docs: replace AGENTS.md with a repo-specific CLAUDE.md
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped

One agent doc per repo, and it is CLAUDE.md. Written fresh rather than
reformatted, and shaped around the fact that this repo holds two
components with nothing in common: ESP-IDF firmware flashed over USB, and
a Python gateway that ships tag to CI to Watchtower.

The rule that a protocol change must update docs/architecture.md is
carried forward, as is the standing one that tatlock is never modified
from here -- this repo consumes its public API only.

Liveness is recorded per component rather than as one claim. The gateway
is confirmed up from the container; the firmware is written down as
undetermined, because no device was attached and there is no remote
telemetry path, and an invented method would have been worse than an
admission. The one figure carried over without re-measuring, a boot time
taken from the old file, is marked as carried rather than verified.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-09 03:17:00 +02:00
co-authored by Claude
parent b6fdd3313a
commit 101816de71
2 changed files with 157 additions and 161 deletions
-131
View File
@@ -1,131 +0,0 @@
# AGENTS.md
> Operational protocols and architecture for AI assistants working on DeskLock.
> Read [docs/architecture.md](docs/architecture.md) before making design changes.
## What this project is
DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
(`/mnt/media/Projects/tatlock`, API at `http://tatlock:8000` on `docker-dataplane`).
Two halves, one repo:
- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
- `gateway/` — Python FastAPI container on tower-of-joy orchestrating STT → chat
(Tatlock `/v1/chat/completions`) → TTS. Listens on port **8600**. STT/TTS models live
in the shared **Speaches** container (live on port 8601, OpenAI-format API), not in
the gateway image; `stt.py`/`tts.py` are pluggable backends (`speaches` default,
`embedded` fallback needing the `[speech]` extra). Gateway needs **Python ≥ 3.11**
(no ceiling; the container runs 3.13) — but system python3 on tower-of-joy is 3.8,
so `make setup` explicitly uses `python3.12`. No local audio resampling in the
default path: the gateway requests 16 kHz output via Speaches' `sample_rate`
extension (verified live).
The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`.
**That doc is the contract** — update it in the same change as any protocol edit on
either side.
The face (black screen, ASCII glyph expressions, matrix rain as activity signal) is
designed in `sim/face/index.html` — the design source of truth — and specified in the
"Face design" section of `docs/architecture.md`. Change the sim and the doc together;
the LVGL implementation follows them. Verify sim changes visually with
`~/bin/claude-screenshot` (note: the tool uses `--virtual-time-budget`, which starves
`requestAnimationFrame` — drive sim animation with `setInterval`, which also mirrors
LVGL timers).
## Hard rules
- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device.
If a feature needs intelligence, it goes in the gateway or in Tatlock itself.
- **Never modify Tatlock from this repo.** It is a separate project with its own repo.
DeskLock consumes its public API only.
- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware
gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS;
the gateway via environment variables (`DESKLOCK_*`).
## Firmware (`firmware/`)
- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`.
- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc)
from the ESP Component Registry (pulled automatically via `main/idf_component.yml`).
- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
official example first; do not guess pin mappings.
```bash
# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell:
source ~/esp-idf/export.sh
# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge)
cd firmware
idf.py build
sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'"
```
- `sg dialout -c '…'` is needed because the login session predates the user's dialout
membership; a plain `idf.py flash` works after any re-login.
- **Radio stack: esp_hosted ≥ 2.x on BOTH chips, non-negotiable.** esp-hosted 1.x is
formally incompatible with IDF 5.5 (esp-hosted-mcu#47) — symptom: RPC/scan/connect
all work, but NO data frames ever flow (no DHCP, no ARP, no ping). Waveshare's
examples pin 1.4.* and the factory C6 slave firmware is ancient — both wrong. The
host manifest pins `espressif/esp_hosted: "^2.12"`; the matching slave image is
embedded as `main/c6_slave.bin` and `c6_ota.c` flashes the C6 **over SDIO** at boot
whenever the C6 reports a version < 2.x (build a new bin from the component's
`slave/` project for esp32c6 when bumping versions).
- **Internal-RAM famine assert**: `assert failed: xTaskCreateStaticPinnedToCore …
xPortcheckValidStackMem` in a pre-app_main boot loop means static+early allocations
starved internal SRAM (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer
counts in sdkconfig.defaults; check `heap_init:` pool lines when the binary grows.
- SDIO clock is set conservatively (`CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`),
ample for 16 kHz voice; raising to 40 MHz is untested on this board's data path.
- **L1 driver diagnosis mode**: set `WIFI_DIAG_MODE 1` in desklock_main.c — the device
becomes AP `DESKLOCK-DIAG` (pass `desklock123`, page at http://192.168.4.1/) proving
radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control → L1
softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns
(`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog).
`CONFIG_SPIRAM_SPEED_200M` only takes effect together with
`CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you
get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config.
- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot
is ~1.6 s from reset to `desklock: DeskLock up`.
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gateway (`gateway/`)
```bash
cd gateway
make setup # venv + dev deps (no ML models)
make setup-speech # additionally install faster-whisper + piper
make run # uvicorn on :8600 with reload
make test # pytest
make lint # ruff check + format check
make typecheck # mypy
```
- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema
and defaults.
- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech`
extra — keep it that way so protocol tests stay fast.
- Deployment is CI-driven: pushing a `v*` tag makes Gitea Actions test, build, and push
`desklock-gateway:{latest,tag}` to the registry and trigger Watchtower
(`.gitea/workflows/build.yml`; needs `REGISTRY_USER`/`REGISTRY_PASSWORD`/
`WATCHTOWER_HTTP_API_TOKEN` secrets). Plain pushes to `main` run lint + tests only. The
gateway deploys as part of the **`tatlock-ui` Portainer stack** —
`system-admin-toj/containers/stacks/tatlock-ui.yml` (registered in `CONTAINERS.md`,
port 8600). Stack updates go through the Portainer API on :8001 (JWT auth; recipe in
`system-admin-toj/containers/setup-new-host.md`), not by editing files on disk.
- Verify speech changes against the live Speaches container with a real round trip
(TTS → STT of a known phrase, expect the transcript back); warm timings to expect:
STT ~0.3 s, TTS ~2 s per sentence.
## Homelab context
- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
- Machine-to-machine traffic uses container names on the `docker-dataplane` network —
`http://tatlock:8000` (the deployed stack overrides `DESKLOCK_TATLOCK_BASE_URL` to
the host LAN IP while Tatlock runs on the host). Browser-facing URLs use
`https://<name>.schweitz.net` (NPM, Authentik SSO) — never for M2M traffic.
- Git remote: `git.schweitz.net` (Gitea).
+157 -30
View File
@@ -1,45 +1,172 @@
# CLAUDE.md # CLAUDE.md — desklock
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. Two components, one repo, coupled by a shared WebSocket protocol:
Claude Code-specific notes for this project. For architecture, hard rules, and full - `firmware/` — ESP-IDF (C, LVGL 9) app for a Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
command reference — see [AGENTS.md](AGENTS.md), and read it before starting work. (3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
Flashed over USB; not containerized.
- `gateway/` — Python/FastAPI container `desklock-gateway`, port **8600**, part of the
**`tatlock-ui`** Portainer stack (`system-admin-toj/containers/stacks/tatlock-ui.yml`,
verified against the live container and `CONTAINERS.md`). Orchestrates STT → Tatlock
chat → TTS; carries no ML dependencies itself.
## Quick orientation The device↔gateway protocol is specified in `docs/architecture.md` under "WebSocket
protocol (device ↔ gateway)". **Any protocol change updates that file in the same
change** — it is the contract, not a description of one side's behavior.
DeskLock = firmware for a Waveshare ESP32-P4 round-display device (`firmware/`, ESP-IDF/C/LVGL) **Never modify Tatlock from this repo.** desklock consumes Tatlock's public API only
plus a voice gateway container (`gateway/`, Python/FastAPI, port 8600) that bridges device (`http://tatlock:8000` on `docker-dataplane`); this is a standing cross-repo rule, not
audio to the Tatlock butler API. The device↔gateway WebSocket protocol lives in local policy.
`docs/architecture.md` and must stay in sync with both implementations.
**Keep the firmware thin.** No STT, no TTS, no conversation logic on the device — that
intelligence belongs in the gateway or in Tatlock itself.
**Secrets never go in source.** Firmware gets them via gitignored `firmware/main/secrets.h`
(verified: gitignored, and `#include`d by `gw_client.c`/`net.c`) or NVS; the gateway via
`DESKLOCK_*` environment variables.
## Gotchas — firmware
- Toolchain: **ESP-IDF ≥ 5.4** (this box has 5.5, installed at `~/esp-idf`), not
Arduino, not PlatformIO. Target `esp32p4`. `source ~/esp-idf/export.sh` is required
every shell — `idf.py` is not on the non-interactive PATH otherwise.
- BSP: `waveshare/esp32_p4_wifi6_touch_lcd_xc` from the ESP Component Registry, pulled
automatically via `main/idf_component.yml`. Reference implementations:
[waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/` — `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi). Check the official example before guessing a pin mapping.
- Flashing needs the `dialout` group; a login session started before that membership
took effect needs `sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null &&
idf.py -p /dev/ttyACM0 flash'"` — a plain `idf.py flash` works after any re-login.
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`.** Without it,
`CONFIG_SPIRAM_SPEED_200M` is silently dropped to 20 MHz and the 800×800 MIPI-DSI
framebuffer underruns (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees,
task watchdog). Verified both settings present in `firmware/sdkconfig.defaults`.
- **Wi-Fi radio stack must be esp_hosted ≥ 2.x on both the P4 host and the C6 slave,
non-negotiable.** 1.x is formally incompatible with IDF 5.5 (esp-hosted-mcu#47) —
symptom is control-plane-only: RPC/scan/connect all work, but no data frame ever
flows (no DHCP, no ARP, no ping). Waveshare's examples and the factory C6 slave
firmware both pin the wrong (1.x-era) version. The host manifest pins
`espressif/esp_hosted: "^2.12"` (verified in `firmware/main/idf_component.yml`); the
matching slave image is embedded as `main/c6_slave.bin`, and `c6_ota.c` flashes the
C6 over SDIO at boot whenever it reports a version below 2.x.
- **Boot-loop assert `xTaskCreateStaticPinnedToCore … xPortcheckValidStackMem`** before
`app_main` means internal SRAM starvation (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer counts
in `sdkconfig.defaults` (both verified present); check `heap_init:` pool lines in the
boot log when the binary grows.
- SDIO clock is conservative by design: `CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`
(verified), ample for 16 kHz voice — raising it to 40 MHz is untested on this board's
data path.
- **Wi-Fi diagnosis ladder**: set `WIFI_DIAG_MODE 1` in `desklock_main.c` (verified the
macro and `#if` guard exist, currently `0`) — the device becomes AP `DESKLOCK-DIAG`
(password `desklock123`, page at `http://192.168.4.1/`, verified in `wifi_diag.c`),
proving radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control →
L1 softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- Non-interactive boot-log capture: avoid `idf.py monitor` (interactive) — open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8s. Reported boot
time (~1.6s to `desklock: DeskLock up`) is carried from `AGENTS.md` and was **not**
re-timed this pass — no device was connected in this session (see Liveness below).
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gotchas — gateway
- Gateway speech deps (`faster-whisper`, `piper-tts`) are an optional extra —
`make setup` alone runs the app and the test suite without them. `make setup` invokes
`python3.12` explicitly; system `python3` on tower-of-joy is 3.8.
- `ruff` and `mypy` are **not** on the non-interactive PATH — they exist only inside
`gateway/.venv/bin/` once `make setup` has run. Use `make lint` / `make typecheck`, or
invoke `.venv/bin/ruff` / `.venv/bin/mypy` directly; a bare `ruff`/`mypy` will fail to
resolve, which is why the `.claude/settings.json` allow list uses the venv-relative
paths and `make` targets rather than bare tool names.
- Tatlock replies open with a `<think>` block — always strip it via
`tatlock.strip_reasoning()` (`gateway/src/desklock_gateway/tatlock.py`) before TTS or
display. Verified present and called at the one call site.
- Low power is a stated hardware requirement — read "Power management" in
`docs/architecture.md` before touching the face/render loop.
- Gateway health check is `GET /healthz` (verified in `main.py` and matches the
container healthcheck in `tatlock-ui.yml`), not `/health`.
## Commands ## Commands
```bash ```bash
# Firmware (requires `source ~/esp-idf/export.sh` first; IDF ≥ 5.4) # Firmware (requires `source ~/esp-idf/export.sh` first; ESP-IDF ≥ 5.4, this box has 5.5)
cd firmware && idf.py build cd firmware && idf.py build
idf.py -p /dev/ttyACM0 flash monitor idf.py -p /dev/ttyACM0 flash monitor
# Gateway # Gateway
cd gateway && make setup # once cd gateway && make setup # venv + dev deps (no ML models)
make run # dev server :8600 make setup-speech # additionally install faster-whisper + piper
make test # pytest; single test: .venv/bin/pytest tests/test_health.py -k healthz make run # uvicorn on :8600 with reload
make lint typecheck make test # pytest (tests/test_health.py, test_tatlock.py, test_commands.py)
make lint typecheck # .venv/bin/ruff + .venv/bin/mypy
``` ```
## Gotchas ## Liveness
- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via - **Gateway (`desklock-gateway` container, port 8600):** confirmed live — `docker ps`
`sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership. shows the container running under that name (method: direct container inspection;
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the blind spot: none relevant here, this confirms the process is up, not that every route
option silently degrades to 20 MHz and the DSI display underruns into a watchdog behaves correctly — that would need a request against it, not checked this pass).
loop. Details in AGENTS.md. - **Firmware / device:** liveness is **undetermined** and cannot be established the way
- **Wi-Fi = esp_hosted 2.x on BOTH chips** (host manifest + C6 slave, auto-OTA'd from the gateway's can. No `/dev/ttyACM0` was present in this session (checked: `ls
`main/c6_slave.bin`). 1.x on IDF 5.5 gives working control RPC but a dead data path /dev/ttyACM*` found nothing) and there is no remote telemetry — the device only proves
(the great July 14th debugging night). Boot-loop assert on itself alive over a physical USB serial connection or by joining the LAN and speaking
`xTaskCreateStaticPinnedToCore` = internal-RAM famine. Details in AGENTS.md. the WebSocket protocol, neither of which this session had access to. Do not infer
- Gateway speech deps are optional extras; `make setup` alone runs the app and tests device state from repo contents or from the gateway being up.
without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8). - `strip_reasoning()` reachability: confirmed by direct read of
- Tatlock replies open with a `<think>` block — always strip via `gateway/src/desklock_gateway/tatlock.py` (method: source read of the one call site;
`tatlock.strip_reasoning()` before TTS or display. blind spot: does not confirm it's exercised by a live request — that's what
- Low power is a prime user requirement: see "Power management" in `tests/test_tatlock.py` is for, not re-run this pass).
docs/architecture.md before touching the face/render loop.
## Work tracking
This repo's vault is standalone — its tickets and internal decisions live in its own
`.pql/` and `governance/`, and travel with a clone (`.pql/changelog/` is committed).
```bash
/home/jpmschweitzer/.local/bin/pql ticket list
/home/jpmschweitzer/.local/bin/pql plan whatsnext
/home/jpmschweitzer/.local/bin/pql decisions list
```
`pql` is not on the non-interactive PATH — use the absolute path above. From inside this
repo no `--vault` flag is needed (pql anchors at the nearest `.git/` ancestor, which is
this repo) — but that also means a bare `pql` run from the **workspace root** will not
see this repo's tickets, and a write from the workspace root would go to the wrong
vault. Cross-repo/stack-level decisions (host, network, deploy mechanics — none specific
to desklock were found at the time of writing) live in the workspace vault instead:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain desklock
```
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that
is fast-forwarded and deleted. This is the workspace-wide policy; there is no
per-repo exception here.
- Conventional Commits (`feat:`, `fix:`, `refactor:`, `docs:`, `chore:`).
- Stage explicitly — never `git add -A` (denied by `.claude/settings.json` policy).
- Update `CHANGELOG.md` under `[Unreleased]` for user-facing changes.
## Releasing (gateway only — firmware has no release flow)
Deploy is not automatic — confirm one is wanted first.
1. Bump `version` in `gateway/pyproject.toml`.
2. Move `[Unreleased]` entries into a dated `CHANGELOG.md` section.
3. Commit, tag `vX.Y.Z`, push with tags.
4. `.gitea/workflows/build.yml` runs lint + pytest on every push to `main`; on a `v*`
tag it additionally builds and pushes
`git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}` and pings Watchtower.
5. Verify: `curl http://192.168.86.149:8600/healthz`.
## Architecture
`docs/architecture.md` is the source of truth for system design, the face design
(`sim/face/index.html` is its visual source — change both together and verify with
`~/bin/claude-screenshot`, noting its `--virtual-time-budget` starves
`requestAnimationFrame`, so sim animation is driven by `setInterval` instead), power
budget, latency budget, and the full WebSocket protocol spec. Not restated here because
it is detailed enough to drift if duplicated — read it directly.