docs: replace AGENTS.md with a repo-specific CLAUDE.md
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped

One agent doc per repo, and it is CLAUDE.md. Written fresh rather than
reformatted, and shaped around the fact that this repo holds two
components with nothing in common: ESP-IDF firmware flashed over USB, and
a Python gateway that ships tag to CI to Watchtower.

The rule that a protocol change must update docs/architecture.md is
carried forward, as is the standing one that tatlock is never modified
from here -- this repo consumes its public API only.

Liveness is recorded per component rather than as one claim. The gateway
is confirmed up from the container; the firmware is written down as
undetermined, because no device was attached and there is no remote
telemetry path, and an invented method would have been worse than an
admission. The one figure carried over without re-measuring, a boot time
taken from the old file, is marked as carried rather than verified.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-09 03:17:00 +02:00
co-authored by Claude
parent b6fdd3313a
commit 101816de71
2 changed files with 157 additions and 161 deletions
-131
View File
@@ -1,131 +0,0 @@
# AGENTS.md
> Operational protocols and architecture for AI assistants working on DeskLock.
> Read [docs/architecture.md](docs/architecture.md) before making design changes.
## What this project is
DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
(`/mnt/media/Projects/tatlock`, API at `http://tatlock:8000` on `docker-dataplane`).
Two halves, one repo:
- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
- `gateway/` — Python FastAPI container on tower-of-joy orchestrating STT → chat
(Tatlock `/v1/chat/completions`) → TTS. Listens on port **8600**. STT/TTS models live
in the shared **Speaches** container (live on port 8601, OpenAI-format API), not in
the gateway image; `stt.py`/`tts.py` are pluggable backends (`speaches` default,
`embedded` fallback needing the `[speech]` extra). Gateway needs **Python ≥ 3.11**
(no ceiling; the container runs 3.13) — but system python3 on tower-of-joy is 3.8,
so `make setup` explicitly uses `python3.12`. No local audio resampling in the
default path: the gateway requests 16 kHz output via Speaches' `sample_rate`
extension (verified live).
The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`.
**That doc is the contract** — update it in the same change as any protocol edit on
either side.
The face (black screen, ASCII glyph expressions, matrix rain as activity signal) is
designed in `sim/face/index.html` — the design source of truth — and specified in the
"Face design" section of `docs/architecture.md`. Change the sim and the doc together;
the LVGL implementation follows them. Verify sim changes visually with
`~/bin/claude-screenshot` (note: the tool uses `--virtual-time-budget`, which starves
`requestAnimationFrame` — drive sim animation with `setInterval`, which also mirrors
LVGL timers).
## Hard rules
- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device.
If a feature needs intelligence, it goes in the gateway or in Tatlock itself.
- **Never modify Tatlock from this repo.** It is a separate project with its own repo.
DeskLock consumes its public API only.
- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware
gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS;
the gateway via environment variables (`DESKLOCK_*`).
## Firmware (`firmware/`)
- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`.
- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc)
from the ESP Component Registry (pulled automatically via `main/idf_component.yml`).
- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
official example first; do not guess pin mappings.
```bash
# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell:
source ~/esp-idf/export.sh
# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge)
cd firmware
idf.py build
sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'"
```
- `sg dialout -c '…'` is needed because the login session predates the user's dialout
membership; a plain `idf.py flash` works after any re-login.
- **Radio stack: esp_hosted ≥ 2.x on BOTH chips, non-negotiable.** esp-hosted 1.x is
formally incompatible with IDF 5.5 (esp-hosted-mcu#47) — symptom: RPC/scan/connect
all work, but NO data frames ever flow (no DHCP, no ARP, no ping). Waveshare's
examples pin 1.4.* and the factory C6 slave firmware is ancient — both wrong. The
host manifest pins `espressif/esp_hosted: "^2.12"`; the matching slave image is
embedded as `main/c6_slave.bin` and `c6_ota.c` flashes the C6 **over SDIO** at boot
whenever the C6 reports a version < 2.x (build a new bin from the component's
`slave/` project for esp32c6 when bumping versions).
- **Internal-RAM famine assert**: `assert failed: xTaskCreateStaticPinnedToCore …
xPortcheckValidStackMem` in a pre-app_main boot loop means static+early allocations
starved internal SRAM (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer
counts in sdkconfig.defaults; check `heap_init:` pool lines when the binary grows.
- SDIO clock is set conservatively (`CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`),
ample for 16 kHz voice; raising to 40 MHz is untested on this board's data path.
- **L1 driver diagnosis mode**: set `WIFI_DIAG_MODE 1` in desklock_main.c — the device
becomes AP `DESKLOCK-DIAG` (pass `desklock123`, page at http://192.168.4.1/) proving
radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control → L1
softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns
(`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog).
`CONFIG_SPIRAM_SPEED_200M` only takes effect together with
`CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you
get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config.
- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot
is ~1.6 s from reset to `desklock: DeskLock up`.
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gateway (`gateway/`)
```bash
cd gateway
make setup # venv + dev deps (no ML models)
make setup-speech # additionally install faster-whisper + piper
make run # uvicorn on :8600 with reload
make test # pytest
make lint # ruff check + format check
make typecheck # mypy
```
- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema
and defaults.
- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech`
extra — keep it that way so protocol tests stay fast.
- Deployment is CI-driven: pushing a `v*` tag makes Gitea Actions test, build, and push
`desklock-gateway:{latest,tag}` to the registry and trigger Watchtower
(`.gitea/workflows/build.yml`; needs `REGISTRY_USER`/`REGISTRY_PASSWORD`/
`WATCHTOWER_HTTP_API_TOKEN` secrets). Plain pushes to `main` run lint + tests only. The
gateway deploys as part of the **`tatlock-ui` Portainer stack** —
`system-admin-toj/containers/stacks/tatlock-ui.yml` (registered in `CONTAINERS.md`,
port 8600). Stack updates go through the Portainer API on :8001 (JWT auth; recipe in
`system-admin-toj/containers/setup-new-host.md`), not by editing files on disk.
- Verify speech changes against the live Speaches container with a real round trip
(TTS → STT of a known phrase, expect the transcript back); warm timings to expect:
STT ~0.3 s, TTS ~2 s per sentence.
## Homelab context
- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
- Machine-to-machine traffic uses container names on the `docker-dataplane` network —
`http://tatlock:8000` (the deployed stack overrides `DESKLOCK_TATLOCK_BASE_URL` to
the host LAN IP while Tatlock runs on the host). Browser-facing URLs use
`https://<name>.schweitz.net` (NPM, Authentik SSO) — never for M2M traffic.
- Git remote: `git.schweitz.net` (Gitea).
+157 -30
View File
@@ -1,45 +1,172 @@
# CLAUDE.md
# CLAUDE.md — desklock
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Two components, one repo, coupled by a shared WebSocket protocol:
Claude Code-specific notes for this project. For architecture, hard rules, and full
command reference — see [AGENTS.md](AGENTS.md), and read it before starting work.
- `firmware/` — ESP-IDF (C, LVGL 9) app for a Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
Flashed over USB; not containerized.
- `gateway/` — Python/FastAPI container `desklock-gateway`, port **8600**, part of the
**`tatlock-ui`** Portainer stack (`system-admin-toj/containers/stacks/tatlock-ui.yml`,
verified against the live container and `CONTAINERS.md`). Orchestrates STT → Tatlock
chat → TTS; carries no ML dependencies itself.
## Quick orientation
The device↔gateway protocol is specified in `docs/architecture.md` under "WebSocket
protocol (device ↔ gateway)". **Any protocol change updates that file in the same
change** — it is the contract, not a description of one side's behavior.
DeskLock = firmware for a Waveshare ESP32-P4 round-display device (`firmware/`, ESP-IDF/C/LVGL)
plus a voice gateway container (`gateway/`, Python/FastAPI, port 8600) that bridges device
audio to the Tatlock butler API. The device↔gateway WebSocket protocol lives in
`docs/architecture.md` and must stay in sync with both implementations.
**Never modify Tatlock from this repo.** desklock consumes Tatlock's public API only
(`http://tatlock:8000` on `docker-dataplane`); this is a standing cross-repo rule, not
local policy.
**Keep the firmware thin.** No STT, no TTS, no conversation logic on the device — that
intelligence belongs in the gateway or in Tatlock itself.
**Secrets never go in source.** Firmware gets them via gitignored `firmware/main/secrets.h`
(verified: gitignored, and `#include`d by `gw_client.c`/`net.c`) or NVS; the gateway via
`DESKLOCK_*` environment variables.
## Gotchas — firmware
- Toolchain: **ESP-IDF ≥ 5.4** (this box has 5.5, installed at `~/esp-idf`), not
Arduino, not PlatformIO. Target `esp32p4`. `source ~/esp-idf/export.sh` is required
every shell — `idf.py` is not on the non-interactive PATH otherwise.
- BSP: `waveshare/esp32_p4_wifi6_touch_lcd_xc` from the ESP Component Registry, pulled
automatically via `main/idf_component.yml`. Reference implementations:
[waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/``08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi). Check the official example before guessing a pin mapping.
- Flashing needs the `dialout` group; a login session started before that membership
took effect needs `sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null &&
idf.py -p /dev/ttyACM0 flash'"` — a plain `idf.py flash` works after any re-login.
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`.** Without it,
`CONFIG_SPIRAM_SPEED_200M` is silently dropped to 20 MHz and the 800×800 MIPI-DSI
framebuffer underruns (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees,
task watchdog). Verified both settings present in `firmware/sdkconfig.defaults`.
- **Wi-Fi radio stack must be esp_hosted ≥ 2.x on both the P4 host and the C6 slave,
non-negotiable.** 1.x is formally incompatible with IDF 5.5 (esp-hosted-mcu#47) —
symptom is control-plane-only: RPC/scan/connect all work, but no data frame ever
flows (no DHCP, no ARP, no ping). Waveshare's examples and the factory C6 slave
firmware both pin the wrong (1.x-era) version. The host manifest pins
`espressif/esp_hosted: "^2.12"` (verified in `firmware/main/idf_component.yml`); the
matching slave image is embedded as `main/c6_slave.bin`, and `c6_ota.c` flashes the
C6 over SDIO at boot whenever it reports a version below 2.x.
- **Boot-loop assert `xTaskCreateStaticPinnedToCore … xPortcheckValidStackMem`** before
`app_main` means internal SRAM starvation (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer counts
in `sdkconfig.defaults` (both verified present); check `heap_init:` pool lines in the
boot log when the binary grows.
- SDIO clock is conservative by design: `CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`
(verified), ample for 16 kHz voice — raising it to 40 MHz is untested on this board's
data path.
- **Wi-Fi diagnosis ladder**: set `WIFI_DIAG_MODE 1` in `desklock_main.c` (verified the
macro and `#if` guard exist, currently `0`) — the device becomes AP `DESKLOCK-DIAG`
(password `desklock123`, page at `http://192.168.4.1/`, verified in `wifi_diag.c`),
proving radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control →
L1 softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- Non-interactive boot-log capture: avoid `idf.py monitor` (interactive) — open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8s. Reported boot
time (~1.6s to `desklock: DeskLock up`) is carried from `AGENTS.md` and was **not**
re-timed this pass — no device was connected in this session (see Liveness below).
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gotchas — gateway
- Gateway speech deps (`faster-whisper`, `piper-tts`) are an optional extra —
`make setup` alone runs the app and the test suite without them. `make setup` invokes
`python3.12` explicitly; system `python3` on tower-of-joy is 3.8.
- `ruff` and `mypy` are **not** on the non-interactive PATH — they exist only inside
`gateway/.venv/bin/` once `make setup` has run. Use `make lint` / `make typecheck`, or
invoke `.venv/bin/ruff` / `.venv/bin/mypy` directly; a bare `ruff`/`mypy` will fail to
resolve, which is why the `.claude/settings.json` allow list uses the venv-relative
paths and `make` targets rather than bare tool names.
- Tatlock replies open with a `<think>` block — always strip it via
`tatlock.strip_reasoning()` (`gateway/src/desklock_gateway/tatlock.py`) before TTS or
display. Verified present and called at the one call site.
- Low power is a stated hardware requirement — read "Power management" in
`docs/architecture.md` before touching the face/render loop.
- Gateway health check is `GET /healthz` (verified in `main.py` and matches the
container healthcheck in `tatlock-ui.yml`), not `/health`.
## Commands
```bash
# Firmware (requires `source ~/esp-idf/export.sh` first; IDF ≥ 5.4)
# Firmware (requires `source ~/esp-idf/export.sh` first; ESP-IDF ≥ 5.4, this box has 5.5)
cd firmware && idf.py build
idf.py -p /dev/ttyACM0 flash monitor
# Gateway
cd gateway && make setup # once
make run # dev server :8600
make test # pytest; single test: .venv/bin/pytest tests/test_health.py -k healthz
make lint typecheck
cd gateway && make setup # venv + dev deps (no ML models)
make setup-speech # additionally install faster-whisper + piper
make run # uvicorn on :8600 with reload
make test # pytest (tests/test_health.py, test_tatlock.py, test_commands.py)
make lint typecheck # .venv/bin/ruff + .venv/bin/mypy
```
## Gotchas
## Liveness
- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via
`sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership.
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the
option silently degrades to 20 MHz and the DSI display underruns into a watchdog
loop. Details in AGENTS.md.
- **Wi-Fi = esp_hosted 2.x on BOTH chips** (host manifest + C6 slave, auto-OTA'd from
`main/c6_slave.bin`). 1.x on IDF 5.5 gives working control RPC but a dead data path
(the great July 14th debugging night). Boot-loop assert on
`xTaskCreateStaticPinnedToCore` = internal-RAM famine. Details in AGENTS.md.
- Gateway speech deps are optional extras; `make setup` alone runs the app and tests
without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8).
- Tatlock replies open with a `<think>` block — always strip via
`tatlock.strip_reasoning()` before TTS or display.
- Low power is a prime user requirement: see "Power management" in
docs/architecture.md before touching the face/render loop.
- **Gateway (`desklock-gateway` container, port 8600):** confirmed live — `docker ps`
shows the container running under that name (method: direct container inspection;
blind spot: none relevant here, this confirms the process is up, not that every route
behaves correctly — that would need a request against it, not checked this pass).
- **Firmware / device:** liveness is **undetermined** and cannot be established the way
the gateway's can. No `/dev/ttyACM0` was present in this session (checked: `ls
/dev/ttyACM*` found nothing) and there is no remote telemetry — the device only proves
itself alive over a physical USB serial connection or by joining the LAN and speaking
the WebSocket protocol, neither of which this session had access to. Do not infer
device state from repo contents or from the gateway being up.
- `strip_reasoning()` reachability: confirmed by direct read of
`gateway/src/desklock_gateway/tatlock.py` (method: source read of the one call site;
blind spot: does not confirm it's exercised by a live request — that's what
`tests/test_tatlock.py` is for, not re-run this pass).
## Work tracking
This repo's vault is standalone — its tickets and internal decisions live in its own
`.pql/` and `governance/`, and travel with a clone (`.pql/changelog/` is committed).
```bash
/home/jpmschweitzer/.local/bin/pql ticket list
/home/jpmschweitzer/.local/bin/pql plan whatsnext
/home/jpmschweitzer/.local/bin/pql decisions list
```
`pql` is not on the non-interactive PATH — use the absolute path above. From inside this
repo no `--vault` flag is needed (pql anchors at the nearest `.git/` ancestor, which is
this repo) — but that also means a bare `pql` run from the **workspace root** will not
see this repo's tickets, and a write from the workspace root would go to the wrong
vault. Cross-repo/stack-level decisions (host, network, deploy mechanics — none specific
to desklock were found at the time of writing) live in the workspace vault instead:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain desklock
```
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that
is fast-forwarded and deleted. This is the workspace-wide policy; there is no
per-repo exception here.
- Conventional Commits (`feat:`, `fix:`, `refactor:`, `docs:`, `chore:`).
- Stage explicitly — never `git add -A` (denied by `.claude/settings.json` policy).
- Update `CHANGELOG.md` under `[Unreleased]` for user-facing changes.
## Releasing (gateway only — firmware has no release flow)
Deploy is not automatic — confirm one is wanted first.
1. Bump `version` in `gateway/pyproject.toml`.
2. Move `[Unreleased]` entries into a dated `CHANGELOG.md` section.
3. Commit, tag `vX.Y.Z`, push with tags.
4. `.gitea/workflows/build.yml` runs lint + pytest on every push to `main`; on a `v*`
tag it additionally builds and pushes
`git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}` and pings Watchtower.
5. Verify: `curl http://192.168.86.149:8600/healthz`.
## Architecture
`docs/architecture.md` is the source of truth for system design, the face design
(`sim/face/index.html` is its visual source — change both together and verify with
`~/bin/claude-screenshot`, noting its `--virtual-time-budget` starves
`requestAnimationFrame`, so sim animation is driven by `setInterval` instead), power
budget, latency budget, and the full WebSocket protocol spec. Not restated here because
it is detailed enough to drift if duplicated — read it directly.