desklock/AGENTS.md

# AGENTS.md

> Operational protocols and architecture for AI assistants working on DeskLock.
> Read [docs/architecture.md](docs/architecture.md) before making design changes.

## What this project is

DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
(`/mnt/media/Projects/tatlock`, API at `http://tatlock.schweitz.internal:8000`). Two halves,
one repo:

- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
  (3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
- `gateway/` — Python FastAPI container on tower-of-joy orchestrating STT → chat
  (Tatlock `/v1/chat/completions`) → TTS. Listens on port **8600**. STT/TTS models live
  in the shared **Speaches** container (live on port 8601, OpenAI-format API), not in
  the gateway image; `stt.py`/`tts.py` are pluggable backends (`speaches` default,
  `embedded` fallback needing the `[speech]` extra). Gateway needs **Python ≥ 3.11**
  (no ceiling; the container runs 3.13) — but system python3 on tower-of-joy is 3.8,
  so `make setup` explicitly uses `python3.12`. No local audio resampling in the
  default path: the gateway requests 16 kHz output via Speaches' `sample_rate`
  extension (verified live).

The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`.
**That doc is the contract** — update it in the same change as any protocol edit on
either side.

The face (black screen, ASCII glyph expressions, matrix rain as activity signal) is
designed in `sim/face/index.html` — the design source of truth — and specified in the
"Face design" section of `docs/architecture.md`. Change the sim and the doc together;
the LVGL implementation follows them. Verify sim changes visually with
`~/bin/claude-screenshot` (note: the tool uses `--virtual-time-budget`, which starves
`requestAnimationFrame` — drive sim animation with `setInterval`, which also mirrors
LVGL timers).

## Hard rules

- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device.
  If a feature needs intelligence, it goes in the gateway or in Tatlock itself.
- **Never modify Tatlock from this repo.** It is a separate project with its own repo.
  DeskLock consumes its public API only.
- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware
  gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS;
  the gateway via environment variables (`DESKLOCK_*`).

## Firmware (`firmware/`)

- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`.
- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc)
  from the ESP Component Registry (pulled automatically via `main/idf_component.yml`).
- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
  `examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
  `04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
  official example first; do not guess pin mappings.
- ⚠️ **Unverified scaffold**: the BSP API calls in `desklock_main.c` and the
  `sdkconfig.defaults` values were written before the first successful build. Validate
  against the official examples on first bring-up, then delete this warning.

```bash
# One-time: install ESP-IDF (not yet installed on tower-of-joy)
git clone -b v5.5 --recursive https://github.com/espressif/esp-idf.git ~/esp-idf
~/esp-idf/install.sh esp32p4

# Every shell:
source ~/esp-idf/export.sh

# Build / flash / monitor (device is on USB-C; check `ls /dev/ttyACM*`)
cd firmware
idf.py set-target esp32p4        # once
idf.py build
idf.py -p /dev/ttyACM0 flash monitor   # Ctrl+] exits monitor
```

Flashing requires the `dialout` group (or run with sudo once and fix the group). If the
device doesn't enumerate, hold BOOT while pressing RESET to enter download mode.

## Gateway (`gateway/`)

```bash
cd gateway
make setup          # venv + dev deps (no ML models)
make setup-speech   # additionally install faster-whisper + piper
make run            # uvicorn on :8600 with reload
make test           # pytest
make lint           # ruff check + format check
make typecheck      # mypy
```

- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema
  and defaults.
- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech`
  extra — keep it that way so protocol tests stay fast.
- Deployment is CI-driven: pushing a `v*` tag makes Gitea Actions test, build, and push
  `desklock-gateway:{latest,tag}` to the registry and trigger Watchtower
  (`.gitea/workflows/build.yml`; needs `REGISTRY_USER`/`REGISTRY_PASSWORD`/
  `WATCHTOWER_TOKEN` secrets). Plain pushes to `main` run lint + tests only. The stack
  file is `deploy/desklock-gateway.yml` — copy into
  `system-admin-toj/containers/stacks/` and register the service + port 8600 in
  `CONTAINERS.md` on first deploy.
- Verify speech changes against the live Speaches container with a real round trip
  (TTS → STT of a known phrase, expect the transcript back); warm timings to expect:
  STT ~0.3 s, TTS ~2 s per sentence.

## Homelab context

- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
- Use `tatlock.schweitz.internal:8000` (direct, no SSO) — the public
  `tatlock.schweitz.net` route sits behind Authentik and is not for machine-to-machine
  traffic.
- Git remote: `git.schweitz.net` (Gitea).