Initial scaffold: ESP32-P4 firmware + voice gateway for Tatlock endpoint
DeskLock gives the Tatlock butler a face and voice on a Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C round display in the living room. - firmware/: ESP-IDF project targeting esp32p4 with the Waveshare XC BSP - gateway/: FastAPI voice bridge (faster-whisper STT, Tatlock chat, Piper TTS) - docs/architecture.md: component design and device<->gateway WS protocol Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,88 @@
|
||||
# AGENTS.md
|
||||
|
||||
> Operational protocols and architecture for AI assistants working on DeskLock.
|
||||
> Read [docs/architecture.md](docs/architecture.md) before making design changes.
|
||||
|
||||
## What this project is
|
||||
|
||||
DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
|
||||
(`/mnt/media/Projects/tatlock`, API at `http://tatlock.schweitz.internal:8000`). Two halves,
|
||||
one repo:
|
||||
|
||||
- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
|
||||
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
|
||||
- `gateway/` — Python FastAPI container on tower-of-joy doing STT (faster-whisper, GPU),
|
||||
chat (Tatlock `/v1/chat/completions`), and TTS (Piper). Listens on port **8600**.
|
||||
|
||||
The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`.
|
||||
**That doc is the contract** — update it in the same change as any protocol edit on
|
||||
either side.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device.
|
||||
If a feature needs intelligence, it goes in the gateway or in Tatlock itself.
|
||||
- **Never modify Tatlock from this repo.** It is a separate project with its own repo.
|
||||
DeskLock consumes its public API only.
|
||||
- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware
|
||||
gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS;
|
||||
the gateway via environment variables (`DESKLOCK_*`).
|
||||
|
||||
## Firmware (`firmware/`)
|
||||
|
||||
- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`.
|
||||
- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc)
|
||||
from the ESP Component Registry (pulled automatically via `main/idf_component.yml`).
|
||||
- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
|
||||
`examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
|
||||
`04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
|
||||
official example first; do not guess pin mappings.
|
||||
- ⚠️ **Unverified scaffold**: the BSP API calls in `desklock_main.c` and the
|
||||
`sdkconfig.defaults` values were written before the first successful build. Validate
|
||||
against the official examples on first bring-up, then delete this warning.
|
||||
|
||||
```bash
|
||||
# One-time: install ESP-IDF (not yet installed on tower-of-joy)
|
||||
git clone -b v5.5 --recursive https://github.com/espressif/esp-idf.git ~/esp-idf
|
||||
~/esp-idf/install.sh esp32p4
|
||||
|
||||
# Every shell:
|
||||
source ~/esp-idf/export.sh
|
||||
|
||||
# Build / flash / monitor (device is on USB-C; check `ls /dev/ttyACM*`)
|
||||
cd firmware
|
||||
idf.py set-target esp32p4 # once
|
||||
idf.py build
|
||||
idf.py -p /dev/ttyACM0 flash monitor # Ctrl+] exits monitor
|
||||
```
|
||||
|
||||
Flashing requires the `dialout` group (or run with sudo once and fix the group). If the
|
||||
device doesn't enumerate, hold BOOT while pressing RESET to enter download mode.
|
||||
|
||||
## Gateway (`gateway/`)
|
||||
|
||||
```bash
|
||||
cd gateway
|
||||
make setup # venv + dev deps (no ML models)
|
||||
make setup-speech # additionally install faster-whisper + piper
|
||||
make run # uvicorn on :8600 with reload
|
||||
make test # pytest
|
||||
make lint # ruff check + format check
|
||||
make typecheck # mypy
|
||||
```
|
||||
|
||||
- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema
|
||||
and defaults.
|
||||
- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech`
|
||||
extra — keep it that way so protocol tests stay fast.
|
||||
- Deployment: Docker image built from `gateway/Dockerfile`, deployed like other
|
||||
tower-of-joy stacks (see `/mnt/media/Projects/system-admin-toj/containers/`). Register
|
||||
the service + port in `CONTAINERS.md` when it first deploys.
|
||||
|
||||
## Homelab context
|
||||
|
||||
- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
|
||||
- Use `tatlock.schweitz.internal:8000` (direct, no SSO) — the public
|
||||
`tatlock.schweitz.net` route sits behind Authentik and is not for machine-to-machine
|
||||
traffic.
|
||||
- Git remote: `git.schweitz.net` (Gitea).
|
||||
Reference in New Issue
Block a user