desklock/.claude/skills/device-screenshot/SKILL.md

name, description

name

description

device-screenshot

Capture a screenshot of the live DeskLock ESP32-P4 display over USB and view it as a PNG. Use whenever you need to SEE what's on the device screen — verify a face/UI change, check the tap controls overlay, confirm a state (idle, listening, effort), or debug layout remotely without the user's camera. Triggers: "screenshot the device", "what's on the screen", "capture the display", "show me the face", "did the UI change land".

DeskLock device screenshot over USB

Grabs the current LVGL screen off the device and rebuilds it as a PNG on the host. No camera, no gateway — it rides the USB serial that's already attached for flashing.

How it works (so you can debug it)

The firmware (main/face.c face_screenshot_dump, gated by DESKLOCK_DEVMODE in desklock_main.c) runs a task that watches the USB-serial-JTAG RX FIFO for a trigger byte, renders the active screen with lv_snapshot_take_to_draw_buf into a PSRAM buffer, 2×-downscales to 400×400, and streams it as raw binary straight to the USB-serial-JTAG TX FIFO — framed by a text header ###SHOT_BEGIN … bytes=N crc=0x… bin=1### + N bytes + ###SHOT_END###. The host tool firmware/tools/device_shot.py sends the trigger, reads N bytes, checks the CRC (retrying on the rare dropped frame), and writes a PNG.

Two things this design is deliberately built around, learned the hard way:

• Don't use printf. The primary console is UART at 115200 baud (~11 KB/s) — a frame would take ~37 s. Writing straight to the USB FIFO runs at USB speed (~1 s). That's why the dump uses usb_serial_jtag_ll_write_txfifo, not stdout.
• Opening the port does NOT reset the P4 (unlike esptool), and there's no USB stdin, so the trigger is a byte the firmware polls for — on-demand, no reboot, captures whatever state is currently on screen.

Prerequisites

• Firmware flashed with dev mode ON — it's OFF by default (production spends no internal RAM on the watcher and nothing extra runs on the render path). Flash a dev build with:

  cd firmware && sg dialout -c "bash -c 'source ~/esp-idf/export.sh && idf.py -DDESKLOCK_DEVMODE=ON -p /dev/ttyACM0 flash'"
  

  If screenshots return "no frame", the flashed build is production — reflash with -DDESKLOCK_DEVMODE=ON. Back to production: -DDESKLOCK_DEVMODE=OFF (the value sticks in the CMake cache until you flip it). Leave it ON for a UI-dev session (you're reflashing for UI changes anyway); flip OFF for the final/production flash.
• Device on /dev/ttyACM0. The port needs the dialout group, so run the tool under sg dialout -c '…' (this login session predates dialout membership).
• Nothing else holding the port (no idf.py monitor running) — one owner only.

Take a shot

From the desklock repo root (/mnt/media/Projects/desklock):

# current screen (whatever state the device is in right now)
sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png"

# force the tap controls overlay in-frame (mic + volume) — for verifying it
# remotely since you can't physically tap
sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png --overlay"

Then Read /tmp/shot.png to view it. A clean run prints [try 1] 400x400 320000B crc … OK and takes ~5 s.

Trigger bytes: s = current screen, o = force overlay. The tool retries up to 4× on a CRC mismatch (occasional console contention), so a transient bad frame self-heals.

Notes / caveats

• RAM / render path: the watcher costs one ~5 KB internal-RAM task, so it's behind DESKLOCK_DEVMODE (off by default). Internal RAM is the scarce resource on this board (it caps the rain sprite pool). Never ship a production build with it on.
• 400×400 is plenty for layout/UI checks. To change resolution, adjust SHOT_DS in face.c (the downscale factor) — the host reads the size from the header.
• If you just reflashed, wait ~7 s for boot before the first shot.
• Only the USB console path is used; this does not touch the device↔gateway WebSocket protocol.