28 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 5.5 fc7f0cc1fd chore: file T-1 — per-device token for the gateway
Test, Build and Push / test-gateway (push) Successful in 14s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
The first ticket on this board: /ws/voice and /devices accept any
client on the LAN, which reaches Tatlock and its Home Assistant
controls. Found in the workspace security sweep of 2026-09-23.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 13:45:19 +02:00
jpmschweitzerandClaude f36f0fe431 fix(permissions): narrow rm -rf deny globs to their exact forms
Test, Build and Push / test-gateway (push) Successful in 13s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
The trailing wildcard on the three rm -rf deny entries spanned path
separators, so Bash(rm -rf /*) matched every absolute path on the
machine rather than the filesystem root, and the ~ and $HOME entries
had the same shape. Narrowed to the exact literal forms.

These rules match literal command text, so they still stop a typo on
rm -rf /, rm -rf ~ or rm -rf $HOME exactly, but they no longer stop a
recursive delete aimed at any other path. That reduced cover is
deliberate, not an oversight.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-25 20:33:17 +02:00
jpmschweitzerandClaude 8bdf950fcf build(lint): select ruff's rules explicitly instead of inheriting them
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
The gate had no `select`, so it linted with whatever the installed ruff
version defaults to. `dev` pins only `ruff>=0.6` and CI installs that
extra fresh on every run, which made the rule set a function of when pip
last resolved rather than of this code. Two developers on one commit
could get different answers, and so could CI and a laptop.

This surfaced when T-47's converged setup reinstalled ruff and pulled
0.16.3: `make lint` failed on UP017 and BLE001 in main.py, a file the
commit before it had not touched. Under the previous install the same
code passed. Nothing about the code changed — only the linter's idea of
what to look at, which had grown to 413 rules with nobody choosing them.

Naming the families fixes that; pinning the version would only have
frozen the symptom and moved the surprise to whoever unpinned it. 217
rules now, selected on purpose, and a future ruff release becomes a
decision instead of a broken push.

ASYNC is included deliberately — this is a websocket gateway, and it is
the family whose findings would be real bugs rather than style. BLE is
deliberately excluded: main.py catches bare Exception when a device
disappears mid-send, which is correct there, and selecting BLE would
mean a noqa on every such site to say so.

UP017 is fixed rather than suppressed (datetime.timezone.utc -> UTC,
identical semantics, and requires-python is already >=3.11); isort then
reordered the import, which is the whole of the main.py diff.

Verified the selection is load-bearing rather than decorative: a probe
file with a mutable default argument fails the explicit set (B006, exit
1) and passes ruff's minimal default set (exit 0), so the rules named
here are doing work the fallback would not. Probe deleted; lint,
typecheck and the 9-test suite all green after.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:36:37 +02:00
jpmschweitzer 78d9b276ee build(make): prove make setup actually works before exiting 0 (T-47)
pip install exiting 0 is not evidence the gateway environment is usable
(D-24) — a resolved-but-broken dependency or a stale venv from another
Python both look identical to a clean install at the point setup exits.
End the target with a cheap positive check instead: collect the test
suite (imports every src module each test pulls in) and confirm ruff
and mypy resolve inside the venv, the only place either binary exists.

Also states explicitly, in a comment, that setup covers the gateway
half only — the firmware half needs `source ~/esp-idf/export.sh` in
every shell, which a Makefile recipe cannot leave sourced in the
caller's shell, so build-firmware sources it itself instead.
2026-08-17 12:04:12 +02:00
jpmschweitzerandClaude 3f8e4cf593 fix(gateway): tell mypy the speech extra may be absent
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
`make typecheck` failed on four missing stubs — piper, faster_whisper and numpy
twice — which made the pre-push gate red on a machine that had followed the
documented setup. `make setup` deliberately omits the speech extra; only
`make setup-speech` installs it, because faster-whisper and piper-tts pull
several GB of ML wheels for a backend the deployment does not use.
settings.tts_backend defaults to "speaches", a network call to the shared
service on 8601, and both imports are lazy inside the functions that need them.
So the absence is a runtime fact the code already handles, not a defect.

The gate was therefore failing for doing the right thing, which is how a gate
stops being read. The correct assertion is "these modules may be absent", not
"install several GB so the type checker is satisfied" — on a disk at 76%, for a
path this deployment does not take.

There was no [tool.mypy] section at all, so this adds one. numpy is listed for
the same reason as the other two: nothing depends on it directly, it arrives
with faster-whisper.

Note the packaging was already correct — speech is an optional extra and always
has been. I initially reported these as required dependencies that were missing
from the venv, having grepped for the package names and read the hits without
checking which table they sat under; `mypy>=1.11` was three lines below in the
same output, which should have said "these are extras". CLAUDE.md states it
outright. The fix is smaller than the one I first described because the repo
was already doing the right thing.

Gate now passes: secrets, ruff, mypy, 9 tests. Firmware and sim still report
undetermined, which is accurate — neither has a suite.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 14:30:56 +02:00
jpmschweitzerandClaude 7846e48595 build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude eca10a0dd0 docs: qualify the workspace decision references
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
D-26 and D-27 are workspace decisions, and this repo's own vault has none, so
a bare citation here means nothing resolvable — workspace D-21 requires the
vault to be named. Found by `make verify` in the workspace, which is the case
that rule was written for.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 17:58:14 +02:00
jpmschweitzerandClaude 5297249e58 ci(make): reserve exit 69 for "could not run" (D-26)
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:25 +02:00
jpmschweitzerandClaude 2ef00c98de build: add the root Makefile the previous commit should have carried
Test, Build and Push / test-gateway (push) Successful in 12s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
ff19320 removed gateway/Makefile but the git add that was meant to stage its
replacement aborted on an already-staged pathspec, so the deletion landed
alone and main briefly had no Makefile at all. This is the other half.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:04:43 +02:00
jpmschweitzerandClaude ff193203c9 refactor: merge the component Makefiles into one at the root
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
desklock is a three-component repo and only the gateway had a Makefile, so
make test meant "the gateway suite" or "no such target" depending on which
directory you happened to be standing in. One root Makefile makes it mean the
same thing everywhere (D-27), and gateway/Makefile is removed rather than
delegated to, so there is one place to look.

The firmware targets now source ~/esp-idf/export.sh themselves. Verified that
idf.py does not resolve on PATH without it and does after — the same class of
failure that has cost time on four other tools on this host, and the reason
D-10 puts path resolution in the Makefile rather than in callers. They fail
loudly with a hint when the toolchain is absent instead of reporting command
not found.

make test never reports green for the firmware. It has no suite, so it prints
undetermined rather than skipping silently — a no-op target that exits 0 would
claim a pass for something never run (D-26).

Verified: make test runs the real 9-test gateway suite, make lint passes, the
missing-toolchain guard fires, and make help lists every target. The firmware
build itself was not run.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:03:58 +02:00
jpmschweitzerandClaude 8cd2c05a00 chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
Test, Build and Push / test-gateway (push) Successful in 11s
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:49:04 +02:00
jpmschweitzerandClaude 984e43aa8f chore(claude): deny toj in the sub-repos
Test, Build and Push / test-gateway (push) Successful in 10s
Test, Build and Push / build-gateway (push) Skipped
Test, Build and Push / release (push) Skipped
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:09 +02:00
jpmschweitzerandClaude 2cdefefecb ci: gate pushes on a gitleaks scan of the outgoing commits
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:48:58 +02:00
jpmschweitzerandClaude 101816de71 docs: replace AGENTS.md with a repo-specific CLAUDE.md
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
One agent doc per repo, and it is CLAUDE.md. Written fresh rather than
reformatted, and shaped around the fact that this repo holds two
components with nothing in common: ESP-IDF firmware flashed over USB, and
a Python gateway that ships tag to CI to Watchtower.

The rule that a protocol change must update docs/architecture.md is
carried forward, as is the standing one that tatlock is never modified
from here -- this repo consumes its public API only.

Liveness is recorded per component rather than as one claim. The gateway
is confirmed up from the container; the firmware is written down as
undetermined, because no device was attached and there is no remote
telemetry path, and an invented method would have been worse than an
admission. The one figure carried over without re-measuring, a boot time
taken from the old file, is marked as carried rather than verified.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:17:00 +02:00
jpmschweitzerandClaude b6fdd3313a chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:16:40 +02:00
jpmschweitzerandClaude 67bee80dc8 docs(architecture): sync with measured 2026-08-07 state
Test, Build and Push / test-gateway (push) Successful in 37s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
Every figure in the latency budget was stale, in both directions. TTS was
listed at ~1.9 s per sentence but measures ~0.24 s warm for 4.5 s of audio;
the full Tatlock flow was listed at 11-25 s but measures ~10-13 s for simple
turns. Both sets of numbers predate the current model.

The VRAM section now carries real figures and the reason they matter: on
2026-08-07 Tatlock ran against a 9.3 GB model, leaving 7 MiB free, and every
transcription failed with CUDA out of memory while the Speaches container
still reported healthy. The budget is the constraint, not slack.

Also replaces the retired tatlock.schweitz.internal hostname in the topology
diagram with the docker container name.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 15:05:15 +02:00
jpmschweitzerandClaude Fable 5 7c7ce1541c release v0.2.2
Test, Build and Push / test-gateway (push) Successful in 9s
Test, Build and Push / release (push) Successful in 3s
Test, Build and Push / build-gateway (push) Successful in 48s
Network migration: the gateway's default Tatlock URL is now the docker
container name (http://tatlock:8000); the retiring tatlock.schweitz.internal
domain is gone from config and docs. Deployments that set
DESKLOCK_TATLOCK_BASE_URL are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:32:13 +02:00
jpmschweitzerandClaude Fable 5 014c86f51e chore(gateway): drop schweitz.internal, default to http://tatlock:8000
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
The homelab is retiring the *.schweitz.internal domain; in-network
machine-to-machine traffic uses docker container names on the
docker-dataplane network. The deployed tatlock-ui stack already overrides
DESKLOCK_TATLOCK_BASE_URL (Tatlock runs on the host), so only the
fallback default changes.

Docs follow: AGENTS.md M2M guidance now points at container names with
*.schweitz.net reserved for browsers, architecture.md drops the retired
domain (the registry name now matches what CI actually pushes since
c477019), and the README diagram loses the stale hostname.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:22:25 +02:00
jpmschweitzerandClaude Fable 5 c4770194d9 chore(ci): push images via git.schweitz.net registry
Test, Build and Push / test-gateway (push) Successful in 59s
Test, Build and Push / release (push) Skipped
Test, Build and Push / build-gateway (push) Skipped
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:09:59 +02:00
jpmschweitzerandClaude Opus 4.8 ccb46d45b4 release v0.2.1
Test, Build and Push / test-gateway (push) Successful in 11s
Test, Build and Push / release (push) Successful in 2s
Test, Build and Push / build-gateway (push) Successful in 1m27s
Fixes: volume voice command no longer hangs in the thinking spinner (gong
stops holding the busy state), and the butler filler is reworded to avoid
a text-to-speech mid-phrase pause.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 22:39:46 +02:00
jpmschweitzerandClaude Opus 4.8 9a62233775 fix(gateway): reword butler filler to avoid a TTS mid-phrase pause
Kokoro inserts an unnatural ~0.2s pause before "for you", so "Let me check
on that for you, sir." came out as two phrases. Reword to "Let me check on
that, sir." — same intent, clean pacing (measured: no internal silence gap).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 22:39:03 +02:00
jpmschweitzerandClaude Opus 4.8 24e9375095 fix(firmware): don't let the feedback gong swallow return-to-idle
A spoken volume command left the face stuck in the thinking spinner. The
gateway sends state:thinking -> command -> state:idle, but the device's
state:idle handler is gated on !audio_is_playing(), and the feedback gong
had been setting s_playing for its (up to 5 s) duration — so the idle was
ignored and nothing re-sent it. The gong is a UI cue, not reply playback,
so it no longer sets s_playing. This also drops the 5 s wake-gate the gong
was imposing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 21:50:53 +02:00
jpmschweitzerandClaude Opus 4.8 e3021baf4d release v0.2.0
Test, Build and Push / test-gateway (push) Successful in 10s
Test, Build and Push / release (push) Successful in 3s
Test, Build and Push / build-gateway (push) Successful in 1m48s
Volume goes to eleven (0–11 scale, gong feedback, mute), a voice command
service that handles volume/mute without the LLM, and an immediate butler
filler line with a spinner during the Tatlock wait.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 21:32:31 +02:00
jpmschweitzerandClaude Opus 4.8 09f8afcadd feat(gateway): butler filler line while Tatlock works
Tatlock turns take 10-25s, which is a long silence after a request. Speak
a canned "Let me check on that for you, sir" immediately (synthesized once
and cached), then re-assert the thinking state so the device keeps its
effort-face spinner up until the real reply arrives. On the device, guard
the playback-done handler so the filler audio finishing doesn't drop the
spinner back to idle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 21:29:35 +02:00
jpmschweitzerandClaude Opus 4.8 6b7fbb60c9 feat(gateway): voice command service
Recognize simple device commands in the transcript and act on them without
a Tatlock round-trip. commands.match() maps volume up/down, mute/unmute,
"set volume to N", and "goes to eleven"/max to a "command" message sent
straight to the device; the utterance never reaches the LLM. Matching is
deliberately precise so real requests ("set an alarm for a quarter to
eleven") are not hijacked. Adds the "command" message to the device
protocol in docs/architecture.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 21:29:00 +02:00
jpmschweitzerandClaude Opus 4.8 cdb23bda05 feat(firmware): volume control goes to eleven
Rework the speaker volume from a 0-100 percentage to an 11-step level
(0..11, mapped to the codec's percent), with mute that remembers the
prior level so unmute restores it. Any non-silent change plays the gong
as feedback; a new change cuts the in-flight gong off and restarts it
rather than queueing another. The tap overlay shows the level number and
0..11 bar, and a "11" drifts up off the bar when you hit maximum.

Also lands the device side of gateway volume commands: gw_client routes a
"command" message to face_volume_command, which applies the change and
shows a compact auto-hiding volume HUD (a centered bar) — the gateway
half that sends these lives in a following commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 21:27:48 +02:00
jpmschweitzerandClaude Opus 4.8 9521deddc5 feat(dev): device screenshot over USB
Adds a way to capture the live LVGL screen off the device and rebuild it
as a PNG on the host, so UI changes can be verified remotely without a
camera. A watcher task polls the USB-serial-JTAG RX for a trigger byte and
streams the current screen as raw RGB565 straight to the USB FIFO (framed
by ###SHOT_BEGIN/END### with a CRC); firmware/tools/device_shot.py and the
device-screenshot skill drive it from the host.

Writing straight to the USB FIFO bypasses the primary UART console, which
at 115200 baud would take ~37s per frame. The whole capability is behind
DESKLOCK_DEVMODE (off by default, enable at deploy time with
-DDESKLOCK_DEVMODE=ON) so production spends no internal RAM on the watcher
and nothing extra runs on the render path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 20:24:41 +02:00
jpmschweitzerandClaude Opus 4.8 23f001ceb5 feat(face): tap-to-reveal volume controls
Tapping the face now brings up an overlay with a microphone button and
volume down/up, plus a live level bar showing the current output volume.
Icons are drawn from a subset of the Phosphor glyph font. Tapping the dim
scrim behind the controls dismisses them; they also auto-hide after a few
seconds of inactivity.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LKPbR6DY2JygHbyLjxm7Uu
2026-07-15 20:24:04 +02:00
38 changed files with 3652 additions and 232 deletions
+70
View File
@@ -0,0 +1,70 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/desklock"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(make -C gateway *)",
"Bash(make setup:*)",
"Bash(make run:*)",
"Bash(make test:*)",
"Bash(make lint:*)",
"Bash(make typecheck:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(.venv/bin/ruff:*)",
"Bash(.venv/bin/mypy:*)",
"Bash(docker logs desklock-gateway:*)",
"Bash(curl -s http://localhost:8600/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(rm -rf $HOME)",
"Bash(rm -rf /)",
"Bash(rm -rf ~)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+83
View File
@@ -0,0 +1,83 @@
---
name: device-screenshot
description: >
Capture a screenshot of the live DeskLock ESP32-P4 display over USB and view
it as a PNG. Use whenever you need to SEE what's on the device screen — verify
a face/UI change, check the tap controls overlay, confirm a state (idle,
listening, effort), or debug layout remotely without the user's camera.
Triggers: "screenshot the device", "what's on the screen", "capture the
display", "show me the face", "did the UI change land".
---
# DeskLock device screenshot over USB
Grabs the current LVGL screen off the device and rebuilds it as a PNG on the
host. No camera, no gateway — it rides the USB serial that's already attached for
flashing.
## How it works (so you can debug it)
The firmware (`main/face.c` `face_screenshot_dump`, gated by `DESKLOCK_DEVMODE`
in `desklock_main.c`) runs a task that watches the **USB-serial-JTAG RX FIFO**
for a trigger byte, renders the active screen with `lv_snapshot_take_to_draw_buf`
into a PSRAM buffer, 2×-downscales to 400×400, and streams it as **raw binary**
straight to the USB-serial-JTAG TX FIFO — framed by a text header
`###SHOT_BEGIN … bytes=N crc=0x… bin=1###` + N bytes + `###SHOT_END###`. The host
tool `firmware/tools/device_shot.py` sends the trigger, reads N bytes, checks the
CRC (retrying on the rare dropped frame), and writes a PNG.
Two things this design is deliberately built around, learned the hard way:
- **Don't use `printf`.** The primary console is UART at 115200 baud (~11 KB/s) —
a frame would take ~37 s. Writing straight to the USB FIFO runs at USB speed
(~1 s). That's why the dump uses `usb_serial_jtag_ll_write_txfifo`, not stdout.
- **Opening the port does NOT reset the P4** (unlike esptool), and there's no USB
stdin, so the trigger is a byte the firmware polls for — on-demand, no reboot,
captures whatever state is currently on screen.
## Prerequisites
- Firmware flashed with **dev mode ON** — it's OFF by default (production spends
no internal RAM on the watcher and nothing extra runs on the render path). Flash
a dev build with:
```bash
cd firmware && sg dialout -c "bash -c 'source ~/esp-idf/export.sh && idf.py -DDESKLOCK_DEVMODE=ON -p /dev/ttyACM0 flash'"
```
If screenshots return "no frame", the flashed build is production — reflash with
`-DDESKLOCK_DEVMODE=ON`. Back to production: `-DDESKLOCK_DEVMODE=OFF` (the value
sticks in the CMake cache until you flip it). Leave it ON for a UI-dev session
(you're reflashing for UI changes anyway); flip OFF for the final/production flash.
- Device on `/dev/ttyACM0`. The port needs the `dialout` group, so run the tool
under `sg dialout -c '…'` (this login session predates dialout membership).
- Nothing else holding the port (no `idf.py monitor` running) — one owner only.
## Take a shot
From the `desklock` repo root (`/mnt/media/Projects/desklock`):
```bash
# current screen (whatever state the device is in right now)
sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png"
# force the tap controls overlay in-frame (mic + volume) — for verifying it
# remotely since you can't physically tap
sg dialout -c "python3 firmware/tools/device_shot.py /tmp/shot.png --overlay"
```
Then **Read `/tmp/shot.png`** to view it. A clean run prints
`[try 1] 400x400 320000B crc … OK` and takes ~5 s.
Trigger bytes: `s` = current screen, `o` = force overlay. The tool retries up to
4× on a CRC mismatch (occasional console contention), so a transient bad frame
self-heals.
## Notes / caveats
- **RAM / render path:** the watcher costs one ~5 KB internal-RAM task, so it's
behind `DESKLOCK_DEVMODE` (off by default). Internal RAM is the scarce resource
on this board (it caps the rain sprite pool). Never ship a production build with
it on.
- 400×400 is plenty for layout/UI checks. To change resolution, adjust `SHOT_DS`
in `face.c` (the downscale factor) — the host reads the size from the header.
- If you just reflashed, wait ~7 s for boot before the first shot.
- Only the USB console path is used; this does not touch the device↔gateway
WebSocket protocol.
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+3 -3
View File
@@ -49,7 +49,7 @@ jobs:
- name: Login to Gitea Registry
uses: docker/login-action@v3
with:
registry: git.schweitz.internal
registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -61,8 +61,8 @@ jobs:
provenance: false
sbom: false
tags: |
git.schweitz.internal/jpmschweitzer/desklock-gateway:latest
git.schweitz.internal/jpmschweitzer/desklock-gateway:${{ github.ref_name }}
git.schweitz.net/jpmschweitzer/desklock-gateway:latest
git.schweitz.net/jpmschweitzer/desklock-gateway:${{ github.ref_name }}
- name: Trigger Watchtower update
if: success()
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+14
View File
@@ -21,3 +21,17 @@ dist/
.idea/
*.swp
.DS_Store
# Claude Code local overrides (per-machine, may hold credentials)
.claude/settings.local.json
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+19
View File
@@ -0,0 +1,19 @@
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCW5B4X073XJCWS4QTHRAVSG', 'description', NULL, 'The gateway accepts any client. `/ws/voice` and `/devices` check no credential, so anything on the LAN (or the tailnet) can open `ws://192.168.86.149:8600/ws/voice` and talk to Tatlock through the gateway — including Tatlock''s Home Assistant controls — or list the devices. Found in the workspace security sweep on 2026-09-23 by reading the gateway source: no token, auth or secret handling anywhere in `gateway/src/desklock_gateway/`.
Port 8600 has to stay LAN-open: it is the one address the firmware connects to (`GATEWAY_WS_URI` in `firmware/main/secrets.h`). So the fix is a credential, not a lockdown.
## Shape
- A per-device token, provisioned into firmware through `secrets.h` (already the secrets path, and already gitignored — check), sent on the WebSocket handshake (an `Authorization: Bearer` header, or a first frame if the ESP-IDF client makes headers awkward).
- The gateway compares it in constant time against its configured set (one token per device, so one can be revoked alone) and closes the socket on a mismatch before any audio or text is processed.
- `/devices` requires the same credential, or an admin one.
- Tokens reach the gateway container as a Portainer stack variable, never in the tracked stack file.
- The protocol change is recorded in `docs/architecture.md` (standing rule: device–gateway protocol changes update it).
## Rollout without a dark period
Gateway first in a mode that accepts both token and no-token and logs which each device used; then flash the devices; then make the token required. Verify: a connection without a token is refused, each flashed device still converses, and `/devices` without a credential is refused.
## Tests
A socket without a token is closed before the first frame is read; a wrong token likewise; a correct one converses; comparison is constant-time. Each mutation-checked.', NULL, '2026-09-23 11:45:03', '2026-09-23 11:45:03.865', '2026-09-23 11:45:03.865', NULL, '93027bc356d0bbc60ae0e38b9afdeb88', 2) ON CONFLICT(hash) DO NOTHING;
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+1
View File
@@ -0,0 +1 @@
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCW5B4X073XJCWS4QTHRAVSG', 'T-1', '2026-09-23 11:45:03.721', '2026-09-23 11:45:03.721', NULL, '4549dc91ac440f4b47a95e9072256904', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+20
View File
@@ -0,0 +1,20 @@
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCW5B4X073XJCWS4QTHRAVSG', 'story', NULL, 'Gateway: per-device token on /ws/voice and /devices', NULL, 'backlog', 'high', NULL, NULL, NULL, '2026-09-23 11:45:03.720', '2026-09-23 11:45:03.720', NULL, '7ef01f11d6709f6e094634f5aacc6789', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCW5B4X073XJCWS4QTHRAVSG', 'story', NULL, 'Gateway: per-device token on /ws/voice and /devices', 'The gateway accepts any client. `/ws/voice` and `/devices` check no credential, so anything on the LAN (or the tailnet) can open `ws://192.168.86.149:8600/ws/voice` and talk to Tatlock through the gateway — including Tatlock''s Home Assistant controls — or list the devices. Found in the workspace security sweep on 2026-09-23 by reading the gateway source: no token, auth or secret handling anywhere in `gateway/src/desklock_gateway/`.
Port 8600 has to stay LAN-open: it is the one address the firmware connects to (`GATEWAY_WS_URI` in `firmware/main/secrets.h`). So the fix is a credential, not a lockdown.
## Shape
- A per-device token, provisioned into firmware through `secrets.h` (already the secrets path, and already gitignored — check), sent on the WebSocket handshake (an `Authorization: Bearer` header, or a first frame if the ESP-IDF client makes headers awkward).
- The gateway compares it in constant time against its configured set (one token per device, so one can be revoked alone) and closes the socket on a mismatch before any audio or text is processed.
- `/devices` requires the same credential, or an admin one.
- Tokens reach the gateway container as a Portainer stack variable, never in the tracked stack file.
- The protocol change is recorded in `docs/architecture.md` (standing rule: device–gateway protocol changes update it).
## Rollout without a dark period
Gateway first in a mode that accepts both token and no-token and logs which each device used; then flash the devices; then make the token required. Verify: a connection without a token is refused, each flashed device still converses, and `/devices` without a credential is refused.
## Tests
A socket without a token is closed before the first frame is read; a wrong token likewise; a correct one converses; comparison is constant-time. Each mutation-checked.', 'backlog', 'high', NULL, NULL, NULL, '2026-09-23 11:45:03.720', '2026-09-23 11:45:03.865', NULL, 'de24fb675828ba55b4fdd3defd406cdb', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
-130
View File
@@ -1,130 +0,0 @@
# AGENTS.md
> Operational protocols and architecture for AI assistants working on DeskLock.
> Read [docs/architecture.md](docs/architecture.md) before making design changes.
## What this project is
DeskLock is the living-room visual/audio endpoint for **Tatlock**, the homelab butler
(`/mnt/media/Projects/tatlock`, API at `http://tatlock.schweitz.internal:8000`). Two halves,
one repo:
- `firmware/` — ESP-IDF (C, LVGL 9) app for the Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
- `gateway/` — Python FastAPI container on tower-of-joy orchestrating STT → chat
(Tatlock `/v1/chat/completions`) → TTS. Listens on port **8600**. STT/TTS models live
in the shared **Speaches** container (live on port 8601, OpenAI-format API), not in
the gateway image; `stt.py`/`tts.py` are pluggable backends (`speaches` default,
`embedded` fallback needing the `[speech]` extra). Gateway needs **Python ≥ 3.11**
(no ceiling; the container runs 3.13) — but system python3 on tower-of-joy is 3.8,
so `make setup` explicitly uses `python3.12`. No local audio resampling in the
default path: the gateway requests 16 kHz output via Speaches' `sample_rate`
extension (verified live).
The device and gateway speak a WebSocket protocol defined in `docs/architecture.md`.
**That doc is the contract** — update it in the same change as any protocol edit on
either side.
The face (black screen, ASCII glyph expressions, matrix rain as activity signal) is
designed in `sim/face/index.html` — the design source of truth — and specified in the
"Face design" section of `docs/architecture.md`. Change the sim and the doc together;
the LVGL implementation follows them. Verify sim changes visually with
`~/bin/claude-screenshot` (note: the tool uses `--virtual-time-budget`, which starves
`requestAnimationFrame` — drive sim animation with `setInterval`, which also mirrors
LVGL timers).
## Hard rules
- **Keep the firmware thin.** No STT, no TTS, no conversation logic on the device.
If a feature needs intelligence, it goes in the gateway or in Tatlock itself.
- **Never modify Tatlock from this repo.** It is a separate project with its own repo.
DeskLock consumes its public API only.
- **Secrets** (Wi-Fi credentials, any future API keys) never go in source. Firmware
gets them via a gitignored `firmware/secrets.h` (see AGENTS notes below) or NVS;
the gateway via environment variables (`DESKLOCK_*`).
## Firmware (`firmware/`)
- Toolchain: **ESP-IDF ≥ 5.4** (not Arduino, not PlatformIO). Target `esp32p4`.
- BSP: [`waveshare/esp32_p4_wifi6_touch_lcd_xc`](https://components.espressif.com/components/waveshare/esp32_p4_wifi6_touch_lcd_xc)
from the ESP Component Registry (pulled automatically via `main/idf_component.yml`).
- Reference implementations: [waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/` — notably `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi via ESP-Hosted). When wiring a new peripheral, check the
official example first; do not guess pin mappings.
```bash
# ESP-IDF v5.5 is installed at ~/esp-idf. Every shell:
source ~/esp-idf/export.sh
# Build / flash (device on USB-C at /dev/ttyACM0, CH343 bridge)
cd firmware
idf.py build
sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null && idf.py -p /dev/ttyACM0 flash'"
```
- `sg dialout -c '…'` is needed because the login session predates the user's dialout
membership; a plain `idf.py flash` works after any re-login.
- **Radio stack: esp_hosted ≥ 2.x on BOTH chips, non-negotiable.** esp-hosted 1.x is
formally incompatible with IDF 5.5 (esp-hosted-mcu#47) — symptom: RPC/scan/connect
all work, but NO data frames ever flow (no DHCP, no ARP, no ping). Waveshare's
examples pin 1.4.* and the factory C6 slave firmware is ancient — both wrong. The
host manifest pins `espressif/esp_hosted: "^2.12"`; the matching slave image is
embedded as `main/c6_slave.bin` and `c6_ota.c` flashes the C6 **over SDIO** at boot
whenever the C6 reports a version < 2.x (build a new bin from the component's
`slave/` project for esp32c6 when bumping versions).
- **Internal-RAM famine assert**: `assert failed: xTaskCreateStaticPinnedToCore …
xPortcheckValidStackMem` in a pre-app_main boot loop means static+early allocations
starved internal SRAM (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer
counts in sdkconfig.defaults; check `heap_init:` pool lines when the binary grows.
- SDIO clock is set conservatively (`CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`),
ample for 16 kHz voice; raising to 40 MHz is untested on this board's data path.
- **L1 driver diagnosis mode**: set `WIFI_DIAG_MODE 1` in desklock_main.c — the device
becomes AP `DESKLOCK-DIAG` (pass `desklock123`, page at http://192.168.4.1/) proving
radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control → L1
softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- **PSRAM must run at 200 MHz** or the 800×800 MIPI-DSI framebuffer underruns
(`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees, task watchdog).
`CONFIG_SPIRAM_SPEED_200M` only takes effect together with
`CONFIG_IDF_EXPERIMENTAL_FEATURES=y` — otherwise it is **silently dropped** and you
get 20 MHz. `sdkconfig.defaults` mirrors the official `08_lvgl_demo_v9` config.
- Non-interactive boot-log capture (avoid `idf.py monitor`, it's interactive): open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8 s. Verified boot
is ~1.6 s from reset to `desklock: DeskLock up`.
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gateway (`gateway/`)
```bash
cd gateway
make setup # venv + dev deps (no ML models)
make setup-speech # additionally install faster-whisper + piper
make run # uvicorn on :8600 with reload
make test # pytest
make lint # ruff check + format check
make typecheck # mypy
```
- Config via `DESKLOCK_*` env vars — see `src/desklock_gateway/config.py` for the schema
and defaults.
- `stt.py` / `tts.py` defer their heavy imports so the app boots without the `speech`
extra — keep it that way so protocol tests stay fast.
- Deployment is CI-driven: pushing a `v*` tag makes Gitea Actions test, build, and push
`desklock-gateway:{latest,tag}` to the registry and trigger Watchtower
(`.gitea/workflows/build.yml`; needs `REGISTRY_USER`/`REGISTRY_PASSWORD`/
`WATCHTOWER_HTTP_API_TOKEN` secrets). Plain pushes to `main` run lint + tests only. The
gateway deploys as part of the **`tatlock-ui` Portainer stack** —
`system-admin-toj/containers/stacks/tatlock-ui.yml` (registered in `CONTAINERS.md`,
port 8600). Stack updates go through the Portainer API on :8001 (JWT auth; recipe in
`system-admin-toj/containers/setup-new-host.md`), not by editing files on disk.
- Verify speech changes against the live Speaches container with a real round trip
(TTS → STT of a known phrase, expect the transcript back); warm timings to expect:
STT ~0.3 s, TTS ~2 s per sentence.
## Homelab context
- This server **is** tower-of-joy; the device, gateway, and Tatlock all share the LAN.
- Use `tatlock.schweitz.internal:8000` (direct, no SSO) — the public
`tatlock.schweitz.net` route sits behind Authentik and is not for machine-to-machine
traffic.
- Git remote: `git.schweitz.net` (Gitea).
+44
View File
@@ -8,6 +8,48 @@ until the first tagged release.
## [Unreleased]
### Changed
- One Makefile at the repo root now drives firmware, gateway and sim; `gateway/Makefile` is
removed. `make test` means the same thing from any directory.
- Firmware targets source `~/esp-idf/export.sh` themselves, so `idf.py` resolves without
having to remember. Override the location with `IDF_EXPORT=`.
- `make setup` now proves the gateway environment actually works instead of trusting a
clean `pip install` exit code: it collects the test suite and checks `ruff`/`mypy`
resolve in the venv, and fails the target if any of that is broken (T-47).
## [0.2.2] — 2026-07-19
### Changed
- The gateway's default Tatlock URL is now `http://tatlock:8000` (docker
container name) — the retiring `tatlock.schweitz.internal` domain is gone
from config and docs. Deployments setting `DESKLOCK_TATLOCK_BASE_URL` are
unaffected.
## [0.2.1] — 2026-07-15
### Fixed
- A spoken volume command no longer leaves the face stuck in the thinking
spinner — the feedback gong is a UI cue and no longer holds the busy state,
so the return-to-idle isn't swallowed.
- The butler filler line reads as one phrase instead of two — reworded to "Let
me check on that, sir." to avoid a text-to-speech pause before "for you".
## [0.2.0] — 2026-07-15
### Added
- After a request, the butler immediately says "Let me check on that for you,
sir" and keeps a spinner up while it works, so the long wait isn't dead air.
- Spoken volume commands ("volume up/down", "mute/unmute", "set volume to N",
"this one goes to eleven") are handled instantly on the gateway, bypassing the
assistant — no waiting on a reply just to change the volume.
- Tap the screen to reveal on-screen controls — a microphone button plus volume
down/up and a live level bar, using Phosphor icon glyphs. Tapping the dimmed
backdrop dismisses them.
### Fixed
- Fixed the blue-screen flicker during voice activity — the matrix rain now
@@ -16,5 +58,7 @@ until the first tagged release.
### Changed
- Volume is now an 0–11 scale (it goes to eleven) with a gong cue on each change
and a little "11" that floats off the bar at maximum.
- Matrix rain is drawn from a reused pool of pre-rendered streak sprites instead
of live text labels, and eases off while the device is listening or speaking.
+178 -34
View File
@@ -1,45 +1,189 @@
# CLAUDE.md
# CLAUDE.md — desklock
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
Two components, one repo, coupled by a shared WebSocket protocol:
Claude Code-specific notes for this project. For architecture, hard rules, and full
command reference — see [AGENTS.md](AGENTS.md), and read it before starting work.
- `firmware/` — ESP-IDF (C, LVGL 9) app for a Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C
(3.4" round 800×800 touch display, dual mics + ES7210 AEC, ES8311 codec + speaker).
Flashed over USB; not containerized.
- `gateway/` — Python/FastAPI container `desklock-gateway`, port **8600**, part of the
**`tatlock-ui`** Portainer stack (`system-admin-toj/containers/stacks/tatlock-ui.yml`,
verified against the live container and `CONTAINERS.md`). Orchestrates STT → Tatlock
chat → TTS; carries no ML dependencies itself.
## Quick orientation
The device↔gateway protocol is specified in `docs/architecture.md` under "WebSocket
protocol (device ↔ gateway)". **Any protocol change updates that file in the same
change** — it is the contract, not a description of one side's behavior.
DeskLock = firmware for a Waveshare ESP32-P4 round-display device (`firmware/`, ESP-IDF/C/LVGL)
plus a voice gateway container (`gateway/`, Python/FastAPI, port 8600) that bridges device
audio to the Tatlock butler API. The device↔gateway WebSocket protocol lives in
`docs/architecture.md` and must stay in sync with both implementations.
**Never modify Tatlock from this repo.** desklock consumes Tatlock's public API only
(`http://tatlock:8000` on `docker-dataplane`); this is a standing cross-repo rule, not
local policy.
**Keep the firmware thin.** No STT, no TTS, no conversation logic on the device — that
intelligence belongs in the gateway or in Tatlock itself.
**Secrets never go in source.** Firmware gets them via gitignored `firmware/main/secrets.h`
(verified: gitignored, and `#include`d by `gw_client.c`/`net.c`) or NVS; the gateway via
`DESKLOCK_*` environment variables.
## Gotchas — firmware
- Toolchain: **ESP-IDF ≥ 5.4** (this box has 5.5, installed at `~/esp-idf`), not
Arduino, not PlatformIO. Target `esp32p4`. `source ~/esp-idf/export.sh` is required
every shell — `idf.py` is not on the non-interactive PATH otherwise.
- BSP: `waveshare/esp32_p4_wifi6_touch_lcd_xc` from the ESP Component Registry, pulled
automatically via `main/idf_component.yml`. Reference implementations:
[waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC](https://github.com/waveshareteam/ESP32-P4-WIFI6-Touch-LCD-XC)
`examples/esp-idf/` — `08_lvgl_demo_v9` (display), `06_I2SCodec` (audio),
`04_wifistation` (Wi-Fi). Check the official example before guessing a pin mapping.
- Flashing needs the `dialout` group; a login session started before that membership
took effect needs `sg dialout -c "bash -lc 'source ~/esp-idf/export.sh >/dev/null &&
idf.py -p /dev/ttyACM0 flash'"` — a plain `idf.py flash` works after any re-login.
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`.** Without it,
`CONFIG_SPIRAM_SPEED_200M` is silently dropped to 20 MHz and the 800×800 MIPI-DSI
framebuffer underruns (`lcd.dsi.dpi: can't fetch data…` spam, LVGL lock never frees,
task watchdog). Verified both settings present in `firmware/sdkconfig.defaults`.
- **Wi-Fi radio stack must be esp_hosted ≥ 2.x on both the P4 host and the C6 slave,
non-negotiable.** 1.x is formally incompatible with IDF 5.5 (esp-hosted-mcu#47) —
symptom is control-plane-only: RPC/scan/connect all work, but no data frame ever
flows (no DHCP, no ARP, no ping). Waveshare's examples and the factory C6 slave
firmware both pin the wrong (1.x-era) version. The host manifest pins
`espressif/esp_hosted: "^2.12"` (verified in `firmware/main/idf_component.yml`); the
matching slave image is embedded as `main/c6_slave.bin`, and `c6_ota.c` flashes the
C6 over SDIO at boot whenever it reports a version below 2.x.
- **Boot-loop assert `xTaskCreateStaticPinnedToCore … xPortcheckValidStackMem`** before
`app_main` means internal SRAM starvation (hosted 2.x is hungry). Keep
`CONFIG_ESP_HOSTED_MEMPOOL_PREFER_SPIRAM=y` and the reduced `WIFI_RMT_*` buffer counts
in `sdkconfig.defaults` (both verified present); check `heap_init:` pool lines in the
boot log when the binary grows.
- SDIO clock is conservative by design: `CONFIG_ESP_HOSTED_SDIO_CLOCK_FREQ_KHZ=20000`
(verified), ample for 16 kHz voice — raising it to 40 MHz is untested on this board's
data path.
- **Wi-Fi diagnosis ladder**: set `WIFI_DIAG_MODE 1` in `desklock_main.c` (verified the
macro and `#if` guard exist, currently `0`) — the device becomes AP `DESKLOCK-DIAG`
(password `desklock123`, page at `http://192.168.4.1/`, verified in `wifi_diag.c`),
proving radio+SDIO+IP with zero external network variables. Ladder: L0 SDIO control →
L1 softap data → L2 STA to any network → L3 STA to "Outside" → L4 gateway.
- Non-interactive boot-log capture: avoid `idf.py monitor` (interactive) — open
`/dev/ttyACM0` at 115200 with pyserial, pulse RTS to reset, read ~8s. Reported boot
time (~1.6s to `desklock: DeskLock up`) is carried from `AGENTS.md` and was **not**
re-timed this pass — no device was connected in this session (see Liveness below).
- If the device doesn't enumerate, hold BOOT while pressing RESET for download mode.
## Gotchas — gateway
- Gateway speech deps (`faster-whisper`, `piper-tts`) are an optional extra —
`make setup` alone runs the app and the test suite without them. `make setup` invokes
`python3.12` explicitly; system `python3` on tower-of-joy is 3.8.
- `ruff` and `mypy` are **not** on the non-interactive PATH — they exist only inside
`gateway/.venv/bin/` once `make setup` has run. Use `make lint` / `make typecheck`, or
invoke `.venv/bin/ruff` / `.venv/bin/mypy` directly; a bare `ruff`/`mypy` will fail to
resolve, which is why the `.claude/settings.json` allow list uses the venv-relative
paths and `make` targets rather than bare tool names.
- Tatlock replies open with a `<think>` block — always strip it via
`tatlock.strip_reasoning()` (`gateway/src/desklock_gateway/tatlock.py`) before TTS or
display. Verified present and called at the one call site.
- Low power is a stated hardware requirement — read "Power management" in
`docs/architecture.md` before touching the face/render loop.
- Gateway health check is `GET /healthz` (verified in `main.py` and matches the
container healthcheck in `tatlock-ui.yml`), not `/health`.
## Commands
```bash
# Firmware (requires `source ~/esp-idf/export.sh` first; IDF ≥ 5.4)
cd firmware && idf.py build
idf.py -p /dev/ttyACM0 flash monitor
**One Makefile at the root drives all three components.** There is deliberately no
`gateway/Makefile` any more — `make test` meant "the gateway's tests" or "nothing"
depending on which directory you were standing in, and now it means the same thing
everywhere (workspace D-27).
# Gateway
cd gateway && make setup # once
make run # dev server :8600
make test # pytest; single test: .venv/bin/pytest tests/test_health.py -k healthz
make lint typecheck
```bash
make help # every target, self-documenting
make test # gateway pytest; reports firmware + sim as undetermined
make lint # ruff check + format --check
make typecheck # mypy
make setup # gateway venv + dev deps (no ML models)
make setup-speech # additionally faster-whisper + piper
make run # uvicorn on :8600 with reload
make build-firmware # sources export.sh for you, then idf.py build
make flash PORT=/dev/ttyACM0 # flash + monitor
make serve-sim # face simulator on :8601
```
## Gotchas
**The firmware targets source `~/esp-idf/export.sh` themselves.** `idf.py` is not on
`PATH` until that runs, so the old `cd firmware && idf.py build` fails with "command
not found" for anyone who forgets — the same class of failure as four other tool
misses on this host. Override with `IDF_EXPORT=<path>/export.sh` on another machine;
the target fails loudly with that hint if the file is absent.
- ESP-IDF v5.5 lives at `~/esp-idf` (`source ~/esp-idf/export.sh`). Flash via
`sg dialout -c …` (see AGENTS.md) — the login session predates dialout membership.
- **PSRAM 200 MHz requires `CONFIG_IDF_EXPERIMENTAL_FEATURES=y`** — without it the
option silently degrades to 20 MHz and the DSI display underruns into a watchdog
loop. Details in AGENTS.md.
- **Wi-Fi = esp_hosted 2.x on BOTH chips** (host manifest + C6 slave, auto-OTA'd from
`main/c6_slave.bin`). 1.x on IDF 5.5 gives working control RPC but a dead data path
(the great July 14th debugging night). Boot-loop assert on
`xTaskCreateStaticPinnedToCore` = internal-RAM famine. Details in AGENTS.md.
- Gateway speech deps are optional extras; `make setup` alone runs the app and tests
without GPU/ML packages. `make setup` uses `python3.12` (system python3 is 3.8).
- Tatlock replies open with a `<think>` block — always strip via
`tatlock.strip_reasoning()` before TTS or display.
- Low power is a prime user requirement: see "Power management" in
docs/architecture.md before touching the face/render loop.
`make test` never reports green for the firmware. It has no suite, so it is
**undetermined**, printed explicitly rather than skipped silently (workspace D-26).
## Liveness
- **Gateway (`desklock-gateway` container, port 8600):** confirmed live — `docker ps`
shows the container running under that name (method: direct container inspection;
blind spot: none relevant here, this confirms the process is up, not that every route
behaves correctly — that would need a request against it, not checked this pass).
- **Firmware / device:** liveness is **undetermined** and cannot be established the way
the gateway's can. No `/dev/ttyACM0` was present in this session (checked: `ls
/dev/ttyACM*` found nothing) and there is no remote telemetry — the device only proves
itself alive over a physical USB serial connection or by joining the LAN and speaking
the WebSocket protocol, neither of which this session had access to. Do not infer
device state from repo contents or from the gateway being up.
- `strip_reasoning()` reachability: confirmed by direct read of
`gateway/src/desklock_gateway/tatlock.py` (method: source read of the one call site;
blind spot: does not confirm it's exercised by a live request — that's what
`tests/test_tatlock.py` is for, not re-run this pass).
## Work tracking
This repo's vault is standalone — its tickets and internal decisions live in its own
`.pql/` and `governance/`, and travel with a clone (`.pql/changelog/` is committed).
```bash
/home/jpmschweitzer/.local/bin/pql ticket list
/home/jpmschweitzer/.local/bin/pql plan whatsnext
/home/jpmschweitzer/.local/bin/pql decisions list
```
`pql` is not on the non-interactive PATH — use the absolute path above. From inside this
repo no `--vault` flag is needed (pql anchors at the nearest `.git/` ancestor, which is
this repo) — but that also means a bare `pql` run from the **workspace root** will not
see this repo's tickets, and a write from the workspace root would go to the wrong
vault. Cross-repo/stack-level decisions (host, network, deploy mechanics — none specific
to desklock were found at the time of writing) live in the workspace vault instead:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain desklock
```
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that
is fast-forwarded and deleted. This is the workspace-wide policy; there is no
per-repo exception here.
- Conventional Commits (`feat:`, `fix:`, `refactor:`, `docs:`, `chore:`).
- Stage explicitly — never `git add -A` (denied by `.claude/settings.json` policy).
- Update `CHANGELOG.md` under `[Unreleased]` for user-facing changes.
## Releasing (gateway only — firmware has no release flow)
Deploy is not automatic — confirm one is wanted first.
1. Bump `version` in `gateway/pyproject.toml`.
2. Move `[Unreleased]` entries into a dated `CHANGELOG.md` section.
3. Commit, tag `vX.Y.Z`, push with tags.
4. `.gitea/workflows/build.yml` runs lint + pytest on every push to `main`; on a `v*`
tag it additionally builds and pushes
`git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}` and pings Watchtower.
5. Verify: `curl http://192.168.86.149:8600/healthz`.
## Architecture
`docs/architecture.md` is the source of truth for system design, the face design
(`sim/face/index.html` is its visual source — change both together and verify with
`~/bin/claude-screenshot`, noting its `--virtual-time-budget` starves
`requestAnimationFrame`, so sim animation is driven by `setInterval` instead), power
budget, latency budget, and the full WebSocket protocol spec. Not restated here because
it is detailed enough to drift if duplicated — read it directly.
+140
View File
@@ -0,0 +1,140 @@
# desklock — one entry point for a three-component repo.
#
# firmware/ ESP-IDF application for the device. No tests, no release flow.
# gateway/ Python service on :8600. The only component with a test suite.
# sim/ a static page that mimics the device face in a browser.
#
# This lives at the root and the components have no Makefiles of their own, so
# `make test` means the same thing wherever you are standing. A per-component
# Makefile makes it mean "some of the tests" depending on your working
# directory, which is the `git -C` failure in another costume (D-27).
#
# Paths resolve here rather than in callers (D-10). Two of them bite:
#
# ESP-IDF is invisible until export.sh is sourced, so `idf.py` is
# "command not found" for anyone who forgets — the same class of failure as
# the four tool-resolution misses recorded in D-24. The firmware targets
# source it themselves.
#
# `python3` on this host is 3.8, which cannot parse the gateway's sources.
# PYTHON names 3.12 explicitly and is overridable for other machines.
PYTHON ?= python3.12
IDF_EXPORT ?= $(HOME)/esp-idf/export.sh
GATEWAY := $(CURDIR)/gateway
VENV := $(GATEWAY)/.venv
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-16s\033[0m %s\n", $$1, $$2}'
# --- the three D-27 required targets -----------------------------------------
.PHONY: test
test: test-gateway ## Run every component's tests that exist
@echo " -- firmware: no test suite (undetermined, not passing)"
@echo " -- sim: a static page, nothing to test"
.PHONY: lint
lint: lint-gateway ## Lint every component that has a linter
# --- gateway ------------------------------------------------------------------
.PHONY: setup
setup: ## Gateway venv + dev deps + prove it works (firmware needs export.sh — see below)
cd $(GATEWAY) && $(PYTHON) -m venv .venv && .venv/bin/pip install -e ".[dev]"
@# This covers the gateway half only, deliberately. The firmware half needs
@# `source ~/esp-idf/export.sh` in every shell (see the firmware gotchas
@# above); a Makefile recipe runs in its own subshell, so it cannot leave
@# that sourced in the caller's shell. A `setup` that appeared to prepare
@# firmware and silently left `idf.py` unresolved would be worse than one
@# that says plainly it does not touch that half — hence `build-firmware`
@# sources export.sh itself, per target, instead.
@#
@# Exit 0 from `pip install` is not evidence (D-24) — pip reports success
@# even when the result is unusable (e.g. a dependency that resolved but
@# doesn't actually import, or a stale .venv left over from a different
@# Python). Prove the environment works instead of trusting the install
@# step: `--collect-only` imports every test module and therefore every
@# src module each one pulls in (T-47). It runs zero tests, so it stays
@# cheap, and it also confirms ruff/mypy landed in .venv/bin — the venv
@# is the only place either binary exists (see gateway gotchas above);
@# `--version` is enough to prove each resolves and runs.
cd $(GATEWAY) && .venv/bin/python -m pytest tests/ --collect-only -q
cd $(GATEWAY) && .venv/bin/ruff --version >/dev/null
cd $(GATEWAY) && .venv/bin/mypy --version >/dev/null
.PHONY: setup-speech
setup-speech: ## Additionally install faster-whisper and piper
cd $(GATEWAY) && .venv/bin/pip install -e ".[dev,speech]"
.PHONY: run
run: ## Run the gateway on :8600 with reload
cd $(GATEWAY) && .venv/bin/uvicorn desklock_gateway.main:app --host 0.0.0.0 --port 8600 --reload
.PHONY: test-gateway
test-gateway: ## Gateway pytest suite
@cd $(GATEWAY) && .venv/bin/pytest
.PHONY: lint-gateway
lint-gateway: ## ruff check and format --check over the gateway
cd $(GATEWAY) && .venv/bin/ruff check src tests && .venv/bin/ruff format --check src tests
.PHONY: typecheck
typecheck: ## mypy over the gateway sources
cd $(GATEWAY) && .venv/bin/mypy src
# --- firmware -----------------------------------------------------------------
#
# Each target sources export.sh in its own shell. That is deliberate: make runs
# every recipe line in a fresh shell, so exporting in one target would not carry
# to the next, and a caller who sources it by hand still works because sourcing
# twice is harmless.
.PHONY: build-firmware
build-firmware: ## Build the ESP-IDF firmware (sources export.sh for you)
@test -f $(IDF_EXPORT) || { echo "FAIL — no ESP-IDF at $(IDF_EXPORT); set IDF_EXPORT=<path>/export.sh"; exit 69; }
. $(IDF_EXPORT) && cd firmware && idf.py build
.PHONY: flash
flash: ## Flash and monitor the device (PORT=/dev/ttyACM0 by default)
@test -f $(IDF_EXPORT) || { echo "FAIL — no ESP-IDF at $(IDF_EXPORT); set IDF_EXPORT=<path>/export.sh"; exit 69; }
. $(IDF_EXPORT) && cd firmware && idf.py -p $(or $(PORT),/dev/ttyACM0) flash monitor
# --- sim ----------------------------------------------------------------------
.PHONY: serve-sim
serve-sim: ## Serve the face simulator on :8601
cd sim/face && $(PYTHON) -m http.server 8601
# --- housekeeping -------------------------------------------------------------
.PHONY: clean
clean: ## Remove the gateway venv and caches
rm -rf $(VENV) $(GATEWAY)/.pytest_cache $(GATEWAY)/.ruff_cache $(GATEWAY)/.mypy_cache
find $(GATEWAY) -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets lint typecheck test ## Everything the pre-push hook runs
+2 -2
View File
@@ -38,8 +38,8 @@ happens on this server.
▼ │ Speaches (container, GPU) │
┌────────────────────┐ │ • STT: faster-whisper │
│ Tatlock (butler) │ │ • TTS: Kokoro / Piper │
│ tatlock.schweitz. │ │ also usable by Open WebUI, │
│ internal :8000 │ │ Home Assistant, … │
│ http://tatlock │ │ also usable by Open WebUI, │
│ :8000 │ │ Home Assistant, … │
└────────────────────┘ └─────────────────────────────┘
```
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+41 -23
View File
@@ -24,8 +24,8 @@ tower-of-joy. Everything is local — no audio, transcript, or reply ever leaves
▼ │ Speaches (container, GPU) │
┌────────────────────┐ │ • STT: faster-whisper │
│ Tatlock (butler) │ │ • TTS: Kokoro / Piper │
│ tatlock.schweitz. │ │ also usable by Open WebUI, │
│ internal :8000 │ │ Home Assistant, … │
│ container name: │ │ also usable by Open WebUI, │
│ tatlock:8000 │ │ Home Assistant, … │
└────────────────────┘ └─────────────────────────────┘
```
@@ -74,7 +74,7 @@ models — the container stays a slim pure-Python image with no CUDA/ML dependen
2. Buffers inbound PCM until end-of-utterance (client-signalled in phase 1; VAD later).
3. **STT**: POST to Speaches `/v1/audio/transcriptions`.
4. **Chat**: POST the transcript to Tatlock `/v1/chat/completions`
(`http://tatlock.schweitz.internal:8000`, OpenAI-compatible, **streaming**),
(`http://tatlock:8000`, OpenAI-compatible, **streaming**),
maintaining the conversation history so follow-ups have context.
5. **TTS**: as Tatlock's token stream completes each sentence, POST it to Speaches
`/v1/audio/speech` and forward the PCM immediately — see
@@ -103,19 +103,25 @@ we may adopt later for streaming transcription.
extension, verified live; default voice `bm_george`, en-GB male). LAN-only like the
Tatlock internal route — do not expose through NPM without auth. Register in
`CONTAINERS.md`.
- **Measured** (live round trip through the gateway code, warm): STT ~0.3 s for a
~3 s utterance; TTS ~1.9 s for a ~3 s sentence. Cold start after model TTL offload
adds ~5–10 s to the first request.
- **Measured** (live round trip, warm, 2026-08-07): STT ~0.30 s for a ~4.8 s utterance;
TTS ~0.24 s for a ~4.5 s sentence (real-time factor ~0.05). The first call after an
idle gap costs ~1.2 s; a full cold start after model TTL offload adds ~4 s.
- **Why a shared layer instead of models inside the gateway**: one GPU-resident model
instance serves the whole homelab. Open WebUI is currently configured with
`AUDIO_STT_ENGINE=openai` / `AUDIO_TTS_ENGINE=openai` (OpenAI *cloud*) — pointing its
audio base URL at Speaches makes it fully local with a config change. Home Assistant
can share it too. Meanwhile the gateway image needs no CUDA and rebuilds in seconds.
- **VRAM budget**: RTX 2080 Ti, 11 GB, shared with Ollama (~3.6 GB in use as of
2026-07). whisper `small` at int8 is <1 GB; Kokoro is a few hundred MB. Speaches'
model TTL offload keeps idle pressure near zero. If VRAM contention ever bites,
faster-whisper `small` on CPU is an acceptable fallback (int8, a few seconds per
utterance).
- **VRAM budget**: RTX 2080 Ti, 11,264 MiB, shared with Ollama. As of 2026-08-07 the
steady state is ~4.9 GB used / ~5.9 GB free with everything resident: `gemma4:e2b`
1.9 GB and `nomic-embed-text` 0.3 GB (both pinned), whisper `small` int8 <1 GB,
Kokoro a few hundred MB. Speaches' model TTL offload keeps idle pressure near zero.
**This budget is not slack — it is the constraint.** On 2026-08-07 Tatlock was
deployed against `mistral-nemo:latest` (9.3 GB, 2 h keep-alive), which left 7 MiB
free and made every transcription fail with `CUDA failed with error out of memory`
while the Speaches container still reported healthy. Keep Tatlock's model at or below
~4 GB resident, and check `nvidia-smi` free VRAM before changing it. If contention
ever bites anyway, faster-whisper `small` on CPU is an acceptable fallback (int8, a
few seconds per utterance).
### 4. Tatlock — existing backend (`/mnt/media/Projects/tatlock`)
@@ -220,19 +226,22 @@ it in phase 5.
## Latency budget & streaming
Measured/known numbers that shape the design (Tatlock figures per tatlock CLAUDE.md,
GPU-resident benchmarks of 2026-07-14, gemma4:e2b at ~100 tok/s):
Measured 2026-08-07 against the deployed stack (`gemma4:e2b` at ~95 tok/s, GPU-resident):
| Stage | Cost |
|-------|------|
| STT (Speaches whisper `small`) | ~0.3 s warm (measured) |
| TTS (Speaches Kokoro) | ~1.9 s per ~3 s sentence, warm (measured) |
| Tatlock Steward analysis | ~6 s warm |
| **Tatlock, full local flow** | **11–25 s end-to-end** (librarian-routed ~20–25 s) |
| Tatlock cold start (>2 h idle) | +~8 s (`OLLAMA_KEEP_ALIVE=2h`) |
| STT (Speaches whisper `small`) | ~0.30 s warm, for ~4.8 s of audio |
| TTS (Speaches Kokoro) | ~0.24 s warm, for ~4.5 s of audio (RTF ~0.05) |
| **Tatlock, full local flow** | **~10–13 s end-to-end** for simple turns |
| Tatlock cold model load | +~36 s — avoided while the model is pinned |
(Older "~35 s Steward / ~2 min flow" figures were from a CPU-only driver-mismatch era —
do not plan against them.)
A Tatlock turn costs **3 sequential Ollama calls** (Steward routing → tool orchestration →
butler-tone synthesis) and ~710 generated tokens even for "what is 61 plus 12?". Most of
that is the model's own reasoning: gemma4 thinks by default, and the effort is spent three
times per turn.
(Older figures — "~35 s Steward / ~2 min flow" from the CPU-only era, and "11–25 s full
flow" from 2026-07-14 — are superseded. Do not plan against them.)
Speech is not the bottleneck — **Tatlock is**, by one to two orders of magnitude.
Constraints this imposes:
@@ -241,11 +250,11 @@ Constraints this imposes:
sentence-by-sentence**, forwarding audio as each sentence is ready. The device starts
speaking after the first sentence instead of waiting for the full reply — with
streaming, first audio should land roughly at Steward-time + first-sentence-time,
well under the 11–25 s full-flow figure. The WS protocol already supports this: one
well under the ~10–13 s full-flow figure. The WS protocol already supports this: one
`audio_start` … PCM … `audio_end` envelope with chunks arriving as they're
synthesized — the device just plays a continuous stream.
2. **The `thinking` face state is a first-class feature**, not decoration — it's what
makes a 10–25 s Tatlock turn feel intentional instead of broken. Consider progress
makes a ~10 s Tatlock turn feel intentional instead of broken. Consider progress
cues (e.g. surface Tatlock's reasoning summaries on-screen) later.
3. A **fast lane** may eventually be needed: MultiNet on-device commands for instant
home-automation phrases, and/or a low-latency intent path in Tatlock itself. Out of
@@ -266,8 +275,17 @@ gateway → device: {"type": "reply_text", "text": "..."}
gateway → device: {"type": "audio_start", "sample_rate": 16000}
gateway → device: <binary PCM frames> (may arrive sentence-by-sentence; play as a stream)
gateway → device: {"type": "audio_end"}
gateway → device: {"type": "command", "action": "volume_up"} (LLM-bypass; see below)
```
`command` (gateway → device) is an **alternative to the reply path**: when the
gateway recognizes a simple device command in the transcript (volume/mute), it
sends a `command` instead of calling Tatlock — no `reply_text`/audio — then returns
to `idle`. Actions: `volume_up`, `volume_down`, `mute`, `unmute`, and `volume_set`
with an extra `"level"` field (0–11, the on-device volume scale). Matched by the
gateway's `commands.py`; applied on the device in `gw_client.c` → `face.c`.
Planned additions (documented before implemented, here first):
- `reply_delta` (gateway → device): incremental reply text for on-screen streaming while
@@ -326,7 +344,7 @@ Gitea Actions (`.gitea/workflows/build.yml`), following the tatlock/tatlock-ui p
- **Every push to `main`**: lint + tests for the gateway (Python 3.12).
- **Version tags (`v0.1.0`, …)**: tests, then build `gateway/` into
`git.schweitz.internal/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
`git.schweitz.net/jpmschweitzer/desklock-gateway:{latest,tag}`, push to the
Gitea registry, create a release, and trigger Watchtower to roll the running
container.
- Required repo/org secrets: `REGISTRY_USER`, `REGISTRY_PASSWORD`,
+12
View File
@@ -10,9 +10,21 @@ idf_component_register(
"fonts/font_face_140.c"
"fonts/font_rain_22.c"
"fonts/font_rage_64.c"
"fonts/font_phosphor_64.c"
INCLUDE_DIRS "."
EMBED_FILES "c6_slave.bin"
PRIV_REQUIRES nvs_flash esp_wifi esp_event esp_netif json esp_timer esp_app_format esp_http_server esp-sr
)
target_compile_definitions(${COMPONENT_LIB} PRIVATE LV_LVGL_H_INCLUDE_SIMPLE)
# Dev-only features (screenshot watcher). OFF in production so nothing extra runs
# on the render path / spends internal RAM. Deploy dev with:
# idf.py -DDESKLOCK_DEVMODE=ON build flash
# and back to production with -DDESKLOCK_DEVMODE=OFF (the value sticks in the
# CMake cache until you flip it).
option(DESKLOCK_DEVMODE "Enable dev-only features (screenshot watcher)" OFF)
if(DESKLOCK_DEVMODE)
target_compile_definitions(${COMPONENT_LIB} PRIVATE DESKLOCK_DEVMODE=1)
message(STATUS "DeskLock: DEVMODE ON — screenshot watcher enabled")
endif()
+65 -5
View File
@@ -29,12 +29,20 @@ static const char *TAG = "audio";
#define GONG_SECONDS 5
#define GONG_SAMPLES (RATE * GONG_SECONDS)
#define GONG_VOLUME 75
#define GONG_PEAK 14000.0f
/* Speaker volume is an 11-step level (0..AUDIO_VOL_MAX). This one goes to eleven. */
#define VOL_DEFAULT_LEVEL 8 /* ~73%, matches the previous 75 default */
static esp_codec_dev_handle_t s_spk;
static esp_codec_dev_handle_t s_mic;
static int s_level = VOL_DEFAULT_LEVEL; /* speaker volume, 0..AUDIO_VOL_MAX */
static bool s_muted;
static int level_to_pct(int level) { return level * 100 / AUDIO_VOL_MAX; }
static int16_t *s_gong;
#define GONG_CHUNK 1600 /* 100 ms — cut-off granularity for a restart */
static volatile uint32_t s_gong_gen; /* bumped per request; a change mid-play restarts */
static volatile bool s_gong_active;
static uint8_t *s_reply;
static volatile size_t s_reply_len;
@@ -152,7 +160,21 @@ static void audio_task(void *arg)
* ourselves. The tail delay covers the DMA that plays after write()
* returns, preventing a playback-boundary false wake. */
if (job == JOB_GONG && s_gong != NULL) {
esp_codec_dev_write(s_spk, s_gong, GONG_SAMPLES * sizeof(int16_t));
/* Play in chunks so a new request (a fresh volume change) cuts the
* current gong off and restarts, instead of queueing another 5 s.
* Note: the gong is a UI cue, not "playback" — it deliberately does
* NOT set s_playing, so a following state:idle isn't swallowed (that
* guard is for reply audio) and it doesn't gate wake for 5 s. */
uint32_t gen;
do {
gen = s_gong_gen;
for (size_t off = 0; off < GONG_SAMPLES && s_gong_gen == gen; off += GONG_CHUNK) {
size_t n = GONG_SAMPLES - off;
if (n > GONG_CHUNK) n = GONG_CHUNK;
esp_codec_dev_write(s_spk, s_gong + off, n * sizeof(int16_t));
}
} while (s_gong_gen != gen); /* a new request arrived mid-play → restart */
s_gong_active = false;
} else if (job == JOB_CHIME && s_chime != NULL) {
s_playing = true;
esp_codec_dev_write(s_spk, s_chime, CHIME_SAMPLES * sizeof(int16_t));
@@ -276,7 +298,7 @@ void audio_init(void)
};
s_spk = bsp_audio_codec_speaker_init();
if (s_spk != NULL) {
esp_codec_dev_set_out_vol(s_spk, GONG_VOLUME);
esp_codec_dev_set_out_vol(s_spk, level_to_pct(s_level));
esp_codec_dev_open(s_spk, &fs);
}
s_mic = bsp_audio_codec_microphone_init();
@@ -303,8 +325,14 @@ void audio_play_chime(void)
void audio_play_gong(void)
{
job_t job = JOB_GONG;
xQueueSend(s_jobs, &job, 0);
s_gong_gen++; /* (re)start; cuts any in-flight gong */
if (!s_gong_active) {
s_gong_active = true;
job_t job = JOB_GONG;
if (xQueueSend(s_jobs, &job, 0) != pdTRUE) {
s_gong_active = false; /* queue full — nothing will run it */
}
}
}
/* Streaming = the detect_task forwards AFE audio upstream. Wake word and touch
@@ -348,3 +376,35 @@ bool audio_is_playing(void)
{
return s_playing;
}
void audio_level_set(int level)
{
if (level < 0) {
level = 0;
} else if (level > AUDIO_VOL_MAX) {
level = AUDIO_VOL_MAX;
}
s_level = level;
s_muted = false; /* setting a level unmutes */
if (s_spk != NULL) {
esp_codec_dev_set_out_vol(s_spk, level_to_pct(s_level));
}
}
int audio_level_get(void)
{
return s_level;
}
void audio_mute(bool on)
{
s_muted = on; /* gate output; s_level is preserved */
if (s_spk != NULL) {
esp_codec_dev_set_out_vol(s_spk, on ? 0 : level_to_pct(s_level));
}
}
bool audio_is_muted(void)
{
return s_muted;
}
+7
View File
@@ -22,6 +22,8 @@ void face_set(face_state_t state); /* safe from any task */
face_state_t face_get(void);
void face_activity(void);
void face_status(const char *text); /* bottom status line (diag/boot) */ /* reset the power ladder to active */
void face_screenshot_dump(bool overlay); /* dev: raw RGB565 screen dump over USB */
void face_volume_command(const char *action, int level); /* apply a gateway volume command */
/* audio.c */
void audio_init(void);
@@ -34,6 +36,11 @@ void audio_playback_begin(void);
void audio_playback_feed(const uint8_t *data, size_t len);
void audio_playback_end(void);
bool audio_is_playing(void);
#define AUDIO_VOL_MAX 11 /* volume level range 0..11 (goes to eleven) */
void audio_level_set(int level); /* 0..AUDIO_VOL_MAX; also unmutes */
int audio_level_get(void);
void audio_mute(bool on); /* gate output; keeps the set level */
bool audio_is_muted(void);
/* net.c */
void net_start(void);
+43 -1
View File
@@ -30,6 +30,16 @@
* 0 = normal app. */
#define FACE_LOADTEST 0
/* Dev mode: enabled at deploy time with `idf.py -DDESKLOCK_DEVMODE=ON build flash`
* (see main/CMakeLists.txt), OFF by default in production. Currently gates only the
* screenshot watcher — a task that watches the USB-serial-JTAG RX for a trigger
* byte and dumps the current screen (see the device-screenshot skill /
* firmware/tools/device_shot.py). Off by default so production spends no internal
* RAM on it and nothing extra runs on the render path. */
#ifndef DESKLOCK_DEVMODE
#define DESKLOCK_DEVMODE 0
#endif
static const char *TAG = "desklock";
/* All utterance framing (blocking WS sends, chime, face) runs on app_task, never
@@ -112,7 +122,12 @@ static void app_task(void *arg)
void app_on_playback_done(void)
{
face_set(FACE_IDLE);
/* Only fall back to idle if we're still the speaker. After the butler filler
* line, the gateway re-asserts "thinking" (FACE_EFFORT) while Tatlock works —
* don't clobber that spinner when the filler audio finishes. */
if (face_get() == FACE_SPEAKING) {
face_set(FACE_IDLE);
}
}
#if FACE_LOADTEST
@@ -232,6 +247,29 @@ void sdio_tx_selftest_kick(void)
}
#endif
#if DESKLOCK_DEVMODE
#include "hal/usb_serial_jtag_ll.h"
/* Watch the USB-serial-JTAG RX FIFO (LL reads, no driver install -> no conflict
* with the secondary console's TX) for a trigger byte and dump one frame. */
static void screenshot_task(void *arg)
{
(void)arg;
vTaskDelay(pdMS_TO_TICKS(6000)); /* let boot + first render settle */
ESP_LOGI(TAG, "screenshot: ready — send 's' on USB serial to capture a frame");
uint8_t rx[16];
for (;;) {
if (usb_serial_jtag_ll_rxfifo_data_available()) {
int n = usb_serial_jtag_ll_read_rxfifo(rx, sizeof(rx));
for (int i = 0; i < n; i++) {
if (rx[i] == 's') { face_screenshot_dump(false); break; } /* current screen */
if (rx[i] == 'o') { face_screenshot_dump(true); break; } /* force overlay */
}
}
vTaskDelay(pdMS_TO_TICKS(150));
}
}
#endif
void app_main(void)
{
ESP_LOGI(TAG, "DeskLock starting");
@@ -287,4 +325,8 @@ void app_main(void)
#if FACE_LOADTEST
xTaskCreate(loadtest_task, "loadtest", 4096, NULL, 4, NULL);
#endif
#if DESKLOCK_DEVMODE
xTaskCreate(screenshot_task, "shot", 5120, NULL, 4, NULL);
#endif
}
+351 -1
View File
@@ -7,8 +7,12 @@
#include <string.h>
#include <time.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_random.h"
#include "esp_heap_caps.h"
#include "esp_rom_crc.h"
#include "hal/usb_serial_jtag_ll.h"
#include "bsp/esp-bsp.h"
#include "lvgl.h"
@@ -16,6 +20,17 @@
LV_FONT_DECLARE(font_face_140);
LV_FONT_DECLARE(font_rain_22);
LV_FONT_DECLARE(font_phosphor_64);
/* Phosphor icon glyphs (Private Use Area, UTF-8) baked into font_phosphor_64. */
#define ICON_MIC "\xEE\x8C\xA6" /* ph-microphone U+E326 */
#define ICON_MIC_OFF "\xEE\x8C\xA8" /* ph-microphone-slash U+E328 */
#define ICON_VOL_UP "\xEE\x91\x8A" /* ph-speaker-high U+E44A */
#define ICON_VOL_DOWN "\xEE\x91\x8C" /* ph-speaker-low U+E44C */
#define ICON_VOL_MUTE "\xEE\x91\x9A" /* ph-speaker-slash U+E45A */
#define ICON_EQ "\xEE\xAE\xBC" /* ph-equalizer U+EBBC */
#define ICON_SLIDERS "\xEE\x90\xB2" /* ph-sliders U+E432 */
#define ICON_SLIDERS_H "\xEE\x90\xB4" /* ph-sliders-horizontal U+E434 */
LV_FONT_DECLARE(font_rage_64);
#define SCREEN 800
@@ -111,6 +126,14 @@ static struct {
lv_obj_t *status;
lv_obj_t *elapsed;
lv_obj_t *arc;
lv_obj_t *controls; /* tap-to-reveal control overlay (mic + volume) */
lv_obj_t *vol_bar;
lv_obj_t *vol_lbl; /* level number 0..11, or "MUTE" */
lv_timer_t *ctl_timer; /* one-shot auto-hide for the controls */
lv_obj_t *vol_hud; /* lightweight volume feedback for voice commands */
lv_obj_t *hud_bar;
lv_obj_t *hud_lbl;
lv_timer_t *hud_timer; /* one-shot auto-hide for the HUD */
stream_t streams[RAIN_MAX];
int arc_rot;
uint32_t state_since;
@@ -407,10 +430,200 @@ static void clock_cb(lv_timer_t *timer)
}
}
/* --- tap-to-reveal controls: mic + volume down/up + level bar --- */
static void controls_update_vol(void)
{
int lvl = audio_level_get();
lv_bar_set_value(F.vol_bar, lvl, LV_ANIM_OFF);
if (audio_is_muted()) {
lv_label_set_text(F.vol_lbl, "MUTE");
} else {
char buf[8];
snprintf(buf, sizeof(buf), "%d", lvl);
lv_label_set_text(F.vol_lbl, buf);
}
}
static void controls_hide(void)
{
lv_obj_add_flag(F.controls, LV_OBJ_FLAG_HIDDEN);
lv_timer_pause(F.ctl_timer);
}
static void controls_show(void)
{
controls_update_vol();
lv_obj_clear_flag(F.controls, LV_OBJ_FLAG_HIDDEN);
face_activity(); /* wake the screen from any dim state */
lv_timer_reset(F.ctl_timer);
lv_timer_resume(F.ctl_timer);
}
/* Audible feedback at the new volume: any non-silent change plays the gong. A new
* change cuts the in-flight gong off and restarts it (audio.c) rather than queueing. */
static void volume_feedback(void)
{
if (!audio_is_muted() && audio_level_get() > 0) {
audio_play_gong();
}
}
/* "This one goes to eleven": a big 11 drifts up and fades when max is hit. */
static void eleven_ty_cb(void *o, int32_t v) { lv_obj_set_style_translate_y((lv_obj_t *)o, v, 0); }
static void eleven_opa_cb(void *o, int32_t v) { lv_obj_set_style_text_opa((lv_obj_t *)o, (lv_opa_t)v, 0); }
static void eleven_del_cb(lv_anim_t *a) { lv_obj_del((lv_obj_t *)a->var); }
static void eleven_pop(lv_obj_t *anchor)
{
lv_obj_t *l = lv_label_create(lv_layer_top()); /* floats above everything */
lv_label_set_text(l, "11");
lv_obj_set_style_text_font(l, &lv_font_montserrat_48, 0);
lv_obj_set_style_text_color(l, lv_color_hex(0xC3FFD7), 0);
lv_obj_update_layout(l);
lv_area_t a; /* start just above the bar */
lv_obj_get_coords(anchor, &a);
lv_obj_set_pos(l, (a.x1 + a.x2) / 2 - lv_obj_get_width(l) / 2,
a.y1 - lv_obj_get_height(l) - 4);
lv_anim_t up; /* drift up (via translate, keeps align) */
lv_anim_init(&up);
lv_anim_set_var(&up, l);
lv_anim_set_exec_cb(&up, eleven_ty_cb);
lv_anim_set_values(&up, 0, -200);
lv_anim_set_duration(&up, 1100);
lv_anim_set_path_cb(&up, lv_anim_path_ease_out);
lv_anim_set_deleted_cb(&up, eleven_del_cb); /* deletes the label (cancels the fade too) */
lv_anim_start(&up);
lv_anim_t fade;
lv_anim_init(&fade);
lv_anim_set_var(&fade, l);
lv_anim_set_exec_cb(&fade, eleven_opa_cb);
lv_anim_set_values(&fade, LV_OPA_COVER, LV_OPA_TRANSP);
lv_anim_set_duration(&fade, 1100);
lv_anim_start(&fade);
}
/* --- voice-command volume HUD (shown when a gateway command changes volume) --- */
static void hud_update(void)
{
int lvl = audio_level_get();
if (audio_is_muted()) {
lv_bar_set_value(F.hud_bar, 0, LV_ANIM_OFF);
lv_label_set_text(F.hud_lbl, "MUTE");
} else {
lv_bar_set_value(F.hud_bar, lvl, LV_ANIM_OFF);
char b[8];
snprintf(b, sizeof(b), "%d", lvl);
lv_label_set_text(F.hud_lbl, b);
}
}
static void hud_autohide_cb(lv_timer_t *t)
{
(void)t;
lv_obj_add_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
lv_timer_pause(F.hud_timer);
}
static void hud_show(void)
{
hud_update();
lv_obj_clear_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
face_activity();
lv_timer_reset(F.hud_timer);
lv_timer_resume(F.hud_timer);
}
/* Apply a gateway volume command (runs on the websocket task). */
void face_volume_command(const char *action, int level)
{
if (strcmp(action, "volume_up") == 0) {
audio_level_set(audio_level_get() + 1);
} else if (strcmp(action, "volume_down") == 0) {
audio_level_set(audio_level_get() - 1);
} else if (strcmp(action, "volume_set") == 0) {
audio_level_set(level);
} else if (strcmp(action, "mute") == 0) {
audio_mute(true);
} else if (strcmp(action, "unmute") == 0) {
audio_mute(false);
} else {
return;
}
bsp_display_lock(UINT32_MAX);
hud_show();
if (!audio_is_muted() && audio_level_get() == AUDIO_VOL_MAX) {
eleven_pop(F.hud_bar);
}
bsp_display_unlock();
volume_feedback();
}
static void ctl_autohide_cb(lv_timer_t *t) { (void)t; controls_hide(); }
static void mic_cb(lv_event_t *e)
{
(void)e;
controls_hide();
app_on_touch(); /* the existing tap-to-talk action */
}
static void volup_cb(lv_event_t *e)
{
(void)e;
audio_level_set(audio_level_get() + 1);
controls_update_vol();
if (audio_level_get() == AUDIO_VOL_MAX) {
eleven_pop(F.vol_bar);
}
volume_feedback();
lv_timer_reset(F.ctl_timer);
}
static void voldown_cb(lv_event_t *e)
{
(void)e;
audio_level_set(audio_level_get() - 1);
controls_update_vol();
volume_feedback();
lv_timer_reset(F.ctl_timer);
}
/* tap anywhere (the underlying catcher, reachable only while controls are hidden)
* reveals the controls; a tap on the dim scrim behind them dismisses. */
static void touch_cb(lv_event_t *e)
{
(void)e;
app_on_touch();
controls_show();
}
static void scrim_cb(lv_event_t *e)
{
(void)e;
controls_hide();
}
static lv_obj_t *icon_btn(lv_obj_t *parent, const char *icon, int diam, lv_event_cb_t cb)
{
lv_obj_t *b = lv_button_create(parent);
lv_obj_set_size(b, diam, diam);
lv_obj_set_style_radius(b, LV_RADIUS_CIRCLE, 0);
lv_obj_set_style_bg_color(b, lv_color_hex(0x0A140E), 0);
lv_obj_set_style_bg_opa(b, LV_OPA_COVER, 0);
lv_obj_set_style_border_color(b, lv_color_hex(0x00A050), 0);
lv_obj_set_style_border_width(b, 2, 0);
lv_obj_set_style_shadow_width(b, 0, 0);
lv_obj_t *lbl = lv_label_create(b);
lv_obj_set_style_text_font(lbl, &font_phosphor_64, 0);
lv_obj_set_style_text_color(lbl, lv_color_hex(0xC3FFD7), 0);
lv_label_set_text(lbl, icon);
lv_obj_center(lbl);
lv_obj_add_event_cb(b, cb, LV_EVENT_CLICKED, NULL);
return b;
}
void face_init(void)
@@ -507,6 +720,77 @@ void face_init(void)
lv_obj_add_flag(touch, LV_OBJ_FLAG_CLICKABLE);
lv_obj_add_event_cb(touch, touch_cb, LV_EVENT_CLICKED, NULL);
/* --- tap-to-reveal control overlay: dim scrim + mic + volume down/up + bar --- */
F.controls = lv_obj_create(scr); /* above the touch catcher */
lv_obj_remove_style_all(F.controls);
lv_obj_set_size(F.controls, SCREEN, SCREEN);
lv_obj_set_style_bg_color(F.controls, lv_color_black(), 0);
lv_obj_set_style_bg_opa(F.controls, 200, 0); /* dim the face behind */
lv_obj_add_flag(F.controls, LV_OBJ_FLAG_CLICKABLE);
lv_obj_clear_flag(F.controls, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_add_event_cb(F.controls, scrim_cb, LV_EVENT_CLICKED, NULL);
lv_obj_add_flag(F.controls, LV_OBJ_FLAG_HIDDEN); /* hidden until tapped */
lv_obj_t *mic = icon_btn(F.controls, ICON_MIC, 150, mic_cb);
lv_obj_align(mic, LV_ALIGN_CENTER, 0, -30);
lv_obj_t *vdn = icon_btn(F.controls, ICON_VOL_DOWN, 110, voldown_cb);
lv_obj_align(vdn, LV_ALIGN_CENTER, -230, -30);
lv_obj_t *vup = icon_btn(F.controls, ICON_VOL_UP, 110, volup_cb);
lv_obj_align(vup, LV_ALIGN_CENTER, 230, -30);
F.vol_bar = lv_bar_create(F.controls);
lv_obj_set_size(F.vol_bar, 360, 16);
lv_obj_align(F.vol_bar, LV_ALIGN_CENTER, 0, 140);
lv_bar_set_range(F.vol_bar, 0, AUDIO_VOL_MAX);
lv_obj_set_style_bg_color(F.vol_bar, lv_color_hex(0x0A140E), LV_PART_MAIN);
lv_obj_set_style_bg_opa(F.vol_bar, LV_OPA_COVER, LV_PART_MAIN);
lv_obj_set_style_bg_color(F.vol_bar, lv_color_hex(0x00C860), LV_PART_INDICATOR);
lv_obj_set_style_radius(F.vol_bar, 8, LV_PART_MAIN);
lv_obj_set_style_radius(F.vol_bar, 8, LV_PART_INDICATOR);
F.vol_lbl = lv_label_create(F.controls);
lv_obj_set_style_text_font(F.vol_lbl, &lv_font_montserrat_28, 0);
lv_obj_set_style_text_color(F.vol_lbl, lv_color_hex(0xC3FFD7), 0);
lv_label_set_text(F.vol_lbl, "");
lv_obj_align(F.vol_lbl, LV_ALIGN_CENTER, 0, 180);
F.ctl_timer = lv_timer_create(ctl_autohide_cb, 5000, NULL);
lv_timer_pause(F.ctl_timer);
/* voice-command volume HUD: just a slim volume bar (+ level number), centered
* and auto-hiding — separate from the tap overlay so a spoken "volume up"
* doesn't pop the mic button. The "11" animation flows off this bar. */
F.vol_hud = lv_obj_create(scr);
lv_obj_remove_style_all(F.vol_hud);
lv_obj_set_size(F.vol_hud, 560, 64);
lv_obj_center(F.vol_hud);
lv_obj_set_style_bg_color(F.vol_hud, lv_color_black(), 0);
lv_obj_set_style_bg_opa(F.vol_hud, 180, 0);
lv_obj_set_style_radius(F.vol_hud, 16, 0);
lv_obj_set_style_pad_all(F.vol_hud, 10, 0);
lv_obj_clear_flag(F.vol_hud, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_add_flag(F.vol_hud, LV_OBJ_FLAG_HIDDEN);
/* bar centered (symmetric on screen); level number hangs off to the right */
F.hud_bar = lv_bar_create(F.vol_hud);
lv_obj_set_size(F.hud_bar, 360, 18);
lv_obj_align(F.hud_bar, LV_ALIGN_CENTER, 0, 0);
lv_bar_set_range(F.hud_bar, 0, AUDIO_VOL_MAX);
lv_obj_set_style_bg_color(F.hud_bar, lv_color_hex(0x0A140E), LV_PART_MAIN);
lv_obj_set_style_bg_opa(F.hud_bar, LV_OPA_COVER, LV_PART_MAIN);
lv_obj_set_style_bg_color(F.hud_bar, lv_color_hex(0x00C860), LV_PART_INDICATOR);
lv_obj_set_style_radius(F.hud_bar, 9, LV_PART_MAIN);
lv_obj_set_style_radius(F.hud_bar, 9, LV_PART_INDICATOR);
F.hud_lbl = lv_label_create(F.vol_hud);
lv_obj_set_style_text_font(F.hud_lbl, &lv_font_montserrat_28, 0);
lv_obj_set_style_text_color(F.hud_lbl, lv_color_hex(0xC3FFD7), 0);
lv_label_set_text(F.hud_lbl, "");
lv_obj_align(F.hud_lbl, LV_ALIGN_RIGHT_MID, -6, 0);
F.hud_timer = lv_timer_create(hud_autohide_cb, 2000, NULL);
lv_timer_pause(F.hud_timer);
/* Boot straight into the calm idle face (dry — rain gate holds it off until
* the gateway connects). No "CONNECTING" banner churn on every recovery. */
F.state = FACE_IDLE;
@@ -571,3 +855,69 @@ void face_activity(void)
bsp_display_unlock();
}
}
/* --- dev screenshot over USB (see the device-screenshot skill) --------------- */
/* Blocking write straight to the USB-serial-JTAG TX FIFO. Bypasses stdout/printf
* — the primary console is UART at 115200 baud (~11 KB/s), far too slow for a
* frame; the USB FIFO runs at USB speed. Waits for FIFO space so nothing drops. */
static void shot_usb_write(const uint8_t *data, size_t len)
{
size_t sent = 0;
while (sent < len) {
if (!usb_serial_jtag_ll_txfifo_writable()) { vTaskDelay(1); continue; }
sent += usb_serial_jtag_ll_write_txfifo(data + sent, len - sent);
usb_serial_jtag_ll_txfifo_flush();
}
}
/* Render the live screen to an RGB565 buffer, 2x-downscale it, and stream it as
* RAW BINARY straight over the USB-serial-JTAG, framed by a ###SHOT_BEGIN ...###
* text header (carrying byte count + CRC) and ###SHOT_END###, so the host can
* rebuild a PNG. `overlay` forces the tap controls in-frame. Called only from the
* DESKLOCK_DEVMODE screenshot watcher; not part of normal operation. */
#define SHOT_DS 2 /* downscale factor (2 -> 400x400) */
void face_screenshot_dump(bool overlay)
{
/* Snapshot into our OWN PSRAM buffer — the default lv_snapshot_take() allocs
* the 1.28 MB frame from the small LVGL heap and returns NULL. */
const int W = SCREEN, H = SCREEN;
const uint32_t stride = lv_draw_buf_width_to_stride(W, LV_COLOR_FORMAT_RGB565);
const size_t buf_size = (size_t)stride * H;
uint8_t *mem = heap_caps_malloc(buf_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (!mem) { printf("###SHOT_FAIL mem###\n"); return; }
lv_draw_buf_t dbuf;
lv_draw_buf_init(&dbuf, W, H, LV_COLOR_FORMAT_RGB565, stride, mem, buf_size);
bsp_display_lock(UINT32_MAX);
if (overlay) { /* force the tap overlay in-frame */
controls_show();
lv_timer_pause(F.ctl_timer); /* don't let it auto-hide mid-capture */
}
lv_result_t rc = lv_snapshot_take_to_draw_buf(lv_screen_active(), LV_COLOR_FORMAT_RGB565, &dbuf);
bsp_display_unlock();
if (rc != LV_RESULT_OK) { printf("###SHOT_FAIL take=%d###\n", (int)rc); heap_caps_free(mem); return; }
const int ow = W / SHOT_DS, oh = H / SHOT_DS;
const size_t olen = (size_t)ow * oh * 2;
uint8_t *out = heap_caps_malloc(olen, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (!out) { printf("###SHOT_FAIL malloc###\n"); heap_caps_free(mem); return; }
for (int y = 0; y < oh; y++) {
const uint16_t *src = (const uint16_t *)(dbuf.data + (size_t)(y * SHOT_DS) * dbuf.header.stride);
uint16_t *dst = (uint16_t *)(out + (size_t)y * ow * 2);
for (int x = 0; x < ow; x++) dst[x] = src[x * SHOT_DS];
}
heap_caps_free(mem);
const uint32_t crc = esp_rom_crc32_le(0, out, olen);
char hdr[128];
int hlen = snprintf(hdr, sizeof(hdr),
"\n###SHOT_BEGIN w=%d h=%d fmt=rgb565le bytes=%u crc=0x%08x bin=1###\n",
ow, oh, (unsigned)olen, (unsigned)crc);
shot_usb_write((const uint8_t *)hdr, hlen);
shot_usb_write(out, olen); /* raw RGB565-LE, exactly `bytes` long */
static const char end[] = "\n###SHOT_END###\n";
shot_usb_write((const uint8_t *)end, sizeof(end) - 1);
heap_caps_free(out);
}
File diff suppressed because it is too large Load Diff
+6
View File
@@ -52,6 +52,12 @@ static void handle_text(const char *data, int len)
audio_playback_end();
} else if (strcmp(type->valuestring, "error") == 0) {
face_set(FACE_RAGE);
} else if (strcmp(type->valuestring, "command") == 0) {
const cJSON *action = cJSON_GetObjectItem(root, "action");
const cJSON *level = cJSON_GetObjectItem(root, "level");
if (cJSON_IsString(action)) {
face_volume_command(action->valuestring, cJSON_IsNumber(level) ? level->valueint : 0);
}
}
cJSON_Delete(root);
}
+4
View File
@@ -32,6 +32,10 @@ CONFIG_ESP_WIFI_SOFTAP_SUPPORT=y
# custom fonts are uncompressed, but enable the decoder as belt-and-braces
CONFIG_LV_USE_FONT_COMPRESSED=y
# lv_snapshot_take() — render the live screen to a buffer for the dev
# screenshot-over-USB dump (face_screenshot_dump)
CONFIG_LV_USE_SNAPSHOT=y
# ESP-Hosted board variant + wifi-remote data-path tuning (from factory brookesia config;
# without these the RPC control path works but data frames never flow)
CONFIG_SLAVE_IDF_TARGET_ESP32C6=y
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""Capture a screenshot from the DeskLock device over USB.
The firmware (SCREENSHOT_ENABLE) watches the USB-serial-JTAG RX for a trigger
byte and streams the current screen as a text header
###SHOT_BEGIN w=.. h=.. fmt=rgb565le bytes=N crc=0xXXXX bin=1###\n
followed by exactly N raw RGB565-LE bytes, then \n###SHOT_END###. We send the
trigger, read the payload, verify the CRC, and write a PNG.
Usage (run in the dialout group, e.g. under `sg dialout -c '...'`):
python3 device_shot.py [out.png] [--overlay] [--port /dev/ttyACM0]
--overlay send 'o' (force the tap controls overlay) instead of 's' (current screen)
"""
import os, sys, termios, select, time, zlib, re
port = "/dev/ttyACM0"
out = "shot.png"
trig = b"s"
args = sys.argv[1:]
while args:
a = args.pop(0)
if a == "--overlay": trig = b"o"
elif a == "--port": port = args.pop(0)
elif not a.startswith("-"): out = a
else: sys.stderr.write(f"[warn] ignoring {a}\n")
ATTEMPTS = 4
PER_TRY = 8.0
begin_re = re.compile(
rb"###SHOT_BEGIN w=(\d+) h=(\d+) fmt=(\S+) bytes=(\d+) crc=0x([0-9a-fA-F]+) bin=1###\n")
fd = os.open(port, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
a = termios.tcgetattr(fd)
a[0] = 0; a[1] = 0; a[3] = 0 # raw: no CR/NL translation, no canon/echo
a[4] = termios.B115200; a[5] = termios.B115200
termios.tcsetattr(fd, termios.TCSANOW, a)
def drain(sec=0.3):
t = time.time()
while time.time() - t < sec:
r, _, _ = select.select([fd], [], [], 0.05)
if r:
try: os.read(fd, 65536)
except BlockingIOError: pass
def capture_once():
drain(0.3) # clear any in-flight tail from a prior dump
buf = bytearray(); hdr = None; t0 = time.time(); last_trig = 0.0
while time.time() - t0 < PER_TRY:
now = time.time()
if hdr is None and now - last_trig >= 2.0:
try: os.write(fd, trig)
except OSError: pass
last_trig = now
r, _, _ = select.select([fd], [], [], 0.2)
if r:
try: chunk = os.read(fd, 65536)
except BlockingIOError: chunk = b""
if chunk: buf += chunk
if hdr is None:
m = begin_re.search(buf)
if m:
hdr = (int(m.group(1)), int(m.group(2)), int(m.group(4)), int(m.group(5), 16))
buf = bytearray(buf[m.end():]) # everything after header = raw payload
if hdr is not None and len(buf) >= hdr[2]:
return (hdr[0], hdr[1], hdr[2], hdr[3], bytes(buf[:hdr[2]]))
return None
result = None
for attempt in range(1, ATTEMPTS + 1):
res = capture_once()
if res is None:
sys.stderr.write(f"[try {attempt}] no frame\n"); continue
w, h, nbytes, crc_dev, raw = res
crc_host = zlib.crc32(raw) & 0xffffffff
ok = crc_host == crc_dev
sys.stderr.write(f"[try {attempt}] {w}x{h} {len(raw)}B "
f"crc dev=0x{crc_dev:08x} host=0x{crc_host:08x} {'OK' if ok else 'RETRY'}\n")
if ok:
result = (w, h, raw); break
os.close(fd)
if result is None:
sys.stderr.write("[fail] no clean frame — SCREENSHOT_ENABLE flashed? device up?\n")
sys.exit(2)
w, h, raw = result
from PIL import Image
img = Image.new("RGB", (w, h)); px = img.load(); i = 0
for y in range(h):
for x in range(w):
v = raw[i] | (raw[i+1] << 8); i += 2
r5 = (v >> 11) & 0x1f; g6 = (v >> 5) & 0x3f; b5 = v & 0x1f
px[x, y] = ((r5 << 3) | (r5 >> 2), (g6 << 2) | (g6 >> 4), (b5 << 3) | (b5 >> 2))
img.save(out)
sys.stderr.write(f"[ok] wrote {out} ({w}x{h})\n")
print(out)
-28
View File
@@ -1,28 +0,0 @@
.PHONY: setup run test lint typecheck clean
# any Python >= 3.11 works; system python3 on tower-of-joy is 3.8, hence explicit
PYTHON ?= python3.12
setup:
$(PYTHON) -m venv .venv
.venv/bin/pip install -e ".[dev]"
setup-speech:
.venv/bin/pip install -e ".[dev,speech]"
run:
.venv/bin/uvicorn desklock_gateway.main:app --host 0.0.0.0 --port 8600 --reload
test:
.venv/bin/pytest
lint:
.venv/bin/ruff check src tests
.venv/bin/ruff format --check src tests
typecheck:
.venv/bin/mypy src
clean:
rm -rf .venv .pytest_cache .ruff_cache .mypy_cache
find . -type d -name __pycache__ -exec rm -rf {} +
+48 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "desklock-gateway"
version = "0.1.0"
version = "0.2.2"
description = "Voice gateway bridging the DeskLock device to the Tatlock butler (STT/chat/TTS)"
requires-python = ">=3.11"
dependencies = [
@@ -33,6 +33,53 @@ where = ["src"]
line-length = 100
src = ["src"]
# Selected explicitly, because the default set is not a constant.
#
# With no `select` here, ruff lints with whatever its installed version
# defaults to — 413 rules under 0.16.3. `dev` pins only `ruff>=0.6`, and CI
# installs that extra fresh on every run, so the gate's scope was a function of
# when pip last resolved rather than of this code. Two findings appeared here
# the first time a converged environment ran the gate, in a file nobody had
# touched (T-47).
#
# That is the failure this repo keeps meeting from the other side: a check
# whose result depends on something other than the thing it checks. Pinning the
# ruff version would freeze the symptom; naming the rules fixes it, and makes
# a future ruff release a decision rather than a surprise.
#
# ASYNC is here on purpose — this is a websocket gateway, and it is the one
# family whose findings would be genuine bugs rather than style.
# BLE (blind except) is deliberately absent: main.py catches bare Exception
# when a device disappears mid-send, which is correct there and would need a
# noqa on every occurrence to say so.
[tool.ruff.lint]
select = ["E", "W", "F", "I", "UP", "B", "ASYNC", "SIM", "C4"]
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
[tool.mypy]
python_version = "3.11"
# The speech extra is deliberately absent from a default setup. `make setup`
# installs the gateway without it; `make setup-speech` adds faster-whisper and
# piper-tts, which pull several GB of ML wheels for a backend the deployment
# does not use — settings.tts_backend defaults to "speaches", a network call to
# the shared service on 8601. Both imports are lazy, inside the functions that
# need them, so their absence is a runtime fact rather than a defect.
#
# numpy is here for the same reason: nothing depends on it directly, it arrives
# with faster-whisper.
#
# Without these overrides, `make typecheck` fails on a machine that followed the
# documented setup — the pre-push gate turning red for doing the right thing,
# which is how a gate stops being read (T-56). The right assertion is "these
# modules may be absent", not "install several GB so the type checker is happy".
[[tool.mypy.overrides]]
module = [
"piper", "piper.*",
"faster_whisper", "faster_whisper.*",
"numpy", "numpy.*",
]
ignore_missing_imports = true
+95
View File
@@ -0,0 +1,95 @@
"""Voice command service.
Intercepts simple device commands from the STT transcript so they bypass the LLM
(no 10-25s Tatlock round-trip). `match()` returns a gateway->device ``command``
message dict, or ``None`` if the transcript is not a recognized command and should
go to Tatlock.
Matching is deliberately precise, not clever: a false positive hijacks a real
request, so we only fire on explicit volume/mute wording. Extend ``match()`` with
new rules as the command vocabulary grows.
"""
import re
VOL_MAX = 11
_WORD_NUMBERS = {
"zero": 0,
"one": 1,
"two": 2,
"three": 3,
"four": 4,
"five": 5,
"six": 6,
"seven": 7,
"eight": 8,
"nine": 9,
"ten": 10,
"eleven": 11,
}
Command = dict[str, str | int]
def _norm(text: str) -> str:
"""Lower-case, drop punctuation, collapse whitespace."""
return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", text.lower())).strip()
def _has(t: str, *words: str) -> bool:
return any(re.search(rf"\b{w}\b", t) for w in words)
def _to_int(tok: str) -> int | None:
if tok.isdigit():
return int(tok)
return _WORD_NUMBERS.get(tok)
def _cmd(action: str, level: int | None = None) -> Command:
msg: Command = {"type": "command", "action": action}
if level is not None:
msg["level"] = max(0, min(VOL_MAX, level))
return msg
def match(transcript: str) -> Command | None:
"""Return a ``command`` message if the transcript is a device command, else None."""
t = _norm(transcript)
if not t:
return None
# mute / unmute — check unmute first ("unmute" contains "mute")
if _has(t, "unmute"):
return _cmd("unmute")
if _has(t, "mute"):
return _cmd("mute")
# explicit "set volume to N" (digit or number word), requires the word "volume"
m = re.search(r"\bvolume\s+(?:to\s+|at\s+|is\s+)?(\w+)\b", t)
if m:
val = _to_int(m.group(1))
if val is not None:
return _cmd("volume_set", level=val)
# max / the meme — "this one goes to eleven", "crank it", "max volume"
if (
re.search(r"\bgoes to eleven\b", t)
or _has(t, "crank")
or (_has(t, "max", "maximum", "full") and _has(t, "volume", "sound"))
):
return _cmd("volume_set", level=VOL_MAX)
# relative up / down — need explicit volume/sound context or an unambiguous verb
vol_ctx = _has(t, "volume", "sound")
if _has(t, "louder") or (vol_ctx and _has(t, "up")) or re.search(r"\bturn it up\b", t):
return _cmd("volume_up")
if (
_has(t, "quieter", "softer")
or (vol_ctx and _has(t, "down"))
or re.search(r"\bturn it down\b", t)
):
return _cmd("volume_down")
return None
+5 -1
View File
@@ -4,7 +4,7 @@ from pydantic_settings import BaseSettings
class Settings(BaseSettings):
"""Gateway configuration, overridable via DESKLOCK_* environment variables."""
tatlock_base_url: str = "http://tatlock.schweitz.internal:8000"
tatlock_base_url: str = "http://tatlock:8000"
tatlock_model: str = "Tatlock"
# PCM rate of the device WebSocket contract (docs/architecture.md)
@@ -19,6 +19,10 @@ class Settings(BaseSettings):
tts_model: str = "speaches-ai/Kokoro-82M-v1.0-ONNX"
tts_voice: str = "bm_george"
# Spoken immediately after a query is accepted, to fill the long Tatlock wait.
# Avoid "for you" — Kokoro inserts an unnatural mid-phrase pause before it.
filler_text: str = "Let me check on that, sir."
# embedded fallback only (requires the [speech] extra)
embedded_stt_model: str = "small"
embedded_stt_device: str = "cuda"
+36 -3
View File
@@ -7,11 +7,11 @@ Protocol (see docs/architecture.md — keep in sync):
import asyncio
import logging
from datetime import datetime, timezone
from datetime import UTC, datetime
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from . import stt, tts
from . import commands, stt, tts
from .config import settings
from .tatlock import TatlockClient
@@ -24,9 +24,24 @@ PCM_CHUNK_BYTES = 4096
# device registry: every endpoint that ever connected, keyed by client IP
DEVICES: dict[str, dict] = {}
# Butler filler audio (the "let me check…" line) is synthesized once, then replayed.
_filler_pcm: bytes | None = None
_filler_tried = False
async def _ensure_filler() -> bytes | None:
global _filler_pcm, _filler_tried
if not _filler_tried:
_filler_tried = True
try:
_filler_pcm = await asyncio.to_thread(tts.synthesize, settings.filler_text)
except Exception:
logger.exception("filler synth failed; continuing without it")
return _filler_pcm
def _now() -> str:
return datetime.now(timezone.utc).astimezone().isoformat(timespec="seconds")
return datetime.now(UTC).astimezone().isoformat(timespec="seconds")
@app.get("/healthz")
@@ -104,6 +119,24 @@ async def _handle_utterance(ws: WebSocket, tatlock: TatlockClient, pcm: bytes) -
await ws.send_json({"type": "state", "value": "idle"})
return
# Simple device commands (volume/mute) bypass the LLM entirely.
command = commands.match(transcript)
if command is not None:
await ws.send_json(command)
await ws.send_json({"type": "state", "value": "idle"})
return
# Acknowledge immediately with a canned line so the long Tatlock wait isn't dead
# air, then re-assert "thinking" to keep the spinner up while it works.
filler = await _ensure_filler()
if filler:
await ws.send_json({"type": "reply_text", "text": settings.filler_text})
await ws.send_json({"type": "audio_start", "sample_rate": settings.sample_rate})
for offset in range(0, len(filler), PCM_CHUNK_BYTES):
await ws.send_bytes(filler[offset : offset + PCM_CHUNK_BYTES])
await ws.send_json({"type": "audio_end"})
await ws.send_json({"type": "state", "value": "thinking"})
reply = await tatlock.ask(transcript)
await ws.send_json({"type": "reply_text", "text": reply})
+61
View File
@@ -0,0 +1,61 @@
from desklock_gateway import commands
def test_mute_and_unmute() -> None:
assert commands.match("mute") == {"type": "command", "action": "mute"}
assert commands.match("mute the volume") == {"type": "command", "action": "mute"}
# "unmute" contains "mute" — must resolve to unmute
assert commands.match("unmute") == {"type": "command", "action": "unmute"}
def test_relative_up_down() -> None:
assert commands.match("volume up") == {"type": "command", "action": "volume_up"}
assert commands.match("louder please") == {"type": "command", "action": "volume_up"}
assert commands.match("turn it up") == {"type": "command", "action": "volume_up"}
assert commands.match("volume down") == {"type": "command", "action": "volume_down"}
assert commands.match("a bit quieter") == {"type": "command", "action": "volume_down"}
assert commands.match("turn it down") == {"type": "command", "action": "volume_down"}
def test_set_to_number_digit_and_word() -> None:
assert commands.match("set volume to 7") == {
"type": "command",
"action": "volume_set",
"level": 7,
}
assert commands.match("volume to seven") == {
"type": "command",
"action": "volume_set",
"level": 7,
}
assert commands.match("volume 3") == {"type": "command", "action": "volume_set", "level": 3}
# clamps into 0..11
assert commands.match("set volume to 50") == {
"type": "command",
"action": "volume_set",
"level": 11,
}
def test_goes_to_eleven() -> None:
assert commands.match("this one goes to eleven") == {
"type": "command",
"action": "volume_set",
"level": 11,
}
assert commands.match("set volume to eleven") == {
"type": "command",
"action": "volume_set",
"level": 11,
}
assert commands.match("crank it") == {"type": "command", "action": "volume_set", "level": 11}
assert commands.match("max volume") == {"type": "command", "action": "volume_set", "level": 11}
def test_non_commands_fall_through_to_llm() -> None:
# these must NOT be hijacked from Tatlock
assert commands.match("what's the weather tomorrow") is None
assert commands.match("set an alarm for a quarter to eleven") is None
assert commands.match("turn up the heating in the lounge") is None
assert commands.match("") is None
assert commands.match("tell me a joke") is None
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_