Compare commits

...
16 Commits
Author SHA1 Message Date
jpmschweitzerandClaude f8059771ce docs: fold AGENTS.md into CLAUDE.md and record the backend traps
One agent doc per repo, and it is CLAUDE.md. Unlike elsewhere, the
existing CLAUDE.md was not a stub -- it carried seven hard-won gotchas,
all of which survive intact. AGENTS.md supplied the deployment and
release material, minus its feature-branch mandate and its `git add -A`
snippet, and minus its pointer to portainer-core, which is deprecated and
must not be used as a source of infra facts. README.md and
docs/philosophy.md linked to the retired file, so those pointers move
with it.

The new material is two traps that both make the runtime look like the
opposite of what it is.

A cold import inside the container loads src/anthropic but not
src/ollama, and Ollama is the primary backend. The only import of
src/ollama is a function-body one at src/anthropic/model_selector.py:230,
while PREFER_CLOUD_BACKEND=false keeps the Claude path off. Read the
module list naively and the disabled fallback looks live while the hot
path looks dead. This matters because the Claude migration is abandoned
and its remnants are supposed to read as vestigial, not as unfinished
work; the doc carries the decision id so that reasoning is fetchable.

Second, get_household_registry() in a fresh `docker exec python` returns
zero members while the running app serves two models from it. It is
populated at startup, so importing the singleton from outside the app and
reading it as empty is a measurement error, not a finding.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:16:29 +02:00
jpmschweitzerandClaude 6b1c892bc6 chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:16:12 +02:00
jpmschweitzerandClaude 84467c121a chore: release v2.4.3
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m14s
Ships the Steward capability-extraction fix (a905363), which has been on
main since earlier today while production continued to route on prose:
the running v2.4.2 still matches capability domains as substrings across
the Steward's whole response, so "description" selects housekeeper and
"acknowledge" selects librarian and biographer.

Patch rather than minor: no new capability, and the JSON on the wire is
unchanged. What changes is which agents get invoked, and only in the
cases that were already wrong.

Also carries the routing benchmark, its fixtures, the shared GPU
residency guard and the findings document, none of which are
user-visible.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 18:30:28 +02:00
jpmschweitzerandClaude 2290320e9c docs: record the Steward routing and thinking findings
No change shipped. The Steward stays on gemma4:e2b with thinking left at
its default, and this records why so the experiment is not repeated on
the premise that started it.

That premise was wrong. The Steward appeared to pay ~300 tokens per turn
for reasoning that was generated and discarded, since no `thinking` field
comes back. The reasoning is emitted inline in the response instead, and
it is what produces a correct DELEGATE line — suppressing it costs 12.5
points of routing accuracy, entirely on multi-capability queries where
the model stops decomposing and names one capability.

e4b is disqualified by memory rather than quality: Ollama predicts
10.6 GiB for it against ~7.9 GiB available, so it evicts every
co-resident before loading, including nomic-embed-text. Lowering context
length does not rescue it — an 8x reduction moved the prediction only
1.1 GiB — and per-request num_ctx reloads the shared runner, dropping the
keep_alive pin and evicting nomic.

Also records that the two axes are independent: model choice governs
VRAM and co-residency, think setting governs tokens and latency and
costs nothing in VRAM.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 18:01:11 +02:00
jpmschweitzerandClaude bf13f9f0de refactor(bench): share the GPU residency guard, and guard tool calling too
Extracts the residency snapshot/restore into scripts/ollama_residency.py
so the two benchmarks cannot drift, and applies it to
benchmark_tool_calling.py, which had no protection at all.

That script was the more dangerous of the two. It rewrites
OLLAMA_DEFAULT_MODEL in .env and lets uvicorn reload onto it, restoring
the original only after the loop — so any crash or interrupt left the
*running server* pointed at the benchmark model. Its DEFAULT_MODELS
begins with mistral-nemo-large, the 9.2G model implicated in the
2026-08-07 VRAM outage. Both the .env restore and the residency restore
now run from `finally`.

SIGTERM is handled explicitly in the shared module. Python runs `finally`
for SIGINT, which arrives as KeyboardInterrupt, but the default SIGTERM
action terminates outright, so `timeout` or a plain `kill` skipped the
guard entirely.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 17:03:53 +02:00
jpmschweitzerandClaude 4f42bc047a test(bench): restore GPU residency after a benchmark run
Benchmarking swaps models on the GPU production is serving from. Ollama
evicts to make room, so the first run unpinned gemma4:e2b and left
gemma4:e4b resident: the next voice turn would have paid a ~36s cold
load, and only the monitoring noticing unexpected_models caught it.

Snapshot residency and pinning before the run, then evict whatever the
benchmark loaded and re-pin what was pinned before.

The restore is wired to SIGTERM as well as the normal exit path. Python
runs `finally` for SIGINT, which arrives as KeyboardInterrupt, but the
default SIGTERM action terminates outright — so a `timeout`, a systemd
stop or a plain `kill` skipped the guard entirely. That was not
theoretical: the first SIGTERM after adding this bypassed it, and the
pinned model survived only because the run had not reached the second
model yet.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 16:08:03 +02:00
jpmschweitzerandClaude 738ff10b93 test(bench): add labelled routing fixtures and router benchmark
Measures Steward routing against model and thinking settings by talking
to Ollama directly. No server, no agents, nothing executed — the
mutating fixtures only ever produce a routing decision — so the run is
cheap, repeatable and isolates routing from everything downstream. The
request body mirrors StewardAgent._call_ollama, so the `unset` cell is
exactly what production sends today.

Three thinking settings rather than two. `unset` is production, and it
is not neutral: gemma4 reasons by default and returns no `thinking`
field, so those tokens are generated and discarded.

Scoring is asymmetric on purpose. Each fixture carries `forbid` as well
as `expect`, because over-routing is the predicted failure when thinking
is off and it is the expensive one — a spurious librarian is a real web
call on a query that asked for arithmetic.

The adversarial group is regression coverage for the extraction fix in
a905363: those queries invite the vocabulary that used to select agents
by substring, so they now assert that routing follows what the Steward
decided rather than the words it used while explaining.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 15:22:01 +02:00
jpmschweitzerandClaude a90536314e fix(steward): route on the declared DELEGATE line, not on prose
The prompt tells the Steward to state its choice on a DELEGATE line and
to explain itself on REASON, COMPLEXITY and CONTEXT lines. Extraction
ignored that structure and substring-matched capability domains across
the entire response, so ordinary English in the explanation selected
agents: "description" contains the housekeeper domain "script",
"discover" contains "cover", "acknowledge" contains "knowledge" and
"know", "economy" contains the biographer domain "my".

Every one of those was a real delegation. A spurious librarian is a
multi-second web call on a query that asked for arithmetic.

It also made prose length a routing input, which would have quietly
corrupted the thinking benchmark this was found during: anything that
shortened the Steward's output reduces accidental substring hits and so
reads as improved routing.

Resolution is now layered, most explicit first — a DELEGATE line opening
with a capability name, then a capability named anywhere on that line,
then a domain on that line. With no DELEGATE line at all the response is
matched on capability names only, never domains, so the conversational
path still answers with no capabilities. Matching is whole-word
throughout.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 14:37:59 +02:00
jpmschweitzerandClaude 19e32cfbd6 docs(tests): correct e2e prerequisites in module docstring
Missed in the previous sweep: this docstring still named wakeup.sh, which
the Makefile replaced, and mistral-nemo, which gemma4:e2b replaced.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 15:10:09 +02:00
jpmschweitzerandClaude 99569e786e docs: correct stale tooling and model references
Three migrations left their documentation behind:

wakeup.sh was replaced by the Makefile during the project structure
consolidation, but AGENTS.md and the e2e README still tell you to run it.
The log path moved to build/logs/server.log at the same time.

The local model moved to gemma4:e2b, but the e2e prerequisites and the
benchmark recommendation still name mistral-nemo.

The benchmark figures in CLAUDE.md predate the current model. Measured
2026-08-07: ~95 tok/s, full flow ~10-13s for simple turns, cold model load
~36s rather than ~8s. A turn costs three sequential Ollama calls and ~710
generated tokens regardless of how trivial the question is.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 15:07:10 +02:00
jpmschweitzerandClaude Fable 5 2cf3252a19 docs(claude-integration): registry is git.schweitz.net not git.schweitz.internal
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 17:11:18 +02:00
jpmschweitzerandClaude Fable 5 cdd5a55613 chore: release v2.4.2
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m3s
Fixes the v2.4.1 crash-loop: fresh image builds resolved
opentelemetry-api 1.44.0, which removed the private _events module
that pydantic-ai 1.27 imports at startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 13:10:57 +02:00
jpmschweitzerandClaude Fable 5 287d66fff7 chore: release v2.4.1
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m44s
Move the container-name network-defaults change from [Unreleased] into
the 2.4.1 section and bump pyproject.toml. Patch release: the change
corrects service-host defaults (SEARXNG_HOST, LIBRARY_DESK_HOST,
CORE_API_HOST) for the docker-dataplane deployment, including the
wrong CORE_API_HOST port.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:32:10 +02:00
jpmschweitzerandClaude Fable 5 65debb6e44 fix(config): default service hosts to docker container names
The homelab is retiring *.schweitz.internal and will rebind host
ports to loopback; container-to-container traffic must use container
names on docker-dataplane.

- SEARXNG_HOST: http://localhost:8087 -> http://searxng:8080
  (SearXNG's internal port is 8080; 8087 was the host-published port)
- LIBRARY_DESK_HOST: http://localhost:8089 -> http://library-desk:8089
- CORE_API_HOST: http://localhost:8090 -> http://core-api:8083
  (8090 is the Scheduler's host port; Core-API serves 8083 internally,
  confirmed by the housekeeper client and test suite hitting :8083)
- scripts/test_housekeeper.sh: reach Core-API via localhost:8083
  instead of the LAN IP, which will refuse after loopback rebinding

Local development against host-published ports keeps working via .env
overrides (.env.example unchanged; localhost stays valid on the host).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:21:40 +02:00
jpmschweitzerandClaude Fable 5 0d7514b90e chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:10:04 +02:00
jpmschweitzerandClaude Fable 5 99683357d2 docs: replace CPU-era latency figures with measured GPU numbers
The ~35s steward / ~2 min flow figures dated from the driver-mismatch era
and were being inherited by downstream consumers (desklock architecture
doc) as planning baselines. Current measured: steward ~6s warm, full flow
11-25s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 17:17:13 +02:00
31 changed files with 1877 additions and 173 deletions
+68
View File
@@ -0,0 +1,68 @@
{
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(make test:*)",
"Bash(make test-unit:*)",
"Bash(make test-contracts:*)",
"Bash(make lint:*)",
"Bash(make typecheck:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(ruff check:*)",
"Bash(mypy:*)",
"Bash(docker logs tatlock:*)",
"Bash(curl -s http://localhost:8000/*)",
"Bash(curl -s http://localhost:8777/*)"
],
"deny": [
"Bash(sudo *)",
"Bash(su *)",
"Bash(rm -rf /*)",
"Bash(rm -rf ~*)",
"Bash(rm -rf $HOME*)",
"Bash(mkfs*)",
"Bash(dd if=*)",
"Bash(chmod 777 *)",
"Bash(chmod -R 777 *)",
"Bash(find * -exec*)",
"Bash(find * -delete*)",
"Bash(git add -A*)",
"Bash(git * add -A*)",
"Bash(git add --all*)",
"Bash(git * add --all*)",
"Bash(git add .)",
"Bash(git * add .)",
"Bash(git push --force*)",
"Bash(git * push --force*)",
"Bash(git push -f*)",
"Bash(git * push -f*)",
"Bash(git reset --hard*)",
"Bash(git * reset --hard*)",
"Bash(git clean -fd*)",
"Bash(git * clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git * clean -fdx*)",
"Bash(git branch -D *)",
"Bash(git * branch -D *)",
"Bash(git checkout -- *)",
"Bash(git * checkout -- *)",
"Bash(git restore .*)",
"Bash(git * restore .*)",
"Bash(git commit --no-verify*)",
"Bash(git * commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git * merge --no-ff*)",
"Bash(redis-cli * FLUSHALL*)",
"Bash(redis-cli * FLUSHDB*)",
"Bash(ollama rm *)"
]
}
}
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+3 -3
View File
@@ -25,7 +25,7 @@ jobs:
- name: Login to Gitea Registry
uses: docker/login-action@v3
with:
registry: git.schweitz.internal
registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -37,8 +37,8 @@ jobs:
provenance: false
sbom: false
tags: |
git.schweitz.internal/jpmschweitzer/tatlock:latest
git.schweitz.internal/jpmschweitzer/tatlock:${{ github.ref_name }}
git.schweitz.net/jpmschweitzer/tatlock:latest
git.schweitz.net/jpmschweitzer/tatlock:${{ github.ref_name }}
- name: Trigger Watchtower update
if: success()
+5
View File
@@ -103,3 +103,8 @@ ollama_data/
ehthumbs.db
Thumbs.db
Desktop.ini
# Claude Code user-specific settings
.claude/settings.local.json
.pql/*
!.pql/changelog/
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
-105
View File
@@ -1,105 +0,0 @@
# LLM Agent Instructions
This document contains instructions and documentation references for AI assistants working with this codebase.
> **📖 Important**: Before working on this project, read [docs/philosophy.md](docs/philosophy.md) to understand the system vision, architectural patterns, and design goals. All development should work towards realizing those patterns.
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🧪 Local Development Setup
* **Always test locally first** before committing and deploying. The build-deploy loop is slow.
* **Start the local server** with `./wakeup.sh` - logs are written to `logs/server.log` for easy tailing
* **Auto-reload**: The wakeup script runs uvicorn in reload mode - code changes are picked up automatically without restart (except for requirements.txt changes)
* **Test REST endpoints** against `http://localhost:8777` using curl or similar tools
* **Only deploy** when a phase or feature is complete and tested locally
* **Environment**: Copy `.env.example` to `.env` and configure for your local setup (Ollama, Redis, Qdrant hosts)
* **Running tests**: Always use the venv explicitly to avoid environment mismatches:
```bash
.venv/bin/python -m pytest tests/ # All tests
.venv/bin/python -m pytest tests/core/ -v # Core tests only
```
### 🌐 Internal Service Access
* **git.schweitz.net**: Access via `http://localhost:3002` (direct Gitea) to bypass Authentik SSO
* Example: `curl http://localhost:3002/jpmschweitzer/library-desk/raw/branch/main/README.md`
* Public repos are readable without authentication
* Related repos: `library-desk`, `scheduler`, `core-api`, `portainer-core`
### 🐳 Deployment & Infrastructure
* **Full stack documentation**: Available in the `portainer-core` repo
* Access: `curl http://localhost:3002/jpmschweitzer/portainer-core/raw/branch/main/CONTAINERS.md`
* Contains: All service ports, URLs, Redis DB allocations, external domains
* **Tatlock deployment**:
* LAN: `http://192.168.86.149:8000`
* External: `tatlock.schweitz.net` (behind Authentik SSO)
* Redis DBs: 1 (memory), 6 (benchmarks)
* **Health check**: `curl http://192.168.86.149:8000/health`
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired**
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8000/health`
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
src/
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point
+24
View File
@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.4.3] - 2026-08-08
### Fixed
- Steward routing no longer triggers on words inside its own explanation. Capability
extraction reads the declared `DELEGATE:` line instead of substring-matching
capability domains across the whole response, where ordinary English routed
requests — "description" contains the housekeeper domain "script", "acknowledge"
contains "knowledge" and "know". A spurious capability meant a real agent call,
including web searches, on queries that needed none.
## [2.4.2] - 2026-07-19
### Fixed
- Container crash-loop on fresh builds: cap `opentelemetry-api` below 1.44,
which removed the private `_events` module that pydantic-ai 1.27 imports
## [2.4.1] - 2026-07-19
### Changed
- **Container-name network defaults** - `SEARXNG_HOST`, `LIBRARY_DESK_HOST`, and `CORE_API_HOST` now default to docker container names on the docker-dataplane network (`http://searxng:8080`, `http://library-desk:8089`, `http://core-api:8083`) instead of host `localhost` ports, ahead of the loopback port rebinding; this also fixes `CORE_API_HOST` pointing at port 8090 (the Scheduler's host port) rather than Core-API's 8083. `scripts/test_housekeeper.sh` now reaches Core-API via `localhost:8083` instead of the LAN IP. Local development against host-published ports still works via `.env` overrides
## [2.4.0] - 2026-07-14
### Removed
+182 -20
View File
@@ -1,34 +1,196 @@
# CLAUDE.md
# CLAUDE.md — tatlock
Claude Code-specific notes for this project. For general development instructions, architecture, coding standards, and deployment — see [AGENTS.md](AGENTS.md).
Privacy-first homelab butler. An OpenAI-compatible orchestration API over local models, with
household staff agents built on PydanticAI. Python 3.12 / FastAPI, `version = "2.4.3"`.
Container `tatlock` on `docker-dataplane`, port **8000**. Redis DB **1** (memory), Qdrant for
vectors.
## Setup & Commands
## Ports
| | Port | How |
|---|---|---|
| Local dev | **8777** | `make run` — uvicorn reload, logs to `build/logs/server.log` |
| Production | **8000** | container; `http://192.168.86.149:8000/health`, external `tatlock.schweitz.net` behind Authentik |
Test endpoints against `localhost:8777` while developing. `localhost:8000` is the *container*.
## Live contract
`http://localhost:8000/openapi.json` — **5 paths**, `title: OpenAI-Compatible API`, `version:
2.4.3` (verified 2026-08-09): `/`, `/health`, `/v1/models`, `/v1/chat/completions`,
`/v1/responses`. `/v1/responses` is primary; `/v1/chat/completions` exists for Open WebUI.
**The spec is the public surface, not the system.** The household capability registry is internal
and appears nowhere in those 5 paths. Absence from the spec means "not exposed", not "does not
exist".
## Two traps that make the runtime look like the opposite of what it is
**1. `src/anthropic` loads at startup; `src/ollama` does not — and Ollama is the primary
backend.** A cold `import src.main` inside the container shows `agents, anthropic, chat, core,
main, models, responses` — no `ollama`. The only import of it is a *function-body* one at
`src/anthropic/model_selector.py:230`. Meanwhile `PREFER_CLOUD_BACKEND=false`, so every request
actually goes to Ollama and the Claude path is off (see **D-11**). Reading the module list
naively gives you exactly the wrong answer: the package that looks live is the disabled fallback,
and the one that looks dead is the hot path. Do not conclude anything about backends from
`sys.modules`; read the config.
**2. In-process singletons are empty outside the app.** `get_household_registry()`
(`src/core/household_registry.py:334`) in a fresh `docker exec python` returns **0 members**,
while the running app serves 2 models from it — it is populated at startup. Import the
module-level definitions or ask the endpoint; never import a singleton and assume it is
populated.
## Stack decisions that bind this repo
Recorded in the workspace vault, not here. Read before assuming anything about the LLM backend:
```bash
make setup # Create venv and install all dependencies
make test # Unit tests (no external services)
make test-integration # Integration tests (needs Claude/Ollama)
make test-contracts # Wire-level contract tests against live service boundaries
make run # Start dev server on port 8777
make lint # Ruff linter + formatter check
make typecheck # Mypy
make clean # Remove caches and build artifacts
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions read D-11
```
Dependencies are in `pyproject.toml` (`[project.dependencies]` and `[project.optional-dependencies.dev]`).
**D-11 — the Claude migration is abandoned. Tatlock stays on Ollama.** Do not resume it and do
not treat its remnants as unfinished work. What you will find, and why none of it is a TODO:
`ANTHROPIC_MODEL` is set on the container (`claude-sonnet-4-20250514`) and never used because
`PREFER_CLOUD_BACKEND=false`; `ANTHROPIC_API_KEY` is a variable reference whose literal was
revoked 2026-08-09; `docs/claude-integration.md` documents a capability that exists but is
switched off. The cost is deliberate: reasoning stays at `gemma4:e2b` scale because VRAM is
shared with Speaches.
## Critical Gotchas
`REDIS_BENCHMARK_DB=6` is allocated on the container but the benchmarking module was never
implemented — see the gotcha below. Vestigial, like the Anthropic settings.
**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app` fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`. Without this, the Ollama/Claude health checks never run: `_ollama_available` stays `None` (treated as available, so requests go to Ollama) and `_claude_available` stays `None` (treated as unavailable, so the Claude fallback never engages).
## Critical gotchas
**AsyncIO scope mismatch.** `asyncio_default_fixture_loop_scope = function` is set in `pyproject.toml`. Session-scoped async fixtures cause `ScopeMismatch` errors. The fix is to use a sync fixture with `asyncio.run()` for session-scoped initialization.
**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app`
fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`.
Without it the Ollama/Claude health checks never run: `_ollama_available` stays `None` (treated
as available, so requests go to Ollama) and `_claude_available` stays `None` (treated as
unavailable, so the Claude fallback never engages).
**The butler persona prompt suppresses local-model tool calling.** With `TATLOCK_SYSTEM_PROMPT` attached, gemma4 reasons about calling the calculator, then answers from memory with wrong arithmetic (a different wrong product each run). `orchestrate_tool_calls()` therefore uses the terse `TATLOCK_ORCHESTRATION_PROMPT`; the persona is applied in `synthesize_from_results()`. Do not reattach the persona prompt to a tool-phase agent. `tool_choice: "required"` via extra_body does NOT force Ollama to call tools — it is advisory at best.
**AsyncIO scope mismatch.** `asyncio_default_fixture_loop_scope = function` is set in
`pyproject.toml`. Session-scoped async fixtures raise `ScopeMismatch`. Use a sync fixture with
`asyncio.run()` for session-scoped initialization.
**Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return a 400. Use `get_sampling_settings()` from the model selector instead of passing `ModelSettings(temperature=...)` directly to agents that can run on the Claude fallback. The contract test suite pins this (`make test-contracts`).
**The butler persona prompt suppresses local-model tool calling.** With `TATLOCK_SYSTEM_PROMPT`
attached, gemma4 reasons about calling the calculator, then answers from memory with wrong
arithmetic — a different wrong product each run. `orchestrate_tool_calls()` therefore uses the
terse `TATLOCK_ORCHESTRATION_PROMPT`; the persona is applied in `synthesize_from_results()`. Do
not reattach the persona prompt to a tool-phase agent. `tool_choice: "required"` via `extra_body`
does **not** force Ollama to call tools — advisory at best.
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). The full local Steward → orchestrate → synthesize flow takes ~2 minutes on gemma4. Steward analysis alone needs ~35s warm — `STEWARD_TIMEOUT` defaults to 60s.
**Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return 400. Use
`get_sampling_settings()` from the model selector rather than passing `ModelSettings(temperature=…)`
to agents that can run on the Claude fallback. `make test-contracts` pins this.
**`get_benchmark_store` does not exist.** The benchmarking module (`src/core/benchmarks.py`) was never implemented. `scripts/benchmark_analysis.py` also references it and is broken. Do not add mocks for it in tests.
**Integration test timeouts** are 120s to match `OLLAMA_TIMEOUT` (300s for the pure-Ollama
fallback test, which Claude cannot rescue). GPU-resident numbers measured 2026-08-07 with
gemma4:e2b at ~95 tok/s: full Steward → orchestrate → synthesize ~10–13s for simple turns;
librarian-routed ~20–25s (not re-measured). **A single turn costs 3 sequential Ollama calls and
~710 generated tokens even for "what is 61 plus 12?"** — mostly the model's own reasoning, paid
three times. Cold model load is ~36s, avoided while pinned with `keep_alive: -1`; the
`OLLAMA_KEEP_ALIVE=2h` default reintroduces it. Older "~35s steward / ~2 min flow" and "11–25s"
figures are superseded — do not plan against them. `STEWARD_TIMEOUT` defaults to 60s.
**Steward tests need household registry.** Use `register_household_members()` (sync) in fixtures, not `initialize_application()` (async). The steward extracts capabilities from the registry.
**`get_benchmark_store` does not exist.** `src/core/benchmarks.py` was never implemented, and
`scripts/benchmark_analysis.py` references it and is broken. Do not add mocks for it in tests.
**Steward tests need the household registry.** Use `register_household_members()` (sync) in
fixtures, not `initialize_application()` (async). The steward extracts capabilities from the
registry.
## Commands
```bash
make setup # venv + all dependencies
make run # dev server on 8777, reload, logs to build/logs/server.log
make test # unit tests, no external services
make test-integration # needs Ollama (and Claude, if enabled)
make test-contracts # wire-level contract tests against live service boundaries
make lint # ruff linter + formatter check
make typecheck # mypy
make clean # remove caches and build artifacts
```
Always run pytest through the venv explicitly, to avoid environment mismatch:
```bash
.venv/bin/python -m pytest tests/
.venv/bin/python -m pytest tests/core/ -v
```
Dependencies live in `pyproject.toml` (`[project.dependencies]`, `[project.optional-dependencies.dev]`).
Copy `.env.example` to `.env` and configure Ollama, Redis and Qdrant hosts.
**Contract tests before code review.** When the question is "do these two services still agree?",
`make test-contracts` answers it by observing the live boundary; reading both codebases only tells
you what should happen. Semantics: unreachable → skip, reachable-but-wrong-shape → fail.
## Architecture
Domain-first under `src/`: `agents/` (steward, librarian, biographer, housekeeper, tatlock_core),
`core/`, `chat/`, `responses/`, `models/`, `ollama/`, `anthropic/`. Two tiers — the Steward routes,
Tatlock coordinates. Group new work by domain, not by file type.
## Internal service access
`http://localhost:3002` reaches Gitea directly, bypassing Authentik SSO — verified returning
`{"version":"1.27.1"}`. Useful for reading a sibling repo's raw files:
```bash
curl http://localhost:3002/jpmschweitzer/library-desk/raw/branch/main/README.md
```
The old AGENTS.md pointed at **`portainer-core`** for full-stack documentation. That repo is
**deprecated** and must not be used as a source of infra facts; it was merged into
`system-admin-toj/containers/`, where `CONTAINERS.md` is the live inventory.
## Work tracking
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets and
internal decisions live here in `.pql/` and `governance/`, and travel with a clone, because
`.pql/changelog/` is committed and replayed by the git hooks (D-15). The databases are gitignored
and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as `/home/jpmschweitzer/.local/bin/pql`.
From inside this repo no `--vault` is needed; pql anchors at the nearest `.git/` ancestor.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack decisions that constrain this service need the flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain tatlock-api
```
The workspace domain is `tatlock-api`, not `tatlock` — pql rejects a domain stem that prefixes
another, and `tatlock` prefixes `tatlock-ui`. A `tatlock-api -> tatlock` symlink at the workspace
root makes the directory answer to both (D-15).
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that is
fast-forwarded and deleted. This repo's AGENTS.md mandated a feature branch for every change;
that rule was retired workspace-wide on 2026-08-08 and does not apply.
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Stage explicitly. Never `git add -A`** — denied by policy, and it sweeps in whatever else is
dirty, including secrets.
- Update `CHANGELOG.md` for every user-facing change, under `[Unreleased]`.
## Releasing
Test locally first — the build-deploy loop is slow. Deploy only when a feature is complete.
1. Ask whether a deploy is wanted; it is not automatic.
2. Bump `version` in `pyproject.toml` (patch for fixes, minor for features).
3. Move `[Unreleased]` entries into a dated section in `CHANGELOG.md`.
4. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
5. Gitea CI builds and pushes on the tag; Watchtower deploys.
6. Verify: `curl http://192.168.86.149:8000/health`.
+2 -2
View File
@@ -406,7 +406,7 @@ tatlock/
## Development
For LLM agent development guidelines and architectural decisions, see [AGENTS.md](AGENTS.md).
For LLM agent development guidelines and architectural decisions, see [CLAUDE.md](CLAUDE.md).
## Contributing
@@ -420,7 +420,7 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md
- **System Philosophy**: [docs/philosophy.md](docs/philosophy.md) - Vision, goals, and architectural patterns
- **Development Roadmap**: [docs/roadmap.md](docs/roadmap.md) - Open work and planned phases
- **Developer Guidelines**: [AGENTS.md](AGENTS.md) - LLM agent development patterns
- **Developer Guidelines**: [CLAUDE.md](CLAUDE.md) - LLM agent development patterns
- **Version History**: [CHANGELOG.md](CHANGELOG.md) - Changes and releases
### External References
+1 -1
View File
@@ -28,7 +28,7 @@ src/mcp/
```yaml
tatlock-mcp:
image: git.schweitz.internal/jpmschweitzer/tatlock:latest
image: git.schweitz.net/jpmschweitzer/tatlock:latest
command: ["python", "-m", "src.mcp.server"]
ports:
- "8002:8002"
+2 -2
View File
@@ -10,7 +10,7 @@ This document establishes the foundational philosophy and architectural patterns
- When new architectural insights require rethinking core principles
**When NOT to modify this document**:
- During implementation of these patterns (use README.md, AGENTS.md, or code comments for technical details)
- During implementation of these patterns (use README.md, CLAUDE.md, or code comments for technical details)
- For adding new household members or capabilities within the existing pattern
- For tactical decisions about specific technologies or tools
@@ -270,7 +270,7 @@ The user never directly interacts with the Steward or individual expert agents
**Related Documents**:
- **README.md**: User-facing documentation and usage guide
- **AGENTS.md**: LLM agent development guidelines and technical patterns
- **CLAUDE.md**: LLM agent development guidelines and technical patterns
- **CHANGELOG.md**: Version history and implemented features
---
+110
View File
@@ -0,0 +1,110 @@
# Steward Routing & Thinking — Findings
**Outcome: no change shipped.** The Steward stays on `gemma4:e2b` with model
thinking left at its default (on). Every alternative was measured and every one
loses. This document exists so the experiment is not repeated on the same
premise.
Run 2026-08-08 with `scripts/benchmark_routing.py` and
`scripts/fixtures/routing_fixtures.py` (40 labelled queries, one repeat per
cell, temperature 0.3 as production sends).
---
## The premise was wrong
The experiment was designed around an observation that the Steward pays ~300
tokens per turn for reasoning that is generated and thrown away: it calls
`/api/generate`, gemma4 reasons by default, and **no `thinking` field comes back
in the response**. Disabling thinking therefore looked close to free.
It is not. The reasoning is not discarded — it is emitted inline in `response`,
and it is what produces a correct `DELEGATE:` line. Those tokens are the work,
not waste. Suppressing them costs 12.5 points of routing accuracy.
## Results
| config | exact | under | over | tokens | latency | resident | predicted | co-resident with nomic |
|---|---|---|---|---|---|---|---|---|
| **e2b, thinking** *(production)* | **97.5%** | 2.5% | 0% | 361 | 5179 ms | 1778 MB | 7.8 GiB | yes |
| e2b, `think: false` | 85.0% | 12.5% | 5.0% | 48 | 1435 ms | 1778 MB | 7.8 GiB | yes |
| e4b, thinking | 100% | 0% | 0% | 192 | 4726 ms | 3089 MB | 10.6 GiB | **no** |
| e4b, `think: false` | 97.5% | 2.5% | 0% | 52 | 2269 ms | 3089 MB | 10.6 GiB | **no** |
`think: true` was also measured and landed within one fixture of the default on
both models, so production's implicit thinking is the same thing as asking for
it explicitly. Format compliance was 100% in every cell — a `DELEGATE:` line is
always emitted.
With 40 fixtures and one repeat, each result is worth 2.5 points, so the
97.5-vs-100 gaps are single fixtures and inside the noise. The latency and token
medians (40 calls each) and the e2b `think: false` degradation (6 failures with a
consistent mechanism) are the parts worth trusting.
## Why each alternative loses
**`think: false` on e2b** — 85% exact, and the failures are not random. All three
multi-capability fixtures under-route, each missing a second capability. Without
reasoning the model names one capability and stops decomposing. It is not
degraded across the board; it specifically stops handling compound requests,
which is where a user would most notice the Butler quietly doing half the job.
**e4b, either setting** — disqualified by memory, not by quality. Ollama predicts
**10.6 GiB** for it at 16k context. Maximum available on this card is ~7.9 GiB
(10.4 free − 2.0 GPU overhead − 0.46 minimum), so e4b *always* exceeds the budget
and evicts every co-resident before loading. Observed directly: loading it threw
out both `gemma4:e2b` and `nomic-embed-text`. Losing nomic means Tatlock memory
and library-desk thrash on every embedding call. Note this is not caused by the
2 GiB reservation — without it, available would be ~9.7 GiB, still under 10.6.
**Lower `OLLAMA_CONTEXT_LENGTH`** — the obvious way to free headroom, and it does
not work. Dropping 16384 → 2048, an 8× reduction, moved the prediction only from
7.8 to 6.7 GiB. The prediction is dominated by weights and batch size, not KV
cache. It would also truncate the Librarian's retrieved passages and webber's
code context for a 14% saving that funds nothing.
**Per-request `num_ctx`** — worse. A single request with a different `num_ctx`
reloads the shared runner, which **drops the `keep_alive: -1` pin** (expiry fell
from year-2318 to a 2-hour default) and evicts nomic. Three services share this
Ollama, so mixed context sizes are a thrash generator, and it fails silently.
**`OLLAMA_NUM_PARALLEL > 1`** — never viable here. e2b already predicts 7.8 GiB
against ~7.9 available, so there is no room for a second slot at any context
length. It is also set to 1 deliberately, to avoid batch overflow panics.
## What the two axes actually control
They do not interact, which is the useful part:
- **Model choice** governs VRAM and co-residency. e2b 1778 MB, e4b 3089 MB.
- **Think setting** governs tokens, latency and routing quality — and costs
**nothing** in VRAM. Verified: e2b is resident at 1778 MB with `think` unset,
true and false alike, because the KV cache is allocated for the full context at
load time and `think` is a per-request generation parameter.
So the only real question is whether 313 tokens and 3.7 seconds are worth 12.5
points of compound-query routing. On a turn that is already three sequential
Ollama calls, they are.
## Prerequisite: the extraction fix
These numbers are only meaningful because `_extract_capabilities` was fixed first
(commit `a905363`). It previously substring-matched capability *domains* across
the Steward's entire response, so ordinary English in the `REASON:` line selected
agents — "description" contains the housekeeper domain "script", "acknowledge"
contains "knowledge" and "know".
That made **prose length a routing input**. Benchmarking against it would have
shown `think: false` improving routing purely because shorter output produces
fewer accidental substring hits — a thinking policy derived from a parsing
artefact. The `adversarial` fixture group is regression coverage for exactly this.
## If this is revisited
The constraint is the single 11 GB card, not the model. A second inference host
(*forge*) removes it entirely, and e4b's 100% routing becomes reachable without
evicting anything. Re-run then; on this card the answer is settled.
`scripts/benchmark_routing.py` takes `--models`, `--think` and `--repeats`, and
restores GPU residency on exit — including on SIGTERM, which the first version
did not.
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+4 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "tatlock"
version = "2.4.0"
version = "2.4.3"
description = "OpenAI-compatible API with Ollama backend"
requires-python = ">=3.12"
dependencies = [
@@ -13,6 +13,9 @@ dependencies = [
"pydantic>=2.11,<2.13",
"pydantic-settings>=2.12,<2.13",
"pydantic-ai-slim[openai,anthropic]>=1.27,<1.28",
# pydantic-ai 1.27 imports the private opentelemetry._events module,
# removed in opentelemetry-api 1.44 — cap until pydantic-ai is bumped
"opentelemetry-api>=1.30,<1.44",
"anthropic>=0.77,<1.0",
"httpx>=0.28,<0.29",
"sse-starlette>=3.0,<3.1",
+235
View File
@@ -0,0 +1,235 @@
"""
Benchmark Steward routing quality against model and thinking settings.
Talks to Ollama directly. No Tatlock server, no agents, no tools, nothing is
executed — the mutating fixtures ("turn on the lights", "update the wiki") only
ever produce a routing decision. That makes this cheap and repeatable, and it
isolates the question: does the Steward still pick the right capabilities when
the model reasons less?
The request body is byte-identical to StewardAgent._call_ollama, plus the
`think` flag under test, so a cell labelled `unset` is exactly what production
sends today.
Three thinking settings, because "on vs off" hides the interesting case:
unset what production sends now. gemma4 reasons by default, and the
response carries no `thinking` field, so those tokens are generated
and discarded.
true reasoning requested explicitly and returned in `thinking`.
false reasoning suppressed.
Scoring is deliberately asymmetric. A missing capability under-routes and the
Butler answers without a tool it needed; a spurious one over-routes, and that is
a real agent call — a stray librarian is a multi-second web search on a query
that asked for arithmetic. Over-routing is the predicted failure when thinking
is off, so `forbid` violations are reported separately rather than folded into
one accuracy number.
Usage:
.venv/bin/python scripts/benchmark_routing.py
.venv/bin/python scripts/benchmark_routing.py --models gemma4:e2b
.venv/bin/python scripts/benchmark_routing.py --think false --repeats 3
"""
from __future__ import annotations
import argparse
import json
import statistics
import sys
import time
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
import httpx
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT))
from scripts.fixtures.routing_fixtures import FIXTURES # noqa: E402
from scripts.ollama_residency import ( # noqa: E402
install_sigterm_handler,
residency_guard,
)
from src.agents.steward.agent import build_steward_prompt # noqa: E402
from src.agents.steward.service import _DELEGATE_LINE_RE, _extract_capabilities # noqa: E402
from src.core.startup import register_household_members # noqa: E402
OLLAMA_URL = "http://localhost:11434"
DEFAULT_MODELS = ["gemma4:e2b", "gemma4:e4b"]
DEFAULT_THINK = ["unset", "true", "false"]
RESULTS_DIR = PROJECT_ROOT / "logs"
def build_body(model: str, prompt: str, think: str) -> dict[str, Any]:
"""Mirror StewardAgent._call_ollama exactly, then add the flag under test."""
body: dict[str, Any] = {
"model": model,
"prompt": prompt,
"stream": False,
"options": {
"temperature": 0.3, # Lower = more consistent
"top_p": 0.9,
},
}
if think != "unset":
body["think"] = think == "true"
return body
def call(client: httpx.Client, body: dict[str, Any]) -> dict[str, Any] | None:
try:
response = client.post(f"{OLLAMA_URL}/api/generate", json=body)
response.raise_for_status()
return response.json()
except Exception as exc: # noqa: BLE001 - a failed cell must not abort the run
print(f" ! {exc}", file=sys.stderr)
return None
def score(fixture: dict, found: list[str]) -> dict[str, Any]:
expected = set(fixture["expect"])
forbidden = set(fixture["forbid"])
got = set(found)
missing = sorted(expected - got)
spurious = sorted(got & forbidden)
return {
"found": found,
"missing": missing,
"spurious": spurious,
# Exact only when everything expected arrived and nothing forbidden did.
"exact": not missing and not spurious,
"under_routed": bool(missing),
"over_routed": bool(spurious),
}
def run_cell(client: httpx.Client, model: str, think: str, repeats: int) -> list[dict[str, Any]]:
rows: list[dict[str, Any]] = []
for fixture in FIXTURES:
prompt = build_steward_prompt(fixture["query"], [])
body = build_body(model, prompt, think)
for rep in range(repeats):
started = time.perf_counter()
data = call(client, body)
elapsed_ms = (time.perf_counter() - started) * 1000
if data is None:
rows.append({
"id": fixture["id"], "group": fixture["group"], "rep": rep,
"error": True, "exact": False, "under_routed": False, "over_routed": False,
})
continue
text = data.get("response", "") or ""
found = _extract_capabilities(text)
rows.append({
"id": fixture["id"],
"group": fixture["group"],
"rep": rep,
"error": False,
"latency_ms": round(elapsed_ms, 1),
"eval_tokens": data.get("eval_count"),
"prompt_tokens": data.get("prompt_eval_count"),
# Did the model obey the documented output shape at all?
"has_delegate_line": bool(_DELEGATE_LINE_RE.search(text)),
# Whether reasoning came back, as opposed to being generated and dropped.
"thinking_returned": bool(data.get("thinking")),
"response_chars": len(text),
**score(fixture, found),
})
return rows
def summarise(rows: list[dict[str, Any]]) -> dict[str, Any]:
ok = [r for r in rows if not r["error"]]
if not ok:
return {"n": 0, "errors": len(rows)}
latencies = [r["latency_ms"] for r in ok]
tokens = [r["eval_tokens"] for r in ok if r["eval_tokens"] is not None]
return {
"n": len(ok),
"errors": len(rows) - len(ok),
"exact_pct": round(100 * sum(r["exact"] for r in ok) / len(ok), 1),
"under_routed_pct": round(100 * sum(r["under_routed"] for r in ok) / len(ok), 1),
"over_routed_pct": round(100 * sum(r["over_routed"] for r in ok) / len(ok), 1),
"format_ok_pct": round(100 * sum(r["has_delegate_line"] for r in ok) / len(ok), 1),
"thinking_returned_pct": round(100 * sum(r["thinking_returned"] for r in ok) / len(ok), 1),
"latency_ms_median": round(statistics.median(latencies), 1),
"latency_ms_mean": round(statistics.fmean(latencies), 1),
"eval_tokens_median": round(statistics.median(tokens), 1) if tokens else None,
"eval_tokens_total": sum(tokens) if tokens else None,
}
def main() -> int:
install_sigterm_handler()
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--models", default=",".join(DEFAULT_MODELS))
parser.add_argument("--think", default=",".join(DEFAULT_THINK),
help="comma-separated subset of unset,true,false")
parser.add_argument("--repeats", type=int, default=1)
parser.add_argument("--timeout", type=float, default=180.0)
args = parser.parse_args()
models = [m.strip() for m in args.models.split(",") if m.strip()]
think_modes = [t.strip() for t in args.think.split(",") if t.strip()]
# build_steward_prompt reads the registry, and the registry is populated at
# application startup. Without this the prompt lists no capabilities and every
# cell scores zero for reasons that have nothing to do with the model.
register_household_members()
print(f"{len(FIXTURES)} fixtures x {len(models)} models x {len(think_modes)} think "
f"x {args.repeats} repeats = {len(FIXTURES) * len(models) * len(think_modes) * args.repeats} calls\n")
cells: dict[str, Any] = {}
# The guard restores production's pinned models however this exits — a
# finished run, a failed cell, Ctrl-C or SIGTERM.
with residency_guard(models_used=models), httpx.Client(timeout=args.timeout) as client:
for model in models:
# Absorb the cold load (~36s) outside the measurements.
print(f"warming {model} ...", flush=True)
call(client, build_body(model, "hi", "false"))
for think in think_modes:
key = f"{model}|think={think}"
print(f" {key} ...", end=" ", flush=True)
started = time.perf_counter()
rows = run_cell(client, model, think, args.repeats)
summary = summarise(rows)
cells[key] = {"summary": summary, "rows": rows}
print(f"exact={summary.get('exact_pct')}% "
f"over={summary.get('over_routed_pct')}% "
f"median={summary.get('latency_ms_median')}ms "
f"({time.perf_counter() - started:.0f}s)")
RESULTS_DIR.mkdir(parents=True, exist_ok=True)
stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
out = RESULTS_DIR / f"routing-bench-{stamp}.json"
out.write_text(json.dumps({
"generated_at": datetime.now(UTC).isoformat(),
"fixtures": len(FIXTURES),
"repeats": args.repeats,
"cells": cells,
}, indent=2))
print(f"\n{'cell':28} {'exact':>7} {'under':>7} {'over':>7} {'fmt':>6} {'tok':>7} {'ms':>8}")
print("-" * 76)
for key, cell in cells.items():
s = cell["summary"]
print(f"{key:28} {s.get('exact_pct'):>6}% {s.get('under_routed_pct'):>6}% "
f"{s.get('over_routed_pct'):>6}% {s.get('format_ok_pct'):>5}% "
f"{str(s.get('eval_tokens_median')):>7} {s.get('latency_ms_median'):>8}")
print(f"\nwritten to {out}")
return 0
if __name__ == "__main__":
try:
sys.exit(main())
except KeyboardInterrupt:
# The residency guard has already run by the time this is caught;
# a traceback here would just bury its output.
print("\ninterrupted", file=sys.stderr)
sys.exit(130)
+1 -1
View File
@@ -268,7 +268,7 @@ async def run_benchmarks(iterations: int = 10, verbose: bool = False):
print(f" Max: {overall_max:.3f}s (target: ≤5.0s)")
print(f" Avg: {overall_avg:.3f}s (target: ≤1.67s)")
print(f"\n Recommendations:")
print(f" - Switch to a faster model (current: mistral-nemo)")
print(f" - Switch to a faster model (current: gemma4:e2b)")
print(f" - Reduce system prompt complexity")
print(f" - Limit tool calls (currently limited to 3)")
print(f" - Consider caching household registry responses")
+26 -9
View File
@@ -17,12 +17,16 @@ import asyncio
import json
import re
import statistics
import sys
import time
from dataclasses import dataclass, field
from pathlib import Path
import httpx
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from scripts.ollama_residency import install_sigterm_handler, residency_guard
# ---------------------------------------------------------------------------
# Configuration
# ---------------------------------------------------------------------------
@@ -525,18 +529,31 @@ async def main():
original_env = ENV_PATH.read_text()
all_stats = []
async with httpx.AsyncClient() as client:
for model in models:
stats = await benchmark_model(client, model, args.iterations)
all_stats.append(stats)
# Restore original .env
ENV_PATH.write_text(original_env)
print(f"\n .env restored to original")
# Both restores must survive a crash or an interrupt. The .env one especially:
# this script rewrites OLLAMA_DEFAULT_MODEL and lets uvicorn reload onto it,
# so bailing out mid-run used to leave the *running server* pointed at the
# benchmark model — and DEFAULT_MODELS starts at mistral-nemo-large, the 9.2G
# model implicated in the 2026-08-07 VRAM outage.
install_sigterm_handler()
try:
with residency_guard(models_used=models):
async with httpx.AsyncClient() as client:
for model in models:
stats = await benchmark_model(client, model, args.iterations)
all_stats.append(stats)
finally:
ENV_PATH.write_text(original_env)
print("\n .env restored to original")
print_comparison(all_stats)
save_results(all_stats, Path(args.output))
if __name__ == "__main__":
asyncio.run(main())
try:
asyncio.run(main())
except KeyboardInterrupt:
# .env and GPU residency are both restored by now; do not bury that
# output under a traceback.
print("\ninterrupted", file=sys.stderr)
raise SystemExit(130) from None
View File
+159
View File
@@ -0,0 +1,159 @@
"""
Labelled queries for the Steward routing benchmark.
Each fixture carries both `expect` and `forbid`:
expect capabilities that must appear. Missing one is under-routing — the
Butler answers without a tool it needed.
forbid capabilities that must not appear. Over-routing is not cosmetic: a
spurious librarian is a real multi-second web call, and a spurious
housekeeper can actuate hardware.
`forbid` matters more than `expect` here, because over-recommendation is the
predicted failure when model thinking is disabled and the Steward has less room
to discriminate.
The `adversarial` group deserves explanation. Until 2026-08-08 the extractor
substring-matched capability *domains* across the Steward's whole response, so
ordinary English in its REASON line selected agents: "description" contains the
housekeeper domain "script", "acknowledge" contains "knowledge" and "know",
"economy" contains the biographer domain "my". Those queries invite exactly that
vocabulary. They now serve as an end-to-end regression: routing must depend on
what the Steward *decided*, not on the words it happened to use while explaining.
Expectations follow the routing rules stated in the Steward prompt itself
(src/agents/steward/agent.py), not on what a capability could plausibly cover.
"""
CORE = "tatlock_core"
LIB = "librarian"
BIO = "biographer"
HOUSE = "housekeeper"
ALL = [CORE, LIB, BIO, HOUSE]
def _others(*keep: str) -> list[str]:
return [c for c in ALL if c not in keep]
FIXTURES: list[dict] = [
# --- arithmetic and computation -> tatlock_core --------------------------
{"id": "math_add", "group": "math", "query": "What is 61 plus 12?",
"expect": [CORE], "forbid": _others(CORE)},
{"id": "math_percent", "group": "math", "query": "What is 15% of 240?",
"expect": [CORE], "forbid": _others(CORE)},
{"id": "math_compound", "group": "math", "query": "If I save 200 a month for 3 years, how much is that?",
"expect": [CORE], "forbid": _others(CORE)},
{"id": "math_sqrt", "group": "math", "query": "What is the square root of 1764?",
"expect": [CORE], "forbid": _others(CORE)},
# --- date and time -> tatlock_core ---------------------------------------
{"id": "time_now", "group": "datetime", "query": "What time is it?",
"expect": [CORE], "forbid": _others(CORE)},
{"id": "time_date", "group": "datetime", "query": "What is today's date?",
"expect": [CORE], "forbid": _others(CORE)},
{"id": "time_delta", "group": "datetime", "query": "How many days until Christmas?",
"expect": [CORE], "forbid": _others(CORE)},
# --- personal memory -> biographer ---------------------------------------
{"id": "bio_location", "group": "biographer", "query": "Where do I live?",
"expect": [BIO], "forbid": [LIB, HOUSE]},
{"id": "bio_name", "group": "biographer", "query": "What's my name?",
"expect": [BIO], "forbid": [LIB, HOUSE]},
{"id": "bio_car", "group": "biographer", "query": "What car do I drive?",
"expect": [BIO], "forbid": [LIB, HOUSE]},
{"id": "bio_store", "group": "biographer", "query": "Remember that I prefer my coffee black.",
"expect": [BIO], "forbid": [LIB, HOUSE]},
{"id": "bio_list", "group": "biographer", "query": "What do you know about me?",
"expect": [BIO], "forbid": [LIB, HOUSE]},
{"id": "bio_forget", "group": "biographer", "query": "Forget my old address.",
"expect": [BIO], "forbid": [LIB, HOUSE]},
# --- research and current information -> librarian ------------------------
{"id": "lib_weather", "group": "librarian", "query": "What's the weather in Rotterdam tomorrow?",
"expect": [LIB], "forbid": [HOUSE]},
{"id": "lib_news", "group": "librarian", "query": "What's in the news today?",
"expect": [LIB], "forbid": [HOUSE, BIO]},
{"id": "lib_url", "group": "librarian", "query": "Read https://example.com/article and summarise it.",
"expect": [LIB], "forbid": [HOUSE, BIO]},
{"id": "lib_research", "group": "librarian", "query": "Research how tidal power stations work.",
"expect": [LIB], "forbid": [HOUSE, BIO]},
{"id": "lib_wiki_create", "group": "librarian", "query": "Create a wiki page about our network topology.",
"expect": [LIB], "forbid": [HOUSE, BIO]},
# --- home automation -> housekeeper --------------------------------------
{"id": "house_lights_on", "group": "housekeeper", "query": "Turn on the kitchen lights.",
"expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
{"id": "house_lights_off", "group": "housekeeper", "query": "Switch off all the lights downstairs.",
"expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
{"id": "house_thermostat", "group": "housekeeper", "query": "Set the thermostat to 20 degrees.",
"expect": [HOUSE], "forbid": [LIB, BIO]},
{"id": "house_blinds", "group": "housekeeper", "query": "Close the blinds in the living room.",
"expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
# --- conversational -> nothing at all -------------------------------------
# The expensive failure mode: a greeting that triggers a web search.
{"id": "chat_greeting", "group": "conversational", "query": "Hello!",
"expect": [], "forbid": ALL},
{"id": "chat_thanks", "group": "conversational", "query": "Thanks, that's helpful.",
"expect": [], "forbid": ALL},
{"id": "chat_joke", "group": "conversational", "query": "Tell me a joke.",
"expect": [], "forbid": ALL},
{"id": "chat_howareyou", "group": "conversational", "query": "How are you doing today?",
"expect": [], "forbid": ALL},
{"id": "chat_prior_turn", "group": "conversational", "query": "What did I just say?",
"expect": [], "forbid": ALL},
# --- genuinely multi-capability -------------------------------------------
{"id": "multi_weather_home", "group": "multi",
"query": "What's the weather here, and remember that I like it warm?",
"expect": [LIB, BIO], "forbid": []},
{"id": "multi_recall_search", "group": "multi",
"query": "Look up the best route from my home address to Utrecht.",
"expect": [BIO, LIB], "forbid": []},
{"id": "multi_math_memory", "group": "multi",
"query": "Remember that my budget is 500 euro, then work out 12% of it.",
"expect": [BIO, CORE], "forbid": [LIB, HOUSE]},
# --- adversarial: vocabulary that used to select agents by substring ------
# "temperature" is a housekeeper domain, but this is a unit conversion.
{"id": "adv_temperature", "group": "adversarial", "query": "Convert 98.6 Fahrenheit to Celsius.",
"expect": [CORE], "forbid": [HOUSE, LIB, BIO]},
# "description" contains "script"; "discover" contains "cover".
{"id": "adv_description", "group": "adversarial",
"query": "Give me a short description of what 17 times 23 comes to.",
"expect": [CORE], "forbid": [HOUSE, LIB]},
# "acknowledge" contains "knowledge" and "know".
{"id": "adv_acknowledge", "group": "adversarial",
"query": "Just acknowledge this and add 5 and 6 for me.",
"expect": [CORE], "forbid": [LIB, BIO]},
# "my" appears inside "economy".
{"id": "adv_economy", "group": "adversarial",
"query": "How many zeros are in one trillion?",
"expect": [CORE], "forbid": [BIO, HOUSE]},
# "fan" inside "fantastic"; also a climate word without a home-control intent.
{"id": "adv_fantastic", "group": "adversarial",
"query": "That's fantastic. What is 8 squared?",
"expect": [CORE], "forbid": [HOUSE, LIB]},
# "home" without any actuation intent.
{"id": "adv_home_word", "group": "adversarial", "query": "What time do I usually get home?",
"expect": [BIO], "forbid": [HOUSE]},
# "search" as ordinary English, not a web-search request.
{"id": "adv_search_word", "group": "adversarial",
"query": "No need to search anything, just tell me what 9 times 9 is.",
"expect": [CORE], "forbid": [LIB]},
# "create"/"write" are librarian domains but this is conversational.
{"id": "adv_write_word", "group": "adversarial", "query": "Can you write that more simply?",
"expect": [], "forbid": [LIB, HOUSE]},
# --- mutating intents: routing only, nothing is ever executed -------------
{"id": "mutate_wiki_update", "group": "mutating", "query": "Update the dossier page with today's findings.",
"expect": [LIB], "forbid": [HOUSE, CORE]},
{"id": "mutate_scene", "group": "mutating", "query": "Run the movie night scene.",
"expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
]
GROUPS = sorted({f["group"] for f in FIXTURES})
assert len({f["id"] for f in FIXTURES}) == len(FIXTURES), "duplicate fixture id"
+121
View File
@@ -0,0 +1,121 @@
"""
Guard production's GPU residency across a benchmark run.
Benchmarks swap models on the card production is serving from. Ollama evicts to
make room, so a run leaves its own models resident and the production one gone:
the next voice turn pays a ~36s cold load, and the pin that prevented it is
silently lost. That happened on 2026-08-08 — a routing benchmark evicted
gemma4:e2b and left gemma4:e4b behind, and only the monitoring noticing
`unexpected_models` caught it.
Snapshot before, restore after, and wire the restore to SIGTERM as well as the
normal path. Python runs `finally` for SIGINT, which arrives as
KeyboardInterrupt, but the default SIGTERM action terminates outright — so
`timeout`, a systemd stop or a plain `kill` would skip the guard entirely.
from scripts.ollama_residency import residency_guard, install_sigterm_handler
install_sigterm_handler()
with residency_guard(models_used=["gemma4:e4b"]):
...
"""
from __future__ import annotations
import signal
from collections.abc import Iterator
from contextlib import contextmanager
from datetime import UTC, datetime
from typing import Any
import httpx
OLLAMA_URL = "http://localhost:11434"
# keep_alive:-1 yields a year-2318 expiry, so "pinned" is simply "expires more
# than a day out". Matches check-ai-pipeline.sh in system-admin-toj.
PINNED_THRESHOLD_SECONDS = 86400
def install_sigterm_handler() -> None:
"""Make SIGTERM raise, so `finally` blocks and context managers still run."""
def _raise(signum, _frame):
raise KeyboardInterrupt(f"signal {signum}")
signal.signal(signal.SIGTERM, _raise)
def snapshot_residency(client: httpx.Client | None = None) -> dict[str, bool]:
"""Resident models mapped to whether each is pinned."""
owns = client is None
client = client or httpx.Client(timeout=30)
try:
data = client.get(f"{OLLAMA_URL}/api/ps", timeout=10).json()
except Exception: # noqa: BLE001 - a missing snapshot must not abort the run
return {}
finally:
if owns:
client.close()
resident: dict[str, bool] = {}
now = datetime.now(UTC)
for model in data.get("models", []):
pinned = False
try:
expires = datetime.fromisoformat(model.get("expires_at", "").replace("Z", "+00:00"))
pinned = (expires - now).total_seconds() > PINNED_THRESHOLD_SECONDS
except ValueError:
pass
resident[model["name"]] = pinned
return resident
def set_keep_alive(model: str, keep_alive: Any, client: httpx.Client | None = None) -> bool:
"""Load, unload or pin a model. Embedding models reject /api/generate."""
owns = client is None
client = client or httpx.Client(timeout=180)
payload = {"model": model, "keep_alive": keep_alive}
try:
for endpoint in ("generate", "embed"):
try:
response = client.post(f"{OLLAMA_URL}/api/{endpoint}", json=payload, timeout=180)
except Exception: # noqa: BLE001
return False
if response.status_code == 200:
return True
if response.status_code == 400 and "does not support generate" in response.text:
continue # embedding-only model; try /api/embed
return False
return False
finally:
if owns:
client.close()
def restore_residency(before: dict[str, bool], used: list[str]) -> None:
"""Evict what the benchmark loaded, then re-pin what was pinned before."""
base = {name.split(":")[0] for name in before}
with httpx.Client(timeout=180) as client:
for model in used:
if model not in before and model.split(":")[0] not in base:
print(f" residency: unloading benchmark model {model}")
set_keep_alive(model, 0, client)
for name, pinned in before.items():
if not pinned:
continue
ok = set_keep_alive(name, -1, client)
print(f" residency: re-pinned {name}" if ok
else f" residency: FAILED to re-pin {name} -- run warmup-ollama.sh")
@contextmanager
def residency_guard(models_used: list[str]) -> Iterator[dict[str, bool]]:
"""Snapshot residency on entry, restore it on exit however that happens."""
before = snapshot_residency()
pinned = [n for n, p in before.items() if p]
print(f" residency: resident before {sorted(before)}"
f"{f' (pinned: {pinned})' if pinned else ''}")
try:
yield before
finally:
print(" residency: restoring ...")
restore_residency(before, models_used)
+1 -1
View File
@@ -3,7 +3,7 @@
# Verifies room groups are controlled by checking actual state changes
API_URL="http://localhost:8777/v1/chat/completions"
CORE_API="http://192.168.86.149:8083"
CORE_API="http://localhost:8083"
RESULTS_FILE="/tmp/housekeeper_test_results.txt"
GREEN='\033[0;32m'
+67 -14
View File
@@ -19,35 +19,88 @@ from .schemas import ConversationContext, StewardRecommendation
logger = get_logger(__name__)
_DELEGATE_LINE_RE = re.compile(r"^[ \t]*DELEGATE:[ \t]*(.+)$", re.IGNORECASE | re.MULTILINE)
def _mentions(needle: str, haystack: str) -> bool:
"""Whole-word containment. Substring matching is what made this go wrong."""
return re.search(rf"(?<!\w){re.escape(needle)}(?!\w)", haystack) is not None
def _extract_capabilities(text: str) -> list[str]:
"""
Extract capability names from Steward's text response.
Extract capability names from the Steward's declared delegation.
Uses keyword matching to find mentioned capabilities.
The prompt instructs the Steward to answer in a fixed shape::
DELEGATE: <capability> to <action> <task>
REASON: ...
COMPLEXITY: ...
CONTEXT: ...
Only the DELEGATE line states intent; the rest is free prose. An earlier
version substring-matched capability *domains* across the whole response,
which routed on ordinary English: "description" contains "script" and
"discover" contains "cover" (both housekeeper domains), "acknowledge"
contains "knowledge" and "know" (librarian, biographer), and "economy"
contains "my" (biographer). Any REASON line could therefore summon agents
the Steward never asked for, and a spurious librarian is a real
multi-second web call.
It also made prose length a routing input, so anything that shortened the
Steward's output — such as disabling model thinking — would look like it had
improved routing.
Resolution is layered, most explicit first:
1. a DELEGATE line beginning with a capability name — the documented shape
2. a capability named anywhere on a DELEGATE line
3. a capability *domain* on a DELEGATE line, for a loosely worded answer
4. no DELEGATE line: capability names only, never domains
Args:
text: Steward's plain text analysis
Returns:
List of capability names (e.g., ['tatlock_core'])
List of capability names (e.g. ['tatlock_core']), de-duplicated.
"""
text_lower = text.lower()
registry = get_household_registry()
capabilities = registry.get_all_capabilities()
delegate_lines = [line.strip().lower() for line in _DELEGATE_LINE_RE.findall(text or "")]
found_caps = []
found_caps: list[str] = []
for cap in capabilities:
# Check if capability name is mentioned
if cap.name.lower() in text_lower:
found_caps.append(cap.name)
def _add(name: str) -> None:
if name not in found_caps:
found_caps.append(name)
if not delegate_lines:
# Either the Steward judged no capability necessary — the prompt's
# conversational path, whose correct answer is [] — or it ignored the
# format. Names only: domain words are ordinary English and would fire
# on any prose, which is the bug described above.
haystack = (text or "").lower()
for cap in capabilities:
if _mentions(cap.name.lower(), haystack):
_add(cap.name)
return found_caps
for line in delegate_lines:
leading = next((c for c in capabilities if line.startswith(c.name.lower())), None)
if leading is not None:
_add(leading.name)
continue
# Check if any domains are mentioned
for domain in cap.domains:
if domain.lower() in text_lower:
found_caps.append(cap.name)
break
named = [c for c in capabilities if _mentions(c.name.lower(), line)]
if named:
for cap in named:
_add(cap.name)
continue
# Last resort. Scoped to this line, so the REASON and CONTEXT prose that
# caused the original misrouting can no longer reach it.
for cap in capabilities:
if any(_mentions(domain.lower(), line) for domain in cap.domains):
_add(cap.name)
return found_caps
+6 -6
View File
@@ -110,8 +110,8 @@ class Config(BaseSettings):
# SearXNG Configuration
SEARXNG_HOST: HttpUrl = Field(
default="http://localhost:8087",
description="SearXNG server URL"
default="http://searxng:8080",
description="SearXNG server URL (container name; internal port 8080)"
)
SEARXNG_TIMEOUT: int = Field(
default=30,
@@ -138,8 +138,8 @@ class Config(BaseSettings):
description="Total time budget for a librarian delegation in seconds"
)
LIBRARY_DESK_HOST: HttpUrl = Field(
default="http://localhost:8089",
description="Library-Desk API URL"
default="http://library-desk:8089",
description="Library-Desk API URL (container name; internal port 8089)"
)
LIBRARY_DESK_API_KEY: str = Field(
default="",
@@ -152,8 +152,8 @@ class Config(BaseSettings):
# Core-API Configuration (The Housekeeper backend)
CORE_API_HOST: HttpUrl = Field(
default="http://localhost:8090",
description="Core-API URL for Home Assistant integration"
default="http://core-api:8083",
description="Core-API URL for Home Assistant integration (container name; internal port 8083)"
)
CORE_API_KEY: str = Field(
default="",
+92 -1
View File
@@ -8,7 +8,12 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from src.agents.steward.schemas import ConversationContext, StewardRecommendation
from src.agents.steward.service import analyze_request, format_steward_note, _build_enriched_query
from src.agents.steward.service import (
_build_enriched_query,
_extract_capabilities,
analyze_request,
format_steward_note,
)
from src.core.startup import register_household_members
@@ -283,3 +288,89 @@ class TestBuildEnrichedQuery:
result = _build_enriched_query(query, memory_context)
assert result == query
class TestExtractCapabilities:
"""Capability extraction reads the declared DELEGATE line, not free prose.
The prompt tells the Steward to state its choice on a DELEGATE line and to
explain itself on REASON/COMPLEXITY/CONTEXT lines. An earlier version
substring-matched capability domains across the entire response, so ordinary
English in the explanation routed requests: "description" contains the
housekeeper domain "script", "acknowledge" contains "know". These tests pin
that the explanation can no longer influence routing.
"""
# (prose, why it used to misroute)
SUBSTRING_TRAPS = [
("The user wants a description of the algorithm.", "script -> housekeeper"),
("I should discover what the answer is.", "cover -> housekeeper"),
("That sounds fantastic, let me compute it.", "fan -> housekeeper"),
("I acknowledge the request to add two numbers.", "knowledge/know -> librarian, biographer"),
("The user asks about the economy myth.", "my -> biographer"),
("Convert 98.6 Fahrenheit to Celsius.", "temperature is a housekeeper domain"),
]
@pytest.mark.parametrize("prose,reason", SUBSTRING_TRAPS)
def test_reason_prose_cannot_add_capabilities(self, prose, reason):
"""Explanatory prose must not summon agents the Steward did not request."""
text = f"DELEGATE: tatlock_core to calculate\nREASON: {prose}\nCOMPLEXITY: simple"
assert _extract_capabilities(text) == ["tatlock_core"], f"regression: {reason}"
def test_delegate_line_task_text_does_not_leak(self):
"""A domain word inside the task description must not add a capability.
"home" is a housekeeper domain, but this is plainly a memory recall.
"""
text = "DELEGATE: biographer to recall the user's home address\nREASON: personal data"
assert _extract_capabilities(text) == ["biographer"]
def test_multiple_delegate_lines(self):
"""Each DELEGATE line contributes its capability, in order, deduplicated."""
text = (
"DELEGATE: biographer to recall the user's location\n"
"DELEGATE: librarian to search_web for the forecast\n"
"DELEGATE: biographer to recall preferences\n"
)
assert _extract_capabilities(text) == ["biographer", "librarian"]
def test_capability_named_later_on_the_line(self):
"""A loosely worded DELEGATE line still resolves by name."""
text = "DELEGATE: ask the librarian to search the web"
assert _extract_capabilities(text) == ["librarian"]
def test_domain_fallback_within_delegate_line(self):
"""With no capability named, domains on the DELEGATE line still resolve."""
text = "DELEGATE: turn on the lights in the kitchen"
assert _extract_capabilities(text) == ["housekeeper"]
def test_conversational_response_selects_nothing(self):
"""No DELEGATE line means no capability, which is the prompt's chat path."""
text = "This is a simple greeting. No capabilities are needed. COMPLEXITY: simple"
assert _extract_capabilities(text) == []
def test_malformed_response_still_routes_by_name(self):
"""If the format is ignored, a named capability is still honoured."""
text = "I think the librarian should handle this research request."
assert _extract_capabilities(text) == ["librarian"]
def test_malformed_response_does_not_route_on_domains(self):
"""...but bare prose must not route on domain words alone."""
text = "The user wants a description of home automation, and I acknowledge it."
assert _extract_capabilities(text) == []
def test_case_insensitive_delegate_marker(self):
text = "delegate: Librarian to search_web"
assert _extract_capabilities(text) == ["librarian"]
def test_empty_input(self):
assert _extract_capabilities("") == []
+5 -5
View File
@@ -4,8 +4,8 @@ These tests make real HTTP requests to the running Tatlock API server to verify
## Prerequisites
1. **Server must be running** on `http://localhost:8777` (use `./wakeup.sh`)
2. **Ollama must be running** with `mistral-nemo:latest` model
1. **Server must be running** on `http://localhost:8777` (use `make run`)
2. **Ollama must be running** with the `gemma4:e2b` model
3. **Redis must be running** (for benchmarking)
4. **Qdrant must be running** on `http://localhost:6333` (for memory tests)
@@ -15,9 +15,9 @@ These tests make real HTTP requests to the running Tatlock API server to verify
```bash
# Terminal 1: Start the server (auto-reload enabled)
./wakeup.sh
make run
# Logs are written to logs/server.log - tail them in another terminal:
# Logs are written to build/logs/server.log - tail them in another terminal:
tail -f logs/server.log
```
@@ -132,7 +132,7 @@ memory = await qdrant.find_memory_by_key("memories_llm_tester", "favorite_color"
Make sure the server is running:
```bash
./wakeup.sh
make run
curl http://localhost:8777/health # Should return 200
```
+2 -2
View File
@@ -11,9 +11,9 @@ These tests hit the actual running server and verify data persistence.
They use the `llm_tester` user for isolation from production data.
Requirements:
- Server running on localhost:8777 (use ./wakeup.sh)
- Server running on localhost:8777 (use `make run`)
- Qdrant running on localhost:6333
- Ollama running with mistral-nemo model
- Ollama running with the gemma4:e2b model
Note: LLM outputs are non-deterministic. Tests use flexible assertions
that check for behavioral patterns rather than exact text matches.