chore: release v2.4.3

Ships the Steward capability-extraction fix (a905363), which has been on
main since earlier today while production continued to route on prose:
the running v2.4.2 still matches capability domains as substrings across
the Steward's whole response, so "description" selects housekeeper and
"acknowledge" selects librarian and biographer.

Patch rather than minor: no new capability, and the JSON on the wire is
unchanged. What changes is which agents get invoked, and only in the
cases that were already wrong.

Also carries the routing benchmark, its fixtures, the shared GPU
residency guard and the findings document, none of which are
user-visible.

Co-Authored-By: Claude <noreply@anthropic.com>

AGENTS.md

@@ -17,8 +17,8 @@ This document contains instructions and documentation references for AI assistan
 
 ### 🧪 Local Development Setup
 * **Always test locally first** before committing and deploying. The build-deploy loop is slow.
-* **Start the local server** with `./wakeup.sh` - logs are written to `logs/server.log` for easy tailing
-* **Auto-reload**: The wakeup script runs uvicorn in reload mode - code changes are picked up automatically without restart (except for requirements.txt changes)
+* **Start the local server** with `make run` - logs are written to `build/logs/server.log` for easy tailing
+* **Auto-reload**: `make run` runs uvicorn in reload mode - code changes are picked up automatically without restart (except for dependency changes)
 * **Test REST endpoints** against `http://localhost:8777` using curl or similar tools
 * **Only deploy** when a phase or feature is complete and tested locally
 * **Environment**: Copy `.env.example` to `.env` and configure for your local setup (Ollama, Redis, Qdrant hosts)

CHANGELOG.md

@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.4.3] - 2026-08-08
+
+### Fixed
+
+- Steward routing no longer triggers on words inside its own explanation. Capability
+  extraction reads the declared `DELEGATE:` line instead of substring-matching
+  capability domains across the whole response, where ordinary English routed
+  requests — "description" contains the housekeeper domain "script", "acknowledge"
+  contains "knowledge" and "know". A spurious capability meant a real agent call,
+  including web searches, on queries that needed none.
+
 ## [2.4.2] - 2026-07-19
 
 ### Fixed

CLAUDE.md

@@ -27,7 +27,7 @@ Dependencies are in `pyproject.toml` (`[project.dependencies]` and `[project.opt
 
 **Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return a 400. Use `get_sampling_settings()` from the model selector instead of passing `ModelSettings(temperature=...)` directly to agents that can run on the Claude fallback. The contract test suite pins this (`make test-contracts`).
 
-**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). Current GPU-resident numbers (2026-07-14, driver 570, gemma4:e2b at ~100 tok/s): Steward analysis ~6s warm, full Steward → orchestrate → synthesize flow 11–25s, librarian-routed queries ~20-25s. The old "~35s steward / ~2 min flow" figures were measured during the CPU-only era (driver mismatch, 13 tok/s) — do not plan against them. Cold start after 2h idle adds ~8s (`OLLAMA_KEEP_ALIVE=2h`). `STEWARD_TIMEOUT` defaults to 60s.
+**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). Current GPU-resident numbers (measured 2026-08-07, gemma4:e2b at ~95 tok/s): full Steward → orchestrate → synthesize flow ~10–13s for simple turns; librarian-routed queries ~20-25s (not re-measured). A single turn costs **3 sequential Ollama calls and ~710 generated tokens** even for "what is 61 plus 12?" — most of it the model's own reasoning, paid three times. Cold model load is ~36s, avoided while the model is pinned with `keep_alive: -1`; the `OLLAMA_KEEP_ALIVE=2h` default otherwise reintroduces it. The old "~35s steward / ~2 min flow" and "11–25s flow" figures are superseded — do not plan against them. `STEWARD_TIMEOUT` defaults to 60s.
 
 **`get_benchmark_store` does not exist.** The benchmarking module (`src/core/benchmarks.py`) was never implemented. `scripts/benchmark_analysis.py` also references it and is broken. Do not add mocks for it in tests.
 

docs/claude-integration.md

@@ -28,7 +28,7 @@ src/mcp/
 
 ```yaml
 tatlock-mcp:
-  image: git.schweitz.internal/jpmschweitzer/tatlock:latest
+  image: git.schweitz.net/jpmschweitzer/tatlock:latest
   command: ["python", "-m", "src.mcp.server"]
   ports:
     - "8002:8002"

docs/steward-routing-findings.md

@@ -0,0 +1,110 @@
+# Steward Routing & Thinking — Findings
+
+**Outcome: no change shipped.** The Steward stays on `gemma4:e2b` with model
+thinking left at its default (on). Every alternative was measured and every one
+loses. This document exists so the experiment is not repeated on the same
+premise.
+
+Run 2026-08-08 with `scripts/benchmark_routing.py` and
+`scripts/fixtures/routing_fixtures.py` (40 labelled queries, one repeat per
+cell, temperature 0.3 as production sends).
+
+---
+
+## The premise was wrong
+
+The experiment was designed around an observation that the Steward pays ~300
+tokens per turn for reasoning that is generated and thrown away: it calls
+`/api/generate`, gemma4 reasons by default, and **no `thinking` field comes back
+in the response**. Disabling thinking therefore looked close to free.
+
+It is not. The reasoning is not discarded — it is emitted inline in `response`,
+and it is what produces a correct `DELEGATE:` line. Those tokens are the work,
+not waste. Suppressing them costs 12.5 points of routing accuracy.
+
+## Results
+
+| config | exact | under | over | tokens | latency | resident | predicted | co-resident with nomic |
+|---|---|---|---|---|---|---|---|---|
+| **e2b, thinking** *(production)* | **97.5%** | 2.5% | 0% | 361 | 5179 ms | 1778 MB | 7.8 GiB | yes |
+| e2b, `think: false` | 85.0% | 12.5% | 5.0% | 48 | 1435 ms | 1778 MB | 7.8 GiB | yes |
+| e4b, thinking | 100% | 0% | 0% | 192 | 4726 ms | 3089 MB | 10.6 GiB | **no** |
+| e4b, `think: false` | 97.5% | 2.5% | 0% | 52 | 2269 ms | 3089 MB | 10.6 GiB | **no** |
+
+`think: true` was also measured and landed within one fixture of the default on
+both models, so production's implicit thinking is the same thing as asking for
+it explicitly. Format compliance was 100% in every cell — a `DELEGATE:` line is
+always emitted.
+
+With 40 fixtures and one repeat, each result is worth 2.5 points, so the
+97.5-vs-100 gaps are single fixtures and inside the noise. The latency and token
+medians (40 calls each) and the e2b `think: false` degradation (6 failures with a
+consistent mechanism) are the parts worth trusting.
+
+## Why each alternative loses
+
+**`think: false` on e2b** — 85% exact, and the failures are not random. All three
+multi-capability fixtures under-route, each missing a second capability. Without
+reasoning the model names one capability and stops decomposing. It is not
+degraded across the board; it specifically stops handling compound requests,
+which is where a user would most notice the Butler quietly doing half the job.
+
+**e4b, either setting** — disqualified by memory, not by quality. Ollama predicts
+**10.6 GiB** for it at 16k context. Maximum available on this card is ~7.9 GiB
+(10.4 free − 2.0 GPU overhead − 0.46 minimum), so e4b *always* exceeds the budget
+and evicts every co-resident before loading. Observed directly: loading it threw
+out both `gemma4:e2b` and `nomic-embed-text`. Losing nomic means Tatlock memory
+and library-desk thrash on every embedding call. Note this is not caused by the
+2 GiB reservation — without it, available would be ~9.7 GiB, still under 10.6.
+
+**Lower `OLLAMA_CONTEXT_LENGTH`** — the obvious way to free headroom, and it does
+not work. Dropping 16384 → 2048, an 8× reduction, moved the prediction only from
+7.8 to 6.7 GiB. The prediction is dominated by weights and batch size, not KV
+cache. It would also truncate the Librarian's retrieved passages and webber's
+code context for a 14% saving that funds nothing.
+
+**Per-request `num_ctx`** — worse. A single request with a different `num_ctx`
+reloads the shared runner, which **drops the `keep_alive: -1` pin** (expiry fell
+from year-2318 to a 2-hour default) and evicts nomic. Three services share this
+Ollama, so mixed context sizes are a thrash generator, and it fails silently.
+
+**`OLLAMA_NUM_PARALLEL > 1`** — never viable here. e2b already predicts 7.8 GiB
+against ~7.9 available, so there is no room for a second slot at any context
+length. It is also set to 1 deliberately, to avoid batch overflow panics.
+
+## What the two axes actually control
+
+They do not interact, which is the useful part:
+
+- **Model choice** governs VRAM and co-residency. e2b 1778 MB, e4b 3089 MB.
+- **Think setting** governs tokens, latency and routing quality — and costs
+  **nothing** in VRAM. Verified: e2b is resident at 1778 MB with `think` unset,
+  true and false alike, because the KV cache is allocated for the full context at
+  load time and `think` is a per-request generation parameter.
+
+So the only real question is whether 313 tokens and 3.7 seconds are worth 12.5
+points of compound-query routing. On a turn that is already three sequential
+Ollama calls, they are.
+
+## Prerequisite: the extraction fix
+
+These numbers are only meaningful because `_extract_capabilities` was fixed first
+(commit `a905363`). It previously substring-matched capability *domains* across
+the Steward's entire response, so ordinary English in the `REASON:` line selected
+agents — "description" contains the housekeeper domain "script", "acknowledge"
+contains "knowledge" and "know".
+
+That made **prose length a routing input**. Benchmarking against it would have
+shown `think: false` improving routing purely because shorter output produces
+fewer accidental substring hits — a thinking policy derived from a parsing
+artefact. The `adversarial` fixture group is regression coverage for exactly this.
+
+## If this is revisited
+
+The constraint is the single 11 GB card, not the model. A second inference host
+(*forge*) removes it entirely, and e4b's 100% routing becomes reachable without
+evicting anything. Re-run then; on this card the answer is settled.
+
+`scripts/benchmark_routing.py` takes `--models`, `--think` and `--repeats`, and
+restores GPU residency on exit — including on SIGTERM, which the first version
+did not.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "tatlock"
-version = "2.4.2"
+version = "2.4.3"
 description = "OpenAI-compatible API with Ollama backend"
 requires-python = ">=3.12"
 dependencies = [

scripts/benchmark_routing.py

@@ -0,0 +1,235 @@
+"""
+Benchmark Steward routing quality against model and thinking settings.
+
+Talks to Ollama directly. No Tatlock server, no agents, no tools, nothing is
+executed — the mutating fixtures ("turn on the lights", "update the wiki") only
+ever produce a routing decision. That makes this cheap and repeatable, and it
+isolates the question: does the Steward still pick the right capabilities when
+the model reasons less?
+
+The request body is byte-identical to StewardAgent._call_ollama, plus the
+`think` flag under test, so a cell labelled `unset` is exactly what production
+sends today.
+
+Three thinking settings, because "on vs off" hides the interesting case:
+
+    unset   what production sends now. gemma4 reasons by default, and the
+            response carries no `thinking` field, so those tokens are generated
+            and discarded.
+    true    reasoning requested explicitly and returned in `thinking`.
+    false   reasoning suppressed.
+
+Scoring is deliberately asymmetric. A missing capability under-routes and the
+Butler answers without a tool it needed; a spurious one over-routes, and that is
+a real agent call — a stray librarian is a multi-second web search on a query
+that asked for arithmetic. Over-routing is the predicted failure when thinking
+is off, so `forbid` violations are reported separately rather than folded into
+one accuracy number.
+
+Usage:
+    .venv/bin/python scripts/benchmark_routing.py
+    .venv/bin/python scripts/benchmark_routing.py --models gemma4:e2b
+    .venv/bin/python scripts/benchmark_routing.py --think false --repeats 3
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import statistics
+import sys
+import time
+from datetime import UTC, datetime
+from pathlib import Path
+from typing import Any
+
+import httpx
+
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(PROJECT_ROOT))
+
+from scripts.fixtures.routing_fixtures import FIXTURES  # noqa: E402
+from scripts.ollama_residency import (  # noqa: E402
+    install_sigterm_handler,
+    residency_guard,
+)
+from src.agents.steward.agent import build_steward_prompt  # noqa: E402
+from src.agents.steward.service import _DELEGATE_LINE_RE, _extract_capabilities  # noqa: E402
+from src.core.startup import register_household_members  # noqa: E402
+
+OLLAMA_URL = "http://localhost:11434"
+DEFAULT_MODELS = ["gemma4:e2b", "gemma4:e4b"]
+DEFAULT_THINK = ["unset", "true", "false"]
+RESULTS_DIR = PROJECT_ROOT / "logs"
+
+
+def build_body(model: str, prompt: str, think: str) -> dict[str, Any]:
+    """Mirror StewardAgent._call_ollama exactly, then add the flag under test."""
+    body: dict[str, Any] = {
+        "model": model,
+        "prompt": prompt,
+        "stream": False,
+        "options": {
+            "temperature": 0.3,  # Lower = more consistent
+            "top_p": 0.9,
+        },
+    }
+    if think != "unset":
+        body["think"] = think == "true"
+    return body
+
+
+def call(client: httpx.Client, body: dict[str, Any]) -> dict[str, Any] | None:
+    try:
+        response = client.post(f"{OLLAMA_URL}/api/generate", json=body)
+        response.raise_for_status()
+        return response.json()
+    except Exception as exc:  # noqa: BLE001 - a failed cell must not abort the run
+        print(f"    ! {exc}", file=sys.stderr)
+        return None
+
+
+def score(fixture: dict, found: list[str]) -> dict[str, Any]:
+    expected = set(fixture["expect"])
+    forbidden = set(fixture["forbid"])
+    got = set(found)
+    missing = sorted(expected - got)
+    spurious = sorted(got & forbidden)
+    return {
+        "found": found,
+        "missing": missing,
+        "spurious": spurious,
+        # Exact only when everything expected arrived and nothing forbidden did.
+        "exact": not missing and not spurious,
+        "under_routed": bool(missing),
+        "over_routed": bool(spurious),
+    }
+
+
+def run_cell(client: httpx.Client, model: str, think: str, repeats: int) -> list[dict[str, Any]]:
+    rows: list[dict[str, Any]] = []
+    for fixture in FIXTURES:
+        prompt = build_steward_prompt(fixture["query"], [])
+        body = build_body(model, prompt, think)
+        for rep in range(repeats):
+            started = time.perf_counter()
+            data = call(client, body)
+            elapsed_ms = (time.perf_counter() - started) * 1000
+            if data is None:
+                rows.append({
+                    "id": fixture["id"], "group": fixture["group"], "rep": rep,
+                    "error": True, "exact": False, "under_routed": False, "over_routed": False,
+                })
+                continue
+
+            text = data.get("response", "") or ""
+            found = _extract_capabilities(text)
+            rows.append({
+                "id": fixture["id"],
+                "group": fixture["group"],
+                "rep": rep,
+                "error": False,
+                "latency_ms": round(elapsed_ms, 1),
+                "eval_tokens": data.get("eval_count"),
+                "prompt_tokens": data.get("prompt_eval_count"),
+                # Did the model obey the documented output shape at all?
+                "has_delegate_line": bool(_DELEGATE_LINE_RE.search(text)),
+                # Whether reasoning came back, as opposed to being generated and dropped.
+                "thinking_returned": bool(data.get("thinking")),
+                "response_chars": len(text),
+                **score(fixture, found),
+            })
+    return rows
+
+
+def summarise(rows: list[dict[str, Any]]) -> dict[str, Any]:
+    ok = [r for r in rows if not r["error"]]
+    if not ok:
+        return {"n": 0, "errors": len(rows)}
+    latencies = [r["latency_ms"] for r in ok]
+    tokens = [r["eval_tokens"] for r in ok if r["eval_tokens"] is not None]
+    return {
+        "n": len(ok),
+        "errors": len(rows) - len(ok),
+        "exact_pct": round(100 * sum(r["exact"] for r in ok) / len(ok), 1),
+        "under_routed_pct": round(100 * sum(r["under_routed"] for r in ok) / len(ok), 1),
+        "over_routed_pct": round(100 * sum(r["over_routed"] for r in ok) / len(ok), 1),
+        "format_ok_pct": round(100 * sum(r["has_delegate_line"] for r in ok) / len(ok), 1),
+        "thinking_returned_pct": round(100 * sum(r["thinking_returned"] for r in ok) / len(ok), 1),
+        "latency_ms_median": round(statistics.median(latencies), 1),
+        "latency_ms_mean": round(statistics.fmean(latencies), 1),
+        "eval_tokens_median": round(statistics.median(tokens), 1) if tokens else None,
+        "eval_tokens_total": sum(tokens) if tokens else None,
+    }
+
+
+def main() -> int:
+    install_sigterm_handler()
+
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--models", default=",".join(DEFAULT_MODELS))
+    parser.add_argument("--think", default=",".join(DEFAULT_THINK),
+                        help="comma-separated subset of unset,true,false")
+    parser.add_argument("--repeats", type=int, default=1)
+    parser.add_argument("--timeout", type=float, default=180.0)
+    args = parser.parse_args()
+
+    models = [m.strip() for m in args.models.split(",") if m.strip()]
+    think_modes = [t.strip() for t in args.think.split(",") if t.strip()]
+
+    # build_steward_prompt reads the registry, and the registry is populated at
+    # application startup. Without this the prompt lists no capabilities and every
+    # cell scores zero for reasons that have nothing to do with the model.
+    register_household_members()
+
+    print(f"{len(FIXTURES)} fixtures x {len(models)} models x {len(think_modes)} think "
+          f"x {args.repeats} repeats = {len(FIXTURES) * len(models) * len(think_modes) * args.repeats} calls\n")
+
+    cells: dict[str, Any] = {}
+    # The guard restores production's pinned models however this exits — a
+    # finished run, a failed cell, Ctrl-C or SIGTERM.
+    with residency_guard(models_used=models), httpx.Client(timeout=args.timeout) as client:
+        for model in models:
+            # Absorb the cold load (~36s) outside the measurements.
+            print(f"warming {model} ...", flush=True)
+            call(client, build_body(model, "hi", "false"))
+            for think in think_modes:
+                key = f"{model}|think={think}"
+                print(f"  {key} ...", end=" ", flush=True)
+                started = time.perf_counter()
+                rows = run_cell(client, model, think, args.repeats)
+                summary = summarise(rows)
+                cells[key] = {"summary": summary, "rows": rows}
+                print(f"exact={summary.get('exact_pct')}% "
+                      f"over={summary.get('over_routed_pct')}% "
+                      f"median={summary.get('latency_ms_median')}ms "
+                      f"({time.perf_counter() - started:.0f}s)")
+
+    RESULTS_DIR.mkdir(parents=True, exist_ok=True)
+    stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
+    out = RESULTS_DIR / f"routing-bench-{stamp}.json"
+    out.write_text(json.dumps({
+        "generated_at": datetime.now(UTC).isoformat(),
+        "fixtures": len(FIXTURES),
+        "repeats": args.repeats,
+        "cells": cells,
+    }, indent=2))
+
+    print(f"\n{'cell':28} {'exact':>7} {'under':>7} {'over':>7} {'fmt':>6} {'tok':>7} {'ms':>8}")
+    print("-" * 76)
+    for key, cell in cells.items():
+        s = cell["summary"]
+        print(f"{key:28} {s.get('exact_pct'):>6}% {s.get('under_routed_pct'):>6}% "
+              f"{s.get('over_routed_pct'):>6}% {s.get('format_ok_pct'):>5}% "
+              f"{str(s.get('eval_tokens_median')):>7} {s.get('latency_ms_median'):>8}")
+    print(f"\nwritten to {out}")
+    return 0
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main())
+    except KeyboardInterrupt:
+        # The residency guard has already run by the time this is caught;
+        # a traceback here would just bury its output.
+        print("\ninterrupted", file=sys.stderr)
+        sys.exit(130)

scripts/benchmark_steward.py

@@ -268,7 +268,7 @@ async def run_benchmarks(iterations: int = 10, verbose: bool = False):
             print(f"   Max: {overall_max:.3f}s (target: ≤5.0s)")
             print(f"   Avg: {overall_avg:.3f}s (target: ≤1.67s)")
             print(f"\n   Recommendations:")
-            print(f"   - Switch to a faster model (current: mistral-nemo)")
+            print(f"   - Switch to a faster model (current: gemma4:e2b)")
             print(f"   - Reduce system prompt complexity")
             print(f"   - Limit tool calls (currently limited to 3)")
             print(f"   - Consider caching household registry responses")

scripts/benchmark_tool_calling.py

@@ -17,12 +17,16 @@ import asyncio
 import json
 import re
 import statistics
+import sys
 import time
 from dataclasses import dataclass, field
 from pathlib import Path
 
 import httpx
 
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+from scripts.ollama_residency import install_sigterm_handler, residency_guard
+
 # ---------------------------------------------------------------------------
 # Configuration
 # ---------------------------------------------------------------------------
@@ -525,18 +529,31 @@ async def main():
     original_env = ENV_PATH.read_text()
 
     all_stats = []
-    async with httpx.AsyncClient() as client:
-        for model in models:
-            stats = await benchmark_model(client, model, args.iterations)
-            all_stats.append(stats)
-
-    # Restore original .env
-    ENV_PATH.write_text(original_env)
-    print(f"\n  .env restored to original")
+    # Both restores must survive a crash or an interrupt. The .env one especially:
+    # this script rewrites OLLAMA_DEFAULT_MODEL and lets uvicorn reload onto it,
+    # so bailing out mid-run used to leave the *running server* pointed at the
+    # benchmark model — and DEFAULT_MODELS starts at mistral-nemo-large, the 9.2G
+    # model implicated in the 2026-08-07 VRAM outage.
+    install_sigterm_handler()
+    try:
+        with residency_guard(models_used=models):
+            async with httpx.AsyncClient() as client:
+                for model in models:
+                    stats = await benchmark_model(client, model, args.iterations)
+                    all_stats.append(stats)
+    finally:
+        ENV_PATH.write_text(original_env)
+        print("\n  .env restored to original")
 
     print_comparison(all_stats)
     save_results(all_stats, Path(args.output))
 
 
 if __name__ == "__main__":
-    asyncio.run(main())
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        # .env and GPU residency are both restored by now; do not bury that
+        # output under a traceback.
+        print("\ninterrupted", file=sys.stderr)
+        raise SystemExit(130) from None

scripts/fixtures/routing_fixtures.py

@@ -0,0 +1,159 @@
+"""
+Labelled queries for the Steward routing benchmark.
+
+Each fixture carries both `expect` and `forbid`:
+
+  expect   capabilities that must appear. Missing one is under-routing — the
+           Butler answers without a tool it needed.
+  forbid   capabilities that must not appear. Over-routing is not cosmetic: a
+           spurious librarian is a real multi-second web call, and a spurious
+           housekeeper can actuate hardware.
+
+`forbid` matters more than `expect` here, because over-recommendation is the
+predicted failure when model thinking is disabled and the Steward has less room
+to discriminate.
+
+The `adversarial` group deserves explanation. Until 2026-08-08 the extractor
+substring-matched capability *domains* across the Steward's whole response, so
+ordinary English in its REASON line selected agents: "description" contains the
+housekeeper domain "script", "acknowledge" contains "knowledge" and "know",
+"economy" contains the biographer domain "my". Those queries invite exactly that
+vocabulary. They now serve as an end-to-end regression: routing must depend on
+what the Steward *decided*, not on the words it happened to use while explaining.
+
+Expectations follow the routing rules stated in the Steward prompt itself
+(src/agents/steward/agent.py), not on what a capability could plausibly cover.
+"""
+
+CORE = "tatlock_core"
+LIB = "librarian"
+BIO = "biographer"
+HOUSE = "housekeeper"
+ALL = [CORE, LIB, BIO, HOUSE]
+
+
+def _others(*keep: str) -> list[str]:
+    return [c for c in ALL if c not in keep]
+
+
+FIXTURES: list[dict] = [
+    # --- arithmetic and computation -> tatlock_core --------------------------
+    {"id": "math_add", "group": "math", "query": "What is 61 plus 12?",
+     "expect": [CORE], "forbid": _others(CORE)},
+    {"id": "math_percent", "group": "math", "query": "What is 15% of 240?",
+     "expect": [CORE], "forbid": _others(CORE)},
+    {"id": "math_compound", "group": "math", "query": "If I save 200 a month for 3 years, how much is that?",
+     "expect": [CORE], "forbid": _others(CORE)},
+    {"id": "math_sqrt", "group": "math", "query": "What is the square root of 1764?",
+     "expect": [CORE], "forbid": _others(CORE)},
+
+    # --- date and time -> tatlock_core ---------------------------------------
+    {"id": "time_now", "group": "datetime", "query": "What time is it?",
+     "expect": [CORE], "forbid": _others(CORE)},
+    {"id": "time_date", "group": "datetime", "query": "What is today's date?",
+     "expect": [CORE], "forbid": _others(CORE)},
+    {"id": "time_delta", "group": "datetime", "query": "How many days until Christmas?",
+     "expect": [CORE], "forbid": _others(CORE)},
+
+    # --- personal memory -> biographer ---------------------------------------
+    {"id": "bio_location", "group": "biographer", "query": "Where do I live?",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+    {"id": "bio_name", "group": "biographer", "query": "What's my name?",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+    {"id": "bio_car", "group": "biographer", "query": "What car do I drive?",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+    {"id": "bio_store", "group": "biographer", "query": "Remember that I prefer my coffee black.",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+    {"id": "bio_list", "group": "biographer", "query": "What do you know about me?",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+    {"id": "bio_forget", "group": "biographer", "query": "Forget my old address.",
+     "expect": [BIO], "forbid": [LIB, HOUSE]},
+
+    # --- research and current information -> librarian ------------------------
+    {"id": "lib_weather", "group": "librarian", "query": "What's the weather in Rotterdam tomorrow?",
+     "expect": [LIB], "forbid": [HOUSE]},
+    {"id": "lib_news", "group": "librarian", "query": "What's in the news today?",
+     "expect": [LIB], "forbid": [HOUSE, BIO]},
+    {"id": "lib_url", "group": "librarian", "query": "Read https://example.com/article and summarise it.",
+     "expect": [LIB], "forbid": [HOUSE, BIO]},
+    {"id": "lib_research", "group": "librarian", "query": "Research how tidal power stations work.",
+     "expect": [LIB], "forbid": [HOUSE, BIO]},
+    {"id": "lib_wiki_create", "group": "librarian", "query": "Create a wiki page about our network topology.",
+     "expect": [LIB], "forbid": [HOUSE, BIO]},
+
+    # --- home automation -> housekeeper --------------------------------------
+    {"id": "house_lights_on", "group": "housekeeper", "query": "Turn on the kitchen lights.",
+     "expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
+    {"id": "house_lights_off", "group": "housekeeper", "query": "Switch off all the lights downstairs.",
+     "expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
+    {"id": "house_thermostat", "group": "housekeeper", "query": "Set the thermostat to 20 degrees.",
+     "expect": [HOUSE], "forbid": [LIB, BIO]},
+    {"id": "house_blinds", "group": "housekeeper", "query": "Close the blinds in the living room.",
+     "expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
+
+    # --- conversational -> nothing at all -------------------------------------
+    # The expensive failure mode: a greeting that triggers a web search.
+    {"id": "chat_greeting", "group": "conversational", "query": "Hello!",
+     "expect": [], "forbid": ALL},
+    {"id": "chat_thanks", "group": "conversational", "query": "Thanks, that's helpful.",
+     "expect": [], "forbid": ALL},
+    {"id": "chat_joke", "group": "conversational", "query": "Tell me a joke.",
+     "expect": [], "forbid": ALL},
+    {"id": "chat_howareyou", "group": "conversational", "query": "How are you doing today?",
+     "expect": [], "forbid": ALL},
+    {"id": "chat_prior_turn", "group": "conversational", "query": "What did I just say?",
+     "expect": [], "forbid": ALL},
+
+    # --- genuinely multi-capability -------------------------------------------
+    {"id": "multi_weather_home", "group": "multi",
+     "query": "What's the weather here, and remember that I like it warm?",
+     "expect": [LIB, BIO], "forbid": []},
+    {"id": "multi_recall_search", "group": "multi",
+     "query": "Look up the best route from my home address to Utrecht.",
+     "expect": [BIO, LIB], "forbid": []},
+    {"id": "multi_math_memory", "group": "multi",
+     "query": "Remember that my budget is 500 euro, then work out 12% of it.",
+     "expect": [BIO, CORE], "forbid": [LIB, HOUSE]},
+
+    # --- adversarial: vocabulary that used to select agents by substring ------
+    # "temperature" is a housekeeper domain, but this is a unit conversion.
+    {"id": "adv_temperature", "group": "adversarial", "query": "Convert 98.6 Fahrenheit to Celsius.",
+     "expect": [CORE], "forbid": [HOUSE, LIB, BIO]},
+    # "description" contains "script"; "discover" contains "cover".
+    {"id": "adv_description", "group": "adversarial",
+     "query": "Give me a short description of what 17 times 23 comes to.",
+     "expect": [CORE], "forbid": [HOUSE, LIB]},
+    # "acknowledge" contains "knowledge" and "know".
+    {"id": "adv_acknowledge", "group": "adversarial",
+     "query": "Just acknowledge this and add 5 and 6 for me.",
+     "expect": [CORE], "forbid": [LIB, BIO]},
+    # "my" appears inside "economy".
+    {"id": "adv_economy", "group": "adversarial",
+     "query": "How many zeros are in one trillion?",
+     "expect": [CORE], "forbid": [BIO, HOUSE]},
+    # "fan" inside "fantastic"; also a climate word without a home-control intent.
+    {"id": "adv_fantastic", "group": "adversarial",
+     "query": "That's fantastic. What is 8 squared?",
+     "expect": [CORE], "forbid": [HOUSE, LIB]},
+    # "home" without any actuation intent.
+    {"id": "adv_home_word", "group": "adversarial", "query": "What time do I usually get home?",
+     "expect": [BIO], "forbid": [HOUSE]},
+    # "search" as ordinary English, not a web-search request.
+    {"id": "adv_search_word", "group": "adversarial",
+     "query": "No need to search anything, just tell me what 9 times 9 is.",
+     "expect": [CORE], "forbid": [LIB]},
+    # "create"/"write" are librarian domains but this is conversational.
+    {"id": "adv_write_word", "group": "adversarial", "query": "Can you write that more simply?",
+     "expect": [], "forbid": [LIB, HOUSE]},
+
+    # --- mutating intents: routing only, nothing is ever executed -------------
+    {"id": "mutate_wiki_update", "group": "mutating", "query": "Update the dossier page with today's findings.",
+     "expect": [LIB], "forbid": [HOUSE, CORE]},
+    {"id": "mutate_scene", "group": "mutating", "query": "Run the movie night scene.",
+     "expect": [HOUSE], "forbid": [LIB, BIO, CORE]},
+]
+
+
+GROUPS = sorted({f["group"] for f in FIXTURES})
+
+assert len({f["id"] for f in FIXTURES}) == len(FIXTURES), "duplicate fixture id"

scripts/ollama_residency.py

@@ -0,0 +1,121 @@
+"""
+Guard production's GPU residency across a benchmark run.
+
+Benchmarks swap models on the card production is serving from. Ollama evicts to
+make room, so a run leaves its own models resident and the production one gone:
+the next voice turn pays a ~36s cold load, and the pin that prevented it is
+silently lost. That happened on 2026-08-08 — a routing benchmark evicted
+gemma4:e2b and left gemma4:e4b behind, and only the monitoring noticing
+`unexpected_models` caught it.
+
+Snapshot before, restore after, and wire the restore to SIGTERM as well as the
+normal path. Python runs `finally` for SIGINT, which arrives as
+KeyboardInterrupt, but the default SIGTERM action terminates outright — so
+`timeout`, a systemd stop or a plain `kill` would skip the guard entirely.
+
+    from scripts.ollama_residency import residency_guard, install_sigterm_handler
+
+    install_sigterm_handler()
+    with residency_guard(models_used=["gemma4:e4b"]):
+        ...
+"""
+from __future__ import annotations
+
+import signal
+from collections.abc import Iterator
+from contextlib import contextmanager
+from datetime import UTC, datetime
+from typing import Any
+
+import httpx
+
+OLLAMA_URL = "http://localhost:11434"
+
+# keep_alive:-1 yields a year-2318 expiry, so "pinned" is simply "expires more
+# than a day out". Matches check-ai-pipeline.sh in system-admin-toj.
+PINNED_THRESHOLD_SECONDS = 86400
+
+
+def install_sigterm_handler() -> None:
+    """Make SIGTERM raise, so `finally` blocks and context managers still run."""
+    def _raise(signum, _frame):
+        raise KeyboardInterrupt(f"signal {signum}")
+
+    signal.signal(signal.SIGTERM, _raise)
+
+
+def snapshot_residency(client: httpx.Client | None = None) -> dict[str, bool]:
+    """Resident models mapped to whether each is pinned."""
+    owns = client is None
+    client = client or httpx.Client(timeout=30)
+    try:
+        data = client.get(f"{OLLAMA_URL}/api/ps", timeout=10).json()
+    except Exception:  # noqa: BLE001 - a missing snapshot must not abort the run
+        return {}
+    finally:
+        if owns:
+            client.close()
+
+    resident: dict[str, bool] = {}
+    now = datetime.now(UTC)
+    for model in data.get("models", []):
+        pinned = False
+        try:
+            expires = datetime.fromisoformat(model.get("expires_at", "").replace("Z", "+00:00"))
+            pinned = (expires - now).total_seconds() > PINNED_THRESHOLD_SECONDS
+        except ValueError:
+            pass
+        resident[model["name"]] = pinned
+    return resident
+
+
+def set_keep_alive(model: str, keep_alive: Any, client: httpx.Client | None = None) -> bool:
+    """Load, unload or pin a model. Embedding models reject /api/generate."""
+    owns = client is None
+    client = client or httpx.Client(timeout=180)
+    payload = {"model": model, "keep_alive": keep_alive}
+    try:
+        for endpoint in ("generate", "embed"):
+            try:
+                response = client.post(f"{OLLAMA_URL}/api/{endpoint}", json=payload, timeout=180)
+            except Exception:  # noqa: BLE001
+                return False
+            if response.status_code == 200:
+                return True
+            if response.status_code == 400 and "does not support generate" in response.text:
+                continue  # embedding-only model; try /api/embed
+            return False
+        return False
+    finally:
+        if owns:
+            client.close()
+
+
+def restore_residency(before: dict[str, bool], used: list[str]) -> None:
+    """Evict what the benchmark loaded, then re-pin what was pinned before."""
+    base = {name.split(":")[0] for name in before}
+    with httpx.Client(timeout=180) as client:
+        for model in used:
+            if model not in before and model.split(":")[0] not in base:
+                print(f"  residency: unloading benchmark model {model}")
+                set_keep_alive(model, 0, client)
+        for name, pinned in before.items():
+            if not pinned:
+                continue
+            ok = set_keep_alive(name, -1, client)
+            print(f"  residency: re-pinned {name}" if ok
+                  else f"  residency: FAILED to re-pin {name} -- run warmup-ollama.sh")
+
+
+@contextmanager
+def residency_guard(models_used: list[str]) -> Iterator[dict[str, bool]]:
+    """Snapshot residency on entry, restore it on exit however that happens."""
+    before = snapshot_residency()
+    pinned = [n for n, p in before.items() if p]
+    print(f"  residency: resident before {sorted(before)}"
+          f"{f' (pinned: {pinned})' if pinned else ''}")
+    try:
+        yield before
+    finally:
+        print("  residency: restoring ...")
+        restore_residency(before, models_used)

src/agents/steward/service.py

@@ -19,35 +19,88 @@ from .schemas import ConversationContext, StewardRecommendation
 logger = get_logger(__name__)
 
 
+_DELEGATE_LINE_RE = re.compile(r"^[ \t]*DELEGATE:[ \t]*(.+)$", re.IGNORECASE | re.MULTILINE)
+
+
+def _mentions(needle: str, haystack: str) -> bool:
+    """Whole-word containment. Substring matching is what made this go wrong."""
+    return re.search(rf"(?<!\w){re.escape(needle)}(?!\w)", haystack) is not None
+
+
 def _extract_capabilities(text: str) -> list[str]:
     """
-    Extract capability names from Steward's text response.
+    Extract capability names from the Steward's declared delegation.
 
-    Uses keyword matching to find mentioned capabilities.
+    The prompt instructs the Steward to answer in a fixed shape::
+
+        DELEGATE: <capability> to <action> <task>
+        REASON: ...
+        COMPLEXITY: ...
+        CONTEXT: ...
+
+    Only the DELEGATE line states intent; the rest is free prose. An earlier
+    version substring-matched capability *domains* across the whole response,
+    which routed on ordinary English: "description" contains "script" and
+    "discover" contains "cover" (both housekeeper domains), "acknowledge"
+    contains "knowledge" and "know" (librarian, biographer), and "economy"
+    contains "my" (biographer). Any REASON line could therefore summon agents
+    the Steward never asked for, and a spurious librarian is a real
+    multi-second web call.
+
+    It also made prose length a routing input, so anything that shortened the
+    Steward's output — such as disabling model thinking — would look like it had
+    improved routing.
+
+    Resolution is layered, most explicit first:
+      1. a DELEGATE line beginning with a capability name — the documented shape
+      2. a capability named anywhere on a DELEGATE line
+      3. a capability *domain* on a DELEGATE line, for a loosely worded answer
+      4. no DELEGATE line: capability names only, never domains
 
     Args:
         text: Steward's plain text analysis
 
     Returns:
-        List of capability names (e.g., ['tatlock_core'])
+        List of capability names (e.g. ['tatlock_core']), de-duplicated.
     """
-    text_lower = text.lower()
     registry = get_household_registry()
     capabilities = registry.get_all_capabilities()
+    delegate_lines = [line.strip().lower() for line in _DELEGATE_LINE_RE.findall(text or "")]
 
-    found_caps = []
+    found_caps: list[str] = []
 
-    for cap in capabilities:
-        # Check if capability name is mentioned
-        if cap.name.lower() in text_lower:
-            found_caps.append(cap.name)
+    def _add(name: str) -> None:
+        if name not in found_caps:
+            found_caps.append(name)
+
+    if not delegate_lines:
+        # Either the Steward judged no capability necessary — the prompt's
+        # conversational path, whose correct answer is [] — or it ignored the
+        # format. Names only: domain words are ordinary English and would fire
+        # on any prose, which is the bug described above.
+        haystack = (text or "").lower()
+        for cap in capabilities:
+            if _mentions(cap.name.lower(), haystack):
+                _add(cap.name)
+        return found_caps
+
+    for line in delegate_lines:
+        leading = next((c for c in capabilities if line.startswith(c.name.lower())), None)
+        if leading is not None:
+            _add(leading.name)
             continue
 
-        # Check if any domains are mentioned
-        for domain in cap.domains:
-            if domain.lower() in text_lower:
-                found_caps.append(cap.name)
-                break
+        named = [c for c in capabilities if _mentions(c.name.lower(), line)]
+        if named:
+            for cap in named:
+                _add(cap.name)
+            continue
+
+        # Last resort. Scoped to this line, so the REASON and CONTEXT prose that
+        # caused the original misrouting can no longer reach it.
+        for cap in capabilities:
+            if any(_mentions(domain.lower(), line) for domain in cap.domains):
+                _add(cap.name)
 
     return found_caps
 

tests/agents/steward/test_steward_service.py

@@ -8,7 +8,12 @@ from unittest.mock import AsyncMock, MagicMock, patch
 import pytest
 
 from src.agents.steward.schemas import ConversationContext, StewardRecommendation
-from src.agents.steward.service import analyze_request, format_steward_note, _build_enriched_query
+from src.agents.steward.service import (
+    _build_enriched_query,
+    _extract_capabilities,
+    analyze_request,
+    format_steward_note,
+)
 from src.core.startup import register_household_members
 
 
@@ -283,3 +288,89 @@ class TestBuildEnrichedQuery:
         result = _build_enriched_query(query, memory_context)
 
         assert result == query
+
+
+class TestExtractCapabilities:
+    """Capability extraction reads the declared DELEGATE line, not free prose.
+
+    The prompt tells the Steward to state its choice on a DELEGATE line and to
+    explain itself on REASON/COMPLEXITY/CONTEXT lines. An earlier version
+    substring-matched capability domains across the entire response, so ordinary
+    English in the explanation routed requests: "description" contains the
+    housekeeper domain "script", "acknowledge" contains "know". These tests pin
+    that the explanation can no longer influence routing.
+    """
+
+    # (prose, why it used to misroute)
+    SUBSTRING_TRAPS = [
+        ("The user wants a description of the algorithm.", "script -> housekeeper"),
+        ("I should discover what the answer is.", "cover -> housekeeper"),
+        ("That sounds fantastic, let me compute it.", "fan -> housekeeper"),
+        ("I acknowledge the request to add two numbers.", "knowledge/know -> librarian, biographer"),
+        ("The user asks about the economy myth.", "my -> biographer"),
+        ("Convert 98.6 Fahrenheit to Celsius.", "temperature is a housekeeper domain"),
+    ]
+
+    @pytest.mark.parametrize("prose,reason", SUBSTRING_TRAPS)
+    def test_reason_prose_cannot_add_capabilities(self, prose, reason):
+        """Explanatory prose must not summon agents the Steward did not request."""
+        text = f"DELEGATE: tatlock_core to calculate\nREASON: {prose}\nCOMPLEXITY: simple"
+
+        assert _extract_capabilities(text) == ["tatlock_core"], f"regression: {reason}"
+
+    def test_delegate_line_task_text_does_not_leak(self):
+        """A domain word inside the task description must not add a capability.
+
+        "home" is a housekeeper domain, but this is plainly a memory recall.
+        """
+        text = "DELEGATE: biographer to recall the user's home address\nREASON: personal data"
+
+        assert _extract_capabilities(text) == ["biographer"]
+
+    def test_multiple_delegate_lines(self):
+        """Each DELEGATE line contributes its capability, in order, deduplicated."""
+        text = (
+            "DELEGATE: biographer to recall the user's location\n"
+            "DELEGATE: librarian to search_web for the forecast\n"
+            "DELEGATE: biographer to recall preferences\n"
+        )
+
+        assert _extract_capabilities(text) == ["biographer", "librarian"]
+
+    def test_capability_named_later_on_the_line(self):
+        """A loosely worded DELEGATE line still resolves by name."""
+        text = "DELEGATE: ask the librarian to search the web"
+
+        assert _extract_capabilities(text) == ["librarian"]
+
+    def test_domain_fallback_within_delegate_line(self):
+        """With no capability named, domains on the DELEGATE line still resolve."""
+        text = "DELEGATE: turn on the lights in the kitchen"
+
+        assert _extract_capabilities(text) == ["housekeeper"]
+
+    def test_conversational_response_selects_nothing(self):
+        """No DELEGATE line means no capability, which is the prompt's chat path."""
+        text = "This is a simple greeting. No capabilities are needed. COMPLEXITY: simple"
+
+        assert _extract_capabilities(text) == []
+
+    def test_malformed_response_still_routes_by_name(self):
+        """If the format is ignored, a named capability is still honoured."""
+        text = "I think the librarian should handle this research request."
+
+        assert _extract_capabilities(text) == ["librarian"]
+
+    def test_malformed_response_does_not_route_on_domains(self):
+        """...but bare prose must not route on domain words alone."""
+        text = "The user wants a description of home automation, and I acknowledge it."
+
+        assert _extract_capabilities(text) == []
+
+    def test_case_insensitive_delegate_marker(self):
+        text = "delegate: Librarian to search_web"
+
+        assert _extract_capabilities(text) == ["librarian"]
+
+    def test_empty_input(self):
+        assert _extract_capabilities("") == []

tests/e2e/README.md

@@ -4,8 +4,8 @@ These tests make real HTTP requests to the running Tatlock API server to verify
 
 ## Prerequisites
 
-1. **Server must be running** on `http://localhost:8777` (use `./wakeup.sh`)
-2. **Ollama must be running** with `mistral-nemo:latest` model
+1. **Server must be running** on `http://localhost:8777` (use `make run`)
+2. **Ollama must be running** with the `gemma4:e2b` model
 3. **Redis must be running** (for benchmarking)
 4. **Qdrant must be running** on `http://localhost:6333` (for memory tests)
 
@@ -15,9 +15,9 @@ These tests make real HTTP requests to the running Tatlock API server to verify
 
 ```bash
 # Terminal 1: Start the server (auto-reload enabled)
-./wakeup.sh
+make run
 
-# Logs are written to logs/server.log - tail them in another terminal:
+# Logs are written to build/logs/server.log - tail them in another terminal:
 tail -f logs/server.log
 ```
 
@@ -132,7 +132,7 @@ memory = await qdrant.find_memory_by_key("memories_llm_tester", "favorite_color"
 
 Make sure the server is running:
 ```bash
-./wakeup.sh
+make run
 curl http://localhost:8777/health  # Should return 200
 ```
 

tests/e2e/test_orchestration_e2e.py

@@ -11,9 +11,9 @@ These tests hit the actual running server and verify data persistence.
 They use the `llm_tester` user for isolation from production data.
 
 Requirements:
-- Server running on localhost:8777 (use ./wakeup.sh)
+- Server running on localhost:8777 (use `make run`)
 - Qdrant running on localhost:6333
-- Ollama running with mistral-nemo model
+- Ollama running with the gemma4:e2b model
 
 Note: LLM outputs are non-deterministic. Tests use flexible assertions
 that check for behavioral patterns rather than exact text matches.