fix(steward): route on the declared DELEGATE line, not on prose

The prompt tells the Steward to state its choice on a DELEGATE line and
to explain itself on REASON, COMPLEXITY and CONTEXT lines. Extraction
ignored that structure and substring-matched capability domains across
the entire response, so ordinary English in the explanation selected
agents: "description" contains the housekeeper domain "script",
"discover" contains "cover", "acknowledge" contains "knowledge" and
"know", "economy" contains the biographer domain "my".

Every one of those was a real delegation. A spurious librarian is a
multi-second web call on a query that asked for arithmetic.

It also made prose length a routing input, which would have quietly
corrupted the thinking benchmark this was found during: anything that
shortened the Steward's output reduces accidental substring hits and so
reads as improved routing.

Resolution is now layered, most explicit first — a DELEGATE line opening
with a capability name, then a capability named anywhere on that line,
then a domain on that line. With no DELEGATE line at all the response is
matched on capability names only, never domains, so the conversational
path still answers with no capabilities. Matching is whole-word
throughout.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-08 14:37:59 +02:00
co-authored by Claude
parent 19e32cfbd6
commit a90536314e
3 changed files with 168 additions and 15 deletions
+67 -14
View File
@@ -19,35 +19,88 @@ from .schemas import ConversationContext, StewardRecommendation
logger = get_logger(__name__)
_DELEGATE_LINE_RE = re.compile(r"^[ \t]*DELEGATE:[ \t]*(.+)$", re.IGNORECASE | re.MULTILINE)
def _mentions(needle: str, haystack: str) -> bool:
"""Whole-word containment. Substring matching is what made this go wrong."""
return re.search(rf"(?<!\w){re.escape(needle)}(?!\w)", haystack) is not None
def _extract_capabilities(text: str) -> list[str]:
"""
Extract capability names from Steward's text response.
Extract capability names from the Steward's declared delegation.
Uses keyword matching to find mentioned capabilities.
The prompt instructs the Steward to answer in a fixed shape::
DELEGATE: <capability> to <action> <task>
REASON: ...
COMPLEXITY: ...
CONTEXT: ...
Only the DELEGATE line states intent; the rest is free prose. An earlier
version substring-matched capability *domains* across the whole response,
which routed on ordinary English: "description" contains "script" and
"discover" contains "cover" (both housekeeper domains), "acknowledge"
contains "knowledge" and "know" (librarian, biographer), and "economy"
contains "my" (biographer). Any REASON line could therefore summon agents
the Steward never asked for, and a spurious librarian is a real
multi-second web call.
It also made prose length a routing input, so anything that shortened the
Steward's output — such as disabling model thinking — would look like it had
improved routing.
Resolution is layered, most explicit first:
1. a DELEGATE line beginning with a capability name — the documented shape
2. a capability named anywhere on a DELEGATE line
3. a capability *domain* on a DELEGATE line, for a loosely worded answer
4. no DELEGATE line: capability names only, never domains
Args:
text: Steward's plain text analysis
Returns:
List of capability names (e.g., ['tatlock_core'])
List of capability names (e.g. ['tatlock_core']), de-duplicated.
"""
text_lower = text.lower()
registry = get_household_registry()
capabilities = registry.get_all_capabilities()
delegate_lines = [line.strip().lower() for line in _DELEGATE_LINE_RE.findall(text or "")]
found_caps = []
found_caps: list[str] = []
for cap in capabilities:
# Check if capability name is mentioned
if cap.name.lower() in text_lower:
found_caps.append(cap.name)
def _add(name: str) -> None:
if name not in found_caps:
found_caps.append(name)
if not delegate_lines:
# Either the Steward judged no capability necessary — the prompt's
# conversational path, whose correct answer is [] — or it ignored the
# format. Names only: domain words are ordinary English and would fire
# on any prose, which is the bug described above.
haystack = (text or "").lower()
for cap in capabilities:
if _mentions(cap.name.lower(), haystack):
_add(cap.name)
return found_caps
for line in delegate_lines:
leading = next((c for c in capabilities if line.startswith(c.name.lower())), None)
if leading is not None:
_add(leading.name)
continue
# Check if any domains are mentioned
for domain in cap.domains:
if domain.lower() in text_lower:
found_caps.append(cap.name)
break
named = [c for c in capabilities if _mentions(c.name.lower(), line)]
if named:
for cap in named:
_add(cap.name)
continue
# Last resort. Scoped to this line, so the REASON and CONTEXT prose that
# caused the original misrouting can no longer reach it.
for cap in capabilities:
if any(_mentions(domain.lower(), line) for domain in cap.domains):
_add(cap.name)
return found_caps