chore: release v1.9.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

AGENTS.md

@@ -22,6 +22,11 @@ This document contains instructions and documentation references for AI assistan
 * **Test REST endpoints** against `http://localhost:8777` using curl or similar tools
 * **Only deploy** when a phase or feature is complete and tested locally
 * **Environment**: Copy `.env.example` to `.env` and configure for your local setup (Ollama, Redis, Qdrant hosts)
+* **Running tests**: Always use the venv explicitly to avoid environment mismatches:
+  ```bash
+  .venv/bin/python -m pytest tests/           # All tests
+  .venv/bin/python -m pytest tests/core/ -v   # Core tests only
+  ```
 
 ### 🌐 Internal Service Access
 * **git.schweitz.net**: Access via `http://localhost:3002` (direct Gitea) to bypass Authentik SSO
@@ -51,6 +56,32 @@ This document contains instructions and documentation references for AI assistan
 * **Update `CHANGELOG.md`** with every user-facing change.
 * Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
 
+### 🚀 Release Flow
+When changes are ready for deployment:
+
+1. **Ask user if deploy cycle is desired**
+
+2. **Update version** in `pyproject.toml`:
+   - Bug fixes: bump patch version (1.8.3 → 1.8.4)
+   - New features: bump minor version (1.8.4 → 1.9.0)
+
+3. **Update CHANGELOG.md**:
+   - Move items from `[Unreleased]` to new version section
+   - Add release date: `## [1.8.4] - 2025-12-16`
+
+4. **Commit and tag**:
+   ```bash
+   git add -A
+   git commit -m "fix: description of changes"
+   git tag v1.8.4
+   git push origin main --tags
+   ```
+
+5. **CI/CD triggers automatically**:
+   - Gitea CI builds Docker image on new tag
+   - Watchtower pulls and deploys to production
+   - Verify deployment: `curl http://192.168.86.149:8000/health`
+
 ---
 
 ## 2. FastAPI Architecture & Best Practices

CHANGELOG.md

@@ -7,6 +7,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.9.0] - 2025-12-18
+
+### Changed
+
+- **Housekeeper prompt optimization** - Rewrote system prompt for Mistral-Nemo function calling with negative constraints, step-by-step process, and explicit entity ID format guidance
+- **Housekeeper temperature setting** - Set temperature to 0.1 for deterministic tool calling behavior
+- **Device list room group priority** - Room groups now appear first in `list_devices` output with `[ROOM GROUP]` marker to address positional bias
+- **Tool docstring improvements** - Updated turn_on/turn_off/toggle with explicit `entity_id=` parameter examples
+
+### Added
+
+- **Housekeeper optimization findings** - Added `docs/housekeeper-optimization-findings.md` documenting the experiment journey from 0% to 100% success rate
+- **Housekeeper test script** - Added `scripts/test_housekeeper.sh` for room group detection regression testing
+
+## [1.8.6] - 2025-12-17
+
+### Fixed
+
+- **Housekeeper API paths** - Updated all client endpoints to use `/housekeeping/` prefix to match core-api routes
+- **Housekeeper entity hallucination** - Improved system prompt with critical rule requiring `list_devices()` before any control action to prevent guessing entity IDs
+
+### Added
+
+- **Housekeeping API spec** - Added `docs/housekeeping-api-spec.md` documenting the core-api home automation interface
+
+## [1.8.5] - 2025-12-16
+
+### Fixed
+
+- **Redis benchmark boolean storage** - Convert booleans to strings for Redis `hset` (Redis doesn't accept bool type directly)
+- **Tool tracking capability matching** - `delegate_to_librarian` now correctly recognized as using "librarian" capability when checking Steward recommendations
+- **E2E test fixture scope** - Fixed pytest-asyncio ScopeMismatch error by using `loop_scope="module"` for module-scoped async fixtures
+
+## [1.8.4] - 2025-12-16
+
+### Fixed
+
+- **Remove `<think>` wrappers from think messages** - Messages in `reasoning_content` should be plain text
+- Removed `<think>` wrappers from delegation.py household think messages
+- Removed `<think>` wrappers from orchestration.py status messages
+- Think messages now appear cleanly in Open WebUI's reasoning block
+
+## [1.8.3] - 2025-12-16
+
+### Fixed
+
+- **Open WebUI streaming rendering** - Use `reasoning_content` field for thinking (DeepSeek R1 format) instead of `<think>` tags in `content`
+- Open WebUI now renders thinking as proper collapsible blocks instead of broken HTML
+
+## [1.8.2] - 2025-12-16
+
+### Fixed
+
+- **HybridRAG keywords schema mismatch** - library-desk now returns `keywords` as dict with `core_keywords`, client now handles both formats
+
 ## [1.8.1] - 2025-12-16
 
 ### Fixed

docs/housekeeper-optimization-findings.md

@@ -0,0 +1,246 @@
+# Housekeeper Agent Optimization Findings
+
+## Background
+
+Research with Gemini identified key issues with mistral-nemo and tool calling:
+- "Pre-computation Hallucination" - model answers before using tools
+- High default temperature (0.7-0.8) causes wandering
+- Model is "chatty and confident" - needs explicit constraints
+
+## Key Recommendations from Gemini Research
+
+1. **Temperature 0.0** for tool-calling agents (deterministic, follows schema)
+2. **Chain of Thought (CoT)** - force step-by-step reasoning
+3. **Negative constraints** - tell model what NOT to do (Nemo responds better)
+4. **Explicit tool descriptions** - verbose docstrings with "never estimate yourself"
+5. **"Strictly tool-based assistant"** pattern - NO internal knowledge claim
+
+---
+
+## Experiment Log
+
+### Baseline (v1.8.6)
+- **Date**: 2025-12-17
+- **Configuration**: Default temperature, improved prompt requiring list_devices first
+- **Results**:
+  - Called list_devices first ✓
+  - Still hallucinated `light.study_desk` despite seeing list with only `light.study` and `light.study_main`
+  - Partial success: turned off `light.study_main`, failed on hallucinated entity
+- **Success rate**: ~50% (1 of 2 study lights controlled correctly)
+
+---
+
+### Experiment 1: Temperature 0.0
+- **Date**: 2025-12-18
+- **Change**: Set `model_settings=ModelSettings(temperature=0.0)` for Housekeeper
+- **Hypothesis**: Deterministic output will force model to use exact entity IDs from tool results
+- **Results**:
+
+  **Study lights test:**
+  - Called `list_devices()` first ✓ (but no domain filter)
+  - Used wrong parameter `device_id` instead of `entity_id` (recovered after validation error)
+  - Only identified `light.studeerlamp` as "study" related (Dutch name)
+  - **Missed `light.study` and `light.study_main`** - didn't match English "study"
+  - Turned off 1 wrong light, missed 2 actual study lights
+
+  **Kitchen lights test:**
+  - Called `list_devices()` first ✓ (no domain filter)
+  - Saw full device list including `light.kitchen`
+  - Used wrong parameter `device_id` instead of `entity_id` (recovered after validation)
+  - After correction, dropped domain prefix: used `kitchen` instead of `light.kitchen`
+  - 404 error - device not found
+
+- **Success rate**: 0% (no target lights successfully controlled)
+- **Observations**:
+  - Temperature 0.0 alone is insufficient
+  - Model consistently confuses `device_id` vs `entity_id` parameter name
+  - After validation error correction, model truncates entity_id (drops domain prefix)
+  - Semantic matching of room names to devices is weak
+  - Model doesn't understand entity_id format: `domain.name`
+
+---
+
+### Experiment 2: Negative Constraints + CoT
+- **Date**: 2025-12-18
+- **Change**: Complete prompt rewrite with:
+  - "You have NO Internal Knowledge" - negative framing
+  - Explicit entity_id format with WRONG/RIGHT examples
+  - Step-by-step process (ALWAYS FOLLOW)
+  - Explicit parameter names section
+  - "What NOT To Do" negative constraints
+- **Hypothesis**: Negative constraints work better with Mistral-Nemo
+- **Results**:
+
+  **Study lights test:**
+  - Called `list_devices(domain="light")` ✓ with domain filter (improvement!)
+  - Still used `device_id` first, recovered to `entity_id` after validation error
+  - After recovery, used correct full format: `light.studeerlamp`
+  - **Still only matched `studeerlamp` not `light.study` or `light.study_main`**
+
+  **Kitchen lights test:**
+  - Called `list_devices(domain="light")` ✓
+  - Called `turn_off(entity_id="light.kitchen")` ✓ correct format!
+  - All 4 kitchen lights turned off (light.kitchen is a group)
+  - **100% success for kitchen!**
+
+- **Success rate**:
+  - Study: 0% (wrong semantic match)
+  - Kitchen: 100% (4/4 lights off)
+  - Combined: ~50% (1 of 2 tests successful)
+- **Observations**:
+  - Domain filter now consistently used ✓
+  - Entity_id format correct after recovery ✓
+  - Semantic matching still fails for "study" → prefers Dutch "studeerlamp" over English "study"
+  - Parameter name confusion persists (`device_id` vs `entity_id`)
+  - Simple room names (kitchen) work; mixed language fails (study/studeerlamp)
+
+---
+
+### Experiment 3: Temperature 0.1 + Explicit Tool Docstrings
+- **Date**: 2025-12-18
+- **Change**:
+  - Temperature 0.1
+  - Updated turn_on/turn_off docstrings with explicit `entity_id=` in examples
+- **Results**:
+  - Still uses `device_id` first, recovers to `entity_id` after validation
+  - Still picks wrong entity (studeerlamp over study)
+- **Success rate**: 0%
+
+---
+
+### Experiment 4: Room Group Priority (with explicit examples)
+- **Date**: 2025-12-18
+- **Change**: Updated prompt with:
+  - Explicit instruction: "Look for EXACT match `light.<room_name>` first!"
+  - Concrete examples: "For 'study lights' → look for `light.study`"
+  - Working example showing `turn_off(entity_id="light.study")`
+- **Hypothesis**: Explicit examples will guide model to use room groups
+- **Results**:
+
+  **Test 1 & 2 (consecutive):**
+  - Called `list_devices(domain="light")` ✓
+  - Device list clearly shows `light.study` at the bottom
+  - First call: `turn_off({"devices":["studeerlamp"]})` - wrong param AND wrong device
+  - After validation error: `turn_off(entity_id="light.studeerlamp")` - correct param, still wrong device
+  - **Completely ignored `light.study` despite prompt explicitly saying to use it**
+
+- **Success rate**: 0% (wrong device controlled)
+- **Observations**:
+  - Model ignores explicit step-by-step instructions in favor of substring matching
+  - Dutch "studeerlamp" contains "studer" which the model prefers over exact "study" match
+  - Even when prompt has a literal example `turn_off(entity_id="light.study")`, model uses `light.studeerlamp`
+  - Positional bias possible - `light.study` appears at end of 21-item list
+  - **Fundamental limitation**: Mistral-Nemo cannot follow explicit matching rules
+
+---
+
+### Experiment 5: Room Groups First (Tool Output Ordering)
+- **Date**: 2025-12-18
+- **Change**: Modified `list_devices` to sort room groups to top of list using HA attributes (`is_hue_group`, `hue_type="room"`)
+- **Hypothesis**: Positional bias - model focuses on items earlier in list
+- **Results**:
+  - Room groups (`light.study`, `light.kitchen`, etc.) now appear first in device list
+  - Combined with improved prompt, model now consistently uses room groups
+  - **70% success rate** (7/10 tests) with default q4 quantization
+
+---
+
+### Experiment 6: Model Quantization (q5_1)
+- **Date**: 2025-12-18
+- **Change**: Upgraded from default Mistral-Nemo quantization (q4) to `mistral-nemo:12b-instruct-2407-q5_1`
+- **Hypothesis**: Higher precision weights improve tool calling accuracy
+- **Results**:
+
+  | Test | Action | Result |
+  |------|--------|--------|
+  | 1 | Turn off study | PASS |
+  | 2 | Turn on study | PASS |
+  | 3 | Toggle study | PASS |
+  | 4 | Turn off kitchen | PASS |
+  | 5 | Turn on kitchen | PASS |
+  | 6 | Toggle kitchen | PASS |
+  | 7 | Turn off bedroom | PASS |
+  | 8 | Turn on bedroom | PASS |
+  | 9 | Turn off living room | PASS |
+  | 10 | Turn on living room | PASS |
+
+- **Success rate**: **100%** (10/10 tests)
+- **Observations**:
+  - q5_1 quantization dramatically improves tool calling accuracy
+  - All room groups correctly identified and used
+  - No parameter confusion (`entity_id` used correctly)
+  - No entity_id truncation issues
+  - Toggle operations now work reliably
+  - Model fits within 10GB VRAM (q6 did not)
+
+---
+
+### Experiment 7: Device List in System Prompt (Context Injection)
+- **Date**: [PENDING]
+- **Change**: Store device list in database (per user/household) and inject into system prompt
+- **Approach**:
+  1. Periodically sync device list from Home Assistant to PostgreSQL
+  2. On each Housekeeper invocation, fetch device list and include in prompt
+  3. Remove need for model to call list_devices() - just match from context
+- **Hypothesis**:
+  - Eliminates tool call step where errors occur
+  - Reduces context size by not returning full device list as tool output
+  - Makes entity matching a language task (in prompt) rather than tool result parsing
+- **Trade-offs**:
+  - Stale data if sync is infrequent
+  - Prompt size increase (but less than tool call response)
+  - Need sync mechanism and storage
+- **Results**: [TO BE RECORDED]
+- **Success rate**: [TO BE RECORDED]
+
+---
+
+## Key Problem Identified (Solved)
+
+The model struggled with:
+1. **Parameter schema adherence** - uses `device_id` when schema requires `entity_id`
+2. **Value preservation** - truncates values after validation errors (drops `light.` prefix)
+3. **Semantic matching** - prefers substring matches ("studeerlamp" contains "studer") over exact matches (`light.study`)
+4. **Following explicit instructions** - ignores step-by-step processes even when examples are provided
+5. **Positional bias** - may not "see" items at the end of long lists
+
+**Solution**: These issues were resolved by:
+1. Using q5_1 quantization instead of default q4 (higher precision weights)
+2. Sorting room groups to top of device list (address positional bias)
+3. Explicit prompt guidance with negative constraints and examples
+
+---
+
+## Potential Next Experiments
+
+### Experiment 5: Room Groups First (List Ordering)
+- **Hypothesis**: Positional bias - model focuses on items earlier in list
+- **Change**: Sort device list to put room groups (entities matching `light.<single_word>`) at the TOP
+- **Effort**: Low - modify list_devices output formatting
+- **Risk**: May affect other use cases where individual devices are needed
+
+### Experiment 6: Simplified Device List Format
+- **Hypothesis**: Markdown formatting adds noise that confuses the model
+- **Change**: Return simple list: `light.study (Study - GROUP), light.study_main (Ceiling light), ...`
+- **Effort**: Low - modify list_devices output
+- **Risk**: Less human-readable responses
+
+---
+
+## Learnings to Apply Elsewhere
+
+1. **Quantization matters** - q5_1 dramatically outperforms q4 for tool calling (100% vs 70%)
+2. **Positional bias is real** - sort important items to top of lists
+3. **Smaller models need simpler workflows** - fewer tool calls, more context injection
+4. **Validation errors don't teach** - model often makes worse mistakes on retry
+5. **Entity IDs are hard** - domain.name format confuses the model
+6. **Consider pre-computation** - move matching logic to code, not LLM
+7. **Use explicit negative constraints** - "NEVER do X" works better than "always do Y"
+
+---
+
+## Notes
+
+- Librarian may need higher temperature for creative synthesis
+- All "action" agents (Housekeeper, future agents) should use low temperature
+- Consider testing with Gemma 2 9B for better function calling (Google, open weights)

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "tatlock"
-version = "1.8.1"
+version = "1.9.0"
 description = "OpenAI-compatible API with Ollama backend"
 requires-python = ">=3.12"
 dependencies = []

scripts/test_housekeeper.sh

@@ -0,0 +1,141 @@
+#!/bin/bash
+# Housekeeper Room Group Detection Test Suite
+# Verifies room groups are controlled by checking actual state changes
+
+API_URL="http://localhost:8777/v1/chat/completions"
+CORE_API="http://192.168.86.149:8083"
+RESULTS_FILE="/tmp/housekeeper_test_results.txt"
+
+GREEN='\033[0;32m'
+RED='\033[0;31m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+get_state() {
+    curl -s "$CORE_API/housekeeping/devices/$1" 2>/dev/null | jq -r '.state' 2>/dev/null
+}
+
+echo "=========================================="
+echo "Housekeeper Room Group Test Suite"
+echo "=========================================="
+echo ""
+
+> "$RESULTS_FILE"
+
+run_toggle_test() {
+    local test_num=$1
+    local room=$2
+    local entity="light.$room"
+    local prompt_room="${room//_/ }"
+
+    printf "Test %2d: Toggle %-12s lights ... " "$test_num" "$prompt_room"
+
+    local before=$(get_state "$entity")
+    if [ -z "$before" ] || [ "$before" = "null" ]; then
+        echo -e "${YELLOW}SKIP${NC} (cannot get state)"
+        echo "SKIP|$test_num|Toggle $room|error" >> "$RESULTS_FILE"
+        return
+    fi
+
+    curl -s -X POST "$API_URL" \
+        -H "Content-Type: application/json" \
+        -d "{\"model\": \"tatlock\", \"messages\": [{\"role\": \"user\", \"content\": \"Toggle the $prompt_room lights\"}]}" > /dev/null
+
+    sleep 4
+
+    local after=$(get_state "$entity")
+
+    if [ "$before" != "$after" ]; then
+        echo -e "${GREEN}PASS${NC} ($before -> $after)"
+        echo "PASS|$test_num|Toggle $room|$before->$after" >> "$RESULTS_FILE"
+    else
+        echo -e "${RED}FAIL${NC} (state unchanged: $before)"
+        echo "FAIL|$test_num|Toggle $room|unchanged:$before" >> "$RESULTS_FILE"
+    fi
+}
+
+run_onoff_test() {
+    local test_num=$1
+    local room=$2
+    local action=$3
+    local expected_state=$4
+    # Entity uses underscore, prompt uses space
+    local entity="light.${room//_/ }"
+    entity="light.$room"
+    local prompt_room="${room//_/ }"
+
+    printf "Test %2d: %-8s %-12s lights ... " "$test_num" "$action" "$prompt_room"
+
+    curl -s -X POST "$API_URL" \
+        -H "Content-Type: application/json" \
+        -d "{\"model\": \"tatlock\", \"messages\": [{\"role\": \"user\", \"content\": \"$action the $prompt_room lights\"}]}" > /dev/null
+
+    sleep 4
+
+    local after=$(get_state "$entity")
+
+    if [ "$after" = "$expected_state" ]; then
+        echo -e "${GREEN}PASS${NC} ($after)"
+        echo "PASS|$test_num|$action $room|$after" >> "$RESULTS_FILE"
+    else
+        echo -e "${RED}FAIL${NC} (got $after, expected $expected_state)"
+        echo "FAIL|$test_num|$action $room|got:$after,expected:$expected_state" >> "$RESULTS_FILE"
+    fi
+}
+
+echo "Running tests (~4s each)..."
+echo ""
+
+# Study tests
+run_onoff_test 1 "study" "Turn off" "off"
+run_onoff_test 2 "study" "Turn on" "on"
+run_toggle_test 3 "study"
+
+# Kitchen tests
+run_onoff_test 4 "kitchen" "Turn off" "off"
+run_onoff_test 5 "kitchen" "Turn on" "on"
+run_toggle_test 6 "kitchen"
+
+# Bedroom tests
+run_onoff_test 7 "bedroom" "Turn off" "off"
+run_onoff_test 8 "bedroom" "Turn on" "on"
+
+# Living room tests (entity is light.living_room)
+run_onoff_test 9 "living_room" "Turn off" "off"
+run_onoff_test 10 "living_room" "Turn on" "on"
+
+# Ensure all lights end up ON
+echo ""
+echo "Restoring all lights to ON..."
+for room in "study" "kitchen" "bedroom" "living room"; do
+    curl -s -X POST "$API_URL" \
+        -H "Content-Type: application/json" \
+        -d "{\"model\": \"tatlock\", \"messages\": [{\"role\": \"user\", \"content\": \"Turn on the $room lights\"}]}" > /dev/null
+    sleep 3
+done
+echo "Done."
+
+echo ""
+echo "=========================================="
+echo "Results"
+echo "=========================================="
+
+PASS=$(grep -c "^PASS" "$RESULTS_FILE" 2>/dev/null || echo 0)
+FAIL=$(grep -c "^FAIL" "$RESULTS_FILE" 2>/dev/null || echo 0)
+SKIP=$(grep -c "^SKIP" "$RESULTS_FILE" 2>/dev/null || echo 0)
+TOTAL=$((PASS + FAIL))
+
+echo "Passed:  $PASS"
+echo "Failed:  $FAIL"
+echo "Skipped: $SKIP"
+
+if [ "$TOTAL" -gt 0 ]; then
+    echo ""
+    echo "Success Rate: $((PASS * 100 / TOTAL))% ($PASS/$TOTAL)"
+fi
+
+if [ "$FAIL" -gt 0 ]; then
+    echo ""
+    echo "Failures:"
+    grep "^FAIL" "$RESULTS_FILE"
+fi

src/agents/delegation.py

@@ -40,45 +40,46 @@ class ActionType(Enum):
 # =============================================================================
 
 HOUSEHOLD_THINK_MESSAGES: dict[str, dict[ActionType, dict[str, str]]] = {
+    # Note: No <think> wrappers needed - these go to reasoning_content field
     "librarian": {
         ActionType.RETRIEVE: {
-            "start": "<think>Allow me to consult the archives, sir.</think>",
-            "success": "<think>The Librarian has compiled the relevant findings.</think>",
-            "error": "<think>I'm afraid the archives proved difficult to access.</think>",
+            "start": "Allow me to consult the archives, sir.",
+            "success": "The Librarian has compiled the relevant findings.",
+            "error": "I'm afraid the archives proved difficult to access.",
         },
         ActionType.RESEARCH: {
-            "start": "<think>I've dispatched the Librarian to conduct some fresh research.</think>",
-            "success": "<think>The Librarian has returned with findings, sir.</think>",
-            "error": "<think>The research proved inconclusive, I'm afraid.</think>",
+            "start": "I've dispatched the Librarian to conduct some fresh research.",
+            "success": "The Librarian has returned with findings, sir.",
+            "error": "The research proved inconclusive, I'm afraid.",
         },
         ActionType.CREATE: {
-            "start": "<think>I'm having the Librarian prepare a new entry.</think>",
-            "success": "<think>The new material has been properly catalogued, sir.</think>",
-            "error": "<think>I'm afraid there was difficulty filing the entry.</think>",
+            "start": "I'm having the Librarian prepare a new entry.",
+            "success": "The new material has been properly catalogued, sir.",
+            "error": "I'm afraid there was difficulty filing the entry.",
         },
     },
     "biographer": {
         ActionType.RETRIEVE: {
-            "start": "<think>Let me consult the household records.</think>",
-            "success": "<think>The Biographer has located the relevant information, sir.</think>",
-            "error": "<think>I'm unable to locate those particular records.</think>",
+            "start": "Let me consult the household records.",
+            "success": "The Biographer has located the relevant information, sir.",
+            "error": "I'm unable to locate those particular records.",
         },
         ActionType.RECORD: {
-            "start": "<think>I've asked the Biographer to take note of this, sir.</think>",
-            "success": "<think>The household records have been updated accordingly.</think>",
-            "error": "<think>I'm afraid there was difficulty recording the entry.</think>",
+            "start": "I've asked the Biographer to take note of this, sir.",
+            "success": "The household records have been updated accordingly.",
+            "error": "I'm afraid there was difficulty recording the entry.",
         },
     },
     "housekeeper": {
         ActionType.RETRIEVE: {
-            "start": "<think>Allow me to inquire with the household staff.</think>",
-            "success": "<think>The staff reports the current status, sir.</think>",
-            "error": "<think>The household staff is momentarily unavailable, I'm afraid.</think>",
+            "start": "Allow me to inquire with the household staff.",
+            "success": "The staff reports the current status, sir.",
+            "error": "The household staff is momentarily unavailable, I'm afraid.",
         },
         ActionType.CONTROL: {
-            "start": "<think>I'm instructing the household staff now, sir.</think>",
-            "success": "<think>The household has been configured as requested.</think>",
-            "error": "<think>I'm afraid the staff reports an issue with that request.</think>",
+            "start": "I'm instructing the household staff now, sir.",
+            "success": "The household has been configured as requested.",
+            "error": "I'm afraid the staff reports an issue with that request.",
         },
     },
 }
@@ -139,7 +140,7 @@ def get_think_message(expert: str, task: str, phase: str) -> str:
     action_type = _detect_action_type(expert, task)
     expert_messages = HOUSEHOLD_THINK_MESSAGES.get(expert, {})
     action_messages = expert_messages.get(action_type, expert_messages.get(ActionType.RETRIEVE, {}))
-    return action_messages.get(phase, f"<think>Consulting {expert}...</think>")
+    return action_messages.get(phase, f"Consulting {expert}...")
 
 
 @dataclass

src/agents/housekeeper/agent.py

@@ -32,79 +32,69 @@ from src.core.logging_config import get_logger
 
 logger = get_logger(__name__)
 
-# Housekeeper system prompt
-HOUSEKEEPER_SYSTEM_PROMPT = """You are The Housekeeper, an expert home automation assistant in the Tatlock household.
+# Housekeeper system prompt - Optimized for Mistral-Nemo function calling
+HOUSEKEEPER_SYSTEM_PROMPT = """You are a strictly tool-based home automation assistant.
 
-Your role is to help users control and monitor their smart home through Home Assistant:
-- Lights, switches, and other devices
-- Scenes (pre-configured device states)
-- Scripts (automation sequences)
-- Automations (event-triggered rules)
+## CRITICAL: You Have NO Internal Knowledge
 
-## Your Personality
-- Efficient and practical
-- Safety-conscious (confirm destructive actions)
-- Proactive in suggesting optimizations
-- Clear about what actions you're taking
+You do NOT know what devices exist. You do NOT know any entity IDs.
+Entity IDs are different in every installation. You MUST discover them using tools.
 
-## Your Tools
+## Entity ID Format
 
-### Discovery Tools
-- **list_areas**: See all rooms/areas configured in Home Assistant
-- **list_devices**: Find devices by type (domain) or location (area)
-- **get_device_state**: Check a device's current state and attributes
+Entity IDs follow the format: `domain.name`
+Examples: `light.kitchen`, `light.study_main`, `switch.coffee_maker`
 
-### Control Tools
-- **turn_on**: Turn on lights, switches, etc. (supports brightness/color for lights)
-- **turn_off**: Turn off devices
-- **toggle**: Flip a device's state
+The `entity_id` parameter MUST be the COMPLETE value including the domain prefix.
+WRONG: `entity_id="kitchen"`
+RIGHT: `entity_id="light.kitchen"`
 
-### Scene Tools
-- **list_scenes**: See available scene presets
-- **activate_scene**: Activate a scene (e.g., "movie night", "good morning")
+## Step-by-Step Process (ALWAYS FOLLOW)
 
-### Script Tools
-- **list_scripts**: See available automation scripts
-- **run_script**: Execute a script
+When asked to control devices in a room:
 
-### Automation Tools
-- **list_automations**: See all automations and their status
-- **toggle_automation**: Enable or disable an automation
+1. THINK: What domain? (light, switch, climate, etc.)
+2. CALL: list_devices(domain="light") to discover available devices
+3. CHECK: Look for EXACT match `light.<room_name>` first!
+   - For "study lights" → look for `light.study` (not light.study_main, not light.studeerlamp)
+   - For "kitchen lights" → look for `light.kitchen` (not light.kitchen_spot_1)
+   - These room groups control ALL lights in that room at once
+   - If found, use ONLY the group (stop looking for individual lights)
+4. FALLBACK: Only if no exact room group exists, find entity_ids containing the room name
+5. CALL: turn_on/turn_off using the EXACT entity_id from step 3 or 4
 
-### History Tools
-- **get_history**: Check a device's state history
+Example for "Turn off study lights":
+1. Domain is "light"
+2. Call list_devices(domain="light")
+3. Look for room group: `light.study` - FOUND!
+4. Call turn_off(entity_id="light.study")  # This controls all study lights
 
-## Best Practices
+Example for "Turn off hallway lights" (no room group):
+1. Domain is "light"
+2. Call list_devices(domain="light")
+3. Look for room group: `light.hallway` - NOT FOUND
+4. Find all with "hallway": light.hallway_spot_1, light.hallway_spot_2
+5. Call turn_off for each
 
-1. **Device Discovery First**: If the user asks about devices without being specific,
-   use list_devices to find what's available before acting.
+## Tool Parameter Names
 
-2. **Confirm State After Actions**: After turning something on/off, you can verify
-   with get_device_state if needed.
+- turn_on, turn_off, toggle: Use `entity_id` (NOT device_id, NOT id)
+- activate_scene: Use `scene_id`
+- run_script: Use `script_id`
 
-3. **Use Entity IDs**: Devices are identified by entity_id (e.g., light.living_room).
-   Always use the exact entity_id from list_devices.
+## What NOT To Do
 
-4. **Area-Aware**: When users say "living room lights", filter by area="living_room".
-
-5. **Safety**: For actions affecting multiple devices or automations, summarize
-   what you're about to do.
-
-## Common Patterns
-
-- "Turn on the lights" → list_devices(domain="light"), then turn_on each
-- "What's on?" → list_devices() and filter for state="on"
-- "Movie time" → Either activate_scene("scene.movie_night") or run_script if available
-- "Dim the bedroom" → turn_on("light.bedroom", brightness=64)
+- NEVER guess an entity_id
+- NEVER construct an entity_id from the room name
+- NEVER drop the domain prefix (light., switch., etc.)
+- NEVER use "device_id" - the parameter is called "entity_id"
+- NEVER provide an answer without calling list_devices first
 
 ## Response Format
-Your responses are returned to Tatlock (the butler) who will synthesize them into
-a final answer for the user. Keep this in mind:
-- Lead with confirmation of what you did or found
-- Be specific about which devices were affected
-- Include relevant state information
-- Note any issues or failures
-- Be concise - Tatlock will format the final response
+
+After completing actions, briefly confirm:
+- Which devices were affected (list the entity_ids)
+- Whether each action succeeded or failed
 """
 
 # Lazy initialization to avoid connection issues during imports
@@ -217,9 +207,13 @@ async def run_housekeeper(
     )
 
     try:
+        # Use temperature 0.1 for slight exploration
+        from pydantic_ai.settings import ModelSettings
+
         result = await agent.run(
             prompt,
             message_history=message_history,
+            model_settings=ModelSettings(temperature=0.1),
         )
 
         logger.info(
@@ -275,9 +269,13 @@ async def run_housekeeper_stream(
     )
 
     try:
+        # Use temperature 0.1 for slight exploration
+        from pydantic_ai.settings import ModelSettings
+
         async with agent.run_stream(
             prompt,
             message_history=message_history,
+            model_settings=ModelSettings(temperature=0.1),
         ) as response:
             async for delta in response.stream_text(delta=True):
                 yield delta

src/agents/housekeeper/client.py

@@ -182,7 +182,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_list_devices", domain=domain, area=area)
 
-        response = await client.get("/devices", params=params or None)
+        response = await client.get("/housekeeping/devices", params=params or None)
         response.raise_for_status()
 
         data = response.json()
@@ -199,7 +199,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_list_areas")
 
-        response = await client.get("/areas")
+        response = await client.get("/housekeeping/areas")
         response.raise_for_status()
 
         data = response.json()
@@ -219,7 +219,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_get_state", entity_id=entity_id)
 
-        response = await client.get(f"/entities/{entity_id}")
+        response = await client.get(f"/housekeeping/devices/{entity_id}")
         response.raise_for_status()
 
         return DeviceState(**response.json())
@@ -260,7 +260,7 @@ class CoreAPIClient:
         logger.info("core_api_turn_on", entity_id=entity_id, payload=payload)
 
         response = await client.post(
-            f"/devices/{entity_id}/control",
+            f"/housekeeping/devices/{entity_id}/control",
             json=payload,
         )
         response.raise_for_status()
@@ -288,7 +288,7 @@ class CoreAPIClient:
         logger.info("core_api_turn_off", entity_id=entity_id)
 
         response = await client.post(
-            f"/devices/{entity_id}/control",
+            f"/housekeeping/devices/{entity_id}/control",
             json={"action": "turn_off"},
         )
         response.raise_for_status()
@@ -316,7 +316,7 @@ class CoreAPIClient:
         logger.info("core_api_toggle", entity_id=entity_id)
 
         response = await client.post(
-            f"/devices/{entity_id}/control",
+            f"/housekeeping/devices/{entity_id}/control",
             json={"action": "toggle"},
         )
         response.raise_for_status()
@@ -344,7 +344,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_list_scenes")
 
-        response = await client.get("/scenes")
+        response = await client.get("/housekeeping/scenes")
         response.raise_for_status()
 
         data = response.json()
@@ -364,7 +364,7 @@ class CoreAPIClient:
 
         logger.info("core_api_activate_scene", scene_id=scene_id)
 
-        response = await client.post(f"/scenes/{scene_id}/activate")
+        response = await client.post(f"/housekeeping/scenes/{scene_id}/activate")
         response.raise_for_status()
 
         data = response.json()
@@ -390,7 +390,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_list_scripts")
 
-        response = await client.get("/scripts")
+        response = await client.get("/housekeeping/scripts")
         response.raise_for_status()
 
         data = response.json()
@@ -420,7 +420,7 @@ class CoreAPIClient:
         logger.info("core_api_run_script", script_id=script_id)
 
         response = await client.post(
-            f"/scripts/{script_id}/run",
+            f"/housekeeping/scripts/{script_id}/run",
             json=payload or None,
         )
         response.raise_for_status()
@@ -448,7 +448,7 @@ class CoreAPIClient:
 
         logger.debug("core_api_list_automations")
 
-        response = await client.get("/automations")
+        response = await client.get("/housekeeping/automations")
         response.raise_for_status()
 
         data = response.json()
@@ -478,7 +478,7 @@ class CoreAPIClient:
         )
 
         response = await client.post(
-            f"/automations/{automation_id}/toggle",
+            f"/housekeeping/automations/{automation_id}/toggle",
             json={"enable": enable},
         )
         response.raise_for_status()
@@ -515,7 +515,7 @@ class CoreAPIClient:
         logger.debug("core_api_get_history", entity_id=entity_id, hours=hours)
 
         response = await client.get(
-            "/history",
+            "/housekeeping/history",
             params={"entity_id": entity_id, "hours": hours},
         )
         response.raise_for_status()
@@ -536,7 +536,7 @@ class CoreAPIClient:
         """
         try:
             client = self._ensure_client()
-            response = await client.get("/health")
+            response = await client.get("/housekeeping/health")
             return response.status_code == 200
         except Exception as e:
             logger.warning("core_api_health_check_failed", error=str(e))

src/agents/housekeeper/tools.py

@@ -59,10 +59,27 @@ async def list_devices(
 
             for dom, dom_devices in sorted(by_domain.items()):
                 output_parts.append(f"### {dom.title()}s")
-                for device in dom_devices:
+
+                # Sort devices: room groups first (using Home Assistant's is_hue_group attribute)
+                def is_room_group(d: object) -> bool:
+                    """Check if device is a room group based on HA attributes."""
+                    attrs = getattr(d, "attributes", {})
+                    # Check for Hue room groups
+                    if attrs.get("is_hue_group") and attrs.get("hue_type") == "room":
+                        return True
+                    # Check for other group indicators (icon or entity_id list)
+                    if "entity_id" in attrs and isinstance(attrs["entity_id"], list):
+                        return True
+                    return False
+
+                sorted_devices = sorted(dom_devices, key=lambda d: (not is_room_group(d), d.entity_id))
+
+                for device in sorted_devices:
                     state_icon = "on" if device.state == "on" else "off" if device.state == "off" else device.state
                     area_str = f" ({device.area})" if device.area else ""
-                    output_parts.append(f"- **{device.name}**{area_str}: {state_icon}")
+                    # Mark room groups clearly using actual HA data
+                    group_marker = " [ROOM GROUP]" if is_room_group(device) else ""
+                    output_parts.append(f"- **{device.name}**{area_str}{group_marker}: {state_icon}")
                     output_parts.append(f"  ID: `{device.entity_id}`")
                 output_parts.append("")
 
@@ -164,12 +181,12 @@ async def turn_on(
     color_temp: int | None = None,
 ) -> str:
     """
-    Turn on a device.
+    Turn on a device. Use the entity_id parameter with the EXACT value from list_devices.
 
     For lights, can optionally set brightness and color temperature.
 
     Args:
-        entity_id: Device to turn on (e.g., light.living_room, switch.coffee_maker)
+        entity_id: The EXACT entity ID from list_devices including domain prefix.
         brightness: Optional brightness for lights (0-255, where 255 is full brightness)
         color_temp: Optional color temperature in Kelvin (2700=warm, 6500=cool)
 
@@ -177,10 +194,9 @@ async def turn_on(
         Confirmation of the action
 
     Examples:
-        turn_on("light.living_room")  # Turn on at current brightness
-        turn_on("light.bedroom", brightness=128)  # Turn on at 50% brightness
-        turn_on("light.office", brightness=255, color_temp=4000)  # Full, neutral white
-        turn_on("switch.coffee_maker")  # Turn on a switch
+        turn_on(entity_id="light.living_room")
+        turn_on(entity_id="light.bedroom", brightness=128)
+        turn_on(entity_id="switch.coffee_maker")
     """
     try:
         async with CoreAPIClient() as client:
@@ -209,17 +225,18 @@ async def turn_on(
 
 async def turn_off(entity_id: str) -> str:
     """
-    Turn off a device.
+    Turn off a device. Use the entity_id parameter with the EXACT value from list_devices.
 
     Args:
-        entity_id: Device to turn off (e.g., light.living_room, switch.coffee_maker)
+        entity_id: The EXACT entity ID from list_devices including domain prefix.
 
     Returns:
         Confirmation of the action
 
     Examples:
-        turn_off("light.living_room")
-        turn_off("switch.coffee_maker")
+        turn_off(entity_id="light.living_room")
+        turn_off(entity_id="switch.coffee_maker")
+        turn_off(entity_id="light.kitchen")
     """
     try:
         async with CoreAPIClient() as client:
@@ -239,15 +256,17 @@ async def toggle(entity_id: str) -> str:
     """
     Toggle a device's state (on becomes off, off becomes on).
 
+    Use the entity_id parameter with the EXACT value from list_devices.
+
     Args:
-        entity_id: Device to toggle
+        entity_id: The EXACT entity ID from list_devices including domain prefix.
 
     Returns:
         Confirmation with the new state
 
     Examples:
-        toggle("light.living_room")
-        toggle("switch.fan")
+        toggle(entity_id="light.living_room")
+        toggle(entity_id="switch.fan")
     """
     try:
         async with CoreAPIClient() as client:

src/agents/librarian/client.py

@@ -281,9 +281,16 @@ class LibraryDeskClient:
                 metadata=r.get("metadata", {}),
             ))
 
+        # Handle keywords being either a list or a dict with core_keywords
+        raw_keywords = data.get("keywords", [])
+        if isinstance(raw_keywords, dict):
+            keywords = raw_keywords.get("core_keywords", [])
+        else:
+            keywords = raw_keywords
+
         return HybridRAGResponse(
             results=results,
-            keywords=data.get("keywords", []),
+            keywords=keywords,
             synonyms=data.get("synonyms", []),
             related_dossiers=data.get("related_dossiers", []),
             formatted_context=data.get("formatted_context", ""),

src/agents/orchestration.py

@@ -176,19 +176,19 @@ async def orchestrate_with_think_updates(
     if delegation_task.expert_name == "librarian":
         expert_display_name = "The Librarian"
 
-    yield f"<think>🤝 Consulting {expert_display_name}...</think>\n"
+    yield f"🤝 Consulting {expert_display_name}...\n"
 
     # Execute delegation (uses run() internally)
     result = await execute_delegation(delegation_task)
 
     if result.success:
-        yield f"<think>✅ {expert_display_name} completed research.</think>\n"
+        yield f"✅ {expert_display_name} completed research.\n"
 
         # Yield the expert's findings
         if result.output:
             yield f"\n{result.output}"
     else:
-        yield f"<think>⚠️ {expert_display_name} encountered an issue: {result.error}</think>\n"
+        yield f"⚠️ {expert_display_name} encountered an issue: {result.error}\n"
 
     logger.info(
         "orchestration_complete",
@@ -449,12 +449,12 @@ async def orchestrate_multi_expert(
         return
 
     # Stream: Starting multi-expert coordination
-    yield f"<think>🎯 Starting multi-expert coordination ({len(tasks)} tasks, {mode.value})...</think>\n"
+    yield f"🎯 Starting multi-expert coordination ({len(tasks)} tasks, {mode.value})...\n"
 
     if mode == ExecutionMode.PARALLEL:
         # Parallel execution - emit one update then run all at once
         expert_names = ", ".join(_get_display_name(t.expert_name) for t in tasks)
-        yield f"<think>🔄 Consulting in parallel: {expert_names}...</think>\n"
+        yield f"🔄 Consulting in parallel: {expert_names}...\n"
 
         result = await execute_parallel(tasks)
 
@@ -462,9 +462,9 @@ async def orchestrate_multi_expert(
         for expert_name, expert_result in result.results.items():
             display_name = _get_display_name(expert_name)
             if expert_result.success:
-                yield f"<think>✅ {display_name} completed.</think>\n"
+                yield f"✅ {display_name} completed.\n"
             else:
-                yield f"<think>⚠️ {display_name} failed: {expert_result.error}</think>\n"
+                yield f"⚠️ {display_name} failed: {expert_result.error}\n"
 
     else:
         # Sequential execution - emit updates for each task
@@ -472,27 +472,27 @@ async def orchestrate_multi_expert(
 
         for task in tasks:
             display_name = _get_display_name(task.expert_name)
-            yield f"<think>🤝 Consulting {display_name}...</think>\n"
+            yield f"🤝 Consulting {display_name}...\n"
 
             task_result = await execute_delegation(task)
             result.add_result(task_result)
 
             if task_result.success:
-                yield f"<think>✅ {display_name} completed.</think>\n"
+                yield f"✅ {display_name} completed.\n"
             else:
-                yield f"<think>⚠️ {display_name} failed: {task_result.error}</think>\n"
+                yield f"⚠️ {display_name} failed: {task_result.error}\n"
                 if stop_on_failure:
-                    yield "<think>🛑 Stopping due to failure.</think>\n"
+                    yield "🛑 Stopping due to failure.\n"
                     break
 
         result.aggregate_outputs()
 
     # Stream: Summary
     if result.all_succeeded:
-        yield "<think>🎉 All experts completed successfully.</think>\n"
+        yield "🎉 All experts completed successfully.\n"
     else:
         failed_names = ", ".join(_get_display_name(e) for e in result.failed_experts)
-        yield f"<think>⚠️ Some experts failed: {failed_names}</think>\n"
+        yield f"⚠️ Some experts failed: {failed_names}\n"
 
     # Yield combined output
     if result.combined_output:

src/chat/schemas.py

@@ -55,6 +55,7 @@ class ChatCompletionChunkDelta(CustomBaseModel):
     """Delta in streaming chunk."""
     role: str | None = None
     content: str | None = None
+    reasoning_content: str | None = None  # For thinking/reasoning (DeepSeek R1 format)
 
 
 class ChatCompletionChunkChoice(CustomBaseModel):

src/chat/service.py

@@ -172,24 +172,9 @@ async def create_chat_completion_stream(
 
     async for event in stream_generator:
         if event.event == StreamEventType.REASONING_SUMMARY_DELTA:
-            # Start <think> block if needed
-            if not in_reasoning:
-                yield ChatCompletionChunk(
-                    id=completion_id,
-                    object=constants.CHAT_COMPLETION_CHUNK_OBJECT,
-                    created=created_at,
-                    model=request.model,
-                    choices=[
-                        ChatCompletionChunkChoice(
-                            index=0,
-                            delta=ChatCompletionChunkDelta(content="<think>\n"),
-                            finish_reason=None,
-                        )
-                    ],
-                )
-                in_reasoning = True
-
-            # Stream reasoning delta
+            # Stream reasoning via reasoning_content field (DeepSeek R1 format)
+            # Open WebUI renders this as collapsible thinking block
+            in_reasoning = True
             yield ChatCompletionChunk(
                 id=completion_id,
                 object=constants.CHAT_COMPLETION_CHUNK_OBJECT,
@@ -198,29 +183,15 @@ async def create_chat_completion_stream(
                 choices=[
                     ChatCompletionChunkChoice(
                         index=0,
-                        delta=ChatCompletionChunkDelta(content=event.delta),
+                        delta=ChatCompletionChunkDelta(reasoning_content=event.delta),
                         finish_reason=None,
                     )
                 ],
             )
 
         elif event.event == StreamEventType.REASONING_SUMMARY_DONE:
-            # Close <think> block
-            if in_reasoning:
-                yield ChatCompletionChunk(
-                    id=completion_id,
-                    object=constants.CHAT_COMPLETION_CHUNK_OBJECT,
-                    created=created_at,
-                    model=request.model,
-                    choices=[
-                        ChatCompletionChunkChoice(
-                            index=0,
-                            delta=ChatCompletionChunkDelta(content="</think>\n\n"),
-                            finish_reason=None,
-                        )
-                    ],
-                )
-                in_reasoning = False
+            # Signal end of reasoning block (no content needed)
+            in_reasoning = False
 
         elif event.event == StreamEventType.OUTPUT_TEXT_DELTA:
             # Stream message content

src/core/benchmarks.py

@@ -47,6 +47,10 @@ class PerformanceBenchmark(BaseModel):
         data = self.model_dump()
         data["timestamp"] = self.timestamp.isoformat()
         data["metadata"] = json.dumps(self.metadata)
+        # Convert booleans to strings (Redis doesn't accept bool type)
+        for key, value in data.items():
+            if isinstance(value, bool):
+                data[key] = str(value)
         return data
 
     @classmethod
@@ -54,6 +58,10 @@ class PerformanceBenchmark(BaseModel):
         """Reconstruct from Redis dict."""
         data["timestamp"] = datetime.fromisoformat(data["timestamp"])
         data["metadata"] = json.loads(data.get("metadata", "{}"))
+        # Convert string booleans back to bool
+        for key in ["success", "was_recommended", "was_actually_used"]:
+            if key in data and isinstance(data[key], str):
+                data[key] = data[key] == "True"
         return cls(**data)
 
 

src/core/tool_tracking.py

@@ -43,6 +43,16 @@ class ToolCallTracker:
             conversation_id=conversation_id,
         )
 
+    def _extract_capability(self, tool_name: str) -> str:
+        """
+        Extract capability name from tool name.
+
+        Tool names like 'delegate_to_librarian' map to capability 'librarian'.
+        """
+        if tool_name.startswith("delegate_to_"):
+            return tool_name.replace("delegate_to_", "")
+        return tool_name
+
     async def track_call(self, tool_name: str, duration: float):
         """
         Record a tool call with timing.
@@ -56,8 +66,9 @@ class ToolCallTracker:
             self.actual_calls[tool_name] = []
         self.actual_calls[tool_name].append(duration)
 
-        # Check if tool was recommended
-        was_recommended = tool_name in self.recommended_capabilities
+        # Check if tool was recommended (normalize tool name to capability)
+        capability = self._extract_capability(tool_name)
+        was_recommended = capability in self.recommended_capabilities
 
         if not was_recommended:
             logger.warning(
@@ -98,8 +109,12 @@ class ToolCallTracker:
         Called after Tatlock completes its response to identify
         tools that were recommended but never used.
         """
+        # Normalize actual tool names to capabilities for comparison
+        used_capabilities = {
+            self._extract_capability(tool) for tool in self.actual_calls.keys()
+        }
         # Find tools that were recommended but not used
-        unused_tools = self.recommended_capabilities - set(self.actual_calls.keys())
+        unused_tools = self.recommended_capabilities - used_capabilities
 
         if unused_tools:
             logger.info(
@@ -145,7 +160,11 @@ class ToolCallTracker:
             Dict with tracking statistics
         """
         total_calls = sum(len(durations) for durations in self.actual_calls.values())
-        unused = self.recommended_capabilities - set(self.actual_calls.keys())
+        # Normalize actual tool names to capabilities for comparison
+        used_capabilities = {
+            self._extract_capability(tool) for tool in self.actual_calls.keys()
+        }
+        unused = self.recommended_capabilities - used_capabilities
 
         return {
             "recommended_capabilities": list(self.recommended_capabilities),
@@ -154,11 +173,11 @@ class ToolCallTracker:
             "total_calls": total_calls,
             "accuracy": {
                 "recommended_and_used": len(
-                    self.recommended_capabilities & set(self.actual_calls.keys())
+                    self.recommended_capabilities & used_capabilities
                 ),
                 "recommended_but_unused": len(unused),
                 "not_recommended_but_used": len(
-                    set(self.actual_calls.keys()) - self.recommended_capabilities
+                    used_capabilities - self.recommended_capabilities
                 ),
             },
         }

tests/agents/test_delegation.py

@@ -248,13 +248,16 @@ class TestHouseholdThinkMessages:
                 assert "success" in messages, f"{expert}/{action_type} missing 'success'"
                 assert "error" in messages, f"{expert}/{action_type} missing 'error'"
 
-    def test_messages_are_think_tags(self):
-        """Test messages are wrapped in <think> tags."""
+    def test_messages_are_plain_text(self):
+        """Test messages are plain text (no <think> wrappers - those go to reasoning_content)."""
         for expert, action_types in HOUSEHOLD_THINK_MESSAGES.items():
             for action_type, messages in action_types.items():
                 for phase, msg in messages.items():
-                    assert msg.startswith("<think>"), f"{expert}/{action_type}/{phase}"
-                    assert msg.endswith("</think>"), f"{expert}/{action_type}/{phase}"
+                    # Messages should NOT have <think> wrappers - they go to reasoning_content field
+                    assert "<think>" not in msg, f"{expert}/{action_type}/{phase} should not have <think> wrapper"
+                    assert "</think>" not in msg, f"{expert}/{action_type}/{phase} should not have </think> wrapper"
+                    # Messages should be non-empty strings
+                    assert isinstance(msg, str) and len(msg) > 0, f"{expert}/{action_type}/{phase}"
 
 
 @pytest.mark.unit
@@ -310,31 +313,32 @@ class TestGetThinkMessage:
     def test_librarian_retrieve_start(self):
         """Test getting librarian retrieve start message."""
         msg = get_think_message("librarian", "search for Docker", "start")
-        assert "<think>" in msg
-        assert "</think>" in msg
+        # No <think> wrappers - messages go to reasoning_content field
+        assert "<think>" not in msg
+        assert "archives" in msg.lower() or "consult" in msg.lower()
 
     def test_librarian_create_success(self):
         """Test getting librarian create success message."""
         msg = get_think_message("librarian", "create a wiki page", "success")
-        assert "<think>" in msg
+        assert "<think>" not in msg
         assert "catalogued" in msg.lower()
 
     def test_biographer_record_start(self):
         """Test getting biographer record start message."""
         msg = get_think_message("biographer", "remember my preference", "start")
-        assert "<think>" in msg
+        assert "<think>" not in msg
         assert "note" in msg.lower() or "biographer" in msg.lower()
 
     def test_housekeeper_control_success(self):
         """Test getting housekeeper control success message."""
         msg = get_think_message("housekeeper", "turn on the lights", "success")
-        assert "<think>" in msg
+        assert "<think>" not in msg
         assert "configured" in msg.lower()
 
     def test_unknown_expert_fallback(self):
         """Test unknown expert gets fallback message."""
         msg = get_think_message("unknown_expert", "some task", "start")
-        assert "<think>" in msg
+        assert "<think>" not in msg
         assert "unknown_expert" in msg.lower()
 
 

tests/agents/test_orchestration.py

@@ -208,8 +208,8 @@ class TestOrchestrateWithThinkUpdates:
             ):
                 updates.append(update)
 
-            # First update should be think tag about consulting
-            assert any("<think>" in u and "Consulting" in u for u in updates)
+            # First update should be about consulting (no <think> wrappers anymore)
+            assert any("Consulting" in u for u in updates)
 
     @pytest.mark.asyncio
     async def test_orchestrate_emits_think_after_delegation(self):
@@ -233,8 +233,8 @@ class TestOrchestrateWithThinkUpdates:
             ):
                 updates.append(update)
 
-            # Should have think tag about completion
-            assert any("<think>" in u and "completed" in u for u in updates)
+            # Should have message about completion (no <think> wrappers anymore)
+            assert any("completed" in u for u in updates)
 
     @pytest.mark.asyncio
     async def test_orchestrate_yields_expert_output(self):

tests/chat/test_streaming_wrapper.py

@@ -4,7 +4,7 @@ Tests for chat completions streaming wrapper.
 Tests that the wrapper correctly:
 - Wraps Responses API
 - Enables reasoning automatically
-- Converts reasoning to <think> tags
+- Streams reasoning via reasoning_content field (DeepSeek R1 format)
 - Streams both reasoning and content
 """
 import json
@@ -17,7 +17,7 @@ from src.chat import constants
 @pytest.mark.unit
 @pytest.mark.asyncio
 async def test_streaming_wrapper_enables_reasoning(async_client: AsyncClient):
-    """Test that streaming wrapper automatically enables reasoning."""
+    """Test that streaming wrapper automatically enables reasoning via reasoning_content."""
     request_data = {
         "model": "lorem-tester",
         "messages": [
@@ -27,7 +27,7 @@ async def test_streaming_wrapper_enables_reasoning(async_client: AsyncClient):
     }
 
     chunks_received = []
-    think_tags_found = False
+    reasoning_content_found = False
 
     async with async_client.stream(
         "POST",
@@ -51,12 +51,12 @@ async def test_streaming_wrapper_enables_reasoning(async_client: AsyncClient):
                     chunk = json.loads(data_str)
                     chunks_received.append(chunk)
 
-                    # Check for <think> tags in delta content
+                    # Check for reasoning_content in delta (DeepSeek R1 format)
                     if "choices" in chunk and len(chunk["choices"]) > 0:
                         delta = chunk["choices"][0].get("delta", {})
-                        content = delta.get("content")
-                        if content and ("<think>" in content or "</think>" in content):
-                            think_tags_found = True
+                        reasoning = delta.get("reasoning_content")
+                        if reasoning:
+                            reasoning_content_found = True
 
                 except json.JSONDecodeError:
                     pass
@@ -64,14 +64,14 @@ async def test_streaming_wrapper_enables_reasoning(async_client: AsyncClient):
     # Should have received chunks
     assert len(chunks_received) > 0
 
-    # Should have found <think> tags (reasoning enabled automatically)
-    assert think_tags_found, "Expected <think> tags in streaming output"
+    # Should have found reasoning_content (reasoning enabled automatically)
+    assert reasoning_content_found, "Expected reasoning_content in streaming output"
 
 
 @pytest.mark.unit
 @pytest.mark.asyncio
 async def test_streaming_wrapper_reasoning_before_content(async_client: AsyncClient):
-    """Test that reasoning (<think> tags) comes before actual content."""
+    """Test that reasoning_content comes before regular content."""
     request_data = {
         "model": "lorem-tester",
         "messages": [
@@ -80,10 +80,7 @@ async def test_streaming_wrapper_reasoning_before_content(async_client: AsyncCli
         "stream": True
     }
 
-    all_content = []
-    found_think_opening = False
-    found_think_closing = False
-    found_content_after_think = False
+    chunk_types = []  # Track order: 'reasoning' or 'content'
 
     async with async_client.stream(
         "POST",
@@ -106,28 +103,22 @@ async def test_streaming_wrapper_reasoning_before_content(async_client: AsyncCli
                     chunk = json.loads(data_str)
                     if "choices" in chunk and len(chunk["choices"]) > 0:
                         delta = chunk["choices"][0].get("delta", {})
-                        content = delta.get("content", "")
-                        if content:
-                            all_content.append(content)
+                        reasoning = delta.get("reasoning_content")
+                        content = delta.get("content")
 
-                            if "<think>" in content:
-                                found_think_opening = True
-                            if "</think>" in content:
-                                found_think_closing = True
-                            # Content after closing think tag
-                            if found_think_closing and content.strip() and "<think>" not in content and "</think>" not in content:
-                                found_content_after_think = True
+                        if reasoning:
+                            chunk_types.append("reasoning")
+                        if content:
+                            chunk_types.append("content")
 
                 except json.JSONDecodeError:
                     pass
 
-    # Verify ordering
-    full_text = "".join(all_content)
-    if found_think_opening and found_think_closing:
-        # Reasoning should come before main content
-        think_start = full_text.index("<think>")
-        think_end = full_text.index("</think>")
-        assert think_start < think_end, "Opening <think> should come before closing </think>"
+    # Verify reasoning comes before content
+    if "reasoning" in chunk_types and "content" in chunk_types:
+        first_reasoning = chunk_types.index("reasoning")
+        first_content = chunk_types.index("content")
+        assert first_reasoning < first_content, "reasoning_content should come before content"
 
 
 @pytest.mark.unit

tests/core/test_benchmarks.py

@@ -61,7 +61,7 @@ class TestPerformanceBenchmark:
         redis_dict = benchmark.to_redis_dict()
         assert redis_dict["operation"] == "test_op"
         assert redis_dict["duration_seconds"] == 1.0
-        assert redis_dict["success"] is True
+        assert redis_dict["success"] == "True"  # Booleans stored as strings in Redis
         assert isinstance(redis_dict["timestamp"], str)
         assert isinstance(redis_dict["metadata"], str)
 
@@ -72,7 +72,7 @@ class TestPerformanceBenchmark:
             "timestamp": now.isoformat(),
             "operation": "test_op",
             "duration_seconds": 1.5,
-            "success": True,
+            "success": "True",  # Booleans stored as strings in Redis
             "metadata": json.dumps({"test": "data"}),
             "recommendation_count": None,
             "confidence": None,
@@ -85,6 +85,7 @@ class TestPerformanceBenchmark:
         benchmark = PerformanceBenchmark.from_redis_dict(redis_dict)
         assert benchmark.operation == "test_op"
         assert benchmark.duration_seconds == 1.5
+        assert benchmark.success is True  # Converted back to bool
         assert benchmark.metadata == {"test": "data"}
 
 
@@ -162,12 +163,12 @@ class TestBenchmarkStore:
         mock_key = f"benchmark:test_op:{int(now.timestamp() * 1000)}"
         mock_redis.zrevrangebyscore.return_value = [mock_key]
 
-        # Mock hgetall to return proper data
+        # Mock hgetall to return proper data (booleans as strings, like Redis)
         mock_redis.hgetall.return_value = {
             "timestamp": now.isoformat(),
             "operation": "test_op",
             "duration_seconds": 1.5,  # Numeric, not string
-            "success": True,
+            "success": "True",  # Booleans stored as strings in Redis
             "metadata": "{}",
             "recommendation_count": None,
             "confidence": None,
@@ -237,7 +238,7 @@ class TestBenchmarkStore:
                 "timestamp": now.isoformat(),
                 "operation": "test_op",
                 "duration_seconds": float(data["duration_seconds"]),
-                "success": data["success"] == "True",
+                "success": data["success"],  # Pass string through, from_redis_dict converts
                 "metadata": "{}",
                 "recommendation_count": None,
                 "confidence": None,
@@ -296,14 +297,14 @@ class TestBenchmarkStore:
                 "timestamp": now.isoformat(),
                 "operation": "tool_call",
                 "duration_seconds": 1.0,
-                "success": True,
+                "success": "True",  # Booleans stored as strings in Redis
                 "metadata": "{}",
                 "recommendation_count": None,
                 "confidence": None,
                 "tool_name": "test_tool",
                 "conversation_id": None,
-                "was_recommended": data["was_recommended"] == "True",
-                "was_actually_used": data["was_actually_used"] == "True",
+                "was_recommended": data["was_recommended"],  # Already strings
+                "was_actually_used": data["was_actually_used"],  # Already strings
             }
 
         mock_redis.hgetall.side_effect = mock_hgetall

tests/core/test_tool_tracking.py

@@ -0,0 +1,101 @@
+"""
+Tests for tool call tracking.
+
+Tests capability extraction and recommendation matching.
+"""
+from unittest.mock import AsyncMock, patch
+
+import pytest
+
+from src.core.tool_tracking import ToolCallTracker
+
+
+class TestToolCallTracker:
+    """Test ToolCallTracker functionality."""
+
+    def test_extract_capability_delegation_tool(self):
+        """Test extracting capability from delegation tool name."""
+        tracker = ToolCallTracker(recommended_capabilities=["librarian"])
+
+        assert tracker._extract_capability("delegate_to_librarian") == "librarian"
+        assert tracker._extract_capability("delegate_to_biographer") == "biographer"
+        assert tracker._extract_capability("delegate_to_housekeeper") == "housekeeper"
+
+    def test_extract_capability_non_delegation_tool(self):
+        """Test that non-delegation tools return unchanged."""
+        tracker = ToolCallTracker(recommended_capabilities=[])
+
+        assert tracker._extract_capability("calculate") == "calculate"
+        assert tracker._extract_capability("search_web") == "search_web"
+
+    @pytest.mark.asyncio
+    async def test_track_call_recognizes_delegation_as_recommended(self):
+        """Test that delegate_to_X is recognized when X is recommended."""
+        tracker = ToolCallTracker(
+            recommended_capabilities=["librarian", "biographer"]
+        )
+
+        with patch("src.core.tool_tracking.get_benchmark_store") as mock_store:
+            mock_store.return_value.record = AsyncMock()
+
+            await tracker.track_call("delegate_to_librarian", 1.0)
+
+            # Should NOT log warning since librarian was recommended
+            call_args = mock_store.return_value.record.call_args
+            benchmark = call_args[0][0]
+            assert benchmark.was_recommended is True
+
+    @pytest.mark.asyncio
+    async def test_track_call_detects_not_recommended(self):
+        """Test that unrecommended tools are flagged."""
+        tracker = ToolCallTracker(
+            recommended_capabilities=["librarian"]
+        )
+
+        with patch("src.core.tool_tracking.get_benchmark_store") as mock_store:
+            mock_store.return_value.record = AsyncMock()
+
+            await tracker.track_call("delegate_to_housekeeper", 1.0)
+
+            call_args = mock_store.return_value.record.call_args
+            benchmark = call_args[0][0]
+            assert benchmark.was_recommended is False
+
+    def test_get_summary_with_delegation_tools(self):
+        """Test summary correctly maps delegation tools to capabilities."""
+        tracker = ToolCallTracker(
+            recommended_capabilities=["librarian", "biographer"]
+        )
+        tracker.actual_calls = {
+            "delegate_to_librarian": [1.0, 2.0],
+            "delegate_to_housekeeper": [0.5],  # Not recommended
+        }
+
+        summary = tracker.get_summary()
+
+        assert summary["accuracy"]["recommended_and_used"] == 1  # librarian
+        assert summary["accuracy"]["recommended_but_unused"] == 1  # biographer
+        assert summary["accuracy"]["not_recommended_but_used"] == 1  # housekeeper
+
+    @pytest.mark.asyncio
+    async def test_finalize_with_delegation_tools(self):
+        """Test finalize correctly identifies unused recommendations."""
+        tracker = ToolCallTracker(
+            recommended_capabilities=["librarian", "biographer"]
+        )
+        tracker.actual_calls = {
+            "delegate_to_librarian": [1.0],
+        }
+
+        with patch("src.core.tool_tracking.get_benchmark_store") as mock_store:
+            mock_store.return_value.record = AsyncMock()
+
+            await tracker.finalize()
+
+            # Should record benchmark for unused biographer
+            assert mock_store.return_value.record.called
+            call_args = mock_store.return_value.record.call_args
+            benchmark = call_args[0][0]
+            assert benchmark.tool_name == "biographer"
+            assert benchmark.was_recommended is True
+            assert benchmark.was_actually_used is False

tests/e2e/test_api_endpoints.py

@@ -8,8 +8,8 @@ These tests hit the actual running server and test the full stack:
 - Response formatting
 """
 import pytest
+import pytest_asyncio
 import httpx
-import asyncio
 from typing import AsyncGenerator
 
 # Test server base URL (assumes server is running on localhost:8777 via ./wakeup.sh)
@@ -17,15 +17,7 @@ BASE_URL = "http://localhost:8777"
 API_TIMEOUT = 120.0  # 120 second timeout for LLM calls
 
 
-@pytest.fixture(scope="module")
-def event_loop():
-    """Create event loop for async tests."""
-    loop = asyncio.get_event_loop_policy().new_event_loop()
-    yield loop
-    loop.close()
-
-
-@pytest.fixture(scope="module")
+@pytest_asyncio.fixture(loop_scope="module", scope="module")
 async def client() -> AsyncGenerator[httpx.AsyncClient, None]:
     """HTTP client for making requests."""
     async with httpx.AsyncClient(base_url=BASE_URL, timeout=API_TIMEOUT) as client:

wakeup.sh

@@ -47,4 +47,4 @@ echo -e "${GREEN}Starting uvicorn server on http://localhost:8777${NC}"
 echo -e "${YELLOW}Press Ctrl+C to stop the server${NC}"
 echo ""
 
-uvicorn src.main:app --reload --host 0.0.0.0 --port 8777 2>&1 | tee "$LOG_FILE"
+uvicorn src.main:app --reload --host localhost --port 8777 2>&1 | tee "$LOG_FILE"