odysseus/docs/ODYSSEUS_FIX_WORKSTREAMS.md

Odysseus Fix Workstreams

Evidence source: 626 historical sft_alex_creator contract sessions, deduplicated to 54 flows and replayed through the current 7011 Agent runtime on 2026-09-11.

Harness

• Resolved — canonical item limits: Notes and Calendar now honor explicit limits such as “at most three” while retaining hidden expansion payloads.
• Evaluate separately — shell/files: two WebUI failures occurred because bash is not consistently offered on follow-up. Shell/files belongs to the validated odysseus-native workspace runtime; do not train the model on WebUI refusals.
• Resolved — Calendar argument continuity: referential repeats preserve the preceding successful range; an explicitly new period still replaces it.
• Resolved — evaluator: historical one-turn probes are now retained, and the judge treats HTML-comment expansion rows as hidden rather than visible overflow.

Model / SFT

• Remaining — browser evidence use: the IKEA task routes correctly to private_browser, but the model clicks opaque refs repeatedly and never extracts a chair answer. This is the confirmed SFT repair class.
• Remaining — identity attribution: after successful Email → Calendar switching, “Who are you?” can add the false phrase “trained by Google.” Keep this as SFT data; do not restore a forced harness identity response.
• Resolved in harness — Memory synthesis: row evidence is compacted before the observation cap instead of being truncated inside invalid JSON; Memory is 3/3.
• Resolved in harness — Search recovery and source rendering: equivalent empty queries stop after two attempts, freshness words survive query shortening, and exact source-link requests render the best relevant first-party result. Search is 15/16, with only the browser reasoning case above remaining.
• Resolved in harness — Cookbook synthesis: configured server rows use a bounded evidence-owned renderer; Cookbook is 3/3.

Build repair examples from these behavior classes only after exact replay confirms the failure with the intended runtime and rendering owner.

Backend / Data

• The Python packaging query returned an unrelated OWASP result. The model reported the failure honestly, but should attempt a bounded recovery before stopping.
• Synthetic email account servers are unavailable. The harness now renders that as an outage and blocks invented message IDs; restore the fixture separately.

Current measurement

• Historical source sessions: 626
• Unique replay flows: 54
• Initial judge result: 36 pass / 18 flagged
• Post-renderer replay for Notes, Calendar, and switching: 14 pass / 2 flagged.
• Final Notes + Calendar replay after continuity and judge fixes: 9 pass / 0 flagged.
• Latest Search replay: 15 pass / 1 confirmed SFT failure.
• Memory replay: 3 pass / 0 flagged; Cookbook replay: 3 pass / 0 flagged.
• Final WebUI-valid historical matrix: 49 pass / 2 confirmed SFT failures = 96.1%.

Artifacts:

• Full run: tmp/odysseus-conversation-qa/run-20260911-092930.json
• Post-renderer replay: tmp/odysseus-conversation-qa/run-20260911-093333.json
• Final Notes + Calendar replay: tmp/odysseus-conversation-qa/run-20260911-093752.json
• Latest Search replay: tmp/odysseus-conversation-qa/run-20260911-100239.json
• Memory replay: tmp/odysseus-conversation-qa/run-20260911-095320.json
• Final WebUI-valid matrix: tmp/odysseus-conversation-qa/run-20260911-101229.json
• Deduplicated queue: tmp/odysseus-conversation-qa/historical-sft-alex-queue.json