jpmschweitzer/odysseus
1
0
Fork 0
mirror of https://github.com/pewdiepie-archdaemon/odysseus.git synced 2026-10-06 06:52:20 +02:00
Code Issues Packages Projects Releases Wiki Activity

Record controlled retrieval and recovery-history findings

Browse Source
This commit is contained in:
pewdiepie-archdaemon
2026-09-17 21:26:46 +00:00
parent 71784cd16f
commit 5a855b5db3
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/search-quality-audit-20260917.md
+4
View File
@@ -75,6 +75,10 @@ Further provider inspection found that the news-to-general fallback dropped the
### Additional informal/multi-part live checks
`reports/search-synthesis-probe-1789680321559.json` compares identical saved native tool history with/without `_harness_control` messages, same canonical base prompt, no offered tools. Full trace: short answer without links, 2.59s. Controls removed: longer answer with links, 6.39s, but introduced a Do Not Track URL not established by the recorded evidence. This is not grounds to remove recovery controls wholesale or claim a factual quality win.
Weekly-news replay `reports/clean-v3-search-quality-2026-09-17T21-24-11-361Z.json` corrected the missing query but still returned no evidence (16.35s). Direct simultaneous provider control with exact query `AI developments this week`, `time_filter=week`: general returned zero, news five. `3ea5a348` recognizes time-qualified developments as news intent while retaining general routing for tutorials, historical discussion, software versions and documentation. Provider/publication/query-relaxation tests: 80 passed. Live weekly-news replay launched after deployment; returned results still require relevance/source review.
Timed `reports/clean-v3-search-quality-2026-09-17T21-22-21-304Z.json`: Firefox 31.0s/five rounds, tool execution 5.779s; news 69.8s/eight rounds, tool execution 1.672s. Remaining time includes inference, streaming and orchestration—not proven pure GPU time. The source-link retry still failed on Firefox. Main observed delay is outside tool execution, not search-provider time in these cached runs.
`4b6a9721` applies the explicit-query requirement to initial calls too: the weekly-news trace had copied a whole compound request into a missing query. Regression suites: 1,249 passed. Weekly-news replay launched. `reports/search-synthesis-probe-1789680251422.json` feeds the saved Firefox evidence to the same model without live recovery history/offered tools: canonical-system answer took 4.98s and concise research-system answer 4.66s; both supplied a link. This proves the model can emit the link in simplified context, not factual correctness—the linked support page was access-blocked, and some feature assertions still need grounding. Do not infer the system prompt alone or lack of tool schemas uniquely explains the difference.