odysseus/docs/CLEAN_V3_UI_PREVIEW.md

# Tools v3 — No-RAG preview

Select this endpoint in the 7011 model picker, with model
`odysseus-qwen3.5-tools-pre-heretic`. This endpoint owns its complete tool loop
and enters Agent mode server-side on every turn, including ambiguous follow-ups;
it does not depend on the legacy per-message intent classifier. Start a new chat
for an uncontaminated comparison. Enable Web for searches. Clean routing is
owned by the exact model identity, so both the normal `preheret` endpoint and
the `cleanv3` alias use this runtime. Every other model remains on legacy RAG.

Endpoint ID: `cleanv3`. Its base URL uses the same inference server's Tailscale
DNS name, `http://odysseus.tailb895f4.ts.net:18182/v1`, to distinguish it from
the original IP-address route when existing chats omit endpoint IDs.

## Implementation

- `src/clean_agent_preview.py` is a separate streamed native-tool loop, entered
  before legacy routing and substitutions. It uses real authenticated tool
  dispatch, the tool-work `compact_contract_v5` builder, temperature 0,
  and thinking disabled. No weights change or inference server was started.
- The offered tool inventory is stable except for permissions/toggles. Safe,
  explicit personal creates/updates are enabled for notes, tasks, calendar,
  memory, skills and documents. Destructive operations, shell/code, outbound
  email, browser interaction, deployment/admin changes and unrelated-family
  write substitution remain blocked. No tool or argument substitution is
  applied by the loop.
- Native calls and matching results persist in `clean_v3_turn` metadata so
  follow-ups use actual evidence. History retains at most eight complete turns,
  trimming oldest whole turns for size; individual outputs cap at 8000 chars.
- Real search still uses the existing search backend and its provider handling;
  this does not claim that provider quality or every backend transform is fixed.
- All routing, privileges and default settings outside this exact Odysseus model
  remain unchanged. The loop has six execution/eight-round limits.
- Write completion is evidence-bound: affirmative success text is replaced
  unless a private-write tool succeeded during the turn. Proposed call batches
  are policy-preflighted atomically, so a batch containing a blocked operation
  cannot partially execute before denial.

## Verification

399 focused Python tests passed after route integration. Browser runs r1/r2
accidentally exercised the old loop and are not preview evidence. The runner
now explicitly asserts `selection_mode=clean_compact_v3_preview`.

`reports/clean-v3-live-ui-r3-20260909.json` confirms the preview route, real notes
execution, correct repetition from history, successful search and no-tool
summary, plus visible incremental growth. Its notes assertions were for the
old routed contract: they prohibited offering web tools even with Web enabled,
and required another notes call for a verbatim repeat. The updated preview
checks permit stable offers and accept an exact match to the preceding saved
answer without re-execution; execution permissions are still asserted.

`reports/clean-v3-live-ui-r4-20260909.json` is the corrected four-turn check,
including notes with Web off and search with Web on: **4/4 passed**, with the
preview selection mode explicitly confirmed on every turn.
These are UI smoke tests, not all-family or factual-answer benchmark scores.

## Disable

Disabling only endpoint `cleanv3` removes the duplicate picker alias; it does
not disable this model-owned runtime. To roll back the runtime, revert the exact
model route in `routes/chat_routes.py`. Do not delete weights, adapters, or user
chats. The v3 schema builder dependency is
`/home/pewds/odysseus-tool-work/scripts/eval_alltools_unseen_compare.py` and its
schema-dropout helper; preserve those with this deployment.

## Expanded UI checks — 2026-09-09

24 additional turns completed through the preview: 23 automated passes and one
checker false alarm. The Cookbook follow-up correctly shortened the previous
six-server result to the first three requested names without another call. The
checker required either a fresh call or a verbatim repeat; manual inspection
confirmed the requested subset. Raw failure evidence is retained, not rescored.

Covered notes/misspellings/second-note selection, calendar/second-event time,
tasks, documents, memory, skills, Cookbook listing, misspelled search, and Web
toggle changes. Cross-family flows passed: Germany news → “whats my notes”,
notes → “seach current stock mraket news”, and calendar → “now show my noes”.
The model chose `current stock market news` itself. Every completed turn's
audit confirmed the preview mode. Search source factual accuracy is not graded
by this suite, and successful reads do not establish mutation coverage.

Email was separately attempted but the test guard stopped it because the
stable offered inventory exceeded its metadata-only verified scope. Email
therefore remains unverified in this expanded run; the guard was not weakened.
No production code, service settings or weights changed during these tests.

Evidence under `reports/`:

- `clean-v3-broader-ui-20260909.json`: 16 turns, 15 automatic passes, Cookbook caveat.
- `clean-v3-topic-switch-ui-20260909.json`: 6/6 passed.
- `clean-v3-second-note-ui-20260909.json`: 2/2 passed.
- `clean-v3-email-notes-ui-20260909.json`: blocked email attempt; notes not run in that file.

## Picker route fix

The previous tests selected sessions through the API, missing a real picker
bug: local entries were deduplicated by model ID, hiding alternative endpoints
with the same weights. The picker now uses endpoint+model identity for local
routes too, displays the endpoint name, and scopes its last-picked send override
to the current chat. `/api/sessions` returns owner-filtered endpoint identity
for unambiguous saved URLs, so reload labels do not depend on loading the model
catalog. Ambiguous identical URLs are not guessed.

The user-authorized chat `ec0683a2-015f-41d7-aa1f-34135c9640cb` was switched to
`cleanv3` using the authenticated session PATCH API; no messages were inserted
and no tool actions ran in that chat. Defaults and other chats were unchanged.

The runner's `--picker-route true` starts on the original route, clicks the
preview in the real picker, sends a greeting, reloads the chat permalink, then
asks for notes. Early picker/reload reports are incomplete, not passes: their
label check exposed the unloaded-catalog issue. Focused route/picker/history
tests: 16 passed.

Final picker test: `reports/clean-v3-picker-reload-r5-20260909.json`, **2/2
passed**. Real picker click, greeting, permalink reload, and notes follow-up
all confirmed the preview route. The label survived reload. R4 retained a
history/DOM mismatch from sending before restored history was ready; the final
driver explicitly waits for the saved first answer to render before sending.
This does not claim a general fix for sending during unfinished history loading.

## Native image/VL status

The inference launcher previously set `--limit-mm-per-prompt` to zero images,
so vLLM rejected attachments before the model saw them. The durable Odysseus
launcher now permits up to three images per prompt; video remains disabled.

`reports/clean-v3-vl-live-r4-20260909.json` proves the real 7011 attachment
path, clean compact route, object/color/spatial recognition, permalink reload,
and ambiguous image follow-up. Those checks pass. Exact OCR of the deterministic
`ODYSSEUS 42` heading fails in both the untouched Qwen 3.5 9B base and the
fine-tune, so it remains a base/runtime capability limitation rather than a
fine-tune regression or harness failure.

The same native path also passes JPEG and lossless WebP transport, object
recognition, reload, and follow-up grounding. Evidence:
`reports/clean-v3-vl-jpeg-r1-20260909.json` and
`reports/clean-v3-vl-webp-r1-20260909.json`. Both remain `partial` only because
the shared OCR check fails.

## Reversible write check

`scripts/verify_clean_v3_write.mjs` runs against only `sft_alex_creator`. It
creates one UUID-named note through the real 7011 UI, verifies that exact row,
requests a destructive bulk deletion, verifies the row still exists, and then
deletes only its own test row through the authenticated API. The cleanup is
verified by a 404 lookup.

Final evidence: `reports/clean-v3-write-ui-r8-20260909.json`, **passed**. Both
turns reported `selection_mode=clean_compact_v3_preview`; creation executed via
`manage_notes(action=add)`, the destructive action did not execute, and the
canonical response was “No changes were made.” The earlier r3/r5 files are
startup/placement failures, while r4/r6/r7 retained genuine intermediate
harness and verifier failures; none should be interpreted as passes.

## Stateful, search, and email checks

The reversible stateful runner passes all six mutation families in one run:
calendar, notes, tasks, documents, memory, and skills (**6/6**). Each flow
creates a UUID-only artifact through the real Agent UI, verifies it by
owner-scoped API, applies a noun-free correction, verifies persistence, and
removes only that artifact. A direct database audit found zero active synthetic
calendar, note, task, or document rows afterward.

The document failure was harness-owned. Compact description dropout left a
vague free-form `command` field, error envelopes defaulted to exit code 0, and
the clean loop dropped the active document ID. Compact v5 now exposes only
required structured `edits`, reports errors truthfully, and executes against
the request's explicit active document. Fresh document and combined stateful
runs pass.

Search Web-toggle combinations `00`, `01`, `10`, and `11` pass **8/8** across
two turns. A web question can no longer silently enable Bash because it says
“official source”, and an unavailable Web capability exposes no unrelated
fallback family. The quality suite passes **3/3**: evidence reuse without a
second call, explicit official-page inspection with `web_fetch`, correction of
“stock mraket” in actual search arguments, and a truthful unsupported result
for a synthetic company.

Production-path email reads pass **3/3** through the running email MCP: account
list, latest inbox list, and referential read of the first result. The report
retains no account names, addresses, subjects, bodies, prompts, or answers.

Post-fix representative direct/follow-up coverage also passes for every family:
notes/calendar 4/4, tasks/documents/memory/skills/Cookbook/search/shell 14/14,
and email 3/3 in its privacy-preserving runner. The combined legacy verifier's
metadata-only email guard correctly refused its broader stable inventory; that
stopped report is not counted as a model failure.

Evidence:

- `reports/clean-v3-stateful-all-r3-20260909.json`
- `reports/clean-v3-stateful-documents-r2-20260909.json`
- `reports/clean-v3-search-toggle-final-r6-20260909.json`
- `reports/clean-v3-search-quality-r3-20260909.json`
- `reports/clean-v3-email-read-r1-20260909.json`
- `reports/clean-v3-ten-family-tail-postfix-r1-20260909.json`

These checks verify routing, execution, persistence, follow-up, and selected
answer-quality invariants. They are not yet the sealed all-action ship score.

## Compact v5 and corrected contract evidence

Compact v5 keeps the compact-v3 surface and adds only development-positive
field hints for Email, Search/Hugging Face quant selection, and Shell/files.
A Calendar date hint regressed development and was excluded. The Python tool now
emits one final bare expression, REPL-style, without duplicating explicit
`print(...)`; this turns otherwise correct computation calls into visible tool
evidence for all models.

Under frozen scorer `odysseus.contract.v2.5`, development is 327/344 raw
(95.06%) and 327/336 scorable (97.32%). Sealed blind is 311/344 raw (90.41%)
and 311/336 scorable (92.56%), with zero reasoning leakage. Calendar, Shell,
and Tasks remain below the 90% family ship floor, so the model is not yet a
full benchmark ship candidate.

Fresh post-deploy real-UI evidence passes: stateful flows 6/6, Email 3/3,
Search quality/recovery 3/3, private browser 3/3, and VL workflow 3/3. The
Search check accepts a failed attempt only when a later tool succeeds and the
final answer remains grounded.