odysseus/docs/REGULAR_MODEL_TOOL_COMPATIBILITY.md

Regular-model tool compatibility

Last verified: 2026-09-09 through the authenticated 7011 Agent UI as sft_alex_creator.

This is the legacy-RAG track. The exact model odysseus-qwen3.5-tools-pre-heretic is excluded and remains on its model-owned clean compact runtime.

Current baseline

Endpoint	Model	Ten-family result	State
DeepSeek	deepseek-v4-flash	10/10	passed
DeepSeek	deepseek-v4-pro	10/10	passed
OpenAI	gpt-5.5	10/10	passed
OpenAI	gpt-5.6-sol	10/10	passed
OpenAI	gpt-5.6-terra	10/10	passed
OpenAI	gpt-5.6-luna	10/10	passed
OpenRouter	moonshotai/kimi-k3	10/10	passed
OpenRouter	x-ai/grok-4.5	10/10	passed
OpenRouter	qwen/qwen3-vl-235b-a22b-instruct	10/10	passed
OpenRouter	openai/gpt-5-image	n/a	image generation; chat tools unsupported
Local 100.69.120.65:8062	Qwen/Qwen3.5-9B	not run	endpoint unavailable
Local 100.69.120.65:8062	GLM-5.3-Flash-Alis-MLX-4bit	not run	endpoint unavailable

The ten-family baseline covers one read-only functional turn each for notes, calendar, email accounts, tasks, documents, memory, skills, Cookbook/admin, web search, and shell. It verifies the legacy route, expected native tool call, execution result, visible UI answer, and absence of reasoning leakage. It is not yet a claim that every mutation/action variant, typo, or follow-up passes.

Typo and follow-up profile

The stricter real-UI profile sends one misspelled read-only request to every family, followed immediately by a noun-free reference to the returned result. Read-only follow-ups must not call any tool; search follow-ups may either use the existing evidence or fetch the prior link. Across the nine chat-capable API models, the composited post-repair result is 178/180 turns (98.89%):

Model	Conversation result
deepseek-v4-flash	20/20
deepseek-v4-pro	20/20
gpt-5.5	20/20
gpt-5.6-sol	20/20
gpt-5.6-terra	20/20
gpt-5.6-luna	18/20
moonshotai/kimi-k3	20/20
x-ai/grok-4.5	20/20
qwen/qwen3-vl-235b-a22b-instruct	20/20

Luna's only remaining family miss is a deliberately misspelled Shell request. The correct-spelling baseline passes. The harness does not auto-execute a shell command to hide that model-owned limitation.

The shared repair recognizes a uniquely misspelled action verb and family noun, then seals only declared safe private reads with immutable canonical arguments. This repaired Tasks/Documents/Memory and adjacent read families across providers without widening mutation or Shell authority. A compact native-tool instruction also tells regular API models to map clear typos to a currently offered tool.

Conversation evidence:

• reports/regular-model-conversation-flash-r3-20260909.json
• reports/regular-model-conversation-remaining-r1-20260909.json
• reports/regular-model-conversation-repair-r1-20260909.json
• reports/regular-model-conversation-shell-r1-20260909.json
• reports/regular-model-conversation-qwen-repair-r1-20260909.json
• reports/regular-model-conversation-qwen-tail-r1-20260909.json
• reports/regular-model-conversation-qwen-search-r1-20260909.json

Evidence:

• reports/regular-model-tools-provider-final-r4-20260909.json — Flash, GPT-5.5, Kimi: 30/30.
• reports/regular-model-tools-repair-r3-20260909.json — Pro and Sol: 20/20; retained Qwen pre-final 9/10 miss.
• reports/regular-qwen-vl-full-r4-20260909.json — Qwen-VL final family-switch run: 10/10.
• reports/regular-model-tools-remaining-20260909.json — Terra, Luna, Grok: 30/30; records unavailable/unsupported models and pre-repair failures.
• reports/regular-model-tools-postfix-r1-20260909.json — post-hardening rerun: nine chat-capable API models passed 90/90 family turns with zero model failures. Its overall status is non-passing only because the two configured local endpoints were offline; the image-only model remains unsupported.

Family switch and page inspection

The six-turn switch/back flow covers notes → calendar → notes from prior evidence → web search → explicit web_fetch → calendar from prior evidence. All nine API models have a clean 6/6 reproduction (54/54). Kimi skipped search once in the retained first run and passed a fresh reproduction; that variability remains visible instead of being erased.

Evidence:

• reports/regular-model-switchback-flash-r2-20260909.json
• reports/regular-model-switchback-remaining-r1-20260909.json
• reports/regular-model-switchback-kimi-r1-20260909.json

Repair that produced the clean baseline

Regular models no longer inherit up to three stale tool families into every explicit new request. Referential follow-ups still resolve from typed recent tool evidence, while explicit family switches receive the current family only. Safe required reads use active_capabilities, so stale offered context cannot disable their immutable operation. The stream layer also stops an exact long block repeated twice instead of waiting for a provider's full timeout.

The composer no longer treats generic words such as “source”, “system”, “app”, or “review” as authority to silently enable Bash. Explicit shell, terminal, repository, code-file, and direct coding requests retain workspace auto-escalation. This is a shared UI authority fix, not a model-name exception.

Run a bounded subset with:

MODELS='deepseek-v4-flash,gpt-5.5' \
FAMILIES='notes,calendar' WORKERS=2 \
REPORT_PATH=reports/regular-model-check.json \
node scripts/verify_regular_model_tools.mjs

Set PROFILE=conversation to run the typo plus follow-up profile.

The runner discovers only enabled pinned models (visible cached local models when no pins exist), retains no tool outputs or private rows, and deletes only the exact sessions it creates.