feat(settings): opt-in reasoning effort for utility model calls (#6572)

* feat(settings): opt-in reasoning effort for utility model calls

Add utility_reasoning_effort (empty = provider default) and let utility
calls inherit a chat session's effort only on that session's matching
endpoint+model route. Explicit caller effort wins; otherwise the utility
setting wins over session inheritance on utility-chain routes. Resolve
the chain and setting per user and validate levels against model evidence.
No-session work uses the setting only, with no chat-effort crossover.

- src/utility_effort.py owns effort_for_call and candidate_effort_factory
- remove global effort fill from llm_call/llm_call_async; utility call
  sites pass effort explicitly, so unrelated calls on the same route do
  not pick it up
- wire auto-title, memory/persona/skill extraction, memory suggest/import/
  audit, compaction, auto-sort, and per-owner background task candidates
- wire no-session email writing-style, summary, and translation calls,
  plus skills evaluation, necessity/retrieval audits, and improvement
- deliberately exclude email replies because thinking_mode='off'
- add the optional candidate_request_factory to llm_call_async_with_fallback
- add sync llm_call reasoning_effort and include it in the cache key
- show the Utility Model effort selector only for advertised levels

Fixes #6569

* fix(utility): exclude email replies and support inherited effort in settings

Pass reasoning_effort='' to task_llm_call_async for scheduled reply drafting to honor the email reply exclusion. In AI settings, resolve the effective Default Chat endpoint and model when Utility is set to 'Same as chat' so supported reasoning effort levels can be selected.

* feat(utility): support same-as-chat effort and session model fallback in compaction

In AI settings, show 'Same as chat' as the default effort option when Utility Model is inherited, and support an explicit 'Provider default' (off) override. Pass the active session fallback in context compaction callers so compaction shares the live chat session's model and reasoning effort for maximum prompt/KV cache affinity.

* fix(settings): clarify chat effort inheritance and keep option visible

Label inherited effort as 'Same as chat (when supported)' and mention
cache affinity and latency speedups in the help text. Ensure the effort
control remains visible when the utility model is set to 'Same as chat'
even if the default chat model lacks advertised reasoning levels, and
populate candidate levels across configured endpoints so the user can
configure effort or keep provider default.

* feat(settings): per-fallback reasoning effort for utility model

Each utility fallback row gets an Effort select, shown only when the
selected model advertises reasoning levels. The choice is stored as
reasoning_effort on the utility_model_fallbacks entry and sent only to
that candidate; an empty value means provider default.

* fix(utility): transmit reasoning effort in non-streaming calls

---------

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>

routes/chat_helpers.py

@@ -551,6 +551,7 @@ async def auto_name_session(session_manager, sess):
     try:
         from src.llm_core import llm_call_async
         from src.task_endpoint import resolve_task_endpoint
+        from src.utility_effort import effort_for_call
 
         # Find first user message
         first_msg = ""
@@ -603,6 +604,7 @@ async def auto_name_session(session_manager, sess):
             max_tokens=64,
             headers=t_headers,
             timeout=15,
+            reasoning_effort=effort_for_call(t_url, t_model, owner, sess),
         )
 
         title = title.strip().strip('"\'').strip()

routes/email/email_helpers.py

@@ -443,9 +443,11 @@ async def _generate_email_summary(
     headers: dict | None = None,
     max_tokens: int = 8192,
     timeout: int = 180,
+    owner: str | None = None,
 ) -> str:
     """Generate an interactive email summary through the shared LLM adapter."""
     from src.llm_core import llm_call_async
+    from src.utility_effort import effort_for_call
 
     raw = await llm_call_async(
         url=url,
@@ -456,6 +458,7 @@ async def _generate_email_summary(
         headers=headers,
         timeout=timeout,
         workload="foreground",
+        reasoning_effort=effort_for_call(url, model, owner),
     )
     return _normalize_email_summary(raw)
 

routes/email/email_pollers.py

@@ -1031,6 +1031,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
                             fallback_url=url, fallback_model=model, fallback_headers=headers,
                             owner=account_owner or None,
                             temperature=0.7, max_tokens=1024, timeout=90,
+                            reasoning_effort="",
                         )
                         reply = _apply_email_style_mechanics(_extract_reply(reply or ""))
                         if reply:

routes/email/email_routes.py

@@ -5920,7 +5920,12 @@ def setup_email_routes():
                 },
             ]
 
-            style = await llm_call_async(url, model, messages, headers=headers, max_tokens=2048)
+            from src.utility_effort import effort_for_call
+
+            style = await llm_call_async(
+                url, model, messages, headers=headers, max_tokens=2048,
+                reasoning_effort=effort_for_call(url, model, owner),
+            )
             style = _strip_think(style or "")
             if not style:
                 return {"success": False, "error": "LLM failed to generate style description"}
@@ -6004,6 +6009,7 @@ def setup_email_routes():
                     headers=req_headers,
                     max_tokens=8192,
                     timeout=180,
+                    owner=owner,
                 )
             except Exception as e:
                 logger.warning(
@@ -6125,6 +6131,8 @@ def setup_email_routes():
             if not candidates:
                 return {"success": False, "error": "No LLM endpoint configured"}
 
+            from src.utility_effort import candidate_effort_factory
+
             content = await llm_call_async_with_fallback(
                 candidates,
                 messages=[
@@ -6152,6 +6160,7 @@ def setup_email_routes():
                 temperature=0.2,
                 max_tokens=8192,
                 timeout=180,
+                candidate_request_factory=candidate_effort_factory(owner),
             )
             model = candidates[0][1] if candidates else ""
             content = (content or "").strip()

routes/history/history_routes.py

@@ -237,6 +237,18 @@ def _keep_count_before_message(db_messages, before_msg_id: str | None) -> int |
     return None
 
 
+def _effort_mode(model: str, raw_effort: Any) -> str:
+    """Session ``thinking_mode`` for a picked effort: ``effort:<level>`` or ``off``.
+
+    Validated like the chat routes do, so llama-swap advertised efforts persist
+    as well as ChatGPT subscription levels.
+    """
+    from src.chatgpt_subscription import validate_reasoning_effort
+
+    effort = validate_reasoning_effort(model, raw_effort)
+    return f"effort:{effort}" if effort else "off"
+
+
 def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
     router = APIRouter(
         tags=["history"],
@@ -1031,26 +1043,11 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
         mode = getattr(session, "thinking_mode", "off") or "off"
         raw_effort = body.get("reasoning_effort")
         if raw_effort is not None:
-            clean_effort = str(raw_effort).strip().lower()
-            if clean_effort in {"", "default"}:
-                mode = "off"
-            else:
-                from src.chatgpt_subscription import get_chatgpt_model_metadata
-                meta = get_chatgpt_model_metadata(session.model)
-                if meta and clean_effort in [lvl.lower() for lvl in meta.get("supported_reasoning_levels", [])]:
-                    mode = f"effort:{clean_effort}"
-                else:
-                    mode = "off"
+            mode = _effort_mode(session.model, raw_effort)
         elif "thinking_mode" in body:
             raw_mode = str(body.get("thinking_mode") or "").strip().lower()
             if raw_mode.startswith("effort:"):
-                clean_effort = raw_mode[7:].strip()
-                from src.chatgpt_subscription import get_chatgpt_model_metadata
-                meta = get_chatgpt_model_metadata(session.model)
-                if meta and clean_effort in [lvl.lower() for lvl in meta.get("supported_reasoning_levels", [])]:
-                    mode = f"effort:{clean_effort}"
-                else:
-                    mode = "off"
+                mode = _effort_mode(session.model, raw_mode[7:])
             elif raw_mode in {"", "on", "off"}:
                 mode = raw_mode
                 from src.model_profiles import supports_user_thinking_toggle
@@ -1129,7 +1126,14 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
             )
 
             # Use utility model if available
-            util_url, util_model, util_headers = resolve_endpoint("utility", owner=owner or None)
+            from src.utility_effort import effort_for_call
+            util_url, util_model, util_headers = resolve_endpoint(
+                "utility",
+                fallback_url=getattr(session, "endpoint_url", None),
+                fallback_model=getattr(session, "model", None),
+                fallback_headers=getattr(session, "headers", None),
+                owner=owner or None,
+            )
             compact_url = util_url or session.endpoint_url
             compact_model = util_model or session.model
             compact_headers = util_headers if util_url else session.headers
@@ -1145,6 +1149,7 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
                 ],
                 temperature=0.2, max_tokens=1024,
                 headers=compact_headers, timeout=30,
+                reasoning_effort=effort_for_call(compact_url, compact_model, owner, session),
             )
             summary = normalize_compaction_summary(summary)
 

routes/memory/memory_routes.py

@@ -261,6 +261,8 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
             sess.endpoint_url, sess.model, sess.headers, owner=_owner(request)
         )
 
+        from src.utility_effort import effort_for_call
+
         try:
             suggestion_text = await llm_call_async(
                 t_url,
@@ -269,6 +271,7 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
                 temperature=0.2,
                 max_tokens=500,
                 headers=t_headers,
+                reasoning_effort=effort_for_call(t_url, t_model, _owner(request), sess),
             )
             try:
                 suggestions = json.loads(suggestion_text)
@@ -349,6 +352,7 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
         endpoint_url = None
         model = None
         headers = {}
+        sess = None
 
         user = _owner(request)
 
@@ -453,6 +457,8 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
             "Return ONLY valid JSON, no markdown fences."
         )
 
+        from src.utility_effort import effort_for_call
+
         try:
             raw = await llm_call_async(
                 endpoint_url,
@@ -464,6 +470,7 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
                 temperature=0.2,
                 max_tokens=2000,
                 headers=headers,
+                reasoning_effort=effort_for_call(endpoint_url, model, user, sess),
             )
 
             # Parse JSON

routes/session_routes.py

@@ -509,6 +509,7 @@ def setup_session_routes(
                     "has_documents": s.id in doc_session_ids,
                     "has_images": s.id in img_session_ids,
                     "mode": s.mode,
+                    "thinking_mode": s.thinking_mode or "off",
                     "message_count": s.message_count or 0,
                 })
         finally:
@@ -1042,6 +1043,7 @@ def setup_session_routes(
                     "id": s.id,
                     "name": s.name,
                     "model": s.model,
+                    "thinking_mode": s.thinking_mode or "off",
                     "message_count": s.message_count or 0,
                     "created_at": s.created_at.isoformat() if s.created_at else None,
                     "updated_at": s.updated_at.isoformat() if s.updated_at else None,
@@ -1238,9 +1240,16 @@ def setup_session_routes(
         from src.context_compactor import SELF_SUMMARY_SYSTEM_PROMPT
         from src.endpoint_resolver import resolve_endpoint
         from src.llm_core import llm_call_async
+        from src.utility_effort import effort_for_call
 
         owner = getattr(session, "owner", None) or effective_user(request)
-        url, model, headers = resolve_endpoint("utility", owner=owner)
+        url, model, headers = resolve_endpoint(
+            "utility",
+            fallback_url=getattr(session, "endpoint_url", None),
+            fallback_model=getattr(session, "model", None),
+            fallback_headers=getattr(session, "headers", None),
+            owner=owner,
+        )
         if not url or not model:
             url, model, headers = session.endpoint_url, session.model, session.headers
         if not url or not model:
@@ -1268,6 +1277,7 @@ def setup_session_routes(
                 max_tokens=1024,
                 headers=headers,
                 timeout=60,
+                reasoning_effort=effort_for_call(url, model, owner, session),
             )
         except Exception as e:
             logger.error("Manual compaction failed: %s", e)
@@ -1311,6 +1321,7 @@ def setup_session_routes(
         users can clean junk without spending tokens.
         """
         from src.llm_core import llm_call
+        from src.utility_effort import effort_for_call
         user = effective_user(request)
         single_user_mode = not user and _auth_disabled()
         user_sessions = session_manager.get_sessions_for_user(user)
@@ -1483,7 +1494,8 @@ def setup_session_routes(
             # reasoning model spends tokens thinking first — 4096 truncated the
             # JSON mid-output, so it never parsed ("invalid JSON for auto-sort").
             raw = llm_call(url, model, [{"role": "user", "content": prompt}],
-                           temperature=0.3, max_tokens=16384, headers=headers, timeout=120)
+                           temperature=0.3, max_tokens=16384, headers=headers, timeout=120,
+                           reasoning_effort=effort_for_call(url, model, user))
             logger.info(f"Auto-sort raw response ({len(raw)} chars): {raw[:300]}")
             # Extract JSON from response — handle markdown fences, leading text,
             # reasoning-model <think> blocks, and trailing commas.

routes/skills_routes.py

@@ -21,6 +21,7 @@ from services.memory.skills import SkillsManager
 from src.auth_helpers import get_current_user
 from src.prompt_security import untrusted_context_message
 from src.text_helpers import strip_closed_think_blocks
+from src.utility_effort import effort_for_call
 from core.middleware import require_admin
 
 logger = logging.getLogger(__name__)
@@ -170,7 +171,7 @@ async def _eval_skill_run(skill_md: str, task: str, transcript: str,
                           baseline_transcript: str = "",
                           skill_stats: Optional[dict] = None,
                           baseline_stats: Optional[dict] = None,
-                          workload: str = "foreground") -> dict:
+                          workload: str = "foreground", owner: Optional[str] = None) -> dict:
     """LLM-as-judge: grade a skill test run from its transcript. Advisory only.
 
     Robust against local reasoning models (strips <think>, lenient JSON,
@@ -349,6 +350,7 @@ async def _eval_skill_run(skill_md: str, task: str, transcript: str,
                 url, model, msgs,
                 temperature=0.1, max_tokens=32768, headers=headers, timeout=180,
                 workload=workload,
+                reasoning_effort=effort_for_call(url, model, owner),
             )
         except Exception as e:
             # Don't give up on a transient first-attempt error — let the second
@@ -368,7 +370,7 @@ async def _eval_skill_run(skill_md: str, task: str, transcript: str,
 
 async def _eval_skill_necessity(skill_md: str, others: list, url: str, model: str,
                                 headers: Optional[dict],
-                                workload: str = "foreground") -> Optional[dict]:
+                                workload: str = "foreground", owner: Optional[str] = None) -> Optional[dict]:
     """Advisory judge: is this skill worth keeping, or is it redundant / trivially
     unnecessary? Sees the OTHER skills' names+descriptions so it can spot
     duplicates. Returns {necessary, redundant_with, reason} or None. Never acts —
@@ -401,6 +403,7 @@ async def _eval_skill_necessity(skill_md: str, others: list, url: str, model: st
             [{"role": "system", "content": sys_prompt}, {"role": "user", "content": user_msg}],
             temperature=0.1, max_tokens=8192, headers=headers, timeout=120,
             workload=workload,
+            reasoning_effort=effort_for_call(url, model, owner),
         )
     except Exception as e:
         logger.warning(f"Necessity check failed: {e}")
@@ -453,7 +456,7 @@ def _should_check_retrieval_precision(skill: dict) -> bool:
 async def _eval_skill_retrieval_precision(skill_md: str, others: list,
                                           url: str, model: str,
                                           headers: Optional[dict],
-                                          workload: str = "foreground") -> Optional[dict]:
+                                          workload: str = "foreground", owner: Optional[str] = None) -> Optional[dict]:
     """Advisory judge: would this skill's metadata make retrieval over-select it?
 
     This is distinct from "does the procedure work?". It asks whether tags,
@@ -491,6 +494,7 @@ async def _eval_skill_retrieval_precision(skill_md: str, others: list,
             [{"role": "system", "content": sys_prompt}, {"role": "user", "content": user_msg}],
             temperature=0.1, max_tokens=4096, headers=headers, timeout=90,
             workload=workload,
+            reasoning_effort=effort_for_call(url, model, owner),
         )
     except Exception as e:
         logger.warning(f"Retrieval precision check failed: {e}")
@@ -667,6 +671,7 @@ async def _run_skill_test_job(
             baseline_transcript=baseline_transcript,
             skill_stats=skill_stats,
             baseline_stats=baseline_stats,
+            owner=owner,
         )
     except Exception as e:
         job["verdict"] = {"verdict": "unknown", "confidence": 0, "summary": f"Eval failed: {e}", "issues": []}
@@ -1157,12 +1162,13 @@ async def _run_skill_test_once(md: str, task: str, url, model, headers, owner,
         headers,
         skill_stats=stats,
         workload=workload,
+        owner=owner,
     )
     return text, verdict
 
 
 async def _improve_skill_md(skill_md: str, verdict: dict, transcript: str, url, model, headers,
-                            workload: str = "foreground"):
+                            workload: str = "foreground", owner: Optional[str] = None):
     """Have a model rewrite SKILL.md to fix the reviewer's issues. Returns the
     corrected markdown, or None if it couldn't produce a usable change."""
     import re as _re
@@ -1192,7 +1198,8 @@ async def _improve_skill_md(skill_md: str, verdict: dict, transcript: str, url,
                                    [{"role": "system", "content": sys_prompt},
                                     {"role": "user", "content": user_msg}],
                                    temperature=0.2, max_tokens=16384, headers=headers, timeout=180,
-                                   workload=workload)
+                                   workload=workload,
+                                   reasoning_effort=effort_for_call(url, model, owner))
     except Exception as e:
         logger.warning(f"Audit: improve call failed: {e}")
         return None
@@ -1262,6 +1269,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
         ]
         nec = await _eval_skill_necessity(
             md, others, url, model, headers, workload=workload,
+            owner=owner,
         )
         if nec is not None:
             skills_manager.set_necessity(name, nec.get("necessary", True),
@@ -1278,6 +1286,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
         if _should_check_retrieval_precision(skill):
             rp = await _eval_skill_retrieval_precision(
                 md, others, url, model, headers, workload=workload,
+                owner=owner,
             )
             if rp and not rp.get("ok"):
                 issues = rp.get("issues") or ["metadata: retrieval: narrow tags and when_to_use to the intended trigger"]
@@ -1288,7 +1297,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
                     "summary": rp.get("summary") or "Retrieval metadata is too broad.",
                     "issues": issues,
                 }, "Retrieval audit only: the procedure may work, but matching metadata is too broad.",
-                    url, model, headers, workload=workload)
+                    url, model, headers, workload=workload, owner=owner)
                 if fixed and fixed.strip() != md.strip() and _apply_skill_md(skills_manager, name, fixed, owner):
                     md = fixed
                     refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
@@ -1364,6 +1373,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
             skill_stats=skill_stats,
             baseline_stats=baseline_stats,
             workload=workload,
+            owner=owner,
         )
     v = verdict.get("verdict")
     log(f"{name}: verdict = {v} ({verdict.get('summary', '')[:80]})")
@@ -1397,6 +1407,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
             log(f"{name}: pass, but fixing {len(meta_issues)} metadata issue(s)…")
             fixed = await _improve_skill_md(
                 md, verdict, transcript, url, model, headers, workload=workload,
+                owner=owner,
             )
             if fixed and fixed.strip() != md.strip():
                 _apply_skill_md(skills_manager, name, fixed, owner)
@@ -1430,6 +1441,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
     log(f"{name}: self-editing to fix issues…")
     new_md = await _improve_skill_md(
         md, verdict, transcript, url, model, headers, workload=workload,
+        owner=owner,
     )
     if new_md and new_md.strip() != md.strip() and _apply_skill_md(skills_manager, name, new_md, owner):
         md = new_md
@@ -1464,6 +1476,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
         log(f"{name}: teacher {t_model} rewriting the skill…")
         t_md = await _improve_skill_md(
             md, verdict, transcript, t_url, t_model, t_headers, workload=workload,
+            owner=owner,
         )
         if t_md and t_md.strip() != md.strip() and _apply_skill_md(skills_manager, name, t_md, owner):
             md = t_md