feat(settings): opt-in reasoning effort for utility model calls (#6572)

* feat(settings): opt-in reasoning effort for utility model calls

Add utility_reasoning_effort (empty = provider default) and let utility
calls inherit a chat session's effort only on that session's matching
endpoint+model route. Explicit caller effort wins; otherwise the utility
setting wins over session inheritance on utility-chain routes. Resolve
the chain and setting per user and validate levels against model evidence.
No-session work uses the setting only, with no chat-effort crossover.

- src/utility_effort.py owns effort_for_call and candidate_effort_factory
- remove global effort fill from llm_call/llm_call_async; utility call
  sites pass effort explicitly, so unrelated calls on the same route do
  not pick it up
- wire auto-title, memory/persona/skill extraction, memory suggest/import/
  audit, compaction, auto-sort, and per-owner background task candidates
- wire no-session email writing-style, summary, and translation calls,
  plus skills evaluation, necessity/retrieval audits, and improvement
- deliberately exclude email replies because thinking_mode='off'
- add the optional candidate_request_factory to llm_call_async_with_fallback
- add sync llm_call reasoning_effort and include it in the cache key
- show the Utility Model effort selector only for advertised levels

Fixes #6569

* fix(utility): exclude email replies and support inherited effort in settings

Pass reasoning_effort='' to task_llm_call_async for scheduled reply drafting to honor the email reply exclusion. In AI settings, resolve the effective Default Chat endpoint and model when Utility is set to 'Same as chat' so supported reasoning effort levels can be selected.

* feat(utility): support same-as-chat effort and session model fallback in compaction

In AI settings, show 'Same as chat' as the default effort option when Utility Model is inherited, and support an explicit 'Provider default' (off) override. Pass the active session fallback in context compaction callers so compaction shares the live chat session's model and reasoning effort for maximum prompt/KV cache affinity.

* fix(settings): clarify chat effort inheritance and keep option visible

Label inherited effort as 'Same as chat (when supported)' and mention
cache affinity and latency speedups in the help text. Ensure the effort
control remains visible when the utility model is set to 'Same as chat'
even if the default chat model lacks advertised reasoning levels, and
populate candidate levels across configured endpoints so the user can
configure effort or keep provider default.

* feat(settings): per-fallback reasoning effort for utility model

Each utility fallback row gets an Effort select, shown only when the
selected model advertises reasoning levels. The choice is stored as
reasoning_effort on the utility_model_fallbacks entry and sent only to
that candidate; an empty value means provider default.

* fix(utility): transmit reasoning effort in non-streaming calls

---------

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>

src/context_compactor.py

@@ -13,6 +13,7 @@ from typing import Any, Dict, List, Optional
 from src.model_context import estimate_text_tokens, get_context_length, estimate_tokens
 from src.llm_core import llm_call_async
 from src.endpoint_resolver import resolve_endpoint
+from src.utility_effort import effort_for_call
 from src.settings import get_setting
 from core.models import ChatMessage
 
@@ -556,7 +557,13 @@ async def maybe_compact(
     )
 
     # Use utility model if configured, otherwise fall back to session model
-    util_url, util_model, util_headers = resolve_endpoint("utility", owner=owner)
+    util_url, util_model, util_headers = resolve_endpoint(
+        "utility",
+        fallback_url=endpoint_url,
+        fallback_model=model,
+        fallback_headers=headers,
+        owner=owner,
+    )
     compact_url = util_url or endpoint_url
     compact_model = util_model or model
     compact_headers = util_headers if util_url else headers
@@ -580,6 +587,7 @@ async def maybe_compact(
             max_tokens=SUMMARY_MAX_TOKENS,
             headers=compact_headers,
             timeout=30,
+            reasoning_effort=effort_for_call(compact_url, compact_model, owner, session),
         )
     except Exception as e:
         logger.error(f"Compaction summary failed: {e}")

src/llm_core.py

@@ -2268,10 +2268,25 @@ def normalize_model_id(
             return a
     return None
 
+
+def _apply_explicit_nonstream_reasoning_effort(
+    payload: Dict, reasoning_effort: Optional[str]
+) -> None:
+    """Apply explicit effort to an OpenAI-compatible non-streaming request.
+
+    No explicit effort preserves existing provider-specific defaults.
+    Native Ollama, Anthropic and ChatGPT Responses use separate builders.
+    """
+    effort = str(reasoning_effort or "").strip().lower()
+    if effort and effort != "default":
+        payload["reasoning_effort"] = effort
+
+
 def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LLMConfig.DEFAULT_TEMPERATURE,
              max_tokens: int = LLMConfig.DEFAULT_MAX_TOKENS, headers: Optional[Dict] = None,
              timeout: int = LLMConfig.DEFAULT_TIMEOUT, prompt_type: Optional[str] = None,
-             thinking_mode: Optional[str] = None) -> str:
+             thinking_mode: Optional[str] = None,
+             reasoning_effort: Optional[str] = None) -> str:
     """Synchronous LLM call with optional prompt type enhancement."""
     h = _provider_headers(_detect_provider(url))
     # Tolerate headers that arrive as a JSON string (some sessions stored them
@@ -2303,7 +2318,7 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
     provider = _detect_provider(url)
     cache_key = _get_cache_key(
         url, model, messages_copy, temperature, max_tokens, headers=headers,
-        thinking_mode=thinking_mode,
+        thinking_mode=thinking_mode, reasoning_effort=reasoning_effort,
     )
     cached_response = _get_cached_response(cache_key)
     if cached_response:
@@ -2343,6 +2358,7 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
         )
         if provider == "mistral" and _supports_thinking(model):
             payload["reasoning_effort"] = _MISTRAL_REASONING_EFFORT
+        _apply_explicit_nonstream_reasoning_effort(payload, reasoning_effort)
     try:
         note_model_activity(target_url, model)
         r = httpx_post_kimi_aware(target_url, h, json=payload, timeout=timeout)
@@ -2465,14 +2481,30 @@ def llm_call_with_fallback(candidates, messages, **kwargs) -> str:
 
 
 async def llm_call_async_with_fallback(candidates, messages, **kwargs) -> str:
-    """Async variant of `llm_call_with_fallback` — same semantics."""
+    """Async variant of `llm_call_with_fallback` — same semantics.
+
+    An optional ``candidate_request_factory(index, url, model, headers)`` may
+    return ``{"messages": ..., "kwargs": {...}}`` to adjust the request per
+    candidate (same contract as `llm_call_async_with_route_fallback`).
+    """
+    candidate_request_factory = kwargs.pop("candidate_request_factory", None)
     cands = dedupe_model_candidates(candidates)
     if not cands:
         raise HTTPException(503, "No model endpoint configured")
     last_err = None
     for i, (url, model, headers) in enumerate(cands):
         try:
-            return await llm_call_async(url, model, messages, headers=headers, **kwargs)
+            candidate_messages = messages
+            candidate_kwargs = kwargs
+            if candidate_request_factory is not None:
+                request = candidate_request_factory(i, url, model, headers) or {}
+                if hasattr(request, "__await__"):
+                    request = await request
+                candidate_messages = request.get("messages", messages)
+                candidate_kwargs = {**kwargs, **(request.get("kwargs") or {})}
+            return await llm_call_async(
+                url, model, candidate_messages, headers=headers, **candidate_kwargs
+            )
         except Exception as e:
             last_err = e
             tag = "primary" if i == 0 else "candidate"
@@ -2740,6 +2772,7 @@ async def llm_call_async(
         _apply_deepseek_v4_reasoning_defaults(
             payload, target_url, model, thinking_mode
         )
+        _apply_explicit_nonstream_reasoning_effort(payload, reasoning_effort)
 
     if _is_host_dead(target_url):
         raise HTTPException(503, f"Upstream {_host_key(target_url)} marked unreachable (cooldown active)")

src/settings.py

@@ -160,6 +160,10 @@ DEFAULT_SETTINGS = {
     # Ordered fallback chain for the Utility model (summarization, naming,
     # tidy actions, etc.).
     "utility_model_fallbacks": [],
+    # Opt-in reasoning effort for utility/background LLM calls. Empty keeps the
+    # provider default; a value is sent only to utility-chain candidates whose
+    # model advertises that level.
+    "utility_reasoning_effort": "",
     "teacher_model": "",
     "teacher_enabled": False,
     "teacher_tier2_enabled": False,
@@ -290,6 +294,7 @@ _PER_USER_KEYS = {
     # got injected into the chat composer on first open.
     "default_endpoint_id", "default_model",
     "utility_endpoint_id", "utility_model", "utility_model_fallbacks",
+    "utility_reasoning_effort",
     "research_endpoint_id", "research_model",
 }
 

src/task_endpoint.py

@@ -106,4 +106,11 @@ async def task_llm_call_async(
         raise RuntimeError("No LLM endpoint available for background task")
     await wait_for_interactive_quiet("background task LLM")
     kwargs.setdefault("workload", "background")
+    if kwargs.get("reasoning_effort") is None:
+        kwargs.pop("reasoning_effort", None)
+        from src.utility_effort import candidate_effort_factory
+
+        factory = candidate_effort_factory(owner)
+        if factory is not None:
+            kwargs.setdefault("candidate_request_factory", factory)
     return await llm_call_async_with_fallback(candidates, messages=messages, **kwargs)

src/utility_effort.py

@@ -0,0 +1,150 @@
+"""Reasoning effort for utility-role LLM calls.
+
+One owner for the rule.  Precedence for a utility call:
+
+1. an explicit caller effort (never touched here);
+2. the ``utility_reasoning_effort`` setting, sent only to routes on the
+   utility chain (the Utility endpoint/model plus its fallbacks, matched by
+   ``(url, model)``);
+3. the chat session's own effort (``thinking_mode == "effort:<level>"``), when
+   the call is made for a chat session and its route is that session's route.
+
+Every level is validated against what the model advertises; a model without
+evidence gets nothing.  Calls with no session only ever use (2).
+"""
+from typing import Optional
+
+
+def utility_routes(owner: Optional[str] = None) -> set:
+    """The ``(url, model)`` routes that make up the utility chain."""
+    from src.endpoint_resolver import resolve_endpoint, resolve_utility_fallback_candidates
+
+    return {
+        (url, model)
+        for url, model, _headers in [
+            resolve_endpoint("utility", owner=owner),
+            *resolve_utility_fallback_candidates(owner=owner),
+        ]
+        if url and model
+    }
+
+
+def configured_effort(owner: Optional[str] = None) -> str:
+    """The raw setting, lowercased; empty when unset (the default)."""
+    from src.settings import get_user_setting
+
+    return str(get_user_setting("utility_reasoning_effort", owner or "", "") or "").strip().lower()
+
+
+def fallback_route_efforts(owner: Optional[str] = None) -> dict[tuple[str, str], str]:
+    """Configured effort per fallback candidate in ``utility_model_fallbacks``."""
+    from src.endpoint_resolver import resolve_endpoint_by_id
+    from src.settings import get_user_setting, load_settings
+
+    try:
+        settings = load_settings()
+        chain = get_user_setting(
+            "utility_model_fallbacks",
+            owner or "",
+            settings.get("utility_model_fallbacks") or [],
+        ) or []
+    except Exception:
+        return {}
+
+    res = {}
+    for entry in chain:
+        if not isinstance(entry, dict) or "reasoning_effort" not in entry:
+            continue
+        effort = str(entry.get("reasoning_effort") or "").strip().lower()
+        ep = resolve_endpoint_by_id(
+            entry.get("endpoint_id", ""),
+            entry.get("model", ""),
+            owner=owner,
+        )
+        if ep:
+            url, model, _ = ep
+            if url and model:
+                res[(url, model)] = effort
+    return res
+
+
+def effort_for_route(url: str, model: str, owner: Optional[str] = None) -> Optional[str]:
+    """Validated effort for ``(url, model)``, or None when it must not be sent."""
+    fb_efforts = fallback_route_efforts(owner)
+    if (url, model) in fb_efforts:
+        fb_effort = fb_efforts[(url, model)]
+        if not fb_effort or fb_effort in {"off", "none", "default"}:
+            return None
+        from src.chatgpt_subscription import validate_reasoning_effort
+
+        return validate_reasoning_effort(model, fb_effort)
+
+    effort = configured_effort(owner)
+    if not effort or (url, model) not in utility_routes(owner):
+        return None
+    from src.chatgpt_subscription import validate_reasoning_effort
+
+    return validate_reasoning_effort(model, effort)
+
+
+def session_effort(session) -> Optional[str]:
+    """The raw effort level the chat session is set to, or None."""
+    mode = str(getattr(session, "thinking_mode", "") or "")
+    if not mode.startswith("effort:"):
+        return None
+    return mode[len("effort:"):].strip().lower() or None
+
+
+def _is_session_route(url: str, model: str, session) -> bool:
+    from src.endpoint_resolver import same_endpoint_base
+
+    return bool(
+        model
+        and model == getattr(session, "model", None)
+        and same_endpoint_base(url, getattr(session, "endpoint_url", None))
+    )
+
+
+def effort_for_call(url: str, model: str, owner: Optional[str] = None, session=None) -> Optional[str]:
+    """Effort an owner-aware utility call should send to ``(url, model)``.
+
+    ``session`` is the chat session the call is made for, or None for
+    session-less work (scheduled tasks, pollers, auto-sort).  Never raises:
+    a lookup failure means "send nothing".
+    """
+    try:
+        raw = configured_effort(owner)
+        if raw in {"off", "none", "default"}:
+            return None
+        effort = effort_for_route(url, model, owner)
+        if effort or session is None:
+            return effort
+        level = session_effort(session)
+        if not level or not _is_session_route(url, model, session):
+            return None
+        from src.chatgpt_subscription import validate_reasoning_effort
+
+        return validate_reasoning_effort(model, level)
+    except Exception:
+        return None
+
+
+def candidate_effort_factory(owner=None):
+    """Per-candidate factory for `llm_call_async_with_fallback`.
+
+    Returns None when neither the setting nor any fallback effort is configured.
+    Otherwise a factory sending the effort only to utility-chain candidates
+    (owner-scoped) whose model advertises it.
+    """
+    raw = configured_effort(owner)
+    has_primary = bool(raw and raw not in {"off", "none", "default"})
+    fb_efforts = fallback_route_efforts(owner)
+    has_fb = any(eff and eff not in {"off", "none", "default"} for eff in fb_efforts.values())
+    if not has_primary and not has_fb:
+        return None
+
+    def factory(_index, url, model, _headers):
+        effort = effort_for_route(url, model, owner)
+        return {"kwargs": {"reasoning_effort": effort}} if effort else {}
+
+    return factory