fix: 2FA bypassed when enabled but TOTP secret is missing (fail-open) (#1286)

* fix: fail closed when 2FA is enabled but the TOTP secret is missing

* test: totp_verify fails closed when secret missing, passes when 2FA off

.env.example

@@ -16,6 +16,10 @@ LLM_HOST=localhost
 # when started with OLLAMA_HOST=0.0.0.0:11434.
 # OLLAMA_BASE_URL=http://host.docker.internal:11434/v1
 
+# Optional LM Studio URL. In Docker, host LM Studio is reachable here
+# when LM Studio is set to serve on all interfaces (0.0.0.0).
+# LM_STUDIO_URL=http://host.docker.internal:1234
+
 # OpenAI API key (only needed if using OpenAI models).
 # Do not commit real keys. Keep this commented until needed.
 # OPENAI_API_KEY=your_openai_api_key_here
@@ -145,7 +149,8 @@ SEARXNG_INSTANCE=http://localhost:8080
 #
 # AMD ROCm (requires ROCm drivers on the host and the GID of the render group):
 # COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
-# RENDER_GID=992
+# Find the render GID with: getent group render | cut -d: -f3
+# RENDER_GID=989
 #
 # These overlays only expose the GPU devices. The slim Odysseus image
 # still needs CUDA/ROCm userspace via Cookbook -> Dependencies (vLLM,

README.md

@@ -77,8 +77,10 @@ python setup.py
 python -m uvicorn app:app --host 127.0.0.1 --port 7000
 ```
 Requirements: Python 3.11+. Cookbook also needs `tmux` for background model
-downloads and serves. Use `--host 0.0.0.0` only when you intentionally want
-LAN/reverse-proxy access.
+downloads and serves. The app itself is lightweight; local model serving is the
+heavy part and depends on the model, runtime, GPU, and VRAM, so small hosts can
+connect to API or remote model servers instead. Use `--host 0.0.0.0` only when
+you intentionally want LAN/reverse-proxy access.
 
 ### Apple Silicon
 Docker on macOS cannot use the Metal GPU. For GPU-accelerated Cookbook on an
@@ -97,7 +99,11 @@ ODYSSEUS_HOST=0.0.0.0 ./start-macos.sh
 # then open http://<tailscale-ip>:7860
 ```
 
-Keep auth enabled when binding outside loopback, and do not expose this port directly to the public internet. To build a clickable app wrapper:
+The script also reads `.env` at startup, so `APP_BIND=0.0.0.0` and `APP_PORT`
+set there are picked up automatically without a command-line override each run.
+
+Keep `AUTH_ENABLED=true` (the default) before binding outside loopback. Do not
+expose this port directly to the public internet. To build a clickable app wrapper:
 
 ```bash
 ./build-macos-app.sh
@@ -124,11 +130,13 @@ Odysseus SSH key and add the public key to the remote server's
 ssh-copy-id -i data/ssh/id_ed25519.pub user@server
 ```
 
-**NVIDIA Docker GPU overlay.** CPU-only users can skip this section.
-`scripts/check-docker-gpu.sh` diagnoses GPU passthrough and can optionally
-install the host runtime or update `.env`. Cookbook can only detect GPUs that
-Docker exposes to the container — if the host runtime is not configured,
-Cookbook sees the iGPU, another card, or CPU instead of your NVIDIA GPU.
+**Docker GPU overlays.** CPU-only users can skip this section. Cookbook can
+only detect GPUs that Docker exposes to the container — if the host runtime or
+device passthrough is not configured, Cookbook sees the iGPU, another card, or
+CPU instead of your intended GPU.
+
+For NVIDIA, `scripts/check-docker-gpu.sh` diagnoses GPU passthrough and can
+optionally install the host runtime or update `.env`.
 
 ```bash
 # Read-only diagnostic (default — installs nothing, never edits .env):
@@ -162,17 +170,27 @@ To enable manually without the script, add this to `.env`:
 COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
 ```
 
-**AMD / ROCm.** AMD GPU passthrough is not automated. Add manually:
+**AMD / ROCm.** AMD setup is read-only diagnostic plus manual `.env` edit. Run:
+
+```bash
+scripts/check-docker-amd-gpu.sh
+```
+
+Then add the reported values to `.env`, replacing `RENDER_GID` with your host's
+numeric render group id:
 
 ```bash
 COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
+RENDER_GID=989
 ```
 
+For NVIDIA/AMD GPU support, also read the comments in the selected overlay file: docker/gpu.nvidia.yml or docker/gpu.amd.yml.
+
 Verify after enabling either overlay:
 
 ```bash
 docker compose exec odysseus nvidia-smi -L   # NVIDIA
-docker compose exec odysseus rocm-smi        # AMD
+docker compose exec odysseus sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls -l /dev/kfd /dev/dri/renderD*'  # AMD
 ```
 
 > **GPU passthrough ≠ llama.cpp CUDA.** `nvidia-smi` passing inside the
@@ -182,6 +200,11 @@ docker compose exec odysseus rocm-smi        # AMD
 > tensors/layers assigned to CPU, that is a Cookbook/llama.cpp build issue —
 > not a Docker passthrough failure. Re-install the serve engine via
 > **Cookbook → Dependencies** to get a CUDA-enabled build.
+>
+> The same split applies to AMD/ROCm: seeing `/dev/kfd` and `/dev/dri` inside
+> the container confirms device passthrough, not ROCm userspace or a
+> ROCm-enabled vLLM/llama.cpp build. `rocm-smi` and `rocminfo` are not expected
+> inside the slim Odysseus image.
 
 **Ollama with Docker.** If Ollama runs on the host, add this endpoint in
 Settings:

ROADMAP.md

@@ -37,6 +37,12 @@ the codebase, you are probably right to stay away.
   fetched pages, and memories should be treated as untrusted data. Keep testing
   whether models follow malicious instructions from those surfaces.
 - Better degraded-state reporting for ChromaDB, SearXNG, email, ntfy, and provider probes.
+- Email performance audit. Fetching, searching, opening, deleting, and sending
+  email can feel slow, especially over IMAP/SMTP providers with high latency.
+  Need someone who knows mail performance to profile the current flow, identify
+  whether the bottleneck is IMAP folder select/fetch, cache invalidation,
+  attachment/body loading, SMTP handshakes, or frontend refresh behavior, then
+  propose safer caching/prefetch/batching without breaking multi-account state.
 - Provider setup/probing audit for Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, and DeepSeek.
 
 ## Refactor Targets

THREAT_MODEL.md

@@ -0,0 +1,81 @@
+# Threat Model
+
+Odysseus is a **self-hosted AI workspace with privileged local access**. This document states the trust boundary so contributors can reason about security decisions without reading through the full auth and middleware stack.
+
+## Trust Boundary
+
+Odysseus is designed for **trusted users on a private network**, not public exposure. The README describes it as "treat it like an admin console" — that framing is accurate. A logged-in admin can execute shell commands, read and write files, send email, and control model serving. This is intentional. The threat model does not try to prevent admins from doing these things. It does try to prevent:
+
+- Unauthenticated access
+- Non-admins reaching admin-only capabilities
+- The AI agent acting on instructions injected through untrusted content (web results, emails, fetched pages, memories)
+- Internal services (ChromaDB, Ollama, SearXNG, etc.) being reachable from outside the host
+
+## Roles and Capabilities
+
+| Capability | Admin | Non-admin (default) |
+|---|---|---|
+| Chat with agent | ✓ | ✓ |
+| Browser tool | ✓ | ✓ |
+| Documents | ✓ | ✓ |
+| Research mode | ✓ | ✓ |
+| Image generation | ✓ | ✓ |
+| Memory management | ✓ | ✓ |
+| Shell / Python execution | ✓ | ✗ |
+| File read / write | ✓ | ✗ |
+| Email send / read | ✓ | ✗ |
+| MCP tools | ✓ | ✗ |
+| Calendar management | ✓ | ✗ |
+| Token / webhook management | ✓ | ✗ |
+| Model serving | ✓ | ✗ |
+| Vault | ✓ | ✗ |
+| Settings | ✓ | ✗ |
+
+Non-admin defaults are in `core/auth.py:DEFAULT_PRIVILEGES`. Tool enforcement is in `src/tool_security.py:NON_ADMIN_BLOCKED_TOOLS`. Any tool whose name starts with `mcp__` is also blocked for non-admins. Admins always get full access regardless of stored privilege values.
+
+## Authentication
+
+- **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`.
+- **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance.
+- **Reserved usernames:** `internal-tool`, `api`, `demo`, `system` cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`.
+  - `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check.
+- **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate.
+
+## Internal Tool Loopback
+
+Agent tool calls reach admin-gated HTTP routes over an in-process HTTP loopback. The mechanism:
+
+1. At app startup, `core/middleware.py` generates a random `INTERNAL_TOOL_TOKEN` via `secrets.token_hex(32)`. It is never persisted and never sent to clients.
+2. Loopback requests carry `X-Odysseus-Internal-Token: <token>` or have `request.state.current_user` already set to `"internal-tool"` by the auth middleware.
+3. `require_admin` recognises either signal and grants access without checking the session user.
+
+The agent may be running in a non-admin user's session, but tool dispatch first calls `src/tool_security.py:owner_is_admin_or_single_user` to verify the session owner is an admin before issuing any loopback call. Non-admin users cannot invoke admin tools even via the agent.
+
+## Prompt-Injection Hardening
+
+External content that reaches the LLM is treated as untrusted via `src/prompt_security.py`:
+
+- `untrusted_context_message(label, content)` wraps the content in a `user`-role message with a header block instructing the model not to follow instructions inside it. Content goes in as data, not as a system instruction.
+- `UNTRUSTED_CONTEXT_POLICY` is a system-prompt preamble that states the same policy at the top of every session where untrusted data may appear.
+
+**Untrusted surfaces that must go through this wrapper:** web search results, fetched URLs, emails (read), saved memories, skill text, notes, and any tool output sourced from outside the server. Injecting untrusted content directly into the system role is a security bug.
+
+## Security Headers
+
+`core/middleware.py:SecurityHeadersMiddleware` sets headers on every response:
+
+- `X-Frame-Options: DENY` + `frame-ancestors 'none'` on all routes except tool-render iframes (which are sandboxed at the HTML level).
+- `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer` everywhere.
+- **CSP:** nonce-based `script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net`. `style-src 'unsafe-inline'` is intentionally kept — `static/index.html` ships inline `<style>` blocks and JS modules set `style=""` attributes at runtime. Inline styles do not execute script so the risk is visual-only. Removing this requires templating the HTML files and auditing all JS-set style attributes.
+
+## Known Gaps
+
+These are open, acknowledged, and contributor help is welcome:
+
+1. **No shell/filesystem sandbox.** The agent `bash` and `read_file`/`write_file` tools run as the app process user with no network egress filtering or filesystem confinement. A successful prompt-injection reaching a shell-enabled admin session can make outbound requests to internal services. See #1058 for the sandbox proposal.
+
+2. **SSRF via `/api/v1/chat` `base_url` parameter.** A chat-scoped API token can supply an arbitrary `base_url`; the server forwards the LLM request to that host without validating the scheme or address. PR #1039 fixes this.
+
+3. **`src/search/` partial consolidation.** `src.search.core` and `src.search.providers` correctly alias `services.search` via `sys.modules` replacement. `analytics`, `cache`, `content`, `query`, and `ranking` are still independent copies that can drift. The SSRF regression tests in `tests/test_webhook_ssrf_resilience.py` test `src.webhook_manager` directly (separate from search), so the safety net there is intact. See #1058.
+
+4. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.

app.py

@@ -772,6 +772,17 @@ async def get_version():
 async def health_check() -> Dict[str, str]:
     return {"status": "healthy", "timestamp": datetime.utcnow().isoformat()}
 
+@app.get("/api/ready")
+async def readiness_check() -> JSONResponse:
+    """Readiness / integrity self-check — DB, data dir, local-first storage.
+
+    Unlike /api/health (liveness), this returns 503 unless every critical
+    subsystem is whole, so an orchestrator can gate traffic on real readiness.
+    """
+    from src.readiness import check_readiness
+    result = check_readiness()
+    return JSONResponse(status_code=200 if result.get("ready") else 503, content=result)
+
 @app.get("/api/runtime")
 async def runtime_info() -> Dict[str, object]:
     in_docker = os.path.exists("/.dockerenv")

build-macos-app.sh

@@ -119,7 +119,11 @@ fi
 
 notify "Starting…"
 cd "$INSTALL_DIR" || die_gui "Install folder not found: $INSTALL_DIR"
-"$UVICORN" app:app --host 127.0.0.1 --port "$PORT" >>"$LOG" 2>&1 &
+if [ "$(uname -m)" = "arm64" ]; then
+  arch -arm64 "$UVICORN" app:app --host 127.0.0.1 --port "$PORT" >>"$LOG" 2>&1 &
+else
+  "$UVICORN" app:app --host 127.0.0.1 --port "$PORT" >>"$LOG" 2>&1 &
+fi
 SERVER_PID=$!
 
 # Quitting the app stops the server it started.

companion/routes.py

@@ -21,6 +21,7 @@ import html
 from fastapi import APIRouter, Request
 from fastapi.responses import HTMLResponse
 
+from core.middleware import require_admin
 from src.auth_helpers import get_current_user
 
 from companion import pairing as _pairing

core/auth.py

@@ -266,7 +266,8 @@ class AuthManager:
         renamed_sessions = 0
         with self._sessions_lock:
             for sess in self._sessions.values():
-                if (sess or {}).get("username") == old_username:
+                sess_user = str((sess or {}).get("username") or "").strip().lower()
+                if sess_user == old_username:
                     sess["username"] = new_username
                     renamed_sessions += 1
         if renamed_sessions:
@@ -375,7 +376,10 @@ class AuthManager:
             return True  # 2FA not enabled, always pass
         secret = user.get("totp_secret")
         if not secret:
-            return True
+            # 2FA is enabled but no secret is stored (corrupt/partially-written
+            # auth.json). Fail closed — returning True here bypassed the second
+            # factor entirely.
+            return False
         # Check backup codes first
         backup = user.get("totp_backup_codes", [])
         if code in backup:

core/database.py

@@ -1,7 +1,9 @@
 import os
 import logging
+import sqlite3
 from datetime import datetime
-from sqlalchemy import create_engine, Column, String, Text, Boolean, DateTime, Integer, ForeignKey, JSON, Index, func, text
+from sqlalchemy import event, create_engine, Column, String, Text, Boolean, DateTime, Integer, ForeignKey, JSON, Index, func, text
+from sqlalchemy.engine import Engine
 from sqlalchemy.types import TypeDecorator
 from sqlalchemy.ext.declarative import declarative_base, declared_attr
 from sqlalchemy.orm import relationship, sessionmaker, backref
@@ -34,6 +36,18 @@ engine = create_engine(
 SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
 
 
+# Listening on the Engine class ensures this listener fires for all Engine
+# instances created within the process, not just the primary application engine.
+# The isinstance(sqlite3.Connection) check ensures that this PRAGMA foreign_keys=ON
+# configuration remains a no-op when using non-SQLite database backends.
+@event.listens_for(Engine, "connect")
+def set_sqlite_pragma(dbapi_connection, connection_record):
+    if isinstance(dbapi_connection, sqlite3.Connection):
+        cursor = dbapi_connection.cursor()
+        cursor.execute("PRAGMA foreign_keys=ON")
+        cursor.close()
+
+
 class EncryptedText(TypeDecorator):
     """Text column transparently encrypted at rest via src.secret_storage.
 
@@ -1484,6 +1498,10 @@ def _migrate_seed_email_account():
         logging.getLogger(__name__).warning(f"seed email account migration: {e}")
 
 
+# WARNING: Foreign-key enforcement is enabled globally for all SQLite connections.
+# Any future migrations or schema changes that temporarily violate foreign-key
+# constraints will fail. To perform such operations, foreign_keys must be
+# temporarily disabled around the migration workflow.
 def init_db():
     """
     Initialize the database by creating all tables.

core/platform_compat.py

@@ -14,6 +14,7 @@ Design rules:
 from __future__ import annotations
 
 import os
+import ntpath
 import shutil
 import subprocess
 from pathlib import Path
@@ -134,11 +135,40 @@ _BASH_CACHE: Optional[str] = None
 _BASH_PROBED = False
 
 # Common Git-for-Windows install locations to probe when bash isn't on PATH.
-_WINDOWS_BASH_FALLBACKS = (
-    r"C:\Program Files\Git\bin\bash.exe",
-    r"C:\Program Files\Git\usr\bin\bash.exe",
-    r"C:\Program Files (x86)\Git\bin\bash.exe",
+_WINDOWS_BASH_ROOT_ENV_VARS = (
+    "ProgramFiles",
+    "ProgramW6432",
+    "ProgramFiles(x86)",
+    "LocalAppData",
 )
+_WINDOWS_BASH_DEFAULT_ROOTS = (
+    r"C:\Program Files\Git",
+    r"C:\Program Files (x86)\Git",
+)
+_WINDOWS_BASH_RELATIVE_PATHS = (
+    ("bin", "bash.exe"),
+    ("usr", "bin", "bash.exe"),
+)
+
+
+def _windows_bash_fallbacks() -> List[str]:
+    roots: List[str] = []
+    for env_name in _WINDOWS_BASH_ROOT_ENV_VARS:
+        base = os.environ.get(env_name)
+        if base:
+            roots.append(ntpath.join(base, "Git"))
+    roots.extend(_WINDOWS_BASH_DEFAULT_ROOTS)
+
+    paths: List[str] = []
+    seen = set()
+    for root in roots:
+        for rel in _WINDOWS_BASH_RELATIVE_PATHS:
+            path = ntpath.join(root, *rel)
+            key = path.lower()
+            if key not in seen:
+                seen.add(key)
+                paths.append(path)
+    return paths
 
 
 def find_bash() -> Optional[str]:
@@ -153,9 +183,9 @@ def find_bash() -> Optional[str]:
     if _BASH_PROBED:
         return _BASH_CACHE
     _BASH_PROBED = True
-    found = shutil.which("bash")
+    found = which_tool("bash")
     if not found and IS_WINDOWS:
-        for cand in _WINDOWS_BASH_FALLBACKS:
+        for cand in _windows_bash_fallbacks():
             if os.path.exists(cand):
                 found = cand
                 break

core/session_manager.py

@@ -29,6 +29,21 @@ def _message_timestamp_iso(value: Optional[datetime]) -> Optional[str]:
     return value.isoformat().replace("+00:00", "Z")
 
 
+def _parse_msg_content(raw):
+    """Parse message content from DB — deserialises JSON arrays back to lists
+    (multimodal content with image/audio attachments)."""
+    if isinstance(raw, list):
+        return raw
+    if isinstance(raw, str) and raw.startswith('[{') and '"type"' in raw:
+        try:
+            parsed = json.loads(raw)
+            if isinstance(parsed, list) and all(isinstance(p, dict) for p in parsed):
+                return parsed
+        except (json.JSONDecodeError, ValueError):
+            pass
+    return raw
+
+
 class SessionManager:
     """
     Manages chat sessions with database persistence.
@@ -119,7 +134,7 @@ class SessionManager:
                 meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp))
                 history.append(ChatMessage(
                     role=db_msg.role,
-                    content=db_msg.content,
+                    content=_parse_msg_content(db_msg.content),
                     metadata=meta,
                 ))
         else:
@@ -134,7 +149,7 @@ class SessionManager:
                 meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp))
                 history.append(ChatMessage(
                     role=db_msg.role,
-                    content=db_msg.content,
+                    content=_parse_msg_content(db_msg.content),
                     metadata=meta,
                 ))
 
@@ -192,11 +207,17 @@ class SessionManager:
             if message.metadata is None:
                 message.metadata = {}
             message.metadata.setdefault('timestamp', _message_timestamp_iso(msg_time))
+            # Multimodal content (image/audio attachments) is a list — serialize
+            # to JSON so the Text column can store it.  On reload, _db_to_session
+            # detects the JSON-array prefix and parses it back.
+            _content = message.content
+            if isinstance(_content, list):
+                _content = json.dumps(_content)
             db_message = DbChatMessage(
                 id=msg_id,
                 session_id=session_id,
                 role=message.role,
-                content=message.content,
+                content=_content,
                 meta_data=json.dumps(message.metadata) if message.metadata else None,
                 timestamp=msg_time,
             )
@@ -466,11 +487,17 @@ class SessionManager:
             db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
             if db_session:
                 db.delete(db_session)
+
+            # Drop the in-memory copy even when there is no DB row. A "ghost"
+            # session lives only here (never persisted, or its row was removed
+            # out-of-band); without this it can never be cleared and keeps
+            # 404ing on every operation (issue #1044).
+            removed_in_memory = self.sessions.pop(session_id, None) is not None
+
+            if db_session or removed_in_memory:
+                # Commit the document-detach / message-delete above (a no-op when
+                # the ghost had no rows) together with the session delete.
                 db.commit()
-
-                if session_id in self.sessions:
-                    del self.sessions[session_id]
-
                 logger.info(f"Deleted session {session_id}")
                 return True
             return False

docker/gpu.amd.yml

@@ -1,5 +1,6 @@
 # AMD ROCm GPU overlay. Enable by setting COMPOSE_FILE in .env:
 #   COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
+#   RENDER_GID=<numeric output of: getent group render | cut -d: -f3>
 #
 # Requires ROCm drivers on the host (kfd + DRI devices). The host user
 # running Docker must be in the `video` and `render` groups.

docs/index.html

@@ -25,7 +25,7 @@
     --radius: 8px;
   }
   * { box-sizing: border-box; }
-  html { scroll-behavior: smooth; scroll-snap-type: y mandatory; scroll-padding-top: 60px; }
+  html { scroll-behavior: smooth; scroll-snap-type: y proximity; scroll-padding-top: 60px; }
   /* Each section is a full-viewport "page" with its content centered, so only
      one shows at a time and the snap is obvious. */
   .hero, section {

launch-windows.ps1

@@ -30,6 +30,26 @@ function Fail($msg) {
     exit 1
 }
 
+function Find-GitBash {
+    $cmd = Get-Command bash -ErrorAction SilentlyContinue
+    if ($cmd) { return $cmd.Source }
+
+    $roots = @()
+    foreach ($name in @("ProgramFiles", "ProgramW6432", "ProgramFiles(x86)", "LocalAppData")) {
+        $base = [Environment]::GetEnvironmentVariable($name)
+        if ($base) { $roots += (Join-Path $base "Git") }
+    }
+    $roots += @("C:\Program Files\Git", "C:\Program Files (x86)\Git")
+
+    foreach ($root in ($roots | Select-Object -Unique)) {
+        foreach ($relative in @("bin\bash.exe", "usr\bin\bash.exe")) {
+            $candidate = Join-Path $root $relative
+            if (Test-Path $candidate) { return $candidate }
+        }
+    }
+    return $null
+}
+
 # 1. Locate a Python interpreter (3.11+ required)
 Write-Step "Checking for Python"
 function Get-PythonVersionText($launcher, $launcherArgs) {
@@ -101,7 +121,7 @@ Write-Step "Running first-time setup"
 if ($LASTEXITCODE -ne 0) { Fail "setup.py failed." }
 
 # 5. Friendly note about Git Bash (full Cookbook / agent-shell parity)
-if (-not (Get-Command bash -ErrorAction SilentlyContinue)) {
+if (-not (Find-GitBash)) {
     Write-Host ""
     Write-Host "NOTE: Git Bash (bash.exe) was not found on PATH." -ForegroundColor Yellow
     Write-Host "      The core app works without it. For full Cookbook background" -ForegroundColor Yellow

routes/admin_wipe_routes.py

@@ -27,6 +27,7 @@ from core.database import (
     Document,
     DocumentVersion,
     GalleryImage,
+    GalleryAlbum,
     CalendarEvent,
     CalendarCal,
 )
@@ -145,8 +146,9 @@ def setup_admin_wipe_routes(session_manager):
                 return {"status": "deleted", "kind": kind, "count": count}
 
             if kind == "gallery":
-                count = db.query(GalleryImage).count()
+                count = db.query(GalleryImage).count() + db.query(GalleryAlbum).count()
                 db.query(GalleryImage).delete()
+                db.query(GalleryAlbum).delete()
                 db.commit()
                 # Also drop the upload dir so disk doesn't keep orphans.
                 _rmtree_quiet(os.path.join(DATA_DIR, "gallery"))

routes/auth_routes.py

@@ -297,6 +297,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
         # owner-scoped DB rows before changing auth so the account keeps
         # access to its sessions, docs, email accounts, tasks, etc.
         try:
+            from sqlalchemy import func
             from core.database import Base, SessionLocal
             db = SessionLocal()
             try:
@@ -306,7 +307,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
                         continue
                     (
                         db.query(model)
-                        .filter(model.owner == old_username)
+                        .filter(func.lower(model.owner) == old_username)
                         .update({"owner": new_username}, synchronize_session=False)
                     )
                 db.commit()
@@ -324,9 +325,15 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
             from routes.prefs_routes import _load as _load_prefs, _save as _save_prefs
             prefs = _load_prefs()
             users = prefs.get("_users") if isinstance(prefs, dict) else None
-            if isinstance(users, dict) and old_username in users and new_username not in users:
-                users[new_username] = users.pop(old_username)
-                _save_prefs(prefs)
+            if isinstance(users, dict):
+                prefs_key = next(
+                    (k for k in users if str(k).strip().lower() == old_username),
+                    None,
+                )
+                new_taken = any(str(k).strip().lower() == new_username for k in users)
+                if prefs_key is not None and not new_taken:
+                    users[new_username] = users.pop(prefs_key)
+                    _save_prefs(prefs)
         except Exception as e:
             logger.warning("Failed to rename user prefs %s -> %s: %s", old_username, new_username, e)
 

routes/calendar_routes.py

@@ -12,7 +12,7 @@ from dateutil.rrule import rrulestr, rruleset
 from dateutil.rrule import DAILY, WEEKLY, MONTHLY, YEARLY
 
 from core.database import SessionLocal, CalendarCal, CalendarEvent
-from src.auth_helpers import get_current_user
+from src.auth_helpers import get_current_user, require_user
 
 logger = logging.getLogger(__name__)
 
@@ -28,16 +28,17 @@ _SINGLE_USER_MODE = _os.environ.get("ODYSSEUS_SINGLE_USER", "1") != "0"
 
 
 def _require_user(request: Request) -> str:
-    """Return the authenticated user. In multi-user mode an unauthenticated
-    request raises 401; in single-user mode it falls through to
-    FALLBACK_OWNER. Prevents the silent cross-user data write that would
-    happen if a request slipped past auth middleware in a real deployment."""
-    u = get_current_user(request)
-    if u:
-        return u
-    if _SINGLE_USER_MODE:
-        return FALLBACK_OWNER
-    raise HTTPException(401, "Authentication required")
+    """Return the authenticated user. Uses require_user so AUTH_ENABLED=false
+    and single-user mode both work: require_user returns "" when auth is
+    disabled or unconfigured, and only raises 401 when auth is configured but
+    the caller is unauthenticated. Falls back to FALLBACK_OWNER for calendar
+    writes so data isn't stored under an empty owner in single-user mode."""
+    user = require_user(request)
+    if user:
+        return user
+    # require_user returned "" — auth is off or unconfigured (single-user).
+    # Use FALLBACK_OWNER so calendar rows have a stable owner for filtering.
+    return FALLBACK_OWNER
 
 
 def _get_or_404_calendar(db, cal_id: str, owner: str) -> CalendarCal:
@@ -64,6 +65,24 @@ def _get_or_404_event(db, uid: str, owner: str) -> CalendarEvent:
     return ev
 
 
+def _ics_escape(text: str) -> str:
+    """Escape a value for an iCalendar TEXT field (RFC 5545 §3.3.11).
+
+    Backslash, semicolon and comma are structural in TEXT values and must be
+    escaped, and newlines become a literal ``\\n``. Backslash is escaped first
+    so the escapes we add aren't re-escaped.
+    """
+    return (
+        (text or "")
+        .replace("\\", "\\\\")
+        .replace(";", "\\;")
+        .replace(",", "\\,")
+        .replace("\r\n", "\\n")
+        .replace("\n", "\\n")
+        .replace("\r", "\\n")
+    )
+
+
 def _resolve_base_uid(uid: str) -> str:
     """Extract the base series UID from a compound occurrence UID.
 
@@ -1032,23 +1051,23 @@ def setup_calendar_routes() -> APIRouter:
                 "BEGIN:VCALENDAR",
                 "VERSION:2.0",
                 "PRODID:-//Odysseus//Calendar//EN",
-                f"X-WR-CALNAME:{cal.name}",
+                f"X-WR-CALNAME:{_ics_escape(cal.name)}",
             ]
             for ev in events:
                 lines.append("BEGIN:VEVENT")
                 lines.append(f"UID:{ev.uid}")
-                lines.append(f"SUMMARY:{ev.summary or ''}")
+                lines.append(f"SUMMARY:{_ics_escape(ev.summary or '')}")
                 if ev.all_day:
                     lines.append(f"DTSTART;VALUE=DATE:{ev.dtstart.strftime('%Y%m%d')}")
                     lines.append(f"DTEND;VALUE=DATE:{ev.dtend.strftime('%Y%m%d')}")
                 else:
-                    lines.append(f"DTSTART:{ev.dtstart.strftime('%Y%m%dT%H%M%S')}")
-                    lines.append(f"DTEND:{ev.dtend.strftime('%Y%m%dT%H%M%S')}")
+                    _dt_suffix = "Z" if getattr(ev, "is_utc", False) else ""
+                    lines.append(f"DTSTART:{ev.dtstart.strftime('%Y%m%dT%H%M%S')}{_dt_suffix}")
+                    lines.append(f"DTEND:{ev.dtend.strftime('%Y%m%dT%H%M%S')}{_dt_suffix}")
                 if ev.description:
-                    desc = ev.description.replace(chr(10), '\\n')
-                    lines.append(f"DESCRIPTION:{desc}")
+                    lines.append(f"DESCRIPTION:{_ics_escape(ev.description)}")
                 if ev.location:
-                    lines.append(f"LOCATION:{ev.location}")
+                    lines.append(f"LOCATION:{_ics_escape(ev.location)}")
                 if ev.rrule:
                     lines.append(f"RRULE:{ev.rrule}")
                 lines.append("END:VEVENT")

routes/chat_helpers.py

@@ -3,6 +3,7 @@
 import asyncio
 import json
 import logging
+import os
 import re
 from dataclasses import dataclass, field
 from typing import Any, Optional
@@ -11,6 +12,7 @@ from core.models import ChatMessage
 from core.database import SessionLocal
 from core.database import Session as DBSession, ModelEndpoint
 from src.llm_core import normalize_model_id
+from src.endpoint_resolver import normalize_base
 from src.context_compactor import maybe_compact, trim_for_context
 from src.auth_helpers import get_current_user
 from src.prompt_security import untrusted_context_message
@@ -149,6 +151,9 @@ async def auto_name_session(session_manager, sess):
         t_url, t_model, t_headers = resolve_task_endpoint(
             sess.endpoint_url, sess.model, sess.headers,
         )
+        if not t_model:
+            logger.debug("[auto-name] No model provided, skipping")
+            return
 
         # max_tokens big enough that reasoning models (Minimax M2,
         # DeepSeek R1, QwQ, etc.) have headroom for <think>…</think>
@@ -334,6 +339,63 @@ def resolve_session_auth(sess, session_id: str):
         logger.warning(f"Failed to resolve session headers: {e}")
 
 
+def _match_cached_model_id(requested: str, models) -> Optional[str]:
+    if not requested or not models:
+        return None
+    model_ids = [str(m) for m in models if m]
+    if requested in model_ids:
+        return requested
+
+    req_base = os.path.basename(requested.rstrip("/"))
+    for model_id in model_ids:
+        if os.path.basename(model_id.rstrip("/")) == req_base:
+            return model_id
+    return None
+
+
+def _normalize_model_id_from_cache(sess) -> Optional[str]:
+    """Use stored endpoint model IDs before falling back to a live /models probe."""
+    endpoint_url = getattr(sess, "endpoint_url", "") or ""
+    requested = getattr(sess, "model", "") or ""
+    if not endpoint_url or not requested:
+        return None
+
+    try:
+        session_base = normalize_base(endpoint_url)
+    except Exception:
+        session_base = endpoint_url.rstrip("/")
+    if not session_base:
+        return None
+
+    db = SessionLocal()
+    try:
+        endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
+        for ep in endpoints:
+            try:
+                if normalize_base(getattr(ep, "base_url", "") or "") != session_base:
+                    continue
+            except Exception:
+                continue
+
+            raw_models = getattr(ep, "cached_models", None)
+            if not raw_models:
+                continue
+            try:
+                models = json.loads(raw_models) if isinstance(raw_models, str) else raw_models
+            except Exception:
+                continue
+
+            matched = _match_cached_model_id(requested, models)
+            if matched:
+                return matched
+    except Exception as e:
+        logger.debug("Cached model normalization skipped: %s", e)
+    finally:
+        db.close()
+
+    return None
+
+
 async def build_chat_context(
     sess,
     request,
@@ -434,8 +496,9 @@ async def build_chat_context(
     for transcript in preprocessed.youtube_transcripts:
         preface.append(untrusted_context_message("youtube transcript", transcript))
 
-    # Normalize model ID
-    norm = normalize_model_id(sess.endpoint_url, sess.model)
+    # Normalize model ID. Prefer cached endpoint models so group chat does not
+    # re-hit slow local /models endpoints on every participant turn.
+    norm = _normalize_model_id_from_cache(sess) or normalize_model_id(sess.endpoint_url, sess.model)
     if norm:
         sess.model = norm
 

routes/chat_routes.py

@@ -28,6 +28,7 @@ from core.database import SessionLocal, get_session_mode, set_session_mode
 from core.database import Session as DBSession, ChatMessage as DBChatMessage
 from core.database import Document as DBDocument, ModelEndpoint
 from routes.research_routes import _resolve_research_endpoint
+from routes.model_routes import _visible_models
 from routes.chat_helpers import (
     resolve_session_auth,
     build_chat_context,
@@ -42,6 +43,7 @@ logger = logging.getLogger(__name__)
 
 # Track active streams for partial-save safety net
 _active_streams: Dict[str, dict] = {}
+_IMAGE_MODEL_PREFIXES = ("gpt-image", "dall-e", "chatgpt-image")
 
 
 def _stream_set(session_id: str, **fields) -> None:
@@ -97,6 +99,59 @@ def _clear_orphaned_session_endpoint(sess) -> bool:
         db.close()
 
 
+def _endpoint_cache_contains_model(endpoint, model: str) -> bool:
+    """Return True when a populated endpoint model cache includes ``model``.
+
+    Empty/malformed caches are treated as unknown rather than a negative match
+    so older image endpoints without cached models still work.
+    """
+    raw = getattr(endpoint, "cached_models", None)
+    if not raw:
+        return True
+    try:
+        models = json.loads(raw) if isinstance(raw, str) else raw
+    except Exception:
+        return True
+    if not isinstance(models, list) or not models:
+        return True
+    wanted = (model or "").strip()
+    return wanted in {str(item).strip() for item in models}
+
+
+def _is_image_generation_session(sess) -> bool:
+    """Whether this chat session should bypass text chat and generate images.
+
+    Model-name prefixes are explicit image models. Endpoint type is only used
+    when the current session endpoint actually matches that image endpoint, and
+    when a populated endpoint model cache includes the selected model. This
+    prevents an image endpoint on the same host from misrouting ordinary text
+    models into the image-generation path.
+    """
+    model = (getattr(sess, "model", "") or "").strip()
+    if any(model.lower().startswith(prefix) for prefix in _IMAGE_MODEL_PREFIXES):
+        return True
+
+    endpoint_url = (getattr(sess, "endpoint_url", "") or "").strip()
+    if not endpoint_url:
+        return False
+
+    db = SessionLocal()
+    try:
+        endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
+        for endpoint in endpoints:
+            if (getattr(endpoint, "model_type", None) or "llm") != "image":
+                continue
+            if not _session_url_matches_endpoint(endpoint_url, getattr(endpoint, "base_url", "") or ""):
+                continue
+            if _endpoint_cache_contains_model(endpoint, model):
+                return True
+    except Exception:
+        return False
+    finally:
+        db.close()
+    return False
+
+
 def _recover_empty_session_model(sess, session_id: str) -> bool:
     """Re-populate sess.model from the matching endpoint's cached models.
 
@@ -130,7 +185,13 @@ def _recover_empty_session_model(sess, session_id: str) -> bool:
             cached = []
         if not cached:
             return False
-        model = cached[0]
+        try:
+            visible = _visible_models(cached, getattr(ep, "hidden_models", None))
+        except Exception:
+            visible = cached
+        if not visible:
+            return False
+        model = visible[0]
         if not isinstance(model, str) or not model.strip():
             return False
         model = model.strip()
@@ -436,10 +497,11 @@ def setup_chat_routes(
                 else:
                     logger.warning(f"[doc-inject] NOT FOUND by ID {active_doc_id}")
             if not active_doc:
-                active_doc = _doc_db.query(DBDocument).filter(
+                _session_doc_q = _doc_db.query(DBDocument).filter(
                     DBDocument.session_id == session,
                     DBDocument.is_active == True
-                ).order_by(DBDocument.updated_at.desc()).first()
+                )
+                active_doc = _owner_session_filter(_session_doc_q, ctx.user).order_by(DBDocument.updated_at.desc()).first()
                 if active_doc:
                     logger.info(f"[doc-inject] found by session fallback: title={active_doc.title!r}")
             # Last resort: the document the agent itself just created/edited
@@ -453,7 +515,8 @@ def setup_chat_routes(
                     from src.tool_implementations import get_active_document
                     _mem_id = get_active_document()
                     if _mem_id:
-                        cand = _doc_db.query(DBDocument).filter(DBDocument.id == _mem_id).first()
+                        _mem_q = _doc_db.query(DBDocument).filter(DBDocument.id == _mem_id)
+                        cand = _owner_session_filter(_mem_q, ctx.user).first()
                         if cand and (not cand.session_id or cand.session_id == session):
                             active_doc = cand
                             logger.info(f"[doc-inject] found by in-memory active id: title={active_doc.title!r} (session_id={cand.session_id!r})")
@@ -717,28 +780,7 @@ def setup_chat_routes(
                 _model_info["character_name"] = ctx.preset.character_name
             yield f'data: {json.dumps(_model_info)}\n\n'
 
-            # Detect image models and route directly to image generation
-            _IMAGE_MODEL_PREFIXES = ("gpt-image", "dall-e", "chatgpt-image")
-            _is_image_model = any(sess.model.lower().startswith(p) for p in _IMAGE_MODEL_PREFIXES)
-
-            # Also check if the endpoint is registered as an image-type endpoint
-            if not _is_image_model:
-                try:
-                    from src.endpoint_resolver import normalize_base as _nb
-                    _ep_base = _nb(sess.endpoint_url)
-                    _db = SessionLocal()
-                    try:
-                        _is_image_model = _db.query(ModelEndpoint).filter(
-                            ModelEndpoint.model_type == "image",
-                            ModelEndpoint.is_enabled == True,
-                            ModelEndpoint.base_url.contains(_ep_base.split("://")[-1].split("/")[0]),
-                        ).first() is not None
-                    finally:
-                        _db.close()
-                except Exception:
-                    pass
-
-            if _is_image_model:
+            if _is_image_generation_session(sess):
                 from src.settings import get_setting
                 if not get_setting("image_gen_enabled", True):
                     yield f'data: {json.dumps({"delta": "Image generation is disabled by the administrator."})}\n\n'
@@ -814,6 +856,15 @@ def setup_chat_routes(
                                         pct = min(round((last_metrics["input_tokens"] / ctx.context_length) * 100, 1), 100.0)
                                         last_metrics["context_percent"] = pct
                                         last_metrics["context_length"] = ctx.context_length
+                                    # The frontend reads `tokens_per_second`; the raw usage event
+                                    # carries the backend's true gen speed as `gen_tps` (llama.cpp
+                                    # timings). Map it through so this direct-chat path shows real
+                                    # t/s instead of "n/a" → falling back to a bare token count.
+                                    if last_metrics.get("gen_tps") and not last_metrics.get("tokens_per_second"):
+                                        last_metrics["tokens_per_second"] = last_metrics["gen_tps"]
+                                        last_metrics["tps_source"] = "backend"
+                                    # Wall-clock response time for the stats popup ("Time").
+                                    last_metrics.setdefault("response_time", round(time.time() - _chat_start, 2))
                                     yield f'data: {json.dumps({"type": "metrics", "data": last_metrics})}\n\n'
                             except json.JSONDecodeError:
                                 yield chunk

routes/cookbook_helpers.py

@@ -148,15 +148,39 @@ def _local_tooling_path_export(executable: str) -> str:
     return f'export PATH="{esc}:$PATH"'
 
 
-def _pip_install_fallback_chain(package: str, *, python_cmd: str = "python3 -m pip", upgrade: bool = False) -> str:
-    """Build a bash pip install fallback chain.
+def _pip_install_attempt(pip_cmd: str) -> str:
+    """Wrap a single pip install command so its exit status survives the
+    fallback chain and its stderr is visible in the tmux log on failure.
 
-    Try the active interpreter/environment first. `--user` is invalid inside
-    many venvs, so only attempt the --user fallback when NOT inside a venv.
+    Without this wrapper, `pip … 2>&1 | tail -5` returns ``tail``'s exit
+    code (0), masking pip's real failure and preventing the next fallback
+    from running.  The generated snippet captures all output to a temp
+    file, prints the last 5 lines on failure (so the Cookbook log panel
+    shows useful diagnostics), cleans up, and exits with pip's original
+    status.
+    """
+    return (
+        "bash -c '"
+        f'_out=$(mktemp) && {pip_cmd} >"$_out" 2>&1; _rc=$?; '
+        'tail -5 "$_out"; rm -f "$_out"; exit $_rc'
+        "'"
+    )
+
+
+def _pip_install_fallback_chain(package: str, *, python_cmd: str = "python3 -m pip", upgrade: bool = False) -> str:
+    """Build a bash pip install fallback chain that surfaces errors.
+
+    Try the active interpreter/environment first. ``--user`` is invalid
+    inside many venvs, so only attempt the ``--user`` fallback when NOT
+    inside a venv.
+
+    Each attempt is wrapped via :func:`_pip_install_attempt` so pip's real
+    exit code is preserved (no ``| tail`` masking) and the last 5 lines of
+    pip output appear in the Cookbook log on failure.
     """
     upgrade_flag = " -U" if upgrade else ""
-    base = f"{python_cmd} install -q{upgrade_flag} {package} 2>/dev/null"
-    user = f"{python_cmd} install --user --break-system-packages -q{upgrade_flag} {package} 2>/dev/null"
+    base = _pip_install_attempt(f"{python_cmd} install -q{upgrade_flag} {package}")
+    user = _pip_install_attempt(f"{python_cmd} install --user --break-system-packages -q{upgrade_flag} {package}")
     # Derive the python executable for the venv detection check.
     # Must use the same interpreter that pip belongs to; hardcoding
     # python3 breaks when pip lives in a venv that only has "python".
@@ -169,8 +193,41 @@ def _pip_install_fallback_chain(package: str, *, python_cmd: str = "python3 -m p
     else:
         python_exe = "python3"
     venv_check = f'{python_exe} -c "import sys; sys.exit(0 if sys.prefix != sys.base_prefix else 1)"'
-    # venv_check exits 0 (true) when IN a venv; --user is only valid outside one.
-    return f"{base} || {{ {venv_check} || {user}; }}"
+    # Negated: `! venv_check` succeeds (exit 0) when NOT in a venv → `&&` tries
+    # --user.  When IN a venv `! venv_check` fails → `&&` skips --user and the
+    # group exits non-zero, propagating the base-install failure instead of
+    # masking it as success (the `|| { venv_check || … }` shape from #903
+    # swallowed the exit code because venv_check's exit-0 became the group's
+    # result).
+    return f"{base} || {{ ! {venv_check} && {user}; }}"
+
+
+def _venv_safe_local_pip_install_cmd(cmd: str, *, local: bool, in_venv: bool) -> str:
+    """Drop pip user-install flags that are invalid for local venv installs.
+
+    Cookbook dependency installs run through the model-serve task path so users
+    can watch progress in the same log UI. For local POSIX runs, that task
+    prepends Odysseus' own interpreter directory to PATH. If Odysseus itself is
+    running from a venv, `python3` resolves to the venv Python and pip rejects
+    `--user` with "User site-packages are not visible in this virtualenv".
+
+    Keep remote and non-venv installs unchanged: remotes may intentionally use
+    system Python, and Docker/non-venv installs still need user-site fallback.
+    """
+    if not local or not in_venv:
+        return cmd
+    if "pip install" not in (cmd or ""):
+        return cmd
+    try:
+        parts = shlex.split(cmd)
+    except ValueError:
+        return cmd
+    stripped = [
+        part
+        for part in parts
+        if part not in {"--user", "--break-system-packages"}
+    ]
+    return shlex.join(stripped)
 
 
 def _cached_model_scan_script(model_dirs: list[str] | None = None) -> str:
@@ -458,8 +515,49 @@ def _append_serve_exit_code_lines(runner_lines: list[str], *, keep_shell_open: b
         runner_lines.append('echo ""; echo "=== Process exited with code $ODYSSEUS_CMD_EXIT ==="; exec "${SHELL:-/bin/bash}"')
     else:
         runner_lines.append('echo ""; echo "=== Process exited with code $ODYSSEUS_CMD_EXIT ==="')
+        runner_lines.append('exit "$ODYSSEUS_CMD_EXIT"')
 
 
+def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
+    """Append Linux llama.cpp build lines that prefer ROCm/HIP when available.
+
+    Cookbook already detects AMD GPUs elsewhere, but the llama.cpp bootstrap used
+    to hard-wire CUDA on Linux. That made ROCm hosts attempt a CUDA configure and
+    fail with "CUDA Toolkit not found" instead of building with HIP.
+    """
+    # Detect pip-installed nvcc (from vLLM/nvidia CUDA wheels) and put it on PATH
+    # so cmake's CUDA configure can find it. We keep this after the ROCm/HIP
+    # check — a machine with both stacks should honor the native HIP toolchain on
+    # AMD hosts instead of accidentally preferring a stray nvcc wheel.
+    runner_lines.append('    for _cudir in ~/.local/lib/python*/site-packages/nvidia/cu13 ~/.local/lib/python*/site-packages/nvidia/cu12 ~/.local/lib/python*/site-packages/nvidia/cuda_nvcc; do')
+    runner_lines.append('      [ -x "$_cudir/bin/nvcc" ] && export CUDA_HOME="$_cudir" && export PATH="$_cudir/bin:$PATH" && break')
+    runner_lines.append('    done')
+    # rm -rf build so a prior poisoned CMakeCache.txt (e.g. from a failed CUDA
+    # or HIP attempt) doesn't cause the next configure to reuse stale settings.
+    runner_lines.append('    cd ~/llama.cpp && rm -rf build')
+    runner_lines.append('    if command -v hipconfig &>/dev/null || [ -d /opt/rocm ] || [ -n "$ROCM_PATH" ] || [ -n "$HIP_PATH" ]; then')
+    runner_lines.append('      if command -v hipconfig &>/dev/null; then')
+    runner_lines.append('        export HIPCXX="${HIPCXX:-$(hipconfig -l)/clang}"')
+    runner_lines.append('        export HIP_PATH="${HIP_PATH:-$(hipconfig -R)}"')
+    runner_lines.append('      fi')
+    runner_lines.append('      echo "[odysseus] ROCm/HIP detected — building llama-server with HIP support..."')
+    runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON \\\\')
+    runner_lines.append('        && cmake --build build -j"$NPROC" --target llama-server \\\\')
+    runner_lines.append('        && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    runner_lines.append('    elif command -v nvcc &>/dev/null; then')
+    runner_lines.append('      echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
+    runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \\\\')
+    runner_lines.append('        && cmake --build build -j"$NPROC" --target llama-server \\\\')
+    runner_lines.append('        && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    runner_lines.append('    else')
+    runner_lines.append('      echo "[odysseus] WARNING: no HIP/CUDA toolchain found — building llama-server for CPU only."')
+    runner_lines.append('      echo "[odysseus]   GPU inference will not be available for this llama.cpp build."')
+    runner_lines.append('      echo "[odysseus]   Install ROCm for AMD GPUs or vLLM/CUDA tooling for NVIDIA, then re-launch this serve task."')
+    runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release \\\\')
+    runner_lines.append('        && cmake --build build -j"$NPROC" --target llama-server \\\\')
+    runner_lines.append('        && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    runner_lines.append('    fi')
+
 class ModelDownloadRequest(BaseModel):
     repo_id: str
     include: str | None = None  # glob pattern e.g. "*Q4_K_M*"

routes/cookbook_routes.py

@@ -37,8 +37,9 @@ from routes.cookbook_helpers import (
     _validate_local_dir, _validate_ssh_port, _validate_gpus, _shell_path,
     _ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase,
     _safe_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
-    _append_serve_exit_code_lines, _cached_model_scan_script, _ollama_bind_from_cmd,
-    _pip_install_fallback_chain, ModelDownloadRequest, ServeRequest,
+    _append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script,
+    _ollama_bind_from_cmd, _pip_install_fallback_chain, _venv_safe_local_pip_install_cmd,
+    ModelDownloadRequest, ServeRequest,
 )
 
 _HF_TOKEN_STATUS_SNIPPET = (
@@ -545,12 +546,18 @@ def setup_cookbook_routes() -> APIRouter:
                 )
             # Ensure pip-user scripts (e.g. hf CLI installed via --user) are on PATH
             runner_lines.append('export PATH="$HOME/.local/bin:$PATH"')
-            # Install hf CLI + hf_transfer best-effort so future runs get the fast path.
+            # Install hf CLI + optional hf_transfer best-effort. Retries disable
+            # hf_transfer because the Rust parallel path is fast but has been
+            # flaky near the end of very large multi-file downloads.
             # Use --break-system-packages on PEP-668 systems (Arch, newer Debian) so it doesn't bail.
             runner_lines.append(f"command -v hf >/dev/null 2>&1 || {_pip_install_fallback_chain('huggingface_hub', python_cmd='pip', upgrade=True)}")
-            runner_lines.append(f"python3 -c 'import hf_transfer' 2>/dev/null || {_pip_install_fallback_chain('hf_transfer', python_cmd='pip')}")
-            runner_lines.append("python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
-            runner_lines.append("export HF_HUB_DOWNLOAD_MAX_WORKERS=8")
+            if req.disable_hf_transfer:
+                runner_lines.append("export HF_HUB_ENABLE_HF_TRANSFER=0")
+                runner_lines.append("export HF_HUB_DOWNLOAD_MAX_WORKERS=4")
+            else:
+                runner_lines.append(f"python3 -c 'import hf_transfer' 2>/dev/null || {_pip_install_fallback_chain('hf_transfer', python_cmd='pip')}")
+                runner_lines.append("python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
+                runner_lines.append("export HF_HUB_DOWNLOAD_MAX_WORKERS=8")
             # Surface whether the HF token actually reached THIS server, so a gated
             # download's "not authorized" failure can be told apart from a missing
             # token (the token is masked — we only print applied / not-set).
@@ -561,13 +568,17 @@ def setup_cookbook_routes() -> APIRouter:
             runner_lines.append(f'  {hf_cmd} < /dev/null')
             runner_lines.append('elif python3 -c "import huggingface_hub" 2>/dev/null; then')
             runner_lines.append('  echo "hf CLI not found, using Python huggingface_hub..."')
-            runner_lines.append(f'  python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers=8)"')
+            runner_lines.append(f'  python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers={4 if req.disable_hf_transfer else 8})"')
             runner_lines.append('else')
             runner_lines.append('  echo "Installing huggingface-hub and dependencies..."')
             runner_lines.append('  pip install --no-deps -q huggingface-hub 2>/dev/null')
-            runner_lines.append('  pip install -q filelock fsspec packaging pyyaml tqdm typer httpx requests hf_transfer 2>/dev/null')
-            runner_lines.append("  python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
-            runner_lines.append(f'  python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers=8)"')
+            if req.disable_hf_transfer:
+                runner_lines.append('  pip install -q filelock fsspec packaging pyyaml tqdm typer httpx requests 2>/dev/null')
+                runner_lines.append('  export HF_HUB_ENABLE_HF_TRANSFER=0')
+            else:
+                runner_lines.append('  pip install -q filelock fsspec packaging pyyaml tqdm typer httpx requests hf_transfer 2>/dev/null')
+                runner_lines.append("  python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
+            runner_lines.append(f'  python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers={4 if req.disable_hf_transfer else 8})"')
             runner_lines.append('fi')
             runner_lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
             runner_lines.append(f"rm -f {remote_runner}")
@@ -819,6 +830,11 @@ def setup_cookbook_routes() -> APIRouter:
         # many downstream `"engine" in req.cmd` membership checks can't hit
         # `TypeError: argument of type 'NoneType'` (a 500 instead of a clean 400).
         req.cmd = _validate_serve_cmd(req.cmd) or ""
+        req.cmd = _venv_safe_local_pip_install_cmd(
+            req.cmd,
+            local=not bool(req.remote_host),
+            in_venv=sys.prefix != sys.base_prefix,
+        )
         is_pip_install = bool(req.cmd and "pip install" in req.cmd)
         if is_pip_install:
             # PEP-508-style package spec — letters, digits, `.-_` for the
@@ -963,43 +979,7 @@ def setup_cookbook_routes() -> APIRouter:
                 runner_lines.append('      && cmake --build build -j"$NPROC" --target llama-server \\')
                 runner_lines.append('      && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
                 runner_lines.append('  else')
-                # Detect pip-installed nvcc (from vLLM/nvidia CUDA wheels) and put
-                # it on PATH so cmake's CUDA configure can find it.  We check the
-                # same three layouts as entrypoint.sh:
-                #   nvidia/cu13       — nvidia-nvcc-cu13
-                #   nvidia/cu12       — nvidia-nvcc-cu12
-                #   nvidia/cuda_nvcc  — nvidia-cuda-nvcc-cu12 (sub-package style)
-                runner_lines.append('    for _cudir in ~/.local/lib/python*/site-packages/nvidia/cu13 ~/.local/lib/python*/site-packages/nvidia/cu12 ~/.local/lib/python*/site-packages/nvidia/cuda_nvcc; do')
-                runner_lines.append('      [ -x "$_cudir/bin/nvcc" ] && export CUDA_HOME="$_cudir" && export PATH="$_cudir/bin:$PATH" && break')
-                runner_lines.append('    done')
-                # rm -rf build so a prior poisoned CMakeCache.txt (e.g. from a
-                # failed CUDA attempt) doesn't cause the next configure to reuse
-                # stale settings and silently produce a CPU-only binary.
-                runner_lines.append('    cd ~/llama.cpp && rm -rf build')
-                runner_lines.append('    _ody_has_cuda_runtime=0')
-                runner_lines.append('    if command -v nvcc &>/dev/null; then')
-                runner_lines.append('      for _cudalib in "${CUDA_HOME:-}/lib64"/libcudart.so* "${CUDA_HOME:-}/lib"/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib*/libcudart.so*; do')
-                runner_lines.append('        [ -e "$_cudalib" ] && _ody_has_cuda_runtime=1 && break')
-                runner_lines.append('      done')
-                runner_lines.append('    fi')
-                runner_lines.append('    if command -v nvcc &>/dev/null && [ "$_ody_has_cuda_runtime" = "1" ]; then')
-                runner_lines.append('      echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
-                runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \\')
-                runner_lines.append('        && cmake --build build -j"$NPROC" --target llama-server \\')
-                runner_lines.append('        && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
-                runner_lines.append('    else')
-                runner_lines.append('      if command -v nvcc &>/dev/null; then')
-                runner_lines.append('        echo "[odysseus] WARNING: nvcc found but CUDA runtime library was not found — building llama-server for CPU only."')
-                runner_lines.append('      else')
-                runner_lines.append('        echo "[odysseus] WARNING: nvcc not found — building llama-server for CPU only."')
-                runner_lines.append('      fi')
-                runner_lines.append('      echo "[odysseus]   GPU inference will not be available for this llama.cpp build."')
-                runner_lines.append('      echo "[odysseus]   To get a GPU build, first install vLLM via Cookbook -> Dependencies"')
-                runner_lines.append('      echo "[odysseus]   (its CUDA wheels include nvcc), then re-launch this serve task."')
-                runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release \\')
-                runner_lines.append('        && cmake --build build -j"$NPROC" --target llama-server \\')
-                runner_lines.append('        && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
-                runner_lines.append('    fi')
+                _append_llama_cpp_linux_accel_build_lines(runner_lines)
                 runner_lines.append('  fi')
                 runner_lines.append('  # If the native build failed, fall back to the Python bindings.')
                 runner_lines.append('  if ! command -v llama-server &>/dev/null && ! python3 -c "import llama_cpp" 2>/dev/null; then')
@@ -1024,16 +1004,20 @@ def setup_cookbook_routes() -> APIRouter:
                 runner_lines.append(f'ODYSSEUS_OLLAMA_HOST={_bash_squote(_ollama_host)}')
                 runner_lines.append(f'ODYSSEUS_OLLAMA_PORT="{_ollama_port}"')
                 runner_lines.append('ODYSSEUS_OLLAMA_URL=""')
-                runner_lines.append('for _ody_ollama_port in "$ODYSSEUS_OLLAMA_PORT" 11434; do')
-                runner_lines.append('  [ -z "$_ody_ollama_port" ] && continue')
-                runner_lines.append('  for _ody_ollama_host in 127.0.0.1 localhost host.docker.internal; do')
-                runner_lines.append('    _ody_ollama_url="http://${_ody_ollama_host}:${_ody_ollama_port}"')
-                runner_lines.append('    if curl -sf "$_ody_ollama_url/api/tags" >/dev/null 2>&1; then')
-                runner_lines.append('      ODYSSEUS_OLLAMA_URL="$_ody_ollama_url"')
-                runner_lines.append('      ODYSSEUS_OLLAMA_PORT="$_ody_ollama_port"')
-                runner_lines.append('      break 2')
-                runner_lines.append('    fi')
+                runner_lines.append('for _ody_ollama_try in $(seq 1 20); do')
+                runner_lines.append('  for _ody_ollama_port in "$ODYSSEUS_OLLAMA_PORT" 11434; do')
+                runner_lines.append('    [ -z "$_ody_ollama_port" ] && continue')
+                runner_lines.append('    for _ody_ollama_host in 127.0.0.1 localhost host.docker.internal; do')
+                runner_lines.append('      _ody_ollama_url="http://${_ody_ollama_host}:${_ody_ollama_port}"')
+                runner_lines.append('      if curl -sf "$_ody_ollama_url/api/tags" >/dev/null 2>&1; then')
+                runner_lines.append('        ODYSSEUS_OLLAMA_URL="$_ody_ollama_url"')
+                runner_lines.append('        ODYSSEUS_OLLAMA_PORT="$_ody_ollama_port"')
+                runner_lines.append('        break 3')
+                runner_lines.append('      fi')
+                runner_lines.append('    done')
                 runner_lines.append('  done')
+                runner_lines.append('  [ "$_ody_ollama_try" -eq 1 ] && echo "[odysseus] Waiting for an existing Ollama API on ports ${ODYSSEUS_OLLAMA_PORT}/11434..."')
+                runner_lines.append('  sleep 1')
                 runner_lines.append('done')
                 runner_lines.append('if [ -n "$ODYSSEUS_OLLAMA_URL" ]; then')
                 runner_lines.append('  if [ "$ODYSSEUS_OLLAMA_PORT" != "' + _ollama_port + '" ]; then')
@@ -1850,6 +1834,43 @@ def setup_cookbook_routes() -> APIRouter:
     def _cookbook_tasks_status_sync():
         import subprocess
 
+        def _download_cache_complete(repo_id: str, remote_host: str = "", ssh_port: str = "") -> bool:
+            """Best-effort check for a completed HF cache entry.
+
+            tmux output can stop at a stale progress line if the pane/session
+            disappears before Cookbook captures the final DOWNLOAD_OK marker.
+            In that case, trust the cache shape: a snapshot directory with files
+            and no *.incomplete blobs means HuggingFace finished materializing the
+            model.
+            """
+            if not repo_id or "/" not in repo_id:
+                return False
+            py = (
+                "import os,sys;"
+                "repo=sys.argv[1];"
+                "base=os.environ.get('HUGGINGFACE_HUB_CACHE') or os.path.join(os.environ.get('HF_HOME', os.path.expanduser('~/.cache/huggingface')), 'hub');"
+                "d=os.path.join(base,'models--'+repo.replace('/','--'));"
+                "snap=os.path.join(d,'snapshots');"
+                "ok=os.path.isdir(snap) and any(os.path.isdir(os.path.join(snap,x)) and os.listdir(os.path.join(snap,x)) for x in os.listdir(snap));"
+                "inc=False;"
+                "blobs=os.path.join(d,'blobs');"
+                "inc=os.path.isdir(blobs) and any(x.endswith('.incomplete') for x in os.listdir(blobs));"
+                "sys.exit(0 if ok and not inc else 1)"
+            )
+            cmd = ["python3", "-c", py, repo_id]
+            try:
+                if remote_host:
+                    ssh_base = ["ssh"]
+                    if ssh_port and ssh_port != "22":
+                        ssh_base.extend(["-p", str(ssh_port)])
+                    shell_cmd = " ".join(shlex.quote(x) for x in cmd)
+                    proc = subprocess.run(ssh_base + [remote_host, shell_cmd], timeout=12, capture_output=True)
+                else:
+                    proc = subprocess.run(cmd, timeout=12, capture_output=True)
+                return proc.returncode == 0
+            except Exception:
+                return False
+
         # Load saved tasks from cookbook state
         tasks = []
         if _cookbook_state_path.exists():
@@ -2026,7 +2047,14 @@ def setup_cookbook_routes() -> APIRouter:
                     status = "running"
             else:
                 # Session is dead — check if it completed or crashed
-                status = "stopped"
+                if task_type == "download" and _download_cache_complete(_payload.get("repo_id") or model, remote, str(_tport or "")):
+                    status = "completed"
+                    if not progress_text:
+                        progress_text = "Download complete"
+                    if not full_snapshot:
+                        full_snapshot = "DOWNLOAD_OK"
+                else:
+                    status = "stopped"
 
             # Parse structured phase info — single source of truth for the UI
             phase_info = _parse_serve_phase(full_snapshot, task_type) if (task_type == "serve" and status == "running" and full_snapshot) else {}

routes/document_routes.py

@@ -145,7 +145,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
             create_form_markdown_document,
             create_plain_pdf_document,
         )
-        from src.document_processor import _process_pdf
+        from src.document_processor import _process_pdf, strip_pdf_content_marker
         import os
 
         from src.auth_helpers import require_privilege
@@ -184,7 +184,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
 
         title = os.path.splitext(meta.get("original_name") or meta.get("name") or upload_id)[0]
         try:
-            body_text = _process_pdf(pdf_path).lstrip("\n[PDF content]:").strip()
+            body_text = strip_pdf_content_marker(_process_pdf(pdf_path))
         except Exception:
             body_text = None
 
@@ -402,7 +402,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
         text extraction was wired, plus for scanned/image-only PDFs where the
         VL model picks up text the basic pypdf path missed."""
         import re
-        from src.document_processor import _process_pdf
+        from src.document_processor import _process_pdf, strip_pdf_content_marker
         from src.pdf_form_doc import find_source_upload_id
 
         user = get_current_user(request)
@@ -423,7 +423,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
                 raise HTTPException(404, "Source PDF could not be located")
 
             try:
-                body_text = _process_pdf(pdf_path).lstrip("\n[PDF content]:").strip()
+                body_text = strip_pdf_content_marker(_process_pdf(pdf_path))
             except Exception as e:
                 logger.error(f"extract_pdf_text failed for {pdf_path}: {e}")
                 raise HTTPException(500, f"Extraction failed: {e}")

routes/email_helpers.py

@@ -608,7 +608,32 @@ def _list_email_accounts() -> list[dict]:
 
 # ── IMAP helpers ──
 
-_IMAP_TIMEOUT_SECONDS = 15
+def _coerce_imap_timeout_seconds(raw: str | None) -> int:
+    try:
+        value = int(raw or "30")
+    except (TypeError, ValueError):
+        value = 30
+    return max(5, min(value, 300))
+
+
+_IMAP_TIMEOUT_SECONDS = _coerce_imap_timeout_seconds(os.environ.get("ODYSSEUS_IMAP_TIMEOUT_SECONDS"))
+
+
+def _open_imap_connection(host: str, port: int, *, starttls: bool, timeout: int = _IMAP_TIMEOUT_SECONDS):
+    """Open an IMAP connection using the configured security mode."""
+    port = int(port or 993)
+    if starttls:
+        conn = imaplib.IMAP4(host, port, timeout=timeout)
+        conn.starttls()
+    elif port == 993:
+        conn = imaplib.IMAP4_SSL(host, port, timeout=timeout)
+    else:
+        conn = imaplib.IMAP4(host, port, timeout=timeout)
+    try:
+        conn.sock.settimeout(timeout)
+    except Exception:
+        pass
+    return conn
 
 def _imap_connect(account_id: str | None = None, owner: str = ""):
     # SECURITY: passing `owner` scopes the fallback config lookup so a brand
@@ -622,17 +647,12 @@ def _imap_connect(account_id: str | None = None, owner: str = ""):
     # The last branch is critical: previously this fell into IMAP4_SSL
     # for any non-STARTTLS port, which would fail the TLS handshake on
     # plain local servers (Dovecot on 31143, etc.).
-    if cfg.get("imap_starttls"):
-        conn = imaplib.IMAP4(cfg["imap_host"], cfg["imap_port"], timeout=_IMAP_TIMEOUT_SECONDS)
-        conn.starttls()
-    elif int(cfg.get("imap_port") or 993) == 993:
-        conn = imaplib.IMAP4_SSL(cfg["imap_host"], cfg["imap_port"], timeout=_IMAP_TIMEOUT_SECONDS)
-    else:
-        conn = imaplib.IMAP4(cfg["imap_host"], cfg["imap_port"], timeout=_IMAP_TIMEOUT_SECONDS)
-    try:
-        conn.sock.settimeout(_IMAP_TIMEOUT_SECONDS)
-    except Exception:
-        pass
+    conn = _open_imap_connection(
+        cfg["imap_host"],
+        cfg["imap_port"],
+        starttls=bool(cfg.get("imap_starttls")),
+        timeout=_IMAP_TIMEOUT_SECONDS,
+    )
     conn.login(cfg["imap_user"], cfg["imap_password"])
     return conn
 
@@ -1134,7 +1154,10 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
         try:
             from routes.contacts_routes import _fetch_contacts
             for c in _fetch_contacts() or []:
-                if (c.get("email") or "").lower() == sender_addr:
+                # Contacts are normalized to plural `emails` lists (see
+                # contacts_routes._normalize_contact); the old `c.get("email")`
+                # singular key never exists, so known senders were never matched.
+                if sender_addr in [(e or "").lower() for e in (c.get("emails") or [])]:
                     is_known = True
                     break
         except Exception:
@@ -1228,13 +1251,13 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
                 t_lower = term.lower()
                 matches = [c for c in all_contacts
                            if t_lower in (c.get("name") or "").lower()
-                           or t_lower in (c.get("email") or "").lower()]
+                           or any(t_lower in (e or "").lower() for e in (c.get("emails") or []))]
                 for c in matches[:2]:
                     parts = [f"Name: {c.get('name','')}"]
-                    if c.get("email"):
-                        parts.append(f"Email: {c['email']}")
-                    if c.get("phone"):
-                        parts.append(f"Phone: {c['phone']}")
+                    if c.get("emails"):
+                        parts.append(f"Email: {', '.join(c['emails'])}")
+                    if c.get("phones"):
+                        parts.append(f"Phone: {', '.join(c['phones'])}")
                     context_snippets.append(f"[Contact match for \"{term}\"] " + ", ".join(parts))
         except Exception:
             pass

routes/email_pollers.py

@@ -132,7 +132,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
     import sqlite3 as _sql3
     import requests as _req
     from src.endpoint_resolver import resolve_endpoint
-    from src.llm_core import _uses_max_completion_tokens
+    from src.llm_core import _uses_max_completion_tokens, _restricts_temperature
 
     settings = _load_settings()
     auto_sum = settings.get("email_auto_summarize", False)
@@ -355,6 +355,9 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
                         "temperature": 0.3,
                         "stream": False,
                     }
+                    # Reasoning models (o1/o3/o4/gpt-5) reject an explicit temperature.
+                    if _restricts_temperature(model):
+                        payload.pop("temperature", None)
                     try:
                         # Use to_thread so this sync HTTP call doesn't freeze
                         # the entire event loop while the LLM thinks (240s).
@@ -806,6 +809,9 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
                             "temperature": 0.1,
                             "stream": False,
                         }
+                        # Reasoning models (o1/o3/o4/gpt-5) reject an explicit temperature.
+                        if _restricts_temperature(model):
+                            payload.pop("temperature", None)
                         # to_thread keeps the event loop responsive during the LLM call
                         resp = await asyncio.to_thread(
                             _req.post, url, json=payload, headers=req_headers, timeout=120

routes/email_routes.py

@@ -17,7 +17,6 @@ import sqlite3 as _sql3
 import email as email_mod
 import email.header
 import email.utils
-import imaplib
 import smtplib
 import json
 import re
@@ -41,6 +40,7 @@ from routes.email_helpers import (
     _q, _attach_compose_uploads, _cleanup_compose_uploads,
     _load_settings, _save_settings, _get_email_config,
     _send_smtp_message, _smtp_security_mode,
+    _IMAP_TIMEOUT_SECONDS, _open_imap_connection,
     _imap_connect, _imap, _decode_header, _detect_sent_folder, _detect_drafts_folder,
     _extract_attachment_text, _list_attachments_from_msg,
     _extract_attachment_to_disk, _extract_html, _extract_text,
@@ -457,7 +457,7 @@ def setup_email_routes():
     _IMAP_POOL = {}   # account_id → (conn, last_used_at)
     _IMAP_IDLE_MAX = 60.0
     _WARMING_READS = set()
-    _WARM_READ_LIMIT = 3
+    _WARM_READ_LIMIT = 1
     _WARM_MAX_BYTES = 128 * 1024
     _WARM_RECENT_SECONDS = 7 * 24 * 60 * 60
     _pool_lock = _threading.Lock()
@@ -1046,7 +1046,7 @@ def setup_email_routes():
 
                 # Escape backslash and quote for the IMAP-SEARCH quoted-string.
                 q_escaped = q.replace('\\', '\\\\').replace('"', '\\"')
-                search_cmd = f'(OR FROM "{q_escaped}" TEXT "{q_escaped}")'
+                search_cmd = f'(OR OR FROM "{q_escaped}" SUBJECT "{q_escaped}" TEXT "{q_escaped}")'
 
                 status, data = _imap_uid_search(conn, search_cmd)
                 if status != "OK" or not data[0]:
@@ -2419,7 +2419,7 @@ def setup_email_routes():
         """Generate a quick AI summary of an email body."""
         try:
             from src.endpoint_resolver import resolve_endpoint
-            from src.llm_core import _uses_max_completion_tokens
+            from src.llm_core import _uses_max_completion_tokens, _restricts_temperature
             import requests as _req
 
             body = data.get("body", "")
@@ -2476,6 +2476,9 @@ def setup_email_routes():
                 "temperature": 0.3,
                 "stream": False,
             }
+            # Reasoning models (o1/o3/o4/gpt-5) reject an explicit temperature.
+            if _restricts_temperature(model):
+                payload.pop("temperature", None)
             resp = await asyncio.to_thread(
                 _req.post, url, json=payload, headers=req_headers, timeout=180
             )
@@ -3100,13 +3103,12 @@ def setup_email_routes():
             # port (Dovecot on 31143, etc.) would always fail the SSL
             # handshake because they're not actually wrapped in TLS.
             try:
-                if imap_starttls:
-                    conn = imaplib.IMAP4(imap_host, imap_port, timeout=10)
-                    conn.starttls()
-                elif imap_port == 993:
-                    conn = imaplib.IMAP4_SSL(imap_host, imap_port, timeout=10)
-                else:
-                    conn = imaplib.IMAP4(imap_host, imap_port, timeout=10)
+                conn = _open_imap_connection(
+                    imap_host,
+                    imap_port,
+                    starttls=imap_starttls,
+                    timeout=_IMAP_TIMEOUT_SECONDS,
+                )
                 try:
                     conn.login(imap_user, imap_pass)
                     imap_result = {"ok": True}

routes/embedding_routes.py

@@ -242,6 +242,18 @@ def setup_embedding_routes():
         if not url:
             raise HTTPException(400, "URL is required")
 
+        # SSRF hardening: validate the user-supplied URL before any outbound
+        # request. Local-first means loopback/LAN endpoints are allowed by
+        # default; non-HTTP(S) schemes and the cloud metadata range are always
+        # rejected. Set EMBEDDING_BLOCK_PRIVATE_IPS=true for full lockdown.
+        from src.url_safety import check_outbound_url
+        ok, reason = check_outbound_url(
+            url,
+            block_private=os.getenv("EMBEDDING_BLOCK_PRIVATE_IPS", "false").lower() == "true",
+        )
+        if not ok:
+            raise HTTPException(400, f"Rejected endpoint URL: {reason}")
+
         # Quick health check
         try:
             import httpx

routes/gallery_routes.py

@@ -3,6 +3,9 @@
 import os
 import hashlib
 import logging
+import re
+import uuid
+from pathlib import Path
 from typing import Dict, Any, Optional
 
 from fastapi import APIRouter, HTTPException, Query, Request
@@ -17,6 +20,14 @@ from routes.gallery_helpers import (
 
 logger = logging.getLogger(__name__)
 
+
+def _sanitize_gallery_filename(filename: str) -> str:
+    """Return a local filename safe to join under generated_images."""
+    safe_name = re.sub(r"[^A-Za-z0-9._-]", "_", Path(filename or "").name)[:128]
+    if not safe_name or safe_name in {".", ".."}:
+        safe_name = uuid.uuid4().hex[:12]
+    return safe_name
+
 def setup_gallery_routes() -> APIRouter:
     router = APIRouter(tags=["gallery"])
 
@@ -122,7 +133,7 @@ def setup_gallery_routes() -> APIRouter:
             content = await file.read()
             img_dir = Path("data/generated_images")
             img_dir.mkdir(parents=True, exist_ok=True)
-            img_path = img_dir / img.filename
+            img_path = img_dir / _sanitize_gallery_filename(img.filename)
             img_path.write_bytes(content)
 
             # Refresh dimensions in case the editor resized the canvas.
@@ -1125,7 +1136,7 @@ def setup_gallery_routes() -> APIRouter:
             db = SessionLocal()
             try:
                 for ep in db.query(ModelEndpoint).all():
-                    if ep.base_url.rstrip("/").rstrip("/v1") == base.rstrip("/v1"):
+                    if ep.base_url.rstrip("/").removesuffix("/v1").rstrip("/") == base.rstrip("/").removesuffix("/v1").rstrip("/"):
                         api_key = ep.api_key
                         break
             finally:
@@ -1696,7 +1707,7 @@ def setup_gallery_routes() -> APIRouter:
                 return {"error": "No vision-capable endpoint configured"}
 
             # Call vision model — format differs between Anthropic and OpenAI
-            from src.llm_core import _detect_provider
+            from src.llm_core import _detect_provider, _restricts_temperature, _uses_max_completion_tokens
             provider = _detect_provider(chat_url)
             tag_prompt = (
                 "Analyze this photo. Return ONLY a comma-separated list of tags. "
@@ -1721,6 +1732,7 @@ def setup_gallery_routes() -> APIRouter:
                     }],
                 }
             else:
+                _tok_key = "max_completion_tokens" if _uses_max_completion_tokens(model_name) else "max_tokens"
                 payload = {
                     "model": model_name,
                     "messages": [{
@@ -1730,9 +1742,12 @@ def setup_gallery_routes() -> APIRouter:
                             {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}},
                         ],
                     }],
-                    "max_tokens": 200,
+                    _tok_key: 200,
                     "temperature": 0.3,
                 }
+                # Reasoning models (o1/o3/o4/gpt-5) reject an explicit temperature.
+                if _restricts_temperature(model_name):
+                    payload.pop("temperature", None)
 
             h = {"Content-Type": "application/json"}
             if headers:

routes/hwfit_routes.py

@@ -4,85 +4,102 @@ from copy import deepcopy
 from fastapi import APIRouter
 
 
+# Backends the manual hardware simulator accepts. Must stay a subset of what
+# services.hwfit.fit understands so a simulated box ranks like a real one:
+# "metal" routes through the Apple-Silicon path (GGUF-only, llama.cpp/Ollama),
+# the CPU backends through the RAM/offload path, cuda/rocm through vLLM.
+_MANUAL_BACKENDS = {"cuda", "rocm", "metal", "cpu_x86", "cpu_arm"}
+
+
+def _apply_manual_hardware(system, manual_mode="", manual_gpu_count="", manual_vram_gb="", manual_ram_gb="", manual_backend=""):
+    """Manual hardware is a "what if I had this setup" simulator —
+    REPLACES the detected hardware entirely instead of adding to it.
+
+    The previous additive behavior averaged the manual VRAM across
+    all GPUs (base + manual), which meant adding "1× 400 GB" on top
+    of "2× 70 GB" only nudged the per-GPU cap from 70 to 180 GB
+    (= 540 / 3), so GGUF models bigger than that still didn't surface
+    — exactly the "cap stuck at detected level" bug the user hit.
+    """
+    manual_mode = (manual_mode or "").lower()
+    if manual_mode not in {"gpu", "ram"}:
+        return system
+
+    try:
+        override_ram_gb = float(manual_ram_gb) if manual_ram_gb else 0
+    except ValueError:
+        override_ram_gb = 0
+    override_ram_gb = max(0.0, override_ram_gb)
+    if override_ram_gb:
+        # Replace RAM, don't add. The number in the field is the
+        # TOTAL system memory the user wants to simulate.
+        system["available_ram_gb"] = round(override_ram_gb, 1)
+        system["total_ram_gb"] = round(override_ram_gb, 1)
+    system["manual_hardware"] = True
+
+    if manual_mode == "ram":
+        # RAM-only simulation — wipe GPU entirely so the ranker uses
+        # CPU/RAM paths.
+        system["has_gpu"] = False
+        system["gpu_name"] = None
+        system["gpu_vram_gb"] = 0
+        system["gpu_count"] = 0
+        system["gpus"] = []
+        system["gpu_groups"] = []
+        system["backend"] = "cpu_x86"
+        system.pop("unified_memory", None)
+        return system
+
+    try:
+        count = int(manual_gpu_count) if manual_gpu_count else 1
+    except ValueError:
+        count = 1
+    try:
+        vram_each = float(manual_vram_gb) if manual_vram_gb else 8.0
+    except ValueError:
+        vram_each = 8.0
+    count = max(1, min(count, 16))
+    vram_each = max(1.0, vram_each)
+    backend = (manual_backend or system.get("backend") or "cuda").lower()
+    if backend not in _MANUAL_BACKENDS:
+        backend = "cuda"
+    total_vram = round(vram_each * count, 1)
+    gpu_name = f"Simulated {backend.upper()} GPU" + (f" × {count}" if count > 1 else "")
+    system["has_gpu"] = True
+    system["gpu_name"] = gpu_name
+    system["gpu_vram_gb"] = total_vram
+    system["gpu_count"] = count
+    system["gpus"] = [
+        {"index": i, "name": gpu_name, "vram_gb": vram_each}
+        for i in range(count)
+    ]
+    # Single homogeneous pool — vram_each here is the ACTUAL per-GPU
+    # VRAM the user entered, not an average. That's the whole point:
+    # raising vram_each lifts the per-GPU cap (GGUF, tensor-parallel
+    # math) all the way up, not just by a small fraction.
+    system["gpu_groups"] = [{
+        "name": gpu_name,
+        "vram_each": vram_each,
+        "count": count,
+        "indices": list(range(count)),
+        "vram_total": total_vram,
+    }]
+    system["homogeneous"] = True
+    system["backend"] = backend
+    # Apple Silicon shares one unified memory pool with the GPU; flag it so
+    # the API/UI report it the way real Metal detection does. Discrete GPUs
+    # (cuda/rocm) and the CPU backends carry separate VRAM, so clear any
+    # stale flag a previous detection left on the dict.
+    if backend == "metal":
+        system["unified_memory"] = True
+    else:
+        system.pop("unified_memory", None)
+    return system
+
+
 def setup_hwfit_routes():
     router = APIRouter(prefix="/api/hwfit", tags=["hwfit"])
 
-    def _apply_manual_hardware(system, manual_mode="", manual_gpu_count="", manual_vram_gb="", manual_ram_gb="", manual_backend=""):
-        """Manual hardware is a "what if I had this setup" simulator —
-        REPLACES the detected hardware entirely instead of adding to it.
-
-        The previous additive behavior averaged the manual VRAM across
-        all GPUs (base + manual), which meant adding "1× 400 GB" on top
-        of "2× 70 GB" only nudged the per-GPU cap from 70 to 180 GB
-        (= 540 / 3), so GGUF models bigger than that still didn't surface
-        — exactly the "cap stuck at detected level" bug the user hit.
-        """
-        manual_mode = (manual_mode or "").lower()
-        if manual_mode not in {"gpu", "ram"}:
-            return system
-
-        try:
-            override_ram_gb = float(manual_ram_gb) if manual_ram_gb else 0
-        except ValueError:
-            override_ram_gb = 0
-        override_ram_gb = max(0.0, override_ram_gb)
-        if override_ram_gb:
-            # Replace RAM, don't add. The number in the field is the
-            # TOTAL system memory the user wants to simulate.
-            system["available_ram_gb"] = round(override_ram_gb, 1)
-            system["total_ram_gb"] = round(override_ram_gb, 1)
-        system["manual_hardware"] = True
-
-        if manual_mode == "ram":
-            # RAM-only simulation — wipe GPU entirely so the ranker uses
-            # CPU/RAM paths.
-            system["has_gpu"] = False
-            system["gpu_name"] = None
-            system["gpu_vram_gb"] = 0
-            system["gpu_count"] = 0
-            system["gpus"] = []
-            system["gpu_groups"] = []
-            system["backend"] = "cpu_x86"
-            return system
-
-        try:
-            count = int(manual_gpu_count) if manual_gpu_count else 1
-        except ValueError:
-            count = 1
-        try:
-            vram_each = float(manual_vram_gb) if manual_vram_gb else 8.0
-        except ValueError:
-            vram_each = 8.0
-        count = max(1, min(count, 16))
-        vram_each = max(1.0, vram_each)
-        backend = (manual_backend or system.get("backend") or "cuda").lower()
-        if backend not in {"cuda", "rocm", "cpu_x86", "cpu_arm"}:
-            backend = "cuda"
-        total_vram = round(vram_each * count, 1)
-        gpu_name = f"Simulated {backend.upper()} GPU" + (f" × {count}" if count > 1 else "")
-        system["has_gpu"] = True
-        system["gpu_name"] = gpu_name
-        system["gpu_vram_gb"] = total_vram
-        system["gpu_count"] = count
-        system["gpus"] = [
-            {"index": i, "name": gpu_name, "vram_gb": vram_each}
-            for i in range(count)
-        ]
-        # Single homogeneous pool — vram_each here is the ACTUAL per-GPU
-        # VRAM the user entered, not an average. That's the whole point:
-        # raising vram_each lifts the per-GPU cap (GGUF, tensor-parallel
-        # math) all the way up, not just by a small fraction.
-        system["gpu_groups"] = [{
-            "name": gpu_name,
-            "vram_each": vram_each,
-            "count": count,
-            "indices": list(range(count)),
-            "vram_total": total_vram,
-        }]
-        system["homogeneous"] = True
-        system["backend"] = backend
-        return system
-
     @router.get("/system")
     def get_system(host: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False):
         """Detect and return current system hardware info. Pass host=user@server for remote.
@@ -91,7 +108,7 @@ def setup_hwfit_routes():
         return detect_system(host=host, ssh_port=ssh_port, platform=platform, fresh=fresh)
 
     @router.get("/models")
-    def get_models(use_case: str = "", sort: str = "score", limit: int = 50, search: str = "", host: str = "", quant: str = "", gpu_count: str = "", gpu_group: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False):
+    def get_models(use_case: str = "", sort: str = "score", limit: int = 50, search: str = "", host: str = "", quant: str = "", ctx: str = "", gpu_count: str = "", gpu_group: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False):
         """Rank LLM models against detected hardware and return scored results.
         gpu_count: override GPU count (0 = CPU only, 1-N = simulate N GPUs of the
             active group). gpu_group: index into system.gpu_groups (the homogeneous
@@ -172,7 +189,14 @@ def setup_hwfit_routes():
             # gpu_only stays off here so the default view still surfaces offload.
             _apply_group(grp, grp["count"])
 
-        results = rank_models(system, use_case=use_case or None, limit=limit, search=search or None, sort=sort, quant=quant or None)
+        try:
+            target_context = int(ctx) if ctx else None
+        except ValueError:
+            target_context = None
+        if target_context is not None:
+            target_context = max(1024, min(target_context, 1000000))
+
+        results = rank_models(system, use_case=use_case or None, limit=limit, search=search or None, sort=sort, quant=quant or None, target_context=target_context)
         return {"system": system, "models": results}
 
     @router.get("/profiles")

routes/memory_routes.py

@@ -27,7 +27,7 @@ from src.request_models import MemoryAddRequest
 from core.database import SessionLocal
 from src.llm_core import llm_call_async
 from services.memory.memory_extractor import audit_memories
-from src.auth_helpers import get_current_user
+from src.auth_helpers import get_current_user, require_user
 from src.endpoint_resolver import resolve_endpoint
 
 logger = logging.getLogger(__name__)
@@ -191,8 +191,7 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
     @router.post("/extract")
     async def extract_memory(request: Request, session: str = Form(...)) -> Dict[str, List[str]]:
         """Analyze a session's chat history and return memory suggestions."""
-        if not get_current_user(request):
-            raise HTTPException(401, "Not authenticated")
+        require_user(request)
         try:
             sess = session_manager.get_session(session)
         except KeyError:

routes/model_routes.py

@@ -1,14 +1,16 @@
 # routes/model_routes.py
 """Routes for model and provider management."""
+import os
 import re
 import uuid
 import json
+import socket
 import time as _time
 import logging
 import httpx
 from datetime import datetime
 from typing import List, Dict, Any, Optional
-from urllib.parse import urlparse
+from urllib.parse import urlparse, urlunparse
 from fastapi import APIRouter, HTTPException, Form, Query, Body, Request
 from pydantic import BaseModel
 from fastapi.responses import StreamingResponse
@@ -26,6 +28,159 @@ from src.auth_helpers import _auth_disabled, owner_filter
 
 logger = logging.getLogger(__name__)
 
+_SPEECH_ENDPOINT_SETTINGS = (
+    ("tts_provider", "tts_model", "tts-1", "Text to Speech"),
+    ("stt_provider", "stt_model", "base", "Speech to Text"),
+)
+
+_ENDPOINT_SETTING_FIELDS = {
+    "default_endpoint_id":  ("default_model",  "Default Model"),
+    "utility_endpoint_id":  ("utility_model",   "Utility Model"),
+    "research_endpoint_id": ("research_model",  "Deep Research"),
+    "task_endpoint_id":     ("task_model",       "Background Tasks"),
+}
+
+_ENDPOINT_FALLBACK_FIELDS = {
+    "default_model_fallbacks": "Default Model Fallbacks",
+    "utility_model_fallbacks": "Utility Model Fallbacks",
+    "vision_model_fallbacks":  "Vision Model Fallbacks",
+}
+
+
+def _speech_settings_using_endpoint(settings: dict, ep_id: str) -> list:
+    """Return speech settings that reference a model endpoint."""
+    endpoint_ref = f"endpoint:{ep_id}"
+    return [
+        label
+        for provider_key, _, _, label in _SPEECH_ENDPOINT_SETTINGS
+        if (settings.get(provider_key) or "") == endpoint_ref
+    ]
+
+
+def _clear_speech_settings_for_endpoint(settings: dict, ep_id: str) -> list:
+    """Reset speech settings that reference a model endpoint."""
+    endpoint_ref = f"endpoint:{ep_id}"
+    cleared = []
+    for provider_key, model_key, default_model, label in _SPEECH_ENDPOINT_SETTINGS:
+        if (settings.get(provider_key) or "") == endpoint_ref:
+            settings[provider_key] = "disabled"
+            settings[model_key] = default_model
+            cleared.append(label)
+    return cleared
+
+
+def _endpoint_settings_using_endpoint(settings: dict, ep_id: str, *, include_speech: bool = False) -> list:
+    """Return labels for settings and fallback chains that reference an endpoint."""
+    affected = []
+    for ep_key, (_, label) in _ENDPOINT_SETTING_FIELDS.items():
+        if (settings.get(ep_key) or "") == ep_id:
+            affected.append(label)
+    for fallback_key, label in _ENDPOINT_FALLBACK_FIELDS.items():
+        chain = settings.get(fallback_key) or []
+        if any(isinstance(entry, dict) and (entry.get("endpoint_id") or "") == ep_id for entry in chain):
+            affected.append(label)
+    if include_speech:
+        affected.extend(_speech_settings_using_endpoint(settings, ep_id))
+    return affected
+
+
+def _clear_endpoint_settings_for_endpoint(settings: dict, ep_id: str, *, include_speech: bool = False) -> list:
+    """Remove an endpoint from direct settings and model fallback chains."""
+    cleared = []
+    for ep_key, (model_key, label) in _ENDPOINT_SETTING_FIELDS.items():
+        if (settings.get(ep_key) or "") == ep_id:
+            settings[ep_key] = ""
+            settings[model_key] = ""
+            cleared.append(label)
+    for fallback_key, label in _ENDPOINT_FALLBACK_FIELDS.items():
+        chain = settings.get(fallback_key)
+        if not isinstance(chain, list):
+            continue
+        kept = [
+            entry for entry in chain
+            if not (isinstance(entry, dict) and (entry.get("endpoint_id") or "") == ep_id)
+        ]
+        if len(kept) != len(chain):
+            settings[fallback_key] = kept
+            cleared.append(label)
+    if include_speech:
+        cleared.extend(_clear_speech_settings_for_endpoint(settings, ep_id))
+    return cleared
+
+
+def _clear_user_pref_endpoint_refs(all_prefs: dict, ep_id: str) -> int:
+    """Remove endpoint references from scoped or legacy-flat user preferences."""
+    if not isinstance(all_prefs, dict):
+        return 0
+    users = all_prefs.get("_users")
+    pref_sets = users.values() if isinstance(users, dict) else [all_prefs]
+    cleared_users = 0
+    for prefs in pref_sets:
+        if isinstance(prefs, dict) and _clear_endpoint_settings_for_endpoint(prefs, ep_id):
+            cleared_users += 1
+    return cleared_users
+
+
+# Loopback hosts a user might type for a local model server (LM Studio,
+# llama.cpp, vLLM, …). Inside Docker these point at the *container*, not the
+# host the server actually runs on.
+_ANY_BIND_HOSTS = {"0.0.0.0", "::"}
+_LOOPBACK_HOSTS = {"localhost", "127.0.0.1", "::1", *_ANY_BIND_HOSTS}
+
+
+def _docker_host_gateway_reachable() -> bool:
+    """True when we run inside a container whose host is reachable via
+    ``host.docker.internal`` (compose maps it to ``host-gateway``). Returns
+    False on native installs and on container setups without the mapping, so
+    the loopback rewrite below stays a no-op there."""
+    in_container = os.path.exists("/.dockerenv")
+    if not in_container:
+        try:
+            with open("/proc/1/cgroup", encoding="utf-8") as fh:
+                in_container = any(t in fh.read() for t in ("docker", "containerd", "kubepods"))
+        except OSError:
+            in_container = False
+    if not in_container:
+        return False
+    try:
+        socket.getaddrinfo("host.docker.internal", None)
+        return True
+    except OSError:
+        return False
+
+
+def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False) -> str:
+    """Rewrite a loopback model-endpoint URL to ``host.docker.internal`` when
+    running in Docker. A URL like ``http://localhost:1234/v1`` (the LM Studio
+    default) otherwise targets the Odysseus container itself, so the probe gets
+    a connection error and the endpoint is rejected with a misleading "No
+    models found for that provider/key".
+
+    Cookbook local serves are the opposite case: Odysseus started the model
+    server inside the same container/process environment, so the saved endpoint
+    must remain container-local. In that mode, normalize a bind address such as
+    0.0.0.0 to a connectable loopback host, but do not jump to the Docker host.
+    """
+    try:
+        parsed = urlparse(base_url)
+    except Exception:
+        return base_url
+    host = (parsed.hostname or "").lower()
+    if host not in _LOOPBACK_HOSTS:
+        return base_url
+    if container_local:
+        if host in _ANY_BIND_HOSTS:
+            netloc = "127.0.0.1" + (f":{parsed.port}" if parsed.port else "")
+            return urlunparse(parsed._replace(netloc=netloc))
+        return base_url
+    if host in _ANY_BIND_HOSTS and not _docker_host_gateway_reachable():
+        netloc = "127.0.0.1" + (f":{parsed.port}" if parsed.port else "")
+        return urlunparse(parsed._replace(netloc=netloc))
+    if not _docker_host_gateway_reachable():
+        return base_url
+    netloc = "host.docker.internal" + (f":{parsed.port}" if parsed.port else "")
+    return urlunparse(parsed._replace(netloc=netloc))
+
 
 # ── Curated model lists per provider ──
 # For cloud providers that return 100+ models, only show these by default.
@@ -41,10 +196,13 @@ _PROVIDER_CURATED = {
         "claude-sonnet-4-5", "claude-haiku-3-5",
     ],
     "zai": [
-        "glm-5", "glm-4.7", "glm-4.7-flash",
+        "glm-5", "glm-5.1", "glm-5v-turbo", "glm-4.7", "glm-4.7-flash",
         "glm-4.6", "glm-4.6v",
         "glm-4.5", "glm-4.5v", "glm-4.5-air", "glm-4.5-flash",
     ],
+    "zai-coding": [
+        "glm-5.1", "glm-5v-turbo", "glm-5-turbo", "glm-4.7", "glm-4.5-air",
+    ],
     "deepseek": [
         "deepseek-chat", "deepseek-reasoner",
     ],
@@ -103,9 +261,14 @@ _HOST_TO_CURATED = (
 def _match_provider_curated(base_url: str, provider: str) -> str:
     """Return the curated-list key for a given endpoint.
 
-    Matches the base URL's hostname against known providers; falls back to
-    the raw provider string from _detect_provider().
+    Checks path-based overrides first (for hosts serving multiple plans),
+    then matches the base URL's hostname against known providers, and
+    finally falls back to the raw provider string from _detect_provider().
     """
+    # Path-based overrides for hosts that serve multiple curated lists.
+    parsed = urlparse(base_url)
+    if _host_match(base_url, "z.ai") and "/api/coding" in (parsed.path or ""):
+        return "zai-coding"
     for domain, key in _HOST_TO_CURATED:
         if _host_match(base_url, domain):
             return key
@@ -203,9 +366,13 @@ def _probe_single_model(base: str, api_key: str, model_id: str, timeout: int = 1
         target_url = build_chat_url(base)
         h = build_headers(api_key, base)
         h["Content-Type"] = "application/json"
-        from src.llm_core import _uses_max_completion_tokens
+        from src.llm_core import _uses_max_completion_tokens, _restricts_temperature
         _max_key = "max_completion_tokens" if _uses_max_completion_tokens(model_id) else "max_tokens"
-        payload = {"model": model_id, "messages": messages, _max_key: 5, "temperature": 0.0}
+        payload = {"model": model_id, "messages": messages, _max_key: 5}
+        # Reasoning models (o1/o3/o4/gpt-5) reject an explicit temperature, so a
+        # probe that hardcodes one falsely reports a working endpoint as failing.
+        if not _restricts_temperature(model_id):
+            payload["temperature"] = 0.0
         if _test_tools:
             payload["tools"] = _test_tools
 
@@ -304,6 +471,13 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
         if not models:
             models = [m.get("name") or m.get("model") for m in (data.get("models") or []) if m.get("name") or m.get("model")]
         if models:
+            # Z.AI coding plan omits some working models from /models;
+            # append curated-only entries for that endpoint only.
+            if _host_match(base, "z.ai") and "/api/coding" in (urlparse(base).path or ""):
+                _ck = _match_provider_curated(base, None)
+                for _e in _PROVIDER_CURATED.get(_ck, []):
+                    if _e not in set(models) and not any(m.startswith(_e) for m in models):
+                        models.append(_e)
             return models
     except httpx.HTTPStatusError as e:
         if api_key:
@@ -348,7 +522,24 @@ def _ping_endpoint(base_url: str, api_key: str = None, timeout: float = 1.5) ->
     if api_key:
         headers["Authorization"] = f"Bearer {api_key}"
 
+    # Ollama exposes /v1/models (OpenAI-compatible) AND native /api/version,
+    # /api/tags. The OpenAI-style GET base + "/models" returns 404 when the
+    # base is the host root or the native /api root (e.g. http://localhost:11434,
+    # http://localhost:11434/api) because /models lives under /v1 there. Treat
+    # 4xx on a port-11434 / Ollama-named base as "try the native paths" rather
+    # than as a definitive offline verdict — Ollama is reachable, it just
+    # doesn't speak OpenAI on that prefix. Without this gate the quickstart
+    # marks an alive Ollama as offline whenever cached_models is empty (issue
+    # #1025): _probe_endpoint() falls through to /api/tags on the same 404, but
+    # _ping_endpoint() was returning before that fallback could run.
+    parsed_base = urlparse(base)
+    looks_like_ollama = (
+        parsed_base.port == 11434
+        or "ollama" in (parsed_base.hostname or "").lower()
+    )
+
     url = base + "/models"
+    last_error: Optional[str] = None
     try:
         r = httpx.get(url, headers=headers, timeout=timeout)
         if 300 <= r.status_code < 400:
@@ -360,17 +551,21 @@ def _ping_endpoint(base_url: str, api_key: str = None, timeout: float = 1.5) ->
                     "error": "That is Odysseus, not a model server. Use the Ollama URL, usually http://host.docker.internal:11434/v1 in Docker.",
                 }
             return {"reachable": False, "status_code": r.status_code, "error": f"HTTP {r.status_code} redirect"}
-        if r.status_code < 500:
-            return {"reachable": r.status_code < 400, "status_code": r.status_code, "error": None if r.status_code < 400 else f"HTTP {r.status_code}"}
+        if r.status_code < 400:
+            return {"reachable": True, "status_code": r.status_code, "error": None}
+        if r.status_code < 500 and not looks_like_ollama:
+            return {"reachable": False, "status_code": r.status_code, "error": f"HTTP {r.status_code}"}
+        last_error = f"HTTP {r.status_code}"
     except Exception as e:
         last_error = str(e)[:120]
-    else:
-        last_error = f"HTTP {r.status_code}"
 
     try:
-        parsed = urlparse(base)
-        if parsed.port == 11434 or "ollama" in (parsed.hostname or "").lower():
-            root = base[:-3].rstrip("/") if base.endswith("/v1") else base
+        if looks_like_ollama:
+            root = base
+            for suffix in ("/v1", "/api"):
+                if root.endswith(suffix):
+                    root = root[: -len(suffix)].rstrip("/")
+                    break
             for path in ("/api/version", "/api/tags"):
                 try:
                     r = httpx.get(root + path, timeout=timeout)
@@ -410,6 +605,15 @@ def _model_endpoint_error_message(base_url: str, ping: Dict[str, Any] = None) ->
     return "No models found for that provider/key."
 
 
+def _visible_models(cached_models, hidden_models):
+    """Filter cached model IDs by hidden_models. Returns list of visible IDs."""
+    all_models = json.loads(cached_models) if isinstance(cached_models, str) else (cached_models or [])
+    if not hidden_models:
+        return all_models
+    hidden = set(json.loads(hidden_models) if isinstance(hidden_models, str) else (hidden_models or []))
+    return [m for m in all_models if m not in hidden]
+
+
 def setup_model_routes(model_discovery):
     router = APIRouter(prefix="/api")
 
@@ -926,6 +1130,7 @@ def setup_model_routes(model_discovery):
         require_models: str = Form("false"),
         model_type: str = Form("llm"),
         supports_tools: str = Form(""),  # "true"/"false"/"" (unknown)
+        container_local: str = Form("false"),
         # Default `shared=true` → endpoints are visible to all users (the
         # app's historical behaviour). Admins can pass `shared=false` to
         # scope a new endpoint to their own account only.
@@ -938,6 +1143,10 @@ def setup_model_routes(model_discovery):
         # Resolve hostname via Tailscale if DNS fails
         from src.endpoint_resolver import resolve_url
         base_url = resolve_url(base_url)
+        # In Docker, manually added loopback URLs usually point at a host-local
+        # server. Cookbook local serves are launched inside Odysseus itself, so
+        # keep those container-local when the frontend marks them as such.
+        base_url = _rewrite_loopback_for_docker(base_url, container_local=_truthy(container_local))
 
         # Auto-generate name from URL if not provided
         if not name.strip():
@@ -1046,6 +1255,7 @@ def setup_model_routes(model_discovery):
             raise HTTPException(400, "Base URL is required")
         from src.endpoint_resolver import resolve_url
         base_url = resolve_url(base_url)
+        base_url = _rewrite_loopback_for_docker(base_url)
         probe_timeout = 3 if (":11434" in base_url or "ollama" in base_url.lower()) else 2
         models = _probe_endpoint(base_url, api_key.strip() or None, timeout=probe_timeout)
         ping = {"reachable": True, "error": None} if models else _ping_endpoint(base_url, api_key.strip() or None, timeout=probe_timeout)
@@ -1258,9 +1468,9 @@ def setup_model_routes(model_discovery):
             chat_url = build_chat_url(base)
             if not model and getattr(ep, "cached_models", None):
                 try:
-                    models = _json.loads(ep.cached_models) if isinstance(ep.cached_models, str) else ep.cached_models
-                    if models:
-                        model = models[0]
+                    visible = _visible_models(ep.cached_models, getattr(ep, "hidden_models", None))
+                    if visible:
+                        model = visible[0]
                 except Exception:
                     pass
             return {"endpoint_id": ep.id, "endpoint_url": chat_url, "model": model}
@@ -1326,44 +1536,31 @@ def setup_model_routes(model_discovery):
         finally:
             db.close()
 
-    # ── Settings fields that store an endpoint ID ──
-    _EP_SETTING_FIELDS = {
-        "default_endpoint_id":  ("default_model",  "Default Model"),
-        "utility_endpoint_id":  ("utility_model",   "Utility Model"),
-        "research_endpoint_id": ("research_model",  "Deep Research"),
-        "task_endpoint_id":     ("task_model",       "Background Tasks"),
-    }
-
     def _settings_using_endpoint(ep_id: str) -> list:
         """Return human-readable labels for settings that reference this endpoint."""
-        settings = _load_settings()
-        affected = []
-        for ep_key, (_, label) in _EP_SETTING_FIELDS.items():
-            if (settings.get(ep_key) or "") == ep_id:
-                affected.append(label)
-        tts_prov = settings.get("tts_provider") or ""
-        if tts_prov == f"endpoint:{ep_id}":
-            affected.append("Text to Speech")
-        return affected
+        return _endpoint_settings_using_endpoint(_load_settings(), ep_id, include_speech=True)
 
     def _clear_settings_for_endpoint(ep_id: str) -> list:
         """Clear all settings that reference this endpoint. Returns list of cleared labels."""
         settings = _load_settings()
-        cleared = []
-        for ep_key, (model_key, label) in _EP_SETTING_FIELDS.items():
-            if (settings.get(ep_key) or "") == ep_id:
-                settings[ep_key] = ""
-                settings[model_key] = ""
-                cleared.append(label)
-        tts_prov = settings.get("tts_provider") or ""
-        if tts_prov == f"endpoint:{ep_id}":
-            settings["tts_provider"] = "disabled"
-            settings["tts_model"] = "tts-1"
-            cleared.append("Text to Speech")
+        cleared = _clear_endpoint_settings_for_endpoint(settings, ep_id, include_speech=True)
         if cleared:
             _save_settings(settings)
         return cleared
 
+    def _clear_user_prefs_for_endpoint(ep_id: str) -> int:
+        """Clear per-user endpoint selections and fallback chains."""
+        try:
+            from routes.prefs_routes import _load as _load_prefs, _save as _save_prefs
+            all_prefs = _load_prefs()
+            cleared_users = _clear_user_pref_endpoint_refs(all_prefs, ep_id)
+            if cleared_users:
+                _save_prefs(all_prefs)
+            return cleared_users
+        except Exception as e:
+            logger.warning("Failed to clear user prefs for endpoint %s: %s", ep_id, e)
+            return 0
+
     def _session_uses_endpoint_url(session_url: str, base_url: str) -> bool:
         if not session_url or not base_url:
             return False
@@ -1428,6 +1625,7 @@ def setup_model_routes(model_discovery):
                 raise HTTPException(404, "Endpoint not found")
             # Clean up any settings that reference this endpoint
             cleared = _clear_settings_for_endpoint(ep_id)
+            cleared_user_preferences = _clear_user_prefs_for_endpoint(ep_id)
             cleared_sessions = _clear_sessions_for_endpoint(db, ep.base_url)
             cleared_loaded_sessions = _clear_loaded_sessions_for_endpoint(ep.base_url)
             db.delete(ep)
@@ -1437,6 +1635,7 @@ def setup_model_routes(model_discovery):
             return {
                 "deleted": True,
                 "cleared_settings": cleared,
+                "cleared_user_preferences": cleared_user_preferences,
                 "cleared_sessions": cleared_sessions,
                 "cleared_loaded_sessions": cleared_loaded_sessions,
             }

routes/note_routes.py

@@ -683,9 +683,8 @@ def setup_note_routes(task_scheduler=None):
         Returns {synthesis, email_sent}.
         """
         # Gate against anonymous callers — LLM synthesis can burn tokens.
-        from src.auth_helpers import get_current_user as _gcu
-        if not _gcu(request):
-            raise HTTPException(401, "Not authenticated")
+        from src.auth_helpers import require_user as _ru
+        _ru(request)
         body = await request.json()
         note_id = body.get("note_id")
         title = (body.get("title") or "").strip()
@@ -697,7 +696,7 @@ def setup_note_routes(task_scheduler=None):
         # the same dispatch without an HTTP roundtrip + auth cookie.
         return await dispatch_reminder(
             title=title, note_body=note_body, note_id=note_id,
-            owner=_gcu(request) or "",
+            owner=_owner(request) or "",
             queue_browser=False,
         )
 

routes/session_routes.py

@@ -14,20 +14,45 @@ from core.database import Session as DbSession, SessionLocal, Document, GalleryI
 from src.auth_helpers import get_current_user, effective_user
 
 
-def _verify_session_owner(request: Request, session_id: str):
-    """Verify the current user owns the session. Raises 404 if not."""
+def _sanitize_export_filename(name: str) -> str:
+    """Return a conservative filename safe for Content-Disposition."""
+    name = name or ""
+    name = re.sub(r"[^A-Za-z0-9._-]", "_", name)
+    return name[:128]
+
+
+def _verify_session_owner(request: Request, session_id: str, session_manager=None):
+    """Verify the current user owns the session. Raises 404 if not.
+
+    Ownership is checked against the DB row when one exists (unchanged). If
+    there is no DB row but the caller owns an in-memory "ghost" session — one
+    that lives only in ``session_manager`` because it was never persisted, or
+    its DB row was removed out-of-band — fall back to the in-memory owner so the
+    user can still manage and delete it. Without this fallback such sessions are
+    listed by ``/api/sessions`` (they come from the in-memory manager) yet every
+    per-session operation 404s, making them impossible to delete (issue #1044).
+
+    ``session_manager`` is optional and defaults to ``None`` so existing callers
+    that only care about persisted sessions keep their exact prior behavior.
+    """
     user = effective_user(request)
     if not user:
         raise HTTPException(403, "Authentication required")
     db = SessionLocal()
     try:
         row = db.query(DbSession.owner).filter(DbSession.id == session_id).first()
-        if not row:
-            raise HTTPException(404, f"Session {session_id} not found")
-        if row.owner != user:
-            raise HTTPException(404, f"Session {session_id} not found")
     finally:
         db.close()
+    if row is not None:
+        if row.owner != user:
+            raise HTTPException(404, f"Session {session_id} not found")
+        return
+    # No DB row — allow the caller to act on an in-memory ghost they own.
+    if session_manager is not None:
+        ghost = getattr(session_manager, "sessions", {}).get(session_id)
+        if ghost is not None and getattr(ghost, "owner", None) == user:
+            return
+    raise HTTPException(404, f"Session {session_id} not found")
 
 logger = logging.getLogger(__name__)
 
@@ -356,7 +381,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
             ids = []
         for sid in ids:
             try:
-                _verify_session_owner(request, sid)
+                _verify_session_owner(request, sid, session_manager)
                 session_manager.delete_session(sid)
                 db = SessionLocal()
                 try:
@@ -374,7 +399,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
     @router.delete("/session/{sid}")
     def delete_session(request: Request, sid: str):
         """Permanently delete a session and all its messages."""
-        _verify_session_owner(request, sid)
+        _verify_session_owner(request, sid, session_manager)
         try:
             # Block deletion of starred/favorited sessions
             db = SessionLocal()
@@ -558,6 +583,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
 
         safe_name = re.sub(r'[^\w\-_]', '_', session.name)
         timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+        filename = _sanitize_export_filename(filename)
 
         if fmt == "json":
             import json as _json

routes/shell_routes.py

@@ -118,6 +118,7 @@ def _running_in_container(dockerenv_path="/.dockerenv", cgroup_path="/proc/1/cgr
 
 
 DockerRowStatus = namedtuple("DockerRowStatus", ["applicable", "install_hint"])
+PackageUpdateStatus = namedtuple("PackageUpdateStatus", ["available", "note"])
 
 
 def _docker_row_status(*, on_remote, in_container, installed, default_hint):
@@ -127,6 +128,24 @@ def _docker_row_status(*, on_remote, in_container, installed, default_hint):
     return DockerRowStatus(applicable=True, install_hint=default_hint)
 
 
+def _pip_dist_name(pkg: dict) -> str:
+    """Distribution name for importlib.metadata lookups.
+
+    The Cookbook package catalog carries both the import name (``name``, e.g.
+    ``llama_cpp``) and the pip spec (``pip``, e.g. ``llama-cpp-python[server]``).
+    The distribution is NOT always the import name with underscores swapped for
+    dashes — ``llama_cpp`` ships in the ``llama-cpp-python`` distribution — so
+    derive it from the pip spec (stripping any ``[extras]`` and version markers)
+    and fall back to the munged import name only when no pip spec is declared.
+    """
+    pip = (pkg.get("pip") or "").strip()
+    if pip:
+        base = re.split(r"[\[<>=!~;\s]", pip, maxsplit=1)[0].strip()
+        if base:
+            return base
+    return (pkg.get("name") or "").replace("_", "-")
+
+
 def _package_installed_from_probe(name: str, probe: dict) -> bool:
     """Return whether an optional dependency is usable by Cookbook.
 
@@ -162,7 +181,10 @@ def _package_status_note(name: str, probe: dict) -> str:
     locations = module.get("locations") or []
     if name == "vllm":
         if binaries.get("vllm"):
-            return f"vLLM CLI: {binaries['vllm']}"
+            parts = [f"vLLM CLI: {binaries['vllm']}"]
+            if dists.get("vllm"):
+                parts.append(f"python package: vllm {dists['vllm']}")
+            return "; ".join(parts)
         if module.get("found") and not dists.get("vllm"):
             loc = locations[0] if locations else module.get("origin") or "unknown path"
             return f"Python sees a vllm namespace at {loc}, but no vLLM CLI is on PATH."
@@ -183,6 +205,35 @@ def _package_status_note(name: str, probe: dict) -> str:
     return ""
 
 
+def _package_pip_update_status(pkg: dict, probe: dict | None = None) -> PackageUpdateStatus:
+    """Return whether the Dependencies UI should offer a generic pip update.
+
+    "Installed" means Cookbook can use the dependency. It does not always mean
+    the dependency is a Python package that Cookbook should update with pip:
+    native llama-server can come from a package manager/source build, and a CLI
+    may be on PATH without matching Python package metadata.
+    """
+    if pkg.get("kind") == "system" or not pkg.get("pip"):
+        return PackageUpdateStatus(False, "Update this system dependency outside Odysseus.")
+
+    name = pkg.get("name")
+    binaries = probe.get("binaries") if isinstance(probe, dict) and isinstance(probe.get("binaries"), dict) else {}
+    dists = probe.get("dists") if isinstance(probe, dict) and isinstance(probe.get("dists"), dict) else {}
+
+    if name == "llama_cpp" and binaries.get("llama-server"):
+        return PackageUpdateStatus(
+            False,
+            "Using native llama-server on PATH; update it with its package manager or source checkout.",
+        )
+    if name == "vllm" and binaries.get("vllm") and not dists.get("vllm"):
+        return PackageUpdateStatus(
+            False,
+            "Using a vLLM CLI on PATH without Python package metadata; update it outside Odysseus.",
+        )
+
+    return PackageUpdateStatus(True, "Update uses pip in the selected Python environment.")
+
+
 def _prepend_user_install_bins_to_path() -> None:
     """Make pip --user console scripts visible to dependency probes.
 
@@ -926,6 +977,7 @@ def setup_shell_routes() -> APIRouter:
 
         for pkg in packages:
             on_remote = bool(host and pkg.get("target") == "remote")
+            probe = None
             if on_remote:
                 pkg["installed"] = bool(remote_status.get(pkg["name"], False))
                 probe = remote_details.get(pkg["name"])
@@ -939,19 +991,36 @@ def setup_shell_routes() -> APIRouter:
             elif pkg["name"] == "llama_cpp" and shutil.which("llama-server"):
                 pkg["installed"] = True
                 pkg["status_note"] = f"native llama-server: {shutil.which('llama-server')}"
+                probe = {"binaries": {"llama-server": shutil.which("llama-server")}, "dists": {}}
+            elif pkg["name"] == "vllm":
+                _vllm_cli = shutil.which("vllm")
+                pkg["installed"] = _vllm_cli is not None
+                if pkg["installed"]:
+                    try:
+                        _vllm_version = importlib_metadata.version(_pip_dist_name(pkg))
+                    except importlib_metadata.PackageNotFoundError:
+                        _vllm_version = None
+                    probe = {
+                        "binaries": {"vllm": _vllm_cli},
+                        "dists": {"vllm": _vllm_version} if _vllm_version else {},
+                    }
+                    pkg["status_note"] = _package_status_note("vllm", probe)
             else:
                 try:
                     importlib.import_module(pkg["name"])
-                    if pkg["name"] == "vllm":
-                        pkg["installed"] = shutil.which("vllm") is not None
-                    else:
-                        importlib_metadata.version(pkg["name"].replace("_", "-"))
-                        pkg["installed"] = True
+                    importlib_metadata.version(_pip_dist_name(pkg))
+                    pkg["installed"] = True
                 except ImportError:
                     pkg["installed"] = False
                 except importlib_metadata.PackageNotFoundError:
                     pkg["installed"] = False
 
+            if pkg.get("installed"):
+                update_status = _package_pip_update_status(pkg, probe)
+                pkg["pip_update_available"] = update_status.available
+                if update_status.note:
+                    pkg["update_note"] = update_status.note
+
             if pkg["name"] == "docker":
                 status = _docker_row_status(
                     on_remote=on_remote,

routes/skills_routes.py

@@ -469,7 +469,7 @@ async def _run_skill_test_job(key, name, md, task, url, model, headers, owner, s
         conf = {"pass": 0.95, "needs_work": 0.6, "fail": 0.4}.get(v)
         if conf is not None:
             try:
-                skills_manager.update_skill(name, {"confidence": conf})
+                skills_manager.update_skill(name, {"confidence": conf}, owner=owner)
             except Exception:
                 pass
     job["status"] = "done"
@@ -638,7 +638,7 @@ def _audit_finalize_status(skills_manager, name: str, owner, verdict: str,
     c = float(confidence or 0.0)
     status = "published" if (auto_publish and necessary and verdict == "pass" and c >= min_conf) else "draft"
     try:
-        skills_manager.update_skill(name, {"status": status})
+        skills_manager.update_skill(name, {"status": status}, owner=owner)
     except Exception:
         pass
     return status
@@ -662,7 +662,7 @@ def _apply_skill_md(skills_manager, name: str, md: str, owner) -> bool:
             "teacher_model": sk.teacher_model, "owner": sk.owner or owner,
             "when_to_use": sk.when_to_use, "procedure": sk.procedure,
             "pitfalls": sk.pitfalls, "verification": sk.verification, "body_extra": sk.body_extra,
-        }))
+        }, owner=owner))
     except Exception as e:
         logger.warning(f"Audit: could not save edited skill {name}: {e}")
         return False
@@ -762,7 +762,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
     # earns a bit less; a skill that still fails is marked low.
     def _set_conf(c):
         try:
-            skills_manager.update_skill(name, {"confidence": c})
+            skills_manager.update_skill(name, {"confidence": c}, owner=owner)
         except Exception:
             pass
 
@@ -799,7 +799,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
     if generic_reason or duplicate_of or (isinstance(nec, dict) and nec.get("necessary") is False):
         reason = generic_reason or (f"Lower-priority duplicate of {duplicate_of}" if duplicate_of else str((nec or {}).get("reason") or "Unnecessary skill"))
         try:
-            skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35})
+            skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35}, owner=owner)
             skills_manager.set_audit(name, "skipped", by_teacher=False, worker_model=model)
             if duplicate_of:
                 skills_manager.set_necessity(name, False, [duplicate_of], reason)
@@ -901,7 +901,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
 
     # Still failing → demote to draft + low confidence + flag (do NOT delete).
     try:
-        skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35})
+        skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35}, owner=owner)
     except Exception:
         pass
     skills_manager.set_audit(
@@ -1474,7 +1474,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
             "pitfalls": sk.pitfalls,
             "verification": sk.verification,
             "body_extra": sk.body_extra,
-        })
+        }, owner=user)
         if not ok:
             raise HTTPException(500, "Update failed")
         # Manual markdown edits can create or substantially rewrite a draft
@@ -1496,7 +1496,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
         updates = body.dict(exclude_none=True)
         if not updates:
             return {"ok": True}
-        ok = skills_manager.update_skill(match.get("name"), updates)
+        ok = skills_manager.update_skill(match.get("name"), updates, owner=user)
         if not ok:
             raise HTTPException(404, "Skill not found")
         if not match.get("audit_verdict"):
@@ -1511,7 +1511,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
         if not match:
             raise HTTPException(404, "Skill not found")
         _verify_owner(match, user)
-        ok = skills_manager.delete_skill(match.get("name"))
+        ok = skills_manager.delete_skill(match.get("name"), owner=user)
         if not ok:
             raise HTTPException(404, "Skill not found")
         return {"ok": True}

routes/vault_routes.py

@@ -61,7 +61,8 @@ def _find_bw() -> str:
 def _load_config() -> dict:
     if VAULT_FILE.exists():
         try:
-            return json.loads(VAULT_FILE.read_text(encoding="utf-8"))
+            data = json.loads(VAULT_FILE.read_text(encoding="utf-8"))
+            return data if isinstance(data, dict) else {}
         except Exception:
             pass
     return {}

routes/webhook_routes.py

@@ -26,6 +26,25 @@ MAX_MESSAGE_LEN = 32_000
 from core.middleware import require_admin as _require_admin
 
 
+def _first_enabled_endpoint(db, owner):
+    """First enabled ModelEndpoint VISIBLE to `owner` — their own rows plus
+    legacy null-owner ("shared") rows. Owner-scoped on purpose: ModelEndpoint
+    is per-user (core/database.py — "when non-null, the model picker only shows
+    the endpoint to that user"), and the sync-chat fallback uses the row's
+    decrypted `api_key`. An unscoped ``.first()`` would let a chat-scoped token
+    (e.g. a paired mobile device) fall back onto ANOTHER user's private
+    endpoint and silently spend that owner's API key / quota — and reach
+    whatever internal base_url they configured. Mirrors the owner_filter scoping
+    in routes/model_routes.py and companion/routes.py. A null/empty owner is a
+    no-op (single-user / legacy mode), preserving the original behaviour.
+    """
+    from core.database import ModelEndpoint
+    from src.auth_helpers import owner_filter
+    q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)  # noqa: E712
+    q = owner_filter(q, ModelEndpoint, owner)
+    return q.first()
+
+
 def _caller_owns_session(sess_owner, caller) -> bool:
     """Strict session-ownership gate for the token-authenticated sync-chat
     endpoint (`POST /api/v1/chat`).
@@ -178,6 +197,7 @@ def setup_webhook_routes(
         "openrouter": "https://openrouter.ai/api/v1",
         "ollama": "https://ollama.com/api",
         "fireworks": "https://api.fireworks.ai/inference/v1",
+        "venice": "https://api.venice.ai/api/v1",
     }
 
     # Model prefix → provider mapping for auto-detection
@@ -222,7 +242,6 @@ def setup_webhook_routes(
 
         from core.models import ChatMessage
         from src.llm_core import llm_call_async
-        from core.database import ModelEndpoint
         from src.endpoint_resolver import build_chat_url, build_headers, build_models_url, normalize_base
 
         message = body.message.strip()
@@ -287,7 +306,9 @@ def setup_webhook_routes(
         if not sess:
             db = SessionLocal()
             try:
-                ep = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).first()
+                # Owner-scoped: only THIS token owner's endpoints + legacy
+                # shared rows, never another user's private endpoint/api_key.
+                ep = _first_enabled_endpoint(db, token_owner)
             finally:
                 db.close()
 

scripts/add_hwfit_models.py

@@ -9,7 +9,9 @@ Adds:
 
 Metadata is taken from the HF Hub `list_models(full=True)` response plus the
 repo name (which encodes the param size, e.g. "Qwen3.6-35B-A3B"). Param-less
-names fall back to a single per-repo model_info() call to read safetensors.
+names fall back, in order, to the parent `base_model:` tag, the repo's
+`config.json` (computed from `hidden_size` / `num_hidden_layers` / MoE
+fields), and finally a per-repo `model_info()` call to read safetensors.
 
 Re-runnable: merges by `name`, leaving existing entries untouched unless
 --overwrite is passed. Writes a .bak first.
@@ -23,7 +25,8 @@ import re
 import sys
 from datetime import datetime
 
-from huggingface_hub import HfApi
+from huggingface_hub import HfApi, hf_hub_download
+from huggingface_hub.utils import EntryNotFoundError, RepositoryNotFoundError
 
 DATA_PATH = os.path.join(os.path.dirname(__file__), "..", "services", "hwfit", "data", "hf_models.json")
 DATA_PATH = os.path.abspath(DATA_PATH)
@@ -70,6 +73,128 @@ def _parse_params(name):
     return total, active
 
 
+def _params_from_config(cfg):
+    """Estimate (total, active) parameter counts from a HF config.json dict.
+
+    Returns (None, None) when the architecture fields aren't usable. Covers:
+      * explicit ``num_parameters`` / ``n_params`` (rare but authoritative)
+      * dense transformers (LLaMA / Qwen / Mistral / GLM-dense / etc.) via
+        embeddings + per-layer attention + MLP
+      * MoE (Qwen3-MoE, GLM-4-MoE, DeepSeek-style) using ``num_experts`` or
+        ``n_routed_experts`` (+ ``n_shared_experts``). Active count assumes
+        ``num_experts_per_tok`` routed experts plus any shared experts.
+
+    The estimate is intentionally coarse — within ~5-10% of the true count for
+    standard decoder-only architectures — which is fine for the downstream
+    ``min_vram_gb`` heuristic (it already buckets via ``parameter_count`` to
+    one decimal place of "B").
+    """
+    if not isinstance(cfg, dict):
+        return None, None
+
+    # Authoritative fields first. Some custom configs embed the trained
+    # parameter count directly.
+    for key in ("num_parameters", "n_params", "total_params"):
+        v = cfg.get(key)
+        if isinstance(v, (int, float)) and v > 0:
+            return int(v), None
+
+    def _i(key, default=None):
+        v = cfg.get(key, default)
+        try:
+            return int(v) if v is not None else None
+        except (TypeError, ValueError):
+            return None
+
+    h = _i("hidden_size")
+    L = _i("num_hidden_layers")
+    if not h or not L:
+        return None, None
+
+    vocab = _i("vocab_size") or 0
+    ffn = _i("intermediate_size") or (4 * h)
+    n_heads = _i("num_attention_heads") or 0
+    n_kv = _i("num_key_value_heads") or n_heads
+    head_dim = _i("head_dim") or (h // n_heads if n_heads else h)
+
+    # Attention: Q is hidden_size wide, KV is grouped (GQA / MQA).
+    q_proj = h * (n_heads * head_dim if n_heads else h)
+    kv_proj = 2 * h * (n_kv * head_dim if n_kv else h)
+    o_proj = (n_heads * head_dim if n_heads else h) * h
+    per_layer_attn = q_proj + kv_proj + o_proj
+
+    # Dense MLP: gate + up + down (SwiGLU / GeGLU). Configs without a gate
+    # (plain GELU) are within the noise floor of this estimate.
+    per_layer_dense_mlp = 3 * h * ffn
+
+    # MoE routing. Both naming conventions are seen in the wild.
+    n_experts = _i("num_experts") or _i("n_routed_experts") or 0
+    n_shared = _i("n_shared_experts") or 0
+    n_active = _i("num_experts_per_tok") or 0
+    moe_ffn = _i("moe_intermediate_size") or ffn
+    # Some configs (GLM-4-MoE, DeepSeek-V3) keep the first K layers dense.
+    first_dense = _i("first_k_dense_replace") or 0
+
+    if n_experts > 0 and n_active > 0:
+        moe_layers = max(0, L - first_dense)
+        dense_layers = L - moe_layers
+        per_expert = 3 * h * moe_ffn
+        total_mlp = (
+            dense_layers * per_layer_dense_mlp
+            + moe_layers * (n_experts + n_shared) * per_expert
+        )
+        active_mlp = (
+            dense_layers * per_layer_dense_mlp
+            + moe_layers * (n_active + n_shared) * per_expert
+        )
+    else:
+        total_mlp = L * per_layer_dense_mlp
+        active_mlp = total_mlp
+
+    embed = vocab * h
+    # Untied output head doubles the embedding contribution.
+    head = 0 if cfg.get("tie_word_embeddings", True) else vocab * h
+
+    total = embed + head + L * per_layer_attn + total_mlp
+    active = embed + head + L * per_layer_attn + active_mlp
+    if total <= 0:
+        return None, None
+    if active == total or n_experts == 0:
+        return int(total), None
+    return int(total), int(active)
+
+
+_CONFIG_CACHE = {}
+
+
+def _fetch_config_json(repo_id):
+    """Download and cache a repo's config.json. Returns a dict or None.
+
+    Network / 404 / private-repo failures are swallowed — the caller already
+    has a safetensors fallback below this. We rely on huggingface_hub's own
+    on-disk cache so repeated script runs don't re-hit the Hub.
+    """
+    if repo_id in _CONFIG_CACHE:
+        return _CONFIG_CACHE[repo_id]
+    try:
+        path = hf_hub_download(repo_id=repo_id, filename="config.json")
+    except (EntryNotFoundError, RepositoryNotFoundError):
+        _CONFIG_CACHE[repo_id] = None
+        return None
+    except Exception:
+        # Network hiccup, gated repo, etc. — don't crash the bulk run.
+        _CONFIG_CACHE[repo_id] = None
+        return None
+    try:
+        with open(path, encoding="utf-8") as f:
+            cfg = json.load(f)
+    except (OSError, ValueError):
+        _CONFIG_CACHE[repo_id] = None
+        return None
+    _CONFIG_CACHE[repo_id] = cfg
+    return cfg
+
+
 def _base_model_tag(tags):
     """Return the `base_model:...` repo id from tags, if any."""
     for t in (tags or []):
@@ -141,6 +266,27 @@ def _entry_from_modelinfo(mi, overrides):
                     active = ba
     # Determine quant first — we need it to unpack the safetensors fallback.
     quant = _quant_from_name(name)
+    # Next-to-last resort: parse config.json. This is robust against
+    # parameter-less repo names (e.g. "GLM-4.5" with no "9B" suffix) where
+    # both the regex and the base_model tag come up empty. We try this
+    # before safetensors so non-standard names still resolve without a
+    # per-repo manual override in EXTRA_REPOS. Source repo first (works for
+    # unquantized models) then the quantized parent via base_model:.
+    if total is None:
+        config_targets = [name]
+        bm = _base_model_tag(getattr(mi, "tags", None))
+        if bm and bm != name:
+            config_targets.append(bm)
+        for target in config_targets:
+            cfg = _fetch_config_json(target)
+            if not cfg:
+                continue
+            ct, ca = _params_from_config(cfg)
+            if ct:
+                total = ct
+                if ca and active is None:
+                    active = ca
+                break
     # Last resort: read safetensors element counts. For pre-quantized repos
     # (AWQ/GPTQ/MLX-Int4 etc.) the weights are packed: 8× 4-bit weights per
     # I32 element, 4× 8-bit weights per I32. The bare safetensors total

scripts/check-docker-amd-gpu.sh

@@ -0,0 +1,205 @@
+#!/usr/bin/env bash
+# check-docker-amd-gpu.sh - read-only AMD/ROCm Docker passthrough diagnostic.
+#
+# This script does not install packages, edit .env, or restart Docker. It only
+# checks host AMD device nodes, Docker access, and whether a small container can
+# see /dev/kfd and /dev/dri. The Odysseus slim image does not include ROCm tools
+# such as rocm-smi, so container verification checks devices instead.
+
+set -u
+
+PASS=0
+FAIL=0
+WARN=0
+RENDER_GID=""
+VIDEO_GID=""
+TEST_IMAGE="${ODYSSEUS_AMD_TEST_IMAGE:-alpine:3.20}"
+
+_pass() { printf '\033[32m[PASS]\033[0m %s\n' "$*"; PASS=$((PASS + 1)); }
+_fail() { printf '\033[31m[FAIL]\033[0m %s\n' "$*"; FAIL=$((FAIL + 1)); }
+_warn() { printf '\033[33m[WARN]\033[0m %s\n' "$*"; WARN=$((WARN + 1)); }
+_info() { printf '\033[34m[INFO]\033[0m %s\n' "$*"; }
+
+_usage() {
+    cat <<'USAGE'
+Usage: scripts/check-docker-amd-gpu.sh
+
+Read-only AMD/ROCm Docker GPU diagnostic. Installs nothing, edits nothing, and
+does not restart Docker.
+
+Checks:
+  - host /dev/kfd and /dev/dri/renderD* exist
+  - host render group GID for RENDER_GID in .env
+  - optional host rocminfo visibility
+  - Docker can pass AMD device nodes into a small container
+
+Environment:
+  ODYSSEUS_AMD_TEST_IMAGE   Docker image for the passthrough smoke
+                            (default: alpine:3.20)
+USAGE
+}
+
+case "${1:-}" in
+    --help|-h)
+        _usage
+        exit 0
+        ;;
+    "")
+        ;;
+    *)
+        printf 'Unknown option: %s\n\n' "$1" >&2
+        _usage >&2
+        exit 1
+        ;;
+esac
+
+_find_cmd() {
+    if command -v "$1" >/dev/null 2>&1; then
+        command -v "$1"
+        return 0
+    fi
+    if [ -x "/opt/rocm/bin/$1" ]; then
+        printf '/opt/rocm/bin/%s\n' "$1"
+        return 0
+    fi
+    return 1
+}
+
+_check_host_devices() {
+    _info "Checking host AMD device nodes..."
+    if [ -e /dev/kfd ]; then
+        _pass "/dev/kfd exists"
+    else
+        _fail "/dev/kfd is missing - ROCm kernel driver access is not available."
+    fi
+
+    if [ -d /dev/dri ]; then
+        _pass "/dev/dri exists"
+    else
+        _fail "/dev/dri is missing - render devices are not available."
+        return
+    fi
+
+    render_nodes="$(find /dev/dri -maxdepth 1 -type c -name 'renderD*' -print 2>/dev/null | sort)"
+    if [ -n "${render_nodes}" ]; then
+        _pass "Render nodes found:"
+        printf '%s\n' "${render_nodes}" | sed 's/^/        /'
+    else
+        _fail "No /dev/dri/renderD* node found."
+    fi
+    echo
+}
+
+_check_groups() {
+    _info "Checking host render/video groups..."
+    RENDER_GID="$(getent group render | awk -F: '{print $3; exit}')"
+    VIDEO_GID="$(getent group video | awk -F: '{print $3; exit}')"
+
+    if [ -n "${RENDER_GID}" ]; then
+        _pass "render group GID: ${RENDER_GID}"
+    else
+        _fail "render group not found - set RENDER_GID manually if your distro uses a different group."
+    fi
+
+    if [ -n "${VIDEO_GID}" ]; then
+        _pass "video group GID: ${VIDEO_GID}"
+    else
+        _warn "video group not found. /dev/kfd and renderD* may still be enough on some hosts."
+    fi
+    echo
+}
+
+_check_host_rocm() {
+    _info "Checking host ROCm tools..."
+    rocminfo_cmd="$(_find_cmd rocminfo || true)"
+    if [ -n "${rocminfo_cmd}" ]; then
+        if "${rocminfo_cmd}" 2>/dev/null | grep -Eq 'gfx[0-9a-f]+'; then
+            _pass "rocminfo works on the host: ${rocminfo_cmd}"
+            "${rocminfo_cmd}" 2>/dev/null \
+                | grep -E 'Marketing Name:|Name:[[:space:]]+gfx' \
+                | head -12 \
+                | sed 's/^/        /'
+        else
+            _warn "rocminfo exists but did not list a gfx target."
+        fi
+    else
+        _warn "rocminfo not found on PATH or /opt/rocm/bin. This does not block Docker passthrough, but host ROCm may be incomplete."
+    fi
+    echo
+}
+
+_check_docker() {
+    _info "Checking Docker..."
+    if ! command -v docker >/dev/null 2>&1; then
+        _fail "docker not found - install Docker first."
+        echo
+        return 1
+    fi
+    if docker info >/dev/null 2>&1; then
+        _pass "Docker daemon is running."
+    else
+        _fail "Docker daemon is not running or this user lacks Docker permission."
+        echo
+        return 1
+    fi
+    echo
+}
+
+_check_docker_passthrough() {
+    if [ -z "${RENDER_GID}" ]; then
+        _fail "Skipping Docker passthrough smoke because render GID is unknown."
+        echo
+        return
+    fi
+
+    _info "Testing AMD device passthrough with ${TEST_IMAGE} (may pull on first run)..."
+    group_args=(--group-add "${RENDER_GID}")
+    if [ -n "${VIDEO_GID}" ]; then
+        group_args+=(--group-add "${VIDEO_GID}")
+    fi
+
+    if docker run --rm \
+        --device=/dev/kfd \
+        --device=/dev/dri \
+        "${group_args[@]}" \
+        "${TEST_IMAGE}" \
+        sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls /dev/dri/renderD* >/dev/null' \
+        >/dev/null 2>&1; then
+        _pass "Docker can pass /dev/kfd and /dev/dri render nodes into a container."
+    else
+        _fail "Docker AMD device passthrough failed."
+        _info "Check that Docker can access /dev/kfd and /dev/dri, then retry."
+    fi
+    echo
+}
+
+_print_next_steps() {
+    echo "=== Suggested .env values ==="
+    if [ -n "${RENDER_GID}" ]; then
+        printf 'COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml\n'
+        printf 'RENDER_GID=%s\n' "${RENDER_GID}"
+    else
+        printf 'COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml\n'
+        printf 'RENDER_GID=<numeric render group id>\n'
+    fi
+    echo
+    echo "After restarting Odysseus, verify the slim app container sees devices:"
+    echo "  docker compose exec odysseus sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls -l /dev/kfd /dev/dri/renderD*'"
+    echo
+    echo "Note: rocm-smi/rocminfo are not expected inside the slim Odysseus image."
+    echo "Device passthrough is necessary but not sufficient for GPU serving; vLLM and"
+    echo "llama.cpp still need ROCm-compatible builds or ROCm-specific Docker images."
+}
+
+echo "=== Odysseus AMD Docker GPU diagnostic ==="
+echo
+_check_host_devices
+_check_groups
+_check_host_rocm
+if _check_docker; then
+    _check_docker_passthrough
+fi
+_print_next_steps
+echo
+echo "=== Results: ${PASS} passed, ${WARN} warnings, ${FAIL} failed ==="
+[ "${FAIL}" -eq 0 ]

services/hwfit/data/hf_models.json

@@ -4375,7 +4375,14 @@
   "hf_downloads": 51135,
   "hf_likes": 2,
   "release_date": "2025-09-23",
-  "_discovered": true
+  "_discovered": true,
+  "gguf_sources": [
+   {
+    "repo": "typhoon-ai/typhoon2.5-qwen3-4b-gguf",
+    "file": "typhoon2.5-qwen3-4b-q4_k_m.gguf",
+    "quant": "Q4_K_M"
+   }
+  ]
  },
  {
   "name": "JunHowie/Qwen3-4B-Instruct-2507-GPTQ-Int4",
@@ -8994,7 +9001,14 @@
   "num_experts": 128,
   "active_experts": 8,
   "active_parameters": 3339450907,
-  "_discovered": true
+  "_discovered": true,
+  "gguf_sources": [
+   {
+    "repo": "typhoon-ai/typhoon2.5-qwen3-30b-a3b-gguf",
+    "file": "typhoon2.5-qwen3-30b-a3b-q4_k_m.gguf",
+    "quant": "Q4_K_M"
+   }
+  ]
  },
  {
   "name": "QuantTrio/Qwen3-Coder-30B-A3B-Instruct-AWQ",

services/hwfit/fit.py

@@ -18,7 +18,7 @@ GPU_BANDWIDTH = {
     "7900 xtx": 960, "7900 xt": 800, "7900 gre": 576, "7800 xt": 624, "7700 xt": 432, "7600": 288,
     "6950 xt": 576, "6900 xt": 512, "6800 xt": 512, "6800": 512, "6700 xt": 384, "6600 xt": 256, "6600": 224,
     "mi300x": 5300, "mi300": 5300, "mi250x": 3277, "mi250": 3277, "mi210": 1638, "mi100": 1229,
-    "9070 xt": 624, "9070": 488,
+    "9070 xt": 624, "9070": 488, "9060 xt": 322, "9060": 322,
     # Apple Silicon unified-memory bandwidth (GB/s). Keyed off the chip name
     # reported by sysctl machdep.cpu.brand_string (e.g. "Apple M4 Max"). Listed
     # before the bare "m_" keys matters less than length-sorting (done below),
@@ -70,8 +70,18 @@ def _lookup_bandwidth(gpu_name):
     return None
 
 
-def _estimate_speed(model, quant, run_mode, system):
-    """Estimate tok/s. Uses active params for MoE (only active experts run per token)."""
+def _estimate_speed(model, quant, run_mode, system, offload_frac=0.0):
+    """Estimate tok/s. Uses active params for MoE (only active experts run per token).
+
+    offload_frac (0..1): fraction of the model's weights that spill to system RAM
+    (CPU) because they don't fit VRAM. Generation reads every active weight per
+    token, so when part lives in CPU RAM the per-token time is dominated by the
+    slow path. We model effective bandwidth as a blend of GPU VRAM bandwidth and
+    system-RAM bandwidth weighted by what's where — far more accurate than a flat
+    "halve it" for partial offload, which under/over-shoots depending on amount.
+    Calibrated against a measured RX 9060 XT: DeepSeek-Coder-V2-Lite Q4_K_M with
+    light offload → ~59 t/s est vs 59.8 measured.
+    """
     pb = _active_params_b(model)
     is_moe = model.get("is_moe", False)
     bw = _lookup_bandwidth(system.get("gpu_name"))
@@ -83,14 +93,24 @@ def _estimate_speed(model, quant, run_mode, system):
         if model_gb <= 0:
             return 0.0
         efficiency = 0.55
-        raw_tps = (bw / model_gb) * efficiency
         if run_mode == "cpu_offload":
-            mode_factor = 0.5
-        elif is_moe:
-            mode_factor = 0.8
-        else:
-            mode_factor = 1.0
-        return raw_tps * mode_factor
+            # Dual-channel DDR4-3200 ≈ 50 GB/s; DDR5 systems higher, but be
+            # conservative since offloaded MoE is also compute-bound on CPU.
+            cpu_bw = 55.0
+            frac = min(max(offload_frac, 0.0), 1.0)
+            # If we don't know the fraction (legacy callers pass 0 with
+            # cpu_offload), assume a meaningful spill so we don't overestimate.
+            if frac <= 0.0:
+                frac = 0.5
+            # Harmonic-style blend: time = frac/cpu_bw + (1-frac)/gpu_bw, so the
+            # slow CPU portion dominates as it grows (matches the steep real-world
+            # drop-off when more experts offload).
+            eff_bw = 1.0 / (frac / cpu_bw + (1.0 - frac) / bw)
+            raw_tps = (eff_bw / model_gb) * efficiency
+            return raw_tps * (0.8 if is_moe else 1.0)
+        # Fully on GPU.
+        raw_tps = (bw / model_gb) * efficiency
+        return raw_tps * (0.8 if is_moe else 1.0)
 
     k = FALLBACK_K.get(backend, 70)
     if pb <= 0:
@@ -155,8 +175,15 @@ def _quality_score(model, quant, use_case):
     model_uc = infer_use_case(model)
     if model_uc == "coding" and use_case == "coding":
         base += 6
+    elif model_uc == "coding" and use_case in ("general", "chat"):
+        # Coder-specialized models are still useful generally, but they should
+        # not dominate the default scan. If the user wants code, the Coding
+        # filter gives them the boost above.
+        base -= 10
     if model_uc == "reasoning" and use_case == "reasoning" and pb >= 13:
         base += 5
+    elif model_uc == "reasoning" and use_case == "chat":
+        base -= 4
     if model_uc == "multimodal" and use_case == "multimodal":
         base += 6
 
@@ -242,7 +269,30 @@ def _quant_bits(q):
     return 0
 
 
-def analyze_model(model, system, target_quant=None, scoring_use_case=None):
+def _native_quant(model):
+    native_quant = model.get("quantization", "Q4_K_M")
+    name = (model.get("name") or "").lower()
+    fmt = (model.get("format") or "").lower()
+    text = f"{name} {fmt}"
+    if "nvfp4" in text:
+        return "NVFP4"
+    if re.search(r"(^|[-_/])fp8($|[-_/\s])", text):
+        return "FP8"
+    if "gptq" in text:
+        m = re.search(r"(?:gptq|int|w)(?:[-_]?)(\d{1,2})(?:bit)?", text)
+        return f"GPTQ-{m.group(1)}bit" if m else "GPTQ"
+    if "awq" in text:
+        m = re.search(r"(?:awq|int|w)(?:[-_]?)(\d{1,2})(?:bit)?", text)
+        return f"AWQ-{m.group(1)}bit" if m else "AWQ"
+    if "mlx" in text:
+        m = re.search(r"mlx[-_]?(\d{1,2})bit", text)
+        return f"mlx-{m.group(1)}bit" if m else native_quant
+    if not (model.get("is_gguf") or model.get("gguf_sources")) and re.search(r"(^|[-_/])(?:int)?8bit($|[-_/\s])", text):
+        return "INT8"
+    return native_quant
+
+
+def analyze_model(model, system, target_quant=None, scoring_use_case=None, target_context=None):
     pb = params_b(model)
     if pb <= 0:
         return None
@@ -262,11 +312,14 @@ def analyze_model(model, system, target_quant=None, scoring_use_case=None):
     gpu_only = bool(system.get("gpu_only")) and has_gpu and gpu_vram > 0
     eff_ram = 0 if gpu_only else available_ram
     is_moe = model.get("is_moe", False)
-    ctx = model.get("context_length", 4096) or 4096
+    model_ctx = model.get("context_length", 4096) or 4096
+    try:
+        target_context = int(target_context or 0)
+    except (TypeError, ValueError):
+        target_context = 0
+    ctx = min(model_ctx, target_context) if target_context > 0 else model_ctx
 
-    native_quant = model.get("quantization", "Q4_K_M")
-    if "nvfp4" in (model.get("name") or "").lower():
-        native_quant = "NVFP4"
+    native_quant = _native_quant(model)
     preq = is_prequantized(model)
 
     # GGUF models can't be sharded across GPUs — use single GPU VRAM
@@ -335,7 +388,8 @@ def analyze_model(model, system, target_quant=None, scoring_use_case=None):
             "score": 0,
             "scores": {"quality": 0, "speed": 0, "fit": 0, "context": 0},
             "gguf_sources": model.get("gguf_sources", []),
-            "context_length": model.get("context_length", 4096),
+            "context_length": model_ctx,
+            "target_context": target_context or None,
         }
 
     run_mode, quant, fit_ctx, required_gb = result
@@ -357,7 +411,12 @@ def analyze_model(model, system, target_quant=None, scoring_use_case=None):
     else:
         fit_level = "marginal"
 
-    tps = _estimate_speed(model, quant, run_mode, system)
+    # Fraction of the model that spills to CPU RAM (drives the offload speed
+    # model). When offloading, anything beyond the GPU's VRAM lives in system RAM.
+    offload_frac = 0.0
+    if run_mode == "cpu_offload" and required_gb > 0 and effective_vram > 0:
+        offload_frac = max(0.0, (required_gb - effective_vram) / required_gb)
+    tps = _estimate_speed(model, quant, run_mode, system, offload_frac=offload_frac)
 
     q_score = _quality_score(model, quant, score_use_case)
     s_score = _speed_score(tps, score_use_case)
@@ -388,7 +447,9 @@ def analyze_model(model, system, target_quant=None, scoring_use_case=None):
             "context": round(c_score, 1),
         },
         "gguf_sources": model.get("gguf_sources", []),
-        "context_length": model.get("context_length", 4096),
+        "context_length": model_ctx,
+        "release_date": model.get("release_date", ""),
+        "target_context": target_context or None,
     }
 
 
@@ -398,10 +459,14 @@ SORT_KEYS = {
     "vram": lambda r: r["required_gb"],
     "params": lambda r: r["params_b"],
     "context": lambda r: r["context"],
+    # Newest first. release_date is an ISO-ish string ("2026-05-30"); plain
+    # string sort is chronological. Missing dates sort last (empty < any date,
+    # and we sort reverse=True for newest, so "" lands at the bottom).
+    "newest": lambda r: r.get("release_date") or "",
 }
 
 
-def rank_models(system, use_case=None, limit=50, search=None, sort="score", quant=None):
+def rank_models(system, use_case=None, limit=50, search=None, sort="score", quant=None, target_context=None):
     """Rank all models against detected hardware. Returns sorted list of fit results."""
     models = get_models()
     results = []
@@ -454,10 +519,18 @@ def rank_models(system, use_case=None, limit=50, search=None, sort="score", quan
     apple_silicon = system_backend in ("mps", "metal", "apple")
     rocm = system_backend == "rocm"
 
+    # Consumer AMD Radeon (RDNA, gfx10/11/12): the practical local serving path
+    # is GGUF via llama.cpp. vLLM/SGLang on ROCm are validated for datacenter
+    # Instinct (CDNA, gfx9xx) but are unreliable on consumer RDNA — AWQ kernels
+    # are largely unsupported there and FP8 needs out-of-tree patches. So treat
+    # consumer RDNA like Apple Silicon (GGUF-only) and leave CDNA untouched.
+    # Unknown family (no rocminfo) is left untouched to avoid hiding models from
+    # a possibly-capable Instinct box on a misdetect.
+    gpu_family = (system.get("gpu_family") or "").lower()
+    consumer_amd = system_backend == "rocm" and gpu_family == "rdna"
+
     for m in models:
-        native_q = m.get("quantization", "")
-        if "nvfp4" in (m.get("name") or "").lower():
-            native_q = "NVFP4"
+        native_q = _native_quant(m)
 
         # MLX needs the mlx_lm runtime, which Odysseus does not generate serve
         # commands for. Hide it on every backend, including Metal.
@@ -479,7 +552,12 @@ def rank_models(system, use_case=None, limit=50, search=None, sort="score", quan
         # default GGUF quant) and vLLM-only AWQ/GPTQ/FP8 builds alike. Without
         # this the Cookbook recommends models the Mac can't run; on CUDA these
         # stay visible because vLLM serves safetensors directly.
-        if apple_silicon and not (m.get("is_gguf") or m.get("gguf_sources")):
+        #
+        # Consumer AMD (RDNA) is the same story: GGUF via llama.cpp is the
+        # servable path, so a model needs a real GGUF to be recommended.
+        # Otherwise the Cookbook rates vLLM-only AWQ/GPTQ builds "GOOD" on a
+        # Radeon that can't actually serve them.
+        if (apple_silicon or consumer_amd) and not (m.get("is_gguf") or m.get("gguf_sources")):
             continue
 
         # Format filter: AWQ tab -> only AWQ models, FP4 tab -> FP4-family models, etc.
@@ -503,7 +581,7 @@ def rank_models(system, use_case=None, limit=50, search=None, sort="score", quan
             if search.lower() not in name and search.lower() not in provider:
                 continue
 
-        result = analyze_model(m, system, target_quant=quant, scoring_use_case=(use_case or "general"))
+        result = analyze_model(m, system, target_quant=quant, scoring_use_case=(use_case or "general"), target_context=target_context)
         if result is None:
             continue
 

services/hwfit/hardware.py

@@ -1,5 +1,6 @@
 import os
 import platform
+import re
 import shutil
 import subprocess
 import time
@@ -130,6 +131,33 @@ def _detect_nvidia():
     }
 
 
+def classify_amd_gfx(gfx):
+    """Map an AMD ISA target (e.g. "gfx1200") to (gfx, family).
+
+    family is one of:
+      "rdna"    — consumer Radeon RX (gfx10xx RDNA1/2, gfx11xx RDNA3, gfx12xx RDNA4)
+      "cdna"    — datacenter Instinct (gfx908 MI100, gfx90a MI200, gfx94x/95x MI300+)
+      "gcn"     — older GCN/Vega (gfx900/906)
+      "unknown" — empty/unrecognized; callers must treat conservatively
+
+    This drives the serving decision: vLLM/SGLang on ROCm are validated on CDNA
+    but fragile on consumer RDNA (AWQ kernels largely unsupported, FP8 needs
+    out-of-tree patches), so RDNA is steered to GGUF/llama.cpp.
+    """
+    gfx = (gfx or "").lower().strip()
+    m = re.fullmatch(r"gfx(\d+[a-f]?)", gfx)
+    if not m:
+        return "", "unknown"
+    digits = m.group(1)
+    if digits[:2] in ("10", "11", "12"):
+        return gfx, "rdna"
+    if digits in ("908", "90a") or digits[:2] in ("94", "95"):
+        return gfx, "cdna"
+    if digits[:1] == "9":
+        return gfx, "gcn"
+    return gfx, "unknown"
+
+
 def _detect_amd():
     """Detect AMD GPUs. Handles both discrete cards (with mem_info_vram_total)
     and APUs / unified-memory SoCs like Strix Halo (which expose
@@ -155,6 +183,17 @@ def _detect_amd():
         except Exception:
             return []
 
+    def _amd_arch():
+        """Best-effort AMD GPU ISA + family from rocminfo.
+
+        rocminfo is the source of truth; its GPU agents report a `Name: gfxNNNN`
+        line (CPU agents report a brand string, not a gfx target), so the first
+        gfx match is the GPU ISA. Returns (gfx, family) — see classify_amd_gfx.
+        """
+        info = _run(["rocminfo"]) or _run(["/opt/rocm/bin/rocminfo"]) or ""
+        m = re.search(r"gfx\d+[a-f]?", info)
+        return classify_amd_gfx(m.group(0) if m else "")
+
     try:
         cards = []
         is_apu = False
@@ -187,6 +226,7 @@ def _detect_amd():
             return None
         total_vram = sum(c["vram_gb"] for c in cards)
         groups = _group_gpus(cards)
+        gfx, family = _amd_arch()
         # NOTE: for APUs with BIOS UMA carveout (e.g. Strix Halo), vis_vram_total
         # is the real usable GPU memory — it's physically backed but reserved
         # by BIOS so it doesn't appear in /proc/meminfo. Don't cap it at system
@@ -200,6 +240,13 @@ def _detect_amd():
             "homogeneous": len(groups) <= 1,
             "backend": "rocm",
             "unified_memory": is_apu,
+            # AMD ISA/family so downstream can tell datacenter Instinct (CDNA,
+            # where vLLM/SGLang run AWQ/GPTQ reliably) from consumer Radeon
+            # (RDNA, where the practical path is GGUF via llama.cpp). Empty/
+            # "unknown" when rocminfo isn't available — callers must treat
+            # unknown conservatively, not assume vLLM works.
+            "gpu_arch": gfx,
+            "gpu_family": family,
         }
     except Exception:
         return None
@@ -409,7 +456,7 @@ def _detect_windows():
         "    $gpus = @(); "
         "    foreach ($line in $nv -split \"`n\") { "
         "      $p = $line -split ','; "
-        "      if ($p.Count -ge 2) { $gpus += @{name=$p[1].Trim(); vram_mb=[double]$p[0].Trim()} } "
+        "      if ($p.Count -ge 2) { $gpus += [pscustomobject]@{name=$p[1].Trim(); vram_mb=[double]$p[0].Trim()} } "
         "    }; "
         "    $r.gpu_name = $gpus[0].name; "
         "    $r.gpu_vram_gb = [math]::Round(($gpus | Measure-Object -Property vram_mb -Sum).Sum / 1024, 1); "

services/hwfit/models.py

@@ -101,7 +101,16 @@ def _normalize_model_entry(model):
 
 def is_prequantized(model):
     q = model.get("quantization", "")
-    return any(q.startswith(p) for p in PREQUANTIZED_PREFIXES)
+    name = (model.get("name") or "").lower()
+    fmt = (model.get("format") or "").lower()
+    text = f"{name} {fmt}"
+    return (
+        "nvfp4" in text
+        or re.search(r"(^|[-_/])fp8($|[-_/\s])", text) is not None
+        or (not (model.get("is_gguf") or model.get("gguf_sources")) and re.search(r"(^|[-_/])(?:int)?8bit($|[-_/\s])", text) is not None)
+        or any(x in text for x in ("awq", "gptq", "mlx"))
+        or any(q.startswith(p) for p in PREQUANTIZED_PREFIXES)
+    )
 
 
 def params_b(model):

services/memory/memory_extractor.py

@@ -235,6 +235,10 @@ async def extract_and_store(
     Designed to run as a background task (asyncio.create_task).
     Errors are logged, never raised.
     """
+    if not endpoint_url or not model:
+        logger.debug("[memory-extract] No model or URL provided, skipping")
+        return
+
     try:
         from src.llm_core import llm_call_async
 
@@ -245,11 +249,30 @@ async def extract_and_store(
         if len(recent) < 2:
             return  # Need at least a user message and assistant response
 
-        fallback_facts = _fallback_memory_candidates(recent)
+        # Strip media (images/audio) from messages — background memory extraction
+        # only needs the text. The VL-generated descriptions are already in the
+        # text content of the messages. This avoids sending image tokens to
+        # non-vision models and prevents accidental "vision grounding" triggers.
+        stripped_recent = []
+        for msg in recent:
+            role = msg.get("role")
+            content = msg.get("content", "")
+            if isinstance(content, list):
+                # Filter out multimodal blocks that aren't text
+                text_only = [b for b in content if isinstance(b, dict) and b.get("type") == "text"]
+                if not text_only and content:
+                    continue
+                content = text_only
+            stripped_recent.append({"role": role, "content": content})
+
+        if not stripped_recent:
+            return
+
+        fallback_facts = _fallback_memory_candidates(stripped_recent)
 
         extraction_messages = [
             {"role": "system", "content": EXTRACT_SYSTEM_PROMPT},
-        ] + recent
+        ] + stripped_recent
 
         facts = []
         try:

services/memory/skill_extractor.py

@@ -59,6 +59,10 @@ async def maybe_extract_skill(
     owner: Optional[str] = None,
 ):
     """Extract a skill if the agent run was complex enough."""
+    if not model:
+        logger.debug("[skill-extract] No model provided, skipping")
+        return None
+
     # Quiet by default; flip to DEBUG when chasing extractor issues.
     logger.debug(
         "[skill-extract] start: rounds=%d tools=%d model=%s owner=%s",
@@ -78,9 +82,23 @@ async def maybe_extract_skill(
             logger.debug("[skill-extract] no recent messages, skipping")
             return None
 
+        # Strip media (images/audio) from messages
+        stripped_recent = []
+        for msg in recent:
+            content = msg.get("content", "")
+            if isinstance(content, list):
+                text_only = [b for b in content if isinstance(b, dict) and b.get("type") == "text"]
+                if not text_only and content:
+                    continue
+                content = text_only
+            stripped_recent.append({"role": msg.get("role"), "content": content})
+
+        if not stripped_recent:
+            return None
+
         # Build conversation summary for extraction
         conv_lines = []
-        for msg in recent:
+        for msg in stripped_recent:
             role = msg.get("role", "?")
             content = msg.get("content", "")
             if isinstance(content, list):

services/search/content.py

@@ -1,5 +1,6 @@
 """Webpage content fetching with caching, PDF extraction, and summarization helpers."""
 
+import copy
 import io
 import ipaddress
 import json
@@ -115,6 +116,28 @@ def _extract_meta(soup: BeautifulSoup) -> dict:
     return {"description": description, "keywords": keywords}
 
 
+def _extract_og_image(soup: BeautifulSoup) -> str:
+    """Extract the best representative image URL from meta tags.
+
+    Only returns absolute http(s) URLs -- skips relative paths and data URIs.
+    """
+    candidates = []
+    for prop in ("og:image", "og:image:url", "og:image:secure_url"):
+        tag = soup.find("meta", attrs={"property": prop})
+        if tag and tag.get("content", "").strip():
+            candidates.append(tag["content"].strip())
+    tag = soup.find("meta", attrs={"name": "twitter:image"})
+    if tag and tag.get("content", "").strip():
+        candidates.append(tag["content"].strip())
+    tag = soup.find("meta", attrs={"name": "thumbnail"})
+    if tag and tag.get("content", "").strip():
+        candidates.append(tag["content"].strip())
+    for url in candidates:
+        if url.startswith(("https://", "http://")) and not url.endswith((".svg", ".ico")):
+            return url
+    return ""
+
+
 def _extract_lists(soup: BeautifulSoup) -> List[List[str]]:
     """Return a list of lists, each inner list representing a <ul>/<ol>."""
     all_lists = []
@@ -275,10 +298,12 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
     title_tag = soup.find("title")
     title_text = title_tag.get_text(strip=True) if title_tag else ""
     meta_info = _extract_meta(soup)
+    og_image = _extract_og_image(soup)
     js_rendered = _detect_js_frameworks(soup)
     js_message = "Page appears to be rendered by a JavaScript framework; content may be incomplete." if js_rendered else ""
 
-    # Main textual content (heuristic)
+    # Main textual content (heuristic): prefer semantic / "content"-classed
+    # containers to skip nav/footer/boilerplate; tuned for article pages.
     main_content = ""
     content_areas = soup.find_all(
         ["main", "article", "section", "div"],
@@ -287,12 +312,23 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
     if content_areas:
         for area in content_areas[:3]:
             main_content += area.get_text(separator=" ", strip=True) + " "
-    if not main_content:
+    main_content = re.sub(r"\s+", " ", main_content).strip()
+
+    # If the heuristic finds only a tiny wrapper, fall back to body text with
+    # obvious boilerplate stripped so UI/deep-research search results do not
+    # look empty for app/landing pages.
+    THIN_CONTENT_CHARS = 600
+    if len(main_content) < THIN_CONTENT_CHARS:
         body = soup.find("body")
         if body:
-            main_content = body.get_text(separator=" ", strip=True)
-
-    main_content = re.sub(r"\s+", " ", main_content).strip()[:8000]
+            body_copy = copy.copy(body)
+            for noise in body_copy.find_all(
+                ["script", "style", "noscript", "template", "nav", "header", "footer", "aside"]
+            ):
+                noise.extract()
+            body_text = re.sub(r"\s+", " ", body_copy.get_text(separator=" ", strip=True)).strip()
+            if len(body_text) > len(main_content):
+                main_content = body_text
 
     result = {
         "url": url,
@@ -303,6 +339,7 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
         "code_blocks": _extract_code_blocks(soup),
         "meta_description": meta_info.get("description", ""),
         "meta_keywords": meta_info.get("keywords", ""),
+        "og_image": og_image,
         "js_rendered": js_rendered,
         "js_message": js_message,
         "success": True,
@@ -348,13 +385,18 @@ def get_tldr(text: str, max_sentences: int = 3) -> str:
 
 def extract_quotes(text: str) -> List[str]:
     """Return quoted excerpts that are at least 15 characters long."""
-    return [m.group(1).strip() for m in re.finditer(r'["\']([^"\']{15,}?)["\']', text)]
+    # Backreference the opening quote so the closing quote must match it —
+    # otherwise `"text'` (open double, close single) is treated as a quote.
+    return [m.group(2).strip() for m in re.finditer(r'(["\'])([^"\']{15,}?)\1', text)]
 
 
 def extract_statistics(text: str) -> List[str]:
     """Find numbers, percentages, dates and simple measurements."""
+    # Match a comma-grouped number (1,000,000) OR a plain digit run (50000) —
+    # the old `\d{1,3}(?:,\d{3})*` matched only the first 3 digits of a
+    # comma-less number, and the trailing `\b` dropped a closing `%`.
     pattern = re.compile(
-        r"\b\d{1,3}(?:,\d{3})*(?:\.\d+)?\s*(%|percent|‰|per cent|[a-zA-Z]+)?\b",
+        r"\b(?:\d{1,3}(?:,\d{3})+|\d+)(?:\.\d+)?\s*(%|percent|‰|per cent|[a-zA-Z]+)?",
         re.IGNORECASE,
     )
     return [m.group(0).strip() for m in pattern.finditer(text)]

services/search/core.py

@@ -30,6 +30,7 @@ from .providers import (
     tavily_search,
     serper_search,
     _get_search_settings,
+    _get_provider_key,
     _get_result_count,
 )
 from .content import (
@@ -54,7 +55,7 @@ def get_search_config() -> Dict[str, Any]:
     settings = _get_search_settings()
     provider = settings.get("search_provider", "searxng")
     config["active_provider"] = provider
-    config["has_api_key"] = bool((settings.get("search_api_key") or "").strip())
+    config["has_api_key"] = bool(_get_provider_key(provider))
     config["result_count"] = _get_result_count()
     if provider == "searxng":
         from .providers import _get_search_instance

services/search/providers.py

@@ -64,7 +64,17 @@ def _get_provider_key(provider: str) -> str:
         if val:
             return val
     # Legacy fallback: old shared search_api_key field
-    return (settings.get("search_api_key") or "").strip()
+    legacy = (settings.get("search_api_key") or "").strip()
+    if legacy:
+        return legacy
+    env_map = {
+        "brave": "DATA_BRAVE_API_KEY",
+        "google_pse": "GOOGLE_API_KEY",
+        "tavily": "TAVILY_API_KEY",
+        "serper": "SERPER_API_KEY",
+    }
+    env_name = env_map.get(provider, "")
+    return (os.environ.get(env_name) or "").strip() if env_name else ""
 
 
 def _get_result_count() -> int:
@@ -76,6 +86,43 @@ def _get_result_count() -> int:
         return 5
 
 
+# Canonical SafeSearch levels: "strict" (default), "moderate", "off".
+# Each provider has its own knob name and value space -- see _safesearch_for(...).
+_SAFESEARCH_LEVELS = ("strict", "moderate", "off")
+
+
+def _get_safesearch_level() -> str:
+    """Return configured SafeSearch level normalized to a canonical value."""
+    settings = _get_search_settings()
+    raw = (settings.get("search_safesearch") or "strict").strip().lower()
+    if raw in _SAFESEARCH_LEVELS:
+        return raw
+    aliases = {
+        "on": "strict", "high": "strict", "2": "strict",
+        "medium": "moderate", "1": "moderate", "default": "moderate",
+        "none": "off", "disabled": "off", "0": "off",
+    }
+    return aliases.get(raw, "strict")
+
+
+def _safesearch_for(provider: str) -> Optional[str]:
+    """Translate the canonical SafeSearch level into provider-specific values."""
+    level = _get_safesearch_level()
+    if provider == "searxng":
+        return {"strict": "2", "moderate": "1", "off": "0"}[level]
+    if provider == "brave":
+        return level
+    if provider == "duckduckgo_lib":
+        return {"strict": "on", "moderate": "moderate", "off": "off"}[level]
+    if provider == "duckduckgo_html":
+        return {"strict": "1", "moderate": "-1", "off": "-2"}[level]
+    if provider == "google_pse":
+        return None if level == "off" else "active"
+    if provider == "serper":
+        return None if level == "off" else "active"
+    return None
+
+
 # ── SearXNG ──
 
 _NEWS_HINTS = ("news", "nyheter", "headlines", "breaking", "latest", "today", "idag")
@@ -105,7 +152,12 @@ def searxng_search_api(query: str, count: int = 10, categories: str = "general",
     # languages and brand-ambiguous terms bleed in foreign SEO pages (e.g.
     # "Odyssey" → Honda Japan, "Trojan" → Japanese malware blogs, "Polyphemus"
     # → Chinese math forums). The news path already did this; general didn't.
-    params = {"q": query, "format": "json", "language": "en"}
+    params = {
+        "q": query,
+        "format": "json",
+        "language": "en",
+        "safesearch": _safesearch_for("searxng"),
+    }
     q_lc = query.lower()
     is_news = time_filter is not None or any(h in q_lc for h in _NEWS_HINTS)
     if is_news and categories == "general":
@@ -154,6 +206,7 @@ def searxng_search_api(query: str, count: int = 10, categories: str = "general",
                 "format": "json",
                 "language": "en",
                 "categories": "general",
+                "safesearch": _safesearch_for("searxng"),
             }
             if _GENERAL_ENGINES:
                 fallback["engines"] = _GENERAL_ENGINES
@@ -204,7 +257,7 @@ def searxng_search(query, max_results=10):
     try:
         response = httpx.get(
             f"{instance}/search",
-            params={"q": query},
+            params={"q": query, "safesearch": _safesearch_for("searxng")},
             headers=req_headers,
             timeout=10,
         )
@@ -249,7 +302,11 @@ def _brave_search_impl(query: str, count: int, time_filter: Optional[str] = None
         return []
 
     headers = {"X-Subscription-Token": brave_api_key, "Accept": "application/json"}
-    params = {"q": enhanced_query, "count": count}
+    params = {
+        "q": enhanced_query,
+        "count": count,
+        "safesearch": _safesearch_for("brave"),
+    }
     if time_filter:
         time_map = {"day": "day", "week": "week", "month": "month", "year": "year"}
         if time_filter in time_map:
@@ -298,32 +355,40 @@ def _brave_search_impl(query: str, count: int, time_filter: Optional[str] = None
 
 # ── DuckDuckGo (free, no key) ──
 
+def _is_duckduckgo_host(host: str) -> bool:
+    """True only for duckduckgo.com and its subdomains."""
+    host = (host or "").lower()
+    return host == "duckduckgo.com" or host.endswith(".duckduckgo.com")
+
+
+def _resolve_ddg_redirect(raw: str) -> str:
+    """Resolve a DuckDuckGo /l/?uddg= redirect URL to its destination."""
+    if not raw:
+        return raw
+    resolved = raw
+    if resolved.startswith("//"):
+        resolved = "https:" + resolved
+    elif resolved.startswith("/"):
+        resolved = urljoin("https://html.duckduckgo.com", resolved)
+    try:
+        parsed = urlparse(resolved)
+        if _is_duckduckgo_host(parsed.hostname) and parsed.path.rstrip("/") == "/l":
+            qs = parse_qs(parsed.query)
+            if "uddg" in qs:
+                return qs["uddg"][0]
+    except Exception:
+        pass
+    return resolved
+
+
 def duckduckgo_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
     """Search using DuckDuckGo via the duckduckgo-search library. No API key needed."""
-    def _resolve_url(raw: str) -> str:
-        """Resolve DuckDuckGo redirect URL to the actual destination URL."""
-        if not raw:
-            return raw
-        resolved = raw
-        if resolved.startswith("//"):
-            resolved = "https:" + resolved
-        elif resolved.startswith("/"):
-            resolved = urljoin("https://html.duckduckgo.com", resolved)
-        try:
-            parsed = urlparse(resolved)
-            if "duckduckgo.com" in (parsed.hostname or "") and parsed.path.rstrip("/") == "/l":
-                qs = parse_qs(parsed.query)
-                if "uddg" in qs:
-                    return qs["uddg"][0]
-        except Exception:
-            pass
-        return resolved
 
     def _html_fallback() -> List[dict]:
         try:
             response = httpx.get(
                 "https://html.duckduckgo.com/html/",
-                params={"q": query},
+                params={"q": query, "kp": _safesearch_for("duckduckgo_html")},
                 headers={"User-Agent": "Mozilla/5.0"},
                 timeout=REQUEST_TIMEOUT,
             )
@@ -334,7 +399,7 @@ def duckduckgo_search(query: str, count: int = 10, time_filter: Optional[str] =
                 link = result.select_one(".result__a")
                 if not link:
                     continue
-                url = _resolve_url(link.get("href", ""))
+                url = _resolve_ddg_redirect(link.get("href", ""))
                 if not url:
                     continue
                 snippet_el = result.select_one(".result__snippet")
@@ -362,7 +427,12 @@ def duckduckgo_search(query: str, count: int = 10, time_filter: Optional[str] =
 
     try:
         ddgs = DDGS()
-        raw = ddgs.text(query, max_results=count, timelimit=timelimit)
+        raw = ddgs.text(
+            query,
+            max_results=count,
+            timelimit=timelimit,
+            safesearch=_safesearch_for("duckduckgo_lib"),
+        )
         results = []
         for item in raw:
             url = item.get("href", "")
@@ -404,6 +474,9 @@ def google_pse_search(query: str, count: int = 10, time_filter: Optional[str] =
         "q": query,
         "num": min(count, 10),  # Google PSE max is 10 per request
     }
+    safe = _safesearch_for("google_pse")
+    if safe:
+        params["safe"] = safe
     if time_filter:
         # dateRestrict: d[number], w[number], m[number], y[number]
         time_map = {"day": "d1", "week": "w1", "month": "m1", "year": "y1"}
@@ -508,6 +581,9 @@ def serper_search(query: str, count: int = 10, time_filter: Optional[str] = None
         "q": query,
         "num": count,
     }
+    safe = _safesearch_for("serper")
+    if safe:
+        payload["safe"] = safe
     if time_filter:
         time_map = {"day": "qdr:d", "week": "qdr:w", "month": "qdr:m", "year": "qdr:y"}
         if time_filter in time_map:

services/search/query.py

@@ -15,7 +15,10 @@ def _detect_question_type(query: str) -> Optional[str]:
     """Return the leading question word if present (who, what, when, where, why, how)."""
     q = query.strip().lower()
     for word in ("who", "what", "when", "where", "why", "how"):
-        if q.startswith(word):
+        # Require a whole-word match: a bare prefix mis-flags ordinary queries
+        # like "whatsapp pricing" (-> what) or "however ..." (-> how), which
+        # then get spurious boost terms OR-appended in enhance_query.
+        if q == word or q.startswith(word + " "):
             return word
     return None
 

services/stt/stt_service.py

@@ -91,6 +91,7 @@ class STTService:
         model = self._get_whisper()
         if not model:
             return None
+        tmp_path = None
         try:
             # Write to temp file (faster-whisper needs a file path or file-like)
             with tempfile.NamedTemporaryFile(suffix=".webm", delete=False) as tmp:
@@ -104,14 +105,14 @@ class STTService:
             segments, info = model.transcribe(tmp_path, **kwargs)
             text = " ".join(seg.text.strip() for seg in segments)
 
-            # Cleanup
-            Path(tmp_path).unlink(missing_ok=True)
-
             logger.info(f"Local STT: {len(text)} chars, lang={info.language}, prob={info.language_probability:.2f}")
             return text
         except Exception as e:
             logger.error(f"Local STT transcription failed: {e}", exc_info=True)
             return None
+        finally:
+            if tmp_path:
+                Path(tmp_path).unlink(missing_ok=True)
 
     # ── API endpoint ──
 

services/tts/tts_service.py

@@ -188,7 +188,7 @@ class TTSService:
         provider = settings["tts_provider"]
         tts_enabled = settings.get("tts_enabled", True)
 
-        cache_files = list(self.cache_dir.glob("*.wav"))
+        cache_files = list(self.cache_dir.glob("*.wav")) + list(self.cache_dir.glob("*.mp3"))
         cache_size = sum(f.stat().st_size for f in cache_files)
 
         is_available = self.available and tts_enabled

src/agent_loop.py

@@ -115,6 +115,7 @@ _API_AGENT_RULES = """\
 - Keep answers concise unless the user asks for depth.
 - For long code or content, use document tools instead of pasting large blocks into chat.
 - Editing an existing document: ALWAYS use `edit_document` with find/replace. Only use `update_document` for genuine full rewrites (>50% changed) — do NOT echo the entire file back for small edits.
+- If the active editor document is an email draft/compose window, treat that open email as the target for "write this", "write the email", "reply with...", "make it say...", "draft this", and similar requests. Do NOT create another document, search/list/manage documents, or open a different reply unless the user explicitly asks. Edit the open email draft with `edit_document` or `update_document`; preserve To/Cc/Bcc/Subject/In-Reply-To/References/X-* header lines unless the user asks to change them.
 - "Give suggestions / feedback / review / how can I improve this / what would make it better" about the OPEN document → call `suggest_document`, do NOT write a prose list of ideas in chat. It creates inline accept/reject bubbles on the doc. Give concrete `find`/`replace`/`reason` items. To suggest an ADDITION (e.g. "add a bow to the SVG", a new section), set `find` to a short existing anchor snippet and `replace` to that same snippet PLUS the new content. Only answer in prose when no document is open, or the request is purely conceptual with no concrete change to propose.
 - BIAS TOWARD ACTION on edit requests. If the user says "edit out X", "remove the Y paragraph", "change Z" — call the edit tool with your best interpretation. Don't ask for clarification on minor ambiguity. The user can undo.
 - AFTER A TOOL SUCCEEDS, do not second-guess. A success response means it worked. Reply in ONE short sentence confirming what was done. No verification thinking, no re-analyzing — move on.
@@ -456,7 +457,7 @@ _API_HOSTS = frozenset([
     "api.deepseek.com", "deepseek.com",
     "api.together.xyz", "api.fireworks.ai",
     "api.perplexity.ai", "api.x.ai",
-    "ollama.com",
+    "ollama.com", "api.venice.ai",
     # Local OpenAI-compatible endpoints (llama.cpp, vLLM, LM Studio, etc.).
     # Without these, `_is_api_model` falls back to keyword sniffing on the
     # model name, so well-behaved local servers don't get native tool
@@ -642,6 +643,7 @@ def _build_system_prompt(
                 f'ACTIVE EMAIL DRAFT (open in editor — the user is looking at this right now)\n'
                 f'Title: "{active_document.title}"\n'
                 f'```\n{_doc_raw}\n```\n\n'
+                f'This is the current email compose window, not a normal document library item. If the user says "write", "draft", "reply", "make it say", or "write the email" without naming another target, edit THIS email draft.\n\n'
                 f'When the user asks you to write, reply to, or improve this email:\n'
                 f'1. Use `update_document` to replace the ENTIRE content — keep all the header lines (To, Subject, In-Reply-To, References, X-Source-UID, X-Source-Folder, X-Attachments) and the `---` separator EXACTLY as they are.\n'
                 f'2. Replace ONLY the body text (the part after `---`). If there is a quoted original email (lines starting with `>`), keep that quoted block unchanged BELOW your new reply.\n'
@@ -792,7 +794,7 @@ def _build_system_prompt(
     # When creating email documents, instruct the AI on the format
     if relevant_tools and (_EMAIL_TOOL_HINTS & set(relevant_tools)):
         agent_prompt += (
-            '\n\n📧 EMAIL DOCUMENT FORMAT: When drafting email replies, use create_document with language="email". '
+            '\n\n📧 EMAIL DOCUMENT FORMAT: If no email draft is already open and you need to create an email draft, use create_document with language="email". '
             'The content format is:\n'
             'To: recipient@example.com\n'
             'Subject: Re: Original subject\n'
@@ -800,8 +802,8 @@ def _build_system_prompt(
             'References: <original-message-id>\n'
             '---\n'
             'Body text here...\n\n'
-            'The user can then edit and click Send or Draft in the editor. For an already-open email draft, '
-            'edit the current document instead of creating another one.'
+            'The user can then edit and click Send or Draft in the editor. If an email draft is already open, '
+            'that open draft is the target: use update_document/edit_document on it instead of creating another document.'
         )
 
     # Inject relevant skills based on the user's last message. The
@@ -1176,6 +1178,8 @@ def _compute_final_metrics(
     model: str = "",
     last_round_input_tokens: int = 0,
     prep_timings: Optional[Dict[str, float]] = None,
+    backend_gen_tps: float = 0,
+    backend_prefill_tps: float = 0,
 ) -> dict:
     """Compute token counts, TPS, and build the final metrics dict."""
     if has_real_usage:
@@ -1188,7 +1192,15 @@ def _compute_final_metrics(
                 input_content += msg["content"] + "\n"
         input_tokens = len(input_content) // 4
         output_tokens = len(full_response) // 4
-    tps = output_tokens / total_duration if total_duration > 0 else 0
+    # Prefer the backend's true generation speed (llama.cpp
+    # timings.predicted_per_second) — pure decode, no prefill/tool/network time.
+    # Fall back to tokens/wall-clock only when the backend didn't report it
+    # (e.g. cloud APIs without timings); that figure reads low because
+    # total_duration includes prefill + agent overhead.
+    if backend_gen_tps and backend_gen_tps > 0:
+        tps = backend_gen_tps
+    else:
+        tps = output_tokens / total_duration if total_duration > 0 else 0
     # Use last round's input tokens for context % (peak usage) when available
     ctx_tokens = last_round_input_tokens if last_round_input_tokens > 0 else input_tokens
     ctx_pct = min(round((ctx_tokens / context_length) * 100, 1), 100.0) if context_length else 0
@@ -1199,12 +1211,17 @@ def _compute_final_metrics(
         "input_tokens": input_tokens,
         "output_tokens": output_tokens,
         "tokens_per_second": round(tps, 2),
+        # True decode speed when the backend reported it; "computed" = the
+        # tokens/wall-clock fallback (reads low — includes prefill/overhead).
+        "tps_source": "backend" if (backend_gen_tps and backend_gen_tps > 0) else "computed",
         "total_tokens": input_tokens + output_tokens,
         "context_length": context_length,
         "context_percent": ctx_pct,
         "usage_source": "real" if has_real_usage else "estimated",
         "model": model,
     }
+    if backend_prefill_tps and backend_prefill_tps > 0:
+        metrics["prefill_tps"] = round(backend_prefill_tps, 2)
     if prep_timings:
         prep_total = round(sum(prep_timings.values()), 3)
         metrics["agent_prep_time"] = prep_total
@@ -1433,7 +1450,7 @@ async def stream_agent_loop(
     except Exception as _e:
         logger.debug(f"endpoint supports_tools lookup failed: {_e}")
     _model_supports_tools = any(kw in _model_lc for kw in (
-        "deepseek", "gpt-4", "gpt-5", "gpt-o", "claude", "gemini", "gemma",
+        "gpt-4", "gpt-5", "gpt-o", "claude", "gemini", "gemma",
         "qwen3", "qwen2.5", "mixtral", "mistral", "llama-3.1", "llama-3.2",
         "llama-3.3", "llama-4",
         # Local-served models that follow OpenAI-style function calling
@@ -1441,10 +1458,20 @@ async def stream_agent_loop(
         # with the per-endpoint flag above.
         "minimax", "kimi", "yi-", "phi-3", "phi-4", "command-r",
         "glm-4", "internlm", "hermes",
+        # deepseek-v2/v3/chat support tools via the cloud API; deepseek-r1
+        # (reasoning model) does not — handled by the blocklist below.
+        "deepseek-v", "deepseek-chat",
+    ))
+    # Models known to reject tool schemas at the Ollama/local level even when
+    # the endpoint URL would otherwise enable native function calling.
+    # The per-endpoint supports_tools flag (True/False) always takes priority
+    # and can override this list for users who know their setup.
+    _model_no_tools = any(kw in _model_lc for kw in (
+        "deepseek-r1",
     ))
     if _endpoint_supports is True:
         _is_api_model = True
-    elif _endpoint_supports is False:
+    elif _endpoint_supports is False or _model_no_tools:
         _is_api_model = False
     else:
         _is_api_model = any(h in endpoint_url for h in _API_HOSTS) or _model_supports_tools
@@ -1460,12 +1487,21 @@ async def stream_agent_loop(
     _t3 = time.time()
     try:
         from src.context_compactor import trim_for_context
+        from src.context_budget import compute_input_token_budget
+        from src.settings import is_setting_overridden
 
         soft_budget = int(get_setting("agent_input_token_budget", 6000) or 0)
         if soft_budget > 0:
             before_trim_tokens = estimate_tokens(messages)
             reserve_tokens = min(max(max_tokens or 1024, 512), 2048)
-            effective_budget = min(context_length or soft_budget, soft_budget)
+            # Scale the default budget to the model's context window so long-context
+            # models aren't silently capped at 6000; an explicit user setting is
+            # still honoured (clamped to the window). (#1170)
+            effective_budget = compute_input_token_budget(
+                soft_budget,
+                context_length,
+                is_setting_overridden("agent_input_token_budget"),
+            )
             trimmed_messages = trim_for_context(
                 messages,
                 effective_budget,
@@ -1506,6 +1542,8 @@ async def stream_agent_loop(
     real_output_tokens = 0
     last_round_input_tokens = 0  # Last round's input tokens (for context % peak)
     has_real_usage = False
+    backend_gen_tps = 0      # backend-reported true gen speed (llama.cpp timings)
+    backend_prefill_tps = 0  # backend-reported prefill speed
     total_tool_calls = 0  # for budget enforcement
 
     # Loop-breaker state. Small models (e.g. deepseek-v4-flash) can get
@@ -1655,6 +1693,14 @@ async def stream_agent_loop(
                         real_output_tokens += u.get("output_tokens", 0)
                         last_round_input_tokens = round_input
                         has_real_usage = True
+                        # Backend-reported TRUE generation speed (llama.cpp
+                        # timings.predicted_per_second) — pure decode, excludes
+                        # prefill/network. Preferred over tokens/wall-clock, which
+                        # reads low. Keep the last round's value (the gen phase).
+                        if u.get("gen_tps"):
+                            backend_gen_tps = u["gen_tps"]
+                        if u.get("prefill_tps"):
+                            backend_prefill_tps = u["prefill_tps"]
                     elif data.get("type") == "fallback":
                         # The selected model failed and another answered; surface
                         # the notice so a misconfigured provider isn't masked.
@@ -2181,6 +2227,8 @@ async def stream_agent_loop(
         has_real_usage, tool_events, round_texts, model=model,
         last_round_input_tokens=last_round_input_tokens,
         prep_timings=prep_timings,
+        backend_gen_tps=backend_gen_tps,
+        backend_prefill_tps=backend_prefill_tps,
     )
     yield f"data: {json.dumps({'type': 'metrics', 'data': metrics})}\n\n"
 

src/api_key_manager.py

@@ -1,7 +1,10 @@
 import os
 import json
+import logging
 from typing import Dict
-from cryptography.fernet import Fernet
+from cryptography.fernet import Fernet, InvalidToken
+
+logger = logging.getLogger(__name__)
 
 class APIKeyManager:
     def __init__(self, data_dir: str):
@@ -47,8 +50,12 @@ class APIKeyManager:
             return {}
         with open(self.api_keys_file, 'r', encoding="utf-8") as f:
             encrypted_keys = json.load(f)
-        return {
-            provider: self.decrypt_api_key(key)
-            for provider, key in encrypted_keys.items()
-        }
+
+        decrypted = {}
+        for provider, key in encrypted_keys.items():
+            try:
+                decrypted[provider] = self.decrypt_api_key(key)
+            except (InvalidToken, ValueError) as e:
+                logger.warning("Failed to decrypt API key for %s: %s", provider, e)
+        return decrypted
 

src/chat_handler.py

@@ -14,7 +14,7 @@ from src.constants import (
     UPLOAD_DIR,
 )
 from core.models import ChatMessage
-from src.chat_helpers import extract_urls, is_vision_model
+from src.chat_helpers import extract_urls, model_supports_vision
 from src.document_processor import build_user_content, analyze_image_with_vl_result
 from src.youtube_handler import (
     is_youtube_url,
@@ -146,7 +146,9 @@ class ChatHandler:
         # Analyze images — skip if vision disabled, or if main model is vision-capable
         from src.settings import get_setting
         vision_enabled = get_setting("vision_enabled", True)
-        main_is_vision = is_vision_model(sess.model or "")
+        main_is_vision = await asyncio.to_thread(
+            model_supports_vision, sess.model or "", getattr(sess, "endpoint_url", "") or ""
+        )
 
         # Resolve uploads once with the session owner. Attachment IDs are
         # bearer-like references; never trust them without an owner check.

src/chat_helpers.py

@@ -4,10 +4,14 @@
 import re
 import os
 import json
+import time
+import ipaddress
 import logging
+import httpx
+from urllib.parse import urlparse
 from fastapi import HTTPException
 from fastapi import UploadFile
-from typing import List
+from typing import List, Optional
 
 logger = logging.getLogger(__name__)
 
@@ -34,6 +38,8 @@ _VISION_MODEL_KEYWORDS = (
     # open / local
     "vision", "llava", "bakllava", "moondream", "pixtral", "minicpm",
     "internvl", "cogvlm", "qwen-vl", "qwen2-vl", "qwen3-vl", "qwen3vl",
+    # zhipu / glm (glm-4.5v, glm-4.6v, glm-5v-turbo, etc.)
+    "glm-4.5v", "glm-4.6v", "glm-5v",
 )
 # Catches the "*-VL-*" / "*VL*" family not covered by a literal keyword above
 # (e.g. Qwen2.5-VL and various tags): a standalone "vl" token, plus "vlm".
@@ -53,6 +59,96 @@ def is_vision_model(model_name: str) -> bool:
     return bool(_VISION_VL_RE.search(m))
 
 
+_PROVIDER_FINGERPRINT_TTL = 60.0
+# (host, port) -> (models_list | None, expiry); list = LM Studio, None = not LM Studio.
+_lmstudio_models_cache: dict = {}
+
+
+def _is_local_host(host: Optional[str]) -> bool:
+    """True for loopback/LAN/Tailscale hosts (never public domains)."""
+    host = (host or "").lower()
+    if not host:
+        return False
+    if host in {"localhost", "host.docker.internal"} or host.endswith(".local"):
+        return True
+    try:
+        ip = ipaddress.ip_address(host)
+    except ValueError:
+        return "." not in host
+    if ip.is_loopback or ip.is_private or ip.is_link_local:
+        return True
+    return ip in ipaddress.ip_network("100.64.0.0/10")
+
+
+def _probe_lmstudio_models(url: str) -> Optional[list]:
+    """Return LM Studio's native /api/v1/models list, or None when the endpoint
+    isn't LM Studio or is unreachable (short-TTL cached; transient errors uncached)."""
+    parsed = urlparse(url)
+    host = parsed.hostname or ""
+    key = (host, parsed.port)
+    now = time.time()
+    cached = _lmstudio_models_cache.get(key)
+    if cached is not None and cached[1] > now:
+        return cached[0]
+    authority = host if parsed.port is None else f"{host}:{parsed.port}"
+    probe_url = f"{parsed.scheme or 'http'}://{authority}/api/v1/models"
+    try:
+        r = httpx.get(probe_url, timeout=1.0)
+    except Exception:
+        return None
+    try:
+        data = r.json() if r.is_success else {}
+    except Exception:
+        data = {}
+    models = data.get("models")
+    valid = (
+        isinstance(models, list) and bool(models)
+        and isinstance(models[0], dict)
+        and "key" in models[0] and "architecture" in models[0]
+    )
+    models = models if valid else None
+    _lmstudio_models_cache[key] = (models, now + _PROVIDER_FINGERPRINT_TTL)
+    return models
+
+
+def lmstudio_supports_vision(url: str, model: str) -> Optional[bool]:
+    """Read `model`'s capabilities.vision flag from LM Studio, or None when the
+    endpoint isn't LM Studio or doesn't report it (so callers fall back)."""
+    if not model:
+        return None
+    # Never probe a remote provider; LM Studio is always a local/LAN host.
+    if not _is_local_host(urlparse(url).hostname):
+        return None
+    models = _probe_lmstudio_models(url)
+    if not models:
+        return None
+    want = model.strip().lower()
+    for m in models:
+        if not isinstance(m, dict):
+            continue
+        names = {str(m.get("key", "")).lower(), str(m.get("display_name", "")).lower()}
+        if want in names:
+            caps = m.get("capabilities")
+            if isinstance(caps, dict) and "vision" in caps:
+                return bool(caps.get("vision"))
+            return None
+    return None
+
+
+def model_supports_vision(model_name: str, endpoint_url: str = "") -> bool:
+    """Whether a model accepts images, using the endpoint's reported
+    capability when available (LM Studio) and falling back to name-based
+    detection otherwise."""
+    if endpoint_url:
+        try:
+            advertised = lmstudio_supports_vision(endpoint_url, model_name or "")
+        except Exception:
+            advertised = None
+        if advertised is not None:
+            return advertised
+    return is_vision_model(model_name)
+
+
 def validate_message(message: str) -> str:
     """Validate message input."""
     if not message:

src/context_budget.py

@@ -0,0 +1,55 @@
+"""Adaptive input-token budget for the agent loop (#1170).
+
+The agent soft-trims its input context to ``agent_input_token_budget`` (default
+6000). The old computation was ``min(context_length or budget, budget)``, which
+made the 6000 default a hard ceiling for *every* model — so a 128K or 1M context
+model was silently capped at 6000 input tokens even though it can hold far more.
+
+This derives the effective budget from the model's discovered context window when
+the user has NOT set an explicit budget, while still honouring an explicit setting
+exactly (clamped to the window). Pure and side-effect free so it is unit-testable.
+"""
+
+# Generous ceiling so long-context models are unblocked without sending a
+# pathologically large prompt every agent turn. Tunable; chosen to fully cover
+# 128K models and give 1M models a large but bounded budget.
+DEFAULT_HARD_MAX = 200_000
+DEFAULT_BUDGET = 6000
+DEFAULT_HEADROOM = 0.85
+
+
+def compute_input_token_budget(
+    configured: int,
+    context_length: int,
+    explicit: bool,
+    *,
+    default: int = DEFAULT_BUDGET,
+    headroom: float = DEFAULT_HEADROOM,
+    hard_max: int = DEFAULT_HARD_MAX,
+) -> int:
+    """Return the effective soft input-token budget.
+
+    Args:
+        configured: the value read from settings (may be the default).
+        context_length: the model's discovered context window (0/unknown if none).
+        explicit: True if the user explicitly set ``agent_input_token_budget``.
+
+    Rules:
+        - Explicit user budget is honoured exactly, only clamped to the model's
+          window when that window is known (never send more than the model holds).
+        - Otherwise (default), scale to ``headroom`` of the context window, capped
+          at ``hard_max`` — so long-context models use their capacity.
+        - When the window is unknown, fall back to the configured/default value
+          (preserving the previous behaviour).
+    """
+    configured = int(configured or 0)
+    context_length = int(context_length or 0)
+
+    if explicit and configured > 0:
+        return min(configured, context_length) if context_length > 0 else configured
+
+    if context_length > 0:
+        scaled = int(context_length * headroom)
+        return max(1, min(scaled, hard_max))
+
+    return configured if configured > 0 else default

src/deep_research.py

@@ -535,7 +535,9 @@ class DeepResearcher:
                 return []
 
             # Try primary provider, then fallbacks
-            for prov in _build_provider_chain(provider):
+            chain = _build_provider_chain(provider)
+            raised = False
+            for prov in chain:
                 try:
                     results = await asyncio.to_thread(_call_provider, prov, query, 10)
                     if results:
@@ -544,8 +546,20 @@ class DeepResearcher:
                             self.providers_used.append(prov)
                         return results
                 except Exception as e:
+                    raised = True
                     logger.warning(f"Research search: {prov} failed: {e}")
                     self._last_search_error = f"{prov}: {e}"
+            # Every provider ran but none returned results. If none of them
+            # raised, record an actionable reason here — otherwise this empty
+            # path leaves `_last_search_error` unset and the caller surfaces a
+            # bare "unknown error" (issue #344). This is exactly the SearXNG
+            # case where the service is reachable but all its engines fail, so
+            # each provider returns [] without throwing.
+            if not raised:
+                self._last_search_error = (
+                    f"no results from search provider(s): "
+                    f"{', '.join(chain) if chain else provider}"
+                )
             return []
         except Exception as e:
             logger.error(f"Search failed for '{query}': {e}")

src/document_processor.py

@@ -190,6 +190,22 @@ def _process_office_document(path: str, display_name: str) -> str:
         return f"\n\n[Attached document: {display_name} — {exc}]"
 
 
+# Marker that _process_pdf prepends to extracted text.
+_PDF_CONTENT_MARKER = "\n\n[PDF content]:"
+
+
+def strip_pdf_content_marker(text: str) -> str:
+    """Remove the leading ``[PDF content]:`` wrapper that ``_process_pdf`` adds.
+
+    Uses ``str.removeprefix`` rather than ``str.lstrip(chars)``: ``lstrip``
+    treats its argument as a *set of characters*, so ``lstrip("\\n[PDF content]:")``
+    keeps chewing into the page text that follows the marker. For example
+    ``"\\n\\n[PDF content]:\\n\\n[Page 1 text]:\\nto the board"`` would lose the
+    leading "to" because 't' and 'o' are in the marker's character set.
+    """
+    return (text or "").removeprefix(_PDF_CONTENT_MARKER).strip()
+
+
 def _load_vl_settings() -> dict:
     """Load admin settings from disk."""
     try:

src/email_thread_parser.py

@@ -57,7 +57,8 @@ _CCBCC = r"(?:Cc|Bcc|Kopie|Skrytá kopie|Копия)"
 _HDR_KEYS = rf"(?:{_FROM}|{_SENT}|{_SUBJ}|{_TO}|{_CCBCC}|Importance|Priority)"
 
 _ORIG_RE = re.compile(
-    r"(?:^|\n)[\s>]*[-_=]{3,}\s*(?:Original\s+Message|Ursprüngliche\s+Nachricht|"
+    r"(?:^|\n)[\s>]*[-_=]{3,}\s*(?:Original\s+Message|Forwarded\s+message|"
+    r"Ursprüngliche\s+Nachricht|"
     r"Mensaje\s+original|Messaggio\s+originale|Message\s+d['’]origine|"
     r"Oorspronkelijk\s+bericht|Original\s+meddelande|原文|原始邮件|転送)"
     r"\s*[-_=]{3,}",

src/endpoint_resolver.py

@@ -234,9 +234,14 @@ def resolve_endpoint(
     ep_id = _stg(f"{setting_prefix}_endpoint_id")
     model = _stg(f"{setting_prefix}_model")
 
-    # Unset Utility means "same as Default Chat Model". This keeps background
-    # features usable out of the box and lets users override Utility only when
-    # they explicitly want a separate cheaper/faster model.
+    # If the specific endpoint is not configured, but the caller provided a 
+    # valid fallback (e.g. the active session model), use that immediately.
+    # This prevents background tasks from jumping to the global default_model
+    # when the user is mid-conversation with a different model.
+    if not ep_id and fallback_url and fallback_model:
+        return fallback_url, fallback_model, fallback_headers
+
+    # Unset Utility means "same as Default Chat Model".
     if setting_prefix == "utility" and not ep_id:
         ep_id = _stg("default_endpoint_id")
         model = _stg("default_model")

src/llm_core.py

@@ -8,6 +8,7 @@ import hashlib
 import threading
 from fastapi import HTTPException
 from typing import Optional, Dict, List
+from src.model_context import get_context_length, DEFAULT_CONTEXT
 from urllib.parse import urlparse
 
 logger = logging.getLogger(__name__)
@@ -238,7 +239,19 @@ def _build_ollama_payload(
     max_tokens: int,
     stream: bool = False,
     tools: Optional[List[Dict]] = None,
+    num_ctx: Optional[int] = None,
 ) -> Dict:
+    """Build the JSON payload for Ollama's /api/chat endpoint.
+
+    ``num_ctx`` sets the input context window. Ollama defaults to 2048
+    when the option is omitted, so a model with a larger advertised
+    window is silently truncated there, and a model with a smaller one
+    gets an oversized window it can't service. Pass the discovered
+    context length through ``num_ctx``; this builder only emits it when
+    the value is trusted (not the ``DEFAULT_CONTEXT`` fallback), so we
+    don't guess for unknown models but do tell Ollama the real window
+    when we know it — even if it's smaller than 2048.
+    """
     payload: Dict = {
         "model": model,
         "messages": _ollama_normalize_tool_messages(messages),
@@ -249,6 +262,8 @@ def _build_ollama_payload(
         options["temperature"] = temperature
     if max_tokens and max_tokens > 0:
         options["num_predict"] = max_tokens
+    if num_ctx is not None and num_ctx > 0 and num_ctx != DEFAULT_CONTEXT:
+        options["num_ctx"] = num_ctx
     if options:
         payload["options"] = options
     if tools:
@@ -388,6 +403,22 @@ def _uses_max_completion_tokens(model: str) -> bool:
     m = model.lower()
     return any(m.startswith(p) or f"/{p}" in m for p in _MAX_COMPLETION_TOKENS_MODELS)
 
+# OpenAI reasoning models (o1, o3, o4, gpt-5 families) only accept the default
+# temperature. Sending any explicit value — even 0.0 — returns HTTP 400
+# ("Only the default (1) value is supported"). That otherwise breaks chat when a
+# preset sets a non-default temperature, and makes endpoint probing report a
+# perfectly good model as failing. For these models we omit the field and let
+# the API use its required default. (gpt-4.5 is intentionally excluded — it is
+# not a reasoning model and accepts temperature normally.)
+_FIXED_TEMPERATURE_MODELS = ("o1", "o3", "o4", "gpt-5")
+
+def _restricts_temperature(model: str) -> bool:
+    """Check if a model rejects any non-default temperature."""
+    if not model:
+        return False
+    m = model.lower()
+    return any(m.startswith(p) or f"/{p}" in m for p in _FIXED_TEMPERATURE_MODELS)
+
 # Models that support structured thinking — may output </think> without opening tag
 _THINKING_MODEL_PATTERNS = ("qwen3", "qwq", "deepseek-r1", "deepseek-reasoner", "minimax", "m2-reap")
 
@@ -531,11 +562,32 @@ def _build_anthropic_headers(headers):
     return h
 
 def _parse_anthropic_response(data: dict) -> str:
-    """Extract text from Anthropic response."""
-    for block in data.get("content", []):
-        if block.get("type") == "text":
-            return block.get("text", "")
-    return ""
+    """Extract text from an Anthropic response.
+
+    The Messages API `content` is an array that can hold more than one text
+    block (e.g. text split around a tool_use block, or citation-segmented
+    text). Concatenate them all instead of returning only the first, which
+    silently dropped the rest of the reply.
+    """
+    return "".join(
+        block.get("text", "")
+        for block in data.get("content", [])
+        if isinstance(block, dict) and block.get("type") == "text"
+    )
+
+
+def _as_content_blocks(content) -> List[Dict]:
+    """Coerce a message `content` into a list of content blocks.
+
+    A list (multimodal: text + image parts) passes through; a non-empty string
+    becomes a single text block; None/empty yields no blocks. Used when merging
+    consecutive user messages so multimodal content isn't str()-ed away.
+    """
+    if isinstance(content, list):
+        return content
+    if content:
+        return [{"type": "text", "text": str(content)}]
+    return []
 
 
 def _sanitize_llm_messages(messages: List[Dict]) -> List[Dict]:
@@ -571,7 +623,110 @@ def _sanitize_llm_messages(messages: List[Dict]) -> List[Dict]:
                 cleaned.append(item)
         elif "content" in item:
             cleaned.append(item)
-    return cleaned
+
+    # Repair tool-call adjacency before sending to any OpenAI-compatible
+    # provider. Trimming/compaction/retries can leave `role:"tool"` messages
+    # without their immediately-preceding assistant `tool_calls` parent, which
+    # DeepSeek rejects with:
+    # "Messages with role 'tool' must be a response to a preceding message with
+    # 'tool_calls'". Also strip unanswered assistant tool_calls; some providers
+    # reject those as incomplete conversations.
+    repaired: List[Dict] = []
+    i = 0
+    while i < len(cleaned):
+        msg = cleaned[i]
+        role = msg.get("role")
+
+        if role == "tool":
+            # Orphan tool result. There is no valid assistant tool_calls parent
+            # immediately before this batch, so it cannot be sent.
+            logger.debug("Dropping orphan tool message before provider request")
+            i += 1
+            continue
+
+        tool_calls = msg.get("tool_calls") if role == "assistant" else None
+        if not tool_calls:
+            repaired.append(msg)
+            i += 1
+            continue
+
+        call_ids = [
+            str(tc.get("id"))
+            for tc in tool_calls
+            if isinstance(tc, dict) and tc.get("id")
+        ]
+        expected = set(call_ids)
+        answered_ids = []
+        tool_batch = []
+        j = i + 1
+        while j < len(cleaned) and cleaned[j].get("role") == "tool":
+            tid = str(cleaned[j].get("tool_call_id") or "")
+            if tid in expected and tid not in answered_ids:
+                answered_ids.append(tid)
+                tool_batch.append(cleaned[j])
+            else:
+                logger.debug("Dropping unmatched/duplicate tool message before provider request")
+            j += 1
+
+        if not tool_batch:
+            plain = {k: v for k, v in msg.items() if k != "tool_calls"}
+            if (plain.get("content") or "").strip():
+                repaired.append(plain)
+            else:
+                logger.debug("Dropping unanswered assistant tool_calls before provider request")
+            i = j
+            continue
+
+        answered = set(answered_ids)
+        pruned_calls = [
+            tc for tc in tool_calls
+            if isinstance(tc, dict) and str(tc.get("id")) in answered
+        ]
+        fixed = dict(msg)
+        fixed["tool_calls"] = pruned_calls
+        if "content" not in fixed:
+            fixed["content"] = None
+        repaired.append(fixed)
+        repaired.extend(tool_batch)
+        if len(pruned_calls) != len(tool_calls):
+            logger.debug("Pruned unanswered assistant tool_calls before provider request")
+        i = j
+
+    # Merge consecutive user messages to satisfy strict role alternation
+    # requirements after invalid tool-call fragments have been removed.
+    merged: List[Dict] = []
+    for item in repaired:
+        if not merged:
+            merged.append(item)
+            continue
+
+        last = merged[-1]
+        if last.get("role") == "user" and item.get("role") == "user":
+            last_copy = dict(last)
+            lc = last_copy.get("content")
+            ic = item.get("content")
+            if isinstance(lc, list) or isinstance(ic, list):
+                # Preserve multimodal content blocks (e.g. an image part) by
+                # concatenating the block lists. str()-ing a list turned an
+                # image message into its Python repr and dropped the image.
+                merged_blocks = _as_content_blocks(lc) + _as_content_blocks(ic)
+                if merged_blocks:
+                    last_copy["content"] = merged_blocks
+                else:
+                    last_copy.pop("content", None)
+            else:
+                last_str = str(lc) if lc is not None else ""
+                item_str = str(ic) if ic is not None else ""
+                new_content = "\n\n".join(part for part in (last_str, item_str) if part)
+                if new_content:
+                    last_copy["content"] = new_content
+                else:
+                    last_copy.pop("content", None)
+            merged[-1] = last_copy
+        else:
+            merged.append(item)
+
+    return merged
 
 def _normalize_anthropic_url(url: str) -> str:
     """Ensure Anthropic URL points to /v1/messages."""
@@ -675,7 +830,10 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
         payload = _build_anthropic_payload(model, messages_copy, temperature, max_tokens)
     elif provider == "ollama":
         target_url = _normalize_ollama_url(url)
-        payload = _build_ollama_payload(model, messages_copy, temperature, max_tokens, stream=False)
+        payload = _build_ollama_payload(
+            model, messages_copy, temperature, max_tokens,
+            stream=False, num_ctx=get_context_length(url, model),
+        )
     else:
         target_url = url
         payload = {
@@ -683,6 +841,8 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
             "messages": messages_copy,
             "temperature": temperature,
         }
+        if _restricts_temperature(model):
+            payload.pop("temperature", None)
         if max_tokens and max_tokens > 0:
             tok_key = "max_completion_tokens" if _uses_max_completion_tokens(model) else "max_tokens"
             payload[tok_key] = max_tokens
@@ -790,7 +950,10 @@ async def llm_call_async(
         h = {"Content-Type": "application/json"}
         if headers:
             h.update(headers)
-        payload = _build_ollama_payload(model, messages_copy, temperature, max_tokens, stream=False)
+        payload = _build_ollama_payload(
+            model, messages_copy, temperature, max_tokens,
+            stream=False, num_ctx=get_context_length(url, model),
+        )
     else:
         target_url = url
         h = _provider_headers(provider, headers)
@@ -799,6 +962,8 @@ async def llm_call_async(
             "messages": messages_copy,
             "temperature": temperature,
         }
+        if _restricts_temperature(model):
+            payload.pop("temperature", None)
         if max_tokens and max_tokens > 0:
             tok_key = "max_completion_tokens" if _uses_max_completion_tokens(model) else "max_tokens"
             payload[tok_key] = max_tokens
@@ -888,7 +1053,10 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
         h = {"Content-Type": "application/json"}
         if headers:
             h.update(headers)
-        payload = _build_ollama_payload(model, messages_copy, temperature, max_tokens, stream=True, tools=tools)
+        payload = _build_ollama_payload(
+            model, messages_copy, temperature, max_tokens,
+            stream=True, tools=tools, num_ctx=get_context_length(url, model),
+        )
     else:
         target_url = url
         payload = {
@@ -897,6 +1065,8 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
             "temperature": temperature,
             "stream": True,
         }
+        if _restricts_temperature(model):
+            payload.pop("temperature", None)
         if provider not in {"openrouter", "groq"}:
             payload["stream_options"] = {"include_usage": True}
         if max_tokens and max_tokens > 0:
@@ -1122,7 +1292,19 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
                                 _delta0 = _choices[0].get("delta") if _choices else None
                                 if "usage" in j and _delta0 in (None, {}, {"content": None}):
                                     u = j["usage"]
-                                    yield f'data: {json.dumps({"type": "usage", "data": {"input_tokens": u.get("prompt_tokens", 0), "output_tokens": u.get("completion_tokens", 0)}})}\n\n'
+                                    _usage_data = {"input_tokens": u.get("prompt_tokens", 0), "output_tokens": u.get("completion_tokens", 0)}
+                                    # llama.cpp puts a `timings` block alongside `usage` with the
+                                    # TRUE generation speed (predicted_per_second) — pure decode,
+                                    # excluding prefill/network. Pass it through so the UI shows the
+                                    # real gen t/s instead of recomputing tokens/wall-clock (which
+                                    # includes prefill and reads ~20-40% low). Prefill speed too.
+                                    _tm = j.get("timings")
+                                    if isinstance(_tm, dict):
+                                        if _tm.get("predicted_per_second"):
+                                            _usage_data["gen_tps"] = round(_tm["predicted_per_second"], 2)
+                                        if _tm.get("prompt_per_second"):
+                                            _usage_data["prefill_tps"] = round(_tm["prompt_per_second"], 2)
+                                    yield f'data: {json.dumps({"type": "usage", "data": _usage_data})}\n\n'
                                 elif "choices" in j:
                                     delta = j["choices"][0].get("delta") or {}
                                     if isinstance(delta, dict):

src/mcp_manager.py

@@ -43,6 +43,8 @@ class McpManager:
         self._sessions: Dict[str, Any] = {}
         # server_id -> exit stack (for cleanup)
         self._stacks: Dict[str, Any] = {}
+        # Tracking updates to tools/connections for RAG indexing
+        self._generation = 0
 
     async def connect_server(
         self,
@@ -57,16 +59,20 @@ class McpManager:
         """Connect to an MCP server via stdio or SSE transport."""
         try:
             if transport == "stdio":
-                return await self._connect_stdio(server_id, name, command, args or [], env or {})
+                res = await self._connect_stdio(server_id, name, command, args or [], env or {})
             elif transport == "sse":
-                return await self._connect_sse(server_id, name, url)
+                res = await self._connect_sse(server_id, name, url)
             else:
                 logger.error(f"Unknown MCP transport: {transport}")
-                return False
+                res = False
+            if res:
+                self._generation += 1
+            return res
         except Exception as e:
             logger.error(f"Failed to connect MCP server {name} ({server_id}): {e}")
             error_message = _format_mcp_connection_error(name, command or "", args or [], e)
             self._connections[server_id] = {"status": "error", "error": error_message, "name": name}
+            self._generation += 1
             return False
 
     async def _connect_stdio(self, server_id: str, name: str, command: str, args: List[str], env: Dict[str, str]) -> bool:
@@ -182,6 +188,7 @@ class McpManager:
         self._sessions.pop(server_id, None)
         self._tools.pop(server_id, None)
         self._connections.pop(server_id, None)
+        self._generation += 1
         logger.info(f"MCP server disconnected: {server_id}")
 
     async def disconnect_all(self):
@@ -387,7 +394,11 @@ class McpManager:
 
     def get_tool_descriptions_for_prompt(self, disabled_map: Optional[Dict[str, set]] = None) -> str:
         """Generate text describing MCP tools for the agent system prompt. Cached."""
-        cache_key = (frozenset((k, frozenset(v)) for k, v in (disabled_map or {}).items()), len(self._tools))
+        cache_key = (
+            frozenset((k, frozenset(v)) for k, v in (disabled_map or {}).items()),
+            len(self._tools),
+            self._generation,
+        )
         if self._cached_prompt_desc is not None and self._cached_prompt_desc_key == cache_key:
             return self._cached_prompt_desc
         tools = self.get_all_tools(disabled_map)

src/model_context.py

@@ -184,14 +184,22 @@ def get_context_length(endpoint_url: str, model: str) -> int:
 
 
 def _lookup_known(model: str) -> Optional[int]:
-    """Check known context windows by substring match."""
+    """Check known context windows by substring match.
+
+    Picks the LONGEST matching key so a short key never shadows a more specific
+    one. Without this, 'o1' (200k) precedes 'o1-mini' (128k) in the table and a
+    first-match return would report o1-mini's window as 200k.
+    """
     name = model.lower()
     basename = name.split("/")[-1] if "/" in name else name
     basename = basename.split(":")[0]  # strip :free, :extended etc.
+    best_key: Optional[str] = None
+    best_ctx: Optional[int] = None
     for key, ctx in KNOWN_CONTEXT_WINDOWS.items():
         if key in basename or key in name:
-            return ctx
-    return None
+            if best_key is None or len(key) > len(best_key):
+                best_key, best_ctx = key, ctx
+    return best_ctx
 
 
 def _query_context_length(endpoint_url: str, model: str) -> int:

src/model_discovery.py

@@ -74,15 +74,33 @@ class ModelDiscovery:
         self.default_host = default_host
         self.openai_api_key = openai_api_key
         self.openai_compat_path = "/v1/chat/completions"
+        # Custom ports from env vars, merged into the scan list by discover_models.
+        self._extra_ports: set = set()
 
     def _get_hosts(self) -> List[str]:
         """Get all hosts to scan, using env override, Tailscale, or default."""
+        self._extra_ports = set()
+
         def _append_host(out: List[str], host: str) -> None:
             host = (host or "").strip()
             if not host or host in out:
                 return
             out.append(host)
 
+        def _append_env_hosts(out: List[str]) -> None:
+            """Add hosts (and any custom ports) from provider-specific env vars."""
+            for env_name in ("OLLAMA_BASE_URL", "OLLAMA_URL", "LM_STUDIO_URL"):
+                raw = os.getenv(env_name, "").strip()
+                if not raw:
+                    continue
+                try:
+                    parsed = urlparse(raw if "://" in raw else "http://" + raw)
+                    _append_host(out, parsed.hostname or "")
+                    if parsed.port:
+                        self._extra_ports.add(parsed.port)
+                except Exception:
+                    pass
+
         # Manual override takes priority
         extra = os.getenv("LLM_HOSTS", "").strip()
         if extra:
@@ -91,6 +109,7 @@ class ModelDiscovery:
             if self.default_host not in hosts:
                 hosts.insert(0, self.default_host)
             _append_host(hosts, "host.docker.internal")
+            _append_env_hosts(hosts)
             return hosts
 
         # Try Tailscale discovery
@@ -100,23 +119,30 @@ class ModelDiscovery:
             if self.default_host not in ts_hosts:
                 ts_hosts.insert(0, self.default_host)
             _append_host(ts_hosts, "host.docker.internal")
+            _append_env_hosts(ts_hosts)
             return ts_hosts
 
         hosts = [self.default_host]
         # Docker desktop/Linux compose maps this to the host machine. That is
         # the common "I started Ollama normally on this computer" case.
         _append_host(hosts, "host.docker.internal")
-        for env_name in ("OLLAMA_BASE_URL", "OLLAMA_URL"):
-            raw = os.getenv(env_name, "").strip()
-            if not raw:
-                continue
-            try:
-                parsed = urlparse(raw if "://" in raw else "http://" + raw)
-                _append_host(hosts, parsed.hostname or "")
-            except Exception:
-                pass
+        _append_env_hosts(hosts)
         return hosts
 
+    def _fingerprint_provider(self, host: str, port: int) -> Optional[str]:
+        """Identify the server software via its native API, independent of port."""
+        try:
+            r = httpx.get(f"http://{host}:{port}/api/v1/models", timeout=1.5)
+            if r.is_success:
+                models = (r.json() or {}).get("models")
+                if (isinstance(models, list) and models
+                        and isinstance(models[0], dict)
+                        and "key" in models[0] and "architecture" in models[0]):
+                    return "lmstudio"
+        except Exception:
+            pass
+        return None
+
     def _check_port(self, host: str, port: int) -> Optional[Dict[str, Any]]:
         """Check a single host:port for models."""
         base = f"http://{host}:{port}/v1"
@@ -132,7 +158,8 @@ class ModelDiscovery:
                     "port": port,
                     "url": f"http://{host}:{port}{self.openai_compat_path}",
                     "models": ids,
-                    "models_display": [i.lstrip("/") for i in ids]
+                    "models_display": [i.lstrip("/") for i in ids],
+                    "provider": self._fingerprint_provider(host, port),
                 }
         except Exception:
             pass
@@ -145,9 +172,10 @@ class ModelDiscovery:
 
         logger.info(f"Scanning {len(hosts)} hosts for models: {hosts}")
 
-        # Build list of (host, port) to check. 8000-8020 catches vLLM,
-        # llama.cpp, SGLang, and Cookbook serves; 11434 catches Ollama.
-        ports = list(range(8000, 8021)) + [11434]
+        # Well-known ports: 8000-8020 (vLLM, llama.cpp, SGLang, Cookbook),
+        # 1234 (LM Studio), 11434 (Ollama)
+        ports = list(range(8000, 8021)) + [1234, 11434]
+        ports += [p for p in sorted(self._extra_ports) if p not in ports]
         targets = [(h, p) for h in hosts for p in ports]
 
         seen_models = set()  # dedupe by (port, model_ids) to avoid same machine via different IPs

src/personal_docs.py

@@ -246,8 +246,15 @@ class PersonalDocsManager:
         # Normalize the path
         directory = os.path.abspath(directory)
 
-        # Clear any exclusions for files in this directory
-        self.excluded_files = {p for p in self.excluded_files if not p.startswith(directory)}
+        # Clear any exclusions for files in this directory. Match on a path
+        # boundary (the directory itself or paths under it) rather than a raw
+        # string prefix: a bare ``startswith(directory)`` also matches sibling
+        # directories that merely share a name prefix (e.g. adding ``/docs``
+        # would wrongly un-exclude files under ``/docs2``).
+        self.excluded_files = {
+            p for p in self.excluded_files
+            if not (p == directory or p.startswith(directory + os.sep))
+        }
         self._save_excluded()
 
         if directory not in self.indexed_directories:

src/preset_manager.py

@@ -92,6 +92,18 @@ Use precise language. Show causal relationships explicitly. Quantify uncertainty
                     custom.setdefault("inject_prefix", "")
                     custom.setdefault("inject_suffix", "")
                     self.save(presets)
+            # Heal a forward-incompatible file the same way the legacy `custom`
+            # migration above does: fill in any built-in presets an older or
+            # partial presets.json is missing, so they reach existing installs
+            # (a missing built-in is otherwise silently absent from the picker
+            # served by GET /api/presets). There is no delete path for the
+            # built-in keys, so this never clobbers an intentional removal.
+            # Defaults first, loaded values win — user edits are preserved.
+            if isinstance(presets, dict) and any(
+                k not in presets for k in self.DEFAULT_PRESETS
+            ):
+                presets = {**self.DEFAULT_PRESETS, **presets}
+                self.save(presets)
             return presets
         except Exception as e:
             logger.error(f"Error loading presets: {e}")

src/rag_vector.py

@@ -7,6 +7,7 @@ configurable embedding endpoint via EMBEDDING_URL env var.
 """
 
 import os
+import hashlib
 import re
 import logging
 import numpy as np
@@ -26,6 +27,10 @@ KEYWORD_WEIGHT = 0.3
 COLLECTION_NAME = "odysseus_rag"
 
 
+def _generate_doc_id(text: str) -> str:
+    return f"doc_{hashlib.sha256(text.encode('utf-8')).hexdigest()[:16]}"
+
+
 class VectorRAG:
     """RAG system using ChromaDB vector storage with hybrid search."""
 
@@ -99,7 +104,7 @@ class VectorRAG:
             return False
 
         try:
-            doc_id = f"doc_{hash(text) % 10**16}"
+            doc_id = _generate_doc_id(text)
             # Check if already exists
             existing = self._collection.get(ids=[doc_id])
             if existing["ids"]:
@@ -135,7 +140,7 @@ class VectorRAG:
             new_metas = []
             new_ids = []
             for t, m in valid:
-                doc_id = f"doc_{hash(t) % 10**16}"
+                doc_id = _generate_doc_id(t)
                 existing = self._collection.get(ids=[doc_id])
                 if not existing["ids"]:
                     new_texts.append(t)

src/readiness.py

@@ -0,0 +1,61 @@
+"""Ithaca anchor — local-instance readiness / integrity self-check.
+
+Beyond ``/api/health``'s liveness ping, this confirms the self-hosted instance is
+whole and at home: the database is reachable, the data directory is present and
+writable, and storage is local-first. Served by ``GET /api/ready`` and suitable
+for an orchestrator readiness probe (200 only when every critical check passes).
+"""
+
+import os
+import uuid
+from datetime import datetime
+from typing import Dict
+
+
+def check_readiness() -> Dict[str, object]:
+    """Run the readiness checks and return a JSON-serialisable report.
+
+    ``ready`` is True only when every critical check (database, data_dir) passes.
+    ``local_first`` is informational — a remote database is a valid deployment, so
+    it never fails readiness, it only reports whether storage stays on this host.
+    """
+    from core.constants import APP_VERSION, DATA_DIR
+    from core.database import DATABASE_URL, engine
+    from sqlalchemy import text as sql_text
+
+    checks: Dict[str, Dict[str, object]] = {}
+
+    # Database reachable — the simplest honest probe that the engine is live.
+    try:
+        with engine.connect() as conn:
+            conn.execute(sql_text("SELECT 1"))
+        checks["database"] = {"ok": True}
+    except Exception as e:
+        checks["database"] = {"ok": False, "error": str(e)}
+
+    # Data directory present and writable — home must be able to hold its own data.
+    try:
+        os.makedirs(DATA_DIR, exist_ok=True)
+        probe = os.path.join(DATA_DIR, f".ready_probe_{uuid.uuid4().hex}")
+        with open(probe, "w", encoding="utf-8") as fh:
+            fh.write("ok")
+        os.remove(probe)
+        checks["data_dir"] = {"ok": True, "path": DATA_DIR}
+    except Exception as e:
+        checks["data_dir"] = {"ok": False, "error": str(e)}
+
+    # Local-first: storage stays on the home machine (informational, never fatal).
+    local_first = (
+        DATABASE_URL.startswith("sqlite")
+        or "localhost" in DATABASE_URL
+        or "127.0.0.1" in DATABASE_URL
+    )
+    checks["local_first"] = {"ok": True, "local": local_first}
+
+    ready = all(bool(c.get("ok")) for c in checks.values())
+    return {
+        "ready": ready,
+        "version": APP_VERSION,
+        "checks": checks,
+        "timestamp": datetime.utcnow().isoformat(),
+    }

src/research_handler.py

@@ -30,6 +30,24 @@ def _bounded_int(value, *, default: int, minimum: int, maximum: int) -> int:
     return max(minimum, min(maximum, n))
 
 
+def _format_probe_failure(model: str, exc: Exception) -> str:
+    """Turn a failed research model probe into a user-facing message."""
+    detail = getattr(exc, "detail", None)
+    status = getattr(exc, "status_code", None)
+    err = str(detail if detail is not None else exc).strip()
+
+    if status in {401, 403} or "401" in err or "API key" in err or "Unauthorized" in err:
+        return f"Model '{model}' requires an API key. Check your endpoint configuration."
+
+    if status and err:
+        return f"Model '{model}' probe failed: {err}"
+
+    if err:
+        return f"Cannot reach model '{model}' — {err}"
+
+    return f"Cannot reach model '{model}' — check that the endpoint is running and accessible."
+
+
 class ResearchHandler:
     """Handles research service operations with iterative deep research."""
 
@@ -216,15 +234,25 @@ class ResearchHandler:
         """
         # Resolve the hard wall-clock timeout from settings when the caller
         # didn't pin one. Local / edge models routinely need more than the
-        # old 600s default to finish a deep-research synthesis.
+        # old 600s default to finish a deep-research synthesis. A setting of
+        # 0 disables the cap entirely (unlimited run); any other value is
+        # bounded to [60, 86400] so a misconfigured settings.json can't
+        # explode into a multi-day hang.
         if hard_timeout is None:
             from src.settings import get_setting
-            hard_timeout = _bounded_int(
-                get_setting("research_run_timeout_seconds", 1800),
-                default=1800,
-                minimum=60,
-                maximum=86400,
-            )
+            try:
+                raw_timeout = int(get_setting("research_run_timeout_seconds", 1800))
+            except (TypeError, ValueError):
+                raw_timeout = 1800
+            if raw_timeout <= 0:
+                hard_timeout = None  # 0 = no wall-clock cap (asyncio.wait_for timeout=None)
+            else:
+                hard_timeout = _bounded_int(
+                    raw_timeout,
+                    default=1800,
+                    minimum=60,
+                    maximum=86400,
+                )
 
         # Cancel any existing research for this session
         if session_id in self._active_tasks:
@@ -624,14 +652,7 @@ class ResearchHandler:
             logger.info(f"Endpoint probe OK: {model}")
         except Exception as e:
             logger.error(f"Probe failed for {model}: {e}")
-            err = str(e)
-            if "401" in err or "API key" in err or "Unauthorized" in err:
-                raise RuntimeError(
-                    f"Model '{model}' requires an API key. Check your endpoint configuration."
-                ) from e
-            raise RuntimeError(
-                f"Cannot reach model '{model}' — check that the endpoint is running and accessible."
-            ) from e
+            raise RuntimeError(_format_probe_failure(model, e)) from e
 
     async def call_research_service(
         self,

src/search/analytics.py

@@ -45,32 +45,37 @@ class RateLimitError(SearchEngineError):
 # ----------------------------------------------------------------------
 # Analytics helpers
 # ----------------------------------------------------------------------
+def _default_analytics() -> Dict[str, Any]:
+    """A fresh analytics document with every counter present."""
+    return {
+        "total_queries": 0,
+        "successful_queries": 0,
+        "failed_queries": 0,
+        "cache_hits": 0,
+        "cache_misses": 0,
+        "query_patterns": {},
+    }
+
+
 def _load_analytics() -> Dict[str, Any]:
     """Load analytics data from the JSON file, creating defaults if missing."""
     if not ANALYTICS_FILE.exists():
-        default = {
-            "total_queries": 0,
-            "successful_queries": 0,
-            "failed_queries": 0,
-            "cache_hits": 0,
-            "cache_misses": 0,
-            "query_patterns": {},
-        }
+        default = _default_analytics()
         _save_analytics(default)
         return default
     try:
         with open(ANALYTICS_FILE, "r", encoding="utf-8") as f:
-            return json.load(f)
+            data = json.load(f)
+        # Merge over defaults so a file written by an older schema (or a
+        # partial write) still has every counter — _record_query indexes
+        # these keys directly and would otherwise raise KeyError.
+        merged = _default_analytics()
+        if isinstance(data, dict):
+            merged.update(data)
+        return merged
     except Exception as e:
         logger.warning(f"Failed to load analytics file: {e}")
-        return {
-            "total_queries": 0,
-            "successful_queries": 0,
-            "failed_queries": 0,
-            "cache_hits": 0,
-            "cache_misses": 0,
-            "query_patterns": {},
-        }
+        return _default_analytics()
 
 
 def _save_analytics(data: Dict[str, Any]) -> None:

src/search/content.py

@@ -390,13 +390,18 @@ def get_tldr(text: str, max_sentences: int = 3) -> str:
 
 def extract_quotes(text: str) -> List[str]:
     """Return quoted excerpts that are at least 15 characters long."""
-    return [m.group(1).strip() for m in re.finditer(r'["\']([^"\']{15,}?)["\']', text)]
+    # Backreference the opening quote so the closing quote must match it —
+    # otherwise `"text'` (open double, close single) is treated as a quote.
+    return [m.group(2).strip() for m in re.finditer(r'(["\'])([^"\']{15,}?)\1', text)]
 
 
 def extract_statistics(text: str) -> List[str]:
     """Find numbers, percentages, dates and simple measurements."""
+    # Match a comma-grouped number (1,000,000) OR a plain digit run (50000) —
+    # the old `\d{1,3}(?:,\d{3})*` matched only the first 3 digits of a
+    # comma-less number, and the trailing `\b` dropped a closing `%`.
     pattern = re.compile(
-        r"\b\d{1,3}(?:,\d{3})*(?:\.\d+)?\s*(%|percent|‰|per cent|[a-zA-Z]+)?\b",
+        r"\b(?:\d{1,3}(?:,\d{3})+|\d+)(?:\.\d+)?\s*(%|percent|‰|per cent|[a-zA-Z]+)?",
         re.IGNORECASE,
     )
     return [m.group(0).strip() for m in pattern.finditer(text)]

src/search/core.py

@@ -1,450 +1,12 @@
-"""Core search orchestrators: searxng_search_results, comprehensive_web_search, config, cache invalidation."""
+"""Compatibility wrapper for the canonical services.search.core module.
 
-import json
-import logging
-from concurrent.futures import ThreadPoolExecutor, as_completed
-from datetime import datetime, timedelta
-from typing import Dict, Any, Optional, List, Set
-from urllib.parse import urlparse
+``src.search.core`` remains importable for older agent/deep-research code, but
+the implementation now lives in ``services.search.core`` so provider ordering,
+cache invalidation, and search route behavior cannot drift between copies.
+"""
 
-from .analytics import (
-    NetworkError,
-    ParseError,
-    RateLimitError,
-    error_logger,
-    _record_query,
-)
-from .cache import (
-    SEARCH_CACHE_DIR,
-    search_cache_index,
-    generate_cache_key,
-    cleanup_cache,
-)
-from .query import _cache_duration_for_query
-from .ranking import rank_search_results
-from .providers import (
-    searxng_search_api,
-    brave_search,
-    duckduckgo_search,
-    google_pse_search,
-    tavily_search,
-    serper_search,
-    _get_search_settings,
-    _get_result_count,
-)
-from .content import (
-    fetch_webpage_content,
-    extract_key_points,
-    get_tldr,
-    extract_quotes,
-    extract_statistics,
-)
+import sys
 
-logger = logging.getLogger(__name__)
+from services.search import core as _core
 
-# ========= CONFIG =========
-SEARCH_CONFIG: Dict[str, Any] = {
-    "primary_provider": "searxng",
-}
-
-
-def get_search_config() -> Dict[str, Any]:
-    """Get current search configuration including active provider info."""
-    config = SEARCH_CONFIG.copy()
-    settings = _get_search_settings()
-    provider = settings.get("search_provider", "searxng")
-    config["active_provider"] = provider
-    config["has_api_key"] = bool((settings.get("search_api_key") or "").strip())
-    config["result_count"] = _get_result_count()
-    if provider == "searxng":
-        from .providers import _get_search_instance
-        config["search_url"] = _get_search_instance()
-    return config
-
-
-def update_search_config(api_key: str = None, **kwargs):
-    """Update search configuration (e.g. Brave API key)."""
-    if api_key:
-        SEARCH_CONFIG["brave_api_key"] = api_key
-
-
-def _call_provider(provider_name: str, query: str, count: int, time_filter: str = None) -> List[dict]:
-    """Call a search provider by name. Returns list of results or empty list."""
-    if provider_name == "searxng":
-        return searxng_search_api(query, count, time_filter=time_filter)
-    elif provider_name == "brave":
-        return brave_search(query, count, time_filter)
-    elif provider_name == "duckduckgo":
-        return duckduckgo_search(query, count, time_filter)
-    elif provider_name == "google_pse":
-        return google_pse_search(query, count, time_filter)
-    elif provider_name == "tavily":
-        return tavily_search(query, count, time_filter)
-    elif provider_name == "serper":
-        return serper_search(query, count, time_filter)
-    return []
-
-
-# If the self-hosted SearXNG instance is up but all enabled engines return
-# empty, fall back to the no-key provider so "search X" still works on fresh
-# installs. Users can override/disable with `search_fallback_chain`.
-_FALLBACK_ORDER = ["duckduckgo"]
-
-
-def _build_provider_chain(primary: str) -> List[str]:
-    """Build ordered list: primary first, then fallbacks (skipping primary
-    and dedupes). The fallback list comes from
-    `settings.search_fallback_chain` if the user configured one, otherwise
-    the hardcoded default above."""
-    chain = [primary]
-    settings = _get_search_settings()
-    user_chain = settings.get("search_fallback_chain") or []
-    if isinstance(user_chain, str):
-        # Tolerate comma-separated form from older payloads.
-        user_chain = [s.strip() for s in user_chain.split(",") if s.strip()]
-    fallbacks = user_chain if user_chain else _FALLBACK_ORDER
-    for fb in fallbacks:
-        if fb and fb != primary and fb not in chain and fb != "disabled":
-            chain.append(fb)
-    return chain
-
-
-# ----------------------------------------------------------------------
-# Unified search with caching and retry
-# ----------------------------------------------------------------------
-def searxng_search_results(query: str, count: int = 10, time_filter: str = None) -> list[dict]:
-    """Perform a web search using configured provider with caching and retry."""
-    settings = _get_search_settings()
-    search_provider = settings.get("search_provider", "searxng")
-    result_count = _get_result_count()
-    # Use configured count if caller used default
-    if count == 10:
-        count = result_count
-
-    cache_key = generate_cache_key(f"{query}|{count}|{time_filter}")
-    cache_file = SEARCH_CACHE_DIR / f"{cache_key}.cache"
-
-    # Check cache
-    if cache_file.exists():
-        try:
-            with open(cache_file, "r", encoding="utf-8") as f:
-                cached_data = json.load(f)
-            expiry_raw = cached_data.get("expiry")
-            expiry = datetime.fromisoformat(expiry_raw) if expiry_raw else None
-            if expiry and datetime.now() < expiry:
-                logger.debug(f"Search cache hit for query: {query}")
-                results = cached_data["data"]
-                _record_query(query, bool(results), cache_hit=True)
-                return results
-            else:
-                cache_file.unlink(missing_ok=True)
-                search_cache_index.pop(cache_key, None)
-        except Exception as e:
-            logger.warning(f"Failed to read search cache for {query}: {e}")
-            cache_file.unlink(missing_ok=True)
-            search_cache_index.pop(cache_key, None)
-
-    logger.debug(f"Search cache miss for query: {query}")
-
-    if search_provider == "disabled":
-        logger.info("Search is disabled via admin settings")
-        return []
-
-    provider_chain = _build_provider_chain(search_provider)
-
-    results: List[dict] = []
-    for provider_name in provider_chain:
-        for attempt in range(2):
-            try:
-                logger.info(f"Attempting {provider_name} search (attempt {attempt + 1})")
-                results = _call_provider(provider_name, query, count, time_filter)
-                if results:
-                    logger.info(f"{provider_name} search succeeded with {len(results)} results")
-                    break
-            except (NetworkError, ParseError, RateLimitError) as e:
-                error_logger.error(f"{provider_name} search error (attempt {attempt + 1}): {e}")
-            except Exception as e:
-                error_logger.error(f"Unexpected error during {provider_name} search (attempt {attempt + 1}): {e}")
-        if results:
-            break
-
-    success = bool(results)
-    _record_query(query, success, cache_hit=False)
-
-    if success:
-        results = rank_search_results(query, results)
-        try:
-            expiry = datetime.now() + _cache_duration_for_query(query)
-            cache_data = {
-                "timestamp": datetime.now().isoformat(),
-                "expiry": expiry.isoformat(),
-                "data": results,
-            }
-            with open(cache_file, "w", encoding="utf-8") as f:
-                json.dump(cache_data, f)
-            search_cache_index[cache_key] = datetime.now()
-            cleanup_cache(SEARCH_CACHE_DIR, search_cache_index, timedelta(hours=1))
-        except Exception as e:
-            logger.warning(f"Failed to write search cache for {query}: {e}")
-
-    if not success:
-        logger.error(f"All search providers failed for query: {query}")
-
-    return results
-
-
-# ----------------------------------------------------------------------
-# Cache invalidation
-# ----------------------------------------------------------------------
-def invalidate_search_cache(query: Optional[str] = None) -> None:
-    """Invalidate cached search results. None clears all, otherwise just the given query."""
-    if query is None:
-        for file in SEARCH_CACHE_DIR.glob("*.cache"):
-            try:
-                file.unlink(missing_ok=True)
-            except Exception as e:
-                error_logger.warning(f"Failed to delete cache file {file}: {e}")
-        search_cache_index.clear()
-        logger.info("All search cache entries have been cleared.")
-    else:
-        # Match the key the write path stores: searxng_search_results replaces
-        # the caller's default count with the configured _get_result_count()
-        # (default 5), so a hardcoded "|10|None" never matched a real entry.
-        cache_key = generate_cache_key(f"{query}|{_get_result_count()}|None")
-        cache_file = SEARCH_CACHE_DIR / f"{cache_key}.cache"
-        if cache_file.exists():
-            try:
-                cache_file.unlink(missing_ok=True)
-                search_cache_index.pop(cache_key, None)
-                logger.info(f"Cache entry for query '{query}' has been invalidated.")
-            except Exception as e:
-                error_logger.warning(f"Failed to delete cache file for query '{query}': {e}")
-        else:
-            logger.info(f"No cache entry found for query '{query}'.")
-
-
-# ----------------------------------------------------------------------
-# Comprehensive web search (with advanced filtering)
-# ----------------------------------------------------------------------
-def comprehensive_web_search(
-    query: str,
-    max_pages: int = 3,
-    max_workers: int = 4,
-    time_filter: str = None,
-    domain_whitelist: Optional[Set[str]] = None,
-    domain_blacklist: Optional[Set[str]] = None,
-    content_type: Optional[str] = None,
-    language: Optional[str] = None,
-    min_content_length: int = 0,
-    return_sources: bool = False,
-):
-    """Perform comprehensive web search with content fetching and advanced filtering."""
-    logger.info(f"Starting comprehensive search for: {query}")
-    if time_filter:
-        logger.info(f"Applying time filter: {time_filter}")
-
-    settings = _get_search_settings()
-    search_provider = settings.get("search_provider", "searxng")
-    result_count = _get_result_count()
-
-    if search_provider == "disabled":
-        logger.info("Search is disabled via admin settings")
-        msg = "Web search is disabled by the administrator."
-        return (msg, []) if return_sources else msg
-
-    # Use configured result count (at least max_pages for content fetching)
-    fetch_count = max(result_count, max_pages)
-
-    provider_chain = _build_provider_chain(search_provider)
-
-    # Each provider gets 2 attempts (matches the inner unified_search behavior).
-    # Empty results are tracked separately from exceptions so the failure
-    # message can tell a soft-fail (provider returned []) apart from a real
-    # error (network blow-up, rate limit, etc.) — useful both for logging
-    # and for the model when it sees the response.
-    search_results = []
-    provider_attempts = {}  # provider -> "ok N", "empty", "error: ..."
-    for provider_name in provider_chain:
-        last_err = None
-        empty = False
-        for attempt in range(2):
-            try:
-                search_results = _call_provider(provider_name, query, fetch_count, time_filter)
-                if search_results:
-                    provider_attempts[provider_name] = f"ok ({len(search_results)})"
-                    logger.info(f"Comprehensive search: {provider_name} returned {len(search_results)} results")
-                    break
-                # Empty result — try once more (transient empties are common on flaky instances)
-                empty = True
-            except Exception as e:
-                last_err = e
-                logger.warning(f"Comprehensive search: {provider_name} attempt {attempt + 1} failed: {e}")
-        if search_results:
-            break
-        if last_err is not None:
-            provider_attempts[provider_name] = f"error: {last_err}"
-        elif empty:
-            provider_attempts[provider_name] = "empty"
-
-    if not search_results:
-        # Build a per-provider tally so the model (and logs) see which
-        # providers were tried and how each one fared, instead of the
-        # uninformative "No search results found".
-        tally = ", ".join(f"{p}:{r}" for p, r in provider_attempts.items()) or "no providers configured"
-        any_errors = any(r.startswith("error") for r in provider_attempts.values())
-        if any_errors:
-            msg = f"Web search failed — all providers errored or returned empty. Tried: {tally}"
-            logger.error(msg)
-        else:
-            msg = (
-                f"No search results found. Tried: {tally}. "
-                "All providers returned empty — possibly a niche query or upstream rate-limiting; "
-                "rephrasing or using the browser tool for a specific URL may help."
-            )
-            logger.warning(msg)
-        return (msg, []) if return_sources else msg
-
-    search_results = rank_search_results(query, search_results)
-
-    # URL filter helper
-    def url_passes_filters(url: str) -> bool:
-        try:
-            netloc = urlparse(url).netloc.lower()
-        except Exception:
-            return False
-        if domain_whitelist is not None and netloc not in domain_whitelist:
-            return False
-        if domain_blacklist is not None and netloc in domain_blacklist:
-            return False
-        if content_type:
-            ct = content_type.lower()
-            if ct == "article":
-                if not any(k in url.lower() for k in ("article", "blog", "news", "post")):
-                    return False
-            elif ct == "forum":
-                if not any(k in url.lower() for k in ("forum", "discussion", "thread", "topic")):
-                    return False
-            elif ct == "academic":
-                if not any(k in url.lower() for k in ("pdf", "doi", "scholar", "arxiv", "journal", "research")):
-                    return False
-        if language:
-            lang_pat = language.lower()
-            if not (f"/{lang_pat}/" in url.lower() or f"?lang={lang_pat}" in url.lower() or f"&lang={lang_pat}" in url.lower()):
-                return False
-        return True
-
-    filtered_urls = [r["url"] for r in search_results[:max_pages] if url_passes_filters(r["url"])]
-    if not filtered_urls:
-        logger.warning("All URLs filtered out by advanced criteria")
-        msg = "No suitable results after applying filters."
-        return (msg, []) if return_sources else msg
-
-    # Build sources list for the frontend (before content fetching)
-    _source_list = [
-        {"url": r.get("url", ""), "title": r.get("title", "")}
-        for r in search_results if r.get("url")
-    ]
-
-    # Fetch content in parallel
-    fetched_content = []
-    with ThreadPoolExecutor(max_workers=max_workers) as executor:
-        future_to_url = {
-            executor.submit(fetch_webpage_content, url, 8, retry_attempt=0): url
-            for url in filtered_urls
-        }
-        for future in as_completed(future_to_url):
-            url = future_to_url[future]
-            try:
-                result = future.result()
-                if result["success"] and result["content"] and len(result["content"]) >= min_content_length:
-                    fetched_content.append(result)
-            except Exception as e:
-                logger.error(f"Exception while fetching {url}: {str(e)}")
-
-    logger.info(f"Successfully fetched content from {len(fetched_content)} pages")
-
-    # Format results
-    output_parts = []
-
-    if search_results:
-        output_parts.append("```sources")
-        for i, result in enumerate(search_results, 1):
-            output_parts.append(f"[{i}] {result['title']}")
-            output_parts.append(f"    {result['url']}")
-            if result.get("age"):
-                output_parts.append(f"    {result['age']}")
-        output_parts.append("```")
-        output_parts.append("")
-
-    output_parts.append("=" * 70)
-    output_parts.append("WEB SEARCH RESULTS AND FETCHED CONTENT")
-    output_parts.append(f"Query: {query}")
-    output_parts.append(f"Searched {len(search_results)} results, fetched {len(fetched_content)} pages")
-    output_parts.append("=" * 70)
-    output_parts.append("")
-
-    output_parts.append("SEARCH RESULTS SUMMARY:")
-    output_parts.append("-" * 50)
-    for i, result in enumerate(search_results, 1):
-        output_parts.append(f"\n[{i}] {result['title']}")
-        output_parts.append(f"    URL: {result['url']}")
-        output_parts.append(f"    Snippet: {result['snippet'][:200]}...")
-        if result.get("age"):
-            output_parts.append(f"    Age: {result['age']}")
-
-    if fetched_content:
-        output_parts.append("\n" + "=" * 70)
-        output_parts.append("FETCHED PAGE CONTENT:")
-        output_parts.append("-" * 50)
-
-        for i, content in enumerate(fetched_content, 1):
-            output_parts.append(f"\n[CONTENT {i}] From: {content['url']}")
-            output_parts.append(f"Title: {content['title']}")
-            output_parts.append("-" * 30)
-
-            text = content["content"][:3000]
-            if len(content["content"]) > 3000:
-                text += "... [truncated]"
-            output_parts.append(text)
-
-            key_points = extract_key_points(content["content"])
-            if key_points:
-                output_parts.append("\nKey Points:")
-                for pt in key_points[:5]:
-                    output_parts.append(f"- {pt}")
-
-            tldr = get_tldr(content["content"])
-            if tldr:
-                output_parts.append("\nTL;DR:")
-                output_parts.append(tldr)
-
-            quotes = extract_quotes(content["content"])
-            if quotes:
-                output_parts.append("\nImportant Quotes:")
-                for q in quotes[:3]:
-                    output_parts.append(f"\u201c{q}\u201d")
-
-            stats = extract_statistics(content["content"])
-            if stats:
-                output_parts.append("\nData / Statistics:")
-                for s in stats[:5]:
-                    output_parts.append(f"- {s}")
-
-            output_parts.append("")
-
-    output_parts.append("=" * 70)
-    output_parts.append("END OF WEB SEARCH RESULTS")
-    output_parts.append("=" * 70)
-
-    instructions = (
-        "\n\nIMPORTANT INSTRUCTIONS:\n"
-        "1. Use the above web search results and fetched content to answer the user's question\n"
-        "2. Prioritize information from the FETCHED PAGE CONTENT section as it contains actual page data\n"
-        "3. Cross-reference multiple sources when possible\n"
-        "4. If the information is time-sensitive, pay attention to the age of the results\n"
-        "5. Be explicit if the search results don't contain sufficient information to fully answer the question"
-    )
-    output_parts.append(instructions)
-
-    result = "\n".join(output_parts)
-    return (result, _source_list) if return_sources else result
+sys.modules[__name__] = _core

src/search/providers.py

@@ -1,618 +1,12 @@
-"""Search provider implementations: SearXNG, Brave, DuckDuckGo, Google PSE, Tavily, Serper."""
+"""Compatibility wrapper for the canonical services.search.providers module.
 
-import json
-import logging
-import os
-from typing import List, Optional
-from urllib.parse import urljoin, urlparse, parse_qs
+Historically Odysseus carried duplicate provider implementations under both
+``src.search`` and ``services.search``. Keep the old import path working, but
+make provider behavior come from one source of truth.
+"""
 
-import httpx
-from bs4 import BeautifulSoup
+import sys
 
-from src.constants import SEARXNG_INSTANCE
-from .analytics import RateLimitError, error_logger
-from .query import build_enhanced_query
+from services.search import providers as _providers
 
-logger = logging.getLogger(__name__)
-
-REQUEST_TIMEOUT = 20
-
-# Provider registry — maps setting value to (label, needs_key, needs_url)
-PROVIDER_INFO = {
-    "searxng":  ("SearXNG",           False, True),
-    "brave":    ("Brave Search",      True,  False),
-    "duckduckgo": ("DuckDuckGo",      False, False),
-    "google_pse": ("Google PSE",      True,  False),
-    "tavily":   ("Tavily",            True,  False),
-    "serper":   ("Serper",            True,  False),
-    "disabled": ("Disabled",          False, False),
-}
-
-
-# ── Settings helpers ──
-
-def _get_search_settings() -> dict:
-    """Return search settings from admin config, falling back to env defaults."""
-    try:
-        from src.settings import load_settings
-        return load_settings()
-    except Exception:
-        return {}
-
-
-def _get_search_instance() -> str:
-    """Return the active search API URL from admin settings, falling back to env var."""
-    settings = _get_search_settings()
-    url = (settings.get("search_url") or "").strip()
-    if url:
-        return url.rstrip("/")
-    return SEARXNG_INSTANCE
-
-
-def _get_provider_key(provider: str) -> str:
-    """Return the API key for a specific provider, with legacy fallback."""
-    settings = _get_search_settings()
-    key_map = {
-        "brave": "brave_api_key",
-        "google_pse": "google_pse_key",
-        "tavily": "tavily_api_key",
-        "serper": "serper_api_key",
-    }
-    field = key_map.get(provider, "")
-    if field:
-        val = (settings.get(field) or "").strip()
-        if val:
-            return val
-    # Legacy fallback: old shared search_api_key field
-    return (settings.get("search_api_key") or "").strip()
-
-
-def _get_result_count() -> int:
-    """Return configured result count, default 5."""
-    settings = _get_search_settings()
-    try:
-        return int(settings.get("search_result_count", 5))
-    except (ValueError, TypeError):
-        return 5
-
-
-# Canonical SafeSearch levels: "strict" (default), "moderate", "off".
-# Each provider has its own knob name and value space — see _safesearch_for(...).
-_SAFESEARCH_LEVELS = ("strict", "moderate", "off")
-
-
-def _get_safesearch_level() -> str:
-    """Return the configured SafeSearch level, normalized to one of
-    _SAFESEARCH_LEVELS. Defaults to 'strict' to avoid adult / spammy URLs
-    bleeding into research and web_search results."""
-    settings = _get_search_settings()
-    raw = (settings.get("search_safesearch") or "strict").strip().lower()
-    if raw in _SAFESEARCH_LEVELS:
-        return raw
-    # Accept a few common aliases so a manually-edited config doesn't
-    # silently lose SafeSearch — fall back to strict on anything unknown.
-    aliases = {
-        "on": "strict", "high": "strict", "2": "strict",
-        "medium": "moderate", "1": "moderate", "default": "moderate",
-        "none": "off", "disabled": "off", "0": "off",
-    }
-    return aliases.get(raw, "strict")
-
-
-def _safesearch_for(provider: str) -> Optional[str]:
-    """Translate the canonical level into the per-provider param value.
-    Returns None when SafeSearch should be omitted entirely for a provider
-    (some APIs default to filtered and treat missing-param as "off")."""
-    level = _get_safesearch_level()
-    if provider == "searxng":
-        # SearXNG: integer 0/1/2
-        return {"strict": "2", "moderate": "1", "off": "0"}[level]
-    if provider == "brave":
-        # Brave: strict / moderate / off
-        return level
-    if provider == "duckduckgo_lib":
-        # duckduckgo-search library: on / moderate / off
-        return {"strict": "on", "moderate": "moderate", "off": "off"}[level]
-    if provider == "duckduckgo_html":
-        # DDG HTML endpoint kp: 1 strict / -1 moderate / -2 off
-        return {"strict": "1", "moderate": "-1", "off": "-2"}[level]
-    if provider == "google_pse":
-        # Google PSE: 'active' filters explicit; 'off' disables. Treat
-        # moderate the same as active — Google PSE has no middle tier.
-        return None if level == "off" else "active"
-    if provider == "serper":
-        # Serper proxies Google's `safe` param.
-        return None if level == "off" else "active"
-    return None
-
-
-# ── SearXNG ──
-
-_NEWS_HINTS = ("news", "nyheter", "headlines", "breaking", "latest", "today", "idag")
-
-# The instance's DEFAULT general engines (google/duckduckgo/brave/startpage/
-# wikipedia) are routinely rate-limited / CAPTCHA-blocked and return nothing,
-# so a plain general query comes back empty. Pin engines that actually respond
-# (verified working on this instance) so non-news queries get results without
-# enabling any third-party API fallback. Override via the SEARXNG_GENERAL_ENGINES
-# env var if the working set changes.
-_GENERAL_ENGINES = os.environ.get("SEARXNG_GENERAL_ENGINES", "bing,mojeek,presearch")
-
-
-def searxng_search_api(query: str, count: int = 10, categories: str = "general",
-                       time_filter: Optional[str] = None) -> List[dict]:
-    """Search using SearXNG JSON API. Returns list of {title, url, snippet}."""
-    instance = _get_search_instance()
-    api_key = ""
-    headers = {"User-Agent": "Mozilla/5.0"}
-    if api_key:
-        headers["Authorization"] = f"Bearer {api_key}"
-    # News/fresh queries do badly in the 'general' category — it favours
-    # encyclopedic/tourism pages, ignores recency, and (with no language pin)
-    # bleeds in foreign-language results. When the agent layer detected
-    # freshness (time_filter) or the query reads like a news lookup, switch to
-    # the 'news' category, constrain recency, and pin language to English so a
-    # search like "Canada latest news" returns actual news instead of Wikipedia.
-    # Pin English for ALL searches — without it SearXNG mixes languages and
-    # brand-ambiguous terms bleed in foreign SEO pages (Honda "Odyssey" JP,
-    # Japanese "Trojan" malware blogs, Chinese math forums for "Polyphemus").
-    params = {"q": query, "format": "json", "language": "en",
-              "safesearch": _safesearch_for("searxng")}
-    q_lc = query.lower()
-    is_news = time_filter is not None or any(h in q_lc for h in _NEWS_HINTS)
-    if is_news and categories == "general":
-        params["categories"] = "news"
-        if time_filter in ("day", "week", "month", "year"):
-            # 'day' is too sparse on most SearXNG news engines — widen to a week
-            # so there's enough volume; the news category already biases recent.
-            params["time_range"] = "week" if time_filter in ("day", "week") else time_filter
-    else:
-        params["categories"] = categories
-        # Route general queries to engines that aren't blocked (the default
-        # general set returns 0 on this instance — see _GENERAL_ENGINES).
-        if categories == "general" and _GENERAL_ENGINES:
-            params["engines"] = _GENERAL_ENGINES
-    try:
-        def _parse_results(results):
-            return [
-                {
-                    "title": r.get("title", ""),
-                    "url": r.get("url", ""),
-                    "snippet": r.get("content", ""),
-                }
-                for r in results[:count]
-                if r.get("url")
-            ]
-
-        def _run(search_params):
-            response = httpx.get(
-                f"{instance}/search",
-                params=search_params,
-                headers=headers or None,
-                timeout=15,
-            )
-            response.raise_for_status()
-            data = response.json()
-            return _parse_results(data.get("results", [])), data
-
-        active_params = params
-        parsed, data = _run(active_params)
-        if not parsed and is_news and categories == "general":
-            # Some self-hosted SearXNG configs have no working news engines.
-            # Fall back to the known-good general engines before reporting an
-            # empty search, otherwise common queries like "Canada news" fail.
-            fallback = {
-                "q": query,
-                "format": "json",
-                "language": "en",
-                "categories": "general",
-                "safesearch": _safesearch_for("searxng"),
-            }
-            if _GENERAL_ENGINES:
-                fallback["engines"] = _GENERAL_ENGINES
-            logger.info(
-                "SearXNG news search returned 0 results for %r; retrying general engines",
-                query,
-            )
-            active_params = fallback
-            parsed, data = _run(active_params)
-        if not parsed and active_params.get("language"):
-            fallback = dict(active_params)
-            fallback.pop("language", None)
-            logger.info(
-                "SearXNG language-pinned search returned 0 results for %r; retrying without language",
-                query,
-            )
-            active_params = fallback
-            parsed, data = _run(active_params)
-        if not parsed and active_params.get("engines"):
-            fallback = dict(active_params)
-            fallback.pop("engines", None)
-            logger.info(
-                "SearXNG pinned engines returned 0 results for %r; retrying default engines",
-                query,
-            )
-            parsed, data = _run(fallback)
-        logger.info(f"SearXNG JSON API returned {len(parsed)} results for: {query}")
-        if not parsed:
-            unresponsive = data.get("unresponsive_engines") if isinstance(data, dict) else None
-            if unresponsive:
-                logger.info(f"SearXNG unresponsive engines for {query!r}: {unresponsive}")
-        return parsed
-    except Exception as e:
-        logger.warning(f"SearXNG JSON API search failed: {e}")
-        html_results = searxng_search(query, max_results=count)
-        if html_results:
-            logger.info(f"SearXNG HTML fallback returned {len(html_results)} results for: {query}")
-        return html_results
-
-
-def searxng_search(query, max_results=10):
-    """Search using SearXNG instance - parsing HTML."""
-    instance = _get_search_instance()
-    api_key = ""
-    req_headers = {"User-Agent": "Mozilla/5.0"}
-    if api_key:
-        req_headers["Authorization"] = f"Bearer {api_key}"
-    try:
-        response = httpx.get(
-            f"{instance}/search",
-            params={"q": query, "safesearch": _safesearch_for("searxng")},
-            headers=req_headers,
-            timeout=10,
-        )
-        if response.is_success:
-            soup = BeautifulSoup(response.text, "html.parser")
-            results = []
-            for article in soup.select("article.result")[:max_results]:
-                title_elem = article.select_one("h3 a")
-                if not title_elem:
-                    continue
-                title = title_elem.get_text(strip=True)
-                url = title_elem.get("href", "")
-                snippet_elem = article.select_one("p.content")
-                snippet = snippet_elem.get_text(strip=True) if snippet_elem else ""
-                results.append({"title": title, "url": url, "snippet": snippet})
-            logger.info(f"SearXNG search (HTML) returned {len(results)} results")
-            return results
-    except Exception as e:
-        logger.error(f"SearXNG search failed: {e}")
-    return []
-
-
-# ── Brave ──
-
-def brave_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
-    """Search using Brave API with key from admin settings or env var."""
-    api_key = _get_provider_key("brave") or os.environ.get("DATA_BRAVE_API_KEY") or ""
-    return _brave_search_impl(query, count, time_filter, search_config={"brave_api_key": api_key})
-
-
-def _brave_search_impl(query: str, count: int, time_filter: Optional[str] = None, search_config: dict = None) -> List[dict]:
-    """Core Brave API call. Returns a list of result dicts or an empty list on failure."""
-    enhanced_query = build_enhanced_query(query, time_filter)
-    config = search_config or {}
-
-    brave_api_key = config.get("brave_api_key")
-    if not brave_api_key:
-        brave_api_key = os.environ.get("DATA_BRAVE_API_KEY")
-
-    if not brave_api_key:
-        logger.warning("Brave API key not found, returning empty results for fallback")
-        return []
-
-    headers = {"X-Subscription-Token": brave_api_key, "Accept": "application/json"}
-    params = {"q": enhanced_query, "count": count,
-              "safesearch": _safesearch_for("brave")}
-    if time_filter:
-        time_map = {"day": "day", "week": "week", "month": "month", "year": "year"}
-        if time_filter in time_map:
-            params["freshness"] = time_map[time_filter]
-
-    logger.info(f"Executing Brave search with query: {enhanced_query}")
-    try:
-        response = httpx.get(
-            "https://api.search.brave.com/res/v1/web/search",
-            headers=headers,
-            params=params,
-            timeout=REQUEST_TIMEOUT,
-        )
-        if response.status_code == 429:
-            raise RateLimitError("Brave rate limit hit")
-        response.raise_for_status()
-    except httpx.RequestError as e:
-        error_logger.error(f"NetworkError during Brave search: {e}")
-        return []
-    except RateLimitError as e:
-        error_logger.error(str(e))
-        return []
-
-    try:
-        data = response.json()
-    except json.JSONDecodeError as e:
-        logger.error(f"Failed to parse Brave API response: {e}")
-        return []
-
-    results = []
-    if "web" in data and "results" in data["web"]:
-        for item in data["web"]["results"][:count]:
-            url = item.get("url", "")
-            if not url:
-                continue
-            results.append({
-                "title": item.get("title", ""),
-                "url": url,
-                "snippet": item.get("description", "") or item.get("content", ""),
-                "age": item.get("date", "") if item.get("date") else "",
-            })
-
-    logger.info(f"Brave search returned {len(results)} results")
-    return results
-
-
-# ── DuckDuckGo (free, no key) ──
-
-def _is_duckduckgo_host(host: str) -> bool:
-    """True only for duckduckgo.com and its subdomains — not substring look-alikes
-    such as ``duckduckgo.com.evil.com`` or ``notduckduckgo.com``."""
-    host = (host or "").lower()
-    return host == "duckduckgo.com" or host.endswith(".duckduckgo.com")
-
-
-def _resolve_ddg_redirect(raw: str) -> str:
-    """Resolve a DuckDuckGo /l/?uddg= redirect URL to its destination."""
-    if not raw:
-        return raw
-    # Handle protocol-relative URLs
-    resolved = raw
-    if resolved.startswith("//"):
-        resolved = "https:" + resolved
-    elif resolved.startswith("/"):
-        resolved = urljoin("https://html.duckduckgo.com", resolved)
-    # Extract the actual URL from DuckDuckGo's /l/?uddg= redirect
-    try:
-        parsed = urlparse(resolved)
-        if _is_duckduckgo_host(parsed.hostname) and parsed.path.rstrip("/") == "/l":
-            qs = parse_qs(parsed.query)
-            if "uddg" in qs:
-                return qs["uddg"][0]
-    except Exception:
-        pass
-    return resolved
-
-
-def duckduckgo_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
-    """Search using DuckDuckGo via the duckduckgo-search library. No API key needed."""
-    def _html_fallback() -> List[dict]:
-        try:
-            response = httpx.get(
-                "https://html.duckduckgo.com/html/",
-                params={"q": query, "kp": _safesearch_for("duckduckgo_html")},
-                headers={"User-Agent": "Mozilla/5.0"},
-                timeout=REQUEST_TIMEOUT,
-            )
-            response.raise_for_status()
-            soup = BeautifulSoup(response.text, "html.parser")
-            parsed = []
-            for result in soup.select(".result")[:count]:
-                link = result.select_one(".result__a")
-                if not link:
-                    continue
-                url = _resolve_ddg_redirect(link.get("href", ""))
-                if not url:
-                    continue
-                snippet_el = result.select_one(".result__snippet")
-                parsed.append({
-                    "title": link.get_text(" ", strip=True),
-                    "url": url,
-                    "snippet": snippet_el.get_text(" ", strip=True) if snippet_el else "",
-                })
-            logger.info(f"DuckDuckGo HTML search returned {len(parsed)} results")
-            return parsed
-        except Exception as e:
-            logger.warning(f"DuckDuckGo HTML search failed: {e}")
-            return []
-
-    try:
-        from duckduckgo_search import DDGS
-    except ImportError:
-        logger.warning("duckduckgo-search package not installed; using HTML fallback")
-        return _html_fallback()
-
-    timelimit = None
-    if time_filter:
-        time_map = {"day": "d", "week": "w", "month": "m", "year": "y"}
-        timelimit = time_map.get(time_filter)
-
-    try:
-        ddgs = DDGS()
-        raw = ddgs.text(query, max_results=count, timelimit=timelimit,
-                        safesearch=_safesearch_for("duckduckgo_lib"))
-        results = []
-        for item in raw:
-            url = item.get("href", "")
-            if not url:
-                continue
-            results.append({
-                "title": item.get("title", ""),
-                "url": url,
-                "snippet": item.get("body", ""),
-            })
-        logger.info(f"DuckDuckGo search returned {len(results)} results")
-        return results or _html_fallback()
-    except Exception as e:
-        logger.warning(f"DuckDuckGo search failed: {e}")
-        return _html_fallback()
-
-
-# ── Google Programmable Search Engine ──
-
-def google_pse_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
-    """Search using Google PSE (Custom Search JSON API).
-
-    Requires two keys in settings:
-      - search_api_key: Google API key
-      - google_pse_cx: Programmable Search Engine ID (cx)
-    Or env vars GOOGLE_API_KEY and GOOGLE_PSE_CX.
-    """
-    settings = _get_search_settings()
-    api_key = _get_provider_key("google_pse") or os.environ.get("GOOGLE_API_KEY", "")
-    cx = (settings.get("google_pse_cx") or "").strip() or os.environ.get("GOOGLE_PSE_CX", "")
-
-    if not api_key or not cx:
-        logger.warning("Google PSE: missing API key or CX ID")
-        return []
-
-    params = {
-        "key": api_key,
-        "cx": cx,
-        "q": query,
-        "num": min(count, 10),  # Google PSE max is 10 per request
-    }
-    safe = _safesearch_for("google_pse")
-    if safe:
-        params["safe"] = safe
-    if time_filter:
-        # dateRestrict: d[number], w[number], m[number], y[number]
-        time_map = {"day": "d1", "week": "w1", "month": "m1", "year": "y1"}
-        if time_filter in time_map:
-            params["dateRestrict"] = time_map[time_filter]
-
-    try:
-        response = httpx.get(
-            "https://www.googleapis.com/customsearch/v1",
-            params=params,
-            timeout=REQUEST_TIMEOUT,
-        )
-        if response.status_code == 429:
-            raise RateLimitError("Google PSE rate limit hit")
-        response.raise_for_status()
-        data = response.json()
-    except httpx.RequestError as e:
-        error_logger.error(f"Google PSE search failed: {e}")
-        return []
-    except RateLimitError as e:
-        error_logger.error(str(e))
-        return []
-
-    results = []
-    for item in data.get("items", [])[:count]:
-        url = item.get("link", "")
-        if not url:
-            continue
-        results.append({
-            "title": item.get("title", ""),
-            "url": url,
-            "snippet": item.get("snippet", ""),
-        })
-
-    logger.info(f"Google PSE returned {len(results)} results")
-    return results
-
-
-# ── Tavily ──
-
-def tavily_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
-    """Search using Tavily API. Requires search_api_key or TAVILY_API_KEY env var."""
-    api_key = _get_provider_key("tavily") or os.environ.get("TAVILY_API_KEY", "")
-    if not api_key:
-        logger.warning("Tavily: no API key configured")
-        return []
-
-    payload = {
-        "query": query,
-        "max_results": count,
-        "include_answer": False,
-    }
-    if time_filter:
-        time_map = {"day": "day", "week": "week", "month": "month", "year": "year"}
-        if time_filter in time_map:
-            payload["days"] = {"day": 1, "week": 7, "month": 30, "year": 365}[time_filter]
-
-    try:
-        response = httpx.post(
-            "https://api.tavily.com/search",
-            json=payload,
-            headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
-            timeout=REQUEST_TIMEOUT,
-        )
-        if response.status_code == 429:
-            raise RateLimitError("Tavily rate limit hit")
-        response.raise_for_status()
-        data = response.json()
-    except httpx.RequestError as e:
-        error_logger.error(f"Tavily search failed: {e}")
-        return []
-    except RateLimitError as e:
-        error_logger.error(str(e))
-        return []
-
-    results = []
-    for item in data.get("results", [])[:count]:
-        url = item.get("url", "")
-        if not url:
-            continue
-        results.append({
-            "title": item.get("title", ""),
-            "url": url,
-            "snippet": item.get("content", ""),
-            "age": item.get("published_date", ""),
-        })
-
-    logger.info(f"Tavily returned {len(results)} results")
-    return results
-
-
-# ── Serper.dev ──
-
-def serper_search(query: str, count: int = 10, time_filter: Optional[str] = None) -> List[dict]:
-    """Search using Serper.dev API. Requires search_api_key or SERPER_API_KEY env var."""
-    api_key = _get_provider_key("serper") or os.environ.get("SERPER_API_KEY", "")
-    if not api_key:
-        logger.warning("Serper: no API key configured")
-        return []
-
-    payload = {
-        "q": query,
-        "num": count,
-    }
-    safe = _safesearch_for("serper")
-    if safe:
-        payload["safe"] = safe
-    if time_filter:
-        time_map = {"day": "qdr:d", "week": "qdr:w", "month": "qdr:m", "year": "qdr:y"}
-        if time_filter in time_map:
-            payload["tbs"] = time_map[time_filter]
-
-    try:
-        response = httpx.post(
-            "https://google.serper.dev/search",
-            json=payload,
-            headers={"X-API-KEY": api_key, "Content-Type": "application/json"},
-            timeout=REQUEST_TIMEOUT,
-        )
-        if response.status_code == 429:
-            raise RateLimitError("Serper rate limit hit")
-        response.raise_for_status()
-        data = response.json()
-    except httpx.RequestError as e:
-        error_logger.error(f"Serper search failed: {e}")
-        return []
-    except RateLimitError as e:
-        error_logger.error(str(e))
-        return []
-
-    results = []
-    for item in data.get("organic", [])[:count]:
-        url = item.get("link", "")
-        if not url:
-            continue
-        results.append({
-            "title": item.get("title", ""),
-            "url": url,
-            "snippet": item.get("snippet", ""),
-            "age": item.get("date", ""),
-        })
-
-    logger.info(f"Serper returned {len(results)} results")
-    return results
+sys.modules[__name__] = _providers

src/search/query.py

@@ -15,7 +15,10 @@ def _detect_question_type(query: str) -> Optional[str]:
     """Return the leading question word if present (who, what, when, where, why, how)."""
     q = query.strip().lower()
     for word in ("who", "what", "when", "where", "why", "how"):
-        if q.startswith(word):
+        # Require a whole-word match: a bare prefix mis-flags ordinary queries
+        # like "whatsapp pricing" (-> what) or "however ..." (-> how), which
+        # then get spurious boost terms OR-appended in enhance_query.
+        if q == word or q.startswith(word + " "):
             return word
     return None
 

src/search/ranking.py

@@ -2,12 +2,49 @@
 
 import re
 import logging
-from datetime import datetime
+from datetime import datetime, timezone
 from typing import List, Optional
 from urllib.parse import urlparse
 
 logger = logging.getLogger(__name__)
 
+_AGE_FORMATS = ("%Y-%m-%d", "%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S")
+
+
+def _utcnow_naive() -> datetime:
+    """Naive UTC 'now'. Matches the naive, UTC-style published dates parsed below,
+    and is safe on Python 3.14 where ``datetime.utcnow()`` is removed (#1116)."""
+    return datetime.now(timezone.utc).replace(tzinfo=None)
+
+
+def recency_score(age_str: Optional[str], now: Optional[datetime] = None) -> float:
+    """Score how recent a result is: 1.0 for <=7 days old, 0.0 for >=30 days.
+
+    The age is measured against UTC, not local time. The previous code used
+    ``datetime.now()`` (local) against UTC-style published dates, so the age was
+    skewed by the host's UTC offset; it was also a latent crash once neighbouring
+    code moves to timezone-aware datetimes (#1116). ``now`` is injectable for tests.
+    """
+    if not age_str:
+        return 0.0
+    dt = None
+    for fmt in _AGE_FORMATS:
+        try:
+            dt = datetime.strptime(age_str, fmt)
+            break
+        except Exception:
+            dt = None
+    if not dt:
+        return 0.0
+    now = now or _utcnow_naive()
+    days_old = (now - dt).days
+    if days_old <= 7:
+        return 1.0
+    if days_old >= 30:
+        return 0.0
+    return (30 - days_old) / 23
+
+
 _NEWS_HINTS = {"news", "nyheter", "headlines", "breaking", "latest", "today", "idag"}
 _SPORTS_HINTS = {
     "sport", "sports", "soccer", "football", "hockey", "nba", "nfl", "mlb",
@@ -68,24 +105,6 @@ def rank_search_results(query: str, results: List[dict]) -> List[dict]:
             return 0.7
         return 0.4
 
-    def recency_score(age_str: Optional[str]) -> float:
-        if not age_str:
-            return 0.0
-        for fmt in ("%Y-%m-%d", "%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S"):
-            try:
-                dt = datetime.strptime(age_str, fmt)
-                break
-            except Exception:
-                dt = None
-        if not dt:
-            return 0.0
-        days_old = (datetime.now() - dt).days
-        if days_old <= 7:
-            return 1.0
-        if days_old >= 30:
-            return 0.0
-        return (30 - days_old) / 23
-
     def news_quality_adjustment(title: str, snippet: str, url: str) -> float:
         if not is_news_query:
             return 0.0

src/settings.py

@@ -89,11 +89,19 @@ DEFAULT_SETTINGS = {
     # Hard wall-clock cap on a single deep-research run. The previous 600s
     # (10 min) default cut off slow local / edge LLMs mid-synthesis; 1800s
     # (30 min) is comfortable for most local setups while still bounding
-    # runaway jobs. Tune via Settings or by editing data/settings.json.
+    # runaway jobs. Set to 0 to disable the cap entirely (unlimited) — only
+    # for very long deep-research runs, since a stalled job then runs an
+    # unbounded model/API bill. Other values are bounded to [60, 86400].
+    # Tune via Settings or by editing data/settings.json.
     "research_run_timeout_seconds": 1800,
     "agent_max_tool_calls": 0,
     "agent_input_token_budget": 6000,
     "agent_stream_timeout_seconds": 300,
+    # Extra directory roots that read_file / write_file may access, in
+    # addition to the built-in project data/ and system temp dirs. Each
+    # entry is an absolute path. Sensitive subpaths (.ssh, .gnupg, shell
+    # rc files, SSH key files) are always blocked regardless of roots.
+    "tool_path_extra_roots": [],
     "task_endpoint_id": "",
     "task_model": "",
     "default_endpoint_id": "",
@@ -187,6 +195,21 @@ def get_setting(key: str, default: Any = None) -> Any:
     return load_settings().get(key, default)
 
 
+def is_setting_overridden(key: str) -> bool:
+    """True if ``key`` is explicitly present in the saved settings file.
+
+    ``load_settings`` merges DEFAULT_SETTINGS with the saved file, so a value
+    equal to its default is indistinguishable from "never set" via get_setting.
+    Callers that need to treat an explicit user choice differently from the
+    default (e.g. adaptive budgets) use this to read the raw saved file.
+    """
+    try:
+        with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
+            return key in json.load(f)
+    except (FileNotFoundError, json.JSONDecodeError):
+        return False
+
+
 # Per-user settings (user prefs override the global admin default). Used for
 # keys that a user is allowed to choose individually — currently the vision
 # model + image-generation model. The owner argument is the authed username

src/task_scheduler.py

@@ -192,7 +192,7 @@ HOUSEKEEPING_DEFAULTS = {
     "draft_email_replies":  {"name": "Email AI Auto Reply",      "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 */2 * * *", "ship_paused": True, "legacy_names": ["Tidy Email (Replies)", "AI Auto Reply"]},
     "extract_email_events": {"name": "Email Calendar Events",    "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 */1 * * *", "ship_paused": True, "legacy_names": ["Email → Calendar Events"]},
     "classify_events":      {"name": "Calendar Classify Events", "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 6,18 * * *", "ship_paused": True, "legacy_names": ["Classify Calendar Events"]},
-    "mark_email_boundaries": {"name": "Email Mark Boundaries",   "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 */2 * * *", "legacy_names": ["Mark Email Boundaries"]},
+    "mark_email_boundaries": {"name": "Email Mark Boundaries",   "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 */2 * * *", "ship_paused": True, "legacy_names": ["Mark Email Boundaries"]},
     "check_email_urgency":   {"name": "Email Tags",               "schedule": "cron",  "scheduled_time": None,    "cron_expression": "0 * * * *", "ship_paused": True, "old_cron_expressions": ["*/15 * * * *"], "legacy_names": ["Email Triage", "Urgent Email"]},
     "audit_skills":          {"name": "Skills Audit",             "trigger_type": "event", "trigger_event": "skill_added", "trigger_count": 5, "schedule": None, "scheduled_time": None, "cron_expression": None, "legacy_names": ["Audit Skills"]},
 }
@@ -203,6 +203,20 @@ RETIRED_HOUSEKEEPING_ACTIONS = frozenset({
 })
 
 
+def _digest_windows(now):
+    """(label, start, end) buckets for the calendar check-in digest.
+
+    The windows are contiguous so no event is dropped between buckets — an
+    earlier version started the 30-day window at now+8d while the week window
+    ended at now+7d, so events ~7-8 days out fell into no bucket.
+    """
+    return [
+        ("today_tomorrow", now, now + timedelta(days=2)),
+        ("this_week", now + timedelta(days=2), now + timedelta(days=7)),
+        ("next_30_days", now + timedelta(days=7), now + timedelta(days=30)),
+    ]
+
+
 class TaskScheduler:
     def __init__(self, session_manager):
         self._session_manager = session_manager
@@ -241,6 +255,35 @@ class TaskScheduler:
         except Exception:
             logger.debug("Task progress update failed", exc_info=True)
 
+    def _mark_run_aborted(self, task_id: str, run_id: str | None = None, message: str = "Stopped by user") -> bool:
+        """Mark an active run as aborted. Used by stop/cancel paths."""
+        try:
+            from core.database import SessionLocal, TaskRun
+            db = SessionLocal()
+            try:
+                q = db.query(TaskRun)
+                if run_id:
+                    q = q.filter(TaskRun.id == run_id)
+                else:
+                    q = q.filter(
+                        TaskRun.task_id == task_id,
+                        TaskRun.status.in_(("queued", "running")),
+                    ).order_by(TaskRun.started_at.desc())
+                run = q.first()
+                if not run or run.status not in ("queued", "running"):
+                    return False
+                run.status = "aborted"
+                run.error = message
+                run.result = run.result or message
+                run.finished_at = datetime.utcnow()
+                db.commit()
+                return True
+            finally:
+                db.close()
+        except Exception:
+            logger.debug("Task abort marker failed for %s", task_id, exc_info=True)
+            return False
+
     def add_notification(self, task_name: str, status: str, task_id: str = None, owner: str = None, body: str = None):
         """Store a notification about a completed task run. Tagged with the
         task's owner so `pop_notifications` can return only that user's
@@ -581,12 +624,25 @@ class TaskScheduler:
         finally:
             _q_db.close()
 
-        if bypass_model_slot or not self._task_needs_model_slot(task_id):
-            await self._execute_task_locked(task_id, run_id, release_executing=release_executing)
-            return
+        try:
+            if bypass_model_slot or not self._task_needs_model_slot(task_id):
+                await self._execute_task_locked(task_id, run_id, release_executing=release_executing)
+                return
 
-        async with self._run_semaphore:
-            await self._execute_task_locked(task_id, run_id, release_executing=release_executing)
+            async with self._run_semaphore:
+                await self._execute_task_locked(task_id, run_id, release_executing=release_executing)
+        except asyncio.CancelledError:
+            # If cancellation happens while queued behind the semaphore,
+            # _execute_task_locked never runs and cannot update the Activity row.
+            self._mark_run_aborted(task_id, run_id)
+            raise
+        finally:
+            handle = self._task_handles.get(task_id)
+            if handle is current:
+                self._task_handles.pop(task_id, None)
+            if release_executing:
+                async with self._executing_lock:
+                    self._executing.discard(task_id)
 
     async def _execute_task_locked(self, task_id: str, run_id: str, *, release_executing: bool = True):
         from core.database import SessionLocal, ScheduledTask, TaskRun
@@ -1040,11 +1096,7 @@ class TaskScheduler:
             from core.database import SessionLocal as _SL, CalendarEvent as _CE
             _db = _SL()
             try:
-                for label, start, end in [
-                    ("today_tomorrow", now, now + timedelta(days=2)),
-                    ("this_week",      now + timedelta(days=2), now + timedelta(days=7)),
-                    ("next_30_days",   now + timedelta(days=8), now + timedelta(days=30)),
-                ]:
+                for label, start, end in _digest_windows(now):
                     # Strip timezone for naive DB comparison
                     _s = start.replace(tzinfo=None) if start.tzinfo else start
                     _e = end.replace(tzinfo=None) if end.tzinfo else end
@@ -1839,24 +1891,7 @@ class TaskScheduler:
                 self._executing.discard(task_id)
                 stopped = True
 
-        from core.database import SessionLocal, TaskRun
-        db = SessionLocal()
-        try:
-            run = (
-                db.query(TaskRun)
-                .filter(TaskRun.task_id == task_id, TaskRun.status.in_(("queued", "running")))
-                .order_by(TaskRun.started_at.desc())
-                .first()
-            )
-            if run:
-                run.status = "aborted"
-                run.error = "Stopped by user"
-                run.result = run.result or "Stopped by user"
-                run.finished_at = datetime.utcnow()
-                db.commit()
-                stopped = True
-        finally:
-            db.close()
+        stopped = self._mark_run_aborted(task_id) or stopped
         return stopped
 
     async def ensure_defaults(self, owner: str):

src/teacher_escalation.py

@@ -42,7 +42,7 @@ _SOTA_HOSTS = frozenset({
     "api.together.xyz", "api.fireworks.ai",
     "api.perplexity.ai", "api.x.ai",
     "generativelanguage.googleapis.com", "api.groq.com",
-    "openrouter.ai", "ollama.com",
+    "openrouter.ai", "ollama.com", "api.venice.ai",
 })
 
 

src/text_helpers.py

@@ -20,9 +20,9 @@ import re
 _THINK_CLOSED_RE = re.compile(r"<think(?:ing)?>[\s\S]*?</think(?:ing)?>\s*", re.IGNORECASE)
 # Orphan opening or closing tags that survive after the closed-pass.
 _THINK_TAG_RE = re.compile(r"</?think(?:ing)?[^>]*>\s*", re.IGNORECASE)
-# Dangling opener at the top of the response with no closer — strip everything
-# from `<think>` up to either `</think>` (if it ever shows) or end of string.
-_THINK_OPEN_RE = re.compile(r"^\s*<think(?:ing)?>.*?(?:</think(?:ing)?>|$)", re.DOTALL | re.IGNORECASE)
+# Dangling opener anywhere in the response with no closer — strip everything
+# from `<think>` to the end of string.
+_THINK_OPEN_RE = re.compile(r"<think(?:ing)?>[\s\S]*$", re.IGNORECASE)
 # Streaming models occasionally emit `<thinking time="0.42">`-style attributes.
 # Normalize to a plain `<think>` so the regexes above catch them.
 _THINK_ATTR_RE = re.compile(r"<think(?:ing)?\s+[^>]*>", re.IGNORECASE)

src/tool_execution.py

@@ -21,6 +21,143 @@ from src.tool_security import is_public_blocked_tool, owner_is_admin_or_single_u
 MAX_OUTPUT_CHARS = 10_000
 MAX_READ_CHARS = 20_000
 
+# ---------------------------------------------------------------------------
+# Path confinement for read_file / write_file
+# ---------------------------------------------------------------------------
+# read_file + write_file are admin-only tools, but the path the agent
+# supplies is model-controlled. Prompt-injection in an admin's chat can
+# weaponise "read /etc/shadow" or "write ~/.ssh/authorized_keys" without
+# the admin noticing.
+#
+# Policy:
+#   1. Sensitive-subpath deny list — checked FIRST. Blocks .ssh,
+#      .gnupg, shell rc files, token/env files even if the root above
+#      them is on the allowlist.
+#   2. Allowlist — only the directories the agent legitimately needs
+#      (project data/, system tmp). $HOME is NOT on the default list.
+#   3. Opt-in extra roots — admin can add broader roots via the
+#      "tool_path_extra_roots" setting (list of path strings).
+# ---------------------------------------------------------------------------
+
+_SENSITIVE_BASENAMES: set[str] = {
+    ".ssh", ".gnupg", ".gitconfig",
+    ".bashrc", ".bash_profile", ".bash_logout",
+    ".zshrc", ".zprofile", ".zshenv",
+    ".profile", ".tcshrc", ".cshrc",
+    ".env", ".netrc",
+}
+
+_SENSITIVE_FILE_PATTERNS: tuple[str, ...] = (
+    "authorized_keys", "id_rsa", "id_ed25519", "id_ecdsa",
+    "known_hosts",
+)
+
+
+def _is_sensitive_path(resolved: str) -> bool:
+    """Return True if *resolved* falls under a sensitive directory or
+    matches a sensitive filename — regardless of what root it sits under.
+    """
+    parts = resolved.split(os.sep)
+    filenames: set[str] = {parts[-1]} if parts else set()
+
+    # Check if any path component is a sensitive directory.
+    for part in parts:
+        if part in _SENSITIVE_BASENAMES:
+            return True
+
+    # Check filename against known sensitive files.
+    for pat in _SENSITIVE_FILE_PATTERNS:
+        if pat in filenames:
+            return True
+
+    return False
+
+
+def _tool_path_roots() -> list[str]:
+    """Return the list of directory roots that read_file / write_file
+    may touch. Default: project data/ + system temp dirs. Extra roots
+    are loaded from the ``tool_path_extra_roots`` setting.
+    """
+    roots: list[str] = []
+
+    # Project data directory — the agent's primary workspace.
+    from src.constants import DATA_DIR
+    roots.append(DATA_DIR)
+
+    # /tmp (and its macOS realpath /private/tmp).
+    roots.append("/tmp")
+    try:
+        private_tmp = os.path.realpath("/tmp")
+        if private_tmp != "/tmp":
+            roots.append(private_tmp)
+    except OSError:
+        pass
+
+    # $TMPDIR — per-user temp root on macOS (e.g. /var/folders/.../T/).
+    tmpdir = os.environ.get("TMPDIR")
+    if tmpdir:
+        roots.append(tmpdir)
+
+    # Opt-in extra roots from settings.
+    try:
+        from src.settings import get_setting
+        extra = get_setting("tool_path_extra_roots")
+        if isinstance(extra, list):
+            roots.extend(str(r) for r in extra if r)
+    except Exception:
+        pass
+
+    # Deduplicate; resolve symlinks so containment is unambiguous.
+    seen: set[str] = set()
+    out: list[str] = []
+    for r in roots:
+        try:
+            real = os.path.realpath(r)
+        except OSError:
+            continue
+        if real in seen:
+            continue
+        seen.add(real)
+        out.append(real)
+    return out
+
+
+def _resolve_tool_path(raw_path: str) -> str:
+    """Resolve and confine a model-supplied path.
+
+    Order of checks:
+      1. Non-empty path.
+      2. Sensitive-subpath deny list (blocks .ssh, .gnupg, etc.
+         even when the root is on the allowlist).
+      3. Allowlist containment (must land under one of the roots).
+
+    Returns the realpath on success. Raises ValueError on rejection.
+    Symlinks are resolved before comparison.
+    """
+    if raw_path is None or not str(raw_path).strip():
+        raise ValueError("path is required")
+    expanded = os.path.expanduser(str(raw_path).strip())
+    resolved = os.path.realpath(expanded)
+
+    if _is_sensitive_path(resolved):
+        raise ValueError(
+            f"path '{raw_path}' is inside a sensitive directory "
+            f"(e.g. .ssh, .gnupg) or matches a sensitive filename"
+        )
+
+    for root in _tool_path_roots():
+        if resolved == root:
+            return resolved
+        try:
+            common = os.path.commonpath([resolved, root])
+        except ValueError:
+            continue
+        if common == root:
+            return resolved
+    raise ValueError(
+        f"path '{raw_path}' is outside the allowed roots"
+    )
+
 # Bash + python tools used to share a single 60s timeout. That's
 # enough for one-shot commands but starves real workloads (pip
 # install, ffmpeg conversions, etc.) — and worse, the agent saw the
@@ -168,6 +305,7 @@ async def _run_subprocess_streaming(
     )
 
 _ADMIN_TOOLS = {
+    "app_api",
     "manage_endpoints",
     "manage_mcp",
     "manage_webhooks",
@@ -175,6 +313,7 @@ _ADMIN_TOOLS = {
     "manage_settings",
     "download_model",
     "serve_model",
+    "serve_preset",
     "stop_served_model",
     "cancel_download",
 }
@@ -373,9 +512,11 @@ async def _direct_fallback(
             return {"output": output or "(no output)", "exit_code": rc or 0}
 
         if tool == "read_file":
-            path = os.path.expanduser(content.split("\n", 1)[0].strip())
-            if not path:
-                return {"error": "read_file: path required", "exit_code": 1}
+            raw_path = content.split("\n", 1)[0].strip()
+            try:
+                path = _resolve_tool_path(raw_path)
+            except ValueError as e:
+                return {"error": f"read_file: {e}", "exit_code": 1}
             try:
                 # Run blocking read in a thread to keep the loop responsive
                 def _read():
@@ -395,13 +536,14 @@ async def _direct_fallback(
 
         if tool == "write_file":
             lines = content.split("\n", 1)
-            path = os.path.expanduser(lines[0].strip())
+            raw_path = lines[0].strip()
             body = lines[1] if len(lines) > 1 else ""
-            if not path:
-                return {"error": "write_file: path required", "exit_code": 1}
+            try:
+                path = _resolve_tool_path(raw_path)
+            except ValueError as e:
+                return {"error": f"write_file: {e}", "exit_code": 1}
             try:
                 def _write():
-                    import os
                     d = os.path.dirname(path)
                     if d:
                         os.makedirs(d, exist_ok=True)

src/tool_implementations.py

@@ -90,7 +90,11 @@ def get_active_document():
 
 def _owned_document_query(query, Document, owner: Optional[str]):
     if owner is None:
-        return query.filter(False)
+        # A bare Python `False` is not a valid SQL expression — SQLAlchemy 1.4
+        # deprecates it and 2.0 raises ArgumentError. Use the SQL `false()`
+        # literal to return zero rows for an unscoped (owner-less) query.
+        from sqlalchemy import false
+        return query.filter(false())
     return query.filter(Document.owner == owner)
 
 
@@ -1921,9 +1925,22 @@ async def do_manage_notes(content: str, owner: Optional[str] = None) -> Dict:
                 return {"error": f"Note '{note_id}' not found", "exit_code": 1}
             if owner is not None and note.owner and note.owner != owner:
                 return {"error": "Note not found", "exit_code": 1}
-            for field in ("title", "content", "note_type", "color", "label", "due_date"):
+            for field in ("title", "content", "note_type", "color", "label"):
                 if field in args and args[field] is not None:
                     setattr(note, field, args[field])
+            # Parse due_date the same way the `add` action does. The schema
+            # advertises natural language ("tomorrow at 9am"), and naive ISO
+            # strings need the user's tz offset attached so the frontend's
+            # `new Date()` resolves the right absolute moment. Storing the raw
+            # value here left updated reminders as unparseable literals that
+            # never fired.
+            if args.get("due_date") is not None:
+                due_raw = args["due_date"]
+                try:
+                    from routes.calendar_routes import parse_due_for_user as _pdt_user
+                    note.due_date = _pdt_user(due_raw)
+                except Exception:
+                    note.due_date = due_raw  # fall through; trust the model
             new_items = args.get("checklist_items")
             if new_items is None:
                 new_items = args.get("items")

src/tool_index.py

@@ -65,7 +65,7 @@ BUILTIN_TOOL_DESCRIPTIONS: Dict[str, str] = {
     "web_fetch": "Fetch and read the text content of a specific URL/website the user names (e.g. 'check example.com', 'open this link'). Use when you have a concrete URL; for open-ended lookups use web_search instead.",
     "read_file": "Read a file from disk and return its contents. View source code, config files, logs.",
     "write_file": "Write content to a file on disk. Create new files, save output, update configs.",
-    "create_document": "Create a new document in the editor panel. For code, articles, text content longer than 15 lines. Specify title, language, and content.",
+    "create_document": "Create a new document in the editor panel. For code, articles, text content longer than 15 lines, unless an already-open document/email draft is the obvious target. If an email compose draft is open, edit that draft instead of creating another document.",
     "edit_document": "Preferred tool for editing an existing document — targeted find-and-replace. Use for any small change: add a function, fix a bug, tweak a section, rename things.",
     "update_document": "Replace the entire active document content. ONLY for full rewrites (>50% changed). Do not use for small edits — use edit_document instead.",
     "suggest_document": "Suggest changes to the active document with explanations. For code review, proofreading, feedback requests.",
@@ -431,10 +431,14 @@ class ToolIndex:
         base = set(always_include or ALWAYS_AVAILABLE)
         retrieved = self.retrieve(query, k=k)
         base.update(retrieved)
-        # Keyword-based force-include for common intents
+        # Keyword-based force-include for common intents. Match on word
+        # boundaries, not raw substrings, so short hints like "fix", "line",
+        # "serve", "reply" or "unread" don't fire inside unrelated words
+        # ("prefix", "deadline"/"online", "observe"/"reserve", "replying",
+        # "unreadable"). Same word-boundary matching used in topic_analyzer.
         ql = query.lower()
         for keywords, tools in self._KEYWORD_HINTS.items():
-            if any(kw in ql for kw in keywords):
+            if any(re.search(rf"\b{re.escape(kw)}\b", ql) for kw in keywords):
                 base.update(tools)
         # Structural scheduling-intent detection — typo-resilient (the literal
         # keyword "every day" misses "every dya"). Catches "every <word>",

src/tool_schemas.py

@@ -111,7 +111,7 @@ FUNCTION_TOOL_SCHEMAS = [
         "type": "function",
         "function": {
             "name": "create_document",
-            "description": "Create a new document in the editor panel. ALWAYS use this when the user asks to write, create, build, or generate code, scripts, programs, games, apps, or any substantial content (>15 lines). NEVER put large code blocks directly in chat — use this tool instead.",
+            "description": "Create a new document in the editor panel. Use this when the user asks to write, create, build, or generate code, scripts, programs, games, apps, or any substantial content (>15 lines) AND there is no already-open document/email draft that the request refers to. If an email compose draft is open, edit that draft instead of creating another document. NEVER put large code blocks directly in chat — use this tool instead.",
             "parameters": {
                 "type": "object",
                 "properties": {

src/tool_security.py

@@ -40,6 +40,7 @@ NON_ADMIN_BLOCKED_TOOLS = {
     "vault_unlock",
     "download_model",
     "serve_model",
+    "serve_preset",
     "stop_served_model",
     "cancel_download",
     "adopt_served_model",

src/topic_analyzer.py

@@ -49,7 +49,15 @@ def analyze_topics(session_manager, owner: str = None) -> Dict[str, Any]:
         if sess_owner != owner:
             continue
 
-        for msg in session_data.get("history", []):
+        # Hydrate session to load history from DB if needed
+        if hasattr(session_manager, "get_session"):
+            hydrated_session = session_manager.get_session(session_id)
+            history = hydrated_session.history
+        else:
+            hydrated_session = session_data
+            history = session_data.get("history", [])
+
+        for msg in history:
             content_raw = msg.get("content") if isinstance(msg, dict) else getattr(msg, "content", None)
             if not content_raw:
                 continue

src/upload_handler.py

@@ -6,6 +6,8 @@ import uuid
 import time
 import hashlib
 import mimetypes
+import shutil
+import tempfile
 import threading
 from datetime import datetime, timedelta
 from typing import Dict, Any, Optional
@@ -29,7 +31,11 @@ import logging
 
 logger = logging.getLogger(__name__)
 
-UPLOAD_ID_RE = re.compile(r"^[0-9a-fA-F]{32}\.[A-Za-z0-9]+$")
+# The extension is optional: save_upload builds the id as `{uuid.hex}{ext}`,
+# and a file with no extension (Dockerfile, README, ...) yields a bare 32-hex
+# id. Requiring `.ext` made those ids fail validation, so the stored file
+# could never be resolved or downloaded again.
+UPLOAD_ID_RE = re.compile(r"^[0-9a-fA-F]{32}(?:\.[A-Za-z0-9]+)?$")
 
 
 def is_valid_upload_id(upload_id: str) -> bool:
@@ -52,6 +58,13 @@ class UploadHandler:
         self._upload_rate_lock = threading.Lock()
         self._upload_rate_counter = 0
         self._upload_rate_max_entries = 1000
+        # Serialise the read-modify-write of uploads.json within one
+        # Python process. Scope: single FastAPI worker (the default
+        # uvicorn deployment). Cross-process / multi-worker deployments
+        # need an additional file-level lock (flock) or a database;
+        # the atomic-rename write below keeps on-disk state consistent
+        # on its own but does not serialise writers across processes.
+        self._index_lock = threading.Lock()
         
         # Create upload directory
         os.makedirs(self.upload_dir, exist_ok=True)
@@ -247,17 +260,52 @@ class UploadHandler:
         except Exception:
             return False
 
+    def _atomic_write_json(self, path: str, data: dict) -> None:
+        """Write `data` to `path` atomically: write to a temp file in the
+        same directory, then `os.replace` onto the target. The kernel
+        guarantees `os.replace` is atomic on POSIX, so a reader either
+        sees the old contents or the new contents, never a half-written
+        file. Also keeps a `.bak` sibling of the previous good state.
+        """
+        directory = os.path.dirname(path) or "."
+        fd, tmp = tempfile.mkstemp(prefix=".uploads-", suffix=".tmp", dir=directory)
+        try:
+            with os.fdopen(fd, "w", encoding="utf-8") as f:
+                json.dump(data, f, indent=2)
+                f.flush()
+                os.fsync(f.fileno())
+            if os.path.exists(path):
+                bak = path + ".bak"
+                try:
+                    shutil.copy2(path, bak)
+                except OSError:
+                    pass
+            os.replace(tmp, path)
+        except Exception:
+            try:
+                os.unlink(tmp)
+            except OSError:
+                pass
+            raise
+
     def _load_upload_index(self) -> Dict[str, Any]:
         uploads_db_path = os.path.join(self.upload_dir, "uploads.json")
         if not os.path.exists(uploads_db_path):
             return {}
-        try:
-            with open(uploads_db_path, "r") as f:
-                data = json.load(f)
-            return data if isinstance(data, dict) else {}
-        except Exception as e:
-            logger.warning(f"Failed to read uploads database: {e}")
-            return {}
+        # Try the live file first, fall back to the .bak sibling if the
+        # live file is truncated/corrupted (e.g. a previous writer was
+        # SIGKILL'd mid-rename before the new code path was deployed).
+        for candidate in (uploads_db_path, uploads_db_path + ".bak"):
+            if not os.path.exists(candidate):
+                continue
+            try:
+                with open(candidate, "r", encoding="utf-8") as f:
+                    data = json.load(f)
+                return data if isinstance(data, dict) else {}
+            except Exception as e:
+                logger.warning(f"Failed to read uploads database ({candidate}): {e}")
+                continue
+        return {}
 
     def get_upload_info(self, upload_id: str) -> Optional[Dict[str, Any]]:
         """Return the uploads.json metadata row for an upload ID, if present."""
@@ -458,52 +506,76 @@ class UploadHandler:
         # Calculate file hash for deduplication
         file_hash = self.calculate_file_hash(file_obj)
         
-        # Check for duplicate files
+        # Check for duplicate files.
+        # The duplicate-detection lookup AND the write must both happen
+        # under _index_lock: a duplicate upload racing with a new-entry
+        # insert must not overwrite a newer snapshot of the index with
+        # the stale one read before the insert.
         uploads_db_path = os.path.join(self.upload_dir, "uploads.json")
-        existing_files = {}
-        
-        if os.path.exists(uploads_db_path):
-            try:
-                with open(uploads_db_path, "r", encoding="utf-8") as f:
-                    existing_files = json.load(f)
-            except Exception as e:
-                logger.warning(f"Failed to read uploads database: {e}")
-        
-        # Check if this hash already exists for the same owner. Uploads are
-        # access-controlled by owner, so cross-user dedupe must not return a
-        # shared file ID.
-        existing_key = None
         existing_file = None
-        for key, info in existing_files.items():
-            if info.get("hash") == file_hash and info.get("owner") == owner:
-                existing_key = key
-                existing_file = info
-                break
+        existing_key = None
+        with self._index_lock:
+            existing_files = self._load_upload_index()
+            stale_keys = []
+            for key, info in existing_files.items():
+                if info.get("hash") == file_hash and info.get("owner") == owner:
+                    stored_path = info.get("path")
+                    if stored_path and os.path.exists(stored_path) and self._inside_upload_dir(stored_path):
+                        existing_key = key
+                        existing_file = info
+                        break
+                    stale_keys.append(key)
+            if stale_keys:
+                for key in stale_keys:
+                    existing_files.pop(key, None)
+                try:
+                    self._atomic_write_json(uploads_db_path, existing_files)
+                    logger.info("Removed %d stale upload index entries for missing duplicates", len(stale_keys))
+                except Exception as e:
+                    logger.warning(f"Failed to remove stale upload index entries: {e}")
         if existing_file:
             logger.info(f"Duplicate file upload detected: {original_filename} -> {existing_file['id']}")
-            
+
             existing_file["last_accessed"] = datetime.now().isoformat()
-            existing_files[existing_key] = existing_file
-            
-            try:
-                with open(uploads_db_path, "w", encoding="utf-8") as f:
-                    json.dump(existing_files, f, indent=2)
-            except Exception as e:
-                logger.warning(f"Failed to update uploads database: {e}")
-            
-            return {
-                "id": existing_file["id"],
-                "path": existing_file["path"],
-                "mime": existing_file["mime"],
-                "size": existing_file["size"],
-                "name": existing_file["original_name"],
-                "hash": file_hash,
-                "uploaded_at": existing_file["uploaded_at"],
-                "owner": existing_file.get("owner"),
-                "width": existing_file.get("width"),
-                "height": existing_file.get("height"),
-                "is_duplicate": True
-            }
+            with self._index_lock:
+                try:
+                    current = self._load_upload_index()
+                    # Re-resolve the key inside the lock: a concurrent
+                    # insert can have changed the dict's keys.
+                    live_key = existing_key
+                    if live_key not in current:
+                        for k, v in current.items():
+                            if v.get("hash") == file_hash and v.get("owner") == owner:
+                                live_key = k
+                                existing_file = v
+                                break
+                    if live_key is None:
+                        # No matching entry anymore (e.g. cleaned up between
+                        # the outer read and the write). Fall through to the
+                        # fresh-insert path below; release the lock first.
+                        raise LookupError("upload entry vanished mid-dedupe")
+                    existing_file["last_accessed"] = datetime.now().isoformat()
+                    current[live_key] = existing_file
+                    self._atomic_write_json(uploads_db_path, current)
+                except LookupError:
+                    existing_file = None
+                except Exception as e:
+                    logger.warning(f"Failed to update uploads database: {e}")
+
+            if existing_file:
+                return {
+                    "id": existing_file["id"],
+                    "path": existing_file["path"],
+                    "mime": existing_file["mime"],
+                    "size": existing_file["size"],
+                    "name": existing_file["original_name"],
+                    "hash": file_hash,
+                    "uploaded_at": existing_file["uploaded_at"],
+                    "owner": existing_file.get("owner"),
+                    "width": existing_file.get("width"),
+                    "height": existing_file.get("height"),
+                    "is_duplicate": True
+                }
         
         # Generate unique ID and determine save location
         _, ext = os.path.splitext(safe_filename)
@@ -548,24 +620,14 @@ class UploadHandler:
                 logger.warning(f"Failed to read image dimensions for {file_id}: {e}")
         
         # Update uploads database
-        try:
-            if os.path.exists(uploads_db_path):
-                try:
-                    with open(uploads_db_path, "r", encoding="utf-8") as f:
-                        all_files = json.load(f)
-                except Exception:
-                    all_files = {}
-            else:
-                all_files = {}
-            
-            storage_key = f"{owner}:{file_hash}" if owner else file_hash
-            all_files[storage_key] = file_metadata
-            
-            with open(uploads_db_path, "w", encoding="utf-8") as f:
-                json.dump(all_files, f, indent=2)
-                
-        except Exception as e:
-            logger.warning(f"Failed to update uploads database: {e}")
+        with self._index_lock:
+            try:
+                current = self._load_upload_index() if os.path.exists(uploads_db_path) else {}
+                storage_key = f"{owner}:{file_hash}" if owner else file_hash
+                current[storage_key] = file_metadata
+                self._atomic_write_json(uploads_db_path, current)
+            except Exception as e:
+                logger.warning(f"Failed to update uploads database: {e}")
         
         logger.info(f"File uploaded successfully: {original_filename} ({file_size} bytes)")
         return file_metadata

src/url_safety.py

@@ -0,0 +1,88 @@
+"""Outbound URL safety checks (SSRF hardening).
+
+Run before the server makes a request to a *user-supplied* URL — e.g. the custom
+embedding endpoint set via ``POST /api/embeddings/endpoint``, which then triggers
+an outbound ``httpx`` call.
+
+Odysseus is local-first: pointing the embedding endpoint at a loopback or LAN
+address (a local vLLM / llama.cpp / Ollama server) is a normal, intended setup.
+So this guard does **not** blanket-block private addresses by default — that would
+break the primary use case. What it *always* rejects:
+
+  - a non-HTTP(S) scheme (``file://``, ``gopher://``, ``ftp://`` …), and
+  - the link-local range (``169.254.0.0/16`` / ``fe80::/10``), i.e. the cloud
+    instance-metadata SSRF credential-exfil vector — nobody serves embeddings
+    there — plus multicast / reserved / unspecified addresses.
+
+For exposed multi-tenant deployments, set ``EMBEDDING_BLOCK_PRIVATE_IPS=true`` to
+additionally reject all private and loopback targets (full SSRF lockdown).
+"""
+
+import ipaddress
+import socket
+from typing import Callable, List, Optional, Tuple
+from urllib.parse import urlparse
+
+ALLOWED_SCHEMES = ("http", "https")
+
+
+def _default_resolver(host: str) -> List[str]:
+    """Resolve a hostname to the list of IP strings it maps to (A + AAAA)."""
+    return [info[4][0] for info in socket.getaddrinfo(host, None)]
+
+
+def _classify(ip: ipaddress._BaseAddress, *, block_private: bool) -> Optional[str]:
+    """Return a rejection reason for an IP, or None if it is allowed."""
+    # IPv4-mapped IPv6 (e.g. ::ffff:169.254.169.254) — judge the embedded v4.
+    if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped is not None:
+        ip = ip.ipv4_mapped
+    if ip.is_link_local:
+        return f"link-local address blocked (SSRF metadata risk): {ip}"
+    if ip.is_multicast or ip.is_reserved or ip.is_unspecified:
+        return f"disallowed address: {ip}"
+    if block_private and (ip.is_private or ip.is_loopback):
+        return f"private/loopback address blocked: {ip}"
+    return None
+
+
+def check_outbound_url(
+    url: str,
+    *,
+    block_private: bool = False,
+    resolver: Optional[Callable[[str], List[str]]] = None,
+) -> Tuple[bool, str]:
+    """Validate a user-supplied outbound URL.
+
+    Returns ``(ok, reason)``. ``ok`` is True only when the URL is safe to fetch.
+    ``resolver`` is injectable so callers/tests can avoid real DNS.
+    """
+    if not url or not url.strip():
+        return False, "URL is required"
+    try:
+        parsed = urlparse(url.strip())
+    except Exception as e:  # pragma: no cover - urlparse is very tolerant
+        return False, f"unparseable URL: {e}"
+
+    if parsed.scheme.lower() not in ALLOWED_SCHEMES:
+        return False, f"scheme must be http or https, got '{parsed.scheme or '(none)'}'"
+    host = parsed.hostname
+    if not host:
+        return False, "URL has no host"
+
+    resolve = resolver or _default_resolver
+    try:
+        raw_ips = resolve(host)
+    except Exception as e:
+        return False, f"host does not resolve: {e}"
+    if not raw_ips:
+        return False, "host does not resolve"
+
+    for raw in raw_ips:
+        try:
+            ip = ipaddress.ip_address(raw.split("%")[0])  # strip IPv6 zone id
+        except ValueError:
+            continue
+        reason = _classify(ip, block_private=block_private)
+        if reason:
+            return False, reason
+    return True, "ok"

src/webhook_manager.py

@@ -38,6 +38,20 @@ _PRIVATE_NETWORKS = [
 
 
 def _ip_is_private(addr: ipaddress._BaseAddress) -> bool:
+    # If the address is IPv4-mapped IPv6, extract and evaluate the embedded IPv4
+    if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
+        addr = addr.ipv4_mapped
+
+    if (
+        addr.is_private
+        or addr.is_loopback
+        or addr.is_link_local
+        or addr.is_reserved
+        or addr.is_multicast
+        or addr.is_unspecified
+    ):
+        return True
+
     return any(addr in net for net in _PRIVATE_NETWORKS)
 
 

src/youtube_handler.py

@@ -198,15 +198,24 @@ async def fetch_youtube_comments(
             f"https://www.youtube.com/watch?v={video_id}",
         ]
 
-        proc = await asyncio.wait_for(
-            asyncio.create_subprocess_exec(
-                *cmd,
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            ),
-            timeout=timeout,
+        proc = await asyncio.create_subprocess_exec(
+            *cmd,
+            stdout=asyncio.subprocess.PIPE,
+            stderr=asyncio.subprocess.PIPE,
         )
-        stdout, stderr = await proc.communicate()
+        # Bound the wait on the process actually finishing, not on spawning it.
+        # create_subprocess_exec returns as soon as the child starts, so wrapping
+        # it in wait_for never enforces the timeout — proc.communicate() is the
+        # blocking step. Kill and reap the child if it overruns so it does not
+        # linger after we return.
+        try:
+            stdout, stderr = await asyncio.wait_for(
+                proc.communicate(), timeout=timeout
+            )
+        except asyncio.TimeoutError:
+            proc.kill()
+            await proc.wait()
+            raise
 
         if proc.returncode != 0:
             return {"success": False, "error": f"yt-dlp failed: {stderr.decode()[:200]}", "comments": []}

start-macos.sh

@@ -16,8 +16,24 @@ set -e
 REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 cd "$REPO_DIR"
 
-PORT="${ODYSSEUS_PORT:-7860}"   # 7860, not 7000 — macOS AirPlay Receiver holds 7000.
-HOST="${ODYSSEUS_HOST:-127.0.0.1}" # Set ODYSSEUS_HOST=0.0.0.0 for LAN/Tailscale access.
+# Load .env so APP_PORT and APP_BIND are available without re-typing them on
+# the command line every run — consistent with how app.py reads them via
+# python-dotenv. Variables already set in the shell take priority over .env.
+if [ -f .env ]; then
+  while IFS='=' read -r key value; do
+    [[ "$key" =~ ^[[:space:]]*# ]] && continue
+    [[ -z "${key// }" ]] && continue
+    value="${value%%#*}"
+    value="${value#"${value%%[![:space:]]*}"}"
+    value="${value%"${value##*[![:space:]]}"}"
+    [ -n "$key" ] && [ -z "${!key+x}" ] && export "$key=$value"
+  done < .env
+fi
+
+# Shell overrides (ODYSSEUS_PORT / ODYSSEUS_HOST) take top priority, then .env
+# values (APP_PORT / APP_BIND), then built-in defaults.
+PORT="${ODYSSEUS_PORT:-${APP_PORT:-7860}}"   # 7860, not 7000 — macOS AirPlay Receiver holds 7000.
+HOST="${ODYSSEUS_HOST:-${APP_BIND:-127.0.0.1}}" # Set APP_BIND=0.0.0.0 in .env for LAN/Tailscale access.
 PROBE_HOST="$HOST"
 if [ "$PROBE_HOST" = "0.0.0.0" ] || [ "$PROBE_HOST" = "::" ]; then
   PROBE_HOST="127.0.0.1"
@@ -117,10 +133,11 @@ if [ ! -d venv ]; then
   echo "▶ Creating Python environment…"
   "$PY" -m venv venv
 fi
+VENV_PY="./venv/bin/python3"
 echo "▶ Installing Python packages (first run downloads a few — can take a few minutes)…"
-"$PY" -m pip install --quiet --upgrade pip
+"$VENV_PY" -m pip install --quiet --upgrade pip
 # Not --quiet: this is the slow step, so show progress (and any real errors).
-"$PY" -m pip install -r requirements.txt
+"$VENV_PY" -m pip install -r requirements.txt
 
 # 4. First-run setup: creates data dirs and prints an initial admin password
 #    the first time (idempotent — does nothing if already set up). Suppress its
@@ -179,4 +196,4 @@ if [ -n "$TAILSCALE_URL" ]; then
 fi
 echo "  (this takes a few seconds; press Ctrl+C here to stop)"
 echo
-"$PY" -m uvicorn app:app --host "$HOST" --port "$PORT"
+"$VENV_PY" -m uvicorn app:app --host "$HOST" --port "$PORT"

static/app.js

@@ -85,6 +85,39 @@ async function _refreshDefaultChat() {
 // synchronously; later reads should call _refreshDefaultChat() first.
 _refreshDefaultChat();
 
+async function _createDirectChatFromPreferredModel() {
+  if (!sessionModule) return false;
+
+  const pending = sessionModule.getPendingChat && sessionModule.getPendingChat();
+  if (pending && pending.url && pending.modelId) {
+    sessionModule.createDirectChat(pending.url, pending.modelId, pending.endpointId);
+    return true;
+  }
+
+  const sessions = sessionModule.getSessions();
+  const currentId = sessionModule.getCurrentSessionId();
+  const current = sessions.find(s => s.id === currentId);
+  if (current && current.endpoint_url && current.model) {
+    sessionModule.createDirectChat(current.endpoint_url, current.model, current.endpoint_id);
+    return true;
+  }
+
+  const dc = await _refreshDefaultChat();
+  if (dc) {
+    sessionModule.createDirectChat(dc.endpoint_url, dc.model, dc.endpoint_id);
+    return true;
+  }
+
+  const withModel = sessions.filter(s => s.endpoint_url && s.model);
+  if (withModel.length > 0) {
+    const last = withModel[0]; // sessions are sorted by recent
+    sessionModule.createDirectChat(last.endpoint_url, last.model, last.endpoint_id);
+    return true;
+  }
+
+  return false;
+}
+
 // ============================================
 // EVENT LISTENERS INITIALIZATION
 // ============================================
@@ -1564,6 +1597,8 @@ function initializeEventListeners() {
       saveToggleState(st);
       agentBtn.classList.toggle('active', mode === 'agent');
       chatBtn.classList.toggle('active', mode === 'chat');
+      agentBtn.setAttribute('aria-pressed', String(mode === 'agent'));
+      chatBtn.setAttribute('aria-pressed', String(mode === 'chat'));
       // Slide the pill to the active button
       const toggle = agentBtn.closest('.mode-toggle');
       if (toggle) toggle.classList.toggle('mode-chat', mode === 'chat');
@@ -1621,11 +1656,13 @@ function initializeEventListeners() {
     const chk = el(checkboxId);
     if (chk) chk.checked = saved;
     btn.classList.toggle('active', saved);
+    btn.setAttribute('aria-pressed', String(saved));
     btn.addEventListener('click', () => {
       const curMode = (loadToggleState().mode) || 'chat';
       const chk = el(checkboxId);
       chk.checked = !chk.checked;
       btn.classList.toggle('active', chk.checked);
+      btn.setAttribute('aria-pressed', String(chk.checked));
       saveToolPref(stateKey, curMode, chk.checked);
       showToolToggleToast(stateKey, chk.checked);
       if (chk.checked) _showToolSplash(stateKey);
@@ -3011,27 +3048,7 @@ function initializeEventListeners() {
       // Clear research mode if active
       const _resChk = el('research-toggle');
       if (_resChk && _resChk.checked) _syncResearchIndicator(false);
-      // Use default chat if configured — always re-fetch so setting changes apply immediately
-      const dc = await _refreshDefaultChat();
-      if (dc) {
-        sessionModule.createDirectChat(dc.endpoint_url, dc.model, dc.endpoint_id);
-        return;
-      }
-      const sessions = sessionModule.getSessions();
-      const currentId = sessionModule.getCurrentSessionId();
-      const current = sessions.find(s => s.id === currentId);
-      // Try current session's model first
-      if (current && current.endpoint_url && current.model) {
-        sessionModule.createDirectChat(current.endpoint_url, current.model, current.endpoint_id);
-        return;
-      }
-      // Fallback: find any recent session with a model
-      const withModel = sessions.filter(s => s.endpoint_url && s.model);
-      if (withModel.length > 0) {
-        const last = withModel[0]; // sessions are sorted by recent
-        sessionModule.createDirectChat(last.endpoint_url, last.model, last.endpoint_id);
-        return;
-      }
+      if (await _createDirectChatFromPreferredModel()) return;
       // No models at all — show welcome screen
       sessionModule.setCurrentSessionId(null);
       if (documentModule && documentModule.isPanelOpen && documentModule.isPanelOpen()) documentModule.closePanel();
@@ -3076,23 +3093,7 @@ function initializeEventListeners() {
       if (presetsModule && presetsModule.deactivateCharacter) presetsModule.deactivateCharacter();
       // Clear research toggle when starting a fresh chat (not via research button)
       _syncResearchIndicator(false);
-      const dc = await _refreshDefaultChat();
-      if (dc) {
-        sessionModule.createDirectChat(dc.endpoint_url, dc.model, dc.endpoint_id);
-        return;
-      }
-      const sessions = sessionModule.getSessions();
-      const currentId = sessionModule.getCurrentSessionId();
-      const current = sessions.find(s => s.id === currentId);
-      if (current && current.endpoint_url && current.model) {
-        sessionModule.createDirectChat(current.endpoint_url, current.model, current.endpoint_id);
-        return;
-      }
-      const withModel = sessions.filter(s => s.endpoint_url && s.model);
-      if (withModel.length > 0) {
-        sessionModule.createDirectChat(withModel[0].endpoint_url, withModel[0].model, withModel[0].endpoint_id);
-        return;
-      }
+      if (await _createDirectChatFromPreferredModel()) return;
       // No models at all — show welcome screen
       sessionModule.setCurrentSessionId(null);
       if (documentModule && documentModule.isPanelOpen && documentModule.isPanelOpen()) documentModule.closePanel();
@@ -3129,10 +3130,7 @@ function initializeEventListeners() {
         const idx = sessions.findIndex(s => s.id === currentId);
         const nextSession = sessions.filter(s => !s.archived && s.id !== currentId)[Math.max(0, idx)] ||
                             sessions.find(s => !s.archived && s.id !== currentId);
-        const res = await fetch(`${API_BASE}/api/session/${currentId}/archive`, {
-          method: 'POST',
-          headers: { 'Content-Type': 'application/json' },
-        });
+        const res = await fetch(`${API_BASE}/api/session/${currentId}`, { method: 'DELETE' });
         if (res.ok) {
           await sessionModule.loadSessions();
           if (nextSession) {
@@ -3159,7 +3157,7 @@ function initializeEventListeners() {
       setTimeout(() => uiModule.autoResize(textarea), 1);
     });
     textarea.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey) {
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
         // If ghost autocomplete is active, accept the suggestion instead of submitting
         if (window._ghostAutocomplete && window._ghostAutocomplete.isActive()) {
           e.preventDefault();
@@ -3732,7 +3730,7 @@ function startOdysseusApp() {
   // Enter to send (shift+enter for newline), or new chat when empty
   if (messageInput) {
     messageInput.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey) {
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
         e.preventDefault();
         // Flush the debounced icon update so dataset.mode reflects the current
         // text state. Without this, a fast type-and-Enter would still see the

static/index.html

@@ -265,7 +265,7 @@
             <p class="memory-desc doclib-desc" style="margin-top:6px;">Long-term facts the AI remembers across chats — recall, edit, or curate.</p>
             <div class="memory-toolbar">
               <div class="memory-toolbar-row">
-                <select id="memory-sort" class="memory-sort-select">
+                <select id="memory-sort" class="memory-sort-select" aria-label="Sort memories">
                   <option value="newest">Newest</option>
                   <option value="oldest">Oldest</option>
                   <option value="alpha">A-Z</option>
@@ -274,7 +274,7 @@
                 <button id="memory-select-btn" class="memory-toolbar-btn" title="Select multiple memories">Select</button>
                 <button id="memory-tidy-btn" class="memory-toolbar-btn" title="AI tidy: deduplicate and clean up memories"><svg width="11" height="11" viewBox="0 0 24 24" fill="currentColor" style="vertical-align:-1px;margin-right:2px;"><path d="M12 0L14.59 8.41L23 12L14.59 15.59L12 24L9.41 15.59L1 12L9.41 8.41Z"/></svg> Tidy</button>
               </div>
-              <input type="text" id="memory-search" placeholder="Search memories…" class="memory-search-input" />
+              <input type="text" id="memory-search" placeholder="Search memories…" class="memory-search-input" aria-label="Search memories" />
               <div id="memory-category-filters" class="memory-category-filters">
                 <button class="memory-cat-chip active" data-cat="all">all</button>
               </div>
@@ -304,7 +304,7 @@
             </p>
             <div class="memory-add-row" style="margin-top:8px;">
               <div class="skill-ph-wrap" style="flex:1;min-width:0;">
-                <input type="text" id="new-memory-input" placeholder=" " class="memory-add-input skill-hint-input" />
+                <input type="text" id="new-memory-input" placeholder=" " class="memory-add-input skill-hint-input" aria-label="New memory text" />
                 <span class="skill-rich-ph"><span class="k">Add a memory</span> &mdash; e.g. 'I prefer concise replies' <svg class="k" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="vertical-align:-2px;margin-left:4px;" aria-hidden="true"><polyline points="9 10 4 15 9 20"/><path d="M20 4v7a4 4 0 0 1-4 4H4"/></svg></span>
               </div>
             </div>
@@ -315,19 +315,19 @@
             </div>
             <p class="memory-desc doclib-desc" style="margin-top:6px;">Create a skill by hand — title, what it solves, and an approach.</p>
             <div class="skill-ph-wrap" style="margin-top:4px;margin-bottom:6px;">
-              <input type="text" id="new-skill-title" placeholder=" " class="memory-add-input skill-hint-input" />
+              <input type="text" id="new-skill-title" placeholder=" " class="memory-add-input skill-hint-input" aria-label="Skill title" />
               <span class="skill-rich-ph"><span class="k">Title</span> — short name, e.g. “build-vllm-wheel”</span>
             </div>
             <div class="skill-ph-wrap" style="margin-bottom:6px;">
-              <input type="text" id="new-skill-problem" placeholder=" " class="memory-add-input skill-hint-input" />
+              <input type="text" id="new-skill-problem" placeholder=" " class="memory-add-input skill-hint-input" aria-label="When to use this skill" />
               <span class="skill-rich-ph"><span class="k">When to use</span> — what problem does this skill solve?</span>
             </div>
             <div class="skill-ph-wrap" style="margin-bottom:6px;">
-              <textarea id="new-skill-solution" placeholder=" " class="memory-add-input skill-hint-input" rows="2" style="resize:vertical;"></textarea>
+              <textarea id="new-skill-solution" placeholder=" " class="memory-add-input skill-hint-input" rows="2" style="resize:vertical;" aria-label="How — the approach or steps"></textarea>
               <span class="skill-rich-ph skill-rich-ph-top"><span class="k">How</span> — the approach, steps, commands, or rules to follow</span>
             </div>
             <div class="skill-ph-wrap" style="margin-bottom:8px;">
-              <input type="text" id="new-skill-tags" placeholder=" " class="memory-add-input skill-hint-input" />
+              <input type="text" id="new-skill-tags" placeholder=" " class="memory-add-input skill-hint-input" aria-label="Tags" />
               <span class="skill-rich-ph"><span class="k">Tags</span> — comma-separated, e.g. python, build, vllm</span>
             </div>
             <div style="display:flex;justify-content:flex-end;">
@@ -368,7 +368,7 @@
                 <button id="skills-select-btn" class="memory-toolbar-btn" title="Select multiple skills">Select</button>
                 <button id="skills-audit-btn" class="memory-toolbar-btn" title="Test every skill, auto-fix the weak ones, flag what still fails"><svg width="11" height="11" viewBox="0 0 24 24" fill="currentColor" style="vertical-align:-1px;margin-right:3px;"><path d="M12 0L14.59 8.41L23 12L14.59 15.59L12 24L9.41 15.59L1 12L9.41 8.41Z"/></svg>Audit all</button>
               </div>
-              <input type="text" id="skills-search" placeholder="Search skills…" class="memory-search-input" />
+              <input type="text" id="skills-search" placeholder="Search skills…" class="memory-search-input" aria-label="Search skills" />
             </div>
             <div id="skills-audit-panel" class="skills-audit-panel hidden"></div>
             <div id="skills-bulk-bar" class="memory-bulk-bar hidden">
@@ -407,7 +407,7 @@
             <span class="admin-toggle-sub" style="display:block;margin-top:6px;opacity:0.6">Controls how many relevant published or approved skills are added to each agent request.</span>
             <div style="display:flex;align-items:center;justify-content:space-between;gap:12px;margin-top:8px">
               <span class="admin-toggle-sub" style="margin:0">Max skills per request</span>
-              <input type="number" id="skill-max-input" min="0" max="12" step="1" value="3" style="flex-shrink:0;width:72px;background:var(--input-bg,var(--panel));color:var(--fg);border:1px solid var(--border);border-radius:6px;padding:4px 6px;font-size:12px;text-align:right;font-variant-numeric:tabular-nums" />
+              <input type="number" id="skill-max-input" min="0" max="12" step="1" value="3" aria-label="Max skills to inject" style="flex-shrink:0;width:72px;background:var(--input-bg,var(--panel));color:var(--fg);border:1px solid var(--border);border-radius:6px;padding:4px 6px;font-size:12px;text-align:right;font-variant-numeric:tabular-nums" />
             </div>
             <span class="admin-toggle-sub" style="display:block;margin-top:6px;opacity:0.5">Set to 0 to disable skill injection.</span>
           </div>
@@ -464,12 +464,12 @@
       <div class="admin-card">
         <h2>Colors</h2>
         <div class="theme-custom" id="themeCustom">
-          <div class="color-row"><label>Background</label><input type="color" id="clr-bg"><button class="color-reset-btn" data-reset="bg" title="Reset this color">&#x21BA;</button></div>
-          <div class="color-row"><label>Text</label><input type="color" id="clr-fg"><button class="color-reset-btn" data-reset="fg" title="Reset this color">&#x21BA;</button></div>
-          <div class="color-row"><label>Panel</label><input type="color" id="clr-panel"><button class="color-reset-btn" data-reset="panel" title="Reset this color">&#x21BA;</button></div>
-          <div class="color-row"><label>Sidebar</label><input type="color" id="adv-sidebarBg"><button class="color-reset-btn" data-reset-adv="sidebarBg" title="Reset this color">&#x21BA;</button></div>
-          <div class="color-row"><label>Border</label><input type="color" id="clr-border"><button class="color-reset-btn" data-reset="border" title="Reset this color">&#x21BA;</button></div>
-          <div class="color-row"><label>Accent</label><input type="color" id="clr-red"><button class="color-reset-btn" data-reset="red" title="Reset this color">&#x21BA;</button></div>
+          <div class="color-row"><label>Background</label><input type="color" id="clr-bg"><button class="color-reset-btn" data-reset="bg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+          <div class="color-row"><label>Text</label><input type="color" id="clr-fg"><button class="color-reset-btn" data-reset="fg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+          <div class="color-row"><label>Panel</label><input type="color" id="clr-panel"><button class="color-reset-btn" data-reset="panel" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+          <div class="color-row"><label>Sidebar</label><input type="color" id="adv-sidebarBg"><button class="color-reset-btn" data-reset-adv="sidebarBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+          <div class="color-row"><label>Border</label><input type="color" id="clr-border"><button class="color-reset-btn" data-reset="border" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+          <div class="color-row"><label>Accent</label><input type="color" id="clr-red"><button class="color-reset-btn" data-reset="red" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
         </div>
       </div>
       <div class="theme-adv-toggle" id="theme-adv-toggle">
@@ -479,38 +479,38 @@
         <div class="theme-adv-group">
           <div class="theme-adv-group-label">Chat Bubbles</div>
           <div class="theme-custom">
-            <div class="color-row"><label>User Chat Bubble</label><input type="color" id="adv-userBubbleBg"><button class="color-reset-btn" data-reset-adv="userBubbleBg" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>AI Chat Bubble</label><input type="color" id="adv-aiBubbleBg"><button class="color-reset-btn" data-reset-adv="aiBubbleBg" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>Border Chat Bubble</label><input type="color" id="adv-bubbleBorder"><button class="color-reset-btn" data-reset-adv="bubbleBorder" title="Reset this color">&#x21BA;</button></div>
+            <div class="color-row"><label>User Chat Bubble</label><input type="color" id="adv-userBubbleBg"><button class="color-reset-btn" data-reset-adv="userBubbleBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>AI Chat Bubble</label><input type="color" id="adv-aiBubbleBg"><button class="color-reset-btn" data-reset-adv="aiBubbleBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>Border Chat Bubble</label><input type="color" id="adv-bubbleBorder"><button class="color-reset-btn" data-reset-adv="bubbleBorder" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
           </div>
         </div>
         <div class="theme-adv-group">
           <div class="theme-adv-group-label">Sidebar</div>
           <div class="theme-custom">
-            <div class="color-row"><label>Odysseus Logo</label><input type="color" id="adv-brandColor"><button class="color-reset-btn" data-reset-adv="brandColor" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label title="Hamburger menu"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" style="vertical-align:-2px;"><line x1="3" y1="6" x2="21" y2="6"/><line x1="3" y1="12" x2="21" y2="12"/><line x1="3" y1="18" x2="21" y2="18"/></svg></label><input type="color" id="adv-hamburgerColor"><button class="color-reset-btn" data-reset-adv="hamburgerColor" title="Reset this color">&#x21BA;</button></div>
+            <div class="color-row"><label>Odysseus Logo</label><input type="color" id="adv-brandColor"><button class="color-reset-btn" data-reset-adv="brandColor" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label title="Hamburger menu"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" style="vertical-align:-2px;"><line x1="3" y1="6" x2="21" y2="6"/><line x1="3" y1="12" x2="21" y2="12"/><line x1="3" y1="18" x2="21" y2="18"/></svg></label><input type="color" id="adv-hamburgerColor"><button class="color-reset-btn" data-reset-adv="hamburgerColor" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
           </div>
         </div>
         <div class="theme-adv-group">
           <div class="theme-adv-group-label">Chat Input / Prompt Area</div>
           <div class="theme-custom">
-            <div class="color-row"><label>Input Bg</label><input type="color" id="adv-inputBg"><button class="color-reset-btn" data-reset-adv="inputBg" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>Input Border</label><input type="color" id="adv-inputBorder"><button class="color-reset-btn" data-reset-adv="inputBorder" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>Send Btn</label><input type="color" id="adv-sendBtnBg"><button class="color-reset-btn" data-reset-adv="sendBtnBg" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>Send Hover</label><input type="color" id="adv-sendBtnHover"><button class="color-reset-btn" data-reset-adv="sendBtnHover" title="Reset this color">&#x21BA;</button></div>
+            <div class="color-row"><label>Input Bg</label><input type="color" id="adv-inputBg"><button class="color-reset-btn" data-reset-adv="inputBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>Input Border</label><input type="color" id="adv-inputBorder"><button class="color-reset-btn" data-reset-adv="inputBorder" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>Send Btn</label><input type="color" id="adv-sendBtnBg"><button class="color-reset-btn" data-reset-adv="sendBtnBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>Send Hover</label><input type="color" id="adv-sendBtnHover"><button class="color-reset-btn" data-reset-adv="sendBtnHover" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
           </div>
         </div>
         <div class="theme-adv-group">
           <div class="theme-adv-group-label">Code Blocks</div>
           <div class="theme-custom">
-            <div class="color-row"><label>Code Bg</label><input type="color" id="adv-codeBg"><button class="color-reset-btn" data-reset-adv="codeBg" title="Reset this color">&#x21BA;</button></div>
-            <div class="color-row"><label>Code Text</label><input type="color" id="adv-codeFg"><button class="color-reset-btn" data-reset-adv="codeFg" title="Reset this color">&#x21BA;</button></div>
+            <div class="color-row"><label>Code Bg</label><input type="color" id="adv-codeBg"><button class="color-reset-btn" data-reset-adv="codeBg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
+            <div class="color-row"><label>Code Text</label><input type="color" id="adv-codeFg"><button class="color-reset-btn" data-reset-adv="codeFg" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
           </div>
         </div>
         <div class="theme-adv-group">
           <div class="theme-adv-group-label">Controls</div>
           <div class="theme-custom">
-            <div class="color-row"><label>Toggle On</label><input type="color" id="adv-toggleActive"><button class="color-reset-btn" data-reset-adv="toggleActive" title="Reset this color">&#x21BA;</button></div>
+            <div class="color-row"><label>Toggle On</label><input type="color" id="adv-toggleActive"><button class="color-reset-btn" data-reset-adv="toggleActive" title="Reset this color" aria-label="Reset color">&#x21BA;</button></div>
           </div>
         </div>
         <div class="theme-adv-group">
@@ -559,7 +559,7 @@
         <div class="theme-fd-row">
           <div class="theme-fd-group">
             <label class="theme-fd-label">Font</label>
-            <select id="theme-font-select" class="theme-fd-select">
+            <select id="theme-font-select" class="theme-fd-select" aria-label="Font">
               <option value="mono">Monospace</option>
               <option value="sans">Sans-serif</option>
               <option value="serif">Serif</option>
@@ -567,7 +567,7 @@
           </div>
           <div class="theme-fd-group">
             <label class="theme-fd-label">Density</label>
-            <select id="theme-density-select" class="theme-fd-select">
+            <select id="theme-density-select" class="theme-fd-select" aria-label="Density">
               <option value="compact">Compact</option>
               <option value="comfortable">Comfortable</option>
               <option value="spacious">Spacious</option>
@@ -981,7 +981,7 @@
     <input type="checkbox" id="research-toggle" style="display:none;">
     <input type="checkbox" id="rag-toggle" style="display:none;">
     <input type="checkbox" id="incognito-toggle" style="display:none;">
-    <input type="file" id="file-input" class="hidden" multiple accept="image/*,application/pdf,video/*,.txt,.py,.html,.htm,.md,.json,.csv,.log,audio/*" />
+    <input type="file" id="file-input" class="hidden" multiple />
 
     <!-- Unified chat input bar -->
     <div class="chat-input-bar">
@@ -993,7 +993,7 @@
           <button type="button" class="model-picker-btn" id="model-picker-btn" title="Switch model"><span id="model-picker-label">Select model</span> <svg width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><polyline points="6 15 12 9 18 15"/></svg></button>
           <div class="model-picker-menu hidden" id="model-picker-menu">
             <div class="model-picker-search-row">
-              <input type="text" id="model-picker-search" placeholder="Search models..." autocomplete="off">
+              <input type="text" id="model-picker-search" placeholder="Search models..." autocomplete="off" aria-label="Search models">
               <button type="button" class="model-picker-action-btn primary" id="model-picker-add-models-btn" title="Add model endpoints" aria-label="Add model endpoints">
                 <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round"><path d="M12 5v14"/><path d="M5 12h14"/></svg>
               </button>
@@ -1007,7 +1007,7 @@
         <div class="chat-input-left">
           <!-- Overflow menu (+) — always first/left -->
           <div class="overflow-wrapper">
-            <button type="button" class="input-icon-btn overflow-plus-btn" id="overflow-plus-btn" title="More tools">
+            <button type="button" class="input-icon-btn overflow-plus-btn" id="overflow-plus-btn" title="More tools" aria-label="More tools" aria-haspopup="true">
               <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round">
                 <polyline points="6 15 12 9 18 15"/>
               </svg>
@@ -1051,13 +1051,13 @@
             </div>
           </div>
           <!-- Web search (magnifying glass) -->
-          <button type="button" class="input-icon-btn" title="Web search" id="web-toggle-btn" data-mode-tool="true">
+          <button type="button" class="input-icon-btn" title="Web search" id="web-toggle-btn" data-mode-tool="true" aria-label="Web search" aria-pressed="false">
             <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
               <circle cx="11" cy="11" r="8"/><line x1="21" y1="21" x2="16.65" y2="16.65"/>
             </svg>
           </button>
           <!-- Shell commands (terminal) -->
-          <button type="button" class="input-icon-btn" title="Shell Access" id="bash-toggle-btn" data-mode-tool="true">
+          <button type="button" class="input-icon-btn" title="Shell Access" id="bash-toggle-btn" data-mode-tool="true" aria-label="Shell access" aria-pressed="false">
             <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
               <polyline points="4 17 10 11 4 5"/><line x1="12" y1="19" x2="20" y2="19"/>
             </svg>
@@ -1099,8 +1099,8 @@
         <div class="chat-input-right">
           <!-- Agent / Chat mode toggle -->
           <div class="mode-toggle">
-            <button type="button" class="mode-toggle-btn active" id="mode-agent-btn">Agent</button>
-            <button type="button" class="mode-toggle-btn" id="mode-chat-btn">Chat</button>
+            <button type="button" class="mode-toggle-btn active" id="mode-agent-btn" aria-pressed="true">Agent</button>
+            <button type="button" class="mode-toggle-btn" id="mode-chat-btn" aria-pressed="false">Chat</button>
           </div>
           <button type="submit" form="chat-form" class="send-btn newchat-mode" data-mode="newchat" aria-label="New chat">
             <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round"><line x1="12" y1="5" x2="12" y2="19"/><line x1="5" y1="12" x2="19" y2="12"/></svg><span class="send-btn-label">+ New</span>
@@ -1463,6 +1463,10 @@
                 <label class="settings-label">Extract Parallel</label>
                 <input id="set-researchExtractConcurrency" type="text" inputmode="numeric" placeholder="3" class="settings-select" style="width:120px;">
               </div>
+              <div class="settings-row">
+                <label class="settings-label">Max Time</label>
+                <input id="set-researchRunTimeout" type="text" inputmode="numeric" placeholder="1800 sec (0 = no limit)" class="settings-select" style="width:120px;">
+              </div>
               <div id="set-researchMsg" style="font-size:11px;color:color-mix(in srgb, var(--fg) 45%, transparent);"></div>
             </div>
           </div>
@@ -2014,6 +2018,9 @@
                     <option value="image">Image</option>
                   </select>
                 </div>
+                <div class="admin-model-form-row">
+                  <input id="adm-epLocalApiKey" type="password" placeholder="API key (optional — for protected local endpoints)" autocomplete="off" style="flex:1">
+                </div>
                 <div class="admin-model-form-row">
                   <span style="flex:1"></span>
                   <button class="admin-btn-sm" id="adm-epLocalTestBtn" style="width:55px;text-align:center;">Test</button>
@@ -2068,6 +2075,7 @@
                   <option value="https://generativelanguage.googleapis.com/v1beta/openai" data-logo="gemini">Google Gemini</option>
                   <option value="https://api.x.ai/v1" data-logo="grok">xAI Grok</option>
                   <option value="https://api.z.ai/api/paas/v4" data-logo="zhipu">Z.AI (Zhipu)</option>
+                  <option value="https://api.z.ai/api/coding/paas/v4" data-logo="zhipu">Z.AI Coding Plan</option>
                 </select>
                 <div class="admin-model-form-row">
                   <input id="adm-epApiKey" type="password" placeholder="API key">

static/js/admin.js

@@ -871,11 +871,14 @@ function initEndpointForm() {
       const raw = (el('adm-epLocalUrl').value || '').trim();
       if (!raw) { msg.textContent = 'Enter a base URL to test'; msg.className = 'admin-error'; return; }
       const url = _normalizeBaseUrl(raw);
+      const keyEl = el('adm-epLocalApiKey');
+      const apiKey = keyEl ? keyEl.value.trim() : '';
       localTestBtn.disabled = true;
       localTestBtn.textContent = 'Testing...';
       try {
         const fd = new FormData();
         fd.append('base_url', url);
+        if (apiKey) fd.append('api_key', apiKey);
         const res = await fetch('/api/model-endpoints/test', { method: 'POST', body: fd, credentials: 'same-origin' });
         const d = await res.json();
         _renderEndpointTestResult(msg, res, d);
@@ -894,10 +897,13 @@ function initEndpointForm() {
       const raw = (el('adm-epLocalUrl').value || '').trim();
       if (!raw) { msg.textContent = 'Enter a base URL (e.g. http://localhost:8002/v1)'; msg.className = 'admin-error'; return; }
       const url = _normalizeBaseUrl(raw);
+      const keyEl = el('adm-epLocalApiKey');
+      const apiKey = keyEl ? keyEl.value.trim() : '';
       localAddBtn.disabled = true; localAddBtn.textContent = 'Adding...';
       try {
         const fd = new FormData();
         fd.append('base_url', url);
+        if (apiKey) fd.append('api_key', apiKey);
         const lt = el('adm-epLocalType');
         if (lt) fd.append('model_type', lt.value);
         fd.append('skip_probe', 'false');
@@ -905,6 +911,7 @@ function initEndpointForm() {
         const d = await res.json();
         if (res.ok) {
           el('adm-epLocalUrl').value = '';
+          if (keyEl) keyEl.value = '';
           if (lt) lt.value = 'llm';
           if (d.id) _recentlyAddedEpId = String(d.id);
           await loadEndpoints();

static/js/calendar.js

@@ -13,6 +13,7 @@ import {
   CAL_PALETTE, CAL_COLORS, _CAL_CUSTOM_GRADIENT, _TYPE_PALETTE,
   _trashIcon, _moreIcon, _bellIcon,
   _isCalBgImage, _calBgImageUrl, _calBgCss,
+  _calReadableTextColor,
   _ds, _addDays, _shiftDT, _tzOffset, _localDateOf,
 } from './calendar/utils.js';
 
@@ -371,6 +372,10 @@ function _calColor(ev) {
   return c?.color || 'var(--accent)';
 }
 
+function _calEventFg(ev) {
+  return _calReadableTextColor(_calColor(ev));
+}
+
 // Extra inline style for an event row when the event has a custom BG image.
 // Returns '' for normal solid-color events.
 function _calItemBgStyle(ev) {
@@ -975,7 +980,7 @@ async function _renderMonth() {
       const startColInt = Math.round(startCol);
       const endColInt = Math.round(endCol);
       const span = endColInt - startColInt + 1;
-      h += `<div class="cal-multiday" style="--col:${startColInt};--span:${span};--slot:${barSlot};background:${_calColor(md)}" draggable="true" data-uid="${_e(md.uid)}" title="${_e(md.summary)}">${_e(md.summary)}</div>`;
+      h += `<div class="cal-multiday" style="--col:${startColInt};--span:${span};--slot:${barSlot};background:${_calColor(md)};--cal-event-fg:${_calEventFg(md)}" draggable="true" data-uid="${_e(md.uid)}" title="${_e(md.summary)}">${_e(md.summary)}</div>`;
       barSlot++;
     }
     h += '</div>';
@@ -1141,7 +1146,7 @@ async function _renderWeek() {
     // All-day strip
     colsHtml += `<div class="cal-wk-allday">`;
     for (const ev of allDayEvents) {
-      colsHtml += `<div class="cal-wk-allday-event" data-uid="${_e(ev.uid)}" style="background:${_calColor(ev)};" title="${_e(ev.summary)}">${_e(ev.summary)}</div>`;
+      colsHtml += `<div class="cal-wk-allday-event" data-uid="${_e(ev.uid)}" style="background:${_calColor(ev)};--cal-event-fg:${_calEventFg(ev)};" title="${_e(ev.summary)}">${_e(ev.summary)}</div>`;
     }
     colsHtml += `</div>`;
     // Hour-grid body

static/js/calendar/utils.js

@@ -74,6 +74,42 @@ export function _calBgCss(c, fallback) {
   return c || fallback || 'var(--accent)';
 }
 
+function _hexToRgb(c) {
+  if (typeof c !== 'string') return null;
+  const m = c.trim().match(/^#([0-9a-f]{3}|[0-9a-f]{6})$/i);
+  if (!m) return null;
+  const hex = m[1].length === 3
+    ? m[1].split('').map(ch => ch + ch).join('')
+    : m[1];
+  return {
+    r: parseInt(hex.slice(0, 2), 16),
+    g: parseInt(hex.slice(2, 4), 16),
+    b: parseInt(hex.slice(4, 6), 16),
+  };
+}
+
+function _relativeLuminance({ r, g, b }) {
+  return [r, g, b].map(v => {
+    const c = v / 255;
+    return c <= 0.03928 ? c / 12.92 : Math.pow((c + 0.055) / 1.055, 2.4);
+  }).reduce((sum, c, i) => sum + c * [0.2126, 0.7152, 0.0722][i], 0);
+}
+
+function _contrastRatio(a, b) {
+  const light = Math.max(a, b);
+  const dark = Math.min(a, b);
+  return (light + 0.05) / (dark + 0.05);
+}
+
+export function _calReadableTextColor(bg) {
+  const rgb = _hexToRgb(bg);
+  if (!rgb) return 'var(--fg)';
+  const lum = _relativeLuminance(rgb);
+  const white = _contrastRatio(lum, 1);
+  const ink = _contrastRatio(lum, 0.006);
+  return ink >= white ? '#111820' : '#ffffff';
+}
+
 // ── date helpers ──
 
 // `YYYY-MM-DD` string from a Date.

static/js/chat.js

@@ -964,6 +964,11 @@ import createResearchSynapse from './researchSynapse.js';
         return;
       }
 
+      // Mark the chat log busy while streaming so screen readers wait for the
+      // settled response instead of announcing every token. Cleared in finally.
+      const _chatLog = document.getElementById('chat-history');
+      if (_chatLog) _chatLog.setAttribute('aria-busy', 'true');
+
       const reader = res.body.getReader();
       const decoder = new TextDecoder();
       let buffer = '';
@@ -2702,6 +2707,9 @@ import createResearchSynapse from './researchSynapse.js';
       }
     } finally {
       clearProcessingProbe();
+      // Streaming done — let screen readers announce the settled response.
+      const _chatLogDone = document.getElementById('chat-history');
+      if (_chatLogDone) _chatLogDone.setAttribute('aria-busy', 'false');
       // Always clean up research tracking regardless of background state
       _researchingStreamIds.delete(streamSessionId);
       if (_researchingStreamIds.size === 0) {
@@ -3416,7 +3424,7 @@ import createResearchSynapse from './researchSynapse.js';
 
     // Also submit on Enter (without shift)
     editor.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey) {
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
         e.preventDefault();
         saveBtn.click();
       }

static/js/color/hex.js

@@ -0,0 +1,14 @@
+// static/js/color/hex.js
+//
+// Parse a CSS hex color into {r, g, b}. Pure — no DOM — so it can be reused
+// across modules and unit-tested under node.
+
+// Accepts "#rgb", "#rrggbb" (with or without the leading '#'). Returns null
+// for anything that isn't a valid 3- or 6-digit hex color.
+export function hexToRgb(hex) {
+  let h = String(hex || '').trim().replace(/^#/, '');
+  if (h.length === 3) h = h.split('').map((c) => c + c).join('');
+  if (!/^[0-9a-fA-F]{6}$/.test(h)) return null;
+  const n = parseInt(h, 16);
+  return { r: (n >> 16) & 255, g: (n >> 8) & 255, b: n & 255 };
+}

static/js/cookbook-hwfit.js

@@ -213,9 +213,33 @@ export function _renderGpuToggles(system) {
 // reload paints instantly, then we refresh in the background and swap.
 const _SCAN_CACHE_KEY = 'hwfit_scan_cache_v1';
 const _MANUAL_HW_KEY = 'hwfit_manual_hardware_v1';
+const _CTX_KEY = 'hwfit_target_context_v1';
+const _CTX_PRESETS = [8192, 16384, 32768, 50000, 131072, 0]; // 0 = model max
 const _SCAN_CACHE_MAX = 12;            // keep the newest N signatures
 const _SCAN_CACHE_TTL = 6 * 3600 * 1000; // 6 h — hardware rarely changes
 
+function _ctxLabel(value) {
+  const n = Number(value) || 0;
+  if (!n) return 'Max';
+  return n >= 1000 ? Math.round(n / 1000) + 'k' : String(n);
+}
+
+function _ctxValue() {
+  const slider = document.getElementById('hwfit-context');
+  const idx = Math.max(0, Math.min(_CTX_PRESETS.length - 1, Number(slider?.value ?? 3) || 0));
+  return _CTX_PRESETS[idx] || 0;
+}
+
+function _syncCtxControl() {
+  const slider = document.getElementById('hwfit-context');
+  const label = document.getElementById('hwfit-context-label');
+  if (!slider) return;
+  const saved = localStorage.getItem(_CTX_KEY);
+  const savedIdx = saved == null ? 3 : _CTX_PRESETS.indexOf(Number(saved));
+  slider.value = String(savedIdx >= 0 ? savedIdx : 3);
+  if (label) label.textContent = _ctxLabel(_ctxValue());
+}
+
 function _manualHwState() {
   try {
     const s = JSON.parse(localStorage.getItem(_MANUAL_HW_KEY) || '{}');
@@ -316,6 +340,7 @@ function _scanSig() {
     o: sortEl?.value || 'score',
     r: sortEl?.dataset.reverse === '1' ? 1 : 0,
     q: document.getElementById('hwfit-quant')?.value || '',
+    c: _ctxValue(),
     g: (tc && typeof tc._activeCount === 'number') ? String(tc._activeCount) : '',
     gg: (tc && tc._activeGroup) ? String(tc._activeGroup) : '',
     m: _manualHwParams(),
@@ -440,6 +465,7 @@ export async function _hwfitFetch(fresh = false) {
   try {
     const sortBy = document.getElementById('hwfit-sort')?.value || 'score';
     const quantPref = document.getElementById('hwfit-quant')?.value || '';
+    const targetCtx = _ctxValue();
     // Get active GPU count from toggles
     const toggleContainer = document.getElementById('hwfit-gpu-toggles');
     let gpuCountOverride = '';
@@ -475,6 +501,7 @@ export async function _hwfitFetch(fresh = false) {
     if (!isImageMode) {
       if (useCase) params.set('use_case', useCase);
       if (quantPref) params.set('quant', quantPref);
+      if (targetCtx) params.set('ctx', String(targetCtx));
     }
     const endpoint = isImageMode ? `/api/hwfit/image-models?${params}` : `/api/hwfit/models?${params}`;
     const res = await fetch(endpoint);
@@ -770,6 +797,20 @@ function _wireManualHardwareControls(el) {
 
 export const _fitColors = { perfect: 'var(--green, #50fa7b)', good: 'var(--yellow, #f1fa8c)', marginal: 'var(--orange, #ffb86c)', too_tight: 'var(--red, #ff5555)' };
 
+function _requiresAcceleratorBackend(model) {
+  const q = String(model?.quant || model?.quantization || '').toUpperCase();
+  const text = `${model?.name || ''} ${model?.repo_id || ''} ${model?.path || ''}`.toLowerCase();
+  return /^AWQ|^GPTQ|^NVFP4/.test(q) || q === 'FP8' || /\b(awq|gptq|fp8|nvfp4)\b/i.test(text);
+}
+
+function _modeLabel(model) {
+  if (model?.is_image_gen) return 'image';
+  if (_requiresAcceleratorBackend(model)) return 'vLLM/SGLang';
+  const detected = _detectBackend(model);
+  if (detected?.label) return detected.label;
+  return String(model?.run_mode || '').replace('_', '+');
+}
+
 export const _hwfitColumns = [
   { key: 'fit', label: 'Fit',    cls: 'hwfit-fit' },
   { key: null,    label: 'Model',  cls: 'hwfit-name' },
@@ -827,9 +868,7 @@ export function _hwfitRenderList(el, models) {
     const pcount = m.parameter_count || '?';
     const ctx = m.context ? (m.context >= 1024 ? (m.context / 1024).toFixed(0) + 'k' : m.context) : '?';
     const fitLabel = (m.fit_level || '').replace('_', ' ');
-    const modeLabel = m.run_mode === 'cpu_offload'
-      ? 'cpu+offload'
-      : (m.run_mode || '').replace(/_/g, ' ');
+    const modeLabel = _modeLabel(m);
     const vramLabel = m.required_gb ? m.required_gb.toFixed(1) + 'G' : '?';
     const moeBadge = m.is_moe ? '<span class="hwfit-badge hwfit-moe">MoE</span>' : '';
     const imgBadge = m.is_image_gen ? '<span class="hwfit-badge" style="background:color-mix(in srgb, var(--red) 20%, transparent);color:var(--red);font-size:8px;padding:1px 4px;border-radius:3px;margin-left:4px;">IMG</span>' : '';
@@ -843,7 +882,7 @@ export function _hwfitRenderList(el, models) {
     html += `<span class="hwfit-col hwfit-c-ctx">${m.is_image_gen ? '\u2014' : ctx}</span>`;
     html += `<span class="hwfit-col hwfit-c-speed">${m.is_image_gen ? '\u2014' : tps + ' t/s'}</span>`;
     html += `<span class="hwfit-col hwfit-c-score">${score}</span>`;
-    html += `<span class="hwfit-col hwfit-c-mode">${m.is_image_gen ? 'image' : esc(modeLabel)}</span>`;
+    html += `<span class="hwfit-col hwfit-c-mode" title="${_requiresAcceleratorBackend(m) ? 'Requires vLLM or SGLang with a visible CUDA/ROCm accelerator. llama.cpp and Ollama need GGUF files.' : ''}">${esc(modeLabel)}</span>`;
     html += `</div>`;
   }
   el.innerHTML = html;
@@ -943,6 +982,8 @@ export function _expandModelRow(row, modelData) {
   html += `</div>`;
   if (modelData.is_image_gen) {
     html += `<div style="font-size:10px;opacity:0.5;margin-top:4px;">${esc((modelData.capabilities || []).join(' \u00B7 ') || '')}${modelData.description ? ' \u2014 ' + esc(modelData.description) : ''}</div>`;
+  } else if (_requiresAcceleratorBackend(modelData)) {
+    html += `<div class="hwfit-panel-note">This is a safetensors GPU-serving format. Use vLLM/SGLang with a visible CUDA/ROCm accelerator, or pick a GGUF download for llama.cpp/Ollama.</div>`;
   }
   html += `</div>`;
 
@@ -1139,8 +1180,11 @@ export function _hwfitInit() {
   const uc = document.getElementById('hwfit-usecase');
   const sort = document.getElementById('hwfit-sort');
   const qpref = document.getElementById('hwfit-quant');
+  const ctx = document.getElementById('hwfit-context');
+  const ctxLabel = document.getElementById('hwfit-context-label');
   const search = document.getElementById('hwfit-search');
   const remote = document.getElementById('hwfit-host');
+  _syncCtxControl();
   if (uc) uc.addEventListener('change', () => _hwfitFetch());
   if (sort) sort.addEventListener('change', () => _hwfitFetch());
   if (qpref) qpref.addEventListener('change', () => _hwfitFetch());
@@ -1155,6 +1199,28 @@ export function _hwfitInit() {
       _hwfitFetch();
     }
   });
+  if (ctx && !ctx.dataset.bound) {
+    ctx.dataset.bound = '1';
+    ctx.addEventListener('input', () => {
+      if (ctxLabel) ctxLabel.textContent = _ctxLabel(_ctxValue());
+    });
+    ctx.addEventListener('change', () => {
+      const targetCtx = _ctxValue();
+      try { localStorage.setItem(_CTX_KEY, String(targetCtx)); } catch {}
+      const sortSel = document.getElementById('hwfit-sort');
+      if (sortSel) {
+        if (targetCtx) {
+          sortSel.value = 'fit';
+          sortSel.dataset.reverse = '1';
+        } else {
+          sortSel.value = 'score';
+          sortSel.dataset.reverse = '';
+        }
+      }
+      _hwfitCache = null;
+      _hwfitFetch();
+    });
+  }
   // Rescan — force a fresh hardware probe (bypasses the per-host cache).
   const rescan = document.getElementById('hwfit-rescan');
   if (rescan && !rescan.dataset.bound) {

static/js/cookbook.js

@@ -366,6 +366,8 @@ export function _buildServeCmd(f, modelName, backend) {
     cmd += ` --gpu-memory-utilization ${f.gpu_mem || '0.90'}`;
     if (f.swap && f.swap !== '0') cmd += ` --swap-space ${f.swap}`;
     cmd += ` --dtype ${f.dtype || 'auto'}`;
+    const _kv = (f.vllm_kv_cache_dtype ?? '').toString().trim();
+    if (_kv === 'fp8') cmd += ' --kv-cache-dtype fp8';
     if (f.max_seqs && f.max_seqs.toString().trim()) cmd += ` --max-num-seqs ${f.max_seqs.toString().trim()}`;
     if (f.enforce_eager) cmd += ' --enforce-eager';
     if (f.trust_remote) cmd += ' --trust-remote-code';
@@ -616,6 +618,10 @@ async function _fetchDependencies() {
     const _statusTag = (pkg, isLocal, isSystemDep, winBlocked) => {
       if (winBlocked) return `<span class="cookbook-dep-tag cookbook-dep-na">N/A</span>`;
       if (pkg.installed && isSystemDep) return `<span class="cookbook-dep-tag cookbook-dep-installed" title="Found on selected server">Installed</span>`;
+      if (pkg.installed && pkg.pip_update_available === false) {
+        const tip = esc(pkg.update_note || pkg.status_note || 'Found externally; update outside Odysseus.');
+        return `<span class="cookbook-dep-tag cookbook-dep-installed" title="${tip}">Installed</span>`;
+      }
       if (pkg.installed) return `<button class="cookbook-dep-tag cookbook-dep-installed cookbook-dep-installed-btn" title="Installed — click for actions"><span class="cookbook-dep-installed-label">Installed</span><span class="cookbook-dep-caret">&#9662;</span></button>`;
       if (isSystemDep) {
         const depTip = esc(pkg.install_hint || 'Install this OS package on the selected server.');
@@ -630,11 +636,13 @@ async function _fetchDependencies() {
       const isSystemDep = pkg.kind === 'system';
       const winBlocked = !isLocal && _isWindows() && _winUnsupported.has(pkg.name);
       const note = pkg.status_note ? `<div class="memory-item-meta" style="font-size:10px;opacity:0.65;margin-top:3px;">${esc(pkg.status_note)}</div>` : '';
+      const updateNote = pkg.installed && pkg.pip_update_available === false && pkg.update_note ? `<div class="memory-item-meta" style="font-size:10px;opacity:0.55;margin-top:3px;">${esc(pkg.update_note)}</div>` : '';
       return `<div class="cookbook-dep-row${winBlocked ? ' cookbook-dep-blocked' : ''}" data-pkg-name="${esc(pkg.name)}" data-dep-pip="${esc(pkg.pip || '')}" data-dep-target="${isLocal ? 'local' : 'remote'}" data-dep-kind="${esc(pkg.kind || 'python')}">`
         + `<div class="cookbook-dep-info">`
         + `<div class="memory-item-title">${esc(pkg.name)}</div>`
         + `<div class="memory-item-meta" style="font-size:10px;opacity:0.5;margin-top:2px;">${esc(pkg.desc)}</div>`
         + note
+        + updateNote
         + `</div>`
         + `<span class="cookbook-dep-tag cookbook-dep-cat">${esc(pkg.category)}</span>`
         + _statusTag(pkg, isLocal, isSystemDep, winBlocked)
@@ -1528,6 +1536,10 @@ function _renderRecipes() {
   html += '<option value="vllm">vLLM</option>';
   html += '<option value="sglang">SGLang</option>';
   html += '</select>';
+  html += '<span class="hwfit-help-chip" title="Higher numbers usually mean better quality, but they need more memory. Lower numbers fit on more hardware.">?</span>';
+  html += '<label class="hwfit-ctx-control" title="Context length for fit estimates. Lower it to find more models that could fit your hardware.">';
+  html += '<span>Ctx</span><span class="hwfit-help-chip hwfit-help-chip-inline" title="Context length. Lower it to find more models that could fit your hardware; raise it when you need longer chats or documents.">?</span><input type="range" id="hwfit-context" min="0" max="5" step="1" value="3" />';
+  html += '<output id="hwfit-context-label">50k</output></label>';
   html += '</div>';
   html += '<div class="hwfit-toolbar" style="margin-top:7px;">';
   html += '<select class="cookbook-field-input hwfit-server-select" id="hwfit-server-select" style="height:28px;min-width:88px;position:relative;top:0px;">';

static/js/cookbookDownload.js

@@ -495,6 +495,10 @@ export async function _runModelDownload(panel, model, backend, hostOverride) {
 
   const payload = { repo_id: repo };
   if (include) payload.include = include;
+  // Large downloads are where hf_transfer most often dies near the end. Use the
+  // plain HuggingFace downloader up front for big model files; it is slower, but
+  // resumes cached partials more reliably.
+  if ((model.required_gb || 0) >= 10 || backend === 'llamacpp') payload.disable_hf_transfer = true;
   if (_envState.hfToken) payload.hf_token = _envState.hfToken;
   if (host) { payload.remote_host = host; const _sp = _getPort(host); if (_sp) payload.ssh_port = _sp; }
   if (platform) payload.platform = platform;
@@ -519,6 +523,18 @@ export async function _runModelDownload(panel, model, backend, hostOverride) {
   const targetHost = host || 'local';
 
   const tasks = _loadTasks();
+  const sameDownload = (t) => {
+    if (!t || t.type !== 'download') return false;
+    const tRepo = t?.payload?.repo_id || t?.repo_id || t?.repo || t?.name || '';
+    const tHost = t?.remoteHost || t?.payload?.remote_host || 'local';
+    return String(tRepo) === String(payload.repo_id) && String(tHost || 'local') === String(targetHost);
+  };
+  const duplicate = tasks.find(t => sameDownload(t) && (t.status === 'running' || t.status === 'queued'));
+  if (duplicate) {
+    _renderRunningTab();
+    uiModule.showToast(`${shortName} is already ${duplicate.status === 'queued' ? 'queued' : 'downloading'}`);
+    return;
+  }
   const activeOnHost = tasks.find(t => t.type === 'download' && (t.status === 'running' || t.status === 'queued') && (t.remoteHost || 'local') === targetHost);
 
   if (activeOnHost) {

static/js/cookbookRunning.js

@@ -37,7 +37,6 @@ function _taskBadge(task) {
 function _canClearTask(task) {
   if (!task || task.status === 'running') return false;
   if (task.type === 'serve' && (task.status === 'ready' || task._serveReady)) return false;
-  if (task.type === 'download' && task.status === 'done' && !task.payload?._dep) return false;
   return ['done', 'stopped', 'error', 'crashed', 'failed'].includes(task.status);
 }
 
@@ -376,10 +375,40 @@ function _refreshModelsAfterEndpointChange() {
   }, 1500);
 }
 
+function _appendCookbookEndpointScope(fd, remoteHost) {
+  const host = String(remoteHost || '').trim();
+  if (!host || host === 'local' || host === 'localhost' || host === '127.0.0.1') {
+    fd.append('container_local', 'true');
+  }
+}
+
+function _connectHostFromRemote(remoteHost, fallback = 'localhost') {
+  const host = String(remoteHost || '').trim();
+  if (!host || host === 'local') return fallback;
+  return host.includes('@') ? host.split('@').pop() : host;
+}
+
+function _isAnyBindHost(host) {
+  const h = String(host || '').trim().toLowerCase();
+  return h === '0.0.0.0' || h === '::' || h === '[::]';
+}
+
+function _endpointFromAdvertisedUrl(rawUrl, currentHost, fallbackPort = '11434') {
+  try {
+    const u = new URL(rawUrl);
+    const host = _isAnyBindHost(u.hostname) ? currentHost : (u.hostname || currentHost);
+    const port = u.port || fallbackPort;
+    const bracketedHost = host.includes(':') && !host.startsWith('[') ? `[${host}]` : host;
+    return { host, port, baseUrl: `${u.protocol}//${bracketedHost}${port ? `:${port}` : ''}/v1` };
+  } catch {
+    return null;
+  }
+}
+
 // ── Download queue — runs one at a time per server ──
 
 function _processQueue() {
-  const tasks = _loadTasks();
+  const tasks = _loadPrunedTasks();
   const running = tasks.filter(t => t.type === 'download' && t.status === 'running');
   const queued = tasks.filter(t => t.type === 'download' && t.status === 'queued');
   if (!queued.length) return;
@@ -433,14 +462,24 @@ async function _startQueuedDownload(task) {
       return;
     }
     const oldId = task.sessionId;
-    const tasks = _loadTasks();
-    const t = tasks.find(t => t.sessionId === oldId);
-    if (t) {
-      t.sessionId = data.session_id;
-      t.id = data.session_id;
-      t.status = 'running';
-      _saveTasks(tasks);
-    }
+    const launchedTask = { ...task, sessionId: data.session_id, id: data.session_id, status: 'running' };
+    const key = _downloadDedupeKey(launchedTask);
+    let found = false;
+    const tasks = _loadTasks().filter(t => {
+      if (t.sessionId === oldId) {
+        found = true;
+        t.sessionId = data.session_id;
+        t.id = data.session_id;
+        t.status = 'running';
+        t._startLaunched = true;
+        return true;
+      }
+      if (t.sessionId === data.session_id) return false;
+      return !(key && t.type === 'download' && t.status === 'queued' && _downloadDedupeKey(t) === key);
+    });
+    if (!found) tasks.push(_stripTaskSecrets(launchedTask));
+    _saveTasks(tasks);
+    _renderRunningTab();
     _startBackgroundMonitor();
     await new Promise(r => setTimeout(r, 2000));
     _renderRunningTab();
@@ -473,6 +512,53 @@ export function _loadTasks() {
   catch { return []; }
 }
 
+function _downloadRepoKey(task) {
+  return String(task?.payload?.repo_id || task?.repo_id || task?.repo || task?.name || '').trim();
+}
+
+function _downloadHostKey(task) {
+  return String(task?.remoteHost || task?.payload?.remote_host || 'local').trim() || 'local';
+}
+
+function _downloadDedupeKey(task) {
+  if (!task || task.type !== 'download') return '';
+  const repo = _downloadRepoKey(task);
+  if (!repo) return '';
+  return `${_downloadHostKey(task)}\n${repo}`;
+}
+
+function _pruneQueuedDownloadDuplicates(tasks) {
+  if (!Array.isArray(tasks) || !tasks.length) return tasks || [];
+  const launched = new Set();
+  for (const task of tasks) {
+    if (task?.type !== 'download' || task.status === 'queued') continue;
+    const key = _downloadDedupeKey(task);
+    if (key) launched.add(key);
+  }
+
+  let changed = false;
+  const seenQueued = new Set();
+  const next = tasks.filter(task => {
+    if (task?.type !== 'download' || task.status !== 'queued') return true;
+    const key = _downloadDedupeKey(task);
+    if (!key) return true;
+    if (launched.has(key) || seenQueued.has(key)) {
+      changed = true;
+      return false;
+    }
+    seenQueued.add(key);
+    return true;
+  });
+  return changed ? next : tasks;
+}
+
+function _loadPrunedTasks() {
+  const tasks = _loadTasks();
+  const pruned = _pruneQueuedDownloadDuplicates(tasks);
+  if (pruned !== tasks) _saveTasks(pruned);
+  return pruned;
+}
+
 // Tombstones for removed tasks. Without these, removing a task only deletes it
 // locally — but the server still has it (its own POST guard even re-preserves
 // recently-added ones), so the next sync/poll merges it right back ("I removed
@@ -535,6 +621,13 @@ export function _addTask(sessionId, name, type, payload) {
     const _repoId = payload.repo_id;
     tasks = tasks.filter(t => !(t.type === 'download' && t.status === 'done' && t.payload && t.payload.repo_id === _repoId));
   }
+  if (type === 'download' && payload && payload.repo_id) {
+    const key = _downloadDedupeKey({ type: 'download', payload, remoteHost });
+    tasks = tasks.filter(t => {
+      if (t.sessionId === sessionId) return false;
+      return !(key && t.type === 'download' && t.status === 'queued' && _downloadDedupeKey(t) === key);
+    });
+  }
   const task = _stripTaskSecrets({ id: sessionId, sessionId, name, type, status: 'running', output: '', ts: Date.now(), payload: payload || null, remoteHost, sshPort, platform });
   tasks.push(task);
   _saveTasks(tasks);
@@ -651,6 +744,52 @@ function _tmuxGracefulKill(task) {
   return `tmux send-keys -t ${task.sessionId} C-c 2>/dev/null; sleep 2; tmux kill-session -t ${task.sessionId} 2>/dev/null`;
 }
 
+function _shQuote(value) {
+  return "'" + String(value ?? '').replace(/'/g, "'\\''") + "'";
+}
+
+function _taskLooksOllama(task, outputText = '') {
+  const haystack = `${task?.payload?.backend || ''} ${task?.payload?._cmd || ''} ${task?.payload?._fields?.backend || ''} ${outputText || ''}`;
+  return /\bollama\b/i.test(haystack) || /Ollama API ready on port\s+\d+/i.test(haystack);
+}
+
+function _ollamaBaseUrlForTask(task, outputText = '') {
+  const out = String(outputText || '');
+  const ready = out.match(/Ollama API ready on port\s+\d+:\s*(http:\/\/[^\s]+)/i);
+  if (ready) return ready[1].replace(/\/+$/, '');
+  const cmd = String(task?.payload?._cmd || '');
+  const host = cmd.match(/OLLAMA_HOST=([^\s]+)/)?.[1] || '';
+  const port = host.match(/:(\d+)$/)?.[1] || '11434';
+  return `http://127.0.0.1:${port}`;
+}
+
+function _ollamaModelForTask(task) {
+  return String(task?.payload?.model || task?.payload?.repo_id || task?.name || '').trim();
+}
+
+function _ollamaUnloadCommand(task, outputText = '') {
+  if (!_taskLooksOllama(task, outputText)) return '';
+  const model = _ollamaModelForTask(task);
+  if (!model) return '';
+  const base = _ollamaBaseUrlForTask(task, outputText);
+  const body = JSON.stringify({ model, prompt: '', keep_alive: 0, stream: false });
+  const inner = `curl -sf -X POST ${_shQuote(base + '/api/generate')} -H 'Content-Type: application/json' -d ${_shQuote(body)} >/dev/null 2>&1 || true`;
+  if (task.remoteHost) {
+    return `ssh ${_sshPrefix(_getPort(task))}${task.remoteHost} ${_shQuote(inner)}`;
+  }
+  return inner;
+}
+
+function _endpointUrlForTask(task, outputText = '') {
+  if (_taskLooksOllama(task, outputText)) {
+    return _ollamaBaseUrlForTask(task, outputText) + '/v1';
+  }
+  const host = _connectHostFromRemote(task.remoteHost);
+  const portMatch = task.payload?._cmd?.match(/--port\s+(\d+)/);
+  const port = portMatch ? portMatch[1] : '8000';
+  return `http://${host}:${port}/v1`;
+}
+
 // ── Wave animation ──
 
 const _waveFrames = ['▁▂▃', '▂▃▄', '▃▄▅', '▄▅▆', '▅▆▅', '▆▅▄', '▅▄▃', '▄▃▂', '▃▂▁'];
@@ -909,17 +1048,23 @@ async function _retryTask(el, task) {
       body: JSON.stringify({ command: _tmuxGracefulKill(task) }),
     });
   } catch {}
-  _removeTask(task.sessionId);
   if (task.payload) {
     if (task.type === 'serve' && task.payload._cmd) {
+      _removeTask(task.sessionId);
       _launchServeTask(task.name, task.payload.repo_id, task.payload._cmd, task.payload._fields, task.remoteHost || '');
     } else {
-      _retryDownload(task.name, task.payload);
+      uiModule.showToast('Retrying download — progress may look reset while HuggingFace checks cached files, then it should resume.', 7000);
+      _updateTask(task.sessionId, {
+        status: 'running',
+        output: `${task.output || ''}\n\n[odysseus] Retrying download. Progress may briefly look like a fresh download while HuggingFace checks cached/incomplete files; cached partial files will be reused when available.`.trim(),
+        _retrying: true,
+      });
+      _retryDownload(task.name, task.payload, task.sessionId);
     }
   }
 }
 
-async function _retryDownload(name, payload) {
+async function _retryDownload(name, payload, replaceSessionId = '') {
   try {
     // A retry means the fast hf_transfer path already failed once — fall back to
     // the plain, reliable downloader for this and any further attempt (it resumes
@@ -932,17 +1077,40 @@ async function _retryDownload(name, payload) {
     });
     if (!res.ok) {
       uiModule.showToast('Download failed: HTTP ' + res.status);
+      if (replaceSessionId) _updateTask(replaceSessionId, { status: 'crashed', _retrying: false });
       return;
     }
     const data = await res.json();
     if (!data.ok) {
       uiModule.showToast('Download failed: ' + (data.error || ''));
+      if (replaceSessionId) _updateTask(replaceSessionId, { status: 'crashed', _retrying: false });
       return;
     }
-    _addTask(data.session_id, name, 'download', _payload);
+    if (replaceSessionId) {
+      const tasks = _loadTasks();
+      const task = tasks.find(t => t.sessionId === replaceSessionId);
+      if (task) {
+        task.id = data.session_id;
+        task.sessionId = data.session_id;
+        task.status = 'running';
+        task.output = '';
+        task.ts = Date.now();
+        task.payload = _payload;
+        task._retrying = false;
+        _saveTasks(tasks);
+        _soloExpandTaskId = data.session_id;
+        _renderRunningTab();
+        _startBackgroundMonitor();
+      } else {
+        _addTask(data.session_id, name, 'download', _payload);
+      }
+    } else {
+      _addTask(data.session_id, name, 'download', _payload);
+    }
     uiModule.showToast(`Downloading ${name}...`);
   } catch (e) {
     uiModule.showToast('Download failed: ' + e.message);
+    if (replaceSessionId) _updateTask(replaceSessionId, { status: 'crashed', _retrying: false });
   }
 }
 
@@ -1193,6 +1361,7 @@ function _parseServeCmdToFields(cmd) {
     gpu_mem: ex(/--gpu-memory-utilization\s+([\d.]+)/) || '0.90',
     swap: ex(/--swap-space\s+(\d+)/) || '',
     dtype: ex(/--dtype\s+(\w+)/) || 'auto',
+    vllm_kv_cache_dtype: ex(/--kv-cache-dtype\s+([\w.-]+)/) || 'auto',
     max_seqs: ex(/--max-num-seqs\s+(\d+)/) || '',
     gpus: ex(/CUDA_VISIBLE_DEVICES=(\S+)/) || '',
     cache_type: ex(/(?:--cache-type-k|-ctk)\s+(\S+)/) || '',
@@ -1326,7 +1495,7 @@ export function _renderRunningTab() {
   // event but the matching clear only ran on modal-open, so the highlight
   // persisted indefinitely after tasks finished in the background.
   try {
-    const _activeTasks = _loadTasks().filter(t => t.status === 'running' || t.status === 'queued' || t.status === 'error');
+    const _activeTasks = _loadPrunedTasks().filter(t => t.status === 'running' || t.status === 'queued' || t.status === 'error');
     if (!_activeTasks.length) _clearCookbookNotif();
   } catch {}
 
@@ -1600,6 +1769,8 @@ export function _renderRunningTab() {
         const label = check.querySelector('.cookbook-task-done-label');
         if (label) label.textContent = _clearPillLabel(task);
       }
+      const startNow = el.querySelector('.cookbook-task-start-now');
+      if (startNow) startNow.style.display = (task.type === 'download' && task.status === 'queued') ? '' : 'none';
       const terminalDiag = _terminalServeDiagnosis(task, el.querySelector('.cookbook-output-pre')?.textContent || task.output || '');
       if (terminalDiag) _showDiagnosis(el, terminalDiag, el.querySelector('.cookbook-output-pre')?.textContent || task.output || '');
     }
@@ -1626,6 +1797,7 @@ export function _renderRunningTab() {
         <span class="cookbook-task-type${(task.status === 'done' && task.type === 'download') ? ' cookbook-task-type-done' : ''}" data-type="${esc(task.type)}">${esc((task.status === 'done' && task.type === 'download') ? 'finished' : task.type)}</span>
         <span class="cookbook-task-name">${modelLogo(task.name)}${esc(task.name)}</span>
         <span class="cookbook-task-indicator"><span class="cookbook-task-wave" style="display:${task.status === 'running' ? '' : 'none'}"></span><span class="cookbook-task-check" title="Clear" style="display:${_canClearTask(task) ? '' : 'none'}"><svg class="cookbook-task-check-ico" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="#50fa7b" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><polyline points="20 6 9 17 4 12"/></svg><svg class="cookbook-task-clear-ico" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg><span class="cookbook-task-done-label">${esc(_clearPillLabel(task))}</span><span class="cookbook-task-clear-label">clear</span></span></span>
+        <button type="button" class="cookbook-task-start-now" title="Start this queued download now" style="display:${(task.type === 'download' && task.status === 'queued') ? '' : 'none'}"><svg width="11" height="11" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><polygon points="8 5 19 12 8 19 8 5"/></svg><span>start now</span></button>
         <span class="cookbook-task-status ${_bdg.cls}"${_bdgTitle}>${esc(_bdg.text)}</span>
         <button class="cookbook-task-menu-btn" title="Actions">&#8942;</button>
       </div>
@@ -1702,6 +1874,14 @@ export function _renderRunningTab() {
       });
     }
 
+    const _startNowBtn = el.querySelector('.cookbook-task-start-now');
+    if (_startNowBtn) {
+      _startNowBtn.addEventListener('click', (e) => {
+        e.stopPropagation();
+        _startQueuedDownload(task);
+      });
+    }
+
     // Wire header click to collapse/expand output
     el.querySelector('.cookbook-task-header').addEventListener('click', (e) => {
       if (e.target.closest('button')) return;
@@ -1802,8 +1982,7 @@ export function _renderRunningTab() {
         // serve to the model-endpoints list regardless of prior flag state.
         if (task.type === 'serve' && task.payload?._cmd) {
           items.push({ label: 'Register endpoint', action: 'register-endpoint', custom: async () => {
-            const rawHost = task.remoteHost || 'localhost';
-            const host = rawHost.includes('@') ? rawHost.split('@').pop() : rawHost;
+            const host = _connectHostFromRemote(task.remoteHost);
             const portMatch = task.payload?._cmd?.match(/--port\s+(\d+)/);
             const port = portMatch ? portMatch[1] : '8000';
             const baseUrl = `http://${host}:${port}/v1`;
@@ -1826,6 +2005,7 @@ export function _renderRunningTab() {
               fd.append('base_url', baseUrl);
               fd.append('name', task.name);
               fd.append('skip_probe', 'true');
+              _appendCookbookEndpointScope(fd, task.remoteHost || '');
               if (task.payload?._cmd?.includes('diffusion_server')) fd.append('model_type', 'image');
               const res = await fetch('/api/model-endpoints', { method: 'POST', credentials: 'same-origin', body: fd });
               if (res.ok) {
@@ -1986,13 +2166,20 @@ export function _renderRunningTab() {
       const badge = el.querySelector('.cookbook-task-status');
       if (badge) { badge.textContent = 'stopping...'; badge.className = 'cookbook-task-status cookbook-task-stopping'; }
       el.dataset.status = 'stopped';
+      const outputText = el.querySelector('.cookbook-output-pre')?.textContent || task.output || '';
       // Drop the model endpoint so the picker stops listing it.
       if (task.type === 'serve' && task.payload) {
-        const rawHost = task.remoteHost || 'localhost';
-        const host = rawHost.includes('@') ? rawHost.split('@').pop() : rawHost;
-        const portMatch = task.payload._cmd?.match(/--port\s+(\d+)/);
-        const port = portMatch ? portMatch[1] : '8000';
-        _removeEndpointByUrl(`http://${host}:${port}/v1`);
+        _removeEndpointByUrl(_endpointUrlForTask(task, outputText));
+      }
+      const ollamaUnload = _ollamaUnloadCommand(task, outputText);
+      if (ollamaUnload) {
+        try {
+          await fetch('/api/shell/exec', {
+            method: 'POST', credentials: 'same-origin',
+            headers: { 'Content-Type': 'application/json' },
+            body: JSON.stringify({ command: ollamaUnload }),
+          });
+        } catch {}
       }
       // Gracefully stop (C-c, then kill the session) so it's fully down...
       try {
@@ -2009,23 +2196,29 @@ export function _renderRunningTab() {
 
     // Wire kill
     el.querySelector('.cookbook-task-action-kill').addEventListener('click', () => {
+      const outputText = el.querySelector('.cookbook-output-pre')?.textContent || task.output || '';
+      const ollamaUnload = _ollamaUnloadCommand(task, outputText);
+      if (ollamaUnload) {
+        fetch('/api/shell/exec', {
+          method: 'POST', credentials: 'same-origin',
+          headers: { 'Content-Type': 'application/json' },
+          body: JSON.stringify({ command: ollamaUnload }),
+        }).catch(() => {});
+      }
       fetch('/api/shell/exec', {
         method: 'POST', credentials: 'same-origin',
         headers: { 'Content-Type': 'application/json' },
         body: JSON.stringify({ command: _tmuxGracefulKill(task) }),
       }).catch(() => {});
       if (task.type === 'serve' && task.payload) {
-        const rawHost = task.remoteHost || 'localhost';
-        const host = rawHost.includes('@') ? rawHost.split('@').pop() : rawHost;
-        const portMatch = task.payload._cmd?.match(/--port\s+(\d+)/);
-        const port = portMatch ? portMatch[1] : '8000';
-        _removeEndpointByUrl(`http://${host}:${port}/v1`);
+        const endpointUrl = _endpointUrlForTask(task, outputText);
+        _removeEndpointByUrl(endpointUrl);
         const modelName = task.payload.model || task.name || '';
         if (modelName) {
           fetch('/api/model-endpoints', { credentials: 'same-origin' })
             .then(r => r.json())
             .then(eps => {
-              const ep = eps.find(e => e.name === modelName || (e.base_url && e.base_url.includes(':' + port)));
+              const ep = eps.find(e => e.name === modelName || e.base_url === endpointUrl);
               if (ep) fetch(`/api/model-endpoints/${ep.id}`, { method: 'DELETE', credentials: 'same-origin' }).then(() => _refreshModelsAfterEndpointChange());
             }).catch(() => {});
         }
@@ -2168,6 +2361,33 @@ async function _reconnectTask(el, task) {
                 fixes: [{ label: 'Edit serve', action: (panel) => _openServeEditForTask(task) }],
               };
               _showDiagnosis(el, diag, lastOutput);
+            } else if (task.type === 'download') {
+              const isDisk = /no space left|disk quota|enospc/i.test(lastOutput);
+              const isNetwork = /connection|timeout|timed out|incompleteread|chunkedencoding|reset by peer|protocolerror|all connection attempts failed/i.test(lastOutput);
+              const progressMatch = String(lastOutput || '').match(/(\d+)%\|/);
+              const nearDone = progressMatch && Number(progressMatch[1]) >= 80;
+              const diag = {
+                message: isDisk
+                  ? 'Download stopped because this server ran out of disk space.'
+                  : isNetwork
+                  ? 'Download stopped after the HuggingFace connection was interrupted.'
+                  : nearDone
+                  ? 'Download stopped near the end before the final completion marker was captured.'
+                  : 'Download stopped before HuggingFace reported completion.',
+                suggestion: isDisk
+                  ? 'Suggested action: free disk space, then retry the download. HuggingFace resumes incomplete files when possible.'
+                  : nearDone
+                  ? 'Suggested action: retry the download. It may briefly look like it restarted while cached files are checked, then it should reuse incomplete files.'
+                  : 'Suggested action: retry the download. HuggingFace resumes incomplete files when possible.',
+                fixes: [
+                  { label: 'Retry download', action: () => _retryTask(el, task) },
+                  { label: 'Copy last 50 lines', action: () => {
+                    const last = String(lastOutput || '').split('\n').slice(-50).join('\n');
+                    _copyText(last || 'No download log available.');
+                  } },
+                ],
+              };
+              _showDiagnosis(el, diag, lastOutput);
             }
             _showCookbookNotif(true);
           } else {
@@ -2175,7 +2395,7 @@ async function _reconnectTask(el, task) {
             el.dataset.status = 'done';
             const badge = el.querySelector('.cookbook-task-status');
             if (badge) { badge.textContent = _statusLabel('done', task.type); badge.className = 'cookbook-task-status cookbook-task-done'; }
-            const _chk = el.querySelector('.cookbook-task-check'); if (_chk && task.type !== 'download') _chk.style.display = '';
+            const _chk = el.querySelector('.cookbook-task-check'); if (_chk) _chk.style.display = '';
             const _sb = el.querySelector('.cookbook-task-serve-btn'); if (_sb) _sb.style.display = '';
             _showCookbookNotif();
             _refreshDepsAfterInstall(task);
@@ -2444,8 +2664,7 @@ async function _reconnectTask(el, task) {
         // first one's dedup check can observe the newly-added row.
         if (task.type === 'serve' && !task._endpointAdded && !task._endpointAddInFlight && task._serveReady) {
           task._endpointAddInFlight = true;
-          const rawHost = task.remoteHost || 'localhost';
-          let host = rawHost.includes('@') ? rawHost.split('@').pop() : rawHost;
+          let host = _connectHostFromRemote(task.remoteHost);
           const portMatch = task.payload?._cmd?.match(/--port[=\s]+(\d+)/)
             || task.payload?._cmd?.match(/(?:^|\s)-p[=\s]+(\d+)/)
             || snapshot.match(/Uvicorn running on\D*?:(\d+)/i)
@@ -2456,12 +2675,8 @@ async function _reconnectTask(el, task) {
           let baseUrl = `http://${host}:${port}/v1`;
           const ollamaUrlMatch = snapshot.match(/Ollama API ready on port\s+\d+:\s*(http:\/\/[^\s]+)/i);
           if (ollamaUrlMatch) {
-            try {
-              const u = new URL(ollamaUrlMatch[1]);
-              host = u.hostname || host;
-              port = u.port || '11434';
-              baseUrl = `${u.origin}/v1`;
-            } catch {}
+            const endpoint = _endpointFromAdvertisedUrl(ollamaUrlMatch[1], host, '11434');
+            if (endpoint) ({ host, port, baseUrl } = endpoint);
           }
           fetch('/api/model-endpoints', { credentials: 'same-origin' })
             .then(r => r.json())
@@ -2489,6 +2704,7 @@ async function _reconnectTask(el, task) {
               fd.append('base_url', baseUrl);
               fd.append('name', task.name);
               fd.append('skip_probe', 'true');
+              _appendCookbookEndpointScope(fd, task.remoteHost || '');
               if (_isDiffusion) fd.append('model_type', 'image');
               return fetch('/api/model-endpoints', { method: 'POST', credentials: 'same-origin', body: fd });
             })
@@ -2592,8 +2808,7 @@ async function _checkServeReachability() {
     ]);
   } catch { return; }
   for (const task of serveTasks) {
-    const rawHost = task.remoteHost || 'localhost';
-    const host = rawHost.includes('@') ? rawHost.split('@').pop() : rawHost;
+    const host = _connectHostFromRemote(task.remoteHost);
     const portMatch = task.payload?._cmd?.match(/--port\s+(\d+)/);
     const port = portMatch ? portMatch[1] : '8000';
     const baseUrl = `http://${host}:${port}/v1`;
@@ -2804,13 +3019,24 @@ async function _pollBackgroundStatus() {
         const updates = {};
         const nextStatus = live.status === 'completed'
           ? 'done'
-          : (live.status === 'error' ? 'error' : null);
+          : (live.status === 'error'
+            ? 'error'
+            : (live.status === 'stopped' ? (task.type === 'download' ? 'crashed' : 'stopped') : null));
         if (nextStatus && task.status !== nextStatus) {
           updates.status = nextStatus;
           if (nextStatus === 'done' && task.payload?._dep) completedDeps.push(task);
         }
+        if ((live.status === 'running' || live.status === 'ready') && task.status !== live.status) {
+          updates.status = live.status === 'ready' ? 'ready' : 'running';
+        }
         if (live.progress && live.progress !== task.progress) updates.progress = live.progress;
-        if (live.output_tail && live.output_tail !== task.output) updates.output = live.output_tail;
+        if (live.output_tail) {
+          const previous = String(task.output || '');
+          const tail = String(live.output_tail || '');
+          if (tail && !previous.endsWith(tail)) {
+            updates.output = `${previous ? `${previous}\n` : ''}${tail}`.slice(-5000);
+          }
+        }
         if (Object.keys(updates).length) {
           Object.assign(task, updates);
           changed = true;
@@ -2835,8 +3061,7 @@ async function _pollBackgroundStatus() {
       const localTask = localTasks.find(lt => lt.sessionId === t.session_id);
       if (localTask && localTask._endpointAdded) continue;
 
-      const rawHost = localTask?.remoteHost || t.remote || 'localhost';
-      let host = rawHost.includes('@') ? rawHost.split('@').pop() : (rawHost === 'local' ? 'localhost' : rawHost);
+      let host = _connectHostFromRemote(localTask?.remoteHost || t.remote);
       const portMatch = localTask?.payload?._cmd?.match(/--port\s+(\d+)/)
         || localTask?.payload?._cmd?.match(/OLLAMA_HOST=[^\s:]+:(\d+)/);
       let port = portMatch ? portMatch[1] : '8000';
@@ -2844,12 +3069,8 @@ async function _pollBackgroundStatus() {
       const snapshot = t.output || localTask?.output || '';
       const ollamaUrlMatch = snapshot.match(/Ollama API ready on port\s+\d+:\s*(http:\/\/[^\s]+)/i);
       if (ollamaUrlMatch) {
-        try {
-          const u = new URL(ollamaUrlMatch[1]);
-          host = u.hostname || host;
-          port = u.port || '11434';
-          baseUrl = `${u.origin}/v1`;
-        } catch {}
+        const endpoint = _endpointFromAdvertisedUrl(ollamaUrlMatch[1], host, '11434');
+        if (endpoint) ({ host, port, baseUrl } = endpoint);
       }
       const _isDiffusion = localTask?.payload?._cmd?.includes('diffusion_server');
 
@@ -2880,6 +3101,7 @@ async function _pollBackgroundStatus() {
           fd.append('base_url', baseUrl);
           fd.append('name', t.model);
           fd.append('skip_probe', 'true');
+          _appendCookbookEndpointScope(fd, localTask?.remoteHost || t.remote || '');
           if (_isDiffusion) fd.append('model_type', 'image');
           if (_supportsTools) fd.append('supports_tools', 'true');
           return fetch('/api/model-endpoints', { method: 'POST', credentials: 'same-origin', body: fd });