fix(integrations): prevent blank API integrations (#3840)

* fix(integrations): validate unified API form fields

* fix(integrations): validate API integration fields server-side

.dockerignore

@@ -10,6 +10,12 @@ dist/
 build/
 .env
 .env.bak.*
+# Secrets: keep plaintext and every transient secrets.env variant out of
+# the build context. If an encrypted secrets.env is used, it is mounted
+# at runtime — never baked into the image. Mirrored in .gitignore.
+secrets.env
+secrets.env.*
+!secrets.env.example
 /data/
 /logs/
 .git/

.github/CODEOWNERS

@@ -1,8 +1,9 @@
 # Code owners.
 #
-# Every file is owned by the maintainer, so that when branch protection has
-# "Require review from Code Owners" turned on, no pull request can be merged
-# without the maintainer's review. This is the human gate that backs up the
-# automated security checks. See docs/security-ci.md for how to turn it on.
-
-*       @pewdiepie-archdaemon
+# Intentionally empty for now. The catch-all rule that mapped every path to a
+# single owner froze all merges the moment "Require review from Code Owners"
+# was enabled, because no other maintainer's approval could satisfy the gate.
+# A per-area ownership map (security/auth, CI, frontend, agent internals, with
+# multiple named owners per line) is being worked out in issue #593; once
+# agreed it replaces this file. Until then, required reviews and the security
+# CI gate (docs/security-ci.md) remain in force via branch protection.

.gitignore

@@ -14,6 +14,15 @@ venv/
 .env
 .env.bak.*
 !.env.example
+# Local uv lockfile (optional, per-platform — see "Faster installs with uv" in README)
+requirements.lock
+
+# SOPS workflow — encrypted `secrets.env` is intentionally committable,
+# but every variant (plaintext, manual decrypt copy, editor backup)
+# must stay out of git. Mirrored in .dockerignore so the same artifacts
+# also cannot enter image build layers.
+secrets.env.*
+!secrets.env.example
 
 # Data — all user data stays local
 data/

README.md

@@ -73,6 +73,10 @@ binds the web UI to `127.0.0.1` by default. If the port is taken, set
 `APP_PORT=7001` in `.env` and recreate the container. Set `APP_BIND=0.0.0.0`
 only when you intentionally want LAN/reverse-proxy access.
 
+> **On Apple Silicon (M-series) Macs:** Docker can't reach the Metal GPU, so
+> Cookbook serves local models on CPU only. For GPU-accelerated model serving,
+> run natively instead — see [Apple Silicon](#apple-silicon) below.
+
 ### Native Linux / macOS
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
@@ -333,6 +337,25 @@ To expose Odysseus on a local network or Tailscale with HTTPS:
 | `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
 | `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
 
+### Faster, reproducible installs with uv (optional)
+[uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
+venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
+
+```bash
+uv venv venv --python 3.13
+uv pip install -r requirements.txt
+# then continue as usual: python setup.py, uvicorn, ...
+```
+
+`requirements.txt` is intentionally unpinned, so two installs at different times can produce different package versions. If you want a reproducible environment (e.g. across your own machines, or to roll back after a bad upgrade), snapshot and restore exact versions with:
+
+```bash
+uv pip compile requirements.txt -o requirements.lock   # snapshot current resolution
+uv pip sync requirements.lock                          # reproduce it exactly later
+```
+
+`requirements.lock` is gitignored and platform-specific (compile it on the OS you deploy to). Regenerate it deliberately when you want to take upgrades. The plain `uv pip install -r requirements.txt` keeps following the unpinned requirements like pip does.
+
 ### Outlook / Office 365 email
 Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
 and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
@@ -364,6 +387,7 @@ Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and th
 4. Keep raw service and model ports internal-only.
 
 Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
+`ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
 
 Common internal-only ports from the default docs/compose setup:
 
@@ -397,6 +421,7 @@ Key settings:
 | `APP_PORT` | `7000` | Docker Compose host port for the web UI. |
 | `AUTH_ENABLED` | `true` | Enable/disable login |
 | `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
+| `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
 | `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
 | `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
 | `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |

routes/chat_helpers.py

@@ -159,9 +159,17 @@ async def auto_name_session(session_manager, sess):
             return
 
         owner = getattr(sess, "owner", None)
-        t_url, t_model, t_headers = resolve_task_endpoint(
-            sess.endpoint_url, sess.model, sess.headers, owner=owner,
-        )
+        t_url, t_model, t_headers = resolve_task_endpoint(owner=owner)
+        if not t_model:
+            # If no task/utility model is configured at all, fall back to
+            # the session's own model so auto-naming still works even on
+            # minimal setups.
+            from src.endpoint_resolver import resolve_endpoint
+            _fallback = resolve_endpoint("default", owner=owner)
+            if _fallback and _fallback[1]:
+                t_url, t_model, t_headers = _fallback
+            else:
+                t_url, t_model, t_headers = sess.endpoint_url, sess.model, sess.headers
         if not t_model:
             logger.debug("[auto-name] No model provided, skipping")
             return

routes/chat_routes.py

@@ -696,7 +696,12 @@ def setup_chat_routes(
         # by default without having to send allow_bash in every request.
         if allow_bash is not None and str(allow_bash).lower() != "true":
             disabled_tools.add("bash")
-        if allow_web_search is not None and str(allow_web_search).lower() != "true":
+        _explicit_web_intent = bool(_tool_intent and _tool_intent.category == "web")
+        if (
+            allow_web_search is not None
+            and str(allow_web_search).lower() != "true"
+            and not _explicit_web_intent
+        ):
             disabled_tools.add("web_search")
             disabled_tools.add("web_fetch")
 

routes/codex_routes.py

@@ -18,6 +18,7 @@ from fastapi.responses import StreamingResponse
 from src.auth_helpers import require_authenticated_request, require_user
 from src.tool_implementations import do_manage_notes
 from src.constants import COOKBOOK_STATE_FILE
+from routes._validators import validate_remote_host, validate_ssh_port
 
 
 COOKBOOK_READ_SCOPES = {"cookbook:read", "cookbook:launch"}
@@ -36,6 +37,21 @@ DOCS_WRITE_SCOPES = {"documents:write"}
 WRITE_ACTIONS = {"add", "create", "new", "save", "remind", "update", "delete", "toggle_item", "remove", "remove_item"}
 
 
+def _ssh_prefix_for_task(task: dict) -> tuple[str, str]:
+    """Resolve a cookbook task's stored SSH target into ``(host, port_flag)``.
+
+    ``host`` is ``""`` for a local task. ``remoteHost`` / ``sshPort`` come from
+    cookbook_state.json and get interpolated into an ``ssh`` command string, so
+    validate them the same way the cookbook routes do. A tampered entry with
+    shell metacharacters in ``remoteHost`` is rejected with 400 rather than
+    injected.
+    """
+    host = validate_remote_host((task.get("remoteHost") or "").strip() or None) or ""
+    ssh_port = validate_ssh_port((task.get("sshPort") or "").strip() or None) or ""
+    port_flag = f"-p {ssh_port} " if ssh_port and ssh_port != "22" else ""
+    return host, port_flag
+
+
 async def _as_owner(request: Request, owner: str, fn, *args, **kwargs):
     """Run an existing route handler with request.state.current_user temporarily
     set to ``owner`` so its internal get_current_user/require_user calls see
@@ -486,8 +502,7 @@ def setup_codex_routes(
         task = next((t for t in tasks if t.get("sessionId") == session_id), None)
         if task is None:
             raise HTTPException(404, "task not found")
-        host = (task.get("remoteHost") or "").strip()
-        ssh_port = (task.get("sshPort") or "").strip()
+        host, port_flag = _ssh_prefix_for_task(task)
         # Prefer the persisted log file over the tmux pane. The pane gets
         # overwritten by the post-crash neofetch banner + bash prompt the
         # moment vllm exits; the log file is the raw stdout/stderr and
@@ -499,7 +514,6 @@ def setup_codex_routes(
             f"else tmux capture-pane -t {session_id} -p -S -{tail}; fi"
         )
         if host:
-            port_flag = f"-p {ssh_port} " if ssh_port and ssh_port != "22" else ""
             import shlex
             cmd = f"ssh {port_flag}{host} {shlex.quote(inner)}"
         else:
@@ -561,10 +575,8 @@ def setup_codex_routes(
         state = _read_cookbook_state()
         tasks = state.get("tasks") or []
         task = next((t for t in tasks if t.get("sessionId") == session_id), None)
-        host = ((task or {}).get("remoteHost") or "").strip()
-        ssh_port = ((task or {}).get("sshPort") or "").strip()
+        host, port_flag = _ssh_prefix_for_task(task or {})
         if host:
-            port_flag = f"-p {ssh_port} " if ssh_port and ssh_port != "22" else ""
             cmd = f"ssh {port_flag}{host} \"tmux kill-session -t {session_id}\""
         else:
             cmd = f"tmux kill-session -t {session_id}"

routes/cookbook_helpers.py

@@ -573,6 +573,36 @@ _GGUF_PRELUDE_RE = re.compile(
 _OLLAMA_HOST_ASSIGNMENT_RE = re.compile(r"(?:^|\s)OLLAMA_HOST=([^\s]+)")
 _OLLAMA_BIND_RE = re.compile(r"^\[([^\]]+)\]:(\d+)$|^([^:]+):(\d+)$")
 _OLLAMA_BIND_HOST_RE = re.compile(r"^[A-Za-z0-9._:-]+$")
+_LLAMA_CPP_PYTHON_GGML_TYPES = {
+    "f32": "0",
+    "f16": "1",
+    "q4_0": "2",
+    "q4_1": "3",
+    "q5_0": "6",
+    "q5_1": "7",
+    "q8_0": "8",
+    "q8_1": "9",
+    "q2_k": "10",
+    "q3_k": "11",
+    "q4_k": "12",
+    "q5_k": "13",
+    "q6_k": "14",
+    "q8_k": "15",
+    "iq2_xxs": "16",
+    "iq2_xs": "17",
+    "iq3_xxs": "18",
+    "iq1_s": "19",
+    "iq4_nl": "20",
+    "iq3_s": "21",
+    "iq2_s": "22",
+    "iq4_xs": "23",
+    "mxfp4": "39",
+    "nvfp4": "40",
+    "q1_0": "41",
+}
+_LLAMA_CPP_PYTHON_TYPE_FLAG_RE = re.compile(
+    r"(?P<flag>--type_[kv])(?P<sep>\s+|=)(?P<quote>['\"]?)(?P<value>[A-Za-z0-9_]+)(?P=quote)"
+)
 
 
 def _ollama_bind_from_cmd(cmd: str | None, *, default_host: str = "127.0.0.1") -> tuple[str, str]:
@@ -604,6 +634,22 @@ def _ollama_bind_from_cmd(cmd: str | None, *, default_host: str = "127.0.0.1") -
     return f"[{host}]" if bracketed_host else host, port
 
 
+def _normalize_llama_cpp_python_cache_types(cmd: str | None) -> str | None:
+    """Map llama.cpp KV cache type names to llama-cpp-python's integer enum."""
+    if not cmd or "llama_cpp.server" not in cmd:
+        return cmd
+
+    def repl(match: re.Match[str]) -> str:
+        value = match.group("value")
+        mapped = _LLAMA_CPP_PYTHON_GGML_TYPES.get(value.lower())
+        if not mapped:
+            return match.group(0)
+        quote = match.group("quote")
+        return f"{match.group('flag')}{match.group('sep')}{quote}{mapped}{quote}"
+
+    return _LLAMA_CPP_PYTHON_TYPE_FLAG_RE.sub(repl, cmd)
+
+
 def _check_serve_binary(seg: str) -> None:
     """Validate that a single command segment starts with an allowlisted binary
     (after skipping leading env-var assignments like `CUDA_VISIBLE_DEVICES=0`)."""
@@ -742,6 +788,7 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
     runner_lines.append('    done')
     # rm -rf build so a prior poisoned CMakeCache.txt (e.g. from a failed CUDA
     # or HIP attempt) doesn't cause the next configure to reuse stale settings.
+    runner_lines.append('    mkdir -p ~/bin')
     runner_lines.append('    cd ~/llama.cpp && rm -rf build')
     runner_lines.append('    if command -v hipconfig &>/dev/null || [ -d /opt/rocm ] || [ -n "$ROCM_PATH" ] || [ -n "$HIP_PATH" ]; then')
     runner_lines.append('      if command -v hipconfig &>/dev/null; then')
@@ -1046,6 +1093,16 @@ def _diagnose_serve_output(text: str) -> dict | None:
             "vLLM is not installed or not in PATH on this server.",
             [{"label": "install vLLM in Cookbook Dependencies", "op": "dependency", "package": "vllm"}],
         ),
+        (
+            r"sgl_kernel[\s\S]*(Python\.h|libnuma\.so\.1|common_ops)|"
+            r"(Python\.h|libnuma\.so\.1|common_ops)[\s\S]*sgl_kernel|"
+            r"Please ensure sgl_kernel is properly installed",
+            "SGLang native dependencies are missing on this server.",
+            [
+                {"label": "install OS packages: libnuma-dev python3.12-dev build-essential", "op": "manual"},
+                {"label": "upgrade sglang-kernel after OS packages are installed", "op": "manual"},
+            ],
+        ),
         (
             r"sglang.*command not found|No module named sglang|SGLang is not installed",
             "SGLang is not installed or not in PATH on this server.",

routes/cookbook_output.py

@@ -4,6 +4,62 @@ Kept dependency-free (no FastAPI / SQLAlchemy imports) so the behavior can be
 unit-tested without standing up the whole app.
 """
 
+import re
+
+_FETCHING_ZERO_FILES_RE = re.compile(r"Fetching\s+0\s+files", re.IGNORECASE)
+
+# Probe scripts for the dead-session download check, run as
+# `python3 -c <PROBE> <repo_id> <cache_root>` (locally or over SSH).
+# cache_root is the task's custom download dir, '' for the default HF cache.
+# It has to be passed explicitly: the download runner exports
+# HF_HOME=<local_dir>, so that task's cache lives under <local_dir>/hub, and
+# the probe process's own environment knows nothing about it.
+HF_CACHE_COMPLETE_PROBE = (
+    "import os,sys;"
+    "repo=sys.argv[1];"
+    "root=os.path.expanduser(sys.argv[2]) if len(sys.argv)>2 and sys.argv[2] else '';"
+    "base=os.path.join(root,'hub') if root else (os.environ.get('HUGGINGFACE_HUB_CACHE') or os.path.join(os.environ.get('HF_HOME', os.path.expanduser('~/.cache/huggingface')), 'hub'));"
+    "d=os.path.join(base,'models--'+repo.replace('/','--'));"
+    "snap=os.path.join(d,'snapshots');"
+    "ok=os.path.isdir(snap) and any(os.path.isdir(os.path.join(snap,x)) and os.listdir(os.path.join(snap,x)) for x in os.listdir(snap));"
+    "inc=False;"
+    "blobs=os.path.join(d,'blobs');"
+    "inc=os.path.isdir(blobs) and any(x.endswith('.incomplete') for x in os.listdir(blobs));"
+    "sys.exit(0 if ok and not inc else 1)"
+)
+
+HF_CACHE_INCOMPLETE_PROBE = (
+    "import os,sys;"
+    "repo=sys.argv[1];"
+    "root=os.path.expanduser(sys.argv[2]) if len(sys.argv)>2 and sys.argv[2] else '';"
+    "base=os.path.join(root,'hub') if root else (os.environ.get('HUGGINGFACE_HUB_CACHE') or os.path.join(os.environ.get('HF_HOME', os.path.expanduser('~/.cache/huggingface')), 'hub'));"
+    "d=os.path.join(base,'models--'+repo.replace('/','--'));"
+    "blobs=os.path.join(d,'blobs');"
+    "inc=os.path.isdir(blobs) and any(x.endswith('.incomplete') for x in os.listdir(blobs));"
+    "sys.exit(0 if inc else 1)"
+)
+
+
+def classify_dead_download(full_snapshot: str):
+    """Resolve a dead download session's status from its runner markers.
+
+    The runner prints DOWNLOAD_OK only after exiting 0 (and DOWNLOAD_FAILED
+    otherwise), so the markers stay trustworthy after the tmux pane is gone.
+    Returns (status, zero_files), or None when the snapshot carries no marker
+    and the caller has to fall back to the cache probe. Same precedence as
+    the live-session branch: DOWNLOAD_OK wins, except a "Fetching 0 files"
+    run is an error (nothing matched the include/quant pattern).
+    """
+    if not full_snapshot:
+        return None
+    if "DOWNLOAD_OK" in full_snapshot:
+        if _FETCHING_ZERO_FILES_RE.search(full_snapshot):
+            return ("error", True)
+        return ("completed", False)
+    if "DOWNLOAD_FAILED" in full_snapshot:
+        return ("error", False)
+    return None
+
 
 def error_aware_output_tail(full_snapshot: str, status: str) -> str:
     """Return the trailing slice of a task log for the status response.

routes/cookbook_routes.py

@@ -30,7 +30,10 @@ from core.platform_compat import (
     which_tool,
 )
 from routes.shell_routes import TMUX_LOG_DIR
-from routes.cookbook_output import error_aware_output_tail
+from routes.cookbook_output import (
+    error_aware_output_tail, classify_dead_download,
+    HF_CACHE_COMPLETE_PROBE, HF_CACHE_INCOMPLETE_PROBE,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -46,6 +49,7 @@ from routes.cookbook_helpers import (
     _diagnose_serve_output, run_ssh_command_async,
     _ollama_bind_from_cmd, _pip_install_fallback_chain, _pip_install_no_cache,
     _user_shell_path_bootstrap, _venv_safe_local_pip_install_cmd,
+    _normalize_llama_cpp_python_cache_types,
     ModelDownloadRequest, ServeRequest,
 )
 
@@ -54,7 +58,7 @@ _HF_TOKEN_STATUS_SNIPPET = (
     'echo "[odysseus] HF token: applied"; '
     'else '
     'echo "[odysseus] HF token: NOT SET — gated/private models will be denied. '
-    'Add one in Odysseus Settings -> Cookbook -> HuggingFace Token."; '
+    'Add one in Odysseus Cookbook -> Settings -> HuggingFace Token."; '
     'fi'
 )
 
@@ -170,6 +174,16 @@ def setup_cookbook_routes() -> APIRouter:
                 "vLLM is not installed or not in PATH on this server.",
                 [{"label": "install vLLM in Cookbook Dependencies", "op": "dependency", "package": "vllm"}],
             ),
+            (
+                r"sgl_kernel[\s\S]*(Python\.h|libnuma\.so\.1|common_ops)|"
+                r"(Python\.h|libnuma\.so\.1|common_ops)[\s\S]*sgl_kernel|"
+                r"Please ensure sgl_kernel is properly installed",
+                "SGLang native dependencies are missing on this server.",
+                [
+                    {"label": "install OS packages: libnuma-dev python3.12-dev build-essential", "op": "manual"},
+                    {"label": "upgrade sglang-kernel after OS packages are installed", "op": "manual"},
+                ],
+            ),
             (
                 r"sglang.*command not found|No module named sglang|SGLang is not installed",
                 "SGLang is not installed or not in PATH on this server.",
@@ -353,7 +367,11 @@ def setup_cookbook_routes() -> APIRouter:
             # all output to the log the poller reads. Paths handed to bash use
             # POSIX form + shell-quoting so drive paths / spaces survive.
             inner = TMUX_LOG_DIR / f"{session_id}_run.sh"
-            inner.write_text("\n".join(bash_lines) + "\n", encoding="utf-8")
+            pp = shlex.quote(pid_path.as_posix())
+            inner.write_text(
+                f"printf '%s\\n' \"$$\" > {pp}\n" + "\n".join(bash_lines) + "\n",
+                encoding="utf-8",
+            )
             lp = shlex.quote(log_path.as_posix())
             ip = shlex.quote(inner.as_posix())
             script_path = TMUX_LOG_DIR / f"{session_id}.sh"
@@ -1211,6 +1229,7 @@ def setup_cookbook_routes() -> APIRouter:
         # many downstream `"engine" in req.cmd` membership checks can't hit
         # `TypeError: argument of type 'NoneType'` (a 500 instead of a clean 400).
         req.cmd = _validate_serve_cmd(req.cmd) or ""
+        req.cmd = _normalize_llama_cpp_python_cache_types(req.cmd) or ""
         req.cmd = _venv_safe_local_pip_install_cmd(
             req.cmd,
             local=not bool(req.remote_host),
@@ -2620,30 +2639,20 @@ def setup_cookbook_routes() -> APIRouter:
     def _cookbook_tasks_status_sync():
         import subprocess
 
-        def _download_cache_complete(repo_id: str, remote_host: str = "", ssh_port: str = "") -> bool:
+        def _download_cache_complete(repo_id: str, remote_host: str = "", ssh_port: str = "", cache_root: str = "") -> bool:
             """Best-effort check for a completed HF cache entry.
 
             tmux output can stop at a stale progress line if the pane/session
             disappears before Cookbook captures the final DOWNLOAD_OK marker.
             In that case, trust the cache shape: a snapshot directory with files
             and no *.incomplete blobs means HuggingFace finished materializing the
-            model.
+            model. cache_root is the task's custom download dir — the runner
+            pointed HF_HOME there, so the cache lives under <cache_root>/hub,
+            not wherever this probe's environment says.
             """
             if not repo_id or "/" not in repo_id:
                 return False
-            py = (
-                "import os,sys;"
-                "repo=sys.argv[1];"
-                "base=os.environ.get('HUGGINGFACE_HUB_CACHE') or os.path.join(os.environ.get('HF_HOME', os.path.expanduser('~/.cache/huggingface')), 'hub');"
-                "d=os.path.join(base,'models--'+repo.replace('/','--'));"
-                "snap=os.path.join(d,'snapshots');"
-                "ok=os.path.isdir(snap) and any(os.path.isdir(os.path.join(snap,x)) and os.listdir(os.path.join(snap,x)) for x in os.listdir(snap));"
-                "inc=False;"
-                "blobs=os.path.join(d,'blobs');"
-                "inc=os.path.isdir(blobs) and any(x.endswith('.incomplete') for x in os.listdir(blobs));"
-                "sys.exit(0 if ok and not inc else 1)"
-            )
-            cmd = ["python3", "-c", py, repo_id]
+            cmd = ["python3", "-c", HF_CACHE_COMPLETE_PROBE, repo_id, cache_root or ""]
             try:
                 if remote_host:
                     ssh_base = ["ssh"]
@@ -2657,7 +2666,7 @@ def setup_cookbook_routes() -> APIRouter:
             except Exception:
                 return False
 
-        def _download_cache_incomplete(repo_id: str, remote_host: str = "", ssh_port: str = "") -> bool:
+        def _download_cache_incomplete(repo_id: str, remote_host: str = "", ssh_port: str = "", cache_root: str = "") -> bool:
             """Best-effort check for resumable HF partial blobs.
 
             A lost SSH/tmux session can leave a real download still incomplete.
@@ -2666,16 +2675,7 @@ def setup_cookbook_routes() -> APIRouter:
             """
             if not repo_id or "/" not in repo_id:
                 return False
-            py = (
-                "import os,sys;"
-                "repo=sys.argv[1];"
-                "base=os.environ.get('HUGGINGFACE_HUB_CACHE') or os.path.join(os.environ.get('HF_HOME', os.path.expanduser('~/.cache/huggingface')), 'hub');"
-                "d=os.path.join(base,'models--'+repo.replace('/','--'));"
-                "blobs=os.path.join(d,'blobs');"
-                "inc=os.path.isdir(blobs) and any(x.endswith('.incomplete') for x in os.listdir(blobs));"
-                "sys.exit(0 if inc else 1)"
-            )
-            cmd = ["python3", "-c", py, repo_id]
+            cmd = ["python3", "-c", HF_CACHE_INCOMPLETE_PROBE, repo_id, cache_root or ""]
             try:
                 if remote_host:
                     ssh_base = ["ssh"]
@@ -2880,7 +2880,7 @@ def setup_cookbook_routes() -> APIRouter:
                 and (
                     ".incomplete" in full_snapshot
                     or bool(re.search(r'model-\d+-of-\d+\.[A-Za-z0-9_.-]+:\s+(?:[0-9]|[1-8][0-9])%', full_snapshot))
-                    or _download_cache_incomplete(_payload.get("repo_id") or model, remote, str(_tport or ""))
+                    or _download_cache_incomplete(_payload.get("repo_id") or model, remote, str(_tport or ""), _payload.get("local_dir") or "")
                 )
             )
             if is_alive or (local_win_task and full_snapshot):
@@ -2921,11 +2921,19 @@ def setup_cookbook_routes() -> APIRouter:
                 else:
                     status = "running"
             else:
-                # Session is dead — check if it completed or crashed
-                if (
+                # Session is dead — check if it completed or crashed. The
+                # runner markers in the retained output are conclusive
+                # (DOWNLOAD_OK only prints after exit 0), so check them before
+                # the cache probe, which can't see ollama pulls at all.
+                marker = classify_dead_download(full_snapshot) if task_type == "download" else None
+                if marker is not None:
+                    status, download_zero_files = marker
+                    if status == "completed" and not progress_text:
+                        progress_text = "Download complete"
+                elif (
                     task_type == "download"
                     and not download_has_incomplete_evidence
-                    and _download_cache_complete(_payload.get("repo_id") or model, remote, str(_tport or ""))
+                    and _download_cache_complete(_payload.get("repo_id") or model, remote, str(_tport or ""), _payload.get("local_dir") or "")
                 ):
                     status = "completed"
                     if not progress_text:

routes/email_routes.py

@@ -1087,7 +1087,10 @@ def setup_email_routes():
             return {"contacts": [], "error": "Mail operation failed"}
 
     @router.get("/search")
-    async def search_emails(
+    # Sync def: the body is blocking IMAP I/O with no awaits. As `async def` it ran
+    # directly on the event loop and stalled the whole app during a search; as a sync
+    # def FastAPI runs it in a threadpool, keeping the loop responsive.
+    def search_emails(
         q: str = Query(""),
         folder: str = Query("INBOX"),
         limit: int = Query(50),
@@ -1736,7 +1739,9 @@ def setup_email_routes():
             return {"success": False, "error": "Mail operation failed"}
 
     @router.post("/archive/{uid}")
-    async def archive_email(uid: str, folder: str = Query("INBOX"), account_id: str | None = Query(None), owner: str = Depends(require_owner)):
+    # Sync def: blocking IMAP I/O with no awaits — see search_emails above. Runs in a
+    # threadpool instead of blocking the event loop.
+    def archive_email(uid: str, folder: str = Query("INBOX"), account_id: str | None = Query(None), owner: str = Depends(require_owner)):
         """Move email to Archive folder."""
         try:
             with _imap(account_id, owner=owner) as conn:

routes/gallery_routes.py

@@ -19,6 +19,7 @@ from src.upload_limits import (
     GALLERY_TRANSFORM_UPLOAD_MAX_BYTES,
 )
 from src.constants import GENERATED_IMAGES_DIR
+from src.optional_deps import patch_realesrgan_torchvision_compat
 
 from routes.gallery_helpers import (
     GalleryPatch, _extract_exif, _image_to_dict, _owner_filter, _human_size,
@@ -108,6 +109,32 @@ def _visible_image_endpoint_for_base(db, base: str, owner: str | None):
     return fallback
 
 
+async def _fetch_result_image_b64(url: str) -> Optional[str]:
+    """Fetch an image URL returned in an upstream response body, base64-encoded
+    (or None on a non-200).
+
+    The URL comes from the diffusion/OpenAI server's response, not from our own
+    config, so a malicious or compromised endpoint could otherwise steer this
+    fetch at an internal or cloud-metadata address. Validate it the same way the
+    client-supplied endpoint is validated before the first request.
+    """
+    import base64
+    import httpx
+    from src.url_safety import check_outbound_url
+
+    ok, reason = check_outbound_url(
+        url,
+        block_private=os.getenv("IMAGE_BLOCK_PRIVATE_IPS", "false").lower() == "true",
+    )
+    if not ok:
+        raise HTTPException(502, f"Upstream returned an unsafe image URL: {reason}")
+    async with httpx.AsyncClient(timeout=60) as c2:
+        ir = await c2.get(url)
+        if ir.status_code == 200:
+            return base64.b64encode(ir.content).decode()
+    return None
+
+
 def setup_gallery_routes() -> APIRouter:
     router = APIRouter(tags=["gallery"])
 
@@ -1142,10 +1169,7 @@ def setup_gallery_routes() -> APIRouter:
                         if item.get("b64_json"):
                             raw_b64 = item["b64_json"]
                         elif item.get("url"):
-                            async with httpx.AsyncClient(timeout=60) as c2:
-                                img_r = await c2.get(item["url"])
-                                if img_r.status_code == 200:
-                                    raw_b64 = base64.b64encode(img_r.content).decode()
+                            raw_b64 = await _fetch_result_image_b64(item["url"])
                     if not raw_b64:
                         raise HTTPException(502, "OpenAI returned no image")
 
@@ -1206,7 +1230,7 @@ def setup_gallery_routes() -> APIRouter:
         original and regenerates `strength` fraction. With strength ~0.4
         you get edge blending + lighting unification while keeping the
         composition recognisable."""
-        import httpx, base64 as _b64
+        import httpx
         user = require_privilege(request, "can_generate_images")
         body = await request.json()
 
@@ -1382,10 +1406,9 @@ def setup_gallery_routes() -> APIRouter:
                             if item.get("b64_json"):
                                 return {"image": item["b64_json"]}
                             if item.get("url"):
-                                async with httpx.AsyncClient(timeout=60) as c2:
-                                    ir = await c2.get(item["url"])
-                                    if ir.status_code == 200:
-                                        return {"image": _b64.b64encode(ir.content).decode()}
+                                img_b64 = await _fetch_result_image_b64(item["url"])
+                                if img_b64:
+                                    return {"image": img_b64}
                     last_err = f"{path}: server returned no image"
                 except httpx.ConnectError as e:
                     raise HTTPException(502, f"Can't reach diffusion server at {base}: {e}")
@@ -1445,6 +1468,7 @@ def setup_gallery_routes() -> APIRouter:
         img_bytes = base64.b64decode(image_b64)
         src = Image.open(io.BytesIO(img_bytes)).convert("RGB")
         try:
+            patch_realesrgan_torchvision_compat()
             from realesrgan import RealESRGANer
         except ImportError:
             return {"error": "realesrgan not installed. Install it from Cookbook → Dependencies (search 'realesrgan')."}
@@ -1494,6 +1518,7 @@ def setup_gallery_routes() -> APIRouter:
         img_bytes = base64.b64decode(image_b64)
         src = Image.open(io.BytesIO(img_bytes)).convert("RGB")
         try:
+            patch_realesrgan_torchvision_compat()
             from basicsr.archs.rrdbnet_arch import RRDBNet
             from realesrgan import RealESRGANer
         except ImportError:

routes/mcp_routes.py

@@ -108,6 +108,12 @@ def _load_disabled_map():
         db.close()
 
 
+def _mcp_oauth_redirect_uri() -> str:
+    """Shared callback URL for legacy Google and generic MCP OAuth flows."""
+    from src.mcp_oauth import REDIRECT_URI
+    return REDIRECT_URI
+
+
 def setup_mcp_routes(mcp_manager: McpManager):
     """Setup MCP routes with the provided manager."""
 
@@ -445,9 +451,9 @@ def setup_mcp_routes(mcp_manager: McpManager):
             client_id = keys["client_id"]
             scopes = oauth_cfg.get("scopes", [])
 
-            # For Desktop App creds, redirect to localhost — the user will
+            # For Desktop App creds, default to localhost — the user will
             # paste the resulting URL back if they're on a different device.
-            redirect_uri = "http://localhost:7000/api/mcp/oauth/callback"
+            redirect_uri = _mcp_oauth_redirect_uri()
 
             params = {
                 "client_id": client_id,
@@ -469,7 +475,7 @@ def setup_mcp_routes(mcp_manager: McpManager):
                 return RedirectResponse(auth_url)
             else:
                 # Remote device — show paste-back page
-                return HTMLResponse(_oauth_authorize_page(auth_url, server_id, host))
+                return HTMLResponse(_oauth_authorize_page(auth_url, server_id, host, redirect_uri))
         finally:
             db.close()
 
@@ -536,7 +542,7 @@ def setup_mcp_routes(mcp_manager: McpManager):
             client_id = keys["client_id"]
             client_secret = keys["client_secret"]
 
-            redirect_uri = "http://localhost:7000/api/mcp/oauth/callback"
+            redirect_uri = _mcp_oauth_redirect_uri()
 
             async with httpx.AsyncClient() as client:
                 resp = await client.post(
@@ -603,13 +609,19 @@ def setup_mcp_routes(mcp_manager: McpManager):
     return router
 
 
-def _oauth_authorize_page(auth_url: str, server_id: str, host: str) -> str:
+def _oauth_authorize_page(
+    auth_url: str,
+    server_id: str,
+    host: str,
+    redirect_uri: str = "http://localhost:7000/api/mcp/oauth/callback",
+) -> str:
     """Page with Google sign-in link and URL paste-back form for remote access."""
     # Escape values interpolated into the page: `host` comes from the request
     # Host header and `server_id` from the OAuth state — neither is trusted.
     auth_url = html.escape(auth_url, quote=True)
     server_id = html.escape(server_id, quote=True)
     host = html.escape(host, quote=True)
+    redirect_uri = html.escape(redirect_uri, quote=True)
     return f"""<!DOCTYPE html>
 <html><head>
 <meta charset="UTF-8"><title>Authorize — Odysseus</title>
@@ -654,7 +666,7 @@ def _oauth_authorize_page(auth_url: str, server_id: str, host: str) -> str:
   <div class="divider"></div>
   <form method="POST" action="http://{host}/api/mcp/oauth/exchange/{server_id}">
     <p>Paste the URL from your browser after signing in:</p>
-    <input type="text" name="callback_url" placeholder="http://localhost:7000/api/mcp/oauth/callback?code=..." required>
+    <input type="text" name="callback_url" placeholder="{redirect_uri}?code=..." required>
     <br><button type="submit">Connect</button>
   </form>
 </div></body></html>"""

routes/memory_routes.py

@@ -29,6 +29,7 @@ from src.llm_core import llm_call_async
 from services.memory.memory_extractor import audit_memories
 from src.auth_helpers import get_current_user, require_user
 from src.endpoint_resolver import resolve_endpoint
+from src.task_endpoint import resolve_task_endpoint
 from src.upload_limits import read_upload_limited, MEMORY_IMPORT_MAX_BYTES
 
 logger = logging.getLogger(__name__)
@@ -240,14 +241,18 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
         }
         messages = [system_msg] + sess.get_context_messages()
 
+        t_url, t_model, t_headers = resolve_task_endpoint(
+            sess.endpoint_url, sess.model, sess.headers, owner=_owner(request)
+        )
+
         try:
             suggestion_text = await llm_call_async(
-                sess.endpoint_url,
-                sess.model,
+                t_url,
+                t_model,
                 messages,
                 temperature=0.2,
                 max_tokens=500,
-                headers=sess.headers,
+                headers=t_headers,
             )
             try:
                 suggestions = json.loads(suggestion_text)
@@ -278,42 +283,50 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
         endpoint_url = model = None
         headers = {}
 
-        # Try default model from settings first
-        settings = _load_settings()
-        ep_id = settings.get("default_endpoint_id", "")
-        default_model = settings.get("default_model", "")
-        if ep_id:
-            db = SessionLocal()
-            try:
-                ep = db.query(ModelEndpoint).filter(
-                    ModelEndpoint.id == ep_id, ModelEndpoint.is_enabled == True
-                ).first()
-                if ep:
-                    base = _normalize_base(ep.base_url)
-                    endpoint_url = build_chat_url(base)
-                    model = default_model
-                    if not model and ep.models:
-                        try:
-                            models = _json.loads(ep.models) if isinstance(ep.models, str) else ep.models
-                            if models:
-                                model = models[0]
-                        except Exception:
-                            pass
-                    if ep.api_key:
-                        headers = {"Authorization": f"Bearer {ep.api_key}"}
-            finally:
-                db.close()
+        # Try utility model from settings first — memory audit is a background
+        # task and should prefer the lighter utility model over the main chat model.
+        from src.task_endpoint import resolve_task_endpoint
+        user = _owner(request)
+        t_url, t_model, t_headers = resolve_task_endpoint(owner=user)
+        if t_url and t_model:
+            endpoint_url, model, headers = t_url, t_model, t_headers
+        else:
+            # Fall back to default model if no task/utility model configured
+            settings = _load_settings()
+            ep_id = settings.get("default_endpoint_id", "")
+            default_model = settings.get("default_model", "")
+            if ep_id:
+                db = SessionLocal()
+                try:
+                    ep = db.query(ModelEndpoint).filter(
+                        ModelEndpoint.id == ep_id, ModelEndpoint.is_enabled == True
+                    ).first()
+                    if ep:
+                        base = _normalize_base(ep.base_url)
+                        endpoint_url = build_chat_url(base)
+                        model = default_model
+                        if not model and ep.models:
+                            try:
+                                models = _json.loads(ep.models) if isinstance(ep.models, str) else ep.models
+                                if models:
+                                    model = models[0]
+                            except Exception:
+                                pass
+                        if ep.api_key:
+                            headers = {"Authorization": f"Bearer {ep.api_key}"}
+                finally:
+                    db.close()
 
-        # Fall back to session model if no default configured
-        if not endpoint_url and session:
-            try:
-                sess = session_manager.get_session(session)
-                _assert_session_owner(sess, _owner(request))
-                endpoint_url = sess.endpoint_url
-                model = sess.model
-                headers = sess.headers
-            except KeyError:
-                pass
+            # Fall back to session model if no default configured
+            if not endpoint_url and session:
+                try:
+                    sess = session_manager.get_session(session)
+                    _assert_session_owner(sess, _owner(request))
+                    endpoint_url = sess.endpoint_url
+                    model = sess.model
+                    headers = sess.headers
+                except KeyError:
+                    pass
 
         if not endpoint_url or not model:
             raise HTTPException(400, "No default model configured — set one in Settings")
@@ -360,13 +373,13 @@ def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionM
             try:
                 sess = session_manager.get_session(session)
                 _assert_session_owner(sess, _owner(request))
-                endpoint_url = sess.endpoint_url
-                model = sess.model
-                headers = sess.headers
+                endpoint_url, model, headers = resolve_task_endpoint(
+                    sess.endpoint_url, sess.model, sess.headers, owner=_owner(request)
+                )
             except KeyError:
                  raise HTTPException(404, "Session not found — needed for LLM config")
         else:
-            endpoint_url, model, headers = resolve_endpoint("utility", owner=_owner(request))
+            endpoint_url, model, headers = resolve_task_endpoint(owner=_owner(request))
     
         if not endpoint_url or not model:
             raise HTTPException(400, "No LLM model configured. Set a default model in Settings.")

routes/model_routes.py

@@ -870,15 +870,52 @@ def _ping_endpoint(base_url: str, api_key: str = None, timeout: float = 1.5) ->
 
 
 def _model_endpoint_error_message(base_url: str, ping: Dict[str, Any] = None) -> str:
-    """Return a provider-aware error message for failed endpoint probes."""
+    """Return a provider-aware error message for failed endpoint probes.
+
+    Surfaces the URL we actually probed and, when the endpoint looks like
+    LM Studio (port 1234 or hostname match), adds a hint about loading a
+    model and confirming the Developer Server is running. The user previously
+    saw a generic "No models found for that provider/key" with no way to
+    tell whether the URL was wrong, the server was down, or the server was
+    reachable but had no model loaded (issue #25).
+    """
     ping = ping or {}
     error = ping.get("error")
+    from src.endpoint_resolver import build_models_url
+    try:
+        probed = build_models_url(base_url) or base_url
+    except Exception:
+        probed = base_url
     parsed = urlparse(base_url)
     host = (parsed.hostname or "").lower()
     is_ollama = parsed.port == 11434 or "ollama" in host or "ollama" in base_url.lower()
+    is_lmstudio = (
+        parsed.port == 1234
+        or "lmstudio" in host
+        or "lm-studio" in host
+        or "lm_studio" in host
+    )
+
+    if is_lmstudio:
+        parts = [
+            "LM Studio is reachable, but no models were reported.",
+            f"Probed {probed}.",
+        ]
+        if error:
+            parts.append(f"Last probe error: {error}.")
+        parts.append(
+            "Open LM Studio, load at least one model, and confirm the "
+            "Developer Server is running on port 1234."
+        )
+        parts.append(
+            "Base URL should be http://localhost:1234/v1 (native) or "
+            "http://host.docker.internal:1234/v1 (Docker)."
+        )
+        return " ".join(parts)
 
     if is_ollama:
         parts = ["No Ollama models found for that endpoint."]
+        parts.append(f"Probed {probed}.")
         if error:
             parts.append(f"Last probe error: {error}.")
         parts.append("Check that Ollama is running and that the base URL is correct.")
@@ -888,9 +925,9 @@ def _model_endpoint_error_message(base_url: str, ping: Dict[str, Any] = None) ->
         return " ".join(parts)
 
     if error:
-        return f"No models found for that provider/key. Last probe error: {error}."
+        return f"No models found for that provider/key. Probed {probed}. Last probe error: {error}."
 
-    return "No models found for that provider/key."
+    return f"No models found for that provider/key. Probed {probed}."
 
 
 def _normalize_model_ids(value):

routes/personal_routes.py

@@ -160,8 +160,11 @@ def setup_personal_routes(personal_docs_manager, rag_manager, rag_available):
             JSON response confirming removal
         """
         try:
-            if not directory:
-                raise HTTPException(400, "Directory path is required")
+            # Confine to PERSONAL_DIR — parity with add_directory_to_rag (which
+            # resolves the path the same way). Without this, an arbitrary or
+            # `..`-escaping path is passed straight to
+            # personal_docs_manager.remove_directory / rag.remove_directory.
+            directory = _resolve_allowed_personal_dir(directory)
 
             logger.info(f"Removing directory from RAG: {directory}")
 

routes/shell_routes.py

@@ -1,6 +1,7 @@
 """Shell routes — user-facing command execution endpoint."""
 
 import asyncio
+import importlib
 import json
 import logging
 import os
@@ -14,6 +15,7 @@ from collections import namedtuple
 from pathlib import Path
 from typing import Dict, Any
 from core.platform_compat import IS_APPLE_SILICON, which_tool
+from src.optional_deps import prepare_optional_dependency_import
 
 # POSIX-only: `pty`/`fcntl` transitively import `termios`, which does NOT exist
 # on Windows, so importing them unconditionally crashed app startup there
@@ -149,6 +151,11 @@ def _pip_dist_name(pkg: dict) -> str:
     return (pkg.get("name") or "").replace("_", "-")
 
 
+def _import_optional_dependency_for_status(name: str):
+    prepare_optional_dependency_import(name)
+    return importlib.import_module(name)
+
+
 def _package_installed_from_probe(name: str, probe: dict) -> bool:
     """Return whether an optional dependency is usable by Cookbook.
 
@@ -970,7 +977,6 @@ def setup_shell_routes() -> APIRouter:
         """
         _require_admin(request)
         _reject_cross_site(request)
-        import importlib
         import importlib.metadata as importlib_metadata
         import shlex
         import json as _json
@@ -1057,6 +1063,13 @@ def setup_shell_routes() -> APIRouter:
                 "category": "Image",
                 "target": "remote",
             },
+            {
+                "name": "transformers",
+                "pip": "transformers",
+                "desc": "Hugging Face model components used by SD/Flux pipelines and image tools",
+                "category": "Image",
+                "target": "remote",
+            },
             {
                 "name": "rembg",
                 "pip": "rembg[gpu]",
@@ -1202,7 +1215,7 @@ def setup_shell_routes() -> APIRouter:
                     pkg["status_note"] = _package_status_note("vllm", probe)
             else:
                 try:
-                    importlib.import_module(pkg["name"])
+                    _import_optional_dependency_for_status(pkg["name"])
                     importlib_metadata.version(_pip_dist_name(pkg))
                     pkg["installed"] = True
                 except ImportError:
@@ -1251,6 +1264,7 @@ def setup_shell_routes() -> APIRouter:
             "sglang[all]",
             "diffusers",
             "diffusers[torch]",
+            "transformers",
             "TTS",
             "bark",
             "faster-whisper",

services/hwfit/profiles.py

@@ -188,12 +188,18 @@ def compute_serve_profiles(system, model, serve_weights_gb=None, serve_quant=Non
         # Shrink context if even the chosen KV won't fit alongside weights.
         # Start from the smaller of the profile's target and the model's limit.
         cur_ctx = min(ctx, model_ctx_max)
-        while cur_ctx >= 8192:
+        # Floor the context-shrink loop at 8192, but never above the model's own
+        # trained limit. A model with a sub-8192 context (e.g. a 2048-token
+        # SmolLM) starts below 8192, so a hard-coded 8192 guard skipped the loop
+        # entirely and produced NO profile — the serve UI then fell back to
+        # manual flags even though the model fits the GPU trivially.
+        ctx_floor = min(8192, model_ctx_max)
+        while cur_ctx >= ctx_floor:
             kv = _kv_gb(model, cur_ctx, kv_type)
             n_cpu_moe, fits = _cpu_moe_for_budget(model, quant, kv, budget, fixed_gb=serve_weights_gb)
             est = _weights_gb(model, quant, serve_weights_gb) + kv + 0.6
             # If a non-MoE model can't fit even fully offloaded, try less context.
-            if model.get("is_moe") or fits or cur_ctx <= 8192:
+            if model.get("is_moe") or fits or cur_ctx <= ctx_floor:
                 profiles.append({
                     "key": key,
                     "label": label,

services/youtube/youtube_handler.py

@@ -64,20 +64,40 @@ def is_youtube_url(url: str) -> bool:
     return "youtube.com" in url or "youtu.be" in url
 
 
+# youtube.com-shaped hosts. music.youtube.com serves the same /watch and
+# /shorts paths, so links shared from YouTube Music must resolve too.
+_YT_HOSTS = ("www.youtube.com", "youtube.com", "m.youtube.com", "music.youtube.com")
+# Path prefixes whose first following segment is the video id. Covers the
+# /embed/ player, Shorts (/shorts/), live streams (/live/), and the legacy
+# /v/ embed — all of which `is_youtube_url` already treats as YouTube, so
+# they must be extractable or the link is silently dropped (neither web-fetched
+# nor transcript-fetched) by the chat pipeline.
+_YT_PATH_PREFIXES = ("/embed/", "/shorts/", "/live/", "/v/")
+
+
 def extract_youtube_id(url: str) -> Optional[str]:
-    """Extract YouTube video ID from various URL formats."""
+    """Extract a YouTube video ID from the common URL shapes:
+    watch?v=, youtu.be/<id>, /embed/<id>, /shorts/<id>, /live/<id>, /v/<id>,
+    across youtube.com / m.youtube.com / music.youtube.com / youtu.be."""
     if not isinstance(url, str):
         return None
     parsed = urllib.parse.urlparse(url)
-    if parsed.hostname in ("www.youtube.com", "youtube.com", "m.youtube.com"):
+    host = (parsed.hostname or "").lower()
+    if host in _YT_HOSTS:
         if parsed.path == "/watch":
             params = urllib.parse.parse_qs(parsed.query)
-            if "v" in params:
+            if params.get("v"):
                 return params["v"][0]
-        elif parsed.path.startswith("/embed/"):
-            return parsed.path.split("/")[-1]
-    elif parsed.hostname == "youtu.be":
-        return parsed.path[1:]
+        else:
+            for prefix in _YT_PATH_PREFIXES:
+                if parsed.path.startswith(prefix):
+                    vid = parsed.path[len(prefix):].split("/")[0]
+                    if vid:
+                        return vid
+    elif host == "youtu.be":
+        vid = parsed.path.lstrip("/").split("/")[0]
+        if vid:
+            return vid
     return None
 
 
@@ -170,6 +190,8 @@ def format_transcript_for_context(
     if segments:
         ctx += "Timestamped Transcript:\n"
         for seg in segments:
+            if not isinstance(seg, dict):
+                continue
             ctx += f"[{seg['timestamp']}] {seg['text']}\n"
         # Check length — fall back to plain text if too long
         if len(ctx) > 12000:
@@ -202,15 +224,24 @@ async def fetch_youtube_comments(
             f"https://www.youtube.com/watch?v={video_id}",
         ]
 
-        proc = await asyncio.wait_for(
-            asyncio.create_subprocess_exec(
-                *cmd,
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            ),
-            timeout=timeout,
+        proc = await asyncio.create_subprocess_exec(
+            *cmd,
+            stdout=asyncio.subprocess.PIPE,
+            stderr=asyncio.subprocess.PIPE,
         )
-        stdout, stderr = await proc.communicate()
+        # Bound the wait on the process actually finishing, not on spawning it.
+        # create_subprocess_exec returns as soon as the child starts, so wrapping
+        # it in wait_for never enforces the timeout — proc.communicate() is the
+        # blocking step. Kill and reap the child if it overruns so it does not
+        # linger after we return.
+        try:
+            stdout, stderr = await asyncio.wait_for(
+                proc.communicate(), timeout=timeout
+            )
+        except asyncio.TimeoutError:
+            proc.kill()
+            await proc.wait()
+            raise
 
         if proc.returncode != 0:
             return {"success": False, "error": f"yt-dlp failed: {stderr.decode()[:200]}", "comments": []}

src/action_intents.py

@@ -91,6 +91,9 @@ _ROUTING_PATTERNS: tuple[tuple[str, str, Pattern[str]], ...] = tuple(
         ("ui", "tool or feature toggle request", r"\b(?:disable|enable|turn\s+(?:on|off))\s+(?:the\s+)?(?:shell|search|web|browser|documents?|memory|skills|images?|calendar|email|mail|research|incognito)\b"),
 
         # Deep research jobs, not quick conceptual mentions of research.
+        ("web", "explicit web search request", rf"{_PLEASE}(?:do|run|use|perform|make)\s+(?:a\s+)?(?:web\s+search|search\s+the\s+web)\b.+"),
+        ("web", "web lookup imperative request", rf"{_PLEASE}(?:web\s+search|search\s+the\s+web|search\s+online|look\s+up|google)\b.+"),
+        ("web", "assistant web lookup request", rf"{_ACTION_QUESTION}(?:web\s+search|search\s+the\s+web|search\s+online|look\s+up|google)\b.+"),
         ("research", "deep research imperative request", rf"{_PLEASE}(?:research|deep\s+dive|look\s+into|investigate)\s+.+"),
         ("research", "assistant deep research request", rf"{_ACTION_QUESTION}(?:research|do\s+research|deep\s+dive|look\s+into|investigate)\s+.+"),
 

src/agent_loop.py

@@ -262,6 +262,11 @@ _DOMAIN_RULES = {
 - Use `manage_settings` for preferences and tool enable/disable.
 - Use named tools over `app_api` when a named wrapper exists.
 - `app_api` is only for safe UI/API actions without a named tool; do not use it for shell, package installs, engine rebuilds, or sensitive auth/admin paths.""",
+    "contacts": """\
+## Contacts rules
+- Use `resolve_contact` to look up a contact's email or phone number by name. Searches the CardDAV address book and sent email history.
+- Use `manage_contact` to list, add, update, or delete contacts in the address book.
+- Do NOT use `manage_memory` for contact lookups — contact details live in the address book, not memory.""",
 }
 
 _DOMAIN_TOOL_MAP = {
@@ -274,6 +279,7 @@ _DOMAIN_TOOL_MAP = {
     "sessions": {"create_session", "list_sessions", "manage_session", "send_to_session", "search_chats"},
     "files": {"bash", "python", "read_file", "write_file", "edit_file", "grep", "glob", "ls", "get_workspace"},
     "settings": {"manage_settings", "manage_endpoints", "manage_mcp", "manage_webhooks", "manage_tokens", "app_api"},
+    "contacts": {"resolve_contact", "manage_contact"},
 }
 
 def _domain_rules_for_tools(tool_names: set) -> list[str]:
@@ -787,6 +793,12 @@ def _classify_agent_request(messages: List[Dict], last_user: str) -> Dict[str, o
         domains.add("documents")
     if has(r"\b(search|web|google|look up|latest|news|current|weather|forecast|stock price|price of|website|url|https?://|www\.)\b"):
         domains.add("web")
+    if has(
+        r"\b(wyszukaj|wyszukać|wyszukac)\b.*\b(internet|internecie|online|web)\b",
+        r"\b(sprawd[zź]|znajd[zź])\b.*\b(internet|internecie|online|web)\b",
+        r"\b(aktualn\w*|bieżąc\w*|biezac\w*|dzisiaj|teraz)\b.*\b(pogod\w*|temperatur\w*)\b",
+    ):
+        domains.add("web")
     if has(r"\b(research|deep dive|investigate|look into)\b"):
         domains.add("web")
     if has(r"\b(open|show|toggle|turn on|turn off|disable|enable|switch model|change model|settings|theme|panel)\b"):
@@ -797,6 +809,8 @@ def _classify_agent_request(messages: List[Dict], last_user: str) -> Dict[str, o
         domains.add("files")
     if has(r"\b(endpoint|api token|mcp|webhook|preference|configure|config|setting)\b"):
         domains.add("settings")
+    if has(r"\b(contact|contacts|phone|phone number|address book|vcard)\b"):
+        domains.add("contacts")
 
     low_signal = not continuation and not domains
     return {
@@ -1801,18 +1815,21 @@ async def stream_agent_loop(
         logger.info(f"[tool-rag] Using caller-provided relevant_tools ({len(_relevant_tools)} tools)")
     if not guide_only and not _relevant_tools and bool(_intent.get("low_signal")):
         from src.tool_index import ALWAYS_AVAILABLE
-        _relevant_tools = set(ALWAYS_AVAILABLE)
         if workspace:
             # An active workspace IS the file-work signal: a vague "look at the
             # project" means explore this folder. Surface only the READ-ONLY file
             # tools (intersection with the plan-mode read-only allowlist) so the
             # agent can investigate; write/shell tools stay out until the request
             # actually calls for them (RAG retrieval adds those on a real ask).
+            _relevant_tools = set(ALWAYS_AVAILABLE)
             from src.tool_security import PLAN_MODE_READONLY_TOOLS
             _relevant_tools |= (_DOMAIN_TOOL_MAP["files"] & PLAN_MODE_READONLY_TOOLS)
             logger.info("[tool-rag] Low-signal but workspace active; including read-only file tools")
         else:
-            logger.info("[tool-rag] Low-signal agent message; skipping retrieval and using always-available tools only")
+            # Don't short-circuit: fall through to RAG retrieval below.
+            # Non-English queries are flagged low_signal by the English-only
+            # intent classifier, but fastembed retrieval works across languages.
+            logger.info("[tool-rag] Low-signal query; will run RAG retrieval")
     if not guide_only and not _relevant_tools:
         try:
             from src.tool_index import get_tool_index, ALWAYS_AVAILABLE
@@ -1937,6 +1954,10 @@ async def stream_agent_loop(
     # and can override this list for users who know their setup.
     _model_no_tools = any(kw in _model_lc for kw in (
         "deepseek-r1",
+        # Open-weight GPT-OSS models are commonly served through llama.cpp /
+        # llama-cpp-python. Their names contain "gpt-o", but they do not use
+        # OpenAI's native tool-call channel unless the endpoint opts in.
+        "gpt-oss",
     ))
     # Native Ollama endpoints (/api/chat) handle tool schemas differently from
     # the OpenAI-compat path. Models like gemma4, qwen3.5, ministral respond to
@@ -1998,30 +2019,34 @@ async def stream_agent_loop(
     _t3 = time.time()
     try:
         from src.context_compactor import trim_for_context
-        from src.context_budget import compute_input_token_budget, DEFAULT_HARD_MAX
-        from src.settings import is_setting_overridden
+        from src.context_budget import compute_input_token_budget, DEFAULT_HARD_MAX, DEFAULT_BUDGET, budget_is_explicit as _budget_is_explicit
+        from src.model_context import budget_context_for_model
 
-        soft_budget = int(get_setting("agent_input_token_budget", 6000) or 0)
+        soft_budget = int(get_setting("agent_input_token_budget", DEFAULT_BUDGET) or 0)
         if soft_budget > 0:
             before_trim_tokens = estimate_tokens(messages)
             reserve_tokens = min(max(max_tokens or 1024, 512), 2048)
-            # Honour the configurable ceiling for the auto-derived budget path.
-            # No-op when the user has an explicit `agent_input_token_budget`
-            # (that branch ignores hard_max). Falls back to DEFAULT_HARD_MAX
-            # on missing/malformed values so misconfig can't zero the budget.
+            # Ceiling for the auto-derived budget (no effect on an explicit budget;
+            # see #1230). Falls back to DEFAULT_HARD_MAX on missing/malformed values
+            # so misconfig can't zero the budget.
             try:
                 hard_max = int(get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX) or DEFAULT_HARD_MAX)
             except (TypeError, ValueError):
                 hard_max = DEFAULT_HARD_MAX
             if hard_max <= 0:
                 hard_max = DEFAULT_HARD_MAX
-            # Scale the default budget to the model's context window so long-context
-            # models aren't silently capped at 6000; an explicit user setting is
-            # still honoured (clamped to the window). (#1170)
+            # Default value = auto sentinel (scale to the window); any other value =
+            # explicit cap. Value-based, not presence-based, because the save path
+            # materializes defaults so a persisted default must still read as auto (#4121).
+            budget_is_explicit = _budget_is_explicit(soft_budget)
+            # Scale only off a window we actually discovered, bound to the value it
+            # proves (else 0) — not the passed-in context_length, which can be stale
+            # or unset for some callers (#4122 review).
+            ctx_for_budget = budget_context_for_model(endpoint_url, model, fallback=context_length)
             effective_budget = compute_input_token_budget(
                 soft_budget,
-                context_length,
-                is_setting_overridden("agent_input_token_budget"),
+                ctx_for_budget,
+                budget_is_explicit,
                 hard_max=hard_max,
             )
             trimmed_messages = trim_for_context(
@@ -2096,11 +2121,12 @@ async def stream_agent_loop(
     # tool, so we don't nudge on harmless transitional text like "let me
     # know what you think".
     _INTENT_RE = re.compile(
-        r"(?:^|\n)\s*(?:let me|i'?ll|i will|going to|let's)\s+"
+        r"(?:^|\n)\s*(?:let me|i'?ll|i will|i need to|we need to|need to|"
+        r"i should|we should|i must|we must|going to|let's)\s+"
         r"(?:tail|check|investigate|look at|see|tail|read|fetch|inspect|"
         r"verify|diagnose|examine|debug|capture|grab|pull|view|run|call|"
         r"trigger|launch|start|kick off|stop|kill|restart|adopt|serve|"
-        r"register|adopt|list|search|find|query|hit|ping|test)"
+        r"register|adopt|list|search|find|query|hit|ping|test|use|perform|do)"
         r"\b[^.\n]{0,140}",
         re.IGNORECASE,
     )

src/api_key_manager.py

@@ -4,6 +4,8 @@ import logging
 from typing import Dict
 from cryptography.fernet import Fernet, InvalidToken
 
+from core.platform_compat import safe_chmod
+
 logger = logging.getLogger(__name__)
 
 class APIKeyManager:
@@ -15,12 +17,20 @@ class APIKeyManager:
     def get_or_create_key(self) -> bytes:
         """Get or create encryption key for API keys"""
         if os.path.exists(self.key_file):
+            # Older versions wrote .key with the process umask (often 0o644,
+            # i.e. group/world-readable). Re-restrict on read so existing
+            # installs heal without needing the key to be regenerated.
+            safe_chmod(self.key_file, 0o600)
             with open(self.key_file, 'rb') as f:
                 return f.read()
         else:
             key = Fernet.generate_key()
             with open(self.key_file, 'wb') as f:
                 f.write(key)
+            # This key decrypts every stored provider credential, so restrict it
+            # to the owner (0o600) — it must not be group/world-readable. No-op
+            # on Windows (files there are ACL-restricted to the user already).
+            safe_chmod(self.key_file, 0o600)
             return key
     
     def encrypt_api_key(self, api_key: str) -> str:

src/bg_monitor.py

@@ -55,6 +55,8 @@ async def _drain_agent(sess, messages):
         if "delta" in d:
             delta = d.get("delta")
             if isinstance(delta, str):
+                if d.get("thinking"):
+                    continue
                 full += delta
         elif d.get("type") == "agent_step":
             round_num = d.get("round", round_num)

src/builtin_mcp.py

@@ -5,12 +5,13 @@ Auto-registration of built-in MCP servers on startup.
 Each server runs as a stdio subprocess managed by McpManager.
 """
 
+import asyncio
+import json
 import logging
 import os
 import shutil
 import subprocess
 import sys
-import asyncio
 
 from core.platform_compat import IS_WINDOWS, which_tool
 
@@ -197,12 +198,13 @@ def _npx_package_from_args(args):
 async def _is_npx_package_cached(npx_path, package_spec, timeout_s=5):
     """Probe whether an npx package is already in the local cache.
 
-    Runs `npx --no-install <pkg> --version`. --no-install tells npx to
-    fail instead of downloading, so a cache miss returns fast. We treat
-    "exited 0 with non-empty stdout" as proof of a working cached copy.
-    Anything else (non-zero exit, empty stdout, timeout, missing npx,
-    network error) means we should skip the server.
+    First checks the local `_npx` cache for an installed package. If the
+    package is not found there, falls back to `npx --no-install <pkg>
+    --version` so older npm layouts still work without downloading.
     """
+    if _is_package_in_npx_cache(package_spec):
+        return True
+
     try:
         proc = await asyncio.create_subprocess_exec(
             npx_path, "--no-install", package_spec, "--version",
@@ -231,3 +233,68 @@ async def _is_npx_package_cached(npx_path, package_spec, timeout_s=5):
             pass
         return False
     return proc.returncode == 0 and bool(stdout.strip())
+
+
+def _is_package_in_npx_cache(package_spec):
+    """Return True when npm's `_npx` cache already contains package_spec."""
+    package_name = _npx_package_name(package_spec)
+    if not package_name:
+        return False
+
+    for cache_root in _npm_cache_roots():
+        npx_root = os.path.join(cache_root, "_npx")
+        if _npx_cache_contains_package(npx_root, package_name):
+            return True
+    return False
+
+
+def _npx_package_name(package_spec):
+    """Strip a version/range suffix from an npm package spec."""
+    if not package_spec:
+        return ""
+    if package_spec.startswith("@"):
+        parts = package_spec.split("@", 2)
+        if len(parts) >= 3:
+            return f"@{parts[1]}"
+        return package_spec
+    return package_spec.split("@", 1)[0]
+
+
+def _npm_cache_roots():
+    roots = []
+    configured = os.environ.get("npm_config_cache")
+    if configured:
+        roots.append(os.path.expanduser(configured))
+    roots.append(os.path.join(os.path.expanduser("~"), ".npm"))
+    local_app_data = os.environ.get("LOCALAPPDATA")
+    if local_app_data:
+        roots.append(os.path.join(local_app_data, "npm-cache"))
+    return list(dict.fromkeys(roots))
+
+
+def _npx_cache_contains_package(npx_root, package_name):
+    if not os.path.isdir(npx_root):
+        return False
+    package_path = os.path.join("node_modules", *package_name.split("/"), "package.json")
+    try:
+        entries = list(os.scandir(npx_root))
+    except OSError:
+        return False
+    for entry in entries:
+        try:
+            is_dir = entry.is_dir()
+        except OSError:
+            continue
+        cached_name = _cached_package_name(os.path.join(entry.path, package_path))
+        if is_dir and cached_name == package_name:
+            return True
+    return False
+
+
+def _cached_package_name(package_json_path):
+    try:
+        with open(package_json_path, encoding="utf-8") as fh:
+            data = json.load(fh)
+    except (OSError, ValueError):
+        return ""
+    return str(data.get("name", "")).strip()

src/context_budget.py

@@ -31,16 +31,22 @@ def compute_input_token_budget(
 
     Args:
         configured: the value read from settings (may be the default).
-        context_length: the model's discovered context window (0/unknown if none).
-        explicit: True if the user explicitly set ``agent_input_token_budget``.
+        context_length: the model's discovered context window. Pass 0 when the
+            window is unknown / only a bare fallback — auto-scaling then stays
+            conservative instead of trusting an unproven window (review on #4122).
+        explicit: True if the user set a NON-default budget. The default value is
+            the "auto" sentinel (scale to the window); any other value is an
+            explicit cap. (A deliberately-chosen default can't be distinguished
+            from a materialized default by value, so the default reads as auto.)
 
     Rules:
         - Explicit user budget is honoured exactly, only clamped to the model's
-          window when that window is known (never send more than the model holds).
-        - Otherwise (default), scale to ``headroom`` of the context window, capped
-          at ``hard_max`` — so long-context models use their capacity.
-        - When the window is unknown, fall back to the configured/default value
-          (preserving the previous behaviour).
+          window when that window is known (the user's deliberate choice wins;
+          ``hard_max`` is an auto-budget ceiling only — see #1230).
+        - Otherwise (auto), scale to ``headroom`` of the context window, capped at
+          ``hard_max`` — so long-context models use their capacity.
+        - When the window is unknown (context_length <= 0), use the conservative
+          ``default`` budget and do NOT scale off the fallback.
     """
     configured = int(configured or 0)
     context_length = int(context_length or 0)
@@ -53,3 +59,17 @@ def compute_input_token_budget(
         return max(1, min(scaled, hard_max))
 
     return configured if configured > 0 else default
+
+
+def budget_is_explicit(configured: int, *, default: int = DEFAULT_BUDGET) -> bool:
+    """Whether a configured agent_input_token_budget is a deliberate explicit cap.
+
+    The default value is the "auto" sentinel (scale to the model's window), so only
+    a NON-default positive value counts as explicit. This keys off the VALUE, not
+    settings *presence* — the settings-save path materializes every default into
+    settings.json, so a persisted default must still read as auto (the regression
+    #4121 / #1230 are about). Centralised here so the materialized-default contract
+    is unit-testable and can't silently regress to a presence check.
+    """
+    configured = int(configured or 0)
+    return configured > 0 and configured != default

src/cookbook_serve_lifecycle.py

@@ -136,7 +136,8 @@ async def _tick() -> None:
         return
     try:
         state = json.loads(state_path.read_text(encoding="utf-8"))
-    except Exception:
+    except Exception as e:
+        logger.warning("cookbook_serve_lifecycle: state file unreadable (%s), skipping tick", e)
         return
     tasks = state.get("tasks") or []
     now_ms = int(time.time() * 1000)
@@ -178,8 +179,26 @@ async def _tick() -> None:
     if stopped_any:
         try:
             from core.atomic_io import atomic_write_json
-            state["tasks"] = tasks
-            atomic_write_json(state_path, state)
+            # Re-read the state file so concurrent UI writes (task adds,
+            # status flips, config edits) are not silently overwritten.
+            # Apply only our stop mutations to the fresh snapshot.
+            try:
+                fresh = json.loads(state_path.read_text(encoding="utf-8"))
+                fresh_tasks = fresh.get("tasks") or []
+            except Exception:
+                fresh = state
+                fresh_tasks = tasks
+            stopped_sids = {sid for sid, _, _ in to_stop}
+            for ft in fresh_tasks:
+                if not isinstance(ft, dict):
+                    continue
+                ft_sid = ft.get("sessionId") or ft.get("id")
+                if ft_sid in stopped_sids:
+                    ft["status"] = "stopped"
+                    ft["_scheduledStopAtMs"] = None
+                    ft["_lastStatusFlipAt"] = now_ms
+            fresh["tasks"] = fresh_tasks
+            atomic_write_json(state_path, fresh)
         except Exception as e:
             logger.warning(f"cookbook_serve_lifecycle: state write failed: {e}")
 

src/endpoint_resolver.py

@@ -183,7 +183,16 @@ def build_chat_url(base: str) -> str:
 
 
 def build_models_url(base: str) -> Optional[str]:
-    """Return the provider-specific model-list endpoint URL for a base."""
+    """Return the provider-specific model-list endpoint URL for a base.
+
+    For OpenAI-compatible servers (LM Studio, llama.cpp, vLLM,
+    text-generation-webui, etc.) the model list is exposed at ``/v1/models``.
+    When the user-supplied base has no path — e.g. ``http://localhost:1234`` —
+    we still need to land on ``/v1/models`` (issue #25); insert the ``/v1``
+    segment only when the path is empty, leaving any explicit non-empty path
+    untouched (so custom prefixes like ``/openai`` or ``/api/openai/v1`` keep
+    their semantics).
+    """
     base = normalize_base(resolve_url(base))
     provider = _detect_provider(base)
     if provider == "anthropic":
@@ -192,6 +201,12 @@ def build_models_url(base: str) -> Optional[str]:
         return _ollama_api_root(base) + "/tags"
     if provider == "chatgpt-subscription":
         return None
+    # Generic OpenAI-compatible fallback: ensure the path lands on /v1/models
+    # when the user omitted a path entirely. If a non-empty path is already
+    # present (e.g. /openai, /api/openai/v1, /v1), trust the caller — the
+    # /models suffix is appended as-is and the caller's prefix is preserved.
+    if not urlparse(base).path:
+        base = base + "/v1"
     return base + "/models"
 
 
@@ -250,27 +265,23 @@ def resolve_endpoint(
     ep_id = _stg(f"{setting_prefix}_endpoint_id")
     model = _stg(f"{setting_prefix}_model")
 
-    # If the specific endpoint is not configured, but the caller provided a
+    # Fall back to utility model for task/research/auto-naming if not specifically configured.
+    if not ep_id and setting_prefix not in ("utility", "default"):
+        ep_id = _stg("utility_endpoint_id")
+        model = _stg("utility_model")
+
+    # If the endpoint is STILL not configured, but the caller provided a
     # valid fallback (e.g. the active session model), use that immediately.
     # This prevents background tasks from jumping to the global default_model
     # when the user is mid-conversation with a different model.
     if not ep_id and fallback_url and fallback_model:
         return fallback_url, fallback_model, fallback_headers
 
-    # Unset Utility means "same as Default Chat Model".
-    if setting_prefix == "utility" and not ep_id:
+    # Unset Utility (or anything else that didn't have a fallback) means "same as Default Chat Model".
+    if not ep_id:
         ep_id = _stg("default_endpoint_id")
         model = _stg("default_model")
 
-    # Fall back to utility model for task/research/auto-naming if not specifically configured.
-    # If Utility itself is unset, the block above makes that resolve to Default Chat.
-    if not ep_id and setting_prefix != "utility":
-        ep_id = _stg("utility_endpoint_id")
-        model = _stg("utility_model")
-        if not ep_id:
-            ep_id = _stg("default_endpoint_id")
-            model = _stg("default_model")
-
     if not ep_id:
         return fallback_url, fallback_model, fallback_headers
 

src/integrations.py

@@ -6,6 +6,7 @@ import re
 from typing import Dict, List, Optional, Any
 
 import httpx
+from fastapi import HTTPException
 
 from core.atomic_io import atomic_write_json
 from core.platform_compat import safe_chmod
@@ -258,6 +259,11 @@ def add_integration(data: Dict[str, Any]) -> Dict[str, Any]:
     integration.setdefault("name", "")
     integration.setdefault("base_url", "")
 
+    if not isinstance(integration.get("name"), str) or not integration["name"].strip():
+        raise HTTPException(400, "Integration name is required")
+    if not isinstance(integration.get("base_url"), str) or not integration["base_url"].strip():
+        raise HTTPException(400, "Integration base URL is required")
+
     integrations = load_integrations()
     integrations.append(integration)
     save_integrations(integrations)
@@ -266,6 +272,11 @@ def add_integration(data: Dict[str, Any]) -> Dict[str, Any]:
 
 def update_integration(integration_id: str, data: Dict[str, Any]) -> Optional[Dict[str, Any]]:
     """Update fields on an existing integration. Returns updated integration or None."""
+    if "name" in data and (not isinstance(data["name"], str) or not data["name"].strip()):
+        raise HTTPException(400, "Integration name is required")
+    if "base_url" in data and (not isinstance(data["base_url"], str) or not data["base_url"].strip()):
+        raise HTTPException(400, "Integration base URL is required")
+
     integrations = load_integrations()
     for item in integrations:
         if item.get("id") == integration_id:

src/llm_core.py

@@ -7,6 +7,7 @@ import logging
 import hashlib
 import threading
 import re
+import os
 from fastapi import HTTPException
 from typing import Optional, Dict, List, Tuple
 from src.model_context import get_context_length, DEFAULT_CONTEXT
@@ -22,6 +23,24 @@ class LLMConfig:
     MAX_RETRIES = 3
     RETRY_DELAY = 0.5
     STREAM_TIMEOUT = 300
+    # TCP+TLS connect budget for a SINGLE attempt. The old hard-coded 3.0s
+    # assumed LAN/Tailscale peers ('SYN in <100ms'); it is too tight for public
+    # cloud endpoints (offshore APIs take ~0.5-1.5s cold, with jitter), so a
+    # brief blip on the first connect of an idle chat surfaced as a 503 on the
+    # streaming path (which, unlike llm_call, does not retry the connect). A
+    # genuinely dead upstream stays bounded by the dead-host cooldown. Override
+    # with env LLM_CONNECT_TIMEOUT (seconds).
+    CONNECT_TIMEOUT = float(os.getenv('LLM_CONNECT_TIMEOUT', '10') or '10')
+
+
+def _call_timeout(read_timeout) -> httpx.Timeout:
+    """Per-request timeout for non-streaming LLM calls (connect from config)."""
+    return httpx.Timeout(connect=LLMConfig.CONNECT_TIMEOUT, read=float(read_timeout), write=10.0, pool=5.0)
+
+
+def _stream_timeout(read_timeout) -> httpx.Timeout:
+    """Per-request timeout for streaming LLM calls (connect from config)."""
+    return httpx.Timeout(connect=LLMConfig.CONNECT_TIMEOUT, read=float(read_timeout), write=30.0, pool=5.0)
 
 
 # Cache for LLM responses
@@ -1446,7 +1465,7 @@ async def llm_call_async(
     if _is_host_dead(target_url):
         raise HTTPException(503, f"Upstream {_host_key(target_url)} marked unreachable (cooldown active)")
 
-    call_timeout = httpx.Timeout(connect=3.0, read=float(timeout), write=10.0, pool=5.0)
+    call_timeout = _call_timeout(timeout)
     attempt = 0
     while attempt < max_retries:
         attempt += 1
@@ -1570,9 +1589,12 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
             from src.copilot import apply_request_headers
             apply_request_headers(h, messages_copy)
 
-    # Short connect timeout: a reachable peer answers SYN in <100ms even on
-    # Tailscale. 3s is plenty; 30s let one dead upstream wedge the UI.
-    stream_timeout = httpx.Timeout(connect=3.0, read=float(timeout), write=30.0, pool=5.0)
+    # Connect budget from LLMConfig.CONNECT_TIMEOUT (env LLM_CONNECT_TIMEOUT).
+    # The dead-host cooldown still bounds a genuinely unreachable upstream, so a
+    # wider connect budget only affects first contact and stops a brief cold
+    # connect blip (offshore/public endpoints) surfacing as a 503 on this stream
+    # path, which -- unlike llm_call -- does not retry the connect.
+    stream_timeout = _stream_timeout(timeout)
 
     if _is_host_dead(target_url):
         yield f'event: error\ndata: {json.dumps({"error": f"Upstream {_host_key(target_url)} unreachable (cooldown active)", "status": 503})}\n\n'

src/model_context.py

@@ -222,16 +222,12 @@ KNOWN_CONTEXT_WINDOWS = {
 # ---------------------------------------------------------------------------
 # Cache
 # ---------------------------------------------------------------------------
-_context_cache: Dict[Tuple[str, str], int] = {}
+_context_cache: Dict[Tuple[str, str], Tuple[int, bool]] = {}
 
 
-def get_context_length(endpoint_url: str, model: str) -> int:
-    """Get the context window size for a model.
-
-    Queries /v1/models on the endpoint and looks for context_length
-    or context_window fields. Caches result per (endpoint, model).
-    Falls back to DEFAULT_CONTEXT if unavailable.
-    """
+def _get_context_length_cached(endpoint_url: str, model: str) -> Tuple[int, bool]:
+    """Return (context_length, known). ``known`` is False only when the value is a
+    bare DEFAULT_CONTEXT fallback (no endpoint report and not in the known table)."""
     configured_kind = _configured_endpoint_kind(endpoint_url)
     is_local = is_local_endpoint(endpoint_url)
     # Key on (endpoint_url, model): the same model id can be served by two
@@ -242,14 +238,50 @@ def get_context_length(endpoint_url: str, model: str) -> int:
     if not is_local and cache_key in _context_cache:
         return _context_cache[cache_key]
 
-    ctx = _query_context_length(endpoint_url, model)
+    ctx, known = _query_context_length(endpoint_url, model)
     # Only cache non-default values to allow retry on next request.
     # Local endpoints can restart with a different --max-model-len while keeping
     # the same model id, so always re-query them instead of serving stale cache.
     if not is_local and (ctx != DEFAULT_CONTEXT or configured_kind in ("api", "proxy")):
-        _context_cache[cache_key] = ctx
+        _context_cache[cache_key] = (ctx, known)
     logger.info(f"Context length for {model}: {ctx}")
-    return ctx
+    return ctx, known
+
+
+def get_context_length(endpoint_url: str, model: str) -> int:
+    """Get the context window size for a model.
+
+    Queries /v1/models on the endpoint and looks for context_length
+    or context_window fields. Caches result per (endpoint, model).
+    Falls back to DEFAULT_CONTEXT if unavailable.
+    """
+    return _get_context_length_cached(endpoint_url, model)[0]
+
+
+def get_context_length_known(endpoint_url: str, model: str) -> Tuple[int, bool]:
+    """Like ``get_context_length`` but also returns whether the window was actually
+    discovered (endpoint-reported or in the known-models table) rather than the bare
+    DEFAULT_CONTEXT fallback. Callers that *scale* a budget off the window must not
+    trust an unknown value — a fallback 128K isn't proof the model holds 128K
+    (review on #4122)."""
+    return _get_context_length_cached(endpoint_url, model)
+
+
+def budget_context_for_model(endpoint_url: str, model: str, *, fallback: int = 0) -> int:
+    """Context window to scale the agent input budget against.
+
+    Returns the *freshly discovered* window when it was actually proven
+    (endpoint-reported / known table), else 0 so auto-scaling stays conservative.
+    Crucially this binds the ``known`` flag to the value it proves — callers must
+    not pair this flag with a context length from a *different* lookup (a stale
+    local re-query, or a caller that didn't pass one), which would budget off an
+    unproven number (review on #4122). On probe error, returns ``fallback`` (the
+    caller's best-known value) to preserve prior behaviour."""
+    try:
+        ctx, known = get_context_length_known(endpoint_url, model)
+        return ctx if known else 0
+    except Exception:
+        return fallback
 
 
 def _lookup_known(model: str) -> Optional[int]:
@@ -271,8 +303,9 @@ def _lookup_known(model: str) -> Optional[int]:
     return best_ctx
 
 
-def _query_context_length(endpoint_url: str, model: str) -> int:
-    """Query the model API for context length."""
+def _query_context_length(endpoint_url: str, model: str) -> Tuple[int, bool]:
+    """Query the model API for context length. Returns (context_length, known) where
+    ``known`` is False only for the bare DEFAULT_CONTEXT fallback."""
     known = _lookup_known(model)
     api_ctx = None
     configured_kind = _configured_endpoint_kind(endpoint_url)
@@ -283,8 +316,8 @@ def _query_context_length(endpoint_url: str, model: str) -> int:
     if configured_kind in ("api", "proxy"):
         if known:
             logger.info(f"Using known context window for {model}: {known}")
-            return known
-        return DEFAULT_CONTEXT
+            return known, True
+        return DEFAULT_CONTEXT, False
 
     # Try llama.cpp /slots endpoint first — reports actual serving context
     if is_local_endpoint(endpoint_url):
@@ -297,7 +330,7 @@ def _query_context_length(endpoint_url: str, model: str) -> int:
                     n_ctx = slots[0].get("n_ctx")
                     if n_ctx and isinstance(n_ctx, int) and n_ctx > 0:
                         logger.info(f"llama.cpp /slots reports n_ctx={n_ctx} for {model}")
-                        return n_ctx
+                        return n_ctx, True
         except Exception:
             pass
 
@@ -309,7 +342,8 @@ def _query_context_length(endpoint_url: str, model: str) -> int:
     if is_copilot_base(endpoint_url):
         if known:
             logger.info(f"Using known context window for {model}: {known}")
-        return known or DEFAULT_CONTEXT
+            return known, True
+        return DEFAULT_CONTEXT, False
 
     from src.endpoint_resolver import build_models_url
 
@@ -354,18 +388,18 @@ def _query_context_length(endpoint_url: str, model: str) -> int:
         _is_local = is_local_endpoint(endpoint_url)
         if _is_local and api_ctx < known:
             logger.info(f"Local endpoint reports {api_ctx} for {model} (known max: {known}) — using API value")
-            return api_ctx
+            return api_ctx, True
         result = max(api_ctx, known)
         if api_ctx < known:
             logger.info(f"API reported {api_ctx} for {model}, using known {known} instead")
-        return result
+        return result, True
     if api_ctx:
-        return api_ctx
+        return api_ctx, True
     if known:
         logger.info(f"Using known context window for {model}: {known}")
-        return known
+        return known, True
 
-    return DEFAULT_CONTEXT
+    return DEFAULT_CONTEXT, False
 
 
 def estimate_tokens(messages: List[Dict]) -> int:

src/optional_deps.py

@@ -0,0 +1,32 @@
+"""Compatibility helpers for optional third-party dependencies."""
+
+from __future__ import annotations
+
+import sys
+import types
+
+
+def patch_realesrgan_torchvision_compat() -> None:
+    """Restore the torchvision import path expected by BasicSR/Real-ESRGAN."""
+    module_name = "torchvision.transforms.functional_tensor"
+    if module_name in sys.modules:
+        return
+    try:
+        from torchvision.transforms import functional
+    except Exception:
+        return
+
+    rgb_to_grayscale = getattr(functional, "rgb_to_grayscale", None)
+    if rgb_to_grayscale is None:
+        return
+
+    shim = types.ModuleType(module_name)
+    shim.rgb_to_grayscale = rgb_to_grayscale
+    shim.__getattr__ = lambda name: getattr(functional, name)
+    sys.modules[module_name] = shim
+
+
+def prepare_optional_dependency_import(name: str) -> None:
+    """Apply known import-time compatibility shims before probing a package."""
+    if name == "realesrgan":
+        patch_realesrgan_torchvision_compat()

src/settings.py

@@ -101,14 +101,22 @@ DEFAULT_SETTINGS = {
     "research_run_timeout_seconds": 1800,
     "agent_max_tool_calls": 0,
     "agent_max_rounds": 20,  # per-message agent step cap (clamped 1..200)
+    # Soft input-token budget for the agent loop. The DEFAULT value (6000) is the
+    # "auto" sentinel: it means "scale the budget to the model's context window"
+    # (#1230) — so long-context models aren't capped at 6000. Set ANY OTHER value
+    # to enforce an explicit cap (clamped to the window only — hard_max does not
+    # apply to explicit budgets, #1230); set 0 to disable soft-trimming. The
+    # default is treated as auto because the settings-save path materializes
+    # defaults, so a persisted 6000 can't be told apart from a deliberate 6000 —
+    # to pin a budget near the default, use a nearby value (e.g. 5999).
     "agent_input_token_budget": 6000,
-    # Ceiling on the *auto-derived* input budget that #1230 introduced. Has
-    # no effect when `agent_input_token_budget` is explicitly set (the user's
-    # value is honoured regardless). Default matches
-    # `src.context_budget.DEFAULT_HARD_MAX`; lower this for cost-paranoid
-    # setups, raise it on premium APIs with very large windows that you
+    # Ceiling on the *auto-derived* input budget; a configurable setting since #1273
+    # (the merged #1230 left it a module constant). No effect on an explicit budget
+    # — a deliberate value is honoured (#1230). Default matches
+    # `src.context_budget.DEFAULT_HARD_MAX`; lower this for
+    # cost-paranoid setups, raise it on premium APIs with very large windows you
     # want to actually use (e.g. 900_000 to fill a 1M-context model). See
-    # `compute_input_token_budget` in src/context_budget.py.
+    # `compute_input_token_budget`.
     "agent_input_token_hard_max": 200_000,
     "agent_stream_timeout_seconds": 300,
     # Extra directory roots that read_file / write_file may access, in
@@ -223,8 +231,10 @@ def is_setting_overridden(key: str) -> bool:
 
     ``load_settings`` merges DEFAULT_SETTINGS with the saved file, so a value
     equal to its default is indistinguishable from "never set" via get_setting.
-    Callers that need to treat an explicit user choice differently from the
-    default (e.g. adaptive budgets) use this to read the raw saved file.
+    Callers that must distinguish an explicit user choice from a default read
+    the raw saved file via this. (Note: a materialized default is also "present",
+    so value-sensitive callers should compare against the default — see
+    ``context_budget.budget_is_explicit``.)
     """
     try:
         with open(SETTINGS_FILE, "r", encoding="utf-8") as f:

src/task_scheduler.py

@@ -1649,6 +1649,8 @@ class TaskScheduler:
                     data = json.loads(event_str[6:])
                     # Capture text from all event types, not just delta
                     if "delta" in data:
+                        if data.get("thinking"):
+                            continue
                         full_text += data["delta"]
                     elif data.get("type") == "tool_output":
                         # Tool results — capture summary so we have SOMETHING even

src/teacher_escalation.py

@@ -594,6 +594,8 @@ async def run_teacher_inline(
                         "exit_code": payload.get("exit_code"),
                     })
                 if "delta" in payload and isinstance(payload["delta"], str):
+                    if payload.get("thinking"):
+                        continue
                     captured_text_parts.append(payload["delta"])
                 yield 'data: ' + json.dumps(payload) + '\n\n'
                 continue

src/tool_implementations.py

@@ -1643,6 +1643,9 @@ async def do_manage_calendar(content: str, owner: Optional[str] = None) -> Dict:
             except ValueError as e:
                 return {"error": f"Invalid date format: {e}", "exit_code": 1}
 
+            if end_dt <= start_dt:
+                end_dt = start_dt + timedelta(days=1)
+
             q = _event_query().filter(
                 CalendarEvent.dtstart < end_dt,
                 CalendarEvent.dtend > start_dt,

src/tool_index.py

@@ -384,6 +384,10 @@ class ToolIndex:
                    "delegate to", "have model"}):
             {"chat_with_model", "ask_teacher", "list_models"},
         # Deep research intent (incl. common typo "reserach")
+        frozenset({"web search", "search the web", "search online", "look up",
+                   "google", "latest", "current", "news", "weather",
+                   "forecast", "stock price", "price of"}):
+            {"web_search", "web_fetch"},
         frozenset({"research", "reserach", "reasearch", "look into", "investigate",
                    "deep dive", "deep research", "find out about", "study up on",
                    "report on", "do research", "look up everything"}):

src/tool_parsing.py

@@ -188,6 +188,12 @@ _MISFENCED_WEB_TOOL_NAMES = {
     "fetch_url": "web_fetch",
 }
 
+_RAW_WEB_JSON_TOOL_RE = re.compile(
+    r"\b(?:web_search|websearch|google_search|google_search_retrieval|google_search_grounding)\b",
+    re.IGNORECASE,
+)
+_RAW_WEB_JSON_ALLOWED_KEYS = {"query", "queries", "time_filter", "freshness", "max_pages"}
+
 
 # ---------------------------------------------------------------------------
 # Parsing functions
@@ -279,6 +285,73 @@ def _parse_misfenced_web_lookup(content: str) -> Optional[ToolBlock]:
         return None
     return ToolBlock("web_fetch", url)
 
+
+def _coerce_raw_web_query(value) -> Optional[str]:
+    if isinstance(value, str) and value.strip():
+        return value.strip()
+    if isinstance(value, list):
+        for item in value:
+            if isinstance(item, str) and item.strip():
+                return item.strip()
+    return None
+
+
+def _raw_web_json_to_tool_block(payload) -> Optional[ToolBlock]:
+    if not isinstance(payload, dict):
+        return None
+    if set(payload) - _RAW_WEB_JSON_ALLOWED_KEYS:
+        return None
+
+    query = _coerce_raw_web_query(payload.get("query"))
+    if not query:
+        query = _coerce_raw_web_query(payload.get("queries"))
+    if not query:
+        return None
+
+    content = {"query": query}
+    for key in ("time_filter", "freshness"):
+        value = payload.get(key)
+        if isinstance(value, str) and value.strip().lower() in ("day", "week", "month", "year"):
+            content[key] = value.strip().lower()
+
+    max_pages = payload.get("max_pages")
+    if isinstance(max_pages, int) and 1 <= max_pages <= 10:
+        content["max_pages"] = max_pages
+
+    if len(content) == 1:
+        return ToolBlock("web_search", query)
+    return ToolBlock("web_search", json.dumps(content))
+
+
+def _parse_raw_web_json_lookup(text: str) -> Optional[tuple[ToolBlock, tuple[int, int]]]:
+    """Recover local text-model web_search calls emitted as prose + bare JSON.
+
+    Some non-native tool models leak the intended call as:
+
+        Need to do web_search for ...
+        {"query": "...", "time_filter": "week"}
+
+    Keep this narrower than fenced/tool markup: it only runs when a known web
+    tool name appears shortly before a JSON object shaped like web_search args.
+    """
+    if not isinstance(text, str):
+        return None
+
+    decoder = json.JSONDecoder()
+    for mention in _RAW_WEB_JSON_TOOL_RE.finditer(text):
+        search_start = mention.end()
+        search_end = min(len(text), search_start + 1200)
+        for brace in re.finditer(r"\{", text[search_start:search_end]):
+            start = search_start + brace.start()
+            try:
+                parsed, end = decoder.raw_decode(text[start:])
+            except json.JSONDecodeError:
+                continue
+            block = _raw_web_json_to_tool_block(parsed)
+            if block:
+                return block, (start, start + end)
+    return None
+
 def _parse_tool_call_block(raw: str) -> Optional[ToolBlock]:
     """Parse a [TOOL_CALL] block into a ToolBlock.
 
@@ -436,6 +509,8 @@ def parse_tool_blocks(text: str, skip_fenced: bool = False) -> List[ToolBlock]:
     3. XML-style <tool_call>/<invoke> blocks
     4. <tool_code> blocks (MiniMax-M2.5 style)
     5. DeepSeek DSML markup (normalized to <invoke> first)
+    6. Non-native local model fallback: prose mentioning web_search followed by
+       bare JSON args, e.g. {"query":"...", "time_filter":"week"}
 
     `skip_fenced`: when True, Pattern 1 (fenced ```bash/```python/```json code
     blocks) is not matched at all. Native function-calling models (GPT/Claude/
@@ -509,6 +584,12 @@ def parse_tool_blocks(text: str, skip_fenced: bool = False) -> List[ToolBlock]:
             if block:
                 blocks.append(block)
 
+    # Pattern 6: local text-model web_search call leaked as prose + bare JSON.
+    if not blocks and not skip_fenced:
+        raw_web_json = _parse_raw_web_json_lookup(text)
+        if raw_web_json:
+            blocks.append(raw_web_json[0])
+
     return blocks
 
 
@@ -532,6 +613,11 @@ def strip_tool_blocks(text: str, skip_fenced: bool = False) -> str:
     cleaned = _TOOL_CALL_RE.sub('', cleaned)
     cleaned = _XML_TOOL_CALL_RE.sub('', cleaned)
     cleaned = _TOOL_CODE_RE.sub('', cleaned)
+    if not skip_fenced:
+        raw_web_json = _parse_raw_web_json_lookup(cleaned)
+        if raw_web_json:
+            _, (start, end) = raw_web_json
+            cleaned = cleaned[:start] + cleaned[end:]
     # Strip bare <invoke> blocks not wrapped in <tool_call>
     cleaned = re.sub(r'<invoke\s+name=["\'].*?</invoke>', '', cleaned, flags=re.DOTALL | re.IGNORECASE)
     cleaned = re.sub(r'\n{3,}', '\n\n', cleaned)

src/tool_security.py

@@ -177,13 +177,16 @@ def owner_is_admin_or_single_user(owner: Optional[str]) -> bool:
     defense-in-depth for callers that bypass it (e.g. trusted loopback).
     """
     try:
+        from src.auth_helpers import _auth_disabled
+
+        if _auth_disabled():
+            return True
+
         from core.auth import AuthManager
 
         auth = AuthManager()
         if not auth.is_configured:
-            from src.auth_helpers import _auth_disabled
-
-            return _auth_disabled()
+            return False
         return bool(owner and auth.is_admin(owner))
     except Exception as exc:
         logger.warning("Unable to evaluate owner admin status: %s", exc)

src/youtube_handler.py

@@ -1,278 +1,23 @@
-"""
-YouTube handling — transcript extraction, comment fetching (yt-dlp),
-and context formatting for LLM injection. Used by chat_handler.py.
+"""Compatibility wrapper for the canonical services.youtube.youtube_handler module.
+
+Odysseus historically carried two independent copies of the YouTube handler —
+one here under ``src`` and one under ``services.youtube``. They drifted: the
+comment-fetch timeout fix landed only in the ``src`` copy, while ``app.py``
+calls ``services.youtube.init_youtube()`` at startup. Because the chat flow
+imported ``extract_transcript_async`` from ``src.youtube_handler`` (a different
+module object), the ``YOUTUBE_AVAILABLE`` / ``YouTubeTranscriptApi`` globals set
+by ``init_youtube`` never reached it and transcript extraction always reported
+"YouTube transcript API not available".
+
+Keep the old ``src.youtube_handler`` import path working, but make it resolve to
+the single source of truth so module state and behavior can't diverge again.
 """
 
-import asyncio
-import json
-import logging
-import shutil
+import importlib
 import sys
-import urllib.parse
-from pathlib import Path
-from typing import Dict, Any, Optional
 
-logger = logging.getLogger(__name__)
+# Import the canonical module directly (services.youtube.youtube_handler)
+# without triggering the heavy services/__init__.py top-level imports.
+_youtube_handler = importlib.import_module("services.youtube.youtube_handler")
 
-# ---------------------------------------------------------------------------
-# Constants
-# ---------------------------------------------------------------------------
-
-YOUTUBE_INSTRUCTION_PROMPT = """When the user shares a YouTube video, respond with a structured breakdown:
-
-1. **Summary** — Concise overview of the video's content and main thesis (2-4 sentences)
-2. **Key Points** — Bullet list of the most important topics, arguments, or moments
-3. **Notable Timestamps** — If timestamps are available from the transcript, highlight 3-5 interesting moments with their approximate timestamps (e.g. "03:45 — discusses X")
-4. **Audience Reception** — If comments are available, summarize what viewers think: general sentiment, top reactions, any debate or controversy
-
-Keep it conversational and concise. Do NOT web search for this video — use only the transcript and comments provided."""
-
-# ---------------------------------------------------------------------------
-# Init / helpers
-# ---------------------------------------------------------------------------
-
-# Will be set at startup by init_youtube()
-YouTubeTranscriptApi = None
-YOUTUBE_AVAILABLE = False
-
-
-def _find_ytdlp() -> str:
-    """Find the yt-dlp binary: venv bin first, then system PATH."""
-    venv_bin = Path(sys.executable).parent / "yt-dlp"
-    if venv_bin.exists():
-        return str(venv_bin)
-    found = shutil.which("yt-dlp")
-    return found or "yt-dlp"
-
-
-def init_youtube():
-    """Import and cache the YouTube transcript API."""
-    global YouTubeTranscriptApi, YOUTUBE_AVAILABLE
-    try:
-        from youtube_transcript_api import YouTubeTranscriptApi as _Api
-        YouTubeTranscriptApi = _Api
-        YOUTUBE_AVAILABLE = True
-        logger.info("YouTube transcript API available")
-    except ImportError as e:
-        logger.warning(f"youtube-transcript-api not installed: {e}")
-        YOUTUBE_AVAILABLE = False
-
-
-def is_youtube_url(url: str) -> bool:
-    if not isinstance(url, str):
-        return False
-    return "youtube.com" in url or "youtu.be" in url
-
-
-def extract_youtube_id(url: str) -> Optional[str]:
-    """Extract YouTube video ID from various URL formats."""
-    parsed = urllib.parse.urlparse(url)
-    if parsed.hostname in ("www.youtube.com", "youtube.com", "m.youtube.com"):
-        if parsed.path == "/watch":
-            params = urllib.parse.parse_qs(parsed.query)
-            if "v" in params:
-                return params["v"][0]
-        elif parsed.path.startswith("/embed/"):
-            return parsed.path.split("/")[-1]
-    elif parsed.hostname == "youtu.be":
-        return parsed.path[1:]
-    return None
-
-
-async def extract_transcript_async(
-    url: str, video_id: str, max_retries: int = 3
-) -> Dict[str, Any]:
-    """
-    Async YouTube transcript extraction with retries.
-
-    Args:
-        url: Full YouTube URL
-        video_id: Extracted video ID
-        max_retries: Number of attempts
-
-    Returns:
-        Dict with success/error/transcript keys
-    """
-    if not YOUTUBE_AVAILABLE or YouTubeTranscriptApi is None:
-        return {"success": False, "error": "YouTube transcript API not available", "transcript": None}
-
-    for attempt in range(max_retries):
-        try:
-            api = YouTubeTranscriptApi()
-            transcript = api.fetch(video_id)
-            transcript_list = list(transcript)
-
-            formatted = []
-            for snippet in transcript_list:
-                text = snippet.text.strip()
-                if not text:
-                    continue
-                start = snippet.start
-                formatted.append({
-                    "text": text,
-                    "start": start,
-                    "duration": snippet.duration,
-                    "timestamp": f"{int(start // 60):02d}:{int(start % 60):02d}",
-                })
-
-            full_text = " ".join(e["text"] for e in formatted)
-            max_len = 8000
-            if len(full_text) > max_len:
-                full_text = full_text[:max_len] + "... [transcript truncated]"
-
-            return {
-                "success": True,
-                "transcript": full_text,
-                "video_id": video_id,
-                "language": "en",
-                "is_generated": False,
-                "segments": formatted,
-            }
-        except Exception as e:
-            logger.warning(f"Transcript attempt {attempt + 1} failed: {e}")
-            if attempt < max_retries - 1:
-                await asyncio.sleep(1 * (attempt + 1))
-
-    return {"success": False, "error": f"Failed after {max_retries} attempts", "transcript": None}
-
-
-def format_transcript_for_context(
-    transcript_data: Dict[str, Any], url: str,
-    title: str = "", channel: str = ""
-) -> str:
-    """Format transcript data for inclusion in LLM context."""
-    if not transcript_data.get("success"):
-        header = ""
-        if title:
-            header = f" \"{title}\""
-            if channel:
-                header += f" by {channel}"
-        return f"\n[YouTube Video{header}: Transcript unavailable ({transcript_data.get('error', 'Unknown error')}). Use the comments below if available, do NOT web search for this video.]"
-
-    transcript = transcript_data.get("transcript", "")
-    video_id = transcript_data.get("video_id", "")
-    language = transcript_data.get("language", "unknown")
-    is_generated = transcript_data.get("is_generated", False)
-    segments = transcript_data.get("segments", [])
-
-    ctx = "\n[YOUTUBE VIDEO TRANSCRIPT]\n"
-    if title:
-        ctx += f"Title: {title}\n"
-    if channel:
-        ctx += f"Channel: {channel}\n"
-    ctx += f"Video ID: {video_id}\n"
-    ctx += f"Language: {language}\n"
-    ctx += f"Source: {'Auto-generated' if is_generated else 'Manual'}\n"
-    ctx += f"URL: {url}\n\n"
-    # Include timestamped segments for the LLM to reference
-    if segments:
-        ctx += "Timestamped Transcript:\n"
-        for seg in segments:
-            if not isinstance(seg, dict):
-                continue
-            ctx += f"[{seg['timestamp']}] {seg['text']}\n"
-        # Check length — fall back to plain text if too long
-        if len(ctx) > 12000:
-            ctx = ctx[:ctx.index("Timestamped Transcript:\n")]
-            ctx += "Transcript:\n"
-            ctx += transcript
-    else:
-        ctx += "Transcript:\n"
-        ctx += transcript
-    ctx += "\n[END TRANSCRIPT]\n"
-    return ctx
-
-
-async def fetch_youtube_comments(
-    video_id: str, max_comments: int = 25, timeout: int = 30
-) -> Dict[str, Any]:
-    """Fetch top comments for a YouTube video using yt-dlp.
-
-    Returns dict with 'success', 'comments' list, 'error'.
-    """
-    try:
-        cmd = [
-            _find_ytdlp(),
-            "--skip-download",
-            "--write-comments",
-            "--extractor-args", f"youtube:max_comments={max_comments},all,100,0",
-            "--dump-json",
-            "--js-runtimes", "node",
-            "--remote-components", "ejs:github",
-            f"https://www.youtube.com/watch?v={video_id}",
-        ]
-
-        proc = await asyncio.create_subprocess_exec(
-            *cmd,
-            stdout=asyncio.subprocess.PIPE,
-            stderr=asyncio.subprocess.PIPE,
-        )
-        # Bound the wait on the process actually finishing, not on spawning it.
-        # create_subprocess_exec returns as soon as the child starts, so wrapping
-        # it in wait_for never enforces the timeout — proc.communicate() is the
-        # blocking step. Kill and reap the child if it overruns so it does not
-        # linger after we return.
-        try:
-            stdout, stderr = await asyncio.wait_for(
-                proc.communicate(), timeout=timeout
-            )
-        except asyncio.TimeoutError:
-            proc.kill()
-            await proc.wait()
-            raise
-
-        if proc.returncode != 0:
-            return {"success": False, "error": f"yt-dlp failed: {stderr.decode()[:200]}", "comments": []}
-
-        data = json.loads(stdout.decode())
-        title = data.get("title", "")
-        channel = data.get("channel", "") or data.get("uploader", "")
-        raw_comments = data.get("comments", [])
-
-        comments = []
-        for c in raw_comments[:max_comments]:
-            text = (c.get("text") or "").strip()
-            if not text:
-                continue
-            comments.append({
-                "author": c.get("author", "Unknown"),
-                "text": text,
-                "likes": c.get("like_count", 0),
-            })
-
-        # Sort by likes descending — most popular comments first
-        comments.sort(key=lambda x: x.get("likes", 0), reverse=True)
-
-        return {"success": True, "comments": comments, "count": len(comments),
-                "title": title, "channel": channel}
-
-    except asyncio.TimeoutError:
-        logger.warning(f"Comment fetch timed out for {video_id}")
-        return {"success": False, "error": "Comment fetch timed out", "comments": []}
-    except FileNotFoundError:
-        logger.warning("yt-dlp not installed — cannot fetch comments")
-        return {"success": False, "error": "yt-dlp not installed", "comments": []}
-    except Exception as e:
-        logger.warning(f"Failed to fetch comments for {video_id}: {e}")
-        return {"success": False, "error": str(e), "comments": []}
-
-
-def format_comments_for_context(comments_data: Dict[str, Any], url: str) -> str:
-    """Format YouTube comments for inclusion in LLM context."""
-    if not comments_data.get("success") or not comments_data.get("comments"):
-        return ""
-
-    comments = comments_data["comments"]
-    ctx = f"\n[YOUTUBE VIDEO COMMENTS — Top {len(comments)} by popularity]\n"
-    ctx += f"URL: {url}\n\n"
-
-    for i, c in enumerate(comments, 1):
-        likes = c.get("likes", 0)
-        likes_str = f" [{likes} likes]" if likes else ""
-        ctx += f"{i}. @{c['author']}{likes_str}: {c['text']}\n\n"
-
-    if len(ctx) > 4000:
-        ctx = ctx[:4000] + "\n[Comments truncated]\n"
-
-    ctx += "[END COMMENTS]\n"
-    return ctx
+sys.modules[__name__] = _youtube_handler

static/app.js

@@ -3135,7 +3135,9 @@ function initializeEventListeners() {
       setTimeout(() => uiModule.autoResize(textarea), 1);
     });
     textarea.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
+      const isMobile = window.innerWidth <= 768
+
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing && !isMobile) {
         // If ghost autocomplete is active, accept the suggestion instead of submitting
         if (window._ghostAutocomplete && window._ghostAutocomplete.isActive()) {
           e.preventDefault();
@@ -3708,7 +3710,9 @@ function startOdysseusApp() {
   // Enter to send (shift+enter for newline), or new chat when empty
   if (messageInput) {
     messageInput.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
+      const isMobile = window.innerWidth <= 768
+
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing && !isMobile) {
         e.preventDefault();
         // Flush the debounced icon update so dataset.mode reflects the current
         // text state. Without this, a fast type-and-Enter would still see the

static/js/calendar.js

@@ -9,7 +9,7 @@ import { makeWindowDraggable } from './windowDrag.js';
 import { attachColorPicker } from './colorPicker.js';
 import { bindMenuDismiss } from './escMenuStack.js';
 import {
-  WEEKDAYS, MONTHS, MON_SHORT,
+  WEEKDAYS, WEEKDAYS_SUN, MONTHS, MON_SHORT,
   CAL_PALETTE, CAL_COLORS, _CAL_CUSTOM_GRADIENT, _TYPE_PALETTE,
   _trashIcon, _moreIcon, _bellIcon,
   _isCalBgImage, _calBgImageUrl, _calBgCss,
@@ -64,6 +64,8 @@ let _hiddenTypes = new Set();   // event_type values to hide
 let _onlyImportant = false;
 
 let _filtersCollapsed = localStorage.getItem('cal-filters-collapsed') === '1';
+// Week-start preference: 'mon' (default, Mon=first col) or 'sun' (Sun=first col).
+let _weekStartSun = localStorage.getItem('cal-week-start') === 'sun';
 let _selectedDay = null;
 let _view = 'month';
 let _searchQuery = '';
@@ -360,14 +362,14 @@ function _today() { return _ds(new Date()); }
 function _monthRange(d) {
   const y = d.getFullYear(), m = d.getMonth();
   const first = new Date(y, m, 1);
-  const dow = (first.getDay() + 6) % 7;
+  const dow = _weekStartSun ? first.getDay() : (first.getDay() + 6) % 7;
   const gs = new Date(y, m, 1 - dow);
   const ge = new Date(gs); ge.setDate(gs.getDate() + 42);
   return [_ds(gs), _ds(ge)];
 }
 
 function _weekRange(d) {
-  const dow = (d.getDay() + 6) % 7;
+  const dow = _weekStartSun ? d.getDay() : (d.getDay() + 6) % 7;
   const s = new Date(d); s.setDate(d.getDate() - dow);
   const e = new Date(s); e.setDate(s.getDate() + 7);
   return [_ds(s), _ds(e)];
@@ -928,11 +930,11 @@ async function _renderMonth() {
   _slideDir = 0;
   let h = _headerHTML() + _filtersRowHTML() + `<div class="cal-grid${slideClass}">`;
   h += '<div class="cal-week-headers">';
-  for (const wd of WEEKDAYS) h += `<div class="cal-weekday">${wd}</div>`;
+  for (const wd of (_weekStartSun ? WEEKDAYS_SUN : WEEKDAYS)) h += `<div class="cal-weekday">${wd}</div>`;
   h += '</div>';
 
   const first = new Date(y, m, 1);
-  const dow = (first.getDay() + 6) % 7;
+  const dow = _weekStartSun ? first.getDay() : (first.getDay() + 6) % 7;
   const gs = new Date(y, m, 1 - dow);
 
   const multiDay = _events.filter(e => {
@@ -1141,13 +1143,13 @@ function _wkEventTopHeight(ev, dayStr) {
   // Date math if the string isn't shaped as expected.
   const _toMin = (iso, fallbackDate) => {
     if (!iso) return null;
-    const m = iso.match(/T(\d{2}):(\d{2})/);
-    if (m) {
+    const mins = _timeToMin(iso);
+    if (mins !== null && iso.includes('T')) {
       // If the event spans into a previous/next day, clamp to today's bounds.
-      const evDate = iso.slice(0, 10);
+      const evDate = _localDateOf(iso);
       if (evDate < fallbackDate) return 0;             // event started before today
       if (evDate > fallbackDate) return 24 * 60;       // event ends after today
-      return parseInt(m[1], 10) * 60 + parseInt(m[2], 10);
+      return mins;
     }
     // All-day or date-only — treat as start of day.
     return 0;
@@ -1204,8 +1206,8 @@ async function _renderWeek() {
     const timedEvents  = _eventsForDay(ds).filter(e => _eventVisible(e) && !e.all_day);
 
     const isSun = d.getDay() === 0;
-    colsHtml += `<div class="cal-wk-col${isToday ? ' cal-wk-today' : ''}${isSun ? ' cal-wk-sun' : ''}" data-date="${ds}">`;
-    colsHtml += `<div class="cal-wk-col-head"><span class="cal-wk-dn">${WEEKDAYS[idx]}</span><span class="cal-wk-dt">${d.getDate()}</span></div>`;
+    colsHtml += `<div class="cal-wk-col${isToday ? ' cal-wk-today' : ''}${isSun && !_weekStartSun ? ' cal-wk-sun' : ''}" data-date="${ds}">`;
+    colsHtml += `<div class="cal-wk-col-head"><span class="cal-wk-dn">${(_weekStartSun ? WEEKDAYS_SUN : WEEKDAYS)[idx]}</span><span class="cal-wk-dt">${d.getDate()}</span></div>`;
     // All-day strip
     colsHtml += `<div class="cal-wk-allday">`;
     for (const ev of allDayEvents) {
@@ -1286,12 +1288,17 @@ async function _renderWeek() {
       if (!ev) return;
       const cols = Array.from(body.querySelectorAll('.cal-wk-grid'));
       if (!cols.length) return;
-      // Original timing
-      const m1 = (ev.dtstart || '').match(/T(\d{2}):(\d{2})/);
-      const m2 = (ev.dtend || '').match(/T(\d{2}):(\d{2})/);
-      const startMin0 = m1 ? parseInt(m1[1], 10) * 60 + parseInt(m1[2], 10) : 0;
-      const endMin0   = m2 ? parseInt(m2[1], 10) * 60 + parseInt(m2[2], 10) : startMin0 + 60;
-      const durationMin = Math.max(15, endMin0 - startMin0);
+      // Local/display timing
+      const startMin0 = _timeToMin(ev.dtstart) ?? 0;
+      const endMin0   = _timeToMin(ev.dtend) ?? startMin0 + 60;
+
+      let durationMin = endMin0 - startMin0;
+      const startDs = _localDateOf(ev.dtstart);
+      const endDs = ev.dtend ? _localDateOf(ev.dtend) : startDs;
+      if (endDs > startDs && endMin0 <= startMin0) {
+        durationMin += 24 * 60;
+      }
+      durationMin = Math.max(15, durationMin);
 
       // Where did the cursor grab the block? (offset from block-top in px)
       const blockRect = block.getBoundingClientRect();
@@ -1365,7 +1372,7 @@ async function _renderWeek() {
         // a plain click (no movement) must still open the event.
         if (moved) block.dataset.justResized = '1';
         // Decide whether anything actually moved.
-        const oldDs = (ev.dtstart || '').slice(0, 10);
+        const oldDs = _localDateOf(ev.dtstart);
         if (!nextDs) return;
         if (nextDs === oldDs && nextStartMin === startMin0) return;
         // Snapshot the original times so we can offer an Undo.
@@ -1374,11 +1381,10 @@ async function _renderWeek() {
         const newEndMin = nextStartMin + durationMin;
         const hh = String(Math.floor(nextStartMin / 60)).padStart(2, '0');
         const mm = String(nextStartMin % 60).padStart(2, '0');
-        const hh2 = String(Math.floor(newEndMin / 60)).padStart(2, '0');
-        const mm2 = String((newEndMin) % 60).padStart(2, '0');
-        const _tz = _tzOffset();
+        const newDtstartDate = new Date(`${nextDs}T${hh}:${mm}:00`);
+        const _tz = _tzOffsetForDate(newDtstartDate);
         const newDtstart = `${nextDs}T${hh}:${mm}:00${_tz}`;
-        const newDtend   = `${nextDs}T${hh2}:${mm2}:00${_tz}`;
+        const newDtend = _addMinutesToLocalIso(newDtstart, durationMin);
         try {
           await _updateEvent(uid, { dtstart: newDtstart, dtend: newDtend });
           _render();
@@ -1410,10 +1416,7 @@ async function _renderWeek() {
       const uid = block.dataset.uid;
       const ev = _events.find(x => x.uid === uid);
       if (!ev || !grid || !ds) return;
-      const startMin = (() => {
-        const m = (ev.dtstart || '').match(/T(\d{2}):(\d{2})/);
-        return m ? parseInt(m[1], 10) * 60 + parseInt(m[2], 10) : 0;
-      })();
+      const startMin = _timeToMin(ev.dtstart) ?? 0;
       const initialTop = parseFloat(block.style.top || '0');
       const gridRect = grid.getBoundingClientRect();
       let newEndMin = startMin;
@@ -1438,9 +1441,8 @@ async function _renderWeek() {
         if (resized) block.dataset.justResized = '1';
         if (newEndMin === startMin) return;
         const prevDtend = ev.dtend;
-        const hh = String(Math.floor(newEndMin / 60)).padStart(2, '0');
-        const mm = String(newEndMin % 60).padStart(2, '0');
-        const newDtend = `${ds}T${hh}:${mm}:00${_tzOffset()}`;
+        const durationMin = newEndMin - startMin;
+        const newDtend = _addMinutesToLocalIso(ev.dtstart, durationMin);
         try {
           await _updateEvent(uid, { dtend: newDtend });
           _render();
@@ -1724,9 +1726,9 @@ async function _renderYear() {
   for (let m = 0; m < 12; m++) {
     h += `<div class="cal-year-month" data-month="${m}"><div class="cal-year-month-title">${MON_SHORT[m]}</div>`;
     h += '<div class="cal-year-grid">';
-    for (const wd of ['M', 'T', 'W', 'T', 'F', 'S', 'S']) h += `<div class="cal-year-wd">${wd}</div>`;
+    for (const wd of (_weekStartSun ? ['S','M','T','W','T','F','S'] : ['M','T','W','T','F','S','S'])) h += `<div class="cal-year-wd">${wd}</div>`;
     const first = new Date(y, m, 1);
-    const dow = (first.getDay() + 6) % 7;
+    const dow = _weekStartSun ? first.getDay() : (first.getDay() + 6) % 7;
     const daysInMonth = new Date(y, m + 1, 0).getDate();
     for (let p = 0; p < dow; p++) h += '<div class="cal-year-cell"></div>';
     for (let d = 1; d <= daysInMonth; d++) {
@@ -1966,10 +1968,10 @@ function _wireAll(body) {
             const ad = document.getElementById('cal-f-allday');
             if (ad && !ad.checked) { ad.checked = true; ad.dispatchEvent(new Event('change')); }
           } else {
-            const t1 = (ev.dtstart || '').match(/T(\d{2}:\d{2})/);
-            const t2 = (ev.dtend || '').match(/T(\d{2}:\d{2})/);
-            if (t1) set('cal-f-start', t1[1]);
-            if (t2) set('cal-f-end', t2[1]);
+            const t1 = _fmtTime(ev.dtstart);
+            const t2 = _fmtTime(ev.dtend);
+            if (t1) set('cal-f-start', t1);
+            if (t2) set('cal-f-end', t2);
             document.getElementById('cal-f-start')?.dispatchEvent(new Event('input'));
           }
           // Make sure the details panel is open so the user can verify time.
@@ -2474,6 +2476,13 @@ async function _showCalSettings() {
           </div>
           <div style="font-size:10px;opacity:0.4;margin-top:4px;">Download a calendar as .ics for backup or to import into another app.</div>
         </div>
+        <div style="border-top:1px solid var(--border);padding-top:12px;">
+          <div style="font-size:11px;opacity:0.5;margin-bottom:6px;">Week starts on</div>
+          <div style="display:flex;gap:6px;">
+            <button id="cal-wstart-mon" type="button" style="font-size:12px;padding:3px 10px;border-radius:4px;border:1px solid var(--border);background:${!_weekStartSun ? 'color-mix(in srgb, var(--accent,var(--red)) 18%, var(--panel))' : 'var(--panel)'};color:var(--fg);cursor:pointer;transition:background 0.1s,border-color 0.1s;outline:none;">Monday</button>
+            <button id="cal-wstart-sun" type="button" style="font-size:12px;padding:3px 10px;border-radius:4px;border:1px solid var(--border);background:${_weekStartSun ? 'color-mix(in srgb, var(--accent,var(--red)) 18%, var(--panel))' : 'var(--panel)'};color:var(--fg);cursor:pointer;transition:background 0.1s,border-color 0.1s;outline:none;">Sunday</button>
+          </div>
+        </div>
         <div style="border-top:1px solid var(--border);padding-top:12px;">
           <div style="font-size:11px;opacity:0.5;margin-bottom:6px;">Sync</div>
           <div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
@@ -2494,6 +2503,28 @@ async function _showCalSettings() {
   overlay.querySelector('#cal-settings-close').addEventListener('click', cleanup);
   overlay.addEventListener('click', (e) => { if (e.target === overlay) cleanup(); });
 
+  // Week-start toggle: save to localStorage, update module state, re-render.
+  const _monBtn = overlay.querySelector('#cal-wstart-mon');
+  const _sunBtn = overlay.querySelector('#cal-wstart-sun');
+  const _activeStyle  = 'color-mix(in srgb, var(--accent,var(--red)) 18%, var(--panel))';
+  const _inactiveStyle = 'var(--panel)';
+  const _applyWeekStartActive = () => {
+    if (_monBtn) _monBtn.style.background = _weekStartSun ? _inactiveStyle : _activeStyle;
+    if (_sunBtn) _sunBtn.style.background = _weekStartSun ? _activeStyle : _inactiveStyle;
+  };
+  _monBtn?.addEventListener('click', () => {
+    _weekStartSun = false;
+    localStorage.setItem('cal-week-start', 'mon');
+    _applyWeekStartActive();
+    if (_open) _render();
+  });
+  _sunBtn?.addEventListener('click', () => {
+    _weekStartSun = true;
+    localStorage.setItem('cal-week-start', 'sun');
+    _applyWeekStartActive();
+    if (_open) _render();
+  });
+
   // Create a new (local) calendar. Defaults the name + next palette color, then
   // reopens the panel so the user can rename it inline and pick a color.
   overlay.querySelector('#cal-settings-add')?.addEventListener('click', async (e) => {
@@ -2918,35 +2949,68 @@ function _showEventForm(existing, defaultDate, defaultEndDate) {
     const startEl = document.getElementById('cal-f-start');
     const endEl = document.getElementById('cal-f-end');
     if (!startEl || !endEl) return;
+
     const _toMin = (v) => {
       if (!v || !/^\d{2}:\d{2}$/.test(v)) return null;
       const [h, m] = v.split(':').map(n => parseInt(n, 10));
       return h * 60 + m;
     };
+
     const _toHHMM = (mins) => {
       let m = ((mins % 1440) + 1440) % 1440;
       const hh = String(Math.floor(m / 60)).padStart(2, '0');
       const mm = String(m % 60).padStart(2, '0');
       return `${hh}:${mm}`;
     };
+
+    const _autoAdvanceEndDate = () => {
+      const isAD = document.getElementById('cal-f-allday')?.checked;
+      if (isAD) return;
+
+      const dv = document.getElementById('cal-f-date')?.value;
+      const dvEndEl = document.getElementById('cal-f-date-end');
+      if (!dv || !dvEndEl || dvEndEl.value !== dv) return;
+
+      const sVal = startEl.value;
+      const eVal = endEl.value;
+
+      if (sVal && eVal && eVal <= sVal) {
+        const d = new Date(`${dv}T00:00:00`);
+        d.setDate(d.getDate() + 1);
+
+        dvEndEl.value = _ds(d);
+      }
+    };
+
     let prevStartMin = _toMin(startEl.value);
-    endEl.addEventListener('input', () => { endEl.dataset.userEdited = '1'; });
+
+    endEl.addEventListener('input', () => {
+      endEl.dataset.userEdited = '1';
+    });
+
+    endEl.addEventListener('change', _autoAdvanceEndDate);
+
     startEl.addEventListener('change', () => {
       const newStartMin = _toMin(startEl.value);
       const endMin = _toMin(endEl.value);
-      if (newStartMin == null) { prevStartMin = newStartMin; return; }
-      // Compute the duration before the change. Use the user's existing
-      // start→end gap, fallback to 1 hour.
-      let durationMin = 60;
-      if (prevStartMin != null && endMin != null && endMin > prevStartMin) {
-        durationMin = endMin - prevStartMin;
-      } else if (endMin != null && newStartMin != null && endMin > newStartMin && endEl.dataset.userEdited === '1') {
-        // User already set a custom end before changing start — leave it.
+
+      if (newStartMin == null) {
         prevStartMin = newStartMin;
         return;
       }
+
+      let durationMin = 60;
+
+      if (prevStartMin != null && endMin != null && endMin > prevStartMin) {
+        durationMin = endMin - prevStartMin;
+      } else if (endMin != null && newStartMin != null && endMin > newStartMin && endEl.dataset.userEdited === '1') {
+        prevStartMin = newStartMin;
+        return;
+      }
+
       endEl.value = _toHHMM(newStartMin + durationMin);
       prevStartMin = newStartMin;
+      _autoAdvanceEndDate();
     });
   })();
   // Custom reminder picker
@@ -3007,6 +3071,20 @@ function _showEventForm(existing, defaultDate, defaultEndDate) {
     // proper UTC instants (is_utc=True). Without this, naive "10:00" gets
     // re-interpreted as local elsewhere — the timezone-misfire bug.
     const _tz = _tzOffset();
+    
+    if (!isAD) {
+      const startVal = document.getElementById('cal-f-start').value;
+      const endVal = document.getElementById('cal-f-end').value;
+
+      const startDt = new Date(`${dv}T${startVal}:00`);
+      const endDt = new Date(`${dvEnd}T${endVal}:00`);
+
+      if (endDt <= startDt) {
+        uiModule.showToast('End time must be after start time');
+        return;
+      }
+    }
+
     const payload = {
       summary,
       dtstart: isAD ? dv : `${dv}T${document.getElementById('cal-f-start').value}:00${_tz}`,
@@ -3215,6 +3293,37 @@ function _fmtTime(s) {
   }
   return s.slice(11, 16);
 }
+
+function _timeToMin(iso) {
+  const hm = _fmtTime(iso);
+  if (!hm) return null;
+  const m = hm.match(/^(\d{1,2}):(\d{2})$/);
+  if (!m) return null;
+  const h = parseInt(m[1], 10);
+  const min = parseInt(m[2], 10);
+  if (h < 0 || h > 23 || min < 0 || min > 59) return null;
+  return h * 60 + min;
+}
+
+function _tzOffsetForDate(d) {
+  const off = -d.getTimezoneOffset();
+  const sign = off >= 0 ? '+' : '-';
+  const abs = Math.abs(off);
+  const hh = String(Math.floor(abs / 60)).padStart(2, '0');
+  const mm = String(abs % 60).padStart(2, '0');
+  return `${sign}${hh}:${mm}`;
+}
+
+function _addMinutesToLocalIso(baseIso, addMinutes) {
+  const d = new Date(new Date(baseIso).getTime() + addMinutes * 60000);
+  const y = d.getFullYear();
+  const mo = String(d.getMonth() + 1).padStart(2, '0');
+  const da = String(d.getDate()).padStart(2, '0');
+  const h = String(d.getHours()).padStart(2, '0');
+  const m = String(d.getMinutes()).padStart(2, '0');
+  return `${y}-${mo}-${da}T${h}:${m}:00${_tzOffsetForDate(d)}`;
+}
+
 function _e(s) { return uiModule.esc ? uiModule.esc(s || '') : (s || '').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;'); }
 
 // Linkify a location string: URLs become clickable, plain addresses get a Maps link.

static/js/calendar/utils.js

@@ -3,7 +3,9 @@
 // Pure constants + zero-state helpers for the calendar UI.
 // No DOM, no fetch, no global mutable state — safe to import anywhere.
 
-export const WEEKDAYS = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'];
+export const WEEKDAYS     = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'];
+export const WEEKDAYS_SUN = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'];
+
 
 export const MONTHS = ['January', 'February', 'March', 'April', 'May', 'June',
   'July', 'August', 'September', 'October', 'November', 'December'];

static/js/chat.js

@@ -1564,9 +1564,12 @@ import { wireArrowUpRecall, getLastUserMessageFromChatHistory } from './composer
                       .replace(/<channel\|>/gi, '');
                     thinkText = thinkText.replace(/^\s*Thinking(?:\s+Process)?:\s*/i, '');
                     _liveThinkInner.innerHTML = markdownModule.mdToHtml(thinkText);
-                    // Keep thinking box scrolled to bottom
+                    // Keep thinking box scrolled to bottom, but let user scroll up
                     var thinkBox = _liveThinkInner.closest('.thinking-content');
-                    if (thinkBox) thinkBox.scrollTop = thinkBox.scrollHeight;
+                    if (thinkBox) {
+                      var nearBottom = thinkBox.scrollHeight - thinkBox.clientHeight - thinkBox.scrollTop < 80;
+                      if (nearBottom) thinkBox.scrollTop = thinkBox.scrollHeight;
+                    }
                   }
                   uiModule.scrollHistory();
                   continue;
@@ -3865,7 +3868,9 @@ import { wireArrowUpRecall, getLastUserMessageFromChatHistory } from './composer
 
     // Also submit on Enter (without shift)
     editor.addEventListener('keydown', (e) => {
-      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
+      const isMobile = window.innerWidth <= 768
+
+      if (e.key === 'Enter' && !e.shiftKey && !e.isComposing && !isMobile) {
         e.preventDefault();
         saveBtn.click();
       }
@@ -3873,9 +3878,11 @@ import { wireArrowUpRecall, getLastUserMessageFromChatHistory } from './composer
   }
 
   /**
-   * Resend a user message — truncates history to that point and resubmits.
+   * Resend a user message. Normal resend appends a fresh copy at the end of
+   * the current thread; regenerate flows can opt into replacing from here.
    */
-  export async function resendUserMessage(userMsgElement) {
+  export async function resendUserMessage(userMsgElement, opts = {}) {
+    const replaceFromHere = Boolean(opts && opts.replaceFromHere);
     const box = document.getElementById('chat-history');
     const allMsgs = Array.from(box.querySelectorAll('.msg'));
     const msgIndex = allMsgs.indexOf(userMsgElement);
@@ -3921,25 +3928,28 @@ import { wireArrowUpRecall, getLastUserMessageFromChatHistory } from './composer
     const sessionId = sessionModule.getCurrentSessionId();
     if (!sessionId) return;
 
-    // Truncate backend to keep everything before this user message
-    const keepCount = msgIndex;
     try {
-      await fetch(`${API_BASE}/api/session/${sessionId}/truncate`, {
-        method: 'POST',
-        headers: { 'Content-Type': 'application/json' },
-        body: JSON.stringify({ keep_count: keepCount })
-      });
+      if (replaceFromHere) {
+        // Regenerate flows intentionally trim history to this point before
+        // resubmitting. The plain "Resend message" action must not do this.
+        const keepCount = msgIndex;
+        await fetch(`${API_BASE}/api/session/${sessionId}/truncate`, {
+          method: 'POST',
+          headers: { 'Content-Type': 'application/json' },
+          body: JSON.stringify({ keep_count: keepCount })
+        });
 
-      // Drop the AI replies after the user message but KEEP the user bubble
-      // itself (so its photo stays visible). Then suppress the new user
-      // bubble that send would otherwise add — same pattern as regenerate.
-      let sibling = userMsgElement.nextSibling;
-      while (sibling) {
-        const next = sibling.nextSibling;
-        sibling.remove();
-        sibling = next;
+        // Drop the AI replies after the user message but KEEP the user bubble
+        // itself (so its photo stays visible). Then suppress the new user
+        // bubble that send would otherwise add — same pattern as regenerate.
+        let sibling = userMsgElement.nextSibling;
+        while (sibling) {
+          const next = sibling.nextSibling;
+          sibling.remove();
+          sibling = next;
+        }
+        _hideUserBubble = true;
       }
-      _hideUserBubble = true;
       _pendingRegenAttachments = _ids;
 
       // Resubmit
@@ -4473,6 +4483,15 @@ import { wireArrowUpRecall, getLastUserMessageFromChatHistory } from './composer
    * Delete an AI message and its preceding user message from the conversation.
    */
   export async function deleteMessage(msgElement) {
+    if (uiModule && uiModule.styledConfirm) {
+      const ok = await uiModule.styledConfirm('Delete this message?', {
+        confirmText: 'Delete',
+        cancelText: 'Cancel',
+        danger: true,
+      });
+      if (!ok) return;
+    }
+
     const box = document.getElementById('chat-history');
     const allMsgs = Array.from(box.querySelectorAll('.msg'));
     const clickedIndex = allMsgs.indexOf(msgElement);

static/js/chatRenderer.js

@@ -362,7 +362,7 @@ function _openVisionEditor(att, userMsgEl) {
       await _saveVisionText();
       _closeVisionEditor();
       if (userMsgEl && window.chatModule?.resendUserMessage) {
-        window.chatModule.resendUserMessage(userMsgEl);
+        window.chatModule.resendUserMessage(userMsgEl, { replaceFromHere: true });
       } else if (uiModule?.showToast) {
         uiModule.showToast('Saved');
       }

static/js/cookbook-diagnosis.js

@@ -320,6 +320,15 @@ export const ERROR_PATTERNS = [
       }},
     ],
   },
+  {
+    pattern: /sgl_kernel[\s\S]*(Python\.h|libnuma\.so\.1|common_ops)|(Python\.h|libnuma\.so\.1|common_ops)[\s\S]*sgl_kernel|Please ensure sgl_kernel is properly installed/i,
+    message: 'SGLang native dependencies are missing on this server.',
+    fixes: [
+      { label: 'Copy OS package command', action: () => _copyText('sudo apt-get install -y libnuma-dev python3.12-dev build-essential') },
+      { label: 'Copy kernel upgrade', action: () => _copyText('python3 -m pip install --upgrade sglang-kernel') },
+      { label: 'Open Dependencies', action: () => _openCookbookDependencies('sglang') },
+    ],
+  },
   {
     pattern: /sglang.*command not found|No module named sglang|SGLang is not installed/i,
     message: 'SGLang is not installed or not in PATH.',

static/js/cookbook.js

@@ -597,7 +597,8 @@ export function _buildServeCmd(f, modelName, backend) {
   } else if (backend === 'diffusers') {
     const gpuStr = f.gpus?.trim();
     if (gpuStr) cmd += `CUDA_VISIBLE_DEVICES=${gpuStr} `;
-    cmd += `python3 scripts/diffusion_server.py --model ${modelName} --port ${f.port || '8100'}`;
+    const diffusersPy = _isWindows() ? 'python' : _py3Bin;
+    cmd += `${diffusersPy} scripts/diffusion_server.py --model ${modelName} --port ${f.port || '8100'}`;
     if (f.diff_dtype && f.diff_dtype !== 'bfloat16') cmd += ` --dtype ${f.diff_dtype}`;
     if (f.diff_device_map && f.diff_device_map !== 'balanced') cmd += ` --device-map ${f.diff_device_map}`;
     if (f.diff_steps) cmd += ` --steps ${f.diff_steps}`;
@@ -718,7 +719,7 @@ async function _fetchDependencies() {
     const data = await resp.json();
     const pkgs = data.packages || [];
     if (!pkgs.length) { list.innerHTML = '<div class="hwfit-loading">No packages found</div>'; return; }
-    const _winUnsupported = new Set(['diffusers', 'hf_transfer', 'vllm', 'rembg', 'gfpgan']);
+    const _winUnsupported = new Set(['hf_transfer', 'vllm', 'rembg', 'gfpgan']);
 
     const _statusTag = (pkg, isLocal, isSystemDep, winBlocked) => {
       if (winBlocked) return `<span class="cookbook-dep-tag cookbook-dep-na">N/A</span>`;

static/js/cookbookRunning.js

@@ -793,9 +793,10 @@ function _winSessionCmd(task, tmuxArgs) {
     return host ? `ssh ${pf}${host} "powershell -Command \\"${ps}\\""` : `powershell -Command "${ps}"`;
   }
   if (tmuxArgs.includes('kill-session')) {
+    const stopTree = `function Stop-Tree([int]$Id) { Get-CimInstance Win32_Process -Filter "ParentProcessId = $Id" -ErrorAction SilentlyContinue | ForEach-Object { Stop-Tree ([int]$_.ProcessId) }; Stop-Process -Id $Id -Force -ErrorAction SilentlyContinue }`;
     const ps = host
-      ? `$p = Get-Content '${sd}\\${sid}.pid' -ErrorAction SilentlyContinue; if ($p) { Stop-Process -Id $p -Force -ErrorAction SilentlyContinue }; Remove-Item '${sd}\\${sid}.*' -Force -ErrorAction SilentlyContinue`
-      : `$p = Get-Content (Join-Path $env:TEMP 'odysseus-tmux\\${sid}.pid') -ErrorAction SilentlyContinue; if ($p) { Stop-Process -Id $p -Force -ErrorAction SilentlyContinue }; Remove-Item (Join-Path $env:TEMP 'odysseus-tmux\\${sid}.*') -Force -ErrorAction SilentlyContinue`;
+      ? `${stopTree}; $p = Get-Content '${sd}\\${sid}.pid' -ErrorAction SilentlyContinue; if ($p -match '^\\d+$') { Stop-Tree ([int]$p) }; Remove-Item '${sd}\\${sid}.*' -Force -ErrorAction SilentlyContinue`
+      : `${stopTree}; $p = Get-Content (Join-Path $env:TEMP 'odysseus-tmux\\${sid}.pid') -ErrorAction SilentlyContinue; if ($p -match '^\\d+$') { Stop-Tree ([int]$p) }; Remove-Item (Join-Path $env:TEMP 'odysseus-tmux\\${sid}.*') -Force -ErrorAction SilentlyContinue`;
     return host ? `ssh ${pf}${host} "powershell -Command \\"${ps}\\""` : `powershell -Command "${ps}"`;
   }
   if (tmuxArgs.includes('send-keys') && tmuxArgs.includes('C-c')) {
@@ -3532,12 +3533,22 @@ async function _pollBackgroundStatus() {
         // dead-session check inspects). Recover "done" from the retained output's
         // exit-0 sentinel so a clean install isn't downgraded to crashed.
         const depDone = !!task.payload?._dep && _depInstallSucceeded(task.output);
+        // A finished model download whose tmux pane is gone is also reported
+        // "stopped" (the dead-session check can miss the landed snapshot).
+        // Recover "done" from the terminal `DOWNLOAD_OK` sentinel — emitted
+        // only after the runner exits 0 — so a completed download isn't
+        // downgraded to crashed. This background poll runs blind (no live
+        // stream to debounce against), so unlike the reconnect loop it keys
+        // off the conclusive exit sentinel only, never the `/snapshots/` path,
+        // which can be printed mid-stream for multi-file downloads.
+        const downloadDone = task.type === 'download'
+          && String(task.output || '').includes('DOWNLOAD_OK');
         const nextStatus = live.status === 'completed'
           ? 'done'
           : (live.status === 'error'
             ? 'error'
             : (live.status === 'stopped'
-                ? (depDone ? 'done' : (task.type === 'download' ? 'crashed' : 'stopped'))
+                ? ((depDone || downloadDone) ? 'done' : (task.type === 'download' ? 'crashed' : 'stopped'))
                 : null));
         if (nextStatus && task.status !== nextStatus) {
           updates.status = nextStatus;

static/js/cookbookServe.js

@@ -530,7 +530,7 @@ function _rerenderCachedModels() {
         : (_lastUsed || (_isLegacyFlat ? _allSs : {}));
       const detectedBackend = _detectBackend(m).backend;
       const _allowedBackends = new Set(_isWindows()
-        ? ['llamacpp']
+        ? ['llamacpp', 'diffusers']
         : (_isMetal() ? ['llamacpp', 'ollama'] : ['vllm', 'sglang', 'llamacpp', 'ollama', 'diffusers']));
       const defaultBackend = (ss._forceBackend && ss.backend && _allowedBackends.has(ss.backend))
         ? ss.backend
@@ -590,7 +590,7 @@ function _rerenderCachedModels() {
       // Row 1: Backend + Server + Env
       panelHtml += `<div class="hwfit-serve-row">`;
       const _backendChoices = _isWindows()
-        ? [['llamacpp','llama.cpp']]
+        ? [['llamacpp','llama.cpp'],['diffusers','Diffusers']]
         : _isMetal()
         // Diffusers (diffusion_server.py) is CUDA-only — omit it on Metal.
         ? [['llamacpp','llama.cpp'],['ollama','Ollama']]

static/js/modalSnap.js

@@ -994,7 +994,7 @@ export function makeEdgeDockController(modal, side = 'right', dockClass) {
   stripe.style.bottom = '0';
   stripe.style.width = '10px';
   stripe.style.cursor = 'col-resize';
-  stripe.style.zIndex = '9999';
+  stripe.style.zIndex = '261';
   stripe.style.background = 'linear-gradient(to right, transparent 0 3px, color-mix(in srgb, var(--accent, var(--red)) 35%, transparent) 3px 7px, transparent 7px 10px)';
   stripe.style.pointerEvents = 'auto';
   stripe.style.touchAction = 'none';

static/js/research/panel.js

@@ -354,7 +354,7 @@ function _buildPanelHTML() {
           <span>Multi-step web research with an LLM-in-the-loop agent</span>
         </p>
         <div id="research-no-past-hint" class="memory-desc doclib-desc" style="display:none;margin-top:-2px;font-size:11px;opacity:0.7;">All past research found in <button type="button" class="research-library-link">Library, Research</button></div>
-        <textarea id="research-query" class="research-query" placeholder="e.g. Trace Odysseus's ten-year journey home from Troy — every island, monster, and detour, and why each one cost him" rows="4"></textarea>
+        <textarea id="research-query" class="research-query" placeholder="e.g. Trace Odysseus's ten-year journey home from Troy — every island, monster, and detour, and what each one cost him." rows="4"></textarea>
         <div class="research-category-row" id="research-category-row">
           <button class="research-cat active" data-cat="" title="LLM auto-detects the best format">Auto</button>
           <button class="research-cat" data-cat="product">Product</button>

static/js/settings.js

@@ -3644,7 +3644,11 @@ async function initUnifiedIntegrations() {
     el('uf-api-cancel').addEventListener('click', () => { formEl.style.display = 'none'; });
     el('uf-api-save').addEventListener('click', async () => {
       const presetKey = preset.value || undefined;
-      const body = { name: name.value, base_url: url.value, auth_type: auth.value, auth_header: header.value, preset: presetKey };
+      const nameValue = name.value.trim();
+      const urlValue = url.value.trim();
+      if (!nameValue) { el('uf-api-msg').textContent = 'Name required'; el('uf-api-msg').style.color = 'var(--red)'; return; }
+      if (!urlValue) { el('uf-api-msg').textContent = 'Base URL required'; el('uf-api-msg').style.color = 'var(--red)'; return; }
+      const body = { name: nameValue, base_url: urlValue, auth_type: auth.value, auth_header: header.value, preset: presetKey };
       if (key.value) body.api_key = key.value;
       try {
         const u = _editId ? `/api/auth/integrations/${_editId}` : '/api/auth/integrations';

static/js/slashCommands.js

@@ -339,10 +339,13 @@ function _submitComposedMessage(text) {
   const msgInput = document.getElementById('message');
   const form = document.getElementById('chat-form');
   if (!msgInput || !form) return false;
-  msgInput.value = text;
-  msgInput.dispatchEvent(new Event('input', { bubbles: true }));
-  if (typeof form.requestSubmit === 'function') form.requestSubmit();
-  else form.dispatchEvent(new Event('submit', { cancelable: true, bubbles: true }));
+  // The slash handler and app-level form debounce must both release before
+  // sending the pinned prompt, otherwise the follow-up submit is dropped.
+  setTimeout(() => {
+    msgInput.value = text;
+    msgInput.dispatchEvent(new Event('input', { bubbles: true }));
+    form.dispatchEvent(new Event('submit', { cancelable: true, bubbles: true }));
+  }, 350);
   return true;
 }
 

static/style.css

@@ -4649,7 +4649,7 @@ body.bg-pattern-sparkles {
       #email-lib-modal .email-reader-actions .memory-toolbar-btn.reader-icon-btn,
       .email-reader-tab-modal .email-reader-actions .memory-toolbar-btn.reader-icon-btn,
       .email-window-modal .email-reader-actions .memory-toolbar-btn.reader-icon-btn {
-        width: 44px !important;
+        width: auto !important;
         height: 44px !important;
         flex: 0 0 auto !important;
         display: inline-flex !important;
@@ -15244,6 +15244,10 @@ body.right-dock-active:not(.email-doc-split-active) .doc-editor-pane {
   overflow-y: auto !important;
   overscroll-behavior: contain;
 }
+.cookbook-group[data-backend-group="Serve"] > .admin-card > .hwfit-cached-list .doclib-card.doclib-card-expanded {
+  flex: 0 0 auto !important;
+  overflow: visible !important;
+}
 /* Drag-and-drop visual hint for the email compose pane. Subtle accent
    outline + tinted overlay so it's obvious files will attach if dropped. */
 .doc-editor-pane.email-dragover {
@@ -29147,9 +29151,9 @@ body.doc-find-active mark.doc-find-mark.current {
 
 /* Email reader icon buttons — vertical icon + label stack. */
 .memory-toolbar-btn.reader-icon-btn {
-  width: 48px;
+  width: auto;
   height: 44px;
-  padding: 4px 2px;
+  /* padding: 4px 2px; */
   position: relative;
   top: 1px;
   display: inline-flex;

tests/test_action_intents.py

@@ -49,6 +49,13 @@ def test_research_action_promotes_to_agent():
     assert message_needs_tools("can you look into GPU hosting options")
 
 
+def test_explicit_web_search_promotes_to_agent():
+    assert message_needs_tools("use web search and find a recipe for chocolate chip cookies")
+    assert message_needs_tools("do a web search for the best chocolate chip cookies")
+    assert message_needs_tools("search the web for current RTX 3090 prices")
+    assert classify_tool_intent("use web search and find a recipe").category == "web"
+
+
 def test_explanatory_calendar_questions_stay_plain_chat():
     assert not message_needs_tools("How do I add an entry to my calendar?")
     assert not message_needs_tools("What about the built-in Odysseus calendar, is that linked to email?")

tests/test_agent_loop.py

@@ -36,6 +36,7 @@ _IMPORTED_AGENT_LOOP = None
 try:
     from src.agent_loop import (
         _detect_admin_intent,
+        _classify_agent_request,
         _compute_final_metrics,
         _append_tool_results,
         _MCP_KEYWORDS,
@@ -62,6 +63,16 @@ def test_mcp_keyword_gate_matches_literal_mcp_requests():
     assert "mcp" in _MCP_KEYWORDS
 
 
+def test_polish_internet_search_request_classifies_as_web():
+    intent = _classify_agent_request(
+        [],
+        "Wyszukaj w internecie i podaj temperaturę w Lubartowie dzisiaj",
+    )
+
+    assert intent["low_signal"] is False
+    assert "web" in intent["domains"]
+
+
 # ---------------------------------------------------------------------------
 # _detect_admin_intent
 # ---------------------------------------------------------------------------

tests/test_api_key_file_permissions.py

@@ -0,0 +1,51 @@
+"""Regression: the API-key encryption key file (data/.key) must be owner-only
+(0o600).
+
+``APIKeyManager.get_or_create_key`` writes the Fernet key that decrypts *every*
+stored provider credential. Older versions created it with the process umask
+(commonly 0o644 — group/world-readable). It must be locked to the owner, both
+when freshly created and when an older, too-permissive key is read back.
+
+POSIX-only: ``core.platform_compat.safe_chmod`` is a documented no-op on Windows
+(files under the user profile are ACL-restricted), so the mode assertions are
+skipped there.
+"""
+import os
+import stat
+import sys
+
+import pytest
+
+from src.api_key_manager import APIKeyManager
+
+_WINDOWS = sys.platform.startswith("win")
+
+
+def _mode(path: str) -> int:
+    return stat.S_IMODE(os.stat(path).st_mode)
+
+
+@pytest.mark.skipif(_WINDOWS, reason="POSIX permission bits only")
+def test_new_key_file_is_owner_only(tmp_path):
+    mgr = APIKeyManager(str(tmp_path))
+    mgr.get_or_create_key()
+    assert _mode(mgr.key_file) == 0o600, f"expected 0o600, got {oct(_mode(mgr.key_file))}"
+
+
+@pytest.mark.skipif(_WINDOWS, reason="POSIX permission bits only")
+def test_existing_world_readable_key_is_relocked(tmp_path):
+    mgr = APIKeyManager(str(tmp_path))
+    # Simulate a key written by an older version with a permissive umask.
+    with open(mgr.key_file, "wb") as f:
+        f.write(b"x" * 44)
+    os.chmod(mgr.key_file, 0o644)
+    mgr.get_or_create_key()  # existing-file branch should re-lock it
+    assert _mode(mgr.key_file) == 0o600, f"expected re-lock to 0o600, got {oct(_mode(mgr.key_file))}"
+
+
+def test_encrypt_decrypt_roundtrip_still_works(tmp_path):
+    # The permission hardening must not change functional behaviour.
+    mgr = APIKeyManager(str(tmp_path))
+    enc = mgr.encrypt_api_key("sk-secret")
+    assert enc and enc != "sk-secret"
+    assert mgr.decrypt_api_key(enc) == "sk-secret"

tests/test_budget_auto_sentinel.py

@@ -0,0 +1,111 @@
+"""Agent input-token budget contract (review on #4122).
+
+- The DEFAULT value is the AUTO sentinel: it scales to the model's context window.
+  Any non-default value is an explicit cap. A materialized default 6000 can't be
+  told apart from a deliberate 6000 (the settings-save path persists defaults), so
+  the default reads as auto — pin a cap with a nearby value (e.g. 5999).
+- Auto-scaling only trusts a DISCOVERED context window; a bare DEFAULT_CONTEXT
+  fallback stays conservative instead of scaling off an unproven window.
+"""
+
+import json
+from unittest.mock import patch
+
+import src.settings as settings
+import src.model_context as mc
+from src.context_budget import compute_input_token_budget, DEFAULT_BUDGET, budget_is_explicit
+
+
+def test_default_value_is_the_auto_sentinel():
+    # The settings default equals DEFAULT_BUDGET, so the agent loop (which compares
+    # the configured value to DEFAULT_BUDGET) treats the default as "auto".
+    assert settings.DEFAULT_SETTINGS["agent_input_token_budget"] == DEFAULT_BUDGET
+
+
+def test_saving_an_unrelated_setting_does_not_re_cap_the_budget(tmp_path, monkeypatch):
+    """End-to-end regression (WGlynn, #4121): changing ANY setting makes the
+    settings-save path persist the merged dict, which materializes the budget
+    default into settings.json. The budget must still AUTO-SCALE — it must not be
+    re-read as an explicit 6000 cap. This locks the exact reopening shut.
+    """
+    settings_file = tmp_path / "settings.json"
+    monkeypatch.setattr(settings, "SETTINGS_FILE", str(settings_file))
+    settings._settings_cache = None
+
+    # Simulate a real settings save: a handler loads the merged dict (defaults +
+    # saved) and persists it after the user changes one *unrelated* setting.
+    merged = settings.load_settings()
+    merged["search_result_count"] = 9                  # unrelated user change
+    settings.save_settings(merged)
+    settings._settings_cache = None
+
+    # The budget default is now physically materialized into the file...
+    raw = json.loads(settings_file.read_text())
+    assert raw["agent_input_token_budget"] == DEFAULT_BUDGET
+    assert raw["search_result_count"] == 9
+
+    # ...yet it must read as AUTO (value == default), not an explicit cap — even
+    # though is_setting_overridden would report True for it now.
+    assert settings.is_setting_overridden("agent_input_token_budget") is True
+    soft = int(settings.get_setting("agent_input_token_budget", DEFAULT_BUDGET) or 0)
+    assert budget_is_explicit(soft) is False
+    # And the effective budget scales to the window rather than capping at 6000.
+    assert compute_input_token_budget(soft, 131072, explicit=budget_is_explicit(soft)) == int(131072 * 0.85)
+
+
+def test_auto_scales_on_a_known_window():
+    assert compute_input_token_budget(DEFAULT_BUDGET, 131072, explicit=False) == int(131072 * 0.85)
+
+
+def test_auto_stays_conservative_on_unknown_window():
+    # P2 #2: the budget block passes context_length=0 when the window is only a
+    # fallback, so auto-scaling must NOT inflate to the unproven window.
+    assert compute_input_token_budget(DEFAULT_BUDGET, 0, explicit=False) == DEFAULT_BUDGET
+
+
+def test_nondefault_value_is_an_explicit_cap():
+    assert compute_input_token_budget(20000, 131072, explicit=True) == 20000      # honoured
+    assert compute_input_token_budget(200000, 32000, explicit=True) == 32000      # clamped to window
+
+
+def test_get_context_length_known_surfaces_endpoint_proven_vs_fallback():
+    mc._context_cache.clear()
+    with patch.object(mc, "_query_context_length", return_value=(131072, True)):
+        assert mc.get_context_length_known("http://proven/v1", "m1") == (131072, True)
+    mc._context_cache.clear()
+    with patch.object(mc, "_query_context_length", return_value=(mc.DEFAULT_CONTEXT, False)):
+        ctx, known = mc.get_context_length_known("http://unknown/v1", "m2")
+        assert ctx == mc.DEFAULT_CONTEXT and known is False
+    # get_context_length keeps its plain-int contract for existing callers
+    mc._context_cache.clear()
+    with patch.object(mc, "_query_context_length", return_value=(64000, True)):
+        assert mc.get_context_length("http://proven/v1", "m3") == 64000
+
+
+def test_budget_context_binds_known_flag_to_its_own_value():
+    """Regression (RaresKeY, #4122): scale the budget off the value the `known`
+    flag actually proves — never a stale/missing context_length from a different
+    lookup. Covers the local-restaleness case (fresh proven value beats a stale
+    fallback) and the no-arg-caller case (discovers a long window despite fallback=0).
+    """
+    # unknown / bare fallback -> 0 (don't scale off an unproven window)
+    with patch.object(mc, "get_context_length_known", return_value=(128000, False)):
+        assert mc.budget_context_for_model("u", "m", fallback=128000) == 0
+    # known -> the freshly-proven value, NOT the (stale) fallback the caller passed
+    with patch.object(mc, "get_context_length_known", return_value=(4096, True)):
+        assert mc.budget_context_for_model("u", "m", fallback=128000) == 4096
+    # no-arg caller (fallback=0) still gets the discovered long window
+    with patch.object(mc, "get_context_length_known", return_value=(131072, True)):
+        assert mc.budget_context_for_model("u", "m", fallback=0) == 131072
+    # probe error -> caller's fallback (prior behaviour)
+    with patch.object(mc, "get_context_length_known", side_effect=RuntimeError):
+        assert mc.budget_context_for_model("u", "m", fallback=4096) == 4096
+
+
+def test_no_arg_caller_scales_from_discovered_window_not_6000():
+    """End-to-end of the fix: a caller that passes no context_length (scheduled
+    tasks, teacher escalation, ...) but whose endpoint reports 131072 now scales to
+    ~111k instead of being capped at the conservative 6000."""
+    with patch.object(mc, "get_context_length_known", return_value=(131072, True)):
+        ctx = mc.budget_context_for_model("u", "m", fallback=0)
+    assert compute_input_token_budget(DEFAULT_BUDGET, ctx, explicit=False) == int(131072 * 0.85)

tests/test_builtin_mcp_npx_cache.py

@@ -36,7 +36,38 @@ def test_npx_package_from_args_prefers_package_after_y_flag(monkeypatch):
     ) == "@playwright/mcp@latest"
 
 
-def test_npx_cache_check_falls_back_when_async_subprocess_is_unsupported(monkeypatch):
+def test_npx_cache_check_detects_scoped_package_in_npx_cache(monkeypatch, tmp_path):
+    builtin_mcp = _load_builtin_mcp(monkeypatch)
+    package_json = (
+        tmp_path
+        / ".npm"
+        / "_npx"
+        / "9833c18b2d85bc59"
+        / "node_modules"
+        / "@playwright"
+        / "mcp"
+        / "package.json"
+    )
+    package_json.parent.mkdir(parents=True)
+    package_json.write_text('{"name":"@playwright/mcp","version":"0.0.76"}', encoding="utf-8")
+
+    async def unexpected_exec(*args, **kwargs):
+        raise AssertionError("cache hit should not shell out to npx")
+
+    monkeypatch.setenv("HOME", str(tmp_path))
+    monkeypatch.delenv("npm_config_cache", raising=False)
+    monkeypatch.setattr(builtin_mcp.asyncio, "create_subprocess_exec", unexpected_exec)
+
+    assert asyncio.run(
+        builtin_mcp._is_npx_package_cached(
+            "npx",
+            "@playwright/mcp@latest",
+            timeout_s=2,
+        )
+    ) is True
+
+
+def test_npx_cache_check_falls_back_when_async_subprocess_is_unsupported(monkeypatch, tmp_path):
     builtin_mcp = _load_builtin_mcp(monkeypatch)
 
     async def unsupported_exec(*args, **kwargs):
@@ -51,6 +82,8 @@ def test_npx_cache_check_falls_back_when_async_subprocess_is_unsupported(monkeyp
 
     monkeypatch.setattr(builtin_mcp.asyncio, "create_subprocess_exec", unsupported_exec)
     monkeypatch.setattr(builtin_mcp.subprocess, "run", fake_run)
+    monkeypatch.setenv("HOME", str(tmp_path))
+    monkeypatch.delenv("npm_config_cache", raising=False)
 
     assert asyncio.run(
         builtin_mcp._is_npx_package_cached(
@@ -69,7 +102,7 @@ def test_npx_cache_check_falls_back_when_async_subprocess_is_unsupported(monkeyp
     assert captured["kwargs"]["timeout"] == 2
 
 
-def test_npx_cache_check_fallback_treats_timeout_as_cache_miss(monkeypatch):
+def test_npx_cache_check_fallback_treats_timeout_as_cache_miss(monkeypatch, tmp_path):
     builtin_mcp = _load_builtin_mcp(monkeypatch)
 
     async def unsupported_exec(*args, **kwargs):
@@ -80,6 +113,8 @@ def test_npx_cache_check_fallback_treats_timeout_as_cache_miss(monkeypatch):
 
     monkeypatch.setattr(builtin_mcp.asyncio, "create_subprocess_exec", unsupported_exec)
     monkeypatch.setattr(builtin_mcp.subprocess, "run", fake_run)
+    monkeypatch.setenv("HOME", str(tmp_path))
+    monkeypatch.delenv("npm_config_cache", raising=False)
 
     assert asyncio.run(
         builtin_mcp._is_npx_package_cached(

tests/test_chat_route_tool_policy.py

@@ -89,6 +89,9 @@ def test_disabled_tools_does_not_bash_when_allow_bash_is_none():
     assert "allow_web_search is not None" in source, (
         "disabled_tools check must guard against allow_web_search being None"
     )
+    assert "_explicit_web_intent" in source and "not _explicit_web_intent" in source, (
+        "explicit web-search requests must override an off web toggle for that turn"
+    )
 
 
 # ── Functional tests of the disabled-tools logic ───────────────
@@ -99,6 +102,7 @@ def _build_disabled_tools(
     allow_web_search=None,
     can_use_bash=True,
     can_use_browser=True,
+    explicit_web_intent=False,
 ):
     """Replicate the disabled-tools logic from chat_stream for unit testing.
 
@@ -109,7 +113,11 @@ def _build_disabled_tools(
     # Issue #3229 fix: only disable when explicitly set to a falsy value.
     if allow_bash is not None and str(allow_bash).lower() != "true":
         disabled_tools.add("bash")
-    if allow_web_search is not None and str(allow_web_search).lower() != "true":
+    if (
+        allow_web_search is not None
+        and str(allow_web_search).lower() != "true"
+        and not explicit_web_intent
+    ):
         disabled_tools.add("web_search")
         disabled_tools.add("web_fetch")
 
@@ -148,6 +156,17 @@ def test_json_body_allow_web_search_false_disables_web():
     assert "web_fetch" in disabled
 
 
+def test_explicit_web_intent_overrides_false_web_toggle_for_turn():
+    """A stale/off web toggle must not remove web tools when the message
+    explicitly asks to use web search."""
+    disabled = _build_disabled_tools(
+        allow_web_search="false",
+        explicit_web_intent=True,
+    )
+    assert "web_search" not in disabled
+    assert "web_fetch" not in disabled
+
+
 def test_admin_user_gets_bash_enabled_by_default():
     """When allow_bash is not set and user has can_use_bash privilege,
     bash must NOT be disabled.

tests/test_codex_ssh_host_validation.py

@@ -0,0 +1,49 @@
+"""The Codex cookbook bridge resolves a task's SSH target (remoteHost / sshPort)
+from cookbook_state.json and interpolates it into an ``ssh ...`` command string
+that runs through a shell. The command body is shlex-quoted, but the host and
+port were not validated, so a tampered task entry carrying shell metacharacters
+in ``remoteHost`` would be injected into that command.
+
+These pin validation on the host/port before they reach the ssh string, matching
+the validators the rest of the cookbook routes already apply.
+"""
+import pytest
+from fastapi import HTTPException
+
+import routes.codex_routes as codex_routes
+
+
+def test_rejects_remote_host_with_shell_metacharacters():
+    task = {"remoteHost": "box; rm -rf ~", "sshPort": ""}
+    with pytest.raises(HTTPException) as exc:
+        codex_routes._ssh_prefix_for_task(task)
+    assert exc.value.status_code == 400
+
+
+def test_rejects_non_numeric_ssh_port():
+    task = {"remoteHost": "box", "sshPort": "22; evil"}
+    with pytest.raises(HTTPException) as exc:
+        codex_routes._ssh_prefix_for_task(task)
+    assert exc.value.status_code == 400
+
+
+def test_local_task_has_no_host():
+    host, port_flag = codex_routes._ssh_prefix_for_task({})
+    assert host == ""
+    assert port_flag == ""
+
+
+def test_valid_remote_builds_port_flag():
+    host, port_flag = codex_routes._ssh_prefix_for_task(
+        {"remoteHost": "user@box", "sshPort": "2222"}
+    )
+    assert host == "user@box"
+    assert port_flag == "-p 2222 "
+
+
+def test_default_ssh_port_omits_flag():
+    host, port_flag = codex_routes._ssh_prefix_for_task(
+        {"remoteHost": "box", "sshPort": "22"}
+    )
+    assert host == "box"
+    assert port_flag == ""

tests/test_context_budget.py

@@ -47,11 +47,11 @@ def test_is_setting_overridden_reads_raw_saved_file(tmp_path, monkeypatch):
 
 
 # ---------------------------------------------------------------------------
-# Configurable hard_max — completes the reviewer requirement from #1190 that
-# was carried over but not implemented in #1230: the ceiling on the auto-
-# derived path should be a setting, not a hidden constant. Without this,
-# admins on premium APIs with very large windows (1M+ context) can only
-# raise the ceiling by editing src/context_budget.py.
+# Configurable hard_max — the ceiling on the auto-derived path is a setting
+# (`agent_input_token_hard_max`), not a hidden constant. History: a reviewer
+# required it on #1190, the merged #1230 shipped without it, and #1273 added it.
+# This test pins the function-level override (the `hard_max` parameter); without
+# a raisable ceiling, admins on 1M+ context APIs would be stuck at the 200K default.
 # ---------------------------------------------------------------------------
 
 def test_custom_hard_max_overrides_default_in_auto_branch():

tests/test_context_cache_per_endpoint.py

@@ -13,7 +13,7 @@ def _setup(monkeypatch, windows):
     """windows: {endpoint_url: context_length}. Force the remote path."""
     monkeypatch.setattr(mc, "is_local_endpoint", lambda url: False)
     monkeypatch.setattr(mc, "_configured_endpoint_kind", lambda url: "api")
-    monkeypatch.setattr(mc, "_query_context_length", lambda url, model: windows[url])
+    monkeypatch.setattr(mc, "_query_context_length", lambda url, model: (windows[url], True))
     mc._context_cache.clear()
 
 
@@ -34,6 +34,6 @@ def test_cache_hit_still_works_per_endpoint(monkeypatch):
 
     # Both endpoints are now cached under their own key; flip the underlying
     # query to prove subsequent reads come from the per-endpoint cache, not a re-query.
-    monkeypatch.setattr(mc, "_query_context_length", lambda url, model: 999)
+    monkeypatch.setattr(mc, "_query_context_length", lambda url, model: (999, True))
     assert mc.get_context_length(a, "shared-model") == 8000
     assert mc.get_context_length(b, "shared-model") == 200000

tests/test_cookbook_cpu_only_serve.py

@@ -15,6 +15,7 @@ import re
 from pathlib import Path
 
 SRC = Path(__file__).resolve().parent.parent / "static/js/cookbook.js"
+SERVE_SRC = Path(__file__).resolve().parent.parent / "static/js/cookbookServe.js"
 
 
 def test_cpu_only_drops_gpu_only_flags():
@@ -28,3 +29,25 @@ def test_cpu_only_drops_gpu_only_flags():
     # The CUDA unified-memory env must be suppressed for CPU-only too.
     assert "f.unified_mem && !_cpuOnly" in text, \
         "GGML_CUDA_ENABLE_UNIFIED_MEMORY must be gated on !_cpuOnly"
+
+
+def test_diffusers_is_not_blocked_on_windows_dependencies_panel():
+    text = SRC.read_text(encoding="utf-8")
+
+    assert "const _winUnsupported = new Set(['hf_transfer', 'vllm', 'rembg', 'gfpgan']);" in text
+    assert "new Set(['diffusers'" not in text
+
+
+def test_diffusers_is_available_on_windows_serve_panel():
+    text = SERVE_SRC.read_text(encoding="utf-8")
+
+    assert "? ['llamacpp', 'diffusers']" in text
+    assert "? [['llamacpp','llama.cpp'],['diffusers','Diffusers']]" in text
+
+
+def test_windows_diffusers_uses_python_not_python3():
+    text = SRC.read_text(encoding="utf-8")
+
+    assert "const diffusersPy = _isWindows() ? 'python' : _py3Bin;" in text
+    assert "cmd += `${diffusersPy} scripts/diffusion_server.py" in text
+    assert "cmd += `python3 scripts/diffusion_server.py" not in text

tests/test_cookbook_dead_download_status.py

@@ -0,0 +1,124 @@
+"""Behavioral guards for dead-session download classification (issue #4017).
+
+A download whose tmux pane is gone must not be reported as stopped when its
+retained output carries DOWNLOAD_OK, or when the files landed in a custom
+download dir. The runner exports HF_HOME=<local_dir>, so the cache lives
+under <local_dir>/hub — the probe only finds it if the task's dir is passed
+in explicitly rather than read from the probe process's environment.
+"""
+import os
+import subprocess
+import sys
+
+from routes.cookbook_output import (
+    classify_dead_download,
+    HF_CACHE_COMPLETE_PROBE,
+    HF_CACHE_INCOMPLETE_PROBE,
+)
+
+REPO = "org/some-model-GGUF"
+
+
+# ── Marker classification ──
+
+
+def test_download_ok_resolves_completed():
+    snap = "Fetching 4 files: 100%|####| 4/4\nDownload complete\n\nDOWNLOAD_OK\n$"
+    assert classify_dead_download(snap) == ("completed", False)
+
+
+def test_download_failed_resolves_error():
+    snap = "some progress\n\nDOWNLOAD_FAILED (exit 1 after 3 attempts)"
+    assert classify_dead_download(snap) == ("error", False)
+
+
+def test_download_ok_with_zero_files_resolves_error():
+    # A DOWNLOAD_OK from a run that matched no files (bad include/quant
+    # pattern) is still a failure — same guard as the live-session branch.
+    snap = "Fetching 0 files: 0it [00:00, ?it/s]\n\nDOWNLOAD_OK"
+    assert classify_dead_download(snap) == ("error", True)
+
+
+def test_no_marker_returns_none():
+    # Mid-download tail with no terminal marker — caller must fall back to
+    # the cache probe.
+    assert classify_dead_download("Downloading model.gguf:  42%") is None
+    assert classify_dead_download("") is None
+
+
+def test_ollama_pull_output_resolves_completed():
+    snap = "pulling manifest\npulling 8f39d1c3...: 100%\nsuccess\n\nDOWNLOAD_OK"
+    assert classify_dead_download(snap) == ("completed", False)
+
+
+# ── Cache probe scripts ──
+
+
+def _make_cache(root, repo=REPO, incomplete=False, empty_snapshot=False):
+    d = os.path.join(root, "hub", "models--" + repo.replace("/", "--"))
+    snap = os.path.join(d, "snapshots", "abc123")
+    os.makedirs(snap)
+    if not empty_snapshot:
+        with open(os.path.join(snap, "model.gguf"), "w") as f:
+            f.write("x")
+    if incomplete:
+        blobs = os.path.join(d, "blobs")
+        os.makedirs(blobs)
+        with open(os.path.join(blobs, "deadbeef.incomplete"), "w") as f:
+            f.write("x")
+
+
+def _run_probe(probe, repo, cache_root, env=None):
+    # Strip the HF cache vars so the probe can't accidentally find a real
+    # cache on the machine running the tests.
+    full_env = {k: v for k, v in os.environ.items()
+                if k not in ("HF_HOME", "HUGGINGFACE_HUB_CACHE", "HF_HUB_CACHE")}
+    full_env.update(env or {})
+    return subprocess.run(
+        [sys.executable, "-c", probe, repo, cache_root],
+        env=full_env, capture_output=True, timeout=30,
+    ).returncode
+
+
+def test_complete_probe_finds_custom_dir_cache(tmp_path):
+    # Model materialized under <local_dir>/hub — found only via the explicit
+    # cache_root argument (issue #4017).
+    root = str(tmp_path)
+    _make_cache(root)
+    assert _run_probe(HF_CACHE_COMPLETE_PROBE, REPO, root) == 0
+
+
+def test_complete_probe_misses_without_cache_root(tmp_path):
+    # Same on-disk layout, but without the cache_root argument the probe
+    # falls back to the default cache and misses it.
+    _make_cache(str(tmp_path))
+    assert _run_probe(HF_CACHE_COMPLETE_PROBE, REPO, "") == 1
+
+
+def test_complete_probe_rejects_incomplete_blobs(tmp_path):
+    root = str(tmp_path)
+    _make_cache(root, incomplete=True)
+    assert _run_probe(HF_CACHE_COMPLETE_PROBE, REPO, root) == 1
+
+
+def test_complete_probe_rejects_empty_snapshot(tmp_path):
+    root = str(tmp_path)
+    _make_cache(root, empty_snapshot=True)
+    assert _run_probe(HF_CACHE_COMPLETE_PROBE, REPO, root) == 1
+
+
+def test_complete_probe_env_fallback_still_works(tmp_path):
+    # No custom dir on the task — the probe must keep honoring the standard
+    # HF env vars so default-cache downloads classify as before.
+    root = str(tmp_path)
+    _make_cache(root)
+    hub = os.path.join(root, "hub")
+    assert _run_probe(HF_CACHE_COMPLETE_PROBE, REPO, "", env={"HUGGINGFACE_HUB_CACHE": hub}) == 0
+
+
+def test_incomplete_probe_sees_custom_dir_partials(tmp_path):
+    root = str(tmp_path)
+    _make_cache(root, incomplete=True)
+    assert _run_probe(HF_CACHE_INCOMPLETE_PROBE, REPO, root) == 0
+    # Clean cache → no resumable partials.
+    assert _run_probe(HF_CACHE_INCOMPLETE_PROBE, "org/other-model", root) == 1

tests/test_cookbook_dependency_completion_regression.py

@@ -74,7 +74,23 @@ def test_background_poll_recovers_done_for_stopped_dependency_install():
     source = _read("static/js/cookbookRunning.js")
 
     assert "const depDone = !!task.payload?._dep && _depInstallSucceeded(task.output);" in source
-    assert "depDone ? 'done' : (task.type === 'download' ? 'crashed' : 'stopped')" in source
+    assert "(depDone || downloadDone) ? 'done' : (task.type === 'download' ? 'crashed' : 'stopped')" in source
+
+
+def test_background_poll_recovers_done_for_completed_download():
+    """When the backend reports a finished model download as "stopped" (its
+    tmux pane is gone after DOWNLOAD_OK, so the dead-session check can miss the
+    landed snapshot), the reconciler must recover "done" from the terminal
+    DOWNLOAD_OK sentinel instead of downgrading the card to crashed. The
+    background poll keys off DOWNLOAD_OK only (not the "/snapshots/" path, which
+    can appear mid-stream for multi-file downloads)."""
+    source = _read("static/js/cookbookRunning.js")
+
+    normalized = " ".join(source.split())
+    assert (
+        "const downloadDone = task.type === 'download' "
+        "&& String(task.output || '').includes('DOWNLOAD_OK');"
+    ) in normalized
 
 
 def test_dependency_install_payload_keeps_env_path_for_refresh():

tests/test_cookbook_diagnosis.py

@@ -13,3 +13,24 @@ def test_diagnose_vllm_modelopt_lm_head_error():
     assert "ModelOpt LM-head" in diagnosis["message"]
     assert diagnosis["suggestions"][0]["op"] == "manual"
     assert "provides this CLI" in diagnosis["suggestions"][0]["label"]
+
+
+def test_diagnose_sglang_native_dependency_errors():
+    output = """
+    /tmp/cuda_utils.c:7:10: fatal error: Python.h: No such file or directory
+    ImportError:
+    [sgl_kernel] CRITICAL: Could not load any common_ops library!
+    Please ensure sgl_kernel is properly installed with:
+    pip install --upgrade sglang-kernel
+    Error details from previous import attempts:
+    - ImportError: libnuma.so.1: cannot open shared object file
+    """
+
+    diagnosis = _diagnose_serve_output(output)
+
+    assert diagnosis is not None
+    assert "SGLang native dependencies" in diagnosis["message"]
+    labels = [suggestion["label"] for suggestion in diagnosis["suggestions"]]
+    assert any("libnuma-dev" in label for label in labels)
+    assert any("python3.12-dev" in label for label in labels)
+    assert any("sglang-kernel" in label for label in labels)

tests/test_cookbook_diagnosis_js.py

@@ -10,3 +10,13 @@ def test_repair_kernels_pip_spec_is_shell_quoted():
 
     assert '"kernels<0.15"' in source
     assert " --break-system-packages kernels<0.15" not in source
+
+
+def test_sglang_native_dependency_diagnosis_is_exposed_to_browser():
+    source = DIAGNOSIS_JS.read_text(encoding="utf-8")
+
+    assert r"Python\.h" in source
+    assert r"libnuma\.so\.1" in source
+    assert "SGLang native dependencies" in source
+    assert "libnuma-dev python3.12-dev build-essential" in source
+    assert "sglang-kernel" in source

tests/test_cookbook_helpers.py

@@ -2,6 +2,7 @@ import json
 import os
 import subprocess
 import sys
+from pathlib import Path
 
 import pytest
 from fastapi import HTTPException
@@ -21,6 +22,7 @@ from routes.cookbook_helpers import (
     _safe_env_prefix,
     _user_shell_path_bootstrap,
     _venv_safe_local_pip_install_cmd,
+    _normalize_llama_cpp_python_cache_types,
     _validate_gpus,
     _validate_local_dir,
     _validate_repo_id,
@@ -549,6 +551,35 @@ def test_validate_serve_cmd_accepts_windows_printf_format():
     assert _validate_serve_cmd(cmd) == cmd
 
 
+def test_normalize_llama_cpp_python_cache_types_for_stale_client_cmd():
+    cmd = (
+        "python -m llama_cpp.server --model model.gguf --host 0.0.0.0 --port 8000 "
+        "--type_k q4_0 --type_v q4_0"
+    )
+
+    assert _normalize_llama_cpp_python_cache_types(cmd).endswith("--type_k 2 --type_v 2")
+
+
+def test_normalize_llama_cpp_python_cache_types_preserves_native_cache_flags():
+    cmd = (
+        "llama-server --model model.gguf --cache-type-k q4_0 --cache-type-v q4_0 "
+        "|| python3 -m llama_cpp.server --model model.gguf --type_k=q8_0 --type_v='f16'"
+    )
+
+    normalized = _normalize_llama_cpp_python_cache_types(cmd)
+    assert "--cache-type-k q4_0 --cache-type-v q4_0" in normalized
+    assert "--type_k=8" in normalized
+    assert "--type_v='1'" in normalized
+
+
+def test_model_serve_normalizes_llama_cpp_python_cache_types_after_validation():
+    src = (Path(__file__).resolve().parents[1] / "routes" / "cookbook_routes.py").read_text(encoding="utf-8")
+
+    assert "req.cmd = _validate_serve_cmd(req.cmd) or \"\"" in src
+    assert "req.cmd = _normalize_llama_cpp_python_cache_types(req.cmd) or \"\"" in src
+    assert src.index("_validate_serve_cmd(req.cmd)") < src.index("_normalize_llama_cpp_python_cache_types(req.cmd)")
+
+
 def test_ollama_serve_defaults_to_loopback_bind():
     assert _ollama_bind_from_cmd("ollama serve") == ("127.0.0.1", "11434")
     assert _ollama_bind_from_cmd("ollama run qwen2.5:0.5b") == ("127.0.0.1", "11434")
@@ -588,6 +619,8 @@ def test_llama_cpp_linux_bootstrap_prefers_rocm_before_cuda():
     _append_llama_cpp_linux_accel_build_lines(runner_lines)
     script = "\n".join(runner_lines)
 
+    assert "mkdir -p ~/bin" in script
+    assert script.index("mkdir -p ~/bin") < script.index("cd ~/llama.cpp && rm -rf build")
     assert 'command -v hipconfig &>/dev/null || [ -d /opt/rocm ] || [ -n "$ROCM_PATH" ] || [ -n "$HIP_PATH" ]' in script
     assert 'cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON' in script
     assert 'cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON' in script
@@ -673,6 +706,16 @@ def test_llama_cpp_rebuild_cmd_clears_cached_build_paths():
     assert 'curl' not in cmd and 'wget' not in cmd
 
 
+def test_local_windows_download_pid_tracks_inner_bash_and_stop_kills_tree():
+    routes_src = (Path(__file__).resolve().parents[1] / "routes" / "cookbook_routes.py").read_text(encoding="utf-8")
+    running_src = (Path(__file__).resolve().parents[1] / "static" / "js" / "cookbookRunning.js").read_text(encoding="utf-8")
+
+    assert 'printf \'%s\\\\n\' \\"$$\\" > {pp}' in routes_src
+    assert "function Stop-Tree([int]$Id)" in running_src
+    assert "ParentProcessId = $Id" in running_src
+    assert "Stop-Tree ([int]$p)" in running_src
+
+
 def test_llama_cpp_rebuild_cmd_runs_clean_on_a_fresh_home(tmp_path):
     """The command should succeed even when neither path exists yet."""
     import os

tests/test_cookbook_package_detection.py

@@ -23,6 +23,7 @@ def test_llama_cpp_maps_to_llama_cpp_python_distribution():
 
 def test_extras_and_version_markers_are_stripped():
     assert _pip_dist_name({"name": "diffusers", "pip": "diffusers[torch]"}) == "diffusers"
+    assert _pip_dist_name({"name": "transformers", "pip": "transformers"}) == "transformers"
     assert _pip_dist_name({"name": "sglang", "pip": "sglang[all]"}) == "sglang"
     assert _pip_dist_name({"name": "rembg", "pip": "rembg[gpu]"}) == "rembg"
     assert _pip_dist_name({"name": "x", "pip": "foo>=1.2,<2"}) == "foo"
@@ -48,3 +49,11 @@ def test_route_uses_dist_name_helper_not_munged_import_name():
     src = (Path(__file__).resolve().parents[1] / "routes" / "shell_routes.py").read_text(encoding="utf-8")
     assert "importlib_metadata.version(_pip_dist_name(pkg))" in src
     assert 'importlib_metadata.version(pkg["name"].replace("_", "-"))' not in src
+
+
+def test_transformers_is_listed_as_image_dependency():
+    src = (Path(__file__).resolve().parents[1] / "routes" / "shell_routes.py").read_text(encoding="utf-8")
+
+    assert '"name": "transformers"' in src
+    assert '"pip": "transformers"' in src
+    assert '"transformers",' in src

tests/test_document_editor_scroll.py

@@ -12,8 +12,8 @@ from pathlib import Path
 
 
 ROOT = Path(__file__).resolve().parents[1]
-DOC_JS = (ROOT / "static/js/document.js").read_text()
-STYLE_CSS = (ROOT / "static/style.css").read_text()
+DOC_JS = (ROOT / "static/js/document.js").read_text(encoding="utf-8")
+STYLE_CSS = (ROOT / "static/style.css").read_text(encoding="utf-8")
 
 
 def test_document_textarea_scrollbar_is_visible():

tests/test_gallery_result_image_ssrf.py

@@ -0,0 +1,69 @@
+"""The gallery image-edit proxies (inpaint, harmonize) accept an upstream
+diffusion / OpenAI response that may carry an image *URL* instead of inline
+base64, and then fetch that URL server-side. That URL is controlled by whatever
+server the request was sent to, so a malicious or compromised endpoint can
+return e.g. ``http://169.254.169.254/...`` and turn the result fetch into an
+SSRF primitive (cloud-metadata credential exfil).
+
+The client-supplied ``_endpoint`` is already validated through
+``check_outbound_url`` before the first request; this pins the same guard on the
+*result* URL pulled from the response body, which previously went unchecked.
+"""
+import base64
+
+import pytest
+from fastapi import HTTPException
+
+import routes.gallery_routes as gallery_routes
+
+
+class _FakeResp:
+    def __init__(self, status_code: int, content: bytes = b""):
+        self.status_code = status_code
+        self.content = content
+
+
+class _FakeAsyncClient:
+    instances: list["_FakeAsyncClient"] = []
+
+    def __init__(self, *args, **kwargs):
+        self.gets: list[str] = []
+        _FakeAsyncClient.instances.append(self)
+
+    async def __aenter__(self):
+        return self
+
+    async def __aexit__(self, *exc):
+        return False
+
+    async def get(self, url, **kwargs):
+        self.gets.append(url)
+        return _FakeResp(200, b"PNGDATA")
+
+
+@pytest.fixture(autouse=True)
+def _fake_httpx(monkeypatch):
+    import httpx
+
+    _FakeAsyncClient.instances = []
+    monkeypatch.setattr(httpx, "AsyncClient", _FakeAsyncClient)
+
+
+async def test_rejects_link_local_result_url():
+    # A compromised upstream returns the cloud-metadata address as the image
+    # URL. The helper must refuse it and never issue the fetch.
+    with pytest.raises(HTTPException) as exc:
+        await gallery_routes._fetch_result_image_b64(
+            "http://169.254.169.254/latest/meta-data"
+        )
+    assert exc.value.status_code == 502
+    assert all(c.gets == [] for c in _FakeAsyncClient.instances), (
+        "the unsafe result URL must not be fetched"
+    )
+
+
+async def test_fetches_safe_result_url():
+    # A normal loopback/LAN diffusion server result URL is allowed (local-first)
+    # and returned base64-encoded, matching the prior inline behavior.
+    out = await gallery_routes._fetch_result_image_b64("http://127.0.0.1/img.png")
+    assert out == base64.b64encode(b"PNGDATA").decode()

tests/test_integrations_store_shape.py

@@ -1,4 +1,8 @@
 import json
+import asyncio
+from types import SimpleNamespace
+
+import pytest
 
 from src import integrations
 
@@ -9,3 +13,117 @@ def test_load_integrations_skips_non_object_rows(tmp_path, monkeypatch):
     monkeypatch.setattr(integrations, "DATA_FILE", str(data_file))
 
     assert integrations.load_integrations() == [{"id": "good", "name": "Good"}]
+
+
+@pytest.fixture
+def integrations_routes(tmp_path, monkeypatch):
+    fastapi = pytest.importorskip("fastapi")
+    from routes import auth_routes
+
+    monkeypatch.setattr(integrations, "DATA_FILE", str(tmp_path / "integrations.json"))
+    monkeypatch.setattr(auth_routes, "migrate_from_settings", lambda: None)
+
+    class _AuthManager:
+        def get_username_for_token(self, token):
+            return "admin" if token == "session-token" else None
+
+        def is_admin(self, user):
+            return user == "admin"
+
+    router = auth_routes.setup_auth_routes(_AuthManager())
+
+    def endpoint(path, method):
+        for route in router.routes:
+            if getattr(route, "path", "") == path and method in getattr(route, "methods", set()):
+                return route.endpoint
+        raise AssertionError(f"{method} {path} route not registered")
+
+    return endpoint, auth_routes.SESSION_COOKIE, fastapi.HTTPException
+
+
+class _JsonRequest(SimpleNamespace):
+    def __init__(self, body, session_cookie):
+        super().__init__(
+            cookies={session_cookie: "session-token"},
+            client=SimpleNamespace(host="127.0.0.1"),
+            _body=body,
+        )
+
+    async def json(self):
+        return self._body
+
+
+@pytest.mark.parametrize("blank_name", ["", "   "])
+def test_create_integration_rejects_blank_name_without_persisting(integrations_routes, blank_name):
+    endpoint, session_cookie, http_exception = integrations_routes
+    create_integration = endpoint("/api/auth/integrations", "POST")
+
+    with pytest.raises(http_exception) as exc:
+        asyncio.run(create_integration(
+            _JsonRequest({"name": blank_name, "base_url": "https://example.test"}, session_cookie)
+        ))
+
+    assert exc.value.status_code == 400
+    assert exc.value.detail == "Integration name is required"
+    assert integrations.load_integrations() == []
+
+
+@pytest.mark.parametrize("blank_base_url", ["", "   "])
+def test_create_integration_rejects_blank_base_url_without_persisting(integrations_routes, blank_base_url):
+    endpoint, session_cookie, http_exception = integrations_routes
+    create_integration = endpoint("/api/auth/integrations", "POST")
+
+    with pytest.raises(http_exception) as exc:
+        asyncio.run(create_integration(
+            _JsonRequest({"name": "Example", "base_url": blank_base_url}, session_cookie)
+        ))
+
+    assert exc.value.status_code == 400
+    assert exc.value.detail == "Integration base URL is required"
+    assert integrations.load_integrations() == []
+
+
+@pytest.mark.parametrize("blank_name", ["", "   "])
+def test_update_integration_rejects_blank_name_without_changing_existing(integrations_routes, blank_name):
+    endpoint, session_cookie, http_exception = integrations_routes
+    update_integration = endpoint("/api/auth/integrations/{integration_id}", "PUT")
+    integrations.save_integrations([
+        {
+            "id": "existing",
+            "name": "Original",
+            "base_url": "https://example.test",
+        }
+    ])
+
+    with pytest.raises(http_exception) as exc:
+        asyncio.run(update_integration(
+            integration_id="existing",
+            request=_JsonRequest({"name": blank_name}, session_cookie),
+        ))
+
+    assert exc.value.status_code == 400
+    assert exc.value.detail == "Integration name is required"
+    assert integrations.load_integrations()[0]["name"] == "Original"
+
+
+@pytest.mark.parametrize("blank_base_url", ["", "   "])
+def test_update_integration_rejects_blank_base_url_without_changing_existing(integrations_routes, blank_base_url):
+    endpoint, session_cookie, http_exception = integrations_routes
+    update_integration = endpoint("/api/auth/integrations/{integration_id}", "PUT")
+    integrations.save_integrations([
+        {
+            "id": "existing",
+            "name": "Original",
+            "base_url": "https://example.test",
+        }
+    ])
+
+    with pytest.raises(http_exception) as exc:
+        asyncio.run(update_integration(
+            integration_id="existing",
+            request=_JsonRequest({"base_url": blank_base_url}, session_cookie),
+        ))
+
+    assert exc.value.status_code == 400
+    assert exc.value.detail == "Integration base URL is required"
+    assert integrations.load_integrations()[0]["base_url"] == "https://example.test"

tests/test_llama_server_models_url.py

@@ -51,7 +51,7 @@ def test_model_context_queries_models_for_v1_base(monkeypatch):
 
     monkeypatch.setattr(model_context.httpx, "get", fake_get)
 
-    assert model_context._query_context_length("http://127.0.0.1:8080/v1", "qwen3") == 32768
+    assert model_context._query_context_length("http://127.0.0.1:8080/v1", "qwen3") == (32768, True)
     assert seen == [
         "http://127.0.0.1:8080/slots",
         "http://127.0.0.1:8080/v1/models",

tests/test_llm_core_connect_timeout.py

@@ -0,0 +1,57 @@
+"""Regression tests for the configurable LLM connect timeout.
+
+Background: chat uses the streaming path, which (unlike llm_call) does not retry
+a connect error -- it marks the host and emits a 503 immediately. With the old
+hard-coded connect=3.0s, a brief blip on the first (cold) connect of an idle
+chat to an offshore/public endpoint surfaced as an intermittent 503 that cleared
+on resend. The connect budget is now LLMConfig.CONNECT_TIMEOUT (env
+LLM_CONNECT_TIMEOUT), applied via _call_timeout/_stream_timeout helpers.
+"""
+import importlib
+import httpx
+import pytest
+
+from src import llm_core
+from src.llm_core import LLMConfig, _call_timeout, _stream_timeout
+
+
+def test_default_connect_timeout_is_widened_not_three():
+    # Regression guard: must not regress to the old too-tight 3.0s default.
+    assert LLMConfig.CONNECT_TIMEOUT >= 8.0
+    assert LLMConfig.CONNECT_TIMEOUT != 3.0
+    assert LLMConfig.CONNECT_TIMEOUT == 10.0
+
+
+def test_call_timeout_uses_config_connect_and_passes_read():
+    t = _call_timeout(45)
+    assert isinstance(t, httpx.Timeout)
+    assert t.connect == LLMConfig.CONNECT_TIMEOUT
+    assert t.read == 45.0
+    assert t.write == 10.0
+    assert t.pool == 5.0
+
+
+def test_stream_timeout_uses_config_connect_and_passes_read():
+    t = _stream_timeout(300)
+    assert isinstance(t, httpx.Timeout)
+    assert t.connect == LLMConfig.CONNECT_TIMEOUT
+    assert t.read == 300.0
+    assert t.write == 30.0
+    assert t.pool == 5.0
+
+
+def test_helpers_are_config_driven(monkeypatch):
+    # Helpers read LLMConfig at call time, so ops can tune without code edits.
+    monkeypatch.setattr(LLMConfig, "CONNECT_TIMEOUT", 4.5)
+    assert _call_timeout(30).connect == 4.5
+    assert _stream_timeout(30).connect == 4.5
+
+
+def test_env_override_is_honoured(monkeypatch):
+    monkeypatch.setenv("LLM_CONNECT_TIMEOUT", "6.5")
+    reloaded = importlib.reload(llm_core)
+    try:
+        assert reloaded.LLMConfig.CONNECT_TIMEOUT == 6.5
+    finally:
+        monkeypatch.delenv("LLM_CONNECT_TIMEOUT", raising=False)
+        importlib.reload(llm_core)  # restore module-level default for other tests

tests/test_lmstudio_models_url.py

@@ -0,0 +1,160 @@
+"""Regression coverage for LM Studio /v1 model-list endpoints (issue #25).
+
+LM Studio's OpenAI-compatible surface exposes its model list at
+``/v1/models`` (just like llama-server, vLLM, text-generation-webui). Two
+distinct failure modes were reported by users:
+
+1. Pasting ``http://localhost:1234`` (no ``/v1``) — ``build_models_url``
+   used to return ``http://localhost:1234/models``, which LM Studio does
+   not expose, so the user got a generic "No models found" error even
+   though the server was running and reachable.
+2. Pasting ``http://localhost:1234/v1`` (with ``/v1``) — the model list
+   fetch was correct, but the error message gave the user no way to tell
+   whether the URL was wrong, the server was down, or the server was
+   reachable but had no model loaded.
+
+This module pins both behaviors so future refactors don't regress them.
+"""
+
+import httpx
+
+from src import endpoint_resolver, llm_core
+
+
+def _neutralize_provider_detection(monkeypatch):
+    """``_is_ollama_native_url`` matches any localhost host with an empty
+    path, which would route ``http://localhost:1234`` (LM Studio) into the
+    Ollama branch and probe ``/api/tags`` instead of ``/v1/models``. Force
+    provider detection to "openai" so the URL builder takes the LM Studio
+    path the user actually intends."""
+    monkeypatch.setattr(llm_core, "_is_ollama_native_url", lambda url: False)
+
+
+# ── build_models_url: handle LM Studio base shapes ────────────────────
+
+
+def test_build_models_url_inserts_v1_for_bare_host_port(monkeypatch):
+    """`http://localhost:1234` must probe `/v1/models` for LM Studio."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    _neutralize_provider_detection(monkeypatch)
+
+    assert (
+        endpoint_resolver.build_models_url("http://localhost:1234")
+        == "http://localhost:1234/v1/models"
+    )
+
+
+def test_build_models_url_accepts_v1_base(monkeypatch):
+    """`http://localhost:1234/v1` must probe `/v1/models` (no double v1)."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    _neutralize_provider_detection(monkeypatch)
+
+    assert (
+        endpoint_resolver.build_models_url("http://localhost:1234/v1")
+        == "http://localhost:1234/v1/models"
+    )
+
+
+def test_build_models_url_idempotent_for_explicit_models(monkeypatch):
+    """`/v1/models` must probe `/v1/models` (normalize_base strips it)."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    _neutralize_provider_detection(monkeypatch)
+
+    assert (
+        endpoint_resolver.build_models_url("http://localhost:1234/v1/models")
+        == "http://localhost:1234/v1/models"
+    )
+
+
+def test_build_models_url_strips_chat_completions(monkeypatch):
+    """`/v1/chat/completions` must collapse to `/v1/models` (parity with #3330)."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    _neutralize_provider_detection(monkeypatch)
+
+    assert (
+        endpoint_resolver.build_models_url("http://localhost:1234/v1/chat/completions")
+        == "http://localhost:1234/v1/models"
+    )
+
+
+def test_build_models_url_preserves_explicit_non_v1_path(monkeypatch):
+    """User-supplied non-empty paths (e.g. `/openai`) must not be overridden
+    with `/v1`. We only insert `/v1` when the path is empty — that matches
+    the documented contract: a custom path is the caller's intent."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    _neutralize_provider_detection(monkeypatch)
+
+    assert (
+        endpoint_resolver.build_models_url("http://proxy.example.com/openai")
+        == "http://proxy.example.com/openai/models"
+    )
+
+
+# ── list_model_ids: parse LM Studio's response ─────────────────────────
+
+
+def test_llm_core_list_model_ids_queries_v1_models_for_lmstudio(monkeypatch):
+    """Issue #25: probing `http://localhost:1234/v1` must hit `/v1/models`."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    monkeypatch.setattr(llm_core, "_configured_cached_model_ids", lambda url, **kwargs: [])
+    seen = []
+
+    def fake_get(url, headers=None, timeout=None):
+        seen.append(url)
+        request = httpx.Request("GET", url)
+        return httpx.Response(
+            200,
+            json={
+                "object": "list",
+                "data": [
+                    {"id": "lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF"},
+                    {"id": "qwen2.5-7b-instruct"},
+                ],
+            },
+            request=request,
+        )
+
+    monkeypatch.setattr(llm_core.httpx, "get", fake_get)
+
+    assert llm_core.list_model_ids("http://localhost:1234/v1", timeout=1) == [
+        "lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF",
+        "qwen2.5-7b-instruct",
+    ]
+    assert seen == ["http://localhost:1234/v1/models"]
+
+
+def test_llm_core_list_model_ids_queries_v1_models_for_bare_lmstudio(monkeypatch):
+    """Issue #25: probing `http://localhost:1234` (no /v1) must hit `/v1/models`."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    monkeypatch.setattr(llm_core, "_configured_cached_model_ids", lambda url, **kwargs: [])
+    # Localhost with empty path would otherwise be misclassified as Ollama
+    # (llm_core._is_ollama_native_url); neutralise that for the test.
+    monkeypatch.setattr(llm_core, "_is_ollama_native_url", lambda url: False)
+    seen = []
+
+    def fake_get(url, headers=None, timeout=None):
+        seen.append(url)
+        request = httpx.Request("GET", url)
+        return httpx.Response(200, json={"data": [{"id": "model-a"}]}, request=request)
+
+    monkeypatch.setattr(llm_core.httpx, "get", fake_get)
+
+    assert llm_core.list_model_ids("http://localhost:1234", timeout=1) == ["model-a"]
+    assert seen == ["http://localhost:1234/v1/models"]
+
+
+def test_llm_core_list_model_ids_handles_empty_lmstudio_list(monkeypatch):
+    """LM Studio returns `{"object":"list","data":[]}` when no model is loaded.
+    The helper must return `[]` cleanly so the caller can surface a clear
+    error (issue #25: previously the empty case was indistinguishable from
+    a connection failure)."""
+    monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
+    monkeypatch.setattr(llm_core, "_configured_cached_model_ids", lambda url, **kwargs: [])
+
+    def fake_get(url, headers=None, timeout=None):
+        request = httpx.Request("GET", url)
+        return httpx.Response(200, json={"object": "list", "data": []}, request=request)
+
+    monkeypatch.setattr(llm_core.httpx, "get", fake_get)
+
+    assert llm_core.list_model_ids("http://localhost:1234/v1", timeout=1) == []

tests/test_model_context.py

@@ -192,7 +192,7 @@ class TestGetContextLength:
 
         def fake_query(endpoint_url, model):
             calls.append((endpoint_url, model))
-            return 8192 if len(calls) == 1 else 27000
+            return (8192, True) if len(calls) == 1 else (27000, True)
 
         monkeypatch.setattr(model_context, "_query_context_length", fake_query)
 
@@ -211,7 +211,7 @@ class TestGetContextLength:
 
         def fake_query(endpoint_url, model):
             calls.append((endpoint_url, model))
-            return 200000 if len(calls) == 1 else 12345
+            return (200000, True) if len(calls) == 1 else (12345, True)
 
         monkeypatch.setattr(model_context, "_query_context_length", fake_query)
 

tests/test_model_routes.py

@@ -625,7 +625,39 @@ def test_generic_endpoint_error_message_preserves_probe_error():
         {"error": "HTTP 401"},
     )
 
-    assert msg == "No models found for that provider/key. Last probe error: HTTP 401."
+    # Issue #25: the message must include the probed URL so the user can
+    # self-diagnose (was opaque "No models found for that provider/key").
+    assert "No models found for that provider/key" in msg
+    assert "HTTP 401" in msg
+    assert "https://api.example.com/v1/models" in msg
+
+
+def test_lmstudio_endpoint_error_message_includes_hint_and_probed_url():
+    # Issue #25: when the user pastes an LM Studio URL, surface a port-aware
+    # hint and the URL we actually probed (not the bare base URL).
+    msg = model_routes._model_endpoint_error_message(
+        "http://localhost:1234/v1",
+        {"error": "HTTP 200"},  # 200-with-empty-list is the LM Studio trap
+    )
+
+    assert "LM Studio" in msg
+    assert "port 1234" in msg
+    assert "http://localhost:1234/v1/models" in msg
+    assert "Developer Server" in msg
+
+
+def test_lmstudio_error_for_bare_host_port_probes_v1_models(monkeypatch):
+    # Regression: build_models_url must add /v1 for path-less LM Studio URLs
+    # (the OpenAI-compatible branch lands on /v1/models for LM Studio).
+    # _is_ollama_native_url would otherwise match localhost+empty path and
+    # route to /api/tags, masking the LM Studio URL we want to assert on.
+    monkeypatch.setattr("src.llm_core._is_ollama_native_url", lambda url: False)
+    msg = model_routes._model_endpoint_error_message(
+        "http://localhost:1234",
+        {"error": "HTTP 200"},
+    )
+    assert "LM Studio" in msg
+    assert "http://localhost:1234/v1/models" in msg
 
 
 # ── _rewrite_loopback_for_docker (issue #25: LM Studio on host loopback) ──

tests/test_personal_remove_dir_confinement.py

@@ -0,0 +1,43 @@
+"""Regression: remove_directory_from_rag must confine its path to PERSONAL_DIR.
+
+DELETE /api/personal/remove_directory took a raw ``directory`` query parameter
+and passed it straight to ``personal_docs_manager.remove_directory`` /
+``rag.remove_directory`` with no containment check — unlike add_directory_to_rag,
+which resolves the path via ``_resolve_allowed_personal_dir`` first. This pins
+the parity fix.
+
+``_resolve_allowed_personal_dir`` is a closure inside ``setup_personal_routes``,
+so this is a source-level test, matching test_personal_dir_symlink_escape.py.
+"""
+import ast
+from pathlib import Path
+
+SRC = Path(__file__).resolve().parent.parent / "routes" / "personal_routes.py"
+
+
+def _function_source(src_text: str, name: str) -> str:
+    tree = ast.parse(src_text)
+    for node in ast.walk(tree):
+        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == name:
+            return ast.get_source_segment(src_text, node)
+    raise AssertionError(f"{name} not found in {SRC}")
+
+
+def test_remove_directory_confines_path():
+    body = _function_source(SRC.read_text(), "remove_directory_from_rag")
+    assert "_resolve_allowed_personal_dir(" in body, (
+        "remove_directory_from_rag must call _resolve_allowed_personal_dir to "
+        "confine the user-supplied directory to PERSONAL_DIR (parity with "
+        "add_directory_to_rag)"
+    )
+
+
+def test_confinement_runs_before_removal_sinks():
+    """The confinement must happen before the path reaches either removal sink."""
+    body = _function_source(SRC.read_text(), "remove_directory_from_rag")
+    resolve_idx = body.index("_resolve_allowed_personal_dir(")
+    for sink in ("personal_docs_manager.remove_directory(", "rag.remove_directory("):
+        assert sink in body, f"expected sink {sink} in remove_directory_from_rag"
+        assert body.index(sink) > resolve_idx, (
+            f"{sink} runs before _resolve_allowed_personal_dir — path not confined"
+        )

tests/test_provider_endpoints.py

@@ -58,7 +58,7 @@ PROVIDER_CASES = [
      "https://api.x.ai/v1/models"),
     ("deepseek", "https://api.deepseek.com",
      "https://api.deepseek.com/chat/completions",
-     "https://api.deepseek.com/models"),
+     "https://api.deepseek.com/v1/models"),
     # Gemini's OpenAI-compatible surface — treated as a generic OpenAI endpoint.
     ("gemini_openai", "https://generativelanguage.googleapis.com/v1beta/openai",
      "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions",

tests/test_realesrgan_torchvision_compat.py

@@ -0,0 +1,47 @@
+import sys
+import types
+
+from src.optional_deps import (
+    patch_realesrgan_torchvision_compat,
+    prepare_optional_dependency_import,
+)
+
+
+def test_realesrgan_patch_restores_removed_functional_tensor_module(monkeypatch):
+    for name in list(sys.modules):
+        if name.startswith("torchvision"):
+            monkeypatch.delitem(sys.modules, name, raising=False)
+
+    sentinel = object()
+    torchvision = types.ModuleType("torchvision")
+    transforms = types.ModuleType("torchvision.transforms")
+    functional = types.ModuleType("torchvision.transforms.functional")
+    functional.rgb_to_grayscale = sentinel
+    transforms.functional = functional
+    torchvision.transforms = transforms
+    monkeypatch.setitem(sys.modules, "torchvision", torchvision)
+    monkeypatch.setitem(sys.modules, "torchvision.transforms", transforms)
+    monkeypatch.setitem(sys.modules, "torchvision.transforms.functional", functional)
+
+    patch_realesrgan_torchvision_compat()
+
+    shim = sys.modules["torchvision.transforms.functional_tensor"]
+    assert shim.rgb_to_grayscale is sentinel
+    assert shim.rgb_to_grayscale is functional.rgb_to_grayscale
+
+
+def test_prepare_optional_dependency_import_scopes_patch_to_realesrgan(monkeypatch):
+    import src.optional_deps as optional_deps
+
+    calls = []
+    monkeypatch.setattr(
+        optional_deps,
+        "patch_realesrgan_torchvision_compat",
+        lambda: calls.append("patched"),
+    )
+
+    prepare_optional_dependency_import("diffusers")
+    assert calls == []
+
+    prepare_optional_dependency_import("realesrgan")
+    assert calls == ["patched"]

tests/test_resend_message_nondestructive.py

@@ -0,0 +1,43 @@
+"""Regression guard for #4149: normal Resend must not delete chat history.
+
+chat.js is browser-heavy, so this pins the source-level contract: the footer's
+plain "Resend message" path appends a fresh send, while regenerate-only paths
+must opt into truncating/replacing from the selected message.
+"""
+
+from pathlib import Path
+
+
+_REPO = Path(__file__).resolve().parent.parent
+_CHAT_JS = _REPO / "static" / "js" / "chat.js"
+_CHAT_RENDERER_JS = _REPO / "static" / "js" / "chatRenderer.js"
+
+
+def _resend_body() -> str:
+    src = _CHAT_JS.read_text(encoding="utf-8")
+    start = src.index("export async function resendUserMessage(")
+    end = src.index("export async function regenerateFrom(", start)
+    return src[start:end]
+
+
+def test_resend_message_does_not_truncate_by_default():
+    body = _resend_body()
+
+    assert "opts = {}" in body
+    assert "const replaceFromHere = Boolean(opts && opts.replaceFromHere);" in body
+
+    guard_idx = body.index("if (replaceFromHere)")
+    truncate_idx = body.index("/api/session/${sessionId}/truncate")
+    hide_idx = body.index("_hideUserBubble = true;")
+
+    assert guard_idx < truncate_idx
+    assert guard_idx < hide_idx
+    assert "/truncate" not in body[:guard_idx]
+    assert "_hideUserBubble = true;" not in body[:guard_idx]
+
+
+def test_only_regenerate_callers_opt_into_replace_from_here():
+    renderer = _CHAT_RENDERER_JS.read_text(encoding="utf-8")
+
+    assert "window.chatModule.resendUserMessage(msgElement);" in renderer
+    assert "window.chatModule.resendUserMessage(userMsgEl, { replaceFromHere: true });" in renderer

tests/test_review_regressions.py

@@ -701,6 +701,34 @@ def test_single_user_mode_keeps_full_tool_access_when_auth_disabled(monkeypatch)
     assert blocked_tools_for_owner(None) == set()
 
 
+def test_auth_disabled_configured_mode_keeps_full_tool_access(monkeypatch):
+    """AUTH_ENABLED=false is still intentional single-user mode after setup.
+
+    Once an admin account exists, AuthManager.is_configured becomes true. The
+    tool gate must still honor explicit auth-disabled mode before requiring an
+    owner/admin match, otherwise agent mode hides email/MCP/local tools from the
+    operator.
+    """
+    monkeypatch.setenv("AUTH_ENABLED", "false")
+    auth_mod = _install_core_auth_stub(monkeypatch)
+
+    class FakeAuth:
+        is_configured = True
+
+        def is_admin(self, username):
+            return False
+
+    monkeypatch.setattr(auth_mod, "AuthManager", lambda: FakeAuth())
+
+    from src.tool_security import (
+        blocked_tools_for_owner,
+        owner_is_admin_or_single_user,
+    )
+
+    assert owner_is_admin_or_single_user(None) is True
+    assert blocked_tools_for_owner(None) == set()
+
+
 @pytest.mark.asyncio
 async def test_webhook_tool_reuses_private_url_validation():
     class FakeDb:

tests/test_security_regressions.py

@@ -972,7 +972,7 @@ def test_mcp_oauth_page_escapes_reflected_values():
     src = Path(__file__).resolve().parents[1] / "routes" / "mcp_routes.py"
     text = src.read_text()
     body = text.split("def _oauth_authorize_page(", 1)[1].split("return f", 1)[0]
-    for var in ("auth_url", "server_id", "host"):
+    for var in ("auth_url", "server_id", "host", "redirect_uri"):
         assert f"{var} = html.escape({var}" in body, var
 
 
@@ -981,6 +981,18 @@ def _import_mcp_routes():
     return importlib.import_module("routes.mcp_routes")
 
 
+def test_google_mcp_oauth_uses_configured_redirect_base(monkeypatch):
+    monkeypatch.setenv("OAUTH_REDIRECT_BASE_URL", "https://odysseus.example/app/")
+    monkeypatch.delenv("APP_PUBLIC_URL", raising=False)
+    sys.modules.pop("src.mcp_oauth", None)
+    mcp_routes = _import_mcp_routes()
+
+    assert (
+        mcp_routes._mcp_oauth_redirect_uri()
+        == "https://odysseus.example/app/api/mcp/oauth/callback"
+    )
+
+
 def test_mcp_oauth_paths_resolve_under_data_dir(tmp_path, monkeypatch):
     mcp_routes = _import_mcp_routes()
     monkeypatch.setattr(mcp_routes, "MCP_OAUTH_DIR", str(tmp_path / "data" / "mcp_oauth"))

tests/test_serve_profiles.py

@@ -81,6 +81,18 @@ def test_context_capped_at_model_limit():
         assert p["ctx"] <= 32768, p
 
 
+def test_small_context_model_still_gets_profiles():
+    """A model whose trained context is below the 8192 shrink floor must still
+    produce serve profiles, capped at its own limit — the loop floor must not
+    exclude it entirely (125 of the catalog models have context_length < 8192)."""
+    small_ctx_model = dict(_DENSE_8B, name="SmolLM-135M", context_length=2048)
+    profs = compute_serve_profiles(_sys(24.0), small_ctx_model)
+    assert profs, "sub-8192-context model produced no profiles"
+    for p in profs:
+        assert p["ctx"] <= 2048, p          # never exceeds the model's trained limit
+        assert p["ctx"] > 0
+
+
 def test_no_gpu_returns_empty():
     """No VRAM detected → no GPU profiles (caller falls back to manual flags)."""
     assert compute_serve_profiles({"backend": "cpu_x86", "gpu_vram_gb": 0}, _QWEN_35B_MOE) == []

tests/test_shell_routes.py

@@ -13,6 +13,7 @@ import pytest
 
 from routes.shell_routes import (
     _find_line_break,
+    _import_optional_dependency_for_status,
     _running_in_container,
     _docker_row_status,
     _package_installed_from_probe,
@@ -376,6 +377,26 @@ class TestPackageProbeStatus:
         assert "add_user_install_bins_to_path()" in script
         assert "shutil.which(b)" in script
 
+    def test_status_import_prepares_optional_dependency(self, monkeypatch):
+        import routes.shell_routes as shell_routes
+
+        calls = []
+        monkeypatch.setattr(
+            shell_routes,
+            "prepare_optional_dependency_import",
+            lambda name: calls.append(name),
+        )
+        monkeypatch.setattr(
+            shell_routes.importlib,
+            "import_module",
+            lambda name: SimpleNamespace(__name__=name),
+        )
+
+        module = _import_optional_dependency_for_status("realesrgan")
+
+        assert module.__name__ == "realesrgan"
+        assert calls == ["realesrgan"]
+
 
 class TestSshBaseArgv:
     def test_basic_host_no_port(self):

tests/test_tool_rag_contacts_domain.py

@@ -0,0 +1,72 @@
+"""Regression: the agent tool-RAG domain classifier had no contacts domain,
+so contact-lookup requests matched no domain, were flagged low_signal, and had
+tool retrieval SKIPPED entirely — the model only received ALWAYS_AVAILABLE tools
+(manage_memory, ask_user, update_plan) and never `resolve_contact`/`manage_contact`,
+so it could not look up contacts from the CardDAV address book (it looped on
+manage_memory instead).
+
+Root cause: `_classify_agent_request` in src/agent_loop.py sets
+`low_signal = not continuation and not domains`; with no `contacts` domain,
+prompts like "What is Massimo's contact?" matched nothing → low_signal →
+retrieval skipped.
+
+The classifier is deterministic string matching (no embeddings / no DB), so it
+can be exercised directly.
+"""
+
+from src.agent_loop import (
+    _classify_agent_request,
+    _DOMAIN_TOOL_MAP,
+    _DOMAIN_RULES,
+    _domain_rules_for_tools,
+)
+
+
+def _classify(text):
+    return _classify_agent_request([{"role": "user", "content": text}], text)
+
+
+def test_contact_lookup_requests_get_contacts_domain():
+    """Contact-lookup phrasings must match the `contacts` domain and NOT be
+    treated as low-signal (which would skip tool retrieval)."""
+    prompts = [
+        "What is Massimo's contact?",
+        "What's John's phone number?",
+        "Show me my contacts",
+        "Look up Kevin's contact info",
+        "Find Alice's phone number",
+    ]
+    for p in prompts:
+        intent = _classify(p)
+        assert "contacts" in intent["domains"], f"expected contacts domain for: {p!r}"
+        assert intent["low_signal"] is False, f"must not be low_signal: {p!r}"
+
+
+def test_contact_management_requests_get_contacts_domain():
+    """Add/update/delete contact phrasings also resolve to the contacts domain."""
+    for p in ("add a new contact", "update Bob's phone number", "delete that contact",
+              "save this person to contacts"):
+        intent = _classify(p)
+        assert "contacts" in intent["domains"], f"expected contacts domain for: {p!r}"
+
+
+def test_contacts_domain_seeds_resolve_and_manage_contact():
+    """The domain must seed the actual contacts tools so they are offered even
+    when semantic retrieval misses."""
+    assert _DOMAIN_TOOL_MAP["contacts"] == {"resolve_contact", "manage_contact"}
+
+
+def test_contacts_domain_has_a_rule_pack():
+    """Every domain in _DOMAIN_TOOL_MAP needs a matching _DOMAIN_RULES entry,
+    otherwise _domain_rules_for_tools raises KeyError when the tools are selected."""
+    assert "contacts" in _DOMAIN_RULES
+    rules = _domain_rules_for_tools({"resolve_contact"})
+    assert any("Contacts rules" in r for r in rules)
+
+
+def test_non_contact_requests_do_not_match_contacts_domain():
+    """Guard against over-triggering: ordinary prompts must not be flagged contacts."""
+    assert "contacts" not in _classify("what is the capital of France")["domains"]
+    assert "contacts" not in _classify("reply to the latest email in my inbox")["domains"]
+    assert "contacts" not in _classify("generate an image of a sunset")["domains"]
+    assert "contacts" not in _classify("what's 2 plus 2")["domains"]

tests/test_tool_rag_keyword_hints.py

@@ -40,6 +40,14 @@ def test_tell_in_web_query_does_not_force_email_tools():
     assert "web_search" in tools and "web_fetch" in tools
 
 
+def test_explicit_web_search_query_gets_web_tools_without_retrieval():
+    """Explicit web-search phrasing must surface web tools even if embeddings
+    return nothing."""
+    ti = _index_without_embeddings()
+    tools = ti.get_tools_for_query("use web search and find a recipe for chocolate chip cookies")
+    assert "web_search" in tools and "web_fetch" in tools
+
+
 def test_genuine_email_query_still_gets_email_tools():
     """Removing 'tell' must not break real email intent — the actual email
     keywords still force-include the toolset."""

tests/test_tool_support_heuristic.py

@@ -25,6 +25,7 @@ def _compute_is_api_model(model: str, endpoint_url: str, endpoint_supports=None)
     ))
     model_no_tools = any(kw in model_lc for kw in (
         "deepseek-r1",
+        "gpt-oss",
     ))
 
     if endpoint_supports is True:
@@ -72,6 +73,11 @@ class TestDeepSeekToolSupport:
             "gemma4:e4b", "http://host.docker.internal:11434/v1"
         ) is False
 
+    def test_gpt_oss_local_openai_compat_defaults_to_fenced_tools(self):
+        assert _compute_is_api_model(
+            "gpt-oss-20b", "http://localhost:8000/v1"
+        ) is False
+
     def test_qwen_native_ollama_defaults_to_fenced_tools(self):
         assert _compute_is_api_model(
             "qwen3.5:4b", "http://localhost:11434/api/chat"
@@ -117,6 +123,12 @@ class TestDeepSeekToolSupport:
         )
         assert result is True
 
+    def test_endpoint_supports_true_overrides_gpt_oss_default(self):
+        result = _compute_is_api_model(
+            "gpt-oss-20b", "http://localhost:8000/v1", endpoint_supports=True
+        )
+        assert result is True
+
     def test_endpoint_supports_false_overrides_cloud(self):
         """supports_tools=False on an endpoint gates even cloud APIs."""
         result = _compute_is_api_model(

tests/test_web_search_raw_json_tool_call.py

@@ -0,0 +1,71 @@
+"""Local text models can leak web_search calls as prose plus bare JSON.
+
+gpt-oss-20b sometimes writes:
+
+    Need to do web_search for ...
+    {"query":"...", "time_filter":"week"}
+
+That is an intended tool call in non-native/textual tool mode, but older parsing
+only recognized fenced blocks, [TOOL_CALL], XML invoke, and tool_code markup.
+"""
+import json
+import sys
+from unittest.mock import MagicMock
+
+for mod in ['src.agent_tools', 'src.tool_parsing', 'src.tool_schemas', 'src.tool_execution']:
+    sys.modules.pop(mod, None)
+for mod in [
+    'sqlalchemy', 'sqlalchemy.orm', 'sqlalchemy.ext', 'sqlalchemy.ext.declarative',
+    'sqlalchemy.ext.hybrid', 'sqlalchemy.sql', 'sqlalchemy.sql.expression',
+    'src.database', 'core.models', 'core.database', 'core.auth'
+]:
+    if mod not in sys.modules:
+        sys.modules[mod] = MagicMock()
+
+import src.agent_tools  # noqa: E402, F401
+from src.tool_parsing import parse_tool_blocks, strip_tool_blocks  # noqa: E402
+
+
+def test_raw_json_after_web_search_phrase_runs_as_web_search():
+    text = (
+        "Need to do web_search for best chocolate chip cookies. Use web_search function.\n\n"
+        '{"query":"best chocolate chip cookie recipe","time_filter":"week"}'
+    )
+
+    blocks = parse_tool_blocks(text)
+
+    assert len(blocks) == 1
+    assert blocks[0].tool_type == "web_search"
+    payload = json.loads(blocks[0].content)
+    assert payload == {
+        "query": "best chocolate chip cookie recipe",
+        "time_filter": "week",
+    }
+
+
+def test_raw_json_without_web_tool_name_is_ignored():
+    text = 'Here is a saved search config:\n\n{"query":"private customer name"}'
+
+    assert parse_tool_blocks(text) == []
+
+
+def test_raw_json_fallback_is_disabled_for_native_parser_gate():
+    text = (
+        "Need to do web_search for best chocolate chip cookies.\n\n"
+        '{"query":"best chocolate chip cookie recipe"}'
+    )
+
+    assert parse_tool_blocks(text, skip_fenced=True) == []
+
+
+def test_strip_tool_blocks_removes_executed_raw_json():
+    text = (
+        "Need to do web_search for best chocolate chip cookies. Use web_search function.\n\n"
+        '{"query":"best chocolate chip cookie recipe","time_filter":"week"}'
+    )
+
+    cleaned = strip_tool_blocks(text)
+
+    assert '{"query"' not in cleaned
+    assert "best chocolate chip cookie recipe" not in cleaned
+    assert "Need to do web_search" in cleaned

tests/test_webhook_task_refs.py

@@ -7,16 +7,20 @@ releases it on completion.
 """
 import asyncio
 import sys
+import types
 
-# webhook_manager does `from src.database import SessionLocal, Webhook` at import
-# time. The shared test harness stubs src.database without Webhook, so ensure the
-# attribute exists before importing the manager. These tests never touch the DB
-# (the manager is built via __new__), so a placeholder class is sufficient.
-_db = sys.modules.get("src.database")
-if _db is not None and not hasattr(_db, "Webhook"):
+from tests.helpers.import_state import clear_module, preserve_import_state
+
+# Import the manager against a private database stub, then restore both modules
+# so collection does not mutate shared import state.
+with preserve_import_state("src.database", "src.webhook_manager"):
+    clear_module("src.database")
+    clear_module("src.webhook_manager")
+    _db = types.ModuleType("src.database")
+    _db.SessionLocal = object()
     _db.Webhook = type("Webhook", (), {})
-
-from src.webhook_manager import WebhookManager  # noqa: E402
+    sys.modules["src.database"] = _db
+    from src.webhook_manager import WebhookManager
 
 
 def test_spawn_tracked_holds_then_releases_reference():

tests/test_youtube_handler_consolidation.py

@@ -0,0 +1,104 @@
+"""Regression: the YouTube handler must live in a single module.
+
+Odysseus carried two independent copies of the handler — ``src.youtube_handler``
+and ``services.youtube.youtube_handler`` — that silently drifted:
+
+* ``app.py`` calls ``services.youtube.init_youtube()`` at startup, but the chat
+  flow imported ``extract_transcript_async`` from ``src.youtube_handler``. Those
+  were different module objects, so the ``YOUTUBE_AVAILABLE`` /
+  ``YouTubeTranscriptApi`` globals set by ``init_youtube`` never reached the chat
+  path and transcript extraction always reported "not available".
+* The comment-fetch timeout fix (PR #1002) landed only in the ``src`` copy.
+
+These tests pin the two import paths to one module object and verify the shared
+state and the broadened URL parsing.
+"""
+import sys
+import types
+
+import pytest
+
+
+def test_src_and_service_youtube_are_same_module():
+    """Both historical import paths must resolve to one module object so
+    behavior and module-level state cannot diverge again."""
+    import src.youtube_handler as src_yt
+    import services.youtube.youtube_handler as svc_yt
+
+    assert src_yt is svc_yt
+
+
+def test_init_youtube_visible_through_chat_import_path(monkeypatch):
+    """init_youtube() is invoked via services.youtube (as app.py does), but the
+    chat flow reads the API globals through src.youtube_handler. After
+    consolidation the globals set by init must be visible on both paths."""
+    import src.youtube_handler as src_yt
+    from services.youtube import init_youtube
+
+    # Pin the globals so monkeypatch restores them after the test, regardless
+    # of whether youtube_transcript_api is actually installed in this env.
+    monkeypatch.setattr(src_yt, "YOUTUBE_AVAILABLE", False, raising=False)
+    monkeypatch.setattr(src_yt, "YouTubeTranscriptApi", None, raising=False)
+
+    # Stand in for the real transcript package so init_youtube() succeeds
+    # without a network/library dependency.
+    stub = types.ModuleType("youtube_transcript_api")
+
+    class _StubApi:
+        pass
+
+    stub.YouTubeTranscriptApi = _StubApi
+    monkeypatch.setitem(sys.modules, "youtube_transcript_api", stub)
+
+    init_youtube()  # called exactly the way app.py calls it
+
+    assert src_yt.YOUTUBE_AVAILABLE is True
+    assert src_yt.YouTubeTranscriptApi is _StubApi
+
+
+@pytest.mark.parametrize(
+    "url,expected",
+    [
+        # Classic watch URLs across the youtube.com hosts.
+        ("https://www.youtube.com/watch?v=dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        ("https://youtube.com/watch?v=dQw4w9WgXcQ&t=42s", "dQw4w9WgXcQ"),
+        ("https://m.youtube.com/watch?v=dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        # YouTube Music shares the same paths and must resolve.
+        ("https://music.youtube.com/watch?v=dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        # Short links.
+        ("https://youtu.be/dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        ("https://youtu.be/dQw4w9WgXcQ?si=ab_cd", "dQw4w9WgXcQ"),
+        # Player/embed and the legacy /v/ embed.
+        ("https://www.youtube.com/embed/dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        ("https://www.youtube.com/embed/dQw4w9WgXcQ/", "dQw4w9WgXcQ"),
+        ("https://www.youtube.com/v/dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        # Shorts and live — previously unrecognized, so the chat pipeline
+        # dropped them entirely (excluded from web-fetch as a YouTube URL, but
+        # no id meant no transcript fetch either).
+        ("https://www.youtube.com/shorts/dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        ("https://www.youtube.com/shorts/dQw4w9WgXcQ?feature=share", "dQw4w9WgXcQ"),
+        ("https://www.youtube.com/live/dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        # Host matching is case-insensitive.
+        ("https://WWW.YouTube.com/watch?v=dQw4w9WgXcQ", "dQw4w9WgXcQ"),
+        # Non-video paths and non-YouTube hosts yield no id.
+        ("https://www.youtube.com/", None),
+        ("https://www.youtube.com/feed/subscriptions", None),
+        ("https://example.com/watch?v=dQw4w9WgXcQ", None),
+        ("https://vimeo.com/76979871", None),
+    ],
+)
+def test_extract_youtube_id(url, expected):
+    from src.youtube_handler import extract_youtube_id
+
+    assert extract_youtube_id(url) == expected
+
+
+def test_shorts_url_is_recognized_and_extractable():
+    """A Shorts URL is treated as a YouTube link (so the chat pipeline excludes
+    it from generic web-fetch). It must therefore yield an id, or the video is
+    silently dropped — fetched by neither path."""
+    from src.youtube_handler import is_youtube_url, extract_youtube_id
+
+    url = "https://www.youtube.com/shorts/dQw4w9WgXcQ"
+    assert is_youtube_url(url)
+    assert extract_youtube_id(url) == "dQw4w9WgXcQ"