Tighten PR template and CONTRIBUTING to gate visual/style changes

The post-launch PR flood from LLM coding agents drowned the repo in PRs
that don't run the app, attach no screenshots, and invent parallel
component styling. Even tiny correctness fixes accumulated into a visual
mess. Make the rules explicit in both the PR template and CONTRIBUTING:

- Run the app and view the change in a browser before submitting.
- Required screenshot for any UI/render touch (no longer "delete if not UI").
- Explicit style requirements: reuse CSS variables, no Unicode emoji
  (use SVG icons), monospaced font, dark theme, no parallel widgets.
- Direct callout for bulk agent-generated PRs: open an issue first.

PRs that ignore these will be closed without merge, regardless of code
correctness.

.github/pull_request_template.md

@@ -23,6 +23,7 @@ Fixes #
 - [ ] I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues) and [open PRs](https://github.com/pewdiepie-archdaemon/odysseus/pulls) — this is not a duplicate.
 - [ ] This PR targets `main`
 - [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
+- [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough.
 
 ## How to Test
 
@@ -33,6 +34,20 @@ Fixes #
 2.
 3.
 
-## Screenshots (UI changes only)
+## Visual / UI changes — REQUIRED if you touched anything that renders
 
-<!-- Drag and drop images or a screen recording here. Delete this section if not a UI change. -->
+**Anything that changes what the UI looks like — buttons, icons, padding, colors, fonts, spacing, layout, CSS, HTML, SVG, or any `static/js/` module that draws to the DOM — needs all of the following. PRs that change rendering without these WILL be closed.**
+
+- [ ] **Screenshot or short clip** of the change in the running app, attached below. Mobile screenshot too if the change affects mobile.
+- [ ] **Style match**: the change uses Odysseus's existing visual language. Specifically:
+  - Reuse existing CSS variables (`--red`, `--fg`, `--bg`, `--card`, `--border`, etc.) — do not introduce new color values, font sizes, or spacing units.
+  - Reuse existing button/input/card/border classes. Don't invent parallel styling.
+  - **No Unicode emoji in UI or code.** Use inline SVG (matching the monochrome icon style already in `static/index.html`) or plain text.
+  - Monospaced font (`Fira Code`) for primary UI text. Don't override.
+  - Dark theme is the default; any light-mode work must be wired through the existing theme system, not hard-coded.
+- [ ] **No new component patterns.** If a similar widget already exists in the app, extend it instead of writing a parallel one.
+- [ ] **I am not an LLM agent submitting a bulk PR.** If you are, please open an issue describing the problem first — bulk auto-generated PRs that don't match the project's visual style are closed on sight, even when the underlying fix is correct.
+
+### Screenshots / clips
+
+<!-- Drag and drop images or a screen recording here. Required for any UI/visual change. -->

ACKNOWLEDGMENTS.md

@@ -33,8 +33,8 @@ The full license texts are kept in [`licenses/`](licenses/).
 - **[Tongyi DeepResearch](https://github.com/Alibaba-NLP/DeepResearch)** by
   **Alibaba-NLP / Tongyi Lab** — the multi-step deep-research agent pipeline.
   Copyright © Alibaba-NLP / Tongyi Lab. **Apache-2.0.** Adapted for Odysseus's
-  Deep Research feature (`api/research_*.py`, `routes/research_routes.py`,
-  `services/search/`). Full text in
+  Deep Research feature (`services/research/`, `src/research_handler.py`,
+  `routes/research_routes.py`, `services/search/`). Full text in
   [`licenses/DeepResearch-Apache-2.0.txt`](licenses/DeepResearch-Apache-2.0.txt).
 
 ---
@@ -47,7 +47,7 @@ just composed.
 
 | Service | Image | Purpose | License |
 |---|---|---|---|
-| [SearXNG](https://github.com/searxng/searxng) | `searxng/searxng:latest` | Default metasearch backend | AGPL-3.0 |
+| [SearXNG](https://github.com/searxng/searxng) | `searxng/searxng:2026.5.31-7159b8aed` (pinned tag; see compose) | Default metasearch backend | AGPL-3.0 |
 | [ChromaDB](https://github.com/chroma-core/chroma) | `chromadb/chroma:latest` | Vector store for memory / RAG | Apache-2.0 |
 | [ntfy](https://github.com/binwiederhier/ntfy) | `binwiederhier/ntfy` | Push notifications (self-hosted reminders) | Apache-2.0 / GPL-2.0 |
 

CONTRIBUTING.md

@@ -57,12 +57,32 @@ Good pull requests usually include:
 
 - A short explanation of the bug or feature.
 - The files or areas changed.
-- Manual test steps or automated test results.
+- Manual test steps or automated test results from running the actual app, not just the test suite.
 - Screenshots or short recordings for UI changes.
 - Links to related issues, for example `Fixes #123`.
 
 Please keep PRs small. Large PRs that mix unrelated cleanup, formatting, refactors, and behavior changes are much harder to review.
 
+> **Auto-generated PRs.** If you are running an LLM agent (Devin, Cursor, OpenHands, Claude Code, etc.) against this repo: please open an issue describing the problem first instead of opening a PR directly. Bulk agent-generated PRs that don't match the project's visual style or contribution format will be closed without review, even when the underlying fix is correct.
+
+## Style and visual changes
+
+Odysseus has an intentional visual style. PRs that ignore it will be closed without merge, no matter how correct the underlying code is.
+
+Before submitting any change that affects what the app looks like — buttons, icons, fonts, colors, spacing, layout, CSS, HTML, SVG, or any `static/js/` module that draws to the DOM — please:
+
+1. **Run the app locally** and view the change in a browser. Type-checks and unit tests are not enough.
+2. **Attach a screenshot or short clip** of the change in the running app. Add a mobile screenshot too if the change affects mobile.
+3. **Match the existing visual language.** Specifically:
+   - Reuse existing CSS variables (`--red`, `--fg`, `--bg`, `--card`, `--border`, …). Do not introduce new color values, font sizes, or spacing units.
+   - Reuse existing button, input, card, and border classes. Don't invent parallel styling for similar widgets.
+   - **No Unicode emoji in UI or code.** Use inline SVG (matching the monochrome icon style already in `static/index.html`) or plain text.
+   - Monospaced font (`Fira Code`) for primary UI text. Don't override.
+   - Dark theme is the default; any light-mode work goes through the existing theme system, not hard-coded.
+4. **Don't add parallel components.** If a similar widget already exists in the app, extend it instead of writing a new one.
+
+If you are unsure whether a change is "visual," it is. Default to attaching a screenshot.
+
 ## Issue Reports
 
 For bugs, include:

README.md

@@ -281,6 +281,41 @@ Local GPU *serving* of vLLM/SGLang needs Linux/WSL2; for a local model on Window
 Open `http://localhost:7000`, log in with the generated admin password,
 and configure everything else inside **Settings**.
 
+## Troubleshooting & Advanced Setup
+
+### `chromadb-client` conflicts with embedded ChromaDB
+If `chromadb-client` (the lightweight HTTP-only package) is installed alongside the full `chromadb` package, Odysseus starts but ChromaDB silently falls back to HTTP-only mode and fails.
+
+**Fix:** uninstall `chromadb-client` and force-reinstall the full package:
+```bash
+./venv/bin/pip uninstall chromadb-client -y
+./venv/bin/pip install --force-reinstall chromadb
+```
+
+### HTTPS + LAN/Tailscale exposure
+To expose Odysseus on a local network or Tailscale with HTTPS:
+1. Change the bind address to `0.0.0.0` in `.env` (`APP_BIND=0.0.0.0` or `ODYSSEUS_HOST=0.0.0.0`).
+2. Generate a locally-trusted cert for your LAN/Tailscale IPs using [mkcert](https://github.com/FiloSottile/mkcert):
+   ```bash
+   mkcert -install
+   mkcert -cert-file cert.pem -key-file key.pem 192.168.1.100 tailscale-ip
+   ```
+3. Run `uvicorn` with the generated certs:
+   ```bash
+   python -m uvicorn app:app --host 0.0.0.0 --port 7000 --ssl-certfile=cert.pem --ssl-keyfile=key.pem
+   ```
+4. Install the `mkcert` CA on any other device you want to access Odysseus from (e.g., for iOS, email the `rootCA.pem` to yourself, install the profile, and trust it in Certificate Trust Settings).
+
+### Optional Dependencies
+`requirements-optional.txt` contains packages that unlock extra features. It is not installed by default.
+
+| Package | Feature unlocked |
+|---------|-----------------|
+| `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
+| `duckduckgo-search` | DuckDuckGo as a search provider option. |
+| `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
+| `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
+
 ## Security Notes
 Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
 

ROADMAP.md

@@ -1,6 +1,6 @@
 # Roadmap / Help Wanted
 
-Odysseus is on a voyage, but not home yet. It works great for me (lol), but this is ship is moving fast and feedback/help would be appreciated! (I dont know what I'm doing hlep).
+Odysseus is on a voyage, but not home yet. It works great for me (lol), but this ship is moving fast and feedback/help would be appreciated! (I don't know what I'm doing, help).
 
 If you see weird CSS, strange layout behavior, or a suspiciously murky corner of
 the codebase, you are probably right to stay away.

app.py

@@ -42,6 +42,7 @@ import secrets
 from datetime import datetime
 from typing import Dict
 
+from contextlib import asynccontextmanager
 from fastapi import FastAPI, Request, HTTPException
 from fastapi.responses import JSONResponse, FileResponse, HTMLResponse
 from fastapi.middleware.cors import CORSMiddleware
@@ -74,6 +75,9 @@ logging.basicConfig(
 logger = logging.getLogger(__name__)
 
 # ========= APP =========
+# Lifespan is defined below (after all helpers it references are in scope)
+# and passed to FastAPI so we can use the modern context-manager lifecycle
+# instead of the deprecated @app.on_event("startup"/"shutdown") decorators.
 app = FastAPI(
     title="AI Chat Application",
     description="Comprehensive AI chat with memory, research, and multi-modal capabilities",
@@ -805,8 +809,19 @@ async def runtime_info() -> Dict[str, object]:
 
 # ========= LIFECYCLE =========
 
-@app.on_event("startup")
-async def startup_event():
+@asynccontextmanager
+async def _lifespan(app):
+    """Modern lifespan context manager replacing deprecated @app.on_event."""
+    # ── STARTUP ──
+    await _startup_event()
+    yield
+    # ── SHUTDOWN ──
+    await _shutdown_event()
+
+app.router.lifespan_context = _lifespan
+
+
+async def _startup_event():
     global upload_cleanup_task
     logger.info("Application starting up...")
     webhook_manager.set_loop(asyncio.get_running_loop())
@@ -1030,8 +1045,7 @@ async def startup_event():
     _startup_tasks.append(asyncio.create_task(_skill_audit_nightly_loop()))
     logger.info("Application startup complete")
 
-@app.on_event("shutdown")
-async def shutdown_event():
+async def _shutdown_event():
     logger.info("Application shutting down...")
     if upload_cleanup_task:
         upload_cleanup_task.cancel()

companion/pairing.py

@@ -64,11 +64,16 @@ def find_admin_user() -> str | None:
     auth_path = os.path.join("data", "auth.json")
     try:
         with open(auth_path, "r", encoding="utf-8") as f:
-            users = (json.load(f) or {}).get("users", {})
+            data = json.load(f)
     except (OSError, json.JSONDecodeError):
         return None
+    if not isinstance(data, dict):
+        return None
+    users = data.get("users") or {}
+    if not isinstance(users, dict):
+        return None
     for uname, udata in users.items():
-        if udata.get("is_admin") is True:
+        if isinstance(udata, dict) and udata.get("is_admin") is True:
             return uname
     return next(iter(users), None)
 

core/session_manager.py

@@ -202,6 +202,15 @@ class SessionManager:
         """Persist a single message to the database."""
         db = SessionLocal()
         try:
+            db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
+            if db_session is None:
+                # A stream/tool callback can outlive a session delete. Do not
+                # create a chat_messages row with no parent session; also drop
+                # any stale cached session so later writes fail closed too.
+                self.sessions.pop(session_id, None)
+                logger.warning("Dropping message for deleted session %s", session_id)
+                return
+
             msg_id = str(uuid.uuid4())
             msg_time = datetime.utcnow()
             if message.metadata is None:
@@ -223,15 +232,13 @@ class SessionManager:
             )
             db.add(db_message)
 
-            db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
-            if db_session:
-                db_session.message_count = len(self.sessions.get(session_id, {}).history) if session_id in self.sessions else 0
-                _now = datetime.now(timezone.utc)
-                db_session.last_accessed = _now
-                # Clean "last conversation" timestamp — only bumped here on a
-                # real message persist, so it powers an accurate "Last active"
-                # sort that ignores renames / model swaps / mere opens.
-                db_session.last_message_at = _now
+            db_session.message_count = len(self.sessions.get(session_id, {}).history) if session_id in self.sessions else 0
+            _now = datetime.now(timezone.utc)
+            db_session.last_accessed = _now
+            # Clean "last conversation" timestamp — only bumped here on a
+            # real message persist, so it powers an accurate "Last active"
+            # sort that ignores renames / model swaps / mere opens.
+            db_session.last_message_at = _now
 
             db.commit()
 
@@ -297,7 +304,15 @@ class SessionManager:
                     id=msg_id,
                     session_id=session_id,
                     role=message.role,
-                    content=message.content,
+                    # Multimodal content (image/audio attachments) is a list;
+                    # serialize to JSON so the Text column round-trips via
+                    # _parse_msg_content. Storing the raw list let SQLAlchemy
+                    # bind its single-quoted repr, which _parse_msg_content
+                    # cannot parse (it looks for double-quoted "type"), so the
+                    # attachment was destroyed on reload. Mirrors _persist_message.
+                    content=(json.dumps(message.content)
+                             if isinstance(message.content, list)
+                             else message.content),
                     meta_data=json.dumps(message.metadata) if message.metadata else None,
                     timestamp=now + timedelta(microseconds=i),
                 )

docker/entrypoint.sh

@@ -80,6 +80,11 @@ export VLLM_USE_FLASHINFER_SAMPLER="${VLLM_USE_FLASHINFER_SAMPLER:-0}"
 # vLLM and helper scripts land here because /app is the non-root user's HOME.
 export PATH="/app/.local/bin:$PATH"
 
+# Run first-time setup as the app user so data/ files get the right ownership.
+# setup.py is idempotent — skips auth.json / .env if they already exist.
+# || true so a setup failure never prevents the container from starting.
+gosu "$PUID:$PGID" python /app/setup.py || true
+
 # Drop root and run the actual app. `gosu` is preferred over `su` /
 # `sudo` because it cleans up the process tree (no extra shell layer)
 # so signals (SIGTERM from `docker stop`) reach uvicorn directly.

mcp_servers/email_server.py

@@ -337,14 +337,25 @@ def _decode_header(raw):
     """Decode MIME encoded header."""
     if not raw:
         return ""
-    parts = email.header.decode_header(raw)
-    decoded = []
-    for data, charset in parts:
-        if isinstance(data, bytes):
-            decoded.append(data.decode(charset or "utf-8", errors="replace"))
-        else:
-            decoded.append(data)
-    return " ".join(decoded)
+    try:
+        # make_header concatenates per RFC 2047: no spurious space between an
+        # encoded-word and adjacent plain text (plain runs keep their own
+        # whitespace), and whitespace between two adjacent encoded-words is
+        # dropped. The old " ".join produced "Re:  Jose" style double spaces
+        # on every non-ASCII subject or sender.
+        return str(email.header.make_header(email.header.decode_header(raw)))
+    except Exception:
+        # Malformed header or unknown charset: lossy per-part decode
+        decoded = []
+        for data, charset in email.header.decode_header(raw):
+            if isinstance(data, bytes):
+                try:
+                    decoded.append(data.decode(charset or "utf-8", errors="replace"))
+                except LookupError:
+                    decoded.append(data.decode("utf-8", errors="replace"))
+            else:
+                decoded.append(data)
+        return "".join(decoded)
 
 
 def _extract_text(msg):

mcp_servers/memory_server.py

@@ -161,10 +161,9 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
                 deleted_text = m.get("text", "")
                 deleted_category = m.get("category", "")
                 break
-        original_len = len(memories)
-        memories = [m for m in memories if not m.get("id", "").startswith(memory_id)]
-        if len(memories) == original_len:
+        if not full_id:
             return [TextContent(type="text", text=f"Error: Memory '{memory_id}' not found")]
+        memories = [m for m in memories if m.get("id") != full_id]
         _memory_manager.save(memories)
         if _memory_vector and _memory_vector.healthy and full_id:
             try:

mcp_servers/rag_server.py

@@ -105,7 +105,9 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
         directory = _dir.strip() if isinstance(_dir, str) else ""
         if not directory:
             return [TextContent(type="text", text="Error: add_directory needs a directory path")]
-        directory = os.path.expanduser(directory)
+        # Store an absolute path so indexed `source` metadata is absolute and
+        # remove_directory (which abspath-normalizes) can match it later (#1660).
+        directory = os.path.abspath(os.path.expanduser(directory))
         if not os.path.isdir(directory):
             return [TextContent(type="text", text=f"Error: Directory not found: {directory}")]
         if not _rag_manager:

routes/backup_routes.py

@@ -77,7 +77,12 @@ def setup_backup_routes(memory_manager, preset_manager, skills_manager) -> APIRo
         # ── Memories ──
         if "memories" in body and isinstance(body["memories"], list):
             existing = memory_manager.load_all()
-            existing_texts = {e.get("text", "").strip().lower() for e in existing}
+            # Dedup against THIS user's own memories only. Using every tenant's
+            # rows (load_all) meant a memory whose text matched any other
+            # user's was silently skipped, so the importing user lost their own
+            # data. The full store is still saved back below.
+            existing_texts = {e.get("text", "").strip().lower()
+                              for e in existing if e.get("owner") == user}
             added = 0
             for mem in body["memories"]:
                 if not isinstance(mem, dict) or not mem.get("text"):

routes/calendar_routes.py

@@ -16,6 +16,23 @@ from src.auth_helpers import get_current_user, require_user
 
 logger = logging.getLogger(__name__)
 
+
+def _ics_naive_dtstart(dt):
+    """Naive value matching how import_ics STORES CalendarEvent.dtstart.
+
+    Timed tz-aware events are stored as UTC with tzinfo stripped, all-day
+    dates as midnight datetimes, naive datetimes unchanged. The ICS dedup
+    must compute the same value or a re-import never matches the stored row.
+    """
+    if isinstance(dt, datetime):
+        if dt.tzinfo is not None:
+            from datetime import timezone as _tz
+            return dt.astimezone(_tz.utc).replace(tzinfo=None)
+        return dt
+    if isinstance(dt, date):
+        return datetime(dt.year, dt.month, dt.day)
+    return dt
+
 # Single-user fallback identity. Used only when:
 #   1. The app is configured for single-user (no auth middleware), AND
 #   2. The request didn't resolve to an authenticated user.
@@ -338,8 +355,8 @@ def _parse_dt(s: str) -> datetime:
             return None
         return h, mn
 
-    # today/tomorrow/yesterday [at] TIME
-    m = _re.match(r'^(today|tomorrow|tmrw|yesterday)(?:\s+at)?\s*(.*)$', lower)
+    # today/tonight/tomorrow/yesterday [at] TIME
+    m = _re.match(r'^(today|tonight|tomorrow|tmrw|yesterday)(?:\s+at)?\s*(.*)$', lower)
     if m:
         word, rest = m.group(1), m.group(2).strip()
         base = today
@@ -453,8 +470,21 @@ def _expand_rrule(
         return [d]
 
     # Parse the rrule, applying it to the base dtstart.
+    rrule_str = ev.rrule
+    if ev.dtstart is not None and getattr(ev.dtstart, "tzinfo", None) is None:
+        # Events are stored with a naive (UTC) dtstart, but standard .ics
+        # exporters (Google/Apple/Outlook/Fastmail) write the bound as an
+        # absolute UTC value, e.g. UNTIL=20240105T090000Z. dateutil refuses to
+        # mix a tz-aware UNTIL with a naive DTSTART ("RRULE UNTIL values must be
+        # specified in UTC when DTSTART is timezone-aware"), so the except branch
+        # below would silently collapse the whole series to a single event.
+        # Drop the trailing Z so UNTIL matches the naive DTSTART.
+        import re as _re
+        rrule_str = _re.sub(
+            r"(UNTIL=\d{8}(?:T\d{6})?)Z", r"\1", rrule_str, flags=_re.IGNORECASE
+        )
     try:
-        rule = rrulestr(ev.rrule, dtstart=ev.dtstart)
+        rule = rrulestr(rrule_str, dtstart=ev.dtstart)
     except Exception as ex:
         logger.warning(
             "Failed to parse rrule=%r for event %s: %s", ev.rrule, ev.uid, ex
@@ -629,6 +659,18 @@ def setup_calendar_routes() -> APIRouter:
                     headers={"Depth": "0", "Content-Type": "application/xml"},
                     content=propfind_body,
                 )
+                # If the server demands Digest (Baïkal default, SabreDAV-based
+                # servers, Radicale with htdigest), the Basic attempt above
+                # 401s. Retry once with httpx.DigestAuth so this test matches
+                # what the real sync does via caldav.DAVClient in
+                # src/caldav_sync.py (which negotiates the scheme).
+                if r.status_code == 401 and "digest" in r.headers.get("www-authenticate", "").lower():
+                    r = await cx.request(
+                        "PROPFIND", url,
+                        auth=httpx.DigestAuth(user, pw),
+                        headers={"Depth": "0", "Content-Type": "application/xml"},
+                        content=propfind_body,
+                    )
             # 207 = Multi-Status — standard CalDAV success. 200 also
             # acceptable. Anything else (401/403/404/5xx) means trouble.
             if r.status_code in (200, 207):
@@ -1011,7 +1053,12 @@ def setup_calendar_routes() -> APIRouter:
                 source_uid = str(comp.get("uid", "")) or None
                 if source_uid:
                     src_dtstart = dtstart.dt
-                    naive_src = src_dtstart.replace(tzinfo=None) if hasattr(src_dtstart, 'tzinfo') and src_dtstart.tzinfo else src_dtstart
+                    # Normalize to the SAME naive form import_ics stores, so a
+                    # re-import of a tz-aware event matches the existing row.
+                    # The old code stripped tzinfo WITHOUT converting to UTC
+                    # (wall clock), while storage converts to UTC first, so
+                    # every re-import of a TZID event created a duplicate.
+                    naive_src = _ics_naive_dtstart(src_dtstart)
                     existing = (
                         db.query(CalendarEvent)
                         .filter(

routes/chat_helpers.py

@@ -121,7 +121,7 @@ def needs_auto_name(name: str) -> bool:
     if name.startswith("Chat:") or name == "Chat":
         return True
     # Default frontend name: "modelname HH:MM:SS AM/PM"
-    if re.match(r'^.+ \d{1,2}:\d{2}:\d{2}\s*(AM|PM)$', name):
+    if re.match(r"^.+ \d{1,2}:\d{2}:\d{2}(\s*(AM|PM))?$", name, re.IGNORECASE):
         return True
     return False
 

routes/chat_routes.py

@@ -1266,7 +1266,7 @@ def setup_chat_routes(
                                 db_msg = (
                                     db.query(DBChatMessage)
                                     .filter(DBChatMessage.session_id == session_id, DBChatMessage.role == 'assistant')
-                                    .order_by(DBChatMessage.created_at.desc())
+                                    .order_by(DBChatMessage.timestamp.desc())
                                     .first()
                                 )
                                 if db_msg:

routes/contacts_routes.py

@@ -130,21 +130,28 @@ def _parse_vcards(text: str) -> List[Dict]:
         contact = {"name": "", "emails": [], "phones": [], "uid": ""}
         for line in block.split("\n"):
             line = line.strip()
-            if line.startswith("FN:") or line.startswith("FN;"):
-                contact["name"] = _vunesc(line.split(":", 1)[1]) if ":" in line else ""
-            elif line.startswith("EMAIL"):
+            # Strip an optional RFC 6350 group prefix (e.g. "item1.EMAIL;...")
+            # that Apple Contacts / iCloud / many CardDAV servers emit by
+            # default — without this the property-name checks below miss those
+            # lines and silently drop the email / phone. The group token only
+            # precedes the property name, so it is safe to strip for matching
+            # and value extraction, and a no-op for non-grouped lines.
+            name_part = re.sub(r"^[A-Za-z0-9-]+\.", "", line, count=1)
+            if name_part.startswith("FN:") or name_part.startswith("FN;"):
+                contact["name"] = _vunesc(name_part.split(":", 1)[1]) if ":" in name_part else ""
+            elif name_part.startswith("EMAIL"):
                 # Handle EMAIL:foo@bar OR EMAIL;TYPE=...:foo@bar OR EMAIL;PREF=1:foo@bar
-                if ":" in line:
-                    email_addr = _vunesc(line.split(":", 1)[1])
+                if ":" in name_part:
+                    email_addr = _vunesc(name_part.split(":", 1)[1])
                     if email_addr and email_addr not in contact["emails"]:
                         contact["emails"].append(email_addr)
-            elif line.startswith("TEL"):
-                if ":" in line:
-                    phone = _vunesc(line.split(":", 1)[1])
+            elif name_part.startswith("TEL"):
+                if ":" in name_part:
+                    phone = _vunesc(name_part.split(":", 1)[1])
                     if phone and phone not in contact["phones"]:
                         contact["phones"].append(phone)
-            elif line.startswith("UID:"):
-                contact["uid"] = _vunesc(line[4:])
+            elif name_part.startswith("UID:"):
+                contact["uid"] = _vunesc(name_part[4:])
         if contact["name"] or contact["emails"]:
             contacts.append(contact)
     return contacts
@@ -676,8 +683,8 @@ def setup_contacts_routes():
     @router.post("/add")
     async def add_contact(data: dict, _admin: str = Depends(require_admin)):
         """Add a new contact."""
-        name = data.get("name", "").strip()
-        email = data.get("email", "").strip()
+        name = (data.get("name") or "").strip()
+        email = (data.get("email") or "").strip()
         if not email:
             return {"success": False, "error": "Email required"}
         # Check if already exists

routes/cookbook_helpers.py

@@ -148,6 +148,21 @@ def _local_tooling_path_export(executable: str) -> str:
     return f'export PATH="{esc}:$PATH"'
 
 
+def _pip_install_no_cache(cmd: str) -> str:
+    """Add ``--no-cache-dir`` to a pip install command.
+
+    Cookbook dependency installs (vLLM, llama-cpp-python, …) build large wheels;
+    pip's default cache lives under ``$HOME/.cache/pip`` and these builds can fill
+    a small home filesystem with ``[Errno 28] No space left on device`` mid-build
+    (issue #1219), leaving the dependency "installed" but unusable (#1459).
+    Disabling the cache for these one-off installs keeps them off the home disk
+    (the maintainer's suggested ``PIP_CACHE_DIR=`` workaround, made the default).
+    Idempotent; leaves non-pip-install commands untouched."""
+    if not cmd or "pip install" not in cmd or "--no-cache-dir" in cmd:
+        return cmd
+    return cmd.replace("pip install", "pip install --no-cache-dir", 1)
+
+
 def _pip_install_attempt(pip_cmd: str) -> str:
     """Wrap a single pip install command so its exit status survives the
     fallback chain and its stderr is visible in the tmux log on failure.
@@ -179,8 +194,13 @@ def _pip_install_fallback_chain(package: str, *, python_cmd: str = "python3 -m p
     pip output appear in the Cookbook log on failure.
     """
     upgrade_flag = " -U" if upgrade else ""
-    base = _pip_install_attempt(f"{python_cmd} install -q{upgrade_flag} {package}")
-    user = _pip_install_attempt(f"{python_cmd} install --user --break-system-packages -q{upgrade_flag} {package}")
+    # Shell-quote the package spec: an extras spec like ``llama-cpp-python[server]``
+    # contains brackets that bash would treat as a glob, so it must be quoted
+    # before being embedded in the install command. Plain names (e.g.
+    # ``huggingface_hub``) are returned unchanged by ``shlex.quote``.
+    pkg = shlex.quote(package)
+    base = _pip_install_attempt(f"{python_cmd} install -q{upgrade_flag} {pkg}")
+    user = _pip_install_attempt(f"{python_cmd} install --user --break-system-packages -q{upgrade_flag} {pkg}")
     # Derive the python executable for the venv detection check.
     # Must use the same interpreter that pip belongs to; hardcoding
     # python3 breaks when pip lives in a venv that only has "python".
@@ -543,8 +563,29 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
     runner_lines.append('      echo "[odysseus] ROCm/HIP detected — building llama-server with HIP support..."')
     runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
     runner_lines.append('    elif command -v nvcc &>/dev/null; then')
-    runner_lines.append('      echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
-    runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    # nvcc alone is not sufficient — pip-installed CUDA wheels or incomplete
+    # tooling can expose nvcc without shipping libcudart, causing cmake to fail
+    # mid-build with "CUDA runtime library not found". Check cudart explicitly
+    # via a small helper so the guard stays readable.
+    runner_lines.append('      _odysseus_has_cudart() {')
+    runner_lines.append('        ldconfig -p 2>/dev/null | grep -q \'libcudart\\.so\' && return 0')
+    runner_lines.append('        local _cuh="${CUDA_HOME:-/usr/local/cuda}"')
+    runner_lines.append('        ls "$_cuh/lib64/libcudart.so"* &>/dev/null && return 0')
+    runner_lines.append('        ls "$_cuh/lib/libcudart.so"* &>/dev/null && return 0')
+    runner_lines.append('        ls /usr/local/cuda/lib64/libcudart.so* &>/dev/null && return 0')
+    runner_lines.append('        ls /usr/local/cuda/lib/libcudart.so* &>/dev/null && return 0')
+    runner_lines.append('        ls "${_cuh%/cuda_nvcc}/cuda_runtime/lib/libcudart.so"* &>/dev/null && return 0')
+    runner_lines.append('        return 1')
+    runner_lines.append('      }')
+    runner_lines.append('      if _odysseus_has_cudart; then')
+    runner_lines.append('        echo "[odysseus] CUDA nvcc + cudart found — building llama-server with CUDA (GPU) support..."')
+    runner_lines.append('        cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    runner_lines.append('      else')
+    runner_lines.append('        echo "[odysseus] WARNING: nvcc found but CUDA runtime (libcudart.so) is not visible — building llama-server for CPU only."')
+    runner_lines.append('        echo "[odysseus]   GPU inference will not be available for this llama.cpp build."')
+    runner_lines.append('        echo "[odysseus]   Ensure libcudart is installed (e.g. cuda-runtime package) and visible via ldconfig or CUDA_HOME."')
+    runner_lines.append('        cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
+    runner_lines.append('      fi')
     runner_lines.append('    else')
     runner_lines.append('      echo "[odysseus] WARNING: no HIP/CUDA toolchain found — building llama-server for CPU only."')
     runner_lines.append('      echo "[odysseus]   GPU inference will not be available for this llama.cpp build."')
@@ -552,6 +593,27 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
     runner_lines.append('      cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
     runner_lines.append('    fi')
 
+
+def _llama_cpp_rebuild_cmd() -> str:
+    """Shell command that clears the Cookbook-managed llama.cpp build.
+
+    Removes the cached ``llama-server`` symlink and the ``~/llama.cpp/build``
+    directory so the next llama.cpp serve recompiles from source, picking up a
+    CUDA or HIP toolchain if one is now available. The serve bootstrap only
+    builds when ``llama-server`` is missing from PATH, so without this an
+    existing CPU-only build is reused forever. It deliberately installs and
+    downloads nothing; the rebuild itself happens on the next serve.
+    """
+    return (
+        'mkdir -p "$HOME/bin" && '
+        'rm -f "$HOME/bin/llama-server" && '
+        'rm -rf "$HOME/llama.cpp/build" && '
+        'echo "[odysseus] Cleared the cached llama.cpp build. '
+        'Re-launch the serve task to rebuild llama-server from source '
+        '(CUDA or HIP will be used if a toolchain is now available)."'
+    )
+
+
 class ModelDownloadRequest(BaseModel):
     repo_id: str
     include: str | None = None  # glob pattern e.g. "*Q4_K_M*"

routes/cookbook_routes.py

@@ -38,7 +38,7 @@ from routes.cookbook_helpers import (
     _ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase,
     _safe_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
     _append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script,
-    _ollama_bind_from_cmd, _pip_install_fallback_chain, _venv_safe_local_pip_install_cmd,
+    _ollama_bind_from_cmd, _pip_install_fallback_chain, _pip_install_no_cache, _venv_safe_local_pip_install_cmd,
     ModelDownloadRequest, ServeRequest,
 )
 
@@ -809,6 +809,80 @@ def setup_cookbook_routes() -> APIRouter:
         finally:
             db.close()
 
+    def _auto_register_llm_endpoint(req: ServeRequest, remote: str | None) -> str | None:
+        """Register a freshly-served LLM as a model endpoint so it appears in the
+        model picker without a manual /setup step — the text-model sibling of
+        _auto_register_image_endpoint.
+
+        Cookbook serve commands launch an OpenAI-compatible server (llama.cpp's
+        llama-server, vLLM, SGLang, or Ollama) on a known port. We point an
+        endpoint at that server's /v1; the picker auto-discovers the model id by
+        probing /v1/models and dims the endpoint until the server is reachable,
+        so registering immediately (before the server finishes loading) is safe.
+        """
+        import re
+        from core.database import SessionLocal, ModelEndpoint
+
+        # Port: an explicit --port wins. Otherwise fall back by backend — Ollama
+        # is the only server in our generated commands that omits --port.
+        port_match = re.search(r'--port\s+(\d+)', req.cmd)
+        if port_match:
+            port = int(port_match.group(1))
+        elif "ollama" in req.cmd:
+            port = 11434
+        else:
+            port = 8080  # llama.cpp's llama-server default — the Apple Silicon path
+
+        # Determine host (mirrors the image path: SSH alias for remote serves).
+        if remote:
+            host = remote.split("@")[-1] if "@" in remote else remote
+        else:
+            host = "localhost"
+
+        base_url = f"http://{host}:{port}/v1"
+
+        short_name = req.repo_id.split("/")[-1] if "/" in req.repo_id else req.repo_id
+        display_name = short_name or "Local model"
+
+        # If the serve command opts models into OpenAI tool-calling, record it so
+        # agent_loop trusts emitted tool_calls instead of the name heuristic.
+        supports_tools = True if "--enable-auto-tool-choice" in req.cmd else None
+
+        db = SessionLocal()
+        try:
+            # Reuse an endpoint already pointed at this URL instead of duplicating.
+            existing = db.query(ModelEndpoint).filter(ModelEndpoint.base_url == base_url).first()
+            if existing:
+                existing.is_enabled = True
+                existing.model_type = "llm"
+                existing.name = display_name
+                if supports_tools is not None:
+                    existing.supports_tools = supports_tools
+                db.commit()
+                logger.info(f"Updated existing local model endpoint: {base_url}")
+                return existing.id
+
+            ep_id = f"local-{uuid.uuid4().hex[:8]}"
+            ep = ModelEndpoint(
+                id=ep_id,
+                name=display_name,
+                base_url=base_url,
+                api_key=None,
+                is_enabled=True,
+                model_type="llm",
+                supports_tools=supports_tools,
+            )
+            db.add(ep)
+            db.commit()
+            logger.info(f"Auto-registered local model endpoint: {display_name} @ {base_url}")
+            return ep_id
+        except Exception as e:
+            logger.error(f"Failed to auto-register local model endpoint: {e}")
+            db.rollback()
+            return None
+        finally:
+            db.close()
+
     @router.post("/api/model/serve")
     async def model_serve(request: Request, req: ServeRequest):
         """Launch a model server in a tmux session (or PowerShell background process on Windows).
@@ -841,6 +915,10 @@ def setup_cookbook_routes() -> APIRouter:
         )
         is_pip_install = bool(req.cmd and "pip install" in req.cmd)
         if is_pip_install:
+            # Keep big dependency wheel builds (vLLM, …) off the home filesystem's
+            # pip cache so they don't fail mid-build with "No space left" (#1219)
+            # and leave the dep installed-but-unusable (#1459).
+            req.cmd = _pip_install_no_cache(req.cmd)
             # PEP-508-style package spec — letters, digits, `.-_` for the
             # name; `[` `]` for extras; `<>=!~,` for version specifiers.
             # v2 review HIGH-14: tightened from the previous regex which
@@ -961,7 +1039,7 @@ def setup_cookbook_routes() -> APIRouter:
                 runner_lines.append('  if ! python3 -c "import llama_cpp" 2>/dev/null; then')
                 runner_lines.append('    pkg install -y cmake 2>/dev/null')
                 runner_lines.append('    pip install numpy diskcache jinja2 2>/dev/null')
-                runner_lines.append('    CMAKE_ARGS="-DGGML_BLAS=OFF -DGGML_LLAMAFILE=OFF" pip install llama-cpp-python --no-build-isolation --no-cache-dir 2>&1 || true')
+                runner_lines.append('    CMAKE_ARGS="-DGGML_BLAS=OFF -DGGML_LLAMAFILE=OFF" pip install \'llama-cpp-python[server]\' --no-build-isolation --no-cache-dir 2>&1 || true')
                 runner_lines.append('  fi')
                 runner_lines.append('elif ! command -v llama-server &>/dev/null; then')
                 runner_lines.append('  echo "Native llama-server not found — building from source (one-time, may take a few minutes)..."')
@@ -988,7 +1066,7 @@ def setup_cookbook_routes() -> APIRouter:
                 runner_lines.append('  # If the native build failed, fall back to the Python bindings.')
                 runner_lines.append('  if ! command -v llama-server &>/dev/null && ! python3 -c "import llama_cpp" 2>/dev/null; then')
                 runner_lines.append('    echo "llama-server build failed — installing Python bindings as fallback..."')
-                runner_lines.append(f"    {_pip_install_fallback_chain('llama-cpp-python', python_cmd='pip')} || true")
+                runner_lines.append(f"    {_pip_install_fallback_chain('llama-cpp-python[server]', python_cmd='pip')} || true")
                 runner_lines.append('  fi')
                 runner_lines.append('  if ! command -v llama-server &>/dev/null && ! python3 -c "import llama_cpp" 2>/dev/null; then')
                 runner_lines.append('    echo "ERROR: llama.cpp serving is not available after install/build attempts."')
@@ -1148,11 +1226,16 @@ def setup_cookbook_routes() -> APIRouter:
                 stderr = (await proc.stderr.read()).decode(errors="replace")
                 return {"ok": False, "error": stderr, "session_id": session_id}
 
-        # Auto-register as model endpoint if serving a diffusion model
+        # Auto-register a model endpoint so the served model shows up in the model
+        # picker with no manual /setup step. Diffusion models get an image
+        # endpoint; any other real model serve (i.e. not a pip-install task) gets
+        # a local LLM endpoint pointed at its /v1.
         endpoint_id = None
         is_diffusion = "diffusion_server.py" in req.cmd
         if is_diffusion:
             endpoint_id = _auto_register_image_endpoint(req, remote)
+        elif not is_pip_install:
+            endpoint_id = _auto_register_llm_endpoint(req, remote)
 
         # Log to assistant
         try:

routes/document_helpers.py

@@ -152,7 +152,7 @@ def _resolve_user_upload_path(
         owner=owner,
         auth_manager=auth_manager,
     )
-    if not resolved:
+    if not isinstance(resolved, dict) or not resolved:
         return None
     path = resolved.get("path")
     upload_dir = getattr(upload_handler, "upload_dir", None)
@@ -203,6 +203,8 @@ def _assert_pdf_marker_upload_owned(
 def _derive_title(content: str) -> str:
     """Derive a title from document content."""
     import re
+    if not isinstance(content, str):
+        return "Untitled"
     text = content.strip()
     if not text:
         return "Untitled"

routes/document_routes.py

@@ -15,6 +15,21 @@ from src.auth_helpers import get_current_user
 logger = logging.getLogger(__name__)
 
 
+def _aggregate_language_facets(lang_rows):
+    """Sum document counts per display language for the library facet.
+
+    NULL-language and explicit "text" rows share the "text" bucket (the
+    language filter treats them as one), so they must be ADDED. The old dict
+    comprehension keyed both to "text", silently overwriting one group and
+    undercounting the facet versus what the filter actually returns.
+    """
+    out = {}
+    for lang, cnt in lang_rows:
+        key = lang or "text"
+        out[key] = out.get(key, 0) + cnt
+    return out
+
+
 
 from routes.document_helpers import (
     DocumentCreate, DocumentUpdate, DocumentPatch,
@@ -258,7 +273,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
             )
             lang_q = _owner_session_filter(lang_q, user)
             lang_rows = lang_q.group_by(Document.language).all()
-            languages = {lang or "text": cnt for lang, cnt in lang_rows}
+            languages = _aggregate_language_facets(lang_rows)
 
             # Session count (owner-filtered)
             sc_q = (

routes/editor_draft_routes.py

@@ -67,6 +67,14 @@ def _summary(d: EditorDraft) -> Dict[str, Any]:
     }
 
 
+def _load_payload(raw: Optional[str]) -> Dict[str, Any]:
+    try:
+        payload = json.loads(raw) if raw else {}
+    except Exception:
+        return {}
+    return payload if isinstance(payload, dict) else {}
+
+
 def setup_editor_draft_routes() -> APIRouter:
     router = APIRouter(tags=["editor-drafts"])
 
@@ -93,13 +101,9 @@ def setup_editor_draft_routes() -> APIRouter:
             ).first()
             if not d or not _owns(d, user):
                 raise HTTPException(404, "Draft not found")
-            try:
-                payload = json.loads(d.payload) if d.payload else {}
-            except Exception:
-                payload = {}
             return {
                 **_summary(d),
-                "payload": payload,
+                "payload": _load_payload(d.payload),
             }
         finally:
             db.close()

routes/email_helpers.py

@@ -751,7 +751,13 @@ def _decode_header(raw):
     decoded = []
     for data, charset in parts:
         if isinstance(data, bytes):
-            decoded.append(data.decode(charset or "utf-8", errors="replace"))
+            try:
+                decoded.append(data.decode(charset or "utf-8", errors="replace"))
+            except (LookupError, ValueError):
+                # Unknown/invalid MIME charset (e.g. a malformed or spam header
+                # like =?x-unknown-charset?B?...?=). errors="replace" only covers
+                # byte-decode errors, not codec lookup, so fall back to utf-8.
+                decoded.append(data.decode("utf-8", errors="replace"))
         else:
             decoded.append(data)
     return " ".join(decoded)

routes/email_pollers.py

@@ -99,6 +99,36 @@ async def _run_auto_summarize_once(do_summary: bool = True, do_reply: bool = Tru
         _save_settings(s2)
 
 
+def _latest_inbox_fallback_uids(conn, reconnect):
+    """Latest INBOX UIDs via ``SEARCH ALL``, with a poisoned-socket guard (#1613).
+
+    On a large Gmail mailbox the fallback ``SEARCH ALL`` can time out mid-reply,
+    leaving its enormous ``* SEARCH <uids…>`` line unread on the socket. The next
+    command (the downstream re-select / EXAMINE) then reads those leftover bytes
+    and fails with ``EXAMINE => unexpected response: b'325188 …'``. Reconnecting
+    on failure guarantees the downstream command starts from a clean socket.
+
+    Returns ``(uids, conn)`` — ``conn`` is the live connection to keep using: the
+    same one on success, a fresh one (via ``reconnect()``) if we had to recover.
+    """
+    try:
+        conn.select("INBOX", readonly=True)
+        status, data = conn.uid("SEARCH", None, "ALL")
+        uids = []
+        if status == "OK" and data and data[0]:
+            for u in reversed(data[0].split()[-8:]):
+                uids.append(("INBOX", u))
+            logger.info("Email task SINCE scan found no messages; fell back to latest INBOX messages")
+        return uids, conn
+    except Exception as _e:
+        logger.warning(f"Latest-INBOX fallback scan failed: {_e}")
+        try:
+            conn.logout()
+        except Exception:
+            pass
+        return [], reconnect()
+
+
 async def _auto_summarize_pass(days_back: int = 1, account_id: str | None = None, progress_cb=None) -> str:
     """Single pass of the auto-summarize/reply scan.
 
@@ -201,16 +231,12 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
         # the latest visible inbox messages so Clear cache -> Run again can
         # actually repopulate AI reply/summary/tag caches.
         if not uid_list:
-            try:
-                conn.select("INBOX", readonly=True)
-                status, data = conn.uid("SEARCH", None, "ALL")
-                if status == "OK" and data and data[0]:
-                    for u in reversed(data[0].split()[-8:]):
-                        uid_list.append(("INBOX", u))
-                    logger.info("Email task SINCE scan found no messages; fell back to latest INBOX messages")
-            except Exception as _e:
-                logger.warning(f"Latest-INBOX fallback scan failed: {_e}")
-        # Re-select INBOX as default for downstream code
+            _fb_uids, conn = _latest_inbox_fallback_uids(
+                conn, lambda: _imap_connect(account_id, owner=account_owner)
+            )
+            uid_list.extend(_fb_uids)
+        # Re-select INBOX as default for downstream code (on a clean socket even
+        # if the SEARCH ALL fallback above failed — see #1613).
         conn.select("INBOX", readonly=True)
         if not uid_list:
             return "No recent emails"

routes/email_routes.py

@@ -322,6 +322,20 @@ def _apply_odysseus_headers(msg, kind: str | None = None, ref_id: str | None = N
         msg["X-Odysseus-Ref"] = re.sub(r"[^A-Za-z0-9_.:-]", "-", ref_id)[:128]
 
 
+def _envelope_recipients(*fields: str) -> list:
+    """Extract bare SMTP envelope addresses from one or more To/Cc/Bcc header
+    strings. A naive `field.split(",")` corrupts display names that contain a
+    comma (e.g. `"Smith, John" <john@corp.com>`, the canonical Outlook form):
+    it splits into `"Smith` and `John" <john@corp.com>`, breaking delivery.
+    email.utils.getaddresses parses the address grammar correctly."""
+    out = []
+    for _name, addr in email.utils.getaddresses([f for f in fields if f]):
+        addr = (addr or "").strip()
+        if addr:
+            out.append(addr)
+    return out
+
+
 def _md_to_email_html(text: str) -> str:
     """Render the compose markdown body to a SAFE HTML fragment for the email's
     text/html part. Everything is HTML-escaped FIRST (so a pasted <script> /
@@ -1943,11 +1957,7 @@ def setup_email_routes():
             outer.attach(body_container)
             _attach_compose_uploads(outer, attachments)
 
-        recipients = [r.strip() for r in to.split(",") if r.strip()]
-        if cc:
-            recipients.extend([r.strip() for r in cc.split(",") if r.strip()])
-        if bcc:
-            recipients.extend([r.strip() for r in bcc.split(",") if r.strip()])
+        recipients = _envelope_recipients(to, cc, bcc)
 
         _send_smtp_message(cfg, cfg["from_address"], recipients, outer.as_string())
 
@@ -1979,6 +1989,15 @@ def setup_email_routes():
             # minute doesn't trip the past-time guard.
             if parsed_at < now_utc:
                 return {"success": False, "error": "send_at must be in the future"}
+            # Normalize to naive UTC before storing: the poller selects due
+            # rows with a lexicographic string compare against a naive
+            # datetime.utcnow().isoformat(), so storing the raw client string
+            # makes "+02:00" schedules fire hours late, negative offsets fire
+            # hours early, and a "Z" suffix compares after the fractional
+            # seconds of the poller timestamp.
+            if parsed_at.tzinfo:
+                parsed_at = parsed_at.astimezone(_tz.utc).replace(tzinfo=None)
+            send_at = parsed_at.isoformat()
 
             sid = _uuid.uuid4().hex[:16]
             conn = sqlite3.connect(SCHEDULED_DB)
@@ -2152,12 +2171,9 @@ def setup_email_routes():
             outer.attach(body_container)
             _attach_compose_uploads(outer, req.attachments)
 
-        # Build recipient list
-        recipients = [r.strip() for r in req.to.split(",") if r.strip()]
-        if req.cc:
-            recipients.extend([r.strip() for r in req.cc.split(",") if r.strip()])
-        if req.bcc:
-            recipients.extend([r.strip() for r in req.bcc.split(",") if r.strip()])
+        # Build recipient list (parse the address grammar so display names with
+        # commas don't get split into broken envelope addresses)
+        recipients = _envelope_recipients(req.to, req.cc, req.bcc)
 
         # Serialize what the background task needs so the request object can be GC'd
         outer_bytes = outer.as_bytes()

routes/embedding_routes.py

@@ -86,7 +86,8 @@ def _load_custom_endpoint() -> dict:
     """Load the saved custom embedding endpoint, if any."""
     try:
         if os.path.exists(_ENDPOINT_FILE):
-            return json.loads(Path(_ENDPOINT_FILE).read_text(encoding="utf-8"))
+            data = json.loads(Path(_ENDPOINT_FILE).read_text(encoding="utf-8"))
+            return data if isinstance(data, dict) else {}
     except Exception:
         pass
     return {}

routes/font_routes.py

@@ -5,6 +5,15 @@ from fastapi import APIRouter
 
 CUSTOM_FONTS_DIR = os.path.join("static", "fonts", "custom")
 FONT_EXTENSIONS = {".ttf", ".otf", ".woff", ".woff2"}
+FAMILY_SUFFIX_WORDS = ("Display", "Rounded", "Serif", "Sans", "Mono", "Code", "Text")
+
+
+def _split_family_token(token):
+    """Split common compact font-family suffixes without breaking brand names."""
+    for suffix in FAMILY_SUFFIX_WORDS:
+        if token.endswith(suffix) and len(token) > len(suffix):
+            return f"{token[:-len(suffix)]} {suffix}"
+    return re.sub(r'(?<=[a-z])(?=[A-Z])', ' ', token)
 
 
 def _derive_family(filename):
@@ -15,10 +24,9 @@ def _derive_family(filename):
         r'[-_ ]?(Thin|ExtraLight|UltraLight|Light|Regular|Medium|SemiBold|DemiBold|Bold|ExtraBold|UltraBold|Black|Heavy|Italic|Oblique|Variable|VF)$',
         '', name, flags=re.IGNORECASE
     )
-    # Insert spaces before uppercase runs: "JetBrainsMono" → "Jet Brains Mono"
-    name = re.sub(r'(?<=[a-z])(?=[A-Z])', ' ', name)
     # Replace dashes/underscores with spaces
     name = re.sub(r'[-_]+', ' ', name).strip()
+    name = " ".join(_split_family_token(part) for part in name.split())
     return name or filename
 
 

routes/gallery_helpers.py

@@ -32,10 +32,21 @@ def _extract_exif(content: bytes) -> dict:
         from PIL import Image
         from io import BytesIO
         img = Image.open(BytesIO(content))
+        # Read the raw EXIF before any transpose: exif_transpose strips the
+        # orientation tag and with it the parsed EXIF view.
+        exif = img._getexif() if hasattr(img, '_getexif') else None
+
+        # Record DISPLAY dimensions (EXIF-rotated), matching upload_handler.
+        # A phone photo with Orientation 6/8 is stored landscape but shown
+        # portrait, so the raw width/height swap the aspect ratio.
+        try:
+            from PIL import ImageOps
+            img = ImageOps.exif_transpose(img) or img
+        except Exception:
+            pass
         result["width"] = img.width
         result["height"] = img.height
 
-        exif = img._getexif() if hasattr(img, '_getexif') else None
         if not exif:
             return result
 
@@ -110,9 +121,17 @@ def _image_to_dict(img: GalleryImage, session_name: str = None) -> Dict[str, Any
 
 
 def _owner_filter(q, user):
-    """Apply owner filtering to a gallery query."""
+    """Apply owner filtering to a gallery query.
+
+    When auth is disabled (single-user mode) get_current_user returns None
+    and there is no per-user scoping. The main library list and stats already
+    treat None as "show everything" (`if user is not None`), so this helper
+    must too — otherwise the tag/model filter sidebars come back empty and the
+    tag-cleanup endpoints (clear-user-tags, clear-ai-tags, dedupe-tags)
+    silently affect zero rows in the most common self-hosted deployment.
+    """
     if user is None:
-        return q.filter(False)
+        return q
     return q.filter(GalleryImage.owner == user)
 
 

routes/gallery_routes.py

@@ -923,6 +923,16 @@ def setup_gallery_routes() -> APIRouter:
         body = await request.json()
         # Use endpoint from request body (editor dropdown) or fall back to DB lookup
         base = (body.pop("_endpoint", "") or "").rstrip("/")
+        # SSRF hardening: validate a client-supplied endpoint before any
+        # outbound request (mirrors routes/embedding_routes.py).
+        if base:
+            from src.url_safety import check_outbound_url
+            ok, reason = check_outbound_url(
+                base,
+                block_private=os.getenv("IMAGE_BLOCK_PRIVATE_IPS", "false").lower() == "true",
+            )
+            if not ok:
+                raise HTTPException(400, f"Rejected endpoint URL: {reason}")
         chosen_model = (body.pop("_model", "") or "").strip()
         api_key = None
         if not base:
@@ -1115,6 +1125,18 @@ def setup_gallery_routes() -> APIRouter:
             raise HTTPException(400, "No image provided")
 
         endpoint = (body.get("_endpoint") or "").rstrip("/")
+        # SSRF hardening: a client-supplied endpoint is fetched server-side
+        # below, so validate it first (mirrors routes/embedding_routes.py).
+        # Local-first means loopback/LAN is allowed by default; the cloud
+        # metadata range and non-HTTP(S) schemes are always rejected.
+        if endpoint:
+            from src.url_safety import check_outbound_url
+            ok, reason = check_outbound_url(
+                endpoint,
+                block_private=os.getenv("IMAGE_BLOCK_PRIVATE_IPS", "false").lower() == "true",
+            )
+            if not ok:
+                raise HTTPException(400, f"Rejected endpoint URL: {reason}")
         model = (body.get("_model") or "").strip()
 
         base = endpoint

routes/history_routes.py

@@ -58,7 +58,7 @@ def setup_history_routes(session_manager) -> APIRouter:
                     .all()
                 )
                 import json as _json
-                history_dict = []
+                db_history = []
                 for m in db_messages:
                     entry = {"role": m.role, "content": m.content}
                     meta = {}
@@ -71,12 +71,19 @@ def setup_history_routes(session_manager) -> APIRouter:
                         meta["timestamp"] = m.timestamp.isoformat() + "Z"
                     if meta:
                         entry["metadata"] = meta
-                    history_dict.append(entry)
-                if history_dict:
+                    db_history.append(entry)
+                if db_history:
+                    # Rebuild in-memory history from the full set so hidden
+                    # messages (e.g. compaction summaries) are kept for AI context.
                     session.history = [
                         ChatMessage(role=m["role"], content=m["content"], metadata=m.get("metadata"))
-                        for m in history_dict
+                        for m in db_history
                     ]
+                # Response excludes hidden messages, matching the in-memory path.
+                history_dict = [
+                    m for m in db_history
+                    if not (m.get("metadata") or {}).get("hidden")
+                ]
             except Exception as e:
                 logger.error(f"DB fallback failed for {session_id}: {e}")
             finally:
@@ -265,7 +272,7 @@ def setup_history_routes(session_manager) -> APIRouter:
                 db_messages = (
                     db.query(DbChatMessage)
                     .filter(DbChatMessage.session_id == session_id, DbChatMessage.role == 'assistant')
-                    .order_by(DbChatMessage.created_at.desc())
+                    .order_by(DbChatMessage.timestamp.desc())
                     .first()
                 )
                 if db_messages:
@@ -320,7 +327,7 @@ def setup_history_routes(session_manager) -> APIRouter:
                 db_msg = (
                     db.query(DbChatMessage)
                     .filter(DbChatMessage.session_id == session_id, DbChatMessage.role == 'assistant')
-                    .order_by(DbChatMessage.created_at.desc())
+                    .order_by(DbChatMessage.timestamp.desc())
                     .first()
                 )
                 if db_msg:
@@ -401,7 +408,7 @@ def setup_history_routes(session_manager) -> APIRouter:
                 db_messages = (
                     db.query(DbChatMessage)
                     .filter(DbChatMessage.session_id == session_id)
-                    .order_by(DbChatMessage.created_at)
+                    .order_by(DbChatMessage.timestamp)
                     .all()
                 )
                 # Find last two assistant messages in DB

routes/personal_routes.py

@@ -69,9 +69,12 @@ def setup_personal_routes(personal_docs_manager, rag_manager, rag_available):
         if not directory:
             raise HTTPException(400, "Directory path is required")
 
-        base_abs = os.path.abspath(PERSONAL_DIR)
+        # realpath (not abspath) so a symlink inside PERSONAL_DIR that points
+        # outside it is resolved before the commonpath confinement check below;
+        # abspath only normalises `..` and would let such a symlink escape.
+        base_abs = os.path.realpath(PERSONAL_DIR)
         candidate = directory if os.path.isabs(directory) else os.path.join(base_abs, directory)
-        resolved = os.path.abspath(candidate)
+        resolved = os.path.realpath(candidate)
         try:
             in_base = os.path.commonpath([resolved, base_abs]) == base_abs
         except ValueError:

routes/prefs_routes.py

@@ -12,7 +12,8 @@ def _load():
     """Load the raw prefs file (internal use only)."""
     try:
         with open(PREFS_FILE, "r", encoding="utf-8") as f:
-            return json.load(f)
+            data = json.load(f)
+            return data if isinstance(data, dict) else {}
     except (FileNotFoundError, json.JSONDecodeError):
         return {}
 
@@ -40,7 +41,18 @@ def _save_for_user(user: Optional[str], prefs: dict):
     """Save preferences for a specific user."""
     all_prefs = _load()
     if user is None:
-        # Auth disabled — save flat
+        # Auth disabled. If the store is already multi-user (e.g. auth was
+        # turned off on a deployment that previously ran multi-user), writing
+        # `prefs` flat would overwrite the whole `_users` map and destroy every
+        # other user's preferences. Instead write back into the same (first)
+        # slot _load_for_user(None) reads from, preserving the others.
+        if "_users" in all_prefs:
+            users = all_prefs["_users"]
+            first_key = next(iter(users), None)
+            if first_key is not None:
+                users[first_key] = prefs
+                _save(all_prefs)
+                return
         _save(prefs)
         return
     if "_users" not in all_prefs:

routes/session_routes.py

@@ -618,7 +618,12 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
                 safe_search = search.replace('%', r'\%').replace('_', r'\_')
                 q = q.filter(DbSession.name.ilike(f"%{safe_search}%", escape='\\'))
             if model:
-                q = q.filter(DbSession.model.ilike(f"%{model}"))
+                # Contains match (mirrors the name filter above). The old
+                # f"%{model}" was a SUFFIX-only match, so filtering by "gpt-4"
+                # dropped "gpt-4o" and over-matched on shared suffixes; it also
+                # left LIKE wildcards in the user value unescaped.
+                safe_model = model.replace('%', r'\%').replace('_', r'\_')
+                q = q.filter(DbSession.model.ilike(f"%{safe_model}%", escape='\\'))
             total = q.count()
             sort_map = {
                 "recent": DbSession.updated_at.desc(),

routes/shell_routes.py

@@ -1058,4 +1058,39 @@ def setup_shell_routes() -> APIRouter:
             return {"ok": True, "output": stdout.decode()[-200:]}
         return {"ok": False, "error": stderr.decode()[-300:]}
 
+    @router.post("/api/cookbook/rebuild-engine")
+    async def rebuild_engine(request: Request):
+        """Clear the cached llama.cpp build so the next serve recompiles.
+
+        Admin only — this removes the Cookbook-managed ``~/bin/llama-server``
+        symlink and ``~/llama.cpp/build`` directory, locally or on the selected
+        remote server. It installs and downloads nothing; the next llama.cpp
+        serve rebuilds from source and picks up CUDA/HIP if a toolchain is now
+        present. This is the missing "force a fresh GPU build" lever for hosts
+        stuck on a CPU-only llama-server.
+        """
+        _require_admin(request)
+        from routes.cookbook_helpers import _llama_cpp_rebuild_cmd
+        body = await request.json()
+        engine = str(body.get("engine") or "llamacpp").strip()
+        if engine != "llamacpp":
+            return {"ok": False, "error": f"Unsupported engine: {engine}"}
+        host = str(body.get("remote_host") or "").strip()
+        ssh_port = body.get("ssh_port")
+        cmd = _llama_cpp_rebuild_cmd()
+        try:
+            argv = (_ssh_base_argv(host, ssh_port) + [cmd]) if host else ["bash", "-lc", cmd]
+        except ValueError as e:
+            raise HTTPException(400, str(e))
+        try:
+            proc = await asyncio.create_subprocess_exec(
+                *argv, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
+            )
+            out, err = await asyncio.wait_for(proc.communicate(), timeout=30)
+        except asyncio.TimeoutError:
+            return {"ok": False, "error": "Rebuild-engine command timed out."}
+        if proc.returncode == 0:
+            return {"ok": True, "output": out.decode("utf-8", errors="replace")[-400:]}
+        return {"ok": False, "error": err.decode("utf-8", errors="replace")[-400:]}
+
     return router

routes/upload_routes.py

@@ -111,7 +111,7 @@ def setup_upload_routes(upload_handler):
         if os.path.exists(uploads_db):
             with open(uploads_db, encoding="utf-8") as f:
                 db = json.load(f)
-            info = next((fi for fi in db.values() if fi["id"] == file_id), None)
+            info = next((fi for fi in db.values() if fi.get("id") == file_id), None)
             if info:
                 original_name = info.get("name", file_id)
         auth_mgr = getattr(request.app.state, "auth_manager", None)
@@ -159,7 +159,7 @@ def setup_upload_routes(upload_handler):
         if os.path.exists(uploads_db):
             with open(uploads_db, encoding="utf-8") as f:
                 db = json.load(f)
-            info = next((fi for fi in db.values() if fi["id"] == file_id), None)
+            info = next((fi for fi in db.values() if fi.get("id") == file_id), None)
         return info
 
     def _vision_cache_path(file_id: str) -> str:

scripts/claim_ownerless.py

@@ -13,6 +13,18 @@ import json
 
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
+
+def claim_json_entries(entries, owner):
+    count = 0
+    for entry in entries:
+        if not isinstance(entry, dict):
+            continue
+        if not entry.get("owner"):
+            entry["owner"] = owner
+            count += 1
+    return count
+
+
 def main():
     if len(sys.argv) < 2:
         print("Usage: python scripts/claim_ownerless.py <username>")
@@ -31,11 +43,7 @@ def main():
             continue
         with open(path, "r", encoding="utf-8") as f:
             entries = json.load(f)
-        count = 0
-        for e in entries:
-            if not e.get("owner"):
-                e["owner"] = owner
-                count += 1
+        count = claim_json_entries(entries, owner)
         if count:
             with open(path, "w", encoding="utf-8") as f:
                 json.dump(entries, f, ensure_ascii=False, indent=2)

scripts/migrate_faiss_to_chroma.py

@@ -26,6 +26,39 @@ logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(mess
 logger = logging.getLogger("migrate")
 
 
+def _load_json(path, default):
+    try:
+        with open(path, encoding="utf-8") as f:
+            return json.load(f)
+    except (OSError, json.JSONDecodeError):
+        return default
+
+
+def _memory_map(rows):
+    memories = {}
+    if not isinstance(rows, list):
+        return memories
+    for row in rows:
+        if not isinstance(row, dict):
+            continue
+        memory_id = row.get("id", "")
+        if memory_id:
+            memories[memory_id] = row
+    return memories
+
+
+def _rag_docstore(data):
+    if not isinstance(data, dict):
+        return [], [], []
+    ids = data.get("ids", [])
+    documents = data.get("documents", [])
+    metadatas = data.get("metadatas", [])
+    if not isinstance(ids, list) or not isinstance(documents, list) or not isinstance(metadatas, list):
+        return [], [], []
+    count = min(len(ids), len(documents), len(metadatas))
+    return ids[:count], documents[:count], metadatas[:count]
+
+
 def migrate_memories():
     """Migrate memory vectors from FAISS to ChromaDB."""
     from src.chroma_client import get_chroma_client
@@ -39,7 +72,9 @@ def migrate_memories():
         logger.info("No memory FAISS index found, skipping memory migration")
         return
 
-    ids = json.loads(open(ids_path, encoding="utf-8").read())
+    ids = _load_json(ids_path, [])
+    if not isinstance(ids, list):
+        ids = []
     if not ids:
         logger.info("Memory FAISS index is empty, skipping")
         return
@@ -47,8 +82,7 @@ def migrate_memories():
     # Load memory texts
     memories = {}
     if os.path.exists(memory_path):
-        for mem in json.loads(open(memory_path, encoding="utf-8").read()):
-            memories[mem.get("id", "")] = mem
+        memories = _memory_map(_load_json(memory_path, []))
 
     embed = get_embedding_client()
     if not embed:
@@ -97,10 +131,7 @@ def migrate_rag():
         logger.info("No RAG DocStore found, skipping RAG migration")
         return
 
-    data = json.loads(open(docs_path, encoding="utf-8").read())
-    ids = data.get("ids", [])
-    documents = data.get("documents", [])
-    metadatas = data.get("metadatas", [])
+    ids, documents, metadatas = _rag_docstore(_load_json(docs_path, {}))
 
     if not ids:
         logger.info("RAG DocStore is empty, skipping")

scripts/odysseus-calendar

@@ -69,11 +69,17 @@ def _parse_dt(s: str) -> datetime:
     return datetime.fromisoformat(s.replace("Z", "+00:00"))
 
 
+def _calendar_name(ev: "CalendarEvent") -> str:
+    cal = getattr(ev, "calendar", None)
+    name = getattr(cal, "name", "") if cal else ""
+    return name if isinstance(name, str) else ""
+
+
 def _serialize_event(ev: "CalendarEvent") -> dict:
     return {
         "uid": ev.uid,
         "calendar_id": ev.calendar_id,
-        "calendar_name": ev.calendar.name if ev.calendar else "",
+        "calendar_name": _calendar_name(ev),
         "summary": ev.summary,
         "description": ev.description or "",
         "location": ev.location or "",

scripts/odysseus-contacts

@@ -60,13 +60,17 @@ def fail(msg: str, code: int = 1) -> None:
     sys.exit(code)
 
 
+def _contact_rows(contacts):
+    return [c for c in contacts or [] if isinstance(c, dict)]
+
+
 # ─── list ────────────────────────────────────────────────────────────
 
 def cmd_list(args) -> None:
     cfg = _get_carddav_config()
     if not cfg["url"]:
         fail("CardDAV not configured. Set carddav_url/username/password in the web UI.")
-    contacts = _fetch_contacts(force=args.refresh)
+    contacts = _contact_rows(_fetch_contacts(force=args.refresh))
     emit(contacts, args)
 
 
@@ -77,7 +81,7 @@ def cmd_search(args) -> None:
     if not cfg["url"]:
         fail("CardDAV not configured.")
     q = args.query.lower()
-    contacts = _fetch_contacts()
+    contacts = _contact_rows(_fetch_contacts())
     matches = [
         c for c in contacts
         if q in (c.get("name") or "").lower() or q in (c.get("email") or "").lower()

scripts/odysseus-cookbook

@@ -411,6 +411,8 @@ def cmd_state_set(args) -> None:
         obj = json.loads(data)
     except json.JSONDecodeError as e:
         fail(f"invalid JSON on stdin: {e}")
+    if not isinstance(obj, dict):
+        fail("invalid cookbook state: expected a JSON object")
     _STATE_PATH.parent.mkdir(parents=True, exist_ok=True)
     # Backup the existing state — undo button if a bad pipe clobbers it.
     if _STATE_PATH.exists():

scripts/odysseus-docs

@@ -33,6 +33,10 @@ except ModuleNotFoundError as e:
     sys.exit(2)
 
 
+def _text_len(value) -> int:
+    return len(value) if isinstance(value, str) else 0
+
+
 def _serialize(d: "Document", include_content: bool = False) -> dict:
     out = {
         "id": d.id,
@@ -42,7 +46,7 @@ def _serialize(d: "Document", include_content: bool = False) -> dict:
         "version_count": d.version_count or 1,
         "is_active": bool(d.is_active),
         "tidy_verdict": d.tidy_verdict or "",
-        "content_length": len(d.current_content or ""),
+        "content_length": _text_len(d.current_content),
         "created_at": d.created_at.isoformat() if d.created_at else "",
         "updated_at": d.updated_at.isoformat() if d.updated_at else "",
     }
@@ -90,7 +94,7 @@ def cmd_versions(args):
                 "version_number": v.version_number,
                 "summary": v.summary or "",
                 "source": v.source or "ai",
-                "content_length": len(v.content or ""),
+                "content_length": _text_len(v.content),
             } for v in rows
         ], args)
     finally:

scripts/odysseus-gallery

@@ -59,6 +59,14 @@ def _serialize_image(i: "GalleryImage") -> dict:
     }
 
 
+def _album_image_count(album) -> int:
+    images = getattr(album, "images", None)
+    try:
+        return len(images) if images is not None else 0
+    except TypeError:
+        return 0
+
+
 def cmd_list(args):
     db = SessionLocal()
     try:
@@ -100,7 +108,7 @@ def cmd_albums(args):
     try:
         rows = db.query(GalleryAlbum).order_by(GalleryAlbum.name.asc()).all()
         emit([
-            {"id": a.id, "name": a.name, "image_count": len(a.images)}
+            {"id": a.id, "name": a.name, "image_count": _album_image_count(a)}
             for a in rows
         ], args)
     finally:

scripts/odysseus-mail

@@ -190,7 +190,7 @@ def cmd_read(args) -> None:
         if st != "OK":
             fail(f"select {args.folder!r} failed: {st}")
         st, msg_data = conn.fetch(args.uid.encode(), "(BODY.PEEK[])")
-        if st != "OK":
+        if st != "OK" or not msg_data or not msg_data[0]:
             fail(f"fetch UID {args.uid} failed: {st}")
         raw = msg_data[0][1]
         msg = email_mod.message_from_bytes(raw)

scripts/odysseus-mcp

@@ -33,15 +33,25 @@ except ModuleNotFoundError as e:
     sys.exit(2)
 
 
+def _json_list(raw) -> list:
+    try:
+        value = json.loads(raw) if raw else []
+    except (TypeError, json.JSONDecodeError):
+        return []
+    return value if isinstance(value, list) else []
+
+
+def _json_dict(raw) -> dict:
+    try:
+        value = json.loads(raw) if raw else {}
+    except (TypeError, json.JSONDecodeError):
+        return {}
+    return value if isinstance(value, dict) else {}
+
+
 def _serialize(s: "McpServer", redact_env: bool = True) -> dict:
-    try:
-        args_arr = json.loads(s.args) if s.args else []
-    except json.JSONDecodeError:
-        args_arr = []
-    try:
-        env_obj = json.loads(s.env) if s.env else {}
-    except json.JSONDecodeError:
-        env_obj = {}
+    args_arr = _json_list(s.args)
+    env_obj = _json_dict(s.env)
     if redact_env and isinstance(env_obj, dict):
         env_obj = {k: ("***" if v else "") for k, v in env_obj.items()}
     return {

scripts/odysseus-memory

@@ -47,8 +47,12 @@ def _manager() -> MemoryManager:
     return _mgr
 
 
+def _memory_entries(entries):
+    return [e for e in entries or [] if isinstance(e, dict)]
+
+
 def cmd_list(args):
-    entries = _manager().load_all()
+    entries = _memory_entries(_manager().load_all())
     if args.category:
         entries = [e for e in entries if (e.get("category") or "fact") == args.category]
     if args.source:
@@ -62,14 +66,14 @@ def cmd_list(args):
 
 def cmd_search(args):
     q = args.query.lower()
-    entries = _manager().load_all()
+    entries = _memory_entries(_manager().load_all())
     matches = [e for e in entries if q in (e.get("text") or "").lower()]
     matches = sorted(matches, key=lambda e: e.get("timestamp", 0), reverse=True)
     emit(matches[: args.limit], args)
 
 
 def cmd_show(args):
-    for e in _manager().load_all():
+    for e in _memory_entries(_manager().load_all()):
         if e.get("id") == args.id:
             emit(e, args)
             return
@@ -93,7 +97,7 @@ def cmd_add(args):
 
 
 def cmd_delete(args):
-    entries = _manager().load_all()
+    entries = _memory_entries(_manager().load_all())
     target = next((e for e in entries if e.get("id") == args.id), None)
     if not target:
         fail(f"no memory with id {args.id!r}")
@@ -104,7 +108,7 @@ def cmd_delete(args):
 
 def cmd_categories(args):
     counts: dict[str, int] = {}
-    for e in _manager().load_all():
+    for e in _memory_entries(_manager().load_all()):
         cat = e.get("category") or "fact"
         counts[cat] = counts.get(cat, 0) + 1
     rows = sorted(counts.items(), key=lambda kv: -kv[1])

scripts/odysseus-notes

@@ -29,12 +29,22 @@ except ModuleNotFoundError as e:
     sys.exit(2)
 
 
+def _load_items(raw) -> list:
+    if not raw:
+        return []
+    try:
+        items = json.loads(raw)
+    except (TypeError, json.JSONDecodeError):
+        return []
+    return items if isinstance(items, list) else []
+
+
 def _serialize(n: "Note") -> dict:
     return {
         "id": n.id,
         "title": n.title or "",
         "content": n.content or "",
-        "items": json.loads(n.items) if n.items else [],
+        "items": _load_items(n.items),
         "note_type": n.note_type or "note",
         "color": n.color or "",
         "label": n.label or "",

scripts/odysseus-personal

@@ -42,8 +42,12 @@ def _manager() -> PersonalDocsManager:
     return _mgr
 
 
+def _file_rows(files):
+    return [f for f in files or [] if isinstance(f, dict)]
+
+
 def cmd_list(args):
-    files = getattr(_manager(), "index", []) or []
+    files = _file_rows(getattr(_manager(), "index", []) or [])
     out = [
         {"name": f.get("name"), "size": f.get("size"), "path": f.get("path", "")}
         for f in files

scripts/odysseus-preset

@@ -85,7 +85,8 @@ def cmd_set(args):
     if prompt is None and args.temperature is None:
         fail("nothing to set — pass --prompt, --prompt-file, or --temperature")
     presets = _load()
-    entry = dict(presets.get(args.name) or {})
+    current = presets.get(args.name)
+    entry = dict(current) if isinstance(current, dict) else {}
     entry.setdefault("name", args.name)
     if prompt is not None:
         entry["system_prompt"] = prompt

scripts/odysseus-research

@@ -26,14 +26,19 @@ from pathlib import Path
 _DATA_DIR = _REPO_ROOT / "data" / "deep_research"
 
 
+def _load_path(path: Path) -> dict | None:
+    try:
+        data = json.loads(path.read_text())
+    except (json.JSONDecodeError, OSError):
+        return None
+    return data if isinstance(data, dict) else None
+
+
 def _load(rp_id: str) -> dict | None:
     path = _DATA_DIR / f"{rp_id}.json"
     if not path.exists():
         return None
-    try:
-        return json.loads(path.read_text())
-    except json.JSONDecodeError:
-        return None
+    return _load_path(path)
 
 
 def _preview_text(value, limit: int = 200) -> str:
@@ -64,9 +69,8 @@ def cmd_list(args):
     out = []
     for path in sorted(_DATA_DIR.glob("*.json")):
         rp_id = path.stem
-        try:
-            data = json.loads(path.read_text())
-        except Exception:
+        data = _load_path(path)
+        if data is None:
             continue
         if args.status and (data.get("status") or "") != args.status:
             continue
@@ -108,9 +112,8 @@ def cmd_search(args):
     out = []
     for path in _DATA_DIR.glob("*.json"):
         rp_id = path.stem
-        try:
-            data = json.loads(path.read_text())
-        except Exception:
+        data = _load_path(path)
+        if data is None:
             continue
         haystack = " ".join([
             (data.get("query") or "").lower(),

scripts/odysseus-signature

@@ -34,9 +34,12 @@ def _decode_png_data(data_png: str) -> bytes:
     if "," in raw:
         raw = raw.split(",", 1)[1]
     try:
-        return base64.b64decode(raw, validate=True)
+        decoded = base64.b64decode(raw, validate=True)
     except Exception as e:
         fail(f"data_png is not valid base64: {e}")
+    if not decoded.startswith(b"\x89PNG\r\n\x1a\n"):
+        fail("data_png is not a PNG image")
+    return decoded
 
 
 def cmd_list(args):

scripts/odysseus-skills

@@ -52,6 +52,10 @@ def _preview_text(value, limit: int = 200) -> str:
     return text[:limit]
 
 
+def _skill_entries(skills):
+    return [s for s in skills or [] if isinstance(s, dict)]
+
+
 def _summary(skill: dict) -> dict:
     return {
         "name": skill.get("name", ""),
@@ -65,7 +69,7 @@ def _summary(skill: dict) -> dict:
 
 
 def cmd_list(args):
-    out = _manager().load_all()
+    out = _skill_entries(_manager().load_all())
     if args.category:
         out = [s for s in out if (s.get("category") or "general") == args.category]
     out.sort(key=lambda s: (-int(s.get("uses") or 0), s.get("name", "")))
@@ -73,7 +77,7 @@ def cmd_list(args):
 
 
 def cmd_show(args):
-    for s in _manager().load_all():
+    for s in _skill_entries(_manager().load_all()):
         if s.get("name") == args.name:
             emit(s, args)
             return
@@ -82,7 +86,7 @@ def cmd_show(args):
 
 def cmd_categories(args):
     counts: dict[str, int] = {}
-    for s in _manager().load_all():
+    for s in _skill_entries(_manager().load_all()):
         c = s.get("category") or "general"
         counts[c] = counts.get(c, 0) + 1
     emit([{"category": c, "count": n} for c, n in sorted(counts.items())], args)
@@ -91,7 +95,7 @@ def cmd_categories(args):
 def cmd_delete(args):
     # Locate the skill's directory and rm -rf it.
     skills_root = Path(_DATA_DIR) / "skills"
-    for s in _manager().load_all():
+    for s in _skill_entries(_manager().load_all()):
         if s.get("name") != args.name:
             continue
         cat = s.get("category") or "general"
@@ -105,7 +109,7 @@ def cmd_delete(args):
 
 
 def cmd_export(args):
-    for s in _manager().load_all():
+    for s in _skill_entries(_manager().load_all()):
         if s.get("name") != args.name:
             continue
         cat = s.get("category") or "general"

scripts/odysseus-tasks

@@ -26,13 +26,18 @@ except ModuleNotFoundError as e:
     sys.exit(2)
 
 
+def _preview_text(value, limit: int = 200) -> str:
+    text = value if isinstance(value, str) else ""
+    return text[:limit] + ("…" if len(text) > limit else "")
+
+
 def _serialize_task(t: "ScheduledTask") -> dict:
     return {
         "id": t.id,
         "name": t.name,
         "task_type": t.task_type,
         "action": t.action,
-        "prompt": (t.prompt or "")[:200] + ("…" if t.prompt and len(t.prompt) > 200 else ""),
+        "prompt": _preview_text(t.prompt),
         "schedule": t.schedule,
         "scheduled_time": t.scheduled_time,
         "next_run": t.next_run.isoformat() if t.next_run else "",
@@ -51,7 +56,7 @@ def _serialize_run(r: "TaskRun") -> dict:
         "started_at": r.started_at.isoformat() if r.started_at else "",
         "completed_at": r.completed_at.isoformat() if r.completed_at else "",
         "status": r.status,
-        "output_preview": (getattr(r, "output", "") or "")[:200],
+        "output_preview": _preview_text(getattr(r, "output", "")),
     }
 
 

scripts/odysseus-theme

@@ -36,10 +36,14 @@ def _load_prefs() -> dict:
         return {"_users": {}}
     try:
         data = json.loads(_USER_PREFS_PATH.read_text())
-        data.setdefault("_users", {})
-        return data
     except json.JSONDecodeError as e:
         fail(f"user_prefs.json is corrupt: {e}")
+    if not isinstance(data, dict):
+        fail("user_prefs.json is corrupt: expected an object")
+    users = data.setdefault("_users", {})
+    if not isinstance(users, dict):
+        fail("user_prefs.json is corrupt: _users must be an object")
+    return data
 
 
 def _save_prefs(data: dict) -> None:

scripts/odysseus-webhook

@@ -30,6 +30,17 @@ except ModuleNotFoundError as e:
     sys.exit(2)
 
 
+def _mask_token(token: str, reveal: bool = False) -> str:
+    token = token or ""
+    if reveal:
+        return token
+    if not token:
+        return ""
+    if len(token) <= 10:
+        return "***"
+    return token[:6] + "…" + token[-4:]
+
+
 def _summary(t: "ScheduledTask", reveal: bool = False) -> dict:
     tok = t.webhook_token or ""
     return {
@@ -37,7 +48,7 @@ def _summary(t: "ScheduledTask", reveal: bool = False) -> dict:
         "name": t.name,
         "status": t.status,
         "task_type": t.task_type,
-        "webhook_token": tok if reveal else (tok[:6] + "…" + tok[-4:]) if tok else "",
+        "webhook_token": _mask_token(tok, reveal),
         "has_token": bool(tok),
     }
 

services/docs/service.py

@@ -57,6 +57,7 @@ class DocsService:
                 metadata=r.get("metadata"),
             )
             for r in results
+            if isinstance(r, dict)
         ]
 
     async def index(self, directory: str) -> IndexResult:

services/hwfit/fit.py

@@ -61,7 +61,7 @@ CONTEXT_TARGET = {
 
 
 def _lookup_bandwidth(gpu_name):
-    if not gpu_name:
+    if not isinstance(gpu_name, str) or not gpu_name:
         return None
     gn = gpu_name.lower()
     for key in _BW_KEYS_SORTED:
@@ -280,10 +280,14 @@ def _native_quant(model):
         return "FP8"
     if "gptq" in text:
         m = re.search(r"(?:gptq|int|w)(?:[-_]?)(\d{1,2})(?:bit)?", text)
-        return f"GPTQ-{m.group(1)}bit" if m else "GPTQ"
+        # Canonical catalog label is "GPTQ-Int4"/"GPTQ-Int8" (see models.py
+        # QUANT_BPP / QUANT_QUALITY_PENALTY keys); "GPTQ-4bit" misses both
+        # maps, so BPP and the quality penalty silently fall to defaults.
+        return f"GPTQ-Int{m.group(1)}" if m else "GPTQ-Int4"
     if "awq" in text:
         m = re.search(r"(?:awq|int|w)(?:[-_]?)(\d{1,2})(?:bit)?", text)
-        return f"AWQ-{m.group(1)}bit" if m else "AWQ"
+        # Catalog keys are "AWQ-4bit"/"AWQ-8bit"; bare "AWQ" misses the maps.
+        return f"AWQ-{m.group(1)}bit" if m else "AWQ-4bit"
     if "mlx" in text:
         m = re.search(r"mlx[-_]?(\d{1,2})bit", text)
         return f"mlx-{m.group(1)}bit" if m else native_quant

services/hwfit/models.py

@@ -123,7 +123,13 @@ def params_b(model):
         pc = pc.strip().upper()
         m = re.match(r"^([\d.]+)\s*([BKMGT]?)$", pc)
         if m:
-            val = float(m.group(1))
+            try:
+                val = float(m.group(1))
+            except ValueError:
+                # Malformed count like "1.5.3B" — [\d.]+ matches but float()
+                # rejects it. One bad catalog row must not abort the whole
+                # ranking pass, so treat it as unknown size.
+                return 0.0
             suffix = m.group(2)
             if suffix == "B":
                 return val

services/memory/memory.py

@@ -12,7 +12,7 @@ logger = logging.getLogger(__name__)
 
 def tokenize(text: str) -> List[str]:
     """Simple tokenizer that splits on whitespace and removes punctuation."""
-    return [word.strip('.,!?";') for word in text.split()]
+    return [cleaned for word in text.split() if (cleaned := word.strip('.,!?";'))]
 
 def get_text_similarity(text1: str, text2: str) -> float:
     """Calculate Jaccard similarity between two texts."""
@@ -59,14 +59,18 @@ class MemoryManager:
                     line = line.strip()
                     # Look for bullet points or numbered lists that might contain memories
                     if re.match(r'^[-*•]|\d+\.', line):
-                        # Extract the text after the bullet/number
-                        text_match = re.match(r'^[-*•]|\d+\.\s*(.*)', line)
+                        # Extract the text after the bullet/number. Group both
+                        # markers so the capture applies to either. The previous
+                        # `^[-*•]|\d+\.\s*(.*)` put the group on the numbered
+                        # branch only, so a bullet line matched with group(1)=None
+                        # and crashed on .strip().
+                        text_match = re.match(r'^(?:[-*•]|\d+\.)\s*(.*)', line)
                         if text_match:
                             text = text_match.group(1).strip()
                             if text:
                                 memories.append({
                                     "text": text,
-                                    "timestamp": int(datetime.now().timestamp()),
+                                    "timestamp": int(time.time()),
                                     "session_id": session_id
                                 })
                     # If we see a heading that suggests memories
@@ -101,6 +105,7 @@ class MemoryManager:
     def ensure_file_exists(self):
         """Create memory file if it doesn't exist."""
         if not os.path.exists(self.memory_file):
+            os.makedirs(os.path.dirname(self.memory_file), exist_ok=True)
             with open(self.memory_file, 'w', encoding='utf-8') as f:
                 json.dump([], f, ensure_ascii=False, indent=2)
     

services/memory/memory_extractor.py

@@ -34,7 +34,7 @@ def _fingerprint_entries(entries) -> str:
     only on id+text+category. Any add/edit/delete invalidates it."""
     items = sorted(
         (str(e.get("id", "")), e.get("text", ""), e.get("category", ""))
-        for e in entries
+        for e in _memory_dicts(entries)
     )
     h = hashlib.sha256()
     for triple in items:
@@ -42,6 +42,12 @@ def _fingerprint_entries(entries) -> str:
     return h.hexdigest()
 
 
+def _memory_dicts(entries):
+    for entry in entries or []:
+        if isinstance(entry, dict):
+            yield entry
+
+
 def _load_tidy_state(memory_manager) -> dict:
     path = _tidy_state_path(memory_manager)
     try:
@@ -211,7 +217,7 @@ def _is_text_duplicate(new_text: str, existing: list, threshold: float = 0.6) ->
     new_tokens = set(new_text.lower().split())
     if not new_tokens:
         return False
-    for entry in existing:
+    for entry in _memory_dicts(existing):
         old_tokens = set(entry.get("text", "").lower().split())
         if not old_tokens:
             continue
@@ -547,17 +553,20 @@ async def audit_memories(
             for e in all_entries:
                 if e.get("owner") is None and e["id"] not in audited_ids and e["id"] not in {o["id"] for o in other_entries}:
                     other_entries.append(e)
-            memory_manager.save(final_entries + other_entries)
+            saved_entries = final_entries + other_entries
         else:
-            memory_manager.save(final_entries)
+            saved_entries = final_entries
+        memory_manager.save(saved_entries)
         logger.info(
             f"Memory audit complete: {before_count} -> {after_count} entries "
             f"({before_count - after_count} removed/merged)"
         )
 
-        # Rebuild vector index
+        # Rebuild vector index from the full saved set, not just this owner's
+        # slice — otherwise the shared collection is wiped of every other
+        # owner's entries until they happen to run their own audit.
         if memory_vector and memory_vector.healthy:
-            memory_vector.rebuild(final_entries)
+            memory_vector.rebuild(saved_entries)
 
         # Persist the post-tidy fingerprint so the next call short-circuits
         # if nothing has changed in the meantime.

services/memory/service.py

@@ -103,6 +103,7 @@ class MemoryService:
                     metadata=r.get("metadata", {}),
                 )
                 for r in results
+                if isinstance(r, dict)
             ]
             return MemorySearchResult(memories=memories, query=query, total=len(memories))
 

services/memory/skill_extractor.py

@@ -48,6 +48,21 @@ MIN_CONFIDENCE = 0.6
 CONTEXT_WINDOW = 12
 
 
+def _skill_dicts(skills):
+    for skill in skills or []:
+        if isinstance(skill, dict):
+            yield skill
+
+
+def _has_duplicate_title(skills, title: str) -> bool:
+    wanted = title.lower()
+    for skill in _skill_dicts(skills):
+        existing = skill.get("title", "")
+        if isinstance(existing, str) and existing.lower() == wanted:
+            return True
+    return False
+
+
 async def maybe_extract_skill(
     session,
     skills_manager,
@@ -191,10 +206,9 @@ async def maybe_extract_skill(
 
         # Check for duplicate skills
         existing = skills_manager.load(owner=owner)
-        for sk in existing:
-            if sk.get("title", "").lower() == title.lower():
-                logger.debug("[skill-extract] '%s' already exists — dropped as duplicate", title)
-                return None
+        if _has_duplicate_title(existing, title):
+            logger.debug("[skill-extract] '%s' already exists — dropped as duplicate", title)
+            return None
 
         entry = skills_manager.add_skill(
             title=title,

services/memory/skills.py

@@ -650,7 +650,10 @@ class SkillsManager:
             ])
             score = _jaccard(query_tokens, _tokenize(text))
             for tag in sk.get("tags", []) or []:
-                if tag and tag in query.lower():
+                # Match tags as whole tokens, not substrings: `tag in query`
+                # boosted e.g. a "ai" tag for any query containing "email".
+                tag_tokens = _tokenize(tag)
+                if tag_tokens and tag_tokens <= query_tokens:
                     score = max(score, 0.3) * 1.3
             if query.lower() in (sk.get("description") or "").lower():
                 score = max(score, 0.6)

services/research/research_handler.py

@@ -14,6 +14,8 @@ import time
 from pathlib import Path
 from typing import Optional, Dict
 
+from src.research_utils import is_low_quality
+
 logger = logging.getLogger(__name__)
 
 RESEARCH_DATA_DIR = Path("data/deep_research")
@@ -179,13 +181,14 @@ class ResearchHandler:
 
     @staticmethod
     def _extract_sources(findings: list) -> list:
-        """Extract deduplicated [{url, title}] from findings."""
+        """Extract deduplicated [{url, title}] from findings, filtering low-quality ones."""
         seen = set()
         sources = []
         for f in findings:
             url = f.get("url", "")
             title = f.get("title", "") or url
-            if url and url not in seen:
+            summary = f.get("summary", "") or f.get("evidence", "")
+            if url and url not in seen and not is_low_quality(summary):
                 seen.add(url)
                 sources.append({"url": url, "title": title})
         return sources
@@ -346,7 +349,8 @@ class ResearchHandler:
             for f in findings:
                 url = f.get("url", "")
                 title = f.get("title", "") or url
-                if url and url not in seen_urls:
+                summary = f.get("summary", "") or f.get("evidence", "")
+                if url and url not in seen_urls and not is_low_quality(summary):
                     seen_urls.add(url)
                     source_lines.append(f"- [{title}]({url})")
             if source_lines:

services/search/analytics.py

@@ -45,32 +45,36 @@ class RateLimitError(SearchEngineError):
 # ----------------------------------------------------------------------
 # Analytics helpers
 # ----------------------------------------------------------------------
+def _default_analytics() -> Dict[str, Any]:
+    return {
+        "total_queries": 0,
+        "successful_queries": 0,
+        "failed_queries": 0,
+        "cache_hits": 0,
+        "cache_misses": 0,
+        "query_patterns": {},
+    }
+
+
 def _load_analytics() -> Dict[str, Any]:
     """Load analytics data from the JSON file, creating defaults if missing."""
     if not ANALYTICS_FILE.exists():
-        default = {
-            "total_queries": 0,
-            "successful_queries": 0,
-            "failed_queries": 0,
-            "cache_hits": 0,
-            "cache_misses": 0,
-            "query_patterns": {},
-        }
+        default = _default_analytics()
         _save_analytics(default)
         return default
     try:
         with open(ANALYTICS_FILE, "r", encoding="utf-8") as f:
-            return json.load(f)
+            data = json.load(f)
+        # Merge over defaults so a file written by an older schema (or a
+        # partial write) still has every counter — _record_query indexes
+        # these keys directly and would otherwise raise KeyError.
+        merged = _default_analytics()
+        if isinstance(data, dict):
+            merged.update(data)
+        return merged
     except Exception as e:
         logger.warning(f"Failed to load analytics file: {e}")
-        return {
-            "total_queries": 0,
-            "successful_queries": 0,
-            "failed_queries": 0,
-            "cache_hits": 0,
-            "cache_misses": 0,
-            "query_patterns": {},
-        }
+        return _default_analytics()
 
 
 def _save_analytics(data: Dict[str, Any]) -> None:

services/search/core.py

@@ -49,8 +49,18 @@ SEARCH_CONFIG: Dict[str, Any] = {
 }
 
 
+def _is_secret_key(name: str) -> bool:
+    """True for config keys that hold a credential (e.g. ``brave_api_key``)."""
+    return name.endswith(("_api_key", "_key", "_token", "_secret"))
+
+
 def get_search_config() -> Dict[str, Any]:
-    """Get current search configuration including active provider info."""
+    """Get current search configuration including active provider info.
+
+    Never returns stored API keys: callers — including the unauthenticated
+    ``GET /api/search/config`` route — only need key *presence* via
+    ``has_api_key``, not the secret itself (#1661).
+    """
     config = SEARCH_CONFIG.copy()
     settings = _get_search_settings()
     provider = settings.get("search_provider", "searxng")
@@ -60,13 +70,27 @@ def get_search_config() -> Dict[str, Any]:
     if provider == "searxng":
         from .providers import _get_search_instance
         config["search_url"] = _get_search_instance()
-    return config
+    # Strip any string-valued credential so secrets never reach the response;
+    # the boolean has_api_key flag (presence only) is preserved.
+    return {
+        k: v for k, v in config.items()
+        if not (isinstance(v, str) and _is_secret_key(k))
+    }
 
 
 def update_search_config(api_key: str = None, **kwargs):
-    """Update search configuration (e.g. Brave API key)."""
-    if api_key:
-        SEARCH_CONFIG["brave_api_key"] = api_key
+    """Merge non-secret search config into SEARCH_CONFIG.
+
+    Provider API keys are intentionally NOT cached here. They are read on demand
+    from settings/env via ``_get_provider_key`` (e.g. ``brave_search``), so the
+    previous ``SEARCH_CONFIG["brave_api_key"] = api_key`` cache was never used
+    for search and only leaked the decrypted key through ``get_search_config`` /
+    ``GET /api/search/config`` (#1661). ``api_key`` is accepted for backward
+    compatibility but no longer stored.
+    """
+    for k, v in kwargs.items():
+        if not _is_secret_key(k):
+            SEARCH_CONFIG[k] = v
 
 
 def _call_provider(provider_name: str, query: str, count: int, time_filter: str = None) -> List[dict]:
@@ -332,6 +356,12 @@ def comprehensive_web_search(
         for r in search_results if r.get("url")
     ]
 
+    # Map each URL to its [i] number in the sources list so fetched content
+    # blocks can be labeled with the SAME index the model cites.
+    _url_index = {
+        r["url"]: i for i, r in enumerate(search_results, 1) if r.get("url")
+    }
+
     # Fetch content in parallel
     fetched_content = []
     with ThreadPoolExecutor(max_workers=max_workers) as executor:
@@ -344,6 +374,10 @@ def comprehensive_web_search(
             try:
                 result = future.result()
                 if result["success"] and result["content"] and len(result["content"]) >= min_content_length:
+                    # Remember which source this fetch belongs to: redirects
+                    # can change result["url"] and completion order is
+                    # arbitrary, so the block label cannot be recomputed later.
+                    result["source_index"] = _url_index.get(url)
                     fetched_content.append(result)
             except Exception as e:
                 logger.error(f"Exception while fetching {url}: {str(e)}")
@@ -384,8 +418,15 @@ def comprehensive_web_search(
         output_parts.append("FETCHED PAGE CONTENT:")
         output_parts.append("-" * 50)
 
-        for i, content in enumerate(fetched_content, 1):
-            output_parts.append(f"\n[CONTENT {i}] From: {content['url']}")
+        # Emit blocks in source order, numbered with the same [i] as the
+        # sources list, so [CONTENT 2] really is content from source [2].
+        # Before this, blocks were numbered 1..N in fetch COMPLETION order,
+        # which matched neither the sources list nor each other run to run.
+        fetched_content.sort(key=lambda c: c.get("source_index") or len(search_results) + 1)
+        for content in fetched_content:
+            _idx = content.get("source_index")
+            _label = f"[CONTENT {_idx}]" if _idx else "[CONTENT]"
+            output_parts.append(f"\n{_label} From: {content['url']}")
             output_parts.append(f"Title: {content['title']}")
             output_parts.append("-" * 30)
 

services/search/providers.py

@@ -492,7 +492,6 @@ def google_pse_search(query: str, count: int = 10, time_filter: Optional[str] =
         if response.status_code == 429:
             raise RateLimitError("Google PSE rate limit hit")
         response.raise_for_status()
-        data = response.json()
     except httpx.RequestError as e:
         error_logger.error(f"Google PSE search failed: {e}")
         return []
@@ -500,6 +499,12 @@ def google_pse_search(query: str, count: int = 10, time_filter: Optional[str] =
         error_logger.error(str(e))
         return []
 
+    try:
+        data = response.json()
+    except json.JSONDecodeError as e:
+        error_logger.error(f"Google PSE returned invalid JSON: {e}")
+        return []
+
     results = []
     for item in data.get("items", [])[:count]:
         url = item.get("link", "")
@@ -544,7 +549,6 @@ def tavily_search(query: str, count: int = 10, time_filter: Optional[str] = None
         if response.status_code == 429:
             raise RateLimitError("Tavily rate limit hit")
         response.raise_for_status()
-        data = response.json()
     except httpx.RequestError as e:
         error_logger.error(f"Tavily search failed: {e}")
         return []
@@ -552,6 +556,12 @@ def tavily_search(query: str, count: int = 10, time_filter: Optional[str] = None
         error_logger.error(str(e))
         return []
 
+    try:
+        data = response.json()
+    except json.JSONDecodeError as e:
+        error_logger.error(f"Tavily returned invalid JSON: {e}")
+        return []
+
     results = []
     for item in data.get("results", [])[:count]:
         url = item.get("url", "")
@@ -599,7 +609,6 @@ def serper_search(query: str, count: int = 10, time_filter: Optional[str] = None
         if response.status_code == 429:
             raise RateLimitError("Serper rate limit hit")
         response.raise_for_status()
-        data = response.json()
     except httpx.RequestError as e:
         error_logger.error(f"Serper search failed: {e}")
         return []
@@ -607,6 +616,12 @@ def serper_search(query: str, count: int = 10, time_filter: Optional[str] = None
         error_logger.error(str(e))
         return []
 
+    try:
+        data = response.json()
+    except json.JSONDecodeError as e:
+        error_logger.error(f"Serper returned invalid JSON: {e}")
+        return []
+
     results = []
     for item in data.get("organic", [])[:count]:
         url = item.get("link", "")

services/search/query.py

@@ -27,6 +27,8 @@ def _detect_question_type(query: str) -> Optional[str]:
 
 def _extract_entities(query: str) -> Dict[str, List[str]]:
     """Lightweight entity extraction: capitalized words and date patterns."""
+    if not isinstance(query, str):
+        return {"names": [], "dates": []}
     entities: Dict[str, List[str]] = {"names": [], "dates": []}
     qtype = _detect_question_type(query)
     cleaned = query

services/search/ranking.py

@@ -13,6 +13,11 @@ _SPORTS_HINTS = {
     "sport", "sports", "soccer", "football", "hockey", "nba", "nfl", "mlb",
     "fifa", "world cup", "championship", "quarterfinal", "eliminates",
 }
+# Word-boundary match so "sport" does not fire inside "transport"/"passport"
+# and a domain like "transport.gov" is not mistaken for a sports site.
+_SPORTS_HINT_RE = re.compile(
+    r"\b(?:" + "|".join(re.escape(h) for h in _SPORTS_HINTS) + r")\b"
+)
 _LOW_VALUE_NEWS_DOMAINS = {
     "facebook.com", "www.facebook.com", "sports.yahoo.com", "yahoo.com",
     "www.yahoo.com", "msn.com", "www.msn.com",
@@ -39,7 +44,7 @@ def rank_search_results(query: str, results: List[dict]) -> List[dict]:
     query_terms = [t.lower() for t in re.findall(r"\b\w+\b", query)]
     query_lc = query.lower()
     is_news_query = any(term in _NEWS_HINTS for term in query_terms)
-    is_sports_query = any(hint in query_lc for hint in _SPORTS_HINTS)
+    is_sports_query = bool(_SPORTS_HINT_RE.search(query_lc))
 
     def title_score(title: str) -> float:
         if not title:
@@ -98,7 +103,7 @@ def rank_search_results(query: str, results: List[dict]) -> List[dict]:
             adjustment += 0.4
         if netloc in _LOW_VALUE_NEWS_DOMAINS:
             adjustment -= 0.8
-        if not is_sports_query and any(hint in text or hint in netloc for hint in _SPORTS_HINTS):
+        if not is_sports_query and (_SPORTS_HINT_RE.search(text) or _SPORTS_HINT_RE.search(netloc)):
             adjustment -= 1.5
         # A country/news query should not rank a page whose title/snippet barely
         # mentions the country above actual news pages for that country.

services/search/service.py

@@ -62,17 +62,24 @@ class SearchService:
             SearchResponse with results
         """
         depth = depth or self.default_depth
-        fetch_content = fetch_content if fetch_content is not None else self.fetch_content
 
-        # Use existing search implementation
-        raw_results = await comprehensive_web_search(
+        # comprehensive_web_search is synchronous and, with return_sources=True,
+        # returns (context_str, [{"url", "title"}, ...]). Run it off the event
+        # loop so we don't block it, and use the source list as the result rows.
+        # `fetch_content` is accepted for API compatibility; the comprehensive
+        # search always fetches page content.
+        import asyncio
+        _context, raw_results = await asyncio.to_thread(
+            comprehensive_web_search,
             query,
-            max_results=10 * depth,
-            fetch_content=fetch_content,
+            max_pages=10 * depth,
+            return_sources=True,
         )
 
         results = []
         for r in raw_results:
+            if not isinstance(r, dict):
+                continue
             results.append(SearchResult(
                 url=r.get("url", ""),
                 title=r.get("title", ""),

services/shell/service.py

@@ -125,10 +125,11 @@ class ShellService:
                 asyncio.create_task(_reader(proc.stderr, "stderr")),
             ]
 
+            loop = asyncio.get_running_loop()
             finished = 0
-            deadline = asyncio.get_event_loop().time() + timeout
+            deadline = loop.time() + timeout
             while finished < 2:
-                remaining = deadline - asyncio.get_event_loop().time()
+                remaining = deadline - loop.time()
                 if remaining <= 0:
                     raise asyncio.TimeoutError()
 

services/tts/tts_service.py

@@ -12,6 +12,18 @@ from typing import Optional, Dict, Any
 logger = logging.getLogger(__name__)
 
 
+def _safe_speed(value, default: float = 1.0) -> float:
+    """Parse the stored tts_speed defensively. The settings layer tolerates
+    corrupt/agent-written config, so a non-numeric or empty value (e.g. an agent
+    setting "speech speed" = "fast", or a hand-edited settings.json) must not
+    crash synthesis or the stats endpoint with a ValueError."""
+    try:
+        speed = float(value)
+    except (TypeError, ValueError):
+        return default
+    return speed if speed > 0 else default
+
+
 class TTSService:
     """Multi-provider TTS service.
 
@@ -136,7 +148,7 @@ class TTSService:
         provider = settings["tts_provider"]
         model = settings["tts_model"]
         voice = settings["tts_voice"]
-        speed = float(settings.get("tts_speed", "1"))
+        speed = _safe_speed(settings.get("tts_speed", "1"))
 
         if provider in ("disabled", "browser"):
             return None
@@ -198,7 +210,7 @@ class TTSService:
             "provider": provider,
             "model": settings["tts_model"],
             "voice": settings["tts_voice"],
-            "speed": float(settings.get("tts_speed", "1")),
+            "speed": _safe_speed(settings.get("tts_speed", "1")),
             "cache_entries": len(cache_files),
             "cache_size_mb": round(cache_size / (1024 * 1024), 2),
         }

services/youtube/youtube_handler.py

@@ -59,11 +59,15 @@ def init_youtube():
 
 
 def is_youtube_url(url: str) -> bool:
+    if not isinstance(url, str):
+        return False
     return "youtube.com" in url or "youtu.be" in url
 
 
 def extract_youtube_id(url: str) -> Optional[str]:
     """Extract YouTube video ID from various URL formats."""
+    if not isinstance(url, str):
+        return None
     parsed = urllib.parse.urlparse(url)
     if parsed.hostname in ("www.youtube.com", "youtube.com", "m.youtube.com"):
         if parsed.path == "/watch":
@@ -254,6 +258,8 @@ def format_comments_for_context(comments_data: Dict[str, Any], url: str) -> str:
     ctx += f"URL: {url}\n\n"
 
     for i, c in enumerate(comments, 1):
+        if not isinstance(c, dict):
+            continue
         likes = c.get("likes", 0)
         likes_str = f" [{likes} likes]" if likes else ""
         ctx += f"{i}. @{c['author']}{likes_str}: {c['text']}\n\n"

src/action_intents.py

@@ -63,7 +63,12 @@ _TOOL_INTENT_PATTERNS: tuple[Pattern[str], ...] = tuple(
         r"\bssh\s+\w+",
         r"\b(run|execute)\s+.{1,40}\bon\s+\w+",
         r"\b(can|could|please|would)\s+you\s+(run|execute|exec)\b",
-        r"\b(deploy|build|install|restart|reboot|kill|tail|grep|cat|ls|cd|cp|mv|rm)\b\s+\S+",
+        # Shell verbs only count in imperative position (start of message,
+        # optionally after "please") or as a "can you ..." request. A bare
+        # word match promoted informational questions ("What does the grep
+        # command do?") and incidental uses ("My cat ate my homework").
+        rf"{_PLEASE}(deploy|build|install|restart|reboot|kill|tail|grep|cat|ls|cd|cp|mv|rm)\b\s+\S+",
+        rf"{_ACTION_QUESTION}(deploy|build|install|restart|reboot|kill|tail|grep|cat|ls|cd|cp|mv|rm)\b\s+\S+",
         r"\b(check|see)\s+(if|whether|what)\s+.{1,40}\b(running|process|service|port|file|exists?)\b",
     )
 )

src/agent_loop.py

@@ -14,7 +14,7 @@ import time
 import logging
 from typing import AsyncGenerator, List, Dict, Optional, Set
 
-from src.llm_core import stream_llm, stream_llm_with_fallback
+from src.llm_core import stream_llm, stream_llm_with_fallback, _is_ollama_native_url
 from src.model_context import estimate_tokens
 from src.settings import get_setting
 from src.prompt_security import untrusted_context_message
@@ -1494,9 +1494,18 @@ async def stream_agent_loop(
     _model_no_tools = any(kw in _model_lc for kw in (
         "deepseek-r1",
     ))
+    # Native Ollama endpoints (/api/chat) handle tool schemas differently from
+    # the OpenAI-compat path. Models like gemma4, qwen3.5, ministral respond to
+    # tool schemas by emitting a single native tool_call token then stopping,
+    # rather than writing a fenced block — the agent loop sees 1 token and no
+    # recognised tool, so the round terminates immediately (issue #1567).
+    # Unless the endpoint is explicitly marked supports_tools=True by the user
+    # (via the endpoint settings toggle), treat Ollama-native as text-only so
+    # the fenced-block path is used instead of native function calling.
+    _is_ollama_native = _is_ollama_native_url(endpoint_url or "")
     if _endpoint_supports is True:
         _is_api_model = True
-    elif _endpoint_supports is False or _model_no_tools:
+    elif _endpoint_supports is False or _model_no_tools or _is_ollama_native:
         _is_api_model = False
     else:
         _is_api_model = any(h in endpoint_url for h in _API_HOSTS) or _model_supports_tools

src/agent_tools.py

@@ -80,6 +80,11 @@ def get_mcp_manager():
 # Helpers (kept here — used by sub-modules)
 # ---------------------------------------------------------------------------
 def _truncate(text: str, limit: int = MAX_OUTPUT_CHARS) -> str:
+    # Callers treat the result as text, so always return a string: coerce a
+    # non-string (None -> "", otherwise str(...)) instead of returning it raw,
+    # which would just move the crash downstream.
+    if not isinstance(text, str):
+        text = "" if text is None else str(text)
     if len(text) > limit:
         return text[:limit] + f"\n... (truncated, {len(text)} chars total)"
     return text

src/ai_interaction.py

@@ -517,7 +517,7 @@ async def do_list_sessions(content: str, session_id: Optional[str] = None, owner
         return {"error": str(e)}
 
 
-async def do_send_to_session(content: str, session_id: Optional[str] = None) -> Dict:
+async def do_send_to_session(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict:
     """Send a message to an existing session and get a response.
 
     Content format:
@@ -541,6 +541,10 @@ async def do_send_to_session(content: str, session_id: Optional[str] = None) ->
     if not sess:
         return {"error": f"Session '{target_sid}' not found"}
 
+    # Owner-scope: reject access to another user's session
+    if owner and getattr(sess, "owner", None) and sess.owner != owner:
+        return {"error": f"Session '{target_sid}' not found"}
+
     if not message:
         return {"error": "No message provided"}
 
@@ -1228,9 +1232,11 @@ async def do_manage_rag(content: str, session_id: Optional[str] = None) -> Dict:
 
         try:
             if hasattr(_personal_docs_manager, 'remove_directory'):
+                # Performs a targeted per-directory delete (#1660). The previous
+                # unconditional _rag_manager.rebuild_index() here wiped the whole
+                # collection on every remove (even for untracked dirs) and has
+                # been removed.
                 _personal_docs_manager.remove_directory(directory)
-            if _rag_manager and hasattr(_rag_manager, 'rebuild_index'):
-                _rag_manager.rebuild_index()
             return {"action": "remove_directory", "directory": directory,
                     "results": f"Directory '{directory}' removed from RAG index"}
         except Exception as e:
@@ -1288,7 +1294,7 @@ async def do_ui_control(content: str, session_id: Optional[str] = None) -> Dict:
             "private": "incognito",
         }
         toggle_name = _toggle_aliases.get(toggle_name, toggle_name)
-        valid_toggles = {"web", "bash", "research", "incognito", "document_editor"}
+        valid_toggles = {"web", "bash", "rag", "research", "incognito", "document_editor"}
         if toggle_name not in valid_toggles:
             return {"error": f"Unknown toggle '{toggle_name}'. Valid: {', '.join(sorted(valid_toggles))}"}
         return {
@@ -1769,7 +1775,7 @@ async def dispatch_ai_tool(
     elif tool == "send_to_session":
         sid = content.split("\n")[0].strip()[:20]
         desc = f"send_to_session: {sid}"
-        result = await do_send_to_session(content, session_id)
+        result = await do_send_to_session(content, session_id, owner=owner)
 
     elif tool == "pipeline":
         desc = "pipeline: running steps"

src/auth_helpers.py

@@ -107,6 +107,8 @@ def require_privilege(request: Request, key: str) -> str:
         privs = auth_mgr.get_privileges(user) or {}
     except Exception:
         return user
+    if not isinstance(privs, dict):
+        privs = {}
     # True = permitted; missing key defaults to permitted (unknown privileges
     # fail open — the UI gates display-side).
     if not privs.get(key, True):

src/bg_jobs.py

@@ -55,7 +55,10 @@ _RETENTION_S = 3600  # 1 hour after follow-up
 def _load() -> Dict[str, Dict[str, Any]]:
     try:
         if _STORE.exists():
-            return json.loads(_STORE.read_text(encoding="utf-8")) or {}
+            data = json.loads(_STORE.read_text(encoding="utf-8")) or {}
+            if not isinstance(data, dict):
+                return {}
+            return {str(job_id): rec for job_id, rec in data.items() if isinstance(rec, dict)}
     except Exception:
         pass
     return {}

src/bg_monitor.py

@@ -53,7 +53,9 @@ async def _drain_agent(sess, messages):
         if not isinstance(d, dict):
             continue
         if "delta" in d:
-            full += d["delta"]
+            delta = d.get("delta")
+            if isinstance(delta, str):
+                full += delta
         elif d.get("type") == "agent_step":
             round_num = d.get("round", round_num)
         elif d.get("type") == "tool_output":

src/builtin_actions.py

@@ -956,6 +956,17 @@ async def action_mark_email_boundaries(owner: str, **kwargs) -> Tuple[str, bool]
         return str(e), False
 
 
+# Sender local-parts (matched exactly or by prefix) whose mail never carries a
+# personal signature worth learning. These compare against the local-part
+# (before "@"), so role names must NOT include a trailing "@" — "support@" etc.
+# could never match a local-part of "support" and were silently dead.
+_SIG_SKIP_PREFIXES = (
+    "noreply", "no-reply", "donotreply", "do-not-reply",
+    "mailer-daemon", "notifications", "notification", "bounce",
+    "newsletter", "support", "info", "admin",
+)
+
+
 async def action_learn_sender_signatures(owner: str, **kwargs) -> Tuple[str, bool]:
     """For each sender with ≥3 recent inbox emails, ask the LLM to extract
     the common signature block across their messages. The cached sig is
@@ -1013,16 +1024,11 @@ async def action_learn_sender_signatures(owner: str, **kwargs) -> Tuple[str, boo
             return "No emails to scan", True
 
         # 2. Group by sender; drop addresses that don't carry useful sigs.
-        SKIP_PREFIXES = (
-            "noreply", "no-reply", "donotreply", "do-not-reply",
-            "mailer-daemon", "notifications", "notification", "bounce",
-            "newsletter", "support@", "info@", "admin@",
-        )
         by_sender: dict[str, list[dict]] = {}
         for m in mails:
             addr = m["from_address"]
             local = addr.split("@", 1)[0]
-            if any(local == p or local.startswith(p) for p in SKIP_PREFIXES):
+            if any(local == p or local.startswith(p) for p in _SIG_SKIP_PREFIXES):
                 continue
             # Skip plus-aliases / list-style addresses too.
             if "+" in local or "-noreply" in addr or "-bounces" in addr:

src/caldav_writeback.py

@@ -157,10 +157,14 @@ async def writeback_event(owner: str, calendar_source: str, calendar_id: str,
         return {"skipped": "not a caldav calendar"}
     try:
         from routes.prefs_routes import _load_for_user
+        from src.secret_storage import decrypt
         cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
         url = (cfg.get("url") or "").strip()
         user = (cfg.get("username") or "").strip()
-        pw = cfg.get("password") or ""
+        # Stored encrypted by routes/calendar_routes; decrypt before use so
+        # the remote sees the real password (decrypt is a no-op on legacy
+        # plaintext). The pull path src/caldav_sync.py already does this.
+        pw = decrypt(cfg.get("password") or "")
         if not (url and user and pw):
             return {"skipped": "caldav not configured"}
         result = await asyncio.to_thread(_writeback_blocking, calendar_id, ev, delete, url, user, pw)

src/chat_helpers.py

@@ -40,12 +40,16 @@ _VISION_MODEL_KEYWORDS = (
     "internvl", "cogvlm", "qwen-vl", "qwen2-vl", "qwen3-vl", "qwen3vl",
     # multimodal families whose names don't contain "vision"/"vl" but DO accept
     # images — without these the image is silently dropped for common Ollama tags
-    # like gemma3:4b (issue #1274). Gemma 3 (4b+), Llama 4 (all), and Mistral
-    # Small 3.1/3.2 are vision-capable; per the err-toward-True policy (#124) a
-    # rare text-only tag (e.g. gemma3:1b) being treated as vision is the safer
-    # failure than dropping a real image.
-    "gemma-3", "gemma3", "llama-4", "llama4",
+    # like gemma3:4b or gemma4:12b (issue #1274). Gemma 3/4 (4b+), Llama 4 (all),
+    # Mistral Small 3.1/3.2, and Phi-4 multimodal are vision-capable; per the
+    # err-toward-True policy (#124) a rare text-only tag being treated as vision is
+    # the safer failure than silently dropping a real image.
+    "gemma-3", "gemma3", "gemma-4", "gemma4",
+    "llama-4", "llama4",
     "mistral-small-3.1", "mistral-small3.1", "mistral-small-3.2", "mistral-small3.2",
+    # Microsoft Phi-4 ships a dedicated multimodal variant ("phi-4-multimodal-instruct")
+    # but users often load it under the bare "phi-4" or "phi4" Ollama tag.
+    "phi-4", "phi4",
     # zhipu / glm (glm-4.5v, glm-4.6v, glm-5v-turbo, etc.)
     "glm-4.5v", "glm-4.6v", "glm-5v",
 )

src/cleanup_service.py

@@ -1,10 +1,20 @@
 # src/cleanup_service.py
 import logging
-from datetime import datetime, timedelta
+from datetime import datetime, timedelta, timezone
 from typing import Tuple, Dict, Any, Optional
 
 logger = logging.getLogger(__name__)
 
+
+def _utcnow() -> datetime:
+    """Naive UTC for this module's DB-bound timestamps.
+
+    Mirrors the naive DateTime columns these values are compared against,
+    without the deprecated stdlib UTC-now call (removed in Python 3.14).
+    """
+    return datetime.now(timezone.utc).replace(tzinfo=None)
+
+
 class CleanupConfig:
     """Configuration constants for cleanup operations."""
     ARCHIVE_AFTER_DAYS = 7
@@ -38,7 +48,7 @@ async def archive_inactive_sessions(session_manager, owner: Optional[str] = None
     Returns:
         Number of sessions archived
     """
-    cutoff_date = datetime.utcnow() - timedelta(days=CleanupConfig.ARCHIVE_AFTER_DAYS)
+    cutoff_date = _utcnow() - timedelta(days=CleanupConfig.ARCHIVE_AFTER_DAYS)
     archived_count = 0
 
     from src.database import SessionLocal, Session as DbSession
@@ -53,7 +63,7 @@ async def archive_inactive_sessions(session_manager, owner: Optional[str] = None
 
         for session in sessions_to_archive:
             session.archived = True
-            session.updated_at = datetime.utcnow()
+            session.updated_at = _utcnow()
             archived_count += 1
 
         if archived_count > 0:
@@ -79,7 +89,7 @@ async def cleanup_old_sessions(session_manager, owner: Optional[str] = None) ->
     Returns:
         Tuple of (number of sessions deleted, space freed in MB)
     """
-    cutoff_date = datetime.utcnow() - timedelta(days=CleanupConfig.DELETE_AFTER_DAYS)
+    cutoff_date = _utcnow() - timedelta(days=CleanupConfig.DELETE_AFTER_DAYS)
     deleted_count = 0
     space_freed = 0
 
@@ -158,8 +168,8 @@ async def get_cleanup_preview(owner: Optional[str] = None) -> Dict[str, Any]:
     Returns:
         Dictionary containing preview information
     """
-    cutoff_archive = datetime.utcnow() - timedelta(days=CleanupConfig.ARCHIVE_AFTER_DAYS)
-    cutoff_delete = datetime.utcnow() - timedelta(days=CleanupConfig.DELETE_AFTER_DAYS)
+    cutoff_archive = _utcnow() - timedelta(days=CleanupConfig.ARCHIVE_AFTER_DAYS)
+    cutoff_delete = _utcnow() - timedelta(days=CleanupConfig.DELETE_AFTER_DAYS)
 
     sessions_to_archive = []
     sessions_to_delete = []

src/context_compactor.py

@@ -15,6 +15,26 @@ from core.models import ChatMessage
 
 logger = logging.getLogger(__name__)
 
+
+def _content_as_text(content: Any) -> str:
+    """Flatten a message's content to plain text.
+
+    Handles the three shapes that flow through history: a plain string, a
+    multimodal list of content blocks (vision/image attachments), and None
+    (assistant turns that carried only native tool_calls persist content as
+    None). Returns "" for anything without text so callers can safely slice
+    the result.
+    """
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        return " ".join(
+            b.get("text", "") for b in content
+            if isinstance(b, dict) and b.get("text")
+        )
+    return ""
+
+
 COMPACT_THRESHOLD = 0.85  # Trigger compaction at 85% of context window
 SUMMARY_MAX_TOKENS = 1024
 SMALL_CONTEXT_LIMIT = 8192  # Models with context <= this get aggressive trimming
@@ -96,6 +116,8 @@ def _sanitize_tool_messages(msgs: List[Dict]) -> List[Dict]:
 
 
 def _message_text_token_estimate(text: str) -> int:
+    if not isinstance(text, str):
+        return 4
     return int(len(text) * 0.3) + 4
 
 
@@ -104,6 +126,11 @@ def _truncate_text_to_token_budget(text: str, token_budget: int) -> str:
     if token_budget <= 32:
         return "[Current user message omitted: it exceeded the model context window.]"
 
+    if not isinstance(text, str):
+        # This helper is typed/used as text downstream, so return an empty
+        # string rather than the raw non-string (which would move the crash
+        # into the caller that concatenates/measures the result).
+        return ""
     # Match src.model_context.estimate_tokens' rough chars * 0.3 estimate.
     max_chars = max(200, int((token_budget - 16) / 0.3))
     if len(text) <= max_chars:
@@ -274,7 +301,7 @@ async def maybe_compact(
 
     # Build the text to summarize
     convo_text = "\n".join(
-        f"{msg['role'].upper()}: {msg.get('content', '')[:2000]}"
+        f"{msg.get('role', 'user').upper()}: {_content_as_text(msg.get('content'))[:2000]}"
         for msg in older
     )
 

src/deep_research.py

@@ -800,6 +800,17 @@ class DeepResearcher:
         except json.JSONDecodeError:
             pass
 
+        # Handle truncated arrays — e.g. '["query one", "query two", "query thr'
+        # Repair from the LAST array start so an echoed example array earlier
+        # in the reply is not harvested into the real query set.
+        last_start = text.rfind('[')
+        truncated = last_start != -1 and ']' not in text[last_start:]
+        if truncated:
+            complete_items = re.findall(r'"([^"]*)"', text[last_start:])
+            if complete_items:
+                logger.info(f"Repaired truncated JSON array: recovered {len(complete_items)} items")
+                return complete_items
+
         # Greedy match to capture the full outermost array
         match = re.search(r'\[[\s\S]*\]', text)
         if match:
@@ -810,8 +821,22 @@ class DeepResearcher:
             except json.JSONDecodeError:
                 pass
 
-        # Handle truncated arrays — e.g. '["query one", "query two", "query thr'
-        # Try to find the start of an array and repair it
+        # Multiple complete arrays in one reply (e.g. the model echoes the
+        # prompt's Example: [...] before the real array). The greedy match
+        # above spans them all and fails to parse, so scan non-greedily and
+        # keep the LAST parseable array, which is the model's actual answer.
+        last_parsed = None
+        for m in re.finditer(r'\[[\s\S]*?\]', text):
+            try:
+                parsed = json.loads(m.group())
+                if isinstance(parsed, list):
+                    last_parsed = parsed
+            except json.JSONDecodeError:
+                continue
+        if last_parsed is not None:
+            return [str(item) for item in last_parsed]
+
+        # Last resort: harvest quoted strings from the first array start
         arr_start = text.find('[')
         if arr_start != -1:
             fragment = text[arr_start:]

src/document_actions.py

@@ -6,6 +6,7 @@ Reusable document actions callable from both REST routes and the task scheduler.
 
 import logging
 import re
+from datetime import datetime
 
 logger = logging.getLogger(__name__)
 
@@ -140,7 +141,20 @@ async def run_document_tidy(owner: str) -> str:
             # Keep the most complete (longest real content), then most recent.
             def _updated(d):
                 return d.updated_at or d.created_at
-            members.sort(key=lambda d: (_real_len(d.current_content), _updated(d)), reverse=True)
+            # Sort key must be total-order safe: a document with both
+            # updated_at and created_at NULL would otherwise make Python
+            # compare None against a datetime on a real-length tie, raising
+            # TypeError and aborting the whole tidy run. Rank "has a
+            # timestamp" before the timestamp itself so a None is never
+            # compared against a datetime.
+            members.sort(
+                key=lambda d: (
+                    _real_len(d.current_content),
+                    _updated(d) is not None,
+                    _updated(d) or datetime.min,
+                ),
+                reverse=True,
+            )
             keeper = members[0]
             kept += 1
             dupes = members[1:]

src/document_processor.py

@@ -12,6 +12,9 @@ from src.llm_core import llm_call
 
 logger = logging.getLogger(__name__)
 
+MAX_INLINE_ATTACHMENT_CHARS = 24000
+MIN_INLINE_ATTACHMENT_SLICE = 500
+
 
 def _is_text_file(path: str) -> bool:
     """Check if file has text extension."""
@@ -160,6 +163,41 @@ def _truncate_inline(text: str, limit: int = 15000) -> tuple[str, str]:
     return text, ""
 
 
+def _fit_inline_attachment_text(
+    text: str,
+    remaining: int,
+    display_name: str,
+) -> tuple[str, int]:
+    """Fit extracted attachment text into the shared inline attachment budget.
+
+    Individual processors already cap single files, but multi-file batches can
+    still add N capped bodies to one user turn. Keep the first files readable,
+    keep later files visible by name, and mark exactly where inline content was
+    reduced so the model does not silently miss attachments.
+    """
+    text = text or ""
+    if len(text) <= remaining:
+        return text, remaining - len(text)
+
+    name = os.path.basename(display_name or "attachment")
+    if remaining < MIN_INLINE_ATTACHMENT_SLICE:
+        return (
+            f"\n\n[Attachment omitted from inline context: {name}. "
+            f"The {MAX_INLINE_ATTACHMENT_CHARS:,}-character shared inline "
+            "attachment budget was already used by earlier attachments. Ask "
+            "to inspect this file specifically if more detail is needed.]",
+            0,
+        )
+    marker = (
+        f"\n\n[Attachment content truncated: {name}. "
+        f"Only {remaining:,} characters of this attachment fit within "
+        f"the {MAX_INLINE_ATTACHMENT_CHARS:,}-character shared inline "
+        "attachment budget. Ask to inspect this file specifically if more "
+        "detail is needed.]"
+    )
+    return text[:remaining] + marker, 0
+
+
 def _process_office_document(path: str, display_name: str) -> str:
     """Extract an Office/EPUB document to Markdown via the optional markitdown dep.
 
@@ -323,6 +361,7 @@ def build_user_content(
     frontend can switch to the new doc immediately.
     """
     content = [{"type": "text", "text": text}]
+    inline_attachment_remaining = MAX_INLINE_ATTACHMENT_CHARS
 
     for fid in attachment_ids or []:
         upload_info = (resolved_uploads or {}).get(fid)
@@ -394,9 +433,7 @@ def build_user_content(
                         # Pull the PDF prose once — used as either intro_text
                         # (form path) or the doc body (plain path).
                         try:
-                            pdf_body_text = _process_pdf(path).lstrip(
-                                "\n[PDF content]:"
-                            ).strip()
+                            pdf_body_text = strip_pdf_content_marker(_process_pdf(path))
                         except Exception:
                             pdf_body_text = None
 
@@ -485,6 +522,11 @@ def build_user_content(
             else:
                 extracted_text = _process_office_document(path, display_name)
 
+            extracted_text, inline_attachment_remaining = _fit_inline_attachment_text(
+                extracted_text,
+                inline_attachment_remaining,
+                display_name,
+            )
             if content and content[0]["type"] == "text":
                 content[0]["text"] += extracted_text
             else:

src/email_thread_parser.py

@@ -605,10 +605,10 @@ def _parse_html(html: str) -> list[dict[str, Any]] | None:
 def parse_thread(body_html: str | None, body_text: str | None) -> list[dict[str, Any]] | None:
     """Public entry point. Prefer HTML when available, else plaintext.
     Returns None if no quoted material found (caller renders flat)."""
-    if body_html:
+    if isinstance(body_html, str) and body_html:
         out = _parse_html(body_html)
         if out:
             return out
-    if body_text:
+    if isinstance(body_text, str) and body_text:
         return _parse_plaintext(body_text)
     return None

src/integrations.py

@@ -197,6 +197,10 @@ def load_integrations() -> List[Dict[str, Any]]:
         if not isinstance(integrations, list):
             log.error("Invalid integrations file shape: expected a list")
             return []
+        valid_integrations = [item for item in integrations if isinstance(item, dict)]
+        if len(valid_integrations) != len(integrations):
+            log.error("Invalid integrations file rows: ignored non-object entries")
+        integrations = valid_integrations
         if _has_plaintext_api_key(integrations):
             save_integrations(_decrypt_integration_secrets(integrations))
         return _decrypt_integration_secrets(integrations)

src/llm_core.py

@@ -420,7 +420,7 @@ def _restricts_temperature(model: str) -> bool:
     return any(m.startswith(p) or f"/{p}" in m for p in _FIXED_TEMPERATURE_MODELS)
 
 # Models that support structured thinking — may output </think> without opening tag
-_THINKING_MODEL_PATTERNS = ("qwen3", "qwq", "deepseek-r1", "deepseek-reasoner", "minimax", "m2-reap")
+_THINKING_MODEL_PATTERNS = ("qwen3", "qwq", "deepseek-r1", "deepseek-reasoner", "minimax", "m2-reap", "gemma")
 
 def _supports_thinking(model: str) -> bool:
     """Check if model supports structured thinking output."""
@@ -512,6 +512,12 @@ def _build_anthropic_payload(model, messages, temperature, max_tokens, stream=Fa
             # Convert multimodal content (image_url → image) for Anthropic
             content = _convert_openai_content_to_anthropic(m["content"])
             chat_messages.append({"role": m["role"], "content": content})
+    # Anthropic only accepts temperature in [0.0, 1.0] and 400s on anything above
+    # 1.0. Clamp here (in the Anthropic builder only) so presets/sliders that use
+    # the wider OpenAI 0.0-2.0 range — e.g. the shipped "Nietzsche" preset at 1.2
+    # — don't hard-break every Claude request. OpenAI's own path is left untouched.
+    if temperature is not None:
+        temperature = max(0.0, min(temperature, 1.0))
     payload = {
         "model": model,
         "messages": chat_messages,
@@ -868,6 +874,31 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
         raise HTTPException(502, f"Unexpected schema from {target_url}: {str(data)[:400]}")
 
 
+def _dedupe_candidates(candidates):
+    """Filter malformed entries and drop a later repeat of an already-seen
+    ``(url, model)`` route, preserving order (first occurrence wins).
+
+    The chain is the primary target followed by the configured fallbacks, so a
+    fallback that repeats the session's current model — a common misconfiguration,
+    since callers prepend the live ``(url, model)`` to ``default_model_fallbacks``
+    — would otherwise make the chain re-attempt the very route that just failed:
+    a wasted round-trip plus a spurious ``fallback`` notice for a switch that did
+    not happen. Headers are not part of the key; the first tuple (with its
+    headers) is the one kept.
+    """
+    seen = set()
+    out = []
+    for c in candidates or []:
+        if not c or not c[0] or not c[1]:
+            continue
+        key = (c[0], c[1])
+        if key in seen:
+            continue
+        seen.add(key)
+        out.append(c)
+    return out
+
+
 def llm_call_with_fallback(candidates, messages, **kwargs) -> str:
     """Sync `llm_call` with an ordered fallback chain.
 
@@ -876,7 +907,7 @@ def llm_call_with_fallback(candidates, messages, **kwargs) -> str:
     the next candidate. The dead-host cooldown inside `llm_call` makes repeat
     attempts at an offline primary effectively free.
     """
-    cands = [c for c in (candidates or []) if c and c[0] and c[1]]
+    cands = _dedupe_candidates(candidates)
     if not cands:
         raise HTTPException(503, "No model endpoint configured")
     last_err = None
@@ -893,7 +924,7 @@ def llm_call_with_fallback(candidates, messages, **kwargs) -> str:
 
 async def llm_call_async_with_fallback(candidates, messages, **kwargs) -> str:
     """Async variant of `llm_call_with_fallback` — same semantics."""
-    cands = [c for c in (candidates or []) if c and c[0] and c[1]]
+    cands = _dedupe_candidates(candidates)
     if not cands:
         raise HTTPException(503, "No model endpoint configured")
     last_err = None
@@ -1161,9 +1192,13 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
                     yield f'event: error\ndata: {json.dumps({"status": r.status_code, "text": friendly, "raw": raw[:500]})}\n\n'
                     return
                 async for line in r.aiter_lines():
-                    if not line or not line.startswith("data: "):
+                    # SSE allows "data:value" with no space after the colon
+                    # (the space is optional per the spec). Some gateways and
+                    # local servers omit it; gating on "data: " dropped their
+                    # entire stream.
+                    if not line or not line.startswith("data:"):
                         continue
-                    data = line[6:].strip()
+                    data = line[5:].strip()
                     if not data or not data.startswith("{"):
                         continue
                     try:
@@ -1276,8 +1311,11 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
                 if not line:
                     continue
 
-                if line.startswith("data: "):
-                    data = line[6:].strip()
+                # SSE allows "data:value" with no space after the colon; gating
+                # on "data: " silently dropped content + usage from providers
+                # that omit it.
+                if line.startswith("data:"):
+                    data = line[5:].strip()
                     if data == "[DONE]":
                         tc_event = _emit_tool_calls()
                         if tc_event:
@@ -1292,7 +1330,19 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
                                 # Usage chunk (from stream_options)
                                 _choices = j.get("choices") or []
                                 _delta0 = _choices[0].get("delta") if _choices else None
-                                if "usage" in j and _delta0 in (None, {}, {"content": None}):
+                                # Capture usage whenever the chunk carries it and
+                                # the delta has no actual output. Some gateways /
+                                # local servers attach usage to the FINAL delta,
+                                # which also carries role/finish_reason (so it is
+                                # not exactly None/{}/{"content": None}); gating on
+                                # those exact shapes discarded their token counts.
+                                _delta_has_output = isinstance(_delta0, dict) and (
+                                    _delta0.get("content")
+                                    or _delta0.get("reasoning_content")
+                                    or _delta0.get("reasoning")
+                                    or _delta0.get("tool_calls")
+                                )
+                                if "usage" in j and not _delta_has_output:
                                     u = j["usage"]
                                     _usage_data = {"input_tokens": u.get("prompt_tokens", 0), "output_tokens": u.get("completion_tokens", 0)}
                                     # llama.cpp puts a `timings` block alongside `usage` with the
@@ -1430,7 +1480,7 @@ async def stream_llm_with_fallback(candidates, messages, **kwargs):
 
     Yields the same SSE chunk protocol as stream_llm.
     """
-    cands = [c for c in (candidates or []) if c and c[0] and c[1]]
+    cands = _dedupe_candidates(candidates)
     if not cands:
         yield f'event: error\ndata: {json.dumps({"error": "No model endpoint configured", "status": 503})}\n\n'
         return

src/mcp_manager.py

@@ -89,14 +89,18 @@ class McpManager:
             )
 
             stack = AsyncExitStack()
-            transport = await stack.enter_async_context(stdio_client(server_params))
-            read_stream, write_stream = transport
-            session = await stack.enter_async_context(ClientSession(read_stream, write_stream))
+            try:
+                transport = await stack.enter_async_context(stdio_client(server_params))
+                read_stream, write_stream = transport
+                session = await stack.enter_async_context(ClientSession(read_stream, write_stream))
 
-            await session.initialize()
+                await session.initialize()
 
-            # Discover tools
-            tools_result = await session.list_tools()
+                # Discover tools
+                tools_result = await session.list_tools()
+            except Exception:
+                await stack.aclose()
+                raise
             tools = []
             for tool in tools_result.tools:
                 tools.append({
@@ -142,14 +146,18 @@ class McpManager:
             from contextlib import AsyncExitStack
 
             stack = AsyncExitStack()
-            transport = await stack.enter_async_context(sse_client(url))
-            read_stream, write_stream = transport
-            session = await stack.enter_async_context(ClientSession(read_stream, write_stream))
+            try:
+                transport = await stack.enter_async_context(sse_client(url))
+                read_stream, write_stream = transport
+                session = await stack.enter_async_context(ClientSession(read_stream, write_stream))
 
-            await session.initialize()
+                await session.initialize()
 
-            # Discover tools
-            tools_result = await session.list_tools()
+                # Discover tools
+                tools_result = await session.list_tools()
+            except Exception:
+                await stack.aclose()
+                raise
             tools = []
             for tool in tools_result.tools:
                 tools.append({

src/memory.py

@@ -51,6 +51,8 @@ class MemoryManager:
         memories = []
         
         for msg in chat_history:
+            if not isinstance(msg, dict):
+                continue
             if msg.get("role") == "assistant":
                 content = str(msg.get("content", ""))
                 lines = content.split('\n')
@@ -135,6 +137,8 @@ class MemoryManager:
         """Ensure all entries have required fields."""
         validated = []
         for entry in entries:
+            if not isinstance(entry, dict):
+                continue
             if "id" not in entry:
                 entry["id"] = str(uuid.uuid4())
             if "timestamp" not in entry:

src/model_context.py

@@ -83,6 +83,7 @@ KNOWN_CONTEXT_WINDOWS = {
     'gemini-2.0-flash': 1048576,
     'gemini-1.5-pro': 1048576,
     'gemini-1.5-flash': 1048576,
+    'gemma-4': 262144,
     'gemma-3': 128000,
     'gemma-2': 8192,
 

src/model_discovery.py

@@ -16,6 +16,23 @@ _hosts_cache_time: float = 0
 _HOSTS_CACHE_TTL = 60  # seconds
 
 
+def _parse_tailscale_status(raw: str) -> Dict[str, Any]:
+    try:
+        data = json.loads(raw)
+    except (TypeError, json.JSONDecodeError):
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def _first_tailscale_ipv4(value: Any) -> Optional[str]:
+    if not isinstance(value, list):
+        return None
+    for ip in value:
+        if isinstance(ip, str) and "." in ip:
+            return ip
+    return None
+
+
 def discover_tailscale_hosts() -> List[str]:
     """Discover online Tailscale peers, returning their IPv4 addresses."""
     global _hosts_cache, _hosts_cache_time
@@ -33,17 +50,21 @@ def discover_tailscale_hosts() -> List[str]:
         if result.returncode != 0:
             return hosts
 
-        data = json.loads(result.stdout)
+        data = _parse_tailscale_status(result.stdout)
+        if not data:
+            return hosts
 
         # Add self
-        self_ips = data.get("Self", {}).get("TailscaleIPs", [])
-        for ip in self_ips:
-            if "." in ip:  # IPv4 only
-                hosts.append(ip)
-                break
+        self_data = data.get("Self") if isinstance(data.get("Self"), dict) else {}
+        self_ip = _first_tailscale_ipv4(self_data.get("TailscaleIPs"))
+        if self_ip:
+            hosts.append(self_ip)
 
         # Add online peers (skip funnel-ingress-nodes and android devices)
-        for peer in data.get("Peer", {}).values():
+        peers = data.get("Peer") if isinstance(data.get("Peer"), dict) else {}
+        for peer in peers.values():
+            if not isinstance(peer, dict):
+                continue
             if not peer.get("Online"):
                 continue
             hostname = peer.get("HostName", "")
@@ -52,11 +73,9 @@ def discover_tailscale_hosts() -> List[str]:
             os_name = peer.get("OS", "")
             if os_name == "android":
                 continue
-            peer_ips = peer.get("TailscaleIPs", [])
-            for ip in peer_ips:
-                if "." in ip:  # IPv4 only
-                    hosts.append(ip)
-                    break
+            peer_ip = _first_tailscale_ipv4(peer.get("TailscaleIPs"))
+            if peer_ip:
+                hosts.append(peer_ip)
 
         _hosts_cache = hosts
         _hosts_cache_time = now

src/pdf_form_doc.py

@@ -126,8 +126,13 @@ def _decode_name(enc: str) -> str:
     """Inverse of _encode_name."""
     import urllib.parse
     return urllib.parse.unquote(enc or "")
-_TEXT_VALUE_RE = re.compile(r'\*\*[^*]+:\*\*\s*(?P<value>.*)$')
-_CHOICE_VALUE_RE = re.compile(r'\*\*[^*]+\*\*\s*\[[^\]]*\]\s*:\s*(?P<value>.*)$')
+# Label segment is non-greedy (.+?) so labels containing '*' — the near-universal
+# required-field marker, e.g. "Email *" — are tolerated, while still splitting at
+# the FIRST ':**' / '**[' so a value that itself contains ':**' is preserved.
+# (The old [^*]+ refused to match any label with an asterisk and silently
+# dropped that field's value on export.)
+_TEXT_VALUE_RE = re.compile(r'\*\*.+?:\*\*\s*(?P<value>.*)$')
+_CHOICE_VALUE_RE = re.compile(r'\*\*.+?\*\*\s*\[[^\]]*\]\s*:\s*(?P<value>.*)$')
 _CHECKBOX_VALUE_RE = re.compile(r'^\s*\[(?P<state>[xX ])\]')
 
 _PLACEHOLDERS = {"_(empty)_", "_(not selected)_", "_(empty)_.", "_(unsigned)_"}

src/personal_docs.py

@@ -134,10 +134,12 @@ def retrieve_personal_keyword(personal_index: List[Dict], query: str, k: int = 5
 
     scored = []
     for f in personal_index:
-        for idx, ch in enumerate(f["chunks"]):
+        if not isinstance(f, dict):
+            continue
+        for idx, ch in enumerate(f.get("chunks") or []):
             score = len(q & tokenize(ch))
             if score > 0:
-                scored.append((score, f["name"], idx, ch))
+                scored.append((score, f.get("name", ""), idx, ch))
     scored.sort(key=lambda x: x[0], reverse=True)
 
     out = []
@@ -306,18 +308,17 @@ class PersonalDocsManager:
             # Refresh the index to exclude the removed directory
             self.refresh_index()
             
-            # If RAG manager is available, we should rebuild the index
-            # This is a simple approach - in production you might want more sophisticated removal
+            # Targeted delete of just this directory's chunks. This previously
+            # called rag_manager.rebuild_index(), which delete+recreates the
+            # entire shared collection (every owner + the base index) and then
+            # re-indexed only the remaining tracked dirs — ownerless and never
+            # personal_dir — a catastrophic wipe (#1660). remove_directory now
+            # removes exactly this directory's chunks and leaves the rest intact.
             if self.rag_manager:
                 try:
-                    logger.info("Rebuilding RAG index after directory removal")
-                    self.rag_manager.rebuild_index()
-                    # Re-index remaining directories
-                    for dir_path in self.indexed_directories:
-                        if os.path.exists(dir_path):
-                            self.rag_manager.index_personal_documents(dir_path)
+                    self.rag_manager.remove_directory(directory)
                 except Exception as e:
-                    logger.error(f"Failed to rebuild RAG index: {e}")
+                    logger.error(f"Failed to remove directory from RAG index: {e}")
         else:
             logger.info(f"Directory not in index: {directory}")
 

src/preset_manager.py

@@ -77,6 +77,9 @@ Use precise language. Show causal relationships explicitly. Quantify uncertainty
         try:
             with open(self.presets_file, 'r', encoding="utf-8") as f:
                 presets = json.load(f)
+            if not isinstance(presets, dict):
+                logger.error("Error loading presets: expected an object")
+                return self.DEFAULT_PRESETS.copy()
             custom = presets.get("custom") if isinstance(presets, dict) else None
             if isinstance(custom, dict) and "enabled" not in custom:
                 legacy_prompt = "You are a helpful, balanced assistant. Match your response style to the user's needs."

src/rag_vector.py

@@ -27,8 +27,14 @@ KEYWORD_WEIGHT = 0.3
 COLLECTION_NAME = "odysseus_rag"
 
 
-def _generate_doc_id(text: str) -> str:
-    return f"doc_{hashlib.sha256(text.encode('utf-8')).hexdigest()[:16]}"
+def _generate_doc_id(text: str, owner: str = "") -> str:
+    # Owner-scope the id so two owners can index byte-identical chunks
+    # without the second one's add early-returning on the first's id and
+    # being silently dropped from their owner-filtered search results.
+    # Empty owner reproduces the legacy text-only id so the unowned/base
+    # index keeps its existing ids and isn't re-churned.
+    key = f"{owner}\x00{text}" if owner else text
+    return f"doc_{hashlib.sha256(key.encode('utf-8')).hexdigest()[:16]}"
 
 
 class VectorRAG:
@@ -104,7 +110,7 @@ class VectorRAG:
             return False
 
         try:
-            doc_id = _generate_doc_id(text)
+            doc_id = _generate_doc_id(text, metadata.get("owner") or "")
             # Check if already exists
             existing = self._collection.get(ids=[doc_id])
             if existing["ids"]:
@@ -140,7 +146,7 @@ class VectorRAG:
             new_metas = []
             new_ids = []
             for t, m in valid:
-                doc_id = _generate_doc_id(t)
+                doc_id = _generate_doc_id(t, m.get("owner") or "")
                 existing = self._collection.get(ids=[doc_id])
                 if not existing["ids"]:
                     new_texts.append(t)
@@ -254,8 +260,11 @@ class VectorRAG:
             for i, doc in enumerate(all_docs["documents"]):
                 meta = all_docs["metadatas"][i]
                 if owner:
-                    doc_owner = meta.get("owner")
-                    if doc_owner and doc_owner != owner:
+                    # Match the primary path's strict where={"owner": owner}
+                    # filter. The old `if doc_owner and doc_owner != owner`
+                    # let docs with a missing/empty owner fall through, leaking
+                    # owner-less documents into another user's results.
+                    if meta.get("owner") != owner:
                         continue
                 doc_lower = doc.lower()
                 score = sum(1 for w in query_words if w in doc_lower)
@@ -374,20 +383,36 @@ class VectorRAG:
             return {'success': False, 'indexed_count': indexed, 'failed_count': failed, 'message': str(e)}
 
     def remove_directory(self, directory: str) -> Dict[str, Any]:
-        """Remove all chunks from a directory. O(1) per chunk via ChromaDB."""
+        """Remove all chunks under ``directory`` (recursively), and nothing else.
+
+        Selection is a Python-side path-boundary match on each chunk's stored
+        ``source`` full path, NOT a Chroma metadata ``where`` filter. No Chroma
+        metadata operator selects a scalar string by path prefix (``$contains``
+        targets document content / list membership, not a ``source`` substring),
+        and a plain substring would over-delete siblings — removing ``/docs``
+        must not touch ``/docs2`` or ``/docs_personal``. We therefore match
+        ``source == directory`` or ``source`` startswith ``directory + os.sep``,
+        the same boundary rule add_directory uses for exclusions. ``directory``
+        is abspath-normalized so it matches the absolute ``source`` that indexing
+        always stores, regardless of how the caller passed it in.
+        """
         if not self.healthy:
             return {"success": False, "message": "Collection not initialized"}
+        directory = os.path.abspath(directory)
         try:
-            # Use ChromaDB where filter to find all docs from this directory
-            results = self._collection.get(
-                where={"source": {"$contains": directory}} if "/" in directory else {"directory": directory},
-                include=["metadatas"],
-            )
-            if not results['ids']:
+            results = self._collection.get(include=["metadatas"])
+            ids = [
+                results["ids"][i]
+                for i, m in enumerate(results["metadatas"])
+                if isinstance(m, dict)
+                and isinstance(m.get("source"), str)
+                and (m["source"] == directory or m["source"].startswith(directory + os.sep))
+            ]
+            if not ids:
                 return {"success": True, "removed_count": 0, "message": "No docs found"}
 
-            self._collection.delete(ids=results['ids'])
-            n = len(results['ids'])
+            self._collection.delete(ids=ids)
+            n = len(ids)
             logger.info(f"Removed {n} chunks from {directory}")
             return {"success": True, "removed_count": n, "message": f"Removed {n} chunks"}
         except Exception as e:

src/research_handler.py

@@ -461,6 +461,8 @@ class ResearchHandler:
         seen = set()
         sources = []
         for f in findings:
+            if not isinstance(f, dict):
+                continue
             url = f.get("url", "")
             title = f.get("title", "") or url
             summary = f.get("summary", "") or f.get("evidence", "")
@@ -479,6 +481,8 @@ class ResearchHandler:
         try:
             items = []
             for f in findings:
+                if not isinstance(f, dict):
+                    continue
                 url = f.get("url", "")
                 title = f.get("title", "") or "Untitled"
                 summary = f.get("summary", "")

src/research_utils.py

@@ -55,7 +55,7 @@ LOW_QUALITY_MARKERS = [
 def is_low_quality(summary: str) -> bool:
     """Check if a finding summary indicates useless or irrelevant content."""
     try:
-        if not summary:
+        if not isinstance(summary, str) or not summary:
             return True
         low = summary.lower()
         return any(marker in low for marker in LOW_QUALITY_MARKERS)