mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-11-06 13:12:27 +01:00
Compare commits
93
Commits
926a4c59cb
...
aa5e3f6884
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aa5e3f6884 | ||
|
|
fc220f760f | ||
|
|
2d94e38d23 | ||
|
|
03ddc5d2c4 | ||
|
|
3175d7ca21 | ||
|
|
d818117d4c | ||
|
|
6b2618dab4 | ||
|
|
0051023056 | ||
|
|
8783f12c4c | ||
|
|
82c09dd768 | ||
|
|
c3bf32d1b1 | ||
|
|
df3864bd15 | ||
|
|
ffeb7d8c97 | ||
|
|
a6b7a7bc60 | ||
|
|
0cc1814658 | ||
|
|
8051e25c65 | ||
|
|
0ad5cd783b | ||
|
|
953305a5af | ||
|
|
15a3b71802 | ||
|
|
e68d0448b8 | ||
|
|
db3a5c17b0 | ||
|
|
f39c87561b | ||
|
|
f7df069ca1 | ||
|
|
eacb99f963 | ||
|
|
fb1341b629 | ||
|
|
19b6cbac12 | ||
|
|
258fe455eb | ||
|
|
667c663668 | ||
|
|
0283216e67 | ||
|
|
77313170c6 | ||
|
|
0d25dfb5f4 | ||
|
|
6df0f5e6df | ||
|
|
382d49d887 | ||
|
|
0d88c9989e | ||
|
|
5462e36d10 | ||
|
|
5b0c7442ee | ||
|
|
441905bc13 | ||
|
|
24be3f3ca8 | ||
|
|
cf60a14d74 | ||
|
|
bd0845e081 | ||
|
|
dc3421c34e | ||
|
|
1f743970dd | ||
|
|
c9361262df | ||
|
|
0e6cbd8315 | ||
|
|
5c6bd0fc2b | ||
|
|
57abe69173 | ||
|
|
583df3dd6a | ||
|
|
70103d8719 | ||
|
|
8450cee02a | ||
|
|
1ecd113808 | ||
|
|
4d1829add0 | ||
|
|
5fd71f68e8 | ||
|
|
0ec8415f0e | ||
|
|
fd37ccebae | ||
|
|
abbc073429 | ||
|
|
a5282e9748 | ||
|
|
35c40bce75 | ||
|
|
77b63ed942 | ||
|
|
1f2a06facd | ||
|
|
69d6fe44b3 | ||
|
|
7af168fa59 | ||
|
|
7ce2db2771 | ||
|
|
cb114d6e87 | ||
|
|
50a486b608 | ||
|
|
b6843c7621 | ||
|
|
b54468291e | ||
|
|
66c9349ee3 | ||
|
|
c3fd969965 | ||
|
|
ce7f5dbbdd | ||
|
|
b2291fad49 | ||
|
|
56656de5bc | ||
|
|
54a221b367 | ||
|
|
4771d80eb2 | ||
|
|
80de69ebb0 | ||
|
|
4019283eba | ||
|
|
97f855b40d | ||
|
|
5ee30cc144 | ||
|
|
1adf21a7e5 | ||
|
|
e73545f64f | ||
|
|
34918d9921 | ||
|
|
e392be0d65 | ||
|
|
dc6711b3c5 | ||
|
|
1a763b0539 | ||
|
|
5452bc96b1 | ||
|
|
9c68ceafeb | ||
|
|
a8395b4e4c | ||
|
|
adde94e430 | ||
|
|
1507d140b8 | ||
|
|
7504fedb17 | ||
|
|
257f7ee7b2 | ||
|
|
8852c7ea4a | ||
|
|
c39d8db12a | ||
|
|
3505a5ff27 |
@@ -0,0 +1,103 @@
|
||||
name: Bug Report
|
||||
description: Report a reproducible bug in Odysseus.
|
||||
labels: ["bug"]
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||
Duplicate reports slow things down.
|
||||
|
||||
For security vulnerabilities, **do not open a public issue** —
|
||||
use [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new)
|
||||
and read [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md) first.
|
||||
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
options:
|
||||
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and did not find an existing report of this bug.
|
||||
required: true
|
||||
- label: This is **not** a security vulnerability. (Vulnerabilities go to [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new) — see [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md).)
|
||||
required: true
|
||||
- label: I am running the latest code from `main`.
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: install-method
|
||||
attributes:
|
||||
label: Install Method
|
||||
options:
|
||||
- Docker (docker compose up)
|
||||
- Manual Python install (pip / venv)
|
||||
- Windows native (launch-windows.ps1)
|
||||
- macOS app (build-macos-app.sh / start-macos.sh)
|
||||
- Other (describe in the reproduction steps below)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: os
|
||||
attributes:
|
||||
label: Operating System
|
||||
options:
|
||||
- Linux
|
||||
- macOS
|
||||
- Windows
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: steps
|
||||
attributes:
|
||||
label: Steps to Reproduce
|
||||
description: Exact steps that reliably trigger the bug. The more specific, the faster this gets fixed.
|
||||
placeholder: |
|
||||
1. Go to ...
|
||||
2. Click / type ...
|
||||
3. Observe ...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
label: Expected Behaviour
|
||||
description: What should have happened?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: actual
|
||||
attributes:
|
||||
label: Actual Behaviour
|
||||
description: What actually happened? Include the full error message if there is one.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: logs
|
||||
attributes:
|
||||
label: Logs / Screenshots
|
||||
description: Paste relevant terminal output or attach screenshots. Remove API keys, passwords, and personal data before pasting.
|
||||
render: text
|
||||
|
||||
- type: input
|
||||
id: model-backend
|
||||
attributes:
|
||||
label: Model / Backend (if relevant)
|
||||
description: "e.g. Ollama + llama3.2:latest, vLLM + mistral-7b, OpenAI API, Anthropic API"
|
||||
placeholder: "Ollama + llama3.2:latest"
|
||||
|
||||
- type: textarea
|
||||
id: additional-info
|
||||
attributes:
|
||||
label: Additional Information
|
||||
description: Anything else that might help — browser console errors, related issues, things you already tried, or environment quirks.
|
||||
placeholder: |
|
||||
- Any other context goes here.
|
||||
- If you are willing to submit a PR that fixes this, mention it here.
|
||||
@@ -0,0 +1,13 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Question / Need Help
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/q-a
|
||||
about: Ask how-to questions, setup help, and model configuration questions here. Issues are for confirmed bugs and concrete proposals only.
|
||||
|
||||
- name: Idea or Suggestion
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas
|
||||
about: Discuss ideas and gauge interest before opening a formal feature request. If there is already a discussion, link it in your feature request.
|
||||
|
||||
- name: Security Vulnerability
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new
|
||||
about: Report vulnerabilities privately via GitHub Security Advisories — never as a public issue. Read SECURITY.md before reporting.
|
||||
@@ -0,0 +1,90 @@
|
||||
name: Feature Request
|
||||
description: Propose a new feature or a concrete improvement to Odysseus.
|
||||
labels: ["enhancement"]
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||
Feature requests that duplicate [ROADMAP.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/ROADMAP.md)
|
||||
or an existing open issue will be closed as duplicates.
|
||||
|
||||
If your idea needs community input before it becomes a concrete proposal,
|
||||
start a [discussion](https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas) instead.
|
||||
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
options:
|
||||
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and this has not already been proposed.
|
||||
required: true
|
||||
- label: I searched [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and this is not already being debated there.
|
||||
required: true
|
||||
- label: This is a concrete, actionable proposal — not a vague "it would be nice if..." request.
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: area
|
||||
attributes:
|
||||
label: Area
|
||||
description: Which part of the application does this affect?
|
||||
options:
|
||||
- Chat / Agent
|
||||
- Email
|
||||
- Calendar
|
||||
- Documents / RAG
|
||||
- Memory
|
||||
- Cookbook / Local Models / GPU
|
||||
- Search
|
||||
- Notes / Editor
|
||||
- Auth / Security
|
||||
- Docker / Deployment
|
||||
- UI / Frontend
|
||||
- API / Backend
|
||||
- MCP
|
||||
- Testing / CI
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem or Motivation
|
||||
description: What problem does this solve, or what use case does it enable? Be specific — "it would be better" is not enough.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Proposed Solution
|
||||
description: Describe the behaviour or change you want to see. Include API shape, UI sketch, or code snippets if that helps make it concrete.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Alternatives Considered
|
||||
description: What other approaches did you consider and why did you rule them out? If there is an existing workaround, describe it.
|
||||
|
||||
- type: textarea
|
||||
id: prior-art
|
||||
attributes:
|
||||
label: Prior Art / Related Issues
|
||||
description: Link any related issues, discussions, or external references that informed this proposal.
|
||||
|
||||
- type: dropdown
|
||||
id: willing_to_implement
|
||||
attributes:
|
||||
label: Are you willing to implement this?
|
||||
options:
|
||||
- "Yes — I can open a PR"
|
||||
- "Partially — I can help but need guidance"
|
||||
- "No — I am only filing the request"
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,38 @@
|
||||
## Summary
|
||||
|
||||
<!-- One paragraph: what changed and why. "Fixed bug" and "Added feature" are not summaries. -->
|
||||
|
||||
## Linked Issue
|
||||
|
||||
<!-- Every PR should be linked to an issue.
|
||||
Use one of: Fixes #NNN | Part of #NNN | Closes #NNN -->
|
||||
|
||||
Fixes #
|
||||
|
||||
## Type of Change
|
||||
|
||||
- [ ] Bug fix (non-breaking — fixes a confirmed issue)
|
||||
- [ ] New feature (non-breaking — adds new behaviour)
|
||||
- [ ] Breaking change (changes or removes existing behaviour)
|
||||
- [ ] Refactor / cleanup (behaviour unchanged)
|
||||
- [ ] Documentation only
|
||||
- [ ] CI / tooling / configuration
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues) and [open PRs](https://github.com/pewdiepie-archdaemon/odysseus/pulls) — this is not a duplicate.
|
||||
- [ ] This PR targets `main`
|
||||
- [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
|
||||
|
||||
## How to Test
|
||||
|
||||
<!-- Step-by-step instructions a reviewer can follow to verify this works.
|
||||
Do not leave this empty — a PR without test steps will be sent back. -->
|
||||
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
|
||||
## Screenshots (UI changes only)
|
||||
|
||||
<!-- Drag and drop images or a screen recording here. Delete this section if not a UI change. -->
|
||||
@@ -1,7 +1,10 @@
|
||||
# Odysseus
|
||||
|
||||
```
|
||||
───────────────────────────────────────────────
|
||||
⊹ ࣪ ˖ ૮( ˶ᵔ ᵕ ᵔ˶ )っ Odysseus vers. 1.0
|
||||
───────────────────────────────────────────────
|
||||
```
|
||||
|
||||

|
||||
|
||||
@@ -219,6 +222,13 @@ Ollama must listen outside its own loopback interface:
|
||||
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
||||
```
|
||||
|
||||
This connects Odysseus in Docker to an Ollama server that is already running on
|
||||
your host machine; it does not start Ollama inside the container.
|
||||
`host.docker.internal` is Docker's hostname for the host machine from inside the
|
||||
container. Cookbook **Serve** is a separate workflow for serving downloaded
|
||||
models through Odysseus/llama.cpp, so Windows users with an existing Ollama
|
||||
install usually only need to add the endpoint in Settings.
|
||||
|
||||
**Useful checks.**
|
||||
|
||||
```bash
|
||||
|
||||
+6
-1
@@ -79,7 +79,12 @@ services:
|
||||
restart: unless-stopped
|
||||
|
||||
searxng:
|
||||
image: docker.io/searxng/searxng:latest
|
||||
# Pinned, not :latest — odysseus waits on searxng's healthcheck
|
||||
# (depends_on: condition: service_healthy), so a broken upstream `latest`
|
||||
# tag blocks the whole app from starting. 2026.6.2 crashes on boot with
|
||||
# `KeyError: 'default_doi_resolver'`, failing the healthcheck (issue #1414).
|
||||
# Bump this deliberately after verifying a newer tag boots clean.
|
||||
image: docker.io/searxng/searxng:2026.5.31-7159b8aed
|
||||
entrypoint:
|
||||
- /bin/sh
|
||||
- -c
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 52 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 51 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 90 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 125 KiB |
@@ -13,6 +13,10 @@ SEARCH_TIMEOUT = 30
|
||||
|
||||
def truncate(text: str, limit: int = MAX_OUTPUT_CHARS) -> str:
|
||||
"""Truncate text to *limit* characters with a suffix note."""
|
||||
if not isinstance(text, str):
|
||||
# Tool output is occasionally None or a non-string; len(None) would
|
||||
# raise. Coerce so this shared helper never crashes a tool response.
|
||||
text = "" if text is None else str(text)
|
||||
if len(text) > limit:
|
||||
return text[:limit] + f"\n... (truncated, {len(text)} chars total)"
|
||||
return text
|
||||
|
||||
@@ -417,6 +417,11 @@ def _list_emails(folder="INBOX", max_results=20, unresponded_only=False,
|
||||
status, data = conn.uid("SEARCH", None, "(UNSEEN UNANSWERED)")
|
||||
elif unread_only:
|
||||
status, data = conn.uid("SEARCH", None, "(UNSEEN)")
|
||||
elif unresponded_only:
|
||||
# Was missing — unresponded_only=True (without unread_only) fell through
|
||||
# to "ALL" and returned answered mail too, despite the documented
|
||||
# "emails without replies" behaviour.
|
||||
status, data = conn.uid("SEARCH", None, "(UNANSWERED)")
|
||||
else:
|
||||
# Include read too — IMAP search "ALL" returns the entire folder
|
||||
status, data = conn.uid("SEARCH", None, "ALL")
|
||||
|
||||
@@ -101,7 +101,8 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
return [TextContent(type="text", text=f"Error: {e}")]
|
||||
|
||||
elif action == "add_directory":
|
||||
directory = arguments.get("directory", "").strip()
|
||||
_dir = arguments.get("directory")
|
||||
directory = _dir.strip() if isinstance(_dir, str) else ""
|
||||
if not directory:
|
||||
return [TextContent(type="text", text="Error: add_directory needs a directory path")]
|
||||
directory = os.path.expanduser(directory)
|
||||
@@ -112,14 +113,27 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
try:
|
||||
result = _rag_manager.index_personal_documents(directory)
|
||||
indexed = result.get("indexed_count", 0) if isinstance(result, dict) else 0
|
||||
# Record the directory so `list` and `remove_directory` can see it.
|
||||
# Indexing was just done above, so pass index=False to avoid a second
|
||||
# (ownerless) pass. Without this the directory was indexed but never
|
||||
# tracked in indexed_directories, so it was invisible/unremovable.
|
||||
if _personal_docs_manager and hasattr(_personal_docs_manager, "add_directory"):
|
||||
try:
|
||||
_personal_docs_manager.add_directory(directory, index=False)
|
||||
except Exception:
|
||||
pass
|
||||
return [TextContent(type="text", text=f"Directory '{directory}' added to RAG index ({indexed} chunks indexed)")]
|
||||
except Exception as e:
|
||||
return [TextContent(type="text", text=f"Error: Failed to index directory: {e}")]
|
||||
|
||||
elif action == "remove_directory":
|
||||
directory = arguments.get("directory", "").strip()
|
||||
_dir = arguments.get("directory")
|
||||
directory = _dir.strip() if isinstance(_dir, str) else ""
|
||||
if not directory:
|
||||
return [TextContent(type="text", text="Error: remove_directory needs a directory path")]
|
||||
# Expand ~ to match add_directory, which indexes the expanded path.
|
||||
# Without this, removing "~/docs" never matches the stored absolute path.
|
||||
directory = os.path.expanduser(directory)
|
||||
if not _personal_docs_manager:
|
||||
return [TextContent(type="text", text="Error: Personal docs manager not available")]
|
||||
try:
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@ Type=simple
|
||||
# CHANGE THESE to match your user and install path:
|
||||
User=YOURUSER
|
||||
WorkingDirectory=/home/YOURUSER/odysseus-ui
|
||||
ExecStart=/home/YOURUSER/odysseus-ui/venv/bin/uvicorn app:app --port 8000 --host 0.0.0.0
|
||||
ExecStart=/home/YOURUSER/odysseus-ui/venv/bin/uvicorn app:app --port 7000 --host 0.0.0.0
|
||||
Restart=always
|
||||
RestartSec=3
|
||||
EnvironmentFile=-/home/YOURUSER/odysseus-ui/.env
|
||||
|
||||
@@ -528,13 +528,20 @@ def setup_calendar_routes() -> APIRouter:
|
||||
owner = _require_user(request)
|
||||
from routes.prefs_routes import _load_for_user
|
||||
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||
caldav_password = cfg.get("password") or ""
|
||||
if caldav_password:
|
||||
try:
|
||||
from src.secret_storage import decrypt
|
||||
caldav_password = decrypt(caldav_password)
|
||||
except Exception:
|
||||
pass
|
||||
# Surface url+username but never hand the password back to the
|
||||
# client — saved-state UI shouldn't leak the credential.
|
||||
return {
|
||||
"url": cfg.get("url", "") or "",
|
||||
"username": cfg.get("username", "") or "",
|
||||
"password": "",
|
||||
"has_password": bool(cfg.get("password")),
|
||||
"has_password": bool(caldav_password),
|
||||
"local": not bool(cfg.get("url")),
|
||||
}
|
||||
|
||||
@@ -553,12 +560,20 @@ def setup_calendar_routes() -> APIRouter:
|
||||
prefs.pop("caldav", None)
|
||||
_save_for_user(owner, prefs)
|
||||
return {"ok": True, "cleared": True}
|
||||
cfg["url"] = body.get("url", "").strip()
|
||||
from src.caldav_sync import validate_caldav_url
|
||||
try:
|
||||
cfg["url"] = validate_caldav_url(body.get("url", ""))
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e))
|
||||
cfg["username"] = (body.get("username") or "").strip()
|
||||
# Preserve the stored password when the client sends an empty
|
||||
# one (edit form re-submitted without re-typing the password).
|
||||
if body.get("password"):
|
||||
cfg["password"] = body["password"]
|
||||
from src.secret_storage import encrypt
|
||||
cfg["password"] = encrypt(body["password"])
|
||||
elif cfg.get("password"):
|
||||
from src.secret_storage import encrypt
|
||||
cfg["password"] = encrypt(cfg["password"])
|
||||
prefs["caldav"] = cfg
|
||||
_save_for_user(owner, prefs)
|
||||
return {"ok": True}
|
||||
@@ -585,9 +600,21 @@ def setup_calendar_routes() -> APIRouter:
|
||||
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||
url = url or (cfg.get("url") or "")
|
||||
user = user or (cfg.get("username") or "")
|
||||
pw = pw or (cfg.get("password") or "")
|
||||
if not pw:
|
||||
pw = cfg.get("password") or ""
|
||||
if pw:
|
||||
try:
|
||||
from src.secret_storage import decrypt
|
||||
pw = decrypt(pw)
|
||||
except Exception:
|
||||
pass
|
||||
if not (url and user and pw):
|
||||
return {"ok": False, "error": "Missing URL, username, or password"}
|
||||
from src.caldav_sync import validate_caldav_url
|
||||
try:
|
||||
url = validate_caldav_url(url)
|
||||
except ValueError as e:
|
||||
return {"ok": False, "error": str(e)}
|
||||
import httpx
|
||||
propfind_body = (
|
||||
'<?xml version="1.0" encoding="UTF-8"?>\n'
|
||||
@@ -595,7 +622,7 @@ def setup_calendar_routes() -> APIRouter:
|
||||
'</d:prop></d:propfind>'
|
||||
)
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=8.0, follow_redirects=True) as cx:
|
||||
async with httpx.AsyncClient(timeout=8.0, follow_redirects=False, trust_env=False) as cx:
|
||||
r = await cx.request(
|
||||
"PROPFIND", url,
|
||||
auth=(user, pw),
|
||||
@@ -612,6 +639,8 @@ def setup_calendar_routes() -> APIRouter:
|
||||
return {"ok": False, "error": "Forbidden — user can't access that URL"}
|
||||
if r.status_code == 404:
|
||||
return {"ok": False, "error": "Not found — check the URL path"}
|
||||
if 300 <= r.status_code < 400:
|
||||
return {"ok": False, "error": "Redirects are not followed for CalDAV safety; use the final URL"}
|
||||
return {"ok": False, "error": f"HTTP {r.status_code}"}
|
||||
except httpx.ConnectError as e:
|
||||
return {"ok": False, "error": f"Connection refused: {e}"[:200]}
|
||||
@@ -758,6 +787,16 @@ def setup_calendar_routes() -> APIRouter:
|
||||
)
|
||||
db.add(ev)
|
||||
db.commit()
|
||||
if cal.source == "caldav":
|
||||
# Push the new event to the remote so it appears on the user's
|
||||
# other devices — the sync is otherwise pull-only (#800).
|
||||
from src.caldav_writeback import writeback_event
|
||||
await writeback_event(owner, cal.source, cal.id, {
|
||||
"uid": uid, "summary": data.summary, "description": data.description,
|
||||
"location": data.location, "dtstart": dtstart, "dtend": dtend,
|
||||
"all_day": data.all_day, "is_utc": _is_utc and not data.all_day,
|
||||
"rrule": data.rrule or "",
|
||||
})
|
||||
return {"ok": True, "uid": uid}
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -804,6 +843,14 @@ def setup_calendar_routes() -> APIRouter:
|
||||
if data.color is not None:
|
||||
ev.color = data.color if data.color else None
|
||||
db.commit()
|
||||
cal = db.query(CalendarCal).filter(CalendarCal.id == ev.calendar_id).first()
|
||||
if cal and cal.source == "caldav":
|
||||
from src.caldav_writeback import writeback_event
|
||||
await writeback_event(owner, cal.source, cal.id, {
|
||||
"uid": ev.uid, "summary": ev.summary, "description": ev.description,
|
||||
"location": ev.location, "dtstart": ev.dtstart, "dtend": ev.dtend,
|
||||
"all_day": ev.all_day, "is_utc": ev.is_utc, "rrule": ev.rrule or "",
|
||||
})
|
||||
return {"ok": True}
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -824,8 +871,15 @@ def setup_calendar_routes() -> APIRouter:
|
||||
db = SessionLocal()
|
||||
try:
|
||||
ev = _get_or_404_event(db, base_uid, owner)
|
||||
# Capture what the remote push needs BEFORE the row is gone.
|
||||
_cal = db.query(CalendarCal).filter(CalendarCal.id == ev.calendar_id).first()
|
||||
_is_caldav = bool(_cal and _cal.source == "caldav")
|
||||
_cal_id, _ev_uid = ev.calendar_id, ev.uid
|
||||
db.delete(ev)
|
||||
db.commit()
|
||||
if _is_caldav:
|
||||
from src.caldav_writeback import writeback_event
|
||||
await writeback_event(owner, "caldav", _cal_id, {"uid": _ev_uid}, delete=True)
|
||||
return {"ok": True}
|
||||
except HTTPException:
|
||||
raise
|
||||
|
||||
+47
-15
@@ -148,8 +148,9 @@ async def auto_name_session(session_manager, sess):
|
||||
if not first_msg:
|
||||
return
|
||||
|
||||
owner = getattr(sess, "owner", None)
|
||||
t_url, t_model, t_headers = resolve_task_endpoint(
|
||||
sess.endpoint_url, sess.model, sess.headers,
|
||||
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||
)
|
||||
if not t_model:
|
||||
logger.debug("[auto-name] No model provided, skipping")
|
||||
@@ -311,7 +312,24 @@ def fire_message_event(request, webhook_manager, session_id: str, sess, message:
|
||||
fire_event("message_sent", user)
|
||||
|
||||
|
||||
def resolve_session_auth(sess, session_id: str):
|
||||
def _session_url_matches_endpoint(session_url: str, endpoint_base: str) -> bool:
|
||||
if not session_url or not endpoint_base:
|
||||
return False
|
||||
try:
|
||||
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||
|
||||
sess_url = session_url.rstrip("/")
|
||||
base = normalize_base(endpoint_base).rstrip("/")
|
||||
return sess_url in {
|
||||
base,
|
||||
base + "/chat/completions",
|
||||
build_chat_url(base).rstrip("/"),
|
||||
}
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def resolve_session_auth(sess, session_id: str, owner: Optional[str] = None):
|
||||
"""Ensure session has auth headers — resolve from endpoint DB if missing."""
|
||||
has_auth = sess.headers and isinstance(sess.headers, dict) and any(
|
||||
k.lower() in ('authorization', 'x-api-key') for k in sess.headers
|
||||
@@ -320,19 +338,33 @@ def resolve_session_auth(sess, session_id: str):
|
||||
return
|
||||
|
||||
try:
|
||||
from src.endpoint_resolver import build_headers
|
||||
from src.endpoint_resolver import build_headers, normalize_base
|
||||
db = SessionLocal()
|
||||
try:
|
||||
domain = sess.endpoint_url.split("//")[1].split("/")[0] if "//" in sess.endpoint_url else ""
|
||||
if domain:
|
||||
ep = db.query(ModelEndpoint).filter(ModelEndpoint.base_url.contains(domain)).first()
|
||||
if ep and ep.api_key:
|
||||
sess.headers = build_headers(ep.api_key, ep.base_url)
|
||||
db.query(DBSession).filter(DBSession.id == session_id).update(
|
||||
{"headers": json.dumps(sess.headers)}
|
||||
)
|
||||
db.commit()
|
||||
logger.info(f"Resolved and persisted auth headers for session {session_id} from endpoint {ep.name}")
|
||||
target_url = getattr(sess, "endpoint_url", "") or ""
|
||||
if not target_url:
|
||||
return
|
||||
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||
if owner:
|
||||
# Missing headers usually means "recover from the saved endpoint".
|
||||
# Scope that lookup to the session owner, otherwise two users
|
||||
# with similar endpoint URLs can borrow each other's API key.
|
||||
from src.auth_helpers import owner_filter
|
||||
q = owner_filter(q, ModelEndpoint, owner)
|
||||
for ep in q.all():
|
||||
if not _session_url_matches_endpoint(target_url, ep.base_url or ""):
|
||||
continue
|
||||
if not ep.api_key:
|
||||
return
|
||||
base = normalize_base(ep.base_url or "")
|
||||
sess.headers = build_headers(ep.api_key, base)
|
||||
update_q = db.query(DBSession).filter(DBSession.id == session_id)
|
||||
if owner:
|
||||
update_q = update_q.filter(DBSession.owner == owner)
|
||||
update_q.update({"headers": sess.headers})
|
||||
db.commit()
|
||||
logger.info(f"Resolved and persisted auth headers for session {session_id} from endpoint {ep.name}")
|
||||
return
|
||||
finally:
|
||||
db.close()
|
||||
except Exception as e:
|
||||
@@ -806,7 +838,7 @@ def run_post_response_tasks(
|
||||
from services.memory.memory_extractor import extract_and_store
|
||||
from src.task_endpoint import resolve_task_endpoint
|
||||
t_url, t_model, t_headers = resolve_task_endpoint(
|
||||
sess.endpoint_url, sess.model, sess.headers,
|
||||
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||
)
|
||||
asyncio.create_task(extract_and_store(
|
||||
sess, memory_manager, memory_vector,
|
||||
@@ -843,7 +875,7 @@ def run_post_response_tasks(
|
||||
from services.memory.skill_extractor import maybe_extract_skill
|
||||
from src.task_endpoint import resolve_task_endpoint
|
||||
s_url, s_model, s_headers = resolve_task_endpoint(
|
||||
sess.endpoint_url, sess.model, sess.headers,
|
||||
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||
)
|
||||
logger.debug("[skill-extract] dispatching extractor (model=%s)", s_model)
|
||||
asyncio.create_task(maybe_extract_skill(
|
||||
|
||||
+29
-14
@@ -72,13 +72,17 @@ def _session_url_matches_endpoint(session_url: str, endpoint_base: str) -> bool:
|
||||
return sess in variants or sess.startswith(base + "/")
|
||||
|
||||
|
||||
def _clear_orphaned_session_endpoint(sess) -> bool:
|
||||
def _clear_orphaned_session_endpoint(sess, owner: str | None = None) -> bool:
|
||||
"""Clear a session model if its endpoint was deleted from ModelEndpoint."""
|
||||
if not getattr(sess, "endpoint_url", ""):
|
||||
return False
|
||||
db = SessionLocal()
|
||||
try:
|
||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
||||
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||
if owner:
|
||||
from src.auth_helpers import owner_filter
|
||||
q = owner_filter(q, ModelEndpoint, owner)
|
||||
endpoints = q.all()
|
||||
for ep in endpoints:
|
||||
if _session_url_matches_endpoint(sess.endpoint_url or "", ep.base_url or ""):
|
||||
return False
|
||||
@@ -118,7 +122,7 @@ def _endpoint_cache_contains_model(endpoint, model: str) -> bool:
|
||||
return wanted in {str(item).strip() for item in models}
|
||||
|
||||
|
||||
def _is_image_generation_session(sess) -> bool:
|
||||
def _is_image_generation_session(sess, owner: str | None = None) -> bool:
|
||||
"""Whether this chat session should bypass text chat and generate images.
|
||||
|
||||
Model-name prefixes are explicit image models. Endpoint type is only used
|
||||
@@ -137,7 +141,11 @@ def _is_image_generation_session(sess) -> bool:
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
||||
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||
if owner:
|
||||
from src.auth_helpers import owner_filter
|
||||
q = owner_filter(q, ModelEndpoint, owner)
|
||||
endpoints = q.all()
|
||||
for endpoint in endpoints:
|
||||
if (getattr(endpoint, "model_type", None) or "llm") != "image":
|
||||
continue
|
||||
@@ -152,7 +160,7 @@ def _is_image_generation_session(sess) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _recover_empty_session_model(sess, session_id: str) -> bool:
|
||||
def _recover_empty_session_model(sess, session_id: str, owner: str | None = None) -> bool:
|
||||
"""Re-populate sess.model from the matching endpoint's cached models.
|
||||
|
||||
Covers the window between endpoint setup and the first chat send: the
|
||||
@@ -172,7 +180,11 @@ def _recover_empty_session_model(sess, session_id: str) -> bool:
|
||||
# cached model is the most defensible default.
|
||||
ep = None
|
||||
if getattr(sess, "endpoint_url", ""):
|
||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
||||
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||
if owner:
|
||||
from src.auth_helpers import owner_filter
|
||||
q = owner_filter(q, ModelEndpoint, owner)
|
||||
endpoints = q.all()
|
||||
for cand in endpoints:
|
||||
if _session_url_matches_endpoint(sess.endpoint_url or "", cand.base_url or ""):
|
||||
ep = cand
|
||||
@@ -251,13 +263,14 @@ def setup_chat_routes(
|
||||
sess = session_manager.get_session(session)
|
||||
except KeyError:
|
||||
raise HTTPException(404, f"Session '{session}' not found")
|
||||
if _clear_orphaned_session_endpoint(sess):
|
||||
owner = get_current_user(request)
|
||||
if _clear_orphaned_session_endpoint(sess, owner=owner):
|
||||
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
||||
|
||||
# Empty model + live endpoint = setup race (Issue #587). Repair from
|
||||
# the endpoint's cached model list before privilege checks, which
|
||||
# otherwise see "" and behave inconsistently with the allowlist.
|
||||
_recover_empty_session_model(sess, session)
|
||||
_recover_empty_session_model(sess, session, owner=owner)
|
||||
if not getattr(sess, "model", "").strip():
|
||||
raise HTTPException(
|
||||
400,
|
||||
@@ -401,7 +414,8 @@ def setup_chat_routes(
|
||||
# but BEFORE loading. Prevents cross-user session hijack.
|
||||
_verify_session_owner(request, session)
|
||||
sess = session_manager.get_session(session)
|
||||
if _clear_orphaned_session_endpoint(sess):
|
||||
owner = get_current_user(request)
|
||||
if _clear_orphaned_session_endpoint(sess, owner=owner):
|
||||
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
||||
# Issue #587: picker shows a model from the endpoint cache but
|
||||
# s.model never made it onto the DB row (first-send race after
|
||||
@@ -409,7 +423,7 @@ def setup_chat_routes(
|
||||
# the first cached model off the matching endpoint so the
|
||||
# upstream isn't called with model="" (which surfaces as a
|
||||
# generic 401/503).
|
||||
_recover_empty_session_model(sess, session)
|
||||
_recover_empty_session_model(sess, session, owner=owner)
|
||||
if not getattr(sess, "model", "").strip():
|
||||
raise HTTPException(
|
||||
400,
|
||||
@@ -431,7 +445,7 @@ def setup_chat_routes(
|
||||
_enforce_chat_privileges(request, sess)
|
||||
|
||||
# Ensure session has auth headers
|
||||
resolve_session_auth(sess, session)
|
||||
resolve_session_auth(sess, session, owner=get_current_user(request))
|
||||
|
||||
# Check for research_pending BEFORE mode persist overwrites it
|
||||
do_research = str(use_research).lower() == "true"
|
||||
@@ -711,6 +725,7 @@ def setup_chat_routes(
|
||||
prior_findings=_prior_findings,
|
||||
prior_urls=_prior_urls,
|
||||
on_complete=_on_research_done,
|
||||
owner=_user,
|
||||
)
|
||||
|
||||
_heartbeat_counter = 0
|
||||
@@ -767,7 +782,7 @@ def setup_chat_routes(
|
||||
# output. Resolved once per request.
|
||||
try:
|
||||
from src.endpoint_resolver import resolve_chat_fallback_candidates
|
||||
_fallback_candidates = resolve_chat_fallback_candidates()
|
||||
_fallback_candidates = resolve_chat_fallback_candidates(owner=_user)
|
||||
except Exception:
|
||||
_fallback_candidates = []
|
||||
|
||||
@@ -780,7 +795,7 @@ def setup_chat_routes(
|
||||
_model_info["character_name"] = ctx.preset.character_name
|
||||
yield f'data: {json.dumps(_model_info)}\n\n'
|
||||
|
||||
if _is_image_generation_session(sess):
|
||||
if _is_image_generation_session(sess, owner=_user):
|
||||
from src.settings import get_setting
|
||||
if not get_setting("image_gen_enabled", True):
|
||||
yield f'data: {json.dumps({"delta": "Image generation is disabled by the administrator."})}\n\n'
|
||||
@@ -791,7 +806,7 @@ def setup_chat_routes(
|
||||
_user_msg = message or ""
|
||||
yield f'data: {json.dumps({"type": "tool_start", "tool": "generate_image", "command": _user_msg[:100]})}\n\n'
|
||||
yield ": heartbeat\n\n"
|
||||
_img_result = await do_generate_image(f"{_user_msg}\n{sess.model}", session)
|
||||
_img_result = await do_generate_image(f"{_user_msg}\n{sess.model}", session, owner=_user)
|
||||
_img_output = _img_result.get("results", _img_result.get("error", ""))
|
||||
_img_tool_data = {"type": "tool_output", "tool": "generate_image", "command": _user_msg[:100], "output": _img_output, "exit_code": 0 if "error" not in _img_result else 1}
|
||||
for _k in ("image_url", "image_id", "image_prompt", "image_model", "image_size", "image_quality"):
|
||||
|
||||
@@ -541,21 +541,15 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
|
||||
runner_lines.append(' export HIP_PATH="${HIP_PATH:-$(hipconfig -R)}"')
|
||||
runner_lines.append(' fi')
|
||||
runner_lines.append(' echo "[odysseus] ROCm/HIP detected — building llama-server with HIP support..."')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON \\\\')
|
||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' elif command -v nvcc &>/dev/null; then')
|
||||
runner_lines.append(' echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \\\\')
|
||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' else')
|
||||
runner_lines.append(' echo "[odysseus] WARNING: no HIP/CUDA toolchain found — building llama-server for CPU only."')
|
||||
runner_lines.append(' echo "[odysseus] GPU inference will not be available for this llama.cpp build."')
|
||||
runner_lines.append(' echo "[odysseus] Install ROCm for AMD GPUs or vLLM/CUDA tooling for NVIDIA, then re-launch this serve task."')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release \\\\')
|
||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||
runner_lines.append(' fi')
|
||||
|
||||
class ModelDownloadRequest(BaseModel):
|
||||
|
||||
@@ -383,11 +383,15 @@ def setup_cookbook_routes() -> APIRouter:
|
||||
encoding="utf-8",
|
||||
)
|
||||
argv = [os.environ.get("ComSpec", "cmd.exe"), "/c", str(script_path)]
|
||||
env = os.environ.copy()
|
||||
env["PYTHONUTF8"] = "1"
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
proc = subprocess.Popen(
|
||||
argv,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
stdin=subprocess.DEVNULL,
|
||||
env=env,
|
||||
**detached_popen_kwargs(),
|
||||
)
|
||||
pid_path.write_text(str(proc.pid), encoding="utf-8")
|
||||
@@ -580,7 +584,7 @@ def setup_cookbook_routes() -> APIRouter:
|
||||
runner_lines.append(" python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
|
||||
runner_lines.append(f' python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers={4 if req.disable_hf_transfer else 8})"')
|
||||
runner_lines.append('fi')
|
||||
runner_lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
||||
runner_lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||
runner_lines.append(f"rm -f {remote_runner}")
|
||||
runner_lines.append('exec "${SHELL:-/bin/bash}"')
|
||||
runner_path = TMUX_LOG_DIR / f"{session_id}_run.sh"
|
||||
@@ -611,11 +615,11 @@ def setup_cookbook_routes() -> APIRouter:
|
||||
# Detached path: no controlling TTY, so skip `< /dev/null`
|
||||
# (handled by Popen stdin=DEVNULL) and don't keep a shell open.
|
||||
lines.append(hf_cmd)
|
||||
lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
||||
lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||
else:
|
||||
# < /dev/null suppresses interactive "update available? [Y/n]" prompt
|
||||
lines.append(f"{hf_cmd} < /dev/null")
|
||||
lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
||||
lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||
lines.append(f"rm -f '{wrapper_script}'")
|
||||
lines.append('exec "${SHELL:-/bin/bash}"')
|
||||
wrapper_script.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
@@ -593,6 +593,15 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
||||
if req.session_id is not None:
|
||||
# Empty string = unlink from session
|
||||
doc.session_id = req.session_id if req.session_id else None
|
||||
if not req.session_id:
|
||||
# Tab closed / doc detached from its session — drop the
|
||||
# in-memory active-doc pointer so the last-resort injection
|
||||
# path doesn't re-surface this doc in a later chat (#1160).
|
||||
try:
|
||||
from src.tool_implementations import clear_active_document
|
||||
clear_active_document(doc_id)
|
||||
except Exception:
|
||||
pass
|
||||
db.commit()
|
||||
db.refresh(doc)
|
||||
return _doc_to_dict(doc)
|
||||
@@ -615,6 +624,13 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
||||
raise HTTPException(404, "Document not found")
|
||||
_verify_doc_owner(db, doc, user)
|
||||
doc.is_active = False
|
||||
# Closed/deleted — drop the in-memory active-doc pointer so it isn't
|
||||
# re-injected into a later, unrelated chat (#1160).
|
||||
try:
|
||||
from src.tool_implementations import clear_active_document
|
||||
clear_active_document(doc_id)
|
||||
except Exception:
|
||||
pass
|
||||
db.commit()
|
||||
return {"status": "deleted", "id": doc_id}
|
||||
except HTTPException:
|
||||
@@ -885,7 +901,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
||||
for i, doc in enumerate(batch):
|
||||
if i >= len(verdicts):
|
||||
break
|
||||
verdict = verdicts[i].lower().strip()
|
||||
verdict = str(verdicts[i] or "").lower().strip()
|
||||
if verdict == "junk":
|
||||
doc.tidy_verdict = "junk"
|
||||
db.delete(doc)
|
||||
|
||||
+79
-21
@@ -297,7 +297,8 @@ def _init_scheduled_db():
|
||||
send_at TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
error TEXT
|
||||
error TEXT,
|
||||
owner TEXT DEFAULT ''
|
||||
)
|
||||
""")
|
||||
# Email summary cache (keyed by Message-ID)
|
||||
@@ -435,6 +436,35 @@ def _init_scheduled_db():
|
||||
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN account_id TEXT")
|
||||
if "odysseus_kind" not in cols:
|
||||
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN odysseus_kind TEXT")
|
||||
if "owner" not in cols:
|
||||
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN owner TEXT DEFAULT ''")
|
||||
conn.execute("CREATE INDEX IF NOT EXISTS ix_scheduled_emails_owner_status ON scheduled_emails(owner, status)")
|
||||
# Backfill owner on legacy rows from the owning email account so the
|
||||
# owner-scoped list/cancel routes surface pre-migration scheduled
|
||||
# sends to the right user (the poller already resolves these by
|
||||
# account at send time; this aligns the UI with that).
|
||||
legacy_accounts = conn.execute(
|
||||
"SELECT DISTINCT account_id FROM scheduled_emails "
|
||||
"WHERE (owner IS NULL OR owner = '') AND account_id IS NOT NULL AND account_id != ''"
|
||||
).fetchall()
|
||||
if legacy_accounts:
|
||||
try:
|
||||
from core.database import SessionLocal as _SL, EmailAccount as _EA
|
||||
_db = _SL()
|
||||
try:
|
||||
for (acct_id,) in legacy_accounts:
|
||||
row = _db.query(_EA.owner).filter(_EA.id == acct_id).first()
|
||||
acct_owner = (row[0] or "") if row else ""
|
||||
if acct_owner:
|
||||
conn.execute(
|
||||
"UPDATE scheduled_emails SET owner = ? "
|
||||
"WHERE account_id = ? AND (owner IS NULL OR owner = '')",
|
||||
(acct_owner, acct_id),
|
||||
)
|
||||
finally:
|
||||
_db.close()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
# Lazy migration: add turns_json to email_boundaries for server-side
|
||||
@@ -815,22 +845,27 @@ def _detect_spam_folder(conn):
|
||||
return None
|
||||
|
||||
|
||||
def _imap_move(uid, dest, src="INBOX"):
|
||||
def _imap_move(uid, dest, src="INBOX", account_id: str | None = None, owner: str = ""):
|
||||
"""Move a single IMAP UID from src folder to dest. Returns True on success."""
|
||||
c = None
|
||||
try:
|
||||
c = _imap_connect()
|
||||
c = _imap_connect(account_id, owner=owner)
|
||||
c.select(_q(src))
|
||||
status, _ = c.copy(uid, _q(dest))
|
||||
if status != "OK":
|
||||
c.logout()
|
||||
return False
|
||||
c.store(uid, "+FLAGS", "\\Deleted")
|
||||
c.expunge()
|
||||
c.logout()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning(f"IMAP move {uid} → {dest} failed: {e}")
|
||||
return False
|
||||
finally:
|
||||
if c:
|
||||
try:
|
||||
c.logout()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _extract_attachment_text(msg, max_chars: int = 6000) -> str:
|
||||
@@ -1021,7 +1056,9 @@ def _fetch_sender_thread_context(sender_addr: str,
|
||||
exclude_folder: str = "INBOX",
|
||||
limit: int = 3,
|
||||
max_chars_per_email: int = 1500,
|
||||
max_attachment_chars: int = 4000) -> str:
|
||||
max_attachment_chars: int = 4000,
|
||||
account_id: str | None = None,
|
||||
owner: str = "") -> str:
|
||||
"""Pull the last N emails from `sender_addr` (across common folders),
|
||||
extract their body snippets + attachment text, and return one formatted
|
||||
block ready to be glued into an LLM system prompt as "REFERENCED MATERIAL".
|
||||
@@ -1043,7 +1080,7 @@ def _fetch_sender_thread_context(sender_addr: str,
|
||||
seen_uids.add((exclude_folder or "INBOX", str(exclude_uid)))
|
||||
|
||||
try:
|
||||
conn = _imap_connect()
|
||||
conn = _imap_connect(account_id, owner=owner)
|
||||
except Exception as e:
|
||||
logger.warning(f"sender-thread-context: imap connect failed: {e}")
|
||||
return ""
|
||||
@@ -1126,7 +1163,12 @@ def _fetch_sender_thread_context(sender_addr: str,
|
||||
return "\n\n=====\n\n".join(blocks)
|
||||
|
||||
|
||||
def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
||||
def _pre_retrieve_context(
|
||||
body: str,
|
||||
sender: str,
|
||||
account_id: str | None = None,
|
||||
owner: str = "",
|
||||
) -> tuple:
|
||||
"""Extract key terms from an incoming email and search past emails + contacts.
|
||||
|
||||
Returns (context_snippets, terms_list). Best-effort; never raises.
|
||||
@@ -1150,21 +1192,37 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
||||
# ── Known-sender check: only retrieve context for senders we already
|
||||
# have a relationship with. New / cold senders get an empty context.
|
||||
sender_addr = email.utils.parseaddr(sender or "")[1].lower()
|
||||
is_known = False
|
||||
# The CardDAV address book is global admin data backed by a single
|
||||
# Radicale instance, so only fold it into reply context for an admin /
|
||||
# single-user owner. Non-admin owners still get their own (owner-scoped)
|
||||
# IMAP history below, just not the shared contacts.
|
||||
try:
|
||||
from routes.contacts_routes import _fetch_contacts
|
||||
for c in _fetch_contacts() or []:
|
||||
# Contacts are normalized to plural `emails` lists (see
|
||||
# contacts_routes._normalize_contact); the old `c.get("email")`
|
||||
# singular key never exists, so known senders were never matched.
|
||||
if sender_addr in [(e or "").lower() for e in (c.get("emails") or [])]:
|
||||
is_known = True
|
||||
break
|
||||
from src.tool_security import owner_is_admin_or_single_user
|
||||
contacts_allowed = owner_is_admin_or_single_user(owner or None)
|
||||
except Exception:
|
||||
pass
|
||||
contacts_allowed = not bool(owner)
|
||||
is_known = False
|
||||
if contacts_allowed:
|
||||
try:
|
||||
from routes.contacts_routes import _fetch_contacts
|
||||
for c in _fetch_contacts() or []:
|
||||
# Contacts are normalized to plural `emails` lists, but
|
||||
# keep the legacy singular key fallback for older data.
|
||||
contact_emails = []
|
||||
raw_emails = c.get("emails")
|
||||
if isinstance(raw_emails, list):
|
||||
contact_emails.extend(str(e or "") for e in raw_emails)
|
||||
legacy_email = c.get("email")
|
||||
if legacy_email:
|
||||
contact_emails.append(str(legacy_email))
|
||||
if any((addr or "").strip().lower() == sender_addr for addr in contact_emails):
|
||||
is_known = True
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
if not is_known and sender_addr:
|
||||
try:
|
||||
with _imap() as _ck:
|
||||
with _imap(account_id, owner=owner) as _ck:
|
||||
_ck.select("INBOX", readonly=True)
|
||||
st_known, dk = _ck.search(None, f'(FROM "{sender_addr}")')
|
||||
if st_known == "OK" and dk and dk[0]:
|
||||
@@ -1202,7 +1260,7 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
||||
return context_snippets, terms_list
|
||||
|
||||
try:
|
||||
ctx_conn = _imap_connect()
|
||||
ctx_conn = _imap_connect(account_id, owner=owner)
|
||||
for folder in ["INBOX", "Sent", "Archive", "Drafts"]:
|
||||
try:
|
||||
st_sel, _sd = ctx_conn.select(_q(folder), readonly=True)
|
||||
@@ -1246,7 +1304,7 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
||||
|
||||
try:
|
||||
from routes.contacts_routes import _fetch_contacts
|
||||
all_contacts = _fetch_contacts()
|
||||
all_contacts = _fetch_contacts() if contacts_allowed else []
|
||||
for term in terms_list:
|
||||
t_lower = term.lower()
|
||||
matches = [c for c in all_contacts
|
||||
|
||||
+55
-36
@@ -45,6 +45,21 @@ from routes.email_helpers import (
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _owner_for_email_account(account_id: str | None) -> str:
|
||||
if not account_id:
|
||||
return ""
|
||||
try:
|
||||
from core.database import SessionLocal as _SL, EmailAccount as _EA
|
||||
db = _SL()
|
||||
try:
|
||||
row = db.query(_EA.owner).filter(_EA.id == account_id).first()
|
||||
return (row[0] or "") if row else ""
|
||||
finally:
|
||||
db.close()
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
# ── Routes ──
|
||||
|
||||
async def _emit_progress(progress_cb, message: str):
|
||||
@@ -143,25 +158,18 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
if not auto_sum and not auto_reply and not auto_tag and not auto_spam and not auto_cal:
|
||||
return "Nothing to do"
|
||||
|
||||
# Owner of the account being processed. All calendar reads/writes below are
|
||||
# scoped to this user: the multi-account fan-out runs every user's mailbox,
|
||||
# so an unscoped pass would disclose and mutate other tenants' calendars.
|
||||
_acct_owner = None
|
||||
try:
|
||||
from core.database import SessionLocal as _SLo, EmailAccount as _EAo
|
||||
_dbo = _SLo()
|
||||
try:
|
||||
if account_id:
|
||||
_arow = _dbo.query(_EAo).filter(_EAo.id == account_id).first()
|
||||
_acct_owner = _arow.owner if _arow else None
|
||||
finally:
|
||||
_dbo.close()
|
||||
except Exception:
|
||||
_acct_owner = None
|
||||
# Owner of the account being processed. All calendar + mailbox reads/writes
|
||||
# below are scoped to this user: the multi-account fan-out runs every user's
|
||||
# mailbox, so an unscoped pass would disclose/mutate other tenants' data.
|
||||
# One resolution feeds both the mailbox path (account_owner) and upstream's
|
||||
# calendar path (_acct_owner, which expects None rather than "").
|
||||
account_owner = _owner_for_email_account(account_id)
|
||||
_acct_owner = account_owner or None
|
||||
|
||||
conn = None
|
||||
try:
|
||||
await _emit_progress(progress_cb, "Connecting to mail…")
|
||||
conn = _imap_connect(account_id)
|
||||
conn = _imap_connect(account_id, owner=account_owner)
|
||||
from datetime import timedelta as _td
|
||||
since = (datetime.utcnow() - _td(days=max(1, days_back))).strftime("%d-%b-%Y")
|
||||
# uid_list carries real IMAP UIDs, matching the email UI/read routes.
|
||||
@@ -205,14 +213,19 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
# Re-select INBOX as default for downstream code
|
||||
conn.select("INBOX", readonly=True)
|
||||
if not uid_list:
|
||||
conn.logout()
|
||||
return "No recent emails"
|
||||
await _emit_progress(progress_cb, f"Found {len(uid_list)} recent email(s); checking cache…")
|
||||
|
||||
_c = _sql3.connect(SCHEDULED_DB)
|
||||
_sum_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_summaries").fetchall()}
|
||||
_reply_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_ai_replies").fetchall()}
|
||||
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags").fetchall()} if (auto_tag or auto_spam) else set()
|
||||
if auto_tag or auto_spam:
|
||||
if account_owner:
|
||||
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags WHERE owner=?", (account_owner,)).fetchall()}
|
||||
else:
|
||||
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags WHERE owner='' OR owner IS NULL").fetchall()}
|
||||
else:
|
||||
_tag_existing = set()
|
||||
_cal_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_calendar_extractions").fetchall()} if auto_cal else set()
|
||||
# Urgency is handled by the built-in `check_email_urgency` task. Keep
|
||||
# this legacy poller path disabled so users don't get two independent
|
||||
@@ -225,7 +238,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
# this per-iteration was making big inbox scans crawl. Used by the
|
||||
# urgency self-loop check below.
|
||||
try:
|
||||
_self_self_addr = (_get_email_config(account_id).get("from_address") or "").strip().lower()
|
||||
_self_self_addr = (_get_email_config(account_id, owner=account_owner).get("from_address") or "").strip().lower()
|
||||
except Exception:
|
||||
_self_self_addr = ""
|
||||
|
||||
@@ -233,11 +246,10 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
if auto_spam and not spam_folder:
|
||||
logger.warning("Auto-spam enabled but no Junk/Spam folder detected — will classify but not move")
|
||||
|
||||
url, model, headers = resolve_endpoint("utility")
|
||||
url, model, headers = resolve_endpoint("utility", owner=account_owner)
|
||||
if not url:
|
||||
url, model, headers = resolve_endpoint("default")
|
||||
url, model, headers = resolve_endpoint("default", owner=account_owner)
|
||||
if not url or not model:
|
||||
conn.logout()
|
||||
return "No model configured"
|
||||
|
||||
writing_style = settings.get("email_writing_style", "")
|
||||
@@ -395,8 +407,8 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
await _emit_progress(progress_cb, f"Drafting reply {processed + 1}/{_max_process} · checked {examined}/{len(uid_list)}")
|
||||
# Background reply drafting should not make the whole app
|
||||
# feel busy. Keep it lightweight: no extra IMAP context
|
||||
# mining here; manual AI Reply can still do that when the
|
||||
# user explicitly asks for a draft on one email.
|
||||
# mining here; manual AI Reply can still do that (owner-scoped)
|
||||
# when the user explicitly asks for a draft on one email.
|
||||
context_snippets, _terms = [], []
|
||||
sys_prompt = _EMAIL_REPLY_SYS_PROMPT_BASE
|
||||
if att_text:
|
||||
@@ -711,7 +723,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
# Send alert email immediately if critical or high
|
||||
if urgency in ("critical", "high"):
|
||||
try:
|
||||
cfg = _get_email_config(account_id)
|
||||
cfg = _get_email_config(account_id, owner=account_owner)
|
||||
to_addr = cfg["from_address"] # self-email
|
||||
|
||||
# Deep-link to open the original email in Odysseus (if public URL is configured).
|
||||
@@ -719,8 +731,8 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
from src.settings import load_settings as _ls
|
||||
_pub = (_ls().get("app_public_url") or "").rstrip("/")
|
||||
uid_str = uid.decode() if isinstance(uid, bytes) else str(uid)
|
||||
from urllib.parse import quote as _q
|
||||
open_url = f"{_pub}/#email={_q(_folder, safe='')}:{uid_str}" if _pub else ""
|
||||
from urllib.parse import quote as _url_q
|
||||
open_url = f"{_pub}/#email={_url_q(_folder, safe='')}:{uid_str}" if _pub else ""
|
||||
|
||||
alert_subject = f"[{urgency.upper()}] {subject}"
|
||||
alert_body = (
|
||||
@@ -817,7 +829,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
_req.post, url, json=payload, headers=req_headers, timeout=120
|
||||
)
|
||||
if not resp.ok:
|
||||
logger.warning(f"Auto-classify {uid.decode()} HTTP {resp.status_code}: {resp.text[:200]}")
|
||||
logger.warning(f"Auto-classify {uid.decode() if isinstance(uid, bytes) else str(uid)} HTTP {resp.status_code}: {resp.text[:200]}")
|
||||
else:
|
||||
rdata = resp.json()
|
||||
m = (rdata.get("choices") or [{}])[0].get("message", {})
|
||||
@@ -846,17 +858,17 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
|
||||
moved_to = ""
|
||||
if is_spam and auto_spam and spam_folder:
|
||||
if _imap_move(uid, spam_folder):
|
||||
if _imap_move(uid, spam_folder, account_id=account_id, owner=account_owner):
|
||||
moved_to = spam_folder
|
||||
logger.info(f"Auto-spam moved uid={uid.decode()} to {spam_folder}: {spam_reason}")
|
||||
|
||||
_c = _sql3.connect(SCHEDULED_DB)
|
||||
_c.execute("""
|
||||
INSERT OR REPLACE INTO email_tags
|
||||
(message_id, uid, folder, subject, sender, tags, spam_verdict,
|
||||
(message_id, owner, uid, folder, subject, sender, tags, spam_verdict,
|
||||
spam_reason, moved_to, model_used, created_at)
|
||||
VALUES (?, ?, 'INBOX', ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (message_id, uid.decode(), subject, sender,
|
||||
VALUES (?, ?, ?, 'INBOX', ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (message_id, account_owner or "", uid.decode(), subject, sender,
|
||||
json.dumps(tags), 1 if is_spam else 0,
|
||||
spam_reason, moved_to, model, datetime.utcnow().isoformat()))
|
||||
_c.commit()
|
||||
@@ -871,7 +883,6 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
logger.warning(f"Auto-process {uid} failed: {e}")
|
||||
continue
|
||||
|
||||
conn.logout()
|
||||
await _emit_progress(progress_cb, "Finishing…")
|
||||
if processed > 0:
|
||||
logger.info(f"Auto-processed {processed} new email(s) for summary/reply/classify")
|
||||
@@ -908,6 +919,12 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
||||
except Exception as e:
|
||||
logger.warning(f"Auto-summarize pass error: {e}")
|
||||
return f"Error: {e}"
|
||||
finally:
|
||||
if conn:
|
||||
try:
|
||||
conn.logout()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
async def _auto_summarize_poller():
|
||||
@@ -936,8 +953,9 @@ def _scheduled_poll_once() -> dict:
|
||||
conn = sqlite3.connect(SCHEDULED_DB)
|
||||
cols = [row[1] for row in conn.execute("PRAGMA table_info(scheduled_emails)").fetchall()]
|
||||
kind_expr = "odysseus_kind" if "odysseus_kind" in cols else "'scheduled' AS odysseus_kind"
|
||||
owner_expr = "owner" if "owner" in cols else "'' AS owner"
|
||||
rows = conn.execute(f"""
|
||||
SELECT id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, account_id, {kind_expr}
|
||||
SELECT id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, account_id, {kind_expr}, {owner_expr}
|
||||
FROM scheduled_emails
|
||||
WHERE status = 'pending' AND send_at <= ?
|
||||
""", (now_iso,)).fetchall()
|
||||
@@ -949,7 +967,8 @@ def _scheduled_poll_once() -> dict:
|
||||
attachments = json.loads(r[8] or "[]")
|
||||
row_account_id = r[9] if len(r) > 9 else None
|
||||
odysseus_kind = r[10] if len(r) > 10 else "scheduled"
|
||||
cfg = _get_email_config(row_account_id)
|
||||
row_owner = (r[11] if len(r) > 11 else "") or _owner_for_email_account(row_account_id)
|
||||
cfg = _get_email_config(row_account_id, owner=row_owner)
|
||||
has_atts = bool(attachments)
|
||||
if has_atts:
|
||||
outer = MIMEMultipart("mixed")
|
||||
@@ -986,7 +1005,7 @@ def _scheduled_poll_once() -> dict:
|
||||
|
||||
# Append to local Sent folder
|
||||
try:
|
||||
with _imap() as imap:
|
||||
with _imap(row_account_id, owner=row_owner) as imap:
|
||||
sent_folder = _detect_sent_folder(imap)
|
||||
imap.append(sent_folder, "\\Seen", None, outer.as_bytes())
|
||||
except Exception as e:
|
||||
|
||||
+58
-30
@@ -90,6 +90,16 @@ def _email_tag_owner_aliases(account_id: str | None, owner: str = "") -> list[st
|
||||
return out or [""]
|
||||
|
||||
|
||||
def _email_tag_owner_clause(account_id: str | None, owner: str = "") -> tuple[str, list[str]]:
|
||||
aliases = _email_tag_owner_aliases(account_id, owner)
|
||||
placeholders = ",".join("?" * len(aliases))
|
||||
# In configured multi-user mode, do not treat legacy owner='' rows as
|
||||
# visible to everyone. Single-user/unconfigured mode keeps legacy rows.
|
||||
if owner:
|
||||
return f"owner IN ({placeholders})", aliases
|
||||
return f"(owner IN ({placeholders}) OR owner IS NULL)", aliases
|
||||
|
||||
|
||||
def _record_email_received_events(owner: str, account_id: str | None, folder: str, emails: list[dict]):
|
||||
"""Baseline inbox messages, then fire `email_received` for new arrivals."""
|
||||
if not owner or (folder or "INBOX").upper() != "INBOX" or not emails:
|
||||
@@ -591,11 +601,11 @@ def setup_email_routes():
|
||||
SECURITY: `owner` is propagated so when `account_id` is missing,
|
||||
the fallback config lookup is scoped to this user's accounts only.
|
||||
"""
|
||||
conn = None
|
||||
try:
|
||||
conn = _imap_connect(account_id, owner=owner)
|
||||
select_status, _ = conn.select(_q(folder), readonly=True)
|
||||
if select_status != "OK":
|
||||
conn.logout()
|
||||
return {"emails": [], "total": 0, "folder": folder, "error": f"Folder not found: {folder}"}
|
||||
|
||||
from_clause = ""
|
||||
@@ -645,8 +655,7 @@ def setup_email_routes():
|
||||
try:
|
||||
import sqlite3 as _sql3t
|
||||
_ct = _sql3t.connect(SCHEDULED_DB)
|
||||
_owner_aliases = _email_tag_owner_aliases(account_id, owner)
|
||||
_owner_ph = ",".join("?" * len(_owner_aliases))
|
||||
_owner_clause, _owner_params = _email_tag_owner_clause(account_id, owner)
|
||||
# SECURITY: owner-scope the lookup (review C2/H8). Without
|
||||
# this, user A's `tag:urgent` filter would surface UIDs
|
||||
# written by user B and IMAP would return whatever
|
||||
@@ -658,8 +667,8 @@ def setup_email_routes():
|
||||
rows_t = _ct.execute(
|
||||
"SELECT message_id, uid FROM email_tags "
|
||||
"WHERE folder=? AND spam_verdict=1 "
|
||||
f"AND (owner IN ({_owner_ph}) OR owner IS NULL)",
|
||||
(folder, *_owner_aliases),
|
||||
f"AND {_owner_clause}",
|
||||
(folder, *_owner_params),
|
||||
).fetchall()
|
||||
for mid, uid in rows_t:
|
||||
if mid:
|
||||
@@ -670,8 +679,8 @@ def setup_email_routes():
|
||||
rows_t = _ct.execute(
|
||||
"SELECT message_id, uid, tags FROM email_tags "
|
||||
"WHERE folder=? AND tags IS NOT NULL AND tags != '' "
|
||||
f"AND (owner IN ({_owner_ph}) OR owner IS NULL)",
|
||||
(folder, *_owner_aliases),
|
||||
f"AND {_owner_clause}",
|
||||
(folder, *_owner_params),
|
||||
).fetchall()
|
||||
for r in rows_t:
|
||||
try:
|
||||
@@ -743,12 +752,11 @@ def setup_email_routes():
|
||||
_uid_strs = [u.decode() for u in uid_list]
|
||||
if _uid_strs:
|
||||
placeholders = ",".join("?" * len(_uid_strs))
|
||||
_owner_aliases = _email_tag_owner_aliases(account_id, owner)
|
||||
_owner_ph = ",".join("?" * len(_owner_aliases))
|
||||
_owner_clause, _owner_params = _email_tag_owner_clause(account_id, owner)
|
||||
rows = _c.execute(
|
||||
f"SELECT uid, tags, spam_verdict FROM email_tags "
|
||||
f"WHERE folder=? AND (owner IN ({_owner_ph}) OR owner IS NULL) AND uid IN ({placeholders})",
|
||||
[folder, *_owner_aliases, *_uid_strs],
|
||||
f"WHERE folder=? AND {_owner_clause} AND uid IN ({placeholders})",
|
||||
[folder, *_owner_params, *_uid_strs],
|
||||
).fetchall()
|
||||
for r in rows:
|
||||
try:
|
||||
@@ -805,14 +813,13 @@ def setup_email_routes():
|
||||
if header_ids:
|
||||
import sqlite3 as _sql3m
|
||||
_cm = _sql3m.connect(SCHEDULED_DB)
|
||||
_owner_aliases_m = _email_tag_owner_aliases(account_id, owner)
|
||||
_owner_ph_m = ",".join("?" * len(_owner_aliases_m))
|
||||
_owner_clause_m, _owner_params_m = _email_tag_owner_clause(account_id, owner)
|
||||
_mid_ph = ",".join("?" * len(header_ids))
|
||||
rows_m = _cm.execute(
|
||||
f"SELECT message_id, tags, spam_verdict FROM email_tags "
|
||||
f"WHERE folder=? AND (owner IN ({_owner_ph_m}) OR owner IS NULL) "
|
||||
f"WHERE folder=? AND {_owner_clause_m} "
|
||||
f"AND message_id IN ({_mid_ph})",
|
||||
[folder, *_owner_aliases_m, *header_ids],
|
||||
[folder, *_owner_params_m, *header_ids],
|
||||
).fetchall()
|
||||
_cm.close()
|
||||
for mid, tags_raw, spam_raw in rows_m:
|
||||
@@ -924,12 +931,17 @@ def setup_email_routes():
|
||||
except Exception as _summary_err:
|
||||
logger.debug(f"Bulk summary attach skipped: {_summary_err}")
|
||||
|
||||
conn.logout()
|
||||
return {"emails": emails, "total": total, "folder": folder, "offset": offset}
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to list emails: {e}")
|
||||
detail = str(e).strip()
|
||||
return {"emails": [], "total": 0, "error": f"Mail operation failed: {detail[:180]}" if detail else "Mail operation failed"}
|
||||
finally:
|
||||
if conn:
|
||||
try:
|
||||
conn.logout()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@router.get("/list")
|
||||
async def list_emails(
|
||||
@@ -971,10 +983,11 @@ def setup_email_routes():
|
||||
async def unflag_spam(uid: str, owner: str = Depends(require_owner)):
|
||||
"""User override — mark email as not spam."""
|
||||
try:
|
||||
owner_clause, owner_params = _email_tag_owner_clause(None, owner)
|
||||
_c = _sql3.connect(SCHEDULED_DB)
|
||||
_c.execute(
|
||||
"UPDATE email_tags SET spam_verdict=0, spam_reason='' WHERE uid=?",
|
||||
(uid,),
|
||||
f"UPDATE email_tags SET spam_verdict=0, spam_reason='' WHERE uid=? AND {owner_clause}",
|
||||
[uid, *owner_params],
|
||||
)
|
||||
_c.commit()
|
||||
_c.close()
|
||||
@@ -997,8 +1010,10 @@ def setup_email_routes():
|
||||
ql = (q or "").strip().lower()
|
||||
try:
|
||||
conn = _sql3.connect(SCHEDULED_DB)
|
||||
owner_clause, owner_params = _email_tag_owner_clause(None, owner)
|
||||
rows = conn.execute(
|
||||
"SELECT sender FROM email_tags WHERE sender IS NOT NULL AND sender != ''"
|
||||
f"SELECT sender FROM email_tags WHERE sender IS NOT NULL AND sender != '' AND {owner_clause}",
|
||||
owner_params,
|
||||
).fetchall()
|
||||
conn.close()
|
||||
seen = {}
|
||||
@@ -1969,8 +1984,8 @@ def setup_email_routes():
|
||||
conn = sqlite3.connect(SCHEDULED_DB)
|
||||
conn.execute("""
|
||||
INSERT INTO scheduled_emails
|
||||
(id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, send_at, created_at, status, account_id, odysseus_kind)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)
|
||||
(id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, send_at, created_at, status, account_id, odysseus_kind, owner)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?, ?)
|
||||
""", (
|
||||
sid,
|
||||
req.get("to", ""),
|
||||
@@ -1985,6 +2000,7 @@ def setup_email_routes():
|
||||
datetime.utcnow().isoformat(),
|
||||
req.get("account_id") or None,
|
||||
req.get("odysseus_kind") or "scheduled",
|
||||
owner or "",
|
||||
))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
@@ -2003,9 +2019,9 @@ def setup_email_routes():
|
||||
rows = conn.execute("""
|
||||
SELECT id, to_addr, cc, subject, send_at, created_at, status, error
|
||||
FROM scheduled_emails
|
||||
WHERE status IN ('pending', 'failed')
|
||||
WHERE status IN ('pending', 'failed') AND owner = ?
|
||||
ORDER BY send_at ASC
|
||||
""").fetchall()
|
||||
""", (owner or "",)).fetchall()
|
||||
conn.close()
|
||||
return {"scheduled": [
|
||||
{
|
||||
@@ -2023,7 +2039,10 @@ def setup_email_routes():
|
||||
import sqlite3
|
||||
try:
|
||||
conn = sqlite3.connect(SCHEDULED_DB)
|
||||
conn.execute("DELETE FROM scheduled_emails WHERE id = ? AND status = 'pending'", (sid,))
|
||||
conn.execute(
|
||||
"DELETE FROM scheduled_emails WHERE id = ? AND status = 'pending' AND owner = ?",
|
||||
(sid, owner or ""),
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
return {"success": True}
|
||||
@@ -2035,7 +2054,7 @@ def setup_email_routes():
|
||||
async def resolve_contact(name: str = Query(..., description="Name to search for"), owner: str = Depends(require_owner)):
|
||||
"""Search Sent folder for a contact by name. Returns matching email addresses."""
|
||||
try:
|
||||
with _imap() as conn:
|
||||
with _imap(owner=owner) as conn:
|
||||
matches = {}
|
||||
for folder in ["Sent", "INBOX", "Drafts"]:
|
||||
try:
|
||||
@@ -2590,7 +2609,7 @@ def setup_email_routes():
|
||||
# `api_key` field.
|
||||
from core.database import SessionLocal as _SL, Session as _CS
|
||||
_db = _SL()
|
||||
sess = _db.query(_CS).filter(_CS.id == session_id).first()
|
||||
sess = _db.query(_CS).filter(_CS.id == session_id, _CS.owner == owner).first()
|
||||
if sess and sess.endpoint_url:
|
||||
url = sess.endpoint_url
|
||||
# Some sessions stored headers double-encoded (a JSON
|
||||
@@ -2649,9 +2668,10 @@ def setup_email_routes():
|
||||
# Manual AI Reply should feel immediate. The heavier context mining
|
||||
# can involve multiple IMAP folder searches and attachment parsing;
|
||||
# reserve that for callers that explicitly opt out of fast mode.
|
||||
# Owner-scoped so pre-retrieval never crosses tenants.
|
||||
context_snippets, _terms = ([], [])
|
||||
if not fast_reply:
|
||||
context_snippets, _terms = _pre_retrieve_context(original_body, to)
|
||||
context_snippets, _terms = _pre_retrieve_context(original_body, to, owner=owner)
|
||||
|
||||
# NEW: also pull the last few emails from the original sender +
|
||||
# their attachments. The "to" field on this endpoint is the
|
||||
@@ -2667,6 +2687,7 @@ def setup_email_routes():
|
||||
exclude_uid=source_uid,
|
||||
exclude_folder=source_folder,
|
||||
limit=3,
|
||||
owner=owner,
|
||||
)
|
||||
except Exception as _e:
|
||||
logger.warning(f"sender-thread-context failed: {_e}")
|
||||
@@ -2728,7 +2749,7 @@ def setup_email_routes():
|
||||
# Configured fallback chains last.
|
||||
for cand in resolve_utility_fallback_candidates(owner=owner) or []:
|
||||
_add(*cand)
|
||||
for cand in resolve_chat_fallback_candidates() or []:
|
||||
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
|
||||
_add(*cand)
|
||||
try:
|
||||
reply = await llm_call_async_with_fallback(
|
||||
@@ -2819,9 +2840,12 @@ def setup_email_routes():
|
||||
import uuid as _uuid
|
||||
db = SessionLocal()
|
||||
try:
|
||||
row = db.query(EmailAccount).filter(EmailAccount.is_default == True).first() # noqa: E712
|
||||
q = db.query(EmailAccount).filter(EmailAccount.is_default == True) # noqa: E712
|
||||
if owner:
|
||||
q = q.filter(EmailAccount.owner == owner)
|
||||
row = q.first()
|
||||
if row is None:
|
||||
row = EmailAccount(id=_uuid.uuid4().hex, name="Default", is_default=True, enabled=True)
|
||||
row = EmailAccount(id=_uuid.uuid4().hex, owner=owner, name="Default", is_default=True, enabled=True)
|
||||
db.add(row)
|
||||
field_map = {
|
||||
"smtp_host": "smtp_host", "smtp_port": "smtp_port", "smtp_user": "smtp_user",
|
||||
@@ -2843,6 +2867,10 @@ def setup_email_routes():
|
||||
row.imap_password = _enc(data["imap_password"])
|
||||
if data.get("smtp_password"):
|
||||
row.smtp_password = _enc(data["smtp_password"])
|
||||
clear_q = db.query(EmailAccount).filter(EmailAccount.id != row.id)
|
||||
if owner:
|
||||
clear_q = clear_q.filter(EmailAccount.owner == owner)
|
||||
clear_q.update({EmailAccount.is_default: False})
|
||||
db.commit()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
@@ -148,6 +148,32 @@ def _docker_host_gateway_reachable() -> bool:
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
def _container_loopback_reachable(base_url: str, timeout: float = 0.2) -> bool:
|
||||
"""True when the requested loopback host:port is already reachable from
|
||||
inside the current container.
|
||||
|
||||
This distinguishes "a model server running alongside Odysseus in the same
|
||||
container" from "a model server running on the Docker host". Only the
|
||||
latter should be rewritten to host.docker.internal.
|
||||
"""
|
||||
try:
|
||||
parsed = urlparse(base_url)
|
||||
except Exception:
|
||||
return False
|
||||
host = (parsed.hostname or "").lower()
|
||||
port = parsed.port
|
||||
if host not in _LOOPBACK_HOSTS or not port:
|
||||
return False
|
||||
probe_host = "::1" if host == "::1" else "127.0.0.1"
|
||||
family = socket.AF_INET6 if probe_host == "::1" else socket.AF_INET
|
||||
try:
|
||||
with socket.socket(family, socket.SOCK_STREAM) as sock:
|
||||
sock.settimeout(timeout)
|
||||
sock.connect((probe_host, port))
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False) -> str:
|
||||
"""Rewrite a loopback model-endpoint URL to ``host.docker.internal`` when
|
||||
@@ -176,6 +202,8 @@ def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False
|
||||
if host in _ANY_BIND_HOSTS and not _docker_host_gateway_reachable():
|
||||
netloc = "127.0.0.1" + (f":{parsed.port}" if parsed.port else "")
|
||||
return urlunparse(parsed._replace(netloc=netloc))
|
||||
if _container_loopback_reachable(base_url):
|
||||
return base_url
|
||||
if not _docker_host_gateway_reachable():
|
||||
return base_url
|
||||
netloc = "host.docker.internal" + (f":{parsed.port}" if parsed.port else "")
|
||||
|
||||
+115
-32
@@ -16,7 +16,7 @@ from src.auth_helpers import get_current_user, effective_user
|
||||
|
||||
def _sanitize_export_filename(name: str) -> str:
|
||||
"""Return a conservative filename safe for Content-Disposition."""
|
||||
name = name or ""
|
||||
name = name if isinstance(name, str) else ""
|
||||
name = re.sub(r"[^A-Za-z0-9._-]", "_", name)
|
||||
return name[:128]
|
||||
|
||||
@@ -58,23 +58,71 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["sessions"])
|
||||
|
||||
def _pick_endpoint_for_sort():
|
||||
|
||||
def _current_user_is_admin(request: Request, user: str | None) -> bool:
|
||||
if not user:
|
||||
return False
|
||||
auth_mgr = getattr(request.app.state, "auth_manager", None)
|
||||
is_admin = getattr(auth_mgr, "is_admin", None)
|
||||
if not callable(is_admin):
|
||||
return False
|
||||
try:
|
||||
return bool(is_admin(user))
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _reject_raw_endpoint_url_for_non_admin(
|
||||
request: Request,
|
||||
user: str | None,
|
||||
endpoint_id: str | None,
|
||||
endpoint_url: str | None,
|
||||
) -> None:
|
||||
"""Require registered endpoints for signed-in non-admin session changes."""
|
||||
if endpoint_id and endpoint_id.strip():
|
||||
return
|
||||
if not endpoint_url:
|
||||
return
|
||||
# Raw URLs make the server dial whatever host the request supplies. For
|
||||
# non-admin users, require a saved endpoint row so normal owner scoping and
|
||||
# endpoint validation have already happened.
|
||||
if user and not _current_user_is_admin(request, user):
|
||||
raise HTTPException(403, "Choose a registered model endpoint")
|
||||
|
||||
|
||||
def _persist_session_headers(session_id: str, headers: dict | None) -> None:
|
||||
"""Persist endpoint auth headers for DB-backed session metadata."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
|
||||
if db_session:
|
||||
db_session.headers = headers or {}
|
||||
db_session.updated_at = datetime.utcnow()
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def _pick_endpoint_for_sort(owner=None):
|
||||
"""Pick model endpoint for auto-sort LLM call — uses utility endpoint setting, falls back to default."""
|
||||
from src.endpoint_resolver import resolve_endpoint
|
||||
# Try utility endpoint first (what the user configured for background tasks)
|
||||
url, model, headers = resolve_endpoint("utility")
|
||||
url, model, headers = resolve_endpoint("utility", owner=owner)
|
||||
if url and model:
|
||||
return url, model, headers
|
||||
# Fall back to task endpoint
|
||||
try:
|
||||
from src.task_endpoint import resolve_task_endpoint
|
||||
url, model, headers = resolve_task_endpoint()
|
||||
url, model, headers = resolve_task_endpoint(owner=owner)
|
||||
if url and model:
|
||||
return url, model, headers
|
||||
except Exception:
|
||||
pass
|
||||
# Fall back to default
|
||||
url, model, headers = resolve_endpoint("default")
|
||||
url, model, headers = resolve_endpoint("default", owner=owner)
|
||||
if url and model:
|
||||
return url, model, headers
|
||||
return None, None, None
|
||||
@@ -197,11 +245,41 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
endpoint_id: str = Form(""),
|
||||
):
|
||||
skip_val = str(skip_validation).lower() == "true"
|
||||
user = get_current_user(request)
|
||||
endpoint_api_key = ""
|
||||
endpoint_base_url = ""
|
||||
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
|
||||
if endpoint_id and endpoint_id.strip():
|
||||
from core.database import ModelEndpoint
|
||||
from src.auth_helpers import owner_filter
|
||||
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||
_db = SessionLocal()
|
||||
try:
|
||||
q = _db.query(ModelEndpoint).filter(
|
||||
ModelEndpoint.id == endpoint_id.strip(),
|
||||
ModelEndpoint.is_enabled == True,
|
||||
)
|
||||
if user:
|
||||
q = owner_filter(q, ModelEndpoint, user)
|
||||
endpoint_row = q.first()
|
||||
if not endpoint_row:
|
||||
raise HTTPException(400, "Model endpoint no longer exists")
|
||||
endpoint_base_url = endpoint_row.base_url or ""
|
||||
endpoint_api_key = endpoint_row.api_key or ""
|
||||
endpoint_url = build_chat_url(normalize_base(endpoint_base_url))
|
||||
finally:
|
||||
_db.close()
|
||||
|
||||
if not endpoint_url and not skip_val:
|
||||
raise HTTPException(400, "endpoint_url is required (choose from /api/models)")
|
||||
|
||||
model_to_use = model
|
||||
request_api_key = api_key.strip() if api_key else ""
|
||||
effective_api_key = request_api_key or endpoint_api_key
|
||||
validation_headers = None
|
||||
if effective_api_key:
|
||||
from src.endpoint_resolver import build_headers
|
||||
validation_headers = build_headers(effective_api_key, endpoint_base_url or endpoint_url)
|
||||
|
||||
if skip_val:
|
||||
# skip_validation = trust the caller and do NOT probe /v1/models.
|
||||
@@ -212,7 +290,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
elif not model_to_use:
|
||||
from src.llm_core import list_model_ids
|
||||
ids = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
||||
headers={"Authorization": f"Bearer {api_key}"} if api_key.strip() else None)
|
||||
headers=validation_headers)
|
||||
if not ids:
|
||||
raise HTTPException(400, "Cannot reach /v1/models")
|
||||
# Default to the first CHAT model — endpoints often list embedding/
|
||||
@@ -227,7 +305,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
import os as _os
|
||||
req_base = _os.path.basename(model_to_use.rstrip("/"))
|
||||
avail = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
||||
headers={"Authorization": f"Bearer {api_key}"} if api_key.strip() else None)
|
||||
headers=validation_headers)
|
||||
if not avail:
|
||||
raise HTTPException(400, "Cannot reach /v1/models")
|
||||
if model_to_use not in avail:
|
||||
@@ -252,22 +330,15 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
owner=user,
|
||||
)
|
||||
# Set auth headers for custom API-key endpoints
|
||||
resolved_key = api_key.strip() if api_key else ""
|
||||
resolved_key = request_api_key
|
||||
resolved_base = endpoint_url
|
||||
if not resolved_key and endpoint_id and endpoint_id.strip():
|
||||
from core.database import ModelEndpoint
|
||||
_db = SessionLocal()
|
||||
try:
|
||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id.strip()).first()
|
||||
if ep and ep.api_key:
|
||||
resolved_key = ep.api_key
|
||||
resolved_base = ep.base_url
|
||||
finally:
|
||||
_db.close()
|
||||
if not resolved_key and endpoint_api_key:
|
||||
resolved_key = endpoint_api_key
|
||||
resolved_base = endpoint_base_url
|
||||
if resolved_key:
|
||||
from src.endpoint_resolver import build_headers
|
||||
session.headers = build_headers(resolved_key, resolved_base)
|
||||
session_manager.save_sessions()
|
||||
_persist_session_headers(sid, session.headers)
|
||||
# Fire webhook (sync-safe)
|
||||
if webhook_manager:
|
||||
webhook_manager.fire_and_forget("session.created", {
|
||||
@@ -313,27 +384,38 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
db.close()
|
||||
# Switch model/endpoint mid-session
|
||||
if model is not None and endpoint_url is not None:
|
||||
user = get_current_user(request)
|
||||
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
|
||||
endpoint_api_key = ""
|
||||
endpoint_base_url = ""
|
||||
if endpoint_id:
|
||||
from core.database import ModelEndpoint
|
||||
from src.auth_helpers import owner_filter
|
||||
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||
_db = SessionLocal()
|
||||
try:
|
||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id).first()
|
||||
q = _db.query(ModelEndpoint).filter(
|
||||
ModelEndpoint.id == endpoint_id,
|
||||
ModelEndpoint.is_enabled == True,
|
||||
)
|
||||
if user:
|
||||
q = owner_filter(q, ModelEndpoint, user)
|
||||
ep = q.first()
|
||||
if not ep:
|
||||
raise HTTPException(400, "Model endpoint no longer exists")
|
||||
endpoint_base_url = ep.base_url or ""
|
||||
endpoint_api_key = ep.api_key or ""
|
||||
endpoint_url = build_chat_url(normalize_base(endpoint_base_url))
|
||||
finally:
|
||||
_db.close()
|
||||
session.model = model
|
||||
session.endpoint_url = endpoint_url
|
||||
# Update auth headers from the endpoint's stored API key
|
||||
if endpoint_id:
|
||||
_db = SessionLocal()
|
||||
try:
|
||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id).first()
|
||||
if ep and ep.api_key:
|
||||
from src.endpoint_resolver import build_headers
|
||||
session.headers = build_headers(ep.api_key, ep.base_url)
|
||||
finally:
|
||||
_db.close()
|
||||
if endpoint_api_key:
|
||||
from src.endpoint_resolver import build_headers
|
||||
session.headers = build_headers(endpoint_api_key, endpoint_base_url)
|
||||
else:
|
||||
session.headers = {}
|
||||
# Persist to DB
|
||||
db = SessionLocal()
|
||||
try:
|
||||
@@ -341,6 +423,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
if db_session:
|
||||
db_session.model = model
|
||||
db_session.endpoint_url = endpoint_url
|
||||
db_session.headers = session.headers or {}
|
||||
db_session.updated_at = datetime.utcnow()
|
||||
db.commit()
|
||||
finally:
|
||||
@@ -754,7 +837,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
from src.endpoint_resolver import resolve_endpoint
|
||||
from src.llm_core import llm_call_async
|
||||
|
||||
url, model, headers = resolve_endpoint("utility")
|
||||
url, model, headers = resolve_endpoint("utility", owner=get_current_user(request))
|
||||
if not url or not model:
|
||||
url, model, headers = session.endpoint_url, session.model, session.headers
|
||||
if not url or not model:
|
||||
@@ -954,9 +1037,9 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
||||
|
||||
# Pick an endpoint — prefer admin-configured task endpoint
|
||||
from src.task_endpoint import resolve_task_endpoint
|
||||
url, model, headers = resolve_task_endpoint()
|
||||
url, model, headers = resolve_task_endpoint(owner=user)
|
||||
if not url:
|
||||
url, model, headers = _pick_endpoint_for_sort()
|
||||
url, model, headers = _pick_endpoint_for_sort(owner=user)
|
||||
if not url:
|
||||
raise HTTPException(503, "No available model endpoint for auto-sort")
|
||||
|
||||
|
||||
+29
-17
@@ -79,6 +79,8 @@ def _skill_test_task(skill: dict) -> str:
|
||||
an email); if we just hand over the 'when to use' text the agent has nothing
|
||||
to work on and stalls asking for input. So we tell it to create its own
|
||||
realistic fixture first, then apply the skill end-to-end."""
|
||||
if not isinstance(skill, dict):
|
||||
skill = {}
|
||||
ctx = (skill.get("when_to_use") or skill.get("description") or skill.get("name") or "").strip()
|
||||
return (
|
||||
"Test this skill end-to-end. FIRST, set up a small realistic scenario it "
|
||||
@@ -310,6 +312,8 @@ def _should_check_retrieval_precision(skill: dict) -> bool:
|
||||
"installation", "install", "system", "ssh", "document", "documents",
|
||||
"search", "email", "calendar", "gpu", "server", "python",
|
||||
}
|
||||
if not isinstance(skill, dict):
|
||||
return False
|
||||
tags = {str(t or "").strip().lower() for t in (skill.get("tags") or [])}
|
||||
if tags & broad:
|
||||
return True
|
||||
@@ -463,7 +467,7 @@ async def _run_skill_test_job(key, name, md, task, url, model, headers, owner, s
|
||||
if skills_manager is not None:
|
||||
v = (job["verdict"] or {}).get("verdict") or "unknown"
|
||||
try:
|
||||
skills_manager.set_audit(name, v, by_teacher=False, worker_model=model)
|
||||
skills_manager.set_audit(name, v, by_teacher=False, worker_model=model, owner=owner)
|
||||
except Exception:
|
||||
pass
|
||||
conf = {"pass": 0.95, "needs_work": 0.6, "fail": 0.4}.get(v)
|
||||
@@ -563,6 +567,7 @@ def _skill_duplicate_blocker(skills_manager, name: str, owner) -> Optional[str]:
|
||||
False,
|
||||
[keeper_name],
|
||||
f"Lower-priority duplicate of {keeper_name}",
|
||||
owner=owner,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
@@ -629,7 +634,7 @@ def _audit_finalize_status(skills_manager, name: str, owner, verdict: str,
|
||||
if generic_reason:
|
||||
necessary = False
|
||||
try:
|
||||
skills_manager.set_necessity(name, False, [], generic_reason)
|
||||
skills_manager.set_necessity(name, False, [], generic_reason, owner=owner)
|
||||
except Exception:
|
||||
pass
|
||||
duplicate_of = _skill_duplicate_blocker(skills_manager, name, owner) if verdict == "pass" else None
|
||||
@@ -788,7 +793,8 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
nec = await _eval_skill_necessity(md, others, url, model, headers)
|
||||
if nec is not None:
|
||||
skills_manager.set_necessity(name, nec.get("necessary", True),
|
||||
nec.get("redundant_with"), nec.get("reason"))
|
||||
nec.get("redundant_with"), nec.get("reason"),
|
||||
owner=owner)
|
||||
if not nec.get("necessary", True):
|
||||
log(f"{name}: possibly unnecessary — {nec.get('reason', '')[:80]}")
|
||||
except Exception as e:
|
||||
@@ -800,11 +806,11 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
reason = generic_reason or (f"Lower-priority duplicate of {duplicate_of}" if duplicate_of else str((nec or {}).get("reason") or "Unnecessary skill"))
|
||||
try:
|
||||
skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35}, owner=owner)
|
||||
skills_manager.set_audit(name, "skipped", by_teacher=False, worker_model=model)
|
||||
skills_manager.set_audit(name, "skipped", by_teacher=False, worker_model=model, owner=owner)
|
||||
if duplicate_of:
|
||||
skills_manager.set_necessity(name, False, [duplicate_of], reason)
|
||||
skills_manager.set_necessity(name, False, [duplicate_of], reason, owner=owner)
|
||||
else:
|
||||
skills_manager.set_necessity(name, False, [], reason)
|
||||
skills_manager.set_necessity(name, False, [], reason, owner=owner)
|
||||
except Exception:
|
||||
pass
|
||||
log(f"{name}: draft — skipped functional test ({reason[:100]})")
|
||||
@@ -848,13 +854,13 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
if fixed and fixed.strip() != md.strip():
|
||||
_apply_skill_md(skills_manager, name, fixed, owner)
|
||||
_set_conf(0.95)
|
||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model)
|
||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model, owner=owner)
|
||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.95, (refreshed or {}).get("necessity"), verdict)
|
||||
log(f"{name}: {status} — confidence 95%")
|
||||
return {"skill": name, "result": "pass", "verdict": verdict, "confidence": 0.95, "status": status}
|
||||
if v in ("unknown", "inconclusive"):
|
||||
skills_manager.set_audit(name, "inconclusive", by_teacher=False, worker_model=model)
|
||||
skills_manager.set_audit(name, "inconclusive", by_teacher=False, worker_model=model, owner=owner)
|
||||
status = _audit_finalize_status(skills_manager, name, owner, "inconclusive", skill.get("confidence") or 0.0, skill.get("necessity"))
|
||||
log(f"{name}: {status} — inconclusive")
|
||||
return {"skill": name, "result": "inconclusive", "verdict": verdict, "status": status}
|
||||
@@ -869,7 +875,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
log(f"{name}: retry (self) = {v}")
|
||||
if v == "pass":
|
||||
_set_conf(0.85)
|
||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model)
|
||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model, owner=owner)
|
||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.85, (refreshed or {}).get("necessity"), verdict)
|
||||
log(f"{name}: {status} — confidence 85% after self-edit")
|
||||
@@ -893,7 +899,9 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
log(f"{name}: retry on student after teacher rewrite = {v}")
|
||||
if v == "pass":
|
||||
_set_conf(0.8)
|
||||
skills_manager.set_audit(name, "pass", by_teacher=True, worker_model=model, teacher_model=t_model)
|
||||
skills_manager.set_audit(
|
||||
name, "pass", by_teacher=True, worker_model=model, teacher_model=t_model, owner=owner
|
||||
)
|
||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.8, (refreshed or {}).get("necessity"), verdict)
|
||||
log(f"{name}: {status} — confidence 80% after teacher rewrite")
|
||||
@@ -908,6 +916,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
||||
name, v or "fail", by_teacher=teacher_ran,
|
||||
worker_model=model,
|
||||
teacher_model=(teacher[1] if teacher_ran and teacher else ""),
|
||||
owner=owner,
|
||||
)
|
||||
log(f"{name}: flagged — confidence lowered, kept as draft for manual review")
|
||||
return {"skill": name, "result": "flagged", "verdict": verdict, "confidence": 0.35}
|
||||
@@ -976,7 +985,7 @@ async def _run_audit_all_job(key, skills_manager, names, url, model, headers, te
|
||||
job.pop("task", None)
|
||||
|
||||
|
||||
def _resolve_audit_models():
|
||||
def _resolve_audit_models(owner=None):
|
||||
"""Resolve (url, model, headers, teacher) for an audit run from Settings.
|
||||
|
||||
Worker = Utility model (falling back to Default, normalized to a served
|
||||
@@ -985,7 +994,7 @@ def _resolve_audit_models():
|
||||
ValueError if no worker model.
|
||||
"""
|
||||
from src.endpoint_resolver import resolve_endpoint
|
||||
url, model, headers = resolve_endpoint("utility")
|
||||
url, model, headers = resolve_endpoint("utility", owner=owner)
|
||||
if not url or not model:
|
||||
raise ValueError("No model configured — set a Default or Utility model in Settings.")
|
||||
try:
|
||||
@@ -1029,7 +1038,7 @@ async def run_scheduled_skill_audit(skills_manager: SkillsManager,
|
||||
return {"status": "running", "skipped": True}
|
||||
|
||||
try:
|
||||
url, model, headers, teacher = _resolve_audit_models()
|
||||
url, model, headers, teacher = _resolve_audit_models(owner=owner)
|
||||
except ValueError as e:
|
||||
logger.info(f"Scheduled skill audit skipped — {e}")
|
||||
return {"status": "skipped", "reason": str(e)}
|
||||
@@ -1280,7 +1289,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
||||
|
||||
# Prefer the configured DEFAULT (→ Utility) model — not the current chat
|
||||
# session's model. Fall back to the caller's session model only if unset.
|
||||
url, model, headers = resolve_endpoint("default")
|
||||
url, model, headers = resolve_endpoint("default", owner=user)
|
||||
if not url or not model:
|
||||
url = url or ((body.get("endpoint_url") or "").strip() or None)
|
||||
model = model or ((body.get("model") or "").strip() or None)
|
||||
@@ -1360,7 +1369,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
||||
|
||||
# Worker model (Default, normalized) + optional teacher — shared resolver.
|
||||
try:
|
||||
url, model, headers, teacher = _resolve_audit_models()
|
||||
url, model, headers, teacher = _resolve_audit_models(owner=user)
|
||||
except ValueError as e:
|
||||
raise HTTPException(400, str(e))
|
||||
|
||||
@@ -1437,7 +1446,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
||||
@router.post("/{skill_id}/markdown")
|
||||
async def save_skill_markdown(request: Request, skill_id: str):
|
||||
"""Replace SKILL.md with new raw content. Parses + validates first."""
|
||||
from services.memory.skill_format import Skill, slugify
|
||||
from services.memory.skill_format import Skill
|
||||
user = _owner(request)
|
||||
body = await request.json()
|
||||
new_content = body.get("markdown")
|
||||
@@ -1452,7 +1461,10 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
||||
sk = Skill.from_markdown(new_content)
|
||||
except Exception as e:
|
||||
raise HTTPException(400, f"Could not parse SKILL.md: {e}")
|
||||
sk.name = slugify(sk.name or match.get("name"))
|
||||
# Never rename on save: a changed `name` in the markdown would move
|
||||
# the skill dir (update_skill) and orphan the original id, so a later
|
||||
# delete 404s (#1333). Pin to the stored name, like _apply_skill_md.
|
||||
sk.name = match.get("name")
|
||||
if not sk.owner:
|
||||
sk.owner = match.get("owner") or user
|
||||
ok = skills_manager.update_skill(match.get("name"), {
|
||||
|
||||
+12
-8
@@ -8,6 +8,7 @@ from typing import List
|
||||
import logging
|
||||
from core.middleware import require_admin
|
||||
from src.auth_helpers import get_current_user
|
||||
from src.upload_handler import count_recent_uploads
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -24,15 +25,18 @@ def setup_upload_routes(upload_handler):
|
||||
|
||||
client_ip = request.client.host if request.client else "unknown"
|
||||
out = []
|
||||
|
||||
# Limit concurrent uploads per IP
|
||||
ip_upload_count = sum(
|
||||
1 for f in files
|
||||
if client_ip in upload_handler.upload_rate_log and
|
||||
any(now > time.time() - 10 for now in upload_handler.upload_rate_log[client_ip][-len(files):])
|
||||
|
||||
# Limit concurrent uploads per IP. Count genuine recent upload events —
|
||||
# NOT the number of files in this batch. The previous check summed over
|
||||
# `files`, so a single multi-file request counted itself as N concurrent
|
||||
# uploads and tripped the limit (issue #1346: "attach more than one file
|
||||
# → the model doesn't even see them"). save_upload still enforces the
|
||||
# per-minute sliding-window rate limit per file.
|
||||
recent_uploads = count_recent_uploads(
|
||||
upload_handler.upload_rate_log.get(client_ip, []), time.time()
|
||||
)
|
||||
|
||||
if ip_upload_count >= upload_handler.max_concurrent_uploads:
|
||||
|
||||
if recent_uploads >= upload_handler.max_concurrent_uploads:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=f"Maximum concurrent uploads ({upload_handler.max_concurrent_uploads}) exceeded"
|
||||
|
||||
+8
-10
@@ -83,10 +83,9 @@ async def _run_bw(args: list, session: str = None, input_text: str = None,
|
||||
if session:
|
||||
env["BW_SESSION"] = session
|
||||
# Secrets must never be passed as argv — process arguments are world-readable
|
||||
# via `ps` / `/proc/<pid>/cmdline` to any local user. Hand the master password
|
||||
# to `bw` through the environment instead (paired with `--passwordenv
|
||||
# BW_PASSWORD` in args); /proc/<pid>/environ is readable only by the process
|
||||
# owner. This mirrors how BW_SESSION is already passed above.
|
||||
# via `ps` / `/proc/<pid>/cmdline` to any local user. Keep --passwordenv
|
||||
# support for bw commands that need it; unlock/login callers should prefer
|
||||
# stdin so the master password is not left in the child environment either.
|
||||
if bw_password is not None:
|
||||
env["BW_PASSWORD"] = bw_password
|
||||
bw_path = _find_bw()
|
||||
@@ -184,13 +183,12 @@ def setup_vault_routes():
|
||||
async def unlock(req: VaultUnlockRequest, request: Request):
|
||||
"""Unlock the vault and save the session key."""
|
||||
require_admin(request)
|
||||
# Pass the master password via the environment (--passwordenv), NOT as
|
||||
# an argv element — argv is visible to every local user through `ps` /
|
||||
# /proc/<pid>/cmdline. (The sibling /login handler already keeps the
|
||||
# password off argv by feeding it on stdin.)
|
||||
# Pass the master password on stdin, not argv. argv is visible through
|
||||
# `ps` / /proc/<pid>/cmdline; stdin also avoids leaving the secret in
|
||||
# the child process environment.
|
||||
stdout, stderr, rc = await _run_bw(
|
||||
["unlock", "--passwordenv", "BW_PASSWORD", "--raw"],
|
||||
bw_password=req.master_password,
|
||||
["unlock", "--raw"],
|
||||
input_text=req.master_password + "\n",
|
||||
)
|
||||
if rc != 0:
|
||||
return {"ok": False, "error": f"Unlock failed: {stderr[:300]}"}
|
||||
|
||||
@@ -39,19 +39,19 @@ Environment:
|
||||
USAGE
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
--help|-h)
|
||||
_usage
|
||||
exit 0
|
||||
;;
|
||||
"")
|
||||
;;
|
||||
*)
|
||||
printf 'Unknown option: %s\n\n' "$1" >&2
|
||||
_usage >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
for _arg in "$@"; do
|
||||
case "${_arg}" in
|
||||
--help|-h)
|
||||
_usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
printf 'Unknown option: %s\n\n' "${_arg}" >&2
|
||||
_usage >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
_find_cmd() {
|
||||
if command -v "$1" >/dev/null 2>&1; then
|
||||
|
||||
@@ -58,10 +58,12 @@ def main():
|
||||
count = db.query(Session).filter(Session.owner == None).update({"owner": owner})
|
||||
print(f" sessions: claimed {count}")
|
||||
|
||||
# Documents
|
||||
count = db.query(Document).filter(Document.session_id.in_(
|
||||
db.query(Session.id).filter(Session.owner == owner)
|
||||
)).update({"session_id": Document.session_id}, synchronize_session=False)
|
||||
# Documents (have their own owner column; claim the ownerless ones,
|
||||
# mirroring the sessions/gallery/comparisons blocks). The old query set
|
||||
# session_id to itself — a no-op — and never set owner, so ownerless
|
||||
# documents stayed ownerless and invisible in the user's Library.
|
||||
count = db.query(Document).filter(Document.owner == None).update({"owner": owner})
|
||||
print(f" documents: claimed {count}")
|
||||
|
||||
# Gallery
|
||||
if GalleryImage:
|
||||
|
||||
+6
-2
@@ -68,6 +68,10 @@ def _short_help(path: Path) -> str:
|
||||
return first
|
||||
|
||||
|
||||
def _is_runnable_subcommand(path: Path) -> bool:
|
||||
return path.exists() and path.is_file() and os.access(path, os.X_OK)
|
||||
|
||||
|
||||
def _print_listing() -> None:
|
||||
"""`odysseus` with no args (or `odysseus help`) — print the table."""
|
||||
sys.stdout.write(f"odysseus {VERSION} — every feature, on the shell.\n\n")
|
||||
@@ -101,7 +105,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
_print_listing()
|
||||
return 0
|
||||
sub = SCRIPTS_DIR / f"odysseus-{argv[1]}"
|
||||
if not sub.exists():
|
||||
if not _is_runnable_subcommand(sub):
|
||||
sys.stderr.write(f"odysseus: unknown subcommand {argv[1]!r}\n")
|
||||
return 1
|
||||
return subprocess.call([str(sub), "--help"])
|
||||
@@ -109,7 +113,7 @@ def main(argv: list[str] | None = None) -> int:
|
||||
# `odysseus foo ...` → exec `odysseus-foo ...` under the project venv.
|
||||
name = argv[0]
|
||||
sub = SCRIPTS_DIR / f"odysseus-{name}"
|
||||
if not sub.exists():
|
||||
if not _is_runnable_subcommand(sub):
|
||||
sys.stderr.write(
|
||||
f"odysseus: unknown subcommand {name!r}. "
|
||||
f"Try `odysseus help` to see available ones.\n"
|
||||
|
||||
@@ -56,6 +56,16 @@ def _sqlite_safe_copy(src: Path, dst: Path) -> None:
|
||||
dst.write_bytes(src.read_bytes())
|
||||
|
||||
|
||||
def _reject_output_inside_data(out_path: Path) -> None:
|
||||
try:
|
||||
resolved = out_path.resolve()
|
||||
data_root = _DATA_DIR.resolve()
|
||||
resolved.relative_to(data_root)
|
||||
except ValueError:
|
||||
return
|
||||
fail("backup output path must be outside data/")
|
||||
|
||||
|
||||
def cmd_snapshot(args):
|
||||
"""Write a tar.gz of the entire data/ directory.
|
||||
|
||||
@@ -68,6 +78,7 @@ def cmd_snapshot(args):
|
||||
out_path = Path(args.out) if args.out else (
|
||||
_BACKUP_DIR / f"odysseus-backup-{datetime.now().strftime('%Y%m%d-%H%M%S')}.tar.gz"
|
||||
)
|
||||
_reject_output_inside_data(out_path)
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
sqlite_dbs = [p for p in _DATA_DIR.rglob("*.db") if p.is_file() and not p.is_symlink()]
|
||||
|
||||
@@ -30,11 +30,19 @@ except ModuleNotFoundError as e:
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def _preview_text(value, limit: int = 200) -> str:
|
||||
"""Truncated preview tolerant of non-string values. A gallery row whose
|
||||
``prompt`` is a non-string would crash ``(value or "")[:200]`` with a
|
||||
TypeError; coerce non-strings to ""."""
|
||||
text = value if isinstance(value, str) else ""
|
||||
return text[:limit]
|
||||
|
||||
|
||||
def _serialize_image(i: "GalleryImage") -> dict:
|
||||
return {
|
||||
"id": i.id,
|
||||
"filename": i.filename,
|
||||
"prompt": (i.prompt or "")[:200],
|
||||
"prompt": _preview_text(i.prompt),
|
||||
"model": i.model or "",
|
||||
"size": i.size or "",
|
||||
"tags": i.tags or "",
|
||||
|
||||
@@ -58,6 +58,8 @@ def _resolve(name: str) -> Path | None:
|
||||
"""Match a log by exact filename, basename-without-extension, or
|
||||
substring. Returns the most-recently-modified match if there are
|
||||
ties."""
|
||||
if not isinstance(name, str):
|
||||
return None
|
||||
candidates = []
|
||||
for base in (_APP_LOGS, _TMUX_LOGS):
|
||||
if not base.is_dir():
|
||||
|
||||
+14
-5
@@ -107,6 +107,19 @@ def _q(name: str) -> str:
|
||||
return '"' + (name or "").replace("\\", "\\\\").replace('"', '\\"') + '"'
|
||||
|
||||
|
||||
def _split_recipients(value: str) -> list[str]:
|
||||
return [r.strip() for r in (value or "").split(",") if r.strip()]
|
||||
|
||||
|
||||
def _recipient_list(to: str, cc: str = "", bcc: str = "") -> list[str]:
|
||||
recipients = _split_recipients(to)
|
||||
recipients.extend(_split_recipients(cc))
|
||||
recipients.extend(_split_recipients(bcc))
|
||||
if not recipients:
|
||||
fail("at least one recipient is required")
|
||||
return recipients
|
||||
|
||||
|
||||
# ─── list ────────────────────────────────────────────────────────────
|
||||
|
||||
def cmd_list(args) -> None:
|
||||
@@ -302,11 +315,7 @@ def cmd_send(args) -> None:
|
||||
outer["Date"] = datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S +0000")
|
||||
outer.attach(MIMEText(body, "plain", "utf-8"))
|
||||
|
||||
recipients = [r.strip() for r in args.to.split(",") if r.strip()]
|
||||
if args.cc:
|
||||
recipients.extend([r.strip() for r in args.cc.split(",") if r.strip()])
|
||||
if args.bcc:
|
||||
recipients.extend([r.strip() for r in args.bcc.split(",") if r.strip()])
|
||||
recipients = _recipient_list(args.to, args.cc, args.bcc)
|
||||
|
||||
if args.dry_run:
|
||||
emit({
|
||||
|
||||
@@ -42,7 +42,7 @@ def _serialize(s: "McpServer", redact_env: bool = True) -> dict:
|
||||
env_obj = json.loads(s.env) if s.env else {}
|
||||
except json.JSONDecodeError:
|
||||
env_obj = {}
|
||||
if redact_env and env_obj:
|
||||
if redact_env and isinstance(env_obj, dict):
|
||||
env_obj = {k: ("***" if v else "") for k, v in env_obj.items()}
|
||||
return {
|
||||
"id": s.id,
|
||||
|
||||
+16
-7
@@ -28,9 +28,12 @@ def _load() -> dict:
|
||||
if not _PATH.exists():
|
||||
return {}
|
||||
try:
|
||||
return json.loads(_PATH.read_text())
|
||||
data = json.loads(_PATH.read_text())
|
||||
except json.JSONDecodeError as e:
|
||||
fail(f"presets.json corrupt: {e}")
|
||||
if not isinstance(data, dict):
|
||||
fail("presets.json corrupt: expected an object")
|
||||
return data
|
||||
|
||||
|
||||
def _save(data: dict) -> None:
|
||||
@@ -46,6 +49,15 @@ def _save(data: dict) -> None:
|
||||
tmp.replace(_PATH)
|
||||
|
||||
|
||||
def _entry_or_fail(presets: dict, name: str) -> dict:
|
||||
if name not in presets:
|
||||
fail(f"no preset named {name!r}")
|
||||
entry = presets[name]
|
||||
if not isinstance(entry, dict):
|
||||
fail(f"preset {name!r} is corrupt: expected an object")
|
||||
return entry
|
||||
|
||||
|
||||
def cmd_list(args):
|
||||
presets = _load()
|
||||
rows = []
|
||||
@@ -63,9 +75,7 @@ def cmd_list(args):
|
||||
|
||||
def cmd_get(args):
|
||||
presets = _load()
|
||||
if args.name not in presets:
|
||||
fail(f"no preset named {args.name!r}")
|
||||
emit({"id": args.name, **presets[args.name]}, args)
|
||||
emit({"id": args.name, **_entry_or_fail(presets, args.name)}, args)
|
||||
|
||||
|
||||
def cmd_set(args):
|
||||
@@ -90,9 +100,8 @@ def cmd_set(args):
|
||||
|
||||
def cmd_delete(args):
|
||||
presets = _load()
|
||||
if args.name not in presets:
|
||||
fail(f"no preset named {args.name!r}")
|
||||
snap = presets.pop(args.name)
|
||||
snap = _entry_or_fail(presets, args.name)
|
||||
presets.pop(args.name)
|
||||
_save(presets)
|
||||
emit({"ok": True, "deleted": {"id": args.name, **snap}}, args)
|
||||
|
||||
|
||||
@@ -36,10 +36,18 @@ def _load(rp_id: str) -> dict | None:
|
||||
return None
|
||||
|
||||
|
||||
def _preview_text(value, limit: int = 200) -> str:
|
||||
"""Truncated preview tolerant of non-string values. A stored research
|
||||
record whose ``query`` is a non-string (legacy/corrupt JSON) would crash
|
||||
``(value or "")[:200]`` with a TypeError; coerce non-strings to ""."""
|
||||
text = value if isinstance(value, str) else ""
|
||||
return text[:limit]
|
||||
|
||||
|
||||
def _summarize(rp_id: str, data: dict) -> dict:
|
||||
return {
|
||||
"id": rp_id,
|
||||
"query": (data.get("query") or "")[:200],
|
||||
"query": _preview_text(data.get("query")),
|
||||
"category": data.get("category") or "",
|
||||
"status": data.get("status") or "",
|
||||
"started_at": data.get("started_at") or "",
|
||||
|
||||
@@ -27,6 +27,12 @@ except ModuleNotFoundError as e:
|
||||
|
||||
|
||||
def _serialize(s: "DbSession") -> dict:
|
||||
def _int_or_zero(value) -> int:
|
||||
try:
|
||||
return int(value or 0)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
return {
|
||||
"id": s.id,
|
||||
"name": s.name,
|
||||
@@ -37,9 +43,9 @@ def _serialize(s: "DbSession") -> dict:
|
||||
"archived": bool(s.archived),
|
||||
"rag": bool(s.rag),
|
||||
"is_important": bool(s.is_important),
|
||||
"message_count": s.message_count or 0,
|
||||
"total_input_tokens": s.total_input_tokens or 0,
|
||||
"total_output_tokens": s.total_output_tokens or 0,
|
||||
"message_count": _int_or_zero(s.message_count),
|
||||
"total_input_tokens": _int_or_zero(s.total_input_tokens),
|
||||
"total_output_tokens": _int_or_zero(s.total_output_tokens),
|
||||
"last_accessed": s.last_accessed.isoformat() if s.last_accessed else "",
|
||||
"created_at": s.created_at.isoformat() if s.created_at else "",
|
||||
}
|
||||
|
||||
@@ -29,6 +29,16 @@ except ModuleNotFoundError as e:
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def _decode_png_data(data_png: str) -> bytes:
|
||||
raw = data_png or ""
|
||||
if "," in raw:
|
||||
raw = raw.split(",", 1)[1]
|
||||
try:
|
||||
return base64.b64decode(raw, validate=True)
|
||||
except Exception as e:
|
||||
fail(f"data_png is not valid base64: {e}")
|
||||
|
||||
|
||||
def cmd_list(args):
|
||||
"""No `Signature` SQLAlchemy model is registered for the
|
||||
`signatures` table — query via raw SQL so we don't depend on it."""
|
||||
@@ -85,13 +95,7 @@ def cmd_export(args):
|
||||
), {"id": args.id}).mappings().first()
|
||||
if not row:
|
||||
fail(f"no signature with id {args.id!r}")
|
||||
raw = row["data_png"] or ""
|
||||
if "," in raw:
|
||||
raw = raw.split(",", 1)[1]
|
||||
try:
|
||||
png_bytes = base64.b64decode(raw)
|
||||
except Exception as e:
|
||||
fail(f"data_png is not valid base64: {e}")
|
||||
png_bytes = _decode_png_data(row["data_png"] or "")
|
||||
out = Path(args.png)
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
out.write_bytes(png_bytes)
|
||||
|
||||
+12
-1
@@ -41,11 +41,22 @@ def _manager() -> SkillsManager:
|
||||
return _mgr
|
||||
|
||||
|
||||
def _preview_text(value, limit: int = 200) -> str:
|
||||
"""Truncated preview of a text field, tolerant of non-string values.
|
||||
|
||||
A skill whose ``description`` is a non-string (e.g. a number from a
|
||||
hand-edited/legacy store) would crash ``(value or "")[:200]`` with a
|
||||
TypeError; coerce non-strings to "" instead.
|
||||
"""
|
||||
text = value if isinstance(value, str) else ""
|
||||
return text[:limit]
|
||||
|
||||
|
||||
def _summary(skill: dict) -> dict:
|
||||
return {
|
||||
"name": skill.get("name", ""),
|
||||
"category": skill.get("category", "general"),
|
||||
"description": (skill.get("description") or "")[:200],
|
||||
"description": _preview_text(skill.get("description")),
|
||||
"status": skill.get("status", ""),
|
||||
"uses": skill.get("uses", 0),
|
||||
"last_used": skill.get("last_used") or "",
|
||||
|
||||
@@ -166,116 +166,3 @@ def update_database():
|
||||
|
||||
if __name__ == "__main__":
|
||||
update_database()
|
||||
"""
|
||||
update_database.py
|
||||
|
||||
This script updates the database schema by adding new columns to the sessions table
|
||||
if they don't already exist. It uses raw SQL ALTER TABLE statements to modify
|
||||
the existing SQLite database.
|
||||
|
||||
The following columns are added:
|
||||
- last_accessed (DateTime): Set to created_at for existing records
|
||||
- is_important (Boolean): Set to False for existing records
|
||||
- message_count (Integer): Calculated from the number of messages in chat_messages table
|
||||
|
||||
Usage:
|
||||
python update_database.py
|
||||
"""
|
||||
|
||||
import os
|
||||
from datetime import datetime
|
||||
from sqlalchemy import create_engine, text
|
||||
from database import DATABASE_URL, SessionLocal
|
||||
|
||||
def update_database():
|
||||
"""Update the database schema and populate new columns."""
|
||||
# Create engine from DATABASE_URL
|
||||
engine = create_engine(DATABASE_URL)
|
||||
|
||||
# Start a transaction
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Add last_accessed column if it doesn't exist
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN last_accessed DATETIME"))
|
||||
conn.commit()
|
||||
print("Added last_accessed column to sessions table")
|
||||
except Exception as e:
|
||||
if "duplicate column name" in str(e).lower():
|
||||
print("last_accessed column already exists")
|
||||
else:
|
||||
print(f"Error adding last_accessed column: {e}")
|
||||
|
||||
# Add is_important column if it doesn't exist
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN is_important BOOLEAN DEFAULT FALSE"))
|
||||
conn.commit()
|
||||
print("Added is_important column to sessions table")
|
||||
except Exception as e:
|
||||
if "duplicate column name" in str(e).lower():
|
||||
print("is_important column already exists")
|
||||
else:
|
||||
print(f"Error adding is_important column: {e}")
|
||||
|
||||
# Add message_count column if it doesn't exist
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN message_count INTEGER DEFAULT 0"))
|
||||
conn.commit()
|
||||
print("Added message_count column to sessions table")
|
||||
except Exception as e:
|
||||
if "duplicate column name" in str(e).lower():
|
||||
print("message_count column already exists")
|
||||
else:
|
||||
print(f"Error adding message_count column: {e}")
|
||||
|
||||
# Populate last_accessed with created_at for existing records where last_accessed is NULL
|
||||
print("Populating last_accessed column...")
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("""
|
||||
UPDATE sessions
|
||||
SET last_accessed = created_at
|
||||
WHERE last_accessed IS NULL
|
||||
"""))
|
||||
conn.commit()
|
||||
|
||||
# Populate is_important with FALSE for existing records where is_important is NULL
|
||||
print("Populating is_important column...")
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("""
|
||||
UPDATE sessions
|
||||
SET is_important = 0
|
||||
WHERE is_important IS NULL
|
||||
"""))
|
||||
conn.commit()
|
||||
|
||||
# Calculate and populate message_count from chat_messages table
|
||||
print("Calculating and populating message_count column...")
|
||||
with engine.connect() as conn:
|
||||
# First, set all message_count to 0
|
||||
conn.execute(text("UPDATE sessions SET message_count = 0"))
|
||||
|
||||
# Then, count messages for each session and update
|
||||
conn.execute(text("""
|
||||
UPDATE sessions
|
||||
SET message_count = (
|
||||
SELECT COUNT(*)
|
||||
FROM chat_messages
|
||||
WHERE chat_messages.session_id = sessions.id
|
||||
)
|
||||
"""))
|
||||
conn.commit()
|
||||
|
||||
print("Database update completed successfully!")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error updating database: {e}")
|
||||
db.rollback()
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
if __name__ == "__main__":
|
||||
update_database()
|
||||
|
||||
@@ -105,6 +105,8 @@ def _detect_nvidia():
|
||||
return None
|
||||
|
||||
gpus = []
|
||||
# Devices nvidia-smi lists with a real name but a non-numeric memory.total.
|
||||
unified = []
|
||||
# nvidia-smi lists GPUs in index order (0,1,2,...), so the row position is
|
||||
# the CUDA device index we'd pass to CUDA_VISIBLE_DEVICES.
|
||||
for idx, line in enumerate(out.strip().split("\n")):
|
||||
@@ -114,9 +116,32 @@ def _detect_nvidia():
|
||||
vram_mb = float(parts[0])
|
||||
gpus.append({"index": idx, "name": parts[1], "vram_gb": vram_mb / 1024.0})
|
||||
except ValueError:
|
||||
# Grace Blackwell GB10 / DGX Spark and other unified-memory
|
||||
# NVIDIA parts report memory.total as "[N/A]"/"Not Supported"
|
||||
# because the GPU shares the system LPDDR pool instead of
|
||||
# carrying discrete VRAM. Don't drop the device — remember it so
|
||||
# we report a unified-memory GPU below rather than "No GPU" (#1340).
|
||||
if parts[1]:
|
||||
unified.append({"index": idx, "name": parts[1]})
|
||||
continue
|
||||
|
||||
if not gpus:
|
||||
if unified:
|
||||
# Unified-memory CUDA box: report the GPU backed by system RAM so the
|
||||
# Cookbook recommends models and serving works. The pool is shared
|
||||
# (not per-GPU discrete VRAM), so report the RAM total once.
|
||||
ram_gb = round(_get_ram_gb(), 1)
|
||||
gpus = [{"index": g["index"], "name": g["name"], "vram_gb": ram_gb} for g in unified]
|
||||
return {
|
||||
"gpu_name": gpus[0]["name"],
|
||||
"gpu_vram_gb": ram_gb,
|
||||
"gpu_count": len(gpus),
|
||||
"gpus": gpus,
|
||||
"gpu_groups": _group_gpus(gpus),
|
||||
"homogeneous": True,
|
||||
"backend": "cuda",
|
||||
"unified_memory": True,
|
||||
}
|
||||
return None
|
||||
total_vram = sum(g["vram_gb"] for g in gpus)
|
||||
groups = _group_gpus(gpus)
|
||||
|
||||
+36
-15
@@ -6,8 +6,8 @@ YAML frontmatter and a structured markdown body (When to Use / Procedure /
|
||||
Pitfalls / Verification). See `skill_format.py` for the format.
|
||||
|
||||
Usage counters (`uses`, `last_used`) live in a sidecar
|
||||
`data/skills/_usage.json` keyed by skill name so the SKILL.md content
|
||||
doesn't churn on every retrieval.
|
||||
`data/skills/_usage.json` keyed by owner plus skill name so the SKILL.md
|
||||
content doesn't churn on every retrieval.
|
||||
|
||||
Ownership: skills declare `owner: <username>` in frontmatter. Single-user
|
||||
deployments can leave that blank.
|
||||
@@ -105,14 +105,29 @@ class SkillsManager:
|
||||
json.dump(usage, f, indent=2)
|
||||
os.replace(tmp, self.usage_file)
|
||||
|
||||
@staticmethod
|
||||
def _usage_key(name: str, owner: Optional[str] = None) -> str:
|
||||
# Skill names are not globally unique once multiple owners are present.
|
||||
# Keep the usage sidecar keyed the same way the skill file is scoped.
|
||||
return f"{owner}::{name}" if owner else name
|
||||
|
||||
def _usage_entry(self, usage: Dict[str, Dict], name: str, owner: Optional[str] = None) -> Dict:
|
||||
key = self._usage_key(name, owner)
|
||||
entry = usage.get(key)
|
||||
if isinstance(entry, dict):
|
||||
return entry
|
||||
return {}
|
||||
|
||||
def set_audit(self, name: str, verdict: str, by_teacher: bool = False,
|
||||
worker_model: str = "", teacher_model: str = "") -> None:
|
||||
worker_model: str = "", teacher_model: str = "",
|
||||
owner: Optional[str] = None) -> None:
|
||||
"""Record the last test/audit result for a skill in the usage sidecar
|
||||
(so it surfaces in load() without touching SKILL.md). Drives the
|
||||
'verified' check + teacher mark on the card."""
|
||||
import time as _t
|
||||
usage = self._load_usage()
|
||||
e = usage.setdefault(name, {"uses": 0, "last_used": None})
|
||||
key = self._usage_key(name, owner)
|
||||
e = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||
e["audit_verdict"] = verdict
|
||||
e["audit_by_teacher"] = bool(by_teacher)
|
||||
if worker_model:
|
||||
@@ -123,11 +138,13 @@ class SkillsManager:
|
||||
self._save_usage(usage)
|
||||
|
||||
def set_necessity(self, name: str, necessary: bool,
|
||||
redundant_with=None, reason: str = "") -> None:
|
||||
redundant_with=None, reason: str = "",
|
||||
owner: Optional[str] = None) -> None:
|
||||
"""Record the advisory 'is this skill necessary?' judgment in the usage
|
||||
sidecar. Surfaced on the card as a flag; never acts on the skill."""
|
||||
usage = self._load_usage()
|
||||
e = usage.setdefault(name, {"uses": 0, "last_used": None})
|
||||
key = self._usage_key(name, owner)
|
||||
e = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||
e["necessity"] = {
|
||||
"necessary": bool(necessary),
|
||||
"redundant_with": list(redundant_with or []),
|
||||
@@ -207,7 +224,7 @@ class SkillsManager:
|
||||
if not sk:
|
||||
continue
|
||||
d = sk.to_dict()
|
||||
u = usage.get(sk.name) or {}
|
||||
u = self._usage_entry(usage, sk.name, sk.owner)
|
||||
d["uses"] = int(u.get("uses", 0))
|
||||
d["last_used"] = u.get("last_used")
|
||||
d["audit_verdict"] = u.get("audit_verdict")
|
||||
@@ -308,6 +325,7 @@ class SkillsManager:
|
||||
# never auto-skipped — a human asked for it. The every-X AI audit
|
||||
# handles the fuzzier near-duplicates this cheap check won't catch.
|
||||
_all = self.load_all()
|
||||
_dedup_pool = _all if owner is None else [s for s in _all if s.get("owner") == owner]
|
||||
if source != "user":
|
||||
cand = _tokenize(" ".join([
|
||||
nm, (description or title or ""),
|
||||
@@ -315,7 +333,7 @@ class SkillsManager:
|
||||
" ".join(procedure if procedure is not None else (steps or [])),
|
||||
]))
|
||||
if cand:
|
||||
for s in _all:
|
||||
for s in _dedup_pool:
|
||||
ex = _tokenize(" ".join([
|
||||
s.get("name", ""), s.get("description", ""),
|
||||
s.get("when_to_use", ""),
|
||||
@@ -326,7 +344,7 @@ class SkillsManager:
|
||||
# existing skill's usage and return it so the caller
|
||||
# knows it already exists.
|
||||
try:
|
||||
self.record_use(s["name"])
|
||||
self.record_use(s["name"], owner=s.get("owner"))
|
||||
except Exception:
|
||||
pass
|
||||
return {**s, "_deduped": True, "_duplicate_of": s.get("name")}
|
||||
@@ -428,8 +446,9 @@ class SkillsManager:
|
||||
os.rename(old_dir, new_dir)
|
||||
# Also rename usage key
|
||||
usage = self._load_usage()
|
||||
if skill_id in usage:
|
||||
usage[sk.name] = usage.pop(skill_id)
|
||||
old_usage_key = self._usage_key(skill_id, sk.owner)
|
||||
if old_usage_key in usage:
|
||||
usage[self._usage_key(sk.name, sk.owner)] = usage.pop(old_usage_key)
|
||||
self._save_usage(usage)
|
||||
self._write_skill(sk)
|
||||
return True
|
||||
@@ -455,15 +474,17 @@ class SkillsManager:
|
||||
logger.warning(f"Failed to remove skill dir {skill_dir}: {e}")
|
||||
return False
|
||||
usage = self._load_usage()
|
||||
if skill_id in usage:
|
||||
del usage[skill_id]
|
||||
usage_key = self._usage_key(skill_id, sk.owner)
|
||||
if usage_key in usage:
|
||||
del usage[usage_key]
|
||||
self._save_usage(usage)
|
||||
return True
|
||||
return False
|
||||
|
||||
def record_use(self, skill_id: str) -> None:
|
||||
def record_use(self, skill_id: str, owner: Optional[str] = None) -> None:
|
||||
usage = self._load_usage()
|
||||
entry = usage.setdefault(skill_id, {"uses": 0, "last_used": None})
|
||||
key = self._usage_key(skill_id, owner)
|
||||
entry = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||
entry["uses"] = int(entry.get("uses", 0)) + 1
|
||||
entry["last_used"] = int(time.time())
|
||||
self._save_usage(usage)
|
||||
|
||||
@@ -92,6 +92,7 @@ class ResearchService:
|
||||
relevance=s.get("relevance", 0.0),
|
||||
)
|
||||
for s in result.get("sources", [])
|
||||
if isinstance(s, dict)
|
||||
]
|
||||
return ResearchResult(
|
||||
query=topic,
|
||||
|
||||
@@ -13,6 +13,8 @@ logger = logging.getLogger(__name__)
|
||||
# ----------------------------------------------------------------------
|
||||
def _detect_question_type(query: str) -> Optional[str]:
|
||||
"""Return the leading question word if present (who, what, when, where, why, how)."""
|
||||
if not isinstance(query, str):
|
||||
return None
|
||||
q = query.strip().lower()
|
||||
for word in ("who", "what", "when", "where", "why", "how"):
|
||||
# Require a whole-word match: a bare prefix mis-flags ordinary queries
|
||||
@@ -45,12 +47,16 @@ def _extract_entities(query: str) -> Dict[str, List[str]]:
|
||||
|
||||
def _split_multi_part(query: str) -> List[str]:
|
||||
"""Split a query into sub-queries on common conjunctions."""
|
||||
if not isinstance(query, str):
|
||||
return []
|
||||
parts = re.split(r"\s+and\s+|\s+or\s+|;", query, flags=re.I)
|
||||
return [p.strip() for p in parts if p.strip()]
|
||||
|
||||
|
||||
def _extract_site_filter(query: str) -> Tuple[str, Optional[str]]:
|
||||
"""Detect a 'site:example.com' token. Returns (query_without_token, site_or_None)."""
|
||||
if not isinstance(query, str):
|
||||
return "", None
|
||||
match = re.search(r"\bsite:([^\s]+)", query, flags=re.I)
|
||||
if match:
|
||||
site = match.group(1)
|
||||
@@ -71,6 +77,8 @@ def _boost_entities_in_query(base_query: str, entities: Dict[str, List[str]]) ->
|
||||
|
||||
def enhance_query(original_query: str) -> Tuple[str, Optional[str]]:
|
||||
"""Process the original query: site filter, question type boosts, entity extraction."""
|
||||
if not isinstance(original_query, str):
|
||||
original_query = ""
|
||||
query_without_site, site = _extract_site_filter(original_query)
|
||||
sub_queries = _split_multi_part(query_without_site)
|
||||
|
||||
@@ -120,6 +128,8 @@ def build_enhanced_query(query: str, time_filter: str = None) -> str:
|
||||
def _is_news_query(query: str) -> bool:
|
||||
"""Lightweight heuristic to decide if a query is news-oriented."""
|
||||
news_terms = {"news", "latest", "breaking", "today", "today's", "current", "updates", "happening"}
|
||||
if not isinstance(query, str):
|
||||
return False
|
||||
tokens = set(re.findall(r"\b\w+\b", query.lower()))
|
||||
return bool(tokens & news_terms)
|
||||
|
||||
|
||||
+46
-9
@@ -276,7 +276,7 @@ Generate an image. Line 1 = description, line 2 = model name, line 3 = WxH (e.g.
|
||||
"manage_webhooks": "- ```manage_webhooks``` — Configure outgoing webhooks (HTTP notifications on events like chat completion). Args (JSON): {\"action\": \"list|add|delete|enable|disable\", ...}",
|
||||
"manage_tokens": "- ```manage_tokens``` — Generate or revoke API access tokens for external integrations. Args (JSON): {\"action\": \"list|create|delete\", ...}",
|
||||
"manage_documents": "- ```manage_documents``` — List, read/open, delete, or tidy documents in the editor panel. Args (JSON): {\"action\": \"list|read|delete|tidy\", ...}. `list` returns rows like `[Title](#document-<id>) — lang, size, updated 5m ago` sorted MOST-RECENT FIRST; the user clicks the anchor to open. `read` (aliases: view/open/get) takes `document_id` and returns the content. When the user asks \"open/show/read my notes\" or \"what documents do I have\", use this — do NOT shell out, do NOT curl.",
|
||||
"manage_research": "- ```manage_research``` — List, read/open, or delete saved DEEP RESEARCH results from the Library. Args (JSON): {\"action\": \"list|read|delete\", \"id\": \"<id>\", \"search\": \"...\"}. `list` returns rows like `[query](#research-<id>) — N sources` MOST-RECENT FIRST; the user clicks to open. `read` (aliases: open/view/get) takes `id` and returns the report + sources. Use when the user says \"open/read/find/delete my research\" or \"that report\". To START new research, use trigger_research instead.",
|
||||
"manage_research": "- ```manage_research``` — List, read/open, or delete saved DEEP RESEARCH results from the Library. Args (JSON): {\"action\": \"list|read|delete\", \"id\": \"<id>\", \"search\": \"...\"}. `list` returns rows like `[query](#research-<id>) — N sources` MOST-RECENT FIRST; the user clicks to open. `read` (aliases: open/view/get) takes `id` and returns the report text + sources. Use when the user says \"open/read/find/delete my research\" or \"that report\". This IS how you read a finished report: when the user refers to a just-completed deep-research job (\"check it out\", \"read that report\", \"summarize the research\") WITHOUT giving an id, call `manage_research` with `action:list` to get the most-recent id, then `action:read` with that id, and answer from the returned text. Do NOT `web_fetch`/`app_api` the `/api/research/report/{id}` URL — that endpoint renders HTML for the browser, not clean text — and do NOT start a fresh `web_search`/`trigger_research` just to read an existing report. To START new research, use trigger_research instead.",
|
||||
"manage_settings": "- ```manage_settings``` — View/change the REAL app settings (same ones the Settings panel writes) AND turn tools on/off. Change a setting: `{\"action\":\"set\",\"key\":\"...\",\"value\":\"...\"}` — keys accept friendly aliases, e.g. voice→tts_voice, \"search engine\"→search_provider, \"default model\"→default_model, \"teacher model\"→teacher_model, \"task/background model\"→task_model, \"image quality\"→image_quality, \"reminder channel\"→reminder_channel (browser|email|ntfy), \"agent timeout\"/\"max tool calls\"/\"token budget\". Read: `{\"action\":\"get\",\"key\":\"...\"}`; see all: `{\"action\":\"list\"}`; reset one: `{\"action\":\"reset\",\"key\":\"...\"}`. Use this when the user asks to change ANY preference instead of making them open Settings. Secrets/API keys are read-only (tell them to set those in the panel). Tool toggles: `{\"action\":\"disable_tool|enable_tool\",\"tool\":\"shell\"}` (aliases: shell/search/browser/documents/memory/skills/images/tasks/notes/calendar/email), list disabled: `{\"action\":\"list_tools\"}`.",
|
||||
"manage_notes": """\
|
||||
```manage_notes
|
||||
@@ -315,9 +315,10 @@ Bulk delete/archive/mark emails. Use this for "delete all those" after listing e
|
||||
{"action": "create_event", "summary": "<event title>", "dtstart": "<natural language or ISO datetime>"}
|
||||
```
|
||||
Calendar event management (CalDAV). Actions: `list_events`, `create_event`, `update_event`, `delete_event`, `list_calendars`. \
|
||||
For `create_event`: {summary, dtstart, dtend?, duration?, calendar?, location?, description?, reminder_minutes?}. \
|
||||
For `create_event`: {summary, dtstart, dtend?, duration?, calendar?, location?, description?, reminder_minutes?, rrule?}. \
|
||||
`dtstart` accepts natural language ("tomorrow at 1pm", "in 2 hours", "next monday 9am") or ISO ("2026-05-12T13:00:00"). \
|
||||
If `dtend` omitted, defaults to dtstart+1h (or +1d when `all_day: true`). \
|
||||
For a RECURRING event pass `rrule` as an iCalendar RRULE string, e.g. `"FREQ=WEEKLY;BYDAY=MO"` (every Monday), `"FREQ=DAILY;COUNT=10"`, or `"FREQ=MONTHLY;BYMONTHDAY=1"` — create ONE event with the rrule, do not loop creating many events. \
|
||||
If the user asks for a reminder/alarm before the event, pass `reminder_minutes` as an integer; do not write reminder text into the event description and do NOT also call `manage_notes` for the same reminder because calendar reminders are routed through Notes automatically. \
|
||||
`calendar` accepts a name ("Main") or short-id prefix.""",
|
||||
"create_session": "- ```create_session``` — Create a new chat. Line 1 = chat name, line 2 = model name. Use for background/parallel work.",
|
||||
@@ -353,7 +354,7 @@ GENERIC LOOPBACK to ANY Odysseus internal endpoint. Use this whenever the user w
|
||||
- Sessions: `/api/sessions`, `/api/session/{id}`, `/api/session/{id}/truncate`
|
||||
- Themes: `/api/prefs/themes`, `/api/prefs/custom-themes`
|
||||
- Settings: `/api/settings`, `/api/prefs/{key}`
|
||||
- Research: `/api/research/start`, `/api/research/tasks`, `/api/research/report/{id}`
|
||||
- Research: `/api/research/start`, `/api/research/tasks` (note: `/api/research/report/{id}` renders HTML — to READ a report's text use the `manage_research` tool with `action:read`, not this endpoint)
|
||||
- Compare: `/api/compare/sessions`, `/api/compare/start`
|
||||
- Email: use named email tools (`list_email_accounts`, `list_emails`, `read_email`, `send_email`, `reply_to_email`). Do NOT use `/api/email/accounts`; it is owner-filtered in tool context and may falsely return empty.
|
||||
- Endpoints (model providers): `/api/endpoints`, `/api/endpoints/{id}`
|
||||
@@ -862,7 +863,7 @@ def _build_system_prompt(
|
||||
# matter how often it's been matched and applied.
|
||||
for _sk in relevant_skills:
|
||||
try:
|
||||
sm.record_use(_sk.get('name', ''))
|
||||
sm.record_use(_sk.get('name', ''), owner=owner)
|
||||
except Exception:
|
||||
pass
|
||||
lines.append("## Relevant skills for this request")
|
||||
@@ -1314,6 +1315,30 @@ async def _run_verifier_subagent(
|
||||
return [r.strip() for r in reasons.split(";") if r.strip()]
|
||||
|
||||
|
||||
def _empty_response_fallback(
|
||||
full_response: str,
|
||||
round_reasoning: str,
|
||||
tool_events: list,
|
||||
) -> tuple:
|
||||
"""Return (final_response, sse_chunk_or_none) for the end-of-loop empty-response guard.
|
||||
|
||||
When a thinking model routes all tokens to reasoning_content (leaving
|
||||
content=""), full_response is empty but round_reasoning has content.
|
||||
The reasoning was already streamed as {thinking:true} chunks — do not
|
||||
re-emit it as a normal delta. Just persist it and yield nothing.
|
||||
|
||||
Returns:
|
||||
(final_response: str, chunk: str | None)
|
||||
chunk is the SSE string to yield, or None if nothing should be emitted.
|
||||
"""
|
||||
if full_response.strip() or tool_events:
|
||||
return full_response, None
|
||||
if round_reasoning.strip():
|
||||
return round_reasoning, None
|
||||
_error_msg = "The model returned an empty response. Please try again or switch to a different model."
|
||||
return _error_msg, f'data: {json.dumps({"delta": _error_msg})}\n\n'
|
||||
|
||||
|
||||
async def stream_agent_loop(
|
||||
endpoint_url: str,
|
||||
model: str,
|
||||
@@ -1487,13 +1512,23 @@ async def stream_agent_loop(
|
||||
_t3 = time.time()
|
||||
try:
|
||||
from src.context_compactor import trim_for_context
|
||||
from src.context_budget import compute_input_token_budget
|
||||
from src.context_budget import compute_input_token_budget, DEFAULT_HARD_MAX
|
||||
from src.settings import is_setting_overridden
|
||||
|
||||
soft_budget = int(get_setting("agent_input_token_budget", 6000) or 0)
|
||||
if soft_budget > 0:
|
||||
before_trim_tokens = estimate_tokens(messages)
|
||||
reserve_tokens = min(max(max_tokens or 1024, 512), 2048)
|
||||
# Honour the configurable ceiling for the auto-derived budget path.
|
||||
# No-op when the user has an explicit `agent_input_token_budget`
|
||||
# (that branch ignores hard_max). Falls back to DEFAULT_HARD_MAX
|
||||
# on missing/malformed values so misconfig can't zero the budget.
|
||||
try:
|
||||
hard_max = int(get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX) or DEFAULT_HARD_MAX)
|
||||
except (TypeError, ValueError):
|
||||
hard_max = DEFAULT_HARD_MAX
|
||||
if hard_max <= 0:
|
||||
hard_max = DEFAULT_HARD_MAX
|
||||
# Scale the default budget to the model's context window so long-context
|
||||
# models aren't silently capped at 6000; an explicit user setting is
|
||||
# still honoured (clamped to the window). (#1170)
|
||||
@@ -1501,6 +1536,7 @@ async def stream_agent_loop(
|
||||
soft_budget,
|
||||
context_length,
|
||||
is_setting_overridden("agent_input_token_budget"),
|
||||
hard_max=hard_max,
|
||||
)
|
||||
trimmed_messages = trim_for_context(
|
||||
messages,
|
||||
@@ -2214,10 +2250,11 @@ async def stream_agent_loop(
|
||||
|
||||
# If the response is completely empty and no tools were executed,
|
||||
# yield a fallback message so the user is not left hanging.
|
||||
if not full_response.strip() and not tool_events:
|
||||
_error_msg = "The model returned an empty response. Please try again or switch to a different model."
|
||||
yield f'data: {json.dumps({"delta": _error_msg})}\n\n'
|
||||
full_response = _error_msg
|
||||
full_response, _fallback_chunk = _empty_response_fallback(
|
||||
full_response, round_reasoning, tool_events
|
||||
)
|
||||
if _fallback_chunk:
|
||||
yield _fallback_chunk
|
||||
|
||||
# --- Final metrics ---
|
||||
total_duration = time.time() - total_start
|
||||
|
||||
+12
-2
@@ -48,8 +48,18 @@ class APIKeyManager:
|
||||
"""Load and decrypt API keys"""
|
||||
if not os.path.exists(self.api_keys_file):
|
||||
return {}
|
||||
with open(self.api_keys_file, 'r', encoding="utf-8") as f:
|
||||
encrypted_keys = json.load(f)
|
||||
try:
|
||||
with open(self.api_keys_file, 'r', encoding="utf-8") as f:
|
||||
encrypted_keys = json.load(f)
|
||||
except (json.JSONDecodeError, OSError) as e:
|
||||
# A corrupt/truncated api_keys.json must not crash load() (called on
|
||||
# startup via app_initializer) — treat it as no stored keys.
|
||||
logger.warning("Failed to read API keys file: %s", e)
|
||||
return {}
|
||||
if not isinstance(encrypted_keys, dict):
|
||||
# Legacy/wrong shape (e.g. a list) — .items() would raise. Ignore it.
|
||||
logger.warning("API keys file has unexpected shape (%s); ignoring", type(encrypted_keys).__name__)
|
||||
return {}
|
||||
|
||||
decrypted = {}
|
||||
for provider, key in encrypted_keys.items():
|
||||
|
||||
@@ -22,6 +22,8 @@ def abs_join(base_dir: str, rel: str) -> str:
|
||||
|
||||
def inside_base_dir(base_dir: str, path: str) -> bool:
|
||||
"""Check if path is inside base directory."""
|
||||
if not isinstance(base_dir, str) or not isinstance(path, str):
|
||||
return False
|
||||
base = os.path.realpath(base_dir)
|
||||
p = os.path.realpath(path)
|
||||
try:
|
||||
|
||||
@@ -478,7 +478,7 @@ def _result_has_work(result: str | None) -> bool:
|
||||
'No new emails to summarize', 'Tagged 0 / Moved 0', etc. when nothing
|
||||
was done. Used to decide whether to record the run or noop it.
|
||||
"""
|
||||
if not result:
|
||||
if not isinstance(result, str) or not result:
|
||||
return False
|
||||
low = result.lower()
|
||||
if "processed 0" in low or "no new" in low or "nothing to" in low:
|
||||
@@ -554,7 +554,7 @@ _HEURISTIC_CRITICAL = ["surgery", "court", "wedding day", "funeral", "delivery d
|
||||
|
||||
def _classify_event_heuristic(summary: str) -> tuple:
|
||||
"""Quick heuristic classification — returns (event_type, importance) or (None, None) if unclear."""
|
||||
s = (summary or "").lower()
|
||||
s = (summary if isinstance(summary, str) else "").lower()
|
||||
etype = None
|
||||
for t, kws in _HEURISTIC_TYPES.items():
|
||||
if any(k in s for k in kws):
|
||||
@@ -1313,7 +1313,7 @@ async def action_test_skills(owner: str, **kwargs) -> Tuple[str, bool]:
|
||||
if not names:
|
||||
raise TaskNoop("no skills to test")
|
||||
|
||||
url, model, headers = resolve_endpoint("default")
|
||||
url, model, headers = resolve_endpoint("default", owner=owner)
|
||||
if not url or not model:
|
||||
return "No Default/Utility model configured — set one in Settings.", False
|
||||
|
||||
@@ -1374,7 +1374,7 @@ async def action_test_skills(owner: str, **kwargs) -> Tuple[str, bool]:
|
||||
# user-set value (e.g. 1.0 → 0.95) is destructive.
|
||||
if v in ("pass", "needs_work", "fail", "inconclusive"):
|
||||
try:
|
||||
sm.set_audit(name, v, by_teacher=False, worker_model=model)
|
||||
sm.set_audit(name, v, by_teacher=False, worker_model=model, owner=owner)
|
||||
except Exception as _e:
|
||||
logger.warning(f"test_skills set_audit({name}) failed: {_e}")
|
||||
if v == "unknown":
|
||||
|
||||
@@ -24,9 +24,12 @@ Design notes:
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import logging
|
||||
import os
|
||||
import uuid
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
from urllib.parse import urlparse, urlunparse
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -35,6 +38,52 @@ logger = logging.getLogger(__name__)
|
||||
# events still come through via RRULE expansion on the frontend.
|
||||
_LOOKBACK_DAYS = 90
|
||||
_LOOKAHEAD_DAYS = 365
|
||||
_BLOCKED_HOSTS = {
|
||||
"localhost",
|
||||
"localhost.",
|
||||
"ip6-localhost",
|
||||
"metadata.google.internal",
|
||||
}
|
||||
|
||||
|
||||
def _private_caldav_allowed() -> bool:
|
||||
return os.environ.get("ODYSSEUS_ALLOW_PRIVATE_CALDAV", "0").lower() in {"1", "true", "yes"}
|
||||
|
||||
|
||||
def _validate_caldav_ip(host: str) -> None:
|
||||
try:
|
||||
ip = ipaddress.ip_address(host.strip("[]"))
|
||||
except ValueError:
|
||||
return
|
||||
if ip.is_loopback or ip.is_link_local or ip.is_multicast or ip.is_unspecified:
|
||||
raise ValueError("CalDAV URL host is not allowed")
|
||||
if ip.is_private and not _private_caldav_allowed():
|
||||
raise ValueError("Private CalDAV IPs require ODYSSEUS_ALLOW_PRIVATE_CALDAV=1")
|
||||
|
||||
|
||||
def validate_caldav_url(raw_url: str) -> str:
|
||||
"""Validate and normalize a user-provided CalDAV URL before server-side use."""
|
||||
url = (raw_url if isinstance(raw_url, str) else "").strip()
|
||||
if not url:
|
||||
raise ValueError("CalDAV URL is required")
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in {"http", "https"}:
|
||||
raise ValueError("CalDAV URL must start with http:// or https://")
|
||||
if not parsed.hostname:
|
||||
raise ValueError("CalDAV URL must include a host")
|
||||
if parsed.username or parsed.password:
|
||||
raise ValueError("Put CalDAV credentials in the username/password fields, not the URL")
|
||||
if parsed.fragment:
|
||||
raise ValueError("CalDAV URL fragments are not allowed")
|
||||
try:
|
||||
parsed.port
|
||||
except ValueError:
|
||||
raise ValueError("CalDAV URL has an invalid port")
|
||||
host = (parsed.hostname or "").lower()
|
||||
if host in _BLOCKED_HOSTS or host.endswith(".localhost"):
|
||||
raise ValueError("CalDAV URL host is not allowed")
|
||||
_validate_caldav_ip(host)
|
||||
return urlunparse(parsed._replace(fragment="")).rstrip("/")
|
||||
|
||||
|
||||
def _stable_cal_id(remote_url: str) -> str:
|
||||
@@ -250,13 +299,21 @@ async def sync_caldav(owner: str) -> dict:
|
||||
url = (cfg.get("url") or "").strip()
|
||||
user = (cfg.get("username") or "").strip()
|
||||
pw = cfg.get("password") or ""
|
||||
try:
|
||||
from src.secret_storage import decrypt
|
||||
pw = decrypt(pw)
|
||||
except Exception:
|
||||
pass
|
||||
if not (url and user and pw):
|
||||
return {
|
||||
"calendars": 0, "events": 0, "deleted": 0,
|
||||
"errors": ["CalDAV is not configured"],
|
||||
}
|
||||
try:
|
||||
url = validate_caldav_url(url)
|
||||
return await asyncio.to_thread(_sync_blocking, owner, url, user, pw)
|
||||
except ValueError as e:
|
||||
return {"calendars": 0, "events": 0, "deleted": 0, "errors": [str(e)]}
|
||||
except Exception as e:
|
||||
logger.exception("CalDAV sync raised")
|
||||
return {"calendars": 0, "events": 0, "deleted": 0, "errors": [str(e)[:200]]}
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
"""CalDAV write-back: push local create/update/delete out to the remote (#800).
|
||||
|
||||
``src/caldav_sync.py`` is a one-way pull (remote → local). So events created,
|
||||
edited, or deleted in Odysseus on a CalDAV-backed calendar only changed the local
|
||||
SQLite copy and never reached the server (iCloud/Nextcloud/Radicale/Fastmail) —
|
||||
they'd silently disappear on the next pull and never show on the user's phone.
|
||||
|
||||
This adds the missing write half. The remote calendar URL isn't stored locally
|
||||
(the local calendar id is a one-way hash of it), so we re-discover the remote
|
||||
calendar by matching that same hash, then PUT/DELETE the VEVENT by its UID via
|
||||
the `caldav` lib. Writes are best-effort: the local DB stays the source of truth,
|
||||
and a remote failure is reported, never fatal to the local operation.
|
||||
|
||||
The pure pieces (``build_event_ical``, ``find_remote_calendar``, ``push_event``)
|
||||
take their inputs by argument so they unit-test against a fake client with no
|
||||
network.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import timezone
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _stable_cal_id(remote_url: str) -> str:
|
||||
# Reuse the sync module's hashing so a local CalDAV calendar id maps back to
|
||||
# the same remote URL it was pulled from.
|
||||
from src.caldav_sync import _stable_cal_id as _sync_id
|
||||
return _sync_id(remote_url)
|
||||
|
||||
|
||||
def build_event_ical(ev: dict) -> str:
|
||||
"""Serialize a local event dict to a VCALENDAR/VEVENT iCalendar string.
|
||||
|
||||
``ev`` keys: uid, summary, description, location, dtstart (datetime),
|
||||
dtend (datetime), all_day (bool), is_utc (bool), rrule (str).
|
||||
Mirrors how the pull path interprets is_utc/all_day so a round-trip is stable.
|
||||
"""
|
||||
from icalendar import Calendar, Event as iEvent
|
||||
from icalendar.prop import vRecur
|
||||
|
||||
cal = Calendar()
|
||||
cal.add("prodid", "-//Odysseus//CalDAV write-back//EN")
|
||||
cal.add("version", "2.0")
|
||||
|
||||
ve = iEvent()
|
||||
ve.add("uid", ev["uid"])
|
||||
ve.add("summary", ev.get("summary") or "")
|
||||
if ev.get("description"):
|
||||
ve.add("description", ev["description"])
|
||||
if ev.get("location"):
|
||||
ve.add("location", ev["location"])
|
||||
|
||||
dtstart = ev["dtstart"]
|
||||
dtend = ev["dtend"]
|
||||
if ev.get("all_day"):
|
||||
ve.add("dtstart", dtstart.date())
|
||||
ve.add("dtend", dtend.date())
|
||||
elif ev.get("is_utc"):
|
||||
# Stored as naive-UTC instants — re-attach UTC so the server gets a Z time.
|
||||
ve.add("dtstart", dtstart.replace(tzinfo=timezone.utc))
|
||||
ve.add("dtend", dtend.replace(tzinfo=timezone.utc))
|
||||
else:
|
||||
# Legacy naive-local ("floating") time — emit without a TZ.
|
||||
ve.add("dtstart", dtstart)
|
||||
ve.add("dtend", dtend)
|
||||
|
||||
if ev.get("rrule"):
|
||||
try:
|
||||
ve.add("rrule", vRecur.from_ical(ev["rrule"]))
|
||||
except Exception:
|
||||
logger.debug("CalDAV write-back: skipping unparseable rrule %r", ev.get("rrule"))
|
||||
|
||||
cal.add_component(ve)
|
||||
return cal.to_ical().decode("utf-8")
|
||||
|
||||
|
||||
def find_remote_calendar(calendars, local_cal_id: str):
|
||||
"""Find the remote calendar whose URL hashes to ``local_cal_id``, or None."""
|
||||
for cal in calendars:
|
||||
try:
|
||||
if _stable_cal_id(str(cal.url)) == local_cal_id:
|
||||
return cal
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def push_event(calendars, local_cal_id: str, ev: dict, *, delete: bool = False) -> dict:
|
||||
"""Create/update (or delete) ``ev`` on the matching remote calendar.
|
||||
|
||||
Returns ``{"ok": bool, ...}``. ``calendars`` is the discovered caldav
|
||||
calendar list (injected so this is unit-testable with fakes).
|
||||
"""
|
||||
uid = (ev or {}).get("uid") if isinstance(ev, dict) else None
|
||||
if not uid:
|
||||
return {"ok": False, "error": "event uid is required"}
|
||||
|
||||
remote = find_remote_calendar(calendars, local_cal_id)
|
||||
if remote is None:
|
||||
return {"ok": False, "error": "remote calendar not found"}
|
||||
|
||||
try:
|
||||
existing = remote.event_by_uid(uid)
|
||||
except Exception:
|
||||
existing = None
|
||||
|
||||
if delete:
|
||||
if existing is None:
|
||||
return {"ok": True, "note": "already absent on remote"}
|
||||
existing.delete()
|
||||
return {"ok": True}
|
||||
|
||||
ical = build_event_ical(ev)
|
||||
if existing is not None:
|
||||
existing.data = ical
|
||||
existing.save()
|
||||
return {"ok": True, "updated": True}
|
||||
remote.save_event(ical)
|
||||
return {"ok": True, "created": True}
|
||||
|
||||
|
||||
def _discover_calendars(client):
|
||||
"""Discover the principal's calendars, falling back to the URL itself —
|
||||
same strategy as the pull path."""
|
||||
from caldav.lib.error import AuthorizationError, NotFoundError
|
||||
try:
|
||||
return client.principal().calendars()
|
||||
except (AuthorizationError, NotFoundError):
|
||||
raise
|
||||
except Exception:
|
||||
try:
|
||||
return [client.calendar(url=str(client.url))]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def _writeback_blocking(local_cal_id, ev, delete, url, username, password) -> dict:
|
||||
import caldav
|
||||
client = caldav.DAVClient(url=url, username=username, password=password)
|
||||
calendars = _discover_calendars(client)
|
||||
if not calendars:
|
||||
return {"ok": False, "error": "no remote calendars discovered"}
|
||||
return push_event(calendars, local_cal_id, ev, delete=delete)
|
||||
|
||||
|
||||
async def writeback_event(owner: str, calendar_source: str, calendar_id: str,
|
||||
ev: dict, *, delete: bool = False) -> dict:
|
||||
"""Best-effort push of a local change to the remote CalDAV server.
|
||||
|
||||
No-ops (``{"skipped": ...}``) when the calendar isn't CalDAV-backed or no
|
||||
credentials are configured. Never raises — a remote failure is logged and
|
||||
returned, the local DB remaining the source of truth.
|
||||
"""
|
||||
if calendar_source != "caldav":
|
||||
return {"skipped": "not a caldav calendar"}
|
||||
try:
|
||||
from routes.prefs_routes import _load_for_user
|
||||
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||
url = (cfg.get("url") or "").strip()
|
||||
user = (cfg.get("username") or "").strip()
|
||||
pw = cfg.get("password") or ""
|
||||
if not (url and user and pw):
|
||||
return {"skipped": "caldav not configured"}
|
||||
result = await asyncio.to_thread(_writeback_blocking, calendar_id, ev, delete, url, user, pw)
|
||||
if not result.get("ok"):
|
||||
logger.warning("CalDAV write-back did not apply: %s", result.get("error") or result)
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.exception("CalDAV write-back raised")
|
||||
return {"ok": False, "error": str(e)[:200]}
|
||||
+9
-1
@@ -36,8 +36,16 @@ _VISION_MODEL_KEYWORDS = (
|
||||
"gpt-4o", "gpt-4.1", "gpt-4.5", "gpt-4-turbo", "gpt-4-vision",
|
||||
"claude-sonnet", "claude-opus", "claude-haiku", "gemini",
|
||||
# open / local
|
||||
"vision", "llava", "bakllava", "moondream", "pixtral", "minicpm",
|
||||
"vision", "multimodal", "llava", "bakllava", "moondream", "pixtral", "minicpm",
|
||||
"internvl", "cogvlm", "qwen-vl", "qwen2-vl", "qwen3-vl", "qwen3vl",
|
||||
# multimodal families whose names don't contain "vision"/"vl" but DO accept
|
||||
# images — without these the image is silently dropped for common Ollama tags
|
||||
# like gemma3:4b (issue #1274). Gemma 3 (4b+), Llama 4 (all), and Mistral
|
||||
# Small 3.1/3.2 are vision-capable; per the err-toward-True policy (#124) a
|
||||
# rare text-only tag (e.g. gemma3:1b) being treated as vision is the safer
|
||||
# failure than dropping a real image.
|
||||
"gemma-3", "gemma3", "llama-4", "llama4",
|
||||
"mistral-small-3.1", "mistral-small3.1", "mistral-small-3.2", "mistral-small3.2",
|
||||
# zhipu / glm (glm-4.5v, glm-4.6v, glm-5v-turbo, etc.)
|
||||
"glm-4.5v", "glm-4.6v", "glm-5v",
|
||||
)
|
||||
|
||||
+47
-3
@@ -11,6 +11,7 @@ import json
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from datetime import datetime
|
||||
from typing import Callable, Dict, List, Optional, Set
|
||||
|
||||
from src.research_utils import strip_thinking, is_low_quality
|
||||
@@ -19,6 +20,20 @@ from src.goal_based_extractor import EXTRACTOR_PROMPT
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def current_date_context() -> str:
|
||||
"""Preamble that grounds query-generation/planning LLMs in the real current
|
||||
date. Without it the model falls back to its training-cutoff year and emits
|
||||
queries like "best Python tutorials 2025" when the year is actually 2026.
|
||||
System TZ-local so it matches what the user sees. Portable strftime only."""
|
||||
now = datetime.now().astimezone()
|
||||
return (
|
||||
f"Today's date is {now.strftime('%B %d, %Y')} ({now.strftime('%Y-%m-%d')}). "
|
||||
f"When a search query needs a year or refers to 'latest'/'current'/"
|
||||
f"'this year', use {now.strftime('%Y')} or relative wording — never a "
|
||||
f"year inferred from training data.\n\n"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prompts
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -329,6 +344,16 @@ class DeepResearcher:
|
||||
self._emit(phase="writing", total_sources=len(self.urls_fetched),
|
||||
total_findings=len(findings))
|
||||
if not report:
|
||||
# Synthesis can fail (e.g. the LLM timed out) even though the search
|
||||
# rounds did gather findings. Don't throw that work away — return the
|
||||
# gathered findings as a basic compiled report instead of claiming
|
||||
# nothing was found (#1551).
|
||||
if findings:
|
||||
logger.warning(
|
||||
"Synthesis produced no report; returning %d gathered "
|
||||
"finding(s) as a fallback", len(findings)
|
||||
)
|
||||
return self._fallback_report(question, findings)
|
||||
return "No information could be gathered for this question."
|
||||
|
||||
self.evolving_report = report # preserve pre-synthesis report
|
||||
@@ -364,7 +389,7 @@ class DeepResearcher:
|
||||
# ------------------------------------------------------------------
|
||||
async def _create_plan(self, question: str) -> str:
|
||||
"""LLM analyzes the question and creates a research plan."""
|
||||
prompt = RESEARCH_PLAN_PROMPT.format(question=question)
|
||||
prompt = current_date_context() + RESEARCH_PLAN_PROMPT.format(question=question)
|
||||
try:
|
||||
response = await self._llm(
|
||||
[{"role": "user", "content": prompt}],
|
||||
@@ -439,7 +464,7 @@ class DeepResearcher:
|
||||
"that the report doesn't yet cover well."
|
||||
)
|
||||
|
||||
prompt = QUERY_GEN_PROMPT.format(
|
||||
prompt = current_date_context() + QUERY_GEN_PROMPT.format(
|
||||
question=question,
|
||||
research_plan=self.research_plan or "(No plan — search broadly.)",
|
||||
report=report or "(No findings yet.)",
|
||||
@@ -647,7 +672,11 @@ class DeepResearcher:
|
||||
[{"role": "user", "content": prompt}],
|
||||
temperature=0.3,
|
||||
max_tokens=self.max_report_tokens,
|
||||
timeout=60,
|
||||
# Synthesis is a heavy generation call like the final report
|
||||
# (which gets 180s); a slow local model (e.g. a 20B served from
|
||||
# LM Studio) routinely needs >60s for it. The old 60s cap timed
|
||||
# out mid-stream and discarded the round's findings (#1551).
|
||||
timeout=180,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Synthesis failed: {e}")
|
||||
@@ -826,6 +855,21 @@ class DeepResearcher:
|
||||
parts.append(f"**Finding {i}** — [{title}]({url})\n{content}")
|
||||
return "\n\n".join(parts)
|
||||
|
||||
def _fallback_report(self, question: str, findings: List[Dict]) -> str:
|
||||
"""Compile gathered findings into a basic report.
|
||||
|
||||
Used when the LLM synthesis step produced no report (e.g. it timed out)
|
||||
but the search rounds did collect findings — so the user still gets the
|
||||
material that was gathered instead of "No information could be gathered"
|
||||
(#1551).
|
||||
"""
|
||||
return (
|
||||
f"# {question}\n\n"
|
||||
"_Automatic synthesis did not complete, so this report lists the "
|
||||
f"{len(findings)} finding(s) gathered during research._\n\n"
|
||||
f"{self._format_findings(findings)}"
|
||||
)
|
||||
|
||||
def get_stats(self) -> Dict:
|
||||
"""Return research statistics."""
|
||||
elapsed = time.time() - self._start_time if self._start_time else 0
|
||||
|
||||
@@ -21,7 +21,8 @@ _JUNK_TITLES = {
|
||||
|
||||
def _norm_title(t: str) -> str:
|
||||
"""Normalize a title for grouping: trim, collapse whitespace, lowercase."""
|
||||
return re.sub(r"\s+", " ", (t or "").strip()).lower()
|
||||
t = t if isinstance(t, str) else ""
|
||||
return re.sub(r"\s+", " ", t.strip()).lower()
|
||||
|
||||
|
||||
def _content_fingerprint(content: str) -> str:
|
||||
@@ -32,7 +33,7 @@ def _content_fingerprint(content: str) -> str:
|
||||
that N imports of the same file collapse to one fingerprint. Whitespace is
|
||||
collapsed and the result lowercased.
|
||||
"""
|
||||
c = content or ""
|
||||
c = content if isinstance(content, str) else ""
|
||||
c = re.sub(r'upload_id="[^"]*"', "upload_id", c) # pdf_source re-imports
|
||||
c = re.sub(r"\bid=ann-[A-Za-z0-9_-]+", "id=ann", c) # annotation ids
|
||||
c = re.sub(r"\s+", " ", c).strip().lower()
|
||||
@@ -41,7 +42,8 @@ def _content_fingerprint(content: str) -> str:
|
||||
|
||||
def _real_len(content: str) -> int:
|
||||
"""Length of content with markdown noise stripped — a 'completeness' proxy."""
|
||||
stripped = re.sub(r"^#{1,6}\s+", "", content or "", flags=re.MULTILINE)
|
||||
content = content if isinstance(content, str) else ""
|
||||
stripped = re.sub(r"^#{1,6}\s+", "", content, flags=re.MULTILINE)
|
||||
stripped = re.sub(r"[*_`>\-=]+", "", stripped)
|
||||
stripped = re.sub(r"\s+", " ", stripped).strip()
|
||||
return len(stripped)
|
||||
|
||||
@@ -234,7 +234,7 @@ def resolve_endpoint(
|
||||
ep_id = _stg(f"{setting_prefix}_endpoint_id")
|
||||
model = _stg(f"{setting_prefix}_model")
|
||||
|
||||
# If the specific endpoint is not configured, but the caller provided a
|
||||
# If the specific endpoint is not configured, but the caller provided a
|
||||
# valid fallback (e.g. the active session model), use that immediately.
|
||||
# This prevents background tasks from jumping to the global default_model
|
||||
# when the user is mid-conversation with a different model.
|
||||
@@ -295,7 +295,7 @@ def resolve_endpoint(
|
||||
|
||||
|
||||
def resolve_endpoint_by_id(
|
||||
ep_id: str, model: Optional[str] = None
|
||||
ep_id: str, model: Optional[str] = None, owner: Optional[str] = None
|
||||
) -> Optional[Tuple[str, str, Dict]]:
|
||||
"""Resolve a specific endpoint id (+ optional model) to (chat_url, model, headers).
|
||||
|
||||
@@ -306,10 +306,14 @@ def resolve_endpoint_by_id(
|
||||
return None
|
||||
db = SessionLocal()
|
||||
try:
|
||||
ep = db.query(ModelEndpoint).filter(
|
||||
q = db.query(ModelEndpoint).filter(
|
||||
ModelEndpoint.id == ep_id,
|
||||
ModelEndpoint.is_enabled == True,
|
||||
).first()
|
||||
)
|
||||
if owner:
|
||||
from src.auth_helpers import owner_filter
|
||||
q = owner_filter(q, ModelEndpoint, owner)
|
||||
ep = q.first()
|
||||
if not ep:
|
||||
return None
|
||||
base = normalize_base(ep.base_url)
|
||||
@@ -332,14 +336,14 @@ def resolve_endpoint_by_id(
|
||||
db.close()
|
||||
|
||||
|
||||
def resolve_chat_fallback_candidates() -> list:
|
||||
def resolve_chat_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||
"""Build the configured default-chat fallback chain as a list of
|
||||
(chat_url, model, headers) tuples, skipping any that can't resolve.
|
||||
|
||||
The primary model is NOT included — callers prepend their session's
|
||||
current (url, model, headers) so per-session model overrides are honored.
|
||||
"""
|
||||
return _resolve_fallback_candidates("default_model_fallbacks")
|
||||
return _resolve_fallback_candidates("default_model_fallbacks", owner=owner)
|
||||
|
||||
|
||||
def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||
@@ -355,9 +359,9 @@ def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||
return _resolve_fallback_candidates("utility_model_fallbacks", owner=owner)
|
||||
|
||||
|
||||
def resolve_vision_fallback_candidates() -> list:
|
||||
def resolve_vision_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||
"""Configured fallback chain for the Vision model (`vision_model_fallbacks`)."""
|
||||
return _resolve_fallback_candidates("vision_model_fallbacks")
|
||||
return _resolve_fallback_candidates("vision_model_fallbacks", owner=owner)
|
||||
|
||||
|
||||
def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None) -> list:
|
||||
@@ -371,7 +375,7 @@ def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None)
|
||||
for entry in chain:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
resolved = resolve_endpoint_by_id(entry.get("endpoint_id", ""), entry.get("model", ""))
|
||||
resolved = resolve_endpoint_by_id(entry.get("endpoint_id", ""), entry.get("model", ""), owner=owner)
|
||||
if resolved:
|
||||
out.append(resolved)
|
||||
return out
|
||||
|
||||
+4
-2
@@ -860,7 +860,8 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
|
||||
elif provider == "ollama":
|
||||
response = _parse_ollama_response(data)
|
||||
else:
|
||||
response = data["choices"][0]["message"]["content"]
|
||||
msg = data["choices"][0]["message"]
|
||||
response = msg.get("content") or msg.get("reasoning_content") or ""
|
||||
_set_cached_response(cache_key, response)
|
||||
return response
|
||||
except Exception:
|
||||
@@ -997,7 +998,8 @@ async def llm_call_async(
|
||||
elif provider == "ollama":
|
||||
response = _parse_ollama_response(data)
|
||||
else:
|
||||
response = data["choices"][0]["message"]["content"]
|
||||
msg = data["choices"][0]["message"]
|
||||
response = msg.get("content") or msg.get("reasoning_content") or ""
|
||||
_set_cached_response(cache_key, response)
|
||||
return response
|
||||
except Exception:
|
||||
|
||||
@@ -26,6 +26,8 @@ MARKITDOWN_EXTS = frozenset({".docx", ".pptx", ".xlsx", ".xls", ".epub"})
|
||||
|
||||
def is_markitdown_format(path: str) -> bool:
|
||||
"""True if the file extension is one we route through markitdown."""
|
||||
if not isinstance(path, str):
|
||||
return False
|
||||
return os.path.splitext(path)[1].lower() in MARKITDOWN_EXTS
|
||||
|
||||
|
||||
|
||||
+20
-4
@@ -77,6 +77,11 @@ def split_chunks(text: str, size: int = config.CHUNK_SIZE, overlap: int = config
|
||||
while i < n:
|
||||
j = min(i + size, n)
|
||||
chunks.append(text[i:j])
|
||||
if j >= n:
|
||||
# Reached the end. Without this, the next start (j - overlap) is
|
||||
# still > i, so the loop appended one extra chunk duplicating the
|
||||
# last `overlap` chars of the text.
|
||||
break
|
||||
i = j - overlap if j - overlap > i else j
|
||||
return chunks
|
||||
|
||||
@@ -179,6 +184,11 @@ def retrieve_personal(personal_index: List[Dict], query: str, k: int = 5,
|
||||
# Fall back to keyword search
|
||||
return retrieve_personal_keyword(personal_index, query, k)
|
||||
|
||||
|
||||
def _string_list(values) -> list[str]:
|
||||
return [value for value in values or [] if isinstance(value, str)]
|
||||
|
||||
|
||||
class PersonalDocsManager:
|
||||
"""Manager class for personal document indexing and retrieval."""
|
||||
|
||||
@@ -199,7 +209,10 @@ class PersonalDocsManager:
|
||||
try:
|
||||
if os.path.exists(self.directories_file):
|
||||
with open(self.directories_file, 'r', encoding="utf-8") as f:
|
||||
self.indexed_directories = json.load(f)
|
||||
directories = json.load(f)
|
||||
if not isinstance(directories, list):
|
||||
raise ValueError("indexed directories must be a list")
|
||||
self.indexed_directories = _string_list(directories)
|
||||
logger.info(f"Loaded {len(self.indexed_directories)} indexed directories")
|
||||
else:
|
||||
self.indexed_directories = []
|
||||
@@ -211,7 +224,7 @@ class PersonalDocsManager:
|
||||
"""Save the list of indexed directories to persistent storage."""
|
||||
try:
|
||||
with open(self.directories_file, 'w', encoding="utf-8") as f:
|
||||
json.dump(self.indexed_directories, f, indent=2)
|
||||
json.dump(_string_list(self.indexed_directories), f, indent=2)
|
||||
logger.info(f"Saved {len(self.indexed_directories)} indexed directories")
|
||||
except Exception as e:
|
||||
logger.error(f"Error saving directories: {e}")
|
||||
@@ -221,7 +234,10 @@ class PersonalDocsManager:
|
||||
try:
|
||||
if os.path.exists(self._excluded_file):
|
||||
with open(self._excluded_file, 'r', encoding="utf-8") as f:
|
||||
self.excluded_files = set(json.load(f))
|
||||
excluded = json.load(f)
|
||||
if not isinstance(excluded, list):
|
||||
raise ValueError("excluded files must be a list")
|
||||
self.excluded_files = set(_string_list(excluded))
|
||||
else:
|
||||
self.excluded_files = set()
|
||||
except Exception as e:
|
||||
@@ -231,7 +247,7 @@ class PersonalDocsManager:
|
||||
def _save_excluded(self):
|
||||
try:
|
||||
with open(self._excluded_file, 'w', encoding="utf-8") as f:
|
||||
json.dump(list(self.excluded_files), f)
|
||||
json.dump(_string_list(self.excluded_files), f)
|
||||
except Exception as e:
|
||||
logger.error(f"Error saving excluded files: {e}")
|
||||
|
||||
|
||||
@@ -161,10 +161,10 @@ class ResearchHandler:
|
||||
) -> Optional[dict]:
|
||||
"""Generate a research plan for user review before starting research."""
|
||||
try:
|
||||
from src.deep_research import RESEARCH_PLAN_PROMPT
|
||||
from src.deep_research import RESEARCH_PLAN_PROMPT, current_date_context
|
||||
from src.llm_core import llm_call_async
|
||||
|
||||
prompt = RESEARCH_PLAN_PROMPT.format(question=query)
|
||||
prompt = current_date_context() + RESEARCH_PLAN_PROMPT.format(question=query)
|
||||
response = await llm_call_async(
|
||||
url=llm_endpoint,
|
||||
model=llm_model,
|
||||
|
||||
+16
-3
@@ -96,6 +96,14 @@ DEFAULT_SETTINGS = {
|
||||
"research_run_timeout_seconds": 1800,
|
||||
"agent_max_tool_calls": 0,
|
||||
"agent_input_token_budget": 6000,
|
||||
# Ceiling on the *auto-derived* input budget that #1230 introduced. Has
|
||||
# no effect when `agent_input_token_budget` is explicitly set (the user's
|
||||
# value is honoured regardless). Default matches
|
||||
# `src.context_budget.DEFAULT_HARD_MAX`; lower this for cost-paranoid
|
||||
# setups, raise it on premium APIs with very large windows that you
|
||||
# want to actually use (e.g. 900_000 to fill a 1M-context model). See
|
||||
# `compute_input_token_budget` in src/context_budget.py.
|
||||
"agent_input_token_hard_max": 200_000,
|
||||
"agent_stream_timeout_seconds": 300,
|
||||
# Extra directory roots that read_file / write_file may access, in
|
||||
# addition to the built-in project data/ and system temp dirs. Each
|
||||
@@ -176,8 +184,10 @@ def load_settings() -> dict:
|
||||
try:
|
||||
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
||||
saved = json.load(f)
|
||||
if not isinstance(saved, dict):
|
||||
raise ValueError("settings must be an object")
|
||||
merged = {**DEFAULT_SETTINGS, **saved}
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
except (FileNotFoundError, json.JSONDecodeError, ValueError):
|
||||
merged = dict(DEFAULT_SETTINGS)
|
||||
_settings_cache = (now, merged)
|
||||
return merged
|
||||
@@ -205,7 +215,8 @@ def is_setting_overridden(key: str) -> bool:
|
||||
"""
|
||||
try:
|
||||
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
||||
return key in json.load(f)
|
||||
saved = json.load(f)
|
||||
return isinstance(saved, dict) and key in saved
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
return False
|
||||
|
||||
@@ -256,8 +267,10 @@ def load_features() -> dict:
|
||||
try:
|
||||
with open(FEATURES_FILE, "r", encoding="utf-8") as f:
|
||||
saved = json.load(f)
|
||||
if not isinstance(saved, dict):
|
||||
raise ValueError("features must be an object")
|
||||
merged = {**DEFAULT_FEATURES, **saved}
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
except (FileNotFoundError, json.JSONDecodeError, ValueError):
|
||||
merged = dict(DEFAULT_FEATURES)
|
||||
_features_cache = (now, merged)
|
||||
return merged
|
||||
|
||||
@@ -3,11 +3,11 @@
|
||||
from src.endpoint_resolver import resolve_endpoint
|
||||
|
||||
|
||||
def resolve_task_endpoint(fallback_url=None, fallback_model=None, fallback_headers=None):
|
||||
def resolve_task_endpoint(fallback_url=None, fallback_model=None, fallback_headers=None, owner=None):
|
||||
"""Return (endpoint_url, model, headers) for background tasks.
|
||||
|
||||
Reads task_endpoint_id / task_model from admin settings.
|
||||
Falls back to the provided values when the setting is empty or the
|
||||
endpoint cannot be resolved.
|
||||
"""
|
||||
return resolve_endpoint("task", fallback_url, fallback_model, fallback_headers)
|
||||
return resolve_endpoint("task", fallback_url, fallback_model, fallback_headers, owner=owner)
|
||||
|
||||
@@ -115,7 +115,7 @@ def compute_next_run(schedule: str, scheduled_time: str,
|
||||
return None
|
||||
|
||||
if schedule == "once":
|
||||
if scheduled_date and scheduled_date > (now.replace(tzinfo=None) if tz is not None else now):
|
||||
if scheduled_date and scheduled_date > (_to_utc_naive(now) if tz is not None else now):
|
||||
return scheduled_date
|
||||
return None
|
||||
|
||||
|
||||
@@ -327,7 +327,7 @@ def _extract_skill_json(teacher_response: str) -> Optional[Dict[str, Any]]:
|
||||
treated as "teacher declined to write a skill", per the prompt
|
||||
contract.
|
||||
"""
|
||||
if not teacher_response:
|
||||
if not isinstance(teacher_response, str) or not teacher_response:
|
||||
return None
|
||||
import json
|
||||
m = re.search(r"```(?:json)?\s*\n(\{[\s\S]*?\})\s*\n```", teacher_response)
|
||||
|
||||
@@ -88,6 +88,24 @@ def get_active_document():
|
||||
return _active_document_id
|
||||
|
||||
|
||||
def clear_active_document(doc_id: Optional[str] = None) -> bool:
|
||||
"""Clear the in-memory active-document pointer.
|
||||
|
||||
With ``doc_id`` given, only clears when it matches the current pointer, so a
|
||||
different active document is left untouched. Returns True if it was cleared.
|
||||
|
||||
Called when a document is detached from its session or deleted (its tab is
|
||||
closed): without this, the stale pointer makes the last-resort doc-injection
|
||||
path re-surface a closed document in a later, unrelated chat — even one whose
|
||||
session no longer matches — because an unlinked doc has session_id NULL (#1160).
|
||||
"""
|
||||
global _active_document_id
|
||||
if doc_id is None or _active_document_id == doc_id:
|
||||
_active_document_id = None
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _owned_document_query(query, Document, owner: Optional[str]):
|
||||
if owner is None:
|
||||
# A bare Python `False` is not a valid SQL expression — SQLAlchemy 1.4
|
||||
@@ -892,7 +910,9 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
|
||||
)
|
||||
|
||||
task_id = str(_uuid.uuid4())
|
||||
name = args.get("name") or args.get("prompt", args.get("action_name", "Task"))[:50]
|
||||
# Guard each fallback with `or`: args.get("prompt", default) returns
|
||||
# None when the key is present but null, and None[:50] raises.
|
||||
name = args.get("name") or (args.get("prompt") or args.get("action_name") or "Task")[:50]
|
||||
|
||||
task = ScheduledTask(
|
||||
id=task_id,
|
||||
@@ -1195,7 +1215,17 @@ async def do_manage_mcp(content: str, owner: Optional[str] = None) -> Dict:
|
||||
try:
|
||||
srv = db2.query(McpServer).filter(McpServer.id == sid).first()
|
||||
if srv:
|
||||
await mcp.connect_server(sid)
|
||||
_args = json.loads(srv.args) if srv.args else []
|
||||
_env = json.loads(srv.env) if srv.env else {}
|
||||
await mcp.connect_server(
|
||||
server_id=sid,
|
||||
name=srv.name,
|
||||
transport=srv.transport,
|
||||
command=srv.command,
|
||||
args=_args,
|
||||
env=_env,
|
||||
url=srv.url,
|
||||
)
|
||||
st = mcp.get_server_status(sid)
|
||||
return {"response": f"Reconnected '{srv.name}' ({st.get('tool_count', 0)} tools)", "exit_code": 0}
|
||||
return {"error": f"Server {sid} not found", "exit_code": 1}
|
||||
@@ -1507,7 +1537,14 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
|
||||
"tavily_api_key", "serper_api_key", "app_public_url",
|
||||
}
|
||||
def _is_secret(k):
|
||||
return k in _SECRET_KEYS or any(t in k for t in ("api_key", "_key", "token", "secret", "password"))
|
||||
# `token` must be a suffix, not a substring: otherwise the int
|
||||
# setting `agent_input_token_budget` (which even has a "token budget"
|
||||
# alias to set it from chat) is wrongly classified as a credential.
|
||||
return (
|
||||
k in _SECRET_KEYS
|
||||
or k.endswith("token")
|
||||
or any(t in k for t in ("api_key", "_key", "secret", "password"))
|
||||
)
|
||||
|
||||
# Friendly aliases → real keys, so natural phrasing resolves.
|
||||
_ALIASES_SET = {
|
||||
@@ -1530,7 +1567,10 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
|
||||
"ntfy topic": "reminder_ntfy_topic",
|
||||
"agent tool calls": "agent_max_tool_calls", "max tool calls": "agent_max_tool_calls",
|
||||
"agent timeout": "agent_stream_timeout_seconds", "stream timeout": "agent_stream_timeout_seconds",
|
||||
"token budget": "agent_input_token_budget",
|
||||
"token budget": "agent_input_token_budget", "input budget": "agent_input_token_budget",
|
||||
"hard max": "agent_input_token_hard_max",
|
||||
"token budget cap": "agent_input_token_hard_max",
|
||||
"input budget cap": "agent_input_token_hard_max",
|
||||
}
|
||||
def _resolve(k):
|
||||
k2 = (k or "").strip().lower()
|
||||
@@ -4064,7 +4104,9 @@ async def do_vault_unlock(content: str, owner: Optional[str] = None) -> Dict:
|
||||
if not master_password:
|
||||
return {"error": "master_password is required", "exit_code": 1}
|
||||
|
||||
stdout, stderr, rc = await _run_bw(["unlock", master_password, "--raw"])
|
||||
# Do not pass the master password as an argv element. Local process lists
|
||||
# can expose argv to other users; stdin keeps the secret out of `ps`.
|
||||
stdout, stderr, rc = await _run_bw(["unlock", "--raw"], input_text=master_password + "\n")
|
||||
if rc != 0:
|
||||
return {"error": f"Unlock failed: {stderr[:300]}", "exit_code": 1}
|
||||
|
||||
|
||||
@@ -69,6 +69,8 @@ _TOOL_CODE_RE = re.compile(
|
||||
# fullwidth (U+FF5C) and ascii '|' in any count.
|
||||
_DSML_PIPES = r"[||]+"
|
||||
def _normalize_dsml(text: str) -> str:
|
||||
if not isinstance(text, str):
|
||||
return ""
|
||||
if "DSML" not in text:
|
||||
return text
|
||||
t = text
|
||||
|
||||
+2
-1
@@ -442,7 +442,8 @@ FUNCTION_TOOL_SCHEMAS = [
|
||||
"end": {"type": "string", "description": "list_events range end (ISO datetime); defaults to +14 days"},
|
||||
"event_type": {"type": "string", "description": "Tag / category for the event. Common values: work, personal, health, travel, meal, social, admin, other. Aliases accepted: tag, category, type."},
|
||||
"importance": {"type": "string", "enum": ["low", "normal", "high", "critical"], "description": "Priority level (defaults to 'normal')"},
|
||||
"reminder_minutes": {"type": "integer", "description": "For create_event: create an Odysseus reminder this many minutes before the event, e.g. 5 for 'reminder 5 min before'."}
|
||||
"reminder_minutes": {"type": "integer", "description": "For create_event: create an Odysseus reminder this many minutes before the event, e.g. 5 for 'reminder 5 min before'."},
|
||||
"rrule": {"type": "string", "description": "Recurrence rule in iCalendar RRULE format, e.g. 'FREQ=WEEKLY;BYDAY=MO' for weekly on Monday. Use with create_event or update_event."}
|
||||
},
|
||||
"required": ["action"]
|
||||
}
|
||||
|
||||
+19
-1
@@ -43,6 +43,18 @@ def is_valid_upload_id(upload_id: str) -> bool:
|
||||
return UPLOAD_ID_RE.fullmatch(upload_id or "") is not None
|
||||
|
||||
|
||||
def count_recent_uploads(timestamps, now: float, window: float = 10.0) -> int:
|
||||
"""Number of upload events in *timestamps* within the last *window* seconds.
|
||||
|
||||
Used by the per-IP concurrency guard. The count is of genuine prior upload
|
||||
events — it must NOT scale with how many files are in the *current* request,
|
||||
or a single multi-file batch would reject itself (issue #1346)."""
|
||||
if not timestamps:
|
||||
return 0
|
||||
cutoff = now - window
|
||||
return sum(1 for t in timestamps if t > cutoff)
|
||||
|
||||
|
||||
class UploadHandler:
|
||||
def __init__(self, base_dir: str, upload_dir: str):
|
||||
self.base_dir = base_dir
|
||||
@@ -50,7 +62,13 @@ class UploadHandler:
|
||||
self.max_upload_size = 10 * 1024 * 1024 # 10MB
|
||||
self.max_concurrent_uploads = 3
|
||||
self.cleanup_days = 30
|
||||
self.upload_rate_limit = 5 # Max 5 uploads per minute per IP
|
||||
# Per-IP per-minute cap. save_upload() counts EACH file, and the chat
|
||||
# composer lets a user attach up to MAX_FILES (10, static/js/fileHandler.js)
|
||||
# in one batch — so this must comfortably exceed 10, or a single 6+ file
|
||||
# attach is rejected mid-batch (issue #1346: "5 work, 6 fail"). Burst abuse
|
||||
# is separately bounded by max_concurrent_uploads. Headroom for a few full
|
||||
# batches per minute.
|
||||
self.upload_rate_limit = 60 # max 60 file-uploads per minute per IP
|
||||
self.upload_rate_window = 60 # 60 seconds
|
||||
|
||||
# Track upload rates
|
||||
|
||||
@@ -56,6 +56,8 @@ def check_outbound_url(
|
||||
Returns ``(ok, reason)``. ``ok`` is True only when the URL is safe to fetch.
|
||||
``resolver`` is injectable so callers/tests can avoid real DNS.
|
||||
"""
|
||||
if not isinstance(url, str):
|
||||
return False, "URL must be a string"
|
||||
if not url or not url.strip():
|
||||
return False, "URL is required"
|
||||
try:
|
||||
|
||||
+8
-1
@@ -532,6 +532,13 @@ function initializeEventListeners() {
|
||||
return;
|
||||
}
|
||||
|
||||
// Model picker popup — close before opening any modals
|
||||
const modelPickerMenu = document.getElementById('model-picker-menu');
|
||||
if (modelPickerMenu && modelPickerMenu.classList.contains('open')) {
|
||||
modelPickerMenu.classList.remove('open');
|
||||
return;
|
||||
}
|
||||
|
||||
// Close one modal at a time (last in DOM = topmost)
|
||||
// Map modal id → sidebar list-item id to clear active state
|
||||
const modalItemMap = {
|
||||
@@ -543,7 +550,7 @@ function initializeEventListeners() {
|
||||
};
|
||||
|
||||
// Dynamic modals (removed from DOM on close)
|
||||
const dynamicModals = ['library-modal', 'archive-modal', 'doclib-modal', 'gallery-modal', 'tasks-modal'];
|
||||
const dynamicModals = ['library-modal', 'archive-modal', 'doclib-modal', 'gallery-modal', 'tasks-modal', 'email-lib-modal'];
|
||||
for (const id of dynamicModals) {
|
||||
const m = document.getElementById(id);
|
||||
if (id === 'gallery-modal') {
|
||||
|
||||
+8
-3
@@ -4037,8 +4037,11 @@ import createResearchSynapse from './researchSynapse.js';
|
||||
const clickedIndex = allMsgs.indexOf(msgElement);
|
||||
if (clickedIndex < 0) return;
|
||||
|
||||
// No early-out on a missing session: an output shown before any model was
|
||||
// selected (issue #1428) has no session/persisted rows, but its "x" must
|
||||
// still remove it. We only need the session id for the server-side delete
|
||||
// below; without one we fall back to removing the DOM.
|
||||
const sessionId = sessionModule.getCurrentSessionId();
|
||||
if (!sessionId) return;
|
||||
|
||||
const clickedIsUser = msgElement.classList.contains('msg-user');
|
||||
|
||||
@@ -4114,8 +4117,10 @@ import createResearchSynapse from './researchSynapse.js';
|
||||
}
|
||||
}
|
||||
|
||||
if (!msgIds.length) {
|
||||
// Fallback: just remove DOM elements if no DB IDs available
|
||||
if (!msgIds.length || !sessionId) {
|
||||
// No persisted rows to delete (no DB IDs, or no session at all — e.g. an
|
||||
// error output shown before a model was selected, #1428). Just remove the
|
||||
// DOM so the "x" works regardless.
|
||||
domToRemove.forEach(el => el.remove());
|
||||
if (uiModule) uiModule.showToast('Message deleted');
|
||||
return;
|
||||
|
||||
@@ -1217,6 +1217,17 @@ export function showWelcomeScreen() {
|
||||
const cc = document.getElementById('chat-container');
|
||||
if (ws) ws.classList.remove('hidden');
|
||||
if (cc) cc.classList.add('welcome-active');
|
||||
// Entering the New Chat / welcome state: discard any stale draft left in the
|
||||
// composer from the previous session so the input starts empty (issue #1343).
|
||||
// Switching between existing sessions loads them directly and does NOT call
|
||||
// this, so genuine drafts are not erased. Reset the autosized height and fire
|
||||
// an `input` event so the send button + autosize listeners update.
|
||||
const _msg = document.getElementById('message');
|
||||
if (_msg) {
|
||||
_msg.value = '';
|
||||
_msg.style.height = '';
|
||||
_msg.dispatchEvent(new Event('input', { bubbles: true }));
|
||||
}
|
||||
// Re-trigger the L→R clip-wipe reveal on the welcome name each time the
|
||||
// welcome screen is shown (new session, deleted last session, etc.) — without
|
||||
// this, the CSS animation only fires on initial DOM insertion.
|
||||
|
||||
@@ -401,13 +401,17 @@ export function _buildServeCmd(f, modelName, backend) {
|
||||
const ggufPath = f._gguf_path || 'model.gguf';
|
||||
const gpuId = f.gpu_id?.trim() || '';
|
||||
const py = _isWindows() ? 'python' : 'python3';
|
||||
// CPU-only serve (-ngl 0): drop the GPU-only flags, otherwise the command
|
||||
// mixes "zero GPU layers" with CUDA unified-memory + flash-attn and fails to
|
||||
// start (issue #1291). Only affects the ngl=0 path; GPU serving is unchanged.
|
||||
const _cpuOnly = String(f.ngl).trim() === '0';
|
||||
const lcPrefix = (() => {
|
||||
let p = '';
|
||||
if (f.unified_mem && !_isWindows()) p += `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 `;
|
||||
if (f.unified_mem && !_cpuOnly && !_isWindows()) p += `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 `;
|
||||
if (gpuId && !_isWindows()) p += `CUDA_VISIBLE_DEVICES=${gpuId} `;
|
||||
return p;
|
||||
})();
|
||||
if (f.unified_mem && _isWindows()) cmd += `$env:GGML_CUDA_ENABLE_UNIFIED_MEMORY="1"; `;
|
||||
if (f.unified_mem && !_cpuOnly && _isWindows()) cmd += `$env:GGML_CUDA_ENABLE_UNIFIED_MEMORY="1"; `;
|
||||
if (gpuId && _isWindows()) cmd += `$env:CUDA_VISIBLE_DEVICES="${gpuId}"; `;
|
||||
if (!_isWindows()) {
|
||||
// Resolve GGUF path once, fail loudly if nothing matched (prevents
|
||||
@@ -439,7 +443,7 @@ export function _buildServeCmd(f, modelName, backend) {
|
||||
_lcExtra += ` --n-cpu-moe ${_ncm}`;
|
||||
_lcpExtra += ` --n_cpu_moe ${_ncm}`; // llama-cpp-python uses underscores
|
||||
}
|
||||
if (f.flash_attn) {
|
||||
if (f.flash_attn && !_cpuOnly) {
|
||||
_lcExtra += ' --flash-attn on';
|
||||
_lcpExtra += ' --flash_attn true';
|
||||
}
|
||||
@@ -1524,7 +1528,7 @@ function _renderRecipes() {
|
||||
html += '<option value="Q4_K_M">Q4</option><option value="Q8_0">Q8</option>';
|
||||
html += '<option value="Q6_K">Q6</option><option value="Q5_K_M">Q5</option>';
|
||||
html += '<option value="Q3_K_M">Q3</option><option value="Q2_K">Q2</option>';
|
||||
html += '<option value="AWQ-4bit">AWQ</option><option value="FP8">FP8</option><option value="FP4">FP4</option>';
|
||||
html += '<option value="AWQ-4bit">AWQ</option><option value="FP8">FP8</option><option value="FP4">FP4</option><option value="NVFP4">NVFP4</option>';
|
||||
html += '<option value="">Native</option></select>';
|
||||
// Engine filter: show only models whose serve engine matches. "llama.cpp"
|
||||
// (GGUF) runs everywhere incl. consumer AMD/Apple; vLLM/SGLang are CUDA /
|
||||
|
||||
@@ -555,18 +555,20 @@ export async function _runModelDownload(panel, model, backend, hostOverride) {
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
if (!res.ok) {
|
||||
uiModule.showToast('Download failed: HTTP ' + res.status);
|
||||
// Errors carry actionable text (e.g. "tmux is required …"); keep them up
|
||||
// long enough to read, matching the serve path's duration (issue #1355).
|
||||
uiModule.showToast('Download failed: HTTP ' + res.status, 9000);
|
||||
return;
|
||||
}
|
||||
const data = await res.json();
|
||||
if (!data.ok) {
|
||||
uiModule.showToast('Download failed: ' + (data.error || ''));
|
||||
uiModule.showToast('Download failed: ' + (data.error || ''), 9000);
|
||||
return;
|
||||
}
|
||||
_addTask(data.session_id, shortName, 'download', payload);
|
||||
uiModule.showToast(`Downloading ${shortName}...`);
|
||||
} catch (e) {
|
||||
uiModule.showToast('Download failed: ' + e.message);
|
||||
uiModule.showToast('Download failed: ' + e.message, 9000);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -246,10 +246,20 @@ function _selectedGgufExpr(model, repo, relPath) {
|
||||
const base = String(model.path || '').replace(/\/+$/, '');
|
||||
return `$(printf %s ${_shellPathExpr(`${base}/${repo}/${rel}`)})`;
|
||||
}
|
||||
if (model.path) {
|
||||
const base = String(model.path || '').replace(/\/+$/, '');
|
||||
return `$(printf %s ${_shellPathExpr(`${base}/models--${repo.replace(/\//g, '--')}/snapshots/${rel}`)})`;
|
||||
}
|
||||
const cacheRepo = repo.replace(/\//g, '--');
|
||||
return `$(printf %s \${HOME}${_shellQuote(`/.cache/huggingface/hub/models--${cacheRepo}/snapshots/${rel}`)})`;
|
||||
}
|
||||
|
||||
function _ggufSearchDirExpr(model, repo) {
|
||||
if (model.is_local_dir && model.path) return _shellQuote(`${String(model.path || '').replace(/\/+$/, '')}/${repo}`);
|
||||
if (model.path) return _shellQuote(`${String(model.path || '').replace(/\/+$/, '')}/models--${repo.replace(/\//g, '--')}/snapshots`);
|
||||
return `"$HOME/.cache/huggingface/hub/models--${repo.replace(/\//g, '--')}/snapshots"`;
|
||||
}
|
||||
|
||||
function _rerenderCachedModels() {
|
||||
const list = document.getElementById('hwfit-cached-list');
|
||||
const tagContainer = document.getElementById('serve-tags');
|
||||
@@ -736,13 +746,12 @@ function _rerenderCachedModels() {
|
||||
// For multi-part GGUFs, llama.cpp requires the first split
|
||||
// (-00001-of-NNNNN.gguf). Prefer it (sorted, so UD-IQ4_XS/001 comes
|
||||
// before Q4_K_M/001 etc); fall back to any single GGUF sorted.
|
||||
// Use $HOME (not ~) so tilde survives variable interpolation inside $(...).
|
||||
const dir = `"$HOME/.cache/huggingface/hub/models--${repo.replace(/\//g, '--')}/snapshots"`;
|
||||
const dir = _ggufSearchDirExpr(m, repo);
|
||||
// GGUF needs the actual .gguf FILE, not the folder. For a custom-dir
|
||||
// model the file lives under "<path>/<repo>" — search there just like we
|
||||
// search the HF snapshots dir, so serving a GGUF from a custom dir works
|
||||
// instead of handing llama.cpp a directory (which fails).
|
||||
const _ldir = `"${m.path}/${repo}"`;
|
||||
const _ldir = m.path ? _shellQuote(`${m.path}/${repo}`) : '""';
|
||||
f._gguf_path = selectedGguf
|
||||
? _selectedGgufExpr(m, repo, selectedGguf.rel_path)
|
||||
: m.is_local_dir && m.path
|
||||
|
||||
@@ -1721,7 +1721,7 @@ async function _loadEmails({ force = false, useCache = true } = {}) {
|
||||
async function _loadScheduled(grid, sp) {
|
||||
const res = await fetch(`${API_BASE}/api/email/scheduled`);
|
||||
const data = await res.json();
|
||||
sp.destroy();
|
||||
if (sp) sp.destroy();
|
||||
const items = data.scheduled || [];
|
||||
grid.innerHTML = '';
|
||||
const stats = document.getElementById('email-lib-stats');
|
||||
|
||||
@@ -112,7 +112,7 @@ function _createChip(f, idx) {
|
||||
chip.classList.add('thumb-image'); // lets CSS overlay the remove-X on the corner (mobile)
|
||||
const img = document.createElement('img');
|
||||
img.className = 'thumb-img';
|
||||
img.src = URL.createObjectURL(f);
|
||||
img.src = _getPreviewUrl(f);
|
||||
img.alt = f.name || 'image';
|
||||
chip.appendChild(img);
|
||||
} else {
|
||||
@@ -172,6 +172,17 @@ export async function uploadPending() {
|
||||
method: 'POST',
|
||||
body: fd
|
||||
});
|
||||
if (!res.ok) {
|
||||
// Surface the failure instead of swallowing it. Previously a non-OK
|
||||
// response (e.g. 429 rate limit, 413 too large) was ignored: the files
|
||||
// silently vanished and the chat sent with no attachments, so the model
|
||||
// "didn't even see them" (issue #1346). Show the server's reason and keep
|
||||
// pendingFiles so the strip re-renders for a retry (see finally below).
|
||||
let detail = '';
|
||||
try { const e = await res.json(); detail = e.detail || e.error || ''; } catch (_) {}
|
||||
_showToast('Upload failed' + (detail ? ': ' + detail : ` (HTTP ${res.status})`));
|
||||
return [];
|
||||
}
|
||||
const data = await res.json();
|
||||
uploaded = (data.files || []);
|
||||
pendingFiles = []; // clear only on success
|
||||
|
||||
+5
-3
@@ -8,6 +8,7 @@ import spinnerModule from './spinner.js';
|
||||
import { providerLogo } from './providers.js';
|
||||
import { PROMPT_TEMPLATES, getAllPresets } from './presets.js';
|
||||
import { sortModelObjects } from './modelSort.js';
|
||||
import Storage from './storage.js';
|
||||
|
||||
let API_BASE = '';
|
||||
let _active = false;
|
||||
@@ -57,7 +58,7 @@ function _initGroupTab() {
|
||||
});
|
||||
});
|
||||
_modelsCache = sortModelObjects(result);
|
||||
return result;
|
||||
return _modelsCache;
|
||||
}
|
||||
|
||||
function _render() {
|
||||
@@ -409,7 +410,7 @@ export async function showModelPicker() {
|
||||
});
|
||||
});
|
||||
_cachedModels = sortModelObjects(result);
|
||||
return result;
|
||||
return _cachedModels;
|
||||
}
|
||||
|
||||
async function render(filter) {
|
||||
@@ -546,7 +547,8 @@ export async function startGroup(models, parentSessionId) {
|
||||
_parentSessionId = pdata.id;
|
||||
// Register as group session for sidebar icon
|
||||
try {
|
||||
const gids = JSON.parse(localStorage.getItem('odysseus-group-sessions') || '[]');
|
||||
const storedGroupSessions = Storage.getJSON('odysseus-group-sessions', []);
|
||||
const gids = Array.isArray(storedGroupSessions) ? storedGroupSessions : [];
|
||||
if (!gids.includes(_parentSessionId)) { gids.push(_parentSessionId); localStorage.setItem('odysseus-group-sessions', JSON.stringify(gids)); }
|
||||
} catch (e) {}
|
||||
} catch (e) {
|
||||
|
||||
@@ -536,16 +536,18 @@ export function mdToHtml(src) {
|
||||
let html = '<table style="border-collapse: collapse; width: 100%; margin: 10px 0;">';
|
||||
|
||||
rows.forEach((row, idx) => {
|
||||
if (idx === 1 && /^[\s|:\-]+$/.test(row)) {
|
||||
html += '<tbody>';
|
||||
return;
|
||||
}
|
||||
const cells = splitTableRow(row);
|
||||
if (cells.length === 0) return;
|
||||
|
||||
html += idx === 1 ? '<tbody>' : '';
|
||||
html += '<tr>';
|
||||
|
||||
cells.forEach(cell => {
|
||||
const tag = idx === 0 ? 'th' : 'td';
|
||||
const style = idx === 1 ? 'style="border-top: 2px solid var(--red);"' : '';
|
||||
html += `<${tag} ${style} style="padding: 8px; text-align: left; border-bottom: 1px solid var(--border);">${cell.trim()}</${tag}>`;
|
||||
html += `<${tag} style="padding: 8px; text-align: left; border-bottom: 1px solid var(--border);">${cell.trim()}</${tag}>`;
|
||||
});
|
||||
|
||||
html += '</tr>';
|
||||
|
||||
+23
-5
@@ -8,6 +8,24 @@ let API_BASE = '';
|
||||
let selectedPreset = null;
|
||||
let presets = {};
|
||||
|
||||
export function loadStoredArray(key) {
|
||||
try {
|
||||
const value = JSON.parse(localStorage.getItem(key) || '[]');
|
||||
return Array.isArray(value) ? value : [];
|
||||
} catch (e) {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
export function loadStoredObject(key) {
|
||||
try {
|
||||
const value = JSON.parse(localStorage.getItem(key) || '{}');
|
||||
return value && typeof value === 'object' && !Array.isArray(value) ? value : {};
|
||||
} catch (e) {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
// Built-in prompt templates (moved from cot_prompts.py)
|
||||
export const PROMPT_TEMPLATES = [
|
||||
{
|
||||
@@ -228,7 +246,7 @@ function initNameDropdown() {
|
||||
}
|
||||
// Hide built-in preset
|
||||
if (isBuiltin) {
|
||||
const hidden = JSON.parse(localStorage.getItem('odysseus-hidden-presets') || '[]');
|
||||
const hidden = loadStoredArray('odysseus-hidden-presets');
|
||||
if (!hidden.includes(charName)) hidden.push(charName);
|
||||
localStorage.setItem('odysseus-hidden-presets', JSON.stringify(hidden));
|
||||
}
|
||||
@@ -311,7 +329,7 @@ function _populateCharSelect() {
|
||||
select.appendChild(group);
|
||||
}
|
||||
|
||||
const hiddenPresets = JSON.parse(localStorage.getItem('odysseus-hidden-presets') || '[]');
|
||||
const hiddenPresets = loadStoredArray('odysseus-hidden-presets');
|
||||
const builtins = PROMPT_TEMPLATES.filter(t => !savedNames.has(t.name) && !hiddenPresets.includes(t.name));
|
||||
if (builtins.length) {
|
||||
const group = document.createElement('optgroup');
|
||||
@@ -405,7 +423,7 @@ function initPersistentChat() {
|
||||
await fetch(`${API_BASE}/api/session/${sessionId}/important`, { method: 'POST', body: favFd });
|
||||
|
||||
// Save session → character mapping so it restores on switch
|
||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
||||
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||
charSessions[sessionId] = charName;
|
||||
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
||||
|
||||
@@ -1011,7 +1029,7 @@ function _syncCharIndicator() {
|
||||
let _prevSessionId = null;
|
||||
|
||||
export function onSessionSwitch(sessionId) {
|
||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
||||
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||
|
||||
// Leaving a persistent chat — deactivate for this switch only
|
||||
if (window._persistentChatSession) {
|
||||
@@ -1059,7 +1077,7 @@ export function isPersistentChat() {
|
||||
* Remove a session from persistent chat mappings (call when session is deleted).
|
||||
*/
|
||||
export function removePersistentChat(sessionId) {
|
||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
||||
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||
if (charSessions[sessionId]) {
|
||||
delete charSessions[sessionId];
|
||||
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
||||
|
||||
+2
-1
@@ -7,6 +7,7 @@ import markdownModule from './markdown.js';
|
||||
import * as spinnerModule from './spinner.js';
|
||||
import { makeWindowDraggable } from './windowDrag.js';
|
||||
import { sortModelIds } from './modelSort.js';
|
||||
import { ordinalSuffix } from './util/ordinal.js';
|
||||
|
||||
const API_BASE = window.location.origin;
|
||||
let _open = false;
|
||||
@@ -244,7 +245,7 @@ function _scheduleLabel(task) {
|
||||
}
|
||||
if (task.schedule === 'monthly') {
|
||||
const d = task.scheduled_day ?? 1;
|
||||
const suffix = d === 1 ? 'st' : d === 2 ? 'nd' : d === 3 ? 'rd' : 'th';
|
||||
const suffix = ordinalSuffix(d);
|
||||
return `Monthly on ${d}${suffix} at ${localTime}`;
|
||||
}
|
||||
return task.schedule || '—';
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
// Pure (browser-free) English ordinal suffix, e.g. 1 -> "st", 21 -> "st",
|
||||
// 22 -> "nd", 23 -> "rd", 11/12/13 -> "th". Extracted so it can be unit-tested.
|
||||
export function ordinalSuffix(n) {
|
||||
const a = Math.abs(Math.trunc(Number(n) || 0));
|
||||
const mod100 = a % 100;
|
||||
if (mod100 >= 11 && mod100 <= 13) return 'th';
|
||||
switch (a % 10) {
|
||||
case 1: return 'st';
|
||||
case 2: return 'nd';
|
||||
case 3: return 'rd';
|
||||
default: return 'th';
|
||||
}
|
||||
}
|
||||
+63
-1
@@ -1141,7 +1141,10 @@ body.bg-pattern-sparkles {
|
||||
display: flex; align-items: center; gap: 6px;
|
||||
margin: 0; padding: 0; border: none;
|
||||
font-size: 10px; font-weight: 400; font-family: inherit;
|
||||
line-height: 1; letter-spacing: 0; text-transform: none;
|
||||
/* 1.3 (not 1) so Fira Code's tall glyph box isn't vertically clipped in
|
||||
Chromium/Edge — mirrors the .list-item fix. The title is flex-centred
|
||||
in a fixed-height (29px) header, so this adds headroom without reflow. */
|
||||
line-height: 1.3; letter-spacing: 0; text-transform: none;
|
||||
color: var(--fg);
|
||||
}
|
||||
.section-icon,
|
||||
@@ -1801,6 +1804,15 @@ body.bg-pattern-sparkles {
|
||||
max-height: 60px;
|
||||
overflow: hidden;
|
||||
}
|
||||
/* The tip is a full sentence that wraps to 4-5 lines on narrow phones,
|
||||
where the welcome block shrink-wraps small; the shared 60px ceiling
|
||||
(sized for the one-line sub/version) clipped its last line. Give the tip
|
||||
a taller ceiling so it isn't truncated. Kept above the max-height:650px
|
||||
block below so that rule's max-height:0 still collapses it on short
|
||||
viewports. */
|
||||
#welcome-screen .welcome-tip {
|
||||
max-height: 120px;
|
||||
}
|
||||
@media (max-height: 650px) {
|
||||
#welcome-screen { top: 28%; }
|
||||
#welcome-screen .welcome-tip { opacity: 0; max-height: 0; margin: 0; }
|
||||
@@ -35357,3 +35369,53 @@ body.theme-frosted .modal {
|
||||
font-family: 'Fira Code', ui-monospace, monospace;
|
||||
color: var(--accent, var(--red));
|
||||
}
|
||||
|
||||
/* ══ iOS focus-zoom fix — touch devices only; desktop sizes untouched ══
|
||||
16px is the threshold below which iOS Safari auto-zooms on focus.
|
||||
Selects and date/time inputs are excluded on purpose — they open native
|
||||
pickers and never zoom. */
|
||||
@media (hover: none) and (pointer: coarse) {
|
||||
|
||||
/* 1 ── Catch-all: every text-entry control NOT pinned with its own
|
||||
!important. !important here beats any non-important rule regardless of
|
||||
specificity, so this clears the long tail (settings, admin, memory,
|
||||
notes, calendar, email, gallery, tasks, model picker, etc.). */
|
||||
input[type="text"],
|
||||
input[type="search"],
|
||||
input[type="email"],
|
||||
input[type="url"],
|
||||
input[type="tel"],
|
||||
input[type="password"],
|
||||
input[type="number"],
|
||||
input:not([type]),
|
||||
textarea {
|
||||
font-size: 16px !important;
|
||||
}
|
||||
|
||||
/* 2 ── Fields that pin their own !important at specificity our catch-all
|
||||
can't beat. Each is matched at equal-or-higher specificity and, being
|
||||
later in the file, wins the tie. */
|
||||
#message { font-size: 16px !important; } /* chat composer (was 13px !important) */
|
||||
.cookbook-dl-repo,
|
||||
.hwfit-search { font-size: 16px !important; } /* cookbook repo path + hardware search */
|
||||
.ge-topbar input { font-size: 16px !important; } /* image-editor topbar input */
|
||||
.ge-transform-field > input.ge-transform-popup-input { /* image-editor transform values */
|
||||
font-size: 16px !important;
|
||||
}
|
||||
}
|
||||
|
||||
@media (hover: none) and (pointer: coarse) {
|
||||
/* Only the sub-16px tiers need bumping; large lands ABOVE 16 so it
|
||||
stays zoom-safe AND visibly larger than medium (otherwise L collapses
|
||||
onto M on touch). All three editor layers move together so the
|
||||
highlight/line-number overlay stays metrically aligned with the textarea. */
|
||||
.doc-font-m .doc-editor-textarea, .doc-font-m .doc-editor-highlight, .doc-font-m .doc-line-numbers {
|
||||
font-size: 16px !important; /* was 13px */
|
||||
}
|
||||
.doc-font-l .doc-editor-textarea, .doc-font-l .doc-editor-highlight, .doc-font-l .doc-line-numbers {
|
||||
font-size: 18px !important; /* was 15px — keep L > M */
|
||||
}
|
||||
/* Email compose rich-body. Medium (15px) zooms, so bump it; large (17px)
|
||||
is already ≥16px and never zoomed — leave it so we don't shrink it. */
|
||||
.doc-email-richbody.doc-font-m { font-size: 16px !important; }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Issue #1160 — a closed document must not stay 'active' and leak into new chats.
|
||||
|
||||
Closing a document tab detaches it (session_id -> NULL) or deletes it, but the
|
||||
in-memory active-document pointer was never cleared, so the last-resort doc
|
||||
injection re-surfaced the closed doc in later, unrelated chats. The document
|
||||
routes now call clear_active_document() on detach/delete; this pins that helper.
|
||||
"""
|
||||
|
||||
from src.tool_implementations import (
|
||||
set_active_document,
|
||||
get_active_document,
|
||||
clear_active_document,
|
||||
)
|
||||
|
||||
|
||||
def test_clear_matching_id_resets_pointer():
|
||||
set_active_document("doc-123")
|
||||
assert get_active_document() == "doc-123"
|
||||
assert clear_active_document("doc-123") is True
|
||||
assert get_active_document() is None
|
||||
|
||||
|
||||
def test_clear_non_matching_id_leaves_other_active_doc():
|
||||
set_active_document("doc-abc")
|
||||
# Closing a DIFFERENT document must not clobber the currently active one.
|
||||
assert clear_active_document("doc-xyz") is False
|
||||
assert get_active_document() == "doc-abc"
|
||||
|
||||
|
||||
def test_clear_without_id_clears_unconditionally():
|
||||
set_active_document("doc-abc")
|
||||
assert clear_active_document() is True
|
||||
assert get_active_document() is None
|
||||
|
||||
|
||||
def test_clear_when_already_none_is_safe():
|
||||
set_active_document(None)
|
||||
assert clear_active_document("doc-123") is False
|
||||
assert get_active_document() is None
|
||||
@@ -0,0 +1,21 @@
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "check-docker-amd-gpu.sh"
|
||||
|
||||
|
||||
def test_amd_gpu_check_rejects_unknown_extra_arg_before_diagnostics():
|
||||
proc = subprocess.run(
|
||||
["bash", str(SCRIPT), "--bad-option"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert proc.returncode == 1
|
||||
assert "Unknown option: --bad-option" in proc.stderr
|
||||
|
||||
|
||||
def test_amd_gpu_check_shell_syntax():
|
||||
subprocess.run(["bash", "-n", str(SCRIPT)], check=True)
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Regression: APIKeyManager.load() must not crash on a corrupt/wrong-shape file.
|
||||
|
||||
load() is called during startup (app_initializer). It had no try/except around
|
||||
`json.load` and called `encrypted_keys.items()` directly, so a corrupt/truncated
|
||||
api_keys.json raised JSONDecodeError and a legacy list-shaped file raised
|
||||
AttributeError — both crashing app startup. It now returns {} instead.
|
||||
"""
|
||||
from src.api_key_manager import APIKeyManager
|
||||
|
||||
|
||||
def _mgr(tmp_path):
|
||||
return APIKeyManager(str(tmp_path))
|
||||
|
||||
|
||||
def test_corrupt_json_returns_empty(tmp_path):
|
||||
(tmp_path / "api_keys.json").write_text("{not valid json", encoding="utf-8")
|
||||
assert _mgr(tmp_path).load() == {}
|
||||
|
||||
|
||||
def test_list_shape_returns_empty(tmp_path):
|
||||
(tmp_path / "api_keys.json").write_text('["openai", "anthropic"]', encoding="utf-8")
|
||||
assert _mgr(tmp_path).load() == {}
|
||||
|
||||
|
||||
def test_missing_file_returns_empty(tmp_path):
|
||||
assert _mgr(tmp_path).load() == {}
|
||||
|
||||
|
||||
def test_valid_roundtrip(tmp_path):
|
||||
mgr = _mgr(tmp_path)
|
||||
mgr.save("openai", "sk-secret")
|
||||
assert mgr.load() == {"openai": "sk-secret"}
|
||||
@@ -30,6 +30,17 @@ def _verify_args(path: Path):
|
||||
return SimpleNamespace(path=str(path), pretty=False)
|
||||
|
||||
|
||||
def test_snapshot_rejects_output_inside_data_dir(tmp_path, monkeypatch):
|
||||
backup = _load_backup_cli()
|
||||
repo = tmp_path / "repo"
|
||||
data = repo / "data"
|
||||
data.mkdir(parents=True)
|
||||
_patch_repo(backup, monkeypatch, repo)
|
||||
|
||||
with pytest.raises(SystemExit):
|
||||
backup._reject_output_inside_data(data / "self.tar.gz")
|
||||
|
||||
|
||||
def test_restore_rejects_symlink_escape(tmp_path, monkeypatch):
|
||||
backup = _load_backup_cli()
|
||||
repo = tmp_path / "repo"
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Regression: builtin_actions heuristics must tolerate non-string input.
|
||||
|
||||
_result_has_work did `result.lower()` after a falsy-only guard, and
|
||||
_classify_event_heuristic did `(summary or "").lower()`; a truthy non-string
|
||||
(e.g. a dict) raised AttributeError. They now coerce/guard non-strings.
|
||||
"""
|
||||
from src.builtin_actions import _result_has_work, _classify_event_heuristic
|
||||
|
||||
|
||||
def test_result_has_work_non_string():
|
||||
assert _result_has_work({"x": 1}) is False
|
||||
assert _result_has_work(123) is False
|
||||
|
||||
|
||||
def test_classify_event_heuristic_non_string():
|
||||
out = _classify_event_heuristic(123)
|
||||
assert isinstance(out, tuple)
|
||||
|
||||
|
||||
def test_valid_inputs_unchanged():
|
||||
assert _result_has_work("Processed 0 emails") is False
|
||||
@@ -0,0 +1,97 @@
|
||||
import asyncio
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from src import caldav_sync
|
||||
|
||||
|
||||
def test_validate_caldav_url_normalizes_safe_url():
|
||||
assert (
|
||||
caldav_sync.validate_caldav_url(" https://calendar.example.com/dav/ ")
|
||||
== "https://calendar.example.com/dav"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"url, message",
|
||||
[
|
||||
("ftp://calendar.example.com/dav", "must start with"),
|
||||
("https://alice:secret@calendar.example.com/dav", "credentials"),
|
||||
("https://calendar.example.com/dav#frag", "fragments"),
|
||||
("http://localhost:5232/dav", "host is not allowed"),
|
||||
("http://service.localhost/dav", "host is not allowed"),
|
||||
("http://127.0.0.1:5232/dav", "host is not allowed"),
|
||||
("http://[::1]:5232/dav", "host is not allowed"),
|
||||
("http://169.254.169.254/latest", "host is not allowed"),
|
||||
],
|
||||
)
|
||||
def test_validate_caldav_url_rejects_unsafe_urls(url, message):
|
||||
with pytest.raises(ValueError, match=message):
|
||||
caldav_sync.validate_caldav_url(url)
|
||||
|
||||
|
||||
def test_validate_caldav_url_blocks_private_ips_unless_explicitly_allowed(monkeypatch):
|
||||
monkeypatch.delenv("ODYSSEUS_ALLOW_PRIVATE_CALDAV", raising=False)
|
||||
with pytest.raises(ValueError, match="Private CalDAV IPs require"):
|
||||
caldav_sync.validate_caldav_url("http://10.0.0.5:5232/dav")
|
||||
|
||||
monkeypatch.setenv("ODYSSEUS_ALLOW_PRIVATE_CALDAV", "1")
|
||||
assert caldav_sync.validate_caldav_url("http://10.0.0.5:5232/dav") == "http://10.0.0.5:5232/dav"
|
||||
|
||||
|
||||
def test_sync_caldav_decrypts_stored_password_and_validates_url(monkeypatch):
|
||||
prefs_mod = types.ModuleType("routes.prefs_routes")
|
||||
prefs_mod._load_for_user = lambda owner: {
|
||||
"caldav": {
|
||||
"url": " https://calendar.example.com/dav/ ",
|
||||
"username": owner,
|
||||
"password": "enc:stored",
|
||||
}
|
||||
}
|
||||
monkeypatch.setitem(sys.modules, "routes.prefs_routes", prefs_mod)
|
||||
|
||||
secret_mod = types.ModuleType("src.secret_storage")
|
||||
secret_mod.decrypt = lambda value: "decrypted-password" if value == "enc:stored" else value
|
||||
monkeypatch.setitem(sys.modules, "src.secret_storage", secret_mod)
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_sync_blocking(owner, url, username, password):
|
||||
captured.update(
|
||||
{
|
||||
"owner": owner,
|
||||
"url": url,
|
||||
"username": username,
|
||||
"password": password,
|
||||
}
|
||||
)
|
||||
return {"calendars": 1, "events": 0, "deleted": 0, "errors": []}
|
||||
|
||||
async def inline_to_thread(func, *args, **kwargs):
|
||||
return func(*args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(caldav_sync, "_sync_blocking", fake_sync_blocking)
|
||||
monkeypatch.setattr(caldav_sync.asyncio, "to_thread", inline_to_thread)
|
||||
|
||||
result = asyncio.run(caldav_sync.sync_caldav("alice"))
|
||||
|
||||
assert result["calendars"] == 1
|
||||
assert captured == {
|
||||
"owner": "alice",
|
||||
"url": "https://calendar.example.com/dav",
|
||||
"username": "alice",
|
||||
"password": "decrypted-password",
|
||||
}
|
||||
|
||||
|
||||
def test_calendar_routes_use_hardened_caldav_client_and_secret_storage():
|
||||
text = Path("routes/calendar_routes.py").read_text(encoding="utf-8")
|
||||
|
||||
assert "validate_caldav_url(body.get(\"url\", \"\"))" in text
|
||||
assert "cfg[\"password\"] = encrypt(body[\"password\"])" in text
|
||||
assert "pw = decrypt(pw)" in text
|
||||
assert "follow_redirects=False, trust_env=False" in text
|
||||
assert "Redirects are not followed for CalDAV safety" in text
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Regression: validate_caldav_url must reject a non-string via its normal
|
||||
ValueError path, not crash with TypeError.
|
||||
|
||||
It did `(raw_url or "").strip()`, so a non-string scalar (e.g. an int from a
|
||||
mis-typed config) reached `.strip()` and raised TypeError instead of the
|
||||
function\'s own ValueError.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from src.caldav_sync import validate_caldav_url
|
||||
|
||||
|
||||
def test_non_string_raises_valueerror_not_typeerror():
|
||||
with pytest.raises(ValueError):
|
||||
validate_caldav_url(12345)
|
||||
with pytest.raises(ValueError):
|
||||
validate_caldav_url(None)
|
||||
|
||||
|
||||
def test_valid_url_passes():
|
||||
out = validate_caldav_url("https://dav.example.com/calendars/")
|
||||
assert "example.com" in out
|
||||
@@ -0,0 +1,125 @@
|
||||
"""Issue #800 — CalDAV write-back pushes local changes to the remote server.
|
||||
|
||||
Unit-tests the pure pieces against a fake caldav calendar (no network): the
|
||||
iCalendar serialization, hash-based remote-calendar discovery, and the
|
||||
create/update/delete orchestration.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from src.caldav_writeback import (
|
||||
build_event_ical,
|
||||
find_remote_calendar,
|
||||
push_event,
|
||||
_stable_cal_id,
|
||||
)
|
||||
|
||||
REMOTE_URL = "https://p69-caldav.icloud.com/123/calendars/home/"
|
||||
CAL_ID = _stable_cal_id(REMOTE_URL)
|
||||
|
||||
|
||||
class FakeEvent:
|
||||
def __init__(self):
|
||||
self.data = "OLD"
|
||||
self.saved = False
|
||||
self.deleted = False
|
||||
|
||||
def save(self):
|
||||
self.saved = True
|
||||
|
||||
def delete(self):
|
||||
self.deleted = True
|
||||
|
||||
|
||||
class FakeCalendar:
|
||||
def __init__(self, url, existing=None):
|
||||
self.url = url
|
||||
self._existing = existing
|
||||
self.saved_ical = None
|
||||
|
||||
def event_by_uid(self, uid):
|
||||
if self._existing is None:
|
||||
raise Exception("not found")
|
||||
return self._existing
|
||||
|
||||
def save_event(self, ical):
|
||||
self.saved_ical = ical
|
||||
|
||||
|
||||
def _ev(**over):
|
||||
base = dict(
|
||||
uid="evt-1", summary="Dentist", description="bring x-rays",
|
||||
location="Clinic", dtstart=datetime(2026, 6, 10, 14, 0),
|
||||
dtend=datetime(2026, 6, 10, 15, 0), all_day=False, is_utc=True, rrule="",
|
||||
)
|
||||
base.update(over)
|
||||
return base
|
||||
|
||||
|
||||
def test_build_ical_timed_event_has_core_fields():
|
||||
ical = build_event_ical(_ev())
|
||||
assert "BEGIN:VEVENT" in ical and "END:VEVENT" in ical
|
||||
assert "UID:evt-1" in ical
|
||||
assert "SUMMARY:Dentist" in ical
|
||||
# is_utc -> UTC instant (Z suffix)
|
||||
assert "DTSTART:20260610T140000Z" in ical
|
||||
assert "DTEND:20260610T150000Z" in ical
|
||||
|
||||
|
||||
def test_build_ical_all_day_uses_date_values():
|
||||
ical = build_event_ical(_ev(all_day=True, is_utc=False))
|
||||
assert "DTSTART;VALUE=DATE:20260610" in ical
|
||||
|
||||
|
||||
def test_build_ical_includes_rrule():
|
||||
ical = build_event_ical(_ev(rrule="FREQ=WEEKLY;BYDAY=MO"))
|
||||
assert "RRULE:FREQ=WEEKLY" in ical
|
||||
|
||||
|
||||
def test_find_remote_calendar_matches_by_hash():
|
||||
cals = [FakeCalendar("https://other/x/"), FakeCalendar(REMOTE_URL)]
|
||||
found = find_remote_calendar(cals, CAL_ID)
|
||||
assert found is cals[1]
|
||||
assert find_remote_calendar([FakeCalendar("https://nope/")], CAL_ID) is None
|
||||
|
||||
|
||||
def test_push_create_calls_save_event():
|
||||
cal = FakeCalendar(REMOTE_URL, existing=None) # event_by_uid raises -> create
|
||||
res = push_event([cal], CAL_ID, _ev(), delete=False)
|
||||
assert res["ok"] and res.get("created")
|
||||
assert cal.saved_ical and "UID:evt-1" in cal.saved_ical
|
||||
|
||||
|
||||
def test_push_update_overwrites_existing():
|
||||
existing = FakeEvent()
|
||||
cal = FakeCalendar(REMOTE_URL, existing=existing)
|
||||
res = push_event([cal], CAL_ID, _ev(summary="Moved"), delete=False)
|
||||
assert res["ok"] and res.get("updated")
|
||||
assert existing.saved and "SUMMARY:Moved" in existing.data
|
||||
assert cal.saved_ical is None # used update path, not create
|
||||
|
||||
|
||||
def test_push_delete_removes_existing():
|
||||
existing = FakeEvent()
|
||||
cal = FakeCalendar(REMOTE_URL, existing=existing)
|
||||
res = push_event([cal], CAL_ID, _ev(), delete=True)
|
||||
assert res["ok"] and existing.deleted
|
||||
|
||||
|
||||
def test_push_delete_absent_is_ok():
|
||||
cal = FakeCalendar(REMOTE_URL, existing=None)
|
||||
res = push_event([cal], CAL_ID, _ev(), delete=True)
|
||||
assert res["ok"] and "absent" in res.get("note", "")
|
||||
|
||||
|
||||
def test_push_unknown_calendar_reports_not_found():
|
||||
cal = FakeCalendar("https://different/")
|
||||
res = push_event([cal], CAL_ID, _ev())
|
||||
assert res["ok"] is False and "not found" in res["error"]
|
||||
|
||||
|
||||
def test_push_missing_uid_reports_input_error_before_remote_lookup():
|
||||
cal = FakeCalendar(REMOTE_URL, existing=FakeEvent())
|
||||
res = push_event([cal], CAL_ID, _ev(uid=""))
|
||||
assert res["ok"] is False and "uid" in res["error"]
|
||||
assert cal._existing.saved is False
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Issue #800 — the calendar write handlers actually trigger CalDAV write-back.
|
||||
|
||||
Route-level: proves POST/DELETE /api/calendar/events fire writeback_event for a
|
||||
CalDAV-backed calendar and not for a local one.
|
||||
|
||||
Calls the async route handlers DIRECTLY (extracted from the router) rather than
|
||||
through Starlette's TestClient — the TestClient middleware-app + threadpool could
|
||||
hang in some environments; a direct call with a minimal fake request keeps the
|
||||
same coverage and completes reliably.
|
||||
"""
|
||||
|
||||
import tempfile
|
||||
import uuid
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
import core.database as cdb
|
||||
import routes.calendar_routes as croutes
|
||||
import src.caldav_writeback as wb
|
||||
from core.database import CalendarCal
|
||||
from routes.calendar_routes import EventCreate
|
||||
|
||||
_TMPDB = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||
_ENGINE = create_engine(
|
||||
f"sqlite:///{_TMPDB.name}",
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=NullPool,
|
||||
)
|
||||
cdb.Base.metadata.create_all(_ENGINE)
|
||||
_TS = sessionmaker(bind=_ENGINE, autoflush=False, autocommit=False)
|
||||
croutes.SessionLocal = _TS
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def calls(monkeypatch):
|
||||
recorded = []
|
||||
|
||||
async def _fake_writeback(owner, source, cal_id, ev, *, delete=False):
|
||||
recorded.append({"source": source, "cal_id": cal_id, "uid": ev.get("uid"), "delete": delete})
|
||||
return {"ok": True}
|
||||
|
||||
monkeypatch.setattr(wb, "writeback_event", _fake_writeback)
|
||||
return recorded
|
||||
|
||||
|
||||
def _req():
|
||||
return SimpleNamespace(state=SimpleNamespace(current_user="tester"))
|
||||
|
||||
|
||||
def _endpoint(method, suffix):
|
||||
router = croutes.setup_calendar_routes()
|
||||
for r in router.routes:
|
||||
if getattr(r, "path", "").endswith(suffix) and method in getattr(r, "methods", set()):
|
||||
return r.endpoint
|
||||
raise RuntimeError(f"{method} *{suffix} not found")
|
||||
|
||||
|
||||
def _make_cal(source):
|
||||
cid = ("caldav-" if source == "caldav" else "loc-") + uuid.uuid4().hex[:10]
|
||||
db = _TS()
|
||||
try:
|
||||
db.add(CalendarCal(id=cid, owner="tester", name="C", source=source))
|
||||
db.commit()
|
||||
return cid
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
async def test_create_on_caldav_calendar_pushes_to_remote(calls):
|
||||
create_event = _endpoint("POST", "/events")
|
||||
cal_id = _make_cal("caldav")
|
||||
res = await create_event(_req(), EventCreate(
|
||||
summary="Dentist", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||
assert res["ok"] is True
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["source"] == "caldav" and calls[0]["cal_id"] == cal_id
|
||||
assert calls[0]["delete"] is False
|
||||
|
||||
|
||||
async def test_create_on_local_calendar_does_not_push(calls):
|
||||
create_event = _endpoint("POST", "/events")
|
||||
cal_id = _make_cal("local")
|
||||
res = await create_event(_req(), EventCreate(
|
||||
summary="Local", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||
assert res["ok"] is True
|
||||
assert calls == []
|
||||
|
||||
|
||||
async def test_delete_on_caldav_calendar_pushes_delete(calls):
|
||||
create_event = _endpoint("POST", "/events")
|
||||
delete_event = _endpoint("DELETE", "/events/{uid}")
|
||||
cal_id = _make_cal("caldav")
|
||||
res = await create_event(_req(), EventCreate(
|
||||
summary="Temp", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||
uid = res["uid"]
|
||||
calls.clear()
|
||||
rd = await delete_event(_req(), uid)
|
||||
assert rd["ok"] is True
|
||||
assert len(calls) == 1 and calls[0]["delete"] is True and calls[0]["uid"] == uid
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Issue #1320 — the agent's manage_calendar tool can create a recurring event.
|
||||
|
||||
The create_event handler already persists `rrule`, but it wasn't documented in the
|
||||
tool schema, so the agent took "a roundabout way". This pins the end-to-end path:
|
||||
calling do_manage_calendar with an rrule stores a single event carrying that RRULE.
|
||||
"""
|
||||
|
||||
import json
|
||||
import tempfile
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
import core.database as cdb
|
||||
from core.database import CalendarEvent
|
||||
|
||||
_TMPDB = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||
_ENGINE = create_engine(
|
||||
f"sqlite:///{_TMPDB.name}",
|
||||
connect_args={"check_same_thread": False},
|
||||
poolclass=NullPool,
|
||||
)
|
||||
cdb.Base.metadata.create_all(_ENGINE)
|
||||
_TS = sessionmaker(bind=_ENGINE, autoflush=False, autocommit=False)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _bind_temp_db(monkeypatch):
|
||||
# do_manage_calendar does `from core.database import SessionLocal` at call
|
||||
# time, so patch the module attribute to our temp DB — via monkeypatch so it
|
||||
# is RESTORED after each test and can't leak into later tests in the process.
|
||||
monkeypatch.setattr(cdb, "SessionLocal", _TS)
|
||||
yield
|
||||
|
||||
|
||||
async def test_create_event_with_rrule_persists_recurrence():
|
||||
from src.tool_implementations import do_manage_calendar
|
||||
|
||||
owner = "tester-" + uuid.uuid4().hex[:6]
|
||||
rrule = "FREQ=WEEKLY;BYDAY=MO"
|
||||
res = await do_manage_calendar(json.dumps({
|
||||
"action": "create_event",
|
||||
"summary": "Standup",
|
||||
"dtstart": "2026-06-08T09:00:00Z",
|
||||
"rrule": rrule,
|
||||
}), owner=owner)
|
||||
assert res.get("exit_code", 0) == 0, res
|
||||
uid = res.get("uid")
|
||||
assert uid, res
|
||||
|
||||
db = _TS()
|
||||
try:
|
||||
ev = db.query(CalendarEvent).filter(CalendarEvent.uid == uid).first()
|
||||
assert ev is not None
|
||||
assert ev.rrule == rrule # ONE event carrying the recurrence rule
|
||||
assert ev.summary == "Standup"
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
async def test_create_event_without_rrule_is_single():
|
||||
from src.tool_implementations import do_manage_calendar
|
||||
|
||||
owner = "tester-" + uuid.uuid4().hex[:6]
|
||||
res = await do_manage_calendar(json.dumps({
|
||||
"action": "create_event",
|
||||
"summary": "One-off",
|
||||
"dtstart": "2026-06-09T10:00:00Z",
|
||||
}), owner=owner)
|
||||
assert res.get("exit_code", 0) == 0, res
|
||||
db = _TS()
|
||||
try:
|
||||
ev = db.query(CalendarEvent).filter(CalendarEvent.uid == res["uid"]).first()
|
||||
assert ev is not None and (ev.rrule or "") == ""
|
||||
finally:
|
||||
db.close()
|
||||
@@ -0,0 +1,14 @@
|
||||
"""Regression: check_outbound_url must reject a non-string URL, not crash.
|
||||
|
||||
The `if not url or not url.strip()` guard only handled falsy values; a truthy
|
||||
non-string (e.g. an int) reached `.strip()` and raised AttributeError out of
|
||||
this SSRF check. Non-strings now fail closed with a clear message.
|
||||
"""
|
||||
from src.url_safety import check_outbound_url
|
||||
|
||||
|
||||
def test_non_string_fails_closed():
|
||||
ok, _ = check_outbound_url(123)
|
||||
assert ok is False
|
||||
ok2, _ = check_outbound_url(None)
|
||||
assert ok2 is False
|
||||
@@ -44,3 +44,75 @@ def test_is_setting_overridden_reads_raw_saved_file(tmp_path, monkeypatch):
|
||||
|
||||
f.write_text(json.dumps({}), encoding="utf-8")
|
||||
assert settings.is_setting_overridden("agent_input_token_budget") is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Configurable hard_max — completes the reviewer requirement from #1190 that
|
||||
# was carried over but not implemented in #1230: the ceiling on the auto-
|
||||
# derived path should be a setting, not a hidden constant. Without this,
|
||||
# admins on premium APIs with very large windows (1M+ context) can only
|
||||
# raise the ceiling by editing src/context_budget.py.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_custom_hard_max_overrides_default_in_auto_branch():
|
||||
"""A caller-supplied hard_max lifts the auto-derived ceiling."""
|
||||
# Without override: 1M ctx -> capped at DEFAULT_HARD_MAX (200K)
|
||||
assert compute_input_token_budget(6000, 1_000_000, explicit=False) == DEFAULT_HARD_MAX
|
||||
# With explicit raise: 1M ctx -> 850K (85% of 1M), under the raised ceiling
|
||||
assert compute_input_token_budget(6000, 1_000_000, explicit=False, hard_max=900_000) == int(1_000_000 * 0.85)
|
||||
|
||||
|
||||
def test_custom_hard_max_lowers_default_for_cost_paranoid_setups():
|
||||
"""A lower ceiling caps the auto-derived budget below the default."""
|
||||
# 128K ctx, default ceiling 200K -> 85% of 128K = 108800
|
||||
assert compute_input_token_budget(6000, 128_000, explicit=False) == int(128_000 * 0.85)
|
||||
# Same ctx, ceiling lowered to 50K -> capped at 50K instead
|
||||
assert compute_input_token_budget(6000, 128_000, explicit=False, hard_max=50_000) == 50_000
|
||||
|
||||
|
||||
def test_hard_max_has_no_effect_on_explicit_branch():
|
||||
"""When the user set an explicit budget, hard_max must not silently cap it."""
|
||||
# User chose 900K explicitly; ctx is 1M; ceiling is 100K — user's choice wins.
|
||||
assert compute_input_token_budget(900_000, 1_000_000, explicit=True, hard_max=100_000) == 900_000
|
||||
|
||||
|
||||
def test_default_settings_registers_hard_max_key():
|
||||
"""Required so /api/auth/settings and manage_settings can persist the key."""
|
||||
from src.settings import DEFAULT_SETTINGS
|
||||
assert "agent_input_token_hard_max" in DEFAULT_SETTINGS
|
||||
assert DEFAULT_SETTINGS["agent_input_token_hard_max"] == DEFAULT_HARD_MAX
|
||||
|
||||
|
||||
def test_alias_map_registers_friendly_names():
|
||||
"""`manage_settings` should accept 'hard max' and friends."""
|
||||
from pathlib import Path
|
||||
src = Path("src/tool_implementations.py").read_text()
|
||||
assert '"hard max": "agent_input_token_hard_max"' in src
|
||||
assert '"token budget cap": "agent_input_token_hard_max"' in src
|
||||
assert '"input budget cap": "agent_input_token_hard_max"' in src
|
||||
|
||||
|
||||
def test_agent_loop_reads_hard_max_setting(tmp_path, monkeypatch):
|
||||
"""End-to-end: a saved settings.json value for agent_input_token_hard_max
|
||||
must reach compute_input_token_budget on the real agent_loop call path."""
|
||||
import src.settings as settings
|
||||
# Point SETTINGS_FILE at a temp file with our override.
|
||||
f = tmp_path / "settings.json"
|
||||
f.write_text(json.dumps({"agent_input_token_hard_max": 750_000}), encoding="utf-8")
|
||||
monkeypatch.setattr(settings, "SETTINGS_FILE", str(f))
|
||||
monkeypatch.setattr(settings, "_settings_cache", None)
|
||||
# Read via the same import path the agent loop uses.
|
||||
assert settings.get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX) == 750_000
|
||||
|
||||
# Malformed value falls back to DEFAULT_HARD_MAX (defensive, matches the
|
||||
# try/except in src/agent_loop.py).
|
||||
f.write_text(json.dumps({"agent_input_token_hard_max": "huge"}), encoding="utf-8")
|
||||
monkeypatch.setattr(settings, "_settings_cache", None)
|
||||
raw = settings.get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX)
|
||||
try:
|
||||
parsed = int(raw)
|
||||
except (TypeError, ValueError):
|
||||
parsed = DEFAULT_HARD_MAX
|
||||
if parsed <= 0:
|
||||
parsed = DEFAULT_HARD_MAX
|
||||
assert parsed == DEFAULT_HARD_MAX
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Regression guard for issue #1291 — CPU-only serve still emitted GPU-only flags.
|
||||
|
||||
The llama.cpp serve command builder (static/js/cookbook.js) added
|
||||
`--flash-attn on` and exported `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` from
|
||||
independent toggles, so a CPU-only config (`-ngl 0`, often with flash-attn left
|
||||
on by an Auto profile) produced a command that mixes "zero GPU layers" with
|
||||
CUDA/flash-attn and fails to start. The builder now drops those GPU-only flags
|
||||
when ngl == 0, per the maintainer's guidance.
|
||||
|
||||
cookbook.js pulls in browser globals so it can't run under node; guard the fix
|
||||
at the source level: a `_cpuOnly` gate exists and is applied to flash-attn and
|
||||
the CUDA unified-memory env.
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
SRC = Path(__file__).resolve().parent.parent / "static/js/cookbook.js"
|
||||
|
||||
|
||||
def test_cpu_only_drops_gpu_only_flags():
|
||||
text = SRC.read_text(encoding="utf-8")
|
||||
# A CPU-only flag derived from ngl == 0.
|
||||
assert re.search(r"_cpuOnly\s*=\s*String\(f\.ngl\)\.trim\(\)\s*===\s*'0'", text), \
|
||||
"expected a _cpuOnly gate derived from ngl==0"
|
||||
# flash-attn must be suppressed for CPU-only.
|
||||
assert re.search(r"if\s*\(\s*f\.flash_attn\s*&&\s*!_cpuOnly\s*\)", text), \
|
||||
"flash-attn must be gated on !_cpuOnly"
|
||||
# The CUDA unified-memory env must be suppressed for CPU-only too.
|
||||
assert "f.unified_mem && !_cpuOnly" in text, \
|
||||
"GGML_CUDA_ENABLE_UNIFIED_MEMORY must be gated on !_cpuOnly"
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Regression guard for issue #1355 — the Cookbook *download* error toast used
|
||||
the default ~1.2s duration, so an actionable message like "tmux is required …"
|
||||
vanished before it could be read. The serve path already used multi-second
|
||||
durations; the download-failure toasts now match.
|
||||
|
||||
cookbookDownload.js pulls in browser globals so it can't run under node; this
|
||||
guards the durations at the source level.
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
SRC = Path(__file__).resolve().parent.parent / "static/js/cookbookDownload.js"
|
||||
_MIN_MS = 5000
|
||||
|
||||
|
||||
def test_download_failure_toasts_stay_visible():
|
||||
# Each download-failure toast is a single line; assert each carries an
|
||||
# explicit duration >= _MIN_MS so the actionable error stays readable.
|
||||
lines = [
|
||||
ln for ln in SRC.read_text(encoding="utf-8").splitlines()
|
||||
if "showToast(" in ln and "Download failed:" in ln
|
||||
]
|
||||
assert lines, "expected at least one 'Download failed' showToast call"
|
||||
for ln in lines:
|
||||
m = re.search(r",\s*(\d{3,})\s*\)\s*;?\s*$", ln)
|
||||
assert m, f"download-failure toast has no explicit duration: {ln.strip()}"
|
||||
assert int(m.group(1)) >= _MIN_MS, f"duration too short to read: {ln.strip()}"
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Regression tests for issue #1341 — deep research used the model's
|
||||
training-cutoff year (e.g. "best Python tutorials 2025") because the
|
||||
query-generation and planning prompts never told the LLM the current date.
|
||||
|
||||
The chat/agent path already injects "Today is ..." (src/agent_loop.py); deep
|
||||
research had no equivalent. These tests pin that the current year now reaches
|
||||
the LLM at both the planning and query-generation steps, without needing a live
|
||||
LLM or DB.
|
||||
"""
|
||||
import asyncio
|
||||
from datetime import datetime
|
||||
|
||||
from src.deep_research import (
|
||||
DeepResearcher,
|
||||
current_date_context,
|
||||
RESEARCH_PLAN_PROMPT,
|
||||
)
|
||||
|
||||
|
||||
def _this_year() -> str:
|
||||
return datetime.now().astimezone().strftime("%Y")
|
||||
|
||||
|
||||
def test_current_date_context_names_the_real_year():
|
||||
ctx = current_date_context()
|
||||
assert _this_year() in ctx
|
||||
# It must actively steer the model away from training-data years.
|
||||
assert "training data" in ctx.lower()
|
||||
|
||||
|
||||
def test_generate_queries_prompt_carries_the_current_year():
|
||||
# Build without the heavy __init__; _generate_queries only needs these.
|
||||
r = DeepResearcher.__new__(DeepResearcher)
|
||||
r.research_plan = ""
|
||||
r.queries_used = set()
|
||||
|
||||
seen = {}
|
||||
|
||||
async def _fake_llm(messages, **kwargs):
|
||||
seen["prompt"] = messages[0]["content"]
|
||||
return '["python tutorials", "python guides"]'
|
||||
|
||||
r._llm = _fake_llm
|
||||
|
||||
queries = asyncio.run(r._generate_queries("best python tutorials", "", 1))
|
||||
|
||||
assert queries # sanity: the JSON array parsed
|
||||
# The fix: the real current year is in the prompt the LLM actually sees.
|
||||
assert _this_year() in seen["prompt"]
|
||||
|
||||
|
||||
def test_plan_prompt_carries_the_current_year():
|
||||
r = DeepResearcher.__new__(DeepResearcher)
|
||||
|
||||
seen = {}
|
||||
|
||||
async def _fake_llm(messages, **kwargs):
|
||||
seen["prompt"] = messages[0]["content"]
|
||||
return "{}"
|
||||
|
||||
r._llm = _fake_llm
|
||||
|
||||
asyncio.run(r._create_plan("what changed this year"))
|
||||
|
||||
assert _this_year() in seen["prompt"]
|
||||
# The base template itself stays year-agnostic; the year comes from the
|
||||
# prepended context, proving the wiring (not a hard-coded prompt edit).
|
||||
assert _this_year() not in RESEARCH_PLAN_PROMPT
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user