mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-11-08 14:12:22 +01:00
Compare commits
93
Commits
926a4c59cb
...
aa5e3f6884
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aa5e3f6884 | ||
|
|
fc220f760f | ||
|
|
2d94e38d23 | ||
|
|
03ddc5d2c4 | ||
|
|
3175d7ca21 | ||
|
|
d818117d4c | ||
|
|
6b2618dab4 | ||
|
|
0051023056 | ||
|
|
8783f12c4c | ||
|
|
82c09dd768 | ||
|
|
c3bf32d1b1 | ||
|
|
df3864bd15 | ||
|
|
ffeb7d8c97 | ||
|
|
a6b7a7bc60 | ||
|
|
0cc1814658 | ||
|
|
8051e25c65 | ||
|
|
0ad5cd783b | ||
|
|
953305a5af | ||
|
|
15a3b71802 | ||
|
|
e68d0448b8 | ||
|
|
db3a5c17b0 | ||
|
|
f39c87561b | ||
|
|
f7df069ca1 | ||
|
|
eacb99f963 | ||
|
|
fb1341b629 | ||
|
|
19b6cbac12 | ||
|
|
258fe455eb | ||
|
|
667c663668 | ||
|
|
0283216e67 | ||
|
|
77313170c6 | ||
|
|
0d25dfb5f4 | ||
|
|
6df0f5e6df | ||
|
|
382d49d887 | ||
|
|
0d88c9989e | ||
|
|
5462e36d10 | ||
|
|
5b0c7442ee | ||
|
|
441905bc13 | ||
|
|
24be3f3ca8 | ||
|
|
cf60a14d74 | ||
|
|
bd0845e081 | ||
|
|
dc3421c34e | ||
|
|
1f743970dd | ||
|
|
c9361262df | ||
|
|
0e6cbd8315 | ||
|
|
5c6bd0fc2b | ||
|
|
57abe69173 | ||
|
|
583df3dd6a | ||
|
|
70103d8719 | ||
|
|
8450cee02a | ||
|
|
1ecd113808 | ||
|
|
4d1829add0 | ||
|
|
5fd71f68e8 | ||
|
|
0ec8415f0e | ||
|
|
fd37ccebae | ||
|
|
abbc073429 | ||
|
|
a5282e9748 | ||
|
|
35c40bce75 | ||
|
|
77b63ed942 | ||
|
|
1f2a06facd | ||
|
|
69d6fe44b3 | ||
|
|
7af168fa59 | ||
|
|
7ce2db2771 | ||
|
|
cb114d6e87 | ||
|
|
50a486b608 | ||
|
|
b6843c7621 | ||
|
|
b54468291e | ||
|
|
66c9349ee3 | ||
|
|
c3fd969965 | ||
|
|
ce7f5dbbdd | ||
|
|
b2291fad49 | ||
|
|
56656de5bc | ||
|
|
54a221b367 | ||
|
|
4771d80eb2 | ||
|
|
80de69ebb0 | ||
|
|
4019283eba | ||
|
|
97f855b40d | ||
|
|
5ee30cc144 | ||
|
|
1adf21a7e5 | ||
|
|
e73545f64f | ||
|
|
34918d9921 | ||
|
|
e392be0d65 | ||
|
|
dc6711b3c5 | ||
|
|
1a763b0539 | ||
|
|
5452bc96b1 | ||
|
|
9c68ceafeb | ||
|
|
a8395b4e4c | ||
|
|
adde94e430 | ||
|
|
1507d140b8 | ||
|
|
7504fedb17 | ||
|
|
257f7ee7b2 | ||
|
|
8852c7ea4a | ||
|
|
c39d8db12a | ||
|
|
3505a5ff27 |
@@ -0,0 +1,103 @@
|
|||||||
|
name: Bug Report
|
||||||
|
description: Report a reproducible bug in Odysseus.
|
||||||
|
labels: ["bug"]
|
||||||
|
|
||||||
|
body:
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: |
|
||||||
|
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||||
|
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||||
|
Duplicate reports slow things down.
|
||||||
|
|
||||||
|
For security vulnerabilities, **do not open a public issue** —
|
||||||
|
use [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new)
|
||||||
|
and read [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md) first.
|
||||||
|
|
||||||
|
- type: checkboxes
|
||||||
|
id: prerequisites
|
||||||
|
attributes:
|
||||||
|
label: Prerequisites
|
||||||
|
options:
|
||||||
|
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and did not find an existing report of this bug.
|
||||||
|
required: true
|
||||||
|
- label: This is **not** a security vulnerability. (Vulnerabilities go to [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new) — see [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md).)
|
||||||
|
required: true
|
||||||
|
- label: I am running the latest code from `main`.
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: dropdown
|
||||||
|
id: install-method
|
||||||
|
attributes:
|
||||||
|
label: Install Method
|
||||||
|
options:
|
||||||
|
- Docker (docker compose up)
|
||||||
|
- Manual Python install (pip / venv)
|
||||||
|
- Windows native (launch-windows.ps1)
|
||||||
|
- macOS app (build-macos-app.sh / start-macos.sh)
|
||||||
|
- Other (describe in the reproduction steps below)
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: dropdown
|
||||||
|
id: os
|
||||||
|
attributes:
|
||||||
|
label: Operating System
|
||||||
|
options:
|
||||||
|
- Linux
|
||||||
|
- macOS
|
||||||
|
- Windows
|
||||||
|
- Other
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: steps
|
||||||
|
attributes:
|
||||||
|
label: Steps to Reproduce
|
||||||
|
description: Exact steps that reliably trigger the bug. The more specific, the faster this gets fixed.
|
||||||
|
placeholder: |
|
||||||
|
1. Go to ...
|
||||||
|
2. Click / type ...
|
||||||
|
3. Observe ...
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: expected
|
||||||
|
attributes:
|
||||||
|
label: Expected Behaviour
|
||||||
|
description: What should have happened?
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: actual
|
||||||
|
attributes:
|
||||||
|
label: Actual Behaviour
|
||||||
|
description: What actually happened? Include the full error message if there is one.
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: logs
|
||||||
|
attributes:
|
||||||
|
label: Logs / Screenshots
|
||||||
|
description: Paste relevant terminal output or attach screenshots. Remove API keys, passwords, and personal data before pasting.
|
||||||
|
render: text
|
||||||
|
|
||||||
|
- type: input
|
||||||
|
id: model-backend
|
||||||
|
attributes:
|
||||||
|
label: Model / Backend (if relevant)
|
||||||
|
description: "e.g. Ollama + llama3.2:latest, vLLM + mistral-7b, OpenAI API, Anthropic API"
|
||||||
|
placeholder: "Ollama + llama3.2:latest"
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: additional-info
|
||||||
|
attributes:
|
||||||
|
label: Additional Information
|
||||||
|
description: Anything else that might help — browser console errors, related issues, things you already tried, or environment quirks.
|
||||||
|
placeholder: |
|
||||||
|
- Any other context goes here.
|
||||||
|
- If you are willing to submit a PR that fixes this, mention it here.
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
blank_issues_enabled: false
|
||||||
|
contact_links:
|
||||||
|
- name: Question / Need Help
|
||||||
|
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/q-a
|
||||||
|
about: Ask how-to questions, setup help, and model configuration questions here. Issues are for confirmed bugs and concrete proposals only.
|
||||||
|
|
||||||
|
- name: Idea or Suggestion
|
||||||
|
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas
|
||||||
|
about: Discuss ideas and gauge interest before opening a formal feature request. If there is already a discussion, link it in your feature request.
|
||||||
|
|
||||||
|
- name: Security Vulnerability
|
||||||
|
url: https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new
|
||||||
|
about: Report vulnerabilities privately via GitHub Security Advisories — never as a public issue. Read SECURITY.md before reporting.
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
name: Feature Request
|
||||||
|
description: Propose a new feature or a concrete improvement to Odysseus.
|
||||||
|
labels: ["enhancement"]
|
||||||
|
|
||||||
|
body:
|
||||||
|
- type: markdown
|
||||||
|
attributes:
|
||||||
|
value: |
|
||||||
|
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||||
|
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||||
|
Feature requests that duplicate [ROADMAP.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/ROADMAP.md)
|
||||||
|
or an existing open issue will be closed as duplicates.
|
||||||
|
|
||||||
|
If your idea needs community input before it becomes a concrete proposal,
|
||||||
|
start a [discussion](https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas) instead.
|
||||||
|
|
||||||
|
- type: checkboxes
|
||||||
|
id: prerequisites
|
||||||
|
attributes:
|
||||||
|
label: Prerequisites
|
||||||
|
options:
|
||||||
|
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and this has not already been proposed.
|
||||||
|
required: true
|
||||||
|
- label: I searched [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and this is not already being debated there.
|
||||||
|
required: true
|
||||||
|
- label: This is a concrete, actionable proposal — not a vague "it would be nice if..." request.
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: dropdown
|
||||||
|
id: area
|
||||||
|
attributes:
|
||||||
|
label: Area
|
||||||
|
description: Which part of the application does this affect?
|
||||||
|
options:
|
||||||
|
- Chat / Agent
|
||||||
|
- Email
|
||||||
|
- Calendar
|
||||||
|
- Documents / RAG
|
||||||
|
- Memory
|
||||||
|
- Cookbook / Local Models / GPU
|
||||||
|
- Search
|
||||||
|
- Notes / Editor
|
||||||
|
- Auth / Security
|
||||||
|
- Docker / Deployment
|
||||||
|
- UI / Frontend
|
||||||
|
- API / Backend
|
||||||
|
- MCP
|
||||||
|
- Testing / CI
|
||||||
|
- Other
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: problem
|
||||||
|
attributes:
|
||||||
|
label: Problem or Motivation
|
||||||
|
description: What problem does this solve, or what use case does it enable? Be specific — "it would be better" is not enough.
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: solution
|
||||||
|
attributes:
|
||||||
|
label: Proposed Solution
|
||||||
|
description: Describe the behaviour or change you want to see. Include API shape, UI sketch, or code snippets if that helps make it concrete.
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: alternatives
|
||||||
|
attributes:
|
||||||
|
label: Alternatives Considered
|
||||||
|
description: What other approaches did you consider and why did you rule them out? If there is an existing workaround, describe it.
|
||||||
|
|
||||||
|
- type: textarea
|
||||||
|
id: prior-art
|
||||||
|
attributes:
|
||||||
|
label: Prior Art / Related Issues
|
||||||
|
description: Link any related issues, discussions, or external references that informed this proposal.
|
||||||
|
|
||||||
|
- type: dropdown
|
||||||
|
id: willing_to_implement
|
||||||
|
attributes:
|
||||||
|
label: Are you willing to implement this?
|
||||||
|
options:
|
||||||
|
- "Yes — I can open a PR"
|
||||||
|
- "Partially — I can help but need guidance"
|
||||||
|
- "No — I am only filing the request"
|
||||||
|
validations:
|
||||||
|
required: true
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
## Summary
|
||||||
|
|
||||||
|
<!-- One paragraph: what changed and why. "Fixed bug" and "Added feature" are not summaries. -->
|
||||||
|
|
||||||
|
## Linked Issue
|
||||||
|
|
||||||
|
<!-- Every PR should be linked to an issue.
|
||||||
|
Use one of: Fixes #NNN | Part of #NNN | Closes #NNN -->
|
||||||
|
|
||||||
|
Fixes #
|
||||||
|
|
||||||
|
## Type of Change
|
||||||
|
|
||||||
|
- [ ] Bug fix (non-breaking — fixes a confirmed issue)
|
||||||
|
- [ ] New feature (non-breaking — adds new behaviour)
|
||||||
|
- [ ] Breaking change (changes or removes existing behaviour)
|
||||||
|
- [ ] Refactor / cleanup (behaviour unchanged)
|
||||||
|
- [ ] Documentation only
|
||||||
|
- [ ] CI / tooling / configuration
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- [ ] I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues) and [open PRs](https://github.com/pewdiepie-archdaemon/odysseus/pulls) — this is not a duplicate.
|
||||||
|
- [ ] This PR targets `main`
|
||||||
|
- [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
|
||||||
|
|
||||||
|
## How to Test
|
||||||
|
|
||||||
|
<!-- Step-by-step instructions a reviewer can follow to verify this works.
|
||||||
|
Do not leave this empty — a PR without test steps will be sent back. -->
|
||||||
|
|
||||||
|
1.
|
||||||
|
2.
|
||||||
|
3.
|
||||||
|
|
||||||
|
## Screenshots (UI changes only)
|
||||||
|
|
||||||
|
<!-- Drag and drop images or a screen recording here. Delete this section if not a UI change. -->
|
||||||
@@ -1,7 +1,10 @@
|
|||||||
# Odysseus
|
# Odysseus
|
||||||
|
|
||||||
|
```
|
||||||
───────────────────────────────────────────────
|
───────────────────────────────────────────────
|
||||||
⊹ ࣪ ˖ ૮( ˶ᵔ ᵕ ᵔ˶ )っ Odysseus vers. 1.0
|
⊹ ࣪ ˖ ૮( ˶ᵔ ᵕ ᵔ˶ )っ Odysseus vers. 1.0
|
||||||
───────────────────────────────────────────────
|
───────────────────────────────────────────────
|
||||||
|
```
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@@ -219,6 +222,13 @@ Ollama must listen outside its own loopback interface:
|
|||||||
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
||||||
```
|
```
|
||||||
|
|
||||||
|
This connects Odysseus in Docker to an Ollama server that is already running on
|
||||||
|
your host machine; it does not start Ollama inside the container.
|
||||||
|
`host.docker.internal` is Docker's hostname for the host machine from inside the
|
||||||
|
container. Cookbook **Serve** is a separate workflow for serving downloaded
|
||||||
|
models through Odysseus/llama.cpp, so Windows users with an existing Ollama
|
||||||
|
install usually only need to add the endpoint in Settings.
|
||||||
|
|
||||||
**Useful checks.**
|
**Useful checks.**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
+6
-1
@@ -79,7 +79,12 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
searxng:
|
searxng:
|
||||||
image: docker.io/searxng/searxng:latest
|
# Pinned, not :latest — odysseus waits on searxng's healthcheck
|
||||||
|
# (depends_on: condition: service_healthy), so a broken upstream `latest`
|
||||||
|
# tag blocks the whole app from starting. 2026.6.2 crashes on boot with
|
||||||
|
# `KeyError: 'default_doi_resolver'`, failing the healthcheck (issue #1414).
|
||||||
|
# Bump this deliberately after verifying a newer tag boots clean.
|
||||||
|
image: docker.io/searxng/searxng:2026.5.31-7159b8aed
|
||||||
entrypoint:
|
entrypoint:
|
||||||
- /bin/sh
|
- /bin/sh
|
||||||
- -c
|
- -c
|
||||||
|
|||||||
Binary file not shown.
|
Before Width: | Height: | Size: 52 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 51 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 90 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 125 KiB |
@@ -13,6 +13,10 @@ SEARCH_TIMEOUT = 30
|
|||||||
|
|
||||||
def truncate(text: str, limit: int = MAX_OUTPUT_CHARS) -> str:
|
def truncate(text: str, limit: int = MAX_OUTPUT_CHARS) -> str:
|
||||||
"""Truncate text to *limit* characters with a suffix note."""
|
"""Truncate text to *limit* characters with a suffix note."""
|
||||||
|
if not isinstance(text, str):
|
||||||
|
# Tool output is occasionally None or a non-string; len(None) would
|
||||||
|
# raise. Coerce so this shared helper never crashes a tool response.
|
||||||
|
text = "" if text is None else str(text)
|
||||||
if len(text) > limit:
|
if len(text) > limit:
|
||||||
return text[:limit] + f"\n... (truncated, {len(text)} chars total)"
|
return text[:limit] + f"\n... (truncated, {len(text)} chars total)"
|
||||||
return text
|
return text
|
||||||
|
|||||||
@@ -417,6 +417,11 @@ def _list_emails(folder="INBOX", max_results=20, unresponded_only=False,
|
|||||||
status, data = conn.uid("SEARCH", None, "(UNSEEN UNANSWERED)")
|
status, data = conn.uid("SEARCH", None, "(UNSEEN UNANSWERED)")
|
||||||
elif unread_only:
|
elif unread_only:
|
||||||
status, data = conn.uid("SEARCH", None, "(UNSEEN)")
|
status, data = conn.uid("SEARCH", None, "(UNSEEN)")
|
||||||
|
elif unresponded_only:
|
||||||
|
# Was missing — unresponded_only=True (without unread_only) fell through
|
||||||
|
# to "ALL" and returned answered mail too, despite the documented
|
||||||
|
# "emails without replies" behaviour.
|
||||||
|
status, data = conn.uid("SEARCH", None, "(UNANSWERED)")
|
||||||
else:
|
else:
|
||||||
# Include read too — IMAP search "ALL" returns the entire folder
|
# Include read too — IMAP search "ALL" returns the entire folder
|
||||||
status, data = conn.uid("SEARCH", None, "ALL")
|
status, data = conn.uid("SEARCH", None, "ALL")
|
||||||
|
|||||||
@@ -101,7 +101,8 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
|||||||
return [TextContent(type="text", text=f"Error: {e}")]
|
return [TextContent(type="text", text=f"Error: {e}")]
|
||||||
|
|
||||||
elif action == "add_directory":
|
elif action == "add_directory":
|
||||||
directory = arguments.get("directory", "").strip()
|
_dir = arguments.get("directory")
|
||||||
|
directory = _dir.strip() if isinstance(_dir, str) else ""
|
||||||
if not directory:
|
if not directory:
|
||||||
return [TextContent(type="text", text="Error: add_directory needs a directory path")]
|
return [TextContent(type="text", text="Error: add_directory needs a directory path")]
|
||||||
directory = os.path.expanduser(directory)
|
directory = os.path.expanduser(directory)
|
||||||
@@ -112,14 +113,27 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
|||||||
try:
|
try:
|
||||||
result = _rag_manager.index_personal_documents(directory)
|
result = _rag_manager.index_personal_documents(directory)
|
||||||
indexed = result.get("indexed_count", 0) if isinstance(result, dict) else 0
|
indexed = result.get("indexed_count", 0) if isinstance(result, dict) else 0
|
||||||
|
# Record the directory so `list` and `remove_directory` can see it.
|
||||||
|
# Indexing was just done above, so pass index=False to avoid a second
|
||||||
|
# (ownerless) pass. Without this the directory was indexed but never
|
||||||
|
# tracked in indexed_directories, so it was invisible/unremovable.
|
||||||
|
if _personal_docs_manager and hasattr(_personal_docs_manager, "add_directory"):
|
||||||
|
try:
|
||||||
|
_personal_docs_manager.add_directory(directory, index=False)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
return [TextContent(type="text", text=f"Directory '{directory}' added to RAG index ({indexed} chunks indexed)")]
|
return [TextContent(type="text", text=f"Directory '{directory}' added to RAG index ({indexed} chunks indexed)")]
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
return [TextContent(type="text", text=f"Error: Failed to index directory: {e}")]
|
return [TextContent(type="text", text=f"Error: Failed to index directory: {e}")]
|
||||||
|
|
||||||
elif action == "remove_directory":
|
elif action == "remove_directory":
|
||||||
directory = arguments.get("directory", "").strip()
|
_dir = arguments.get("directory")
|
||||||
|
directory = _dir.strip() if isinstance(_dir, str) else ""
|
||||||
if not directory:
|
if not directory:
|
||||||
return [TextContent(type="text", text="Error: remove_directory needs a directory path")]
|
return [TextContent(type="text", text="Error: remove_directory needs a directory path")]
|
||||||
|
# Expand ~ to match add_directory, which indexes the expanded path.
|
||||||
|
# Without this, removing "~/docs" never matches the stored absolute path.
|
||||||
|
directory = os.path.expanduser(directory)
|
||||||
if not _personal_docs_manager:
|
if not _personal_docs_manager:
|
||||||
return [TextContent(type="text", text="Error: Personal docs manager not available")]
|
return [TextContent(type="text", text="Error: Personal docs manager not available")]
|
||||||
try:
|
try:
|
||||||
|
|||||||
+1
-1
@@ -9,7 +9,7 @@ Type=simple
|
|||||||
# CHANGE THESE to match your user and install path:
|
# CHANGE THESE to match your user and install path:
|
||||||
User=YOURUSER
|
User=YOURUSER
|
||||||
WorkingDirectory=/home/YOURUSER/odysseus-ui
|
WorkingDirectory=/home/YOURUSER/odysseus-ui
|
||||||
ExecStart=/home/YOURUSER/odysseus-ui/venv/bin/uvicorn app:app --port 8000 --host 0.0.0.0
|
ExecStart=/home/YOURUSER/odysseus-ui/venv/bin/uvicorn app:app --port 7000 --host 0.0.0.0
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=3
|
RestartSec=3
|
||||||
EnvironmentFile=-/home/YOURUSER/odysseus-ui/.env
|
EnvironmentFile=-/home/YOURUSER/odysseus-ui/.env
|
||||||
|
|||||||
@@ -528,13 +528,20 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
owner = _require_user(request)
|
owner = _require_user(request)
|
||||||
from routes.prefs_routes import _load_for_user
|
from routes.prefs_routes import _load_for_user
|
||||||
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||||
|
caldav_password = cfg.get("password") or ""
|
||||||
|
if caldav_password:
|
||||||
|
try:
|
||||||
|
from src.secret_storage import decrypt
|
||||||
|
caldav_password = decrypt(caldav_password)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
# Surface url+username but never hand the password back to the
|
# Surface url+username but never hand the password back to the
|
||||||
# client — saved-state UI shouldn't leak the credential.
|
# client — saved-state UI shouldn't leak the credential.
|
||||||
return {
|
return {
|
||||||
"url": cfg.get("url", "") or "",
|
"url": cfg.get("url", "") or "",
|
||||||
"username": cfg.get("username", "") or "",
|
"username": cfg.get("username", "") or "",
|
||||||
"password": "",
|
"password": "",
|
||||||
"has_password": bool(cfg.get("password")),
|
"has_password": bool(caldav_password),
|
||||||
"local": not bool(cfg.get("url")),
|
"local": not bool(cfg.get("url")),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -553,12 +560,20 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
prefs.pop("caldav", None)
|
prefs.pop("caldav", None)
|
||||||
_save_for_user(owner, prefs)
|
_save_for_user(owner, prefs)
|
||||||
return {"ok": True, "cleared": True}
|
return {"ok": True, "cleared": True}
|
||||||
cfg["url"] = body.get("url", "").strip()
|
from src.caldav_sync import validate_caldav_url
|
||||||
|
try:
|
||||||
|
cfg["url"] = validate_caldav_url(body.get("url", ""))
|
||||||
|
except ValueError as e:
|
||||||
|
raise HTTPException(400, str(e))
|
||||||
cfg["username"] = (body.get("username") or "").strip()
|
cfg["username"] = (body.get("username") or "").strip()
|
||||||
# Preserve the stored password when the client sends an empty
|
# Preserve the stored password when the client sends an empty
|
||||||
# one (edit form re-submitted without re-typing the password).
|
# one (edit form re-submitted without re-typing the password).
|
||||||
if body.get("password"):
|
if body.get("password"):
|
||||||
cfg["password"] = body["password"]
|
from src.secret_storage import encrypt
|
||||||
|
cfg["password"] = encrypt(body["password"])
|
||||||
|
elif cfg.get("password"):
|
||||||
|
from src.secret_storage import encrypt
|
||||||
|
cfg["password"] = encrypt(cfg["password"])
|
||||||
prefs["caldav"] = cfg
|
prefs["caldav"] = cfg
|
||||||
_save_for_user(owner, prefs)
|
_save_for_user(owner, prefs)
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
@@ -585,9 +600,21 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||||
url = url or (cfg.get("url") or "")
|
url = url or (cfg.get("url") or "")
|
||||||
user = user or (cfg.get("username") or "")
|
user = user or (cfg.get("username") or "")
|
||||||
pw = pw or (cfg.get("password") or "")
|
if not pw:
|
||||||
|
pw = cfg.get("password") or ""
|
||||||
|
if pw:
|
||||||
|
try:
|
||||||
|
from src.secret_storage import decrypt
|
||||||
|
pw = decrypt(pw)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
if not (url and user and pw):
|
if not (url and user and pw):
|
||||||
return {"ok": False, "error": "Missing URL, username, or password"}
|
return {"ok": False, "error": "Missing URL, username, or password"}
|
||||||
|
from src.caldav_sync import validate_caldav_url
|
||||||
|
try:
|
||||||
|
url = validate_caldav_url(url)
|
||||||
|
except ValueError as e:
|
||||||
|
return {"ok": False, "error": str(e)}
|
||||||
import httpx
|
import httpx
|
||||||
propfind_body = (
|
propfind_body = (
|
||||||
'<?xml version="1.0" encoding="UTF-8"?>\n'
|
'<?xml version="1.0" encoding="UTF-8"?>\n'
|
||||||
@@ -595,7 +622,7 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
'</d:prop></d:propfind>'
|
'</d:prop></d:propfind>'
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(timeout=8.0, follow_redirects=True) as cx:
|
async with httpx.AsyncClient(timeout=8.0, follow_redirects=False, trust_env=False) as cx:
|
||||||
r = await cx.request(
|
r = await cx.request(
|
||||||
"PROPFIND", url,
|
"PROPFIND", url,
|
||||||
auth=(user, pw),
|
auth=(user, pw),
|
||||||
@@ -612,6 +639,8 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
return {"ok": False, "error": "Forbidden — user can't access that URL"}
|
return {"ok": False, "error": "Forbidden — user can't access that URL"}
|
||||||
if r.status_code == 404:
|
if r.status_code == 404:
|
||||||
return {"ok": False, "error": "Not found — check the URL path"}
|
return {"ok": False, "error": "Not found — check the URL path"}
|
||||||
|
if 300 <= r.status_code < 400:
|
||||||
|
return {"ok": False, "error": "Redirects are not followed for CalDAV safety; use the final URL"}
|
||||||
return {"ok": False, "error": f"HTTP {r.status_code}"}
|
return {"ok": False, "error": f"HTTP {r.status_code}"}
|
||||||
except httpx.ConnectError as e:
|
except httpx.ConnectError as e:
|
||||||
return {"ok": False, "error": f"Connection refused: {e}"[:200]}
|
return {"ok": False, "error": f"Connection refused: {e}"[:200]}
|
||||||
@@ -758,6 +787,16 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
)
|
)
|
||||||
db.add(ev)
|
db.add(ev)
|
||||||
db.commit()
|
db.commit()
|
||||||
|
if cal.source == "caldav":
|
||||||
|
# Push the new event to the remote so it appears on the user's
|
||||||
|
# other devices — the sync is otherwise pull-only (#800).
|
||||||
|
from src.caldav_writeback import writeback_event
|
||||||
|
await writeback_event(owner, cal.source, cal.id, {
|
||||||
|
"uid": uid, "summary": data.summary, "description": data.description,
|
||||||
|
"location": data.location, "dtstart": dtstart, "dtend": dtend,
|
||||||
|
"all_day": data.all_day, "is_utc": _is_utc and not data.all_day,
|
||||||
|
"rrule": data.rrule or "",
|
||||||
|
})
|
||||||
return {"ok": True, "uid": uid}
|
return {"ok": True, "uid": uid}
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
raise
|
raise
|
||||||
@@ -804,6 +843,14 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
if data.color is not None:
|
if data.color is not None:
|
||||||
ev.color = data.color if data.color else None
|
ev.color = data.color if data.color else None
|
||||||
db.commit()
|
db.commit()
|
||||||
|
cal = db.query(CalendarCal).filter(CalendarCal.id == ev.calendar_id).first()
|
||||||
|
if cal and cal.source == "caldav":
|
||||||
|
from src.caldav_writeback import writeback_event
|
||||||
|
await writeback_event(owner, cal.source, cal.id, {
|
||||||
|
"uid": ev.uid, "summary": ev.summary, "description": ev.description,
|
||||||
|
"location": ev.location, "dtstart": ev.dtstart, "dtend": ev.dtend,
|
||||||
|
"all_day": ev.all_day, "is_utc": ev.is_utc, "rrule": ev.rrule or "",
|
||||||
|
})
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
raise
|
raise
|
||||||
@@ -824,8 +871,15 @@ def setup_calendar_routes() -> APIRouter:
|
|||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
ev = _get_or_404_event(db, base_uid, owner)
|
ev = _get_or_404_event(db, base_uid, owner)
|
||||||
|
# Capture what the remote push needs BEFORE the row is gone.
|
||||||
|
_cal = db.query(CalendarCal).filter(CalendarCal.id == ev.calendar_id).first()
|
||||||
|
_is_caldav = bool(_cal and _cal.source == "caldav")
|
||||||
|
_cal_id, _ev_uid = ev.calendar_id, ev.uid
|
||||||
db.delete(ev)
|
db.delete(ev)
|
||||||
db.commit()
|
db.commit()
|
||||||
|
if _is_caldav:
|
||||||
|
from src.caldav_writeback import writeback_event
|
||||||
|
await writeback_event(owner, "caldav", _cal_id, {"uid": _ev_uid}, delete=True)
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
raise
|
raise
|
||||||
|
|||||||
+47
-15
@@ -148,8 +148,9 @@ async def auto_name_session(session_manager, sess):
|
|||||||
if not first_msg:
|
if not first_msg:
|
||||||
return
|
return
|
||||||
|
|
||||||
|
owner = getattr(sess, "owner", None)
|
||||||
t_url, t_model, t_headers = resolve_task_endpoint(
|
t_url, t_model, t_headers = resolve_task_endpoint(
|
||||||
sess.endpoint_url, sess.model, sess.headers,
|
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||||
)
|
)
|
||||||
if not t_model:
|
if not t_model:
|
||||||
logger.debug("[auto-name] No model provided, skipping")
|
logger.debug("[auto-name] No model provided, skipping")
|
||||||
@@ -311,7 +312,24 @@ def fire_message_event(request, webhook_manager, session_id: str, sess, message:
|
|||||||
fire_event("message_sent", user)
|
fire_event("message_sent", user)
|
||||||
|
|
||||||
|
|
||||||
def resolve_session_auth(sess, session_id: str):
|
def _session_url_matches_endpoint(session_url: str, endpoint_base: str) -> bool:
|
||||||
|
if not session_url or not endpoint_base:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||||
|
|
||||||
|
sess_url = session_url.rstrip("/")
|
||||||
|
base = normalize_base(endpoint_base).rstrip("/")
|
||||||
|
return sess_url in {
|
||||||
|
base,
|
||||||
|
base + "/chat/completions",
|
||||||
|
build_chat_url(base).rstrip("/"),
|
||||||
|
}
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_session_auth(sess, session_id: str, owner: Optional[str] = None):
|
||||||
"""Ensure session has auth headers — resolve from endpoint DB if missing."""
|
"""Ensure session has auth headers — resolve from endpoint DB if missing."""
|
||||||
has_auth = sess.headers and isinstance(sess.headers, dict) and any(
|
has_auth = sess.headers and isinstance(sess.headers, dict) and any(
|
||||||
k.lower() in ('authorization', 'x-api-key') for k in sess.headers
|
k.lower() in ('authorization', 'x-api-key') for k in sess.headers
|
||||||
@@ -320,19 +338,33 @@ def resolve_session_auth(sess, session_id: str):
|
|||||||
return
|
return
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from src.endpoint_resolver import build_headers
|
from src.endpoint_resolver import build_headers, normalize_base
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
domain = sess.endpoint_url.split("//")[1].split("/")[0] if "//" in sess.endpoint_url else ""
|
target_url = getattr(sess, "endpoint_url", "") or ""
|
||||||
if domain:
|
if not target_url:
|
||||||
ep = db.query(ModelEndpoint).filter(ModelEndpoint.base_url.contains(domain)).first()
|
return
|
||||||
if ep and ep.api_key:
|
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||||
sess.headers = build_headers(ep.api_key, ep.base_url)
|
if owner:
|
||||||
db.query(DBSession).filter(DBSession.id == session_id).update(
|
# Missing headers usually means "recover from the saved endpoint".
|
||||||
{"headers": json.dumps(sess.headers)}
|
# Scope that lookup to the session owner, otherwise two users
|
||||||
)
|
# with similar endpoint URLs can borrow each other's API key.
|
||||||
db.commit()
|
from src.auth_helpers import owner_filter
|
||||||
logger.info(f"Resolved and persisted auth headers for session {session_id} from endpoint {ep.name}")
|
q = owner_filter(q, ModelEndpoint, owner)
|
||||||
|
for ep in q.all():
|
||||||
|
if not _session_url_matches_endpoint(target_url, ep.base_url or ""):
|
||||||
|
continue
|
||||||
|
if not ep.api_key:
|
||||||
|
return
|
||||||
|
base = normalize_base(ep.base_url or "")
|
||||||
|
sess.headers = build_headers(ep.api_key, base)
|
||||||
|
update_q = db.query(DBSession).filter(DBSession.id == session_id)
|
||||||
|
if owner:
|
||||||
|
update_q = update_q.filter(DBSession.owner == owner)
|
||||||
|
update_q.update({"headers": sess.headers})
|
||||||
|
db.commit()
|
||||||
|
logger.info(f"Resolved and persisted auth headers for session {session_id} from endpoint {ep.name}")
|
||||||
|
return
|
||||||
finally:
|
finally:
|
||||||
db.close()
|
db.close()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -806,7 +838,7 @@ def run_post_response_tasks(
|
|||||||
from services.memory.memory_extractor import extract_and_store
|
from services.memory.memory_extractor import extract_and_store
|
||||||
from src.task_endpoint import resolve_task_endpoint
|
from src.task_endpoint import resolve_task_endpoint
|
||||||
t_url, t_model, t_headers = resolve_task_endpoint(
|
t_url, t_model, t_headers = resolve_task_endpoint(
|
||||||
sess.endpoint_url, sess.model, sess.headers,
|
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||||
)
|
)
|
||||||
asyncio.create_task(extract_and_store(
|
asyncio.create_task(extract_and_store(
|
||||||
sess, memory_manager, memory_vector,
|
sess, memory_manager, memory_vector,
|
||||||
@@ -843,7 +875,7 @@ def run_post_response_tasks(
|
|||||||
from services.memory.skill_extractor import maybe_extract_skill
|
from services.memory.skill_extractor import maybe_extract_skill
|
||||||
from src.task_endpoint import resolve_task_endpoint
|
from src.task_endpoint import resolve_task_endpoint
|
||||||
s_url, s_model, s_headers = resolve_task_endpoint(
|
s_url, s_model, s_headers = resolve_task_endpoint(
|
||||||
sess.endpoint_url, sess.model, sess.headers,
|
sess.endpoint_url, sess.model, sess.headers, owner=owner,
|
||||||
)
|
)
|
||||||
logger.debug("[skill-extract] dispatching extractor (model=%s)", s_model)
|
logger.debug("[skill-extract] dispatching extractor (model=%s)", s_model)
|
||||||
asyncio.create_task(maybe_extract_skill(
|
asyncio.create_task(maybe_extract_skill(
|
||||||
|
|||||||
+29
-14
@@ -72,13 +72,17 @@ def _session_url_matches_endpoint(session_url: str, endpoint_base: str) -> bool:
|
|||||||
return sess in variants or sess.startswith(base + "/")
|
return sess in variants or sess.startswith(base + "/")
|
||||||
|
|
||||||
|
|
||||||
def _clear_orphaned_session_endpoint(sess) -> bool:
|
def _clear_orphaned_session_endpoint(sess, owner: str | None = None) -> bool:
|
||||||
"""Clear a session model if its endpoint was deleted from ModelEndpoint."""
|
"""Clear a session model if its endpoint was deleted from ModelEndpoint."""
|
||||||
if not getattr(sess, "endpoint_url", ""):
|
if not getattr(sess, "endpoint_url", ""):
|
||||||
return False
|
return False
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||||
|
if owner:
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
q = owner_filter(q, ModelEndpoint, owner)
|
||||||
|
endpoints = q.all()
|
||||||
for ep in endpoints:
|
for ep in endpoints:
|
||||||
if _session_url_matches_endpoint(sess.endpoint_url or "", ep.base_url or ""):
|
if _session_url_matches_endpoint(sess.endpoint_url or "", ep.base_url or ""):
|
||||||
return False
|
return False
|
||||||
@@ -118,7 +122,7 @@ def _endpoint_cache_contains_model(endpoint, model: str) -> bool:
|
|||||||
return wanted in {str(item).strip() for item in models}
|
return wanted in {str(item).strip() for item in models}
|
||||||
|
|
||||||
|
|
||||||
def _is_image_generation_session(sess) -> bool:
|
def _is_image_generation_session(sess, owner: str | None = None) -> bool:
|
||||||
"""Whether this chat session should bypass text chat and generate images.
|
"""Whether this chat session should bypass text chat and generate images.
|
||||||
|
|
||||||
Model-name prefixes are explicit image models. Endpoint type is only used
|
Model-name prefixes are explicit image models. Endpoint type is only used
|
||||||
@@ -137,7 +141,11 @@ def _is_image_generation_session(sess) -> bool:
|
|||||||
|
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||||
|
if owner:
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
q = owner_filter(q, ModelEndpoint, owner)
|
||||||
|
endpoints = q.all()
|
||||||
for endpoint in endpoints:
|
for endpoint in endpoints:
|
||||||
if (getattr(endpoint, "model_type", None) or "llm") != "image":
|
if (getattr(endpoint, "model_type", None) or "llm") != "image":
|
||||||
continue
|
continue
|
||||||
@@ -152,7 +160,7 @@ def _is_image_generation_session(sess) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _recover_empty_session_model(sess, session_id: str) -> bool:
|
def _recover_empty_session_model(sess, session_id: str, owner: str | None = None) -> bool:
|
||||||
"""Re-populate sess.model from the matching endpoint's cached models.
|
"""Re-populate sess.model from the matching endpoint's cached models.
|
||||||
|
|
||||||
Covers the window between endpoint setup and the first chat send: the
|
Covers the window between endpoint setup and the first chat send: the
|
||||||
@@ -172,7 +180,11 @@ def _recover_empty_session_model(sess, session_id: str) -> bool:
|
|||||||
# cached model is the most defensible default.
|
# cached model is the most defensible default.
|
||||||
ep = None
|
ep = None
|
||||||
if getattr(sess, "endpoint_url", ""):
|
if getattr(sess, "endpoint_url", ""):
|
||||||
endpoints = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all()
|
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
|
||||||
|
if owner:
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
q = owner_filter(q, ModelEndpoint, owner)
|
||||||
|
endpoints = q.all()
|
||||||
for cand in endpoints:
|
for cand in endpoints:
|
||||||
if _session_url_matches_endpoint(sess.endpoint_url or "", cand.base_url or ""):
|
if _session_url_matches_endpoint(sess.endpoint_url or "", cand.base_url or ""):
|
||||||
ep = cand
|
ep = cand
|
||||||
@@ -251,13 +263,14 @@ def setup_chat_routes(
|
|||||||
sess = session_manager.get_session(session)
|
sess = session_manager.get_session(session)
|
||||||
except KeyError:
|
except KeyError:
|
||||||
raise HTTPException(404, f"Session '{session}' not found")
|
raise HTTPException(404, f"Session '{session}' not found")
|
||||||
if _clear_orphaned_session_endpoint(sess):
|
owner = get_current_user(request)
|
||||||
|
if _clear_orphaned_session_endpoint(sess, owner=owner):
|
||||||
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
||||||
|
|
||||||
# Empty model + live endpoint = setup race (Issue #587). Repair from
|
# Empty model + live endpoint = setup race (Issue #587). Repair from
|
||||||
# the endpoint's cached model list before privilege checks, which
|
# the endpoint's cached model list before privilege checks, which
|
||||||
# otherwise see "" and behave inconsistently with the allowlist.
|
# otherwise see "" and behave inconsistently with the allowlist.
|
||||||
_recover_empty_session_model(sess, session)
|
_recover_empty_session_model(sess, session, owner=owner)
|
||||||
if not getattr(sess, "model", "").strip():
|
if not getattr(sess, "model", "").strip():
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
400,
|
400,
|
||||||
@@ -401,7 +414,8 @@ def setup_chat_routes(
|
|||||||
# but BEFORE loading. Prevents cross-user session hijack.
|
# but BEFORE loading. Prevents cross-user session hijack.
|
||||||
_verify_session_owner(request, session)
|
_verify_session_owner(request, session)
|
||||||
sess = session_manager.get_session(session)
|
sess = session_manager.get_session(session)
|
||||||
if _clear_orphaned_session_endpoint(sess):
|
owner = get_current_user(request)
|
||||||
|
if _clear_orphaned_session_endpoint(sess, owner=owner):
|
||||||
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
|
||||||
# Issue #587: picker shows a model from the endpoint cache but
|
# Issue #587: picker shows a model from the endpoint cache but
|
||||||
# s.model never made it onto the DB row (first-send race after
|
# s.model never made it onto the DB row (first-send race after
|
||||||
@@ -409,7 +423,7 @@ def setup_chat_routes(
|
|||||||
# the first cached model off the matching endpoint so the
|
# the first cached model off the matching endpoint so the
|
||||||
# upstream isn't called with model="" (which surfaces as a
|
# upstream isn't called with model="" (which surfaces as a
|
||||||
# generic 401/503).
|
# generic 401/503).
|
||||||
_recover_empty_session_model(sess, session)
|
_recover_empty_session_model(sess, session, owner=owner)
|
||||||
if not getattr(sess, "model", "").strip():
|
if not getattr(sess, "model", "").strip():
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
400,
|
400,
|
||||||
@@ -431,7 +445,7 @@ def setup_chat_routes(
|
|||||||
_enforce_chat_privileges(request, sess)
|
_enforce_chat_privileges(request, sess)
|
||||||
|
|
||||||
# Ensure session has auth headers
|
# Ensure session has auth headers
|
||||||
resolve_session_auth(sess, session)
|
resolve_session_auth(sess, session, owner=get_current_user(request))
|
||||||
|
|
||||||
# Check for research_pending BEFORE mode persist overwrites it
|
# Check for research_pending BEFORE mode persist overwrites it
|
||||||
do_research = str(use_research).lower() == "true"
|
do_research = str(use_research).lower() == "true"
|
||||||
@@ -711,6 +725,7 @@ def setup_chat_routes(
|
|||||||
prior_findings=_prior_findings,
|
prior_findings=_prior_findings,
|
||||||
prior_urls=_prior_urls,
|
prior_urls=_prior_urls,
|
||||||
on_complete=_on_research_done,
|
on_complete=_on_research_done,
|
||||||
|
owner=_user,
|
||||||
)
|
)
|
||||||
|
|
||||||
_heartbeat_counter = 0
|
_heartbeat_counter = 0
|
||||||
@@ -767,7 +782,7 @@ def setup_chat_routes(
|
|||||||
# output. Resolved once per request.
|
# output. Resolved once per request.
|
||||||
try:
|
try:
|
||||||
from src.endpoint_resolver import resolve_chat_fallback_candidates
|
from src.endpoint_resolver import resolve_chat_fallback_candidates
|
||||||
_fallback_candidates = resolve_chat_fallback_candidates()
|
_fallback_candidates = resolve_chat_fallback_candidates(owner=_user)
|
||||||
except Exception:
|
except Exception:
|
||||||
_fallback_candidates = []
|
_fallback_candidates = []
|
||||||
|
|
||||||
@@ -780,7 +795,7 @@ def setup_chat_routes(
|
|||||||
_model_info["character_name"] = ctx.preset.character_name
|
_model_info["character_name"] = ctx.preset.character_name
|
||||||
yield f'data: {json.dumps(_model_info)}\n\n'
|
yield f'data: {json.dumps(_model_info)}\n\n'
|
||||||
|
|
||||||
if _is_image_generation_session(sess):
|
if _is_image_generation_session(sess, owner=_user):
|
||||||
from src.settings import get_setting
|
from src.settings import get_setting
|
||||||
if not get_setting("image_gen_enabled", True):
|
if not get_setting("image_gen_enabled", True):
|
||||||
yield f'data: {json.dumps({"delta": "Image generation is disabled by the administrator."})}\n\n'
|
yield f'data: {json.dumps({"delta": "Image generation is disabled by the administrator."})}\n\n'
|
||||||
@@ -791,7 +806,7 @@ def setup_chat_routes(
|
|||||||
_user_msg = message or ""
|
_user_msg = message or ""
|
||||||
yield f'data: {json.dumps({"type": "tool_start", "tool": "generate_image", "command": _user_msg[:100]})}\n\n'
|
yield f'data: {json.dumps({"type": "tool_start", "tool": "generate_image", "command": _user_msg[:100]})}\n\n'
|
||||||
yield ": heartbeat\n\n"
|
yield ": heartbeat\n\n"
|
||||||
_img_result = await do_generate_image(f"{_user_msg}\n{sess.model}", session)
|
_img_result = await do_generate_image(f"{_user_msg}\n{sess.model}", session, owner=_user)
|
||||||
_img_output = _img_result.get("results", _img_result.get("error", ""))
|
_img_output = _img_result.get("results", _img_result.get("error", ""))
|
||||||
_img_tool_data = {"type": "tool_output", "tool": "generate_image", "command": _user_msg[:100], "output": _img_output, "exit_code": 0 if "error" not in _img_result else 1}
|
_img_tool_data = {"type": "tool_output", "tool": "generate_image", "command": _user_msg[:100], "output": _img_output, "exit_code": 0 if "error" not in _img_result else 1}
|
||||||
for _k in ("image_url", "image_id", "image_prompt", "image_model", "image_size", "image_quality"):
|
for _k in ("image_url", "image_id", "image_prompt", "image_model", "image_size", "image_quality"):
|
||||||
|
|||||||
@@ -541,21 +541,15 @@ def _append_llama_cpp_linux_accel_build_lines(runner_lines: list[str]) -> None:
|
|||||||
runner_lines.append(' export HIP_PATH="${HIP_PATH:-$(hipconfig -R)}"')
|
runner_lines.append(' export HIP_PATH="${HIP_PATH:-$(hipconfig -R)}"')
|
||||||
runner_lines.append(' fi')
|
runner_lines.append(' fi')
|
||||||
runner_lines.append(' echo "[odysseus] ROCm/HIP detected — building llama-server with HIP support..."')
|
runner_lines.append(' echo "[odysseus] ROCm/HIP detected — building llama-server with HIP support..."')
|
||||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON \\\\')
|
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_HIP=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
|
||||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
|
||||||
runner_lines.append(' elif command -v nvcc &>/dev/null; then')
|
runner_lines.append(' elif command -v nvcc &>/dev/null; then')
|
||||||
runner_lines.append(' echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
|
runner_lines.append(' echo "[odysseus] CUDA nvcc found — building llama-server with CUDA (GPU) support..."')
|
||||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \\\\')
|
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
|
||||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
|
||||||
runner_lines.append(' else')
|
runner_lines.append(' else')
|
||||||
runner_lines.append(' echo "[odysseus] WARNING: no HIP/CUDA toolchain found — building llama-server for CPU only."')
|
runner_lines.append(' echo "[odysseus] WARNING: no HIP/CUDA toolchain found — building llama-server for CPU only."')
|
||||||
runner_lines.append(' echo "[odysseus] GPU inference will not be available for this llama.cpp build."')
|
runner_lines.append(' echo "[odysseus] GPU inference will not be available for this llama.cpp build."')
|
||||||
runner_lines.append(' echo "[odysseus] Install ROCm for AMD GPUs or vLLM/CUDA tooling for NVIDIA, then re-launch this serve task."')
|
runner_lines.append(' echo "[odysseus] Install ROCm for AMD GPUs or vLLM/CUDA tooling for NVIDIA, then re-launch this serve task."')
|
||||||
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release \\\\')
|
runner_lines.append(' cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j"$NPROC" --target llama-server && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
||||||
runner_lines.append(' && cmake --build build -j"$NPROC" --target llama-server \\\\')
|
|
||||||
runner_lines.append(' && ln -sf ~/llama.cpp/build/bin/llama-server ~/bin/llama-server')
|
|
||||||
runner_lines.append(' fi')
|
runner_lines.append(' fi')
|
||||||
|
|
||||||
class ModelDownloadRequest(BaseModel):
|
class ModelDownloadRequest(BaseModel):
|
||||||
|
|||||||
@@ -383,11 +383,15 @@ def setup_cookbook_routes() -> APIRouter:
|
|||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
argv = [os.environ.get("ComSpec", "cmd.exe"), "/c", str(script_path)]
|
argv = [os.environ.get("ComSpec", "cmd.exe"), "/c", str(script_path)]
|
||||||
|
env = os.environ.copy()
|
||||||
|
env["PYTHONUTF8"] = "1"
|
||||||
|
env["PYTHONIOENCODING"] = "utf-8"
|
||||||
proc = subprocess.Popen(
|
proc = subprocess.Popen(
|
||||||
argv,
|
argv,
|
||||||
stdout=subprocess.DEVNULL,
|
stdout=subprocess.DEVNULL,
|
||||||
stderr=subprocess.DEVNULL,
|
stderr=subprocess.DEVNULL,
|
||||||
stdin=subprocess.DEVNULL,
|
stdin=subprocess.DEVNULL,
|
||||||
|
env=env,
|
||||||
**detached_popen_kwargs(),
|
**detached_popen_kwargs(),
|
||||||
)
|
)
|
||||||
pid_path.write_text(str(proc.pid), encoding="utf-8")
|
pid_path.write_text(str(proc.pid), encoding="utf-8")
|
||||||
@@ -580,7 +584,7 @@ def setup_cookbook_routes() -> APIRouter:
|
|||||||
runner_lines.append(" python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
|
runner_lines.append(" python3 -c 'import hf_transfer' 2>/dev/null && export HF_HUB_ENABLE_HF_TRANSFER=1")
|
||||||
runner_lines.append(f' python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers={4 if req.disable_hf_transfer else 8})"')
|
runner_lines.append(f' python3 -c "import os; from huggingface_hub import snapshot_download; snapshot_download(\'{req.repo_id}\'{_dl_pyarg}, max_workers={4 if req.disable_hf_transfer else 8})"')
|
||||||
runner_lines.append('fi')
|
runner_lines.append('fi')
|
||||||
runner_lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
runner_lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||||
runner_lines.append(f"rm -f {remote_runner}")
|
runner_lines.append(f"rm -f {remote_runner}")
|
||||||
runner_lines.append('exec "${SHELL:-/bin/bash}"')
|
runner_lines.append('exec "${SHELL:-/bin/bash}"')
|
||||||
runner_path = TMUX_LOG_DIR / f"{session_id}_run.sh"
|
runner_path = TMUX_LOG_DIR / f"{session_id}_run.sh"
|
||||||
@@ -611,11 +615,11 @@ def setup_cookbook_routes() -> APIRouter:
|
|||||||
# Detached path: no controlling TTY, so skip `< /dev/null`
|
# Detached path: no controlling TTY, so skip `< /dev/null`
|
||||||
# (handled by Popen stdin=DEVNULL) and don't keep a shell open.
|
# (handled by Popen stdin=DEVNULL) and don't keep a shell open.
|
||||||
lines.append(hf_cmd)
|
lines.append(hf_cmd)
|
||||||
lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||||
else:
|
else:
|
||||||
# < /dev/null suppresses interactive "update available? [Y/n]" prompt
|
# < /dev/null suppresses interactive "update available? [Y/n]" prompt
|
||||||
lines.append(f"{hf_cmd} < /dev/null")
|
lines.append(f"{hf_cmd} < /dev/null")
|
||||||
lines.append('if [ $? -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $?)"; fi')
|
lines.append('_ec=$?; if [ $_ec -eq 0 ]; then echo ""; echo "DOWNLOAD_OK"; else echo ""; echo "DOWNLOAD_FAILED (exit $_ec)"; fi')
|
||||||
lines.append(f"rm -f '{wrapper_script}'")
|
lines.append(f"rm -f '{wrapper_script}'")
|
||||||
lines.append('exec "${SHELL:-/bin/bash}"')
|
lines.append('exec "${SHELL:-/bin/bash}"')
|
||||||
wrapper_script.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
wrapper_script.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
|||||||
@@ -593,6 +593,15 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
|||||||
if req.session_id is not None:
|
if req.session_id is not None:
|
||||||
# Empty string = unlink from session
|
# Empty string = unlink from session
|
||||||
doc.session_id = req.session_id if req.session_id else None
|
doc.session_id = req.session_id if req.session_id else None
|
||||||
|
if not req.session_id:
|
||||||
|
# Tab closed / doc detached from its session — drop the
|
||||||
|
# in-memory active-doc pointer so the last-resort injection
|
||||||
|
# path doesn't re-surface this doc in a later chat (#1160).
|
||||||
|
try:
|
||||||
|
from src.tool_implementations import clear_active_document
|
||||||
|
clear_active_document(doc_id)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
db.commit()
|
db.commit()
|
||||||
db.refresh(doc)
|
db.refresh(doc)
|
||||||
return _doc_to_dict(doc)
|
return _doc_to_dict(doc)
|
||||||
@@ -615,6 +624,13 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
|||||||
raise HTTPException(404, "Document not found")
|
raise HTTPException(404, "Document not found")
|
||||||
_verify_doc_owner(db, doc, user)
|
_verify_doc_owner(db, doc, user)
|
||||||
doc.is_active = False
|
doc.is_active = False
|
||||||
|
# Closed/deleted — drop the in-memory active-doc pointer so it isn't
|
||||||
|
# re-injected into a later, unrelated chat (#1160).
|
||||||
|
try:
|
||||||
|
from src.tool_implementations import clear_active_document
|
||||||
|
clear_active_document(doc_id)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
db.commit()
|
db.commit()
|
||||||
return {"status": "deleted", "id": doc_id}
|
return {"status": "deleted", "id": doc_id}
|
||||||
except HTTPException:
|
except HTTPException:
|
||||||
@@ -885,7 +901,7 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
|
|||||||
for i, doc in enumerate(batch):
|
for i, doc in enumerate(batch):
|
||||||
if i >= len(verdicts):
|
if i >= len(verdicts):
|
||||||
break
|
break
|
||||||
verdict = verdicts[i].lower().strip()
|
verdict = str(verdicts[i] or "").lower().strip()
|
||||||
if verdict == "junk":
|
if verdict == "junk":
|
||||||
doc.tidy_verdict = "junk"
|
doc.tidy_verdict = "junk"
|
||||||
db.delete(doc)
|
db.delete(doc)
|
||||||
|
|||||||
+79
-21
@@ -297,7 +297,8 @@ def _init_scheduled_db():
|
|||||||
send_at TEXT NOT NULL,
|
send_at TEXT NOT NULL,
|
||||||
created_at TEXT NOT NULL,
|
created_at TEXT NOT NULL,
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
status TEXT NOT NULL DEFAULT 'pending',
|
||||||
error TEXT
|
error TEXT,
|
||||||
|
owner TEXT DEFAULT ''
|
||||||
)
|
)
|
||||||
""")
|
""")
|
||||||
# Email summary cache (keyed by Message-ID)
|
# Email summary cache (keyed by Message-ID)
|
||||||
@@ -435,6 +436,35 @@ def _init_scheduled_db():
|
|||||||
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN account_id TEXT")
|
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN account_id TEXT")
|
||||||
if "odysseus_kind" not in cols:
|
if "odysseus_kind" not in cols:
|
||||||
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN odysseus_kind TEXT")
|
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN odysseus_kind TEXT")
|
||||||
|
if "owner" not in cols:
|
||||||
|
conn.execute("ALTER TABLE scheduled_emails ADD COLUMN owner TEXT DEFAULT ''")
|
||||||
|
conn.execute("CREATE INDEX IF NOT EXISTS ix_scheduled_emails_owner_status ON scheduled_emails(owner, status)")
|
||||||
|
# Backfill owner on legacy rows from the owning email account so the
|
||||||
|
# owner-scoped list/cancel routes surface pre-migration scheduled
|
||||||
|
# sends to the right user (the poller already resolves these by
|
||||||
|
# account at send time; this aligns the UI with that).
|
||||||
|
legacy_accounts = conn.execute(
|
||||||
|
"SELECT DISTINCT account_id FROM scheduled_emails "
|
||||||
|
"WHERE (owner IS NULL OR owner = '') AND account_id IS NOT NULL AND account_id != ''"
|
||||||
|
).fetchall()
|
||||||
|
if legacy_accounts:
|
||||||
|
try:
|
||||||
|
from core.database import SessionLocal as _SL, EmailAccount as _EA
|
||||||
|
_db = _SL()
|
||||||
|
try:
|
||||||
|
for (acct_id,) in legacy_accounts:
|
||||||
|
row = _db.query(_EA.owner).filter(_EA.id == acct_id).first()
|
||||||
|
acct_owner = (row[0] or "") if row else ""
|
||||||
|
if acct_owner:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE scheduled_emails SET owner = ? "
|
||||||
|
"WHERE account_id = ? AND (owner IS NULL OR owner = '')",
|
||||||
|
(acct_owner, acct_id),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
_db.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
# Lazy migration: add turns_json to email_boundaries for server-side
|
# Lazy migration: add turns_json to email_boundaries for server-side
|
||||||
@@ -815,22 +845,27 @@ def _detect_spam_folder(conn):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def _imap_move(uid, dest, src="INBOX"):
|
def _imap_move(uid, dest, src="INBOX", account_id: str | None = None, owner: str = ""):
|
||||||
"""Move a single IMAP UID from src folder to dest. Returns True on success."""
|
"""Move a single IMAP UID from src folder to dest. Returns True on success."""
|
||||||
|
c = None
|
||||||
try:
|
try:
|
||||||
c = _imap_connect()
|
c = _imap_connect(account_id, owner=owner)
|
||||||
c.select(_q(src))
|
c.select(_q(src))
|
||||||
status, _ = c.copy(uid, _q(dest))
|
status, _ = c.copy(uid, _q(dest))
|
||||||
if status != "OK":
|
if status != "OK":
|
||||||
c.logout()
|
|
||||||
return False
|
return False
|
||||||
c.store(uid, "+FLAGS", "\\Deleted")
|
c.store(uid, "+FLAGS", "\\Deleted")
|
||||||
c.expunge()
|
c.expunge()
|
||||||
c.logout()
|
|
||||||
return True
|
return True
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"IMAP move {uid} → {dest} failed: {e}")
|
logger.warning(f"IMAP move {uid} → {dest} failed: {e}")
|
||||||
return False
|
return False
|
||||||
|
finally:
|
||||||
|
if c:
|
||||||
|
try:
|
||||||
|
c.logout()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
def _extract_attachment_text(msg, max_chars: int = 6000) -> str:
|
def _extract_attachment_text(msg, max_chars: int = 6000) -> str:
|
||||||
@@ -1021,7 +1056,9 @@ def _fetch_sender_thread_context(sender_addr: str,
|
|||||||
exclude_folder: str = "INBOX",
|
exclude_folder: str = "INBOX",
|
||||||
limit: int = 3,
|
limit: int = 3,
|
||||||
max_chars_per_email: int = 1500,
|
max_chars_per_email: int = 1500,
|
||||||
max_attachment_chars: int = 4000) -> str:
|
max_attachment_chars: int = 4000,
|
||||||
|
account_id: str | None = None,
|
||||||
|
owner: str = "") -> str:
|
||||||
"""Pull the last N emails from `sender_addr` (across common folders),
|
"""Pull the last N emails from `sender_addr` (across common folders),
|
||||||
extract their body snippets + attachment text, and return one formatted
|
extract their body snippets + attachment text, and return one formatted
|
||||||
block ready to be glued into an LLM system prompt as "REFERENCED MATERIAL".
|
block ready to be glued into an LLM system prompt as "REFERENCED MATERIAL".
|
||||||
@@ -1043,7 +1080,7 @@ def _fetch_sender_thread_context(sender_addr: str,
|
|||||||
seen_uids.add((exclude_folder or "INBOX", str(exclude_uid)))
|
seen_uids.add((exclude_folder or "INBOX", str(exclude_uid)))
|
||||||
|
|
||||||
try:
|
try:
|
||||||
conn = _imap_connect()
|
conn = _imap_connect(account_id, owner=owner)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"sender-thread-context: imap connect failed: {e}")
|
logger.warning(f"sender-thread-context: imap connect failed: {e}")
|
||||||
return ""
|
return ""
|
||||||
@@ -1126,7 +1163,12 @@ def _fetch_sender_thread_context(sender_addr: str,
|
|||||||
return "\n\n=====\n\n".join(blocks)
|
return "\n\n=====\n\n".join(blocks)
|
||||||
|
|
||||||
|
|
||||||
def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
def _pre_retrieve_context(
|
||||||
|
body: str,
|
||||||
|
sender: str,
|
||||||
|
account_id: str | None = None,
|
||||||
|
owner: str = "",
|
||||||
|
) -> tuple:
|
||||||
"""Extract key terms from an incoming email and search past emails + contacts.
|
"""Extract key terms from an incoming email and search past emails + contacts.
|
||||||
|
|
||||||
Returns (context_snippets, terms_list). Best-effort; never raises.
|
Returns (context_snippets, terms_list). Best-effort; never raises.
|
||||||
@@ -1150,21 +1192,37 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
|||||||
# ── Known-sender check: only retrieve context for senders we already
|
# ── Known-sender check: only retrieve context for senders we already
|
||||||
# have a relationship with. New / cold senders get an empty context.
|
# have a relationship with. New / cold senders get an empty context.
|
||||||
sender_addr = email.utils.parseaddr(sender or "")[1].lower()
|
sender_addr = email.utils.parseaddr(sender or "")[1].lower()
|
||||||
is_known = False
|
# The CardDAV address book is global admin data backed by a single
|
||||||
|
# Radicale instance, so only fold it into reply context for an admin /
|
||||||
|
# single-user owner. Non-admin owners still get their own (owner-scoped)
|
||||||
|
# IMAP history below, just not the shared contacts.
|
||||||
try:
|
try:
|
||||||
from routes.contacts_routes import _fetch_contacts
|
from src.tool_security import owner_is_admin_or_single_user
|
||||||
for c in _fetch_contacts() or []:
|
contacts_allowed = owner_is_admin_or_single_user(owner or None)
|
||||||
# Contacts are normalized to plural `emails` lists (see
|
|
||||||
# contacts_routes._normalize_contact); the old `c.get("email")`
|
|
||||||
# singular key never exists, so known senders were never matched.
|
|
||||||
if sender_addr in [(e or "").lower() for e in (c.get("emails") or [])]:
|
|
||||||
is_known = True
|
|
||||||
break
|
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
contacts_allowed = not bool(owner)
|
||||||
|
is_known = False
|
||||||
|
if contacts_allowed:
|
||||||
|
try:
|
||||||
|
from routes.contacts_routes import _fetch_contacts
|
||||||
|
for c in _fetch_contacts() or []:
|
||||||
|
# Contacts are normalized to plural `emails` lists, but
|
||||||
|
# keep the legacy singular key fallback for older data.
|
||||||
|
contact_emails = []
|
||||||
|
raw_emails = c.get("emails")
|
||||||
|
if isinstance(raw_emails, list):
|
||||||
|
contact_emails.extend(str(e or "") for e in raw_emails)
|
||||||
|
legacy_email = c.get("email")
|
||||||
|
if legacy_email:
|
||||||
|
contact_emails.append(str(legacy_email))
|
||||||
|
if any((addr or "").strip().lower() == sender_addr for addr in contact_emails):
|
||||||
|
is_known = True
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
if not is_known and sender_addr:
|
if not is_known and sender_addr:
|
||||||
try:
|
try:
|
||||||
with _imap() as _ck:
|
with _imap(account_id, owner=owner) as _ck:
|
||||||
_ck.select("INBOX", readonly=True)
|
_ck.select("INBOX", readonly=True)
|
||||||
st_known, dk = _ck.search(None, f'(FROM "{sender_addr}")')
|
st_known, dk = _ck.search(None, f'(FROM "{sender_addr}")')
|
||||||
if st_known == "OK" and dk and dk[0]:
|
if st_known == "OK" and dk and dk[0]:
|
||||||
@@ -1202,7 +1260,7 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
|||||||
return context_snippets, terms_list
|
return context_snippets, terms_list
|
||||||
|
|
||||||
try:
|
try:
|
||||||
ctx_conn = _imap_connect()
|
ctx_conn = _imap_connect(account_id, owner=owner)
|
||||||
for folder in ["INBOX", "Sent", "Archive", "Drafts"]:
|
for folder in ["INBOX", "Sent", "Archive", "Drafts"]:
|
||||||
try:
|
try:
|
||||||
st_sel, _sd = ctx_conn.select(_q(folder), readonly=True)
|
st_sel, _sd = ctx_conn.select(_q(folder), readonly=True)
|
||||||
@@ -1246,7 +1304,7 @@ def _pre_retrieve_context(body: str, sender: str) -> tuple:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
from routes.contacts_routes import _fetch_contacts
|
from routes.contacts_routes import _fetch_contacts
|
||||||
all_contacts = _fetch_contacts()
|
all_contacts = _fetch_contacts() if contacts_allowed else []
|
||||||
for term in terms_list:
|
for term in terms_list:
|
||||||
t_lower = term.lower()
|
t_lower = term.lower()
|
||||||
matches = [c for c in all_contacts
|
matches = [c for c in all_contacts
|
||||||
|
|||||||
+55
-36
@@ -45,6 +45,21 @@ from routes.email_helpers import (
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _owner_for_email_account(account_id: str | None) -> str:
|
||||||
|
if not account_id:
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
from core.database import SessionLocal as _SL, EmailAccount as _EA
|
||||||
|
db = _SL()
|
||||||
|
try:
|
||||||
|
row = db.query(_EA.owner).filter(_EA.id == account_id).first()
|
||||||
|
return (row[0] or "") if row else ""
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
except Exception:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
# ── Routes ──
|
# ── Routes ──
|
||||||
|
|
||||||
async def _emit_progress(progress_cb, message: str):
|
async def _emit_progress(progress_cb, message: str):
|
||||||
@@ -143,25 +158,18 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
if not auto_sum and not auto_reply and not auto_tag and not auto_spam and not auto_cal:
|
if not auto_sum and not auto_reply and not auto_tag and not auto_spam and not auto_cal:
|
||||||
return "Nothing to do"
|
return "Nothing to do"
|
||||||
|
|
||||||
# Owner of the account being processed. All calendar reads/writes below are
|
# Owner of the account being processed. All calendar + mailbox reads/writes
|
||||||
# scoped to this user: the multi-account fan-out runs every user's mailbox,
|
# below are scoped to this user: the multi-account fan-out runs every user's
|
||||||
# so an unscoped pass would disclose and mutate other tenants' calendars.
|
# mailbox, so an unscoped pass would disclose/mutate other tenants' data.
|
||||||
_acct_owner = None
|
# One resolution feeds both the mailbox path (account_owner) and upstream's
|
||||||
try:
|
# calendar path (_acct_owner, which expects None rather than "").
|
||||||
from core.database import SessionLocal as _SLo, EmailAccount as _EAo
|
account_owner = _owner_for_email_account(account_id)
|
||||||
_dbo = _SLo()
|
_acct_owner = account_owner or None
|
||||||
try:
|
|
||||||
if account_id:
|
|
||||||
_arow = _dbo.query(_EAo).filter(_EAo.id == account_id).first()
|
|
||||||
_acct_owner = _arow.owner if _arow else None
|
|
||||||
finally:
|
|
||||||
_dbo.close()
|
|
||||||
except Exception:
|
|
||||||
_acct_owner = None
|
|
||||||
|
|
||||||
|
conn = None
|
||||||
try:
|
try:
|
||||||
await _emit_progress(progress_cb, "Connecting to mail…")
|
await _emit_progress(progress_cb, "Connecting to mail…")
|
||||||
conn = _imap_connect(account_id)
|
conn = _imap_connect(account_id, owner=account_owner)
|
||||||
from datetime import timedelta as _td
|
from datetime import timedelta as _td
|
||||||
since = (datetime.utcnow() - _td(days=max(1, days_back))).strftime("%d-%b-%Y")
|
since = (datetime.utcnow() - _td(days=max(1, days_back))).strftime("%d-%b-%Y")
|
||||||
# uid_list carries real IMAP UIDs, matching the email UI/read routes.
|
# uid_list carries real IMAP UIDs, matching the email UI/read routes.
|
||||||
@@ -205,14 +213,19 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
# Re-select INBOX as default for downstream code
|
# Re-select INBOX as default for downstream code
|
||||||
conn.select("INBOX", readonly=True)
|
conn.select("INBOX", readonly=True)
|
||||||
if not uid_list:
|
if not uid_list:
|
||||||
conn.logout()
|
|
||||||
return "No recent emails"
|
return "No recent emails"
|
||||||
await _emit_progress(progress_cb, f"Found {len(uid_list)} recent email(s); checking cache…")
|
await _emit_progress(progress_cb, f"Found {len(uid_list)} recent email(s); checking cache…")
|
||||||
|
|
||||||
_c = _sql3.connect(SCHEDULED_DB)
|
_c = _sql3.connect(SCHEDULED_DB)
|
||||||
_sum_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_summaries").fetchall()}
|
_sum_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_summaries").fetchall()}
|
||||||
_reply_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_ai_replies").fetchall()}
|
_reply_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_ai_replies").fetchall()}
|
||||||
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags").fetchall()} if (auto_tag or auto_spam) else set()
|
if auto_tag or auto_spam:
|
||||||
|
if account_owner:
|
||||||
|
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags WHERE owner=?", (account_owner,)).fetchall()}
|
||||||
|
else:
|
||||||
|
_tag_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_tags WHERE owner='' OR owner IS NULL").fetchall()}
|
||||||
|
else:
|
||||||
|
_tag_existing = set()
|
||||||
_cal_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_calendar_extractions").fetchall()} if auto_cal else set()
|
_cal_existing = {r[0] for r in _c.execute("SELECT message_id FROM email_calendar_extractions").fetchall()} if auto_cal else set()
|
||||||
# Urgency is handled by the built-in `check_email_urgency` task. Keep
|
# Urgency is handled by the built-in `check_email_urgency` task. Keep
|
||||||
# this legacy poller path disabled so users don't get two independent
|
# this legacy poller path disabled so users don't get two independent
|
||||||
@@ -225,7 +238,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
# this per-iteration was making big inbox scans crawl. Used by the
|
# this per-iteration was making big inbox scans crawl. Used by the
|
||||||
# urgency self-loop check below.
|
# urgency self-loop check below.
|
||||||
try:
|
try:
|
||||||
_self_self_addr = (_get_email_config(account_id).get("from_address") or "").strip().lower()
|
_self_self_addr = (_get_email_config(account_id, owner=account_owner).get("from_address") or "").strip().lower()
|
||||||
except Exception:
|
except Exception:
|
||||||
_self_self_addr = ""
|
_self_self_addr = ""
|
||||||
|
|
||||||
@@ -233,11 +246,10 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
if auto_spam and not spam_folder:
|
if auto_spam and not spam_folder:
|
||||||
logger.warning("Auto-spam enabled but no Junk/Spam folder detected — will classify but not move")
|
logger.warning("Auto-spam enabled but no Junk/Spam folder detected — will classify but not move")
|
||||||
|
|
||||||
url, model, headers = resolve_endpoint("utility")
|
url, model, headers = resolve_endpoint("utility", owner=account_owner)
|
||||||
if not url:
|
if not url:
|
||||||
url, model, headers = resolve_endpoint("default")
|
url, model, headers = resolve_endpoint("default", owner=account_owner)
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
conn.logout()
|
|
||||||
return "No model configured"
|
return "No model configured"
|
||||||
|
|
||||||
writing_style = settings.get("email_writing_style", "")
|
writing_style = settings.get("email_writing_style", "")
|
||||||
@@ -395,8 +407,8 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
await _emit_progress(progress_cb, f"Drafting reply {processed + 1}/{_max_process} · checked {examined}/{len(uid_list)}")
|
await _emit_progress(progress_cb, f"Drafting reply {processed + 1}/{_max_process} · checked {examined}/{len(uid_list)}")
|
||||||
# Background reply drafting should not make the whole app
|
# Background reply drafting should not make the whole app
|
||||||
# feel busy. Keep it lightweight: no extra IMAP context
|
# feel busy. Keep it lightweight: no extra IMAP context
|
||||||
# mining here; manual AI Reply can still do that when the
|
# mining here; manual AI Reply can still do that (owner-scoped)
|
||||||
# user explicitly asks for a draft on one email.
|
# when the user explicitly asks for a draft on one email.
|
||||||
context_snippets, _terms = [], []
|
context_snippets, _terms = [], []
|
||||||
sys_prompt = _EMAIL_REPLY_SYS_PROMPT_BASE
|
sys_prompt = _EMAIL_REPLY_SYS_PROMPT_BASE
|
||||||
if att_text:
|
if att_text:
|
||||||
@@ -711,7 +723,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
# Send alert email immediately if critical or high
|
# Send alert email immediately if critical or high
|
||||||
if urgency in ("critical", "high"):
|
if urgency in ("critical", "high"):
|
||||||
try:
|
try:
|
||||||
cfg = _get_email_config(account_id)
|
cfg = _get_email_config(account_id, owner=account_owner)
|
||||||
to_addr = cfg["from_address"] # self-email
|
to_addr = cfg["from_address"] # self-email
|
||||||
|
|
||||||
# Deep-link to open the original email in Odysseus (if public URL is configured).
|
# Deep-link to open the original email in Odysseus (if public URL is configured).
|
||||||
@@ -719,8 +731,8 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
from src.settings import load_settings as _ls
|
from src.settings import load_settings as _ls
|
||||||
_pub = (_ls().get("app_public_url") or "").rstrip("/")
|
_pub = (_ls().get("app_public_url") or "").rstrip("/")
|
||||||
uid_str = uid.decode() if isinstance(uid, bytes) else str(uid)
|
uid_str = uid.decode() if isinstance(uid, bytes) else str(uid)
|
||||||
from urllib.parse import quote as _q
|
from urllib.parse import quote as _url_q
|
||||||
open_url = f"{_pub}/#email={_q(_folder, safe='')}:{uid_str}" if _pub else ""
|
open_url = f"{_pub}/#email={_url_q(_folder, safe='')}:{uid_str}" if _pub else ""
|
||||||
|
|
||||||
alert_subject = f"[{urgency.upper()}] {subject}"
|
alert_subject = f"[{urgency.upper()}] {subject}"
|
||||||
alert_body = (
|
alert_body = (
|
||||||
@@ -817,7 +829,7 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
_req.post, url, json=payload, headers=req_headers, timeout=120
|
_req.post, url, json=payload, headers=req_headers, timeout=120
|
||||||
)
|
)
|
||||||
if not resp.ok:
|
if not resp.ok:
|
||||||
logger.warning(f"Auto-classify {uid.decode()} HTTP {resp.status_code}: {resp.text[:200]}")
|
logger.warning(f"Auto-classify {uid.decode() if isinstance(uid, bytes) else str(uid)} HTTP {resp.status_code}: {resp.text[:200]}")
|
||||||
else:
|
else:
|
||||||
rdata = resp.json()
|
rdata = resp.json()
|
||||||
m = (rdata.get("choices") or [{}])[0].get("message", {})
|
m = (rdata.get("choices") or [{}])[0].get("message", {})
|
||||||
@@ -846,17 +858,17 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
|
|
||||||
moved_to = ""
|
moved_to = ""
|
||||||
if is_spam and auto_spam and spam_folder:
|
if is_spam and auto_spam and spam_folder:
|
||||||
if _imap_move(uid, spam_folder):
|
if _imap_move(uid, spam_folder, account_id=account_id, owner=account_owner):
|
||||||
moved_to = spam_folder
|
moved_to = spam_folder
|
||||||
logger.info(f"Auto-spam moved uid={uid.decode()} to {spam_folder}: {spam_reason}")
|
logger.info(f"Auto-spam moved uid={uid.decode()} to {spam_folder}: {spam_reason}")
|
||||||
|
|
||||||
_c = _sql3.connect(SCHEDULED_DB)
|
_c = _sql3.connect(SCHEDULED_DB)
|
||||||
_c.execute("""
|
_c.execute("""
|
||||||
INSERT OR REPLACE INTO email_tags
|
INSERT OR REPLACE INTO email_tags
|
||||||
(message_id, uid, folder, subject, sender, tags, spam_verdict,
|
(message_id, owner, uid, folder, subject, sender, tags, spam_verdict,
|
||||||
spam_reason, moved_to, model_used, created_at)
|
spam_reason, moved_to, model_used, created_at)
|
||||||
VALUES (?, ?, 'INBOX', ?, ?, ?, ?, ?, ?, ?, ?)
|
VALUES (?, ?, ?, 'INBOX', ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
""", (message_id, uid.decode(), subject, sender,
|
""", (message_id, account_owner or "", uid.decode(), subject, sender,
|
||||||
json.dumps(tags), 1 if is_spam else 0,
|
json.dumps(tags), 1 if is_spam else 0,
|
||||||
spam_reason, moved_to, model, datetime.utcnow().isoformat()))
|
spam_reason, moved_to, model, datetime.utcnow().isoformat()))
|
||||||
_c.commit()
|
_c.commit()
|
||||||
@@ -871,7 +883,6 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
logger.warning(f"Auto-process {uid} failed: {e}")
|
logger.warning(f"Auto-process {uid} failed: {e}")
|
||||||
continue
|
continue
|
||||||
|
|
||||||
conn.logout()
|
|
||||||
await _emit_progress(progress_cb, "Finishing…")
|
await _emit_progress(progress_cb, "Finishing…")
|
||||||
if processed > 0:
|
if processed > 0:
|
||||||
logger.info(f"Auto-processed {processed} new email(s) for summary/reply/classify")
|
logger.info(f"Auto-processed {processed} new email(s) for summary/reply/classify")
|
||||||
@@ -908,6 +919,12 @@ async def _auto_summarize_pass_single(days_back: int = 1, account_id: str | None
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"Auto-summarize pass error: {e}")
|
logger.warning(f"Auto-summarize pass error: {e}")
|
||||||
return f"Error: {e}"
|
return f"Error: {e}"
|
||||||
|
finally:
|
||||||
|
if conn:
|
||||||
|
try:
|
||||||
|
conn.logout()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
async def _auto_summarize_poller():
|
async def _auto_summarize_poller():
|
||||||
@@ -936,8 +953,9 @@ def _scheduled_poll_once() -> dict:
|
|||||||
conn = sqlite3.connect(SCHEDULED_DB)
|
conn = sqlite3.connect(SCHEDULED_DB)
|
||||||
cols = [row[1] for row in conn.execute("PRAGMA table_info(scheduled_emails)").fetchall()]
|
cols = [row[1] for row in conn.execute("PRAGMA table_info(scheduled_emails)").fetchall()]
|
||||||
kind_expr = "odysseus_kind" if "odysseus_kind" in cols else "'scheduled' AS odysseus_kind"
|
kind_expr = "odysseus_kind" if "odysseus_kind" in cols else "'scheduled' AS odysseus_kind"
|
||||||
|
owner_expr = "owner" if "owner" in cols else "'' AS owner"
|
||||||
rows = conn.execute(f"""
|
rows = conn.execute(f"""
|
||||||
SELECT id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, account_id, {kind_expr}
|
SELECT id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, account_id, {kind_expr}, {owner_expr}
|
||||||
FROM scheduled_emails
|
FROM scheduled_emails
|
||||||
WHERE status = 'pending' AND send_at <= ?
|
WHERE status = 'pending' AND send_at <= ?
|
||||||
""", (now_iso,)).fetchall()
|
""", (now_iso,)).fetchall()
|
||||||
@@ -949,7 +967,8 @@ def _scheduled_poll_once() -> dict:
|
|||||||
attachments = json.loads(r[8] or "[]")
|
attachments = json.loads(r[8] or "[]")
|
||||||
row_account_id = r[9] if len(r) > 9 else None
|
row_account_id = r[9] if len(r) > 9 else None
|
||||||
odysseus_kind = r[10] if len(r) > 10 else "scheduled"
|
odysseus_kind = r[10] if len(r) > 10 else "scheduled"
|
||||||
cfg = _get_email_config(row_account_id)
|
row_owner = (r[11] if len(r) > 11 else "") or _owner_for_email_account(row_account_id)
|
||||||
|
cfg = _get_email_config(row_account_id, owner=row_owner)
|
||||||
has_atts = bool(attachments)
|
has_atts = bool(attachments)
|
||||||
if has_atts:
|
if has_atts:
|
||||||
outer = MIMEMultipart("mixed")
|
outer = MIMEMultipart("mixed")
|
||||||
@@ -986,7 +1005,7 @@ def _scheduled_poll_once() -> dict:
|
|||||||
|
|
||||||
# Append to local Sent folder
|
# Append to local Sent folder
|
||||||
try:
|
try:
|
||||||
with _imap() as imap:
|
with _imap(row_account_id, owner=row_owner) as imap:
|
||||||
sent_folder = _detect_sent_folder(imap)
|
sent_folder = _detect_sent_folder(imap)
|
||||||
imap.append(sent_folder, "\\Seen", None, outer.as_bytes())
|
imap.append(sent_folder, "\\Seen", None, outer.as_bytes())
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|||||||
+58
-30
@@ -90,6 +90,16 @@ def _email_tag_owner_aliases(account_id: str | None, owner: str = "") -> list[st
|
|||||||
return out or [""]
|
return out or [""]
|
||||||
|
|
||||||
|
|
||||||
|
def _email_tag_owner_clause(account_id: str | None, owner: str = "") -> tuple[str, list[str]]:
|
||||||
|
aliases = _email_tag_owner_aliases(account_id, owner)
|
||||||
|
placeholders = ",".join("?" * len(aliases))
|
||||||
|
# In configured multi-user mode, do not treat legacy owner='' rows as
|
||||||
|
# visible to everyone. Single-user/unconfigured mode keeps legacy rows.
|
||||||
|
if owner:
|
||||||
|
return f"owner IN ({placeholders})", aliases
|
||||||
|
return f"(owner IN ({placeholders}) OR owner IS NULL)", aliases
|
||||||
|
|
||||||
|
|
||||||
def _record_email_received_events(owner: str, account_id: str | None, folder: str, emails: list[dict]):
|
def _record_email_received_events(owner: str, account_id: str | None, folder: str, emails: list[dict]):
|
||||||
"""Baseline inbox messages, then fire `email_received` for new arrivals."""
|
"""Baseline inbox messages, then fire `email_received` for new arrivals."""
|
||||||
if not owner or (folder or "INBOX").upper() != "INBOX" or not emails:
|
if not owner or (folder or "INBOX").upper() != "INBOX" or not emails:
|
||||||
@@ -591,11 +601,11 @@ def setup_email_routes():
|
|||||||
SECURITY: `owner` is propagated so when `account_id` is missing,
|
SECURITY: `owner` is propagated so when `account_id` is missing,
|
||||||
the fallback config lookup is scoped to this user's accounts only.
|
the fallback config lookup is scoped to this user's accounts only.
|
||||||
"""
|
"""
|
||||||
|
conn = None
|
||||||
try:
|
try:
|
||||||
conn = _imap_connect(account_id, owner=owner)
|
conn = _imap_connect(account_id, owner=owner)
|
||||||
select_status, _ = conn.select(_q(folder), readonly=True)
|
select_status, _ = conn.select(_q(folder), readonly=True)
|
||||||
if select_status != "OK":
|
if select_status != "OK":
|
||||||
conn.logout()
|
|
||||||
return {"emails": [], "total": 0, "folder": folder, "error": f"Folder not found: {folder}"}
|
return {"emails": [], "total": 0, "folder": folder, "error": f"Folder not found: {folder}"}
|
||||||
|
|
||||||
from_clause = ""
|
from_clause = ""
|
||||||
@@ -645,8 +655,7 @@ def setup_email_routes():
|
|||||||
try:
|
try:
|
||||||
import sqlite3 as _sql3t
|
import sqlite3 as _sql3t
|
||||||
_ct = _sql3t.connect(SCHEDULED_DB)
|
_ct = _sql3t.connect(SCHEDULED_DB)
|
||||||
_owner_aliases = _email_tag_owner_aliases(account_id, owner)
|
_owner_clause, _owner_params = _email_tag_owner_clause(account_id, owner)
|
||||||
_owner_ph = ",".join("?" * len(_owner_aliases))
|
|
||||||
# SECURITY: owner-scope the lookup (review C2/H8). Without
|
# SECURITY: owner-scope the lookup (review C2/H8). Without
|
||||||
# this, user A's `tag:urgent` filter would surface UIDs
|
# this, user A's `tag:urgent` filter would surface UIDs
|
||||||
# written by user B and IMAP would return whatever
|
# written by user B and IMAP would return whatever
|
||||||
@@ -658,8 +667,8 @@ def setup_email_routes():
|
|||||||
rows_t = _ct.execute(
|
rows_t = _ct.execute(
|
||||||
"SELECT message_id, uid FROM email_tags "
|
"SELECT message_id, uid FROM email_tags "
|
||||||
"WHERE folder=? AND spam_verdict=1 "
|
"WHERE folder=? AND spam_verdict=1 "
|
||||||
f"AND (owner IN ({_owner_ph}) OR owner IS NULL)",
|
f"AND {_owner_clause}",
|
||||||
(folder, *_owner_aliases),
|
(folder, *_owner_params),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
for mid, uid in rows_t:
|
for mid, uid in rows_t:
|
||||||
if mid:
|
if mid:
|
||||||
@@ -670,8 +679,8 @@ def setup_email_routes():
|
|||||||
rows_t = _ct.execute(
|
rows_t = _ct.execute(
|
||||||
"SELECT message_id, uid, tags FROM email_tags "
|
"SELECT message_id, uid, tags FROM email_tags "
|
||||||
"WHERE folder=? AND tags IS NOT NULL AND tags != '' "
|
"WHERE folder=? AND tags IS NOT NULL AND tags != '' "
|
||||||
f"AND (owner IN ({_owner_ph}) OR owner IS NULL)",
|
f"AND {_owner_clause}",
|
||||||
(folder, *_owner_aliases),
|
(folder, *_owner_params),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
for r in rows_t:
|
for r in rows_t:
|
||||||
try:
|
try:
|
||||||
@@ -743,12 +752,11 @@ def setup_email_routes():
|
|||||||
_uid_strs = [u.decode() for u in uid_list]
|
_uid_strs = [u.decode() for u in uid_list]
|
||||||
if _uid_strs:
|
if _uid_strs:
|
||||||
placeholders = ",".join("?" * len(_uid_strs))
|
placeholders = ",".join("?" * len(_uid_strs))
|
||||||
_owner_aliases = _email_tag_owner_aliases(account_id, owner)
|
_owner_clause, _owner_params = _email_tag_owner_clause(account_id, owner)
|
||||||
_owner_ph = ",".join("?" * len(_owner_aliases))
|
|
||||||
rows = _c.execute(
|
rows = _c.execute(
|
||||||
f"SELECT uid, tags, spam_verdict FROM email_tags "
|
f"SELECT uid, tags, spam_verdict FROM email_tags "
|
||||||
f"WHERE folder=? AND (owner IN ({_owner_ph}) OR owner IS NULL) AND uid IN ({placeholders})",
|
f"WHERE folder=? AND {_owner_clause} AND uid IN ({placeholders})",
|
||||||
[folder, *_owner_aliases, *_uid_strs],
|
[folder, *_owner_params, *_uid_strs],
|
||||||
).fetchall()
|
).fetchall()
|
||||||
for r in rows:
|
for r in rows:
|
||||||
try:
|
try:
|
||||||
@@ -805,14 +813,13 @@ def setup_email_routes():
|
|||||||
if header_ids:
|
if header_ids:
|
||||||
import sqlite3 as _sql3m
|
import sqlite3 as _sql3m
|
||||||
_cm = _sql3m.connect(SCHEDULED_DB)
|
_cm = _sql3m.connect(SCHEDULED_DB)
|
||||||
_owner_aliases_m = _email_tag_owner_aliases(account_id, owner)
|
_owner_clause_m, _owner_params_m = _email_tag_owner_clause(account_id, owner)
|
||||||
_owner_ph_m = ",".join("?" * len(_owner_aliases_m))
|
|
||||||
_mid_ph = ",".join("?" * len(header_ids))
|
_mid_ph = ",".join("?" * len(header_ids))
|
||||||
rows_m = _cm.execute(
|
rows_m = _cm.execute(
|
||||||
f"SELECT message_id, tags, spam_verdict FROM email_tags "
|
f"SELECT message_id, tags, spam_verdict FROM email_tags "
|
||||||
f"WHERE folder=? AND (owner IN ({_owner_ph_m}) OR owner IS NULL) "
|
f"WHERE folder=? AND {_owner_clause_m} "
|
||||||
f"AND message_id IN ({_mid_ph})",
|
f"AND message_id IN ({_mid_ph})",
|
||||||
[folder, *_owner_aliases_m, *header_ids],
|
[folder, *_owner_params_m, *header_ids],
|
||||||
).fetchall()
|
).fetchall()
|
||||||
_cm.close()
|
_cm.close()
|
||||||
for mid, tags_raw, spam_raw in rows_m:
|
for mid, tags_raw, spam_raw in rows_m:
|
||||||
@@ -924,12 +931,17 @@ def setup_email_routes():
|
|||||||
except Exception as _summary_err:
|
except Exception as _summary_err:
|
||||||
logger.debug(f"Bulk summary attach skipped: {_summary_err}")
|
logger.debug(f"Bulk summary attach skipped: {_summary_err}")
|
||||||
|
|
||||||
conn.logout()
|
|
||||||
return {"emails": emails, "total": total, "folder": folder, "offset": offset}
|
return {"emails": emails, "total": total, "folder": folder, "offset": offset}
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Failed to list emails: {e}")
|
logger.error(f"Failed to list emails: {e}")
|
||||||
detail = str(e).strip()
|
detail = str(e).strip()
|
||||||
return {"emails": [], "total": 0, "error": f"Mail operation failed: {detail[:180]}" if detail else "Mail operation failed"}
|
return {"emails": [], "total": 0, "error": f"Mail operation failed: {detail[:180]}" if detail else "Mail operation failed"}
|
||||||
|
finally:
|
||||||
|
if conn:
|
||||||
|
try:
|
||||||
|
conn.logout()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
@router.get("/list")
|
@router.get("/list")
|
||||||
async def list_emails(
|
async def list_emails(
|
||||||
@@ -971,10 +983,11 @@ def setup_email_routes():
|
|||||||
async def unflag_spam(uid: str, owner: str = Depends(require_owner)):
|
async def unflag_spam(uid: str, owner: str = Depends(require_owner)):
|
||||||
"""User override — mark email as not spam."""
|
"""User override — mark email as not spam."""
|
||||||
try:
|
try:
|
||||||
|
owner_clause, owner_params = _email_tag_owner_clause(None, owner)
|
||||||
_c = _sql3.connect(SCHEDULED_DB)
|
_c = _sql3.connect(SCHEDULED_DB)
|
||||||
_c.execute(
|
_c.execute(
|
||||||
"UPDATE email_tags SET spam_verdict=0, spam_reason='' WHERE uid=?",
|
f"UPDATE email_tags SET spam_verdict=0, spam_reason='' WHERE uid=? AND {owner_clause}",
|
||||||
(uid,),
|
[uid, *owner_params],
|
||||||
)
|
)
|
||||||
_c.commit()
|
_c.commit()
|
||||||
_c.close()
|
_c.close()
|
||||||
@@ -997,8 +1010,10 @@ def setup_email_routes():
|
|||||||
ql = (q or "").strip().lower()
|
ql = (q or "").strip().lower()
|
||||||
try:
|
try:
|
||||||
conn = _sql3.connect(SCHEDULED_DB)
|
conn = _sql3.connect(SCHEDULED_DB)
|
||||||
|
owner_clause, owner_params = _email_tag_owner_clause(None, owner)
|
||||||
rows = conn.execute(
|
rows = conn.execute(
|
||||||
"SELECT sender FROM email_tags WHERE sender IS NOT NULL AND sender != ''"
|
f"SELECT sender FROM email_tags WHERE sender IS NOT NULL AND sender != '' AND {owner_clause}",
|
||||||
|
owner_params,
|
||||||
).fetchall()
|
).fetchall()
|
||||||
conn.close()
|
conn.close()
|
||||||
seen = {}
|
seen = {}
|
||||||
@@ -1969,8 +1984,8 @@ def setup_email_routes():
|
|||||||
conn = sqlite3.connect(SCHEDULED_DB)
|
conn = sqlite3.connect(SCHEDULED_DB)
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
INSERT INTO scheduled_emails
|
INSERT INTO scheduled_emails
|
||||||
(id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, send_at, created_at, status, account_id, odysseus_kind)
|
(id, to_addr, cc, bcc, subject, body, in_reply_to, references_hdr, attachments, send_at, created_at, status, account_id, odysseus_kind, owner)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?, ?)
|
||||||
""", (
|
""", (
|
||||||
sid,
|
sid,
|
||||||
req.get("to", ""),
|
req.get("to", ""),
|
||||||
@@ -1985,6 +2000,7 @@ def setup_email_routes():
|
|||||||
datetime.utcnow().isoformat(),
|
datetime.utcnow().isoformat(),
|
||||||
req.get("account_id") or None,
|
req.get("account_id") or None,
|
||||||
req.get("odysseus_kind") or "scheduled",
|
req.get("odysseus_kind") or "scheduled",
|
||||||
|
owner or "",
|
||||||
))
|
))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
conn.close()
|
conn.close()
|
||||||
@@ -2003,9 +2019,9 @@ def setup_email_routes():
|
|||||||
rows = conn.execute("""
|
rows = conn.execute("""
|
||||||
SELECT id, to_addr, cc, subject, send_at, created_at, status, error
|
SELECT id, to_addr, cc, subject, send_at, created_at, status, error
|
||||||
FROM scheduled_emails
|
FROM scheduled_emails
|
||||||
WHERE status IN ('pending', 'failed')
|
WHERE status IN ('pending', 'failed') AND owner = ?
|
||||||
ORDER BY send_at ASC
|
ORDER BY send_at ASC
|
||||||
""").fetchall()
|
""", (owner or "",)).fetchall()
|
||||||
conn.close()
|
conn.close()
|
||||||
return {"scheduled": [
|
return {"scheduled": [
|
||||||
{
|
{
|
||||||
@@ -2023,7 +2039,10 @@ def setup_email_routes():
|
|||||||
import sqlite3
|
import sqlite3
|
||||||
try:
|
try:
|
||||||
conn = sqlite3.connect(SCHEDULED_DB)
|
conn = sqlite3.connect(SCHEDULED_DB)
|
||||||
conn.execute("DELETE FROM scheduled_emails WHERE id = ? AND status = 'pending'", (sid,))
|
conn.execute(
|
||||||
|
"DELETE FROM scheduled_emails WHERE id = ? AND status = 'pending' AND owner = ?",
|
||||||
|
(sid, owner or ""),
|
||||||
|
)
|
||||||
conn.commit()
|
conn.commit()
|
||||||
conn.close()
|
conn.close()
|
||||||
return {"success": True}
|
return {"success": True}
|
||||||
@@ -2035,7 +2054,7 @@ def setup_email_routes():
|
|||||||
async def resolve_contact(name: str = Query(..., description="Name to search for"), owner: str = Depends(require_owner)):
|
async def resolve_contact(name: str = Query(..., description="Name to search for"), owner: str = Depends(require_owner)):
|
||||||
"""Search Sent folder for a contact by name. Returns matching email addresses."""
|
"""Search Sent folder for a contact by name. Returns matching email addresses."""
|
||||||
try:
|
try:
|
||||||
with _imap() as conn:
|
with _imap(owner=owner) as conn:
|
||||||
matches = {}
|
matches = {}
|
||||||
for folder in ["Sent", "INBOX", "Drafts"]:
|
for folder in ["Sent", "INBOX", "Drafts"]:
|
||||||
try:
|
try:
|
||||||
@@ -2590,7 +2609,7 @@ def setup_email_routes():
|
|||||||
# `api_key` field.
|
# `api_key` field.
|
||||||
from core.database import SessionLocal as _SL, Session as _CS
|
from core.database import SessionLocal as _SL, Session as _CS
|
||||||
_db = _SL()
|
_db = _SL()
|
||||||
sess = _db.query(_CS).filter(_CS.id == session_id).first()
|
sess = _db.query(_CS).filter(_CS.id == session_id, _CS.owner == owner).first()
|
||||||
if sess and sess.endpoint_url:
|
if sess and sess.endpoint_url:
|
||||||
url = sess.endpoint_url
|
url = sess.endpoint_url
|
||||||
# Some sessions stored headers double-encoded (a JSON
|
# Some sessions stored headers double-encoded (a JSON
|
||||||
@@ -2649,9 +2668,10 @@ def setup_email_routes():
|
|||||||
# Manual AI Reply should feel immediate. The heavier context mining
|
# Manual AI Reply should feel immediate. The heavier context mining
|
||||||
# can involve multiple IMAP folder searches and attachment parsing;
|
# can involve multiple IMAP folder searches and attachment parsing;
|
||||||
# reserve that for callers that explicitly opt out of fast mode.
|
# reserve that for callers that explicitly opt out of fast mode.
|
||||||
|
# Owner-scoped so pre-retrieval never crosses tenants.
|
||||||
context_snippets, _terms = ([], [])
|
context_snippets, _terms = ([], [])
|
||||||
if not fast_reply:
|
if not fast_reply:
|
||||||
context_snippets, _terms = _pre_retrieve_context(original_body, to)
|
context_snippets, _terms = _pre_retrieve_context(original_body, to, owner=owner)
|
||||||
|
|
||||||
# NEW: also pull the last few emails from the original sender +
|
# NEW: also pull the last few emails from the original sender +
|
||||||
# their attachments. The "to" field on this endpoint is the
|
# their attachments. The "to" field on this endpoint is the
|
||||||
@@ -2667,6 +2687,7 @@ def setup_email_routes():
|
|||||||
exclude_uid=source_uid,
|
exclude_uid=source_uid,
|
||||||
exclude_folder=source_folder,
|
exclude_folder=source_folder,
|
||||||
limit=3,
|
limit=3,
|
||||||
|
owner=owner,
|
||||||
)
|
)
|
||||||
except Exception as _e:
|
except Exception as _e:
|
||||||
logger.warning(f"sender-thread-context failed: {_e}")
|
logger.warning(f"sender-thread-context failed: {_e}")
|
||||||
@@ -2728,7 +2749,7 @@ def setup_email_routes():
|
|||||||
# Configured fallback chains last.
|
# Configured fallback chains last.
|
||||||
for cand in resolve_utility_fallback_candidates(owner=owner) or []:
|
for cand in resolve_utility_fallback_candidates(owner=owner) or []:
|
||||||
_add(*cand)
|
_add(*cand)
|
||||||
for cand in resolve_chat_fallback_candidates() or []:
|
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
|
||||||
_add(*cand)
|
_add(*cand)
|
||||||
try:
|
try:
|
||||||
reply = await llm_call_async_with_fallback(
|
reply = await llm_call_async_with_fallback(
|
||||||
@@ -2819,9 +2840,12 @@ def setup_email_routes():
|
|||||||
import uuid as _uuid
|
import uuid as _uuid
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
row = db.query(EmailAccount).filter(EmailAccount.is_default == True).first() # noqa: E712
|
q = db.query(EmailAccount).filter(EmailAccount.is_default == True) # noqa: E712
|
||||||
|
if owner:
|
||||||
|
q = q.filter(EmailAccount.owner == owner)
|
||||||
|
row = q.first()
|
||||||
if row is None:
|
if row is None:
|
||||||
row = EmailAccount(id=_uuid.uuid4().hex, name="Default", is_default=True, enabled=True)
|
row = EmailAccount(id=_uuid.uuid4().hex, owner=owner, name="Default", is_default=True, enabled=True)
|
||||||
db.add(row)
|
db.add(row)
|
||||||
field_map = {
|
field_map = {
|
||||||
"smtp_host": "smtp_host", "smtp_port": "smtp_port", "smtp_user": "smtp_user",
|
"smtp_host": "smtp_host", "smtp_port": "smtp_port", "smtp_user": "smtp_user",
|
||||||
@@ -2843,6 +2867,10 @@ def setup_email_routes():
|
|||||||
row.imap_password = _enc(data["imap_password"])
|
row.imap_password = _enc(data["imap_password"])
|
||||||
if data.get("smtp_password"):
|
if data.get("smtp_password"):
|
||||||
row.smtp_password = _enc(data["smtp_password"])
|
row.smtp_password = _enc(data["smtp_password"])
|
||||||
|
clear_q = db.query(EmailAccount).filter(EmailAccount.id != row.id)
|
||||||
|
if owner:
|
||||||
|
clear_q = clear_q.filter(EmailAccount.owner == owner)
|
||||||
|
clear_q.update({EmailAccount.is_default: False})
|
||||||
db.commit()
|
db.commit()
|
||||||
finally:
|
finally:
|
||||||
db.close()
|
db.close()
|
||||||
|
|||||||
@@ -148,6 +148,32 @@ def _docker_host_gateway_reachable() -> bool:
|
|||||||
except OSError:
|
except OSError:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
def _container_loopback_reachable(base_url: str, timeout: float = 0.2) -> bool:
|
||||||
|
"""True when the requested loopback host:port is already reachable from
|
||||||
|
inside the current container.
|
||||||
|
|
||||||
|
This distinguishes "a model server running alongside Odysseus in the same
|
||||||
|
container" from "a model server running on the Docker host". Only the
|
||||||
|
latter should be rewritten to host.docker.internal.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
parsed = urlparse(base_url)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
host = (parsed.hostname or "").lower()
|
||||||
|
port = parsed.port
|
||||||
|
if host not in _LOOPBACK_HOSTS or not port:
|
||||||
|
return False
|
||||||
|
probe_host = "::1" if host == "::1" else "127.0.0.1"
|
||||||
|
family = socket.AF_INET6 if probe_host == "::1" else socket.AF_INET
|
||||||
|
try:
|
||||||
|
with socket.socket(family, socket.SOCK_STREAM) as sock:
|
||||||
|
sock.settimeout(timeout)
|
||||||
|
sock.connect((probe_host, port))
|
||||||
|
return True
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False) -> str:
|
def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False) -> str:
|
||||||
"""Rewrite a loopback model-endpoint URL to ``host.docker.internal`` when
|
"""Rewrite a loopback model-endpoint URL to ``host.docker.internal`` when
|
||||||
@@ -176,6 +202,8 @@ def _rewrite_loopback_for_docker(base_url: str, *, container_local: bool = False
|
|||||||
if host in _ANY_BIND_HOSTS and not _docker_host_gateway_reachable():
|
if host in _ANY_BIND_HOSTS and not _docker_host_gateway_reachable():
|
||||||
netloc = "127.0.0.1" + (f":{parsed.port}" if parsed.port else "")
|
netloc = "127.0.0.1" + (f":{parsed.port}" if parsed.port else "")
|
||||||
return urlunparse(parsed._replace(netloc=netloc))
|
return urlunparse(parsed._replace(netloc=netloc))
|
||||||
|
if _container_loopback_reachable(base_url):
|
||||||
|
return base_url
|
||||||
if not _docker_host_gateway_reachable():
|
if not _docker_host_gateway_reachable():
|
||||||
return base_url
|
return base_url
|
||||||
netloc = "host.docker.internal" + (f":{parsed.port}" if parsed.port else "")
|
netloc = "host.docker.internal" + (f":{parsed.port}" if parsed.port else "")
|
||||||
|
|||||||
+115
-32
@@ -16,7 +16,7 @@ from src.auth_helpers import get_current_user, effective_user
|
|||||||
|
|
||||||
def _sanitize_export_filename(name: str) -> str:
|
def _sanitize_export_filename(name: str) -> str:
|
||||||
"""Return a conservative filename safe for Content-Disposition."""
|
"""Return a conservative filename safe for Content-Disposition."""
|
||||||
name = name or ""
|
name = name if isinstance(name, str) else ""
|
||||||
name = re.sub(r"[^A-Za-z0-9._-]", "_", name)
|
name = re.sub(r"[^A-Za-z0-9._-]", "_", name)
|
||||||
return name[:128]
|
return name[:128]
|
||||||
|
|
||||||
@@ -58,23 +58,71 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
router = APIRouter(prefix="/api", tags=["sessions"])
|
router = APIRouter(prefix="/api", tags=["sessions"])
|
||||||
|
|
||||||
def _pick_endpoint_for_sort():
|
|
||||||
|
def _current_user_is_admin(request: Request, user: str | None) -> bool:
|
||||||
|
if not user:
|
||||||
|
return False
|
||||||
|
auth_mgr = getattr(request.app.state, "auth_manager", None)
|
||||||
|
is_admin = getattr(auth_mgr, "is_admin", None)
|
||||||
|
if not callable(is_admin):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(is_admin(user))
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_raw_endpoint_url_for_non_admin(
|
||||||
|
request: Request,
|
||||||
|
user: str | None,
|
||||||
|
endpoint_id: str | None,
|
||||||
|
endpoint_url: str | None,
|
||||||
|
) -> None:
|
||||||
|
"""Require registered endpoints for signed-in non-admin session changes."""
|
||||||
|
if endpoint_id and endpoint_id.strip():
|
||||||
|
return
|
||||||
|
if not endpoint_url:
|
||||||
|
return
|
||||||
|
# Raw URLs make the server dial whatever host the request supplies. For
|
||||||
|
# non-admin users, require a saved endpoint row so normal owner scoping and
|
||||||
|
# endpoint validation have already happened.
|
||||||
|
if user and not _current_user_is_admin(request, user):
|
||||||
|
raise HTTPException(403, "Choose a registered model endpoint")
|
||||||
|
|
||||||
|
|
||||||
|
def _persist_session_headers(session_id: str, headers: dict | None) -> None:
|
||||||
|
"""Persist endpoint auth headers for DB-backed session metadata."""
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
|
||||||
|
if db_session:
|
||||||
|
db_session.headers = headers or {}
|
||||||
|
db_session.updated_at = datetime.utcnow()
|
||||||
|
db.commit()
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _pick_endpoint_for_sort(owner=None):
|
||||||
"""Pick model endpoint for auto-sort LLM call — uses utility endpoint setting, falls back to default."""
|
"""Pick model endpoint for auto-sort LLM call — uses utility endpoint setting, falls back to default."""
|
||||||
from src.endpoint_resolver import resolve_endpoint
|
from src.endpoint_resolver import resolve_endpoint
|
||||||
# Try utility endpoint first (what the user configured for background tasks)
|
# Try utility endpoint first (what the user configured for background tasks)
|
||||||
url, model, headers = resolve_endpoint("utility")
|
url, model, headers = resolve_endpoint("utility", owner=owner)
|
||||||
if url and model:
|
if url and model:
|
||||||
return url, model, headers
|
return url, model, headers
|
||||||
# Fall back to task endpoint
|
# Fall back to task endpoint
|
||||||
try:
|
try:
|
||||||
from src.task_endpoint import resolve_task_endpoint
|
from src.task_endpoint import resolve_task_endpoint
|
||||||
url, model, headers = resolve_task_endpoint()
|
url, model, headers = resolve_task_endpoint(owner=owner)
|
||||||
if url and model:
|
if url and model:
|
||||||
return url, model, headers
|
return url, model, headers
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
# Fall back to default
|
# Fall back to default
|
||||||
url, model, headers = resolve_endpoint("default")
|
url, model, headers = resolve_endpoint("default", owner=owner)
|
||||||
if url and model:
|
if url and model:
|
||||||
return url, model, headers
|
return url, model, headers
|
||||||
return None, None, None
|
return None, None, None
|
||||||
@@ -197,11 +245,41 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
endpoint_id: str = Form(""),
|
endpoint_id: str = Form(""),
|
||||||
):
|
):
|
||||||
skip_val = str(skip_validation).lower() == "true"
|
skip_val = str(skip_validation).lower() == "true"
|
||||||
|
user = get_current_user(request)
|
||||||
|
endpoint_api_key = ""
|
||||||
|
endpoint_base_url = ""
|
||||||
|
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
|
||||||
|
if endpoint_id and endpoint_id.strip():
|
||||||
|
from core.database import ModelEndpoint
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||||
|
_db = SessionLocal()
|
||||||
|
try:
|
||||||
|
q = _db.query(ModelEndpoint).filter(
|
||||||
|
ModelEndpoint.id == endpoint_id.strip(),
|
||||||
|
ModelEndpoint.is_enabled == True,
|
||||||
|
)
|
||||||
|
if user:
|
||||||
|
q = owner_filter(q, ModelEndpoint, user)
|
||||||
|
endpoint_row = q.first()
|
||||||
|
if not endpoint_row:
|
||||||
|
raise HTTPException(400, "Model endpoint no longer exists")
|
||||||
|
endpoint_base_url = endpoint_row.base_url or ""
|
||||||
|
endpoint_api_key = endpoint_row.api_key or ""
|
||||||
|
endpoint_url = build_chat_url(normalize_base(endpoint_base_url))
|
||||||
|
finally:
|
||||||
|
_db.close()
|
||||||
|
|
||||||
if not endpoint_url and not skip_val:
|
if not endpoint_url and not skip_val:
|
||||||
raise HTTPException(400, "endpoint_url is required (choose from /api/models)")
|
raise HTTPException(400, "endpoint_url is required (choose from /api/models)")
|
||||||
|
|
||||||
model_to_use = model
|
model_to_use = model
|
||||||
|
request_api_key = api_key.strip() if api_key else ""
|
||||||
|
effective_api_key = request_api_key or endpoint_api_key
|
||||||
|
validation_headers = None
|
||||||
|
if effective_api_key:
|
||||||
|
from src.endpoint_resolver import build_headers
|
||||||
|
validation_headers = build_headers(effective_api_key, endpoint_base_url or endpoint_url)
|
||||||
|
|
||||||
if skip_val:
|
if skip_val:
|
||||||
# skip_validation = trust the caller and do NOT probe /v1/models.
|
# skip_validation = trust the caller and do NOT probe /v1/models.
|
||||||
@@ -212,7 +290,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
elif not model_to_use:
|
elif not model_to_use:
|
||||||
from src.llm_core import list_model_ids
|
from src.llm_core import list_model_ids
|
||||||
ids = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
ids = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
||||||
headers={"Authorization": f"Bearer {api_key}"} if api_key.strip() else None)
|
headers=validation_headers)
|
||||||
if not ids:
|
if not ids:
|
||||||
raise HTTPException(400, "Cannot reach /v1/models")
|
raise HTTPException(400, "Cannot reach /v1/models")
|
||||||
# Default to the first CHAT model — endpoints often list embedding/
|
# Default to the first CHAT model — endpoints often list embedding/
|
||||||
@@ -227,7 +305,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
import os as _os
|
import os as _os
|
||||||
req_base = _os.path.basename(model_to_use.rstrip("/"))
|
req_base = _os.path.basename(model_to_use.rstrip("/"))
|
||||||
avail = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
avail = list_model_ids(endpoint_url, timeout=REQUEST_TIMEOUT,
|
||||||
headers={"Authorization": f"Bearer {api_key}"} if api_key.strip() else None)
|
headers=validation_headers)
|
||||||
if not avail:
|
if not avail:
|
||||||
raise HTTPException(400, "Cannot reach /v1/models")
|
raise HTTPException(400, "Cannot reach /v1/models")
|
||||||
if model_to_use not in avail:
|
if model_to_use not in avail:
|
||||||
@@ -252,22 +330,15 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
owner=user,
|
owner=user,
|
||||||
)
|
)
|
||||||
# Set auth headers for custom API-key endpoints
|
# Set auth headers for custom API-key endpoints
|
||||||
resolved_key = api_key.strip() if api_key else ""
|
resolved_key = request_api_key
|
||||||
resolved_base = endpoint_url
|
resolved_base = endpoint_url
|
||||||
if not resolved_key and endpoint_id and endpoint_id.strip():
|
if not resolved_key and endpoint_api_key:
|
||||||
from core.database import ModelEndpoint
|
resolved_key = endpoint_api_key
|
||||||
_db = SessionLocal()
|
resolved_base = endpoint_base_url
|
||||||
try:
|
|
||||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id.strip()).first()
|
|
||||||
if ep and ep.api_key:
|
|
||||||
resolved_key = ep.api_key
|
|
||||||
resolved_base = ep.base_url
|
|
||||||
finally:
|
|
||||||
_db.close()
|
|
||||||
if resolved_key:
|
if resolved_key:
|
||||||
from src.endpoint_resolver import build_headers
|
from src.endpoint_resolver import build_headers
|
||||||
session.headers = build_headers(resolved_key, resolved_base)
|
session.headers = build_headers(resolved_key, resolved_base)
|
||||||
session_manager.save_sessions()
|
_persist_session_headers(sid, session.headers)
|
||||||
# Fire webhook (sync-safe)
|
# Fire webhook (sync-safe)
|
||||||
if webhook_manager:
|
if webhook_manager:
|
||||||
webhook_manager.fire_and_forget("session.created", {
|
webhook_manager.fire_and_forget("session.created", {
|
||||||
@@ -313,27 +384,38 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
db.close()
|
db.close()
|
||||||
# Switch model/endpoint mid-session
|
# Switch model/endpoint mid-session
|
||||||
if model is not None and endpoint_url is not None:
|
if model is not None and endpoint_url is not None:
|
||||||
|
user = get_current_user(request)
|
||||||
|
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
|
||||||
|
endpoint_api_key = ""
|
||||||
|
endpoint_base_url = ""
|
||||||
if endpoint_id:
|
if endpoint_id:
|
||||||
from core.database import ModelEndpoint
|
from core.database import ModelEndpoint
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
from src.endpoint_resolver import build_chat_url, normalize_base
|
||||||
_db = SessionLocal()
|
_db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id).first()
|
q = _db.query(ModelEndpoint).filter(
|
||||||
|
ModelEndpoint.id == endpoint_id,
|
||||||
|
ModelEndpoint.is_enabled == True,
|
||||||
|
)
|
||||||
|
if user:
|
||||||
|
q = owner_filter(q, ModelEndpoint, user)
|
||||||
|
ep = q.first()
|
||||||
if not ep:
|
if not ep:
|
||||||
raise HTTPException(400, "Model endpoint no longer exists")
|
raise HTTPException(400, "Model endpoint no longer exists")
|
||||||
|
endpoint_base_url = ep.base_url or ""
|
||||||
|
endpoint_api_key = ep.api_key or ""
|
||||||
|
endpoint_url = build_chat_url(normalize_base(endpoint_base_url))
|
||||||
finally:
|
finally:
|
||||||
_db.close()
|
_db.close()
|
||||||
session.model = model
|
session.model = model
|
||||||
session.endpoint_url = endpoint_url
|
session.endpoint_url = endpoint_url
|
||||||
# Update auth headers from the endpoint's stored API key
|
# Update auth headers from the endpoint's stored API key
|
||||||
if endpoint_id:
|
if endpoint_api_key:
|
||||||
_db = SessionLocal()
|
from src.endpoint_resolver import build_headers
|
||||||
try:
|
session.headers = build_headers(endpoint_api_key, endpoint_base_url)
|
||||||
ep = _db.query(ModelEndpoint).filter(ModelEndpoint.id == endpoint_id).first()
|
else:
|
||||||
if ep and ep.api_key:
|
session.headers = {}
|
||||||
from src.endpoint_resolver import build_headers
|
|
||||||
session.headers = build_headers(ep.api_key, ep.base_url)
|
|
||||||
finally:
|
|
||||||
_db.close()
|
|
||||||
# Persist to DB
|
# Persist to DB
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
@@ -341,6 +423,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
if db_session:
|
if db_session:
|
||||||
db_session.model = model
|
db_session.model = model
|
||||||
db_session.endpoint_url = endpoint_url
|
db_session.endpoint_url = endpoint_url
|
||||||
|
db_session.headers = session.headers or {}
|
||||||
db_session.updated_at = datetime.utcnow()
|
db_session.updated_at = datetime.utcnow()
|
||||||
db.commit()
|
db.commit()
|
||||||
finally:
|
finally:
|
||||||
@@ -754,7 +837,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
from src.endpoint_resolver import resolve_endpoint
|
from src.endpoint_resolver import resolve_endpoint
|
||||||
from src.llm_core import llm_call_async
|
from src.llm_core import llm_call_async
|
||||||
|
|
||||||
url, model, headers = resolve_endpoint("utility")
|
url, model, headers = resolve_endpoint("utility", owner=get_current_user(request))
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
url, model, headers = session.endpoint_url, session.model, session.headers
|
url, model, headers = session.endpoint_url, session.model, session.headers
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
@@ -954,9 +1037,9 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
|
|||||||
|
|
||||||
# Pick an endpoint — prefer admin-configured task endpoint
|
# Pick an endpoint — prefer admin-configured task endpoint
|
||||||
from src.task_endpoint import resolve_task_endpoint
|
from src.task_endpoint import resolve_task_endpoint
|
||||||
url, model, headers = resolve_task_endpoint()
|
url, model, headers = resolve_task_endpoint(owner=user)
|
||||||
if not url:
|
if not url:
|
||||||
url, model, headers = _pick_endpoint_for_sort()
|
url, model, headers = _pick_endpoint_for_sort(owner=user)
|
||||||
if not url:
|
if not url:
|
||||||
raise HTTPException(503, "No available model endpoint for auto-sort")
|
raise HTTPException(503, "No available model endpoint for auto-sort")
|
||||||
|
|
||||||
|
|||||||
+29
-17
@@ -79,6 +79,8 @@ def _skill_test_task(skill: dict) -> str:
|
|||||||
an email); if we just hand over the 'when to use' text the agent has nothing
|
an email); if we just hand over the 'when to use' text the agent has nothing
|
||||||
to work on and stalls asking for input. So we tell it to create its own
|
to work on and stalls asking for input. So we tell it to create its own
|
||||||
realistic fixture first, then apply the skill end-to-end."""
|
realistic fixture first, then apply the skill end-to-end."""
|
||||||
|
if not isinstance(skill, dict):
|
||||||
|
skill = {}
|
||||||
ctx = (skill.get("when_to_use") or skill.get("description") or skill.get("name") or "").strip()
|
ctx = (skill.get("when_to_use") or skill.get("description") or skill.get("name") or "").strip()
|
||||||
return (
|
return (
|
||||||
"Test this skill end-to-end. FIRST, set up a small realistic scenario it "
|
"Test this skill end-to-end. FIRST, set up a small realistic scenario it "
|
||||||
@@ -310,6 +312,8 @@ def _should_check_retrieval_precision(skill: dict) -> bool:
|
|||||||
"installation", "install", "system", "ssh", "document", "documents",
|
"installation", "install", "system", "ssh", "document", "documents",
|
||||||
"search", "email", "calendar", "gpu", "server", "python",
|
"search", "email", "calendar", "gpu", "server", "python",
|
||||||
}
|
}
|
||||||
|
if not isinstance(skill, dict):
|
||||||
|
return False
|
||||||
tags = {str(t or "").strip().lower() for t in (skill.get("tags") or [])}
|
tags = {str(t or "").strip().lower() for t in (skill.get("tags") or [])}
|
||||||
if tags & broad:
|
if tags & broad:
|
||||||
return True
|
return True
|
||||||
@@ -463,7 +467,7 @@ async def _run_skill_test_job(key, name, md, task, url, model, headers, owner, s
|
|||||||
if skills_manager is not None:
|
if skills_manager is not None:
|
||||||
v = (job["verdict"] or {}).get("verdict") or "unknown"
|
v = (job["verdict"] or {}).get("verdict") or "unknown"
|
||||||
try:
|
try:
|
||||||
skills_manager.set_audit(name, v, by_teacher=False, worker_model=model)
|
skills_manager.set_audit(name, v, by_teacher=False, worker_model=model, owner=owner)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
conf = {"pass": 0.95, "needs_work": 0.6, "fail": 0.4}.get(v)
|
conf = {"pass": 0.95, "needs_work": 0.6, "fail": 0.4}.get(v)
|
||||||
@@ -563,6 +567,7 @@ def _skill_duplicate_blocker(skills_manager, name: str, owner) -> Optional[str]:
|
|||||||
False,
|
False,
|
||||||
[keeper_name],
|
[keeper_name],
|
||||||
f"Lower-priority duplicate of {keeper_name}",
|
f"Lower-priority duplicate of {keeper_name}",
|
||||||
|
owner=owner,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
@@ -629,7 +634,7 @@ def _audit_finalize_status(skills_manager, name: str, owner, verdict: str,
|
|||||||
if generic_reason:
|
if generic_reason:
|
||||||
necessary = False
|
necessary = False
|
||||||
try:
|
try:
|
||||||
skills_manager.set_necessity(name, False, [], generic_reason)
|
skills_manager.set_necessity(name, False, [], generic_reason, owner=owner)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
duplicate_of = _skill_duplicate_blocker(skills_manager, name, owner) if verdict == "pass" else None
|
duplicate_of = _skill_duplicate_blocker(skills_manager, name, owner) if verdict == "pass" else None
|
||||||
@@ -788,7 +793,8 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
nec = await _eval_skill_necessity(md, others, url, model, headers)
|
nec = await _eval_skill_necessity(md, others, url, model, headers)
|
||||||
if nec is not None:
|
if nec is not None:
|
||||||
skills_manager.set_necessity(name, nec.get("necessary", True),
|
skills_manager.set_necessity(name, nec.get("necessary", True),
|
||||||
nec.get("redundant_with"), nec.get("reason"))
|
nec.get("redundant_with"), nec.get("reason"),
|
||||||
|
owner=owner)
|
||||||
if not nec.get("necessary", True):
|
if not nec.get("necessary", True):
|
||||||
log(f"{name}: possibly unnecessary — {nec.get('reason', '')[:80]}")
|
log(f"{name}: possibly unnecessary — {nec.get('reason', '')[:80]}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -800,11 +806,11 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
reason = generic_reason or (f"Lower-priority duplicate of {duplicate_of}" if duplicate_of else str((nec or {}).get("reason") or "Unnecessary skill"))
|
reason = generic_reason or (f"Lower-priority duplicate of {duplicate_of}" if duplicate_of else str((nec or {}).get("reason") or "Unnecessary skill"))
|
||||||
try:
|
try:
|
||||||
skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35}, owner=owner)
|
skills_manager.update_skill(name, {"status": "draft", "confidence": 0.35}, owner=owner)
|
||||||
skills_manager.set_audit(name, "skipped", by_teacher=False, worker_model=model)
|
skills_manager.set_audit(name, "skipped", by_teacher=False, worker_model=model, owner=owner)
|
||||||
if duplicate_of:
|
if duplicate_of:
|
||||||
skills_manager.set_necessity(name, False, [duplicate_of], reason)
|
skills_manager.set_necessity(name, False, [duplicate_of], reason, owner=owner)
|
||||||
else:
|
else:
|
||||||
skills_manager.set_necessity(name, False, [], reason)
|
skills_manager.set_necessity(name, False, [], reason, owner=owner)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
log(f"{name}: draft — skipped functional test ({reason[:100]})")
|
log(f"{name}: draft — skipped functional test ({reason[:100]})")
|
||||||
@@ -848,13 +854,13 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
if fixed and fixed.strip() != md.strip():
|
if fixed and fixed.strip() != md.strip():
|
||||||
_apply_skill_md(skills_manager, name, fixed, owner)
|
_apply_skill_md(skills_manager, name, fixed, owner)
|
||||||
_set_conf(0.95)
|
_set_conf(0.95)
|
||||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model)
|
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model, owner=owner)
|
||||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.95, (refreshed or {}).get("necessity"), verdict)
|
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.95, (refreshed or {}).get("necessity"), verdict)
|
||||||
log(f"{name}: {status} — confidence 95%")
|
log(f"{name}: {status} — confidence 95%")
|
||||||
return {"skill": name, "result": "pass", "verdict": verdict, "confidence": 0.95, "status": status}
|
return {"skill": name, "result": "pass", "verdict": verdict, "confidence": 0.95, "status": status}
|
||||||
if v in ("unknown", "inconclusive"):
|
if v in ("unknown", "inconclusive"):
|
||||||
skills_manager.set_audit(name, "inconclusive", by_teacher=False, worker_model=model)
|
skills_manager.set_audit(name, "inconclusive", by_teacher=False, worker_model=model, owner=owner)
|
||||||
status = _audit_finalize_status(skills_manager, name, owner, "inconclusive", skill.get("confidence") or 0.0, skill.get("necessity"))
|
status = _audit_finalize_status(skills_manager, name, owner, "inconclusive", skill.get("confidence") or 0.0, skill.get("necessity"))
|
||||||
log(f"{name}: {status} — inconclusive")
|
log(f"{name}: {status} — inconclusive")
|
||||||
return {"skill": name, "result": "inconclusive", "verdict": verdict, "status": status}
|
return {"skill": name, "result": "inconclusive", "verdict": verdict, "status": status}
|
||||||
@@ -869,7 +875,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
log(f"{name}: retry (self) = {v}")
|
log(f"{name}: retry (self) = {v}")
|
||||||
if v == "pass":
|
if v == "pass":
|
||||||
_set_conf(0.85)
|
_set_conf(0.85)
|
||||||
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model)
|
skills_manager.set_audit(name, "pass", by_teacher=False, worker_model=model, owner=owner)
|
||||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.85, (refreshed or {}).get("necessity"), verdict)
|
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.85, (refreshed or {}).get("necessity"), verdict)
|
||||||
log(f"{name}: {status} — confidence 85% after self-edit")
|
log(f"{name}: {status} — confidence 85% after self-edit")
|
||||||
@@ -893,7 +899,9 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
log(f"{name}: retry on student after teacher rewrite = {v}")
|
log(f"{name}: retry on student after teacher rewrite = {v}")
|
||||||
if v == "pass":
|
if v == "pass":
|
||||||
_set_conf(0.8)
|
_set_conf(0.8)
|
||||||
skills_manager.set_audit(name, "pass", by_teacher=True, worker_model=model, teacher_model=t_model)
|
skills_manager.set_audit(
|
||||||
|
name, "pass", by_teacher=True, worker_model=model, teacher_model=t_model, owner=owner
|
||||||
|
)
|
||||||
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
refreshed = next((s for s in skills_manager.load(owner=owner) if s.get("name") == name), None)
|
||||||
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.8, (refreshed or {}).get("necessity"), verdict)
|
status = _audit_finalize_status(skills_manager, name, owner, "pass", 0.8, (refreshed or {}).get("necessity"), verdict)
|
||||||
log(f"{name}: {status} — confidence 80% after teacher rewrite")
|
log(f"{name}: {status} — confidence 80% after teacher rewrite")
|
||||||
@@ -908,6 +916,7 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
|
|||||||
name, v or "fail", by_teacher=teacher_ran,
|
name, v or "fail", by_teacher=teacher_ran,
|
||||||
worker_model=model,
|
worker_model=model,
|
||||||
teacher_model=(teacher[1] if teacher_ran and teacher else ""),
|
teacher_model=(teacher[1] if teacher_ran and teacher else ""),
|
||||||
|
owner=owner,
|
||||||
)
|
)
|
||||||
log(f"{name}: flagged — confidence lowered, kept as draft for manual review")
|
log(f"{name}: flagged — confidence lowered, kept as draft for manual review")
|
||||||
return {"skill": name, "result": "flagged", "verdict": verdict, "confidence": 0.35}
|
return {"skill": name, "result": "flagged", "verdict": verdict, "confidence": 0.35}
|
||||||
@@ -976,7 +985,7 @@ async def _run_audit_all_job(key, skills_manager, names, url, model, headers, te
|
|||||||
job.pop("task", None)
|
job.pop("task", None)
|
||||||
|
|
||||||
|
|
||||||
def _resolve_audit_models():
|
def _resolve_audit_models(owner=None):
|
||||||
"""Resolve (url, model, headers, teacher) for an audit run from Settings.
|
"""Resolve (url, model, headers, teacher) for an audit run from Settings.
|
||||||
|
|
||||||
Worker = Utility model (falling back to Default, normalized to a served
|
Worker = Utility model (falling back to Default, normalized to a served
|
||||||
@@ -985,7 +994,7 @@ def _resolve_audit_models():
|
|||||||
ValueError if no worker model.
|
ValueError if no worker model.
|
||||||
"""
|
"""
|
||||||
from src.endpoint_resolver import resolve_endpoint
|
from src.endpoint_resolver import resolve_endpoint
|
||||||
url, model, headers = resolve_endpoint("utility")
|
url, model, headers = resolve_endpoint("utility", owner=owner)
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
raise ValueError("No model configured — set a Default or Utility model in Settings.")
|
raise ValueError("No model configured — set a Default or Utility model in Settings.")
|
||||||
try:
|
try:
|
||||||
@@ -1029,7 +1038,7 @@ async def run_scheduled_skill_audit(skills_manager: SkillsManager,
|
|||||||
return {"status": "running", "skipped": True}
|
return {"status": "running", "skipped": True}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
url, model, headers, teacher = _resolve_audit_models()
|
url, model, headers, teacher = _resolve_audit_models(owner=owner)
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
logger.info(f"Scheduled skill audit skipped — {e}")
|
logger.info(f"Scheduled skill audit skipped — {e}")
|
||||||
return {"status": "skipped", "reason": str(e)}
|
return {"status": "skipped", "reason": str(e)}
|
||||||
@@ -1280,7 +1289,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
|||||||
|
|
||||||
# Prefer the configured DEFAULT (→ Utility) model — not the current chat
|
# Prefer the configured DEFAULT (→ Utility) model — not the current chat
|
||||||
# session's model. Fall back to the caller's session model only if unset.
|
# session's model. Fall back to the caller's session model only if unset.
|
||||||
url, model, headers = resolve_endpoint("default")
|
url, model, headers = resolve_endpoint("default", owner=user)
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
url = url or ((body.get("endpoint_url") or "").strip() or None)
|
url = url or ((body.get("endpoint_url") or "").strip() or None)
|
||||||
model = model or ((body.get("model") or "").strip() or None)
|
model = model or ((body.get("model") or "").strip() or None)
|
||||||
@@ -1360,7 +1369,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
|||||||
|
|
||||||
# Worker model (Default, normalized) + optional teacher — shared resolver.
|
# Worker model (Default, normalized) + optional teacher — shared resolver.
|
||||||
try:
|
try:
|
||||||
url, model, headers, teacher = _resolve_audit_models()
|
url, model, headers, teacher = _resolve_audit_models(owner=user)
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
raise HTTPException(400, str(e))
|
raise HTTPException(400, str(e))
|
||||||
|
|
||||||
@@ -1437,7 +1446,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
|||||||
@router.post("/{skill_id}/markdown")
|
@router.post("/{skill_id}/markdown")
|
||||||
async def save_skill_markdown(request: Request, skill_id: str):
|
async def save_skill_markdown(request: Request, skill_id: str):
|
||||||
"""Replace SKILL.md with new raw content. Parses + validates first."""
|
"""Replace SKILL.md with new raw content. Parses + validates first."""
|
||||||
from services.memory.skill_format import Skill, slugify
|
from services.memory.skill_format import Skill
|
||||||
user = _owner(request)
|
user = _owner(request)
|
||||||
body = await request.json()
|
body = await request.json()
|
||||||
new_content = body.get("markdown")
|
new_content = body.get("markdown")
|
||||||
@@ -1452,7 +1461,10 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
|||||||
sk = Skill.from_markdown(new_content)
|
sk = Skill.from_markdown(new_content)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
raise HTTPException(400, f"Could not parse SKILL.md: {e}")
|
raise HTTPException(400, f"Could not parse SKILL.md: {e}")
|
||||||
sk.name = slugify(sk.name or match.get("name"))
|
# Never rename on save: a changed `name` in the markdown would move
|
||||||
|
# the skill dir (update_skill) and orphan the original id, so a later
|
||||||
|
# delete 404s (#1333). Pin to the stored name, like _apply_skill_md.
|
||||||
|
sk.name = match.get("name")
|
||||||
if not sk.owner:
|
if not sk.owner:
|
||||||
sk.owner = match.get("owner") or user
|
sk.owner = match.get("owner") or user
|
||||||
ok = skills_manager.update_skill(match.get("name"), {
|
ok = skills_manager.update_skill(match.get("name"), {
|
||||||
|
|||||||
+10
-6
@@ -8,6 +8,7 @@ from typing import List
|
|||||||
import logging
|
import logging
|
||||||
from core.middleware import require_admin
|
from core.middleware import require_admin
|
||||||
from src.auth_helpers import get_current_user
|
from src.auth_helpers import get_current_user
|
||||||
|
from src.upload_handler import count_recent_uploads
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -25,14 +26,17 @@ def setup_upload_routes(upload_handler):
|
|||||||
client_ip = request.client.host if request.client else "unknown"
|
client_ip = request.client.host if request.client else "unknown"
|
||||||
out = []
|
out = []
|
||||||
|
|
||||||
# Limit concurrent uploads per IP
|
# Limit concurrent uploads per IP. Count genuine recent upload events —
|
||||||
ip_upload_count = sum(
|
# NOT the number of files in this batch. The previous check summed over
|
||||||
1 for f in files
|
# `files`, so a single multi-file request counted itself as N concurrent
|
||||||
if client_ip in upload_handler.upload_rate_log and
|
# uploads and tripped the limit (issue #1346: "attach more than one file
|
||||||
any(now > time.time() - 10 for now in upload_handler.upload_rate_log[client_ip][-len(files):])
|
# → the model doesn't even see them"). save_upload still enforces the
|
||||||
|
# per-minute sliding-window rate limit per file.
|
||||||
|
recent_uploads = count_recent_uploads(
|
||||||
|
upload_handler.upload_rate_log.get(client_ip, []), time.time()
|
||||||
)
|
)
|
||||||
|
|
||||||
if ip_upload_count >= upload_handler.max_concurrent_uploads:
|
if recent_uploads >= upload_handler.max_concurrent_uploads:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=429,
|
status_code=429,
|
||||||
detail=f"Maximum concurrent uploads ({upload_handler.max_concurrent_uploads}) exceeded"
|
detail=f"Maximum concurrent uploads ({upload_handler.max_concurrent_uploads}) exceeded"
|
||||||
|
|||||||
+8
-10
@@ -83,10 +83,9 @@ async def _run_bw(args: list, session: str = None, input_text: str = None,
|
|||||||
if session:
|
if session:
|
||||||
env["BW_SESSION"] = session
|
env["BW_SESSION"] = session
|
||||||
# Secrets must never be passed as argv — process arguments are world-readable
|
# Secrets must never be passed as argv — process arguments are world-readable
|
||||||
# via `ps` / `/proc/<pid>/cmdline` to any local user. Hand the master password
|
# via `ps` / `/proc/<pid>/cmdline` to any local user. Keep --passwordenv
|
||||||
# to `bw` through the environment instead (paired with `--passwordenv
|
# support for bw commands that need it; unlock/login callers should prefer
|
||||||
# BW_PASSWORD` in args); /proc/<pid>/environ is readable only by the process
|
# stdin so the master password is not left in the child environment either.
|
||||||
# owner. This mirrors how BW_SESSION is already passed above.
|
|
||||||
if bw_password is not None:
|
if bw_password is not None:
|
||||||
env["BW_PASSWORD"] = bw_password
|
env["BW_PASSWORD"] = bw_password
|
||||||
bw_path = _find_bw()
|
bw_path = _find_bw()
|
||||||
@@ -184,13 +183,12 @@ def setup_vault_routes():
|
|||||||
async def unlock(req: VaultUnlockRequest, request: Request):
|
async def unlock(req: VaultUnlockRequest, request: Request):
|
||||||
"""Unlock the vault and save the session key."""
|
"""Unlock the vault and save the session key."""
|
||||||
require_admin(request)
|
require_admin(request)
|
||||||
# Pass the master password via the environment (--passwordenv), NOT as
|
# Pass the master password on stdin, not argv. argv is visible through
|
||||||
# an argv element — argv is visible to every local user through `ps` /
|
# `ps` / /proc/<pid>/cmdline; stdin also avoids leaving the secret in
|
||||||
# /proc/<pid>/cmdline. (The sibling /login handler already keeps the
|
# the child process environment.
|
||||||
# password off argv by feeding it on stdin.)
|
|
||||||
stdout, stderr, rc = await _run_bw(
|
stdout, stderr, rc = await _run_bw(
|
||||||
["unlock", "--passwordenv", "BW_PASSWORD", "--raw"],
|
["unlock", "--raw"],
|
||||||
bw_password=req.master_password,
|
input_text=req.master_password + "\n",
|
||||||
)
|
)
|
||||||
if rc != 0:
|
if rc != 0:
|
||||||
return {"ok": False, "error": f"Unlock failed: {stderr[:300]}"}
|
return {"ok": False, "error": f"Unlock failed: {stderr[:300]}"}
|
||||||
|
|||||||
@@ -39,19 +39,19 @@ Environment:
|
|||||||
USAGE
|
USAGE
|
||||||
}
|
}
|
||||||
|
|
||||||
case "${1:-}" in
|
for _arg in "$@"; do
|
||||||
--help|-h)
|
case "${_arg}" in
|
||||||
_usage
|
--help|-h)
|
||||||
exit 0
|
_usage
|
||||||
;;
|
exit 0
|
||||||
"")
|
;;
|
||||||
;;
|
*)
|
||||||
*)
|
printf 'Unknown option: %s\n\n' "${_arg}" >&2
|
||||||
printf 'Unknown option: %s\n\n' "$1" >&2
|
_usage >&2
|
||||||
_usage >&2
|
exit 1
|
||||||
exit 1
|
;;
|
||||||
;;
|
esac
|
||||||
esac
|
done
|
||||||
|
|
||||||
_find_cmd() {
|
_find_cmd() {
|
||||||
if command -v "$1" >/dev/null 2>&1; then
|
if command -v "$1" >/dev/null 2>&1; then
|
||||||
|
|||||||
@@ -58,10 +58,12 @@ def main():
|
|||||||
count = db.query(Session).filter(Session.owner == None).update({"owner": owner})
|
count = db.query(Session).filter(Session.owner == None).update({"owner": owner})
|
||||||
print(f" sessions: claimed {count}")
|
print(f" sessions: claimed {count}")
|
||||||
|
|
||||||
# Documents
|
# Documents (have their own owner column; claim the ownerless ones,
|
||||||
count = db.query(Document).filter(Document.session_id.in_(
|
# mirroring the sessions/gallery/comparisons blocks). The old query set
|
||||||
db.query(Session.id).filter(Session.owner == owner)
|
# session_id to itself — a no-op — and never set owner, so ownerless
|
||||||
)).update({"session_id": Document.session_id}, synchronize_session=False)
|
# documents stayed ownerless and invisible in the user's Library.
|
||||||
|
count = db.query(Document).filter(Document.owner == None).update({"owner": owner})
|
||||||
|
print(f" documents: claimed {count}")
|
||||||
|
|
||||||
# Gallery
|
# Gallery
|
||||||
if GalleryImage:
|
if GalleryImage:
|
||||||
|
|||||||
+6
-2
@@ -68,6 +68,10 @@ def _short_help(path: Path) -> str:
|
|||||||
return first
|
return first
|
||||||
|
|
||||||
|
|
||||||
|
def _is_runnable_subcommand(path: Path) -> bool:
|
||||||
|
return path.exists() and path.is_file() and os.access(path, os.X_OK)
|
||||||
|
|
||||||
|
|
||||||
def _print_listing() -> None:
|
def _print_listing() -> None:
|
||||||
"""`odysseus` with no args (or `odysseus help`) — print the table."""
|
"""`odysseus` with no args (or `odysseus help`) — print the table."""
|
||||||
sys.stdout.write(f"odysseus {VERSION} — every feature, on the shell.\n\n")
|
sys.stdout.write(f"odysseus {VERSION} — every feature, on the shell.\n\n")
|
||||||
@@ -101,7 +105,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
_print_listing()
|
_print_listing()
|
||||||
return 0
|
return 0
|
||||||
sub = SCRIPTS_DIR / f"odysseus-{argv[1]}"
|
sub = SCRIPTS_DIR / f"odysseus-{argv[1]}"
|
||||||
if not sub.exists():
|
if not _is_runnable_subcommand(sub):
|
||||||
sys.stderr.write(f"odysseus: unknown subcommand {argv[1]!r}\n")
|
sys.stderr.write(f"odysseus: unknown subcommand {argv[1]!r}\n")
|
||||||
return 1
|
return 1
|
||||||
return subprocess.call([str(sub), "--help"])
|
return subprocess.call([str(sub), "--help"])
|
||||||
@@ -109,7 +113,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
# `odysseus foo ...` → exec `odysseus-foo ...` under the project venv.
|
# `odysseus foo ...` → exec `odysseus-foo ...` under the project venv.
|
||||||
name = argv[0]
|
name = argv[0]
|
||||||
sub = SCRIPTS_DIR / f"odysseus-{name}"
|
sub = SCRIPTS_DIR / f"odysseus-{name}"
|
||||||
if not sub.exists():
|
if not _is_runnable_subcommand(sub):
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
f"odysseus: unknown subcommand {name!r}. "
|
f"odysseus: unknown subcommand {name!r}. "
|
||||||
f"Try `odysseus help` to see available ones.\n"
|
f"Try `odysseus help` to see available ones.\n"
|
||||||
|
|||||||
@@ -56,6 +56,16 @@ def _sqlite_safe_copy(src: Path, dst: Path) -> None:
|
|||||||
dst.write_bytes(src.read_bytes())
|
dst.write_bytes(src.read_bytes())
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_output_inside_data(out_path: Path) -> None:
|
||||||
|
try:
|
||||||
|
resolved = out_path.resolve()
|
||||||
|
data_root = _DATA_DIR.resolve()
|
||||||
|
resolved.relative_to(data_root)
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
fail("backup output path must be outside data/")
|
||||||
|
|
||||||
|
|
||||||
def cmd_snapshot(args):
|
def cmd_snapshot(args):
|
||||||
"""Write a tar.gz of the entire data/ directory.
|
"""Write a tar.gz of the entire data/ directory.
|
||||||
|
|
||||||
@@ -68,6 +78,7 @@ def cmd_snapshot(args):
|
|||||||
out_path = Path(args.out) if args.out else (
|
out_path = Path(args.out) if args.out else (
|
||||||
_BACKUP_DIR / f"odysseus-backup-{datetime.now().strftime('%Y%m%d-%H%M%S')}.tar.gz"
|
_BACKUP_DIR / f"odysseus-backup-{datetime.now().strftime('%Y%m%d-%H%M%S')}.tar.gz"
|
||||||
)
|
)
|
||||||
|
_reject_output_inside_data(out_path)
|
||||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
sqlite_dbs = [p for p in _DATA_DIR.rglob("*.db") if p.is_file() and not p.is_symlink()]
|
sqlite_dbs = [p for p in _DATA_DIR.rglob("*.db") if p.is_file() and not p.is_symlink()]
|
||||||
|
|||||||
@@ -30,11 +30,19 @@ except ModuleNotFoundError as e:
|
|||||||
sys.exit(2)
|
sys.exit(2)
|
||||||
|
|
||||||
|
|
||||||
|
def _preview_text(value, limit: int = 200) -> str:
|
||||||
|
"""Truncated preview tolerant of non-string values. A gallery row whose
|
||||||
|
``prompt`` is a non-string would crash ``(value or "")[:200]`` with a
|
||||||
|
TypeError; coerce non-strings to ""."""
|
||||||
|
text = value if isinstance(value, str) else ""
|
||||||
|
return text[:limit]
|
||||||
|
|
||||||
|
|
||||||
def _serialize_image(i: "GalleryImage") -> dict:
|
def _serialize_image(i: "GalleryImage") -> dict:
|
||||||
return {
|
return {
|
||||||
"id": i.id,
|
"id": i.id,
|
||||||
"filename": i.filename,
|
"filename": i.filename,
|
||||||
"prompt": (i.prompt or "")[:200],
|
"prompt": _preview_text(i.prompt),
|
||||||
"model": i.model or "",
|
"model": i.model or "",
|
||||||
"size": i.size or "",
|
"size": i.size or "",
|
||||||
"tags": i.tags or "",
|
"tags": i.tags or "",
|
||||||
|
|||||||
@@ -58,6 +58,8 @@ def _resolve(name: str) -> Path | None:
|
|||||||
"""Match a log by exact filename, basename-without-extension, or
|
"""Match a log by exact filename, basename-without-extension, or
|
||||||
substring. Returns the most-recently-modified match if there are
|
substring. Returns the most-recently-modified match if there are
|
||||||
ties."""
|
ties."""
|
||||||
|
if not isinstance(name, str):
|
||||||
|
return None
|
||||||
candidates = []
|
candidates = []
|
||||||
for base in (_APP_LOGS, _TMUX_LOGS):
|
for base in (_APP_LOGS, _TMUX_LOGS):
|
||||||
if not base.is_dir():
|
if not base.is_dir():
|
||||||
|
|||||||
+14
-5
@@ -107,6 +107,19 @@ def _q(name: str) -> str:
|
|||||||
return '"' + (name or "").replace("\\", "\\\\").replace('"', '\\"') + '"'
|
return '"' + (name or "").replace("\\", "\\\\").replace('"', '\\"') + '"'
|
||||||
|
|
||||||
|
|
||||||
|
def _split_recipients(value: str) -> list[str]:
|
||||||
|
return [r.strip() for r in (value or "").split(",") if r.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _recipient_list(to: str, cc: str = "", bcc: str = "") -> list[str]:
|
||||||
|
recipients = _split_recipients(to)
|
||||||
|
recipients.extend(_split_recipients(cc))
|
||||||
|
recipients.extend(_split_recipients(bcc))
|
||||||
|
if not recipients:
|
||||||
|
fail("at least one recipient is required")
|
||||||
|
return recipients
|
||||||
|
|
||||||
|
|
||||||
# ─── list ────────────────────────────────────────────────────────────
|
# ─── list ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def cmd_list(args) -> None:
|
def cmd_list(args) -> None:
|
||||||
@@ -302,11 +315,7 @@ def cmd_send(args) -> None:
|
|||||||
outer["Date"] = datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S +0000")
|
outer["Date"] = datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S +0000")
|
||||||
outer.attach(MIMEText(body, "plain", "utf-8"))
|
outer.attach(MIMEText(body, "plain", "utf-8"))
|
||||||
|
|
||||||
recipients = [r.strip() for r in args.to.split(",") if r.strip()]
|
recipients = _recipient_list(args.to, args.cc, args.bcc)
|
||||||
if args.cc:
|
|
||||||
recipients.extend([r.strip() for r in args.cc.split(",") if r.strip()])
|
|
||||||
if args.bcc:
|
|
||||||
recipients.extend([r.strip() for r in args.bcc.split(",") if r.strip()])
|
|
||||||
|
|
||||||
if args.dry_run:
|
if args.dry_run:
|
||||||
emit({
|
emit({
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ def _serialize(s: "McpServer", redact_env: bool = True) -> dict:
|
|||||||
env_obj = json.loads(s.env) if s.env else {}
|
env_obj = json.loads(s.env) if s.env else {}
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
env_obj = {}
|
env_obj = {}
|
||||||
if redact_env and env_obj:
|
if redact_env and isinstance(env_obj, dict):
|
||||||
env_obj = {k: ("***" if v else "") for k, v in env_obj.items()}
|
env_obj = {k: ("***" if v else "") for k, v in env_obj.items()}
|
||||||
return {
|
return {
|
||||||
"id": s.id,
|
"id": s.id,
|
||||||
|
|||||||
+16
-7
@@ -28,9 +28,12 @@ def _load() -> dict:
|
|||||||
if not _PATH.exists():
|
if not _PATH.exists():
|
||||||
return {}
|
return {}
|
||||||
try:
|
try:
|
||||||
return json.loads(_PATH.read_text())
|
data = json.loads(_PATH.read_text())
|
||||||
except json.JSONDecodeError as e:
|
except json.JSONDecodeError as e:
|
||||||
fail(f"presets.json corrupt: {e}")
|
fail(f"presets.json corrupt: {e}")
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
fail("presets.json corrupt: expected an object")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
def _save(data: dict) -> None:
|
def _save(data: dict) -> None:
|
||||||
@@ -46,6 +49,15 @@ def _save(data: dict) -> None:
|
|||||||
tmp.replace(_PATH)
|
tmp.replace(_PATH)
|
||||||
|
|
||||||
|
|
||||||
|
def _entry_or_fail(presets: dict, name: str) -> dict:
|
||||||
|
if name not in presets:
|
||||||
|
fail(f"no preset named {name!r}")
|
||||||
|
entry = presets[name]
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
fail(f"preset {name!r} is corrupt: expected an object")
|
||||||
|
return entry
|
||||||
|
|
||||||
|
|
||||||
def cmd_list(args):
|
def cmd_list(args):
|
||||||
presets = _load()
|
presets = _load()
|
||||||
rows = []
|
rows = []
|
||||||
@@ -63,9 +75,7 @@ def cmd_list(args):
|
|||||||
|
|
||||||
def cmd_get(args):
|
def cmd_get(args):
|
||||||
presets = _load()
|
presets = _load()
|
||||||
if args.name not in presets:
|
emit({"id": args.name, **_entry_or_fail(presets, args.name)}, args)
|
||||||
fail(f"no preset named {args.name!r}")
|
|
||||||
emit({"id": args.name, **presets[args.name]}, args)
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_set(args):
|
def cmd_set(args):
|
||||||
@@ -90,9 +100,8 @@ def cmd_set(args):
|
|||||||
|
|
||||||
def cmd_delete(args):
|
def cmd_delete(args):
|
||||||
presets = _load()
|
presets = _load()
|
||||||
if args.name not in presets:
|
snap = _entry_or_fail(presets, args.name)
|
||||||
fail(f"no preset named {args.name!r}")
|
presets.pop(args.name)
|
||||||
snap = presets.pop(args.name)
|
|
||||||
_save(presets)
|
_save(presets)
|
||||||
emit({"ok": True, "deleted": {"id": args.name, **snap}}, args)
|
emit({"ok": True, "deleted": {"id": args.name, **snap}}, args)
|
||||||
|
|
||||||
|
|||||||
@@ -36,10 +36,18 @@ def _load(rp_id: str) -> dict | None:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _preview_text(value, limit: int = 200) -> str:
|
||||||
|
"""Truncated preview tolerant of non-string values. A stored research
|
||||||
|
record whose ``query`` is a non-string (legacy/corrupt JSON) would crash
|
||||||
|
``(value or "")[:200]`` with a TypeError; coerce non-strings to ""."""
|
||||||
|
text = value if isinstance(value, str) else ""
|
||||||
|
return text[:limit]
|
||||||
|
|
||||||
|
|
||||||
def _summarize(rp_id: str, data: dict) -> dict:
|
def _summarize(rp_id: str, data: dict) -> dict:
|
||||||
return {
|
return {
|
||||||
"id": rp_id,
|
"id": rp_id,
|
||||||
"query": (data.get("query") or "")[:200],
|
"query": _preview_text(data.get("query")),
|
||||||
"category": data.get("category") or "",
|
"category": data.get("category") or "",
|
||||||
"status": data.get("status") or "",
|
"status": data.get("status") or "",
|
||||||
"started_at": data.get("started_at") or "",
|
"started_at": data.get("started_at") or "",
|
||||||
|
|||||||
@@ -27,6 +27,12 @@ except ModuleNotFoundError as e:
|
|||||||
|
|
||||||
|
|
||||||
def _serialize(s: "DbSession") -> dict:
|
def _serialize(s: "DbSession") -> dict:
|
||||||
|
def _int_or_zero(value) -> int:
|
||||||
|
try:
|
||||||
|
return int(value or 0)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return 0
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"id": s.id,
|
"id": s.id,
|
||||||
"name": s.name,
|
"name": s.name,
|
||||||
@@ -37,9 +43,9 @@ def _serialize(s: "DbSession") -> dict:
|
|||||||
"archived": bool(s.archived),
|
"archived": bool(s.archived),
|
||||||
"rag": bool(s.rag),
|
"rag": bool(s.rag),
|
||||||
"is_important": bool(s.is_important),
|
"is_important": bool(s.is_important),
|
||||||
"message_count": s.message_count or 0,
|
"message_count": _int_or_zero(s.message_count),
|
||||||
"total_input_tokens": s.total_input_tokens or 0,
|
"total_input_tokens": _int_or_zero(s.total_input_tokens),
|
||||||
"total_output_tokens": s.total_output_tokens or 0,
|
"total_output_tokens": _int_or_zero(s.total_output_tokens),
|
||||||
"last_accessed": s.last_accessed.isoformat() if s.last_accessed else "",
|
"last_accessed": s.last_accessed.isoformat() if s.last_accessed else "",
|
||||||
"created_at": s.created_at.isoformat() if s.created_at else "",
|
"created_at": s.created_at.isoformat() if s.created_at else "",
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -29,6 +29,16 @@ except ModuleNotFoundError as e:
|
|||||||
sys.exit(2)
|
sys.exit(2)
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_png_data(data_png: str) -> bytes:
|
||||||
|
raw = data_png or ""
|
||||||
|
if "," in raw:
|
||||||
|
raw = raw.split(",", 1)[1]
|
||||||
|
try:
|
||||||
|
return base64.b64decode(raw, validate=True)
|
||||||
|
except Exception as e:
|
||||||
|
fail(f"data_png is not valid base64: {e}")
|
||||||
|
|
||||||
|
|
||||||
def cmd_list(args):
|
def cmd_list(args):
|
||||||
"""No `Signature` SQLAlchemy model is registered for the
|
"""No `Signature` SQLAlchemy model is registered for the
|
||||||
`signatures` table — query via raw SQL so we don't depend on it."""
|
`signatures` table — query via raw SQL so we don't depend on it."""
|
||||||
@@ -85,13 +95,7 @@ def cmd_export(args):
|
|||||||
), {"id": args.id}).mappings().first()
|
), {"id": args.id}).mappings().first()
|
||||||
if not row:
|
if not row:
|
||||||
fail(f"no signature with id {args.id!r}")
|
fail(f"no signature with id {args.id!r}")
|
||||||
raw = row["data_png"] or ""
|
png_bytes = _decode_png_data(row["data_png"] or "")
|
||||||
if "," in raw:
|
|
||||||
raw = raw.split(",", 1)[1]
|
|
||||||
try:
|
|
||||||
png_bytes = base64.b64decode(raw)
|
|
||||||
except Exception as e:
|
|
||||||
fail(f"data_png is not valid base64: {e}")
|
|
||||||
out = Path(args.png)
|
out = Path(args.png)
|
||||||
out.parent.mkdir(parents=True, exist_ok=True)
|
out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
out.write_bytes(png_bytes)
|
out.write_bytes(png_bytes)
|
||||||
|
|||||||
+12
-1
@@ -41,11 +41,22 @@ def _manager() -> SkillsManager:
|
|||||||
return _mgr
|
return _mgr
|
||||||
|
|
||||||
|
|
||||||
|
def _preview_text(value, limit: int = 200) -> str:
|
||||||
|
"""Truncated preview of a text field, tolerant of non-string values.
|
||||||
|
|
||||||
|
A skill whose ``description`` is a non-string (e.g. a number from a
|
||||||
|
hand-edited/legacy store) would crash ``(value or "")[:200]`` with a
|
||||||
|
TypeError; coerce non-strings to "" instead.
|
||||||
|
"""
|
||||||
|
text = value if isinstance(value, str) else ""
|
||||||
|
return text[:limit]
|
||||||
|
|
||||||
|
|
||||||
def _summary(skill: dict) -> dict:
|
def _summary(skill: dict) -> dict:
|
||||||
return {
|
return {
|
||||||
"name": skill.get("name", ""),
|
"name": skill.get("name", ""),
|
||||||
"category": skill.get("category", "general"),
|
"category": skill.get("category", "general"),
|
||||||
"description": (skill.get("description") or "")[:200],
|
"description": _preview_text(skill.get("description")),
|
||||||
"status": skill.get("status", ""),
|
"status": skill.get("status", ""),
|
||||||
"uses": skill.get("uses", 0),
|
"uses": skill.get("uses", 0),
|
||||||
"last_used": skill.get("last_used") or "",
|
"last_used": skill.get("last_used") or "",
|
||||||
|
|||||||
@@ -166,116 +166,3 @@ def update_database():
|
|||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
update_database()
|
update_database()
|
||||||
"""
|
|
||||||
update_database.py
|
|
||||||
|
|
||||||
This script updates the database schema by adding new columns to the sessions table
|
|
||||||
if they don't already exist. It uses raw SQL ALTER TABLE statements to modify
|
|
||||||
the existing SQLite database.
|
|
||||||
|
|
||||||
The following columns are added:
|
|
||||||
- last_accessed (DateTime): Set to created_at for existing records
|
|
||||||
- is_important (Boolean): Set to False for existing records
|
|
||||||
- message_count (Integer): Calculated from the number of messages in chat_messages table
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python update_database.py
|
|
||||||
"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
from datetime import datetime
|
|
||||||
from sqlalchemy import create_engine, text
|
|
||||||
from database import DATABASE_URL, SessionLocal
|
|
||||||
|
|
||||||
def update_database():
|
|
||||||
"""Update the database schema and populate new columns."""
|
|
||||||
# Create engine from DATABASE_URL
|
|
||||||
engine = create_engine(DATABASE_URL)
|
|
||||||
|
|
||||||
# Start a transaction
|
|
||||||
db = SessionLocal()
|
|
||||||
try:
|
|
||||||
# Add last_accessed column if it doesn't exist
|
|
||||||
try:
|
|
||||||
with engine.connect() as conn:
|
|
||||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN last_accessed DATETIME"))
|
|
||||||
conn.commit()
|
|
||||||
print("Added last_accessed column to sessions table")
|
|
||||||
except Exception as e:
|
|
||||||
if "duplicate column name" in str(e).lower():
|
|
||||||
print("last_accessed column already exists")
|
|
||||||
else:
|
|
||||||
print(f"Error adding last_accessed column: {e}")
|
|
||||||
|
|
||||||
# Add is_important column if it doesn't exist
|
|
||||||
try:
|
|
||||||
with engine.connect() as conn:
|
|
||||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN is_important BOOLEAN DEFAULT FALSE"))
|
|
||||||
conn.commit()
|
|
||||||
print("Added is_important column to sessions table")
|
|
||||||
except Exception as e:
|
|
||||||
if "duplicate column name" in str(e).lower():
|
|
||||||
print("is_important column already exists")
|
|
||||||
else:
|
|
||||||
print(f"Error adding is_important column: {e}")
|
|
||||||
|
|
||||||
# Add message_count column if it doesn't exist
|
|
||||||
try:
|
|
||||||
with engine.connect() as conn:
|
|
||||||
conn.execute(text("ALTER TABLE sessions ADD COLUMN message_count INTEGER DEFAULT 0"))
|
|
||||||
conn.commit()
|
|
||||||
print("Added message_count column to sessions table")
|
|
||||||
except Exception as e:
|
|
||||||
if "duplicate column name" in str(e).lower():
|
|
||||||
print("message_count column already exists")
|
|
||||||
else:
|
|
||||||
print(f"Error adding message_count column: {e}")
|
|
||||||
|
|
||||||
# Populate last_accessed with created_at for existing records where last_accessed is NULL
|
|
||||||
print("Populating last_accessed column...")
|
|
||||||
with engine.connect() as conn:
|
|
||||||
conn.execute(text("""
|
|
||||||
UPDATE sessions
|
|
||||||
SET last_accessed = created_at
|
|
||||||
WHERE last_accessed IS NULL
|
|
||||||
"""))
|
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
# Populate is_important with FALSE for existing records where is_important is NULL
|
|
||||||
print("Populating is_important column...")
|
|
||||||
with engine.connect() as conn:
|
|
||||||
conn.execute(text("""
|
|
||||||
UPDATE sessions
|
|
||||||
SET is_important = 0
|
|
||||||
WHERE is_important IS NULL
|
|
||||||
"""))
|
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
# Calculate and populate message_count from chat_messages table
|
|
||||||
print("Calculating and populating message_count column...")
|
|
||||||
with engine.connect() as conn:
|
|
||||||
# First, set all message_count to 0
|
|
||||||
conn.execute(text("UPDATE sessions SET message_count = 0"))
|
|
||||||
|
|
||||||
# Then, count messages for each session and update
|
|
||||||
conn.execute(text("""
|
|
||||||
UPDATE sessions
|
|
||||||
SET message_count = (
|
|
||||||
SELECT COUNT(*)
|
|
||||||
FROM chat_messages
|
|
||||||
WHERE chat_messages.session_id = sessions.id
|
|
||||||
)
|
|
||||||
"""))
|
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
print("Database update completed successfully!")
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Error updating database: {e}")
|
|
||||||
db.rollback()
|
|
||||||
raise
|
|
||||||
finally:
|
|
||||||
db.close()
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
update_database()
|
|
||||||
|
|||||||
@@ -105,6 +105,8 @@ def _detect_nvidia():
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
gpus = []
|
gpus = []
|
||||||
|
# Devices nvidia-smi lists with a real name but a non-numeric memory.total.
|
||||||
|
unified = []
|
||||||
# nvidia-smi lists GPUs in index order (0,1,2,...), so the row position is
|
# nvidia-smi lists GPUs in index order (0,1,2,...), so the row position is
|
||||||
# the CUDA device index we'd pass to CUDA_VISIBLE_DEVICES.
|
# the CUDA device index we'd pass to CUDA_VISIBLE_DEVICES.
|
||||||
for idx, line in enumerate(out.strip().split("\n")):
|
for idx, line in enumerate(out.strip().split("\n")):
|
||||||
@@ -114,9 +116,32 @@ def _detect_nvidia():
|
|||||||
vram_mb = float(parts[0])
|
vram_mb = float(parts[0])
|
||||||
gpus.append({"index": idx, "name": parts[1], "vram_gb": vram_mb / 1024.0})
|
gpus.append({"index": idx, "name": parts[1], "vram_gb": vram_mb / 1024.0})
|
||||||
except ValueError:
|
except ValueError:
|
||||||
|
# Grace Blackwell GB10 / DGX Spark and other unified-memory
|
||||||
|
# NVIDIA parts report memory.total as "[N/A]"/"Not Supported"
|
||||||
|
# because the GPU shares the system LPDDR pool instead of
|
||||||
|
# carrying discrete VRAM. Don't drop the device — remember it so
|
||||||
|
# we report a unified-memory GPU below rather than "No GPU" (#1340).
|
||||||
|
if parts[1]:
|
||||||
|
unified.append({"index": idx, "name": parts[1]})
|
||||||
continue
|
continue
|
||||||
|
|
||||||
if not gpus:
|
if not gpus:
|
||||||
|
if unified:
|
||||||
|
# Unified-memory CUDA box: report the GPU backed by system RAM so the
|
||||||
|
# Cookbook recommends models and serving works. The pool is shared
|
||||||
|
# (not per-GPU discrete VRAM), so report the RAM total once.
|
||||||
|
ram_gb = round(_get_ram_gb(), 1)
|
||||||
|
gpus = [{"index": g["index"], "name": g["name"], "vram_gb": ram_gb} for g in unified]
|
||||||
|
return {
|
||||||
|
"gpu_name": gpus[0]["name"],
|
||||||
|
"gpu_vram_gb": ram_gb,
|
||||||
|
"gpu_count": len(gpus),
|
||||||
|
"gpus": gpus,
|
||||||
|
"gpu_groups": _group_gpus(gpus),
|
||||||
|
"homogeneous": True,
|
||||||
|
"backend": "cuda",
|
||||||
|
"unified_memory": True,
|
||||||
|
}
|
||||||
return None
|
return None
|
||||||
total_vram = sum(g["vram_gb"] for g in gpus)
|
total_vram = sum(g["vram_gb"] for g in gpus)
|
||||||
groups = _group_gpus(gpus)
|
groups = _group_gpus(gpus)
|
||||||
|
|||||||
+36
-15
@@ -6,8 +6,8 @@ YAML frontmatter and a structured markdown body (When to Use / Procedure /
|
|||||||
Pitfalls / Verification). See `skill_format.py` for the format.
|
Pitfalls / Verification). See `skill_format.py` for the format.
|
||||||
|
|
||||||
Usage counters (`uses`, `last_used`) live in a sidecar
|
Usage counters (`uses`, `last_used`) live in a sidecar
|
||||||
`data/skills/_usage.json` keyed by skill name so the SKILL.md content
|
`data/skills/_usage.json` keyed by owner plus skill name so the SKILL.md
|
||||||
doesn't churn on every retrieval.
|
content doesn't churn on every retrieval.
|
||||||
|
|
||||||
Ownership: skills declare `owner: <username>` in frontmatter. Single-user
|
Ownership: skills declare `owner: <username>` in frontmatter. Single-user
|
||||||
deployments can leave that blank.
|
deployments can leave that blank.
|
||||||
@@ -105,14 +105,29 @@ class SkillsManager:
|
|||||||
json.dump(usage, f, indent=2)
|
json.dump(usage, f, indent=2)
|
||||||
os.replace(tmp, self.usage_file)
|
os.replace(tmp, self.usage_file)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _usage_key(name: str, owner: Optional[str] = None) -> str:
|
||||||
|
# Skill names are not globally unique once multiple owners are present.
|
||||||
|
# Keep the usage sidecar keyed the same way the skill file is scoped.
|
||||||
|
return f"{owner}::{name}" if owner else name
|
||||||
|
|
||||||
|
def _usage_entry(self, usage: Dict[str, Dict], name: str, owner: Optional[str] = None) -> Dict:
|
||||||
|
key = self._usage_key(name, owner)
|
||||||
|
entry = usage.get(key)
|
||||||
|
if isinstance(entry, dict):
|
||||||
|
return entry
|
||||||
|
return {}
|
||||||
|
|
||||||
def set_audit(self, name: str, verdict: str, by_teacher: bool = False,
|
def set_audit(self, name: str, verdict: str, by_teacher: bool = False,
|
||||||
worker_model: str = "", teacher_model: str = "") -> None:
|
worker_model: str = "", teacher_model: str = "",
|
||||||
|
owner: Optional[str] = None) -> None:
|
||||||
"""Record the last test/audit result for a skill in the usage sidecar
|
"""Record the last test/audit result for a skill in the usage sidecar
|
||||||
(so it surfaces in load() without touching SKILL.md). Drives the
|
(so it surfaces in load() without touching SKILL.md). Drives the
|
||||||
'verified' check + teacher mark on the card."""
|
'verified' check + teacher mark on the card."""
|
||||||
import time as _t
|
import time as _t
|
||||||
usage = self._load_usage()
|
usage = self._load_usage()
|
||||||
e = usage.setdefault(name, {"uses": 0, "last_used": None})
|
key = self._usage_key(name, owner)
|
||||||
|
e = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||||
e["audit_verdict"] = verdict
|
e["audit_verdict"] = verdict
|
||||||
e["audit_by_teacher"] = bool(by_teacher)
|
e["audit_by_teacher"] = bool(by_teacher)
|
||||||
if worker_model:
|
if worker_model:
|
||||||
@@ -123,11 +138,13 @@ class SkillsManager:
|
|||||||
self._save_usage(usage)
|
self._save_usage(usage)
|
||||||
|
|
||||||
def set_necessity(self, name: str, necessary: bool,
|
def set_necessity(self, name: str, necessary: bool,
|
||||||
redundant_with=None, reason: str = "") -> None:
|
redundant_with=None, reason: str = "",
|
||||||
|
owner: Optional[str] = None) -> None:
|
||||||
"""Record the advisory 'is this skill necessary?' judgment in the usage
|
"""Record the advisory 'is this skill necessary?' judgment in the usage
|
||||||
sidecar. Surfaced on the card as a flag; never acts on the skill."""
|
sidecar. Surfaced on the card as a flag; never acts on the skill."""
|
||||||
usage = self._load_usage()
|
usage = self._load_usage()
|
||||||
e = usage.setdefault(name, {"uses": 0, "last_used": None})
|
key = self._usage_key(name, owner)
|
||||||
|
e = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||||
e["necessity"] = {
|
e["necessity"] = {
|
||||||
"necessary": bool(necessary),
|
"necessary": bool(necessary),
|
||||||
"redundant_with": list(redundant_with or []),
|
"redundant_with": list(redundant_with or []),
|
||||||
@@ -207,7 +224,7 @@ class SkillsManager:
|
|||||||
if not sk:
|
if not sk:
|
||||||
continue
|
continue
|
||||||
d = sk.to_dict()
|
d = sk.to_dict()
|
||||||
u = usage.get(sk.name) or {}
|
u = self._usage_entry(usage, sk.name, sk.owner)
|
||||||
d["uses"] = int(u.get("uses", 0))
|
d["uses"] = int(u.get("uses", 0))
|
||||||
d["last_used"] = u.get("last_used")
|
d["last_used"] = u.get("last_used")
|
||||||
d["audit_verdict"] = u.get("audit_verdict")
|
d["audit_verdict"] = u.get("audit_verdict")
|
||||||
@@ -308,6 +325,7 @@ class SkillsManager:
|
|||||||
# never auto-skipped — a human asked for it. The every-X AI audit
|
# never auto-skipped — a human asked for it. The every-X AI audit
|
||||||
# handles the fuzzier near-duplicates this cheap check won't catch.
|
# handles the fuzzier near-duplicates this cheap check won't catch.
|
||||||
_all = self.load_all()
|
_all = self.load_all()
|
||||||
|
_dedup_pool = _all if owner is None else [s for s in _all if s.get("owner") == owner]
|
||||||
if source != "user":
|
if source != "user":
|
||||||
cand = _tokenize(" ".join([
|
cand = _tokenize(" ".join([
|
||||||
nm, (description or title or ""),
|
nm, (description or title or ""),
|
||||||
@@ -315,7 +333,7 @@ class SkillsManager:
|
|||||||
" ".join(procedure if procedure is not None else (steps or [])),
|
" ".join(procedure if procedure is not None else (steps or [])),
|
||||||
]))
|
]))
|
||||||
if cand:
|
if cand:
|
||||||
for s in _all:
|
for s in _dedup_pool:
|
||||||
ex = _tokenize(" ".join([
|
ex = _tokenize(" ".join([
|
||||||
s.get("name", ""), s.get("description", ""),
|
s.get("name", ""), s.get("description", ""),
|
||||||
s.get("when_to_use", ""),
|
s.get("when_to_use", ""),
|
||||||
@@ -326,7 +344,7 @@ class SkillsManager:
|
|||||||
# existing skill's usage and return it so the caller
|
# existing skill's usage and return it so the caller
|
||||||
# knows it already exists.
|
# knows it already exists.
|
||||||
try:
|
try:
|
||||||
self.record_use(s["name"])
|
self.record_use(s["name"], owner=s.get("owner"))
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
return {**s, "_deduped": True, "_duplicate_of": s.get("name")}
|
return {**s, "_deduped": True, "_duplicate_of": s.get("name")}
|
||||||
@@ -428,8 +446,9 @@ class SkillsManager:
|
|||||||
os.rename(old_dir, new_dir)
|
os.rename(old_dir, new_dir)
|
||||||
# Also rename usage key
|
# Also rename usage key
|
||||||
usage = self._load_usage()
|
usage = self._load_usage()
|
||||||
if skill_id in usage:
|
old_usage_key = self._usage_key(skill_id, sk.owner)
|
||||||
usage[sk.name] = usage.pop(skill_id)
|
if old_usage_key in usage:
|
||||||
|
usage[self._usage_key(sk.name, sk.owner)] = usage.pop(old_usage_key)
|
||||||
self._save_usage(usage)
|
self._save_usage(usage)
|
||||||
self._write_skill(sk)
|
self._write_skill(sk)
|
||||||
return True
|
return True
|
||||||
@@ -455,15 +474,17 @@ class SkillsManager:
|
|||||||
logger.warning(f"Failed to remove skill dir {skill_dir}: {e}")
|
logger.warning(f"Failed to remove skill dir {skill_dir}: {e}")
|
||||||
return False
|
return False
|
||||||
usage = self._load_usage()
|
usage = self._load_usage()
|
||||||
if skill_id in usage:
|
usage_key = self._usage_key(skill_id, sk.owner)
|
||||||
del usage[skill_id]
|
if usage_key in usage:
|
||||||
|
del usage[usage_key]
|
||||||
self._save_usage(usage)
|
self._save_usage(usage)
|
||||||
return True
|
return True
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def record_use(self, skill_id: str) -> None:
|
def record_use(self, skill_id: str, owner: Optional[str] = None) -> None:
|
||||||
usage = self._load_usage()
|
usage = self._load_usage()
|
||||||
entry = usage.setdefault(skill_id, {"uses": 0, "last_used": None})
|
key = self._usage_key(skill_id, owner)
|
||||||
|
entry = usage.setdefault(key, {"uses": 0, "last_used": None})
|
||||||
entry["uses"] = int(entry.get("uses", 0)) + 1
|
entry["uses"] = int(entry.get("uses", 0)) + 1
|
||||||
entry["last_used"] = int(time.time())
|
entry["last_used"] = int(time.time())
|
||||||
self._save_usage(usage)
|
self._save_usage(usage)
|
||||||
|
|||||||
@@ -92,6 +92,7 @@ class ResearchService:
|
|||||||
relevance=s.get("relevance", 0.0),
|
relevance=s.get("relevance", 0.0),
|
||||||
)
|
)
|
||||||
for s in result.get("sources", [])
|
for s in result.get("sources", [])
|
||||||
|
if isinstance(s, dict)
|
||||||
]
|
]
|
||||||
return ResearchResult(
|
return ResearchResult(
|
||||||
query=topic,
|
query=topic,
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ logger = logging.getLogger(__name__)
|
|||||||
# ----------------------------------------------------------------------
|
# ----------------------------------------------------------------------
|
||||||
def _detect_question_type(query: str) -> Optional[str]:
|
def _detect_question_type(query: str) -> Optional[str]:
|
||||||
"""Return the leading question word if present (who, what, when, where, why, how)."""
|
"""Return the leading question word if present (who, what, when, where, why, how)."""
|
||||||
|
if not isinstance(query, str):
|
||||||
|
return None
|
||||||
q = query.strip().lower()
|
q = query.strip().lower()
|
||||||
for word in ("who", "what", "when", "where", "why", "how"):
|
for word in ("who", "what", "when", "where", "why", "how"):
|
||||||
# Require a whole-word match: a bare prefix mis-flags ordinary queries
|
# Require a whole-word match: a bare prefix mis-flags ordinary queries
|
||||||
@@ -45,12 +47,16 @@ def _extract_entities(query: str) -> Dict[str, List[str]]:
|
|||||||
|
|
||||||
def _split_multi_part(query: str) -> List[str]:
|
def _split_multi_part(query: str) -> List[str]:
|
||||||
"""Split a query into sub-queries on common conjunctions."""
|
"""Split a query into sub-queries on common conjunctions."""
|
||||||
|
if not isinstance(query, str):
|
||||||
|
return []
|
||||||
parts = re.split(r"\s+and\s+|\s+or\s+|;", query, flags=re.I)
|
parts = re.split(r"\s+and\s+|\s+or\s+|;", query, flags=re.I)
|
||||||
return [p.strip() for p in parts if p.strip()]
|
return [p.strip() for p in parts if p.strip()]
|
||||||
|
|
||||||
|
|
||||||
def _extract_site_filter(query: str) -> Tuple[str, Optional[str]]:
|
def _extract_site_filter(query: str) -> Tuple[str, Optional[str]]:
|
||||||
"""Detect a 'site:example.com' token. Returns (query_without_token, site_or_None)."""
|
"""Detect a 'site:example.com' token. Returns (query_without_token, site_or_None)."""
|
||||||
|
if not isinstance(query, str):
|
||||||
|
return "", None
|
||||||
match = re.search(r"\bsite:([^\s]+)", query, flags=re.I)
|
match = re.search(r"\bsite:([^\s]+)", query, flags=re.I)
|
||||||
if match:
|
if match:
|
||||||
site = match.group(1)
|
site = match.group(1)
|
||||||
@@ -71,6 +77,8 @@ def _boost_entities_in_query(base_query: str, entities: Dict[str, List[str]]) ->
|
|||||||
|
|
||||||
def enhance_query(original_query: str) -> Tuple[str, Optional[str]]:
|
def enhance_query(original_query: str) -> Tuple[str, Optional[str]]:
|
||||||
"""Process the original query: site filter, question type boosts, entity extraction."""
|
"""Process the original query: site filter, question type boosts, entity extraction."""
|
||||||
|
if not isinstance(original_query, str):
|
||||||
|
original_query = ""
|
||||||
query_without_site, site = _extract_site_filter(original_query)
|
query_without_site, site = _extract_site_filter(original_query)
|
||||||
sub_queries = _split_multi_part(query_without_site)
|
sub_queries = _split_multi_part(query_without_site)
|
||||||
|
|
||||||
@@ -120,6 +128,8 @@ def build_enhanced_query(query: str, time_filter: str = None) -> str:
|
|||||||
def _is_news_query(query: str) -> bool:
|
def _is_news_query(query: str) -> bool:
|
||||||
"""Lightweight heuristic to decide if a query is news-oriented."""
|
"""Lightweight heuristic to decide if a query is news-oriented."""
|
||||||
news_terms = {"news", "latest", "breaking", "today", "today's", "current", "updates", "happening"}
|
news_terms = {"news", "latest", "breaking", "today", "today's", "current", "updates", "happening"}
|
||||||
|
if not isinstance(query, str):
|
||||||
|
return False
|
||||||
tokens = set(re.findall(r"\b\w+\b", query.lower()))
|
tokens = set(re.findall(r"\b\w+\b", query.lower()))
|
||||||
return bool(tokens & news_terms)
|
return bool(tokens & news_terms)
|
||||||
|
|
||||||
|
|||||||
+46
-9
@@ -276,7 +276,7 @@ Generate an image. Line 1 = description, line 2 = model name, line 3 = WxH (e.g.
|
|||||||
"manage_webhooks": "- ```manage_webhooks``` — Configure outgoing webhooks (HTTP notifications on events like chat completion). Args (JSON): {\"action\": \"list|add|delete|enable|disable\", ...}",
|
"manage_webhooks": "- ```manage_webhooks``` — Configure outgoing webhooks (HTTP notifications on events like chat completion). Args (JSON): {\"action\": \"list|add|delete|enable|disable\", ...}",
|
||||||
"manage_tokens": "- ```manage_tokens``` — Generate or revoke API access tokens for external integrations. Args (JSON): {\"action\": \"list|create|delete\", ...}",
|
"manage_tokens": "- ```manage_tokens``` — Generate or revoke API access tokens for external integrations. Args (JSON): {\"action\": \"list|create|delete\", ...}",
|
||||||
"manage_documents": "- ```manage_documents``` — List, read/open, delete, or tidy documents in the editor panel. Args (JSON): {\"action\": \"list|read|delete|tidy\", ...}. `list` returns rows like `[Title](#document-<id>) — lang, size, updated 5m ago` sorted MOST-RECENT FIRST; the user clicks the anchor to open. `read` (aliases: view/open/get) takes `document_id` and returns the content. When the user asks \"open/show/read my notes\" or \"what documents do I have\", use this — do NOT shell out, do NOT curl.",
|
"manage_documents": "- ```manage_documents``` — List, read/open, delete, or tidy documents in the editor panel. Args (JSON): {\"action\": \"list|read|delete|tidy\", ...}. `list` returns rows like `[Title](#document-<id>) — lang, size, updated 5m ago` sorted MOST-RECENT FIRST; the user clicks the anchor to open. `read` (aliases: view/open/get) takes `document_id` and returns the content. When the user asks \"open/show/read my notes\" or \"what documents do I have\", use this — do NOT shell out, do NOT curl.",
|
||||||
"manage_research": "- ```manage_research``` — List, read/open, or delete saved DEEP RESEARCH results from the Library. Args (JSON): {\"action\": \"list|read|delete\", \"id\": \"<id>\", \"search\": \"...\"}. `list` returns rows like `[query](#research-<id>) — N sources` MOST-RECENT FIRST; the user clicks to open. `read` (aliases: open/view/get) takes `id` and returns the report + sources. Use when the user says \"open/read/find/delete my research\" or \"that report\". To START new research, use trigger_research instead.",
|
"manage_research": "- ```manage_research``` — List, read/open, or delete saved DEEP RESEARCH results from the Library. Args (JSON): {\"action\": \"list|read|delete\", \"id\": \"<id>\", \"search\": \"...\"}. `list` returns rows like `[query](#research-<id>) — N sources` MOST-RECENT FIRST; the user clicks to open. `read` (aliases: open/view/get) takes `id` and returns the report text + sources. Use when the user says \"open/read/find/delete my research\" or \"that report\". This IS how you read a finished report: when the user refers to a just-completed deep-research job (\"check it out\", \"read that report\", \"summarize the research\") WITHOUT giving an id, call `manage_research` with `action:list` to get the most-recent id, then `action:read` with that id, and answer from the returned text. Do NOT `web_fetch`/`app_api` the `/api/research/report/{id}` URL — that endpoint renders HTML for the browser, not clean text — and do NOT start a fresh `web_search`/`trigger_research` just to read an existing report. To START new research, use trigger_research instead.",
|
||||||
"manage_settings": "- ```manage_settings``` — View/change the REAL app settings (same ones the Settings panel writes) AND turn tools on/off. Change a setting: `{\"action\":\"set\",\"key\":\"...\",\"value\":\"...\"}` — keys accept friendly aliases, e.g. voice→tts_voice, \"search engine\"→search_provider, \"default model\"→default_model, \"teacher model\"→teacher_model, \"task/background model\"→task_model, \"image quality\"→image_quality, \"reminder channel\"→reminder_channel (browser|email|ntfy), \"agent timeout\"/\"max tool calls\"/\"token budget\". Read: `{\"action\":\"get\",\"key\":\"...\"}`; see all: `{\"action\":\"list\"}`; reset one: `{\"action\":\"reset\",\"key\":\"...\"}`. Use this when the user asks to change ANY preference instead of making them open Settings. Secrets/API keys are read-only (tell them to set those in the panel). Tool toggles: `{\"action\":\"disable_tool|enable_tool\",\"tool\":\"shell\"}` (aliases: shell/search/browser/documents/memory/skills/images/tasks/notes/calendar/email), list disabled: `{\"action\":\"list_tools\"}`.",
|
"manage_settings": "- ```manage_settings``` — View/change the REAL app settings (same ones the Settings panel writes) AND turn tools on/off. Change a setting: `{\"action\":\"set\",\"key\":\"...\",\"value\":\"...\"}` — keys accept friendly aliases, e.g. voice→tts_voice, \"search engine\"→search_provider, \"default model\"→default_model, \"teacher model\"→teacher_model, \"task/background model\"→task_model, \"image quality\"→image_quality, \"reminder channel\"→reminder_channel (browser|email|ntfy), \"agent timeout\"/\"max tool calls\"/\"token budget\". Read: `{\"action\":\"get\",\"key\":\"...\"}`; see all: `{\"action\":\"list\"}`; reset one: `{\"action\":\"reset\",\"key\":\"...\"}`. Use this when the user asks to change ANY preference instead of making them open Settings. Secrets/API keys are read-only (tell them to set those in the panel). Tool toggles: `{\"action\":\"disable_tool|enable_tool\",\"tool\":\"shell\"}` (aliases: shell/search/browser/documents/memory/skills/images/tasks/notes/calendar/email), list disabled: `{\"action\":\"list_tools\"}`.",
|
||||||
"manage_notes": """\
|
"manage_notes": """\
|
||||||
```manage_notes
|
```manage_notes
|
||||||
@@ -315,9 +315,10 @@ Bulk delete/archive/mark emails. Use this for "delete all those" after listing e
|
|||||||
{"action": "create_event", "summary": "<event title>", "dtstart": "<natural language or ISO datetime>"}
|
{"action": "create_event", "summary": "<event title>", "dtstart": "<natural language or ISO datetime>"}
|
||||||
```
|
```
|
||||||
Calendar event management (CalDAV). Actions: `list_events`, `create_event`, `update_event`, `delete_event`, `list_calendars`. \
|
Calendar event management (CalDAV). Actions: `list_events`, `create_event`, `update_event`, `delete_event`, `list_calendars`. \
|
||||||
For `create_event`: {summary, dtstart, dtend?, duration?, calendar?, location?, description?, reminder_minutes?}. \
|
For `create_event`: {summary, dtstart, dtend?, duration?, calendar?, location?, description?, reminder_minutes?, rrule?}. \
|
||||||
`dtstart` accepts natural language ("tomorrow at 1pm", "in 2 hours", "next monday 9am") or ISO ("2026-05-12T13:00:00"). \
|
`dtstart` accepts natural language ("tomorrow at 1pm", "in 2 hours", "next monday 9am") or ISO ("2026-05-12T13:00:00"). \
|
||||||
If `dtend` omitted, defaults to dtstart+1h (or +1d when `all_day: true`). \
|
If `dtend` omitted, defaults to dtstart+1h (or +1d when `all_day: true`). \
|
||||||
|
For a RECURRING event pass `rrule` as an iCalendar RRULE string, e.g. `"FREQ=WEEKLY;BYDAY=MO"` (every Monday), `"FREQ=DAILY;COUNT=10"`, or `"FREQ=MONTHLY;BYMONTHDAY=1"` — create ONE event with the rrule, do not loop creating many events. \
|
||||||
If the user asks for a reminder/alarm before the event, pass `reminder_minutes` as an integer; do not write reminder text into the event description and do NOT also call `manage_notes` for the same reminder because calendar reminders are routed through Notes automatically. \
|
If the user asks for a reminder/alarm before the event, pass `reminder_minutes` as an integer; do not write reminder text into the event description and do NOT also call `manage_notes` for the same reminder because calendar reminders are routed through Notes automatically. \
|
||||||
`calendar` accepts a name ("Main") or short-id prefix.""",
|
`calendar` accepts a name ("Main") or short-id prefix.""",
|
||||||
"create_session": "- ```create_session``` — Create a new chat. Line 1 = chat name, line 2 = model name. Use for background/parallel work.",
|
"create_session": "- ```create_session``` — Create a new chat. Line 1 = chat name, line 2 = model name. Use for background/parallel work.",
|
||||||
@@ -353,7 +354,7 @@ GENERIC LOOPBACK to ANY Odysseus internal endpoint. Use this whenever the user w
|
|||||||
- Sessions: `/api/sessions`, `/api/session/{id}`, `/api/session/{id}/truncate`
|
- Sessions: `/api/sessions`, `/api/session/{id}`, `/api/session/{id}/truncate`
|
||||||
- Themes: `/api/prefs/themes`, `/api/prefs/custom-themes`
|
- Themes: `/api/prefs/themes`, `/api/prefs/custom-themes`
|
||||||
- Settings: `/api/settings`, `/api/prefs/{key}`
|
- Settings: `/api/settings`, `/api/prefs/{key}`
|
||||||
- Research: `/api/research/start`, `/api/research/tasks`, `/api/research/report/{id}`
|
- Research: `/api/research/start`, `/api/research/tasks` (note: `/api/research/report/{id}` renders HTML — to READ a report's text use the `manage_research` tool with `action:read`, not this endpoint)
|
||||||
- Compare: `/api/compare/sessions`, `/api/compare/start`
|
- Compare: `/api/compare/sessions`, `/api/compare/start`
|
||||||
- Email: use named email tools (`list_email_accounts`, `list_emails`, `read_email`, `send_email`, `reply_to_email`). Do NOT use `/api/email/accounts`; it is owner-filtered in tool context and may falsely return empty.
|
- Email: use named email tools (`list_email_accounts`, `list_emails`, `read_email`, `send_email`, `reply_to_email`). Do NOT use `/api/email/accounts`; it is owner-filtered in tool context and may falsely return empty.
|
||||||
- Endpoints (model providers): `/api/endpoints`, `/api/endpoints/{id}`
|
- Endpoints (model providers): `/api/endpoints`, `/api/endpoints/{id}`
|
||||||
@@ -862,7 +863,7 @@ def _build_system_prompt(
|
|||||||
# matter how often it's been matched and applied.
|
# matter how often it's been matched and applied.
|
||||||
for _sk in relevant_skills:
|
for _sk in relevant_skills:
|
||||||
try:
|
try:
|
||||||
sm.record_use(_sk.get('name', ''))
|
sm.record_use(_sk.get('name', ''), owner=owner)
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
lines.append("## Relevant skills for this request")
|
lines.append("## Relevant skills for this request")
|
||||||
@@ -1314,6 +1315,30 @@ async def _run_verifier_subagent(
|
|||||||
return [r.strip() for r in reasons.split(";") if r.strip()]
|
return [r.strip() for r in reasons.split(";") if r.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_response_fallback(
|
||||||
|
full_response: str,
|
||||||
|
round_reasoning: str,
|
||||||
|
tool_events: list,
|
||||||
|
) -> tuple:
|
||||||
|
"""Return (final_response, sse_chunk_or_none) for the end-of-loop empty-response guard.
|
||||||
|
|
||||||
|
When a thinking model routes all tokens to reasoning_content (leaving
|
||||||
|
content=""), full_response is empty but round_reasoning has content.
|
||||||
|
The reasoning was already streamed as {thinking:true} chunks — do not
|
||||||
|
re-emit it as a normal delta. Just persist it and yield nothing.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(final_response: str, chunk: str | None)
|
||||||
|
chunk is the SSE string to yield, or None if nothing should be emitted.
|
||||||
|
"""
|
||||||
|
if full_response.strip() or tool_events:
|
||||||
|
return full_response, None
|
||||||
|
if round_reasoning.strip():
|
||||||
|
return round_reasoning, None
|
||||||
|
_error_msg = "The model returned an empty response. Please try again or switch to a different model."
|
||||||
|
return _error_msg, f'data: {json.dumps({"delta": _error_msg})}\n\n'
|
||||||
|
|
||||||
|
|
||||||
async def stream_agent_loop(
|
async def stream_agent_loop(
|
||||||
endpoint_url: str,
|
endpoint_url: str,
|
||||||
model: str,
|
model: str,
|
||||||
@@ -1487,13 +1512,23 @@ async def stream_agent_loop(
|
|||||||
_t3 = time.time()
|
_t3 = time.time()
|
||||||
try:
|
try:
|
||||||
from src.context_compactor import trim_for_context
|
from src.context_compactor import trim_for_context
|
||||||
from src.context_budget import compute_input_token_budget
|
from src.context_budget import compute_input_token_budget, DEFAULT_HARD_MAX
|
||||||
from src.settings import is_setting_overridden
|
from src.settings import is_setting_overridden
|
||||||
|
|
||||||
soft_budget = int(get_setting("agent_input_token_budget", 6000) or 0)
|
soft_budget = int(get_setting("agent_input_token_budget", 6000) or 0)
|
||||||
if soft_budget > 0:
|
if soft_budget > 0:
|
||||||
before_trim_tokens = estimate_tokens(messages)
|
before_trim_tokens = estimate_tokens(messages)
|
||||||
reserve_tokens = min(max(max_tokens or 1024, 512), 2048)
|
reserve_tokens = min(max(max_tokens or 1024, 512), 2048)
|
||||||
|
# Honour the configurable ceiling for the auto-derived budget path.
|
||||||
|
# No-op when the user has an explicit `agent_input_token_budget`
|
||||||
|
# (that branch ignores hard_max). Falls back to DEFAULT_HARD_MAX
|
||||||
|
# on missing/malformed values so misconfig can't zero the budget.
|
||||||
|
try:
|
||||||
|
hard_max = int(get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX) or DEFAULT_HARD_MAX)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
hard_max = DEFAULT_HARD_MAX
|
||||||
|
if hard_max <= 0:
|
||||||
|
hard_max = DEFAULT_HARD_MAX
|
||||||
# Scale the default budget to the model's context window so long-context
|
# Scale the default budget to the model's context window so long-context
|
||||||
# models aren't silently capped at 6000; an explicit user setting is
|
# models aren't silently capped at 6000; an explicit user setting is
|
||||||
# still honoured (clamped to the window). (#1170)
|
# still honoured (clamped to the window). (#1170)
|
||||||
@@ -1501,6 +1536,7 @@ async def stream_agent_loop(
|
|||||||
soft_budget,
|
soft_budget,
|
||||||
context_length,
|
context_length,
|
||||||
is_setting_overridden("agent_input_token_budget"),
|
is_setting_overridden("agent_input_token_budget"),
|
||||||
|
hard_max=hard_max,
|
||||||
)
|
)
|
||||||
trimmed_messages = trim_for_context(
|
trimmed_messages = trim_for_context(
|
||||||
messages,
|
messages,
|
||||||
@@ -2214,10 +2250,11 @@ async def stream_agent_loop(
|
|||||||
|
|
||||||
# If the response is completely empty and no tools were executed,
|
# If the response is completely empty and no tools were executed,
|
||||||
# yield a fallback message so the user is not left hanging.
|
# yield a fallback message so the user is not left hanging.
|
||||||
if not full_response.strip() and not tool_events:
|
full_response, _fallback_chunk = _empty_response_fallback(
|
||||||
_error_msg = "The model returned an empty response. Please try again or switch to a different model."
|
full_response, round_reasoning, tool_events
|
||||||
yield f'data: {json.dumps({"delta": _error_msg})}\n\n'
|
)
|
||||||
full_response = _error_msg
|
if _fallback_chunk:
|
||||||
|
yield _fallback_chunk
|
||||||
|
|
||||||
# --- Final metrics ---
|
# --- Final metrics ---
|
||||||
total_duration = time.time() - total_start
|
total_duration = time.time() - total_start
|
||||||
|
|||||||
+12
-2
@@ -48,8 +48,18 @@ class APIKeyManager:
|
|||||||
"""Load and decrypt API keys"""
|
"""Load and decrypt API keys"""
|
||||||
if not os.path.exists(self.api_keys_file):
|
if not os.path.exists(self.api_keys_file):
|
||||||
return {}
|
return {}
|
||||||
with open(self.api_keys_file, 'r', encoding="utf-8") as f:
|
try:
|
||||||
encrypted_keys = json.load(f)
|
with open(self.api_keys_file, 'r', encoding="utf-8") as f:
|
||||||
|
encrypted_keys = json.load(f)
|
||||||
|
except (json.JSONDecodeError, OSError) as e:
|
||||||
|
# A corrupt/truncated api_keys.json must not crash load() (called on
|
||||||
|
# startup via app_initializer) — treat it as no stored keys.
|
||||||
|
logger.warning("Failed to read API keys file: %s", e)
|
||||||
|
return {}
|
||||||
|
if not isinstance(encrypted_keys, dict):
|
||||||
|
# Legacy/wrong shape (e.g. a list) — .items() would raise. Ignore it.
|
||||||
|
logger.warning("API keys file has unexpected shape (%s); ignoring", type(encrypted_keys).__name__)
|
||||||
|
return {}
|
||||||
|
|
||||||
decrypted = {}
|
decrypted = {}
|
||||||
for provider, key in encrypted_keys.items():
|
for provider, key in encrypted_keys.items():
|
||||||
|
|||||||
@@ -22,6 +22,8 @@ def abs_join(base_dir: str, rel: str) -> str:
|
|||||||
|
|
||||||
def inside_base_dir(base_dir: str, path: str) -> bool:
|
def inside_base_dir(base_dir: str, path: str) -> bool:
|
||||||
"""Check if path is inside base directory."""
|
"""Check if path is inside base directory."""
|
||||||
|
if not isinstance(base_dir, str) or not isinstance(path, str):
|
||||||
|
return False
|
||||||
base = os.path.realpath(base_dir)
|
base = os.path.realpath(base_dir)
|
||||||
p = os.path.realpath(path)
|
p = os.path.realpath(path)
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -478,7 +478,7 @@ def _result_has_work(result: str | None) -> bool:
|
|||||||
'No new emails to summarize', 'Tagged 0 / Moved 0', etc. when nothing
|
'No new emails to summarize', 'Tagged 0 / Moved 0', etc. when nothing
|
||||||
was done. Used to decide whether to record the run or noop it.
|
was done. Used to decide whether to record the run or noop it.
|
||||||
"""
|
"""
|
||||||
if not result:
|
if not isinstance(result, str) or not result:
|
||||||
return False
|
return False
|
||||||
low = result.lower()
|
low = result.lower()
|
||||||
if "processed 0" in low or "no new" in low or "nothing to" in low:
|
if "processed 0" in low or "no new" in low or "nothing to" in low:
|
||||||
@@ -554,7 +554,7 @@ _HEURISTIC_CRITICAL = ["surgery", "court", "wedding day", "funeral", "delivery d
|
|||||||
|
|
||||||
def _classify_event_heuristic(summary: str) -> tuple:
|
def _classify_event_heuristic(summary: str) -> tuple:
|
||||||
"""Quick heuristic classification — returns (event_type, importance) or (None, None) if unclear."""
|
"""Quick heuristic classification — returns (event_type, importance) or (None, None) if unclear."""
|
||||||
s = (summary or "").lower()
|
s = (summary if isinstance(summary, str) else "").lower()
|
||||||
etype = None
|
etype = None
|
||||||
for t, kws in _HEURISTIC_TYPES.items():
|
for t, kws in _HEURISTIC_TYPES.items():
|
||||||
if any(k in s for k in kws):
|
if any(k in s for k in kws):
|
||||||
@@ -1313,7 +1313,7 @@ async def action_test_skills(owner: str, **kwargs) -> Tuple[str, bool]:
|
|||||||
if not names:
|
if not names:
|
||||||
raise TaskNoop("no skills to test")
|
raise TaskNoop("no skills to test")
|
||||||
|
|
||||||
url, model, headers = resolve_endpoint("default")
|
url, model, headers = resolve_endpoint("default", owner=owner)
|
||||||
if not url or not model:
|
if not url or not model:
|
||||||
return "No Default/Utility model configured — set one in Settings.", False
|
return "No Default/Utility model configured — set one in Settings.", False
|
||||||
|
|
||||||
@@ -1374,7 +1374,7 @@ async def action_test_skills(owner: str, **kwargs) -> Tuple[str, bool]:
|
|||||||
# user-set value (e.g. 1.0 → 0.95) is destructive.
|
# user-set value (e.g. 1.0 → 0.95) is destructive.
|
||||||
if v in ("pass", "needs_work", "fail", "inconclusive"):
|
if v in ("pass", "needs_work", "fail", "inconclusive"):
|
||||||
try:
|
try:
|
||||||
sm.set_audit(name, v, by_teacher=False, worker_model=model)
|
sm.set_audit(name, v, by_teacher=False, worker_model=model, owner=owner)
|
||||||
except Exception as _e:
|
except Exception as _e:
|
||||||
logger.warning(f"test_skills set_audit({name}) failed: {_e}")
|
logger.warning(f"test_skills set_audit({name}) failed: {_e}")
|
||||||
if v == "unknown":
|
if v == "unknown":
|
||||||
|
|||||||
@@ -24,9 +24,12 @@ Design notes:
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import ipaddress
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import date, datetime, timedelta, timezone
|
from datetime import date, datetime, timedelta, timezone
|
||||||
|
from urllib.parse import urlparse, urlunparse
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -35,6 +38,52 @@ logger = logging.getLogger(__name__)
|
|||||||
# events still come through via RRULE expansion on the frontend.
|
# events still come through via RRULE expansion on the frontend.
|
||||||
_LOOKBACK_DAYS = 90
|
_LOOKBACK_DAYS = 90
|
||||||
_LOOKAHEAD_DAYS = 365
|
_LOOKAHEAD_DAYS = 365
|
||||||
|
_BLOCKED_HOSTS = {
|
||||||
|
"localhost",
|
||||||
|
"localhost.",
|
||||||
|
"ip6-localhost",
|
||||||
|
"metadata.google.internal",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _private_caldav_allowed() -> bool:
|
||||||
|
return os.environ.get("ODYSSEUS_ALLOW_PRIVATE_CALDAV", "0").lower() in {"1", "true", "yes"}
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_caldav_ip(host: str) -> None:
|
||||||
|
try:
|
||||||
|
ip = ipaddress.ip_address(host.strip("[]"))
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
if ip.is_loopback or ip.is_link_local or ip.is_multicast or ip.is_unspecified:
|
||||||
|
raise ValueError("CalDAV URL host is not allowed")
|
||||||
|
if ip.is_private and not _private_caldav_allowed():
|
||||||
|
raise ValueError("Private CalDAV IPs require ODYSSEUS_ALLOW_PRIVATE_CALDAV=1")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_caldav_url(raw_url: str) -> str:
|
||||||
|
"""Validate and normalize a user-provided CalDAV URL before server-side use."""
|
||||||
|
url = (raw_url if isinstance(raw_url, str) else "").strip()
|
||||||
|
if not url:
|
||||||
|
raise ValueError("CalDAV URL is required")
|
||||||
|
parsed = urlparse(url)
|
||||||
|
if parsed.scheme not in {"http", "https"}:
|
||||||
|
raise ValueError("CalDAV URL must start with http:// or https://")
|
||||||
|
if not parsed.hostname:
|
||||||
|
raise ValueError("CalDAV URL must include a host")
|
||||||
|
if parsed.username or parsed.password:
|
||||||
|
raise ValueError("Put CalDAV credentials in the username/password fields, not the URL")
|
||||||
|
if parsed.fragment:
|
||||||
|
raise ValueError("CalDAV URL fragments are not allowed")
|
||||||
|
try:
|
||||||
|
parsed.port
|
||||||
|
except ValueError:
|
||||||
|
raise ValueError("CalDAV URL has an invalid port")
|
||||||
|
host = (parsed.hostname or "").lower()
|
||||||
|
if host in _BLOCKED_HOSTS or host.endswith(".localhost"):
|
||||||
|
raise ValueError("CalDAV URL host is not allowed")
|
||||||
|
_validate_caldav_ip(host)
|
||||||
|
return urlunparse(parsed._replace(fragment="")).rstrip("/")
|
||||||
|
|
||||||
|
|
||||||
def _stable_cal_id(remote_url: str) -> str:
|
def _stable_cal_id(remote_url: str) -> str:
|
||||||
@@ -250,13 +299,21 @@ async def sync_caldav(owner: str) -> dict:
|
|||||||
url = (cfg.get("url") or "").strip()
|
url = (cfg.get("url") or "").strip()
|
||||||
user = (cfg.get("username") or "").strip()
|
user = (cfg.get("username") or "").strip()
|
||||||
pw = cfg.get("password") or ""
|
pw = cfg.get("password") or ""
|
||||||
|
try:
|
||||||
|
from src.secret_storage import decrypt
|
||||||
|
pw = decrypt(pw)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
if not (url and user and pw):
|
if not (url and user and pw):
|
||||||
return {
|
return {
|
||||||
"calendars": 0, "events": 0, "deleted": 0,
|
"calendars": 0, "events": 0, "deleted": 0,
|
||||||
"errors": ["CalDAV is not configured"],
|
"errors": ["CalDAV is not configured"],
|
||||||
}
|
}
|
||||||
try:
|
try:
|
||||||
|
url = validate_caldav_url(url)
|
||||||
return await asyncio.to_thread(_sync_blocking, owner, url, user, pw)
|
return await asyncio.to_thread(_sync_blocking, owner, url, user, pw)
|
||||||
|
except ValueError as e:
|
||||||
|
return {"calendars": 0, "events": 0, "deleted": 0, "errors": [str(e)]}
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.exception("CalDAV sync raised")
|
logger.exception("CalDAV sync raised")
|
||||||
return {"calendars": 0, "events": 0, "deleted": 0, "errors": [str(e)[:200]]}
|
return {"calendars": 0, "events": 0, "deleted": 0, "errors": [str(e)[:200]]}
|
||||||
|
|||||||
@@ -0,0 +1,172 @@
|
|||||||
|
"""CalDAV write-back: push local create/update/delete out to the remote (#800).
|
||||||
|
|
||||||
|
``src/caldav_sync.py`` is a one-way pull (remote → local). So events created,
|
||||||
|
edited, or deleted in Odysseus on a CalDAV-backed calendar only changed the local
|
||||||
|
SQLite copy and never reached the server (iCloud/Nextcloud/Radicale/Fastmail) —
|
||||||
|
they'd silently disappear on the next pull and never show on the user's phone.
|
||||||
|
|
||||||
|
This adds the missing write half. The remote calendar URL isn't stored locally
|
||||||
|
(the local calendar id is a one-way hash of it), so we re-discover the remote
|
||||||
|
calendar by matching that same hash, then PUT/DELETE the VEVENT by its UID via
|
||||||
|
the `caldav` lib. Writes are best-effort: the local DB stays the source of truth,
|
||||||
|
and a remote failure is reported, never fatal to the local operation.
|
||||||
|
|
||||||
|
The pure pieces (``build_event_ical``, ``find_remote_calendar``, ``push_event``)
|
||||||
|
take their inputs by argument so they unit-test against a fake client with no
|
||||||
|
network.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from datetime import timezone
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _stable_cal_id(remote_url: str) -> str:
|
||||||
|
# Reuse the sync module's hashing so a local CalDAV calendar id maps back to
|
||||||
|
# the same remote URL it was pulled from.
|
||||||
|
from src.caldav_sync import _stable_cal_id as _sync_id
|
||||||
|
return _sync_id(remote_url)
|
||||||
|
|
||||||
|
|
||||||
|
def build_event_ical(ev: dict) -> str:
|
||||||
|
"""Serialize a local event dict to a VCALENDAR/VEVENT iCalendar string.
|
||||||
|
|
||||||
|
``ev`` keys: uid, summary, description, location, dtstart (datetime),
|
||||||
|
dtend (datetime), all_day (bool), is_utc (bool), rrule (str).
|
||||||
|
Mirrors how the pull path interprets is_utc/all_day so a round-trip is stable.
|
||||||
|
"""
|
||||||
|
from icalendar import Calendar, Event as iEvent
|
||||||
|
from icalendar.prop import vRecur
|
||||||
|
|
||||||
|
cal = Calendar()
|
||||||
|
cal.add("prodid", "-//Odysseus//CalDAV write-back//EN")
|
||||||
|
cal.add("version", "2.0")
|
||||||
|
|
||||||
|
ve = iEvent()
|
||||||
|
ve.add("uid", ev["uid"])
|
||||||
|
ve.add("summary", ev.get("summary") or "")
|
||||||
|
if ev.get("description"):
|
||||||
|
ve.add("description", ev["description"])
|
||||||
|
if ev.get("location"):
|
||||||
|
ve.add("location", ev["location"])
|
||||||
|
|
||||||
|
dtstart = ev["dtstart"]
|
||||||
|
dtend = ev["dtend"]
|
||||||
|
if ev.get("all_day"):
|
||||||
|
ve.add("dtstart", dtstart.date())
|
||||||
|
ve.add("dtend", dtend.date())
|
||||||
|
elif ev.get("is_utc"):
|
||||||
|
# Stored as naive-UTC instants — re-attach UTC so the server gets a Z time.
|
||||||
|
ve.add("dtstart", dtstart.replace(tzinfo=timezone.utc))
|
||||||
|
ve.add("dtend", dtend.replace(tzinfo=timezone.utc))
|
||||||
|
else:
|
||||||
|
# Legacy naive-local ("floating") time — emit without a TZ.
|
||||||
|
ve.add("dtstart", dtstart)
|
||||||
|
ve.add("dtend", dtend)
|
||||||
|
|
||||||
|
if ev.get("rrule"):
|
||||||
|
try:
|
||||||
|
ve.add("rrule", vRecur.from_ical(ev["rrule"]))
|
||||||
|
except Exception:
|
||||||
|
logger.debug("CalDAV write-back: skipping unparseable rrule %r", ev.get("rrule"))
|
||||||
|
|
||||||
|
cal.add_component(ve)
|
||||||
|
return cal.to_ical().decode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def find_remote_calendar(calendars, local_cal_id: str):
|
||||||
|
"""Find the remote calendar whose URL hashes to ``local_cal_id``, or None."""
|
||||||
|
for cal in calendars:
|
||||||
|
try:
|
||||||
|
if _stable_cal_id(str(cal.url)) == local_cal_id:
|
||||||
|
return cal
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def push_event(calendars, local_cal_id: str, ev: dict, *, delete: bool = False) -> dict:
|
||||||
|
"""Create/update (or delete) ``ev`` on the matching remote calendar.
|
||||||
|
|
||||||
|
Returns ``{"ok": bool, ...}``. ``calendars`` is the discovered caldav
|
||||||
|
calendar list (injected so this is unit-testable with fakes).
|
||||||
|
"""
|
||||||
|
uid = (ev or {}).get("uid") if isinstance(ev, dict) else None
|
||||||
|
if not uid:
|
||||||
|
return {"ok": False, "error": "event uid is required"}
|
||||||
|
|
||||||
|
remote = find_remote_calendar(calendars, local_cal_id)
|
||||||
|
if remote is None:
|
||||||
|
return {"ok": False, "error": "remote calendar not found"}
|
||||||
|
|
||||||
|
try:
|
||||||
|
existing = remote.event_by_uid(uid)
|
||||||
|
except Exception:
|
||||||
|
existing = None
|
||||||
|
|
||||||
|
if delete:
|
||||||
|
if existing is None:
|
||||||
|
return {"ok": True, "note": "already absent on remote"}
|
||||||
|
existing.delete()
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
ical = build_event_ical(ev)
|
||||||
|
if existing is not None:
|
||||||
|
existing.data = ical
|
||||||
|
existing.save()
|
||||||
|
return {"ok": True, "updated": True}
|
||||||
|
remote.save_event(ical)
|
||||||
|
return {"ok": True, "created": True}
|
||||||
|
|
||||||
|
|
||||||
|
def _discover_calendars(client):
|
||||||
|
"""Discover the principal's calendars, falling back to the URL itself —
|
||||||
|
same strategy as the pull path."""
|
||||||
|
from caldav.lib.error import AuthorizationError, NotFoundError
|
||||||
|
try:
|
||||||
|
return client.principal().calendars()
|
||||||
|
except (AuthorizationError, NotFoundError):
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
try:
|
||||||
|
return [client.calendar(url=str(client.url))]
|
||||||
|
except Exception:
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _writeback_blocking(local_cal_id, ev, delete, url, username, password) -> dict:
|
||||||
|
import caldav
|
||||||
|
client = caldav.DAVClient(url=url, username=username, password=password)
|
||||||
|
calendars = _discover_calendars(client)
|
||||||
|
if not calendars:
|
||||||
|
return {"ok": False, "error": "no remote calendars discovered"}
|
||||||
|
return push_event(calendars, local_cal_id, ev, delete=delete)
|
||||||
|
|
||||||
|
|
||||||
|
async def writeback_event(owner: str, calendar_source: str, calendar_id: str,
|
||||||
|
ev: dict, *, delete: bool = False) -> dict:
|
||||||
|
"""Best-effort push of a local change to the remote CalDAV server.
|
||||||
|
|
||||||
|
No-ops (``{"skipped": ...}``) when the calendar isn't CalDAV-backed or no
|
||||||
|
credentials are configured. Never raises — a remote failure is logged and
|
||||||
|
returned, the local DB remaining the source of truth.
|
||||||
|
"""
|
||||||
|
if calendar_source != "caldav":
|
||||||
|
return {"skipped": "not a caldav calendar"}
|
||||||
|
try:
|
||||||
|
from routes.prefs_routes import _load_for_user
|
||||||
|
cfg = (_load_for_user(owner) or {}).get("caldav", {}) or {}
|
||||||
|
url = (cfg.get("url") or "").strip()
|
||||||
|
user = (cfg.get("username") or "").strip()
|
||||||
|
pw = cfg.get("password") or ""
|
||||||
|
if not (url and user and pw):
|
||||||
|
return {"skipped": "caldav not configured"}
|
||||||
|
result = await asyncio.to_thread(_writeback_blocking, calendar_id, ev, delete, url, user, pw)
|
||||||
|
if not result.get("ok"):
|
||||||
|
logger.warning("CalDAV write-back did not apply: %s", result.get("error") or result)
|
||||||
|
return result
|
||||||
|
except Exception as e:
|
||||||
|
logger.exception("CalDAV write-back raised")
|
||||||
|
return {"ok": False, "error": str(e)[:200]}
|
||||||
+9
-1
@@ -36,8 +36,16 @@ _VISION_MODEL_KEYWORDS = (
|
|||||||
"gpt-4o", "gpt-4.1", "gpt-4.5", "gpt-4-turbo", "gpt-4-vision",
|
"gpt-4o", "gpt-4.1", "gpt-4.5", "gpt-4-turbo", "gpt-4-vision",
|
||||||
"claude-sonnet", "claude-opus", "claude-haiku", "gemini",
|
"claude-sonnet", "claude-opus", "claude-haiku", "gemini",
|
||||||
# open / local
|
# open / local
|
||||||
"vision", "llava", "bakllava", "moondream", "pixtral", "minicpm",
|
"vision", "multimodal", "llava", "bakllava", "moondream", "pixtral", "minicpm",
|
||||||
"internvl", "cogvlm", "qwen-vl", "qwen2-vl", "qwen3-vl", "qwen3vl",
|
"internvl", "cogvlm", "qwen-vl", "qwen2-vl", "qwen3-vl", "qwen3vl",
|
||||||
|
# multimodal families whose names don't contain "vision"/"vl" but DO accept
|
||||||
|
# images — without these the image is silently dropped for common Ollama tags
|
||||||
|
# like gemma3:4b (issue #1274). Gemma 3 (4b+), Llama 4 (all), and Mistral
|
||||||
|
# Small 3.1/3.2 are vision-capable; per the err-toward-True policy (#124) a
|
||||||
|
# rare text-only tag (e.g. gemma3:1b) being treated as vision is the safer
|
||||||
|
# failure than dropping a real image.
|
||||||
|
"gemma-3", "gemma3", "llama-4", "llama4",
|
||||||
|
"mistral-small-3.1", "mistral-small3.1", "mistral-small-3.2", "mistral-small3.2",
|
||||||
# zhipu / glm (glm-4.5v, glm-4.6v, glm-5v-turbo, etc.)
|
# zhipu / glm (glm-4.5v, glm-4.6v, glm-5v-turbo, etc.)
|
||||||
"glm-4.5v", "glm-4.6v", "glm-5v",
|
"glm-4.5v", "glm-4.6v", "glm-5v",
|
||||||
)
|
)
|
||||||
|
|||||||
+47
-3
@@ -11,6 +11,7 @@ import json
|
|||||||
import logging
|
import logging
|
||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
|
from datetime import datetime
|
||||||
from typing import Callable, Dict, List, Optional, Set
|
from typing import Callable, Dict, List, Optional, Set
|
||||||
|
|
||||||
from src.research_utils import strip_thinking, is_low_quality
|
from src.research_utils import strip_thinking, is_low_quality
|
||||||
@@ -19,6 +20,20 @@ from src.goal_based_extractor import EXTRACTOR_PROMPT
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def current_date_context() -> str:
|
||||||
|
"""Preamble that grounds query-generation/planning LLMs in the real current
|
||||||
|
date. Without it the model falls back to its training-cutoff year and emits
|
||||||
|
queries like "best Python tutorials 2025" when the year is actually 2026.
|
||||||
|
System TZ-local so it matches what the user sees. Portable strftime only."""
|
||||||
|
now = datetime.now().astimezone()
|
||||||
|
return (
|
||||||
|
f"Today's date is {now.strftime('%B %d, %Y')} ({now.strftime('%Y-%m-%d')}). "
|
||||||
|
f"When a search query needs a year or refers to 'latest'/'current'/"
|
||||||
|
f"'this year', use {now.strftime('%Y')} or relative wording — never a "
|
||||||
|
f"year inferred from training data.\n\n"
|
||||||
|
)
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Prompts
|
# Prompts
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -329,6 +344,16 @@ class DeepResearcher:
|
|||||||
self._emit(phase="writing", total_sources=len(self.urls_fetched),
|
self._emit(phase="writing", total_sources=len(self.urls_fetched),
|
||||||
total_findings=len(findings))
|
total_findings=len(findings))
|
||||||
if not report:
|
if not report:
|
||||||
|
# Synthesis can fail (e.g. the LLM timed out) even though the search
|
||||||
|
# rounds did gather findings. Don't throw that work away — return the
|
||||||
|
# gathered findings as a basic compiled report instead of claiming
|
||||||
|
# nothing was found (#1551).
|
||||||
|
if findings:
|
||||||
|
logger.warning(
|
||||||
|
"Synthesis produced no report; returning %d gathered "
|
||||||
|
"finding(s) as a fallback", len(findings)
|
||||||
|
)
|
||||||
|
return self._fallback_report(question, findings)
|
||||||
return "No information could be gathered for this question."
|
return "No information could be gathered for this question."
|
||||||
|
|
||||||
self.evolving_report = report # preserve pre-synthesis report
|
self.evolving_report = report # preserve pre-synthesis report
|
||||||
@@ -364,7 +389,7 @@ class DeepResearcher:
|
|||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
async def _create_plan(self, question: str) -> str:
|
async def _create_plan(self, question: str) -> str:
|
||||||
"""LLM analyzes the question and creates a research plan."""
|
"""LLM analyzes the question and creates a research plan."""
|
||||||
prompt = RESEARCH_PLAN_PROMPT.format(question=question)
|
prompt = current_date_context() + RESEARCH_PLAN_PROMPT.format(question=question)
|
||||||
try:
|
try:
|
||||||
response = await self._llm(
|
response = await self._llm(
|
||||||
[{"role": "user", "content": prompt}],
|
[{"role": "user", "content": prompt}],
|
||||||
@@ -439,7 +464,7 @@ class DeepResearcher:
|
|||||||
"that the report doesn't yet cover well."
|
"that the report doesn't yet cover well."
|
||||||
)
|
)
|
||||||
|
|
||||||
prompt = QUERY_GEN_PROMPT.format(
|
prompt = current_date_context() + QUERY_GEN_PROMPT.format(
|
||||||
question=question,
|
question=question,
|
||||||
research_plan=self.research_plan or "(No plan — search broadly.)",
|
research_plan=self.research_plan or "(No plan — search broadly.)",
|
||||||
report=report or "(No findings yet.)",
|
report=report or "(No findings yet.)",
|
||||||
@@ -647,7 +672,11 @@ class DeepResearcher:
|
|||||||
[{"role": "user", "content": prompt}],
|
[{"role": "user", "content": prompt}],
|
||||||
temperature=0.3,
|
temperature=0.3,
|
||||||
max_tokens=self.max_report_tokens,
|
max_tokens=self.max_report_tokens,
|
||||||
timeout=60,
|
# Synthesis is a heavy generation call like the final report
|
||||||
|
# (which gets 180s); a slow local model (e.g. a 20B served from
|
||||||
|
# LM Studio) routinely needs >60s for it. The old 60s cap timed
|
||||||
|
# out mid-stream and discarded the round's findings (#1551).
|
||||||
|
timeout=180,
|
||||||
)
|
)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Synthesis failed: {e}")
|
logger.error(f"Synthesis failed: {e}")
|
||||||
@@ -826,6 +855,21 @@ class DeepResearcher:
|
|||||||
parts.append(f"**Finding {i}** — [{title}]({url})\n{content}")
|
parts.append(f"**Finding {i}** — [{title}]({url})\n{content}")
|
||||||
return "\n\n".join(parts)
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
def _fallback_report(self, question: str, findings: List[Dict]) -> str:
|
||||||
|
"""Compile gathered findings into a basic report.
|
||||||
|
|
||||||
|
Used when the LLM synthesis step produced no report (e.g. it timed out)
|
||||||
|
but the search rounds did collect findings — so the user still gets the
|
||||||
|
material that was gathered instead of "No information could be gathered"
|
||||||
|
(#1551).
|
||||||
|
"""
|
||||||
|
return (
|
||||||
|
f"# {question}\n\n"
|
||||||
|
"_Automatic synthesis did not complete, so this report lists the "
|
||||||
|
f"{len(findings)} finding(s) gathered during research._\n\n"
|
||||||
|
f"{self._format_findings(findings)}"
|
||||||
|
)
|
||||||
|
|
||||||
def get_stats(self) -> Dict:
|
def get_stats(self) -> Dict:
|
||||||
"""Return research statistics."""
|
"""Return research statistics."""
|
||||||
elapsed = time.time() - self._start_time if self._start_time else 0
|
elapsed = time.time() - self._start_time if self._start_time else 0
|
||||||
|
|||||||
@@ -21,7 +21,8 @@ _JUNK_TITLES = {
|
|||||||
|
|
||||||
def _norm_title(t: str) -> str:
|
def _norm_title(t: str) -> str:
|
||||||
"""Normalize a title for grouping: trim, collapse whitespace, lowercase."""
|
"""Normalize a title for grouping: trim, collapse whitespace, lowercase."""
|
||||||
return re.sub(r"\s+", " ", (t or "").strip()).lower()
|
t = t if isinstance(t, str) else ""
|
||||||
|
return re.sub(r"\s+", " ", t.strip()).lower()
|
||||||
|
|
||||||
|
|
||||||
def _content_fingerprint(content: str) -> str:
|
def _content_fingerprint(content: str) -> str:
|
||||||
@@ -32,7 +33,7 @@ def _content_fingerprint(content: str) -> str:
|
|||||||
that N imports of the same file collapse to one fingerprint. Whitespace is
|
that N imports of the same file collapse to one fingerprint. Whitespace is
|
||||||
collapsed and the result lowercased.
|
collapsed and the result lowercased.
|
||||||
"""
|
"""
|
||||||
c = content or ""
|
c = content if isinstance(content, str) else ""
|
||||||
c = re.sub(r'upload_id="[^"]*"', "upload_id", c) # pdf_source re-imports
|
c = re.sub(r'upload_id="[^"]*"', "upload_id", c) # pdf_source re-imports
|
||||||
c = re.sub(r"\bid=ann-[A-Za-z0-9_-]+", "id=ann", c) # annotation ids
|
c = re.sub(r"\bid=ann-[A-Za-z0-9_-]+", "id=ann", c) # annotation ids
|
||||||
c = re.sub(r"\s+", " ", c).strip().lower()
|
c = re.sub(r"\s+", " ", c).strip().lower()
|
||||||
@@ -41,7 +42,8 @@ def _content_fingerprint(content: str) -> str:
|
|||||||
|
|
||||||
def _real_len(content: str) -> int:
|
def _real_len(content: str) -> int:
|
||||||
"""Length of content with markdown noise stripped — a 'completeness' proxy."""
|
"""Length of content with markdown noise stripped — a 'completeness' proxy."""
|
||||||
stripped = re.sub(r"^#{1,6}\s+", "", content or "", flags=re.MULTILINE)
|
content = content if isinstance(content, str) else ""
|
||||||
|
stripped = re.sub(r"^#{1,6}\s+", "", content, flags=re.MULTILINE)
|
||||||
stripped = re.sub(r"[*_`>\-=]+", "", stripped)
|
stripped = re.sub(r"[*_`>\-=]+", "", stripped)
|
||||||
stripped = re.sub(r"\s+", " ", stripped).strip()
|
stripped = re.sub(r"\s+", " ", stripped).strip()
|
||||||
return len(stripped)
|
return len(stripped)
|
||||||
|
|||||||
@@ -295,7 +295,7 @@ def resolve_endpoint(
|
|||||||
|
|
||||||
|
|
||||||
def resolve_endpoint_by_id(
|
def resolve_endpoint_by_id(
|
||||||
ep_id: str, model: Optional[str] = None
|
ep_id: str, model: Optional[str] = None, owner: Optional[str] = None
|
||||||
) -> Optional[Tuple[str, str, Dict]]:
|
) -> Optional[Tuple[str, str, Dict]]:
|
||||||
"""Resolve a specific endpoint id (+ optional model) to (chat_url, model, headers).
|
"""Resolve a specific endpoint id (+ optional model) to (chat_url, model, headers).
|
||||||
|
|
||||||
@@ -306,10 +306,14 @@ def resolve_endpoint_by_id(
|
|||||||
return None
|
return None
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
ep = db.query(ModelEndpoint).filter(
|
q = db.query(ModelEndpoint).filter(
|
||||||
ModelEndpoint.id == ep_id,
|
ModelEndpoint.id == ep_id,
|
||||||
ModelEndpoint.is_enabled == True,
|
ModelEndpoint.is_enabled == True,
|
||||||
).first()
|
)
|
||||||
|
if owner:
|
||||||
|
from src.auth_helpers import owner_filter
|
||||||
|
q = owner_filter(q, ModelEndpoint, owner)
|
||||||
|
ep = q.first()
|
||||||
if not ep:
|
if not ep:
|
||||||
return None
|
return None
|
||||||
base = normalize_base(ep.base_url)
|
base = normalize_base(ep.base_url)
|
||||||
@@ -332,14 +336,14 @@ def resolve_endpoint_by_id(
|
|||||||
db.close()
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
def resolve_chat_fallback_candidates() -> list:
|
def resolve_chat_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||||
"""Build the configured default-chat fallback chain as a list of
|
"""Build the configured default-chat fallback chain as a list of
|
||||||
(chat_url, model, headers) tuples, skipping any that can't resolve.
|
(chat_url, model, headers) tuples, skipping any that can't resolve.
|
||||||
|
|
||||||
The primary model is NOT included — callers prepend their session's
|
The primary model is NOT included — callers prepend their session's
|
||||||
current (url, model, headers) so per-session model overrides are honored.
|
current (url, model, headers) so per-session model overrides are honored.
|
||||||
"""
|
"""
|
||||||
return _resolve_fallback_candidates("default_model_fallbacks")
|
return _resolve_fallback_candidates("default_model_fallbacks", owner=owner)
|
||||||
|
|
||||||
|
|
||||||
def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
|
def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||||
@@ -355,9 +359,9 @@ def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
|
|||||||
return _resolve_fallback_candidates("utility_model_fallbacks", owner=owner)
|
return _resolve_fallback_candidates("utility_model_fallbacks", owner=owner)
|
||||||
|
|
||||||
|
|
||||||
def resolve_vision_fallback_candidates() -> list:
|
def resolve_vision_fallback_candidates(owner: Optional[str] = None) -> list:
|
||||||
"""Configured fallback chain for the Vision model (`vision_model_fallbacks`)."""
|
"""Configured fallback chain for the Vision model (`vision_model_fallbacks`)."""
|
||||||
return _resolve_fallback_candidates("vision_model_fallbacks")
|
return _resolve_fallback_candidates("vision_model_fallbacks", owner=owner)
|
||||||
|
|
||||||
|
|
||||||
def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None) -> list:
|
def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None) -> list:
|
||||||
@@ -371,7 +375,7 @@ def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None)
|
|||||||
for entry in chain:
|
for entry in chain:
|
||||||
if not isinstance(entry, dict):
|
if not isinstance(entry, dict):
|
||||||
continue
|
continue
|
||||||
resolved = resolve_endpoint_by_id(entry.get("endpoint_id", ""), entry.get("model", ""))
|
resolved = resolve_endpoint_by_id(entry.get("endpoint_id", ""), entry.get("model", ""), owner=owner)
|
||||||
if resolved:
|
if resolved:
|
||||||
out.append(resolved)
|
out.append(resolved)
|
||||||
return out
|
return out
|
||||||
|
|||||||
+4
-2
@@ -860,7 +860,8 @@ def llm_call(url: str, model: str, messages: List[Dict], temperature: float = LL
|
|||||||
elif provider == "ollama":
|
elif provider == "ollama":
|
||||||
response = _parse_ollama_response(data)
|
response = _parse_ollama_response(data)
|
||||||
else:
|
else:
|
||||||
response = data["choices"][0]["message"]["content"]
|
msg = data["choices"][0]["message"]
|
||||||
|
response = msg.get("content") or msg.get("reasoning_content") or ""
|
||||||
_set_cached_response(cache_key, response)
|
_set_cached_response(cache_key, response)
|
||||||
return response
|
return response
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -997,7 +998,8 @@ async def llm_call_async(
|
|||||||
elif provider == "ollama":
|
elif provider == "ollama":
|
||||||
response = _parse_ollama_response(data)
|
response = _parse_ollama_response(data)
|
||||||
else:
|
else:
|
||||||
response = data["choices"][0]["message"]["content"]
|
msg = data["choices"][0]["message"]
|
||||||
|
response = msg.get("content") or msg.get("reasoning_content") or ""
|
||||||
_set_cached_response(cache_key, response)
|
_set_cached_response(cache_key, response)
|
||||||
return response
|
return response
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|||||||
@@ -26,6 +26,8 @@ MARKITDOWN_EXTS = frozenset({".docx", ".pptx", ".xlsx", ".xls", ".epub"})
|
|||||||
|
|
||||||
def is_markitdown_format(path: str) -> bool:
|
def is_markitdown_format(path: str) -> bool:
|
||||||
"""True if the file extension is one we route through markitdown."""
|
"""True if the file extension is one we route through markitdown."""
|
||||||
|
if not isinstance(path, str):
|
||||||
|
return False
|
||||||
return os.path.splitext(path)[1].lower() in MARKITDOWN_EXTS
|
return os.path.splitext(path)[1].lower() in MARKITDOWN_EXTS
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+20
-4
@@ -77,6 +77,11 @@ def split_chunks(text: str, size: int = config.CHUNK_SIZE, overlap: int = config
|
|||||||
while i < n:
|
while i < n:
|
||||||
j = min(i + size, n)
|
j = min(i + size, n)
|
||||||
chunks.append(text[i:j])
|
chunks.append(text[i:j])
|
||||||
|
if j >= n:
|
||||||
|
# Reached the end. Without this, the next start (j - overlap) is
|
||||||
|
# still > i, so the loop appended one extra chunk duplicating the
|
||||||
|
# last `overlap` chars of the text.
|
||||||
|
break
|
||||||
i = j - overlap if j - overlap > i else j
|
i = j - overlap if j - overlap > i else j
|
||||||
return chunks
|
return chunks
|
||||||
|
|
||||||
@@ -179,6 +184,11 @@ def retrieve_personal(personal_index: List[Dict], query: str, k: int = 5,
|
|||||||
# Fall back to keyword search
|
# Fall back to keyword search
|
||||||
return retrieve_personal_keyword(personal_index, query, k)
|
return retrieve_personal_keyword(personal_index, query, k)
|
||||||
|
|
||||||
|
|
||||||
|
def _string_list(values) -> list[str]:
|
||||||
|
return [value for value in values or [] if isinstance(value, str)]
|
||||||
|
|
||||||
|
|
||||||
class PersonalDocsManager:
|
class PersonalDocsManager:
|
||||||
"""Manager class for personal document indexing and retrieval."""
|
"""Manager class for personal document indexing and retrieval."""
|
||||||
|
|
||||||
@@ -199,7 +209,10 @@ class PersonalDocsManager:
|
|||||||
try:
|
try:
|
||||||
if os.path.exists(self.directories_file):
|
if os.path.exists(self.directories_file):
|
||||||
with open(self.directories_file, 'r', encoding="utf-8") as f:
|
with open(self.directories_file, 'r', encoding="utf-8") as f:
|
||||||
self.indexed_directories = json.load(f)
|
directories = json.load(f)
|
||||||
|
if not isinstance(directories, list):
|
||||||
|
raise ValueError("indexed directories must be a list")
|
||||||
|
self.indexed_directories = _string_list(directories)
|
||||||
logger.info(f"Loaded {len(self.indexed_directories)} indexed directories")
|
logger.info(f"Loaded {len(self.indexed_directories)} indexed directories")
|
||||||
else:
|
else:
|
||||||
self.indexed_directories = []
|
self.indexed_directories = []
|
||||||
@@ -211,7 +224,7 @@ class PersonalDocsManager:
|
|||||||
"""Save the list of indexed directories to persistent storage."""
|
"""Save the list of indexed directories to persistent storage."""
|
||||||
try:
|
try:
|
||||||
with open(self.directories_file, 'w', encoding="utf-8") as f:
|
with open(self.directories_file, 'w', encoding="utf-8") as f:
|
||||||
json.dump(self.indexed_directories, f, indent=2)
|
json.dump(_string_list(self.indexed_directories), f, indent=2)
|
||||||
logger.info(f"Saved {len(self.indexed_directories)} indexed directories")
|
logger.info(f"Saved {len(self.indexed_directories)} indexed directories")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error saving directories: {e}")
|
logger.error(f"Error saving directories: {e}")
|
||||||
@@ -221,7 +234,10 @@ class PersonalDocsManager:
|
|||||||
try:
|
try:
|
||||||
if os.path.exists(self._excluded_file):
|
if os.path.exists(self._excluded_file):
|
||||||
with open(self._excluded_file, 'r', encoding="utf-8") as f:
|
with open(self._excluded_file, 'r', encoding="utf-8") as f:
|
||||||
self.excluded_files = set(json.load(f))
|
excluded = json.load(f)
|
||||||
|
if not isinstance(excluded, list):
|
||||||
|
raise ValueError("excluded files must be a list")
|
||||||
|
self.excluded_files = set(_string_list(excluded))
|
||||||
else:
|
else:
|
||||||
self.excluded_files = set()
|
self.excluded_files = set()
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -231,7 +247,7 @@ class PersonalDocsManager:
|
|||||||
def _save_excluded(self):
|
def _save_excluded(self):
|
||||||
try:
|
try:
|
||||||
with open(self._excluded_file, 'w', encoding="utf-8") as f:
|
with open(self._excluded_file, 'w', encoding="utf-8") as f:
|
||||||
json.dump(list(self.excluded_files), f)
|
json.dump(_string_list(self.excluded_files), f)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error saving excluded files: {e}")
|
logger.error(f"Error saving excluded files: {e}")
|
||||||
|
|
||||||
|
|||||||
@@ -161,10 +161,10 @@ class ResearchHandler:
|
|||||||
) -> Optional[dict]:
|
) -> Optional[dict]:
|
||||||
"""Generate a research plan for user review before starting research."""
|
"""Generate a research plan for user review before starting research."""
|
||||||
try:
|
try:
|
||||||
from src.deep_research import RESEARCH_PLAN_PROMPT
|
from src.deep_research import RESEARCH_PLAN_PROMPT, current_date_context
|
||||||
from src.llm_core import llm_call_async
|
from src.llm_core import llm_call_async
|
||||||
|
|
||||||
prompt = RESEARCH_PLAN_PROMPT.format(question=query)
|
prompt = current_date_context() + RESEARCH_PLAN_PROMPT.format(question=query)
|
||||||
response = await llm_call_async(
|
response = await llm_call_async(
|
||||||
url=llm_endpoint,
|
url=llm_endpoint,
|
||||||
model=llm_model,
|
model=llm_model,
|
||||||
|
|||||||
+16
-3
@@ -96,6 +96,14 @@ DEFAULT_SETTINGS = {
|
|||||||
"research_run_timeout_seconds": 1800,
|
"research_run_timeout_seconds": 1800,
|
||||||
"agent_max_tool_calls": 0,
|
"agent_max_tool_calls": 0,
|
||||||
"agent_input_token_budget": 6000,
|
"agent_input_token_budget": 6000,
|
||||||
|
# Ceiling on the *auto-derived* input budget that #1230 introduced. Has
|
||||||
|
# no effect when `agent_input_token_budget` is explicitly set (the user's
|
||||||
|
# value is honoured regardless). Default matches
|
||||||
|
# `src.context_budget.DEFAULT_HARD_MAX`; lower this for cost-paranoid
|
||||||
|
# setups, raise it on premium APIs with very large windows that you
|
||||||
|
# want to actually use (e.g. 900_000 to fill a 1M-context model). See
|
||||||
|
# `compute_input_token_budget` in src/context_budget.py.
|
||||||
|
"agent_input_token_hard_max": 200_000,
|
||||||
"agent_stream_timeout_seconds": 300,
|
"agent_stream_timeout_seconds": 300,
|
||||||
# Extra directory roots that read_file / write_file may access, in
|
# Extra directory roots that read_file / write_file may access, in
|
||||||
# addition to the built-in project data/ and system temp dirs. Each
|
# addition to the built-in project data/ and system temp dirs. Each
|
||||||
@@ -176,8 +184,10 @@ def load_settings() -> dict:
|
|||||||
try:
|
try:
|
||||||
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
||||||
saved = json.load(f)
|
saved = json.load(f)
|
||||||
|
if not isinstance(saved, dict):
|
||||||
|
raise ValueError("settings must be an object")
|
||||||
merged = {**DEFAULT_SETTINGS, **saved}
|
merged = {**DEFAULT_SETTINGS, **saved}
|
||||||
except (FileNotFoundError, json.JSONDecodeError):
|
except (FileNotFoundError, json.JSONDecodeError, ValueError):
|
||||||
merged = dict(DEFAULT_SETTINGS)
|
merged = dict(DEFAULT_SETTINGS)
|
||||||
_settings_cache = (now, merged)
|
_settings_cache = (now, merged)
|
||||||
return merged
|
return merged
|
||||||
@@ -205,7 +215,8 @@ def is_setting_overridden(key: str) -> bool:
|
|||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
with open(SETTINGS_FILE, "r", encoding="utf-8") as f:
|
||||||
return key in json.load(f)
|
saved = json.load(f)
|
||||||
|
return isinstance(saved, dict) and key in saved
|
||||||
except (FileNotFoundError, json.JSONDecodeError):
|
except (FileNotFoundError, json.JSONDecodeError):
|
||||||
return False
|
return False
|
||||||
|
|
||||||
@@ -256,8 +267,10 @@ def load_features() -> dict:
|
|||||||
try:
|
try:
|
||||||
with open(FEATURES_FILE, "r", encoding="utf-8") as f:
|
with open(FEATURES_FILE, "r", encoding="utf-8") as f:
|
||||||
saved = json.load(f)
|
saved = json.load(f)
|
||||||
|
if not isinstance(saved, dict):
|
||||||
|
raise ValueError("features must be an object")
|
||||||
merged = {**DEFAULT_FEATURES, **saved}
|
merged = {**DEFAULT_FEATURES, **saved}
|
||||||
except (FileNotFoundError, json.JSONDecodeError):
|
except (FileNotFoundError, json.JSONDecodeError, ValueError):
|
||||||
merged = dict(DEFAULT_FEATURES)
|
merged = dict(DEFAULT_FEATURES)
|
||||||
_features_cache = (now, merged)
|
_features_cache = (now, merged)
|
||||||
return merged
|
return merged
|
||||||
|
|||||||
@@ -3,11 +3,11 @@
|
|||||||
from src.endpoint_resolver import resolve_endpoint
|
from src.endpoint_resolver import resolve_endpoint
|
||||||
|
|
||||||
|
|
||||||
def resolve_task_endpoint(fallback_url=None, fallback_model=None, fallback_headers=None):
|
def resolve_task_endpoint(fallback_url=None, fallback_model=None, fallback_headers=None, owner=None):
|
||||||
"""Return (endpoint_url, model, headers) for background tasks.
|
"""Return (endpoint_url, model, headers) for background tasks.
|
||||||
|
|
||||||
Reads task_endpoint_id / task_model from admin settings.
|
Reads task_endpoint_id / task_model from admin settings.
|
||||||
Falls back to the provided values when the setting is empty or the
|
Falls back to the provided values when the setting is empty or the
|
||||||
endpoint cannot be resolved.
|
endpoint cannot be resolved.
|
||||||
"""
|
"""
|
||||||
return resolve_endpoint("task", fallback_url, fallback_model, fallback_headers)
|
return resolve_endpoint("task", fallback_url, fallback_model, fallback_headers, owner=owner)
|
||||||
|
|||||||
@@ -115,7 +115,7 @@ def compute_next_run(schedule: str, scheduled_time: str,
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
if schedule == "once":
|
if schedule == "once":
|
||||||
if scheduled_date and scheduled_date > (now.replace(tzinfo=None) if tz is not None else now):
|
if scheduled_date and scheduled_date > (_to_utc_naive(now) if tz is not None else now):
|
||||||
return scheduled_date
|
return scheduled_date
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|||||||
@@ -327,7 +327,7 @@ def _extract_skill_json(teacher_response: str) -> Optional[Dict[str, Any]]:
|
|||||||
treated as "teacher declined to write a skill", per the prompt
|
treated as "teacher declined to write a skill", per the prompt
|
||||||
contract.
|
contract.
|
||||||
"""
|
"""
|
||||||
if not teacher_response:
|
if not isinstance(teacher_response, str) or not teacher_response:
|
||||||
return None
|
return None
|
||||||
import json
|
import json
|
||||||
m = re.search(r"```(?:json)?\s*\n(\{[\s\S]*?\})\s*\n```", teacher_response)
|
m = re.search(r"```(?:json)?\s*\n(\{[\s\S]*?\})\s*\n```", teacher_response)
|
||||||
|
|||||||
@@ -88,6 +88,24 @@ def get_active_document():
|
|||||||
return _active_document_id
|
return _active_document_id
|
||||||
|
|
||||||
|
|
||||||
|
def clear_active_document(doc_id: Optional[str] = None) -> bool:
|
||||||
|
"""Clear the in-memory active-document pointer.
|
||||||
|
|
||||||
|
With ``doc_id`` given, only clears when it matches the current pointer, so a
|
||||||
|
different active document is left untouched. Returns True if it was cleared.
|
||||||
|
|
||||||
|
Called when a document is detached from its session or deleted (its tab is
|
||||||
|
closed): without this, the stale pointer makes the last-resort doc-injection
|
||||||
|
path re-surface a closed document in a later, unrelated chat — even one whose
|
||||||
|
session no longer matches — because an unlinked doc has session_id NULL (#1160).
|
||||||
|
"""
|
||||||
|
global _active_document_id
|
||||||
|
if doc_id is None or _active_document_id == doc_id:
|
||||||
|
_active_document_id = None
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def _owned_document_query(query, Document, owner: Optional[str]):
|
def _owned_document_query(query, Document, owner: Optional[str]):
|
||||||
if owner is None:
|
if owner is None:
|
||||||
# A bare Python `False` is not a valid SQL expression — SQLAlchemy 1.4
|
# A bare Python `False` is not a valid SQL expression — SQLAlchemy 1.4
|
||||||
@@ -892,7 +910,9 @@ async def do_manage_tasks(content: str, owner: Optional[str] = None) -> Dict:
|
|||||||
)
|
)
|
||||||
|
|
||||||
task_id = str(_uuid.uuid4())
|
task_id = str(_uuid.uuid4())
|
||||||
name = args.get("name") or args.get("prompt", args.get("action_name", "Task"))[:50]
|
# Guard each fallback with `or`: args.get("prompt", default) returns
|
||||||
|
# None when the key is present but null, and None[:50] raises.
|
||||||
|
name = args.get("name") or (args.get("prompt") or args.get("action_name") or "Task")[:50]
|
||||||
|
|
||||||
task = ScheduledTask(
|
task = ScheduledTask(
|
||||||
id=task_id,
|
id=task_id,
|
||||||
@@ -1195,7 +1215,17 @@ async def do_manage_mcp(content: str, owner: Optional[str] = None) -> Dict:
|
|||||||
try:
|
try:
|
||||||
srv = db2.query(McpServer).filter(McpServer.id == sid).first()
|
srv = db2.query(McpServer).filter(McpServer.id == sid).first()
|
||||||
if srv:
|
if srv:
|
||||||
await mcp.connect_server(sid)
|
_args = json.loads(srv.args) if srv.args else []
|
||||||
|
_env = json.loads(srv.env) if srv.env else {}
|
||||||
|
await mcp.connect_server(
|
||||||
|
server_id=sid,
|
||||||
|
name=srv.name,
|
||||||
|
transport=srv.transport,
|
||||||
|
command=srv.command,
|
||||||
|
args=_args,
|
||||||
|
env=_env,
|
||||||
|
url=srv.url,
|
||||||
|
)
|
||||||
st = mcp.get_server_status(sid)
|
st = mcp.get_server_status(sid)
|
||||||
return {"response": f"Reconnected '{srv.name}' ({st.get('tool_count', 0)} tools)", "exit_code": 0}
|
return {"response": f"Reconnected '{srv.name}' ({st.get('tool_count', 0)} tools)", "exit_code": 0}
|
||||||
return {"error": f"Server {sid} not found", "exit_code": 1}
|
return {"error": f"Server {sid} not found", "exit_code": 1}
|
||||||
@@ -1507,7 +1537,14 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
|
|||||||
"tavily_api_key", "serper_api_key", "app_public_url",
|
"tavily_api_key", "serper_api_key", "app_public_url",
|
||||||
}
|
}
|
||||||
def _is_secret(k):
|
def _is_secret(k):
|
||||||
return k in _SECRET_KEYS or any(t in k for t in ("api_key", "_key", "token", "secret", "password"))
|
# `token` must be a suffix, not a substring: otherwise the int
|
||||||
|
# setting `agent_input_token_budget` (which even has a "token budget"
|
||||||
|
# alias to set it from chat) is wrongly classified as a credential.
|
||||||
|
return (
|
||||||
|
k in _SECRET_KEYS
|
||||||
|
or k.endswith("token")
|
||||||
|
or any(t in k for t in ("api_key", "_key", "secret", "password"))
|
||||||
|
)
|
||||||
|
|
||||||
# Friendly aliases → real keys, so natural phrasing resolves.
|
# Friendly aliases → real keys, so natural phrasing resolves.
|
||||||
_ALIASES_SET = {
|
_ALIASES_SET = {
|
||||||
@@ -1530,7 +1567,10 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
|
|||||||
"ntfy topic": "reminder_ntfy_topic",
|
"ntfy topic": "reminder_ntfy_topic",
|
||||||
"agent tool calls": "agent_max_tool_calls", "max tool calls": "agent_max_tool_calls",
|
"agent tool calls": "agent_max_tool_calls", "max tool calls": "agent_max_tool_calls",
|
||||||
"agent timeout": "agent_stream_timeout_seconds", "stream timeout": "agent_stream_timeout_seconds",
|
"agent timeout": "agent_stream_timeout_seconds", "stream timeout": "agent_stream_timeout_seconds",
|
||||||
"token budget": "agent_input_token_budget",
|
"token budget": "agent_input_token_budget", "input budget": "agent_input_token_budget",
|
||||||
|
"hard max": "agent_input_token_hard_max",
|
||||||
|
"token budget cap": "agent_input_token_hard_max",
|
||||||
|
"input budget cap": "agent_input_token_hard_max",
|
||||||
}
|
}
|
||||||
def _resolve(k):
|
def _resolve(k):
|
||||||
k2 = (k or "").strip().lower()
|
k2 = (k or "").strip().lower()
|
||||||
@@ -4064,7 +4104,9 @@ async def do_vault_unlock(content: str, owner: Optional[str] = None) -> Dict:
|
|||||||
if not master_password:
|
if not master_password:
|
||||||
return {"error": "master_password is required", "exit_code": 1}
|
return {"error": "master_password is required", "exit_code": 1}
|
||||||
|
|
||||||
stdout, stderr, rc = await _run_bw(["unlock", master_password, "--raw"])
|
# Do not pass the master password as an argv element. Local process lists
|
||||||
|
# can expose argv to other users; stdin keeps the secret out of `ps`.
|
||||||
|
stdout, stderr, rc = await _run_bw(["unlock", "--raw"], input_text=master_password + "\n")
|
||||||
if rc != 0:
|
if rc != 0:
|
||||||
return {"error": f"Unlock failed: {stderr[:300]}", "exit_code": 1}
|
return {"error": f"Unlock failed: {stderr[:300]}", "exit_code": 1}
|
||||||
|
|
||||||
|
|||||||
@@ -69,6 +69,8 @@ _TOOL_CODE_RE = re.compile(
|
|||||||
# fullwidth (U+FF5C) and ascii '|' in any count.
|
# fullwidth (U+FF5C) and ascii '|' in any count.
|
||||||
_DSML_PIPES = r"[||]+"
|
_DSML_PIPES = r"[||]+"
|
||||||
def _normalize_dsml(text: str) -> str:
|
def _normalize_dsml(text: str) -> str:
|
||||||
|
if not isinstance(text, str):
|
||||||
|
return ""
|
||||||
if "DSML" not in text:
|
if "DSML" not in text:
|
||||||
return text
|
return text
|
||||||
t = text
|
t = text
|
||||||
|
|||||||
+2
-1
@@ -442,7 +442,8 @@ FUNCTION_TOOL_SCHEMAS = [
|
|||||||
"end": {"type": "string", "description": "list_events range end (ISO datetime); defaults to +14 days"},
|
"end": {"type": "string", "description": "list_events range end (ISO datetime); defaults to +14 days"},
|
||||||
"event_type": {"type": "string", "description": "Tag / category for the event. Common values: work, personal, health, travel, meal, social, admin, other. Aliases accepted: tag, category, type."},
|
"event_type": {"type": "string", "description": "Tag / category for the event. Common values: work, personal, health, travel, meal, social, admin, other. Aliases accepted: tag, category, type."},
|
||||||
"importance": {"type": "string", "enum": ["low", "normal", "high", "critical"], "description": "Priority level (defaults to 'normal')"},
|
"importance": {"type": "string", "enum": ["low", "normal", "high", "critical"], "description": "Priority level (defaults to 'normal')"},
|
||||||
"reminder_minutes": {"type": "integer", "description": "For create_event: create an Odysseus reminder this many minutes before the event, e.g. 5 for 'reminder 5 min before'."}
|
"reminder_minutes": {"type": "integer", "description": "For create_event: create an Odysseus reminder this many minutes before the event, e.g. 5 for 'reminder 5 min before'."},
|
||||||
|
"rrule": {"type": "string", "description": "Recurrence rule in iCalendar RRULE format, e.g. 'FREQ=WEEKLY;BYDAY=MO' for weekly on Monday. Use with create_event or update_event."}
|
||||||
},
|
},
|
||||||
"required": ["action"]
|
"required": ["action"]
|
||||||
}
|
}
|
||||||
|
|||||||
+19
-1
@@ -43,6 +43,18 @@ def is_valid_upload_id(upload_id: str) -> bool:
|
|||||||
return UPLOAD_ID_RE.fullmatch(upload_id or "") is not None
|
return UPLOAD_ID_RE.fullmatch(upload_id or "") is not None
|
||||||
|
|
||||||
|
|
||||||
|
def count_recent_uploads(timestamps, now: float, window: float = 10.0) -> int:
|
||||||
|
"""Number of upload events in *timestamps* within the last *window* seconds.
|
||||||
|
|
||||||
|
Used by the per-IP concurrency guard. The count is of genuine prior upload
|
||||||
|
events — it must NOT scale with how many files are in the *current* request,
|
||||||
|
or a single multi-file batch would reject itself (issue #1346)."""
|
||||||
|
if not timestamps:
|
||||||
|
return 0
|
||||||
|
cutoff = now - window
|
||||||
|
return sum(1 for t in timestamps if t > cutoff)
|
||||||
|
|
||||||
|
|
||||||
class UploadHandler:
|
class UploadHandler:
|
||||||
def __init__(self, base_dir: str, upload_dir: str):
|
def __init__(self, base_dir: str, upload_dir: str):
|
||||||
self.base_dir = base_dir
|
self.base_dir = base_dir
|
||||||
@@ -50,7 +62,13 @@ class UploadHandler:
|
|||||||
self.max_upload_size = 10 * 1024 * 1024 # 10MB
|
self.max_upload_size = 10 * 1024 * 1024 # 10MB
|
||||||
self.max_concurrent_uploads = 3
|
self.max_concurrent_uploads = 3
|
||||||
self.cleanup_days = 30
|
self.cleanup_days = 30
|
||||||
self.upload_rate_limit = 5 # Max 5 uploads per minute per IP
|
# Per-IP per-minute cap. save_upload() counts EACH file, and the chat
|
||||||
|
# composer lets a user attach up to MAX_FILES (10, static/js/fileHandler.js)
|
||||||
|
# in one batch — so this must comfortably exceed 10, or a single 6+ file
|
||||||
|
# attach is rejected mid-batch (issue #1346: "5 work, 6 fail"). Burst abuse
|
||||||
|
# is separately bounded by max_concurrent_uploads. Headroom for a few full
|
||||||
|
# batches per minute.
|
||||||
|
self.upload_rate_limit = 60 # max 60 file-uploads per minute per IP
|
||||||
self.upload_rate_window = 60 # 60 seconds
|
self.upload_rate_window = 60 # 60 seconds
|
||||||
|
|
||||||
# Track upload rates
|
# Track upload rates
|
||||||
|
|||||||
@@ -56,6 +56,8 @@ def check_outbound_url(
|
|||||||
Returns ``(ok, reason)``. ``ok`` is True only when the URL is safe to fetch.
|
Returns ``(ok, reason)``. ``ok`` is True only when the URL is safe to fetch.
|
||||||
``resolver`` is injectable so callers/tests can avoid real DNS.
|
``resolver`` is injectable so callers/tests can avoid real DNS.
|
||||||
"""
|
"""
|
||||||
|
if not isinstance(url, str):
|
||||||
|
return False, "URL must be a string"
|
||||||
if not url or not url.strip():
|
if not url or not url.strip():
|
||||||
return False, "URL is required"
|
return False, "URL is required"
|
||||||
try:
|
try:
|
||||||
|
|||||||
+8
-1
@@ -532,6 +532,13 @@ function initializeEventListeners() {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Model picker popup — close before opening any modals
|
||||||
|
const modelPickerMenu = document.getElementById('model-picker-menu');
|
||||||
|
if (modelPickerMenu && modelPickerMenu.classList.contains('open')) {
|
||||||
|
modelPickerMenu.classList.remove('open');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
// Close one modal at a time (last in DOM = topmost)
|
// Close one modal at a time (last in DOM = topmost)
|
||||||
// Map modal id → sidebar list-item id to clear active state
|
// Map modal id → sidebar list-item id to clear active state
|
||||||
const modalItemMap = {
|
const modalItemMap = {
|
||||||
@@ -543,7 +550,7 @@ function initializeEventListeners() {
|
|||||||
};
|
};
|
||||||
|
|
||||||
// Dynamic modals (removed from DOM on close)
|
// Dynamic modals (removed from DOM on close)
|
||||||
const dynamicModals = ['library-modal', 'archive-modal', 'doclib-modal', 'gallery-modal', 'tasks-modal'];
|
const dynamicModals = ['library-modal', 'archive-modal', 'doclib-modal', 'gallery-modal', 'tasks-modal', 'email-lib-modal'];
|
||||||
for (const id of dynamicModals) {
|
for (const id of dynamicModals) {
|
||||||
const m = document.getElementById(id);
|
const m = document.getElementById(id);
|
||||||
if (id === 'gallery-modal') {
|
if (id === 'gallery-modal') {
|
||||||
|
|||||||
+8
-3
@@ -4037,8 +4037,11 @@ import createResearchSynapse from './researchSynapse.js';
|
|||||||
const clickedIndex = allMsgs.indexOf(msgElement);
|
const clickedIndex = allMsgs.indexOf(msgElement);
|
||||||
if (clickedIndex < 0) return;
|
if (clickedIndex < 0) return;
|
||||||
|
|
||||||
|
// No early-out on a missing session: an output shown before any model was
|
||||||
|
// selected (issue #1428) has no session/persisted rows, but its "x" must
|
||||||
|
// still remove it. We only need the session id for the server-side delete
|
||||||
|
// below; without one we fall back to removing the DOM.
|
||||||
const sessionId = sessionModule.getCurrentSessionId();
|
const sessionId = sessionModule.getCurrentSessionId();
|
||||||
if (!sessionId) return;
|
|
||||||
|
|
||||||
const clickedIsUser = msgElement.classList.contains('msg-user');
|
const clickedIsUser = msgElement.classList.contains('msg-user');
|
||||||
|
|
||||||
@@ -4114,8 +4117,10 @@ import createResearchSynapse from './researchSynapse.js';
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!msgIds.length) {
|
if (!msgIds.length || !sessionId) {
|
||||||
// Fallback: just remove DOM elements if no DB IDs available
|
// No persisted rows to delete (no DB IDs, or no session at all — e.g. an
|
||||||
|
// error output shown before a model was selected, #1428). Just remove the
|
||||||
|
// DOM so the "x" works regardless.
|
||||||
domToRemove.forEach(el => el.remove());
|
domToRemove.forEach(el => el.remove());
|
||||||
if (uiModule) uiModule.showToast('Message deleted');
|
if (uiModule) uiModule.showToast('Message deleted');
|
||||||
return;
|
return;
|
||||||
|
|||||||
@@ -1217,6 +1217,17 @@ export function showWelcomeScreen() {
|
|||||||
const cc = document.getElementById('chat-container');
|
const cc = document.getElementById('chat-container');
|
||||||
if (ws) ws.classList.remove('hidden');
|
if (ws) ws.classList.remove('hidden');
|
||||||
if (cc) cc.classList.add('welcome-active');
|
if (cc) cc.classList.add('welcome-active');
|
||||||
|
// Entering the New Chat / welcome state: discard any stale draft left in the
|
||||||
|
// composer from the previous session so the input starts empty (issue #1343).
|
||||||
|
// Switching between existing sessions loads them directly and does NOT call
|
||||||
|
// this, so genuine drafts are not erased. Reset the autosized height and fire
|
||||||
|
// an `input` event so the send button + autosize listeners update.
|
||||||
|
const _msg = document.getElementById('message');
|
||||||
|
if (_msg) {
|
||||||
|
_msg.value = '';
|
||||||
|
_msg.style.height = '';
|
||||||
|
_msg.dispatchEvent(new Event('input', { bubbles: true }));
|
||||||
|
}
|
||||||
// Re-trigger the L→R clip-wipe reveal on the welcome name each time the
|
// Re-trigger the L→R clip-wipe reveal on the welcome name each time the
|
||||||
// welcome screen is shown (new session, deleted last session, etc.) — without
|
// welcome screen is shown (new session, deleted last session, etc.) — without
|
||||||
// this, the CSS animation only fires on initial DOM insertion.
|
// this, the CSS animation only fires on initial DOM insertion.
|
||||||
|
|||||||
@@ -401,13 +401,17 @@ export function _buildServeCmd(f, modelName, backend) {
|
|||||||
const ggufPath = f._gguf_path || 'model.gguf';
|
const ggufPath = f._gguf_path || 'model.gguf';
|
||||||
const gpuId = f.gpu_id?.trim() || '';
|
const gpuId = f.gpu_id?.trim() || '';
|
||||||
const py = _isWindows() ? 'python' : 'python3';
|
const py = _isWindows() ? 'python' : 'python3';
|
||||||
|
// CPU-only serve (-ngl 0): drop the GPU-only flags, otherwise the command
|
||||||
|
// mixes "zero GPU layers" with CUDA unified-memory + flash-attn and fails to
|
||||||
|
// start (issue #1291). Only affects the ngl=0 path; GPU serving is unchanged.
|
||||||
|
const _cpuOnly = String(f.ngl).trim() === '0';
|
||||||
const lcPrefix = (() => {
|
const lcPrefix = (() => {
|
||||||
let p = '';
|
let p = '';
|
||||||
if (f.unified_mem && !_isWindows()) p += `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 `;
|
if (f.unified_mem && !_cpuOnly && !_isWindows()) p += `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 `;
|
||||||
if (gpuId && !_isWindows()) p += `CUDA_VISIBLE_DEVICES=${gpuId} `;
|
if (gpuId && !_isWindows()) p += `CUDA_VISIBLE_DEVICES=${gpuId} `;
|
||||||
return p;
|
return p;
|
||||||
})();
|
})();
|
||||||
if (f.unified_mem && _isWindows()) cmd += `$env:GGML_CUDA_ENABLE_UNIFIED_MEMORY="1"; `;
|
if (f.unified_mem && !_cpuOnly && _isWindows()) cmd += `$env:GGML_CUDA_ENABLE_UNIFIED_MEMORY="1"; `;
|
||||||
if (gpuId && _isWindows()) cmd += `$env:CUDA_VISIBLE_DEVICES="${gpuId}"; `;
|
if (gpuId && _isWindows()) cmd += `$env:CUDA_VISIBLE_DEVICES="${gpuId}"; `;
|
||||||
if (!_isWindows()) {
|
if (!_isWindows()) {
|
||||||
// Resolve GGUF path once, fail loudly if nothing matched (prevents
|
// Resolve GGUF path once, fail loudly if nothing matched (prevents
|
||||||
@@ -439,7 +443,7 @@ export function _buildServeCmd(f, modelName, backend) {
|
|||||||
_lcExtra += ` --n-cpu-moe ${_ncm}`;
|
_lcExtra += ` --n-cpu-moe ${_ncm}`;
|
||||||
_lcpExtra += ` --n_cpu_moe ${_ncm}`; // llama-cpp-python uses underscores
|
_lcpExtra += ` --n_cpu_moe ${_ncm}`; // llama-cpp-python uses underscores
|
||||||
}
|
}
|
||||||
if (f.flash_attn) {
|
if (f.flash_attn && !_cpuOnly) {
|
||||||
_lcExtra += ' --flash-attn on';
|
_lcExtra += ' --flash-attn on';
|
||||||
_lcpExtra += ' --flash_attn true';
|
_lcpExtra += ' --flash_attn true';
|
||||||
}
|
}
|
||||||
@@ -1524,7 +1528,7 @@ function _renderRecipes() {
|
|||||||
html += '<option value="Q4_K_M">Q4</option><option value="Q8_0">Q8</option>';
|
html += '<option value="Q4_K_M">Q4</option><option value="Q8_0">Q8</option>';
|
||||||
html += '<option value="Q6_K">Q6</option><option value="Q5_K_M">Q5</option>';
|
html += '<option value="Q6_K">Q6</option><option value="Q5_K_M">Q5</option>';
|
||||||
html += '<option value="Q3_K_M">Q3</option><option value="Q2_K">Q2</option>';
|
html += '<option value="Q3_K_M">Q3</option><option value="Q2_K">Q2</option>';
|
||||||
html += '<option value="AWQ-4bit">AWQ</option><option value="FP8">FP8</option><option value="FP4">FP4</option>';
|
html += '<option value="AWQ-4bit">AWQ</option><option value="FP8">FP8</option><option value="FP4">FP4</option><option value="NVFP4">NVFP4</option>';
|
||||||
html += '<option value="">Native</option></select>';
|
html += '<option value="">Native</option></select>';
|
||||||
// Engine filter: show only models whose serve engine matches. "llama.cpp"
|
// Engine filter: show only models whose serve engine matches. "llama.cpp"
|
||||||
// (GGUF) runs everywhere incl. consumer AMD/Apple; vLLM/SGLang are CUDA /
|
// (GGUF) runs everywhere incl. consumer AMD/Apple; vLLM/SGLang are CUDA /
|
||||||
|
|||||||
@@ -555,18 +555,20 @@ export async function _runModelDownload(panel, model, backend, hostOverride) {
|
|||||||
body: JSON.stringify(payload),
|
body: JSON.stringify(payload),
|
||||||
});
|
});
|
||||||
if (!res.ok) {
|
if (!res.ok) {
|
||||||
uiModule.showToast('Download failed: HTTP ' + res.status);
|
// Errors carry actionable text (e.g. "tmux is required …"); keep them up
|
||||||
|
// long enough to read, matching the serve path's duration (issue #1355).
|
||||||
|
uiModule.showToast('Download failed: HTTP ' + res.status, 9000);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const data = await res.json();
|
const data = await res.json();
|
||||||
if (!data.ok) {
|
if (!data.ok) {
|
||||||
uiModule.showToast('Download failed: ' + (data.error || ''));
|
uiModule.showToast('Download failed: ' + (data.error || ''), 9000);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
_addTask(data.session_id, shortName, 'download', payload);
|
_addTask(data.session_id, shortName, 'download', payload);
|
||||||
uiModule.showToast(`Downloading ${shortName}...`);
|
uiModule.showToast(`Downloading ${shortName}...`);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
uiModule.showToast('Download failed: ' + e.message);
|
uiModule.showToast('Download failed: ' + e.message, 9000);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -246,10 +246,20 @@ function _selectedGgufExpr(model, repo, relPath) {
|
|||||||
const base = String(model.path || '').replace(/\/+$/, '');
|
const base = String(model.path || '').replace(/\/+$/, '');
|
||||||
return `$(printf %s ${_shellPathExpr(`${base}/${repo}/${rel}`)})`;
|
return `$(printf %s ${_shellPathExpr(`${base}/${repo}/${rel}`)})`;
|
||||||
}
|
}
|
||||||
|
if (model.path) {
|
||||||
|
const base = String(model.path || '').replace(/\/+$/, '');
|
||||||
|
return `$(printf %s ${_shellPathExpr(`${base}/models--${repo.replace(/\//g, '--')}/snapshots/${rel}`)})`;
|
||||||
|
}
|
||||||
const cacheRepo = repo.replace(/\//g, '--');
|
const cacheRepo = repo.replace(/\//g, '--');
|
||||||
return `$(printf %s \${HOME}${_shellQuote(`/.cache/huggingface/hub/models--${cacheRepo}/snapshots/${rel}`)})`;
|
return `$(printf %s \${HOME}${_shellQuote(`/.cache/huggingface/hub/models--${cacheRepo}/snapshots/${rel}`)})`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function _ggufSearchDirExpr(model, repo) {
|
||||||
|
if (model.is_local_dir && model.path) return _shellQuote(`${String(model.path || '').replace(/\/+$/, '')}/${repo}`);
|
||||||
|
if (model.path) return _shellQuote(`${String(model.path || '').replace(/\/+$/, '')}/models--${repo.replace(/\//g, '--')}/snapshots`);
|
||||||
|
return `"$HOME/.cache/huggingface/hub/models--${repo.replace(/\//g, '--')}/snapshots"`;
|
||||||
|
}
|
||||||
|
|
||||||
function _rerenderCachedModels() {
|
function _rerenderCachedModels() {
|
||||||
const list = document.getElementById('hwfit-cached-list');
|
const list = document.getElementById('hwfit-cached-list');
|
||||||
const tagContainer = document.getElementById('serve-tags');
|
const tagContainer = document.getElementById('serve-tags');
|
||||||
@@ -736,13 +746,12 @@ function _rerenderCachedModels() {
|
|||||||
// For multi-part GGUFs, llama.cpp requires the first split
|
// For multi-part GGUFs, llama.cpp requires the first split
|
||||||
// (-00001-of-NNNNN.gguf). Prefer it (sorted, so UD-IQ4_XS/001 comes
|
// (-00001-of-NNNNN.gguf). Prefer it (sorted, so UD-IQ4_XS/001 comes
|
||||||
// before Q4_K_M/001 etc); fall back to any single GGUF sorted.
|
// before Q4_K_M/001 etc); fall back to any single GGUF sorted.
|
||||||
// Use $HOME (not ~) so tilde survives variable interpolation inside $(...).
|
const dir = _ggufSearchDirExpr(m, repo);
|
||||||
const dir = `"$HOME/.cache/huggingface/hub/models--${repo.replace(/\//g, '--')}/snapshots"`;
|
|
||||||
// GGUF needs the actual .gguf FILE, not the folder. For a custom-dir
|
// GGUF needs the actual .gguf FILE, not the folder. For a custom-dir
|
||||||
// model the file lives under "<path>/<repo>" — search there just like we
|
// model the file lives under "<path>/<repo>" — search there just like we
|
||||||
// search the HF snapshots dir, so serving a GGUF from a custom dir works
|
// search the HF snapshots dir, so serving a GGUF from a custom dir works
|
||||||
// instead of handing llama.cpp a directory (which fails).
|
// instead of handing llama.cpp a directory (which fails).
|
||||||
const _ldir = `"${m.path}/${repo}"`;
|
const _ldir = m.path ? _shellQuote(`${m.path}/${repo}`) : '""';
|
||||||
f._gguf_path = selectedGguf
|
f._gguf_path = selectedGguf
|
||||||
? _selectedGgufExpr(m, repo, selectedGguf.rel_path)
|
? _selectedGgufExpr(m, repo, selectedGguf.rel_path)
|
||||||
: m.is_local_dir && m.path
|
: m.is_local_dir && m.path
|
||||||
|
|||||||
@@ -1721,7 +1721,7 @@ async function _loadEmails({ force = false, useCache = true } = {}) {
|
|||||||
async function _loadScheduled(grid, sp) {
|
async function _loadScheduled(grid, sp) {
|
||||||
const res = await fetch(`${API_BASE}/api/email/scheduled`);
|
const res = await fetch(`${API_BASE}/api/email/scheduled`);
|
||||||
const data = await res.json();
|
const data = await res.json();
|
||||||
sp.destroy();
|
if (sp) sp.destroy();
|
||||||
const items = data.scheduled || [];
|
const items = data.scheduled || [];
|
||||||
grid.innerHTML = '';
|
grid.innerHTML = '';
|
||||||
const stats = document.getElementById('email-lib-stats');
|
const stats = document.getElementById('email-lib-stats');
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ function _createChip(f, idx) {
|
|||||||
chip.classList.add('thumb-image'); // lets CSS overlay the remove-X on the corner (mobile)
|
chip.classList.add('thumb-image'); // lets CSS overlay the remove-X on the corner (mobile)
|
||||||
const img = document.createElement('img');
|
const img = document.createElement('img');
|
||||||
img.className = 'thumb-img';
|
img.className = 'thumb-img';
|
||||||
img.src = URL.createObjectURL(f);
|
img.src = _getPreviewUrl(f);
|
||||||
img.alt = f.name || 'image';
|
img.alt = f.name || 'image';
|
||||||
chip.appendChild(img);
|
chip.appendChild(img);
|
||||||
} else {
|
} else {
|
||||||
@@ -172,6 +172,17 @@ export async function uploadPending() {
|
|||||||
method: 'POST',
|
method: 'POST',
|
||||||
body: fd
|
body: fd
|
||||||
});
|
});
|
||||||
|
if (!res.ok) {
|
||||||
|
// Surface the failure instead of swallowing it. Previously a non-OK
|
||||||
|
// response (e.g. 429 rate limit, 413 too large) was ignored: the files
|
||||||
|
// silently vanished and the chat sent with no attachments, so the model
|
||||||
|
// "didn't even see them" (issue #1346). Show the server's reason and keep
|
||||||
|
// pendingFiles so the strip re-renders for a retry (see finally below).
|
||||||
|
let detail = '';
|
||||||
|
try { const e = await res.json(); detail = e.detail || e.error || ''; } catch (_) {}
|
||||||
|
_showToast('Upload failed' + (detail ? ': ' + detail : ` (HTTP ${res.status})`));
|
||||||
|
return [];
|
||||||
|
}
|
||||||
const data = await res.json();
|
const data = await res.json();
|
||||||
uploaded = (data.files || []);
|
uploaded = (data.files || []);
|
||||||
pendingFiles = []; // clear only on success
|
pendingFiles = []; // clear only on success
|
||||||
|
|||||||
+5
-3
@@ -8,6 +8,7 @@ import spinnerModule from './spinner.js';
|
|||||||
import { providerLogo } from './providers.js';
|
import { providerLogo } from './providers.js';
|
||||||
import { PROMPT_TEMPLATES, getAllPresets } from './presets.js';
|
import { PROMPT_TEMPLATES, getAllPresets } from './presets.js';
|
||||||
import { sortModelObjects } from './modelSort.js';
|
import { sortModelObjects } from './modelSort.js';
|
||||||
|
import Storage from './storage.js';
|
||||||
|
|
||||||
let API_BASE = '';
|
let API_BASE = '';
|
||||||
let _active = false;
|
let _active = false;
|
||||||
@@ -57,7 +58,7 @@ function _initGroupTab() {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
_modelsCache = sortModelObjects(result);
|
_modelsCache = sortModelObjects(result);
|
||||||
return result;
|
return _modelsCache;
|
||||||
}
|
}
|
||||||
|
|
||||||
function _render() {
|
function _render() {
|
||||||
@@ -409,7 +410,7 @@ export async function showModelPicker() {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
_cachedModels = sortModelObjects(result);
|
_cachedModels = sortModelObjects(result);
|
||||||
return result;
|
return _cachedModels;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function render(filter) {
|
async function render(filter) {
|
||||||
@@ -546,7 +547,8 @@ export async function startGroup(models, parentSessionId) {
|
|||||||
_parentSessionId = pdata.id;
|
_parentSessionId = pdata.id;
|
||||||
// Register as group session for sidebar icon
|
// Register as group session for sidebar icon
|
||||||
try {
|
try {
|
||||||
const gids = JSON.parse(localStorage.getItem('odysseus-group-sessions') || '[]');
|
const storedGroupSessions = Storage.getJSON('odysseus-group-sessions', []);
|
||||||
|
const gids = Array.isArray(storedGroupSessions) ? storedGroupSessions : [];
|
||||||
if (!gids.includes(_parentSessionId)) { gids.push(_parentSessionId); localStorage.setItem('odysseus-group-sessions', JSON.stringify(gids)); }
|
if (!gids.includes(_parentSessionId)) { gids.push(_parentSessionId); localStorage.setItem('odysseus-group-sessions', JSON.stringify(gids)); }
|
||||||
} catch (e) {}
|
} catch (e) {}
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
|
|||||||
@@ -536,16 +536,18 @@ export function mdToHtml(src) {
|
|||||||
let html = '<table style="border-collapse: collapse; width: 100%; margin: 10px 0;">';
|
let html = '<table style="border-collapse: collapse; width: 100%; margin: 10px 0;">';
|
||||||
|
|
||||||
rows.forEach((row, idx) => {
|
rows.forEach((row, idx) => {
|
||||||
|
if (idx === 1 && /^[\s|:\-]+$/.test(row)) {
|
||||||
|
html += '<tbody>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
const cells = splitTableRow(row);
|
const cells = splitTableRow(row);
|
||||||
if (cells.length === 0) return;
|
if (cells.length === 0) return;
|
||||||
|
|
||||||
html += idx === 1 ? '<tbody>' : '';
|
|
||||||
html += '<tr>';
|
html += '<tr>';
|
||||||
|
|
||||||
cells.forEach(cell => {
|
cells.forEach(cell => {
|
||||||
const tag = idx === 0 ? 'th' : 'td';
|
const tag = idx === 0 ? 'th' : 'td';
|
||||||
const style = idx === 1 ? 'style="border-top: 2px solid var(--red);"' : '';
|
html += `<${tag} style="padding: 8px; text-align: left; border-bottom: 1px solid var(--border);">${cell.trim()}</${tag}>`;
|
||||||
html += `<${tag} ${style} style="padding: 8px; text-align: left; border-bottom: 1px solid var(--border);">${cell.trim()}</${tag}>`;
|
|
||||||
});
|
});
|
||||||
|
|
||||||
html += '</tr>';
|
html += '</tr>';
|
||||||
|
|||||||
+23
-5
@@ -8,6 +8,24 @@ let API_BASE = '';
|
|||||||
let selectedPreset = null;
|
let selectedPreset = null;
|
||||||
let presets = {};
|
let presets = {};
|
||||||
|
|
||||||
|
export function loadStoredArray(key) {
|
||||||
|
try {
|
||||||
|
const value = JSON.parse(localStorage.getItem(key) || '[]');
|
||||||
|
return Array.isArray(value) ? value : [];
|
||||||
|
} catch (e) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadStoredObject(key) {
|
||||||
|
try {
|
||||||
|
const value = JSON.parse(localStorage.getItem(key) || '{}');
|
||||||
|
return value && typeof value === 'object' && !Array.isArray(value) ? value : {};
|
||||||
|
} catch (e) {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Built-in prompt templates (moved from cot_prompts.py)
|
// Built-in prompt templates (moved from cot_prompts.py)
|
||||||
export const PROMPT_TEMPLATES = [
|
export const PROMPT_TEMPLATES = [
|
||||||
{
|
{
|
||||||
@@ -228,7 +246,7 @@ function initNameDropdown() {
|
|||||||
}
|
}
|
||||||
// Hide built-in preset
|
// Hide built-in preset
|
||||||
if (isBuiltin) {
|
if (isBuiltin) {
|
||||||
const hidden = JSON.parse(localStorage.getItem('odysseus-hidden-presets') || '[]');
|
const hidden = loadStoredArray('odysseus-hidden-presets');
|
||||||
if (!hidden.includes(charName)) hidden.push(charName);
|
if (!hidden.includes(charName)) hidden.push(charName);
|
||||||
localStorage.setItem('odysseus-hidden-presets', JSON.stringify(hidden));
|
localStorage.setItem('odysseus-hidden-presets', JSON.stringify(hidden));
|
||||||
}
|
}
|
||||||
@@ -311,7 +329,7 @@ function _populateCharSelect() {
|
|||||||
select.appendChild(group);
|
select.appendChild(group);
|
||||||
}
|
}
|
||||||
|
|
||||||
const hiddenPresets = JSON.parse(localStorage.getItem('odysseus-hidden-presets') || '[]');
|
const hiddenPresets = loadStoredArray('odysseus-hidden-presets');
|
||||||
const builtins = PROMPT_TEMPLATES.filter(t => !savedNames.has(t.name) && !hiddenPresets.includes(t.name));
|
const builtins = PROMPT_TEMPLATES.filter(t => !savedNames.has(t.name) && !hiddenPresets.includes(t.name));
|
||||||
if (builtins.length) {
|
if (builtins.length) {
|
||||||
const group = document.createElement('optgroup');
|
const group = document.createElement('optgroup');
|
||||||
@@ -405,7 +423,7 @@ function initPersistentChat() {
|
|||||||
await fetch(`${API_BASE}/api/session/${sessionId}/important`, { method: 'POST', body: favFd });
|
await fetch(`${API_BASE}/api/session/${sessionId}/important`, { method: 'POST', body: favFd });
|
||||||
|
|
||||||
// Save session → character mapping so it restores on switch
|
// Save session → character mapping so it restores on switch
|
||||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||||
charSessions[sessionId] = charName;
|
charSessions[sessionId] = charName;
|
||||||
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
||||||
|
|
||||||
@@ -1011,7 +1029,7 @@ function _syncCharIndicator() {
|
|||||||
let _prevSessionId = null;
|
let _prevSessionId = null;
|
||||||
|
|
||||||
export function onSessionSwitch(sessionId) {
|
export function onSessionSwitch(sessionId) {
|
||||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||||
|
|
||||||
// Leaving a persistent chat — deactivate for this switch only
|
// Leaving a persistent chat — deactivate for this switch only
|
||||||
if (window._persistentChatSession) {
|
if (window._persistentChatSession) {
|
||||||
@@ -1059,7 +1077,7 @@ export function isPersistentChat() {
|
|||||||
* Remove a session from persistent chat mappings (call when session is deleted).
|
* Remove a session from persistent chat mappings (call when session is deleted).
|
||||||
*/
|
*/
|
||||||
export function removePersistentChat(sessionId) {
|
export function removePersistentChat(sessionId) {
|
||||||
const charSessions = JSON.parse(localStorage.getItem('odysseus-char-sessions') || '{}');
|
const charSessions = loadStoredObject('odysseus-char-sessions');
|
||||||
if (charSessions[sessionId]) {
|
if (charSessions[sessionId]) {
|
||||||
delete charSessions[sessionId];
|
delete charSessions[sessionId];
|
||||||
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
localStorage.setItem('odysseus-char-sessions', JSON.stringify(charSessions));
|
||||||
|
|||||||
+2
-1
@@ -7,6 +7,7 @@ import markdownModule from './markdown.js';
|
|||||||
import * as spinnerModule from './spinner.js';
|
import * as spinnerModule from './spinner.js';
|
||||||
import { makeWindowDraggable } from './windowDrag.js';
|
import { makeWindowDraggable } from './windowDrag.js';
|
||||||
import { sortModelIds } from './modelSort.js';
|
import { sortModelIds } from './modelSort.js';
|
||||||
|
import { ordinalSuffix } from './util/ordinal.js';
|
||||||
|
|
||||||
const API_BASE = window.location.origin;
|
const API_BASE = window.location.origin;
|
||||||
let _open = false;
|
let _open = false;
|
||||||
@@ -244,7 +245,7 @@ function _scheduleLabel(task) {
|
|||||||
}
|
}
|
||||||
if (task.schedule === 'monthly') {
|
if (task.schedule === 'monthly') {
|
||||||
const d = task.scheduled_day ?? 1;
|
const d = task.scheduled_day ?? 1;
|
||||||
const suffix = d === 1 ? 'st' : d === 2 ? 'nd' : d === 3 ? 'rd' : 'th';
|
const suffix = ordinalSuffix(d);
|
||||||
return `Monthly on ${d}${suffix} at ${localTime}`;
|
return `Monthly on ${d}${suffix} at ${localTime}`;
|
||||||
}
|
}
|
||||||
return task.schedule || '—';
|
return task.schedule || '—';
|
||||||
|
|||||||
@@ -0,0 +1,13 @@
|
|||||||
|
// Pure (browser-free) English ordinal suffix, e.g. 1 -> "st", 21 -> "st",
|
||||||
|
// 22 -> "nd", 23 -> "rd", 11/12/13 -> "th". Extracted so it can be unit-tested.
|
||||||
|
export function ordinalSuffix(n) {
|
||||||
|
const a = Math.abs(Math.trunc(Number(n) || 0));
|
||||||
|
const mod100 = a % 100;
|
||||||
|
if (mod100 >= 11 && mod100 <= 13) return 'th';
|
||||||
|
switch (a % 10) {
|
||||||
|
case 1: return 'st';
|
||||||
|
case 2: return 'nd';
|
||||||
|
case 3: return 'rd';
|
||||||
|
default: return 'th';
|
||||||
|
}
|
||||||
|
}
|
||||||
+63
-1
@@ -1141,7 +1141,10 @@ body.bg-pattern-sparkles {
|
|||||||
display: flex; align-items: center; gap: 6px;
|
display: flex; align-items: center; gap: 6px;
|
||||||
margin: 0; padding: 0; border: none;
|
margin: 0; padding: 0; border: none;
|
||||||
font-size: 10px; font-weight: 400; font-family: inherit;
|
font-size: 10px; font-weight: 400; font-family: inherit;
|
||||||
line-height: 1; letter-spacing: 0; text-transform: none;
|
/* 1.3 (not 1) so Fira Code's tall glyph box isn't vertically clipped in
|
||||||
|
Chromium/Edge — mirrors the .list-item fix. The title is flex-centred
|
||||||
|
in a fixed-height (29px) header, so this adds headroom without reflow. */
|
||||||
|
line-height: 1.3; letter-spacing: 0; text-transform: none;
|
||||||
color: var(--fg);
|
color: var(--fg);
|
||||||
}
|
}
|
||||||
.section-icon,
|
.section-icon,
|
||||||
@@ -1801,6 +1804,15 @@ body.bg-pattern-sparkles {
|
|||||||
max-height: 60px;
|
max-height: 60px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
}
|
}
|
||||||
|
/* The tip is a full sentence that wraps to 4-5 lines on narrow phones,
|
||||||
|
where the welcome block shrink-wraps small; the shared 60px ceiling
|
||||||
|
(sized for the one-line sub/version) clipped its last line. Give the tip
|
||||||
|
a taller ceiling so it isn't truncated. Kept above the max-height:650px
|
||||||
|
block below so that rule's max-height:0 still collapses it on short
|
||||||
|
viewports. */
|
||||||
|
#welcome-screen .welcome-tip {
|
||||||
|
max-height: 120px;
|
||||||
|
}
|
||||||
@media (max-height: 650px) {
|
@media (max-height: 650px) {
|
||||||
#welcome-screen { top: 28%; }
|
#welcome-screen { top: 28%; }
|
||||||
#welcome-screen .welcome-tip { opacity: 0; max-height: 0; margin: 0; }
|
#welcome-screen .welcome-tip { opacity: 0; max-height: 0; margin: 0; }
|
||||||
@@ -35357,3 +35369,53 @@ body.theme-frosted .modal {
|
|||||||
font-family: 'Fira Code', ui-monospace, monospace;
|
font-family: 'Fira Code', ui-monospace, monospace;
|
||||||
color: var(--accent, var(--red));
|
color: var(--accent, var(--red));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ══ iOS focus-zoom fix — touch devices only; desktop sizes untouched ══
|
||||||
|
16px is the threshold below which iOS Safari auto-zooms on focus.
|
||||||
|
Selects and date/time inputs are excluded on purpose — they open native
|
||||||
|
pickers and never zoom. */
|
||||||
|
@media (hover: none) and (pointer: coarse) {
|
||||||
|
|
||||||
|
/* 1 ── Catch-all: every text-entry control NOT pinned with its own
|
||||||
|
!important. !important here beats any non-important rule regardless of
|
||||||
|
specificity, so this clears the long tail (settings, admin, memory,
|
||||||
|
notes, calendar, email, gallery, tasks, model picker, etc.). */
|
||||||
|
input[type="text"],
|
||||||
|
input[type="search"],
|
||||||
|
input[type="email"],
|
||||||
|
input[type="url"],
|
||||||
|
input[type="tel"],
|
||||||
|
input[type="password"],
|
||||||
|
input[type="number"],
|
||||||
|
input:not([type]),
|
||||||
|
textarea {
|
||||||
|
font-size: 16px !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* 2 ── Fields that pin their own !important at specificity our catch-all
|
||||||
|
can't beat. Each is matched at equal-or-higher specificity and, being
|
||||||
|
later in the file, wins the tie. */
|
||||||
|
#message { font-size: 16px !important; } /* chat composer (was 13px !important) */
|
||||||
|
.cookbook-dl-repo,
|
||||||
|
.hwfit-search { font-size: 16px !important; } /* cookbook repo path + hardware search */
|
||||||
|
.ge-topbar input { font-size: 16px !important; } /* image-editor topbar input */
|
||||||
|
.ge-transform-field > input.ge-transform-popup-input { /* image-editor transform values */
|
||||||
|
font-size: 16px !important;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (hover: none) and (pointer: coarse) {
|
||||||
|
/* Only the sub-16px tiers need bumping; large lands ABOVE 16 so it
|
||||||
|
stays zoom-safe AND visibly larger than medium (otherwise L collapses
|
||||||
|
onto M on touch). All three editor layers move together so the
|
||||||
|
highlight/line-number overlay stays metrically aligned with the textarea. */
|
||||||
|
.doc-font-m .doc-editor-textarea, .doc-font-m .doc-editor-highlight, .doc-font-m .doc-line-numbers {
|
||||||
|
font-size: 16px !important; /* was 13px */
|
||||||
|
}
|
||||||
|
.doc-font-l .doc-editor-textarea, .doc-font-l .doc-editor-highlight, .doc-font-l .doc-line-numbers {
|
||||||
|
font-size: 18px !important; /* was 15px — keep L > M */
|
||||||
|
}
|
||||||
|
/* Email compose rich-body. Medium (15px) zooms, so bump it; large (17px)
|
||||||
|
is already ≥16px and never zoomed — leave it so we don't shrink it. */
|
||||||
|
.doc-email-richbody.doc-font-m { font-size: 16px !important; }
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,39 @@
|
|||||||
|
"""Issue #1160 — a closed document must not stay 'active' and leak into new chats.
|
||||||
|
|
||||||
|
Closing a document tab detaches it (session_id -> NULL) or deletes it, but the
|
||||||
|
in-memory active-document pointer was never cleared, so the last-resort doc
|
||||||
|
injection re-surfaced the closed doc in later, unrelated chats. The document
|
||||||
|
routes now call clear_active_document() on detach/delete; this pins that helper.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from src.tool_implementations import (
|
||||||
|
set_active_document,
|
||||||
|
get_active_document,
|
||||||
|
clear_active_document,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_clear_matching_id_resets_pointer():
|
||||||
|
set_active_document("doc-123")
|
||||||
|
assert get_active_document() == "doc-123"
|
||||||
|
assert clear_active_document("doc-123") is True
|
||||||
|
assert get_active_document() is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_clear_non_matching_id_leaves_other_active_doc():
|
||||||
|
set_active_document("doc-abc")
|
||||||
|
# Closing a DIFFERENT document must not clobber the currently active one.
|
||||||
|
assert clear_active_document("doc-xyz") is False
|
||||||
|
assert get_active_document() == "doc-abc"
|
||||||
|
|
||||||
|
|
||||||
|
def test_clear_without_id_clears_unconditionally():
|
||||||
|
set_active_document("doc-abc")
|
||||||
|
assert clear_active_document() is True
|
||||||
|
assert get_active_document() is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_clear_when_already_none_is_safe():
|
||||||
|
set_active_document(None)
|
||||||
|
assert clear_active_document("doc-123") is False
|
||||||
|
assert get_active_document() is None
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "check-docker-amd-gpu.sh"
|
||||||
|
|
||||||
|
|
||||||
|
def test_amd_gpu_check_rejects_unknown_extra_arg_before_diagnostics():
|
||||||
|
proc = subprocess.run(
|
||||||
|
["bash", str(SCRIPT), "--bad-option"],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert proc.returncode == 1
|
||||||
|
assert "Unknown option: --bad-option" in proc.stderr
|
||||||
|
|
||||||
|
|
||||||
|
def test_amd_gpu_check_shell_syntax():
|
||||||
|
subprocess.run(["bash", "-n", str(SCRIPT)], check=True)
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
"""Regression: APIKeyManager.load() must not crash on a corrupt/wrong-shape file.
|
||||||
|
|
||||||
|
load() is called during startup (app_initializer). It had no try/except around
|
||||||
|
`json.load` and called `encrypted_keys.items()` directly, so a corrupt/truncated
|
||||||
|
api_keys.json raised JSONDecodeError and a legacy list-shaped file raised
|
||||||
|
AttributeError — both crashing app startup. It now returns {} instead.
|
||||||
|
"""
|
||||||
|
from src.api_key_manager import APIKeyManager
|
||||||
|
|
||||||
|
|
||||||
|
def _mgr(tmp_path):
|
||||||
|
return APIKeyManager(str(tmp_path))
|
||||||
|
|
||||||
|
|
||||||
|
def test_corrupt_json_returns_empty(tmp_path):
|
||||||
|
(tmp_path / "api_keys.json").write_text("{not valid json", encoding="utf-8")
|
||||||
|
assert _mgr(tmp_path).load() == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_list_shape_returns_empty(tmp_path):
|
||||||
|
(tmp_path / "api_keys.json").write_text('["openai", "anthropic"]', encoding="utf-8")
|
||||||
|
assert _mgr(tmp_path).load() == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_file_returns_empty(tmp_path):
|
||||||
|
assert _mgr(tmp_path).load() == {}
|
||||||
|
|
||||||
|
|
||||||
|
def test_valid_roundtrip(tmp_path):
|
||||||
|
mgr = _mgr(tmp_path)
|
||||||
|
mgr.save("openai", "sk-secret")
|
||||||
|
assert mgr.load() == {"openai": "sk-secret"}
|
||||||
@@ -30,6 +30,17 @@ def _verify_args(path: Path):
|
|||||||
return SimpleNamespace(path=str(path), pretty=False)
|
return SimpleNamespace(path=str(path), pretty=False)
|
||||||
|
|
||||||
|
|
||||||
|
def test_snapshot_rejects_output_inside_data_dir(tmp_path, monkeypatch):
|
||||||
|
backup = _load_backup_cli()
|
||||||
|
repo = tmp_path / "repo"
|
||||||
|
data = repo / "data"
|
||||||
|
data.mkdir(parents=True)
|
||||||
|
_patch_repo(backup, monkeypatch, repo)
|
||||||
|
|
||||||
|
with pytest.raises(SystemExit):
|
||||||
|
backup._reject_output_inside_data(data / "self.tar.gz")
|
||||||
|
|
||||||
|
|
||||||
def test_restore_rejects_symlink_escape(tmp_path, monkeypatch):
|
def test_restore_rejects_symlink_escape(tmp_path, monkeypatch):
|
||||||
backup = _load_backup_cli()
|
backup = _load_backup_cli()
|
||||||
repo = tmp_path / "repo"
|
repo = tmp_path / "repo"
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""Regression: builtin_actions heuristics must tolerate non-string input.
|
||||||
|
|
||||||
|
_result_has_work did `result.lower()` after a falsy-only guard, and
|
||||||
|
_classify_event_heuristic did `(summary or "").lower()`; a truthy non-string
|
||||||
|
(e.g. a dict) raised AttributeError. They now coerce/guard non-strings.
|
||||||
|
"""
|
||||||
|
from src.builtin_actions import _result_has_work, _classify_event_heuristic
|
||||||
|
|
||||||
|
|
||||||
|
def test_result_has_work_non_string():
|
||||||
|
assert _result_has_work({"x": 1}) is False
|
||||||
|
assert _result_has_work(123) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_classify_event_heuristic_non_string():
|
||||||
|
out = _classify_event_heuristic(123)
|
||||||
|
assert isinstance(out, tuple)
|
||||||
|
|
||||||
|
|
||||||
|
def test_valid_inputs_unchanged():
|
||||||
|
assert _result_has_work("Processed 0 emails") is False
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src import caldav_sync
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_caldav_url_normalizes_safe_url():
|
||||||
|
assert (
|
||||||
|
caldav_sync.validate_caldav_url(" https://calendar.example.com/dav/ ")
|
||||||
|
== "https://calendar.example.com/dav"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"url, message",
|
||||||
|
[
|
||||||
|
("ftp://calendar.example.com/dav", "must start with"),
|
||||||
|
("https://alice:secret@calendar.example.com/dav", "credentials"),
|
||||||
|
("https://calendar.example.com/dav#frag", "fragments"),
|
||||||
|
("http://localhost:5232/dav", "host is not allowed"),
|
||||||
|
("http://service.localhost/dav", "host is not allowed"),
|
||||||
|
("http://127.0.0.1:5232/dav", "host is not allowed"),
|
||||||
|
("http://[::1]:5232/dav", "host is not allowed"),
|
||||||
|
("http://169.254.169.254/latest", "host is not allowed"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_validate_caldav_url_rejects_unsafe_urls(url, message):
|
||||||
|
with pytest.raises(ValueError, match=message):
|
||||||
|
caldav_sync.validate_caldav_url(url)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_caldav_url_blocks_private_ips_unless_explicitly_allowed(monkeypatch):
|
||||||
|
monkeypatch.delenv("ODYSSEUS_ALLOW_PRIVATE_CALDAV", raising=False)
|
||||||
|
with pytest.raises(ValueError, match="Private CalDAV IPs require"):
|
||||||
|
caldav_sync.validate_caldav_url("http://10.0.0.5:5232/dav")
|
||||||
|
|
||||||
|
monkeypatch.setenv("ODYSSEUS_ALLOW_PRIVATE_CALDAV", "1")
|
||||||
|
assert caldav_sync.validate_caldav_url("http://10.0.0.5:5232/dav") == "http://10.0.0.5:5232/dav"
|
||||||
|
|
||||||
|
|
||||||
|
def test_sync_caldav_decrypts_stored_password_and_validates_url(monkeypatch):
|
||||||
|
prefs_mod = types.ModuleType("routes.prefs_routes")
|
||||||
|
prefs_mod._load_for_user = lambda owner: {
|
||||||
|
"caldav": {
|
||||||
|
"url": " https://calendar.example.com/dav/ ",
|
||||||
|
"username": owner,
|
||||||
|
"password": "enc:stored",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
monkeypatch.setitem(sys.modules, "routes.prefs_routes", prefs_mod)
|
||||||
|
|
||||||
|
secret_mod = types.ModuleType("src.secret_storage")
|
||||||
|
secret_mod.decrypt = lambda value: "decrypted-password" if value == "enc:stored" else value
|
||||||
|
monkeypatch.setitem(sys.modules, "src.secret_storage", secret_mod)
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
def fake_sync_blocking(owner, url, username, password):
|
||||||
|
captured.update(
|
||||||
|
{
|
||||||
|
"owner": owner,
|
||||||
|
"url": url,
|
||||||
|
"username": username,
|
||||||
|
"password": password,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {"calendars": 1, "events": 0, "deleted": 0, "errors": []}
|
||||||
|
|
||||||
|
async def inline_to_thread(func, *args, **kwargs):
|
||||||
|
return func(*args, **kwargs)
|
||||||
|
|
||||||
|
monkeypatch.setattr(caldav_sync, "_sync_blocking", fake_sync_blocking)
|
||||||
|
monkeypatch.setattr(caldav_sync.asyncio, "to_thread", inline_to_thread)
|
||||||
|
|
||||||
|
result = asyncio.run(caldav_sync.sync_caldav("alice"))
|
||||||
|
|
||||||
|
assert result["calendars"] == 1
|
||||||
|
assert captured == {
|
||||||
|
"owner": "alice",
|
||||||
|
"url": "https://calendar.example.com/dav",
|
||||||
|
"username": "alice",
|
||||||
|
"password": "decrypted-password",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_calendar_routes_use_hardened_caldav_client_and_secret_storage():
|
||||||
|
text = Path("routes/calendar_routes.py").read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
assert "validate_caldav_url(body.get(\"url\", \"\"))" in text
|
||||||
|
assert "cfg[\"password\"] = encrypt(body[\"password\"])" in text
|
||||||
|
assert "pw = decrypt(pw)" in text
|
||||||
|
assert "follow_redirects=False, trust_env=False" in text
|
||||||
|
assert "Redirects are not followed for CalDAV safety" in text
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""Regression: validate_caldav_url must reject a non-string via its normal
|
||||||
|
ValueError path, not crash with TypeError.
|
||||||
|
|
||||||
|
It did `(raw_url or "").strip()`, so a non-string scalar (e.g. an int from a
|
||||||
|
mis-typed config) reached `.strip()` and raised TypeError instead of the
|
||||||
|
function\'s own ValueError.
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.caldav_sync import validate_caldav_url
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_string_raises_valueerror_not_typeerror():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
validate_caldav_url(12345)
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
validate_caldav_url(None)
|
||||||
|
|
||||||
|
|
||||||
|
def test_valid_url_passes():
|
||||||
|
out = validate_caldav_url("https://dav.example.com/calendars/")
|
||||||
|
assert "example.com" in out
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
"""Issue #800 — CalDAV write-back pushes local changes to the remote server.
|
||||||
|
|
||||||
|
Unit-tests the pure pieces against a fake caldav calendar (no network): the
|
||||||
|
iCalendar serialization, hash-based remote-calendar discovery, and the
|
||||||
|
create/update/delete orchestration.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from src.caldav_writeback import (
|
||||||
|
build_event_ical,
|
||||||
|
find_remote_calendar,
|
||||||
|
push_event,
|
||||||
|
_stable_cal_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
REMOTE_URL = "https://p69-caldav.icloud.com/123/calendars/home/"
|
||||||
|
CAL_ID = _stable_cal_id(REMOTE_URL)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeEvent:
|
||||||
|
def __init__(self):
|
||||||
|
self.data = "OLD"
|
||||||
|
self.saved = False
|
||||||
|
self.deleted = False
|
||||||
|
|
||||||
|
def save(self):
|
||||||
|
self.saved = True
|
||||||
|
|
||||||
|
def delete(self):
|
||||||
|
self.deleted = True
|
||||||
|
|
||||||
|
|
||||||
|
class FakeCalendar:
|
||||||
|
def __init__(self, url, existing=None):
|
||||||
|
self.url = url
|
||||||
|
self._existing = existing
|
||||||
|
self.saved_ical = None
|
||||||
|
|
||||||
|
def event_by_uid(self, uid):
|
||||||
|
if self._existing is None:
|
||||||
|
raise Exception("not found")
|
||||||
|
return self._existing
|
||||||
|
|
||||||
|
def save_event(self, ical):
|
||||||
|
self.saved_ical = ical
|
||||||
|
|
||||||
|
|
||||||
|
def _ev(**over):
|
||||||
|
base = dict(
|
||||||
|
uid="evt-1", summary="Dentist", description="bring x-rays",
|
||||||
|
location="Clinic", dtstart=datetime(2026, 6, 10, 14, 0),
|
||||||
|
dtend=datetime(2026, 6, 10, 15, 0), all_day=False, is_utc=True, rrule="",
|
||||||
|
)
|
||||||
|
base.update(over)
|
||||||
|
return base
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_ical_timed_event_has_core_fields():
|
||||||
|
ical = build_event_ical(_ev())
|
||||||
|
assert "BEGIN:VEVENT" in ical and "END:VEVENT" in ical
|
||||||
|
assert "UID:evt-1" in ical
|
||||||
|
assert "SUMMARY:Dentist" in ical
|
||||||
|
# is_utc -> UTC instant (Z suffix)
|
||||||
|
assert "DTSTART:20260610T140000Z" in ical
|
||||||
|
assert "DTEND:20260610T150000Z" in ical
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_ical_all_day_uses_date_values():
|
||||||
|
ical = build_event_ical(_ev(all_day=True, is_utc=False))
|
||||||
|
assert "DTSTART;VALUE=DATE:20260610" in ical
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_ical_includes_rrule():
|
||||||
|
ical = build_event_ical(_ev(rrule="FREQ=WEEKLY;BYDAY=MO"))
|
||||||
|
assert "RRULE:FREQ=WEEKLY" in ical
|
||||||
|
|
||||||
|
|
||||||
|
def test_find_remote_calendar_matches_by_hash():
|
||||||
|
cals = [FakeCalendar("https://other/x/"), FakeCalendar(REMOTE_URL)]
|
||||||
|
found = find_remote_calendar(cals, CAL_ID)
|
||||||
|
assert found is cals[1]
|
||||||
|
assert find_remote_calendar([FakeCalendar("https://nope/")], CAL_ID) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_create_calls_save_event():
|
||||||
|
cal = FakeCalendar(REMOTE_URL, existing=None) # event_by_uid raises -> create
|
||||||
|
res = push_event([cal], CAL_ID, _ev(), delete=False)
|
||||||
|
assert res["ok"] and res.get("created")
|
||||||
|
assert cal.saved_ical and "UID:evt-1" in cal.saved_ical
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_update_overwrites_existing():
|
||||||
|
existing = FakeEvent()
|
||||||
|
cal = FakeCalendar(REMOTE_URL, existing=existing)
|
||||||
|
res = push_event([cal], CAL_ID, _ev(summary="Moved"), delete=False)
|
||||||
|
assert res["ok"] and res.get("updated")
|
||||||
|
assert existing.saved and "SUMMARY:Moved" in existing.data
|
||||||
|
assert cal.saved_ical is None # used update path, not create
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_delete_removes_existing():
|
||||||
|
existing = FakeEvent()
|
||||||
|
cal = FakeCalendar(REMOTE_URL, existing=existing)
|
||||||
|
res = push_event([cal], CAL_ID, _ev(), delete=True)
|
||||||
|
assert res["ok"] and existing.deleted
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_delete_absent_is_ok():
|
||||||
|
cal = FakeCalendar(REMOTE_URL, existing=None)
|
||||||
|
res = push_event([cal], CAL_ID, _ev(), delete=True)
|
||||||
|
assert res["ok"] and "absent" in res.get("note", "")
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_unknown_calendar_reports_not_found():
|
||||||
|
cal = FakeCalendar("https://different/")
|
||||||
|
res = push_event([cal], CAL_ID, _ev())
|
||||||
|
assert res["ok"] is False and "not found" in res["error"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_push_missing_uid_reports_input_error_before_remote_lookup():
|
||||||
|
cal = FakeCalendar(REMOTE_URL, existing=FakeEvent())
|
||||||
|
res = push_event([cal], CAL_ID, _ev(uid=""))
|
||||||
|
assert res["ok"] is False and "uid" in res["error"]
|
||||||
|
assert cal._existing.saved is False
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
"""Issue #800 — the calendar write handlers actually trigger CalDAV write-back.
|
||||||
|
|
||||||
|
Route-level: proves POST/DELETE /api/calendar/events fire writeback_event for a
|
||||||
|
CalDAV-backed calendar and not for a local one.
|
||||||
|
|
||||||
|
Calls the async route handlers DIRECTLY (extracted from the router) rather than
|
||||||
|
through Starlette's TestClient — the TestClient middleware-app + threadpool could
|
||||||
|
hang in some environments; a direct call with a minimal fake request keeps the
|
||||||
|
same coverage and completes reliably.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
import uuid
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import sessionmaker
|
||||||
|
from sqlalchemy.pool import NullPool
|
||||||
|
|
||||||
|
import core.database as cdb
|
||||||
|
import routes.calendar_routes as croutes
|
||||||
|
import src.caldav_writeback as wb
|
||||||
|
from core.database import CalendarCal
|
||||||
|
from routes.calendar_routes import EventCreate
|
||||||
|
|
||||||
|
_TMPDB = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||||
|
_ENGINE = create_engine(
|
||||||
|
f"sqlite:///{_TMPDB.name}",
|
||||||
|
connect_args={"check_same_thread": False},
|
||||||
|
poolclass=NullPool,
|
||||||
|
)
|
||||||
|
cdb.Base.metadata.create_all(_ENGINE)
|
||||||
|
_TS = sessionmaker(bind=_ENGINE, autoflush=False, autocommit=False)
|
||||||
|
croutes.SessionLocal = _TS
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def calls(monkeypatch):
|
||||||
|
recorded = []
|
||||||
|
|
||||||
|
async def _fake_writeback(owner, source, cal_id, ev, *, delete=False):
|
||||||
|
recorded.append({"source": source, "cal_id": cal_id, "uid": ev.get("uid"), "delete": delete})
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
monkeypatch.setattr(wb, "writeback_event", _fake_writeback)
|
||||||
|
return recorded
|
||||||
|
|
||||||
|
|
||||||
|
def _req():
|
||||||
|
return SimpleNamespace(state=SimpleNamespace(current_user="tester"))
|
||||||
|
|
||||||
|
|
||||||
|
def _endpoint(method, suffix):
|
||||||
|
router = croutes.setup_calendar_routes()
|
||||||
|
for r in router.routes:
|
||||||
|
if getattr(r, "path", "").endswith(suffix) and method in getattr(r, "methods", set()):
|
||||||
|
return r.endpoint
|
||||||
|
raise RuntimeError(f"{method} *{suffix} not found")
|
||||||
|
|
||||||
|
|
||||||
|
def _make_cal(source):
|
||||||
|
cid = ("caldav-" if source == "caldav" else "loc-") + uuid.uuid4().hex[:10]
|
||||||
|
db = _TS()
|
||||||
|
try:
|
||||||
|
db.add(CalendarCal(id=cid, owner="tester", name="C", source=source))
|
||||||
|
db.commit()
|
||||||
|
return cid
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_on_caldav_calendar_pushes_to_remote(calls):
|
||||||
|
create_event = _endpoint("POST", "/events")
|
||||||
|
cal_id = _make_cal("caldav")
|
||||||
|
res = await create_event(_req(), EventCreate(
|
||||||
|
summary="Dentist", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||||
|
assert res["ok"] is True
|
||||||
|
assert len(calls) == 1
|
||||||
|
assert calls[0]["source"] == "caldav" and calls[0]["cal_id"] == cal_id
|
||||||
|
assert calls[0]["delete"] is False
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_on_local_calendar_does_not_push(calls):
|
||||||
|
create_event = _endpoint("POST", "/events")
|
||||||
|
cal_id = _make_cal("local")
|
||||||
|
res = await create_event(_req(), EventCreate(
|
||||||
|
summary="Local", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||||
|
assert res["ok"] is True
|
||||||
|
assert calls == []
|
||||||
|
|
||||||
|
|
||||||
|
async def test_delete_on_caldav_calendar_pushes_delete(calls):
|
||||||
|
create_event = _endpoint("POST", "/events")
|
||||||
|
delete_event = _endpoint("DELETE", "/events/{uid}")
|
||||||
|
cal_id = _make_cal("caldav")
|
||||||
|
res = await create_event(_req(), EventCreate(
|
||||||
|
summary="Temp", dtstart="2026-06-10T14:00:00Z", calendar_href=cal_id))
|
||||||
|
uid = res["uid"]
|
||||||
|
calls.clear()
|
||||||
|
rd = await delete_event(_req(), uid)
|
||||||
|
assert rd["ok"] is True
|
||||||
|
assert len(calls) == 1 and calls[0]["delete"] is True and calls[0]["uid"] == uid
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
"""Issue #1320 — the agent's manage_calendar tool can create a recurring event.
|
||||||
|
|
||||||
|
The create_event handler already persists `rrule`, but it wasn't documented in the
|
||||||
|
tool schema, so the agent took "a roundabout way". This pins the end-to-end path:
|
||||||
|
calling do_manage_calendar with an rrule stores a single event carrying that RRULE.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import sessionmaker
|
||||||
|
from sqlalchemy.pool import NullPool
|
||||||
|
|
||||||
|
import core.database as cdb
|
||||||
|
from core.database import CalendarEvent
|
||||||
|
|
||||||
|
_TMPDB = tempfile.NamedTemporaryFile(suffix=".db", delete=False)
|
||||||
|
_ENGINE = create_engine(
|
||||||
|
f"sqlite:///{_TMPDB.name}",
|
||||||
|
connect_args={"check_same_thread": False},
|
||||||
|
poolclass=NullPool,
|
||||||
|
)
|
||||||
|
cdb.Base.metadata.create_all(_ENGINE)
|
||||||
|
_TS = sessionmaker(bind=_ENGINE, autoflush=False, autocommit=False)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _bind_temp_db(monkeypatch):
|
||||||
|
# do_manage_calendar does `from core.database import SessionLocal` at call
|
||||||
|
# time, so patch the module attribute to our temp DB — via monkeypatch so it
|
||||||
|
# is RESTORED after each test and can't leak into later tests in the process.
|
||||||
|
monkeypatch.setattr(cdb, "SessionLocal", _TS)
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_event_with_rrule_persists_recurrence():
|
||||||
|
from src.tool_implementations import do_manage_calendar
|
||||||
|
|
||||||
|
owner = "tester-" + uuid.uuid4().hex[:6]
|
||||||
|
rrule = "FREQ=WEEKLY;BYDAY=MO"
|
||||||
|
res = await do_manage_calendar(json.dumps({
|
||||||
|
"action": "create_event",
|
||||||
|
"summary": "Standup",
|
||||||
|
"dtstart": "2026-06-08T09:00:00Z",
|
||||||
|
"rrule": rrule,
|
||||||
|
}), owner=owner)
|
||||||
|
assert res.get("exit_code", 0) == 0, res
|
||||||
|
uid = res.get("uid")
|
||||||
|
assert uid, res
|
||||||
|
|
||||||
|
db = _TS()
|
||||||
|
try:
|
||||||
|
ev = db.query(CalendarEvent).filter(CalendarEvent.uid == uid).first()
|
||||||
|
assert ev is not None
|
||||||
|
assert ev.rrule == rrule # ONE event carrying the recurrence rule
|
||||||
|
assert ev.summary == "Standup"
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_create_event_without_rrule_is_single():
|
||||||
|
from src.tool_implementations import do_manage_calendar
|
||||||
|
|
||||||
|
owner = "tester-" + uuid.uuid4().hex[:6]
|
||||||
|
res = await do_manage_calendar(json.dumps({
|
||||||
|
"action": "create_event",
|
||||||
|
"summary": "One-off",
|
||||||
|
"dtstart": "2026-06-09T10:00:00Z",
|
||||||
|
}), owner=owner)
|
||||||
|
assert res.get("exit_code", 0) == 0, res
|
||||||
|
db = _TS()
|
||||||
|
try:
|
||||||
|
ev = db.query(CalendarEvent).filter(CalendarEvent.uid == res["uid"]).first()
|
||||||
|
assert ev is not None and (ev.rrule or "") == ""
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
"""Regression: check_outbound_url must reject a non-string URL, not crash.
|
||||||
|
|
||||||
|
The `if not url or not url.strip()` guard only handled falsy values; a truthy
|
||||||
|
non-string (e.g. an int) reached `.strip()` and raised AttributeError out of
|
||||||
|
this SSRF check. Non-strings now fail closed with a clear message.
|
||||||
|
"""
|
||||||
|
from src.url_safety import check_outbound_url
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_string_fails_closed():
|
||||||
|
ok, _ = check_outbound_url(123)
|
||||||
|
assert ok is False
|
||||||
|
ok2, _ = check_outbound_url(None)
|
||||||
|
assert ok2 is False
|
||||||
@@ -44,3 +44,75 @@ def test_is_setting_overridden_reads_raw_saved_file(tmp_path, monkeypatch):
|
|||||||
|
|
||||||
f.write_text(json.dumps({}), encoding="utf-8")
|
f.write_text(json.dumps({}), encoding="utf-8")
|
||||||
assert settings.is_setting_overridden("agent_input_token_budget") is False
|
assert settings.is_setting_overridden("agent_input_token_budget") is False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Configurable hard_max — completes the reviewer requirement from #1190 that
|
||||||
|
# was carried over but not implemented in #1230: the ceiling on the auto-
|
||||||
|
# derived path should be a setting, not a hidden constant. Without this,
|
||||||
|
# admins on premium APIs with very large windows (1M+ context) can only
|
||||||
|
# raise the ceiling by editing src/context_budget.py.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_custom_hard_max_overrides_default_in_auto_branch():
|
||||||
|
"""A caller-supplied hard_max lifts the auto-derived ceiling."""
|
||||||
|
# Without override: 1M ctx -> capped at DEFAULT_HARD_MAX (200K)
|
||||||
|
assert compute_input_token_budget(6000, 1_000_000, explicit=False) == DEFAULT_HARD_MAX
|
||||||
|
# With explicit raise: 1M ctx -> 850K (85% of 1M), under the raised ceiling
|
||||||
|
assert compute_input_token_budget(6000, 1_000_000, explicit=False, hard_max=900_000) == int(1_000_000 * 0.85)
|
||||||
|
|
||||||
|
|
||||||
|
def test_custom_hard_max_lowers_default_for_cost_paranoid_setups():
|
||||||
|
"""A lower ceiling caps the auto-derived budget below the default."""
|
||||||
|
# 128K ctx, default ceiling 200K -> 85% of 128K = 108800
|
||||||
|
assert compute_input_token_budget(6000, 128_000, explicit=False) == int(128_000 * 0.85)
|
||||||
|
# Same ctx, ceiling lowered to 50K -> capped at 50K instead
|
||||||
|
assert compute_input_token_budget(6000, 128_000, explicit=False, hard_max=50_000) == 50_000
|
||||||
|
|
||||||
|
|
||||||
|
def test_hard_max_has_no_effect_on_explicit_branch():
|
||||||
|
"""When the user set an explicit budget, hard_max must not silently cap it."""
|
||||||
|
# User chose 900K explicitly; ctx is 1M; ceiling is 100K — user's choice wins.
|
||||||
|
assert compute_input_token_budget(900_000, 1_000_000, explicit=True, hard_max=100_000) == 900_000
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_settings_registers_hard_max_key():
|
||||||
|
"""Required so /api/auth/settings and manage_settings can persist the key."""
|
||||||
|
from src.settings import DEFAULT_SETTINGS
|
||||||
|
assert "agent_input_token_hard_max" in DEFAULT_SETTINGS
|
||||||
|
assert DEFAULT_SETTINGS["agent_input_token_hard_max"] == DEFAULT_HARD_MAX
|
||||||
|
|
||||||
|
|
||||||
|
def test_alias_map_registers_friendly_names():
|
||||||
|
"""`manage_settings` should accept 'hard max' and friends."""
|
||||||
|
from pathlib import Path
|
||||||
|
src = Path("src/tool_implementations.py").read_text()
|
||||||
|
assert '"hard max": "agent_input_token_hard_max"' in src
|
||||||
|
assert '"token budget cap": "agent_input_token_hard_max"' in src
|
||||||
|
assert '"input budget cap": "agent_input_token_hard_max"' in src
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_loop_reads_hard_max_setting(tmp_path, monkeypatch):
|
||||||
|
"""End-to-end: a saved settings.json value for agent_input_token_hard_max
|
||||||
|
must reach compute_input_token_budget on the real agent_loop call path."""
|
||||||
|
import src.settings as settings
|
||||||
|
# Point SETTINGS_FILE at a temp file with our override.
|
||||||
|
f = tmp_path / "settings.json"
|
||||||
|
f.write_text(json.dumps({"agent_input_token_hard_max": 750_000}), encoding="utf-8")
|
||||||
|
monkeypatch.setattr(settings, "SETTINGS_FILE", str(f))
|
||||||
|
monkeypatch.setattr(settings, "_settings_cache", None)
|
||||||
|
# Read via the same import path the agent loop uses.
|
||||||
|
assert settings.get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX) == 750_000
|
||||||
|
|
||||||
|
# Malformed value falls back to DEFAULT_HARD_MAX (defensive, matches the
|
||||||
|
# try/except in src/agent_loop.py).
|
||||||
|
f.write_text(json.dumps({"agent_input_token_hard_max": "huge"}), encoding="utf-8")
|
||||||
|
monkeypatch.setattr(settings, "_settings_cache", None)
|
||||||
|
raw = settings.get_setting("agent_input_token_hard_max", DEFAULT_HARD_MAX)
|
||||||
|
try:
|
||||||
|
parsed = int(raw)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
parsed = DEFAULT_HARD_MAX
|
||||||
|
if parsed <= 0:
|
||||||
|
parsed = DEFAULT_HARD_MAX
|
||||||
|
assert parsed == DEFAULT_HARD_MAX
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
"""Regression guard for issue #1291 — CPU-only serve still emitted GPU-only flags.
|
||||||
|
|
||||||
|
The llama.cpp serve command builder (static/js/cookbook.js) added
|
||||||
|
`--flash-attn on` and exported `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` from
|
||||||
|
independent toggles, so a CPU-only config (`-ngl 0`, often with flash-attn left
|
||||||
|
on by an Auto profile) produced a command that mixes "zero GPU layers" with
|
||||||
|
CUDA/flash-attn and fails to start. The builder now drops those GPU-only flags
|
||||||
|
when ngl == 0, per the maintainer's guidance.
|
||||||
|
|
||||||
|
cookbook.js pulls in browser globals so it can't run under node; guard the fix
|
||||||
|
at the source level: a `_cpuOnly` gate exists and is applied to flash-attn and
|
||||||
|
the CUDA unified-memory env.
|
||||||
|
"""
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SRC = Path(__file__).resolve().parent.parent / "static/js/cookbook.js"
|
||||||
|
|
||||||
|
|
||||||
|
def test_cpu_only_drops_gpu_only_flags():
|
||||||
|
text = SRC.read_text(encoding="utf-8")
|
||||||
|
# A CPU-only flag derived from ngl == 0.
|
||||||
|
assert re.search(r"_cpuOnly\s*=\s*String\(f\.ngl\)\.trim\(\)\s*===\s*'0'", text), \
|
||||||
|
"expected a _cpuOnly gate derived from ngl==0"
|
||||||
|
# flash-attn must be suppressed for CPU-only.
|
||||||
|
assert re.search(r"if\s*\(\s*f\.flash_attn\s*&&\s*!_cpuOnly\s*\)", text), \
|
||||||
|
"flash-attn must be gated on !_cpuOnly"
|
||||||
|
# The CUDA unified-memory env must be suppressed for CPU-only too.
|
||||||
|
assert "f.unified_mem && !_cpuOnly" in text, \
|
||||||
|
"GGML_CUDA_ENABLE_UNIFIED_MEMORY must be gated on !_cpuOnly"
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""Regression guard for issue #1355 — the Cookbook *download* error toast used
|
||||||
|
the default ~1.2s duration, so an actionable message like "tmux is required …"
|
||||||
|
vanished before it could be read. The serve path already used multi-second
|
||||||
|
durations; the download-failure toasts now match.
|
||||||
|
|
||||||
|
cookbookDownload.js pulls in browser globals so it can't run under node; this
|
||||||
|
guards the durations at the source level.
|
||||||
|
"""
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SRC = Path(__file__).resolve().parent.parent / "static/js/cookbookDownload.js"
|
||||||
|
_MIN_MS = 5000
|
||||||
|
|
||||||
|
|
||||||
|
def test_download_failure_toasts_stay_visible():
|
||||||
|
# Each download-failure toast is a single line; assert each carries an
|
||||||
|
# explicit duration >= _MIN_MS so the actionable error stays readable.
|
||||||
|
lines = [
|
||||||
|
ln for ln in SRC.read_text(encoding="utf-8").splitlines()
|
||||||
|
if "showToast(" in ln and "Download failed:" in ln
|
||||||
|
]
|
||||||
|
assert lines, "expected at least one 'Download failed' showToast call"
|
||||||
|
for ln in lines:
|
||||||
|
m = re.search(r",\s*(\d{3,})\s*\)\s*;?\s*$", ln)
|
||||||
|
assert m, f"download-failure toast has no explicit duration: {ln.strip()}"
|
||||||
|
assert int(m.group(1)) >= _MIN_MS, f"duration too short to read: {ln.strip()}"
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
"""Regression tests for issue #1341 — deep research used the model's
|
||||||
|
training-cutoff year (e.g. "best Python tutorials 2025") because the
|
||||||
|
query-generation and planning prompts never told the LLM the current date.
|
||||||
|
|
||||||
|
The chat/agent path already injects "Today is ..." (src/agent_loop.py); deep
|
||||||
|
research had no equivalent. These tests pin that the current year now reaches
|
||||||
|
the LLM at both the planning and query-generation steps, without needing a live
|
||||||
|
LLM or DB.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from src.deep_research import (
|
||||||
|
DeepResearcher,
|
||||||
|
current_date_context,
|
||||||
|
RESEARCH_PLAN_PROMPT,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _this_year() -> str:
|
||||||
|
return datetime.now().astimezone().strftime("%Y")
|
||||||
|
|
||||||
|
|
||||||
|
def test_current_date_context_names_the_real_year():
|
||||||
|
ctx = current_date_context()
|
||||||
|
assert _this_year() in ctx
|
||||||
|
# It must actively steer the model away from training-data years.
|
||||||
|
assert "training data" in ctx.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_queries_prompt_carries_the_current_year():
|
||||||
|
# Build without the heavy __init__; _generate_queries only needs these.
|
||||||
|
r = DeepResearcher.__new__(DeepResearcher)
|
||||||
|
r.research_plan = ""
|
||||||
|
r.queries_used = set()
|
||||||
|
|
||||||
|
seen = {}
|
||||||
|
|
||||||
|
async def _fake_llm(messages, **kwargs):
|
||||||
|
seen["prompt"] = messages[0]["content"]
|
||||||
|
return '["python tutorials", "python guides"]'
|
||||||
|
|
||||||
|
r._llm = _fake_llm
|
||||||
|
|
||||||
|
queries = asyncio.run(r._generate_queries("best python tutorials", "", 1))
|
||||||
|
|
||||||
|
assert queries # sanity: the JSON array parsed
|
||||||
|
# The fix: the real current year is in the prompt the LLM actually sees.
|
||||||
|
assert _this_year() in seen["prompt"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_plan_prompt_carries_the_current_year():
|
||||||
|
r = DeepResearcher.__new__(DeepResearcher)
|
||||||
|
|
||||||
|
seen = {}
|
||||||
|
|
||||||
|
async def _fake_llm(messages, **kwargs):
|
||||||
|
seen["prompt"] = messages[0]["content"]
|
||||||
|
return "{}"
|
||||||
|
|
||||||
|
r._llm = _fake_llm
|
||||||
|
|
||||||
|
asyncio.run(r._create_plan("what changed this year"))
|
||||||
|
|
||||||
|
assert _this_year() in seen["prompt"]
|
||||||
|
# The base template itself stays year-agnostic; the year comes from the
|
||||||
|
# prepended context, proving the wiring (not a hard-coded prompt edit).
|
||||||
|
assert _this_year() not in RESEARCH_PLAN_PROMPT
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user