Compare commits

...
Author SHA1 Message Date
red person ef3467e8d4 fix(tasks): render daily brief calendars in local time (#6926) 2026-10-31 13:54:14 +00:00
red person 72e8520fa3 fix(models): honor visible pinned IDs in route selection (#6922) 2026-10-31 13:51:59 +00:00
red person 9276db9c11 fix(models): skip non-chat models when resolving chat defaults (#6907) 2026-10-31 13:49:26 +00:00
GrzegorzandClaude Opus 5.5 a797cee0b3 fix(cookbook): diagnose vLLM serves on an NVIDIA driver that is too old (#6727) (#6728)
- Add a diagnosis for "NVIDIA driver on your system is too old" and
  "CUDA driver version is insufficient for CUDA runtime version", the
  errors PyPI's CUDA 13 torch/vLLM builds hit on drivers older than 580.
- Remove the nested _diagnose_serve_output copy in cookbook_routes.py.
  It shadowed the helper and had drifted, so /api/cookbook/tasks/status
  never used the helper's newer diagnoses.
- Capture the tmux pane with -J in the status poll so lines wrapped at
  the pane width do not split error text mid-word.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 13:47:30 +00:00
Grzegorz 76061498c8 fix(gallery): validate album rename and image_ids input (#6822) (#6823) 2026-10-31 13:45:02 +00:00
Merajul Arefin 04f8546d9d fix(search): gallery library per-word search with LIKE escaping (#6584)
Gallery search built a single `%{search}%` LIKE, so multi-word queries only matched the exact adjacent phrase ("red car" missed "a car that is red") and literal % / _ acted as SQL wildcards. Split on whitespace and require each token to match (prompt OR tags OR ai_tags), escaping LIKE wildcards per token; same escaping for the tag filter. Promote _escape_like to public escape_like_pattern() shared helper.

Part of #2123.

Tests: tests/test_gallery_search_multiword.py (5 tests); 4 fail pre-fix, 5 pass post-fix. Neighbors (session_search, gallery null-user, shims): 25 passed.
2026-10-31 13:43:20 +00:00
Robert Vind-Gardoș 9835c1a7f9 fix(skills): refuse renaming a skill onto a name the owner already uses (#6941)
update_skill only checked whether the target directory existed. A skill
with that name in another category lives elsewhere on disk, so the
rename went through and left two skills with one name. Update and
delete look skills up by name, so the next delete removed whichever
file os.walk reached first.

Check the owner's skills in every category before renaming, and answer
409 from PUT /api/skills/{id} instead of the misleading 404.
2026-10-31 13:34:05 +00:00
Isharakandisharak7m 239f09cbc4 fix(docker): retry PyPI fetches in realesrgan wheel builder (#6407)
Replace urllib-based PyPI metadata retrieval with curl --retry 8 to
handle transient TLS/SSL EOF errors that abort the Docker image build.
The sdist download also gets retry logic. No functional change to the
patching or wheel-building steps.

Co-authored-by: isharak7m <192635824+isharak7m@users.noreply.github.com>
2026-10-31 13:32:28 +00:00
GrzegorzandClaude Opus 5.5 a1a7f8fc9a fix(agent): don't clamp an unknown context window to the small-server budget (#6678) (#6706)
budget_context_for_model() returns 0 when the endpoint does not prove its
context window, and compute_input_token_budget() answers 0 with the
conservative default budget (6000). The agent loop's small-local-server
clamp (`candidate_context <= 8192`) also matched 0 and cut the request to
max(1200, 0) tokens, so on any endpoint with an unreported window an open
document or a few thousand tokens of earlier context were trimmed away
before the model saw them ("5519 tokens > 1200 budget (ctx=1800)").

Apply the clamp only to a known window.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 13:30:29 +00:00
Prince 87193d23f5 fix(agent-loop): release buffered prose when '<' is not a tool tag (#6757) (#6758) 2026-10-31 13:26:43 +00:00
VykosandClaude 516f046fef fix(chat): label the vision description injected for text-only models (#6223)
When the chat model cannot see images, the separate vision call's description
is appended to the user's message. Nothing marked it as machine-derived, so the
chat model read it as the user's own prose and answered the description instead
of the question.

The label goes below the `[Image: <name>]` marker line because the client
strips the block by matching that exact prefix; renaming the marker would leave
the raw description rendered in the user's bubble.

Fixes #5573

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-31 13:24:32 +00:00
red person 48c3a7fc36 fix(pdf): convert CMYK images before vision analysis (#6917) 2026-10-31 13:22:55 +00:00
red person 70fc58225c fix(documents): treat library search punctuation literally (#6944) 2026-10-31 13:21:10 +00:00
red person 9efedc8b07 fix(documents): retain language extensions in ZIP exports (#6848) 2026-10-31 13:18:58 +00:00
gloss-stack 1d251ef867 Fix transcript search snippet positioning (#6206) 2026-10-31 13:05:35 +00:00
Nguyen Thanh Dat bf5bf57bc1 fix(diagnostics): report a failed config source instead of "disabled" (#6154) (#6169)
`_gather_inputs()` fails soft: a source that raises leaves its inventory as
`[]`. Every probe reads an empty inventory as "not configured" and answers
`disabled`, so a broken integrations file, email store, or endpoint database is
indistinguishable from a subsystem the operator deliberately turned off. With
the remaining subsystems disabled, `GET /api/diagnostics/services` then reports
`overall: "ok"` while three of its inputs failed to load — the exceptions appear
only in DEBUG logs.

Record each failure alongside the empty value (`failed`: source -> controlled
category) and report the affected subsystem through `_source_unavailable()`
rather than probing it with an inventory that is empty for the wrong reason.

- `degraded`, not `disabled`: the configured state is unknown, not known-off.
  Not `down` either — nothing reached the subsystem and failed; the inventory
  could not be read. Severity is non-zero either way, so a report containing a
  failed source can no longer roll up to `ok`.
- A source that loads and is genuinely empty still reports `disabled`.
- One broken source degrades one entry; the others are still probed.
- `meta` carries the source name and the controlled `_classify_error()`
  category only, never exception text, credentials, or database detail.
- `inputs.get("failed")` so an inputs payload without the key still probes
  normally.

Verified with the reporter's harness: settings loading with
`search_provider: "disabled"` while the other three sources raise now yields
`overall: "degraded"` with `searxng` still `disabled`; before, all five read
`disabled` and `overall` was `ok`.
2026-10-31 13:04:18 +00:00
MBGAandBolt4243 2019a97339 fix(skills): preserve escaped quotes in frontmatter lists (#6265)
* fix(skills): preserve escaped quotes in frontmatter lists

* test(skills): use descriptive frontmatter examples

---------

Co-authored-by: Bolt4243 <277300782+Bolt4243@users.noreply.github.com>
2026-10-31 13:02:02 +00:00
Aashish Mahato 9a1fdb192e fix(cookbook): resolve NameError in hf-gguf-files exception handler (#6341) (#6369)
In routes/cookbook_routes.py, the exception handler for GET /api/cookbook/hf-gguf-files referenced repo instead of the parameter repo_id, causing an unhandled NameError and 500 status when upstream requests to Hugging Face failed.

Fixes #6341.
2026-10-31 12:59:17 +00:00
35 changed files with 1841 additions and 264 deletions
+33 -11
View File
@@ -31,21 +31,43 @@ for spec in $SPECS; do
name="${spec%%==*}" name="${spec%%==*}"
ver="${spec##*==}" ver="${spec##*==}"
# pip download builds metadata (and trips the same bug), so fetch the raw # pip download builds metadata (and trips the same bug), so fetch the raw
# sdist URL from the PyPI JSON API instead. # sdist URL from the PyPI JSON API instead. Use curl with retries to
url="$(python - "$name" "$ver" <<'PY' # survive transient TLS / network failures that urllib cannot recover from.
import json, sys, urllib.request metadata="${name}-${ver}.json"
name, ver = sys.argv[1], sys.argv[2] curl --fail --silent --show-error --location \
data = json.load(urllib.request.urlopen(f"https://pypi.org/pypi/{name}/{ver}/json")) --retry 8 \
for f in data["urls"]: --retry-delay 3 \
if f["packagetype"] == "sdist": --retry-max-time 120 \
print(f["url"]); break --retry-all-errors \
"https://pypi.org/pypi/${name}/${ver}/json" \
-o "$metadata"
url="$(python - "$metadata" <<'PY'
import json, sys
with open(sys.argv[1], encoding="utf-8") as f:
data = json.load(f)
for item in data["urls"]:
if item["packagetype"] == "sdist":
print(item["url"])
break
else: else:
sys.exit(f"no sdist found for {name}=={ver}") raise SystemExit("no sdist found")
PY PY
)" )"
rm -f "$metadata"
echo ">> fetching ${name} ${ver}: ${url}" echo ">> fetching ${name} ${ver}: ${url}"
curl -fsSL "$url" -o "${name}.tar.gz" curl --fail --silent --show-error --location \
--retry 8 \
--retry-delay 3 \
--retry-max-time 120 \
--retry-all-errors \
"$url" \
-o "${name}.tar.gz"
tar xzf "${name}.tar.gz" tar xzf "${name}.tar.gz"
rm -f "${name}.tar.gz"
done done
echo ">> patching get_version()" echo ">> patching get_version()"
+4 -1
View File
@@ -1901,7 +1901,7 @@ def _ts_or_zero(value) -> float:
def _recover_empty_session_model(sess, session_id: str, owner: str | None = None) -> bool: def _recover_empty_session_model(sess, session_id: str, owner: str | None = None) -> bool:
"""Re-populate sess.model from the matching endpoint's cached models. """Recover sess.model from the endpoint's visible cached and pinned models.
Covers the window between endpoint setup and the first chat send: the Covers the window between endpoint setup and the first chat send: the
picker showed a model in the dropdown but the session record never got picker showed a model in the dropdown but the session record never got
@@ -1965,6 +1965,9 @@ def _recover_empty_session_model(sess, session_id: str, owner: str | None = None
visible = _visible_models(cached, getattr(ep, "hidden_models", None)) visible = _visible_models(cached, getattr(ep, "hidden_models", None))
except Exception: except Exception:
visible = cached visible = cached
if not is_chatgpt_subscription:
from src.endpoint_resolver import _endpoint_enabled_models
visible = _endpoint_enabled_models(ep)
if current_model and current_model in {str(item).strip() for item in visible}: if current_model and current_model in {str(item).strip() for item in visible}:
return False return False
if is_chatgpt_subscription: if is_chatgpt_subscription:
+14
View File
@@ -1491,6 +1491,20 @@ def _diagnose_serve_output(text: str) -> dict | None:
"Port is already in use.", "Port is already in use.",
[{"label": "retry on port 8001", "op": "replace", "flag": "--port", "value": "8001"}], [{"label": "retry on port 8001", "op": "replace", "flag": "--port", "value": "8001"}],
), ),
(
# PyPI torch/vLLM wheels now target CUDA 13, which needs driver 580+.
# torch: "The NVIDIA driver on your system is too old (found version 12080)";
# vLLM kernels: "CUDA driver version is insufficient for CUDA runtime version".
r"NVIDIA driver on your system is too old|CUDA driver version is insufficient",
"The installed PyTorch/vLLM build needs a newer NVIDIA driver than this server has.",
[
{
"label": "install builds for the driver's CUDA version, e.g. vLLM's +cu129 wheel from its GitHub release with uv pip install … --torch-backend cu129",
"op": "manual",
},
{"label": "or update the NVIDIA driver (CUDA 13 builds need driver 580 or newer)", "op": "manual"},
],
),
( (
r"No CUDA GPUs are available|no GPU.*found|CUDA_VISIBLE_DEVICES.*invalid", r"No CUDA GPUs are available|no GPU.*found|CUDA_VISIBLE_DEVICES.*invalid",
"No GPUs are visible to the serve process.", "No GPUs are visible to the serve process.",
+5 -172
View File
@@ -468,175 +468,6 @@ def setup_cookbook_routes() -> APIRouter:
task["payload"].pop("hf_token", None) task["payload"].pop("hf_token", None)
return state return state
def _diagnose_serve_output(text: str) -> dict | None:
"""Server-side mirror of the Cookbook UI's common serve diagnoses.
The browser uses cookbook-diagnosis.js for clickable fixes. This gives
the agent/tool path the same structured signal so it can retry with an
adjusted command instead of guessing from raw tmux output.
"""
if not text:
return None
tail = text[-6000:]
patterns = [
(
r"No available memory for the cache blocks|Available KV cache memory:.*-",
"No GPU memory left for KV cache after loading model.",
[
{"label": "retry with GPU memory utilization 0.95", "op": "replace", "flag": "--gpu-memory-utilization", "value": "0.95"},
{"label": "retry with context 2048", "op": "replace", "flag": "--max-model-len", "value": "2048"},
],
),
(
r"CUDA out of memory|torch\.cuda\.OutOfMemoryError|CUDA error: out of memory|warming up sampler|max_num_seqs.*gpu_memory_utilization",
"GPU ran out of memory during startup or warmup.",
[
{"label": "retry with context 4096", "op": "replace", "flag": "--max-model-len", "value": "4096"},
{"label": "retry with GPU memory utilization 0.80", "op": "replace", "flag": "--gpu-memory-utilization", "value": "0.80"},
{"label": "retry with --enforce-eager", "op": "append", "arg": "--enforce-eager"},
],
),
(
r"not divisib|must be divisible|attention heads.*divisible",
"Tensor parallel size is incompatible with the model.",
[
{"label": "retry with tensor parallel size 1", "op": "replace", "flag": "--tensor-parallel-size", "value": "1"},
{"label": "retry with tensor parallel size 2", "op": "replace", "flag": "--tensor-parallel-size", "value": "2"},
],
),
(
r"KV cache.*too (small|large)|max_model_len.*exceeds|maximum.*context",
"Context length is too large for available GPU memory.",
[
{"label": "retry with context 8192", "op": "replace", "flag": "--max-model-len", "value": "8192"},
{"label": "retry with context 4096", "op": "replace", "flag": "--max-model-len", "value": "4096"},
],
),
(
r"enable-auto-tool-choice requires --tool-call-parser",
"Auto tool choice requires an explicit tool call parser.",
[{"label": "retry with Hermes tool parser", "op": "append", "arg": "--tool-call-parser hermes"}],
),
(
r"Please pass.*trust.remote.code=True|contains custom code which must be executed to correctly load|does not recognize this architecture|model type.*but Transformers does not",
"Model requires custom code or newer model support.",
[{"label": "retry with --trust-remote-code", "op": "append", "arg": "--trust-remote-code"}],
),
(
r"Either a revision or a version must be specified|transformers\.integrations\.hub_kernels|kernels/layer",
"vLLM/Transformers kernel package mismatch.",
[{"label": "update vLLM, Transformers, and kernels on this server", "op": "dependency", "package": "vllm transformers kernels"}],
),
(
r"Address already in use|bind.*address.*in use",
"Port is already in use.",
[{"label": "retry on port 8001", "op": "replace", "flag": "--port", "value": "8001"}],
),
(
r"No CUDA GPUs are available|no GPU.*found|CUDA_VISIBLE_DEVICES.*invalid",
"No GPUs are visible to the serve process.",
[{"label": "clear Cookbook GPU selection or choose available GPUs", "op": "settings", "field": "gpus", "value": ""}],
),
(
r"Failed to infer device type|NVML Shared Library Not Found|No module named 'amdsmi'|platform is not available",
"vLLM could not find a supported GPU (CUDA or ROCm). "
"This machine may have integrated or unsupported graphics only.",
[
{"label": "switch to llama.cpp (CPU/Metal, works without a discrete GPU)", "op": "manual"},
{"label": "switch to Ollama (CPU/Metal, works without a discrete GPU)", "op": "manual"},
],
),
(
r"vllm.*command not found|No module named vllm|ERROR: vLLM is not installed",
"vLLM is not installed or not in PATH on this server.",
[{"label": "install vLLM in Cookbook Dependencies", "op": "dependency", "package": "vllm"}],
),
(
r"sgl_kernel[\s\S]*(Python\.h|libnuma\.so\.1|common_ops|libnvrtc\.so)|"
r"(Python\.h|libnuma\.so\.1|common_ops|libnvrtc\.so)[\s\S]*sgl_kernel|"
r"Could not load any common_ops library|"
r"Please ensure sgl_kernel is properly installed",
"SGLang native kernel/runtime is missing or mismatched on this server.",
[
{"label": "repair sglang-kernel in this Python environment", "op": "dependency", "package": "sglang-kernel"},
{"label": "if libnvrtc is still missing, install the matching CUDA/NVRTC runtime on this host", "op": "manual"},
],
),
(
r"sglang.*command not found|No module named sglang|SGLang is not installed",
"SGLang is not installed or not in PATH on this server.",
[{"label": "install SGLang in Cookbook Dependencies", "op": "dependency", "package": "sglang[all]"}],
),
(
r"No module named ['\"]?mlx_lm|mlx_lm.*command not found|MLX is not installed|MLX LM is not installed",
"MLX LM is not installed on this server.",
[{"label": "install mlx-lm in Cookbook Dependencies", "op": "dependency", "package": "mlx-lm"}],
),
(
r"OmniGen2Pipeline|module diffusers has no attribute .*Pipeline|custom_pipeline=.*failed",
"This image model uses a custom Diffusers pipeline that the launch environment does not know yet.",
[{"label": "update Diffusers image dependencies", "op": "dependency", "package": "diffusers transformers accelerate"}],
),
(
r"Unable to quantize model of type <class ['\"]mlx_lm\.models\.switch_layers\.QuantizedSwitchLinear['\"]>|QuantizedSwitchLinear",
"MLX-LM tried to quantize an already-quantized DeepSeek switch layer.",
[
{"label": "relaunch from the cached local Hugging Face snapshot path on this Mac", "op": "manual"},
{"label": "Odysseus now rewrites MLX repo-id launches to a cached snapshot when one exists", "op": "manual"},
],
),
# System build deps come BEFORE the generic llama.cpp catch-all
# so cmake / build-essential / git missing → a specific OS-package
# remediation instead of "install llama-cpp-python[server]" (which
# itself fails to compile when cmake is absent).
(
r"cmake: command not found|cmake.*not found.*[Cc]ould not",
"cmake is required to build llama.cpp from source but isn't installed on this server.",
[{"label": "install build deps for llama.cpp (apt: cmake build-essential git / pacman: cmake base-devel git / dnf: cmake gcc-c++ make git / brew: cmake git)", "op": "dependency", "package": "llama-cpp-python[server]"}],
),
(
r"^(make|g\+\+|gcc): command not found|Could not find C\+\+ compiler",
"A C/C++ compiler (build-essential) is required to build llama.cpp from source.",
[{"label": "install build deps for llama.cpp on this server", "op": "dependency", "package": "llama-cpp-python[server]"}],
),
(
r"^git: command not found",
"git is required to clone the llama.cpp source tree.",
[{"label": "install build deps for llama.cpp on this server", "op": "dependency", "package": "llama-cpp-python[server]"}],
),
(
r"llama-server.*command not found|llama\.cpp.*not found|No module named.*llama_cpp|No module named 'starlette_context'",
"llama.cpp / llama-cpp-python dependencies are missing.",
[{"label": "install llama.cpp dependencies or llama-cpp-python[server]", "op": "dependency", "package": "llama-cpp-python[server]"}],
),
(
r"No GGUF found on this host|no \.gguf file|No GGUF file found",
"No GGUF file found for this model on this host. The llama.cpp backend needs a .gguf file.",
[{"label": "download a GGUF build of this model (repo name usually ends in -GGUF, file like Q4_K_M.gguf)", "op": "manual"}],
),
(
r"No module named 'torch'|No module named torch|No module named 'torchvision'|No module named torchvision|No module named 'diffusers'|No module named diffusers|No module named 'scipy'|No module named scipy|install scipy if you want to use beta sigmas|requires the Torchvision library",
"Diffusion serving requires PyTorch, Torchvision, Diffusers, Accelerate, and SciPy.",
[{"label": "install Diffusers image deps in Cookbook Dependencies", "op": "dependency", "package": "diffusers[torch] torchvision accelerate scipy python-multipart"}],
),
(
r"403 Forbidden|401 Unauthorized|Access to model.*is restricted|gated repo|not in the authorized list|awaiting a review",
"Model access is gated or unauthorized.",
[{"label": "set HF token and request model access on HuggingFace", "op": "manual"}],
),
]
for pattern, message, suggestions in patterns:
if re.search(pattern, tail, re.I):
return {"message": message, "suggestions": suggestions}
if re.search(r"Traceback \(most recent call last\)", tail, re.I) and not re.search(
r"Application startup complete|GET /v1/|Uvicorn running on", tail, re.I
):
return {
"message": "Python traceback detected during serve startup.",
"suggestions": [{"label": "inspect traceback and retry with adjusted backend/settings", "op": "manual"}],
}
return None
def _state_for_client(state): def _state_for_client(state):
"""Return cookbook state without raw secrets for browser clients.""" """Return cookbook state without raw secrets for browser clients."""
_strip_task_secrets(state) _strip_task_secrets(state)
@@ -3930,7 +3761,7 @@ def setup_cookbook_routes() -> APIRouter:
return {"ok": False, "files": [], "error": f"HF API HTTP {resp.status_code}"} return {"ok": False, "files": [], "error": f"HF API HTTP {resp.status_code}"}
data = resp.json() data = resp.json()
except Exception: except Exception:
logger.exception("HF GGUF file scan failed for %s", repo) logger.exception("HF GGUF file scan failed for %s", repo_id)
return {"ok": False, "files": [], "error": "HF API request failed"} return {"ok": False, "files": [], "error": "HF API request failed"}
files = [ files = [
str(s.get("rfilename") or "") str(s.get("rfilename") or "")
@@ -4446,7 +4277,9 @@ def setup_cookbook_routes() -> APIRouter:
# fills the visible tail. Without this, output_tail ends up # fills the visible tail. Without this, output_tail ends up
# as just "Locale: C / Ubuntu_Odysseus ❯" and the agent # as just "Locale: C / Ubuntu_Odysseus ❯" and the agent
# can't diagnose the actual error. # can't diagnose the actual error.
capture_cmd = ssh_base + [remote, _remote_tmux_command("capture-pane", "-t", session_id, "-p", "-S", "-500")] # -J joins lines tmux wrapped at the pane width, so an error
# split mid-word still matches the serve diagnosis patterns.
capture_cmd = ssh_base + [remote, _remote_tmux_command("capture-pane", "-t", session_id, "-p", "-J", "-S", "-500")]
elif IS_WINDOWS: elif IS_WINDOWS:
# LOCAL Windows task: launched as a detached process (no tmux). # LOCAL Windows task: launched as a detached process (no tmux).
# Liveness comes from the <session>.pid file, output from the # Liveness comes from the <session>.pid file, output from the
@@ -4455,7 +4288,7 @@ def setup_cookbook_routes() -> APIRouter:
capture_cmd = None capture_cmd = None
else: else:
check_cmd = ["tmux", "has-session", "-t", session_id] check_cmd = ["tmux", "has-session", "-t", session_id]
capture_cmd = ["tmux", "capture-pane", "-t", session_id, "-p", "-S", "-500"] capture_cmd = ["tmux", "capture-pane", "-t", session_id, "-p", "-J", "-S", "-500"]
local_win_task = (not remote) and IS_WINDOWS local_win_task = (not remote) and IS_WINDOWS
+9 -4
View File
@@ -16,6 +16,7 @@ from core.database import SessionLocal, Document, DocumentVersion
from core.database import Session as DbSession from core.database import Session as DbSession
from src.auth_helpers import get_current_user, _auth_disabled from src.auth_helpers import get_current_user, _auth_disabled
from src.constants import MAIL_ATTACHMENTS_DIR from src.constants import MAIL_ATTACHMENTS_DIR
from src.session_search import _escape_like
from src.upload_handler import reserve_upload_references from src.upload_handler import reserve_upload_references
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -571,9 +572,10 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
# match docs containing both words regardless of position/order. # match docs containing both words regardless of position/order.
if search: if search:
for tok in search.split(): for tok in search.split():
term = f"%{tok}%" term = f"%{_escape_like(tok)}%"
q = q.filter( q = q.filter(
Document.title.ilike(term) | Document.current_content.ilike(term) Document.title.ilike(term, escape="\\")
| Document.current_content.ilike(term, escape="\\")
) )
# Language filter. "pdf" is a display language derived from the # Language filter. "pdf" is a display language derived from the
@@ -812,10 +814,13 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
ext = _ext.get(doc.language or "text", ".txt") ext = _ext.get(doc.language or "text", ".txt")
base = (doc.title or "document").strip() or "document" base = (doc.title or "document").strip() or "document"
base = re.sub(r"[^\w\-. ]+", "", base)[:60].strip() or doc.id base = re.sub(r"[^\w\-. ]+", "", base)[:60].strip() or doc.id
name = base if "." in base else base + ext has_ext = base.lower().endswith(ext)
stem = base[:-len(ext)] if has_ext else base
suffix = base[-len(ext):] if has_ext else ext
name = stem + suffix
i = 1 i = 1
while name in used: while name in used:
name = f"{base}-{i}" + ("" if "." in base else ext) name = f"{stem}-{i}{suffix}"
i += 1 i += 1
used.add(name) used.add(name)
content = doc.current_content or "" content = doc.current_content or ""
+44 -16
View File
@@ -18,6 +18,7 @@ from fastapi import APIRouter, HTTPException, Query, Request
from core.database import SessionLocal, GalleryImage, GalleryAlbum, ModelEndpoint from core.database import SessionLocal, GalleryImage, GalleryAlbum, ModelEndpoint
from core.database import Session as DbSession from core.database import Session as DbSession
from src.auth_helpers import get_current_user, owner_filter, require_privilege from src.auth_helpers import get_current_user, owner_filter, require_privilege
from src.session_search import escape_like_pattern
from src.upload_limits import ( from src.upload_limits import (
read_upload_limited, read_upload_limited,
GALLERY_UPLOAD_MAX_BYTES, GALLERY_UPLOAD_MAX_BYTES,
@@ -305,6 +306,23 @@ def _b64_bytes(value: str, *, label: str = "image") -> bytes:
raise HTTPException(400, f"Invalid {label}") from exc raise HTTPException(400, f"Invalid {label}") from exc
async def _json_object(request: Request) -> dict:
try:
data = await request.json()
except ValueError:
raise HTTPException(400, "Invalid JSON body")
if not isinstance(data, dict):
raise HTTPException(400, "JSON object body required")
return data
async def _image_ids(request: Request) -> list:
ids = (await _json_object(request)).get("image_ids", [])
if not isinstance(ids, list) or not all(isinstance(i, str) for i in ids):
raise HTTPException(400, "image_ids must be a list of image ids")
return ids
def _pil_image_to_b64(img, *, fmt: str = "PNG") -> str: def _pil_image_to_b64(img, *, fmt: str = "PNG") -> str:
buf = io.BytesIO() buf = io.BytesIO()
img.save(buf, format=fmt) img.save(buf, format=fmt)
@@ -1184,14 +1202,17 @@ def setup_gallery_routes() -> APIRouter:
) )
q = _owner_filter(q, user) q = _owner_filter(q, user)
# Search filter (prompt + tags + ai_tags) # Per-term AND search: a single `%foo bar%` LIKE only matched the
# exact phrase, so "red car" missed "a car that is red".
# LIKE wildcards are escaped per token.
if search: if search:
term = f"%{search}%"
from sqlalchemy import or_ from sqlalchemy import or_
for tok in search.split():
term = f"%{escape_like_pattern(tok)}%"
q = q.filter(or_( q = q.filter(or_(
GalleryImage.prompt.ilike(term), GalleryImage.prompt.ilike(term, escape="\\"),
GalleryImage.tags.ilike(term), GalleryImage.tags.ilike(term, escape="\\"),
GalleryImage.ai_tags.ilike(term), GalleryImage.ai_tags.ilike(term, escape="\\"),
)) ))
# Tag filter. The UI stacks multiple tag pills by passing them # Tag filter. The UI stacks multiple tag pills by passing them
@@ -1203,9 +1224,10 @@ def setup_gallery_routes() -> APIRouter:
for one in (t.strip() for t in tag.split(",")): for one in (t.strip() for t in tag.split(",")):
if not one: if not one:
continue continue
safe_tag = f"%{escape_like_pattern(one)}%"
q = q.filter(_or( q = q.filter(_or(
GalleryImage.tags.ilike(f"%{one}%"), GalleryImage.tags.ilike(safe_tag, escape="\\"),
GalleryImage.ai_tags.ilike(f"%{one}%"), GalleryImage.ai_tags.ilike(safe_tag, escape="\\"),
)) ))
# Model filter # Model filter
@@ -2939,14 +2961,22 @@ def setup_gallery_routes() -> APIRouter:
@router.put("/api/gallery/albums/{album_id}") @router.put("/api/gallery/albums/{album_id}")
async def update_album(request: Request, album_id: str): async def update_album(request: Request, album_id: str):
user = get_current_user(request) user = get_current_user(request)
data = await request.json() data = await _json_object(request)
name = data.get("name")
if name is not None:
if not isinstance(name, str) or not name.strip():
raise HTTPException(400, "Album name required")
name = name.strip()
description = data.get("description")
if description is not None and not isinstance(description, str):
raise HTTPException(400, "Album description must be a string")
db = SessionLocal() db = SessionLocal()
try: try:
album = _get_or_404_album(db, album_id, user) album = _get_or_404_album(db, album_id, user)
if data.get("name") is not None: if name is not None:
album.name = data["name"] album.name = name
if data.get("description") is not None: if description is not None:
album.description = data["description"] album.description = description
if data.get("cover_id") is not None: if data.get("cover_id") is not None:
cover_id = data["cover_id"] or None cover_id = data["cover_id"] or None
if cover_id: if cover_id:
@@ -2976,8 +3006,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/albums/{album_id}/add") @router.post("/api/gallery/albums/{album_id}/add")
async def add_to_album(request: Request, album_id: str): async def add_to_album(request: Request, album_id: str):
user = get_current_user(request) user = get_current_user(request)
data = await request.json() ids = await _image_ids(request)
ids = data.get("image_ids", [])
db = SessionLocal() db = SessionLocal()
try: try:
_get_or_404_album(db, album_id, user) _get_or_404_album(db, album_id, user)
@@ -2994,8 +3023,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/albums/{album_id}/remove") @router.post("/api/gallery/albums/{album_id}/remove")
async def remove_from_album(request: Request, album_id: str): async def remove_from_album(request: Request, album_id: str):
user = get_current_user(request) user = get_current_user(request)
data = await request.json() ids = await _image_ids(request)
ids = data.get("image_ids", [])
db = SessionLocal() db = SessionLocal()
try: try:
_get_or_404_album(db, album_id, user) _get_or_404_album(db, album_id, user)
+4 -3
View File
@@ -659,7 +659,7 @@ _NON_CHAT_CONTAINS = (
"gliner", "riva", "-parse", "-embedqa", "-nemoretriever", "gliner", "riva", "-parse", "-embedqa", "-nemoretriever",
"topic-control", "calibration", "topic-control", "calibration",
"ai-synthetic-video", "cosmos-reason2", "ai-synthetic-video", "cosmos-reason2",
"bge", "llama-guard", "bge", "llama-guard", "-embed", "rerank",
) )
_NON_CHAT_EXACT_PREFIXES = ( _NON_CHAT_EXACT_PREFIXES = (
"gpt-audio", # gpt-audio, gpt-audio-mini etc. (not gpt-4o-audio-preview which is chat) "gpt-audio", # gpt-audio, gpt-audio-mini etc. (not gpt-4o-audio-preview which is chat)
@@ -2846,11 +2846,12 @@ def setup_model_routes(model_discovery):
return {"endpoint_id": "", "endpoint_url": "", "model": ""} return {"endpoint_id": "", "endpoint_url": "", "model": ""}
base = _normalize_base(ep.base_url) base = _normalize_base(ep.base_url)
chat_url = build_chat_url(base) chat_url = build_chat_url(base)
if model and not _is_chat_model(model):
model = ""
if not model and (getattr(ep, "cached_models", None) or getattr(ep, "pinned_models", None)): if not model and (getattr(ep, "cached_models", None) or getattr(ep, "pinned_models", None)):
try: try:
visible = _visible_models(ep.cached_models, getattr(ep, "hidden_models", None), getattr(ep, "pinned_models", None)) visible = _visible_models(ep.cached_models, getattr(ep, "hidden_models", None), getattr(ep, "pinned_models", None))
if visible: model = next((mid for mid in visible if _is_chat_model(mid)), "")
model = visible[0]
except Exception: except Exception:
pass pass
return {"endpoint_id": ep.id, "endpoint_url": chat_url, "model": model} return {"endpoint_id": ep.id, "endpoint_url": chat_url, "model": model}
+5 -14
View File
@@ -183,6 +183,7 @@ def _resolve_endpoint_runtime(ep, owner=None, model: Optional[str] = None):
OAuth tokens in ProviderAuthSession, so ep.api_key is intentionally empty. OAuth tokens in ProviderAuthSession, so ep.api_key is intentionally empty.
""" """
from src.endpoint_resolver import ( from src.endpoint_resolver import (
_endpoint_enabled_models,
build_chat_url, build_chat_url,
build_headers, build_headers,
resolve_endpoint_runtime as resolve_model_endpoint_runtime, resolve_endpoint_runtime as resolve_model_endpoint_runtime,
@@ -197,7 +198,7 @@ def _resolve_endpoint_runtime(ep, owner=None, model: Optional[str] = None):
ep_model = (model or "").strip() ep_model = (model or "").strip()
if not ep_model: if not ep_model:
try: try:
models = json.loads(ep.cached_models) if ep.cached_models else [] models = _endpoint_enabled_models(ep)
if models: if models:
ep_model = _first_chat_model(models) ep_model = _first_chat_model(models)
except Exception: except Exception:
@@ -824,23 +825,13 @@ def setup_research_routes(research_handler, session_manager=None) -> APIRouter:
if not ep_url or not ep_model: if not ep_url or not ep_model:
# Last resort: this user's enabled endpoint, plus legacy shared rows. # Last resort: this user's enabled endpoint, plus legacy shared rows.
from src.database import SessionLocal from src.database import SessionLocal
from src.endpoint_resolver import normalize_base, build_chat_url, build_headers
db = SessionLocal() db = SessionLocal()
try: try:
ep = _owned_enabled_endpoint(db, user) ep = _owned_enabled_endpoint(db, user)
if ep: if ep:
base = normalize_base(ep.base_url) resolved = _resolve_endpoint_runtime(ep, owner=user)
fallback_url = build_chat_url(base) if resolved:
fallback_headers = build_headers(ep.api_key, base) _merge(*resolved)
fallback_model = ""
if ep.cached_models:
try:
models = json.loads(ep.cached_models)
if models:
fallback_model = _first_chat_model(models)
except Exception:
pass
_merge(fallback_url, fallback_model, fallback_headers)
finally: finally:
db.close() db.close()
+5
View File
@@ -15,6 +15,7 @@ import httpx
from fastapi import APIRouter, HTTPException, Request from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from services.memory.skill_format import slugify
from services.memory.skills import SkillsManager from services.memory.skills import SkillsManager
from src.auth_helpers import get_current_user from src.auth_helpers import get_current_user
from services.skills import evaluation as _evaluation from services.skills import evaluation as _evaluation
@@ -836,6 +837,10 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
updates = body.dict(exclude_none=True) updates = body.dict(exclude_none=True)
if not updates: if not updates:
return {"ok": True} return {"ok": True}
if "name" in updates:
new_name = slugify(updates["name"] or match.get("name"))
if new_name != match.get("name") and skills_manager.has_skill(new_name, owner=user):
raise HTTPException(409, f"A skill named '{new_name}' already exists")
ok = skills_manager.update_skill(match.get("name"), updates, owner=user) ok = skills_manager.update_skill(match.get("name"), updates, owner=user)
if not ok: if not ok:
raise HTTPException(404, "Skill not found") raise HTTPException(404, "Skill not found")
+8 -8
View File
@@ -289,15 +289,15 @@ def _resolve_run_endpoint(db, task: ScheduledTask, run: TaskRun) -> str:
try: try:
from core.database import ModelEndpoint from core.database import ModelEndpoint
eps = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True).all() from src.auth_helpers import owner_filter
from src.endpoint_resolver import _endpoint_enabled_models
eps = owner_filter(
db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True),
ModelEndpoint, getattr(task, "owner", None),
).all()
for ep in eps: for ep in eps:
cached = [] if model in _endpoint_enabled_models(ep):
if ep.cached_models:
try:
cached = json.loads(ep.cached_models) or []
except Exception:
cached = []
if model in cached:
return ep.base_url or "" return ep.base_url or ""
except Exception: except Exception:
pass pass
+6 -1
View File
@@ -130,10 +130,15 @@ def _parse_scalar(raw: str, *, string_only: bool = False) -> Any:
def _split_top_level(s: str, sep: str) -> List[str]: def _split_top_level(s: str, sep: str) -> List[str]:
"""Split `s` on `sep` ignoring separators inside [] or quotes.""" """Split `s` on `sep` ignoring separators inside [] or quotes."""
out, buf, depth, quote = [], [], 0, None out, buf, depth, quote = [], [], 0, None
escaped = False
for ch in s: for ch in s:
if quote: if quote:
buf.append(ch) buf.append(ch)
if ch == quote: if escaped:
escaped = False
elif ch == "\\" and quote == '"':
escaped = True
elif ch == quote:
quote = None quote = None
continue continue
if ch in ("'", '"'): if ch in ("'", '"'):
+16
View File
@@ -500,6 +500,19 @@ class SkillsManager:
sk.path = self._skill_file(cat, nm) sk.path = self._skill_file(cat, nm)
return sk.to_dict() return sk.to_dict()
def has_skill(self, name: str, owner: Optional[str] = None) -> bool:
"""Whether `owner` already has a skill called `name`, in any category.
Update and delete find a skill by name within its owner, so two of
one owner's skills sharing a name would make them act on whichever
file is walked first.
"""
for path in self._iter_skill_files():
sk = self._read_skill(path)
if sk and sk.name == name and (sk.owner or "") == (owner or ""):
return True
return False
def update_skill(self, skill_id: str, updates: Dict, owner: Optional[str] = None) -> bool: def update_skill(self, skill_id: str, updates: Dict, owner: Optional[str] = None) -> bool:
"""`skill_id` is the slug name. Allows updating any field plus """`skill_id` is the slug name. Allows updating any field plus
renames if `name` changes (file is moved on disk). renames if `name` changes (file is moved on disk).
@@ -551,6 +564,9 @@ class SkillsManager:
# Rename # Rename
new_name = slugify(updates.get("name") or sk.name) new_name = slugify(updates.get("name") or sk.name)
if new_name != sk.name: if new_name != sk.name:
if self.has_skill(new_name, owner=owner):
logger.warning(f"Skill rename target name in use: {new_name}")
return False
sk.name = new_name sk.name = new_name
# Write to potentially new path # Write to potentially new path
+13 -1
View File
@@ -24193,11 +24193,14 @@ async def stream_agent_loop(
budget_is_explicit, budget_is_explicit,
hard_max=hard_max, hard_max=hard_max,
) )
if candidate_context <= 8192: if 0 < candidate_context <= 8192:
# Small local servers often tokenize chat wrappers and tool # Small local servers often tokenize chat wrappers and tool
# results much more generously than our rough estimator. Keep # results much more generously than our rough estimator. Keep
# substantial headroom for those wrappers and generation so a # substantial headroom for those wrappers and generation so a
# follow-up tool round cannot exceed the server's n_ctx. # follow-up tool round cannot exceed the server's n_ctx.
# 0 means the window is unknown, which already gets the
# conservative default budget; clamping it as well cut every
# request to 1200 tokens and dropped open documents (#6678).
effective_budget = min( effective_budget = min(
effective_budget, effective_budget,
max(1200, int(candidate_context * 0.40)), max(1200, int(candidate_context * 0.40)),
@@ -27833,7 +27836,16 @@ async def stream_agent_loop(
if _streamed_tool_markup or _streamed_tool_markup_starts(_delta_text): if _streamed_tool_markup or _streamed_tool_markup_starts(_delta_text):
_streamed_tool_markup += _delta_text _streamed_tool_markup += _delta_text
if not _streamed_tool_markup_complete(_streamed_tool_markup): if not _streamed_tool_markup_complete(_streamed_tool_markup):
if _streamed_tool_markup_starts(_streamed_tool_markup):
continue continue
# False alarm: a bare "<" (e.g. ``x < 2``) looked like
# a tag prefix, but the buffer can no longer become a
# tool tag. Release it as prose instead of holding the
# rest of the answer forever.
_delta_text = _streamed_tool_markup
_streamed_tool_markup = ""
data["delta"] = _delta_text
else:
_visible_markup_tail = strip_tool_blocks( _visible_markup_tail = strip_tool_blocks(
_streamed_tool_markup, _streamed_tool_markup,
skip_fenced=True, skip_fenced=True,
+35 -3
View File
@@ -10,7 +10,7 @@ import os
import json import json
import re import re
import time import time
from datetime import datetime from datetime import datetime, timezone
from typing import Tuple from typing import Tuple
from src.auth_helpers import owner_filter from src.auth_helpers import owner_filter
@@ -1957,6 +1957,23 @@ async def action_learn_sender_signatures(owner: str, **kwargs) -> Tuple[str, boo
return str(e), False return str(e), False
def _daily_brief_utc_boundary(local_dt, tz=None):
"""Convert a local wall-clock boundary using the offset at that boundary."""
if local_dt.tzinfo is None and tz is not None:
local_dt = local_dt.replace(tzinfo=tz)
return local_dt.astimezone(timezone.utc).replace(tzinfo=None)
def _daily_brief_local_start(event, tz=None):
"""Keep date-only/floating rows local; convert only stored UTC instants."""
start = event.dtstart
if event.is_utc and not event.all_day:
if start.tzinfo is None:
start = start.replace(tzinfo=timezone.utc)
return start.astimezone(tz).replace(tzinfo=None)
return start
async def action_daily_brief(owner: str, **kwargs) -> Tuple[str, bool]: async def action_daily_brief(owner: str, **kwargs) -> Tuple[str, bool]:
"""Build a short morning digest: today's calendar events, unread email count """Build a short morning digest: today's calendar events, unread email count
+ top-N senders/subjects, active todos.""" + top-N senders/subjects, active todos."""
@@ -1966,10 +1983,13 @@ async def action_daily_brief(owner: str, **kwargs) -> Tuple[str, bool]:
from core.database import SessionLocal, CalendarEvent, CalendarCal, Note from core.database import SessionLocal, CalendarEvent, CalendarCal, Note
from routes.email_helpers import _imap_connect, _decode_header from routes.email_helpers import _imap_connect, _decode_header
from sqlalchemy import and_, or_
# ----- Calendar: today's events ----- # ----- Calendar: today's events -----
today = _dt.now().replace(hour=0, minute=0, second=0, microsecond=0) today = _dt.now().replace(hour=0, minute=0, second=0, microsecond=0)
tomorrow = today + _td(days=1) tomorrow = today + _td(days=1)
utc_today = _daily_brief_utc_boundary(today)
utc_tomorrow = _daily_brief_utc_boundary(tomorrow)
# v2 review HIGH-12: gate the OR-null branch on single-user # v2 review HIGH-12: gate the OR-null branch on single-user
# (unconfigured) deploys only. In a multi-user deploy, one # (unconfigured) deploys only. In a multi-user deploy, one
# user's daily brief must not include another user's notes or # user's daily brief must not include another user's notes or
@@ -1982,13 +2002,25 @@ async def action_daily_brief(owner: str, **kwargs) -> Tuple[str, bool]:
db = SessionLocal() db = SessionLocal()
try: try:
ev_q = db.query(CalendarEvent).join(CalendarCal).filter( ev_q = db.query(CalendarEvent).join(CalendarCal).filter(
or_(
and_(
CalendarEvent.is_utc.is_(True),
CalendarEvent.all_day.is_not(True),
CalendarEvent.dtstart < utc_tomorrow,
CalendarEvent.dtend > utc_today,
),
and_(
or_(CalendarEvent.is_utc.is_not(True), CalendarEvent.all_day.is_(True)),
CalendarEvent.dtstart < tomorrow, CalendarEvent.dtstart < tomorrow,
CalendarEvent.dtend > today, CalendarEvent.dtend > today,
),
),
CalendarEvent.status != "cancelled", CalendarEvent.status != "cancelled",
) )
if owner: if owner:
ev_q = owner_filter(ev_q, CalendarCal, owner, include_shared=_allow_null) ev_q = owner_filter(ev_q, CalendarCal, owner, include_shared=_allow_null)
events = ev_q.order_by(CalendarEvent.dtstart).all() events = ev_q.all()
events.sort(key=_daily_brief_local_start)
# ----- Notes: pinned + non-archived todos with at least one undone item ----- # ----- Notes: pinned + non-archived todos with at least one undone item -----
n_q = db.query(Note).filter(Note.archived == False) # noqa: E712 n_q = db.query(Note).filter(Note.archived == False) # noqa: E712
if owner: if owner:
@@ -2058,7 +2090,7 @@ async def action_daily_brief(owner: str, **kwargs) -> Tuple[str, bool]:
if events: if events:
plain.append("Calendar:") plain.append("Calendar:")
for e in events: for e in events:
t = e.dtstart.strftime("%H:%M") if not e.all_day else "all day" t = _daily_brief_local_start(e).strftime("%H:%M") if not e.all_day else "all day"
loc = f" @ {e.location}" if e.location else "" loc = f" @ {e.location}" if e.location else ""
plain.append(f" {t} {e.summary}{loc}") plain.append(f" {t} {e.summary}{loc}")
plain.append("") plain.append("")
+7 -1
View File
@@ -282,7 +282,13 @@ class ChatHandler:
_sync_upload_vision_to_gallery(file_info, owner, vl_desc) _sync_upload_vision_to_gallery(file_info, owner, vl_desc)
except Exception: except Exception:
pass pass
enhanced_message = f"{enhanced_message}\n\n[Image: {file_info['name']}]\n{vl_desc}" enhanced_message = f"{enhanced_message}\n\n[Image: {file_info['name']}]\n"
if vl_desc:
# A separate vision call wrote this, but the chat model sees
# one message and answers the prose as if the user wrote it
# (#5573). Label it below the marker line — the client strips
# the block by matching `[Image: <name>]\n`.
enhanced_message += f"[Image description — context only, not part of the user's message]:\n{vl_desc}"
# Surface the description to the client live so it renders as a # Surface the description to the client live so it renders as a
# collapsible "image description" on the user bubble (not just # collapsible "image description" on the user bubble (not just
# after a refresh that re-parses the stored message). # after a refresh that re-parses the stored message).
+4 -1
View File
@@ -142,7 +142,10 @@ def _process_pdf(
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp: with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
temp_img_path = tmp.name temp_img_path = tmp.name
try: try:
img.image.save(temp_img_path, "PNG") # pypdf -> PIL image image = img.image
if image.mode == "CMYK":
image = image.convert("RGB")
image.save(temp_img_path, "PNG")
ocr_text = analyze_image_with_vl(temp_img_path, owner=owner) ocr_text = analyze_image_with_vl(temp_img_path, owner=owner)
if ocr_text and "unavailable" not in ocr_text.lower(): if ocr_text and "unavailable" not in ocr_text.lower():
pdf_text += f"\n\n[Page {page_num + 1} image {img_index + 1} text]: {ocr_text}" pdf_text += f"\n\n[Page {page_num + 1} image {img_index + 1} text]: {ocr_text}"
+63 -8
View File
@@ -76,9 +76,23 @@ _ERROR_DETAIL = {
"auth_or_protocol_error": "authentication or protocol error", "auth_or_protocol_error": "authentication or protocol error",
"no_models": "endpoint returned no models", "no_models": "endpoint returned no models",
"no_host": "no host configured", "no_host": "no host configured",
"config_source_error": "configuration source could not be read",
"error": "probe failed", "error": "probe failed",
} }
# Config/inventory source behind each network subsystem. `_gather_inputs()`
# fails soft, so a source that could not be read hands its probe an empty
# inventory — indistinguishable from "nothing configured", which every probe
# answers with `disabled`. The collector therefore reports the source failure
# instead of running a probe whose verdict would be a lie.
CONFIG_SOURCE_ERROR = "config_source_error"
_SUBSYSTEM_SOURCE = {
"searxng": "settings",
"ntfy": "integrations",
"email": "accounts",
"providers": "endpoints",
}
def _svc(name: str, status: str, detail: str, **meta: Any) -> Dict[str, Any]: def _svc(name: str, status: str, detail: str, **meta: Any) -> Dict[str, Any]:
return {"name": name, "status": status, "detail": detail, "meta": dict(meta)} return {"name": name, "status": status, "detail": detail, "meta": dict(meta)}
@@ -411,26 +425,33 @@ def _gather_inputs() -> Dict[str, Any]:
"""Pull live config/account/endpoint lists from the app's data sources. """Pull live config/account/endpoint lists from the app's data sources.
Each lookup fails soft: a broken source yields an empty/neutral value so a Each lookup fails soft: a broken source yields an empty/neutral value so a
single failure can't take down the whole health report. single failure can't take down the whole health report. The failure is also
*recorded* under `failed` (source -> controlled category), because the empty
value alone cannot be told apart from a source that loaded and is genuinely
empty — and the probes read empty as `disabled`.
""" """
settings: Dict[str, Any] = {} settings: Dict[str, Any] = {}
integrations: List[Dict[str, Any]] = [] integrations: List[Dict[str, Any]] = []
accounts: List[Dict[str, Any]] = [] accounts: List[Dict[str, Any]] = []
endpoints: List[Dict[str, Any]] = [] endpoints: List[Dict[str, Any]] = []
failed: Dict[str, str] = {}
try: try:
from src.settings import load_settings from src.settings import load_settings
settings = load_settings() or {} settings = load_settings() or {}
except Exception as e: except Exception as e:
failed["settings"] = _classify_error(e)
logger.debug(f"service_health: settings load failed: {e}") logger.debug(f"service_health: settings load failed: {e}")
try: try:
from src.integrations import load_integrations from src.integrations import load_integrations
integrations = load_integrations() or [] integrations = load_integrations() or []
except Exception as e: except Exception as e:
failed["integrations"] = _classify_error(e)
logger.debug(f"service_health: integrations load failed: {e}") logger.debug(f"service_health: integrations load failed: {e}")
try: try:
from routes.email_helpers import _list_email_accounts from routes.email_helpers import _list_email_accounts
accounts = _list_email_accounts() or [] accounts = _list_email_accounts() or []
except Exception as e: except Exception as e:
failed["accounts"] = _classify_error(e)
logger.debug(f"service_health: email accounts load failed: {e}") logger.debug(f"service_health: email accounts load failed: {e}")
try: try:
from core.database import SessionLocal, ModelEndpoint from core.database import SessionLocal, ModelEndpoint
@@ -443,9 +464,10 @@ def _gather_inputs() -> Dict[str, Any]:
finally: finally:
db.close() db.close()
except Exception as e: except Exception as e:
failed["endpoints"] = _classify_error(e)
logger.debug(f"service_health: endpoint load failed: {e}") logger.debug(f"service_health: endpoint load failed: {e}")
return {"settings": settings, "integrations": integrations, return {"settings": settings, "integrations": integrations,
"accounts": accounts, "endpoints": endpoints} "accounts": accounts, "endpoints": endpoints, "failed": failed}
async def _run_subsystem(name: str, fn: Callable, *args: Any) -> Dict[str, Any]: async def _run_subsystem(name: str, fn: Callable, *args: Any) -> Dict[str, Any]:
@@ -464,6 +486,24 @@ async def _run_subsystem(name: str, fn: Callable, *args: Any) -> Dict[str, Any]:
return _svc(name, DOWN, _detail_for(category), error=category) return _svc(name, DOWN, _detail_for(category), error=category)
async def _source_unavailable(name: str, source: str,
category: str) -> Dict[str, Any]:
"""Controlled entry for a subsystem whose config source could not be read.
Deliberately not `disabled`: the configured state is *unknown*, not known to
be off. `degraded` rather than `down` because the failure is in reading the
inventory, not a probe that reached the subsystem and failed — but it still
carries non-zero severity, so a report containing one can never roll up to
`ok`. `meta` names the source and the controlled category of the underlying
exception; never its text.
"""
return _svc(name, DEGRADED,
f"Could not read configuration "
f"({_detail_for(CONFIG_SOURCE_ERROR)}).",
error=CONFIG_SOURCE_ERROR, source=source,
source_error=category)
async def collect_service_health(rag_manager: Any = None, async def collect_service_health(rag_manager: Any = None,
memory_vector: Any = None) -> Dict[str, Any]: memory_vector: Any = None) -> Dict[str, Any]:
"""Run every probe and return {overall, services, timestamp}. """Run every probe and return {overall, services, timestamp}.
@@ -472,6 +512,10 @@ async def collect_service_health(rag_manager: Any = None,
four network subsystems run concurrently, each under `_SUBSYSTEM_DEADLINE`, four network subsystems run concurrently, each under `_SUBSYSTEM_DEADLINE`,
with an overall `_AGGREGATE_DEADLINE` backstop. Per-item probes inside with an overall `_AGGREGATE_DEADLINE` backstop. Per-item probes inside
providers/email are themselves bounded by `_FANOUT_BUDGET`. providers/email are themselves bounded by `_FANOUT_BUDGET`.
A subsystem whose config/inventory source failed to load is reported via
`_source_unavailable()` instead of being probed with an empty inventory,
which every probe would otherwise report as `disabled`.
""" """
from datetime import datetime, timezone from datetime import datetime, timezone
@@ -482,12 +526,23 @@ async def collect_service_health(rag_manager: Any = None,
chroma = chromadb_health(rag_manager, memory_vector) chroma = chromadb_health(rag_manager, memory_vector)
names = ["searxng", "ntfy", "email", "providers"] names = ["searxng", "ntfy", "email", "providers"]
coros = [ probes = {
_run_subsystem("searxng", searxng_health, settings), "searxng": (searxng_health, (settings,)),
_run_subsystem("ntfy", ntfy_health, inputs["integrations"], settings), "ntfy": (ntfy_health, (inputs["integrations"], settings)),
_run_subsystem("email", email_health, inputs["accounts"]), "email": (email_health, (inputs["accounts"],)),
_run_subsystem("providers", providers_health, inputs["endpoints"]), "providers": (providers_health, (inputs["endpoints"],)),
] }
# A subsystem whose inventory could not be read is reported as such. The
# others are still probed normally — one broken source degrades one entry.
failed = inputs.get("failed") or {}
coros = []
for name in names:
source = _SUBSYSTEM_SOURCE[name]
if source in failed:
coros.append(_source_unavailable(name, source, failed[source]))
else:
fn, args = probes[name]
coros.append(_run_subsystem(name, fn, *args))
try: try:
results = await asyncio.wait_for(asyncio.gather(*coros), results = await asyncio.wait_for(asyncio.gather(*coros),
timeout=_AGGREGATE_DEADLINE) timeout=_AGGREGATE_DEADLINE)
+33 -5
View File
@@ -57,25 +57,53 @@ def _message_to_context(msg: DBChatMessage) -> dict[str, Any]:
} }
def _escape_like(value: str) -> str: def escape_like_pattern(value: str) -> str:
"""Escape SQL LIKE wildcards (``%``, ``_``) and the escape char itself.
Public shared helper for building ``ilike(f"%{...}%", escape="\\\\")``
patterns from user input. Must escape backslash first.
"""
return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_") return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
def _escape_like(value: str) -> str:
return escape_like_pattern(value)
def _snippet(content: str, query: str, radius: int = 60) -> str: def _snippet(content: str, query: str, radius: int = 60) -> str:
content = content or "" content = content or ""
query = query or "" query = query or ""
if not query: if not query:
return content[: radius * 2] return content[: radius * 2]
idx = content.lower().find(query.lower()) match_start = -1
if idx == -1: match_length = 0
lowered_content = content.lower()
for candidate in _query_terms(query):
idx = lowered_content.find(candidate.lower())
if idx != -1 and (match_start == -1 or idx < match_start):
match_start = idx
match_length = len(candidate)
if match_start == -1:
return content[: radius * 2] return content[: radius * 2]
start = max(0, idx - radius) start = max(0, match_start - radius)
end = min(len(content), idx + len(query) + radius) end = min(len(content), match_start + match_length + radius)
return ("..." if start > 0 else "") + content[start:end] + ("..." if end < len(content) else "") return ("..." if start > 0 else "") + content[start:end] + ("..." if end < len(content) else "")
def _query_terms(query: str) -> list[str]:
"""Return meaningful quoted phrases and tokens in user-entered order."""
terms: list[str] = []
for match in re.finditer(r'"([^"]+)"|[\w][\w._-]*', query, flags=re.UNICODE):
term = match.group(1) if match.group(1) is not None else match.group(0).strip("._-")
term = term.strip()
if term:
terms.append(term)
return terms
def _sanitize_fts_query(query: str) -> str | None: def _sanitize_fts_query(query: str) -> str | None:
"""Convert free text into a conservative FTS5 MATCH query. """Convert free text into a conservative FTS5 MATCH query.
@@ -0,0 +1,71 @@
"""An unknown context window must not be budgeted like a tiny local server (#6678).
``budget_context_for_model`` returns 0 when the endpoint does not prove its
window, and ``compute_input_token_budget`` answers 0 with the conservative
default budget. The agent loop's small-server clamp (``<= 8192``) also caught
0 and cut the request to 1200 tokens, so an open document or earlier context
of a few thousand tokens was trimmed away before the model saw it.
"""
import inspect
import json
import pytest
DONE = 'data: [DONE]\n\n'
MARKER = 'NIGHTJAR-7'
# ~3000 estimated tokens: over the old 1200 clamp, under the 6000 default.
CONTEXT = ''.join(
f'## Project-{n}\nOwner: team {n}. Status: active. Notes: routine maintenance '
'and quarterly review scheduled for every workstream in this registry.\n\n'
for n in range(60)
) + f'## Project-Alpha\nAlias: {MARKER}\n'
def _event(payload):
return 'data: ' + json.dumps(payload) + '\n\n'
async def _first_request(monkeypatch, tmp_path, context_window):
import src.agent_loop as al
from tests.test_agent_runtime_context import _patch_fake_skills
_patch_fake_skills(monkeypatch)
monkeypatch.setattr('src.tool_index.get_tool_index', lambda: None)
monkeypatch.setattr('src.model_context.budget_context_for_model',
lambda *args, **kwargs: context_window)
seen = []
async def provider(candidates, messages, **kwargs):
# Like llm_core: the per-candidate factory shapes and trims the request.
url, model, headers = candidates[0][:3]
request = kwargs['candidate_request_factory'](0, url, model, headers)
if inspect.isawaitable(request):
request = await request
seen.append((request or {}).get('messages', messages))
yield _event({'delta': 'ok'})
yield DONE
monkeypatch.setattr(al, 'stream_llm_with_fallback', provider)
history = [
{'role': 'user', 'content': 'Keep this registry for later:\n\n' + CONTEXT},
{'role': 'assistant', 'content': 'Saved.'},
{'role': 'user', 'content': 'What is the alias of Project-Alpha?'},
]
async for _ in al.stream_agent_loop('http://127.0.0.1:9/v1', 'model', history,
max_rounds=1, workspace=str(tmp_path)):
pass
assert seen, 'the provider was never called'
return json.dumps(seen[0])
@pytest.mark.asyncio
async def test_unknown_context_window_keeps_earlier_context(monkeypatch, tmp_path):
request = await _first_request(monkeypatch, tmp_path, 0)
assert MARKER in request
@pytest.mark.asyncio
async def test_known_small_window_is_still_clamped(monkeypatch, tmp_path):
# The clamp exists for small local servers whose window is known (n_ctx
# 2048 here); it must keep protecting them.
request = await _first_request(monkeypatch, tmp_path, 2048)
assert MARKER not in request
+60
View File
@@ -70,3 +70,63 @@ def test_no_gguf_found_pattern():
result = _diagnose_serve_output(out) result = _diagnose_serve_output(out)
assert result is not None assert result is not None
assert "GGUF" in result["message"] assert "GGUF" in result["message"]
def test_cookbook_routes_uses_shared_diagnosis_helper():
# /api/cookbook/tasks/status must use the maintained helper. A nested copy in
# cookbook_routes.py shadowed the import and silently drifted (missing the
# ModelOpt, mflux and MLX bridge diagnoses).
import ast
from pathlib import Path
source = (Path(__file__).resolve().parents[1] / "routes" / "cookbook_routes.py").read_text(encoding="utf-8")
local_defs = [
node.lineno
for node in ast.walk(ast.parse(source))
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == "_diagnose_serve_output"
]
assert local_defs == []
def test_modelopt_lm_head_checkpoint_returns_diagnosis():
out = "ValueError: There is no module or parameter named 'lm_head.input_scale' in Qwen3ForCausalLM"
result = _diagnose_serve_output(out)
assert result is not None
assert "ModelOpt" in result["message"]
def test_nvidia_driver_too_old_for_torch_returns_diagnosis():
# vLLM 0.31 from PyPI ships torch cu130; on driver 570 (CUDA 12.8) the engine dies here.
out = (
"(EngineCore pid=1) ERROR [core.py:1433] self.driver_worker.init_device()\n"
"(EngineCore pid=1) ERROR [core.py:1433] RuntimeError: The NVIDIA driver on your system is too old "
"(found version 12080). Please update your GPU driver by downloading and installing a new version\n"
"Traceback (most recent call last):\n"
"RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}\n"
)
result = _diagnose_serve_output(out)
assert result is not None
assert "driver" in result["message"].lower()
assert any("cu129" in s["label"] for s in result["suggestions"])
def test_cuda_runtime_newer_than_driver_returns_diagnosis():
# torch switched to cu128 but the vLLM wheel's own kernels still need CUDA 13.
out = (
"RuntimeError: get_cuda_view_from_cpu_tensor, /workspace/csrc/libtorch_stable/cuda_view.cu:38, "
"cudaHostGetDevicePointer failed: CUDA driver version is insufficient for CUDA runtime version\n"
"Traceback (most recent call last):\n"
)
result = _diagnose_serve_output(out)
assert result is not None
assert "driver" in result["message"].lower()
def test_tasks_status_capture_joins_wrapped_tmux_lines():
# tmux wraps at the pane width (often mid-word), which hid errors like
# "insufficie|nt for CUDA runtime version" from the diagnosis patterns.
from pathlib import Path
source = (Path(__file__).resolve().parents[1] / "routes" / "cookbook_routes.py").read_text(encoding="utf-8")
assert '_remote_tmux_command("capture-pane", "-t", session_id, "-p", "-J", "-S", "-500")' in source
assert '["tmux", "capture-pane", "-t", session_id, "-p", "-J", "-S", "-500"]' in source
+62
View File
@@ -0,0 +1,62 @@
import pytest
import httpx
from unittest.mock import patch, AsyncMock
from fastapi import FastAPI
from fastapi.testclient import TestClient
from routes.cookbook_routes import setup_cookbook_routes
from src.auth_helpers import require_user
app = FastAPI()
router = setup_cookbook_routes()
app.include_router(router)
app.dependency_overrides[require_user] = lambda: "test_user"
client = TestClient(app)
def test_hf_gguf_files_success():
mock_resp = httpx.Response(
200,
json={
"siblings": [
{"rfilename": "model-q4_k_m.gguf"},
{"rfilename": "model-q8_0.gguf"},
{"rfilename": "README.md"},
]
},
request=httpx.Request("GET", "https://huggingface.co/api/models/TheBloke/Llama-2-7B-GGUF"),
)
with patch("httpx.AsyncClient.get", new_callable=AsyncMock, return_value=mock_resp):
response = client.get("/api/cookbook/hf-gguf-files?repo_id=TheBloke/Llama-2-7B-GGUF")
assert response.status_code == 200
data = response.json()
assert data["ok"] is True
assert data["repo_id"] == "TheBloke/Llama-2-7B-GGUF"
assert data["files"] == ["model-q4_k_m.gguf", "model-q8_0.gguf"]
def test_hf_gguf_files_http_error():
mock_resp = httpx.Response(
404,
json={"error": "Model not found"},
request=httpx.Request("GET", "https://huggingface.co/api/models/invalid/repo"),
)
with patch("httpx.AsyncClient.get", new_callable=AsyncMock, return_value=mock_resp):
response = client.get("/api/cookbook/hf-gguf-files?repo_id=invalid/repo")
assert response.status_code == 200
data = response.json()
assert data["ok"] is False
assert "HF API HTTP 404" in data["error"]
def test_hf_gguf_files_exception_handling():
with patch("httpx.AsyncClient.get", new_callable=AsyncMock, side_effect=httpx.ConnectError("Network unreachable")):
response = client.get("/api/cookbook/hf-gguf-files?repo_id=TheBloke/Llama-2-7B-GGUF")
# Should gracefully return JSON error payload, not raise NameError / 500
assert response.status_code == 200
data = response.json()
assert data["ok"] is False
assert data["error"] == "HF API request failed"
+103
View File
@@ -0,0 +1,103 @@
"""Daily brief must query and display each calendar storage kind in local time."""
import asyncio
import os
import time
from contextlib import contextmanager
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
from zoneinfo import ZoneInfo
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import StaticPool
from core import database
from src import builtin_actions
@contextmanager
def system_timezone(name):
previous = os.environ.get('TZ')
os.environ['TZ'] = name
time.tzset()
try:
yield
finally:
if previous is None:
os.environ.pop('TZ', None)
else:
os.environ['TZ'] = previous
time.tzset()
@pytest.mark.skipif(not hasattr(time, 'tzset'), reason='requires changing the process local timezone')
@pytest.mark.parametrize('zone', ['Europe/Berlin', 'America/Los_Angeles'])
def test_daily_brief_selects_local_day_and_orders_mixed_storage(monkeypatch, zone):
engine = create_engine('sqlite://', poolclass=StaticPool)
for model in [database.CalendarCal, database.CalendarEvent, database.Note]:
model.__table__.create(engine)
sessions = sessionmaker(bind=engine)
monkeypatch.setattr(database, 'SessionLocal', sessions)
from core.auth import AuthManager
monkeypatch.setattr(AuthManager, 'is_configured', property(lambda self: True))
from routes import email_helpers
def no_email(*args):
raise RuntimeError('No email account in this calendar-only test')
monkeypatch.setattr(email_helpers, '_imap_connect', no_email)
try:
with system_timezone(zone):
today = datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)
def utc(value):
return value.astimezone(timezone.utc).replace(tzinfo=None)
with sessions() as db:
for owner in ['alice', 'bob', None]:
db.add(database.CalendarCal(id=owner or 'shared', owner=owner, name=owner or 'shared'))
db.flush()
def add(uid, start, end, *, utc_row=False, all_day=False, owner='alice', status='confirmed'):
db.add(database.CalendarEvent(uid=uid, calendar_id=owner or 'shared', summary=uid,
dtstart=utc(start) if utc_row and not all_day else start,
dtend=utc(end) if utc_row and not all_day else end,
is_utc=utc_row, all_day=all_day, status=status))
add('early', today+timedelta(minutes=30), today+timedelta(hours=1), utc_row=True)
add('utc-ten', today+timedelta(hours=10), today+timedelta(hours=11), utc_row=True)
add('local-nine', today+timedelta(hours=9), today+timedelta(hours=10))
add('all-day', today, today+timedelta(days=1), utc_row=True, all_day=True)
add('next-day', today+timedelta(days=1,minutes=30), today+timedelta(days=1,hours=1), utc_row=True)
add('ended', today-timedelta(hours=1), today, utc_row=True)
add('cancelled', today+timedelta(hours=12), today+timedelta(hours=13), utc_row=True, status='cancelled')
add('foreign', today+timedelta(hours=12), today+timedelta(hours=13), utc_row=True, owner='bob')
add('unowned', today+timedelta(hours=12), today+timedelta(hours=13), owner=None)
db.commit()
text, success = asyncio.run(builtin_actions.action_daily_brief('alice'))
assert success, text
lines = ['all day all-day', '00:30 early', '09:00 local-nine', '10:00 utc-ten']
assert all(line in text for line in lines), text
assert [text.index(line) for line in lines] == sorted(text.index(line) for line in lines), text
for excluded in ['next-day', 'ended', 'cancelled', 'foreign', 'unowned']:
assert excluded not in text, text
finally:
engine.dispose()
@pytest.mark.parametrize('day, hours', [('2026-03-29', 23), ('2026-10-25', 25)])
def test_utc_day_window_uses_offset_at_each_boundary(day, hours):
start = datetime.fromisoformat(day)
zone = ZoneInfo('Europe/Berlin')
begin = builtin_actions._daily_brief_utc_boundary(start, zone)
end = builtin_actions._daily_brief_utc_boundary(start+timedelta(days=1), zone)
assert end-begin == timedelta(hours=hours)
if hasattr(time, 'tzset'):
with system_timezone('Europe/Berlin'):
assert builtin_actions._daily_brief_utc_boundary(start) == begin
assert builtin_actions._daily_brief_utc_boundary(start+timedelta(days=1)) == end
@pytest.mark.parametrize('utc_row, all_day, expected', [
(True, False, datetime(2026, 6, 1, 10)),
(False, False, datetime(2026, 6, 1, 8)),
(True, True, datetime(2026, 6, 1, 8)),
])
def test_local_start_preserves_floating_and_date_only_values(utc_row, all_day, expected):
event = SimpleNamespace(dtstart=datetime(2026,6,1,8), is_utc=utc_row, all_day=all_day)
assert builtin_actions._daily_brief_local_start(event, ZoneInfo('Europe/Berlin')) == expected
+86
View File
@@ -0,0 +1,86 @@
"""Issue #2137: never preselect an embedding or other non-chat model."""
import json
from types import SimpleNamespace
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from core.database import ModelEndpoint
from routes import model_routes, prefs_routes
from tests.helpers.database import disposable_database
@pytest.fixture
def defaults_api(monkeypatch, tmp_path):
defaults = {}
with disposable_database(tmp_path) as factory:
monkeypatch.setattr(model_routes, "SessionLocal", factory)
monkeypatch.setattr(model_routes, "_load_settings", lambda: {})
monkeypatch.setattr(prefs_routes, "_load_for_user", lambda owner: defaults)
app = FastAPI()
app.state.auth_manager = SimpleNamespace(is_admin=lambda owner: False)
@app.middleware("http")
async def owner(request, call_next):
request.state.current_user = "alice"
return await call_next(request)
app.include_router(model_routes.setup_model_routes(model_discovery=None))
with TestClient(app) as client:
yield client, factory, defaults
def _endpoint(factory, *, cached=(), pinned=(), hidden=(), owner="alice", endpoint_id="ep"):
with factory() as db:
db.add(ModelEndpoint(
id=endpoint_id, name="Fixture", base_url="http://model.invalid/v1", owner=owner,
is_enabled=True, cached_models=json.dumps(cached),
pinned_models=json.dumps(pinned), hidden_models=json.dumps(hidden),
))
db.commit()
@pytest.mark.parametrize("non_chat", [
"text-embedding-3-small", "nomic-embed-text:latest", "jina-reranker-v2-base",
"tts-1", "whisper-1", "gpt-image-1",
])
def test_automatic_default_skips_non_chat_models(defaults_api, non_chat):
client, factory, _ = defaults_api
_endpoint(factory, cached=[non_chat, "chat-model"])
assert client.get("/api/default-chat").json()["model"] == "chat-model"
def test_non_chat_saved_default_falls_back_to_visible_pinned_chat(defaults_api):
client, factory, defaults = defaults_api
_endpoint(factory, cached=["text-embedding-3-small", "hidden-chat"],
pinned=["jina-reranker-v2-base", "pinned-chat"], hidden=["hidden-chat"])
defaults.update(default_endpoint_id="ep", default_model="text-embedding-3-small")
assert client.get("/api/default-chat").json()["model"] == "pinned-chat"
def test_no_chat_candidate_leaves_model_empty(defaults_api):
client, factory, defaults = defaults_api
_endpoint(factory, cached=["text-embedding-3-small", "hidden-chat"],
pinned=["jina-reranker-v2-base"], hidden=["hidden-chat"])
defaults.update(default_endpoint_id="ep", default_model="text-embedding-3-small")
result = client.get("/api/default-chat").json()
assert result == {"endpoint_id": "ep", "endpoint_url": "http://model.invalid/v1/chat/completions", "model": ""}
def test_valid_explicit_model_survives_stale_catalog(defaults_api):
client, factory, defaults = defaults_api
_endpoint(factory, cached=["text-embedding-3-small", "other-chat"])
defaults.update(default_endpoint_id="ep", default_model="my-custom-chat")
assert client.get("/api/default-chat").json()["model"] == "my-custom-chat"
def test_model_filter_preserves_owner_scope(defaults_api):
client, factory, defaults = defaults_api
_endpoint(factory, owner="bob", endpoint_id="bobs-endpoint", cached=["bobs-chat"])
_endpoint(factory, cached=["text-embedding-3-small", "alices-chat"])
defaults.update(default_endpoint_id="bobs-endpoint")
result = client.get("/api/default-chat").json()
assert result["endpoint_id"] == "ep"
assert result["model"] == "alices-chat"
@@ -0,0 +1,96 @@
"""ZIP exports keep language extensions and place collision suffixes before them."""
import io
import zipfile
import pytest
from fastapi import FastAPI, Request
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from core.database import Base, Document
from routes.document import document_routes
@pytest.fixture
def exports(tmp_path, monkeypatch):
engine = create_engine(f"sqlite:///{tmp_path / 'documents.db'}", connect_args={"check_same_thread": False})
Base.metadata.create_all(engine)
factory = sessionmaker(bind=engine)
monkeypatch.setattr(document_routes, "SessionLocal", factory)
monkeypatch.setenv("AUTH_ENABLED", "true")
app = FastAPI()
@app.middleware("http")
async def fixture_user(request: Request, call_next):
request.state.current_user = "alice"
return await call_next(request)
app.include_router(document_routes.setup_document_routes(None))
def add(doc_id, title, language, content="body\n", owner="alice"):
with factory() as db:
db.add(Document(id=doc_id, title=title, language=language,
current_content=content, owner=owner, is_active=True))
db.commit()
with TestClient(app) as client:
yield client, add
engine.dispose()
@pytest.mark.parametrize("title,language,filename", [
("config.local", "python", "config.local.py"),
("report.v2", "markdown", "report.v2.md"),
("data.2024", "json", "data.2024.json"),
("module.ts", "typescript", "module.ts"),
("SCRIPT.PY", "python", "SCRIPT.PY"),
("résumé.v2", "text", "résumé.v2.txt"),
("unknown.v2", "unknown", "unknown.v2.txt"),
("draft", None, "draft.txt"),
("project/report.v2", "markdown", "projectreport.v2.md"),
])
def test_export_uses_the_language_extension_even_for_dotted_titles(exports, title, language, filename):
client, add = exports
body = "Original text: α\n"
add("doc", title, language, body)
response = client.post("/api/documents/export-zip", json={"ids": ["doc"]})
assert response.status_code == 200
assert response.headers["content-type"] == "application/zip"
with zipfile.ZipFile(io.BytesIO(response.content)) as archive:
assert archive.namelist() == [filename]
assert archive.read(filename).decode() == body
@pytest.mark.parametrize("title", ["config.local", "config.local.py"])
def test_duplicate_names_keep_the_extension_at_the_end(exports, title):
client, add = exports
for i in range(3):
add(str(i), title, "python", f"body {i}")
response = client.post("/api/documents/export-zip", json={"ids": ["0", "1", "2"]})
assert response.status_code == 200
with zipfile.ZipFile(io.BytesIO(response.content)) as archive:
assert set(archive.namelist()) == {"config.local.py", "config.local-1.py", "config.local-2.py"}
assert {archive.read(name).decode() for name in archive.namelist()} == {"body 0", "body 1", "body 2"}
def test_added_extension_collides_safely_with_an_existing_extension(exports):
client, add = exports
add("a", "report.v2", "markdown", "first")
add("b", "report.v2.md", "markdown", "second")
response = client.post("/api/documents/export-zip", json={"ids": ["a", "b"]})
assert response.status_code == 200
with zipfile.ZipFile(io.BytesIO(response.content)) as archive:
assert set(archive.namelist()) == {"report.v2.md", "report.v2-1.md"}
assert {archive.read(name) for name in archive.namelist()} == {b"first", b"second"}
def test_foreign_documents_are_still_excluded_and_email_body_is_preserved(exports):
client, add = exports
add("owned", "reply.v2", "email", "Subject: Hello\n---\nBody")
add("foreign", "private", "python", "secret", owner="bob")
response = client.post("/api/documents/export-zip", json={"ids": ["owned", "foreign"]})
assert response.status_code == 200
with zipfile.ZipFile(io.BytesIO(response.content)) as archive:
assert archive.namelist() == ["reply.v2.eml"]
assert archive.read("reply.v2.eml") == b"Subject: Hello\r\n\r\nBody"
@@ -0,0 +1,74 @@
"""Literal punctuation in library queries must not become SQL wildcards."""
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from core.database import Document
from routes.document import document_routes
from tests.helpers.database import disposable_database
CASES = [
("100%", "100% complete", "1000 complete"),
("a_b", "a_b", "axb"),
("%", "price 10%", "price 100"),
("_", "a_b", "axb"),
(r"path\notes", r"path\notes", "pathnotes"),
(r"path\_tag", r"path\_tag", r"path\Xtag"),
("50%_done", "50%_done", "500Xdone"),
("can't", "can't stop", "cannot stop"),
("mixed case", "Case ordered Mixed", "Mixed without"),
]
@pytest.mark.parametrize("column", ["title", "current_content"])
@pytest.mark.parametrize("query,literal,lookalike", CASES)
@pytest.mark.asyncio
async def test_library_search_matches_literal_tokens(tmp_path, monkeypatch, column, query, literal, lookalike):
with disposable_database(tmp_path) as factory:
monkeypatch.setattr(document_routes, "SessionLocal", factory)
with factory() as db:
for doc_id, text, owner, active, archived in [
("literal", literal, "alice", True, False),
("lookalike", lookalike, "alice", True, False),
("foreign", literal, "bob", True, False),
("inactive", literal, "alice", False, False),
("archived", literal, "alice", True, True),
]:
fields = {"title": "Document", "current_content": "Body", column: text}
db.add(Document(id=doc_id, owner=owner, is_active=active, archived=archived,
language="text", version_count=1, **fields))
db.commit()
router = document_routes.setup_document_routes(MagicMock())
endpoint = next(r.endpoint for r in router.routes if r.path == "/api/documents/library")
request = SimpleNamespace(state=SimpleNamespace(current_user="alice"))
result = await endpoint(request, search=query, language=None, sort="recent", offset=0, limit=20, archived=False)
assert result["total"] == 1
assert [row["id"] for row in result["documents"]] == ["literal"]
archived = await endpoint(request, search=query, language=None, sort="recent", offset=0, limit=20, archived=True)
assert [row["id"] for row in archived["documents"]] == ["archived"]
@pytest.mark.asyncio
async def test_literal_search_keeps_cross_field_and_pagination_behavior(tmp_path, monkeypatch):
with disposable_database(tmp_path) as factory:
monkeypatch.setattr(document_routes, "SessionLocal", factory)
with factory() as db:
for doc_id, title, content in [
("first", "100% ready", "done"),
("second", "done", "100% ready"),
("lookalike", "1000 ready", "done"),
("missing-term", "100% ready", "waiting"),
]:
db.add(Document(id=doc_id, owner="alice", title=title, current_content=content,
is_active=True, archived=False, language="text", version_count=1))
db.commit()
router = document_routes.setup_document_routes(MagicMock())
endpoint = next(r.endpoint for r in router.routes if r.path == "/api/documents/library")
request = SimpleNamespace(state=SimpleNamespace(current_user="alice"))
results = [await endpoint(request, search="100% done", language="text", sort="alpha",
offset=offset, limit=1, archived=False) for offset in [0, 1]]
assert all(result["total"] == 2 for result in results)
assert {row["id"] for result in results for row in result["documents"]} == {"first", "second"}
+49
View File
@@ -0,0 +1,49 @@
import asyncio
import pytest
from fastapi import HTTPException
import routes.gallery_routes as gallery
def route(path, method):
return next(r for r in gallery.setup_gallery_routes().routes if r.path == path and method in r.methods)
def call(path, method, monkeypatch, body):
monkeypatch.setattr(gallery, 'get_current_user', lambda request: 'alice')
class Request:
async def json(self):
return body
return asyncio.run(route(path, method).endpoint(Request(), 'album-1'))
@pytest.mark.parametrize('body', [
{'name': ''},
{'name': ' '},
{'name': 123},
{'name': ['x']},
{'description': {'a': 1}},
[1],
])
def test_album_update_rejects_bad_input(body, monkeypatch):
with pytest.raises(HTTPException) as exc:
call('/api/gallery/albums/{album_id}', 'PUT', monkeypatch, body)
assert exc.value.status_code == 400
@pytest.mark.parametrize('op', ['add', 'remove'])
@pytest.mark.parametrize('body', [
{'image_ids': 'abc'},
{'image_ids': 5},
{'image_ids': None},
{'image_ids': [{'a': 1}]},
{'image_ids': ['ok', 1]},
[1],
])
def test_album_image_ids_must_be_a_list_of_strings(op, body, monkeypatch):
with pytest.raises(HTTPException) as exc:
call(f'/api/gallery/albums/{{album_id}}/{op}', 'POST', monkeypatch, body)
assert exc.value.status_code == 400
+135
View File
@@ -0,0 +1,135 @@
"""Regression for #2123 (gallery part): library search must match multi-word
queries in any order and treat % / _ literally.
Old code built a single ``%{search}%`` LIKE over prompt/tags/ai_tags, so
"red car" missed "a car that is red", and a literal "%" matched every row.
"""
import uuid
from fastapi import FastAPI
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import NullPool
import core.database as cdb
from core.database import GalleryImage
import routes.gallery_routes as gallery_routes
def _client_with_gallery(monkeypatch, tmp_path):
engine = create_engine(
f"sqlite:///{tmp_path / 'gallery_search.db'}",
connect_args={"check_same_thread": False},
poolclass=NullPool,
)
cdb.Base.metadata.create_all(engine)
session_factory = sessionmaker(bind=engine, autoflush=False, autocommit=False)
monkeypatch.setattr(gallery_routes, "SessionLocal", session_factory)
db = session_factory()
try:
db.add_all(
[
GalleryImage(
id="img-red-car",
filename=f"{uuid.uuid4().hex}.png",
prompt="a car that is red",
model="model-a",
tags="",
ai_tags="",
is_active=True,
file_size=10,
),
GalleryImage(
id="img-boat",
filename=f"{uuid.uuid4().hex}.png",
prompt="blue boat on the lake",
model="model-a",
tags="",
ai_tags="",
is_active=True,
file_size=10,
),
GalleryImage(
id="img-percent",
filename=f"{uuid.uuid4().hex}.png",
prompt="100% cotton shirt",
model="model-a",
tags="sale_2024",
ai_tags="",
is_active=True,
file_size=10,
),
GalleryImage(
id="img-backslash",
filename=f"{uuid.uuid4().hex}.png",
prompt=r"C:\photos\sunset",
model="model-a",
tags="",
ai_tags="",
is_active=True,
file_size=10,
),
]
)
db.commit()
finally:
db.close()
app = FastAPI()
app.include_router(gallery_routes.setup_gallery_routes())
return TestClient(app)
def test_gallery_search_matches_words_in_any_order(monkeypatch, tmp_path):
monkeypatch.setenv("AUTH_ENABLED", "false")
client = _client_with_gallery(monkeypatch, tmp_path)
body = client.get("/api/gallery/library", params={"search": "red car"}).json()
ids = {item["id"] for item in body["items"]}
assert "img-red-car" in ids
assert "img-boat" not in ids
def test_gallery_search_escapes_percent_wildcard(monkeypatch, tmp_path):
monkeypatch.setenv("AUTH_ENABLED", "false")
client = _client_with_gallery(monkeypatch, tmp_path)
body = client.get("/api/gallery/library", params={"search": "%"}).json()
ids = {item["id"] for item in body["items"]}
assert ids == {"img-percent"}
def test_gallery_search_escapes_underscore_wildcard(monkeypatch, tmp_path):
monkeypatch.setenv("AUTH_ENABLED", "false")
client = _client_with_gallery(monkeypatch, tmp_path)
# Only img-percent has a literal underscore (in tags "sale_2024").
# Unescaped, "_" matches any single char so every row matches.
body = client.get("/api/gallery/library", params={"search": "_"}).json()
ids = {item["id"] for item in body["items"]}
assert ids == {"img-percent"}
def test_gallery_search_escapes_backslash(monkeypatch, tmp_path):
monkeypatch.setenv("AUTH_ENABLED", "false")
client = _client_with_gallery(monkeypatch, tmp_path)
# Only img-backslash has a literal backslash. The escape char itself
# must survive escaping instead of escaping the surrounding wildcards.
body = client.get("/api/gallery/library", params={"search": "\\"}).json()
ids = {item["id"] for item in body["items"]}
assert ids == {"img-backslash"}
def test_gallery_tag_filter_escapes_wildcards(monkeypatch, tmp_path):
monkeypatch.setenv("AUTH_ENABLED", "false")
client = _client_with_gallery(monkeypatch, tmp_path)
# Unescaped, "%" / "_" match every row; escaped, only literal matches.
assert client.get("/api/gallery/library", params={"tag": "%"}).json()["items"] == []
body = client.get("/api/gallery/library", params={"tag": "_"}).json()
assert {item["id"] for item in body["items"]} == {"img-percent"}
+98
View File
@@ -0,0 +1,98 @@
"""Actual embedded PDF images must reach vision as a decodable PNG."""
from pathlib import Path
from PIL import Image
from pypdf import PdfReader, PdfWriter
from pypdf.generic import DecodedStreamObject, DictionaryObject, NameObject, NumberObject
import pytest
from src import document_processor as dp
def make_pdf(tmp_path, mode):
colors = {"CMYK": (0, 255, 255, 0), "RGB": (255, 0, 0), "L": 127}
path = tmp_path / (mode + ".pdf")
image = Image.new(mode, (128, 96), colors[mode])
if mode == "CMYK":
# A raw DeviceCMYK XObject avoids JPEG/Adobe inversion differences in
# Pillow-generated PDFs; the extracted color samples are unambiguous.
writer = PdfWriter()
page = writer.add_blank_page(width=128, height=96)
stream = DecodedStreamObject()
stream.set_data(image.tobytes())
stream.update({NameObject(key): value for key, value in {
"/Type": NameObject("/XObject"), "/Subtype": NameObject("/Image"),
"/ColorSpace": NameObject("/DeviceCMYK"), "/BitsPerComponent": NumberObject(8),
"/Width": NumberObject(128), "/Height": NumberObject(96),
}.items()})
page[NameObject("/Resources")] = DictionaryObject({NameObject("/XObject"):
DictionaryObject({NameObject("/Im0"): writer._add_object(stream)})})
content = DecodedStreamObject()
content.set_data(b"q 128 0 0 96 0 0 cm /Im0 Do Q")
page[NameObject("/Contents")] = writer._add_object(content)
writer.write(path)
else:
image.save(path, "PDF")
assert PdfReader(path).pages[0].images[0].image.mode == mode
return path
@pytest.mark.parametrize("mode,expected", [("CMYK", "RGB"), ("RGB", "RGB"), ("L", "L")])
def test_pdf_image_reaches_vision_with_owner_and_png_pixels(tmp_path, monkeypatch, mode, expected):
pdf = make_pdf(tmp_path, mode)
calls = []
def analyze(path, owner=None):
with Image.open(path) as image:
image.load()
assert image.format == "PNG" and image.mode == expected
assert image.size == (128, 96)
if mode == "CMYK":
red, green, blue = image.getpixel((64, 48))
assert red > 250 and green < 5 and blue < 5
calls.append((Path(path), owner))
return "Synthetic OCR result"
monkeypatch.setattr(dp, "analyze_image_with_vl", analyze)
output = dp._process_pdf(str(pdf), owner="alice")
assert "[Page 1 image 1 text]: Synthetic OCR result" in output
assert len(calls) == 1 and calls[0][1] == "alice"
assert not calls[0][0].exists(), "Temporary PNG must be removed after analysis"
assert PdfReader(pdf).pages[0].images[0].image.mode == mode, "Source PDF is unchanged"
def test_mixed_pdf_keeps_images_on_both_pages(tmp_path, monkeypatch):
writer = PdfWriter()
for mode in ("CMYK", "RGB"):
writer.append(str(make_pdf(tmp_path, mode)))
pdf = tmp_path / "mixed.pdf"
writer.write(pdf)
calls = []
def analyze(path, owner=None):
with Image.open(path) as image:
image.load()
calls.append(image.mode)
return "Image content"
monkeypatch.setattr(dp, "analyze_image_with_vl", analyze)
output = dp._process_pdf(str(pdf), owner="alice")
assert calls == ["RGB", "RGB"]
assert "Page 1 image 1 text" in output and "Page 2 image 1 text" in output
def test_text_only_extraction_does_not_enable_vision(tmp_path, monkeypatch):
pdf = make_pdf(tmp_path, "CMYK")
def unexpected(*args, **kwargs):
pytest.fail("Text-only extraction must not call vision")
monkeypatch.setattr(dp, "analyze_image_with_vl", unexpected)
assert "no readable content" in dp.extract_local_document(str(pdf), owner="alice")
def test_vision_failure_still_removes_converted_png(tmp_path, monkeypatch):
pdf = make_pdf(tmp_path, "CMYK")
paths = []
def unavailable(path, owner=None):
paths.append(Path(path))
with Image.open(path) as image:
assert image.mode == "RGB" and image.format == "PNG"
raise RuntimeError("Synthetic unavailable vision endpoint")
monkeypatch.setattr(dp, "analyze_image_with_vl", unavailable)
assert "no readable content" in dp._process_pdf(str(pdf), owner="alice")
assert len(paths) == 1 and not paths[0].exists()
+108
View File
@@ -0,0 +1,108 @@
"""Route defaults must use manually pinned IDs without widening ownership."""
import json
from types import SimpleNamespace
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
import core.database as cdb
import routes.chat_routes as chat
from routes.task.task_routes import _resolve_run_endpoint
from routes.research.research_routes import _resolve_endpoint_runtime
@pytest.fixture
def database(monkeypatch):
engine = create_engine("sqlite:///:memory:")
cdb.Base.metadata.create_all(engine)
factory = sessionmaker(bind=engine)
monkeypatch.setattr(chat, "SessionLocal", factory)
with factory() as db:
yield db
engine.dispose()
def endpoint(db, eid="owned", *, owner="alice", cached=(), pinned=("pinned-chat",), hidden=()):
row = cdb.ModelEndpoint(
id=eid, name=eid, owner=owner, is_enabled=True,
base_url=f"https://{eid}.example.test/v1", api_key=None,
cached_models=json.dumps(cached), pinned_models=json.dumps(pinned),
hidden_models=json.dumps(hidden),
)
db.add(row)
db.commit()
return row
@pytest.mark.parametrize("cached", [(), ("hidden-chat",)])
def test_empty_session_recovers_a_visible_pinned_model(database, cached):
ep = endpoint(database, cached=cached, hidden=("hidden-chat",))
database.add(cdb.Session(id="chat", name="Chat", owner="alice", model="", endpoint_url=ep.base_url, endpoint_id=ep.id))
database.commit()
session = SimpleNamespace(model="", endpoint_url=ep.base_url, endpoint_id=ep.id)
assert chat._recover_empty_session_model(session, "chat", "alice") is True
assert session.model == "pinned-chat"
database.expire_all()
assert database.get(cdb.Session, "chat").model == "pinned-chat"
def test_empty_session_does_not_recover_a_hidden_pin(database):
ep = endpoint(database, hidden=("pinned-chat",))
session = SimpleNamespace(model="", endpoint_url=ep.base_url, endpoint_id=ep.id)
assert chat._recover_empty_session_model(session, "missing", "alice") is False
assert session.model == ""
def test_subscription_recovery_still_requires_its_live_catalog(database):
ep = endpoint(database)
ep.base_url = "https://chatgpt.com/backend-api/codex"
database.commit()
session = SimpleNamespace(model="", endpoint_url=ep.base_url, endpoint_id=ep.id)
assert chat._recover_empty_session_model(session, "missing", "alice") is False
assert session.model == ""
def test_task_run_reopens_against_its_owners_pinned_endpoint(database):
endpoint(database, "foreign", owner="bob", cached=("pinned-chat",))
own = endpoint(database)
task = SimpleNamespace(owner="alice", endpoint_url="", session_id=None, model="pinned-chat")
assert _resolve_run_endpoint(database, task, SimpleNamespace(model="pinned-chat")) == own.base_url
@pytest.mark.parametrize("owner", ["alice", None])
def test_task_run_can_find_visible_pins_on_owned_or_shared_endpoints(database, owner):
ep = endpoint(database, owner=owner)
task = SimpleNamespace(owner="alice", endpoint_url="", session_id=None, model="pinned-chat")
assert _resolve_run_endpoint(database, task, SimpleNamespace(model=None)) == ep.base_url
def test_task_run_does_not_autoselect_a_hidden_pin(database):
endpoint(database, hidden=("pinned-chat",))
task = SimpleNamespace(owner="alice", endpoint_url="", session_id=None, model="pinned-chat")
assert _resolve_run_endpoint(database, task, SimpleNamespace(model="pinned-chat")) == ""
@pytest.mark.parametrize("cached", [(), ("hidden-chat",)])
def test_research_default_uses_the_visible_pinned_id(database, cached):
ep = endpoint(database, cached=cached, hidden=("hidden-chat",))
assert _resolve_endpoint_runtime(ep, owner="alice") == (
ep.base_url + "/chat/completions", "pinned-chat", {},
)
def test_research_default_does_not_autoselect_a_hidden_pin(database):
ep = endpoint(database, hidden=("pinned-chat",))
assert _resolve_endpoint_runtime(ep, owner="alice") is None
def test_research_explicit_model_selection_is_preserved(database):
ep = endpoint(database, cached=("cached-chat",))
assert _resolve_endpoint_runtime(ep, owner="alice", model="explicit-chat")[1] == "explicit-chat"
def test_research_cached_default_keeps_priority_over_pins(database):
ep = endpoint(database, cached=("cached-chat",))
assert _resolve_endpoint_runtime(ep, owner="alice")[1] == "cached-chat"
+167
View File
@@ -137,3 +137,170 @@ def test_collect_aggregate_deadline_yields_controlled_result(monkeypatch):
net = [s for s in out["services"] if s["name"] != "chromadb"] net = [s for s in out["services"] if s["name"] != "chromadb"]
assert all(s["status"] == sh.DOWN and s["meta"].get("error") == "timeout" assert all(s["status"] == sh.DOWN and s["meta"].get("error") == "timeout"
for s in net) for s in net)
# ── Discovery failure is not "disabled" (#6154) ──
#
# `_gather_inputs()` fails soft to an empty inventory, and every probe reads an
# empty inventory as `disabled`. Without a record of *why* it is empty, a broken
# integrations file / email store / endpoint DB is reported as an intentionally
# unconfigured subsystem — and, with the rest disabled, `overall: "ok"`.
def _boom(*_a, **_k):
raise RuntimeError("secret-bearing detail: postgres://u:p@db/x")
def test_gather_inputs_records_each_failed_source(monkeypatch):
import core.database
import routes.email_helpers
import src.integrations
import src.settings
monkeypatch.setattr(src.settings, "load_settings", lambda: {"a": 1})
monkeypatch.setattr(src.integrations, "load_integrations", _boom)
monkeypatch.setattr(routes.email_helpers, "_list_email_accounts", _boom)
monkeypatch.setattr(core.database, "SessionLocal", _boom)
out = sh._gather_inputs()
assert out["settings"] == {"a": 1}
assert set(out["failed"]) == {"integrations", "accounts", "endpoints"}
# Controlled categories only — never the exception text.
assert set(out["failed"].values()) <= set(sh._ERROR_DETAIL)
assert "postgres" not in repr(out)
def test_gather_inputs_reports_no_failures_when_sources_load(monkeypatch):
import core.database
import routes.email_helpers
import src.integrations
import src.settings
class _DB:
def query(self, *_a):
return self
def filter(self, *_a):
return self
def all(self):
return []
def close(self):
pass
monkeypatch.setattr(src.settings, "load_settings", lambda: {})
monkeypatch.setattr(src.integrations, "load_integrations", lambda: [])
monkeypatch.setattr(routes.email_helpers, "_list_email_accounts", lambda: [])
monkeypatch.setattr(core.database, "SessionLocal", lambda: _DB())
assert sh._gather_inputs()["failed"] == {}
@pytest.mark.parametrize("source,service", [
("settings", "searxng"),
("integrations", "ntfy"),
("accounts", "email"),
("endpoints", "providers"),
])
def test_failed_source_is_not_reported_as_disabled(monkeypatch, source, service):
import asyncio
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {"search_provider": "disabled"},
"integrations": [], "accounts": [], "endpoints": [],
"failed": {source: "network_error"},
})
out = asyncio.run(sh.collect_service_health(None, None))
entry = next(s for s in out["services"] if s["name"] == service)
assert entry["status"] != sh.DISABLED
assert entry["meta"]["error"] == sh.CONFIG_SOURCE_ERROR
assert entry["meta"]["source"] == source
assert entry["meta"]["source_error"] == "network_error"
# A failed source must reach the aggregate verdict.
assert out["overall"] != sh.OK
assert set(out) == {"overall", "services", "timestamp"}
def test_all_sources_failing_cannot_report_overall_ok(monkeypatch):
# The exact shape from the report: settings loads and says search is off,
# the other three sources raise. Before the fix this was overall "ok" with
# five "disabled" entries.
import asyncio
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {"search_provider": "disabled"},
"integrations": [], "accounts": [], "endpoints": [],
"failed": {"integrations": "error", "accounts": "error",
"endpoints": "error"},
})
out = asyncio.run(sh.collect_service_health(None, None))
by_name = {s["name"]: s for s in out["services"]}
assert out["overall"] == sh.DEGRADED
# Loaded-and-genuinely-empty still reads as disabled.
assert by_name["searxng"]["status"] == sh.DISABLED
for name in ("ntfy", "email", "providers"):
assert by_name[name]["status"] == sh.DEGRADED
def test_empty_inventory_without_failure_stays_disabled(monkeypatch):
import asyncio
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {"search_provider": "disabled"},
"integrations": [], "accounts": [], "endpoints": [], "failed": {},
})
out = asyncio.run(sh.collect_service_health(_Store(True), _Store(True)))
assert out["overall"] == sh.OK
assert all(s["status"] == sh.DISABLED
for s in out["services"] if s["name"] != "chromadb")
def test_one_failed_source_still_probes_the_others(monkeypatch):
# A broken endpoint DB must not stop ntfy/email from being probed.
import asyncio
probed = []
def _probe(name):
def _fn(*_a, **_k):
probed.append(name)
return sh._svc(name, sh.OK, "probed")
return _fn
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {}, "integrations": [], "accounts": [], "endpoints": [],
"failed": {"endpoints": "timeout"},
})
monkeypatch.setattr(sh, "searxng_health", _probe("searxng"))
monkeypatch.setattr(sh, "ntfy_health", _probe("ntfy"))
monkeypatch.setattr(sh, "email_health", _probe("email"))
monkeypatch.setattr(sh, "providers_health", _probe("providers"))
out = asyncio.run(sh.collect_service_health(None, None))
assert sorted(probed) == ["email", "ntfy", "searxng"]
assert "providers" not in probed
by_name = {s["name"]: s for s in out["services"]}
assert by_name["providers"]["status"] == sh.DEGRADED
assert by_name["ntfy"]["status"] == sh.OK
def test_source_failure_entry_carries_no_raw_detail(monkeypatch):
import asyncio
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {}, "integrations": [], "accounts": [], "endpoints": [],
"failed": {"accounts": "auth_or_protocol_error"},
})
out = asyncio.run(sh.collect_service_health(None, None))
entry = next(s for s in out["services"] if s["name"] == "email")
assert entry["detail"] == (
"Could not read configuration (configuration source could not be read).")
assert set(entry["meta"]) == {"error", "source", "source_error"}
def test_collect_tolerates_inputs_without_a_failed_key(monkeypatch):
# `_gather_inputs()` is monkeypatched in a number of existing tests (and by
# anything calling the collector with a stub); a payload predating `failed`
# must still probe normally rather than raise.
import asyncio
monkeypatch.setattr(sh, "_gather_inputs", lambda: {
"settings": {"search_provider": "disabled"},
"integrations": [], "accounts": [], "endpoints": [],
})
out = asyncio.run(sh.collect_service_health(_Store(True), _Store(True)))
assert out["overall"] == sh.OK
+42 -1
View File
@@ -8,7 +8,7 @@ from sqlalchemy.orm import sessionmaker
from core.database import Base from core.database import Base
from core.database import ChatMessage as DbChatMessage from core.database import ChatMessage as DbChatMessage
from core.database import Session as DbSession from core.database import Session as DbSession
from src.session_search import SessionSearchResult, search_session_messages from src.session_search import SessionSearchResult, _snippet, search_session_messages
def _db(with_fts=True): def _db(with_fts=True):
@@ -61,6 +61,47 @@ def _has_fts(db):
) )
def test_snippet_locates_quoted_phrase_instead_of_raw_query():
content = "A" * 100 + "The session search result is useful." + "Z" * 100
snippet = _snippet(content, '"session search"', radius=20)
assert "session search" in snippet
assert snippet.startswith("...")
assert snippet.endswith("...")
def test_snippet_locates_punctuation_bearing_term_case_insensitively():
content = "A" * 100 + "Use FOO_BAR.py for routing." + "Z" * 100
snippet = _snippet(content, "foo_bar.py!", radius=20)
assert "FOO_BAR.py" in snippet
def test_snippet_uses_first_meaningful_matching_term():
content = "A" * 100 + "second appears here; first appears later." + "Z" * 100
snippet = _snippet(content, "first second", radius=12)
assert "second appears" in snippet
assert "first appears" not in snippet
def test_snippet_falls_back_to_content_start_when_no_term_matches():
content = "Beginning of the transcript, followed by more detail."
assert _snippet(content, '"missing phrase"', radius=10) == content[:20]
def test_snippet_preserves_boundary_markers_and_truncation():
content = "A" * 50 + "target" + "Z" * 50
snippet = _snippet(content, "target", radius=5)
assert snippet == "...AAAAAtargetZZZZZ..."
def test_session_search_uses_fts_and_returns_context(): def test_session_search_uses_fts_and_returns_context():
db = _db(with_fts=True) db = _db(with_fts=True)
try: try:
@@ -0,0 +1,54 @@
"""Inline frontmatter lists must preserve quoted values across save/load cycles."""
import importlib.util
from pathlib import Path
import sys
import unittest
# Load the dependency-free formatter without starting the services package.
_PATH = Path(__file__).resolve().parents[1] / "services" / "memory" / "skill_format.py"
_SPEC = importlib.util.spec_from_file_location("_skill_list_format_under_test", _PATH)
_FORMAT = importlib.util.module_from_spec(_SPEC)
sys.modules[_SPEC.name] = _FORMAT
_SPEC.loader.exec_module(_FORMAT)
class FrontmatterListEscapesTests(unittest.TestCase):
def test_skill_tags_survive_repeated_save_load_cycles(self):
tags = ['review "release, notes', 'use "search", then summarize', 'publish']
skill = _FORMAT.Skill(name="release-review", tags=tags)
for _ in range(5):
skill = _FORMAT.Skill.from_markdown(skill.to_markdown())
self.assertEqual(skill.tags, tags)
def test_escaped_quotes_preserve_each_list_field(self):
for field in ("tags", "platforms", "requires_toolsets", "fallback_for_toolsets"):
with self.subTest(field=field):
values = ['review "release, notes', 'publish']
skill = _FORMAT.Skill(name="release-review", **{field: values})
restored = _FORMAT.Skill.from_markdown(skill.to_markdown())
self.assertEqual(getattr(restored, field), values)
def test_backslash_parity_does_not_hide_quotes_or_list_separators(self):
for count in range(1, 5):
with self.subTest(backslashes=count):
values = ['review ' + '\\' * count + '"release, notes', 'path, ' + '\\' * count, 'publish']
text = _FORMAT.emit_frontmatter({"tags": values})
parsed, _ = _FORMAT.parse_frontmatter(f"---\n{text}\n---\n")
self.assertEqual(parsed["tags"], values)
def test_single_quoted_backslashes_remain_literal(self):
text = "---\ntags: ['C:\\', next]\n---\n"
parsed, _ = _FORMAT.parse_frontmatter(text)
self.assertEqual(parsed["tags"], ["C:\\", "next"])
def test_nested_lists_preserve_quoted_brackets_and_commas(self):
values = [['review "], notes', 'publish'], 'archive']
text = _FORMAT.emit_frontmatter({"tags": values})
parsed, _ = _FORMAT.parse_frontmatter(f"---\n{text}\n---\n")
self.assertEqual(parsed["tags"], values)
if __name__ == "__main__":
unittest.main()
+83
View File
@@ -0,0 +1,83 @@
"""Renaming a skill onto a name the owner already uses must be refused.
``update_skill`` only checked whether the target directory existed. A
skill in another category with the wanted name lives in a different
directory, so the rename went through and the owner ended up with two
skills sharing one name. Update and delete look skills up by name, so the
next DELETE removed whichever of the two ``os.walk`` reached first, which
was often the one the user did not pick.
"""
import pytest
from routes.skills_routes import SkillUpdateRequest, setup_skills_routes
from services.memory.skills import SkillsManager
from tests.test_skills_routes_owner_update import _request, _route_handler, _write_skill_md
def _two_skills(tmp_path, owner="alice"):
mgr = SkillsManager(str(tmp_path))
mgr.add_skill(name="deploy-app", description="Deploy the web app", category="ops",
source="user", procedure=["ship it"], owner=owner)
mgr.add_skill(name="draft-notes", description="Draft meeting notes", category="writing",
source="user", procedure=["write"], owner=owner)
return mgr
def _names(mgr, owner="alice"):
return sorted((s["name"], s["category"]) for s in mgr.load(owner=owner))
def test_rename_onto_a_name_in_another_category_is_refused(tmp_path):
mgr = _two_skills(tmp_path)
assert mgr.update_skill("draft-notes", {"name": "deploy-app"}, owner="alice") is False
assert _names(mgr) == [("deploy-app", "ops"), ("draft-notes", "writing")]
def test_delete_after_refused_rename_removes_only_the_named_skill(tmp_path):
mgr = _two_skills(tmp_path)
mgr.update_skill("draft-notes", {"name": "deploy-app"}, owner="alice")
assert mgr.delete_skill("draft-notes", owner="alice") is True
assert _names(mgr) == [("deploy-app", "ops")]
def test_rename_to_a_free_name_still_moves_the_skill(tmp_path):
mgr = _two_skills(tmp_path)
assert mgr.update_skill("draft-notes", {"name": "meeting-notes"}, owner="alice") is True
assert _names(mgr) == [("deploy-app", "ops"), ("meeting-notes", "writing")]
def test_another_owners_skill_does_not_block_the_rename(tmp_path):
skills_root = tmp_path / "skills"
_write_skill_md(skills_root, "bob-cat", "caveman-mode", "bob")
_write_skill_md(skills_root, "alice-cat", "terse-mode", "alice")
mgr = SkillsManager(str(tmp_path))
assert mgr.update_skill("terse-mode", {"name": "caveman-mode"}, owner="alice") is True
assert [s["name"] for s in mgr.load(owner="alice")] == ["caveman-mode"]
assert [s["name"] for s in mgr.load(owner="bob")] == ["caveman-mode"]
def test_keeping_the_same_name_is_not_a_conflict(tmp_path):
mgr = _two_skills(tmp_path)
assert mgr.update_skill("draft-notes", {"name": "Draft Notes", "status": "published"},
owner="alice") is True
assert [s["status"] for s in mgr.load(owner="alice") if s["name"] == "draft-notes"] == ["published"]
@pytest.mark.asyncio
async def test_put_route_answers_409_for_a_taken_name(tmp_path):
from fastapi import HTTPException
mgr = _two_skills(tmp_path)
update_route = _route_handler(setup_skills_routes(mgr), "/api/skills/{skill_id}", "PUT")
with pytest.raises(HTTPException) as exc:
await update_route(_request("alice"), "draft-notes", SkillUpdateRequest(name="deploy-app"))
assert exc.value.status_code == 409
assert _names(mgr) == [("deploy-app", "ops"), ("draft-notes", "writing")]
+99
View File
@@ -27,3 +27,102 @@ def test_dsml_tool_markup_strips_to_empty_visible_text():
def test_chunk_split_tool_call_prefix_is_buffered_and_not_persisted(): def test_chunk_split_tool_call_prefix_is_buffered_and_not_persisted():
assert _streamed_tool_markup_starts('I will search<tool_cal') assert _streamed_tool_markup_starts('I will search<tool_cal')
assert _strip_incomplete_tool_markup_tail('I will search<tool_cal') == 'I will search' assert _strip_incomplete_tool_markup_tail('I will search<tool_cal') == 'I will search'
def test_bare_less_than_and_comparison_operators_false_alarm_recognition():
# A bare "<" starts buffering because it could be a tool-call tag prefix like "<tool_call>"
assert _streamed_tool_markup_starts(' <')
assert _streamed_tool_markup_starts('x <')
assert _streamed_tool_markup_starts('<tool_call')
assert _streamed_tool_markup_starts('<invoke')
# Once subsequent characters arrive and cannot become a tool-call tag, false alarm is detected
assert not _streamed_tool_markup_starts(' < 2')
assert not _streamed_tool_markup_starts('x < 2')
assert not _streamed_tool_markup_starts('len(items) < 2')
assert not _streamed_tool_markup_starts(' < 2:')
assert not _streamed_tool_markup_starts('x <= y')
assert not _streamed_tool_markup_starts('a < b')
async def test_stream_agent_loop_does_not_truncate_at_less_than_operator(monkeypatch):
import json
import src.agent_loop as al
async def fake_stream(*args, **kwargs):
for d in ["def is_short(items):\n", " return len(items)", " <", " 2\n"]:
yield "data: " + json.dumps({"delta": d}) + "\n\n"
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "get_setting", lambda k, d=None: d, raising=False)
monkeypatch.setattr(al, "get_mcp_manager", lambda: None, raising=False)
monkeypatch.setattr(al, "estimate_tokens", lambda *a, **k: 10, raising=False)
monkeypatch.setattr(al, "stream_llm_with_fallback", fake_stream, raising=False)
deltas = []
async for chunk in al.stream_agent_loop(
"http://local.test/v1",
"qwen3.6-35b-a3b-heretic",
[{"role": "user", "content": "Write python function"}],
max_rounds=1,
owner="test_user",
):
for line in chunk.split("\n"):
line = line.strip()
if line.startswith("data: ") and line[6:] != "[DONE]":
try:
payload = json.loads(line[6:])
if "delta" in payload:
deltas.append(payload["delta"])
except Exception:
pass
full_text = "".join(deltas)
assert full_text == "def is_short(items):\n return len(items) < 2\n"
async def test_stream_agent_loop_still_hides_real_tool_markup(monkeypatch):
import json
import src.agent_loop as al
async def fake_stream(*args, **kwargs):
chunks = [
"Sure.",
" <",
"tool",
"_call",
'>{"name":"host_shell","arguments":{"command":"pwd"}}',
"</",
"tool",
"_call",
">",
" done",
]
for c in chunks:
yield "data: " + json.dumps({"delta": c}) + "\n\n"
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "get_setting", lambda k, d=None: d, raising=False)
monkeypatch.setattr(al, "get_mcp_manager", lambda: None, raising=False)
monkeypatch.setattr(al, "estimate_tokens", lambda *a, **k: 10, raising=False)
monkeypatch.setattr(al, "stream_llm_with_fallback", fake_stream, raising=False)
deltas = []
async for chunk in al.stream_agent_loop(
"http://local.test/v1",
"qwen3.6-35b-a3b-heretic",
[{"role": "user", "content": "run command"}],
max_rounds=1,
owner="test_user",
):
for line in chunk.split("\n"):
line = line.strip()
if line.startswith("data: ") and line[6:] != "[DONE]":
try:
payload = json.loads(line[6:])
if "delta" in payload:
deltas.append(payload["delta"])
except Exception:
pass
assert "".join(deltas) == "Sure. done"
+132
View File
@@ -0,0 +1,132 @@
"""Issue #5573 — the vision model's description must not read as the user's own words.
For a text-only chat model the description comes from a separate vision call and
is appended to the user's message. Unlabelled, the chat model answers the prose
instead of the image — reviewing the user's "writing" back to them.
"""
import importlib
from types import SimpleNamespace
import pytest
DESCRIPTION = "A chocolate cake, plated with a mirror glaze and gold leaf."
QUESTION = "What am I looking at?"
class _UploadHandler:
def __init__(self, file_info):
self._file_info = file_info
def resolve_upload(self, att_id, owner=None):
return self._file_info if att_id == self._file_info["id"] else None
def is_image_file(self, *_args, **_kwargs):
return True
def _no_vision_call(*_args, **_kwargs):
raise AssertionError("the vision model must not be called on this path")
@pytest.fixture
def vision_env(monkeypatch, tmp_path):
"""Text-only chat model with one image attachment, no DB and no real vision call.
Other tests drop src.chat_handler from sys.modules, so resolve it here rather
than at import time — otherwise the patches land on a different module object
than the one under test.
"""
module = importlib.import_module("src.chat_handler")
monkeypatch.setattr(module, "UPLOAD_DIR", str(tmp_path))
monkeypatch.setattr(module, "model_supports_vision", lambda *a, **k: False)
monkeypatch.setattr(module, "_sync_upload_vision_to_gallery", lambda *a, **k: None)
monkeypatch.setattr(module, "build_user_content", lambda text, *a, **k: text)
monkeypatch.setattr(
importlib.import_module("src.settings"), "get_setting", lambda key, default=None: default
)
return SimpleNamespace(module=module, tmp_path=tmp_path, file_info={
"id": "img-1", "name": "cake.jpg", "path": str(tmp_path / "cake.jpg"),
"mime": "image/jpeg", "size": 1, "hash": "h",
})
async def _preprocess(env):
handler = env.module.ChatHandler(
session_manager=None, memory_manager=None, chat_processor=None,
research_handler=None, preset_manager=None,
upload_handler=_UploadHandler(env.file_info),
)
sess = SimpleNamespace(model="chat-model", endpoint_url="", owner="user", id="s1")
enhanced, *_ = await handler.preprocess_message(
QUESTION, ["img-1"], sess, auto_opened_docs=[],
)
return enhanced
def _describe(env, monkeypatch, text):
monkeypatch.setattr(env.module, "analyze_image_with_vl_result",
lambda *a, **k: {"text": text, "model": "vl-model"})
def _label_above(enhanced, description):
"""The line the description is introduced by."""
return enhanced.split(description)[0].rstrip("\n").rsplit("\n", 1)[-1]
@pytest.mark.asyncio
async def test_description_is_marked_as_not_the_users_message(vision_env, monkeypatch):
_describe(vision_env, monkeypatch, DESCRIPTION)
enhanced = await _preprocess(vision_env)
assert QUESTION in enhanced, "the user's own words must survive untouched"
label = _label_above(enhanced, DESCRIPTION)
assert label.startswith("[") and "user" in label.lower(), (
"the description is appended as bare prose, so the chat model reads it as "
f"the user's own writing. Got:\n{enhanced}"
)
@pytest.mark.asyncio
async def test_cached_description_is_labelled_too(vision_env, monkeypatch):
"""The cache holds an auto description or a hand-corrected one, and the label
has to be true either way — it is a description of the image regardless."""
cache = vision_env.tmp_path / ".vision"
cache.mkdir()
(cache / "img-1.txt").write_text(DESCRIPTION, encoding="utf-8")
monkeypatch.setattr(vision_env.module, "analyze_image_with_vl_result", _no_vision_call)
enhanced = await _preprocess(vision_env)
label = _label_above(enhanced, DESCRIPTION)
assert label.startswith("[") and "user" in label.lower(), (
f"a cached description is injected with no label. Got:\n{enhanced}"
)
@pytest.mark.asyncio
async def test_client_side_image_marker_is_preserved(vision_env, monkeypatch):
"""chatRenderer.js and sessions.js strip the block by matching `[Image: <name>]\\n`.
Renaming that marker leaves the raw description rendered in the user's bubble.
"""
_describe(vision_env, monkeypatch, DESCRIPTION)
assert "[Image: cake.jpg]\n" in await _preprocess(vision_env)
@pytest.mark.asyncio
async def test_no_dangling_label_when_the_vision_model_returns_nothing(vision_env, monkeypatch):
_describe(vision_env, monkeypatch, "")
enhanced = await _preprocess(vision_env)
assert enhanced.endswith("[Image: cake.jpg]\n"), (
"an empty description must leave the bare marker, trailing newline included "
f"— the client's strip regex needs it. Got:\n{enhanced!r}"
)
@pytest.mark.asyncio
async def test_vision_capable_model_path_is_unchanged(vision_env, monkeypatch):
monkeypatch.setattr(vision_env.module, "model_supports_vision", lambda *a, **k: True)
monkeypatch.setattr(vision_env.module, "analyze_image_with_vl_result", _no_vision_call)
assert await _preprocess(vision_env) == f"{QUESTION}\n\n[Image attached: cake.jpg]"