Compare commits

...
8 Commits
Author SHA1 Message Date
Maruf HasanandKenny Van de Maele c3fcaf15b7 feat(providers): add NVIDIA AI provider endpoint support (#3456)
* feat: add NVIDIA as an AI provider (integrate.api.nvidia.com)

* feat: add NVIDIA option to provider settings dropdown and aliases

* test: add NVIDIA provider detection and endpoint tests

* Add NVIDIA to _HOST_TO_CURATED and expand non-chat model filtering

- nvidia.com -> 'nvidia' curated key for proper provider routing
- _NON_CHAT_PREFIXES: bge, snowflake/arctic-embed, nvidia/nv-embed
- _NON_CHAT_CONTAINS: content-safety, -safety, -reward, nvclip,
  kosmos, fuyu, deplot, vila, neva, gliner, riva, -parse,
  -embedqa, -nemoretriever

* Expand non-chat model filtering for NVIDIA embedding/guard/video models

Add _NON_CHAT_PREFIXES: embed, recurrent
Add _NON_CHAT_CONTAINS: topic-control, guard, calibration,
  ai-synthetic-video, cosmos-reason2

Catches remaining unfiltered non-chat models from NVIDIA catalog:
embedding (llama-nemotron-embed, embed-qa), guard (llama-guard,
nemoguard-topic-control), calibration (ising-calibration),
video (ai-synthetic-video-detector, cosmos-reason2),
recurrent (recurrentgemma-2b)

* Filter non-chat models in _probe_endpoint via _is_chat_model()

Previously _is_chat_model() was only used in the per-model probe
and _first_chat_model(), so non-chat models still appeared in the
model picker even though they were filtered in those specific paths.
Applying the filter at _probe_endpoint() return ensures non-chat
models (embeddings, safety guards, reward, calibration, video
detectors, CLIP, VLM, translation, parsing, recurrent, etc.) never
enter cached_models and never appear in the picker.

* Fix _NON_CHAT_CONTAINS to catch org-prefixed embedding models

Prefix checks (mid.startswith) miss models with org prefixes like
baai/bge-m3, nvidia/embed-qa-4, google/recurrentgemma-2b, etc.
Adding the same terms to _NON_CHAT_CONTAINS ensures they are caught
regardless of the org prefix.

Adds: embed, bge, recurrent, starcoder, gemma-2b

* fix(model-routes): drop collision-prone substrings from global non-chat filter

The NVIDIA PR added several substrings to the shared _NON_CHAT_PREFIXES
and _NON_CHAT_CONTAINS tuples. These are intended to filter out
embedding, retrieval, safety, and vision models from NVIDIA's catalog
that are not chat-completions-capable. However, four of the added
substrings collide with legitimate chat models served by other providers:

  - gemma-2b  matches google/gemma-2b-it (instruct chat model)
  - starcoder matches bigcode/starcoder2-15b (code completion model)
  - recurrent matches google/recurrentgemma-2b (language model)
  - guard     matches meta-llama/Llama-Guard-3-8B (safety classifier)

Removing these four from the global tuples keeps the NVIDIA-specific
filtering intact (safety, embedding, retrieval, and vision models are
still caught by other tokens such as content-safety, -safety, -reward,
embed, bge, -embedqa, -nemoretriever, nvclip, deplot, etc.) while
preventing false negatives for instruct/code models on other providers.

Tests added for gemma-2b-it, google/gemma-2b-it, and
bigcode/starcoder2-15b-instruct asserting they are recognized as chat
models.

Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be>

* fix(nvidia): remove duplicate bge/embed tokens from _NON_CHAT_CONTAINS

Tokens already present in _NON_CHAT_PREFIXES, making the CONTAINS
entries redundant since the prefix check runs first.

Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be>

* fix(nvidia): move bge to CONTAINS, add llama-guard, remove stray blanks

Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be>

* style: fix indentation of groq and xai test cases in test_provider_endpoints.py

---------

Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be>
2026-06-09 11:06:12 +02:00
Mazen Tamer Salah 3c4ec8828b fix(embeddings): survive numpy embeddings when restoring a reset lane (#3410)
When a lane reset fails to rewrite the recreated collection, the recovery path
re-adds the preserved rows. It read the embeddings with
`preserved.get("embeddings") or []` and gated the loop with
`if ids and docs and old_embeddings:`. chromadb returns embeddings as a numpy
ndarray, whose truth value is ambiguous, so both expressions raise ValueError
inside the except block — the restore is abandoned and every preserved row is
lost (the collection was already deleted), exactly when the code is trying to
avoid data loss.

Use an explicit `is None` check and `len(...)`, and convert ndarray batches to
lists before re-adding.

Adds tests/test_embedding_lane_ndarray_restore.py (preserved embeddings come
back as np.ndarray); existing test_embedding_lanes.py still passes.
2026-06-09 10:40:17 +02:00
Ashvin 2fdb4813db fix(auth): sync file-backed and in-memory owner caches on user rename (#3397)
The DB owner-rename loop in rename_user patched every SQL column named
owner, but three non-SQL stores were left behind:

1. session_manager.sessions -- in-memory Session objects carry s.owner
   set at server-boot time. get_sessions_for_user() does an exact
   s.owner == username check, so the renamed user chat sidebar goes empty
   until a server restart.

2. data/deep_research/*.json -- each completed research report is a
   standalone JSON file with an owner field. research_routes filters
   by d.get(owner) == user, making every report invisible to the
   renamed user.

3. data/memory.json -- a flat JSON array; each entry carries an owner
   field. memory_manager.load(owner=user) filters on it, so all memories
   vanish from the memory panel.

Fix: after the SQL loop, patch all three:
- iterate sm.sessions and update owner in-place (exposed via app.state)
- walk data/deep_research/*.json and rewrite owner with atomic_write_json
- update matching entries in memory.json with atomic_write_json

All three use the same case-insensitive lower() comparison the SQL loop
already uses. Each step is independently wrapped so a single failure
does not abort the others or the rename itself.

Fixes #3362
2026-06-09 10:19:45 +02:00
nubsandKenny Van de Maele f1cda91683 fix(agent): scope skill index to owner (#2404)
Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be>
2026-06-09 09:51:29 +02:00
Kenny Van de Maele 0aba00f4cf refactor(tools): remove dead workspace-confinement plumbing (#3590)
Commit e6b1009 removed the workspace feature's entry point (deleted
routes/workspace_routes.py + static/js/workspace.js and dropped the
workspace-param parsing in chat_routes), but left the downstream backend
plumbing dangling: chat_routes passed a hardcoded workspace=None into
stream_agent_loop, which forwarded it to execute_tool_block, so the
workspace value was permanently None and every workspace-gated branch
was unreachable.

Remove the now-dead code (no behavior change, since workspace was always
None):
- src/tool_execution.py: drop _resolve_tool_path_in_workspace and the
  workspace params/branches on execute_tool_block, _direct_fallback,
  _call_mcp_tool, _do_edit_file, and _resolve_search_root; restore the
  bash/python/bg cwd to _AGENT_WORKDIR.
- src/agent_loop.py: drop the workspace param on stream_agent_loop, the
  dead 'ACTIVE WORKSPACE' system-prompt block, and the workspace forward.
- routes/chat_routes.py: drop the hardcoded workspace=None arg and var.
- tests: delete test_workspace_confine.py (tested the removed feature) and
  the workspace assertion in test_tool_policy.py.

Full suite: 2903 passed, 1 skipped.
2026-06-09 08:30:50 +02:00
Afonso Coutinho fbed9027b0 fix: backup import dropping a user's skill on cross-tenant title/id collision (#2057)
* Fix backup import dropping a user's skill on cross-tenant title/id collision

The skills block of import_data deduped incoming skills against
skills_manager.load_all(), which returns EVERY tenant's skills. So when
a user imports their own backup, any skill whose id or title collides
with another user's skill was silently skipped — the importing user
lost their own data. This is the same cross-tenant bug already fixed
for the memories block just above (#1743); the skills block was left
with the old pattern. Filter the dedup sets to the importing user's own
skills (owner == user); the full store is still saved back, preserving
other users' skills.

* Restore sys.modules after stubbing so backup test does not break collection of later src.* test modules

* Patch backup_routes auth helpers via monkeypatch instead of sys.modules stubs so the test is import-order robust

* Give FakeSkillsManager an add_skill method matching the disk-backed skills API
2026-06-09 08:04:22 +02:00
Disorder AAandClaude Opus 4.8 d9141c6e56 fix(cookbook): allow spaces and non-ASCII characters in model directory paths (#3473)
* fix(cookbook): allow spaces in model directory paths

Allow POSIX external-drive paths and Windows drive paths with spaces while keeping shell metacharacters rejected.

* fix(cookbook): also allow non-ASCII (Unicode) characters in model dir paths

The ASCII-only allowlist that rejected spaces also rejected Cyrillic,
accented Latin and CJK folder names (e.g. /Volumes/Модели,
D:\AI Models\Модели) with 400 Invalid local_dir. Switch the path
character class from [A-Za-z0-9._ -] to [\w. -] (\w is Unicode-aware on
Python 3 str patterns) so localized folder names validate, while shell
metacharacters (; & | ` $ quotes newlines) stay rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cookbook): reject local_dir path segments starting with '-'

The local_dir allowlist includes '-', so a directory like /models/-rf
(or D:\models\-rf) could be parsed as a CLI flag by hf/etc. (option
injection) — and quoting does not stop a value from being read as an
option. Guard against it inside the validator so the safety stays fully
self-contained there rather than depending on consumers' quoting.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 07:58:38 +02:00
onemorethan0 8ae2b5f58c fix(llm): suppress thinking mode for qwen3/gemma4 on Ollama /v1 endpoint (#3228)
* fix(llm): suppress thinking for qwen3/gemma4 on Ollama /v1 compat endpoint

When using qwen3, QwQ, gemma4, or other thinking models via Ollama's
OpenAI-compatible /v1 endpoint, the model routes all output into its
<think>...</think> reasoning block. Since Odysseus strips thinking
content from round_response and only accumulates native tool_calls,
this produces a round with 0 chars, 0 native calls, 0 tool blocks —
the agent appears to silently do nothing.

Root cause: Odysseus classifies the /v1 endpoint as provider="openai"
(not "ollama"), so the payload is built as a standard OpenAI payload
without any Ollama-specific options. Ollama's /v1 endpoint accepts
"think": false as a top-level parameter to suppress extended thinking,
but this was never sent.

Fix:
- Add _is_ollama_openai_compat_url() to detect local Ollama /v1 URLs
- Inject "think": false in both stream_llm and llm_call_async for
  thinking models (qwen3, QwQ, gemma4, DeepSeek-R1, etc.) on this
  endpoint

Verified with qwen3:14b on Ollama 0.24: with think=False the model
correctly emits native tool_calls in a single streaming chunk and
the agent executes bash/file/web tools as expected.

* fix(llm): extend _is_ollama_openai_compat_url to match localhost on any port

Per reviewer feedback on PR #3228:

1. Generalize host detection to mirror _is_ollama_native_url: match any
   localhost/127.0.0.1/0.0.0.0/::1 host (not just port 11434) so that
   custom OLLAMA_HOST ports and container remaps are also covered.

2. Add tests/test_llm_core_ollama_thinking.py covering:
   - _is_ollama_openai_compat_url for all positive/negative URL cases
     including IPv6, non-default port, native /api path, and real OpenAI
   - Payload injection: think:false set for Ollama /v1 thinking model,
     not set for non-thinking model, not set for real OpenAI endpoint,
     and set for localhost on a non-default port (the new case)
2026-06-09 07:35:15 +02:00
24 changed files with 1092 additions and 252 deletions
+1
View File
@@ -472,6 +472,7 @@ components = initialize_managers(BASE_DIR, rag_manager)
session_manager = components["session_manager"]
from src.assistant_log import set_session_manager as _set_asst_sm
_set_asst_sm(session_manager)
app.state.session_manager = session_manager
memory_manager = components["memory_manager"]
memory_vector = components.get("memory_vector")
upload_handler = components["upload_handler"]
+100 -5
View File
@@ -7,7 +7,13 @@ import asyncio
import logging
import os
import json
import re
from pathlib import Path
from core.atomic_io import atomic_write_json, atomic_write_text
from core.auth import AuthManager
from src.constants import DEEP_RESEARCH_DIR, MEMORY_FILE, SKILLS_DIR
from src.rate_limiter import RateLimiter
from src.settings_scrub import scrub_settings
from src.settings import (
@@ -291,9 +297,17 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
if new_username in auth_manager.users:
raise HTTPException(409, "Username already taken")
# Gate on auth first. Every mutation below is contingent on this
# succeeding — doing it last meant a rejected rename (e.g. reserved
# username) left file-backed owner fields already rewritten with no
# way to roll them back.
ok = auth_manager.rename_user(old_username, new_username, user)
if not ok:
raise HTTPException(400, "Cannot rename user")
# Usernames are ownership keys for user data. Rename the common
# owner-scoped DB rows before changing auth so the account keeps
# access to its sessions, docs, email accounts, tasks, etc.
# owner-scoped DB rows so the account keeps access to its sessions,
# docs, email accounts, tasks, etc.
try:
from sqlalchemy import func
from core.database import Base, SessionLocal
@@ -335,9 +349,90 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
except Exception as e:
logger.warning("Failed to rename user prefs %s -> %s: %s", old_username, new_username, e)
ok = auth_manager.rename_user(old_username, new_username, user)
if not ok:
raise HTTPException(400, "Cannot rename user")
# deep_research: each completed report is a standalone JSON file with
# an `owner` field. research_routes filters by d.get("owner") == user,
# so a stale owner makes every report invisible to the renamed user.
try:
dr_dir = Path(DEEP_RESEARCH_DIR)
if dr_dir.is_dir():
for p in dr_dir.glob("*.json"):
try:
d = json.loads(p.read_text(encoding="utf-8"))
if str(d.get("owner", "")).strip().lower() == old_username:
d["owner"] = new_username
atomic_write_json(str(p), d)
except Exception as err:
logger.warning("Failed to update research owner in %s: %s", p.name, err)
except Exception as e:
logger.warning("Failed to rename research owner references %s -> %s: %s", old_username, new_username, e)
# memory.json: a flat JSON array where each entry carries an `owner`
# field. memory_manager.load(owner=user) filters on it, so stale
# entries disappear from the memory panel.
try:
if os.path.isfile(MEMORY_FILE):
with open(MEMORY_FILE, encoding="utf-8") as fh:
entries = json.loads(fh.read())
if isinstance(entries, list):
changed = False
for entry in entries:
if isinstance(entry, dict) and str(entry.get("owner", "")).strip().lower() == old_username:
entry["owner"] = new_username
changed = True
if changed:
atomic_write_json(MEMORY_FILE, entries)
except Exception as e:
logger.warning("Failed to rename memory.json owner references %s -> %s: %s", old_username, new_username, e)
# skills: SKILL.md frontmatter carries owner: <username>; the usage
# sidecar (_usage.json) keys entries as owner::skill-name. Both must
# be updated or the renamed user's Skills panel goes empty.
try:
skills_root = Path(SKILLS_DIR)
if skills_root.is_dir():
_owner_re = re.compile(
r'(?m)^(owner:\s*)' + re.escape(old_username) + r'\s*$'
)
for p in skills_root.rglob("SKILL.md"):
try:
text = p.read_text(encoding="utf-8")
new_text = _owner_re.sub(r'\g<1>' + new_username, text)
if new_text != text:
atomic_write_text(str(p), new_text)
except Exception as err:
logger.warning("Failed to update skill owner in %s: %s", p, err)
usage_path = skills_root / "_usage.json"
if usage_path.is_file():
try:
usage = json.loads(usage_path.read_text(encoding="utf-8"))
if isinstance(usage, dict):
prefix = old_username + "::"
new_usage = {}
changed = False
for k, v in usage.items():
if k.startswith(prefix):
new_usage[new_username + "::" + k[len(prefix):]] = v
changed = True
else:
new_usage[k] = v
if changed:
atomic_write_json(str(usage_path), new_usage)
except Exception as err:
logger.warning("Failed to update skills usage keys %s -> %s: %s", old_username, new_username, err)
except Exception as e:
logger.warning("Failed to rename skills owner references %s -> %s: %s", old_username, new_username, e)
# The in-memory session cache (session_manager.sessions) stores each
# session's owner at load time. Without this patch the renamed user's
# sessions are invisible on the next /api/sessions call because
# get_sessions_for_user does an exact `s.owner == username` comparison
# against stale in-memory values.
sm = getattr(request.app.state, "session_manager", None)
if sm is not None:
for sess in list(getattr(sm, "sessions", {}).values()):
if str(getattr(sess, "owner", None) or "").strip().lower() == old_username:
sess.owner = new_username
# The owner-rename loop above updated ApiToken.owner in the DB, but the
# bearer-token cache still maps each token to the OLD owner. Without
# refreshing it, the renamed user's API tokens resolve to the old (now
+9 -3
View File
@@ -101,11 +101,17 @@ def setup_backup_routes(memory_manager, preset_manager, skills_manager) -> APIRo
# ── Skills ──
if "skills" in body and isinstance(body["skills"], list):
existing = skills_manager.load_all()
existing_names = {s.get("name") for s in existing if s.get("name")}
existing_ids = {s.get("id") for s in existing if s.get("id")}
# Dedup against THIS user's own skills only. Using every tenant's
# rows (load_all) meant a skill whose id/name/title matched any
# other user's was silently skipped, so the importing user lost
# their own data — same cross-tenant bug fixed for memories above.
# The full store is still saved back below.
own = [s for s in existing if s.get("owner") == user]
existing_names = {s.get("name") for s in own if s.get("name")}
existing_ids = {s.get("id") for s in own if s.get("id")}
existing_titles = {
(s.get("title") or s.get("description") or "").strip().lower()
for s in existing
for s in own
}
added = 0
for skill in body["skills"]:
-2
View File
@@ -456,7 +456,6 @@ def setup_chat_routes(
# manual form posts that still send plan_mode=true.
plan_mode = False
chat_mode = str(form_data.get("mode", "")).lower() # 'chat' or 'agent'
workspace = ""
# Plan mode is a modifier on agent mode — it only makes sense with tools.
if plan_mode:
chat_mode = "agent"
@@ -1135,7 +1134,6 @@ def setup_chat_routes(
tool_policy=tool_policy,
owner=_user,
fallbacks=_fallback_candidates,
workspace=None,
plan_mode=plan_mode,
approved_plan=approved_plan or None,
):
+24 -6
View File
@@ -42,9 +42,16 @@ _SESSION_ID_RE = re.compile(r"^[A-Za-z0-9_-]{1,64}$")
_SSH_PORT_RE = re.compile(r"^\d{1,5}$")
_GPU_LIST_RE = re.compile(r"^\d+(?:,\d+)*$")
# A download target directory. Absolute or ~-relative path; safe path glyphs
# only (no quotes, shell metacharacters, or spaces) since it lands in a shell
# command. A leading ~ is expanded to $HOME at command-build time.
_LOCAL_DIR_RE = re.compile(r"^~?/[A-Za-z0-9._/-]*$|^~$")
# only (no quotes or shell metacharacters). Spaces are allowed because command
# builders pass the value through quoted shell/Python contexts. The character
# class uses ``\w`` — Unicode word characters under Python 3's default str
# matching — so non-ASCII folder names pass validation too: Cyrillic, accented
# Latin, CJK, e.g. ``/Volumes/Модели`` or ``D:\AI Models\Модели``. This stays
# shell-safe: none of ``; & | ` $ '' "" () {}`` newlines etc. are in ``[\w. -]``,
# so injection vectors remain rejected. A leading ~ is expanded to $HOME at
# command-build time. (Drive letters stay ASCII: ``[A-Za-z]:``.)
_LOCAL_DIR_RE = re.compile(r"^~?(?:/[\w. -]*)+$|^~$")
_WINDOWS_LOCAL_DIR_RE = re.compile(r"^[A-Za-z]:[\\/](?:[\w. -]+(?:[\\/][\w. -]+)*[\\/]?)?$")
_WINDOWS_DRIVE_PATH_RE = re.compile(r"^[A-Za-z]:[\\/]")
@@ -97,9 +104,19 @@ def _validate_token(v: str | None) -> str | None:
def _validate_local_dir(v: str | None) -> str | None:
if v is None or v == "":
return None
if len(v) >= 2 and v[0] == v[-1] and v[0] in {"'", '"'}:
v = v[1:-1]
v = v.rstrip("/") or "/"
if not _LOCAL_DIR_RE.match(v):
raise HTTPException(400, "Invalid local_dir — must be an absolute or ~ path with no spaces or shell metacharacters")
if not (_LOCAL_DIR_RE.match(v) or _WINDOWS_LOCAL_DIR_RE.match(v)):
raise HTTPException(400, "Invalid local_dir — must be an absolute or ~ path with no shell metacharacters")
# Reject path segments that start with '-' (option injection). '-' is in the
# allowlist, so a dir like ``/models/-rf`` or ``D:\models\-rf`` could be read
# as a CLI flag by hf/etc. — and quoting does NOT stop a value from being
# parsed as an option. This is the one residual that command-build-time
# quoting can't cover, so the guard lives here, keeping the safety wholly
# inside the validator rather than relying on consumers.
if any(seg.startswith("-") for seg in re.split(r"[\\/]", v) if seg):
raise HTTPException(400, "Invalid local_dir — path segments cannot start with '-'")
return v
@@ -125,7 +142,7 @@ def _validate_gpus(v: str | None) -> str | None:
def _shell_path(p: str) -> str:
"""Render a validated path for a double-quoted shell context, expanding a
leading ~ to $HOME (single quotes wouldn't expand it). Safe because
_validate_local_dir already restricts the charset."""
_validate_local_dir already rejects quotes and shell metacharacters."""
if p == "~":
return '"$HOME"'
if p.startswith("~/"):
@@ -386,6 +403,7 @@ def _cached_model_scan_script(model_dirs: list[str] | None = None, add_hf_cache:
" for root, dirs, fns in safe_walk(base):",
" for fn in sorted(fns):",
" if not fn.lower().endswith('.gguf'): continue",
" if fn.startswith('._'): continue # macOS AppleDouble sidecar, not a real GGUF",
" fp = os.path.join(root, fn)",
" try: size = os.path.getsize(fp)",
" except Exception: size = 0",
+11 -3
View File
@@ -283,6 +283,7 @@ _HOST_TO_CURATED = (
("fireworks.ai", "fireworks"),
("googleapis.com", "google"),
("x.ai", "xai"),
("nvidia.com", "nvidia"),
("openrouter.ai", "openrouter"),
("ollama.com", "ollama"),
)
@@ -477,10 +478,17 @@ _NON_CHAT_PREFIXES = (
"dall-e", "tts-", "whisper", "text-embedding", "embedding",
"davinci", "babbage", "moderation", "omni-moderation",
"sora", "gpt-image", "chatgpt-image",
# embedding / retrieval / non-chat models (common across providers)
"snowflake/arctic-embed", "nvidia/nv-embed", "embed",
)
_NON_CHAT_CONTAINS = (
"-realtime", "-transcribe", "-tts", "-codex",
"codex-",
"codex-", "content-safety", "-safety", "-reward", "nvclip",
"kosmos", "fuyu", "deplot", "vila", "neva",
"gliner", "riva", "-parse", "-embedqa", "-nemoretriever",
"topic-control", "calibration",
"ai-synthetic-video", "cosmos-reason2",
"bge", "llama-guard",
)
_NON_CHAT_EXACT_PREFIXES = (
"gpt-audio", # gpt-audio, gpt-audio-mini etc. (not gpt-4o-audio-preview which is chat)
@@ -731,7 +739,7 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
for _e in _PROVIDER_CURATED.get(_ck, []):
if _e not in set(models) and not any(m.startswith(_e) for m in models):
models.append(_e)
return models
return [m for m in models if _is_chat_model(m)]
except httpx.HTTPStatusError as e:
if api_key:
status = e.response.status_code if e.response is not None else "unknown"
@@ -755,7 +763,7 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
data = r.json()
models = [m.get("name") or m.get("model") for m in (data.get("models") or []) if m.get("name") or m.get("model")]
if models:
return models
return [m for m in models if _is_chat_model(m)]
except Exception as e:
logger.debug(f"Ollama /api/tags probe failed for {base}: {e}")
# Fall back to curated list if the provider has a URL-based match (e.g. z.ai has no /models endpoint)
+5 -26
View File
@@ -855,7 +855,7 @@ def _build_system_prompt(
_ov_sig = _hl.sha256(_json.dumps(get_builtin_overrides() or {}, sort_keys=True).encode()).hexdigest()
except Exception:
_ov_sig = ""
cache_key = (frozenset(disabled_tools or []), bool(mcp_mgr), needs_admin, _rt_key, compact, _ov_sig, suppress_local_context)
cache_key = (frozenset(disabled_tools or []), bool(mcp_mgr), needs_admin, _rt_key, compact, _ov_sig, owner, suppress_local_context)
if _cached_base_prompt and _cached_base_prompt_key == cache_key and not active_document:
agent_prompt = _cached_base_prompt
# Skill index is user-editable (name + description), so it must never
@@ -863,7 +863,7 @@ def _build_system_prompt(
# when the cache hits.
_, _skill_index_block = _build_base_prompt(
disabled_tools, mcp_mgr, needs_admin, relevant_tools,
mcp_disabled_map=mcp_disabled_map, compact=compact,
mcp_disabled_map=mcp_disabled_map, compact=compact, owner=owner,
suppress_local_context=suppress_local_context,
)
else:
@@ -874,6 +874,7 @@ def _build_system_prompt(
relevant_tools,
mcp_disabled_map=mcp_disabled_map,
compact=compact,
owner=owner,
suppress_local_context=suppress_local_context,
)
if not active_document:
@@ -1246,6 +1247,7 @@ def _build_base_prompt(
relevant_tools=None,
mcp_disabled_map=None,
compact: bool = False,
owner: Optional[str] = None,
suppress_local_context: bool = False,
):
"""Build the agent prompt with only relevant tools included.
@@ -1299,7 +1301,7 @@ def _build_base_prompt(
from src.constants import DATA_DIR
_sm = SkillsManager(DATA_DIR)
active_tools = list(set(TOOL_SECTIONS.keys()) - set(disabled or []))
skill_idx = _sm.index_for(owner=None, active_toolsets=active_tools)
skill_idx = _sm.index_for(owner=owner, active_toolsets=active_tools)
if skill_idx:
lines = ["## Available skills",
"Procedures the assistant should consult before doing domain work. "
@@ -1707,7 +1709,6 @@ async def stream_agent_loop(
owner: Optional[str] = None,
relevant_tools: Optional[Set[str]] = None,
fallbacks: Optional[List[tuple]] = None,
workspace: Optional[str] = None,
plan_mode: bool = False,
approved_plan: Optional[str] = None,
tool_policy: Optional[ToolPolicy] = None,
@@ -1935,27 +1936,6 @@ async def stream_agent_loop(
owner=owner,
suppress_local_context=guide_only,
)
if workspace and not guide_only:
# PREPEND (not append) so it dominates the large base prompt — appended
# at the end, small models ignored it and asked the user for code. The
# folder IS the project; the agent must explore it, not ask.
_ws_note = (
f"## ACTIVE WORKSPACE — READ FIRST\n"
f"The user is working in this folder: {workspace}\n"
f"It IS the project. bash/python run with cwd set here and "
f"read_file/write_file are confined to it (paths outside are rejected).\n"
f"When the user says \"the code\" / \"this project\" / \"the workspace\" "
f"or asks to review/find/edit something WITHOUT a path, they mean THIS "
f"folder. Do NOT ask the user for code or a path, and do NOT read a file "
f"literally named \"workspace\". ALWAYS start by exploring it yourself: "
f"run `bash` → `git ls-files` (or `ls -R`) to see the files, then "
f"read_file the relevant ones by path RELATIVE to the workspace."
)
if messages and messages[0].get("role") == "system":
messages[0]["content"] = _ws_note + "\n\n" + (messages[0].get("content") or "")
else:
messages.insert(0, {"role": "system", "content": _ws_note})
logger.info("[workspace] active for this turn: %s", workspace)
if plan_mode and not guide_only:
# Steer the model to investigate-then-propose. Hard tool gating handles
# every write path except shell; this directive is what keeps the
@@ -2649,7 +2629,6 @@ async def stream_agent_loop(
tool_policy=tool_policy,
owner=owner,
progress_cb=_push_progress,
workspace=workspace,
)
finally:
# Sentinel so the drainer knows to stop.
+11 -2
View File
@@ -196,13 +196,22 @@ def _get_or_reset_collection(chroma_client, name: str, metadata: Dict[str, Any],
try:
chroma_client.delete_collection(name)
restored = chroma_client.get_or_create_collection(name=name, metadata=current)
old_embeddings = preserved.get("embeddings") or []
if ids and docs and old_embeddings:
# chromadb returns embeddings as a numpy ndarray, whose truth value
# is ambiguous — `preserved.get("embeddings") or []` and a bare
# `if ... and old_embeddings:` both raise ValueError, which aborts
# the restore and loses the rows the reset was supposed to keep.
# Use explicit None/len checks instead.
old_embeddings = preserved.get("embeddings")
if old_embeddings is None:
old_embeddings = []
if ids and docs and len(old_embeddings):
for start in range(0, len(ids), 100):
batch_ids = ids[start:start + 100]
batch_docs = docs[start:start + 100]
batch_metas = metas[start:start + 100]
batch_embeddings = old_embeddings[start:start + 100]
if hasattr(batch_embeddings, "tolist"):
batch_embeddings = batch_embeddings.tolist()
if len(batch_metas) < len(batch_ids):
batch_metas += [{}] * (len(batch_ids) - len(batch_metas))
restored.add(
+29
View File
@@ -276,6 +276,24 @@ def _is_ollama_native_url(url: str) -> bool:
return local_ollama_host and (path == "" or path == "/api" or path.startswith("/api/"))
def _is_ollama_openai_compat_url(url: str) -> bool:
"""Return True for local Ollama's OpenAI-compatible /v1 surface.
Mirrors the host detection used by ``_is_ollama_native_url`` so that the
two helpers stay in lockstep: a localhost Ollama on a non-default port
(custom ``OLLAMA_HOST``, reverse proxy, container port remap) is treated
the same way here as it is on the native ``/api`` path.
"""
try:
parsed = urlparse(url or "")
except Exception:
return False
host = parsed.hostname or ""
path = (parsed.path or "").rstrip("/")
local_ollama_host = host in {"localhost", "127.0.0.1", "0.0.0.0", "::1"} or parsed.port == 11434
return local_ollama_host and (path == "/v1" or path.startswith("/v1/"))
def _ollama_api_root(url: str) -> str:
"""Return a native Ollama API root such as https://ollama.com/api."""
url = (url or "").strip().rstrip("/")
@@ -426,6 +444,8 @@ def _detect_provider(url: str) -> str:
return "openrouter"
if _host_match(url, "groq.com"):
return "groq"
if _host_match(url, "nvidia.com"):
return "nvidia"
from src.chatgpt_subscription import is_chatgpt_subscription_base
if is_chatgpt_subscription_base(url):
return "chatgpt-subscription"
@@ -471,6 +491,7 @@ def _provider_label(url: str) -> str:
if is_copilot_base(url): return "GitHub Copilot"
if _host_match(url, "mistral.ai"): return "Mistral"
if _host_match(url, "deepseek.com"): return "DeepSeek"
if _host_match(url, "nvidia.com"): return "NVIDIA"
if _host_match(url, "googleapis.com"): return "Google"
if _host_match(url, "together.xyz", "together.ai"): return "Together"
if _host_match(url, "fireworks.ai"): return "Fireworks"
@@ -1344,6 +1365,9 @@ async def llm_call_async(
if max_tokens and max_tokens > 0:
tok_key = "max_completion_tokens" if _uses_max_completion_tokens(model) else "max_tokens"
payload[tok_key] = max_tokens
# Suppress thinking for qwen3/gemma4 on Ollama /v1 — same as stream_llm.
if _is_ollama_openai_compat_url(url) and _supports_thinking(model):
payload["think"] = False
if _is_host_dead(target_url):
raise HTTPException(503, f"Upstream {_host_key(target_url)} marked unreachable (cooldown active)")
@@ -1461,6 +1485,11 @@ async def stream_llm(url: str, model: str, messages: List[Dict], temperature: fl
payload[tok_key] = max_tokens
if tools:
payload["tools"] = tools
# For Ollama's OpenAI-compat /v1 endpoint with thinking models (qwen3,
# gemma4, etc.), suppress thinking so tool calls aren't swallowed inside
# <think> blocks. Ollama /v1 accepts "think": false as a top-level param.
if _is_ollama_openai_compat_url(url) and _supports_thinking(model):
payload["think"] = False
h = _provider_headers(provider, headers)
if provider == "copilot":
from src.copilot import apply_request_headers
+19 -67
View File
@@ -67,13 +67,12 @@ def _unified_diff(old: str, new: str, path: str) -> Optional[Dict[str, Any]]:
}
async def _do_edit_file(content: str, workspace: Optional[str] = None) -> Dict[str, Any]:
async def _do_edit_file(content: str) -> Dict[str, Any]:
"""Exact string-replacement edit of an on-disk file.
content is JSON: {"path", "old_string", "new_string", "replace_all"?}.
Fails if old_string is missing or non-unique (unless replace_all) so the
model can't silently edit the wrong place. Returns a unified diff for the UI.
Confined to the workspace when one is set (same policy as write_file).
"""
try:
args = json.loads(content) if content.strip().startswith("{") else {}
@@ -85,11 +84,9 @@ async def _do_edit_file(content: str, workspace: Optional[str] = None) -> Dict[s
replace_all = bool(args.get("replace_all", False))
if not raw_path:
return {"error": "edit_file: path required", "exit_code": 1}
# Confine to the workspace when set, else the same allowlist + sensitive-file
# policy as read/write_file.
# Allowlist + sensitive-file policy as read/write_file.
try:
path = (_resolve_tool_path_in_workspace(workspace, raw_path)
if workspace else _resolve_tool_path(raw_path))
path = _resolve_tool_path(raw_path)
except ValueError as e:
return {"error": f"edit_file: {e}", "exit_code": 1}
if old == "":
@@ -272,39 +269,6 @@ def _resolve_tool_path(raw_path: str) -> str:
)
def _resolve_tool_path_in_workspace(workspace: str, raw_path: str) -> str:
"""Confine a model-supplied path to the active workspace.
Layered on top of upstream's path policy: the workspace is the allowed
root (relative paths resolve under it; paths that escape it are rejected),
and the sensitive-file deny list (.ssh, .gnupg, id_rsa, …) still applies
inside it. When no workspace is set, callers use _resolve_tool_path (the
default data/tmp allowlist) instead.
"""
if raw_path is None or not str(raw_path).strip():
raise ValueError("path is required")
base = os.path.realpath(workspace)
expanded = os.path.expanduser(str(raw_path).strip())
candidate = expanded if os.path.isabs(expanded) else os.path.join(base, expanded)
resolved = os.path.realpath(candidate)
if _is_sensitive_path(resolved):
raise ValueError(
f"path '{raw_path}' is inside a sensitive directory "
f"(e.g. .ssh, .gnupg) or matches a sensitive filename"
)
if resolved != base:
# normcase so containment holds on case-insensitive filesystems
# (Windows, default macOS): it lowercases on Windows and is a no-op on
# POSIX. commonpath raises ValueError across Windows drives (C: vs D:)
# or mixed abs/rel — both mean "outside", so the except rejects them.
nbase = os.path.normcase(base)
try:
if os.path.commonpath([os.path.normcase(resolved), nbase]) != nbase:
raise ValueError
except ValueError:
raise ValueError(f"path '{raw_path}' is outside the workspace ({workspace})")
return resolved
# Bash + python tools used to share a single 60s timeout. That's
# enough for one-shot commands but starves real workloads (pip
# install, ffmpeg conversions, etc.) — and worse, the agent saw the
@@ -341,19 +305,13 @@ _CODENAV_MAX_HITS = 200
_CODENAV_MAX_LINE = 400
def _resolve_search_root(raw_path: str, workspace: Optional[str] = None) -> str:
def _resolve_search_root(raw_path: str) -> str:
"""Resolve + confine a code-nav path (grep/glob/ls).
With a workspace set, the workspace folder is the root and supplied paths are
confined inside it (same policy as read_file). Without one, an empty path
defaults to the agent's primary root (project data dir) and a supplied path
is confined by the global allowlist + sensitive-file policy.
An empty path defaults to the agent's primary root (project data dir) and a
supplied path is confined by the global allowlist + sensitive-file policy.
"""
raw = (raw_path or "").strip()
if workspace:
if not raw:
return os.path.realpath(workspace)
return _resolve_tool_path_in_workspace(workspace, raw)
if not raw:
roots = _tool_path_roots()
return roots[0] if roots else os.path.realpath(".")
@@ -564,12 +522,11 @@ async def _call_mcp_tool(
tool: str,
content: str,
progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None,
workspace: Optional[str] = None,
) -> Dict:
"""Route a legacy tool call through the MCP manager, with direct fallbacks."""
mcp = get_mcp_manager()
if not mcp:
return await _direct_fallback(tool, content, progress_cb=progress_cb, workspace=workspace) or {"error": f"MCP manager not available for tool '{tool}'", "exit_code": 1}
return await _direct_fallback(tool, content, progress_cb=progress_cb) or {"error": f"MCP manager not available for tool '{tool}'", "exit_code": 1}
server_id, tool_name = _MCP_TOOL_MAP[tool]
qualified = f"mcp__{server_id}__{tool_name}"
@@ -578,7 +535,7 @@ async def _call_mcp_tool(
# If MCP server not connected, try direct fallback
if isinstance(result, dict) and result.get("exit_code") == 1 and "not connected" in result.get("error", ""):
fallback = await _direct_fallback(tool, content, progress_cb=progress_cb, workspace=workspace)
fallback = await _direct_fallback(tool, content, progress_cb=progress_cb)
if fallback:
return fallback
@@ -636,7 +593,6 @@ async def _direct_fallback(
tool: str,
content: str,
progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None,
workspace: Optional[str] = None,
) -> Optional[Dict]:
"""In-process execution path for the eight tools that used to live as
stdio MCP servers under mcp_servers/. Those servers were deleted in
@@ -670,7 +626,7 @@ async def _direct_fallback(
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=_subproc_env,
cwd=workspace or _AGENT_WORKDIR,
cwd=_AGENT_WORKDIR,
)
stdout, stderr, rc, timed_out = await _run_subprocess_streaming(
proc,
@@ -697,7 +653,7 @@ async def _direct_fallback(
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=_subproc_env,
cwd=workspace or _AGENT_WORKDIR,
cwd=_AGENT_WORKDIR,
)
stdout, stderr, rc, timed_out = await _run_subprocess_streaming(
proc,
@@ -727,8 +683,7 @@ async def _direct_fallback(
except (json.JSONDecodeError, TypeError, ValueError):
pass
try:
path = (_resolve_tool_path_in_workspace(workspace, raw_path)
if workspace else _resolve_tool_path(raw_path))
path = _resolve_tool_path(raw_path)
except ValueError as e:
return {"error": f"read_file: {e}", "exit_code": 1}
try:
@@ -771,8 +726,7 @@ async def _direct_fallback(
raw_path = lines[0].strip()
body = lines[1] if len(lines) > 1 else ""
try:
path = (_resolve_tool_path_in_workspace(workspace, raw_path)
if workspace else _resolve_tool_path(raw_path))
path = _resolve_tool_path(raw_path)
except ValueError as e:
return {"error": f"write_file: {e}", "exit_code": 1}
try:
@@ -825,7 +779,7 @@ async def _direct_fallback(
max_hits = _CODENAV_MAX_HITS
max_hits = max(1, min(max_hits, _CODENAV_MAX_HITS))
try:
root = _resolve_search_root(str(args.get("path", "")), workspace)
root = _resolve_search_root(str(args.get("path", "")))
except ValueError as e:
return {"error": f"grep: {e}", "exit_code": 1}
@@ -909,7 +863,7 @@ async def _direct_fallback(
if not pattern:
return {"error": "glob: pattern is required", "exit_code": 1}
try:
root = _resolve_search_root(str(args.get("path", "")), workspace)
root = _resolve_search_root(str(args.get("path", "")))
except ValueError as e:
return {"error": f"glob: {e}", "exit_code": 1}
@@ -956,7 +910,7 @@ async def _direct_fallback(
else:
raw_path = _s.split("\n", 1)[0].strip()
try:
root = _resolve_search_root(raw_path, workspace)
root = _resolve_search_root(raw_path)
except ValueError as e:
return {"error": f"ls: {e}", "exit_code": 1}
@@ -1121,7 +1075,6 @@ async def execute_tool_block(
tool_policy: Optional[ToolPolicy] = None,
owner: Optional[str] = None,
progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None,
workspace: Optional[str] = None,
) -> Tuple[str, Dict]:
"""Execute a single tool block. Returns (description, result_dict).
@@ -1296,7 +1249,7 @@ async def execute_tool_block(
_is_bg, _bg_cmd = _split_bg_marker(content)
if _is_bg and _bg_cmd:
from src import bg_jobs
rec = bg_jobs.launch(_bg_cmd, session_id=session_id, cwd=workspace or _AGENT_WORKDIR)
rec = bg_jobs.launch(_bg_cmd, session_id=session_id, cwd=_AGENT_WORKDIR)
short = _bg_cmd.strip().split(chr(10))[0][:80]
desc = f"bash (background): {short}"
result = {
@@ -1318,13 +1271,12 @@ async def execute_tool_block(
if tool in _MCP_TOOL_MAP:
first_line = content.split(chr(10))[0][:80]
desc = f"{tool}: {first_line}"
result = await _call_mcp_tool(tool, content, progress_cb=progress_cb, workspace=workspace)
result = await _call_mcp_tool(tool, content, progress_cb=progress_cb)
elif tool in ("grep", "glob", "ls"):
# Code-navigation tools — no MCP server; run the direct implementation.
# Confined to the workspace when one is set (same policy as read_file).
first_line = content.split(chr(10))[0][:80]
desc = f"{tool}: {first_line}"
result = await _direct_fallback(tool, content, progress_cb=progress_cb, workspace=workspace) \
result = await _direct_fallback(tool, content, progress_cb=progress_cb) \
or {"error": f"{tool}: execution failed", "exit_code": 1}
elif tool == "create_document":
title = content.split("\n")[0].strip()[:60]
@@ -1429,7 +1381,7 @@ async def execute_tool_block(
desc = "edit_image"
result = await do_edit_image(content, owner=owner)
elif tool == "edit_file":
result = await _do_edit_file(content, workspace=workspace)
result = await _do_edit_file(content)
desc = result.get("output") or result.get("error") or "edit_file"
elif tool == "trigger_research":
desc = "trigger_research"
+1
View File
@@ -2095,6 +2095,7 @@
<option value="https://opencode.ai/zen/v1" data-logo="opencode">OpenCode Zen</option>
<option value="https://opencode.ai/zen/go/v1" data-logo="opencode">OpenCode Go</option>
<option value="https://api.z.ai/api/coding/paas/v4" data-logo="zhipu">Z.AI Coding Plan</option>
<option value="https://integrate.api.nvidia.com/v1" data-logo="nvidia">NVIDIA</option>
</select>
<!-- API key row stays in DOM, hidden until Key button is
clicked. Mirrors the Local section pattern: most users
+1
View File
@@ -118,6 +118,7 @@ const _ENDPOINT_LABELS = [
[/(^|\.)together\.(ai|xyz)$/i, "Together"],
[/(^|\.)fireworks\.ai$/i, "Fireworks"],
[/(^|\.)perplexity\.ai$/i, "Perplexity"],
[/(^|\.)nvidia\.com$/i, "NVIDIA"],
[/(^|\.)x\.ai$/i, "xAI"],
];
+5 -1
View File
@@ -43,6 +43,7 @@ const PROVIDER_PATTERNS = [
{ re: /^gsk_/, name: 'Groq', url: 'https://api.groq.com/openai/v1' },
{ re: /^AIza/, name: 'Gemini', url: 'https://generativelanguage.googleapis.com/v1beta/openai' },
{ re: /^xai-/, name: 'xAI', url: 'https://api.x.ai/v1' },
{ re: /^nvapi-/, name: 'NVIDIA', url: 'https://integrate.api.nvidia.com/v1' },
];
const SETUP_PROVIDER_URLS = {
deepseek: { name: 'DeepSeek', url: 'https://api.deepseek.com/v1' },
@@ -56,8 +57,9 @@ const SETUP_PROVIDER_URLS = {
google: { name: 'Gemini', url: 'https://generativelanguage.googleapis.com/v1beta/openai' },
'opencode-zen': { name: 'OpenCode Zen', url: 'https://opencode.ai/zen/v1' },
'opencode-go': { name: 'OpenCode Go', url: 'https://opencode.ai/zen/go/v1' },
nvidia: { name: 'NVIDIA', url: 'https://integrate.api.nvidia.com/v1' },
};
const SETUP_PROVIDER_NAMES = ['deepseek', 'openai', 'openrouter', 'ollama', 'xai', 'anthropic', 'groq', 'gemini', 'opencode-zen', 'opencode-go'];
const SETUP_PROVIDER_NAMES = ['deepseek', 'openai', 'openrouter', 'ollama', 'xai', 'anthropic', 'groq', 'gemini', 'opencode-zen', 'opencode-go', 'nvidia'];
const SETUP_DEVICE_AUTH_PROVIDERS = [
{ key: 'copilot', name: 'GitHub Copilot', aliases: ['github'], command: '/setup copilot' },
{ key: 'chatgpt-subscription', name: 'ChatGPT Subscription', aliases: ['chatgptsubscription', 'chatgpt-sub', 'codex'], command: '/setup chatgpt-subscription' },
@@ -97,6 +99,7 @@ function _setupProviderFromInput(input) {
google: 'gemini',
xai: 'xai',
grok: 'xai',
nvidia: 'nvidia',
};
return SETUP_PROVIDER_URLS[aliases[raw] || raw] || null;
}
@@ -124,6 +127,7 @@ function _extractSetupProviderCredential(input) {
['groq', 'groq'],
['google', 'gemini'], ['gemini', 'gemini'],
['x ai', 'xai'], ['xai', 'xai'], ['grok', 'xai'],
['nvidia', 'nvidia'],
];
for (const [alias, key] of providerAliases) {
const re = new RegExp('(^|\\s|[,;:])(' + alias.replace(/\s+/g, '\\s+') + ')(?=$|\\s|[,;:])', 'i');
+112
View File
@@ -0,0 +1,112 @@
"""Regression test for routes/backup_routes.py import_data skills dedup.
BUG: the skills import block deduplicates against EVERY tenant's skills
(skills_manager.load_all()) instead of the importing user's own skills.
So importing your own backup silently drops any skill whose title (or id)
collides with ANOTHER user's skill — the same cross-tenant data-loss bug
that was already fixed for memories in the block just above.
"""
import pytest
from fastapi import FastAPI, Request
from fastapi.testclient import TestClient
import routes.backup_routes as backup_routes
from routes.backup_routes import setup_backup_routes
# require_admin / get_current_user are bound into routes.backup_routes at import
# time (`from x import name`). We patch them on that module directly per-test
# via monkeypatch — robust to import order and reverted at teardown. (Stubbing
# them through sys.modules only works if backup_routes has not been imported
# yet, which is not guaranteed in a full-suite run.)
class FakeMemoryManager:
def __init__(self):
self.rows = []
def load(self, owner=None):
return [r for r in self.rows if r.get("owner") == owner]
def load_all(self):
return list(self.rows)
def save(self, rows):
self.rows = list(rows)
class FakePresetManager:
def get_all(self):
return {}
def save(self, d):
pass
class FakeSkillsManager:
"""Mimics services.memory.skills: load_all() = all owners,
load(owner) = that owner's skills only."""
def __init__(self, rows):
self.rows = list(rows)
def load(self, owner=None):
return [s for s in self.rows if s.get("owner") == owner]
def load_all(self):
return list(self.rows)
def save(self, rows):
self.rows = list(rows)
def add_skill(self, title=None, name=None, owner=None, **kwargs):
# Mirrors services.memory.skills.add_skill: persists a SKILL.md row and
# returns its identity. source="user" skips auto-dedup, so no _deduped.
entry = {"id": f"new-{len(self.rows)}", "title": title, "name": name, "owner": owner}
self.rows.append(entry)
return {"name": name, "id": entry["id"]}
def _make_client(skills_mgr, monkeypatch):
# Bypass the admin gate and read the importer straight off request.state.
monkeypatch.setattr(backup_routes, "require_admin", lambda *a, **k: None)
monkeypatch.setattr(backup_routes, "get_current_user",
lambda req: getattr(req.state, "user", None))
app = FastAPI()
@app.middleware("http")
async def _set_user(request: Request, call_next):
request.state.user = "alice"
return await call_next(request)
router = setup_backup_routes(FakeMemoryManager(), FakePresetManager(), skills_mgr)
app.include_router(router)
return TestClient(app)
def test_import_skill_not_dropped_by_other_users_title_collision(monkeypatch):
# Bob already owns a skill titled "Deploy". Alice (the importer) has none.
skills_mgr = FakeSkillsManager([
{"id": "bob-1", "title": "Deploy", "name": "Deploy", "owner": "bob"},
])
client = _make_client(skills_mgr, monkeypatch)
# Alice imports HER OWN backup containing a skill also titled "Deploy".
payload = {
"skills": [
{"id": "alice-1", "title": "Deploy", "name": "Deploy"},
],
}
resp = client.post("/api/import", json=payload)
assert resp.status_code == 200, resp.text
# Alice's skill must have been imported and assigned to her.
alice_skills = skills_mgr.load(owner="alice")
titles = {s["title"] for s in alice_skills}
assert "Deploy" in titles, (
"Alice's own 'Deploy' skill was silently dropped because Bob owns a "
"skill with the same title (cross-tenant dedup bug)."
)
if __name__ == "__main__":
raise SystemExit(pytest.main([__file__, "-v"]))
+85
View File
@@ -22,10 +22,12 @@ from routes.cookbook_helpers import (
_user_shell_path_bootstrap,
_venv_safe_local_pip_install_cmd,
_validate_gpus,
_validate_local_dir,
_validate_repo_id,
_validate_serve_cmd,
_validate_serve_model_id,
_validate_ssh_port,
_shell_path,
run_ssh_command_async,
)
@@ -110,6 +112,89 @@ def test_validate_ssh_port_rejects_shell_payload():
assert _validate_ssh_port("2222") == "2222"
def test_validate_local_dir_accepts_external_drive_paths_with_spaces():
path = "/Volumes/T7 2TB/AI Models/llamacpp"
assert _validate_local_dir(path) == path
assert _validate_local_dir(f'"{path}"') == path
assert _shell_path(f"{path}/Qwen3-8B") == '"/Volumes/T7 2TB/AI Models/llamacpp/Qwen3-8B"'
def test_validate_local_dir_accepts_windows_drive_paths_with_spaces():
backslash_path = r"D:\AI Models\llamacpp"
slash_path = "D:/AI Models/llamacpp"
assert _validate_local_dir(backslash_path) == backslash_path
assert _validate_local_dir(f"'{backslash_path}'") == backslash_path
assert _validate_local_dir(slash_path) == slash_path
assert _shell_path(backslash_path + r"\Qwen3-8B") == '"D:\\AI Models\\llamacpp\\Qwen3-8B"'
def test_validate_local_dir_still_rejects_shell_metacharacters():
for path in [
"/Volumes/T7 2TB/AI Models; touch /tmp/pwned",
"/Volumes/T7 2TB/AI Models/$(touch pwned)",
"/Volumes/T7 2TB/AI Models/`touch pwned`",
"/Volumes/T7 2TB/AI Models/model\nnext",
]:
with pytest.raises(HTTPException):
_validate_local_dir(path)
def test_validate_local_dir_rejects_windows_shell_metacharacters():
for path in [
r"D:\AI Models\llamacpp; touch C:\pwned",
r"D:\AI Models\llamacpp\$(touch pwned)",
r"D:\AI Models\llamacpp\`touch pwned`",
"D:\\AI Models\\llamacpp\nnext",
]:
with pytest.raises(HTTPException):
_validate_local_dir(path)
def test_validate_local_dir_accepts_non_ascii_unicode_paths():
# Folder names are routinely non-ASCII on localized systems; the validator
# must accept them the same way it accepts spaces (see issue: spaces AND
# non-ASCII chars were both rejected by the old ASCII-only allowlist).
for path in [
"/Volumes/Модели/llamacpp", # Cyrillic (POSIX / external drive)
"/home/josé/models", # accented Latin
"/Volumes/モデル/llm", # CJK
r"D:\AI Models\Модели", # Cyrillic (Windows drive path)
]:
assert _validate_local_dir(path) == path
def test_validate_local_dir_rejects_metacharacters_in_unicode_paths():
# Widening the allowlist to Unicode must not reopen the injection surface:
# shell metacharacters stay rejected even alongside non-ASCII segments.
for path in [
"/Volumes/Модели; touch /tmp/pwned",
"/Volumes/Модели/$(touch pwned)",
"/Volumes/Модели/`touch pwned`",
"/Volumes/Модели/a|b",
"/Volumes/Модели\nnext",
r"D:\Модели\llamacpp & calc.exe",
]:
with pytest.raises(HTTPException):
_validate_local_dir(path)
def test_validate_local_dir_rejects_leading_dash_segments():
# A path segment starting with '-' could be parsed as a CLI option by hf/etc.
# (option injection) even when quoted, since quoting doesn't stop a value from
# being read as a flag. The validator must reject it on every platform.
for path in [
"/models/-rf",
"/models/-rf/llamacpp",
"/-oStrictHostKeyChecking=no",
r"D:\models\-rf",
"D:/models/-rf",
]:
with pytest.raises(HTTPException):
_validate_local_dir(path)
def test_validate_gpus_accepts_indexes_only():
assert _validate_gpus("0,1,2") == "0,1,2"
with pytest.raises(HTTPException):
@@ -0,0 +1,68 @@
"""Embedding-lane reset must restore rows even when chromadb returns the
preserved embeddings as a numpy ndarray.
Real chromadb returns collection.get(include=["embeddings"]) as a numpy
ndarray. The restore-after-failed-rewrite path used `embeddings or []` and a
bare `if ... and embeddings:`, both of which raise
"truth value of an array ... is ambiguous" on an ndarray — aborting the
restore and wiping the collection the reset was meant to preserve.
This mirrors test_lane_reset_restores_existing_collection_when_rewrite_fails
in test_embedding_lanes.py, but the preserved embeddings come back as ndarray.
"""
import numpy as np
from src.embedding_lanes import build_embedding_lanes
from tests.test_embedding_lanes import FakeChroma, FakeEmbedder, _patch_chroma
def test_lane_reset_restores_when_chroma_returns_numpy_embeddings(monkeypatch):
fake = FakeChroma()
old_custom = fake.get_or_create_collection(
"odysseus_memories_custom",
metadata={
"embedding_lane": "custom",
"embedding_dimension": 384,
"embedding_fingerprint": "old",
},
)
old_custom.add(
ids=["existing-memory"],
embeddings=[[0.0] * 384],
documents=["existing custom memory"],
metadatas=[{"source": "memory"}],
)
# Make the preserved embeddings come back as a numpy ndarray, like real
# chromadb does.
real_get = old_custom.get
def ndarray_get(*args, **kwargs):
result = real_get(*args, **kwargs)
result["embeddings"] = np.array(result["embeddings"])
return result
old_custom.get = ndarray_get
# Force the post-reset rewrite to fail so the restore branch runs.
fake.fail_next_add_for["odysseus_memories_custom"] = 1
_patch_chroma(monkeypatch, fake)
import src.embedding_lanes as lanes
monkeypatch.setattr(lanes, "_build_custom_client", lambda: FakeEmbedder(768, "nomic", "http://embeddings/v1"))
def fail_fastembed():
raise RuntimeError("fastembed missing")
monkeypatch.setattr(lanes, "_build_fastembed_client", fail_fastembed)
built = build_embedding_lanes("odysseus_memories")
# Both lanes are unavailable, but the existing row must survive — not be
# wiped by an ndarray-truthiness crash in the restore path.
assert built == []
restored = fake.collections["odysseus_memories_custom"]
assert restored.count() == 1
assert restored.get()["ids"] == ["existing-memory"]
assert len(restored.rows["existing-memory"]["embedding"]) == 384
+165
View File
@@ -0,0 +1,165 @@
"""Tests for Ollama /v1 thinking-suppression helpers.
Covers:
- _is_ollama_openai_compat_url: URL classification (local host + /v1 path)
- think: false is injected into the payload for Ollama /v1 thinking models
- think: false is NOT injected for non-thinking models or non-Ollama /v1 endpoints
"""
import asyncio
import json
from src import llm_core
# ---------------------------------------------------------------------------
# Fake HTTP client — captures the outgoing payload without network I/O
# ---------------------------------------------------------------------------
class _FakeResp:
status_code = 200
async def aiter_lines(self):
# Yield a minimal done event so stream_llm exits cleanly
yield json.dumps({"choices": [{"delta": {"content": "ok"}, "finish_reason": "stop"}]})
yield "data: [DONE]"
async def aread(self):
return b""
class _FakeStreamCtx:
def __init__(self, captured):
self._captured = captured
async def __aenter__(self):
return _FakeResp()
async def __aexit__(self, *a):
return False
class _FakeClient:
"""Minimal stand-in for httpx.AsyncClient that captures request payload."""
def __init__(self):
self.captured_payload = {}
def stream(self, method, url, **kw):
self.captured_payload = kw.get("json") or {}
return _FakeStreamCtx(self.captured_payload)
def _capture_payload(monkeypatch, url, model):
"""Run stream_llm, intercept the HTTP payload, and return it."""
client = _FakeClient()
monkeypatch.setattr(llm_core, "_get_http_client", lambda: client)
monkeypatch.setattr(llm_core, "_is_host_dead", lambda u: False)
monkeypatch.setattr(llm_core, "note_model_activity", lambda *a, **k: None)
monkeypatch.setattr(llm_core, "_clear_host_dead", lambda *a, **k: None)
monkeypatch.setattr(llm_core, "get_context_length", lambda u, m: 32768)
async def run():
return [c async for c in llm_core.stream_llm(
url, model, [{"role": "user", "content": "hi"}],
)]
asyncio.run(run())
return client.captured_payload
# ---------------------------------------------------------------------------
# _is_ollama_openai_compat_url — pure function, no I/O
# ---------------------------------------------------------------------------
class TestIsOllamaOpenAICompatUrl:
"""Unit tests for the URL classifier that gates think-suppression."""
# Positive cases — should be True
def test_default_port_v1_root(self):
assert llm_core._is_ollama_openai_compat_url("http://127.0.0.1:11434/v1")
def test_default_port_chat_completions(self):
assert llm_core._is_ollama_openai_compat_url("http://127.0.0.1:11434/v1/chat/completions")
def test_localhost_default_port(self):
assert llm_core._is_ollama_openai_compat_url("http://localhost:11434/v1")
def test_localhost_default_port_with_path(self):
assert llm_core._is_ollama_openai_compat_url("http://localhost:11434/v1/chat/completions")
def test_loopback_ipv6(self):
# IPv6 addresses in URLs require square brackets per RFC 3986
assert llm_core._is_ollama_openai_compat_url("http://[::1]:11434/v1")
def test_any_local_non_default_port(self):
"""Localhost on a non-default port (custom OLLAMA_HOST) must also match."""
assert llm_core._is_ollama_openai_compat_url("http://127.0.0.1:11435/v1")
def test_localhost_non_default_port(self):
assert llm_core._is_ollama_openai_compat_url("http://localhost:8080/v1/chat/completions")
def test_zero_dot_zero_host(self):
assert llm_core._is_ollama_openai_compat_url("http://0.0.0.0:11434/v1")
# Negative cases — should be False
def test_openai_api_v1(self):
"""Real OpenAI endpoint must never match, even though path is /v1."""
assert not llm_core._is_ollama_openai_compat_url("https://api.openai.com/v1")
def test_openai_chat_completions(self):
assert not llm_core._is_ollama_openai_compat_url("https://api.openai.com/v1/chat/completions")
def test_ollama_native_api_path(self):
"""The native /api path is a different surface and must not match /v1."""
assert not llm_core._is_ollama_openai_compat_url("http://localhost:11434/api")
def test_ollama_native_api_chat(self):
assert not llm_core._is_ollama_openai_compat_url("http://localhost:11434/api/chat")
def test_remote_openrouter(self):
assert not llm_core._is_ollama_openai_compat_url("https://openrouter.ai/api/v1")
def test_empty_string(self):
assert not llm_core._is_ollama_openai_compat_url("")
def test_none_like_empty(self):
assert not llm_core._is_ollama_openai_compat_url(None) # type: ignore[arg-type]
# ---------------------------------------------------------------------------
# Payload injection — think: false only when both conditions hold
# ---------------------------------------------------------------------------
class TestThinkSuppression:
"""Assert think:false is present/absent in the outgoing HTTP payload."""
def test_think_false_for_ollama_v1_thinking_model(self, monkeypatch):
"""think:false must be set for qwen3 on Ollama /v1."""
payload = _capture_payload(
monkeypatch, "http://127.0.0.1:11434/v1/chat/completions", "qwen3:14b"
)
assert payload.get("think") is False
def test_no_think_for_ollama_v1_non_thinking_model(self, monkeypatch):
"""think must NOT be set for a plain (non-thinking) model on Ollama /v1."""
payload = _capture_payload(
monkeypatch, "http://127.0.0.1:11434/v1/chat/completions", "llama3.2:3b"
)
assert "think" not in payload
def test_no_think_for_openai_endpoint_with_thinking_model_name(self, monkeypatch):
"""think must NOT leak to a real OpenAI endpoint even if the model name
matches a thinking pattern — the URL guard is what matters."""
payload = _capture_payload(
monkeypatch, "https://api.openai.com/v1/chat/completions", "qwen3:14b"
)
assert "think" not in payload
def test_think_false_for_non_default_port_thinking_model(self, monkeypatch):
"""Custom-port localhost Ollama (e.g. OLLAMA_HOST=0.0.0.0:11435) must
also receive think:false — this is the regression guarded by the
host-set check added in this fix."""
payload = _capture_payload(
monkeypatch, "http://127.0.0.1:11435/v1/chat/completions", "qwen3:14b"
)
assert payload.get("think") is False
+2
View File
@@ -347,6 +347,8 @@ class TestIsChatModel:
"gpt-4o", "gpt-4o-mini", "claude-sonnet-4", "llama-3.3-70b",
"deepseek-chat", "gemini-2.0-flash", "o3",
"llama-4-scout-17b-16e-instruct",
"gemma-2b-it", "google/gemma-2b-it",
"bigcode/starcoder2-15b-instruct",
])
def test_chat_models(self, model_id):
assert _is_chat_model(model_id) is True
+2
View File
@@ -40,6 +40,7 @@ class TestDetectProvider:
("https://anthropic.com/v1", "anthropic"),
("https://openrouter.ai/api/v1", "openrouter"),
("https://api.groq.com/openai/v1", "groq"),
("https://integrate.api.nvidia.com/v1", "nvidia"),
("http://localhost:11434/api", "ollama"),
("https://ollama.com", "ollama"),
# xAI, DeepSeek and Gemini's OpenAI-compatible surface are NOT
@@ -84,6 +85,7 @@ class TestProviderLabel:
("https://api.openai.com/v1", "OpenAI"),
("https://openrouter.ai/api/v1", "OpenRouter"),
("https://api.groq.com/openai/v1", "Groq"),
("https://integrate.api.nvidia.com/v1", "NVIDIA"),
("https://api.mistral.ai/v1", "Mistral"),
("https://api.deepseek.com", "DeepSeek"),
("https://generativelanguage.googleapis.com/v1beta/openai", "Google"),
+4
View File
@@ -50,6 +50,9 @@ PROVIDER_CASES = [
("groq", "https://api.groq.com/openai/v1",
"https://api.groq.com/openai/v1/chat/completions",
"https://api.groq.com/openai/v1/models"),
("nvidia", "https://integrate.api.nvidia.com/v1",
"https://integrate.api.nvidia.com/v1/chat/completions",
"https://integrate.api.nvidia.com/v1/models"),
("xai", "https://api.x.ai/v1",
"https://api.x.ai/v1/chat/completions",
"https://api.x.ai/v1/models"),
@@ -112,6 +115,7 @@ def test_headers_anthropic_without_key_still_sends_version():
"https://api.x.ai/v1",
"https://api.deepseek.com",
"https://api.groq.com/openai/v1",
"https://integrate.api.nvidia.com/v1",
"https://generativelanguage.googleapis.com/v1beta/openai",
])
def test_headers_openai_style_use_bearer(base):
+384
View File
@@ -0,0 +1,384 @@
"""Renaming a user must update all three owner caches, not just the SQL DB.
The DB owner-rename loop in the rename_user route updates every SQL-backed
owner column, but three file-backed / in-memory stores are left stale:
1. session_manager.sessions — in-memory session objects carry s.owner set at
load time; get_sessions_for_user does an exact `s.owner == username` check,
so the renamed user's sidebar empties until a server restart.
2. data/deep_research/*.json — each report JSON has an `owner` field;
research_routes filters by `d.get("owner") == user`, making every report
invisible after rename.
3. data/memory.json — a flat array where every entry has an `owner` field;
memory_manager.load(owner=user) filters on it, so all memories vanish.
Regression coverage: these bugs are invisible in unit tests that mock the DB
loop but don't exercise the file/cache patches added to the route.
"""
import asyncio
import json
import sys
import types
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
def _route(router, name):
for r in router.routes:
if getattr(getattr(r, "endpoint", None), "__name__", "") == name:
return r.endpoint
raise AssertionError(name)
@pytest.fixture
def rename_endpoint(monkeypatch, tmp_path):
import routes.auth_routes as ar
import core.database as cdb
# Neutralize the DB owner-rename loop.
monkeypatch.setattr(cdb, "SessionLocal", lambda: MagicMock())
monkeypatch.setattr(cdb, "Base", SimpleNamespace(registry=SimpleNamespace(mappers=[])), raising=False)
# Neutralize the JSON-prefs rename.
pr = types.ModuleType("routes.prefs_routes")
pr._load = lambda: {}
pr._save = lambda d: None
monkeypatch.setitem(sys.modules, "routes.prefs_routes", pr)
# Patch the module-level constants so file-update steps write to tmp_path.
# (Patching sc.DATA_DIR wouldn't work — auth_routes binds DEEP_RESEARCH_DIR
# and MEMORY_FILE at import time, so we must patch those names on the module.)
monkeypatch.setattr(ar, "DEEP_RESEARCH_DIR", str(tmp_path / "deep_research"))
monkeypatch.setattr(ar, "MEMORY_FILE", str(tmp_path / "memory.json"))
monkeypatch.setattr(ar, "SKILLS_DIR", str(tmp_path / "skills"))
am = MagicMock()
am.is_admin.return_value = True
am.get_username_for_token.return_value = "admin"
am.users = {"alice": {}}
am.rename_user.return_value = True
return _route(ar.setup_auth_routes(am), "rename_user"), am, tmp_path
def _request(tmp_path, session_manager=None):
state = SimpleNamespace(
invalidate_token_cache=lambda: None,
session_manager=session_manager,
)
return SimpleNamespace(
cookies={"odysseus_session": "t"},
app=SimpleNamespace(state=state),
state=SimpleNamespace(current_user="admin"),
)
# ---------------------------------------------------------------------------
# 1. In-memory session cache
# ---------------------------------------------------------------------------
def test_rename_updates_in_memory_session_owner(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
# Build a fake session_manager with one session owned by alice.
sess = SimpleNamespace(owner="alice")
sm = SimpleNamespace(sessions={"s1": sess})
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path, sm)))
assert sess.owner == "alice2", "in-memory session owner was not updated on rename"
def test_rename_session_owner_case_insensitive(rename_endpoint):
"""Stored owner 'Alice' (mixed case) must match rename of 'alice'."""
endpoint, _am, tmp_path = rename_endpoint
sess = SimpleNamespace(owner="Alice")
sm = SimpleNamespace(sessions={"s1": sess})
asyncio.run(endpoint("alice", SimpleNamespace(username="bob"), _request(tmp_path, sm)))
assert sess.owner == "bob"
def test_rename_leaves_other_sessions_untouched(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
sess_alice = SimpleNamespace(owner="alice")
sess_other = SimpleNamespace(owner="carol")
sm = SimpleNamespace(sessions={"s1": sess_alice, "s2": sess_other})
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path, sm)))
assert sess_alice.owner == "alice2"
assert sess_other.owner == "carol", "unrelated session owner was modified"
def test_rename_no_session_manager_does_not_crash(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
# app.state without a session_manager must not raise.
req = SimpleNamespace(
cookies={"odysseus_session": "t"},
app=SimpleNamespace(state=SimpleNamespace(invalidate_token_cache=lambda: None)),
state=SimpleNamespace(current_user="admin"),
)
res = asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), req))
assert res["ok"] is True
# ---------------------------------------------------------------------------
# 2. deep_research JSON files
# ---------------------------------------------------------------------------
def test_rename_updates_research_json_owner(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
dr_dir = tmp_path / "deep_research"
dr_dir.mkdir()
report = {"query": "test", "owner": "alice", "status": "done"}
p = dr_dir / "abc123.json"
p.write_text(json.dumps(report), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
updated = json.loads(p.read_text(encoding="utf-8"))
assert updated["owner"] == "alice2", "deep_research JSON owner was not updated on rename"
def test_rename_research_json_case_insensitive(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
dr_dir = tmp_path / "deep_research"
dr_dir.mkdir()
p = (dr_dir / "r1.json")
p.write_text(json.dumps({"owner": "Alice"}), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="bob"), _request(tmp_path)))
assert json.loads(p.read_text())["owner"] == "bob"
def test_rename_leaves_other_research_untouched(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
dr_dir = tmp_path / "deep_research"
dr_dir.mkdir()
p_alice = dr_dir / "a.json"
p_carol = dr_dir / "c.json"
p_alice.write_text(json.dumps({"owner": "alice"}), encoding="utf-8")
p_carol.write_text(json.dumps({"owner": "carol"}), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert json.loads(p_alice.read_text())["owner"] == "alice2"
assert json.loads(p_carol.read_text())["owner"] == "carol"
def test_rename_no_deep_research_dir_does_not_crash(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
# No deep_research dir — must not crash.
res = asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert res["ok"] is True
def test_rename_research_respects_custom_data_dir(monkeypatch, tmp_path):
"""DEEP_RESEARCH_DIR (which honours ODYSSEUS_DATA_DIR) is used, not a
hardcoded relative path. Before the fix, setting ODYSSEUS_DATA_DIR made
the rename silently patch a different directory from where research files
actually live, so reports still disappeared after rename."""
import routes.auth_routes as ar
import core.database as cdb
custom_dr = tmp_path / "custom_data" / "deep_research"
custom_dr.mkdir(parents=True)
p = custom_dr / "rp-abc.json"
p.write_text(json.dumps({"query": "q", "owner": "alice", "status": "done"}), encoding="utf-8")
monkeypatch.setattr(cdb, "SessionLocal", lambda: MagicMock())
monkeypatch.setattr(cdb, "Base", SimpleNamespace(registry=SimpleNamespace(mappers=[])), raising=False)
pr = types.ModuleType("routes.prefs_routes")
pr._load = lambda: {}
pr._save = lambda d: None
monkeypatch.setitem(sys.modules, "routes.prefs_routes", pr)
monkeypatch.setattr(ar, "DEEP_RESEARCH_DIR", str(custom_dr))
monkeypatch.setattr(ar, "MEMORY_FILE", str(tmp_path / "memory.json"))
am = MagicMock()
am.is_admin.return_value = True
am.get_username_for_token.return_value = "admin"
am.users = {"alice": {}}
am.rename_user.return_value = True
endpoint = _route(ar.setup_auth_routes(am), "rename_user")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert json.loads(p.read_text(encoding="utf-8"))["owner"] == "alice2", (
"research JSON at custom DATA_DIR was not patched — DEEP_RESEARCH_DIR constant not used"
)
# ---------------------------------------------------------------------------
# 3. memory.json
# ---------------------------------------------------------------------------
def test_rename_updates_memory_json_owner(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
entries = [
{"id": "1", "text": "Lives in Berlin", "owner": "alice"},
{"id": "2", "text": "Likes Python", "owner": "carol"},
]
(tmp_path / "memory.json").write_text(json.dumps(entries), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
updated = json.loads((tmp_path / "memory.json").read_text(encoding="utf-8"))
assert updated[0]["owner"] == "alice2", "memory.json entry owner was not updated on rename"
assert updated[1]["owner"] == "carol", "unrelated memory entry was modified"
def test_rename_memory_json_case_insensitive(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
entries = [{"id": "1", "text": "x", "owner": "Alice"}]
(tmp_path / "memory.json").write_text(json.dumps(entries), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="bob"), _request(tmp_path)))
assert json.loads((tmp_path / "memory.json").read_text())[0]["owner"] == "bob"
def test_rename_no_memory_json_does_not_crash(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
# No memory.json — must not crash.
res = asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert res["ok"] is True
# ---------------------------------------------------------------------------
# 4. Skills (SKILL.md frontmatter + _usage.json sidecar)
# ---------------------------------------------------------------------------
_SKILL_MD = """\
---
name: test-skill
description: A test skill.
version: 1.0.0
category: general
status: published
confidence: 0.9
source: learned
owner: {owner}
---
## When to Use
When testing.
"""
def test_rename_updates_skill_md_owner(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
skill_dir = tmp_path / "skills" / "general" / "test-skill"
skill_dir.mkdir(parents=True)
(skill_dir / "SKILL.md").write_text(_SKILL_MD.format(owner="alice"), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
content = (skill_dir / "SKILL.md").read_text(encoding="utf-8")
assert "owner: alice2" in content
assert "owner: alice\n" not in content
def test_rename_leaves_other_skill_owners_untouched(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
for owner, name in [("alice", "alice-skill"), ("carol", "carol-skill")]:
d = tmp_path / "skills" / "general" / name
d.mkdir(parents=True)
(d / "SKILL.md").write_text(_SKILL_MD.format(owner=owner).replace("test-skill", name), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert "owner: alice2" in (tmp_path / "skills" / "general" / "alice-skill" / "SKILL.md").read_text()
assert "owner: carol" in (tmp_path / "skills" / "general" / "carol-skill" / "SKILL.md").read_text()
def test_rename_updates_usage_sidecar_keys(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
skills_root = tmp_path / "skills"
skills_root.mkdir(parents=True)
usage = {
"alice::test-skill": {"uses": 3, "last_used": 1000},
"carol::other-skill": {"uses": 1, "last_used": 500},
"unscoped-skill": {"uses": 2, "last_used": 200},
}
(skills_root / "_usage.json").write_text(json.dumps(usage), encoding="utf-8")
asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
updated = json.loads((skills_root / "_usage.json").read_text(encoding="utf-8"))
assert "alice2::test-skill" in updated
assert "alice::test-skill" not in updated
assert "carol::other-skill" in updated
assert "unscoped-skill" in updated
def test_rename_no_skills_dir_does_not_crash(rename_endpoint):
endpoint, _am, tmp_path = rename_endpoint
res = asyncio.run(endpoint("alice", SimpleNamespace(username="alice2"), _request(tmp_path)))
assert res["ok"] is True
# ---------------------------------------------------------------------------
# 5. P1 regression: rejected auth rename must not mutate file-backed stores
# ---------------------------------------------------------------------------
def test_rejected_rename_does_not_mutate_files(monkeypatch, tmp_path):
"""If auth_manager.rename_user() returns False, no file-backed store
should be touched. Before the fix the deep_research and memory writes
ran before the auth check, so a rejected rename (e.g. reserved username)
silently moved owner fields to the new name."""
import routes.auth_routes as ar
import core.database as cdb
monkeypatch.setattr(cdb, "SessionLocal", lambda: MagicMock())
monkeypatch.setattr(cdb, "Base", SimpleNamespace(registry=SimpleNamespace(mappers=[])), raising=False)
pr = types.ModuleType("routes.prefs_routes")
pr._load = lambda: {}
pr._save = lambda d: None
monkeypatch.setitem(sys.modules, "routes.prefs_routes", pr)
monkeypatch.setattr(ar, "DEEP_RESEARCH_DIR", str(tmp_path / "deep_research"))
monkeypatch.setattr(ar, "MEMORY_FILE", str(tmp_path / "memory.json"))
monkeypatch.setattr(ar, "SKILLS_DIR", str(tmp_path / "skills"))
# Seed files for alice.
dr = tmp_path / "deep_research"
dr.mkdir()
rp = dr / "rp-abc.json"
rp.write_text(json.dumps({"owner": "alice", "query": "q"}), encoding="utf-8")
mem = tmp_path / "memory.json"
mem.write_text(json.dumps([{"owner": "alice", "text": "x"}]), encoding="utf-8")
skill_dir = tmp_path / "skills" / "general" / "s"
skill_dir.mkdir(parents=True)
(skill_dir / "SKILL.md").write_text(_SKILL_MD.format(owner="alice"), encoding="utf-8")
# Auth rejects the rename (reserved name, race, etc.).
am = MagicMock()
am.is_admin.return_value = True
am.get_username_for_token.return_value = "admin"
am.users = {"alice": {}}
am.rename_user.return_value = False
endpoint = _route(ar.setup_auth_routes(am), "rename_user")
with pytest.raises(Exception):
asyncio.run(endpoint("alice", SimpleNamespace(username="api"), _request(tmp_path)))
assert json.loads(rp.read_text())["owner"] == "alice", "research owner mutated after rejected rename"
assert json.loads(mem.read_text())[0]["owner"] == "alice", "memory owner mutated after rejected rename"
assert "owner: alice" in (skill_dir / "SKILL.md").read_text(), "skill owner mutated after rejected rename"
@@ -76,6 +76,23 @@ def _seed_index_skill(tmp_path: Path) -> Path:
return data_dir
def _write_index_skill(data_dir: Path, name: str, description: str, owner: str) -> None:
skill_dir = data_dir / "skills" / owner / name
skill_dir.mkdir(parents=True, exist_ok=True)
(skill_dir / "SKILL.md").write_text(
"---\n"
f"name: {name}\n"
f"description: {description}\n"
"when_to_use: when this owner needs a private workflow\n"
"category: private\n"
"status: published\n"
f"owner: {owner}\n"
"---\n\n"
f"# {name}\n",
encoding="utf-8",
)
def _patch_prefs(monkeypatch, data_dir):
"""Mirror the helpers from test_skill_prompt_injection.py: point
`src.constants.DATA_DIR` at our tmp, and patch the prefs loader so
@@ -152,3 +169,40 @@ def test_skill_index_lands_in_untrusted_user_message(tmp_path, monkeypatch):
)
assert untrusted[0]["role"] == "user"
assert "Source: skills" in untrusted[0]["content"]
def test_skill_index_is_owner_scoped_across_prompt_cache_hits(tmp_path, monkeypatch):
"""Authenticated users must not receive another user's skill index.
This calls the prompt builder twice without clearing the base-prompt cache,
so the second call exercises the cache-hit path as well as owner scoping.
"""
data_dir = tmp_path / "data"
_write_index_skill(data_dir, "alice-only", "Alice private procedure", "alice")
_write_index_skill(data_dir, "bob-only", "Bob private procedure", "bob")
_patch_prefs(monkeypatch, data_dir)
from src.agent_loop import _build_system_prompt # noqa: WPS433
messages = [{"role": "user", "content": "use my workflow"}]
alice_out, _ = _build_system_prompt(
messages=messages, model="test-model",
active_document=None, mcp_mgr=None, owner="alice",
)
bob_out, _ = _build_system_prompt(
messages=messages, model="test-model",
active_document=None, mcp_mgr=None, owner="bob",
)
alice_text = "\n".join(m.get("content", "") or "" for m in alice_out)
bob_text = "\n".join(m.get("content", "") or "" for m in bob_out)
assert "alice-only" in alice_text
assert "Alice private procedure" in alice_text
assert "bob-only" not in alice_text
assert "Bob private procedure" not in alice_text
assert "bob-only" in bob_text
assert "Bob private procedure" in bob_text
assert "alice-only" not in bob_text
assert "Alice private procedure" not in bob_text
-30
View File
@@ -238,36 +238,6 @@ def test_guide_only_blocks_later_round_document_streaming(monkeypatch):
assert not any(event.get("type") == "doc_stream_delta" for event in events)
def test_guide_only_directive_dominates_workspace_prompt(monkeypatch):
_patch_loop_basics(monkeypatch)
system_prompts = []
async def _fake_stream(_candidates, messages, **kwargs):
system_prompts.append(messages[0]["content"])
yield _delta_chunk("ok")
yield "data: [DONE]\n\n"
monkeypatch.setattr(al, "stream_llm_with_fallback", _fake_stream, raising=False)
policy = build_effective_tool_policy(last_user_message="Do not use tools.")
_collect(
al.stream_agent_loop(
"http://local.test/v1",
"local-model",
[{"role": "user", "content": "Do not use tools."}],
max_rounds=1,
relevant_tools={"bash"},
tool_policy=policy,
workspace="/tmp/project",
)
)
assert system_prompts
assert system_prompts[0].startswith("## GUIDE-ONLY MODE")
assert "ACTIVE WORKSPACE" not in system_prompts[0]
assert "ALWAYS start by exploring" not in system_prompts[0]
def test_guide_only_skips_intent_without_action_nudge(monkeypatch):
_patch_loop_basics(monkeypatch)
-107
View File
@@ -1,107 +0,0 @@
"""Workspace confinement: file tools are hard-bounded to the workspace folder
(layered on upstream's sensitive-path policy); bash runs with cwd there."""
import os
import tempfile
import pytest
from src.tool_execution import _resolve_tool_path_in_workspace, _direct_fallback
def test_workspace_resolver_confines():
ws = tempfile.mkdtemp()
open(os.path.join(ws, "a.txt"), "w").write("x")
real = os.path.realpath(os.path.join(ws, "a.txt"))
# relative path resolves under the workspace
assert _resolve_tool_path_in_workspace(ws, "a.txt") == real
# absolute path inside the workspace is allowed
assert _resolve_tool_path_in_workspace(ws, os.path.join(ws, "a.txt")) == real
# absolute path outside is rejected (sibling temp dir, portable across OSes)
outside = tempfile.mkdtemp()
with pytest.raises(ValueError):
_resolve_tool_path_in_workspace(ws, os.path.join(outside, "x.txt"))
# parent-escape is rejected
with pytest.raises(ValueError):
_resolve_tool_path_in_workspace(ws, os.path.join("..", "..", "escape.txt"))
def test_workspace_resolver_blocks_sensitive():
"""Upstream's sensitive-file deny list still applies inside the workspace."""
ws = tempfile.mkdtemp()
os.makedirs(os.path.join(ws, ".ssh"), exist_ok=True)
with pytest.raises(ValueError):
_resolve_tool_path_in_workspace(ws, ".ssh/authorized_keys")
@pytest.mark.asyncio
async def test_read_write_confined_in_workspace():
ws = tempfile.mkdtemp()
# Write inside the workspace (relative path) succeeds.
res = await _direct_fallback("write_file", "note.txt\nhello", workspace=ws)
assert res["exit_code"] == 0
assert os.path.isfile(os.path.join(ws, "note.txt"))
# Read it back.
res = await _direct_fallback("read_file", "note.txt", workspace=ws)
assert res["exit_code"] == 0 and res["output"] == "hello"
# Reading outside the workspace is rejected (sibling temp dir, portable).
outside = tempfile.mkdtemp()
outside_file = os.path.join(outside, "secret.txt")
open(outside_file, "w").write("nope")
res = await _direct_fallback("read_file", outside_file, workspace=ws)
assert res["exit_code"] == 1 and "outside the workspace" in res["error"]
# Writing outside is rejected (file must not be created).
escape = os.path.join(outside, "_ws_escape.txt")
res = await _direct_fallback("write_file", f"{escape}\nx", workspace=ws)
assert res["exit_code"] == 1 and "outside the workspace" in res["error"]
assert not os.path.exists(escape)
@pytest.mark.asyncio
async def test_subprocess_runs_with_workspace_cwd():
"""bash/python subprocesses run with cwd set to the workspace. Use the
python tool for an OS-agnostic cwd probe (Windows cmd has no `pwd`)."""
ws = tempfile.mkdtemp()
res = await _direct_fallback("python", "import os; print(os.getcwd())", workspace=ws)
assert res["exit_code"] == 0
assert os.path.realpath(res["output"].strip()) == os.path.realpath(ws)
# --- Tools that landed after this PR, now wired into the workspace -----------
@pytest.mark.asyncio
async def test_edit_file_confined_in_workspace():
import json
from src.tool_execution import _do_edit_file
ws = tempfile.mkdtemp()
open(os.path.join(ws, "f.txt"), "w").write("foo bar")
# Edit inside the workspace succeeds.
res = await _do_edit_file(json.dumps(
{"path": "f.txt", "old_string": "foo", "new_string": "baz"}), workspace=ws)
assert res["exit_code"] == 0
assert open(os.path.join(ws, "f.txt")).read() == "baz bar"
# Editing outside the workspace is rejected (sibling temp dir, portable).
outside = tempfile.mkdtemp()
outside_file = os.path.join(outside, "f.txt")
open(outside_file, "w").write("a")
res = await _do_edit_file(json.dumps(
{"path": outside_file, "old_string": "a", "new_string": "b"}), workspace=ws)
assert res["exit_code"] == 1 and "outside the workspace" in res["error"]
@pytest.mark.asyncio
async def test_grep_and_ls_confined_in_workspace():
import json
ws = tempfile.mkdtemp()
open(os.path.join(ws, "doc.txt"), "w").write("hello workspace\n")
# grep with no path searches the workspace root and finds the match.
res = await _direct_fallback("grep", json.dumps({"pattern": "hello"}), workspace=ws)
assert res["exit_code"] == 0 and "doc.txt" in res["output"]
# grep pointed outside the workspace is rejected (sibling temp dir, portable).
outside = tempfile.mkdtemp()
res = await _direct_fallback("grep", json.dumps({"pattern": "x", "path": outside}), workspace=ws)
assert res["exit_code"] == 1 and "outside the workspace" in res["error"]
# ls of the workspace lists its files; ls outside is rejected.
res = await _direct_fallback("ls", "", workspace=ws)
assert res["exit_code"] == 0 and "doc.txt" in res["output"]
res = await _direct_fallback("ls", outside, workspace=ws)
assert res["exit_code"] == 1 and "outside the workspace" in res["error"]