Compare commits

...
Author SHA1 Message Date
RaresKeY 7e28d8b34a chore(release): sync main with dev 2026-09-07 08:06:17 +00:00
nopozandRaresKeY 934d23c0be Merge commit from fork
* fix(security): keep agent file tools out of the app state directory

The agent's read tools (read_file, grep, glob, ls) resolved model-supplied
paths against a root list whose first entry was the whole data directory.
That directory holds the session store, the auth database, the app
encryption key and the settings file, so prompt-injected content could ask
for any of them. No approval prompt stood in the way: reads are classified
read_workspace and pass the untrusted-context gate untouched, which is
correct for reading a workspace and wrong for reading the app's own state.

The agent gets data/agent_workspace/ instead, and the subprocess cwd and
HOME move with it so bash and read_file agree on where scratch files live.

The deny itself is a property of the path, not of the root it arrived
through, because three routes reach the same bytes and closing only the
first leaves the other two working:

  - the default root list
  - a workspace bound at or above the data directory, which vet_workspace
    accepted and chat_routes auto-binds from a path named in the message
  - a tool_path_extra_roots setting covering the data directory

_resolve_search_root also returned the workspace root unchecked when the
path was empty, so a bare ls enumerated the directory whatever the deny
list said. It now resolves that case through the same guards.

A containment rule rather than a filename deny list, so state files added
later are covered without anyone remembering to list them, and so a user's
own settings.json or app.db inside a real workspace is not caught.

Four directories of user content stay readable, because the application
hands their paths to the model and tells it to open them: the chat upload
manifest, downloaded mail attachments, personal docs (which covers the
runbook) and personal uploads.

* fix: enforce state deny during recursive file search

* fix: bound protected filesystem searches

* fix(security): reject inode aliases and workspace redirects

* fix(security): harden partitioned agent searches

* fix(security): report fallback worker exits promptly

* fix(security): clean up search readers and retain relative data roots

---------

Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
2026-09-05 19:21:12 +02:00
nopozandRaresKeY f88e2d1f7f Merge commit from fork
* fix(security): stop API tokens reaching privileged agent tools

A bearer API token resolves to the human who minted it, and minting is admin-only, so every owner-keyed privilege check in the agent path answers "admin". A token issued for a narrow integration therefore reached bash and python with the authority of the account that created it.

Three independent routes to that sink, each closed here.

The token could answer its own tool-approval prompt. An approval records that a person authorized one dangerous action, and a token cannot make that statement, so /api/chat_stream now refuses an approval resume from a bearer caller.

The chat-session grant was reconstructable from caller-supplied message metadata. Two routes persist a metadata blob on the caller's behalf, so the shape of a resolved approval card could be written straight into a transcript and was then read back as authority. The server now signs the grant when it resolves an approval and verifies that signature when reading it back, binding it to the chat and the approval it was issued for. Both routes also drop server-owned keys from an inbound blob.

A run driven by a token inherited its owner's tool set. Such a run is now capped at the non-admin policy regardless of who minted the credential, which holds even where no approval is raised at all.

The human path is unchanged: a browser session still receives the prompt, still approves, and a granted chat-session scope still carries to later turns in that chat.

Scope enforcement across the wider route surface is a separate gap and is not addressed here.

* fix scoped chat delegation boundaries

* fix(auth): reject malformed chat approval signatures

---------

Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
2026-09-05 19:20:49 +02:00
rauljuaandRaul c7a8637475 fix(docker): repair app cache parent ownership (#6158)
* fix(docker): repair app cache parent ownership

* fix(docker): avoid walking mounted model cache

* test(docker): exercise nested cache ownership

---------

Co-authored-by: Raul <9117159+raultcj@users.noreply.github.com>
2026-09-05 18:05:38 +02:00
RaresKeYandnopoz 451900fc15 fix(image): prepare v1.0.3 hotfix (#6167)
* fix(image): pin standalone service model selection

Port the reviewed standalone image-service hardening from dev to the curated release branch and include its focused regressions.

Evidence: all four focused regression tests pass in an isolated runtime, and the staged diff check is clean. Coordinated-disclosure identifiers are intentionally omitted pending publication.

* chore(release): bump version to 1.0.3

Set the canonical application version for the curated hotfix release.

Evidence: APP_VERSION imports as 1.0.3 and the release workflow reads this source. No tag, package, or release artifact is created by this commit.

---------

Co-authored-by: nopoz <bill.lowney@gmail.com>
2026-08-25 10:22:41 +01:00
pewdiepie-archdaemon cf4e240ad1 Merge verified Odysseus fixes 2026-07-23 14:49:08 +00:00
27 changed files with 2636 additions and 103 deletions
+10
View File
@@ -11,6 +11,8 @@ from typing import Dict, List, Any, Optional, TYPE_CHECKING
from src.tool_approval_scopes import (
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
CHAT_SESSION_APPROVAL_DECISION,
CHAT_SESSION_APPROVAL_SIGNATURE_FIELD,
verify_chat_session_grant,
)
if TYPE_CHECKING:
@@ -60,6 +62,14 @@ def _history_grants_chat_session_approval(
ask_user.get("kind") == "tool_approval"
and ask_user.get("resolved") == CHAT_SESSION_APPROVAL_DECISION
and str(ask_user.get("session_id") or "") == expected_session
# Shape proves nothing here: routes that accept a
# caller-supplied metadata blob write into this same history.
and verify_chat_session_grant(
ask_user.get(CHAT_SESSION_APPROVAL_SIGNATURE_FIELD),
expected_session,
ask_user.get("approval_id"),
CHAT_SESSION_APPROVAL_DECISION,
)
):
return True
return False
+10 -1
View File
@@ -96,7 +96,16 @@ repair_bind_mount_ownership() {
# Repair image-owned writable paths without walking into bind-mounted host
# trees, then repair the app-owned mount roots separately.
repair_app_tree_ownership
for dir in /app/data /app/logs /app/.ssh /app/.cache/huggingface /app/.local; do
# Docker creates the parent of the HuggingFace bind mount as root before this
# entrypoint runs. Repair only the parent directory itself so app-user caches
# such as /app/.cache/vllm and /app/.cache/flashinfer can be created without
# recursively walking the mounted model cache.
chown "$PUID:$PGID" /app/.cache 2>/dev/null || true
# The Hugging Face cache can contain hundreds of gigabytes and is a nested
# mount with its own ownership contract. Repair its mount root so new cache
# entries are writable, but never traverse or rewrite existing model files.
chown "$PUID:$PGID" /app/.cache/huggingface 2>/dev/null || true
for dir in /app/data /app/logs /app/.ssh /app/.local; do
repair_bind_mount_ownership "$dir"
done
+7
View File
@@ -14,6 +14,13 @@ import threading
import time
import webbrowser
# PyInstaller multiprocessing children re-enter this executable with a private
# bootstrap argument. Consume it before splash/UI or application imports so a
# spawn-based worker does not relaunch the full desktop application.
if __name__ == "__main__":
import multiprocessing
multiprocessing.freeze_support()
# Define a dummy NullWriter to suppress standard stream crashes (isatty etc.) in GUI mode
class NullWriter:
def write(self, text):
+46 -3
View File
@@ -9,7 +9,7 @@ import logging
from datetime import datetime
from typing import Dict, Any, AsyncGenerator, List, Optional
from fastapi import APIRouter, Request, HTTPException, Form, Query
from fastapi import APIRouter, Request, HTTPException, Form, Query, Depends
from fastapi.responses import StreamingResponse
from pydantic import ValidationError
@@ -40,7 +40,13 @@ from src.foreground_model_routing import (
from src.session_search import search_session_messages
from src.prompt_security import untrusted_context_message
from core.exceptions import SessionNotFoundError
from src.auth_helpers import effective_user, get_current_user
from src.auth_helpers import (
effective_user,
get_current_user,
is_delegated_credential,
require_api_token_scope,
require_chat_api_token_scope,
)
from routes.session_routes import _verify_session_owner
from routes.document_helpers import _owner_session_filter
from core.database import SessionLocal, get_session_mode, set_session_mode
@@ -68,6 +74,8 @@ from src.tool_policy import (
web_search_enabled_for_turn,
)
from src.tool_approvals import tool_approval_store
from src.tool_approval_scopes import stamp_chat_session_grant
from src.tool_security import delegated_credential_blocked_tools
logger = logging.getLogger(__name__)
@@ -89,6 +97,23 @@ def _stream_failure_status(chunk: str) -> Optional[int]:
return None
def _reject_delegated_tool_approval(request: Request) -> None:
"""Refuse an approval answered by a bearer API token.
A tool approval records that a HUMAN authorized one dangerous action. A
token is a delegated credential handed to an integration, so when it
answers the prompt it triggered, nobody is asked and the gate collapses
into an extra round trip. Owner and session already match here: the token
is answering on behalf of the account that minted it.
"""
if is_delegated_credential(request):
raise HTTPException(
403,
"Tool approvals require an interactive session. "
"API tokens cannot authorize a gated action.",
)
def _mark_tool_approval_resolved(sess, approval_id: Any, decision: Any) -> bool:
"""Persist a consumed approval decision on its existing tool event."""
@@ -113,6 +138,11 @@ def _mark_tool_approval_resolved(sess, approval_id: Any, decision: Any) -> bool:
if str(ask_user.get("approval_id") or "") != approval_key:
continue
ask_user["resolved"] = normalized_decision
stamp_chat_session_grant(
ask_user,
getattr(sess, "id", ""),
normalized_decision,
)
message_id = metadata.get("_db_id")
resolved_metadata = {
key: value for key, value in metadata.items() if key != "_db_id"
@@ -730,13 +760,17 @@ def setup_chat_routes(
webhook_manager=None,
skills_manager=None,
) -> APIRouter:
router = APIRouter(tags=["chat"])
router = APIRouter(
tags=["chat"],
dependencies=[Depends(require_chat_api_token_scope)],
)
# ------------------------------------------------------------------ #
# POST /api/chat (non-streaming)
# ------------------------------------------------------------------ #
@router.post("/api/chat", response_model=Dict[str, Any])
async def chat_endpoint(request: Request, chat_request: ChatRequest) -> Dict[str, Any]:
require_api_token_scope(request, "chat")
_set_user_time_from_request(request)
message = chat_request.message
@@ -927,6 +961,7 @@ def setup_chat_routes(
# ------------------------------------------------------------------ #
@router.post("/api/chat_stream")
async def chat_stream(request: Request) -> StreamingResponse:
require_api_token_scope(request, "chat")
body = None
try:
if request.headers.get("content-type", "").startswith("application/json"):
@@ -1125,6 +1160,7 @@ def setup_chat_routes(
sess = session_manager.get_session(session)
owner = effective_user(request)
if tool_approval_id:
_reject_delegated_tool_approval(request)
pending_tool_approval = tool_approval_store.peek(tool_approval_id)
normalized_owner = str(owner or "").strip().casefold()
if (
@@ -1442,6 +1478,12 @@ def setup_chat_routes(
# Build disabled-tools set from frontend toggles + user privileges
disabled_tools = set()
# Minting is admin-only, so every owner-keyed check below answers
# "admin" for a token. Cap it at the non-admin policy instead.
# stream_agent_loop repeats this from delegated_credential.
_delegated_credential = is_delegated_credential(request)
if _delegated_credential:
disabled_tools.update(delegated_credential_blocked_tools())
# Only disable bash when the caller *explicitly* set it to a falsy
# value. When unset (None), defer to per-user privilege checks below.
# Web search is per-turn opt-in: either the chat pre-search setting
@@ -2327,6 +2369,7 @@ def setup_chat_routes(
uploaded_files=ctx.uploaded_files,
defer_context_shaping=_foreground_policy.enabled,
external_untrusted_context_seen=external_untrusted_context_seen,
delegated_credential=_delegated_credential,
exact_approval=exact_tool_approval,
):
if chunk.startswith("data: ") and not chunk.startswith("data: [DONE]"):
+8 -4
View File
@@ -6,13 +6,14 @@ import logging
import re
from typing import Dict, Any, Optional
from fastapi import APIRouter, Request, HTTPException
from fastapi import APIRouter, Request, HTTPException, Depends
from core.models import ChatMessage
from core.database import SessionLocal, ChatMessage as DbChatMessage, Session as DbSession
from src.auth_helpers import effective_user
from src.auth_helpers import effective_user, require_chat_api_token_scope
from src.topic_analyzer import analyze_topics
from src.upload_handler import reserve_message_upload_references
from src.tool_approval_scopes import sanitize_client_message_metadata
from routes.session_routes import (
_message_role,
_message_text,
@@ -101,7 +102,10 @@ def _merge_continue_rows_to_delete(db_messages, db1, db2):
def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
router = APIRouter(tags=["history"])
router = APIRouter(
tags=["history"],
dependencies=[Depends(require_chat_api_token_scope)],
)
def _reserve_message_uploads(
request: Request,
@@ -268,7 +272,7 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
content = body.get("content", "")
if not content:
raise HTTPException(400, "content is required")
metadata = body.get("metadata")
metadata = sanitize_client_message_metadata(body.get("metadata"))
_reserve_message_uploads(request, content, metadata)
msg = ChatMessage(role=role, content=content, metadata=metadata)
session_manager.add_message(session_id, msg)
+44 -4
View File
@@ -4,17 +4,24 @@ import html
import json
import uuid
from datetime import datetime
from fastapi import APIRouter, Form, HTTPException, Response, Request
from fastapi import APIRouter, Form, HTTPException, Response, Request, Depends
import logging
from core.session_manager import SessionManager
from core.models import ChatMessage
from src.request_models import SessionResponse
from core.database import Session as DbSession, SessionLocal, Document, GalleryImage, utcnow_naive
from src.auth_helpers import effective_user, _auth_disabled, owner_filter
from src.auth_helpers import (
effective_user,
_auth_disabled,
owner_filter,
is_delegated_credential,
require_chat_api_token_scope,
)
from src.session_image_cleanup import _generated_image_path_for_cleanup, session_image_refs
from src.session_actions import is_session_recently_active
from src.upload_handler import reserve_message_upload_references
from src.tool_approval_scopes import sanitize_client_message_metadata
def _sanitize_export_filename(name: str) -> str:
@@ -124,9 +131,15 @@ def _verify_session_owner(request: Request, session_id: str, session_manager=Non
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["sessions"])
router = APIRouter(
prefix="/api",
tags=["sessions"],
dependencies=[Depends(require_chat_api_token_scope)],
)
def _current_user_is_admin(request: Request, user: str | None) -> bool:
if is_delegated_credential(request):
return False
if not user:
return False
auth_mgr = getattr(request.app.state, "auth_manager", None)
@@ -157,6 +170,22 @@ def _reject_raw_endpoint_url_for_non_admin(
raise HTTPException(403, "Choose a registered model endpoint")
def _reject_delegated_session_options(
request: Request,
*,
skip_validation: bool = False,
api_key: str | None = None,
) -> None:
"""Keep bearer credentials from exercising interactive-admin options."""
if is_delegated_credential(request) and (
skip_validation or bool((api_key or "").strip())
):
raise HTTPException(
403,
"API tokens cannot supply endpoint credentials or skip endpoint validation",
)
def _persist_session_headers(session_id: str, headers: dict | None) -> None:
"""Persist endpoint auth headers for DB-backed session metadata."""
db = SessionLocal()
@@ -340,6 +369,11 @@ def setup_session_routes(
):
skip_val = str(skip_validation).lower() == "true"
user = effective_user(request)
_reject_delegated_session_options(
request,
skip_validation=skip_val,
api_key=api_key,
)
endpoint_api_key = ""
endpoint_base_url = ""
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
@@ -564,7 +598,11 @@ def setup_session_routes(
except (AttributeError, TypeError, ValueError) as exc:
raise HTTPException(400, "Invalid message attachment metadata") from exc
for m in messages:
sess.add_message(ChatMessage(m["role"], m["content"], metadata=m.get("metadata")))
sess.add_message(ChatMessage(
m["role"],
m["content"],
metadata=sanitize_client_message_metadata(m.get("metadata")),
))
session_manager.save_sessions()
return {"ok": True, "count": len(messages)}
@@ -906,6 +944,8 @@ def setup_session_routes(
model: str = Form("gpt-4o"),
rag: str = Form(None)
):
if is_delegated_credential(request):
raise HTTPException(403, "This session type requires an interactive session")
if not OPENAI_API_KEY:
raise HTTPException(400, "Server missing OPENAI_API_KEY")
sid = str(uuid.uuid4())
+2 -1
View File
@@ -16,7 +16,7 @@ sys.path.insert(0, BASE_DIR)
from src.constants import (
DATA_DIR, AUTH_FILE, UPLOAD_DIR, PERSONAL_DIR, PERSONAL_UPLOADS_DIR,
TTS_CACHE_DIR, GENERATED_IMAGES_DIR, DEEP_RESEARCH_DIR, CHROMA_DIR,
RAG_DIR, MEMORY_VECTORS_DIR, PASSWORD_MIN_LENGTH,
RAG_DIR, MEMORY_VECTORS_DIR, AGENT_WORKSPACE_DIR, PASSWORD_MIN_LENGTH,
)
from core.auth import RESERVED_USERNAMES
@@ -31,6 +31,7 @@ DIRS = [
CHROMA_DIR,
RAG_DIR,
MEMORY_VECTORS_DIR,
AGENT_WORKSPACE_DIR,
os.path.join(BASE_DIR, "logs"),
]
+11
View File
@@ -33,6 +33,7 @@ from src.settings import get_setting
from src.prompt_security import untrusted_context_message
from src.tool_security import (
blocked_tools_for_owner,
delegated_credential_blocked_tools,
email_tool_policy_names,
plan_mode_disabled_tools,
)
@@ -3443,6 +3444,7 @@ async def stream_agent_loop(
uploaded_files: Optional[List[Dict]] = None,
workload: str = "foreground",
external_untrusted_context_seen: bool = False,
delegated_credential: bool = False,
exact_approval: Optional[ExactToolApproval] = None,
_is_teacher_run: bool = False,
history_session=None,
@@ -3471,6 +3473,7 @@ async def stream_agent_loop(
approval_gate_bypassed=bool(
exact_approval and exact_approval.allow_remaining_actions
),
delegated_credential=bool(delegated_credential),
)
mcp_mgr = get_mcp_manager()
prep_timings: Dict[str, float] = {}
@@ -3490,6 +3493,10 @@ async def stream_agent_loop(
mcp_mgr = None
guide_only = bool(tool_policy and tool_policy.mode == "guide_only")
public_blocked_tools = blocked_tools_for_owner(owner)
if delegated_credential:
# owner is the admin who minted the token, so the call above returns
# nothing. Cap the run regardless of who it acts for.
public_blocked_tools.update(delegated_credential_blocked_tools())
if public_blocked_tools:
disabled_tools.update(public_blocked_tools)
# MCP tools are namespaced dynamically, so hide all MCP schemas for
@@ -6434,6 +6441,10 @@ async def stream_agent_loop(
tool_policy=tool_policy,
active_document=active_document,
active_email=active_email,
external_untrusted_context_seen=(
run_security.external_untrusted_context_seen
),
delegated_credential=delegated_credential,
):
yield evt
except Exception as _esc_err:
+428 -61
View File
@@ -3,8 +3,8 @@ import json
import os
import re
import difflib
import fnmatch
import shutil
import time
from typing import Optional, Dict, Any, Tuple, List
from src.constants import MAX_READ_CHARS, MAX_DIFF_LINES, MAX_OUTPUT_CHARS
@@ -16,6 +16,8 @@ _CODENAV_SKIP_DIRS = frozenset({
})
_CODENAV_MAX_HITS = 200
_CODENAV_MAX_LINE = 400
_GREP_TIMEOUT_SECONDS = 20
_GREP_STDERR_PREFIX = 20_000
def _glob_to_regex(pat: str) -> "re.Pattern":
@@ -42,6 +44,113 @@ def _glob_to_regex(pat: str) -> "re.Pattern":
i += 1
return re.compile("".join(out))
def _python_grep_worker(payload: dict, output_queue) -> None:
"""Spawn-safe fallback grep worker used when ripgrep is unavailable.
Keep this at module scope: a frozen Windows executable cannot safely be
relaunched as ``sys.executable -c ...``, while multiprocessing can invoke a
top-level target through its frozen-process bootstrap.
"""
try:
flags = re.IGNORECASE if payload["ignore_case"] else 0
try:
regex = re.compile(payload["pattern"], flags)
glob_regex = (
_glob_to_regex(payload["glob"].replace("\\", "/"))
if payload["glob"]
else None
)
except re.error as exc:
output_queue.put(("error", f"grep: bad pattern: {exc}"))
return
requested_root = payload["root"]
skip_dirs = set(payload["skip_dirs"])
sensitive = {name.casefold() for name in payload["sensitive_names"]}
max_hits = payload["max_hits"]
hits = 0
def within(path: str, root: str) -> bool:
try:
return os.path.commonpath(
[os.path.normcase(path), os.path.normcase(root)]
) == os.path.normcase(root)
except ValueError:
return False
def safe_file(path: str, target: str) -> Optional[str]:
if os.path.islink(path):
return None
canonical = os.path.realpath(path)
if not within(canonical, requested_root) or not within(canonical, target):
return None
parts = [part.casefold() for part in canonical.split(os.sep)]
if any(part in sensitive for part in parts):
return None
try:
if not os.path.isfile(canonical) or os.stat(canonical).st_nlink > 1:
return None
except OSError:
return None
return canonical
for target in payload["targets"]:
if hits >= max_hits:
break
if os.path.isfile(target):
file_iter = iter((target,))
else:
def walk_files():
for directory, dirnames, filenames in os.walk(
target, followlinks=False
):
dirnames[:] = [
name
for name in dirnames
if name not in skip_dirs
and name.casefold() not in sensitive
and not os.path.islink(os.path.join(directory, name))
]
for name in filenames:
yield os.path.join(directory, name)
file_iter = walk_files()
for candidate in file_iter:
path = safe_file(candidate, target)
if path is None:
continue
relative = os.path.relpath(path, requested_root).replace(os.sep, "/")
if glob_regex and not (
glob_regex.fullmatch(relative)
or glob_regex.fullmatch(os.path.basename(path))
):
continue
try:
with open(path, "r", encoding="utf-8", errors="strict") as handle:
for number, line in enumerate(handle, 1):
if regex.search(line):
output_queue.put((
"match",
path,
number,
line.rstrip()[:_CODENAV_MAX_LINE],
))
hits += 1
if hits >= max_hits:
break
except (UnicodeDecodeError, OSError):
continue
if hits >= max_hits:
break
output_queue.put(("done",))
except BaseException as exc:
try:
output_queue.put(("error", f"grep: fallback worker failed: {exc}"))
except BaseException:
pass
def _unified_diff(old: str, new: str, path: str) -> Optional[Dict[str, Any]]:
if old == new:
return None
@@ -407,7 +516,11 @@ def _apply_patch_hunks(original: str, hunks: List[List[str]], label: str) -> str
class LsTool:
async def execute(self, content: str, ctx: dict) -> dict:
from src.tool_execution import _resolve_tool_path, _resolve_search_root, _truncate
from src.tool_execution import (
_is_denied_tool_path,
_resolve_search_root,
_truncate,
)
raw_path = ""
_s = (content or "").strip()
if _s.startswith("{"):
@@ -431,6 +544,8 @@ class LsTool:
for entry in it:
if entry.name.startswith("."):
continue
if _is_denied_tool_path(os.path.realpath(entry.path)):
continue
try:
is_dir = entry.is_dir(follow_symlinks=False)
size = entry.stat(follow_symlinks=False).st_size if not is_dir else 0
@@ -458,7 +573,8 @@ class GlobTool:
async def execute(self, content: str, ctx: dict) -> dict:
from src.tool_execution import (
_SENSITIVE_BASENAMES,
_is_sensitive_path,
_can_traverse_tool_path,
_is_denied_tool_path,
_resolve_tool_path,
_resolve_search_root,
_truncate,
@@ -507,7 +623,7 @@ class GlobTool:
# .ssh/id_rsa, …) falls through to the walk, which skips it —
# otherwise glob would surface secret paths that read_file /
# grep already refuse to touch.
if inside and os.path.exists(cand) and not _is_sensitive_path(cand):
if inside and os.path.exists(cand) and not _is_denied_tool_path(cand):
return [cand], None
# Literal not at exact path — fall through to walk so
# e.g. "foo.py" still matches at any depth (like rglob).
@@ -517,13 +633,18 @@ class GlobTool:
cap = _CODENAV_MAX_HITS * 5
try:
for dp, dns, fns in os.walk(base):
if not _can_traverse_tool_path(os.path.realpath(dp)):
dns[:] = []
continue
# Prune skipped dirs before descending (unlike rglob which
# descends first then filters — fatal on large node_modules).
# Sensitive dirs (.ssh, .gnupg, …) are pruned too so glob
# never enumerates the keys/tokens inside them.
dns[:] = [
d for d in dns
if d not in _CODENAV_SKIP_DIRS and d not in _SENSITIVE_BASENAMES
if d not in _CODENAV_SKIP_DIRS
and d not in _SENSITIVE_BASENAMES
and _can_traverse_tool_path(os.path.realpath(os.path.join(dp, d)))
]
for name in fns + dns:
full = os.path.join(dp, name)
@@ -531,7 +652,7 @@ class GlobTool:
if regex.fullmatch(rel) or regex.fullmatch(name):
# Skip deny-listed sensitive files (.env, id_rsa,
# known_hosts, …) the same way grep does.
if _is_sensitive_path(os.path.realpath(full)):
if _is_denied_tool_path(os.path.realpath(full)):
continue
try:
mtime = os.stat(full).st_mtime
@@ -558,9 +679,12 @@ class GlobTool:
class GrepTool:
async def execute(self, content: str, ctx: dict) -> dict:
from src.tool_execution import (
_SENSITIVE_BASENAMES,
_SENSITIVE_FILE_PATTERNS,
_agent_readable_data_subdirs,
_is_denied_tool_path,
_is_sensitive_path,
_resolve_tool_path,
_path_within,
_resolve_search_root,
_truncate,
)
@@ -589,64 +713,307 @@ class GrepTool:
return {"error": f"grep: {e}", "exit_code": 1}
def _grep():
import re as _re
import shutil
import multiprocessing
import queue
import subprocess
import threading
from src.constants import DATA_DIR
rg = shutil.which("rg")
if rg:
cmd = [rg, "--line-number", "--no-heading", "--color=never",
"--max-count", str(max_hits)]
if ignore_case:
cmd.append("--ignore-case")
if glob_pat:
cmd += ["--glob", glob_pat]
# --iglob (not --glob) so the exclusion is case-insensitive:
# on a case-insensitive filesystem "ID_RSA"/"Known_Hosts"
# resolve to the same secret as their lowercase forms, and the
# Python fallback below already folds case via _is_sensitive_path.
for _pat in _SENSITIVE_FILE_PATTERNS:
cmd += ["--iglob", f"!*{_pat}*"]
for _d in _CODENAV_SKIP_DIRS:
cmd += ["--glob", f"!**/{_d}/**"]
cmd += ["--regexp", pattern, root]
real_root = os.path.realpath(root)
data_dir = os.path.realpath(DATA_DIR)
spans_state = _path_within(data_dir, real_root)
def is_top_level_safe(path: str, *, partition_generated: bool) -> bool:
lexical = os.path.abspath(path)
if os.path.islink(lexical):
return False
canonical = os.path.realpath(lexical)
if not _path_within(canonical, real_root):
return False
if partition_generated and os.path.basename(lexical) in _CODENAV_SKIP_DIRS:
return False
if _is_sensitive_path(canonical) or _is_denied_tool_path(canonical):
return False
return True
def safe_targets() -> tuple[list[str], Optional[str]]:
candidates: list[tuple[str, bool]] = []
if not spans_state:
# Preserve direct-root compatibility: skip-directory policy
# prunes descendants, but an explicitly requested allowed
# root named node_modules remains searchable.
candidates.append((real_root, False))
else:
current = real_root
if current != data_dir:
for part in os.path.relpath(data_dir, current).split(os.sep):
try:
with os.scandir(current) as entries:
for entry in entries:
if entry.name != part:
# Reject a sibling link lexically before
# canonicalizing or treating it as a target.
if entry.is_symlink():
continue
candidates.append((entry.path, True))
except OSError as exc:
return [], f"grep: {exc}"
current = os.path.join(current, part)
for readable in _agent_readable_data_subdirs():
if (
_path_within(readable, data_dir)
and _path_within(readable, real_root)
and os.path.exists(readable)
):
candidates.append((readable, True))
targets: list[str] = []
seen: set[str] = set()
for candidate, partition_generated in candidates:
if not is_top_level_safe(
candidate, partition_generated=partition_generated
):
continue
canonical = os.path.realpath(candidate)
if canonical not in seen:
seen.add(canonical)
targets.append(canonical)
return targets, None
targets, target_error = safe_targets()
if target_error:
return None, target_error
base = real_root if os.path.isdir(real_root) else os.path.dirname(real_root)
deadline = time.monotonic() + _GREP_TIMEOUT_SECONDS
lines: list[str] = []
def parse_rg_result(raw: str) -> Optional[str]:
try:
import subprocess
p = subprocess.run(cmd, capture_output=True, text=True, timeout=20)
lines = [ln for ln in (p.stdout or "").splitlines() if ln][:max_hits]
return lines, None
except subprocess.TimeoutExpired:
return None, "grep: timed out"
except Exception as _e:
return None, f"grep: {_e}"
try:
rx = _re.compile(pattern, _re.IGNORECASE if ignore_case else 0)
except _re.error as _e:
return None, f"grep: bad pattern: {_e}"
hits = []
if os.path.isfile(root):
file_iter = [root]
else:
file_iter = []
for dp, dns, fns in os.walk(root):
dns[:] = [d for d in dns if d not in _CODENAV_SKIP_DIRS]
for fn in fns:
if glob_pat and not fnmatch.fnmatch(fn, glob_pat):
record = json.loads(raw)
except (TypeError, json.JSONDecodeError):
return None
if record.get("type") != "match":
return None
data = record.get("data") or {}
path = (data.get("path") or {}).get("text")
text_value = (data.get("lines") or {}).get("text")
number = data.get("line_number")
if not isinstance(path, str) or not isinstance(text_value, str):
return None
absolute = path if os.path.isabs(path) else os.path.join(base, path)
canonical = os.path.realpath(absolute)
if not _path_within(canonical, real_root) or _is_denied_tool_path(canonical):
return None
return f"{os.path.abspath(absolute)}:{number}:{text_value.rstrip()[:_CODENAV_MAX_LINE]}"
def run_rg(cmd: list[str]) -> Optional[str]:
try:
process = subprocess.Popen(
cmd,
cwd=base,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
bufsize=1,
)
except Exception as exc:
return f"grep: {exc}"
output: queue.Queue[Optional[str]] = queue.Queue(maxsize=max_hits + 2)
stderr_prefix: list[str] = []
stderr_size = 0
stop_reader = threading.Event()
def enqueue_stdout(value: Optional[str]) -> bool:
# The consumer stops at the result cap or deadline. Never
# leave a producer blocked on its bounded queue afterward.
while not stop_reader.is_set():
try:
output.put(value, timeout=0.05)
return True
except queue.Full:
continue
file_iter.append(os.path.join(dp, fn))
for fp in file_iter:
if len(hits) >= max_hits:
break
if _is_sensitive_path(os.path.realpath(fp)):
continue
return False
def read_stdout() -> None:
assert process.stdout is not None
try:
for line in process.stdout:
if not enqueue_stdout(line.rstrip("\n")):
break
finally:
enqueue_stdout(None)
def read_stderr() -> None:
nonlocal stderr_size
assert process.stderr is not None
while True:
chunk = process.stderr.read(4096)
if not chunk:
break
if stderr_size < _GREP_STDERR_PREFIX:
kept = chunk[:_GREP_STDERR_PREFIX - stderr_size]
stderr_prefix.append(kept)
stderr_size += len(kept)
stdout_thread = threading.Thread(target=read_stdout, daemon=True)
stderr_thread = threading.Thread(target=read_stderr, daemon=True)
stdout_thread.start()
stderr_thread.start()
timed_out = False
capped = False
try:
with open(fp, "r", encoding="utf-8", errors="strict") as f:
for i, line in enumerate(f, 1):
if rx.search(line):
hits.append(f"{fp}:{i}:{line.rstrip()[:_CODENAV_MAX_LINE]}")
if len(hits) >= max_hits:
break
except (UnicodeDecodeError, OSError):
continue
return hits, None
while len(lines) < max_hits:
remaining = deadline - time.monotonic()
if remaining <= 0:
timed_out = True
break
try:
raw = output.get(timeout=remaining)
except queue.Empty:
timed_out = True
break
if raw is None:
break
parsed = parse_rg_result(raw)
if parsed and parsed not in lines:
lines.append(parsed)
capped = len(lines) >= max_hits
finally:
stop_reader.set()
if (timed_out or capped) and process.poll() is None:
process.terminate()
try:
remaining = max(0.01, deadline - time.monotonic())
return_code = process.wait(timeout=min(1, remaining))
except subprocess.TimeoutExpired:
process.kill()
return_code = process.wait()
stdout_thread.join()
stderr_thread.join()
if timed_out:
return "grep: timed out"
if not capped and return_code not in (0, 1):
detail = "".join(stderr_prefix).strip()
return f"grep: {detail or f'process exited {return_code}'}"
return None
if rg:
# Validate even when policy filtering leaves no search targets.
if not targets:
error = run_rg([rg, "--json", "--no-config", "--regexp", pattern])
return (None, error) if error else ([], None)
relative_targets = [os.path.relpath(target, base) for target in targets]
for offset in range(0, len(relative_targets), 128):
if len(lines) >= max_hits:
break
cmd = [
rg, "--json", "--no-config", "--no-follow",
"--max-count", str(max_hits - len(lines)),
"--max-columns", str(_CODENAV_MAX_LINE),
"--max-columns-preview",
]
if ignore_case:
cmd.append("--ignore-case")
if glob_pat:
cmd += ["--glob", glob_pat]
for sensitive_pattern in _SENSITIVE_FILE_PATTERNS:
cmd += ["--iglob", f"!{sensitive_pattern}"]
for skipped_dir in _CODENAV_SKIP_DIRS:
cmd += ["--glob", f"!**/{skipped_dir}/**"]
cmd += ["--regexp", pattern, "--", *relative_targets[offset:offset + 128]]
error = run_rg(cmd)
if error:
return None, error
return lines, None
# This runs inside asyncio.to_thread(), so forking would clone a
# multithreaded process and can deadlock. Spawn is platform-safe and
# PyInstaller-compatible via launcher's early freeze_support().
payload = {
"root": real_root,
"targets": targets,
"pattern": pattern,
"ignore_case": ignore_case,
"glob": glob_pat,
"max_hits": max_hits,
"skip_dirs": tuple(_CODENAV_SKIP_DIRS),
"sensitive_names": tuple(
set(_SENSITIVE_BASENAMES) | set(_SENSITIVE_FILE_PATTERNS)
),
}
try:
context = multiprocessing.get_context("spawn")
output_queue = context.Queue(maxsize=max_hits + 2)
worker = context.Process(
target=_python_grep_worker, args=(payload, output_queue)
)
worker.start()
except Exception as exc:
try:
output_queue.close()
except (NameError, OSError, ValueError):
pass
return None, f"grep: could not start fallback worker: {exc}"
error = None
completed = False
try:
while len(lines) < max_hits:
remaining = deadline - time.monotonic()
if remaining <= 0:
error = "grep: timed out"
break
try:
# Keep queue waits short enough to observe a spawn
# worker that dies during bootstrap/import before it
# can enqueue either an error or the done sentinel.
record = output_queue.get(timeout=min(0.05, remaining))
except queue.Empty:
if worker.is_alive():
continue
worker.join(timeout=0)
try:
# A multiprocessing queue's feeder can make the
# final record visible at process-exit time. Give
# that record precedence over the exit status.
remaining = deadline - time.monotonic()
record = output_queue.get(
timeout=min(0.05, max(0, remaining))
)
except queue.Empty:
error = f"grep: fallback worker exited {worker.exitcode}"
break
if record[0] == "done":
completed = True
break
if record[0] == "error":
error = record[1]
break
_, path, number, text_value = record
canonical = os.path.realpath(path)
if not _path_within(canonical, real_root) or _is_denied_tool_path(canonical):
continue
rendered = f"{path}:{number}:{text_value}"
if rendered not in lines:
lines.append(rendered)
finally:
if completed:
worker.join(timeout=min(1, max(0.01, deadline - time.monotonic())))
if worker.is_alive():
worker.terminate()
worker.join(timeout=1)
if worker.is_alive():
worker.kill()
worker.join()
output_queue.close()
if error:
return None, error
if worker.exitcode not in (0, None) and len(lines) < max_hits:
return None, f"grep: fallback worker exited {worker.exitcode}"
return lines, None
lines, err = await asyncio.to_thread(_grep)
if err:
+31 -2
View File
@@ -2,10 +2,11 @@
"""Initialize all application components and dependencies."""
import os
import logging
import stat
from typing import Dict, Any
from src.constants import (
DATA_DIR, PERSONAL_DIR, RUNBOOK_DIR, UPLOAD_DIR,
DATA_DIR, PERSONAL_DIR, RUNBOOK_DIR, UPLOAD_DIR, AGENT_WORKSPACE_DIR,
SESSIONS_FILE, DEFAULT_HOST, OPENAI_API_KEY
)
from src.memory import MemoryManager
@@ -30,7 +31,35 @@ def create_directories():
"""Create necessary directories if they don't exist."""
for directory in (DATA_DIR, PERSONAL_DIR, RUNBOOK_DIR, UPLOAD_DIR):
os.makedirs(directory, exist_ok=True)
# The model-controlled workspace must be a real child of DATA_DIR. Never
# follow a pre-existing symlink here: it would silently move the default
# native-file root outside the application volume before any resolver runs.
data_root = os.path.realpath(os.path.abspath(os.path.expanduser(DATA_DIR)))
workspace = os.path.abspath(os.path.expanduser(AGENT_WORKSPACE_DIR))
expected_workspace = os.path.join(data_root, "agent_workspace")
# Validate the real parent so a supported DATA_DIR bind/symlink works, but
# require the fixed internal carve-out name and reject a link at the model-
# controlled workspace entry itself.
if (
os.path.basename(workspace) != "agent_workspace"
or os.path.realpath(os.path.dirname(workspace)) != data_root
):
raise RuntimeError("agent workspace must be the canonical child of DATA_DIR")
if os.path.lexists(workspace):
mode = os.lstat(workspace).st_mode
if stat.S_ISLNK(mode) or not stat.S_ISDIR(mode):
raise RuntimeError("agent workspace must be a real directory")
else:
os.mkdir(workspace, 0o700)
resolved_workspace = os.path.realpath(workspace)
if resolved_workspace != expected_workspace:
raise RuntimeError("agent workspace must be the canonical child of DATA_DIR")
try:
os.chmod(workspace, 0o700)
except OSError:
pass
def initialize_managers(base_dir: str, rag_manager=None) -> Dict[str, Any]:
"""
Initialize all manager and handler instances.
+39
View File
@@ -41,6 +41,45 @@ def _is_api_token_request(request: Request) -> bool:
return bool(getattr(request.state, "api_token", False))
def is_delegated_credential(request: Request) -> bool:
"""Whether this request arrived on a credential acting FOR a human.
A bearer API token is minted by a person and then handed to something
else: an integration, a script, a third party. :func:`effective_user`
resolves it back to that person for ownership and attribution, which is
correct for data but wrong for authority. Only admins can mint tokens, so
every token resolves to an admin, and any gate that asks "is the owner an
admin?" answers yes for a credential the owner has given away.
Security decisions about what the AGENT may do should ask this instead, so
a token cannot inherit the shell merely because its owner could use one.
"""
return _is_api_token_request(request)
def require_api_token_scope(request: Request, scope: str) -> Optional[str]:
"""Require ``scope`` when the request is authenticated by an API token.
Browser sessions are unaffected. Scoped bearer routes use this before
touching owner data so resolving the token back to its owner never also
grants the owner's interactive-session authority.
"""
if not _is_api_token_request(request):
return get_current_user(request)
scopes = set(getattr(request.state, "api_token_scopes", []) or [])
if scope not in scopes:
raise HTTPException(403, f"API token missing required scope: {scope}")
owner = getattr(request.state, "api_token_owner", None)
if not owner:
raise HTTPException(403, "API token has no owner")
return owner
def require_chat_api_token_scope(request: Request) -> Optional[str]:
"""FastAPI dependency for chat/session/history bearer surfaces."""
return require_api_token_scope(request, "chat")
def require_authenticated_request(request: Request) -> str:
"""Allow either a browser session or a valid bearer API token.
+5
View File
@@ -54,6 +54,11 @@ GALLERY_DIR = os.path.join(DATA_DIR, "gallery")
GALLERY_UPLOADS_DIR = os.path.join(DATA_DIR, "gallery_uploads")
MEMORY_VECTORS_DIR = os.path.join(DATA_DIR, "memory_vectors")
# The only part of DATA_DIR the agent's file tools and subprocesses may touch.
# Everything else under DATA_DIR is application state (session store, auth
# database, encryption key, settings), and the agent has no business reading it.
AGENT_WORKSPACE_DIR = os.path.join(DATA_DIR, "agent_workspace")
# Paths with an intentional dedicated env override, defaulting under DATA_DIR.
MAIL_ATTACHMENTS_DIR = os.getenv("ODYSSEUS_MAIL_ATTACHMENTS_DIR", os.path.join(DATA_DIR, "mail-attachments"))
# `or` (not os.getenv's default arg) so a PRESENT-but-EMPTY value falls back to
+4
View File
@@ -524,6 +524,8 @@ async def run_teacher_inline(
tool_policy: Any = None,
active_document: Any = None,
active_email: Optional[Dict[str, str]] = None,
external_untrusted_context_seen: bool = False,
delegated_credential: bool = False,
):
"""Async generator. Yields SSE event strings.
@@ -636,6 +638,8 @@ async def run_teacher_inline(
tool_policy=tool_policy,
active_document=active_document,
active_email=active_email,
external_untrusted_context_seen=external_untrusted_context_seen,
delegated_credential=delegated_credential,
_is_teacher_run=True,
):
# Swallow teacher's own [DONE] — outer loop emits the real one
+125
View File
@@ -2,7 +2,12 @@
from __future__ import annotations
import hmac
import logging
from enum import Enum
from hashlib import sha256
logger = logging.getLogger(__name__)
# Keep the existing wire values so the current route and no-build frontend do
@@ -16,6 +21,126 @@ DENY_APPROVAL_DECISION = "deny"
# session history contains a matching, resolved chat-session approval.
CHAT_SESSION_APPROVAL_CONTEXT_MARKER = "_tool_approval_chat_session_granted"
# The server's proof that IT resolved this approval. More than one route
# writes caller-supplied metadata into session history, so a client can write
# the shape of a resolved card directly; only the server can produce this.
CHAT_SESSION_APPROVAL_SIGNATURE_FIELD = "_server_grant"
def _grant_key() -> bytes | None:
"""Key material for grant signatures, or None when it is unavailable.
Reuses the persistent application key so a grant survives a restart the
way the transcript holding it does.
"""
try:
from src.secret_storage import _load_or_create_key
return _load_or_create_key()
except Exception as exc:
logger.warning("Tool approval grant key unavailable: %s", exc)
return None
def sign_chat_session_grant(
session_id: object,
approval_id: object,
decision: object,
) -> str | None:
"""Return the server's signature for one resolved chat-session grant."""
key = _grant_key()
if key is None:
return None
payload = "\x00".join(
(
str(session_id or ""),
str(approval_id or ""),
str(decision or "").strip().lower(),
)
)
return hmac.new(key, payload.encode("utf-8"), sha256).hexdigest()
# Message-metadata keys the server writes and a caller never should. Both are
# read back as authority: ``tool_events`` carries the approval cards, and the
# context marker is projected onto a turn once a grant is found.
_SERVER_OWNED_METADATA_KEYS = (
"tool_events",
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
)
def sanitize_client_message_metadata(metadata):
"""Drop server-owned keys from a caller-supplied message metadata blob.
Routes that persist a message on the caller's behalf accept this blob
verbatim, which lets a caller write the shape of a resolved approval into
its own transcript. The grant check verifies a signature, so this is not
the control that closes that path; it keeps the state out of the
transcript in the first place. Anything else in the blob is left alone.
"""
if not isinstance(metadata, dict):
return metadata
if not any(key in metadata for key in _SERVER_OWNED_METADATA_KEYS):
return metadata
return {
key: value
for key, value in metadata.items()
if key not in _SERVER_OWNED_METADATA_KEYS
}
def stamp_chat_session_grant(
ask_user: dict,
session_id: object,
decision: object,
) -> None:
"""Record the server's grant on a card it has just resolved.
Call this only from the server-side resolve path. A decision that does not
grant chat-session scope leaves no signature behind, so downgrading a
``deny`` to an ``approve`` in the transcript does not carry a usable one.
"""
if not isinstance(ask_user, dict):
return
if str(decision or "").strip().lower() != CHAT_SESSION_APPROVAL_DECISION:
ask_user.pop(CHAT_SESSION_APPROVAL_SIGNATURE_FIELD, None)
return
signature = sign_chat_session_grant(
session_id,
ask_user.get("approval_id"),
CHAT_SESSION_APPROVAL_DECISION,
)
if signature:
ask_user[CHAT_SESSION_APPROVAL_SIGNATURE_FIELD] = signature
def verify_chat_session_grant(
signature: object,
session_id: object,
approval_id: object,
decision: object,
) -> bool:
"""Whether *signature* is this server's grant for that exact approval.
Fails CLOSED: an absent, malformed, or unverifiable signature is not a
grant. Binding the session and approval ids into the payload means a
signature lifted from one chat cannot be replayed into another.
"""
# compare_digest accepts only ASCII strings. Treat arbitrary persisted
# metadata as untrusted and require the exact representation we sign.
if (
not isinstance(signature, str)
or len(signature) != sha256().digest_size * 2
or any(character not in "0123456789abcdef" for character in signature)
):
return False
expected = sign_chat_session_grant(session_id, approval_id, decision)
if expected is None:
return False
return hmac.compare_digest(signature, expected)
class ToolApprovalScope(str, Enum):
# Surfaces without a resumable chat (the skill tester, unattended audits)
+23 -1
View File
@@ -15,7 +15,7 @@ from types import MappingProxyType
from typing import Any, Iterable, Mapping
from src.tool_approval_scopes import CHAT_SESSION_APPROVAL_CONTEXT_MARKER
from src.tool_security import BUILTIN_EMAIL_TOOLS
from src.tool_security import BUILTIN_EMAIL_TOOLS, is_public_blocked_tool
class ToolEffect(str, Enum):
@@ -624,10 +624,21 @@ class ToolRunSecurityContext:
# The bypass affects only this automatic gate; current tool policy, ownership,
# workspace confinement, and execution/sandbox restrictions still apply.
approval_gate_bypassed: bool = False
# Driven by a bearer API token, not a person at a browser. Privileged
# tools are refused outright and no approval can lift that.
delegated_credential: bool = False
def observe_messages(self, messages: Iterable[dict]) -> None:
"""Apply server-owned chat scope and promote untrusted prompt context."""
message_list = list(messages or ())
if self.delegated_credential:
# A delegated run has no human to grant chat-session scope, so a
# grant sitting in this chat's history (left by the owner's own
# browser) must not be picked up by a token driving the same chat.
self.approval_gate_bypassed = False
if messages_contain_external_untrusted_context(message_list):
self.external_untrusted_context_seen = True
return
if any(
isinstance(message, dict)
and isinstance(message.get("metadata"), dict)
@@ -641,6 +652,17 @@ class ToolRunSecurityContext:
self.external_untrusted_context_seen = True
def decision_for(self, tool_name: Any, content: Any = None) -> ToolGateDecision:
# Checked before the bypasses below, because neither may lift it, and
# kept independent of external_untrusted_context_seen so it holds on a
# run where that gate never arms and raises no prompt to bypass.
if self.delegated_credential and is_public_blocked_tool(tool_name):
return ToolGateDecision(
False,
(
f"Tool '{tool_name}' is not available to API-token callers. "
"It requires an interactive session."
),
)
if self.approval_gate_bypassed:
return ToolGateDecision(True)
if not self.external_untrusted_context_seen:
+235 -18
View File
@@ -15,6 +15,7 @@ import logging
import os
import pathlib
import re
import stat
import sys
import time
from typing import Any, Awaitable, Callable, Dict, Optional, Tuple
@@ -30,7 +31,12 @@ from src.tool_security import (
from src.tool_capabilities import ToolRunSecurityContext, blocked_tool_result
from src.tool_approvals import ExactToolApproval
from src.tool_policy import ToolPolicy
from src.constants import MAX_OUTPUT_CHARS, MAX_READ_CHARS, MAX_DIFF_LINES, DATA_DIR
from src.constants import (
MAX_OUTPUT_CHARS,
MAX_READ_CHARS,
MAX_DIFF_LINES,
AGENT_WORKSPACE_DIR,
)
from src.tool_utils import _truncate, get_mcp_manager
@@ -46,11 +52,11 @@ _MISSING_TOOL_SECURITY_CONTEXT = _MissingToolSecurityContext()
NO_TOOL_SECURITY_CONTEXT = _NoToolSecurityContext()
# Persistent working directory for agent subprocesses.
# Resolves to <repo_root>/data, which is the bind-mounted volume in Docker
# (/app/data) and the local data directory for manual installs.
# Using this as cwd and HOME prevents the agent from silently creating files
# in ephemeral container layers that are lost on the next rebuild.
_AGENT_WORKDIR = DATA_DIR
# Resolves to <repo_root>/data/agent_workspace, inside the bind-mounted volume
# in Docker (/app/data), so files survive a rebuild as before. The subdirectory
# rather than data/ itself keeps agent scratch files and dotfiles out of the
# directory holding the session store and the auth database.
_AGENT_WORKDIR = AGENT_WORKSPACE_DIR
@@ -66,10 +72,15 @@ _AGENT_WORKDIR = DATA_DIR
# 1. Sensitive-subpath deny list — checked FIRST. Blocks .ssh,
# .gnupg, shell rc files, token/env files even if the root above
# them is on the allowlist.
# 2. Allowlist — only the directories the agent legitimately needs
# (project data/, system tmp). $HOME is NOT on the default list.
# 3. Opt-in extra roots — admin can add broader roots via the
# "tool_path_extra_roots" setting (list of path strings).
# 2. Application-state deny (_is_app_state_path) - DATA_DIR holds the
# session store, auth database, app key and settings, so only
# _agent_readable_data_subdirs() is readable inside it.
# 3. Allowlist - only the directories the agent legitimately needs
# (its data/ workspace, user content, system tmp). $HOME is NOT on
# the default list.
# 4. Opt-in extra roots - admin can add broader roots via the
# "tool_path_extra_roots" setting. These cannot re-open DATA_DIR;
# rule 2 is independent of which root a path arrived through.
# ---------------------------------------------------------------------------
_SENSITIVE_BASENAMES: set[str] = {
@@ -116,6 +127,184 @@ def _is_sensitive_path(resolved: str) -> bool:
return filename in _SENSITIVE_FILE_PATTERNS_CF
def _path_within(resolved: str, root: str) -> bool:
"""True when *resolved* is *root* itself or sits underneath it.
Use the platform's path-case rules. This helper participates in allow
decisions, so unconditional case-folding would let a distinct ``/DATA``
tree masquerade as a descendant of ``/data`` on case-sensitive systems.
"""
resolved, root = os.path.normcase(resolved), os.path.normcase(root)
if resolved == root:
return True
try:
if os.path.commonpath([resolved, root]) == root:
return True
except ValueError:
return False
# normcase is intentionally conservative about assumptions (notably on
# POSIX), so consult the filesystem when paths exist. This recognizes a
# case alias on a case-insensitive volume without treating distinct
# case-sensitive paths as the same allow root.
if os.path.exists(root):
candidate = resolved
while True:
try:
if os.path.exists(candidate) and os.path.samefile(candidate, root):
return True
except OSError:
pass
parent = os.path.dirname(candidate)
if parent == candidate:
break
candidate = parent
return False
def _path_within_conservative(resolved: str, root: str) -> bool:
"""Containment for deny decisions, folding case to fail closed."""
resolved, root = resolved.casefold(), root.casefold()
if resolved == root:
return True
try:
return os.path.commonpath([resolved, root]) == root
except ValueError:
return False
def _agent_readable_data_subdirs() -> tuple[str, ...]:
"""The only parts of DATA_DIR the agent's file tools may reach.
The agent's own scratch folder, plus the directories of user content whose
paths the application itself gives to the model, which it would then be
unable to open. These normally live under DATA_DIR; the documented mail
attachment override may instead name a disjoint external directory:
UPLOAD_DIR the chat upload manifest renders "path=<p>" and
says to read it with read_file (agent_loop.py)
MAIL_ATTACHMENTS_DIR download_attachment returns the path and its own
description tells the model to read it
PERSONAL_DIR GET /api/personal returns a path per file and is
reachable through the app_api tool; RUNBOOK_DIR
nests under it
PERSONAL_UPLOADS_DIR indexed as a personal-docs directory, which
manage_rag lists as an absolute path
Order matters: the first entry is roots[0], which _resolve_search_root uses
when grep/glob/ls are called with no path.
"""
from src.constants import (
DATA_DIR,
MAIL_ATTACHMENTS_DIR,
PERSONAL_DIR,
PERSONAL_UPLOADS_DIR,
UPLOAD_DIR,
)
configured = (
(AGENT_WORKSPACE_DIR, "agent_workspace", False),
(UPLOAD_DIR, "uploads", False),
# This has a documented environment override and may legitimately
# live outside DATA_DIR, but it must never equal/contain DATA_DIR.
(MAIL_ATTACHMENTS_DIR, "mail-attachments", True),
(PERSONAL_DIR, "personal_docs", False),
(PERSONAL_UPLOADS_DIR, "personal_uploads", False),
)
configured_data_dir = os.path.abspath(os.path.expanduser(str(DATA_DIR)))
data_dir = os.path.realpath(configured_data_dir)
safe: list[str] = []
for raw, internal_name, external_ok in configured:
value = str(raw or "").strip()
# These paths are security-policy roots, not ordinary allowlist
# entries. Internal roles may inherit a relative DATA_DIR, but must
# still resolve to their exact canonical child below. External mail
# overrides require an absolute, disjoint directory.
if not value:
continue
expanded = os.path.abspath(os.path.expanduser(value))
# A policy root must not acquire an exemption by redirecting its final
# path component to protected state or to an unrelated external tree.
if os.path.islink(expanded):
continue
resolved = os.path.realpath(expanded)
if os.path.exists(resolved) and not os.path.isdir(resolved):
continue
expected_internal = os.path.join(data_dir, internal_name)
expected_configured = os.path.join(configured_data_dir, internal_name)
inside_data = (
os.path.normcase(expanded)
in {
os.path.normcase(expected_configured),
os.path.normcase(expected_internal),
}
and resolved == expected_internal
)
external_safe = (
external_ok
and os.path.isabs(os.path.expanduser(value))
and resolved != data_dir
and os.path.dirname(resolved) != resolved
and not _path_within(data_dir, resolved)
and not _path_within(resolved, data_dir)
)
if not (inside_data or external_safe) or _is_sensitive_path(resolved):
continue
safe.append(resolved)
return tuple(safe)
def _is_app_state_path(resolved: str) -> bool:
"""True for anything under DATA_DIR that is not agent-readable.
DATA_DIR holds the session store, the auth database, the app encryption key
and the settings file. A model-supplied path must not reach those through
any root, so this is checked in both resolvers rather than expressed as an
absence from the allowlist: a workspace bound at or above the data
directory, or an opt-in tool_path_extra_roots entry covering it, would
otherwise put them back in reach.
A containment rule rather than a filename deny list, so state files added
later are covered without anyone remembering to list them, and so a user's
own settings.json or app.db inside a real workspace is not caught.
"""
from src.constants import DATA_DIR
if not _path_within_conservative(resolved, os.path.realpath(DATA_DIR)):
return False
return not any(
_path_within(resolved, d)
for d in _agent_readable_data_subdirs()
)
def _is_hardlinked_regular_file(resolved: str) -> bool:
"""Reject inode aliases that can smuggle DATA_DIR state into an allow root."""
try:
target = os.stat(resolved, follow_symlinks=False)
except OSError:
return False
return stat.S_ISREG(target.st_mode) and getattr(target, "st_nlink", 1) > 1
def _is_denied_tool_path(resolved: str) -> bool:
"""Apply every path deny to a canonical traversal result."""
return (
_is_sensitive_path(resolved)
or _is_app_state_path(resolved)
or _is_hardlinked_regular_file(resolved)
)
def _can_traverse_tool_path(resolved: str) -> bool:
"""Allow walking a denied state parent only to reach safe carve-outs."""
if _is_sensitive_path(resolved):
return False
if not _is_app_state_path(resolved):
return True
return any(
_path_within(readable, resolved)
for readable in _agent_readable_data_subdirs()
)
def _tool_path_roots() -> list[str]:
"""Return the list of directory roots that read_file / write_file
may touch. Default: project data/ + system temp dirs. Extra roots
@@ -123,9 +312,9 @@ def _tool_path_roots() -> list[str]:
"""
roots: list[str] = []
# Project data directory — the agent's primary workspace.
from src.constants import DATA_DIR
roots.append(DATA_DIR)
# The agent's workspace plus the user-content directories inside data/.
# The rest of DATA_DIR is denied by _is_app_state_path.
roots.extend(_agent_readable_data_subdirs())
# /tmp (and its macOS realpath /private/tmp).
roots.append("/tmp")
@@ -193,6 +382,12 @@ def _resolve_tool_path(raw_path: str) -> str:
f"path '{raw_path}' is inside a sensitive directory "
f"(e.g. .ssh, .gnupg) or matches a sensitive filename"
)
if _is_app_state_path(resolved):
raise ValueError(
f"path '{raw_path}' is inside the application state directory"
)
if _is_hardlinked_regular_file(resolved):
raise ValueError(f"path '{raw_path}' is a hard-linked file")
for root in _tool_path_roots():
if resolved == root:
@@ -228,6 +423,12 @@ def _resolve_tool_path_in_workspace(workspace: str, raw_path: str) -> str:
f"path '{raw_path}' is inside a sensitive directory "
f"(e.g. .ssh, .gnupg) or matches a sensitive filename"
)
if _is_app_state_path(resolved):
raise ValueError(
f"path '{raw_path}' is inside the application state directory"
)
if _is_hardlinked_regular_file(resolved):
raise ValueError(f"path '{raw_path}' is a hard-linked file")
if resolved != base:
# normcase so containment holds on case-insensitive filesystems
# (Windows, default macOS): it lowercases on Windows and is a no-op on
@@ -277,6 +478,10 @@ def vet_workspace(raw: str) -> Optional[str]:
resolved = os.path.realpath(os.path.expanduser(raw))
if not os.path.isdir(resolved) or _is_sensitive_path(resolved):
return None
# Refuse the bind rather than binding a workspace where every subsequent
# tool call would fail on the same deny list.
if _is_app_state_path(resolved):
return None
# Reject filesystem roots: binding / (or a Windows drive/UNC root) as the
# workspace would make every absolute path "inside" it, collapsing the
# confinement into host-wide file access. A root is its own dirname, which
@@ -289,7 +494,13 @@ def vet_workspace(raw: str) -> Optional[str]:
def agent_cwd() -> str:
"""Working directory for agent subprocesses (bash/python/background jobs):
the active workspace when set, else the persistent data dir."""
return get_active_workspace() or _AGENT_WORKDIR
workspace = get_active_workspace()
if workspace:
return workspace
resolved = os.path.realpath(_AGENT_WORKDIR)
if resolved not in _agent_readable_data_subdirs():
raise RuntimeError("agent workspace is not a safe real directory")
return resolved
def get_mcp_manager():
@@ -304,16 +515,22 @@ def _resolve_search_root(raw_path: str) -> str:
With a workspace active, the workspace folder is the root and a supplied
path is confined inside it. Otherwise an empty path defaults to the agent's
primary root (project data dir) and a supplied path is confined by the
global allowlist + sensitive-file policy.
primary root (its workspace under the project data dir) and a supplied path
is confined by the global allowlist + sensitive-file policy.
"""
raw = (raw_path or "").strip()
ws = get_active_workspace()
if ws:
return os.path.realpath(ws) if not raw else _resolve_tool_path_in_workspace(ws, raw)
# Resolve the empty case as the workspace path rather than returning
# it directly: returned unchecked it skipped both deny lists, so a
# bare ls listed whatever the workspace was bound to.
return _resolve_tool_path_in_workspace(ws, raw or ws)
if not raw:
roots = _tool_path_roots()
return roots[0] if roots else os.path.realpath(".")
default_root = os.path.realpath(AGENT_WORKSPACE_DIR)
if default_root in roots and not _is_denied_tool_path(default_root):
return default_root
raise ValueError("default agent workspace is not a safe readable data subdirectory")
return _resolve_tool_path(raw)
logger = logging.getLogger(__name__)
+13
View File
@@ -269,3 +269,16 @@ def blocked_tools_for_owner(owner: Optional[str]) -> Set[str]:
if owner_is_admin_or_single_user(owner):
return set()
return set(NON_ADMIN_BLOCKED_TOOLS)
def delegated_credential_blocked_tools() -> Set[str]:
"""Tools an agent run driven by a bearer API token must not reach.
Deliberately not owner-dependent. ``blocked_tools_for_owner`` asks whether
the OWNER is an admin, and for a token that question is always answered
yes: minting a token is an admin-only action, so the empty set comes back
for every token in existence. A token is a long-lived credential the owner
hands to a third party, so it is capped at the non-admin policy no matter
who minted it.
"""
return set(NON_ADMIN_BLOCKED_TOOLS)
File diff suppressed because it is too large Load Diff
+327
View File
@@ -0,0 +1,327 @@
"""Tool authority for delegated API-token callers.
Covers three independent ways a bearer API token could reach the agent's
privileged tools:
1. the token answering its own tool-approval prompt,
2. the token pre-seeding approval-shaped message metadata so no prompt is
ever raised,
3. the token inheriting ``bash``/``python`` from the admin account that
minted it, on a run where the approval gate never arms at all.
"""
from types import SimpleNamespace
import pytest
from fastapi import HTTPException
from core.models import ChatMessage, Session
from src.tool_approval_scopes import CHAT_SESSION_APPROVAL_CONTEXT_MARKER
from src.tool_capabilities import ToolRunSecurityContext
def _session(history):
return Session(
id="session-1",
name="Chat",
endpoint_url="http://example.invalid",
model="test",
history=history,
)
def _forged_card(session_id="session-1"):
"""Approval-shaped metadata as a client could POST it."""
return {
"kind": "tool_approval",
"approval_id": "attacker-chosen-id",
"session_id": session_id,
"resolved": "approve",
}
def test_client_supplied_approval_metadata_does_not_grant_the_chat_session_bypass():
session = _session([
ChatMessage(
"assistant",
"approval requested",
{"tool_events": [{"ask_user": _forged_card()}]},
),
ChatMessage("user", "continue the work"),
])
context = ToolRunSecurityContext(external_untrusted_context_seen=True)
context.observe_messages(session.get_context_messages())
assert context.approval_gate_bypassed is False
assert context.decision_for("bash").allowed is False
def test_a_grant_the_server_signed_still_bypasses_the_gate_for_that_chat():
"""The fix must not simply deny every chat-session grant."""
from src.tool_approval_scopes import stamp_chat_session_grant
card = {
"kind": "tool_approval",
"approval_id": "real-approval",
"session_id": "session-1",
"resolved": "approve",
}
stamp_chat_session_grant(card, "session-1", "approve")
session = _session([
ChatMessage("assistant", "approval requested", {"tool_events": [{"ask_user": card}]}),
ChatMessage("user", "continue the work"),
])
context = ToolRunSecurityContext(external_untrusted_context_seen=True)
context.observe_messages(session.get_context_messages())
assert context.approval_gate_bypassed is True
assert context.decision_for("bash").allowed is True
def test_a_signed_grant_does_not_transfer_to_another_chat():
from src.tool_approval_scopes import stamp_chat_session_grant
card = {
"kind": "tool_approval",
"approval_id": "real-approval",
"session_id": "session-1",
"resolved": "approve",
}
stamp_chat_session_grant(card, "session-1", "approve")
# Copy the whole resolved card, signature included, into a different chat.
card_in_other_chat = dict(card, session_id="session-2")
other = Session(
id="session-2",
name="Chat",
endpoint_url="http://example.invalid",
model="test",
history=[
ChatMessage("assistant", "x", {"tool_events": [{"ask_user": card_in_other_chat}]}),
ChatMessage("user", "continue"),
],
)
context = ToolRunSecurityContext(external_untrusted_context_seen=True)
context.observe_messages(other.get_context_messages())
assert context.approval_gate_bypassed is False
@pytest.mark.parametrize("signature", [
None, 17, [], {}, b"a" * 64, "", "a" * 63, "a" * 65,
"g" * 64, "A" * 64, "\u00e9" * 64, "\ud800" * 64,
])
def test_malformed_grant_is_rejected_without_breaking_chat_context(monkeypatch, signature):
import json
from src import tool_approval_scopes as scopes
monkeypatch.setattr(scopes, "_grant_key", lambda: b"test-only-grant-key")
assert scopes.verify_chat_session_grant(
signature, "session-1", "attacker-chosen-id", "approve"
) is False
# JSON can persist non-ASCII text and escaped lone surrogates in history.
# Bytes are not JSON-serializable, but still exercise the direct verifier.
if isinstance(signature, bytes):
return
card = _forged_card()
card[scopes.CHAT_SESSION_APPROVAL_SIGNATURE_FIELD] = signature
metadata = json.loads(json.dumps({"tool_events": [{"ask_user": card}]}))
session = _session([
ChatMessage("assistant", "approval requested", metadata),
ChatMessage("user", "continue the work"),
])
messages = session.get_context_messages()
assert messages[-1]["content"] == "continue the work"
context = ToolRunSecurityContext(external_untrusted_context_seen=True)
context.observe_messages(messages)
assert context.approval_gate_bypassed is False
assert context.decision_for("bash").allowed is False
def _bearer_request(owner="admin"):
return SimpleNamespace(state=SimpleNamespace(
api_token=True, api_token_owner=owner, api_token_scopes=["todos:read"],
current_user="api",
))
def _cookie_request(user="admin"):
return SimpleNamespace(state=SimpleNamespace(api_token=False, current_user=user))
def test_a_bearer_token_may_not_answer_a_tool_approval_prompt():
"""An approval asserts a human authorized the action; a token is not one."""
from routes.chat_routes import _reject_delegated_tool_approval
with pytest.raises(HTTPException) as raised:
_reject_delegated_tool_approval(_bearer_request())
assert raised.value.status_code == 403
def test_a_browser_session_may_still_answer_a_tool_approval_prompt():
from routes.chat_routes import _reject_delegated_tool_approval
_reject_delegated_tool_approval(_cookie_request())
def test_chat_scope_is_required_before_bearer_chat_state_is_touched():
from src.auth_helpers import require_chat_api_token_scope
with pytest.raises(HTTPException) as raised:
require_chat_api_token_scope(_bearer_request())
assert raised.value.status_code == 403
def test_chat_scope_allows_owner_attribution_for_bearer_chat_routes():
from src.auth_helpers import require_chat_api_token_scope
request = _bearer_request()
request.state.api_token_scopes = ["chat"]
assert require_chat_api_token_scope(request) == "admin"
@pytest.mark.asyncio
async def test_todos_read_token_is_denied_before_inline_memory_persistence():
from routes.chat_routes import setup_chat_routes
from src.request_models import ChatRequest
class MemoryGuard:
async def handle_memory_command(self, *args, **kwargs):
raise AssertionError("memory command ran before bearer scope policy")
router = setup_chat_routes(
session_manager=SimpleNamespace(),
chat_handler=MemoryGuard(),
chat_processor=SimpleNamespace(),
memory_manager=SimpleNamespace(),
research_handler=SimpleNamespace(),
upload_handler=SimpleNamespace(),
)
endpoint = next(
route.endpoint
for route in router.routes
if route.path == "/api/chat" and "POST" in route.methods
)
with pytest.raises(HTTPException) as raised:
await endpoint(
_bearer_request(),
ChatRequest(message="remember this", session="session-1"),
)
assert raised.value.status_code == 403
def test_a_delegated_run_is_denied_the_shell_even_when_the_gate_never_arms():
"""The approval prompt is raised only once untrusted context is seen.
An agent run driven by a token that carries no untrusted context reaches
``bash`` with no prompt to bypass at all, so refusing token-answered
approvals does not by itself close the path.
"""
context = ToolRunSecurityContext(
external_untrusted_context_seen=False,
delegated_credential=True,
)
assert context.decision_for("bash").allowed is False
assert context.decision_for("python").allowed is False
def test_a_delegated_run_cannot_be_handed_the_gate_bypass():
context = ToolRunSecurityContext(
external_untrusted_context_seen=True,
delegated_credential=True,
approval_gate_bypassed=True,
)
assert context.decision_for("bash").allowed is False
def test_a_delegated_run_still_allows_tools_that_are_not_privileged():
context = ToolRunSecurityContext(
external_untrusted_context_seen=False,
delegated_credential=True,
)
assert context.decision_for("web_search").allowed is True
assert context.decision_for("manage_notes").allowed is True
def test_delegated_runs_lose_the_tools_a_non_admin_would_lose():
"""A token's authority is capped at the non-admin policy, not its owner's.
Only admins can mint tokens, so ``blocked_tools_for_owner`` returns an
empty set for every token that exists. This is the set that should apply
instead.
"""
from src.tool_security import delegated_credential_blocked_tools
blocked = delegated_credential_blocked_tools()
assert {"bash", "python", "read_file", "write_file", "send_email"} <= blocked
assert "web_search" not in blocked
assert "manage_notes" not in blocked
def test_caller_supplied_metadata_is_stripped_of_server_owned_tool_events():
"""Defence in depth for the two routes that accept a metadata blob.
The grant check is signature-based, so this is not what closes the hole.
It keeps a caller from writing server-owned keys into a transcript at all.
"""
from src.tool_approval_scopes import sanitize_client_message_metadata
cleaned = sanitize_client_message_metadata({
"source": "slash",
"tool_events": [{"ask_user": _forged_card()}],
CHAT_SESSION_APPROVAL_CONTEXT_MARKER: True,
})
assert cleaned == {"source": "slash"}
def test_sanitizing_metadata_leaves_ordinary_payloads_alone():
from src.tool_approval_scopes import sanitize_client_message_metadata
payload = {"source": "slash", "attachments": [{"attachment_id": "abc"}]}
assert sanitize_client_message_metadata(payload) == payload
assert sanitize_client_message_metadata(None) is None
def test_a_token_cannot_reuse_the_grant_its_owner_made_in_the_browser():
"""The grant is genuine and correctly signed, so only the delegated check
stops it. Confirmed live: exploitable before this change, closed after."""
from src.tool_approval_scopes import stamp_chat_session_grant
card = {
"kind": "tool_approval",
"approval_id": "owners-real-approval",
"session_id": "session-1",
"resolved": "approve",
}
stamp_chat_session_grant(card, "session-1", "approve")
session = _session([
ChatMessage("assistant", "approval requested", {"tool_events": [{"ask_user": card}]}),
ChatMessage("user", "continue"),
])
messages = session.get_context_messages()
owner_turn = ToolRunSecurityContext(external_untrusted_context_seen=True)
owner_turn.observe_messages(messages)
assert owner_turn.decision_for("bash").allowed is True
token_turn = ToolRunSecurityContext(
external_untrusted_context_seen=True, delegated_credential=True)
token_turn.observe_messages(messages)
assert token_turn.approval_gate_bypassed is False
assert token_turn.decision_for("bash").allowed is False
+11
View File
@@ -91,6 +91,17 @@ def test_grep_python_fallback_when_no_rg(repo, monkeypatch):
assert ".git/config" not in r["output"]
def test_grep_python_fallback_uses_relative_glob_paths(repo, monkeypatch):
monkeypatch.setattr(shutil, "which", lambda name: None)
r = _run(
"grep",
f'{{"pattern": "needle|python", "glob": "**/*.py", "path": "{repo}"}}',
)
assert r["exit_code"] == 0
assert "a.py" in r["output"]
assert "sub/deep/c.py" in r["output"]
@pytest.mark.skipif(shutil.which("rg") is None, reason="targets the ripgrep fast-path")
def test_grep_skips_case_variant_sensitive_files_rg(repo):
"""The rg fast-path must exclude deny-listed key files case-insensitively.
+84
View File
@@ -1,9 +1,14 @@
"""Static regressions for Docker/devops hardening contracts."""
import ast
import os
import re
import shutil
import subprocess
import uuid
from pathlib import Path
import pytest
import yaml
from starlette.applications import Starlette
from starlette.middleware.cors import CORSMiddleware
@@ -115,6 +120,85 @@ def test_docker_entrypoint_ownership_repair_stays_inside_expected_mounts():
assert "Skipping recursive ownership repair" in script
def test_docker_entrypoint_repairs_cache_parent_without_recursive_walk():
"""Pin the hard-coded container-path contract without running entrypoint as root."""
script = (ROOT / "docker" / "entrypoint.sh").read_text(encoding="utf-8")
app_repair = script.index("repair_app_tree_ownership\n")
cache_parent_repair = script.index(
'chown "$PUID:$PGID" /app/.cache 2>/dev/null || true'
)
mounted_cache_root_repair = script.index(
'chown "$PUID:$PGID" /app/.cache/huggingface 2>/dev/null || true'
)
assert app_repair < cache_parent_repair < mounted_cache_root_repair
assert 'repair_tree_ownership "/app/.cache"' not in script
assert 'repair_bind_mount_ownership "/app/.cache/huggingface"' not in script
@pytest.mark.skipif(shutil.which("docker") is None, reason="Docker CLI is unavailable")
def test_docker_entrypoint_cache_parent_with_nested_volume():
"""Run the real entrypoint against a disposable nested-volume layout."""
image = os.environ.get("ODYSSEUS_DOCKER_TEST_IMAGE", "odysseus-odysseus:latest")
if subprocess.run(
["docker", "image", "inspect", image],
capture_output=True,
text=True,
check=False,
).returncode != 0:
pytest.skip(f"Docker test image is unavailable: {image}")
volume = f"odysseus-cache-parent-test-{uuid.uuid4().hex}"
subprocess.run(
["docker", "volume", "create", volume],
capture_output=True,
text=True,
check=True,
)
try:
subprocess.run(
[
"docker", "run", "--rm", "--pull=never",
"--entrypoint", "sh",
"-v", f"{volume}:/fixture",
image,
"-c", "mkdir -p /fixture/nested && touch /fixture/nested/sentinel",
],
capture_output=True,
text=True,
check=True,
)
result = subprocess.run(
[
"docker", "run", "--rm", "--pull=never",
"-e", "PUID=23456",
"-e", "PGID=23456",
"-v", f"{volume}:/app/.cache/huggingface",
image,
"sh", "-c",
"mkdir -p /app/.cache/vllm && "
"touch /app/.cache/vllm/probe && "
"printf 'CACHE_TEST %s %s %s %s\\n' "
"\"$(stat -c %u /app/.cache)\" "
"\"$(stat -c %u /app/.cache/vllm/probe)\" "
"\"$(stat -c %u /app/.cache/huggingface)\" "
"\"$(stat -c %u /app/.cache/huggingface/nested/sentinel)\"",
],
capture_output=True,
text=True,
check=True,
)
finally:
subprocess.run(
["docker", "volume", "rm", "-f", volume],
capture_output=True,
text=True,
check=False,
)
assert "CACHE_TEST 23456 23456 23456 0" in result.stdout
def test_dockerignore_excludes_secrets_editor_backups():
patterns = set((ROOT / ".dockerignore").read_text(encoding="utf-8").splitlines())
assert {
+52
View File
@@ -1291,6 +1291,58 @@ def test_approval_pause_does_not_trigger_teacher_takeover(monkeypatch):
)
def test_teacher_takeover_inherits_delegated_and_tainted_run_authority(monkeypatch):
from src.prompt_security import untrusted_context_message
import src.agent_loop as agent_loop
import src.teacher_escalation as teacher_escalation
monkeypatch.setattr(
agent_loop,
"get_setting",
lambda key, default=None: default,
raising=False,
)
monkeypatch.setattr(agent_loop, "get_mcp_manager", lambda: None, raising=False)
monkeypatch.setattr(agent_loop, "estimate_tokens", lambda *args, **kwargs: 10)
monkeypatch.setattr(
agent_loop,
"blocked_tools_for_owner",
lambda owner: set(),
raising=False,
)
async def fake_stream(*args, **kwargs):
yield "data: " + json.dumps({"delta": "finished"}) + "\n\n"
yield "data: [DONE]\n\n"
captured = {}
async def capture_teacher(*args, **kwargs):
captured.update(kwargs)
if False:
yield "" # pragma: no cover
monkeypatch.setattr(agent_loop, "stream_llm_with_fallback", fake_stream)
monkeypatch.setattr(teacher_escalation, "run_teacher_inline", capture_teacher)
_collect_agent_events(
agent_loop.stream_agent_loop(
"http://local.test/v1",
"qwen-local-model",
[
{"role": "user", "content": "finish it"},
untrusted_context_message("stored context", "untrusted"),
],
session_id="session-1",
max_rounds=1,
delegated_credential=True,
)
)
assert captured["delegated_credential"] is True
assert captured["external_untrusted_context_seen"] is True
def test_frontend_tool_approval_uses_opaque_id_and_fixed_decisions():
root = Path(__file__).parents[1]
chat = (root / "static/js/chat.js").read_text()
+10
View File
@@ -1,12 +1,22 @@
# tests/test_launcher.py
import sys
import os
from pathlib import Path
from unittest import mock
import pytest
from launcher import NullWriter, create_tray_image, on_open_browser, on_exit, open_browser
def test_frozen_multiprocessing_bootstrap_precedes_gui_and_app_imports():
source = Path("launcher.py").read_text(encoding="utf-8")
freeze = source.index("multiprocessing.freeze_support()")
splash = source.index("if getattr(sys, 'frozen', False):")
app_import = source.index("from app import app")
assert freeze < splash < app_import
def test_null_writer():
writer = NullWriter()
# writing and flushing should not raise any exceptions
+52 -3
View File
@@ -6,13 +6,21 @@ from fastapi import HTTPException
# Import the route helper during collection so sibling session tests that use
# partial import stubs do not become the first loader of core.session_manager.
from routes.session_routes import _reject_raw_endpoint_url_for_non_admin
from routes.session_routes import (
_reject_delegated_session_options,
_reject_raw_endpoint_url_for_non_admin,
)
def _request(user, *, admin=False):
def _request(user, *, admin=False, api_token=False, scopes=None):
auth_manager = SimpleNamespace(is_admin=lambda username: bool(admin))
return SimpleNamespace(
state=SimpleNamespace(current_user=user),
state=SimpleNamespace(
current_user="api" if api_token else user,
api_token=api_token,
api_token_owner=user if api_token else None,
api_token_scopes=scopes or [],
),
app=SimpleNamespace(state=SimpleNamespace(auth_manager=auth_manager)),
)
@@ -44,6 +52,47 @@ def test_admin_and_registered_endpoint_can_use_endpoint_url():
)
def test_bearer_token_does_not_inherit_owner_admin_raw_endpoint_authority():
request = _request("admin", admin=True, api_token=True, scopes=["chat"])
with pytest.raises(HTTPException) as exc:
_reject_raw_endpoint_url_for_non_admin(
request,
"admin",
"",
"http://127.0.0.1:8000/v1/chat/completions",
)
assert exc.value.status_code == 403
def test_chat_scoped_bearer_can_still_choose_an_owner_registered_endpoint():
_reject_raw_endpoint_url_for_non_admin(
_request("admin", admin=True, api_token=True, scopes=["chat"]),
"admin",
"owner-endpoint-id",
"http://127.0.0.1:8000/v1/chat/completions",
)
@pytest.mark.parametrize(
("skip_validation", "api_key"),
[(True, ""), (False, "caller-secret")],
)
def test_bearer_token_cannot_use_interactive_session_options(
skip_validation,
api_key,
):
with pytest.raises(HTTPException) as exc:
_reject_delegated_session_options(
_request("admin", admin=True, api_token=True, scopes=["chat"]),
skip_validation=skip_validation,
api_key=api_key,
)
assert exc.value.status_code == 403
def test_chat_endpoint_recovery_paths_are_owner_scoped():
root = Path(__file__).resolve().parents[1]
chat_routes = (root / "routes" / "chat_routes.py").read_text(encoding="utf-8")
+4
View File
@@ -367,6 +367,8 @@ async def test_teacher_approval_keeps_parent_authority_and_skips_skill_save(
tool_policy=policy,
active_document=active_document,
active_email=active_email,
external_untrusted_context_seen=True,
delegated_credential=True,
):
events.append(evt)
@@ -376,6 +378,8 @@ async def test_teacher_approval_keeps_parent_authority_and_skips_skill_save(
assert captured["tool_policy"] is policy
assert captured["active_document"] is active_document
assert captured["active_email"] == active_email
assert captured["external_untrusted_context_seen"] is True
assert captured["delegated_credential"] is True
assert any("opaque-id" in event for event in events)
assert not any("skill_saved" in event for event in events)
+4
View File
@@ -10,6 +10,7 @@ from core.models import ChatMessage, Session
from src.tool_approval_scopes import (
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
ToolApprovalScope,
stamp_chat_session_grant,
)
from src.tool_approvals import ExactToolApproval, ToolApprovalStore
from src.tool_capabilities import ToolRunSecurityContext, capabilities_for_action
@@ -111,6 +112,9 @@ def test_allow_for_chat_session_applies_to_later_turns_in_only_that_chat():
resolved_card = pending.public_payload()
resolved_card["resolved"] = "approve"
# Resolving is a server action, and only the server's signature on the card
# makes it a grant. A card that merely looks resolved is not one.
stamp_chat_session_grant(resolved_card, "session-1", "approve")
history = [
ChatMessage(
"assistant",
+7 -5
View File
@@ -161,12 +161,14 @@ def test_blocks_netrc():
_resolve_tool_path("~/.netrc")
def test_allows_project_data(tmp_path):
"""Paths under project data/ must resolve cleanly."""
def test_allows_agent_workspace(tmp_path):
"""Paths under the agent's workspace in project data/ must resolve
cleanly. The rest of data/ is application state and is rejected;
tests/test_agent_state_dir_confinement.py covers that side."""
from src.tool_execution import _resolve_tool_path
from src.constants import DATA_DIR
target = os.path.join(DATA_DIR, "test-confinement-ok.txt")
os.makedirs(DATA_DIR, exist_ok=True)
from src.constants import AGENT_WORKSPACE_DIR
target = os.path.join(AGENT_WORKSPACE_DIR, "test-confinement-ok.txt")
os.makedirs(AGENT_WORKSPACE_DIR, exist_ok=True)
with open(target, "w") as f:
f.write("ok")
try: