From 33fa27b4c88c23b4935a457caff86a7726b197b3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?L=C3=A9o?= Date: Thu, 1 Oct 2026 17:55:33 +0200 Subject: [PATCH] feat(runtime): add the containment boundary and its failure contract Agent-reachable execution has 25 independent spawn sites and no single place deciding where a process runs or under what limits. All three consequences are visible on this SHA. When bwrap is absent the workspace namespace degrades to a regex that rewrites /workspace to the real path, and nothing in the tool result says which one you got. No spawn site passes start_new_session, so a wall-clock kill reaches the wrapper shell and leaves its backgrounded grandchildren running while reporting the process killed. Teardown stops at SIGTERM without ever checking death. src/containment.py gives those paths one boundary. acquire() establishes containment or refuses -- a string rewrite is not a mechanism it can select -- and the grant states which dimensions actually hold, which were best-effort and are missing, and which were required and are missing. run() enforces the wall clock and the output cap. release() signals the process group, escalates to SIGKILL, and reports dead only for a group it observed go empty. Containment never sees the command: acquire() takes a workspace and limits, and the command text only reaches run(). Nothing in a request can widen a boundary it is never shown. CONTAINMENT_MODE chooses between refusing an unestablishable required dimension and recording it. It ships report-only, so landing this changes no behaviour on a host without bwrap -- which is every host today. No call sites move here; they follow on this branch. The configuration reference is regenerated because the page records src/constants.py line numbers and the new path constant shifts two of them. --- src/constants.py | 1 + src/containment.py | 1222 ++++++++++++++++++++++++ tests/test_containment_contract.py | 509 ++++++++++ tests/test_containment_process_tree.py | 378 ++++++++ website/configuration-reference.md | 4 +- 5 files changed, 2112 insertions(+), 2 deletions(-) create mode 100644 src/containment.py create mode 100644 tests/test_containment_contract.py create mode 100644 tests/test_containment_process_tree.py diff --git a/src/constants.py b/src/constants.py index 633141b22..40f859f3e 100644 --- a/src/constants.py +++ b/src/constants.py @@ -75,6 +75,7 @@ APP_KEY_FILE = os.path.join(DATA_DIR, ".app_key") EMBEDDING_ENDPOINT_FILE = os.path.join(DATA_DIR, "embedding_endpoint.json") COOKBOOK_STATE_FILE = os.path.join(DATA_DIR, "cookbook_state.json") BG_JOBS_FILE = os.path.join(DATA_DIR, "bg_jobs.json") +CONTAINMENT_STATE_FILE = os.path.join(DATA_DIR, "containment_grants.json") VAULT_FILE = os.path.join(DATA_DIR, "vault.json") TIDY_CALENDAR_STATE_FILE = os.path.join(DATA_DIR, "tidy_calendar_state.json") SKILLS_FILE = os.path.join(DATA_DIR, "skills.json") diff --git a/src/containment.py b/src/containment.py new file mode 100644 index 000000000..1f46f07d5 --- /dev/null +++ b/src/containment.py @@ -0,0 +1,1222 @@ +"""The runtime containment boundary. + +One place decides *where* and *under what limits* an already-authorized process +may run. Nothing here decides *whether* it may run — that is request authority, +and it lives elsewhere. The chain is: request → authority decides whether → +containment decides how and where → effect inside the boundary. + +Three properties this module exists to hold, in order of how badly the tree +needed them: + +1. **No silent downgrade.** Today a missing sandbox binary turns into a regex + that rewrites ``/workspace`` to the real path, with no log line and no field + in the tool result — ``namespaced or _replace_workspace_alias(...)``. A + string rewrite is not a containment mechanism and :func:`acquire` cannot + return one, so that line becomes unwritable through this API. +2. **Truthful reporting.** A grant states which dimensions are actually + enforced, which were asked for best-effort and are missing, and which were + required and are missing. "Was that command contained?" gets one answer + instead of none. +3. **Authoritative teardown.** :func:`release` escalates SIGTERM → SIGKILL, + signals the whole process group, and verifies death before reporting it. + Nothing here marks a process killed that it did not observe die. + +**Containment never reads the command.** :func:`acquire` is given a spec and an +owner; the command text only reaches :func:`run`, after the boundary is fixed. +That is structural, not a convention: no model output, tool argument or chain of +reasoning can widen a boundary it is never shown to. Limits come from +:data:`DEFAULT_REQUIRED` and the caller's configuration, never from the request. + +Enforcement mode +---------------- +:data:`CONTAINMENT_MODE` is a module-level constant, deliberately not a setting +and not an ``ODYSSEUS_*`` variable, so that changing the posture of every +agent-reachable spawn site is a one-line reviewable diff rather than a +deployment detail. + +* :data:`MODE_ENFORCING` — a required dimension that cannot be established + raises :class:`ContainmentUnavailable` and the command does not run. +* :data:`MODE_REPORT_ONLY` — the same shortfall is recorded on the grant as + ``unenforced_required``, logged once, and the command runs. + +The shipped default is report-only. On macOS and in the shipped Docker image +there is no ``bwrap``, so enforcing filesystem containment by default would turn +every ``bash`` call into a refusal the moment this module is wired up. Starting +report-only makes that landing observable instead of breaking, and flipping the +constant is reversible in a way that breaking every host is not. +""" + +from __future__ import annotations + +import asyncio +import json +import logging +import os +import shutil +import signal +import subprocess +import sys +import time +import uuid +from dataclasses import dataclass, replace +from pathlib import Path, PurePosixPath +from types import MappingProxyType +from typing import Any, Awaitable, Callable, Mapping, Optional + +from core.atomic_io import atomic_write_json +from core.platform_compat import IS_WINDOWS, find_bash, pid_alive + +from src.constants import CONTAINMENT_STATE_FILE, MAX_OUTPUT_CHARS + +logger = logging.getLogger(__name__) + + +# ── Enforcement mode ──────────────────────────────────────────────────────── +MODE_ENFORCING = "enforcing" +MODE_REPORT_ONLY = "report_only" + +#: Ship report-only; see the module docstring for why this is the reversible +#: direction. Flip to MODE_ENFORCING to make an unestablishable required +#: dimension refuse the command instead of reporting on it. +CONTAINMENT_MODE = MODE_REPORT_ONLY + + +# ── Dimensions ────────────────────────────────────────────────────────────── +FILESYSTEM = "filesystem" +PROCESS_TREE = "process_tree" +WALL_CLOCK = "wall_clock" +NETWORK = "network" +MEMORY = "memory" +PROCESS_COUNT = "process_count" + +DIMENSIONS = frozenset({ + FILESYSTEM, PROCESS_TREE, WALL_CLOCK, NETWORK, MEMORY, PROCESS_COUNT, +}) + +#: What every model-reachable spawn must have. Filesystem scope and an +#: authoritative kill are the two the tree currently lacks; a wall clock it has +#: but does not enforce past the leader process. Network, memory and process +#: count stay best-effort until mechanisms for them exist on every platform +#: (Waves 5A/5B), because requiring a dimension no mechanism provides refuses +#: every command on every host. +DEFAULT_REQUIRED = frozenset({FILESYSTEM, PROCESS_TREE, WALL_CLOCK}) + +NETWORK_INHERIT = "inherit" +NETWORK_NONE = "none" + +# A grant record is kept this long after release so a restart can tell a reaped +# job from one it never saw, then pruned so the store cannot grow without bound. +_RETENTION_S = 3600 + +# Teardown reads the group liveness probe this often while waiting out the +# grace period. Short enough that a cooperative child is not waited on for the +# full grace, long enough not to spin. +_DEATH_POLL_S = 0.05 + +# Destinations a bind must never overlay: replacing the private root, the +# private /tmp or the workspace itself with a host directory would undo the +# namespace from inside the argv that builds it. +_RESERVED_BIND_DESTS = frozenset({ + "/", "/tmp", "/proc", "/dev", "/sys", "/workspace", +}) + +#: Where the workspace is mounted inside a namespace. The public tool contract +#: already promises this path, so it is the one path a contained command may +#: assume. +WORKSPACE_MOUNT = "/workspace" + + +class ContainmentUnavailable(RuntimeError): + """A required dimension could not be established. Never downgraded. + + Raised by :func:`acquire` under :data:`MODE_ENFORCING`, and by :func:`run` + whenever it is handed a grant whose postcondition does not hold — so a + hand-built grant claiming containment it does not have cannot reach a + spawn. + """ + + def __init__(self, missing: frozenset[str], mechanism_tried: str) -> None: + self.missing = frozenset(missing) + self.mechanism_tried = str(mechanism_tried or "none") + listed = ", ".join(sorted(self.missing)) + super().__init__( + f"containment unavailable ({listed}); strongest mechanism available " + f"was {self.mechanism_tried!r}" + ) + + +# ── Records ───────────────────────────────────────────────────────────────── +@dataclass(frozen=True) +class ContainmentSpec: + """What the caller needs. Declarative, and contains no policy decision. + + ``required`` is the whole contract: those dimensions hold or the command + does not run. Everything else is best-effort and is reported as fact rather + than assumed. + """ + + workspace: str + env: Mapping[str, str] + wall_clock_s: int + required: frozenset[str] = DEFAULT_REQUIRED + network: str = NETWORK_INHERIT + writable_extra: tuple[str, ...] = () + readonly_extra: tuple[str, ...] = () + max_output_bytes: int = MAX_OUTPUT_CHARS + max_memory_bytes: Optional[int] = None + max_processes: Optional[int] = None + + def __post_init__(self) -> None: + # Freeze env into a read-only view over a private copy. The child's + # environment is part of the boundary, so a caller holding the dict it + # passed in must not be able to edit it after acquire() validated it. + object.__setattr__(self, "env", MappingProxyType(dict(self.env or {}))) + object.__setattr__(self, "required", frozenset(self.required or ())) + object.__setattr__(self, "writable_extra", tuple(self.writable_extra or ())) + object.__setattr__(self, "readonly_extra", tuple(self.readonly_extra or ())) + + @property + def requested(self) -> frozenset[str]: + """Dimensions this spec actually asks about. + + A spec that leaves ``network`` inherited is not asking for network + containment, so a mechanism without it is not degraded — it gave the + spec everything the spec wanted. + """ + asked = {FILESYSTEM, PROCESS_TREE, WALL_CLOCK} + if self.network == NETWORK_NONE: + asked.add(NETWORK) + if self.max_memory_bytes is not None: + asked.add(MEMORY) + if self.max_processes is not None: + asked.add(PROCESS_COUNT) + return frozenset(asked) + + +@dataclass(frozen=True) +class ContainmentGrant: + """What was actually established. Never a superset of the spec.""" + + id: str + mechanism: str + workspace: str + enforced: frozenset[str] + degraded: tuple[str, ...] + unenforced_required: tuple[str, ...] + owner: str + mode: str + spec: ContainmentSpec + external: bool = False + pid: Optional[int] = None + #: The child's process group, captured at spawn. Teardown needs it because + #: it outlives the leader's pid: the leader can exit while the processes it + #: backgrounded keep running in the same group. + pgid: Optional[int] = None + + @property + def contained(self) -> bool: + """True when every required dimension is actually enforced.""" + return not self.unenforced_required + + def to_dict(self) -> dict[str, Any]: + """The ``containment`` block a tool result carries. + + Deliberately omits ``env``: it is part of the boundary but it is also + where credentials live, and a tool result is model-visible. + """ + return { + "id": self.id, + "mechanism": self.mechanism, + "mode": self.mode, + "workspace": self.workspace, + "enforced": sorted(self.enforced), + "degraded": list(self.degraded), + "unenforced_required": list(self.unenforced_required), + "contained": self.contained, + "external": self.external, + } + + +@dataclass(frozen=True) +class ContainmentResult: + stdout: str + stderr: str + exit_code: Optional[int] + timed_out: bool + output_truncated: bool + grant: ContainmentGrant + release: Optional["ReleaseOutcome"] = None + + +@dataclass(frozen=True) +class ReleaseOutcome: + """Whether the tree is actually gone, not whether a signal was sent.""" + + dead: bool + escalated: bool + survivors: tuple[int, ...] = () + mechanism: str = "" + + def to_dict(self) -> dict[str, Any]: + return { + "dead": self.dead, + "escalated": self.escalated, + "survivors": list(self.survivors), + "mechanism": self.mechanism, + } + + +# ── Mechanisms ────────────────────────────────────────────────────────────── +@dataclass(frozen=True) +class Mechanism: + """A way to establish containment, and exactly what it is good for. + + ``provides`` is a function of the spec alone — never of the command — so + mechanism selection cannot be influenced by request text. + """ + + name: str + rank: int + available: Callable[[], bool] + provides: Callable[[ContainmentSpec], frozenset[str]] + + +def _bwrap_available() -> bool: + return not IS_WINDOWS and bool(shutil.which("bwrap")) + + +def _posix_group_available() -> bool: + return not IS_WINDOWS + + +def _windows_available() -> bool: + return IS_WINDOWS + + +#: macOS advertises an infinite ``RLIMIT_AS`` hard limit and then refuses every +#: attempt to lower it ("current limit exceeds maximum limit"), so an +#: address-space ceiling is a Linux-only mechanism. Claiming it anywhere else +#: would produce a grant saying `memory` is enforced and a spawn that dies in +#: ``preexec_fn`` — a false claim is worse than an honest absence. +_ADDRESS_SPACE_LIMIT_SUPPORTED = sys.platform.startswith("linux") + + +def _rlimit_fits(name: str, requested: int) -> bool: + """True when ``requested`` is within the inherited hard limit for ``name``. + + A soft limit above the hard limit is rejected by ``setrlimit``, so asking + for one would abort the spawn. Checked here, where it can be reported, not + in the child, where it can only crash. + """ + try: + import resource + except ImportError: # pragma: no cover - POSIX always has it + return False + which = getattr(resource, name, None) + if which is None: + return False + try: + _soft, hard = resource.getrlimit(which) + except (OSError, ValueError): # pragma: no cover - platform dependent + return False + return hard in (resource.RLIM_INFINITY, -1) or requested <= hard + + +def _rlimit_dimensions(spec: ContainmentSpec) -> set[str]: + """Resource dimensions a POSIX ``setrlimit`` in the child can actually hold. + + Probed rather than assumed: a dimension is only claimed when the limit + exists on this platform and the requested value is applicable. + """ + if IS_WINDOWS: + return set() + provided: set[str] = set() + if ( + spec.max_memory_bytes is not None + and _ADDRESS_SPACE_LIMIT_SUPPORTED + and _rlimit_fits("RLIMIT_AS", spec.max_memory_bytes) + ): + provided.add(MEMORY) + if ( + spec.max_processes is not None + and _rlimit_fits("RLIMIT_NPROC", spec.max_processes) + ): + provided.add(PROCESS_COUNT) + return provided + + +def _bwrap_provides(spec: ContainmentSpec) -> frozenset[str]: + # bwrap gives the private root and the workspace bind (filesystem), a new + # session plus --die-with-parent (process_tree), and --unshare-net when the + # spec asked for no network. The wall clock and the resource limits are + # ours either way, applied to the bwrap process itself so its descendants + # inherit them. + provided = {FILESYSTEM, PROCESS_TREE, WALL_CLOCK} | _rlimit_dimensions(spec) + if spec.network == NETWORK_NONE: + provided.add(NETWORK) + return frozenset(provided) + + +def _posix_group_provides(spec: ContainmentSpec) -> frozenset[str]: + # A process group plus setsid makes the kill authoritative and the wall + # clock real for the whole tree. It says nothing about the filesystem: a + # cwd is not a boundary. + return frozenset({PROCESS_TREE, WALL_CLOCK} | _rlimit_dimensions(spec)) + + +def _windows_provides(spec: ContainmentSpec) -> frozenset[str]: + # taskkill /T /F walks the child tree, which is the Windows equivalent of + # signalling a group. There is no setrlimit and no namespace. + return frozenset({PROCESS_TREE, WALL_CLOCK}) + + +#: Strongest first. Selection walks this in order and stops at the first +#: mechanism that covers ``spec.required``; if none does, the strongest +#: available one is used and the shortfall is reported (or raised, under +#: MODE_ENFORCING). Tests substitute this list to drive selection +#: deterministically without needing a real sandbox. +MECHANISMS: tuple[Mechanism, ...] = ( + Mechanism("bubblewrap", 30, _bwrap_available, _bwrap_provides), + Mechanism("process_group", 20, _posix_group_available, _posix_group_provides), + Mechanism("windows_tree", 10, _windows_available, _windows_provides), +) + + +# ── Durable grant records ─────────────────────────────────────────────────── +def _store_path() -> Path: + return Path(CONTAINMENT_STATE_FILE) + + +def _load_records() -> dict[str, dict[str, Any]]: + try: + path = _store_path() + if path.exists(): + data = json.loads(path.read_text(encoding="utf-8")) or {} + if isinstance(data, dict): + return { + str(key): value + for key, value in data.items() + if isinstance(value, dict) + } + except Exception: + # A corrupt or unreadable store must not take out execution. The grant + # itself is authoritative for this process; the file exists so a + # *restart* can reap rather than orphan. + logger.warning("containment: grant store unreadable; starting empty", exc_info=True) + return {} + + +def _save_records(records: Mapping[str, dict[str, Any]]) -> bool: + try: + atomic_write_json(str(_store_path()), dict(records), indent=2) + return True + except Exception: + logger.warning("containment: could not persist grant store", exc_info=True) + return False + + +def _prune(records: dict[str, dict[str, Any]]) -> dict[str, dict[str, Any]]: + now = time.time() + kept = {} + for grant_id, record in records.items(): + released = record.get("released_at") + if released and (now - float(released)) > _RETENTION_S: + continue + kept[grant_id] = record + return kept + + +def _write_record(grant: ContainmentGrant) -> None: + records = _prune(_load_records()) + records[grant.id] = { + "id": grant.id, + "owner": grant.owner, + "mechanism": grant.mechanism, + "mode": grant.mode, + "workspace": grant.workspace, + "enforced": sorted(grant.enforced), + "degraded": list(grant.degraded), + "unenforced_required": list(grant.unenforced_required), + "required": sorted(grant.spec.required), + "wall_clock_s": grant.spec.wall_clock_s, + "max_memory_bytes": grant.spec.max_memory_bytes, + "max_processes": grant.spec.max_processes, + "network": grant.spec.network, + "external": grant.external, + "pid": grant.pid, + "pgid": grant.pgid, + "acquired_at": time.time(), + "released_at": None, + "release": None, + } + _save_records(records) + + +def _update_record(grant_id: str, **fields: Any) -> None: + records = _load_records() + record = records.get(grant_id) + if record is None: + return + record.update(fields) + records[grant_id] = record + _save_records(records) + + +def active_grants() -> list[dict[str, Any]]: + """Grant records that were never released — a restart's reaping input. + + One owner, one record, one place to ask what is running on whose behalf. + """ + return [ + record + for record in _prune(_load_records()).values() + if not record.get("released_at") + ] + + +def forget(grant_id: str) -> None: + """Drop a record outright. For a reaper that has finished with it.""" + records = _load_records() + if records.pop(str(grant_id), None) is not None: + _save_records(records) + + +# ── Spec validation ───────────────────────────────────────────────────────── +def _validate_abs_path(value: str, *, label: str) -> str: + text = str(value or "") + if not text or "\x00" in text: + raise ValueError(f"containment: {label} must be a non-empty path") + if not os.path.isabs(text): + raise ValueError(f"containment: {label} must be absolute, got {text!r}") + if ".." in PurePosixPath(text.replace(os.sep, "/")).parts: + raise ValueError(f"containment: {label} must not contain '..', got {text!r}") + return os.path.normpath(text) + + +def _validate_spec(spec: ContainmentSpec) -> ContainmentSpec: + """Reject a malformed spec loudly, before any mechanism is considered. + + These are caller bugs, not platform shortfalls, so they raise ValueError in + both modes: there is no report-only version of a workspace that is not a + directory. + """ + unknown = set(spec.required) - DIMENSIONS + if unknown: + raise ValueError( + f"containment: unknown required dimension(s) {sorted(unknown)}; " + f"known dimensions are {sorted(DIMENSIONS)}" + ) + # Requiring a dimension the spec never asked for can never be satisfied, + # so it is a contradiction rather than an unavailable mechanism. + contradictory = set(spec.required) - set(spec.requested) + if contradictory: + raise ValueError( + f"containment: required {sorted(contradictory)} but the spec does not " + "request it (set network='none', max_memory_bytes or max_processes)" + ) + if spec.network not in (NETWORK_INHERIT, NETWORK_NONE): + raise ValueError(f"containment: network must be 'inherit' or 'none', got {spec.network!r}") + if not isinstance(spec.wall_clock_s, int) or isinstance(spec.wall_clock_s, bool): + raise ValueError("containment: wall_clock_s must be an int") + if spec.wall_clock_s <= 0: + raise ValueError(f"containment: wall_clock_s must be positive, got {spec.wall_clock_s}") + if spec.max_output_bytes <= 0: + raise ValueError("containment: max_output_bytes must be positive") + for name, value in (("max_memory_bytes", spec.max_memory_bytes), + ("max_processes", spec.max_processes)): + if value is not None and (not isinstance(value, int) or value <= 0): + raise ValueError(f"containment: {name} must be a positive int or None") + for key, value in spec.env.items(): + if not isinstance(key, str) or not isinstance(value, str): + raise ValueError("containment: env keys and values must be str") + if "\x00" in key or "\x00" in value: + raise ValueError("containment: env must not contain NUL") + + workspace = _validate_abs_path(spec.workspace, label="workspace") + if not os.path.isdir(workspace): + raise ValueError(f"containment: workspace is not a directory: {workspace}") + writable = tuple( + _validate_abs_path(path, label="writable_extra") for path in spec.writable_extra + ) + readonly = tuple( + _validate_abs_path(path, label="readonly_extra") for path in spec.readonly_extra + ) + for path in writable + readonly: + if path in _RESERVED_BIND_DESTS: + raise ValueError(f"containment: refusing to bind over reserved path {path}") + return replace(spec, workspace=workspace, writable_extra=writable, readonly_extra=readonly) + + +def agent_spec( + workspace: str, + env: Mapping[str, str], + wall_clock_s: int, + **overrides: Any, +) -> ContainmentSpec: + """Build the spec for a model-reachable spawn. + + One factory so no call site can quietly pass a weaker ``required`` set: + ``required`` is :data:`DEFAULT_REQUIRED` and is not overridable here. + Widening or narrowing it is a change to this module, reviewed as one. + """ + overrides.pop("required", None) + return ContainmentSpec( + workspace=workspace, + env=env, + wall_clock_s=wall_clock_s, + required=DEFAULT_REQUIRED, + **overrides, + ) + + +# ── acquire ───────────────────────────────────────────────────────────────── +def _select(spec: ContainmentSpec) -> tuple[Optional[Mechanism], frozenset[str]]: + """Strongest-first selection. Returns the mechanism and what it provides. + + Deterministic: the only inputs are the spec and each mechanism's + availability probe. The command is not an input and is not in scope here. + """ + best: Optional[Mechanism] = None + best_provided: frozenset[str] = frozenset() + for mechanism in sorted(MECHANISMS, key=lambda item: item.rank, reverse=True): + try: + if not mechanism.available(): + continue + except Exception: + logger.warning( + "containment: availability probe for %s failed; treating as unavailable", + mechanism.name, exc_info=True, + ) + continue + provided = frozenset(mechanism.provides(spec)) & DIMENSIONS + if best is None: + best, best_provided = mechanism, provided + if spec.required <= provided: + return mechanism, provided + return best, best_provided + + +def acquire(spec: ContainmentSpec, *, owner: str) -> ContainmentGrant: + """Establish containment, or refuse. + + Postcondition under :data:`MODE_ENFORCING`, asserted rather than assumed:: + + spec.required <= grant.enforced + + Picks the strongest available mechanism and never substitutes a weaker one + for a required dimension. Under :data:`MODE_REPORT_ONLY` the same shortfall + lands in ``grant.unenforced_required`` and is logged, so the run is + distinguishable from a contained one after the fact. + + :raises ValueError: the spec is malformed (a caller bug, in either mode). + :raises ContainmentUnavailable: a required dimension is unavailable, under + :data:`MODE_ENFORCING`. + """ + owner_id = str(owner or "").strip() + if not owner_id: + # A process with no owner is a process nothing will reap. + raise ValueError("containment: every grant needs an owner") + spec = _validate_spec(spec) + + mechanism, provided = _select(spec) + enforced = provided & spec.requested + missing_required = frozenset(spec.required) - enforced + name = mechanism.name if mechanism else "none" + + if missing_required and CONTAINMENT_MODE == MODE_ENFORCING: + # The command does not run. This is the whole point: "not executed" is + # the one outcome a model cannot mistake for success. + raise ContainmentUnavailable(missing_required, name) + + degraded = tuple(sorted(spec.requested - enforced - spec.required)) + grant = ContainmentGrant( + id=uuid.uuid4().hex[:12], + mechanism=name, + workspace=spec.workspace, + enforced=enforced, + degraded=degraded, + unenforced_required=tuple(sorted(missing_required)), + owner=owner_id, + mode=CONTAINMENT_MODE, + spec=spec, + ) + if missing_required: + logger.warning( + "containment: grant %s for owner %s is NOT contained — required %s " + "not enforced by mechanism %s (report-only mode)", + grant.id, owner_id, sorted(missing_required), name, + ) + elif degraded: + logger.info( + "containment: grant %s enforced %s; best-effort %s unavailable under %s", + grant.id, sorted(enforced), list(degraded), name, + ) + _write_record(grant) + return grant + + +def declare_external_bridge( + spec: ContainmentSpec, *, owner: str, endpoint: str, +) -> ContainmentGrant: + """Record that execution leaves this backend entirely. + + A bridged tool runs in a process this backend does not own, so no local + mechanism can contain it. The honest record is ``enforced=frozenset()`` + rather than a grant implying confinement; this exists so that path has a + record at all instead of looking like an absence of one. + """ + owner_id = str(owner or "").strip() + if not owner_id: + raise ValueError("containment: every grant needs an owner") + spec = _validate_spec(spec) + grant = ContainmentGrant( + id=uuid.uuid4().hex[:12], + mechanism="external_bridge", + workspace=spec.workspace, + enforced=frozenset(), + degraded=(), + unenforced_required=tuple(sorted(spec.required)), + owner=owner_id, + mode=CONTAINMENT_MODE, + spec=spec, + external=True, + ) + logger.info( + "containment: grant %s is external (%s); nothing local contains it", + grant.id, endpoint, + ) + _write_record(grant) + return grant + + +def unavailable_tool_result(exc: ContainmentUnavailable, *, tool: str) -> dict[str, Any]: + """The tool result for a request that could not be contained. + + "not executed" is stated in the error text, not inferred from a missing + output field, so a run that could not be contained reads differently from a + contained run that failed. + """ + listed = ", ".join(sorted(exc.missing)) + return { + "error": f"{tool}: containment unavailable ({listed}); command not executed", + "exit_code": 1, + "containment": { + "mechanism": exc.mechanism_tried, + "mode": CONTAINMENT_MODE, + "enforced": [], + "unenforced_required": sorted(exc.missing), + "contained": False, + "executed": False, + }, + } + + +# ── Launch plumbing ───────────────────────────────────────────────────────── +def _dir_chain(path: str) -> list[str]: + """``--dir`` args for every ancestor of ``path`` inside the private root. + + bwrap mounts into a tmpfs root, so the destination's parents have to exist + before the bind. Stops at the mount points the argv already creates. + """ + args: list[str] = [] + parents: list[str] = [] + parent = os.path.dirname(path) + while parent not in ("/", "", "/tmp", "/etc", "/usr", WORKSPACE_MOUNT): + parents.append(parent) + parent = os.path.dirname(parent) + for directory in reversed(parents): + args.extend(("--dir", directory)) + return args + + +def _bwrap_prefix(spec: ContainmentSpec) -> list[str]: + """The bubblewrap argv establishing the boundary this spec asked for. + + Note what is *not* here, versus the namespace this replaces: ``/home`` and + ``/mnt`` are not bound read-write. Binding the user's whole home directory + into a "workspace confinement" namespace gives back most of what the + namespace was for. Anything a command legitimately needs outside the + workspace is named by the spec, as ``readonly_extra`` or ``writable_extra``. + """ + args = [ + "bwrap", "--die-with-parent", "--new-session", + "--tmpfs", "/", + "--dir", "/usr", "--ro-bind", "/usr", "/usr", + "--symlink", "usr/bin", "/bin", + "--symlink", "usr/lib", "/lib", + "--symlink", "usr/lib64", "/lib64", + "--symlink", "usr/bin", "/sbin", + "--dir", "/etc", "--ro-bind", "/etc", "/etc", + "--dir", "/tmp", "--tmpfs", "/tmp", + "--dev-bind", "/dev", "/dev", "--proc", "/proc", + "--dir", WORKSPACE_MOUNT, "--bind", spec.workspace, WORKSPACE_MOUNT, + ] + for path in spec.readonly_extra: + args.extend(_dir_chain(path)) + args.extend(("--ro-bind", path, path)) + for path in spec.writable_extra: + args.extend(_dir_chain(path)) + args.extend(("--bind", path, path)) + if spec.network == NETWORK_NONE: + args.append("--unshare-net") + args.extend(("--chdir", WORKSPACE_MOUNT)) + return args + + +def _rlimit_preexec(grant: ContainmentGrant) -> Optional[Callable[[], None]]: + """A child-side hook applying the limits the grant actually claimed, or None. + + Only ever applies a dimension in ``grant.enforced``, so the child cannot + attempt a limit the probe already said this platform will refuse. If a limit + nevertheless fails to apply, the exception aborts the spawn: an unlimited + run under a grant that promised a ceiling is the one outcome worse than a + loud failure. + """ + if IS_WINDOWS: + return None + spec = grant.spec + memory = spec.max_memory_bytes if MEMORY in grant.enforced else None + processes = spec.max_processes if PROCESS_COUNT in grant.enforced else None + if memory is None and processes is None: + return None + try: + import resource + except ImportError: # pragma: no cover - POSIX always has it + return None + + def _apply() -> None: # pragma: no cover - runs in the forked child + if memory is not None: + resource.setrlimit(resource.RLIMIT_AS, (memory, memory)) + if processes is not None: + resource.setrlimit(resource.RLIMIT_NPROC, (processes, processes)) + + return _apply + + +def _launch_argv(grant: ContainmentGrant, command: Any, *, argv: bool) -> list[str]: + spec = grant.spec + if argv: + parts = [str(part) for part in command] + if not parts: + raise ValueError("containment: empty argv") + else: + text = str(command or "") + if not text.strip(): + raise ValueError("containment: empty command") + if grant.mechanism == "bubblewrap": + # The namespace brings its own /bin/bash via the read-only /usr. + parts = ["/bin/bash", "-lc", text] + else: + shell = find_bash() + if not shell: + raise RuntimeError( + "containment: no POSIX shell available to run a shell command" + ) + parts = [shell, "-c", text] + if grant.mechanism == "bubblewrap": + return _bwrap_prefix(spec) + parts + return parts + + +def _spawn_kwargs(grant: ContainmentGrant) -> dict[str, Any]: + kwargs: dict[str, Any] = {} + if IS_WINDOWS: + # No setsid; the child gets its own group so a console event cannot + # reach it, and teardown walks the tree with taskkill /T. + kwargs["creationflags"] = getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0x00000200) + return kwargs + # setsid is what makes PROCESS_TREE real: without it a timeout kill reaches + # the wrapper shell and nothing it backgrounded. + kwargs["start_new_session"] = True + preexec = _rlimit_preexec(grant) + if preexec is not None: + kwargs["preexec_fn"] = preexec + return kwargs + + +async def _drain(stream, buffer: list[str], budget: list[int]) -> None: + """Read a stream to EOF, keeping at most ``budget[0]`` bytes. + + Reading past the cap and discarding is deliberate: stopping the read would + block the child on a full pipe, which turns an output cap into a hang. + ``budget[0]`` is set to -1 once anything has actually been dropped, so the + caller reports truncation only when bytes were lost — output that exactly + fills the cap is not truncated. + Each stream gets its own budget so the split between stdout and stderr does + not depend on which reader happened to be scheduled first. + """ + if stream is None: + return + while True: + line = await stream.readline() + if not line: + break + if budget[0] < 0: + continue + if len(line) <= budget[0]: + budget[0] -= len(line) + buffer.append(line.decode("utf-8", errors="replace")) + continue + chunk = line[: budget[0]] + if chunk: + buffer.append(chunk.decode("utf-8", errors="replace")) + budget[0] = -1 + + +async def run( + grant: ContainmentGrant, + command: Any, + *, + argv: bool = False, + stdin: Optional[bytes] = None, + progress_cb: Optional[Callable[[dict], Awaitable[None]]] = None, +) -> ContainmentResult: + """Execute inside an existing grant. + + Enforces the wall clock and the output cap, and on timeout tears the tree + down through :func:`release` so the reported outcome is the observed one. + + :raises ContainmentUnavailable: the grant's postcondition does not hold + under :data:`MODE_ENFORCING`. Re-checked here, at the point of effect, + so a grant that was not produced by :func:`acquire` cannot buy a spawn + by claiming dimensions it does not have. + """ + if grant.external: + raise ValueError( + "containment: an external-bridge grant describes execution this " + "backend does not own; it cannot be run locally" + ) + missing = frozenset(grant.spec.required) - frozenset(grant.enforced) + if missing and grant.mode == MODE_ENFORCING: + raise ContainmentUnavailable(missing, grant.mechanism) + + spec = grant.spec + launch = _launch_argv(grant, command, argv=argv) + proc = await asyncio.create_subprocess_exec( + *launch, + stdin=asyncio.subprocess.PIPE if stdin is not None else asyncio.subprocess.DEVNULL, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + cwd=spec.workspace, + env=dict(spec.env), + **_spawn_kwargs(grant), + ) + # start_new_session makes the child its own group leader, so the group id + # is the child's pid. Captured here rather than at teardown: once the leader + # exits, getpgid can no longer tell us which group its children are in. + pgid = None if IS_WINDOWS else (_pgid_of(proc.pid) or proc.pid) + live = replace(grant, pid=proc.pid, pgid=pgid) + _update_record(grant.id, pid=proc.pid, pgid=pgid, started_at=time.time()) + + out_buf: list[str] = [] + err_buf: list[str] = [] + out_budget = [int(spec.max_output_bytes)] + err_budget = [int(spec.max_output_bytes)] + started = time.time() + readers = [ + asyncio.create_task(_drain(proc.stdout, out_buf, out_budget)), + asyncio.create_task(_drain(proc.stderr, err_buf, err_budget)), + ] + if stdin is not None and proc.stdin is not None: + try: + proc.stdin.write(stdin) + await proc.stdin.drain() + except Exception: + pass + try: + proc.stdin.close() + except Exception: + pass + + async def _progress() -> None: + while True: + await asyncio.sleep(2.0) + if progress_cb: + try: + await progress_cb({"elapsed_s": round(time.time() - started, 1)}) + except Exception: + pass + + progress_task = asyncio.create_task(_progress()) if progress_cb else None + timed_out = False + outcome: Optional[ReleaseOutcome] = None + try: + try: + await asyncio.wait_for(proc.wait(), timeout=spec.wall_clock_s) + except asyncio.TimeoutError: + timed_out = True + outcome = await _release_awaited(live, proc) + except asyncio.CancelledError: + await _release_awaited(live, proc) + raise + finally: + if progress_task is not None: + progress_task.cancel() + try: + await progress_task + except (asyncio.CancelledError, Exception): + pass + for task in readers: + try: + await asyncio.wait_for(task, timeout=1) + except (asyncio.TimeoutError, asyncio.CancelledError, Exception): + task.cancel() + + if not timed_out: + # Even a clean exit goes through teardown: a command that backgrounded + # something leaves the group populated, and leaving it running is the + # leak this boundary exists to close. + outcome = await _release_awaited(live, proc) + else: + _update_record(grant.id, timed_out=True) + + return ContainmentResult( + stdout="".join(out_buf), + stderr="".join(err_buf), + exit_code=proc.returncode, + timed_out=timed_out, + output_truncated=out_budget[0] < 0 or err_budget[0] < 0, + grant=live, + release=outcome, + ) + + +# ── release ───────────────────────────────────────────────────────────────── +# These primitives duplicate the escalating teardown that PR #46 adds to +# core/platform_compat.kill_process_tree. They are here because this branch is +# cut from a lab SHA that predates it, and containment cannot ship a teardown +# that only sends SIGTERM. When #46 lands, release() should delegate to that +# function and the helpers below should go — carrying two copies of a +# process-group kill is exactly the divergence this module exists to end. +def _own_pgid() -> int: + try: + return os.getpgid(0) + except OSError: # pragma: no cover - getpgid(0) does not fail in practice + return -1 + + +def _pgid_of(pid: Optional[int]) -> Optional[int]: + if not pid or IS_WINDOWS: + return None + try: + return os.getpgid(int(pid)) + except (OSError, ProcessLookupError, ValueError): + return None + + +def _group_present(pgid: Optional[int]) -> bool: + """True while any process remains in ``pgid``. + + ``killpg(pgid, 0)`` is the authoritative probe: it raises + ``ProcessLookupError`` once the group is empty, which a per-pid check cannot + tell you — the leader can be gone while its children keep running. The + group id outlives the leader's pid, which is why teardown captures it at + spawn rather than deriving it afterwards. + + Our own group is never reported as present: if ``setsid`` had not applied, + probing it would describe the server, not the child. + """ + if not pgid or pgid <= 0 or IS_WINDOWS: + return False + if pgid == _own_pgid(): + return False + try: + os.killpg(pgid, 0) + return True + except (OSError, ProcessLookupError): + return False + + +def _signal_tree(pid: Optional[int], pgid: Optional[int], sig: int) -> None: + """Signal the whole group, falling back to the leader alone. + + A group that is also *our* group is never signalled: if setsid failed, + killpg would take the server down with the child. + """ + if pgid and pgid > 0 and pgid != _own_pgid(): + try: + os.killpg(pgid, sig) + return + except (OSError, ProcessLookupError): + pass + if pid: + try: + os.kill(int(pid), sig) + except (OSError, ProcessLookupError, ValueError): + pass + + +def _reap_if_child(pid: Optional[int]) -> None: + """Clear a zombie we parented, so "alive" means running. + + ``os.kill(pid, 0)`` succeeds for a zombie and a zombie is still a member of + its process group, so without this a process we just killed is reported as a + survivor indefinitely — nothing else is going to reap it. A pid that is not + our child raises ``ChildProcessError`` and there is nothing to do. + + Only the synchronous :func:`release` reaps. A child being awaited is reaped + through ``proc.wait()`` instead, so this never races the event loop's own + child watcher. + """ + if not pid or IS_WINDOWS: + return + try: + os.waitpid(int(pid), os.WNOHANG) + except (ChildProcessError, OSError, ValueError): + pass + + +def _tree_gone(pid: Optional[int], pgid: Optional[int], *, reap: bool = False) -> bool: + if reap: + _reap_if_child(pid) + return not _group_present(pgid) and not pid_alive(pid) + + +def _outcome_for( + grant: ContainmentGrant, *, dead: bool, escalated: bool, +) -> ReleaseOutcome: + survivors: tuple[int, ...] = () + if not dead: + survivors = tuple(dict.fromkeys( + value for value in (grant.pid, grant.pgid) if value + )) + logger.warning( + "containment: grant %s left survivors after escalation: %s", + grant.id, survivors, + ) + return ReleaseOutcome( + dead=dead, + escalated=escalated, + survivors=survivors, + mechanism=grant.mechanism, + ) + + +def release(grant: ContainmentGrant, *, grace_s: float = 2.0) -> ReleaseOutcome: + """Authoritative teardown: signal the group, escalate, then verify. + + Returns whether the tree is **observed** gone. A caller must not record a + process as killed on anything weaker than ``dead=True`` — reporting an + outcome you did not achieve is how a surviving process becomes invisible. + + This is the synchronous form, for a grant whose process this caller is not + awaiting: a restart reaper, or a detached job. For a child being awaited, + :func:`run` uses the async form, which reaps the leader before verifying — + a zombie still belongs to its process group, so the group probe would + otherwise report a tree that is already gone. + """ + pid, pgid = grant.pid, grant.pgid + if pid is None or (pgid is None and not IS_WINDOWS): + record = _load_records().get(grant.id) or {} + pid = pid if pid is not None else record.get("pid") + pgid = pgid if pgid is not None else record.get("pgid") + try: + pid = int(pid) if pid else 0 + except (TypeError, ValueError): + pid = 0 + try: + pgid = int(pgid) if pgid else None + except (TypeError, ValueError): + pgid = None + grant = replace(grant, pid=pid or None, pgid=pgid) + + if not pid and not _group_present(pgid): + outcome = _outcome_for(grant, dead=True, escalated=False) + _finish_release(grant, outcome) + return outcome + + if IS_WINDOWS: + try: + subprocess.run( + ["taskkill", "/F", "/T", "/PID", str(pid)], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0), + ) + except Exception: + logger.warning("containment: taskkill failed for pid %s", pid, exc_info=True) + deadline = time.monotonic() + max(grace_s, 0.0) + while time.monotonic() < deadline and pid_alive(pid): + time.sleep(_DEATH_POLL_S) + outcome = _outcome_for(grant, dead=not pid_alive(pid), escalated=True) + _finish_release(grant, outcome) + return outcome + + if _tree_gone(pid, pgid, reap=True): + outcome = _outcome_for(grant, dead=True, escalated=False) + _finish_release(grant, outcome) + return outcome + + _signal_tree(pid, pgid, signal.SIGTERM) + escalated = False + deadline = time.monotonic() + max(grace_s, 0.0) + while time.monotonic() < deadline and not _tree_gone(pid, pgid, reap=True): + time.sleep(_DEATH_POLL_S) + if not _tree_gone(pid, pgid, reap=True): + escalated = True + _signal_tree(pid, pgid, signal.SIGKILL) + # SIGKILL cannot be caught, so a short verification window is enough. + # Anything still here is out of our reach — a zombie whose parent is + # not us, or a pid we never owned. + deadline = time.monotonic() + 1.0 + while time.monotonic() < deadline and not _tree_gone(pid, pgid, reap=True): + time.sleep(_DEATH_POLL_S) + + outcome = _outcome_for(grant, dead=_tree_gone(pid, pgid, reap=True), escalated=escalated) + _finish_release(grant, outcome) + return outcome + + +async def _release_awaited( + grant: ContainmentGrant, + proc: "asyncio.subprocess.Process", + *, + grace_s: float = 2.0, +) -> ReleaseOutcome: + """Teardown for a child this coroutine owns. + + Identical contract to :func:`release`, with one necessary difference: the + leader is reaped through ``proc.wait()`` before the group is probed. An + unreaped child is a zombie, a zombie is still a member of its process + group, and so ``killpg(pgid, 0)`` would report survivors for a tree that + has entirely exited — turning every timeout into a false "survivors" + report. + """ + if IS_WINDOWS: + return release(grant, grace_s=grace_s) + + pid, pgid = grant.pid, grant.pgid + _signal_tree(pid, pgid, signal.SIGTERM) + try: + await asyncio.wait_for(proc.wait(), timeout=max(grace_s, 0.05)) + except (asyncio.TimeoutError, ProcessLookupError): + pass + deadline = time.monotonic() + max(grace_s, 0.0) + while time.monotonic() < deadline and not _tree_gone(pid, pgid): + await asyncio.sleep(_DEATH_POLL_S) + + escalated = False + if not _tree_gone(pid, pgid): + escalated = True + _signal_tree(pid, pgid, signal.SIGKILL) + try: + await asyncio.wait_for(proc.wait(), timeout=1.0) + except (asyncio.TimeoutError, ProcessLookupError): + pass + deadline = time.monotonic() + 1.0 + while time.monotonic() < deadline and not _tree_gone(pid, pgid): + await asyncio.sleep(_DEATH_POLL_S) + + outcome = _outcome_for(grant, dead=_tree_gone(pid, pgid), escalated=escalated) + _finish_release(grant, outcome) + return outcome + + +def _finish_release(grant: ContainmentGrant, outcome: ReleaseOutcome) -> None: + if outcome.dead: + _update_record(grant.id, released_at=time.time(), release=outcome.to_dict()) + else: + # Deliberately NOT released: the record stays active so a reaper sees it + # again. A record claiming teardown it did not achieve is the defect + # this reverses. + _update_record(grant.id, release=outcome.to_dict()) diff --git a/tests/test_containment_contract.py b/tests/test_containment_contract.py new file mode 100644 index 000000000..390cd743d --- /dev/null +++ b/tests/test_containment_contract.py @@ -0,0 +1,509 @@ +"""The containment API's own invariants. + +These pin the contract rather than any one mechanism, so they run identically on +a host with bubblewrap and one without: every test substitutes +``containment.MECHANISMS`` with fake mechanisms whose availability and provided +dimensions are stated in the test. No real sandbox, no real process. + +The one thing these tests must prove above all others: a request that cannot be +contained does not execute. That is asserted by recording every spawn attempt +and showing the list is empty. +""" + +import asyncio + +import pytest + +from src import containment + + +@pytest.fixture(autouse=True) +def _isolated_store(tmp_path, monkeypatch): + """Keep grant records out of ./data for every test in this module.""" + store = tmp_path / "containment_grants.json" + monkeypatch.setattr(containment, "_store_path", lambda: store) + return store + + +@pytest.fixture +def workspace(tmp_path): + path = tmp_path / "ws" + path.mkdir() + return str(path) + + +@pytest.fixture +def no_spawn(monkeypatch): + """Record spawn attempts and refuse them, so "did not execute" is provable.""" + attempts = [] + + async def _refuse(*args, **kwargs): + attempts.append(args) + raise AssertionError("containment spawned a process it should not have") + + monkeypatch.setattr(asyncio, "create_subprocess_exec", _refuse) + return attempts + + +def mechanism(name, rank, provides, *, available=True): + return containment.Mechanism( + name=name, + rank=rank, + available=lambda: available, + provides=lambda spec, _provides=frozenset(provides): _provides, + ) + + +def install(monkeypatch, *mechanisms): + monkeypatch.setattr(containment, "MECHANISMS", tuple(mechanisms)) + + +def enforcing(monkeypatch): + monkeypatch.setattr(containment, "CONTAINMENT_MODE", containment.MODE_ENFORCING) + + +def report_only(monkeypatch): + monkeypatch.setattr(containment, "CONTAINMENT_MODE", containment.MODE_REPORT_ONLY) + + +def spec_for(workspace, **kwargs): + kwargs.setdefault("env", {"PATH": "/usr/bin"}) + kwargs.setdefault("wall_clock_s", 5) + return containment.ContainmentSpec(workspace=workspace, **kwargs) + + +ALL = tuple(sorted(containment.DIMENSIONS)) + + +# ── The postcondition ─────────────────────────────────────────────────────── +@pytest.mark.parametrize("provided", [ + frozenset(containment.DEFAULT_REQUIRED), + containment.DIMENSIONS, +]) +def test_required_is_always_a_subset_of_enforced(monkeypatch, workspace, provided): + """spec.required <= grant.enforced, for any mechanism that can satisfy it.""" + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, provided)) + grant = containment.acquire(spec_for(workspace), owner="session-1") + assert grant.spec.required <= grant.enforced + assert grant.contained is True + assert grant.unenforced_required == () + + +def test_enforced_never_exceeds_what_the_spec_requested(monkeypatch, workspace): + """A grant is never a superset of its spec: unrequested dimensions are not claimed.""" + enforcing(monkeypatch) + install(monkeypatch, mechanism("generous", 10, containment.DIMENSIONS)) + grant = containment.acquire(spec_for(workspace), owner="session-1") + # network/memory/process_count were not asked for, so they are not enforced + # even though the mechanism offers them. + assert grant.enforced == frozenset(containment.DEFAULT_REQUIRED) + assert containment.NETWORK not in grant.enforced + assert grant.degraded == () + + +def test_enforced_is_always_within_the_known_dimension_set(monkeypatch, workspace): + """A mechanism cannot invent a dimension the API does not define.""" + enforcing(monkeypatch) + install(monkeypatch, mechanism( + "liar", 10, frozenset(containment.DEFAULT_REQUIRED) | {"telepathy"}, + )) + grant = containment.acquire(spec_for(workspace), owner="session-1") + assert grant.enforced <= containment.DIMENSIONS + + +# ── Deterministic failure: the request does not execute ───────────────────── +async def test_uncontainable_request_does_not_execute(monkeypatch, workspace, no_spawn): + """The headline contract: no mechanism for a required dimension → no process. + + Written as a call site would use the API — acquire, then run — so the proof + covers the whole path and not just the raising function. + """ + enforcing(monkeypatch) + # The only mechanism available cannot do filesystem, which is required. + install(monkeypatch, mechanism( + "group_only", 10, {containment.PROCESS_TREE, containment.WALL_CLOCK}, + )) + + spec = containment.agent_spec(workspace, {"PATH": "/usr/bin"}, 5) + try: + grant = containment.acquire(spec, owner="session-1") + except containment.ContainmentUnavailable as exc: + result = containment.unavailable_tool_result(exc, tool="bash") + else: # pragma: no cover - the point of the test is that this is unreachable + result = await containment.run(grant, "echo hello") + + assert no_spawn == [], "an uncontainable request reached a spawn" + assert result["exit_code"] == 1 + assert "command not executed" in result["error"] + assert "filesystem" in result["error"] + assert result["containment"]["contained"] is False + assert result["containment"]["executed"] is False + assert result["containment"]["unenforced_required"] == ["filesystem"] + # A run that could not be contained is distinguishable from a contained run + # that failed: there is no output key at all, and `executed` is explicit. + assert "output" not in result + + +def test_unavailable_names_every_missing_dimension_and_the_mechanism_tried( + monkeypatch, workspace, +): + enforcing(monkeypatch) + install(monkeypatch, mechanism("group_only", 10, {containment.WALL_CLOCK})) + spec = containment.agent_spec(workspace, {}, 5) + with pytest.raises(containment.ContainmentUnavailable) as caught: + containment.acquire(spec, owner="session-1") + assert caught.value.missing == frozenset({ + containment.FILESYSTEM, containment.PROCESS_TREE, + }) + assert caught.value.mechanism_tried == "group_only" + assert "containment unavailable" in str(caught.value) + + +def test_no_mechanism_at_all_still_refuses_rather_than_running( + monkeypatch, workspace, no_spawn, +): + """With nothing available there is no weaker thing to fall back to.""" + enforcing(monkeypatch) + install(monkeypatch, mechanism("absent", 10, containment.DIMENSIONS, available=False)) + with pytest.raises(containment.ContainmentUnavailable) as caught: + containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + assert caught.value.mechanism_tried == "none" + assert no_spawn == [] + + +async def test_a_forged_grant_cannot_buy_a_spawn(monkeypatch, workspace, no_spawn): + """run() re-checks the postcondition at the point of effect. + + A grant is a plain record, so a caller could construct one claiming + dimensions it does not have. run() refuses it rather than trusting the + record it was handed. + """ + enforcing(monkeypatch) + spec = containment.agent_spec(workspace, {}, 5) + forged = containment.ContainmentGrant( + id="forged", + mechanism="bubblewrap", + workspace=workspace, + enforced=frozenset({containment.WALL_CLOCK}), + degraded=(), + unenforced_required=(), # the lie: claims nothing is missing + owner="session-1", + mode=containment.MODE_ENFORCING, + spec=spec, + ) + with pytest.raises(containment.ContainmentUnavailable): + await containment.run(forged, "echo hello") + assert no_spawn == [] + + +# ── Best-effort dimensions degrade, they do not refuse ────────────────────── +def test_unavailable_best_effort_dimension_is_reported_not_refused(monkeypatch, workspace): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DEFAULT_REQUIRED)) + spec = containment.agent_spec( + workspace, {}, 5, network=containment.NETWORK_NONE, max_memory_bytes=1 << 30, + ) + grant = containment.acquire(spec, owner="session-1") + assert grant.contained is True + assert grant.degraded == (containment.MEMORY, containment.NETWORK) + # And it is reported as fact in the model-visible block, never as permission. + block = grant.to_dict() + assert block["degraded"] == ["memory", "network"] + assert block["contained"] is True + assert "env" not in block + + +def test_a_degraded_dimension_is_never_also_enforced(monkeypatch, workspace): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DEFAULT_REQUIRED)) + spec = containment.agent_spec(workspace, {}, 5, max_processes=16) + grant = containment.acquire(spec, owner="session-1") + assert set(grant.degraded).isdisjoint(grant.enforced) + + +# ── Mechanism selection: strongest first, command-independent ─────────────── +def test_selection_is_strongest_first(monkeypatch, workspace): + enforcing(monkeypatch) + install( + monkeypatch, + mechanism("weak", 10, containment.DEFAULT_REQUIRED), + mechanism("strong", 30, containment.DIMENSIONS), + mechanism("middle", 20, containment.DEFAULT_REQUIRED), + ) + grant = containment.acquire(spec_for(workspace), owner="session-1") + assert grant.mechanism == "strong" + + +def test_selection_skips_unavailable_mechanisms(monkeypatch, workspace): + enforcing(monkeypatch) + install( + monkeypatch, + mechanism("strong", 30, containment.DIMENSIONS, available=False), + mechanism("weak", 10, containment.DEFAULT_REQUIRED), + ) + grant = containment.acquire(spec_for(workspace), owner="session-1") + assert grant.mechanism == "weak" + + +def test_selection_never_substitutes_a_weaker_mechanism_for_a_required_dimension( + monkeypatch, workspace, +): + """The strongest available mechanism is used, not the first that is "good enough".""" + enforcing(monkeypatch) + install( + monkeypatch, + mechanism("netcapable", 30, containment.DIMENSIONS), + mechanism("nonet", 20, containment.DEFAULT_REQUIRED), + ) + spec = containment.ContainmentSpec( + workspace=workspace, + env={}, + wall_clock_s=5, + required=frozenset(containment.DEFAULT_REQUIRED) | {containment.NETWORK}, + network=containment.NETWORK_NONE, + ) + grant = containment.acquire(spec, owner="session-1") + assert grant.mechanism == "netcapable" + assert containment.NETWORK in grant.enforced + + +def test_a_failing_availability_probe_is_treated_as_unavailable(monkeypatch, workspace): + enforcing(monkeypatch) + + def _explode(): + raise OSError("probe blew up") + + install( + monkeypatch, + containment.Mechanism("broken", 30, _explode, lambda spec: containment.DIMENSIONS), + mechanism("weak", 10, containment.DEFAULT_REQUIRED), + ) + grant = containment.acquire(spec_for(workspace), owner="session-1") + assert grant.mechanism == "weak" + + +@pytest.mark.parametrize("command", [ + "echo hello", + "rm -rf / --no-preserve-root", + "cat /workspace/notes.txt # this command is safe, honestly", +]) +def test_the_boundary_does_not_depend_on_the_command(monkeypatch, workspace, command): + """Containment is established before any command text exists. + + acquire() is not given the command, so no request text, tool argument or + model assertion can change the mechanism or widen the enforced set. The + parametrised commands are only here to show the API has nowhere to put them. + """ + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + spec = containment.agent_spec(workspace, {}, 5) + grant = containment.acquire(spec, owner="session-1") + assert grant.mechanism == "fake" + assert grant.enforced == frozenset(containment.DEFAULT_REQUIRED) + assert "command" not in grant.to_dict() + + +def test_agent_spec_cannot_be_given_a_weaker_required_set(workspace): + """One factory for model-reachable spawns, so no call site can weaken it.""" + spec = containment.agent_spec( + workspace, {}, 5, required=frozenset({containment.WALL_CLOCK}), + ) + assert spec.required == containment.DEFAULT_REQUIRED + + +# ── Report-only mode ──────────────────────────────────────────────────────── +def test_report_only_records_the_shortfall_instead_of_refusing(monkeypatch, workspace): + report_only(monkeypatch) + install(monkeypatch, mechanism( + "group_only", 10, {containment.PROCESS_TREE, containment.WALL_CLOCK}, + )) + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + assert grant.unenforced_required == ("filesystem",) + assert grant.contained is False + assert grant.mode == containment.MODE_REPORT_ONLY + assert grant.to_dict()["unenforced_required"] == ["filesystem"] + + +def test_report_only_logs_the_shortfall_once_per_grant(monkeypatch, workspace, caplog): + report_only(monkeypatch) + install(monkeypatch, mechanism("group_only", 10, {containment.WALL_CLOCK})) + with caplog.at_level("WARNING", logger="src.containment"): + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + messages = [record.getMessage() for record in caplog.records] + assert sum("NOT contained" in message for message in messages) == 1 + assert grant.id in messages[0] + + +def test_the_two_modes_differ_only_in_whether_the_shortfall_refuses(monkeypatch, workspace): + install(monkeypatch, mechanism("group_only", 10, {containment.WALL_CLOCK})) + spec = containment.agent_spec(workspace, {}, 5) + + report_only(monkeypatch) + reported = containment.acquire(spec, owner="s") + + enforcing(monkeypatch) + with pytest.raises(containment.ContainmentUnavailable) as caught: + containment.acquire(spec, owner="s") + + assert frozenset(reported.unenforced_required) == caught.value.missing + + +def test_report_only_is_the_shipped_default(): + """Pinned deliberately: merging this must not change behaviour on a host + without bubblewrap. Flipping it is a one-line diff, reviewed as one.""" + assert containment.CONTAINMENT_MODE == containment.MODE_REPORT_ONLY + + +# ── Spec validation: caller bugs raise in both modes ──────────────────────── +@pytest.mark.parametrize("mode", [containment.MODE_ENFORCING, containment.MODE_REPORT_ONLY]) +@pytest.mark.parametrize("overrides, fragment", [ + ({"required": frozenset({"telepathy"})}, "unknown required dimension"), + ({"required": frozenset({containment.NETWORK})}, "does not request it"), + ({"required": frozenset({containment.MEMORY})}, "does not request it"), + ({"wall_clock_s": 0}, "must be positive"), + ({"wall_clock_s": -1}, "must be positive"), + ({"max_output_bytes": 0}, "max_output_bytes must be positive"), + ({"max_memory_bytes": 0}, "max_memory_bytes must be a positive int"), + ({"max_processes": -4}, "max_processes must be a positive int"), + ({"network": "maybe"}, "network must be"), + ({"env": {"A": 1}}, "env keys and values must be str"), + ({"env": {"A": "x\x00y"}}, "must not contain NUL"), + ({"writable_extra": ("relative/path",)}, "must be absolute"), + ({"writable_extra": ("/",)}, "reserved path"), + ({"readonly_extra": ("/workspace",)}, "reserved path"), +]) +def test_a_malformed_spec_raises_in_both_modes( + monkeypatch, workspace, mode, overrides, fragment, +): + monkeypatch.setattr(containment, "CONTAINMENT_MODE", mode) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + spec = spec_for(workspace, **overrides) + with pytest.raises(ValueError, match=fragment): + containment.acquire(spec, owner="session-1") + + +@pytest.mark.parametrize("bad_workspace, fragment", [ + ("", "non-empty path"), + ("relative/ws", "must be absolute"), +]) +def test_a_malformed_workspace_raises(monkeypatch, bad_workspace, fragment): + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + spec = containment.ContainmentSpec( + workspace=bad_workspace, env={}, wall_clock_s=5, + ) + with pytest.raises(ValueError, match=fragment): + containment.acquire(spec, owner="session-1") + + +def test_a_workspace_that_is_not_a_directory_raises(monkeypatch, tmp_path): + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + missing = tmp_path / "nope" + spec = containment.ContainmentSpec(workspace=str(missing), env={}, wall_clock_s=5) + with pytest.raises(ValueError, match="not a directory"): + containment.acquire(spec, owner="session-1") + + +def test_a_grant_without_an_owner_raises(monkeypatch, workspace): + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + with pytest.raises(ValueError, match="needs an owner"): + containment.acquire(spec_for(workspace), owner=" ") + + +def test_the_child_environment_cannot_be_edited_after_acquire(monkeypatch, workspace): + """env is part of the boundary, so the caller's dict is copied and frozen.""" + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + caller_env = {"PATH": "/usr/bin"} + grant = containment.acquire( + spec_for(workspace, env=caller_env), owner="session-1", + ) + caller_env["LD_PRELOAD"] = "/tmp/evil.so" + assert dict(grant.spec.env) == {"PATH": "/usr/bin"} + with pytest.raises(TypeError): + grant.spec.env["LD_PRELOAD"] = "/tmp/evil.so" + + +# ── Durable records: one owner, one record ───────────────────────────────── +def test_a_grant_is_recorded_with_its_owner_and_declared_limits( + monkeypatch, workspace, _isolated_store, +): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + spec = containment.agent_spec(workspace, {}, 7, max_processes=8) + grant = containment.acquire(spec, owner="session-42") + + active = containment.active_grants() + assert [record["id"] for record in active] == [grant.id] + record = active[0] + assert record["owner"] == "session-42" + assert record["wall_clock_s"] == 7 + assert record["max_processes"] == 8 + assert record["required"] == sorted(containment.DEFAULT_REQUIRED) + assert record["pid"] is None + + +def test_releasing_a_grant_with_no_process_clears_it_from_active(monkeypatch, workspace): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + outcome = containment.release(grant) + assert outcome.dead is True + assert outcome.escalated is False + assert outcome.survivors == () + assert containment.active_grants() == [] + + +def test_forget_drops_a_record(monkeypatch, workspace): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + containment.forget(grant.id) + assert containment.active_grants() == [] + + +def test_an_unwritable_store_does_not_take_out_execution(monkeypatch, workspace, caplog): + """The record is observability, not a containment dimension. + + Refusing an authorized command because a journal file could not be written + would be a worse failure than running it, so this degrades loudly and the + grant still describes the boundary accurately. + """ + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + + def _explode(*args, **kwargs): + raise OSError("read-only filesystem") + + monkeypatch.setattr(containment, "atomic_write_json", _explode) + with caplog.at_level("WARNING", logger="src.containment"): + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + assert grant.contained is True + assert any("could not persist" in record.getMessage() for record in caplog.records) + + +def test_a_corrupt_store_does_not_take_out_execution(monkeypatch, workspace, _isolated_store): + enforcing(monkeypatch) + install(monkeypatch, mechanism("fake", 10, containment.DIMENSIONS)) + _isolated_store.write_text("{ this is not json", encoding="utf-8") + grant = containment.acquire(containment.agent_spec(workspace, {}, 5), owner="s") + assert [record["id"] for record in containment.active_grants()] == [grant.id] + + +# ── Execution that leaves the box is declared, not pretended ─────────────── +async def test_an_external_bridge_grant_claims_nothing_and_cannot_be_run_locally( + monkeypatch, workspace, no_spawn, +): + enforcing(monkeypatch) + spec = containment.agent_spec(workspace, {}, 5) + grant = containment.declare_external_bridge( + spec, owner="session-1", endpoint="http://127.0.0.1:8777/exec", + ) + assert grant.mechanism == "external_bridge" + assert grant.enforced == frozenset() + assert grant.external is True + assert grant.contained is False + assert grant.to_dict()["external"] is True + with pytest.raises(ValueError, match="does not own"): + await containment.run(grant, "echo hello") + assert no_spawn == [] diff --git a/tests/test_containment_process_tree.py b/tests/test_containment_process_tree.py new file mode 100644 index 000000000..e70fdcac1 --- /dev/null +++ b/tests/test_containment_process_tree.py @@ -0,0 +1,378 @@ +"""Teardown through the containment boundary, against real processes. + +The headline case is the one that fails on an unmodified baseline: a command +that backgrounds a grandchild and then times out leaves the grandchild running, +while the tool result claims "process killed". These tests pin that the boundary +signals the whole process group and verifies death before reporting it. + +POSIX only — the Windows path walks the tree with ``taskkill /T /F`` and has no +host here to run on, which is stated in the PR rather than skipped silently. +""" + +import os +import signal +import subprocess +import sys +import time +from dataclasses import replace + +import pytest + +from core.platform_compat import pid_alive +from src import containment + +pytestmark = pytest.mark.skipif( + sys.platform.startswith("win"), reason="POSIX process groups; Windows path untested here" +) + + +@pytest.fixture(autouse=True) +def _isolated_store(tmp_path, monkeypatch): + store = tmp_path / "containment_grants.json" + monkeypatch.setattr(containment, "_store_path", lambda: store) + return store + + +@pytest.fixture(autouse=True) +def _real_process_group_mechanism(monkeypatch): + """Pin the mechanism to the real POSIX process group. + + Not a fake: this is the mechanism shipped in ``MECHANISMS``, selected by + name so the test behaves the same on a host that happens to have bubblewrap + installed. Filesystem containment is bubblewrap's job and is not what these + tests are about. + """ + selected = [item for item in containment.MECHANISMS if item.name == "process_group"] + assert selected, "process_group mechanism disappeared from MECHANISMS" + monkeypatch.setattr(containment, "MECHANISMS", tuple(selected)) + monkeypatch.setattr(containment, "CONTAINMENT_MODE", containment.MODE_ENFORCING) + + +@pytest.fixture +def workspace(tmp_path): + path = tmp_path / "ws" + path.mkdir() + return str(path) + + +def tree_spec(workspace, **kwargs): + """A spec requiring exactly what a process group can give.""" + kwargs.setdefault("env", {"PATH": "/usr/bin:/bin:/usr/sbin:/sbin"}) + kwargs.setdefault("wall_clock_s", 1) + return containment.ContainmentSpec( + workspace=workspace, + required=frozenset({containment.PROCESS_TREE, containment.WALL_CLOCK}), + **kwargs, + ) + + +def read_pid(path, *, timeout=5.0): + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + try: + text = path.read_text(encoding="utf-8").strip() + except OSError: + text = "" + if text.isdigit(): + return int(text) + time.sleep(0.02) + raise AssertionError(f"{path} never received a pid") + + +def gone(pid, *, timeout=5.0): + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if not pid_alive(pid): + return True + time.sleep(0.02) + return not pid_alive(pid) + + +# ── The regression this lane exists to close ──────────────────────────────── +async def test_a_timeout_leaves_no_surviving_grandchild(tmp_path, workspace): + """A backgrounded grandchild does not survive the wall-clock kill. + + On a baseline spawn site the wrapper shell is killed with ``proc.kill()`` + and the grandchild keeps running, unowned and unreaped, while the tool + result says the process was killed. + """ + pidfile = tmp_path / "grandchild.pid" + command = f"bash -c 'sleep 60 & echo $! > {pidfile}'; sleep 60" + + grant = containment.acquire(tree_spec(workspace), owner="session-1") + result = await containment.run(grant, command) + + grandchild = read_pid(pidfile) + assert result.timed_out is True + assert gone(grandchild), f"grandchild {grandchild} survived the timeout kill" + assert result.release is not None + assert result.release.dead is True + assert result.release.survivors == () + + +async def test_the_timeout_outcome_is_observed_not_asserted(tmp_path, workspace): + """``dead`` reflects a verified empty process group, not a signal that was sent.""" + pidfile = tmp_path / "child.pid" + command = f"echo $$ > {pidfile}; sleep 60" + grant = containment.acquire(tree_spec(workspace), owner="session-1") + result = await containment.run(grant, command) + leader = read_pid(pidfile) + assert result.timed_out is True + assert result.release.dead is True + assert gone(leader) + assert containment._group_present(result.grant.pgid) is False + + +async def test_a_clean_exit_tears_down_anything_left_behind(tmp_path, workspace): + """A command that returns while leaving a background process does not leak it. + + The leftover closes its inherited pipes (``>/dev/null 2>&1``) so the command + really does complete: a background process still holding the output pipes + keeps the grant open until the wall clock, which is the previous test's case + rather than this one's. + """ + pidfile = tmp_path / "leftover.pid" + command = f"sleep 60 >/dev/null 2>&1 & echo $! > {pidfile}; exit 0" + grant = containment.acquire(tree_spec(workspace, wall_clock_s=10), owner="session-1") + result = await containment.run(grant, command) + leftover = read_pid(pidfile) + assert result.timed_out is False + assert result.exit_code == 0 + assert gone(leftover), f"background process {leftover} outlived its grant" + assert result.release.dead is True + + +# ── Escalation ────────────────────────────────────────────────────────────── +def test_release_escalates_to_sigkill_and_reports_only_verified_death(tmp_path, workspace): + """A SIGTERM-ignoring tree is escalated, and ``dead`` is set only once gone.""" + # The child announces itself only after installing the handler. Without that + # the test races process startup and sometimes measures a child that was + # still using the default SIGTERM disposition. + ready = tmp_path / "ignoring-sigterm" + code = ( + "import signal, time\n" + "signal.signal(signal.SIGTERM, signal.SIG_IGN)\n" + f"open({str(ready)!r}, 'w').write('x')\n" + "time.sleep(60)\n" + ) + proc = subprocess.Popen( + [sys.executable, "-c", code], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + start_new_session=True, + ) + try: + deadline = time.monotonic() + 10 + while time.monotonic() < deadline and not ready.exists(): + time.sleep(0.02) + assert ready.exists(), "child never installed its SIGTERM handler" + grant = containment.acquire(tree_spec(workspace), owner="session-1") + grant = replace(grant, pid=proc.pid, pgid=os.getpgid(proc.pid)) + outcome = containment.release(grant, grace_s=0.3) + proc.wait(timeout=5) + assert outcome.escalated is True + assert outcome.dead is True + assert outcome.survivors == () + finally: + if proc.poll() is None: # pragma: no cover - only on an unexpected failure + proc.kill() + proc.wait(timeout=5) + + +def test_release_does_not_escalate_a_cooperative_tree(workspace): + proc = subprocess.Popen( + [sys.executable, "-c", "import time; time.sleep(60)"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + start_new_session=True, + ) + try: + grant = containment.acquire(tree_spec(workspace), owner="session-1") + grant = replace(grant, pid=proc.pid, pgid=os.getpgid(proc.pid)) + outcome = containment.release(grant, grace_s=2.0) + proc.wait(timeout=5) + assert outcome.dead is True + assert outcome.escalated is False + finally: + if proc.poll() is None: # pragma: no cover + proc.kill() + proc.wait(timeout=5) + + +def test_a_surviving_tree_keeps_its_record_active(workspace, monkeypatch): + """A record moves to released only on verified death. + + With teardown unable to signal anything, ``release`` must report + ``dead=False`` with the survivors named, and must not mark the grant + released — the inverse of marking a job killed without checking. + """ + proc = subprocess.Popen( + [sys.executable, "-c", "import time; time.sleep(60)"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + start_new_session=True, + ) + try: + grant = containment.acquire(tree_spec(workspace), owner="session-1") + grant = replace(grant, pid=proc.pid, pgid=os.getpgid(proc.pid)) + monkeypatch.setattr(containment, "_signal_tree", lambda *args, **kwargs: None) + outcome = containment.release(grant, grace_s=0.2) + assert outcome.dead is False + assert outcome.escalated is True + assert proc.pid in outcome.survivors + active = {record["id"] for record in containment.active_grants()} + assert grant.id in active + assert pid_alive(proc.pid) is True + finally: + proc.kill() + proc.wait(timeout=5) + + +def test_release_never_signals_the_servers_own_process_group(workspace, monkeypatch): + """If setsid had not applied, killpg would take the server down with the child. + + Driven by handing teardown our own group id, which is exactly the state a + failed setsid would leave behind. + """ + sent = [] + monkeypatch.setattr(os, "killpg", lambda pgid, sig: sent.append((pgid, sig))) + monkeypatch.setattr(os, "kill", lambda pid, sig: sent.append(("pid", pid, sig))) + containment._signal_tree(os.getpid(), os.getpgid(0), signal.SIGTERM) + assert all(entry[0] != os.getpgid(0) for entry in sent), sent + assert sent == [("pid", os.getpid(), signal.SIGTERM)] + + +def test_our_own_group_is_never_reported_as_a_childs_group(): + assert containment._group_present(os.getpgid(0)) is False + + +# ── run(): the contained happy path ───────────────────────────────────────── +async def test_run_returns_output_exit_code_and_a_released_record(workspace): + grant = containment.acquire(tree_spec(workspace, wall_clock_s=10), owner="session-9") + result = await containment.run(grant, "echo contained; exit 3") + assert result.stdout == "contained\n" + assert result.exit_code == 3 + assert result.timed_out is False + assert result.output_truncated is False + assert result.grant.pid is not None + assert containment.active_grants() == [] + + +async def test_run_executes_in_the_workspace_and_with_the_declared_env_only(workspace): + grant = containment.acquire( + tree_spec(workspace, wall_clock_s=10, env={"PATH": "/usr/bin:/bin", "MARK": "yes"}), + owner="session-9", + ) + result = await containment.run(grant, 'pwd; echo "MARK=$MARK"; echo "HOME=${HOME:-unset}"') + assert result.exit_code == 0 + assert os.path.realpath(workspace) == os.path.realpath(result.stdout.splitlines()[0]) + assert "MARK=yes" in result.stdout + # The child env is exactly what the spec declared, never an implicit + # inherit, so the server's own environment does not leak into it. + assert "HOME=unset" in result.stdout + + +async def test_run_caps_output_and_says_so(workspace): + grant = containment.acquire( + tree_spec(workspace, wall_clock_s=10, max_output_bytes=16), owner="session-9", + ) + result = await containment.run(grant, "printf 'x%.0s' $(seq 1 500); echo") + assert result.output_truncated is True + assert len(result.stdout.encode("utf-8")) <= 16 + + +async def test_run_accepts_an_argv_command_without_a_shell(workspace): + grant = containment.acquire(tree_spec(workspace, wall_clock_s=10), owner="session-9") + result = await containment.run( + grant, [sys.executable, "-c", "print('argv path')"], argv=True, + ) + assert result.exit_code == 0 + assert result.stdout.strip() == "argv path" + + +async def test_run_feeds_stdin_when_given(workspace): + grant = containment.acquire(tree_spec(workspace, wall_clock_s=10), owner="session-9") + result = await containment.run(grant, "cat", stdin=b"piped\n") + assert result.exit_code == 0 + assert result.stdout == "piped\n" + + +@pytest.mark.parametrize("command, argv", [(" ", False), ([], True)]) +async def test_run_rejects_an_empty_command(workspace, command, argv): + grant = containment.acquire(tree_spec(workspace, wall_clock_s=10), owner="session-9") + with pytest.raises(ValueError, match="empty"): + await containment.run(grant, command, argv=argv) + + +# ── Resource limits, where the platform provides them ─────────────────────── +async def test_a_process_count_limit_is_applied_to_the_child(workspace): + """RLIMIT_NPROC is set in the child, so the ceiling is real where it is claimed.""" + spec = containment.ContainmentSpec( + workspace=workspace, + env={"PATH": "/usr/bin:/bin"}, + wall_clock_s=10, + required=frozenset({ + containment.PROCESS_TREE, containment.WALL_CLOCK, containment.PROCESS_COUNT, + }), + max_processes=64, + ) + grant = containment.acquire(spec, owner="session-9") + assert containment.PROCESS_COUNT in grant.enforced + result = await containment.run( + grant, + [sys.executable, "-c", + "import resource; print(resource.getrlimit(resource.RLIMIT_NPROC))"], + argv=True, + ) + assert result.exit_code == 0 + assert result.stdout.strip() == "(64, 64)" + + +async def test_a_memory_limit_is_claimed_only_where_it_can_be_applied(workspace): + """The claim and the reality agree, on whichever platform this runs. + + macOS reports an infinite RLIMIT_AS hard limit and then refuses to lower it, + so `memory` must come back unenforced there rather than enforced-and-crashing. + Written to assert the consistency rather than the platform, so it is a real + test on Linux and a real test here. + """ + limit = 2 * 1024 * 1024 * 1024 + spec = containment.ContainmentSpec( + workspace=workspace, + env={"PATH": "/usr/bin:/bin"}, + wall_clock_s=10, + required=frozenset({containment.PROCESS_TREE, containment.WALL_CLOCK}), + max_memory_bytes=limit, + ) + grant = containment.acquire(spec, owner="session-9") + probe = [sys.executable, "-c", + "import resource; print(resource.getrlimit(resource.RLIMIT_AS)[0])"] + result = await containment.run(grant, probe, argv=True) + assert result.exit_code == 0, result.stderr + if containment.MEMORY in grant.enforced: + assert result.stdout.strip() == str(limit) + else: + assert containment.MEMORY in grant.degraded + assert result.stdout.strip() != str(limit) + + +def test_an_unenforceable_required_limit_refuses_instead_of_crashing_the_spawn(workspace): + """A limit this platform cannot apply is refused at acquire, not in preexec_fn. + + Skipped where the platform *can* apply it, since then there is nothing to + refuse. + """ + if containment._ADDRESS_SPACE_LIMIT_SUPPORTED: + pytest.skip("this platform can lower RLIMIT_AS, so there is no shortfall") + spec = containment.ContainmentSpec( + workspace=workspace, + env={"PATH": "/usr/bin:/bin"}, + wall_clock_s=10, + required=frozenset({ + containment.PROCESS_TREE, containment.WALL_CLOCK, containment.MEMORY, + }), + max_memory_bytes=2 * 1024 * 1024 * 1024, + ) + with pytest.raises(containment.ContainmentUnavailable) as caught: + containment.acquire(spec, owner="session-9") + assert caught.value.missing == frozenset({containment.MEMORY}) diff --git a/website/configuration-reference.md b/website/configuration-reference.md index fc2a3bdeb..a87f574c7 100644 --- a/website/configuration-reference.md +++ b/website/configuration-reference.md @@ -52,7 +52,7 @@ The source tree reads **108** `ODYSSEUS_*` variables: 78 an operator may want to | Variable | Default | Read in | What it does | |---|---|---|---| | `ODYSSEUS_DATA_DIR` | `get_default_data_dir()` | `src/constants.py:56` (+1 more) | Root directory for every persisted file. Prefer this over the per-path overrides; the rest of `src/constants.py` derives from it. | -| `ODYSSEUS_MAIL_ATTACHMENTS_DIR` | `os.path.join(DATA_DIR, 'mail-attachments')` | `src/constants.py:102` | Dedicated override for the mail attachment store, which otherwise lives under the data directory. | +| `ODYSSEUS_MAIL_ATTACHMENTS_DIR` | `os.path.join(DATA_DIR, 'mail-attachments')` | `src/constants.py:103` | Dedicated override for the mail attachment store, which otherwise lives under the data directory. | ### Model routing and providers @@ -167,7 +167,7 @@ The source tree reads **108** `ODYSSEUS_*` variables: 78 an operator may want to | Variable | Default | Read in | What it does | |---|---|---|---| -| `ODYSSEUS_INTERNAL_BASE` | *unset* | `src/constants.py:178` | Base URL the in-app tool layer uses for loopback HTTP calls. Set it when the app is not reachable at the port it thinks it is bound to. | +| `ODYSSEUS_INTERNAL_BASE` | *unset* | `src/constants.py:179` | Base URL the in-app tool layer uses for loopback HTTP calls. Set it when the app is not reachable at the port it thinks it is bound to. | | `ODYSSEUS_INTERNAL_TOKEN` | *unset* | `core/middleware.py:20` | Security-relevant. Token that lets the in-app tool layer reach admin-gated routes over loopback. Unset generates a fresh per-process token, which is what you want unless something outside the process needs the same value. | ### Integrations (Claude, Codex)