Record metrics, triage, invariants, and classifications for the Wave 3
corrective implementation pass. Confirms 0 Wave 3 regressions remaining,
with 12358 tests passing across the repository.
Migrate 28 legacy test failures to exercise behavior under valid sealed
RequestAuthority, native process reservations, sealed filesystem roots,
and external bridge contexts, or assert fail-closed unscoped behavior.
Preserves all design invariants without weakening production authority.
Replace unscrubbed os.environ inheritance with an explicit allowlist
(_SAFE_SUBPROCESS_VARS) containing only variables necessary for bash/python
execution (PATH, locale, terminal, Python virtualenv/site-packages, Windows
essentials) and regex-based credential scrubbing (_SENSITIVE_PATTERN) to
prevent provider tokens, database URLs, and API keys from leaking into child
processes.
Shutdown cleanup must not depend on active record.session capability,
which is cleared on cancellation. Guard cleanup by socket directory
presence so all owned daemons are terminated.
Use monkeypatch.setattr for engine, SessionLocal, ScheduledTask, and TaskRun
in _setup_isolated_db to ensure pytest restores real database engine state on
teardown, preventing downstream test failures like no such table: documents.
Catch ResourceIdentityError during intersection so that normal process exit
or background job termination does not crash child authority creation.
Stale or unverifiable observations are conservatively excluded from the
resulting authority while maintaining identity verification and preventing
PID reuse or renewal.
The suite is 12.1k tests in a single CI job — nearly six minutes of pytest
that every push and every PR waits on in one block, on top of a setup step
that already installs npm, Playwright, FFmpeg and bubblewrap. Split it into
four sections the matrix runs in parallel: 352s becomes a 98s longest pole
locally.
Shards partition by test *file*, not by the area_* taxonomy markers. Those
markers do not partition the suite - a file can carry a hand-applied area_*
mark on top of the one conftest derives from its filename, so a marker-based
split would run those tests in more than one section. Assignment is a total
function of the file path instead, so every file lands in exactly one shard
and the four together run every test exactly once.
Sharding deselects rather than narrowing collection, so every test module is
still imported, in the same order, in every shard. The import-time stubbing
in conftest and the session-scoped static server behave identically whether
the suite runs whole or in sections - this suite has known collection-order
coupling and splitting by path would have walked into it.
Balance uses the existing `slow` marker as its weight signal rather than a
committed duration table that would go stale unnoticed. Files pack
heaviest-first into the lightest shard, which is deterministic for a given
file set, so every parallel job computes the same plan from the same commit.
Verified: the four shards together reproduce the full run exactly - 12141
tests selected across the four, and the same 59 failures, 78 skips and 2
xfails, by node ID and not merely by count.
- Regenerate website/configuration-reference.md: Wave 5B moved the
ODYSSEUS_BROWSER_SCREENSHOT_DIR read in web_tools.py (3458 -> 3479).
- Give the Chrome sweep regression fixture a real process identity (stat
start time, boot id, process_ownership.PROC_ROOT). The sweep now signals
only verified identities; the old cmdline-only fixture borrowed the
identity of whatever real process held pid 101 on the host, so it passed
or failed depending on the machine.
- Import pytest in test_workspace_artifact_tool_floor.py: its existing
bubblewrap capability skip raised NameError on hosts without functional
namespaces.
No production code changes. Required containment still fails closed.
Capture the PTY leader's ProcessIdentity and its own session group
immediately after spawn, while the child is held unreaped, and drive
teardown from that frozen record instead of re-deriving the group from
proc.pid. Every signal re-verifies the leader: OWNED and still leading
the group signals the group; GONE signals only the recorded group, never
the pid; FOREIGN proves the group's lifetime ended and nothing is
signalled; UNVERIFIABLE or a missing spawn identity signals nothing.
The server's own process group is never recorded or signalled.
Extract the generic process lifecycle layer (src/process_lifecycle.py)
shared by runtime-owned subprocesses: process identity (pid + boot-bound
start token), identity-bound observation, group and pidfd probes, the
TERM -> verify -> KILL -> verify escalation with re-gating before
escalation, identity-scoped sweeps, and the termination receipt.
Containment, the PTY shell, the Cookbook survivor sweep, the browser
lifecycle, web_tools browser cleanup, kill_process_tree and the startup
reaper consume it while keeping their own ownership semantics.
Safety corrections:
- browser membership and identity are bound in one snapshot; no identity
is recaptured after membership is decided
- web_tools legacy pid-file and profile-match kills signal only verified
identities; browser CLI groups only while their spawn identity verifies
- Cookbook and legacy-tmux descendant capture bind membership to identity
- PTY teardown never signals the server's own process group
- unverifiable processes are reported, never signalled
- Narrowly guard _request_privileges() in routes/chat_routes.py against
synthetic requests lacking scope['app'] or auth manager state, safely
returning empty privileges without granting agent privileges.
- Add focused regression test in tests/test_context_resolution_route.py
verifying that requests without app scope do not crash and cannot gain
agent privileges or qualify for compact preview runtime.
- Regenerate website/configuration-reference.md mechanically to align with
current source line numbers.
The chat route repeated the compact (clean v3) eligibility decision inline
to prepare the turn's context resolution, while the agent loop dispatched
on the contract stamp set by a separate, later condition. The two could
drift, and already disagreed for a user whose privileges demote the turn
to plain chat: the route prepared a compact resolution that no compact
runtime used.
src/agent_runtime/runtime_selection.py (no imports) now owns the rule:
- uses_compact_preview_runtime(): clean route requested, contract policy
enabled, agent mode, agent permitted, not an image generation session.
- is_compact_preview_contract() and COMPACT_PREVIEW_MODE for the stamp.
The route evaluates the rule once, before context preparation, where all
of its facts are final (the agent privilege is read through the same
_request_privileges helper the later enforcement uses). That one value
gates the typed context resolution and is the _clean_v3_preview flag that
stamps the contract; inside the agent-contract branch it equals the
previous condition, so stamping behavior is unchanged. The agent loop
dispatches through is_compact_preview_contract(), and the compact runtime's
MODE is the shared constant.
A route-level matrix drives the real agent loop and asserts that route
preparation and compact dispatch agree for compact, escalated, configured
compact/full, regular, TUI, privilege-denied and image-generation turns.
The first checkpoint removed terminal-metrics discovery, but a normal
compact chat turn still ran two context systems: build_chat_context's
legacy untyped lookup (directly or inside maybe_compact) and the typed
resolver inside stream_preview.
Resolve the typed ContextResolution once, at the chat route, before
build_chat_context, using the session's provider credentials. The
predicate mirrors _clean_v3_preview; every input it needs is known at
that point and the native-workspace term cannot veto a requested clean
route. The same object then:
- sizes legacy history shaping in build_chat_context through a new
maybe_compact(context_length=...) override, so no legacy probe runs;
an unknown window still shapes with DEFAULT_CONTEXT but gains no
provenance;
- crosses stream_agent_loop (one new parameter, forwarded only at the
compact dispatch) into stream_preview, which reuses it and probes only
for callers that arrive without one or with one bound to another
route.
ContextResolution now records the endpoint and model it describes
(endpoint URL excluded from repr and metrics). The bare legacy
context_length is never converted into typed evidence.
Credential scoping: origins compare with default ports normalized, an
empty host is never trusted, and the probe client never follows
redirects. Tests cover the configured origin, the server-resolved
Tailscale form, scheme/port/lookalike/userinfo/path origins, redirects,
and secret-free errors, logs and metrics.
The conftest guard now replaces only the resolver's I/O edges (HTTP
client and DNS-capable URL building) instead of the whole probe, and
exposes a context_probe_ledger fixture, so route integration tests run
the real resolver offline and can count metadata requests.
The compact (clean v3) runtime had no effective context window: it learned a
limit only reactively from a provider 400/413 and its terminal metrics carried
no context_length. PR #41 addressed the reporting gap by probing provider
metadata between the last model byte and [DONE], unauthenticated, and folded
known-table and endpoint evidence into one "known" flag.
Resolve the window once, before the first model request, instead:
- src/agent_runtime/context_resolution.py adds a typed ContextResolution
(effective value, evidence class, source, all observations, conflicts,
provider_io, cached, secret-free probe errors). Evidence classes stay
distinct: runtime_confirmed (llama.cpp /slots, /props, or a limit the
provider stated this turn), provider_advertised (models catalog),
operator_declared (client_runtime_context.model_context_window),
known_table, unknown (0, never a default).
- Selection is deterministic: runtime beats provider beats table; an
operator declaration caps measured evidence and replaces weaker evidence.
Disagreements are recorded as conflicts; a declaration below a measured
value is a cap, above it a contradiction.
- The provider probe forwards the turn's credentials only to the provider's
own origin, runs URL resolution off the event loop, is bounded by one
deadline, never raises, and caches remote results per credential
fingerprint (shorter TTL for failures; local servers are re-probed).
- stream_preview resolves at preparation (or accepts a supplied resolution),
seeds the proactive trim budget from it when evidence is not unknown, and
terminal metrics report only the stored resolution plus any limit the
provider stated during the turn. Metrics perform no discovery.
src/agent_loop.py and the regular runtime's legacy model_context probe are
unchanged. A conftest guard keeps tests that drive the compact runtime with
placeholder endpoints from performing real DNS/HTTP lookups.
A batch whose open succeeded but whose later command failed was recorded
as a failed navigation, so a following observation was wrongly labelled
stale. Use the per-command rows; when the outcome cannot be determined,
treat the page as unknown instead of claiming either result.
Alexandre Teixeira