"Is this path inside that root" is asked in twenty places in this tree and
answered twenty times by a locally written realpath/commonpath pair. Nine test
files exist because nine call sites each needed their own proof. Each one is
defensible alone; together they are the defect, because the boundary has no
single definition and a site that gets a detail wrong is wrong by itself.
src/path_confinement.py is that definition, and it settles the details the
copies disagreed on. Both sides get canonicalized: comparing a realpath-ed
candidate against a root that was only abspath-ed is the macOS /tmp ->
/private/tmp mismatch that has already produced a false failure here, and
canonicalizing one side is worse than canonicalizing neither. commonpath rather
than startswith, because /a/bc begins with /a/b and is not inside it. A relative
candidate joins the root rather than os.getcwd(), which is whatever directory
the server happens to be running in. NUL and newline are refused with a reason
instead of caught by a bare `except Exception` and reported as an ordinary
escape. Eighteen call sites go through it now. It deliberately does not decide
whether a path is sensitive -- that deny list answers "allowed" rather than
"inside", and it stays with src/tool_execution, which owns it. The one
commonpath left in the tree, in src/workspace_paths.py, stays: that function
translates a host path into a container path, so canonicalizing either side
would change the relative path it computes and break the mapping. It is not a
confinement check.
Two of those sites were weaker than the rest and are fixed rather than moved.
The email attachment check used abspath, which folds `..` but does not resolve
symlinks, so a symlink written into the extraction directory passed it and was
then read through. The skill-reference guard compared a realpath-ed target
against a raw dirname, so on a host where the skills tree is reached through a
symlink the two sides never matched and the guard could not fire.
The execution boundary had two separate holes.
The workspace namespace bound /home and /mnt read-write. On the one platform
where that namespace engages at all, a command inside it reaches outside the
workspace and writes to the user's home directory -- measured by running this
argv on a Linux host with working bubblewrap, not inferred from the source.
Binding the user's whole home directory into a workspace-confinement namespace
gives back most of what the namespace was for. Both are read-only now. The
workspace is also bound writable at its real host path, not only at /workspace:
BashTool's own /tmp redirect rewrites `/tmp/` to `<agent_cwd()>/.tmp/` before
the namespace is built, so the command bwrap receives already names the real
path, and those writes previously landed only because the workspace happened to
sit under the writable /home.
`namespaced or _replace_workspace_alias(...)` chose between a mount namespace
and a regex with nothing in the result saying which one ran. The fallback
rewrites the literal token /workspace in the command string, so a command that
never mentions /workspace is untouched by it and runs on the host unrestricted
-- which is every agent shell command on macOS. Both tools now ask
containment.probe() instead of each deciding for itself, and every bash and
python result carries a containment block naming the mechanism and stating
whether the filesystem dimension actually held. Under enforcing mode the
command is not run and the result says so.
That block reports the filesystem dimension only, and says so in a
reported_dimensions field. The probe knows this host could also give a process
group and a real wall clock, but these two tools still assemble their own
create_subprocess_* call and pass neither, so listing those dimensions would be
exactly the false claim src/containment.py calls worse than an honest absence.
probe() is new on src/containment.py: the same mechanism table and the same
arithmetic as acquire(), stopping before the side effects. acquire() is the
wrong shape for a decision -- it writes a durable grant record, and a record
whose pid is never filled in and whose release() never runs is an entry a
restart reaper keeps finding.
CONTAINMENT_MODE stays report_only. Flipping it refuses every agent shell
command on macOS and on any Linux host without bubblewrap, which is a product
decision rather than a code one.
Smaller things in the same area: the /tmp redirect's makedirs was unguarded, so
a read-only workspace turned a command that merely mentioned `/tmp/` into an
OSError traceback instead of a tool error; it degrades now. WORKSPACE_MOUNT
moved to src/constants.py so the namespace and the path resolvers read one
definition of the contract rather than two. The ".tmp" dirname got a constant,
since it appeared in both tool paths.
One generated artifact moved with it: website/configuration-reference.md pins
the source line where each ODYSSEUS_* variable is read, and three of those
shifted. Regenerated with scripts/generate_env_reference.py; the diff is line
numbers only.
Three existing tests changed. test_workspace_artifact_tool_floor asserted that
an unsafe interpreter prefix produces no `--ro-bind <prefix> <prefix>`, which
now fires on /home because /home is legitimately a read-only base mount.
Asserting the absence of a literal flag string cannot distinguish "the prefix
was rejected" from "the argv mounted that root itself", so it compares the argv
against the no-prefix baseline instead: an unsafe prefix must add nothing.
The Windows bash test asserted dict equality on the
whole result, which makes adding a field to every bash result impossible without
touching a test about tmux; it asserts the shape now. The personal-dir symlink
test grepped the resolver's source for the literal "os.path.realpath", which is
gone because the resolution moved into the shared boundary -- it keeps the
negative assertion that the closure must not grow its own abspath check again,
and the behavioural half now runs against the boundary, where it covers every
call site instead of one closure.
Not verified: the bubblewrap argv is asserted, not executed. There is no bwrap
on macOS, and in Docker it needs --privileged to work at all -- default and
seccomp=unconfined both fail with "Creating new namespace failed", and
--cap-add=SYS_ADMIN fails at pivot_root. The Python tool's
needs_virtual_namespace gate means ordinary Python code gets no namespace even
on a Linux host that could provide one; that is reported now but deliberately
not changed, because it alters the Linux Python path on every call and cannot be
checked from here.
_session_alive collapsed every OSError from killpg(pgid, 0) into "the
group is gone". EPERM means the opposite — the group answered the probe
but holds a process we may not signal — so a session we could not touch
was reported as contained, and a timed-out command that left children
running said it had terminated cleanly.
Resolving PTY_KILL_ESCALATION also named signal.SIGKILL unconditionally,
which does not exist on native Windows. app.py imports this module at
start-up, so that turned a POSIX-only teardown detail into the whole app
failing to import there.
/api/shell/stream starts its PTY child under os.setsid, so the child
leads its own session and process group. The timeout, client-disconnect
and error paths all called proc.kill(), which signals only the group
leader. Creating a group and then signalling only its leader is strictly
worse than never creating one: the descendants are detached from the
server's group as well, so nothing else will ever reach them, while the
route reports "Command timed out after Ns" and exit_code -1 as if the
command were gone.
The kernel's controlling-terminal SIGHUP hid this for well-behaved
children, which is why it reads as working. Anything that ignores
SIGHUP — a nohup'ed job, a daemon, a process that means to outlive its
terminal — survives the kill indefinitely.
Signal the whole group instead, escalate to SIGKILL if it outlives the
grace period, and confirm it is actually gone. The timeout response now
says so when containment could not be established rather than claiming
a clean kill it did not get.
Snapshot current maintainer-preview application changes and regression fixtures for integration into lab. Excludes local runtime data, evaluation outputs and source backups. Focused Python regression selection: 140 passed; full suite not certified.
`add_server` wrapped `json.loads(args)` in a bare `except` that fell back to
`[]`, so an Args value that is not JSON — a bare path, which is what the form's
placeholder invites people to type — registered the server and spawned the
stdio subprocess with an empty argv. Nothing surfaced the loss: the POST
returned 200 and the row persisted with `"args": []`.
The route now returns 400 for an unparseable value, and also for valid JSON of
the wrong shape: `args=5` reached `StdioServerParameters(args=5)` and raised an
unhandled TypeError in the error formatter's `" ".join(...)`.
Both form clients mirror the guard instead of leaving the user to read a 400
they cannot see. `settings.js` stops silently defaulting a bad Args value, and
`admin.js` gains the same client-side parse check plus a `res.ok` branch so a
server-side rejection is not reported as a connection failure.
Ported from public `dev` (`9d5c0319`, #6215 upstream, fixing #6211), with its
test. The `admin.js` hunks are inert on `lab` — `initMcpForm` early-returns
because that form's markup is not in this build — and are carried anyway to
keep the two lines from diverging further.
routes/email_routes.py (7,194 lines), routes/email_helpers.py (2,025) and
routes/email_pollers.py (1,764) were the largest flat email area left in
routes/. They now live in routes/email/, following the pattern fifteen route
areas already use: the canonical module under the package, a shim at the old
path that replaces itself in sys.modules with the canonical object.
The shim matters more here than the move. Dozens of email tests monkeypatch
module attributes, and several pop the module from sys.modules and re-import
it. Without identity between routes.email_routes and routes.email.email_routes
those patches would apply to a different object than the app uses.
Intra-package imports keep the flat path, matching routes/document/ and
routes/gallery/: the shim resolves them and the diff stays a move.
Eight path references in five test modules read these files as source rather
than importing them, so they now read the canonical location. That is the same
adjustment every previous conversion made, and it is why the note/document
shims mention source-introspection tests explicitly.
No behaviour change. Verified with the full suite and under several explicit
collection orders, since a byte-identical routes move has broken tests under
one ordering before.
- Enforce outbound URL policy on all hops via bounded manual redirects in scholarly lookups
- Guard declared Content-Length in EditorDraftRoute before request body parsing
- Keep scholarly lookup timeouts and budgets as internal constants rather than surface env vars
- Narrow TUI threat-model description around demonstrable host shell bridge behavior
- Add behavioral regression tests for redirect security and pre-parsing body size guards
The 256 MiB ceiling was only enforced inside _dump_payload, which runs after
FastAPI has parsed the request and after json.dumps has re-serialised it. By
that point the payload has been materialised several times over, so the limit
rejected an allocation it had already paid for.
Check Content-Length first, on both write routes. A request that omits or lies
about the header still reaches the exact byte count, which remains
authoritative.
* fix(agent): allow remaining actions for an approved task
* fix(agent): make approval continuation control-only
* fix(ci): preserve approval taint and cache-buster contract
* fix(ui): keep tool approvals in current chat
* fix(ui): route tool approvals through chat submit
* test(ui): pin approval submit routing
* fix(agent): complete approval denial flow
* fix(ui): avoid duplicate ask-user close icon
* fix(agent): retain approved tool in continuation set
* revert(ui): keep PR 6113 scoped to approval continuation
* fix(agent): add task and chat approval scopes
* fix(ui): prevent duplicate ask-user close icon
* feat(ui): add ask-user option shortcuts
* fix(compare): route ask-user choices per pane
* fix(agent): keep skill-test approvals to a single action
The chat card now reuses the wire value `approve` to mean chat-session
scope, and `consume()` returned `allow_remaining_actions=True` for it
unconditionally. The skill-test approval route was never updated: it still
sends `approve` meaning "once", and its button still reads "Allow once",
but the grant it got back set `approval_gate_bypassed` for the rest of the
resumed run. That surface wraps the skill body and every transcript byte
as untrusted context, so it is the last place where one click should
ungate everything that follows.
Give `consume()` an explicit `allow_continuation` flag. Callers that own a
resumable chat keep the scope the user picked; callers that do not — the
skill tester, unattended audits — get SINGLE_ACTION and the gate re-arms
behind the sealed action, which is what their label promises.
* fix(ui): cache-bust every module the approval click depends on
chatStream.js, compare/index.js and compare/stream.js all changed
behaviour but kept their old `?v=`, while chat.js and chatRenderer.js were
bumped. A returning browser therefore serves the new chat.js — which now
deliberately leaves the composer empty and clicks the send button — next to
the cached chatStream.js that has no interceptor. With an empty composer
that button sits at `data-mode="newchat"`, so the click opens a new chat
and the approval is dropped.
Bump the three, and version compare/stream.js's chatRenderer import to
match everyone else's so the ask_user keydown listener binds to one module
instance instead of two.
* fix(ui): keep the digit shortcuts off tool approval cards
With an approval card on screen and focus anywhere outside an input, a bare
`1` fired `approve_task` — the widest of the three grants — with no
modifier and no confirmation. That card is the one control whose entire
purpose is deliberate consent after untrusted context influenced the run,
and Deny sits at 3.
Label the card with its kind and skip the shortcut for approvals. Ordinary
ask_user questions keep 1-3.
* fix(compare): restore a pane's ask_user card instead of dropping the choice
renderAskUserCard removes the card as soon as onSubmit accepts, but the
resume loop gave up silently after 10s if the originating stream still owned
the pane. The user saw the click land, the card vanish, and nothing happen,
with no way to get it back.
Re-render the card on that deadline and say why. The reroll case still
returns without sending — that choice belongs to a stream that no longer
exists.
* refactor(chat): drop the unreachable deny branch
`if decision != "deny"` is always true — the deny path returns a
StreamingResponse a few lines above. It reads as if deny still falls
through to the toggle restore.
---------
Co-authored-by: Léo <leograndcontact@gmail.com>
Slice 2p of the route-domain reorganization (#4082/#4071). Moves
task_routes.py (1181 lines) into routes/task/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
The shim uses sys.modules replacement so the `import ... as task_routes` +
`monkeypatch.setattr(task_routes, "SessionLocal", ...)` /
`"get_current_user"` pattern and the `task_routes.__file__` reads in
test_auth_regressions.py all reach the canonical module.
Four source-introspection test sites repointed:
- test_aux_llm_owner_scope.py
- test_model_helper_owner_scope.py
- test_internal_api_base.py
- test_webhook_trigger_auth_exempt.py
Adds tests/test_task_routes_shim.py to pin the sys.modules shim contract.
Verified: compileall clean; full suite 5040 passed, 3 skipped.
* fix(auth): derive the session cookie Secure flag from the request scheme
SECURE_COOKIES only marked the login cookie Secure when it was explicitly
set to true, so an HTTPS login on an install that never set it handed out a
session cookie the browser is happy to send back in cleartext.
Unset now derives the flag from the request: the connection scheme, which
uvicorn's proxy-headers middleware rewrites for the proxies it trusts, or
X-Forwarded-Proto for a terminator that is not on a trusted address. That
is the same test core/middleware.py already applies before sending HSTS, so
the two stop disagreeing about whether a request arrived over TLS. An
explicit true still forces the flag on and an explicit false turns it off
for an install still answering on both HTTP and HTTPS. Strictly more Secure
flags than before and never fewer.
Empty counts as unset, because docker-compose pinned SECURE_COOKIES=false
for every container; the compose files now pass the variable through
unset, the way FASTEMBED_CACHE_PATH already does.
The helper and its decision order come from #3799, which was closed for
being too large to review and whose six replacement PRs dropped this fix.
Part of #3803.
* docs(setup): flag the leftover SECURE_COOKIES=false on upgrades
The old default was false, so an install set up before scheme derivation
can still carry an explicit SECURE_COOKIES=false in its own .env. That
value stays authoritative, so HTTPS logins keep getting a non-Secure
session cookie even after the tracked compose defaults are updated by a
pull. Say so where people look: the security notes and the variable's
own comment in .env.example.
* docs(setup): align TLS guidance with scheme-derived cookies
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
* fix(mcp): stop assuming http://localhost:7000 for the OAuth callback
The MCP OAuth callback origin is wrong on any install not reached at
http://localhost:7000, and on Docker it cannot be corrected at all.
Three sites, one assumption:
- The redirect base fell back to a fixed port 7000. The app binds APP_PORT
natively (app.py, launcher.py) and the macOS launcher defaults to 7860,
where 7000 is AirPlay Receiver, so the callback lands on another service
entirely. The fallback now follows APP_PORT. The hostname stays localhost
rather than internal_api_base()'s 127.0.0.1: this URI is registered with
the authorization server, so changing the host would invalidate the
registrations that already exist.
- The paste-back form hardcoded an http:// action. Serving the page over
HTTPS, Chrome raises its insecure-form interstitial, and overriding that
posts plain HTTP at a TLS port, which fails too. Either way the
authorization code never reaches Odysseus. The action now carries the
scheme the request arrived on.
- OAUTH_REDIRECT_BASE_URL is the only fix available to a Docker install,
because the container listens on 7000 and cannot see the host port map,
but compose never forwarded it and nothing documented it. Both fixed.
* fix(mcp): make the paste-back form action relative and export APP_PORT
Answers the review on #6032. Three of the fixes did not survive contact with
the deployments they targeted.
- The form action derived its scheme from request.url.scheme. uvicorn only
honours X-Forwarded-Proto from a peer inside --forwarded-allow-ips, which
defaults to 127.0.0.1; the Dockerfile CMD sets no override, so a proxy
arriving over the Docker bridge is untrusted and the scheme stays http.
That is mixed content on exactly the HTTPS installs paste-back exists for.
A relative action is resolved by the browser against the origin the page
came from, which is right under every proxy setup, and it drops the Host
header from the page entirely.
- The APP_PORT fallback never fired for the shipped launchers. start-macos.sh,
the generated .app launcher and launch-windows.ps1 all pass --port to
uvicorn without putting the value in the environment, so the motivating
case, macOS on 7860, still registered localhost:7000. Each now exports it.
internal_api_base() and companion pairing read APP_PORT too and were wrong
in the same way.
- .env.example pointed Google MCP servers at OAUTH_REDIRECT_BASE_URL.
add_server writes Desktop App credentials, and Google only accepts loopback
redirects for that client type, so a public origin comes back as
redirect_uri_mismatch. The variable is for the DCR flow; Google stays on the
loopback default and finishes remotely through paste-back.
The Host header is no longer reflected into the page, so the escaping
regression test asserts its absence instead of its escaping.
* fix(personal): run directory indexing off the event loop (#5558)
POST /api/personal/add_directory called rag.index_personal_documents
inline from an async handler, so the whole indexing job (os.walk, file
reads, per-chunk embedding, Chroma inserts) ran on the event loop and
every other request queued behind it. Indexing a real directory froze
the UI and API for 25+ minutes with no sign of life.
Move the blocking section into the threadpool via run_in_threadpool.
personal_docs_manager.add_directory stays inside it because its
refresh_index() re-extracts text across tracked directories, which is
also blocking work. A module-level lock serializes index jobs so the
threadpool move does not introduce parallel jobs racing
PersonalDocsManager's unsynchronized list mutations and file writes;
they previously serialized on the blocked loop, so one-at-a-time is
behavior parity.
* fix(personal): serialize add/remove/reload on an async job lock
The #5558 fix took the job lock INSIDE the threadpool worker and only on the
add path, so (1) remove_directory and /reload mutated PersonalDocsManager's
unsynchronized list/index concurrently with an in-flight add — the inconsistent
state the PR claimed to prevent — and (2) a queued add blocked on the lock while
holding an AnyIO threadpool token, starving the shared pool.
Move the lock to an asyncio.Lock acquired in the async handler BEFORE offloading,
and route add, remove and reload through it. A waiting request now parks on the
event loop instead of pinning a worker, and all three mutators are serialized so
the 'add/remove are serialized and cannot leave inconsistent state' guarantee
holds. remove and reload also run their blocking work off the event loop. The
lock is per-router so each app binds it to its own loop; single-process scope.
Tests: add-vs-remove and add-vs-reload serialization regressions (async via
ASGITransport, since asyncio.Lock deadlocks starlette TestClient's portal); the
existing add-vs-add test converted to the same driver.
* fix(personal): route upload and delete through the index job lock
/api/personal/upload and DELETE /api/personal/file mutated the same
vector and tracking state add/remove/reload serialize on, outside
_index_job_lock and inline on the event loop.
Both now stage async work on the loop, then run the complete transition
(vector writes, disk change, personal_docs_manager update) in one
offloaded critical section under the shared lock, acquired before the
offload so queued requests park on the loop rather than pinning a
threadpool worker.
Adds add-vs-upload and add-vs-file ordering regressions.
* fix(personal): bound multi-file upload memory
---------
Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
* refactor(model-routing): centralize explicit foreground fallback policy
Make foreground fallback an explicit per-user, availability-only policy shared by streaming Chat, non-stream Chat, and Agent runs.
Preserve strict defaults, owner/model and credential boundaries, pinned Agent routes, and truthful per-round provenance/accounting. Carry provider-reported model identifiers through native streaming adapters, non-stream responses, and caches, and keep legacy default_model_fallbacks as tombstoned raw storage that generic settings APIs and agent tools cannot expose or mutate.
* fix(agent-loop): restore rebase-dropped qwen routing, workspace prompt, and temperature clamp
* fix(model-routing): thread selected endpoint identity, fix cost classification and fallback eligibility
* fix(chat): restore stream helpers and harden run stop lifecycle
* fix(model-routing): let numeric provider codes win over symbolic rate-limit statuses
* fix(agent-loop): apply qwen temperature and notes-tool clamps per fallback candidate
* fix(chat): honor queued stop across resend and reload canonical terminal on EOF
* fix(chat): track stop queue and cleanup ownership by per-send generation
* fix(agent-loop): preserve requested temperature for non-qwen fallback candidates
* fix(chat): reserve send ownership before any await and scope stop to the current send
* fix(chat): clear the previous run identity at send reservation
---------
Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
Co-authored-by: StressTestor <212606152+StressTestor@users.noreply.github.com>
* fix(cookbook): record real Windows pid for local serve so Stop kills the model
The Windows-local serve runner recorded Git Bash's `$$`, which is the
MSYS/Cygwin pid, not the Windows pid. Win32 tooling (taskkill,
Get-CimInstance ParentProcessId, Stop-Process) can't match an MSYS pid, so
the frontend Stop-Tree walk found nothing and the llama-server child survived
after Stop, leaving the model loaded and the GPU pinned.
Record the serving shell's true Win32 pid via `/proc/$$/winpid`, falling back
to the outer proc.pid already written from Python when the map is unavailable.
The existing pid-tracking test asserted the buggy `$$` literal at the source
level, so it passed while the feature was broken; update it to the winpid
behavior and add a focused regression test.
* fix(cookbook): make Windows serve pid handoff deterministic
---------
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
Both the authorize and callback routes built the redirect URI with a
hardcoded `http://` and the Host header. Behind any TLS terminator that
produces `http://host:443/api/email/oauth/google/callback` — the wrong
scheme and, on a split-port setup, a dead port. Google then refuses the
authorize request or the token exchange, so OAuth email is unusable on
every HTTPS deployment unless GOOGLE_OAUTH_REDIRECT_URI is pinned by hand.
uvicorn's proxy-headers middleware already rewrites the scheme from
X-Forwarded-Proto for trusted proxies (on by default, trusting 127.0.0.1),
so request.url.scheme is correct both directly and behind a proxy.
Google requires the callback's redirect_uri to match the authorize one
exactly, so both sites change together. An explicit
GOOGLE_OAUTH_REDIRECT_URI still wins, unchanged.
atomic_write_json/atomic_write_text build their temp filename as
"{path}.tmp.{os.getpid()}". os.getpid() is constant for the life of a
process, so it only ever distinguishes concurrent writers that live in
different OS processes. Odysseus runs as a single long-lived process
per container, so two concurrent writers to the same path (e.g. two
request handlers racing a settings save) always compute the identical
temp path. Whichever finishes os.replace() first removes the shared
tmp file out from under the other, which then raises FileNotFoundError
on its own os.replace() instead of landing its write.
Fix: derive the temp suffix from uuid4() instead of the PID, so every
call gets a distinct temp path regardless of process/thread identity.
routes/prefs_routes.py's _save() had an independent, hand-rolled copy
of the exact same PID-suffix logic (not the shared core.atomic_io
helper other routes already use, e.g. routes/auth_routes.py) with the
same bug. Replaced it with a call to atomic_write_json.
Fixes#5596
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
Alexandre Teixeira