Background scheduled agent runs were aborted as "Stopped by user" when
the web UI was merely open, because the idle /api/email/unread-state
poll was counted as foreground activity while its sibling
/api/email/urgency-state was already excluded.
Fixes#5981
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
* refactor(model-routing): centralize explicit foreground fallback policy
Make foreground fallback an explicit per-user, availability-only policy shared by streaming Chat, non-stream Chat, and Agent runs.
Preserve strict defaults, owner/model and credential boundaries, pinned Agent routes, and truthful per-round provenance/accounting. Carry provider-reported model identifiers through native streaming adapters, non-stream responses, and caches, and keep legacy default_model_fallbacks as tombstoned raw storage that generic settings APIs and agent tools cannot expose or mutate.
* fix(agent-loop): restore rebase-dropped qwen routing, workspace prompt, and temperature clamp
* fix(model-routing): thread selected endpoint identity, fix cost classification and fallback eligibility
* fix(chat): restore stream helpers and harden run stop lifecycle
* fix(model-routing): let numeric provider codes win over symbolic rate-limit statuses
* fix(agent-loop): apply qwen temperature and notes-tool clamps per fallback candidate
* fix(chat): honor queued stop across resend and reload canonical terminal on EOF
* fix(chat): track stop queue and cleanup ownership by per-send generation
* fix(agent-loop): preserve requested temperature for non-qwen fallback candidates
* fix(chat): reserve send ownership before any await and scope stop to the current send
* fix(chat): clear the previous run identity at send reservation
---------
Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
Co-authored-by: StressTestor <212606152+StressTestor@users.noreply.github.com>
* fix(cookbook): record real Windows pid for local serve so Stop kills the model
The Windows-local serve runner recorded Git Bash's `$$`, which is the
MSYS/Cygwin pid, not the Windows pid. Win32 tooling (taskkill,
Get-CimInstance ParentProcessId, Stop-Process) can't match an MSYS pid, so
the frontend Stop-Tree walk found nothing and the llama-server child survived
after Stop, leaving the model loaded and the GPU pinned.
Record the serving shell's true Win32 pid via `/proc/$$/winpid`, falling back
to the outer proc.pid already written from Python when the map is unavailable.
The existing pid-tracking test asserted the buggy `$$` literal at the source
level, so it passed while the feature was broken; update it to the winpid
behavior and add a focused regression test.
* fix(cookbook): make Windows serve pid handoff deterministic
---------
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
* fix(thinking): add deepseek-v4 to thinking model patterns
deepseek-v4-flash emits reasoning_content via the API but was not
recognized in _THINKING_MODEL_PATTERNS (only deepseek-r1 and
deepseek-reasoner were listed). Add the deepseek-v4 prefix so
the model is recognized as thinking-capable.
The between-round _thinkOpen leakage was separately fixed by
PR #5931 (perf(chat): batch live thinking rendering).
Related: #3998, #5931
* test(thinking): cover DeepSeek v4 detection
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
parse_tool_blocks fed <tool_call> wrapper bodies only to the XML
iterators (_iter_xml_invoke/_iter_xml_direct), so the canonical
Qwen/Hermes text-mode form — a bare JSON object like
{"name": "bash", "arguments": {"command": "..."}} inside the
wrapper — parsed to zero tool blocks and the agent never executed
anything. Pattern 4d only matches OpenAI-style blobs with a literal
"function" key, which the Hermes format lacks.
Wrapper bodies are now classified first: a JSON-looking body ({ or [)
is parsed by the new _parse_json_tool_call_body, which requires an
object with a string "name" and rejects a non-object "arguments"
instead of coercing it, then converts through the same
function_call_to_tool_block used by the XML paths so aliases and
per-tool argument formatting stay uniform. JSON-looking bodies fail
closed — they are never rescanned by the XML iterators (including the
unclosed-wrapper and bare-invoke fallbacks), so XML-like text inside
JSON argument values stays data instead of selecting a different tool.
Non-JSON bodies keep the existing XML path unchanged.
Fixes#5187
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
gpt-oss (harmony) ships BUILT-IN tools named python/browser, invoked
with the raw body as the argument (to=python + bare source), while
custom functions use to=functions.NAME + JSON. Exposing our own tools
under those names makes the model answer with the built-in convention:
it emits raw code, the server parses it as JSON, and the request dies
with 'error parsing tool call: raw=import sys, ...'. In streaming mode
Ollama does not report it at all — it truncates the stream, so the turn
arrives as an empty response and the agent loop reads it as a model
stall. bash collides the same way in practice.
Measured on gpt-oss:20b via Ollama /v1, fixed agentic prompt, 12 runs
per arm: python+bash as-is 2/12, python renamed 10/12, both renamed
12/12. 74 HTTP 500s were logged server-side during investigation with
zero surfaced to the client.
Rename the colliding tools on the outbound payload and map the names
back on responses. Transport-only and gated on gpt-oss: every other
model's schemas pass through untouched (asserted in tests).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
* fix: stop stripping the word 'assistant' from rendered text
The QWEN_BARE_MARKER_RE regex in both the Python backend (tool_parsing.py)
and JS frontend (chatRenderer.js) was matching any standalone occurrence of
the word 'assistant' separated by any whitespace, then replacing it with a
space. This caused normal English uses like 'Home assistant' to render as
'Home '.
Fixed by narrowing the word-boundary check from [\t\r\n ] (any whitespace)
to [\r\n] (line boundaries only), so only Qwen-format role-token leaks
(where 'assistant' appears alone on a line) are stripped.
* fix(tests): update bare-marker test expectations for #5971
Move 'x assistant y' from STRIPPED to KEPT (mid-sentence must survive).
Add 'Before\nassistant\nAfter' to STRIPPED (bare-marker on own line).
* fix(ui): strip whitespace-padded assistant role markers
---------
Co-authored-by: samy <samy@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
* fix(sidebar): keep minimized icon rail in sync with per-tab visibility
Per-tab visibility (Customize UI / Appearance checkboxes, stored in
localStorage under `odysseus-ui-visibility`) was only applied to the full
sidebar elements — `UI_VIS_MAP` never targeted the collapsed `#icon-rail`
launchers. So a user who turned a tab off (e.g. Email) in the full view saw
every tab reappear when minimizing the sidebar to the icon rail.
Pair each tool/section selector with its `#rail-*` counterpart (mapping
mirrors `_railToolMap`), so `applyUIVis()` hides the rail launcher too.
Admin feature-flag handling is unaffected: the features-fetch reconcile at
app.js already re-applies `applyUIVis()`, so rail launchers now track admin
disables exactly like their sidebar buttons.
Adds a static regression test asserting every customizable tab pairs its
rail button.
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor(sidebar): extract UI visibility into testable module
Move UI_VIS_MAP, UI_VIS_DEFAULT_OFF, and a pure resolveVisibility() into
static/js/ui_visibility.js so the icon-rail visibility rules are unit
testable without a DOM. app.js applies resolveVisibility() to the document,
replacing the ad-hoc tools-section override with an inline parent rule
(tools-section off hides every tool rail launcher). Add edge-case tests
covering per-tool off, the tools-section parent rule, parent+child combos,
email-section, and the tool-library <-> #rail-archive mapping.
Refs #5985
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
_emit_scalar quotes a frontmatter scalar with json.dumps when it holds
punctuation that would change how the line reads back. _parse_scalar undid that
with a bare raw[1:-1]: it stripped the quotes but never decoded the escapes. So
a description containing ü was written as the escape sequence \u00fc, read
back with that escape still sitting literally in the value, and re-escaped on
the next save. The backslash run doubles every save, so a non-English skill
description degrades into backslash noise after a few edits, and the escapes are
shown verbatim in the skills list and the /skills catalog.
This is not limited to non-ASCII. Any description containing a quote takes the
same path, since the quote is itself what forces the quoted form.
Make the two halves symmetric: emit with ensure_ascii=False, since SKILL.md is
UTF-8 at both ends (skills.py reads it, atomic_write_text writes it) and the
ASCII-escaped form bought nothing; and parse double-quoted scalars with
json.loads, falling back to the previous literal reading when the value is not
valid JSON. Files already corrupted heal one level per load.
ensure_ascii=False on its own would open a smaller hole. json.dumps escapes
every C0 control character but passes NEL, LINE SEPARATOR and PARAGRAPH
SEPARATOR through literally, and parse_frontmatter reads one scalar per line via
str.splitlines(), which breaks on all three. Re-escape those three, and add them
plus the remaining splitlines characters to the set that forces a quoted scalar,
so none of them can reach the file bare.
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
llm_call_async returned raw list content for Mistral thinking models,
breaking callers that expect a str (e.g. auto-title). Match the sync
and streaming parsers by running list content through
_normalize_mistral_content.
Fixes#5435
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
_drawWhirlpool re-armed requestAnimationFrame forever whenever its element
had never been connected to the document. The grace period is there so a
spinner can keep drawing between start() and the caller appending the
element, but it had no deadline: while the element has never been connected
_wpWasConnected stays false, so the guard stays true and the else branch is
unreachable. Any caller that starts a spinner and then takes an early return,
such as an aborted request or a panel that resolved from cache, leaves a loop
redrawing an 84-segment spiral into a detached canvas at one frame per
displayed frame until the tab closes.
Put a 2 second deadline on the grace period. Callers append in the same task
as start(), so that is far more slack than any of them need. A spinner that
is actually in the document is unaffected.
Two supporting changes in the same file:
- Both self-terminate paths now call stop() instead of setting isRunning
directly, so termination always runs one cancelAnimationFrame and never
depends solely on inferring DOM connectivity. Both draw functions bail at
the top when they are no longer running, and _requestFrame() clears rafId
as the callback enters so it is a truthful "a frame is pending" flag.
- start() arms a visibilitychange listener and stop() removes it. A hidden
tab cancels the pending frame, a re-shown tab re-arms it. Chrome throttles
background rAF but does not reliably stop the canvas work, and owning the
listener from start/stop means a dead spinner never leaves one behind.
Adds tests/test_spinner_stops_when_never_attached_js.py, which drives the
real module under node with a fake clock and a manual frame pump. It covers
all four exits and, importantly, the converse: a spinner that is attached
keeps running well past the grace window.
Both the authorize and callback routes built the redirect URI with a
hardcoded `http://` and the Host header. Behind any TLS terminator that
produces `http://host:443/api/email/oauth/google/callback` — the wrong
scheme and, on a split-port setup, a dead port. Google then refuses the
authorize request or the token exchange, so OAuth email is unusable on
every HTTPS deployment unless GOOGLE_OAUTH_REDIRECT_URI is pinned by hand.
uvicorn's proxy-headers middleware already rewrites the scheme from
X-Forwarded-Proto for trusted proxies (on by default, trusting 127.0.0.1),
so request.url.scheme is correct both directly and behind a proxy.
Google requires the callback's redirect_uri to match the authorize one
exactly, so both sites change together. An explicit
GOOGLE_OAUTH_REDIRECT_URI still wins, unchanged.
atomic_write_json/atomic_write_text build their temp filename as
"{path}.tmp.{os.getpid()}". os.getpid() is constant for the life of a
process, so it only ever distinguishes concurrent writers that live in
different OS processes. Odysseus runs as a single long-lived process
per container, so two concurrent writers to the same path (e.g. two
request handlers racing a settings save) always compute the identical
temp path. Whichever finishes os.replace() first removes the shared
tmp file out from under the other, which then raises FileNotFoundError
on its own os.replace() instead of landing its write.
Fix: derive the temp suffix from uuid4() instead of the PID, so every
call gets a distinct temp path regardless of process/thread identity.
routes/prefs_routes.py's _save() had an independent, hand-rolled copy
of the exact same PID-suffix logic (not the shared core.atomic_io
helper other routes already use, e.g. routes/auth_routes.py) with the
same bug. Replaced it with a call to atomic_write_json.
Fixes#5596
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
Slice 2o of the route-domain reorganization (#4082/#4071). Moves
mcp_routes.py (697 lines) into routes/mcp/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
The shim uses sys.modules replacement so sys.modules.pop + re-import,
monkeypatch.setattr(mcp_routes, "MCP_OAUTH_DIR", ...), and __file__
introspection in test_security_regressions.py all reach the canonical
module. One source-introspection path string repointed (line 1001).
Canonical module imports only from core/, src/, and stdlib (zero internal
routes/ coupling). Adds tests/test_mcp_routes_shim.py.
Verified: compileall clean; full suite 4804 passed, 3 skipped.
* perf(chat): batch live thinking DOM updates
* test(chat): cover live thinking scheduler lifecycle
* fix(chat): guard background stop-state, restore live thinking text, drop source-text tests
- _closeOpenThinkingMarkup no longer overwrites currentAccumulated for
backgrounded streams. It now mirrors the guard the delta path already uses
(`if (!_isBg) currentAccumulated = accumulated`). Without it a backgrounded
stream's text is written into the foreground session's stop-state, which
abortCurrentRequest and detachCurrentStream then put in the wrong bubble.
- Split _extractLiveThinkingText into _liveThinkingText (strip every think tag)
and _closedThinkingText (via extractThinkingBlocks). Slicing from the first
<think> to the first </think> pinned the live box to "The" for the rest of the
stream on the `<think>The</think>` + untagged-thinking pattern that the
hasUnclosedThink detection deliberately keeps streaming through.
- The background transition now flushes with rich:true, so a stream that
backgrounds mid-thinking isn't left as pre-wrap plain text permanently.
- Move the throttle to static/js/liveThinkingThrottle.js and import it. The
.mjs suite imports the module instead of slicing it out of chat.js with
vm.runInNewContext and marker comments.
- Replace the source-text assertions in tests/test_live_thinking_scheduler_js.py
with behavioral coverage, per tests/TESTING_STANDARD.md. The .mjs suite grows
from 3 to 6 cases.
- Collapse the duplicated tool_start/agent_step finalizers into one
_endLiveThinkingSection().
* fix(chat): hoist thinking teardown out of the try block so catch can reach it
In an ES module a function declared inside `try { }` is scoped to that block,
and `catch` is a sibling scope rather than a nested one. _closeOpenThinkingMarkup
was declared inside the try and called from catch, so the call threw
ReferenceError and killed the rest of the error path: the stream never
finalized and the thinking block was never torn down.
Declare _closeOpenThinkingMarkup and a new _endThinkingOnTerminalPath next to
the existing _flushLiveThinking / _cancelLiveThinkingWork outer lets and assign
them inside the try, which is the pattern those two already use for exactly
this reason.
Verified against a live stream in a browser: before, clicking stop mid-thinking
logged "_closeOpenThinkingMarkup is not defined" and left no finalized thinking
section; after, the block collapses to "View thinking process" correctly.
* perf(chat): extract live thinking at commit cadence
* fix(chat): bound live thinking work
* test(chat): update stream invariant assertions
---------
Co-authored-by: Léo <leograndcontact@gmail.com>
* fix(history): defer full hydration to model sends
* fix(session): key hydration on real rows, fork through get_session
Two regressions from the display/model-context split, both reproducible
against dev.
The hydration gate compared the cached transcript against the
denormalized sessions.message_count column. That column drifts in normal
operation — _persist_message swallows a failed insert while add_message
has already appended in memory, so the next successful persist writes
rows+1 — and _db_to_session re-read the same column after each reload, so
the shortfall never closed. Every send, edit, delete and truncate on a
warm session re-selected the whole message table: the cost this change
set out to remove, relocated onto the hot path. The other direction was
just as bad — a persist for an uncached session writes message_count = 0,
and a stale-low counter with a partly filled cache meant no hydration at
all and a silently truncated transcript for the model.
sync_session_metadata now reconciles message_count against COUNT(*) on
chat_messages (one indexed count inside the connection it already opens),
and _db_to_session trusts the rows it just loaded. A hydrate always
closes the gap, so the next read is a cache hit.
fork_session read session_manager.sessions directly and never hydrated.
keep_count indexes into source.history, and display pagination no longer
fills that cache, so forking after a restart returned HTTP 200 with an
empty conversation and no error surfaced. It goes through get_session
now.
_hydrate_session_history_from_db is gone with its helper: get_session is
the hydration seam, and rebuilding session.history from raw rows in the
display fallback overwrote the parsed multimodal content and the _db_id
edit/delete keys that had just been set.
Tests drive a real SessionManager over a temp DB instead of a stub that
only proved the stub hydrates — both drift directions, the send path
warm and cold, and a fork taken after a restart. All five fail without
this change. The brittle SQL-text assertions are dropped; the page
bounds are already proven by the response body.
* fix(history): route pagination through canonical handler
---------
Co-authored-by: Léo <leograndcontact@gmail.com>
* perf(ui): stop session loading from blocking shell
* fix(startup): open routes on their own data, retire the loader for good
Follow-up to review on #5927.
- Route openers are now classified by the data they actually read. Only
/email touches the hydrated session list (its new-chat path falls back to
the most recent session's model when no default chat is set), so every
other route opens as soon as module wiring completes instead of queueing
behind /api/sessions. This is the deferred-route half of #5926, which the
first pass left unimplemented.
- index.html's 5s fallback removes the loader node again. Leaving it in the
DOM indefinitely kept _shouldPreserveStartupComposer true forever on a
hung /api/sessions, so the composer stopped clearing on session switch.
- A missing session module settles hydration instead of leaving the sidebar
on "Loading chats…" and dropping the user's route on the floor.
- Startup sequencing moved to static/js/startupShell.js so it can be run by
tests. The source-text assertions in test_startup_shell_session_loading.py
are replaced by node-driven behavioural tests, per tests/TESTING_STANDARD.md.
- Reverted the unrequested loader a11y rework, removed the duplicated inert
writes (the module stops the wave interval through a callback), and moved
the bootstrap row's inline styles into .session-list-bootstrap.
* fix: preserve session bootstrap failure state
---------
Co-authored-by: Léo <leograndcontact@gmail.com>
* fix(email): mark opened messages seen in one IMAP operation
* fix(email): collapse unread opens and ignore stale responses
* fix(email): send seen flags as an IMAP flag list
Wrap the authoritative \\Seen STORE operand in parentheses so strict IMAP servers such as GreenMail accept both cache-miss and cached-open transitions. Tighten the focused fake IMAP contract to reject the previously emitted bare flag atom.
* fix(email): guard stale authoritative opens
* fix(email): report a failed \Seen instead of withholding the message
The authoritative-open contract made a failed STORE fatal to the read: the
cold path raised after the body was already fetched and parsed, and the
cached path discarded an in-memory message to return
{"error": "Failed to mark email read"}. A transient IMAP failure therefore
turned a readable message into one that could not be opened at all.
Being authoritative should mean the reported flag state is truthful, not
that the body is withheld. The read now always returns the message and
carries mark_seen_failed so the client can roll its optimistic unread
marker back:
- _read_email_sync logs and reports a rejected STORE rather than raising,
and only writes the local index/list-cache transition when the provider
accepted it, so local state cannot drift ahead of the mailbox.
- A mailbox that refuses a read-write SELECT (shared archives, some
provider folders) falls back to a read-only selection and reports the
flag failure instead of failing the open.
- The route strips mark_seen_failed before caching, so a one-off failure is
never replayed to later readers.
- mark_seen now defaults to False on _read_email_sync. It was inert before
this branch and now mutates provider state; the one caller that wants it
off already passes it explicitly.
emailInbox and emailLibrary keep the message rendered when mark_seen_failed
is set and restore the unread state, rather than showing a failed reader.
---------
Co-authored-by: Léo <leograndcontact@gmail.com>
The Brain > Add tab rendered only a text input and category select with no
submit control, and Enter submission relied on a deprecated keypress
listener that is not guaranteed to fire, so the form could not be
submitted at all (#5828).
Add a labelled submit button styled like the neighbouring Skill Import
button (theme-io-btn, inline SVG icon), switch the Enter handler to
keydown with preventDefault, ignore IME composition, and pin both submit
paths with a source-level regression test.
Fixes#5828
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
static/app.js carried a near-verbatim copy of the prompt-recall logic in
static/js/composerArrowUpRecall.js, wired as a second capture-phase
keydown listener on the same #message textarea. The copy omitted the
draft guard the module has: it called preventDefault() and
stopImmediatePropagation() unconditionally, then recalled history[0]
over whatever the user had typed.
Because it stopped immediate propagation, the copy won regardless of
registration order — if it ran first the module never saw the event, and
if it ran second the module had already declined to stop propagation on
an unmatched draft. The guard at composerArrowUpRecall.js:109 was
unreachable on the real page, so ArrowUp on a multi-line draft replaced
it with the last sent prompt instead of moving the caret up a line.
Delete the duplicate. The module keeps ownership of ArrowUp/ArrowDown
recall, which is the behavior MODULE_SUMMARY.md documents ("on an empty
composer") and the behavior tests/test_composer_arrow_up_recall_js.py
already pins via test_non_empty_composer_does_not_recall and
test_multiline_caret_navigation_preserved.
Also correct a stale comment in the module that described the deleted
behavior and contradicted the guard 35 lines above it, and add a
regression test asserting app.js does not reintroduce a second handler.
Fixes#5862
The `end` branch of _QWEN_BARE_MARKER_RE had both pipes optional
(`\|?end\|?`), so it also matched a bare `end` between whitespace and
replaced it with a space. Messages containing Ruby, Lua or shell code that
closes a block with a lone `end` had those lines deleted, and ordinary prose
lost the word too.
Require at least one pipe so only real turn markers match; `|end`, `end|`,
`|end|` and `/|end|` strip exactly as before. Applied to the duplicated
pattern in static/js/chatRenderer.js as well.
Fixes#5547
* fix(memory): don't let an unreadable store get overwritten with an empty one
load_all() answered a failed read the same way it answered an empty store:
with []. Every mutation path is a read-modify-write (load the whole file,
change it, save it back), so a failed read became
load_all() -> [] -> [].append(new) -> save([new])
and save() is atomic, so the replacement stuck.
The case that actually destroys data is a store that is READABLE but not
parseable - a truncated file, or one holding {} instead of []. Nothing
obstructs the write, so adding a memory returns HTTP 200 and every memory
already stored is gone. Verified end-to-end against a running instance: on the
current code a truncated memory.json plus one add leaves the file holding only
the new entry. Truncation is reachable - core/database.py rewrites memory.json
during migration with a plain open(.., "w") + json.dump, which is not atomic.
A live exclusive lock is not the dangerous case: it blocks the read and the
os.replace alike, so the save fails too and the store survives. That path
currently 500s and loses nothing.
_read_entries() now returns [] only when the file genuinely does not exist and
raises MemoryStoreUnreadable for every other failure, including a store that
parses but is not a JSON array. load_all() keeps the old lenient behaviour so
display, search and context injection still degrade quietly instead of
breaking chat. The read-modify-write callers switch to load_all_for_update(),
which propagates the error: the memory routes turn it into a 503 and change
nothing, backup import refuses rather than saving only the incoming rows, and
auto-extraction and the audit merge skip the write. The audit merge mattered
most - it rebuilds the whole file from one owner's slice plus everyone else's
rows, so an empty read there dropped every other tenant's memories.
The corrupt-JSON path still gets its one shot at the legacy memory.txt
migration before raising, so that recovery is unchanged.
The two updated fakes gained load_all_for_update because the real class has it;
MagicMock would otherwise hand the import path a Mock instead of the seeded list.
Fixes#5673
* fix(memory): fail closed on the remaining read-modify-write add paths
The strict loader landed with the routes, the backup import and the extractor
converted, but three read-modify-write sinks still called load_all(), which
degrades an unreadable store to []. Two of them are the paths users actually
reach, so the data loss in #5673 stayed reproducible:
- src/ai_interaction.py do_manage_memory, action "add" — reached from ordinary
chat via src/tool_execution.py:793 -> dispatch_ai_tool. "Remember that I
prefer X" against an unreadable store wrote a one-entry file over it and
reported success.
- mcp_servers/memory_server.py, action "add" — the same shape through
_scope_entries(), registered as a built-in in src/builtin_mcp.py.
- src/memory_provider.py NativeMemoryProvider.remember and .delete — wired
into app state in src/app_initializer.py but not consumed outside tests yet,
converted here so the pattern is uniform before it goes live.
The MCP server takes _scope_entries(for_update=True) so list keeps the lenient
read. The edit and delete branches on both tool paths were already fail-closed
by accident — an empty view matches nothing and returns before the save — so
they are left alone.
The three new tests drive the real entry points rather than replaying the
shape, and use a truncated store, which is the case that reads back fine so
nothing stops the save. Each asserts memory.json is byte-identical afterwards;
all three fail on the previous commit with the store overwritten.
* fix(integrations): pin api_call to the SSRF-validated IP
execute_api_call runs check_outbound_url on the target, but that guard only
resolves the host to answer (ok, reason) and hands back no address. The request
right after it opened a plain httpx.AsyncClient, which resolves the host again at
connect time. A base_url host on a low TTL can pass the guard as a public IP and
then flip to 169.254.169.254 for the connect, so the call lands on cloud metadata
with the integration's stored auth headers attached.
Resolve once, remember the IPs the guard actually validated, and pin the client's
socket to that set through a small AnyIO-backed transport. SNI and the Host header
still come from the URL, so TLS and vhost routing are unchanged; connect-time
fallback stays inside the approved address set over one shared deadline. This is
the same pinning the webhook sender and web-fetch paths already do -- api_call was
the last outbound path that skipped it.
Fixes#5513
* fix(integrations): de-duplicate the pinned IP list
_default_resolver calls getaddrinfo(host, None) with no socktype filter, so
glibc returns one record per socktype and a single-homed host comes back three
times over. _validated_ips kept every entry, so the transport pinned the same
address repeatedly and the connect fallback could spend its shared deadline
retrying one dead address instead of moving on to a genuinely different one.
Windows getaddrinfo collapses those duplicate records, which is why the
ip-literal pin test only failed on CI and not locally.
Slice 2m of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves document_routes.py
(1810 lines) and document_helpers.py (243 lines) into routes/document/,
leaving backward-compat sys.modules shims at the old paths. Pure file
reorganization, no behavior change.
Both shims use sys.modules replacement so the `import ... as droutes` +
`droutes.SessionLocal = ...` / `monkeypatch.setattr(droutes, ...)` pattern
in multiple tests, and the `sys.modules.pop("routes.document_helpers")` +
re-import pattern in test_security_regressions.py, all reach the canonical
modules.
The canonical document_routes.py imports helpers from the canonical path
(routes.document.document_helpers), not the legacy shim.
Three source-introspection test sites repointed to the new canonical path:
- test_imap_mailbox_quoting.py
- test_model_helper_owner_scope.py
- test_vision_owner_scope.py (shared with other domains; document entry repointed)
Adds tests/test_document_routes_shim.py to pin the sys.modules shim contract
for both modules.
Verified: compileall clean; full suite 4789 passed, 3 skipped.
Slice 2l of the route-domain reorganization (#4082/#4071). Moves
webhook_routes.py into routes/webhook/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
One source-introspection test repointed (test_api_chat_security.py).
Slice 2k of the route-domain reorganization (#4082/#4071). Moves
vault_routes.py into routes/vault/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
The version pattern in _anthropic_rejects_temperature() required a minor
component, so major-only ids like `claude-opus-5` never matched and the
guard reported that the model accepts `temperature`. Anthropic rejects the
field outright on Opus 4.7+, so every such call returned HTTP 400 and the
stream aborted with zero tokens ("the model returned an empty response").
Make the minor optional and read a missing minor as `.0`. The major is also
capped at 1-2 digits with a no-trailing-digit lookahead, mirroring the
minor: once the minor is optional, a greedy major would swallow the date in
`claude-3-opus-20240229` and read it as version 20240229, dropping
temperature from a model that accepts it.
Fixes#5753
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
The placeholder-restore pass in mdToHtml put code, math, mermaid and
allowed-HTML blocks back with a string replacement, so String.replace read
`$&`, `` $` ``, `$'` and `$$` in the *replacement* as substitution patterns.
A fenced block containing them rendered corrupted: `$&` re-inserted the
placeholder (`perl -pe 's/world/$& again/'` became
`s/world/___CODE_BLOCK_0___amp; again/`), `` $` `` and `$'` spliced in the
surrounding document, and `$$` collapsed to a single `$`.
Pass a function replacer at all four sites, matching the inline-code site
below them, which was already fixed this way. A function's return value is
inserted verbatim with no `$` interpretation.
The inline-code comment claimed `echo $1` would be read as a back-reference;
with a string search value there are no capture groups, so `$1` is already
literal. Reworded to name the four sequences that do corrupt.
Fixes#5663
* fix(skills): replace deprecated utcnow in skill timestamp helper
_now_iso() builds the 'created' value in skill frontmatter. datetime.utcnow()
returns a naive datetime and has been deprecated since Python 3.12, scheduled
for removal. Switch to the timezone-aware datetime.now(timezone.utc), keeping
the serialized YYYY-MM-DDTHH:MM:SSZ shape unchanged so existing skill files
keep parsing.
timezone.utc is used rather than the datetime.UTC alias, which is 3.11+ only.
Adds regression tests covering the deprecation, the serialized shape, and
UTC correctness under a non-UTC local timezone -- the last guards against a
bare datetime.now(), which yields the same shape but local wall time.
Fixes#5697
* test(skills): skip timezone mutation where unsupported
---------
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
The issue-close lifecycle change is narrowly scoped and correct. Closed issues remove the stale \`ready for review\` label and return before normal validation can restore it. Focused regressions cover closure and subsequent edits to a closed issue.
The branch was updated onto current \`dev\`. The focused test, merged-result validation, diff checks, and GitHub CI passed. No blocking review threads remain.
Slice 2j of the route-domain reorganization (#4082/#4071). Moves
search_routes.py into routes/search/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
