Compare commits

...
Author SHA1 Message Date
StressTestor 166a6e9cf6 fix(agent): drop the empty assistant turn from an approved-action replay
The approved-action replay appends the sealed tool result with no assistant
prose for that round, which produced an assistant message with content "".
Anthropic's Messages API rejects a non-final assistant message with empty
content, so a resumed turn after a tool approval failed before the model saw
the result. A turn carrying neither prose nor reasoning has nothing to say to
any provider, so it is no longer appended. A round with prose, and a
reasoning-only round that DeepSeek thinking mode needs, both still append.
2026-08-19 03:36:28 -06:00
Dividesbyzer0 43682d4e2e fix(cookbook): activate local Windows venv in bash runner (#5734) 2026-08-18 16:19:33 +02:00
RaresKeY 032967af4b fix(models): show API models by default (#6089) 2026-08-17 13:41:04 +02:00
RaresKeY 0e03aea134 fix(models): align API model checkbox state (#6087) 2026-08-17 11:09:02 +01:00
Joeseph Grey 2a6b09b968 Merge pull request #6081 from ydonghao/refactor/routes-task-to-subdir
refactor(routes): move task domain into routes/task/ subpackage
2026-08-16 22:29:43 -06:00
yuandonghao 1a2d889c33 refactor(routes): move task domain into routes/task/ subpackage
Slice 2p of the route-domain reorganization (#4082/#4071). Moves
task_routes.py (1181 lines) into routes/task/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.

The shim uses sys.modules replacement so the `import ... as task_routes` +
`monkeypatch.setattr(task_routes, "SessionLocal", ...)` /
`"get_current_user"` pattern and the `task_routes.__file__` reads in
test_auth_regressions.py all reach the canonical module.

Four source-introspection test sites repointed:
- test_aux_llm_owner_scope.py
- test_model_helper_owner_scope.py
- test_internal_api_base.py
- test_webhook_trigger_auth_exempt.py

Adds tests/test_task_routes_shim.py to pin the sys.modules shim contract.

Verified: compileall clean; full suite 5040 passed, 3 skipped.
2026-08-17 10:07:17 +08:00
Boody 517946d778 Merge pull request #5911 from Mubelotix/patch-1
docs(readme): Fix Star History section in README
2026-08-17 02:55:27 +03:00
RaresKeYandAlexandre Teixeira 8cb8b074a4 fix(docs): map live VectorRAG result shapes (#5960)
* fix(docs): map live VectorRAG result shapes

* fix(docs): normalize optional VectorRAG fields

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-08-17 00:07:12 +01:00
RaresKeYandAlexandre Teixeira ee252e7cd9 fix(chat): preserve URL prefetch failures in context (#5954)
* fix(chat): preserve URL fetch failures in context

* fix(chat): avoid duplicating signed URLs in fetch failures

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-08-17 00:01:10 +01:00
RaresKeY 0af6a99e81 refactor(search): extract outbound fetch transport (#5953) 2026-08-16 23:43:04 +01:00
RaresKeY f562bfee01 fix(speech): define the Kokoro optional install contract (#5962) 2026-08-16 23:39:12 +01:00
RaresKeY 0728b994d8 fix: discover sessions from persisted messages (#5938)
* fix(session): discover sessions from persisted messages

Use indexed chat-row existence instead of stale derived message_count metadata during startup discovery, then repair the bounded in-memory counts so lazy hydration remains correct. Keep truly empty sessions excluded and cover stale-low and stale-high counts with real SQLite.

* test(session): isolate discovery database

* test(session): use manager database metadata
2026-08-16 23:34:27 +01:00
RaresKeY db05175e3e fix(cli): generate live task webhook URLs (#5956) 2026-08-16 23:28:26 +01:00
RaresKeY e4046aa41f fix(models): bind provider detection to DNS labels (#5961) 2026-08-16 23:25:46 +01:00
RaresKeY 71f30fcc9d fix(issues): require exact bug-report revisions (#5984) 2026-08-16 23:04:23 +01:00
LéoandAlexandre Teixeira b19d327f03 fix(auth): derive the session cookie Secure flag from the request scheme (#6048)
* fix(auth): derive the session cookie Secure flag from the request scheme

SECURE_COOKIES only marked the login cookie Secure when it was explicitly
set to true, so an HTTPS login on an install that never set it handed out a
session cookie the browser is happy to send back in cleartext.

Unset now derives the flag from the request: the connection scheme, which
uvicorn's proxy-headers middleware rewrites for the proxies it trusts, or
X-Forwarded-Proto for a terminator that is not on a trusted address. That
is the same test core/middleware.py already applies before sending HSTS, so
the two stop disagreeing about whether a request arrived over TLS. An
explicit true still forces the flag on and an explicit false turns it off
for an install still answering on both HTTP and HTTPS. Strictly more Secure
flags than before and never fewer.

Empty counts as unset, because docker-compose pinned SECURE_COOKIES=false
for every container; the compose files now pass the variable through
unset, the way FASTEMBED_CACHE_PATH already does.

The helper and its decision order come from #3799, which was closed for
being too large to review and whose six replacement PRs dropped this fix.

Part of #3803.

* docs(setup): flag the leftover SECURE_COOKIES=false on upgrades

The old default was false, so an install set up before scheme derivation
can still carry an explicit SECURE_COOKIES=false in its own .env. That
value stays authoritative, so HTTPS logins keep getting a non-Secure
session cookie even after the tracked compose defaults are updated by a
pull. Say so where people look: the security notes and the variable's
own comment in .env.example.

* docs(setup): align TLS guidance with scheme-derived cookies

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-08-16 22:56:36 +01:00
Léo d0bf771f9d perf(static): vendor KaTeX and Mermaid, and load them on first use (#5994)
* fix(static): vendor KaTeX and Mermaid instead of loading them from a CDN

index.html pulled katex.min.{js,css} and mermaid.min.js from cdn.jsdelivr.net on
every page load. For self-hosted software that is three problems at once: an
air-gapped or offline install renders no math and no diagrams at all, every
session announces its IP, User-Agent and Referer to a third party, and the "runs
on your own hardware" promise quietly isn't true.

static/lib/ already vendors highlight.js, docx, xlsx, mammoth, html2pdf and
qrcode, so the CDN usage was an inconsistency rather than a policy. Vendoring
also pins Mermaid, which was floating on the `11` tag, to 11.16.1.

Behaviour is unchanged: both libraries still load eagerly from <head>, just from
this machine.

- KaTeX goes in its own directory because its stylesheet resolves fonts with a
  relative url(fonts/...), so the vendored CSS needs no rewrite. Only the .woff2
  variants ship, matching static/fonts/, since a browser that supports woff2
  never requests the .woff/.ttf alternatives the stylesheet also lists.
- The service worker precaches KaTeX and its fonts so offline math is typeset
  rather than falling back to system glyphs, and CACHE_NAME is bumped. Mermaid
  is left to the existing cache-first rule: at 3.5 MB, precaching it would mean
  re-downloading it on every cache bump for a library most sessions never touch.
- Licence texts travel with the bundles in licenses/, following the convention
  the repo already uses for OpenDyslexic and DeepResearch.
- .gitattributes turns the whitespace check off for static/lib/ so `git diff
  --check` passes without stripping bytes from the published npm artifacts,
  which would desync them from upstream.

* perf(markdown): load KaTeX and Mermaid on first use, not on every page load

Both libraries loaded eagerly from <head>, costing every session ~985 KB on the
wire (929 KB of that Mermaid) even though most chats contain neither a formula
nor a diagram. Measured on a cold profile via the Resource Timing API: JS bytes
per page load drop from 3,102,141 to 2,098,634, a saving of 1,003,507 bytes, and
third-party requests per load go from 3 to 0.

markdown.js now fetches each library the first time one is actually needed:

- renderMermaid() checks for an unprocessed mermaid fence before touching the
  network, and re-queries the DOM after the load so a diagram replaced mid-stream
  still renders.
- mdToHtml() is synchronous, so when KaTeX is not in yet it banks the math source
  in an inert placeholder and schedules a flush that loads the library and swaps
  the placeholders in. Once KaTeX is loaded it typesets inline exactly as before,
  so callers that never call a render helper still get their math.

Both loaders memoise the promise rather than the module, so concurrent callers
share one fetch and a double trigger cannot start two loads; a failed load clears
the memo so the next formula retries instead of being poisoned for the session.
The flush is scheduled with setTimeout rather than requestAnimationFrame, which
is throttled to a stop in a background tab and never fires at all in a headless
browser, so math would have sat as plain source text until the tab was focused.

If neither library ever loads, math degrades to readable source text and diagrams
to their fence contents, rather than to nothing.

* fix(markdown): unescape &amp; last so math entities survive intact

The math pass unescaped &amp; before &lt; and &gt;. mdToHtml escapes the source
first, so a literal "&lt;" typed inside a formula arrives here as "&amp;lt;",
turns back into "&lt;" on the ampersand pass, and is then eaten by the very next
one. Typing $a &lt; b$ rendered as "a < b" instead of the literal text.

The code-block pass in the same function already unescapes &amp; last; only the
math paths were the outlier, in all four of the copies this branch consolidated
into pushMath(). Reordering to match makes them consistent and clears the
js/double-escaping alert CodeQL raised on this PR.

Math containing a genuinely typed "<" is unaffected, which is why this went
unnoticed for so long. Covered by a regression test asserting both cases.

* fix(markdown): decode entity-spelled math in one pass

mdToHtml escapes the source before the math pass, so a typed "<" reaches
the delimiters as "&lt;" and a typed "&lt;" reaches them as "&amp;lt;".
KaTeX has no entity syntax and reads the leftover "&" as an alignment
marker, so "$a &lt; b$" rendered as a red .katex-error instead of a
formula, on both the inline and the deferred path.

Chained replaces cannot fix it in either order: unescaping "&amp;" first
lets the next pass eat the "&lt;" it just wrote, and unescaping it last
leaves the entity spelling for KaTeX to choke on. One alternation,
longest form first, decodes every spelling and never rescans its own
output.

The tests now drive the vendored KaTeX build rather than a renderer that
echoes its input, which is why the old assertion looked correct.

* fix(document): typeset deferred math before the PDF export

exportAsPdf() renders the document into a detached container and hands
it straight to html2pdf. On a page where KaTeX has not loaded yet,
mdToHtml() returns pending placeholders and schedules a flush scoped to
document, which never reaches a node that was never attached, so the
PDF printed raw formula source.

Render the container's own math first. renderMath() returns immediately
without fetching anything when there is nothing pending, so a document
with no formulas still exports without pulling KaTeX.
2026-08-16 22:43:12 +01:00
LéoandAlexandre Teixeira 04b8829fb2 perf(frontend): share one cached fetch for /api/auth/settings and /api/tools (#5997)
* perf(frontend): share one cached fetch for settings and tools

/api/auth/settings was fetched independently by eight modules and /api/tools by
three on a single load — 4 and 3 requests measured — and any two of those
callers could observe a different snapshot of the same object. chatRenderer.js
is imported under three different ?v= query strings, so it is three separate
module instances each issuing its own /api/tools request.

appConfig.js holds one promise per endpoint, so concurrent and later callers
share it. Every writer invalidates: the settings panel routes its 16 saves
through a single helper, and the admin tools save drops both snapshots because
that route persists disabled_tools into the same settings store. A rejected
fetch clears its slot rather than being memoised, so one blip at boot cannot
leave keybinds, TTS and the search provider on defaults for the session.

The settings panel keeps reading directly: it is the writer and edits what it
reads, so it must see authoritative state.

Cold load, Resource Timing: /api/auth/settings 4 -> 1, /api/tools 3 -> 1, and
0 settings requests on the first load after a login, because the cache now
consumes the sessionStorage prefetch that login.html writes.

Fixes #5996

* fix(admin): refetch tool state when the Agent Tools panel opens

The shared cache made Admin > Tools render the boot snapshot on every
reopen. Its save posts the whole disabled list rebuilt from the checkboxes,
so a tool disabled out of band (the manage_settings tool, another tab) came
back enabled on the next unrelated toggle. Reproduced against the running
app: with api_call disabled by a separate client, toggling app_api off
posted ['app_api'] and silently re-enabled api_call.

The panel now drops the shared entry before reading it, which restores what
dev does today and keeps the startup read that chatRenderer.js shares. Cold
load is still 1 request each for /api/auth/settings and /api/tools, and the
panel costs the same 2 requests per open as dev.

* fix(static): preserve concurrent tool setting changes

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-08-16 21:03:05 +01:00
Léo 895bf896e3 refactor(static): load the image editor on first use (#6074)
galleryEditor.js and its js/editor/ graph are 54 modules / 576 KB, and
gallery.js imported them statically. Every page load paid for the whole
image editor even though most sessions never open the Edit tab: 54 of the
173 JS files on a cold load, and 576 KB of the decoded JS, were for a
panel that was never displayed.

Add a small panel-loader registry (static/js/panels.js) that imports a
panel's module on first use and memoises the promise, so a double-click
cannot start two loads and a failed load can still be retried. Convert
the image editor to it, and route the two existing dynamic imports in
chat.js and chatRenderer.js through the same entry so all three call
sites share one module instance instead of two.

closeEditor() and isEditorOpen() stay synchronous: if the module was
never loaded there is no edit session to close and none can be open.

The service worker keeps precaching the editor, in a separate
PANEL_PRECACHE list, so the panel stays available offline even though
index.html no longer loads it. The two lists now serve different
purposes and the header comment says so.
2026-08-16 17:54:24 +01:00
Léo cc42f38a89 fix(ci): match the screenshot checkbox by wording, not emphasis (#6073)
The PR-description check folded the template's asterisks into the pattern,
so a ticked box written without them read as unchecked while rendering
identically on the PR page. `ready for review` was silently withheld and
the bot reported missing visual evidence even with screenshots attached,
with no way to tell from the rendered PR what was wrong.

The two attestations directly above it already anchor on the wording
alone. This one now does the same, accepting `**bold**`, `*italic*`,
`__underscores__` and plain text.

Fixes #6071
2026-08-16 16:26:08 +01:00
Joeseph Grey d5514da3ab fix(tasks): scope action_tidy_research broken-file sweep to admins (#6069)
action_tidy_research took an `owner` argument and never used it. Any user's
scheduled tidy task swept data/deep_research globally, unlinking every empty
or unparseable file regardless of who owned it.

A broken file has no readable owner stamp, so it cannot be matched against
`owner` the way _find_owned_research_path does, which is why the HTTP path and
manage_research already treat parse failure as not-owned. Clearing one is a
privileged act rather than an ownership one, so gate it on the canonical
owner_is_admin_or_single_user helper: admins and the single-user operator keep
the janitor, a regular user does not, and neither does the pre-setup window
before an admin exists.

Returns before the directory glob rather than filtering inside the loop, so a
denied run reports why instead of reporting "none broken" over files it never
inspected. That reason string surfaces in Activity as a skipped row.
2026-08-16 13:19:56 +01:00
Mubelotix 45fc3938e0 Fixes Star History section in README
Fixes part of #5563
2026-08-06 19:18:56 +02:00
99 changed files with 8302 additions and 1796 deletions
+6 -2
View File
@@ -88,8 +88,12 @@ SEARXNG_INSTANCE=http://localhost:8080
# Keep false for Docker, LAN, reverse proxy, and any shared deployment.
# LOCALHOST_BYPASS=false
# Mark session cookies Secure. Set true when Odysseus is served through HTTPS
# by a trusted reverse proxy or private access gateway.
# Mark session cookies Secure. Left unset, this follows the request scheme:
# an HTTPS login gets a Secure cookie, a plain-HTTP one does not. Set true to
# force it on, or false to force it off while you still serve plain HTTP.
# Upgrading: this used to default to false. Drop a leftover SECURE_COOKIES=false
# from your .env unless you still need that escape hatch — it keeps HTTPS logins
# on a non-Secure cookie.
# SECURE_COOKIES=true
# Optional: pre-seed the first admin password during setup.
+7
View File
@@ -15,6 +15,13 @@ docker/entrypoint.sh text eol=lf
*.cmd text eol=crlf
*.bat text eol=crlf
# Vendored third-party bundles in static/lib/ are published minified artifacts
# and must stay byte-identical to what npm ships — stripping trailing whitespace
# to satisfy `git diff --check` would desync them from the upstream release. Turn
# the whitespace check off for that tree instead, and keep the bundles out of
# GitHub's language statistics.
static/lib/** -whitespace linguist-vendored
# Binary assets — never normalize.
*.png binary
*.jpg binary
+12
View File
@@ -26,6 +26,18 @@ body:
- label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing.
required: true
- type: input
id: revision
attributes:
label: Odysseus Revision
description: |
From the repository root (on the host when using Docker), run
`git show -s --abbrev=12 --format='%h (%cs)' HEAD`
and paste the output exactly.
placeholder: "1fef4929cf1d (2026-08-11)"
validations:
required: true
- type: dropdown
id: install-method
attributes:
@@ -41,6 +41,14 @@ module.exports = async ({ github, context, core }) => {
break;
case 'bug': {
const revisionText = section('Odysseus Revision');
if (!/^[0-9a-f]{12} \(\d{4}-\d{2}-\d{2}\)$/i.test(revisionText)) {
failures.push(
'**Odysseus Revision** — paste the 12-character commit SHA and date, ' +
'for example `1fef4929cf1d (2026-08-11)`',
);
}
if (!section('Install Method')) {
failures.push('**Install Method** — select how you installed Odysseus');
}
+5 -1
View File
@@ -96,7 +96,11 @@ module.exports = async ({ github, context, core }) => {
const appRan = /- \[x\]\s+I actually ran the app\b/i.test(body);
const appNotRun = /- \[x\]\s+I did not run the app\/runtime validation\b/i.test(body);
const screenshotChecked = /- \[x\]\s+\*\*Screenshot or short clip\*\*/i.test(body);
// Anchor on the wording, not the template's emphasis: a ticked box the author
// retyped without the surrounding ** renders identically on the PR page, so
// treating it as unchecked is invisible from their side. Matches the two
// attestations above, which already ignore formatting.
const screenshotChecked = /- \[x\]\s+[*_]{0,2}Screenshot or short clip[*_]{0,2}/i.test(body);
const screenshotSection = section('Screenshots / clips');
const hasVisualEvidence = /!\[[^\]]*\]\([^)]+\)|<(?:img|video|source)\b[^>]*(?:src|href)=|https?:\/\/[^\s)]+/i.test(screenshotSection);
const evidenceGaps = [];
+10 -2
View File
@@ -65,6 +65,16 @@ Vendored in `static/lib/` and served directly:
| [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT |
| [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT |
| [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT |
| [KaTeX](https://github.com/KaTeX/KaTeX) v0.16.22 (`katex/katex.min.{js,css}` + `katex/fonts/*.woff2`) | Math typesetting | MIT ([`licenses/KaTeX-MIT-LICENSE.txt`](licenses/KaTeX-MIT-LICENSE.txt)) |
| [Mermaid](https://github.com/mermaid-js/mermaid) v11.16.1 (`mermaid.min.js`) | Diagrams from text | MIT ([`licenses/Mermaid-MIT-LICENSE.txt`](licenses/Mermaid-MIT-LICENSE.txt)) |
KaTeX and Mermaid are loaded on first use by `static/js/markdown.js` rather than
from `index.html`, so a session that renders no math and no diagram never fetches
either. Only the `.woff2` KaTeX fonts are shipped, matching `static/fonts/`; the
`.woff` and `.ttf` variants its stylesheet also lists are never requested by a
browser that supports `woff2`. The bundles are the published npm artifacts,
unmodified — `.gitattributes` turns the whitespace check off for `static/lib/`
so they can stay byte-identical to upstream.
## Front-end libraries loaded at runtime (CDN)
@@ -72,8 +82,6 @@ Referenced from `cdn.jsdelivr.net` / `cdnjs.cloudflare.com` at runtime — not v
| Library | Purpose | License |
|---|---|---|
| [KaTeX](https://github.com/KaTeX/KaTeX) 0.16.22 | Math typesetting | MIT |
| [Mermaid](https://github.com/mermaid-js/mermaid) 11 | Diagrams from text | MIT |
| [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 |
| [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT |
+4 -4
View File
@@ -68,11 +68,11 @@ Deployment details are in the [setup guide](docs/setup.md#security-notes).
## Star History
<a href="https://www.star-history.com/?repos=odysseus-dev%2Fodysseus&type=date&legend=top-left">
<a href="https://star-history.dera.page/#odysseus-dev/odysseus&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
</picture>
</a>
+1 -1
View File
@@ -10,7 +10,7 @@ Security fixes are handled on the default branch until formal releases are cut.
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development.
- Set `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Set `SECURE_COOKIES=true` to force it on (for a proxy Odysseus cannot see the scheme of), or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS.
- Use HTTPS when exposing the app beyond localhost.
- Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN.
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only.
+1 -1
View File
@@ -784,7 +784,7 @@ from src.task_scheduler import TaskScheduler
task_scheduler = TaskScheduler(session_manager)
from src.event_bus import set_task_scheduler
set_task_scheduler(task_scheduler)
from routes.task_routes import setup_task_routes
from routes.task.task_routes import setup_task_routes
app.include_router(setup_task_routes(task_scheduler))
from routes.assistant_routes import setup_assistant_routes
+17 -1
View File
@@ -14,6 +14,8 @@ import logging
from datetime import datetime, timezone, timedelta
from typing import Dict, Optional
from sqlalchemy import func
from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive
from .models import Session, ChatMessage
from src.attachment_refs import persistable_message_content
@@ -92,14 +94,28 @@ class SessionManager:
try:
db_sessions = db.query(DbSession).filter(
DbSession.archived == False,
DbSession.message_count > 0,
DbSession.messages.any(),
).order_by(DbSession.last_accessed.desc()).limit(100).all()
# message_count is derived metadata and can drift after interrupted
# or legacy writes. Count only the bounded discovery set so startup
# remains metadata-only while lazy hydration sees an authoritative
# positive count for every discovered non-empty session.
message_counts = {}
if db_sessions:
message_counts = dict(
db.query(DbChatMessage.session_id, func.count(DbChatMessage.id))
.filter(DbChatMessage.session_id.in_([row.id for row in db_sessions]))
.group_by(DbChatMessage.session_id)
.all()
)
loaded_count = 0
for db_session in db_sessions:
try:
session = self._db_to_session_meta(db_session)
if session is not None:
session.message_count = message_counts[db_session.id]
self.sessions[db_session.id] = session
loaded_count += 1
except Exception as e:
+1 -1
View File
@@ -50,7 +50,7 @@ services:
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-false}
- SECURE_COOKIES=${SECURE_COOKIES:-}
- EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
+1 -1
View File
@@ -49,7 +49,7 @@ services:
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-false}
- SECURE_COOKIES=${SECURE_COOKIES:-}
- EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
+1 -1
View File
@@ -38,7 +38,7 @@ services:
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-false}
- SECURE_COOKIES=${SECURE_COOKIES:-}
- EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
+30 -8
View File
@@ -441,10 +441,19 @@ A grab-bag of small gotchas that otherwise turn into long debugging sessions.
| Package | Feature unlocked |
|---------|-----------------|
| `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
| `kokoro`, `soundfile` | Local Kokoro-82M text-to-speech on a CUDA GPU. The pinned Kokoro release supports Odysseus installs on Python 3.11-3.12; these packages are intentionally skipped on Python 3.13+ (including the Python 3.14 container image). |
| `ddgs` | DuckDuckGo as a search provider option. |
| `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
| `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
Install the optional set only when you need these features:
```bash
pip install -r requirements-optional.txt
```
The default Docker image currently uses Python 3.14, while Kokoro 0.9.4 declares Python `>=3.10,<3.13`. Odysseus itself continues to support Python 3.11+, but this pinned optional local-TTS feature requires a native Python 3.11 or 3.12 environment. Kokoro declares `torch`, but the local provider only activates when that torch build has CUDA and a GPU is visible; install the CUDA build appropriate for your host. Browser and configured endpoint TTS remain available on Python 3.13+ and in the container image.
### Faster, reproducible installs with uv (optional)
[uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
@@ -475,7 +484,7 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development.
- Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Use `SECURE_COOKIES=true` to force it on for a proxy whose scheme Odysseus cannot see, or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS.
- Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
- Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
- Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
@@ -486,6 +495,14 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
- Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
> **Upgrading an existing install:** `SECURE_COOKIES` used to default to
> `false`, so an install set up before scheme derivation may still carry
> `SECURE_COOKIES=false` in its own `.env`. That explicit value stays
> authoritative, so HTTPS logins keep getting a non-`Secure` session cookie.
> Pulling this change updates the tracked Compose files, but nothing rewrites
> your `.env` — drop the line from it unless you deliberately serve plain HTTP
> alongside HTTPS and want the escape hatch.
### Private or proxied deployments
Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
@@ -494,7 +511,7 @@ Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and th
3. Put the authenticated Odysseus web/API entrypoint behind that layer.
4. Keep raw service and model ports internal-only.
Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true` and `LOCALHOST_BYPASS=false`. Any proxy that forwards `X-Forwarded-Proto: https` gets `Secure` session cookies without configuration, so `SECURE_COOKIES` only needs setting when you want to override that — force it on for a proxy that forwards no scheme at all, or off while you still serve plain HTTP.
`ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
#### Faster over the network: HTTP/2
@@ -582,9 +599,12 @@ Odysseus's own service is unchanged; the proxy runs alongside it. Under Docker,
run the proxy as another container, or on the host pointing at the published
port.
**4. Point Odysseus at the new origin** in `.env`, then restart it:
**4. Point Odysseus at the new origin** in `.env`, then restart it.
A proxy that exposes the HTTPS request scheme to Odysseus needs no `SECURE_COOKIES` setting. Only force it on when the proxy cannot expose that scheme:
```bash
# only if the proxy cannot expose the external HTTPS scheme to Odysseus:
SECURE_COOKIES=true
# only if you use remote MCP servers with OAuth:
OAUTH_REDIRECT_BASE_URL=https://odysseus.example.com
@@ -619,10 +639,12 @@ enable it by right-clicking the column headers.
Three things bite when moving an existing install behind TLS:
- Set `SECURE_COOKIES=true` **at the same time** you stop serving plain HTTP,
not before. The flag is applied to every login regardless of the scheme the
request arrived on, so while an HTTP entrypoint is still reachable the
browser will reject the `Secure` cookie there and login will appear to loop.
- Leave `SECURE_COOKIES` unset when Odysseus can see the external HTTPS scheme;
the cookie then follows the request automatically. If your proxy cannot expose
that scheme, set `SECURE_COOKIES=true` **at the same time** you stop serving
plain HTTP, not before. An explicit `true` applies to every login, so while an
HTTP entrypoint is still reachable the browser will reject the `Secure` cookie
there and login will appear to loop.
- `OAUTH_REDIRECT_BASE_URL` defaults to `http://localhost:7000`. Unlike the
Gmail redirect URI it cannot be derived from a request — it is registered
with each MCP authorization server up front — so set it to the external
@@ -675,7 +697,7 @@ Key settings:
| `AUTH_ENABLED` | `true` | Enable/disable login |
| `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
| `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
| `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
| `SECURE_COOKIES` | derived from the request scheme | Marks session cookies `Secure` on HTTPS requests. Set true to force it on, false to force it off. |
| `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
| `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
| `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
+21
View File
@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2013-2020 Khan Academy and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+21
View File
@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2014 - 2022 Knut Sveidqvist
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+10
View File
@@ -12,6 +12,16 @@
# GPU-accelerated transcription — it's auto-detected, CPU is used otherwise.
faster-whisper
# Local text-to-speech via Kokoro-82M for the "local" TTS provider.
# Kokoro 0.9.4 declares Python >=3.10,<3.13; Odysseus itself requires 3.11+,
# so pip installs these extras on 3.11-3.12 and deliberately skips them on
# Python 3.13+ (including the Python 3.14 container image). Kokoro declares
# torch; the local provider still
# requires a CUDA-enabled torch build and GPU at runtime. SoundFile is separate
# in Kokoro's official install instructions and is not a transitive dependency.
kokoro==0.9.4; python_version >= "3.11" and python_version < "3.13"
soundfile; python_version >= "3.11" and python_version < "3.13"
# DuckDuckGo as a search provider option.
# Install if you want DDG in the search-provider dropdown.
# Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE.
+28 -1
View File
@@ -86,6 +86,33 @@ class SetOpenRegistrationRequest(BaseModel):
SESSION_COOKIE = "odysseus_session"
def _secure_cookie(request: Request) -> bool:
"""Decide the ``Secure`` attribute of the session cookie.
``SECURE_COOKIES`` stays authoritative when it holds an explicit value:
``true`` always marks the cookie Secure (the documented knob for a TLS
proxy), ``false`` never does, which is the escape hatch for an install
that still answers on plain HTTP alongside HTTPS. Anything else —
unset, or the present-but-empty value docker-compose injects for a
variable the host has not defined — derives it from the request, so an
HTTPS login gets a Secure cookie without any configuration.
Either the connection scheme or ``X-Forwarded-Proto`` saying https is
enough, which is the same test ``core/middleware.py`` applies before it
sends HSTS. Uvicorn's proxy-headers middleware already folds that header
into the scheme for the proxies it trusts, so reading it here only adds
the case of a terminator that is not on a trusted address; the cost is
that a client talking to the app directly can set the header and lock
its own session out over plain HTTP.
"""
configured = os.getenv("SECURE_COOKIES", "").strip().lower()
if configured in ("true", "false"):
return configured == "true"
# A chained proxy sends a list — the client-facing hop comes first.
forwarded_proto = request.headers.get("x-forwarded-proto", "").split(",")[0]
return request.url.scheme == "https" or forwarded_proto.strip().lower() == "https"
def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
router = APIRouter(prefix="/api/auth", tags=["auth"])
@@ -159,7 +186,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
value=token,
httponly=True,
samesite="lax",
secure=os.getenv("SECURE_COOKIES", "false").lower() == "true",
secure=_secure_cookie(request),
path="/",
)
if body.remember:
+35
View File
@@ -1204,6 +1204,41 @@ def _safe_env_prefix(ep: str | None) -> str | None:
return f'[ -f "{path}" ] && source "{path}" || true'
def _local_windows_bash_env_prefix(ep: str | None) -> str | None:
"""Convert a frontend PowerShell venv prefix for the local Git Bash runner."""
if not ep:
return ep
prefix = ep.strip()
if not prefix.startswith("&"):
return ep
raw_path = prefix[1:].lstrip()
if not raw_path:
return ep
if raw_path.startswith("'"):
if len(raw_path) < 2 or not raw_path.endswith("'"):
return ep
quoted_path = raw_path[1:-1]
if "'" in quoted_path.replace("''", ""):
return ep
path = quoted_path.replace("''", "'")
else:
path = raw_path.rstrip()
if "'" in path or '"' in path:
return ep
if any(c in path for c in "\r\n;&|`$<>"):
return ep
if not path.replace("\\", "/").casefold().endswith("/scripts/activate.ps1"):
return ep
bash_path = _git_bash_path(path)
if "\\" in bash_path:
return ep
bash_path = bash_path[: -len("Activate.ps1")] + "activate"
return "source " + shlex.quote(bash_path)
def _ssh_ps(host, script_path, port=None):
"""Build SSH command to run a PowerShell script on a Windows remote."""
pf = f"-p {port} " if port and port != "22" else ""
+3 -3
View File
@@ -50,7 +50,7 @@ from routes.cookbook_helpers import (
_SESSION_ID_RE, _validate_repo_id, _validate_serve_model_id, _validate_include, _validate_token,
_validate_local_dir, _validate_gpus, _shell_path,
_ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase, OLLAMA_MISSING_HINT,
_safe_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
_safe_env_prefix, _local_windows_bash_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
_append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script,
load_stored_hf_token,
_append_vllm_linux_preflight_lines, _ollama_bind_from_cmd, _pip_install_fallback_chain,
@@ -1336,7 +1336,7 @@ def setup_cookbook_routes() -> APIRouter:
# Local: run hf download in the background (tmux on POSIX, a detached
# process + logfile on Windows where tmux doesn't exist).
if req.env_prefix:
lines.append(_safe_env_prefix(req.env_prefix))
lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix))
else:
lines.append("deactivate 2>/dev/null; hash -r")
# Show whether the HF token reached this run (masked) — tells a gated
@@ -2166,7 +2166,7 @@ def setup_cookbook_routes() -> APIRouter:
if req.gpus:
runner_lines.append(f"export CUDA_VISIBLE_DEVICES='{req.gpus}'")
if req.env_prefix:
runner_lines.append(_safe_env_prefix(req.env_prefix))
runner_lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix))
else:
runner_lines.append("deactivate 2>/dev/null; hash -r")
_append_venv_nvidia_library_path_lines(runner_lines, cmd=req.cmd)
+5 -7
View File
@@ -1351,14 +1351,14 @@ def _legacy_visible_api_models(ep) -> List[str]:
def _picker_models_for_endpoint(ep, base_url: str, kind: str):
"""Return model IDs that should appear in the picker for an endpoint.
API providers expose remote inventory from /v1/models. Treat that cache as
inventory, not approval: only manually pinned API models should appear in
the picker. Local/self-hosted endpoints keep the older hide-list behavior.
API providers expose remote inventory from /v1/models. Default to that
visible inventory until an explicit pinned-model allow-list is saved.
Local/self-hosted endpoints keep the older hide-list behavior.
"""
pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if _picker_requires_pinning(base_url, kind):
if not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep) if _hidden_model_ids(ep) else []
pinned = _legacy_visible_api_models(ep)
return pinned, pinned
return _visible_models(
_cached_model_ids(ep),
@@ -2342,9 +2342,7 @@ def setup_model_routes(model_discovery):
else:
response.headers["X-Model-Refresh-Status"] = "failed"
response.headers["X-Model-Refresh-Warning"] = "Model refresh failed or returned no models; kept cached models."
pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if picker_requires_pinning and not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep)
_, pinned = _picker_models_for_endpoint(ep, base, kind)
pinned_set = set(pinned)
return [
{
+5
View File
@@ -0,0 +1,5 @@
"""Task route domain package (slice 2p, #4082/#4071).
Contains task_routes.py, migrated from the flat routes/ directory.
Backward-compat shim at routes/task_routes.py re-exports from here.
"""
File diff suppressed because it is too large Load Diff
+14 -1177
View File
File diff suppressed because it is too large Load Diff
+11 -3
View File
@@ -2,7 +2,7 @@
"""odysseus-webhook — shell wrapper for scheduled-task webhook tokens.
Tasks in the scheduled-task system can carry a `webhook_token`. Any
HTTP POST to `/api/webhook/<token>` fires the task. This CLI lists,
HTTP POST to `/api/tasks/<task-id>/webhook/<token>` fires the task. This CLI lists,
rotates, and revokes those tokens.
odysseus-webhook list # tasks that have a token
@@ -21,6 +21,7 @@ quiet_logs()
import argparse, json, logging, os, secrets, sys
from pathlib import Path
from urllib.parse import quote
try:
from core.database import SessionLocal, ScheduledTask
@@ -53,6 +54,14 @@ def _summary(t: "ScheduledTask", reveal: bool = False) -> dict:
}
def _task_webhook_url(base: str, task_id: str, token: str) -> str:
"""Build the live task-route URL without leaking ids into path syntax."""
root = (base or "http://localhost:7000").rstrip("/")
task_part = quote(str(task_id), safe="")
token_part = quote(str(token), safe="")
return f"{root}/api/tasks/{task_part}/webhook/{token_part}"
def cmd_list(args):
db = SessionLocal()
try:
@@ -109,8 +118,7 @@ def cmd_url(args):
fail(f"no task with id {args.id!r}")
if not t.webhook_token:
fail(f"task {args.id!r} has no webhook token (rotate one first)")
base = (args.base or "http://localhost:7000").rstrip("/")
url = f"{base}/api/webhook/{t.webhook_token}"
url = _task_webhook_url(args.base, t.id, t.webhook_token)
emit({
"task_id": t.id,
"name": t.name,
+41 -11
View File
@@ -50,16 +50,46 @@ class DocsService:
List of DocChunk objects
"""
results = self.rag.search(query, k=top_k)
return [
DocChunk(
text=r.get("text", r.get("content", "")),
source=r.get("source", r.get("metadata", {}).get("source", "unknown")),
score=r.get("score", 0.0),
metadata=r.get("metadata"),
chunks = []
for result in results:
if not isinstance(result, dict):
continue
metadata = result.get("metadata")
if not isinstance(metadata, dict):
metadata = {}
text = result.get("document")
if text is None:
text = result.get("text")
if text is None:
text = result.get("content")
if text is None:
text = ""
source = result.get("source")
if source is None:
source = metadata.get("source")
if source is None:
source = "unknown"
score = result.get("similarity")
if score is None:
score = result.get("score")
if score is None:
score = 0.0
chunks.append(
DocChunk(
text=text,
source=source,
score=score,
metadata=metadata,
)
)
for r in results
if isinstance(r, dict)
]
return chunks
async def index(self, directory: str) -> IndexResult:
"""
@@ -73,8 +103,8 @@ class DocsService:
"""
result = self.rag.index_personal_documents(directory)
return IndexResult(
indexed=result.get("indexed", 0),
failed=result.get("failed", 0),
indexed=result.get("indexed_count", result.get("indexed", 0)),
failed=result.get("failed_count", result.get("failed", 0)),
errors=result.get("errors", []),
)
+31 -331
View File
@@ -2,22 +2,18 @@
import copy
import io
import ipaddress
import json
import os
import re
import logging
import socket
import ssl
from datetime import datetime, timedelta
from typing import Iterable, List, cast
from urllib.parse import urljoin, urlparse
from typing import List
import httpx
import httpcore
from bs4 import BeautifulSoup
from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_USER_AGENT
from src import outbound_fetch as _outbound_fetch
from .analytics import RateLimitError, error_logger
from .cache import (
@@ -29,336 +25,40 @@ from .cache import (
logger = logging.getLogger(__name__)
_PRIVATE_NETWORKS = (
ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
)
def _is_private_address(addr):
return _outbound_fetch._is_private_address(addr)
def _is_private_address(addr: ipaddress._BaseAddress) -> bool:
if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
addr = addr.ipv4_mapped
return (
addr.is_private
or addr.is_loopback
or addr.is_link_local
or addr.is_reserved
or addr.is_multicast
or addr.is_unspecified
or any(addr in net for net in _PRIVATE_NETWORKS)
def _resolve_hostname_ips(hostname):
return _outbound_fetch._resolve_hostname_ips(hostname)
def _public_http_url(url):
return _outbound_fetch._public_http_url(url, resolver=_resolve_hostname_ips)
def _resolve_public_ips(url):
return _outbound_fetch._resolve_public_ips(url, resolver=_resolve_hostname_ips)
_PinnedBackend = _outbound_fetch._PinnedBackend
_PinnedTransport = _outbound_fetch._PinnedTransport
BodyTooLargeError = _outbound_fetch.BodyTooLargeError
_CappedFetch = _outbound_fetch._CappedFetch
def _get_public_url(url, headers, timeout, max_redirects=5, max_bytes=None):
return _outbound_fetch._get_public_url(
url,
headers=headers,
timeout=timeout,
max_redirects=max_redirects,
max_bytes=max_bytes,
resolve_public_ips=_resolve_public_ips,
transport_factory=_PinnedTransport,
)
def _resolve_hostname_ips(hostname: str) -> list[ipaddress._BaseAddress]:
try:
infos = socket.getaddrinfo(hostname, None)
except Exception:
return []
out = []
for info in infos:
try:
out.append(ipaddress.ip_address(info[4][0]))
except Exception:
continue
return out
def _public_http_url(url: str) -> bool:
try:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
return False
host = (parsed.hostname or "").strip()
if not host:
return False
lower = host.lower()
if lower in ("localhost", "metadata", "metadata.google.internal"):
return False
if lower.endswith((".local", ".localhost", ".internal", ".lan", ".intranet")):
return False
try:
return not _is_private_address(ipaddress.ip_address(host))
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
return bool(addrs) and not any(_is_private_address(a) for a in addrs)
except Exception:
return False
def _resolve_public_ips(url: str) -> list[ipaddress._BaseAddress]:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise httpx.RequestError(f"Blocked non-public URL: {url}")
host = (parsed.hostname or "").strip().lower()
if host in ("localhost", "metadata", "metadata.google.internal"):
raise httpx.RequestError(f"Blocked non-public hostname: {host}")
try:
ip = ipaddress.ip_address(host)
if _is_private_address(ip):
raise httpx.RequestError(f"Blocked non-public IP literal: {host}")
return [ip]
except httpx.RequestError:
raise
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
if not addrs or any(_is_private_address(a) for a in addrs):
raise httpx.RequestError(f"Blocked non-public URL: {url}")
return addrs
class _PinnedBackend(httpcore.NetworkBackend):
"""Network backend that connects to a pre-resolved IP.
httpcore derives the TLS SNI and the ``Host`` header from the URL's
origin, not from the host argument passed to ``connect_tcp``. So
routing the TCP connect to a resolved IP while leaving the URL
untouched keeps SNI / vhost behaviour correct and closes the
DNS-rebinding TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress):
self._ip = str(ip)
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
return self._real.connect_tcp(
self._ip, port, timeout, local_address, socket_options
)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
# Map httpcore exception classes to their httpx equivalents. Built
# once at import time from the public exception classes; avoids any
# import of httpx's private transport machinery. httpcore's
# ``ConnectionNotAvailable`` is a pool-internal signal (the pool will
# close and retry on its own) — we never expect to see it surface to
# a transport caller, so it has no httpx counterpart here.
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Transport that pins every TCP connect to a pre-resolved IP.
Uses only the public ``httpcore`` and ``httpx`` APIs — no
subclassing of ``httpx.HTTPTransport``, no reads of private
``httpcore.ConnectionPool`` attributes, no imports from
``httpx private transport internals``. The URL is passed through unchanged so SNI
/ vhost work as if httpx had been given the hostname directly;
only the TCP destination is pinned, closing the DNS-rebinding
TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress, *, http2: bool = False):
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
http1=True,
http2=http2,
network_backend=_PinnedBackend(ip),
)
def __enter__(self):
self._pool.__enter__()
return self
def __exit__(self, exc_type=None, exc_value=None, traceback=None) -> None:
self._pool.__exit__(exc_type, exc_value, traceback)
def handle_request(self, request: httpx.Request) -> httpx.Response:
httpcore_req = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
try:
httpcore_resp = self._pool.handle_request(httpcore_req)
# Eager materialisation matches the original
# ``response.text`` usage in fetch_webpage_content. The
# sync pool's stream is a plain Iterable[bytes] despite
# the httpcore type hint unioning the async variant.
content = b"".join(cast(Iterable[bytes], httpcore_resp.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
return httpx.Response(
status_code=httpcore_resp.status,
headers=httpcore_resp.headers,
content=content,
extensions=httpcore_resp.extensions,
)
def close(self) -> None:
self._pool.close()
class BodyTooLargeError(Exception):
"""The server declared a body larger than the hard fetch ceiling."""
def __init__(self, url: str, declared_bytes: int):
self.url = url
self.declared_bytes = declared_bytes
super().__init__(
f"response body is {declared_bytes:,} bytes, over the "
f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
)
class _CappedFetch:
"""Result of a size-capped streaming GET.
Carries just what fetch_webpage_content needs from an httpx.Response,
plus the cap bookkeeping: the (possibly truncated) body, whether the
cap cut it short, and the size the server declared via Content-Length
(wire bytes; None when absent).
"""
__slots__ = ("status_code", "headers", "content", "truncated",
"declared_bytes", "encoding", "url")
def __init__(self, status_code, headers, content, truncated,
declared_bytes, encoding, url):
self.status_code = status_code
self.headers = headers
self.content = content
self.truncated = truncated
self.declared_bytes = declared_bytes
self.encoding = encoding
self.url = url
@property
def text(self) -> str:
return self.content.decode(self.encoding or "utf-8", errors="replace")
def raise_for_status(self):
if self.status_code >= 400:
request = httpx.Request("GET", self.url)
raise httpx.HTTPStatusError(
f"HTTP {self.status_code} for {self.url}",
request=request,
response=httpx.Response(self.status_code, request=request),
)
def _get_public_url(url: str, headers: dict, timeout: int, max_redirects: int = 5,
max_bytes: int = None) -> "_CappedFetch":
"""Capped streaming GET with SSRF-guarded, DNS-pinned manual redirects.
Each hop is resolved once, validated as public, and then the actual TCP
connection is pinned to that resolved IP. The request URL is left unchanged
so Host and TLS SNI keep the original hostname.
"""
cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
current = url
for _ in range(max_redirects + 1):
ips = _resolve_public_ips(current)
# Force identity transfer-encoding. With gzip/deflate the wire bytes
# and Content-Length can be a small fraction of the decoded body, so a
# tiny compressed response could pass the hard-cap preflight and then
# expand past the ceiling in one decoded chunk before the streamed cap
# below can slice it.
req_headers = dict(headers or {})
req_headers["Accept-Encoding"] = "identity"
with httpx.Client(
headers=req_headers,
timeout=timeout,
follow_redirects=False,
transport=_PinnedTransport(ips[0]),
) as client:
with client.stream("GET", current) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
if not location:
return _CappedFetch(response.status_code, response.headers, b"",
False, None, response.encoding, str(response.url))
current = urljoin(str(response.url), location)
continue
# A server can ignore the identity request and still return a
# compressed body; httpx.iter_bytes would then decode it, and a
# tiny gzip can balloon into one decoded chunk far past the cap.
# Refuse compressed Content-Encoding so the streamed cap stays
# a real memory bound.
enc = (response.headers.get("content-encoding") or "").strip().lower()
if enc and enc != "identity":
raise httpx.RequestError(
f"Refusing compressed response (Content-Encoding: {enc}) after "
"requesting identity: cannot bound decoded body size",
request=httpx.Request("GET", current),
)
declared = None
raw_len = response.headers.get("content-length")
if raw_len and raw_len.isdigit():
declared = int(raw_len)
if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
raise BodyTooLargeError(current, declared)
chunks = []
read = 0
truncated = False
for chunk in response.iter_bytes():
read += len(chunk)
if read > cap:
keep = cap - (read - len(chunk))
if keep > 0:
chunks.append(chunk[:keep])
truncated = True
break
chunks.append(chunk)
return _CappedFetch(response.status_code, response.headers,
b"".join(chunks), truncated, declared,
response.encoding, str(response.url))
raise httpx.RequestError("Too many redirects", request=httpx.Request("GET", current))
# PDF extraction (optional dependency)
try:
from pdfminer.high_level import extract_text as pdf_extract_text
+11 -4
View File
@@ -3081,10 +3081,17 @@ def _append_tool_results(
messages.append(result_message)
else:
tool_output_text = "\n\n".join(tool_results)
msg = {"role": "assistant", "content": round_response}
if round_reasoning:
msg["reasoning_content"] = round_reasoning
messages.append(msg)
# An approved-action replay injects the sealed tool result with no
# assistant prose for that round, which used to append an assistant turn
# whose content was "". Anthropic's Messages API rejects a non-final
# assistant message with empty content (HTTP 400), so the resumed turn
# died before the model saw the result. A turn carrying neither prose nor
# reasoning has nothing to say to any provider, so skip it entirely.
if round_response.strip() or round_reasoning:
msg = {"role": "assistant", "content": round_response}
if round_reasoning:
msg["reasoning_content"] = round_reasoning
messages.append(msg)
# Tool output (shell/python stdout, file reads, fetched pages, email
# bodies, MCP results) is sourced from outside the server. Wrap it as
# untrusted data so prompt-injection inside a tool result is treated as
+15 -1
View File
@@ -810,13 +810,27 @@ async def action_tidy_research(owner: str, **kwargs) -> Tuple[str, bool]:
Research history lives entirely in data/deep_research/<id>.json and is NOT
backed by chat-session rows — so a file must never be deleted just because
no chat session matches its id. Only prune files that fail to load."""
no chat session matches its id. Only prune files that fail to load.
A broken file has no readable owner stamp, so it cannot be matched against
`owner`. Clearing one is privileged: admins and the single-user operator
(AUTH_ENABLED=false) may, a regular user may not, and neither may anyone
during the pre-setup window before an admin exists.
"""
try:
from pathlib import Path
import json as _json
from src.tool_security import owner_is_admin_or_single_user
research_dir = Path(DEEP_RESEARCH_DIR)
if not research_dir.exists():
raise TaskNoop("no research directory")
if not owner_is_admin_or_single_user(owner):
# Return before the glob rather than filtering inside the loop: the
# loop reports "none broken" off an empty `removed`, which reaches
# Activity as a false report to a user whose files it skipped, and a
# regular user need not read every owner's file to learn it may
# delete none of them.
raise TaskNoop("not permitted to remove unattributable research files")
files = list(research_dir.glob("*.json"))
removed = []
for p in files:
+26 -1
View File
@@ -462,7 +462,14 @@ class ChatProcessor:
skip_url_fetch = len(message) > 2000 or len(non_yt_urls) > 3
if not skip_url_fetch:
for url in non_yt_urls:
result = fetch_webpage_content(url)
try:
result = fetch_webpage_content(url)
except Exception:
# The URL and exception can both contain signed-query
# credentials or response-controlled text. Keep the log
# diagnostic stable as well as the model-facing context.
logger.warning("Automatic URL fetch failed while building context")
result = {"success": False, "error": ""}
if result.get('success'):
content = result.get('content', '')[:10000]
preface.append(untrusted_context_message(
@@ -470,6 +477,24 @@ class ChatProcessor:
f"Content from {url}:\n\n{content}",
provenance_origin="external",
))
else:
# A failed automatic URL fetch is context too. Never pass
# exception text or response-controlled diagnostics back to
# the model: reduce the result to a small transport-owned
# status and explicitly state that the page was not read.
error = str(result.get("error") or "")
status = "the page was unavailable"
status_match = re.match(r"^HTTP\s+(\d{3})\b", error)
if status_match:
status = f"the server returned HTTP {status_match.group(1)}"
elif error.startswith("TooLarge:"):
status = "the response exceeded the fetch size limit"
elif error.startswith("Rate limit"):
status = "the request was rate limited"
preface.append(untrusted_context_message(
"web page fetch failure",
f"A linked page was not read: {status}.",
))
# Skills index — progressive disclosure. Only injected when the
# model has the `manage_skills` tool available (agent_mode), and
+11 -6
View File
@@ -290,17 +290,22 @@ def detect_vendor(base_url: Any = "", endpoint_kind: Any = "") -> str:
return kind_map[kind]
parsed = urlparse(compact_str(base_url))
host = (parsed.hostname or "").lower()
host = (parsed.hostname or "").lower().rstrip(".")
port = parsed.port
if host.endswith("openrouter.ai"):
def host_matches(domain: str) -> bool:
domain = domain.lower().rstrip(".")
return host == domain or host.endswith(f".{domain}")
if host_matches("openrouter.ai"):
return VENDOR_OPENROUTER
if host.endswith("openai.com"):
if host_matches("openai.com"):
return VENDOR_OPENAI
if host.endswith("anthropic.com"):
if host_matches("anthropic.com"):
return VENDOR_ANTHROPIC
if host.endswith("googleapis.com"):
if host_matches("googleapis.com"):
return VENDOR_GOOGLE
if host.endswith("ollama.com") or port == 11434:
if host_matches("ollama.com") or port == 11434:
return VENDOR_OLLAMA
if port == 1234:
return VENDOR_LMSTUDIO
+354
View File
@@ -0,0 +1,354 @@
"""SSRF-guarded synchronous HTTP fetching primitives.
This module owns outbound URL classification, one-resolution-per-hop DNS
pinning, redirects, and response-body budgets. It deliberately has no search
or content-extraction dependencies so callers outside search can reuse the
same transport boundary.
"""
from __future__ import annotations
import ipaddress
import socket
import ssl
from typing import Callable, Iterable, cast
from urllib.parse import urljoin, urlparse
import httpcore
import httpx
from src.constants import WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_SOFT_MAX_BYTES
_PRIVATE_NETWORKS = (
ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
)
def _is_private_address(addr: ipaddress._BaseAddress) -> bool:
if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
addr = addr.ipv4_mapped
return (
addr.is_private
or addr.is_loopback
or addr.is_link_local
or addr.is_reserved
or addr.is_multicast
or addr.is_unspecified
or any(addr in net for net in _PRIVATE_NETWORKS)
)
def _resolve_hostname_ips(hostname: str) -> list[ipaddress._BaseAddress]:
try:
infos = socket.getaddrinfo(hostname, None)
except Exception:
return []
out = []
for info in infos:
try:
out.append(ipaddress.ip_address(info[4][0]))
except Exception:
continue
return out
def _public_http_url(
url: str,
*,
resolver: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
) -> bool:
resolver = resolver or _resolve_hostname_ips
try:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
return False
host = (parsed.hostname or "").strip()
if not host:
return False
lower = host.lower()
if lower in ("localhost", "metadata", "metadata.google.internal"):
return False
if lower.endswith((".local", ".localhost", ".internal", ".lan", ".intranet")):
return False
try:
return not _is_private_address(ipaddress.ip_address(host))
except ValueError:
pass
addrs = resolver(host)
return bool(addrs) and not any(_is_private_address(a) for a in addrs)
except Exception:
return False
def _resolve_public_ips(
url: str,
*,
resolver: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
) -> list[ipaddress._BaseAddress]:
resolver = resolver or _resolve_hostname_ips
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise httpx.RequestError(f"Blocked non-public URL: {url}")
host = (parsed.hostname or "").strip().lower()
if host in ("localhost", "metadata", "metadata.google.internal"):
raise httpx.RequestError(f"Blocked non-public hostname: {host}")
try:
ip = ipaddress.ip_address(host)
if _is_private_address(ip):
raise httpx.RequestError(f"Blocked non-public IP literal: {host}")
return [ip]
except httpx.RequestError:
raise
except ValueError:
pass
addrs = resolver(host)
if not addrs or any(_is_private_address(a) for a in addrs):
raise httpx.RequestError(f"Blocked non-public URL: {url}")
return addrs
class _PinnedBackend(httpcore.NetworkBackend):
"""Network backend that connects to a pre-resolved IP."""
def __init__(self, ip: ipaddress._BaseAddress):
self._ip = str(ip)
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
return self._real.connect_tcp(
self._ip, port, timeout, local_address, socket_options
)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Transport that pins every TCP connect to a pre-resolved IP."""
def __init__(self, ip: ipaddress._BaseAddress, *, http2: bool = False):
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
http1=True,
http2=http2,
network_backend=_PinnedBackend(ip),
)
def __enter__(self):
self._pool.__enter__()
return self
def __exit__(self, exc_type=None, exc_value=None, traceback=None) -> None:
self._pool.__exit__(exc_type, exc_value, traceback)
def handle_request(self, request: httpx.Request) -> httpx.Response:
httpcore_req = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
try:
httpcore_resp = self._pool.handle_request(httpcore_req)
content = b"".join(cast(Iterable[bytes], httpcore_resp.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
return httpx.Response(
status_code=httpcore_resp.status,
headers=httpcore_resp.headers,
content=content,
extensions=httpcore_resp.extensions,
)
def close(self) -> None:
self._pool.close()
class BodyTooLargeError(Exception):
"""The server declared a body larger than the hard fetch ceiling."""
def __init__(self, url: str, declared_bytes: int):
self.url = url
self.declared_bytes = declared_bytes
super().__init__(
f"response body is {declared_bytes:,} bytes, over the "
f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
)
class _CappedFetch:
"""Result of a size-capped streaming GET."""
__slots__ = (
"status_code",
"headers",
"content",
"truncated",
"declared_bytes",
"encoding",
"url",
)
def __init__(
self,
status_code,
headers,
content,
truncated,
declared_bytes,
encoding,
url,
):
self.status_code = status_code
self.headers = headers
self.content = content
self.truncated = truncated
self.declared_bytes = declared_bytes
self.encoding = encoding
self.url = url
@property
def text(self) -> str:
return self.content.decode(self.encoding or "utf-8", errors="replace")
def raise_for_status(self):
if self.status_code >= 400:
request = httpx.Request("GET", self.url)
raise httpx.HTTPStatusError(
f"HTTP {self.status_code} for {self.url}",
request=request,
response=httpx.Response(self.status_code, request=request),
)
def _get_public_url(
url: str,
headers: dict,
timeout: int,
max_redirects: int = 5,
max_bytes: int | None = None,
*,
resolve_public_ips: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
transport_factory: Callable[[ipaddress._BaseAddress], httpx.BaseTransport] | None = None,
) -> _CappedFetch:
"""Capped streaming GET with SSRF-guarded, DNS-pinned redirects."""
resolve_public_ips = resolve_public_ips or _resolve_public_ips
transport_factory = transport_factory or _PinnedTransport
cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
current = url
for _ in range(max_redirects + 1):
ips = resolve_public_ips(current)
req_headers = dict(headers or {})
req_headers["Accept-Encoding"] = "identity"
with httpx.Client(
headers=req_headers,
timeout=timeout,
follow_redirects=False,
transport=transport_factory(ips[0]),
) as client:
with client.stream("GET", current) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
if not location:
return _CappedFetch(
response.status_code,
response.headers,
b"",
False,
None,
response.encoding,
str(response.url),
)
current = urljoin(str(response.url), location)
continue
enc = (response.headers.get("content-encoding") or "").strip().lower()
if enc and enc != "identity":
raise httpx.RequestError(
f"Refusing compressed response (Content-Encoding: {enc}) after "
"requesting identity: cannot bound decoded body size",
request=httpx.Request("GET", current),
)
declared = None
raw_len = response.headers.get("content-length")
if raw_len and raw_len.isdigit():
declared = int(raw_len)
if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
raise BodyTooLargeError(current, declared)
chunks = []
read = 0
truncated = False
for chunk in response.iter_bytes():
read += len(chunk)
if read > cap:
keep = cap - (read - len(chunk))
if keep > 0:
chunks.append(chunk[:keep])
truncated = True
break
chunks.append(chunk)
return _CappedFetch(
response.status_code,
response.headers,
b"".join(chunks),
truncated,
declared,
response.encoding,
str(response.url),
)
raise httpx.RequestError(
"Too many redirects", request=httpx.Request("GET", current)
)
+7 -8
View File
@@ -50,6 +50,7 @@ import * as researchPanelModule from './js/research/panel.js?v=20260630researcht
import ttsModule from './js/tts-ai.js';
import spinnerModule from './js/spinner.js';
import { initKeyboardShortcuts } from './js/keyboard-shortcuts.js';
import { getSettings } from './js/appConfig.js';
import { initSidebarLayout, syncRailSide } from './js/sidebar-layout.js?v=20260715startupclean';
import { initSectionCollapse, initSectionDrag } from './js/section-management.js';
@@ -1518,13 +1519,11 @@ function initializeEventListeners() {
})
.catch(() => {});
// Hide Gallery when image generation is disabled in settings
const _prefetchedSettings = sessionStorage.getItem('ody-prefetch-settings');
sessionStorage.removeItem('ody-prefetch-settings');
window._initSettingsReady = (_prefetchedSettings
? Promise.resolve(JSON.parse(_prefetchedSettings))
: fetch(`${API_BASE}/api/auth/settings`, { credentials: 'same-origin' }).then(r => r.json())
).then(settings => {
// Hide Gallery when image generation is disabled in settings.
// getSettings() consumes the login prefetch itself, so every other module
// that asks for settings this load gets the same snapshot without a request.
window._initSettingsReady = getSettings()
.then(settings => {
// NOTE: image_gen_enabled only governs *generating* images in chat — the
// tool is blocked server-side (chat_routes / agent_loop). The Gallery
// holds uploads and past images too, so it stays visible regardless;
@@ -3705,7 +3704,7 @@ function startOdysseusApp() {
modelsModule.init(API_BASE);
ragModule.init(API_BASE);
presetsModule.init(API_BASE);
searchModule.init(API_BASE);
searchModule.init();
chatModule.init(API_BASE);
chatModule.initListeners();
groupModule.init(API_BASE);
+5 -17
View File
@@ -231,23 +231,11 @@
}
}
</style>
<!-- KaTeX CSS is loaded with media="print" so it doesn't block render,
then flipped to "all" via JS after load. Mermaid init runs once the
library finishes loading. Both hooks are wired via addEventListener
below (inline onload= attrs are blocked by CSP script-src-attr). -->
<link id="katex-css" rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.22/dist/katex.min.css" media="print">
<script async src="https://cdn.jsdelivr.net/npm/katex@0.16.22/dist/katex.min.js"></script>
<script id="mermaid-script" async src="https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.min.js"></script>
<script nonce="{{CSP_NONCE}}">
(function(){
var k = document.getElementById('katex-css');
if (k) k.addEventListener('load', function(){ k.media = 'all'; }, { once: true });
var m = document.getElementById('mermaid-script');
if (m) m.addEventListener('load', function(){
if (window.odysseusInitMermaid) window.odysseusInitMermaid();
}, { once: true });
})();
</script>
<!-- KaTeX and Mermaid are vendored in /static/lib and pulled in by
static/js/markdown.js the first time a page actually renders math or a
```mermaid fence. They used to load here from cdn.jsdelivr.net on every
page load, which cost ~985 KB on the wire, broke offline installs, and
announced every session to a third party. -->
<!-- Preload the two faces first paint actually uses: Fira Code 400 and 600,
the app font and the weight the sidebar and header text render at. They
are declared in style.css, so without a hint they are only discovered
+65 -17
View File
@@ -6,6 +6,7 @@ import settingsModule from './settings.js';
import { providerLogo, providerLogoFromUrl } from './providers.js';
import { sortModelObjects } from './modelSort.js';
import { PROVIDER_DEVICE_FLOWS, formatDeviceFlowError, runProviderDeviceFlow } from './providerDeviceFlow.js';
import { getSettings, getTools, invalidateSettings, invalidateTools } from './appConfig.js';
let initialized = false;
let modalEl = null;
@@ -345,8 +346,7 @@ function initSignupToggle() {
function initShareDefaultsToggle() {
const toggle = el('adm-shareDefaultsToggle');
fetch('/api/auth/settings', { credentials: 'same-origin' })
.then(r => r.json())
getSettings()
.then(d => { toggle.checked = !!d.share_defaults_with_users; })
.catch(e => console.warn('Settings fetch failed:', e));
toggle.addEventListener('change', async () => {
@@ -361,6 +361,9 @@ function initShareDefaultsToggle() {
toggle.checked = !!data.share_defaults_with_users;
} catch (e) {
toggle.checked = !toggle.checked;
} finally {
// Drop the shared snapshot: it still says what this toggle used to be.
invalidateSettings();
}
});
}
@@ -1893,8 +1896,16 @@ async function loadBuiltinTools() {
const list = el('adm-builtin-tools-list');
if (!list) return;
try {
const res = await fetch('/api/tools', { credentials: 'same-origin' });
const data = await res.json();
// This panel is an editor, and its save posts the whole disabled list
// rebuilt from the checkboxes below. So it has to render authoritative
// state: a snapshot that went stale out of band (the manage_settings tool,
// another tab) would be re-posted wholesale on the next unrelated toggle
// and would silently undo the newer state. refreshAll() calls this on every
// panel open, so drop the shared entry and refill it. The startup read that
// chatRenderer.js shares is unaffected; this panel just never edits a cache,
// which is the same rule the settings panel follows by reading directly.
invalidateTools();
const data = await getTools();
const tools = data.tools || [];
if (!tools.length) { list.innerHTML = '<div class="admin-empty">No tools found</div>'; return; }
@@ -1968,17 +1979,50 @@ async function loadBuiltinTools() {
});
});
// Helper: save disabled tools + update counters
async function _saveToolState() {
const allChecks = list.querySelectorAll('input[data-tool-id]');
const disabled = [];
allChecks.forEach(c => { if (!c.checked) disabled.push(c.dataset.toolId); });
await fetch('/api/tools', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ disabled }),
credentials: 'same-origin',
});
// Merge only the user's intended changes onto authoritative server state.
// /api/tools replaces the full disabled list, so rebuilding it from this
// panel's DOM can undo a change made by another tab or manage_settings
// after the panel was opened.
async function _saveToolState(changes) {
invalidateTools();
const latest = await getTools();
const state = new Map(
(latest.tools || []).map(t => [t.id, !!t.enabled])
);
for (const change of changes) {
if (state.has(change.id)) {
state.set(change.id, !!change.enabled);
}
}
const disabled = Array.from(state.entries())
.filter(([, enabled]) => !enabled)
.map(([id]) => id);
try {
const res = await fetch('/api/tools', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ disabled }),
credentials: 'same-origin',
});
if (!res.ok) throw new Error(`Failed to update tools (${res.status})`);
// Bring the still-open editor forward to the same merged snapshot so an
// out-of-band change is visible instead of leaving stale checkboxes.
list.querySelectorAll('input[data-tool-id]').forEach(c => {
if (state.has(c.dataset.toolId)) {
c.checked = state.get(c.dataset.toolId);
}
});
list.querySelectorAll('.admin-tool-category').forEach(_updateCatCounter);
} finally {
// This route persists disabled_tools into the settings store
// (routes/model_routes.py), so both snapshots are now stale.
invalidateTools();
invalidateSettings();
}
}
function _updateCatCounter(catEl) {
if (!catEl) return;
@@ -1993,7 +2037,9 @@ async function loadBuiltinTools() {
// Wire individual tool toggles
list.querySelectorAll('input[data-tool-id]').forEach(chk => {
chk.addEventListener('change', async () => {
await _saveToolState();
await _saveToolState([
{ id: chk.dataset.toolId, enabled: chk.checked },
]);
_updateCatCounter(chk.closest('.admin-tool-category'));
});
});
@@ -2004,8 +2050,10 @@ async function loadBuiltinTools() {
const catEl = chk.closest('.admin-tool-category');
if (!catEl) return;
const checked = chk.checked;
const changes = Array.from(catEl.querySelectorAll('input[data-tool-id]'))
.map(c => ({ id: c.dataset.toolId, enabled: checked }));
catEl.querySelectorAll('input[data-tool-id]').forEach(c => { c.checked = checked; });
await _saveToolState();
await _saveToolState(changes);
_updateCatCounter(catEl);
});
});
+86
View File
@@ -0,0 +1,86 @@
// static/js/appConfig.js
//
// One shared, invalidatable cache for the two config endpoints that every
// module wants at startup.
//
// Before this, /api/auth/settings was fetched independently by six modules and
// /api/tools by three, none of them aware of the others — 4 and 3 requests on a
// single cold load. Worse than the requests: each caller could observe a
// different snapshot of the same object, and chatRenderer.js is imported under
// three different ?v= query strings, so it is three separate module instances
// each issuing its own /api/tools fetch. Caching here fixes both, because the
// cache lives in one module every instance imports by the same specifier.
//
// URLs are bare paths on purpose. The callers that used `${API_BASE}/api/...`
// resolved to the identical URL — API_BASE is `window.location.origin`
// (app.js) — so nothing about the request changes for them.
//
// WRITERS MUST INVALIDATE. Anything that POSTs /api/auth/settings calls
// invalidateSettings(); anything that POSTs /api/tools calls invalidateTools()
// *and* invalidateSettings(), because that route persists `disabled_tools`
// into the same settings store (routes/model_routes.py). Miss one and the UI
// serves a stale settings object for the rest of the session, which is worse
// than the duplicate fetches this replaces.
//
// The resolved object is shared by reference, so treat it as read-only: copy
// before mutating (`{ ...await getSettings() }`).
// Written by login.html immediately before it redirects to '/', so the first
// load after a login can skip the request entirely. Consumed once per page
// load, by whichever module asks for settings first.
const PREFETCH_KEY = 'ody-prefetch-settings';
const _URLS = { settings: '/api/auth/settings', tools: '/api/tools' };
const _cache = { settings: null, tools: null };
function _readPrefetchedSettings() {
try {
const raw = sessionStorage.getItem(PREFETCH_KEY);
if (!raw) return null;
sessionStorage.removeItem(PREFETCH_KEY);
return JSON.parse(raw);
} catch (_) {
return null;
}
}
// A rejected promise must not stay in the slot. Plain `??=` memoisation would
// keep it, so one transient blip during boot would leave keybinds, TTS and the
// search provider on their defaults for the whole session with no retry. Clear
// the slot on failure — unless a later invalidate/refetch already replaced it —
// and rethrow, so every caller's existing .catch() still runs exactly as before.
function _get(key) {
if (_cache[key]) return _cache[key];
const pending = fetch(_URLS[key], { credentials: 'same-origin' })
.then(r => r.json())
.catch(err => {
if (_cache[key] === pending) _cache[key] = null;
throw err;
});
_cache[key] = pending;
return pending;
}
/** GET /api/auth/settings, once per page load (or once per invalidation). */
export function getSettings() {
if (!_cache.settings) {
const prefetched = _readPrefetchedSettings();
if (prefetched) _cache.settings = Promise.resolve(prefetched);
}
return _get('settings');
}
/** GET /api/tools, once per page load (or once per invalidation). */
export function getTools() {
return _get('tools');
}
/** Call after any write that can change settings. */
export function invalidateSettings() {
_cache.settings = null;
}
/** Call after any write that can change the tool enable/disable state. */
export function invalidateTools() {
_cache.tools = null;
}
+2 -1
View File
@@ -35,6 +35,7 @@ import {
inheritModelRouteState,
} from './chatModelProvenance.js';
import { createTerminalStreamError, isRecoverableStreamError } from './chatStreamErrors.js';
import { loadPanel } from './panels.js';
const RESEARCH_TIMEOUT_MS = 360000;
const DEFAULT_TIMEOUT_MS = 120000;
@@ -6606,7 +6607,7 @@ import { createTerminalStreamError, isRecoverableStreamError } from './chatStrea
// Images → Gallery editor.
if (isImage) {
try {
const gx = await import('./galleryEditor.js');
const gx = await loadPanel('editor');
if (gx.openEditor) { gx.openEditor(url, id, null, name); return; }
} catch (e) { console.warn('gallery open failed', e); }
window.open(url, '_blank');
+9 -3
View File
@@ -9,7 +9,9 @@ import { providerLogo, providerLabel } from './providers.js';
import settingsModule from './settings.js';
import spinnerModule from './spinner.js';
import { bindMenuDismiss } from './escMenuStack.js';
import { loadPanel } from './panels.js';
import { matchModelKey } from './model/matchKey.js';
import { getTools } from './appConfig.js';
const SEARCH_ICON = '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="11" cy="11" r="8"/><path d="M21 21l-4.35-4.35"/></svg>';
const REPORT_ICON = '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/><line x1="10" y1="9" x2="8" y2="9"/></svg>';
@@ -445,8 +447,12 @@ function stripExecutedFence(match, tag, inline, body) {
async function loadExecFenceRegex() {
try {
const res = await fetch('/api/tools', { credentials: 'same-origin' });
const data = await res.json();
// Shared with admin.js, and — more to the point — with the other copies of
// this module: chatRenderer.js is imported under three different ?v= query
// strings, so it is instantiated three times per load and used to issue
// three identical /api/tools requests. appConfig.js is imported by one
// specifier from all of them, so they now share a single fetch.
const data = await getTools();
const tags = (data.tools || [])
.map((t) => t.id)
.filter((id) => id && !EXEC_FENCE_NON_TOOL.has(id));
@@ -1548,7 +1554,7 @@ export function buildImageBubble(imageUrl, prompt, model, size, quality, imageId
try {
const [galleryMod, editorMod] = await Promise.all([
import('./gallery.js'),
import('./galleryEditor.js'),
loadPanel('editor'),
]);
// Ensure the Gallery modal is open so the editor has a container
// to render into; switch its tabs to the Edit tab.
+3 -1
View File
@@ -15,6 +15,7 @@ let _getPlatform;
let _serverByVal;
let _isWindows;
let _buildEnvPrefix;
let _psQuote;
let _buildServeCmd;
let _detectBackend;
let _detectToolParser;
@@ -538,7 +539,7 @@ export async function _runModelDownload(panel, model, backend, hostOverride) {
if (srv.downloadDir) payload.local_dir = srv.downloadDir;
if (isWin) {
if (env === 'venv' && envPath) {
payload.env_prefix = '& ' + (envPath.endsWith('\\Scripts\\Activate.ps1') ? envPath : envPath + '\\Scripts\\Activate.ps1');
payload.env_prefix = '& ' + _psQuote(envPath.endsWith('\\Scripts\\Activate.ps1') ? envPath : envPath + '\\Scripts\\Activate.ps1');
} else if (env === 'conda' && envPath) {
payload.env_prefix = 'conda activate ' + envPath;
}
@@ -652,6 +653,7 @@ export function initDownload(shared) {
_serverByVal = shared._serverByVal;
_isWindows = shared._isWindows;
_buildEnvPrefix = shared._buildEnvPrefix;
_psQuote = shared._psQuote;
_buildServeCmd = shared._buildServeCmd;
_detectBackend = shared._detectBackend;
_detectToolParser = shared._detectToolParser;
+3 -1
View File
@@ -338,6 +338,7 @@ let _sshPrefix;
let _getPlatform;
let _isWindows;
let _buildEnvPrefix;
let _psQuote;
let _loadPresets;
let _savePresets;
let _copyText;
@@ -1971,7 +1972,7 @@ export async function _launchServeTask(shortName, repo, cmd, fields, hostOverrid
let envPrefix = '';
if (_isWindows()) {
if (_envState.env === 'venv' && _envState.envPath) {
envPrefix = '& ' + (_envState.envPath.endsWith('\\Scripts\\Activate.ps1') ? _envState.envPath : _envState.envPath + '\\Scripts\\Activate.ps1');
envPrefix = '& ' + _psQuote(_envState.envPath.endsWith('\\Scripts\\Activate.ps1') ? _envState.envPath : _envState.envPath + '\\Scripts\\Activate.ps1');
} else if (_envState.env === 'conda' && _envState.envPath) {
envPrefix = 'conda activate ' + _envState.envPath;
}
@@ -4402,6 +4403,7 @@ export function initRunning(shared) {
_getPlatform = shared._getPlatform;
_isWindows = shared._isWindows;
_buildEnvPrefix = shared._buildEnvPrefix;
_psQuote = shared._psQuote;
_loadPresets = shared._loadPresets;
_savePresets = shared._savePresets;
_copyText = shared._copyText;
+5
View File
@@ -9738,6 +9738,11 @@ import { bindMenuDismiss, dismissOrRemove } from './escMenuStack.js';
const container = document.createElement('div');
container.style.cssText = 'padding:20px;font-family:sans-serif;font-size:12px;color:#000;background:#fff;line-height:1.6;';
container.innerHTML = html;
// This container is detached, so the document-scoped flush mdToHtml
// schedules never sees it. Typeset the deferred math before html2pdf
// rasterises, or the PDF gets raw formula source. renderMath() returns
// immediately, without loading KaTeX, when there is nothing pending.
await markdownModule.renderMath(container);
const baseName = _getExportBaseName();
window.html2pdf().set({
margin: 10,
+2 -2
View File
@@ -23,6 +23,7 @@ import {
_tryFoldHintSig, _foldSignature, _SIG_ICON, _QUOTE_ICON,
} from './emailLibrary/signatureFold.js';
import { state } from './emailLibrary/state.js';
import { getSettings } from './appConfig.js';
import { collapseSidebarToRail } from './modalSnap.js';
import { emailApiUrl } from './emailShared.js';
import { bindMenuDismiss, dismissOrRemove } from './escMenuStack.js';
@@ -993,8 +994,7 @@ function _syncEmailReminderBellVisibility(enabled) {
async function _loadEmailReminderBellVisibility() {
try {
const res = await fetch('/api/auth/settings', { credentials: 'same-origin' });
const settings = await res.json();
const settings = await getSettings();
_syncEmailReminderBellVisibility(settings.reminder_channel === 'email');
} catch (_) {
_syncEmailReminderBellVisibility(false);
+49 -1
View File
@@ -3,7 +3,7 @@
*/
import uiModule from './ui.js';
import { openEditor, closeEditor, isEditorOpen } from './galleryEditor.js?v=20260708match1';
import { loadPanel } from './panels.js';
import spinnerModule from './spinner.js';
import { makeWindowDraggable } from './windowDrag.js';
import { bindMenuDismiss, dismissOrRemove } from './escMenuStack.js';
@@ -15,6 +15,54 @@ const API_BASE = window.location.origin;
let _open = false;
let _galleryResizeHandler = null;
// ── Image editor, loaded on first use ──
// galleryEditor.js plus everything under js/editor/ is 54 modules / 576 KB.
// It used to be a static import here, so every page load paid for it even
// though most sessions never touch the Edit tab. The wrappers below keep the
// three call shapes the rest of this file already uses.
//
// closeEditor() and isEditorOpen() stay synchronous on purpose: if the module
// was never loaded there is no edit session to close, and none can be open.
let _editorMod = null;
let _editorLoading = false;
async function _loadEditor() {
_editorLoading = true;
try {
_editorMod = await loadPanel('editor');
return _editorMod;
} finally {
_editorLoading = false;
}
}
async function openEditor(...args) {
let mod = _editorMod;
if (!mod) {
try {
mod = await _loadEditor();
} catch (e) {
// Previously unreachable — a static import either loaded or the whole
// page failed. Now it can fail on its own (offline before the panel was
// ever cached), so say so instead of doing nothing.
console.error('[gallery] image editor failed to load', e);
uiModule?.showError?.('Failed to load the image editor');
return;
}
}
return mod.openEditor(...args);
}
function closeEditor(...args) {
return _editorMod ? _editorMod.closeEditor(...args) : undefined;
}
// True while the module is still in flight as well — the gallery-close paths
// use this to refuse to tear the container down under an edit that is opening.
function isEditorOpen() {
return _editorLoading || (_editorMod ? _editorMod.isEditorOpen() : false);
}
// Auto-refresh gallery when new image is generated
window.addEventListener('gallery-refresh', (e) => {
if (e?.detail?.source === 'chat-upload' && _sort !== 'recent') {
+2 -2
View File
@@ -3,6 +3,7 @@
// ============================================
import { IS_MAC, isAltGrEvent } from './platform.js';
import { getSettings } from './appConfig.js';
const _defaultKeybinds = {
search: 'ctrl+k', toggle_sidebar: 'ctrl+alt+b', new_session: 'ctrl+alt+n',
@@ -56,8 +57,7 @@ export function initKeyboardShortcuts(modules) {
window._odysseusKeybinds = { ..._defaultKeybinds };
// Load saved keybinds
fetch('/api/auth/settings', { credentials: 'same-origin' })
.then(r => r.json())
getSettings()
.then(s => { if (s.keybinds) window._odysseusKeybinds = { ..._defaultKeybinds, ...s.keybinds }; })
.catch(() => {});
+202 -63
View File
@@ -10,6 +10,127 @@ import { replaceEmojiShortcodes, hasEmojiShortcode } from './emojiShortcodes.js'
var escapeHtml = uiModule.esc;
// Mermaid and KaTeX are vendored under /static/lib and fetched on first use.
// Loading them from <head> cost every session ~985 KB on the wire even though
// most chats never contain a diagram or a formula. Both loaders memoise the
// *promise* rather than the resolved library, so concurrent callers share one
// fetch and a double trigger cannot start two loads. A failed load clears the
// memo so the next diagram/formula retries instead of being poisoned forever.
const MERMAID_SRC = '/static/lib/mermaid.min.js';
const KATEX_SRC = '/static/lib/katex/katex.min.js';
const KATEX_CSS = '/static/lib/katex/katex.min.css';
// Marks math emitted before KaTeX finished loading; renderMath() swaps these
// for typeset output. The source stays as readable text inside the span, so a
// load that never completes degrades to plain text rather than to nothing.
const MATH_PENDING_CLASS = 'ody-math-pending';
// KaTeX has no entity syntax: it reads a bare "&" as an alignment marker and
// errors out on anything that is not a valid column break, so "a &lt; b" comes
// back as a red .katex-error instead of a formula. mdToHtml escapes the whole
// string before the math pass, which leaves two spellings of the same
// character at the delimiters — a typed "<" arrives as "&lt;", while a typed
// "&lt;" arrives as "&amp;lt;" — and both have to reach KaTeX as "<".
//
// One alternation, longest form first, so nothing this writes is scanned
// again. Chained .replace() calls cannot do it: unescaping "&amp;" first lets
// the next pass eat the "&lt;" it just produced (the double-unescape CodeQL
// flags), and unescaping it last leaves the entity spelling intact and breaks
// the render. The code-block pass upstream keeps its chained order on purpose
// — Markdown does not decode entities inside code, so "&lt;" there is meant to
// stay visible.
const MATH_SOURCE_ENTITY_RE = /&amp;(?:lt|gt|amp|quot|#39);|&lt;|&gt;|&amp;/g;
const MATH_SOURCE_ENTITIES = {
'&amp;lt;': '<',
'&amp;gt;': '>',
'&amp;amp;': '&',
'&amp;quot;': '"',
'&amp;#39;': "'",
'&lt;': '<',
'&gt;': '>',
'&amp;': '&',
};
function decodeMathSource(text) {
return String(text).replace(MATH_SOURCE_ENTITY_RE, (entity) => MATH_SOURCE_ENTITIES[entity]);
}
let _mermaidPromise = null;
let _katexPromise = null;
let _mathFlushScheduled = false;
function _loadScript(src) {
return new Promise((resolve, reject) => {
const script = document.createElement('script');
script.src = src;
script.addEventListener('load', () => resolve(), { once: true });
script.addEventListener('error', () => reject(new Error('Failed to load ' + src)), { once: true });
document.head.appendChild(script);
});
}
function _loadStylesheet(href) {
// Resolves either way: without the stylesheet KaTeX still produces correct
// markup, just unstyled, which beats failing the whole math render.
return new Promise((resolve) => {
const link = document.createElement('link');
link.rel = 'stylesheet';
link.href = href;
link.addEventListener('load', () => resolve(), { once: true });
link.addEventListener('error', () => resolve(), { once: true });
document.head.appendChild(link);
});
}
/**
* Load Mermaid on first use and initialize it once.
*/
export function ensureMermaid() {
return (_mermaidPromise ??= _loadScript(MERMAID_SRC)
.then(() => {
if (!window.mermaid) throw new Error('mermaid global missing after load');
window.mermaid.initialize({ startOnLoad: false, theme: 'dark', securityLevel: 'loose' });
return window.mermaid;
})
.catch((err) => {
_mermaidPromise = null;
throw err;
}));
}
/**
* Load KaTeX (script + stylesheet) on first use.
*/
export function ensureKatex() {
return (_katexPromise ??= Promise.all([_loadScript(KATEX_SRC), _loadStylesheet(KATEX_CSS)])
.then(() => {
if (!window.katex) throw new Error('katex global missing after load');
return window.katex;
})
.catch((err) => {
_katexPromise = null;
throw err;
}));
}
// mdToHtml() is synchronous and its callers insert the returned string into the
// DOM themselves, so the placeholders are usually not attached yet when this
// fires. Loading first and scanning afterwards covers that gap: by the time
// KaTeX is in, the caller's innerHTML assignment has long since happened.
//
// setTimeout, not requestAnimationFrame: this has nothing to do with paint, and
// rAF is throttled to a stop in a background tab (and never fires at all in a
// headless browser), which would leave math untypeset until the tab is focused.
function _scheduleMathFlush() {
if (_mathFlushScheduled) return;
_mathFlushScheduled = true;
setTimeout(() => {
_mathFlushScheduled = false;
ensureKatex()
.then(() => renderMath(document))
.catch((e) => console.warn('KaTeX load error:', e));
}, 0);
}
function safeLinkUrl(rawUrl) {
const url = String(rawUrl || '').trim();
if (url.startsWith('#')) {
@@ -631,49 +752,45 @@ export function mdToHtml(src, opts) {
// KaTeX math rendering (after code blocks are extracted, so math in code is safe)
const mathBlocks = [];
if (window.katex) {
// Display math: \[ ... \] — GPT-style delimiter (gpt-5.x, Claude, etc.).
// Handle before $$/$ so all common delimiters render.
s = s.replace(/\\\[([\s\S]*?)\\\]/g, (match, math) => {
try {
const raw = math.replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>');
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
mathBlocks.push(katex.renderToString(raw.trim(), { displayMode: true, throwOnError: false }));
return placeholder;
} catch (e) { return match; }
});
// Inline math: \( ... \) — GPT-style inline delimiter. Single-line only
// ([^\n]) so a stray escaped paren in prose can't swallow across lines.
s = s.replace(/\\\(([^\n]*?)\\\)/g, (match, math) => {
try {
const raw = math.replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>');
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
mathBlocks.push(katex.renderToString(raw.trim(), { displayMode: false, throwOnError: false }));
return placeholder;
} catch (e) { return match; }
});
// Display math: $$...$$
s = s.replace(/\$\$([\s\S]*?)\$\$/g, (match, math) => {
try {
const raw = math.replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>');
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
mathBlocks.push(katex.renderToString(raw.trim(), { displayMode: true, throwOnError: false }));
return placeholder;
} catch (e) { return match; }
});
// Inline math: $...$ — single line only, and Pandoc-style delimiter rules so
// currency doesn't render as math ("$5 to $10"): the opening $ must be
// immediately followed by a non-space, the closing $ must be immediately
// preceded by a non-space and not followed by a digit.
s = s.replace(/(?<![\$\d])\$(?!\$)(?=\S)([^\$\n]+?)(?<=\S)\$(?!\$|\d)/g, (match, math) => {
try {
const raw = math.replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>');
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
mathBlocks.push(katex.renderToString(raw.trim(), { displayMode: false, throwOnError: false }));
return placeholder;
} catch (e) { return match; }
});
}
let sawPendingMath = false;
// Typeset straight away when KaTeX is already in, otherwise bank the source in
// an inert placeholder for renderMath() to swap once the library lands.
const pushMath = (math, displayMode) => {
const raw = decodeMathSource(math).trim();
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
if (window.katex) {
mathBlocks.push(katex.renderToString(raw, { displayMode, throwOnError: false }));
} else {
sawPendingMath = true;
mathBlocks.push(`<span class="${MATH_PENDING_CLASS}" data-display="${displayMode}">${escapeHtml(raw)}</span>`);
}
return placeholder;
};
// Display math: \[ ... \] — GPT-style delimiter (gpt-5.x, Claude, etc.).
// Handle before $$/$ so all common delimiters render.
s = s.replace(/\\\[([\s\S]*?)\\\]/g, (match, math) => {
try { return pushMath(math, true); } catch (e) { return match; }
});
// Inline math: \( ... \) — GPT-style inline delimiter. Single-line only
// ([^\n]) so a stray escaped paren in prose can't swallow across lines.
s = s.replace(/\\\(([^\n]*?)\\\)/g, (match, math) => {
try { return pushMath(math, false); } catch (e) { return match; }
});
// Display math: $$...$$
s = s.replace(/\$\$([\s\S]*?)\$\$/g, (match, math) => {
try { return pushMath(math, true); } catch (e) { return match; }
});
// Inline math: $...$ — single line only, and Pandoc-style delimiter rules so
// currency doesn't render as math ("$5 to $10"): the opening $ must be
// immediately followed by a non-space, the closing $ must be immediately
// preceded by a non-space and not followed by a digit.
s = s.replace(/(?<![\$\d])\$(?!\$)(?=\S)([^\$\n]+?)(?<=\S)\$(?!\$|\d)/g, (match, math) => {
try { return pushMath(math, false); } catch (e) { return match; }
});
if (sawPendingMath) _scheduleMathFlush();
// Handle pipe tables
s = s.replace(/(?:^|\n)([^\n]*\|[^\n]*\|[^\n]*)(?:\n([^\n]*\|[^\n]*\|[^\n]*))*/g, (table) => {
@@ -826,19 +943,47 @@ export function renderContent(content) {
}
/**
* Initialize any unprocessed Mermaid diagrams in a container (or whole document)
* Initialize any unprocessed Mermaid diagrams in a container (or whole document).
* Returns a promise so callers can await the (lazy) library load if they need to.
*/
export function renderMermaid(container) {
if (!window.mermaid) return;
initMermaid();
const target = container || document;
const pending = target.querySelectorAll('pre.mermaid:not([data-processed])');
if (pending.length === 0) return;
try {
window.mermaid.run({ nodes: pending });
} catch (e) {
console.warn('Mermaid render error:', e);
}
if (!target || typeof target.querySelectorAll !== 'function') return Promise.resolve();
// Cheap pre-check: no fence on the page means Mermaid is never fetched.
if (target.querySelectorAll('pre.mermaid:not([data-processed])').length === 0) return Promise.resolve();
return ensureMermaid()
.then((mermaid) => {
// Re-query after the load: during streaming the renderer replaces the
// message body repeatedly, so the nodes seen before the fetch are stale.
const nodes = [...target.querySelectorAll('pre.mermaid:not([data-processed])')]
.filter((node) => node.isConnected);
if (nodes.length === 0) return;
return mermaid.run({ nodes });
})
.catch((e) => { console.warn('Mermaid render error:', e); });
}
/**
* Typeset any math that mdToHtml() had to defer because KaTeX was not loaded
* yet. Once KaTeX is in, mdToHtml() renders inline and this finds nothing.
*/
export function renderMath(container) {
const target = container || document;
if (!target || typeof target.querySelectorAll !== 'function') return Promise.resolve();
if (target.querySelectorAll('.' + MATH_PENDING_CLASS).length === 0) return Promise.resolve();
return ensureKatex()
.then((katex) => {
target.querySelectorAll('.' + MATH_PENDING_CLASS).forEach((el) => {
const displayMode = el.getAttribute('data-display') === 'true';
try {
el.outerHTML = katex.renderToString(el.textContent || '', { displayMode, throwOnError: false });
} catch (e) {
// Leave the source visible — readable, just not typeset.
el.classList.remove(MATH_PENDING_CLASS);
}
});
})
.catch((e) => { console.warn('KaTeX render error:', e); });
}
const markdownModule = {
@@ -853,20 +998,14 @@ const markdownModule = {
extractThinkingBlocks,
normalizeThinkingMarkup,
startsWithReasoningPrefix,
renderMermaid
renderMermaid,
renderMath,
ensureMermaid,
ensureKatex
};
export default markdownModule;
// Mermaid is loaded async so it cannot delay the app shell.
function initMermaid() {
if (!window.mermaid || window.__odysseusMermaidReady) return;
window.mermaid.initialize({ startOnLoad: false, theme: 'dark', securityLevel: 'loose' });
window.__odysseusMermaidReady = true;
}
window.odysseusInitMermaid = initMermaid;
initMermaid();
// Persist which thinking sections were expanded across page refreshes.
// IDs are render-generated (Date.now-based) so we key by a stable hash of
// the inner text content instead — same content reproduces the same hash on
+53
View File
@@ -0,0 +1,53 @@
/**
* Panel loader registry — imports the modules behind a feature panel the
* first time that panel is actually used.
*
* The panels already populate themselves on open (each fetches its own data).
* They were just not *loaded* on demand: every one of them sat on the critical
* path of every page load, opened or not.
*
* A panel only belongs here once it has been checked for import-time side
* effects that something outside the panel depends on at startup. Entries get
* added one panel at a time, not in bulk.
*
* Note the service worker still precaches these modules (PANEL_PRECACHE in
* sw.js) — they are off the critical path, not off the offline manifest.
*/
const LOADERS = {
editor: () => import('./galleryEditor.js'),
};
/**
* Build a memoising loader over a name -> import-thunk map. Exported so the
* behaviour can be tested without pulling a real panel's module graph in.
*/
export function createPanelLoader(loaders) {
const cache = new Map();
return function load(name) {
const cached = cache.get(name);
if (cached) return cached;
const loader = loaders[name];
if (!loader) throw new Error(`loadPanel: unknown panel "${name}"`);
// A failed load (offline, 404, syntax error) is not memoised — caching the
// rejection would leave the panel broken for the rest of the session even
// after the network came back.
const pending = Promise.resolve().then(loader).catch((err) => {
cache.delete(name);
throw err;
});
cache.set(name, pending);
return pending;
};
}
/**
* Load a panel's module, once. Returns the same promise on every call for the
* same panel; throws for a name that is not registered.
*/
export const loadPanel = createPanelLoader(LOADERS);
/** Registered panel names — the registry is the list, not a second copy of it. */
export function panelNames() {
return Object.keys(LOADERS);
}
+9 -5
View File
@@ -4,20 +4,21 @@
* Search settings management — reads active provider from admin settings.
*/
let API_BASE = '';
import { getSettings, invalidateSettings } from './appConfig.js';
let _provider = 'searxng';
let _loaded = false;
export function init(apiBase) {
API_BASE = apiBase;
// No API base parameter any more: the settings request lives in appConfig.js and
// resolves against the document origin, which is exactly what API_BASE held.
export function init() {
// Fetch provider on init so it's ready when chat needs it
_fetchProvider();
}
async function _fetchProvider() {
try {
const res = await fetch((API_BASE || '') + '/api/auth/settings', { credentials: 'same-origin' });
const s = await res.json();
const s = await getSettings();
_provider = s.search_provider || 'searxng';
_loaded = true;
} catch (e) { /* keep default */ }
@@ -39,6 +40,9 @@ export function getProviderLabel() {
/** Re-fetch after admin saves new settings */
export function refresh() {
// Drop the shared snapshot first: the point of this call is to observe the
// settings that were just written, so it must not be served from cache.
invalidateSettings();
_fetchProvider();
}
+46 -66
View File
@@ -26,11 +26,37 @@ import { sortModelIds } from './modelSort.js';
import { providerLogo } from './providers.js';
import { isAltGrEvent } from './platform.js';
import { bindMenuDismiss } from './escMenuStack.js';
import { invalidateSettings } from './appConfig.js';
let initialized = false;
let modalEl = null;
let _authPolicy = { password_min_length: 8 };
/**
* POST a settings patch, then drop the shared snapshot in appConfig.js.
*
* Every write in this file goes through here so no save path can forget the
* invalidation — a stale settings object served for the rest of the session is
* a worse bug than the duplicate fetches the cache removes. The invalidation is
* in a `finally` because a request that throws on the way back may still have
* been applied server-side.
*
* Reads in this file deliberately stay direct fetches: this panel is the writer
* and edits what it reads, so it must see the authoritative state, not a cache.
*/
async function _postSettings(body) {
try {
return await fetch('/api/auth/settings', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
} finally {
invalidateSettings();
}
}
const el = byId;
function esc(s) { return uiModule.esc(s); }
function safeRasterDataUrl(raw) {
@@ -276,10 +302,7 @@ function _bindFallbackWidget(opts) {
var body = {};
body[settingKey] = clean;
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
});
await _postSettings(body);
} catch (e) { console.warn('[fallback] save failed for ' + settingKey, e); }
}
@@ -389,12 +412,9 @@ async function initDefaultChat() {
async function saveDefault() {
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
default_endpoint_id: epSel.value,
default_model: modelSel.value
})
await _postSettings({
default_endpoint_id: epSel.value,
default_model: modelSel.value
});
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(function() { msg.textContent = ''; }, 2000);
@@ -449,12 +469,9 @@ async function initUtilityModel() {
// no toggle, "—" means "unset, use chat").
async function saveUtility() {
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
utility_endpoint_id: epSel.value || '',
utility_model: modelSel.value || ''
})
await _postSettings({
utility_endpoint_id: epSel.value || '',
utility_model: modelSel.value || ''
});
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(function() { msg.textContent = ''; }, 1500);
@@ -547,10 +564,7 @@ async function initTeacherModel() {
spec = ep ? (modelSel.value + '@' + ep.name) : modelSel.value;
}
var enabled = enabledToggle ? !!enabledToggle.checked : false;
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ teacher_enabled: enabled, teacher_model: spec })
});
await _postSettings({ teacher_enabled: enabled, teacher_model: spec });
msg.textContent = enabled ? (spec ? 'Saved' : 'Pick an endpoint + model') : 'Disabled';
msg.style.color = enabled && !spec ? 'var(--red)' : 'var(--fg)';
setTimeout(function() { msg.textContent = ''; }, 2000);
@@ -625,8 +639,7 @@ async function initImageSettings() {
async function saveSettings() {
try {
const res = await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ image_gen_enabled: enabledToggle ? enabledToggle.checked : false, image_model: modelSel.value, image_quality: qualSel.value }) });
const res = await _postSettings({ image_gen_enabled: enabledToggle ? enabledToggle.checked : false, image_model: modelSel.value, image_quality: qualSel.value });
if (!res.ok) throw new Error(await res.text().catch(() => `HTTP ${res.status}`));
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)'; setTimeout(() => { msg.textContent = ''; }, 2000);
} catch (e) { msg.textContent = 'Failed to save'; msg.style.color = 'var(--red)'; }
@@ -700,8 +713,7 @@ async function initVisionSettings() {
async function saveSettings() {
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ vision_enabled: enabledToggle ? enabledToggle.checked : true, vision_model: vlSel.value }) });
await _postSettings({ vision_enabled: enabledToggle ? enabledToggle.checked : true, vision_model: vlSel.value });
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)'; setTimeout(() => { msg.textContent = ''; }, 2000);
} catch (e) { msg.textContent = 'Failed to save'; msg.style.color = 'var(--red)'; }
}
@@ -782,8 +794,7 @@ async function initTtsSettings() {
async function saveTTS() {
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ tts_enabled: ttsEnabledToggle ? ttsEnabledToggle.checked : true, tts_provider: provSel.value, tts_model: getModel() || 'tts-1', tts_voice: getVoice() || 'alloy', tts_speed: speedSelect.value || '1' }) });
await _postSettings({ tts_enabled: ttsEnabledToggle ? ttsEnabledToggle.checked : true, tts_provider: provSel.value, tts_model: getModel() || 'tts-1', tts_voice: getVoice() || 'alloy', tts_speed: speedSelect.value || '1' });
ttsMsg.textContent = 'Saved'; ttsMsg.style.color = 'var(--fg)'; setTimeout(() => { ttsMsg.textContent = ''; }, 2000);
if (window.aiTTSManager) window.aiTTSManager.checkAvailability();
} catch (e) { ttsMsg.textContent = 'Failed to save'; ttsMsg.style.color = 'var(--red)'; }
@@ -944,9 +955,7 @@ async function initSttSettings() {
async function saveSTT() {
try {
var enabled = sttEnabledToggle ? sttEnabledToggle.checked : false;
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ stt_enabled: enabled, stt_provider: provSel.value, stt_model: getModel() || 'base', stt_language: langInput.value.trim() }) });
await _postSettings({ stt_enabled: enabled, stt_provider: provSel.value, stt_model: getModel() || 'base', stt_language: langInput.value.trim() });
sttMsg.textContent = 'Saved'; sttMsg.style.color = 'var(--fg)'; setTimeout(() => { sttMsg.textContent = ''; }, 2000);
// Notify voiceRecorder of effective provider and update send button icon
if (window.voiceRecorderModule) window.voiceRecorderModule._sttProvider = effectiveProvider();
@@ -1102,10 +1111,7 @@ async function initSearchSettings() {
payload[kf] = keyInput.value.trim();
_settings[kf] = keyInput.value.trim();
}
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
await _postSettings(payload);
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(refreshStatus, 2000);
if (searchModule && searchModule.refresh) searchModule.refresh();
@@ -1257,11 +1263,7 @@ async function initSearchSettings() {
async function _saveFallbackChain(chain) {
_settings.search_fallback_chain = chain;
try {
await fetch('/api/auth/settings', {
method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ search_fallback_chain: chain }),
});
await _postSettings({ search_fallback_chain: chain });
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(refreshStatus, 2000);
} catch (e) { msg.textContent = 'Failed to save'; msg.style.color = 'var(--red)'; }
@@ -1425,10 +1427,7 @@ async function initResearchSettings() {
}
}
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
await _postSettings(payload);
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(showStatus, 2000);
} catch (e) { msg.textContent = 'Failed to save'; msg.style.color = 'var(--red)'; }
@@ -1492,10 +1491,7 @@ async function initResearchSearchSettings() {
async function saveResearchSearch() {
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ research_search_provider: searchSel.value })
});
await _postSettings({ research_search_provider: searchSel.value });
msg.textContent = 'Saved'; msg.style.color = 'var(--fg)';
setTimeout(function() { msg.textContent = ''; }, 2000);
} catch (e) { msg.textContent = 'Failed to save'; msg.style.color = 'var(--red)'; }
@@ -1537,10 +1533,7 @@ async function initAgentSettings() {
if (rounds != null) payload.agent_max_rounds = rounds;
if (supInput) payload.agent_supervisor_ladder = !!supInput.checked;
try {
await fetch('/api/auth/settings', { method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
await _postSettings(payload);
msg.textContent = (tools > 0 ? 'Limit: ' + tools + ' tool calls' : 'Unlimited tool calls') +
(rounds != null ? ' · ' + rounds + ' steps/message' : '') +
(supInput && supInput.checked ? ' · supervisor on' : '');
@@ -1935,11 +1928,7 @@ async function initShortcuts() {
async function saveKeybinds() {
try {
await fetch('/api/auth/settings', {
method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ keybinds }),
});
await _postSettings({ keybinds });
// Update global keybinds so they take effect immediately
window._odysseusKeybinds = keybinds;
if (uiModule && uiModule.showToast) uiModule.showToast('Shortcut saved');
@@ -2232,11 +2221,7 @@ async function initReminderSettings() {
pubDebounce = setTimeout(async () => {
try {
const val = pubUrlIn.value.trim().replace(/\/+$/, '');
await fetch('/api/auth/settings', {
method: 'POST', credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ app_public_url: val }),
});
await _postSettings({ app_public_url: val });
if (pubUrlMsg) {
pubUrlMsg.textContent = val ? 'Saved' : 'Cleared (deep-links disabled)';
pubUrlMsg.style.color = 'var(--green,#50fa7b)';
@@ -2534,12 +2519,7 @@ async function initReminderSettings() {
async function save(patch) {
try {
await fetch('/api/auth/settings', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(patch),
});
await _postSettings(patch);
} catch (e) { console.warn('Failed to save reminder settings', e); }
}
+2 -2
View File
@@ -22,6 +22,7 @@ import settingsModule from './settings.js';
import cookbookModule from './cookbook.js';
import { EVAL_PROMPTS } from './compare/index.js';
import { PROVIDER_DEVICE_FLOWS, formatDeviceFlowError, runProviderDeviceFlow } from './providerDeviceFlow.js';
import { getSettings } from './appConfig.js';
// ── Module state ──────────────────────────────────────────────────────
@@ -5220,8 +5221,7 @@ async function _cmdShortcuts(args, ctx) {
};
try {
const res = await fetch(`${API_BASE}/api/auth/settings`, { credentials: 'same-origin' });
const settings = await res.json();
const settings = await getSettings();
if (settings.keybinds) {
keybinds = { ...keybinds, ...settings.keybinds };
}
+19 -18
View File
@@ -10,6 +10,7 @@ import { topPortalZ } from './toolWindowZOrder.js';
import { sortModelIds } from './modelSort.js';
import { ordinalSuffix } from './util/ordinal.js';
import { bindMenuDismiss, dismissOrRemove } from './escMenuStack.js';
import { getSettings, invalidateSettings } from './appConfig.js';
const API_BASE = window.location.origin;
let _open = false;
@@ -214,31 +215,31 @@ async function _fetchActions() {
return _builtinActions;
}
let _urgentEmailSettings = null;
async function _fetchUrgentEmailSettings() {
if (_urgentEmailSettings) return _urgentEmailSettings;
try {
const res = await fetch('/api/auth/settings', { credentials: 'same-origin' });
_urgentEmailSettings = await res.json();
return await getSettings();
} catch (e) {
_urgentEmailSettings = { urgent_email_prompt: '' };
return { urgent_email_prompt: '' };
}
return _urgentEmailSettings;
}
async function _saveUrgentEmailSettings(prompt) {
_urgentEmailSettings = {
...(_urgentEmailSettings || {}),
urgent_email_prompt: prompt || '',
};
await fetch('/api/auth/settings', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
urgent_email_prompt: prompt || '',
}),
});
try {
await fetch('/api/auth/settings', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
urgent_email_prompt: prompt || '',
}),
});
} finally {
// The shared snapshot still carries the old prompt — drop it so the next
// read (here or in any other module) sees what was just written. In a
// `finally` because a request that throws on the way back may still have
// been applied.
invalidateSettings();
}
}
const _EMAIL_ACCOUNT_ACTIONS = new Set([
+6 -3
View File
@@ -1,6 +1,8 @@
// static/js/tts-ai.js
// AI Text-to-Speech Module — supports server TTS and browser Web Speech API
import { getSettings } from './appConfig.js';
class AITTSManager {
constructor() {
this.currentAudio = null;
@@ -30,10 +32,11 @@ class AITTSManager {
async checkAvailability() {
try {
// Check user setting first — if TTS is disabled in settings, don't show buttons
// Check user setting first — if TTS is disabled in settings, don't show buttons.
// settings.js re-calls this right after saving TTS settings; it invalidates
// the shared cache before doing so, so this still sees the new value.
try {
const settingsRes = await fetch('/api/auth/settings', { credentials: 'same-origin' });
const settings = await settingsRes.json();
const settings = await getSettings();
if (settings.tts_enabled === false) {
this.available = false;
this._provider = 'disabled';
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
File diff suppressed because one or more lines are too long
+1
View File
File diff suppressed because one or more lines are too long
+3587
View File
File diff suppressed because one or more lines are too long
+100 -5
View File
@@ -7,16 +7,42 @@
// - Other static assets (images/fonts/libs): cache-first with bg refresh.
// - API / non-GET: never cached.
// Bump CACHE_NAME whenever the precache list or SW logic changes.
const CACHE_NAME = 'odysseus-v376-settings-title-icons';
const CACHE_NAME = 'odysseus-v380-shared-config-image-editor-lazy-katex-mermaid';
// Core shell precached on install so repeat opens are instant without any
// network wait. Keep this list in sync with the <script type="module"> tags
// and <link rel="stylesheet"> in index.html.
// KaTeX resolves these from its own stylesheet, so caching the CSS without them
// gives offline math fallback glyphs instead of proper typesetting.
const KATEX_FONTS = [
'AMS-Regular', 'Caligraphic-Bold', 'Caligraphic-Regular',
'Fraktur-Bold', 'Fraktur-Regular',
'Main-Bold', 'Main-BoldItalic', 'Main-Italic', 'Main-Regular',
'Math-BoldItalic', 'Math-Italic',
'SansSerif-Bold', 'SansSerif-Italic', 'SansSerif-Regular',
'Script-Regular',
'Size1-Regular', 'Size2-Regular', 'Size3-Regular', 'Size4-Regular',
'Typewriter-Regular',
].map(name => `/static/lib/katex/fonts/KaTeX_${name}.woff2`);
// Two lists, two jobs — they are no longer the same set and must not be
// "resynced" back into one:
//
// PRECACHE = the app shell. Mirrors the <script type="module"> tags
// and <link rel="stylesheet"> in index.html — i.e. what
// loads before first paint.
// PANEL_PRECACHE = modules that index.html deliberately does NOT load,
// because js/panels.js imports them on first use. They are
// off the critical path, not out of the offline manifest:
// without them here, a panel the user never opened while
// online could not open offline at all.
//
// Both are fetched at install time, in the background. Entries must match the
// exact URL the browser requests, query string included.
const PRECACHE = [
'/',
'/static/style.css',
'/static/app.js',
'/static/js/storage.js',
'/static/js/appConfig.js',
'/static/js/ui.js',
'/static/js/markdown.js',
'/static/js/dragSort.js',
@@ -61,6 +87,75 @@ const PRECACHE = [
'/static/js/sidebar-layout.js',
'/static/js/section-management.js',
'/static/lib/highlight.min.js',
// Math turns up in ordinary answers and KaTeX is small, so precaching it and
// its fonts keeps formulas typeset offline. Mermaid is deliberately NOT
// precached: at 3.5 MB it would re-download on every CACHE_NAME bump, a poor
// trade for a library most sessions never touch. The cache-first rule below
// picks it up the first time a diagram renders, which is also when it starts
// mattering offline.
'/static/lib/katex/katex.min.js',
'/static/lib/katex/katex.min.css',
...KATEX_FONTS,
];
// Lazily-imported panel modules (js/panels.js). Not in index.html by design;
// precached so the panel still opens with no network.
const PANEL_PRECACHE = [
// Image editor — galleryEditor.js and its js/editor/ graph.
'/static/js/galleryEditor.js',
'/static/js/editor/ai-inpaint.js?v=20260708match1',
'/static/js/editor/ai-models.js',
'/static/js/editor/ai-rembg.js',
'/static/js/editor/ai-tool-runner.js',
'/static/js/editor/ai-tools-misc.js',
'/static/js/editor/build/controls.js?v=20260708match1',
'/static/js/editor/build/popups.js',
'/static/js/editor/build/right-panel.js',
'/static/js/editor/build/toolbar.js?v=20260708sam3',
'/static/js/editor/build/topbar.js',
'/static/js/editor/build/transform-popup.js',
'/static/js/editor/canvas-coords.js',
'/static/js/editor/canvas-events.js',
'/static/js/editor/canvas-transforms.js',
'/static/js/editor/checkerboard.js',
'/static/js/editor/clipboard-and-drop.js',
'/static/js/editor/composite-helpers.js',
'/static/js/editor/filters/blur.js',
'/static/js/editor/filters/edge-feather.js',
'/static/js/editor/fx/adj-popup.js',
'/static/js/editor/fx/filter-string.js',
'/static/js/editor/fx/histogram.js',
'/static/js/editor/fx/pixel-pass.js',
'/static/js/editor/harmonize-masks.js',
'/static/js/editor/history-panel.js',
'/static/js/editor/keyboard-shortcuts.js',
'/static/js/editor/layer-helpers.js',
'/static/js/editor/layer-panel.js',
'/static/js/editor/mask-utils.js',
'/static/js/editor/shortcuts-popover.js',
'/static/js/editor/slider-ux.js',
'/static/js/editor/snap.js',
'/static/js/editor/state.js',
'/static/js/editor/stroke-pipeline.js',
'/static/js/editor/stroke-tool-sliders.js',
'/static/js/editor/tools/clone.js',
'/static/js/editor/tools/crop.js',
'/static/js/editor/tools/flood-fill.js',
'/static/js/editor/tools/lasso-mask.js',
'/static/js/editor/tools/lasso.js',
'/static/js/editor/tools/move.js',
'/static/js/editor/tools/stroke.js',
'/static/js/editor/tools/transform-drag.js',
'/static/js/editor/tools/transform-handles.js',
'/static/js/editor/tools/transform-session.js',
'/static/js/editor/tools/wand.js',
'/static/js/editor/wire-import.js',
'/static/js/editor/wire-inpaint-controls.js?v=20260708match1',
'/static/js/editor/wire-merge-buttons.js',
'/static/js/editor/wire-selection-controls.js',
'/static/js/editor/wire-topbar-menus.js',
'/static/js/editor/wire-topbar-overflow.js',
'/static/js/editor/wire-topbar.js',
];
self.addEventListener('install', (e) => {
@@ -69,7 +164,7 @@ self.addEventListener('install', (e) => {
// addAll is atomic — if any item fails, none are cached. Use individual
// puts so a single 404 can't block the whole install.
Promise.all(
PRECACHE.map(url =>
[...PRECACHE, ...PANEL_PRECACHE].map(url =>
fetch(url, { cache: 'reload' })
.then(res => res.ok ? cache.put(url, res) : null)
.catch(() => null)
+25
View File
@@ -10,3 +10,28 @@ def test_mask_token_handles_short_values(monkeypatch):
assert cli._mask_token("short") == "***"
assert cli._mask_token("abcdef1234567890") == "abcdef…7890"
assert cli._mask_token("short", reveal=True) == "short"
def test_task_webhook_url_matches_live_route_and_escapes_path_parts(monkeypatch):
make_core_db_stub(monkeypatch, models=["ScheduledTask"])
cli = load_script("odysseus-webhook")
url = cli._task_webhook_url(
"https://ody.example/",
"task/with space",
"token/with space",
)
assert url == (
"https://ody.example/api/tasks/task%2Fwith%20space/"
"webhook/token%2Fwith%20space"
)
def test_default_task_webhook_url_uses_current_task_prefix(monkeypatch):
make_core_db_stub(monkeypatch, models=["ScheduledTask"])
cli = load_script("odysseus-webhook")
assert cli._task_webhook_url(None, "task-1", "secret") == (
"http://localhost:7000/api/tasks/task-1/webhook/secret"
)
@@ -817,6 +817,12 @@ const STUBS = new Map([
bindMenuDismiss() {},
},
],
[
path.join(JS, 'appConfig.js'),
{
invalidateSettings() {},
},
],
[
path.join(JS, 'windowDrag.js'),
{
+375
View File
@@ -0,0 +1,375 @@
"""Pin the shared config cache in static/js/appConfig.js.
Background: /api/auth/settings was fetched independently by six modules and
/api/tools by three (chatRenderer.js is imported under three different ?v=
query strings, so it is three separate module instances) — 4 and 3 requests on
one cold load. Beyond the redundant work, each caller could observe a different
snapshot of the same object. appConfig.js gives them one promise each.
The two properties that matter are opposites, so both are tested here:
concurrent and later callers must NOT refetch, and a caller after a write MUST
see the new value — which only holds if every writer invalidates. The last test
is a source scan that checks exactly that, since a forgotten invalidation
serves a stale settings object for the rest of the session, which is worse than
the duplicate fetches this replaces.
Driven through `node --input-type=module` so the real module runs, same idiom as
test_esc_menu_stack_js.py. The module source is inlined rather than imported by
path because the repo has no `"type": "module"` in package.json; appConfig.js
has no imports of its own, so inlining is exact. `fetch` and `sessionStorage`
are stubbed, so nothing here touches the network or depends on timing.
"""
import json
import re
import shutil
import subprocess
from pathlib import Path
import pytest
_REPO = Path(__file__).resolve().parent.parent
_MODULE = _REPO / "static" / "js" / "appConfig.js"
_HAS_NODE = shutil.which("node") is not None
_SRC = _MODULE.read_text(encoding="utf-8") if _MODULE.exists() else ""
# Browser stand-ins. Every fetch is recorded and resolved from a queue the test
# controls, so "how many requests went out" is an exact count, not a guess.
_STUBS = r"""
const calls = [];
let responses = [];
globalThis.__queue = (fn) => { responses.push(fn); };
globalThis.fetch = (url, opts) => {
calls.push([url, opts]);
const next = responses.shift();
if (!next) throw new Error('unexpected fetch: ' + url);
return next();
};
globalThis.__calls = () => calls;
globalThis.__json = (value) => () => Promise.resolve({ json: () => Promise.resolve(value) });
globalThis.__fail = (msg) => () => Promise.reject(new Error(msg));
const store = new Map();
globalThis.sessionStorage = {
getItem: (k) => (store.has(k) ? store.get(k) : null),
setItem: (k, v) => { store.set(k, String(v)); },
removeItem: (k) => { store.delete(k); },
};
globalThis.__seedPrefetch = (value) => {
store.set('ody-prefetch-settings', JSON.stringify(value));
};
globalThis.__prefetchLeft = () => store.has('ody-prefetch-settings');
"""
def _run(body: str) -> str:
js = _STUBS + "\n" + _SRC + "\n" + body
proc = subprocess.run(
["node", "--input-type=module"],
input=js, capture_output=True, text=True, encoding="utf-8",
cwd=str(_REPO), timeout=30,
)
assert proc.returncode == 0, proc.stderr
return proc.stdout.strip()
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_concurrent_callers_share_one_request():
# The startup case: several modules ask before the first response lands.
body = """
__queue(__json({ tts_enabled: true }));
const [a, b, c] = await Promise.all([getSettings(), getSettings(), getSettings()]);
console.log(JSON.stringify({
requests: __calls().length,
url: __calls()[0][0],
credentials: __calls()[0][1].credentials,
sameObject: a === b && b === c,
value: a.tts_enabled,
}));
"""
assert json.loads(_run(body)) == {
"requests": 1,
"url": "/api/auth/settings",
"credentials": "same-origin",
"sameObject": True,
"value": True,
}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_later_caller_reuses_the_resolved_snapshot():
# A panel opened long after boot must not re-request.
body = """
__queue(__json({ search_provider: 'brave' }));
const first = await getSettings();
const second = await getSettings();
console.log(JSON.stringify({ requests: __calls().length, sameObject: first === second }));
"""
assert json.loads(_run(body)) == {"requests": 1, "sameObject": True}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_invalidate_forces_the_next_read_to_refetch():
# The write path: save a setting, then read it back and see the new value.
body = """
__queue(__json({ tts_enabled: true }));
__queue(__json({ tts_enabled: false }));
const before = await getSettings();
invalidateSettings();
const after = await getSettings();
console.log(JSON.stringify({
requests: __calls().length,
before: before.tts_enabled,
after: after.tts_enabled,
}));
"""
assert json.loads(_run(body)) == {"requests": 2, "before": True, "after": False}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_a_failed_fetch_does_not_poison_the_cache():
# Plain `??=` memoisation would keep the rejected promise, so one blip at
# boot would leave keybinds/TTS/search on defaults for the whole session.
body = """
__queue(__fail('offline'));
__queue(__json({ tts_enabled: true }));
let rejected = false;
try { await getSettings(); } catch (e) { rejected = e.message === 'offline'; }
const retry = await getSettings();
console.log(JSON.stringify({
rejected,
requests: __calls().length,
recovered: retry.tts_enabled,
}));
"""
assert json.loads(_run(body)) == {"rejected": True, "requests": 2, "recovered": True}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_settings_and_tools_are_independent_slots():
body = """
__queue(__json({ tts_enabled: true }));
__queue(__json({ tools: [{ id: 'web_search' }] }));
await getSettings();
const tools = await getTools();
const toolsUrl = __calls()[1][0];
invalidateSettings(); // must not drop the tools snapshot
const toolsAgain = await getTools();
console.log(JSON.stringify({
requests: __calls().length,
toolsUrl,
sameObject: tools === toolsAgain,
}));
"""
assert json.loads(_run(body)) == {
"requests": 2,
"toolsUrl": "/api/tools",
"sameObject": True,
}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_login_prefetch_is_used_once_and_then_consumed():
# login.html stashes a settings snapshot in sessionStorage just before it
# redirects, so the first load after a login should issue no request at all.
body = """
__seedPrefetch({ tts_enabled: false, from: 'prefetch' });
__queue(__json({ tts_enabled: true, from: 'network' }));
const first = await getSettings();
const consumed = !__prefetchLeft();
invalidateSettings();
const second = await getSettings();
console.log(JSON.stringify({
requests: __calls().length,
first: first.from,
consumed,
second: second.from,
}));
"""
assert json.loads(_run(body)) == {
"requests": 1,
"first": "prefetch",
"consumed": True,
"second": "network",
}
# ── Writers must invalidate ─────────────────────────────────────────────────
_WRITE_ENDPOINTS = {
"/api/auth/settings": "invalidateSettings",
"/api/tools": "invalidateTools",
}
# login.html is the pre-app login page: it has no module graph and its only call
# is the prefetch GET, so it is not a writer and cannot import appConfig.js.
_SCANNED = [_REPO / "static" / "app.js"] + sorted((_REPO / "static" / "js").rglob("*.js"))
def _post_sites(source: str, endpoint: str):
"""Yield the 1-based line of every fetch() to `endpoint` that is a POST."""
for m in re.finditer(re.escape(f"'{endpoint}'"), source):
window = source[m.start():m.start() + 240]
if re.search(r"method:\s*'POST'", window):
yield source[:m.start()].count("\n") + 1
def test_every_settings_writer_invalidates_the_shared_cache():
"""A POST that skips the invalidation serves a stale object for the session.
Checked by source scan rather than at runtime: the failure mode is a call
site that was never wired up, which no unit test of the cache itself can
see. `appConfig.js` itself is skipped — it is the cache, not a writer.
"""
missing = []
for path in _SCANNED:
if path.name == "appConfig.js":
continue
source = path.read_text(encoding="utf-8")
for endpoint, invalidator in _WRITE_ENDPOINTS.items():
for line in _post_sites(source, endpoint):
# The invalidation belongs in the same function as the POST;
# accept it anywhere in the surrounding 20 lines either way.
lines = source.splitlines()
near = "\n".join(lines[max(0, line - 20):line + 20])
if invalidator + "(" not in near:
missing.append(f"{path.relative_to(_REPO)}:{line} POST {endpoint}")
assert not missing, (
"POST sites with no nearby cache invalidation — the UI will serve a "
"stale snapshot after these writes:\n " + "\n ".join(missing)
)
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_out_of_band_tool_change_is_not_undone_by_an_unrelated_panel_save():
"""Admin > Tools must render authoritative state, not the startup snapshot.
The save posts the whole disabled-tool list rebuilt from the checkboxes, so
a stale render turns any unrelated toggle into a lost update: a tool
disabled out of band (manage_settings, another tab) comes back enabled.
refreshAll() calls loadBuiltinTools() on every panel open, which is why the
editor drops the shared entry before reading it.
"""
body = """
// Boot: chatRenderer.js reads the tool list for the exec-fence regex.
__queue(__json({ tools: [{ id: 'web_search', enabled: true }, { id: 'shell', enabled: true }] }));
const boot = await getTools();
// Out of band, this page hearing nothing about it: web_search is disabled.
__queue(__json({ tools: [{ id: 'web_search', enabled: false }, { id: 'shell', enabled: true }] }));
// Admin > Tools opens. loadBuiltinTools() invalidates, then reads.
invalidateTools();
const panel = await getTools();
// The user toggles one unrelated tool off. The save posts every unchecked
// box, so the list is only right if the render was authoritative.
const post = (snapshot) => {
const boxes = snapshot.tools.map(t => ({ id: t.id, checked: t.enabled }));
boxes.find(b => b.id === 'shell').checked = false;
return boxes.filter(b => !b.checked).map(b => b.id);
};
console.log(JSON.stringify({
requests: __calls().length,
posted: post(panel),
postedFromStaleSnapshot: post(boot),
}));
"""
assert json.loads(_run(body)) == {
"requests": 2,
# web_search stays disabled, which is the point.
"posted": ["web_search", "shell"],
# What the page-lifetime snapshot would have posted: web_search silently
# re-enabled by a toggle that had nothing to do with it.
"postedFromStaleSnapshot": ["shell"],
}
@pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def test_out_of_band_tool_change_after_panel_open_is_preserved_on_save():
"""Saving must merge the user's edit onto a fresh authoritative snapshot."""
body = """
// Panel opens while both tools are enabled.
__queue(__json({ tools: [
{ id: 'web_search', enabled: true },
{ id: 'shell', enabled: true },
] }));
invalidateTools();
const panel = await getTools();
// Another tab disables web_search after this panel has already rendered.
__queue(__json({ tools: [
{ id: 'web_search', enabled: false },
{ id: 'shell', enabled: true },
] }));
// The user only disables shell. Saving refreshes the authoritative state
// first and applies that one intended change on top of it.
invalidateTools();
const latest = await getTools();
const state = new Map(latest.tools.map(t => [t.id, !!t.enabled]));
state.set('shell', false);
const disabled = Array.from(state.entries())
.filter(([, enabled]) => !enabled)
.map(([id]) => id);
console.log(JSON.stringify({
panelWebSearchEnabled: panel.tools.find(t => t.id === 'web_search').enabled,
requests: __calls().length,
disabled,
}));
"""
assert json.loads(_run(body)) == {
"panelWebSearchEnabled": True,
"requests": 2,
"disabled": ["web_search", "shell"],
}
def test_admin_tool_save_refreshes_before_full_state_post():
"""Pin the lost-update guard in the Admin Tools full-list writer."""
source = (_REPO / "static" / "js" / "admin.js").read_text(encoding="utf-8")
match = re.search(
r"async function _saveToolState\(changes\) \{(.*?)\n \}\n"
r" function _updateCatCounter",
source,
re.S,
)
assert match, "_saveToolState(changes) not found in static/js/admin.js"
body = match.group(1)
invalidate = body.find("invalidateTools()")
refresh = body.find("getTools()")
post = body.find("fetch('/api/tools'")
assert -1 not in (invalidate, refresh, post)
assert invalidate < refresh < post, (
"Admin Tools must invalidate and refresh authoritative tool state before "
"posting the endpoint's full disabled-tools replacement list"
)
assert "for (const change of changes)" in body
def test_the_admin_tools_editor_does_not_read_a_cached_snapshot():
"""Pin the invalidate-before-read in loadBuiltinTools().
A source scan because the failure is an ordering in a call site, not
behaviour of the cache: getTools() is doing exactly its job either way.
"""
source = (_REPO / "static" / "js" / "admin.js").read_text(encoding="utf-8")
match = re.search(r"\nasync function loadBuiltinTools\(\) \{\n(.*?)\n\}\n", source, re.S)
assert match, "loadBuiltinTools() not found in static/js/admin.js"
body = match.group(1)
read = body.find("getTools(")
assert read != -1, "loadBuiltinTools() no longer reads the shared tool cache"
assert "invalidateTools(" in body[:read], (
"loadBuiltinTools() reads the shared /api/tools snapshot without dropping "
"it first, so a reopened panel can render tool state that changed out of "
"band and re-post it on the next unrelated toggle"
)
def test_appconfig_is_precached_by_the_service_worker():
"""PRECACHE is hand-maintained; a module missing from it breaks offline."""
sw = (_REPO / "static" / "sw.js").read_text(encoding="utf-8")
assert "'/static/js/appConfig.js'" in sw
+100
View File
@@ -0,0 +1,100 @@
"""Regression coverage for the assistant turn an approved-action replay appends.
Anthropic's Messages API rejects a non-final assistant message whose content is
empty, so the resumed turn after a tool approval used to fail before the model
ever saw the sealed result. The replay injects its result with no assistant
prose for that round, which is the only path that produced such a turn.
"""
import src.llm_core as llm_core
from src.agent_loop import _append_tool_results
_RESULT = "bash: ok\nhello"
_RECORD = {
"tool_name": "bash",
"content": "printf hello",
"result": {"output": "hello", "exit_code": 0},
"text": _RESULT,
}
def _replay_messages(round_response="", round_reasoning=""):
"""Mirror the approved-action injection in stream_agent_loop."""
messages = [
{"role": "system", "content": "system preface"},
{"role": "user", "content": "run the command and summarise it"},
]
_append_tool_results(
messages,
round_response,
[],
[_RESULT],
[_RESULT],
False,
0,
round_reasoning=round_reasoning,
tool_result_records=[_RECORD],
)
return messages
def _empty_assistant_turns(messages):
return [
index
for index, message in enumerate(messages)
if message.get("role") == "assistant"
and not str(message.get("content") or "").strip()
and not message.get("tool_calls")
]
def test_replay_appends_no_empty_assistant_turn():
assert _empty_assistant_turns(_replay_messages()) == []
def test_replay_payload_is_a_single_non_empty_user_turn_for_anthropic():
# Asserting the whole sequence rather than scanning a slice: once the empty
# spacer is gone the payload is one message, so a "no offenders in
# chat[:-1]" check would pass without inspecting anything.
sanitized = llm_core._sanitize_llm_messages(_replay_messages())
payload = llm_core._build_anthropic_payload(
"claude-sonnet-5", sanitized, 0.2, 512
)
chat = payload["messages"]
assert [message["role"] for message in chat] == ["user"]
assert all(str(message.get("content") or "").strip() for message in chat)
def test_replay_merges_tool_output_after_the_request_with_its_fence_intact():
# Dropping the empty assistant turn leaves two adjacent user messages, which
# the sanitizer merges. Pin that shape: the tool output must still sit
# behind its untrusted fence and must not precede the operator's request.
sanitized = llm_core._sanitize_llm_messages(_replay_messages())
user_turns = [m for m in sanitized if m.get("role") == "user"]
assert len(user_turns) == 1
merged = user_turns[0]["content"]
assert merged.index("run the command") < merged.index("UNTRUSTED SOURCE DATA")
assert merged.index("UNTRUSTED SOURCE DATA") < merged.index("hello")
def test_a_round_with_prose_still_appends_its_assistant_turn():
messages = _replay_messages(round_response="running that now")
assistants = [m for m in messages if m.get("role") == "assistant"]
assert [m["content"] for m in assistants] == ["running that now"]
def test_reasoning_only_round_keeps_its_carrier_and_its_known_empty_content():
"""Characterises the one empty-content turn this change deliberately leaves.
A round with reasoning and no prose still appends its carrier, and that
carrier's content is still "", which Anthropic would still reject. Dropping
it would lose the reasoning DeepSeek thinking mode requires on the next
request, and the approval replay never takes this path because it passes no
reasoning. Left alone on purpose; this test makes the gap visible instead of
silent, and should be updated by whoever closes it.
"""
messages = _replay_messages(round_reasoning="thinking about it")
assistants = [m for m in messages if m.get("role") == "assistant"]
assert len(assistants) == 1
assert assistants[0].get("reasoning_content") == "thinking about it"
assert assistants[0]["content"] == ""
+6 -1
View File
@@ -99,7 +99,12 @@ def test_login_offloads_bcrypt_bearing_calls(monkeypatch):
login = _login_endpoint(auth)
request = SimpleNamespace(client=SimpleNamespace(host="203.0.113.7"), cookies={})
request = SimpleNamespace(
client=SimpleNamespace(host="203.0.113.7"),
cookies={},
url=SimpleNamespace(scheme="http"),
headers={},
)
response = MagicMock()
body = LoginRequest(username="alice", password="hunter2", remember=True)
+93 -2
View File
@@ -242,12 +242,26 @@ def _login_endpoint(auth_manager):
raise AssertionError("login route not found")
def _login_request(scheme="http", forwarded_proto=None):
"""Stand-in for fastapi.Request carrying the fields login reads: the
client host (rate limiter), and the URL scheme plus `X-Forwarded-Proto`
(cookie Secure flag)."""
headers = {}
if forwarded_proto is not None:
headers["x-forwarded-proto"] = forwarded_proto
return SimpleNamespace(
client=SimpleNamespace(host="127.0.0.1"),
url=SimpleNamespace(scheme=scheme),
headers=headers,
)
def test_remember_cookie_max_age_matches_token_ttl(tmp_path):
auth_mod = _auth_module()
mgr = _make_manager(tmp_path)
mgr.create_user("alice", "alice-password", is_admin=False)
endpoint, LoginRequest = _login_endpoint(mgr)
request = SimpleNamespace(client=SimpleNamespace(host="127.0.0.1"))
request = _login_request()
response = _CapturingResponse()
body = LoginRequest(username="alice", password="alice-password", remember=True)
@@ -262,7 +276,7 @@ def test_no_remember_omits_cookie_max_age(tmp_path):
mgr = _make_manager(tmp_path)
mgr.create_user("bob", "bob-password", is_admin=False)
endpoint, LoginRequest = _login_endpoint(mgr)
request = SimpleNamespace(client=SimpleNamespace(host="127.0.0.1"))
request = _login_request()
response = _CapturingResponse()
body = LoginRequest(username="bob", password="bob-password", remember=False)
@@ -270,3 +284,80 @@ def test_no_remember_omits_cookie_max_age(tmp_path):
# Without "remember", the cookie is a session cookie (no max_age).
assert "max_age" not in response.cookie_kwargs
# ── Session cookie Secure flag ─────────────────────────────────────────
def _login_secure_flag(tmp_path, scheme, forwarded_proto=None):
"""Log a user in over ``scheme`` and return the cookie's Secure flag."""
mgr = _make_manager(tmp_path)
mgr.create_user("carol", "carol-password", is_admin=False)
endpoint, LoginRequest = _login_endpoint(mgr)
response = _CapturingResponse()
body = LoginRequest(username="carol", password="carol-password")
request = _login_request(scheme, forwarded_proto)
asyncio.run(endpoint(body=body, request=request, response=response))
return response.cookie_kwargs["secure"]
def test_https_login_marks_cookie_secure_without_config(tmp_path, monkeypatch):
monkeypatch.delenv("SECURE_COOKIES", raising=False)
# A session token handed out over HTTPS must not be allowed to travel
# back in cleartext just because nobody set an env var.
assert _login_secure_flag(tmp_path, "https") is True
def test_plain_http_login_leaves_cookie_insecure_without_config(tmp_path, monkeypatch):
monkeypatch.delenv("SECURE_COOKIES", raising=False)
# Marking it Secure here would make the browser drop the cookie and
# break login on a plain-HTTP install.
assert _login_secure_flag(tmp_path, "http") is False
def test_forwarded_proto_https_marks_cookie_secure(tmp_path, monkeypatch):
monkeypatch.delenv("SECURE_COOKIES", raising=False)
# A terminator that is not on an address uvicorn trusts leaves the
# connection scheme as http, so the header is the only signal there.
assert _login_secure_flag(tmp_path, "http", forwarded_proto="https") is True
# Chained proxies send a list; the client-facing hop is the first entry.
assert _login_secure_flag(tmp_path, "http", forwarded_proto="https, http") is True
def test_forwarded_proto_http_leaves_plain_http_login_insecure(tmp_path, monkeypatch):
monkeypatch.delenv("SECURE_COOKIES", raising=False)
# The header is not a downgrade switch: either signal saying https is
# enough, the same test core/middleware.py applies before sending HSTS.
assert _login_secure_flag(tmp_path, "http", forwarded_proto="http") is False
assert _login_secure_flag(tmp_path, "https", forwarded_proto="http") is True
def test_empty_secure_cookies_still_derives_from_scheme(tmp_path, monkeypatch):
# docker-compose injects `SECURE_COOKIES=${SECURE_COOKIES:-}`, which sets
# the variable to "" when the host has not defined it. Empty means
# unconfigured, not "off".
monkeypatch.setenv("SECURE_COOKIES", "")
assert _login_secure_flag(tmp_path, "https") is True
def test_secure_cookies_true_forces_secure_on_plain_http(tmp_path, monkeypatch):
# The documented knob keeps working for a proxy whose scheme the app
# cannot see, e.g. one that is not on a trusted loopback address.
monkeypatch.setenv("SECURE_COOKIES", "true")
assert _login_secure_flag(tmp_path, "http") is True
def test_secure_cookies_false_forces_insecure_on_https(tmp_path, monkeypatch):
# The escape hatch for an install still answering on both HTTP and
# HTTPS: an explicit false wins over the request scheme.
monkeypatch.setenv("SECURE_COOKIES", "false")
assert _login_secure_flag(tmp_path, "https") is False
+1 -1
View File
@@ -16,7 +16,7 @@ def test_registered_manual_compaction_uses_session_owner_for_utility_endpoint():
def test_task_name_generation_uses_owner_scoped_session_endpoint():
src = _src("routes/task_routes.py")
src = _src("routes/task/task_routes.py")
assert "async def _generate_task_name(prompt: str, owner: Optional[str] = None)" in src
assert "q = q.filter(DbSession.owner == owner)" in src
@@ -0,0 +1,177 @@
"""Automatic chat URL fetch failures must remain visible to the model."""
import logging
from types import SimpleNamespace
import pytest
import src.chat_processor as chat_processor
def _processor():
return chat_processor.ChatProcessor(
memory_manager=SimpleNamespace(load=lambda owner=None: []),
personal_docs_manager=SimpleNamespace(rag_manager=None),
skills_manager=None,
)
def test_failed_url_prefetch_keeps_diagnostics_out_of_model_context(monkeypatch):
monkeypatch.setattr(
chat_processor,
"fetch_webpage_content",
lambda url: {
"success": False,
"error": "NetworkError: connection refused\nattacker supplied detail",
},
)
preface, _, _ = _processor().build_context_preface(
message=(
"Please summarize "
"https://example.invalid/report?token=LEAK_URL_MARKER"
),
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
failure = next(
message
for message in preface
if (message.get("metadata") or {}).get("source")
== "web page fetch failure"
)
assert failure["metadata"]["trusted"] is False
assert "was not read: the page was unavailable" in failure["content"]
assert "connection refused" not in failure["content"]
assert "attacker supplied detail" not in failure["content"]
assert "LEAK_URL_MARKER" not in failure["content"]
assert "LEAK_URL_MARKER" not in str(failure["metadata"])
def test_failed_url_prefetch_exposes_only_stable_http_status(monkeypatch):
monkeypatch.setattr(
chat_processor,
"fetch_webpage_content",
lambda url: {
"success": False,
"error": "HTTP 403: attacker-controlled response detail",
},
)
preface, _, _ = _processor().build_context_preface(
message="Read https://example.test/private",
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
content = preface[-1]["content"]
assert "server returned HTTP 403" in content
assert "attacker-controlled" not in content
@pytest.mark.parametrize(
("error", "expected_category"),
[
(
"TooLarge: response exceeded configured limit; attacker-controlled detail",
"response exceeded the fetch size limit",
),
(
"Rate limit exceeded: attacker-controlled detail",
"request was rate limited",
),
],
)
def test_failed_url_prefetch_exposes_only_stable_failure_category(
monkeypatch, error, expected_category
):
monkeypatch.setattr(
chat_processor,
"fetch_webpage_content",
lambda url: {"success": False, "error": error},
)
preface, _, _ = _processor().build_context_preface(
message="Read https://example.test/report",
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
content = preface[-1]["content"]
assert expected_category in content
assert "attacker-controlled" not in content
def test_unexpected_url_prefetch_exception_does_not_abort_context(monkeypatch):
def fail(url):
raise RuntimeError("socket detail must stay in logs")
monkeypatch.setattr(chat_processor, "fetch_webpage_content", fail)
preface, _, _ = _processor().build_context_preface(
message="Review https://example.test/report",
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
content = preface[-1]["content"]
assert "was not read: the page was unavailable" in content
assert "socket detail" not in content
def test_unexpected_url_prefetch_exception_logs_only_stable_diagnostic(
monkeypatch, caplog
):
def fail(url):
raise RuntimeError("transport detail LEAK_EXCEPTION_MARKER")
monkeypatch.setattr(chat_processor, "fetch_webpage_content", fail)
with caplog.at_level(logging.WARNING, logger=chat_processor.logger.name):
_processor().build_context_preface(
message=(
"Review https://example.test/report?token=LEAK_URL_MARKER"
),
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
assert "Automatic URL fetch failed while building context" in caplog.text
assert "LEAK_URL_MARKER" not in caplog.text
assert "LEAK_EXCEPTION_MARKER" not in caplog.text
def test_successful_url_prefetch_keeps_existing_content_shape(monkeypatch):
monkeypatch.setattr(
chat_processor,
"fetch_webpage_content",
lambda url: {"success": True, "content": "page body"},
)
preface, _, _ = _processor().build_context_preface(
message="Read https://example.test/page",
session=SimpleNamespace(endpoint_url="", model="", headers={}),
use_web=False,
use_rag=False,
use_memory=False,
)
page = next(
message
for message in preface
if (message.get("metadata") or {}).get("source")
== "web page: https://example.test/page"
)
assert page["metadata"]["trusted"] is False
assert "page body" in page["content"]
+65
View File
@@ -16,6 +16,7 @@ from routes.cookbook_helpers import (
_llama_cpp_rebuild_cmd,
_append_vllm_linux_preflight_lines,
_local_tooling_path_export,
_local_windows_bash_env_prefix,
_pip_install_attempt,
_pip_install_fallback_chain,
_ollama_bind_from_cmd,
@@ -107,6 +108,70 @@ def test_safe_env_prefix_accepts_powershell_activation_path():
)
@pytest.mark.parametrize(
("prefix", "expected"),
[
("& 'C:\\Users\\me\\venv\\Scripts\\Activate.ps1'", "source /c/Users/me/venv/Scripts/activate"),
(r"& C:\Users\me\venv\Scripts\Activate.ps1", "source /c/Users/me/venv/Scripts/activate"),
(
r"& C:\Users\me\My Envs\venv\Scripts\Activate.ps1",
"source '/c/Users/me/My Envs/venv/Scripts/activate'",
),
(
"& 'C:\\Users\\me\\My Envs\\venv\\Scripts\\Activate.ps1'",
"source '/c/Users/me/My Envs/venv/Scripts/activate'",
),
(r"& D:/Envs/venv/Scripts/Activate.ps1", "source /d/Envs/venv/Scripts/activate"),
],
)
def test_local_windows_bash_env_prefix_converts_powershell_venv_activation(prefix, expected):
converted = _local_windows_bash_env_prefix(prefix)
assert converted == expected
assert _safe_env_prefix(converted).startswith('[ -f "')
@pytest.mark.parametrize(
"prefix",
[
None,
"",
"source /home/me/venv/bin/activate",
"conda activate qwen35",
'eval "$(conda shell.bash hook)" && conda activate qwen35',
r"& \\server\share\venv\Scripts\Activate.ps1",
],
)
def test_local_windows_bash_env_prefix_leaves_other_prefixes_unchanged(prefix):
assert _local_windows_bash_env_prefix(prefix) == prefix
def test_local_windows_bash_env_prefix_handles_long_whitespace_input():
prefix = "\t" * 100_000
assert _local_windows_bash_env_prefix(prefix) == prefix
@pytest.mark.parametrize(
"relative_path",
["static/js/cookbookRunning.js", "static/js/cookbookDownload.js"],
)
def test_primary_windows_venv_emitters_quote_activation_path(relative_path):
source = (Path(__file__).resolve().parents[1] / relative_path).read_text(encoding="utf-8")
assert "'& ' + _psQuote(" in source
assert "_psQuote = shared._psQuote;" in source
def test_windows_venv_conversion_stays_scoped_to_local_git_bash_runners():
source = (Path(__file__).resolve().parents[1] / "routes/cookbook_routes.py").read_text(encoding="utf-8")
guarded_conversion = (
"_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix"
)
assert source.count(guarded_conversion) == 2
def test_validate_local_dir_accepts_external_drive_paths_with_spaces():
path = "/Volumes/T7 2TB/AI Models/llamacpp"
+70 -1
View File
@@ -9,11 +9,18 @@ class _FakeRag:
def search(self, query, k=5):
return [
{"text": "alpha", "source": "a.txt", "score": 0.9},
{
"document": "alpha",
"metadata": {"source": "a.txt"},
"similarity": 0.9,
},
"corrupt-row",
None,
]
def index_personal_documents(self, directory):
return {"indexed_count": 7, "failed_count": 2, "errors": ["bad.pdf"]}
def test_query_skips_non_dict_rag_rows():
# Bypass __init__ (it builds a real RAGManager / Chroma client) and inject
@@ -24,3 +31,65 @@ def test_query_skips_non_dict_rag_rows():
# old code called r.get(...) on the str/None rows and raised AttributeError.
assert [c.text for c in out] == ["alpha"]
assert out[0].source == "a.txt"
assert out[0].score == 0.9
def test_index_maps_live_vectorrag_result_shape():
svc = DocsService.__new__(DocsService)
svc.rag = _FakeRag()
out = asyncio.run(svc.index("/documents"))
assert out.indexed == 7
assert out.failed == 2
assert out.errors == ["bad.pdf"]
def test_query_normalizes_null_canonical_fields_and_malformed_metadata():
class _MalformedRag:
def search(self, query, k=5):
return [
{
"document": None,
"text": "legacy text",
"source": None,
"similarity": None,
"score": 0.4,
"metadata": "not-a-dict",
},
{
"document": "canonical zero",
"similarity": 0.0,
"metadata": {"source": "nested.txt"},
},
{
"document": None,
"text": None,
"content": "content fallback",
"similarity": 0.2,
"metadata": ["unexpected"],
},
]
svc = DocsService.__new__(DocsService)
svc.rag = _MalformedRag()
out = asyncio.run(svc.query("query"))
assert [chunk.text for chunk in out] == [
"legacy text",
"canonical zero",
"content fallback",
]
assert out[0].source == "unknown"
assert out[0].score == 0.4
assert out[0].metadata == {}
assert out[1].source == "nested.txt"
assert out[1].score == 0.0
assert out[1].metadata == {"source": "nested.txt"}
assert out[2].source == "unknown"
assert out[2].score == 0.2
assert out[2].metadata == {}
+1 -1
View File
@@ -41,7 +41,7 @@ def test_no_hardcoded_loopback_left_in_call_sites():
"src/tools/_common.py",
"src/cookbook_serve_lifecycle.py",
"src/builtin_actions.py",
"routes/task_routes.py",
"routes/task/task_routes.py",
):
text = (root / rel).read_text(encoding="utf-8")
# Allow it only inside comments; flag any code occurrence.
+105
View File
@@ -6,10 +6,12 @@ import subprocess
from pathlib import Path
import pytest
import yaml
_REPO = Path(__file__).resolve().parent.parent
_CHECKER = _REPO / ".github" / "scripts" / "check-issue-description.js"
_BUG_TEMPLATE = _REPO / ".github" / "ISSUE_TEMPLATE" / "bug_report.yml"
_WORKFLOW = _REPO / ".github" / "workflows" / "issue-description-check.yml"
pytestmark = pytest.mark.skipif(not shutil.which("node"), reason="node not on PATH")
@@ -66,11 +68,114 @@ checkIssueDescription({ github, context, core })
return json.loads(proc.stdout)
def _run_bug_issue(revision):
body = f"""# Odysseus Revision
{revision}
# Install Method
Docker
# Operating System
Linux
# Steps to Reproduce
1. Start Odysseus.
# Expected Behaviour
The application starts normally.
# Actual Behaviour
The application exits unexpectedly.
"""
harness = r"""
const checkIssueDescription = require(process.argv[1]);
const body = process.argv[2];
const calls = [];
const github = {
rest: {
issues: {
removeLabel: async (params) => calls.push({ method: 'removeLabel', params }),
getLabel: async () => ({}),
addLabels: async (params) => calls.push({ method: 'addLabels', params }),
listComments: async () => ({ data: [] }),
createComment: async (params) => calls.push({ method: 'createComment', params }),
updateComment: async (params) => calls.push({ method: 'updateComment', params }),
deleteComment: async (params) => calls.push({ method: 'deleteComment', params }),
},
},
};
const context = {
payload: {
action: 'opened',
issue: { number: 42, state: 'open', body, labels: [{ name: 'bug' }] },
},
repo: { owner: 'odysseus-dev', repo: 'odysseus' },
};
const core = {
warning: (message) => calls.push({ method: 'warning', message }),
setFailed: (message) => calls.push({ method: 'setFailed', message }),
};
checkIssueDescription({ github, context, core })
.then(() => process.stdout.write(JSON.stringify(calls)))
.catch((error) => {
console.error(error);
process.exitCode = 1;
});
"""
proc = subprocess.run(
["node", "-e", harness, str(_CHECKER), body],
capture_output=True,
text=True,
cwd=str(_REPO),
timeout=30,
)
assert proc.returncode == 0, proc.stderr
return json.loads(proc.stdout)
def test_workflow_handles_issue_closures():
workflow = _WORKFLOW.read_text()
assert "types: [opened, edited, reopened, closed]" in workflow
def test_bug_template_requires_exact_revision():
template = yaml.safe_load(_BUG_TEMPLATE.read_text())
revision = next(item for item in template["body"] if item.get("id") == "revision")
assert revision["type"] == "input"
assert revision["attributes"]["label"] == "Odysseus Revision"
assert "git show -s --abbrev=12 --format='%h (%cs)' HEAD" in revision["attributes"]["description"]
assert revision["attributes"]["placeholder"] == "1fef4929cf1d (2026-08-11)"
assert revision["validations"]["required"] is True
def test_bug_checker_accepts_exact_revision():
calls = _run_bug_issue("1fef4929cf1d (2026-08-11)")
assert not any(call["method"] in {"createComment", "setFailed"} for call in calls)
assert any(
call["method"] == "addLabels" and call["params"]["labels"] == ["ready for review"]
for call in calls
)
@pytest.mark.parametrize(
"revision",
[
"",
"1fef492",
"1fef4929cf1d",
"1fef4929cf1d (11 August 2026)",
"1fef4929cf1d (2026-08-11) extra",
],
)
def test_bug_checker_rejects_missing_or_malformed_revision(revision):
calls = _run_bug_issue(revision)
comment = next(call["params"]["body"] for call in calls if call["method"] == "createComment")
assert "**Odysseus Revision**" in comment
assert any(call["method"] == "setFailed" for call in calls)
@pytest.mark.parametrize("action", ["closed", "edited"])
def test_closed_issue_only_drops_ready_for_review(action):
assert _run_closed_issue(action) == [
@@ -0,0 +1,40 @@
from pathlib import Path
import pytest
from packaging.requirements import Requirement
ROOT = Path(__file__).resolve().parents[1]
def _optional_requirement(name):
lines = (ROOT / "requirements-optional.txt").read_text(encoding="utf-8").splitlines()
return Requirement(next(line for line in lines if line.startswith(name)))
def test_kokoro_optional_dependency_pins_the_verified_release():
kokoro = _optional_requirement("kokoro")
assert str(kokoro.specifier) == "==0.9.4"
@pytest.mark.parametrize(
("python_version", "selected"),
(("3.11", True), ("3.12", True), ("3.13", False), ("3.14", False)),
)
def test_kokoro_feature_markers_match_supported_python_range(python_version, selected):
for name in ("kokoro", "soundfile"):
requirement = _optional_requirement(name)
assert requirement.marker is not None
assert requirement.marker.evaluate({"python_version": python_version}) is selected
def test_setup_documents_container_constraint_and_install_command():
setup = (ROOT / "docs" / "setup.md").read_text(encoding="utf-8")
assert "pip install -r requirements-optional.txt" in setup
assert "default Docker image currently uses Python 3.14" in setup
assert "Python `>=3.10,<3.13`" in setup
assert "native Python 3.11 or 3.12" in setup
assert "skipped on Python 3.13+" in setup
assert "torch build has CUDA" in setup
+510
View File
@@ -0,0 +1,510 @@
"""KaTeX and Mermaid must be vendored and fetched only on first real use.
They used to load from cdn.jsdelivr.net in every <head>, costing ~985 KB on the
wire per page load, breaking offline installs and announcing each session to a
third party. These tests pin the replacement contract: one fetch per library,
never before a formula or a ```mermaid fence actually shows up, and math still
renders once the library lands.
"""
import json
import re
import shutil
import subprocess
import textwrap
from pathlib import Path
import pytest
_REPO = Path(__file__).resolve().parent.parent
_HAS_NODE = shutil.which("node") is not None
MERMAID_SRC = "/static/lib/mermaid.min.js"
KATEX_SRC = "/static/lib/katex/katex.min.js"
KATEX_CSS = "/static/lib/katex/katex.min.css"
@pytest.fixture(scope="module")
def node_available():
if not _HAS_NODE:
pytest.skip("node binary not on PATH")
def _katex_fonts_block(sw_source: str) -> str:
"""The literal body of sw.js's KATEX_FONTS array."""
match = re.search(r"const KATEX_FONTS = \[(.*?)\]", sw_source, re.S)
assert match, "sw.js no longer defines a KATEX_FONTS array"
return match.group(1)
# A DOM stub small enough to reason about: it records every <script>/<link> the
# module injects and lets the test decide when each one "loads", which is the
# only way to observe that a second call reuses the first fetch.
_HARNESS = r"""
import fs from 'node:fs';
import vm from 'node:vm';
// The vendored KaTeX build itself, not a stand-in. A fake renderer that echoes
// its input cannot tell "a < b" from "a &lt; b" — real KaTeX reads the "&" as
// an alignment marker and returns a .katex-error span, which is the whole
// point of the entity tests below. renderToString needs no DOM, so a bare vm
// context is enough and keeps the library off the harness globals until a test
// installs it deliberately.
function loadRealKatex() {
const context = { console };
context.window = context;
context.self = context;
context.globalThis = context;
vm.createContext(context);
vm.runInContext(fs.readFileSync('./static/lib/katex/katex.min.js', 'utf8'), context);
if (!context.katex) throw new Error('vendored katex.min.js did not define a katex global');
return context.katex;
}
const injected = { scripts: [], links: [] };
function makeEl(tag) {
return {
tagName: String(tag).toUpperCase(),
_listeners: {},
classList: { remove() {} },
addEventListener(type, fn) { (this._listeners[type] ||= []).push(fn); },
fire(type) { (this._listeners[type] || []).forEach((fn) => fn()); },
};
}
function makeTemplate() {
return {
_html: '',
content: { querySelectorAll() { return []; } },
set innerHTML(value) { this._html = value; },
get innerHTML() { return this._html; },
};
}
globalThis.window = { location: { origin: 'http://localhost' }, katex: null, mermaid: null };
globalThis.document = {
readyState: 'complete',
addEventListener() {},
head: {
appendChild(el) {
if (el.tagName === 'SCRIPT') injected.scripts.push(el);
else if (el.tagName === 'LINK') injected.links.push(el);
return el;
},
},
createElement(tag) {
if (tag === 'template') return makeTemplate();
return makeEl(tag);
},
querySelectorAll() { return []; },
};
globalThis.MutationObserver = class { observe() {} };
let source = fs.readFileSync('./static/js/markdown.js', 'utf8');
source = source.replace(/import uiModule from ['"]\.\/ui\.js['"];/, '');
source = source.replace(
/import \{ splitTableRow \} from ['"]\.\/markdown\/tableRow\.js['"];/,
`function splitTableRow(row) {
return (row || '').replace(/^\s*\|/, '').replace(/\|\s*$/, '').split('|').map(c => c.trim());
}`
);
const emojiSource = fs.readFileSync('./static/js/emojiShortcodes.js', 'utf8')
.replace(/^export default .*$/m, '')
.replace(/export const /g, 'const ')
.replace(/export function /g, 'function ');
source = source.replace(
/import \{ replaceEmojiShortcodes, hasEmojiShortcode \} from ['"]\.\/emojiShortcodes\.js['"];/,
() => emojiSource
);
source = source.replace(
/var escapeHtml = uiModule\.esc;/,
`var escapeHtml = (value) => String(value ?? '')
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');`
);
const moduleUrl = 'data:text/javascript;base64,' + Buffer.from(source).toString('base64');
const mod = await import(moduleUrl);
// A container whose querySelectorAll answers from a fixed element list, so a
// test can hand the renderer exactly the nodes it wants it to see.
function makeContainer(elements) {
return {
querySelectorAll(selector) {
return (elements[selector] || []).slice();
},
};
}
const emit = (value) => console.log(JSON.stringify(value));
"""
def _run_node(body: str, timeout: int = 20):
script = _HARNESS + textwrap.dedent(body)
result = subprocess.run(
["node", "--input-type=module", "-e", script],
cwd=_REPO,
capture_output=True,
timeout=timeout,
text=True,
)
if result.returncode != 0:
raise AssertionError(f"node failed:\nSTDERR:\n{result.stderr}\nSTDOUT:\n{result.stdout}")
return json.loads(result.stdout.splitlines()[-1])
def test_ensure_mermaid_shares_one_load_between_concurrent_callers(node_available):
"""Two callers before the library lands must not trigger two fetches."""
out = _run_node(
"""
const p1 = mod.ensureMermaid();
const p2 = mod.ensureMermaid();
const samePromise = p1 === p2;
const srcs = injected.scripts.map((s) => s.src);
let initializeCalls = 0;
globalThis.window.mermaid = {
initialize() { initializeCalls++; },
run() {},
};
injected.scripts[0].fire('load');
const [a, b] = await Promise.all([p1, p2]);
emit({
samePromise,
scriptCount: injected.scripts.length,
srcs,
sameLibrary: a === b && a === globalThis.window.mermaid,
initializeCalls,
});
"""
)
assert out["samePromise"] is True
assert out["scriptCount"] == 1
assert out["srcs"] == [MERMAID_SRC]
assert out["sameLibrary"] is True
assert out["initializeCalls"] == 1
def test_ensure_mermaid_retries_after_a_failed_load(node_available):
"""A blocked first fetch must not poison every later diagram."""
out = _run_node(
"""
const first = mod.ensureMermaid();
injected.scripts[0].fire('error');
let firstError = null;
try { await first; } catch (e) { firstError = e.message; }
const second = mod.ensureMermaid();
const retried = injected.scripts.length === 2;
globalThis.window.mermaid = { initialize() {}, run() {} };
injected.scripts[1].fire('load');
await second;
emit({ firstError, retried, differentPromise: first !== second });
"""
)
assert MERMAID_SRC in out["firstError"]
assert out["retried"] is True
assert out["differentPromise"] is True
def test_render_mermaid_does_not_fetch_when_no_diagram_is_present(node_available):
"""The whole point of the lazy load: no fence, no 3.5 MB download."""
out = _run_node(
"""
const container = makeContainer({});
await mod.renderMermaid(container);
emit({ scriptCount: injected.scripts.length });
"""
)
assert out["scriptCount"] == 0
def test_render_mermaid_fetches_once_a_diagram_is_present(node_available):
out = _run_node(
"""
const node = makeEl('pre');
node.isConnected = true;
const container = makeContainer({ 'pre.mermaid:not([data-processed])': [node] });
const pending = mod.renderMermaid(container);
const srcs = injected.scripts.map((s) => s.src);
let ranWith = null;
globalThis.window.mermaid = {
initialize() {},
run(opts) { ranWith = opts.nodes.length; },
};
injected.scripts[0].fire('load');
await pending;
emit({ srcs, ranWith });
"""
)
assert out["srcs"] == [MERMAID_SRC]
assert out["ranWith"] == 1
def test_ensure_katex_loads_script_and_stylesheet_once(node_available):
out = _run_node(
"""
const p1 = mod.ensureKatex();
const p2 = mod.ensureKatex();
const samePromise = p1 === p2;
const scriptSrcs = injected.scripts.map((s) => s.src);
const linkHrefs = injected.links.map((l) => l.href);
globalThis.window.katex = { renderToString: (src) => src };
injected.scripts[0].fire('load');
injected.links[0].fire('load');
await Promise.all([p1, p2]);
emit({ samePromise, scriptSrcs, linkHrefs });
"""
)
assert out["samePromise"] is True
assert out["scriptSrcs"] == [KATEX_SRC]
assert out["linkHrefs"] == [KATEX_CSS]
def test_md_to_html_defers_math_when_katex_is_not_loaded_yet(node_available):
"""Without KaTeX the source is banked verbatim, not dropped or mangled."""
out = _run_node(
"""
const html = mod.mdToHtml('Inline $x^2 + y_1$ and\\n\\n$$\\\\frac{a}{b}$$\\n');
emit({ html, scriptCount: injected.scripts.length });
"""
)
html = out["html"]
assert 'class="ody-math-pending" data-display="false"' in html
assert 'class="ody-math-pending" data-display="true"' in html
# The raw source survives the escaping passes — `y_1` must not become <em>.
assert "x^2 + y_1" in html
assert "<em>" not in html
# Nothing is fetched during the synchronous render itself.
assert out["scriptCount"] == 0
def test_deferred_math_schedules_a_katex_load(node_available):
"""Deferring is only safe if the follow-up actually fires.
An earlier version scheduled this on requestAnimationFrame, which never runs
in a headless browser and is throttled to a stop in a background tab — math
then sat as plain source text until the tab was focused.
"""
out = _run_node(
"""
mod.mdToHtml('Inline $x^2$ here.');
const duringRender = injected.scripts.length;
await new Promise((r) => setTimeout(r, 0));
emit({ duringRender, scriptSrcs: injected.scripts.map((s) => s.src) });
"""
)
assert out["duringRender"] == 0, "the synchronous render must not block on a fetch"
assert out["scriptSrcs"] == [KATEX_SRC]
def test_entity_math_reaches_katex_as_characters_not_entities(node_available):
""""$a &lt; b$" and "$a < b$" must typeset the same, with no parse error.
mdToHtml escapes the source before the math pass, so a typed "<" arrives at
the delimiters as "&lt;" and a typed "&lt;" arrives as "&amp;lt;". KaTeX
has no entity syntax and treats the "&" as an alignment marker, so anything
still spelled as an entity comes back as a red .katex-error instead of a
formula. Both spellings have to be decoded to the character itself, in one
pass — decoding "&amp;" first and "&lt;" after would let the second pass eat
what the first produced, which is the double-unescape CodeQL flags.
"""
out = _run_node(
"""
const katex = loadRealKatex();
globalThis.window.katex = katex;
globalThis.katex = katex;
emit({
entity: mod.mdToHtml('Math: $a &lt; b$ done.'),
typed: mod.mdToHtml('Math: $a < b$ done.'),
ampersandEntity: mod.mdToHtml('Math: $x &gt; y$ done.'),
});
"""
)
assert "katex-error" not in out["entity"]
assert "katex-error" not in out["typed"]
assert "katex-error" not in out["ampersandEntity"]
# Same formula, same markup, whichever way the author spelled the operator.
assert out["entity"] == out["typed"]
assert 'class="katex"' in out["entity"]
def test_deferred_entity_math_banks_the_decoded_source(node_available):
"""The placeholder has to hold the same source the inline path would use.
renderMath() feeds the span's textContent straight to KaTeX, so an entity
left in the bank is a .katex-error that only appears on a cold page — the
exact case the lazy load made common.
"""
out = _run_node(
"""
emit({
entity: mod.mdToHtml('Math: $a &lt; b$ done.'),
typed: mod.mdToHtml('Math: $a < b$ done.'),
});
"""
)
assert 'class="ody-math-pending"' in out["entity"]
assert out["entity"] == out["typed"]
# Escaped once for transport, so the span's textContent is "a < b".
assert "a &lt; b</span>" in out["entity"]
def test_detached_container_math_typesets_with_the_real_renderer(node_available):
"""The PDF export renders into a container it never attaches to the page.
mdToHtml defers math to a document-scoped flush, which cannot reach a
detached node, so the export has to typeset its own container before
handing it to html2pdf. This is that container: pending spans in, real
KaTeX markup out, no .katex-error and nothing left pending.
"""
out = _run_node(
"""
const katex = loadRealKatex();
const html = mod.mdToHtml('Formula $E = mc^2$ here.');
const el = makeEl('span');
el.textContent = 'E = mc^2';
el.getAttribute = (name) => (name === 'data-display' ? 'false' : null);
let written = null;
Object.defineProperty(el, 'outerHTML', { set(v) { written = v; } });
const container = makeContainer({ '.ody-math-pending': [el] });
const pending = mod.renderMath(container);
globalThis.window.katex = katex;
injected.scripts[0].fire('load');
injected.links[0].fire('load');
await pending;
emit({ html, written });
"""
)
# Cold page: mdToHtml could not typeset, so the export HTML starts pending.
assert 'class="ody-math-pending"' in out["html"]
# After the export's own render pass it is real KaTeX markup.
assert 'class="katex"' in out["written"]
assert "katex-error" not in out["written"]
assert "ody-math-pending" not in out["written"]
def test_pdf_export_typesets_its_container_before_html2pdf():
"""Ordering in a call site, so pin the call site. No node needed."""
source = (_REPO / "static/js/document.js").read_text(encoding="utf-8")
match = re.search(r"\n async function exportAsPdf\(\) \{(.*?)\n \}\n", source, re.S)
assert match, "exportAsPdf not found"
body = match.group(1)
render = "await markdownModule.renderMath(container);"
assert render in body, "the export never typesets its detached container"
assert body.index("container.innerHTML = html;") < body.index(render)
assert body.index(render) < body.index("window.html2pdf()")
def test_md_to_html_renders_inline_once_katex_is_loaded(node_available):
"""After the first load mdToHtml goes back to typesetting synchronously."""
out = _run_node(
"""
globalThis.window.katex = {
renderToString: (src, opts) => `<span class="katex" data-display="${!!(opts && opts.displayMode)}">${src}</span>`,
};
globalThis.katex = globalThis.window.katex;
const html = mod.mdToHtml('Inline $x^2$ here.');
emit({ html });
"""
)
assert '<span class="katex" data-display="false">x^2</span>' in out["html"]
assert "ody-math-pending" not in out["html"]
def test_render_math_typesets_deferred_placeholders(node_available):
out = _run_node(
"""
const el = makeEl('span');
el.textContent = 'x^2';
el.getAttribute = (name) => (name === 'data-display' ? 'false' : null);
let written = null;
Object.defineProperty(el, 'outerHTML', { set(v) { written = v; } });
const container = makeContainer({ '.ody-math-pending': [el] });
const pending = mod.renderMath(container);
const scriptSrcs = injected.scripts.map((s) => s.src);
globalThis.window.katex = {
renderToString: (src, opts) => `<span class="katex" data-display="${!!(opts && opts.displayMode)}">${src}</span>`,
};
injected.scripts[0].fire('load');
injected.links[0].fire('load');
await pending;
emit({ scriptSrcs, written });
"""
)
assert out["scriptSrcs"] == [KATEX_SRC]
assert out["written"] == '<span class="katex" data-display="false">x^2</span>'
def test_render_math_does_not_fetch_without_placeholders(node_available):
out = _run_node(
"""
await mod.renderMath(makeContainer({}));
emit({ scriptCount: injected.scripts.length, linkCount: injected.links.length });
"""
)
assert out["scriptCount"] == 0
assert out["linkCount"] == 0
def test_vendored_assets_exist_and_index_html_has_no_cdn_reference():
"""Guards the offline/privacy half: no node needed, so it always runs."""
for rel in (
"static/lib/mermaid.min.js",
"static/lib/katex/katex.min.js",
"static/lib/katex/katex.min.css",
):
path = _REPO / rel
assert path.is_file(), f"{rel} is not vendored"
assert path.stat().st_size > 1024, f"{rel} looks truncated"
# KaTeX's stylesheet resolves fonts relative to itself; a missing font
# degrades silently to fallback glyphs, so resolve every woff2 the vendored
# CSS actually asks for. (.woff/.ttf are listed too but never requested by a
# browser that supports woff2, which is what static/fonts/ already assumes.)
css_dir = _REPO / "static/lib/katex"
css = (css_dir / "katex.min.css").read_text(encoding="utf-8")
wanted = sorted(set(re.findall(r"url\((fonts/KaTeX_[\w-]+\.woff2)\)", css)))
assert len(wanted) == 20, f"expected 20 woff2 references in the CSS, found {len(wanted)}"
missing = [ref for ref in wanted if not (css_dir / ref).is_file()]
assert missing == [], f"KaTeX stylesheet references fonts that are not vendored: {missing}"
# Everything the CSS needs must also survive an offline install: the font
# names have to be in KATEX_FONTS and that array has to reach PRECACHE.
sw = (_REPO / "static/sw.js").read_text(encoding="utf-8")
assert "...KATEX_FONTS," in sw, "KATEX_FONTS is defined but never spread into PRECACHE"
precached = {
f"fonts/KaTeX_{name}.woff2"
for name in re.findall(r"'([\w-]+)',", _katex_fonts_block(sw))
}
assert precached >= set(wanted), f"not precached: {sorted(set(wanted) - precached)}"
# The shell must fetch no resource from a third party. Scoped to the tags
# that actually load something — an <a href> to an external page is fine,
# and the comment explaining the move can keep naming the CDN it left.
index = (_REPO / "static/index.html").read_text(encoding="utf-8")
remote_loads = re.findall(r"<(?:script|link)\b[^>]*\b(?:src|href)=\"https?://[^\"]+", index)
assert remote_loads == [], f"index.html loads remote resources: {remote_loads}"
sw = (_REPO / "static/sw.js").read_text(encoding="utf-8")
assert KATEX_SRC in sw
assert KATEX_CSS in sw
+9
View File
@@ -30,6 +30,15 @@ def test_detect_vendor_uses_endpoint_kind_then_host_and_common_local_ports():
assert detect_vendor("http://localhost:7000/v1") == VENDOR_GENERIC_OPENAI
def test_detect_vendor_requires_a_dns_label_boundary():
assert detect_vendor("https://api.openai.com./v1") == VENDOR_OPENAI
assert detect_vendor("https://notopenai.com/v1") == VENDOR_GENERIC_OPENAI
assert detect_vendor("https://fakeopenrouter.ai/v1") == VENDOR_GENERIC_OPENAI
assert detect_vendor("https://notgoogleapis.com/v1") == VENDOR_GENERIC_OPENAI
assert detect_vendor("https://evilanthropic.com/v1") == VENDOR_GENERIC_OPENAI
assert detect_vendor("https://fakeollama.com/v1") == VENDOR_GENERIC_OPENAI
def test_generic_openai_reader_keeps_basic_model_payload_unknown():
records = generic_openai.records_from_payload(
{
+1 -1
View File
@@ -27,7 +27,7 @@ def test_calendar_quick_parse_resolves_with_owner_scope():
def test_task_parse_resolves_with_owner_scope():
body = _function_source("routes/task_routes.py", "parse_task")
body = _function_source("routes/task/task_routes.py", "parse_task")
assert "user = _owner(request)" in body
assert 'resolve_endpoint("utility", owner=user or None)' in body
assert 'resolve_endpoint("default", owner=user or None)' in body
+64 -5
View File
@@ -1107,6 +1107,47 @@ def test_get_api_models_marks_picker_as_pinned_only(monkeypatch):
assert by_id["anthropic/claude-sonnet-4"]["is_pinned"] is False
def test_get_fresh_api_models_marks_cached_inventory_as_pinned(monkeypatch):
ep = _make_endpoint(
base_url="https://api.example.test/v1",
cached_models=json.dumps(["openai/gpt-image-1", "anthropic/claude-sonnet-4"]),
pinned_models=None,
)
db = _PinnedFakeDb([ep])
monkeypatch.setattr(model_routes, "SessionLocal", lambda: db)
monkeypatch.setattr(model_routes, "require_admin", lambda request: None)
endpoint = _get_route("/api/model-endpoints/{ep_id}/models", "GET")
result = endpoint("ep1", _PinnedFakeRequest(), SimpleNamespace(headers={}))
assert [row["id"] for row in result] == [
"openai/gpt-image-1",
"anthropic/claude-sonnet-4",
]
assert all(row["picker_requires_pinning"] is True for row in result)
assert all(row["is_pinned"] is True for row in result)
def test_get_legacy_api_models_preserves_hidden_selection_as_pinned(monkeypatch):
ep = _make_endpoint(
base_url="https://api.example.test/v1",
cached_models=json.dumps(["m1", "m2", "m3"]),
hidden_models=json.dumps(["m2"]),
pinned_models=None,
)
db = _PinnedFakeDb([ep])
monkeypatch.setattr(model_routes, "SessionLocal", lambda: db)
monkeypatch.setattr(model_routes, "require_admin", lambda request: None)
endpoint = _get_route("/api/model-endpoints/{ep_id}/models", "GET")
result = endpoint("ep1", _PinnedFakeRequest(), SimpleNamespace(headers={}))
by_id = {row["id"]: row for row in result}
assert by_id["m1"]["is_pinned"] is True
assert by_id["m2"]["is_pinned"] is False
assert by_id["m3"]["is_pinned"] is True
def test_reprobe_preserves_pinned_models(monkeypatch):
ep = _make_endpoint(pinned_models=json.dumps(["deploy-1"]))
db = _PinnedFakeDb([ep])
@@ -1266,7 +1307,7 @@ def test_list_model_endpoints_returns_key_fingerprint(monkeypatch):
assert result[1]["api_key_fingerprint"] == ""
def test_list_api_endpoint_reports_inventory_count_when_none_pinned(monkeypatch):
def test_list_api_endpoint_defaults_inventory_to_visible_when_none_pinned(monkeypatch):
ep = _make_endpoint(
base_url="https://api.example.test/v1",
cached_models=json.dumps(["openai/gpt-image-1", "anthropic/claude-sonnet-4"]),
@@ -1279,12 +1320,31 @@ def test_list_api_endpoint_reports_inventory_count_when_none_pinned(monkeypatch)
result = endpoint(_PinnedFakeRequest())
assert result[0]["models"] == []
assert result[0]["models"] == [
"openai/gpt-image-1",
"anthropic/claude-sonnet-4",
]
assert result[0]["pinned_models"] == result[0]["models"]
assert result[0]["model_count"] == 2
assert result[0]["picker_requires_pinning"] is True
assert result[0]["status"] == "online"
def test_api_picker_preserves_explicit_empty_selection():
ep = _make_endpoint(
base_url="https://api.example.test/v1",
cached_models=json.dumps(["openai/gpt-image-1", "anthropic/claude-sonnet-4"]),
pinned_models=json.dumps([]),
)
models, pinned = model_routes._picker_models_for_endpoint(
ep, ep.base_url, ep.endpoint_kind
)
assert models == []
assert pinned == []
def test_list_api_endpoint_returns_pinned_picker_models(monkeypatch):
ep = _make_endpoint(
base_url="https://api.example.test/v1",
@@ -1767,7 +1827,7 @@ def test_api_models_openrouter_uses_pinned_models_not_hidden(monkeypatch):
assert result["items"][0]["models"] == ["openai/gpt-image-1"]
def test_api_models_openrouter_derives_legacy_visible_models(monkeypatch):
def test_api_models_openrouter_defaults_cached_models_visible(monkeypatch):
row = _route_ep(
"openrouter",
"https://openrouter.ai/api/v1",
@@ -1775,7 +1835,6 @@ def test_api_models_openrouter_derives_legacy_visible_models(monkeypatch):
pinned_models=None,
api_key="fake-key",
)
row.hidden_models = json.dumps(["m2"])
db = _RouteDb([row])
router = model_routes.setup_model_routes(model_discovery=None)
@@ -1788,7 +1847,7 @@ def test_api_models_openrouter_derives_legacy_visible_models(monkeypatch):
result = _route_endpoint(router, "/api/models")(_route_request())
assert result["items"][0]["endpoint_name"] == "openrouter"
assert result["items"][0]["models"] == ["m1", "m3"]
assert result["items"][0]["models"] == ["m1", "m2", "m3"]
@pytest.mark.asyncio
+201
View File
@@ -0,0 +1,201 @@
r"""Behaviour of the panel-loader registry in `static/js/panels.js`.
The registry is what keeps a lazily-loaded panel honest: it must import a
panel's module exactly once no matter how many times the user clicks, it must
fail loudly on a name nobody registered (rather than handing back `undefined`
and blowing up somewhere unrelated), and it must not memoise a *failed* load —
otherwise a panel that failed to load once while offline would stay broken for
the rest of the session.
`createPanelLoader` is exercised with stub thunks so these tests need no DOM and
no real panel module graph. The production `loadPanel` is the same function
applied to the real registry, checked here only for its registered names.
"""
import json
import os
import re
import shutil
import subprocess
import textwrap
from pathlib import Path
import pytest
_REPO = Path(__file__).resolve().parent.parent
_PANELS = (_REPO / "static" / "js" / "panels.js").as_posix()
_HAS_NODE = shutil.which("node") is not None
pytestmark = pytest.mark.skipif(not _HAS_NODE, reason="node binary not on PATH")
def _run(js: str) -> str:
proc = subprocess.run(
["node", "--input-type=module"],
input=js, capture_output=True, text=True, cwd=str(_REPO), timeout=30,
)
assert proc.returncode == 0, proc.stderr
return proc.stdout.strip()
def test_repeated_loads_import_once_and_share_one_promise():
# A double-click on a rail button calls the loader twice before the first
# import has resolved. Both calls must ride the same in-flight import.
js = textwrap.dedent(
f"""
const {{ createPanelLoader }} = await import('{_PANELS}');
let calls = 0;
const load = createPanelLoader({{
demo: () => {{ calls += 1; return Promise.resolve({{ open: () => 'opened' }}); }},
}});
const a = load('demo');
const b = load('demo');
const mod = await a;
const c = load('demo'); // after it resolved, still cached
console.log(JSON.stringify({{
calls,
sameWhilePending: a === b,
sameAfterResolve: a === c,
opened: (await c).open(),
modOpened: mod.open(),
}}));
"""
)
out = json.loads(_run(js))
assert out["calls"] == 1
assert out["sameWhilePending"] is True
assert out["sameAfterResolve"] is True
assert out["opened"] == "opened"
assert out["modOpened"] == "opened"
def test_unknown_panel_throws_instead_of_returning_undefined():
js = textwrap.dedent(
f"""
const {{ createPanelLoader }} = await import('{_PANELS}');
const load = createPanelLoader({{ demo: () => Promise.resolve({{}}) }});
let message = null, returned = 'not-reached';
try {{ returned = load('nope'); }} catch (e) {{ message = e.message; }}
console.log(JSON.stringify({{ message, returned: String(returned) }}));
"""
)
out = json.loads(_run(js))
assert out["message"] is not None
assert 'nope' in out["message"]
assert out["returned"] == "not-reached"
def test_a_failed_load_is_not_memoised_so_a_retry_can_succeed():
# First open happens offline and the import 404s; the second, back online,
# must actually retry rather than replay the cached rejection.
js = textwrap.dedent(
f"""
const {{ createPanelLoader }} = await import('{_PANELS}');
let calls = 0;
const load = createPanelLoader({{
demo: () => {{
calls += 1;
return calls === 1
? Promise.reject(new Error('offline'))
: Promise.resolve({{ open: () => 'opened' }});
}},
}});
let firstError = null;
try {{ await load('demo'); }} catch (e) {{ firstError = e.message; }}
const second = await load('demo');
console.log(JSON.stringify({{ calls, firstError, opened: second.open() }}));
"""
)
out = json.loads(_run(js))
assert out["firstError"] == "offline"
assert out["calls"] == 2
assert out["opened"] == "opened"
def test_a_thunk_that_throws_synchronously_rejects_rather_than_escaping():
js = textwrap.dedent(
f"""
const {{ createPanelLoader }} = await import('{_PANELS}');
const load = createPanelLoader({{ demo: () => {{ throw new Error('boom'); }} }});
let sync = null, rejected = null;
let p;
try {{ p = load('demo'); }} catch (e) {{ sync = e.message; }}
try {{ await p; }} catch (e) {{ rejected = e.message; }}
console.log(JSON.stringify({{ sync, rejected }}));
"""
)
out = json.loads(_run(js))
assert out["sync"] is None, "a broken thunk must not throw at call time"
assert out["rejected"] == "boom"
def test_the_image_editor_is_registered():
js = textwrap.dedent(
f"""
const {{ panelNames }} = await import('{_PANELS}');
console.log(JSON.stringify(panelNames()));
"""
)
assert "editor" in json.loads(_run(js))
# ── Offline coverage ──────────────────────────────────────────────────────
# A lazily-loaded panel is never fetched during a normal page load, so it only
# lands in the service-worker cache if sw.js precaches it explicitly. Miss one
# module and the panel opens fine online and dies offline — the failure mode is
# invisible until someone is on a plane. This walks the editor's import graph
# and pins that every file in it is listed in PANEL_PRECACHE, query string
# included (the SW matches on the full URL).
_SW = _REPO / "static" / "sw.js"
_JS_DIR = _REPO / "static" / "js"
_STATIC_IMPORT = re.compile(
r"""(?:^|\n)\s*(?:import\s+(?:[^;'"()]*?\s+from\s+)?|export\s+(?:\*|\{[^}]*\})\s+from\s+)['"]([^'"]+)['"]"""
)
def _editor_module_graph() -> set[str]:
"""Every module statically reachable from galleryEditor.js, as '<path>[?query]'."""
seen: set[str] = set()
stack = ["galleryEditor.js"]
while stack:
current = stack.pop()
if current in seen:
continue
seen.add(current)
source = (_JS_DIR / current.split("?")[0]).read_text(encoding="utf-8")
for spec in _STATIC_IMPORT.findall(source):
if not spec.startswith("."):
continue
path, _, query = spec.partition("?")
target = os.path.normpath(
os.path.join(os.path.dirname(current.split("?")[0]), path)
)
stack.append(f"{target}?{query}" if query else target)
return seen
def _panel_precache_entries() -> set[str]:
block = re.search(
r"const PANEL_PRECACHE = \[(.*?)\];", _SW.read_text(encoding="utf-8"), re.S
)
assert block, "PANEL_PRECACHE not found in static/sw.js"
return set(re.findall(r"'([^']+)'", block.group(1)))
def test_every_lazy_editor_module_is_precached_for_offline_use():
graph = _editor_module_graph()
precached = _panel_precache_entries()
# Shared modules (ui.js, spinner.js, ...) are still eagerly loaded by
# index.html, so they are cached by the normal page load. Only the part of
# the graph that nothing else pulls in needs an explicit entry.
lazy_only = {
m for m in graph
if m == "galleryEditor.js" or m.split("?")[0].startswith("editor/")
}
missing = {f"/static/js/{m}" for m in lazy_only} - precached
assert not missing, (
"these editor modules load lazily but are not in PANEL_PRECACHE, so the "
f"editor would not open offline: {sorted(missing)}"
)
+14
View File
@@ -169,6 +169,20 @@ def test_ui_checkbox_without_media_still_needs_visual_evidence():
assert not any(call["method"] == "setFailed" for call in calls)
def test_ticked_screenshot_box_counts_without_the_template_bolding():
body = _body(
app_ran=True,
screenshot=True,
media="https://github.com/user-attachments/assets/example",
).replace("**Screenshot or short clip**", "Screenshot or short clip")
calls = _run_checker(["static/js/example.js"], body)
assert _added_labels(calls) == {"ready for review"}
assert not _comment(calls)
assert not any(call["method"] == "setFailed" for call in calls)
def test_explicit_not_run_is_honest_but_not_ready():
calls = _run_checker(
["services/example.py"],
@@ -0,0 +1,62 @@
"""Real-SQLite regressions for session discovery with stale derived counts."""
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import NullPool
import core.session_manager as session_manager
def _make_manager(db_path, monkeypatch):
engine = create_engine(
f"sqlite:///{db_path}",
connect_args={"check_same_thread": False},
poolclass=NullPool,
)
session_manager.DbSession.metadata.create_all(bind=engine)
session_local = sessionmaker(
bind=engine,
autoflush=False,
autocommit=False,
)
monkeypatch.setattr(session_manager, "SessionLocal", session_local)
return session_manager.SessionManager(), session_manager
def test_discovery_uses_persisted_rows_and_repairs_cached_count(tmp_path, monkeypatch):
from core.models import ChatMessage
manager, session_manager = _make_manager(
tmp_path / "session-discovery.db", monkeypatch
)
for session_id in ("stale-low", "empty", "stale-high"):
manager.create_session(
session_id=session_id,
name=session_id,
endpoint_url="http://example.invalid",
model="test-model",
rag=False,
owner="tester",
)
manager.add_message("stale-low", ChatMessage("user", "persisted message"))
db = session_manager.SessionLocal()
try:
db.query(session_manager.DbSession).filter(
session_manager.DbSession.id == "stale-low"
).update({"message_count": 0})
db.query(session_manager.DbSession).filter(
session_manager.DbSession.id == "stale-high"
).update({"message_count": 7})
db.commit()
finally:
db.close()
restarted = session_manager.SessionManager()
assert set(restarted.sessions) == {"stale-low"}
assert restarted.sessions["stale-low"].history == []
assert restarted.sessions["stale-low"].message_count == 1
hydrated = restarted.get_session("stale-low")
assert [message.content for message in hydrated.history] == ["persisted message"]
+20
View File
@@ -0,0 +1,20 @@
"""Regression test for the task route shim (slice 2p, #4082/#4071).
The backward-compat shim at ``routes/task_routes.py`` uses ``sys.modules``
replacement so the legacy import path and the canonical ``routes.task.*``
path resolve to the *same* module object. This is required because multiple
tests do ``import routes.task_routes as task_routes`` followed by
``monkeypatch.setattr(task_routes, "SessionLocal", ...)`` /
``"get_current_user"``, and test_auth_regressions.py reads
``task_routes.__file__`` for source introspection.
"""
import importlib
import routes.task_routes as _shim_task # noqa: F401
def test_legacy_and_canonical_task_module_are_same_object():
legacy = importlib.import_module("routes.task_routes")
canonical = importlib.import_module("routes.task.task_routes")
assert legacy is canonical
+159
View File
@@ -0,0 +1,159 @@
"""Owner-scope tests for action_tidy_research.
Broken research files (empty or unparseable JSON) have no readable owner
stamp. The HTTP path and _find_owned_research_path treat parse failure as
not-owned, so a regular user's tidy task must not unlink them. Clearing a
corrupt record is a privileged act instead: admins, and the operator in
single-user mode (AUTH_ENABLED=false), keep the janitor.
Every case installs a fake AuthManager rather than relying on ambient state.
Setting AUTH_ENABLED=true alone leaves is_configured False, so the admin check
these tests exist to pin would never be reached.
"""
import json
import pytest
from src.builtin_actions import TaskNoop, action_tidy_research
@pytest.fixture
def research_dir(tmp_path, monkeypatch):
data_dir = tmp_path / "deep_research"
data_dir.mkdir()
monkeypatch.setattr("src.builtin_actions.DEEP_RESEARCH_DIR", str(data_dir))
return data_dir
def _install_auth(monkeypatch, *, is_configured: bool):
"""Auth enabled, with `root` as the only admin. `is_configured` False models
the pre-setup window, where no admin exists yet."""
import core.auth as core_auth
configured = is_configured
class FakeAuthManager:
is_configured = configured
def is_admin(self, user):
return user == "root"
monkeypatch.setenv("AUTH_ENABLED", "true")
monkeypatch.setattr(core_auth, "AuthManager", FakeAuthManager)
@pytest.fixture
def configured_auth(monkeypatch):
_install_auth(monkeypatch, is_configured=True)
def _write(data_dir, name: str, text: str):
path = data_dir / f"{name}.json"
path.write_text(text, encoding="utf-8")
return path
def _write_owned(data_dir, name: str, owner: str):
return _write(data_dir, name, json.dumps({"owner": owner, "query": name}))
@pytest.mark.asyncio
async def test_configured_non_admin_skips_unparseable_and_empty_files(
research_dir, configured_auth
):
alice = _write_owned(research_dir, "alice-ok", "alice")
bob = _write_owned(research_dir, "bob-ok", "bob")
empty = _write(research_dir, "broken-empty", "")
garbage = _write(research_dir, "broken-json", "{not json")
alice_txt = alice.read_text(encoding="utf-8")
bob_txt = bob.read_text(encoding="utf-8")
with pytest.raises(TaskNoop, match="not permitted"):
await action_tidy_research("alice")
assert alice.exists()
assert alice.read_text(encoding="utf-8") == alice_txt
assert bob.exists()
assert bob.read_text(encoding="utf-8") == bob_txt
assert empty.exists(), "empty file has no owner stamp; non-admin tidy must not unlink it"
assert garbage.exists(), "unparseable file has no owner stamp; non-admin tidy must not unlink it"
@pytest.mark.asyncio
async def test_configured_non_admin_skip_reason_does_not_claim_none_broken(
research_dir, configured_auth
):
"""The skip reason reaches Activity as `skipped -- <reason>`, so it must not
report a clean scan over files it declined to look at."""
_write(research_dir, "broken-empty", "")
with pytest.raises(TaskNoop) as excinfo:
await action_tidy_research("alice")
assert "none broken" not in str(excinfo.value)
@pytest.mark.asyncio
async def test_configured_admin_still_removes_broken_files(research_dir, configured_auth):
valid = _write_owned(research_dir, "ok", "alice")
empty = _write(research_dir, "broken-empty", " ")
garbage = _write(research_dir, "broken-json", "not-json")
message, ok = await action_tidy_research("root")
assert ok is True
assert "Removed 2" in message
assert valid.exists(), "admin sweep clears corrupt records, not valid ones"
assert not empty.exists()
assert not garbage.exists()
@pytest.mark.asyncio
async def test_pre_setup_denies_owner_who_would_be_admin_once_configured(
research_dir, monkeypatch
):
"""Auth on, no admin created yet: fail closed even for `root`.
The owner here is non-empty and would satisfy is_admin, so this fails if the
is_configured guard is removed. An empty owner would not discriminate -- it
is denied by the `owner and ...` short-circuit either way.
"""
_install_auth(monkeypatch, is_configured=False)
empty = _write(research_dir, "broken-empty", "")
garbage = _write(research_dir, "broken-json", "{")
with pytest.raises(TaskNoop, match="not permitted"):
await action_tidy_research("root")
assert empty.exists()
assert garbage.exists()
@pytest.mark.asyncio
async def test_empty_owner_does_not_delete_broken_files(research_dir, configured_auth):
empty = _write(research_dir, "broken-empty", "")
garbage = _write(research_dir, "broken-json", "{")
with pytest.raises(TaskNoop, match="not permitted"):
await action_tidy_research("")
assert empty.exists()
assert garbage.exists()
@pytest.mark.asyncio
async def test_auth_disabled_still_removes_broken_files(research_dir, monkeypatch):
monkeypatch.setenv("AUTH_ENABLED", "false")
valid = _write_owned(research_dir, "ok", "alice")
empty = _write(research_dir, "broken-empty", " ")
garbage = _write(research_dir, "broken-json", "not-json")
message, ok = await action_tidy_research("alice")
assert ok is True
assert "Removed 2" in message
assert valid.exists()
assert not empty.exists()
assert not garbage.exists()
+6 -1
View File
@@ -6,6 +6,7 @@ notice, per-call override clamped to the hard cap, and a pre-buffer refusal
when Content-Length already exceeds the hard ceiling.
"""
import json
import ipaddress
from contextlib import contextmanager
import pytest
@@ -90,7 +91,11 @@ class _FakeStream:
def no_cache(monkeypatch, tmp_path):
monkeypatch.setattr(content_mod, "CONTENT_CACHE_DIR", tmp_path)
monkeypatch.setattr(content_mod, "_cache_result", lambda *a, **k: None)
monkeypatch.setattr(content_mod, "_public_http_url", lambda u: True)
monkeypatch.setattr(
content_mod,
"_resolve_public_ips",
lambda _url: [ipaddress.ip_address("93.184.216.34")],
)
def _patch_stream(monkeypatch, fake):
@@ -87,6 +87,7 @@ def test_webhook_trigger_handler_still_validates_token():
routes_path = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"routes",
"task",
"task_routes.py",
)
with open(routes_path, encoding="utf-8") as fh: