Compare commits

..
Author SHA1 Message Date
RaresKeY 96c88c27c8 fix(personal): bound multi-file upload memory 2026-08-15 09:02:55 +00:00
Joeseph Grey 6edd771cc9 Merge branch 'dev' into fix/add-directory-event-loop 2026-08-12 10:39:22 -06:00
StressTestor 938251000b fix(personal): route upload and delete through the index job lock
/api/personal/upload and DELETE /api/personal/file mutated the same
vector and tracking state add/remove/reload serialize on, outside
_index_job_lock and inline on the event loop.

Both now stage async work on the loop, then run the complete transition
(vector writes, disk change, personal_docs_manager update) in one
offloaded critical section under the shared lock, acquired before the
offload so queued requests park on the loop rather than pinning a
threadpool worker.

Adds add-vs-upload and add-vs-file ordering regressions.
2026-08-12 10:17:36 -06:00
StressTestor e222e92153 fix(personal): serialize add/remove/reload on an async job lock
The #5558 fix took the job lock INSIDE the threadpool worker and only on the
add path, so (1) remove_directory and /reload mutated PersonalDocsManager's
unsynchronized list/index concurrently with an in-flight add — the inconsistent
state the PR claimed to prevent — and (2) a queued add blocked on the lock while
holding an AnyIO threadpool token, starving the shared pool.

Move the lock to an asyncio.Lock acquired in the async handler BEFORE offloading,
and route add, remove and reload through it. A waiting request now parks on the
event loop instead of pinning a worker, and all three mutators are serialized so
the 'add/remove are serialized and cannot leave inconsistent state' guarantee
holds. remove and reload also run their blocking work off the event loop. The
lock is per-router so each app binds it to its own loop; single-process scope.

Tests: add-vs-remove and add-vs-reload serialization regressions (async via
ASGITransport, since asyncio.Lock deadlocks starlette TestClient's portal); the
existing add-vs-add test converted to the same driver.
2026-07-27 20:43:04 +00:00
StressTestor b91f48f50a fix(personal): run directory indexing off the event loop (#5558)
POST /api/personal/add_directory called rag.index_personal_documents
inline from an async handler, so the whole indexing job (os.walk, file
reads, per-chunk embedding, Chroma inserts) ran on the event loop and
every other request queued behind it. Indexing a real directory froze
the UI and API for 25+ minutes with no sign of life.

Move the blocking section into the threadpool via run_in_threadpool.
personal_docs_manager.add_directory stays inside it because its
refresh_index() re-extracts text across tracked directories, which is
also blocking work. A module-level lock serializes index jobs so the
threadpool move does not introduce parallel jobs racing
PersonalDocsManager's unsynchronized list mutations and file writes;
they previously serialized on the blocked loop, so one-at-a-time is
behavior parity.
2026-07-27 20:43:04 +00:00
235 changed files with 3663 additions and 33039 deletions
+2 -29
View File
@@ -76,24 +76,12 @@ SEARXNG_INSTANCE=http://localhost:8080
# Change this if another local service already uses 7000 (macOS AirPlay often does). # Change this if another local service already uses 7000 (macOS AirPlay often does).
# APP_PORT=7000 # APP_PORT=7000
# Optional HTTP address advertised in companion/mobile pairing codes. Set this
# when Docker would otherwise advertise a container address or loopback. Use a
# LAN or Tailscale IPv4 address, a single-label hostname, or an mDNS *.local
# name that the phone can reach. HTTPS and public hostnames are not supported
# by the current companion client. Do not include credentials, a path, query,
# or fragment.
# COMPANION_BASE_URL=http://192.168.1.50:7000
# Development-only auth bypass for loopback requests. # Development-only auth bypass for loopback requests.
# Keep false for Docker, LAN, reverse proxy, and any shared deployment. # Keep false for Docker, LAN, reverse proxy, and any shared deployment.
# LOCALHOST_BYPASS=false # LOCALHOST_BYPASS=false
# Mark session cookies Secure. Left unset, this follows the request scheme: # Mark session cookies Secure. Set true when Odysseus is served through HTTPS
# an HTTPS login gets a Secure cookie, a plain-HTTP one does not. Set true to # by a trusted reverse proxy or private access gateway.
# force it on, or false to force it off while you still serve plain HTTP.
# Upgrading: this used to default to false. Drop a leftover SECURE_COOKIES=false
# from your .env unless you still need that escape hatch — it keeps HTTPS logins
# on a non-Secure cookie.
# SECURE_COOKIES=true # SECURE_COOKIES=true
# Optional: pre-seed the first admin password during setup. # Optional: pre-seed the first admin password during setup.
@@ -163,21 +151,6 @@ SEARXNG_INSTANCE=http://localhost:8080
# Local HTTP setups may use the callback URL inferred by the application. # Local HTTP setups may use the callback URL inferred by the application.
# GOOGLE_OAUTH_REDIRECT_URI=https://your-domain.com/api/email/oauth/google/callback # GOOGLE_OAUTH_REDIRECT_URI=https://your-domain.com/api/email/oauth/google/callback
# Origin the MCP OAuth callback is sent back to, for remote (Streamable HTTP)
# MCP servers that register it dynamically. Defaults to http://localhost:$APP_PORT,
# which is right only when you reach Odysseus directly on that port. Set it for
# HTTPS, reverse-proxy, hosted, and Docker installs — inside the container the
# app always listens on 7000 and cannot see the host port map, so the default is
# wrong there whenever APP_PORT is not 7000.
#
# Not for Google MCP servers. Those use Desktop App credentials, and Google only
# accepts loopback redirect URIs for that client type, so a public origin here is
# rejected with redirect_uri_mismatch. Leave it unset for a Google-only install:
# the loopback default is what Google wants, and remote users finish through the
# paste-back page, which never has to load the redirect.
# https://developers.google.com/identity/protocols/oauth2/native-app
# OAUTH_REDIRECT_BASE_URL=https://your-domain.com
# ============================================================ # ============================================================
# Misc # Misc
# ============================================================ # ============================================================
-7
View File
@@ -15,13 +15,6 @@ docker/entrypoint.sh text eol=lf
*.cmd text eol=crlf *.cmd text eol=crlf
*.bat text eol=crlf *.bat text eol=crlf
# Vendored third-party bundles in static/lib/ are published minified artifacts
# and must stay byte-identical to what npm ships — stripping trailing whitespace
# to satisfy `git diff --check` would desync them from the upstream release. Turn
# the whitespace check off for that tree instead, and keep the bundles out of
# GitHub's language statistics.
static/lib/** -whitespace linguist-vendored
# Binary assets — never normalize. # Binary assets — never normalize.
*.png binary *.png binary
*.jpg binary *.jpg binary
-12
View File
@@ -26,18 +26,6 @@ body:
- label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing. - label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing.
required: true required: true
- type: input
id: revision
attributes:
label: Odysseus Revision
description: |
From the repository root (on the host when using Docker), run
`git show -s --abbrev=12 --format='%h (%cs)' HEAD`
and paste the output exactly.
placeholder: "1fef4929cf1d (2026-08-11)"
validations:
required: true
- type: dropdown - type: dropdown
id: install-method id: install-method
attributes: attributes:
-1
View File
@@ -28,7 +28,6 @@ Fixes #
- [ ] This PR targets `dev` - [ ] This PR targets `dev`
- [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in. - [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
- [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough. - [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough.
- [ ] I did not run the app/runtime validation and stated that gap in **How to Test**. Leave this unchecked when the app-run box above is checked.
## How to Test ## How to Test
@@ -41,14 +41,6 @@ module.exports = async ({ github, context, core }) => {
break; break;
case 'bug': { case 'bug': {
const revisionText = section('Odysseus Revision');
if (!/^[0-9a-f]{12} \(\d{4}-\d{2}-\d{2}\)$/i.test(revisionText)) {
failures.push(
'**Odysseus Revision** — paste the 12-character commit SHA and date, ' +
'for example `1fef4929cf1d (2026-08-11)`',
);
}
if (!section('Install Method')) { if (!section('Install Method')) {
failures.push('**Install Method** — select how you installed Odysseus'); failures.push('**Install Method** — select how you installed Odysseus');
} }
+32 -142
View File
@@ -21,11 +21,11 @@ module.exports = async ({ github, context, core }) => {
return strip(m?.[0].replace(new RegExp(`#+\\s+${heading}`, 'i'), '') ?? ''); return strip(m?.[0].replace(new RegExp(`#+\\s+${heading}`, 'i'), '') ?? '');
} }
const descriptionProblems = []; const problems = [];
// 1. Summary must be filled in. // 1. Summary must be filled in.
if (section('Summary').length < 20) { if (section('Summary').length < 20) {
descriptionProblems.push('**Summary** is empty or too short — describe what changed and why.'); problems.push('**Summary** is empty or too short — describe what changed and why.');
} }
// 2. Linked Issue must reference a real issue. Accept a bare #NNN, a closing // 2. Linked Issue must reference a real issue. Accept a bare #NNN, a closing
@@ -34,18 +34,18 @@ module.exports = async ({ github, context, core }) => {
const linkedSection = section('Linked Issue'); const linkedSection = section('Linked Issue');
const hasIssueRef = /#\d+\b/.test(linkedSection) || /\/issues\/\d+/.test(linkedSection); const hasIssueRef = /#\d+\b/.test(linkedSection) || /\/issues\/\d+/.test(linkedSection);
if (!linkedSection || !hasIssueRef) { if (!linkedSection || !hasIssueRef) {
descriptionProblems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.'); problems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.');
} }
// 3. At least one Type of Change box must be checked. // 3. At least one Type of Change box must be checked.
const typeBlock = body.match(/##\s+Type of Change[\s\S]*?(?=\n##\s|$)/i)?.[0] ?? ''; const typeBlock = body.match(/##\s+Type of Change[\s\S]*?(?=\n##\s|$)/i)?.[0] ?? '';
if (!/- \[x\]/i.test(typeBlock)) { if (!/- \[x\]/i.test(typeBlock)) {
descriptionProblems.push('**Type of Change** — check at least one box.'); problems.push('**Type of Change** — check at least one box.');
} }
// 4. Duplicate-search checklist item must be checked. // 4. Duplicate-search checklist item must be checked.
if (!/- \[x\] I searched/i.test(body)) { if (!/- \[x\] I searched/i.test(body)) {
descriptionProblems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.'); problems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.');
} }
// 5. How to Test must contain enough real detail for a reviewer to act on. // 5. How to Test must contain enough real detail for a reviewer to act on.
@@ -53,83 +53,7 @@ module.exports = async ({ github, context, core }) => {
// code block — so we only require non-trivial content, not a specific shape. // code block — so we only require non-trivial content, not a specific shape.
const howTo = section('How to Test'); const howTo = section('How to Test');
if (howTo.length < 30) { if (howTo.length < 30) {
descriptionProblems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").'); problems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").');
}
// Classify paths from GitHub's API. This workflow runs in the privileged base
// context, so it must never check out or execute code from the PR branch.
const changedFiles = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: prNum, per_page: 100,
});
const changedPaths = changedFiles.map(file => file.filename);
function isUiSensitivePath(filename) {
const path = filename.toLowerCase();
return path.startsWith('static/')
|| path.startsWith('templates/')
|| /\.(?:html?|css|svg)$/.test(path);
}
function isDocsOnlyPath(filename) {
const path = filename.toLowerCase();
return /\.(?:md|mdx|rst|adoc|txt)$/.test(path)
|| (path.startsWith('docs/') && !isUiSensitivePath(path));
}
function isRuntimeSensitivePath(filename) {
const path = filename.toLowerCase();
if (isUiSensitivePath(path)) return false;
if (path.startsWith('tests/') || path.startsWith('.github/')) return false;
return /^(?:app\.py|routes\/|services\/|src\/|core\/|mcp_servers\/|scripts\/|docker\/)/.test(path)
|| /^(?:dockerfile|docker-compose.*\.ya?ml|requirements(?:-optional)?\.txt|pyproject\.toml|setup\.py)$/.test(path)
|| /\.(?:py|sh|ps1|bat)$/.test(path);
}
let classification = 'tooling';
if (changedPaths.some(isUiSensitivePath)) {
classification = 'UI-sensitive';
} else if (changedPaths.some(isRuntimeSensitivePath)) {
classification = 'backend/runtime';
} else if (changedPaths.length > 0 && changedPaths.every(isDocsOnlyPath)) {
classification = 'docs-only';
}
const appRan = /- \[x\]\s+I actually ran the app\b/i.test(body);
const appNotRun = /- \[x\]\s+I did not run the app\/runtime validation\b/i.test(body);
// Anchor on the wording, not the template's emphasis: a ticked box the author
// retyped without the surrounding ** renders identically on the PR page, so
// treating it as unchecked is invisible from their side. Matches the two
// attestations above, which already ignore formatting.
const screenshotChecked = /- \[x\]\s+[*_]{0,2}Screenshot or short clip[*_]{0,2}/i.test(body);
const screenshotSection = section('Screenshots / clips');
const hasVisualEvidence = /!\[[^\]]*\]\([^)]+\)|<(?:img|video|source)\b[^>]*(?:src|href)=|https?:\/\/[^\s)]+/i.test(screenshotSection);
const evidenceGaps = [];
let needsRuntimeValidation = false;
let needsVisualEvidence = false;
if (classification === 'backend/runtime' || classification === 'UI-sensitive') {
if (appRan && appNotRun) {
needsRuntimeValidation = true;
evidenceGaps.push('The app-run and explicit not-run boxes are both checked. Select the one state that is true.');
} else if (!appRan) {
needsRuntimeValidation = true;
if (appNotRun) {
evidenceGaps.push('The author explicitly reports that app/runtime validation was not performed.');
} else {
evidenceGaps.push('App/runtime validation is not author-attested. Check the run box only after running it, or check the explicit not-run box and describe the gap.');
}
}
}
if (classification === 'UI-sensitive') {
if (!screenshotChecked) {
needsVisualEvidence = true;
evidenceGaps.push('The screenshot/clip checkbox is not checked for this UI-sensitive change.');
}
if (!hasVisualEvidence) {
needsVisualEvidence = true;
evidenceGaps.push('The Screenshots / clips section does not contain an actual attachment or link.');
}
} }
// ── Comment ────────────────────────────────────────────────────────────── // ── Comment ──────────────────────────────────────────────────────────────
@@ -138,43 +62,22 @@ module.exports = async ({ github, context, core }) => {
}); });
const existing = comments.find(c => (c.body ?? '').includes(MARKER)); const existing = comments.find(c => (c.body ?? '').includes(MARKER));
if (descriptionProblems.length === 0 && evidenceGaps.length === 0) { if (problems.length === 0) {
if (existing) { if (existing) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id }); await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
} }
} else { } else {
const commentLines = [MARKER]; const commentBody = [
if (descriptionProblems.length > 0) { MARKER,
commentLines.push( '⚠️ **PR description — action needed**',
'⚠️ **PR description — action needed**', '',
'', 'The following required sections are missing or incomplete. Please update the PR description to address them:',
'The following required sections are missing or incomplete. Please update the PR description to address them:', '',
'', problems.map(p => `- ${p}`).join('\n'),
descriptionProblems.map(problem => `- ${problem}`).join('\n'),
);
} else {
commentLines.push(
'⚠️ **PR description is complete; validation evidence is still outstanding**',
'',
`Changed-file classification: **${classification}**.`,
);
}
if (evidenceGaps.length > 0) {
commentLines.push(
'',
'**Author-reported runtime / visual state**',
'',
evidenceGaps.map(gap => `- ${gap}`).join('\n'),
'',
'Checkboxes are author attestations. GitHub Actions results remain the execution evidence for CI; this check does not prove that a local command ran.',
);
}
commentLines.push(
'', '',
'---', '---',
'_This comment updates automatically when the description or changed files change._', '_This comment is deleted automatically once all sections are complete._',
); ].join('\n');
const commentBody = commentLines.join('\n');
if (existing) { if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: commentBody }); await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: commentBody });
@@ -194,47 +97,34 @@ module.exports = async ({ github, context, core }) => {
return true; return true;
} catch (e) { } catch (e) {
if (e.status === 404) return false; if (e.status === 404) return false;
if (e.status === 403) {
core.warning(`Could not inspect label "${name}" — token lacks label read access; skipping.`);
return false;
}
throw e; throw e;
} }
} }
async function setLabel(name, wanted) { async function swapLabel(num, add, remove) {
if (wanted && await labelExists(name)) { if (await labelExists(add)) {
try { try {
await github.rest.issues.addLabels({ owner, repo, issue_number: prNum, labels: [name] }); await github.rest.issues.addLabels({ owner, repo, issue_number: num, labels: [add] });
} catch (e) { } catch (e) {
// Fail soft on a token that can't write labels so a label permission // Fail soft on a token that can't write labels so a label permission
// problem never masks the actual description verdict. // problem never masks the actual description verdict.
if (e.status !== 403 && e.status !== 404) throw e; if (e.status !== 403) throw e;
core.warning(`Could not add "${name}" — label is unavailable or the token lacks label write access; skipping.`); core.warning(`Could not add "${add}" — token lacks label write here; skipping.`);
} }
} else if (wanted) {
core.warning(`Label "${name}" does not exist in the repo — skipping. Create it once to enable labelling.`);
} else { } else {
try { core.warning(`Label "${add}" does not exist in the repo — skipping. Create it once to enable labelling.`);
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNum, name }); }
} catch (e) { try {
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e; await github.rest.issues.removeLabel({ owner, repo, issue_number: num, name: remove });
} } catch (e) {
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e;
} }
} }
const descriptionComplete = descriptionProblems.length === 0; if (problems.length === 0) {
const evidenceComplete = evidenceGaps.length === 0; await swapLabel(prNum, 'ready for review', 'needs work');
const isDraft = Boolean(context.payload.pull_request.draft); } else {
await setLabel( await swapLabel(prNum, 'needs work', 'ready for review');
'ready for review', core.setFailed(`PR description has ${problems.length} issue(s) — see bot comment for details.`);
descriptionComplete && evidenceComplete && !isDraft,
);
await setLabel('needs work', !descriptionComplete);
await setLabel('needs runtime validation', needsRuntimeValidation);
await setLabel('needs visual evidence', needsVisualEvidence);
if (!descriptionComplete) {
core.setFailed(`PR description has ${descriptionProblems.length} issue(s) — see bot comment for details.`);
} }
}; };
+3 -9
View File
@@ -5,11 +5,7 @@ on:
# works on fork PRs. Safe here: the checkout pins to the base branch (no fork # works on fork PRs. Safe here: the checkout pins to the base branch (no fork
# code runs) and the scripts only read context.payload and call the GitHub API. # code runs) and the scripts only read context.payload and call the GitHub API.
pull_request_target: # zizmor: ignore[dangerous-triggers] pull_request_target: # zizmor: ignore[dangerous-triggers]
types: [opened, edited, synchronize, reopened, ready_for_review, converted_to_draft] types: [opened, edited, synchronize, reopened, ready_for_review]
concurrency:
group: pr-description-${{ github.event.pull_request.number }}
cancel-in-progress: true
# Default-deny at the workflow level; each job opts into only the scopes it needs. # Default-deny at the workflow level; each job opts into only the scopes it needs.
# Note: modifying a PR's labels/comments needs pull-requests:write even though the # Note: modifying a PR's labels/comments needs pull-requests:write even though the
@@ -63,14 +59,12 @@ jobs:
check-mergeable: check-mergeable:
name: Flag unmergeable PRs name: Flag unmergeable PRs
needs: check-description
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
pull-requests: write pull-requests: write
issues: write issues: write
# Run after description validation failures, but never from an obsolete # Skip bots: they open PRs programmatically and have their own process.
# workflow run canceled by a newer PR event. if: github.event.pull_request.user.type != 'Bot'
if: ${{ !cancelled() && github.event.pull_request.user.type != 'Bot' }}
steps: steps:
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with: with:
+2 -10
View File
@@ -65,16 +65,6 @@ Vendored in `static/lib/` and served directly:
| [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT | | [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT |
| [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT | | [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT |
| [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT | | [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT |
| [KaTeX](https://github.com/KaTeX/KaTeX) v0.16.22 (`katex/katex.min.{js,css}` + `katex/fonts/*.woff2`) | Math typesetting | MIT ([`licenses/KaTeX-MIT-LICENSE.txt`](licenses/KaTeX-MIT-LICENSE.txt)) |
| [Mermaid](https://github.com/mermaid-js/mermaid) v11.16.1 (`mermaid.min.js`) | Diagrams from text | MIT ([`licenses/Mermaid-MIT-LICENSE.txt`](licenses/Mermaid-MIT-LICENSE.txt)) |
KaTeX and Mermaid are loaded on first use by `static/js/markdown.js` rather than
from `index.html`, so a session that renders no math and no diagram never fetches
either. Only the `.woff2` KaTeX fonts are shipped, matching `static/fonts/`; the
`.woff` and `.ttf` variants its stylesheet also lists are never requested by a
browser that supports `woff2`. The bundles are the published npm artifacts,
unmodified — `.gitattributes` turns the whitespace check off for `static/lib/`
so they can stay byte-identical to upstream.
## Front-end libraries loaded at runtime (CDN) ## Front-end libraries loaded at runtime (CDN)
@@ -82,6 +72,8 @@ Referenced from `cdn.jsdelivr.net` / `cdnjs.cloudflare.com` at runtime — not v
| Library | Purpose | License | | Library | Purpose | License |
|---|---|---| |---|---|---|
| [KaTeX](https://github.com/KaTeX/KaTeX) 0.16.22 | Math typesetting | MIT |
| [Mermaid](https://github.com/mermaid-js/mermaid) 11 | Diagrams from text | MIT |
| [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 | | [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 |
| [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT | | [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT |
+5 -10
View File
@@ -59,20 +59,15 @@ Help is welcome. The best entry points are fresh-install testing, provider setup
## Security ## Security
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Deployment details are in the [setup guide](docs/setup.md#security-notes).
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development.
Deployment details are in the [setup guide](docs/setup.md#security-notes).
## Star History ## Star History
<a href="https://star-history.dera.page/#odysseus-dev/odysseus&type=date&legend=top-left"> <a href="https://www.star-history.com/?repos=odysseus-dev%2Fodysseus&type=date&legend=top-left">
<picture> <picture>
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" /> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" /> <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
</picture> </picture>
</a> </a>
+1 -1
View File
@@ -10,7 +10,7 @@ Security fixes are handled on the default branch until formal releases are cut.
- Keep `AUTH_ENABLED=true` for any network-accessible deployment. - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development. - Keep `LOCALHOST_BYPASS=false` outside local development.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Set `SECURE_COOKIES=true` to force it on (for a proxy Odysseus cannot see the scheme of), or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS. - Set `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Use HTTPS when exposing the app beyond localhost. - Use HTTPS when exposing the app beyond localhost.
- Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN. - Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN.
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only.
+1 -1
View File
@@ -37,7 +37,7 @@ Non-admin defaults are in `core/auth.py:DEFAULT_PRIVILEGES`. Tool enforcement is
- **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`. - **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`.
- **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance. - **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance.
- **Reserved usernames:** request sentinels and the Default/Local storage owner cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`. - **Reserved usernames:** `internal-tool`, `api`, `demo`, `system` cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`.
- `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check. - `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check.
- **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate. - **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate.
+10 -34
View File
@@ -67,13 +67,7 @@ from core.constants import (
REQUEST_TIMEOUT, OPENAI_API_KEY, AUTH_FILE, REQUEST_TIMEOUT, OPENAI_API_KEY, AUTH_FILE,
) )
from core.database import SessionLocal, ApiToken from core.database import SessionLocal, ApiToken
from core.middleware import ( from core.middleware import SecurityHeadersMiddleware, is_cors_preflight
SecurityHeadersMiddleware,
get_application_route_path,
is_cors_preflight,
path_is_route_or_child,
with_asgi_root_path,
)
from core.auth import AuthManager, normalize_known_username from core.auth import AuthManager, normalize_known_username
from core.exceptions import ( from core.exceptions import (
SessionNotFoundError, InvalidFileUploadError, SessionNotFoundError, InvalidFileUploadError,
@@ -84,7 +78,6 @@ import bcrypt as _bcrypt
from src.app_helpers import abs_join, serve_html_with_nonce from src.app_helpers import abs_join, serve_html_with_nonce
from src.generated_images import GENERATED_IMAGE_HEADERS, resolve_generated_image_path from src.generated_images import GENERATED_IMAGE_HEADERS, resolve_generated_image_path
from src.owner_identity import auth_disabled
from starlette.responses import RedirectResponse from starlette.responses import RedirectResponse
# ========= LOGGING ========= # ========= LOGGING =========
@@ -255,7 +248,7 @@ from routes.auth_routes import setup_auth_routes, SESSION_COOKIE
auth_manager = AuthManager() auth_manager = AuthManager()
app.state.auth_manager = auth_manager app.state.auth_manager = auth_manager
AUTH_ENABLED = not auth_disabled() AUTH_ENABLED = os.getenv("AUTH_ENABLED", "true").lower() != "false"
LOCALHOST_BYPASS = os.getenv("LOCALHOST_BYPASS", "false").lower() == "true" LOCALHOST_BYPASS = os.getenv("LOCALHOST_BYPASS", "false").lower() == "true"
if LOCALHOST_BYPASS: if LOCALHOST_BYPASS:
logger.warning("LOCALHOST_BYPASS is enabled, loopback requests bypass authentication. Do not expose this instance to a network.") logger.warning("LOCALHOST_BYPASS is enabled, loopback requests bypass authentication. Do not expose this instance to a network.")
@@ -291,7 +284,7 @@ if AUTH_ENABLED:
def _is_auth_exempt(path: str) -> bool: def _is_auth_exempt(path: str) -> bool:
if path in AUTH_EXEMPT_EXACT: if path in AUTH_EXEMPT_EXACT:
return True return True
if any(path_is_route_or_child(path, p) for p in AUTH_EXEMPT_PREFIXES): if any(path.startswith(p) for p in AUTH_EXEMPT_PREFIXES):
return True return True
return any(p.match(path) for p in AUTH_EXEMPT_PATTERNS) return any(p.match(path) for p in AUTH_EXEMPT_PATTERNS)
@@ -362,7 +355,7 @@ if AUTH_ENABLED:
class AuthMiddleware(BaseHTTPMiddleware): class AuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next): async def dispatch(self, request: Request, call_next):
path = get_application_route_path(request.scope) path = request.url.path
# A genuine CORS preflight (OPTIONS + Access-Control-Request-Method) # A genuine CORS preflight (OPTIONS + Access-Control-Request-Method)
# carries no credentials by design and must reach CORSMiddleware to be # carries no credentials by design and must reach CORSMiddleware to be
# answered. AuthMiddleware is the outermost middleware, so gating the # answered. AuthMiddleware is the outermost middleware, so gating the
@@ -406,10 +399,7 @@ if AUTH_ENABLED:
if not auth_manager.is_configured: if not auth_manager.is_configured:
# No users yet — redirect to login for first-time setup # No users yet — redirect to login for first-time setup
if not path.startswith("/api/"): if not path.startswith("/api/"):
return RedirectResponse( return RedirectResponse(url="/login", status_code=302)
url=with_asgi_root_path(request.scope, "/login"),
status_code=302,
)
return JSONResponse(status_code=401, content={"error": "Setup required"}) return JSONResponse(status_code=401, content={"error": "Setup required"})
# --- Bearer token auth (API tokens for external integrations) --- # --- Bearer token auth (API tokens for external integrations) ---
@@ -471,10 +461,7 @@ if AUTH_ENABLED:
if not auth_manager.validate_token(token): if not auth_manager.validate_token(token):
if path.startswith("/api/"): if path.startswith("/api/"):
return JSONResponse(status_code=401, content={"error": "Not authenticated"}) return JSONResponse(status_code=401, content={"error": "Not authenticated"})
return RedirectResponse( return RedirectResponse(url="/login", status_code=302)
url=with_asgi_root_path(request.scope, "/login"),
status_code=302,
)
# Attach current username to request state for downstream routes # Attach current username to request state for downstream routes
request.state.current_user = auth_manager.get_username_for_token(token) request.state.current_user = auth_manager.get_username_for_token(token)
@@ -643,24 +630,13 @@ app.include_router(auth_router)
@app.post("/api/activity/heartbeat") @app.post("/api/activity/heartbeat")
async def activity_heartbeat(): async def activity_heartbeat():
from src.interactive_gate import ( from src.interactive_gate import mark_browser_activity
mark_browser_activity,
maybe_stop_background_tasks_for_heartbeat,
)
await mark_browser_activity() await mark_browser_activity()
async def _stop_background(): async def _stop_background():
try: try:
await maybe_stop_background_tasks_for_heartbeat( await task_scheduler.stop_background_tasks_for_foreground(reason="browser heartbeat")
task_scheduler.stop_background_tasks_for_foreground
)
except Exception: except Exception:
logging.getLogger("app.foreground_gate").debug( logging.getLogger("app.foreground_gate").debug("heartbeat task stop failed", exc_info=True)
"heartbeat task stop failed",
exc_info=True,
)
asyncio.create_task(_stop_background()) asyncio.create_task(_stop_background())
return {"ok": True} return {"ok": True}
@@ -784,7 +760,7 @@ from src.task_scheduler import TaskScheduler
task_scheduler = TaskScheduler(session_manager) task_scheduler = TaskScheduler(session_manager)
from src.event_bus import set_task_scheduler from src.event_bus import set_task_scheduler
set_task_scheduler(task_scheduler) set_task_scheduler(task_scheduler)
from routes.task.task_routes import setup_task_routes from routes.task_routes import setup_task_routes
app.include_router(setup_task_routes(task_scheduler)) app.include_router(setup_task_routes(task_scheduler))
from routes.assistant_routes import setup_assistant_routes from routes.assistant_routes import setup_assistant_routes
-4
View File
@@ -73,10 +73,6 @@ cat > "$APP/Contents/MacOS/$APP_NAME.tmpl" <<'LAUNCHER'
INSTALL_DIR="__INSTALL_DIR__" INSTALL_DIR="__INSTALL_DIR__"
PORT="__PORT__" PORT="__PORT__"
URL="http://127.0.0.1:${PORT}" URL="http://127.0.0.1:${PORT}"
# uvicorn is started with --port below, but APP_PORT is what the app itself
# reads when it needs to build a URL for this instance (internal_api_base(),
# companion pairing, the MCP OAuth callback), so export it as well.
export APP_PORT="$PORT"
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH" export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH"
UVICORN="$INSTALL_DIR/venv/bin/uvicorn" UVICORN="$INSTALL_DIR/venv/bin/uvicorn"
-99
View File
@@ -6,14 +6,11 @@ units so the route layer stays thin and the logic is directly testable.
from __future__ import annotations from __future__ import annotations
import ipaddress
import json import json
import os import os
import re
import secrets import secrets
import socket import socket
import uuid import uuid
from urllib.parse import urlsplit
import bcrypt import bcrypt
@@ -23,102 +20,6 @@ PAIRING_VERSION = 1
COMPANION_SCOPE = "chat" COMPANION_SCOPE = "chat"
_COMPANION_IPV4_NETWORKS = tuple(
ipaddress.ip_network(cidr)
for cidr in (
"10.0.0.0/8",
"100.64.0.0/10",
"127.0.0.0/8",
"169.254.0.0/16",
"172.16.0.0/12",
"192.168.0.0/16",
)
)
_DNS_LABEL_RE = re.compile(r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\Z")
def _valid_companion_client_host(host: str) -> bool:
"""Match the host forms supported by the current v1 Expo client."""
if not host or len(host) > 253 or not host.isascii() or "%" in host:
return False
try:
address = ipaddress.ip_address(host)
except ValueError:
labels = host.split(".")
if any(not _DNS_LABEL_RE.fullmatch(label) for label in labels):
return False
if any(label.startswith("xn--") for label in labels):
return False
# WHATWG URL parsers treat a decimal or ``0x`` single-label hostname
# as an IPv4 number even though Python's strict ``ipaddress`` parser
# rejects that spelling. The v1 client interpolates this host back
# into a URL, so accepting e.g. ``134744072`` would make the phone send
# its bearer token to public 8.8.8.8. Keep DNS labels unambiguous.
if len(labels) == 1 and (
labels[0].isdigit()
or re.fullmatch(r"0x[0-9a-f]*", labels[0]) is not None
):
return False
return len(labels) == 1 or (len(labels) >= 2 and labels[-1] == "local")
return isinstance(address, ipaddress.IPv4Address) and any(
address in network for network in _COMPANION_IPV4_NETWORKS
)
def parse_companion_base_url(value: str) -> tuple[str, int]:
"""Validate a v1 companion address and return its legacy (host, port).
The deployed client understands only HTTP plus a LAN-style host and port.
Reject anything outside that exact contract instead of advertising a URL
the client would reject, downgrade, or interpret differently.
"""
if not isinstance(value, str) or not value:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
if not value.isascii():
raise ValueError("COMPANION_BASE_URL must contain only ASCII characters")
if any(
ord(char) <= 32 or ord(char) == 127 or char in {"\\", "%"}
for char in value
):
raise ValueError(
"COMPANION_BASE_URL contains a forbidden character"
)
try:
parsed = urlsplit(value)
port = parsed.port
except ValueError as exc:
raise ValueError("COMPANION_BASE_URL must be a valid HTTP LAN origin") from exc
host = parsed.hostname
if parsed.scheme.lower() != "http" or not parsed.netloc or not host:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
if parsed.username is not None or parsed.password is not None:
raise ValueError("COMPANION_BASE_URL must not contain credentials")
if parsed.path or parsed.query or parsed.fragment:
raise ValueError("COMPANION_BASE_URL must not contain a path, query, or fragment")
if port is not None and not 1 <= port <= 65535:
raise ValueError("COMPANION_BASE_URL port must be between 1 and 65535")
if not _valid_companion_client_host(host):
raise ValueError("COMPANION_BASE_URL host is not supported by companion v1")
netloc = f"{host}:{port}" if port is not None else host
origin = f"http://{netloc}"
if value != origin:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
return host, port or 80
def configured_companion_origin() -> tuple[str, int] | None:
"""Return the validated operator-configured v1 address, if any."""
value = os.environ.get("COMPANION_BASE_URL")
if value is None or value == "":
return None
return parse_companion_base_url(value)
def default_port() -> int: def default_port() -> int:
"""Best guess at the port the server is reachable on. Callers that know the """Best guess at the port the server is reachable on. Callers that know the
real request port should pass it explicitly.""" real request port should pass it explicitly."""
+8 -23
View File
@@ -23,7 +23,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from core.middleware import require_admin from core.middleware import require_admin
from src.auth_helpers import _auth_disabled, get_current_user from src.auth_helpers import get_current_user
from companion import pairing as _pairing from companion import pairing as _pairing
@@ -113,9 +113,8 @@ def setup_companion_routes() -> APIRouter:
The stock /api/models route scopes to get_current_user, which for a The stock /api/models route scopes to get_current_user, which for a
bearer token is the sandboxed pseudo-user "api" (owns nothing). Here we bearer token is the sandboxed pseudo-user "api" (owns nothing). Here we
scope to the token's real owner instead, plus legacy null-owner shared scope to the token's real owner instead, plus legacy null-owner shared
rows -- the same rule as owner_filter. Explicit auth-disabled mode keeps rows -- the same rule as owner_filter. Read-only; never returns api_key
the stock route's single-user all-endpoints view. Read-only; never material.
returns api_key material.
""" """
require_models_scope(request) require_models_scope(request)
import json as _json import json as _json
@@ -124,11 +123,6 @@ def setup_companion_routes() -> APIRouter:
from src.endpoint_resolver import build_chat_url from src.endpoint_resolver import build_chat_url
owner = token_owner(request) owner = token_owner(request)
single_user_mode = (
owner is None
and not getattr(request.state, "api_token", False)
and _auth_disabled()
)
out = [] out = []
db = SessionLocal() db = SessionLocal()
try: try:
@@ -139,7 +133,7 @@ def setup_companion_routes() -> APIRouter:
if owner: if owner:
q = q.filter((ModelEndpoint.owner == owner) | (ModelEndpoint.owner == None)) # noqa: E711 q = q.filter((ModelEndpoint.owner == owner) | (ModelEndpoint.owner == None)) # noqa: E711
for ep in q.all(): for ep in q.all():
if not single_user_mode and not owner_can_see(ep.owner, owner): if not owner_can_see(ep.owner, owner):
continue continue
try: try:
model_ids = _json.loads(ep.cached_models) if ep.cached_models else [] model_ids = _json.loads(ep.cached_models) if ep.cached_models else []
@@ -200,27 +194,19 @@ def setup_companion_routes() -> APIRouter:
the code works immediately, no restart. `?format=json` returns the the code works immediately, no restart. `?format=json` returns the
payload for an in-app pairing screen.""" payload for an in-app pairing screen."""
require_admin(request) require_admin(request)
try:
configured_origin = _pairing.configured_companion_origin()
except ValueError as exc:
raise HTTPException(500, str(exc)) from None
owner = get_current_user(request) owner = get_current_user(request)
invalidate = getattr(request.app.state, "invalidate_token_cache", None) invalidate = getattr(request.app.state, "invalidate_token_cache", None)
token_id, raw_token = mint_pairing_token(owner, invalidate) token_id, raw_token = mint_pairing_token(owner, invalidate)
if configured_origin: hosts = _pairing.lan_ip_candidates()
host, port = configured_origin host = hosts[0] if hosts else "127.0.0.1"
hosts = [host] port = request.url.port or _pairing.default_port()
else:
hosts = _pairing.lan_ip_candidates()
host = hosts[0] if hosts else "127.0.0.1"
port = request.url.port or _pairing.default_port()
payload = _pairing.pairing_payload(host, port, raw_token) payload = _pairing.pairing_payload(host, port, raw_token)
qr = _pairing.pairing_qr_png_data_uri(payload) qr = _pairing.pairing_qr_png_data_uri(payload)
qr_ok = bool(qr and qr.startswith("data:image/png;base64,")) qr_ok = bool(qr and qr.startswith("data:image/png;base64,"))
if (request.query_params.get("format") or "").lower() == "json": if (request.query_params.get("format") or "").lower() == "json":
response = { return {
"host": host, "host": host,
"port": port, "port": port,
"token": raw_token, "token": raw_token,
@@ -229,7 +215,6 @@ def setup_companion_routes() -> APIRouter:
"payload": payload, "payload": payload,
"qr": qr if qr_ok else None, "qr": qr if qr_ok else None,
} }
return response
import json as _json import json as _json
payload_json = _json.dumps(payload, separators=(",", ":")) payload_json = _json.dumps(payload, separators=(",", ":"))
+10 -28
View File
@@ -30,20 +30,11 @@ def atomic_write_json(path: str, data: Any, *, indent: Optional[int] = None) ->
""" """
os.makedirs(os.path.dirname(path) or ".", exist_ok=True) os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
tmp = f"{path}.tmp.{uuid.uuid4().hex}" tmp = f"{path}.tmp.{uuid.uuid4().hex}"
with open(tmp, "w", encoding="utf-8") as f:
try: json.dump(data, f, indent=indent)
with open(tmp, "w", encoding="utf-8") as f: f.flush()
json.dump(data, f, indent=indent) os.fsync(f.fileno())
f.flush() os.replace(tmp, path)
os.fsync(f.fileno())
os.replace(tmp, path)
finally:
# Directly unlink to avoid a check-then-act race condition.
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
try:
os.unlink(tmp)
except OSError:
pass
def atomic_write_text(path: str, text: str) -> None: def atomic_write_text(path: str, text: str) -> None:
@@ -51,17 +42,8 @@ def atomic_write_text(path: str, text: str) -> None:
raise TypeError("atomic_write_text expects a string") raise TypeError("atomic_write_text expects a string")
os.makedirs(os.path.dirname(path) or ".", exist_ok=True) os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
tmp = f"{path}.tmp.{uuid.uuid4().hex}" tmp = f"{path}.tmp.{uuid.uuid4().hex}"
with open(tmp, "w", encoding="utf-8") as f:
try: f.write(text)
with open(tmp, "w", encoding="utf-8") as f: f.flush()
f.write(text) os.fsync(f.fileno())
f.flush() os.replace(tmp, path)
os.fsync(f.fileno())
os.replace(tmp, path)
finally:
# Directly unlink to avoid a check-then-act race condition.
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
try:
os.unlink(tmp)
except OSError:
pass
+16 -9
View File
@@ -20,6 +20,7 @@ logger = logging.getLogger(__name__)
from core.atomic_io import atomic_write_json as _atomic_write_json # noqa: E402 from core.atomic_io import atomic_write_json as _atomic_write_json # noqa: E402
from core.middleware import INTERNAL_TOOL_USER # noqa: E402
DEFAULT_PRIVILEGES = { DEFAULT_PRIVILEGES = {
"can_use_agent": True, "can_use_agent": True,
@@ -48,18 +49,24 @@ ADMIN_PRIVILEGES["allowed_models_restricted"] = False
ADMIN_PRIVILEGES["block_all_models"] = False ADMIN_PRIVILEGES["block_all_models"] = False
from src.constants import AUTH_FILE, PASSWORD_MIN_LENGTH from src.constants import AUTH_FILE, PASSWORD_MIN_LENGTH
from src.owner_identity import RESERVED_AUTH_USERNAMES
DEFAULT_AUTH_PATH = AUTH_FILE DEFAULT_AUTH_PATH = AUTH_FILE
TOKEN_TTL = 60 * 60 * 24 * 7 # 7 days TOKEN_TTL = 60 * 60 * 24 * 7 # 7 days
# Usernames the auth + middleware layer reserves for request sentinels and # Usernames the auth + middleware layer reserve as internal "synthetic owner"
# internal storage owners; they must never belong to a real login account. # sentinels; they must never belong to a real account. The most dangerous is
# "internal-tool" is the most dangerous because `core.middleware.require_admin` # "internal-tool": `core.middleware.require_admin` treats any request whose
# treats it as the in-process tool loopback. "api" collides with bearer-token # `current_user == "internal-tool"` as the in-process tool loopback and grants
# attribution. "demo"/"system" are synthetic owners already special-cased by # admin, and because the cookie auth path sets `current_user` to the raw
# scheduler/assistant/research paths. The Default/Local owner is a storage # username, an account literally named "internal-tool" would be silently
# bucket for explicit auth-disabled no-login mode, not a login username. # treated as an admin by every `require_admin`-gated route. "api" collides with
RESERVED_USERNAMES = frozenset(RESERVED_AUTH_USERNAMES) # the bearer-token owner-attribution sentinel. "demo"/"system" round out the
# synthetic-owner set the rest of the codebase already special-cases (see
# `_SYNTHETIC_OWNERS` in routes/assistant_routes.py and the matching guards in
# src/task_scheduler.py / routes/research_routes.py) — a real account with one
# of those names would be denied an assistant and inconsistently owner-scoped.
# Refuse to create or rename into any of them so the sentinels can't be
# impersonated. (Keep this in sync with that synthetic-owner set.)
RESERVED_USERNAMES = frozenset({INTERNAL_TOOL_USER, "api", "demo", "system"})
def normalize_known_username(users: Dict[str, Any], username: str | None) -> Optional[str]: def normalize_known_username(users: Dict[str, Any], username: str | None) -> Optional[str]:
+2 -19
View File
@@ -1491,25 +1491,8 @@ def _migrate_assign_legacy_owner():
with open(prefs_path, "r", encoding="utf-8") as f: with open(prefs_path, "r", encoding="utf-8") as f:
prefs = _json.load(f) prefs = _json.load(f)
if "_users" not in prefs and prefs: if "_users" not in prefs and prefs:
# Flat format → nest ordinary preferences under the admin # Flat format → nest under admin user
# user. Foreground fallback is an explicit per-owner opt-in, new_prefs = {"_users": {admin_user: prefs}}
# so auth-disabled consent must remain inert at the flat root
# rather than becoming consent for the first named owner.
foreground_keys = {
"foreground_fallback_enabled",
"foreground_model_fallbacks",
}
named_prefs = {
key: value
for key, value in prefs.items()
if key not in foreground_keys
}
new_prefs = {
key: prefs[key]
for key in foreground_keys
if key in prefs
}
new_prefs["_users"] = {admin_user: named_prefs}
with open(prefs_path, "w", encoding="utf-8") as f: with open(prefs_path, "w", encoding="utf-8") as f:
_json.dump(new_prefs, f, indent=2) _json.dump(new_prefs, f, indent=2)
logger.info(f"Migrated user_prefs.json to per-user format under '{admin_user}'") logger.info(f"Migrated user_prefs.json to per-user format under '{admin_user}'")
+3 -29
View File
@@ -3,14 +3,10 @@
import os import os
import secrets import secrets
from collections.abc import Mapping
from fastapi import HTTPException, Request from fastapi import HTTPException, Request
from starlette.middleware.base import BaseHTTPMiddleware from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import Response from starlette.responses import Response
from starlette.routing import get_route_path
from src.owner_identity import INTERNAL_TOOL_USER, auth_disabled
# Per-process token that lets the in-app tool layer hit admin-gated # Per-process token that lets the in-app tool layer hit admin-gated
@@ -19,30 +15,8 @@ from src.owner_identity import INTERNAL_TOOL_USER, auth_disabled
# same value from this module. Never persisted or exposed externally. # same value from this module. Never persisted or exposed externally.
INTERNAL_TOOL_TOKEN = os.environ.get("ODYSSEUS_INTERNAL_TOKEN") or secrets.token_hex(32) INTERNAL_TOOL_TOKEN = os.environ.get("ODYSSEUS_INTERNAL_TOKEN") or secrets.token_hex(32)
INTERNAL_TOOL_HEADER = "X-Odysseus-Internal-Token" INTERNAL_TOOL_HEADER = "X-Odysseus-Internal-Token"
# Pseudo-username on in-process tool-loopback requests; require_admin trusts it and it is reserved.
INTERNAL_TOOL_USER = "internal-tool"
def get_application_route_path(scope: Mapping[str, object]) -> str:
"""Return the application-relative path used by Starlette routing.
Uvicorn prefixes ``scope["path"]`` with a configured ASGI ``root_path``;
Starlette removes that prefix before matching routes. Middleware policy
must use the same path form or a deployment prefix can change which policy
applies to an otherwise unchanged application route.
"""
return get_route_path(scope)
def with_asgi_root_path(scope: Mapping[str, object], path: str) -> str:
"""Prefix an application path for a client-facing redirect target."""
root_path = scope.get("root_path", "")
if not isinstance(root_path, str) or not root_path:
return path
return f"{root_path.rstrip('/')}{path}"
def path_is_route_or_child(path: str, prefix: str) -> bool:
"""Return whether ``path`` is exactly ``prefix`` or below that route."""
return path == prefix or path.startswith(prefix + "/")
def is_cors_preflight(method: str, headers) -> bool: def is_cors_preflight(method: str, headers) -> bool:
@@ -73,7 +47,7 @@ def require_admin(request: Request):
pass pass
auth_mgr = getattr(request.app.state, "auth_manager", None) auth_mgr = getattr(request.app.state, "auth_manager", None)
if auth_disabled(): if os.getenv("AUTH_ENABLED", "true").lower() == "false":
return return
if not auth_mgr or not auth_mgr.is_configured: if not auth_mgr or not auth_mgr.is_configured:
raise HTTPException(403, "Admin only") raise HTTPException(403, "Admin only")
+1 -51
View File
@@ -8,11 +8,6 @@ These are simple datacontainers. All persistence is handled by SessionManager.
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, List, Any, Optional, TYPE_CHECKING from typing import Dict, List, Any, Optional, TYPE_CHECKING
from src.tool_approval_scopes import (
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
CHAT_SESSION_APPROVAL_DECISION,
)
if TYPE_CHECKING: if TYPE_CHECKING:
from .session_manager import SessionManager from .session_manager import SessionManager
@@ -36,35 +31,6 @@ set_session_manager = set_session_manager_instance
get_session_manager = get_session_manager_instance get_session_manager = get_session_manager_instance
def _history_grants_chat_session_approval(
history: List["ChatMessage"],
session_id: str,
) -> bool:
"""Return whether this exact chat has a resolved session-scope grant."""
expected_session = str(session_id or "")
if not expected_session:
return False
for message in reversed(history or []):
metadata = getattr(message, "metadata", None)
if not isinstance(metadata, dict):
continue
tool_events = metadata.get("tool_events")
if not isinstance(tool_events, list):
continue
for event in reversed(tool_events):
ask_user = event.get("ask_user") if isinstance(event, dict) else None
if not isinstance(ask_user, dict):
continue
if (
ask_user.get("kind") == "tool_approval"
and ask_user.get("resolved") == CHAT_SESSION_APPROVAL_DECISION
and str(ask_user.get("session_id") or "") == expected_session
):
return True
return False
@dataclass @dataclass
class ChatMessage: class ChatMessage:
"""A single chat message.""" """A single chat message."""
@@ -150,27 +116,11 @@ class Session:
the model. Display/history-load paths use the raw ``history`` and are the model. Display/history-load paths use the raw ``history`` and are
unaffected. unaffected.
""" """
messages = [ return [
msg.to_dict() msg.to_dict()
for msg in self.history for msg in self.history
if (msg.metadata or {}).get("source") != "slash" if (msg.metadata or {}).get("source") != "slash"
] ]
if not _history_grants_chat_session_approval(self.history, self.id):
return messages
# Keep the grant close to the latest user request so route-neutral
# compaction/trimming preserves it. Copy the metadata instead of
# mutating the durable transcript object.
for index in range(len(messages) - 1, -1, -1):
if messages[index].get("role") != "user":
continue
message = dict(messages[index])
metadata = dict(message.get("metadata") or {})
metadata[CHAT_SESSION_APPROVAL_CONTEXT_MARKER] = True
message["metadata"] = metadata
messages[index] = message
break
return messages
def get(self, key: str, default=None): def get(self, key: str, default=None):
"""Dict-like access for compatibility.""" """Dict-like access for compatibility."""
+1 -17
View File
@@ -14,8 +14,6 @@ import logging
from datetime import datetime, timezone, timedelta from datetime import datetime, timezone, timedelta
from typing import Dict, Optional from typing import Dict, Optional
from sqlalchemy import func
from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive
from .models import Session, ChatMessage from .models import Session, ChatMessage
from src.attachment_refs import persistable_message_content from src.attachment_refs import persistable_message_content
@@ -94,28 +92,14 @@ class SessionManager:
try: try:
db_sessions = db.query(DbSession).filter( db_sessions = db.query(DbSession).filter(
DbSession.archived == False, DbSession.archived == False,
DbSession.messages.any(), DbSession.message_count > 0,
).order_by(DbSession.last_accessed.desc()).limit(100).all() ).order_by(DbSession.last_accessed.desc()).limit(100).all()
# message_count is derived metadata and can drift after interrupted
# or legacy writes. Count only the bounded discovery set so startup
# remains metadata-only while lazy hydration sees an authoritative
# positive count for every discovered non-empty session.
message_counts = {}
if db_sessions:
message_counts = dict(
db.query(DbChatMessage.session_id, func.count(DbChatMessage.id))
.filter(DbChatMessage.session_id.in_([row.id for row in db_sessions]))
.group_by(DbChatMessage.session_id)
.all()
)
loaded_count = 0 loaded_count = 0
for db_session in db_sessions: for db_session in db_sessions:
try: try:
session = self._db_to_session_meta(db_session) session = self._db_to_session_meta(db_session)
if session is not None: if session is not None:
session.message_count = message_counts[db_session.id]
self.sessions[db_session.id] = session self.sessions[db_session.id] = session
loaded_count += 1 loaded_count += 1
except Exception as e: except Exception as e:
+1 -12
View File
@@ -46,11 +46,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -75,11 +74,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -135,17 +129,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
+1 -12
View File
@@ -45,11 +45,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -74,11 +73,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -138,17 +132,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
+1 -12
View File
@@ -34,11 +34,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -63,11 +62,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -116,17 +110,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
+3 -173
View File
@@ -441,19 +441,10 @@ A grab-bag of small gotchas that otherwise turn into long debugging sessions.
| Package | Feature unlocked | | Package | Feature unlocked |
|---------|-----------------| |---------|-----------------|
| `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. | | `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
| `kokoro`, `soundfile` | Local Kokoro-82M text-to-speech on a CUDA GPU. The pinned Kokoro release supports Odysseus installs on Python 3.11-3.12; these packages are intentionally skipped on Python 3.13+ (including the Python 3.14 container image). |
| `ddgs` | DuckDuckGo as a search provider option. | | `ddgs` | DuckDuckGo as a search provider option. |
| `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) | | `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
| `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). | | `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
Install the optional set only when you need these features:
```bash
pip install -r requirements-optional.txt
```
The default Docker image currently uses Python 3.14, while Kokoro 0.9.4 declares Python `>=3.10,<3.13`. Odysseus itself continues to support Python 3.11+, but this pinned optional local-TTS feature requires a native Python 3.11 or 3.12 environment. Kokoro declares `torch`, but the local provider only activates when that torch build has CUDA and a GPU is visible; install the CUDA build appropriate for your host. Browser and configured endpoint TTS remain available on Python 3.13+ and in the container image.
### Faster, reproducible installs with uv (optional) ### Faster, reproducible installs with uv (optional)
[uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the [uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use: venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
@@ -484,7 +475,7 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep `AUTH_ENABLED=true` for any network-accessible deployment. - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development. - Keep `LOCALHOST_BYPASS=false` outside local development.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Use `SECURE_COOKIES=true` to force it on for a proxy whose scheme Odysseus cannot see, or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS. - Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer. - Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
- Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default. - Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
- Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin. - Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
@@ -495,14 +486,6 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer. - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
- Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged. - Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
> **Upgrading an existing install:** `SECURE_COOKIES` used to default to
> `false`, so an install set up before scheme derivation may still carry
> `SECURE_COOKIES=false` in its own `.env`. That explicit value stays
> authoritative, so HTTPS logins keep getting a non-`Secure` session cookie.
> Pulling this change updates the tracked Compose files, but nothing rewrites
> your `.env` — drop the line from it unless you deliberately serve plain HTTP
> alongside HTTPS and want the escape hatch.
### Private or proxied deployments ### Private or proxied deployments
Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is: Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
@@ -511,162 +494,9 @@ Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and th
3. Put the authenticated Odysseus web/API entrypoint behind that layer. 3. Put the authenticated Odysseus web/API entrypoint behind that layer.
4. Keep raw service and model ports internal-only. 4. Keep raw service and model ports internal-only.
Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true` and `LOCALHOST_BYPASS=false`. Any proxy that forwards `X-Forwarded-Proto: https` gets `Secure` session cookies without configuration, so `SECURE_COOKIES` only needs setting when you want to override that — force it on for a proxy that forwards no scheme at all, or off while you still serve plain HTTP. Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
`ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry. `ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
#### Faster over the network: HTTP/2
The frontend is raw ES modules with no bundler, so a page load is a few hundred
small same-origin requests. Over HTTP/1.1 browsers typically allow only a small
number of concurrent connections per host (commonly around six), so many of
those requests are serialized across multiple round trips. On localhost that
costs almost nothing. Over a LAN, VPN, or remote link it can become a major
part of load time, especially as latency increases.
HTTP/2 multiplexes them onto one connection and the serialisation disappears.
Odysseus needs no changes for this — uvicorn keeps speaking HTTP/1.1 on
loopback and the proxy speaks HTTP/2 to the browser. Mainstream browsers
negotiate HTTP/2 for normal web pages over TLS; they do not use the cleartext
h2c mode here, so browser-facing HTTP/2 requires a certificate. The
`--ssl-certfile` route in *HTTPS + LAN/Tailscale exposure* above gives you
HTTPS but not HTTP/2 — uvicorn does not speak it.
**1. Install Caddy.** See the [install docs](https://caddyserver.com/docs/install)
for your platform; on macOS, `brew install caddy`.
**2. Write a `Caddyfile`.** Pick the block that matches how you reach the
machine. Replace `7000` if Odysseus listens elsewhere — the macOS start script
uses `7860`.
Public domain, Caddy obtains and renews the certificate itself:
```
odysseus.example.com {
reverse_proxy 127.0.0.1:7000
}
```
Tailscale, no public DNS needed — `tailscale cert` issues a browser-trusted
certificate for a tailnet name and writes `<domain>.crt` and `<domain>.key`:
```bash
tailscale cert myhost.tailnet-name.ts.net
```
```
myhost.tailnet-name.ts.net {
tls /path/to/myhost.tailnet-name.ts.net.crt /path/to/myhost.tailnet-name.ts.net.key
reverse_proxy 127.0.0.1:7000
}
```
LAN with your own certificate — same shape, your own files:
```
odysseus.lan {
tls /path/to/cert.pem /path/to/key.pem
reverse_proxy 127.0.0.1:7000
}
```
Give `tls` absolute paths: a service starts in a working directory you did not
choose. If port 443 is already taken, append a port to the site address
(`odysseus.example.com:8443`) and use it in the URL. That alone does not free
port 80 — Caddy still binds it for the HTTP-to-HTTPS redirect, and fails to
start with `listen tcp :80: bind: address already in use` if something else
holds it. Turn the redirect off with a global block at the top of the file:
```
{
auto_https disable_redirects
}
```
**3. Run it in the foreground first:**
```bash
caddy run --config ./Caddyfile
```
Once that works, run it as a service:
```bash
brew services start caddy # macOS — reads $(brew --prefix)/etc/Caddyfile, not ./Caddyfile
sudo systemctl enable --now caddy # Linux, if your package installed the unit
```
Odysseus's own service is unchanged; the proxy runs alongside it. Under Docker,
run the proxy as another container, or on the host pointing at the published
port.
**4. Point Odysseus at the new origin** in `.env`, then restart it.
A proxy that exposes the HTTPS request scheme to Odysseus needs no `SECURE_COOKIES` setting. Only force it on when the proxy cannot expose that scheme:
```bash
# only if the proxy cannot expose the external HTTPS scheme to Odysseus:
SECURE_COOKIES=true
# only if you use remote MCP servers with OAuth:
OAUTH_REDIRECT_BASE_URL=https://odysseus.example.com
```
Gmail OAuth needs nothing here when the proxy runs on the same host: the
redirect URI is built from the incoming request, and uvicorn rewrites the
scheme from `X-Forwarded-Proto` for proxies it trusts — by default only
`127.0.0.1`. A proxy in a separate container or on another machine is not
trusted, so pin the URI there:
```bash
GOOGLE_OAUTH_REDIRECT_URI=https://odysseus.example.com/api/email/oauth/google/callback
```
(uvicorn's own `FORWARDED_ALLOW_IPS` widens that trust, but it has to be in the
environment uvicorn starts with — `.env` is read by the app afterwards, too
late for it to take effect.)
**5. Confirm HTTP/2 is really on:**
```bash
curl -s -o /dev/null -w '%{http_version}\n' https://odysseus.example.com/
# 2
```
The status code is not the thing to check here — a logged-out request redirects
to the login page, so `curl -I` shows `HTTP/2 302`, and the `HTTP/2` prefix is
the part that matters. The browser reports the same in the Network panel's
Protocol column (`h2`); in Chrome and Firefox that column is hidden until you
enable it by right-clicking the column headers.
Three things bite when moving an existing install behind TLS:
- Leave `SECURE_COOKIES` unset when Odysseus can see the external HTTPS scheme;
the cookie then follows the request automatically. If your proxy cannot expose
that scheme, set `SECURE_COOKIES=true` **at the same time** you stop serving
plain HTTP, not before. An explicit `true` applies to every login, so while an
HTTP entrypoint is still reachable the browser will reject the `Secure` cookie
there and login will appear to loop.
- `OAUTH_REDIRECT_BASE_URL` defaults to `http://localhost:7000`. Unlike the
Gmail redirect URI it cannot be derived from a request — it is registered
with each MCP authorization server up front — so set it to the external
origin if you use remote MCP servers over OAuth.
- Odysseus sends `Strict-Transport-Security` once it sees `X-Forwarded-Proto:
https`. HSTS applies to the whole hostname and ignores the port, so any other
plain-HTTP service on that same hostname becomes unreachable in browsers that
have visited Odysseus. Give Odysseus its own hostname, or strip the header at
the proxy (`header_down -Strict-Transport-Security` in Caddy).
Server-sent events are not buffered by this configuration, so chat streaming
arrives token by token; add `flush_interval -1` inside the `reverse_proxy`
block if you want that pinned explicitly. nginx needs `proxy_buffering off;`
for the same reason.
Changing the external origin also affects state scoped to it. Service workers
and their caches are origin-scoped, so moving to a different origin starts with
a cold load. Cookies follow their own domain/path/security rules rather than
being port-scoped: changing the hostname normally requires a new login, while
changing only the scheme or port does not by itself guarantee that existing
cookies disappear.
Common internal-only ports from the default docs/compose setup: Common internal-only ports from the default docs/compose setup:
| Port | Service | | Port | Service |
@@ -697,7 +527,7 @@ Key settings:
| `AUTH_ENABLED` | `true` | Enable/disable login | | `AUTH_ENABLED` | `true` | Enable/disable login |
| `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. | | `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
| `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. | | `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
| `SECURE_COOKIES` | derived from the request scheme | Marks session cookies `Secure` on HTTPS requests. Set true to force it on, false to force it off. | | `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
| `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string | | `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
| `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. | | `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
| `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. | | `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
-4
View File
@@ -163,10 +163,6 @@ if (Test-Path $cudaBase) {
} }
# 7. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH) # 7. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH)
# -Port only reaches uvicorn as a flag. Everything that builds a URL for this
# instance - internal_api_base(), companion pairing, the MCP OAuth callback -
# reads APP_PORT, so set it too or they all assume 7000.
$env:APP_PORT = $Port
Write-Step ("Starting Odysseus at http://{0}:{1}" -f $BindHost, $Port) Write-Step ("Starting Odysseus at http://{0}:{1}" -f $BindHost, $Port)
Write-Host "Press Ctrl+C to stop." Write-Host "Press Ctrl+C to stop."
Write-Host "" Write-Host ""
-21
View File
@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2013-2020 Khan Academy and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-21
View File
@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2014 - 2022 Knut Sveidqvist
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+8
View File
@@ -1802,6 +1802,7 @@ async def _ai_draft_reply_to_email(uid, folder="INBOX", reply_all=False, account
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_endpoint, resolve_endpoint,
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
except Exception as exc: except Exception as exc:
@@ -1842,6 +1843,13 @@ async def _ai_draft_reply_to_email(uid, folder="INBOX", reply_all=False, account
utility_fallbacks = resolve_utility_fallback_candidates() or [] utility_fallbacks = resolve_utility_fallback_candidates() or []
for cand in utility_fallbacks: for cand in utility_fallbacks:
_add(*cand) _add(*cand)
try:
chat_fallbacks = resolve_chat_fallback_candidates(owner=None) or []
except TypeError:
chat_fallbacks = resolve_chat_fallback_candidates() or []
for cand in chat_fallbacks:
_add(*cand)
if not candidates: if not candidates:
return {"error": "No LLM endpoint configured for AI reply"} return {"error": "No LLM endpoint configured for AI reply"}
-10
View File
@@ -12,16 +12,6 @@
# GPU-accelerated transcription — it's auto-detected, CPU is used otherwise. # GPU-accelerated transcription — it's auto-detected, CPU is used otherwise.
faster-whisper faster-whisper
# Local text-to-speech via Kokoro-82M for the "local" TTS provider.
# Kokoro 0.9.4 declares Python >=3.10,<3.13; Odysseus itself requires 3.11+,
# so pip installs these extras on 3.11-3.12 and deliberately skips them on
# Python 3.13+ (including the Python 3.14 container image). Kokoro declares
# torch; the local provider still
# requires a CUDA-enabled torch build and GPU at runtime. SoundFile is separate
# in Kokoro's official install instructions and is not a transitive dependency.
kokoro==0.9.4; python_version >= "3.11" and python_version < "3.13"
soundfile; python_version >= "3.11" and python_version < "3.13"
# DuckDuckGo as a search provider option. # DuckDuckGo as a search provider option.
# Install if you want DDG in the search-provider dropdown. # Install if you want DDG in the search-provider dropdown.
# Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE. # Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE.
+3 -4
View File
@@ -16,7 +16,7 @@ from pydantic import BaseModel
from core.database import SessionLocal, CrewMember, ScheduledTask from core.database import SessionLocal, CrewMember, ScheduledTask
from src.auth_helpers import get_current_user from src.auth_helpers import get_current_user
from src.owner_identity import REQUEST_SENTINEL_OWNERS from core.auth import RESERVED_USERNAMES
from src.task_scheduler import compute_next_run from src.task_scheduler import compute_next_run
@@ -90,12 +90,11 @@ def setup_assistant_routes(task_scheduler) -> APIRouter:
# check-in tasks seeded. Hitting any /assistant route under one of these # check-in tasks seeded. Hitting any /assistant route under one of these
# used to seed a full CrewMember + Morning/Midday/Evening tasks under that # used to seed a full CrewMember + Morning/Midday/Evening tasks under that
# owner, which then double-fired alongside the real user's check-ins. # owner, which then double-fired alongside the real user's check-ins.
# REQUEST_SENTINEL_OWNERS covers request-only identities; Default/Local is a # RESERVED_USERNAMES covers the same set; the `not owner` guard handles "".
# reserved login name but remains a valid storage owner.
async def _get_or_create(owner: str) -> CrewMember: async def _get_or_create(owner: str) -> CrewMember:
"""Return the per-owner assistant CrewMember, creating it on demand.""" """Return the per-owner assistant CrewMember, creating it on demand."""
if not owner or owner in REQUEST_SENTINEL_OWNERS: if not owner or owner in RESERVED_USERNAMES:
raise HTTPException(status_code=400, detail=f"Cannot seed assistant for {owner!r}") raise HTTPException(status_code=400, detail=f"Cannot seed assistant for {owner!r}")
db = SessionLocal() db = SessionLocal()
try: try:
+3 -34
View File
@@ -22,8 +22,6 @@ from src.settings import (
load_features as _load_features, load_features as _load_features,
save_features as _save_features, save_features as _save_features,
DEFAULT_SETTINGS, DEFAULT_SETTINGS,
RETIRED_SETTING_KEYS,
without_retired_settings,
) )
from src.integrations import ( from src.integrations import (
load_integrations, load_integrations,
@@ -86,33 +84,6 @@ class SetOpenRegistrationRequest(BaseModel):
SESSION_COOKIE = "odysseus_session" SESSION_COOKIE = "odysseus_session"
def _secure_cookie(request: Request) -> bool:
"""Decide the ``Secure`` attribute of the session cookie.
``SECURE_COOKIES`` stays authoritative when it holds an explicit value:
``true`` always marks the cookie Secure (the documented knob for a TLS
proxy), ``false`` never does, which is the escape hatch for an install
that still answers on plain HTTP alongside HTTPS. Anything else
unset, or the present-but-empty value docker-compose injects for a
variable the host has not defined derives it from the request, so an
HTTPS login gets a Secure cookie without any configuration.
Either the connection scheme or ``X-Forwarded-Proto`` saying https is
enough, which is the same test ``core/middleware.py`` applies before it
sends HSTS. Uvicorn's proxy-headers middleware already folds that header
into the scheme for the proxies it trusts, so reading it here only adds
the case of a terminator that is not on a trusted address; the cost is
that a client talking to the app directly can set the header and lock
its own session out over plain HTTP.
"""
configured = os.getenv("SECURE_COOKIES", "").strip().lower()
if configured in ("true", "false"):
return configured == "true"
# A chained proxy sends a list — the client-facing hop comes first.
forwarded_proto = request.headers.get("x-forwarded-proto", "").split(",")[0]
return request.url.scheme == "https" or forwarded_proto.strip().lower() == "https"
def setup_auth_routes(auth_manager: AuthManager) -> APIRouter: def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
router = APIRouter(prefix="/api/auth", tags=["auth"]) router = APIRouter(prefix="/api/auth", tags=["auth"])
@@ -186,7 +157,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
value=token, value=token,
httponly=True, httponly=True,
samesite="lax", samesite="lax",
secure=_secure_cookie(request), secure=os.getenv("SECURE_COOKIES", "false").lower() == "true",
path="/", path="/",
) )
if body.remember: if body.remember:
@@ -718,7 +689,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
a scrubbed copy with secret keys blanked. The frontend uses this a scrubbed copy with secret keys blanked. The frontend uses this
for keybinds + TTS prefs, so it stays callable without admin.""" for keybinds + TTS prefs, so it stays callable without admin."""
user = _get_current_user(request) user = _get_current_user(request)
settings = without_retired_settings(_load_settings()) settings = _load_settings()
if user and auth_manager.is_admin(user): if user and auth_manager.is_admin(user):
return settings return settings
return scrub_settings(settings) return scrub_settings(settings)
@@ -738,8 +709,6 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
"agent_max_tool_calls": (0, 1000), # 0 = unlimited "agent_max_tool_calls": (0, 1000), # 0 = unlimited
} }
for key in DEFAULT_SETTINGS: for key in DEFAULT_SETTINGS:
if key in RETIRED_SETTING_KEYS:
continue
if key not in body: if key not in body:
continue continue
val = body[key] val = body[key]
@@ -752,7 +721,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
val = max(lo, min(val, hi)) val = max(lo, min(val, hi))
current[key] = val current[key] = val
_save_settings(current) _save_settings(current)
return without_retired_settings(current) return current
# ---- Integrations CRUD ---- # ---- Integrations CRUD ----
+105 -67
View File
@@ -15,7 +15,7 @@ from core.database import Session as DBSession, ModelEndpoint
from src.llm_core import normalize_model_id from src.llm_core import normalize_model_id
from src.endpoint_resolver import normalize_base from src.endpoint_resolver import normalize_base
from src.context_compactor import maybe_compact, trim_for_context from src.context_compactor import maybe_compact, trim_for_context
from src.model_context import estimate_tokens, get_context_length from src.model_context import estimate_tokens
from src.auth_helpers import effective_user from src.auth_helpers import effective_user
from src.prompt_security import untrusted_context_message from src.prompt_security import untrusted_context_message
from src.attachment_refs import attachment_ref from src.attachment_refs import attachment_ref
@@ -152,38 +152,10 @@ class ChatContext:
# Uploads attached to this user turn, resolved and owner-checked for the # Uploads attached to this user turn, resolved and owner-checked for the
# agent's private context. This is not emitted to the browser. # agent's private context. This is not emitted to the browser.
uploaded_files: list = field(default_factory=list) uploaded_files: list = field(default_factory=list)
# Route-neutral prompt before any model-window compaction/trimming. This is
# retained only when explicit foreground fallbacks are enabled so each
# concrete candidate can apply its own context budget independently.
route_messages: list = field(default_factory=list)
# ── Helpers ────────────────────────────────────────────────────────────── # # ── Helpers ────────────────────────────────────────────────────────────── #
def _allowed_models_from_privileges(privs: dict) -> Optional[frozenset[str]]:
if privs.get("block_all_models"):
return frozenset()
allowed_raw = privs.get("allowed_models")
allowed = allowed_raw if isinstance(allowed_raw, list) else []
restricted = bool(privs.get("allowed_models_restricted")) or bool(allowed)
return frozenset(model for model in allowed if isinstance(model, str)) if restricted else None
def _allowed_models_for_request(request) -> Optional[frozenset[str]]:
"""Return the caller's model allowlist, or ``None`` when unrestricted."""
try:
user = effective_user(request)
except Exception:
user = None
if not user:
return None
auth_manager = getattr(getattr(request.app, "state", None), "auth_manager", None)
if not auth_manager:
return None
privs = auth_manager.get_privileges(user) or {}
return _allowed_models_from_privileges(privs)
def _enforce_chat_privileges(request, sess) -> None: def _enforce_chat_privileges(request, sess) -> None:
"""Apply the per-user privilege gates (allowed_models + max_messages_per_day) """Apply the per-user privilege gates (allowed_models + max_messages_per_day)
that both /api/chat and /api/chat_stream must enforce BEFORE any LLM work. that both /api/chat and /api/chat_stream must enforce BEFORE any LLM work.
@@ -213,8 +185,10 @@ def _enforce_chat_privileges(request, sess) -> None:
if privs.get("block_all_models"): if privs.get("block_all_models"):
raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.") raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.")
allowed_models = _allowed_models_from_privileges(privs) allowed_raw = privs.get("allowed_models")
if allowed_models is not None and sess.model and sess.model not in allowed_models: allowed = allowed_raw if isinstance(allowed_raw, list) else []
restricted = bool(privs.get("allowed_models_restricted")) or bool(allowed)
if restricted and sess.model and sess.model not in allowed:
raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.") raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.")
cap = int(privs.get("max_messages_per_day") or 0) cap = int(privs.get("max_messages_per_day") or 0)
@@ -313,6 +287,96 @@ async def auto_name_session(session_manager, sess):
logger.error(f"Auto-name failed for {sess.id}: {e}\n{traceback.format_exc()}") logger.error(f"Auto-name failed for {sess.id}: {e}\n{traceback.format_exc()}")
def try_fallback_endpoint(sess, session_id: str) -> dict | None:
"""Find an alternative working endpoint when the current one fails.
Returns {"model": ..., "endpoint_url": ..., "endpoint_name": ...} or None.
"""
import requests as _req
from src.endpoint_resolver import (
build_chat_url,
build_headers,
build_models_url,
normalize_base,
resolve_endpoint_runtime,
)
from src.chatgpt_subscription import is_chatgpt_subscription_base
current_url = sess.endpoint_url or ""
owner = getattr(sess, "owner", None)
db = SessionLocal()
try:
q = db.query(ModelEndpoint).filter(
ModelEndpoint.is_enabled == True
)
if owner:
from src.auth_helpers import owner_filter
q = owner_filter(q, ModelEndpoint, owner)
endpoints = q.all()
finally:
db.close()
for ep in endpoints:
base = normalize_base(ep.base_url)
# Skip current endpoint
if current_url and base in current_url:
continue
try:
base, api_key = resolve_endpoint_runtime(ep, owner=owner)
except Exception:
continue
ping_url = build_models_url(base)
headers = build_headers(api_key, base)
try:
if ping_url:
r = _req.get(ping_url, headers=headers, timeout=5)
r.raise_for_status()
data = r.json()
models = [m.get("id") for m in (data.get("data") or []) if m.get("id")]
if not models:
models = [
m.get("name") or m.get("model")
for m in (data.get("models") or [])
if m.get("name") or m.get("model")
]
else:
models = json.loads(ep.cached_models or "[]")
if not models:
continue
# Found a working endpoint — update session
new_model = models[0]
chat_url = build_chat_url(base)
new_headers = build_headers(api_key, base)
persisted_headers = {} if is_chatgpt_subscription_base(base) else new_headers
sess.model = new_model
sess.endpoint_url = chat_url
sess.headers = new_headers
# Persist
_db = SessionLocal()
try:
_db.query(DBSession).filter(DBSession.id == session_id).update({
"model": new_model,
"endpoint_url": chat_url,
"headers": persisted_headers,
})
_db.commit()
finally:
_db.close()
logger.info(f"Fallback: switched session {session_id} from {current_url} to {ep.name} ({new_model})")
return {
"model": new_model,
"endpoint_url": chat_url,
"endpoint_name": ep.name,
}
except Exception:
continue
return None
def extract_preset(chat_handler, preset_id) -> PresetInfo: def extract_preset(chat_handler, preset_id) -> PresetInfo:
"""Extract preset parameters via chat_handler.""" """Extract preset parameters via chat_handler."""
temperature, max_tokens, system_prompt, char_name = ( temperature, max_tokens, system_prompt, char_name = (
@@ -623,9 +687,6 @@ async def build_chat_context(
use_enhanced_message: bool = False, use_enhanced_message: bool = False,
agent_mode: bool = False, agent_mode: bool = False,
allow_tool_preprocessing: bool = True, allow_tool_preprocessing: bool = True,
defer_context_shaping: bool = False,
continuation_context_message: str | None = None,
persist_user_message: bool = True,
) -> ChatContext: ) -> ChatContext:
"""Build the full context (preface + messages) for an LLM call. """Build the full context (preface + messages) for an LLM call.
@@ -649,14 +710,14 @@ async def build_chat_context(
# Add user message to history. Nobody/incognito uses a request-local # Add user message to history. Nobody/incognito uses a request-local
# transcript store instead of session history so stale saved chats cannot # transcript store instead of session history so stale saved chats cannot
# bleed into context and the turn is not persisted. # bleed into context and the turn is not persisted.
if persist_user_message and incognito: if incognito:
user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None
_append_incognito_message(session_id, "user", preprocessed.user_content, user_meta) _append_incognito_message(session_id, "user", preprocessed.user_content, user_meta)
elif persist_user_message: else:
add_user_message(sess, chat_handler, preprocessed, incognito=False) add_user_message(sess, chat_handler, preprocessed, incognito=False)
# Fire events # Fire events
if persist_user_message and not incognito: if not incognito:
fire_message_event(request, webhook_manager, session_id, sess, message, compare_mode) fire_message_event(request, webhook_manager, session_id, sess, message, compare_mode)
# Resolve owner-scoped prefs/context. Browser requests keep the cookie user; # Resolve owner-scoped prefs/context. Browser requests keep the cookie user;
@@ -668,12 +729,7 @@ async def build_chat_context(
getattr(chat_handler, "upload_handler", None), getattr(chat_handler, "upload_handler", None),
getattr(sess, "owner", None), getattr(sess, "owner", None),
) )
context_message = ( casual_low_signal = _is_casual_low_signal(message)
str(continuation_context_message).strip()
if continuation_context_message
else message
)
casual_low_signal = _is_casual_low_signal(context_message)
# Memory enabled? # Memory enabled?
mem_enabled = not incognito and not no_memory and uprefs.get("memory_enabled", True) mem_enabled = not incognito and not no_memory and uprefs.get("memory_enabled", True)
@@ -710,15 +766,7 @@ async def build_chat_context(
# Build context preface # Build context preface
# The stream path uses enhanced_message (with CoT/preprocessing applied), # The stream path uses enhanced_message (with CoT/preprocessing applied),
# the sync path uses text_for_context. # the sync path uses text_for_context.
_ctx_msg = ( _ctx_msg = preprocessed.enhanced_message if use_enhanced_message else preprocessed.text_for_context
context_message
if continuation_context_message
else (
preprocessed.enhanced_message
if use_enhanced_message
else preprocessed.text_for_context
)
)
_preface_kwargs = dict( _preface_kwargs = dict(
message=_ctx_msg, message=_ctx_msg,
session=sess, session=sess,
@@ -782,22 +830,13 @@ async def build_chat_context(
except Exception: except Exception:
logger.debug("Failed to add current date/time context", exc_info=True) logger.debug("Failed to add current date/time context", exc_info=True)
route_messages = list(messages) # Auto-compact
# Explicit fallback routing must shape from the same route-neutral prompt messages, context_length, was_compacted = await maybe_compact(
# for every candidate. Running selected-model compaction here would mutate sess, sess.endpoint_url, sess.model, messages, sess.headers, owner=user,
# session history before we know which route can answer and would make a )
# later larger-context candidate unable to recover discarded history.
if defer_context_shaping:
context_length = get_context_length(sess.endpoint_url, sess.model)
was_compacted = False
else:
messages, context_length, was_compacted = await maybe_compact(
sess, sess.endpoint_url, sess.model, messages, sess.headers, owner=user,
)
_before_trim_messages = len(messages) _before_trim_messages = len(messages)
_before_trim_tokens = estimate_tokens(messages) _before_trim_tokens = estimate_tokens(messages)
if not defer_context_shaping: messages = trim_for_context(messages, context_length)
messages = trim_for_context(messages, context_length)
_after_trim_messages = len(messages) _after_trim_messages = len(messages)
_after_trim_tokens = estimate_tokens(messages) _after_trim_tokens = estimate_tokens(messages)
_context_trimmed = _after_trim_messages < _before_trim_messages or _after_trim_tokens < _before_trim_tokens _context_trimmed = _after_trim_messages < _before_trim_messages or _after_trim_tokens < _before_trim_tokens
@@ -821,7 +860,6 @@ async def build_chat_context(
context_tokens_after_trim=_after_trim_tokens, context_tokens_after_trim=_after_trim_tokens,
auto_opened_docs=auto_opened_docs, auto_opened_docs=auto_opened_docs,
uploaded_files=uploaded_files, uploaded_files=uploaded_files,
route_messages=route_messages,
) )
+46 -769
View File
File diff suppressed because it is too large Load Diff
-35
View File
@@ -1204,41 +1204,6 @@ def _safe_env_prefix(ep: str | None) -> str | None:
return f'[ -f "{path}" ] && source "{path}" || true' return f'[ -f "{path}" ] && source "{path}" || true'
def _local_windows_bash_env_prefix(ep: str | None) -> str | None:
"""Convert a frontend PowerShell venv prefix for the local Git Bash runner."""
if not ep:
return ep
prefix = ep.strip()
if not prefix.startswith("&"):
return ep
raw_path = prefix[1:].lstrip()
if not raw_path:
return ep
if raw_path.startswith("'"):
if len(raw_path) < 2 or not raw_path.endswith("'"):
return ep
quoted_path = raw_path[1:-1]
if "'" in quoted_path.replace("''", ""):
return ep
path = quoted_path.replace("''", "'")
else:
path = raw_path.rstrip()
if "'" in path or '"' in path:
return ep
if any(c in path for c in "\r\n;&|`$<>"):
return ep
if not path.replace("\\", "/").casefold().endswith("/scripts/activate.ps1"):
return ep
bash_path = _git_bash_path(path)
if "\\" in bash_path:
return ep
bash_path = bash_path[: -len("Activate.ps1")] + "activate"
return "source " + shlex.quote(bash_path)
def _ssh_ps(host, script_path, port=None): def _ssh_ps(host, script_path, port=None):
"""Build SSH command to run a PowerShell script on a Windows remote.""" """Build SSH command to run a PowerShell script on a Windows remote."""
pf = f"-p {port} " if port and port != "22" else "" pf = f"-p {port} " if port and port != "22" else ""
+3 -3
View File
@@ -50,7 +50,7 @@ from routes.cookbook_helpers import (
_SESSION_ID_RE, _validate_repo_id, _validate_serve_model_id, _validate_include, _validate_token, _SESSION_ID_RE, _validate_repo_id, _validate_serve_model_id, _validate_include, _validate_token,
_validate_local_dir, _validate_gpus, _shell_path, _validate_local_dir, _validate_gpus, _shell_path,
_ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase, OLLAMA_MISSING_HINT, _ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase, OLLAMA_MISSING_HINT,
_safe_env_prefix, _local_windows_bash_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines, _safe_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
_append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script, _append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script,
load_stored_hf_token, load_stored_hf_token,
_append_vllm_linux_preflight_lines, _ollama_bind_from_cmd, _pip_install_fallback_chain, _append_vllm_linux_preflight_lines, _ollama_bind_from_cmd, _pip_install_fallback_chain,
@@ -1336,7 +1336,7 @@ def setup_cookbook_routes() -> APIRouter:
# Local: run hf download in the background (tmux on POSIX, a detached # Local: run hf download in the background (tmux on POSIX, a detached
# process + logfile on Windows where tmux doesn't exist). # process + logfile on Windows where tmux doesn't exist).
if req.env_prefix: if req.env_prefix:
lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix)) lines.append(_safe_env_prefix(req.env_prefix))
else: else:
lines.append("deactivate 2>/dev/null; hash -r") lines.append("deactivate 2>/dev/null; hash -r")
# Show whether the HF token reached this run (masked) — tells a gated # Show whether the HF token reached this run (masked) — tells a gated
@@ -2166,7 +2166,7 @@ def setup_cookbook_routes() -> APIRouter:
if req.gpus: if req.gpus:
runner_lines.append(f"export CUDA_VISIBLE_DEVICES='{req.gpus}'") runner_lines.append(f"export CUDA_VISIBLE_DEVICES='{req.gpus}'")
if req.env_prefix: if req.env_prefix:
runner_lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix)) runner_lines.append(_safe_env_prefix(req.env_prefix))
else: else:
runner_lines.append("deactivate 2>/dev/null; hash -r") runner_lines.append("deactivate 2>/dev/null; hash -r")
_append_venv_nvidia_library_path_lines(runner_lines, cmd=req.cmd) _append_venv_nvidia_library_path_lines(runner_lines, cmd=req.cmd)
+10 -3
View File
@@ -5004,6 +5004,7 @@ def setup_email_routes():
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_endpoint, resolve_endpoint,
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
@@ -5065,6 +5066,8 @@ def setup_email_routes():
pass pass
for cand in resolve_utility_fallback_candidates(owner=owner) or []: for cand in resolve_utility_fallback_candidates(owner=owner) or []:
_add(*cand) _add(*cand)
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
_add(*cand)
if not candidates: if not candidates:
return {"success": False, "error": "No LLM endpoint configured"} return {"success": False, "error": "No LLM endpoint configured"}
@@ -5324,11 +5327,13 @@ def setup_email_routes():
# Build a candidate chain so a stale session-stored API key # Build a candidate chain so a stale session-stored API key
# (the most common cause of "authentication failed" here) # (the most common cause of "authentication failed" here)
# doesn't kill AI Reply outright — fall through to the # doesn't kill AI Reply outright — fall through to the
# user's Utility / Default endpoints and active Utility fallback # user's Utility / Default endpoints and the active Utility
# chain. Dedupe by url+model so we don't retry the same endpoint. # fallback chain. The retired default-fallback hook stays empty.
# Dedupe by url+model so we don't retry the same broken endpoint.
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
_seen = set() _seen = set()
_candidates = [] _candidates = []
@@ -5353,9 +5358,11 @@ def setup_email_routes():
_add(_d_url, _d_model, _d_headers) _add(_d_url, _d_model, _d_headers)
except Exception: except Exception:
pass pass
# Active Utility fallbacks last. # Active Utility fallbacks, then the retired default hook.
for cand in resolve_utility_fallback_candidates(owner=owner) or []: for cand in resolve_utility_fallback_candidates(owner=owner) or []:
_add(*cand) _add(*cand)
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
_add(*cand)
_messages = [ _messages = [
{"role": "system", "content": system_prompt}, {"role": "system", "content": system_prompt},
{"role": "user", "content": user_msg}, {"role": "user", "content": user_msg},
+7 -13
View File
@@ -475,7 +475,7 @@ def setup_mcp_routes(mcp_manager: McpManager):
return RedirectResponse(auth_url) return RedirectResponse(auth_url)
else: else:
# Remote device — show paste-back page # Remote device — show paste-back page
return HTMLResponse(_oauth_authorize_page(auth_url, server_id, redirect_uri)) return HTMLResponse(_oauth_authorize_page(auth_url, server_id, host, redirect_uri))
finally: finally:
db.close() db.close()
@@ -612,13 +612,15 @@ def setup_mcp_routes(mcp_manager: McpManager):
def _oauth_authorize_page( def _oauth_authorize_page(
auth_url: str, auth_url: str,
server_id: str, server_id: str,
redirect_uri: str, host: str,
redirect_uri: str = "http://localhost:7000/api/mcp/oauth/callback",
) -> str: ) -> str:
"""Page with Google sign-in link and URL paste-back form for remote access.""" """Page with Google sign-in link and URL paste-back form for remote access."""
# Escape values interpolated into the page: `server_id` comes from the OAuth # Escape values interpolated into the page: `host` comes from the request
# state and is not trusted. # Host header and `server_id` from the OAuth state — neither is trusted.
auth_url = html.escape(auth_url, quote=True) auth_url = html.escape(auth_url, quote=True)
server_id = html.escape(server_id, quote=True) server_id = html.escape(server_id, quote=True)
host = html.escape(host, quote=True)
redirect_uri = html.escape(redirect_uri, quote=True) redirect_uri = html.escape(redirect_uri, quote=True)
return f"""<!DOCTYPE html> return f"""<!DOCTYPE html>
<html><head> <html><head>
@@ -662,15 +664,7 @@ def _oauth_authorize_page(
</div> </div>
<a class="auth-link" href="{auth_url}" target="_blank" rel="noopener">Sign in with Google</a> <a class="auth-link" href="{auth_url}" target="_blank" rel="noopener">Sign in with Google</a>
<div class="divider"></div> <div class="divider"></div>
<!-- Relative action: the browser resolves it against the origin this page was <form method="POST" action="http://{host}/api/mcp/oauth/exchange/{server_id}">
served from, so the form follows the user through any proxy without the
app having to know the scheme or the host. An absolute http:// action is
blocked as mixed content on exactly the HTTPS deployments that need
paste-back, and request.url.scheme cannot be trusted to spot them
uvicorn only honours X-Forwarded-Proto from a peer in
--forwarded-allow-ips, which defaults to 127.0.0.1 and excludes a proxy
arriving over the Docker bridge. -->
<form method="POST" action="/api/mcp/oauth/exchange/{server_id}">
<p>Paste the URL from your browser after signing in:</p> <p>Paste the URL from your browser after signing in:</p>
<input type="text" name="callback_url" placeholder="{redirect_uri}?code=..." required> <input type="text" name="callback_url" placeholder="{redirect_uri}?code=..." required>
<br><button type="submit">Connect</button> <br><button type="submit">Connect</button>
+8 -12
View File
@@ -46,7 +46,6 @@ _ENDPOINT_SETTING_FIELDS = {
} }
_ENDPOINT_FALLBACK_FIELDS = { _ENDPOINT_FALLBACK_FIELDS = {
"foreground_model_fallbacks": "Foreground Model Fallbacks",
"utility_model_fallbacks": "Utility Model Fallbacks", "utility_model_fallbacks": "Utility Model Fallbacks",
"vision_model_fallbacks": "Vision Model Fallbacks", "vision_model_fallbacks": "Vision Model Fallbacks",
} }
@@ -181,12 +180,7 @@ def _clear_user_pref_endpoint_refs(all_prefs: dict, ep_id: str) -> int:
if not isinstance(all_prefs, dict): if not isinstance(all_prefs, dict):
return 0 return 0
users = all_prefs.get("_users") users = all_prefs.get("_users")
# A mixed store can contain auth-disabled foreground policy at the root pref_sets = users.values() if isinstance(users, dict) else [all_prefs]
# alongside named-owner preferences. Both are active namespaces; legacy
# `default_model_fallbacks` remains untouched by the field allowlist.
pref_sets = [all_prefs]
if isinstance(users, dict):
pref_sets.extend(users.values())
cleared_users = 0 cleared_users = 0
for prefs in pref_sets: for prefs in pref_sets:
if isinstance(prefs, dict) and _clear_endpoint_settings_for_endpoint(prefs, ep_id): if isinstance(prefs, dict) and _clear_endpoint_settings_for_endpoint(prefs, ep_id):
@@ -1351,14 +1345,14 @@ def _legacy_visible_api_models(ep) -> List[str]:
def _picker_models_for_endpoint(ep, base_url: str, kind: str): def _picker_models_for_endpoint(ep, base_url: str, kind: str):
"""Return model IDs that should appear in the picker for an endpoint. """Return model IDs that should appear in the picker for an endpoint.
API providers expose remote inventory from /v1/models. Default to that API providers expose remote inventory from /v1/models. Treat that cache as
visible inventory until an explicit pinned-model allow-list is saved. inventory, not approval: only manually pinned API models should appear in
Local/self-hosted endpoints keep the older hide-list behavior. the picker. Local/self-hosted endpoints keep the older hide-list behavior.
""" """
pinned = _normalize_model_ids(getattr(ep, "pinned_models", None)) pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if _picker_requires_pinning(base_url, kind): if _picker_requires_pinning(base_url, kind):
if not _has_explicit_pinned_models(ep): if not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep) pinned = _legacy_visible_api_models(ep) if _hidden_model_ids(ep) else []
return pinned, pinned return pinned, pinned
return _visible_models( return _visible_models(
_cached_model_ids(ep), _cached_model_ids(ep),
@@ -2342,7 +2336,9 @@ def setup_model_routes(model_discovery):
else: else:
response.headers["X-Model-Refresh-Status"] = "failed" response.headers["X-Model-Refresh-Status"] = "failed"
response.headers["X-Model-Refresh-Warning"] = "Model refresh failed or returned no models; kept cached models." response.headers["X-Model-Refresh-Warning"] = "Model refresh failed or returned no models; kept cached models."
_, pinned = _picker_models_for_endpoint(ep, base, kind) pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if picker_requires_pinning and not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep)
pinned_set = set(pinned) pinned_set = set(pinned)
return [ return [
{ {
+11 -51
View File
@@ -7,10 +7,6 @@ from src.auth_helpers import get_current_user
from src.constants import USER_PREFS_FILE from src.constants import USER_PREFS_FILE
PREFS_FILE = USER_PREFS_FILE PREFS_FILE = USER_PREFS_FILE
_FOREGROUND_POLICY_KEYS = (
"foreground_fallback_enabled",
"foreground_model_fallbacks",
)
def _load(): def _load():
@@ -30,27 +26,14 @@ def _save(prefs):
def _load_for_user(user: Optional[str] = None) -> dict: def _load_for_user(user: Optional[str] = None) -> dict:
"""Load preferences for a specific user.""" """Load preferences for a specific user."""
all_prefs = _load() all_prefs = _load()
users = all_prefs.get("_users") if "_users" in all_prefs:
if isinstance(users, dict):
if user is None: if user is None:
# Auth disabled — return first user's prefs for backward compat # Auth disabled — return first user's prefs for backward compat
prefs = dict(next(iter(users.values()), {})) users = all_prefs["_users"]
# Foreground fallback consent is never borrowed from a named return dict(next(iter(users.values()), {}))
# owner. Auth-disabled operation has a separate flat/root opt-in return dict(all_prefs["_users"].get(user, {}))
# that remains inert when authentication is enabled again. # Legacy flat format — return as-is
for key in _FOREGROUND_POLICY_KEYS: return dict(all_prefs)
prefs.pop(key, None)
if key in all_prefs:
prefs[key] = all_prefs[key]
return prefs
prefs = users.get(user, {})
return dict(prefs) if isinstance(prefs, dict) else {}
# A legacy flat store belongs only to auth-disabled single-user mode.
# Copying it into the first named user's new `_users` record during an
# auth transition would silently transfer another user's preferences and,
# critically, foreground fallback consent. Named owners therefore start
# with an empty record and must write their own preferences explicitly.
return dict(all_prefs) if user is None else {}
def _save_for_user(user: Optional[str], prefs: dict): def _save_for_user(user: Optional[str], prefs: dict):
@@ -62,40 +45,17 @@ def _save_for_user(user: Optional[str], prefs: dict):
# `prefs` flat would overwrite the whole `_users` map and destroy every # `prefs` flat would overwrite the whole `_users` map and destroy every
# other user's preferences. Instead write back into the same (first) # other user's preferences. Instead write back into the same (first)
# slot _load_for_user(None) reads from, preserving the others. # slot _load_for_user(None) reads from, preserving the others.
users = all_prefs.get("_users") if "_users" in all_prefs:
if isinstance(users, dict): users = all_prefs["_users"]
first_key = next(iter(users), None) first_key = next(iter(users), None)
if first_key is not None: if first_key is not None:
existing_named = users.get(first_key) users[first_key] = prefs
existing_named = (
dict(existing_named)
if isinstance(existing_named, dict)
else {}
)
named_foreground = {
key: existing_named[key]
for key in _FOREGROUND_POLICY_KEYS
if key in existing_named
}
users[first_key] = {
key: value
for key, value in prefs.items()
if key not in _FOREGROUND_POLICY_KEYS
}
users[first_key].update(named_foreground)
for key in _FOREGROUND_POLICY_KEYS:
if key in prefs:
all_prefs[key] = prefs[key]
_save(all_prefs) _save(all_prefs)
return return
_save(prefs) _save(prefs)
return return
if not isinstance(all_prefs.get("_users"), dict): if "_users" not in all_prefs:
# Preserve the flat single-user object as inert legacy data while all_prefs = {"_users": {}}
# creating the first named-owner namespace. In particular, historical
# fallback values must not be deleted or copied into the new owner.
all_prefs = dict(all_prefs)
all_prefs["_users"] = {}
all_prefs["_users"][user] = prefs all_prefs["_users"][user] = prefs
_save(all_prefs) _save(all_prefs)
+2 -2
View File
@@ -15,7 +15,7 @@ from pydantic import BaseModel, Field
from core.middleware import INTERNAL_TOOL_USER from core.middleware import INTERNAL_TOOL_USER
from src.endpoint_resolver import resolve_endpoint from src.endpoint_resolver import resolve_endpoint
from src.auth_helpers import _auth_disabled, get_current_user from src.auth_helpers import _auth_disabled, get_current_user
from src.owner_identity import REQUEST_SENTINEL_OWNERS from core.auth import RESERVED_USERNAMES
from src.constants import DEEP_RESEARCH_DIR from src.constants import DEEP_RESEARCH_DIR
_SESSION_ID_RE = re.compile(r"^[a-zA-Z0-9-]{1,128}$") _SESSION_ID_RE = re.compile(r"^[a-zA-Z0-9-]{1,128}$")
@@ -496,7 +496,7 @@ def setup_research_routes(research_handler, session_manager=None) -> APIRouter:
user = require_privilege(request, "can_use_research") user = require_privilege(request, "can_use_research")
if user == INTERNAL_TOOL_USER: if user == INTERNAL_TOOL_USER:
tool_owner = (request.headers.get("X-Odysseus-Owner") or "").strip() tool_owner = (request.headers.get("X-Odysseus-Owner") or "").strip()
if tool_owner and tool_owner not in REQUEST_SENTINEL_OWNERS: if tool_owner and tool_owner not in RESERVED_USERNAMES:
auth_mgr = getattr(request.app.state, "auth_manager", None) auth_mgr = getattr(request.app.state, "auth_manager", None)
if auth_mgr is not None and getattr(auth_mgr, "is_configured", False): if auth_mgr is not None and getattr(auth_mgr, "is_configured", False):
try: try:
+17 -254
View File
@@ -18,7 +18,6 @@ from pydantic import BaseModel, Field
from services.memory.skills import SkillsManager from services.memory.skills import SkillsManager
from src.auth_helpers import get_current_user from src.auth_helpers import get_current_user
from src.prompt_security import untrusted_context_message
from core.middleware import require_admin from core.middleware import require_admin
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -108,23 +107,6 @@ def _skill_test_task(skill: dict) -> str:
) )
def _skill_test_messages(md: str, task: str) -> list[dict]:
"""Keep user-editable skill text out of the trusted system role."""
return [
{
"role": "system",
"content": (
"You are TESTING a skill. Follow the supplied reusable procedure "
"to complete the user's task for real, using available tools step "
"by step. If the skill is wrong, unclear, or references tools that "
"do not exist, do your best; the problems will be reviewed afterward."
),
},
untrusted_context_message("skill under test", md),
{"role": "user", "content": task},
]
async def _eval_skill_run(skill_md: str, task: str, transcript: str, async def _eval_skill_run(skill_md: str, task: str, transcript: str,
url: str, model: str, headers: Optional[dict]) -> dict: url: str, model: str, headers: Optional[dict]) -> dict:
"""LLM-as-judge: grade a skill test run from its transcript. Advisory only. """LLM-as-judge: grade a skill test run from its transcript. Advisory only.
@@ -429,21 +411,7 @@ async def _eval_skill_retrieval_precision(skill_md: str, others: list,
_skill_test_jobs: dict = {} _skill_test_jobs: dict = {}
async def _run_skill_test_job( async def _run_skill_test_job(key, name, md, task, url, model, headers, owner, skills_manager=None):
key,
name,
md,
task,
url,
model,
headers,
owner,
skills_manager=None,
*,
messages=None,
transcript=None,
exact_approval=None,
):
"""Background coroutine: run the skill in an agent loop, capture a condensed """Background coroutine: run the skill in an agent loop, capture a condensed
log + transcript, then have the judge grade it. Writes into _skill_test_jobs.""" log + transcript, then have the judge grade it. Writes into _skill_test_jobs."""
import json as _json import json as _json
@@ -453,7 +421,7 @@ async def _run_skill_test_job(
if job is None: if job is None:
return return
log = job["log"] log = job["log"]
transcript = transcript if isinstance(transcript, list) else [] transcript = []
say_buf = [] say_buf = []
def _flush_say(): def _flush_say():
@@ -461,12 +429,18 @@ async def _run_skill_test_job(
log.append({"type": "say", "text": "".join(say_buf)}) log.append({"type": "say", "text": "".join(say_buf)})
say_buf.clear() say_buf.clear()
messages = list(messages) if isinstance(messages, list) else _skill_test_messages(md, task) messages = [
{"role": "system", "content":
"You are TESTING a skill. Below is a reusable skill (a procedure). Follow it "
"to complete the user's task for real, using your available tools, step by "
"step. If the skill is wrong, unclear, or references tools that don't exist, "
"do your best — the problems will be reviewed afterward.\n\n=== SKILL ===\n" + md},
{"role": "user", "content": task},
]
try: try:
async for chunk in stream_agent_loop( async for chunk in stream_agent_loop(
url, model, messages, headers=headers, url, model, messages, headers=headers,
temperature=0.3, max_tokens=0, max_rounds=8, owner=owner, temperature=0.3, max_tokens=0, max_rounds=8, owner=owner,
exact_approval=exact_approval,
): ):
if not chunk.startswith("data: ") or chunk.strip() == "data: [DONE]": if not chunk.startswith("data: ") or chunk.strip() == "data: [DONE]":
continue continue
@@ -484,25 +458,8 @@ async def _run_skill_test_job(
elif d.get("type") == "tool_output": elif d.get("type") == "tool_output":
_flush_say() _flush_say()
out = str(d.get("output") or "")[:600] out = str(d.get("output") or "")[:600]
tool_log = {"type": "tool_output", "output": out} log.append({"type": "tool_output", "output": out})
approval = d.get("ask_user")
if isinstance(approval, dict):
tool_log["ask_user"] = approval
log.append(tool_log)
transcript.append(f"[output] {out}\n") transcript.append(f"[output] {out}\n")
if (
isinstance(approval, dict)
and approval.get("kind") == "tool_approval"
and approval.get("approval_id")
):
# Manual skill tests have their own polling UI instead of a
# chat session. Pause the run and retain only server-side
# continuation state until the same owner approves/denies
# this exact sealed action.
job["status"] = "awaiting_approval"
job["approval"] = approval
job["_transcript"] = transcript
return
elif d.get("type") == "agent_step": elif d.get("type") == "agent_step":
_flush_say() _flush_say()
log.append({"type": "agent_step", "round": d.get("round")}) log.append({"type": "agent_step", "round": d.get("round")})
@@ -514,9 +471,6 @@ async def _run_skill_test_job(
_flush_say() _flush_say()
log.append({"type": "error", "error": str(e)}) log.append({"type": "error", "error": str(e)})
job.pop("approval", None)
job.pop("_transcript", None)
job.pop("_run", None)
log.append({"type": "evaluating"}) log.append({"type": "evaluating"})
try: try:
job["verdict"] = await _eval_skill_run(md, task, "".join(transcript), url, model, headers) job["verdict"] = await _eval_skill_run(md, task, "".join(transcript), url, model, headers)
@@ -740,8 +694,12 @@ async def _run_skill_test_once(md: str, task: str, url, model, headers, owner) -
import json as _json import json as _json
from src.agent_loop import stream_agent_loop from src.agent_loop import stream_agent_loop
transcript = [] transcript = []
approval_required = None messages = [
messages = _skill_test_messages(md, task) {"role": "system", "content":
"You are TESTING a skill. Follow this skill's procedure to complete the task "
"for real, using your tools, step by step.\n\n=== SKILL ===\n" + md},
{"role": "user", "content": task},
]
try: try:
# max_tokens explicitly set: passing 0 lets some upstreams (Ollama, # max_tokens explicitly set: passing 0 lets some upstreams (Ollama,
# OpenAI-compat) generate an empty completion, which manifested as # OpenAI-compat) generate an empty completion, which manifested as
@@ -761,44 +719,11 @@ async def _run_skill_test_once(md: str, task: str, url, model, headers, owner) -
transcript.append(f"\n[tool {d.get('tool')}] {str(d.get('command') or d.get('args') or '')[:300]}\n") transcript.append(f"\n[tool {d.get('tool')}] {str(d.get('command') or d.get('args') or '')[:300]}\n")
elif d.get("type") == "tool_output": elif d.get("type") == "tool_output":
transcript.append(f"[output] {str(d.get('output') or '')[:600]}\n") transcript.append(f"[output] {str(d.get('output') or '')[:600]}\n")
approval = d.get("ask_user")
if (
isinstance(approval, dict)
and approval.get("kind") == "tool_approval"
):
approval_required = approval
break
elif d.get("type") == "agent_step": elif d.get("type") == "agent_step":
transcript.append(f"\n--- round {d.get('round')} ---\n") transcript.append(f"\n--- round {d.get('round')} ---\n")
except Exception as e: except Exception as e:
transcript.append(f"\n[run error] {e}\n") transcript.append(f"\n[run error] {e}\n")
text = "".join(transcript) text = "".join(transcript)
if approval_required is not None:
# Unattended audits have no authority to approve and no UI that could
# resume this record. Destructively deny it now instead of leaving a
# reusable opaque grant pending until TTL/cap eviction.
try:
from src.tool_approvals import tool_approval_store
tool_approval_store.consume(
approval_required.get("approval_id"),
decision="deny",
owner=owner,
session_id=None,
)
except Exception:
logger.debug("Could not retire unattended skill approval", exc_info=True)
return text, {
"verdict": "inconclusive",
"confidence": 1.0,
"summary": (
"This automated audit reached an exact action that requires "
"a human approval; no action was executed."
),
"issues": [
"Run this skill's manual test and review the sealed action."
],
"approval_required": True,
}
verdict = await _eval_skill_run(md, task, text, url, model, headers) verdict = await _eval_skill_run(md, task, text, url, model, headers)
return text, verdict return text, verdict
@@ -938,26 +863,6 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
transcript, verdict = await _run_skill_test_once(md, task, url, model, headers, owner) transcript, verdict = await _run_skill_test_once(md, task, url, model, headers, owner)
v = verdict.get("verdict") v = verdict.get("verdict")
log(f"{name}: verdict = {v} ({verdict.get('summary', '')[:80]})") log(f"{name}: verdict = {v} ({verdict.get('summary', '')[:80]})")
if verdict.get("approval_required"):
# An unattended audit is not authority for an action influenced by the
# skill under test. Preserve the skill's current publication/confidence
# state and route the exact action to the manual test UI instead of
# letting a safety pause demote, rewrite, or auto-publish the skill.
skills_manager.set_audit(
name,
"inconclusive",
by_teacher=False,
worker_model=model,
owner=owner,
)
status = skill.get("status") or "draft"
log(f"{name}: {status} unchanged — exact action needs manual approval")
return {
"skill": name,
"result": "approval_required",
"verdict": verdict,
"status": status,
}
if v == "pass": if v == "pass":
# Procedure works. If the reviewer still flagged metadata (tags/category/ # Procedure works. If the reviewer still flagged metadata (tags/category/
# when_to_use/description), do ONE fixer pass to correct the frontmatter # when_to_use/description), do ONE fixer pass to correct the frontmatter
@@ -1526,19 +1431,6 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
logger.warning(f"Skill-test model resolve failed: {_e}") logger.warning(f"Skill-test model resolve failed: {_e}")
key = (user or "", name) key = (user or "", name)
previous_job = _skill_test_jobs.get(key) or {}
previous_approval = previous_job.get("approval") or {}
if previous_approval.get("approval_id"):
try:
from src.tool_approvals import tool_approval_store
tool_approval_store.consume(
previous_approval["approval_id"],
decision="deny",
owner=user,
session_id=None,
)
except Exception:
logger.debug("Could not retire replaced skill approval", exc_info=True)
_skill_test_jobs[key] = { _skill_test_jobs[key] = {
"status": "running", "status": "running",
"task": task, "task": task,
@@ -1547,138 +1439,10 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
"started": _time.time(), "started": _time.time(),
"log": [{"type": "skill_test_start", "task": task, "skill": name, "model": model}], "log": [{"type": "skill_test_start", "task": task, "skill": name, "model": model}],
"verdict": None, "verdict": None,
"_run": {
"md": md,
"url": url,
"model": model,
"headers": headers,
"owner": user,
},
} }
_asyncio.create_task(_run_skill_test_job(key, name, md, task, url, model, headers, user, skills_manager)) _asyncio.create_task(_run_skill_test_job(key, name, md, task, url, model, headers, user, skills_manager))
return {"ok": True, "status": "running", "skill": name, "model": model} return {"ok": True, "status": "running", "skill": name, "model": model}
@router.post("/{skill_id}/test-approval")
async def approve_skill_test_action(request: Request, skill_id: str):
"""Resume a manual skill test with one exact server-sealed action."""
import asyncio as _asyncio
from src.tool_approvals import tool_approval_store
user = _owner(request)
skills = skills_manager.load(owner=user)
match = next(
(s for s in skills if s.get("name") == skill_id or s.get("id") == skill_id),
None,
)
if not match:
raise HTTPException(404, "Skill not found")
_verify_owner(match, user)
name = match.get("name")
key = (user or "", name)
job = _skill_test_jobs.get(key)
if not job or job.get("status") != "awaiting_approval":
raise HTTPException(409, "This skill test is not awaiting an approval.")
body = await request.json()
if not isinstance(body, dict):
raise HTTPException(400, "Tool approval body must be a JSON object.")
approval_id = str(body.get("approval_id") or "")
decision = str(body.get("decision") or "").strip().lower()
expected = job.get("approval") or {}
if approval_id != str(expected.get("approval_id") or ""):
raise HTTPException(409, "This approval does not match the pending skill test action.")
if decision not in {"approve", "deny"}:
raise HTTPException(400, "Invalid tool approval decision.")
pending = tool_approval_store.peek(approval_id)
normalized_owner = str(user or "").strip().casefold()
if (
pending is None
or pending.owner != normalized_owner
or pending.session_id != ""
):
raise HTTPException(409, "This tool approval is invalid or expired.")
exact_approval = tool_approval_store.consume(
approval_id,
decision=decision,
owner=user,
session_id=None,
# The button here says "Allow once" and there is no chat to carry a
# scope into, so the gate must re-arm behind the sealed action.
allow_continuation=False,
)
if decision == "approve" and exact_approval is None:
raise HTTPException(409, "This tool approval could not be consumed.")
job.pop("approval", None)
if decision == "deny":
job.pop("_transcript", None)
job.pop("_run", None)
job["log"].append({
"type": "approval_denied",
"text": "Exact action denied; the skill test stopped without executing it.",
})
job["verdict"] = {
"verdict": "inconclusive",
"confidence": 1.0,
"summary": "The test stopped because its exact action was denied.",
"issues": [],
}
job["status"] = "done"
return {"ok": True, "status": "done", "decision": "deny"}
run = job.get("_run") or {}
transcript = job.pop("_transcript", [])
# stream_agent_loop owns its per-round message list internally. Rebuild
# continuation context from the original untrusted skill plus the
# accumulated transcript so repeated approvals do not lose earlier
# approved results, while keeping every transcript byte tainted.
messages = _skill_test_messages(
run.get("md", ""),
job.get("task", ""),
)
if transcript:
messages.append(untrusted_context_message(
"skill test transcript",
"".join(str(item) for item in transcript),
))
messages.extend([
{
"role": "assistant",
"content": str(expected.get("question") or "Allow this exact action once?"),
},
{
"role": "user",
"content": (
f"Approved the exact {exact_approval.pending.tool_name} "
"action shown above once."
),
},
])
job["status"] = "running"
job["log"].append({
"type": "approval_granted",
"text": (
f"Approved exact {exact_approval.pending.tool_name} action once; "
"resuming test."
),
})
_asyncio.create_task(_run_skill_test_job(
key,
name,
run.get("md", ""),
job.get("task", ""),
run.get("url"),
run.get("model"),
run.get("headers"),
run.get("owner"),
skills_manager,
messages=messages,
transcript=transcript,
exact_approval=exact_approval,
))
return {"ok": True, "status": "running", "decision": "approve"}
@router.get("/{skill_id}/test-status") @router.get("/{skill_id}/test-status")
async def test_skill_status(request: Request, skill_id: str): async def test_skill_status(request: Request, skill_id: str):
"""Current background-test state for a skill (status / log / verdict).""" """Current background-test state for a skill (status / log / verdict)."""
@@ -1695,7 +1459,6 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
"model": job.get("model"), "model": job.get("model"),
"log": job.get("log", []), "log": job.get("log", []),
"verdict": job.get("verdict"), "verdict": job.get("verdict"),
"approval": job.get("approval"),
} }
@router.post("/audit-all") @router.post("/audit-all")
-5
View File
@@ -1,5 +0,0 @@
"""Task route domain package (slice 2p, #4082/#4071).
Contains task_routes.py, migrated from the flat routes/ directory.
Backward-compat shim at routes/task_routes.py re-exports from here.
"""
File diff suppressed because it is too large Load Diff
+1177 -14
View File
File diff suppressed because it is too large Load Diff
-166
View File
@@ -1,166 +0,0 @@
#!/usr/bin/env python3
"""Make retained SearXNG settings inherit defaults without replacing them."""
from __future__ import annotations
import os
import stat
import sys
import tempfile
from pathlib import Path
import yaml
from yaml.nodes import MappingNode
from yaml.tokens import BlockMappingStartToken, FlowMappingStartToken
_UTF8_BOM = b"\xef\xbb\xbf"
def _parse_root_mapping(text: str) -> tuple[MappingNode | None, dict]:
"""Parse settings with the same safe YAML semantics SearXNG uses."""
try:
loaded = yaml.safe_load(text)
node = yaml.compose(text, Loader=yaml.SafeLoader)
except yaml.YAMLError:
raise ValueError("settings file is not valid single-document YAML") from None
if loaded is None and node is None:
return None, {}
if not isinstance(loaded, dict) or not isinstance(node, MappingNode):
raise ValueError("settings root is not a mapping")
return node, loaded
def _flow_mapping_start(text: str) -> int:
"""Return the root flow mapping's opening-brace character offset."""
try:
for token in yaml.scan(text, Loader=yaml.SafeLoader):
if isinstance(token, FlowMappingStartToken):
return token.start_mark.index
except yaml.YAMLError:
pass
raise ValueError("flow-style settings mapping has no opening brace")
def _newline_for(contents: bytes) -> bytes:
first_lf = contents.find(b"\n")
if first_lf > 0 and contents[first_lf - 1 : first_lf + 1] == b"\r\n":
return b"\r\n"
return b"\n"
def _block_mapping_position(text: str, root: MappingNode | None) -> tuple[int, int]:
"""Return a safe character offset and indent for a root block mapping key."""
if root is None:
return len(text), 0
try:
for token in yaml.scan(text, Loader=yaml.SafeLoader):
if not isinstance(token, BlockMappingStartToken):
continue
line_start = token.start_mark.index - token.start_mark.column
if not text[line_start : token.start_mark.index].strip():
return line_start, token.start_mark.column
return root.end_mark.index, token.start_mark.column
except yaml.YAMLError:
pass
return root.end_mark.index, root.start_mark.column
def _add_block_default_inheritance(
contents: bytes, text: str, root: MappingNode | None
) -> bytes:
newline = _newline_for(contents)
character_offset, indent_width = _block_mapping_position(text, root)
bom_length = len(_UTF8_BOM) if contents.startswith(_UTF8_BOM) else 0
offset = bom_length + len(text[:character_offset].encode("utf-8"))
separator = b""
if offset not in (0, bom_length) and not contents[:offset].endswith((b"\n", b"\r")):
separator = newline
addition = (
separator
+ b" " * indent_width
+ b"use_default_settings: true"
+ newline
)
return contents[:offset] + addition + contents[offset:]
def migrate_settings(path: Path) -> bool:
"""Add the missing inheritance key atomically; return whether the file changed."""
source_stat = path.lstat()
if not stat.S_ISREG(source_stat.st_mode):
raise ValueError(f"settings path is not a regular file: {path}")
contents = path.read_bytes()
if not contents:
return False
text = contents.decode("utf-8-sig")
root, loaded = _parse_root_mapping(text)
if "use_default_settings" in loaded:
return False
if root is not None and root.flow_style:
start = _flow_mapping_start(text)
bom_length = len(_UTF8_BOM) if contents.startswith(_UTF8_BOM) else 0
offset = bom_length + len(text[: start + 1].encode("utf-8"))
separator = b", " if root.value else b""
updated = (
contents[:offset]
+ b"use_default_settings: true"
+ separator
+ contents[offset:]
)
else:
updated = _add_block_default_inheritance(contents, text, root)
fd, temporary_name = tempfile.mkstemp(
prefix=f".{path.name}.odysseus-", dir=path.parent
)
temporary = Path(temporary_name)
try:
# chmod before chown: the Compose cap set is `cap_drop: ALL` plus
# CHOWN/SETGID/SETUID/DAC_OVERRIDE, with no FOWNER. Once the temporary
# file belongs to searxng:searxng — which every retained settings file
# does, because searxng's entrypoint chowns /etc/searxng — root can no
# longer chmod it and the migration dies with EPERM.
os.fchmod(fd, stat.S_IMODE(source_stat.st_mode))
os.fchown(fd, source_stat.st_uid, source_stat.st_gid)
with os.fdopen(fd, "wb") as handle:
fd = -1
handle.write(updated)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory_fd = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
if fd >= 0:
os.close(fd)
temporary.unlink(missing_ok=True)
return True
def main(argv: list[str]) -> int:
if len(argv) > 2:
print(f"usage: {Path(argv[0]).name} [settings.yml]", file=sys.stderr)
return 2
path = Path(argv[1]) if len(argv) == 2 else Path("/etc/searxng/settings.yml")
try:
changed = migrate_settings(path)
except (OSError, UnicodeError, ValueError) as exc:
print(f"SearXNG settings migration failed: {exc}", file=sys.stderr)
return 1
if changed:
print("Added use_default_settings inheritance to retained SearXNG settings")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+2 -7
View File
@@ -327,12 +327,7 @@ def list_models():
@app.post("/v1/images/generations") @app.post("/v1/images/generations")
def generate(req: ImageRequest): def generate(req: ImageRequest):
# The served model is the one this process was launched with. `req.model` model = req.model or _args.model
# is accepted for OpenAI wire compatibility and ignored, matching
# scripts/diffusion_server.py: honouring it would let a caller point the
# generator at any local directory or Hugging Face repo, and the HiDream
# branch runs a python script from inside that directory.
model = _args.model
width, height = _size(req.size) width, height = _size(req.size)
out_images = [] out_images = []
count = max(1, min(int(req.n or 1), 4)) count = max(1, min(int(req.n or 1), 4))
@@ -398,7 +393,7 @@ async def edit_image(
size: str = Form("1024x1024"), size: str = Form("1024x1024"),
response_format: str = Form("b64_json"), response_format: str = Form("b64_json"),
): ):
active_model = _args.model # pinned; see generate() active_model = model or _args.model
if _is_lama_inpaint(active_model) or _is_ddcolor(active_model): if _is_lama_inpaint(active_model) or _is_ddcolor(active_model):
image_raw = await image.read() image_raw = await image.read()
mask_raw = await mask.read() if mask is not None else None mask_raw = await mask.read() if mask is not None else None
+3 -11
View File
@@ -2,7 +2,7 @@
"""odysseus-webhook — shell wrapper for scheduled-task webhook tokens. """odysseus-webhook — shell wrapper for scheduled-task webhook tokens.
Tasks in the scheduled-task system can carry a `webhook_token`. Any Tasks in the scheduled-task system can carry a `webhook_token`. Any
HTTP POST to `/api/tasks/<task-id>/webhook/<token>` fires the task. This CLI lists, HTTP POST to `/api/webhook/<token>` fires the task. This CLI lists,
rotates, and revokes those tokens. rotates, and revokes those tokens.
odysseus-webhook list # tasks that have a token odysseus-webhook list # tasks that have a token
@@ -21,7 +21,6 @@ quiet_logs()
import argparse, json, logging, os, secrets, sys import argparse, json, logging, os, secrets, sys
from pathlib import Path from pathlib import Path
from urllib.parse import quote
try: try:
from core.database import SessionLocal, ScheduledTask from core.database import SessionLocal, ScheduledTask
@@ -54,14 +53,6 @@ def _summary(t: "ScheduledTask", reveal: bool = False) -> dict:
} }
def _task_webhook_url(base: str, task_id: str, token: str) -> str:
"""Build the live task-route URL without leaking ids into path syntax."""
root = (base or "http://localhost:7000").rstrip("/")
task_part = quote(str(task_id), safe="")
token_part = quote(str(token), safe="")
return f"{root}/api/tasks/{task_part}/webhook/{token_part}"
def cmd_list(args): def cmd_list(args):
db = SessionLocal() db = SessionLocal()
try: try:
@@ -118,7 +109,8 @@ def cmd_url(args):
fail(f"no task with id {args.id!r}") fail(f"no task with id {args.id!r}")
if not t.webhook_token: if not t.webhook_token:
fail(f"task {args.id!r} has no webhook token (rotate one first)") fail(f"task {args.id!r} has no webhook token (rotate one first)")
url = _task_webhook_url(args.base, t.id, t.webhook_token) base = (args.base or "http://localhost:7000").rstrip("/")
url = f"{base}/api/webhook/{t.webhook_token}"
emit({ emit({
"task_id": t.id, "task_id": t.id,
"name": t.name, "name": t.name,
+11 -41
View File
@@ -50,46 +50,16 @@ class DocsService:
List of DocChunk objects List of DocChunk objects
""" """
results = self.rag.search(query, k=top_k) results = self.rag.search(query, k=top_k)
chunks = [] return [
DocChunk(
for result in results: text=r.get("text", r.get("content", "")),
if not isinstance(result, dict): source=r.get("source", r.get("metadata", {}).get("source", "unknown")),
continue score=r.get("score", 0.0),
metadata=r.get("metadata"),
metadata = result.get("metadata")
if not isinstance(metadata, dict):
metadata = {}
text = result.get("document")
if text is None:
text = result.get("text")
if text is None:
text = result.get("content")
if text is None:
text = ""
source = result.get("source")
if source is None:
source = metadata.get("source")
if source is None:
source = "unknown"
score = result.get("similarity")
if score is None:
score = result.get("score")
if score is None:
score = 0.0
chunks.append(
DocChunk(
text=text,
source=source,
score=score,
metadata=metadata,
)
) )
for r in results
return chunks if isinstance(r, dict)
]
async def index(self, directory: str) -> IndexResult: async def index(self, directory: str) -> IndexResult:
""" """
@@ -103,8 +73,8 @@ class DocsService:
""" """
result = self.rag.index_personal_documents(directory) result = self.rag.index_personal_documents(directory)
return IndexResult( return IndexResult(
indexed=result.get("indexed_count", result.get("indexed", 0)), indexed=result.get("indexed", 0),
failed=result.get("failed_count", result.get("failed", 0)), failed=result.get("failed", 0),
errors=result.get("errors", []), errors=result.get("errors", []),
) )
+43 -213
View File
@@ -1,18 +1,16 @@
"""Import SKILL.md bundles from public GitHub (or skills.sh → GitHub) URLs.""" """Import SKILL.md bundles from public GitHub (or skills.sh → GitHub) URLs."""
from __future__ import annotations from __future__ import annotations
import ipaddress
import logging import logging
import os import os
import time import re
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, Iterable, List, Optional, Tuple, cast from typing import Dict, List, Optional, Tuple
from urllib.parse import quote, urljoin, urlparse from urllib.parse import quote, urljoin, urlparse
import httpcore
import httpx import httpx
from src.url_safety import _default_resolver, check_outbound_url from src.url_safety import check_outbound_url
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -27,7 +25,6 @@ TEXT_NAMES = {"skill.md", "license", "license.md", "readme.md"}
_GITHUB_HOSTS = frozenset({ _GITHUB_HOSTS = frozenset({
"github.com", "www.github.com", "api.github.com", "raw.githubusercontent.com", "github.com", "www.github.com", "api.github.com", "raw.githubusercontent.com",
}) })
_SKILLS_SH_HOSTS = frozenset({"skills.sh", "www.skills.sh"})
def _github_host(url: str) -> str: def _github_host(url: str) -> str:
@@ -75,158 +72,18 @@ def _is_text_file(name: str) -> bool:
_MAX_FETCH_REDIRECTS = 5 _MAX_FETCH_REDIRECTS = 5
def _validated_ips(raw_ips: List[str]) -> List[ipaddress._BaseAddress]: def _check_fetch_url(url: str) -> None:
"""Parse and de-duplicate one resolver snapshot in resolver order.""" """SSRF guard for skill-import fetches (defense-in-depth).
ips: List[ipaddress._BaseAddress] = []
seen = set()
for raw in raw_ips:
if not isinstance(raw, str):
continue
try:
ip = ipaddress.ip_address(raw.split("%", 1)[0])
except ValueError:
continue
if ip in seen:
continue
seen.add(ip)
ips.append(ip)
return ips
Skill bundles only ever come from public GitHub, never an internal
def _resolve_and_check_url(url: str) -> List[ipaddress._BaseAddress]: address, so block private/loopback/link-local targets on every hop
"""Return the exact address snapshot approved for one fetch hop.""" matching the hardened web-fetch path in
resolved_ips: List[str] = [] ``services/search/content.py:_get_public_url`` rather than the lenient
default used for admin-configured model endpoints.
def _recording_resolver(host: str) -> List[str]: """
answers = list(_default_resolver(host)) ok, reason = check_outbound_url(url, block_private=True)
resolved_ips[:] = answers
return answers
ok, reason = check_outbound_url(
url,
block_private=True,
resolver=_recording_resolver,
)
if not ok: if not ok:
raise SkillImportError(f"outbound URL blocked: {reason}") raise SkillImportError(reason)
pinned_ips = _validated_ips(resolved_ips)
if not pinned_ips:
raise SkillImportError("outbound URL blocked: host did not resolve to a usable address")
return pinned_ips
# Backward compatibility alias for tests importing _check_fetch_url directly
_check_fetch_url = _resolve_and_check_url
class _PinnedBackend(httpcore.NetworkBackend):
"""Connect only to addresses from one validated DNS snapshot."""
def __init__(self, ips: List[ipaddress._BaseAddress]):
self._ips = [str(ip) for ip in ips]
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
deadline = None if timeout is None else time.monotonic() + timeout
last_exc: Optional[Exception] = None
for ip in self._ips:
remaining = None if deadline is None else max(0.0, deadline - time.monotonic())
try:
return self._real.connect_tcp(
ip,
port,
remaining,
local_address,
socket_options,
)
except (httpcore.ConnectError, httpcore.ConnectTimeout) as exc:
last_exc = exc
if deadline is not None and time.monotonic() >= deadline:
break
if last_exc is not None:
raise last_exc
raise httpcore.ConnectError("no validated address available")
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Pin socket connects while preserving URL authority, Host, and TLS SNI."""
def __init__(self, ips: List[ipaddress._BaseAddress]):
self._pinned_ips = list(ips)
self._pool = httpcore.ConnectionPool(
ssl_context=httpx.create_ssl_context(),
http1=True,
http2=False,
network_backend=_PinnedBackend(ips),
)
def handle_request(self, request: httpx.Request) -> httpx.Response:
core_request = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
core_response = None
try:
core_response = self._pool.handle_request(core_request)
content = b"".join(cast(Iterable[bytes], core_response.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
finally:
if core_response is not None:
core_response.close()
return httpx.Response(
status_code=core_response.status,
headers=core_response.headers,
content=content,
extensions=core_response.extensions,
)
def close(self) -> None:
self._pool.close()
def _get_checked( def _get_checked(
@@ -243,76 +100,49 @@ def _get_checked(
hand lets us re-validate every hop, closing that blind-SSRF gap. hand lets us re-validate every hop, closing that blind-SSRF gap.
""" """
current = url current = url
for _ in range(_MAX_FETCH_REDIRECTS + 1): with httpx.Client(follow_redirects=False, timeout=timeout) as client:
pinned_ips = _resolve_and_check_url(current) for _ in range(_MAX_FETCH_REDIRECTS + 1):
with httpx.Client( _check_fetch_url(current)
transport=_PinnedTransport(pinned_ips),
follow_redirects=False,
timeout=timeout,
) as client:
r = client.get(current, headers=headers) r = client.get(current, headers=headers)
if r.status_code in (301, 302, 303, 307, 308):
if r.status_code in (301, 302, 303, 307, 308): location = r.headers.get("location")
location = r.headers.get("location") if not location:
if not location: return r
return r current = urljoin(str(r.url), location)
current = urljoin(str(r.url), location) continue
continue return r
return r
raise SkillImportError("too many redirects while fetching skill bundle") raise SkillImportError("too many redirects while fetching skill bundle")
def parse_skill_source(url: str) -> ResolvedSource: def parse_skill_source(url: str) -> ResolvedSource:
"""Normalize skills.sh / GitHub web URLs into owner/repo/ref/path.""" """Normalize skills.sh / GitHub web URLs into owner/repo/ref/path."""
url = (url or "").strip() raw = (url or "").strip()
if not url: if not raw:
raise SkillImportError("URL is required") raise SkillImportError("URL is required")
# ``urlparse`` only reports an unambiguous scheme when the URL carries the # skills.sh often links to GitHub; try to unwrap ?url= or redirect target later.
# ``scheme://`` form. Opaque schemes (``mailto:``, ``javascript:``) and a if "skills.sh" in raw and "github.com" not in raw:
# schemeless ``host:port`` both parse a "scheme" that is not one, so they r = _get_checked(raw, timeout=20.0)
# fall through to the host check below and are rejected on the host instead.
scheme = urlparse(url).scheme.lower()
if scheme not in ("http", "https"):
if scheme and url.lower().startswith(f"{scheme}://"):
raise SkillImportError(f"unsupported URL scheme: {scheme}")
# Schemeless "github.com/owner/repo" — accept only a supported host.
rough_host = (urlparse("//" + url).hostname or "").lower()
if rough_host not in _GITHUB_HOSTS and rough_host not in _SKILLS_SH_HOSTS:
raise SkillImportError("Only GitHub or skills.sh URLs are supported")
url = "https://" + url
parsed = urlparse(url)
hostname = (parsed.hostname or "").lower()
if hostname not in _GITHUB_HOSTS and hostname not in _SKILLS_SH_HOSTS:
raise SkillImportError("Only GitHub or skills.sh URLs are supported")
# A skills.sh link is only usable if it redirects to an exact supported
# GitHub host. Scraping the page body for a github.com link cannot work:
# skill pages only ever link the repository root, never the skill's
# subdirectory, so the scrape resolves every skill in a repo to the same
# (wrong) bundle. Fail with an actionable message instead.
if hostname in _SKILLS_SH_HOSTS:
r = _get_checked(url, timeout=20.0)
if r.status_code >= 400: if r.status_code >= 400:
raise _github_response_error(r) raise _github_response_error(r)
final = str(r.url) final = str(r.url)
if _github_host(final) not in _GITHUB_HOSTS: _assert_github_url(final, context="redirect target")
raise SkillImportError( # Page may embed a github link; prefer final URL if redirected.
"skills.sh did not redirect to GitHub — open the skill's " if "github.com" in final:
"repository on GitHub, navigate to the exact skill folder or " raw = final
"SKILL.md file, and paste that URL; the repository-root link " else:
"alone is not sufficient" m = re.search(r"https?://github\.com/[^\s\"')]+", r.text or "")
) if m:
url = final raw = m.group(0).rstrip(".,)")
# Update parsed and hostname to reflect the new GitHub URL parsed = urlparse(raw)
parsed = urlparse(url) host = _github_host(raw)
hostname = (parsed.hostname or "").lower() if host not in _GITHUB_HOSTS:
raise SkillImportError(
"Only GitHub URLs are supported (https://github.com/... or raw.githubusercontent.com/...)"
)
_assert_github_url(url) if host == "raw.githubusercontent.com":
if hostname == "raw.githubusercontent.com":
# /owner/repo/ref/path/to/file # /owner/repo/ref/path/to/file
bits = [p for p in parsed.path.split("/") if p] bits = [p for p in parsed.path.split("/") if p]
if len(bits) < 4: if len(bits) < 4:
+331 -31
View File
@@ -2,18 +2,22 @@
import copy import copy
import io import io
import ipaddress
import json import json
import os import os
import re import re
import logging import logging
import socket
import ssl
from datetime import datetime, timedelta from datetime import datetime, timedelta
from typing import List from typing import Iterable, List, cast
from urllib.parse import urljoin, urlparse
import httpx import httpx
import httpcore
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_USER_AGENT from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_USER_AGENT
from src import outbound_fetch as _outbound_fetch
from .analytics import RateLimitError, error_logger from .analytics import RateLimitError, error_logger
from .cache import ( from .cache import (
@@ -25,40 +29,336 @@ from .cache import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _is_private_address(addr): _PRIVATE_NETWORKS = (
return _outbound_fetch._is_private_address(addr) ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
)
def _resolve_hostname_ips(hostname): def _is_private_address(addr: ipaddress._BaseAddress) -> bool:
return _outbound_fetch._resolve_hostname_ips(hostname) if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
addr = addr.ipv4_mapped
return (
def _public_http_url(url): addr.is_private
return _outbound_fetch._public_http_url(url, resolver=_resolve_hostname_ips) or addr.is_loopback
or addr.is_link_local
or addr.is_reserved
def _resolve_public_ips(url): or addr.is_multicast
return _outbound_fetch._resolve_public_ips(url, resolver=_resolve_hostname_ips) or addr.is_unspecified
or any(addr in net for net in _PRIVATE_NETWORKS)
_PinnedBackend = _outbound_fetch._PinnedBackend
_PinnedTransport = _outbound_fetch._PinnedTransport
BodyTooLargeError = _outbound_fetch.BodyTooLargeError
_CappedFetch = _outbound_fetch._CappedFetch
def _get_public_url(url, headers, timeout, max_redirects=5, max_bytes=None):
return _outbound_fetch._get_public_url(
url,
headers=headers,
timeout=timeout,
max_redirects=max_redirects,
max_bytes=max_bytes,
resolve_public_ips=_resolve_public_ips,
transport_factory=_PinnedTransport,
) )
def _resolve_hostname_ips(hostname: str) -> list[ipaddress._BaseAddress]:
try:
infos = socket.getaddrinfo(hostname, None)
except Exception:
return []
out = []
for info in infos:
try:
out.append(ipaddress.ip_address(info[4][0]))
except Exception:
continue
return out
def _public_http_url(url: str) -> bool:
try:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
return False
host = (parsed.hostname or "").strip()
if not host:
return False
lower = host.lower()
if lower in ("localhost", "metadata", "metadata.google.internal"):
return False
if lower.endswith((".local", ".localhost", ".internal", ".lan", ".intranet")):
return False
try:
return not _is_private_address(ipaddress.ip_address(host))
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
return bool(addrs) and not any(_is_private_address(a) for a in addrs)
except Exception:
return False
def _resolve_public_ips(url: str) -> list[ipaddress._BaseAddress]:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise httpx.RequestError(f"Blocked non-public URL: {url}")
host = (parsed.hostname or "").strip().lower()
if host in ("localhost", "metadata", "metadata.google.internal"):
raise httpx.RequestError(f"Blocked non-public hostname: {host}")
try:
ip = ipaddress.ip_address(host)
if _is_private_address(ip):
raise httpx.RequestError(f"Blocked non-public IP literal: {host}")
return [ip]
except httpx.RequestError:
raise
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
if not addrs or any(_is_private_address(a) for a in addrs):
raise httpx.RequestError(f"Blocked non-public URL: {url}")
return addrs
class _PinnedBackend(httpcore.NetworkBackend):
"""Network backend that connects to a pre-resolved IP.
httpcore derives the TLS SNI and the ``Host`` header from the URL's
origin, not from the host argument passed to ``connect_tcp``. So
routing the TCP connect to a resolved IP while leaving the URL
untouched keeps SNI / vhost behaviour correct and closes the
DNS-rebinding TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress):
self._ip = str(ip)
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
return self._real.connect_tcp(
self._ip, port, timeout, local_address, socket_options
)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
# Map httpcore exception classes to their httpx equivalents. Built
# once at import time from the public exception classes; avoids any
# import of httpx's private transport machinery. httpcore's
# ``ConnectionNotAvailable`` is a pool-internal signal (the pool will
# close and retry on its own) — we never expect to see it surface to
# a transport caller, so it has no httpx counterpart here.
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Transport that pins every TCP connect to a pre-resolved IP.
Uses only the public ``httpcore`` and ``httpx`` APIs no
subclassing of ``httpx.HTTPTransport``, no reads of private
``httpcore.ConnectionPool`` attributes, no imports from
``httpx private transport internals``. The URL is passed through unchanged so SNI
/ vhost work as if httpx had been given the hostname directly;
only the TCP destination is pinned, closing the DNS-rebinding
TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress, *, http2: bool = False):
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
http1=True,
http2=http2,
network_backend=_PinnedBackend(ip),
)
def __enter__(self):
self._pool.__enter__()
return self
def __exit__(self, exc_type=None, exc_value=None, traceback=None) -> None:
self._pool.__exit__(exc_type, exc_value, traceback)
def handle_request(self, request: httpx.Request) -> httpx.Response:
httpcore_req = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
try:
httpcore_resp = self._pool.handle_request(httpcore_req)
# Eager materialisation matches the original
# ``response.text`` usage in fetch_webpage_content. The
# sync pool's stream is a plain Iterable[bytes] despite
# the httpcore type hint unioning the async variant.
content = b"".join(cast(Iterable[bytes], httpcore_resp.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
return httpx.Response(
status_code=httpcore_resp.status,
headers=httpcore_resp.headers,
content=content,
extensions=httpcore_resp.extensions,
)
def close(self) -> None:
self._pool.close()
class BodyTooLargeError(Exception):
"""The server declared a body larger than the hard fetch ceiling."""
def __init__(self, url: str, declared_bytes: int):
self.url = url
self.declared_bytes = declared_bytes
super().__init__(
f"response body is {declared_bytes:,} bytes, over the "
f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
)
class _CappedFetch:
"""Result of a size-capped streaming GET.
Carries just what fetch_webpage_content needs from an httpx.Response,
plus the cap bookkeeping: the (possibly truncated) body, whether the
cap cut it short, and the size the server declared via Content-Length
(wire bytes; None when absent).
"""
__slots__ = ("status_code", "headers", "content", "truncated",
"declared_bytes", "encoding", "url")
def __init__(self, status_code, headers, content, truncated,
declared_bytes, encoding, url):
self.status_code = status_code
self.headers = headers
self.content = content
self.truncated = truncated
self.declared_bytes = declared_bytes
self.encoding = encoding
self.url = url
@property
def text(self) -> str:
return self.content.decode(self.encoding or "utf-8", errors="replace")
def raise_for_status(self):
if self.status_code >= 400:
request = httpx.Request("GET", self.url)
raise httpx.HTTPStatusError(
f"HTTP {self.status_code} for {self.url}",
request=request,
response=httpx.Response(self.status_code, request=request),
)
def _get_public_url(url: str, headers: dict, timeout: int, max_redirects: int = 5,
max_bytes: int = None) -> "_CappedFetch":
"""Capped streaming GET with SSRF-guarded, DNS-pinned manual redirects.
Each hop is resolved once, validated as public, and then the actual TCP
connection is pinned to that resolved IP. The request URL is left unchanged
so Host and TLS SNI keep the original hostname.
"""
cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
current = url
for _ in range(max_redirects + 1):
ips = _resolve_public_ips(current)
# Force identity transfer-encoding. With gzip/deflate the wire bytes
# and Content-Length can be a small fraction of the decoded body, so a
# tiny compressed response could pass the hard-cap preflight and then
# expand past the ceiling in one decoded chunk before the streamed cap
# below can slice it.
req_headers = dict(headers or {})
req_headers["Accept-Encoding"] = "identity"
with httpx.Client(
headers=req_headers,
timeout=timeout,
follow_redirects=False,
transport=_PinnedTransport(ips[0]),
) as client:
with client.stream("GET", current) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
if not location:
return _CappedFetch(response.status_code, response.headers, b"",
False, None, response.encoding, str(response.url))
current = urljoin(str(response.url), location)
continue
# A server can ignore the identity request and still return a
# compressed body; httpx.iter_bytes would then decode it, and a
# tiny gzip can balloon into one decoded chunk far past the cap.
# Refuse compressed Content-Encoding so the streamed cap stays
# a real memory bound.
enc = (response.headers.get("content-encoding") or "").strip().lower()
if enc and enc != "identity":
raise httpx.RequestError(
f"Refusing compressed response (Content-Encoding: {enc}) after "
"requesting identity: cannot bound decoded body size",
request=httpx.Request("GET", current),
)
declared = None
raw_len = response.headers.get("content-length")
if raw_len and raw_len.isdigit():
declared = int(raw_len)
if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
raise BodyTooLargeError(current, declared)
chunks = []
read = 0
truncated = False
for chunk in response.iter_bytes():
read += len(chunk)
if read > cap:
keep = cap - (read - len(chunk))
if keep > 0:
chunks.append(chunk[:keep])
truncated = True
break
chunks.append(chunk)
return _CappedFetch(response.status_code, response.headers,
b"".join(chunks), truncated, declared,
response.encoding, str(response.url))
raise httpx.RequestError("Too many redirects", request=httpx.Request("GET", current))
# PDF extraction (optional dependency) # PDF extraction (optional dependency)
try: try:
from pdfminer.high_level import extract_text as pdf_extract_text from pdfminer.high_level import extract_text as pdf_extract_text
+467 -1661
View File
File diff suppressed because it is too large Load Diff
+26 -84
View File
@@ -17,14 +17,13 @@ close / navigation / refresh). It does NOT survive a server restart.
import asyncio import asyncio
import json import json
import logging import logging
import uuid
from typing import AsyncGenerator, Dict, Optional from typing import AsyncGenerator, Dict, Optional
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class _Run: class _Run:
__slots__ = ("buffer", "subscribers", "status", "task", "evict_task", "run_id") __slots__ = ("buffer", "subscribers", "status", "task", "evict_task")
def __init__(self) -> None: def __init__(self) -> None:
self.buffer: list = [] # ordered SSE event strings (replay log) self.buffer: list = [] # ordered SSE event strings (replay log)
@@ -32,9 +31,6 @@ class _Run:
self.status: str = "running" # running | done | error | stopped self.status: str = "running" # running | done | error | stopped
self.task: Optional[asyncio.Task] = None self.task: Optional[asyncio.Task] = None
self.evict_task: Optional[asyncio.Task] = None self.evict_task: Optional[asyncio.Task] = None
# Stable across every subscription/replay of this exact detached run.
# The browser uses it to make local cost accounting replay-idempotent.
self.run_id: str = uuid.uuid4().hex
_RUNS: Dict[str, _Run] = {} _RUNS: Dict[str, _Run] = {}
@@ -57,24 +53,13 @@ def _publish(run: _Run, ev: str) -> None:
pass pass
def _wake_run_subscribers(run: _Run) -> None: def _schedule_evict(session_id: str) -> None:
"""Close subscribers even when the drain task never reached its body."""
for q in list(run.subscribers):
try:
q.put_nowait((None, None))
except Exception:
pass
def _schedule_evict(session_id: str, expected_run: Optional[_Run] = None) -> None:
"""(Re)arm a grace-period eviction for a terminal run with no subscribers. """(Re)arm a grace-period eviction for a terminal run with no subscribers.
Identity-checked so a run that gets replaced/reused is never evicted by a Identity-checked so a run that gets replaced/reused is never evicted by a
stale timer.""" stale timer."""
run = _RUNS.get(session_id) run = _RUNS.get(session_id)
if run is None: if run is None:
return return
if expected_run is not None and run is not expected_run:
return
if run.evict_task and not run.evict_task.done(): if run.evict_task and not run.evict_task.done():
run.evict_task.cancel() run.evict_task.cancel()
@@ -100,38 +85,25 @@ def get_status(session_id: str) -> Optional[str]:
return r.status if r else None return r.status if r else None
def get_run_id(session_id: str) -> Optional[str]: async def _drain(session_id: str, agen: AsyncGenerator[str, None],
"""Return the opaque identity of the current detached run, if present."""
r = _RUNS.get(session_id)
return r.run_id if r else None
def get_active_run(session_id: str) -> Optional[_Run]:
"""Return the exact active run currently registered for a session."""
r = _RUNS.get(session_id)
return r if r and r.status == "running" else None
async def _drain(session_id: str, run: _Run, agen: AsyncGenerator[str, None],
prev_task: Optional[asyncio.Task] = None) -> None: prev_task: Optional[asyncio.Task] = None) -> None:
"""Pull every event from the wrapped generator into the run buffer, fanning """Pull every event from the wrapped generator into the run buffer, fanning
each out to live subscribers. Runs to completion regardless of subscribers.""" each out to live subscribers. Runs to completion regardless of subscribers."""
subscribers_woken = False run = _RUNS.get(session_id)
if run is None:
def _wake_subscribers() -> None: return
nonlocal subscribers_woken
if subscribers_woken:
return
subscribers_woken = True
_wake_run_subscribers(run)
# If this run replaced an in-flight one (rapid double-send), wait for that # If this run replaced an in-flight one (rapid double-send), wait for that
# one to fully finish first. Its CancelledError handler calls aclose(), which # one to fully finish first. Its CancelledError handler calls aclose(), which
# persists its partial response — letting it complete before we start writing # persists its partial response — letting it complete before we start writing
# keeps the two runs' session saves sequential instead of interleaved. # keeps the two runs' session saves sequential instead of interleaved.
try: if prev_task is not None and not prev_task.done():
if prev_task is not None and not prev_task.done(): try:
await asyncio.wait({prev_task}) await asyncio.wait({prev_task})
except asyncio.CancelledError:
raise # our own cancellation — propagate
except Exception:
pass
try:
async for ev in agen: async for ev in agen:
_publish(run, ev) _publish(run, ev)
if run.status == "running": if run.status == "running":
@@ -144,16 +116,6 @@ async def _drain(session_id: str, run: _Run, agen: AsyncGenerator[str, None],
await agen.aclose() await agen.aclose()
except Exception: except Exception:
pass pass
# A rapid third replacement can cancel this task while it is still
# waiting for its predecessor. Close this run's subscribers promptly,
# but keep the task alive until the predecessor finishes so the next
# run still observes the transitive session-save ordering barrier.
_wake_subscribers()
if prev_task is not None and not prev_task.done():
try:
await asyncio.shield(prev_task)
except (asyncio.CancelledError, Exception):
pass
except Exception as e: except Exception as e:
logger.error("[agent-run] %s failed: %s", session_id, e, exc_info=True) logger.error("[agent-run] %s failed: %s", session_id, e, exc_info=True)
run.status = "error" run.status = "error"
@@ -165,11 +127,15 @@ async def _drain(session_id: str, run: _Run, agen: AsyncGenerator[str, None],
_publish(run, "data: [DONE]\n\n") _publish(run, "data: [DONE]\n\n")
finally: finally:
# Wake every subscriber with the end sentinel so their SSE closes. # Wake every subscriber with the end sentinel so their SSE closes.
_wake_subscribers() for q in list(run.subscribers):
try:
q.put_nowait((None, None))
except Exception:
pass
# Run is terminal — arm the grace timer so it (and its buffer) is # Run is terminal — arm the grace timer so it (and its buffer) is
# eventually freed even if nobody ever reconnects. subscribe() cancels # eventually freed even if nobody ever reconnects. subscribe() cancels
# this on connect and re-arms on disconnect. # this on connect and re-arms on disconnect.
_schedule_evict(session_id, run) _schedule_evict(session_id)
def start(session_id: str, agen: AsyncGenerator[str, None]) -> _Run: def start(session_id: str, agen: AsyncGenerator[str, None]) -> _Run:
@@ -179,37 +145,20 @@ def start(session_id: str, agen: AsyncGenerator[str, None]) -> _Run:
prev_task: Optional[asyncio.Task] = None prev_task: Optional[asyncio.Task] = None
if prev: if prev:
if prev.task and not prev.task.done(): if prev.task and not prev.task.done():
# A task cancelled before its first instruction never enters
# _drain(), so its except/finally blocks cannot update status or
# wake a response already bound to this exact run. Terminalize it
# synchronously before cancelling; _drain's cleanup is idempotent
# when the task had already started.
if prev.status == "running":
prev.status = "stopped"
_wake_run_subscribers(prev)
prev.task.cancel() prev.task.cancel()
prev_task = prev.task # new run awaits this before it starts writing prev_task = prev.task # new run awaits this before it starts writing
if prev.evict_task and not prev.evict_task.done(): if prev.evict_task and not prev.evict_task.done():
prev.evict_task.cancel() prev.evict_task.cancel()
run = _Run() run = _Run()
_RUNS[session_id] = run _RUNS[session_id] = run
run.task = asyncio.create_task(_drain(session_id, run, agen, prev_task)) run.task = asyncio.create_task(_drain(session_id, agen, prev_task))
return run return run
async def subscribe( async def subscribe(session_id: str) -> AsyncGenerator[str, None]:
session_id: str,
expected_run: Optional[_Run] = None,
) -> AsyncGenerator[str, None]:
"""Replay the run's buffer from the start, then stream live until it ends. """Replay the run's buffer from the start, then stream live until it ends.
Safe to call repeatedly (reconnect) and from multiple clients at once. Safe to call repeatedly (reconnect) and from multiple clients at once."""
run = _RUNS.get(session_id)
``expected_run`` binds a lazy StreamingResponse body to the same run whose
identity was put in its response headers. Without that binding, a rapid
replacement between response construction and body iteration could replay
the replacement run under the prior run's identity.
"""
run = expected_run or _RUNS.get(session_id)
if run is None: if run is None:
return return
q: asyncio.Queue = asyncio.Queue() q: asyncio.Queue = asyncio.Queue()
@@ -252,19 +201,12 @@ async def subscribe(
# Last subscriber gone on a finished run — (re)arm eviction so the # Last subscriber gone on a finished run — (re)arm eviction so the
# buffer doesn't linger indefinitely. # buffer doesn't linger indefinitely.
if not run.subscribers and run.status != "running": if not run.subscribers and run.status != "running":
_schedule_evict(session_id, run) _schedule_evict(session_id)
def stop(session_id: str, expected_run_id: Optional[str] = None) -> bool: def stop(session_id: str) -> bool:
"""Cancel the matching in-flight run (which saves its partial output). """Cancel an in-flight run (the wrapped generator saves its partial)."""
A stale browser may issue Stop after another tab has replaced the session's
run. Once the caller knows its opaque run identity, fail closed rather than
cancelling that newer run.
"""
run = _RUNS.get(session_id) run = _RUNS.get(session_id)
if not expected_run_id or run is None or run.run_id != expected_run_id:
return False
if run and run.task and not run.task.done(): if run and run.task and not run.task.done():
run.task.cancel() run.task.cancel()
return True return True
+6 -18
View File
@@ -510,12 +510,7 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
# set/get/list/delete operate on the REAL app settings (the same store # set/get/list/delete operate on the REAL app settings (the same store
# the Settings panel writes), so changing a model / voice / search # the Settings panel writes), so changing a model / voice / search
# engine / reminder channel from chat actually takes effect. # engine / reminder channel from chat actually takes effect.
from src.settings import ( from src.settings import load_settings, save_settings, DEFAULT_SETTINGS
DEFAULT_SETTINGS,
RETIRED_SETTING_KEYS,
load_settings,
save_settings,
)
# Secrets/credentials the agent must NOT write: kept read-only (masked) # Secrets/credentials the agent must NOT write: kept read-only (masked)
# so API keys never flow through chat. User sets these in the panel. # so API keys never flow through chat. User sets these in the panel.
@@ -567,9 +562,6 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
return k2 return k2
return _ALIASES_SET.get(k2, (k or "").strip()) return _ALIASES_SET.get(k2, (k or "").strip())
def _is_managed_key(key):
return key in DEFAULT_SETTINGS and key not in RETIRED_SETTING_KEYS
_ENUMS = { _ENUMS = {
"image_quality": ["low", "medium", "high"], "image_quality": ["low", "medium", "high"],
"reminder_channel": ["browser", "email", "ntfy", "webhook"], "reminder_channel": ["browser", "email", "ntfy", "webhook"],
@@ -632,18 +624,14 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
if action == "list": if action == "list":
s = load_settings() s = load_settings()
shown = { shown = {k: _mask(k, v) for k, v in s.items() if k in DEFAULT_SETTINGS and not isinstance(v, dict)}
k: _mask(k, v)
for k, v in s.items()
if _is_managed_key(k) and not isinstance(v, dict)
}
return {"response": f"{len(shown)} settings (use get/set with a key)", "settings": shown, "exit_code": 0} return {"response": f"{len(shown)} settings (use get/set with a key)", "settings": shown, "exit_code": 0}
elif action == "get": elif action == "get":
key = _resolve(args.get("key", "")) key = _resolve(args.get("key", ""))
if not key: if not key:
return {"error": "key is required", "exit_code": 1} return {"error": "key is required", "exit_code": 1}
if not _is_managed_key(key): if key not in DEFAULT_SETTINGS:
return {"error": f"Unknown setting '{args.get('key')}'. Use action='list' to see them.", "exit_code": 1} return {"error": f"Unknown setting '{args.get('key')}'. Use action='list' to see them.", "exit_code": 1}
val = load_settings().get(key, DEFAULT_SETTINGS.get(key)) val = load_settings().get(key, DEFAULT_SETTINGS.get(key))
return {"response": f"{key} = {_mask(key, val)}", "value": _mask(key, val), "exit_code": 0} return {"response": f"{key} = {_mask(key, val)}", "value": _mask(key, val), "exit_code": 0}
@@ -654,11 +642,11 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
if not raw: if not raw:
return {"error": "key is required", "exit_code": 1} return {"error": "key is required", "exit_code": 1}
key = _resolve(raw) key = _resolve(raw)
if not _is_managed_key(key): if key not in DEFAULT_SETTINGS:
return {"error": f"Unknown setting '{raw}'. Use action='list' to see available settings.", "exit_code": 1} return {"error": f"Unknown setting '{raw}'. Use action='list' to see available settings.", "exit_code": 1}
if _is_secret(key): if _is_secret(key):
return {"response": f"'{key}' is a credential/secret. For security I can't set it from chat. Open Settings and set it there.", "exit_code": 0} return {"response": f"'{key}' is a credential/secret. For security I can't set it from chat. Open Settings and set it there.", "exit_code": 0}
# Structured settings (dicts/lists like keybinds or vision fallbacks) # Structured settings (dicts/lists like keybinds, default_model_fallbacks)
# have no safe scalar coercion; _coerce would pass a bare string # have no safe scalar coercion; _coerce would pass a bare string
# straight through and clobber the structure. Refuse them here; they're # straight through and clobber the structure. Refuse them here; they're
# edited in their dedicated panels. (reset/delete still restore the # edited in their dedicated panels. (reset/delete still restore the
@@ -687,7 +675,7 @@ async def do_manage_settings(content: str, owner: Optional[str] = None) -> Dict:
elif action == "delete" or action == "reset": elif action == "delete" or action == "reset":
key = _resolve(args.get("key", "")) key = _resolve(args.get("key", ""))
if not _is_managed_key(key): if key not in DEFAULT_SETTINGS:
return {"error": f"Unknown setting '{args.get('key')}'.", "exit_code": 1} return {"error": f"Unknown setting '{args.get('key')}'.", "exit_code": 1}
if _is_secret(key): if _is_secret(key):
return {"response": f"'{key}' is a credential. Reset it in the panel.", "exit_code": 0} return {"response": f"'{key}' is a credential. Reset it in the panel.", "exit_code": 0}
-59
View File
@@ -2,7 +2,6 @@ from typing import Any, Dict, List, Optional
import logging import logging
import re import re
from src.constants import MAX_READ_CHARS from src.constants import MAX_READ_CHARS
from src.tool_approvals import document_content_digest
from src.tool_utils import _parse_tool_args, get_upload_handler from src.tool_utils import _parse_tool_args, get_upload_handler
from src.upload_handler import reserve_upload_references from src.upload_handler import reserve_upload_references
@@ -81,40 +80,6 @@ def _most_recent_owned_document(db, Document, owner: Optional[str], active_only:
return q.order_by(Document.updated_at.desc()).first() return q.order_by(Document.updated_at.desc()).first()
def _approved_document_version_error(doc: Any, ctx: dict) -> Optional[Dict]:
"""Reject a sealed document action when its target changed meanwhile."""
expected_version = ctx.get("expected_document_version")
expected_digest = (
str(ctx.get("expected_document_digest") or "").strip().lower()
)
if expected_version is None and not expected_digest:
return None
try:
version_unchanged = (
expected_version is None
or int(getattr(doc, "version_count", -1)) == int(expected_version)
)
except (TypeError, ValueError):
version_unchanged = False
content_unchanged = True
if expected_digest:
content_unchanged = (
doc is not None
and document_content_digest(getattr(doc, "current_content", ""))
== expected_digest
)
if version_unchanged and content_unchanged:
return None
return {
"error": (
"The target document changed after this action was proposed. "
"Review the latest version and request the edit again."
),
"exit_code": 1,
"document_changed": True,
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Document tools — create/update/edit/suggest living documents # Document tools — create/update/edit/suggest living documents
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -489,12 +454,6 @@ class UpdateDocumentTool:
doc = None doc = None
if target_id: if target_id:
doc = _get_owned_document(db, Document, target_id, owner) doc = _get_owned_document(db, Document, target_id, owner)
if (
not doc
and target_id
and ctx.get("expected_document_version") is not None
):
return _approved_document_version_error(None, ctx)
if not doc: if not doc:
doc = _most_recent_owned_document(db, Document, owner) doc = _most_recent_owned_document(db, Document, owner)
if doc: if doc:
@@ -504,10 +463,6 @@ class UpdateDocumentTool:
if not doc: if not doc:
return {"error": "No documents exist to update"} return {"error": "No documents exist to update"}
version_error = _approved_document_version_error(doc, ctx)
if version_error:
return version_error
is_email_doc = doc.language == "email" or _looks_like_email_document(doc.current_content or "", doc.title or "") is_email_doc = doc.language == "email" or _looks_like_email_document(doc.current_content or "", doc.title or "")
new_content = _coerce_email_document_content(doc.current_content or "", content) if is_email_doc else content.strip() new_content = _coerce_email_document_content(doc.current_content or "", content) if is_email_doc else content.strip()
if is_email_doc: if is_email_doc:
@@ -575,12 +530,6 @@ class EditDocumentTool:
doc = None doc = None
if target_id: if target_id:
doc = _get_owned_document(db, Document, target_id, owner) doc = _get_owned_document(db, Document, target_id, owner)
if (
not doc
and target_id
and ctx.get("expected_document_version") is not None
):
return _approved_document_version_error(None, ctx)
if not doc: if not doc:
# Fallback: most recently updated document. Avoids "no active doc" errors # Fallback: most recently updated document. Avoids "no active doc" errors
# after server restart or when the agent loses track of which doc to edit. # after server restart or when the agent loses track of which doc to edit.
@@ -592,10 +541,6 @@ class EditDocumentTool:
if not doc: if not doc:
return {"error": "No documents exist to edit"} return {"error": "No documents exist to edit"}
version_error = _approved_document_version_error(doc, ctx)
if version_error:
return version_error
is_email_doc = doc.language == "email" or _looks_like_email_document(doc.current_content or "", doc.title or "") is_email_doc = doc.language == "email" or _looks_like_email_document(doc.current_content or "", doc.title or "")
blank_find_edits = [e for e in edits if not (e.get("find") or "").strip()] blank_find_edits = [e for e in edits if not (e.get("find") or "").strip()]
if blank_find_edits: if blank_find_edits:
@@ -732,10 +677,6 @@ class SuggestDocumentTool:
if not doc: if not doc:
return {"error": f"Document {target_id} not found"} return {"error": f"Document {target_id} not found"}
version_error = _approved_document_version_error(doc, ctx)
if version_error:
return version_error
# Validate that FIND text exists in document # Validate that FIND text exists in document
valid = [] valid = []
for s in suggestions: for s in suggestions:
+2 -8
View File
@@ -64,10 +64,7 @@ async def chat_with_model(content: str, session_id: Optional[str] = None, owner:
return {"model": model, "response": response} return {"model": model, "response": response}
except Exception as e: except Exception as e:
logger.error(f"chat_with_model failed: {e}") logger.error(f"chat_with_model failed: {e}")
return { return {"error": f"Failed to get response from {model_spec}: {e}"}
"error": f"Failed to get response from {model_spec}: {e}",
"untrusted_content": True,
}
async def ask_teacher(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict: async def ask_teacher(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict:
@@ -113,10 +110,7 @@ async def ask_teacher(content: str, session_id: Optional[str] = None, owner: Opt
return {"model": model, "response": response, "teacher": True} return {"model": model, "response": response, "teacher": True}
except Exception as e: except Exception as e:
logger.error(f"ask_teacher failed: {e}") logger.error(f"ask_teacher failed: {e}")
return { return {"error": f"Teacher call failed ({model_spec}): {e}"}
"error": f"Teacher call failed ({model_spec}): {e}",
"untrusted_content": True,
}
async def list_models(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict: async def list_models(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict:
+1 -4
View File
@@ -240,10 +240,7 @@ async def send_to_session(content: str, session_id: Optional[str] = None, owner:
} }
except Exception as e: except Exception as e:
logger.error(f"send_to_session failed: {e}") logger.error(f"send_to_session failed: {e}")
return { return {"error": f"Failed to send to session: {e}"}
"error": f"Failed to send to session: {e}",
"untrusted_content": True,
}
async def manage_session(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict: async def manage_session(content: str, session_id: Optional[str] = None, owner: Optional[str] = None) -> Dict:
"""Manage sessions: rename, archive, delete, important, truncate, fork. """Manage sessions: rename, archive, delete, important, truncate, fork.
+1 -6
View File
@@ -66,7 +66,6 @@ class WebSearchTool:
return { return {
"error": f"web_search failed: {type(e).__name__}: {str(e) or 'no details'}", "error": f"web_search failed: {type(e).__name__}: {str(e) or 'no details'}",
"exit_code": 1, "exit_code": 1,
"untrusted_content": True,
} }
if progress_cb: if progress_cb:
await progress_cb({ await progress_cb({
@@ -137,11 +136,7 @@ class WebFetchTool:
if not text: if not text:
if err: if err:
return { return {"error": f"web_fetch: {url}: {err}", "exit_code": 1}
"error": f"web_fetch: {url}: {err}",
"exit_code": 1,
"untrusted_content": True,
}
return {"error": f"web_fetch: {url}: no readable text content (not HTML, or the page needs JS/login)", "exit_code": 1} return {"error": f"web_fetch: {url}: no readable text content (not HTML, or the page needs JS/login)", "exit_code": 1}
# Tell the model when the download budget cut the body short and how # Tell the model when the download budget cut the body short and how
+6 -24
View File
@@ -324,10 +324,7 @@ async def do_pipeline(content: str, session_id: Optional[str] = None, owner: Opt
} }
except Exception as e: except Exception as e:
logger.error(f"pipeline failed at step {len(step_outputs) + 1}: {e}") logger.error(f"pipeline failed at step {len(step_outputs) + 1}: {e}")
return { return {"error": f"Pipeline failed at step {len(step_outputs) + 1}: {e}"}
"error": f"Pipeline failed at step {len(step_outputs) + 1}: {e}",
"untrusted_content": True,
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -1092,10 +1089,7 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
error_text = err_json.get("error", {}).get("message", error_text) if isinstance(err_json.get("error"), dict) else str(err_json.get("error", error_text)) error_text = err_json.get("error", {}).get("message", error_text) if isinstance(err_json.get("error"), dict) else str(err_json.get("error", error_text))
except Exception: except Exception:
pass pass
return { return {"error": f"Image generation failed ({resp.status_code}): {error_text}"}
"error": f"Image generation failed ({resp.status_code}): {error_text}",
"untrusted_content": True,
}
data = resp.json() data = resp.json()
images = data.get("data", []) images = data.get("data", [])
@@ -1179,10 +1173,7 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
except httpx.TimeoutException: except httpx.TimeoutException:
return {"error": "Image generation timed out (300s). The model may be overloaded — try again or use quality=low."} return {"error": "Image generation timed out (300s). The model may be overloaded — try again or use quality=low."}
except Exception as e: except Exception as e:
return { return {"error": f"Image generation error: {str(e)}"}
"error": f"Image generation error: {str(e)}",
"untrusted_content": True,
}
async def do_edit_image( async def do_edit_image(
@@ -1319,10 +1310,7 @@ async def do_edit_image(
error_text = err_json.get("detail") or err_json.get("error") or error_text error_text = err_json.get("detail") or err_json.get("error") or error_text
except Exception: except Exception:
pass pass
return { return {"error": f"Image edit fallback failed ({fallback_resp.status_code}): {error_text}"}
"error": f"Image edit fallback failed ({fallback_resp.status_code}): {error_text}",
"untrusted_content": True,
}
fallback_data = fallback_resp.json() fallback_data = fallback_resp.json()
image_b64 = fallback_data.get("image") image_b64 = fallback_data.get("image")
if not image_b64: if not image_b64:
@@ -1406,10 +1394,7 @@ async def do_edit_image(
"model for attached-image prompts." "model for attached-image prompts."
) )
} }
return { return {"error": f"Image edit failed ({resp.status_code}): {error_text}"}
"error": f"Image edit failed ({resp.status_code}): {error_text}",
"untrusted_content": True,
}
data = resp.json() data = resp.json()
images = data.get("data", []) images = data.get("data", [])
@@ -1449,10 +1434,7 @@ async def do_edit_image(
except httpx.TimeoutException: except httpx.TimeoutException:
return {"error": "Image edit timed out. The model may still be loading or overloaded."} return {"error": "Image edit timed out. The model may still be loading or overloaded."}
except Exception as e: except Exception as e:
return { return {"error": f"Image edit error: {str(e)}"}
"error": f"Image edit error: {str(e)}",
"untrusted_content": True,
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+1 -13
View File
@@ -4,8 +4,6 @@ import os
from typing import Optional from typing import Optional
from fastapi import Request, HTTPException from fastapi import Request, HTTPException
from src.owner_identity import auth_disabled, effective_storage_owner
def get_current_user(request: Request) -> Optional[str]: def get_current_user(request: Request) -> Optional[str]:
"""Get current username from request state (set by auth middleware).""" """Get current username from request state (set by auth middleware)."""
@@ -58,17 +56,7 @@ def _auth_disabled() -> bool:
"""True when the operator has explicitly turned off auth via .env. """True when the operator has explicitly turned off auth via .env.
Mirrors the AUTH_ENABLED parse in app.py / core/middleware.py so the Mirrors the AUTH_ENABLED parse in app.py / core/middleware.py so the
three call sites agree on what "off" means.""" three call sites agree on what "off" means."""
return auth_disabled() return os.getenv("AUTH_ENABLED", "true").lower() == "false"
def storage_owner_for_request(request: Request) -> Optional[str]:
"""Resolve the storage owner for code paths that need an owner bucket.
This does not replace route authentication. It only gives auth-disabled
no-login mode a stable storage identity instead of writing new data as
legacy NULL/ownerless state.
"""
return effective_storage_owner(effective_user(request))
def require_user(request: Request) -> str: def require_user(request: Request) -> str:
+9 -20
View File
@@ -15,7 +15,6 @@ import json
import logging import logging
from src import bg_jobs from src import bg_jobs
from src.prompt_security import untrusted_context_message
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -26,16 +25,6 @@ POLL_INTERVAL_S = 5
_FOLLOWUP_MAX_ROUNDS = 12 _FOLLOWUP_MAX_ROUNDS = 12
def _background_result_message(rec):
inject = (
f"[Background job {rec['id']} finished]\n\n"
f"{bg_jobs.result_text(rec)}\n\n"
"Continue the task using this output. Don't repeat work that's already done. "
"If the task is now complete, give the user the final result."
)
return untrusted_context_message("background job output", inject)
async def _drain_agent(sess, messages): async def _drain_agent(sess, messages):
"""Run the agent loop headless against a session. Returns """Run the agent loop headless against a session. Returns
(final_prose, tool_events) tool_events in the same shape the live chat (final_prose, tool_events) tool_events in the same shape the live chat
@@ -73,19 +62,13 @@ async def _drain_agent(sess, messages):
round_num = d.get("round", round_num) round_num = d.get("round", round_num)
elif d.get("type") == "tool_output": elif d.get("type") == "tool_output":
# Mirror the live chat's tool_event shape (chat_routes / chatRenderer). # Mirror the live chat's tool_event shape (chat_routes / chatRenderer).
tool_event = { tool_events.append({
"round": round_num, "round": round_num,
"tool": d.get("tool"), "tool": d.get("tool"),
"command": d.get("command"), "command": d.get("command"),
"output": d.get("output"), "output": d.get("output"),
"exit_code": d.get("exit_code"), "exit_code": d.get("exit_code"),
} })
if isinstance(d.get("ask_user"), dict):
# Preserve exact-approval cards from a tainted background-job
# continuation so the user can authorize the sealed action on
# the next foreground turn instead of losing it headlessly.
tool_event["ask_user"] = d["ask_user"]
tool_events.append(tool_event)
return full, tool_events return full, tool_events
@@ -118,8 +101,14 @@ async def _run_followup(rec: dict) -> bool:
except Exception: except Exception:
pass pass
inject = (
f"[Background job {rec['id']} finished]\n\n"
f"{bg_jobs.result_text(rec)}\n\n"
"Continue the task using this output. Don't repeat work that's already done. "
"If the task is now complete, give the user the final result."
)
context = sess.get_context_messages() context = sess.get_context_messages()
context.append(_background_result_message(rec)) context.append({"role": "user", "content": inject})
full, tool_events = await _drain_agent(sess, context) full, tool_events = await _drain_agent(sess, context)
+1 -15
View File
@@ -810,27 +810,13 @@ async def action_tidy_research(owner: str, **kwargs) -> Tuple[str, bool]:
Research history lives entirely in data/deep_research/<id>.json and is NOT Research history lives entirely in data/deep_research/<id>.json and is NOT
backed by chat-session rows so a file must never be deleted just because backed by chat-session rows so a file must never be deleted just because
no chat session matches its id. Only prune files that fail to load. no chat session matches its id. Only prune files that fail to load."""
A broken file has no readable owner stamp, so it cannot be matched against
`owner`. Clearing one is privileged: admins and the single-user operator
(AUTH_ENABLED=false) may, a regular user may not, and neither may anyone
during the pre-setup window before an admin exists.
"""
try: try:
from pathlib import Path from pathlib import Path
import json as _json import json as _json
from src.tool_security import owner_is_admin_or_single_user
research_dir = Path(DEEP_RESEARCH_DIR) research_dir = Path(DEEP_RESEARCH_DIR)
if not research_dir.exists(): if not research_dir.exists():
raise TaskNoop("no research directory") raise TaskNoop("no research directory")
if not owner_is_admin_or_single_user(owner):
# Return before the glob rather than filtering inside the loop: the
# loop reports "none broken" off an empty `removed`, which reaches
# Activity as a false report to a user whose files it skipped, and a
# regular user need not read every owner's file to learn it may
# delete none of them.
raise TaskNoop("not permitted to remove unattributable research files")
files = list(research_dir.glob("*.json")) files = list(research_dir.glob("*.json"))
removed = [] removed = []
for p in files: for p in files:
+3 -35
View File
@@ -381,10 +381,7 @@ class ChatProcessor:
) )
if len(rag_content) > 10000: if len(rag_content) > 10000:
rag_content = rag_content[:10000] + "\n[Truncated]" rag_content = rag_content[:10000] + "\n[Truncated]"
preface.append(untrusted_context_message( preface.append(untrusted_context_message("retrieved documents", rag_content))
"retrieved documents",
rag_content,
))
except Exception as e: except Exception as e:
logger.warning(f"RAG retrieval failed: {e}") logger.warning(f"RAG retrieval failed: {e}")
@@ -462,38 +459,12 @@ class ChatProcessor:
skip_url_fetch = len(message) > 2000 or len(non_yt_urls) > 3 skip_url_fetch = len(message) > 2000 or len(non_yt_urls) > 3
if not skip_url_fetch: if not skip_url_fetch:
for url in non_yt_urls: for url in non_yt_urls:
try: result = fetch_webpage_content(url)
result = fetch_webpage_content(url)
except Exception:
# The URL and exception can both contain signed-query
# credentials or response-controlled text. Keep the log
# diagnostic stable as well as the model-facing context.
logger.warning("Automatic URL fetch failed while building context")
result = {"success": False, "error": ""}
if result.get('success'): if result.get('success'):
content = result.get('content', '')[:10000] content = result.get('content', '')[:10000]
preface.append(untrusted_context_message( preface.append(untrusted_context_message(
f"web page: {url}", f"web page: {url}",
f"Content from {url}:\n\n{content}", f"Content from {url}:\n\n{content}",
provenance_origin="external",
))
else:
# A failed automatic URL fetch is context too. Never pass
# exception text or response-controlled diagnostics back to
# the model: reduce the result to a small transport-owned
# status and explicitly state that the page was not read.
error = str(result.get("error") or "")
status = "the page was unavailable"
status_match = re.match(r"^HTTP\s+(\d{3})\b", error)
if status_match:
status = f"the server returned HTTP {status_match.group(1)}"
elif error.startswith("TooLarge:"):
status = "the response exceeded the fetch size limit"
elif error.startswith("Rate limit"):
status = "the request was rate limited"
preface.append(untrusted_context_message(
"web page fetch failure",
f"A linked page was not read: {status}.",
)) ))
# Skills index — progressive disclosure. Only injected when the # Skills index — progressive disclosure. Only injected when the
@@ -517,9 +488,6 @@ class ChatProcessor:
for s in sorted(by_cat[cat], key=lambda x: x["name"]): for s in sorted(by_cat[cat], key=lambda x: x["name"]):
desc = s.get("description") or "" desc = s.get("description") or ""
lines.append(f" - {s['name']}: {desc}" if desc else f" - {s['name']}") lines.append(f" - {s['name']}: {desc}" if desc else f" - {s['name']}")
preface.append(untrusted_context_message( preface.append(untrusted_context_message("available skills index", "\n".join(lines)))
"available skills index",
"\n".join(lines),
))
return preface, rag_sources, web_sources return preface, rag_sources, web_sources
+1 -1
View File
@@ -4,7 +4,7 @@ import os
from src.runtime_paths import get_app_root, get_default_data_dir from src.runtime_paths import get_app_root, get_default_data_dir
APP_VERSION = "1.0.3" APP_VERSION = "1.0.2"
# Base paths # Base paths
BASE_DIR = os.path.join(get_app_root(), "") BASE_DIR = os.path.join(get_app_root(), "")
+2 -62
View File
@@ -282,9 +282,7 @@ def trim_for_context(messages: List[Dict], context_length: int, reserve_tokens:
if essential_system: if essential_system:
sys_text = essential_system[0].get("content", "") sys_text = essential_system[0].get("content", "")
if len(sys_text) > 2000: if len(sys_text) > 2000:
truncated_system = dict(essential_system[0]) essential_system[0] = {"role": "system", "content": sys_text[:2000] + "\n[System prompt truncated for context limits]"}
truncated_system["content"] = sys_text[:2000] + "\n[System prompt truncated for context limits]"
essential_system[0] = truncated_system
trimmed = essential_system + convo_msgs trimmed = essential_system + convo_msgs
if estimate_tokens(trimmed) <= budget: if estimate_tokens(trimmed) <= budget:
return _sanitize_tool_messages(essential_system + protected_msgs + convo_msgs) return _sanitize_tool_messages(essential_system + protected_msgs + convo_msgs)
@@ -327,9 +325,6 @@ async def maybe_compact(
messages: List[Dict], messages: List[Dict],
headers: Optional[Dict] = None, headers: Optional[Dict] = None,
owner: Optional[str] = None, owner: Optional[str] = None,
*,
persist: bool = True,
compaction_state: Optional[Dict[str, Any]] = None,
) -> tuple: ) -> tuple:
"""Check context usage and compact if above threshold. """Check context usage and compact if above threshold.
@@ -421,17 +416,7 @@ async def maybe_compact(
# offset — session.history INCLUDES the system messages, but # offset — session.history INCLUDES the system messages, but
# split_point is indexed against convo_msgs which does NOT. Without # split_point is indexed against convo_msgs which does NOT. Without
# this, the slice drops the leading system message(s). # this, the slice drops the leading system message(s).
if compaction_state is not None: _update_session_history(session, split_point, summary, system_msg_count=len(system_msgs))
compaction_state.update({
"split_point": split_point,
"summary": summary,
"system_msg_count": len(system_msgs),
"applied": False,
})
if persist:
_update_session_history(session, split_point, summary, system_msg_count=len(system_msgs))
if compaction_state is not None:
compaction_state["applied"] = True
new_used = estimate_tokens(compacted) new_used = estimate_tokens(compacted)
logger.info( logger.info(
@@ -442,51 +427,6 @@ async def maybe_compact(
return compacted, context_length, True return compacted, context_length, True
def apply_compaction_state(session, compaction_state: Optional[Dict[str, Any]]) -> bool:
"""Persist a route-specific compaction after that route commits output.
Candidate prompts may be compacted speculatively while an explicit
foreground fallback chain is being tried. Persisting at construction time
would let an unavailable route rewrite history before another route answers,
so callers hold this small plan and apply only the winning route's plan.
"""
state = compaction_state if isinstance(compaction_state, dict) else None
if not state or state.get("applied"):
return False
summary = state.get("summary")
split_point = state.get("split_point")
system_msg_count = state.get("system_msg_count", 0)
if not isinstance(summary, str) or not isinstance(split_point, int):
return False
_update_session_history(
session,
split_point,
summary,
system_msg_count=system_msg_count if isinstance(system_msg_count, int) else 0,
)
state["applied"] = True
return True
def apply_compaction_state_for_session(
session_id: Optional[str],
compaction_state: Optional[Dict[str, Any]],
) -> bool:
"""Resolve an in-memory session and apply a deferred compaction plan."""
if not session_id:
return False
try:
from core.models import get_session_manager_instance
manager = get_session_manager_instance()
session = manager.get_session(session_id) if manager else None
except Exception:
session = None
return apply_compaction_state(session, compaction_state) if session else False
def _update_session_history(session, split_point: int, summary: str, def _update_session_history(session, split_point: int, summary: str,
system_msg_count: int = 0): system_msg_count: int = 0):
"""Update the in-memory session history after compaction. """Update the in-memory session history after compaction.
+20 -216
View File
@@ -5,7 +5,6 @@ Consolidates the 4+ copies of normalize_base / resolve_endpoint logic into one p
""" """
import json import json
import ipaddress
import logging import logging
import socket import socket
import subprocess import subprocess
@@ -28,43 +27,6 @@ _NON_CHAT_MODEL = (
) )
def endpoint_cost_tracked(url: str, endpoint_kind: Optional[str] = None) -> bool:
"""Return whether token cost should be tracked for a concrete route.
This is intentionally a non-secret route classification. It mirrors the
frontend's local/subscription exclusions without exposing endpoint URLs to
message metadata.
"""
try:
parsed = urlparse(url or "")
host = (parsed.hostname or "").lower().rstrip(".")
path = (parsed.path or "").rstrip("/")
except Exception:
return False
if not host:
return False
if host == "chatgpt.com" and (
path == "/backend-api/codex" or path.startswith("/backend-api/codex/")
):
return False
kind = str(endpoint_kind or "auto").strip().lower()
if kind == "local":
return False
if kind in {"api", "proxy"}:
return True
if host in {"localhost", "0.0.0.0", "host.docker.internal"} or host.endswith(".local"):
return False
try:
ip = ipaddress.ip_address(host)
return ip.is_global
except ValueError:
pass
if "." not in host:
return False
return True
def _first_chat_model(models) -> Optional[str]: def _first_chat_model(models) -> Optional[str]:
"""First model that isn't an embedding/tts/etc.; falls back to models[0].""" """First model that isn't an embedding/tts/etc.; falls back to models[0]."""
for m in (models or []): for m in (models or []):
@@ -434,14 +396,10 @@ def resolve_endpoint(
db.close() db.close()
def _resolve_endpoint_by_id_with_descriptor( def resolve_endpoint_by_id(
ep_id: str, ep_id: str, model: Optional[str] = None, owner: Optional[str] = None
model: Optional[str] = None, ) -> Optional[Tuple[str, str, Dict]]:
owner: Optional[str] = None, """Resolve a specific endpoint id (+ optional model) to (chat_url, model, headers).
*,
require_exact_model: bool = False,
) -> Optional[Tuple[Tuple[str, str, Dict], dict]]:
"""Resolve a concrete endpoint/model plus its non-secret descriptor.
Returns None if the endpoint doesn't exist or is disabled. Used to turn Returns None if the endpoint doesn't exist or is disabled. Used to turn
a configured fallback entry ({endpoint_id, model}) into a dispatch target. a configured fallback entry ({endpoint_id, model}) into a dispatch target.
@@ -468,34 +426,15 @@ def _resolve_endpoint_by_id_with_descriptor(
chat_url = build_chat_url(base) chat_url = build_chat_url(base)
headers = build_headers(api_key, base) headers = build_headers(api_key, base)
m = (model or "").strip() m = (model or "").strip()
enabled_models = _endpoint_enabled_models(ep) # Drop a model the user disabled on the endpoint, then pick the first
if require_exact_model: # enabled chat model rather than a hidden one.
# Explicit foreground fallback entries are concrete choices. A if m and m in _endpoint_hidden_models(ep):
# hidden or known-missing model must disable the entry instead of m = ""
# silently substituting another model from the endpoint. if not m:
if not m or m in _endpoint_hidden_models(ep): m = _first_chat_model(_endpoint_enabled_models(ep)) or ""
return None
if enabled_models and m not in enabled_models:
return None
else:
# Legacy Utility/Vision chains retain their model-repair behavior.
if m and m in _endpoint_hidden_models(ep):
m = ""
if not m:
m = _first_chat_model(enabled_models) or ""
if not m: if not m:
return None return None
return ( return chat_url, m, headers
(chat_url, m, headers),
{
"endpoint_id": ep.id,
"endpoint_label": getattr(ep, "name", None) or ep.id,
"endpoint_cost_tracked": endpoint_cost_tracked(
chat_url,
getattr(ep, "endpoint_kind", None),
),
},
)
except Exception as e: except Exception as e:
logger.debug(f"Could not resolve endpoint {ep_id}: {e}") logger.debug(f"Could not resolve endpoint {ep_id}: {e}")
return None return None
@@ -503,101 +442,11 @@ def _resolve_endpoint_by_id_with_descriptor(
db.close() db.close()
def resolve_endpoint_by_id( def resolve_chat_fallback_candidates(owner: Optional[str] = None) -> list:
ep_id: str, """Compatibility shim for the retired default-chat fallback chain."""
model: Optional[str] = None,
owner: Optional[str] = None,
*,
require_exact_model: bool = False,
) -> Optional[Tuple[str, str, Dict]]:
"""Resolve a specific endpoint id (+ optional model) to its runtime route."""
resolved = _resolve_endpoint_by_id_with_descriptor( del owner
ep_id, return []
model,
owner=owner,
require_exact_model=require_exact_model,
)
return resolved[0] if resolved else None
def resolve_route_descriptor(
endpoint_url: str,
model: str,
headers: Optional[Dict] = None,
owner: Optional[str] = None,
) -> dict:
"""Return the visible endpoint identity for an already-resolved route.
Headers are compared only inside the process so two endpoints using the
same provider URL/model but different credentials remain distinguishable.
No credential material is returned or logged.
"""
if not endpoint_url or not model:
return {
"endpoint_id": None,
"endpoint_label": "Selected route",
"endpoint_cost_tracked": endpoint_cost_tracked(endpoint_url),
}
db = SessionLocal()
try:
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
if owner:
from src.auth_helpers import owner_filter
q = owner_filter(q, ModelEndpoint, owner)
expected = (endpoint_url.rstrip("/"), model, headers or {})
for ep in q.all():
resolved = _resolve_endpoint_by_id_with_descriptor(
ep.id,
model,
owner=owner,
require_exact_model=True,
)
if not resolved:
continue
candidate, descriptor = resolved
actual = (candidate[0].rstrip("/"), candidate[1], candidate[2] or {})
if actual == expected:
return descriptor
except Exception as e:
logger.debug("Could not identify selected endpoint route: %s", e)
finally:
db.close()
return {
"endpoint_id": None,
"endpoint_label": "Selected route",
"endpoint_cost_tracked": endpoint_cost_tracked(endpoint_url),
}
def resolve_route_descriptor_by_id(
endpoint_id: str,
endpoint_url: str,
model: str,
headers: Optional[Dict] = None,
owner: Optional[str] = None,
) -> Optional[dict]:
"""Resolve a selected route's identity without relying on row order.
The explicit endpoint id is still verified against the resolved runtime
route. This prevents stale or mismatched request metadata from being used
for attribution while disambiguating endpoints whose routes are otherwise
identical.
"""
resolved = _resolve_endpoint_by_id_with_descriptor(
endpoint_id,
model,
owner=owner,
require_exact_model=True,
)
if not resolved:
return None
candidate, descriptor = resolved
expected = ((endpoint_url or "").rstrip("/"), model, headers or {})
actual = (candidate[0].rstrip("/"), candidate[1], candidate[2] or {})
return descriptor if actual == expected else None
def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list: def resolve_utility_fallback_candidates(owner: Optional[str] = None) -> list:
@@ -611,62 +460,17 @@ def resolve_vision_fallback_candidates(owner: Optional[str] = None) -> list:
def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None) -> list: def _resolve_fallback_candidates(setting_key: str, owner: Optional[str] = None) -> list:
out = []
try: try:
from src.settings import get_user_setting, load_settings from src.settings import get_user_setting, load_settings
settings = load_settings() settings = load_settings()
chain = get_user_setting(setting_key, owner or "", settings.get(setting_key) or []) or [] chain = get_user_setting(setting_key, owner or "", settings.get(setting_key) or []) or []
except Exception: except Exception:
return [] return out
return resolve_fallback_entries(chain, owner=owner) for entry in chain:
def resolve_fallback_entries(
entries,
owner: Optional[str] = None,
*,
require_exact_model: bool = False,
) -> list:
"""Resolve ordered endpoint/model entries within the caller's owner scope."""
out = []
for entry in entries or []:
if not isinstance(entry, dict): if not isinstance(entry, dict):
continue continue
resolved = resolve_endpoint_by_id( resolved = resolve_endpoint_by_id(entry.get("endpoint_id", ""), entry.get("model", ""), owner=owner)
entry.get("endpoint_id", ""), if resolved:
entry.get("model", ""),
owner=owner,
require_exact_model=require_exact_model,
)
if resolved and resolved not in out:
out.append(resolved) out.append(resolved)
return out return out
def resolve_fallback_entries_with_descriptors(
entries,
owner: Optional[str] = None,
*,
require_exact_model: bool = False,
) -> list:
"""Resolve ordered entries while retaining safe endpoint provenance."""
out = []
seen = []
for entry in entries or []:
if not isinstance(entry, dict):
continue
resolved = _resolve_endpoint_by_id_with_descriptor(
entry.get("endpoint_id", ""),
entry.get("model", ""),
owner=owner,
require_exact_model=require_exact_model,
)
if not resolved:
continue
candidate, descriptor = resolved
if any(candidate == prior for prior in seen):
continue
seen.append(candidate)
out.append((candidate, descriptor))
return out
+14 -189
View File
@@ -1,155 +1,22 @@
"""Explicit foreground Chat and Agent model-routing policy.""" """Foreground Chat and Agent model-routing policy.
from dataclasses import dataclass The selected session model is strict by default. Historical
from typing import Any, Collection, Dict, FrozenSet, Optional, Tuple ``default_model_fallbacks`` values remain stored for compatibility, but this
policy intentionally does not read or migrate them.
"""
from src.endpoint_resolver import ( from typing import Any, Dict, Optional
endpoint_cost_tracked,
resolve_fallback_entries,
resolve_fallback_entries_with_descriptors,
resolve_route_descriptor,
resolve_route_descriptor_by_id,
)
_DEFAULT_FALLBACK_ENTRY_RESOLVER = resolve_fallback_entries
FOREGROUND_FALLBACK_ENABLED_KEY = "foreground_fallback_enabled"
FOREGROUND_FALLBACK_LIST_KEY = "foreground_model_fallbacks"
FOREGROUND_AVAILABILITY_STATUSES: FrozenSet[int] = frozenset({
408, 425, 429, 500, 502, 503, 504, 507, 508, 529,
})
MAX_FOREGROUND_FALLBACKS = 10
@dataclass(frozen=True)
class ForegroundModelPolicy:
"""Resolved per-user foreground fallback policy."""
enabled: bool = False
fallback_candidates: Tuple[tuple, ...] = ()
fallback_descriptors: Tuple[dict, ...] = ()
eligible_statuses: FrozenSet[int] = FOREGROUND_AVAILABILITY_STATUSES
fallback_on_empty: bool = False
def _load_policy_preferences(owner: Optional[str]) -> dict:
"""Load only preferences that explicitly belong to ``owner``.
The generic preferences loader intentionally treats a legacy flat store as
the single-user preferences object. That compatibility must not cross an
authentication transition: once a named owner is present, foreground
fallback consent exists only in an actual ``_users[owner]`` dictionary.
"""
from routes import prefs_routes
if owner is None:
prefs = prefs_routes._load_for_user(None)
return dict(prefs) if isinstance(prefs, dict) else {}
raw = prefs_routes._load()
users = raw.get("_users") if isinstance(raw, dict) else None
if not isinstance(users, dict):
return {}
prefs = users.get(owner)
return dict(prefs) if isinstance(prefs, dict) else {}
def resolve_foreground_model_policy(
owner: Optional[str] = None,
allowed_models: Optional[Collection[str]] = None,
) -> ForegroundModelPolicy:
"""Resolve an explicit owner-scoped policy, failing closed to strict mode.
The policy is stored in user preferences even when authentication is
disabled. Historical ``default_model_fallbacks`` values are deliberately
unrelated and are never read or migrated.
"""
try:
prefs = _load_policy_preferences(owner)
except Exception:
return ForegroundModelPolicy()
if prefs.get(FOREGROUND_FALLBACK_ENABLED_KEY) is not True:
return ForegroundModelPolicy()
entries = prefs.get(FOREGROUND_FALLBACK_LIST_KEY)
if not isinstance(entries, list) or not entries:
return ForegroundModelPolicy()
if allowed_models is not None:
allowed = frozenset(allowed_models)
entries = [
entry for entry in entries
if (
isinstance(entry, dict)
and isinstance(entry.get("model"), str)
and entry.get("model") in allowed
)
]
if not entries:
return ForegroundModelPolicy()
entries = entries[:MAX_FOREGROUND_FALLBACKS]
if resolve_fallback_entries is not _DEFAULT_FALLBACK_ENTRY_RESOLVER:
# Preserve the long-standing resolver seam used by downstream tests and
# integrations. Production uses the descriptor-aware resolver below.
compatibility_candidates = resolve_fallback_entries(
entries,
owner=owner,
require_exact_model=True,
)
# Known limitation of this test-only seam: alignment matches on model
# alone, so when two entries share a model and the resolver skips the
# first, the surviving candidate inherits the skipped entry's
# endpoint_id. Production uses the descriptor-aware branch below,
# which is unaffected.
resolved_routes = []
remaining_entries = list(entries)
for candidate in compatibility_candidates:
matching_index = next(
(
index for index, entry in enumerate(remaining_entries)
if isinstance(entry, dict)
and entry.get("model") == candidate[1]
),
None,
)
matching_entry = (
remaining_entries.pop(matching_index)
if matching_index is not None
else {}
)
descriptor = {
"endpoint_id": matching_entry.get("endpoint_id"),
"endpoint_label": matching_entry.get("endpoint_id") or "Fallback route",
"endpoint_cost_tracked": endpoint_cost_tracked(candidate[0]),
}
resolved_routes.append((candidate, descriptor))
else:
resolved_routes = resolve_fallback_entries_with_descriptors(
entries,
owner=owner,
require_exact_model=True,
)
candidates = [candidate for candidate, _descriptor in resolved_routes]
if not candidates:
return ForegroundModelPolicy()
return ForegroundModelPolicy(
enabled=True,
fallback_candidates=tuple(candidates),
fallback_descriptors=tuple(
dict(descriptor) for _candidate, descriptor in resolved_routes
),
)
def resolve_foreground_fallback_candidates(owner: Optional[str] = None) -> list: def resolve_foreground_fallback_candidates(owner: Optional[str] = None) -> list:
"""Return only candidates explicitly enabled by the current user.""" """Return fallback candidates for a foreground Chat or Agent request.
return list(resolve_foreground_model_policy(owner).fallback_candidates) Foreground routing is strict, so no alternate endpoint/model is eligible.
``owner`` is accepted to keep this policy boundary owner-aware.
"""
del owner
return []
def build_foreground_model_candidates( def build_foreground_model_candidates(
@@ -157,50 +24,8 @@ def build_foreground_model_candidates(
model: str, model: str,
headers: Optional[Dict[str, Any]] = None, headers: Optional[Dict[str, Any]] = None,
owner: Optional[str] = None, owner: Optional[str] = None,
policy: Optional[ForegroundModelPolicy] = None,
) -> list: ) -> list:
"""Build the ordered candidate list for a foreground request.""" """Build the ordered candidate list for a foreground request."""
policy = policy or resolve_foreground_model_policy(owner)
primary = (endpoint_url, model, headers or {}) primary = (endpoint_url, model, headers or {})
candidates = [primary] return [primary] + resolve_foreground_fallback_candidates(owner=owner)
for candidate in policy.fallback_candidates:
if candidate not in candidates:
candidates.append(candidate)
return candidates
def build_foreground_route_descriptors(
endpoint_url: str,
model: str,
headers: Optional[Dict[str, Any]] = None,
owner: Optional[str] = None,
policy: Optional[ForegroundModelPolicy] = None,
selected_endpoint_id: Optional[str] = None,
) -> list:
"""Build safe route metadata parallel to foreground candidates."""
policy = policy or resolve_foreground_model_policy(owner)
selected = None
if selected_endpoint_id:
selected = resolve_route_descriptor_by_id(
selected_endpoint_id,
endpoint_url,
model,
headers or {},
owner=owner,
)
if selected is None:
selected = resolve_route_descriptor(endpoint_url, model, headers or {}, owner=owner)
primary = (endpoint_url, model, headers or {})
candidates = [primary]
descriptors = [selected]
for candidate, descriptor in zip(
policy.fallback_candidates,
policy.fallback_descriptors,
):
if candidate in candidates:
continue
candidates.append(candidate)
descriptors.append(dict(descriptor))
return descriptors
+1 -8
View File
@@ -719,14 +719,7 @@ async def execute_api_call(
output = f"HTTP {status}\n{formatted}" output = f"HTTP {status}\n{formatted}"
if status >= 400: if status >= 400:
return { return {"error": output, "exit_code": 1}
"error": output,
"exit_code": 1,
# The error string includes the remote response body. Preserve
# it for diagnostics, but make its provenance explicit so the
# agent gate does not treat HTTP failure as content-free.
"untrusted_content": True,
}
return {"output": output, "exit_code": 0} return {"output": output, "exit_code": 0}
-16
View File
@@ -63,11 +63,8 @@ _PASSIVE_EXACT_PATHS = {
"/api/activity/heartbeat", "/api/activity/heartbeat",
"/api/client-perf", "/api/client-perf",
"/api/tasks/notifications", "/api/tasks/notifications",
"/api/tasks/runs/recent",
"/api/research/active", "/api/research/active",
"/api/email/urgency-state", "/api/email/urgency-state",
# UI idle poll sibling of urgency-state; must not pre-empt background tasks.
"/api/email/unread-state",
} }
_PASSIVE_PREFIXES = ( _PASSIVE_PREFIXES = (
@@ -77,19 +74,6 @@ _PASSIVE_PREFIXES = (
) )
async def maybe_stop_background_tasks_for_heartbeat(stop_background) -> bool:
"""Stop background work for browser activity only when the gate is enabled.
``stop_background`` is injected by the application boundary so this policy
remains independently testable without importing the full FastAPI app.
"""
if not _enabled():
return False
await stop_background(reason="browser heartbeat")
return True
def should_track_interactive_request(path: str, method: str = "GET") -> bool: def should_track_interactive_request(path: str, method: str = "GET") -> bool:
if not _enabled(): if not _enabled():
return False return False
+124 -883
View File
File diff suppressed because it is too large Load Diff
-2
View File
@@ -530,8 +530,6 @@ class McpManager:
"stderr": output if is_error else "", "stderr": output if is_error else "",
"exit_code": 1 if is_error else 0, "exit_code": 1 if is_error else 0,
} }
if is_error and output:
result_dict["untrusted_content"] = True
if images: if images:
result_dict["images"] = images result_dict["images"] = images
return result_dict return result_dict
+10 -24
View File
@@ -15,32 +15,18 @@ from urllib.parse import urlparse, parse_qs
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _resolve_redirect_base() -> str:
"""Origin the browser is sent back to after authorizing.
Falls back to the port the app binds natively (APP_PORT, read the same way
by app.py and launcher.py) rather than a fixed 7000: the macOS launcher
defaults to 7860, and a callback on the wrong port reaches nothing. The
hostname stays `localhost` rather than internal_api_base()'s 127.0.0.1 —
this URI is registered with the authorization server (via DCR, or by hand
for Google clients), so changing the host invalidates registrations that
already exist.
"""
return (
os.environ.get("OAUTH_REDIRECT_BASE_URL")
or os.environ.get("APP_PUBLIC_URL")
or f"http://localhost:{os.environ.get('APP_PORT', '7000')}"
).rstrip("/")
# OAuth redirect URI registered with every authorization server via DCR. Loopback # OAuth redirect URI registered with every authorization server via DCR. Loopback
# is allowed for native/desktop clients (RFC 8252); remote users finish via the # is allowed for native/desktop clients (RFC 8252); remote users finish via the
# paste-back flow. Deployments whose externally reachable origin differs from the # paste-back flow. Deployments not reachable at http://localhost:7000 (custom
# port Odysseus binds — reverse proxy, public domain, or Docker, whose host port # port, reverse proxy, or public domain) must set OAUTH_REDIRECT_BASE_URL (or
# map is invisible inside the container — must set OAUTH_REDIRECT_BASE_URL (or # APP_PUBLIC_URL) to their externally reachable origin so the redirect lands back
# APP_PUBLIC_URL), otherwise the redirect never lands back on Odysseus. # on Odysseus. APP_PORT is intentionally not used: it is only the Docker host
_REDIRECT_BASE = _resolve_redirect_base() # port-map; the app always listens on 7000 inside the container.
_REDIRECT_BASE = (
os.environ.get("OAUTH_REDIRECT_BASE_URL")
or os.environ.get("APP_PUBLIC_URL")
or "http://localhost:7000"
).rstrip("/")
REDIRECT_URI = f"{_REDIRECT_BASE}/api/mcp/oauth/callback" REDIRECT_URI = f"{_REDIRECT_BASE}/api/mcp/oauth/callback"
# How long the background connect waits for the user to authorize before giving up. # How long the background connect waits for the user to authorize before giving up.
+6 -11
View File
@@ -290,22 +290,17 @@ def detect_vendor(base_url: Any = "", endpoint_kind: Any = "") -> str:
return kind_map[kind] return kind_map[kind]
parsed = urlparse(compact_str(base_url)) parsed = urlparse(compact_str(base_url))
host = (parsed.hostname or "").lower().rstrip(".") host = (parsed.hostname or "").lower()
port = parsed.port port = parsed.port
if host.endswith("openrouter.ai"):
def host_matches(domain: str) -> bool:
domain = domain.lower().rstrip(".")
return host == domain or host.endswith(f".{domain}")
if host_matches("openrouter.ai"):
return VENDOR_OPENROUTER return VENDOR_OPENROUTER
if host_matches("openai.com"): if host.endswith("openai.com"):
return VENDOR_OPENAI return VENDOR_OPENAI
if host_matches("anthropic.com"): if host.endswith("anthropic.com"):
return VENDOR_ANTHROPIC return VENDOR_ANTHROPIC
if host_matches("googleapis.com"): if host.endswith("googleapis.com"):
return VENDOR_GOOGLE return VENDOR_GOOGLE
if host_matches("ollama.com") or port == 11434: if host.endswith("ollama.com") or port == 11434:
return VENDOR_OLLAMA return VENDOR_OLLAMA
if port == 1234: if port == 1234:
return VENDOR_LMSTUDIO return VENDOR_LMSTUDIO
-354
View File
@@ -1,354 +0,0 @@
"""SSRF-guarded synchronous HTTP fetching primitives.
This module owns outbound URL classification, one-resolution-per-hop DNS
pinning, redirects, and response-body budgets. It deliberately has no search
or content-extraction dependencies so callers outside search can reuse the
same transport boundary.
"""
from __future__ import annotations
import ipaddress
import socket
import ssl
from typing import Callable, Iterable, cast
from urllib.parse import urljoin, urlparse
import httpcore
import httpx
from src.constants import WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_SOFT_MAX_BYTES
_PRIVATE_NETWORKS = (
ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
)
def _is_private_address(addr: ipaddress._BaseAddress) -> bool:
if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
addr = addr.ipv4_mapped
return (
addr.is_private
or addr.is_loopback
or addr.is_link_local
or addr.is_reserved
or addr.is_multicast
or addr.is_unspecified
or any(addr in net for net in _PRIVATE_NETWORKS)
)
def _resolve_hostname_ips(hostname: str) -> list[ipaddress._BaseAddress]:
try:
infos = socket.getaddrinfo(hostname, None)
except Exception:
return []
out = []
for info in infos:
try:
out.append(ipaddress.ip_address(info[4][0]))
except Exception:
continue
return out
def _public_http_url(
url: str,
*,
resolver: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
) -> bool:
resolver = resolver or _resolve_hostname_ips
try:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
return False
host = (parsed.hostname or "").strip()
if not host:
return False
lower = host.lower()
if lower in ("localhost", "metadata", "metadata.google.internal"):
return False
if lower.endswith((".local", ".localhost", ".internal", ".lan", ".intranet")):
return False
try:
return not _is_private_address(ipaddress.ip_address(host))
except ValueError:
pass
addrs = resolver(host)
return bool(addrs) and not any(_is_private_address(a) for a in addrs)
except Exception:
return False
def _resolve_public_ips(
url: str,
*,
resolver: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
) -> list[ipaddress._BaseAddress]:
resolver = resolver or _resolve_hostname_ips
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise httpx.RequestError(f"Blocked non-public URL: {url}")
host = (parsed.hostname or "").strip().lower()
if host in ("localhost", "metadata", "metadata.google.internal"):
raise httpx.RequestError(f"Blocked non-public hostname: {host}")
try:
ip = ipaddress.ip_address(host)
if _is_private_address(ip):
raise httpx.RequestError(f"Blocked non-public IP literal: {host}")
return [ip]
except httpx.RequestError:
raise
except ValueError:
pass
addrs = resolver(host)
if not addrs or any(_is_private_address(a) for a in addrs):
raise httpx.RequestError(f"Blocked non-public URL: {url}")
return addrs
class _PinnedBackend(httpcore.NetworkBackend):
"""Network backend that connects to a pre-resolved IP."""
def __init__(self, ip: ipaddress._BaseAddress):
self._ip = str(ip)
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
return self._real.connect_tcp(
self._ip, port, timeout, local_address, socket_options
)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Transport that pins every TCP connect to a pre-resolved IP."""
def __init__(self, ip: ipaddress._BaseAddress, *, http2: bool = False):
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
http1=True,
http2=http2,
network_backend=_PinnedBackend(ip),
)
def __enter__(self):
self._pool.__enter__()
return self
def __exit__(self, exc_type=None, exc_value=None, traceback=None) -> None:
self._pool.__exit__(exc_type, exc_value, traceback)
def handle_request(self, request: httpx.Request) -> httpx.Response:
httpcore_req = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
try:
httpcore_resp = self._pool.handle_request(httpcore_req)
content = b"".join(cast(Iterable[bytes], httpcore_resp.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
return httpx.Response(
status_code=httpcore_resp.status,
headers=httpcore_resp.headers,
content=content,
extensions=httpcore_resp.extensions,
)
def close(self) -> None:
self._pool.close()
class BodyTooLargeError(Exception):
"""The server declared a body larger than the hard fetch ceiling."""
def __init__(self, url: str, declared_bytes: int):
self.url = url
self.declared_bytes = declared_bytes
super().__init__(
f"response body is {declared_bytes:,} bytes, over the "
f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
)
class _CappedFetch:
"""Result of a size-capped streaming GET."""
__slots__ = (
"status_code",
"headers",
"content",
"truncated",
"declared_bytes",
"encoding",
"url",
)
def __init__(
self,
status_code,
headers,
content,
truncated,
declared_bytes,
encoding,
url,
):
self.status_code = status_code
self.headers = headers
self.content = content
self.truncated = truncated
self.declared_bytes = declared_bytes
self.encoding = encoding
self.url = url
@property
def text(self) -> str:
return self.content.decode(self.encoding or "utf-8", errors="replace")
def raise_for_status(self):
if self.status_code >= 400:
request = httpx.Request("GET", self.url)
raise httpx.HTTPStatusError(
f"HTTP {self.status_code} for {self.url}",
request=request,
response=httpx.Response(self.status_code, request=request),
)
def _get_public_url(
url: str,
headers: dict,
timeout: int,
max_redirects: int = 5,
max_bytes: int | None = None,
*,
resolve_public_ips: Callable[[str], list[ipaddress._BaseAddress]] | None = None,
transport_factory: Callable[[ipaddress._BaseAddress], httpx.BaseTransport] | None = None,
) -> _CappedFetch:
"""Capped streaming GET with SSRF-guarded, DNS-pinned redirects."""
resolve_public_ips = resolve_public_ips or _resolve_public_ips
transport_factory = transport_factory or _PinnedTransport
cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
current = url
for _ in range(max_redirects + 1):
ips = resolve_public_ips(current)
req_headers = dict(headers or {})
req_headers["Accept-Encoding"] = "identity"
with httpx.Client(
headers=req_headers,
timeout=timeout,
follow_redirects=False,
transport=transport_factory(ips[0]),
) as client:
with client.stream("GET", current) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
if not location:
return _CappedFetch(
response.status_code,
response.headers,
b"",
False,
None,
response.encoding,
str(response.url),
)
current = urljoin(str(response.url), location)
continue
enc = (response.headers.get("content-encoding") or "").strip().lower()
if enc and enc != "identity":
raise httpx.RequestError(
f"Refusing compressed response (Content-Encoding: {enc}) after "
"requesting identity: cannot bound decoded body size",
request=httpx.Request("GET", current),
)
declared = None
raw_len = response.headers.get("content-length")
if raw_len and raw_len.isdigit():
declared = int(raw_len)
if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
raise BodyTooLargeError(current, declared)
chunks = []
read = 0
truncated = False
for chunk in response.iter_bytes():
read += len(chunk)
if read > cap:
keep = cap - (read - len(chunk))
if keep > 0:
chunks.append(chunk[:keep])
truncated = True
break
chunks.append(chunk)
return _CappedFetch(
response.status_code,
response.headers,
b"".join(chunks),
truncated,
declared,
response.encoding,
str(response.url),
)
raise httpx.RequestError(
"Too many redirects", request=httpx.Request("GET", current)
)
-56
View File
@@ -1,56 +0,0 @@
"""Shared owner identity constants and helpers."""
from __future__ import annotations
import os
from typing import Optional
DEFAULT_LOCAL_OWNER = "__odysseus_local__"
DEFAULT_LOCAL_OWNER_LABEL = "Local"
INTERNAL_TOOL_USER = "internal-tool"
REQUEST_SENTINEL_OWNERS = frozenset({INTERNAL_TOOL_USER, "api", "demo", "system"})
RESERVED_AUTH_USERNAMES = REQUEST_SENTINEL_OWNERS | {DEFAULT_LOCAL_OWNER}
def auth_disabled() -> bool:
"""Return True only when auth is explicitly disabled by configuration."""
return os.getenv("AUTH_ENABLED", "true").strip().lower() == "false"
def normalize_owner(owner: str | None) -> Optional[str]:
"""Normalize an owner-like value without inventing a fallback identity."""
value = str(owner or "").strip()
return value or None
def owner_key(owner: str | None) -> Optional[str]:
normalized = normalize_owner(owner)
return normalized.lower() if normalized else None
def is_request_sentinel_owner(owner: str | None) -> bool:
return owner_key(owner) in REQUEST_SENTINEL_OWNERS
def effective_storage_owner(owner: str | None, *, auth_is_disabled: bool | None = None) -> Optional[str]:
"""Resolve the owner used for storage writes that need a real bucket.
``None`` still means no authenticated owner when auth is enabled. In the
explicit no-login mode, it resolves to the reserved local owner instead of
conflating local-operator writes with legacy NULL/ownerless rows.
"""
normalized = normalize_owner(owner)
if normalized:
if is_request_sentinel_owner(normalized):
return None
return normalized
disabled = auth_disabled() if auth_is_disabled is None else auth_is_disabled
if disabled:
return DEFAULT_LOCAL_OWNER
return None
def is_default_local_owner(owner: str | None) -> bool:
return owner_key(owner) == DEFAULT_LOCAL_OWNER
+2 -15
View File
@@ -61,13 +61,7 @@ def _sanitize_label(label: str) -> str:
return label return label
def untrusted_context_message( def untrusted_context_message(label: str, content: Any) -> Dict[str, Any]:
label: str,
content: Any,
*,
provenance_origin: str | None = None,
arm_tool_gate: bool = True,
) -> Dict[str, Any]:
"""Return an LLM message that keeps retrieved/source text out of system role. """Return an LLM message that keeps retrieved/source text out of system role.
The template is structured so that *only* the hardcoded The template is structured so that *only* the hardcoded
@@ -79,13 +73,6 @@ def untrusted_context_message(
safe_label = _sanitize_label(label) safe_label = _sanitize_label(label)
text = "" if content is None else str(content) text = "" if content is None else str(content)
text = _escape_guard_markers(text) text = _escape_guard_markers(text)
metadata: Dict[str, Any] = {
"trusted": False,
"source": label,
"tool_gate_untrusted": bool(arm_tool_gate),
}
if provenance_origin:
metadata["provenance_origin"] = provenance_origin
return { return {
"role": "user", "role": "user",
"content": ( "content": (
@@ -95,5 +82,5 @@ def untrusted_context_message(
f"{text}\n" f"{text}\n"
f"{GUARD_CLOSE}" f"{GUARD_CLOSE}"
), ),
"metadata": metadata, "metadata": {"trusted": False, "source": label},
} }
-1
View File
@@ -12,7 +12,6 @@ class ChatRequest(BaseModel):
use_research: Optional[bool] = Field(default=False, description="Enable deep research") use_research: Optional[bool] = Field(default=False, description="Enable deep research")
time_filter: Optional[str] = Field(default=None, description="Time filter for search") time_filter: Optional[str] = Field(default=None, description="Time filter for search")
preset_id: Optional[str] = Field(default=None, description="Preset identifier") preset_id: Optional[str] = Field(default=None, description="Preset identifier")
selected_endpoint_id: Optional[str] = Field(default=None, description="Selected model endpoint ID")
@field_validator('message') @field_validator('message')
@classmethod @classmethod
+1 -19
View File
@@ -14,13 +14,6 @@ from src.constants import SETTINGS_FILE, FEATURES_FILE
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# Keys retained in the raw settings store for compatibility and rollback, but
# deliberately unavailable through generic settings APIs or agent tools. They
# must stay in ``DEFAULT_SETTINGS`` so old files continue to load without data
# loss; callers that present or mutate settings should use this set as a
# tombstone boundary.
RETIRED_SETTING_KEYS = frozenset({"default_model_fallbacks"})
# Tiny TTL cache for settings/features. get_setting() is called on hot paths # Tiny TTL cache for settings/features. get_setting() is called on hot paths
# (every chat, every preprocess); without this it re-parses the JSON each call. # (every chat, every preprocess); without this it re-parses the JSON each call.
# Picks up edits within _CACHE_TTL seconds, which is fine for human-edited config. # Picks up edits within _CACHE_TTL seconds, which is fine for human-edited config.
@@ -204,17 +197,6 @@ DEFAULT_SETTINGS = {
}, },
} }
def without_retired_settings(settings: dict) -> dict:
"""Return a shallow copy suitable for generic settings interfaces."""
if not isinstance(settings, dict):
return {}
return {
key: value
for key, value in settings.items()
if key not in RETIRED_SETTING_KEYS
}
DEFAULT_FEATURES = { DEFAULT_FEATURES = {
"web_search": True, "web_search": True,
"web_fetch": True, "web_fetch": True,
@@ -287,7 +269,7 @@ _PER_USER_KEYS = {
# Default chat endpoint / model — without per-user resolution every new # Default chat endpoint / model — without per-user resolution every new
# account inherited whatever the most-recent admin picked, which then # account inherited whatever the most-recent admin picked, which then
# got injected into the chat composer on first open. # got injected into the chat composer on first open.
"default_endpoint_id", "default_model", "default_endpoint_id", "default_model", "default_model_fallbacks",
"utility_endpoint_id", "utility_model", "utility_model_fallbacks", "utility_endpoint_id", "utility_model", "utility_model_fallbacks",
"research_endpoint_id", "research_model", "research_endpoint_id", "research_model",
} }
+5
View File
@@ -1,6 +1,7 @@
"""Shared resolver for background-task AI endpoints.""" """Shared resolver for background-task AI endpoints."""
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_chat_fallback_candidates,
resolve_endpoint, resolve_endpoint,
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
) )
@@ -31,6 +32,7 @@ def resolve_task_candidates(
2. Utility endpoint/model 2. Utility endpoint/model
3. Default endpoint/model 3. Default endpoint/model
4. Utility fallback chain 4. Utility fallback chain
5. Retired default-fallback compatibility hook (currently empty)
""" """
candidates = [] candidates = []
@@ -47,6 +49,9 @@ def resolve_task_candidates(
_append(*resolve_endpoint("default", owner=owner)) _append(*resolve_endpoint("default", owner=owner))
for url, model, headers in resolve_utility_fallback_candidates(owner=owner): for url, model, headers in resolve_utility_fallback_candidates(owner=owner):
_append(url, model, headers) _append(url, model, headers)
for url, model, headers in resolve_chat_fallback_candidates(owner=owner):
_append(url, model, headers)
return candidates return candidates
+1 -38
View File
@@ -10,7 +10,6 @@ from datetime import datetime, timedelta, timezone
from typing import Any, Awaitable, Callable, Dict, Tuple from typing import Any, Awaitable, Callable, Dict, Tuple
from core.auth import RESERVED_USERNAMES from core.auth import RESERVED_USERNAMES
from src.owner_identity import REQUEST_SENTINEL_OWNERS
from src.task_action_policy import ( from src.task_action_policy import (
is_admin_only_task_action, is_admin_only_task_action,
owner_has_admin_task_privileges, owner_has_admin_task_privileges,
@@ -1884,7 +1883,6 @@ class TaskScheduler:
pass pass
full_text = "" full_text = ""
tool_results = [] tool_results = []
approval_pause = None
# Honor per-task max_steps (defense against runaway agent loops). # Honor per-task max_steps (defense against runaway agent loops).
# Falls back to 20 if not set — the historical default. # Falls back to 20 if not set — the historical default.
@@ -1931,44 +1929,9 @@ class TaskScheduler:
tool_summary = data.get("stdout") or data.get("output") or data.get("result") or "" tool_summary = data.get("stdout") or data.get("output") or data.get("result") or ""
if isinstance(tool_summary, str) and tool_summary.strip(): if isinstance(tool_summary, str) and tool_summary.strip():
tool_results.append(f"[{data.get('tool', '?')}] {tool_summary[:500]}") tool_results.append(f"[{data.get('tool', '?')}] {tool_summary[:500]}")
approval = data.get("ask_user")
if (
isinstance(approval, dict)
and approval.get("kind") == "tool_approval"
):
approval_pause = {
"tool": data.get("tool") or "tool",
"approval_id": approval.get("approval_id"),
}
# Scheduled tasks have no interactive surface that
# can safely resume a one-use grant. Retire the
# record immediately instead of leaving it pending
# and report an explicit manual-action boundary.
try:
from src.tool_approvals import tool_approval_store
tool_approval_store.consume(
approval_pause["approval_id"],
decision="deny",
owner=task.owner,
session_id=session_id,
)
except Exception:
logger.debug(
"Could not retire scheduled-task approval",
exc_info=True,
)
break
except (json.JSONDecodeError, KeyError): except (json.JSONDecodeError, KeyError):
pass pass
if approval_pause is not None:
return (
"Scheduled task paused safely: "
f"{approval_pause['tool']} requested an exact action after "
"untrusted context. That action was not executed. Run this task "
"interactively to inspect and approve the action."
)
# Grace summarization — if the model exhausted rounds on tool calls # Grace summarization — if the model exhausted rounds on tool calls
# without producing a final text response, do one last LLM call # without producing a final text response, do one last LLM call
# asking it to summarize what it did. Guarantees output. # asking it to summarize what it did. Guarantees output.
@@ -2521,7 +2484,7 @@ class TaskScheduler:
# check-ins seeded, which then double-fire alongside the human user's # check-ins seeded, which then double-fire alongside the human user's
# check-ins. This was the root cause of the duplicate 'Morning check-in' # check-ins. This was the root cause of the duplicate 'Morning check-in'
# rows we had to manually clean up. # rows we had to manually clean up.
if not owner or owner in REQUEST_SENTINEL_OWNERS: if not owner or owner in RESERVED_USERNAMES:
logger.info(f"ensure_assistant_defaults: skip synthetic owner {owner!r}") logger.info(f"ensure_assistant_defaults: skip synthetic owner {owner!r}")
return return
from core.database import SessionLocal, CrewMember, ScheduledTask from core.database import SessionLocal, CrewMember, ScheduledTask
+74 -110
View File
@@ -439,11 +439,56 @@ async def escalate_and_learn(
failure_reason: str, failure_reason: str,
owner: Optional[str] = None, owner: Optional[str] = None,
) -> Optional[str]: ) -> Optional[str]:
"""Retire legacy background learning when no approval UI is available.""" """Call the teacher, evaluate ITS attempt, save a skill on success.
logger.info(
"background teacher learning skipped: generated skills require an " Returns the saved skill name (or None if the teacher couldn't
"interactive exact approval" write one). Logs but doesn't raise — escalation is best-effort.
"""
from src.settings import get_setting
teacher_spec = (get_setting("teacher_model", "") or "").strip()
if not teacher_spec:
return None
prompt = _TEACHER_ESCALATION_PROMPT.format(
user_request=user_request or "(no user request captured)",
failure_reason=failure_reason or "(failure reason not captured)",
untrusted_trace_guard=_UNTRUSTED_TRACE_GUARD,
trace=_format_trace(tool_results, agent_reply),
) )
response = await _call_teacher(teacher_spec, prompt, owner=owner)
if not response:
return None
skill = _extract_skill_json(response)
if not skill:
# Teacher chose not to write a skill — see prompt contract.
logger.info("teacher declined to write a skill for this failure")
return None
# Same regex eval applied to the teacher's response — if the
# teacher itself sounded uncertain ("I don't have a tool"), drop
# the skill rather than persist a sketchy one.
status, reason = evaluate_turn_regex([], response)
if status == "failure":
logger.info(f"teacher response failed eval, skipping skill save: {reason}")
return None
# Tag the skill with the escalation source for auditability.
skill.setdefault("source", "teacher-escalation")
skill.setdefault("teacher_model", teacher_spec)
# Force action=add regardless of what the teacher wrote.
skill["action"] = "add"
import json
from src.tool_implementations import do_manage_skills
try:
result = await do_manage_skills(json.dumps(skill), owner=owner)
if isinstance(result, dict) and not result.get("error"):
logger.info(f"teacher wrote skill: {skill.get('name')}")
return skill.get("name")
logger.warning(f"skill save failed: {result}")
except Exception as e:
logger.warning(f"skill save raised: {e}")
return None return None
@@ -518,12 +563,6 @@ async def run_teacher_inline(
student_tool_events: List[Dict[str, Any]], student_tool_events: List[Dict[str, Any]],
student_reply: str, student_reply: str,
owner: Optional[str] = None, owner: Optional[str] = None,
session_id: Optional[str] = None,
workspace: Optional[str] = None,
disabled_tools: Optional[set[str]] = None,
tool_policy: Any = None,
active_document: Any = None,
active_email: Optional[Dict[str, str]] = None,
): ):
"""Async generator. Yields SSE event strings. """Async generator. Yields SSE event strings.
@@ -622,7 +661,6 @@ async def run_teacher_inline(
from src.agent_loop import stream_agent_loop from src.agent_loop import stream_agent_loop
captured_tool_events: List[Dict[str, Any]] = [] captured_tool_events: List[Dict[str, Any]] = []
captured_text_parts: List[str] = [] captured_text_parts: List[str] = []
captured_metrics: Dict[str, Any] = {}
async for evt_str in stream_agent_loop( async for evt_str in stream_agent_loop(
endpoint_url=teacher_url, endpoint_url=teacher_url,
@@ -630,12 +668,6 @@ async def run_teacher_inline(
messages=teacher_messages, messages=teacher_messages,
headers=teacher_headers, headers=teacher_headers,
owner=owner, owner=owner,
session_id=session_id,
workspace=workspace,
disabled_tools=disabled_tools,
tool_policy=tool_policy,
active_document=active_document,
active_email=active_email,
_is_teacher_run=True, _is_teacher_run=True,
): ):
# Swallow teacher's own [DONE] — outer loop emits the real one # Swallow teacher's own [DONE] — outer loop emits the real one
@@ -650,21 +682,13 @@ async def run_teacher_inline(
if isinstance(payload, dict): if isinstance(payload, dict):
payload["teacher"] = True payload["teacher"] = True
typ = payload.get("type") typ = payload.get("type")
if typ == "metrics" and isinstance(payload.get("data"), dict):
# The outer chat route persists only the last metrics
# payload. Keep a copy so any approval produced after the
# recursive teacher run's metrics remains reloadable.
captured_metrics = dict(payload["data"])
if typ == "tool_output": if typ == "tool_output":
captured_tool_event = { captured_tool_events.append({
"tool": payload.get("tool"), "tool": payload.get("tool"),
"command": payload.get("command"), "command": payload.get("command"),
"output": payload.get("output"), "output": payload.get("output"),
"exit_code": payload.get("exit_code"), "exit_code": payload.get("exit_code"),
} })
if isinstance(payload.get("ask_user"), dict):
captured_tool_event["ask_user"] = payload["ask_user"]
captured_tool_events.append(captured_tool_event)
if "delta" in payload and isinstance(payload["delta"], str): if "delta" in payload and isinstance(payload["delta"], str):
if payload.get("thinking"): if payload.get("thinking"):
continue continue
@@ -673,12 +697,6 @@ async def run_teacher_inline(
continue continue
yield evt_str yield evt_str
# A takeover that paused for a question or exact action has not completed
# yet. Its server-owned approval card is already in the live/persisted tool
# events; do not evaluate the partial trace or distill it into a skill.
if any(event.get("ask_user") for event in captured_tool_events):
return
teacher_text = "".join(captured_text_parts).strip() teacher_text = "".join(captured_text_parts).strip()
t_status, t_reason = evaluate_turn_regex(captured_tool_events, teacher_text) t_status, t_reason = evaluate_turn_regex(captured_tool_events, teacher_text)
if t_status == "failure": if t_status == "failure":
@@ -722,85 +740,31 @@ async def run_teacher_inline(
skill.setdefault("source", "teacher-escalation") skill.setdefault("source", "teacher-escalation")
skill.setdefault("teacher_model", teacher_spec) skill.setdefault("teacher_model", teacher_spec)
if not session_id: import json as _json
from src.tool_implementations import do_manage_skills
try:
result = await do_manage_skills(_json.dumps(skill), owner=owner)
if isinstance(result, dict) and not result.get("error"):
logger.info(f"teacher succeeded; saved skill: {skill.get('name')}")
yield (
'data: ' + json.dumps({
"type": "skill_saved",
"name": skill.get("name"),
"category": skill.get("category", "general"),
}) + '\n\n'
)
else:
yield (
'data: ' + json.dumps({
"type": "skill_save_failed",
"reason": str(result),
}) + '\n\n'
)
except Exception as e:
logger.warning(f"skill save raised: {e}")
yield ( yield (
'data: ' + json.dumps({ 'data: ' + json.dumps({
"type": "skill_save_failed", "type": "skill_save_failed",
"reason": ( "reason": str(e),
"Teacher-generated skills require an interactive exact "
"approval before they can be saved."
),
}) + '\n\n' }) + '\n\n'
) )
return
import json as _json
import uuid as _uuid
from src.tool_approvals import tool_approval_store
from src.tool_capabilities import capabilities_for_action
skill_content = _json.dumps(skill, ensure_ascii=False)
pending = tool_approval_store.create(
owner=owner,
session_id=session_id,
origin_run_id=f"teacher-skill-{_uuid.uuid4().hex}",
tool_name="manage_skills",
content=skill_content,
workspace=workspace,
external_untrusted_context_seen=True,
capabilities=capabilities_for_action("manage_skills", skill_content),
)
approval = pending.public_payload(
reason=(
"The teacher generated this reusable skill. Review and approve "
"the complete skill definition before it is saved."
),
)
persisted_metrics = dict(captured_metrics)
persisted_tool_events = list(persisted_metrics.get("tool_events") or [])
persisted_round_texts = list(persisted_metrics.get("round_texts") or [])
prior_rounds = [
event.get("round")
for event in persisted_tool_events
if isinstance(event, dict) and isinstance(event.get("round"), int)
]
approval_round = max([len(persisted_round_texts), *prior_rounds, 0]) + 1
approval_tool_event = {
"round": approval_round,
"model": teacher_model,
"tool": "manage_skills",
"command": str(skill.get("name") or "teacher-generated skill"),
"output": "Waiting for an exact user approval.",
"exit_code": None,
"ask_user": approval,
}
persisted_tool_events.append(approval_tool_event)
persisted_metrics["tool_events"] = persisted_tool_events
persisted_metrics.setdefault("model", teacher_model)
yield (
"data: "
+ json.dumps({"delta": "Review the teacher-generated skill before saving it."})
+ "\n\n"
)
yield (
"data: "
+ json.dumps({
"type": "tool_output",
**approval_tool_event,
"teacher": True,
})
+ "\n\n"
)
yield (
"data: "
+ json.dumps({"type": "ask_user", "data": approval, "teacher": True})
+ "\n\n"
)
# This must be the final metrics event: chat_routes saves only last_metrics
# when the outer stream reaches [DONE]. Without it, the live approval card
# disappears after a reload even though the server grant remains pending.
yield (
"data: "
+ json.dumps({"type": "metrics", "data": persisted_metrics, "teacher": True})
+ "\n\n"
)
-35
View File
@@ -1,35 +0,0 @@
"""Shared wire values and scope markers for tool approval continuations."""
from __future__ import annotations
from enum import Enum
# Keep the existing wire values so the current route and no-build frontend do
# not need a second protocol migration. ``approve`` no longer means one action;
# it now selects chat-session scope.
TASK_APPROVAL_DECISION = "approve_task"
CHAT_SESSION_APPROVAL_DECISION = "approve"
DENY_APPROVAL_DECISION = "deny"
# Session.get_context_messages() adds this server-owned marker only when the
# session history contains a matching, resolved chat-session approval.
CHAT_SESSION_APPROVAL_CONTEXT_MARKER = "_tool_approval_chat_session_granted"
class ToolApprovalScope(str, Enum):
# Surfaces without a resumable chat (the skill tester, unattended audits)
# keep the original one-use meaning: the sealed action runs and the gate
# re-arms immediately for anything after it.
SINGLE_ACTION = "single_action"
TASK = "task"
CHAT_SESSION = "chat_session"
def scope_for_decision(decision: object) -> ToolApprovalScope | None:
normalized = str(decision or "").strip().lower()
if normalized == TASK_APPROVAL_DECISION:
return ToolApprovalScope.TASK
if normalized == CHAT_SESSION_APPROVAL_DECISION:
return ToolApprovalScope.CHAT_SESSION
return None
-513
View File
@@ -1,513 +0,0 @@
"""Opaque exact-action approvals with explicit task and chat scopes.
The server still seals and claims the first displayed action exactly once. The
selected scope then bypasses only the automatic post-external-context approval
gate for the rest of the resumed task or chat session. Browser-visible fields
are display copies, never authority.
"""
from __future__ import annotations
import hashlib
import json
import os
import secrets
import threading
import time
from dataclasses import dataclass, field
from typing import Any
from src.tool_approval_scopes import (
CHAT_SESSION_APPROVAL_DECISION,
DENY_APPROVAL_DECISION,
TASK_APPROVAL_DECISION,
ToolApprovalScope,
scope_for_decision,
)
from src.tool_capabilities import ToolCapabilities, capabilities_for_action
DEFAULT_APPROVAL_TTL_SECONDS = 10 * 60
DEFAULT_MAX_PENDING_APPROVALS = 2048
def _normalized_owner(owner: Any) -> str:
return str(owner or "").strip().casefold()
def _normalized_workspace(workspace: Any) -> str:
if not isinstance(workspace, str) or not workspace.strip():
return ""
return os.path.realpath(os.path.expanduser(workspace))
_MAX_APPROVAL_SELECTED_TOOLS = 512
_MAX_APPROVAL_TOOL_NAME_CHARS = 512
_MAX_APPROVAL_CONTINUATION_QUERY_CHARS = 4000
def _normalized_selected_tools(
selected_tools: Any,
*,
required_tool: Any = None,
) -> tuple[str, ...]:
if isinstance(selected_tools, str):
selected_tools = (selected_tools,)
try:
values = selected_tools or ()
names = {
name.strip()
for name in values
if (
isinstance(name, str)
and name.strip()
and len(name.strip()) <= _MAX_APPROVAL_TOOL_NAME_CHARS
)
}
required_name = str(required_tool or "").strip()
if required_name and len(required_name) <= _MAX_APPROVAL_TOOL_NAME_CHARS:
names.add(required_name)
ordered = sorted(names)
if len(ordered) <= _MAX_APPROVAL_SELECTED_TOOLS:
return tuple(ordered)
kept = ordered[:_MAX_APPROVAL_SELECTED_TOOLS]
if required_name and required_name in names and required_name not in kept:
kept[-1] = required_name
kept.sort()
return tuple(kept)
except TypeError:
return ()
def _normalized_continuation_query(value: Any) -> str:
# The query is server-derived from the interrupted run and already lives in
# session history. Keep the pending copy bounded because approvals are held
# in memory until consumed or expired.
return str(value or "").strip()[:_MAX_APPROVAL_CONTINUATION_QUERY_CHARS]
def _canonical_digest(payload: dict[str, Any]) -> str:
encoded = json.dumps(
payload,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def document_content_digest(content: Any) -> str:
"""Return the stable server-side fingerprint used to seal a document."""
return hashlib.sha256(str(content or "").encode("utf-8")).hexdigest()
def _binding_payload(
*,
owner: Any,
session_id: Any,
origin_run_id: Any,
tool_name: Any,
content: Any,
workspace: Any,
document_id: Any,
document_version: Any,
document_digest: Any,
external_untrusted_context_seen: bool,
selected_tools: Any,
continuation_query: Any,
effects: tuple[str, ...],
result_integrity: str,
) -> dict[str, Any]:
return {
"owner": _normalized_owner(owner),
"session_id": str(session_id or ""),
"origin_run_id": str(origin_run_id or ""),
"tool_name": str(tool_name or ""),
"content": str(content or ""),
"workspace": _normalized_workspace(workspace),
"document_id": str(document_id or ""),
"document_version": (
int(document_version) if document_version is not None else None
),
"document_digest": str(document_digest or "").strip().lower(),
"external_untrusted_context_seen": bool(external_untrusted_context_seen),
"selected_tools": list(
_normalized_selected_tools(selected_tools, required_tool=tool_name)
),
"continuation_query": _normalized_continuation_query(continuation_query),
"effects": list(effects),
"result_integrity": str(result_integrity),
}
@dataclass(frozen=True)
class PendingToolApproval:
approval_id: str
owner: str
session_id: str
origin_run_id: str
tool_name: str
content: str
workspace: str
document_id: str
document_version: int | None
document_digest: str
external_untrusted_context_seen: bool
effects: tuple[str, ...]
result_integrity: str
digest: str
created_at: float
expires_at: float
# Server-only continuation state. Both fields are digest-bound and never
# exposed in the browser payload.
selected_tools: tuple[str, ...] = ()
continuation_query: str = ""
def public_payload(self, *, reason: str | None = None) -> dict[str, Any]:
return {
"kind": "tool_approval",
"approval_id": self.approval_id,
# The browser already owns this chat id. Persisting it with the
# resolved card lets history-derived session grants remain bound to
# this exact chat and prevents inheritance by a forked session.
"session_id": self.session_id,
"question": "Allow this task to continue?",
"description": reason or (
"Untrusted context influenced this run, so continuing with "
"otherwise-gated actions needs your explicit approval."
),
"options": [
{
"label": "Allow for this task",
"value": TASK_APPROVAL_DECISION,
"description": (
"Execute the sealed action and allow every otherwise-gated "
"action needed to finish this request. Current tool, account, "
"workspace, and sandbox restrictions still apply."
),
},
{
"label": "Allow for this chat session",
"value": CHAT_SESSION_APPROVAL_DECISION,
"description": (
"Execute the sealed action and stop asking at this gate for "
"later requests in this chat. Current tool, account, workspace, "
"and sandbox restrictions still apply."
),
},
{
"label": "Deny",
"value": DENY_APPROVAL_DECISION,
"description": "Do not execute the proposed action.",
},
],
"action": {
"tool": self.tool_name,
# Show the complete sealed input so approval never hides
# trailing lines. This is not read back as authority.
"content": self.content,
"digest": self.digest[:16],
"effects": list(self.effects),
"workspace": self.workspace or None,
"document_id": self.document_id or None,
"document_version": self.document_version,
},
}
@dataclass
class ExactToolApproval:
"""A consumed exact first action plus an explicit continuation scope."""
pending: PendingToolApproval
scope: ToolApprovalScope = ToolApprovalScope.TASK
# The seam consumed by agent_loop. Both chat-card allow choices cover the
# complete resumed task, because one-action scope there immediately
# re-entered the same gate on the next round. Callers with no resumable
# chat still get SINGLE_ACTION, which leaves the gate armed behind the
# sealed action.
allow_remaining_actions: bool = True
_claimed: bool = field(default=False, init=False, repr=False)
_lock: threading.Lock = field(default_factory=threading.Lock, init=False, repr=False)
@property
def grants_chat_session(self) -> bool:
return self.scope is ToolApprovalScope.CHAT_SESSION
def _matches_unlocked(
self,
*,
owner: Any,
session_id: Any,
tool_name: Any,
content: Any,
workspace: Any,
) -> bool:
if self._claimed:
return False
capabilities = capabilities_for_action(tool_name, content)
effects = tuple(sorted(effect.value for effect in capabilities.effects))
result_integrity = capabilities.result_integrity.value
if (
effects != self.pending.effects
or result_integrity != self.pending.result_integrity
):
return False
expected = _binding_payload(
owner=owner,
session_id=session_id,
origin_run_id=self.pending.origin_run_id,
tool_name=tool_name,
content=content,
workspace=workspace,
document_id=self.pending.document_id,
document_version=self.pending.document_version,
document_digest=self.pending.document_digest,
external_untrusted_context_seen=(
self.pending.external_untrusted_context_seen
),
selected_tools=self.pending.selected_tools,
continuation_query=self.pending.continuation_query,
effects=effects,
result_integrity=result_integrity,
)
return _canonical_digest(expected) == self.pending.digest
def matches(
self,
*,
owner: Any,
session_id: Any,
tool_name: Any,
content: Any,
workspace: Any,
) -> bool:
with self._lock:
return self._matches_unlocked(
owner=owner,
session_id=session_id,
tool_name=tool_name,
content=content,
workspace=workspace,
)
def claim(
self,
*,
owner: Any,
session_id: Any,
tool_name: Any,
content: Any,
workspace: Any,
) -> bool:
with self._lock:
if not self._matches_unlocked(
owner=owner,
session_id=session_id,
tool_name=tool_name,
content=content,
workspace=workspace,
):
return False
self._claimed = True
return True
class ToolApprovalStore:
"""Thread-safe pending approval registry with destructive consumption."""
def __init__(
self,
*,
ttl_seconds: int = DEFAULT_APPROVAL_TTL_SECONDS,
max_pending: int = DEFAULT_MAX_PENDING_APPROVALS,
):
self._ttl_seconds = max(1, int(ttl_seconds))
self._max_pending = max(1, int(max_pending))
self._pending: dict[str, PendingToolApproval] = {}
self._lock = threading.Lock()
def _purge_expired_locked(self, now: float) -> None:
expired = [
approval_id
for approval_id, pending in self._pending.items()
if pending.expires_at <= now
]
for approval_id in expired:
self._pending.pop(approval_id, None)
def create(
self,
*,
owner: Any,
session_id: Any,
origin_run_id: Any,
tool_name: Any,
content: Any,
workspace: Any,
document_id: Any = None,
document_version: Any = None,
document_digest: Any = None,
selected_tools: Any = None,
continuation_query: Any = None,
external_untrusted_context_seen: bool,
capabilities: ToolCapabilities,
) -> PendingToolApproval:
now = time.time()
effects = tuple(sorted(effect.value for effect in capabilities.effects))
result_integrity = capabilities.result_integrity.value
payload = _binding_payload(
owner=owner,
session_id=session_id,
origin_run_id=origin_run_id,
tool_name=tool_name,
content=content,
workspace=workspace,
document_id=document_id,
document_version=document_version,
document_digest=document_digest,
external_untrusted_context_seen=external_untrusted_context_seen,
selected_tools=selected_tools,
continuation_query=continuation_query,
effects=effects,
result_integrity=result_integrity,
)
pending = PendingToolApproval(
approval_id=secrets.token_urlsafe(32),
owner=payload["owner"],
session_id=payload["session_id"],
origin_run_id=payload["origin_run_id"],
tool_name=payload["tool_name"],
content=payload["content"],
workspace=payload["workspace"],
document_id=payload["document_id"],
document_version=payload["document_version"],
document_digest=payload["document_digest"],
external_untrusted_context_seen=payload[
"external_untrusted_context_seen"
],
effects=effects,
result_integrity=result_integrity,
digest=_canonical_digest(payload),
created_at=now,
expires_at=now + self._ttl_seconds,
selected_tools=tuple(payload["selected_tools"]),
continuation_query=payload["continuation_query"],
)
with self._lock:
self._purge_expired_locked(now)
# The chat UI exposes one pending card per session, so supersede an
# older action there. Headless/manual-test callers use an empty
# session id; keep independent origin runs separate so two skill
# tests owned by the same user cannot invalidate each other.
superseded = [
approval_id
for approval_id, existing in self._pending.items()
if (
existing.owner == pending.owner
and existing.session_id == pending.session_id
and (
bool(pending.session_id)
or existing.origin_run_id == pending.origin_run_id
)
)
]
for approval_id in superseded:
self._pending.pop(approval_id, None)
while len(self._pending) >= self._max_pending:
oldest_id = min(
self._pending,
key=lambda approval_id: self._pending[approval_id].created_at,
)
self._pending.pop(oldest_id, None)
self._pending[pending.approval_id] = pending
return pending
def consume(
self,
approval_id: Any,
*,
decision: Any,
owner: Any,
session_id: Any,
allow_continuation: bool = True,
) -> ExactToolApproval | None:
"""Consume a pending approval.
``allow_continuation`` is the caller's assertion that it owns a
resumable conversation the granted scope can apply to. Callers without
one (the skill tester, unattended audits) pass ``False`` and get the
original one-use grant, so a button labelled "Allow once" cannot widen
into a run-long bypass just because the chat card reuses the same wire
value.
"""
now = time.time()
with self._lock:
self._purge_expired_locked(now)
approval_key = str(approval_id or "")
pending = self._pending.get(approval_key)
if pending is None:
return None
if (
pending.owner != _normalized_owner(owner)
or pending.session_id != str(session_id or "")
):
# Authentication is checked before destructive consumption so
# a leaked/guessed opaque id cannot be used to invalidate
# another owner's pending action.
return None
self._pending.pop(approval_key, None)
normalized_decision = str(decision or "").strip().lower()
scope = scope_for_decision(normalized_decision)
if scope is None:
return None
if not allow_continuation:
return ExactToolApproval(
pending,
scope=ToolApprovalScope.SINGLE_ACTION,
allow_remaining_actions=False,
)
return ExactToolApproval(
pending,
scope=scope,
allow_remaining_actions=True,
)
def peek(self, approval_id: Any) -> PendingToolApproval | None:
now = time.time()
with self._lock:
self._purge_expired_locked(now)
return self._pending.get(str(approval_id or ""))
def retire_for_session(self, *, owner: Any, session_id: Any) -> bool:
"""Discard pending actions superseded by an ordinary user turn.
Returns whether any retired action carried external provenance, so the
caller can preserve that security state without treating the new user
message as an approval continuation.
"""
now = time.time()
normalized_owner = _normalized_owner(owner)
normalized_session = str(session_id or "")
if not normalized_session:
return False
with self._lock:
self._purge_expired_locked(now)
retired_ids = [
approval_id
for approval_id, pending in self._pending.items()
if (
pending.owner == normalized_owner
and pending.session_id == normalized_session
)
]
carried_taint = any(
self._pending[approval_id].external_untrusted_context_seen
for approval_id in retired_ids
)
for approval_id in retired_ids:
self._pending.pop(approval_id, None)
return carried_taint
tool_approval_store = ToolApprovalStore()
-686
View File
@@ -1,686 +0,0 @@
"""Deterministic capability metadata for agent tools.
Model output requests an action; it never supplies the authority for that
action. This module classifies the effects of each built-in tool and applies
run-local integrity gates before dispatch.
"""
from __future__ import annotations
import json
import uuid
from dataclasses import dataclass, field
from enum import Enum
from types import MappingProxyType
from typing import Any, Iterable, Mapping
from src.tool_approval_scopes import CHAT_SESSION_APPROVAL_CONTEXT_MARKER
from src.tool_security import BUILTIN_EMAIL_TOOLS
class ToolEffect(str, Enum):
READ_PUBLIC = "read_public"
READ_WORKSPACE = "read_workspace"
READ_PRIVATE = "read_private"
WRITE_WORKSPACE = "write_workspace"
WRITE_PRIVATE = "write_private"
EXECUTE_CODE = "execute_code"
BROKERED_NETWORK_READ = "brokered_network_read"
NETWORK_EGRESS = "network_egress"
EXTERNAL_SIDE_EFFECT = "external_side_effect"
UI_SIDE_EFFECT = "ui_side_effect"
ADMIN_CHANGE = "admin_change"
DESTRUCTIVE = "destructive"
USER_INTERACTION = "user_interaction"
class ResultIntegrity(str, Enum):
SYSTEM = "system"
WORKSPACE_UNTRUSTED = "workspace_untrusted"
EXTERNAL_UNTRUSTED = "external_untrusted"
@dataclass(frozen=True)
class ToolCapabilities:
effects: frozenset[ToolEffect]
result_integrity: ResultIntegrity = ResultIntegrity.SYSTEM
known: bool = True
def _capabilities(
*effects: ToolEffect,
result_integrity: ResultIntegrity = ResultIntegrity.SYSTEM,
) -> ToolCapabilities:
return ToolCapabilities(frozenset(effects), result_integrity)
_REGISTRY: dict[str, ToolCapabilities] = {}
def _register(
names: Iterable[str],
*effects: ToolEffect,
result_integrity: ResultIntegrity = ResultIntegrity.SYSTEM,
) -> None:
capabilities = _capabilities(*effects, result_integrity=result_integrity)
for name in names:
if name in _REGISTRY:
raise RuntimeError(f"Duplicate tool capability classification: {name}")
_REGISTRY[name] = capabilities
_register(
{"ask_user", "update_plan"},
ToolEffect.USER_INTERACTION,
)
_register(
{
"list_cached_models",
"list_cookbook_servers",
"list_downloads",
"list_models",
"list_serve_presets",
"list_served_models",
},
ToolEffect.READ_PRIVATE,
# These readers return provider-controlled model identifiers or durable
# user/admin-authored Cookbook and process state. Local brokering does not
# make the returned text server-authored.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"search_hf_models"},
ToolEffect.BROKERED_NETWORK_READ,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"get_workspace", "glob", "grep", "ls", "read_file"},
ToolEffect.READ_WORKSPACE,
result_integrity=ResultIntegrity.WORKSPACE_UNTRUSTED,
)
_register(
{"web_search"},
ToolEffect.BROKERED_NETWORK_READ,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"web_fetch"},
ToolEffect.BROKERED_NETWORK_READ,
ToolEffect.NETWORK_EGRESS,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{
"list_email_accounts",
"list_emails",
"read_email",
"resolve_contact",
"scan_email_unsubscribes",
"search_chats",
"search_emails",
"list_sessions",
"tail_serve_output",
"vault_get",
"vault_search",
},
ToolEffect.READ_PRIVATE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"bash", "manage_bg_jobs", "python"},
ToolEffect.EXECUTE_CODE,
result_integrity=ResultIntegrity.WORKSPACE_UNTRUSTED,
)
_register(
{"apply_patch", "edit_file", "write_file"},
ToolEffect.WRITE_WORKSPACE,
# Successful writes include unified diffs that can echo arbitrary existing
# workspace content back into the next model round.
result_integrity=ResultIntegrity.WORKSPACE_UNTRUSTED,
)
_register(
{
"create_document",
"manage_calendar",
"manage_contact",
"manage_documents",
"manage_memory",
"manage_notes",
"manage_research",
"manage_session",
"manage_skills",
"manage_tasks",
"suggest_document",
"todowrite",
},
ToolEffect.WRITE_PRIVATE,
)
_register(
{
"ai_draft_email_reply",
"create_session",
"draft_email",
"draft_email_reply",
},
ToolEffect.WRITE_PRIVATE,
# These tools resolve user-configured endpoints/accounts or read stored
# email content before returning model-visible status text.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"edit_document", "update_document"},
ToolEffect.WRITE_PRIVATE,
# These tools can echo stored document content that was not present in
# their arguments. edit_document returns the complete edited document;
# update_document also preserves stored email headers/thread history.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"pipeline"},
ToolEffect.NETWORK_EGRESS,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"send_to_session"},
ToolEffect.NETWORK_EGRESS,
ToolEffect.WRITE_PRIVATE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"chat_with_model", "ask_teacher"},
ToolEffect.NETWORK_EGRESS,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"download_attachment"},
ToolEffect.READ_PRIVATE,
ToolEffect.WRITE_WORKSPACE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"edit_image", "generate_image", "trigger_research"},
ToolEffect.NETWORK_EGRESS,
ToolEffect.WRITE_PRIVATE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{
"archive_email",
"bulk_email",
"mark_email_read",
"reply_to_email",
"send_email",
"unsubscribe_email",
},
ToolEffect.EXTERNAL_SIDE_EFFECT,
# Email action results can include stored headers/account labels or remote
# SMTP/IMAP responses, even when the action itself succeeded.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"delete_email"},
ToolEffect.EXTERNAL_SIDE_EFFECT,
ToolEffect.DESTRUCTIVE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{"ui_control"},
ToolEffect.UI_SIDE_EFFECT,
# Model switches and custom-theme validation read mutable user settings.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{
"adopt_served_model",
"cancel_download",
"download_model",
"serve_model",
"serve_preset",
"stop_served_model",
"vault_unlock",
},
ToolEffect.ADMIN_CHANGE,
# Cookbook/process operations can return stored presets, provider data,
# remote shell output, and command errors.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_register(
{
"api_call",
"app_api",
"manage_endpoints",
"manage_mcp",
"manage_settings",
"manage_tokens",
"manage_webhooks",
},
ToolEffect.ADMIN_CHANGE,
# api_call/app_api return remote or stored application data, and the
# admin managers can echo user-controlled configuration. Conservatively
# retain the action effect while treating every successful result as data.
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
TOOL_CAPABILITIES: Mapping[str, ToolCapabilities] = MappingProxyType(dict(_REGISTRY))
KNOWN_CAPABILITY_TOOLS = frozenset(TOOL_CAPABILITIES)
_UNKNOWN_CAPABILITIES = _capabilities(
ToolEffect.READ_PRIVATE,
ToolEffect.WRITE_WORKSPACE,
ToolEffect.WRITE_PRIVATE,
ToolEffect.EXECUTE_CODE,
ToolEffect.NETWORK_EGRESS,
ToolEffect.EXTERNAL_SIDE_EFFECT,
ToolEffect.ADMIN_CHANGE,
ToolEffect.DESTRUCTIVE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_UNKNOWN_CAPABILITIES = ToolCapabilities(
_UNKNOWN_CAPABILITIES.effects,
_UNKNOWN_CAPABILITIES.result_integrity,
known=False,
)
_BROWSER_MCP_READ_CAPABILITIES = _capabilities(
ToolEffect.BROKERED_NETWORK_READ,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
_BROWSER_MCP_READ_TOOLS = frozenset(
{
"mcp__builtin_browser__browser_console_messages",
"mcp__builtin_browser__browser_network_requests",
"mcp__builtin_browser__browser_snapshot",
"mcp__builtin_browser__browser_take_screenshot",
}
)
def capabilities_for_tool(tool_name: Any) -> ToolCapabilities:
"""Return deterministic capabilities; malformed and unknown tools fail high."""
if not isinstance(tool_name, str) or not tool_name:
return _UNKNOWN_CAPABILITIES
capabilities = TOOL_CAPABILITIES.get(tool_name)
if capabilities is not None:
return capabilities
if tool_name.startswith("mcp__email__"):
bare_name = tool_name[len("mcp__email__"):]
capabilities = TOOL_CAPABILITIES.get(bare_name)
if bare_name in BUILTIN_EMAIL_TOOLS and capabilities is not None:
return capabilities
if tool_name in _BROWSER_MCP_READ_TOOLS:
return _BROWSER_MCP_READ_CAPABILITIES
return _UNKNOWN_CAPABILITIES
_PRIVATE_ACTION_READS: Mapping[str, frozenset[str]] = MappingProxyType(
{
"manage_calendar": frozenset({"list_calendars", "list_events"}),
"manage_contact": frozenset({"list"}),
"manage_documents": frozenset({"list", "read", "view", "open", "get"}),
"manage_memory": frozenset({"list", "search"}),
"manage_notes": frozenset({"list", "search", "find", "view"}),
"manage_research": frozenset({"list", "read", "open", "view", "get"}),
"manage_session": frozenset({"list", "switch", "open", "select", "view"}),
"manage_skills": frozenset({"list", "index", "view", "view_ref", "search"}),
"manage_tasks": frozenset({"list"}),
}
)
_PRIVATE_ACTION_WRITES: Mapping[str, frozenset[str]] = MappingProxyType(
{
"manage_calendar": frozenset(
{"create_event", "update_event", "delete_event"}
),
"manage_contact": frozenset({"add", "update", "edit", "delete"}),
"manage_documents": frozenset({"delete", "tidy"}),
"manage_memory": frozenset({"add", "edit", "delete"}),
"manage_notes": frozenset({"add", "update", "delete", "toggle_item"}),
"manage_research": frozenset({"delete"}),
"manage_session": frozenset(
{
"rename",
"archive",
"unarchive",
"delete",
"important",
"unimportant",
"truncate",
"fork",
}
),
"manage_skills": frozenset({"add", "edit", "patch", "publish", "delete"}),
"manage_tasks": frozenset({"create", "edit", "delete", "pause", "resume", "run"}),
}
)
_ACTION_DESTRUCTIVE: Mapping[str, frozenset[str]] = MappingProxyType(
{
"manage_calendar": frozenset({"delete_event"}),
"manage_contact": frozenset({"delete"}),
"manage_documents": frozenset({"delete", "tidy"}),
"manage_endpoints": frozenset({"delete"}),
"manage_bg_jobs": frozenset({"kill", "stop", "cancel", "terminate"}),
"manage_memory": frozenset({"delete"}),
"manage_mcp": frozenset({"delete"}),
"manage_notes": frozenset({"delete"}),
"manage_research": frozenset({"delete"}),
"manage_session": frozenset({"delete", "truncate"}),
"manage_settings": frozenset({"delete", "reset"}),
"manage_skills": frozenset({"delete"}),
"manage_tasks": frozenset({"delete"}),
"manage_tokens": frozenset({"delete"}),
"manage_webhooks": frozenset({"delete"}),
}
)
_ACTION_DEFAULTS: Mapping[str, str] = MappingProxyType(
{
"manage_calendar": "list_events",
"manage_documents": "list",
"manage_research": "list",
"manage_tasks": "list",
}
)
_ACTION_ALIASES: Mapping[str, Mapping[str, str]] = MappingProxyType(
{
"manage_calendar": MappingProxyType(
{
"create": "create_event",
"update": "update_event",
"delete": "delete_event",
"list": "list_events",
}
),
"manage_notes": MappingProxyType(
{
"create": "add",
"new": "add",
"save": "add",
"remind": "add",
"reminder": "add",
"remove": "delete",
"remove_item": "toggle_item",
}
),
}
)
_LINE_ACTION_TOOLS = frozenset({"manage_memory", "manage_session"})
def _action_from_content(tool_name: str, content: Any) -> str | None:
"""Extract the action discriminator using the same accepted input shapes."""
if isinstance(content, Mapping):
payload: Any = dict(content)
elif isinstance(content, str):
raw = content.strip()
if tool_name in _LINE_ACTION_TOOLS and raw and not raw.startswith("{"):
return raw.splitlines()[0].strip().replace("-", "_").casefold() or None
try:
payload = json.loads(raw) if raw else {}
except (TypeError, ValueError):
return None
else:
payload = {}
if not isinstance(payload, dict):
return None
if (
len(payload) == 1
and isinstance(payload.get("body"), dict)
and "action" in payload["body"]
):
payload = payload["body"]
action = payload.get("action")
if (
not action
and tool_name == "manage_calendar"
and isinstance(payload.get("events"), list)
):
action = "create_event"
if not action and tool_name == "manage_tasks" and any(
payload.get(key) is not None
for key in ("task", "description", "schedule", "time", "day_of_week")
):
action = "create"
if not isinstance(action, str) or not action.strip():
action = _ACTION_DEFAULTS.get(tool_name)
if not action:
return None
normalized = action.strip().replace("-", "_").casefold()
return _ACTION_ALIASES.get(tool_name, {}).get(normalized, normalized)
def capabilities_for_action(tool_name: Any, content: Any) -> ToolCapabilities:
"""Classify a sealed multiplexed action; ambiguous actions fail high."""
base = capabilities_for_tool(tool_name)
if not isinstance(tool_name, str):
return base
action = _action_from_content(tool_name, content)
destructive = action in _ACTION_DESTRUCTIVE.get(tool_name, ())
if tool_name not in _PRIVATE_ACTION_READS:
if not destructive:
return base
return ToolCapabilities(
frozenset(set(base.effects) | {ToolEffect.DESTRUCTIVE}),
base.result_integrity,
known=base.known,
)
if action in _PRIVATE_ACTION_READS[tool_name]:
return _capabilities(
ToolEffect.READ_PRIVATE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
if action in _PRIVATE_ACTION_WRITES[tool_name]:
effects = set(base.effects)
if destructive:
effects.add(ToolEffect.DESTRUCTIVE)
return ToolCapabilities(
frozenset(effects),
ResultIntegrity.EXTERNAL_UNTRUSTED,
known=base.known,
)
return _capabilities(
ToolEffect.READ_PRIVATE,
ToolEffect.WRITE_PRIVATE,
result_integrity=ResultIntegrity.EXTERNAL_UNTRUSTED,
)
def tool_result_is_successful(result: Any) -> bool:
"""Return whether a result actually introduced successful tool output."""
return bool(
isinstance(result, dict)
and not result.get("blocked")
and not result.get("approval_required")
and not result.get("error")
and result.get("exit_code") in (None, 0)
and result.get("success") is not False
)
def tool_result_should_arm_gate(
tool_name: Any,
result: Any,
content: Any = None,
) -> bool:
"""Return whether a result introduced non-system content to the model.
A blocked/approval placeholder and a genuinely content-free failure do not
change authority. Once a non-system tool returns text or structured data
that will be folded into model context, however, failure status cannot make
that payload trusted: MCP ``isError`` text, provider exception messages,
and HTTP error bodies are all attacker-controlled input surfaces.
"""
if not isinstance(result, dict):
return False
if result.get("blocked") or result.get("approval_required"):
return False
# A producer that knows a particular response body came from a remote or
# stored source overrides a coarse static SYSTEM default.
if result.get("untrusted_content") is True:
return True
capabilities = capabilities_for_action(tool_name, content)
if capabilities.result_integrity is ResultIntegrity.SYSTEM:
return False
if tool_result_is_successful(result):
return True
# ``format_tool_result`` serializes every additional structured field, so
# a fixed allowlist here would inevitably miss model-visible payloads such
# as ``details``, ``events``, or provider-specific response keys. Exclude
# only status/policy controls that carry no producer content; any other
# non-empty field crosses the same integrity boundary even on failure.
non_content_keys = frozenset(
{
"approval_required",
"blocked",
"exit_code",
"policy",
"success",
"untrusted_content",
}
)
return any(
key not in non_content_keys and value not in (None, "", [], {}, ())
for key, value in result.items()
)
POST_EXTERNAL_BLOCKED_EFFECTS = frozenset(
{
ToolEffect.READ_PRIVATE,
ToolEffect.WRITE_WORKSPACE,
ToolEffect.WRITE_PRIVATE,
ToolEffect.EXECUTE_CODE,
ToolEffect.NETWORK_EGRESS,
ToolEffect.EXTERNAL_SIDE_EFFECT,
ToolEffect.UI_SIDE_EFFECT,
ToolEffect.ADMIN_CHANGE,
ToolEffect.DESTRUCTIVE,
}
)
@dataclass(frozen=True)
class ToolGateDecision:
allowed: bool
reason: str | None = None
_EXTERNAL_MESSAGE_SOURCES = frozenset(
{
"injected research context",
"prefetched search context",
"research context",
"web search results",
"youtube transcript",
}
)
_EXTERNAL_MESSAGE_SOURCE_PREFIXES = ("web page:",)
def messages_contain_external_untrusted_context(messages: Iterable[dict]) -> bool:
"""Detect explicitly labelled external context already present in a run."""
for message in messages or ():
if not isinstance(message, dict):
continue
metadata = message.get("metadata")
if not isinstance(metadata, dict) or metadata.get("trusted") is not False:
continue
gate_marker = metadata.get("tool_gate_untrusted")
if gate_marker is True:
return True
if gate_marker is False:
# Explicit current-format opt-outs are authoritative. The source
# label heuristics below exist only for older saved wrappers that
# predate the marker.
continue
if metadata.get("provenance_origin") == "external":
return True
source = metadata.get("source")
if not isinstance(source, str):
continue
normalized_source = source.strip().casefold()
if normalized_source in _EXTERNAL_MESSAGE_SOURCES:
return True
if normalized_source.startswith(_EXTERNAL_MESSAGE_SOURCE_PREFIXES):
return True
return False
@dataclass
class ToolRunSecurityContext:
"""Server-owned integrity state for one agent run."""
external_untrusted_context_seen: bool = False
external_sources: list[str] = field(default_factory=list)
run_id: str = field(default_factory=lambda: uuid.uuid4().hex)
# Task-scope approval sets this for the resumed in-memory run. Chat-scope
# approval is projected from the server-owned session history marker below.
# The bypass affects only this automatic gate; current tool policy, ownership,
# workspace confinement, and execution/sandbox restrictions still apply.
approval_gate_bypassed: bool = False
def observe_messages(self, messages: Iterable[dict]) -> None:
"""Apply server-owned chat scope and promote untrusted prompt context."""
message_list = list(messages or ())
if any(
isinstance(message, dict)
and isinstance(message.get("metadata"), dict)
and message["metadata"].get(
CHAT_SESSION_APPROVAL_CONTEXT_MARKER
) is True
for message in message_list
):
self.approval_gate_bypassed = True
if messages_contain_external_untrusted_context(message_list):
self.external_untrusted_context_seen = True
def decision_for(self, tool_name: Any, content: Any = None) -> ToolGateDecision:
if self.approval_gate_bypassed:
return ToolGateDecision(True)
if not self.external_untrusted_context_seen:
return ToolGateDecision(True)
capabilities = capabilities_for_action(tool_name, content)
blocked_effects = capabilities.effects & POST_EXTERNAL_BLOCKED_EFFECTS
if capabilities.known and not blocked_effects:
return ToolGateDecision(True)
effects = ", ".join(sorted(effect.value for effect in blocked_effects))
if not capabilities.known:
effects = "unknown/high-impact"
return ToolGateDecision(
False,
(
"External untrusted context has already influenced this run. "
f"Tool '{tool_name}' requires a separate user-authorized action "
f"because it can cause {effects}."
),
)
def observe_tool_result(
self,
tool_name: Any,
result: Any,
content: Any = None,
) -> None:
if not tool_result_should_arm_gate(tool_name, result, content):
return
self.external_untrusted_context_seen = True
if isinstance(tool_name, str) and tool_name not in self.external_sources:
self.external_sources.append(tool_name)
def blocked_tool_result(tool_name: Any, reason: str) -> tuple[str, dict]:
return (
f"{tool_name}: BLOCKED",
{
"error": reason,
"exit_code": 1,
"blocked": True,
"policy": "external_untrusted_context",
},
)
+2 -161
View File
@@ -27,24 +27,10 @@ from src.tool_security import (
is_public_blocked_tool, is_public_blocked_tool,
owner_is_admin_or_single_user, owner_is_admin_or_single_user,
) )
from src.tool_capabilities import ToolRunSecurityContext, blocked_tool_result
from src.tool_approvals import ExactToolApproval
from src.tool_policy import ToolPolicy from src.tool_policy import ToolPolicy
from src.constants import MAX_OUTPUT_CHARS, MAX_READ_CHARS, MAX_DIFF_LINES, DATA_DIR from src.constants import MAX_OUTPUT_CHARS, MAX_READ_CHARS, MAX_DIFF_LINES, DATA_DIR
from src.tool_utils import _truncate, get_mcp_manager from src.tool_utils import _truncate, get_mcp_manager
class _MissingToolSecurityContext:
pass
class _NoToolSecurityContext:
"""Explicit sentinel for non-agent callers that have no run provenance."""
_MISSING_TOOL_SECURITY_CONTEXT = _MissingToolSecurityContext()
NO_TOOL_SECURITY_CONTEXT = _NoToolSecurityContext()
# Persistent working directory for agent subprocesses. # Persistent working directory for agent subprocesses.
# Resolves to <repo_root>/data, which is the bind-mounted volume in Docker # Resolves to <repo_root>/data, which is the bind-mounted volume in Docker
# (/app/data) and the local data directory for manual installs. # (/app/data) and the local data directory for manual installs.
@@ -568,19 +554,10 @@ async def _document_tool_dispatch(
content: str, content: str,
session_id: Optional[str] = None, session_id: Optional[str] = None,
owner: Optional[str] = None, owner: Optional[str] = None,
document_id: Optional[str] = None,
document_version: Optional[int] = None,
document_digest: Optional[str] = None,
) -> Optional[Dict]: ) -> Optional[Dict]:
"""Route a document tool through TOOL_HANDLERS with the right ctx shape.""" """Route a document tool through TOOL_HANDLERS with the right ctx shape."""
from src.agent_tools import TOOL_HANDLERS from src.agent_tools import TOOL_HANDLERS
ctx = { ctx = {"session_id": session_id, "owner": owner}
"session_id": session_id,
"owner": owner,
"doc_id": document_id,
"expected_document_version": document_version,
"expected_document_digest": document_digest,
}
if tool in TOOL_HANDLERS: if tool in TOOL_HANDLERS:
return await TOOL_HANDLERS[tool](content, ctx) return await TOOL_HANDLERS[tool](content, ctx)
return None return None
@@ -598,12 +575,6 @@ async def execute_tool_block(
progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None, progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None,
workspace: Optional[str] = None, workspace: Optional[str] = None,
tool_policy: Optional[Any] = None, tool_policy: Optional[Any] = None,
security_context: (
ToolRunSecurityContext
| _NoToolSecurityContext
| _MissingToolSecurityContext
) = _MISSING_TOOL_SECURITY_CONTEXT,
exact_approval: Optional[ExactToolApproval] = None,
) -> Tuple[str, Dict]: ) -> Tuple[str, Dict]:
"""Execute a single tool block. Returns (description, result_dict). """Execute a single tool block. Returns (description, result_dict).
@@ -611,104 +582,6 @@ async def execute_tool_block(
cwd confine to it) for the duration of this call, then delegate. Reset on the cwd confine to it) for the duration of this call, then delegate. Reset on the
way out so the binding never leaks to the next tool call. way out so the binding never leaks to the next tool call.
""" """
if security_context is _MISSING_TOOL_SECURITY_CONTEXT:
raise TypeError(
"execute_tool_block requires security_context; pass a "
"ToolRunSecurityContext or NO_TOOL_SECURITY_CONTEXT explicitly"
)
if (
not isinstance(security_context, ToolRunSecurityContext)
and security_context is not NO_TOOL_SECURITY_CONTEXT
):
raise TypeError(
"security_context must be a ToolRunSecurityContext or "
"NO_TOOL_SECURITY_CONTEXT"
)
approval_claimed = False
if exact_approval is not None:
if (
not isinstance(security_context, ToolRunSecurityContext)
or not security_context.external_untrusted_context_seen
or not exact_approval.pending.external_untrusted_context_seen
):
return (
f"{getattr(block, 'tool_type', None)}: BLOCKED",
{
"error": "Exact-action approval requires an armed run security context.",
"exit_code": 1,
"blocked": True,
"policy": "exact_tool_approval",
},
)
if (
exact_approval.pending.tool_name
in {"edit_document", "suggest_document", "update_document"}
and (
not exact_approval.pending.document_id
or exact_approval.pending.document_version is None
or not exact_approval.pending.document_digest
)
):
return (
f"{getattr(block, 'tool_type', None)}: BLOCKED",
{
"error": (
"The approved document action has no sealed target and "
"cannot be executed."
),
"exit_code": 1,
"blocked": True,
"policy": "exact_tool_approval",
},
)
sealed_workspace = exact_approval.pending.workspace
if sealed_workspace and vet_workspace(sealed_workspace) != sealed_workspace:
return (
f"{getattr(block, 'tool_type', None)}: BLOCKED",
{
"error": (
"The approved workspace is no longer a valid safe "
"directory. Review the action again."
),
"exit_code": 1,
"blocked": True,
"policy": "exact_tool_approval",
},
)
approval_claimed = exact_approval.claim(
owner=owner,
session_id=session_id,
tool_name=getattr(block, "tool_type", None),
content=getattr(block, "content", None),
workspace=workspace,
)
if not approval_claimed:
return (
f"{getattr(block, 'tool_type', None)}: BLOCKED",
{
"error": "The exact-action approval did not match this tool request.",
"exit_code": 1,
"blocked": True,
"policy": "exact_tool_approval",
},
)
if isinstance(security_context, ToolRunSecurityContext) and not approval_claimed:
decision = security_context.decision_for(
getattr(block, "tool_type", None),
getattr(block, "content", None),
)
if not decision.allowed:
logger.warning(
"External-context policy blocked tool=%r",
getattr(block, "tool_type", None),
)
return blocked_tool_result(
getattr(block, "tool_type", None),
decision.reason or "Tool blocked by external-context policy.",
)
token = _active_workspace.set(workspace or None) token = _active_workspace.set(workspace or None)
try: try:
output = await _execute_tool_block_impl( output = await _execute_tool_block_impl(
@@ -718,28 +591,7 @@ async def execute_tool_block(
owner=owner, owner=owner,
progress_cb=progress_cb, progress_cb=progress_cb,
tool_policy=tool_policy, tool_policy=tool_policy,
approved_document_id=(
exact_approval.pending.document_id
if approval_claimed
else None
),
approved_document_version=(
exact_approval.pending.document_version
if approval_claimed
else None
),
approved_document_digest=(
exact_approval.pending.document_digest
if approval_claimed
else None
),
) )
if isinstance(security_context, ToolRunSecurityContext):
security_context.observe_tool_result(
getattr(block, "tool_type", None),
output[1],
getattr(block, "content", None),
)
return output return output
finally: finally:
_active_workspace.reset(token) _active_workspace.reset(token)
@@ -752,9 +604,6 @@ async def _execute_tool_block_impl(
owner: Optional[str] = None, owner: Optional[str] = None,
progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None, progress_cb: Optional[Callable[[Dict], Awaitable[None]]] = None,
tool_policy: Optional[Any] = None, tool_policy: Optional[Any] = None,
approved_document_id: Optional[str] = None,
approved_document_version: Optional[int] = None,
approved_document_digest: Optional[str] = None,
) -> Tuple[str, Dict]: ) -> Tuple[str, Dict]:
"""Execute a single tool block. Returns (description, result_dict). """Execute a single tool block. Returns (description, result_dict).
@@ -916,15 +765,7 @@ async def _execute_tool_block_impl(
elif tool in ("create_document", "update_document", "edit_document", elif tool in ("create_document", "update_document", "edit_document",
"suggest_document", "manage_documents"): "suggest_document", "manage_documents"):
desc = f"{tool}: {content.split(chr(10))[0][:80]}" desc = f"{tool}: {content.split(chr(10))[0][:80]}"
result = await _document_tool_dispatch( result = await _document_tool_dispatch(tool, content, session_id, owner) \
tool,
content,
session_id,
owner,
document_id=approved_document_id,
document_version=approved_document_version,
document_digest=approved_document_digest,
) \
or {"error": f"{tool}: execution failed", "exit_code": 1} or {"error": f"{tool}: execution failed", "exit_code": 1}
if tool in ("edit_document", "suggest_document") and "title" in (result or {}): if tool in ("edit_document", "suggest_document") and "title" in (result or {}):
desc = f"{tool}: {result.get('title', '')}" desc = f"{tool}: {result.get('title', '')}"
+2 -10
View File
@@ -954,11 +954,7 @@ async def _cookbook_kill_session(session_id: str, *, remote_host: str = "",
resp = await client.post(f"{_INTERNAL_BASE}/api/shell/exec", resp = await client.post(f"{_INTERNAL_BASE}/api/shell/exec",
json={"command": cmd}, headers=headers) json={"command": cmd}, headers=headers)
if resp.status_code >= 400: if resp.status_code >= 400:
return { return {"error": f"shell/exec returned HTTP {resp.status_code}: {resp.text[:200]}", "exit_code": 1}
"error": f"shell/exec returned HTTP {resp.status_code}: {resp.text[:200]}",
"exit_code": 1,
"untrusted_content": True,
}
try: try:
data = resp.json() data = resp.json()
except Exception: except Exception:
@@ -1087,11 +1083,7 @@ async def do_tail_serve_output(content: str, owner: Optional[str] = None) -> Dic
resp = await client.post(f"{_INTERNAL_BASE}/api/shell/exec", resp = await client.post(f"{_INTERNAL_BASE}/api/shell/exec",
json={"command": cmd}, headers=headers) json={"command": cmd}, headers=headers)
if resp.status_code >= 400: if resp.status_code >= 400:
return { return {"error": f"shell/exec returned HTTP {resp.status_code}: {resp.text[:200]}", "exit_code": 1}
"error": f"shell/exec returned HTTP {resp.status_code}: {resp.text[:200]}",
"exit_code": 1,
"untrusted_content": True,
}
data = resp.json() if resp.content else {} data = resp.json() if resp.content else {}
output_text = (data.get("stdout") or "").strip() output_text = (data.get("stdout") or "").strip()
stderr_text = (data.get("stderr") or "").strip() stderr_text = (data.get("stderr") or "").strip()
+1 -5
View File
@@ -123,11 +123,7 @@ async def do_trigger_research(content: str, owner: Optional[str] = None) -> Dict
resp = await client.post(f"{_INTERNAL_BASE}/api/research/start", resp = await client.post(f"{_INTERNAL_BASE}/api/research/start",
json=payload, headers=_internal_headers(owner)) json=payload, headers=_internal_headers(owner))
if resp.status_code >= 400: if resp.status_code >= 400:
return { return {"error": f"research/start returned HTTP {resp.status_code}: {resp.text[:200]}", "exit_code": 1}
"error": f"research/start returned HTTP {resp.status_code}: {resp.text[:200]}",
"exit_code": 1,
"untrusted_content": True,
}
data = resp.json() data = resp.json()
sid = data.get("session_id", "?") sid = data.get("session_id", "?")
return { return {
-1
View File
@@ -725,7 +725,6 @@ async def do_app_api(content: str, owner: Optional[str] = None) -> Dict:
"status_code": resp.status_code, "status_code": resp.status_code,
"body": preview, "body": preview,
"exit_code": 1, "exit_code": 1,
"untrusted_content": True,
} }
return { return {
"output": f"{method} {path} -> {resp.status_code}\n{preview}", "output": f"{method} {path} -> {resp.status_code}\n{preview}",
+20 -31
View File
@@ -8,7 +8,7 @@ from __future__ import annotations
import re import re
from contextvars import ContextVar from contextvars import ContextVar
from datetime import datetime, timedelta, timezone, tzinfo from datetime import datetime, timedelta, timezone
from typing import Dict, Optional from typing import Dict, Optional
@@ -65,31 +65,19 @@ def format_utc_offset(offset_min: Optional[int]) -> str:
return f"{sign}{hours:02d}:{minutes:02d}" return f"{sign}{hours:02d}:{minutes:02d}"
def _zoneinfo_from_name(): def user_timezone() -> timezone:
"""Return ZoneInfo for the request's IANA name, or None if missing/invalid.""" """Return the best known user timezone as a fixed-offset tzinfo."""
name = get_user_tz_name()
if not name:
return None
try:
from zoneinfo import ZoneInfo
return ZoneInfo(name)
except Exception:
return None
def user_timezone() -> tzinfo:
"""Return the best known user timezone.
A valid IANA name wins over x-tz-offset. The offset is a fixed number and
can disagree with the name (wrong sign, stale client); the name carries DST.
"""
zone = _zoneinfo_from_name()
if zone is not None:
return zone
offset = get_user_tz_offset() offset = get_user_tz_offset()
if offset is not None: if offset is None:
return timezone(timedelta(minutes=offset)) name = get_user_tz_name()
return datetime.now().astimezone().tzinfo or timezone.utc if name:
try:
from zoneinfo import ZoneInfo
return ZoneInfo(name)
except Exception:
pass
return datetime.now().astimezone().tzinfo or timezone.utc
return timezone(timedelta(minutes=offset))
def now_user_local(now_utc: Optional[datetime] = None) -> datetime: def now_user_local(now_utc: Optional[datetime] = None) -> datetime:
@@ -112,13 +100,14 @@ def _clock_label(dt: datetime) -> str:
def timezone_label(dt: Optional[datetime] = None) -> str: def timezone_label(dt: Optional[datetime] = None) -> str:
"""Return a concise display label such as Australia/Brisbane, UTC+10:00.""" """Return a concise display label such as Australia/Brisbane, UTC+10:00."""
if dt is None: offset = get_user_tz_offset()
dt = now_user_local() if offset is None:
offset = int((dt.utcoffset() or timedelta()).total_seconds() // 60) if dt is None:
dt = datetime.now().astimezone()
offset = int((dt.utcoffset() or timedelta()).total_seconds() // 60)
offset_label = f"UTC{format_utc_offset(offset)}" offset_label = f"UTC{format_utc_offset(offset)}"
if _zoneinfo_from_name() is not None: name = get_user_tz_name()
return f"{get_user_tz_name()}, {offset_label}" return f"{name}, {offset_label}" if name else offset_label
return offset_label
def current_datetime_prompt(now_utc: Optional[datetime] = None) -> str: def current_datetime_prompt(now_utc: Optional[datetime] = None) -> str:
-4
View File
@@ -34,10 +34,6 @@ fi
# values (APP_PORT / APP_BIND), then built-in defaults. # values (APP_PORT / APP_BIND), then built-in defaults.
PORT="${ODYSSEUS_PORT:-${APP_PORT:-7860}}" # 7860, not 7000 — macOS AirPlay Receiver holds 7000. PORT="${ODYSSEUS_PORT:-${APP_PORT:-7860}}" # 7860, not 7000 — macOS AirPlay Receiver holds 7000.
HOST="${ODYSSEUS_HOST:-${APP_BIND:-127.0.0.1}}" # Set APP_BIND=0.0.0.0 in .env for LAN/Tailscale access. HOST="${ODYSSEUS_HOST:-${APP_BIND:-127.0.0.1}}" # Set APP_BIND=0.0.0.0 in .env for LAN/Tailscale access.
# The port only reaches uvicorn as a flag, so export it too: everything that
# builds a URL for this instance — internal_api_base(), the companion pairing
# code, the MCP OAuth callback — reads APP_PORT and would otherwise assume 7000.
export APP_PORT="$PORT"
PROBE_HOST="$HOST" PROBE_HOST="$HOST"
if [ "$PROBE_HOST" = "0.0.0.0" ] || [ "$PROBE_HOST" = "::" ]; then if [ "$PROBE_HOST" = "0.0.0.0" ] || [ "$PROBE_HOST" = "::" ]; then
PROBE_HOST="127.0.0.1" PROBE_HOST="127.0.0.1"
+13 -12
View File
@@ -10,9 +10,9 @@ import modelsModule from './js/models.js?v=20260715startupcalm2';
import ragModule from './js/rag.js'; import ragModule from './js/rag.js';
import presetsModule from './js/presets.js'; import presetsModule from './js/presets.js';
import searchModule from './js/search.js'; import searchModule from './js/search.js';
import chatModule from './js/chat.js?v=20260819approvalcontrol1'; import chatModule from './js/chat.js?v=20260801fix1';
import compareModule from './js/compare/index.js?v=20260819approvalcontrol1'; import compareModule from './js/compare/index.js?v=20260723compareicon2';
import documentModule from './js/document.js?v=20260815approvalsave1'; import documentModule from './js/document.js?v=20260722emailfastindex1';
import searchChatModule from './js/search-chat.js'; import searchChatModule from './js/search-chat.js';
import { makeWindowDraggable } from './js/windowDrag.js'; import { makeWindowDraggable } from './js/windowDrag.js';
import { import {
@@ -22,7 +22,7 @@ import {
settleSessionHydration settleSessionHydration
} from './js/startupShell.js'; } from './js/startupShell.js';
import markdownModule from './js/markdown.js'; import markdownModule from './js/markdown.js';
import chatRenderer from './js/chatRenderer.js?v=20260819approvalcontrol1'; import chatRenderer from './js/chatRenderer.js?v=20260722emailfastindex1';
import sessionModule from './js/sessions.js'; import sessionModule from './js/sessions.js';
import memoryModule from './js/memory.js?v=20260722memoryloading1'; import memoryModule from './js/memory.js?v=20260722memoryloading1';
import voiceRecorderModule from './js/voiceRecorder.js'; import voiceRecorderModule from './js/voiceRecorder.js';
@@ -33,7 +33,7 @@ import tasksModule from './js/tasks.js?v=20260723tasksbulkfeedback1';
import calendarModule from './js/calendar.js'; import calendarModule from './js/calendar.js';
import notesModule from './js/notes.js'; import notesModule from './js/notes.js';
import adminModule from './js/admin.js?v=20260716openrouter3'; import adminModule from './js/admin.js?v=20260716openrouter3';
import settingsModule from './js/settings.js?v=20260815approvalsave1'; import settingsModule from './js/settings.js?v=20260722emailfastindex1';
// Eagerly bind unified minimize/restore behavior across all tool modals. // Eagerly bind unified minimize/restore behavior across all tool modals.
import './js/modalManager.js?v=20260723compareicon2'; import './js/modalManager.js?v=20260723compareicon2';
// Desktop window tiling — drag a modal near an edge/corner to snap. // Desktop window tiling — drag a modal near an edge/corner to snap.
@@ -50,7 +50,6 @@ import * as researchPanelModule from './js/research/panel.js?v=20260630researcht
import ttsModule from './js/tts-ai.js'; import ttsModule from './js/tts-ai.js';
import spinnerModule from './js/spinner.js'; import spinnerModule from './js/spinner.js';
import { initKeyboardShortcuts } from './js/keyboard-shortcuts.js'; import { initKeyboardShortcuts } from './js/keyboard-shortcuts.js';
import { getSettings } from './js/appConfig.js';
import { initSidebarLayout, syncRailSide } from './js/sidebar-layout.js?v=20260715startupclean'; import { initSidebarLayout, syncRailSide } from './js/sidebar-layout.js?v=20260715startupclean';
import { initSectionCollapse, initSectionDrag } from './js/section-management.js'; import { initSectionCollapse, initSectionDrag } from './js/section-management.js';
@@ -1519,11 +1518,13 @@ function initializeEventListeners() {
}) })
.catch(() => {}); .catch(() => {});
// Hide Gallery when image generation is disabled in settings. // Hide Gallery when image generation is disabled in settings
// getSettings() consumes the login prefetch itself, so every other module const _prefetchedSettings = sessionStorage.getItem('ody-prefetch-settings');
// that asks for settings this load gets the same snapshot without a request. sessionStorage.removeItem('ody-prefetch-settings');
window._initSettingsReady = getSettings() window._initSettingsReady = (_prefetchedSettings
.then(settings => { ? Promise.resolve(JSON.parse(_prefetchedSettings))
: fetch(`${API_BASE}/api/auth/settings`, { credentials: 'same-origin' }).then(r => r.json())
).then(settings => {
// NOTE: image_gen_enabled only governs *generating* images in chat — the // NOTE: image_gen_enabled only governs *generating* images in chat — the
// tool is blocked server-side (chat_routes / agent_loop). The Gallery // tool is blocked server-side (chat_routes / agent_loop). The Gallery
// holds uploads and past images too, so it stays visible regardless; // holds uploads and past images too, so it stays visible regardless;
@@ -3704,7 +3705,7 @@ function startOdysseusApp() {
modelsModule.init(API_BASE); modelsModule.init(API_BASE);
ragModule.init(API_BASE); ragModule.init(API_BASE);
presetsModule.init(API_BASE); presetsModule.init(API_BASE);
searchModule.init(); searchModule.init(API_BASE);
chatModule.init(API_BASE); chatModule.init(API_BASE);
chatModule.initListeners(); chatModule.initListeners();
groupModule.init(API_BASE); groupModule.init(API_BASE);
+33 -64
View File
@@ -231,11 +231,23 @@
} }
} }
</style> </style>
<!-- KaTeX and Mermaid are vendored in /static/lib and pulled in by <!-- KaTeX CSS is loaded with media="print" so it doesn't block render,
static/js/markdown.js the first time a page actually renders math or a then flipped to "all" via JS after load. Mermaid init runs once the
```mermaid fence. They used to load here from cdn.jsdelivr.net on every library finishes loading. Both hooks are wired via addEventListener
page load, which cost ~985 KB on the wire, broke offline installs, and below (inline onload= attrs are blocked by CSP script-src-attr). -->
announced every session to a third party. --> <link id="katex-css" rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.22/dist/katex.min.css" media="print">
<script async src="https://cdn.jsdelivr.net/npm/katex@0.16.22/dist/katex.min.js"></script>
<script id="mermaid-script" async src="https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.min.js"></script>
<script nonce="{{CSP_NONCE}}">
(function(){
var k = document.getElementById('katex-css');
if (k) k.addEventListener('load', function(){ k.media = 'all'; }, { once: true });
var m = document.getElementById('mermaid-script');
if (m) m.addEventListener('load', function(){
if (window.odysseusInitMermaid) window.odysseusInitMermaid();
}, { once: true });
})();
</script>
<!-- Preload the two faces first paint actually uses: Fira Code 400 and 600, <!-- Preload the two faces first paint actually uses: Fira Code 400 and 600,
the app font and the weight the sidebar and header text render at. They the app font and the weight the sidebar and header text render at. They
are declared in style.css, so without a hint they are only discovered are declared in style.css, so without a hint they are only discovered
@@ -246,8 +258,8 @@
<link rel="preload" as="font" type="font/woff2" crossorigin href="/static/fonts/FiraCode-Regular.woff2"> <link rel="preload" as="font" type="font/woff2" crossorigin href="/static/fonts/FiraCode-Regular.woff2">
<link rel="preload" as="font" type="font/woff2" crossorigin href="/static/fonts/FiraCode-SemiBold.woff2"> <link rel="preload" as="font" type="font/woff2" crossorigin href="/static/fonts/FiraCode-SemiBold.woff2">
<link rel="stylesheet" href="/static/style.css?v=20260808startupshell1"> <link rel="stylesheet" href="/static/style.css?v=20260808startupshell1">
<link rel="modulepreload" href="/static/app.js?v=20260815toolapproval4"> <link rel="modulepreload" href="/static/app.js?v=20260808startupshell1">
<link rel="modulepreload" href="/static/js/chat.js?v=20260815toolapproval4"> <link rel="modulepreload" href="/static/js/chat.js?v=20260801fix1">
<link rel="modulepreload" href="/static/js/ui.js"> <link rel="modulepreload" href="/static/js/ui.js">
<link rel="modulepreload" href="/static/js/sessions.js"> <link rel="modulepreload" href="/static/js/sessions.js">
<link rel="modulepreload" href="/static/js/markdown.js"> <link rel="modulepreload" href="/static/js/markdown.js">
@@ -1409,55 +1421,6 @@
</div> </div>
<div class="settings-layout"> <div class="settings-layout">
<div class="settings-sidebar"> <div class="settings-sidebar">
<button
type="button"
class="settings-sidebar-toggle"
id="settings-sidebar-toggle"
aria-label="Collapse settings navigation"
title="Collapse settings navigation"
>
<svg class="settings-sidebar-toggle-collapse" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
<polyline points="15 18 9 12 15 6"></polyline>
</svg>
<svg class="settings-sidebar-toggle-expand" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
<polyline points="9 18 15 12 9 6"></polyline>
</svg>
</button>
<div
class="settings-sidebar-resize-handle"
id="settings-sidebar-resize-handle"
role="separator"
aria-orientation="vertical"
aria-label="Resize settings navigation"
aria-valuemin="150"
aria-valuemax="340"
aria-valuenow="220"
tabindex="0"
></div>
<div class="settings-sidebar-content">
<div class="settings-nav-search-wrap">
<svg class="settings-nav-search-icon" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true">
<circle cx="11" cy="11" r="7"></circle>
<path d="M20 20l-4-4"></path>
</svg>
<input
type="search"
id="settings-nav-search"
class="settings-nav-search"
placeholder="Find settings…"
autocomplete="off"
aria-label="Find settings"
aria-controls="settings-nav-search-results"
/>
<div
id="settings-nav-search-results"
class="settings-nav-search-results hidden"
role="listbox"
aria-label="Settings search results"
></div>
</div>
<!-- Section 1: AI plumbing (Add Models → AI Defaults → Search) --> <!-- Section 1: AI plumbing (Add Models → AI Defaults → Search) -->
<button class="settings-nav-item active" data-settings-tab="services"> <button class="settings-nav-item active" data-settings-tab="services">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round"><path d="M12 5v14"/><path d="M5 12h14"/></svg> <svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round"><path d="M12 5v14"/><path d="M5 12h14"/></svg>
@@ -1518,10 +1481,9 @@
<span>Users</span> <span>Users</span>
</button> </button>
<button class="settings-nav-item admin-only" data-settings-tab="system"> <button class="settings-nav-item admin-only" data-settings-tab="system">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 0 1-2.83-2.83l.06-.06A1.65 1.65 0 0 0 4.68 15a1.65 1.65 0 0 0-1.51-1H3a2 2 0 0 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 0 1 2.83-2.83l.06.06A1.65 1.65 0 0 0 9 4.68a1.65 1.65 0 0 0 1-1.51V3a2 2 0 0 1 4 0v-.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 2.83l-.06.06A1.65 1.65 0 0 0 19.4 9a1.65 1.65 0 0 0 1.51 1H21a2 2 0 0 1 0 4h-.09A1.65 1.65 0 0 0 19.4 15z"/></svg> <svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 0 1-2.83-2.83l.06-.06A1.65 1.65 0 0 0 4.68 15a1.65 1.65 0 0 0-1.51-1H3a2 2 0 0 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 0 1 2.83-2.83l.06.06A1.65 1.65 0 0 0 9 4.68a1.65 1.65 0 0 0 1-1.51V3a2 2 0 0 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 2.83l-.06.06A1.65 1.65 0 0 0 19.4 9a1.65 1.65 0 0 0 1.51 1H21a2 2 0 0 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
<span>System</span> <span>System</span>
</button> </button>
</div>
</div> </div>
<div class="settings-panels"> <div class="settings-panels">
@@ -1542,6 +1504,13 @@
<span class="adm-model-logo" id="set-defaultModelSelect-logo" style="display:inline-flex;align-items:center;justify-content:center;width:18px;height:18px;flex-shrink:0;opacity:0.9;color:var(--fg);"></span> <span class="adm-model-logo" id="set-defaultModelSelect-logo" style="display:inline-flex;align-items:center;justify-content:center;width:18px;height:18px;flex-shrink:0;opacity:0.9;color:var(--fg);"></span>
<select id="set-defaultModelSelect" class="settings-select"></select> <select id="set-defaultModelSelect" class="settings-select"></select>
</div> </div>
<div class="settings-row" style="align-items:flex-start;" hidden>
<label class="settings-label" style="margin-top:6px;">Fallbacks</label>
<div style="flex:1;display:flex;flex-direction:column;gap:6px;">
<div id="set-defaultFallbacks" class="settings-fallbacks"></div>
<button type="button" class="settings-fallback-add" id="set-defaultAddFallback" title="Add a model to try if the one above fails">+ Add fallback</button>
</div>
</div>
<div id="set-defaultChatMsg" style="font-size:11px;color:color-mix(in srgb, var(--fg) 45%, transparent);"></div> <div id="set-defaultChatMsg" style="font-size:11px;color:color-mix(in srgb, var(--fg) 45%, transparent);"></div>
</div> </div>
</div> </div>
@@ -2570,20 +2539,20 @@
<script type="module" src="/static/js/search.js"></script> <script type="module" src="/static/js/search.js"></script>
<script type="module" src="/static/js/spinner.js"></script> <script type="module" src="/static/js/spinner.js"></script>
<script type="module" src="/static/js/tts-ai.js"></script> <script type="module" src="/static/js/tts-ai.js"></script>
<script type="module" src="/static/js/document.js?v=20260815approvalsave1"></script> <script type="module" src="/static/js/document.js?v=20260722emailfastindex1"></script>
<script type="module" src="/static/js/gallery.js?v=20260708match1"></script> <script type="module" src="/static/js/gallery.js?v=20260708match1"></script>
<script type="module" src="/static/js/chatRenderer.js?v=20260819approvalcontrol1"></script> <script type="module" src="/static/js/chatRenderer.js?v=20260722emailfastindex1"></script>
<script type="module" src="/static/js/codeRunner.js"></script> <script type="module" src="/static/js/codeRunner.js"></script>
<script type="module" src="/static/js/chatStream.js?v=20260819approvalcontrol1"></script> <script type="module" src="/static/js/chatStream.js?v=20260722emailfastindex1"></script>
<script type="module" src="/static/js/chat.js?v=20260819approvalcontrol1"></script> <script type="module" src="/static/js/chat.js?v=20260801fix1"></script>
<script type="module" src="/static/js/cookbook.js"></script> <script type="module" src="/static/js/cookbook.js"></script>
<script src="/static/js/cookbookSchedule.js"></script> <script src="/static/js/cookbookSchedule.js"></script>
<script type="module" src="/static/js/search-chat.js"></script> <script type="module" src="/static/js/search-chat.js"></script>
<script type="module" src="/static/js/theme.js"></script> <script type="module" src="/static/js/theme.js"></script>
<script type="module" src="/static/js/censor.js"></script> <script type="module" src="/static/js/censor.js"></script>
<script type="module" src="/static/js/settings.js?v=20260815approvalsave1"></script> <script type="module" src="/static/js/settings.js?v=20260723compareicon1"></script>
<script type="module" src="/static/js/assistant.js"></script> <script type="module" src="/static/js/assistant.js"></script>
<script type="module" src="/static/app.js?v=20260815toolapproval4"></script> <!-- app.js must be LAST --> <script type="module" src="/static/app.js?v=20260808startupshell1"></script> <!-- app.js must be LAST -->
<script type="module" src="/static/js/init.js?v=20260715freshroot3"></script> <script type="module" src="/static/js/init.js?v=20260715freshroot3"></script>
<script type="module" src="/static/js/a11y.js"></script> <script type="module" src="/static/js/a11y.js"></script>
<script nonce="{{CSP_NONCE}}">if('serviceWorker' in navigator){navigator.serviceWorker.register('/static/sw.js').catch(()=>{});}</script> <script nonce="{{CSP_NONCE}}">if('serviceWorker' in navigator){navigator.serviceWorker.register('/static/sw.js').catch(()=>{});}</script>
+17 -65
View File
@@ -6,7 +6,6 @@ import settingsModule from './settings.js';
import { providerLogo, providerLogoFromUrl } from './providers.js'; import { providerLogo, providerLogoFromUrl } from './providers.js';
import { sortModelObjects } from './modelSort.js'; import { sortModelObjects } from './modelSort.js';
import { PROVIDER_DEVICE_FLOWS, formatDeviceFlowError, runProviderDeviceFlow } from './providerDeviceFlow.js'; import { PROVIDER_DEVICE_FLOWS, formatDeviceFlowError, runProviderDeviceFlow } from './providerDeviceFlow.js';
import { getSettings, getTools, invalidateSettings, invalidateTools } from './appConfig.js';
let initialized = false; let initialized = false;
let modalEl = null; let modalEl = null;
@@ -346,7 +345,8 @@ function initSignupToggle() {
function initShareDefaultsToggle() { function initShareDefaultsToggle() {
const toggle = el('adm-shareDefaultsToggle'); const toggle = el('adm-shareDefaultsToggle');
getSettings() fetch('/api/auth/settings', { credentials: 'same-origin' })
.then(r => r.json())
.then(d => { toggle.checked = !!d.share_defaults_with_users; }) .then(d => { toggle.checked = !!d.share_defaults_with_users; })
.catch(e => console.warn('Settings fetch failed:', e)); .catch(e => console.warn('Settings fetch failed:', e));
toggle.addEventListener('change', async () => { toggle.addEventListener('change', async () => {
@@ -361,9 +361,6 @@ function initShareDefaultsToggle() {
toggle.checked = !!data.share_defaults_with_users; toggle.checked = !!data.share_defaults_with_users;
} catch (e) { } catch (e) {
toggle.checked = !toggle.checked; toggle.checked = !toggle.checked;
} finally {
// Drop the shared snapshot: it still says what this toggle used to be.
invalidateSettings();
} }
}); });
} }
@@ -1896,16 +1893,8 @@ async function loadBuiltinTools() {
const list = el('adm-builtin-tools-list'); const list = el('adm-builtin-tools-list');
if (!list) return; if (!list) return;
try { try {
// This panel is an editor, and its save posts the whole disabled list const res = await fetch('/api/tools', { credentials: 'same-origin' });
// rebuilt from the checkboxes below. So it has to render authoritative const data = await res.json();
// state: a snapshot that went stale out of band (the manage_settings tool,
// another tab) would be re-posted wholesale on the next unrelated toggle
// and would silently undo the newer state. refreshAll() calls this on every
// panel open, so drop the shared entry and refill it. The startup read that
// chatRenderer.js shares is unaffected; this panel just never edits a cache,
// which is the same rule the settings panel follows by reading directly.
invalidateTools();
const data = await getTools();
const tools = data.tools || []; const tools = data.tools || [];
if (!tools.length) { list.innerHTML = '<div class="admin-empty">No tools found</div>'; return; } if (!tools.length) { list.innerHTML = '<div class="admin-empty">No tools found</div>'; return; }
@@ -1979,50 +1968,17 @@ async function loadBuiltinTools() {
}); });
}); });
// Merge only the user's intended changes onto authoritative server state. // Helper: save disabled tools + update counters
// /api/tools replaces the full disabled list, so rebuilding it from this async function _saveToolState() {
// panel's DOM can undo a change made by another tab or manage_settings const allChecks = list.querySelectorAll('input[data-tool-id]');
// after the panel was opened. const disabled = [];
async function _saveToolState(changes) { allChecks.forEach(c => { if (!c.checked) disabled.push(c.dataset.toolId); });
invalidateTools(); await fetch('/api/tools', {
const latest = await getTools(); method: 'POST',
const state = new Map( headers: { 'Content-Type': 'application/json' },
(latest.tools || []).map(t => [t.id, !!t.enabled]) body: JSON.stringify({ disabled }),
); credentials: 'same-origin',
});
for (const change of changes) {
if (state.has(change.id)) {
state.set(change.id, !!change.enabled);
}
}
const disabled = Array.from(state.entries())
.filter(([, enabled]) => !enabled)
.map(([id]) => id);
try {
const res = await fetch('/api/tools', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ disabled }),
credentials: 'same-origin',
});
if (!res.ok) throw new Error(`Failed to update tools (${res.status})`);
// Bring the still-open editor forward to the same merged snapshot so an
// out-of-band change is visible instead of leaving stale checkboxes.
list.querySelectorAll('input[data-tool-id]').forEach(c => {
if (state.has(c.dataset.toolId)) {
c.checked = state.get(c.dataset.toolId);
}
});
list.querySelectorAll('.admin-tool-category').forEach(_updateCatCounter);
} finally {
// This route persists disabled_tools into the settings store
// (routes/model_routes.py), so both snapshots are now stale.
invalidateTools();
invalidateSettings();
}
} }
function _updateCatCounter(catEl) { function _updateCatCounter(catEl) {
if (!catEl) return; if (!catEl) return;
@@ -2037,9 +1993,7 @@ async function loadBuiltinTools() {
// Wire individual tool toggles // Wire individual tool toggles
list.querySelectorAll('input[data-tool-id]').forEach(chk => { list.querySelectorAll('input[data-tool-id]').forEach(chk => {
chk.addEventListener('change', async () => { chk.addEventListener('change', async () => {
await _saveToolState([ await _saveToolState();
{ id: chk.dataset.toolId, enabled: chk.checked },
]);
_updateCatCounter(chk.closest('.admin-tool-category')); _updateCatCounter(chk.closest('.admin-tool-category'));
}); });
}); });
@@ -2050,10 +2004,8 @@ async function loadBuiltinTools() {
const catEl = chk.closest('.admin-tool-category'); const catEl = chk.closest('.admin-tool-category');
if (!catEl) return; if (!catEl) return;
const checked = chk.checked; const checked = chk.checked;
const changes = Array.from(catEl.querySelectorAll('input[data-tool-id]'))
.map(c => ({ id: c.dataset.toolId, enabled: checked }));
catEl.querySelectorAll('input[data-tool-id]').forEach(c => { c.checked = checked; }); catEl.querySelectorAll('input[data-tool-id]').forEach(c => { c.checked = checked; });
await _saveToolState(changes); await _saveToolState();
_updateCatCounter(catEl); _updateCatCounter(catEl);
}); });
}); });
-86
View File
@@ -1,86 +0,0 @@
// static/js/appConfig.js
//
// One shared, invalidatable cache for the two config endpoints that every
// module wants at startup.
//
// Before this, /api/auth/settings was fetched independently by six modules and
// /api/tools by three, none of them aware of the others — 4 and 3 requests on a
// single cold load. Worse than the requests: each caller could observe a
// different snapshot of the same object, and chatRenderer.js is imported under
// three different ?v= query strings, so it is three separate module instances
// each issuing its own /api/tools fetch. Caching here fixes both, because the
// cache lives in one module every instance imports by the same specifier.
//
// URLs are bare paths on purpose. The callers that used `${API_BASE}/api/...`
// resolved to the identical URL — API_BASE is `window.location.origin`
// (app.js) — so nothing about the request changes for them.
//
// WRITERS MUST INVALIDATE. Anything that POSTs /api/auth/settings calls
// invalidateSettings(); anything that POSTs /api/tools calls invalidateTools()
// *and* invalidateSettings(), because that route persists `disabled_tools`
// into the same settings store (routes/model_routes.py). Miss one and the UI
// serves a stale settings object for the rest of the session, which is worse
// than the duplicate fetches this replaces.
//
// The resolved object is shared by reference, so treat it as read-only: copy
// before mutating (`{ ...await getSettings() }`).
// Written by login.html immediately before it redirects to '/', so the first
// load after a login can skip the request entirely. Consumed once per page
// load, by whichever module asks for settings first.
const PREFETCH_KEY = 'ody-prefetch-settings';
const _URLS = { settings: '/api/auth/settings', tools: '/api/tools' };
const _cache = { settings: null, tools: null };
function _readPrefetchedSettings() {
try {
const raw = sessionStorage.getItem(PREFETCH_KEY);
if (!raw) return null;
sessionStorage.removeItem(PREFETCH_KEY);
return JSON.parse(raw);
} catch (_) {
return null;
}
}
// A rejected promise must not stay in the slot. Plain `??=` memoisation would
// keep it, so one transient blip during boot would leave keybinds, TTS and the
// search provider on their defaults for the whole session with no retry. Clear
// the slot on failure — unless a later invalidate/refetch already replaced it —
// and rethrow, so every caller's existing .catch() still runs exactly as before.
function _get(key) {
if (_cache[key]) return _cache[key];
const pending = fetch(_URLS[key], { credentials: 'same-origin' })
.then(r => r.json())
.catch(err => {
if (_cache[key] === pending) _cache[key] = null;
throw err;
});
_cache[key] = pending;
return pending;
}
/** GET /api/auth/settings, once per page load (or once per invalidation). */
export function getSettings() {
if (!_cache.settings) {
const prefetched = _readPrefetchedSettings();
if (prefetched) _cache.settings = Promise.resolve(prefetched);
}
return _get('settings');
}
/** GET /api/tools, once per page load (or once per invalidation). */
export function getTools() {
return _get('tools');
}
/** Call after any write that can change settings. */
export function invalidateSettings() {
_cache.settings = null;
}
/** Call after any write that can change the tool enable/disable state. */
export function invalidateTools() {
_cache.tools = null;
}
+202 -650
View File
File diff suppressed because it is too large Load Diff
-104
View File
@@ -1,104 +0,0 @@
/** Select and update the response holder for a route-provenance event. */
export function applyModelRouteEventState(event, holder, roundHolder, defaultModel = '') {
const target = event && event.round && roundHolder ? roundHolder : holder;
if (!target) return null;
target._requestedModel = (
event.requested_model
|| event.selected_model
|| target._requestedModel
|| defaultModel
);
target._actualModel = (
event.model
|| event.answered_by
|| target._actualModel
|| target._requestedModel
);
const hasEndpointRoute = Boolean(
event.requested_endpoint_id
|| event.selected_endpoint_id
|| event.endpoint_id
|| event.answered_by_endpoint_id
|| event.requested_endpoint_label
|| event.selected_endpoint_label
|| event.endpoint_label
|| event.answered_by_endpoint_label
|| target._requestedEndpointLabel
);
if (hasEndpointRoute) {
target._requestedEndpointId = (
event.requested_endpoint_id
|| event.selected_endpoint_id
|| target._requestedEndpointId
|| null
);
target._requestedEndpointLabel = (
event.requested_endpoint_label
|| event.selected_endpoint_label
|| target._requestedEndpointLabel
|| 'Selected route'
);
target._actualEndpointId = (
event.endpoint_id
|| event.answered_by_endpoint_id
|| target._actualEndpointId
|| target._requestedEndpointId
|| null
);
target._actualEndpointLabel = (
event.endpoint_label
|| event.answered_by_endpoint_label
|| target._actualEndpointLabel
|| target._requestedEndpointLabel
);
}
return target;
}
/** Copy the active route into the bubble created for the next Agent round. */
export function inheritModelRouteState(holder, roundHolder, target, defaultModel = '') {
if (!target) return null;
const source = roundHolder || holder;
target._requestedModel = source?._requestedModel || defaultModel;
target._actualModel = source?._actualModel || target._requestedModel;
if (source?._requestedEndpointLabel || source?._actualEndpointLabel) {
target._requestedEndpointId = source?._requestedEndpointId || null;
target._requestedEndpointLabel = source?._requestedEndpointLabel || 'Selected route';
target._actualEndpointId = source?._actualEndpointId || target._requestedEndpointId;
target._actualEndpointLabel = source?._actualEndpointLabel || target._requestedEndpointLabel;
}
return target;
}
/** Apply final/metrics provenance to the active round, not the first bubble. */
export function applyModelMetricsState(metrics, holder, roundHolder, defaultModel = '') {
const target = roundHolder || holder;
if (!target || !metrics) return target || null;
const roundModels = Array.isArray(metrics.round_models) ? metrics.round_models : [];
const roundModel = roundHolder && roundModels.length
? roundModels[roundModels.length - 1]
: null;
target._requestedModel = metrics.requested_model || target._requestedModel || defaultModel;
target._actualModel = roundModel || metrics.model || target._actualModel || target._requestedModel;
const roundEndpointIds = Array.isArray(metrics.round_endpoint_ids) ? metrics.round_endpoint_ids : [];
const roundEndpointLabels = Array.isArray(metrics.round_endpoint_labels) ? metrics.round_endpoint_labels : [];
if (
metrics.requested_endpoint_label
|| metrics.endpoint_label
|| roundEndpointLabels.length
|| target._requestedEndpointLabel
) {
target._requestedEndpointId = metrics.requested_endpoint_id || target._requestedEndpointId || null;
target._requestedEndpointLabel = metrics.requested_endpoint_label || target._requestedEndpointLabel || 'Selected route';
const hasRoundEndpointId = Boolean(roundHolder && roundEndpointIds.length);
const hasRoundEndpointLabel = Boolean(roundHolder && roundEndpointLabels.length);
target._actualEndpointId = hasRoundEndpointId
? roundEndpointIds[roundEndpointIds.length - 1]
: (metrics.endpoint_id || target._actualEndpointId || target._requestedEndpointId);
target._actualEndpointLabel = hasRoundEndpointLabel
? roundEndpointLabels[roundEndpointLabels.length - 1]
: (metrics.endpoint_label || target._actualEndpointLabel || target._requestedEndpointLabel);
}
return target;
}
+61 -376
View File
@@ -9,9 +9,7 @@ import { providerLogo, providerLabel } from './providers.js';
import settingsModule from './settings.js'; import settingsModule from './settings.js';
import spinnerModule from './spinner.js'; import spinnerModule from './spinner.js';
import { bindMenuDismiss } from './escMenuStack.js'; import { bindMenuDismiss } from './escMenuStack.js';
import { loadPanel } from './panels.js';
import { matchModelKey } from './model/matchKey.js'; import { matchModelKey } from './model/matchKey.js';
import { getTools } from './appConfig.js';
const SEARCH_ICON = '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="11" cy="11" r="8"/><path d="M21 21l-4.35-4.35"/></svg>'; const SEARCH_ICON = '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="11" cy="11" r="8"/><path d="M21 21l-4.35-4.35"/></svg>';
const REPORT_ICON = '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/><line x1="10" y1="9" x2="8" y2="9"/></svg>'; const REPORT_ICON = '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/><line x1="10" y1="9" x2="8" y2="9"/></svg>';
@@ -447,12 +445,8 @@ function stripExecutedFence(match, tag, inline, body) {
async function loadExecFenceRegex() { async function loadExecFenceRegex() {
try { try {
// Shared with admin.js, and — more to the point — with the other copies of const res = await fetch('/api/tools', { credentials: 'same-origin' });
// this module: chatRenderer.js is imported under three different ?v= query const data = await res.json();
// strings, so it is instantiated three times per load and used to issue
// three identical /api/tools requests. appConfig.js is imported by one
// specifier from all of them, so they now share a single fetch.
const data = await getTools();
const tags = (data.tools || []) const tags = (data.tools || [])
.map((t) => t.id) .map((t) => t.id)
.filter((id) => id && !EXEC_FENCE_NON_TOOL.has(id)); .filter((id) => id && !EXEC_FENCE_NON_TOOL.has(id));
@@ -621,36 +615,10 @@ export function sameModelName(left, right) {
|| shortModel(a).toLowerCase() === shortModel(b).toLowerCase(); || shortModel(a).toLowerCase() === shortModel(b).toLowerCase();
} }
function shortEndpointLabel(label) { export function modelRouteLabel(requestedModel, actualModel) {
const value = modelValue(label);
if (!value) return '';
return value.length > 18 ? value.slice(0, 17) + '…' : value;
}
export function modelRouteLabel(
requestedModel,
actualModel,
requestedEndpointLabel = '',
actualEndpointLabel = '',
requestedEndpointId = '',
actualEndpointId = '',
) {
const requested = modelValue(requestedModel); const requested = modelValue(requestedModel);
const actual = modelValue(actualModel) || requested; const actual = modelValue(actualModel) || requested;
const requestedRoute = modelValue(requestedEndpointId || requestedEndpointLabel); if (!requested || sameModelName(requested, actual)) return shortModel(actual || requested);
const actualRoute = modelValue(actualEndpointId || actualEndpointLabel);
const routeChanged = Boolean(
actualRoute
&& requestedRoute
&& actualRoute !== requestedRoute
);
if (!requested || sameModelName(requested, actual)) {
const model = shortModel(actual || requested);
if (!routeChanged) return model;
const from = shortEndpointLabel(requestedEndpointLabel || 'Selected route');
const to = shortEndpointLabel(actualEndpointLabel || actualEndpointId);
return model + ' (' + from + ' -> ' + to + ')';
}
return shortModel(requested) + ' -> ' + shortModel(actual); return shortModel(requested) + ' -> ' + shortModel(actual);
} }
@@ -661,24 +629,10 @@ export function replyModelPair(modelName, metadata) {
if (actualFromMeta || requestedFromMeta) { if (actualFromMeta || requestedFromMeta) {
const actual = actualFromMeta || requestedFromMeta || modelValue(modelName); const actual = actualFromMeta || requestedFromMeta || modelValue(modelName);
const requested = requestedFromMeta || actual; const requested = requestedFromMeta || actual;
return { return { requestedModel: requested, actualModel: actual };
requestedModel: requested,
actualModel: actual,
requestedEndpointId: meta.requested_endpoint_id || null,
requestedEndpointLabel: meta.requested_endpoint_label || 'Selected route',
actualEndpointId: meta.endpoint_id || null,
actualEndpointLabel: meta.endpoint_label || meta.requested_endpoint_label || 'Selected route',
};
} }
const fallback = modelValue(modelName); const fallback = modelValue(modelName);
return { return { requestedModel: fallback, actualModel: fallback };
requestedModel: fallback,
actualModel: fallback,
requestedEndpointId: null,
requestedEndpointLabel: 'Selected route',
actualEndpointId: null,
actualEndpointLabel: 'Selected route',
};
} }
/** /**
@@ -870,50 +824,12 @@ export function isCostTrackedEndpoint(url) {
} }
/** Cost for the current turn, returning null for non-billable endpoints. */ /** Cost for the current turn, returning null for non-billable endpoints. */
function _billableCost(model, inputTokens, outputTokens, endpointCostTracked, selectedEndpointUrl) { function _billableCost(model, inputTokens, outputTokens) {
// Foreground fallback can answer on a different endpoint than the session's const url = _currentEndpointUrl();
// selected route. Prefer the backend's non-secret actual-route if (!isCostTrackedEndpoint(url)) return null;
// classification; retain the selected-endpoint check for older history.
if (endpointCostTracked === false) return null;
const selectedUrl = selectedEndpointUrl === undefined
? _currentEndpointUrl()
: selectedEndpointUrl;
if (endpointCostTracked !== true && !isCostTrackedEndpoint(selectedUrl)) {
return null;
}
return getModelCost(model, inputTokens, outputTokens); return getModelCost(model, inputTokens, outputTokens);
} }
/** Sum cost using the route/model that produced each Agent round. */
function _metricsBillableCost(metrics, model, inputTokens, outputTokens, selectedEndpointUrl) {
const buckets = Array.isArray(metrics.usage_buckets) ? metrics.usage_buckets : [];
if (!buckets.length) {
return _billableCost(
model,
inputTokens,
outputTokens,
metrics.endpoint_cost_tracked,
selectedEndpointUrl,
);
}
let total = 0;
let hasPricedUsage = false;
for (const bucket of buckets) {
if (!bucket || typeof bucket !== 'object') continue;
const bucketCost = _billableCost(
bucket.model || model,
Number(bucket.input_tokens) || 0,
Number(bucket.output_tokens) || 0,
bucket.endpoint_cost_tracked,
selectedEndpointUrl,
);
if (bucketCost === null) continue;
total += bucketCost;
hasPricedUsage = true;
}
return hasPricedUsage ? total : null;
}
export function getImageCost(model, quality, size) { export function getImageCost(model, quality, size) {
if (!model) return null; if (!model) return null;
const m = model.toLowerCase(); const m = model.toLowerCase();
@@ -928,9 +844,6 @@ export function getImageCost(model, quality, size) {
/* ── Session cost helpers ─────────────────────────────────────────── */ /* ── Session cost helpers ─────────────────────────────────────────── */
const _COST_KEY = 'ody-session-cost'; const _COST_KEY = 'ody-session-cost';
const _COST_RUNS_KEY = 'ody-session-cost-runs';
const _MAX_COST_RUNS_PER_SESSION = 256;
const _COST_LEDGER_LOCK = 'odysseus-session-cost-ledger';
/** Return the accumulated cost for the current (or given) session. */ /** Return the accumulated cost for the current (or given) session. */
export function getSessionCost(sessionId) { export function getSessionCost(sessionId) {
@@ -938,14 +851,7 @@ export function getSessionCost(sessionId) {
if (!sid) return 0; if (!sid) return 0;
try { try {
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}'); const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
const runCosts = JSON.parse(localStorage.getItem(_COST_RUNS_KEY) || '{}'); return costs[sid] || 0;
const recordedRuns = runCosts[sid] && typeof runCosts[sid] === 'object'
? Object.values(runCosts[sid])
: [];
return (costs[sid] || 0) + recordedRuns.reduce(
(total, value) => total + (Number(value) || 0),
0,
);
} catch (_e) { return 0; } } catch (_e) { return 0; }
} }
@@ -957,9 +863,6 @@ export function resetSessionCost(sessionId) {
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}'); const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
delete costs[sid]; delete costs[sid];
localStorage.setItem(_COST_KEY, JSON.stringify(costs)); localStorage.setItem(_COST_KEY, JSON.stringify(costs));
const runCosts = JSON.parse(localStorage.getItem(_COST_RUNS_KEY) || '{}');
delete runCosts[sid];
localStorage.setItem(_COST_RUNS_KEY, JSON.stringify(runCosts));
} catch (_e) { /* ignore */ } } catch (_e) { /* ignore */ }
updateSessionCostUI(); updateSessionCostUI();
} }
@@ -968,8 +871,21 @@ export function resetSessionCost(sessionId) {
export function updateSessionCostUI() { export function updateSessionCostUI() {
const el = document.getElementById('session-cost-display'); const el = document.getElementById('session-cost-display');
if (!el) return; if (!el) return;
// The ledger records billable work already performed in this session. A // Non-billable endpoint? Hide the badge and clear stale cost that a previous
// selected local endpoint does not erase cost from a paid fallback route. // cloud-rate calculation may have left in localStorage for this session.
const _url = _currentEndpointUrl();
if (!isCostTrackedEndpoint(_url)) {
const sid = window.sessionModule && window.sessionModule.getCurrentSessionId();
if (sid && getSessionCost(sid) > 0) {
try {
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
delete costs[sid];
localStorage.setItem(_COST_KEY, JSON.stringify(costs));
} catch (_e) { /* ignore */ }
}
el.style.display = 'none';
return;
}
const cost = getSessionCost(); const cost = getSessionCost();
if (cost > 0) { if (cost > 0) {
el.textContent = '$' + (cost < 0.01 ? cost.toFixed(4) : cost < 1 ? cost.toFixed(3) : cost.toFixed(2)); el.textContent = '$' + (cost < 0.01 ? cost.toFixed(4) : cost < 1 ? cost.toFixed(3) : cost.toFixed(2));
@@ -979,94 +895,6 @@ export function updateSessionCostUI() {
} }
} }
/** Record one metrics payload in a session ledger at most once. */
export function recordSessionMetricsCost(metrics, sessionId, selectedEndpointUrl) {
if (!metrics || typeof metrics !== 'object') return null;
const cost = _metricsBillableCost(
metrics,
metrics.model || 'Unknown',
metrics.input_tokens || 0,
metrics.output_tokens || 0,
selectedEndpointUrl,
);
if (metrics._fromHistory) return cost;
const sid = sessionId || (
window.sessionModule && window.sessionModule.getCurrentSessionId()
);
if (!sid || cost === null) return cost;
const runId = typeof metrics._costRecordId === 'string'
? metrics._costRecordId.trim()
: '';
if ((metrics._costRecorded || metrics._costRecordPending) && !runId) return cost;
// Recorded is only set once the write actually runs; pending covers the
// window while the write waits on the cross-tab lock, so a replay in that
// window cannot double-add and a tab closed mid-queue never claims recorded.
metrics._costRecordPending = true;
const writeCost = () => {
if (runId) {
try {
const runCosts = JSON.parse(localStorage.getItem(_COST_RUNS_KEY) || '{}');
const sessionRuns = runCosts[sid] && typeof runCosts[sid] === 'object'
? runCosts[sid]
: {};
// Assigning by detached-run identity is replay-idempotent even when a
// refresh produces a fresh metrics object. The Web Lock around this
// read/modify/write also keeps distinct runs from two tabs from
// overwriting one another's stale snapshot.
sessionRuns[runId] = cost;
const entries = Object.entries(sessionRuns);
if (entries.length > _MAX_COST_RUNS_PER_SESSION) {
const overflow = entries.slice(0, entries.length - _MAX_COST_RUNS_PER_SESSION);
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
costs[sid] = (costs[sid] || 0) + overflow.reduce(
(total, entry) => total + (Number(entry[1]) || 0),
0,
);
overflow.forEach(([oldRunId]) => delete sessionRuns[oldRunId]);
localStorage.setItem(_COST_KEY, JSON.stringify(costs));
}
runCosts[sid] = sessionRuns;
localStorage.setItem(_COST_RUNS_KEY, JSON.stringify(runCosts));
} catch (_e) { /* ignore */ }
} else {
try {
const costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
costs[sid] = (costs[sid] || 0) + cost;
localStorage.setItem(_COST_KEY, JSON.stringify(costs));
} catch (_e) { /* ignore */ }
}
metrics._costRecorded = true;
metrics._costRecordPending = false;
const currentSid = window.sessionModule && window.sessionModule.getCurrentSessionId();
if (currentSid === sid) updateSessionCostUI();
};
let writeStarted = false;
const guardedWrite = () => {
writeStarted = true;
writeCost();
};
try {
if (
typeof navigator !== 'undefined'
&& navigator.locks
&& typeof navigator.locks.request === 'function'
) {
const pendingWrite = navigator.locks.request(_COST_LEDGER_LOCK, guardedWrite);
if (pendingWrite && typeof pendingWrite.catch === 'function') {
pendingWrite.catch(() => {
if (!writeStarted) guardedWrite();
});
}
} else {
guardedWrite();
}
} catch (_e) {
if (!writeStarted) guardedWrite();
}
return cost;
}
/** Create a timestamp span for role labels. /** Create a timestamp span for role labels.
* Pass an ISO string / Date / epoch-ms to render the message's own time * Pass an ISO string / Date / epoch-ms to render the message's own time
* (used when replaying history). Falls back to "now" when no value is given. */ * (used when replaying history). Falls back to "now" when no value is given. */
@@ -1373,7 +1201,7 @@ document.addEventListener('click', function(e) {
} catch {} } catch {}
}); });
} else if (kind === 'document') { } else if (kind === 'document') {
import('./document.js?v=20260815approvalsave1').then(mod => { import('./document.js?v=20260722emailfastindex1').then(mod => {
const open = mod.loadDocument const open = mod.loadDocument
|| mod.openDocument || mod.openDocument
|| (mod.default && (mod.default.loadDocument || mod.default.openDocument)); || (mod.default && (mod.default.loadDocument || mod.default.openDocument));
@@ -1395,7 +1223,7 @@ document.addEventListener('click', function(e) {
if (open) open(id); if (open) open(id);
}).catch(() => {}); }).catch(() => {});
} else if (kind === 'email') { } else if (kind === 'email') {
import('./emailLibrary.js?v=20260815approvalsave1').then(mod => { import('./emailLibrary.js?v=20260722emailfastindex1').then(mod => {
const open = mod.openEmailLibrary || (mod.default && mod.default.openEmailLibrary); const open = mod.openEmailLibrary || (mod.default && mod.default.openEmailLibrary);
if (open) open({ uid: id }); if (open) open({ uid: id });
}).catch(() => {}); }).catch(() => {});
@@ -1554,7 +1382,7 @@ export function buildImageBubble(imageUrl, prompt, model, size, quality, imageId
try { try {
const [galleryMod, editorMod] = await Promise.all([ const [galleryMod, editorMod] = await Promise.all([
import('./gallery.js'), import('./gallery.js'),
loadPanel('editor'), import('./galleryEditor.js'),
]); ]);
// Ensure the Gallery modal is open so the editor has a container // Ensure the Gallery modal is open so the editor has a container
// to render into; switch its tabs to the Edit tab. // to render into; switch its tabs to the Edit tab.
@@ -2046,19 +1874,23 @@ export function displayMetrics(messageElement, metrics) {
const isReal = metrics.usage_source === 'real'; const isReal = metrics.usage_source === 'real';
const ctxPct = metrics.context_percent; const ctxPct = metrics.context_percent;
const model = metrics.model || 'Unknown'; const model = metrics.model || 'Unknown';
const cost = _metricsBillableCost( const cost = _billableCost(model, inputTokens, outputTokens);
metrics,
model,
inputTokens,
outputTokens,
);
// Nothing useful to show — bail out (only if ALL metrics are missing) // Nothing useful to show — bail out (only if ALL metrics are missing)
if (!responseTime && !inputTokens && !outputTokens && tps == null && !ctxPct) return; if (!responseTime && !inputTokens && !outputTokens && tps == null && !ctxPct) return;
// Rendering can occur when metrics arrive and again after [DONE]. The // Accumulate session cost (only on fresh metrics, not history reload)
// ledger mutation is idempotent for that shared payload. if (!metrics._fromHistory) {
recordSessionMetricsCost(metrics); const _sid = window.sessionModule && window.sessionModule.getCurrentSessionId();
if (_sid && cost !== null) {
try {
const _costs = JSON.parse(localStorage.getItem(_COST_KEY) || '{}');
_costs[_sid] = (_costs[_sid] || 0) + cost;
localStorage.setItem(_COST_KEY, JSON.stringify(_costs));
} catch (_e) { /* ignore */ }
updateSessionCostUI();
}
}
// Keep token counts in the Message Stats popup; the footer should stay slim. // Keep token counts in the Message Stats popup; the footer should stay slim.
const costStr0 = cost !== null ? `$${cost < 0.01 ? cost.toFixed(4) : cost.toFixed(3)}` : null; const costStr0 = cost !== null ? `$${cost < 0.01 ? cost.toFixed(4) : cost.toFixed(3)}` : null;
@@ -2327,42 +2159,6 @@ export function removeAskUserCards(root) {
scope.querySelectorAll('.ask-user-card').forEach((node) => node.remove()); scope.querySelectorAll('.ask-user-card').forEach((node) => node.remove());
} }
// While a choice card is visible, let plain 13 activate the corresponding
// rendered option. Reuse the option's click path so the question keeps its
// existing submission semantics. Tool approval cards are excluded: that card
// exists to make consent deliberate after untrusted context influenced the
// run, and its first option is the widest grant, so a stray digit must not
// answer it.
function _handleAskUserShortcut(event) {
if (
event.defaultPrevented
|| event.repeat
|| event.isComposing
|| event.ctrlKey
|| event.altKey
|| event.metaKey
|| event.shiftKey
) return;
if (!/^[1-3]$/.test(event.key)) return;
const target = event.target;
if (target?.closest?.('input, textarea, select, [contenteditable="true"]')) return;
const focusedCard = document.activeElement?.closest?.('.ask-user-card') || null;
const mainCard = document.querySelector('#chat-history .ask-user-card');
const compareCards = document.querySelectorAll('.compare-pane .ask-user-card');
const card = focusedCard || mainCard || (compareCards.length === 1 ? compareCards[0] : null);
if (!card) return;
if (card.dataset.askUserKind === 'tool_approval') return;
const option = card.querySelectorAll('.ask-user-option')[Number(event.key) - 1];
if (!option || option.disabled) return;
event.preventDefault();
option.click();
}
document.addEventListener('keydown', _handleAskUserShortcut);
/** /**
* Render an ask_user payload as a durable choice card. * Render an ask_user payload as a durable choice card.
* *
@@ -2372,15 +2168,11 @@ document.addEventListener('keydown', _handleAskUserShortcut);
*/ */
export function renderAskUserCard(payload, options) { export function renderAskUserCard(payload, options) {
const aq = payload || {}; const aq = payload || {};
if (aq.resolved) return null;
const opts = Array.isArray(aq.options) ? aq.options : []; const opts = Array.isArray(aq.options) ? aq.options : [];
const renderOptions = options || {}; const chatBox = document.getElementById('chat-history');
const chatBox = renderOptions.root || document.getElementById('chat-history');
const onSubmit = typeof renderOptions.onSubmit === 'function'
? renderOptions.onSubmit
: null;
if (!chatBox || !aq.question || opts.length < 2) return null; if (!chatBox || !aq.question || opts.length < 2) return null;
const renderOptions = options || {};
removeAskUserCards(chatBox); removeAskUserCards(chatBox);
const card = document.createElement('div'); const card = document.createElement('div');
@@ -2388,8 +2180,6 @@ export function renderAskUserCard(payload, options) {
card.setAttribute('role', 'group'); card.setAttribute('role', 'group');
card.tabIndex = -1; card.tabIndex = -1;
const multi = !!aq.multi; const multi = !!aq.multi;
const isToolApproval = aq.kind === 'tool_approval' && !!aq.approval_id;
card.dataset.askUserKind = isToolApproval ? 'tool_approval' : 'question';
const emojiText = (value) => svgifyEmoji(uiModule.esc(String(value))); const emojiText = (value) => svgifyEmoji(uiModule.esc(String(value)));
const head = document.createElement('div'); const head = document.createElement('div');
@@ -2398,6 +2188,7 @@ export function renderAskUserCard(payload, options) {
closeBtn.type = 'button'; closeBtn.type = 'button';
closeBtn.className = 'modal-close ask-user-close'; closeBtn.className = 'modal-close ask-user-close';
closeBtn.setAttribute('aria-label', 'Dismiss question'); closeBtn.setAttribute('aria-label', 'Dismiss question');
closeBtn.textContent = '×';
closeBtn.addEventListener('click', () => { closeBtn.addEventListener('click', () => {
card.remove(); card.remove();
const input = uiModule.el('message'); const input = uiModule.el('message');
@@ -2413,44 +2204,12 @@ export function renderAskUserCard(payload, options) {
card.appendChild(question); card.appendChild(question);
card.setAttribute('aria-labelledby', question.id); card.setAttribute('aria-labelledby', question.id);
if (isToolApproval && aq.action) {
const action = document.createElement('div');
action.className = 'ask-user-option-desc';
const effects = Array.isArray(aq.action.effects)
? aq.action.effects.join(', ')
: '';
action.textContent = [
aq.action.tool || 'tool',
aq.action.content || '',
effects ? `Effects: ${effects}` : '',
aq.action.workspace ? `Workspace: ${aq.action.workspace}` : '',
aq.action.document_id ? `Document: ${aq.action.document_id}` : '',
aq.action.document_version != null
? `Document version: ${aq.action.document_version}`
: '',
aq.action.digest ? `Approval fingerprint: ${aq.action.digest}` : '',
].filter(Boolean).join('\n');
action.style.whiteSpace = 'pre-wrap';
card.appendChild(action);
}
const list = document.createElement('div'); const list = document.createElement('div');
list.className = 'ask-user-options'; list.className = 'ask-user-options';
card.appendChild(list); card.appendChild(list);
const send = (text) => { const send = (text) => {
if (!text) return; if (!text) return;
if (onSubmit) {
const accepted = onSubmit({
kind: 'answer',
text,
label: text,
payload: aq,
card,
});
if (accepted !== false) card.remove();
return;
}
card.remove(); card.remove();
const input = uiModule.el('message'); const input = uiModule.el('message');
if (input) input.value = text; if (input) input.value = text;
@@ -2482,32 +2241,7 @@ export function renderAskUserCard(payload, options) {
} }
if (!multi) { if (!multi) {
row.type = 'button'; row.type = 'button';
row.addEventListener('click', () => { row.addEventListener('click', () => send(label));
if (isToolApproval) {
const detail = {
approval_id: aq.approval_id,
decision: String((opt && opt.value) || '').toLowerCase(),
label,
document_id: aq.action && aq.action.document_id
? String(aq.action.document_id)
: '',
};
if (onSubmit) {
const accepted = onSubmit({
kind: 'tool_approval',
...detail,
payload: aq,
card,
});
if (accepted !== false) card.remove();
} else {
card.remove();
document.dispatchEvent(new CustomEvent('odysseus:tool-approval', { detail }));
}
} else {
send(label);
}
});
} }
list.appendChild(row); list.appendChild(row);
}); });
@@ -2543,7 +2277,7 @@ export function renderAskUserCard(payload, options) {
}); });
other.appendChild(otherInput); other.appendChild(otherInput);
other.appendChild(otherSend); other.appendChild(otherSend);
if (!isToolApproval) card.appendChild(other); card.appendChild(other);
chatBox.appendChild(card); chatBox.appendChild(card);
if (renderOptions.scroll !== false) { if (renderOptions.scroll !== false) {
@@ -2573,19 +2307,9 @@ export function addMessage(role, content, modelName, metadata) {
const textRaw = Array.isArray(content) ? markdownModule.renderContent(content) : content; const textRaw = Array.isArray(content) ? markdownModule.renderContent(content) : content;
// --- Agent multi-bubble reconstruction from saved metadata --- // --- Agent multi-bubble reconstruction from saved metadata ---
if ( if (role === 'assistant' && metadata && metadata.tool_events && metadata.tool_events.length > 0) {
role === 'assistant'
&& metadata
&& (
(Array.isArray(metadata.tool_events) && metadata.tool_events.length > 0)
|| (Array.isArray(metadata.round_texts) && metadata.round_texts.length > 1)
)
) {
const roundTexts = metadata.round_texts || []; const roundTexts = metadata.round_texts || [];
const roundModels = metadata.round_models || []; const toolEvents = metadata.tool_events;
const roundEndpointIds = metadata.round_endpoint_ids || [];
const roundEndpointLabels = metadata.round_endpoint_labels || [];
const toolEvents = metadata.tool_events || [];
let pendingAskUser = null; let pendingAskUser = null;
let lastWrap = null; let lastWrap = null;
let firstMsgAi = null; let firstMsgAi = null;
@@ -2593,20 +2317,16 @@ export function addMessage(role, content, modelName, metadata) {
const toolsByRound = {}; const toolsByRound = {};
for (const ev of toolEvents) { for (const ev of toolEvents) {
const r = ev.round ?? 1; const r = ev.round || 1;
if (!toolsByRound[r]) toolsByRound[r] = []; if (!toolsByRound[r]) toolsByRound[r] = [];
toolsByRound[r].push(ev); toolsByRound[r].push(ev);
} }
const toolRounds = Object.keys(toolsByRound).map(Number); const maxRound = Math.max(...Object.keys(toolsByRound).map(Number), roundTexts.length);
const maxRound = Math.max(toolRounds.length ? Math.max(...toolRounds) : 0, roundTexts.length);
const firstRound = (toolsByRound[0] || []).length ? 0 : 1; for (let r = 0; r < maxRound; r++) {
for (let roundNum = firstRound; roundNum <= maxRound; roundNum++) { const roundNum = r + 1;
const r = roundNum - 1; const txt = resolveDocumentPlaceholderLinks((roundTexts[r] || '').trim(), metadata);
const txt = r >= 0
? resolveDocumentPlaceholderLinks((roundTexts[r] || '').trim(), metadata)
: '';
if (txt) { if (txt) {
const wrap = document.createElement('div'); const wrap = document.createElement('div');
@@ -2614,31 +2334,10 @@ export function addMessage(role, content, modelName, metadata) {
const roleEl = document.createElement('div'); const roleEl = document.createElement('div');
roleEl.className = 'role'; roleEl.className = 'role';
const pair = replyModelPair(modelName, metadata); const pair = replyModelPair(modelName, metadata);
const contModel = roundModels[r] || pair.actualModel || pair.requestedModel; const contModel = pair.actualModel || pair.requestedModel;
const contEndpointId = r < roundEndpointIds.length roleEl.textContent = modelRouteLabel(pair.requestedModel, contModel);
? roundEndpointIds[r] if (pair.requestedModel && contModel && !sameModelName(pair.requestedModel, contModel)) {
: pair.actualEndpointId; roleEl.title = pair.requestedModel + ' -> ' + contModel;
const contEndpointLabel = r < roundEndpointLabels.length
? roundEndpointLabels[r]
: pair.actualEndpointLabel;
roleEl.textContent = modelRouteLabel(
pair.requestedModel,
contModel,
pair.requestedEndpointLabel,
contEndpointLabel,
pair.requestedEndpointId,
contEndpointId,
);
if (
pair.requestedModel
&& contModel
&& (
!sameModelName(pair.requestedModel, contModel)
|| (pair.requestedEndpointId && contEndpointId && pair.requestedEndpointId !== contEndpointId)
)
) {
roleEl.title = pair.requestedModel + ' -> ' + contModel
+ ' (' + pair.requestedEndpointLabel + ' -> ' + contEndpointLabel + ')';
} }
applyModelColor(roleEl, contModel); applyModelColor(roleEl, contModel);
if (r === 0) roleEl.appendChild(roleTimestamp(metadata?.timestamp)); if (r === 0) roleEl.appendChild(roleTimestamp(metadata?.timestamp));
@@ -2688,7 +2387,7 @@ export function addMessage(role, content, modelName, metadata) {
box.appendChild(threadWrap); box.appendChild(threadWrap);
} }
for (const ev of roundTools) { for (const ev of roundTools) {
if (ev.ask_user && !ev.ask_user.resolved) pendingAskUser = ev.ask_user; if (ev.ask_user) pendingAskUser = ev.ask_user;
const ok = (ev.exit_code === 0 || ev.exit_code == null); const ok = (ev.exit_code === 0 || ev.exit_code == null);
let outHtml = ''; let outHtml = '';
if (ev.output && ev.output.trim()) { if (ev.output && ev.output.trim()) {
@@ -2793,14 +2492,7 @@ export function addMessage(role, content, modelName, metadata) {
const isCompacted = metadata?.compacted; const isCompacted = metadata?.compacted;
const replyModels = replyModelPair(modelName, metadata); const replyModels = replyModelPair(modelName, metadata);
const resolvedModel = replyModels.actualModel || replyModels.requestedModel; const resolvedModel = replyModels.actualModel || replyModels.requestedModel;
var _roleText = role === 'user' ? 'You' : (isSlash || isCompacted) ? 'Odysseus' : modelRouteLabel( var _roleText = role === 'user' ? 'You' : (isSlash || isCompacted) ? 'Odysseus' : modelRouteLabel(replyModels.requestedModel, resolvedModel);
replyModels.requestedModel,
resolvedModel,
replyModels.requestedEndpointLabel,
replyModels.actualEndpointLabel,
replyModels.requestedEndpointId,
replyModels.actualEndpointId,
);
if (role === 'assistant' && (metadata?.research || metadata?.research_clarification)) { if (role === 'assistant' && (metadata?.research || metadata?.research_clarification)) {
_roleText += ' (Research)'; _roleText += ' (Research)';
} }
@@ -2811,14 +2503,8 @@ export function addMessage(role, content, modelName, metadata) {
} }
r.textContent = _roleText; r.textContent = _roleText;
if (role !== 'user') { if (role !== 'user') {
const endpointChanged = Boolean( if (!isSlash && !isCompacted && replyModels.requestedModel && resolvedModel && !sameModelName(replyModels.requestedModel, resolvedModel)) {
replyModels.requestedEndpointId r.title = replyModels.requestedModel + ' -> ' + resolvedModel;
&& replyModels.actualEndpointId
&& replyModels.requestedEndpointId !== replyModels.actualEndpointId
);
if (!isSlash && !isCompacted && replyModels.requestedModel && resolvedModel && (!sameModelName(replyModels.requestedModel, resolvedModel) || endpointChanged)) {
r.title = replyModels.requestedModel + ' -> ' + resolvedModel
+ ' (' + replyModels.requestedEndpointLabel + ' -> ' + replyModels.actualEndpointLabel + ')';
} }
if (!isSlash && !isCompacted) applyModelColor(r, resolvedModel); if (!isSlash && !isCompacted) applyModelColor(r, resolvedModel);
r.appendChild(roleTimestamp(metadata?.timestamp)); r.appendChild(roleTimestamp(metadata?.timestamp));
@@ -3102,7 +2788,6 @@ const chatRenderer = {
getSessionCost, getSessionCost,
resetSessionCost, resetSessionCost,
updateSessionCostUI, updateSessionCostUI,
recordSessionMetricsCost,
roleTimestamp, roleTimestamp,
stripToolBlocks, stripToolBlocks,
copyMessageText, copyMessageText,
+3 -32
View File
@@ -7,36 +7,7 @@ import Storage from './storage.js';
import themeModule from './theme.js'; import themeModule from './theme.js';
import markdownModule from './markdown.js'; import markdownModule from './markdown.js';
import sessionModule from './sessions.js'; import sessionModule from './sessions.js';
import documentModule from './document.js?v=20260815approvalsave1'; import documentModule from './document.js?v=20260722emailfastindex1';
// Tool approvals are control-plane submits for the current chat. chat.js
// deliberately leaves the composer untouched, then programmatically clicks the
// shared send button after it records the sealed approval id/decision. That
// button is polymorphic: with an empty composer it can mean New chat or Record
// voice instead of Send. Intercept only the programmatic approval click and
// route it through the form submit path, which already reaches chat.js directly.
document.addEventListener('odysseus:tool-approval', () => {
const sendButton = document.querySelector('.send-btn');
const chatForm = document.getElementById('chat-form');
if (!sendButton || !chatForm) return;
const interceptApprovalClick = (event) => {
// A real user click must retain the normal send/new-chat/STT behavior.
if (event.isTrusted) return;
sendButton.removeEventListener('click', interceptApprovalClick, true);
event.preventDefault();
event.stopImmediatePropagation();
if (chatForm.requestSubmit) chatForm.requestSubmit();
else chatForm.dispatchEvent(new Event('submit', { bubbles: true, cancelable: true }));
};
sendButton.addEventListener('click', interceptApprovalClick, true);
// Fail-safe cleanup if the approval continuation never reaches its deferred
// synthetic click (for example because the surrounding view is torn down).
setTimeout(() => {
sendButton.removeEventListener('click', interceptApprovalClick, true);
}, 60000);
}, true);
/** /**
* Handle a ui_control SSE event AI-driven UI manipulation. * Handle a ui_control SSE event AI-driven UI manipulation.
@@ -185,7 +156,7 @@ export function handleUIControl(uiData) {
if (fn) fn(); if (fn) fn();
}).catch(function(){}); }).catch(function(){});
} else if (panel === 'email') { } else if (panel === 'email') {
import('./emailLibrary.js?v=20260815approvalsave1').then(function(mod) { import('./emailLibrary.js?v=20260722emailfastindex1').then(function(mod) {
var fn = mod.openEmailLibrary || (mod.default && mod.default.openEmailLibrary); var fn = mod.openEmailLibrary || (mod.default && mod.default.openEmailLibrary);
if (fn) fn(); if (fn) fn();
}).catch(function(){}); }).catch(function(){});
@@ -234,7 +205,7 @@ export function handleUIControl(uiData) {
} catch (e) { } catch (e) {
console.warn('open_email_reply existing draft update failed:', e); console.warn('open_email_reply existing draft update failed:', e);
} }
import('./emailInbox.js?v=20260815approvalsave1').then(function(mod) { import('./emailInbox.js?v=20260722emailfastindex1').then(function(mod) {
var fn = mod.openReplyDraft || (mod.default && mod.default.openReplyDraft); var fn = mod.openReplyDraft || (mod.default && mod.default.openReplyDraft);
if (fn) fn(uiData.uid, uiData.folder || 'INBOX', uiData.mode || 'reply', uiData.body || ''); if (fn) fn(uiData.uid, uiData.folder || 'INBOX', uiData.mode || 'reply', uiData.body || '');
}).catch(function(e) { }).catch(function(e) {
-23
View File
@@ -1,23 +0,0 @@
/** Build a terminal stream error while preserving provider-supplied text. */
export function createTerminalStreamError(payload = {}) {
const rawError = payload.error;
const message = (
payload.text
|| (typeof rawError === 'string' ? rawError : rawError?.message)
|| `Error ${payload.status || 'unknown'}`
);
const error = new Error(message);
error.name = 'TerminalStreamError';
error.terminalStreamError = true;
error.status = payload.status;
return error;
}
/** Only connection-class stream failures are safe to resubmit automatically. */
export function isRecoverableStreamError(error) {
if (!error || error.terminalStreamError || error.name === 'TerminalStreamError') return false;
if (error.name === 'TypeError') return true;
const message = (error.message || '').toLowerCase();
if (/\btool\b|unsupported|json|parse|\b4\d\d\b|\b5\d\d\b/.test(message)) return false;
return /network|fetch|connection|reset|closed|aborted|stream|tim(?:e|ed)\s?out|econn|eof/.test(message);
}
+7 -12
View File
@@ -21,7 +21,7 @@ import { EVAL_PROMPTS, WAVE_FRAMES,
import { fetchModels, _persistSelections, _modelDisplayNames, getExcludedModels, setExcludedModels } from './models.js'; import { fetchModels, _persistSelections, _modelDisplayNames, getExcludedModels, setExcludedModels } from './models.js';
import { showModelSelector, disableToolToggles, restoreToolToggles, _syncToolbarIndicator } from './selector.js?v=20260723compareicon2'; import { showModelSelector, disableToolToggles, restoreToolToggles, _syncToolbarIndicator } from './selector.js?v=20260723compareicon2';
import { _checkUnprobed, _clearProbeWaves } from './probe.js'; import { _checkUnprobed, _clearProbeWaves } from './probe.js';
import { streamToPane, _renderSearchResults, _runSynthForPane, _formatMs, registerStreamActions } from './stream.js?v=20260819approvalcontrol1'; import { streamToPane, _renderSearchResults, _runSynthForPane, _formatMs, registerStreamActions } from './stream.js';
import { import {
stopAll, stopPane, rerollPane, shufflePanePositions, resetCompare, stopAll, stopPane, rerollPane, shufflePanePositions, resetCompare,
_addPane, _removePane, toggleExpandPane, togglePanePreview, copyPaneResponse, _addPane, _removePane, toggleExpandPane, togglePanePreview, copyPaneResponse,
@@ -1006,16 +1006,11 @@ async function _executeCompare(message) {
console.error('Compare error:', err); console.error('Compare error:', err);
if (uiModule) uiModule.showError('Compare failed: ' + err.message); if (uiModule) uiModule.showError('Compare failed: ' + err.message);
} finally { } finally {
// A pane may have started its own ask_user/approval continuation while the state._streaming = false;
// original all-pane Promise was settling. Keep Compare busy until every _setSendBtn('send');
// pane-owned controller is gone instead of exposing a second broadcast send. // Re-enable header buttons
const compareStillStreaming = state._abortControllers.some(Boolean); document.querySelectorAll('#compare-shuffle-btn, #compare-check-btn, #compare-add-btn').forEach(b => {
state._streaming = compareStillStreaming; b.disabled = false; b.style.opacity = '0.7'; b.style.pointerEvents = '';
_setSendBtn(compareStillStreaming ? 'stop' : 'send');
document.querySelectorAll('#compare-shuffle-btn, #compare-check-btn, #compare-add-btn').forEach((button) => {
button.disabled = compareStillStreaming;
button.style.opacity = compareStillStreaming ? '0.25' : '0.7';
button.style.pointerEvents = compareStillStreaming ? 'none' : '';
}); });
} }
} }
@@ -1519,7 +1514,7 @@ async function showShufflePoolEditor() {
// ──────────────────────────────────────────────────────────────────────────── // ────────────────────────────────────────────────────────────────────────────
registerCompareActions({ stopAll, resetCompare }); registerCompareActions({ stopAll, resetCompare });
registerStreamActions({ rerollPane, autoPreviewHtml: _autoPreviewHtml, setSendBtn: _setSendBtn }); registerStreamActions({ rerollPane, autoPreviewHtml: _autoPreviewHtml });
registerPaneActions({ setSendBtn: _setSendBtn, deactivate, streamToPane, renderSearchResults: _renderSearchResults, fetchModels }); registerPaneActions({ setSendBtn: _setSendBtn, deactivate, streamToPane, renderSearchResults: _renderSearchResults, fetchModels });
// ──────────────────────────────────────────────────────────────────────────── // ────────────────────────────────────────────────────────────────────────────

Some files were not shown because too many files have changed in this diff Show More