Compare commits

..
Author SHA1 Message Date
RaresKeY 96c88c27c8 fix(personal): bound multi-file upload memory 2026-08-15 09:02:55 +00:00
Joeseph Grey 6edd771cc9 Merge branch 'dev' into fix/add-directory-event-loop 2026-08-12 10:39:22 -06:00
StressTestor 938251000b fix(personal): route upload and delete through the index job lock
/api/personal/upload and DELETE /api/personal/file mutated the same
vector and tracking state add/remove/reload serialize on, outside
_index_job_lock and inline on the event loop.

Both now stage async work on the loop, then run the complete transition
(vector writes, disk change, personal_docs_manager update) in one
offloaded critical section under the shared lock, acquired before the
offload so queued requests park on the loop rather than pinning a
threadpool worker.

Adds add-vs-upload and add-vs-file ordering regressions.
2026-08-12 10:17:36 -06:00
StressTestor e222e92153 fix(personal): serialize add/remove/reload on an async job lock
The #5558 fix took the job lock INSIDE the threadpool worker and only on the
add path, so (1) remove_directory and /reload mutated PersonalDocsManager's
unsynchronized list/index concurrently with an in-flight add — the inconsistent
state the PR claimed to prevent — and (2) a queued add blocked on the lock while
holding an AnyIO threadpool token, starving the shared pool.

Move the lock to an asyncio.Lock acquired in the async handler BEFORE offloading,
and route add, remove and reload through it. A waiting request now parks on the
event loop instead of pinning a worker, and all three mutators are serialized so
the 'add/remove are serialized and cannot leave inconsistent state' guarantee
holds. remove and reload also run their blocking work off the event loop. The
lock is per-router so each app binds it to its own loop; single-process scope.

Tests: add-vs-remove and add-vs-reload serialization regressions (async via
ASGITransport, since asyncio.Lock deadlocks starlette TestClient's portal); the
existing add-vs-add test converted to the same driver.
2026-07-27 20:43:04 +00:00
StressTestor b91f48f50a fix(personal): run directory indexing off the event loop (#5558)
POST /api/personal/add_directory called rag.index_personal_documents
inline from an async handler, so the whole indexing job (os.walk, file
reads, per-chunk embedding, Chroma inserts) ran on the event loop and
every other request queued behind it. Indexing a real directory froze
the UI and API for 25+ minutes with no sign of life.

Move the blocking section into the threadpool via run_in_threadpool.
personal_docs_manager.add_directory stays inside it because its
refresh_index() re-extracts text across tracked directories, which is
also blocking work. A module-level lock serializes index jobs so the
threadpool move does not introduce parallel jobs racing
PersonalDocsManager's unsynchronized list mutations and file writes;
they previously serialized on the blocked loop, so one-at-a-time is
behavior parity.
2026-07-27 20:43:04 +00:00
367 changed files with 4650 additions and 45427 deletions
-2
View File
@@ -30,8 +30,6 @@ secrets.env~
.idea/ .idea/
dev-docs/ dev-docs/
docs/ docs/
website/
assets/branding/
*.md *.md
*.db *.db
*.sqlite *.sqlite
+2 -29
View File
@@ -76,24 +76,12 @@ SEARXNG_INSTANCE=http://localhost:8080
# Change this if another local service already uses 7000 (macOS AirPlay often does). # Change this if another local service already uses 7000 (macOS AirPlay often does).
# APP_PORT=7000 # APP_PORT=7000
# Optional HTTP address advertised in companion/mobile pairing codes. Set this
# when Docker would otherwise advertise a container address or loopback. Use a
# LAN or Tailscale IPv4 address, a single-label hostname, or an mDNS *.local
# name that the phone can reach. HTTPS and public hostnames are not supported
# by the current companion client. Do not include credentials, a path, query,
# or fragment.
# COMPANION_BASE_URL=http://192.168.1.50:7000
# Development-only auth bypass for loopback requests. # Development-only auth bypass for loopback requests.
# Keep false for Docker, LAN, reverse proxy, and any shared deployment. # Keep false for Docker, LAN, reverse proxy, and any shared deployment.
# LOCALHOST_BYPASS=false # LOCALHOST_BYPASS=false
# Mark session cookies Secure. Left unset, this follows the request scheme: # Mark session cookies Secure. Set true when Odysseus is served through HTTPS
# an HTTPS login gets a Secure cookie, a plain-HTTP one does not. Set true to # by a trusted reverse proxy or private access gateway.
# force it on, or false to force it off while you still serve plain HTTP.
# Upgrading: this used to default to false. Drop a leftover SECURE_COOKIES=false
# from your .env unless you still need that escape hatch — it keeps HTTPS logins
# on a non-Secure cookie.
# SECURE_COOKIES=true # SECURE_COOKIES=true
# Optional: pre-seed the first admin password during setup. # Optional: pre-seed the first admin password during setup.
@@ -163,21 +151,6 @@ SEARXNG_INSTANCE=http://localhost:8080
# Local HTTP setups may use the callback URL inferred by the application. # Local HTTP setups may use the callback URL inferred by the application.
# GOOGLE_OAUTH_REDIRECT_URI=https://your-domain.com/api/email/oauth/google/callback # GOOGLE_OAUTH_REDIRECT_URI=https://your-domain.com/api/email/oauth/google/callback
# Origin the MCP OAuth callback is sent back to, for remote (Streamable HTTP)
# MCP servers that register it dynamically. Defaults to http://localhost:$APP_PORT,
# which is right only when you reach Odysseus directly on that port. Set it for
# HTTPS, reverse-proxy, hosted, and Docker installs — inside the container the
# app always listens on 7000 and cannot see the host port map, so the default is
# wrong there whenever APP_PORT is not 7000.
#
# Not for Google MCP servers. Those use Desktop App credentials, and Google only
# accepts loopback redirect URIs for that client type, so a public origin here is
# rejected with redirect_uri_mismatch. Leave it unset for a Google-only install:
# the loopback default is what Google wants, and remote users finish through the
# paste-back page, which never has to load the redirect.
# https://developers.google.com/identity/protocols/oauth2/native-app
# OAUTH_REDIRECT_BASE_URL=https://your-domain.com
# ============================================================ # ============================================================
# Misc # Misc
# ============================================================ # ============================================================
-7
View File
@@ -15,13 +15,6 @@ docker/entrypoint.sh text eol=lf
*.cmd text eol=crlf *.cmd text eol=crlf
*.bat text eol=crlf *.bat text eol=crlf
# Vendored third-party bundles in static/lib/ are published minified artifacts
# and must stay byte-identical to what npm ships — stripping trailing whitespace
# to satisfy `git diff --check` would desync them from the upstream release. Turn
# the whitespace check off for that tree instead, and keep the bundles out of
# GitHub's language statistics.
static/lib/** -whitespace linguist-vendored
# Binary assets — never normalize. # Binary assets — never normalize.
*.png binary *.png binary
*.jpg binary *.jpg binary
+1 -1
View File
@@ -6,4 +6,4 @@
# A per-area ownership map (security/auth, CI, frontend, agent internals, with # A per-area ownership map (security/auth, CI, frontend, agent internals, with
# multiple named owners per line) is being worked out in issue #593; once # multiple named owners per line) is being worked out in issue #593; once
# agreed it replaces this file. Until then, required reviews and the security # agreed it replaces this file. Until then, required reviews and the security
# CI gate (website/security-ci.md) remain in force via branch protection. # CI gate (docs/security-ci.md) remain in force via branch protection.
-12
View File
@@ -26,18 +26,6 @@ body:
- label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing. - label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing.
required: true required: true
- type: input
id: revision
attributes:
label: Odysseus Revision
description: |
From the repository root (on the host when using Docker), run
`git show -s --abbrev=12 --format='%h (%cs)' HEAD`
and paste the output exactly.
placeholder: "1fef4929cf1d (2026-08-11)"
validations:
required: true
- type: dropdown - type: dropdown
id: install-method id: install-method
attributes: attributes:
-1
View File
@@ -28,7 +28,6 @@ Fixes #
- [ ] This PR targets `dev` - [ ] This PR targets `dev`
- [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in. - [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
- [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough. - [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough.
- [ ] I did not run the app/runtime validation and stated that gap in **How to Test**. Leave this unchecked when the app-run box above is checked.
## How to Test ## How to Test
@@ -41,14 +41,6 @@ module.exports = async ({ github, context, core }) => {
break; break;
case 'bug': { case 'bug': {
const revisionText = section('Odysseus Revision');
if (!/^[0-9a-f]{12} \(\d{4}-\d{2}-\d{2}\)$/i.test(revisionText)) {
failures.push(
'**Odysseus Revision** — paste the 12-character commit SHA and date, ' +
'for example `1fef4929cf1d (2026-08-11)`',
);
}
if (!section('Install Method')) { if (!section('Install Method')) {
failures.push('**Install Method** — select how you installed Odysseus'); failures.push('**Install Method** — select how you installed Odysseus');
} }
+32 -142
View File
@@ -21,11 +21,11 @@ module.exports = async ({ github, context, core }) => {
return strip(m?.[0].replace(new RegExp(`#+\\s+${heading}`, 'i'), '') ?? ''); return strip(m?.[0].replace(new RegExp(`#+\\s+${heading}`, 'i'), '') ?? '');
} }
const descriptionProblems = []; const problems = [];
// 1. Summary must be filled in. // 1. Summary must be filled in.
if (section('Summary').length < 20) { if (section('Summary').length < 20) {
descriptionProblems.push('**Summary** is empty or too short — describe what changed and why.'); problems.push('**Summary** is empty or too short — describe what changed and why.');
} }
// 2. Linked Issue must reference a real issue. Accept a bare #NNN, a closing // 2. Linked Issue must reference a real issue. Accept a bare #NNN, a closing
@@ -34,18 +34,18 @@ module.exports = async ({ github, context, core }) => {
const linkedSection = section('Linked Issue'); const linkedSection = section('Linked Issue');
const hasIssueRef = /#\d+\b/.test(linkedSection) || /\/issues\/\d+/.test(linkedSection); const hasIssueRef = /#\d+\b/.test(linkedSection) || /\/issues\/\d+/.test(linkedSection);
if (!linkedSection || !hasIssueRef) { if (!linkedSection || !hasIssueRef) {
descriptionProblems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.'); problems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.');
} }
// 3. At least one Type of Change box must be checked. // 3. At least one Type of Change box must be checked.
const typeBlock = body.match(/##\s+Type of Change[\s\S]*?(?=\n##\s|$)/i)?.[0] ?? ''; const typeBlock = body.match(/##\s+Type of Change[\s\S]*?(?=\n##\s|$)/i)?.[0] ?? '';
if (!/- \[x\]/i.test(typeBlock)) { if (!/- \[x\]/i.test(typeBlock)) {
descriptionProblems.push('**Type of Change** — check at least one box.'); problems.push('**Type of Change** — check at least one box.');
} }
// 4. Duplicate-search checklist item must be checked. // 4. Duplicate-search checklist item must be checked.
if (!/- \[x\] I searched/i.test(body)) { if (!/- \[x\] I searched/i.test(body)) {
descriptionProblems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.'); problems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.');
} }
// 5. How to Test must contain enough real detail for a reviewer to act on. // 5. How to Test must contain enough real detail for a reviewer to act on.
@@ -53,83 +53,7 @@ module.exports = async ({ github, context, core }) => {
// code block — so we only require non-trivial content, not a specific shape. // code block — so we only require non-trivial content, not a specific shape.
const howTo = section('How to Test'); const howTo = section('How to Test');
if (howTo.length < 30) { if (howTo.length < 30) {
descriptionProblems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").'); problems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").');
}
// Classify paths from GitHub's API. This workflow runs in the privileged base
// context, so it must never check out or execute code from the PR branch.
const changedFiles = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: prNum, per_page: 100,
});
const changedPaths = changedFiles.map(file => file.filename);
function isUiSensitivePath(filename) {
const path = filename.toLowerCase();
return path.startsWith('static/')
|| path.startsWith('templates/')
|| /\.(?:html?|css|svg)$/.test(path);
}
function isDocsOnlyPath(filename) {
const path = filename.toLowerCase();
return /\.(?:md|mdx|rst|adoc|txt)$/.test(path)
|| (path.startsWith('docs/') && !isUiSensitivePath(path));
}
function isRuntimeSensitivePath(filename) {
const path = filename.toLowerCase();
if (isUiSensitivePath(path)) return false;
if (path.startsWith('tests/') || path.startsWith('.github/')) return false;
return /^(?:app\.py|routes\/|services\/|src\/|core\/|mcp_servers\/|scripts\/|docker\/)/.test(path)
|| /^(?:dockerfile|docker-compose.*\.ya?ml|requirements(?:-optional)?\.txt|pyproject\.toml|setup\.py)$/.test(path)
|| /\.(?:py|sh|ps1|bat)$/.test(path);
}
let classification = 'tooling';
if (changedPaths.some(isUiSensitivePath)) {
classification = 'UI-sensitive';
} else if (changedPaths.some(isRuntimeSensitivePath)) {
classification = 'backend/runtime';
} else if (changedPaths.length > 0 && changedPaths.every(isDocsOnlyPath)) {
classification = 'docs-only';
}
const appRan = /- \[x\]\s+I actually ran the app\b/i.test(body);
const appNotRun = /- \[x\]\s+I did not run the app\/runtime validation\b/i.test(body);
// Anchor on the wording, not the template's emphasis: a ticked box the author
// retyped without the surrounding ** renders identically on the PR page, so
// treating it as unchecked is invisible from their side. Matches the two
// attestations above, which already ignore formatting.
const screenshotChecked = /- \[x\]\s+[*_]{0,2}Screenshot or short clip[*_]{0,2}/i.test(body);
const screenshotSection = section('Screenshots / clips');
const hasVisualEvidence = /!\[[^\]]*\]\([^)]+\)|<(?:img|video|source)\b[^>]*(?:src|href)=|https?:\/\/[^\s)]+/i.test(screenshotSection);
const evidenceGaps = [];
let needsRuntimeValidation = false;
let needsVisualEvidence = false;
if (classification === 'backend/runtime' || classification === 'UI-sensitive') {
if (appRan && appNotRun) {
needsRuntimeValidation = true;
evidenceGaps.push('The app-run and explicit not-run boxes are both checked. Select the one state that is true.');
} else if (!appRan) {
needsRuntimeValidation = true;
if (appNotRun) {
evidenceGaps.push('The author explicitly reports that app/runtime validation was not performed.');
} else {
evidenceGaps.push('App/runtime validation is not author-attested. Check the run box only after running it, or check the explicit not-run box and describe the gap.');
}
}
}
if (classification === 'UI-sensitive') {
if (!screenshotChecked) {
needsVisualEvidence = true;
evidenceGaps.push('The screenshot/clip checkbox is not checked for this UI-sensitive change.');
}
if (!hasVisualEvidence) {
needsVisualEvidence = true;
evidenceGaps.push('The Screenshots / clips section does not contain an actual attachment or link.');
}
} }
// ── Comment ────────────────────────────────────────────────────────────── // ── Comment ──────────────────────────────────────────────────────────────
@@ -138,43 +62,22 @@ module.exports = async ({ github, context, core }) => {
}); });
const existing = comments.find(c => (c.body ?? '').includes(MARKER)); const existing = comments.find(c => (c.body ?? '').includes(MARKER));
if (descriptionProblems.length === 0 && evidenceGaps.length === 0) { if (problems.length === 0) {
if (existing) { if (existing) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id }); await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
} }
} else { } else {
const commentLines = [MARKER]; const commentBody = [
if (descriptionProblems.length > 0) { MARKER,
commentLines.push( '⚠️ **PR description — action needed**',
'⚠️ **PR description — action needed**', '',
'', 'The following required sections are missing or incomplete. Please update the PR description to address them:',
'The following required sections are missing or incomplete. Please update the PR description to address them:', '',
'', problems.map(p => `- ${p}`).join('\n'),
descriptionProblems.map(problem => `- ${problem}`).join('\n'),
);
} else {
commentLines.push(
'⚠️ **PR description is complete; validation evidence is still outstanding**',
'',
`Changed-file classification: **${classification}**.`,
);
}
if (evidenceGaps.length > 0) {
commentLines.push(
'',
'**Author-reported runtime / visual state**',
'',
evidenceGaps.map(gap => `- ${gap}`).join('\n'),
'',
'Checkboxes are author attestations. GitHub Actions results remain the execution evidence for CI; this check does not prove that a local command ran.',
);
}
commentLines.push(
'', '',
'---', '---',
'_This comment updates automatically when the description or changed files change._', '_This comment is deleted automatically once all sections are complete._',
); ].join('\n');
const commentBody = commentLines.join('\n');
if (existing) { if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: commentBody }); await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: commentBody });
@@ -194,47 +97,34 @@ module.exports = async ({ github, context, core }) => {
return true; return true;
} catch (e) { } catch (e) {
if (e.status === 404) return false; if (e.status === 404) return false;
if (e.status === 403) {
core.warning(`Could not inspect label "${name}" — token lacks label read access; skipping.`);
return false;
}
throw e; throw e;
} }
} }
async function setLabel(name, wanted) { async function swapLabel(num, add, remove) {
if (wanted && await labelExists(name)) { if (await labelExists(add)) {
try { try {
await github.rest.issues.addLabels({ owner, repo, issue_number: prNum, labels: [name] }); await github.rest.issues.addLabels({ owner, repo, issue_number: num, labels: [add] });
} catch (e) { } catch (e) {
// Fail soft on a token that can't write labels so a label permission // Fail soft on a token that can't write labels so a label permission
// problem never masks the actual description verdict. // problem never masks the actual description verdict.
if (e.status !== 403 && e.status !== 404) throw e; if (e.status !== 403) throw e;
core.warning(`Could not add "${name}" — label is unavailable or the token lacks label write access; skipping.`); core.warning(`Could not add "${add}" — token lacks label write here; skipping.`);
} }
} else if (wanted) {
core.warning(`Label "${name}" does not exist in the repo — skipping. Create it once to enable labelling.`);
} else { } else {
try { core.warning(`Label "${add}" does not exist in the repo — skipping. Create it once to enable labelling.`);
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNum, name }); }
} catch (e) { try {
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e; await github.rest.issues.removeLabel({ owner, repo, issue_number: num, name: remove });
} } catch (e) {
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e;
} }
} }
const descriptionComplete = descriptionProblems.length === 0; if (problems.length === 0) {
const evidenceComplete = evidenceGaps.length === 0; await swapLabel(prNum, 'ready for review', 'needs work');
const isDraft = Boolean(context.payload.pull_request.draft); } else {
await setLabel( await swapLabel(prNum, 'needs work', 'ready for review');
'ready for review', core.setFailed(`PR description has ${problems.length} issue(s) — see bot comment for details.`);
descriptionComplete && evidenceComplete && !isDraft,
);
await setLabel('needs work', !descriptionComplete);
await setLabel('needs runtime validation', needsRuntimeValidation);
await setLabel('needs visual evidence', needsVisualEvidence);
if (!descriptionComplete) {
core.setFailed(`PR description has ${descriptionProblems.length} issue(s) — see bot comment for details.`);
} }
}; };
+10 -11
View File
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
continue-on-error: true continue-on-error: true
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
@@ -73,10 +73,10 @@ jobs:
name: Python syntax (compileall) name: Python syntax (compileall)
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: "3.11" python-version: "3.11"
# Byte-compile sources — catches syntax errors without installing deps. # Byte-compile sources — catches syntax errors without installing deps.
@@ -86,10 +86,10 @@ jobs:
name: JS syntax (node --check) name: JS syntax (node --check)
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with: with:
node-version: "20" node-version: "20"
# Syntax-check our own JS (skip vendored libs in static/lib). # Syntax-check our own JS (skip vendored libs in static/lib).
@@ -105,12 +105,12 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
# Make Python test validation authoritative for the configured scope. # Make Python test validation authoritative for the configured scope.
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
# Detect whether this PR only touches repository prose outside the Pages site. # Detect whether this PR only touches documentation files.
# If so, skip the expensive pytest run while still reporting a passing check. # If so, skip the expensive pytest run while still reporting a passing check.
- name: Check for docs-only changes - name: Check for docs-only changes
id: docs-check id: docs-check
@@ -122,10 +122,9 @@ jobs:
BASE="${{ github.event.before }}" BASE="${{ github.event.before }}"
HEAD="${{ github.sha }}" HEAD="${{ github.sha }}"
fi fi
# Keep website/ and assets/branding/ out of this bypass: pytest owns # List all changed files; if every file matches docs/markdown patterns, skip pytest.
# regression guards for their published-file and orphan-asset contracts.
changed=$(git diff --name-only "$BASE" "$HEAD" 2>/dev/null || git diff --name-only HEAD~1 HEAD) changed=$(git diff --name-only "$BASE" "$HEAD" 2>/dev/null || git diff --name-only HEAD~1 HEAD)
non_docs=$(echo "$changed" | grep -Ev '^(docs/|[^/]+\.md$|\.github/[^/]+\.md$)' || true) non_docs=$(echo "$changed" | grep -Ev '^(docs/|.*\.md$|\.github/[^/]+\.md$)' || true)
if [ -z "$non_docs" ]; then if [ -z "$non_docs" ]; then
echo "docs_only=true" >> "$GITHUB_OUTPUT" echo "docs_only=true" >> "$GITHUB_OUTPUT"
echo "Docs-only change detected — skipping pytest." echo "Docs-only change detected — skipping pytest."
@@ -133,7 +132,7 @@ jobs:
echo "docs_only=false" >> "$GITHUB_OUTPUT" echo "docs_only=false" >> "$GITHUB_OUTPUT"
fi fi
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
if: steps.docs-check.outputs.docs_only != 'true' if: steps.docs-check.outputs.docs_only != 'true'
with: with:
python-version: "3.11" python-version: "3.11"
+3 -3
View File
@@ -27,15 +27,15 @@ jobs:
language: [actions, javascript-typescript, python] language: [actions, javascript-typescript, python]
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7 uses: github/codeql-action/init@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
build-mode: none build-mode: none
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7 uses: github/codeql-action/analyze@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
with: with:
category: "/language:${{ matrix.language }}" category: "/language:${{ matrix.language }}"
+2 -2
View File
@@ -37,12 +37,12 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Lint Dockerfile - name: Lint Dockerfile
uses: hadolint/hadolint-action@2a66e89f53d0771bb131a7fa31f3136336094aa6 # v3.4.0 uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with: with:
dockerfile: Dockerfile dockerfile: Dockerfile
# DL3008: pinning apt package versions is impractical on a -slim base # DL3008: pinning apt package versions is impractical on a -slim base
+7 -11
View File
@@ -23,16 +23,12 @@ on:
paths-ignore: paths-ignore:
- '**.md' - '**.md'
- 'docs/**' - 'docs/**'
- 'website/**'
- 'assets/branding/**'
- '.github/ISSUE_TEMPLATE/**' - '.github/ISSUE_TEMPLATE/**'
push: push:
branches: [main] branches: [main]
paths-ignore: paths-ignore:
- '**.md' - '**.md'
- 'docs/**' - 'docs/**'
- 'website/**'
- 'assets/branding/**'
- '.github/ISSUE_TEMPLATE/**' - '.github/ISSUE_TEMPLATE/**'
workflow_dispatch: workflow_dispatch:
@@ -56,17 +52,17 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up Buildx - name: Set up Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
# Build without pushing so a broken Dockerfile is caught here, and the # Build without pushing so a broken Dockerfile is caught here, and the
# exact image we ship is what gets scanned. # exact image we ship is what gets scanned.
- name: Build image - name: Build image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with: with:
context: . context: .
push: false push: false
@@ -97,15 +93,15 @@ jobs:
security-events: write # upload SARIF to the Security tab security-events: write # upload SARIF to the Security tab
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up Buildx - name: Set up Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Build image - name: Build image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with: with:
context: . context: .
push: false push: false
@@ -123,7 +119,7 @@ jobs:
TRIVY_DB_REPOSITORY: ghcr.io/aquasecurity/trivy-db:2 TRIVY_DB_REPOSITORY: ghcr.io/aquasecurity/trivy-db:2
- name: Upload Trivy results - name: Upload Trivy results
uses: github/codeql-action/upload-sarif@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7 uses: github/codeql-action/upload-sarif@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
with: with:
sarif_file: trivy-results.sarif sarif_file: trivy-results.sarif
category: trivy-image category: trivy-image
+3 -3
View File
@@ -36,7 +36,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
@@ -55,12 +55,12 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up Python - name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: '3.12' python-version: '3.12'
-50
View File
@@ -1,50 +0,0 @@
name: Deploy GitHub Pages
on:
push:
branches: [main]
paths:
- 'website/**'
- '.github/workflows/deploy-pages.yml'
workflow_dispatch:
permissions: {}
concurrency:
group: pages
cancel-in-progress: false
jobs:
build:
name: Package static site
runs-on: ubuntu-latest
permissions:
contents: read
pages: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
- uses: actions/jekyll-build-pages@44a6e6beabd48582f863aeeb6cb2151cc1716697 # v1.0.13
with:
source: website
destination: _site
- uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
with:
path: _site
deploy:
name: Deploy static site
needs: build
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
+8 -10
View File
@@ -14,8 +14,6 @@ on:
paths-ignore: paths-ignore:
- '**.md' - '**.md'
- 'docs/**' - 'docs/**'
- 'website/**'
- 'assets/branding/**'
- '.github/ISSUE_TEMPLATE/**' - '.github/ISSUE_TEMPLATE/**'
concurrency: concurrency:
@@ -47,20 +45,20 @@ jobs:
arch: arm64 arch: arm64
runner: ubuntu-24.04-arm runner: ubuntu-24.04-arm
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up Buildx - name: Set up Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Log in to GHCR - name: Log in to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with: with:
registry: ${{ env.REGISTRY }} registry: ${{ env.REGISTRY }}
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest - name: Build and push by digest
id: build id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with: with:
context: . context: .
platforms: ${{ matrix.platform }} platforms: ${{ matrix.platform }}
@@ -88,7 +86,7 @@ jobs:
contents: read contents: read
packages: write packages: write
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Read APP_VERSION + short sha - name: Read APP_VERSION + short sha
@@ -105,16 +103,16 @@ jobs:
pattern: digest-* pattern: digest-*
merge-multiple: true merge-multiple: true
- name: Set up Buildx - name: Set up Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0 uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Log in to GHCR - name: Log in to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with: with:
registry: ${{ env.REGISTRY }} registry: ${{ env.REGISTRY }}
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Compute tags - name: Compute tags
id: meta id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0 uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with: with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: | tags: |
@@ -14,7 +14,7 @@ jobs:
# Skip bots (Dependabot, release-drafter, etc.) # Skip bots (Dependabot, release-drafter, etc.)
if: ${{ github.event.issue.user.type != 'Bot' }} if: ${{ github.event.issue.user.type != 'Bot' }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
sparse-checkout: .github/scripts sparse-checkout: .github/scripts
persist-credentials: false persist-credentials: false
+4 -10
View File
@@ -5,11 +5,7 @@ on:
# works on fork PRs. Safe here: the checkout pins to the base branch (no fork # works on fork PRs. Safe here: the checkout pins to the base branch (no fork
# code runs) and the scripts only read context.payload and call the GitHub API. # code runs) and the scripts only read context.payload and call the GitHub API.
pull_request_target: # zizmor: ignore[dangerous-triggers] pull_request_target: # zizmor: ignore[dangerous-triggers]
types: [opened, edited, synchronize, reopened, ready_for_review, converted_to_draft] types: [opened, edited, synchronize, reopened, ready_for_review]
concurrency:
group: pr-description-${{ github.event.pull_request.number }}
cancel-in-progress: true
# Default-deny at the workflow level; each job opts into only the scopes it needs. # Default-deny at the workflow level; each job opts into only the scopes it needs.
# Note: modifying a PR's labels/comments needs pull-requests:write even though the # Note: modifying a PR's labels/comments needs pull-requests:write even though the
@@ -27,7 +23,7 @@ jobs:
# Skip bots: they open PRs programmatically and have their own process. # Skip bots: they open PRs programmatically and have their own process.
if: github.event.pull_request.user.type != 'Bot' if: github.event.pull_request.user.type != 'Bot'
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
ref: ${{ github.base_ref }} ref: ${{ github.base_ref }}
sparse-checkout: .github/scripts sparse-checkout: .github/scripts
@@ -63,14 +59,12 @@ jobs:
check-mergeable: check-mergeable:
name: Flag unmergeable PRs name: Flag unmergeable PRs
needs: check-description
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
pull-requests: write pull-requests: write
issues: write issues: write
# Run after description validation failures, but never from an obsolete # Skip bots: they open PRs programmatically and have their own process.
# workflow run canceled by a newer PR event. if: github.event.pull_request.user.type != 'Bot'
if: ${{ !cancelled() && github.event.pull_request.user.type != 'Bot' }}
steps: steps:
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with: with:
+1 -1
View File
@@ -35,7 +35,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
# Full history so a secret committed in an earlier commit (and later # Full history so a secret committed in an earlier commit (and later
# deleted) is still caught -- deletion does not remove it from Git. # deleted) is still caught -- deletion does not remove it from Git.
+3 -3
View File
@@ -36,7 +36,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
@@ -61,12 +61,12 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up Python - name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with: with:
python-version: '3.12' python-version: '3.12'
-18
View File
@@ -85,24 +85,6 @@ output.txt.txt
!docs/**/*.gif !docs/**/*.gif
!docs/**/*.webp !docs/**/*.webp
# …except shipped website and branding media.
!website/**/*.jpg
!website/**/*.jpeg
!website/**/*.png
!website/**/*.gif
!website/**/*.bmp
!website/**/*.webp
!website/**/*.tiff
!website/**/*.pdf
!assets/branding/**/*.jpg
!assets/branding/**/*.jpeg
!assets/branding/**/*.png
!assets/branding/**/*.gif
!assets/branding/**/*.bmp
!assets/branding/**/*.webp
!assets/branding/**/*.tiff
!assets/branding/**/*.pdf
# Reports and temp files # Reports and temp files
reports/ reports/
tasks/ tasks/
+2 -10
View File
@@ -65,16 +65,6 @@ Vendored in `static/lib/` and served directly:
| [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT | | [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT |
| [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT | | [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT |
| [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT | | [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT |
| [KaTeX](https://github.com/KaTeX/KaTeX) v0.16.22 (`katex/katex.min.{js,css}` + `katex/fonts/*.woff2`) | Math typesetting | MIT ([`licenses/KaTeX-MIT-LICENSE.txt`](licenses/KaTeX-MIT-LICENSE.txt)) |
| [Mermaid](https://github.com/mermaid-js/mermaid) v11.16.1 (`mermaid.min.js`) | Diagrams from text | MIT ([`licenses/Mermaid-MIT-LICENSE.txt`](licenses/Mermaid-MIT-LICENSE.txt)) |
KaTeX and Mermaid are loaded on first use by `static/js/markdown.js` rather than
from `index.html`, so a session that renders no math and no diagram never fetches
either. Only the `.woff2` KaTeX fonts are shipped, matching `static/fonts/`; the
`.woff` and `.ttf` variants its stylesheet also lists are never requested by a
browser that supports `woff2`. The bundles are the published npm artifacts,
unmodified — `.gitattributes` turns the whitespace check off for `static/lib/`
so they can stay byte-identical to upstream.
## Front-end libraries loaded at runtime (CDN) ## Front-end libraries loaded at runtime (CDN)
@@ -82,6 +72,8 @@ Referenced from `cdn.jsdelivr.net` / `cdnjs.cloudflare.com` at runtime — not v
| Library | Purpose | License | | Library | Purpose | License |
|---|---|---| |---|---|---|
| [KaTeX](https://github.com/KaTeX/KaTeX) 0.16.22 | Math typesetting | MIT |
| [Mermaid](https://github.com/mermaid-js/mermaid) 11 | Diagrams from text | MIT |
| [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 | | [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 |
| [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT | | [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT |
+10 -15
View File
@@ -1,5 +1,5 @@
<p align="center"> <p align="center">
<img src="assets/branding/odysseus-wordmark.png" alt="Odysseus" width="238"> <img src="docs/odysseus-wordmark.png" alt="Odysseus" width="238">
</p> </p>
<p align="center"> <p align="center">
@@ -8,7 +8,7 @@
<p align="center"> <p align="center">
<a href="#quick-start">Quick Start</a> · <a href="#quick-start">Quick Start</a> ·
<a href="website/setup.md">Setup Guide</a> · <a href="docs/setup.md">Setup Guide</a> ·
<a href="CONTRIBUTING.md">Contributing</a> · <a href="CONTRIBUTING.md">Contributing</a> ·
<a href="ROADMAP.md">Roadmap</a> <a href="ROADMAP.md">Roadmap</a>
</p> </p>
@@ -18,7 +18,7 @@
</p> </p>
<p align="center"> <p align="center">
<img src="assets/branding/odysseus-browser.jpg" alt="Odysseus interface"> <img src="docs/odysseus-browser.jpg" alt="Odysseus interface">
</p> </p>
--- ---
@@ -36,7 +36,7 @@ docker compose up -d --build
Open `http://localhost:7000` when the containers are healthy. The first admin password is printed in `docker compose logs odysseus`. Open `http://localhost:7000` when the containers are healthy. The first admin password is printed in `docker compose logs odysseus`.
Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration live in the [setup guide](website/setup.md). Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration live in the [setup guide](docs/setup.md).
## Features ## Features
@@ -51,7 +51,7 @@ Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration
## Demo ## Demo
A full hover-to-play tour lives on the [Odysseus landing page](https://odysseus-dev.github.io/odysseus/). Its source lives under [`website/`](website/). A full hover-to-play tour lives on the landing page: [`docs/index.html`](docs/index.html).
## Contributing ## Contributing
@@ -59,20 +59,15 @@ Help is welcome. The best entry points are fresh-install testing, provider setup
## Security ## Security
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Deployment details are in the [setup guide](docs/setup.md#security-notes).
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development.
Deployment details are in the [setup guide](website/setup.md#security-notes).
## Star History ## Star History
<a href="https://star-history.dera.page/#odysseus-dev/odysseus&type=date&legend=top-left"> <a href="https://www.star-history.com/?repos=odysseus-dev%2Fodysseus&type=date&legend=top-left">
<picture> <picture>
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" /> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" /> <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
</picture> </picture>
</a> </a>
+1 -1
View File
@@ -10,7 +10,7 @@ Security fixes are handled on the default branch until formal releases are cut.
- Keep `AUTH_ENABLED=true` for any network-accessible deployment. - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development. - Keep `LOCALHOST_BYPASS=false` outside local development.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Set `SECURE_COOKIES=true` to force it on (for a proxy Odysseus cannot see the scheme of), or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS. - Set `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Use HTTPS when exposing the app beyond localhost. - Use HTTPS when exposing the app beyond localhost.
- Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN. - Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN.
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only.
+1 -1
View File
@@ -37,7 +37,7 @@ Non-admin defaults are in `core/auth.py:DEFAULT_PRIVILEGES`. Tool enforcement is
- **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`. - **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`.
- **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance. - **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance.
- **Reserved usernames:** request sentinels and the Default/Local storage owner cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`. - **Reserved usernames:** `internal-tool`, `api`, `demo`, `system` cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`.
- `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check. - `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check.
- **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate. - **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate.
+14 -60
View File
@@ -67,13 +67,7 @@ from core.constants import (
REQUEST_TIMEOUT, OPENAI_API_KEY, AUTH_FILE, REQUEST_TIMEOUT, OPENAI_API_KEY, AUTH_FILE,
) )
from core.database import SessionLocal, ApiToken from core.database import SessionLocal, ApiToken
from core.middleware import ( from core.middleware import SecurityHeadersMiddleware, is_cors_preflight
SecurityHeadersMiddleware,
get_application_route_path,
is_cors_preflight,
path_is_route_or_child,
with_asgi_root_path,
)
from core.auth import AuthManager, normalize_known_username from core.auth import AuthManager, normalize_known_username
from core.exceptions import ( from core.exceptions import (
SessionNotFoundError, InvalidFileUploadError, SessionNotFoundError, InvalidFileUploadError,
@@ -84,7 +78,6 @@ import bcrypt as _bcrypt
from src.app_helpers import abs_join, serve_html_with_nonce from src.app_helpers import abs_join, serve_html_with_nonce
from src.generated_images import GENERATED_IMAGE_HEADERS, resolve_generated_image_path from src.generated_images import GENERATED_IMAGE_HEADERS, resolve_generated_image_path
from src.owner_identity import auth_disabled
from starlette.responses import RedirectResponse from starlette.responses import RedirectResponse
# ========= LOGGING ========= # ========= LOGGING =========
@@ -255,7 +248,7 @@ from routes.auth_routes import setup_auth_routes, SESSION_COOKIE
auth_manager = AuthManager() auth_manager = AuthManager()
app.state.auth_manager = auth_manager app.state.auth_manager = auth_manager
AUTH_ENABLED = not auth_disabled() AUTH_ENABLED = os.getenv("AUTH_ENABLED", "true").lower() != "false"
LOCALHOST_BYPASS = os.getenv("LOCALHOST_BYPASS", "false").lower() == "true" LOCALHOST_BYPASS = os.getenv("LOCALHOST_BYPASS", "false").lower() == "true"
if LOCALHOST_BYPASS: if LOCALHOST_BYPASS:
logger.warning("LOCALHOST_BYPASS is enabled, loopback requests bypass authentication. Do not expose this instance to a network.") logger.warning("LOCALHOST_BYPASS is enabled, loopback requests bypass authentication. Do not expose this instance to a network.")
@@ -291,7 +284,7 @@ if AUTH_ENABLED:
def _is_auth_exempt(path: str) -> bool: def _is_auth_exempt(path: str) -> bool:
if path in AUTH_EXEMPT_EXACT: if path in AUTH_EXEMPT_EXACT:
return True return True
if any(path_is_route_or_child(path, p) for p in AUTH_EXEMPT_PREFIXES): if any(path.startswith(p) for p in AUTH_EXEMPT_PREFIXES):
return True return True
return any(p.match(path) for p in AUTH_EXEMPT_PATTERNS) return any(p.match(path) for p in AUTH_EXEMPT_PATTERNS)
@@ -362,7 +355,7 @@ if AUTH_ENABLED:
class AuthMiddleware(BaseHTTPMiddleware): class AuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next): async def dispatch(self, request: Request, call_next):
path = get_application_route_path(request.scope) path = request.url.path
# A genuine CORS preflight (OPTIONS + Access-Control-Request-Method) # A genuine CORS preflight (OPTIONS + Access-Control-Request-Method)
# carries no credentials by design and must reach CORSMiddleware to be # carries no credentials by design and must reach CORSMiddleware to be
# answered. AuthMiddleware is the outermost middleware, so gating the # answered. AuthMiddleware is the outermost middleware, so gating the
@@ -406,10 +399,7 @@ if AUTH_ENABLED:
if not auth_manager.is_configured: if not auth_manager.is_configured:
# No users yet — redirect to login for first-time setup # No users yet — redirect to login for first-time setup
if not path.startswith("/api/"): if not path.startswith("/api/"):
return RedirectResponse( return RedirectResponse(url="/login", status_code=302)
url=with_asgi_root_path(request.scope, "/login"),
status_code=302,
)
return JSONResponse(status_code=401, content={"error": "Setup required"}) return JSONResponse(status_code=401, content={"error": "Setup required"})
# --- Bearer token auth (API tokens for external integrations) --- # --- Bearer token auth (API tokens for external integrations) ---
@@ -471,10 +461,7 @@ if AUTH_ENABLED:
if not auth_manager.validate_token(token): if not auth_manager.validate_token(token):
if path.startswith("/api/"): if path.startswith("/api/"):
return JSONResponse(status_code=401, content={"error": "Not authenticated"}) return JSONResponse(status_code=401, content={"error": "Not authenticated"})
return RedirectResponse( return RedirectResponse(url="/login", status_code=302)
url=with_asgi_root_path(request.scope, "/login"),
status_code=302,
)
# Attach current username to request state for downstream routes # Attach current username to request state for downstream routes
request.state.current_user = auth_manager.get_username_for_token(token) request.state.current_user = auth_manager.get_username_for_token(token)
@@ -516,45 +503,23 @@ async def serve_generated_image(filename: str, request: Request):
# SECURITY: filename is the only key, so anyone who knows / guesses a # SECURITY: filename is the only key, so anyone who knows / guesses a
# 12-hex content hash could pull another user's image bytes. Require # 12-hex content hash could pull another user's image bytes. Require
# auth and verify ownership via the gallery row (when one exists). # auth and verify ownership via the gallery row (when one exists).
_is_bearer = False
try: try:
from src.auth_helpers import ( from src.auth_helpers import get_current_user
effective_user,
get_current_user,
is_bearer_principal,
require_chat_scope,
)
from core.database import SessionLocal as _SL, GalleryImage as _GI from core.database import SessionLocal as _SL, GalleryImage as _GI
_is_bearer = is_bearer_principal(request) _user = get_current_user(request)
if _is_bearer:
# Gallery JSON attributes rows to the token owner. Reuse the same
# owner/scope gate for the binary follow-up so the returned URL is
# actually readable by that bearer principal.
require_chat_scope(request)
_user = effective_user(request)
else:
_user = get_current_user(request)
if _user: if _user:
_db = _SL() _db = _SL()
try: try:
_row = _db.query(_GI).filter(_GI.filename == filename).first() _row = _db.query(_GI).filter(_GI.filename == filename).first()
# Generated-but-not-yet-imported images have no row → allow. # Generated-but-not-yet-imported images have no row → allow.
# A bearer gallery row must have the exact token owner; cookie # Row exists with a different owner → 404 (don't confirm existence).
# callers retain the legacy null-owner compatibility below. if _row is not None and _row.owner and _row.owner != _user:
if _row is not None and (
(_is_bearer and _row.owner != _user)
or (not _is_bearer and _row.owner and _row.owner != _user)
):
raise HTTPException(status_code=404, detail="Image not found") raise HTTPException(status_code=404, detail="Image not found")
finally: finally:
_db.close() _db.close()
except HTTPException: except HTTPException:
raise raise
except Exception as _e: except Exception as _e:
if _is_bearer:
# An authenticated bearer request must not become a public file
# read because ownership lookup degraded or the DB was unavailable.
raise HTTPException(status_code=404, detail="Image not found") from _e
logger.warning("Image ownership verification failed for %r", filename, exc_info=_e) logger.warning("Image ownership verification failed for %r", filename, exc_info=_e)
ext = filename.rsplit('.', 1)[-1].lower() ext = filename.rsplit('.', 1)[-1].lower()
mime = { mime = {
@@ -665,24 +630,13 @@ app.include_router(auth_router)
@app.post("/api/activity/heartbeat") @app.post("/api/activity/heartbeat")
async def activity_heartbeat(): async def activity_heartbeat():
from src.interactive_gate import ( from src.interactive_gate import mark_browser_activity
mark_browser_activity,
maybe_stop_background_tasks_for_heartbeat,
)
await mark_browser_activity() await mark_browser_activity()
async def _stop_background(): async def _stop_background():
try: try:
await maybe_stop_background_tasks_for_heartbeat( await task_scheduler.stop_background_tasks_for_foreground(reason="browser heartbeat")
task_scheduler.stop_background_tasks_for_foreground
)
except Exception: except Exception:
logging.getLogger("app.foreground_gate").debug( logging.getLogger("app.foreground_gate").debug("heartbeat task stop failed", exc_info=True)
"heartbeat task stop failed",
exc_info=True,
)
asyncio.create_task(_stop_background()) asyncio.create_task(_stop_background())
return {"ok": True} return {"ok": True}
@@ -806,7 +760,7 @@ from src.task_scheduler import TaskScheduler
task_scheduler = TaskScheduler(session_manager) task_scheduler = TaskScheduler(session_manager)
from src.event_bus import set_task_scheduler from src.event_bus import set_task_scheduler
set_task_scheduler(task_scheduler) set_task_scheduler(task_scheduler)
from routes.task.task_routes import setup_task_routes from routes.task_routes import setup_task_routes
app.include_router(setup_task_routes(task_scheduler)) app.include_router(setup_task_routes(task_scheduler))
from routes.assistant_routes import setup_assistant_routes from routes.assistant_routes import setup_assistant_routes
+4 -8
View File
@@ -27,13 +27,13 @@ echo " port: $PORT"
rm -rf "$APP" rm -rf "$APP"
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources" mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
# ── Icon (best effort) — center-crop the branding image to a square .icns ── # ── Icon (best effort) — center-crop docs/odysseus.jpg to a square .icns ──
if [ -f "$REPO_DIR/assets/branding/odysseus.jpg" ] && command -v sips >/dev/null 2>&1; then if [ -f "$REPO_DIR/docs/odysseus.jpg" ] && command -v sips >/dev/null 2>&1; then
TMPIMG="$(mktemp -d)" TMPIMG="$(mktemp -d)"
# Center-crop to a square, scale to 512 (sips' icns encoder caps at 512), and # Center-crop to a square, scale to 512 (sips' icns encoder caps at 512), and
# let sips emit the .icns directly — more robust across macOS versions than # let sips emit the .icns directly — more robust across macOS versions than
# building an .iconset by hand. # building an .iconset by hand.
sips -c 720 720 "$REPO_DIR/assets/branding/odysseus.jpg" --out "$TMPIMG/sq.png" >/dev/null 2>&1 || cp "$REPO_DIR/assets/branding/odysseus.jpg" "$TMPIMG/sq.png" sips -c 720 720 "$REPO_DIR/docs/odysseus.jpg" --out "$TMPIMG/sq.png" >/dev/null 2>&1 || cp "$REPO_DIR/docs/odysseus.jpg" "$TMPIMG/sq.png"
sips -z 512 512 "$TMPIMG/sq.png" --out "$TMPIMG/icon.png" >/dev/null 2>&1 sips -z 512 512 "$TMPIMG/sq.png" --out "$TMPIMG/icon.png" >/dev/null 2>&1
if sips -s format icns "$TMPIMG/icon.png" --out "$APP/Contents/Resources/odysseus.icns" >/dev/null 2>&1; then if sips -s format icns "$TMPIMG/icon.png" --out "$APP/Contents/Resources/odysseus.icns" >/dev/null 2>&1; then
echo " icon: odysseus.icns" echo " icon: odysseus.icns"
@@ -42,7 +42,7 @@ if [ -f "$REPO_DIR/assets/branding/odysseus.jpg" ] && command -v sips >/dev/null
fi fi
rm -rf "$TMPIMG" rm -rf "$TMPIMG"
else else
echo " icon: (skipped — no assets/branding/odysseus.jpg)" echo " icon: (skipped — no docs/odysseus.jpg)"
fi fi
# ── Info.plist ── # ── Info.plist ──
@@ -73,10 +73,6 @@ cat > "$APP/Contents/MacOS/$APP_NAME.tmpl" <<'LAUNCHER'
INSTALL_DIR="__INSTALL_DIR__" INSTALL_DIR="__INSTALL_DIR__"
PORT="__PORT__" PORT="__PORT__"
URL="http://127.0.0.1:${PORT}" URL="http://127.0.0.1:${PORT}"
# uvicorn is started with --port below, but APP_PORT is what the app itself
# reads when it needs to build a URL for this instance (internal_api_base(),
# companion pairing, the MCP OAuth callback), so export it as well.
export APP_PORT="$PORT"
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH" export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH"
UVICORN="$INSTALL_DIR/venv/bin/uvicorn" UVICORN="$INSTALL_DIR/venv/bin/uvicorn"
-99
View File
@@ -6,14 +6,11 @@ units so the route layer stays thin and the logic is directly testable.
from __future__ import annotations from __future__ import annotations
import ipaddress
import json import json
import os import os
import re
import secrets import secrets
import socket import socket
import uuid import uuid
from urllib.parse import urlsplit
import bcrypt import bcrypt
@@ -23,102 +20,6 @@ PAIRING_VERSION = 1
COMPANION_SCOPE = "chat" COMPANION_SCOPE = "chat"
_COMPANION_IPV4_NETWORKS = tuple(
ipaddress.ip_network(cidr)
for cidr in (
"10.0.0.0/8",
"100.64.0.0/10",
"127.0.0.0/8",
"169.254.0.0/16",
"172.16.0.0/12",
"192.168.0.0/16",
)
)
_DNS_LABEL_RE = re.compile(r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\Z")
def _valid_companion_client_host(host: str) -> bool:
"""Match the host forms supported by the current v1 Expo client."""
if not host or len(host) > 253 or not host.isascii() or "%" in host:
return False
try:
address = ipaddress.ip_address(host)
except ValueError:
labels = host.split(".")
if any(not _DNS_LABEL_RE.fullmatch(label) for label in labels):
return False
if any(label.startswith("xn--") for label in labels):
return False
# WHATWG URL parsers treat a decimal or ``0x`` single-label hostname
# as an IPv4 number even though Python's strict ``ipaddress`` parser
# rejects that spelling. The v1 client interpolates this host back
# into a URL, so accepting e.g. ``134744072`` would make the phone send
# its bearer token to public 8.8.8.8. Keep DNS labels unambiguous.
if len(labels) == 1 and (
labels[0].isdigit()
or re.fullmatch(r"0x[0-9a-f]*", labels[0]) is not None
):
return False
return len(labels) == 1 or (len(labels) >= 2 and labels[-1] == "local")
return isinstance(address, ipaddress.IPv4Address) and any(
address in network for network in _COMPANION_IPV4_NETWORKS
)
def parse_companion_base_url(value: str) -> tuple[str, int]:
"""Validate a v1 companion address and return its legacy (host, port).
The deployed client understands only HTTP plus a LAN-style host and port.
Reject anything outside that exact contract instead of advertising a URL
the client would reject, downgrade, or interpret differently.
"""
if not isinstance(value, str) or not value:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
if not value.isascii():
raise ValueError("COMPANION_BASE_URL must contain only ASCII characters")
if any(
ord(char) <= 32 or ord(char) == 127 or char in {"\\", "%"}
for char in value
):
raise ValueError(
"COMPANION_BASE_URL contains a forbidden character"
)
try:
parsed = urlsplit(value)
port = parsed.port
except ValueError as exc:
raise ValueError("COMPANION_BASE_URL must be a valid HTTP LAN origin") from exc
host = parsed.hostname
if parsed.scheme.lower() != "http" or not parsed.netloc or not host:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
if parsed.username is not None or parsed.password is not None:
raise ValueError("COMPANION_BASE_URL must not contain credentials")
if parsed.path or parsed.query or parsed.fragment:
raise ValueError("COMPANION_BASE_URL must not contain a path, query, or fragment")
if port is not None and not 1 <= port <= 65535:
raise ValueError("COMPANION_BASE_URL port must be between 1 and 65535")
if not _valid_companion_client_host(host):
raise ValueError("COMPANION_BASE_URL host is not supported by companion v1")
netloc = f"{host}:{port}" if port is not None else host
origin = f"http://{netloc}"
if value != origin:
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
return host, port or 80
def configured_companion_origin() -> tuple[str, int] | None:
"""Return the validated operator-configured v1 address, if any."""
value = os.environ.get("COMPANION_BASE_URL")
if value is None or value == "":
return None
return parse_companion_base_url(value)
def default_port() -> int: def default_port() -> int:
"""Best guess at the port the server is reachable on. Callers that know the """Best guess at the port the server is reachable on. Callers that know the
real request port should pass it explicitly.""" real request port should pass it explicitly."""
+8 -23
View File
@@ -23,7 +23,7 @@ from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse from fastapi.responses import HTMLResponse
from core.middleware import require_admin from core.middleware import require_admin
from src.auth_helpers import _auth_disabled, get_current_user from src.auth_helpers import get_current_user
from companion import pairing as _pairing from companion import pairing as _pairing
@@ -113,9 +113,8 @@ def setup_companion_routes() -> APIRouter:
The stock /api/models route scopes to get_current_user, which for a The stock /api/models route scopes to get_current_user, which for a
bearer token is the sandboxed pseudo-user "api" (owns nothing). Here we bearer token is the sandboxed pseudo-user "api" (owns nothing). Here we
scope to the token's real owner instead, plus legacy null-owner shared scope to the token's real owner instead, plus legacy null-owner shared
rows -- the same rule as owner_filter. Explicit auth-disabled mode keeps rows -- the same rule as owner_filter. Read-only; never returns api_key
the stock route's single-user all-endpoints view. Read-only; never material.
returns api_key material.
""" """
require_models_scope(request) require_models_scope(request)
import json as _json import json as _json
@@ -124,11 +123,6 @@ def setup_companion_routes() -> APIRouter:
from src.endpoint_resolver import build_chat_url from src.endpoint_resolver import build_chat_url
owner = token_owner(request) owner = token_owner(request)
single_user_mode = (
owner is None
and not getattr(request.state, "api_token", False)
and _auth_disabled()
)
out = [] out = []
db = SessionLocal() db = SessionLocal()
try: try:
@@ -139,7 +133,7 @@ def setup_companion_routes() -> APIRouter:
if owner: if owner:
q = q.filter((ModelEndpoint.owner == owner) | (ModelEndpoint.owner == None)) # noqa: E711 q = q.filter((ModelEndpoint.owner == owner) | (ModelEndpoint.owner == None)) # noqa: E711
for ep in q.all(): for ep in q.all():
if not single_user_mode and not owner_can_see(ep.owner, owner): if not owner_can_see(ep.owner, owner):
continue continue
try: try:
model_ids = _json.loads(ep.cached_models) if ep.cached_models else [] model_ids = _json.loads(ep.cached_models) if ep.cached_models else []
@@ -200,27 +194,19 @@ def setup_companion_routes() -> APIRouter:
the code works immediately, no restart. `?format=json` returns the the code works immediately, no restart. `?format=json` returns the
payload for an in-app pairing screen.""" payload for an in-app pairing screen."""
require_admin(request) require_admin(request)
try:
configured_origin = _pairing.configured_companion_origin()
except ValueError as exc:
raise HTTPException(500, str(exc)) from None
owner = get_current_user(request) owner = get_current_user(request)
invalidate = getattr(request.app.state, "invalidate_token_cache", None) invalidate = getattr(request.app.state, "invalidate_token_cache", None)
token_id, raw_token = mint_pairing_token(owner, invalidate) token_id, raw_token = mint_pairing_token(owner, invalidate)
if configured_origin: hosts = _pairing.lan_ip_candidates()
host, port = configured_origin host = hosts[0] if hosts else "127.0.0.1"
hosts = [host] port = request.url.port or _pairing.default_port()
else:
hosts = _pairing.lan_ip_candidates()
host = hosts[0] if hosts else "127.0.0.1"
port = request.url.port or _pairing.default_port()
payload = _pairing.pairing_payload(host, port, raw_token) payload = _pairing.pairing_payload(host, port, raw_token)
qr = _pairing.pairing_qr_png_data_uri(payload) qr = _pairing.pairing_qr_png_data_uri(payload)
qr_ok = bool(qr and qr.startswith("data:image/png;base64,")) qr_ok = bool(qr and qr.startswith("data:image/png;base64,"))
if (request.query_params.get("format") or "").lower() == "json": if (request.query_params.get("format") or "").lower() == "json":
response = { return {
"host": host, "host": host,
"port": port, "port": port,
"token": raw_token, "token": raw_token,
@@ -229,7 +215,6 @@ def setup_companion_routes() -> APIRouter:
"payload": payload, "payload": payload,
"qr": qr if qr_ok else None, "qr": qr if qr_ok else None,
} }
return response
import json as _json import json as _json
payload_json = _json.dumps(payload, separators=(",", ":")) payload_json = _json.dumps(payload, separators=(",", ":"))
+10 -28
View File
@@ -30,20 +30,11 @@ def atomic_write_json(path: str, data: Any, *, indent: Optional[int] = None) ->
""" """
os.makedirs(os.path.dirname(path) or ".", exist_ok=True) os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
tmp = f"{path}.tmp.{uuid.uuid4().hex}" tmp = f"{path}.tmp.{uuid.uuid4().hex}"
with open(tmp, "w", encoding="utf-8") as f:
try: json.dump(data, f, indent=indent)
with open(tmp, "w", encoding="utf-8") as f: f.flush()
json.dump(data, f, indent=indent) os.fsync(f.fileno())
f.flush() os.replace(tmp, path)
os.fsync(f.fileno())
os.replace(tmp, path)
finally:
# Directly unlink to avoid a check-then-act race condition.
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
try:
os.unlink(tmp)
except OSError:
pass
def atomic_write_text(path: str, text: str) -> None: def atomic_write_text(path: str, text: str) -> None:
@@ -51,17 +42,8 @@ def atomic_write_text(path: str, text: str) -> None:
raise TypeError("atomic_write_text expects a string") raise TypeError("atomic_write_text expects a string")
os.makedirs(os.path.dirname(path) or ".", exist_ok=True) os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
tmp = f"{path}.tmp.{uuid.uuid4().hex}" tmp = f"{path}.tmp.{uuid.uuid4().hex}"
with open(tmp, "w", encoding="utf-8") as f:
try: f.write(text)
with open(tmp, "w", encoding="utf-8") as f: f.flush()
f.write(text) os.fsync(f.fileno())
f.flush() os.replace(tmp, path)
os.fsync(f.fileno())
os.replace(tmp, path)
finally:
# Directly unlink to avoid a check-then-act race condition.
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
try:
os.unlink(tmp)
except OSError:
pass
+16 -9
View File
@@ -20,6 +20,7 @@ logger = logging.getLogger(__name__)
from core.atomic_io import atomic_write_json as _atomic_write_json # noqa: E402 from core.atomic_io import atomic_write_json as _atomic_write_json # noqa: E402
from core.middleware import INTERNAL_TOOL_USER # noqa: E402
DEFAULT_PRIVILEGES = { DEFAULT_PRIVILEGES = {
"can_use_agent": True, "can_use_agent": True,
@@ -48,18 +49,24 @@ ADMIN_PRIVILEGES["allowed_models_restricted"] = False
ADMIN_PRIVILEGES["block_all_models"] = False ADMIN_PRIVILEGES["block_all_models"] = False
from src.constants import AUTH_FILE, PASSWORD_MIN_LENGTH from src.constants import AUTH_FILE, PASSWORD_MIN_LENGTH
from src.owner_identity import RESERVED_AUTH_USERNAMES
DEFAULT_AUTH_PATH = AUTH_FILE DEFAULT_AUTH_PATH = AUTH_FILE
TOKEN_TTL = 60 * 60 * 24 * 7 # 7 days TOKEN_TTL = 60 * 60 * 24 * 7 # 7 days
# Usernames the auth + middleware layer reserves for request sentinels and # Usernames the auth + middleware layer reserve as internal "synthetic owner"
# internal storage owners; they must never belong to a real login account. # sentinels; they must never belong to a real account. The most dangerous is
# "internal-tool" is the most dangerous because `core.middleware.require_admin` # "internal-tool": `core.middleware.require_admin` treats any request whose
# treats it as the in-process tool loopback. "api" collides with bearer-token # `current_user == "internal-tool"` as the in-process tool loopback and grants
# attribution. "demo"/"system" are synthetic owners already special-cased by # admin, and because the cookie auth path sets `current_user` to the raw
# scheduler/assistant/research paths. The Default/Local owner is a storage # username, an account literally named "internal-tool" would be silently
# bucket for explicit auth-disabled no-login mode, not a login username. # treated as an admin by every `require_admin`-gated route. "api" collides with
RESERVED_USERNAMES = frozenset(RESERVED_AUTH_USERNAMES) # the bearer-token owner-attribution sentinel. "demo"/"system" round out the
# synthetic-owner set the rest of the codebase already special-cases (see
# `_SYNTHETIC_OWNERS` in routes/assistant_routes.py and the matching guards in
# src/task_scheduler.py / routes/research_routes.py) — a real account with one
# of those names would be denied an assistant and inconsistently owner-scoped.
# Refuse to create or rename into any of them so the sentinels can't be
# impersonated. (Keep this in sync with that synthetic-owner set.)
RESERVED_USERNAMES = frozenset({INTERNAL_TOOL_USER, "api", "demo", "system"})
def normalize_known_username(users: Dict[str, Any], username: str | None) -> Optional[str]: def normalize_known_username(users: Dict[str, Any], username: str | None) -> Optional[str]:
+3 -103
View File
@@ -5,7 +5,7 @@ from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
from urllib.parse import unquote, urlparse from urllib.parse import unquote, urlparse
from sqlalchemy import DDL, event, create_engine, Column, String, Text, Boolean, DateTime, Integer, ForeignKey, JSON, Index, UniqueConstraint, func, inspect, text from sqlalchemy import DDL, event, create_engine, Column, String, Text, Boolean, DateTime, Integer, ForeignKey, JSON, Index, func, inspect, text
from sqlalchemy.engine import Engine, make_url from sqlalchemy.engine import Engine, make_url
from sqlalchemy.types import TypeDecorator from sqlalchemy.types import TypeDecorator
from sqlalchemy.ext.declarative import declarative_base, declared_attr from sqlalchemy.ext.declarative import declarative_base, declared_attr
@@ -187,13 +187,6 @@ class Session(TimestampMixin, Base):
endpoint_url = Column(String, nullable=False) endpoint_url = Column(String, nullable=False)
model = Column(String, nullable=False) model = Column(String, nullable=False)
owner = Column(String, nullable=True, index=True) # username; null = legacy/shared owner = Column(String, nullable=True, index=True) # username; null = legacy/shared
# Bearer-chat sessions must retain the exact server-owned endpoint they
# were created from. Keep this reference non-cascading so endpoint
# disable/delete/owner changes remain observable as an orphan and fail
# closed at the next bearer LLM boundary.
model_endpoint_id = Column(String, nullable=True, index=True)
endpoint_provenance = Column(String, nullable=True)
# Configuration flags # Configuration flags
rag = Column(Boolean, default=False) rag = Column(Boolean, default=False)
@@ -287,47 +280,6 @@ class ChatMessage(Base):
Index('ix_messages_session_time', 'session_id', 'timestamp'), # Composite for efficient message retrieval Index('ix_messages_session_time', 'session_id', 'timestamp'), # Composite for efficient message retrieval
) )
class ChatSessionApprovalGrant(Base):
"""Server-owned, durable approval provenance for one chat session.
A resolved tool-approval card is display/history data, not authority. This
separate row is inserted only by the interactive approval continuation and
is keyed by the real session owner plus session id. It deliberately has no
update path; deleting the owning session cascades the grant so an old id
cannot carry approval authority into a newly-created conversation.
"""
__tablename__ = "chat_session_approval_grants"
id = Column(String, primary_key=True, index=True)
session_id = Column(
String,
ForeignKey("sessions.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
owner = Column(String, nullable=False, index=True)
approval_id = Column(String, nullable=False, index=True)
provenance_version = Column(Integer, nullable=False, default=1)
created_at = Column(DateTime, default=utcnow_naive, nullable=False)
__table_args__ = (
UniqueConstraint(
"session_id",
"owner",
"approval_id",
name="uq_chat_session_approval_grant",
),
Index(
"ix_chat_session_approval_grant_lookup",
"session_id",
"owner",
"provenance_version",
),
)
class Document(TimestampMixin, Base): class Document(TimestampMixin, Base):
"""Living document that the AI can create and edit in-place.""" """Living document that the AI can create and edit in-place."""
__tablename__ = "documents" __tablename__ = "documents"
@@ -1006,40 +958,6 @@ def _migrate_add_owner_column():
except Exception: except Exception:
pass pass
def _migrate_add_session_endpoint_provenance_columns():
"""Add the durable endpoint identity used by bearer session validation."""
import sqlite3
db_path = DATABASE_URL.replace("sqlite:///", "")
if not os.path.exists(db_path):
return
conn = None
try:
conn = sqlite3.connect(db_path)
columns = {row[1] for row in conn.execute("PRAGMA table_info(sessions)")}
if "model_endpoint_id" not in columns:
conn.execute("ALTER TABLE sessions ADD COLUMN model_endpoint_id TEXT")
if "endpoint_provenance" not in columns:
conn.execute("ALTER TABLE sessions ADD COLUMN endpoint_provenance TEXT")
conn.execute(
"CREATE INDEX IF NOT EXISTS ix_sessions_model_endpoint_id "
"ON sessions(model_endpoint_id)"
)
conn.commit()
logging.getLogger(__name__).info(
"Migrated: added session endpoint identity/provenance columns"
)
except Exception as e:
logging.getLogger(__name__).warning(
"Session endpoint provenance migration failed: %s", e
)
finally:
try:
conn.close()
except Exception:
pass
def _migrate_model_endpoints(): def _migrate_model_endpoints():
"""Recreate model_endpoints table if schema changed (url->base_url).""" """Recreate model_endpoints table if schema changed (url->base_url)."""
import sqlite3 import sqlite3
@@ -1573,25 +1491,8 @@ def _migrate_assign_legacy_owner():
with open(prefs_path, "r", encoding="utf-8") as f: with open(prefs_path, "r", encoding="utf-8") as f:
prefs = _json.load(f) prefs = _json.load(f)
if "_users" not in prefs and prefs: if "_users" not in prefs and prefs:
# Flat format → nest ordinary preferences under the admin # Flat format → nest under admin user
# user. Foreground fallback is an explicit per-owner opt-in, new_prefs = {"_users": {admin_user: prefs}}
# so auth-disabled consent must remain inert at the flat root
# rather than becoming consent for the first named owner.
foreground_keys = {
"foreground_fallback_enabled",
"foreground_model_fallbacks",
}
named_prefs = {
key: value
for key, value in prefs.items()
if key not in foreground_keys
}
new_prefs = {
key: prefs[key]
for key in foreground_keys
if key in prefs
}
new_prefs["_users"] = {admin_user: named_prefs}
with open(prefs_path, "w", encoding="utf-8") as f: with open(prefs_path, "w", encoding="utf-8") as f:
_json.dump(new_prefs, f, indent=2) _json.dump(new_prefs, f, indent=2)
logger.info(f"Migrated user_prefs.json to per-user format under '{admin_user}'") logger.info(f"Migrated user_prefs.json to per-user format under '{admin_user}'")
@@ -2193,7 +2094,6 @@ def init_db():
_migrate_add_supports_tools_column() _migrate_add_supports_tools_column()
_migrate_add_task_run_model_column() _migrate_add_task_run_model_column()
_migrate_add_owner_column() _migrate_add_owner_column()
_migrate_add_session_endpoint_provenance_columns()
_migrate_add_document_archived_column() _migrate_add_document_archived_column()
_migrate_add_last_message_at_column() _migrate_add_last_message_at_column()
_migrate_add_folder_column() _migrate_add_folder_column()
+3 -37
View File
@@ -3,15 +3,10 @@
import os import os
import secrets import secrets
from collections.abc import Mapping
from fastapi import HTTPException, Request from fastapi import HTTPException, Request
from starlette.middleware.base import BaseHTTPMiddleware from starlette.middleware.base import BaseHTTPMiddleware
from starlette.responses import Response from starlette.responses import Response
from starlette.routing import get_route_path
from src.owner_identity import INTERNAL_TOOL_USER, auth_disabled
from src.auth_helpers import is_bearer_principal
# Per-process token that lets the in-app tool layer hit admin-gated # Per-process token that lets the in-app tool layer hit admin-gated
@@ -20,30 +15,8 @@ from src.auth_helpers import is_bearer_principal
# same value from this module. Never persisted or exposed externally. # same value from this module. Never persisted or exposed externally.
INTERNAL_TOOL_TOKEN = os.environ.get("ODYSSEUS_INTERNAL_TOKEN") or secrets.token_hex(32) INTERNAL_TOOL_TOKEN = os.environ.get("ODYSSEUS_INTERNAL_TOKEN") or secrets.token_hex(32)
INTERNAL_TOOL_HEADER = "X-Odysseus-Internal-Token" INTERNAL_TOOL_HEADER = "X-Odysseus-Internal-Token"
# Pseudo-username on in-process tool-loopback requests; require_admin trusts it and it is reserved.
INTERNAL_TOOL_USER = "internal-tool"
def get_application_route_path(scope: Mapping[str, object]) -> str:
"""Return the application-relative path used by Starlette routing.
Uvicorn prefixes ``scope["path"]`` with a configured ASGI ``root_path``;
Starlette removes that prefix before matching routes. Middleware policy
must use the same path form or a deployment prefix can change which policy
applies to an otherwise unchanged application route.
"""
return get_route_path(scope)
def with_asgi_root_path(scope: Mapping[str, object], path: str) -> str:
"""Prefix an application path for a client-facing redirect target."""
root_path = scope.get("root_path", "")
if not isinstance(root_path, str) or not root_path:
return path
return f"{root_path.rstrip('/')}{path}"
def path_is_route_or_child(path: str, prefix: str) -> bool:
"""Return whether ``path`` is exactly ``prefix`` or below that route."""
return path == prefix or path.startswith(prefix + "/")
def is_cors_preflight(method: str, headers) -> bool: def is_cors_preflight(method: str, headers) -> bool:
@@ -60,13 +33,6 @@ def require_admin(request: Request):
Allows access when auth is explicitly disabled, or when the request carries Allows access when auth is explicitly disabled, or when the request carries
the in-process internal-tool token used by loopback agent tools. the in-process internal-tool token used by loopback agent tools.
""" """
# A bearer principal never inherits admin authority, even when the token
# carries a legacy cookbook scope or auth is disabled in a direct-entry
# test. Host-control routes use this centralized gate, so rejecting here
# covers shell, model-serving, MCP, runtime, and other admin surfaces.
if is_bearer_principal(request):
raise HTTPException(403, "API tokens cannot use admin host-control surfaces")
# In-process bypass for tool-layer loopback calls. Two paths: # In-process bypass for tool-layer loopback calls. Two paths:
# (a) header-direct (caller set X-Odysseus-Internal-Token), or # (a) header-direct (caller set X-Odysseus-Internal-Token), or
# (b) the auth middleware already validated the token and stamped # (b) the auth middleware already validated the token and stamped
@@ -81,7 +47,7 @@ def require_admin(request: Request):
pass pass
auth_mgr = getattr(request.app.state, "auth_manager", None) auth_mgr = getattr(request.app.state, "auth_manager", None)
if auth_disabled(): if os.getenv("AUTH_ENABLED", "true").lower() == "false":
return return
if not auth_mgr or not auth_mgr.is_configured: if not auth_mgr or not auth_mgr.is_configured:
raise HTTPException(403, "Admin only") raise HTTPException(403, "Admin only")
+5 -56
View File
@@ -8,12 +8,6 @@ These are simple datacontainers. All persistence is handled by SessionManager.
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, List, Any, Optional, TYPE_CHECKING from typing import Dict, List, Any, Optional, TYPE_CHECKING
from src.tool_approval_scopes import (
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
)
from src.message_metadata import sanitize_projected_message_metadata
from src.tool_approval_provenance import has_chat_session_approval_grant
if TYPE_CHECKING: if TYPE_CHECKING:
from .session_manager import SessionManager from .session_manager import SessionManager
@@ -37,18 +31,6 @@ set_session_manager = set_session_manager_instance
get_session_manager = get_session_manager_instance get_session_manager = get_session_manager_instance
def _history_grants_chat_session_approval(
history: List["ChatMessage"],
session_id: str,
) -> bool:
"""Compatibility shim: durable history is never an authority source.
Keep the old private symbol for downstream imports, but deliberately return
false. The live projection checks the separate server-owned grant table.
"""
return False
@dataclass @dataclass
class ChatMessage: class ChatMessage:
"""A single chat message.""" """A single chat message."""
@@ -90,8 +72,6 @@ class Session:
headers: Optional[Dict[str, str]] = None headers: Optional[Dict[str, str]] = None
history: List[ChatMessage] = None history: List[ChatMessage] = None
owner: Optional[str] = None owner: Optional[str] = None
model_endpoint_id: Optional[str] = None
endpoint_provenance: Optional[str] = None
is_important: bool = False is_important: bool = False
message_count: int = 0 message_count: int = 0
@@ -136,42 +116,11 @@ class Session:
the model. Display/history-load paths use the raw ``history`` and are the model. Display/history-load paths use the raw ``history`` and are
unaffected. unaffected.
""" """
messages = [] return [
for msg in self.history: msg.to_dict()
raw_metadata = getattr(msg, "metadata", None) for msg in self.history
if isinstance(raw_metadata, dict) and raw_metadata.get("source") == "slash": if (msg.metadata or {}).get("source") != "slash"
continue ]
projected = msg.to_dict()
metadata = projected.get("metadata")
if not isinstance(metadata, dict):
# Old or malformed durable rows must not make context
# projection fail, and non-mapping metadata has no trusted
# fields that belong in the model context.
projected.pop("metadata", None)
messages.append(projected)
continue
metadata = sanitize_projected_message_metadata(metadata)
if metadata:
projected["metadata"] = metadata
else:
projected.pop("metadata", None)
messages.append(projected)
if not has_chat_session_approval_grant(self.id, self.owner):
return messages
# Keep the grant close to the latest user request so route-neutral
# compaction/trimming preserves it. Copy the metadata instead of
# mutating the durable transcript object.
for index in range(len(messages) - 1, -1, -1):
if messages[index].get("role") != "user":
continue
message = dict(messages[index])
metadata = dict(message.get("metadata") or {})
metadata[CHAT_SESSION_APPROVAL_CONTEXT_MARKER] = True
message["metadata"] = metadata
messages[index] = message
break
return messages
def get(self, key: str, default=None): def get(self, key: str, default=None):
"""Dict-like access for compatibility.""" """Dict-like access for compatibility."""
+5 -89
View File
@@ -14,8 +14,6 @@ import logging
from datetime import datetime, timezone, timedelta from datetime import datetime, timezone, timedelta
from typing import Dict, Optional from typing import Dict, Optional
from sqlalchemy import func
from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive
from .models import Session, ChatMessage from .models import Session, ChatMessage
from src.attachment_refs import persistable_message_content from src.attachment_refs import persistable_message_content
@@ -62,22 +60,6 @@ def _parse_msg_content(raw):
return raw return raw
def _parse_message_metadata(raw) -> dict:
"""Decode only JSON objects from durable message metadata.
Legacy rows may contain a JSON list (including list-of-pairs) or another
scalar. Such values have no trusted message fields and must not reach the
``_db_id``/timestamp merge below or any approval projection.
"""
if not raw:
return {}
try:
parsed = json.loads(raw) if isinstance(raw, str) else raw
except (json.JSONDecodeError, TypeError, ValueError):
return {}
return dict(parsed) if isinstance(parsed, dict) else {}
class SessionManager: class SessionManager:
""" """
Manages chat sessions with database persistence. Manages chat sessions with database persistence.
@@ -110,28 +92,14 @@ class SessionManager:
try: try:
db_sessions = db.query(DbSession).filter( db_sessions = db.query(DbSession).filter(
DbSession.archived == False, DbSession.archived == False,
DbSession.messages.any(), DbSession.message_count > 0,
).order_by(DbSession.last_accessed.desc()).limit(100).all() ).order_by(DbSession.last_accessed.desc()).limit(100).all()
# message_count is derived metadata and can drift after interrupted
# or legacy writes. Count only the bounded discovery set so startup
# remains metadata-only while lazy hydration sees an authoritative
# positive count for every discovered non-empty session.
message_counts = {}
if db_sessions:
message_counts = dict(
db.query(DbChatMessage.session_id, func.count(DbChatMessage.id))
.filter(DbChatMessage.session_id.in_([row.id for row in db_sessions]))
.group_by(DbChatMessage.session_id)
.all()
)
loaded_count = 0 loaded_count = 0
for db_session in db_sessions: for db_session in db_sessions:
try: try:
session = self._db_to_session_meta(db_session) session = self._db_to_session_meta(db_session)
if session is not None: if session is not None:
session.message_count = message_counts[db_session.id]
self.sessions[db_session.id] = session self.sessions[db_session.id] = session
loaded_count += 1 loaded_count += 1
except Exception as e: except Exception as e:
@@ -165,8 +133,6 @@ class SessionManager:
headers=headers, headers=headers,
history=[], history=[],
owner=getattr(db_session, "owner", None), owner=getattr(db_session, "owner", None),
model_endpoint_id=getattr(db_session, "model_endpoint_id", None),
endpoint_provenance=getattr(db_session, "endpoint_provenance", None),
is_important=getattr(db_session, "is_important", False) or False, is_important=getattr(db_session, "is_important", False) or False,
) )
session.message_count = getattr(db_session, "message_count", 0) or 0 session.message_count = getattr(db_session, "message_count", 0) or 0
@@ -179,7 +145,8 @@ class SessionManager:
# Try relationship first, then direct query # Try relationship first, then direct query
if db_session.messages: if db_session.messages:
for db_msg in db_session.messages: for db_msg in db_session.messages:
meta = _parse_message_metadata(db_msg.meta_data) meta = json.loads(db_msg.meta_data) if db_msg.meta_data else {}
if meta is None: meta = {}
meta['_db_id'] = db_msg.id meta['_db_id'] = db_msg.id
meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp)) meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp))
history.append(ChatMessage( history.append(ChatMessage(
@@ -193,7 +160,8 @@ class SessionManager:
).order_by(DbChatMessage.timestamp).all() ).order_by(DbChatMessage.timestamp).all()
for db_msg in db_messages: for db_msg in db_messages:
meta = _parse_message_metadata(db_msg.meta_data) meta = json.loads(db_msg.meta_data) if db_msg.meta_data else {}
if meta is None: meta = {}
meta['_db_id'] = db_msg.id meta['_db_id'] = db_msg.id
meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp)) meta.setdefault('timestamp', _message_timestamp_iso(db_msg.timestamp))
history.append(ChatMessage( history.append(ChatMessage(
@@ -223,8 +191,6 @@ class SessionManager:
headers=headers, headers=headers,
history=history, history=history,
owner=getattr(db_session, 'owner', None), owner=getattr(db_session, 'owner', None),
model_endpoint_id=getattr(db_session, 'model_endpoint_id', None),
endpoint_provenance=getattr(db_session, 'endpoint_provenance', None),
is_important=getattr(db_session, 'is_important', False) or False, is_important=getattr(db_session, 'is_important', False) or False,
) )
@@ -272,8 +238,6 @@ class SessionManager:
logger.warning("Dropping message for deleted session %s", session_id) logger.warning("Dropping message for deleted session %s", session_id)
return return
if not isinstance(message.metadata, dict):
message.metadata = None
missing_upload_id = reserve_message_upload_references( missing_upload_id = reserve_message_upload_references(
getattr(self, "upload_handler", None), getattr(self, "upload_handler", None),
getattr(db_session, "owner", None), getattr(db_session, "owner", None),
@@ -386,8 +350,6 @@ class SessionManager:
# ownership check/access touch and the replacement transaction. # ownership check/access touch and the replacement transaction.
# A failed reservation must leave the existing transcript intact. # A failed reservation must leave the existing transcript intact.
for message in messages: for message in messages:
if not isinstance(message.metadata, dict):
message.metadata = None
missing_upload_id = reserve_message_upload_references( missing_upload_id = reserve_message_upload_references(
getattr(self, "upload_handler", None), getattr(self, "upload_handler", None),
getattr(db_session, "owner", None), getattr(db_session, "owner", None),
@@ -506,8 +468,6 @@ class SessionManager:
session.rag = db_session.rag session.rag = db_session.rag
session.archived = db_session.archived session.archived = db_session.archived
session.owner = getattr(db_session, "owner", None) session.owner = getattr(db_session, "owner", None)
session.model_endpoint_id = getattr(db_session, "model_endpoint_id", None)
session.endpoint_provenance = getattr(db_session, "endpoint_provenance", None)
session.is_important = getattr(db_session, "is_important", False) or False session.is_important = getattr(db_session, "is_important", False) or False
session.message_count = ( session.message_count = (
db.query(DbChatMessage) db.query(DbChatMessage)
@@ -608,50 +568,6 @@ class SessionManager:
finally: finally:
db.close() db.close()
def set_session_endpoint_provenance(
self,
session_id: str,
*,
model_endpoint_id: Optional[str],
endpoint_provenance: str,
) -> bool:
"""Persist the server-owned endpoint provenance for a session.
``registered`` rows carry an exact ModelEndpoint id. ``direct`` rows
deliberately carry no endpoint id and retain direct API-key
compatibility. The values are assigned only after the durable write
succeeds so an in-memory session cannot claim provenance the database
did not accept.
"""
provenance = str(endpoint_provenance or "").strip().lower()
endpoint_id = str(model_endpoint_id or "").strip() or None
if provenance == "registered" and not endpoint_id:
raise ValueError("registered session provenance requires an endpoint id")
if provenance == "direct":
endpoint_id = None
if provenance not in {"registered", "direct"}:
raise ValueError("unsupported session endpoint provenance")
db = SessionLocal()
try:
db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
if db_session is None:
raise KeyError(f"Session {session_id} not found")
db_session.model_endpoint_id = endpoint_id
db_session.endpoint_provenance = provenance
db.commit()
except Exception:
db.rollback()
raise
finally:
db.close()
session = self.sessions.get(session_id)
if session is not None:
session.model_endpoint_id = endpoint_id
session.endpoint_provenance = provenance
return True
def delete_session(self, session_id: str) -> bool: def delete_session(self, session_id: str) -> bool:
"""Permanently delete a session and all its messages.""" """Permanently delete a session and all its messages."""
db = SessionLocal() db = SessionLocal()
+1 -12
View File
@@ -46,11 +46,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -75,11 +74,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -135,17 +129,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
+1 -12
View File
@@ -45,11 +45,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -74,11 +73,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -138,17 +132,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
+1 -12
View File
@@ -34,11 +34,10 @@ services:
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db} - DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
- AUTH_ENABLED=${AUTH_ENABLED:-true} - AUTH_ENABLED=${AUTH_ENABLED:-true}
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false} - LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin} - ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-} - ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1} - ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
- SECURE_COOKIES=${SECURE_COOKIES:-} - SECURE_COOKIES=${SECURE_COOKIES:-false}
- EMBEDDING_URL=${EMBEDDING_URL:-} - EMBEDDING_URL=${EMBEDDING_URL:-}
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-} - EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
@@ -63,11 +62,6 @@ services:
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-} - GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-} - GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-} - GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
# Externally reachable origin for MCP OAuth callbacks. The container
# always listens on 7000 and cannot see the host port map above, so
# remote MCP OAuth needs this set whenever the browser reaches
# Odysseus on anything other than http://localhost:7000.
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
- TAVILY_API_KEY=${TAVILY_API_KEY:-} - TAVILY_API_KEY=${TAVILY_API_KEY:-}
- SERPER_API_KEY=${SERPER_API_KEY:-} - SERPER_API_KEY=${SERPER_API_KEY:-}
# PUID / PGID — the user/group the container drops to before # PUID / PGID — the user/group the container drops to before
@@ -116,17 +110,12 @@ services:
fi fi
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
fi fi
# Advisory: a settings file the migration cannot parse or rewrite must
# not be what stops searxng from booting. It explains itself on stderr
# and we carry on, letting searxng report anything genuinely wrong.
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
exec /usr/local/searxng/entrypoint.sh exec /usr/local/searxng/entrypoint.sh
ports: ports:
- "127.0.0.1:8080:8080" - "127.0.0.1:8080:8080"
volumes: volumes:
- searxng-data:/etc/searxng - searxng-data:/etc/searxng
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z - ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
environment: environment:
- SEARXNG_BASE_URL=http://localhost:8080/ - SEARXNG_BASE_URL=http://localhost:8080/
- SEARXNG_SECRET=${SEARXNG_SECRET:-} - SEARXNG_SECRET=${SEARXNG_SECRET:-}
@@ -1,7 +1,3 @@
---
layout: default
---
# Agent migration manifests # Agent migration manifests
Odysseus should be able to learn from another agent without blindly trusting Odysseus should be able to learn from another agent without blindly trusting
@@ -1,7 +1,3 @@
---
layout: default
---
# Attachment References and Upload Storage # Attachment References and Upload Storage
Odysseus stores uploaded bytes once under the configured upload directory and Odysseus stores uploaded bytes once under the configured upload directory and
@@ -1,7 +1,3 @@
---
layout: default
---
# Backup & Restore # Backup & Restore
Odysseus keeps all of your state in the `data/` directory — the SQLite database Odysseus keeps all of your state in the `data/` directory — the SQLite database
View File
@@ -1,7 +1,3 @@
---
layout: default
---
# Outlook / Office 365 email accounts # Outlook / Office 365 email accounts
Odysseus email accounts currently use IMAP and SMTP with username/password Odysseus email accounts currently use IMAP and SMTP with username/password

Before

Width:  |  Height:  |  Size: 185 KiB

After

Width:  |  Height:  |  Size: 185 KiB

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 79 KiB

@@ -1,7 +1,3 @@
---
layout: default
---
# PR Blocker Audit # PR Blocker Audit
`scripts/pr_blocker_audit.py` is a small, read-only triage helper for maintainers who need to inspect open pull request overlap before reviewing or starting related work. `scripts/pr_blocker_audit.py` is a small, read-only triage helper for maintainers who need to inspect open pull request overlap before reviewing or starting related work.
@@ -1,7 +1,3 @@
---
layout: default
---
# Security CI guide # Security CI guide
This project runs a set of automated security checks on pull requests and This project runs a set of automated security checks on pull requests and
+7 -182
View File
@@ -1,7 +1,3 @@
---
layout: default
---
# Odysseus Setup Guide # Odysseus Setup Guide
This page keeps the detailed install, deployment, troubleshooting, and configuration notes out of the front README. This page keeps the detailed install, deployment, troubleshooting, and configuration notes out of the front README.
@@ -19,7 +15,8 @@ On first setup, Odysseus creates an admin account (`admin` unless
For Docker installs, the same line is in `docker compose logs odysseus`. For Docker installs, the same line is in `docker compose logs odysseus`.
Use that for the first login, then change it in **Settings**. Use that for the first login, then change it in **Settings**.
Contributing? See [CONTRIBUTING.md](https://github.com/odysseus-dev/odysseus/blob/dev/CONTRIBUTING.md) for setup, testing, and pull request guidelines. Contributing? See [CONTRIBUTING.md](../CONTRIBUTING.md) for setup, testing, and
pull request guidelines.
### Docker (recommended) ### Docker (recommended)
```bash ```bash
@@ -208,11 +205,9 @@ failed to fulfil mount request: open /usr/lib/wsl/lib/libdxcore.so: no such file
Check with `snap list docker` or: Check with `snap list docker` or:
<!-- {% raw %} -->
```bash ```bash
docker info --format '{{.DockerRootDir}}' docker info --format '{{.DockerRootDir}}'
``` ```
<!-- {% endraw %} -->
A Docker root under `/var/snap/docker/` means snap confinement can prevent A Docker root under `/var/snap/docker/` means snap confinement can prevent
Docker from seeing WSL2's `/usr/lib/wsl/lib` GPU libraries even when the files Docker from seeing WSL2's `/usr/lib/wsl/lib` GPU libraries even when the files
@@ -446,19 +441,10 @@ A grab-bag of small gotchas that otherwise turn into long debugging sessions.
| Package | Feature unlocked | | Package | Feature unlocked |
|---------|-----------------| |---------|-----------------|
| `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. | | `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
| `kokoro`, `soundfile` | Local Kokoro-82M text-to-speech on a CUDA GPU. The pinned Kokoro release supports Odysseus installs on Python 3.11-3.12; these packages are intentionally skipped on Python 3.13+ (including the Python 3.14 container image). |
| `ddgs` | DuckDuckGo as a search provider option. | | `ddgs` | DuckDuckGo as a search provider option. |
| `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) | | `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
| `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). | | `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
Install the optional set only when you need these features:
```bash
pip install -r requirements-optional.txt
```
The default Docker image currently uses Python 3.14, while Kokoro 0.9.4 declares Python `>=3.10,<3.13`. Odysseus itself continues to support Python 3.11+, but this pinned optional local-TTS feature requires a native Python 3.11 or 3.12 environment. Kokoro declares `torch`, but the local provider only activates when that torch build has CUDA and a GPU is visible; install the CUDA build appropriate for your host. Browser and configured endpoint TTS remain available on Python 3.13+ and in the container image.
### Faster, reproducible installs with uv (optional) ### Faster, reproducible installs with uv (optional)
[uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the [uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use: venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
@@ -481,7 +467,7 @@ uv pip sync requirements.lock # reproduce it exactly la
### Outlook / Office 365 email ### Outlook / Office 365 email
Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
passwords will fail. See [the Outlook email guide](email-outlook.md) for the passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
current limitation and the planned integration direction. current limitation and the planned integration direction.
## Security Notes ## Security Notes
@@ -489,7 +475,7 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep `AUTH_ENABLED=true` for any network-accessible deployment. - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
- Keep `LOCALHOST_BYPASS=false` outside local development. - Keep `LOCALHOST_BYPASS=false` outside local development.
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Use `SECURE_COOKIES=true` to force it on for a proxy whose scheme Odysseus cannot see, or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS. - Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
- Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer. - Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
- Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default. - Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
- Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin. - Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
@@ -500,14 +486,6 @@ Odysseus is a self-hosted workspace with powerful local tools: shell access, fil
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer. - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
- Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged. - Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
> **Upgrading an existing install:** `SECURE_COOKIES` used to default to
> `false`, so an install set up before scheme derivation may still carry
> `SECURE_COOKIES=false` in its own `.env`. That explicit value stays
> authoritative, so HTTPS logins keep getting a non-`Secure` session cookie.
> Pulling this change updates the tracked Compose files, but nothing rewrites
> your `.env` — drop the line from it unless you deliberately serve plain HTTP
> alongside HTTPS and want the escape hatch.
### Private or proxied deployments ### Private or proxied deployments
Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is: Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
@@ -516,162 +494,9 @@ Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and th
3. Put the authenticated Odysseus web/API entrypoint behind that layer. 3. Put the authenticated Odysseus web/API entrypoint behind that layer.
4. Keep raw service and model ports internal-only. 4. Keep raw service and model ports internal-only.
Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true` and `LOCALHOST_BYPASS=false`. Any proxy that forwards `X-Forwarded-Proto: https` gets `Secure` session cookies without configuration, so `SECURE_COOKIES` only needs setting when you want to override that — force it on for a proxy that forwards no scheme at all, or off while you still serve plain HTTP. Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
`ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry. `ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
#### Faster over the network: HTTP/2
The frontend is raw ES modules with no bundler, so a page load is a few hundred
small same-origin requests. Over HTTP/1.1 browsers typically allow only a small
number of concurrent connections per host (commonly around six), so many of
those requests are serialized across multiple round trips. On localhost that
costs almost nothing. Over a LAN, VPN, or remote link it can become a major
part of load time, especially as latency increases.
HTTP/2 multiplexes them onto one connection and the serialisation disappears.
Odysseus needs no changes for this — uvicorn keeps speaking HTTP/1.1 on
loopback and the proxy speaks HTTP/2 to the browser. Mainstream browsers
negotiate HTTP/2 for normal web pages over TLS; they do not use the cleartext
h2c mode here, so browser-facing HTTP/2 requires a certificate. The
`--ssl-certfile` route in *HTTPS + LAN/Tailscale exposure* above gives you
HTTPS but not HTTP/2 — uvicorn does not speak it.
**1. Install Caddy.** See the [install docs](https://caddyserver.com/docs/install)
for your platform; on macOS, `brew install caddy`.
**2. Write a `Caddyfile`.** Pick the block that matches how you reach the
machine. Replace `7000` if Odysseus listens elsewhere — the macOS start script
uses `7860`.
Public domain, Caddy obtains and renews the certificate itself:
```
odysseus.example.com {
reverse_proxy 127.0.0.1:7000
}
```
Tailscale, no public DNS needed — `tailscale cert` issues a browser-trusted
certificate for a tailnet name and writes `<domain>.crt` and `<domain>.key`:
```bash
tailscale cert myhost.tailnet-name.ts.net
```
```
myhost.tailnet-name.ts.net {
tls /path/to/myhost.tailnet-name.ts.net.crt /path/to/myhost.tailnet-name.ts.net.key
reverse_proxy 127.0.0.1:7000
}
```
LAN with your own certificate — same shape, your own files:
```
odysseus.lan {
tls /path/to/cert.pem /path/to/key.pem
reverse_proxy 127.0.0.1:7000
}
```
Give `tls` absolute paths: a service starts in a working directory you did not
choose. If port 443 is already taken, append a port to the site address
(`odysseus.example.com:8443`) and use it in the URL. That alone does not free
port 80 — Caddy still binds it for the HTTP-to-HTTPS redirect, and fails to
start with `listen tcp :80: bind: address already in use` if something else
holds it. Turn the redirect off with a global block at the top of the file:
```
{
auto_https disable_redirects
}
```
**3. Run it in the foreground first:**
```bash
caddy run --config ./Caddyfile
```
Once that works, run it as a service:
```bash
brew services start caddy # macOS — reads $(brew --prefix)/etc/Caddyfile, not ./Caddyfile
sudo systemctl enable --now caddy # Linux, if your package installed the unit
```
Odysseus's own service is unchanged; the proxy runs alongside it. Under Docker,
run the proxy as another container, or on the host pointing at the published
port.
**4. Point Odysseus at the new origin** in `.env`, then restart it.
A proxy that exposes the HTTPS request scheme to Odysseus needs no `SECURE_COOKIES` setting. Only force it on when the proxy cannot expose that scheme:
```bash
# only if the proxy cannot expose the external HTTPS scheme to Odysseus:
SECURE_COOKIES=true
# only if you use remote MCP servers with OAuth:
OAUTH_REDIRECT_BASE_URL=https://odysseus.example.com
```
Gmail OAuth needs nothing here when the proxy runs on the same host: the
redirect URI is built from the incoming request, and uvicorn rewrites the
scheme from `X-Forwarded-Proto` for proxies it trusts — by default only
`127.0.0.1`. A proxy in a separate container or on another machine is not
trusted, so pin the URI there:
```bash
GOOGLE_OAUTH_REDIRECT_URI=https://odysseus.example.com/api/email/oauth/google/callback
```
(uvicorn's own `FORWARDED_ALLOW_IPS` widens that trust, but it has to be in the
environment uvicorn starts with — `.env` is read by the app afterwards, too
late for it to take effect.)
**5. Confirm HTTP/2 is really on:**
```bash
curl -s -o /dev/null -w '%{http_version}\n' https://odysseus.example.com/
# 2
```
The status code is not the thing to check here — a logged-out request redirects
to the login page, so `curl -I` shows `HTTP/2 302`, and the `HTTP/2` prefix is
the part that matters. The browser reports the same in the Network panel's
Protocol column (`h2`); in Chrome and Firefox that column is hidden until you
enable it by right-clicking the column headers.
Three things bite when moving an existing install behind TLS:
- Leave `SECURE_COOKIES` unset when Odysseus can see the external HTTPS scheme;
the cookie then follows the request automatically. If your proxy cannot expose
that scheme, set `SECURE_COOKIES=true` **at the same time** you stop serving
plain HTTP, not before. An explicit `true` applies to every login, so while an
HTTP entrypoint is still reachable the browser will reject the `Secure` cookie
there and login will appear to loop.
- `OAUTH_REDIRECT_BASE_URL` defaults to `http://localhost:7000`. Unlike the
Gmail redirect URI it cannot be derived from a request — it is registered
with each MCP authorization server up front — so set it to the external
origin if you use remote MCP servers over OAuth.
- Odysseus sends `Strict-Transport-Security` once it sees `X-Forwarded-Proto:
https`. HSTS applies to the whole hostname and ignores the port, so any other
plain-HTTP service on that same hostname becomes unreachable in browsers that
have visited Odysseus. Give Odysseus its own hostname, or strip the header at
the proxy (`header_down -Strict-Transport-Security` in Caddy).
Server-sent events are not buffered by this configuration, so chat streaming
arrives token by token; add `flush_interval -1` inside the `reverse_proxy`
block if you want that pinned explicitly. nginx needs `proxy_buffering off;`
for the same reason.
Changing the external origin also affects state scoped to it. Service workers
and their caches are origin-scoped, so moving to a different origin starts with
a cold load. Cookies follow their own domain/path/security rules rather than
being port-scoped: changing the hostname normally requires a new login, while
changing only the scheme or port does not by itself guarantee that existing
cookies disappear.
Common internal-only ports from the default docs/compose setup: Common internal-only ports from the default docs/compose setup:
| Port | Service | | Port | Service |
@@ -702,7 +527,7 @@ Key settings:
| `AUTH_ENABLED` | `true` | Enable/disable login | | `AUTH_ENABLED` | `true` | Enable/disable login |
| `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. | | `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
| `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. | | `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
| `SECURE_COOKIES` | derived from the request scheme | Marks session cookies `Secure` on HTTPS requests. Set true to force it on, false to force it off. | | `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
| `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string | | `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
| `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. | | `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
| `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. | | `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
@@ -738,7 +563,7 @@ src/ llm_core, agent_loop, agent_tools, chat_processor, search/
routes/ chat, session, document, memory, model … endpoints routes/ chat, session, document, memory, model … endpoints
services/ docs, memory, search, hwfit (Cookbook) … services/ docs, memory, search, hwfit (Cookbook) …
static/ index.html + app.js + style.css + js/ (modular front-end) static/ index.html + app.js + style.css + js/ (modular front-end)
website/ landing page (index.html) + preview clips docs/ landing page (index.html) + preview clips
``` ```
## Data ## Data
-4
View File
@@ -163,10 +163,6 @@ if (Test-Path $cudaBase) {
} }
# 7. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH) # 7. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH)
# -Port only reaches uvicorn as a flag. Everything that builds a URL for this
# instance - internal_api_base(), companion pairing, the MCP OAuth callback -
# reads APP_PORT, so set it too or they all assume 7000.
$env:APP_PORT = $Port
Write-Step ("Starting Odysseus at http://{0}:{1}" -f $BindHost, $Port) Write-Step ("Starting Odysseus at http://{0}:{1}" -f $BindHost, $Port)
Write-Host "Press Ctrl+C to stop." Write-Host "Press Ctrl+C to stop."
Write-Host "" Write-Host ""
-21
View File
@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2013-2020 Khan Academy and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-21
View File
@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2014 - 2022 Knut Sveidqvist
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+8
View File
@@ -1802,6 +1802,7 @@ async def _ai_draft_reply_to_email(uid, folder="INBOX", reply_all=False, account
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_endpoint, resolve_endpoint,
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
except Exception as exc: except Exception as exc:
@@ -1842,6 +1843,13 @@ async def _ai_draft_reply_to_email(uid, folder="INBOX", reply_all=False, account
utility_fallbacks = resolve_utility_fallback_candidates() or [] utility_fallbacks = resolve_utility_fallback_candidates() or []
for cand in utility_fallbacks: for cand in utility_fallbacks:
_add(*cand) _add(*cand)
try:
chat_fallbacks = resolve_chat_fallback_candidates(owner=None) or []
except TypeError:
chat_fallbacks = resolve_chat_fallback_candidates() or []
for cand in chat_fallbacks:
_add(*cand)
if not candidates: if not candidates:
return {"error": "No LLM endpoint configured for AI reply"} return {"error": "No LLM endpoint configured for AI reply"}
+4 -4
View File
@@ -5,13 +5,13 @@
"packages": { "packages": {
"": { "": {
"devDependencies": { "devDependencies": {
"@antithesishq/bombadil": "^0.7.0" "@antithesishq/bombadil": "^0.6.1"
} }
}, },
"node_modules/@antithesishq/bombadil": { "node_modules/@antithesishq/bombadil": {
"version": "0.7.0", "version": "0.6.1",
"resolved": "https://registry.npmjs.org/@antithesishq/bombadil/-/bombadil-0.7.0.tgz", "resolved": "https://registry.npmjs.org/@antithesishq/bombadil/-/bombadil-0.6.1.tgz",
"integrity": "sha512-alJmnphJ/iUoL5mCsnV3DwtajGy/sEQ3NJJCiMhgjqXshSq2BUtAs0vqdXEiiSkB8HbsOX5CLrAcaogYdwfAJg==", "integrity": "sha512-d1iufG3MI7gSMSiSmMeNdcMW+qR0yQXL2zdkVynC3n3DYgFJYlYXKUQzygmqU12m4RWlR5iOdQU1hsx5UT6+IA==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"bin": { "bin": {
+1 -1
View File
@@ -4,6 +4,6 @@
"url": "https://github.com/odysseus-dev/odysseus.git" "url": "https://github.com/odysseus-dev/odysseus.git"
}, },
"devDependencies": { "devDependencies": {
"@antithesishq/bombadil": "^0.7.0" "@antithesishq/bombadil": "^0.6.1"
} }
} }
-10
View File
@@ -12,16 +12,6 @@
# GPU-accelerated transcription — it's auto-detected, CPU is used otherwise. # GPU-accelerated transcription — it's auto-detected, CPU is used otherwise.
faster-whisper faster-whisper
# Local text-to-speech via Kokoro-82M for the "local" TTS provider.
# Kokoro 0.9.4 declares Python >=3.10,<3.13; Odysseus itself requires 3.11+,
# so pip installs these extras on 3.11-3.12 and deliberately skips them on
# Python 3.13+ (including the Python 3.14 container image). Kokoro declares
# torch; the local provider still
# requires a CUDA-enabled torch build and GPU at runtime. SoundFile is separate
# in Kokoro's official install instructions and is not a transitive dependency.
kokoro==0.9.4; python_version >= "3.11" and python_version < "3.13"
soundfile; python_version >= "3.11" and python_version < "3.13"
# DuckDuckGo as a search provider option. # DuckDuckGo as a search provider option.
# Install if you want DDG in the search-provider dropdown. # Install if you want DDG in the search-provider dropdown.
# Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE. # Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE.
+7 -12
View File
@@ -11,12 +11,12 @@ import json
from datetime import datetime from datetime import datetime
from typing import Optional from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Request from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel from pydantic import BaseModel
from core.database import SessionLocal, CrewMember, ScheduledTask from core.database import SessionLocal, CrewMember, ScheduledTask
from src.auth_helpers import require_interactive_request from src.auth_helpers import get_current_user
from src.owner_identity import REQUEST_SENTINEL_OWNERS from core.auth import RESERVED_USERNAMES
from src.task_scheduler import compute_next_run from src.task_scheduler import compute_next_run
@@ -78,14 +78,10 @@ def _task_to_checkin_dict(t: ScheduledTask) -> dict:
def setup_assistant_routes(task_scheduler) -> APIRouter: def setup_assistant_routes(task_scheduler) -> APIRouter:
router = APIRouter( router = APIRouter(prefix="/api/assistant", tags=["assistant"])
prefix="/api/assistant",
tags=["assistant"],
dependencies=[Depends(require_interactive_request)],
)
def _owner(request: Request) -> str: def _owner(request: Request) -> str:
owner = require_interactive_request(request) owner = get_current_user(request)
if not owner: if not owner:
raise HTTPException(status_code=401, detail="Not authenticated") raise HTTPException(status_code=401, detail="Not authenticated")
return owner return owner
@@ -94,12 +90,11 @@ def setup_assistant_routes(task_scheduler) -> APIRouter:
# check-in tasks seeded. Hitting any /assistant route under one of these # check-in tasks seeded. Hitting any /assistant route under one of these
# used to seed a full CrewMember + Morning/Midday/Evening tasks under that # used to seed a full CrewMember + Morning/Midday/Evening tasks under that
# owner, which then double-fired alongside the real user's check-ins. # owner, which then double-fired alongside the real user's check-ins.
# REQUEST_SENTINEL_OWNERS covers request-only identities; Default/Local is a # RESERVED_USERNAMES covers the same set; the `not owner` guard handles "".
# reserved login name but remains a valid storage owner.
async def _get_or_create(owner: str) -> CrewMember: async def _get_or_create(owner: str) -> CrewMember:
"""Return the per-owner assistant CrewMember, creating it on demand.""" """Return the per-owner assistant CrewMember, creating it on demand."""
if not owner or owner in REQUEST_SENTINEL_OWNERS: if not owner or owner in RESERVED_USERNAMES:
raise HTTPException(status_code=400, detail=f"Cannot seed assistant for {owner!r}") raise HTTPException(status_code=400, detail=f"Cannot seed assistant for {owner!r}")
db = SessionLocal() db = SessionLocal()
try: try:
+3 -34
View File
@@ -22,8 +22,6 @@ from src.settings import (
load_features as _load_features, load_features as _load_features,
save_features as _save_features, save_features as _save_features,
DEFAULT_SETTINGS, DEFAULT_SETTINGS,
RETIRED_SETTING_KEYS,
without_retired_settings,
) )
from src.integrations import ( from src.integrations import (
load_integrations, load_integrations,
@@ -86,33 +84,6 @@ class SetOpenRegistrationRequest(BaseModel):
SESSION_COOKIE = "odysseus_session" SESSION_COOKIE = "odysseus_session"
def _secure_cookie(request: Request) -> bool:
"""Decide the ``Secure`` attribute of the session cookie.
``SECURE_COOKIES`` stays authoritative when it holds an explicit value:
``true`` always marks the cookie Secure (the documented knob for a TLS
proxy), ``false`` never does, which is the escape hatch for an install
that still answers on plain HTTP alongside HTTPS. Anything else
unset, or the present-but-empty value docker-compose injects for a
variable the host has not defined derives it from the request, so an
HTTPS login gets a Secure cookie without any configuration.
Either the connection scheme or ``X-Forwarded-Proto`` saying https is
enough, which is the same test ``core/middleware.py`` applies before it
sends HSTS. Uvicorn's proxy-headers middleware already folds that header
into the scheme for the proxies it trusts, so reading it here only adds
the case of a terminator that is not on a trusted address; the cost is
that a client talking to the app directly can set the header and lock
its own session out over plain HTTP.
"""
configured = os.getenv("SECURE_COOKIES", "").strip().lower()
if configured in ("true", "false"):
return configured == "true"
# A chained proxy sends a list — the client-facing hop comes first.
forwarded_proto = request.headers.get("x-forwarded-proto", "").split(",")[0]
return request.url.scheme == "https" or forwarded_proto.strip().lower() == "https"
def setup_auth_routes(auth_manager: AuthManager) -> APIRouter: def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
router = APIRouter(prefix="/api/auth", tags=["auth"]) router = APIRouter(prefix="/api/auth", tags=["auth"])
@@ -186,7 +157,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
value=token, value=token,
httponly=True, httponly=True,
samesite="lax", samesite="lax",
secure=_secure_cookie(request), secure=os.getenv("SECURE_COOKIES", "false").lower() == "true",
path="/", path="/",
) )
if body.remember: if body.remember:
@@ -718,7 +689,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
a scrubbed copy with secret keys blanked. The frontend uses this a scrubbed copy with secret keys blanked. The frontend uses this
for keybinds + TTS prefs, so it stays callable without admin.""" for keybinds + TTS prefs, so it stays callable without admin."""
user = _get_current_user(request) user = _get_current_user(request)
settings = without_retired_settings(_load_settings()) settings = _load_settings()
if user and auth_manager.is_admin(user): if user and auth_manager.is_admin(user):
return settings return settings
return scrub_settings(settings) return scrub_settings(settings)
@@ -738,8 +709,6 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
"agent_max_tool_calls": (0, 1000), # 0 = unlimited "agent_max_tool_calls": (0, 1000), # 0 = unlimited
} }
for key in DEFAULT_SETTINGS: for key in DEFAULT_SETTINGS:
if key in RETIRED_SETTING_KEYS:
continue
if key not in body: if key not in body:
continue continue
val = body[key] val = body[key]
@@ -752,7 +721,7 @@ def setup_auth_routes(auth_manager: AuthManager) -> APIRouter:
val = max(lo, min(val, hi)) val = max(lo, min(val, hi))
current[key] = val current[key] = val
_save_settings(current) _save_settings(current)
return without_retired_settings(current) return current
# ---- Integrations CRUD ---- # ---- Integrations CRUD ----
+124 -314
View File
@@ -15,14 +15,8 @@ from core.database import Session as DBSession, ModelEndpoint
from src.llm_core import normalize_model_id from src.llm_core import normalize_model_id
from src.endpoint_resolver import normalize_base from src.endpoint_resolver import normalize_base
from src.context_compactor import maybe_compact, trim_for_context from src.context_compactor import maybe_compact, trim_for_context
from src.model_context import estimate_tokens, get_context_length from src.model_context import estimate_tokens
from src.auth_helpers import ( from src.auth_helpers import effective_user
RequestCapability,
effective_user,
is_bearer_principal,
request_capability as build_request_capability,
)
from src.tool_approval_scopes import CHAT_SESSION_APPROVAL_CONTEXT_MARKER
from src.prompt_security import untrusted_context_message from src.prompt_security import untrusted_context_message
from src.attachment_refs import attachment_ref from src.attachment_refs import attachment_ref
from routes.prefs_routes import _load_for_user as load_prefs_for_user from routes.prefs_routes import _load_for_user as load_prefs_for_user
@@ -110,33 +104,6 @@ def _append_incognito_message(session_id: str, role: str, content: Any, metadata
bundle["updated_at"] = time.time() bundle["updated_at"] = time.time()
def _history_for_request_capability(sess, capability: RequestCapability) -> list[dict[str, Any]]:
"""Project persisted history without interactive approval authority for bearers."""
history = sess.get_context_messages()
if not capability.is_bearer:
return history
# Session.get_context_messages() derives the marker only from the separate
# server-owned grant table. A pure bearer chat may still read its owner's
# ordinary transcript, but it must not receive even that interactive
# approval signal as model context or future tool authority.
projected = []
for item in history or []:
if not isinstance(item, dict):
continue
message = dict(item)
metadata = message.get("metadata")
if isinstance(metadata, dict) and CHAT_SESSION_APPROVAL_CONTEXT_MARKER in metadata:
metadata = dict(metadata)
metadata.pop(CHAT_SESSION_APPROVAL_CONTEXT_MARKER, None)
if metadata:
message["metadata"] = metadata
else:
message.pop("metadata", None)
projected.append(message)
return projected
# ── Data containers ────────────────────────────────────────────────────── # # ── Data containers ────────────────────────────────────────────────────── #
@dataclass @dataclass
@@ -185,43 +152,10 @@ class ChatContext:
# Uploads attached to this user turn, resolved and owner-checked for the # Uploads attached to this user turn, resolved and owner-checked for the
# agent's private context. This is not emitted to the browser. # agent's private context. This is not emitted to the browser.
uploaded_files: list = field(default_factory=list) uploaded_files: list = field(default_factory=list)
# Route-neutral prompt before any model-window compaction/trimming. This is
# retained only when explicit foreground fallbacks are enabled so each
# concrete candidate can apply its own context budget independently.
route_messages: list = field(default_factory=list)
# ── Helpers ────────────────────────────────────────────────────────────── # # ── Helpers ────────────────────────────────────────────────────────────── #
def _allowed_models_from_privileges(privs: dict) -> Optional[frozenset[str]]:
if privs.get("block_all_models"):
return frozenset()
allowed_raw = privs.get("allowed_models")
allowed = allowed_raw if isinstance(allowed_raw, list) else []
restricted = bool(privs.get("allowed_models_restricted")) or bool(allowed)
return frozenset(model for model in allowed if isinstance(model, str)) if restricted else None
def _allowed_models_for_request(request) -> Optional[frozenset[str]]:
"""Return the caller's model allowlist, or ``None`` when unrestricted."""
# ``effective_user`` is an attribution/storage identity for bearers, not a
# browser privilege principal. In particular, an admin-owned token must
# not inherit the owner's ADMIN_PRIVILEGES map through this lookup.
if is_bearer_principal(request):
return None
try:
user = effective_user(request)
except Exception:
user = None
if not user:
return None
auth_manager = getattr(getattr(request.app, "state", None), "auth_manager", None)
if not auth_manager:
return None
privs = auth_manager.get_privileges(user) or {}
return _allowed_models_from_privileges(privs)
def _enforce_chat_privileges(request, sess) -> None: def _enforce_chat_privileges(request, sess) -> None:
"""Apply the per-user privilege gates (allowed_models + max_messages_per_day) """Apply the per-user privilege gates (allowed_models + max_messages_per_day)
that both /api/chat and /api/chat_stream must enforce BEFORE any LLM work. that both /api/chat and /api/chat_stream must enforce BEFORE any LLM work.
@@ -232,12 +166,6 @@ def _enforce_chat_privileges(request, sess) -> None:
(single-user mode). Admins receive ADMIN_PRIVILEGES from get_privileges, (single-user mode). Admins receive ADMIN_PRIVILEGES from get_privileges,
which means unrestricted allowed_models / zero cap -> no-op for them. which means unrestricted allowed_models / zero cap -> no-op for them.
""" """
# Bearer authority is defined by the token scope at the route boundary.
# Do not turn its owner attribution back into a browser privilege lookup;
# that would make an admin-owned token inherit the admin model/cap policy.
if is_bearer_principal(request):
return
try: try:
user = effective_user(request) user = effective_user(request)
except Exception: except Exception:
@@ -257,8 +185,10 @@ def _enforce_chat_privileges(request, sess) -> None:
if privs.get("block_all_models"): if privs.get("block_all_models"):
raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.") raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.")
allowed_models = _allowed_models_from_privileges(privs) allowed_raw = privs.get("allowed_models")
if allowed_models is not None and sess.model and sess.model not in allowed_models: allowed = allowed_raw if isinstance(allowed_raw, list) else []
restricted = bool(privs.get("allowed_models_restricted")) or bool(allowed)
if restricted and sess.model and sess.model not in allowed:
raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.") raise HTTPException(403, f"Your account is not allowed to use model '{sess.model}'.")
cap = int(privs.get("max_messages_per_day") or 0) cap = int(privs.get("max_messages_per_day") or 0)
@@ -357,6 +287,96 @@ async def auto_name_session(session_manager, sess):
logger.error(f"Auto-name failed for {sess.id}: {e}\n{traceback.format_exc()}") logger.error(f"Auto-name failed for {sess.id}: {e}\n{traceback.format_exc()}")
def try_fallback_endpoint(sess, session_id: str) -> dict | None:
"""Find an alternative working endpoint when the current one fails.
Returns {"model": ..., "endpoint_url": ..., "endpoint_name": ...} or None.
"""
import requests as _req
from src.endpoint_resolver import (
build_chat_url,
build_headers,
build_models_url,
normalize_base,
resolve_endpoint_runtime,
)
from src.chatgpt_subscription import is_chatgpt_subscription_base
current_url = sess.endpoint_url or ""
owner = getattr(sess, "owner", None)
db = SessionLocal()
try:
q = db.query(ModelEndpoint).filter(
ModelEndpoint.is_enabled == True
)
if owner:
from src.auth_helpers import owner_filter
q = owner_filter(q, ModelEndpoint, owner)
endpoints = q.all()
finally:
db.close()
for ep in endpoints:
base = normalize_base(ep.base_url)
# Skip current endpoint
if current_url and base in current_url:
continue
try:
base, api_key = resolve_endpoint_runtime(ep, owner=owner)
except Exception:
continue
ping_url = build_models_url(base)
headers = build_headers(api_key, base)
try:
if ping_url:
r = _req.get(ping_url, headers=headers, timeout=5)
r.raise_for_status()
data = r.json()
models = [m.get("id") for m in (data.get("data") or []) if m.get("id")]
if not models:
models = [
m.get("name") or m.get("model")
for m in (data.get("models") or [])
if m.get("name") or m.get("model")
]
else:
models = json.loads(ep.cached_models or "[]")
if not models:
continue
# Found a working endpoint — update session
new_model = models[0]
chat_url = build_chat_url(base)
new_headers = build_headers(api_key, base)
persisted_headers = {} if is_chatgpt_subscription_base(base) else new_headers
sess.model = new_model
sess.endpoint_url = chat_url
sess.headers = new_headers
# Persist
_db = SessionLocal()
try:
_db.query(DBSession).filter(DBSession.id == session_id).update({
"model": new_model,
"endpoint_url": chat_url,
"headers": persisted_headers,
})
_db.commit()
finally:
_db.close()
logger.info(f"Fallback: switched session {session_id} from {current_url} to {ep.name} ({new_model})")
return {
"model": new_model,
"endpoint_url": chat_url,
"endpoint_name": ep.name,
}
except Exception:
continue
return None
def extract_preset(chat_handler, preset_id) -> PresetInfo: def extract_preset(chat_handler, preset_id) -> PresetInfo:
"""Extract preset parameters via chat_handler.""" """Extract preset parameters via chat_handler."""
temperature, max_tokens, system_prompt, char_name = ( temperature, max_tokens, system_prompt, char_name = (
@@ -450,13 +470,7 @@ def build_uploaded_file_manifest(att_ids: list, upload_handler, owner: Optional[
return manifest return manifest
def add_user_message( def add_user_message(sess, chat_handler, preprocessed: PreprocessedMessage, incognito: bool = False):
sess,
chat_handler,
preprocessed: PreprocessedMessage,
incognito: bool = False,
capability: RequestCapability | None = None,
):
"""Add user message to session history and update session name. """Add user message to session history and update session name.
Incognito messages must not mutate persistent session history, even in Incognito messages must not mutate persistent session history, even in
memory, because a later normal turn can persist the same session object.""" memory, because a later normal turn can persist the same session object."""
@@ -464,23 +478,11 @@ def add_user_message(
return return
user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None
sess.add_message(ChatMessage("user", preprocessed.user_content, metadata=user_meta)) sess.add_message(ChatMessage("user", preprocessed.user_content, metadata=user_meta))
if capability is None or capability.allow_auto_naming: chat_handler.update_session_name_if_needed(sess, preprocessed.text_for_context)
chat_handler.update_session_name_if_needed(sess, preprocessed.text_for_context)
def fire_message_event( def fire_message_event(request, webhook_manager, session_id: str, sess, message: str, compare_mode: bool = False):
request,
webhook_manager,
session_id: str,
sess,
message: str,
compare_mode: bool = False,
capability: RequestCapability | None = None,
):
"""Fire webhook and event_bus events for a new user message.""" """Fire webhook and event_bus events for a new user message."""
capability = capability or build_request_capability(request)
if not capability.allow_message_events:
return
if webhook_manager and not compare_mode: if webhook_manager and not compare_mode:
webhook_manager.fire_and_forget("chat.message", { webhook_manager.fire_and_forget("chat.message", {
"session_id": session_id, "model": sess.model, "message": message[:2000], "session_id": session_id, "model": sess.model, "message": message[:2000],
@@ -514,37 +516,16 @@ def _has_auth_keys(headers) -> bool:
) )
def resolve_session_auth( def resolve_session_auth(sess, session_id: str, owner: Optional[str] = None):
sess,
session_id: str,
owner: Optional[str] = None,
*,
allow_live_probes: bool = True,
):
"""Ensure session has auth headers — resolve from endpoint DB if missing.""" """Ensure session has auth headers — resolve from endpoint DB if missing."""
if not allow_live_probes:
# Bearer chat is cache-only and request-local. Do not resolve provider
# credentials or write recovered headers/session state in this mode.
return
try: try:
from src.chatgpt_subscription import is_chatgpt_subscription_base from src.chatgpt_subscription import is_chatgpt_subscription_base
is_chatgpt_subscription = is_chatgpt_subscription_base(getattr(sess, "endpoint_url", "") or "") is_chatgpt_subscription = is_chatgpt_subscription_base(getattr(sess, "endpoint_url", "") or "")
except Exception: except Exception:
is_chatgpt_subscription = False is_chatgpt_subscription = False
provenance = (getattr(sess, "endpoint_provenance", None) or "").strip().lower()
endpoint_id = (getattr(sess, "model_endpoint_id", None) or "").strip()
has_auth = _has_auth_keys(sess.headers) has_auth = _has_auth_keys(sess.headers)
if has_auth and not is_chatgpt_subscription and provenance != "registered": if has_auth and not is_chatgpt_subscription:
return return
if provenance == "direct":
# A direct API-key session owns its request headers; a same-URL
# registered endpoint must never supply another user's credentials by
# coincidence.
return
if provenance == "registered":
# Do not carry a previously persisted key through endpoint rotation or
# an unavailable endpoint while attempting exact re-resolution below.
sess.headers = {}
try: try:
from src.endpoint_resolver import build_headers, resolve_endpoint_runtime from src.endpoint_resolver import build_headers, resolve_endpoint_runtime
@@ -560,10 +541,6 @@ def resolve_session_auth(
# with similar endpoint URLs can borrow each other's API key. # with similar endpoint URLs can borrow each other's API key.
from src.auth_helpers import owner_filter from src.auth_helpers import owner_filter
q = owner_filter(q, ModelEndpoint, owner) q = owner_filter(q, ModelEndpoint, owner)
if provenance == "registered":
if not endpoint_id:
return
q = q.filter(ModelEndpoint.id == endpoint_id)
for ep in q.all(): for ep in q.all():
if not _session_url_matches_endpoint(target_url, ep.base_url or ""): if not _session_url_matches_endpoint(target_url, ep.base_url or ""):
continue continue
@@ -619,7 +596,7 @@ def _match_cached_model_id(requested: str, models) -> Optional[str]:
def _normalize_model_id_from_cache(sess) -> Optional[str]: def _normalize_model_id_from_cache(sess) -> Optional[str]:
"""Use stored ``cached_models``/pinned IDs before a live /models probe.""" """Use stored endpoint model IDs before falling back to a live /models probe."""
endpoint_url = getattr(sess, "endpoint_url", "") or "" endpoint_url = getattr(sess, "endpoint_url", "") or ""
requested = getattr(sess, "model", "") or "" requested = getattr(sess, "model", "") or ""
if not endpoint_url or not requested: if not endpoint_url or not requested:
@@ -632,12 +609,6 @@ def _normalize_model_id_from_cache(sess) -> Optional[str]:
if not session_base: if not session_base:
return None return None
provenance = getattr(sess, "endpoint_provenance", None)
endpoint_id = (getattr(sess, "model_endpoint_id", None) or "").strip()
if provenance == "direct":
# Direct API-key sessions are intentionally outside the registered
# endpoint inventory. Never borrow a same-URL endpoint's model list.
return None
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True) q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
@@ -645,10 +616,6 @@ def _normalize_model_id_from_cache(sess) -> Optional[str]:
if owner: if owner:
from src.auth_helpers import owner_filter from src.auth_helpers import owner_filter
q = owner_filter(q, ModelEndpoint, owner) q = owner_filter(q, ModelEndpoint, owner)
if provenance == "registered":
if not endpoint_id:
return None
q = q.filter(ModelEndpoint.id == endpoint_id)
endpoints = q.all() endpoints = q.all()
for ep in endpoints: for ep in endpoints:
try: try:
@@ -657,12 +624,11 @@ def _normalize_model_id_from_cache(sess) -> Optional[str]:
except Exception: except Exception:
continue continue
raw_models = getattr(ep, "cached_models", None)
if not raw_models:
continue
try: try:
from routes.model_routes import _effective_endpoint_kind, _picker_models_for_endpoint models = json.loads(raw_models) if isinstance(raw_models, str) else raw_models
base_url = getattr(ep, "base_url", "") or ""
kind = _effective_endpoint_kind(ep, base_url)
models, _ = _picker_models_for_endpoint(ep, base_url, kind)
except Exception: except Exception:
continue continue
@@ -677,91 +643,6 @@ def _normalize_model_id_from_cache(sess) -> Optional[str]:
return None return None
def _validate_bearer_session_model(sess, owner: str | None = None) -> Optional[str]:
"""Enforce endpoint-picker authority for a bearer session model.
Direct API-key sessions intentionally have no ``ModelEndpoint`` row and
retain their documented compatibility behavior. Registered endpoint
sessions, including provider-auth-backed rows, must use the visible
server-owned inventory and never trigger a live provider lookup here.
"""
# Lightweight in-memory test doubles from older route tests do not carry
# durable provenance fields. They cannot represent a persisted bearer
# session; retain their historical seam while every SessionManager-loaded
# object (which always has both fields) takes the fail-closed path below.
if not hasattr(sess, "endpoint_provenance") and not hasattr(sess, "model_endpoint_id"):
return None
provenance = (getattr(sess, "endpoint_provenance", None) or "").strip().lower()
endpoint_id = (getattr(sess, "model_endpoint_id", None) or "").strip()
if provenance == "direct":
if endpoint_id:
raise HTTPException(400, "Direct API-key sessions cannot carry a registered endpoint")
# No registered ModelEndpoint row is consulted for this documented
# compatibility path.
return None
if provenance != "registered":
raise HTTPException(400, "Session endpoint provenance is unavailable")
if not owner:
raise HTTPException(403, "A bearer session owner is required")
if not endpoint_id:
raise HTTPException(400, "Registered session endpoint identity is unavailable")
endpoint_url = (getattr(sess, "endpoint_url", "") or "").strip()
requested = (getattr(sess, "model", "") or "").strip()
if not endpoint_url:
raise HTTPException(400, "Registered session endpoint is not configured")
if not requested:
raise HTTPException(400, "Registered session model is not configured")
db = SessionLocal()
try:
from src.auth_helpers import owner_filter
q = db.query(ModelEndpoint).filter(
ModelEndpoint.id == endpoint_id,
ModelEndpoint.is_enabled == True,
)
q = owner_filter(q, ModelEndpoint, owner)
endpoints = q.all()
if len(endpoints) != 1:
# This covers disabled/deleted/owner-mismatched rows as well as
# malformed duplicate results. Do not fall back to URL matching.
raise HTTPException(400, "Registered model endpoint is no longer available")
ep = endpoints[0]
if not _session_url_matches_endpoint(endpoint_url, getattr(ep, "base_url", "") or ""):
raise HTTPException(400, "Session endpoint provenance is stale")
from routes.model_routes import _validate_bearer_model_selection
validated = _validate_bearer_model_selection(ep, requested)
# A session may outlive an endpoint-key rotation. For bearer calls,
# use the current exact-endpoint credentials and never trust a stale
# persisted Authorization header. Provider-auth credentials are
# owner-scoped, request-local, and cache-only in this boundary.
try:
from src.endpoint_resolver import build_headers, resolve_endpoint_runtime
base, api_key = resolve_endpoint_runtime(
ep,
owner=owner,
allow_live_probes=False,
)
sess.headers = build_headers(api_key, base)
except Exception as exc:
logger.warning("Could not refresh bearer session endpoint auth: %s", exc)
sess.headers = {}
if getattr(ep, "provider_auth_id", None):
raise HTTPException(401, "Registered provider credentials are unavailable") from exc
raise HTTPException(400, "Registered endpoint credentials are unavailable") from exc
sess.model = validated
return validated
finally:
db.close()
def _session_is_research_spinoff(sess) -> bool: def _session_is_research_spinoff(sess) -> bool:
"""True if this session was created via research "Discuss" spin-off. """True if this session was created via research "Discuss" spin-off.
@@ -806,18 +687,12 @@ async def build_chat_context(
use_enhanced_message: bool = False, use_enhanced_message: bool = False,
agent_mode: bool = False, agent_mode: bool = False,
allow_tool_preprocessing: bool = True, allow_tool_preprocessing: bool = True,
defer_context_shaping: bool = False,
continuation_context_message: str | None = None,
persist_user_message: bool = True,
capability: RequestCapability | None = None,
) -> ChatContext: ) -> ChatContext:
"""Build the full context (preface + messages) for an LLM call. """Build the full context (preface + messages) for an LLM call.
This is the shared logic between /chat and /chat_stream preset extraction, This is the shared logic between /chat and /chat_stream preset extraction,
message preprocessing, memory/RAG/web injection, compaction, normalization. message preprocessing, memory/RAG/web injection, compaction, normalization.
""" """
capability = capability or build_request_capability(request)
# Preset # Preset
preset = extract_preset(chat_handler, preset_id) preset = extract_preset(chat_handler, preset_id)
@@ -835,29 +710,15 @@ async def build_chat_context(
# Add user message to history. Nobody/incognito uses a request-local # Add user message to history. Nobody/incognito uses a request-local
# transcript store instead of session history so stale saved chats cannot # transcript store instead of session history so stale saved chats cannot
# bleed into context and the turn is not persisted. # bleed into context and the turn is not persisted.
if persist_user_message and incognito: if incognito:
user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None user_meta = {"attachments": preprocessed.attachment_meta} if preprocessed.attachment_meta else None
_append_incognito_message(session_id, "user", preprocessed.user_content, user_meta) _append_incognito_message(session_id, "user", preprocessed.user_content, user_meta)
elif persist_user_message: else:
add_user_message( add_user_message(sess, chat_handler, preprocessed, incognito=False)
sess,
chat_handler,
preprocessed,
incognito=False,
capability=capability,
)
# Fire events # Fire events
if persist_user_message and not incognito: if not incognito:
fire_message_event( fire_message_event(request, webhook_manager, session_id, sess, message, compare_mode)
request,
webhook_manager,
session_id,
sess,
message,
compare_mode,
capability=capability,
)
# Resolve owner-scoped prefs/context. Browser requests keep the cookie user; # Resolve owner-scoped prefs/context. Browser requests keep the cookie user;
# bearer-token chat requests use the token owner instead of the "api" sentinel. # bearer-token chat requests use the token owner instead of the "api" sentinel.
@@ -868,12 +729,7 @@ async def build_chat_context(
getattr(chat_handler, "upload_handler", None), getattr(chat_handler, "upload_handler", None),
getattr(sess, "owner", None), getattr(sess, "owner", None),
) )
context_message = ( casual_low_signal = _is_casual_low_signal(message)
str(continuation_context_message).strip()
if continuation_context_message
else message
)
casual_low_signal = _is_casual_low_signal(context_message)
# Memory enabled? # Memory enabled?
mem_enabled = not incognito and not no_memory and uprefs.get("memory_enabled", True) mem_enabled = not incognito and not no_memory and uprefs.get("memory_enabled", True)
@@ -910,15 +766,7 @@ async def build_chat_context(
# Build context preface # Build context preface
# The stream path uses enhanced_message (with CoT/preprocessing applied), # The stream path uses enhanced_message (with CoT/preprocessing applied),
# the sync path uses text_for_context. # the sync path uses text_for_context.
_ctx_msg = ( _ctx_msg = preprocessed.enhanced_message if use_enhanced_message else preprocessed.text_for_context
context_message
if continuation_context_message
else (
preprocessed.enhanced_message
if use_enhanced_message
else preprocessed.text_for_context
)
)
_preface_kwargs = dict( _preface_kwargs = dict(
message=_ctx_msg, message=_ctx_msg,
session=sess, session=sess,
@@ -931,7 +779,6 @@ async def build_chat_context(
agent_mode=agent_mode, agent_mode=agent_mode,
incognito=incognito, incognito=incognito,
use_skills=skills_enabled, use_skills=skills_enabled,
allow_tool_preprocessing=allow_tool_preprocessing,
) )
if use_rag is not None or is_research_spinoff or casual_low_signal: if use_rag is not None or is_research_spinoff or casual_low_signal:
_preface_kwargs["use_rag"] = use_rag_val _preface_kwargs["use_rag"] = use_rag_val
@@ -950,27 +797,18 @@ async def build_chat_context(
# Normalize model ID. Prefer cached endpoint models so group chat does not # Normalize model ID. Prefer cached endpoint models so group chat does not
# re-hit slow local /models endpoints on every participant turn. # re-hit slow local /models endpoints on every participant turn.
norm = _normalize_model_id_from_cache(sess) norm = _normalize_model_id_from_cache(sess) or normalize_model_id(
# Model normalization falls back to a live /models or /tags request on a sess.endpoint_url,
# cache miss. A bearer chat request may use the stored model as-is, but it sess.model,
# must not implicitly refresh an endpoint catalogue while building context. owner=getattr(sess, "owner", None),
if norm is None and capability.allow_live_probes: )
norm = normalize_model_id(
sess.endpoint_url,
sess.model,
owner=getattr(sess, "owner", None),
)
if norm: if norm:
sess.model = norm sess.model = norm
# Build messages. In Nobody/incognito mode, never read saved session # Build messages. In Nobody/incognito mode, never read saved session
# history: the session id may be a temporary wrapper or, in buggy clients, a # history: the session id may be a temporary wrapper or, in buggy clients, a
# stale normal session id. Only the ephemeral incognito transcript is safe. # stale normal session id. Only the ephemeral incognito transcript is safe.
messages = preface + ( messages = preface + (_incognito_messages(session_id) if incognito else sess.get_context_messages())
_incognito_messages(session_id)
if incognito
else _history_for_request_capability(sess, capability)
)
# Current date/time — injected as a standalone *user*-role context message # Current date/time — injected as a standalone *user*-role context message
# placed immediately before the latest user turn, NOT folded into the # placed immediately before the latest user turn, NOT folded into the
@@ -992,33 +830,13 @@ async def build_chat_context(
except Exception: except Exception:
logger.debug("Failed to add current date/time context", exc_info=True) logger.debug("Failed to add current date/time context", exc_info=True)
route_messages = list(messages) # Auto-compact
# Explicit fallback routing must shape from the same route-neutral prompt messages, context_length, was_compacted = await maybe_compact(
# for every candidate. Running selected-model compaction here would mutate sess, sess.endpoint_url, sess.model, messages, sess.headers, owner=user,
# session history before we know which route can answer and would make a )
# later larger-context candidate unable to recover discarded history.
if defer_context_shaping:
context_kwargs = {}
if not capability.allow_live_probes:
context_kwargs["allow_live_probes"] = False
context_length = get_context_length(sess.endpoint_url, sess.model, **context_kwargs)
was_compacted = False
else:
compact_kwargs = {"owner": user}
if not capability.allow_live_probes:
compact_kwargs["allow_live_probes"] = False
messages, context_length, was_compacted = await maybe_compact(
sess,
sess.endpoint_url,
sess.model,
messages,
sess.headers,
**compact_kwargs,
)
_before_trim_messages = len(messages) _before_trim_messages = len(messages)
_before_trim_tokens = estimate_tokens(messages) _before_trim_tokens = estimate_tokens(messages)
if not defer_context_shaping: messages = trim_for_context(messages, context_length)
messages = trim_for_context(messages, context_length)
_after_trim_messages = len(messages) _after_trim_messages = len(messages)
_after_trim_tokens = estimate_tokens(messages) _after_trim_tokens = estimate_tokens(messages)
_context_trimmed = _after_trim_messages < _before_trim_messages or _after_trim_tokens < _before_trim_tokens _context_trimmed = _after_trim_messages < _before_trim_messages or _after_trim_tokens < _before_trim_tokens
@@ -1042,7 +860,6 @@ async def build_chat_context(
context_tokens_after_trim=_after_trim_tokens, context_tokens_after_trim=_after_trim_tokens,
auto_opened_docs=auto_opened_docs, auto_opened_docs=auto_opened_docs,
uploaded_files=uploaded_files, uploaded_files=uploaded_files,
route_messages=route_messages,
) )
@@ -1393,7 +1210,6 @@ def run_post_response_tasks(
owner: str = None, owner: str = None,
extract_skills: bool = True, extract_skills: bool = True,
allow_background_extraction: bool = True, allow_background_extraction: bool = True,
capability: RequestCapability | None = None,
): ):
"""Fire background tasks after a completed response: memory extraction, webhooks, auto-name, skill extraction. """Fire background tasks after a completed response: memory extraction, webhooks, auto-name, skill extraction.
@@ -1409,12 +1225,6 @@ def run_post_response_tasks(
``_queue_background_extraction`` keeps them from overlapping the *next* ``_queue_background_extraction`` keeps them from overlapping the *next*
turn's request too. turn's request too.
""" """
if capability is not None and not capability.allow_deferred_work:
# Pure bearer chat is intentionally synchronous and request-bound.
# Do not schedule extraction, teacher/model work, callbacks, or
# auto-naming after the authorized request has returned/disconnected.
return
_extraction_jobs: list = [] _extraction_jobs: list = []
# Memory extraction — only every 4th message pair to avoid excess LLM calls # Memory extraction — only every 4th message pair to avoid excess LLM calls
+121 -1121
View File
File diff suppressed because it is too large Load Diff
+41 -85
View File
@@ -1,9 +1,8 @@
"""Codex integration routes. """Codex integration routes.
These are small HTTP surfaces intended for the Codex plugin/MCP bridge. They These are small HTTP surfaces intended for the Codex plugin/MCP bridge. They
reuse existing Odysseus helpers. Owner-scoped data operations support bearer reuse existing Odysseus helpers and enforce API-token scopes before touching
principals with the matching token scope; the Cookbook/plugin host-control user data.
plane remains interactive-only.
""" """
import asyncio import asyncio
@@ -13,16 +12,11 @@ from io import BytesIO
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from fastapi import APIRouter, BackgroundTasks, Body, Depends, HTTPException, Request from fastapi import APIRouter, BackgroundTasks, Body, HTTPException, Request
from fastapi.responses import StreamingResponse from fastapi.responses import StreamingResponse
from core.middleware import require_admin from core.middleware import require_admin
from src.auth_helpers import ( from src.auth_helpers import require_authenticated_request, require_user
require_api_token_owner,
require_authenticated_request,
require_non_bearer_request,
require_user,
)
from src.tool_implementations import do_manage_notes from src.tool_implementations import do_manage_notes
from src.constants import COOKBOOK_STATE_FILE from src.constants import COOKBOOK_STATE_FILE
from routes._validators import validate_remote_host, validate_ssh_port from routes._validators import validate_remote_host, validate_ssh_port
@@ -67,40 +61,9 @@ async def _as_owner(request: Request, owner: str, fn, *args, **kwargs):
"""Run an existing route handler with request.state.current_user temporarily """Run an existing route handler with request.state.current_user temporarily
set to ``owner`` so its internal get_current_user/require_user calls see set to ``owner`` so its internal get_current_user/require_user calls see
the scope-gated owner (not the "api" pseudo-user the bearer middleware sets). the scope-gated owner (not the "api" pseudo-user the bearer middleware sets).
Temporarily hide the bearer header as well: nested legacy handlers classify Restores the original value when done. Works for sync and async handlers."""
the raw header independently of ``request.state.api_token``. Restore every
request value when done. Works for sync and async handlers."""
orig = getattr(request.state, "current_user", None) orig = getattr(request.state, "current_user", None)
orig_api_token = getattr(request.state, "api_token", None) orig_api_token = getattr(request.state, "api_token", None)
missing = object()
scope = getattr(request, "scope", None)
original_scope_headers = missing
original_cached_headers = missing
original_mapping_headers = missing
if isinstance(scope, dict) and "headers" in scope:
original_scope_headers = scope["headers"]
scope["headers"] = [
(name, value)
for name, value in (original_scope_headers or [])
if not (
(isinstance(name, bytes) and name.lower() == b"authorization")
or (isinstance(name, str) and name.casefold() == "authorization")
)
]
request_dict = getattr(request, "__dict__", {})
if "_headers" in request_dict:
original_cached_headers = request_dict["_headers"]
request_dict.pop("_headers", None)
else:
current_headers = getattr(request, "headers", missing)
if isinstance(current_headers, dict):
original_mapping_headers = current_headers
request.headers = {
name: value
for name, value in current_headers.items()
if str(name).casefold() != "authorization"
}
request.state.current_user = owner request.state.current_user = owner
request.state.api_token = False request.state.api_token = False
try: try:
@@ -117,49 +80,46 @@ async def _as_owner(request: Request, owner: str, fn, *args, **kwargs):
pass pass
else: else:
request.state.api_token = orig_api_token request.state.api_token = orig_api_token
if original_scope_headers is not missing:
scope["headers"] = original_scope_headers
request_dict = getattr(request, "__dict__", {})
request_dict.pop("_headers", None)
if original_cached_headers is not missing:
request_dict["_headers"] = original_cached_headers
if original_mapping_headers is not missing:
request.headers = original_mapping_headers
def _scope_owner(request: Request, allowed: set[str]) -> str: def _scope_owner(request: Request, allowed: set[str]) -> str:
"""Return the data owner if the caller is allowed for this Codex action.""" """Return the data owner if the caller is allowed for this Codex action."""
if getattr(request.state, "api_token", False) is True: if getattr(request.state, "api_token", False):
scopes = set(getattr(request.state, "api_token_scopes", []) or []) scopes = set(getattr(request.state, "api_token_scopes", []) or [])
if not scopes.intersection(allowed): if not scopes.intersection(allowed):
required = " or ".join(sorted(allowed)) required = " or ".join(sorted(allowed))
raise HTTPException(403, f"API token missing required scope: {required}") raise HTTPException(403, f"API token missing required scope: {required}")
return require_api_token_owner(request) owner = getattr(request.state, "api_token_owner", None)
if not owner:
raise HTTPException(403, "API token has no owner")
return owner
return require_user(request) return require_user(request)
def _scope_owner_all(request: Request, required: set[str]) -> str: def _scope_owner_all(request: Request, required: set[str]) -> str:
"""Return owner only when an API token has every required scope.""" """Return owner only when an API token has every required scope."""
if getattr(request.state, "api_token", False) is True: if getattr(request.state, "api_token", False):
scopes = set(getattr(request.state, "api_token_scopes", []) or []) scopes = set(getattr(request.state, "api_token_scopes", []) or [])
missing = required - scopes missing = required - scopes
if missing: if missing:
raise HTTPException(403, f"API token missing required scope: {' and '.join(sorted(missing))}") raise HTTPException(403, f"API token missing required scope: {' and '.join(sorted(missing))}")
return require_api_token_owner(request) owner = getattr(request.state, "api_token_owner", None)
if not owner:
raise HTTPException(403, "API token has no owner")
return owner
return require_user(request) return require_user(request)
def _require_cookbook_scope(request: Request, allowed: set[str]) -> str: def _require_cookbook_scope(request: Request, allowed: set[str]) -> str:
"""Authorize a Codex cookbook route. """Authorize a Codex cookbook route.
Bearer callers are rejected by the host-control boundary regardless of For API-token callers, enforce the given scope set.
legacy scope labels. Cookie-session callers additionally require admin For cookie-session callers, additionally require admin privileges
privileges because cookbook surfaces expose host topology, task logs, tmux because cookbook surfaces expose host topology, task logs, tmux
commands, and model-serving controls. commands, and model-serving controls.
""" """
require_non_bearer_request(request)
owner = _scope_owner(request, allowed) owner = _scope_owner(request, allowed)
if getattr(request.state, "api_token", False) is not True: if not getattr(request.state, "api_token", False):
require_admin(request) require_admin(request)
return owner return owner
@@ -191,10 +151,7 @@ def setup_codex_routes(
calendar_router: APIRouter | None = None, calendar_router: APIRouter | None = None,
document_router: APIRouter | None = None, document_router: APIRouter | None = None,
) -> APIRouter: ) -> APIRouter:
router = APIRouter( router = APIRouter(prefix="/api/codex", tags=["codex"])
prefix="/api/codex",
tags=["codex"],
)
email_list_endpoint = _find_endpoint(email_router, "GET", "/api/email/list") email_list_endpoint = _find_endpoint(email_router, "GET", "/api/email/list")
email_read_endpoint = _find_endpoint(email_router, "GET", "/api/email/read/{uid}") email_read_endpoint = _find_endpoint(email_router, "GET", "/api/email/read/{uid}")
email_send_endpoint = _find_endpoint(email_router, "POST", "/api/email/send") email_send_endpoint = _find_endpoint(email_router, "POST", "/api/email/send")
@@ -210,7 +167,7 @@ def setup_codex_routes(
@router.get("/capabilities") @router.get("/capabilities")
def capabilities(request: Request): def capabilities(request: Request):
token_scopes = set(getattr(request.state, "api_token_scopes", []) or []) token_scopes = set(getattr(request.state, "api_token_scopes", []) or [])
has_token = getattr(request.state, "api_token", False) is True has_token = bool(getattr(request.state, "api_token", False))
def scoped(allowed): def scoped(allowed):
return bool(token_scopes.intersection(allowed)) if has_token else True return bool(token_scopes.intersection(allowed)) if has_token else True
return { return {
@@ -258,9 +215,8 @@ def setup_codex_routes(
}, },
} }
@router.get("/plugin.zip", dependencies=[Depends(require_non_bearer_request)]) @router.get("/plugin.zip")
def plugin_zip(request: Request): def plugin_zip(request: Request):
require_non_bearer_request(request)
require_authenticated_request(request) require_authenticated_request(request)
root = Path(__file__).resolve().parent.parent / "integrations" / "codex" root = Path(__file__).resolve().parent.parent / "integrations" / "codex"
if not root.exists(): if not root.exists():
@@ -557,10 +513,15 @@ def setup_codex_routes(
return await _as_owner(request, owner, documents_create_endpoint, request, req) return await _as_owner(request, owner, documents_create_endpoint, request, req)
# ── Cookbook surface ── # ── Cookbook surface ──
# These handlers retain their legacy scope constants for compatibility # Lets the agent run the same launch / monitor / kill loop the user
# with callers and tests, but the bridge is an interactive-only # would do by hand in the Cookbook UI: read the current task list +
# host-control plane. Bearer principals are rejected before any task-list, # tmux output, launch a serve task, stop one. Two scopes:
# tmux-output, launch, stop, or model-serving operation. # cookbook:read — list tasks + tail output + list servers
# cookbook:launch — also start/stop serves (host shell exec)
# `cookbook:launch` is genuinely powerful: /api/model/serve runs SSH'd
# commands on the user's hosts. The existing _validate_serve_cmd
# allowlist (vllm/python3/sglang/llama-server/etc., no shell metachars)
# keeps the agent inside the same sandbox the UI uses.
async def _run_shell(cmd: str, timeout: float = 15.0) -> dict: async def _run_shell(cmd: str, timeout: float = 15.0) -> dict:
"""Run a shell command, return {exit_code, stdout, stderr}.""" """Run a shell command, return {exit_code, stdout, stderr}."""
@@ -604,14 +565,14 @@ def setup_codex_routes(
if k not in ("hf_token", "_secrets")} if k not in ("hf_token", "_secrets")}
return clean return clean
@router.get("/cookbook/tasks", dependencies=[Depends(require_non_bearer_request)]) @router.get("/cookbook/tasks")
async def codex_cookbook_tasks(request: Request): async def codex_cookbook_tasks(request: Request):
_require_cookbook_scope(request, COOKBOOK_READ_SCOPES) _require_cookbook_scope(request, COOKBOOK_READ_SCOPES)
state = _read_cookbook_state() state = _read_cookbook_state()
tasks = state.get("tasks") or [] tasks = state.get("tasks") or []
return {"tasks": [_redact_task(t) for t in tasks]} return {"tasks": [_redact_task(t) for t in tasks]}
@router.get("/cookbook/servers", dependencies=[Depends(require_non_bearer_request)]) @router.get("/cookbook/servers")
async def codex_cookbook_servers(request: Request): async def codex_cookbook_servers(request: Request):
_require_cookbook_scope(request, COOKBOOK_READ_SCOPES) _require_cookbook_scope(request, COOKBOOK_READ_SCOPES)
state = _read_cookbook_state() state = _read_cookbook_state()
@@ -630,7 +591,7 @@ def setup_codex_routes(
}) })
return {"servers": cleaned} return {"servers": cleaned}
@router.get("/cookbook/output/{session_id}", dependencies=[Depends(require_non_bearer_request)]) @router.get("/cookbook/output/{session_id}")
async def codex_cookbook_output(request: Request, session_id: str, tail: int = 400): async def codex_cookbook_output(request: Request, session_id: str, tail: int = 400):
_require_cookbook_scope(request, COOKBOOK_READ_SCOPES) _require_cookbook_scope(request, COOKBOOK_READ_SCOPES)
# Defensive: session_id must be the tmux-style id we issue # Defensive: session_id must be the tmux-style id we issue
@@ -672,7 +633,7 @@ def setup_codex_routes(
"task": _redact_task(task), "task": _redact_task(task),
} }
@router.post("/cookbook/serve", dependencies=[Depends(require_non_bearer_request)]) @router.post("/cookbook/serve")
async def codex_cookbook_serve(request: Request, body: dict[str, Any] = Body(default_factory=dict)): async def codex_cookbook_serve(request: Request, body: dict[str, Any] = Body(default_factory=dict)):
_require_cookbook_scope(request, COOKBOOK_LAUNCH_SCOPES) _require_cookbook_scope(request, COOKBOOK_LAUNCH_SCOPES)
# Wraps /api/model/serve with the SAME validation the UI uses. # Wraps /api/model/serve with the SAME validation the UI uses.
@@ -711,7 +672,7 @@ def setup_codex_routes(
raise HTTPException(503, "model serve endpoint unavailable") raise HTTPException(503, "model serve endpoint unavailable")
return await serve_endpoint(request, req) return await serve_endpoint(request, req)
@router.post("/cookbook/stop/{session_id}", dependencies=[Depends(require_non_bearer_request)]) @router.post("/cookbook/stop/{session_id}")
async def codex_cookbook_stop(request: Request, session_id: str): async def codex_cookbook_stop(request: Request, session_id: str):
_require_cookbook_scope(request, COOKBOOK_LAUNCH_SCOPES) _require_cookbook_scope(request, COOKBOOK_LAUNCH_SCOPES)
import re as _re import re as _re
@@ -728,7 +689,7 @@ def setup_codex_routes(
result = await _run_shell(cmd, timeout=10) result = await _run_shell(cmd, timeout=10)
return {"session_id": session_id, "exit_code": result.get("exit_code"), "host": host or "local"} return {"session_id": session_id, "exit_code": result.get("exit_code"), "host": host or "local"}
@router.get("/cookbook/cached", dependencies=[Depends(require_non_bearer_request)]) @router.get("/cookbook/cached")
async def codex_cookbook_cached(request: Request, host: str | None = None): async def codex_cookbook_cached(request: Request, host: str | None = None):
"""List cached models on a configured server (or local if host is omitted). """List cached models on a configured server (or local if host is omitted).
Mirrors `list_cached_models` from the chat agent so external agents have Mirrors `list_cached_models` from the chat agent so external agents have
@@ -790,7 +751,7 @@ def setup_codex_routes(
platform=params.get("platform") or None, platform=params.get("platform") or None,
) )
@router.get("/cookbook/presets", dependencies=[Depends(require_non_bearer_request)]) @router.get("/cookbook/presets")
async def codex_cookbook_presets(request: Request): async def codex_cookbook_presets(request: Request):
"""List saved serve presets (model + host + port + launch cmd). """List saved serve presets (model + host + port + launch cmd).
Counterpart to `list_serve_presets`. Use BEFORE composing a `serve` Counterpart to `list_serve_presets`. Use BEFORE composing a `serve`
@@ -811,7 +772,7 @@ def setup_codex_routes(
}) })
return {"presets": out, "default_host": (state.get("env") or {}).get("defaultServer", "")} return {"presets": out, "default_host": (state.get("env") or {}).get("defaultServer", "")}
@router.post("/cookbook/preset/{name}", dependencies=[Depends(require_non_bearer_request)]) @router.post("/cookbook/preset/{name}")
async def codex_cookbook_serve_preset(request: Request, name: str): async def codex_cookbook_serve_preset(request: Request, name: str):
"""Launch a saved preset by name. Reuses the working cmd + host the """Launch a saved preset by name. Reuses the working cmd + host the
user already saved, avoiding the cmd-allowlist trial-and-error loop.""" user already saved, avoiding the cmd-allowlist trial-and-error loop."""
@@ -861,7 +822,7 @@ def setup_codex_routes(
raise HTTPException(503, "model serve endpoint unavailable") raise HTTPException(503, "model serve endpoint unavailable")
return await serve_endpoint(request, req) return await serve_endpoint(request, req)
@router.post("/cookbook/adopt", dependencies=[Depends(require_non_bearer_request)]) @router.post("/cookbook/adopt")
async def codex_cookbook_adopt(request: Request, body: dict[str, Any] = Body(default_factory=dict)): async def codex_cookbook_adopt(request: Request, body: dict[str, Any] = Body(default_factory=dict)):
"""Adopt an existing tmux session (one started via raw ssh+tmux) into """Adopt an existing tmux session (one started via raw ssh+tmux) into
cookbook tracking. Needed when serve_model rejects a cmd and the cookbook tracking. Needed when serve_model rejects a cmd and the
@@ -925,15 +886,10 @@ def setup_claude_routes() -> APIRouter:
this router only exists to deliver the skill zip via `/api/claude/plugin.zip` this router only exists to deliver the skill zip via `/api/claude/plugin.zip`
so the user-facing setup commands stay in the Claude namespace. so the user-facing setup commands stay in the Claude namespace.
""" """
router = APIRouter( router = APIRouter(prefix="/api/claude", tags=["claude"])
prefix="/api/claude",
tags=["claude"],
dependencies=[Depends(require_non_bearer_request)],
)
@router.get("/plugin.zip") @router.get("/plugin.zip")
def plugin_zip(request: Request): def plugin_zip(request: Request):
require_non_bearer_request(request)
require_authenticated_request(request) require_authenticated_request(request)
# Only ship the skills/ subtree so extracting at ~/.claude/ doesn't dump # Only ship the skills/ subtree so extracting at ~/.claude/ doesn't dump
# README.md or other bundle metadata into the user's claude config dir. # README.md or other bundle metadata into the user's claude config dir.
+17 -87
View File
@@ -4,24 +4,19 @@ import json
import uuid import uuid
import random import random
from datetime import datetime from datetime import datetime
from fastapi import APIRouter, Depends, Form, HTTPException, Request from fastapi import APIRouter, Form, HTTPException, Request
from typing import List from typing import List
from pydantic import BaseModel from pydantic import BaseModel
import logging import logging
from core.database import Comparison, SessionLocal from core.database import Comparison, SessionLocal
from core.session_manager import SessionManager from core.session_manager import SessionManager
from src.auth_helpers import effective_user, is_bearer_principal, require_chat_scope from src.auth_helpers import get_current_user
from src.session_provenance import persist_session_endpoint_provenance
from routes.session_routes import _reject_raw_endpoint_url_for_non_admin from routes.session_routes import _reject_raw_endpoint_url_for_non_admin
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter( router = APIRouter(prefix="/api/compare", tags=["compare"])
prefix="/api/compare",
tags=["compare"],
dependencies=[Depends(require_chat_scope)],
)
def _owned_endpoint_by_url(db, base_url, owner): def _owned_endpoint_by_url(db, base_url, owner):
@@ -69,37 +64,6 @@ class RecordVoteRequest(BaseModel):
is_blind: bool = True is_blind: bool = True
def _validate_bearer_compare_models(models, owner: str) -> list[str]:
"""Validate record-only comparison models against visible endpoint caches."""
from core.database import ModelEndpoint
from src.auth_helpers import owner_filter
from routes.model_routes import _validate_bearer_model_selection
db = SessionLocal()
try:
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
q = owner_filter(q, ModelEndpoint, owner)
endpoints = q.all()
selected = []
for requested in models:
matches = []
for ep in endpoints:
try:
matches.append(_validate_bearer_model_selection(ep, requested))
except HTTPException:
continue
unique = list(dict.fromkeys(matches))
if len(unique) != 1:
raise HTTPException(
400,
f"Model is not permitted by a visible server endpoint: {requested}",
)
selected.append(unique[0])
return selected
finally:
db.close()
def setup_compare_routes(session_manager: SessionManager): def setup_compare_routes(session_manager: SessionManager):
"""Setup comparison routes.""" """Setup comparison routes."""
@@ -120,9 +84,7 @@ def setup_compare_routes(session_manager: SessionManager):
Returns the comparison ID and the two session IDs so the client Returns the comparison ID and the two session IDs so the client
can fire two independent SSE streams to /api/chat_stream. can fire two independent SSE streams to /api/chat_stream.
""" """
require_chat_scope(request) user = getattr(request.state, 'current_user', None)
user = effective_user(request)
bearer = is_bearer_principal(request)
comp_id = str(uuid.uuid4()) comp_id = str(uuid.uuid4())
sid_a = str(uuid.uuid4()) sid_a = str(uuid.uuid4())
sid_b = str(uuid.uuid4()) sid_b = str(uuid.uuid4())
@@ -198,13 +160,6 @@ def setup_compare_routes(session_manager: SessionManager):
_reject_raw_endpoint_url_for_non_admin( _reject_raw_endpoint_url_for_non_admin(
request, user, str(ep.id) if ep is not None else None, endpoint request, user, str(ep.id) if ep is not None else None, endpoint
) )
selected_model = model
if bearer:
if ep is None:
raise HTTPException(403, "Choose a registered model endpoint")
from routes.model_routes import _validate_bearer_model_selection
selected_model = _validate_bearer_model_selection(ep, model)
# Bind the [CMP] session to the RESOLVED endpoint, not the raw # Bind the [CMP] session to the RESOLVED endpoint, not the raw
# caller-supplied string. When the URL matches a registered # caller-supplied string. When the URL matches a registered
# endpoint visible to the caller, use that row's own normalized # endpoint visible to the caller, use that row's own normalized
@@ -221,24 +176,15 @@ def setup_compare_routes(session_manager: SessionManager):
# `ep` is None (raw admin URL or no match), so a comparison can # `ep` is None (raw admin URL or no match), so a comparison can
# never inherit another user's key/headers. # never inherit another user's key/headers.
headers = build_headers(ep.api_key, ep.base_url) if (ep and ep.api_key) else None headers = build_headers(ep.api_key, ep.base_url) if (ep and ep.api_key) else None
resolved.append( resolved.append((sid, model, session_endpoint_url, headers))
(
sid,
selected_model,
session_endpoint_url,
headers,
str(ep.id) if ep is not None else None,
"registered" if ep is not None else None,
)
)
finally: finally:
db.close() db.close()
# Both endpoints validated — only now create the ephemeral [CMP] # Both endpoints validated — only now create the ephemeral [CMP]
# sessions and copy any resolved headers. # sessions and copy any resolved headers.
for sid, model, session_endpoint_url, headers, endpoint_id, provenance in resolved: for sid, model, session_endpoint_url, headers in resolved:
name = f"[CMP] {slot_name[sid]}" if blind else f"[CMP] {model.split('/')[-1]}" name = f"[CMP] {slot_name[sid]}" if blind else f"[CMP] {model.split('/')[-1]}"
comparison_session = session_manager.create_session( session_manager.create_session(
session_id=sid, session_id=sid,
name=name, name=name,
endpoint_url=session_endpoint_url, endpoint_url=session_endpoint_url,
@@ -246,14 +192,6 @@ def setup_compare_routes(session_manager: SessionManager):
rag=False, rag=False,
owner=user, owner=user,
) )
if provenance in {"registered", "direct"}:
persist_session_endpoint_provenance(
session_manager,
sid,
comparison_session,
model_endpoint_id=endpoint_id,
endpoint_provenance=provenance,
)
if headers: if headers:
s = session_manager.sessions.get(sid) s = session_manager.sessions.get(sid)
if s: if s:
@@ -265,8 +203,8 @@ def setup_compare_routes(session_manager: SessionManager):
comp = Comparison( comp = Comparison(
id=comp_id, id=comp_id,
prompt=prompt, prompt=prompt,
model_a=resolved[0][1], model_a=model_a,
model_b=resolved[1][1], model_b=model_b,
# Record the URL the session actually dials. For URL callers this # Record the URL the session actually dials. For URL callers this
# is their raw input; for id-only callers (empty endpoint_a/_b) # is their raw input; for id-only callers (empty endpoint_a/_b)
# fall back to the resolved endpoint URL so the column stays # fall back to the resolved endpoint URL so the column stays
@@ -303,8 +241,7 @@ def setup_compare_routes(session_manager: SessionManager):
winner: str = Form(...), # "left", "right", or "tie" winner: str = Form(...), # "left", "right", or "tie"
): ):
"""Record the user's vote and reveal model names if blind.""" """Record the user's vote and reveal model names if blind."""
require_chat_scope(request) user = get_current_user(request)
user = effective_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
comp = db.query(Comparison).filter(Comparison.id == comp_id).first() comp = db.query(Comparison).filter(Comparison.id == comp_id).first()
@@ -346,20 +283,15 @@ def setup_compare_routes(session_manager: SessionManager):
@router.post("/record") @router.post("/record")
def record_comparison(request: Request, body: RecordVoteRequest): def record_comparison(request: Request, body: RecordVoteRequest):
"""Lightweight endpoint to record a comparison vote from the frontend.""" """Lightweight endpoint to record a comparison vote from the frontend."""
require_chat_scope(request) user = get_current_user(request)
user = effective_user(request)
comp_id = str(uuid.uuid4()) comp_id = str(uuid.uuid4())
models = list(body.models or []) model_a = body.models[0] if len(body.models) > 0 else ""
if is_bearer_principal(request): model_b = body.models[1] if len(body.models) > 1 else ""
models = _validate_bearer_compare_models(models, user)
model_a = models[0] if len(models) > 0 else ""
model_b = models[1] if len(models) > 1 else ""
# For N>2 models, store the full list as JSON in blind_mapping # For N>2 models, store the full list as JSON in blind_mapping
if len(models) > 2: if len(body.models) > 2:
blind_mapping = json.dumps({"models": models}) blind_mapping = json.dumps({"models": body.models})
else: else:
blind_mapping = None blind_mapping = None
@@ -388,8 +320,7 @@ def setup_compare_routes(session_manager: SessionManager):
@router.get("/history") @router.get("/history")
def list_comparisons(request: Request): def list_comparisons(request: Request):
"""List past comparisons.""" """List past comparisons."""
require_chat_scope(request) user = get_current_user(request)
user = effective_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(Comparison) q = db.query(Comparison)
@@ -415,8 +346,7 @@ def setup_compare_routes(session_manager: SessionManager):
@router.delete("/{comp_id}") @router.delete("/{comp_id}")
def delete_comparison(request: Request, comp_id: str): def delete_comparison(request: Request, comp_id: str):
"""Delete a comparison and its ephemeral sessions.""" """Delete a comparison and its ephemeral sessions."""
require_chat_scope(request) user = get_current_user(request)
user = effective_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
comp = db.query(Comparison).filter(Comparison.id == comp_id).first() comp = db.query(Comparison).filter(Comparison.id == comp_id).first()
-35
View File
@@ -1204,41 +1204,6 @@ def _safe_env_prefix(ep: str | None) -> str | None:
return f'[ -f "{path}" ] && source "{path}" || true' return f'[ -f "{path}" ] && source "{path}" || true'
def _local_windows_bash_env_prefix(ep: str | None) -> str | None:
"""Convert a frontend PowerShell venv prefix for the local Git Bash runner."""
if not ep:
return ep
prefix = ep.strip()
if not prefix.startswith("&"):
return ep
raw_path = prefix[1:].lstrip()
if not raw_path:
return ep
if raw_path.startswith("'"):
if len(raw_path) < 2 or not raw_path.endswith("'"):
return ep
quoted_path = raw_path[1:-1]
if "'" in quoted_path.replace("''", ""):
return ep
path = quoted_path.replace("''", "'")
else:
path = raw_path.rstrip()
if "'" in path or '"' in path:
return ep
if any(c in path for c in "\r\n;&|`$<>"):
return ep
if not path.replace("\\", "/").casefold().endswith("/scripts/activate.ps1"):
return ep
bash_path = _git_bash_path(path)
if "\\" in bash_path:
return ep
bash_path = bash_path[: -len("Activate.ps1")] + "activate"
return "source " + shlex.quote(bash_path)
def _ssh_ps(host, script_path, port=None): def _ssh_ps(host, script_path, port=None):
"""Build SSH command to run a PowerShell script on a Windows remote.""" """Build SSH command to run a PowerShell script on a Windows remote."""
pf = f"-p {port} " if port and port != "22" else "" pf = f"-p {port} " if port and port != "22" else ""
+3 -3
View File
@@ -50,7 +50,7 @@ from routes.cookbook_helpers import (
_SESSION_ID_RE, _validate_repo_id, _validate_serve_model_id, _validate_include, _validate_token, _SESSION_ID_RE, _validate_repo_id, _validate_serve_model_id, _validate_include, _validate_token,
_validate_local_dir, _validate_gpus, _shell_path, _validate_local_dir, _validate_gpus, _shell_path,
_ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase, OLLAMA_MISSING_HINT, _ps_squote, _bash_squote, _validate_serve_cmd, _parse_serve_phase, OLLAMA_MISSING_HINT,
_safe_env_prefix, _local_windows_bash_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines, _safe_env_prefix, _local_tooling_path_export, _append_serve_preflight_exit_lines,
_append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script, _append_serve_exit_code_lines, _append_llama_cpp_linux_accel_build_lines, _cached_model_scan_script,
load_stored_hf_token, load_stored_hf_token,
_append_vllm_linux_preflight_lines, _ollama_bind_from_cmd, _pip_install_fallback_chain, _append_vllm_linux_preflight_lines, _ollama_bind_from_cmd, _pip_install_fallback_chain,
@@ -1336,7 +1336,7 @@ def setup_cookbook_routes() -> APIRouter:
# Local: run hf download in the background (tmux on POSIX, a detached # Local: run hf download in the background (tmux on POSIX, a detached
# process + logfile on Windows where tmux doesn't exist). # process + logfile on Windows where tmux doesn't exist).
if req.env_prefix: if req.env_prefix:
lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix)) lines.append(_safe_env_prefix(req.env_prefix))
else: else:
lines.append("deactivate 2>/dev/null; hash -r") lines.append("deactivate 2>/dev/null; hash -r")
# Show whether the HF token reached this run (masked) — tells a gated # Show whether the HF token reached this run (masked) — tells a gated
@@ -2166,7 +2166,7 @@ def setup_cookbook_routes() -> APIRouter:
if req.gpus: if req.gpus:
runner_lines.append(f"export CUDA_VISIBLE_DEVICES='{req.gpus}'") runner_lines.append(f"export CUDA_VISIBLE_DEVICES='{req.gpus}'")
if req.env_prefix: if req.env_prefix:
runner_lines.append(_safe_env_prefix(_local_windows_bash_env_prefix(req.env_prefix) if local_windows else req.env_prefix)) runner_lines.append(_safe_env_prefix(req.env_prefix))
else: else:
runner_lines.append("deactivate 2>/dev/null; hash -r") runner_lines.append("deactivate 2>/dev/null; hash -r")
_append_venv_nvidia_library_path_lines(runner_lines, cmd=req.cmd) _append_venv_nvidia_library_path_lines(runner_lines, cmd=req.cmd)
+1 -10
View File
@@ -34,7 +34,7 @@ from fastapi import Query, HTTPException, Request
from pydantic import BaseModel from pydantic import BaseModel
from typing import Optional, List from typing import Optional, List
from src.auth_helpers import _auth_disabled, get_current_user, is_bearer_principal from src.auth_helpers import _auth_disabled, get_current_user
from src.secret_storage import decrypt as _decrypt from src.secret_storage import decrypt as _decrypt
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -420,15 +420,6 @@ def _require_auth(request: Request) -> str:
unconfigured mode are only honoured if they're coming from unconfigured mode are only honoured if they're coming from
localhost; everyone else gets 401. localhost; everyone else gets 401.
""" """
# The legacy email router uses one generic dependency for mailbox reads,
# drafts, AI helpers, and SMTP send. It has no per-route token-scope
# contract, so a bearer must not be allowed to enter it as the ``api``
# pseudo-user. Otherwise owner-scoped lookup can miss the token owner and
# fall through to process-wide legacy settings credentials below.
# Scope-aware integrations must use their dedicated route boundary.
if is_bearer_principal(request):
raise HTTPException(403, "API tokens must use a scope-aware email route")
u = get_current_user(request) u = get_current_user(request)
if u: if u:
return u return u
+10 -3
View File
@@ -5004,6 +5004,7 @@ def setup_email_routes():
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_endpoint, resolve_endpoint,
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
@@ -5065,6 +5066,8 @@ def setup_email_routes():
pass pass
for cand in resolve_utility_fallback_candidates(owner=owner) or []: for cand in resolve_utility_fallback_candidates(owner=owner) or []:
_add(*cand) _add(*cand)
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
_add(*cand)
if not candidates: if not candidates:
return {"success": False, "error": "No LLM endpoint configured"} return {"success": False, "error": "No LLM endpoint configured"}
@@ -5324,11 +5327,13 @@ def setup_email_routes():
# Build a candidate chain so a stale session-stored API key # Build a candidate chain so a stale session-stored API key
# (the most common cause of "authentication failed" here) # (the most common cause of "authentication failed" here)
# doesn't kill AI Reply outright — fall through to the # doesn't kill AI Reply outright — fall through to the
# user's Utility / Default endpoints and active Utility fallback # user's Utility / Default endpoints and the active Utility
# chain. Dedupe by url+model so we don't retry the same endpoint. # fallback chain. The retired default-fallback hook stays empty.
# Dedupe by url+model so we don't retry the same broken endpoint.
from src.llm_core import llm_call_async_with_fallback from src.llm_core import llm_call_async_with_fallback
from src.endpoint_resolver import ( from src.endpoint_resolver import (
resolve_utility_fallback_candidates, resolve_utility_fallback_candidates,
resolve_chat_fallback_candidates,
) )
_seen = set() _seen = set()
_candidates = [] _candidates = []
@@ -5353,9 +5358,11 @@ def setup_email_routes():
_add(_d_url, _d_model, _d_headers) _add(_d_url, _d_model, _d_headers)
except Exception: except Exception:
pass pass
# Active Utility fallbacks last. # Active Utility fallbacks, then the retired default hook.
for cand in resolve_utility_fallback_candidates(owner=owner) or []: for cand in resolve_utility_fallback_candidates(owner=owner) or []:
_add(*cand) _add(*cand)
for cand in resolve_chat_fallback_candidates(owner=owner) or []:
_add(*cand)
_messages = [ _messages = [
{"role": "system", "content": system_prompt}, {"role": "system", "content": system_prompt},
{"role": "user", "content": user_msg}, {"role": "user", "content": user_msg},
+38 -92
View File
@@ -10,19 +10,11 @@ import uuid
from pathlib import Path from pathlib import Path
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
from fastapi import APIRouter, Depends, HTTPException, Query, Request from fastapi import APIRouter, HTTPException, Query, Request
from core.database import SessionLocal, GalleryImage, GalleryAlbum, ModelEndpoint from core.database import SessionLocal, GalleryImage, GalleryAlbum, ModelEndpoint
from core.database import Session as DbSession from core.database import Session as DbSession
from src.auth_helpers import ( from src.auth_helpers import get_current_user, owner_filter, require_privilege
effective_user,
get_current_user,
is_bearer_principal,
owner_filter,
require_chat_scope,
require_non_bearer_request,
require_privilege,
)
from src.upload_limits import ( from src.upload_limits import (
read_upload_limited, read_upload_limited,
GALLERY_UPLOAD_MAX_BYTES, GALLERY_UPLOAD_MAX_BYTES,
@@ -41,13 +33,6 @@ _SAM_STATE: Dict[str, Any] = {}
_GROUNDING_STATE: Dict[str, Any] = {} _GROUNDING_STATE: Dict[str, Any] = {}
def _gallery_owner(request: Request) -> Optional[str]:
"""Use the token owner for bearer calls and preserve the legacy seam otherwise."""
if is_bearer_principal(request):
return effective_user(request)
return get_current_user(request)
def _b64_to_pil_image(image_b64: str, *, mode: str = "RGBA"): def _b64_to_pil_image(image_b64: str, *, mode: str = "RGBA"):
if not image_b64: if not image_b64:
raise HTTPException(400, "Missing image") raise HTTPException(400, "Missing image")
@@ -361,10 +346,7 @@ async def _fetch_result_image_b64(url: str) -> Optional[str]:
def setup_gallery_routes() -> APIRouter: def setup_gallery_routes() -> APIRouter:
router = APIRouter( router = APIRouter(tags=["gallery"])
tags=["gallery"],
dependencies=[Depends(require_chat_scope)],
)
# ---- POST /api/gallery/upload ---- # ---- POST /api/gallery/upload ----
@router.post("/api/gallery/upload") @router.post("/api/gallery/upload")
@@ -378,7 +360,7 @@ def setup_gallery_routes() -> APIRouter:
if not file or not hasattr(file, 'filename'): if not file or not hasattr(file, 'filename'):
raise HTTPException(400, "No file provided") raise HTTPException(400, "No file provided")
user = _gallery_owner(request) user = get_current_user(request)
album_id = form.get("album_id") or None album_id = form.get("album_id") or None
content = await read_upload_limited(file, GALLERY_UPLOAD_MAX_BYTES, "Gallery upload") content = await read_upload_limited(file, GALLERY_UPLOAD_MAX_BYTES, "Gallery upload")
@@ -452,7 +434,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/{image_id}/replace") @router.post("/api/gallery/{image_id}/replace")
async def gallery_replace(request: Request, image_id: str): async def gallery_replace(request: Request, image_id: str):
"""Replace an existing gallery image file with a new one.""" """Replace an existing gallery image file with a new one."""
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first() img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first()
@@ -497,7 +479,7 @@ def setup_gallery_routes() -> APIRouter:
"""Rename a gallery photo. Stores the new name in the `prompt` """Rename a gallery photo. Stores the new name in the `prompt`
column (which serves as the user-facing label for uploaded column (which serves as the user-facing label for uploaded
photos that have no AI prompt).""" photos that have no AI prompt)."""
user = _gallery_owner(request) user = get_current_user(request)
data = await request.json() data = await request.json()
new_name = (data.get("name") or "").strip() new_name = (data.get("name") or "").strip()
if not new_name: if not new_name:
@@ -534,7 +516,7 @@ def setup_gallery_routes() -> APIRouter:
if angle not in (90, -90, 180, 270): if angle not in (90, -90, 180, 270):
raise HTTPException(400, "Angle must be 90, -90, 180, or 270") raise HTTPException(400, "Angle must be 90, -90, 180, or 270")
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first() img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first()
@@ -575,10 +557,7 @@ def setup_gallery_routes() -> APIRouter:
db.close() db.close()
# ---- POST /api/gallery/ai-upscale ---- # ---- POST /api/gallery/ai-upscale ----
@router.post( @router.post("/api/gallery/ai-upscale")
"/api/gallery/ai-upscale",
dependencies=[Depends(require_non_bearer_request)],
)
async def gallery_ai_upscale(request: Request): async def gallery_ai_upscale(request: Request):
"""AI upscale using img2img with the diffusion server.""" """AI upscale using img2img with the diffusion server."""
import base64, httpx import base64, httpx
@@ -622,10 +601,7 @@ def setup_gallery_routes() -> APIRouter:
return {"error": "Upscale request failed"} return {"error": "Upscale request failed"}
# ---- POST /api/gallery/style-transfer ---- # ---- POST /api/gallery/style-transfer ----
@router.post( @router.post("/api/gallery/style-transfer")
"/api/gallery/style-transfer",
dependencies=[Depends(require_non_bearer_request)],
)
async def gallery_style_transfer(request: Request): async def gallery_style_transfer(request: Request):
"""Style transfer using img2img with the diffusion server.""" """Style transfer using img2img with the diffusion server."""
import base64, httpx import base64, httpx
@@ -675,7 +651,7 @@ def setup_gallery_routes() -> APIRouter:
@router.get("/api/gallery/tags") @router.get("/api/gallery/tags")
async def gallery_tags(request: Request) -> Dict[str, Any]: async def gallery_tags(request: Request) -> Dict[str, Any]:
"""Return distinct tags across all active gallery images.""" """Return distinct tags across all active gallery images."""
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryImage.tags).filter( q = db.query(GalleryImage.tags).filter(
@@ -707,7 +683,7 @@ def setup_gallery_routes() -> APIRouter:
offset: int = Query(0, ge=0), offset: int = Query(0, ge=0),
limit: int = Query(24, ge=1, le=100), limit: int = Query(24, ge=1, le=100),
) -> Dict[str, Any]: ) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
# Distinct tags for filter UI # Distinct tags for filter UI
@@ -835,7 +811,7 @@ def setup_gallery_routes() -> APIRouter:
@router.get("/api/gallery/albums") @router.get("/api/gallery/albums")
async def list_albums(request: Request): async def list_albums(request: Request):
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryAlbum) q = db.query(GalleryAlbum)
@@ -874,7 +850,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/albums") @router.post("/api/gallery/albums")
async def create_album(request: Request): async def create_album(request: Request):
import uuid import uuid
user = _gallery_owner(request) user = get_current_user(request)
data = await request.json() data = await request.json()
name = (data.get("name") or "").strip() name = (data.get("name") or "").strip()
if not name: if not name:
@@ -894,7 +870,7 @@ def setup_gallery_routes() -> APIRouter:
@router.get("/api/gallery/stats") @router.get("/api/gallery/stats")
async def gallery_stats(request: Request): async def gallery_stats(request: Request):
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
from sqlalchemy import func from sqlalchemy import func
@@ -918,16 +894,13 @@ def setup_gallery_routes() -> APIRouter:
finally: finally:
db.close() db.close()
@router.post( @router.post("/api/gallery/ai-tag-batch")
"/api/gallery/ai-tag-batch",
dependencies=[Depends(require_non_bearer_request)],
)
async def ai_tag_batch( async def ai_tag_batch(
request: Request, request: Request,
album_id: Optional[str] = Query(None), album_id: Optional[str] = Query(None),
limit: int = Query(200), limit: int = Query(200),
): ):
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryImage).filter( q = db.query(GalleryImage).filter(
@@ -946,7 +919,7 @@ def setup_gallery_routes() -> APIRouter:
# ---- GET /api/gallery/{image_id} ---- # ---- GET /api/gallery/{image_id} ----
@router.get("/api/gallery/{image_id}") @router.get("/api/gallery/{image_id}")
async def get_gallery_image(request: Request, image_id: str) -> Dict[str, Any]: async def get_gallery_image(request: Request, image_id: str) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
row = ( row = (
@@ -967,7 +940,7 @@ def setup_gallery_routes() -> APIRouter:
# ---- PATCH /api/gallery/{image_id} ---- # ---- PATCH /api/gallery/{image_id} ----
@router.patch("/api/gallery/{image_id}") @router.patch("/api/gallery/{image_id}")
async def patch_gallery_image(request: Request, image_id: str, req: GalleryPatch) -> Dict[str, Any]: async def patch_gallery_image(request: Request, image_id: str, req: GalleryPatch) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first() img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first()
@@ -1019,7 +992,7 @@ def setup_gallery_routes() -> APIRouter:
# of a flood of individual downloads). # of a flood of individual downloads).
@router.post("/api/gallery/download-zip") @router.post("/api/gallery/download-zip")
async def gallery_download_zip(request: Request): async def gallery_download_zip(request: Request):
user = _gallery_owner(request) user = get_current_user(request)
if not user: if not user:
raise HTTPException(401, "Not authenticated") raise HTTPException(401, "Not authenticated")
try: try:
@@ -1074,7 +1047,7 @@ def setup_gallery_routes() -> APIRouter:
# AI-suggested values you never added. # AI-suggested values you never added.
@router.post("/api/gallery/clear-user-tags") @router.post("/api/gallery/clear-user-tags")
async def clear_gallery_user_tags(request: Request) -> Dict[str, Any]: async def clear_gallery_user_tags(request: Request) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryImage).filter(GalleryImage.is_active == True) q = db.query(GalleryImage).filter(GalleryImage.is_active == True)
@@ -1099,7 +1072,7 @@ def setup_gallery_routes() -> APIRouter:
# "woman" have leaked into the gallery and you want them gone. # "woman" have leaked into the gallery and you want them gone.
@router.post("/api/gallery/clear-ai-tags") @router.post("/api/gallery/clear-ai-tags")
async def clear_gallery_ai_tags(request: Request, image_id: Optional[str] = Query(None)) -> Dict[str, Any]: async def clear_gallery_ai_tags(request: Request, image_id: Optional[str] = Query(None)) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryImage).filter(GalleryImage.is_active == True) q = db.query(GalleryImage).filter(GalleryImage.is_active == True)
@@ -1126,7 +1099,7 @@ def setup_gallery_routes() -> APIRouter:
# Returns how many rows were touched + how many tags removed. # Returns how many rows were touched + how many tags removed.
@router.post("/api/gallery/dedupe-tags") @router.post("/api/gallery/dedupe-tags")
async def dedupe_gallery_tags(request: Request) -> Dict[str, Any]: async def dedupe_gallery_tags(request: Request) -> Dict[str, Any]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
q = db.query(GalleryImage).filter(GalleryImage.is_active == True) q = db.query(GalleryImage).filter(GalleryImage.is_active == True)
@@ -1162,7 +1135,7 @@ def setup_gallery_routes() -> APIRouter:
# ---- DELETE /api/gallery/{image_id} ---- # ---- DELETE /api/gallery/{image_id} ----
@router.delete("/api/gallery/{image_id}") @router.delete("/api/gallery/{image_id}")
async def delete_gallery_image(request: Request, image_id: str) -> Dict[str, str]: async def delete_gallery_image(request: Request, image_id: str) -> Dict[str, str]:
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first() img = db.query(GalleryImage).filter(GalleryImage.id == image_id).first()
@@ -1281,10 +1254,7 @@ def setup_gallery_routes() -> APIRouter:
db.close() db.close()
# ---- POST /api/image/inpaint — proxy to diffusion server OR OpenAI ---- # ---- POST /api/image/inpaint — proxy to diffusion server OR OpenAI ----
@router.post( @router.post("/api/image/inpaint")
"/api/image/inpaint",
dependencies=[Depends(require_non_bearer_request)],
)
async def inpaint_proxy(request: Request): async def inpaint_proxy(request: Request):
"""Forward inpaint request. If the selected endpoint is OpenAI, re-shape """Forward inpaint request. If the selected endpoint is OpenAI, re-shape
the request for /v1/images/edits (multipart, inverted mask). Otherwise the request for /v1/images/edits (multipart, inverted mask). Otherwise
@@ -1542,10 +1512,7 @@ def setup_gallery_routes() -> APIRouter:
# scratch using the prompt", ignoring the source. Real img2img sends # scratch using the prompt", ignoring the source. Real img2img sends
# the image alongside a `strength` (denoising strength) and the model # the image alongside a `strength` (denoising strength) and the model
# mixes that fraction of new noise into the existing pixels. # mixes that fraction of new noise into the existing pixels.
@router.post( @router.post("/api/image/harmonize")
"/api/image/harmonize",
dependencies=[Depends(require_non_bearer_request)],
)
async def harmonize_image(request: Request): async def harmonize_image(request: Request):
"""Harmonize = img2img. The model preserves (1 - strength) of the """Harmonize = img2img. The model preserves (1 - strength) of the
original and regenerates `strength` fraction. With strength ~0.4 original and regenerates `strength` fraction. With strength ~0.4
@@ -1745,10 +1712,7 @@ def setup_gallery_routes() -> APIRouter:
"/v1/images/harmonize, /v1/images/img2img, /v1/images/variations, /sdapi/v1/img2img.") "/v1/images/harmonize, /v1/images/img2img, /v1/images/variations, /sdapi/v1/img2img.")
# ---- POST /api/image/sharpen ---- # ---- POST /api/image/sharpen ----
@router.post( @router.post("/api/image/sharpen")
"/api/image/sharpen",
dependencies=[Depends(require_non_bearer_request)],
)
async def sharpen_image(request: Request): async def sharpen_image(request: Request):
"""Apply unsharp-mask sharpening to an image.""" """Apply unsharp-mask sharpening to an image."""
require_privilege(request, "can_generate_images") require_privilege(request, "can_generate_images")
@@ -1773,10 +1737,7 @@ def setup_gallery_routes() -> APIRouter:
# AI denoise via Real-ESRGAN with the realesr-general-x4v3 weights at # AI denoise via Real-ESRGAN with the realesr-general-x4v3 weights at
# outscale=1 + denoise_strength. Falls back to a "package missing" # outscale=1 + denoise_strength. Falls back to a "package missing"
# error so the client can prompt the user to install via Cookbook. # error so the client can prompt the user to install via Cookbook.
@router.post( @router.post("/api/image/denoise")
"/api/image/denoise",
dependencies=[Depends(require_non_bearer_request)],
)
async def denoise_image(request: Request): async def denoise_image(request: Request):
require_privilege(request, "can_generate_images") require_privilege(request, "can_generate_images")
body = await request.json() body = await request.json()
@@ -1827,10 +1788,7 @@ def setup_gallery_routes() -> APIRouter:
# ---- POST /api/image/upscale-local ---- # ---- POST /api/image/upscale-local ----
# Local Real-ESRGAN upscale (2× or 4×). Self-contained — no diffusion # Local Real-ESRGAN upscale (2× or 4×). Self-contained — no diffusion
# server required. Used by the editor's AI Upscale button. # server required. Used by the editor's AI Upscale button.
@router.post( @router.post("/api/image/upscale-local")
"/api/image/upscale-local",
dependencies=[Depends(require_non_bearer_request)],
)
async def upscale_image_local(request: Request): async def upscale_image_local(request: Request):
require_privilege(request, "can_generate_images") require_privilege(request, "can_generate_images")
body = await request.json() body = await request.json()
@@ -1876,10 +1834,7 @@ def setup_gallery_routes() -> APIRouter:
return {"error": "AI upscale failed"} return {"error": "AI upscale failed"}
# ---- POST /api/image/remove-bg ---- # ---- POST /api/image/remove-bg ----
@router.post( @router.post("/api/image/mask")
"/api/image/mask",
dependencies=[Depends(require_non_bearer_request)],
)
async def smart_mask(request: Request): async def smart_mask(request: Request):
"""Create a neutral segmentation mask from user-provided points or a box. """Create a neutral segmentation mask from user-provided points or a box.
@@ -2005,10 +1960,7 @@ def setup_gallery_routes() -> APIRouter:
logger.exception("smart_mask failed") logger.exception("smart_mask failed")
raise HTTPException(500, f"SAM mask failed: {exc}") from exc raise HTTPException(500, f"SAM mask failed: {exc}") from exc
@router.post( @router.post("/api/image/remove-bg")
"/api/image/remove-bg",
dependencies=[Depends(require_non_bearer_request)],
)
async def remove_background(request: Request): async def remove_background(request: Request):
"""Remove background from an image. If the client passes a `hint_mask` """Remove background from an image. If the client passes a `hint_mask`
(white-where-the-user-wants-the-subject PNG, same dims as the (white-where-the-user-wants-the-subject PNG, same dims as the
@@ -2101,10 +2053,7 @@ def setup_gallery_routes() -> APIRouter:
return {"image": base64.b64encode(buf.getvalue()).decode()} return {"image": base64.b64encode(buf.getvalue()).decode()}
# ---- POST /api/image/enhance-face ---- # ---- POST /api/image/enhance-face ----
@router.post( @router.post("/api/image/enhance-face")
"/api/image/enhance-face",
dependencies=[Depends(require_non_bearer_request)],
)
async def enhance_face(request: Request): async def enhance_face(request: Request):
"""Face/portrait enhancement. Uses GFPGAN if available, falls back to PIL.""" """Face/portrait enhancement. Uses GFPGAN if available, falls back to PIL."""
require_privilege(request, "can_generate_images") require_privilege(request, "can_generate_images")
@@ -2190,7 +2139,7 @@ def setup_gallery_routes() -> APIRouter:
@router.put("/api/gallery/albums/{album_id}") @router.put("/api/gallery/albums/{album_id}")
async def update_album(request: Request, album_id: str): async def update_album(request: Request, album_id: str):
user = _gallery_owner(request) user = get_current_user(request)
data = await request.json() data = await request.json()
db = SessionLocal() db = SessionLocal()
try: try:
@@ -2211,7 +2160,7 @@ def setup_gallery_routes() -> APIRouter:
@router.delete("/api/gallery/albums/{album_id}") @router.delete("/api/gallery/albums/{album_id}")
async def delete_album(request: Request, album_id: str): async def delete_album(request: Request, album_id: str):
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
album = _get_or_404_album(db, album_id, user) album = _get_or_404_album(db, album_id, user)
@@ -2227,7 +2176,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/albums/{album_id}/add") @router.post("/api/gallery/albums/{album_id}/add")
async def add_to_album(request: Request, album_id: str): async def add_to_album(request: Request, album_id: str):
user = _gallery_owner(request) user = get_current_user(request)
data = await request.json() data = await request.json()
ids = data.get("image_ids", []) ids = data.get("image_ids", [])
db = SessionLocal() db = SessionLocal()
@@ -2245,7 +2194,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/albums/{album_id}/remove") @router.post("/api/gallery/albums/{album_id}/remove")
async def remove_from_album(request: Request, album_id: str): async def remove_from_album(request: Request, album_id: str):
user = _gallery_owner(request) user = get_current_user(request)
data = await request.json() data = await request.json()
ids = data.get("image_ids", []) ids = data.get("image_ids", [])
db = SessionLocal() db = SessionLocal()
@@ -2266,7 +2215,7 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/{image_id}/favorite") @router.post("/api/gallery/{image_id}/favorite")
async def toggle_favorite(request: Request, image_id: str): async def toggle_favorite(request: Request, image_id: str):
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = _get_or_404_image(db, image_id, user) img = _get_or_404_image(db, image_id, user)
@@ -2278,16 +2227,13 @@ def setup_gallery_routes() -> APIRouter:
# ---- AI auto-tag ---- # ---- AI auto-tag ----
@router.post( @router.post("/api/gallery/{image_id}/ai-tag")
"/api/gallery/{image_id}/ai-tag",
dependencies=[Depends(require_non_bearer_request)],
)
async def ai_tag_image(request: Request, image_id: str): async def ai_tag_image(request: Request, image_id: str):
"""Send image to vision model for auto-tagging.""" """Send image to vision model for auto-tagging."""
import base64, httpx import base64, httpx
from pathlib import Path from pathlib import Path
user = _gallery_owner(request) user = get_current_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
img = _get_or_404_image(db, image_id, user) img = _get_or_404_image(db, image_id, user)
+54 -167
View File
@@ -6,31 +6,19 @@ import logging
import re import re
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
from fastapi import APIRouter, Depends, Request, HTTPException from fastapi import APIRouter, Request, HTTPException
from core.models import ChatMessage from core.models import ChatMessage
from core.database import SessionLocal, ChatMessage as DbChatMessage, Session as DbSession from core.database import SessionLocal, ChatMessage as DbChatMessage, Session as DbSession
from src.auth_helpers import ( from src.auth_helpers import effective_user
effective_user,
is_bearer_principal,
request_capability,
require_chat_scope,
)
from src.message_metadata import (
sanitize_client_message_metadata,
sanitize_projected_message_metadata,
normalize_client_message_role,
)
from src.topic_analyzer import analyze_topics from src.topic_analyzer import analyze_topics
from src.upload_handler import reserve_message_upload_references from src.upload_handler import reserve_message_upload_references
from src.session_provenance import persist_session_endpoint_provenance
from routes.session_routes import ( from routes.session_routes import (
_message_role, _message_role,
_message_text, _message_text,
_reject_compact_during_active_run, _reject_compact_during_active_run,
_verify_session_owner, _verify_session_owner,
) )
from routes.chat_helpers import _validate_bearer_session_model
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -38,24 +26,6 @@ _HISTORY_INLINE_MEDIA_THRESHOLD = 200_000
_DATA_IMAGE_RE = re.compile(r"data:image/[^;,\"]+;base64,[A-Za-z0-9+/=\s]+") _DATA_IMAGE_RE = re.compile(r"data:image/[^;,\"]+;base64,[A-Za-z0-9+/=\s]+")
def _metadata_dict(value: Any) -> dict:
"""Return only mapping-shaped message metadata.
Legacy rows and client payloads can contain JSON lists/scalars. They are
display noise, not trusted fields, and must not reach ``dict.update`` or
approval projection code.
"""
if isinstance(value, dict):
return value
if isinstance(value, str):
try:
parsed = json.loads(value)
except (json.JSONDecodeError, TypeError, ValueError):
return {}
return parsed if isinstance(parsed, dict) else {}
return {}
def _history_display_content(content: Any) -> Any: def _history_display_content(content: Any) -> Any:
"""Return a lightweight browser-display copy of stored message content. """Return a lightweight browser-display copy of stored message content.
@@ -131,7 +101,7 @@ def _merge_continue_rows_to_delete(db_messages, db1, db2):
def setup_history_routes(session_manager, upload_handler=None) -> APIRouter: def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
router = APIRouter(tags=["history"], dependencies=[Depends(require_chat_scope)]) router = APIRouter(tags=["history"])
def _reserve_message_uploads( def _reserve_message_uploads(
request: Request, request: Request,
@@ -153,19 +123,14 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
f"Referenced upload is no longer available: {missing_id}", f"Referenced upload is no longer available: {missing_id}",
) )
def _display_metadata(value: Any, *, sanitize: bool) -> dict: def _db_history_entry(m: DbChatMessage) -> Dict[str, Any]:
meta = _metadata_dict(value)
if sanitize:
return sanitize_projected_message_metadata(meta) or {}
return dict(meta)
def _db_history_entry(
m: DbChatMessage,
*,
sanitize: bool = False,
) -> Dict[str, Any]:
entry = {"role": m.role, "content": _history_display_content(m.content)} entry = {"role": m.role, "content": _history_display_content(m.content)}
meta = _display_metadata(m.meta_data, sanitize=sanitize) meta = {}
if m.meta_data:
try:
meta = json.loads(m.meta_data) or {}
except (json.JSONDecodeError, ValueError):
meta = {}
if m.timestamp and "timestamp" not in meta: if m.timestamp and "timestamp" not in meta:
meta["timestamp"] = m.timestamp.isoformat() + "Z" meta["timestamp"] = m.timestamp.isoformat() + "Z"
if meta: if meta:
@@ -179,8 +144,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
limit: Optional[int] = None, limit: Optional[int] = None,
offset: Optional[int] = None, offset: Optional[int] = None,
) -> Dict[str, Any]: ) -> Dict[str, Any]:
require_chat_scope(request)
sanitize_history = is_bearer_principal(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
if limit is not None: if limit is not None:
page_limit = max(1, min(int(limit), 100)) page_limit = max(1, min(int(limit), 100))
@@ -208,11 +171,7 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
.all() .all()
) )
history_dict = [ history_dict = [
entry entry for entry in (_db_history_entry(m) for m in rows)
for entry in (
_db_history_entry(m, sanitize=sanitize_history)
for m in rows
)
if not (entry.get("metadata") or {}).get("hidden") if not (entry.get("metadata") or {}).get("hidden")
] ]
return { return {
@@ -238,29 +197,21 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
for msg in session.history: for msg in session.history:
if isinstance(msg, ChatMessage): if isinstance(msg, ChatMessage):
# Skip hidden messages (e.g. compaction summaries for AI context) # Skip hidden messages (e.g. compaction summaries for AI context)
msg_meta = _display_metadata( if msg.metadata and msg.metadata.get("hidden"):
msg.metadata,
sanitize=sanitize_history,
)
if msg_meta.get("hidden"):
continue continue
entry = {"role": msg.role, "content": _history_display_content(msg.content)} entry = {"role": msg.role, "content": _history_display_content(msg.content)}
if msg_meta: if msg.metadata:
entry["metadata"] = msg_meta entry["metadata"] = msg.metadata
history_dict.append(entry) history_dict.append(entry)
elif isinstance(msg, dict): elif isinstance(msg, dict):
msg_meta = _display_metadata( if msg.get("metadata", {}).get("hidden"):
msg.get("metadata"),
sanitize=sanitize_history,
)
if msg_meta.get("hidden"):
continue continue
entry = { entry = {
"role": msg.get("role", ""), "role": msg.get("role", ""),
"content": _history_display_content(msg.get("content", "")), "content": _history_display_content(msg.get("content", "")),
} }
if msg_meta: if msg.get("metadata"):
entry["metadata"] = msg_meta entry["metadata"] = msg["metadata"]
history_dict.append(entry) history_dict.append(entry)
# Fallback: load from DB if in-memory renders empty. Display only — # Fallback: load from DB if in-memory renders empty. Display only —
@@ -278,11 +229,7 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
) )
# Response excludes hidden messages, matching the in-memory path. # Response excludes hidden messages, matching the in-memory path.
history_dict = [ history_dict = [
entry entry for entry in (_db_history_entry(m) for m in db_messages)
for entry in (
_db_history_entry(m, sanitize=sanitize_history)
for m in db_messages
)
if not (entry.get("metadata") or {}).get("hidden") if not (entry.get("metadata") or {}).get("hidden")
] ]
except Exception as e: except Exception as e:
@@ -299,7 +246,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/truncate") @router.post("/api/session/{session_id}/truncate")
async def truncate_session(request: Request, session_id: str): async def truncate_session(request: Request, session_id: str):
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
@@ -315,15 +261,14 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/message") @router.post("/api/session/{session_id}/message")
async def add_message(request: Request, session_id: str): async def add_message(request: Request, session_id: str):
"""Add a message to a session (for slash command persistence).""" """Add a message to a session (for slash command persistence)."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
role = normalize_client_message_role(body.get("role", "assistant")) role = body.get("role", "assistant")
content = body.get("content", "") content = body.get("content", "")
if not content: if not content:
raise HTTPException(400, "content is required") raise HTTPException(400, "content is required")
metadata = sanitize_client_message_metadata(body.get("metadata")) metadata = body.get("metadata")
_reserve_message_uploads(request, content, metadata) _reserve_message_uploads(request, content, metadata)
msg = ChatMessage(role=role, content=content, metadata=metadata) msg = ChatMessage(role=role, content=content, metadata=metadata)
session_manager.add_message(session_id, msg) session_manager.add_message(session_id, msg)
@@ -334,7 +279,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/delete-messages") @router.post("/api/session/{session_id}/delete-messages")
async def delete_messages(request: Request, session_id: str): async def delete_messages(request: Request, session_id: str):
"""Delete specific messages by DB ID (or legacy index).""" """Delete specific messages by DB ID (or legacy index)."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
@@ -398,7 +342,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/edit-message") @router.post("/api/session/{session_id}/edit-message")
async def edit_message(request: Request, session_id: str): async def edit_message(request: Request, session_id: str):
"""Edit the content of a message by its database ID.""" """Edit the content of a message by its database ID."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
@@ -421,8 +364,9 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
db_msg.content = content db_msg.content = content
meta = {} meta = {}
meta = _metadata_dict(db_msg.meta_data) if db_msg.meta_data:
meta = dict(meta) try: meta = json.loads(db_msg.meta_data)
except (json.JSONDecodeError, ValueError): pass
meta['edited'] = True meta['edited'] = True
db_msg.meta_data = json.dumps(meta) db_msg.meta_data = json.dumps(meta)
@@ -453,7 +397,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/mark-stopped") @router.post("/api/session/{session_id}/mark-stopped")
async def mark_stopped(request: Request, session_id: str): async def mark_stopped(request: Request, session_id: str):
"""Mark the last assistant message as stopped by user.""" """Mark the last assistant message as stopped by user."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
session = session_manager.get_session(session_id) session = session_manager.get_session(session_id)
@@ -462,13 +405,13 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
if (isinstance(msg, ChatMessage) and msg.role == 'assistant') or \ if (isinstance(msg, ChatMessage) and msg.role == 'assistant') or \
(isinstance(msg, dict) and msg.get('role') == 'assistant'): (isinstance(msg, dict) and msg.get('role') == 'assistant'):
if isinstance(msg, ChatMessage): if isinstance(msg, ChatMessage):
if not isinstance(msg.metadata, dict): if not msg.metadata:
msg.metadata = {} msg.metadata = {}
msg.metadata['stopped'] = True msg.metadata['stopped'] = True
if not msg.metadata.get('model'): if not msg.metadata.get('model'):
msg.metadata['model'] = session.model msg.metadata['model'] = session.model
else: else:
if not isinstance(msg.get('metadata'), dict): if 'metadata' not in msg:
msg['metadata'] = {} msg['metadata'] = {}
msg['metadata']['stopped'] = True msg['metadata']['stopped'] = True
if not msg['metadata'].get('model'): if not msg['metadata'].get('model'):
@@ -486,8 +429,11 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
) )
if db_messages: if db_messages:
meta = {} meta = {}
meta = _metadata_dict(db_messages.meta_data) if db_messages.meta_data:
meta = dict(meta) try:
meta = _json.loads(db_messages.meta_data)
except (json.JSONDecodeError, ValueError):
pass
meta['stopped'] = True meta['stopped'] = True
if not meta.get('model'): if not meta.get('model'):
meta['model'] = session.model meta['model'] = session.model
@@ -506,11 +452,10 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/update-last-meta") @router.post("/api/session/{session_id}/update-last-meta")
async def update_last_meta(request: Request, session_id: str): async def update_last_meta(request: Request, session_id: str):
"""Merge metadata into the last assistant message (e.g. save variants).""" """Merge metadata into the last assistant message (e.g. save variants)."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
meta_update = sanitize_client_message_metadata(body.get("metadata", {})) or {} meta_update = body.get("metadata", {})
session = session_manager.get_session(session_id) session = session_manager.get_session(session_id)
# Update in-memory # Update in-memory
@@ -518,11 +463,11 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
if (isinstance(msg, ChatMessage) and msg.role == 'assistant') or \ if (isinstance(msg, ChatMessage) and msg.role == 'assistant') or \
(isinstance(msg, dict) and msg.get('role') == 'assistant'): (isinstance(msg, dict) and msg.get('role') == 'assistant'):
if isinstance(msg, ChatMessage): if isinstance(msg, ChatMessage):
if not isinstance(msg.metadata, dict): if not msg.metadata:
msg.metadata = {} msg.metadata = {}
msg.metadata.update(meta_update) msg.metadata.update(meta_update)
else: else:
if not isinstance(msg.get('metadata'), dict): if 'metadata' not in msg:
msg['metadata'] = {} msg['metadata'] = {}
msg['metadata'].update(meta_update) msg['metadata'].update(meta_update)
break break
@@ -538,7 +483,10 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
.first() .first()
) )
if db_msg: if db_msg:
meta = dict(_metadata_dict(db_msg.meta_data)) meta = {}
if db_msg.meta_data:
try: meta = _json.loads(db_msg.meta_data)
except (json.JSONDecodeError, ValueError): pass
meta.update(meta_update) meta.update(meta_update)
db_msg.meta_data = _json.dumps(meta) db_msg.meta_data = _json.dumps(meta)
db.commit() db.commit()
@@ -555,7 +503,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/merge-last-assistant") @router.post("/api/session/{session_id}/merge-last-assistant")
async def merge_last_assistant(request: Request, session_id: str): async def merge_last_assistant(request: Request, session_id: str):
"""Merge the last two assistant messages into one (for continue).""" """Merge the last two assistant messages into one (for continue)."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
@@ -580,12 +527,8 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
merged_content = content1 + separator + content2 merged_content = content1 + separator + content2
# Merge metadata # Merge metadata
meta1 = dict(_metadata_dict( meta1 = (msg1.metadata if isinstance(msg1, ChatMessage) else msg1.get('metadata')) or {}
msg1.metadata if isinstance(msg1, ChatMessage) else msg1.get('metadata') meta2 = (msg2.metadata if isinstance(msg2, ChatMessage) else msg2.get('metadata')) or {}
))
meta2 = dict(_metadata_dict(
msg2.metadata if isinstance(msg2, ChatMessage) else msg2.get('metadata')
))
merged_meta = {**meta1, **meta2} merged_meta = {**meta1, **meta2}
merged_meta.pop('stopped', None) # no longer stopped after continue merged_meta.pop('stopped', None) # no longer stopped after continue
@@ -649,7 +592,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/fork") @router.post("/api/session/{session_id}/fork")
async def fork_session(request: Request, session_id: str): async def fork_session(request: Request, session_id: str):
"""Create a new session with messages copied up to keep_count.""" """Create a new session with messages copied up to keep_count."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
body = await request.json() body = await request.json()
@@ -666,15 +608,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
if not source: if not source:
raise HTTPException(404, "Session not found") raise HTTPException(404, "Session not found")
source_provenance = getattr(source, "endpoint_provenance", None)
source_endpoint_id = getattr(source, "model_endpoint_id", None)
if (
is_bearer_principal(request)
and (hasattr(source, "endpoint_provenance") or hasattr(source, "model_endpoint_id"))
and source_provenance not in {"registered", "direct"}
):
raise HTTPException(400, "Session endpoint provenance is unavailable")
# Create new session # Create new session
new_id = str(uuid.uuid4()) new_id = str(uuid.uuid4())
fork_name = f"\u2ADD {source.name}" fork_name = f"\u2ADD {source.name}"
@@ -686,14 +619,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
rag=False, rag=False,
owner=getattr(source, 'owner', None), owner=getattr(source, 'owner', None),
) )
if source_provenance in {"registered", "direct"}:
persist_session_endpoint_provenance(
session_manager,
new_id,
new_session,
model_endpoint_id=source_endpoint_id,
endpoint_provenance=source_provenance,
)
# Copy messages up to keep_count # Copy messages up to keep_count
msgs_to_copy = source.history[:keep_count] msgs_to_copy = source.history[:keep_count]
@@ -704,15 +629,12 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
# in-memory messages, corrupting their _db_id and breaking # in-memory messages, corrupting their _db_id and breaking
# edit/delete-by-id on the original conversation. # edit/delete-by-id on the original conversation.
meta = dict(msg.metadata) if isinstance(msg.metadata, dict) else None meta = dict(msg.metadata) if isinstance(msg.metadata, dict) else None
if is_bearer_principal(request):
meta = sanitize_projected_message_metadata(meta)
new_session.add_message(ChatMessage(msg.role, msg.content, meta)) new_session.add_message(ChatMessage(msg.role, msg.content, meta))
if not is_bearer_principal(request): try:
try: from src.event_bus import fire_event
from src.event_bus import fire_event fire_event("session_created", getattr(source, 'owner', None))
fire_event("session_created", getattr(source, 'owner', None)) except Exception:
except Exception: logger.debug("session_created event dispatch failed", exc_info=True)
logger.debug("session_created event dispatch failed", exc_info=True)
return { return {
"status": "ok", "status": "ok",
@@ -728,7 +650,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.get("/api/conversations/topics") @router.get("/api/conversations/topics")
async def get_conversation_topics(request: Request) -> Dict[str, Any]: async def get_conversation_topics(request: Request) -> Dict[str, Any]:
require_chat_scope(request)
from src.auth_helpers import require_user from src.auth_helpers import require_user
user = require_user(request) user = require_user(request)
try: try:
@@ -744,8 +665,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
endpoint estimates the persisted session context so the header can show endpoint estimates the persisted session context so the header can show
when the whole chat is approaching compaction. when the whole chat is approaching compaction.
""" """
require_chat_scope(request)
capability = request_capability(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
session = session_manager.get_session(session_id) session = session_manager.get_session(session_id)
@@ -757,23 +676,16 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
messages = session.get_context_messages() messages = session.get_context_messages()
used = int(estimate_tokens(messages)) used = int(estimate_tokens(messages))
context_kwargs = {} ctx_len = int(get_context_length(session.endpoint_url, session.model) or 0)
if not capability.allow_live_probes:
context_kwargs["allow_live_probes"] = False
ctx_len = int(get_context_length(
session.endpoint_url,
session.model,
**context_kwargs,
) or 0)
pct = round((used / ctx_len) * 100, 1) if ctx_len else 0.0 pct = round((used / ctx_len) * 100, 1) if ctx_len else 0.0
pct = max(0.0, min(100.0, pct)) pct = max(0.0, min(100.0, pct))
visible_messages = sum( visible_messages = sum(
1 for m in session.history 1 for m in session.history
if not _metadata_dict(getattr(m, "metadata", None)).get("hidden") if not (getattr(m, "metadata", None) or {}).get("hidden")
) )
compacted_messages = sum( compacted_messages = sum(
1 for m in session.history 1 for m in session.history
if _metadata_dict(getattr(m, "metadata", None)).get("compacted") if (getattr(m, "metadata", None) or {}).get("compacted")
) )
can_compact = used > 0 can_compact = used > 0
return { return {
@@ -797,8 +709,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
@router.post("/api/session/{session_id}/compact") @router.post("/api/session/{session_id}/compact")
async def compact_session(request: Request, session_id: str): async def compact_session(request: Request, session_id: str):
"""Manually trigger context compaction for a session.""" """Manually trigger context compaction for a session."""
require_chat_scope(request)
capability = request_capability(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
from src.auth_helpers import effective_user from src.auth_helpers import effective_user
owner = effective_user(request) owner = effective_user(request)
@@ -811,21 +721,12 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
try: try:
from src.model_context import estimate_tokens, get_context_length from src.model_context import estimate_tokens, get_context_length
from src.llm_core import llm_call_async from src.llm_core import llm_call_async
from src.endpoint_resolver import resolve_endpoint
if len(session.history) < 6: if len(session.history) < 6:
return {"status": "ok", "message": "Not enough messages to compact"} return {"status": "ok", "message": "Not enough messages to compact"}
if capability.is_bearer: ctx_len = get_context_length(session.endpoint_url, session.model)
_validate_bearer_session_model(session, owner=owner)
context_kwargs = {}
if not capability.allow_live_probes:
context_kwargs["allow_live_probes"] = False
ctx_len = get_context_length(
session.endpoint_url,
session.model,
**context_kwargs,
)
messages_before = session.get_context_messages() messages_before = session.get_context_messages()
used_before = estimate_tokens(messages_before) used_before = estimate_tokens(messages_before)
pct_before = round((used_before / ctx_len) * 100, 1) if ctx_len else 0 pct_before = round((used_before / ctx_len) * 100, 1) if ctx_len else 0
@@ -843,26 +744,15 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
for m in older for m in older
) )
# Use the utility model only for interactive/live-capable callers. # Use utility model if available
# Bearer compaction remains on the selected session route. util_url, util_model, util_headers = resolve_endpoint("utility", owner=owner or None)
if capability.allow_live_probes: compact_url = util_url or session.endpoint_url
from src.endpoint_resolver import resolve_endpoint compact_model = util_model or session.model
compact_headers = util_headers if util_url else session.headers
util_url, util_model, util_headers = resolve_endpoint("utility", owner=owner or None)
compact_url = util_url or session.endpoint_url
compact_model = util_model or session.model
compact_headers = util_headers if util_url else session.headers
else:
compact_url = session.endpoint_url
compact_model = session.model
compact_headers = session.headers
from src.context_compactor import SELF_SUMMARY_SYSTEM_PROMPT, normalize_compaction_summary from src.context_compactor import SELF_SUMMARY_SYSTEM_PROMPT, normalize_compaction_summary
compaction_count = sum(1 for m in session.history if isinstance(m, ChatMessage) and "[Conversation summary" in (m.content or "")) compaction_count = sum(1 for m in session.history if isinstance(m, ChatMessage) and "[Conversation summary" in (m.content or ""))
sys_prompt = SELF_SUMMARY_SYSTEM_PROMPT.replace("{count}", str(len(older))).replace("{n}", str(compaction_count + 1)) sys_prompt = SELF_SUMMARY_SYSTEM_PROMPT.replace("{count}", str(len(older))).replace("{n}", str(compaction_count + 1))
compact_kwargs = {}
if not capability.allow_live_probes:
compact_kwargs["allow_live_probes"] = False
summary = await llm_call_async( summary = await llm_call_async(
compact_url, compact_model, compact_url, compact_model,
[ [
@@ -871,7 +761,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
], ],
temperature=0.2, max_tokens=1024, temperature=0.2, max_tokens=1024,
headers=compact_headers, timeout=30, headers=compact_headers, timeout=30,
**compact_kwargs,
) )
summary = normalize_compaction_summary(summary) summary = normalize_compaction_summary(summary)
@@ -949,8 +838,6 @@ def setup_history_routes(session_manager, upload_handler=None) -> APIRouter:
"after": pct_after, "after": pct_after,
} }
except HTTPException:
raise
except Exception as e: except Exception as e:
logger.error(f"Manual compact error {session_id}: {e}") logger.error(f"Manual compact error {session_id}: {e}")
raise HTTPException(500, str(e)) raise HTTPException(500, str(e))
+6 -19
View File
@@ -5,11 +5,10 @@ import shlex
import subprocess import subprocess
from copy import deepcopy from copy import deepcopy
from fastapi import APIRouter, Depends, HTTPException, Request from fastapi import APIRouter, HTTPException
from core.platform_compat import run_ssh_command from core.platform_compat import run_ssh_command
from routes._validators import validate_remote_host, validate_ssh_port from routes._validators import validate_remote_host, validate_ssh_port
from src.auth_helpers import require_non_bearer_request
# Backends the manual hardware simulator accepts. Must stay a subset of what # Backends the manual hardware simulator accepts. Must stay a subset of what
@@ -181,32 +180,24 @@ def _inspect_model_path(model_path: str, host: str = "", ssh_port: str = "") ->
def setup_hwfit_routes(): def setup_hwfit_routes():
router = APIRouter( router = APIRouter(prefix="/api/hwfit", tags=["hwfit"])
prefix="/api/hwfit",
tags=["hwfit"],
dependencies=[Depends(require_non_bearer_request)],
)
@router.get("/system") @router.get("/system")
def get_system(host: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, request: Request = None): def get_system(host: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False):
"""Detect and return current system hardware info. Pass host=user@server for remote. """Detect and return current system hardware info. Pass host=user@server for remote.
fresh=true bypasses the per-host cache (the Rescan button).""" fresh=true bypasses the per-host cache (the Rescan button)."""
if request is not None:
require_non_bearer_request(request)
from services.hwfit.hardware import detect_system from services.hwfit.hardware import detect_system
host, ssh_port = _validate_detection_target(host, ssh_port) host, ssh_port = _validate_detection_target(host, ssh_port)
return detect_system(host=host, ssh_port=ssh_port, platform=platform, fresh=fresh) return detect_system(host=host, ssh_port=ssh_port, platform=platform, fresh=fresh)
@router.get("/models") @router.get("/models")
def get_models(use_case: str = "", sort: str = "newest", limit: int = 50, search: str = "", host: str = "", quant: str = "", ctx: str = "", gpu_count: str = "", gpu_group: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, refresh_catalog: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False, fit_only: bool = False, request: Request = None): def get_models(use_case: str = "", sort: str = "newest", limit: int = 50, search: str = "", host: str = "", quant: str = "", ctx: str = "", gpu_count: str = "", gpu_group: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, refresh_catalog: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False, fit_only: bool = False):
"""Rank LLM models against detected hardware and return scored results. """Rank LLM models against detected hardware and return scored results.
gpu_count: override GPU count (0 = CPU only, 1-N = simulate N GPUs of the gpu_count: override GPU count (0 = CPU only, 1-N = simulate N GPUs of the
active group). gpu_group: index into system.gpu_groups (the homogeneous active group). gpu_group: index into system.gpu_groups (the homogeneous
pools) to target empty/auto = the largest pool. vLLM can only pools) to target empty/auto = the largest pool. vLLM can only
tensor-parallel across identical GPUs, so we never mix pools. tensor-parallel across identical GPUs, so we never mix pools.
fresh=true bypasses the hardware-detection cache.""" fresh=true bypasses the hardware-detection cache."""
if request is not None:
require_non_bearer_request(request)
from services.hwfit.hardware import detect_system from services.hwfit.hardware import detect_system
from services.hwfit.fit import rank_models from services.hwfit.fit import rank_models
from services.hwfit.models import get_models, model_catalog_path, refresh_dynamic_catalogs from services.hwfit.models import get_models, model_catalog_path, refresh_dynamic_catalogs
@@ -325,7 +316,7 @@ def setup_hwfit_routes():
return payload return payload
@router.get("/profiles") @router.get("/profiles")
def get_serve_profiles(model: str = "", model_path: str = "", host: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, serve_weights_gb: float = 0.0, serve_quant: str = "", request: Request = None): def get_serve_profiles(model: str = "", model_path: str = "", host: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, serve_weights_gb: float = 0.0, serve_quant: str = ""):
"""Compute llama.cpp serve profiles (Quality/Balanced/Speed) for `model` """Compute llama.cpp serve profiles (Quality/Balanced/Speed) for `model`
against the detected hardware on `host` (or local). Returns concrete against the detected hardware on `host` (or local). Returns concrete
flags (n_gpu_layers, n_cpu_moe, cache_type, ctx) the serve UI can apply. flags (n_gpu_layers, n_cpu_moe, cache_type, ctx) the serve UI can apply.
@@ -334,8 +325,6 @@ def setup_hwfit_routes():
catalog (e.g. an ad-hoc HF repo), pass enough hints via a minimal synthetic catalog (e.g. an ad-hoc HF repo), pass enough hints via a minimal synthetic
entry isn't possible here, so we return [] and the UI keeps manual flags. entry isn't possible here, so we return [] and the UI keeps manual flags.
""" """
if request is not None:
require_non_bearer_request(request)
from services.hwfit.hardware import detect_system from services.hwfit.hardware import detect_system
from services.hwfit.models import get_models from services.hwfit.models import get_models
from services.hwfit.profiles import compute_serve_profiles from services.hwfit.profiles import compute_serve_profiles
@@ -421,10 +410,8 @@ def setup_hwfit_routes():
} }
@router.get("/image-models") @router.get("/image-models")
def get_image_models(sort: str = "fit", search: str = "", host: str = "", gpu_count: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False, request: Request = None): def get_image_models(sort: str = "fit", search: str = "", host: str = "", gpu_count: str = "", ssh_port: str = "", platform: str = "", fresh: bool = False, manual_mode: str = "", manual_gpu_count: str = "", manual_vram_gb: str = "", manual_ram_gb: str = "", manual_backend: str = "", ignore_detected_gpu: bool = False, ignore_detected_ram: bool = False):
"""Rank image generation models against detected hardware.""" """Rank image generation models against detected hardware."""
if request is not None:
require_non_bearer_request(request)
from services.hwfit.hardware import detect_system from services.hwfit.hardware import detect_system
from services.hwfit.image_models import rank_image_models from services.hwfit.image_models import rank_image_models
host, ssh_port = _validate_detection_target(host, ssh_port) host, ssh_port = _validate_detection_target(host, ssh_port)
+7 -13
View File
@@ -475,7 +475,7 @@ def setup_mcp_routes(mcp_manager: McpManager):
return RedirectResponse(auth_url) return RedirectResponse(auth_url)
else: else:
# Remote device — show paste-back page # Remote device — show paste-back page
return HTMLResponse(_oauth_authorize_page(auth_url, server_id, redirect_uri)) return HTMLResponse(_oauth_authorize_page(auth_url, server_id, host, redirect_uri))
finally: finally:
db.close() db.close()
@@ -612,13 +612,15 @@ def setup_mcp_routes(mcp_manager: McpManager):
def _oauth_authorize_page( def _oauth_authorize_page(
auth_url: str, auth_url: str,
server_id: str, server_id: str,
redirect_uri: str, host: str,
redirect_uri: str = "http://localhost:7000/api/mcp/oauth/callback",
) -> str: ) -> str:
"""Page with Google sign-in link and URL paste-back form for remote access.""" """Page with Google sign-in link and URL paste-back form for remote access."""
# Escape values interpolated into the page: `server_id` comes from the OAuth # Escape values interpolated into the page: `host` comes from the request
# state and is not trusted. # Host header and `server_id` from the OAuth state — neither is trusted.
auth_url = html.escape(auth_url, quote=True) auth_url = html.escape(auth_url, quote=True)
server_id = html.escape(server_id, quote=True) server_id = html.escape(server_id, quote=True)
host = html.escape(host, quote=True)
redirect_uri = html.escape(redirect_uri, quote=True) redirect_uri = html.escape(redirect_uri, quote=True)
return f"""<!DOCTYPE html> return f"""<!DOCTYPE html>
<html><head> <html><head>
@@ -662,15 +664,7 @@ def _oauth_authorize_page(
</div> </div>
<a class="auth-link" href="{auth_url}" target="_blank" rel="noopener">Sign in with Google</a> <a class="auth-link" href="{auth_url}" target="_blank" rel="noopener">Sign in with Google</a>
<div class="divider"></div> <div class="divider"></div>
<!-- Relative action: the browser resolves it against the origin this page was <form method="POST" action="http://{host}/api/mcp/oauth/exchange/{server_id}">
served from, so the form follows the user through any proxy without the
app having to know the scheme or the host. An absolute http:// action is
blocked as mixed content on exactly the HTTPS deployments that need
paste-back, and request.url.scheme cannot be trusted to spot them
uvicorn only honours X-Forwarded-Proto from a peer in
--forwarded-allow-ips, which defaults to 127.0.0.1 and excludes a proxy
arriving over the Docker bridge. -->
<form method="POST" action="/api/mcp/oauth/exchange/{server_id}">
<p>Paste the URL from your browser after signing in:</p> <p>Paste the URL from your browser after signing in:</p>
<input type="text" name="callback_url" placeholder="{redirect_uri}?code=..." required> <input type="text" name="callback_url" placeholder="{redirect_uri}?code=..." required>
<br><button type="submit">Connect</button> <br><button type="submit">Connect</button>
+2 -12
View File
@@ -1,5 +1,5 @@
# routes/memory_routes.py # routes/memory_routes.py
from fastapi import APIRouter, Depends, Form, HTTPException, Request, UploadFile, File from fastapi import APIRouter, Form, HTTPException, Request, UploadFile, File
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional, List
import json import json
import os import os
@@ -53,19 +53,9 @@ def _load_for_update(memory_manager) -> List[Dict[str, Any]]:
def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionManager, memory_vector=None): def setup_memory_routes(memory_manager: MemoryManager, session_manager: SessionManager, memory_vector=None):
"""Set up memory-related routes.""" """Set up memory-related routes."""
router = APIRouter( router = APIRouter(prefix="/api/memory", tags=["memory"])
prefix="/api/memory",
tags=["memory"],
dependencies=[Depends(require_user)],
)
def _owner(request: Request) -> Optional[str]: def _owner(request: Request) -> Optional[str]:
# Router dependencies do not run when a handler is called directly
# (including through an integration router), so keep the same bearer
# rejection at the owner-resolution seam. ``None`` is retained only
# for legacy unit callers; real ASGI requests always carry Request.
if request is not None:
require_user(request)
return get_current_user(request) return get_current_user(request)
def _assert_session_owner(session_obj, user): def _assert_session_owner(session_obj, user):
+26 -123
View File
@@ -29,12 +29,7 @@ from src.endpoint_resolver import (
build_models_url, build_models_url,
build_headers, build_headers,
) )
from src.auth_helpers import ( from src.auth_helpers import _auth_disabled, effective_user, owner_filter
_auth_disabled,
is_bearer_principal,
owner_filter,
require_chat_scope,
)
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -51,7 +46,6 @@ _ENDPOINT_SETTING_FIELDS = {
} }
_ENDPOINT_FALLBACK_FIELDS = { _ENDPOINT_FALLBACK_FIELDS = {
"foreground_model_fallbacks": "Foreground Model Fallbacks",
"utility_model_fallbacks": "Utility Model Fallbacks", "utility_model_fallbacks": "Utility Model Fallbacks",
"vision_model_fallbacks": "Vision Model Fallbacks", "vision_model_fallbacks": "Vision Model Fallbacks",
} }
@@ -186,12 +180,7 @@ def _clear_user_pref_endpoint_refs(all_prefs: dict, ep_id: str) -> int:
if not isinstance(all_prefs, dict): if not isinstance(all_prefs, dict):
return 0 return 0
users = all_prefs.get("_users") users = all_prefs.get("_users")
# A mixed store can contain auth-disabled foreground policy at the root pref_sets = users.values() if isinstance(users, dict) else [all_prefs]
# alongside named-owner preferences. Both are active namespaces; legacy
# `default_model_fallbacks` remains untouched by the field allowlist.
pref_sets = [all_prefs]
if isinstance(users, dict):
pref_sets.extend(users.values())
cleared_users = 0 cleared_users = 0
for prefs in pref_sets: for prefs in pref_sets:
if isinstance(prefs, dict) and _clear_endpoint_settings_for_endpoint(prefs, ep_id): if isinstance(prefs, dict) and _clear_endpoint_settings_for_endpoint(prefs, ep_id):
@@ -1356,14 +1345,14 @@ def _legacy_visible_api_models(ep) -> List[str]:
def _picker_models_for_endpoint(ep, base_url: str, kind: str): def _picker_models_for_endpoint(ep, base_url: str, kind: str):
"""Return model IDs that should appear in the picker for an endpoint. """Return model IDs that should appear in the picker for an endpoint.
API providers expose remote inventory from /v1/models. Default to that API providers expose remote inventory from /v1/models. Treat that cache as
visible inventory until an explicit pinned-model allow-list is saved. inventory, not approval: only manually pinned API models should appear in
Local/self-hosted endpoints keep the older hide-list behavior. the picker. Local/self-hosted endpoints keep the older hide-list behavior.
""" """
pinned = _normalize_model_ids(getattr(ep, "pinned_models", None)) pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if _picker_requires_pinning(base_url, kind): if _picker_requires_pinning(base_url, kind):
if not _has_explicit_pinned_models(ep): if not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep) pinned = _legacy_visible_api_models(ep) if _hidden_model_ids(ep) else []
return pinned, pinned return pinned, pinned
return _visible_models( return _visible_models(
_cached_model_ids(ep), _cached_model_ids(ep),
@@ -1372,68 +1361,6 @@ def _picker_models_for_endpoint(ep, base_url: str, kind: str):
), pinned ), pinned
def _validate_bearer_model_selection(
ep,
requested_model: Optional[str],
*,
allow_empty: bool = False,
) -> str:
"""Validate a bearer-selected model against the server-owned picker.
Bearer requests cannot perform a provider model probe. Their model choice
must therefore come from the same endpoint-local cache/pin inventory that
the server exposes to the model picker. ``allow_empty`` is used only by
default-chat, where an explicitly empty inventory has a deterministic empty
result rather than an implicit provider alias.
"""
if ep is None:
if allow_empty:
return ""
raise HTTPException(400, "A registered model endpoint is required")
base_url = _normalize_base(getattr(ep, "base_url", "") or "")
kind = _effective_endpoint_kind(ep, base_url)
models, _ = _picker_models_for_endpoint(ep, base_url, kind)
models = [model for model in models if isinstance(model, str) and model.strip()]
requested = str(requested_model or "").strip()
if not requested:
if models:
return models[0]
if allow_empty:
return ""
raise HTTPException(400, "No permitted model is configured for this endpoint")
# A registered local endpoint may intentionally have no persisted
# catalog: local models are operator-controlled and bearer requests
# must not discover them live. Preserve that documented compatibility
# path, while still enforcing any inventory that the server does own
# and rejecting explicitly hidden entries below.
raw_inventory = _merge_model_ids(
_normalize_model_ids(getattr(ep, "cached_models", None)),
_normalize_model_ids(getattr(ep, "pinned_models", None)),
)
hidden = set(_normalize_model_ids(getattr(ep, "hidden_models", None)))
if (
requested
and not raw_inventory
and _classify_endpoint(base_url, kind) == "local"
and requested not in hidden
):
return requested
if requested in models:
return requested
requested_base = os.path.basename(requested.rstrip("/"))
matches = [
model for model in models
if os.path.basename(model.rstrip("/")) == requested_base
]
if len(matches) == 1:
return matches[0]
if len(matches) > 1:
raise HTTPException(400, "Model selection is ambiguous for this endpoint")
raise HTTPException(400, f"Model is not permitted for this endpoint: {requested}")
def _api_key_fingerprint(api_key: Optional[str]) -> str: def _api_key_fingerprint(api_key: Optional[str]) -> str:
"""Stable, non-secret label for distinguishing same-URL credentials.""" """Stable, non-secret label for distinguishing same-URL credentials."""
key = (api_key or "").strip() key = (api_key or "").strip()
@@ -1603,12 +1530,7 @@ def setup_model_routes(model_discovery):
_refresh_inflight["v"] = False _refresh_inflight["v"] = False
threading.Thread(target=_do, daemon=True).start() threading.Thread(target=_do, daemon=True).start()
def _fetch_models( def _fetch_models(owner: str = "", is_admin: bool = False):
owner: str = "",
is_admin: bool = False,
*,
read_only: bool = False,
):
"""Return model list from cached data (instant). Background refresh keeps caches fresh. """Return model list from cached data (instant). Background refresh keeps caches fresh.
SECURITY: filters endpoints by `owner` without this the picker SECURITY: filters endpoints by `owner` without this the picker
@@ -1623,7 +1545,7 @@ def setup_model_routes(model_discovery):
db = SessionLocal() db = SessionLocal()
try: try:
if not read_only and _disable_stale_cookbook_local_endpoints(db): if _disable_stale_cookbook_local_endpoints(db):
_invalidate_models_cache() _invalidate_models_cache()
q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True) q = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True)
if owner and not is_admin: if owner and not is_admin:
@@ -1694,7 +1616,13 @@ def setup_model_routes(model_discovery):
# Require auth; "" is the unconfigured single-user mode, treated as # Require auth; "" is the unconfigured single-user mode, treated as
# "see everything" by _fetch_models. # "see everything" by _fetch_models.
try: try:
owner = require_chat_scope(request) or "" if getattr(request.state, "api_token", False):
scopes = set(getattr(request.state, "api_token_scopes", []) or [])
if "chat" not in scopes:
raise HTTPException(403, "API token is not scoped for chat")
if not getattr(request.state, "api_token_owner", None):
raise HTTPException(403, "API token has no owner")
owner = effective_user(request) or ""
# Reject anonymous in configured deployments — no leaking the model # Reject anonymous in configured deployments — no leaking the model
# list to unauthenticated callers. # list to unauthenticated callers.
@@ -1706,17 +1634,6 @@ def setup_model_routes(model_discovery):
except Exception as e: except Exception as e:
logger.error("Auth gate error in GET /api/models, failing closed: %s", e) logger.error("Auth gate error in GET /api/models, failing closed: %s", e)
raise HTTPException(status_code=500, detail="Internal error") raise HTTPException(status_code=500, detail="Internal error")
bearer = is_bearer_principal(request)
if bearer and (refresh or background):
raise HTTPException(
403,
"API tokens may only read the owner-scoped cached model list",
)
if bearer:
# The bearer-compatible path is deliberately read-only: no global
# admin view, stale-row cleanup, process cache writes, background
# probes, stored endpoint credentials, or refresh state changes.
return _fetch_models(owner=owner, is_admin=False, read_only=True)
# Admins see every endpoint (they manage the global pool); regular # Admins see every endpoint (they manage the global pool); regular
# users get the owner-scoped view. # users get the owner-scoped view.
_is_admin = False _is_admin = False
@@ -2419,7 +2336,9 @@ def setup_model_routes(model_discovery):
else: else:
response.headers["X-Model-Refresh-Status"] = "failed" response.headers["X-Model-Refresh-Status"] = "failed"
response.headers["X-Model-Refresh-Warning"] = "Model refresh failed or returned no models; kept cached models." response.headers["X-Model-Refresh-Warning"] = "Model refresh failed or returned no models; kept cached models."
_, pinned = _picker_models_for_endpoint(ep, base, kind) pinned = _normalize_model_ids(getattr(ep, "pinned_models", None))
if picker_requires_pinning and not _has_explicit_pinned_models(ep):
pinned = _legacy_visible_api_models(ep)
pinned_set = set(pinned) pinned_set = set(pinned)
return [ return [
{ {
@@ -2496,11 +2415,11 @@ def setup_model_routes(model_discovery):
# no per-user default yet, we resolve via the owner-scoped endpoint # no per-user default yet, we resolve via the owner-scoped endpoint
# lookup below (last-resort: first enabled endpoint THIS user owns). # lookup below (last-resort: first enabled endpoint THIS user owns).
# Unauthenticated single-user mode keeps the old behavior. # Unauthenticated single-user mode keeps the old behavior.
# Resolve through the same owner/scope gate as the model picker. In an from src.auth_helpers import get_current_user as _gcu
# auth-disabled process there is no middleware to stamp token state, so try:
# raw bearer detection must still prevent a token from resolving _user = _gcu(request) or ""
# global/admin defaults. except Exception:
_user = require_chat_scope(request) or "" _user = ""
# Admins resolve via the global defaults (they own them, and the # Admins resolve via the global defaults (they own them, and the
# scoped resolution was making the picker disappear for them). # scoped resolution was making the picker disappear for them).
# Regular users get per-user prefs with NO global fallback for the # Regular users get per-user prefs with NO global fallback for the
@@ -2510,12 +2429,7 @@ def setup_model_routes(model_discovery):
_is_admin = False _is_admin = False
try: try:
auth_mgr = getattr(request.app.state, "auth_manager", None) auth_mgr = getattr(request.app.state, "auth_manager", None)
if ( if _user and auth_mgr is not None and getattr(auth_mgr, "is_admin", None):
_user
and not is_bearer_principal(request)
and auth_mgr is not None
and getattr(auth_mgr, "is_admin", None)
):
_is_admin = bool(auth_mgr.is_admin(_user)) _is_admin = bool(auth_mgr.is_admin(_user))
except Exception: except Exception:
_is_admin = False _is_admin = False
@@ -2563,13 +2477,7 @@ def setup_model_routes(model_discovery):
return {"endpoint_id": "", "endpoint_url": "", "model": ""} return {"endpoint_id": "", "endpoint_url": "", "model": ""}
base = _normalize_base(ep.base_url) base = _normalize_base(ep.base_url)
chat_url = build_chat_url(base) chat_url = build_chat_url(base)
if is_bearer_principal(request): if not model and (getattr(ep, "cached_models", None) or getattr(ep, "pinned_models", None)):
model = _validate_bearer_model_selection(
ep,
model,
allow_empty=True,
)
elif not model and (getattr(ep, "cached_models", None) or getattr(ep, "pinned_models", None)):
try: try:
visible = _visible_models(ep.cached_models, getattr(ep, "hidden_models", None), getattr(ep, "pinned_models", None)) visible = _visible_models(ep.cached_models, getattr(ep, "hidden_models", None), getattr(ep, "pinned_models", None))
if visible: if visible:
@@ -2780,13 +2688,8 @@ def setup_model_routes(model_discovery):
# ── Tool management ── # ── Tool management ──
@router.get("/tools") @router.get("/tools")
def list_tools(request: Request): def list_tools():
"""List all available tools with their enabled/disabled status.""" """List all available tools with their enabled/disabled status."""
# Tool inventory is an interactive/agent capability description, not
# part of the narrow bearer chat contract. Cookie/local callers retain
# the historical response.
from src.auth_helpers import require_non_bearer_request
require_non_bearer_request(request)
from src.agent_tools import TOOL_TAGS from src.agent_tools import TOOL_TAGS
settings = _load_settings() settings = _load_settings()
disabled = set(settings.get("disabled_tools", [])) disabled = set(settings.get("disabled_tools", []))
+11 -51
View File
@@ -7,10 +7,6 @@ from src.auth_helpers import get_current_user
from src.constants import USER_PREFS_FILE from src.constants import USER_PREFS_FILE
PREFS_FILE = USER_PREFS_FILE PREFS_FILE = USER_PREFS_FILE
_FOREGROUND_POLICY_KEYS = (
"foreground_fallback_enabled",
"foreground_model_fallbacks",
)
def _load(): def _load():
@@ -30,27 +26,14 @@ def _save(prefs):
def _load_for_user(user: Optional[str] = None) -> dict: def _load_for_user(user: Optional[str] = None) -> dict:
"""Load preferences for a specific user.""" """Load preferences for a specific user."""
all_prefs = _load() all_prefs = _load()
users = all_prefs.get("_users") if "_users" in all_prefs:
if isinstance(users, dict):
if user is None: if user is None:
# Auth disabled — return first user's prefs for backward compat # Auth disabled — return first user's prefs for backward compat
prefs = dict(next(iter(users.values()), {})) users = all_prefs["_users"]
# Foreground fallback consent is never borrowed from a named return dict(next(iter(users.values()), {}))
# owner. Auth-disabled operation has a separate flat/root opt-in return dict(all_prefs["_users"].get(user, {}))
# that remains inert when authentication is enabled again. # Legacy flat format — return as-is
for key in _FOREGROUND_POLICY_KEYS: return dict(all_prefs)
prefs.pop(key, None)
if key in all_prefs:
prefs[key] = all_prefs[key]
return prefs
prefs = users.get(user, {})
return dict(prefs) if isinstance(prefs, dict) else {}
# A legacy flat store belongs only to auth-disabled single-user mode.
# Copying it into the first named user's new `_users` record during an
# auth transition would silently transfer another user's preferences and,
# critically, foreground fallback consent. Named owners therefore start
# with an empty record and must write their own preferences explicitly.
return dict(all_prefs) if user is None else {}
def _save_for_user(user: Optional[str], prefs: dict): def _save_for_user(user: Optional[str], prefs: dict):
@@ -62,40 +45,17 @@ def _save_for_user(user: Optional[str], prefs: dict):
# `prefs` flat would overwrite the whole `_users` map and destroy every # `prefs` flat would overwrite the whole `_users` map and destroy every
# other user's preferences. Instead write back into the same (first) # other user's preferences. Instead write back into the same (first)
# slot _load_for_user(None) reads from, preserving the others. # slot _load_for_user(None) reads from, preserving the others.
users = all_prefs.get("_users") if "_users" in all_prefs:
if isinstance(users, dict): users = all_prefs["_users"]
first_key = next(iter(users), None) first_key = next(iter(users), None)
if first_key is not None: if first_key is not None:
existing_named = users.get(first_key) users[first_key] = prefs
existing_named = (
dict(existing_named)
if isinstance(existing_named, dict)
else {}
)
named_foreground = {
key: existing_named[key]
for key in _FOREGROUND_POLICY_KEYS
if key in existing_named
}
users[first_key] = {
key: value
for key, value in prefs.items()
if key not in _FOREGROUND_POLICY_KEYS
}
users[first_key].update(named_foreground)
for key in _FOREGROUND_POLICY_KEYS:
if key in prefs:
all_prefs[key] = prefs[key]
_save(all_prefs) _save(all_prefs)
return return
_save(prefs) _save(prefs)
return return
if not isinstance(all_prefs.get("_users"), dict): if "_users" not in all_prefs:
# Preserve the flat single-user object as inert legacy data while all_prefs = {"_users": {}}
# creating the first named-owner namespace. In particular, historical
# fallback values must not be deleted or copied into the new owner.
all_prefs = dict(all_prefs)
all_prefs["_users"] = {}
all_prefs["_users"][user] = prefs all_prefs["_users"][user] = prefs
_save(all_prefs) _save(all_prefs)
+6 -9
View File
@@ -9,13 +9,13 @@ from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query, Request from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import HTMLResponse, StreamingResponse from fastapi.responses import HTMLResponse, StreamingResponse
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from core.middleware import INTERNAL_TOOL_USER from core.middleware import INTERNAL_TOOL_USER
from src.endpoint_resolver import resolve_endpoint from src.endpoint_resolver import resolve_endpoint
from src.auth_helpers import _auth_disabled, require_interactive_request from src.auth_helpers import _auth_disabled, get_current_user
from src.owner_identity import REQUEST_SENTINEL_OWNERS from core.auth import RESERVED_USERNAMES
from src.constants import DEEP_RESEARCH_DIR from src.constants import DEEP_RESEARCH_DIR
_SESSION_ID_RE = re.compile(r"^[a-zA-Z0-9-]{1,128}$") _SESSION_ID_RE = re.compile(r"^[a-zA-Z0-9-]{1,128}$")
@@ -207,17 +207,14 @@ def _resolve_endpoint_runtime(ep, owner=None, model: Optional[str] = None):
def setup_research_routes(research_handler, session_manager=None) -> APIRouter: def setup_research_routes(research_handler, session_manager=None) -> APIRouter:
router = APIRouter( router = APIRouter(tags=["research"])
tags=["research"],
dependencies=[Depends(require_interactive_request)],
)
def _require_user(request: Request) -> str: def _require_user(request: Request) -> str:
"""All research endpoints require an authenticated user. Research """All research endpoints require an authenticated user. Research
data isn't owner-scoped in the on-disk JSON yet, so we at least data isn't owner-scoped in the on-disk JSON yet, so we at least
block anonymous access. Multi-tenant deploys should additionally block anonymous access. Multi-tenant deploys should additionally
verify the session belongs to this user.""" verify the session belongs to this user."""
user = require_interactive_request(request) user = get_current_user(request)
if not user: if not user:
if _auth_disabled(): if _auth_disabled():
return "" return ""
@@ -499,7 +496,7 @@ def setup_research_routes(research_handler, session_manager=None) -> APIRouter:
user = require_privilege(request, "can_use_research") user = require_privilege(request, "can_use_research")
if user == INTERNAL_TOOL_USER: if user == INTERNAL_TOOL_USER:
tool_owner = (request.headers.get("X-Odysseus-Owner") or "").strip() tool_owner = (request.headers.get("X-Odysseus-Owner") or "").strip()
if tool_owner and tool_owner not in REQUEST_SENTINEL_OWNERS: if tool_owner and tool_owner not in RESERVED_USERNAMES:
auth_mgr = getattr(request.app.state, "auth_manager", None) auth_mgr = getattr(request.app.state, "auth_manager", None)
if auth_mgr is not None and getattr(auth_mgr, "is_configured", False): if auth_mgr is not None and getattr(auth_mgr, "is_configured", False):
try: try:
+4 -12
View File
@@ -3,14 +3,13 @@
import logging import logging
from typing import Dict, Any from typing import Dict, Any
from fastapi import APIRouter, Depends, Request from fastapi import APIRouter, Request
import time import time
from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO
from services.search.core import _call_provider from services.search.core import _call_provider
from services.search.providers import _get_provider_key, _get_search_instance from services.search.providers import _get_provider_key, _get_search_instance
from src.auth_helpers import require_interactive_request
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -38,14 +37,10 @@ async def _request_values(request: Request) -> Dict[str, Any]:
def setup_search_routes(config) -> APIRouter: def setup_search_routes(config) -> APIRouter:
router = APIRouter( router = APIRouter(tags=["search"])
tags=["search"],
dependencies=[Depends(require_interactive_request)],
)
@router.get("/api/search/config") @router.get("/api/search/config")
async def get_search_settings(request: Request) -> Dict[str, Any]: async def get_search_settings() -> Dict[str, Any]:
require_interactive_request(request)
return get_search_config() return get_search_config()
@router.post("/api/search") @router.post("/api/search")
@@ -54,7 +49,6 @@ def setup_search_routes(config) -> APIRouter:
Used by Compare mode to pre-search once and share results across panes. Used by Compare mode to pre-search once and share results across panes.
""" """
require_interactive_request(request)
values = await _request_values(request) values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip() query = str(values.get("query") or values.get("q") or "").strip()
if not query: if not query:
@@ -72,9 +66,8 @@ def setup_search_routes(config) -> APIRouter:
return {"context": "", "sources": [], "error": str(e)} return {"context": "", "sources": [], "error": str(e)}
@router.get("/api/search/providers") @router.get("/api/search/providers")
async def list_search_providers(request: Request): async def list_search_providers():
"""Return available search providers with config status.""" """Return available search providers with config status."""
require_interactive_request(request)
providers = [] providers = []
for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items(): for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items():
if pid == "disabled": if pid == "disabled":
@@ -94,7 +87,6 @@ def setup_search_routes(config) -> APIRouter:
@router.post("/api/search/query") @router.post("/api/search/query")
async def search_with_provider(request: Request) -> Dict[str, Any]: async def search_with_provider(request: Request) -> Dict[str, Any]:
"""Search using a specific provider. Used by compare search mode.""" """Search using a specific provider. Used by compare search mode."""
require_interactive_request(request)
values = await _request_values(request) values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip() query = str(values.get("query") or values.get("q") or "").strip()
provider = str(values.get("provider") or "").strip() provider = str(values.get("provider") or "").strip()
+69 -195
View File
@@ -4,30 +4,17 @@ import html
import json import json
import uuid import uuid
from datetime import datetime from datetime import datetime
from fastapi import APIRouter, Depends, Form, HTTPException, Response, Request from fastapi import APIRouter, Form, HTTPException, Response, Request
import logging import logging
from core.session_manager import SessionManager from core.session_manager import SessionManager
from core.models import ChatMessage from core.models import ChatMessage
from src.request_models import SessionResponse from src.request_models import SessionResponse
from core.database import Session as DbSession, SessionLocal, Document, GalleryImage, utcnow_naive from core.database import Session as DbSession, SessionLocal, Document, GalleryImage, utcnow_naive
from src.auth_helpers import ( from src.auth_helpers import effective_user, _auth_disabled, owner_filter
effective_user,
_auth_disabled,
is_bearer_principal,
owner_filter,
request_capability,
require_chat_scope,
require_interactive_request,
)
from src.message_metadata import (
normalize_client_message_role,
sanitize_client_message_metadata,
)
from src.session_image_cleanup import _generated_image_path_for_cleanup, session_image_refs from src.session_image_cleanup import _generated_image_path_for_cleanup, session_image_refs
from src.session_actions import is_session_recently_active from src.session_actions import is_session_recently_active
from src.upload_handler import reserve_message_upload_references from src.upload_handler import reserve_message_upload_references
from src.session_provenance import persist_session_endpoint_provenance
def _sanitize_export_filename(name: str) -> str: def _sanitize_export_filename(name: str) -> str:
@@ -137,11 +124,7 @@ def _verify_session_owner(request: Request, session_id: str, session_manager=Non
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter( router = APIRouter(prefix="/api", tags=["sessions"])
prefix="/api",
tags=["sessions"],
dependencies=[Depends(require_chat_scope)],
)
def _current_user_is_admin(request: Request, user: str | None) -> bool: def _current_user_is_admin(request: Request, user: str | None) -> bool:
if not user: if not user:
@@ -170,10 +153,7 @@ def _reject_raw_endpoint_url_for_non_admin(
# Raw URLs make the server dial whatever host the request supplies. For # Raw URLs make the server dial whatever host the request supplies. For
# non-admin users, require a saved endpoint row so normal owner scoping and # non-admin users, require a saved endpoint row so normal owner scoping and
# endpoint validation have already happened. # endpoint validation have already happened.
# A bearer may be attributed to an admin owner for storage and endpoint if user and not _current_user_is_admin(request, user):
# visibility, but it is still not an interactive admin principal. Raw
# endpoint URLs therefore remain unavailable to every bearer request.
if is_bearer_principal(request) or (user and not _current_user_is_admin(request, user)):
raise HTTPException(403, "Choose a registered model endpoint") raise HTTPException(403, "Choose a registered model endpoint")
@@ -240,7 +220,6 @@ def setup_session_routes(
@router.get("/sessions") @router.get("/sessions")
def list_sessions(request: Request): def list_sessions(request: Request):
require_chat_scope(request)
user = effective_user(request) user = effective_user(request)
active_incognito_id = str(request.query_params.get("active_incognito_id") or "").strip() active_incognito_id = str(request.query_params.get("active_incognito_id") or "").strip()
# Lazy purge: incognito sessions are ephemeral by design — wipe leftovers # Lazy purge: incognito sessions are ephemeral by design — wipe leftovers
@@ -252,37 +231,32 @@ def setup_session_routes(
# session is current and won't delete the live one — this server-side # session is current and won't delete the live one — this server-side
# purge exists only to catch ghosts the frontend missed (tab close, # purge exists only to catch ghosts the frontend missed (tab close,
# crash). Only clean up rows old enough to be definitely orphaned. # crash). Only clean up rows old enough to be definitely orphaned.
# Listing is an owner-scoped read for bearer integrations. The legacy try:
# incognito cleanup query has no owner predicate and would otherwise from datetime import timedelta as _td
# let a chat token mutate another user's stale sessions before the _cutoff = utcnow_naive() - _td(minutes=10)
# owner-filtered result is assembled. Browser cleanup remains intact. _purge_db = SessionLocal()
if not is_bearer_principal(request):
try: try:
from datetime import timedelta as _td from core.database import ChatMessage as _DbMsg
_cutoff = utcnow_naive() - _td(minutes=10) _ghosts = _purge_db.query(DbSession).filter(
_purge_db = SessionLocal() DbSession.name.in_(("Nobody", "Incognito")),
try: DbSession.created_at < _cutoff,
from core.database import ChatMessage as _DbMsg ).all()
_ghosts = _purge_db.query(DbSession).filter( for _g in _ghosts:
DbSession.name.in_(("Nobody", "Incognito")), if active_incognito_id and _g.id == active_incognito_id:
DbSession.created_at < _cutoff, continue
).all() _purge_db.query(_DbMsg).filter(_DbMsg.session_id == _g.id).delete()
for _g in _ghosts: _purge_db.delete(_g)
if active_incognito_id and _g.id == active_incognito_id: if hasattr(session_manager, "delete_session"):
continue try:
_purge_db.query(_DbMsg).filter(_DbMsg.session_id == _g.id).delete() session_manager.delete_session(_g.id)
_purge_db.delete(_g) except Exception:
if hasattr(session_manager, "delete_session"): pass
try: if _ghosts:
session_manager.delete_session(_g.id) _purge_db.commit()
except Exception: finally:
pass _purge_db.close()
if _ghosts: except Exception:
_purge_db.commit() pass
finally:
_purge_db.close()
except Exception:
pass
user_sessions = session_manager.get_sessions_for_user(user) user_sessions = session_manager.get_sessions_for_user(user)
# Fetch folder info from DB for each session # Fetch folder info from DB for each session
db = SessionLocal() db = SessionLocal()
@@ -364,14 +338,10 @@ def setup_session_routes(
api_key: str = Form(""), api_key: str = Form(""),
endpoint_id: str = Form(""), endpoint_id: str = Form(""),
): ):
require_chat_scope(request)
capability = request_capability(request)
probe_kwargs = {} if capability.allow_live_probes else {"allow_live_probes": False}
skip_val = str(skip_validation).lower() == "true" skip_val = str(skip_validation).lower() == "true"
user = effective_user(request) user = effective_user(request)
endpoint_api_key = "" endpoint_api_key = ""
endpoint_base_url = "" endpoint_base_url = ""
endpoint_row = None
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url) _reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
if endpoint_id and endpoint_id.strip(): if endpoint_id and endpoint_id.strip():
from core.database import ModelEndpoint from core.database import ModelEndpoint
@@ -405,14 +375,7 @@ def setup_session_routes(
from src.endpoint_resolver import build_headers from src.endpoint_resolver import build_headers
validation_headers = build_headers(effective_api_key, endpoint_base_url or endpoint_url) validation_headers = build_headers(effective_api_key, endpoint_base_url or endpoint_url)
if is_bearer_principal(request) and endpoint_row is not None: if skip_val:
# Bearer requests are cache-only, but cache-only does not mean
# caller-authorized. Validate explicit selections and choose an
# empty selection deterministically from the server-owned picker.
from routes.model_routes import _validate_bearer_model_selection
model_to_use = _validate_bearer_model_selection(endpoint_row, model_to_use)
elif skip_val:
# skip_validation = trust the caller and do NOT probe /v1/models. # skip_validation = trust the caller and do NOT probe /v1/models.
# Used for custom endpoints AND for bare placeholder sessions with no # Used for custom endpoints AND for bare placeholder sessions with no
# model at all (e.g. an email reply draft just needs a session to live # model at all (e.g. an email reply draft just needs a session to live
@@ -426,7 +389,6 @@ def setup_session_routes(
headers=validation_headers, headers=validation_headers,
owner=user, owner=user,
endpoint_id=endpoint_id.strip() if endpoint_id else None, endpoint_id=endpoint_id.strip() if endpoint_id else None,
**probe_kwargs,
) )
if not ids: if not ids:
raise HTTPException(400, "Cannot reach /v1/models") raise HTTPException(400, "Cannot reach /v1/models")
@@ -438,35 +400,28 @@ def setup_session_routes(
chat_ids = [m for m in ids if not any(p in m.lower() for p in _NON_CHAT)] chat_ids = [m for m in ids if not any(p in m.lower() for p in _NON_CHAT)]
model_to_use = (chat_ids or ids)[0] model_to_use = (chat_ids or ids)[0]
else: else:
# A bearer with an explicit model is already using an owner-scoped from src.llm_core import list_model_ids
# registered endpoint (raw URLs are rejected above). Do not turn import os as _os
# that synchronous session-creation request into a live catalog req_base = _os.path.basename(model_to_use.rstrip("/"))
# probe merely to validate a value the caller supplied. Interactive avail = list_model_ids(
# requests retain the existing catalog-backed validation. endpoint_url,
if capability.allow_live_probes: timeout=SESSION_MODEL_VALIDATION_TIMEOUT,
from src.llm_core import list_model_ids headers=validation_headers,
import os as _os owner=user,
req_base = _os.path.basename(model_to_use.rstrip("/")) endpoint_id=endpoint_id.strip() if endpoint_id else None,
avail = list_model_ids( )
endpoint_url, if not avail:
timeout=SESSION_MODEL_VALIDATION_TIMEOUT, raise HTTPException(400, "Cannot reach /v1/models")
headers=validation_headers, if model_to_use not in avail:
owner=user, found = None
endpoint_id=endpoint_id.strip() if endpoint_id else None, for a in avail:
**probe_kwargs, if _os.path.basename(a.rstrip("/")) == req_base:
) found = a
if not avail: break
raise HTTPException(400, "Cannot reach /v1/models") if not found:
if model_to_use not in avail: raise HTTPException(400,
found = None f"Model not found at server. Available: {', '.join(avail)}")
for a in avail: model_to_use = found
if _os.path.basename(a.rstrip("/")) == req_base:
found = a
break
if not found:
raise HTTPException(400,
f"Model not found at server. Available: {', '.join(avail)}")
model_to_use = found
sid = str(uuid.uuid4()) sid = str(uuid.uuid4())
user = effective_user(request) user = effective_user(request)
@@ -478,20 +433,6 @@ def setup_session_routes(
rag=str(rag).lower() == "true" if rag else False, rag=str(rag).lower() == "true" if rag else False,
owner=user, owner=user,
) )
if endpoint_row is not None or request_api_key:
try:
persist_session_endpoint_provenance(
session_manager,
sid,
session,
model_endpoint_id=getattr(endpoint_row, "id", None),
endpoint_provenance=(
"registered" if endpoint_row is not None else "direct"
),
)
except Exception as exc:
logger.error("Failed to persist session endpoint provenance for %s: %s", sid, exc)
raise HTTPException(500, "Failed to persist session endpoint provenance") from exc
# Set auth headers for custom API-key endpoints # Set auth headers for custom API-key endpoints
resolved_key = request_api_key resolved_key = request_api_key
resolved_base = endpoint_url resolved_base = endpoint_url
@@ -502,17 +443,14 @@ def setup_session_routes(
from src.endpoint_resolver import build_headers from src.endpoint_resolver import build_headers
session.headers = build_headers(resolved_key, resolved_base) session.headers = build_headers(resolved_key, resolved_base)
_persist_session_headers(sid, session.headers) _persist_session_headers(sid, session.headers)
# A bearer can create owner-attributed chat data, but must not cause # Fire webhook (sync-safe)
# owner lifecycle automation or webhook delivery as a side effect. if webhook_manager:
if not is_bearer_principal(request): webhook_manager.fire_and_forget("session.created", {
# Fire webhook (sync-safe) "session_id": sid, "name": session.name, "model": model_to_use,
if webhook_manager: })
webhook_manager.fire_and_forget("session.created", { # Fire event for automation tasks
"session_id": sid, "name": session.name, "model": model_to_use, from src.event_bus import fire_event
}) fire_event("session_created", user)
# Fire event for automation tasks
from src.event_bus import fire_event
fire_event("session_created", user)
return SessionResponse( return SessionResponse(
id=sid, id=sid,
name=session.name, name=session.name,
@@ -527,7 +465,6 @@ def setup_session_routes(
model: str = Form(None), endpoint_url: str = Form(None), model: str = Form(None), endpoint_url: str = Form(None),
endpoint_id: str = Form(None), endpoint_id: str = Form(None),
): ):
require_chat_scope(request)
_verify_session_owner(request, sid) _verify_session_owner(request, sid)
try: try:
session = session_manager.get_session(sid) session = session_manager.get_session(sid)
@@ -555,7 +492,6 @@ def setup_session_routes(
_reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url) _reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
endpoint_api_key = "" endpoint_api_key = ""
endpoint_base_url = "" endpoint_base_url = ""
endpoint_row = None
if endpoint_id: if endpoint_id:
from core.database import ModelEndpoint from core.database import ModelEndpoint
from src.auth_helpers import owner_filter from src.auth_helpers import owner_filter
@@ -571,23 +507,13 @@ def setup_session_routes(
ep = q.first() ep = q.first()
if not ep: if not ep:
raise HTTPException(400, "Model endpoint no longer exists") raise HTTPException(400, "Model endpoint no longer exists")
endpoint_row = ep
endpoint_base_url = ep.base_url or "" endpoint_base_url = ep.base_url or ""
endpoint_api_key = ep.api_key or "" endpoint_api_key = ep.api_key or ""
endpoint_url = build_chat_url(normalize_base(endpoint_base_url)) endpoint_url = build_chat_url(normalize_base(endpoint_base_url))
finally: finally:
_db.close() _db.close()
if is_bearer_principal(request) and endpoint_row is not None:
from routes.model_routes import _validate_bearer_model_selection
# Validate before mutating either the in-memory or durable
# session. The same server-owned inventory is enforced again
# immediately before each bearer LLM consumer.
model = _validate_bearer_model_selection(endpoint_row, model)
session.model = model session.model = model
session.endpoint_url = endpoint_url session.endpoint_url = endpoint_url
session.model_endpoint_id = getattr(endpoint_row, "id", None)
session.endpoint_provenance = "registered" if endpoint_row is not None else None
# Update auth headers from the endpoint's stored API key # Update auth headers from the endpoint's stored API key
if endpoint_api_key: if endpoint_api_key:
from src.endpoint_resolver import build_headers from src.endpoint_resolver import build_headers
@@ -602,8 +528,6 @@ def setup_session_routes(
db_session.model = model db_session.model = model
db_session.endpoint_url = endpoint_url db_session.endpoint_url = endpoint_url
db_session.headers = session.headers or {} db_session.headers = session.headers or {}
db_session.model_endpoint_id = getattr(endpoint_row, "id", None)
db_session.endpoint_provenance = "registered" if endpoint_row is not None else None
db_session.updated_at = utcnow_naive() db_session.updated_at = utcnow_naive()
db.commit() db.commit()
finally: finally:
@@ -615,7 +539,6 @@ def setup_session_routes(
@router.post("/session/{sid}/inject_messages") @router.post("/session/{sid}/inject_messages")
async def inject_messages(request: Request, sid: str): async def inject_messages(request: Request, sid: str):
"""Bulk-inject messages into a session's history (for group chat sync).""" """Bulk-inject messages into a session's history (for group chat sync)."""
require_chat_scope(request)
_verify_session_owner(request, sid) _verify_session_owner(request, sid)
try: try:
sess = session_manager.get_session(sid) sess = session_manager.get_session(sid)
@@ -631,7 +554,7 @@ def setup_session_routes(
upload_handler, upload_handler,
owner, owner,
message.get("content"), message.get("content"),
sanitize_client_message_metadata(message.get("metadata")), message.get("metadata"),
) )
if missing_id: if missing_id:
raise HTTPException( raise HTTPException(
@@ -641,24 +564,18 @@ def setup_session_routes(
except (AttributeError, TypeError, ValueError) as exc: except (AttributeError, TypeError, ValueError) as exc:
raise HTTPException(400, "Invalid message attachment metadata") from exc raise HTTPException(400, "Invalid message attachment metadata") from exc
for m in messages: for m in messages:
sess.add_message(ChatMessage( sess.add_message(ChatMessage(m["role"], m["content"], metadata=m.get("metadata")))
normalize_client_message_role(m.get("role", "user"), default="user"),
m["content"],
metadata=sanitize_client_message_metadata(m.get("metadata")),
))
session_manager.save_sessions() session_manager.save_sessions()
return {"ok": True, "count": len(messages)} return {"ok": True, "count": len(messages)}
@router.post("/session/{sid}/delete") @router.post("/session/{sid}/delete")
def delete_session_beacon(request: Request, sid: str): def delete_session_beacon(request: Request, sid: str):
"""Delete session via POST (for navigator.sendBeacon on page close).""" """Delete session via POST (for navigator.sendBeacon on page close)."""
require_chat_scope(request)
return delete_session(request, sid) return delete_session(request, sid)
@router.post("/sessions/bulk-delete") @router.post("/sessions/bulk-delete")
async def bulk_delete_sessions(request: Request): async def bulk_delete_sessions(request: Request):
"""Delete multiple sessions (for compare cleanup via sendBeacon).""" """Delete multiple sessions (for compare cleanup via sendBeacon)."""
require_chat_scope(request)
from core.database import ChatMessage as _CM from core.database import ChatMessage as _CM
try: try:
body = await request.json() body = await request.json()
@@ -688,7 +605,6 @@ def setup_session_routes(
@router.delete("/session/{sid}") @router.delete("/session/{sid}")
def delete_session(request: Request, sid: str): def delete_session(request: Request, sid: str):
"""Permanently delete a session and all its messages.""" """Permanently delete a session and all its messages."""
require_chat_scope(request)
_verify_session_owner(request, sid, session_manager) _verify_session_owner(request, sid, session_manager)
try: try:
# Block deletion of starred/favorited sessions # Block deletion of starred/favorited sessions
@@ -723,7 +639,6 @@ def setup_session_routes(
@router.delete("/sessions/all") @router.delete("/sessions/all")
def delete_all_sessions(request: Request): def delete_all_sessions(request: Request):
"""Admin only: permanently delete ALL sessions and their messages.""" """Admin only: permanently delete ALL sessions and their messages."""
require_chat_scope(request)
from core.middleware import require_admin from core.middleware import require_admin
require_admin(request) require_admin(request)
@@ -777,7 +692,6 @@ def setup_session_routes(
@router.post("/session/{sid}/archive") @router.post("/session/{sid}/archive")
def archive_session(request: Request, sid: str): def archive_session(request: Request, sid: str):
"""Archive a session, keeping its data but removing it from active sessions.""" """Archive a session, keeping its data but removing it from active sessions."""
require_chat_scope(request)
_verify_session_owner(request, sid) _verify_session_owner(request, sid)
try: try:
# First check if session exists # First check if session exists
@@ -816,7 +730,6 @@ def setup_session_routes(
@router.post("/session/{sid}/unarchive") @router.post("/session/{sid}/unarchive")
def unarchive_session(request: Request, sid: str): def unarchive_session(request: Request, sid: str):
"""Restore an archived session back to the active session list.""" """Restore an archived session back to the active session list."""
require_chat_scope(request)
_verify_session_owner(request, sid) _verify_session_owner(request, sid)
db = SessionLocal() db = SessionLocal()
try: try:
@@ -847,7 +760,6 @@ def setup_session_routes(
@router.get("/sessions/archived") @router.get("/sessions/archived")
def list_archived_sessions(request: Request, search: str = "", offset: int = 0, limit: int = 20, sort: str = "recent", model: str = ""): def list_archived_sessions(request: Request, search: str = "", offset: int = 0, limit: int = 20, sort: str = "recent", model: str = ""):
"""List archived sessions for the archive browser.""" """List archived sessions for the archive browser."""
require_chat_scope(request)
user = effective_user(request) user = effective_user(request)
db = SessionLocal() db = SessionLocal()
try: try:
@@ -895,7 +807,6 @@ def setup_session_routes(
Supported formats: md (markdown), txt (plain text), json, html Supported formats: md (markdown), txt (plain text), json, html
""" """
require_chat_scope(request)
_verify_session_owner(request, sid) _verify_session_owner(request, sid)
try: try:
session = session_manager.get_session(sid) session = session_manager.get_session(sid)
@@ -982,7 +893,6 @@ def setup_session_routes(
@router.post("/sessions/save") @router.post("/sessions/save")
def sessions_save_now(request: Request): def sessions_save_now(request: Request):
require_chat_scope(request)
user = effective_user(request) user = effective_user(request)
if not user: if not user:
raise HTTPException(401, "Not authenticated") raise HTTPException(401, "Not authenticated")
@@ -996,15 +906,6 @@ def setup_session_routes(
model: str = Form("gpt-4o"), model: str = Form("gpt-4o"),
rag: str = Form(None) rag: str = Form(None)
): ):
require_chat_scope(request)
# This legacy alias uses the server-owned OPENAI_API_KEY rather than a
# caller-selected, owner-visible ModelEndpoint. A bearer must not turn
# that credential into an owner-attributed session whose provenance
# cannot be represented as either a registered endpoint or a direct
# caller-supplied key. Registered bearer chat remains available through
# POST /api/session with endpoint_id.
if is_bearer_principal(request):
raise HTTPException(403, "Bearer callers must choose a registered model endpoint")
if not OPENAI_API_KEY: if not OPENAI_API_KEY:
raise HTTPException(400, "Server missing OPENAI_API_KEY") raise HTTPException(400, "Server missing OPENAI_API_KEY")
sid = str(uuid.uuid4()) sid = str(uuid.uuid4())
@@ -1019,15 +920,13 @@ def setup_session_routes(
) )
session.headers = {"Authorization": f"Bearer {OPENAI_API_KEY}"} session.headers = {"Authorization": f"Bearer {OPENAI_API_KEY}"}
session_manager.save_sessions() session_manager.save_sessions()
if not is_bearer_principal(request): from src.event_bus import fire_event
from src.event_bus import fire_event fire_event("session_created", user)
fire_event("session_created", user)
return {"id": sid, "name": "", "model": model} return {"id": sid, "name": "", "model": model}
@router.post("/session/{session_id}/important") @router.post("/session/{session_id}/important")
async def mark_session_important(request: Request, session_id: str, important: bool = Form(True)): async def mark_session_important(request: Request, session_id: str, important: bool = Form(True)):
"""Mark a session as important to protect it from automatic cleanup.""" """Mark a session as important to protect it from automatic cleanup."""
require_chat_scope(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
# Validate session exists # Validate session exists
@@ -1065,8 +964,6 @@ def setup_session_routes(
@router.post("/session/{session_id}/compact") @router.post("/session/{session_id}/compact")
async def compact_session(request: Request, session_id: str): async def compact_session(request: Request, session_id: str):
"""Summarize older messages into one compacted history entry.""" """Summarize older messages into one compacted history entry."""
require_chat_scope(request)
capability = request_capability(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
try: try:
session = session_manager.get_session(session_id) session = session_manager.get_session(session_id)
@@ -1086,22 +983,13 @@ def setup_session_routes(
if not older: if not older:
raise HTTPException(400, "Nothing old enough to compact") raise HTTPException(400, "Nothing old enough to compact")
if capability.is_bearer:
from routes.chat_helpers import _validate_bearer_session_model
_validate_bearer_session_model(session, owner=effective_user(request))
from src.context_compactor import SELF_SUMMARY_SYSTEM_PROMPT from src.context_compactor import SELF_SUMMARY_SYSTEM_PROMPT
from src.endpoint_resolver import resolve_endpoint
from src.llm_core import llm_call_async from src.llm_core import llm_call_async
owner = getattr(session, "owner", None) or effective_user(request) owner = getattr(session, "owner", None) or effective_user(request)
if capability.allow_live_probes: url, model, headers = resolve_endpoint("utility", owner=owner)
from src.endpoint_resolver import resolve_endpoint if not url or not model:
url, model, headers = resolve_endpoint("utility", owner=owner)
if not url or not model:
url, model, headers = session.endpoint_url, session.model, session.headers
else:
url, model, headers = session.endpoint_url, session.model, session.headers url, model, headers = session.endpoint_url, session.model, session.headers
if not url or not model: if not url or not model:
raise HTTPException(400, "No model configured for compaction") raise HTTPException(400, "No model configured for compaction")
@@ -1120,9 +1008,6 @@ def setup_session_routes(
for m in older for m in older
) )
try: try:
compact_kwargs = {}
if not capability.allow_live_probes:
compact_kwargs["allow_live_probes"] = False
summary = await llm_call_async( summary = await llm_call_async(
url, url,
model, model,
@@ -1131,7 +1016,6 @@ def setup_session_routes(
max_tokens=1024, max_tokens=1024,
headers=headers, headers=headers,
timeout=60, timeout=60,
**compact_kwargs,
) )
except Exception as e: except Exception as e:
logger.error("Manual compaction failed: %s", e) logger.error("Manual compaction failed: %s", e)
@@ -1157,10 +1041,7 @@ def setup_session_routes(
"message_count": len(new_history), "message_count": len(new_history),
} }
@router.post( @router.post("/sessions/auto-sort")
"/sessions/auto-sort",
dependencies=[Depends(require_interactive_request)],
)
def auto_sort_sessions(request: Request, skip_llm: bool = False): def auto_sort_sessions(request: Request, skip_llm: bool = False):
"""Use AI to categorize all sessions into folders. """Use AI to categorize all sessions into folders.
@@ -1169,8 +1050,6 @@ def setup_session_routes(
after Phase 1 used by the "Tidy (no AI)" UI affordance so after Phase 1 used by the "Tidy (no AI)" UI affordance so
users can clean junk without spending tokens. users can clean junk without spending tokens.
""" """
require_chat_scope(request)
require_interactive_request(request)
from src.llm_core import llm_call from src.llm_core import llm_call
user = effective_user(request) user = effective_user(request)
single_user_mode = not user and _auth_disabled() single_user_mode = not user and _auth_disabled()
@@ -1451,8 +1330,6 @@ def setup_session_routes(
@router.get("/session/{session_id}/context_info") @router.get("/session/{session_id}/context_info")
async def get_context_info(request: Request, session_id: str): async def get_context_info(request: Request, session_id: str):
"""Get the real context length for a session's model from the endpoint.""" """Get the real context length for a session's model from the endpoint."""
require_chat_scope(request)
capability = request_capability(request)
_verify_session_owner(request, session_id) _verify_session_owner(request, session_id)
session = session_manager.get_session(session_id) session = session_manager.get_session(session_id)
if not session: if not session:
@@ -1461,10 +1338,7 @@ def setup_session_routes(
return {"context_length": None} return {"context_length": None}
try: try:
from src.model_context import get_context_length from src.model_context import get_context_length
context_kwargs = {} ctx = get_context_length(session.endpoint_url, session.model)
if not capability.allow_live_probes:
context_kwargs["allow_live_probes"] = False
ctx = get_context_length(session.endpoint_url, session.model, **context_kwargs)
return {"context_length": ctx, "model": session.model} return {"context_length": ctx, "model": session.model}
except Exception: except Exception:
return {"context_length": None} return {"context_length": None}
-6
View File
@@ -16,7 +16,6 @@ from pathlib import Path
from typing import Dict, Any from typing import Dict, Any
from core.platform_compat import IS_APPLE_SILICON, which_tool from core.platform_compat import IS_APPLE_SILICON, which_tool
from core.middleware import INTERNAL_TOOL_USER from core.middleware import INTERNAL_TOOL_USER
from src.auth_helpers import is_bearer_principal
from src.host_docker_access import ( from src.host_docker_access import (
HOST_DOCKER_ACCESS_HINT, HOST_DOCKER_ACCESS_HINT,
host_docker_access_enabled as _host_docker_access_enabled, host_docker_access_enabled as _host_docker_access_enabled,
@@ -54,11 +53,6 @@ from core.platform_compat import (
def _require_admin(request: Request): def _require_admin(request: Request):
"""Reject non-admin callers. Shell exec is admin-only — never expose to """Reject non-admin callers. Shell exec is admin-only — never expose to
regular users; that's RCE-after-signup.""" regular users; that's RCE-after-signup."""
# This route predates the shared middleware helper and is also called
# directly by a few integration paths. Reject the credential class before
# trusting a caller-supplied current_user that might look administrative.
if is_bearer_principal(request):
raise HTTPException(403, "API tokens cannot use admin host-control surfaces")
auth_manager = getattr(request.app.state, "auth_manager", None) auth_manager = getattr(request.app.state, "auth_manager", None)
if not auth_manager: if not auth_manager:
# No auth at all — only safe in fully-trusted localhost dev mode # No auth at all — only safe in fully-trusted localhost dev mode
+21 -262
View File
@@ -13,12 +13,11 @@ from typing import List, Optional
import httpx import httpx
from fastapi import APIRouter, Depends, HTTPException, Request from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from services.memory.skills import SkillsManager from services.memory.skills import SkillsManager
from src.auth_helpers import require_interactive_request from src.auth_helpers import get_current_user
from src.prompt_security import untrusted_context_message
from core.middleware import require_admin from core.middleware import require_admin
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -108,23 +107,6 @@ def _skill_test_task(skill: dict) -> str:
) )
def _skill_test_messages(md: str, task: str) -> list[dict]:
"""Keep user-editable skill text out of the trusted system role."""
return [
{
"role": "system",
"content": (
"You are TESTING a skill. Follow the supplied reusable procedure "
"to complete the user's task for real, using available tools step "
"by step. If the skill is wrong, unclear, or references tools that "
"do not exist, do your best; the problems will be reviewed afterward."
),
},
untrusted_context_message("skill under test", md),
{"role": "user", "content": task},
]
async def _eval_skill_run(skill_md: str, task: str, transcript: str, async def _eval_skill_run(skill_md: str, task: str, transcript: str,
url: str, model: str, headers: Optional[dict]) -> dict: url: str, model: str, headers: Optional[dict]) -> dict:
"""LLM-as-judge: grade a skill test run from its transcript. Advisory only. """LLM-as-judge: grade a skill test run from its transcript. Advisory only.
@@ -429,21 +411,7 @@ async def _eval_skill_retrieval_precision(skill_md: str, others: list,
_skill_test_jobs: dict = {} _skill_test_jobs: dict = {}
async def _run_skill_test_job( async def _run_skill_test_job(key, name, md, task, url, model, headers, owner, skills_manager=None):
key,
name,
md,
task,
url,
model,
headers,
owner,
skills_manager=None,
*,
messages=None,
transcript=None,
exact_approval=None,
):
"""Background coroutine: run the skill in an agent loop, capture a condensed """Background coroutine: run the skill in an agent loop, capture a condensed
log + transcript, then have the judge grade it. Writes into _skill_test_jobs.""" log + transcript, then have the judge grade it. Writes into _skill_test_jobs."""
import json as _json import json as _json
@@ -453,7 +421,7 @@ async def _run_skill_test_job(
if job is None: if job is None:
return return
log = job["log"] log = job["log"]
transcript = transcript if isinstance(transcript, list) else [] transcript = []
say_buf = [] say_buf = []
def _flush_say(): def _flush_say():
@@ -461,12 +429,18 @@ async def _run_skill_test_job(
log.append({"type": "say", "text": "".join(say_buf)}) log.append({"type": "say", "text": "".join(say_buf)})
say_buf.clear() say_buf.clear()
messages = list(messages) if isinstance(messages, list) else _skill_test_messages(md, task) messages = [
{"role": "system", "content":
"You are TESTING a skill. Below is a reusable skill (a procedure). Follow it "
"to complete the user's task for real, using your available tools, step by "
"step. If the skill is wrong, unclear, or references tools that don't exist, "
"do your best — the problems will be reviewed afterward.\n\n=== SKILL ===\n" + md},
{"role": "user", "content": task},
]
try: try:
async for chunk in stream_agent_loop( async for chunk in stream_agent_loop(
url, model, messages, headers=headers, url, model, messages, headers=headers,
temperature=0.3, max_tokens=0, max_rounds=8, owner=owner, temperature=0.3, max_tokens=0, max_rounds=8, owner=owner,
exact_approval=exact_approval,
): ):
if not chunk.startswith("data: ") or chunk.strip() == "data: [DONE]": if not chunk.startswith("data: ") or chunk.strip() == "data: [DONE]":
continue continue
@@ -484,25 +458,8 @@ async def _run_skill_test_job(
elif d.get("type") == "tool_output": elif d.get("type") == "tool_output":
_flush_say() _flush_say()
out = str(d.get("output") or "")[:600] out = str(d.get("output") or "")[:600]
tool_log = {"type": "tool_output", "output": out} log.append({"type": "tool_output", "output": out})
approval = d.get("ask_user")
if isinstance(approval, dict):
tool_log["ask_user"] = approval
log.append(tool_log)
transcript.append(f"[output] {out}\n") transcript.append(f"[output] {out}\n")
if (
isinstance(approval, dict)
and approval.get("kind") == "tool_approval"
and approval.get("approval_id")
):
# Manual skill tests have their own polling UI instead of a
# chat session. Pause the run and retain only server-side
# continuation state until the same owner approves/denies
# this exact sealed action.
job["status"] = "awaiting_approval"
job["approval"] = approval
job["_transcript"] = transcript
return
elif d.get("type") == "agent_step": elif d.get("type") == "agent_step":
_flush_say() _flush_say()
log.append({"type": "agent_step", "round": d.get("round")}) log.append({"type": "agent_step", "round": d.get("round")})
@@ -514,9 +471,6 @@ async def _run_skill_test_job(
_flush_say() _flush_say()
log.append({"type": "error", "error": str(e)}) log.append({"type": "error", "error": str(e)})
job.pop("approval", None)
job.pop("_transcript", None)
job.pop("_run", None)
log.append({"type": "evaluating"}) log.append({"type": "evaluating"})
try: try:
job["verdict"] = await _eval_skill_run(md, task, "".join(transcript), url, model, headers) job["verdict"] = await _eval_skill_run(md, task, "".join(transcript), url, model, headers)
@@ -740,8 +694,12 @@ async def _run_skill_test_once(md: str, task: str, url, model, headers, owner) -
import json as _json import json as _json
from src.agent_loop import stream_agent_loop from src.agent_loop import stream_agent_loop
transcript = [] transcript = []
approval_required = None messages = [
messages = _skill_test_messages(md, task) {"role": "system", "content":
"You are TESTING a skill. Follow this skill's procedure to complete the task "
"for real, using your tools, step by step.\n\n=== SKILL ===\n" + md},
{"role": "user", "content": task},
]
try: try:
# max_tokens explicitly set: passing 0 lets some upstreams (Ollama, # max_tokens explicitly set: passing 0 lets some upstreams (Ollama,
# OpenAI-compat) generate an empty completion, which manifested as # OpenAI-compat) generate an empty completion, which manifested as
@@ -761,44 +719,11 @@ async def _run_skill_test_once(md: str, task: str, url, model, headers, owner) -
transcript.append(f"\n[tool {d.get('tool')}] {str(d.get('command') or d.get('args') or '')[:300]}\n") transcript.append(f"\n[tool {d.get('tool')}] {str(d.get('command') or d.get('args') or '')[:300]}\n")
elif d.get("type") == "tool_output": elif d.get("type") == "tool_output":
transcript.append(f"[output] {str(d.get('output') or '')[:600]}\n") transcript.append(f"[output] {str(d.get('output') or '')[:600]}\n")
approval = d.get("ask_user")
if (
isinstance(approval, dict)
and approval.get("kind") == "tool_approval"
):
approval_required = approval
break
elif d.get("type") == "agent_step": elif d.get("type") == "agent_step":
transcript.append(f"\n--- round {d.get('round')} ---\n") transcript.append(f"\n--- round {d.get('round')} ---\n")
except Exception as e: except Exception as e:
transcript.append(f"\n[run error] {e}\n") transcript.append(f"\n[run error] {e}\n")
text = "".join(transcript) text = "".join(transcript)
if approval_required is not None:
# Unattended audits have no authority to approve and no UI that could
# resume this record. Destructively deny it now instead of leaving a
# reusable opaque grant pending until TTL/cap eviction.
try:
from src.tool_approvals import tool_approval_store
tool_approval_store.consume(
approval_required.get("approval_id"),
decision="deny",
owner=owner,
session_id=None,
)
except Exception:
logger.debug("Could not retire unattended skill approval", exc_info=True)
return text, {
"verdict": "inconclusive",
"confidence": 1.0,
"summary": (
"This automated audit reached an exact action that requires "
"a human approval; no action was executed."
),
"issues": [
"Run this skill's manual test and review the sealed action."
],
"approval_required": True,
}
verdict = await _eval_skill_run(md, task, text, url, model, headers) verdict = await _eval_skill_run(md, task, text, url, model, headers)
return text, verdict return text, verdict
@@ -938,26 +863,6 @@ async def _audit_one_skill(skills_manager, skill, url, model, headers,
transcript, verdict = await _run_skill_test_once(md, task, url, model, headers, owner) transcript, verdict = await _run_skill_test_once(md, task, url, model, headers, owner)
v = verdict.get("verdict") v = verdict.get("verdict")
log(f"{name}: verdict = {v} ({verdict.get('summary', '')[:80]})") log(f"{name}: verdict = {v} ({verdict.get('summary', '')[:80]})")
if verdict.get("approval_required"):
# An unattended audit is not authority for an action influenced by the
# skill under test. Preserve the skill's current publication/confidence
# state and route the exact action to the manual test UI instead of
# letting a safety pause demote, rewrite, or auto-publish the skill.
skills_manager.set_audit(
name,
"inconclusive",
by_teacher=False,
worker_model=model,
owner=owner,
)
status = skill.get("status") or "draft"
log(f"{name}: {status} unchanged — exact action needs manual approval")
return {
"skill": name,
"result": "approval_required",
"verdict": verdict,
"status": status,
}
if v == "pass": if v == "pass":
# Procedure works. If the reviewer still flagged metadata (tags/category/ # Procedure works. If the reviewer still flagged metadata (tags/category/
# when_to_use/description), do ONE fixer pass to correct the frontmatter # when_to_use/description), do ONE fixer pass to correct the frontmatter
@@ -1181,14 +1086,10 @@ async def run_scheduled_skill_audit(skills_manager: SkillsManager,
def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter: def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
router = APIRouter( router = APIRouter(prefix="/api/skills", tags=["skills"])
prefix="/api/skills",
tags=["skills"],
dependencies=[Depends(require_interactive_request)],
)
def _owner(request: Request) -> Optional[str]: def _owner(request: Request) -> Optional[str]:
return require_interactive_request(request) return get_current_user(request)
def _verify_owner(skill: dict, user: Optional[str]): def _verify_owner(skill: dict, user: Optional[str]):
if user is None: if user is None:
@@ -1530,19 +1431,6 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
logger.warning(f"Skill-test model resolve failed: {_e}") logger.warning(f"Skill-test model resolve failed: {_e}")
key = (user or "", name) key = (user or "", name)
previous_job = _skill_test_jobs.get(key) or {}
previous_approval = previous_job.get("approval") or {}
if previous_approval.get("approval_id"):
try:
from src.tool_approvals import tool_approval_store
tool_approval_store.consume(
previous_approval["approval_id"],
decision="deny",
owner=user,
session_id=None,
)
except Exception:
logger.debug("Could not retire replaced skill approval", exc_info=True)
_skill_test_jobs[key] = { _skill_test_jobs[key] = {
"status": "running", "status": "running",
"task": task, "task": task,
@@ -1551,138 +1439,10 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
"started": _time.time(), "started": _time.time(),
"log": [{"type": "skill_test_start", "task": task, "skill": name, "model": model}], "log": [{"type": "skill_test_start", "task": task, "skill": name, "model": model}],
"verdict": None, "verdict": None,
"_run": {
"md": md,
"url": url,
"model": model,
"headers": headers,
"owner": user,
},
} }
_asyncio.create_task(_run_skill_test_job(key, name, md, task, url, model, headers, user, skills_manager)) _asyncio.create_task(_run_skill_test_job(key, name, md, task, url, model, headers, user, skills_manager))
return {"ok": True, "status": "running", "skill": name, "model": model} return {"ok": True, "status": "running", "skill": name, "model": model}
@router.post("/{skill_id}/test-approval")
async def approve_skill_test_action(request: Request, skill_id: str):
"""Resume a manual skill test with one exact server-sealed action."""
import asyncio as _asyncio
from src.tool_approvals import tool_approval_store
user = _owner(request)
skills = skills_manager.load(owner=user)
match = next(
(s for s in skills if s.get("name") == skill_id or s.get("id") == skill_id),
None,
)
if not match:
raise HTTPException(404, "Skill not found")
_verify_owner(match, user)
name = match.get("name")
key = (user or "", name)
job = _skill_test_jobs.get(key)
if not job or job.get("status") != "awaiting_approval":
raise HTTPException(409, "This skill test is not awaiting an approval.")
body = await request.json()
if not isinstance(body, dict):
raise HTTPException(400, "Tool approval body must be a JSON object.")
approval_id = str(body.get("approval_id") or "")
decision = str(body.get("decision") or "").strip().lower()
expected = job.get("approval") or {}
if approval_id != str(expected.get("approval_id") or ""):
raise HTTPException(409, "This approval does not match the pending skill test action.")
if decision not in {"approve", "deny"}:
raise HTTPException(400, "Invalid tool approval decision.")
pending = tool_approval_store.peek(approval_id)
normalized_owner = str(user or "").strip().casefold()
if (
pending is None
or pending.owner != normalized_owner
or pending.session_id != ""
):
raise HTTPException(409, "This tool approval is invalid or expired.")
exact_approval = tool_approval_store.consume(
approval_id,
decision=decision,
owner=user,
session_id=None,
# The button here says "Allow once" and there is no chat to carry a
# scope into, so the gate must re-arm behind the sealed action.
allow_continuation=False,
)
if decision == "approve" and exact_approval is None:
raise HTTPException(409, "This tool approval could not be consumed.")
job.pop("approval", None)
if decision == "deny":
job.pop("_transcript", None)
job.pop("_run", None)
job["log"].append({
"type": "approval_denied",
"text": "Exact action denied; the skill test stopped without executing it.",
})
job["verdict"] = {
"verdict": "inconclusive",
"confidence": 1.0,
"summary": "The test stopped because its exact action was denied.",
"issues": [],
}
job["status"] = "done"
return {"ok": True, "status": "done", "decision": "deny"}
run = job.get("_run") or {}
transcript = job.pop("_transcript", [])
# stream_agent_loop owns its per-round message list internally. Rebuild
# continuation context from the original untrusted skill plus the
# accumulated transcript so repeated approvals do not lose earlier
# approved results, while keeping every transcript byte tainted.
messages = _skill_test_messages(
run.get("md", ""),
job.get("task", ""),
)
if transcript:
messages.append(untrusted_context_message(
"skill test transcript",
"".join(str(item) for item in transcript),
))
messages.extend([
{
"role": "assistant",
"content": str(expected.get("question") or "Allow this exact action once?"),
},
{
"role": "user",
"content": (
f"Approved the exact {exact_approval.pending.tool_name} "
"action shown above once."
),
},
])
job["status"] = "running"
job["log"].append({
"type": "approval_granted",
"text": (
f"Approved exact {exact_approval.pending.tool_name} action once; "
"resuming test."
),
})
_asyncio.create_task(_run_skill_test_job(
key,
name,
run.get("md", ""),
job.get("task", ""),
run.get("url"),
run.get("model"),
run.get("headers"),
run.get("owner"),
skills_manager,
messages=messages,
transcript=transcript,
exact_approval=exact_approval,
))
return {"ok": True, "status": "running", "decision": "approve"}
@router.get("/{skill_id}/test-status") @router.get("/{skill_id}/test-status")
async def test_skill_status(request: Request, skill_id: str): async def test_skill_status(request: Request, skill_id: str):
"""Current background-test state for a skill (status / log / verdict).""" """Current background-test state for a skill (status / log / verdict)."""
@@ -1699,7 +1459,6 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
"model": job.get("model"), "model": job.get("model"),
"log": job.get("log", []), "log": job.get("log", []),
"verdict": job.get("verdict"), "verdict": job.get("verdict"),
"approval": job.get("approval"),
} }
@router.post("/audit-all") @router.post("/audit-all")
-5
View File
@@ -1,5 +0,0 @@
"""Task route domain package (slice 2p, #4082/#4071).
Contains task_routes.py, migrated from the flat routes/ directory.
Backward-compat shim at routes/task_routes.py re-exports from here.
"""
File diff suppressed because it is too large Load Diff
+1177 -14
View File
File diff suppressed because it is too large Load Diff
+4 -28
View File
@@ -6,7 +6,7 @@ import asyncio
import shutil import shutil
import uuid import uuid
from pathlib import Path from pathlib import Path
from fastapi import APIRouter, Depends, Request, File, UploadFile, HTTPException, Form from fastapi import APIRouter, Request, File, UploadFile, HTTPException, Form
from typing import List, Optional from typing import List, Optional
import logging import logging
from core.middleware import require_admin from core.middleware import require_admin
@@ -21,12 +21,7 @@ from core.database import (
Note, Note,
Session as DbSession, Session as DbSession,
) )
from src.auth_helpers import ( from src.auth_helpers import effective_user
effective_user,
is_bearer_principal,
require_chat_scope,
require_non_bearer_request,
)
from src.attachment_refs import attachment_refs_from_metadata from src.attachment_refs import attachment_refs_from_metadata
from src.constants import GENERATED_IMAGES_DIR from src.constants import GENERATED_IMAGES_DIR
from src.upload_handler import ( from src.upload_handler import (
@@ -37,11 +32,7 @@ from src.upload_handler import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter( router = APIRouter(prefix="/api/upload", tags=["upload"])
prefix="/api/upload",
tags=["upload"],
dependencies=[Depends(require_chat_scope)],
)
UPLOAD_RESPONSE_HEADERS = {"X-Content-Type-Options": "nosniff"} UPLOAD_RESPONSE_HEADERS = {"X-Content-Type-Options": "nosniff"}
def _upload_ids_from_persisted_text(value: object) -> set[str]: def _upload_ids_from_persisted_text(value: object) -> set[str]:
@@ -270,7 +261,6 @@ def setup_upload_routes(upload_handler):
session_id: Optional[str] = Form(None), session_id: Optional[str] = Form(None),
): ):
"""Upload files with enhanced security and organization.""" """Upload files with enhanced security and organization."""
require_chat_scope(request)
if not isinstance(session_id, str): if not isinstance(session_id, str):
session_id = None session_id = None
if not files: if not files:
@@ -330,7 +320,6 @@ def setup_upload_routes(upload_handler):
@router.post("/cleanup") @router.post("/cleanup")
async def manual_cleanup(request: Request): async def manual_cleanup(request: Request):
"""Manually trigger cleanup of old uploads.""" """Manually trigger cleanup of old uploads."""
require_chat_scope(request)
require_admin(request) require_admin(request)
try: try:
cleaned_count = await asyncio.to_thread( cleaned_count = await asyncio.to_thread(
@@ -354,7 +343,6 @@ def setup_upload_routes(upload_handler):
@router.get("/stats") @router.get("/stats")
async def upload_stats(request: Request): async def upload_stats(request: Request):
"""Get statistics about uploaded files.""" """Get statistics about uploaded files."""
require_chat_scope(request)
require_admin(request) require_admin(request)
try: try:
return upload_handler.get_upload_stats() return upload_handler.get_upload_stats()
@@ -367,7 +355,6 @@ def setup_upload_routes(upload_handler):
"""Serve an uploaded file by its ID. `?thumb=1` returns a small cached """Serve an uploaded file by its ID. `?thumb=1` returns a small cached
JPEG thumbnail for images (used by chat attachment previews) so the JPEG thumbnail for images (used by chat attachment previews) so the
client isn't downloading the full-resolution photo just to show it tiny.""" client isn't downloading the full-resolution photo just to show it tiny."""
require_chat_scope(request)
if not upload_handler.validate_upload_id(file_id): if not upload_handler.validate_upload_id(file_id):
raise HTTPException(400, "Invalid file ID") raise HTTPException(400, "Invalid file ID")
import mimetypes as _mt import mimetypes as _mt
@@ -384,14 +371,7 @@ def setup_upload_routes(upload_handler):
auth_configured = bool(auth_mgr and auth_mgr.is_configured) auth_configured = bool(auth_mgr and auth_mgr.is_configured)
current_user = effective_user(request) current_user = effective_user(request)
file_owner = info.get("owner") if info else None file_owner = info.get("owner") if info else None
if is_bearer_principal(request): if auth_configured:
# A token owner is an owner-bound data principal, even when that
# owner is an administrator. Do not reuse the browser admin
# fallback for bearer downloads or an admin token can read another
# user's upload by ID.
if not current_user or file_owner != current_user:
raise HTTPException(404, "File not found")
elif auth_configured:
if not current_user: if not current_user:
raise HTTPException(403, "Access denied") raise HTTPException(403, "Access denied")
if file_owner != current_user and not auth_mgr.is_admin(current_user): if file_owner != current_user and not auth_mgr.is_admin(current_user):
@@ -473,8 +453,6 @@ def setup_upload_routes(upload_handler):
"""Return the vision-model OCR/description for an uploaded image. """Return the vision-model OCR/description for an uploaded image.
Cached under UPLOAD_DIR/.vision/{file_id}.txt first call computes, Cached under UPLOAD_DIR/.vision/{file_id}.txt first call computes,
subsequent loads are instant. Pass force=1 to recompute.""" subsequent loads are instant. Pass force=1 to recompute."""
require_chat_scope(request)
require_non_bearer_request(request)
if not upload_handler.validate_upload_id(file_id): if not upload_handler.validate_upload_id(file_id):
raise HTTPException(400, "Invalid file ID") raise HTTPException(400, "Invalid file ID")
info = _load_upload_info(file_id) info = _load_upload_info(file_id)
@@ -519,8 +497,6 @@ def setup_upload_routes(upload_handler):
async def put_vision_text(request: Request, file_id: str): async def put_vision_text(request: Request, file_id: str):
"""Persist a user-edited vision/OCR text for an attachment. Stored in """Persist a user-edited vision/OCR text for an attachment. Stored in
the same cache file so the chat send picks it up as the override.""" the same cache file so the chat send picks it up as the override."""
require_chat_scope(request)
require_non_bearer_request(request)
if not upload_handler.validate_upload_id(file_id): if not upload_handler.validate_upload_id(file_id):
raise HTTPException(400, "Invalid file ID") raise HTTPException(400, "Invalid file ID")
info = _load_upload_info(file_id) info = _load_upload_info(file_id)
+41 -157
View File
@@ -2,7 +2,6 @@
import uuid import uuid
import logging import logging
import json
from typing import Optional from typing import Optional
import httpx import httpx
@@ -10,15 +9,9 @@ from fastapi import APIRouter, HTTPException, Request, Form
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from core.database import SessionLocal, Webhook, ModelEndpoint from core.database import SessionLocal, Webhook, ModelEndpoint
from src.auth_helpers import ( from src.auth_helpers import owner_filter
is_bearer_principal,
owner_filter,
request_capability,
require_chat_scope,
)
from src.url_security import validate_public_http_url from src.url_security import validate_public_http_url
from src.webhook_manager import WebhookManager, validate_webhook_url, validate_events from src.webhook_manager import WebhookManager, validate_webhook_url, validate_events
from src.session_provenance import persist_session_endpoint_provenance
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -39,15 +32,14 @@ def _select_api_chat_fallback_endpoint(db, token_owner: Optional[str]):
legacy null-owner ("shared") rows. Owner-scoped: an unscoped .first() would legacy null-owner ("shared") rows. Owner-scoped: an unscoped .first() would
let a chat-scoped token fall back onto another user's private endpoint and let a chat-scoped token fall back onto another user's private endpoint and
silently spend that owner's API key/quota. Prefer owner rows before shared silently spend that owner's API key/quota. Prefer owner rows before shared
rows. Fails closed when token_owner is absent; the sync endpoint requires rows. Fails closed to null-owner rows only when token_owner is absent.
an owner-scoped bearer before this helper is reached.
Does not validate base_url admin-configured local/LAN endpoints remain allowed. Does not validate base_url admin-configured local/LAN endpoints remain allowed.
""" """
query = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True) # noqa: E712 query = db.query(ModelEndpoint).filter(ModelEndpoint.is_enabled == True) # noqa: E712
if not token_owner: if token_owner:
return None query = owner_filter(query, ModelEndpoint, token_owner)
query = owner_filter(query, ModelEndpoint, token_owner) return query.order_by(ModelEndpoint.owner.desc(), ModelEndpoint.created_at).first()
return query.order_by(ModelEndpoint.owner.desc(), ModelEndpoint.created_at).first() return query.filter(ModelEndpoint.owner == None).order_by(ModelEndpoint.created_at).first() # noqa: E711
def _caller_owns_session(sess_owner, caller) -> bool: def _caller_owns_session(sess_owner, caller) -> bool:
@@ -69,89 +61,6 @@ def _caller_owns_session(sess_owner, caller) -> bool:
return sess_owner == caller return sess_owner == caller
def _cached_endpoint_model_ids(endpoint) -> list[str]:
"""Return model IDs already stored for a configured endpoint.
The synchronous bearer integration may use a cached model or the provider's
``auto`` alias, but it must not turn an ordinary chat request into a remote
catalog probe. Malformed/legacy cache shapes are treated as empty.
"""
try:
from routes.model_routes import _effective_endpoint_kind, _picker_models_for_endpoint
base_url = getattr(endpoint, "base_url", "") or ""
kind = _effective_endpoint_kind(endpoint, base_url)
models, _ = _picker_models_for_endpoint(endpoint, base_url, kind)
return models
except Exception:
raw = getattr(endpoint, "cached_models", None)
pinned_raw = getattr(endpoint, "pinned_models", None)
hidden_raw = getattr(endpoint, "hidden_models", None)
if not raw and not pinned_raw:
return []
try:
value = json.loads(raw) if isinstance(raw, str) else raw
pinned = json.loads(pinned_raw) if isinstance(pinned_raw, str) else pinned_raw
hidden = json.loads(hidden_raw) if isinstance(hidden_raw, str) else hidden_raw
except (TypeError, ValueError):
return []
if isinstance(value, dict):
value = value.get("data") or value.get("models") or []
if isinstance(pinned, dict):
pinned = pinned.get("data") or pinned.get("models") or []
if isinstance(hidden, dict):
hidden = hidden.get("data") or hidden.get("models") or []
if not isinstance(value, list):
value = []
if not isinstance(pinned, list):
pinned = []
if not isinstance(hidden, list):
hidden = []
raw_ids = value + pinned
hidden_ids = {str(item).strip() for item in hidden if str(item).strip()}
ids = []
for item in raw_ids:
if isinstance(item, str) and item.strip():
model_id = item.strip()
elif isinstance(item, dict):
model_id = item.get("id") or item.get("name") or item.get("model")
if not isinstance(model_id, str) or not model_id.strip():
continue
model_id = model_id.strip()
else:
continue
if model_id not in hidden_ids and model_id not in ids:
ids.append(model_id)
return ids
def _validate_bearer_sync_model(endpoint, requested_model: str) -> str:
"""Validate a configured sync model without probing its provider."""
try:
from routes.model_routes import _validate_bearer_model_selection
return _validate_bearer_model_selection(endpoint, requested_model)
except ImportError:
# Keep the lightweight webhook test/import seam usable when optional
# route modules are deliberately stubbed. Production uses the
# central picker validator above; this fallback remains cache-only.
models = _cached_endpoint_model_ids(endpoint)
requested = str(requested_model or "").strip()
if requested and requested in models:
return requested
if not requested and models:
return models[0]
if (
requested
and not models
and not getattr(endpoint, "cached_models", None)
and not getattr(endpoint, "pinned_models", None)
and "localhost" in str(getattr(endpoint, "base_url", "")).lower()
):
return requested
raise HTTPException(400, "Model is not permitted for this endpoint")
def setup_webhook_routes( def setup_webhook_routes(
webhook_manager: WebhookManager, webhook_manager: WebhookManager,
auth_manager, auth_manager,
@@ -327,16 +236,16 @@ def setup_webhook_routes(
@router.post("/v1/chat") @router.post("/v1/chat")
async def sync_chat(request: Request, body: SyncChatRequest): async def sync_chat(request: Request, body: SyncChatRequest):
if getattr(request.state, "api_token", False) is not True: if not getattr(request.state, "api_token", False):
raise HTTPException(403, "This endpoint requires an API token") raise HTTPException(403, "This endpoint requires an API token")
token_owner = require_chat_scope(request) scopes = set(getattr(request.state, "api_token_scopes", []) or [])
capability = request_capability(request) if "chat" not in scopes:
if not token_owner: raise HTTPException(403, "API token is not scoped for chat")
raise HTTPException(403, "API token has no owner") token_owner = getattr(request.state, "api_token_owner", None)
from core.models import ChatMessage from core.models import ChatMessage
from src.llm_core import llm_call_async from src.llm_core import llm_call_async
from src.endpoint_resolver import build_chat_url, build_headers, normalize_base from src.endpoint_resolver import build_chat_url, build_headers, build_models_url, normalize_base
message = body.message.strip() message = body.message.strip()
if not message: if not message:
@@ -366,12 +275,6 @@ def setup_webhook_routes(
_sess_owner = getattr(sess, "owner", None) _sess_owner = getattr(sess, "owner", None)
if not _caller_owns_session(_sess_owner, _tok_user): if not _caller_owns_session(_sess_owner, _tok_user):
raise HTTPException(404, "Session not found") raise HTTPException(404, "Session not found")
if is_bearer_principal(request):
from routes.chat_helpers import _validate_bearer_session_model
# Existing-session resume is an LLM boundary too; ownership
# alone must not authorize the persisted endpoint/model.
_validate_bearer_session_model(sess, owner=token_owner)
# --- Case 2: Direct API key + model (no pre-configured endpoint needed) --- # --- Case 2: Direct API key + model (no pre-configured endpoint needed) ---
if not sess and body.api_key: if not sess and body.api_key:
@@ -404,12 +307,6 @@ def setup_webhook_routes(
session_id=sid, name="API Chat", endpoint_url=endpoint_url, session_id=sid, name="API Chat", endpoint_url=endpoint_url,
model=model, owner=token_owner, model=model, owner=token_owner,
) )
persist_session_endpoint_provenance(
session_manager,
sid,
sess,
endpoint_provenance="direct",
)
sess.headers = build_headers(api_key, base_url) sess.headers = build_headers(api_key, base_url)
session_manager.save_sessions() session_manager.save_sessions()
session_id = sid session_id = sid
@@ -429,27 +326,39 @@ def setup_webhook_routes(
base_url = normalize_base(ep.base_url) base_url = normalize_base(ep.base_url)
endpoint_url = build_chat_url(base_url) endpoint_url = build_chat_url(base_url)
model = body.model or "" model = body.model or "auto"
api_key = ep.api_key api_key = ep.api_key
if getattr(ep, "provider_auth_id", None): if getattr(ep, "provider_auth_id", None):
try: try:
from src.endpoint_resolver import resolve_endpoint_runtime from src.endpoint_resolver import resolve_endpoint_runtime
runtime_kwargs = {} base_url, api_key = resolve_endpoint_runtime(ep, owner=token_owner)
if not capability.allow_live_probes:
runtime_kwargs["allow_live_probes"] = False
base_url, api_key = resolve_endpoint_runtime(
ep,
owner=token_owner,
**runtime_kwargs,
)
endpoint_url = build_chat_url(base_url) endpoint_url = build_chat_url(base_url)
except Exception: except Exception:
raise HTTPException(500, "Could not resolve endpoint credentials") raise HTTPException(500, "Could not resolve endpoint credentials")
# This route is bearer-only. Explicit and empty selections both if model == "auto":
# use the same server-owned, cache-only picker inventory; an empty try:
# inventory is an error rather than an implicit provider alias. async with httpx.AsyncClient(timeout=5) as client:
model = _validate_bearer_sync_model(ep, model) models_url = build_models_url(base_url)
hdrs = build_headers(api_key, base_url)
if models_url:
resp = await client.get(models_url, headers=hdrs)
resp.raise_for_status()
data = resp.json()
items = data if isinstance(data, list) else (data.get("data") or [])
ids = [m.get("id") for m in items if isinstance(m, dict) and m.get("id")]
if not ids and isinstance(data, dict):
ids = [
m.get("name") or m.get("model")
for m in (data.get("models") or [])
if m.get("name") or m.get("model")
]
else:
import json as _json
ids = _json.loads(ep.cached_models or "[]")
model = ids[0] if ids else "auto"
except Exception:
raise HTTPException(500, "Could not discover models from endpoint")
if not session_manager: if not session_manager:
raise HTTPException(500, "Session manager not available") raise HTTPException(500, "Session manager not available")
@@ -459,52 +368,27 @@ def setup_webhook_routes(
session_id=sid, name="API Chat", endpoint_url=endpoint_url, session_id=sid, name="API Chat", endpoint_url=endpoint_url,
model=model, owner=token_owner, model=model, owner=token_owner,
) )
endpoint_id = getattr(ep, "id", None)
if endpoint_id:
persist_session_endpoint_provenance(
session_manager,
sid,
sess,
model_endpoint_id=endpoint_id,
endpoint_provenance="registered",
)
if api_key: if api_key:
sess.headers = build_headers(api_key, base_url) sess.headers = build_headers(api_key, base_url)
session_manager.save_sessions() session_manager.save_sessions()
session_id = sid session_id = sid
# --- Send message and get response --- # --- Send message and get response ---
if is_bearer_principal(request):
from routes.chat_helpers import _validate_bearer_session_model
# The fallback branch has just created the session, so it did not
# pass through the existing-session gate above. Recheck the
# durable endpoint identity immediately before the LLM boundary
# for every bearer path, including malformed endpoint rows.
_validate_bearer_session_model(sess, owner=token_owner)
sess.add_message(ChatMessage("user", message)) sess.add_message(ChatMessage("user", message))
messages = [{"role": m.role, "content": m.content} for m in sess.history] messages = [{"role": m.role, "content": m.content} for m in sess.history]
llm_kwargs = {}
if not capability.allow_live_probes:
llm_kwargs["allow_live_probes"] = False
reply = await llm_call_async( reply = await llm_call_async(
sess.endpoint_url, sess.model, messages, sess.endpoint_url, sess.model, messages,
headers=sess.headers, timeout=120, headers=sess.headers, timeout=120,
**llm_kwargs,
) )
sess.add_message(ChatMessage("assistant", reply)) sess.add_message(ChatMessage("assistant", reply))
session_manager.save_sessions() session_manager.save_sessions()
# /api/v1/chat remains a synchronous bearer integration: the response webhook_manager.fire_and_forget("chat.completed", {
# is returned normally, but the token must not fan that content out to "session_id": session_id, "model": sess.model,
# an owner-configured asynchronous callback after authorization ends. "user_message": message[:2000], "response": reply[:2000],
if not is_bearer_principal(request): })
webhook_manager.fire_and_forget("chat.completed", {
"session_id": session_id, "model": sess.model,
"user_message": message[:2000], "response": reply[:2000],
})
return {"response": reply, "session_id": session_id, "model": sess.model} return {"response": reply, "session_id": session_id, "model": sess.model}
+1 -3
View File
@@ -2,7 +2,7 @@
import os import os
from fastapi import APIRouter, Request, HTTPException, Query from fastapi import APIRouter, Request, HTTPException, Query
from src.auth_helpers import get_current_user, require_non_bearer_request from src.auth_helpers import get_current_user
from src.tool_security import owner_is_admin_or_single_user from src.tool_security import owner_is_admin_or_single_user
# Cap entries returned per directory (mirrors filesystem_tools._CODENAV_MAX_HITS). # Cap entries returned per directory (mirrors filesystem_tools._CODENAV_MAX_HITS).
@@ -24,7 +24,6 @@ def setup_workspace_routes():
NON_ADMIN_BLOCKED_TOOLS). A non-admin who can't use those tools must not NON_ADMIN_BLOCKED_TOOLS). A non-admin who can't use those tools must not
be able to map the host's directory tree either. be able to map the host's directory tree either.
""" """
require_non_bearer_request(request)
owner = get_current_user(request) owner = get_current_user(request)
if not owner_is_admin_or_single_user(owner): if not owner_is_admin_or_single_user(owner):
raise HTTPException(status_code=403, detail="Workspace browsing is admin-only") raise HTTPException(status_code=403, detail="Workspace browsing is admin-only")
@@ -76,7 +75,6 @@ def setup_workspace_routes():
instead of being stored client-side and silently dropped at chat time. instead of being stored client-side and silently dropped at chat time.
Admin-gated like /browse: it confirms path existence on the host. Admin-gated like /browse: it confirms path existence on the host.
""" """
require_non_bearer_request(request)
owner = get_current_user(request) owner = get_current_user(request)
if not owner_is_admin_or_single_user(owner): if not owner_is_admin_or_single_user(owner):
raise HTTPException(status_code=403, detail="Workspace selection is admin-only") raise HTTPException(status_code=403, detail="Workspace selection is admin-only")
+2 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Encode a source screen-recording (.mkv) into web-optimized preview clips for # Encode a source screen-recording (.mkv) into web-optimized preview clips for
# the landing page: website/<name>.webm (VP9) + website/<name>.mp4 (H.264). # the landing page: docs/<name>.webm (VP9) + docs/<name>.mp4 (H.264).
# #
# ./encode_previews.sh <input> <name> [max_secs] # ./encode_previews.sh <input> <name> [max_secs]
# #
@@ -13,7 +13,7 @@ set -euo pipefail
IN="${1:?input file}" IN="${1:?input file}"
NAME="${2:?output basename}" NAME="${2:?output basename}"
MAX="${3:-30}" MAX="${3:-30}"
OUT_DIR="$(cd "$(dirname "$0")/../website" && pwd)" OUT_DIR="$(cd "$(dirname "$0")/../docs" && pwd)"
dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$IN" | cut -d. -f1) dur=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$IN" | cut -d. -f1)
dur=${dur:-0} dur=${dur:-0}
-166
View File
@@ -1,166 +0,0 @@
#!/usr/bin/env python3
"""Make retained SearXNG settings inherit defaults without replacing them."""
from __future__ import annotations
import os
import stat
import sys
import tempfile
from pathlib import Path
import yaml
from yaml.nodes import MappingNode
from yaml.tokens import BlockMappingStartToken, FlowMappingStartToken
_UTF8_BOM = b"\xef\xbb\xbf"
def _parse_root_mapping(text: str) -> tuple[MappingNode | None, dict]:
"""Parse settings with the same safe YAML semantics SearXNG uses."""
try:
loaded = yaml.safe_load(text)
node = yaml.compose(text, Loader=yaml.SafeLoader)
except yaml.YAMLError:
raise ValueError("settings file is not valid single-document YAML") from None
if loaded is None and node is None:
return None, {}
if not isinstance(loaded, dict) or not isinstance(node, MappingNode):
raise ValueError("settings root is not a mapping")
return node, loaded
def _flow_mapping_start(text: str) -> int:
"""Return the root flow mapping's opening-brace character offset."""
try:
for token in yaml.scan(text, Loader=yaml.SafeLoader):
if isinstance(token, FlowMappingStartToken):
return token.start_mark.index
except yaml.YAMLError:
pass
raise ValueError("flow-style settings mapping has no opening brace")
def _newline_for(contents: bytes) -> bytes:
first_lf = contents.find(b"\n")
if first_lf > 0 and contents[first_lf - 1 : first_lf + 1] == b"\r\n":
return b"\r\n"
return b"\n"
def _block_mapping_position(text: str, root: MappingNode | None) -> tuple[int, int]:
"""Return a safe character offset and indent for a root block mapping key."""
if root is None:
return len(text), 0
try:
for token in yaml.scan(text, Loader=yaml.SafeLoader):
if not isinstance(token, BlockMappingStartToken):
continue
line_start = token.start_mark.index - token.start_mark.column
if not text[line_start : token.start_mark.index].strip():
return line_start, token.start_mark.column
return root.end_mark.index, token.start_mark.column
except yaml.YAMLError:
pass
return root.end_mark.index, root.start_mark.column
def _add_block_default_inheritance(
contents: bytes, text: str, root: MappingNode | None
) -> bytes:
newline = _newline_for(contents)
character_offset, indent_width = _block_mapping_position(text, root)
bom_length = len(_UTF8_BOM) if contents.startswith(_UTF8_BOM) else 0
offset = bom_length + len(text[:character_offset].encode("utf-8"))
separator = b""
if offset not in (0, bom_length) and not contents[:offset].endswith((b"\n", b"\r")):
separator = newline
addition = (
separator
+ b" " * indent_width
+ b"use_default_settings: true"
+ newline
)
return contents[:offset] + addition + contents[offset:]
def migrate_settings(path: Path) -> bool:
"""Add the missing inheritance key atomically; return whether the file changed."""
source_stat = path.lstat()
if not stat.S_ISREG(source_stat.st_mode):
raise ValueError(f"settings path is not a regular file: {path}")
contents = path.read_bytes()
if not contents:
return False
text = contents.decode("utf-8-sig")
root, loaded = _parse_root_mapping(text)
if "use_default_settings" in loaded:
return False
if root is not None and root.flow_style:
start = _flow_mapping_start(text)
bom_length = len(_UTF8_BOM) if contents.startswith(_UTF8_BOM) else 0
offset = bom_length + len(text[: start + 1].encode("utf-8"))
separator = b", " if root.value else b""
updated = (
contents[:offset]
+ b"use_default_settings: true"
+ separator
+ contents[offset:]
)
else:
updated = _add_block_default_inheritance(contents, text, root)
fd, temporary_name = tempfile.mkstemp(
prefix=f".{path.name}.odysseus-", dir=path.parent
)
temporary = Path(temporary_name)
try:
# chmod before chown: the Compose cap set is `cap_drop: ALL` plus
# CHOWN/SETGID/SETUID/DAC_OVERRIDE, with no FOWNER. Once the temporary
# file belongs to searxng:searxng — which every retained settings file
# does, because searxng's entrypoint chowns /etc/searxng — root can no
# longer chmod it and the migration dies with EPERM.
os.fchmod(fd, stat.S_IMODE(source_stat.st_mode))
os.fchown(fd, source_stat.st_uid, source_stat.st_gid)
with os.fdopen(fd, "wb") as handle:
fd = -1
handle.write(updated)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
directory_fd = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
if fd >= 0:
os.close(fd)
temporary.unlink(missing_ok=True)
return True
def main(argv: list[str]) -> int:
if len(argv) > 2:
print(f"usage: {Path(argv[0]).name} [settings.yml]", file=sys.stderr)
return 2
path = Path(argv[1]) if len(argv) == 2 else Path("/etc/searxng/settings.yml")
try:
changed = migrate_settings(path)
except (OSError, UnicodeError, ValueError) as exc:
print(f"SearXNG settings migration failed: {exc}", file=sys.stderr)
return 1
if changed:
print("Added use_default_settings inheritance to retained SearXNG settings")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+2 -7
View File
@@ -327,12 +327,7 @@ def list_models():
@app.post("/v1/images/generations") @app.post("/v1/images/generations")
def generate(req: ImageRequest): def generate(req: ImageRequest):
# The served model is the one this process was launched with. `req.model` model = req.model or _args.model
# is accepted for OpenAI wire compatibility and ignored, matching
# scripts/diffusion_server.py: honouring it would let a caller point the
# generator at any local directory or Hugging Face repo, and the HiDream
# branch runs a python script from inside that directory.
model = _args.model
width, height = _size(req.size) width, height = _size(req.size)
out_images = [] out_images = []
count = max(1, min(int(req.n or 1), 4)) count = max(1, min(int(req.n or 1), 4))
@@ -398,7 +393,7 @@ async def edit_image(
size: str = Form("1024x1024"), size: str = Form("1024x1024"),
response_format: str = Form("b64_json"), response_format: str = Form("b64_json"),
): ):
active_model = _args.model # pinned; see generate() active_model = model or _args.model
if _is_lama_inpaint(active_model) or _is_ddcolor(active_model): if _is_lama_inpaint(active_model) or _is_ddcolor(active_model):
image_raw = await image.read() image_raw = await image.read()
mask_raw = await mask.read() if mask is not None else None mask_raw = await mask.read() if mask is not None else None
+3 -11
View File
@@ -2,7 +2,7 @@
"""odysseus-webhook — shell wrapper for scheduled-task webhook tokens. """odysseus-webhook — shell wrapper for scheduled-task webhook tokens.
Tasks in the scheduled-task system can carry a `webhook_token`. Any Tasks in the scheduled-task system can carry a `webhook_token`. Any
HTTP POST to `/api/tasks/<task-id>/webhook/<token>` fires the task. This CLI lists, HTTP POST to `/api/webhook/<token>` fires the task. This CLI lists,
rotates, and revokes those tokens. rotates, and revokes those tokens.
odysseus-webhook list # tasks that have a token odysseus-webhook list # tasks that have a token
@@ -21,7 +21,6 @@ quiet_logs()
import argparse, json, logging, os, secrets, sys import argparse, json, logging, os, secrets, sys
from pathlib import Path from pathlib import Path
from urllib.parse import quote
try: try:
from core.database import SessionLocal, ScheduledTask from core.database import SessionLocal, ScheduledTask
@@ -54,14 +53,6 @@ def _summary(t: "ScheduledTask", reveal: bool = False) -> dict:
} }
def _task_webhook_url(base: str, task_id: str, token: str) -> str:
"""Build the live task-route URL without leaking ids into path syntax."""
root = (base or "http://localhost:7000").rstrip("/")
task_part = quote(str(task_id), safe="")
token_part = quote(str(token), safe="")
return f"{root}/api/tasks/{task_part}/webhook/{token_part}"
def cmd_list(args): def cmd_list(args):
db = SessionLocal() db = SessionLocal()
try: try:
@@ -118,7 +109,8 @@ def cmd_url(args):
fail(f"no task with id {args.id!r}") fail(f"no task with id {args.id!r}")
if not t.webhook_token: if not t.webhook_token:
fail(f"task {args.id!r} has no webhook token (rotate one first)") fail(f"task {args.id!r} has no webhook token (rotate one first)")
url = _task_webhook_url(args.base, t.id, t.webhook_token) base = (args.base or "http://localhost:7000").rstrip("/")
url = f"{base}/api/webhook/{t.webhook_token}"
emit({ emit({
"task_id": t.id, "task_id": t.id,
"name": t.name, "name": t.name,
+11 -41
View File
@@ -50,46 +50,16 @@ class DocsService:
List of DocChunk objects List of DocChunk objects
""" """
results = self.rag.search(query, k=top_k) results = self.rag.search(query, k=top_k)
chunks = [] return [
DocChunk(
for result in results: text=r.get("text", r.get("content", "")),
if not isinstance(result, dict): source=r.get("source", r.get("metadata", {}).get("source", "unknown")),
continue score=r.get("score", 0.0),
metadata=r.get("metadata"),
metadata = result.get("metadata")
if not isinstance(metadata, dict):
metadata = {}
text = result.get("document")
if text is None:
text = result.get("text")
if text is None:
text = result.get("content")
if text is None:
text = ""
source = result.get("source")
if source is None:
source = metadata.get("source")
if source is None:
source = "unknown"
score = result.get("similarity")
if score is None:
score = result.get("score")
if score is None:
score = 0.0
chunks.append(
DocChunk(
text=text,
source=source,
score=score,
metadata=metadata,
)
) )
for r in results
return chunks if isinstance(r, dict)
]
async def index(self, directory: str) -> IndexResult: async def index(self, directory: str) -> IndexResult:
""" """
@@ -103,8 +73,8 @@ class DocsService:
""" """
result = self.rag.index_personal_documents(directory) result = self.rag.index_personal_documents(directory)
return IndexResult( return IndexResult(
indexed=result.get("indexed_count", result.get("indexed", 0)), indexed=result.get("indexed", 0),
failed=result.get("failed_count", result.get("failed", 0)), failed=result.get("failed", 0),
errors=result.get("errors", []), errors=result.get("errors", []),
) )
+43 -213
View File
@@ -1,18 +1,16 @@
"""Import SKILL.md bundles from public GitHub (or skills.sh → GitHub) URLs.""" """Import SKILL.md bundles from public GitHub (or skills.sh → GitHub) URLs."""
from __future__ import annotations from __future__ import annotations
import ipaddress
import logging import logging
import os import os
import time import re
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, Iterable, List, Optional, Tuple, cast from typing import Dict, List, Optional, Tuple
from urllib.parse import quote, urljoin, urlparse from urllib.parse import quote, urljoin, urlparse
import httpcore
import httpx import httpx
from src.url_safety import _default_resolver, check_outbound_url from src.url_safety import check_outbound_url
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -27,7 +25,6 @@ TEXT_NAMES = {"skill.md", "license", "license.md", "readme.md"}
_GITHUB_HOSTS = frozenset({ _GITHUB_HOSTS = frozenset({
"github.com", "www.github.com", "api.github.com", "raw.githubusercontent.com", "github.com", "www.github.com", "api.github.com", "raw.githubusercontent.com",
}) })
_SKILLS_SH_HOSTS = frozenset({"skills.sh", "www.skills.sh"})
def _github_host(url: str) -> str: def _github_host(url: str) -> str:
@@ -75,158 +72,18 @@ def _is_text_file(name: str) -> bool:
_MAX_FETCH_REDIRECTS = 5 _MAX_FETCH_REDIRECTS = 5
def _validated_ips(raw_ips: List[str]) -> List[ipaddress._BaseAddress]: def _check_fetch_url(url: str) -> None:
"""Parse and de-duplicate one resolver snapshot in resolver order.""" """SSRF guard for skill-import fetches (defense-in-depth).
ips: List[ipaddress._BaseAddress] = []
seen = set()
for raw in raw_ips:
if not isinstance(raw, str):
continue
try:
ip = ipaddress.ip_address(raw.split("%", 1)[0])
except ValueError:
continue
if ip in seen:
continue
seen.add(ip)
ips.append(ip)
return ips
Skill bundles only ever come from public GitHub, never an internal
def _resolve_and_check_url(url: str) -> List[ipaddress._BaseAddress]: address, so block private/loopback/link-local targets on every hop
"""Return the exact address snapshot approved for one fetch hop.""" matching the hardened web-fetch path in
resolved_ips: List[str] = [] ``services/search/content.py:_get_public_url`` rather than the lenient
default used for admin-configured model endpoints.
def _recording_resolver(host: str) -> List[str]: """
answers = list(_default_resolver(host)) ok, reason = check_outbound_url(url, block_private=True)
resolved_ips[:] = answers
return answers
ok, reason = check_outbound_url(
url,
block_private=True,
resolver=_recording_resolver,
)
if not ok: if not ok:
raise SkillImportError(f"outbound URL blocked: {reason}") raise SkillImportError(reason)
pinned_ips = _validated_ips(resolved_ips)
if not pinned_ips:
raise SkillImportError("outbound URL blocked: host did not resolve to a usable address")
return pinned_ips
# Backward compatibility alias for tests importing _check_fetch_url directly
_check_fetch_url = _resolve_and_check_url
class _PinnedBackend(httpcore.NetworkBackend):
"""Connect only to addresses from one validated DNS snapshot."""
def __init__(self, ips: List[ipaddress._BaseAddress]):
self._ips = [str(ip) for ip in ips]
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
deadline = None if timeout is None else time.monotonic() + timeout
last_exc: Optional[Exception] = None
for ip in self._ips:
remaining = None if deadline is None else max(0.0, deadline - time.monotonic())
try:
return self._real.connect_tcp(
ip,
port,
remaining,
local_address,
socket_options,
)
except (httpcore.ConnectError, httpcore.ConnectTimeout) as exc:
last_exc = exc
if deadline is not None and time.monotonic() >= deadline:
break
if last_exc is not None:
raise last_exc
raise httpcore.ConnectError("no validated address available")
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Pin socket connects while preserving URL authority, Host, and TLS SNI."""
def __init__(self, ips: List[ipaddress._BaseAddress]):
self._pinned_ips = list(ips)
self._pool = httpcore.ConnectionPool(
ssl_context=httpx.create_ssl_context(),
http1=True,
http2=False,
network_backend=_PinnedBackend(ips),
)
def handle_request(self, request: httpx.Request) -> httpx.Response:
core_request = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
core_response = None
try:
core_response = self._pool.handle_request(core_request)
content = b"".join(cast(Iterable[bytes], core_response.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
finally:
if core_response is not None:
core_response.close()
return httpx.Response(
status_code=core_response.status,
headers=core_response.headers,
content=content,
extensions=core_response.extensions,
)
def close(self) -> None:
self._pool.close()
def _get_checked( def _get_checked(
@@ -243,76 +100,49 @@ def _get_checked(
hand lets us re-validate every hop, closing that blind-SSRF gap. hand lets us re-validate every hop, closing that blind-SSRF gap.
""" """
current = url current = url
for _ in range(_MAX_FETCH_REDIRECTS + 1): with httpx.Client(follow_redirects=False, timeout=timeout) as client:
pinned_ips = _resolve_and_check_url(current) for _ in range(_MAX_FETCH_REDIRECTS + 1):
with httpx.Client( _check_fetch_url(current)
transport=_PinnedTransport(pinned_ips),
follow_redirects=False,
timeout=timeout,
) as client:
r = client.get(current, headers=headers) r = client.get(current, headers=headers)
if r.status_code in (301, 302, 303, 307, 308):
if r.status_code in (301, 302, 303, 307, 308): location = r.headers.get("location")
location = r.headers.get("location") if not location:
if not location: return r
return r current = urljoin(str(r.url), location)
current = urljoin(str(r.url), location) continue
continue return r
return r
raise SkillImportError("too many redirects while fetching skill bundle") raise SkillImportError("too many redirects while fetching skill bundle")
def parse_skill_source(url: str) -> ResolvedSource: def parse_skill_source(url: str) -> ResolvedSource:
"""Normalize skills.sh / GitHub web URLs into owner/repo/ref/path.""" """Normalize skills.sh / GitHub web URLs into owner/repo/ref/path."""
url = (url or "").strip() raw = (url or "").strip()
if not url: if not raw:
raise SkillImportError("URL is required") raise SkillImportError("URL is required")
# ``urlparse`` only reports an unambiguous scheme when the URL carries the # skills.sh often links to GitHub; try to unwrap ?url= or redirect target later.
# ``scheme://`` form. Opaque schemes (``mailto:``, ``javascript:``) and a if "skills.sh" in raw and "github.com" not in raw:
# schemeless ``host:port`` both parse a "scheme" that is not one, so they r = _get_checked(raw, timeout=20.0)
# fall through to the host check below and are rejected on the host instead.
scheme = urlparse(url).scheme.lower()
if scheme not in ("http", "https"):
if scheme and url.lower().startswith(f"{scheme}://"):
raise SkillImportError(f"unsupported URL scheme: {scheme}")
# Schemeless "github.com/owner/repo" — accept only a supported host.
rough_host = (urlparse("//" + url).hostname or "").lower()
if rough_host not in _GITHUB_HOSTS and rough_host not in _SKILLS_SH_HOSTS:
raise SkillImportError("Only GitHub or skills.sh URLs are supported")
url = "https://" + url
parsed = urlparse(url)
hostname = (parsed.hostname or "").lower()
if hostname not in _GITHUB_HOSTS and hostname not in _SKILLS_SH_HOSTS:
raise SkillImportError("Only GitHub or skills.sh URLs are supported")
# A skills.sh link is only usable if it redirects to an exact supported
# GitHub host. Scraping the page body for a github.com link cannot work:
# skill pages only ever link the repository root, never the skill's
# subdirectory, so the scrape resolves every skill in a repo to the same
# (wrong) bundle. Fail with an actionable message instead.
if hostname in _SKILLS_SH_HOSTS:
r = _get_checked(url, timeout=20.0)
if r.status_code >= 400: if r.status_code >= 400:
raise _github_response_error(r) raise _github_response_error(r)
final = str(r.url) final = str(r.url)
if _github_host(final) not in _GITHUB_HOSTS: _assert_github_url(final, context="redirect target")
raise SkillImportError( # Page may embed a github link; prefer final URL if redirected.
"skills.sh did not redirect to GitHub — open the skill's " if "github.com" in final:
"repository on GitHub, navigate to the exact skill folder or " raw = final
"SKILL.md file, and paste that URL; the repository-root link " else:
"alone is not sufficient" m = re.search(r"https?://github\.com/[^\s\"')]+", r.text or "")
) if m:
url = final raw = m.group(0).rstrip(".,)")
# Update parsed and hostname to reflect the new GitHub URL parsed = urlparse(raw)
parsed = urlparse(url) host = _github_host(raw)
hostname = (parsed.hostname or "").lower() if host not in _GITHUB_HOSTS:
raise SkillImportError(
"Only GitHub URLs are supported (https://github.com/... or raw.githubusercontent.com/...)"
)
_assert_github_url(url) if host == "raw.githubusercontent.com":
if hostname == "raw.githubusercontent.com":
# /owner/repo/ref/path/to/file # /owner/repo/ref/path/to/file
bits = [p for p in parsed.path.split("/") if p] bits = [p for p in parsed.path.split("/") if p]
if len(bits) < 4: if len(bits) < 4:
+331 -31
View File
@@ -2,18 +2,22 @@
import copy import copy
import io import io
import ipaddress
import json import json
import os import os
import re import re
import logging import logging
import socket
import ssl
from datetime import datetime, timedelta from datetime import datetime, timedelta
from typing import List from typing import Iterable, List, cast
from urllib.parse import urljoin, urlparse
import httpx import httpx
import httpcore
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_USER_AGENT from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES, WEB_FETCH_USER_AGENT
from src import outbound_fetch as _outbound_fetch
from .analytics import RateLimitError, error_logger from .analytics import RateLimitError, error_logger
from .cache import ( from .cache import (
@@ -25,40 +29,336 @@ from .cache import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _is_private_address(addr): _PRIVATE_NETWORKS = (
return _outbound_fetch._is_private_address(addr) ipaddress.ip_network("0.0.0.0/8"),
ipaddress.ip_network("10.0.0.0/8"),
ipaddress.ip_network("127.0.0.0/8"),
ipaddress.ip_network("169.254.0.0/16"),
ipaddress.ip_network("172.16.0.0/12"),
ipaddress.ip_network("192.168.0.0/16"),
ipaddress.ip_network("::1/128"),
ipaddress.ip_network("fc00::/7"),
ipaddress.ip_network("fe80::/10"),
)
def _resolve_hostname_ips(hostname): def _is_private_address(addr: ipaddress._BaseAddress) -> bool:
return _outbound_fetch._resolve_hostname_ips(hostname) if isinstance(addr, ipaddress.IPv6Address) and addr.ipv4_mapped is not None:
addr = addr.ipv4_mapped
return (
def _public_http_url(url): addr.is_private
return _outbound_fetch._public_http_url(url, resolver=_resolve_hostname_ips) or addr.is_loopback
or addr.is_link_local
or addr.is_reserved
def _resolve_public_ips(url): or addr.is_multicast
return _outbound_fetch._resolve_public_ips(url, resolver=_resolve_hostname_ips) or addr.is_unspecified
or any(addr in net for net in _PRIVATE_NETWORKS)
_PinnedBackend = _outbound_fetch._PinnedBackend
_PinnedTransport = _outbound_fetch._PinnedTransport
BodyTooLargeError = _outbound_fetch.BodyTooLargeError
_CappedFetch = _outbound_fetch._CappedFetch
def _get_public_url(url, headers, timeout, max_redirects=5, max_bytes=None):
return _outbound_fetch._get_public_url(
url,
headers=headers,
timeout=timeout,
max_redirects=max_redirects,
max_bytes=max_bytes,
resolve_public_ips=_resolve_public_ips,
transport_factory=_PinnedTransport,
) )
def _resolve_hostname_ips(hostname: str) -> list[ipaddress._BaseAddress]:
try:
infos = socket.getaddrinfo(hostname, None)
except Exception:
return []
out = []
for info in infos:
try:
out.append(ipaddress.ip_address(info[4][0]))
except Exception:
continue
return out
def _public_http_url(url: str) -> bool:
try:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
return False
host = (parsed.hostname or "").strip()
if not host:
return False
lower = host.lower()
if lower in ("localhost", "metadata", "metadata.google.internal"):
return False
if lower.endswith((".local", ".localhost", ".internal", ".lan", ".intranet")):
return False
try:
return not _is_private_address(ipaddress.ip_address(host))
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
return bool(addrs) and not any(_is_private_address(a) for a in addrs)
except Exception:
return False
def _resolve_public_ips(url: str) -> list[ipaddress._BaseAddress]:
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname:
raise httpx.RequestError(f"Blocked non-public URL: {url}")
host = (parsed.hostname or "").strip().lower()
if host in ("localhost", "metadata", "metadata.google.internal"):
raise httpx.RequestError(f"Blocked non-public hostname: {host}")
try:
ip = ipaddress.ip_address(host)
if _is_private_address(ip):
raise httpx.RequestError(f"Blocked non-public IP literal: {host}")
return [ip]
except httpx.RequestError:
raise
except ValueError:
pass
addrs = _resolve_hostname_ips(host)
if not addrs or any(_is_private_address(a) for a in addrs):
raise httpx.RequestError(f"Blocked non-public URL: {url}")
return addrs
class _PinnedBackend(httpcore.NetworkBackend):
"""Network backend that connects to a pre-resolved IP.
httpcore derives the TLS SNI and the ``Host`` header from the URL's
origin, not from the host argument passed to ``connect_tcp``. So
routing the TCP connect to a resolved IP while leaving the URL
untouched keeps SNI / vhost behaviour correct and closes the
DNS-rebinding TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress):
self._ip = str(ip)
self._real = httpcore.SyncBackend()
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options=None,
):
return self._real.connect_tcp(
self._ip, port, timeout, local_address, socket_options
)
def connect_unix_socket(self, path, timeout=None, socket_options=None):
return self._real.connect_unix_socket(path, timeout, socket_options)
def sleep(self, seconds: float) -> None:
return self._real.sleep(seconds)
# Map httpcore exception classes to their httpx equivalents. Built
# once at import time from the public exception classes; avoids any
# import of httpx's private transport machinery. httpcore's
# ``ConnectionNotAvailable`` is a pool-internal signal (the pool will
# close and retry on its own) — we never expect to see it surface to
# a transport caller, so it has no httpx counterpart here.
_HTTPCORE_TO_HTTPX_EXC = {
httpcore.ConnectError: httpx.ConnectError,
httpcore.ConnectTimeout: httpx.ConnectTimeout,
httpcore.LocalProtocolError: httpx.LocalProtocolError,
httpcore.NetworkError: httpx.NetworkError,
httpcore.PoolTimeout: httpx.PoolTimeout,
httpcore.ProtocolError: httpx.ProtocolError,
httpcore.ProxyError: httpx.ProxyError,
httpcore.ReadError: httpx.ReadError,
httpcore.ReadTimeout: httpx.ReadTimeout,
httpcore.RemoteProtocolError: httpx.RemoteProtocolError,
httpcore.TimeoutException: httpx.TimeoutException,
httpcore.UnsupportedProtocol: httpx.UnsupportedProtocol,
httpcore.WriteError: httpx.WriteError,
httpcore.WriteTimeout: httpx.WriteTimeout,
}
class _PinnedTransport(httpx.BaseTransport):
"""Transport that pins every TCP connect to a pre-resolved IP.
Uses only the public ``httpcore`` and ``httpx`` APIs no
subclassing of ``httpx.HTTPTransport``, no reads of private
``httpcore.ConnectionPool`` attributes, no imports from
``httpx private transport internals``. The URL is passed through unchanged so SNI
/ vhost work as if httpx had been given the hostname directly;
only the TCP destination is pinned, closing the DNS-rebinding
TOCTOU between the SSRF check and the connect.
"""
def __init__(self, ip: ipaddress._BaseAddress, *, http2: bool = False):
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
http1=True,
http2=http2,
network_backend=_PinnedBackend(ip),
)
def __enter__(self):
self._pool.__enter__()
return self
def __exit__(self, exc_type=None, exc_value=None, traceback=None) -> None:
self._pool.__exit__(exc_type, exc_value, traceback)
def handle_request(self, request: httpx.Request) -> httpx.Response:
httpcore_req = httpcore.Request(
method=request.method,
url=httpcore.URL(
scheme=request.url.raw_scheme,
host=request.url.raw_host,
port=request.url.port,
target=request.url.raw_path,
),
headers=request.headers.raw,
content=request.stream,
extensions=request.extensions,
)
try:
httpcore_resp = self._pool.handle_request(httpcore_req)
# Eager materialisation matches the original
# ``response.text`` usage in fetch_webpage_content. The
# sync pool's stream is a plain Iterable[bytes] despite
# the httpcore type hint unioning the async variant.
content = b"".join(cast(Iterable[bytes], httpcore_resp.stream))
except Exception as exc:
mapped = _HTTPCORE_TO_HTTPX_EXC.get(type(exc))
if mapped is not None:
raise mapped(str(exc)) from exc
raise
return httpx.Response(
status_code=httpcore_resp.status,
headers=httpcore_resp.headers,
content=content,
extensions=httpcore_resp.extensions,
)
def close(self) -> None:
self._pool.close()
class BodyTooLargeError(Exception):
"""The server declared a body larger than the hard fetch ceiling."""
def __init__(self, url: str, declared_bytes: int):
self.url = url
self.declared_bytes = declared_bytes
super().__init__(
f"response body is {declared_bytes:,} bytes, over the "
f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
)
class _CappedFetch:
"""Result of a size-capped streaming GET.
Carries just what fetch_webpage_content needs from an httpx.Response,
plus the cap bookkeeping: the (possibly truncated) body, whether the
cap cut it short, and the size the server declared via Content-Length
(wire bytes; None when absent).
"""
__slots__ = ("status_code", "headers", "content", "truncated",
"declared_bytes", "encoding", "url")
def __init__(self, status_code, headers, content, truncated,
declared_bytes, encoding, url):
self.status_code = status_code
self.headers = headers
self.content = content
self.truncated = truncated
self.declared_bytes = declared_bytes
self.encoding = encoding
self.url = url
@property
def text(self) -> str:
return self.content.decode(self.encoding or "utf-8", errors="replace")
def raise_for_status(self):
if self.status_code >= 400:
request = httpx.Request("GET", self.url)
raise httpx.HTTPStatusError(
f"HTTP {self.status_code} for {self.url}",
request=request,
response=httpx.Response(self.status_code, request=request),
)
def _get_public_url(url: str, headers: dict, timeout: int, max_redirects: int = 5,
max_bytes: int = None) -> "_CappedFetch":
"""Capped streaming GET with SSRF-guarded, DNS-pinned manual redirects.
Each hop is resolved once, validated as public, and then the actual TCP
connection is pinned to that resolved IP. The request URL is left unchanged
so Host and TLS SNI keep the original hostname.
"""
cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
current = url
for _ in range(max_redirects + 1):
ips = _resolve_public_ips(current)
# Force identity transfer-encoding. With gzip/deflate the wire bytes
# and Content-Length can be a small fraction of the decoded body, so a
# tiny compressed response could pass the hard-cap preflight and then
# expand past the ceiling in one decoded chunk before the streamed cap
# below can slice it.
req_headers = dict(headers or {})
req_headers["Accept-Encoding"] = "identity"
with httpx.Client(
headers=req_headers,
timeout=timeout,
follow_redirects=False,
transport=_PinnedTransport(ips[0]),
) as client:
with client.stream("GET", current) as response:
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("location")
if not location:
return _CappedFetch(response.status_code, response.headers, b"",
False, None, response.encoding, str(response.url))
current = urljoin(str(response.url), location)
continue
# A server can ignore the identity request and still return a
# compressed body; httpx.iter_bytes would then decode it, and a
# tiny gzip can balloon into one decoded chunk far past the cap.
# Refuse compressed Content-Encoding so the streamed cap stays
# a real memory bound.
enc = (response.headers.get("content-encoding") or "").strip().lower()
if enc and enc != "identity":
raise httpx.RequestError(
f"Refusing compressed response (Content-Encoding: {enc}) after "
"requesting identity: cannot bound decoded body size",
request=httpx.Request("GET", current),
)
declared = None
raw_len = response.headers.get("content-length")
if raw_len and raw_len.isdigit():
declared = int(raw_len)
if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
raise BodyTooLargeError(current, declared)
chunks = []
read = 0
truncated = False
for chunk in response.iter_bytes():
read += len(chunk)
if read > cap:
keep = cap - (read - len(chunk))
if keep > 0:
chunks.append(chunk[:keep])
truncated = True
break
chunks.append(chunk)
return _CappedFetch(response.status_code, response.headers,
b"".join(chunks), truncated, declared,
response.encoding, str(response.url))
raise httpx.RequestError("Too many redirects", request=httpx.Request("GET", current))
# PDF extraction (optional dependency) # PDF extraction (optional dependency)
try: try:
from pdfminer.high_level import extract_text as pdf_extract_text from pdfminer.high_level import extract_text as pdf_extract_text
-88
View File
@@ -1,88 +0,0 @@
# Specs DocumentMap
Last updated: dev@e71f8ce | 2026-08-25
This folder is the compact implementation-truth map for humans and coding agents working on Odysseus. Read this file first, then open only the subsystem specs that match the work.
Specs are living notes about current code shape and intended contracts. They are not product marketing, not PR planning, not templates, and not a replacement for source inspection or tests.
This `_readme.md` is the DocumentMap and control document. It is intentionally exempt from subsystem `Scope` and `Current Gaps` sections; keep it limited to the quality contract, working rules, subsystem map, and cross-cutting update triggers.
## Quality Contract
Each subsystem spec should stay compact and useful under context pressure:
- Start with `Last updated: dev@<short-sha> | YYYY-MM-DD`, using the
upstream `dev` commit the spec text was inspected against.
- Use a concrete `Scope` section that names real files, route surfaces, frontend modules, data stores, and integration points.
- Use domain-specific sections. Do not force every spec into the same headings when the subsystem needs `Streaming`, `Tool Results`, `Optional Dependencies`, `Current Gaps`, or another focused section.
- State ownership clearly: which file owns a mapping, which layer only forwards state, and which caller requests behavior without owning implementation.
- Include runtime behavior bullets for flows that matter.
- Include "Current call sites include" when behavior is spread across many files.
- Record transitional compatibility notes, especially `src/` versus `services/` duplication.
- Record degraded, optional, or platform behavior where it changes runtime expectations.
- Record policy/provenance where relevant: untrusted context, encrypted secrets, API token scopes, optional dependency/license implications, generated media, or user data.
- End with `Current Gaps` only when there is a real known gap, not as filler.
If code and specs disagree, treat code as ground truth. Update specs only when
the current task explicitly includes spec maintenance or the PR intentionally
includes specs; otherwise report the drift in the relevant issue, PR review, or
project documentation.
## Working Rules
- Start here before substantial work.
- Read the related subsystem spec before changing code in that area. For cross-cutting work, include the owning domain spec plus route/runtime, auth/security, persistence, frontend, tool/context, integration, and testing/devops specs as applicable.
- Treat specs as read-only context during ordinary project work, PR review, and code review. Do not edit specs unless the user explicitly asks for spec work or the current PR intentionally includes spec changes.
- During explicit spec-maintenance work, update the related spec when source inspection shows behavior, ownership, security boundaries, data shape, import paths, or implementation contracts have changed.
- During ordinary work, record source/spec drift in the relevant issue, PR review, or project documentation instead of mutating specs.
- Keep specs dense but readable. Prefer current facts and invariants over broad explanation.
- Every non-index `specs/*.md` file should appear exactly once in the Subsystem Map with a one-line description and no dead link.
- Specs contain implementation truth. Planning, research, branch notes, and decisions belong in tracked project docs. Drafts, audit reports, raw exports, and exploratory gap lists are not authoritative until promoted into tracked docs or specs.
- Use repo source and these specs as the authority for Odysseus architecture. Do not treat global skill registries or external agent metadata as repo ground truth.
## Subsystem Map
- [runtime.md](runtime.md): FastAPI startup, router registration, static serving, lifespan, app-wide middleware.
- [auth-security.md](auth-security.md): auth, privileges, API tokens, security headers, untrusted data, SSRF and admin boundaries.
- [persistence.md](persistence.md): SQLite models, startup migrations, encrypted columns, ownership columns, data directory rules.
- [chat.md](chat.md): chat routes, sessions, streaming, uploads-in-chat, compare handoff, research/chat mode dispatch.
- [compare.md](compare.md): model A/B comparison runs, voting/history, compare frontend panes, compare ownership.
- [llm-models.md](llm-models.md): LLM provider calls, endpoint discovery, model context length, fallbacks, model endpoints.
- [model-capability-canonical.md](model-capability-canonical.md): canonical provider/model capability shapes, evidence, payload resolution, and safe fallback.
- [model-quirks.md](model-quirks.md): model-specific behavior observations, evidence, and promotion gates.
- [model-providers/_readme.md](model-providers/_readme.md): provider-by-provider API/catalog shape index and compatibility status.
- [agent-tools.md](agent-tools.md): agent loop, tool schemas, tool execution, tool retrieval, tool security, MCP tool exposure.
- [context-building.md](context-building.md): URL/search/RAG/memory/skills/YouTube/email/tool-output context, untrusted wrapping, unavailable context, intent boundaries.
- [search.md](search.md): web search providers, ranking, cache/analytics, URL fetch/content extraction, `src.search`/`services.search` split.
- [documents-rag-uploads.md](documents-rag-uploads.md): uploads, documents, PDF/form handling, personal docs, RAG/vector stores.
- [memory-skills.md](memory-skills.md): memory storage, semantic memory, skill extraction/formatting, owner isolation.
- [research.md](research.md): deep research jobs, synthesis, sources, research library, research UI panel.
- [calendar-tasks-notes.md](calendar-tasks-notes.md): CalDAV calendars, scheduled tasks, reminders, assistant runs, notes/todos.
- [email-contacts.md](email-contacts.md): IMAP/SMTP email, email library, scheduled mail, contacts/CardDAV.
- [gallery-editor-media.md](gallery-editor-media.md): gallery, generated media, image editor drafts, signatures, emoji/font helpers.
- [cookbook-hwfit.md](cookbook-hwfit.md): model downloads, local/remote model serving, hardware detection, fit ranking.
- [speech.md](speech.md): STT and TTS services, routes, settings, optional dependencies.
- [frontend.md](frontend.md): static SPA, module loading, UI conventions, major JS areas, no-build frontend shape.
- [integrations.md](integrations.md): Codex/Claude scoped APIs, companion pairing, webhooks, external agent access.
- [shell-mcp.md](shell-mcp.md): shell execution, background jobs, MCP manager, built-in MCP servers.
- [settings-admin.md](settings-admin.md): settings, preferences, presets, backup/import/export, diagnostics, admin wipe.
- [testing-devops.md](testing-devops.md): pytest, JS tests, Docker, scripts, requirements, local dev expectations.
## Cross-Cutting Spec Update Triggers
Use these triggers only during explicit spec-maintenance work or a PR that
intentionally includes specs. For ordinary work and code review, use the same
list to choose which specs to read and where to report drift.
- New route file or route prefix: update [runtime.md](runtime.md) and the owning subsystem spec.
- New SQLAlchemy model, column migration, durable JSON/local store, data directory, backup/import domain, or non-SQL persistence behavior: update [persistence.md](persistence.md) and the owning subsystem spec.
- New tool, tool schema, agent prompt rule, or tool security behavior: update [agent-tools.md](agent-tools.md) and [context-building.md](context-building.md) if it adds model context.
- New MCP runtime/config/built-in behavior: update [shell-mcp.md](shell-mcp.md), [agent-tools.md](agent-tools.md), and [context-building.md](context-building.md) when MCP tool results enter model context.
- New external content source, tool result, MCP/app API result, or integration result shown to an LLM: update [context-building.md](context-building.md) and [auth-security.md](auth-security.md).
- New API-token scope, scoped external API, webhook, companion/pairing route, generic integration provider, or external-agent helper bundle: update [integrations.md](integrations.md), [auth-security.md](auth-security.md), and the owning subsystem spec.
- New secret store, decrypted-secret return path, settings backup/import/export behavior, diagnostics/log output, vault/tool secret flow, `.env*` policy change, or credential-bearing CLI output: update [auth-security.md](auth-security.md), [settings-admin.md](settings-admin.md), [testing-devops.md](testing-devops.md), and the owning subsystem spec.
- New optional dependency, degraded fallback, platform/Docker/native/launcher difference, GPU overlay behavior, or retired compatibility shim: update [testing-devops.md](testing-devops.md) and the owning subsystem spec; also update [runtime.md](runtime.md), [llm-models.md](llm-models.md), [shell-mcp.md](shell-mcp.md), [cookbook-hwfit.md](cookbook-hwfit.md), or [persistence.md](persistence.md) when that layer owns the behavior.
- New frontend module or modal/tool surface: update [frontend.md](frontend.md) and the owning subsystem spec.
- New static/PWA/service-worker/cache/CSP behavior: update [frontend.md](frontend.md), [runtime.md](runtime.md), and [auth-security.md](auth-security.md) when headers or trust boundaries change.
- New CLI script: update [testing-devops.md](testing-devops.md) and the owning subsystem spec.
-157
View File
@@ -1,157 +0,0 @@
# Agent Tools
Last updated: dev@e71f8ce | 2026-08-25
## Scope
This spec covers agent/tool behavior in:
- `src/agent_loop.py`;
- `src/llm_core.py`;
- `src/tool_schemas.py`;
- `src/tool_execution.py`;
- `src/tool_policy.py`;
- `src/tool_index.py`;
- `src/tool_parsing.py`;
- `src/tool_security.py`;
- `src/tool_capabilities.py`;
- `src/tool_approval_scopes.py`;
- `src/tool_approvals.py`;
- `src/attachment_refs.py` and shared upload lifecycle helpers in
`src/upload_handler.py` / `src/tool_utils.py`;
- `src/tool_implementations.py`;
- `src/tools/*.py`;
- `src/builtin_actions.py`;
- `src/ai_interaction.py`;
- `src/action_intents.py`;
- `src/goal_based_extractor.py`;
- `src/teacher_escalation.py`;
- `src/agent_tools/` modules and compatibility facade;
- `src/mcp_manager.py`;
- `src/builtin_mcp.py`;
- `src/bg_jobs.py` and `src/bg_monitor.py`;
- `routes/chat_routes.py`, `routes/chat_helpers.py`, `routes/model_routes.py`, `routes/skills_routes.py`, canonical `routes/mcp/mcp_routes.py` plus its shim, and `routes/workspace_routes.py`;
- `mcp_servers/*.py`;
- frontend stream/admin/settings files that display tool events, workspaces, and disabled tools;
- `tests/test_agent_loop.py`, `tests/test_tool_*`, and focused MCP/public-policy/schema tests.
## Agent Loop
`src.agent_loop` owns agent prompt assembly, request-local current date/time insertion, tool retrieval, prompted tool-block handling, native tool-call consumption after `llm_core` normalizes provider events, multi-round execution, tool result insertion, final metrics, and fallback responses. It requests context from documents, skills, tool retrieval, and messages; it should not own domain-specific business logic for every tool. Its prompt rules now bias structured/long-form writing toward living documents, route active compose/email drafts back into existing email documents, and prefer first-class `web_search`/`web_fetch` tools over shell/Python/curl for current web lookups when web tools are enabled.
`src.llm_core` owns provider payloads, native tool-schema emission, and provider stream parsing. `agent_loop` consumes normalized tool-call events and decides whether and how to execute them.
Agent mode enters through chat routes, including auto-escalation from intent helpers, detached `agent_runs` streaming, resume/stop behavior, and frontend tool-event rendering.
Guide-only/no-tools turns are runtime policy, not prompt advice. `src.tool_policy` detects strong latest-turn directives such as guide-only mode, no-tools mode, and explicit requests not to use tools; it builds a `ToolPolicy` that hides schemas, disables known native tools, disables MCP for that turn, skips tool retrieval, suppresses local/workspace context injection, blocks document streaming/teacher escalation, and gives `tool_execution` a final execution backstop.
Plan mode is a read-only investigation path inside the same loop. It adds a denylist for known mutating tools, filters write/unknown MCP tools, prepends plan-mode instructions, and uses the `update_plan` tool only after a plan is approved for execution. The backend path still exists for compatibility, but current browser chat forces incoming `plan_mode` off and the old plan-window UI module is gone.
Workspace mode is request-scoped. Admin chat can send a workspace directory selected through `static/js/workspace.js`; `agent_loop` injects that fact early in the prompt and `tool_execution` confines bash, python, read/write/edit-file, and code-navigation tools to that root. `routes.workspace_routes` owns admin-only browse/vet APIs, skips hidden/symlink directory traversal, caps listings, and rejects sensitive/root paths before a workspace reaches chat.
## Tool Registry
Tool registration is split:
- `src.agent_tools` is now a package/facade. `TOOL_HANDLERS` maps native tool names to handler functions across filesystem, subprocess, web, document, interaction, model-interaction, background-job, session, and admin modules, while `TOOL_TAGS` keeps compatibility metadata and the global MCP manager handle;
- `src.tools` owns domain do_* implementations for calendar, contacts, Cookbook, image, notes, research, search, system, and vault tools. `src.tool_implementations` is now a compatibility facade that re-exports those symbols and lazy-loads admin manage_* symbols to avoid circular imports;
- `src.agent_tools.admin_tools` owns admin manage_* tools for endpoints, MCP, webhooks, tokens, and settings, including command validation for `manage_mcp`;
- `src.tool_parsing._TOOL_NAME_MAP` owns aliases and prompted-block parsing;
- `src.tool_schemas.FUNCTION_TOOL_SCHEMAS` and `function_call_to_tool_block()` own native schema and native-call conversion;
- `src.tool_index.BUILTIN_TOOL_DESCRIPTIONS` owns retrieval text;
- `src.tool_execution.execute_tool_block()` owns dispatch and hard execution gates;
- `routes.model_routes.py` and frontend settings/admin surfaces expose global disabled-tool controls.
When adding, removing, or renaming a tool, update the registry chain, execution dispatch, retrieval text, prompt wording, disabled-tool UI, and tests together.
`src.tool_index.ALWAYS_AVAILABLE` is the retrieval catalog for high-frequency tools such as shell/python, web search/fetch, read/write/edit-file, code-nav, `manage_memory`, `ask_user`, `update_plan`, selected Cookbook serve controls, and `app_api`. Current prompt/schema assembly preserves only selected base tools unconditionally, then adds intent-, skill-, and retrieval-relevant tools so unrelated schemas do not flood small contexts.
## Tool Retrieval And Execution
`src.tool_index.ToolIndex` owns candidate retrieval using embeddings/keywords and cached index data. Security filtering is not its hard boundary: `agent_loop` hides unavailable schemas, and `tool_execution` blocks disabled, admin-only, and public-restricted calls before dispatch.
`src.tool_execution` owns built-in tool execution, MCP dispatch, path confinement, background markers, output truncation, internal HTTP loopback, owner/admin checks, policy-blocked execution results, and formatting tool results for the model/UI. File tools support exact edit diffs, full-file writes, read line ranges, and workspace confinement. Code-navigation tools (`grep`, `glob`, `ls`) prefer `rg`/structured filesystem traversal over ad hoc shell commands. Uploaded-file context uses stable `attachment_ref` manifests and owner-checked URIs; a compatibility local path is exposed only after upload-root and tool-root confinement. Shared truncation, upload-handler registration, and MCP manager compatibility helpers live in `src.tool_utils`.
Tool retrieval has domain-specific hooks beyond generic similarity: contact queries can surface `resolve_contact`/`manage_contact`; matched skills can add `manage_skills` and their required toolsets to the relevant tool set; explicit admin intents can include admin schemas so prompt text and native schema emission match.
Interaction/session/model helper tools are native first-class tools, not prompt-only conventions. `ask_user` and `update_plan` live in `src.agent_tools.interaction_tools`, model delegation/listing helpers live in `model_interaction_tools`, session creation/list/send/manage helpers live in `session_tools`, and `manage_bg_jobs` lives in `bg_job_tools`.
Prompted-tool parsing includes recovery paths for local/provider text leaks: bare JSON after a web-tool mention, OpenAI-style raw `{"function": ...}` payloads, StepFun/Gemma/DSML markup, Hermes/Qwen JSON bodies nested inside `tool_call` wrappers, and `<function_model><function_call>...</function_call><parameters>...</parameters></function_model>` wrappers from local MLX/Exo models. The Qwen bare end marker requires its pipe delimiter so ordinary text cannot terminate a tool block. Non-dict JSON arguments are rejected back to empty args instead of crashing the turn, common `tex` typos normalize to `text`, and delimiter scans are forward-only so unterminated tool markup cannot drive quadratic rescans. Executed raw tool JSON is stripped from assistant text afterward; this is still not a general-purpose JSON-command parser.
Current call sites include:
- agent mode tool calls from `src.agent_loop`;
- MCP route configuration and built-in MCP registration;
- background job monitoring and auto-continue;
- skill tests, teacher escalation, scheduled tasks, and background follow-up loops;
- UI-control and AI interaction helpers.
## Streaming And Continuations
Agent streaming emits normal content plus tool progress/output, document stream/update, ask-user choices, plan updates, budget, round exhaustion, loop-breaker, intent-nudge exhaustion, metrics, teacher escalation, research anchor, and finish/error events. Frontend chat stream code and detached replay depend on stable event names. If the stream generator closes while awaiting an in-flight tool, the loop cancels and awaits that tool task so subprocess-backed work is not left orphaned.
Long-running bash jobs can be detached with background markers. `src.bg_jobs` owns persistent job state/result files; `src.bg_monitor` owns auto-continuation when jobs finish. Detached chat runs are in-memory and do not survive server restart, while background job state is disk-backed.
Loop-breaker final-answer rounds, explicit repeated-tool/intent-nudge guard events, round-cap continuation signals, optional verifier retries, and teacher escalation are recovery behavior owned by `agent_loop` and `src.teacher_escalation`.
Approval replay injects the sealed first tool result before the resumed model round. If that replay round has neither assistant prose nor reasoning, `_append_tool_results()` omits the empty assistant spacer so Anthropic-compatible payloads do not contain a rejected non-final empty assistant message; reasoning-only carriers remain a documented compatibility edge.
## Security And Policy
- `src.tool_security` owns non-admin blocked-tool decisions.
- Non-admin users must not reach admin tools through agent mode, MCP, retrieval, or loopback calls.
- Agent owner is passed from chat route `get_current_user(request)`. In `AUTH_ENABLED=false` mode this is `None`, not the `""` value returned by route dependencies. `blocked_tools_for_owner()`, schema hiding, and `execute_tool_block()` all use that owner.
- Current dev tool security treats explicit `AUTH_ENABLED=false` as single-user even when an auth store exists, while auth-enabled pre-setup callers remain non-admin.
- Path-based tools must remain confined to allowed roots and reject sensitive paths. Sensitive-path checks are case-insensitive and apply to direct file tools and code-navigation tools; `grep`/`glob`/`ls` must not become existence or content oracles for `.env`, SSH/GPG material, `id_rsa`, and similar denylisted paths.
- Tool output is bounded/truncated where native execution owns the path, including displayed agent-tool output through the shared truncation helper. MCP output must be treated as untrusted; central MCP-output truncation before model re-entry remains a gap.
- Provider-emitted native tool calls are requests, not authorization. `tool_execution` and route-level policy remain the authority.
- `src.tool_capabilities` classifies each tool's effects and result integrity. Once external/workspace-untrusted content becomes model-visible, the request/session security context permits only explicitly low-impact tools without interruption and requires exact approval for high-impact, unknown, and arbitrary MCP calls.
- `src.tool_approvals` seals an opaque, expiring exact first action plus server-only selected tools and continuation query to owner, session, origin run, tool content, workspace, capability snapshot, and—when relevant—document id/version/content digest. Chat choices grant the resumed task or the same chat session; both consume the exact first action, task scope bypasses the gate only during that resumed run, and chat scope is reconstructed only from a resolved card bound to the exact session id. The browser never receives selected tools/query and submits only task/chat/deny. Non-chat callers retain single-action behavior; new normal turns and superseding actions retire unresolved approvals without clearing taint.
- Tool results that expose remote or stored untrusted content arm the gate even when their tool status is failed. Content-free failures and server-generated policy/approval placeholders do not. Native/provider tool messages and fenced results carry model-visible untrusted metadata/wrapping instead of relying on prompt wording alone.
- Attachment-bearing document, note, and calendar tools owner-reserve internal
upload references before durable writes and fail without mutation when the
referenced upload is unavailable.
- Guide-only/no-tools mode blocks tools before prompt assembly, before execution, and in chat preprocessing paths that would otherwise fetch context or start tool-backed research.
- Plan mode is policy, not prompt advice: mutating native tools are disabled through schema-derived detection plus a static backstop, and write/unknown MCP tools are hidden and runtime-blocked for that turn.
## Internal Loopback
`do_app_api()` is implemented in `src.tools.system` and re-exported by `src.tool_implementations`. It owns generic app API loopback, OpenAPI discovery, method/path blocklists, and fixed local target behavior. `_internal_headers()` adds the process-secret internal-tool token and optional `X-Odysseus-Owner`; `core.middleware.require_admin()` and auth middleware own the corresponding bypass and owner-stamping rules. Route-specific owner handling must still be audited.
## MCP
`src.mcp_manager` owns configured MCP server lifecycle, discovered tool state, qualified MCP names, OpenAI schema conversion, call routing, generation invalidation, and connect/disconnect status. It supports stdio, SSE, and Streamable HTTP transports; Streamable HTTP can publish a `needs_auth` state and uses `src.mcp_oauth` for OAuth/OIDC-style authorization, token refresh, and encrypted token storage. Arbitrary MCP tools classify fail-high for approvals. `src.builtin_mcp` owns built-in server registration and the native-vs-MCP split. `mcp_servers/` owns server-specific tools for email, image generation, memory, RAG, and optional browser tooling.
Native bash, python, file, web search, and web fetch tools continue through native fallback even when MCP is unavailable. Browser MCP is optional and can be skipped when cached Playwright/NPX packages are missing. Public users get no MCP schemas, and any `mcp__*` execution attempt must be blocked.
MCP prompt/schema rendering includes server-provided input schemas, but names, types, and parameter hint text are sanitized and length-capped before entering the prompt. Per-server disabled tools filter listings, prompt descriptions, and function schemas; execution-time disabled-tool enforcement remains a separate hardening item.
## Intent And Recovery Helpers
`src.action_intents` owns deterministic chat-to-agent promotion hints and returns a category/reason so route logs can explain auto-escalation decisions. Explicit web-search language is category `web`; it can promote the turn into agent mode and narrow tools toward web search/fetch, but route policy requires explicit web-search enablement and honors explicit denial. It must avoid promoting explanatory questions into agent mode. `src.builtin_actions` owns scheduler/background actions outside the normal live agent loop. `src.teacher_escalation` owns recovery/escalation and skill-creation flows. `src.goal_based_extractor` is research-adjacent and should stay cross-referenced from research behavior rather than treated as ordinary tool execution.
When an email reader is active, browser chat passes active email metadata and the agent loop injects it as protected, untrusted context so default reply/draft behavior targets the selected message. Active email compose documents are handled as existing email drafts rather than generic new-document requests.
## Degraded Behavior
- ToolIndex can degrade to keyword selection when embeddings, Chroma, index
warmup, or vector retrieval timeouts fail.
- Agent mode can degrade from native function schemas to prompted fenced-block parsing based on provider/tool-support heuristics. Local Ollama `/v1` and native `/api` endpoints default to text tools unless the endpoint explicitly advertises `supports_tools`; `gpt-oss` remains text-tool by default unless the endpoint opts in.
- MCP startup failure is non-critical; route/status surfaces expose per-server errors.
- `ODYSSEUS_DISABLE_MCP`, missing `mcp`, uncached browser MCP packages, and per-server disabled tools can remove tools without blocking the app.
- Global `builtin_browser` disable behavior may not currently match qualified `mcp__builtin_browser__*` tool names.
## Current Gaps
- Tool descriptions are duplicated across `FUNCTION_TOOL_SCHEMAS`, agent prompt sections, and `BUILTIN_TOOL_DESCRIPTIONS`.
- Agent prompts remain heavy for small local context windows.
- Some AI-control helpers are still globally wired from app startup rather than a narrower service layer.
- Tool registry consistency is manual across handler maps, tags, aliases, schemas, retrieval descriptions, execution dispatch, settings/model routes, and frontend toggles.
- MCP disabled-tool changes can stale-cache tool retrieval because disabled maps are not always an index generation input.
- External MCP output still needs a single central size cap before model re-entry; untrusted-result metadata and the post-external-context action gate now cover the prompt-injection/authorization boundary.
- Auth-disabled/no-login owner propagation is inconsistent between route dependencies and chat/agent execution, so tool-security and native tool storage behavior need dedicated regression coverage.
- Agent tests mostly cover helpers and targeted regressions, including round-cap
and disconnect cancellation paths, but not an end-to-end fake-LLM
`stream_agent_loop` path with retrieval, native schemas, prompted blocks,
disabled/admin hiding, MCP tools, plan/workspace state, user-time context, and
tool-result SSE.

Some files were not shown because too many files have changed in this diff Show More