Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ee6502010 |
@@ -10,6 +10,16 @@ dist/
|
||||
build/
|
||||
.env
|
||||
.env.bak.*
|
||||
# Secrets: keep plaintext and every transient secrets.env variant out of
|
||||
# the build context. If an encrypted secrets.env is used, it is mounted
|
||||
# at runtime — never baked into the image. Mirrored in .gitignore.
|
||||
secrets.env
|
||||
secrets.env.*
|
||||
secrets.env~
|
||||
.secrets.env.swp
|
||||
.secrets.env.swo
|
||||
**/#secrets.env#
|
||||
!secrets.env.example
|
||||
/data/
|
||||
/logs/
|
||||
.git/
|
||||
@@ -20,6 +30,8 @@ build/
|
||||
.idea/
|
||||
dev-docs/
|
||||
docs/
|
||||
website/
|
||||
assets/branding/
|
||||
*.md
|
||||
*.db
|
||||
*.sqlite
|
||||
|
||||
@@ -67,6 +67,11 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# Auth & Security
|
||||
# ============================================================
|
||||
|
||||
# Optional backend workspace used automatically by the WebUI when no workspace
|
||||
# is saved in the browser. This must be a directory visible to the backend;
|
||||
# with host-workspace mapping, a host path is translated before vetting.
|
||||
# ODYSSEUS_WORKSPACE_DEFAULT=/workspace/project
|
||||
|
||||
# Enable authentication (default: true)
|
||||
# AUTH_ENABLED=true
|
||||
|
||||
@@ -76,12 +81,32 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# Change this if another local service already uses 7000 (macOS AirPlay often does).
|
||||
# APP_PORT=7000
|
||||
|
||||
# Optional HTTP address advertised in companion/mobile pairing codes. Set this
|
||||
# when Docker would otherwise advertise a container address or loopback. Use a
|
||||
# LAN or Tailscale IPv4 address, a single-label hostname, or an mDNS *.local
|
||||
# name that the phone can reach. HTTPS and public hostnames are not supported
|
||||
# by the current companion client. Do not include credentials, a path, query,
|
||||
# or fragment.
|
||||
# COMPANION_BASE_URL=http://192.168.1.50:7000
|
||||
|
||||
# Development-only auth bypass for loopback requests.
|
||||
# Keep false for Docker, LAN, reverse proxy, and any shared deployment.
|
||||
# LOCALHOST_BYPASS=false
|
||||
|
||||
# Mark session cookies Secure. Set true when Odysseus is served through HTTPS
|
||||
# by a trusted reverse proxy or private access gateway.
|
||||
# Skip the external-context exact-approval pause for unattended local agents.
|
||||
# Keep false for shared or internet-exposed deployments.
|
||||
# ODYSSEUS_UNATTENDED_MODE=false
|
||||
|
||||
# Optional post-external-context tool approval gate. Off by default because it
|
||||
# can block normal agent work; enable only for deployments that want this fence.
|
||||
# ODYSSEUS_TOOL_APPROVAL_GATE=0
|
||||
|
||||
# Mark session cookies Secure. Left unset, this follows the request scheme:
|
||||
# an HTTPS login gets a Secure cookie, a plain-HTTP one does not. Set true to
|
||||
# force it on, or false to force it off while you still serve plain HTTP.
|
||||
# Upgrading: this used to default to false. Drop a leftover SECURE_COOKIES=false
|
||||
# from your .env unless you still need that escape hatch — it keeps HTTPS logins
|
||||
# on a non-Secure cookie.
|
||||
# SECURE_COOKIES=true
|
||||
|
||||
# Optional: pre-seed the first admin password during setup.
|
||||
@@ -130,6 +155,42 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# FASTEMBED_MODEL=sentence-transformers/all-MiniLM-L6-v2
|
||||
# FASTEMBED_CACHE_PATH= # defaults to ~/.cache/fastembed
|
||||
|
||||
# ============================================================
|
||||
# Google OAuth2 (Google Workspace / .edu email accounts)
|
||||
# ============================================================
|
||||
# Required to use the "Connect with Google" OAuth flow in email account setup.
|
||||
# Create credentials at: console.cloud.google.com → APIs & Services → Credentials
|
||||
# 1. Enable the Gmail API for your project.
|
||||
# 2. Configure the OAuth consent screen (User Type: Internal for Workspace orgs).
|
||||
# Add scopes: https://mail.google.com/ and email.
|
||||
# 3. Create an OAuth 2.0 Client ID (type: Web application).
|
||||
# Add your redirect URI: http://localhost:7000/api/email/oauth/google/callback
|
||||
# (replace host/port for hosted installs).
|
||||
# 4. Copy the Client ID and Client Secret below.
|
||||
#
|
||||
# GOOGLE_OAUTH_CLIENT_ID=your-client-id.apps.googleusercontent.com
|
||||
# GOOGLE_OAUTH_CLIENT_SECRET=replace-with-client-secret
|
||||
#
|
||||
# Set this explicitly for HTTPS, reverse-proxy, or hosted deployments. The
|
||||
# value must exactly match an authorized redirect URI in the Google client.
|
||||
# Local HTTP setups may use the callback URL inferred by the application.
|
||||
# GOOGLE_OAUTH_REDIRECT_URI=https://your-domain.com/api/email/oauth/google/callback
|
||||
|
||||
# Origin the MCP OAuth callback is sent back to, for remote (Streamable HTTP)
|
||||
# MCP servers that register it dynamically. Defaults to http://localhost:$APP_PORT,
|
||||
# which is right only when you reach Odysseus directly on that port. Set it for
|
||||
# HTTPS, reverse-proxy, hosted, and Docker installs — inside the container the
|
||||
# app always listens on 7000 and cannot see the host port map, so the default is
|
||||
# wrong there whenever APP_PORT is not 7000.
|
||||
#
|
||||
# Not for Google MCP servers. Those use Desktop App credentials, and Google only
|
||||
# accepts loopback redirect URIs for that client type, so a public origin here is
|
||||
# rejected with redirect_uri_mismatch. Leave it unset for a Google-only install:
|
||||
# the loopback default is what Google wants, and remote users finish through the
|
||||
# paste-back page, which never has to load the redirect.
|
||||
# https://developers.google.com/identity/protocols/oauth2/native-app
|
||||
# OAUTH_REDIRECT_BASE_URL=https://your-domain.com
|
||||
|
||||
# ============================================================
|
||||
# Misc
|
||||
# ============================================================
|
||||
@@ -168,6 +229,58 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=26214400 # email compose attachment (25 MB)
|
||||
# ODYSSEUS_STT_MAX_AUDIO_BYTES=26214400 # speech-to-text audio (25 MB)
|
||||
# ODYSSEUS_ICS_MAX_BYTES=10485760 # calendar .ics import (10 MB)
|
||||
# ODYSSEUS_TTS_CACHE_MAX_BYTES=524288000 # TTS cache (500 MB)
|
||||
|
||||
# ============================================================
|
||||
# Host Docker access (explicit opt-in)
|
||||
# ============================================================
|
||||
# Default Docker Compose does not mount /var/run/docker.sock. Existing
|
||||
# Ollama, vLLM, and other OpenAI-compatible endpoints remain usable without it.
|
||||
#
|
||||
# Enable this only for intentional Cookbook/local Docker-daemon management.
|
||||
# Raw socket access is high-trust and can grant broad control over the host
|
||||
# Docker daemon. Set DOCKER_GID to the host docker group's numeric GID.
|
||||
# Put these values in .env, or export them before running docker compose.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-docker.yml
|
||||
# DOCKER_GID=963
|
||||
# docker/host-docker.yml sets this inside the container. Keep it paired
|
||||
# with the socket overlay; setting it alone is not sufficient.
|
||||
# ODYSSEUS_ENABLE_HOST_DOCKER=true
|
||||
#
|
||||
# Host Docker access can be combined with one GPU overlay:
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml:docker/host-docker.yml
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml:docker/host-docker.yml
|
||||
|
||||
# ============================================================
|
||||
# Host workspace access (explicit opt-in)
|
||||
# ============================================================
|
||||
# Docker installs normally see only the container filesystem and /app/data.
|
||||
# Enable this when the agent should edit a real host workspace like Codex.
|
||||
# This is high-trust: the mounted tree is writable by the Odysseus container.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-workspace.yml
|
||||
# ODYSSEUS_HOST_WORKSPACE_DIR=/home/you
|
||||
# ODYSSEUS_HOST_WORKSPACE_MOUNT=/host/workspace
|
||||
#
|
||||
# Host workspace access can be combined with host Docker access and GPU overlays:
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-workspace.yml:docker/host-docker.yml
|
||||
|
||||
# ============================================================
|
||||
# Host network access (explicit opt-in, Linux Docker)
|
||||
# ============================================================
|
||||
# Docker bridge networking hides some host/LAN/VPN behavior from the agent:
|
||||
# mDNS, some LAN discovery, local VPN/Tailscale state, and host namespace
|
||||
# assumptions may differ from native Codex. Enable this only for high-trust
|
||||
# local installs where the Odysseus container should share the host network.
|
||||
#
|
||||
# With host networking, Docker port publishing is disabled and the app listens
|
||||
# directly on APP_PORT. The bundled SearXNG/Chroma services stay in Docker and
|
||||
# are reached through their host-published loopback ports.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-workspace.yml:docker/host-network.yml
|
||||
# APP_BIND=127.0.0.1
|
||||
# APP_PORT=7000
|
||||
# ODYSSEUS_HOST_NETWORK_SEARXNG_INSTANCE=http://127.0.0.1:8080
|
||||
# ODYSSEUS_HOST_NETWORK_CHROMADB_HOST=127.0.0.1
|
||||
# ODYSSEUS_HOST_NETWORK_CHROMADB_PORT=8100
|
||||
|
||||
# ============================================================
|
||||
# GPU support (Docker Compose)
|
||||
@@ -190,3 +303,12 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# These overlays only expose the GPU devices. The slim Odysseus image
|
||||
# still needs CUDA/ROCm userspace via Cookbook -> Dependencies (vLLM,
|
||||
# llama-cpp-python, etc.) before models can actually serve on GPU.
|
||||
|
||||
# ============================================================
|
||||
# Storage Paths (Docker Compose)
|
||||
# ============================================================
|
||||
|
||||
# APP_DATA_DIR=./data
|
||||
# APP_LOGS_DIR=./logs
|
||||
# Maximum serialized layered photo-editor draft size (default: 256 MiB).
|
||||
ODYSSEUS_EDITOR_DRAFT_MAX_BYTES=268435456
|
||||
|
||||
@@ -15,6 +15,13 @@ docker/entrypoint.sh text eol=lf
|
||||
*.cmd text eol=crlf
|
||||
*.bat text eol=crlf
|
||||
|
||||
# Vendored third-party bundles in static/lib/ are published minified artifacts
|
||||
# and must stay byte-identical to what npm ships — stripping trailing whitespace
|
||||
# to satisfy `git diff --check` would desync them from the upstream release. Turn
|
||||
# the whitespace check off for that tree instead, and keep the bundles out of
|
||||
# GitHub's language statistics.
|
||||
static/lib/** -whitespace linguist-vendored
|
||||
|
||||
# Binary assets — never normalize.
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
# Code owners.
|
||||
#
|
||||
# Intentionally empty for now. The catch-all rule that mapped every path to a
|
||||
# single owner froze all merges the moment "Require review from Code Owners"
|
||||
# was enabled, because no other maintainer's approval could satisfy the gate.
|
||||
# A per-area ownership map (security/auth, CI, frontend, agent internals, with
|
||||
# multiple named owners per line) is being worked out in issue #593; once
|
||||
# agreed it replaces this file. Until then, required reviews and the security
|
||||
# CI gate (website/security-ci.md) remain in force via branch protection.
|
||||
@@ -6,26 +6,38 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||
**Before submitting:** search [open issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
and [discussions](https://github.com/odysseus-dev/odysseus/discussions) first.
|
||||
Duplicate reports slow things down.
|
||||
|
||||
For security vulnerabilities, **do not open a public issue** —
|
||||
use [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new)
|
||||
and read [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md) first.
|
||||
use [GitHub Security Advisories](https://github.com/odysseus-dev/odysseus/security/advisories/new)
|
||||
and read [SECURITY.md](https://github.com/odysseus-dev/odysseus/blob/main/SECURITY.md) first.
|
||||
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
options:
|
||||
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and did not find an existing report of this bug.
|
||||
- label: I searched [open issues](https://github.com/odysseus-dev/odysseus/issues?q=is%3Aissue+is%3Aopen) and [discussions](https://github.com/odysseus-dev/odysseus/discussions) and did not find an existing report of this bug.
|
||||
required: true
|
||||
- label: This is **not** a security vulnerability. (Vulnerabilities go to [GitHub Security Advisories](https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new) — see [SECURITY.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/SECURITY.md).)
|
||||
- label: This is **not** a security vulnerability. (Vulnerabilities go to [GitHub Security Advisories](https://github.com/odysseus-dev/odysseus/security/advisories/new) — see [SECURITY.md](https://github.com/odysseus-dev/odysseus/blob/main/SECURITY.md).)
|
||||
required: true
|
||||
- label: I am running the latest code from the `dev` branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please `git pull` the latest `dev` before filing.
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: revision
|
||||
attributes:
|
||||
label: Odysseus Revision
|
||||
description: |
|
||||
From the repository root (on the host when using Docker), run
|
||||
`git show -s --abbrev=12 --format='%h (%cs)' HEAD`
|
||||
and paste the output exactly.
|
||||
placeholder: "1fef4929cf1d (2026-08-11)"
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: install-method
|
||||
attributes:
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Question / Need Help
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/q-a
|
||||
url: https://github.com/odysseus-dev/odysseus/discussions/categories/q-a
|
||||
about: Ask how-to questions, setup help, and model configuration questions here. Issues are for confirmed bugs and concrete proposals only.
|
||||
|
||||
- name: Idea or Suggestion
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas
|
||||
url: https://github.com/odysseus-dev/odysseus/discussions/categories/ideas
|
||||
about: Discuss ideas and gauge interest before opening a formal feature request. If there is already a discussion, link it in your feature request.
|
||||
|
||||
- name: Security Vulnerability
|
||||
url: https://github.com/pewdiepie-archdaemon/odysseus/security/advisories/new
|
||||
url: https://github.com/odysseus-dev/odysseus/security/advisories/new
|
||||
about: Report vulnerabilities privately via GitHub Security Advisories — never as a public issue. Read SECURITY.md before reporting.
|
||||
|
||||
@@ -6,22 +6,22 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues)
|
||||
and [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) first.
|
||||
Feature requests that duplicate [ROADMAP.md](https://github.com/pewdiepie-archdaemon/odysseus/blob/main/ROADMAP.md)
|
||||
**Before submitting:** search [open issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
and [discussions](https://github.com/odysseus-dev/odysseus/discussions) first.
|
||||
Feature requests that duplicate [ROADMAP.md](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md)
|
||||
or an existing open issue will be closed as duplicates.
|
||||
|
||||
If your idea needs community input before it becomes a concrete proposal,
|
||||
start a [discussion](https://github.com/pewdiepie-archdaemon/odysseus/discussions/categories/ideas) instead.
|
||||
start a [discussion](https://github.com/odysseus-dev/odysseus/discussions/categories/ideas) instead.
|
||||
|
||||
- type: checkboxes
|
||||
id: prerequisites
|
||||
attributes:
|
||||
label: Prerequisites
|
||||
options:
|
||||
- label: I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues?q=is%3Aissue+is%3Aopen) and this has not already been proposed.
|
||||
- label: I searched [open issues](https://github.com/odysseus-dev/odysseus/issues?q=is%3Aissue+is%3Aopen) and this has not already been proposed.
|
||||
required: true
|
||||
- label: I searched [discussions](https://github.com/pewdiepie-archdaemon/odysseus/discussions) and this is not already being debated there.
|
||||
- label: I searched [discussions](https://github.com/odysseus-dev/odysseus/discussions) and this is not already being debated there.
|
||||
required: true
|
||||
- label: This is a concrete, actionable proposal — not a vague "it would be nice if..." request.
|
||||
required: true
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
# Dependabot keeps dependencies and pinned action versions current.
|
||||
#
|
||||
# Why this matters for security: every workflow in this repo pins its GitHub
|
||||
# Actions to an exact commit (a SHA), which is safe but freezes them in time.
|
||||
# Dependabot opens a small, reviewable pull request whenever a newer version
|
||||
# exists -- for Python packages, npm packages, the Docker base image, and the
|
||||
# pinned Actions themselves -- so staying patched does not require manual work.
|
||||
# Updates are grouped so a week's bumps arrive as one PR per ecosystem, not a
|
||||
# flood of separate ones.
|
||||
|
||||
version: 2
|
||||
updates:
|
||||
# Python dependencies (requirements.txt + requirements-optional.txt).
|
||||
- package-ecosystem: pip
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
python:
|
||||
patterns: ["*"]
|
||||
|
||||
# Frontend / tooling npm packages (package.json).
|
||||
- package-ecosystem: npm
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
npm:
|
||||
patterns: ["*"]
|
||||
|
||||
# The pinned action SHAs used across .github/workflows.
|
||||
- package-ecosystem: github-actions
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
groups:
|
||||
actions:
|
||||
patterns: ["*"]
|
||||
|
||||
# The Docker base image in the Dockerfile.
|
||||
- package-ecosystem: docker
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: weekly
|
||||
open-pull-requests-limit: 5
|
||||
@@ -0,0 +1,123 @@
|
||||
# Pull Request Review Template
|
||||
|
||||
Use this shape as a copyable reference for substantive PR reviews; GitHub does
|
||||
not auto-apply this file to review comments. Omit sections that do not add
|
||||
useful signal. Lead with confirmed findings; keep speculative notes out of the
|
||||
public review unless they are framed as a concrete open question.
|
||||
|
||||
## Small PR Path
|
||||
|
||||
For narrow docs, typo, test-only, or obvious local fixes, a short review is
|
||||
enough:
|
||||
|
||||
```md
|
||||
LGTM after checking:
|
||||
- scope:
|
||||
- validation:
|
||||
- residual risk:
|
||||
```
|
||||
|
||||
Use the fuller structure below for larger, risky, multi-finding, or
|
||||
security-sensitive reviews.
|
||||
|
||||
## Findings
|
||||
|
||||
**<sub><sub></sub></sub> issue (test): Short issue title**
|
||||
|
||||
- **Problem:** Concrete broken flow, contract, input, or risk.
|
||||
|
||||
- **Impact:** Why this matters to users, CI, maintainers, data, security, or scale.
|
||||
|
||||
- **Ask:** Smallest practical correction or decision the author should make.
|
||||
|
||||
- **Location:** `path:line`
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **question (scope, non-blocking): Short author question** Ask the concrete
|
||||
intent, scope, or tradeoff question.
|
||||
|
||||
## Validation
|
||||
|
||||
- Ran:
|
||||
- Not run:
|
||||
- Residual risk:
|
||||
|
||||
## PR Hygiene
|
||||
|
||||
- Target/template/checks:
|
||||
- Related, duplicate, or superseding context:
|
||||
|
||||
## No Findings Variant
|
||||
|
||||
```md
|
||||
## Findings
|
||||
|
||||
none confirmed
|
||||
|
||||
## Validation
|
||||
|
||||
- Ran:
|
||||
- Not run:
|
||||
- Residual risk:
|
||||
```
|
||||
|
||||
## Legend
|
||||
|
||||
- **Findings:** Verified, author-actionable issues that should be fixed or
|
||||
consciously accepted before merge.
|
||||
- **Priority badges:** The shields.io badges below are optional formatting for
|
||||
priority labels. Plain `P0`, `P1`, `P2`, or `P3` text is also acceptable when
|
||||
an external image dependency is undesirable or may not render.
|
||||
- **P0:** `` -
|
||||
release-blocking or actively dangerous.
|
||||
- **P1:** `` -
|
||||
serious bug, security risk, data-loss risk, or broken primary flow.
|
||||
- **P2:** `` -
|
||||
meaningful correctness, test, maintainability, or edge-case issue.
|
||||
- **P3:** `` -
|
||||
minor polish or low-risk cleanup.
|
||||
- **Intent labels:**
|
||||
- **`issue`:** A confirmed defect, regression, broken contract, or concrete
|
||||
risk.
|
||||
- **`suggestion`:** A non-blocking improvement that would make the PR clearer,
|
||||
safer, or easier to maintain.
|
||||
- **`nit`:** A tiny, non-blocking cleanup or style note. Use it only when the
|
||||
author can safely ignore it without changing the review outcome.
|
||||
- **`question`:** A real author-facing clarification about intent, scope, or
|
||||
tradeoffs. Do not use questions to hide an issue that should be stated
|
||||
directly.
|
||||
- **`LGTM`:** "Looks good to me." Use only when the review found no blocking
|
||||
issues, or when any remaining notes are clearly optional.
|
||||
- **Decorations:** Optional labels in parentheses that clarify the finding type,
|
||||
scope, or merge impact.
|
||||
- **`security`:** Auth, authorization, ownership, secrets, SSRF, injection,
|
||||
unsafe external input, or other trust-boundary concerns.
|
||||
- **`test`:** Missing, failing, misleading, brittle, or insufficient tests.
|
||||
- **`scope`:** PR scope, feature boundaries, unrelated churn, or work that
|
||||
should be split into a separate issue or PR.
|
||||
- **`ci`:** CI configuration, workflow failures, flaky checks, or validation
|
||||
signal quality.
|
||||
- **`api`:** Route, request/response, public function, schema, persistence, or
|
||||
integration contract changes.
|
||||
- **`docs`:** User-facing docs, contributor docs, examples, or comments that
|
||||
need to change with the code.
|
||||
- **`non-blocking`:** Useful feedback that should not prevent merge by
|
||||
itself.
|
||||
- **Finding fields:**
|
||||
- **Problem:** What is wrong, what contract is ambiguous, or what risk the PR
|
||||
introduces.
|
||||
- **Impact:** Why the problem matters in practical terms.
|
||||
- **Ask:** The smallest concrete fix, test, or decision requested from the PR
|
||||
author.
|
||||
- **Location:** The most useful repo-relative file and line reference for the
|
||||
finding, using `path:line`.
|
||||
- **Optional sections:**
|
||||
- **Open Questions:** Genuine scope or intent questions; omit when there are
|
||||
no real questions.
|
||||
- **Validation:** What the reviewer ran, what was intentionally not run, and
|
||||
what risk remains after review.
|
||||
- **PR Hygiene:** Target-branch, template, CI/check, duplicate, related-work,
|
||||
or superseding-PR notes.
|
||||
- **`none confirmed`:** Use only when no review-worthy findings were confirmed;
|
||||
still list validation gaps or residual risk when relevant.
|
||||
@@ -24,10 +24,11 @@ Fixes #
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] I searched [open issues](https://github.com/pewdiepie-archdaemon/odysseus/issues) and [open PRs](https://github.com/pewdiepie-archdaemon/odysseus/pulls) — this is not a duplicate.
|
||||
- [ ] I searched [open issues](https://github.com/odysseus-dev/odysseus/issues) and [open PRs](https://github.com/odysseus-dev/odysseus/pulls) — this is not a duplicate.
|
||||
- [ ] This PR targets `dev`
|
||||
- [ ] My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
|
||||
- [ ] I actually ran the app (`docker compose up` or `uvicorn app:app`) and verified the change works end-to-end. Type-checks and unit tests are not enough.
|
||||
- [ ] I did not run the app/runtime validation and stated that gap in **How to Test**. Leave this unchecked when the app-run box above is checked.
|
||||
|
||||
## How to Test
|
||||
|
||||
|
||||
@@ -41,6 +41,14 @@ module.exports = async ({ github, context, core }) => {
|
||||
break;
|
||||
|
||||
case 'bug': {
|
||||
const revisionText = section('Odysseus Revision');
|
||||
if (!/^[0-9a-f]{12} \(\d{4}-\d{2}-\d{2}\)$/i.test(revisionText)) {
|
||||
failures.push(
|
||||
'**Odysseus Revision** — paste the 12-character commit SHA and date, ' +
|
||||
'for example `1fef4929cf1d (2026-08-11)`',
|
||||
);
|
||||
}
|
||||
|
||||
if (!section('Install Method')) {
|
||||
failures.push('**Install Method** — select how you installed Odysseus');
|
||||
}
|
||||
@@ -153,6 +161,16 @@ module.exports = async ({ github, context, core }) => {
|
||||
}
|
||||
}
|
||||
|
||||
const LABEL_BAD = 'needs more info';
|
||||
const LABEL_GOOD = 'ready for review';
|
||||
|
||||
// Closed issues are no longer awaiting review.
|
||||
// This also prevents later edits to closed issues from restoring the label.
|
||||
if (issue.state === 'closed') {
|
||||
await dropLabel(LABEL_GOOD);
|
||||
return;
|
||||
}
|
||||
|
||||
// ── Find existing bot comment to update in-place ──────────────────────────
|
||||
const MARKER = '<!-- issue-description-check -->';
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
@@ -160,9 +178,6 @@ module.exports = async ({ github, context, core }) => {
|
||||
});
|
||||
const existing = comments.find(c => c.user.type === 'Bot' && c.body.includes(MARKER));
|
||||
|
||||
const LABEL_BAD = 'needs more info';
|
||||
const LABEL_GOOD = 'ready for review';
|
||||
|
||||
if (failures.length === 0) {
|
||||
if (existing) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
|
||||
|
||||
@@ -21,11 +21,11 @@ module.exports = async ({ github, context, core }) => {
|
||||
return strip(m?.[0].replace(new RegExp(`#+\\s+${heading}`, 'i'), '') ?? '');
|
||||
}
|
||||
|
||||
const problems = [];
|
||||
const descriptionProblems = [];
|
||||
|
||||
// 1. Summary must be filled in.
|
||||
if (section('Summary').length < 20) {
|
||||
problems.push('**Summary** is empty or too short — describe what changed and why.');
|
||||
descriptionProblems.push('**Summary** is empty or too short — describe what changed and why.');
|
||||
}
|
||||
|
||||
// 2. Linked Issue must reference a real issue. Accept a bare #NNN, a closing
|
||||
@@ -34,18 +34,18 @@ module.exports = async ({ github, context, core }) => {
|
||||
const linkedSection = section('Linked Issue');
|
||||
const hasIssueRef = /#\d+\b/.test(linkedSection) || /\/issues\/\d+/.test(linkedSection);
|
||||
if (!linkedSection || !hasIssueRef) {
|
||||
problems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.');
|
||||
descriptionProblems.push('**Linked Issue** — add a reference like `Fixes #NNN`, a bare `#NNN`, or a link to the issue.');
|
||||
}
|
||||
|
||||
// 3. At least one Type of Change box must be checked.
|
||||
const typeBlock = body.match(/##\s+Type of Change[\s\S]*?(?=\n##\s|$)/i)?.[0] ?? '';
|
||||
if (!/- \[x\]/i.test(typeBlock)) {
|
||||
problems.push('**Type of Change** — check at least one box.');
|
||||
descriptionProblems.push('**Type of Change** — check at least one box.');
|
||||
}
|
||||
|
||||
// 4. Duplicate-search checklist item must be checked.
|
||||
if (!/- \[x\] I searched/i.test(body)) {
|
||||
problems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.');
|
||||
descriptionProblems.push('**Checklist** — check the duplicate-search box to confirm you searched existing issues and PRs.');
|
||||
}
|
||||
|
||||
// 5. How to Test must contain enough real detail for a reviewer to act on.
|
||||
@@ -53,7 +53,83 @@ module.exports = async ({ github, context, core }) => {
|
||||
// code block — so we only require non-trivial content, not a specific shape.
|
||||
const howTo = section('How to Test');
|
||||
if (howTo.length < 30) {
|
||||
problems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").');
|
||||
descriptionProblems.push('**How to Test** — explain how a reviewer can verify this change. Numbered steps, the commands you ran, or a short code block all work — give a sentence or two of real detail (not just "tested locally").');
|
||||
}
|
||||
|
||||
// Classify paths from GitHub's API. This workflow runs in the privileged base
|
||||
// context, so it must never check out or execute code from the PR branch.
|
||||
const changedFiles = await github.paginate(github.rest.pulls.listFiles, {
|
||||
owner, repo, pull_number: prNum, per_page: 100,
|
||||
});
|
||||
const changedPaths = changedFiles.map(file => file.filename);
|
||||
|
||||
function isUiSensitivePath(filename) {
|
||||
const path = filename.toLowerCase();
|
||||
return path.startsWith('static/')
|
||||
|| path.startsWith('templates/')
|
||||
|| /\.(?:html?|css|svg)$/.test(path);
|
||||
}
|
||||
|
||||
function isDocsOnlyPath(filename) {
|
||||
const path = filename.toLowerCase();
|
||||
return /\.(?:md|mdx|rst|adoc|txt)$/.test(path)
|
||||
|| (path.startsWith('docs/') && !isUiSensitivePath(path));
|
||||
}
|
||||
|
||||
function isRuntimeSensitivePath(filename) {
|
||||
const path = filename.toLowerCase();
|
||||
if (isUiSensitivePath(path)) return false;
|
||||
if (path.startsWith('tests/') || path.startsWith('.github/')) return false;
|
||||
return /^(?:app\.py|routes\/|services\/|src\/|core\/|mcp_servers\/|scripts\/|docker\/)/.test(path)
|
||||
|| /^(?:dockerfile|docker-compose.*\.ya?ml|requirements(?:-optional)?\.txt|pyproject\.toml|setup\.py)$/.test(path)
|
||||
|| /\.(?:py|sh|ps1|bat)$/.test(path);
|
||||
}
|
||||
|
||||
let classification = 'tooling';
|
||||
if (changedPaths.some(isUiSensitivePath)) {
|
||||
classification = 'UI-sensitive';
|
||||
} else if (changedPaths.some(isRuntimeSensitivePath)) {
|
||||
classification = 'backend/runtime';
|
||||
} else if (changedPaths.length > 0 && changedPaths.every(isDocsOnlyPath)) {
|
||||
classification = 'docs-only';
|
||||
}
|
||||
|
||||
const appRan = /- \[x\]\s+I actually ran the app\b/i.test(body);
|
||||
const appNotRun = /- \[x\]\s+I did not run the app\/runtime validation\b/i.test(body);
|
||||
// Anchor on the wording, not the template's emphasis: a ticked box the author
|
||||
// retyped without the surrounding ** renders identically on the PR page, so
|
||||
// treating it as unchecked is invisible from their side. Matches the two
|
||||
// attestations above, which already ignore formatting.
|
||||
const screenshotChecked = /- \[x\]\s+[*_]{0,2}Screenshot or short clip[*_]{0,2}/i.test(body);
|
||||
const screenshotSection = section('Screenshots / clips');
|
||||
const hasVisualEvidence = /!\[[^\]]*\]\([^)]+\)|<(?:img|video|source)\b[^>]*(?:src|href)=|https?:\/\/[^\s)]+/i.test(screenshotSection);
|
||||
const evidenceGaps = [];
|
||||
let needsRuntimeValidation = false;
|
||||
let needsVisualEvidence = false;
|
||||
|
||||
if (classification === 'backend/runtime' || classification === 'UI-sensitive') {
|
||||
if (appRan && appNotRun) {
|
||||
needsRuntimeValidation = true;
|
||||
evidenceGaps.push('The app-run and explicit not-run boxes are both checked. Select the one state that is true.');
|
||||
} else if (!appRan) {
|
||||
needsRuntimeValidation = true;
|
||||
if (appNotRun) {
|
||||
evidenceGaps.push('The author explicitly reports that app/runtime validation was not performed.');
|
||||
} else {
|
||||
evidenceGaps.push('App/runtime validation is not author-attested. Check the run box only after running it, or check the explicit not-run box and describe the gap.');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (classification === 'UI-sensitive') {
|
||||
if (!screenshotChecked) {
|
||||
needsVisualEvidence = true;
|
||||
evidenceGaps.push('The screenshot/clip checkbox is not checked for this UI-sensitive change.');
|
||||
}
|
||||
if (!hasVisualEvidence) {
|
||||
needsVisualEvidence = true;
|
||||
evidenceGaps.push('The Screenshots / clips section does not contain an actual attachment or link.');
|
||||
}
|
||||
}
|
||||
|
||||
// ── Comment ──────────────────────────────────────────────────────────────
|
||||
@@ -62,22 +138,43 @@ module.exports = async ({ github, context, core }) => {
|
||||
});
|
||||
const existing = comments.find(c => (c.body ?? '').includes(MARKER));
|
||||
|
||||
if (problems.length === 0) {
|
||||
if (descriptionProblems.length === 0 && evidenceGaps.length === 0) {
|
||||
if (existing) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
|
||||
}
|
||||
} else {
|
||||
const commentBody = [
|
||||
MARKER,
|
||||
'⚠️ **PR description — action needed**',
|
||||
'',
|
||||
'The following required sections are missing or incomplete. Please update the PR description to address them:',
|
||||
'',
|
||||
problems.map(p => `- ${p}`).join('\n'),
|
||||
const commentLines = [MARKER];
|
||||
if (descriptionProblems.length > 0) {
|
||||
commentLines.push(
|
||||
'⚠️ **PR description — action needed**',
|
||||
'',
|
||||
'The following required sections are missing or incomplete. Please update the PR description to address them:',
|
||||
'',
|
||||
descriptionProblems.map(problem => `- ${problem}`).join('\n'),
|
||||
);
|
||||
} else {
|
||||
commentLines.push(
|
||||
'⚠️ **PR description is complete; validation evidence is still outstanding**',
|
||||
'',
|
||||
`Changed-file classification: **${classification}**.`,
|
||||
);
|
||||
}
|
||||
if (evidenceGaps.length > 0) {
|
||||
commentLines.push(
|
||||
'',
|
||||
'**Author-reported runtime / visual state**',
|
||||
'',
|
||||
evidenceGaps.map(gap => `- ${gap}`).join('\n'),
|
||||
'',
|
||||
'Checkboxes are author attestations. GitHub Actions results remain the execution evidence for CI; this check does not prove that a local command ran.',
|
||||
);
|
||||
}
|
||||
commentLines.push(
|
||||
'',
|
||||
'---',
|
||||
'_This comment is deleted automatically once all sections are complete._',
|
||||
].join('\n');
|
||||
'_This comment updates automatically when the description or changed files change._',
|
||||
);
|
||||
const commentBody = commentLines.join('\n');
|
||||
|
||||
if (existing) {
|
||||
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body: commentBody });
|
||||
@@ -97,34 +194,47 @@ module.exports = async ({ github, context, core }) => {
|
||||
return true;
|
||||
} catch (e) {
|
||||
if (e.status === 404) return false;
|
||||
if (e.status === 403) {
|
||||
core.warning(`Could not inspect label "${name}" — token lacks label read access; skipping.`);
|
||||
return false;
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
async function swapLabel(num, add, remove) {
|
||||
if (await labelExists(add)) {
|
||||
async function setLabel(name, wanted) {
|
||||
if (wanted && await labelExists(name)) {
|
||||
try {
|
||||
await github.rest.issues.addLabels({ owner, repo, issue_number: num, labels: [add] });
|
||||
await github.rest.issues.addLabels({ owner, repo, issue_number: prNum, labels: [name] });
|
||||
} catch (e) {
|
||||
// Fail soft on a token that can't write labels so a label permission
|
||||
// problem never masks the actual description verdict.
|
||||
if (e.status !== 403) throw e;
|
||||
core.warning(`Could not add "${add}" — token lacks label write here; skipping.`);
|
||||
if (e.status !== 403 && e.status !== 404) throw e;
|
||||
core.warning(`Could not add "${name}" — label is unavailable or the token lacks label write access; skipping.`);
|
||||
}
|
||||
} else if (wanted) {
|
||||
core.warning(`Label "${name}" does not exist in the repo — skipping. Create it once to enable labelling.`);
|
||||
} else {
|
||||
core.warning(`Label "${add}" does not exist in the repo — skipping. Create it once to enable labelling.`);
|
||||
}
|
||||
try {
|
||||
await github.rest.issues.removeLabel({ owner, repo, issue_number: num, name: remove });
|
||||
} catch (e) {
|
||||
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e;
|
||||
try {
|
||||
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNum, name });
|
||||
} catch (e) {
|
||||
if (e.status !== 404 && e.status !== 410 && e.status !== 403) throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (problems.length === 0) {
|
||||
await swapLabel(prNum, 'ready for review', 'needs work');
|
||||
} else {
|
||||
await swapLabel(prNum, 'needs work', 'ready for review');
|
||||
core.setFailed(`PR description has ${problems.length} issue(s) — see bot comment for details.`);
|
||||
const descriptionComplete = descriptionProblems.length === 0;
|
||||
const evidenceComplete = evidenceGaps.length === 0;
|
||||
const isDraft = Boolean(context.payload.pull_request.draft);
|
||||
await setLabel(
|
||||
'ready for review',
|
||||
descriptionComplete && evidenceComplete && !isDraft,
|
||||
);
|
||||
await setLabel('needs work', !descriptionComplete);
|
||||
await setLabel('needs runtime validation', needsRuntimeValidation);
|
||||
await setLabel('needs visual evidence', needsVisualEvidence);
|
||||
|
||||
if (!descriptionComplete) {
|
||||
core.setFailed(`PR description has ${descriptionProblems.length} issue(s) — see bot comment for details.`);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report focused pytest guidance for changed paths under tests/."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Iterable
|
||||
from pathlib import PurePosixPath
|
||||
|
||||
|
||||
def parse_paths(raw_paths: bytes) -> list[str]:
|
||||
"""Decode the NUL-delimited output of ``git diff --name-only -z``."""
|
||||
return [os.fsdecode(path) for path in raw_paths.split(b"\0") if path]
|
||||
|
||||
|
||||
def changed_paths_from_merge_base(base_sha: str, head_sha: str) -> list[str]:
|
||||
"""Return changed ``tests/`` paths using GitHub PR three-dot semantics.
|
||||
|
||||
GitHub PR changed files are based on the merge base and the PR head, not a
|
||||
direct endpoint diff between the current base branch tip and the PR head.
|
||||
Using the direct endpoint diff can include files changed only on the base
|
||||
branch when the PR branch is stale.
|
||||
"""
|
||||
merge_base = subprocess.check_output(
|
||||
["git", "merge-base", base_sha, head_sha],
|
||||
stderr=subprocess.DEVNULL,
|
||||
).strip()
|
||||
raw_paths = subprocess.check_output(
|
||||
[
|
||||
"git",
|
||||
"diff",
|
||||
"--name-only",
|
||||
"--diff-filter=ACMRT",
|
||||
"-z",
|
||||
os.fsdecode(merge_base),
|
||||
head_sha,
|
||||
"--",
|
||||
"tests/",
|
||||
],
|
||||
)
|
||||
return parse_paths(raw_paths)
|
||||
|
||||
|
||||
def select_test_paths(paths: Iterable[str]) -> list[str]:
|
||||
"""Return unique, repository-relative paths contained by tests/."""
|
||||
selected: set[str] = set()
|
||||
for raw_path in paths:
|
||||
path = PurePosixPath(raw_path)
|
||||
if path.is_absolute() or ".." in path.parts:
|
||||
continue
|
||||
parts = tuple(part for part in path.parts if part != ".")
|
||||
if len(parts) >= 2 and parts[0] == "tests":
|
||||
selected.add(PurePosixPath(*parts).as_posix())
|
||||
return sorted(selected)
|
||||
|
||||
|
||||
def is_pytest_file(path: str) -> bool:
|
||||
"""Return whether a changed path follows this repository's pytest naming."""
|
||||
name = PurePosixPath(path).name
|
||||
return name.endswith(".py") and (
|
||||
name.startswith("test_") or name.endswith("_test.py")
|
||||
)
|
||||
|
||||
|
||||
def pytest_command(paths: Iterable[str]) -> str:
|
||||
"""Build a copyable pytest command for changed runnable test files."""
|
||||
command = ["python3", "-m", "pytest", "-q", *paths]
|
||||
return shlex.join(command)
|
||||
|
||||
|
||||
def format_report(paths: Iterable[str]) -> str:
|
||||
"""Format focused guidance for CI logs and the workflow summary."""
|
||||
changed_paths = select_test_paths(paths)
|
||||
runnable_paths = [path for path in changed_paths if is_pytest_file(path)]
|
||||
lines = ["## Focused test guidance (report-only)", ""]
|
||||
if not changed_paths:
|
||||
lines.append("No changed paths under `tests/`.")
|
||||
else:
|
||||
lines.extend(["Changed paths under `tests/`:", ""])
|
||||
lines.extend(f"- `{path}`" for path in changed_paths)
|
||||
lines.extend(["", "Suggested focused validation:", ""])
|
||||
if runnable_paths:
|
||||
lines.append(f"```sh\n{pytest_command(runnable_paths)}\n```")
|
||||
else:
|
||||
lines.append("No directly runnable pytest files changed.")
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"This guidance does not infer tests from source changes. "
|
||||
"Existing blocking CI remains the source of truth.",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _parse_args(argv: list[str]) -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Report focused pytest guidance for changed tests/ paths.",
|
||||
)
|
||||
parser.add_argument("--base-sha", help="Pull request base commit SHA.")
|
||||
parser.add_argument("--head-sha", help="Pull request head commit SHA.")
|
||||
return parser.parse_args(argv)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
args = _parse_args(sys.argv[1:] if argv is None else argv)
|
||||
if bool(args.base_sha) != bool(args.head_sha):
|
||||
raise SystemExit("--base-sha and --head-sha must be provided together")
|
||||
|
||||
if args.base_sha and args.head_sha:
|
||||
paths = changed_paths_from_merge_base(args.base_sha, args.head_sha)
|
||||
else:
|
||||
paths = parse_paths(sys.stdin.buffer.read())
|
||||
|
||||
print(format_report(paths))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -2,7 +2,7 @@ name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
branches: [main, dev]
|
||||
pull_request:
|
||||
|
||||
# Least privilege: none of the jobs write to the repo.
|
||||
@@ -15,14 +15,68 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
focused-test-guidance:
|
||||
name: Focused test guidance (report-only)
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
- name: Report changed test paths
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
report_file="$RUNNER_TEMP/focused-test-guidance.md"
|
||||
publish_report() {
|
||||
cat "$report_file"
|
||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
||||
cat "$report_file" >> "$GITHUB_STEP_SUMMARY" || true
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
report_unavailable() {
|
||||
{
|
||||
printf '%s\n\n' '## Focused test guidance unavailable (report-only)'
|
||||
printf '%s\n\n' "$1"
|
||||
printf '%s\n' 'Existing blocking CI remains the source of truth.'
|
||||
} > "$report_file"
|
||||
publish_report
|
||||
exit 0
|
||||
}
|
||||
|
||||
if [ -z "$BASE_SHA" ] || [ -z "$HEAD_SHA" ]; then
|
||||
report_unavailable "Pull request base/head metadata is missing."
|
||||
fi
|
||||
|
||||
if ! git cat-file -e "${BASE_SHA}^{commit}" 2>/dev/null; then
|
||||
report_unavailable "The pull request base commit is unavailable locally."
|
||||
fi
|
||||
|
||||
if ! git cat-file -e "${HEAD_SHA}^{commit}" 2>/dev/null; then
|
||||
report_unavailable "The pull request head commit is unavailable locally."
|
||||
fi
|
||||
|
||||
if ! python3 .github/scripts/focused_test_guidance.py \
|
||||
--base-sha "$BASE_SHA" \
|
||||
--head-sha "$HEAD_SHA" > "$report_file"; then
|
||||
report_unavailable "The focused test guidance helper could not produce a report."
|
||||
fi
|
||||
|
||||
publish_report
|
||||
|
||||
python-syntax:
|
||||
name: Python syntax (compileall)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.11"
|
||||
# Byte-compile sources — catches syntax errors without installing deps.
|
||||
@@ -32,10 +86,10 @@ jobs:
|
||||
name: JS syntax (node --check)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
with:
|
||||
node-version: "20"
|
||||
# Syntax-check our own JS (skip vendored libs in static/lib).
|
||||
@@ -49,17 +103,14 @@ jobs:
|
||||
python-tests:
|
||||
name: Python tests (pytest)
|
||||
runs-on: ubuntu-latest
|
||||
# Informational for now: the suite has known flaky / environment-dependent
|
||||
# failures (test isolation + embedding-model assertions). Tracked under the
|
||||
# ROADMAP "fresh install smoke tests" item; make this required once green.
|
||||
continue-on-error: true
|
||||
# Make Python test validation authoritative for the configured scope.
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
# Detect whether this PR only touches documentation files.
|
||||
# Detect whether this PR only touches repository prose outside the Pages site.
|
||||
# If so, skip the expensive pytest run while still reporting a passing check.
|
||||
- name: Check for docs-only changes
|
||||
id: docs-check
|
||||
@@ -71,9 +122,10 @@ jobs:
|
||||
BASE="${{ github.event.before }}"
|
||||
HEAD="${{ github.sha }}"
|
||||
fi
|
||||
# List all changed files; if every file matches docs/markdown patterns, skip pytest.
|
||||
# Keep website/ and assets/branding/ out of this bypass: pytest owns
|
||||
# regression guards for their published-file and orphan-asset contracts.
|
||||
changed=$(git diff --name-only "$BASE" "$HEAD" 2>/dev/null || git diff --name-only HEAD~1 HEAD)
|
||||
non_docs=$(echo "$changed" | grep -Ev '^(docs/|.*\.md$|\.github/[^/]+\.md$)' || true)
|
||||
non_docs=$(echo "$changed" | grep -Ev '^(docs/|[^/]+\.md$|\.github/[^/]+\.md$)' || true)
|
||||
if [ -z "$non_docs" ]; then
|
||||
echo "docs_only=true" >> "$GITHUB_OUTPUT"
|
||||
echo "Docs-only change detected — skipping pytest."
|
||||
@@ -81,7 +133,7 @@ jobs:
|
||||
echo "docs_only=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
|
||||
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
if: steps.docs-check.outputs.docs_only != 'true'
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
name: CodeQL
|
||||
|
||||
# Advanced setup so CodeQL also runs on pull requests (including from forks),
|
||||
# surfacing findings before merge instead of only after a change lands on dev.
|
||||
on:
|
||||
push:
|
||||
branches: [dev, main]
|
||||
pull_request:
|
||||
branches: [dev]
|
||||
schedule:
|
||||
- cron: "17 3 * * 1"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze (${{ matrix.language }})
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
security-events: write
|
||||
actions: read
|
||||
contents: read
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: [actions, javascript-typescript, python]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
build-mode: none
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
|
||||
with:
|
||||
category: "/language:${{ matrix.language }}"
|
||||
@@ -0,0 +1,52 @@
|
||||
# Container security: Dockerfile lint
|
||||
#
|
||||
# Purpose: the Docker image is how most people run Odysseus, so it is part of
|
||||
# the attack surface. hadolint lints the Dockerfile for mistakes and insecure
|
||||
# patterns (running as root longer than needed, unpinned base image, bad apt
|
||||
# usage). Blocking.
|
||||
#
|
||||
# The image vulnerability scan (Trivy, advisory) lives in its own file,
|
||||
# container-trivy.yml. Keeping it separate lets that advisory scan be
|
||||
# path-filtered and held to a read-only token on pull requests without
|
||||
# weakening this blocking gate, which must always report so a required check
|
||||
# never hangs.
|
||||
#
|
||||
# Note: a separate open PR (#120) proposes a local `scripts/scan_image.py`.
|
||||
# This job is complementary -- it is a CI gate, not a script a contributor has
|
||||
# to remember to run.
|
||||
|
||||
name: Container scan
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: container-scan-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
hadolint:
|
||||
name: hadolint (Dockerfile lint)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Lint Dockerfile
|
||||
uses: hadolint/hadolint-action@2a66e89f53d0771bb131a7fa31f3136336094aa6 # v3.4.0
|
||||
with:
|
||||
dockerfile: Dockerfile
|
||||
# DL3008: pinning apt package versions is impractical on a -slim base
|
||||
# image. Debian purges old package versions from its repos, so a
|
||||
# pinned version breaks future rebuilds. The base image itself is
|
||||
# what should be pinned (tracked by Dependabot's docker ecosystem).
|
||||
ignore: DL3008
|
||||
@@ -0,0 +1,129 @@
|
||||
# Container image vulnerability scan (advisory)
|
||||
#
|
||||
# Trivy builds the application image and scans it for known-vulnerable OS and
|
||||
# Python packages. Advisory only -- it reports findings to the repo's Security
|
||||
# tab without blocking a merge, because the image inevitably contains
|
||||
# already-known CVEs in upstream packages that are not this project's bug.
|
||||
#
|
||||
# Split from the Dockerfile lint (container-scan.yml) for two reasons:
|
||||
#
|
||||
# - Least privilege. The image build runs Dockerfile instructions, which on a
|
||||
# pull request are attacker-influenceable. That path (the `scan` job) is
|
||||
# held to a read-only token and never publishes results. Only `publish`,
|
||||
# which runs on push to main (curated, fast-forwarded from reviewed dev),
|
||||
# gets security-events:write to upload SARIF.
|
||||
# - Cost. Docs-only changes do not rebuild the image (paths-ignore below),
|
||||
# matching docker-publish.yml. hadolint stays on the broad trigger in
|
||||
# container-scan.yml so the blocking gate always reports.
|
||||
|
||||
name: Container scan (Trivy)
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'website/**'
|
||||
- 'assets/branding/**'
|
||||
- '.github/ISSUE_TEMPLATE/**'
|
||||
push:
|
||||
branches: [main]
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'website/**'
|
||||
- 'assets/branding/**'
|
||||
- '.github/ISSUE_TEMPLATE/**'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: container-trivy-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
# Pull requests and manual runs: build and scan under a read-only token.
|
||||
# The build executes PR-supplied Dockerfile instructions, so this job must
|
||||
# not hold any write scope, and it does not upload to the Security tab.
|
||||
scan:
|
||||
name: Trivy (image scan, advisory)
|
||||
if: github.event_name != 'push'
|
||||
runs-on: ubuntu-latest
|
||||
# Advisory: a CVE in an upstream package must not block a PR.
|
||||
continue-on-error: true
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Buildx
|
||||
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
|
||||
|
||||
# Build without pushing so a broken Dockerfile is caught here, and the
|
||||
# exact image we ship is what gets scanned.
|
||||
- name: Build image
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
push: false
|
||||
load: true
|
||||
tags: odysseus:ci
|
||||
|
||||
- name: Scan image with Trivy
|
||||
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
|
||||
with:
|
||||
image-ref: odysseus:ci
|
||||
format: table
|
||||
ignore-unfixed: true
|
||||
env:
|
||||
# Pin the vuln DB source to GHCR to avoid rate-limited Docker Hub
|
||||
# mirrors that flake on shared runners.
|
||||
TRIVY_DB_REPOSITORY: ghcr.io/aquasecurity/trivy-db:2
|
||||
|
||||
# Push to main only: build, scan, and publish SARIF to the Security tab.
|
||||
# This is the only path that runs trusted code, so it is the only one granted
|
||||
# security-events:write.
|
||||
publish:
|
||||
name: Trivy (image scan + SARIF upload)
|
||||
if: github.event_name == 'push'
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write # upload SARIF to the Security tab
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Buildx
|
||||
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
|
||||
|
||||
- name: Build image
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
push: false
|
||||
load: true
|
||||
tags: odysseus:ci
|
||||
|
||||
- name: Scan image with Trivy
|
||||
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
|
||||
with:
|
||||
image-ref: odysseus:ci
|
||||
format: sarif
|
||||
output: trivy-results.sarif
|
||||
ignore-unfixed: true
|
||||
env:
|
||||
TRIVY_DB_REPOSITORY: ghcr.io/aquasecurity/trivy-db:2
|
||||
|
||||
- name: Upload Trivy results
|
||||
uses: github/codeql-action/upload-sarif@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4.37.7
|
||||
with:
|
||||
sarif_file: trivy-results.sarif
|
||||
category: trivy-image
|
||||
@@ -0,0 +1,71 @@
|
||||
# Supply-chain review
|
||||
#
|
||||
# Purpose: defend against "side-chain" / supply-chain attacks -- a pull request
|
||||
# that adds (or bumps) a dependency to a version with a known vulnerability or a
|
||||
# disallowed license. Two layers:
|
||||
#
|
||||
# - dependency-review: runs ONLY on pull requests. It compares the
|
||||
# dependencies before and after the PR and blocks the merge if the change
|
||||
# pulls in a package with a known security advisory. This is the gate.
|
||||
# - pip-audit: scans the project's current Python requirements against the
|
||||
# advisory database. Advisory only (it never blocks a merge), because it can
|
||||
# flag a pre-existing issue in an already-shipped dependency.
|
||||
|
||||
name: Dependency review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
# Default-deny token; jobs grant only read access.
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: dependency-review-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
dependency-review:
|
||||
name: dependency-review (PR gate)
|
||||
# Only meaningful on a pull request -- it needs a base..head diff to review.
|
||||
if: github.event_name == 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Review dependency changes
|
||||
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
|
||||
with:
|
||||
# Fail the PR on any newly introduced moderate-or-worse advisory.
|
||||
fail-on-severity: moderate
|
||||
|
||||
pip-audit:
|
||||
name: pip-audit (advisory)
|
||||
runs-on: ubuntu-latest
|
||||
# Advisory: report known-vulnerable Python deps without blocking the merge.
|
||||
continue-on-error: true
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Run pip-audit on requirements
|
||||
run: |
|
||||
set -euo pipefail
|
||||
pip install pip-audit==2.10.0
|
||||
pip-audit -r requirements.txt -r requirements-optional.txt --strict
|
||||
@@ -0,0 +1,50 @@
|
||||
name: Deploy GitHub Pages
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'website/**'
|
||||
- '.github/workflows/deploy-pages.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Package static site
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pages: read
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
|
||||
- uses: actions/jekyll-build-pages@44a6e6beabd48582f863aeeb6cb2151cc1716697 # v1.0.13
|
||||
with:
|
||||
source: website
|
||||
destination: _site
|
||||
- uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
|
||||
with:
|
||||
path: _site
|
||||
|
||||
deploy:
|
||||
name: Deploy static site
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pages: write
|
||||
id-token: write
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
|
||||
@@ -14,6 +14,8 @@ on:
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'website/**'
|
||||
- 'assets/branding/**'
|
||||
- '.github/ISSUE_TEMPLATE/**'
|
||||
|
||||
concurrency:
|
||||
@@ -45,20 +47,20 @@ jobs:
|
||||
arch: arm64
|
||||
runner: ubuntu-24.04-arm
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Set up Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
|
||||
- name: Log in to GHCR
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Build and push by digest
|
||||
id: build
|
||||
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
platforms: ${{ matrix.platform }}
|
||||
@@ -86,7 +88,7 @@ jobs:
|
||||
contents: read
|
||||
packages: write
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Read APP_VERSION + short sha
|
||||
@@ -103,16 +105,16 @@ jobs:
|
||||
pattern: digest-*
|
||||
merge-multiple: true
|
||||
- name: Set up Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
|
||||
- name: Log in to GHCR
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
- name: Compute tags
|
||||
id: meta
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
|
||||
@@ -2,7 +2,7 @@ name: ci / issue description check
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened, edited, reopened]
|
||||
types: [opened, edited, reopened, closed]
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
# Skip bots (Dependabot, release-drafter, etc.)
|
||||
if: ${{ github.event.issue.user.type != 'Bot' }}
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
sparse-checkout: .github/scripts
|
||||
persist-credentials: false
|
||||
|
||||
@@ -5,7 +5,11 @@ on:
|
||||
# works on fork PRs. Safe here: the checkout pins to the base branch (no fork
|
||||
# code runs) and the scripts only read context.payload and call the GitHub API.
|
||||
pull_request_target: # zizmor: ignore[dangerous-triggers]
|
||||
types: [opened, edited, synchronize, reopened, ready_for_review]
|
||||
types: [opened, edited, synchronize, reopened, ready_for_review, converted_to_draft]
|
||||
|
||||
concurrency:
|
||||
group: pr-description-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
# Default-deny at the workflow level; each job opts into only the scopes it needs.
|
||||
# Note: modifying a PR's labels/comments needs pull-requests:write even though the
|
||||
@@ -23,7 +27,7 @@ jobs:
|
||||
# Skip bots: they open PRs programmatically and have their own process.
|
||||
if: github.event.pull_request.user.type != 'Bot'
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
ref: ${{ github.base_ref }}
|
||||
sparse-checkout: .github/scripts
|
||||
@@ -59,12 +63,14 @@ jobs:
|
||||
|
||||
check-mergeable:
|
||||
name: Flag unmergeable PRs
|
||||
needs: check-description
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pull-requests: write
|
||||
issues: write
|
||||
# Skip bots: they open PRs programmatically and have their own process.
|
||||
if: github.event.pull_request.user.type != 'Bot'
|
||||
# Run after description validation failures, but never from an obsolete
|
||||
# workflow run canceled by a newer PR event.
|
||||
if: ${{ !cancelled() && github.event.pull_request.user.type != 'Bot' }}
|
||||
steps:
|
||||
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# Secret scanning
|
||||
#
|
||||
# Purpose: stop credentials (API keys, tokens, passwords, private keys) from
|
||||
# ever living in the Git history. Odysseus deliberately keeps real secrets in
|
||||
# files that are gitignored (.env, data/), but a slip in a future commit -- or a
|
||||
# malicious pull request that sneaks one in -- would otherwise go unnoticed.
|
||||
# This job reads the repository and the full commit history and fails if it
|
||||
# finds anything that looks like a secret.
|
||||
#
|
||||
# It runs the official gitleaks BINARY directly (pinned to an exact version and
|
||||
# verified against the project's published SHA-256 checksum) rather than the
|
||||
# gitleaks GitHub Action, because the Action asks for a paid license on
|
||||
# organization-owned repos. The binary is free and behaves identically.
|
||||
|
||||
name: Secret scan
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
# Start with zero permissions; the single job opts back in to read-only.
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: secret-scan-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
gitleaks:
|
||||
name: gitleaks
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
# Full history so a secret committed in an earlier commit (and later
|
||||
# deleted) is still caught -- deletion does not remove it from Git.
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
# Pinned version + checksum so a tampered release binary cannot run here.
|
||||
# Bump VERSION/SHA256 together; the checksum comes from the matching
|
||||
# gitleaks_<version>_checksums.txt on the GitHub release.
|
||||
- name: Run gitleaks (pinned, checksum-verified)
|
||||
env:
|
||||
GITLEAKS_VERSION: 8.30.1
|
||||
GITLEAKS_SHA256: 551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TARBALL="gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
|
||||
curl -fsSL -o "${TARBALL}" \
|
||||
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/${TARBALL}"
|
||||
echo "${GITLEAKS_SHA256} ${TARBALL}" | sha256sum -c -
|
||||
tar -xzf "${TARBALL}" gitleaks
|
||||
# Scan the whole history. Findings print to the log and fail the job.
|
||||
./gitleaks git --no-banner --redact --verbose .
|
||||
@@ -0,0 +1,80 @@
|
||||
# Workflow security (CI that audits the CI)
|
||||
#
|
||||
# Purpose: the GitHub Actions workflows themselves are an attack surface. A
|
||||
# poorly written workflow can leak the repository token, run attacker-supplied
|
||||
# code from a pull request, or pull in a tampered third-party action. These two
|
||||
# tools check every workflow file in this repo for those mistakes:
|
||||
#
|
||||
# - actionlint: catches workflow syntax errors and shell-script bugs inside
|
||||
# `run:` steps before they reach main.
|
||||
# - zizmor: a security linter for Actions. Flags template-injection holes,
|
||||
# unpinned actions, credential persistence, and over-broad token
|
||||
# permissions -- exactly the patterns the rest of this CI is built to avoid.
|
||||
#
|
||||
# Add this early: it then audits every workflow added after it.
|
||||
|
||||
name: Workflow security
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
# Default-deny token; each job grants only read access to the code.
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: workflow-security-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
actionlint:
|
||||
name: actionlint
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
# Pinned version + checksum so a tampered binary cannot run here.
|
||||
- name: Run actionlint (pinned, checksum-verified)
|
||||
env:
|
||||
ACTIONLINT_VERSION: 1.7.12
|
||||
ACTIONLINT_SHA256: 8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TARBALL="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
|
||||
curl -fsSL -o "${TARBALL}" \
|
||||
"https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/${TARBALL}"
|
||||
echo "${ACTIONLINT_SHA256} ${TARBALL}" | sha256sum -c -
|
||||
tar -xzf "${TARBALL}" actionlint
|
||||
./actionlint -color
|
||||
|
||||
zizmor:
|
||||
name: zizmor (Actions SAST)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
# Pinned zizmor release. --offline keeps the audit hermetic (no network
|
||||
# calls about the actions it inspects); --min-severity=low surfaces
|
||||
# everything so nothing slips through under the gate.
|
||||
- name: Run zizmor
|
||||
run: |
|
||||
set -euo pipefail
|
||||
pip install zizmor==1.25.2
|
||||
zizmor --offline --min-severity=low .github/workflows/
|
||||
@@ -14,6 +14,15 @@ venv/
|
||||
.env
|
||||
.env.bak.*
|
||||
!.env.example
|
||||
# Local uv lockfile (optional, per-platform — see "Faster installs with uv" in README)
|
||||
requirements.lock
|
||||
|
||||
# SOPS workflow — encrypted `secrets.env` is intentionally committable,
|
||||
# but every variant (plaintext, manual decrypt copy, editor backup)
|
||||
# must stay out of git. Mirrored in .dockerignore so the same artifacts
|
||||
# also cannot enter image build layers.
|
||||
secrets.env.*
|
||||
!secrets.env.example
|
||||
|
||||
# Data — all user data stays local
|
||||
data/
|
||||
@@ -61,6 +70,9 @@ output.txt.txt
|
||||
*.tiff
|
||||
*.pdf
|
||||
|
||||
# …except shipped static assets
|
||||
!static/icons/*.png
|
||||
|
||||
# …except shipped demo assets in docs/ that the README links to.
|
||||
!docs/*.jpg
|
||||
!docs/*.jpeg
|
||||
@@ -73,6 +85,24 @@ output.txt.txt
|
||||
!docs/**/*.gif
|
||||
!docs/**/*.webp
|
||||
|
||||
# …except shipped website and branding media.
|
||||
!website/**/*.jpg
|
||||
!website/**/*.jpeg
|
||||
!website/**/*.png
|
||||
!website/**/*.gif
|
||||
!website/**/*.bmp
|
||||
!website/**/*.webp
|
||||
!website/**/*.tiff
|
||||
!website/**/*.pdf
|
||||
!assets/branding/**/*.jpg
|
||||
!assets/branding/**/*.jpeg
|
||||
!assets/branding/**/*.png
|
||||
!assets/branding/**/*.gif
|
||||
!assets/branding/**/*.bmp
|
||||
!assets/branding/**/*.webp
|
||||
!assets/branding/**/*.tiff
|
||||
!assets/branding/**/*.pdf
|
||||
|
||||
# Reports and temp files
|
||||
reports/
|
||||
tasks/
|
||||
@@ -89,3 +119,4 @@ docs/windows-port/
|
||||
compound.config.json
|
||||
*.error.log
|
||||
_scratch/
|
||||
/odysseus/
|
||||
|
||||
@@ -65,6 +65,16 @@ Vendored in `static/lib/` and served directly:
|
||||
| [jsPDF](https://github.com/parallax/jsPDF) (bundled in html2pdf) | PDF generation | MIT |
|
||||
| [html2canvas](https://github.com/niklasvh/html2canvas) (bundled in html2pdf) | DOM → canvas rasterization | MIT |
|
||||
| [node-qrcode](https://github.com/soldair/node-qrcode) (`qrcode.min.js`) | QR-code rendering (2FA setup) | MIT |
|
||||
| [KaTeX](https://github.com/KaTeX/KaTeX) v0.16.22 (`katex/katex.min.{js,css}` + `katex/fonts/*.woff2`) | Math typesetting | MIT ([`licenses/KaTeX-MIT-LICENSE.txt`](licenses/KaTeX-MIT-LICENSE.txt)) |
|
||||
| [Mermaid](https://github.com/mermaid-js/mermaid) v11.16.1 (`mermaid.min.js`) | Diagrams from text | MIT ([`licenses/Mermaid-MIT-LICENSE.txt`](licenses/Mermaid-MIT-LICENSE.txt)) |
|
||||
|
||||
KaTeX and Mermaid are loaded on first use by `static/js/markdown.js` rather than
|
||||
from `index.html`, so a session that renders no math and no diagram never fetches
|
||||
either. Only the `.woff2` KaTeX fonts are shipped, matching `static/fonts/`; the
|
||||
`.woff` and `.ttf` variants its stylesheet also lists are never requested by a
|
||||
browser that supports `woff2`. The bundles are the published npm artifacts,
|
||||
unmodified — `.gitattributes` turns the whitespace check off for `static/lib/`
|
||||
so they can stay byte-identical to upstream.
|
||||
|
||||
## Front-end libraries loaded at runtime (CDN)
|
||||
|
||||
@@ -72,8 +82,6 @@ Referenced from `cdn.jsdelivr.net` / `cdnjs.cloudflare.com` at runtime — not v
|
||||
|
||||
| Library | Purpose | License |
|
||||
|---|---|---|
|
||||
| [KaTeX](https://github.com/KaTeX/KaTeX) 0.16.22 | Math typesetting | MIT |
|
||||
| [Mermaid](https://github.com/mermaid-js/mermaid) 11 | Diagrams from text | MIT |
|
||||
| [Pyodide](https://github.com/pyodide/pyodide) 0.27.5 | In-browser Python runtime | MPL-2.0 |
|
||||
| [PDFObject](https://github.com/pipwerks/PDFObject) 2.1.1 | Inline PDF embedding | MIT |
|
||||
|
||||
@@ -86,6 +94,7 @@ Bundled in `static/fonts/`:
|
||||
| [Fira Code](https://github.com/tonsky/FiraCode) | SIL Open Font License 1.1 | Nikita Prokopov & contributors |
|
||||
| [Inter](https://github.com/rsms/inter) | SIL Open Font License 1.1 | Rasmus Andersson |
|
||||
| [GohuFont](https://font.gohu.org/) (`fonts/custom/GohuFont.ttf`) | WTFPL | Hugo Chargois |
|
||||
| [OpenDyslexic](https://opendyslexic.org/) (`fonts/OpenDyslexic-{Regular,Bold}.woff2`) | SIL Open Font License 1.1 ([`licenses/OpenDyslexic-OFL.txt`](licenses/OpenDyslexic-OFL.txt)) | Abbie Gonzalez |
|
||||
|
||||
## Python dependencies
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ End-users cloning the repo will land on `dev` by default. To run the curated/sta
|
||||
Docker is the recommended path for normal testing:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
git clone https://github.com/odysseus-dev/odysseus.git
|
||||
cd odysseus
|
||||
cp .env.example .env
|
||||
docker compose up -d --build
|
||||
@@ -37,7 +37,7 @@ Manual development uses Python 3.11+:
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python -m uvicorn app:app --host 0.0.0.0 --port 7000
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 7000
|
||||
```
|
||||
|
||||
Windows is not actively tested. Docker on Linux or a Linux/macOS manual install is the safer path for now.
|
||||
|
||||
@@ -1,11 +1,27 @@
|
||||
FROM python:3.12-slim
|
||||
# ---- builder: patch + build wheels for Real-ESRGAN's broken-on-3.14 deps ----
|
||||
# basicsr/gfpgan/facexlib read their version via exec()+locals()['__version__'],
|
||||
# which raises KeyError on Python 3.13+ (PEP 667). Build patched wheels here so
|
||||
# the final image / Cookbook never has to compile the broken sdists. See
|
||||
# docker/build-realesrgan-wheels.sh for the full rationale.
|
||||
FROM python:3.14-slim AS realesrgan-wheels
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY docker/build-realesrgan-wheels.sh /usr/local/bin/build-realesrgan-wheels.sh
|
||||
RUN bash /usr/local/bin/build-realesrgan-wheels.sh /wheels
|
||||
|
||||
FROM python:3.14-slim
|
||||
|
||||
# System deps. tmux is required by Cookbook for background downloads/serves.
|
||||
# openssh-client is required for Cookbook remote server tests, setup, probes,
|
||||
# downloads, and serves from Docker installs.
|
||||
# git/cmake are required when Cookbook builds llama.cpp on first llama.cpp
|
||||
# launch inside Docker.
|
||||
# nodejs/npm provide npx for the optional built-in Browser MCP server.
|
||||
# nodejs/npm provide npx for the built-in Browser MCP server.
|
||||
# chromium provides the actual browser binary used by that MCP server.
|
||||
# fontconfig + Noto CJK provide real fallback glyphs for multilingual pages;
|
||||
# Chromium otherwise renders Chinese/Japanese/Korean labels as empty boxes.
|
||||
# iproute2/iputils-ping/net-tools/dnsutils/nmap give Docker-hosted agents the
|
||||
# basic network inspection toolkit expected by local LAN/debugging tasks.
|
||||
# gosu lets the entrypoint drop privileges cleanly so signals still reach
|
||||
# uvicorn directly (no extra shell layer like `su`/`sudo` would add).
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
@@ -15,11 +31,60 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
git \
|
||||
nodejs \
|
||||
npm \
|
||||
chromium \
|
||||
fontconfig \
|
||||
fonts-noto-cjk \
|
||||
tmux \
|
||||
openssh-client \
|
||||
iproute2 \
|
||||
iputils-ping \
|
||||
net-tools \
|
||||
dnsutils \
|
||||
nmap \
|
||||
gosu \
|
||||
libgl1 \
|
||||
libglib2.0-0t64 \
|
||||
libxcb1 \
|
||||
libmagic1 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Private browser automation wrapper used by the native `private_browser` tool.
|
||||
# Chromium is installed above, so agent-browser can drive the existing browser
|
||||
# binary without paying `npx` startup/install overhead on each tool call.
|
||||
RUN npm install -g agent-browser@0.35.0 --omit=dev --loglevel=error
|
||||
|
||||
# libgl1/libglib2.0-0t64/libxcb1 are runtime shared libs (libGL.so.1,
|
||||
# libglib-2.0/libgthread, libxcb.so.1) that opencv-python (cv2) loads. The
|
||||
# slim base omits them, so the Cookbook "install realesrgan" path imports cv2
|
||||
# and dies with `libxcb.so.1: cannot open shared object file` despite a clean
|
||||
# pip install. Using full opencv-python (not -headless) because basicsr/gfpgan/
|
||||
# facexlib/realesrgan all depend on the `opencv-python` distribution by name.
|
||||
#
|
||||
# libmagic1 is the shared lib (libmagic.so.1) that python-magic dlopens for
|
||||
# content-based MIME sniffing in src/upload_handler.py. We install both here
|
||||
# (libmagic1 + the python-magic wrapper, below) rather than in requirements.txt
|
||||
# because python-magic resolves libmagic at import time: where the lib is
|
||||
# absent the import can block or raise, so keeping it image-only avoids
|
||||
# regressing pip/venv installs on hosts without libmagic. Debian always has the
|
||||
# lib here, so the import is instant and detection actually works.
|
||||
|
||||
# Docker CLI (client only — daemon stays on the host via the
|
||||
# /var/run/docker.sock mount). The Debian `docker.io` package ships
|
||||
# dockerd but not the client binary on slim, so grab the static client
|
||||
# tarball from download.docker.com instead.
|
||||
ARG DOCKER_CLI_VERSION=29.6.2
|
||||
RUN ARCH="$(dpkg --print-architecture)" \
|
||||
&& case "$ARCH" in \
|
||||
amd64) DARCH=x86_64 ;; \
|
||||
arm64) DARCH=aarch64 ;; \
|
||||
*) echo "unsupported arch $ARCH"; exit 1 ;; \
|
||||
esac \
|
||||
&& curl -fsSL "https://download.docker.com/linux/static/stable/${DARCH}/docker-${DOCKER_CLI_VERSION}.tgz" \
|
||||
-o /tmp/docker.tgz \
|
||||
&& tar -xzf /tmp/docker.tgz -C /tmp \
|
||||
&& install -m 0755 /tmp/docker/docker /usr/local/bin/docker \
|
||||
&& rm -rf /tmp/docker /tmp/docker.tgz
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install Python deps first (layer cache). Optional extras (PyMuPDF AGPL, etc.)
|
||||
@@ -29,6 +94,20 @@ COPY requirements.txt requirements-optional.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt \
|
||||
&& if [ "$INSTALL_OPTIONAL" = "true" ]; then pip install --no-cache-dir -r requirements-optional.txt; fi
|
||||
|
||||
# python-magic powers content-based MIME sniffing in src/upload_handler.py.
|
||||
# Image-only (not in requirements.txt) because it needs the libmagic1 system
|
||||
# lib installed above; see the apt note near the top of this stage.
|
||||
RUN pip install --no-cache-dir python-magic==0.4.27
|
||||
|
||||
# Pre-install the patched basicsr/gfpgan/facexlib wheels built in the
|
||||
# realesrgan-wheels stage (--no-deps keeps the image lean — torch & friends are
|
||||
# pulled only when realesrgan is actually installed). With these dists already
|
||||
# satisfied, the Cookbook's plain `pip install realesrgan` resolves them from
|
||||
# wheels instead of rebuilding the sdists that fail on Python 3.14.
|
||||
COPY --from=realesrgan-wheels /wheels/ /tmp/odysseus-wheels/
|
||||
RUN pip install --no-cache-dir --no-deps /tmp/odysseus-wheels/*.whl \
|
||||
&& rm -rf /tmp/odysseus-wheels
|
||||
|
||||
# Copy app code
|
||||
COPY . .
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
0.20.5
|
||||
@@ -1,21 +1,235 @@
|
||||
MIT License
|
||||
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||
Version 3, 19 November 2007
|
||||
|
||||
Copyright (c) 2025 Odysseus Contributors
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
Preamble
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software.
|
||||
|
||||
The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.
|
||||
|
||||
Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software.
|
||||
|
||||
A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public.
|
||||
|
||||
The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version.
|
||||
|
||||
An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license.
|
||||
|
||||
The precise terms and conditions for copying, distribution and modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
"Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.
|
||||
|
||||
Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Affero General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# -*- mode: python ; coding: utf-8 -*-
|
||||
|
||||
|
||||
a = Analysis(
|
||||
['launcher.py'],
|
||||
pathex=[],
|
||||
binaries=[],
|
||||
datas=[('static', 'static'), ('scripts', 'scripts'), ('mcp_servers', 'mcp_servers'), ('services/hwfit/data', 'services/hwfit/data'), ('config', 'config'), ('.env.example', '.env.example')],
|
||||
hiddenimports=[],
|
||||
hookspath=[],
|
||||
hooksconfig={},
|
||||
runtime_hooks=[],
|
||||
excludes=[],
|
||||
noarchive=False,
|
||||
optimize=0,
|
||||
)
|
||||
pyz = PYZ(a.pure)
|
||||
|
||||
exe = EXE(
|
||||
pyz,
|
||||
a.scripts,
|
||||
[],
|
||||
exclude_binaries=True,
|
||||
name='Odysseus',
|
||||
debug=False,
|
||||
bootloader_ignore_signals=False,
|
||||
strip=False,
|
||||
upx=True,
|
||||
console=False,
|
||||
disable_windowed_traceback=False,
|
||||
argv_emulation=False,
|
||||
target_arch=None,
|
||||
codesign_identity=None,
|
||||
entitlements_file=None,
|
||||
icon=['static\\icon.ico'],
|
||||
)
|
||||
coll = COLLECT(
|
||||
exe,
|
||||
a.binaries,
|
||||
a.datas,
|
||||
strip=False,
|
||||
upx=True,
|
||||
upx_exclude=[],
|
||||
name='Odysseus',
|
||||
)
|
||||
@@ -1,469 +1,81 @@
|
||||
# Odysseus
|
||||
<p align="center">
|
||||
<img src="assets/branding/odysseus-wordmark.png" alt="Odysseus" width="238">
|
||||
</p>
|
||||
|
||||
> **Branch note:** `dev` is the default branch and contains the latest development changes, but it may be unstable. For the more stable curated branch, use [`main`](https://github.com/pewdiepie-archdaemon/odysseus/tree/main).
|
||||
<p align="center">
|
||||
A self-hosted AI workspace for chat, agents, research, documents, email, notes, calendar, and local model workflows.
|
||||
</p>
|
||||
|
||||
```
|
||||
───────────────────────────────────────────────
|
||||
⊹ ࣪ ˖ ૮( ˶ᵔ ᵕ ᵔ˶ )っ Odysseus vers. 1.0
|
||||
───────────────────────────────────────────────
|
||||
```
|
||||
<p align="center">
|
||||
<a href="#quick-start">Quick Start</a> ·
|
||||
<a href="website/setup.md">Setup Guide</a> ·
|
||||
<a href="CONTRIBUTING.md">Contributing</a> ·
|
||||
<a href="ROADMAP.md">Roadmap</a>
|
||||
</p>
|
||||
|
||||

|
||||
<p align="center">
|
||||
<a href="https://repology.org/project/odysseus-ai/versions"><img src="https://repology.org/badge/vertical-allrepos/odysseus-ai.svg" alt="Packaging status"></a>
|
||||
</p>
|
||||
|
||||
A self-hosted AI workspace -- meant to be the self-hosted version of the UI experience you get from ChatGPT and Claude. But with more jank and fun. Running on your own hardware, with your own data -- local-first, privacy-first, and no trojan.
|
||||
<p align="center">
|
||||
<img src="assets/branding/odysseus-browser.jpg" alt="Odysseus interface">
|
||||
</p>
|
||||
|
||||
## Features
|
||||
- **Chat** -- chat with any local model or API; adding them is super simple.<br> <sub>vLLM · llama.cpp · Ollama · OpenRouter · OpenAI · GitHub Copilot</sub>
|
||||
- **Agent** -- hand it tools and let it run the whole task itself.<br> <sub>built on [opencode](https://github.com/anomalyco/opencode) · MCP · web · files · shell · skills · memory</sub>
|
||||
- **Cookbook** -- Scans your hardware, recommends models, click to download and serve.. easy!<br> <sub>built on [llmfit](https://github.com/AlexsJones/llmfit) · VRAM-aware · GGUF / FP8 / AWQ · fit scoring · vLLM / llama.cpp serving</sub>
|
||||
- **Deep Research** -- multi-step runs that gather, read, and synthesize sources into a nice visual report.<br> <sub>adapted from [Tongyi DeepResearch](https://github.com/Alibaba-NLP/DeepResearch)</sub>
|
||||
- **Compare** -- a fun tool to compare models side by side. Test completely blind, no bias!<br> <sub>multi-model · blind test · synthesis</sub>
|
||||
- **Documents** -- YOU write the text, AI is there to assist, not the opposite.<br> <sub>multi-tab editor · markdown · HTML · CSV · syntax highlighting · AI edits · suggestions</sub>
|
||||
- **Memory / Skills** -- Persistent memory and skills, your agent evolves over time as it better understands you and your tasks!<br> <sub>ChromaDB · fastembed (ONNX) · vector + keyword retrieval · import/export</sub>
|
||||
- **Email** -- IMAP/SMTP inbox with AI triage built in: urgency reminders, auto-tag, auto-summary, auto-reply drafts, auto-spam.<br> <sub>IMAP · SMTP · per-account routing · CalDAV-aware</sub>
|
||||
- **Notes & Tasks** -- Quick notes with reminders, a todo list, and scheduled tasks the agent can act on.<br> <sub>note pings · checklist · cron-style tasks · ntfy / browser / email channels</sub>
|
||||
- **Calendar** -- Local-first calendar with CalDAV sync to Radicale / Nextcloud / Apple / Fastmail.<br> <sub>CalDAV pull · .ics import/export · per-calendar colors · agent-aware</sub>
|
||||
- **Works on mobile** -- looks and runs great on your phone, not just desktop.<br> <sub>responsive · installable (PWA) · touch gestures</sub>
|
||||
- **Extras** -- more to explore, happy if you give it a go!<br> <sub>image editor · theme editor · file uploads (vision + PDF) · web search · presets · sessions · 2FA</sub>
|
||||
|
||||
## Demo
|
||||
A full, hover-to-play tour lives on the landing page (`docs/index.html`).
|
||||
|
||||
<details>
|
||||
<summary>Screenshots / clips</summary>
|
||||
|
||||
### Chat & Agents
|
||||

|
||||
### Deep Research
|
||||

|
||||
### Compare
|
||||

|
||||
### Documents
|
||||

|
||||
### Notes & Tasks
|
||||

|
||||
|
||||
</details>
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
Defaults work out of the box: clone, run, then configure models/search/email
|
||||
inside **Settings**. Only edit `.env` for deployment-level overrides like
|
||||
`APP_BIND`, `APP_PORT`, `AUTH_ENABLED`, `DATABASE_URL`, or a pre-seeded admin password.
|
||||
> `dev` is the default branch and gets the newest changes first. Use [`main`](https://github.com/odysseus-dev/odysseus/tree/main) if you want the more curated branch.
|
||||
|
||||
On first setup, Odysseus creates an admin account (`admin` unless
|
||||
`ODYSSEUS_ADMIN_USER` is set) and prints a temporary password in the terminal.
|
||||
For Docker installs, the same line is in `docker compose logs odysseus`.
|
||||
Use that for the first login, then change it in **Settings**.
|
||||
|
||||
Contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, testing, and
|
||||
pull request guidelines.
|
||||
|
||||
### Docker (recommended)
|
||||
```bash
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
git clone https://github.com/odysseus-dev/odysseus.git
|
||||
cd odysseus
|
||||
cp .env.example .env # optional, but recommended for explicit defaults
|
||||
cp .env.example .env
|
||||
docker compose up -d --build
|
||||
```
|
||||
To include optional extras in the image (PDF viewer, Office extraction; includes AGPL PyMuPDF), build with `docker compose build --build-arg INSTALL_OPTIONAL=true` before `up`.
|
||||
|
||||
Open `http://localhost:7000` when the containers are healthy. Docker Compose
|
||||
binds the web UI to `127.0.0.1` by default. If the port is taken, set
|
||||
`APP_PORT=7001` in `.env` and recreate the container. Set `APP_BIND=0.0.0.0`
|
||||
only when you intentionally want LAN/reverse-proxy access.
|
||||
Open `http://localhost:7000` when the containers are healthy. The first admin password is printed in `docker compose logs odysseus`.
|
||||
|
||||
### Native Linux / macOS
|
||||
```bash
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
cd odysseus
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python setup.py
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 7000
|
||||
```
|
||||
Requirements: Python 3.11+. Cookbook also needs `tmux` for background model
|
||||
downloads and serves. The app itself is lightweight; local model serving is the
|
||||
heavy part and depends on the model, runtime, GPU, and VRAM, so small hosts can
|
||||
connect to API or remote model servers instead. Use `--host 0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
|
||||
Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration live in the [setup guide](website/setup.md).
|
||||
|
||||
### Apple Silicon
|
||||
Docker on macOS cannot use the Metal GPU. For GPU-accelerated Cookbook on an
|
||||
M-series Mac, run Odysseus natively:
|
||||
## Features
|
||||
|
||||
```bash
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
cd odysseus
|
||||
./start-macos.sh
|
||||
```
|
||||
- **Chat + Agents** — local/API models, tools, MCP, files, shell, skills, and memory.
|
||||
- **Cookbook** — hardware-aware model recommendations, downloads, and serving.
|
||||
- **Deep Research** — multi-step web research with source reading and report generation.
|
||||
- **Compare** — blind side-by-side model testing and synthesis.
|
||||
- **Documents** — writing-first editor with AI edits, suggestions, Markdown, HTML, CSV, and syntax highlighting.
|
||||
- **Email** — IMAP/SMTP inbox with triage, tags, summaries, reminders, and reply drafts.
|
||||
- **Notes, Tasks + Calendar** — reminders, todos, scheduled agent tasks, and CalDAV sync.
|
||||
- **Extras** — gallery/image editor, themes, uploads, web search, presets, sessions, and 2FA.
|
||||
|
||||
It launches at `http://127.0.0.1:7860`. To expose it to your phone over a trusted LAN/VPN such as Tailscale, bind all interfaces:
|
||||
## Demo
|
||||
|
||||
```bash
|
||||
ODYSSEUS_HOST=0.0.0.0 ./start-macos.sh
|
||||
# then open http://<tailscale-ip>:7860
|
||||
```
|
||||
A full hover-to-play tour lives on the [Odysseus landing page](https://odysseus-dev.github.io/odysseus/). Its source lives under [`website/`](website/).
|
||||
|
||||
The script also reads `.env` at startup, so `APP_BIND=0.0.0.0` and `APP_PORT`
|
||||
set there are picked up automatically without a command-line override each run.
|
||||
## Contributing
|
||||
|
||||
Keep `AUTH_ENABLED=true` (the default) before binding outside loopback. Do not
|
||||
expose this port directly to the public internet. To build a clickable app wrapper:
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, docs, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
```bash
|
||||
./build-macos-app.sh
|
||||
```
|
||||
## Security
|
||||
|
||||
<details>
|
||||
<summary>Cookbook, GPU, Ollama, and troubleshooting notes</summary>
|
||||
|
||||
**Docker bundled services.** Compose starts Odysseus, ChromaDB, SearXNG, and
|
||||
ntfy. Odysseus and the bundled service ports bind to `127.0.0.1` by default, so
|
||||
they are reachable from the host but not exposed to your LAN/public internet
|
||||
unless you opt in.
|
||||
|
||||
**Cookbook storage in Docker.** Downloads live in `./data/huggingface`
|
||||
(`~/.cache/huggingface` in the container). Cookbook-installed Python CLIs and
|
||||
serve engines live in `./data/local` (`~/.local` in the container), so they
|
||||
survive container recreation.
|
||||
|
||||
**Remote servers.** In **Cookbook -> Settings -> Servers**, generate the
|
||||
Odysseus SSH key and add the public key to the remote server's
|
||||
`~/.ssh/authorized_keys`. From the host you can also run:
|
||||
|
||||
```bash
|
||||
ssh-copy-id -i data/ssh/id_ed25519.pub user@server
|
||||
```
|
||||
|
||||
**Docker GPU overlays.** CPU-only users can skip this section. Cookbook can
|
||||
only detect GPUs that Docker exposes to the container — if the host runtime or
|
||||
device passthrough is not configured, Cookbook sees the iGPU, another card, or
|
||||
CPU instead of your intended GPU.
|
||||
|
||||
For NVIDIA, `scripts/check-docker-gpu.sh` diagnoses GPU passthrough and can
|
||||
optionally install the host runtime or update `.env`.
|
||||
|
||||
```bash
|
||||
# Read-only diagnostic (default — installs nothing, never edits .env):
|
||||
scripts/check-docker-gpu.sh
|
||||
|
||||
# Print OS-specific install commands without running them:
|
||||
scripts/check-docker-gpu.sh --print-install-commands
|
||||
|
||||
# Install NVIDIA Container Toolkit on Ubuntu/Debian (requires sudo):
|
||||
scripts/check-docker-gpu.sh --install-nvidia-toolkit
|
||||
|
||||
# Write COMPOSE_FILE to .env (only when GPU passthrough is confirmed working):
|
||||
scripts/check-docker-gpu.sh --enable-nvidia-overlay
|
||||
|
||||
# Full assisted setup — install toolkit, then enable overlay if passthrough works:
|
||||
scripts/check-docker-gpu.sh --install-nvidia-toolkit --enable-nvidia-overlay
|
||||
```
|
||||
|
||||
Safety notes:
|
||||
- The app never installs host GPU runtime automatically.
|
||||
- The app never edits `.env` automatically.
|
||||
- `.env` is only modified when `--enable-nvidia-overlay` is explicitly passed,
|
||||
and only after GPU passthrough succeeds. `--yes` skips prompts but does not
|
||||
bypass the passthrough gate.
|
||||
- `.env.bak.*` backups created by `--enable-nvidia-overlay` are ignored by
|
||||
Git and the Docker build context.
|
||||
|
||||
To enable manually without the script, add this to `.env`:
|
||||
|
||||
```bash
|
||||
COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
|
||||
```
|
||||
|
||||
**AMD / ROCm.** AMD setup is read-only diagnostic plus manual `.env` edit. Run:
|
||||
|
||||
```bash
|
||||
scripts/check-docker-amd-gpu.sh
|
||||
```
|
||||
|
||||
Then add the reported values to `.env`, replacing `RENDER_GID` with your host's
|
||||
numeric render group id:
|
||||
|
||||
```bash
|
||||
COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
|
||||
RENDER_GID=989
|
||||
```
|
||||
|
||||
For NVIDIA/AMD GPU support, also read the comments in the selected overlay file: docker/gpu.nvidia.yml or docker/gpu.amd.yml.
|
||||
|
||||
**Stack-management UIs (Portainer, Coolify, Dockhand, etc.).** These tools
|
||||
often accept only a single Compose file and do not reliably honor `COMPOSE_FILE`
|
||||
or multiple `-f` overlays. CLI users should keep using the `COMPOSE_FILE`
|
||||
overlay workflow above. For stack UIs, point the stack at one of the standalone
|
||||
files instead, which bundle the base stack plus the GPU settings:
|
||||
|
||||
- `docker-compose.gpu-nvidia.yml` — still requires the NVIDIA Container Toolkit
|
||||
on the host.
|
||||
- `docker-compose.gpu-amd.yml` — still requires host ROCm/kfd/DRI setup, the
|
||||
`video`/`render` group membership, and `RENDER_GID` when needed.
|
||||
|
||||
The base `docker-compose.yml` plus the `docker/gpu.*.yml` overlays remain the
|
||||
source of truth; the standalone files mirror them for single-file deployments.
|
||||
|
||||
Verify after enabling either overlay:
|
||||
|
||||
```bash
|
||||
docker compose exec odysseus nvidia-smi -L # NVIDIA
|
||||
docker compose exec odysseus sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls -l /dev/kfd /dev/dri/renderD*' # AMD
|
||||
```
|
||||
|
||||
> **GPU passthrough ≠ llama.cpp CUDA.** `nvidia-smi` passing inside the
|
||||
> container confirms Docker GPU access, but llama.cpp also needs `cudart` and
|
||||
> the CUDA Toolkit at runtime. If Cookbook logs show `Unable to find cudart
|
||||
> library`, `Could NOT find CUDAToolkit`, `CUDA Toolkit not found`, or
|
||||
> tensors/layers assigned to CPU, that is a Cookbook/llama.cpp build issue —
|
||||
> not a Docker passthrough failure. Re-install the serve engine via
|
||||
> **Cookbook → Dependencies** to get a CUDA-enabled build.
|
||||
>
|
||||
> The same split applies to AMD/ROCm: seeing `/dev/kfd` and `/dev/dri` inside
|
||||
> the container confirms device passthrough, not ROCm userspace or a
|
||||
> ROCm-enabled vLLM/llama.cpp build. `rocm-smi` and `rocminfo` are not expected
|
||||
> inside the slim Odysseus image.
|
||||
|
||||
**Ollama with Docker.** If Ollama runs on the host, add this endpoint in
|
||||
Settings:
|
||||
|
||||
```text
|
||||
http://host.docker.internal:11434/v1
|
||||
```
|
||||
|
||||
Ollama must listen outside its own loopback interface:
|
||||
|
||||
```bash
|
||||
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
||||
```
|
||||
|
||||
This connects Odysseus in Docker to an Ollama server that is already running on
|
||||
your host machine; it does not start Ollama inside the container.
|
||||
`host.docker.internal` is Docker's hostname for the host machine from inside the
|
||||
container. Cookbook **Serve** is a separate workflow for serving downloaded
|
||||
models through Odysseus/llama.cpp, so Windows users with an existing Ollama
|
||||
install usually only need to add the endpoint in Settings.
|
||||
|
||||
**Useful checks.**
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker compose logs --tail=120 odysseus
|
||||
docker compose logs odysseus | grep -E 'ChromaDB|MemoryVectorStore|DEGRADED'
|
||||
```
|
||||
|
||||
**macOS details.** `start-macos.sh` installs Homebrew deps, creates the venv,
|
||||
runs setup, and starts uvicorn on port `7860` because AirPlay often holds
|
||||
`7000`. It uses llama.cpp/Ollama for Metal. vLLM/SGLang are CUDA/ROCm-only and
|
||||
do not run on macOS. MLX-only models are not served by Odysseus.
|
||||
|
||||
</details>
|
||||
|
||||
### Native Windows
|
||||
|
||||
**One-command launcher** (creates the venv, installs deps, runs setup, starts the
|
||||
server; safe to re-run):
|
||||
|
||||
```powershell
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
cd odysseus
|
||||
powershell -ExecutionPolicy Bypass -File .\launch-windows.ps1
|
||||
```
|
||||
|
||||
Or do it by hand:
|
||||
|
||||
```powershell
|
||||
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
|
||||
cd odysseus
|
||||
py -3.11 -m venv venv
|
||||
venv\Scripts\Activate.ps1
|
||||
pip install -r requirements.txt
|
||||
python setup.py
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 7000
|
||||
```
|
||||
|
||||
If `python` points at an older interpreter, use `py -3.12` (or another installed
|
||||
3.11+ version) for the venv step.
|
||||
|
||||
**Requirements:** Python 3.11+. The core app (chat, agent, memory, documents,
|
||||
email, calendar, deep research) runs fully native. For full **Cookbook** background
|
||||
model downloads and the agent shell tool, also install
|
||||
[Git for Windows](https://git-scm.com/download/win) (provides `bash.exe`).
|
||||
Local GPU *serving* of vLLM/SGLang needs Linux/WSL2; for a local model on Windows,
|
||||
[Ollama](https://ollama.com/download) is the easiest path — point Odysseus at
|
||||
`http://localhost:11434/v1` in Settings.
|
||||
|
||||
Open `http://localhost:7000`, log in with the generated admin password,
|
||||
and configure everything else inside **Settings**.
|
||||
|
||||
## Troubleshooting & Advanced Setup
|
||||
|
||||
### `chromadb-client` conflicts with embedded ChromaDB
|
||||
If `chromadb-client` (the lightweight HTTP-only package) is installed alongside the full `chromadb` package, Odysseus starts but ChromaDB silently falls back to HTTP-only mode and fails.
|
||||
|
||||
**Fix:** uninstall `chromadb-client` and force-reinstall the full package:
|
||||
```bash
|
||||
./venv/bin/pip uninstall chromadb-client -y
|
||||
./venv/bin/pip install --force-reinstall chromadb
|
||||
```
|
||||
|
||||
### HTTPS + LAN/Tailscale exposure
|
||||
To expose Odysseus on a local network or Tailscale with HTTPS:
|
||||
1. Change the bind address to `0.0.0.0` in `.env` (`APP_BIND=0.0.0.0` or `ODYSSEUS_HOST=0.0.0.0`).
|
||||
2. Generate a locally-trusted cert for your LAN/Tailscale IPs using [mkcert](https://github.com/FiloSottile/mkcert):
|
||||
```bash
|
||||
mkcert -install
|
||||
mkcert -cert-file cert.pem -key-file key.pem 192.168.1.100 tailscale-ip
|
||||
```
|
||||
3. Run `uvicorn` with the generated certs:
|
||||
```bash
|
||||
python -m uvicorn app:app --host 0.0.0.0 --port 7000 --ssl-certfile=cert.pem --ssl-keyfile=key.pem
|
||||
```
|
||||
4. Install the `mkcert` CA on any other device you want to access Odysseus from (e.g., for iOS, email the `rootCA.pem` to yourself, install the profile, and trust it in Certificate Trust Settings).
|
||||
|
||||
### Optional Dependencies
|
||||
`requirements-optional.txt` contains packages that unlock extra features. It is not installed by default.
|
||||
|
||||
| Package | Feature unlocked |
|
||||
|---------|-----------------|
|
||||
| `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
|
||||
| `duckduckgo-search` | DuckDuckGo as a search provider option. |
|
||||
| `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
|
||||
| `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
|
||||
|
||||
### Outlook / Office 365 email
|
||||
Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
|
||||
and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
|
||||
passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
|
||||
current limitation and the planned integration direction.
|
||||
|
||||
## Security Notes
|
||||
Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
|
||||
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly.
|
||||
|
||||
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
|
||||
- Keep `LOCALHOST_BYPASS=false` outside local development.
|
||||
- Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
|
||||
- Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
|
||||
- Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
|
||||
- Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
|
||||
- Non-admin users do not get shell/Python/file read/write by default, and admin-only routes/tools such as MCP management, API tokens, webhooks, model/cookbook serving, backup/vault, and app settings are admin-gated. Other features are controlled by per-user privileges, so review each user's privileges before exposing a deployment.
|
||||
- Rotate any API keys or tokens that were ever pasted into a shared chat, demo, screenshot, or log.
|
||||
- If you enable API tokens or webhooks, create separate tokens per integration and delete unused ones.
|
||||
- Prefer binding manual development runs to `127.0.0.1`; bind to `0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
|
||||
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
|
||||
- Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
|
||||
|
||||
### Private or proxied deployments
|
||||
Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
|
||||
|
||||
1. Keep Odysseus on localhost, for example `127.0.0.1:7000`.
|
||||
2. Terminate HTTPS at a trusted reverse proxy or private access gateway.
|
||||
3. Put the authenticated Odysseus web/API entrypoint behind that layer.
|
||||
4. Keep raw service and model ports internal-only.
|
||||
|
||||
Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
|
||||
|
||||
Common internal-only ports from the default docs/compose setup:
|
||||
|
||||
| Port | Service |
|
||||
|---|---|
|
||||
| `7000` | Odysseus raw app port |
|
||||
| `8080` | SearXNG |
|
||||
| `8091` | ntfy |
|
||||
| `8100` | ChromaDB host port for manual/compose access |
|
||||
| `11434` | Ollama |
|
||||
| `8000-8020` | Common local model/provider APIs |
|
||||
|
||||
## Contributing
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup
|
||||
bugs, mobile/editor polish, docs, and small focused refactors. See
|
||||
[ROADMAP.md](ROADMAP.md) for the current help-wanted list.
|
||||
|
||||
## Configuration
|
||||
Most setup is done inside the app with `/setup` or **Settings**. Use `.env`
|
||||
for deployment-level defaults and secrets you want present before first boot.
|
||||
Key settings:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `LLM_HOST` | `localhost` | Your LLM server (e.g. `llm-host.local:8000`) |
|
||||
| `LLM_HOSTS` | -- | Comma-separated list for model discovery |
|
||||
| `OPENAI_API_KEY` | -- | Optional OpenAI key. Prefer adding providers in the app unless pre-seeding. |
|
||||
| `SEARXNG_INSTANCE` | `http://localhost:8080` | SearXNG URL. Docker overrides this to `http://searxng:8080`. |
|
||||
| `SEARXNG_SECRET` | generated on first Docker boot | Optional SearXNG cookie/CSRF secret. Leave blank unless you need to pin it. |
|
||||
| `APP_BIND` | `127.0.0.1` | Docker Compose host bind address for the web UI. Use `0.0.0.0` only for intentional LAN/reverse-proxy access. |
|
||||
| `APP_PORT` | `7000` | Docker Compose host port for the web UI. |
|
||||
| `AUTH_ENABLED` | `true` | Enable/disable login |
|
||||
| `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
|
||||
| `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
|
||||
| `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
|
||||
| `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
|
||||
| `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
|
||||
| `EMBEDDING_URL` | -- | OpenAI-compatible embeddings endpoint |
|
||||
| `ODYSSEUS_CHAT_UPLOAD_MAX_BYTES` | `10485760` | Chat/agent attachment cap in bytes. Raise for larger local PDFs or text documents. |
|
||||
| `ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES` | `104857600` | Gallery image upload cap in bytes (100 MB). |
|
||||
| `ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES` | `26214400` | Gallery transform input cap in bytes (25 MB). |
|
||||
| `ODYSSEUS_MEMORY_IMPORT_MAX_BYTES` | `10485760` | Memory import file cap in bytes (10 MB). |
|
||||
| `ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES` | `26214400` | Personal document upload cap in bytes (25 MB). |
|
||||
| `ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES` | `26214400` | Email compose attachment cap in bytes (25 MB). |
|
||||
| `ODYSSEUS_STT_MAX_AUDIO_BYTES` | `26214400` | Speech-to-text audio cap in bytes (25 MB). |
|
||||
| `ODYSSEUS_ICS_MAX_BYTES` | `10485760` | Calendar `.ics` import cap in bytes (10 MB). |
|
||||
|
||||
All upload-limit vars are validated (must be a positive integer) and optional; an invalid value fails fast at startup.
|
||||
|
||||
### Built-in MCP servers (optional setup)
|
||||
|
||||
Odysseus auto-registers a few built-in MCP servers at startup. The npx-based ones (currently the browser server, `@playwright/mcp`) only start when their npm package is already in the local npx cache. If a package isn't cached, that server is skipped with a startup log message explaining what to do, so a fresh install does not block on a multi-minute npm download or hang if Playwright system deps are missing.
|
||||
|
||||
To enable the browser MCP (page navigation, screenshots, vision), run once:
|
||||
|
||||
```bash
|
||||
npx -y @playwright/mcp@latest --version
|
||||
```
|
||||
|
||||
That installs `@playwright/mcp` plus Playwright (~300MB total). Restart Odysseus and the server will register at startup.
|
||||
|
||||
## Architecture
|
||||
```
|
||||
app.py # FastAPI entry point
|
||||
core/ auth, database, middleware, constants
|
||||
src/ llm_core, agent_loop, agent_tools, chat_processor, search/
|
||||
routes/ chat, session, document, memory, model … endpoints
|
||||
services/ docs, memory, search, hwfit (Cookbook) …
|
||||
static/ index.html + app.js + style.css + js/ (modular front-end)
|
||||
docs/ landing page (index.html) + preview clips
|
||||
```
|
||||
|
||||
## Data
|
||||
All user data lives in `data/` (gitignored): `app.db` (sessions, messages, documents),
|
||||
`memory.json`, `presets.json`, `uploads/`, `personal_docs/`, `chroma/`, `settings.json`.
|
||||
Deployment details are in the [setup guide](website/setup.md#security-notes).
|
||||
|
||||
## Star History
|
||||
|
||||
<a href="https://www.star-history.com/?repos=pewdiepie-archdaemon%2Fodysseus&type=date&legend=top-left">
|
||||
<a href="https://star-history.dera.page/#odysseus-dev/odysseus&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=pewdiepie-archdaemon/odysseus&type=date&theme=dark&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=pewdiepie-archdaemon/odysseus&type=date&legend=top-left" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=pewdiepie-archdaemon/odysseus&type=date&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&theme=dark&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
|
||||
<img alt="Star History Chart" src="https://star-history.dera.page/svg?repos=odysseus-dev/odysseus&type=date&legend=top-left" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
## License
|
||||
MIT -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
|
||||
|
||||
```
|
||||
|
|
||||
|||
|
||||
|||||
|
||||
| | | |||||||
|
||||
)_) )_) )_) ~|~
|
||||
)___))___))___)\ |
|
||||
)____)____)_____)\\|
|
||||
_____|____|____|_____\\\__
|
||||
\ /
|
||||
~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~
|
||||
~^~ all aboard! ~^~
|
||||
~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~
|
||||
```
|
||||
AGPL-3.0-or-later -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
|
||||
|
||||
@@ -12,7 +12,6 @@ the codebase, you are probably right to stay away.
|
||||
and WSL all need coverage.
|
||||
|
||||
- Integration audit: do integrations even work? Confirm what works, what needs setup docs, and what should be removed or hidden.
|
||||
- Self-host troubleshooting cookbook. Document the weird 30-second fixes that otherwise become 30-minute searches: Dovecot cleartext auth for local stacks, ntfy Android Instant Delivery for non-ntfy.sh servers, clipboard limits on plain-HTTP Tailscale URLs, Radicale collection URLs, and similar traps.
|
||||
- Cookbook reliability on other computers. This is probably the area most likely to need work across different machines, GPUs, drivers, shells, and Python environments.
|
||||
- Cookbook SGLang support across platforms. Make sure SGLang setup/serve works
|
||||
predictably on Linux, Windows/WSL, macOS where possible, Docker, and common
|
||||
@@ -33,6 +32,14 @@ the codebase, you are probably right to stay away.
|
||||
before the user request really starts. We need slimmer prompts, better tool
|
||||
selection, smaller default tool sets, and clearer guidance for models with
|
||||
4k/8k/16k context windows.
|
||||
- Local model speculative decoding support. For Odysseus-tuned local models,
|
||||
plan to ship or recommend a small same-tokenizer draft model when the serving
|
||||
backend supports it. Early vLLM testing showed a generic `Qwen3-0.6B` draft
|
||||
beside `Qwen3-8B` can materially reduce wall time, while an unsupported
|
||||
DSpark conversion performed poorly. Treat this as a supported draft-model lane
|
||||
first; keep MTP-specific packaging as future work only when the architecture
|
||||
and runtime support are real. Judge this by time-to-success, tool correctness,
|
||||
grammar, and unchanged target output, not tokens/sec alone.
|
||||
- Skill/tool prompt-injection audit. User-editable skills, notes, documents,
|
||||
fetched pages, and memories should be treated as untrusted data. Keep testing
|
||||
whether models follow malicious instructions from those surfaces.
|
||||
|
||||
@@ -10,7 +10,7 @@ Security fixes are handled on the default branch until formal releases are cut.
|
||||
|
||||
- Keep `AUTH_ENABLED=true` for any network-accessible deployment.
|
||||
- Keep `LOCALHOST_BYPASS=false` outside local development.
|
||||
- Set `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
|
||||
- Leave `SECURE_COOKIES` unset unless you need to override it: session cookies are marked `Secure` whenever the request arrives over HTTPS. Set `SECURE_COOKIES=true` to force it on (for a proxy Odysseus cannot see the scheme of), or `SECURE_COOKIES=false` to force it off while you still serve plain HTTP alongside HTTPS.
|
||||
- Use HTTPS when exposing the app beyond localhost.
|
||||
- Put the authenticated Odysseus web/API entrypoint behind a trusted reverse proxy or private access layer such as Cloudflare Access, Tailscale, or a VPN.
|
||||
- Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only.
|
||||
|
||||
@@ -37,7 +37,7 @@ Non-admin defaults are in `core/auth.py:DEFAULT_PRIVILEGES`. Tool enforcement is
|
||||
|
||||
- **Sessions:** bcrypt passwords, 7-day session tokens stored atomically in `data/sessions.json` via `core/atomic_io.py`.
|
||||
- **2FA:** TOTP with 8 single-use backup codes. Verified after password check, before session issuance.
|
||||
- **Reserved usernames:** `internal-tool`, `api`, `demo`, `system` cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`.
|
||||
- **Reserved usernames:** request sentinels and the Default/Local storage owner cannot be registered or renamed into. Defined in `core/auth.py:RESERVED_USERNAMES`.
|
||||
- `internal-tool` is security-critical: `core/middleware.py:require_admin` treats any request where `request.state.current_user == "internal-tool"` as the in-process tool loopback and grants admin unconditionally. A real account with that name would silently pass every `require_admin` check.
|
||||
- **Orphan sessions:** `validate_token` re-checks that the user record still exists on every call. A deleted user's cookie is dropped on next request rather than continuing to authenticate.
|
||||
|
||||
|
||||
@@ -1,6 +1,20 @@
|
||||
# app.py — slim orchestrator
|
||||
import mimetypes
|
||||
import os
|
||||
import sys
|
||||
import asyncio
|
||||
import time
|
||||
import shutil
|
||||
import socket
|
||||
|
||||
# On Windows, asyncio.create_subprocess_exec/shell require the ProactorEventLoop.
|
||||
# When started via `python -m uvicorn` from a terminal, uvicorn sets this
|
||||
# automatically. But the VS Code debugger (and other non-uvicorn entrypoints)
|
||||
# use the default SelectorEventLoop, which raises NotImplementedError on any
|
||||
# subprocess call. Force ProactorEventLoop here so the right loop is always
|
||||
# used, regardless of how the process is launched.
|
||||
if sys.platform == "win32":
|
||||
asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
|
||||
|
||||
|
||||
def register_static_mime_types() -> None:
|
||||
@@ -38,15 +52,16 @@ load_dotenv(encoding="utf-8-sig")
|
||||
import asyncio
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from typing import Dict
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI, Request, HTTPException
|
||||
from fastapi.responses import JSONResponse, FileResponse, HTMLResponse
|
||||
from fastapi.responses import JSONResponse, FileResponse
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.middleware.gzip import GZipMiddleware
|
||||
|
||||
# Core imports
|
||||
from core.constants import (
|
||||
@@ -54,8 +69,14 @@ from core.constants import (
|
||||
REQUEST_TIMEOUT, OPENAI_API_KEY, AUTH_FILE,
|
||||
)
|
||||
from core.database import SessionLocal, ApiToken
|
||||
from core.middleware import SecurityHeadersMiddleware, is_cors_preflight
|
||||
from core.auth import AuthManager
|
||||
from core.middleware import (
|
||||
SecurityHeadersMiddleware,
|
||||
get_application_route_path,
|
||||
is_cors_preflight,
|
||||
path_is_route_or_child,
|
||||
with_asgi_root_path,
|
||||
)
|
||||
from core.auth import AuthManager, normalize_known_username
|
||||
from core.exceptions import (
|
||||
SessionNotFoundError, InvalidFileUploadError,
|
||||
LLMServiceError, WebSearchError,
|
||||
@@ -63,15 +84,43 @@ from core.exceptions import (
|
||||
|
||||
import bcrypt as _bcrypt
|
||||
|
||||
from src.app_helpers import abs_join
|
||||
from src.app_helpers import abs_join, serve_html_with_nonce
|
||||
from src.generated_images import GENERATED_IMAGE_HEADERS, resolve_generated_image_path
|
||||
from src.owner_identity import auth_disabled
|
||||
from starlette.responses import RedirectResponse
|
||||
|
||||
# ========= LOGGING =========
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
||||
)
|
||||
import logging.handlers
|
||||
from core.constants import DATA_DIR
|
||||
|
||||
_root_logger = logging.getLogger()
|
||||
_root_logger.setLevel(logging.INFO)
|
||||
_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
|
||||
|
||||
# Clear existing handlers to avoid duplicates
|
||||
for _h in list(_root_logger.handlers):
|
||||
_root_logger.removeHandler(_h)
|
||||
|
||||
_console_h = logging.StreamHandler()
|
||||
_console_h.setFormatter(_formatter)
|
||||
_root_logger.addHandler(_console_h)
|
||||
|
||||
try:
|
||||
_log_dir = os.path.join(DATA_DIR, "logs")
|
||||
os.makedirs(_log_dir, exist_ok=True)
|
||||
_log_file = os.path.join(_log_dir, "app.log")
|
||||
|
||||
# RotatingFileHandler is not multi-process safe (e.g. if uvicorn is run with --workers N).
|
||||
# Odysseus is single-process by convention, so this is acceptable, but be aware that
|
||||
# concurrent log rotation issues can arise if multiple workers are configured.
|
||||
_file_h = logging.handlers.RotatingFileHandler(
|
||||
_log_file, maxBytes=5 * 1024 * 1024, backupCount=3, encoding="utf-8"
|
||||
)
|
||||
_file_h.setFormatter(_formatter)
|
||||
_root_logger.addHandler(_file_h)
|
||||
except Exception as e:
|
||||
_root_logger.warning(f"Failed to initialize file logging handler (falling back to console-only): {e}")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ========= APP =========
|
||||
@@ -85,12 +134,13 @@ app = FastAPI(
|
||||
)
|
||||
|
||||
# ========= CORS =========
|
||||
CORS_ALLOW_METHODS = ["GET", "POST", "PUT", "PATCH", "DELETE"]
|
||||
allowed_origins = os.getenv("ALLOWED_ORIGINS", "http://localhost,http://127.0.0.1").split(",")
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=allowed_origins,
|
||||
allow_credentials=True,
|
||||
allow_methods=["GET", "POST", "PUT", "DELETE"],
|
||||
allow_methods=CORS_ALLOW_METHODS,
|
||||
allow_headers=[
|
||||
"Accept",
|
||||
"Authorization",
|
||||
@@ -104,6 +154,17 @@ app.add_middleware(
|
||||
],
|
||||
)
|
||||
|
||||
# ========= RESPONSE COMPRESSION (gzip) =========
|
||||
# The frontend's text assets (style.css, index.html, the JS bundles) shipped
|
||||
# uncompressed on every cold load. gzip cuts CSS/JS/HTML by ~75-85% on the wire
|
||||
# with no behavioural change. Starlette's GZipMiddleware excludes
|
||||
# `text/event-stream` by default, so the SSE streams (chat, shell, research,
|
||||
# model-probe — all served with media_type="text/event-stream") are never
|
||||
# compressed or buffered; only complete bodies over minimum_size are. The
|
||||
# security-header middleware composes cleanly on top.
|
||||
if os.getenv("RESPONSE_COMPRESSION_ENABLED", "true").strip().lower() not in {"0", "false", "no", "off"}:
|
||||
app.add_middleware(GZipMiddleware, minimum_size=1024, compresslevel=6)
|
||||
|
||||
# ========= SECURITY HEADERS MIDDLEWARE =========
|
||||
app.add_middleware(SecurityHeadersMiddleware)
|
||||
|
||||
@@ -129,6 +190,7 @@ _TIMEOUT_EXEMPT_PREFIXES = (
|
||||
"/api/cookbook/setup", # remote pacman/apt installs
|
||||
"/api/upload", # large files
|
||||
"/api/image", # diffusion proxies (inpaint/harmonize/upscale/etc.) — own 120s httpx timeout
|
||||
"/api/memory/audit", # retains own 120s LLM inactivity timeout
|
||||
)
|
||||
|
||||
|
||||
@@ -146,14 +208,57 @@ class _RequestTimeoutMiddleware(_BaseHTTPMiddleware):
|
||||
)
|
||||
|
||||
|
||||
class _InteractiveActivityMiddleware(_BaseHTTPMiddleware):
|
||||
async def dispatch(self, request, call_next):
|
||||
from src.interactive_gate import should_track_interactive_request, track_interactive_request
|
||||
|
||||
path = request.url.path or ""
|
||||
if not should_track_interactive_request(path, request.method):
|
||||
return await call_next(request)
|
||||
async def _stop_background():
|
||||
try:
|
||||
await task_scheduler.stop_background_tasks_for_foreground(reason=f"foreground request {request.method} {path}")
|
||||
except Exception:
|
||||
logging.getLogger("app.foreground_gate").debug("foreground task stop failed", exc_info=True)
|
||||
asyncio.create_task(_stop_background())
|
||||
async with track_interactive_request(path, request.method):
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
class _SlowRequestLogMiddleware(_BaseHTTPMiddleware):
|
||||
async def dispatch(self, request, call_next):
|
||||
start = time.perf_counter()
|
||||
status = 500
|
||||
try:
|
||||
response = await call_next(request)
|
||||
status = getattr(response, "status_code", 0) or 0
|
||||
return response
|
||||
finally:
|
||||
elapsed = time.perf_counter() - start
|
||||
try:
|
||||
threshold = float(os.getenv("ODYSSEUS_SLOW_REQUEST_LOG_SECONDS", "0.75") or "0.75")
|
||||
except Exception:
|
||||
threshold = 0.75
|
||||
if elapsed >= threshold:
|
||||
logging.getLogger("app.slow_request").warning(
|
||||
"slow_request method=%s path=%s status=%s elapsed=%.3fs",
|
||||
request.method,
|
||||
request.url.path,
|
||||
status,
|
||||
elapsed,
|
||||
)
|
||||
|
||||
|
||||
app.add_middleware(_RequestTimeoutMiddleware)
|
||||
app.add_middleware(_InteractiveActivityMiddleware)
|
||||
app.add_middleware(_SlowRequestLogMiddleware)
|
||||
|
||||
# ========= AUTH =========
|
||||
from routes.auth_routes import setup_auth_routes, SESSION_COOKIE
|
||||
|
||||
auth_manager = AuthManager()
|
||||
app.state.auth_manager = auth_manager
|
||||
AUTH_ENABLED = os.getenv("AUTH_ENABLED", "true").lower() != "false"
|
||||
AUTH_ENABLED = not auth_disabled()
|
||||
LOCALHOST_BYPASS = os.getenv("LOCALHOST_BYPASS", "false").lower() == "true"
|
||||
if LOCALHOST_BYPASS:
|
||||
logger.warning("LOCALHOST_BYPASS is enabled, loopback requests bypass authentication. Do not expose this instance to a network.")
|
||||
@@ -189,7 +294,7 @@ if AUTH_ENABLED:
|
||||
def _is_auth_exempt(path: str) -> bool:
|
||||
if path in AUTH_EXEMPT_EXACT:
|
||||
return True
|
||||
if any(path.startswith(p) for p in AUTH_EXEMPT_PREFIXES):
|
||||
if any(path_is_route_or_child(path, p) for p in AUTH_EXEMPT_PREFIXES):
|
||||
return True
|
||||
return any(p.match(path) for p in AUTH_EXEMPT_PATTERNS)
|
||||
|
||||
@@ -217,8 +322,16 @@ if AUTH_ENABLED:
|
||||
try:
|
||||
rows = db.query(ApiToken).filter(ApiToken.is_active == True).all()
|
||||
for r in rows:
|
||||
owner_key = normalize_known_username(auth_manager.users, getattr(r, "owner", None))
|
||||
if not owner_key:
|
||||
logger.warning(
|
||||
"Ignoring active API token '%s' for unknown auth user '%s'",
|
||||
getattr(r, "id", ""),
|
||||
getattr(r, "owner", None),
|
||||
)
|
||||
continue
|
||||
scopes = [s.strip() for s in (getattr(r, "scopes", "") or "chat").split(",") if s.strip()]
|
||||
new_map[r.token_prefix].append((r.id, r.token_hash, getattr(r, "owner", None), scopes))
|
||||
new_map[r.token_prefix].append((r.id, r.token_hash, owner_key, scopes))
|
||||
finally:
|
||||
db.close()
|
||||
_token_cache.clear()
|
||||
@@ -252,7 +365,7 @@ if AUTH_ENABLED:
|
||||
|
||||
class AuthMiddleware(BaseHTTPMiddleware):
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
path = request.url.path
|
||||
path = get_application_route_path(request.scope)
|
||||
# A genuine CORS preflight (OPTIONS + Access-Control-Request-Method)
|
||||
# carries no credentials by design and must reach CORSMiddleware to be
|
||||
# answered. AuthMiddleware is the outermost middleware, so gating the
|
||||
@@ -269,7 +382,7 @@ if AUTH_ENABLED:
|
||||
# (no admin cookie available in that context). Restricted to
|
||||
# loopback clients + matching token to keep it locked down.
|
||||
try:
|
||||
from core.middleware import INTERNAL_TOOL_HEADER, INTERNAL_TOOL_TOKEN as _ITT
|
||||
from core.middleware import INTERNAL_TOOL_HEADER, INTERNAL_TOOL_TOKEN as _ITT, INTERNAL_TOOL_USER
|
||||
_hdr = request.headers.get(INTERNAL_TOOL_HEADER)
|
||||
if _hdr and secrets.compare_digest(_hdr, _ITT) and _is_trusted_loopback(request):
|
||||
# Impersonation: when the agent's loopback call sets
|
||||
@@ -281,11 +394,11 @@ if AUTH_ENABLED:
|
||||
if _impersonate and _impersonate in getattr(_auth_mgr, "users", {}):
|
||||
request.state.current_user = _impersonate
|
||||
else:
|
||||
request.state.current_user = "internal-tool"
|
||||
request.state.current_user = INTERNAL_TOOL_USER
|
||||
request.state.api_token = False
|
||||
return await call_next(request)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _e:
|
||||
logger.warning("Internal tool auth header check failed", exc_info=_e)
|
||||
# Allow DIRECT localhost requests (internal service calls from
|
||||
# heartbeats etc.). Tunnel/proxy-forwarded requests are excluded by
|
||||
# _is_trusted_loopback so LOCALHOST_BYPASS can't be abused over a
|
||||
@@ -296,7 +409,10 @@ if AUTH_ENABLED:
|
||||
if not auth_manager.is_configured:
|
||||
# No users yet — redirect to login for first-time setup
|
||||
if not path.startswith("/api/"):
|
||||
return RedirectResponse(url="/login", status_code=302)
|
||||
return RedirectResponse(
|
||||
url=with_asgi_root_path(request.scope, "/login"),
|
||||
status_code=302,
|
||||
)
|
||||
return JSONResponse(status_code=401, content={"error": "Setup required"})
|
||||
|
||||
# --- Bearer token auth (API tokens for external integrations) ---
|
||||
@@ -338,11 +454,10 @@ if AUTH_ENABLED:
|
||||
_db.close()
|
||||
try:
|
||||
await _asyncio.to_thread(_do)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _e:
|
||||
logger.debug("Failed to update token last_used_at", exc_info=_e)
|
||||
_asyncio.create_task(_touch_last_used(matched_id))
|
||||
# Keep bearer-token callers out of normal cookie/user
|
||||
# routes. API-aware routes can read api_token_owner.
|
||||
request.state.current_user = "api"
|
||||
request.state.api_token = True
|
||||
request.state.api_token_id = matched_id
|
||||
@@ -359,7 +474,10 @@ if AUTH_ENABLED:
|
||||
if not auth_manager.validate_token(token):
|
||||
if path.startswith("/api/"):
|
||||
return JSONResponse(status_code=401, content={"error": "Not authenticated"})
|
||||
return RedirectResponse(url="/login", status_code=302)
|
||||
return RedirectResponse(
|
||||
url=with_asgi_root_path(request.scope, "/login"),
|
||||
status_code=302,
|
||||
)
|
||||
|
||||
# Attach current username to request state for downstream routes
|
||||
request.state.current_user = auth_manager.get_username_for_token(token)
|
||||
@@ -391,7 +509,7 @@ class _RevalidatingStatic(StaticFiles):
|
||||
return resp
|
||||
|
||||
|
||||
app.mount("/static", _RevalidatingStatic(directory="static"), name="static")
|
||||
app.mount("/static", _RevalidatingStatic(directory=STATIC_DIR), name="static")
|
||||
|
||||
# ========= GENERATED IMAGES =========
|
||||
@app.get("/api/generated-image/{filename}")
|
||||
@@ -417,8 +535,8 @@ async def serve_generated_image(filename: str, request: Request):
|
||||
_db.close()
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as _e:
|
||||
logger.warning("Image ownership verification failed for %r", filename, exc_info=_e)
|
||||
ext = filename.rsplit('.', 1)[-1].lower()
|
||||
mime = {
|
||||
"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg",
|
||||
@@ -472,14 +590,21 @@ components = initialize_managers(BASE_DIR, rag_manager)
|
||||
session_manager = components["session_manager"]
|
||||
from src.assistant_log import set_session_manager as _set_asst_sm
|
||||
_set_asst_sm(session_manager)
|
||||
# Set the global session manager singleton (used by core.models.Session.add_message)
|
||||
from core.models import set_session_manager_instance
|
||||
set_session_manager_instance(session_manager)
|
||||
app.state.session_manager = session_manager
|
||||
memory_manager = components["memory_manager"]
|
||||
memory_vector = components.get("memory_vector")
|
||||
upload_handler = components["upload_handler"]
|
||||
app.state.upload_handler = upload_handler
|
||||
personal_docs_mgr = components["personal_docs_manager"]
|
||||
app.state.personal_docs_manager = personal_docs_mgr
|
||||
api_key_manager = components["api_key_manager"]
|
||||
preset_manager = components["preset_manager"]
|
||||
chat_processor = components["chat_processor"]
|
||||
research_handler = components["research_handler"]
|
||||
app.state.research_handler = research_handler
|
||||
chat_handler = components["chat_handler"]
|
||||
model_discovery = components["model_discovery"]
|
||||
skills_manager = components["skills_manager"]
|
||||
@@ -518,6 +643,31 @@ webhook_manager = WebhookManager(api_key_manager=api_key_manager)
|
||||
auth_router = setup_auth_routes(auth_manager)
|
||||
app.include_router(auth_router)
|
||||
|
||||
|
||||
@app.post("/api/activity/heartbeat")
|
||||
async def activity_heartbeat():
|
||||
from src.interactive_gate import (
|
||||
mark_browser_activity,
|
||||
maybe_stop_background_tasks_for_heartbeat,
|
||||
)
|
||||
|
||||
await mark_browser_activity()
|
||||
|
||||
async def _stop_background():
|
||||
try:
|
||||
await maybe_stop_background_tasks_for_heartbeat(
|
||||
task_scheduler.stop_background_tasks_for_foreground
|
||||
)
|
||||
except Exception:
|
||||
logging.getLogger("app.foreground_gate").debug(
|
||||
"heartbeat task stop failed",
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
asyncio.create_task(_stop_background())
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
# Uploads
|
||||
from routes.upload_routes import setup_upload_routes
|
||||
upload_router, upload_cleanup_func = setup_upload_routes(upload_handler)
|
||||
@@ -532,14 +682,20 @@ app.include_router(setup_emoji_routes())
|
||||
# Sessions
|
||||
from routes.session_routes import setup_session_routes
|
||||
session_config = {"REQUEST_TIMEOUT": REQUEST_TIMEOUT, "OPENAI_API_KEY": OPENAI_API_KEY, "SESSIONS_FILE": SESSIONS_FILE}
|
||||
app.include_router(setup_session_routes(session_manager, session_config, webhook_manager=webhook_manager))
|
||||
app.include_router(setup_session_routes(
|
||||
session_manager,
|
||||
session_config,
|
||||
webhook_manager=webhook_manager,
|
||||
upload_handler=upload_handler,
|
||||
skills_manager=skills_manager,
|
||||
))
|
||||
|
||||
# Admin Danger Zone wipes (Settings → System → Danger Zone)
|
||||
from routes.admin_wipe_routes import setup_admin_wipe_routes
|
||||
from routes.admin_wipe.admin_wipe_routes import setup_admin_wipe_routes
|
||||
app.include_router(setup_admin_wipe_routes(session_manager))
|
||||
|
||||
# Memory
|
||||
from routes.memory_routes import setup_memory_routes
|
||||
from routes.memory.memory_routes import setup_memory_routes
|
||||
memory_router = setup_memory_routes(memory_manager, session_manager, memory_vector=memory_vector)
|
||||
app.include_router(memory_router)
|
||||
from routes.skills_routes import setup_skills_routes
|
||||
@@ -556,15 +712,15 @@ app.include_router(setup_chat_routes(
|
||||
))
|
||||
|
||||
# Research (background deep-research tasks)
|
||||
from routes.research_routes import setup_research_routes
|
||||
from routes.research.research_routes import setup_research_routes
|
||||
app.include_router(setup_research_routes(research_handler, session_manager=session_manager))
|
||||
|
||||
# History
|
||||
from routes.history_routes import setup_history_routes
|
||||
app.include_router(setup_history_routes(session_manager))
|
||||
from routes.history.history_routes import setup_history_routes
|
||||
app.include_router(setup_history_routes(session_manager, upload_handler=upload_handler))
|
||||
|
||||
# Search
|
||||
from routes.search_routes import setup_search_routes
|
||||
from routes.search.search_routes import setup_search_routes
|
||||
app.include_router(setup_search_routes(config))
|
||||
|
||||
# Presets
|
||||
@@ -573,10 +729,10 @@ app.include_router(setup_preset_routes(preset_manager))
|
||||
|
||||
# Diagnostics
|
||||
from routes.diagnostics_routes import setup_diagnostics_routes
|
||||
app.include_router(setup_diagnostics_routes(rag_manager, rag_available, research_handler))
|
||||
app.include_router(setup_diagnostics_routes(rag_manager, rag_available, research_handler, memory_vector))
|
||||
|
||||
# Cleanup
|
||||
from routes.cleanup_routes import setup_cleanup_routes
|
||||
from routes.cleanup.cleanup_routes import setup_cleanup_routes
|
||||
app.include_router(setup_cleanup_routes(session_manager))
|
||||
|
||||
# Personal docs
|
||||
@@ -611,7 +767,7 @@ app.include_router(setup_stt_routes(stt_service))
|
||||
logger.info("STT service initialized (provider managed via settings)")
|
||||
|
||||
# Documents (artifacts/canvas)
|
||||
from routes.document_routes import setup_document_routes
|
||||
from routes.document.document_routes import setup_document_routes
|
||||
document_router = setup_document_routes(session_manager, upload_handler)
|
||||
app.include_router(document_router)
|
||||
|
||||
@@ -620,7 +776,7 @@ from routes.signature_routes import setup_signature_routes
|
||||
app.include_router(setup_signature_routes())
|
||||
|
||||
# Gallery (image library)
|
||||
from routes.gallery_routes import setup_gallery_routes
|
||||
from routes.gallery.gallery_routes import setup_gallery_routes
|
||||
app.include_router(setup_gallery_routes())
|
||||
|
||||
# Persisted image-editor drafts (server-backed projects)
|
||||
@@ -632,7 +788,7 @@ from src.task_scheduler import TaskScheduler
|
||||
task_scheduler = TaskScheduler(session_manager)
|
||||
from src.event_bus import set_task_scheduler
|
||||
set_task_scheduler(task_scheduler)
|
||||
from routes.task_routes import setup_task_routes
|
||||
from routes.task.task_routes import setup_task_routes
|
||||
app.include_router(setup_task_routes(task_scheduler))
|
||||
|
||||
from routes.assistant_routes import setup_assistant_routes
|
||||
@@ -640,7 +796,7 @@ app.include_router(setup_assistant_routes(task_scheduler))
|
||||
|
||||
# Calendar (CalDAV)
|
||||
from routes.calendar_routes import setup_calendar_routes
|
||||
calendar_router = setup_calendar_routes()
|
||||
calendar_router = setup_calendar_routes(upload_handler=upload_handler)
|
||||
app.include_router(calendar_router)
|
||||
|
||||
# Shell (user-facing command execution)
|
||||
@@ -651,12 +807,15 @@ app.include_router(setup_shell_routes())
|
||||
from routes.cookbook_routes import setup_cookbook_routes
|
||||
app.include_router(setup_cookbook_routes())
|
||||
|
||||
from routes.workspace_routes import setup_workspace_routes
|
||||
app.include_router(setup_workspace_routes())
|
||||
|
||||
# Hardware model fitting (cookbook "What Fits?" tab)
|
||||
from routes.hwfit_routes import setup_hwfit_routes
|
||||
app.include_router(setup_hwfit_routes())
|
||||
|
||||
# Model A/B Comparison
|
||||
from routes.compare_routes import setup_compare_routes
|
||||
from routes.compare.compare_routes import setup_compare_routes
|
||||
app.include_router(setup_compare_routes(session_manager))
|
||||
|
||||
# User Preferences
|
||||
@@ -674,7 +833,7 @@ app.include_router(setup_font_routes())
|
||||
# MCP (Model Context Protocol)
|
||||
from src.mcp_manager import McpManager
|
||||
from src.agent_tools import set_mcp_manager
|
||||
from routes.mcp_routes import setup_mcp_routes
|
||||
from routes.mcp.mcp_routes import setup_mcp_routes
|
||||
|
||||
mcp_manager = McpManager()
|
||||
set_mcp_manager(mcp_manager)
|
||||
@@ -689,7 +848,7 @@ set_ai_rag_manager(rag_manager, personal_docs_mgr)
|
||||
logger.info("AI interaction tools initialized (session, memory, RAG, UI control)")
|
||||
|
||||
# Webhooks
|
||||
from routes.webhook_routes import setup_webhook_routes
|
||||
from routes.webhook.webhook_routes import setup_webhook_routes
|
||||
app.include_router(setup_webhook_routes(webhook_manager, auth_manager, session_manager, api_key_manager))
|
||||
|
||||
# API Tokens
|
||||
@@ -699,8 +858,8 @@ app.include_router(setup_api_token_routes())
|
||||
logger.info("Webhook & API token routes initialized")
|
||||
|
||||
# Notes (Google Keep-style notes/todos)
|
||||
from routes.note_routes import setup_note_routes
|
||||
app.include_router(setup_note_routes(task_scheduler))
|
||||
from routes.note.note_routes import setup_note_routes
|
||||
app.include_router(setup_note_routes(task_scheduler, upload_handler=upload_handler))
|
||||
|
||||
# Email
|
||||
from routes.email_routes import setup_email_routes
|
||||
@@ -721,11 +880,11 @@ app.include_router(setup_codex_routes(
|
||||
))
|
||||
app.include_router(setup_claude_routes())
|
||||
|
||||
from routes.vault_routes import setup_vault_routes
|
||||
from routes.vault.vault_routes import setup_vault_routes
|
||||
app.include_router(setup_vault_routes())
|
||||
|
||||
# Contacts (CardDAV)
|
||||
from routes.contacts_routes import setup_contacts_routes
|
||||
from routes.contacts.contacts_routes import setup_contacts_routes
|
||||
app.include_router(setup_contacts_routes())
|
||||
|
||||
from companion import setup_companion_routes
|
||||
@@ -733,23 +892,17 @@ app.include_router(setup_companion_routes())
|
||||
|
||||
# ========= ROUTES (kept in app.py) =========
|
||||
|
||||
def _serve_html_with_nonce(request: Request, file_path: str) -> HTMLResponse:
|
||||
"""Read an HTML file and inject the CSP nonce into inline <script> tags."""
|
||||
with open(file_path, "r", encoding="utf-8") as f:
|
||||
html = f.read()
|
||||
nonce = getattr(request.state, "csp_nonce", "")
|
||||
html = html.replace("{{CSP_NONCE}}", nonce)
|
||||
return HTMLResponse(html)
|
||||
|
||||
@app.get("/")
|
||||
async def serve_index(request: Request):
|
||||
static_path = abs_join(BASE_DIR, "static/index.html")
|
||||
if os.path.exists(static_path):
|
||||
return _serve_html_with_nonce(request, static_path)
|
||||
root_path = abs_join(BASE_DIR, "index.html")
|
||||
if os.path.exists(root_path):
|
||||
return _serve_html_with_nonce(request, root_path)
|
||||
raise HTTPException(404, "index.html not found")
|
||||
return serve_html_with_nonce(request, static_path)
|
||||
# No static bundle — fall back to a root-level index.html if one is shipped.
|
||||
# If neither exists, serve_html_with_nonce logs it and returns a generic 500:
|
||||
# a missing index.html is a broken deployment (server fault), not a client
|
||||
# "not found". This keeps the app-shell route consistent with the other
|
||||
# bundled-template routes instead of mislabelling the fault as a 404.
|
||||
return serve_html_with_nonce(request, abs_join(BASE_DIR, "index.html"))
|
||||
|
||||
@app.get("/notes")
|
||||
async def serve_notes(request: Request):
|
||||
@@ -790,22 +943,54 @@ async def serve_library(request: Request):
|
||||
@app.get("/backgrounds")
|
||||
async def serve_backgrounds(request: Request):
|
||||
"""Sandbox page for prototyping background effects. No auth required."""
|
||||
return _serve_html_with_nonce(request, abs_join(BASE_DIR, "static/backgrounds.html"))
|
||||
return serve_html_with_nonce(request, abs_join(BASE_DIR, "static/backgrounds.html"))
|
||||
|
||||
@app.get("/login")
|
||||
async def serve_login(request: Request):
|
||||
if not AUTH_ENABLED:
|
||||
return RedirectResponse(url="/", status_code=302)
|
||||
return _serve_html_with_nonce(request, abs_join(BASE_DIR, "static/login.html"))
|
||||
return serve_html_with_nonce(request, abs_join(BASE_DIR, "static/login.html"))
|
||||
|
||||
@app.get("/api/version")
|
||||
async def get_version():
|
||||
from core.constants import APP_VERSION
|
||||
return {"version": APP_VERSION}
|
||||
from core.constants import APP_BUILD_VERSION, APP_SOURCE_COMMIT, APP_VERSION
|
||||
return {
|
||||
"version": APP_VERSION,
|
||||
"build": APP_BUILD_VERSION,
|
||||
"source_commit": APP_SOURCE_COMMIT,
|
||||
}
|
||||
|
||||
@app.get("/api/health")
|
||||
async def health_check() -> Dict[str, str]:
|
||||
return {"status": "healthy", "timestamp": datetime.utcnow().isoformat()}
|
||||
return {"status": "healthy", "timestamp": datetime.now(timezone.utc).isoformat()}
|
||||
|
||||
@app.post("/api/client-perf")
|
||||
async def client_perf(request: Request):
|
||||
"""Low-volume frontend timing reports for stalls that happen before SSE logs."""
|
||||
try:
|
||||
data = await request.json()
|
||||
except Exception:
|
||||
data = {}
|
||||
try:
|
||||
kind = str(data.get("type") or "client").replace("\n", " ")[:80]
|
||||
total_ms = float(data.get("total_ms") or 0)
|
||||
stages = data.get("stages") if isinstance(data.get("stages"), list) else []
|
||||
stage_txt = " ".join(
|
||||
f"{str(s.get('name') or '')[:40]}={float(s.get('delta_ms') or 0):.0f}ms"
|
||||
for s in stages[:20]
|
||||
if isinstance(s, dict)
|
||||
)
|
||||
extra = str(data.get("extra") or "").replace("\n", " ")[:200]
|
||||
logging.getLogger("app.client_perf").warning(
|
||||
"client_perf type=%s total=%.0fms %s%s",
|
||||
kind,
|
||||
total_ms,
|
||||
stage_txt,
|
||||
f" extra={extra}" if extra else "",
|
||||
)
|
||||
except Exception:
|
||||
logging.getLogger("app.client_perf").debug("client_perf log failed", exc_info=True)
|
||||
return {"ok": True}
|
||||
|
||||
@app.get("/api/ready")
|
||||
async def readiness_check() -> JSONResponse:
|
||||
@@ -833,11 +1018,76 @@ async def runtime_info() -> Dict[str, object]:
|
||||
or os.getenv("OLLAMA_URL")
|
||||
or ("http://host.docker.internal:11434/v1" if in_docker else "http://127.0.0.1:11434/v1")
|
||||
)
|
||||
network_mode = os.getenv("ODYSSEUS_CONTAINER_NETWORK_MODE", "").strip()
|
||||
host_gateway_reachable = False
|
||||
host_gateway_address = ""
|
||||
if in_docker and network_mode != "host":
|
||||
try:
|
||||
resolved = socket.getaddrinfo("host.docker.internal", None)
|
||||
for item in resolved:
|
||||
sockaddr = item[4] if len(item) >= 5 else ()
|
||||
candidate = sockaddr[0] if sockaddr else ""
|
||||
if candidate:
|
||||
host_gateway_address = str(candidate)
|
||||
break
|
||||
host_gateway_reachable = True
|
||||
except OSError:
|
||||
host_gateway_reachable = False
|
||||
if not host_gateway_address:
|
||||
host_gateway_address = _docker_default_gateway_ip()
|
||||
container: Dict[str, object] = {
|
||||
"engine": "docker" if in_docker else "",
|
||||
"networkMode": network_mode,
|
||||
"hostAccess": bool(in_docker and network_mode == "host"),
|
||||
"hostGatewayReachable": host_gateway_reachable,
|
||||
}
|
||||
if host_gateway_address:
|
||||
container["hostGatewayAddress"] = host_gateway_address
|
||||
command_names = (
|
||||
"ip",
|
||||
"ss",
|
||||
"arp",
|
||||
"nmap",
|
||||
"ping",
|
||||
"dig",
|
||||
"ssh",
|
||||
"git",
|
||||
"docker",
|
||||
)
|
||||
commands = {name: bool(shutil.which(name)) for name in command_names}
|
||||
capabilities = {
|
||||
"networkInspection": bool(commands["ip"] and (commands["ss"] or commands["arp"])),
|
||||
"lanScan": bool(commands["nmap"]),
|
||||
"dnsLookup": bool(commands["dig"]),
|
||||
"sshClient": bool(commands["ssh"]),
|
||||
"git": bool(commands["git"]),
|
||||
"dockerClient": bool(commands["docker"]),
|
||||
}
|
||||
return {
|
||||
"in_docker": in_docker,
|
||||
"ollama_base_url": ollama_url,
|
||||
"container": container,
|
||||
"commands": commands,
|
||||
"capabilities": capabilities,
|
||||
}
|
||||
|
||||
|
||||
def _docker_default_gateway_ip() -> str:
|
||||
try:
|
||||
with open("/proc/net/route", "r", encoding="utf-8", errors="ignore") as fh:
|
||||
for line in fh.readlines()[1:]:
|
||||
parts = line.split()
|
||||
if len(parts) < 3 or parts[1] != "00000000":
|
||||
continue
|
||||
raw = parts[2]
|
||||
if len(raw) != 8:
|
||||
continue
|
||||
octets = [str(int(raw[i:i + 2], 16)) for i in range(6, -1, -2)]
|
||||
return ".".join(octets)
|
||||
except Exception:
|
||||
return ""
|
||||
return ""
|
||||
|
||||
# ========= LIFECYCLE =========
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -877,6 +1127,15 @@ async def _startup_event():
|
||||
# GC tasks created with `asyncio.create_task(...)` before they finish.
|
||||
_startup_tasks: list[asyncio.Task] = getattr(app.state, "_startup_tasks", [])
|
||||
app.state._startup_tasks = _startup_tasks
|
||||
from src.background_tool_jobs import BackgroundToolJobs
|
||||
from routes.chat_routes import _active_streams
|
||||
from src import agent_runs
|
||||
app.state.background_tool_jobs = BackgroundToolJobs(
|
||||
is_busy=lambda sid: sid in _active_streams or agent_runs.is_active(sid),
|
||||
session_manager=session_manager, research_handler=research_handler,
|
||||
)
|
||||
app.state.background_tool_delivery_task = asyncio.create_task(app.state.background_tool_jobs.run())
|
||||
_startup_tasks.append(app.state.background_tool_delivery_task)
|
||||
if upload_cleanup_func:
|
||||
upload_cleanup_task = asyncio.create_task(upload_cleanup_func())
|
||||
# Always-on monitor that auto-continues the agent when a background bash
|
||||
@@ -895,7 +1154,7 @@ async def _startup_event():
|
||||
except BaseException as e:
|
||||
logger.warning(f"Built-in MCP registration failed (non-critical): {type(e).__name__}: {e}")
|
||||
try:
|
||||
await asyncio.wait_for(mcp_manager.connect_all_enabled(), timeout=20)
|
||||
await mcp_manager.connect_all_enabled()
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("User MCP startup timed out (non-critical)")
|
||||
except BaseException as e:
|
||||
@@ -903,52 +1162,70 @@ async def _startup_event():
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_startup_mcp_connections()))
|
||||
|
||||
# Pre-warm the RAG tool index off the request path. Loading the local
|
||||
# embedding model + opening ChromaDB + indexing the built-in tools is a
|
||||
# one-time ~1-3s cost that otherwise lands on the user's FIRST message
|
||||
# (showing up as a big `tool_selection` time). Doing it here makes the
|
||||
# first turn as fast as subsequent ones (warm embed ≈ a few ms).
|
||||
async def _warmup_tool_index():
|
||||
try:
|
||||
from src.tool_index import get_tool_index
|
||||
idx = await asyncio.to_thread(get_tool_index)
|
||||
if idx:
|
||||
await asyncio.to_thread(idx.get_tools_for_query, "warmup", 8)
|
||||
logger.info("[startup] Tool index pre-warmed")
|
||||
except Exception as e:
|
||||
logger.warning(f"Tool index warmup failed (non-critical): {type(e).__name__}: {e}")
|
||||
# Semantic tool selection is part of the agent serving contract. Initialize
|
||||
# it in a background thread by default so startup remains nonblocking while
|
||||
# harness deployments can wait for the explicit readiness state.
|
||||
from src.tool_index import prewarm_tool_index, tool_index_prewarm_enabled
|
||||
if tool_index_prewarm_enabled():
|
||||
async def _warmup_tool_index():
|
||||
status = await asyncio.to_thread(prewarm_tool_index)
|
||||
if status.get("ready"):
|
||||
logger.info(
|
||||
"[startup] Tool index pre-warmed lanes=%s tools=%s duration_ms=%s",
|
||||
[lane.get("name") for lane in status.get("lanes", [])],
|
||||
status.get("builtin_tools"),
|
||||
status.get("duration_ms"),
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"Tool index warmup degraded (non-critical): %s",
|
||||
status.get("error_type") or status.get("state"),
|
||||
)
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_warmup_tool_index()))
|
||||
# Warmup: ping all known LLM endpoints to prime connections
|
||||
async def _warmup_endpoints():
|
||||
try:
|
||||
import httpx
|
||||
endpoints = model_discovery.get_endpoints() if model_discovery else []
|
||||
for ep in endpoints[:5]:
|
||||
url = ep.get("url", "").replace("/chat/completions", "/models")
|
||||
if url:
|
||||
_startup_tasks.append(asyncio.create_task(_warmup_tool_index()))
|
||||
else:
|
||||
logger.info("Tool index prewarm disabled (ODYSSEUS_TOOL_INDEX_PREWARM=0)")
|
||||
|
||||
# Model endpoint pings remain opt-in. They can compete with the first seconds
|
||||
# of UI use on slow or busy machines and are not required for local startup.
|
||||
_startup_warmups_enabled = str(os.getenv("ODYSSEUS_STARTUP_WARMUPS", "")).lower() in {"1", "true", "yes", "on"}
|
||||
if _startup_warmups_enabled:
|
||||
async def _warmup_endpoints():
|
||||
try:
|
||||
import httpx
|
||||
urls = (
|
||||
await asyncio.to_thread(model_discovery.warmup_ping_urls)
|
||||
if model_discovery else []
|
||||
)
|
||||
for url in urls:
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
await client.get(url)
|
||||
logger.info(f"Warmup ping OK: {url}")
|
||||
except Exception as e:
|
||||
logger.debug(f"Warmup ping failed for endpoint: {e}")
|
||||
except Exception as e:
|
||||
logger.debug(f"Warmup ping skipped: {e}")
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_warmup_endpoints()))
|
||||
|
||||
# Keep-alive: ping endpoints every 60 seconds to prevent cold starts
|
||||
async def _keepalive_loop():
|
||||
while True:
|
||||
try:
|
||||
await asyncio.sleep(60)
|
||||
await _warmup_endpoints()
|
||||
except Exception as e:
|
||||
logger.warning(f"Keepalive loop error: {e}")
|
||||
await asyncio.sleep(300) # Back off on error
|
||||
logger.debug(f"Warmup ping skipped: {e}")
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_keepalive_loop()))
|
||||
_startup_tasks.append(asyncio.create_task(_warmup_endpoints()))
|
||||
else:
|
||||
logger.info("Model endpoint warmups disabled (set ODYSSEUS_STARTUP_WARMUPS=1 to enable)")
|
||||
|
||||
# Keep-alive is opt-in. The ping path performs model discovery, and when
|
||||
# stale LAN endpoints are configured it can add periodic backend pressure
|
||||
# that delays unrelated UI requests such as Notes/Documents.
|
||||
_keepalive_enabled = str(os.getenv("ODYSSEUS_MODEL_KEEPALIVE", "")).lower() in {"1", "true", "yes", "on"}
|
||||
if _keepalive_enabled:
|
||||
async def _keepalive_loop():
|
||||
while True:
|
||||
try:
|
||||
await asyncio.sleep(60)
|
||||
await _warmup_endpoints()
|
||||
except Exception as e:
|
||||
logger.warning(f"Keepalive loop error: {e}")
|
||||
await asyncio.sleep(300) # Back off on error
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_keepalive_loop()))
|
||||
|
||||
async def _ensure_default_tasks():
|
||||
# Create/reconcile default automation tasks + personal assistant for every user.
|
||||
@@ -1000,6 +1277,14 @@ async def _startup_event():
|
||||
# Disk-backed skills are not covered by the DB legacy-owner sweep. Repair
|
||||
# ownerless or deleted/test-owner SKILL.md files so strict owner filtering
|
||||
# does not make an existing library look empty after auth/account changes.
|
||||
try:
|
||||
from services.memory.builtin_skills import install_builtin_skills
|
||||
installed = install_builtin_skills(skills_manager, ())
|
||||
if installed:
|
||||
logger.info("Installed %s built-in skill file(s)", installed)
|
||||
except Exception as e:
|
||||
logger.debug(f"Built-in skill installation skipped: {e}")
|
||||
|
||||
try:
|
||||
import json as _json
|
||||
auth_path = AUTH_FILE
|
||||
@@ -1045,35 +1330,10 @@ async def _startup_event():
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_null_owner_sweep_loop()))
|
||||
|
||||
# Nightly skill audit — at ~02:00 local, test + judge a batch of the
|
||||
# least-recently-checked skills, auto-fixing/escalating weak ones (never
|
||||
# deletes). Rotates through the library so each night covers different
|
||||
# skills. Gated by the `skill_audit_nightly` setting (default on); hour via
|
||||
# `skill_audit_hour` (default 2), batch size via `skill_audit_batch` (8).
|
||||
async def _skill_audit_nightly_loop():
|
||||
from datetime import timedelta
|
||||
while True:
|
||||
try:
|
||||
from src.settings import get_setting
|
||||
hour = int(get_setting("skill_audit_hour", 2) or 2)
|
||||
except Exception:
|
||||
hour = 2
|
||||
now = datetime.now()
|
||||
nxt = now.replace(hour=hour % 24, minute=0, second=0, microsecond=0)
|
||||
if nxt <= now:
|
||||
nxt += timedelta(days=1)
|
||||
await asyncio.sleep(max(60, (nxt - now).total_seconds()))
|
||||
try:
|
||||
from src.settings import get_setting
|
||||
if not get_setting("skill_audit_nightly", True):
|
||||
continue
|
||||
batch = int(get_setting("skill_audit_batch", 8) or 8)
|
||||
from routes.skills_routes import run_scheduled_skill_audit
|
||||
await run_scheduled_skill_audit(skills_manager, owner=None, max_skills=batch)
|
||||
except Exception as e:
|
||||
logger.warning(f"Nightly skill audit failed: {e}")
|
||||
|
||||
_startup_tasks.append(asyncio.create_task(_skill_audit_nightly_loop()))
|
||||
# Skills Audit is scheduled per owner by TaskScheduler. Do not also start
|
||||
# an ownerless audit here: its sidecar results cannot be read back through
|
||||
# an authenticated owner's skill namespace, and its model activity can
|
||||
# defer the real per-owner task at the same time of night.
|
||||
|
||||
# Cookbook serve lifecycle — kills scheduler-launched serves whose
|
||||
# window-end has passed. Paired with the cookbook_serve builtin
|
||||
@@ -1088,6 +1348,18 @@ async def _startup_event():
|
||||
|
||||
async def _shutdown_event():
|
||||
logger.info("Application shutting down...")
|
||||
background_delivery = getattr(app.state, 'background_tool_delivery_task', None)
|
||||
if background_delivery:
|
||||
background_delivery.cancel()
|
||||
try:
|
||||
await background_delivery
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
try:
|
||||
from src.agent_tools.web_tools import shutdown_private_browser_sessions
|
||||
await shutdown_private_browser_sessions()
|
||||
except Exception as e:
|
||||
logger.warning(f"Private browser shutdown error: {e}")
|
||||
if upload_cleanup_task:
|
||||
upload_cleanup_task.cancel()
|
||||
try:
|
||||
@@ -1110,3 +1382,12 @@ async def _shutdown_event():
|
||||
except Exception as e:
|
||||
logger.warning(f"MCP shutdown error: {e}")
|
||||
logger.info("Application shutdown complete")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
|
||||
bind_host = os.getenv("APP_BIND", "127.0.0.1")
|
||||
bind_port = int(os.getenv("APP_PORT", "7011"))
|
||||
|
||||
uvicorn.run(app, host=bind_host, port=bind_port, log_level="info")
|
||||
|
||||
|
After Width: | Height: | Size: 185 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 79 KiB |
@@ -27,13 +27,13 @@ echo " port: $PORT"
|
||||
rm -rf "$APP"
|
||||
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
|
||||
|
||||
# ── Icon (best effort) — center-crop docs/odysseus.jpg to a square .icns ──
|
||||
if [ -f "$REPO_DIR/docs/odysseus.jpg" ] && command -v sips >/dev/null 2>&1; then
|
||||
# ── Icon (best effort) — center-crop the branding image to a square .icns ──
|
||||
if [ -f "$REPO_DIR/assets/branding/odysseus.jpg" ] && command -v sips >/dev/null 2>&1; then
|
||||
TMPIMG="$(mktemp -d)"
|
||||
# Center-crop to a square, scale to 512 (sips' icns encoder caps at 512), and
|
||||
# let sips emit the .icns directly — more robust across macOS versions than
|
||||
# building an .iconset by hand.
|
||||
sips -c 720 720 "$REPO_DIR/docs/odysseus.jpg" --out "$TMPIMG/sq.png" >/dev/null 2>&1 || cp "$REPO_DIR/docs/odysseus.jpg" "$TMPIMG/sq.png"
|
||||
sips -c 720 720 "$REPO_DIR/assets/branding/odysseus.jpg" --out "$TMPIMG/sq.png" >/dev/null 2>&1 || cp "$REPO_DIR/assets/branding/odysseus.jpg" "$TMPIMG/sq.png"
|
||||
sips -z 512 512 "$TMPIMG/sq.png" --out "$TMPIMG/icon.png" >/dev/null 2>&1
|
||||
if sips -s format icns "$TMPIMG/icon.png" --out "$APP/Contents/Resources/odysseus.icns" >/dev/null 2>&1; then
|
||||
echo " icon: odysseus.icns"
|
||||
@@ -42,7 +42,7 @@ if [ -f "$REPO_DIR/docs/odysseus.jpg" ] && command -v sips >/dev/null 2>&1; then
|
||||
fi
|
||||
rm -rf "$TMPIMG"
|
||||
else
|
||||
echo " icon: (skipped — no docs/odysseus.jpg)"
|
||||
echo " icon: (skipped — no assets/branding/odysseus.jpg)"
|
||||
fi
|
||||
|
||||
# ── Info.plist ──
|
||||
@@ -73,6 +73,10 @@ cat > "$APP/Contents/MacOS/$APP_NAME.tmpl" <<'LAUNCHER'
|
||||
INSTALL_DIR="__INSTALL_DIR__"
|
||||
PORT="__PORT__"
|
||||
URL="http://127.0.0.1:${PORT}"
|
||||
# uvicorn is started with --port below, but APP_PORT is what the app itself
|
||||
# reads when it needs to build a URL for this instance (internal_api_base(),
|
||||
# companion pairing, the MCP OAuth callback), so export it as well.
|
||||
export APP_PORT="$PORT"
|
||||
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:$PATH"
|
||||
|
||||
UVICORN="$INSTALL_DIR/venv/bin/uvicorn"
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
#Requires -Version 5.1
|
||||
<#
|
||||
Build a portable Windows distribution for Odysseus.
|
||||
|
||||
Output layout:
|
||||
dist\Odysseus\Odysseus.exe
|
||||
dist\Odysseus\static\...
|
||||
dist\Odysseus\scripts\...
|
||||
dist\Odysseus\mcp_servers\...
|
||||
dist\Odysseus\services\hwfit\data\...
|
||||
|
||||
The app then keeps using its normal filesystem layout when frozen.
|
||||
|
||||
Usage:
|
||||
powershell -ExecutionPolicy Bypass -File .\build-windows-portable.ps1
|
||||
#>
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
Set-Location -Path $PSScriptRoot
|
||||
|
||||
function Write-Step($msg) { Write-Host ""; Write-Host ("==> " + $msg) -ForegroundColor Cyan }
|
||||
function Fail($msg) {
|
||||
Write-Host ""
|
||||
Write-Host ("ERROR: " + $msg) -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
|
||||
Write-Step "Checking for Python"
|
||||
$pyExe = $null
|
||||
if (Test-Path ".\.venv\Scripts\python.exe") {
|
||||
$pyExe = (Resolve-Path ".\.venv\Scripts\python.exe").Path
|
||||
} else {
|
||||
foreach ($c in @("py", "python")) {
|
||||
$cmd = Get-Command $c -ErrorAction SilentlyContinue
|
||||
if ($cmd) { $pyExe = $cmd.Source; break }
|
||||
}
|
||||
if ($pyExe -like "*WindowsApps*python.exe") {
|
||||
$pyCmd = Get-Command py -ErrorAction SilentlyContinue
|
||||
if ($pyCmd) {
|
||||
$pyExe = $pyCmd.Source
|
||||
}
|
||||
}
|
||||
}
|
||||
if (-not $pyExe) {
|
||||
Fail "Python not found on PATH. Install Python 3.11+ first."
|
||||
}
|
||||
Write-Host ("Using Python: " + $pyExe)
|
||||
|
||||
Write-Step "Installing build dependencies"
|
||||
& $pyExe -m pip install --upgrade pip --quiet
|
||||
& $pyExe -m pip install -r requirements.txt pyinstaller pystray Pillow
|
||||
if ($LASTEXITCODE -ne 0) { Fail "Dependency install failed." }
|
||||
|
||||
Write-Step "Building portable exe bundle"
|
||||
Remove-Item -Recurse -Force build, dist -ErrorAction SilentlyContinue
|
||||
|
||||
$dataArgs = @(
|
||||
"--add-data", "static;static",
|
||||
"--add-data", "scripts;scripts",
|
||||
"--add-data", "mcp_servers;mcp_servers",
|
||||
"--add-data", "services/hwfit/data;services/hwfit/data",
|
||||
"--add-data", "config;config",
|
||||
"--add-data", ".env.example;.env.example"
|
||||
)
|
||||
|
||||
& $pyExe -m PyInstaller --noconfirm --clean --onedir --noconsole --icon=static/icon.ico --name Odysseus @dataArgs launcher.py
|
||||
if ($LASTEXITCODE -ne 0) { Fail "PyInstaller build failed." }
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "Build complete." -ForegroundColor Green
|
||||
Write-Host "Portable app folder: $PSScriptRoot\dist\Odysseus" -ForegroundColor Green
|
||||
Write-Host "Distribute the whole folder (or zip it) so static assets and scripts stay with the exe." -ForegroundColor Green
|
||||
@@ -6,11 +6,14 @@ units so the route layer stays thin and the logic is directly testable.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import socket
|
||||
import uuid
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
import bcrypt
|
||||
|
||||
@@ -20,6 +23,102 @@ PAIRING_VERSION = 1
|
||||
COMPANION_SCOPE = "chat"
|
||||
|
||||
|
||||
_COMPANION_IPV4_NETWORKS = tuple(
|
||||
ipaddress.ip_network(cidr)
|
||||
for cidr in (
|
||||
"10.0.0.0/8",
|
||||
"100.64.0.0/10",
|
||||
"127.0.0.0/8",
|
||||
"169.254.0.0/16",
|
||||
"172.16.0.0/12",
|
||||
"192.168.0.0/16",
|
||||
)
|
||||
)
|
||||
_DNS_LABEL_RE = re.compile(r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\Z")
|
||||
|
||||
|
||||
def _valid_companion_client_host(host: str) -> bool:
|
||||
"""Match the host forms supported by the current v1 Expo client."""
|
||||
if not host or len(host) > 253 or not host.isascii() or "%" in host:
|
||||
return False
|
||||
|
||||
try:
|
||||
address = ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
labels = host.split(".")
|
||||
if any(not _DNS_LABEL_RE.fullmatch(label) for label in labels):
|
||||
return False
|
||||
if any(label.startswith("xn--") for label in labels):
|
||||
return False
|
||||
# WHATWG URL parsers treat a decimal or ``0x`` single-label hostname
|
||||
# as an IPv4 number even though Python's strict ``ipaddress`` parser
|
||||
# rejects that spelling. The v1 client interpolates this host back
|
||||
# into a URL, so accepting e.g. ``134744072`` would make the phone send
|
||||
# its bearer token to public 8.8.8.8. Keep DNS labels unambiguous.
|
||||
if len(labels) == 1 and (
|
||||
labels[0].isdigit()
|
||||
or re.fullmatch(r"0x[0-9a-f]*", labels[0]) is not None
|
||||
):
|
||||
return False
|
||||
return len(labels) == 1 or (len(labels) >= 2 and labels[-1] == "local")
|
||||
|
||||
return isinstance(address, ipaddress.IPv4Address) and any(
|
||||
address in network for network in _COMPANION_IPV4_NETWORKS
|
||||
)
|
||||
|
||||
|
||||
def parse_companion_base_url(value: str) -> tuple[str, int]:
|
||||
"""Validate a v1 companion address and return its legacy (host, port).
|
||||
|
||||
The deployed client understands only HTTP plus a LAN-style host and port.
|
||||
Reject anything outside that exact contract instead of advertising a URL
|
||||
the client would reject, downgrade, or interpret differently.
|
||||
"""
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
|
||||
if not value.isascii():
|
||||
raise ValueError("COMPANION_BASE_URL must contain only ASCII characters")
|
||||
if any(
|
||||
ord(char) <= 32 or ord(char) == 127 or char in {"\\", "%"}
|
||||
for char in value
|
||||
):
|
||||
raise ValueError(
|
||||
"COMPANION_BASE_URL contains a forbidden character"
|
||||
)
|
||||
|
||||
try:
|
||||
parsed = urlsplit(value)
|
||||
port = parsed.port
|
||||
except ValueError as exc:
|
||||
raise ValueError("COMPANION_BASE_URL must be a valid HTTP LAN origin") from exc
|
||||
|
||||
host = parsed.hostname
|
||||
if parsed.scheme.lower() != "http" or not parsed.netloc or not host:
|
||||
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
|
||||
if parsed.username is not None or parsed.password is not None:
|
||||
raise ValueError("COMPANION_BASE_URL must not contain credentials")
|
||||
if parsed.path or parsed.query or parsed.fragment:
|
||||
raise ValueError("COMPANION_BASE_URL must not contain a path, query, or fragment")
|
||||
if port is not None and not 1 <= port <= 65535:
|
||||
raise ValueError("COMPANION_BASE_URL port must be between 1 and 65535")
|
||||
if not _valid_companion_client_host(host):
|
||||
raise ValueError("COMPANION_BASE_URL host is not supported by companion v1")
|
||||
|
||||
netloc = f"{host}:{port}" if port is not None else host
|
||||
origin = f"http://{netloc}"
|
||||
if value != origin:
|
||||
raise ValueError("COMPANION_BASE_URL must be a canonical HTTP LAN origin")
|
||||
return host, port or 80
|
||||
|
||||
|
||||
def configured_companion_origin() -> tuple[str, int] | None:
|
||||
"""Return the validated operator-configured v1 address, if any."""
|
||||
value = os.environ.get("COMPANION_BASE_URL")
|
||||
if value is None or value == "":
|
||||
return None
|
||||
return parse_companion_base_url(value)
|
||||
|
||||
|
||||
def default_port() -> int:
|
||||
"""Best guess at the port the server is reachable on. Callers that know the
|
||||
real request port should pass it explicitly."""
|
||||
|
||||
@@ -5,8 +5,9 @@ offers and pair to it, without duplicating any LLM logic.
|
||||
|
||||
Auth is enforced globally by AuthMiddleware (app.py), so reaching a handler here
|
||||
means the caller is authenticated by either a cookie session or a Bearer `ody_`
|
||||
API token. The read endpoints (ping/info/models) accept either; the pairing
|
||||
endpoints are admin-cookie only.
|
||||
API token. Ping/info accept either credential type, models requires a chat-
|
||||
scoped API token for bearer callers, and the pairing endpoints are admin-cookie
|
||||
only.
|
||||
|
||||
Pairing CSRF posture: minting happens ONLY on POST. The session cookie is
|
||||
SameSite=Lax (routes/auth_routes.py), which a browser does not send on a
|
||||
@@ -18,11 +19,11 @@ on a GET would be unsafe (Lax cookies ride top-level GET navigations), so GET
|
||||
|
||||
import html
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from core.middleware import require_admin
|
||||
from src.auth_helpers import get_current_user
|
||||
from src.auth_helpers import _auth_disabled, get_current_user
|
||||
|
||||
from companion import pairing as _pairing
|
||||
|
||||
@@ -52,6 +53,18 @@ def owner_can_see(row_owner, owner) -> bool:
|
||||
return row_owner is None or row_owner == owner
|
||||
|
||||
|
||||
def require_models_scope(request: Request) -> None:
|
||||
"""Require the companion chat scope for bearer-token model inventory."""
|
||||
if not getattr(request.state, "api_token", False):
|
||||
return
|
||||
scopes = getattr(request.state, "api_token_scopes", None) or []
|
||||
if isinstance(scopes, str):
|
||||
scopes = [scope.strip() for scope in scopes.split(",")]
|
||||
scope_set = {str(scope).strip() for scope in scopes if str(scope).strip()}
|
||||
if _pairing.COMPANION_SCOPE not in scope_set:
|
||||
raise HTTPException(403, "API token requires chat scope")
|
||||
|
||||
|
||||
def mint_pairing_token(owner: str, invalidate=None) -> tuple[str, str]:
|
||||
"""Mint a pairing token AND invalidate the auth middleware's in-memory token
|
||||
cache, so the new token is accepted on the very next request without a server
|
||||
@@ -100,15 +113,22 @@ def setup_companion_routes() -> APIRouter:
|
||||
The stock /api/models route scopes to get_current_user, which for a
|
||||
bearer token is the sandboxed pseudo-user "api" (owns nothing). Here we
|
||||
scope to the token's real owner instead, plus legacy null-owner shared
|
||||
rows -- the same rule as owner_filter. Read-only; never returns api_key
|
||||
material.
|
||||
rows -- the same rule as owner_filter. Explicit auth-disabled mode keeps
|
||||
the stock route's single-user all-endpoints view. Read-only; never
|
||||
returns api_key material.
|
||||
"""
|
||||
require_models_scope(request)
|
||||
import json as _json
|
||||
|
||||
from core.database import SessionLocal, ModelEndpoint
|
||||
from src.endpoint_resolver import build_chat_url
|
||||
|
||||
owner = token_owner(request)
|
||||
single_user_mode = (
|
||||
owner is None
|
||||
and not getattr(request.state, "api_token", False)
|
||||
and _auth_disabled()
|
||||
)
|
||||
out = []
|
||||
db = SessionLocal()
|
||||
try:
|
||||
@@ -119,7 +139,7 @@ def setup_companion_routes() -> APIRouter:
|
||||
if owner:
|
||||
q = q.filter((ModelEndpoint.owner == owner) | (ModelEndpoint.owner == None)) # noqa: E711
|
||||
for ep in q.all():
|
||||
if not owner_can_see(ep.owner, owner):
|
||||
if not single_user_mode and not owner_can_see(ep.owner, owner):
|
||||
continue
|
||||
try:
|
||||
model_ids = _json.loads(ep.cached_models) if ep.cached_models else []
|
||||
@@ -180,19 +200,27 @@ def setup_companion_routes() -> APIRouter:
|
||||
the code works immediately, no restart. `?format=json` returns the
|
||||
payload for an in-app pairing screen."""
|
||||
require_admin(request)
|
||||
try:
|
||||
configured_origin = _pairing.configured_companion_origin()
|
||||
except ValueError as exc:
|
||||
raise HTTPException(500, str(exc)) from None
|
||||
owner = get_current_user(request)
|
||||
invalidate = getattr(request.app.state, "invalidate_token_cache", None)
|
||||
token_id, raw_token = mint_pairing_token(owner, invalidate)
|
||||
|
||||
hosts = _pairing.lan_ip_candidates()
|
||||
host = hosts[0] if hosts else "127.0.0.1"
|
||||
port = request.url.port or _pairing.default_port()
|
||||
if configured_origin:
|
||||
host, port = configured_origin
|
||||
hosts = [host]
|
||||
else:
|
||||
hosts = _pairing.lan_ip_candidates()
|
||||
host = hosts[0] if hosts else "127.0.0.1"
|
||||
port = request.url.port or _pairing.default_port()
|
||||
payload = _pairing.pairing_payload(host, port, raw_token)
|
||||
qr = _pairing.pairing_qr_png_data_uri(payload)
|
||||
qr_ok = bool(qr and qr.startswith("data:image/png;base64,"))
|
||||
|
||||
if (request.query_params.get("format") or "").lower() == "json":
|
||||
return {
|
||||
response = {
|
||||
"host": host,
|
||||
"port": port,
|
||||
"token": raw_token,
|
||||
@@ -201,6 +229,7 @@ def setup_companion_routes() -> APIRouter:
|
||||
"payload": payload,
|
||||
"qr": qr if qr_ok else None,
|
||||
}
|
||||
return response
|
||||
|
||||
import json as _json
|
||||
payload_json = _json.dumps(payload, separators=(",", ":"))
|
||||
|
||||
@@ -15,29 +15,53 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import uuid
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
def atomic_write_json(path: str, data: Any, *, indent: Optional[int] = None) -> None:
|
||||
"""Atomically persist `data` as JSON at `path`.
|
||||
|
||||
The temp file uses the live PID as a suffix so two processes saving the
|
||||
same file (e.g. unit tests) don't collide on the rename target.
|
||||
The temp file uses a random suffix so two concurrent writers saving the
|
||||
same file don't collide on the rename target. A PID suffix does not do
|
||||
this: the PID is constant for the life of a process, so two writers on
|
||||
the same path within one process (or one single-process container, where
|
||||
the PID never changes at all) still race for the same temp file.
|
||||
"""
|
||||
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
|
||||
tmp = f"{path}.tmp.{os.getpid()}"
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=indent)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
tmp = f"{path}.tmp.{uuid.uuid4().hex}"
|
||||
|
||||
try:
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=indent)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
finally:
|
||||
# Directly unlink to avoid a check-then-act race condition.
|
||||
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def atomic_write_text(path: str, text: str) -> None:
|
||||
if not isinstance(text, str):
|
||||
raise TypeError("atomic_write_text expects a string")
|
||||
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
|
||||
tmp = f"{path}.tmp.{os.getpid()}"
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
f.write(text)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
tmp = f"{path}.tmp.{uuid.uuid4().hex}"
|
||||
|
||||
try:
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
f.write(text)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
finally:
|
||||
# Directly unlink to avoid a check-then-act race condition.
|
||||
# Swallows FileNotFoundError (on success path) and other cleanup OSErrors.
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
@@ -3,6 +3,7 @@ Authentication module — multi-user password hashing, session tokens, config pe
|
||||
Config stored in data/auth.json. Uses bcrypt directly.
|
||||
"""
|
||||
|
||||
import enum
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
@@ -46,25 +47,27 @@ ADMIN_PRIVILEGES["allowed_models_restricted"] = False
|
||||
# backwards for this sentinel.
|
||||
ADMIN_PRIVILEGES["block_all_models"] = False
|
||||
|
||||
from src.constants import AUTH_FILE
|
||||
from src.constants import AUTH_FILE, PASSWORD_MIN_LENGTH
|
||||
from src.owner_identity import RESERVED_AUTH_USERNAMES
|
||||
DEFAULT_AUTH_PATH = AUTH_FILE
|
||||
TOKEN_TTL = 60 * 60 * 24 * 7 # 7 days
|
||||
|
||||
# Usernames the auth + middleware layer reserve as internal "synthetic owner"
|
||||
# sentinels; they must never belong to a real account. The most dangerous is
|
||||
# "internal-tool": `core.middleware.require_admin` treats any request whose
|
||||
# `current_user == "internal-tool"` as the in-process tool loopback and grants
|
||||
# admin, and because the cookie auth path sets `current_user` to the raw
|
||||
# username, an account literally named "internal-tool" would be silently
|
||||
# treated as an admin by every `require_admin`-gated route. "api" collides with
|
||||
# the bearer-token owner-attribution sentinel. "demo"/"system" round out the
|
||||
# synthetic-owner set the rest of the codebase already special-cases (see
|
||||
# `_SYNTHETIC_OWNERS` in routes/assistant_routes.py and the matching guards in
|
||||
# src/task_scheduler.py / routes/research_routes.py) — a real account with one
|
||||
# of those names would be denied an assistant and inconsistently owner-scoped.
|
||||
# Refuse to create or rename into any of them so the sentinels can't be
|
||||
# impersonated. (Keep this in sync with that synthetic-owner set.)
|
||||
RESERVED_USERNAMES = frozenset({"internal-tool", "api", "demo", "system"})
|
||||
# Usernames the auth + middleware layer reserves for request sentinels and
|
||||
# internal storage owners; they must never belong to a real login account.
|
||||
# "internal-tool" is the most dangerous because `core.middleware.require_admin`
|
||||
# treats it as the in-process tool loopback. "api" collides with bearer-token
|
||||
# attribution. "demo"/"system" are synthetic owners already special-cased by
|
||||
# scheduler/assistant/research paths. The Default/Local owner is a storage
|
||||
# bucket for explicit auth-disabled no-login mode, not a login username.
|
||||
RESERVED_USERNAMES = frozenset(RESERVED_AUTH_USERNAMES)
|
||||
|
||||
|
||||
def normalize_known_username(users: Dict[str, Any], username: str | None) -> Optional[str]:
|
||||
"""Return a normalized username only when it exists in the auth user map."""
|
||||
key = str(username or "").strip().lower()
|
||||
if not key or key not in users:
|
||||
return None
|
||||
return key
|
||||
|
||||
|
||||
def _hash_password(password: str) -> str:
|
||||
@@ -75,6 +78,15 @@ def _verify_password(password: str, hashed: str) -> bool:
|
||||
return bcrypt.checkpw(password.encode("utf-8"), hashed.encode("utf-8"))
|
||||
|
||||
|
||||
class SetAdminResult(enum.Enum):
|
||||
"""Outcome of AuthManager.set_admin, so callers can map each case to a
|
||||
precise response instead of guessing from a bare bool."""
|
||||
OK = "ok"
|
||||
USER_NOT_FOUND = "user_not_found"
|
||||
NOT_AUTHORIZED = "not_authorized" # requester is not an admin
|
||||
LAST_ADMIN = "last_admin" # would remove the last remaining admin
|
||||
|
||||
|
||||
class AuthManager:
|
||||
"""Manages multi-user password + session-token auth system."""
|
||||
|
||||
@@ -96,6 +108,7 @@ class AuthManager:
|
||||
self._load()
|
||||
self._load_sessions()
|
||||
self._migrate_single_user()
|
||||
self._drop_reserved_loaded_users()
|
||||
self._migrate_legacy_admin_role()
|
||||
|
||||
def _load(self):
|
||||
@@ -148,20 +161,52 @@ class AuthManager:
|
||||
def _migrate_single_user(self):
|
||||
"""Migrate old single-user format to multi-user format."""
|
||||
if "password_hash" in self._config and "users" not in self._config:
|
||||
old_user = self._config.get("username", "admin")
|
||||
old_user = str(self._config.get("username", "admin") or "admin").strip().lower()
|
||||
if old_user in RESERVED_USERNAMES:
|
||||
logger.warning(
|
||||
"Migrating legacy single-user reserved username '%s' to 'admin'",
|
||||
old_user,
|
||||
)
|
||||
old_user = "admin"
|
||||
old_hash = self._config["password_hash"]
|
||||
self._config = {
|
||||
"users": {
|
||||
old_user: {
|
||||
"password_hash": old_hash,
|
||||
"created": time.time(),
|
||||
"is_admin": True,
|
||||
with self._config_lock:
|
||||
self._config = {
|
||||
"users": {
|
||||
old_user: {
|
||||
"password_hash": old_hash,
|
||||
"created": time.time(),
|
||||
"is_admin": True,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
self._save()
|
||||
self._save()
|
||||
logger.info(f"Migrated single-user auth to multi-user (admin: {old_user})")
|
||||
|
||||
def _drop_reserved_loaded_users(self):
|
||||
"""Fail closed for legacy/manual auth rows that collide with sentinels."""
|
||||
users = self._config.get("users")
|
||||
if not isinstance(users, dict):
|
||||
return
|
||||
normalized = {}
|
||||
removed = []
|
||||
for username, data in users.items():
|
||||
key = str(username or "").strip().lower()
|
||||
if not key:
|
||||
continue
|
||||
if key in RESERVED_USERNAMES:
|
||||
removed.append(key)
|
||||
continue
|
||||
normalized[key] = data
|
||||
if removed or normalized != users:
|
||||
with self._config_lock:
|
||||
self._config["users"] = normalized
|
||||
self._save()
|
||||
if removed:
|
||||
logger.warning(
|
||||
"Removed reserved username(s) from auth config: %s",
|
||||
", ".join(sorted(set(removed))),
|
||||
)
|
||||
|
||||
def _migrate_legacy_admin_role(self):
|
||||
"""Normalize setup.py's old role='admin' marker to is_admin=True."""
|
||||
changed = False
|
||||
@@ -194,6 +239,15 @@ class AuthManager:
|
||||
def is_configured(self) -> bool:
|
||||
return len(self.users) > 0
|
||||
|
||||
def policy(self) -> dict:
|
||||
"""Return public auth policy constants for the frontend."""
|
||||
return {
|
||||
"password_min_length": PASSWORD_MIN_LENGTH,
|
||||
"reserved_usernames": sorted(RESERVED_USERNAMES),
|
||||
"signup_enabled": self.signup_enabled,
|
||||
"session_days": TOKEN_TTL // 86400,
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Account management
|
||||
# ------------------------------------------------------------------
|
||||
@@ -244,6 +298,22 @@ class AuthManager:
|
||||
return False
|
||||
if not self.users.get(requesting_user, {}).get("is_admin"):
|
||||
return False
|
||||
# Revoke API bearer tokens before removing the auth row. The bearer
|
||||
# path authenticates from ApiToken rows and does not require the
|
||||
# owner to still exist, so a successful delete must not leave active
|
||||
# rows behind. If the token store is unavailable, fail closed and
|
||||
# keep the user/session state intact so the admin can retry.
|
||||
try:
|
||||
from core.database import get_db_session, ApiToken
|
||||
with get_db_session() as db:
|
||||
removed_tokens = db.query(ApiToken).filter(ApiToken.owner == username).delete()
|
||||
if removed_tokens:
|
||||
logger.info(
|
||||
f"Revoked {removed_tokens} API token(s) owned by deleted user '{username}'"
|
||||
)
|
||||
except Exception:
|
||||
logger.warning(f"Failed to revoke API tokens for deleted user '{username}'")
|
||||
return False
|
||||
del self._config["users"][username]
|
||||
self._save()
|
||||
# Purge all sessions belonging to this user. validate_token doesn't
|
||||
@@ -258,18 +328,6 @@ class AuthManager:
|
||||
revoked += 1
|
||||
if revoked:
|
||||
self._save_sessions()
|
||||
# Also revoke API bearer tokens owned by this user. The bearer auth
|
||||
# path authenticates straight against ApiToken rows and never
|
||||
# re-checks that the owner still exists, so leaving the rows behind
|
||||
# would let a deleted user keep full API access indefinitely.
|
||||
try:
|
||||
from core.database import get_db_session, ApiToken
|
||||
with get_db_session() as db:
|
||||
removed = db.query(ApiToken).filter(ApiToken.owner == username).delete()
|
||||
if removed:
|
||||
logger.info(f"Revoked {removed} API token(s) owned by deleted user '{username}'")
|
||||
except Exception:
|
||||
logger.warning(f"Failed to revoke API tokens for deleted user '{username}'")
|
||||
logger.info(f"Deleted user '{username}' (by {requesting_user}); revoked {revoked} active session(s)")
|
||||
return True
|
||||
|
||||
@@ -344,6 +402,69 @@ class AuthManager:
|
||||
logger.info(f"Updated privileges for '{username}': {current}")
|
||||
return True
|
||||
|
||||
def set_admin(self, username: str, is_admin: bool,
|
||||
requesting_user: str) -> SetAdminResult:
|
||||
"""Promote/demote an existing user to/from admin. Admin only.
|
||||
|
||||
Refuses to remove the last remaining admin so the instance can never
|
||||
be locked out of admin access; self-demotion is allowed as long as
|
||||
another admin remains. Admin status is re-checked live on every
|
||||
request, so unlike delete/rename no session or token revocation is
|
||||
needed — a demoted admin simply fails the next is_admin() gate.
|
||||
|
||||
Promotion stashes the user's current privilege map and demotion
|
||||
restores it, so a temporary admin stint can't silently broaden a
|
||||
user's non-admin access; users without a stash (created as admin,
|
||||
or promoted before stashing existed) demote to DEFAULT_PRIVILEGES.
|
||||
|
||||
Counting admins and flipping the flag happen in one critical section
|
||||
so two concurrent demotions can't race the admin count to zero.
|
||||
"""
|
||||
username = (username or "").strip().lower()
|
||||
requesting_user = (requesting_user or "").strip().lower()
|
||||
is_admin = bool(is_admin)
|
||||
with self._config_lock:
|
||||
target = self._config.get("users", {}).get(username)
|
||||
if target is None:
|
||||
return SetAdminResult.USER_NOT_FOUND
|
||||
if not self.users.get(requesting_user, {}).get("is_admin"):
|
||||
return SetAdminResult.NOT_AUTHORIZED
|
||||
currently_admin = bool(target.get("is_admin"))
|
||||
if currently_admin == is_admin:
|
||||
return SetAdminResult.OK # no-op; leave privileges untouched
|
||||
if currently_admin and not is_admin:
|
||||
admin_count = sum(1 for d in self.users.values() if d.get("is_admin"))
|
||||
if admin_count <= 1:
|
||||
return SetAdminResult.LAST_ADMIN
|
||||
# Write order matters for lock-free readers: get_privileges()
|
||||
# reads without _config_lock and trusts is_admin, so the admin
|
||||
# flag must be flipped while the stored map is safe to expose —
|
||||
# before writing admin privileges on promote, after restoring
|
||||
# the pre-admin map on demote.
|
||||
if is_admin:
|
||||
target["is_admin"] = True
|
||||
# Stash the pre-admin map so a later demotion can restore it.
|
||||
# While is_admin is set the stored map is inert: get_privileges
|
||||
# short-circuits to ADMIN_PRIVILEGES and set_privileges refuses
|
||||
# admins, so only set_admin ever touches the stash.
|
||||
target["privileges_before_admin"] = dict(
|
||||
target.get("privileges") or DEFAULT_PRIVILEGES
|
||||
)
|
||||
target["privileges"] = dict(ADMIN_PRIVILEGES)
|
||||
else:
|
||||
# Restore the stashed pre-admin map. Fall back to defaults for
|
||||
# users created as admins (their stored map is ADMIN_PRIVILEGES,
|
||||
# which must not leak past demotion — e.g. can_use_bash) and
|
||||
# for admins promoted before the stash existed.
|
||||
target["privileges"] = dict(
|
||||
target.pop("privileges_before_admin", None)
|
||||
or DEFAULT_PRIVILEGES
|
||||
)
|
||||
target["is_admin"] = False
|
||||
self._save()
|
||||
logger.info("Set is_admin=%s for '%s' (by '%s')", is_admin, username, requesting_user)
|
||||
return SetAdminResult.OK
|
||||
|
||||
def change_password(self, username: str, current_password: str, new_password: str) -> bool:
|
||||
username = username.strip().lower()
|
||||
if username not in self.users:
|
||||
@@ -457,16 +578,20 @@ class AuthManager:
|
||||
return None
|
||||
return self.create_session_trusted(username)
|
||||
|
||||
def create_session_trusted(self, username: str) -> str:
|
||||
def create_session_trusted(self, username: str) -> Optional[str]:
|
||||
"""Issue a session token for an already-verified user.
|
||||
Call only after verify_password (and TOTP if enabled) have passed."""
|
||||
username = username.strip().lower()
|
||||
token = secrets.token_hex(32)
|
||||
with self._sessions_lock:
|
||||
self._sessions[token] = {
|
||||
"username": username,
|
||||
"expiry": time.time() + TOKEN_TTL,
|
||||
}
|
||||
with self._config_lock:
|
||||
if username not in self.users:
|
||||
logger.warning("Refused to issue session for missing user '%s'", username)
|
||||
return None
|
||||
with self._sessions_lock:
|
||||
self._sessions[token] = {
|
||||
"username": username,
|
||||
"expiry": time.time() + TOKEN_TTL,
|
||||
}
|
||||
self._save_sessions()
|
||||
return token
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# src/exceptions.py
|
||||
# core/exceptions.py
|
||||
"""Custom exceptions for the application."""
|
||||
|
||||
class SessionNotFoundError(Exception):
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Helpers for keeping sensitive data out of logs.
|
||||
|
||||
Endpoint URLs configured by admins can embed credentials in the userinfo
|
||||
(``https://user:pass@host``) or query string (``?api_key=...``). Logging them
|
||||
raw leaks those secrets, so route/diagnostic logs run URLs through
|
||||
``redact_url`` first. Reconstructing the URL without userinfo/query/fragment
|
||||
also doubles as a sanitizer barrier for CodeQL's clear-text-logging query.
|
||||
"""
|
||||
|
||||
from urllib.parse import urlparse, urlunparse
|
||||
|
||||
|
||||
def redact_url(url: str) -> str:
|
||||
"""Return a URL safe for logs by removing userinfo and query/fragment.
|
||||
|
||||
Keeps scheme, host, port and path so logs stay useful for debugging.
|
||||
"""
|
||||
try:
|
||||
parsed = urlparse(url or "")
|
||||
host = parsed.hostname or ""
|
||||
if ":" in host: # IPv6 literal — re-bracket so host:port stays unambiguous
|
||||
host = f"[{host}]"
|
||||
if parsed.port:
|
||||
host = f"{host}:{parsed.port}"
|
||||
return urlunparse((parsed.scheme, host, parsed.path, "", "", ""))
|
||||
except Exception:
|
||||
return "<endpoint>"
|
||||
@@ -3,10 +3,14 @@
|
||||
|
||||
import os
|
||||
import secrets
|
||||
from collections.abc import Mapping
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.responses import Response
|
||||
from starlette.routing import get_route_path
|
||||
|
||||
from src.owner_identity import INTERNAL_TOOL_USER, auth_disabled
|
||||
|
||||
|
||||
# Per-process token that lets the in-app tool layer hit admin-gated
|
||||
@@ -17,6 +21,30 @@ INTERNAL_TOOL_TOKEN = os.environ.get("ODYSSEUS_INTERNAL_TOKEN") or secrets.token
|
||||
INTERNAL_TOOL_HEADER = "X-Odysseus-Internal-Token"
|
||||
|
||||
|
||||
def get_application_route_path(scope: Mapping[str, object]) -> str:
|
||||
"""Return the application-relative path used by Starlette routing.
|
||||
|
||||
Uvicorn prefixes ``scope["path"]`` with a configured ASGI ``root_path``;
|
||||
Starlette removes that prefix before matching routes. Middleware policy
|
||||
must use the same path form or a deployment prefix can change which policy
|
||||
applies to an otherwise unchanged application route.
|
||||
"""
|
||||
return get_route_path(scope)
|
||||
|
||||
|
||||
def with_asgi_root_path(scope: Mapping[str, object], path: str) -> str:
|
||||
"""Prefix an application path for a client-facing redirect target."""
|
||||
root_path = scope.get("root_path", "")
|
||||
if not isinstance(root_path, str) or not root_path:
|
||||
return path
|
||||
return f"{root_path.rstrip('/')}{path}"
|
||||
|
||||
|
||||
def path_is_route_or_child(path: str, prefix: str) -> bool:
|
||||
"""Return whether ``path`` is exactly ``prefix`` or below that route."""
|
||||
return path == prefix or path.startswith(prefix + "/")
|
||||
|
||||
|
||||
def is_cors_preflight(method: str, headers) -> bool:
|
||||
"""True for a genuine CORS preflight: an OPTIONS request carrying the
|
||||
Access-Control-Request-Method header. Such requests are credential-less by
|
||||
@@ -39,13 +67,13 @@ def require_admin(request: Request):
|
||||
hdr = request.headers.get(INTERNAL_TOOL_HEADER)
|
||||
if hdr and secrets.compare_digest(hdr, INTERNAL_TOOL_TOKEN):
|
||||
return
|
||||
if getattr(request.state, "current_user", None) == "internal-tool":
|
||||
if getattr(request.state, "current_user", None) == INTERNAL_TOOL_USER:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
auth_mgr = getattr(request.app.state, "auth_manager", None)
|
||||
if os.getenv("AUTH_ENABLED", "true").lower() == "false":
|
||||
if auth_disabled():
|
||||
return
|
||||
if not auth_mgr or not auth_mgr.is_configured:
|
||||
raise HTTPException(403, "Admin only")
|
||||
@@ -65,10 +93,9 @@ class SecurityHeadersMiddleware(BaseHTTPMiddleware):
|
||||
response = await call_next(request)
|
||||
path = request.url.path
|
||||
|
||||
# Tool render endpoints are served inside iframes — allow framing by self
|
||||
# Tool render endpoints
|
||||
is_tool_render = path.startswith("/api/tools/") and path.endswith("/render")
|
||||
# PDF previews are embedded by the in-app document library. Keep the
|
||||
# exception route-scoped so normal app pages remain unframeable.
|
||||
# Document library PDF preview endpoint
|
||||
is_document_pdf_preview = path.startswith("/api/document/") and path.endswith("/render-pdf")
|
||||
# Visual report pages are self-contained HTML — need inline scripts + external images
|
||||
is_report = path.startswith("/api/research/report/")
|
||||
@@ -95,9 +122,7 @@ class SecurityHeadersMiddleware(BaseHTTPMiddleware):
|
||||
"frame-ancestors 'none'"
|
||||
)
|
||||
elif is_tool_render:
|
||||
# Tool iframe content: skip all framing headers — the iframe's
|
||||
# sandbox="allow-scripts" attribute provides isolation.
|
||||
# Don't overwrite the route's own restrictive CSP either.
|
||||
# Skip framing headers for tools.
|
||||
pass
|
||||
elif is_document_pdf_preview:
|
||||
response.headers["X-Frame-Options"] = "SAMEORIGIN"
|
||||
@@ -118,7 +143,7 @@ class SecurityHeadersMiddleware(BaseHTTPMiddleware):
|
||||
f"script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net; "
|
||||
"style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net; "
|
||||
"font-src 'self' https://cdn.jsdelivr.net; "
|
||||
"img-src 'self' data: blob:; "
|
||||
"img-src 'self' data: blob: https:; "
|
||||
"media-src 'self' blob:; "
|
||||
"connect-src 'self'; "
|
||||
"frame-src 'self'; "
|
||||
|
||||
@@ -8,17 +8,61 @@ These are simple datacontainers. All persistence is handled by SessionManager.
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, List, Any, Optional, TYPE_CHECKING
|
||||
|
||||
from src.tool_approval_scopes import (
|
||||
CHAT_SESSION_APPROVAL_CONTEXT_MARKER,
|
||||
CHAT_SESSION_APPROVAL_DECISION,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .session_manager import SessionManager
|
||||
|
||||
# Module-level session manager reference (set at app startup)
|
||||
_session_manager: Optional["SessionManager"] = None
|
||||
# Module-level session manager singleton (single source of truth)
|
||||
_SESSION_MANAGER_INSTANCE: Optional["SessionManager"] = None
|
||||
|
||||
|
||||
def set_session_manager(manager: "SessionManager"):
|
||||
"""Set the global session manager reference."""
|
||||
global _session_manager
|
||||
_session_manager = manager
|
||||
def set_session_manager_instance(manager: "SessionManager"):
|
||||
"""Set the global SessionManager singleton."""
|
||||
global _SESSION_MANAGER_INSTANCE
|
||||
_SESSION_MANAGER_INSTANCE = manager
|
||||
|
||||
|
||||
def get_session_manager_instance() -> Optional["SessionManager"]:
|
||||
"""Get the global SessionManager singleton."""
|
||||
return _SESSION_MANAGER_INSTANCE
|
||||
|
||||
|
||||
# Keep legacy name for backward compatibility
|
||||
set_session_manager = set_session_manager_instance
|
||||
get_session_manager = get_session_manager_instance
|
||||
|
||||
|
||||
def _history_grants_chat_session_approval(
|
||||
history: List["ChatMessage"],
|
||||
session_id: str,
|
||||
) -> bool:
|
||||
"""Return whether this exact chat has a resolved session-scope grant."""
|
||||
|
||||
expected_session = str(session_id or "")
|
||||
if not expected_session:
|
||||
return False
|
||||
for message in reversed(history or []):
|
||||
metadata = getattr(message, "metadata", None)
|
||||
if not isinstance(metadata, dict):
|
||||
continue
|
||||
tool_events = metadata.get("tool_events")
|
||||
if not isinstance(tool_events, list):
|
||||
continue
|
||||
for event in reversed(tool_events):
|
||||
ask_user = event.get("ask_user") if isinstance(event, dict) else None
|
||||
if not isinstance(ask_user, dict):
|
||||
continue
|
||||
if (
|
||||
ask_user.get("kind") == "tool_approval"
|
||||
and ask_user.get("resolved") == CHAT_SESSION_APPROVAL_DECISION
|
||||
and str(ask_user.get("session_id") or "") == expected_session
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -42,7 +86,17 @@ class ChatMessage:
|
||||
|
||||
@dataclass
|
||||
class Session:
|
||||
"""A chat session — pure data container."""
|
||||
"""A chat session — pure data container.
|
||||
|
||||
``.history`` is the authoritative mutable message list. Callers may
|
||||
read, append, pop, or reassign it directly — these changes take
|
||||
effect immediately. ``_history`` remains a compatibility alias that
|
||||
always resolves to the authoritative ``history`` list.
|
||||
|
||||
Each session gets its own unique history list at construction time
|
||||
(the dataclass default is never shared between instances).
|
||||
"""
|
||||
|
||||
id: str
|
||||
name: str
|
||||
endpoint_url: str
|
||||
@@ -54,26 +108,43 @@ class Session:
|
||||
owner: Optional[str] = None
|
||||
is_important: bool = False
|
||||
message_count: int = 0
|
||||
memory_extraction_enabled: bool = True
|
||||
skill_injection_enabled: bool = True
|
||||
thinking_mode: str = "off"
|
||||
temperature_override: Optional[float] = None
|
||||
max_tokens_override: Optional[int] = None
|
||||
cwd: Optional[str] = None
|
||||
|
||||
def __post_init__(self):
|
||||
if self.history is None:
|
||||
self.history = []
|
||||
if self.headers is None:
|
||||
self.headers = {}
|
||||
# Ensure each session gets its OWN list (not the shared dataclass default)
|
||||
if self.history is None:
|
||||
self.history = []
|
||||
|
||||
@property
|
||||
def _history(self) -> List[ChatMessage]:
|
||||
"""Compatibility alias for callers that still reference ``_history``."""
|
||||
return self.history
|
||||
|
||||
@_history.setter
|
||||
def _history(self, messages: List[ChatMessage]):
|
||||
self.history = messages
|
||||
|
||||
def add_message(self, message: ChatMessage):
|
||||
"""
|
||||
Add a message to this session.
|
||||
|
||||
Delegates to SessionManager for persistence if available,
|
||||
otherwise just appends to history.
|
||||
Appends to the authoritative history list and increments
|
||||
message_count. Delegates to SessionManager for persistence
|
||||
if available.
|
||||
"""
|
||||
self.history.append(message)
|
||||
self.message_count = len(self.history)
|
||||
|
||||
# Delegate to session manager for persistence
|
||||
if _session_manager:
|
||||
_session_manager._persist_message(self.id, message)
|
||||
if _SESSION_MANAGER_INSTANCE:
|
||||
_SESSION_MANAGER_INSTANCE._persist_message(self.id, message)
|
||||
|
||||
def get_context_messages(self) -> List[Dict[str, Any]]:
|
||||
"""Get messages in format for LLM API.
|
||||
@@ -85,12 +156,50 @@ class Session:
|
||||
the model. Display/history-load paths use the raw ``history`` and are
|
||||
unaffected.
|
||||
"""
|
||||
return [
|
||||
messages = [
|
||||
msg.to_dict()
|
||||
for msg in self.history
|
||||
if (msg.metadata or {}).get("source") != "slash"
|
||||
]
|
||||
from src.background_tool_jobs import background_result_context
|
||||
messages = [part for message in messages for part in (
|
||||
*background_result_context(message.get('metadata')), message,
|
||||
)]
|
||||
# Resume an interrupted thinking-only response from its actual model
|
||||
# reasoning channel. Restrict this to the latest assistant message so
|
||||
# old traces do not accumulate in context or cause reasoning loops.
|
||||
for index in range(len(messages) - 1, -1, -1):
|
||||
message = messages[index]
|
||||
if message.get("role") != "assistant":
|
||||
continue
|
||||
metadata = message.get("metadata") or {}
|
||||
thinking = str(metadata.get("thinking") or "").strip()
|
||||
if metadata.get("stopped") and thinking:
|
||||
resumed = dict(message)
|
||||
resumed["reasoning_content"] = thinking
|
||||
messages[index] = resumed
|
||||
break
|
||||
if not _history_grants_chat_session_approval(self.history, self.id):
|
||||
return messages
|
||||
|
||||
# Keep the grant close to the latest user request so route-neutral
|
||||
# compaction/trimming preserves it. Copy the metadata instead of
|
||||
# mutating the durable transcript object.
|
||||
for index in range(len(messages) - 1, -1, -1):
|
||||
if messages[index].get("role") != "user":
|
||||
continue
|
||||
message = dict(messages[index])
|
||||
metadata = dict(message.get("metadata") or {})
|
||||
metadata[CHAT_SESSION_APPROVAL_CONTEXT_MARKER] = True
|
||||
message["metadata"] = metadata
|
||||
messages[index] = message
|
||||
break
|
||||
return messages
|
||||
|
||||
def get(self, key: str, default=None):
|
||||
"""Dict-like access for compatibility."""
|
||||
return getattr(self, key, default)
|
||||
|
||||
def __getitem__(self, key: str):
|
||||
"""Allow session['field'] syntax."""
|
||||
return getattr(self, key)
|
||||
|
||||
@@ -191,6 +191,8 @@ def _windows_bash_fallbacks() -> List[str]:
|
||||
base = os.environ.get(env_name)
|
||||
if base:
|
||||
roots.append(ntpath.join(base, "Git"))
|
||||
if env_name == "LocalAppData":
|
||||
roots.append(ntpath.join(base, "Programs", "Git"))
|
||||
roots.extend(_WINDOWS_BASH_DEFAULT_ROOTS)
|
||||
|
||||
paths: List[str] = []
|
||||
@@ -298,7 +300,7 @@ def is_wsl() -> bool:
|
||||
import sys
|
||||
if sys.platform.startswith("linux") or os.name == "posix":
|
||||
try:
|
||||
with open("/proc/version", "r") as f:
|
||||
with open("/proc/version", "r", encoding="utf-8", errors="ignore") as f:
|
||||
if "microsoft" in f.read().lower():
|
||||
return True
|
||||
except Exception:
|
||||
@@ -366,6 +368,10 @@ def _ssh_exec_argv(
|
||||
strict_host_key_checking: bool | None = None,
|
||||
) -> list[str]:
|
||||
"""Build a consistent ssh argv for remote command execution."""
|
||||
remote_value = str(remote or "").strip()
|
||||
remote_host = remote_value.rsplit("@", 1)[-1]
|
||||
if not remote_value or remote_value.startswith("-") or not remote_host or remote_host.startswith("-"):
|
||||
raise ValueError("Invalid SSH remote host")
|
||||
argv = ["ssh"]
|
||||
if connect_timeout is not None:
|
||||
argv.extend(["-o", f"ConnectTimeout={int(connect_timeout)}"])
|
||||
|
||||
@@ -14,8 +14,15 @@ import logging
|
||||
from datetime import datetime, timezone, timedelta
|
||||
from typing import Dict, Optional
|
||||
|
||||
from sqlalchemy import func
|
||||
|
||||
from .database import Session as DbSession, ChatMessage as DbChatMessage, Document as DbDocument, SessionLocal, utcnow_naive
|
||||
from .models import Session, ChatMessage
|
||||
from src.attachment_refs import persistable_message_content
|
||||
from src.upload_handler import reserve_message_upload_references
|
||||
|
||||
# Re-export singleton accessors from models for convenience
|
||||
from .models import set_session_manager_instance, get_session_manager_instance
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -37,7 +44,18 @@ def _parse_msg_content(raw):
|
||||
if isinstance(raw, str) and raw.startswith('[{') and '"type"' in raw:
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
if isinstance(parsed, list) and all(isinstance(p, dict) for p in parsed):
|
||||
# Only treat as serialized multimodal content when EVERY element is
|
||||
# a dict whose "type" is a recognized content-block kind. Otherwise a
|
||||
# plain text message that merely *looks* like a JSON array of objects
|
||||
# (e.g. a user pasting an API schema/sample with a "type" field) was
|
||||
# silently parsed back into a list, destroying the original string.
|
||||
_BLOCK_TYPES = {
|
||||
"text", "image", "image_url", "audio", "input_audio",
|
||||
"input_image", "document", "file",
|
||||
}
|
||||
if (isinstance(parsed, list) and parsed
|
||||
and all(isinstance(p, dict) and p.get("type") in _BLOCK_TYPES
|
||||
for p in parsed)):
|
||||
return parsed
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
pass
|
||||
@@ -58,6 +76,7 @@ class SessionManager:
|
||||
def __init__(self, sessions_file: str = None):
|
||||
# sessions_file kept for backward compat, not used
|
||||
self.sessions: Dict[str, Session] = {}
|
||||
self.upload_handler = None
|
||||
self.load_sessions()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -75,14 +94,28 @@ class SessionManager:
|
||||
try:
|
||||
db_sessions = db.query(DbSession).filter(
|
||||
DbSession.archived == False,
|
||||
DbSession.message_count > 0,
|
||||
DbSession.messages.any(),
|
||||
).order_by(DbSession.last_accessed.desc()).limit(100).all()
|
||||
|
||||
# message_count is derived metadata and can drift after interrupted
|
||||
# or legacy writes. Count only the bounded discovery set so startup
|
||||
# remains metadata-only while lazy hydration sees an authoritative
|
||||
# positive count for every discovered non-empty session.
|
||||
message_counts = {}
|
||||
if db_sessions:
|
||||
message_counts = dict(
|
||||
db.query(DbChatMessage.session_id, func.count(DbChatMessage.id))
|
||||
.filter(DbChatMessage.session_id.in_([row.id for row in db_sessions]))
|
||||
.group_by(DbChatMessage.session_id)
|
||||
.all()
|
||||
)
|
||||
|
||||
loaded_count = 0
|
||||
for db_session in db_sessions:
|
||||
try:
|
||||
session = self._db_to_session_meta(db_session)
|
||||
if session is not None:
|
||||
session.message_count = message_counts[db_session.id]
|
||||
self.sessions[db_session.id] = session
|
||||
loaded_count += 1
|
||||
except Exception as e:
|
||||
@@ -117,6 +150,12 @@ class SessionManager:
|
||||
history=[],
|
||||
owner=getattr(db_session, "owner", None),
|
||||
is_important=getattr(db_session, "is_important", False) or False,
|
||||
memory_extraction_enabled=getattr(db_session, "memory_extraction_enabled", True) is not False,
|
||||
skill_injection_enabled=getattr(db_session, "skill_injection_enabled", True) is not False,
|
||||
thinking_mode=getattr(db_session, "thinking_mode", "") or "off",
|
||||
temperature_override=getattr(db_session, "temperature_override", None),
|
||||
max_tokens_override=getattr(db_session, "max_tokens_override", None),
|
||||
cwd=getattr(db_session, "cwd", None) or None,
|
||||
)
|
||||
session.message_count = getattr(db_session, "message_count", 0) or 0
|
||||
return session
|
||||
@@ -175,9 +214,20 @@ class SessionManager:
|
||||
history=history,
|
||||
owner=getattr(db_session, 'owner', None),
|
||||
is_important=getattr(db_session, 'is_important', False) or False,
|
||||
memory_extraction_enabled=getattr(db_session, 'memory_extraction_enabled', True) is not False,
|
||||
skill_injection_enabled=getattr(db_session, 'skill_injection_enabled', True) is not False,
|
||||
thinking_mode=getattr(db_session, "thinking_mode", "") or "off",
|
||||
temperature_override=getattr(db_session, "temperature_override", None),
|
||||
max_tokens_override=getattr(db_session, "max_tokens_override", None),
|
||||
cwd=getattr(db_session, "cwd", None) or None,
|
||||
)
|
||||
|
||||
session.message_count = getattr(db_session, 'message_count', len(history))
|
||||
# The rows just loaded are the whole transcript, so they — not the
|
||||
# denormalized sessions.message_count column — are the truth for this
|
||||
# cached object. get_session's hydration gate compares against this
|
||||
# number; seeding it from a drifted column would ask for a reload that
|
||||
# can never close the gap.
|
||||
session.message_count = len(history)
|
||||
return session
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -188,12 +238,17 @@ class SessionManager:
|
||||
"""
|
||||
Add a message to a session and persist to database.
|
||||
|
||||
Updates the authoritative history list and persists through this
|
||||
manager directly so tests and temporary managers do not depend on the
|
||||
process-wide session-manager singleton.
|
||||
|
||||
Args:
|
||||
session_id: Session ID
|
||||
message: ChatMessage to add
|
||||
"""
|
||||
session = self.get_session(session_id)
|
||||
session.history.append(message)
|
||||
session._history = session.history
|
||||
session.message_count = len(session.history)
|
||||
|
||||
self._persist_message(session_id, message)
|
||||
@@ -211,17 +266,26 @@ class SessionManager:
|
||||
logger.warning("Dropping message for deleted session %s", session_id)
|
||||
return
|
||||
|
||||
missing_upload_id = reserve_message_upload_references(
|
||||
getattr(self, "upload_handler", None),
|
||||
getattr(db_session, "owner", None),
|
||||
message.content,
|
||||
message.metadata,
|
||||
)
|
||||
if missing_upload_id:
|
||||
raise ValueError(
|
||||
f"Referenced upload is no longer available: {missing_upload_id}"
|
||||
)
|
||||
|
||||
msg_id = str(uuid.uuid4())
|
||||
msg_time = datetime.utcnow()
|
||||
if message.metadata is None:
|
||||
message.metadata = {}
|
||||
message.metadata.setdefault('timestamp', _message_timestamp_iso(msg_time))
|
||||
# Multimodal content (image/audio attachments) is a list — serialize
|
||||
# to JSON so the Text column can store it. On reload, _db_to_session
|
||||
# detects the JSON-array prefix and parses it back.
|
||||
_content = message.content
|
||||
if isinstance(_content, list):
|
||||
_content = json.dumps(_content)
|
||||
# Multimodal content may contain provider data URLs for the live
|
||||
# model call. Persist only readable text plus attachment references
|
||||
# so chat_messages/FTS do not duplicate upload bytes.
|
||||
_content = persistable_message_content(message.content, message.metadata)
|
||||
db_message = DbChatMessage(
|
||||
id=msg_id,
|
||||
session_id=session_id,
|
||||
@@ -232,7 +296,10 @@ class SessionManager:
|
||||
)
|
||||
db.add(db_message)
|
||||
|
||||
db_session.message_count = len(self.sessions.get(session_id, {}).history) if session_id in self.sessions else 0
|
||||
if session_id in self.sessions:
|
||||
db_session.message_count = len(self.sessions[session_id].history)
|
||||
else:
|
||||
db_session.message_count = 0
|
||||
_now = datetime.now(timezone.utc)
|
||||
db_session.last_accessed = _now
|
||||
# Clean "last conversation" timestamp — only bumped here on a
|
||||
@@ -283,6 +350,7 @@ class SessionManager:
|
||||
|
||||
# Update in-memory
|
||||
session.history = session.history[:keep_count]
|
||||
session._history = session.history
|
||||
|
||||
logger.info(f"Truncated session {session_id} to {keep_count} messages")
|
||||
return True
|
||||
@@ -299,6 +367,28 @@ class SessionManager:
|
||||
session = self.get_session(session_id)
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
|
||||
if db_session is None:
|
||||
logger.warning("Cannot replace history for missing session %s", session_id)
|
||||
return False
|
||||
|
||||
# Reserve every incoming attachment before removing any durable
|
||||
# message row. reserve_upload() shares the upload lifecycle lock
|
||||
# with cleanup, so an upload cannot be deleted between this
|
||||
# ownership check/access touch and the replacement transaction.
|
||||
# A failed reservation must leave the existing transcript intact.
|
||||
for message in messages:
|
||||
missing_upload_id = reserve_message_upload_references(
|
||||
getattr(self, "upload_handler", None),
|
||||
getattr(db_session, "owner", None),
|
||||
message.content,
|
||||
message.metadata,
|
||||
)
|
||||
if missing_upload_id:
|
||||
raise ValueError(
|
||||
f"Referenced upload is no longer available: {missing_upload_id}"
|
||||
)
|
||||
|
||||
db.query(DbChatMessage).filter(DbChatMessage.session_id == session_id).delete()
|
||||
now = datetime.now(timezone.utc)
|
||||
for i, message in enumerate(messages):
|
||||
@@ -307,15 +397,9 @@ class SessionManager:
|
||||
id=msg_id,
|
||||
session_id=session_id,
|
||||
role=message.role,
|
||||
# Multimodal content (image/audio attachments) is a list;
|
||||
# serialize to JSON so the Text column round-trips via
|
||||
# _parse_msg_content. Storing the raw list let SQLAlchemy
|
||||
# bind its single-quoted repr, which _parse_msg_content
|
||||
# cannot parse (it looks for double-quoted "type"), so the
|
||||
# attachment was destroyed on reload. Mirrors _persist_message.
|
||||
content=(json.dumps(message.content)
|
||||
if isinstance(message.content, list)
|
||||
else message.content),
|
||||
# Mirrors _persist_message: keep raw media bytes out of the
|
||||
# persisted transcript and search index.
|
||||
content=persistable_message_content(message.content, message.metadata),
|
||||
meta_data=json.dumps(message.metadata) if message.metadata else None,
|
||||
timestamp=now + timedelta(microseconds=i),
|
||||
)
|
||||
@@ -324,15 +408,14 @@ class SessionManager:
|
||||
message.metadata = {}
|
||||
message.metadata["_db_id"] = msg_id
|
||||
|
||||
db_session = db.query(DbSession).filter(DbSession.id == session_id).first()
|
||||
if db_session:
|
||||
db_session.message_count = len(messages)
|
||||
db_session.updated_at = now
|
||||
db_session.last_accessed = now
|
||||
db_session.last_message_at = now
|
||||
db_session.message_count = len(messages)
|
||||
db_session.updated_at = now
|
||||
db_session.last_accessed = now
|
||||
db_session.last_message_at = now
|
||||
|
||||
db.commit()
|
||||
session.history = list(messages)
|
||||
session._history = session.history
|
||||
session.message_count = len(messages)
|
||||
logger.info("Replaced session %s history with %d messages", session_id, len(messages))
|
||||
return True
|
||||
@@ -348,30 +431,50 @@ class SessionManager:
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_session(self, session_id: str) -> Session:
|
||||
"""Get a session by ID, loading from DB if needed.
|
||||
"""Get a session by ID, loading complete DB history when needed.
|
||||
|
||||
Sessions seeded by `load_sessions` start with empty history. The
|
||||
first read here hydrates them with the message rows.
|
||||
Sessions seeded by ``load_sessions`` start with empty history, and a
|
||||
cached session can also become partially stale. Refresh metadata first,
|
||||
then hydrate whenever the cached transcript is short of the stored rows.
|
||||
Model-send routes enter through this method before building context,
|
||||
while paginated display history reads SQLite directly.
|
||||
|
||||
The gate compares against ``sync_session_metadata``'s reconciled count
|
||||
(the real ``chat_messages`` total), never the denormalized column, so a
|
||||
hydrate always closes the gap and the next read is a cache hit.
|
||||
"""
|
||||
if session_id not in self.sessions:
|
||||
self._load_session_from_db(session_id)
|
||||
else:
|
||||
cached = self.sessions[session_id]
|
||||
# Lazy hydrate: metadata-only entries get their messages on first read.
|
||||
if not cached.history and getattr(cached, "message_count", 0) > 0:
|
||||
self._load_session_from_db(session_id)
|
||||
|
||||
# Keep model/endpoint metadata fresh. Endpoint deletion can clear the
|
||||
# DB row while a session object is still cached in RAM.
|
||||
# DB row while a session object is still cached in RAM. Refreshing first
|
||||
# also exposes the authoritative message count before completeness is
|
||||
# checked.
|
||||
self.sync_session_metadata(session_id)
|
||||
|
||||
cached = self.sessions[session_id]
|
||||
cached_count = len(cached.history or [])
|
||||
stored_count = int(getattr(cached, "message_count", 0) or 0)
|
||||
if cached_count < stored_count:
|
||||
self._load_session_from_db(session_id)
|
||||
|
||||
# Update last_accessed
|
||||
self._touch_session(session_id)
|
||||
|
||||
return self.sessions[session_id]
|
||||
|
||||
def sync_session_metadata(self, session_id: str) -> bool:
|
||||
"""Refresh non-message session fields from the DB into the cached object."""
|
||||
"""Refresh non-message session fields from the DB into the cached object.
|
||||
|
||||
``message_count`` is reconciled against the real ``chat_messages`` rows
|
||||
rather than copied from the denormalized ``sessions.message_count``
|
||||
column. That column drifts in normal operation — ``_persist_message``
|
||||
swallows a failed insert but ``add_message`` has already appended in
|
||||
memory, so the next successful persist writes rows+1, and a persist for
|
||||
an uncached session writes 0. Hydration keys off this number: a
|
||||
drifted-high column would reload the whole transcript on every warm
|
||||
read, and a drifted-low one would leave the model a truncated one.
|
||||
"""
|
||||
session = self.sessions.get(session_id)
|
||||
if session is None:
|
||||
return False
|
||||
@@ -394,7 +497,12 @@ class SessionManager:
|
||||
session.archived = db_session.archived
|
||||
session.owner = getattr(db_session, "owner", None)
|
||||
session.is_important = getattr(db_session, "is_important", False) or False
|
||||
session.message_count = getattr(db_session, "message_count", session.message_count) or 0
|
||||
session.cwd = getattr(db_session, "cwd", None) or None
|
||||
session.message_count = (
|
||||
db.query(DbChatMessage)
|
||||
.filter(DbChatMessage.session_id == session_id)
|
||||
.count()
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Error syncing session metadata {session_id}: {e}")
|
||||
@@ -450,9 +558,12 @@ class SessionManager:
|
||||
endpoint_url: str,
|
||||
model: str,
|
||||
rag: bool = False,
|
||||
owner: str = None
|
||||
owner: str = None,
|
||||
cwd: str = None,
|
||||
headers: Optional[Dict[str, str]] = None,
|
||||
) -> Session:
|
||||
"""Create a new session and save to database."""
|
||||
session_headers = dict(headers or {})
|
||||
db = SessionLocal()
|
||||
try:
|
||||
db_session = DbSession(
|
||||
@@ -461,8 +572,9 @@ class SessionManager:
|
||||
endpoint_url=endpoint_url,
|
||||
model=model,
|
||||
rag=rag,
|
||||
headers={},
|
||||
headers=session_headers,
|
||||
owner=owner,
|
||||
cwd=cwd or None,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc)
|
||||
)
|
||||
@@ -475,8 +587,9 @@ class SessionManager:
|
||||
endpoint_url=endpoint_url,
|
||||
model=model,
|
||||
rag=rag,
|
||||
headers={},
|
||||
headers=session_headers,
|
||||
owner=owner,
|
||||
cwd=cwd or None,
|
||||
)
|
||||
|
||||
self.sessions[session_id] = session
|
||||
@@ -493,6 +606,12 @@ class SessionManager:
|
||||
"""Permanently delete a session and all its messages."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
try:
|
||||
from src.session_image_cleanup import cleanup_session_images
|
||||
cleanup_session_images(session_id, db=db)
|
||||
except Exception as e:
|
||||
logger.warning(f"Image cleanup failed while deleting session {session_id}: {e}")
|
||||
|
||||
# Detach documents so they survive as orphans in the library
|
||||
db.query(DbDocument).filter(DbDocument.session_id == session_id).update(
|
||||
{DbDocument.session_id: None}, synchronize_session=False
|
||||
@@ -608,24 +727,52 @@ class SessionManager:
|
||||
def save_sessions(self):
|
||||
"""No-op for DB compatibility."""
|
||||
|
||||
def ensure_task_session(self, session_id: str, name: str, endpoint_url: str, model: str, owner: str = None, task: object = None) -> Session:
|
||||
"""Create a task session if it doesn't exist, or return the existing one.
|
||||
|
||||
Unlike create_session, this checks the cache first and does NOT
|
||||
overwrite an existing in-memory session. The task scheduler must
|
||||
use this instead of direct dict assignment.
|
||||
"""
|
||||
if session_id in self.sessions:
|
||||
return self.sessions[session_id]
|
||||
|
||||
session = self.create_session(session_id, name, endpoint_url, model, owner=owner)
|
||||
if task is not None:
|
||||
task.session_id = session_id
|
||||
return session
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cleanup
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def cleanup_empty_sessions(self, auto_archive_days: int = 30) -> dict:
|
||||
"""Clean up empty and old sessions."""
|
||||
def cleanup_empty_sessions(self, auto_archive_days: int = 30, min_age_hours: int = 1) -> dict:
|
||||
"""Clean up empty and old sessions.
|
||||
|
||||
Args:
|
||||
auto_archive_days: Age in days before non-important sessions are archived.
|
||||
min_age_hours: Minimum age in hours before an empty session can be deleted.
|
||||
Prevents deleting sessions that were just created.
|
||||
"""
|
||||
db = SessionLocal()
|
||||
stats = {'deleted_empty': 0, 'archived_old': 0, 'total_checked': 0}
|
||||
|
||||
try:
|
||||
all_sessions = db.query(DbSession).all()
|
||||
cutoff_date = utcnow_naive() - timedelta(days=auto_archive_days)
|
||||
min_age = utcnow_naive() - timedelta(hours=min_age_hours)
|
||||
|
||||
for db_session in all_sessions:
|
||||
stats['total_checked'] += 1
|
||||
|
||||
# Delete empty sessions
|
||||
# Delete empty sessions only if older than min_age_hours
|
||||
if db_session.message_count == 0:
|
||||
if db_session.created_at is not None:
|
||||
created = db_session.created_at
|
||||
if created.tzinfo is None:
|
||||
created = created.replace(tzinfo=timezone.utc)
|
||||
if created > min_age:
|
||||
continue # Too young to delete
|
||||
if db_session.id in self.sessions:
|
||||
del self.sessions[db_session.id]
|
||||
db.delete(db_session)
|
||||
|
||||
@@ -14,20 +14,20 @@ services:
|
||||
odysseus:
|
||||
build: .
|
||||
ports:
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7000}:7000"
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7011}:7000"
|
||||
volumes:
|
||||
- ./data:/app/data:z
|
||||
- ./logs:/app/logs:z
|
||||
- ${APP_DATA_DIR:-./data}:/app/data:z
|
||||
- ${APP_LOGS_DIR:-./logs}:/app/logs:z
|
||||
# Cookbook remote-server SSH identity. Odysseus can generate a key here;
|
||||
# add the shown public key to each remote server's authorized_keys.
|
||||
- ./data/ssh:/app/.ssh:z
|
||||
- ${APP_DATA_DIR:-./data}/ssh:/app/.ssh:z
|
||||
# Cookbook local model cache. Inside Docker, "Local" means the Odysseus
|
||||
# container, so persist its HuggingFace cache under ./data/huggingface.
|
||||
- ./data/huggingface:/app/.cache/huggingface:z
|
||||
- ${APP_DATA_DIR:-./data}/huggingface:/app/.cache/huggingface:z
|
||||
# Cookbook-installed Python CLIs/packages (vLLM, llama-cpp-python, etc.)
|
||||
# land under /app/.local for the odysseus user. Persist them so a
|
||||
# container recreate does not silently remove installed serve engines.
|
||||
- ./data/local:/app/.local:z
|
||||
- ${APP_DATA_DIR:-./data}/local:/app/.local:z
|
||||
extra_hosts:
|
||||
# Lets the container reach local services on the Docker host, including
|
||||
# Ollama at http://host.docker.internal:11434.
|
||||
@@ -46,10 +46,11 @@ services:
|
||||
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
|
||||
- AUTH_ENABLED=${AUTH_ENABLED:-true}
|
||||
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
|
||||
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
|
||||
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
|
||||
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
|
||||
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-false}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-}
|
||||
- EMBEDDING_URL=${EMBEDDING_URL:-}
|
||||
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
|
||||
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
|
||||
@@ -58,11 +59,39 @@ services:
|
||||
- CLEANUP_INTERVAL_HOURS=${CLEANUP_INTERVAL_HOURS:-24}
|
||||
- ODYSSEUS_INPROCESS_POLLERS=${ODYSSEUS_INPROCESS_POLLERS:-1}
|
||||
- ODYSSEUS_INPROCESS_TASKS=${ODYSSEUS_INPROCESS_TASKS:-1}
|
||||
- ODYSSEUS_UNATTENDED_MODE=${ODYSSEUS_UNATTENDED_MODE:-false}
|
||||
- ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS=${ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS:-1}
|
||||
- ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT=${ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT:-0}
|
||||
- ODYSSEUS_CAPTURE_MODEL_REQUESTS=${ODYSSEUS_CAPTURE_MODEL_REQUESTS:-0}
|
||||
- ODYSSEUS_MCP_EMAIL_OWNER=${ODYSSEUS_MCP_EMAIL_OWNER:-}
|
||||
- ODYSSEUS_SCRIPT_HOST=${ODYSSEUS_SCRIPT_HOST:-localhost}
|
||||
- ODYSSEUS_CHAT_UPLOAD_MAX_BYTES=${ODYSSEUS_CHAT_UPLOAD_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES:-104857600}
|
||||
- ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_MEMORY_IMPORT_MAX_BYTES=${ODYSSEUS_MEMORY_IMPORT_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES=${ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EDITOR_DRAFT_MAX_BYTES=${ODYSSEUS_EDITOR_DRAFT_MAX_BYTES:-268435456}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
# Host workspace translation is opt-in. Keep the public compose file
|
||||
# user-neutral; configure these in a local .env or use the host-workspace
|
||||
# overlay with ODYSSEUS_HOST_WORKSPACE_DIR.
|
||||
- ODYSSEUS_WORKSPACE_HOST_ROOT=${ODYSSEUS_WORKSPACE_HOST_ROOT:-}
|
||||
- ODYSSEUS_WORKSPACE_CONTAINER_ROOT=${ODYSSEUS_WORKSPACE_CONTAINER_ROOT:-/workspace}
|
||||
- ODYSSEUS_WORKSPACE_DEFAULT=${ODYSSEUS_WORKSPACE_DEFAULT:-}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
|
||||
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
|
||||
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
|
||||
# Externally reachable origin for MCP OAuth callbacks. The container
|
||||
# always listens on 7000 and cannot see the host port map above, so
|
||||
# remote MCP OAuth needs this set whenever the browser reaches
|
||||
# Odysseus on anything other than http://localhost:7000.
|
||||
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
|
||||
- TAVILY_API_KEY=${TAVILY_API_KEY:-}
|
||||
- SERPER_API_KEY=${SERPER_API_KEY:-}
|
||||
# PUID / PGID — the user/group the container drops to before
|
||||
@@ -118,12 +147,17 @@ services:
|
||||
fi
|
||||
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
|
||||
fi
|
||||
# Advisory: a settings file the migration cannot parse or rewrite must
|
||||
# not be what stops searxng from booting. It explains itself on stderr
|
||||
# and we carry on, letting searxng report anything genuinely wrong.
|
||||
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
|
||||
exec /usr/local/searxng/entrypoint.sh
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
volumes:
|
||||
- searxng-data:/etc/searxng
|
||||
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
|
||||
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
|
||||
environment:
|
||||
- SEARXNG_BASE_URL=http://localhost:8080/
|
||||
- SEARXNG_SECRET=${SEARXNG_SECRET:-}
|
||||
|
||||
@@ -13,20 +13,20 @@ services:
|
||||
odysseus:
|
||||
build: .
|
||||
ports:
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7000}:7000"
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7011}:7000"
|
||||
volumes:
|
||||
- ./data:/app/data:z
|
||||
- ./logs:/app/logs:z
|
||||
- ${APP_DATA_DIR:-./data}:/app/data:z
|
||||
- ${APP_LOGS_DIR:-./logs}:/app/logs:z
|
||||
# Cookbook remote-server SSH identity. Odysseus can generate a key here;
|
||||
# add the shown public key to each remote server's authorized_keys.
|
||||
- ./data/ssh:/app/.ssh:z
|
||||
- ${APP_DATA_DIR:-./data}/ssh:/app/.ssh:z
|
||||
# Cookbook local model cache. Inside Docker, "Local" means the Odysseus
|
||||
# container, so persist its HuggingFace cache under ./data/huggingface.
|
||||
- ./data/huggingface:/app/.cache/huggingface:z
|
||||
- ${APP_DATA_DIR:-./data}/huggingface:/app/.cache/huggingface:z
|
||||
# Cookbook-installed Python CLIs/packages (vLLM, llama-cpp-python, etc.)
|
||||
# land under /app/.local for the odysseus user. Persist them so a
|
||||
# container recreate does not silently remove installed serve engines.
|
||||
- ./data/local:/app/.local:z
|
||||
- ${APP_DATA_DIR:-./data}/local:/app/.local:z
|
||||
extra_hosts:
|
||||
# Lets the container reach local services on the Docker host, including
|
||||
# Ollama at http://host.docker.internal:11434.
|
||||
@@ -45,10 +45,11 @@ services:
|
||||
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
|
||||
- AUTH_ENABLED=${AUTH_ENABLED:-true}
|
||||
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
|
||||
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
|
||||
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
|
||||
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
|
||||
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-false}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-}
|
||||
- EMBEDDING_URL=${EMBEDDING_URL:-}
|
||||
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
|
||||
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
|
||||
@@ -57,11 +58,39 @@ services:
|
||||
- CLEANUP_INTERVAL_HOURS=${CLEANUP_INTERVAL_HOURS:-24}
|
||||
- ODYSSEUS_INPROCESS_POLLERS=${ODYSSEUS_INPROCESS_POLLERS:-1}
|
||||
- ODYSSEUS_INPROCESS_TASKS=${ODYSSEUS_INPROCESS_TASKS:-1}
|
||||
- ODYSSEUS_UNATTENDED_MODE=${ODYSSEUS_UNATTENDED_MODE:-false}
|
||||
- ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS=${ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS:-1}
|
||||
- ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT=${ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT:-0}
|
||||
- ODYSSEUS_CAPTURE_MODEL_REQUESTS=${ODYSSEUS_CAPTURE_MODEL_REQUESTS:-0}
|
||||
- ODYSSEUS_MCP_EMAIL_OWNER=${ODYSSEUS_MCP_EMAIL_OWNER:-}
|
||||
- ODYSSEUS_SCRIPT_HOST=${ODYSSEUS_SCRIPT_HOST:-localhost}
|
||||
- ODYSSEUS_CHAT_UPLOAD_MAX_BYTES=${ODYSSEUS_CHAT_UPLOAD_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES:-104857600}
|
||||
- ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_MEMORY_IMPORT_MAX_BYTES=${ODYSSEUS_MEMORY_IMPORT_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES=${ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EDITOR_DRAFT_MAX_BYTES=${ODYSSEUS_EDITOR_DRAFT_MAX_BYTES:-268435456}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
# Host workspace translation is opt-in. Keep the public compose file
|
||||
# user-neutral; configure these in a local .env or use the host-workspace
|
||||
# overlay with ODYSSEUS_HOST_WORKSPACE_DIR.
|
||||
- ODYSSEUS_WORKSPACE_HOST_ROOT=${ODYSSEUS_WORKSPACE_HOST_ROOT:-}
|
||||
- ODYSSEUS_WORKSPACE_CONTAINER_ROOT=${ODYSSEUS_WORKSPACE_CONTAINER_ROOT:-/workspace}
|
||||
- ODYSSEUS_WORKSPACE_DEFAULT=${ODYSSEUS_WORKSPACE_DEFAULT:-}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
|
||||
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
|
||||
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
|
||||
# Externally reachable origin for MCP OAuth callbacks. The container
|
||||
# always listens on 7000 and cannot see the host port map above, so
|
||||
# remote MCP OAuth needs this set whenever the browser reaches
|
||||
# Odysseus on anything other than http://localhost:7000.
|
||||
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
|
||||
- TAVILY_API_KEY=${TAVILY_API_KEY:-}
|
||||
- SERPER_API_KEY=${SERPER_API_KEY:-}
|
||||
# PUID / PGID — the user/group the container drops to before
|
||||
@@ -121,12 +150,17 @@ services:
|
||||
fi
|
||||
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
|
||||
fi
|
||||
# Advisory: a settings file the migration cannot parse or rewrite must
|
||||
# not be what stops searxng from booting. It explains itself on stderr
|
||||
# and we carry on, letting searxng report anything genuinely wrong.
|
||||
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
|
||||
exec /usr/local/searxng/entrypoint.sh
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
volumes:
|
||||
- searxng-data:/etc/searxng
|
||||
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
|
||||
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
|
||||
environment:
|
||||
- SEARXNG_BASE_URL=http://localhost:8080/
|
||||
- SEARXNG_SECRET=${SEARXNG_SECRET:-}
|
||||
|
||||
@@ -2,20 +2,20 @@ services:
|
||||
odysseus:
|
||||
build: .
|
||||
ports:
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7000}:7000"
|
||||
- "${APP_BIND:-127.0.0.1}:${APP_PORT:-7011}:7000"
|
||||
volumes:
|
||||
- ./data:/app/data:z
|
||||
- ./logs:/app/logs:z
|
||||
- ${APP_DATA_DIR:-./data}:/app/data:z
|
||||
- ${APP_LOGS_DIR:-./logs}:/app/logs:z
|
||||
# Cookbook remote-server SSH identity. Odysseus can generate a key here;
|
||||
# add the shown public key to each remote server's authorized_keys.
|
||||
- ./data/ssh:/app/.ssh:z
|
||||
- ${APP_DATA_DIR:-./data}/ssh:/app/.ssh:z
|
||||
# Cookbook local model cache. Inside Docker, "Local" means the Odysseus
|
||||
# container, so persist its HuggingFace cache under ./data/huggingface.
|
||||
- ./data/huggingface:/app/.cache/huggingface:z
|
||||
- ${APP_DATA_DIR:-./data}/huggingface:/app/.cache/huggingface:z
|
||||
# Cookbook-installed Python CLIs/packages (vLLM, llama-cpp-python, etc.)
|
||||
# land under /app/.local for the odysseus user. Persist them so a
|
||||
# container recreate does not silently remove installed serve engines.
|
||||
- ./data/local:/app/.local:z
|
||||
- ${APP_DATA_DIR:-./data}/local:/app/.local:z
|
||||
extra_hosts:
|
||||
# Lets the container reach local services on the Docker host, including
|
||||
# Ollama at http://host.docker.internal:11434.
|
||||
@@ -34,10 +34,11 @@ services:
|
||||
- DATABASE_URL=${DATABASE_URL:-sqlite:///./data/app.db}
|
||||
- AUTH_ENABLED=${AUTH_ENABLED:-true}
|
||||
- LOCALHOST_BYPASS=${LOCALHOST_BYPASS:-false}
|
||||
- COMPANION_BASE_URL=${COMPANION_BASE_URL:-}
|
||||
- ODYSSEUS_ADMIN_USER=${ODYSSEUS_ADMIN_USER:-admin}
|
||||
- ODYSSEUS_ADMIN_PASSWORD=${ODYSSEUS_ADMIN_PASSWORD:-}
|
||||
- ALLOWED_ORIGINS=${ALLOWED_ORIGINS:-http://localhost,http://127.0.0.1}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-false}
|
||||
- SECURE_COOKIES=${SECURE_COOKIES:-}
|
||||
- EMBEDDING_URL=${EMBEDDING_URL:-}
|
||||
- EMBEDDING_MODEL=${EMBEDDING_MODEL:-}
|
||||
- EMBEDDING_API_KEY=${EMBEDDING_API_KEY:-}
|
||||
@@ -46,11 +47,39 @@ services:
|
||||
- CLEANUP_INTERVAL_HOURS=${CLEANUP_INTERVAL_HOURS:-24}
|
||||
- ODYSSEUS_INPROCESS_POLLERS=${ODYSSEUS_INPROCESS_POLLERS:-1}
|
||||
- ODYSSEUS_INPROCESS_TASKS=${ODYSSEUS_INPROCESS_TASKS:-1}
|
||||
- ODYSSEUS_UNATTENDED_MODE=${ODYSSEUS_UNATTENDED_MODE:-false}
|
||||
- ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS=${ODYSSEUS_QWEN_NATIVE_COMPACT_BUILTINS:-1}
|
||||
- ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT=${ODYSSEUS_QWEN_SUPPRESS_LOCAL_CONTEXT:-0}
|
||||
- ODYSSEUS_CAPTURE_MODEL_REQUESTS=${ODYSSEUS_CAPTURE_MODEL_REQUESTS:-0}
|
||||
- ODYSSEUS_MCP_EMAIL_OWNER=${ODYSSEUS_MCP_EMAIL_OWNER:-}
|
||||
- ODYSSEUS_SCRIPT_HOST=${ODYSSEUS_SCRIPT_HOST:-localhost}
|
||||
- ODYSSEUS_CHAT_UPLOAD_MAX_BYTES=${ODYSSEUS_CHAT_UPLOAD_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES:-104857600}
|
||||
- ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES=${ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_MEMORY_IMPORT_MAX_BYTES=${ODYSSEUS_MEMORY_IMPORT_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES=${ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_EDITOR_DRAFT_MAX_BYTES=${ODYSSEUS_EDITOR_DRAFT_MAX_BYTES:-268435456}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
# Host workspace translation is opt-in. Keep the public compose file
|
||||
# user-neutral; configure these in a local .env or use the host-workspace
|
||||
# overlay with ODYSSEUS_HOST_WORKSPACE_DIR.
|
||||
- ODYSSEUS_WORKSPACE_HOST_ROOT=${ODYSSEUS_WORKSPACE_HOST_ROOT:-}
|
||||
- ODYSSEUS_WORKSPACE_CONTAINER_ROOT=${ODYSSEUS_WORKSPACE_CONTAINER_ROOT:-/workspace}
|
||||
- ODYSSEUS_WORKSPACE_DEFAULT=${ODYSSEUS_WORKSPACE_DEFAULT:-}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
- GOOGLE_OAUTH_CLIENT_ID=${GOOGLE_OAUTH_CLIENT_ID:-}
|
||||
- GOOGLE_OAUTH_CLIENT_SECRET=${GOOGLE_OAUTH_CLIENT_SECRET:-}
|
||||
- GOOGLE_OAUTH_REDIRECT_URI=${GOOGLE_OAUTH_REDIRECT_URI:-}
|
||||
# Externally reachable origin for MCP OAuth callbacks. The container
|
||||
# always listens on 7000 and cannot see the host port map above, so
|
||||
# remote MCP OAuth needs this set whenever the browser reaches
|
||||
# Odysseus on anything other than http://localhost:7000.
|
||||
- OAUTH_REDIRECT_BASE_URL=${OAUTH_REDIRECT_BASE_URL:-}
|
||||
- TAVILY_API_KEY=${TAVILY_API_KEY:-}
|
||||
- SERPER_API_KEY=${SERPER_API_KEY:-}
|
||||
# PUID / PGID — the user/group the container drops to before
|
||||
@@ -99,12 +128,17 @@ services:
|
||||
fi
|
||||
sed "s|__SEARXNG_SECRET__|$$secret|g" /tmp/searxng-settings.yml.template > /etc/searxng/settings.yml
|
||||
fi
|
||||
# Advisory: a settings file the migration cannot parse or rewrite must
|
||||
# not be what stops searxng from booting. It explains itself on stderr
|
||||
# and we carry on, letting searxng report anything genuinely wrong.
|
||||
/usr/local/searxng/.venv/bin/python /tmp/migrate-searxng-settings.py /etc/searxng/settings.yml || true
|
||||
exec /usr/local/searxng/entrypoint.sh
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
volumes:
|
||||
- searxng-data:/etc/searxng
|
||||
- ./config/searxng/settings.yml:/tmp/searxng-settings.yml.template:ro,z
|
||||
- ./scripts/migrate_searxng_settings.py:/tmp/migrate-searxng-settings.py:ro,z
|
||||
environment:
|
||||
- SEARXNG_BASE_URL=http://localhost:8080/
|
||||
- SEARXNG_SECRET=${SEARXNG_SECRET:-}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build patched wheels for Real-ESRGAN's unmaintained dependencies.
|
||||
#
|
||||
# basicsr / gfpgan / facexlib (xinntao, last released 2022) read their version
|
||||
# in setup.py with:
|
||||
#
|
||||
# exec(compile(f.read(), version_file, 'exec'))
|
||||
# return locals()['__version__']
|
||||
#
|
||||
# Python 3.13+ implements PEP 667: locals() inside a function returns an
|
||||
# independent snapshot that exec() can no longer mutate, so the read raises
|
||||
# `KeyError: '__version__'` and the sdist build fails. That is why the Cookbook
|
||||
# "install realesrgan" button dies on the python:3.14 image. The packages have
|
||||
# no fixed release, so we patch get_version() to exec into an explicit namespace
|
||||
# dict (works on every Python) and build wheels from the patched source.
|
||||
#
|
||||
# Usage: build-realesrgan-wheels.sh [OUTPUT_DIR] (default: /wheels)
|
||||
set -euo pipefail
|
||||
|
||||
OUT="${1:-/wheels}"
|
||||
mkdir -p "$OUT"
|
||||
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
cd "$work"
|
||||
|
||||
# Pinned to the versions Real-ESRGAN 0.3.0 resolves to.
|
||||
SPECS="basicsr==1.4.2 gfpgan==1.3.8 facexlib==0.3.0"
|
||||
|
||||
for spec in $SPECS; do
|
||||
name="${spec%%==*}"
|
||||
ver="${spec##*==}"
|
||||
# pip download builds metadata (and trips the same bug), so fetch the raw
|
||||
# sdist URL from the PyPI JSON API instead.
|
||||
url="$(python - "$name" "$ver" <<'PY'
|
||||
import json, sys, urllib.request
|
||||
name, ver = sys.argv[1], sys.argv[2]
|
||||
data = json.load(urllib.request.urlopen(f"https://pypi.org/pypi/{name}/{ver}/json"))
|
||||
for f in data["urls"]:
|
||||
if f["packagetype"] == "sdist":
|
||||
print(f["url"]); break
|
||||
else:
|
||||
sys.exit(f"no sdist found for {name}=={ver}")
|
||||
PY
|
||||
)"
|
||||
echo ">> fetching ${name} ${ver}: ${url}"
|
||||
curl -fsSL "$url" -o "${name}.tar.gz"
|
||||
tar xzf "${name}.tar.gz"
|
||||
done
|
||||
|
||||
echo ">> patching get_version()"
|
||||
python - <<'PY'
|
||||
import pathlib
|
||||
old_exec = "exec(compile(f.read(), version_file, 'exec'))"
|
||||
new_exec = "_ver_ns = {}\n exec(compile(f.read(), version_file, 'exec'), _ver_ns)"
|
||||
old_ret = "return locals()['__version__']"
|
||||
new_ret = "return _ver_ns['__version__']"
|
||||
patched = 0
|
||||
for setup in pathlib.Path(".").glob("*/setup.py"):
|
||||
s = setup.read_text()
|
||||
if old_exec in s and old_ret in s:
|
||||
setup.write_text(s.replace(old_exec, new_exec).replace(old_ret, new_ret))
|
||||
print(" patched", setup)
|
||||
patched += 1
|
||||
assert patched == 3, f"expected to patch 3 setup.py files, patched {patched}"
|
||||
PY
|
||||
|
||||
echo ">> building wheels into ${OUT}"
|
||||
pip wheel --no-deps -w "$OUT" ./basicsr-* ./gfpgan-* ./facexlib-*
|
||||
ls -l "$OUT"
|
||||
@@ -13,6 +13,8 @@ set -e
|
||||
|
||||
PUID="${PUID:-1000}"
|
||||
PGID="${PGID:-1000}"
|
||||
GOSU_BIN="$(command -v gosu)"
|
||||
PYTHON_BIN="$(command -v python)"
|
||||
|
||||
# Reuse an existing matching group/user if the host's UID/GID already
|
||||
# corresponds to one in /etc/passwd (e.g. when the image is rebuilt
|
||||
@@ -24,26 +26,78 @@ if ! getent passwd "$PUID" >/dev/null 2>&1; then
|
||||
useradd -u "$PUID" -g "$PGID" -M -s /bin/sh -d /app odysseus
|
||||
fi
|
||||
|
||||
# Repair ownership on every writable path the app touches at runtime.
|
||||
#
|
||||
# Bind-mounted dirs (/app/data, /app/logs) are the obvious ones, but
|
||||
# the app ALSO writes inside the image's own source tree at runtime:
|
||||
# - services/cache/{search,content}/* (search cache LRU)
|
||||
# - services/search_analytics.json
|
||||
# - services/search_engine_error.log
|
||||
# - services/tts cache, etc.
|
||||
# These dirs were created as root during `docker build`, so dropping
|
||||
# to PUID:PGID would otherwise crash on the first import that tries
|
||||
# to mkdir them. Chown the whole /app tree — fast (<1s on this size)
|
||||
# and idempotent via the `-not -uid` filter so we only touch files
|
||||
# that need fixing.
|
||||
for dir in /app /app/data /app/logs; do
|
||||
ODY_USER="$(getent passwd "$PUID" | cut -d: -f1)"
|
||||
[ -z "$ODY_USER" ] && ODY_USER=odysseus
|
||||
|
||||
# Docker-socket group plumbing for the explicit host-Docker overlay. When
|
||||
# opted in, the socket is owned by root:<host docker gid>. Add the app user
|
||||
# to that group and later call gosu by username so supplementary groups are
|
||||
# retained.
|
||||
DOCKER_SOCK="${DOCKER_SOCK:-/var/run/docker.sock}"
|
||||
if [ "${ODYSSEUS_ENABLE_HOST_DOCKER:-}" = "true" ] && [ -S "$DOCKER_SOCK" ]; then
|
||||
SOCK_GID="$(stat -c '%g' "$DOCKER_SOCK" 2>/dev/null || echo '')"
|
||||
if [ -n "$SOCK_GID" ] && [ "$SOCK_GID" != "0" ]; then
|
||||
if ! getent group "$SOCK_GID" >/dev/null 2>&1; then
|
||||
groupadd -g "$SOCK_GID" docker_host || true
|
||||
fi
|
||||
SOCK_GROUP="$(getent group "$SOCK_GID" | cut -d: -f1)"
|
||||
if [ -n "$SOCK_GROUP" ]; then
|
||||
usermod -aG "$SOCK_GROUP" "$ODY_USER" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
mount_root_for() {
|
||||
awk -v target="$1" '$5 == target { print $4; exit }' /proc/self/mountinfo 2>/dev/null || true
|
||||
}
|
||||
|
||||
is_broad_mount_root() {
|
||||
case "$1" in
|
||||
/|/home|/srv|/var|/usr|/opt|/tmp|/mnt|/media)
|
||||
return 0
|
||||
;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
repair_tree_ownership() {
|
||||
dir="$1"
|
||||
if [ -d "$dir" ]; then
|
||||
# `find ... -not -uid` keeps this O(touched-files), not
|
||||
# O(everything), so terabyte-sized maildirs don't slow startup.
|
||||
find "$dir" -not -uid "$PUID" -print0 2>/dev/null \
|
||||
find "$dir" -xdev -not -uid "$PUID" -print0 2>/dev/null \
|
||||
| xargs -0 -r chown "$PUID:$PGID" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
|
||||
repair_app_tree_ownership() {
|
||||
if [ -d /app ]; then
|
||||
find /app -xdev \
|
||||
\( -path /app/data -o -path /app/logs -o -path /app/.ssh -o -path /app/.cache -o -path /app/.local \) -prune \
|
||||
-o -not -uid "$PUID" -print0 2>/dev/null \
|
||||
| xargs -0 -r chown "$PUID:$PGID" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
|
||||
repair_bind_mount_ownership() {
|
||||
dir="$1"
|
||||
if [ ! -d "$dir" ]; then
|
||||
return
|
||||
fi
|
||||
|
||||
mount_root="$(mount_root_for "$dir")"
|
||||
if is_broad_mount_root "$mount_root"; then
|
||||
echo "Skipping recursive ownership repair for $dir because it maps to broad host path $mount_root" >&2
|
||||
chown "$PUID:$PGID" "$dir" 2>/dev/null || true
|
||||
return
|
||||
fi
|
||||
|
||||
repair_tree_ownership "$dir"
|
||||
}
|
||||
|
||||
# Repair image-owned writable paths without walking into bind-mounted host
|
||||
# trees, then repair the app-owned mount roots separately.
|
||||
repair_app_tree_ownership
|
||||
for dir in /app/data /app/logs /app/.ssh /app/.cache/huggingface /app/.local; do
|
||||
repair_bind_mount_ownership "$dir"
|
||||
done
|
||||
|
||||
# Cookbook installs vllm/etc. via `pip install --user`, which pulls
|
||||
@@ -70,6 +124,7 @@ for cu in \
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# Disable the FlashInfer JIT sampler unconditionally — it is sampler-only
|
||||
# and has no impact on the attention path, but requires nvcc + matching
|
||||
# CUDA headers at startup. Without this, vLLM crashes with "Could not find
|
||||
@@ -83,9 +138,9 @@ export PATH="/app/.local/bin:$PATH"
|
||||
# Run first-time setup as the app user so data/ files get the right ownership.
|
||||
# setup.py is idempotent — skips auth.json / .env if they already exist.
|
||||
# || true so a setup failure never prevents the container from starting.
|
||||
gosu "$PUID:$PGID" python /app/setup.py || true
|
||||
"$GOSU_BIN" "$ODY_USER" "$PYTHON_BIN" /app/setup.py || true
|
||||
|
||||
# Drop root and run the actual app. `gosu` is preferred over `su` /
|
||||
# `sudo` because it cleans up the process tree (no extra shell layer)
|
||||
# so signals (SIGTERM from `docker stop`) reach uvicorn directly.
|
||||
exec gosu "$PUID:$PGID" "$@"
|
||||
exec "$GOSU_BIN" "$ODY_USER" "$@"
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# High-trust host Docker access. Enable only when local Docker-daemon
|
||||
# management from Cookbook is required and you accept that raw socket access
|
||||
# grants broad control over the host Docker daemon.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-docker.yml
|
||||
# DOCKER_GID=<numeric host Docker group id>
|
||||
services:
|
||||
odysseus:
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
group_add: ["${DOCKER_GID:-963}"]
|
||||
environment:
|
||||
- ODYSSEUS_ENABLE_HOST_DOCKER=true
|
||||
@@ -0,0 +1,21 @@
|
||||
# High-trust host network access. Enable only when the Odysseus agent needs
|
||||
# host-native LAN/VPN/mDNS behavior that Docker bridge networking cannot
|
||||
# provide. Linux only; Docker Desktop does not provide equivalent host
|
||||
# networking semantics.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-workspace.yml:docker/host-network.yml
|
||||
# APP_PORT=7011
|
||||
services:
|
||||
odysseus:
|
||||
network_mode: host
|
||||
ports: !reset []
|
||||
environment:
|
||||
- APP_PORT=${APP_PORT:-7011}
|
||||
- APP_BIND=${APP_BIND:-0.0.0.0}
|
||||
- SEARXNG_INSTANCE=${ODYSSEUS_HOST_NETWORK_SEARXNG_INSTANCE:-http://127.0.0.1:8080}
|
||||
- CHROMADB_HOST=${ODYSSEUS_HOST_NETWORK_CHROMADB_HOST:-127.0.0.1}
|
||||
- CHROMADB_PORT=${ODYSSEUS_HOST_NETWORK_CHROMADB_PORT:-8100}
|
||||
- ODYSSEUS_CONTAINER_NETWORK_MODE=host
|
||||
command:
|
||||
- sh
|
||||
- -c
|
||||
- exec uvicorn app:app --host "$${APP_BIND:-0.0.0.0}" --port "$${APP_PORT:-7011}"
|
||||
@@ -0,0 +1,11 @@
|
||||
# High-trust host workspace access. Enable only when the Odysseus agent should
|
||||
# work on a host directory outside the container's normal /app/data sandbox.
|
||||
# COMPOSE_FILE=docker-compose.yml:docker/host-workspace.yml
|
||||
# ODYSSEUS_HOST_WORKSPACE_DIR=/absolute/host/path
|
||||
# ODYSSEUS_HOST_WORKSPACE_MOUNT=/host/workspace
|
||||
services:
|
||||
odysseus:
|
||||
volumes:
|
||||
- ${ODYSSEUS_HOST_WORKSPACE_DIR:?set ODYSSEUS_HOST_WORKSPACE_DIR}:${ODYSSEUS_HOST_WORKSPACE_MOUNT:-/host/workspace}:rw,z
|
||||
environment:
|
||||
- ODYSSEUS_HOST_WORKSPACE_MOUNT=${ODYSSEUS_HOST_WORKSPACE_MOUNT:-/host/workspace}
|
||||
@@ -0,0 +1,75 @@
|
||||
# Agent turn contract
|
||||
|
||||
Scope: product Agent turns on 7011. Environment-owned native/TUI bridges retain
|
||||
their existing execution contract. No model weights or training settings change.
|
||||
|
||||
## Boundaries
|
||||
|
||||
1. `src/turn_contract.py` classifies capabilities, including explicit compound
|
||||
requests and referential follow-ups. Classification is selection, not permission.
|
||||
2. `routes/chat_routes.py` resolves toggles, privileges, global/plan/incognito
|
||||
restrictions, fixture restrictions and available schema inventory before
|
||||
freezing the offered set. Web enabled alone does not select web tools.
|
||||
3. `TurnContract` checks `required <= offered <= executable`, stores immutable
|
||||
serialized schema copies, and records unavailable requirements. An unavailable
|
||||
request stops without inference or substitution; unknown actions ask for clarity.
|
||||
Exact account-discovery requests narrow selection to account metadata only;
|
||||
compounds retain their declared family scope. Media operations declare their
|
||||
existing tool dependencies rather than falling back to shell generation.
|
||||
4. The agent's prompt/schema route and fallback use that same logical scope.
|
||||
Native versus textual serialization remains model-specific. Answer-only phases
|
||||
can suppress tool calls without granting a different scope.
|
||||
Contract turns preserve the already-compacted conversation and tool-call/result
|
||||
IDs. The standalone specialist prompt's latest-message-only behavior is not used
|
||||
for these product turns. Prompt domains also come from the contract.
|
||||
Accepted in-scope calls retain their model-provided arguments and native IDs;
|
||||
the explicit-intent fallback must not overwrite them with the whole user turn.
|
||||
5. The context-bound dispatcher checks membership **and** existing runtime policy,
|
||||
owner restrictions and exact-action approvals. A contract is not authorization
|
||||
to bypass those gates. Contract work bypasses terminating legacy shortcuts.
|
||||
6. `_AgentRenderState` explicitly identifies streamed versus canonical output.
|
||||
Later synthesis transfers ownership with turn-scoped replacement. The frontend
|
||||
reconciles visible DOM, not just accumulated strings; tool evidence is retained.
|
||||
Ownership is included in saved metrics and `message_saved` events.
|
||||
History and resume honor replacement scope. Single-capability turns retain
|
||||
canonical output: an always-synthesize trial caused a live notes loop and was
|
||||
reverted. Compound turns cannot terminate after only one capability's result.
|
||||
|
||||
## Verification
|
||||
|
||||
Use the project's configured Python environment, not an unrelated system Python:
|
||||
|
||||
```sh
|
||||
/home/pewds/odysseus-cookbook-fresh/.venv/bin/python -m pytest -q \
|
||||
tests/test_turn_contract.py tests/test_turn_contract_integration.py \
|
||||
tests/test_agent_turn_contract_boundaries.py tests/test_turn_rendering_js.py \
|
||||
tests/test_contract_prompt_conversation.py tests/test_product_turn_contract_route.py \
|
||||
tests/test_contract_explicit_fallback.py \
|
||||
tests/test_history_resume_rendering_js.py \
|
||||
tests/test_chat_route_tool_policy.py tests/test_tool_policy.py \
|
||||
tests/test_frontend_module_version_parity.py
|
||||
node scripts/verify_agent_turn_contract.mjs --max-turns 80 --total-ms 900000
|
||||
```
|
||||
|
||||
The browser verifier uses `sft_alex_creator` and actual 7011 Agent controls. It
|
||||
captures request toggles, SSE contract/tool events, visible output and persisted
|
||||
history. Ten families have four initial/follow-up Web-toggle combinations.
|
||||
Blocked or unrun cases are not passes. Email requires verified fixture isolation;
|
||||
do not enable global fixture mode on the user's live service to make a test pass.
|
||||
|
||||
## Remaining limits
|
||||
|
||||
- Classification is deterministic and vocabulary-based, not a proof of semantic
|
||||
understanding. Add independent behavior examples for confirmed misses.
|
||||
- Schema registration and policy permission do not guarantee a remote provider
|
||||
stays healthy throughout a turn. Runtime failure must remain visible.
|
||||
- Separate tool/argument errors, tool-service failures, rendering failures and
|
||||
verifier defects in reports. Do not infer model accuracy from routing alone.
|
||||
- Canonical summaries can still ignore presentation constraints such as a
|
||||
requested item count. Do not count those as full functional passes. Forcing an
|
||||
extra model round is not a validated general repair for this deployed model.
|
||||
- Keep all imports of a local JS module on the same URL identity. Distinct query
|
||||
versions instantiate separate module state even when source files are identical.
|
||||
|
||||
Live baseline and current matrix results are in `reports/agent-turn-contract-*`.
|
||||
The implementation is not a claim that every family has passed live verification.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Background research → originating chat
|
||||
|
||||
Chat `trigger_research` calls carry a **dispatcher-supplied** `origin_chat_id`.
|
||||
The research start route verifies chat ownership before registering a durable
|
||||
`background_tool_jobs` row and starting the existing research service. Panel
|
||||
jobs have no origin and never inject a chat reply.
|
||||
|
||||
- Chat default: **2 rounds**, 120-second *soft* research budget. Explicit
|
||||
deeper/Auto rounds regain the normal research time budget. Panel defaults
|
||||
remain unchanged. This is not a guaranteed two-minute wall-clock deadline.
|
||||
- A completion callback stores the report and sources. A startup worker also
|
||||
reconciles missed callbacks and research errors/restarts.
|
||||
- When the origin has no active foreground/detached run, its model summarizes
|
||||
the report with thinking off and no tools. An outer 75-second deadline also
|
||||
bounds model-slot waits. If synthesis is unavailable, deliver an honest
|
||||
notice plus the report link; preserve the evidence for follow-ups.
|
||||
- Message and delivery marker commit in one transaction with a deterministic
|
||||
message ID. Report context is stored in server message metadata and injected
|
||||
as untrusted evidence in regular and compact model history. Long excerpts
|
||||
are explicitly marked; the saved full research report remains accessible.
|
||||
- The browser polls owner-scoped `/api/research/chat-jobs/{chat_id}`, appending
|
||||
unseen message IDs only when that chat is current and not streaming. No
|
||||
transcript replacement or forced navigation. Reloaded history deduplicates.
|
||||
- Chat uses the existing agent-thread rail and expandable rows. The compact
|
||||
header shows status and a right-aligned BG task label with the shared whirlpool
|
||||
while running; expanding reveals topic, phase/round, source count and report
|
||||
link. Rows update in place, preserving expansion/focus while chat streams.
|
||||
Completed rows remain visible; zero-source runs show a warning, not success.
|
||||
Progress polling excludes reports and internal fields.
|
||||
|
||||
Other tools are **not automatically backgrounded**. The durable handoff can be
|
||||
reused, but each future producer needs explicit launch/result/permission wiring.
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
/home/pewds/odysseus-cookbook-fresh/.venv/bin/pytest -q tests/test_background_tool_jobs.py tests/test_research_chat_runtime.py
|
||||
node --test tests/backgroundToolJobs.test.mjs
|
||||
node scripts/verify_background_delivery_isolation.mjs
|
||||
node scripts/verify_background_research_cards.mjs
|
||||
node scripts/verify_background_research_chat.mjs
|
||||
```
|
||||
|
||||
The last script uses disposable `sft_alex_creator` chats and real research/model
|
||||
calls, then removes only its own reports/chats. Do not use real-user mutations.
|
||||
It checks two-round launch, continued chat, automatic arrival, no transcript
|
||||
rebuild/duplicates, reload, and a follow-up. Inspect retained report excerpts
|
||||
and generated summary when it fails; do not equate job launch with good research.
|
||||
|
||||
Initial live runs verified delivery/navigation/follow-ups but exposed a summary
|
||||
attempt-count bug (fixed: helper requires **1 attempt**, not `max_retries=0`).
|
||||
A later full run was interrupted by an inference endpoint outage. The corrected
|
||||
summary path separately passed a real-model evidence/limitations/citation probe.
|
||||
All targeted Python tests passed (441); real DOM isolation checks passed. A clean
|
||||
full live run with useful retrieved evidence remains to be recorded.
|
||||
@@ -0,0 +1,99 @@
|
||||
# No-RAG clean loop: first diagnostic
|
||||
|
||||
## Setup
|
||||
|
||||
No live UI, service configuration, or weights changed. The standalone loop sends
|
||||
conversation history, native assistant calls and matching tool results directly
|
||||
to the served pre-Heretic model. It never rewrites queries, invents calls, swaps
|
||||
families, or strips output. Invalid calls return errors. Six executions per turn
|
||||
and seven model rounds bound the test.
|
||||
|
||||
Both arms use temperature 0, thinking disabled, 768 output tokens, and the
|
||||
original tool-work evaluator's `tools_for_mode(..., 'compact_contract_v3')`.
|
||||
This matters: the app's plain compact scrubber deletes descriptions, whereas v3
|
||||
retains empirically tested micro-hints. Previous plain-compact tests were not
|
||||
exact reproductions of the passing benchmark setup.
|
||||
|
||||
The 76 tools come from the current app's ten-family inventory, transformed by
|
||||
the original v3 builder. This is not a byte-identical frozen 99-tool benchmark
|
||||
inventory or proof of training-data identity. The report records schema and
|
||||
builder hashes. No schemas are invented for this experiment.
|
||||
|
||||
- **Stable:** same compact inventory on every turn, irrespective of spelling.
|
||||
- **Routed:** same loop, but existing `requested_capabilities` chooses inventory
|
||||
each turn. This isolates that selector; it is not the complete production RAG
|
||||
or Agent UI path. Other production normalizers are absent in both arms.
|
||||
- Private records are synthetic. No real private dispatcher is imported.
|
||||
Only fixture reads and optional public SearXNG calls execute. Other operations
|
||||
return explicit errors, so this does not validate their functionality.
|
||||
- Live search sends the exact model query to local SearXNG Bing/Yep, bypassing
|
||||
app query rewriting/filtering. Source results may vary between arms.
|
||||
|
||||
## Observations, not a blind score
|
||||
|
||||
| Case | Stable compact inventory | Selector arm |
|
||||
|---|---|---|
|
||||
| `whats the current stock mraket` | Selected `web_search`, query `current stock market` | Offered zero tools; declined live lookup |
|
||||
| Exact seeded failed exchange, then `can you look up` | Searched with corrected query | Also searched with corrected query |
|
||||
| Summarize search, explicitly no tools | Answered without tools or permission failure | Same |
|
||||
| Calendar → email → calendar | Recalled second event at 14:30 | Same |
|
||||
| Notes → second note → what does it say | Correct `view` ID and content | Same after fixture correction |
|
||||
| Deliberately irrelevant search result | Did not automatically retry | Did not automatically retry |
|
||||
| User asks for a better source | Refined and executed another search | Proposed search was not offered and was rejected |
|
||||
| Web-disabled lookup | Attempted network access via bash; sandbox rejected it | Invented unsupported current market news without tools |
|
||||
|
||||
The initial stable stock answer listed sources, not current index values. It
|
||||
does not establish that the market question was fully answered. Its subsequent
|
||||
`can you look up` elicited clarification after it had already searched. The
|
||||
separate seeded replay removes that differing-history confound.
|
||||
|
||||
The first notes fixture incorrectly accepted `get/read`, not the real `view`
|
||||
action. Both models selected the correct action, but the fixture rejected it.
|
||||
Those six original turns are invalid for execution comparison. A corrected
|
||||
six-turn rerun succeeded in both arms; the failed evidence is retained.
|
||||
|
||||
Web-off results are a release blocker: removing named web tools alone does not
|
||||
enforce network denial across general-purpose tools. The fixture prevented real
|
||||
execution, but any UI integration must use the real cross-tool permissions and
|
||||
clearly communicate unavailable capabilities. Neither arm is ready for a live
|
||||
switch. Source recovery and grounded completion also remain weak.
|
||||
|
||||
## What this changes
|
||||
|
||||
There is direct evidence that the selector can withhold needed tools, and that
|
||||
the model can repair the misspelled query itself when offered the tool. Clean
|
||||
history also supports the tested topic switches without synthetic substitutions.
|
||||
This supports continuing the clean-path experiment, not retraining or declaring
|
||||
the UI fixed. Full inventory is slower in these requests; overlapping runs and
|
||||
different source content prevent a controlled latency conclusion.
|
||||
|
||||
Next: integrate the clean loop behind a test-only UI profile with real permission
|
||||
enforcement and one renderer, preserving the v3 contract. Test live read-only
|
||||
follow-ups and explicit Web-off behavior before any rollout. Separately compare
|
||||
a generic evidence-check/retry instruction on the weak-result fixture; do not
|
||||
manufacture a retry query in the harness.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Eight boundary tests pass:
|
||||
|
||||
```sh
|
||||
/home/pewds/odysseus-cookbook-fresh/.venv/bin/pytest -q tests/test_clean_tool_loop.py
|
||||
```
|
||||
|
||||
Run with a fresh report filename (existing evidence is never overwritten):
|
||||
|
||||
```sh
|
||||
/home/pewds/odysseus-cookbook-fresh/.venv/bin/python scripts/test_clean_tool_loop.py --live-search --report reports/clean-loop-v3-new-run.json
|
||||
```
|
||||
|
||||
Evidence:
|
||||
|
||||
- `reports/clean-loop-v3-20260909.json`: original 24 turns; notes fixture caveat above.
|
||||
- `reports/clean-loop-v3-stock-seeded-20260909.json`: four matched seeded follow-up turns.
|
||||
- `reports/clean-loop-v3-notes-fixture-corrected-20260909.json`: corrected six notes turns.
|
||||
|
||||
Each report retains model requests, responses, offered inventory and execution
|
||||
results. The `completed` status means the request loop finished, **not** that
|
||||
the answer passed functional evaluation. These are synthetic/public traces, not
|
||||
private user conversations. This test does not measure UI rendering or streaming.
|
||||
@@ -0,0 +1,220 @@
|
||||
# Tools v3 — No-RAG preview
|
||||
|
||||
Select this endpoint in the 7011 model picker, with model
|
||||
`odysseus-qwen3.5-tools-pre-heretic`. This endpoint owns its complete tool loop
|
||||
and enters Agent mode server-side on every turn, including ambiguous follow-ups;
|
||||
it does not depend on the legacy per-message intent classifier. Start a new chat
|
||||
for an uncontaminated comparison. Enable Web for searches. Clean routing is
|
||||
owned by the exact model identity, so both the normal `preheret` endpoint and
|
||||
the `cleanv3` alias use this runtime. Every other model remains on legacy RAG.
|
||||
|
||||
Endpoint ID: `cleanv3`. Its base URL uses the same inference server's Tailscale
|
||||
DNS name, `http://odysseus.tailb895f4.ts.net:18182/v1`, to distinguish it from
|
||||
the original IP-address route when existing chats omit endpoint IDs.
|
||||
|
||||
## Implementation
|
||||
|
||||
- `src/clean_agent_preview.py` is a separate streamed native-tool loop, entered
|
||||
before legacy routing and substitutions. It uses real authenticated tool
|
||||
dispatch, the tool-work `compact_contract_v5` builder, temperature 0,
|
||||
and thinking disabled. No weights change or inference server was started.
|
||||
- The offered tool inventory is stable except for permissions/toggles. Safe,
|
||||
explicit personal creates/updates are enabled for notes, tasks, calendar,
|
||||
memory, skills and documents. Destructive operations, shell/code, outbound
|
||||
email, browser interaction, deployment/admin changes and unrelated-family
|
||||
write substitution remain blocked. No tool or argument substitution is
|
||||
applied by the loop.
|
||||
- Native calls and matching results persist in `clean_v3_turn` metadata so
|
||||
follow-ups use actual evidence. History retains at most eight complete turns,
|
||||
trimming oldest whole turns for size; individual outputs cap at 8000 chars.
|
||||
- Real search still uses the existing search backend and its provider handling;
|
||||
this does not claim that provider quality or every backend transform is fixed.
|
||||
- All routing, privileges and default settings outside this exact Odysseus model
|
||||
remain unchanged. The loop has six execution/eight-round limits.
|
||||
- Write completion is evidence-bound: affirmative success text is replaced
|
||||
unless a private-write tool succeeded during the turn. Proposed call batches
|
||||
are policy-preflighted atomically, so a batch containing a blocked operation
|
||||
cannot partially execute before denial.
|
||||
|
||||
## Verification
|
||||
|
||||
399 focused Python tests passed after route integration. Browser runs r1/r2
|
||||
accidentally exercised the old loop and are not preview evidence. The runner
|
||||
now explicitly asserts `selection_mode=clean_compact_v3_preview`.
|
||||
|
||||
`reports/clean-v3-live-ui-r3-20260909.json` confirms the preview route, real notes
|
||||
execution, correct repetition from history, successful search and no-tool
|
||||
summary, plus visible incremental growth. Its notes assertions were for the
|
||||
old routed contract: they prohibited offering web tools even with Web enabled,
|
||||
and required another notes call for a verbatim repeat. The updated preview
|
||||
checks permit stable offers and accept an exact match to the preceding saved
|
||||
answer without re-execution; execution permissions are still asserted.
|
||||
|
||||
`reports/clean-v3-live-ui-r4-20260909.json` is the corrected four-turn check,
|
||||
including notes with Web off and search with Web on: **4/4 passed**, with the
|
||||
preview selection mode explicitly confirmed on every turn.
|
||||
These are UI smoke tests, not all-family or factual-answer benchmark scores.
|
||||
|
||||
## Disable
|
||||
|
||||
Disabling only endpoint `cleanv3` removes the duplicate picker alias; it does
|
||||
not disable this model-owned runtime. To roll back the runtime, revert the exact
|
||||
model route in `routes/chat_routes.py`. Do not delete weights, adapters, or user
|
||||
chats. The v3 schema builder dependency is
|
||||
`/home/pewds/odysseus-tool-work/scripts/eval_alltools_unseen_compare.py` and its
|
||||
schema-dropout helper; preserve those with this deployment.
|
||||
|
||||
## Expanded UI checks — 2026-09-09
|
||||
|
||||
24 additional turns completed through the preview: 23 automated passes and one
|
||||
checker false alarm. The Cookbook follow-up correctly shortened the previous
|
||||
six-server result to the first three requested names without another call. The
|
||||
checker required either a fresh call or a verbatim repeat; manual inspection
|
||||
confirmed the requested subset. Raw failure evidence is retained, not rescored.
|
||||
|
||||
Covered notes/misspellings/second-note selection, calendar/second-event time,
|
||||
tasks, documents, memory, skills, Cookbook listing, misspelled search, and Web
|
||||
toggle changes. Cross-family flows passed: Germany news → “whats my notes”,
|
||||
notes → “seach current stock mraket news”, and calendar → “now show my noes”.
|
||||
The model chose `current stock market news` itself. Every completed turn's
|
||||
audit confirmed the preview mode. Search source factual accuracy is not graded
|
||||
by this suite, and successful reads do not establish mutation coverage.
|
||||
|
||||
Email was separately attempted but the test guard stopped it because the
|
||||
stable offered inventory exceeded its metadata-only verified scope. Email
|
||||
therefore remains unverified in this expanded run; the guard was not weakened.
|
||||
No production code, service settings or weights changed during these tests.
|
||||
|
||||
Evidence under `reports/`:
|
||||
|
||||
- `clean-v3-broader-ui-20260909.json`: 16 turns, 15 automatic passes, Cookbook caveat.
|
||||
- `clean-v3-topic-switch-ui-20260909.json`: 6/6 passed.
|
||||
- `clean-v3-second-note-ui-20260909.json`: 2/2 passed.
|
||||
- `clean-v3-email-notes-ui-20260909.json`: blocked email attempt; notes not run in that file.
|
||||
|
||||
## Picker route fix
|
||||
|
||||
The previous tests selected sessions through the API, missing a real picker
|
||||
bug: local entries were deduplicated by model ID, hiding alternative endpoints
|
||||
with the same weights. The picker now uses endpoint+model identity for local
|
||||
routes too, displays the endpoint name, and scopes its last-picked send override
|
||||
to the current chat. `/api/sessions` returns owner-filtered endpoint identity
|
||||
for unambiguous saved URLs, so reload labels do not depend on loading the model
|
||||
catalog. Ambiguous identical URLs are not guessed.
|
||||
|
||||
The user-authorized chat `ec0683a2-015f-41d7-aa1f-34135c9640cb` was switched to
|
||||
`cleanv3` using the authenticated session PATCH API; no messages were inserted
|
||||
and no tool actions ran in that chat. Defaults and other chats were unchanged.
|
||||
|
||||
The runner's `--picker-route true` starts on the original route, clicks the
|
||||
preview in the real picker, sends a greeting, reloads the chat permalink, then
|
||||
asks for notes. Early picker/reload reports are incomplete, not passes: their
|
||||
label check exposed the unloaded-catalog issue. Focused route/picker/history
|
||||
tests: 16 passed.
|
||||
|
||||
Final picker test: `reports/clean-v3-picker-reload-r5-20260909.json`, **2/2
|
||||
passed**. Real picker click, greeting, permalink reload, and notes follow-up
|
||||
all confirmed the preview route. The label survived reload. R4 retained a
|
||||
history/DOM mismatch from sending before restored history was ready; the final
|
||||
driver explicitly waits for the saved first answer to render before sending.
|
||||
This does not claim a general fix for sending during unfinished history loading.
|
||||
|
||||
## Native image/VL status
|
||||
|
||||
The inference launcher previously set `--limit-mm-per-prompt` to zero images,
|
||||
so vLLM rejected attachments before the model saw them. The durable Odysseus
|
||||
launcher now permits up to three images per prompt; video remains disabled.
|
||||
|
||||
`reports/clean-v3-vl-live-r4-20260909.json` proves the real 7011 attachment
|
||||
path, clean compact route, object/color/spatial recognition, permalink reload,
|
||||
and ambiguous image follow-up. Those checks pass. Exact OCR of the deterministic
|
||||
`ODYSSEUS 42` heading fails in both the untouched Qwen 3.5 9B base and the
|
||||
fine-tune, so it remains a base/runtime capability limitation rather than a
|
||||
fine-tune regression or harness failure.
|
||||
|
||||
The same native path also passes JPEG and lossless WebP transport, object
|
||||
recognition, reload, and follow-up grounding. Evidence:
|
||||
`reports/clean-v3-vl-jpeg-r1-20260909.json` and
|
||||
`reports/clean-v3-vl-webp-r1-20260909.json`. Both remain `partial` only because
|
||||
the shared OCR check fails.
|
||||
|
||||
## Reversible write check
|
||||
|
||||
`scripts/verify_clean_v3_write.mjs` runs against only `sft_alex_creator`. It
|
||||
creates one UUID-named note through the real 7011 UI, verifies that exact row,
|
||||
requests a destructive bulk deletion, verifies the row still exists, and then
|
||||
deletes only its own test row through the authenticated API. The cleanup is
|
||||
verified by a 404 lookup.
|
||||
|
||||
Final evidence: `reports/clean-v3-write-ui-r8-20260909.json`, **passed**. Both
|
||||
turns reported `selection_mode=clean_compact_v3_preview`; creation executed via
|
||||
`manage_notes(action=add)`, the destructive action did not execute, and the
|
||||
canonical response was “No changes were made.” The earlier r3/r5 files are
|
||||
startup/placement failures, while r4/r6/r7 retained genuine intermediate
|
||||
harness and verifier failures; none should be interpreted as passes.
|
||||
|
||||
## Stateful, search, and email checks
|
||||
|
||||
The reversible stateful runner passes all six mutation families in one run:
|
||||
calendar, notes, tasks, documents, memory, and skills (**6/6**). Each flow
|
||||
creates a UUID-only artifact through the real Agent UI, verifies it by
|
||||
owner-scoped API, applies a noun-free correction, verifies persistence, and
|
||||
removes only that artifact. A direct database audit found zero active synthetic
|
||||
calendar, note, task, or document rows afterward.
|
||||
|
||||
The document failure was harness-owned. Compact description dropout left a
|
||||
vague free-form `command` field, error envelopes defaulted to exit code 0, and
|
||||
the clean loop dropped the active document ID. Compact v5 now exposes only
|
||||
required structured `edits`, reports errors truthfully, and executes against
|
||||
the request's explicit active document. Fresh document and combined stateful
|
||||
runs pass.
|
||||
|
||||
Search Web-toggle combinations `00`, `01`, `10`, and `11` pass **8/8** across
|
||||
two turns. A web question can no longer silently enable Bash because it says
|
||||
“official source”, and an unavailable Web capability exposes no unrelated
|
||||
fallback family. The quality suite passes **3/3**: evidence reuse without a
|
||||
second call, explicit official-page inspection with `web_fetch`, correction of
|
||||
“stock mraket” in actual search arguments, and a truthful unsupported result
|
||||
for a synthetic company.
|
||||
|
||||
Production-path email reads pass **3/3** through the running email MCP: account
|
||||
list, latest inbox list, and referential read of the first result. The report
|
||||
retains no account names, addresses, subjects, bodies, prompts, or answers.
|
||||
|
||||
Post-fix representative direct/follow-up coverage also passes for every family:
|
||||
notes/calendar 4/4, tasks/documents/memory/skills/Cookbook/search/shell 14/14,
|
||||
and email 3/3 in its privacy-preserving runner. The combined legacy verifier's
|
||||
metadata-only email guard correctly refused its broader stable inventory; that
|
||||
stopped report is not counted as a model failure.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `reports/clean-v3-stateful-all-r3-20260909.json`
|
||||
- `reports/clean-v3-stateful-documents-r2-20260909.json`
|
||||
- `reports/clean-v3-search-toggle-final-r6-20260909.json`
|
||||
- `reports/clean-v3-search-quality-r3-20260909.json`
|
||||
- `reports/clean-v3-email-read-r1-20260909.json`
|
||||
- `reports/clean-v3-ten-family-tail-postfix-r1-20260909.json`
|
||||
|
||||
These checks verify routing, execution, persistence, follow-up, and selected
|
||||
answer-quality invariants. They are not yet the sealed all-action ship score.
|
||||
|
||||
## Compact v5 and corrected contract evidence
|
||||
|
||||
Compact v5 keeps the compact-v3 surface and adds only development-positive
|
||||
field hints for Email, Search/Hugging Face quant selection, and Shell/files.
|
||||
A Calendar date hint regressed development and was excluded. The Python tool now
|
||||
emits one final bare expression, REPL-style, without duplicating explicit
|
||||
`print(...)`; this turns otherwise correct computation calls into visible tool
|
||||
evidence for all models.
|
||||
|
||||
Under frozen scorer `odysseus.contract.v2.5`, development is 327/344 raw
|
||||
(95.06%) and 327/336 scorable (97.32%). Sealed blind is 311/344 raw (90.41%)
|
||||
and 311/336 scorable (92.56%), with zero reasoning leakage. Calendar, Shell,
|
||||
and Tasks remain below the 90% family ship floor, so the model is not yet a
|
||||
full benchmark ship candidate.
|
||||
|
||||
Fresh post-deploy real-UI evidence passes: stateful flows 6/6, Email 3/3,
|
||||
Search quality/recovery 3/3, private browser 3/3, and VL workflow 3/3. The
|
||||
Search check accepts a failed attempt only when a later tool succeeds and the
|
||||
final answer remains grounded.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Regular-model tool compatibility
|
||||
|
||||
Last verified: 2026-09-09 through the authenticated 7011 Agent UI as
|
||||
`sft_alex_creator`.
|
||||
|
||||
This is the legacy-RAG track. The exact model
|
||||
`odysseus-qwen3.5-tools-pre-heretic` is excluded and remains on its model-owned
|
||||
clean compact runtime.
|
||||
|
||||
## Current baseline
|
||||
|
||||
| Endpoint | Model | Ten-family result | State |
|
||||
|---|---|---:|---|
|
||||
| DeepSeek | `deepseek-v4-flash` | 10/10 | passed |
|
||||
| DeepSeek | `deepseek-v4-pro` | 10/10 | passed |
|
||||
| OpenAI | `gpt-5.5` | 10/10 | passed |
|
||||
| OpenAI | `gpt-5.6-sol` | 10/10 | passed |
|
||||
| OpenAI | `gpt-5.6-terra` | 10/10 | passed |
|
||||
| OpenAI | `gpt-5.6-luna` | 10/10 | passed |
|
||||
| OpenRouter | `moonshotai/kimi-k3` | 10/10 | passed |
|
||||
| OpenRouter | `x-ai/grok-4.5` | 10/10 | passed |
|
||||
| OpenRouter | `qwen/qwen3-vl-235b-a22b-instruct` | 10/10 | passed |
|
||||
| OpenRouter | `openai/gpt-5-image` | n/a | image generation; chat tools unsupported |
|
||||
| Local `100.69.120.65:8062` | `Qwen/Qwen3.5-9B` | not run | endpoint unavailable |
|
||||
| Local `100.69.120.65:8062` | `GLM-5.3-Flash-Alis-MLX-4bit` | not run | endpoint unavailable |
|
||||
|
||||
The ten-family baseline covers one read-only functional turn each for notes,
|
||||
calendar, email accounts, tasks, documents, memory, skills, Cookbook/admin,
|
||||
web search, and shell. It verifies the legacy route, expected native tool call,
|
||||
execution result, visible UI answer, and absence of reasoning leakage. It is not
|
||||
yet a claim that every mutation/action variant, typo, or follow-up passes.
|
||||
|
||||
## Typo and follow-up profile
|
||||
|
||||
The stricter real-UI profile sends one misspelled read-only request to every
|
||||
family, followed immediately by a noun-free reference to the returned result.
|
||||
Read-only follow-ups must not call any tool; search follow-ups may either use
|
||||
the existing evidence or fetch the prior link. Across the nine chat-capable API
|
||||
models, the composited post-repair result is **178/180 turns (98.89%)**:
|
||||
|
||||
| Model | Conversation result |
|
||||
|---|---:|
|
||||
| `deepseek-v4-flash` | 20/20 |
|
||||
| `deepseek-v4-pro` | 20/20 |
|
||||
| `gpt-5.5` | 20/20 |
|
||||
| `gpt-5.6-sol` | 20/20 |
|
||||
| `gpt-5.6-terra` | 20/20 |
|
||||
| `gpt-5.6-luna` | 18/20 |
|
||||
| `moonshotai/kimi-k3` | 20/20 |
|
||||
| `x-ai/grok-4.5` | 20/20 |
|
||||
| `qwen/qwen3-vl-235b-a22b-instruct` | 20/20 |
|
||||
|
||||
Luna's only remaining family miss is a deliberately misspelled Shell request.
|
||||
The correct-spelling baseline passes. The harness does not auto-execute a shell
|
||||
command to hide that model-owned limitation.
|
||||
|
||||
The shared repair recognizes a uniquely misspelled action verb and family noun,
|
||||
then seals only declared safe private reads with immutable canonical arguments.
|
||||
This repaired Tasks/Documents/Memory and adjacent read families across providers
|
||||
without widening mutation or Shell authority. A compact native-tool instruction
|
||||
also tells regular API models to map clear typos to a currently offered tool.
|
||||
|
||||
Conversation evidence:
|
||||
|
||||
- `reports/regular-model-conversation-flash-r3-20260909.json`
|
||||
- `reports/regular-model-conversation-remaining-r1-20260909.json`
|
||||
- `reports/regular-model-conversation-repair-r1-20260909.json`
|
||||
- `reports/regular-model-conversation-shell-r1-20260909.json`
|
||||
- `reports/regular-model-conversation-qwen-repair-r1-20260909.json`
|
||||
- `reports/regular-model-conversation-qwen-tail-r1-20260909.json`
|
||||
- `reports/regular-model-conversation-qwen-search-r1-20260909.json`
|
||||
|
||||
Evidence:
|
||||
|
||||
- `reports/regular-model-tools-provider-final-r4-20260909.json` — Flash, GPT-5.5, Kimi: 30/30.
|
||||
- `reports/regular-model-tools-repair-r3-20260909.json` — Pro and Sol: 20/20; retained Qwen pre-final 9/10 miss.
|
||||
- `reports/regular-qwen-vl-full-r4-20260909.json` — Qwen-VL final family-switch run: 10/10.
|
||||
- `reports/regular-model-tools-remaining-20260909.json` — Terra, Luna, Grok: 30/30; records unavailable/unsupported models and pre-repair failures.
|
||||
- `reports/regular-model-tools-postfix-r1-20260909.json` — post-hardening
|
||||
rerun: nine chat-capable API models passed 90/90 family turns with zero model
|
||||
failures. Its overall status is non-passing only because the two configured
|
||||
local endpoints were offline; the image-only model remains unsupported.
|
||||
|
||||
## Family switch and page inspection
|
||||
|
||||
The six-turn switch/back flow covers notes → calendar → notes from prior
|
||||
evidence → web search → explicit `web_fetch` → calendar from prior evidence.
|
||||
All nine API models have a clean 6/6 reproduction (**54/54**). Kimi skipped
|
||||
search once in the retained first run and passed a fresh reproduction; that
|
||||
variability remains visible instead of being erased.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `reports/regular-model-switchback-flash-r2-20260909.json`
|
||||
- `reports/regular-model-switchback-remaining-r1-20260909.json`
|
||||
- `reports/regular-model-switchback-kimi-r1-20260909.json`
|
||||
|
||||
## Repair that produced the clean baseline
|
||||
|
||||
Regular models no longer inherit up to three stale tool families into every
|
||||
explicit new request. Referential follow-ups still resolve from typed recent
|
||||
tool evidence, while explicit family switches receive the current family only.
|
||||
Safe required reads use `active_capabilities`, so stale offered context cannot
|
||||
disable their immutable operation. The stream layer also stops an exact long
|
||||
block repeated twice instead of waiting for a provider's full timeout.
|
||||
|
||||
The composer no longer treats generic words such as “source”, “system”, “app”,
|
||||
or “review” as authority to silently enable Bash. Explicit shell, terminal,
|
||||
repository, code-file, and direct coding requests retain workspace
|
||||
auto-escalation. This is a shared UI authority fix, not a model-name exception.
|
||||
|
||||
Run a bounded subset with:
|
||||
|
||||
```sh
|
||||
MODELS='deepseek-v4-flash,gpt-5.5' \
|
||||
FAMILIES='notes,calendar' WORKERS=2 \
|
||||
REPORT_PATH=reports/regular-model-check.json \
|
||||
node scripts/verify_regular_model_tools.mjs
|
||||
```
|
||||
|
||||
Set `PROFILE=conversation` to run the typo plus follow-up profile.
|
||||
|
||||
The runner discovers only enabled pinned models (visible cached local models
|
||||
when no pins exist), retains no tool outputs or private rows, and deletes only
|
||||
the exact sessions it creates.
|
||||
@@ -0,0 +1,92 @@
|
||||
# Search and compact-tool experiment — 2026-09-09
|
||||
|
||||
## Decision
|
||||
|
||||
Keep the normal routed profile on 7011. The all-tools compact experiment is
|
||||
implemented but **disabled**: direct routing success did not translate into a
|
||||
working Agent UI. Do not retrain or promote a profile on these measurements.
|
||||
|
||||
## Changes
|
||||
|
||||
- Short public-web lookups on the target model have an execution budget: two
|
||||
distinct token-normalized searches, one fetch, and up to three browser calls
|
||||
after the two searches. This bounds attempts, not just recovery prose. Existing
|
||||
permissions still apply; this does not make unavailable tools executable.
|
||||
- Failed/weak searches reach the model for evaluation and query refinement,
|
||||
instead of the earlier unconditional terminal evidence veto. Some legacy
|
||||
heuristics and official-site shortcuts remain; this is not a completed rewrite.
|
||||
- Search providers retain query/engine/date provenance. Unconfigured credentialed
|
||||
fallbacks are skipped. When SearXNG is the sole configured usable provider, Yep
|
||||
on the same instance is an additional fallback.
|
||||
- An unavailable warm-only family no longer vetoes an otherwise ordinary reply.
|
||||
- The all-tools experiment offers the trained compact inventory subject to
|
||||
permissions. It requires the exact test-owner environment flag and exact model
|
||||
match. The temporary service flag was removed after failed UI testing.
|
||||
|
||||
## Evidence and limits
|
||||
|
||||
| Measurement | Result | What it establishes |
|
||||
|---|---|---|
|
||||
| Focused Python regression suite | 401 passed | Covered policy, contract, provider and recovery-budget behavior |
|
||||
| Direct family-only compact schemas | 10/10 tool routing | Small public smoke test, not functional or blind accuracy |
|
||||
| Direct all-family compact schemas | 10/10 tool routing | Inventory did not break these first calls; roughly 3–4x slower in this run |
|
||||
| Full compact Agent UI, revision 3 | All eight turns failed one or more checks | Not suitable for activation; leaks, duplicate/incorrect rendering or missing expected calls |
|
||||
| Normal-profile final UI control | Five passed checks, two product failures, one capture error | Not accepted; suite status incomplete |
|
||||
| Clean synthetic notes tool-result continuation | Clean answer with both schema sizes | Model can continue correctly on that isolated input, not proof that UI failure is solely harness |
|
||||
|
||||
Direct probes used temperature 0; the UI target-model sampling path can cap at
|
||||
0.2. Prompts, history and tool-result serialization also differ. Match those
|
||||
before attributing UI failures to weights versus harness. Existing UI checks are
|
||||
not a grounded factual-answer benchmark. Unit tests are not UI acceptance.
|
||||
|
||||
Normal-profile control details: notes initial, both calendar turns, AI search
|
||||
initial, and history initial passed the automated checks. Notes follow-up hit a
|
||||
Playwright `Network.getResponseBody` capture error and is inconclusive. AI search
|
||||
follow-up explicitly requested a summary with no tools, but the contract still
|
||||
required `search_browser` and returned a permission failure. The history search
|
||||
follow-up failed the visible-leak check. These are separate from source relevance;
|
||||
the earlier warm-only fix did not cover classification as an active requirement.
|
||||
There is no matched pre-change control establishing a net improvement.
|
||||
|
||||
Provider isolation bypassed app relevance filters. Bing general often returned
|
||||
broad or unrelated results despite the full query. Google/Mojeek returned no
|
||||
results, DDG hit CAPTCHA, and Presearch timed out. Yep returned useful PostgreSQL
|
||||
documentation, but was weak or empty for several other questions. Engine health
|
||||
and source quality remain unresolved. Fallback cannot help when an earlier weak
|
||||
result survives filtering; there is no claim of universal relevance here.
|
||||
|
||||
## Reproduce and inspect
|
||||
|
||||
- `scripts/audit_search_pipeline.py`: raw provider comparison, no model.
|
||||
- `scripts/compare_compact_tool_inventory.py`: read-only model schema comparison;
|
||||
proposed calls are never executed.
|
||||
- `scripts/verify_agent_turn_contract.mjs`: real 7011 Agent UI and persisted-history
|
||||
checks. Use the dedicated test account; reports can contain private tool data.
|
||||
- `reports/search-provider-isolation.json`, `reports/search-yep-isolation.json`:
|
||||
public provider evidence.
|
||||
- `reports/compact-inventory-ablation.json`: direct routing probe.
|
||||
- `reports/full-compact-ui-audit-r3.json`: completed rejected UI experiment.
|
||||
Earlier experiment reports include an initialization error and an aborted run;
|
||||
do not combine them into an accuracy score.
|
||||
- `reports/routed-control-ui-audit-final.json`: normal-profile control replay;
|
||||
eight attempts, incomplete because of the capture error; failures retained.
|
||||
|
||||
Focused suite:
|
||||
|
||||
```sh
|
||||
/home/pewds/odysseus-cookbook-fresh/.venv/bin/pytest -q tests/test_turn_contract.py tests/test_turn_contract_integration.py tests/test_service_search_provider_guards.py tests/test_web_recovery_budget.py tests/test_tool_policy.py
|
||||
```
|
||||
|
||||
UI replay (read-only prompts, creates test chats):
|
||||
|
||||
```sh
|
||||
node scripts/verify_agent_turn_contract.mjs --families notes,calendar,search_ai,search_history --pairs notes:11,calendar:11,search_ai:11,search_history:11 --max-turns 8 --total-ms 360000 --turn-ms 45000 --report reports/routed-control-ui-audit-final.json
|
||||
```
|
||||
|
||||
## Next discriminating test
|
||||
|
||||
Replay the same captured UI request directly, preserving sampling, compact
|
||||
schemas, history and tool results. Then change one layer at a time. Separately
|
||||
score retrieved-source relevance and supported answers. Replace failing generic
|
||||
boundaries only when the replay identifies them; do not add rules for individual
|
||||
user phrasings or treat successful tool routing as successful execution.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Typo-tolerant tool routing audit
|
||||
|
||||
The 9B SFT model was not retrained. This audit targets the earlier harness
|
||||
stage that decides which complete tool families the model is allowed to see.
|
||||
|
||||
## Method
|
||||
|
||||
- Source prompts: real `sft_alex_creator` sessions from `a37dcb3b-...` onward.
|
||||
- Labels: recorded single-family tool calls, excluding mixed/ambiguous traces.
|
||||
- Variants: deletion, adjacent transposition, duplicated character,
|
||||
keyboard-neighbor substitution, and accidental word split.
|
||||
- Split: deterministic SHA-256 assignment before scoring (75% dev, 25% blind).
|
||||
- Safety: static routing only; no historical mutation or send action is replayed.
|
||||
- Acceptance: at least 95% blind exact-family accuracy and below 1% blind
|
||||
wrong-family authorization. Abstention is measured separately.
|
||||
|
||||
## Results
|
||||
|
||||
| Router | Dev family supplied | Blind family supplied | Blind exact | Blind wrong-family |
|
||||
|---|---:|---:|---:|---:|
|
||||
| Previous exact rules | 63.64% | 65.69% | — | — |
|
||||
| Conservative fuzzy fallback r4 | 96.31% | 98.31% | 96.62% | 0.00% |
|
||||
| Final router + safe-read repair | 98.31% | 98.73% | 97.05% | 0.00% |
|
||||
|
||||
The fallback runs only for action/lookup-shaped requests, resolves exactly one
|
||||
nearby family term, and abstains on ambiguity. Conceptual questions remain
|
||||
tool-free. Complete family schemas are still selected by the immutable turn
|
||||
contract; fuzzy matching never chooses an individual tool or its arguments.
|
||||
|
||||
Authoritative machine reports:
|
||||
|
||||
- `reports/typo-tool-routing-baseline-20260909.json`
|
||||
- `reports/typo-tool-routing-fuzzy-r4-20260909.json`
|
||||
- `reports/typo-tool-routing-final-20260909.json`
|
||||
- `reports/post-followup-agent-80-20260909.json`
|
||||
- `reports/post-typo-routing-agent-80-20260909.json`
|
||||
- `reports/live-typo-agent-20-20260909.json`
|
||||
- `reports/live-typo-unresolved-r3-20260909.json`
|
||||
- `reports/live-typo-agent-final-20-20260909.json`
|
||||
- `reports/post-typo-safe-read-agent-final-80-20260909.json`
|
||||
|
||||
## Live 7011 findings
|
||||
|
||||
The post-deployment standard matrix passed 80/80 through the real Agent UI.
|
||||
The first read-only typo matrix then attempted 17 of 20 planned turns before
|
||||
its total-time limit. Initial Notes, Calendar, Email, Tasks, Documents, and
|
||||
Cookbook calls passed. Completed failing turns still had the correct family
|
||||
and required tool in `turn_contract.offered`; the 9B model sometimes answered
|
||||
without calling that offered tool. Memory and Search also exposed timeouts.
|
||||
|
||||
This separates three failure classes:
|
||||
|
||||
1. **Tool injection:** addressed by conservative fuzzy family routing; blind
|
||||
exact routing is 96.62% with zero blind wrong-family authorizations.
|
||||
2. **Required read execution:** a correctly offered safe list/refresh tool can
|
||||
still be skipped by the model, especially after a typo or on “list those
|
||||
again” follow-ups. This should be handled by the generic deterministic
|
||||
safe-read path, not additional prompt-specific hints.
|
||||
3. **Runtime timeout:** Search and one Memory follow-up require loop/backend
|
||||
diagnosis. A timeout is not counted as a model-accuracy or routing result.
|
||||
|
||||
The generic safe-read parser and search-family precedence were then repaired.
|
||||
The previously unresolved Calendar, Email, Search, and Shell/Files cases passed
|
||||
8/8. The complete typo matrix passed 20/20, including initial requests and
|
||||
follow-ups for all ten families. The final standard Agent UI compatibility
|
||||
matrix passed 80/80 across family, Web-toggle, and follow-up combinations.
|
||||
|
||||
The broad routing regression suite passed 458 tests. The model was not
|
||||
retrained and no DeepSeek API was used: the measured defect was in harness
|
||||
family selection and deterministic safe-read execution, upstream of the
|
||||
model. All 1,535 unique labeled historical turns were statically audited to
|
||||
mine failure categories. Historical write/send/delete actions were not replayed
|
||||
against live data; live verification used the deduplicated read-only matrices.
|
||||
|
Before Width: | Height: | Size: 3.0 MiB |
|
Before Width: | Height: | Size: 3.4 MiB |
|
Before Width: | Height: | Size: 1.1 MiB |
|
Before Width: | Height: | Size: 1003 KiB |
|
Before Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 2.5 MiB |
@@ -0,0 +1,26 @@
|
||||
# Skills lifecycle
|
||||
|
||||
The UI exposes All, Built-in, Approved, and Draft. Draft includes archived
|
||||
records so they remain inspectable and recoverable. Built-ins are not audited.
|
||||
Approved means published, passing, at the configured confidence threshold,
|
||||
and not marked unnecessary. Baseline speed measurements remain evidence, not
|
||||
an additional hidden UI approval gate.
|
||||
|
||||
Automatic audits process at most eight eligible records at a time, oldest first.
|
||||
New records are eligible immediately; inconclusive checks retry after a day;
|
||||
failed repairs retry after a week. Passed, duplicate-skipped, and archived records
|
||||
are excluded. Existing daily Skills Audit tasks drive this queue. Their quiet
|
||||
window deferrals propagate to the scheduler rather than becoming task failures.
|
||||
Automatic runs use background model scheduling. Existing self-repair and teacher
|
||||
repair stages remain in place; failed candidates remain drafts.
|
||||
|
||||
The skill index advertises short descriptions; the agent loads a relevant full
|
||||
procedure on demand and applies already-injected procedures directly. Extraction
|
||||
prefers verified discoveries and specific workarounds over routine tool usage.
|
||||
|
||||
Reference reviewed: NousResearch/hermes-agent, MIT license, commit
|
||||
cfdbbb6e35010ace89fbe8243ee82fa4de143e10, cloned to
|
||||
/home/pewds/hermes-skills-reference. In particular tools/skills_tool.py and
|
||||
agent/prompt_builder.py use progressive disclosure and task-triggered procedure
|
||||
loading. These changes adapt that approach to Odysseus's existing registry;
|
||||
no Hermes implementation code was copied.
|
||||
@@ -102,6 +102,7 @@ python3 ~/.claude/skills/odysseus/scripts/odysseus_api.py POST /api/codex/memory
|
||||
|
||||
## Email draft + send
|
||||
|
||||
- Prefer `POST /api/codex/emails/draft-document` for agent-written email replies. It creates an editable Odysseus Document with `language: "email"` and does not touch IMAP/send.
|
||||
- `POST /api/codex/emails/draft` — body matches `SendEmailRequest` (`to`, `cc`, `bcc`, `subject`, `body`, `body_html`, `attachments`, `account_id`, `in_reply_to`, `references`). Requires `email:draft` (or `email:send`).
|
||||
- `POST /api/codex/emails/send` — same body. Requires `email:send`. Never send without explicit user instruction.
|
||||
|
||||
|
||||
@@ -17,6 +17,11 @@ def _usage() -> int:
|
||||
print(" odysseus_api.py todos add TITLE", file=sys.stderr)
|
||||
print(" odysseus_api.py emails list [limit]", file=sys.stderr)
|
||||
print(" odysseus_api.py emails read UID", file=sys.stderr)
|
||||
print(" odysseus_api.py emails draft-doc JSON_PAYLOAD", file=sys.stderr)
|
||||
print(" odysseus_api.py documents list [limit]", file=sys.stderr)
|
||||
print(" odysseus_api.py documents read DOC_ID", file=sys.stderr)
|
||||
print(" odysseus_api.py documents create JSON_PAYLOAD", file=sys.stderr)
|
||||
print(" odysseus_api.py documents delete DOC_ID", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook tasks", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook servers", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook cached [HOST]", file=sys.stderr)
|
||||
@@ -79,6 +84,33 @@ def main() -> int:
|
||||
method = "GET"
|
||||
path = f"/api/codex/emails/{sys.argv[3]}"
|
||||
body = None
|
||||
elif action in ("draft-doc", "draft_document") and len(sys.argv) >= 4:
|
||||
method = "POST"
|
||||
path = "/api/codex/emails/draft-document"
|
||||
body = " ".join(sys.argv[3:])
|
||||
else:
|
||||
return _usage()
|
||||
elif command in ("documents", "docs"):
|
||||
if len(sys.argv) < 3:
|
||||
return _usage()
|
||||
action = sys.argv[2].lower()
|
||||
if action == "list":
|
||||
method = "GET"
|
||||
limit = sys.argv[3] if len(sys.argv) >= 4 else "50"
|
||||
path = f"/api/codex/documents?limit={limit}"
|
||||
body = None
|
||||
elif action == "read" and len(sys.argv) >= 4:
|
||||
method = "GET"
|
||||
path = f"/api/codex/documents/{sys.argv[3]}"
|
||||
body = None
|
||||
elif action == "create" and len(sys.argv) >= 4:
|
||||
method = "POST"
|
||||
path = "/api/codex/documents"
|
||||
body = " ".join(sys.argv[3:])
|
||||
elif action == "delete" and len(sys.argv) >= 4:
|
||||
method = "DELETE"
|
||||
path = f"/api/codex/documents/{sys.argv[3]}"
|
||||
body = None
|
||||
else:
|
||||
return _usage()
|
||||
elif command == "cookbook":
|
||||
|
||||
@@ -17,6 +17,11 @@ def _usage() -> int:
|
||||
print(" odysseus_api.py todos add TITLE", file=sys.stderr)
|
||||
print(" odysseus_api.py emails list [limit]", file=sys.stderr)
|
||||
print(" odysseus_api.py emails read UID", file=sys.stderr)
|
||||
print(" odysseus_api.py emails draft-doc JSON_PAYLOAD", file=sys.stderr)
|
||||
print(" odysseus_api.py documents list [limit]", file=sys.stderr)
|
||||
print(" odysseus_api.py documents read DOC_ID", file=sys.stderr)
|
||||
print(" odysseus_api.py documents create JSON_PAYLOAD", file=sys.stderr)
|
||||
print(" odysseus_api.py documents delete DOC_ID", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook tasks", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook servers", file=sys.stderr)
|
||||
print(" odysseus_api.py cookbook cached [HOST]", file=sys.stderr)
|
||||
@@ -79,6 +84,33 @@ def main() -> int:
|
||||
method = "GET"
|
||||
path = f"/api/codex/emails/{sys.argv[3]}"
|
||||
body = None
|
||||
elif action in ("draft-doc", "draft_document") and len(sys.argv) >= 4:
|
||||
method = "POST"
|
||||
path = "/api/codex/emails/draft-document"
|
||||
body = " ".join(sys.argv[3:])
|
||||
else:
|
||||
return _usage()
|
||||
elif command in ("documents", "docs"):
|
||||
if len(sys.argv) < 3:
|
||||
return _usage()
|
||||
action = sys.argv[2].lower()
|
||||
if action == "list":
|
||||
method = "GET"
|
||||
limit = sys.argv[3] if len(sys.argv) >= 4 else "50"
|
||||
path = f"/api/codex/documents?limit={limit}"
|
||||
body = None
|
||||
elif action == "read" and len(sys.argv) >= 4:
|
||||
method = "GET"
|
||||
path = f"/api/codex/documents/{sys.argv[3]}"
|
||||
body = None
|
||||
elif action == "create" and len(sys.argv) >= 4:
|
||||
method = "POST"
|
||||
path = "/api/codex/documents"
|
||||
body = " ".join(sys.argv[3:])
|
||||
elif action == "delete" and len(sys.argv) >= 4:
|
||||
method = "DELETE"
|
||||
path = f"/api/codex/documents/{sys.argv[3]}"
|
||||
body = None
|
||||
else:
|
||||
return _usage()
|
||||
elif command == "cookbook":
|
||||
|
||||
@@ -102,6 +102,7 @@ python3 integrations/codex/scripts/odysseus_api.py POST /api/codex/memory '{"tex
|
||||
|
||||
## Email draft + send
|
||||
|
||||
- Prefer `POST /api/codex/emails/draft-document` for Codex-written email replies. It creates an editable Odysseus Document with `language: "email"` and does not touch IMAP/send.
|
||||
- `POST /api/codex/emails/draft` — body matches `SendEmailRequest` (`to`, `cc`, `bcc`, `subject`, `body`, `body_html`, `attachments`, `account_id`, `in_reply_to`, `references`). Requires `email:draft` (or `email:send`).
|
||||
- `POST /api/codex/emails/send` — same body. Requires `email:send`. Never send without explicit user instruction.
|
||||
|
||||
|
||||
@@ -30,14 +30,26 @@ function Fail($msg) {
|
||||
exit 1
|
||||
}
|
||||
|
||||
function Test-WindowsBashStub($path) {
|
||||
if (-not $path) { return $false }
|
||||
$lowered = $path.ToLowerInvariant()
|
||||
foreach ($stub in @("system32\bash.exe", "sysnative\bash.exe", "windowsapps\bash.exe")) {
|
||||
if ($lowered.Contains($stub)) { return $true }
|
||||
}
|
||||
return $false
|
||||
}
|
||||
|
||||
function Find-GitBash {
|
||||
$cmd = Get-Command bash -ErrorAction SilentlyContinue
|
||||
if ($cmd) { return $cmd.Source }
|
||||
if ($cmd -and -not (Test-WindowsBashStub $cmd.Source)) { return $cmd.Source }
|
||||
|
||||
$roots = @()
|
||||
foreach ($name in @("ProgramFiles", "ProgramW6432", "ProgramFiles(x86)", "LocalAppData")) {
|
||||
$base = [Environment]::GetEnvironmentVariable($name)
|
||||
if ($base) { $roots += (Join-Path $base "Git") }
|
||||
if ($base) {
|
||||
$roots += (Join-Path $base "Git")
|
||||
if ($name -eq "LocalAppData") { $roots += (Join-Path $base "Programs\Git") }
|
||||
}
|
||||
}
|
||||
$roots += @("C:\Program Files\Git", "C:\Program Files (x86)\Git")
|
||||
|
||||
@@ -93,6 +105,14 @@ if (-not $pyExe) {
|
||||
}
|
||||
}
|
||||
|
||||
if ($pyExe -like "*WindowsApps*python.exe") {
|
||||
$pyCmd = Get-Command py -ErrorAction SilentlyContinue
|
||||
if ($pyCmd) {
|
||||
$pyExe = $pyCmd.Source
|
||||
$pyArgs = @("-3.11")
|
||||
}
|
||||
}
|
||||
|
||||
if (-not $pyExe) {
|
||||
Fail "Couldn't find Python 3.11+ for Windows setup. Install Python 3.11+ (or open the Python launcher with 'py -3.11') from https://www.python.org/downloads/, then re-run this script."
|
||||
}
|
||||
@@ -129,7 +149,24 @@ if (-not (Find-GitBash)) {
|
||||
Write-Host " https://git-scm.com/download/win" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# 6. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH)
|
||||
# 6. Point CUDA_PATH at a real CUDA toolkit so GPU llama-cpp-python can import.
|
||||
$cudaBase = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA"
|
||||
if (Test-Path $cudaBase) {
|
||||
$cudaBest = Get-ChildItem $cudaBase -Directory -ErrorAction SilentlyContinue |
|
||||
Where-Object { Test-Path (Join-Path $_.FullName "bin") } |
|
||||
Sort-Object { try { [version]($_.Name -replace "^v", "") } catch { [version]"0.0" } } -Descending |
|
||||
Select-Object -First 1
|
||||
if ($cudaBest) {
|
||||
$env:CUDA_PATH = $cudaBest.FullName
|
||||
Write-Host ("Using CUDA_PATH = " + $cudaBest.FullName) -ForegroundColor Cyan
|
||||
}
|
||||
}
|
||||
|
||||
# 7. Start the server (use `python -m uvicorn` - bare `uvicorn` may not be on PATH)
|
||||
# -Port only reaches uvicorn as a flag. Everything that builds a URL for this
|
||||
# instance - internal_api_base(), companion pairing, the MCP OAuth callback -
|
||||
# reads APP_PORT, so set it too or they all assume 7000.
|
||||
$env:APP_PORT = $Port
|
||||
Write-Step ("Starting Odysseus at http://{0}:{1}" -f $BindHost, $Port)
|
||||
Write-Host "Press Ctrl+C to stop."
|
||||
Write-Host ""
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# launcher.py
|
||||
"""Dedicated entrypoint for the standalone Windows portable launcher.
|
||||
|
||||
Handles:
|
||||
- Immediate GUI splash screen creation using tkinter.
|
||||
- Suppressing console stream crashes in windowed GUI mode via NullWriter.
|
||||
- Spawning system tray icon via pystray and Pillow (lazy-loaded).
|
||||
- Auto-opening default browser pointing to the running backend.
|
||||
- Launching the FastAPI server (importing and running app.py).
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import webbrowser
|
||||
|
||||
# Define a dummy NullWriter to suppress standard stream crashes (isatty etc.) in GUI mode
|
||||
class NullWriter:
|
||||
def write(self, text):
|
||||
pass
|
||||
def flush(self):
|
||||
pass
|
||||
def isatty(self):
|
||||
return False
|
||||
|
||||
if sys.stdout is None:
|
||||
sys.stdout = NullWriter()
|
||||
if sys.stderr is None:
|
||||
sys.stderr = NullWriter()
|
||||
|
||||
|
||||
splash_root = None
|
||||
|
||||
# If running from a frozen PyInstaller bundle, launch the splash screen IMMEDIATELY
|
||||
if getattr(sys, 'frozen', False):
|
||||
import tkinter as tk
|
||||
|
||||
def show_splash_instantly():
|
||||
global splash_root
|
||||
try:
|
||||
splash_root = tk.Tk()
|
||||
splash_root.title("Odysseus")
|
||||
splash_root.overrideredirect(True)
|
||||
splash_root.configure(bg="#1a1c23")
|
||||
|
||||
# Accented borders
|
||||
splash_root.config(highlightbackground="#e06c75", highlightcolor="#e06c75", highlightthickness=1)
|
||||
|
||||
w, h = 360, 160
|
||||
ws = splash_root.winfo_screenwidth()
|
||||
hs = splash_root.winfo_screenheight()
|
||||
x = (ws - w) // 2
|
||||
y = (hs - h) // 2
|
||||
splash_root.geometry(f"{w}x{h}+{x}+{y}")
|
||||
|
||||
tk.Label(splash_root, text="⛵ Odysseus", font=("Segoe UI", 22, "bold"), bg="#1a1c23", fg="#e06c75").pack(pady=(22, 2))
|
||||
tk.Label(splash_root, text="Launching background services...", font=("Segoe UI", 10), bg="#1a1c23", fg="#d1d4e0").pack(pady=2)
|
||||
tk.Label(splash_root, text="Please wait, this will take a few seconds.", font=("Segoe UI", 8, "italic"), bg="#1a1c23", fg="#5c6370").pack(pady=(12, 0))
|
||||
|
||||
splash_root.attributes("-topmost", True)
|
||||
splash_root.mainloop()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Launch the GUI splash screen immediately on a background thread
|
||||
threading.Thread(target=show_splash_instantly, daemon=True).start()
|
||||
|
||||
|
||||
def create_tray_image():
|
||||
# Generate a beautiful 64x64 icon matching Odysseus brand red accent (#e06c75)
|
||||
from PIL import Image, ImageDraw
|
||||
image = Image.new('RGBA', (64, 64), (0, 0, 0, 0))
|
||||
dc = ImageDraw.Draw(image)
|
||||
accent_red = (224, 108, 117, 255)
|
||||
light_red = (224, 108, 117, 150)
|
||||
|
||||
# Draw premium sailing boat
|
||||
dc.polygon([(32, 10), (32, 45), (12, 45)], fill=accent_red)
|
||||
dc.polygon([(32, 18), (32, 45), (48, 45)], fill=light_red)
|
||||
dc.polygon([(8, 48), (56, 48), (44, 56), (20, 56)], fill=accent_red)
|
||||
return image
|
||||
|
||||
|
||||
def on_open_browser(icon, item, url):
|
||||
webbrowser.open(url)
|
||||
|
||||
|
||||
def on_exit(icon, item):
|
||||
icon.stop()
|
||||
os._exit(0)
|
||||
|
||||
|
||||
def setup_system_tray(url):
|
||||
try:
|
||||
import pystray
|
||||
icon_img = create_tray_image()
|
||||
menu = (
|
||||
pystray.MenuItem('Open Odysseus', lambda icon, item: on_open_browser(icon, item, url), default=True),
|
||||
pystray.MenuItem('Exit', on_exit)
|
||||
)
|
||||
tray_icon = pystray.Icon(
|
||||
"Odysseus",
|
||||
icon_img,
|
||||
"Odysseus",
|
||||
menu
|
||||
)
|
||||
tray_icon.run()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def open_browser(url):
|
||||
# Allow uvicorn and app lifecycles to complete warmups
|
||||
time.sleep(3.5)
|
||||
|
||||
# Safely close the splash screen
|
||||
try:
|
||||
global splash_root
|
||||
if splash_root:
|
||||
splash_root.after(0, splash_root.destroy)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
webbrowser.open(url)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
# Import the FastAPI app from app.py
|
||||
from app import app
|
||||
|
||||
bind_host = os.getenv("APP_BIND", "127.0.0.1")
|
||||
bind_port = int(os.getenv("APP_PORT", "7011"))
|
||||
url = f"http://{bind_host}:{bind_port}"
|
||||
|
||||
if getattr(sys, 'frozen', False):
|
||||
# Start browser manager thread
|
||||
threading.Thread(target=open_browser, args=(url,), daemon=True).start()
|
||||
# Start system tray manager thread
|
||||
threading.Thread(target=setup_system_tray, args=(url,), daemon=True).start()
|
||||
|
||||
uvicorn.run(app, host=bind_host, port=bind_port, log_level="info")
|
||||
@@ -0,0 +1,21 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2013-2020 Khan Academy and other contributors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,21 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2014 - 2022 Knut Sveidqvist
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,94 @@
|
||||
Copyright (c) 2019-07-29, Abbie Gonzalez (https://abbiecod.es|support@abbiecod.es),
|
||||
with Reserved Font Name OpenDyslexic.
|
||||
Copyright (c) 12/2012 - 2019
|
||||
This Font Software is licensed under the SIL Open Font License, Version 1.1.
|
||||
This license is copied below, and is also available with a FAQ at:
|
||||
http://scripts.sil.org/OFL
|
||||
|
||||
|
||||
-----------------------------------------------------------
|
||||
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
|
||||
-----------------------------------------------------------
|
||||
|
||||
PREAMBLE
|
||||
The goals of the Open Font License (OFL) are to stimulate worldwide
|
||||
development of collaborative font projects, to support the font creation
|
||||
efforts of academic and linguistic communities, and to provide a free and
|
||||
open framework in which fonts may be shared and improved in partnership
|
||||
with others.
|
||||
|
||||
The OFL allows the licensed fonts to be used, studied, modified and
|
||||
redistributed freely as long as they are not sold by themselves. The
|
||||
fonts, including any derivative works, can be bundled, embedded,
|
||||
redistributed and/or sold with any software provided that any reserved
|
||||
names are not used by derivative works. The fonts and derivatives,
|
||||
however, cannot be released under any other type of license. The
|
||||
requirement for fonts to remain under this license does not apply
|
||||
to any document created using the fonts or their derivatives.
|
||||
|
||||
DEFINITIONS
|
||||
"Font Software" refers to the set of files released by the Copyright
|
||||
Holder(s) under this license and clearly marked as such. This may
|
||||
include source files, build scripts and documentation.
|
||||
|
||||
"Reserved Font Name" refers to any names specified as such after the
|
||||
copyright statement(s).
|
||||
|
||||
"Original Version" refers to the collection of Font Software components as
|
||||
distributed by the Copyright Holder(s).
|
||||
|
||||
"Modified Version" refers to any derivative made by adding to, deleting,
|
||||
or substituting -- in part or in whole -- any of the components of the
|
||||
Original Version, by changing formats or by porting the Font Software to a
|
||||
new environment.
|
||||
|
||||
"Author" refers to any designer, engineer, programmer, technical
|
||||
writer or other person who contributed to the Font Software.
|
||||
|
||||
PERMISSION & CONDITIONS
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of the Font Software, to use, study, copy, merge, embed, modify,
|
||||
redistribute, and sell modified and unmodified copies of the Font
|
||||
Software, subject to the following conditions:
|
||||
|
||||
1) Neither the Font Software nor any of its individual components,
|
||||
in Original or Modified Versions, may be sold by itself.
|
||||
|
||||
2) Original or Modified Versions of the Font Software may be bundled,
|
||||
redistributed and/or sold with any software, provided that each copy
|
||||
contains the above copyright notice and this license. These can be
|
||||
included either as stand-alone text files, human-readable headers or
|
||||
in the appropriate machine-readable metadata fields within text or
|
||||
binary files as long as those fields can be easily viewed by the user.
|
||||
|
||||
3) No Modified Version of the Font Software may use the Reserved Font
|
||||
Name(s) unless explicit written permission is granted by the corresponding
|
||||
Copyright Holder. This restriction only applies to the primary font name as
|
||||
presented to the users.
|
||||
|
||||
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
|
||||
Software shall not be used to promote, endorse or advertise any
|
||||
Modified Version, except to acknowledge the contribution(s) of the
|
||||
Copyright Holder(s) and the Author(s) or with their explicit written
|
||||
permission.
|
||||
|
||||
5) The Font Software, modified or unmodified, in part or in whole,
|
||||
must be distributed entirely under this license, and must not be
|
||||
distributed under any other license. The requirement for fonts to
|
||||
remain under this license does not apply to any document created
|
||||
using the Font Software.
|
||||
|
||||
TERMINATION
|
||||
This license becomes null and void if any of the above conditions are
|
||||
not met.
|
||||
|
||||
DISCLAIMER
|
||||
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
|
||||
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
|
||||
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
|
||||
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
|
||||
OTHER DEALINGS IN THE FONT SOFTWARE.
|
||||
@@ -73,7 +73,7 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
if not model_spec:
|
||||
for candidate in ("gpt-image-1.5", "gpt-image-1", "dall-e-3"):
|
||||
try:
|
||||
_resolve_model(candidate)
|
||||
await asyncio.to_thread(_resolve_model, candidate)
|
||||
model_spec = candidate
|
||||
break
|
||||
except ValueError:
|
||||
@@ -81,7 +81,13 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
if not model_spec:
|
||||
return [TextContent(type="text", text="Error: No image model found. Configure one in Admin.")]
|
||||
|
||||
url, model_id, headers = _resolve_model(model_spec)
|
||||
try:
|
||||
url, model_id, headers = await asyncio.to_thread(_resolve_model, model_spec, model_type="image")
|
||||
except ValueError:
|
||||
_lower_model_spec = model_spec.lower()
|
||||
if not any(_name in _lower_model_spec for _name in ("gpt-image", "dall-e")):
|
||||
raise
|
||||
url, model_id, headers = await asyncio.to_thread(_resolve_model, model_spec)
|
||||
|
||||
is_gpt_image = "gpt-image" in model_id.lower()
|
||||
base_url = url.replace("/chat/completions", "").replace("/v1/messages", "").rstrip("/")
|
||||
|
||||
@@ -6,6 +6,7 @@ Imports MemoryManager and MemoryVectorStore from the Odysseus codebase.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
@@ -16,6 +17,8 @@ from mcp.types import Tool, TextContent
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from src.memory import MemoryStoreUnreadable
|
||||
|
||||
server = Server("memory")
|
||||
|
||||
# Late-initialized managers (set during first tool call)
|
||||
@@ -23,6 +26,71 @@ _memory_manager = None
|
||||
_memory_vector = None
|
||||
_initialized = False
|
||||
|
||||
_OWNER_ENV_KEYS = ("ODYSSEUS_MCP_MEMORY_OWNER", "ODYSSEUS_MEMORY_OWNER")
|
||||
_OWNER_SCOPE_ERROR = (
|
||||
"Error: Memory MCP owner is not configured for an owner-scoped memory store. "
|
||||
"Set ODYSSEUS_MCP_MEMORY_OWNER for this server or use the owner-aware native memory tool."
|
||||
)
|
||||
_UNREADABLE_STORE_ERROR = (
|
||||
"Error: Memory store is temporarily unreadable — nothing was saved. "
|
||||
"Repair or restore memory.json, then retry."
|
||||
)
|
||||
|
||||
|
||||
def _configured_owner() -> str | None:
|
||||
for key in _OWNER_ENV_KEYS:
|
||||
owner = os.environ.get(key, "").strip()
|
||||
if owner:
|
||||
return owner
|
||||
return None
|
||||
|
||||
|
||||
def _entry_owner(entry: dict) -> str | None:
|
||||
owner = entry.get("owner")
|
||||
if owner is None:
|
||||
return None
|
||||
owner_text = str(owner).strip()
|
||||
return owner_text or None
|
||||
|
||||
|
||||
def _owner_scoped_store(entries: list[dict]) -> bool:
|
||||
return any(_entry_owner(entry) for entry in entries if isinstance(entry, dict))
|
||||
|
||||
|
||||
def _scope_entries(for_update: bool = False) -> tuple[str | None, list[dict], list[dict], str | None]:
|
||||
"""Return configured owner, all entries, visible entries, and optional error.
|
||||
|
||||
``for_update=True`` is for read-modify-write callers. They save the ``all
|
||||
entries`` list back, so an unreadable store must be reported as an error
|
||||
instead of degrading to ``[]`` — otherwise the save writes their one new
|
||||
entry over the whole store (issue #5673).
|
||||
"""
|
||||
if for_update:
|
||||
try:
|
||||
entries = _memory_manager.load_all_for_update()
|
||||
except MemoryStoreUnreadable as e:
|
||||
return None, [], [], f"{_UNREADABLE_STORE_ERROR} ({e})"
|
||||
else:
|
||||
entries = _memory_manager.load_all()
|
||||
owner = _configured_owner()
|
||||
if owner is None and _owner_scoped_store(entries):
|
||||
return None, entries, [], _OWNER_SCOPE_ERROR
|
||||
if owner is None:
|
||||
visible = [
|
||||
entry for entry in entries
|
||||
if isinstance(entry, dict) and _entry_owner(entry) is None
|
||||
]
|
||||
else:
|
||||
visible = [
|
||||
entry for entry in entries
|
||||
if isinstance(entry, dict) and _entry_owner(entry) == owner
|
||||
]
|
||||
return owner, entries, visible, None
|
||||
|
||||
|
||||
def _text_result(text: str) -> list[TextContent]:
|
||||
return [TextContent(type="text", text=text)]
|
||||
|
||||
|
||||
def _ensure_init():
|
||||
"""Lazy-init memory managers on first use."""
|
||||
@@ -75,43 +143,46 @@ async def list_tools() -> list[Tool]:
|
||||
@server.call_tool()
|
||||
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
if name != "manage_memory":
|
||||
return [TextContent(type="text", text=f"Unknown tool: {name}")]
|
||||
return _text_result(f"Unknown tool: {name}")
|
||||
|
||||
_ensure_init()
|
||||
if not _memory_manager:
|
||||
return [TextContent(type="text", text="Error: Memory manager not available")]
|
||||
return _text_result("Error: Memory manager not available")
|
||||
|
||||
action = arguments.get("action", "")
|
||||
|
||||
if action == "list":
|
||||
category_filter = arguments.get("category", "")
|
||||
memories = _memory_manager.load()
|
||||
_owner, _all_memories, memories, scope_error = _scope_entries()
|
||||
if scope_error:
|
||||
return _text_result(scope_error)
|
||||
if category_filter:
|
||||
memories = [m for m in memories if m.get("category", "").lower() == category_filter.lower()]
|
||||
if not memories:
|
||||
msg = "No memories found"
|
||||
if category_filter:
|
||||
msg += f" in category '{category_filter}'"
|
||||
return [TextContent(type="text", text=msg + ".")]
|
||||
return _text_result(msg + ".")
|
||||
|
||||
lines = [f"Found {len(memories)} memory entries:\n"]
|
||||
for m in memories[:100]:
|
||||
for m in memories:
|
||||
cat = m.get("category", "fact")
|
||||
mid = m.get("id", "?")[:8]
|
||||
text = m.get("text", "")
|
||||
if len(text) > 150:
|
||||
text = text[:150] + "..."
|
||||
lines.append(f"- [{cat}] `{mid}` — {text}")
|
||||
if len(memories) > 100:
|
||||
lines.append(f"... and {len(memories) - 100} more")
|
||||
return [TextContent(type="text", text="\n".join(lines))]
|
||||
return _text_result("\n".join(lines))
|
||||
|
||||
elif action == "add":
|
||||
text = arguments.get("text", "")
|
||||
category = arguments.get("category", "fact")
|
||||
if not text:
|
||||
return [TextContent(type="text", text="Error: Memory text cannot be empty")]
|
||||
entry = _memory_manager.add_entry(text, source="ai_agent", category=category)
|
||||
memories = _memory_manager.load_all()
|
||||
return _text_result("Error: Memory text cannot be empty")
|
||||
owner, memories, _visible, scope_error = _scope_entries(for_update=True)
|
||||
if scope_error:
|
||||
return _text_result(scope_error)
|
||||
entry = _memory_manager.add_entry(text, source="ai_agent", category=category, owner=owner)
|
||||
memories.append(entry)
|
||||
_memory_manager.save(memories)
|
||||
if _memory_vector and _memory_vector.healthy:
|
||||
@@ -119,25 +190,28 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
_memory_vector.add(entry["id"], text)
|
||||
except Exception:
|
||||
pass
|
||||
return [TextContent(type="text", text=f"Memory added: [{category}] {text} (id: {entry['id'][:8]})")]
|
||||
return _text_result(f"Memory added: [{category}] {text} (id: {entry['id'][:8]})")
|
||||
|
||||
elif action == "edit":
|
||||
memory_id = arguments.get("memory_id", "")
|
||||
new_text = arguments.get("text", "")
|
||||
if not memory_id or not new_text:
|
||||
return [TextContent(type="text", text="Error: edit needs memory_id and text")]
|
||||
memories = _memory_manager.load_all()
|
||||
found = False
|
||||
return _text_result("Error: edit needs memory_id and text")
|
||||
_owner, memories, visible, scope_error = _scope_entries()
|
||||
if scope_error:
|
||||
return _text_result(scope_error)
|
||||
full_id = None
|
||||
for m in memories:
|
||||
for m in visible:
|
||||
if m.get("id", "").startswith(memory_id):
|
||||
m["text"] = new_text
|
||||
m["timestamp"] = int(time.time())
|
||||
found = True
|
||||
full_id = m["id"]
|
||||
break
|
||||
if not found:
|
||||
return [TextContent(type="text", text=f"Error: Memory '{memory_id}' not found")]
|
||||
if not full_id:
|
||||
return _text_result(f"Error: Memory '{memory_id}' not found")
|
||||
for m in memories:
|
||||
if m.get("id") == full_id:
|
||||
m["text"] = new_text
|
||||
m["timestamp"] = int(time.time())
|
||||
break
|
||||
_memory_manager.save(memories)
|
||||
if _memory_vector and _memory_vector.healthy and full_id:
|
||||
try:
|
||||
@@ -145,24 +219,26 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
_memory_vector.add(full_id, new_text)
|
||||
except Exception:
|
||||
pass
|
||||
return [TextContent(type="text", text=f"Memory updated: {new_text}")]
|
||||
return _text_result(f"Memory updated: {new_text}")
|
||||
|
||||
elif action == "delete":
|
||||
memory_id = arguments.get("memory_id", "")
|
||||
if not memory_id:
|
||||
return [TextContent(type="text", text="Error: delete needs memory_id")]
|
||||
memories = _memory_manager.load_all()
|
||||
return _text_result("Error: delete needs memory_id")
|
||||
_owner, memories, visible, scope_error = _scope_entries()
|
||||
if scope_error:
|
||||
return _text_result(scope_error)
|
||||
full_id = None
|
||||
deleted_text = ""
|
||||
deleted_category = ""
|
||||
for m in memories:
|
||||
for m in visible:
|
||||
if m.get("id", "").startswith(memory_id):
|
||||
full_id = m["id"]
|
||||
deleted_text = m.get("text", "")
|
||||
deleted_category = m.get("category", "")
|
||||
break
|
||||
if not full_id:
|
||||
return [TextContent(type="text", text=f"Error: Memory '{memory_id}' not found")]
|
||||
return _text_result(f"Error: Memory '{memory_id}' not found")
|
||||
memories = [m for m in memories if m.get("id") != full_id]
|
||||
_memory_manager.save(memories)
|
||||
if _memory_vector and _memory_vector.healthy and full_id:
|
||||
@@ -172,30 +248,32 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
pass
|
||||
cat = f"[{deleted_category}] " if deleted_category else ""
|
||||
snippet = deleted_text if len(deleted_text) <= 120 else deleted_text[:117] + "..."
|
||||
return [TextContent(type="text", text=f"Memory deleted: {cat}{snippet} (id: {memory_id})")]
|
||||
return _text_result(f"Memory deleted: {cat}{snippet} (id: {memory_id})")
|
||||
|
||||
elif action == "search":
|
||||
query = arguments.get("text", "")
|
||||
if not query:
|
||||
return [TextContent(type="text", text="Error: search needs text (query)")]
|
||||
memories = _memory_manager.load()
|
||||
return _text_result("Error: search needs text (query)")
|
||||
_owner, _all_memories, memories, scope_error = _scope_entries()
|
||||
if scope_error:
|
||||
return _text_result(scope_error)
|
||||
if hasattr(_memory_manager, 'get_relevant_memories'):
|
||||
results = _memory_manager.get_relevant_memories(query, memories, threshold=0.05, max_items=20)
|
||||
else:
|
||||
query_lower = query.lower()
|
||||
results = [m for m in memories if query_lower in m.get("text", "").lower()][:20]
|
||||
if not results:
|
||||
return [TextContent(type="text", text=f"No memories found matching '{query}'.")]
|
||||
return _text_result(f"No memories found matching '{query}'.")
|
||||
lines = [f"Found {len(results)} matching memories:\n"]
|
||||
for m in results:
|
||||
cat = m.get("category", "fact")
|
||||
mid = m.get("id", "?")[:8]
|
||||
text = m.get("text", "")
|
||||
lines.append(f"- [{cat}] `{mid}` — {text}")
|
||||
return [TextContent(type="text", text="\n".join(lines))]
|
||||
return _text_result("\n".join(lines))
|
||||
|
||||
else:
|
||||
return [TextContent(type="text", text=f"Error: Unknown action '{action}'. Use: list, add, edit, delete, search")]
|
||||
return _text_result(f"Error: Unknown action '{action}'. Use: list, add, edit, delete, search")
|
||||
|
||||
|
||||
async def run():
|
||||
|
||||
@@ -4,90 +4,84 @@
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.98.0"
|
||||
},
|
||||
"name": "odysseus",
|
||||
"devDependencies": {
|
||||
"@antithesishq/bombadil": "^0.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@anthropic-ai/sdk": {
|
||||
"version": "0.98.0",
|
||||
"resolved": "https://registry.npmjs.org/@anthropic-ai/sdk/-/sdk-0.98.0.tgz",
|
||||
"integrity": "sha512-N7aXtCvC5g6T1Y4V29lJjceu/zTkVkIZF0jdBvagr0TRFHuKeImffalGWEfqZKrvjH+IQbzJWw6TmSmUzrlMgg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"json-schema-to-ts": "^3.1.1",
|
||||
"standardwebhooks": "^1.0.0"
|
||||
},
|
||||
"bin": {
|
||||
"anthropic-ai-sdk": "bin/cli"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"zod": "^3.25.0 || ^4.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"zod": {
|
||||
"optional": true
|
||||
}
|
||||
"@antithesishq/bombadil": "^0.7.0",
|
||||
"@playwright/test": "^1.62.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@antithesishq/bombadil": {
|
||||
"version": "0.3.2",
|
||||
"resolved": "https://registry.npmjs.org/@antithesishq/bombadil/-/bombadil-0.3.2.tgz",
|
||||
"integrity": "sha512-ATy1w9ZY5gbny1H8DFc7rxZitT7DLLLFDiGcRZe+8TQiUrV5tLO+IJGOVNNLp3RpCqjZqSsxGiKoQsx31ipV1g==",
|
||||
"version": "0.7.0",
|
||||
"resolved": "https://registry.npmjs.org/@antithesishq/bombadil/-/bombadil-0.7.0.tgz",
|
||||
"integrity": "sha512-alJmnphJ/iUoL5mCsnV3DwtajGy/sEQ3NJJCiMhgjqXshSq2BUtAs0vqdXEiiSkB8HbsOX5CLrAcaogYdwfAJg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@babel/runtime": {
|
||||
"version": "7.29.7",
|
||||
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.7.tgz",
|
||||
"integrity": "sha512-Nq8OhGWiZIZGV6hLHoyAKLLcJihP/xFeBMGJoUrxTX2psI8dCifzLhZISFb+VWS3wFMRDmCGw5R+dOySCqPLhw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6.9.0"
|
||||
"bin": {
|
||||
"bombadil": "bin/bombadil.js"
|
||||
}
|
||||
},
|
||||
"node_modules/@stablelib/base64": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@stablelib/base64/-/base64-1.0.1.tgz",
|
||||
"integrity": "sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/fast-sha256": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/fast-sha256/-/fast-sha256-1.3.0.tgz",
|
||||
"integrity": "sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==",
|
||||
"license": "Unlicense"
|
||||
},
|
||||
"node_modules/json-schema-to-ts": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
|
||||
"integrity": "sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==",
|
||||
"license": "MIT",
|
||||
"node_modules/@playwright/test": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.62.1.tgz",
|
||||
"integrity": "sha512-DTcUc8qii+cpHvtOwggMtBRMjKZHXYWdw8syRYu2vtzuq4Wxphqq4NfCs5Zt44L6mA8rfDfj+PHnxFc/FeK6mQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@babel/runtime": "^7.18.3",
|
||||
"ts-algebra": "^2.0.0"
|
||||
"playwright": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16"
|
||||
"node": ">=20"
|
||||
}
|
||||
},
|
||||
"node_modules/standardwebhooks": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/standardwebhooks/-/standardwebhooks-1.0.0.tgz",
|
||||
"integrity": "sha512-BbHGOQK9olHPMvQNHWul6MYlrRTAOKn03rOe4A8O3CLWhNf4YHBqq2HJKKC+sfqpxiBY52pNeesD6jIiLDz8jg==",
|
||||
"node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@stablelib/base64": "^1.0.0",
|
||||
"fast-sha256": "^1.3.0"
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/ts-algebra": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/ts-algebra/-/ts-algebra-2.0.0.tgz",
|
||||
"integrity": "sha512-FPAhNPFMrkwz76P7cdjdmiShwMynZYN6SgOujD1urY4oNm80Ou9oMdmbR45LotcKOXoy7wSmHkRFE6Mxbrhefw==",
|
||||
"license": "MIT"
|
||||
"node_modules/playwright": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
|
||||
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright-core": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"fsevents": "2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright-core": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
|
||||
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"playwright-core": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,12 +1,18 @@
|
||||
{
|
||||
"name": "odysseus",
|
||||
"private": true,
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/pewdiepie-archdaemon/odysseus.git"
|
||||
"url": "https://github.com/odysseus-dev/odysseus.git"
|
||||
},
|
||||
"scripts": {
|
||||
"test:photo-editor": "playwright test --config tests/e2e/playwright.config.js",
|
||||
"test:photo-editor:install": "playwright install chromium firefox webkit",
|
||||
"test:photo-editor:firefox": "PHOTO_EDITOR_E2E_BROWSER=firefox playwright test --config tests/e2e/playwright.config.js",
|
||||
"test:photo-editor:webkit": "PHOTO_EDITOR_E2E_BROWSER=webkit playwright test --config tests/e2e/playwright.config.js"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@antithesishq/bombadil": "^0.3.2"
|
||||
},
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.98.0"
|
||||
"@antithesishq/bombadil": "^0.7.0",
|
||||
"@playwright/test": "^1.62.1"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,326 @@
|
||||
# Odysseus Tool Runtime Hardening Plan
|
||||
|
||||
## Objective
|
||||
|
||||
Ship `odysseus-qwen3.5-tools-pre-heretic` with one compact, model-specific tool
|
||||
runtime that supports realistic multi-turn use. Keep the existing RAG runtime
|
||||
unchanged for every other model. Prove routing, execution, answer quality,
|
||||
follow-ups, safety, rendering, latency, and native image/VL understanding through
|
||||
the real 7011 Agent UI.
|
||||
|
||||
Current evidence is a baseline, not a ship claim:
|
||||
|
||||
- Corrected v2.5 + compact-v5 development is 327/344 raw (95.06%) and
|
||||
327/336 scorable (97.32%). Sealed blind is 311/344 raw (90.41%) and
|
||||
311/336 scorable (92.56%), with zero reasoning leakage.
|
||||
- Notes, Skills, and Cookbook/admin clear 95% scorable blind. Calendar 87.5%,
|
||||
Shell/files 86.11%, and Tasks 87.5% remain below the 90% family ship floor.
|
||||
- Compact-v5 hints improved Email, Search/HF quant, and Shell on development;
|
||||
a Calendar hint regressed and was rejected rather than shipped.
|
||||
|
||||
- Ten-family focused baseline: 19/20 functional and 20/20 routing/execution.
|
||||
- Typo and cross-family read flows: 26/26 passed.
|
||||
- Real use exposed untested write correction and search-to-fetch follow-ups.
|
||||
- Email production access, browser interaction, search quality, and broader
|
||||
multi-turn mutations are not yet proven.
|
||||
- Nine enabled chat-capable regular API models pass the ten-family read-only
|
||||
legacy-RAG baseline (90/90 combined). Their stricter typo/follow-up profile is
|
||||
178/180 turns: eight models are 20/20 and Luna is 18/20 due only to its
|
||||
misspelled Shell request. One pinned image-generation model is explicitly
|
||||
unsupported and two visible local models are currently offline.
|
||||
- Native VL object/spatial recognition and reload follow-up pass. Exact OCR
|
||||
fails equally on the fine-tune and untouched 9B base and remains unresolved.
|
||||
PNG, JPEG, and WebP transport all pass.
|
||||
- Reversible create/correct/API-verify/cleanup flows pass 6/6 across every
|
||||
stateful family.
|
||||
- Search Web-toggle combinations pass 8/8 and the focused quality suite passes
|
||||
3/3. Production-path email account/inbox/referential reads pass 3/3.
|
||||
- The latest regular-model regression is 90/90 across the nine enabled
|
||||
chat-capable API models, with zero failed model turns; two local endpoints
|
||||
remain offline and the image-only model is unsupported.
|
||||
- The Epictetus OMLX endpoint was recovered after an unsupported
|
||||
`qwen3_5_mtp` model load wedged the server. Its supported Qwen 27B 4-bit
|
||||
model passes the ten-family real-7011 legacy-RAG smoke 10/10; the unsupported
|
||||
MTP artifact is recorded as a runtime limitation rather than a timeout.
|
||||
- Fresh compact-v5 UI regressions pass stateful 6/6, Email 3/3, Search 3/3,
|
||||
private-browser 3/3, and VL workflow 3/3.
|
||||
- The exact-model, family-scoped compact runtime now passes 20/20 direct and
|
||||
same-family turns across all ten families on the real 7011 Agent UI. A
|
||||
separate 36/36 robustness run passes misspellings, bounded repeats, browser
|
||||
and news continuation, ambiguous follow-ups, family switchbacks, and a
|
||||
greeting before a tool request.
|
||||
- The mobile active-email editor path passes 1/1: `Write reply this email`
|
||||
offers and executes only `update_document`, mutates the open draft, and
|
||||
preserves its reply headers and quoted thread.
|
||||
- The active-editor classifier now also covers short mobile wording without a
|
||||
pronoun (`Write reply` / `Draft a reply`) while explicit note, code, file, and
|
||||
new-object requests retain their own families. Whole-draft requests are bound
|
||||
to the sole offered `update_document` writer until one successful write, then
|
||||
tools are removed for the confirmation round. The deployed real-route email
|
||||
regression passes 3/3—including the exact unspecified `Write reply to this
|
||||
email` form—with one write, verified mutation, and preserved reply headers.
|
||||
Clean-v3 now also emits the established `doc_update` event and flattened
|
||||
document metadata on `tool_output`, so a successful database write updates
|
||||
the already-open editor instead of leaving stale UI beside a success message.
|
||||
- The client now reuses the existing assistant bubble for `agent_step` round 1
|
||||
instead of replacing it before the first token. A real-7011 sampled
|
||||
greeting-to-Notes conversation passes 2/2 with stable first-round DOM
|
||||
identity; round 2+ remains the only continuation-bubble path.
|
||||
- Clean-runtime metrics now expose provider-counted initial injected tokens,
|
||||
all-round input/output, TTFT, tok/s, schema count, agent rounds, and tool-call
|
||||
count. A real 7011 browser run passes 2/2 and visibly renders compact footers
|
||||
plus the full details popup; the sampled Notes turns streamed progressively.
|
||||
- The deployed startup bottleneck was an unindexed quadratic transcript-FTS
|
||||
reconciliation. Live-database import fell from about 36 seconds to 0.54
|
||||
seconds; 7011 now answers in about 3 seconds after a controlled restart.
|
||||
- A controlled identical-compact comparison already proves the fine-tune's
|
||||
accuracy benefit: 94.48% (325/344) versus the untouched base's 77.91%
|
||||
(268/344). Raw serving speed is effectively tied, so product speed comes
|
||||
from the compact contract and fewer failed/redundant rounds.
|
||||
- A fully merged 10,000-row category-repair candidate reached 97.32% scorable
|
||||
development but only 92.26% scorable sealed blind. Calendar (87.5%), Tasks
|
||||
(87.5%), and Shell/files (86.11%) remained below the family floor, so it was
|
||||
rejected and not deployed. Compact-v4/full development A/Bs did not improve
|
||||
Calendar or Tasks over compact-v5; full-schema Shell also fell from 97.22%
|
||||
to 94.44%. This rules out compactness as the primary cause of the remaining
|
||||
blind gaps and supports keeping the compact contract.
|
||||
|
||||
## Non-negotiable architecture rules
|
||||
|
||||
1. Runtime selection follows exact model identity. The trained Odysseus model
|
||||
uses the clean compact runtime across endpoint aliases; all other models use
|
||||
legacy RAG. Add a regression test for both sides.
|
||||
2. Resolve permissions, toggles, and available backends once per turn. Produce
|
||||
one immutable contract satisfying `required ⊆ offered ⊆ executable`.
|
||||
3. Never offer a tool that the preview policy will categorically reject. Add a
|
||||
contract self-check covering every offered action/effect combination.
|
||||
4. Follow-ups consume typed prior evidence: native call, result, success state,
|
||||
family, and object identifiers. Do not infer continuity from keyword RAG.
|
||||
5. Contextual write authority may revise only a recently proven object in the
|
||||
same family. It may not authorize a new object, another family, a destructive
|
||||
action, or an external side effect.
|
||||
6. The model chooses tools and valid arguments. The harness validates and
|
||||
executes; it does not silently substitute another family, rewrite arguments,
|
||||
fabricate success, or replace a failed tool with prose claiming completion.
|
||||
7. One owner renders each turn: streamed prose or canonical structured output.
|
||||
Never both, and never expose hidden prompts or raw untrusted wrappers.
|
||||
8. No exact-prompt production patches. A fix must name the failed layer, add a
|
||||
generic failing invariant test, and cover neighboring cases.
|
||||
|
||||
## Failure layers
|
||||
|
||||
Every failure is assigned to exactly one primary layer before code changes:
|
||||
|
||||
1. **Route:** wrong model runtime or endpoint identity.
|
||||
2. **Contract:** required tool absent, forbidden tool present, or toggle drift.
|
||||
3. **Model:** wrong/no tool or semantically wrong required arguments despite a
|
||||
correct contract.
|
||||
4. **Policy:** valid proposed operation incorrectly allowed or denied.
|
||||
5. **Execution:** canonical arguments, backend dispatch, timeout, or result
|
||||
envelope is wrong.
|
||||
6. **Evidence:** result is empty, irrelevant, truncated badly, or insufficient.
|
||||
7. **Answer:** model misstates or ignores valid tool evidence.
|
||||
8. **Rendering:** duplicate, dump-at-end, missing structured output, or stopped
|
||||
stream.
|
||||
9. **Performance:** startup, TTFT, tool latency, or oversized context.
|
||||
|
||||
Reports store aggregate category, relevant contract/tool metadata, timings, and
|
||||
sanitized outputs. Do not copy private hidden benchmark prompts or create a log
|
||||
dump that nobody can audit.
|
||||
|
||||
## Test matrix
|
||||
|
||||
Use the real authenticated 7011 Agent UI and the normal `preheret` picker alias.
|
||||
Use `sft_alex_creator` for reversible writes. Never mutate the personal account
|
||||
from an automated test.
|
||||
|
||||
### A. Every one of the ten families
|
||||
|
||||
For calendar, notes, email, tasks, documents, memory, skills, Cookbook/admin,
|
||||
search/browser, and shell/files, test:
|
||||
|
||||
- direct request;
|
||||
- natural misspelling;
|
||||
- ambiguous same-family follow-up;
|
||||
- switch to another family and back;
|
||||
- no-tool greeting before the tool request;
|
||||
- requested count/field limit;
|
||||
- backend failure rendered truthfully;
|
||||
- reload the permalink before a follow-up.
|
||||
|
||||
### B. Stateful mutation families
|
||||
|
||||
For notes, calendar, tasks, documents, memory, and skills:
|
||||
|
||||
- create → verify by API → referential correction → verify;
|
||||
- create → list/read → correction → verify;
|
||||
- typo correction such as name/date/title without repeating the family noun;
|
||||
- correction after one unrelated conversational turn;
|
||||
- destructive request is denied atomically;
|
||||
- failed write never produces a success claim;
|
||||
- cleanup deletes only the UUID-owned test artifact and verifies absence.
|
||||
|
||||
### C. Search and browser conversations
|
||||
|
||||
- search → summarize existing results without a new call;
|
||||
- search → inspect one result with `web_fetch`;
|
||||
- poor results → refine query once;
|
||||
- insufficient evidence → say so without fabrication;
|
||||
- Web toggle combinations `00`, `01`, `10`, and `11` across two turns;
|
||||
- private browser open/snapshot/click only after its permission boundary is
|
||||
deliberately enabled and specified; do not smuggle it in via web search.
|
||||
|
||||
Grade source relevance, freshness, authority, and whether claims are supported,
|
||||
not merely whether `web_search` was called.
|
||||
|
||||
### D. Email and shell
|
||||
|
||||
- Separate fixture accuracy from production connectivity. A fixture pass cannot
|
||||
promote production email health.
|
||||
- Test account listing, inbox listing, reading, and referential follow-up against
|
||||
the configured production-like backend before enabling email actions.
|
||||
- Shell remains toggle-gated. Test off/on transitions, canonical raw command
|
||||
dispatch, read-only output, and denial of network/destructive commands.
|
||||
|
||||
### E. Rendering and performance
|
||||
|
||||
- Assert first visible streamed token, monotonic DOM growth, one final answer,
|
||||
persistence/reload equality, stop behavior, and structured list rendering.
|
||||
- Record request preparation, TTFT, tool duration, post-tool TTFT, total time,
|
||||
input/output tokens, and tool-result bytes.
|
||||
- Diagnose the 30–40 second 7011 restart separately from inference latency.
|
||||
- Bound large calendar/search results before replaying them into later rounds,
|
||||
while preserving IDs and fields needed for follow-ups.
|
||||
|
||||
### F. Image/VL recognition
|
||||
|
||||
- Attach real PNG, JPEG, and WebP images through the 7011 UI and verify the
|
||||
trained model receives native multimodal message content on its clean route.
|
||||
- Test object recognition, visible text/OCR, spatial relationships, charts, and
|
||||
screenshots. Score required facts instead of stylistic wording.
|
||||
- Test image → ambiguous follow-up, image → tool request, and tool result → image
|
||||
comparison without requiring the user to attach the same image again.
|
||||
- Verify image references survive persistence and permalink reload without raw
|
||||
base64, local paths, or hidden wrappers appearing in chat output.
|
||||
- Separate direct model vision from `inspect_media`, browser screenshots, and
|
||||
image generation. The harness must not silently substitute one for another.
|
||||
- Compare the fine-tune with its base VL model on the same images to detect
|
||||
whether tool training regressed visual understanding.
|
||||
|
||||
### G. Regular-model legacy RAG and tool coverage
|
||||
|
||||
- Inventory every enabled non-Odysseus endpoint/model visible in 7011, including
|
||||
its provider, schema mode, native-tool support, context limit, and configured
|
||||
permissions. Do not assume every provider supports the same wire format.
|
||||
- Assert that no non-Odysseus model enters the clean-v3 runtime. These models
|
||||
retain the regular RAG/tool loop and are repaired only in that owning path.
|
||||
- For each model, test every tool family the effective user policy offers:
|
||||
direct request, misspelling, ambiguous follow-up, family switch, backend
|
||||
failure, and Web/Bash toggle transitions. Record unsupported families as an
|
||||
explicit capability limitation, not a silent pass.
|
||||
- Test full schemas versus compact schemas only where both are valid for that
|
||||
model. Store the selected schema mode in every report.
|
||||
- Verify provider-native tool calls, textual fallback parsing where required,
|
||||
canonical argument conversion, execution, evidence replay, and rendering.
|
||||
- Group fixes by shared legacy-runtime or provider-adapter defect. Do not add
|
||||
model-name prompt exceptions when a transport, schema, or RAG ranking issue is
|
||||
responsible.
|
||||
- Maintain a per-model compatibility matrix so adding or changing an endpoint
|
||||
cannot silently regress previously working tools.
|
||||
|
||||
## Fix protocol
|
||||
|
||||
For each failure:
|
||||
|
||||
1. Preserve the raw report and reproduce once on a fresh test session.
|
||||
2. Identify the primary failure layer from the taxonomy above.
|
||||
3. Add the smallest generic red test at that layer.
|
||||
4. Fix the owning module or invariant—not the literal prompt.
|
||||
5. Run the focused unit tests, the original scenario, two adjacent scenarios,
|
||||
and the affected family suite.
|
||||
6. After a batch of category fixes, rerun the ten-family matrix and legacy-RAG
|
||||
isolation test. Do not rerun training unless the contract and harness are
|
||||
proven correct and failures remain model-owned.
|
||||
|
||||
If three failures share a layer, pause case-by-case patching and refactor that
|
||||
layer before continuing.
|
||||
|
||||
## Execution phases
|
||||
|
||||
### Phase 1 — Make the runtime auditable
|
||||
|
||||
- Add a sanitized per-turn decision record: model runtime, contract, proposed
|
||||
calls, policy decisions with reason codes, executions, render owner, timings.
|
||||
- Add startup/runtime provenance to the UI so a linked chat proves which harness
|
||||
handled it.
|
||||
- Add the offered-versus-policy compatibility self-test.
|
||||
- Correct stale preview documentation.
|
||||
|
||||
### Phase 2 — Build the conversation suite
|
||||
|
||||
- Extend the current Playwright verifier with reusable multi-turn scenarios and
|
||||
reversible artifact fixtures.
|
||||
- Implement the matrix above, prioritizing search continuations and all
|
||||
stateful corrections because real usage already exposed those gaps.
|
||||
- Run independent family groups in parallel, but serialize writes that share a
|
||||
backend or fixture account.
|
||||
- Add a small versioned VL fixture set with locally generated, non-private
|
||||
images and deterministic answer keys.
|
||||
|
||||
### Phase 3 — Repair by architecture category
|
||||
|
||||
- Consolidate model-specific runtime selection in one function.
|
||||
- Represent prior successful objects explicitly for referential follow-ups.
|
||||
- Align tool capability classification, contract offering, and policy decisions.
|
||||
- Standardize tool results into bounded envelopes with source/object IDs.
|
||||
- Keep search refinement and evidence sufficiency generic.
|
||||
|
||||
### Phase 4 — Accuracy and speed comparison
|
||||
|
||||
- Compare the clean fine-tune with the base model using identical compact tools,
|
||||
prompts, toggles, backend state, and semantic scoring.
|
||||
- Report functional accuracy, argument accuracy, unsupported success claims,
|
||||
TTFT, total latency, and tokens. Do not compare one model on full schemas and
|
||||
another on compact schemas.
|
||||
- Only consider more SFT/RL for failures classified as model-owned after the
|
||||
harness audit.
|
||||
|
||||
### Phase 4B — Regular-model repair and verification
|
||||
|
||||
- Snapshot the enabled non-Odysseus model inventory.
|
||||
- Run the legacy-RAG compatibility matrix in bounded parallel groups, respecting
|
||||
endpoint rate limits and shared backend write serialization.
|
||||
- Fix shared harness/provider defects first, then rerun all affected models.
|
||||
- Publish separate per-model scores and limitations; do not blend them into the
|
||||
Odysseus fine-tune score.
|
||||
|
||||
### Phase 5 — Ship gate
|
||||
|
||||
Ship only when:
|
||||
|
||||
- every family is at least 90% on sealed functional holdout;
|
||||
- overall functional accuracy is at least 95%;
|
||||
- realistic follow-up suite is at least 95%, with no repeated failure category;
|
||||
- image/VL fixture accuracy does not regress materially from the base model and
|
||||
all attachment/follow-up/persistence flows pass;
|
||||
- routing/execution and safety invariants are 100%;
|
||||
- all reversible writes are API-verified and cleaned up;
|
||||
- search quality and production email are reported separately and honestly;
|
||||
- non-Odysseus models demonstrably retain legacy RAG;
|
||||
- every enabled regular model has a complete tested-tool compatibility record,
|
||||
and every tool advertised as supported passes its functional checks;
|
||||
- no hidden prompt leakage, duplicate rendering, or false success remains;
|
||||
- pre-heretic passing weights and merged adapter backups remain recoverable.
|
||||
|
||||
## Immediate next batch
|
||||
|
||||
1. Expand VL fixtures to charts, screenshots, and image-to-tool turns;
|
||||
investigate the shared base-model OCR limitation without hiding it behind a
|
||||
silent external fallback.
|
||||
2. Add deliberately permissioned private-browser open/snapshot/click checks;
|
||||
keep browser interaction unavailable when its boundary is not enabled.
|
||||
3. Bring the two configured local regular models online and run their matrix.
|
||||
4. Compare fine-tune versus untouched base with identical compact contracts,
|
||||
backend state, prompts, and timing instrumentation.
|
||||
5. Run the sealed all-action holdout and prioritize failures by shared
|
||||
layer rather than by prompt.
|
||||
@@ -0,0 +1,445 @@
|
||||
# Plan: Odysseus Professional Photo Editor
|
||||
|
||||
> Source PRD: Conversation goal, "a Photoshop/Photopea clone with Odysseus style"
|
||||
|
||||
## Product boundary
|
||||
|
||||
Odysseus should provide the editing loop people expect from a professional
|
||||
layer-based photo editor without copying Photoshop's visual design or trying to
|
||||
match every specialist feature. The target is a dependable browser editor for
|
||||
real photo work: direct manipulation, non-destructive layers, precise masking,
|
||||
retouching, typography, export, recovery, and optional AI assistance.
|
||||
|
||||
The existing quiet Odysseus interface remains the visual language. Dense tools
|
||||
are acceptable, but controls should stay restrained, compact, predictable, and
|
||||
usable on both desktop and touch devices.
|
||||
|
||||
## Existing foundation
|
||||
|
||||
The current editor already provides meaningful parts of this product:
|
||||
|
||||
- Raster and editable text layers
|
||||
- Multi-layer selection, nested groups, clipping, visibility, opacity, and locks
|
||||
- Layer, group, and selection masks
|
||||
- Marquee, lasso, wand, SAM, Quick Mask, and saved selections
|
||||
- Brush, eraser, clone, crop, transform, and text tools
|
||||
- Blend modes, adjustment stacks, blur, and several image corrections
|
||||
- Rulers, guides, grid, snapping, zooming, and panning
|
||||
- Undo/redo history with a memory budget
|
||||
- Versioned layered-project serialization, autosave drafts, recovery, and export
|
||||
- Optional endpoint-backed inpaint and image-processing tools
|
||||
- Desktop and mobile editor layouts with Playwright release-gate coverage
|
||||
|
||||
## Architectural decisions
|
||||
|
||||
Durable decisions that apply across every phase:
|
||||
|
||||
- **Editor ownership**: The editor remains an Odysseus feature. Do not embed a
|
||||
third-party editor or imitate another product's chrome.
|
||||
- **Document format**: Continue the versioned Odysseus editor document. Every
|
||||
new persistent capability requires a migration, validation, round-trip test,
|
||||
and corrupt-input recovery behavior.
|
||||
- **Layer model**: Grow the document into explicit layer kinds rather than
|
||||
hiding more behavior in raster canvases. The intended kinds are raster, text,
|
||||
shape, adjustment, and placed/smart content.
|
||||
- **Non-destructive default**: Preserve source pixels and editable parameters
|
||||
whenever practical. Destructive actions remain available as explicit Apply,
|
||||
Rasterize, or Merge commands.
|
||||
- **Interaction engine**: Transform, crop, selections, text frames, masks, and
|
||||
shapes share one pointer-session model for hit testing, pointer capture,
|
||||
modifiers, snapping, cancellation, and undo transactions.
|
||||
- **Rendering**: Keep Canvas 2D as the compatibility renderer initially. Move
|
||||
expensive compositing and pixel operations behind renderer/worker boundaries
|
||||
before considering WebGL or WebGPU acceleration.
|
||||
- **History**: One continuous gesture creates one undo entry. Preview frames are
|
||||
never separate history entries, and Cancel restores the exact starting state.
|
||||
- **Persistence routes**: Continue using `/api/editor-drafts` for layered draft
|
||||
persistence and `/api/gallery` for media-library save/replace operations.
|
||||
- **AI boundary**: AI features consume capability-based image endpoints. Core
|
||||
editing never requires a particular model, repository, or provider.
|
||||
- **Responsive behavior**: Desktop favors precision; touch targets gain larger
|
||||
invisible hit areas without visually enlarging the whole interface.
|
||||
- **Testing**: Every phase adds deterministic geometry/unit tests and at least
|
||||
one complete Playwright workflow covering persistence and undo where relevant.
|
||||
- **Incremental architecture**: New behavior leaves the main editor orchestrator
|
||||
through small domain modules. Avoid broad refactors that do not deliver a
|
||||
visible editing improvement in the same phase.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Accurate Transform Frame
|
||||
|
||||
**User stories**: I can clearly see and grab the transform frame at any zoom. I
|
||||
can resize from corners or sides without grabbing invisible or incorrect areas.
|
||||
|
||||
### What to build
|
||||
|
||||
Replace the four-corner-only frame with a shared frame geometry model. Render
|
||||
four corners, four edge handles, a rotation control, and an optional center
|
||||
pivot from the same geometry used for hit testing. Keep handles visually compact
|
||||
while providing touch-sized invisible targets. Make the frame stay aligned
|
||||
during zoom, pan, viewport resize, and when handles extend outside the image.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Eight resize handles, rotation control, and center pivot derive from one geometry result.
|
||||
- [x] Drawn handles and hit targets cannot disagree.
|
||||
- [x] Handles remain a stable visual size from minimum to maximum zoom.
|
||||
- [x] Touch hit targets are at least 40 CSS pixels without oversized visuals.
|
||||
- [x] Outside-canvas handles remain interactive and visible when space permits.
|
||||
- [x] Hover and active cursors match each handle's current screen direction.
|
||||
- [x] Desktop and mobile Playwright tests grab every handle successfully.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Correct Rotated Resize
|
||||
|
||||
**User stories**: I can resize a rotated layer naturally. The opposite side or
|
||||
corner stays fixed, and the frame follows my pointer rather than drifting.
|
||||
|
||||
### What to build
|
||||
|
||||
Calculate drag movement in the frame's rotated local coordinate system. Anchor
|
||||
the opposite handle in document space and derive the new center from that
|
||||
anchor. Support crossing an axis as a deliberate flip instead of clamping to a
|
||||
one-pixel box. Apply the same geometry to one layer, multiple layers, and a
|
||||
selection transform.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Rotated corner and edge drags follow the pointer on the frame's local axes.
|
||||
- [x] The opposite anchor remains fixed within a sub-pixel tolerance.
|
||||
- [x] Crossing width or height zero produces a predictable horizontal or vertical flip.
|
||||
- [x] Shift locks the starting aspect ratio.
|
||||
- [x] Alt/Option scales around the transform center.
|
||||
- [x] Combined Shift+Alt/Option behavior is deterministic.
|
||||
- [x] Rotation snaps to 15-degree increments with Shift and remains smooth otherwise.
|
||||
- [x] Geometry tests cover 0, 45, 90, 135, and arbitrary-degree rotations.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Transform Interaction Polish
|
||||
|
||||
**User stories**: Transform behaves like a professional tool on mouse, pen, and
|
||||
touch. I can see exact values, snap precisely, and never lose a drag at the edge.
|
||||
|
||||
### What to build
|
||||
|
||||
Use a unified pointer session with pointer capture, live modifiers, and a small
|
||||
contextual transform readout. Add accurate rotated-frame interior hit testing,
|
||||
keyboard nudging, frame snapping, and clear Apply/Cancel behavior. Keep the
|
||||
existing compact Odysseus styling and make the numeric popup a precision surface
|
||||
rather than a competing transform implementation.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Pointer capture keeps a drag alive outside the canvas and browser viewport.
|
||||
- [x] Clicking inside a rotated frame moves it; clicking its empty bounding-box corner does not.
|
||||
- [x] Live X, Y, W, H, and angle values stay synchronized with direct manipulation.
|
||||
- [x] Arrow keys nudge, Shift+Arrow performs a larger nudge, Enter applies, and Escape cancels.
|
||||
- [x] Layer edges, document center/edges, guides, and grid participate in transform snapping.
|
||||
- [x] Snap guides clearly identify the active alignment without obscuring the photo.
|
||||
- [x] A complete gesture creates exactly one undo step.
|
||||
- [x] Touch gestures do not conflict with viewport pinch/pan behavior.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Transform Content Correctness
|
||||
|
||||
**User stories**: Transforming layers never unexpectedly damages masks, text,
|
||||
group layout, clipping, or image quality. Saving and reopening preserves it.
|
||||
|
||||
### What to build
|
||||
|
||||
Route raster layers, text layers, linked and unlinked masks, selections, clipped
|
||||
layers, and grouped multi-selection through the same transform contract. Keep
|
||||
immutable source data during previews and validate the final result through
|
||||
undo, cancel, autosave, project download, and reopen.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Raster previews are always derived from the session source, never a prior preview.
|
||||
- [x] Editable text remains editable after scaling, rotation, and flipping.
|
||||
- [x] Linked masks follow the layer while unlinked masks remain in document space.
|
||||
- [x] Multi-layer transforms preserve relative centers, order, clipping, and group membership.
|
||||
- [x] Transforming a selection changes only the selection mask unless content transform is explicitly chosen.
|
||||
- [x] Apply, Cancel, Undo, Redo, autosave reopen, and project-file reopen produce matching pixels and metadata.
|
||||
- [x] Large transforms cannot allocate beyond the editor's documented surface budget.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Shared Direct-Manipulation Sessions
|
||||
|
||||
**User stories**: Crop, selections, masks, text boxes, and shapes feel consistent
|
||||
with Transform instead of each behaving like a separate mini application.
|
||||
|
||||
### What to build
|
||||
|
||||
Generalize the proven transform pointer session into a reusable interaction
|
||||
contract. Migrate crop and selection movement first as a visible tracer bullet,
|
||||
including modifiers, snapping, pointer capture, cancel, and one-step history.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Transform, crop, and selection movement use the same gesture lifecycle.
|
||||
- [x] Tool switching safely commits, cancels, or prompts according to one policy.
|
||||
- [x] No stale pointer session can modify a newly selected tool or document.
|
||||
- [x] Mouse, pen, and touch event behavior is covered by shared tests.
|
||||
- [x] Adding a future frame-based tool does not require another global event stack.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Non-Destructive Placed Layers
|
||||
|
||||
**User stories**: I can import an image, resize it repeatedly without cumulative
|
||||
quality loss, replace its source, and choose when to rasterize it.
|
||||
|
||||
### What to build
|
||||
|
||||
Introduce a placed/smart layer kind containing source pixels and persistent
|
||||
transform metadata. Import-as-layer uses this kind by default. Rendering applies
|
||||
the transform at composite time, while Rasterize produces a normal raster layer.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Repeated transforms render from the original source rather than resampling the last result.
|
||||
- [x] A placed layer can be replaced while preserving its transform and masks.
|
||||
- [x] Rasterize produces a visually matching editable raster layer.
|
||||
- [x] Masks, clipping, groups, blend modes, and opacity work with placed layers.
|
||||
- [x] Version migration and recovery handle missing or corrupt placed sources.
|
||||
- [x] Existing raster projects open without changed output.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Professional Selections And Masks
|
||||
|
||||
**User stories**: I can build, inspect, refine, save, transform, and reuse precise
|
||||
selections without manually repainting every edge.
|
||||
|
||||
### What to build
|
||||
|
||||
Unify marquee, lasso, wand, SAM, Quick Mask, and saved selections around one
|
||||
selection-mask model. Add explicit replace/add/subtract/intersect modes, feather,
|
||||
expand, contract, smooth, border, and a focused refine-edge workflow.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Every selection tool supports replace, add, subtract, and intersect modes.
|
||||
- [x] Feather, expand, contract, smooth, and border preview before applying.
|
||||
- [x] Quick Mask edits the same canonical selection shown by marching ants.
|
||||
- [x] Selection-to-layer-mask and layer-mask-to-selection round-trip accurately.
|
||||
- [x] Saved selections retain names and pixels across reopen.
|
||||
- [x] Edge refinement works without requiring an AI dependency.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Paint And Retouch Workflow
|
||||
|
||||
**User stories**: I can paint and retouch photographs with predictable strokes,
|
||||
reusable presets, and the controls expected for a mouse, pen, or touch device.
|
||||
|
||||
### What to build
|
||||
|
||||
Promote brush behavior into a reusable brush engine. Add spacing, smoothing,
|
||||
pressure mapping, blend mode, sampled color, presets, and stroke preview. Build
|
||||
healing, dodge, and burn as complete retouching paths using that engine.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Brush, eraser, clone, masks, and inpaint share spacing and smoothing behavior.
|
||||
- [x] Pressure can independently affect size, opacity, or flow when supported.
|
||||
- [x] Eyedropper samples composite or active-layer color.
|
||||
- [x] Brush presets can be created, named, selected, and deleted.
|
||||
- [x] Healing, dodge, and burn create one undo entry per stroke.
|
||||
- [x] Long strokes remain smooth without blocking the main interface.
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Editable Text And Shapes
|
||||
|
||||
**User stories**: I can design labels, cards, and overlays with text and vector
|
||||
shapes that remain editable after saving and reopening.
|
||||
|
||||
### What to build
|
||||
|
||||
Add on-canvas text-frame editing, selection, caret behavior, typography, and
|
||||
alignment. Introduce shape layers for rectangle, ellipse, line, and path-backed
|
||||
polygons with editable fill, stroke, corners, and transform metadata.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [x] Text is edited directly on canvas without immediately rasterizing.
|
||||
- [x] Font, size, weight, line height, letter spacing, alignment, and color persist.
|
||||
- [x] Rectangle, ellipse, line, and polygon shapes remain editable.
|
||||
- [x] Shape fill, stroke, width, and corner radius can be changed after creation.
|
||||
- [x] Text and shape layers support masks, clipping, groups, blend modes, and transform.
|
||||
- [x] Missing fonts fall back predictably without corrupting the project.
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Adjustment Layers And Color
|
||||
|
||||
**User stories**: I can correct a photograph non-destructively and return later
|
||||
to modify the correction without reconstructing the edit.
|
||||
|
||||
### What to build
|
||||
|
||||
Promote adjustments into first-class layers with masks and clipping. Deliver
|
||||
Levels and Curves first, then exposure, white balance, hue/saturation, color
|
||||
balance, selective color, gradients, and channel-aware controls.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Adjustment layers affect content below them and can be clipped or grouped.
|
||||
- [ ] Every adjustment has live preview, reset, visibility, opacity, mask, Apply, and Cancel behavior.
|
||||
- [ ] Levels includes histogram, input range, gamma, and output range.
|
||||
- [ ] Curves supports RGB and channel curves with editable points.
|
||||
- [ ] Color results match flattened export and project reopen.
|
||||
- [ ] Large previews are throttled or worker-backed and remain cancellable.
|
||||
|
||||
---
|
||||
|
||||
## Phase 11: Layer Effects And Filters
|
||||
|
||||
**User stories**: I can add common visual effects without permanently altering
|
||||
the layer and can reorder or disable those effects later.
|
||||
|
||||
### What to build
|
||||
|
||||
Create an ordered non-destructive filter/effect stack. Begin with Gaussian blur,
|
||||
sharpen, shadow, stroke, and color overlay; then add filter masks and reusable
|
||||
effect presets.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Effects can be added, reordered, toggled, edited, masked, and removed.
|
||||
- [ ] Drop shadow, stroke, color overlay, blur, and sharpen survive project reopen.
|
||||
- [ ] Effects render correctly inside groups and clipping stacks.
|
||||
- [ ] Apply/rasterize produces a pixel-equivalent raster result.
|
||||
- [ ] Expensive filters expose progress and cancellation.
|
||||
|
||||
---
|
||||
|
||||
## Phase 12: Odysseus Professional Workspace
|
||||
|
||||
**User stories**: I can work quickly without fighting floating windows or losing
|
||||
the active tool, layer, selection, or document context.
|
||||
|
||||
### What to build
|
||||
|
||||
Refine the existing shell into a consistent professional workspace: contextual
|
||||
tool options, properties inspector, panel persistence, command search, status
|
||||
information, multi-document switching, and compact touch sheets. Preserve the
|
||||
current Odysseus palette, typography, restrained borders, and frosted surfaces.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Tool options appear in one predictable location and never duplicate popup state.
|
||||
- [ ] Panels remember size, collapsed state, and position per device class.
|
||||
- [ ] The properties inspector follows the active layer, mask, selection, or tool.
|
||||
- [ ] Command search exposes actions and shortcuts without adding toolbar clutter.
|
||||
- [ ] Switching documents preserves independent history, zoom, pan, and selection.
|
||||
- [ ] Mobile prioritizes canvas area while keeping all commands reachable.
|
||||
|
||||
---
|
||||
|
||||
## Phase 13: File Interchange And Export
|
||||
|
||||
**User stories**: I can bring common assets into Odysseus and export predictable
|
||||
results without losing transparency, dimensions, or color intent.
|
||||
|
||||
### What to build
|
||||
|
||||
Strengthen image import/export first, then add layered interchange where a
|
||||
maintained parser makes it safe. Keep Odysseus project files as the lossless
|
||||
source of truth and clearly report what an external format cannot preserve.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] PNG, JPEG, WebP, and supported modern image imports honor orientation and transparency.
|
||||
- [ ] Export exposes format, dimensions, quality, metadata, and transparency choices.
|
||||
- [ ] Copy/paste and drag/drop preserve alpha and use placed layers when appropriate.
|
||||
- [ ] Layered imports report unsupported features instead of silently flattening them.
|
||||
- [ ] Exported pixels are covered by deterministic visual comparisons.
|
||||
|
||||
---
|
||||
|
||||
## Phase 14: Large-Document Performance And Recovery
|
||||
|
||||
**User stories**: Large photos and layered projects remain responsive, autosave
|
||||
reliably, and recover after a crash or interrupted network connection.
|
||||
|
||||
### What to build
|
||||
|
||||
Move serialization, thumbnails, filters, and suitable pixel operations into
|
||||
workers. Add dirty-region rendering, reusable surfaces, measurable memory
|
||||
budgets, operation cancellation, autosave generations, and recovery diagnostics.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Normal interactions remain responsive on the agreed 4K multi-layer benchmark.
|
||||
- [ ] Compositing avoids rebuilding unaffected layers and thumbnails.
|
||||
- [ ] History and document surfaces stay within explicit memory limits.
|
||||
- [ ] Closing or switching documents cancels stale work safely.
|
||||
- [ ] Autosave never lets an older request overwrite newer state.
|
||||
- [ ] Recovery can identify the last complete generation and explain skipped data.
|
||||
|
||||
---
|
||||
|
||||
## Phase 15: Odysseus-Native Assisted Editing
|
||||
|
||||
**User stories**: I can use an available local or remote image capability as an
|
||||
editing assistant while retaining masks, layers, undo, privacy choices, and
|
||||
normal manual controls.
|
||||
|
||||
### What to build
|
||||
|
||||
Standardize image capability discovery and requests for generation, editing,
|
||||
inpainting, segmentation, restoration, and upscaling. Results enter the document
|
||||
as named layers with provenance and reusable masks. Add orchestration only after
|
||||
the manual operation it assists is dependable.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] The UI describes required capabilities rather than model or provider names.
|
||||
- [ ] Memory and unrelated chat context are not sent to image endpoints.
|
||||
- [ ] Requests show progress, support cancellation, and cannot update a closed document.
|
||||
- [ ] Generated results arrive as reversible layers with prompt/settings metadata.
|
||||
- [ ] A failed endpoint leaves the source document unchanged and offers a useful retry path.
|
||||
- [ ] Manual selection and masking remain available when assisted tools are absent.
|
||||
|
||||
---
|
||||
|
||||
## Phase 16: Professional Release Gate
|
||||
|
||||
**User stories**: I can trust the editor for real work and understand what is
|
||||
unsupported before committing an edit.
|
||||
|
||||
### What to build
|
||||
|
||||
Create a release gate around complete user journeys rather than isolated button
|
||||
tests. Cover accessibility, keyboard-only operation, touch, browser differences,
|
||||
pixel correctness, persistence, failure recovery, and large-document behavior.
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- [ ] Core workflows pass on current Chromium and Firefox desktop builds.
|
||||
- [ ] Mobile workflows pass at representative phone and tablet viewports.
|
||||
- [ ] Keyboard-only users can reach every command and escape every modal state.
|
||||
- [ ] Transform, masks, text, adjustments, export, and reopen have pixel/metadata regression tests.
|
||||
- [ ] No supported action silently flattens or discards editable document data.
|
||||
- [ ] The ALPHA badge can be removed based on explicit reliability metrics.
|
||||
|
||||
---
|
||||
|
||||
## Recommended delivery order
|
||||
|
||||
The first four phases are one focused Transform 2.0 program and should ship in
|
||||
order. Phases 5 and 6 establish the interaction and document foundations needed
|
||||
for the remaining professional tools. After that, phases 7 through 13 can be
|
||||
prioritized by user value, while performance and release-gate work continue as
|
||||
part of every phase rather than being deferred entirely to the end.
|
||||
|
||||
The recommended first milestone is complete when Phases 1 through 4 are live:
|
||||
transforming one layer, multiple layers, text, masks, and selections feels
|
||||
precise on desktop and mobile and remains correct through undo and reopen.
|
||||
@@ -0,0 +1,159 @@
|
||||
# Photo Editor Remaining Scope
|
||||
|
||||
Date: 2026-08-29
|
||||
|
||||
## Current verdict
|
||||
|
||||
Odysseus is now a credible layered everyday editor, not an editor mockup. The
|
||||
first nine roadmap phases are implemented: professional transform geometry,
|
||||
shared direct-manipulation sessions, retained placed content, unified
|
||||
selections and masks, a reusable brush/retouch engine, and retained text and
|
||||
shape layers.
|
||||
|
||||
Phase 10 is functionally advanced but not closed. First-class adjustment layers
|
||||
now support Levels, Curves, Exposure, White Balance, Brightness/Contrast,
|
||||
Hue/Saturation/Lightness, Color Balance, Selective Color, and Gradient Map.
|
||||
They participate in clipping, groups, masks, visibility, opacity, history, the
|
||||
v14 document format, and flattening. Retained effects have since been added as
|
||||
a separate ordered stack with Gaussian Blur, Color Overlay, Drop Shadow, and
|
||||
Stroke, including editable colors, visibility, opacity, reorder, rasterize,
|
||||
history, persistence, and migration.
|
||||
|
||||
Practical readiness estimate:
|
||||
|
||||
- Everyday layered photo editing: **about 88%**
|
||||
- Dependable professional v1 described by the roadmap: **about 62%**
|
||||
- Broad Photoshop/Photopea feature parity: **about 50%**
|
||||
|
||||
The remaining gap is dominated by large-document rendering outside the live
|
||||
composite path, workspace consolidation, interchange/color policy, and release
|
||||
proof rather than basic canvas tools.
|
||||
|
||||
## Verification snapshot
|
||||
|
||||
- The focused editor unit suite currently passes **31 tests** in Docker.
|
||||
- The full photo-editor browser suite currently has **41 passing workflows**;
|
||||
the nested-group selection workflow initially exposed a row-hit regression,
|
||||
which now passes on isolated rerun after the slider-selection fix. The new
|
||||
group-effects workflow also passes.
|
||||
- The new adjustment tests exercise deterministic pixel math, nested parameter
|
||||
normalization, retained metadata, undo/redo, clipping, masks, and draft
|
||||
reopen.
|
||||
- The latest editor changes have not yet been rebuilt into the live `7011`
|
||||
container.
|
||||
|
||||
## Close Phase 10
|
||||
|
||||
This is the immediate release slice.
|
||||
|
||||
1. Finish the bounded preview path for large documents. Downsampled previews
|
||||
now keep control movement responsive and full resolution is restored for
|
||||
commit/export. Live worker composites now use generation checks, latest-only
|
||||
coalescing, and close/reopen invalidation; extend the same guarantees to
|
||||
remaining preview paths.
|
||||
2. Add flattened-export versus reopened-project pixel comparisons for every
|
||||
adjustment family, including groups, clipping, masks, blend mode, and
|
||||
partial opacity.
|
||||
3. Validate the color algorithms visually. White Balance and Selective Color
|
||||
are currently deterministic approximations, not color-managed photographic
|
||||
transforms.
|
||||
4. Test every adjustment popup on phone and desktop viewports, including tall
|
||||
popups, color inputs, drag, Reset, Apply, Cancel, and Escape.
|
||||
5. Decide the migration path for the older per-raster `adjLayers` stack. It can
|
||||
remain readable for compatibility, but new UI should converge on first-class
|
||||
adjustment layers instead of maintaining two competing concepts.
|
||||
6. Bump static cache versions, rebuild the live container, and run a short
|
||||
visual smoke test on `7011`.
|
||||
|
||||
## Phase 11: Retained effects and filters
|
||||
|
||||
The retained-effects slice is implemented for raster/placed/text/shape-compatible
|
||||
layer output: Gaussian Blur, Sharpen, Color Overlay, Drop Shadow, and Stroke
|
||||
have editable colors/parameters, visibility, opacity, reorder, rasterize,
|
||||
history, migration, and reopen support. Effect-specific masks, presets, and
|
||||
group-level effects are also implemented and covered by focused browser tests.
|
||||
Remaining work is:
|
||||
|
||||
1. Extend worker coverage to serialization and remaining preview paths.
|
||||
Thumbnail encoding, retained-effect rasterization, and live composite
|
||||
rendering now use a worker where OffscreenCanvas is available, with
|
||||
synchronous compatibility fallbacks. Generation invalidation, latest-only
|
||||
coalescing, and CPU loop cancellation protect live rendering.
|
||||
2. Add explicit group-effect blend/ordering tests for nested groups and
|
||||
non-default blend modes, plus visual comparisons for effect stacks.
|
||||
|
||||
Introduce the renderer/worker cancellation boundary here rather than adding
|
||||
more synchronous full-canvas filters that Phase 14 must immediately replace.
|
||||
|
||||
## Phase 12: Professional workspace
|
||||
|
||||
Consolidate fragmented popups into one contextual properties surface. Persist
|
||||
panel layout by device class, add command search, expose stable document status,
|
||||
and support multiple open documents with independent history, zoom, pan, and
|
||||
selection. Mobile should use canvas-first sheets rather than compressed desktop
|
||||
panels.
|
||||
|
||||
## Phase 13: Interchange and export
|
||||
|
||||
Harden orientation, transparency, metadata, and color behavior for PNG, JPEG,
|
||||
and WebP first. Add copy/paste and drag/drop through placed layers. Treat
|
||||
layered formats as explicit compatibility projects: unsupported PSD/TIFF/HEIC
|
||||
features must be reported, never silently discarded. Odysseus project files
|
||||
remain the lossless source of truth.
|
||||
|
||||
## Phase 14: Performance and recovery
|
||||
|
||||
Move remaining preview/pixel paths into workers. Thumbnail encoding,
|
||||
autosave serialization, adjustment rendering, and retained-effect rendering
|
||||
now have worker-backed paths with compatibility fallbacks. Add
|
||||
dirty-region compositing, reusable render surfaces, cancellation tokens,
|
||||
operation telemetry, a documented surface/history budget, autosave generations,
|
||||
and a checked-in 4K multi-layer benchmark.
|
||||
|
||||
This phase is the main architectural risk. Canvas 2D remains a valid
|
||||
compatibility renderer, but full-document synchronous passes will not scale to
|
||||
professional documents.
|
||||
|
||||
## Phase 15: Assisted editing
|
||||
|
||||
Normalize generation, editing, inpainting, segmentation, restoration, and
|
||||
upscaling behind capability-based endpoints. Keep model/provider names out of
|
||||
editor logic. Requests must exclude chat memory, show progress, cancel safely,
|
||||
and return named reversible layers with provenance. Manual tools remain fully
|
||||
usable without an endpoint.
|
||||
|
||||
Much of the endpoint plumbing already exists; the remaining work is consistent
|
||||
capability discovery, lifecycle safety, and editor-native result handling.
|
||||
|
||||
## Phase 16: Release gate
|
||||
|
||||
Run complete user journeys on Chromium and Firefox desktop plus representative
|
||||
phone/tablet viewports. Add keyboard-only and accessibility coverage, mixed
|
||||
20-edit persistence/export tests, failure recovery, and large-document stress
|
||||
tests. No supported operation may silently flatten or discard retained state.
|
||||
|
||||
## Architecture debt to control
|
||||
|
||||
- `galleryEditor.js` is still a large orchestrator. Continue extracting domain
|
||||
modules as visible features move, without a broad rewrite.
|
||||
- Legacy raster adjustment sublayers and first-class adjustment layers overlap.
|
||||
Converge on the first-class model.
|
||||
- Pixel effects still rely heavily on synchronous full-canvas work.
|
||||
- `static/style.css` carries substantial editor-specific surface area and needs
|
||||
clearer component boundaries before workspace customization expands.
|
||||
- The repository worktree contains many unrelated changes. Editor release and
|
||||
merge decisions require a scoped diff or clean integration branch.
|
||||
|
||||
## Recommended execution order
|
||||
|
||||
1. Close and deploy Phase 10.
|
||||
2. Build Phase 11 through a cancellable render boundary.
|
||||
3. Consolidate the workspace in Phase 12.
|
||||
4. Define color/metadata policy and complete Phase 13.
|
||||
5. Finish worker rendering, stress, and recovery in Phase 14.
|
||||
6. Normalize assisted editing in Phase 15.
|
||||
7. Run the cross-browser professional release gate in Phase 16.
|
||||
|
||||
Do not expand into full PSD fidelity, CMYK production, RAW development, 3D, or
|
||||
complete Photoshop parity before this critical path passes. Those are separate
|
||||
product decisions, not prerequisites for a strong Odysseus editor.
|
||||
@@ -15,4 +15,8 @@ markers = [
|
||||
"area_helpers: self-tests for the shared test helpers in tests/helpers/",
|
||||
"area_unit: pure parser / utility tests that do not clearly belong elsewhere",
|
||||
"area_uncategorized: tests not yet matched by the taxonomy (fallback)",
|
||||
# Fast-lane marker (issue #3443). Opt-in and orthogonal to the area_*/sub_*
|
||||
# taxonomy. The fast lane runs `not slow`; mark a test slow only with
|
||||
# duration evidence (see tests/run_focus.py --durations and tests/README.md).
|
||||
"slow: opt-in marker for known-slow tests; excluded by the fast lane (not slow)",
|
||||
]
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
# Optional dependencies — install only if you use the corresponding feature.
|
||||
# Local OCR for screenshots, scans, labels, and coordinate-grounded text extraction.
|
||||
rapidocr==3.9.2
|
||||
onnxruntime>=1.20,<2
|
||||
# The app handles their absence gracefully (clear error message on first use).
|
||||
#
|
||||
# Note: chromadb-client + fastembed moved to requirements.txt — RAG, semantic
|
||||
@@ -12,10 +15,20 @@
|
||||
# GPU-accelerated transcription — it's auto-detected, CPU is used otherwise.
|
||||
faster-whisper
|
||||
|
||||
# Local text-to-speech via Kokoro-82M for the "local" TTS provider.
|
||||
# Kokoro 0.9.4 declares Python >=3.10,<3.13; Odysseus itself requires 3.11+,
|
||||
# so pip installs these extras on 3.11-3.12 and deliberately skips them on
|
||||
# Python 3.13+ (including the Python 3.14 container image). Kokoro declares
|
||||
# torch; the local provider still
|
||||
# requires a CUDA-enabled torch build and GPU at runtime. SoundFile is separate
|
||||
# in Kokoro's official install instructions and is not a transitive dependency.
|
||||
kokoro==0.9.4; python_version >= "3.11" and python_version < "3.13"
|
||||
soundfile; python_version >= "3.11" and python_version < "3.13"
|
||||
|
||||
# DuckDuckGo as a search provider option.
|
||||
# Install if you want DDG in the search-provider dropdown.
|
||||
# Alternatives: SearXNG, Brave, Tavily, Serper, Google PSE.
|
||||
duckduckgo-search
|
||||
ddgs
|
||||
|
||||
# PDF form-filling feature (fillable AcroForm detection, field extraction,
|
||||
# value/annotation/signature stamping, page rendering for the form overlay).
|
||||
@@ -33,4 +46,7 @@ PyMuPDF
|
||||
# magika (onnxruntime), already a core dep via fastembed. We avoid the
|
||||
# [all]/Azure/audio extras (cloud + heavy). Pinned to a release >30 days old per
|
||||
# the dependency-age discussion in issue #485.
|
||||
markitdown[docx,pptx,xlsx,xls]==0.1.5
|
||||
markitdown[docx,pptx,xlsx,xls]==0.1.6
|
||||
|
||||
# Photoshop PSD opening / flattened previews / layer inspection.
|
||||
psd-tools
|
||||
|
||||
@@ -3,10 +3,16 @@ uvicorn
|
||||
python-multipart
|
||||
python-dotenv
|
||||
httpx
|
||||
pydantic>=2.0
|
||||
pydantic-settings>=2.0
|
||||
httpcore>=1.0,<2.0
|
||||
pydantic>=2.13.4
|
||||
pydantic-settings>=2.14.1
|
||||
SQLAlchemy
|
||||
pypdf
|
||||
pypdfium2
|
||||
Pillow
|
||||
faster-whisper
|
||||
PyPDF2
|
||||
pdfplumber
|
||||
beautifulsoup4
|
||||
charset-normalizer
|
||||
numpy
|
||||
@@ -18,6 +24,7 @@ numpy
|
||||
chromadb-client
|
||||
fastembed
|
||||
youtube-transcript-api
|
||||
yt-dlp
|
||||
# Markdown rendering for research reports (src/visual_report.py).
|
||||
# Imported at module-top so it's a hard core dep, not optional.
|
||||
markdown
|
||||
@@ -37,9 +44,16 @@ python-dateutil
|
||||
caldav
|
||||
cryptography
|
||||
bcrypt
|
||||
mcp
|
||||
# Built-in servers use the v1 low-level Server decorator API. MCP SDK v2 is a
|
||||
# breaking rewrite, so keep fresh installs on the maintained v1 line until the
|
||||
# servers are migrated together.
|
||||
mcp<2
|
||||
pyotp
|
||||
qrcode[pil]
|
||||
croniter
|
||||
pytest
|
||||
pytest-asyncio
|
||||
# starlette.testclient prefers httpx2 since Starlette 1.2.0 and warns on every
|
||||
# TestClient import when only classic httpx is present. Runtime code keeps
|
||||
# using `httpx` above; this is test-client only.
|
||||
httpx2
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: artifact-completion
|
||||
description: Create requested artifacts early, iterate from concrete output, and verify final deliverables
|
||||
version: 1.0.0
|
||||
category: agent
|
||||
tags: [artifacts, files, verification, workflow]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
owner: ""
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when the task requires a file, patch, report, document, image, archive, configuration, or other persistent deliverable rather than only a text answer.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Extract the required deliverable path, format, content constraints, and acceptance criteria.
|
||||
2. Inspect the source material and existing target without delaying the first valid artifact.
|
||||
3. Create a minimal complete version at the required location, then iterate from that concrete output.
|
||||
4. Use the format's native parser, renderer, compiler, or test tool to inspect the artifact.
|
||||
5. Repair specific validation, content, or presentation failures while preserving correct portions.
|
||||
6. Confirm the final path, file type, required content, and usability before reporting completion.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not spend the full task budget inspecting without creating the requested output.
|
||||
- Do not place the artifact at a convenient path when the task specifies another location.
|
||||
- Do not use a filename extension as proof that the file is valid in that format.
|
||||
- Do not report completion while placeholders, missing sections, parse errors, or failed checks remain.
|
||||
|
||||
## Verification
|
||||
|
||||
- The artifact exists at the required path and opens or parses successfully.
|
||||
- Required sections, fields, labels, or visual elements are present.
|
||||
- Relevant tests, render checks, or validators pass.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: terminal-recovery
|
||||
description: Recover from failed terminal commands using evidence-driven diagnosis and bounded retries
|
||||
version: 1.0.0
|
||||
category: agent
|
||||
tags: [terminal, shell, debugging, recovery]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
owner: ""
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when a command fails, times out, produces incomplete output, or behaves differently from what the task requires.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Read the command, exit status, standard output, and standard error before choosing a response.
|
||||
2. Confirm the working directory, relevant files, executable availability, permissions, and environment assumptions with minimal read-only probes.
|
||||
3. Classify the failure as syntax, missing dependency, wrong path, permissions, resource pressure, timeout, service state, or task logic.
|
||||
4. Change one relevant condition and retry the narrowest command that can test the diagnosis.
|
||||
5. For a long-running command, use the returned session identifier to poll or provide input instead of launching duplicates.
|
||||
6. After recovery, run the original acceptance check and inspect the resulting files or service state.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not rerun an unchanged failing command repeatedly.
|
||||
- Do not install packages or change global configuration before confirming they are missing and necessary.
|
||||
- Do not launch a second server or training job before checking for an existing process and port or device conflicts.
|
||||
- Do not treat partial output or a zero exit status as proof that the requested state was produced.
|
||||
|
||||
## Verification
|
||||
|
||||
- The diagnosed cause is supported by command output or environment state.
|
||||
- The corrected command exits as expected.
|
||||
- The requested artifact, process, or state passes an independent acceptance check.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: tool-discovery
|
||||
description: Discover the smallest capable tool set and confirm argument schemas before acting
|
||||
version: 1.0.0
|
||||
category: agent
|
||||
tags: [tools, discovery, routing, schemas]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
owner: ""
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when a task requires tools whose names, capabilities, or argument shapes are not already clear. This is especially useful when many tools are available or a previous call failed because the wrong tool or parameters were selected.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Translate the request into required capabilities such as reading, searching, editing, executing, browsing, or verifying.
|
||||
2. Search the tool index for those capabilities and inspect the returned tool descriptions and schemas.
|
||||
3. Prefer one direct tool over a chain of indirect tools when it can complete the operation and provide evidence.
|
||||
4. Check required parameters, identifiers, path rules, side effects, and approval requirements before calling the tool.
|
||||
5. Make a small read-only probe when the environment or target is uncertain.
|
||||
6. Execute the selected action, inspect the result, and only broaden the tool search if the result shows a concrete capability gap.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not guess tool names or argument keys from memory when the index or schema is available.
|
||||
- Do not load unrelated tool groups into context.
|
||||
- Do not repeat the same failed call without changing the arguments or strategy.
|
||||
- Do not use a broad shell or browser workaround when a scoped native tool already owns the operation.
|
||||
|
||||
## Verification
|
||||
|
||||
- The chosen tool directly matches the required capability.
|
||||
- Required arguments follow the exposed schema.
|
||||
- The result contains evidence of the requested effect or a specific error that guides the next step.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: verified-state-change
|
||||
description: Make scoped state changes with target confirmation, minimal mutation, and read-back verification
|
||||
version: 1.0.0
|
||||
category: agent
|
||||
tags: [state, mutation, verification, safety]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
owner: ""
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when creating, editing, deleting, moving, sending, scheduling, or otherwise changing persistent state through an application, API, filesystem, or service.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Read the current state and identify the target using stable identifiers plus enough content to disambiguate it.
|
||||
2. Preserve fields the user did not ask to change and choose the narrowest supported mutation.
|
||||
3. For destructive or externally visible actions, confirm that the user's instruction authorizes the exact target and effect.
|
||||
4. Perform the mutation once and capture the returned identifier, status, or revision.
|
||||
5. Read the target again through an independent list, fetch, status, or content operation.
|
||||
6. Compare the observed state with the requested outcome and repair only the specific mismatch.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not infer the target from a stale active item when a stable identifier can be fetched.
|
||||
- Do not report success from an accepted request alone; asynchronous or partial operations may not have completed.
|
||||
- Do not replace an entire object when a field-level update is supported and safer.
|
||||
- Do not silently broaden a mutation to adjacent files, records, accounts, or services.
|
||||
|
||||
## Verification
|
||||
|
||||
- The target identity was confirmed before mutation.
|
||||
- A read-back shows the intended values and preserves unrelated state.
|
||||
- Any external effect has a concrete status, identifier, or observable result.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: action-evidence-synthesis
|
||||
description: "Turn messages, meeting notes, and documents into sourced decisions, actions, dependencies, and risks"
|
||||
version: 1.0.0
|
||||
category: communication
|
||||
tags: [messages, meetings, actions, status, evidence]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when information is fragmented across messages, meeting notes, transcripts, or documents and the user needs an action list, status summary, feasibility assessment, or executive brief.
|
||||
|
||||
Do not use when the source material is unavailable or when the user only wants a verbatim transcript.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Identify the requested scope, audience, time window, and decision to support.
|
||||
2. Gather the relevant records in full and preserve stable source identifiers, authors, and timestamps.
|
||||
3. Extract explicit decisions, commitments, requests, owners, dates, dependencies, blockers, and changed facts.
|
||||
4. Reconcile revisions by preferring the newest authoritative record; keep unresolved conflicts visible instead of guessing.
|
||||
5. Separate observed facts from inferred owners, dates, urgency, feasibility, or recommendations, and label every inference as tentative.
|
||||
6. Produce the requested format with concise source references beside consequential claims and a final list of open questions.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not turn discussion or speculation into a confirmed decision.
|
||||
- Do not invent owners or deadlines when none were assigned.
|
||||
- Do not silently discard older records that explain a changed commitment.
|
||||
- Do not send messages, create tasks, or update calendars unless the user separately authorizes those actions.
|
||||
|
||||
## Verification
|
||||
|
||||
- Every action has a source, status, and explicit or tentative owner and due date.
|
||||
- Conflicting values and revisions are resolved or visibly flagged.
|
||||
- The output covers decisions, actions, dependencies, risks, and open questions relevant to the request.
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: reviewable-external-draft
|
||||
description: "Reconcile source evidence and prepare an accurate external-facing draft without bypassing review"
|
||||
version: 1.0.0
|
||||
category: communication
|
||||
tags: [drafting, email, messages, review, reconciliation]
|
||||
status: published
|
||||
confidence: 1.0
|
||||
source: builtin
|
||||
created: "2026-08-30T00:00:00Z"
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Use when preparing a client, customer, partner, leadership, or other external-facing update from internal messages or documents.
|
||||
|
||||
Do not use this procedure to send immediately unless the user explicitly authorizes the exact recipient and final content.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Confirm the audience, communication channel, requested tone, and whether the user asked for a draft or an immediate send.
|
||||
2. Gather the relevant source records and identify the latest values, dates, commitments, and unresolved discrepancies.
|
||||
3. Resolve recipient identity through the available contact source and avoid inferring internal versus external status from a display name alone.
|
||||
4. Draft only claims supported by the collected evidence; qualify uncertainty and omit internal-only detail that the audience should not receive.
|
||||
5. Save or present a reviewable draft through the native draft or document capability.
|
||||
6. Report the draft identifier or location plus any reconciliation notes that require human review.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not send a draft merely because a send-capable tool is available.
|
||||
- Do not copy stale figures when a later correction exists.
|
||||
- Do not conceal unresolved discrepancies behind polished prose.
|
||||
- Do not expose private internal discussion, credentials, or unrelated personal data.
|
||||
|
||||
## Verification
|
||||
|
||||
- Recipient identity and communication mode match the request.
|
||||
- Dates, figures, status, and commitments map to current source evidence.
|
||||
- The result remains reviewable unless an explicit send-now instruction authorized delivery.
|
||||
|
||||