mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-09-10 18:22:20 +02:00
16b04a9792d2f776d089340c471dbbcefa237062
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 tanmayraut45
|
c161866199 |
CalDAV: close the DAVClient on sync and write-back paths (#4793)
_sync_blocking (src/caldav_sync.py) and _writeback_blocking (src/caldav_writeback.py) each open their own caldav.DAVClient via _build_dav_client, but never close it. The client owns an HTTP session with a pooled connection; without a close() that connection is held until process exit. Previously the fix added explicit client.close() calls before each early return and at the end of the DB finally block. This still leaked the client when SessionLocal() raised before the DB try/finally was entered. Now _sync_blocking wraps the entire post-construction path in an outer try/finally that calls client.close() unconditionally, covering: - AuthorizationError / NotFoundError early return - URL-fallback failure early return - no-calendars early return - normal return after sync - SessionLocal() construction failure (new regression coverage) _writeback_blocking already used a try/finally (unchanged). - src/caldav_sync.py: replace scattered client.close() calls with a single outer try/finally block around the discovery + DB sync path - tests/test_caldav_client_cleanup.py: add CalendarDeletedEvent to the database stub; add regression test for SessionLocal() failure path Closes #4593 |
||
|
tanmayraut45
|
dfad293494 |
fix(mcp): retain builtin startup tasks and reap npx probe
Keep strong references to builtin MCP startup tasks until completion and kill/reap the npx probe subprocess when cancellation interrupts the probe. Includes focused regression coverage for both lifecycle paths. |
||
|
tanmayraut45
|
d3306d63f7 |
fix(ai): offload model resolution from async paths
Wrap blocking _resolve_model calls in asyncio.to_thread across async model interaction paths so endpoint/model resolution does not stall the event loop. Preserve owner-scoped resolution and add focused regression coverage. |
||
|
tanmayraut45
|
1ac3807820 |
Research CLI: alias --status complete to the stored done value (#2515)
`odysseus-research list --status complete` returns an empty result on
any real corpus. The CLI accepts `complete` as a `--status` choice (the
user-facing label), but the writer in
`services/research/research_handler.py` stores `status="done"` when a
run finishes (and the legacy `src/research_handler.py` copy does the
same). The list filter at `scripts/odysseus-research` was a literal
string compare:
if args.status and (data.get("status") or "") != args.status:
continue
so `--status complete` filtered every finished record out, and the user
saw nothing — even though `odysseus-research list` (no filter) listed
them fine and `show RP_ID` worked on the same files. The other
documented choices — `running`, `cancelled`, `error` — are stored
verbatim by the writer, so the surface mismatch is just on `complete`.
Add a small `_STATUS_CLI_TO_STORED = {"complete": "done"}` map and run
`data.get("status")` through `_status_matches(...)` before comparing.
The other CLI choices fall through unchanged, so the filter still
matches them verbatim. A `None` or non-string `status` (corrupt JSON)
is coerced to `""` and never matches `complete`, so a half-written
record can't sneak past the filter.
`tests/test_research_cli_status_filter.py` covers all four documented
choices, the non-string / missing status case, and pins that the
verbatim choices are NOT rewritten — a blanket mapping that turned
every CLI choice into a stored variant would just re-introduce the
empty-result bug on the running/cancelled/error paths.
Part of #2122.
|
||
|
tanmayraut45
|
4c4b3091ba |
Support extra CA bundle for private-CA LLM providers (#769)
Adding GigaChat (Sber) or an on-premise enterprise LLM gateway as a
model endpoint fails on first probe with
CERTIFICATE_VERIFY_FAILED: self-signed certificate in certificate
chain (_ssl.c:1000)
because their TLS chain is signed by a private root CA (Russian Trusted
Root CA for GigaChat; corporate CA for on-prem) that isn't part of the
default system / certifi trust store. The endpoint shows offline in
the picker even though the URL and API key are correct (issue #722).
The right fix is to extend the trust store, not to weaken verification.
This change:
- src/tls_overrides.py: new module that resolves an opt-in env var
LLM_CA_BUNDLE at import time, builds a shared SSLContext via
ssl.create_default_context() (so the system / certifi bundle is
loaded first) and layers the operator's PEM on top with
load_verify_locations(). Exposes llm_verify() returning a value
suitable for httpx `verify=`. Defaults to True (httpx built-in
trust) when the env var is unset, when the file is missing, or
when the PEM fails to load — verification is never silently
disabled, the warning is logged and we fall back to the safe path.
- src/llm_core.py: thread llm_verify() into the shared AsyncClient
used by stream_llm / streaming completions.
- routes/model_routes.py: thread llm_verify() into the five httpx.get
call sites in _probe_endpoint / _ping_endpoint so adding a
private-CA endpoint goes green on the very first probe and the
picker stops showing it offline.
- .env.example: document LLM_CA_BUNDLE with the GigaChat case as the
concrete example.
Deliberately NOT included: a verify=False knob (global or per-host).
Disabling verification exposes the affected endpoint to MITM, and the
operator-supplied bundle is the correct fix for legitimate private-CA
providers — so the only switch in this PR is the safe one.
Closes #722.
|
||
|
tanmayraut45
|
1f0cb83140 |
Hwfit: estimate params from config.json fallback
`add_hwfit_models.py` infers `parameter_count` and `parameters_raw` by regexing the HF repo name for a `<num>B` token, optionally with an `-A<num>B` MoE active-param suffix. Repos that don't encode a size in their name at all (e.g. `zai-org/GLM-4.5`, where the "4.5" is a version not a parameter count) fall through to the safetensors element-count path. That path works for unquantized FP16 / BF16 repos but is brittle in two cases the catalog hits often: 1. Author-bulk runs (`AUTHORS = ["cyankiwi"]`) pull pre-quantized AWQ / GPTQ / MLX repos. The safetensors metadata stores the packed I32 tensors and a per-dtype `parameters` map, which the script unpacks via a per-quant pack factor. When the upload doesn't populate that map (older repos, custom shards), `st.total` is used raw and the parameter count is off by 4-8x. 2. Repos where the safetensors block is absent from `model_info()` entirely. The current code returns `None` and silently drops the model, which then has to be added to `EXTRA_REPOS` by hand with a literal `parameter_count` string. Both are exactly what the issue calls out — the regex / safetensors combo can't size GLM-4.5 by itself because the name has no `<num>B` and the upstream repo's safetensors block doesn't carry a usable param total either. Add a config.json fallback in front of the safetensors path: - `_fetch_config_json(repo_id)` downloads `config.json` via `hf_hub_download` (so the standard HF on-disk cache handles deduplication across runs, no extra cache layer needed). Network / 404 / gated-repo errors return `None` and the caller proceeds to the safetensors fallback. An in-process `_CONFIG_CACHE` dedupes the base-model vs. source-repo lookups within a single run. - `_params_from_config(cfg)` first honours explicit `num_parameters` / `n_params` / `total_params` fields when present. Otherwise it sums embeddings + attention (GQA-aware via `num_key_value_heads` and `head_dim`) + dense MLP (`3 * hidden_size * intermediate_size`, covering SwiGLU / GeGLU). For MoE configs it picks up both naming conventions in the wild — `num_experts` / `num_experts_per_tok` (Qwen3-MoE) and `n_routed_experts` / `n_shared_experts` (GLM-4-MoE, DeepSeek-V3) — uses `moe_intermediate_size`, and respects `first_k_dense_replace` so the first N layers stay dense. Active parameters come out as `num_experts_per_tok + n_shared_experts` of the routed experts, which matches how each architecture reports its active count. - In `_entry_from_modelinfo`, try config.json on the source repo first (works for unquantized models) and then on the `base_model:` parent (covers AWQ / GPTQ children whose own config is just a quantization manifest). Both lookups run only when regex + override + base_model tag all failed, so the normal author-bulk run still resolves sizes from names without touching the Hub. Spot-checks against the three architecture families this script actually pulls — within ~5% of the documented param counts, which is well inside the `parameter_count` rounding (one decimal of "B") and the `min_vram_gb` downstream bucket: Qwen2.5-7B-Instruct 7.62B (HF card: 7.6B) Qwen3-30B-A3B 30.5B / 3.34B active (card: 30.5B / 3.3B) GLM-4.5 352.7B / 33.6B active (card: 355B / 32B) The safetensors path is unchanged and remains the last resort, so repos with neither a parsable name nor a fetchable config.json behave exactly as before. Closes #955. |
||
|
tanmayraut45
|
c9dc64c018 |
Sessions: ignore list keydown while typing
The list keyboard handler (_onSessionListKeydown) treats Backspace and
Delete as "delete the focused session". When the user double-clicks a
chat to rename it, an <input class="session-rename-input"> is mounted
inside the .list-item row. Backspace on the input bubbles up to the list
container, the handler walks closest('.list-item[data-session-id]') from
e.target, finds the parent row and DELETEs the session via the API —
so a single typo correction nukes the whole conversation.
Bail out at the top of the handler when e.target is an INPUT, TEXTAREA,
or contentEditable element. Arrow / Enter / Delete navigation still
works for rows themselves (the row is the focused element then, not the
input). Mirrors the guard pattern already used in ui.js, notes.js,
tasks.js, calendar.js, emailLibrary.js and galleryEditor.js.
Closes #1007.
|
||
|
tanmayraut45
|
e05c54c8a9 |
Models: detect bare Ollama URLs as online
_ping_endpoint() is the reachability fallback the model-endpoint POST handler invokes when _probe_endpoint() returns no model ids. It GETs base + "/models" and, on any sub-500 response, returns immediately with `reachable = (status < 400)`. That early return runs before the Ollama-native /api/version / /api/tags fallback below it. For an Ollama URL without /v1 (the quickstart accepts both http://localhost:11434 and http://127.0.0.1:11434, and the reporter on #1025 explicitly tried both), the OpenAI-style probe target is http://127.0.0.1:11434/models. Ollama returns 404 there because /models only lives under /v1. _ping_endpoint then returned reachable=False and the picker showed "Added (offline — will retry on next load)" on an install that was running fine. /api/version was never tried. Same shape for http://127.0.0.1:11434/api (the native Ollama root): /api/models is also 404, same premature offline verdict. _probe_endpoint() does fall through to /api/tags on a 4xx (the response raises via raise_for_status), so the endpoint quietly recovers once cached_models becomes non-empty on the next background refresh — matching the second commenter's "had to disconnect manually then reconnect for it to be detected" note. The bug is most visible while no models are pulled yet (cached_models stays empty, _ping_endpoint keeps voting offline). Fix: - Hoist the Ollama-shaped-URL test (port == 11434 or "ollama" in hostname — the same condition _probe_endpoint already uses) to the top of the function so both code paths share it. - Stop short-circuiting on 4xx when the URL looks like Ollama: fall through to the existing /api/version + /api/tags reachability loop so an alive Ollama gets recognised even when its OpenAI surface has the wrong prefix for the user's input. - Fix the `root` computation in that loop to strip a trailing /api as well as /v1, so http://127.0.0.1:11434/api no longer gets probed at /api/api/version. - 4xx on non-Ollama hosts keeps the current semantics: a 401 from api.openai.com/v1/models is still a definitive offline verdict, not a reason to GET /api/version on OpenAI. Closes #1025. |
||
|
tanmayraut45
|
446d5863fa |
Apply SafeSearch by default across search providers (#763)
#718 reported Deep Research drifting into adult / spam URLs several rounds into a benign session ("research about https://bhagathgoud.com/ and what he doing currently"). The reporter's log showed Japanese adult sites being crawled even though the model was emitting normal queries like "Bhagath Goud LinkedIn" and "site:bhagathgoud.com". The model wasn't generating those URLs. Every provider call site constructed its params dict without a SafeSearch parameter, so the underlying HTTP backend (the duckduckgo-search library / DDG's HTML endpoint in this case) was free to surface "related search" / trending / spam recommendations that have nothing to do with the user's query. Per provider: - SearXNG: instance-dependent; many self-hosted instances default to safesearch=0. - Brave API: defaults to "off" for new API keys. - duckduckgo-search lib: defaults to "moderate", which still lets related-search recommendations and HTTP-backend fallback URLs surface trending non-English spam topics. - DDG HTML fallback (html.duckduckgo.com): no `kp` param, treated as off. - Google PSE: omitted `safe` is equivalent to off. - Serper: omitted `safe` proxies to Google with safe off. Since the bad URLs entered through the provider layer, not the model, the provider params are the right place to gate this. Changes: - src/settings.py: new `search_safesearch` setting with default "strict". Documented values ("strict" | "moderate" | "off") plus a few aliases ("on", "high", "0/1/2", "disabled", ...) so a hand-edited config doesn't silently fall through to off. - src/search/providers.py: - Add `_get_safesearch_level()` (canonical, normalizing) and `_safesearch_for(provider)` (per-provider param translation). - Thread the per-provider value into every params dict: SearXNG JSON, SearXNG language/engines fallbacks, SearXNG HTML, Brave, DDG library, DDG HTML fallback, Google PSE, Serper. - Tavily is left untouched — its API has no SafeSearch knob and its index already filters explicit content at ingest time. Behavior change for existing installs: default is now "strict", so explicit results get filtered across every supported provider without any user action. Users who deliberately want unfiltered results can set `search_safesearch` to "off" in Settings. No new dependencies, no schema migrations. Closes #718. |
||
|
tanmayraut45
|
f2d2649f47 |
Expose manage_notes via native function calling (#759)
The agent's RAG tool selector retrieves manage_notes as relevant for note / todo / reminder requests, but two gaps stopped it from actually firing on local llama.cpp / vLLM endpoints: 1. FUNCTION_TOOL_SCHEMAS had no entry for manage_notes. Even when the tool was marked relevant, no JSON schema was sent on the function tools list, so native-function-calling models had nothing to call. In practice the model would describe creating the note in prose while the actual note stayed blank — the symptom reported in #713 ("checklist hallucinated as blank"). 2. _API_HOSTS only listed hosted providers (OpenAI, Anthropic, etc.). For local endpoints like http://localhost:8080 or http://host.docker.internal:8000, _is_api_model fell back to keyword-sniffing the model name, so any model whose slug didn't happen to match the keyword list silently lost native tool schemas entirely. Fixes: - src/tool_schemas.py: add a manage_notes function schema covering list/add/update/delete/toggle_item with the full Keep-style field set. note_type is exposed as an enum ("note" | "checklist") so the model picks the mode explicitly instead of inferring it from content shape. Items are named checklist_items in the schema — consistent with the description's wording and avoiding the Python-built-in name clash that #713 calls out. - src/tool_implementations.py: do_manage_notes accepts both checklist_items (new, schema-exposed) and items (legacy / internal). Direct API callers and existing code paths keep working unchanged; native function calls following the new schema route through the same path. - src/agent_loop.py: add localhost, 127.0.0.1, and host.docker.internal to _API_HOSTS so the function-tool path is not gated behind model-name guessing for local servers. Closes #174. Closes #713. |
||
|
tanmayraut45
|
8a1ef47338 |
Support in-place endpoint updates and recover empty-model sessions (#786)
The "don't wipe endpoint_url/model on endpoint delete" half of #587 landed
in
|
||
|
tanmayraut45
|
bc4dcf703b |
Honor AUTH_ENABLED=false in route-level auth gate (#785)
#622 reported "I cant even paste that hash pw and granted So auth_en =false & localbypass= true But then the host still is showing login page?" — the operator turned auth off in .env and still gets bounced to /login on every page load. The flow: The auth middleware in app.py is correctly gated on AUTH_ENABLED, so the middleware itself does not install when AUTH_ENABLED=false. The SPA front-end at static/app.js wraps window.fetch and redirects to /login on ANY 401 response from any API call. So all it takes for the operator to see a login page is one route-level 401. src/auth_helpers.require_user — the shared FastAPI dependency mounted on ~50 routes (email, contacts, personal, …) — was the source. It is documented as defense-in-depth in case the middleware was bypassed unexpectedly (SSRF from a sibling service), but the implementation treated AUTH_ENABLED=false as one of those unexpected bypasses and 401'd anyway. The loopback fall-through that would have admitted the operator does not fire under docker compose / a reverse proxy because the container sees the request arriving from the bridge gateway (172.x.x.x), not 127.0.0.1. require_user now short-circuits to "" when AUTH_ENABLED=false so the explicit operator opt-out reaches the route layer too. While in the file, also mirror LOCALHOST_BYPASS=true the same way for loopback callers — the middleware already lets them through, and routes 401'ing the same caller would produce the same /login bounce. Non-loopback callers under LOCALHOST_BYPASS are still rejected, matching the middleware's _is_trusted_loopback check. Add three focused regression tests in tests/test_security_regressions.py: docker-bridge caller is admitted under AUTH_ENABLED=false, loopback caller is admitted under LOCALHOST_BYPASS=true, LAN caller under LOCALHOST_BYPASS=true is still rejected. The existing test_require_user_rejects_unauthenticated and test_require_user_accepts_loopback_when_unconfigured tests continue to pass because neither sets AUTH_ENABLED, so the AUTH_ENABLED=true default path is unchanged. Closes #622. |
||
|
tanmayraut45
|
612089572a |
Exempt task webhook trigger from session auth (#784)
POSTing to the per-task webhook URL shown in the Tasks UI returned 401
Unauthorized even though the URL is labelled "no auth needed". The
trigger handler at routes/task_routes.py:873 (`POST
/api/tasks/{task_id}/webhook/{token}`) was written as an
unauthenticated endpoint — the 32-byte path-embedded `webhook_token`
generated by `secrets.token_urlsafe(32)` is the credential, and the
handler validates it against the row before doing anything. But
AuthMiddleware in app.py runs first and only knows about
AUTH_EXEMPT_EXACT (static path set) and AUTH_EXEMPT_PREFIXES (only
`/static`), so every external POST (curl, Zapier, n8n, Make,
Activepieces) got rejected before the route ever saw the request.
External callers can't supply a session cookie, which is precisely
why the per-task token exists.
Fix: add an AUTH_EXEMPT_PATTERNS list of compiled regexes for dynamic
public paths and route `^/api/tasks/[^/]+/webhook/[^/]+/?$` through
it. The route handler still enforces `ScheduledTask.webhook_token ==
token` and 404s on mismatch, so an attacker without the token gets a
404 (indistinguishable from a non-existent task), and a holder of the
token gets the documented "POST and a task fires" behaviour. The
sibling endpoint `/{task_id}/webhook-regenerate` is admin-gated and
deliberately does NOT match the pattern — it requires `_owner(request)`
and a session.
Tests: tests/test_webhook_trigger_auth_exempt.py extracts the regex
list out of app.py, applies it to a representative trigger path
(positive) and the four neighbouring task paths that must stay
authenticated (negative — `/api/tasks`, `/api/tasks/{id}`,
`/api/tasks/{id}/webhook-regenerate`, `/api/tasks/{id}/run`), and
pins the handler-side token check so a refactor of the route doesn't
quietly turn the endpoint into a truly anonymous one.
Closes #621.
|
||
|
tanmayraut45
|
af20b8be30 |
Lift deep-research hard timeout into a setting (#783)
The 600s wall-clock cap in research_handler.start_research was too short for local / edge LLMs to finish a deep-research synthesis — long extraction passes plus a slow final report routinely blew past 10 minutes and the run was killed with partial results. Introduce research_run_timeout_seconds (default 1800s = 30 min) in DEFAULT_SETTINGS and resolve it at start_research entry when the caller hasn't pinned hard_timeout. Bound the resolved value at [60, 86400] so a misconfigured settings.json can't either disable the safety net or explode into a multi-day hang. Existing call sites in research_routes.py and chat_routes.py keep working unchanged — they don't pass hard_timeout and now pick up the new default. Closes #595. |
||
|
tanmayraut45
|
12bc2c3025 |
Fix TOCTOU race in chat stream status endpoint
The /api/chat/stream_status handler did a membership test against _active_streams followed by an indexed read of the same key. Between those two ops, a sibling stream's finally block (or a stop / cleanup path) can pop the entry, turning the indexed read into a KeyError that bubbles up as a 500. The race is the exact one _stream_set was already written to avoid; the comment on the helper at the top of the module spells out why a single .get() is the right pattern here too. Collapse the two-step into a single .get() call so the lookup either returns the live record or None, and report 'detached' / 404 based on that single read. No behavior change on the happy path; the failure mode under concurrent stream cleanup is now handled deterministically. Closes #658. |
||
|
tanmayraut45
|
a113ee4f6f |
Fix searxng container permission errors during setup
A fresh `docker compose up -d` shows the searxng container failing its healthcheck with permission errors at setup (reported in #721 — the service comes up under names like `odysseus_searxng_1` and never goes ready, which then blocks the main odysseus container because of the `depends_on: searxng: condition: service_healthy` gate). Root cause: the official `searxng/searxng:latest` image runs as the non-root `searxng` user but its entrypoint still needs to 1. chown /etc/searxng on first boot so the persisted named volume is owned by the searxng user inside the container, 2. su-exec to drop / re-assert privileges before launching uwsgi, and 3. let our wrapper entrypoint (which seeds settings.yml into the named volume on first boot) write the file through the volume mount. Without explicit `cap_add`, the container has neither CHOWN nor DAC_OVERRIDE nor SETUID/SETGID, so the entrypoint aborts at the first chown / su-exec / redirection with EACCES. The upstream searxng-docker compose file solves this with the standard "drop everything, grant only what's needed" capability pattern. Fix: mirror the upstream cap_drop ALL / cap_add CHOWN, SETGID, SETUID, DAC_OVERRIDE on the searxng service. This grants only the four caps the entrypoint actually needs, matches what searxng-docker ships with, and leaves ports, volumes, env, healthcheck, and the wrapper entrypoint unchanged. Closes #721. |