chore: release v1.9.1

Ships the Wiki.js change listener supervision fix. Patch release: no API
change, no migration — the listener now reconnects after a database restart
instead of going silently deaf, and /health reports its subscription state.

Co-Authored-By: Claude <noreply@anthropic.com>

.env.example

@@ -1,15 +1,15 @@
 # Service URLs for local dev (pointing to your server)
 TEST_HOST=192.168.86.149
-WIKIJS_URL=http://192.168.86.149:8088
-NEO4J_URI=bolt://192.168.86.149:7687
+WIKIJS_URL=http://wiki:3000
+NEO4J_URI=bolt://neo4j:7687
 QDRANT_HOST=192.168.86.149
 QDRANT_PORT=6333
-OLLAMA_URL=http://192.168.86.149:11434
-SEARXNG_URL=http://192.168.86.149:8080
+OLLAMA_URL=http://ollama:11434
+SEARXNG_URL=http://searxng:8080
 REDIS_HOST=192.168.86.149
-PAPERLESS_URL=http://192.168.86.149:8091
+PAPERLESS_URL=http://paperless:8000
 
-OLLAMA_MODEL=mistral-nemo-large:latest
+OLLAMA_LLM_MODEL=gemma4:e2b
 OLLAMA_EMBEDDING_MODEL=nomic-embed-text
 
 # Wiki.js auth

.gitea/workflows/build.yml

@@ -1,19 +1,32 @@
 name: Build and Push
 
 on:
-  release:
-    types: [published]
+  push:
+    tags:
+      - 'v*'
 
 jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Create Gitea Release
+        run: |
+          curl -sf -X POST \
+            -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
+            -H "Content-Type: application/json" \
+            -d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
+            "${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
+
   build:
     runs-on: ubuntu-latest
+    needs: release
     steps:
       - uses: actions/checkout@v4
 
       - name: Login to Gitea Registry
         uses: docker/login-action@v3
         with:
-          registry: git.schweitz.internal
+          registry: git.schweitz.net
           username: ${{ secrets.REGISTRY_USER }}
           password: ${{ secrets.REGISTRY_PASSWORD }}
 
@@ -23,8 +36,8 @@ jobs:
           context: .
           push: true
           tags: |
-            git.schweitz.internal/jpmschweitzer/library-desk:latest
-            git.schweitz.internal/jpmschweitzer/library-desk:${{ github.ref_name }}
+            git.schweitz.net/jpmschweitzer/library-desk:latest
+            git.schweitz.net/jpmschweitzer/library-desk:${{ github.ref_name }}
 
       - name: Trigger Watchtower update
         if: success()

AGENTS.md

@@ -47,7 +47,7 @@ When changes are ready for deployment:
 5. **CI/CD triggers automatically**:
    - Gitea CI builds Docker image on new tag
    - Watchtower pulls and deploys to production
-   - Verify deployment: `curl http://192.168.86.149:8000/health`
+   - Verify deployment: `curl http://192.168.86.149:8089/health`
 
 ---
 

CHANGELOG.md

@@ -5,6 +5,182 @@ All notable changes to Library Desk will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+## [1.9.1] - 2026-08-08
+
+### Fixed
+
+- The Wiki.js change listener now survives a database restart. It held a single
+  `LISTEN` connection with no supervision, so when the connection dropped it was
+  gone permanently while `running` stayed `True` — the service kept reporting
+  healthy and silently stopped indexing every page edit until someone restarted
+  the container. This happened for real on 2026-08-08 when `postgres-shared` was
+  redeployed. It now detects the drop via asyncpg's termination callback and
+  reconnects with bounded exponential backoff (1s doubling to a 60s cap),
+  retrying indefinitely because a database in maintenance does come back and
+  giving up would recreate the same silent deafness.
+- `WikiChangeListener.running` is derived from the live connection instead of
+  being assigned once at startup, so it can no longer claim a subscription that
+  does not exist.
+
+### Added
+
+- `/health` reports the change listener under `services.wiki_listener`
+  (`subscribed`, `reconnects`, `last_gap_seconds`). Nothing previously exposed
+  its state anywhere, which is why a dead listener went unnoticed. It is
+  deliberately excluded from the overall healthy/degraded verdict: it recovers on
+  its own, and flipping the container unhealthy for the duration of a database
+  outage would add a restart loop to an incident rather than information.
+- On reconnect the listener logs the outage duration and warns that `NOTIFY`
+  events emitted during the gap were lost and cannot be replayed, pointing at
+  `POST /maintenance/integrity-check` to reconcile. A reconciliation pass is not
+  performed automatically — tenant attribution for a changed page is non-trivial
+  here, and guessing it wrong writes content into the wrong user's namespace.
+
+## [1.9.0] - 2026-07-20
+
+### Security
+
+- The Wiki.js integration buttons no longer embed an API key in the browser. The re-index and entity-link endpoints now authenticate via the NPM `/library-desk/` proxy (Authentik session for external users, LAN bypass for internal), verified by a trusted proxy marker header. The previously-embedded key was a full-privilege key served to every wiki visitor; it has been rotated out of service.
+- `verify_api_key` now uses a constant-time comparison.
+
+### Changed
+
+- `static/wikijs-integration.js` calls library-desk same-origin (`/library-desk/...`) with `credentials: same-origin` and no `Authorization` header. Update the Wiki.js code-injection snippet to `<script src="/library-desk/static/wikijs-integration.js">`.
+- Machine callers (the Scheduler) continue to use the Bearer API key against the container-network endpoints; only the two browser endpoints switched to proxy auth.
+
+## [1.8.1] - 2026-07-19
+
+### Changed
+
+- Container images are now pushed via the `git.schweitz.net` registry endpoint (the `.internal` registry domain is being retired); no change to the image name consumed by Watchtower.
+- `.env.example` service URL defaults now use docker-dataplane container names (`wiki:3000`, `neo4j:7687`, `searxng:8080`, `paperless:8000`, `ollama:11434`) instead of host IP + published port, ahead of the Phase 4 port lockdown; AGENTS.md deploy health-check URL corrected from port 8000 to 8089.
+
+## [1.8.0] - 2026-07-14
+
+### Added
+
+- **Stub endpoints implemented** (`/ingest/check-updates`, `/ingest/status/{job_id}`, `/ingest/repo-status/{repository}`, `/deduplicate/check`) — all previously returned canned "not yet implemented" responses; all now require an explicit `user` (Phase B rule):
+  - `/ingest/check-updates` compares the `content_hash` now recorded on the tenant's Neo4j Document nodes at ingestion time against the SHA-256 of current Wiki.js page content in a single UNWIND Cypher query, returning `changed` / `new` / `deleted` page lists (auto-generated entity stubs excluded; documents whose stored hash predates hash tracking are flagged `stored_hash_missing` and count as changed).
+  - `/ingest/status/{job_id}` is backed by the Redis `JobManager` (jobs are tenant-scoped; other tenants' jobs return 404). `/ingest/page`, `/ingest/batch` and `/ingest/all` now record job entries and return a `job_id`.
+  - `/ingest/repo-status/{repository}` reports wiki page count vs indexed Document-node count under `users/{tenant}/{repository}` plus the tenant's Redis job statistics.
+  - `/deduplicate/check` runs a tenant-scoped Qdrant similarity scan: wiki chunk pairs above the threshold (default 0.9 cosine) grouped per page pair with best score, matching chunk-pair count, and page references. Read-only.
+- **Job + Scheduler task plumbing** — In-process hourly `job_cleanup_loop` (started at app startup, cancelled at shutdown) reclaims expired Redis job-set memberships (`JobManager.cleanup_expired_jobs`). `docs/scheduler-tasks.md` defines the four production Scheduler task payloads for the deploy checklist (nightly integrity 04:30, weekly quality report Sunday 03:00, daily Paperless orphan-cleanup 05:00 on the existing endpoint, and disabling `test_example_task`) with exact HTTP bodies (explicit `user=jpmschweitzer`, `${LIBRARY_API_KEY}` auth placeholder). `scripts/register_scheduler_tasks.py` reads the Scheduler API location from `SCHEDULER_URL` and registers them — dry-run by default (prints payloads), `--execute` gated and requiring `LIBRARY_API_KEY`.
+- **Weekly quality report** — `POST /maintenance/quality-report {user}` runs the duplicate scan, flags stale pages (not updated in N days AND ≤ M SearchQuery hits from the graph data), lists pages missing tags/description, folds in the latest integrity-check results (Redis-cached or run inline), and writes a dated report page to `users/{user}/system/quality-reports/YYYY-MM-DD` (same-day reruns update the same page — the page id is remembered in Redis because the Wiki.js listing lags page creation). Response returns the full report content + page path. Verified end-to-end against the local dev server as `llm_tester`.
+- **Nightly integrity check** — `POST /maintenance/integrity-check {user}` (read-only: reports, never auto-fixes) reports per tenant: wiki pages with ZERO vectors in Qdrant (silent-skip reindex victims), orphaned vectors whose wiki page no longer exists, unexpected Qdrant collections (test-tenant residue and unknown namespaces flagged; other services' collections counted as foreign), Neo4j Document nodes without wiki counterparts, plus counts and duration. The latest report is cached in Redis (30 days) so the weekly quality report can fold it in.
+
+### Fixed (Phase D review batch)
+
+- **Runtime prefetch registration is executable** - `SchedulerClient.register_volatile_fetch` (used by consolidation's `_register_prefetch`) registered tasks that were dead on arrival three ways: the JSON body sat under the ignored `body` key (`rest_api_executor` only reads `config["payload"]`), there was no `auth` block (the scheduled POST would 401 against library-desk's `verify_api_key`), and `user` was in the body while every `/volatile/fetch` endpoint requires it as a QUERY parameter (would 422). The config now puts `user` in the URL query string (URL-encoded), an empty `payload`, and `auth: {type: bearer, token: "${LIBRARY_API_KEY}"}` (substituted from the Scheduler's environment; never stored raw). `SchedulerClient` itself also sent no Authorization to the Scheduler API, so registration failed silently at consolidation time — it now sends `Authorization: Bearer <SCHEDULER_API_KEY>` (new `scheduler_api_key` setting; a client without a key logs a warning).
+- **Document sync uses delete-last reindex order** - `DocumentSyncService._index_vectors` deleted the document's existing chunks (`delete_by_filter`) BEFORE embedding, so a failed embedding pass (e.g. Ollama down) left the Paperless document with zero vectors until the next successful sync — the exact hazard already fixed for wiki pages in `VectorService.update_from_page`. Chunk point ids are now deterministic (uuid5 of `document_{id}_chunk_{i}`, replacing random uuid4) so re-upserting overwrites in place; new points are upserted first and stale points (including legacy uuid4 ones) are pruned afterwards, only after a successful upsert.
+- **5MB response cap now aborts the download** - `ContentExtractor._fetch` buffered the entire response body in memory before truncating to `MAX_RESPONSE_BYTES`, so the cap protected Trafilatura but not memory/bandwidth (a multi-hundred-MB URL was still fully downloaded, on up to `max_urls_per_batch` concurrent fetches). Fetches are now streamed (`client.stream` + `aiter_bytes`) and the connection is closed as soon as the cap is reached.
+- **Registrar authenticates to the Scheduler API** - `scripts/register_scheduler_tasks.py --execute` sent no `Authorization` header while the Scheduler's task-management endpoints are Bearer-guarded (`verify_api_key`: 401 on missing key), so the existence probes 401'd (misread as "task absent") and every registration failed; only the public `/health` gate passed. `--execute` now requires `SCHEDULER_API_KEY` in the environment (refuses to run without it, key is never stored) and sends `Authorization: Bearer $SCHEDULER_API_KEY` on all of its own HTTP calls. Deploy note updated alongside the existing `LIBRARY_API_KEY` requirement.
+
+### Fixed (hazards batch)
+
+- **CORS wildcard + credentials removed** - `allow_origins=["*"]` combined with `allow_credentials=True` told browsers to attach credentials for any site. Credentials are now disabled (all real callers are server-to-server and use the `Authorization` header, which wildcard-origin CORS without credentials still permits) and the origin list is configurable via `CORS_ALLOW_ORIGINS` (comma-separated, default `*`).
+- **Scheduler task auth no longer stores raw tokens** - the Phase C Scheduler task definitions put `Authorization: Bearer ${LIBRARY_API_KEY}` in plain `headers` and the registrar substituted the REAL key client-side on `--execute`, which would persist it in the Scheduler's `scheduled_tasks.config` JSONB column — and the Scheduler's `rest_api_executor` does not substitute env vars in plain headers anyway (only in `url`/`payload`/`auth`). The definitions now use the executor's `auth: {type: bearer, token: "${LIBRARY_API_KEY}"}` block, substituted from the SCHEDULER's environment at execution time; the registrar sends the placeholder verbatim and no longer needs (or accepts) the key. Also fixed: the JSON body moved from the ignored `body` key to `payload` (the executor only reads `config["payload"]`, so the tasks would have POSTed empty bodies and failed Phase B user validation).
+- **Reranker index parser dedupes** - an LLM ranking answer like `3,3,1` inserted the same result into the final ranking twice; parsed indices are now deduplicated preserving first occurrence.
+- **Single HybridRAG wiring point** - three separate constructions existed: an unused `dependencies.get_hybrid_rag_service` singleton lacking `volatile_service`, an inline per-request copy in the `/query/hybrid` router, and another inline copy in `/wiki/pages/smart-create` also lacking `volatile_service`. All callers now use the dependencies singleton, which includes `volatile_service` (smart-create research can now hit the volatile cache leg).
+- **Hot-path Neo4j writes use write transactions** - remaining graph writes ran as auto-commit `execute_query` calls (no retry, no transaction-function semantics): `GraphService` ingestion (`update_from_page` document + entity queries), `delete_page`, `create_entity_mentions`, document/paperless/collection node deletion, orphan-entity purge, stale-document purge, `cleanup_broken_relationships`; the webhook rename/delete cleanup writes; document-sync `_index_graph`; and consolidation's `_mark_search_processed`/`_add_entity_to_graph`. All now go through `execute_write` (managed transaction with driver retry). Read paths are unchanged.
+
+### Changed (performance)
+
+- **Content extractor hardened** - `ContentExtractor` now downloads pages with `httpx.AsyncClient` under real connect (3s) and read timeouts on the event loop; only the CPU-bound Trafilatura parse runs in the thread pool. Previously `trafilatura.fetch_url` ran inside the worker thread with no caller-side timeout control, so an `asyncio.wait_for` timeout abandoned the thread while it kept downloading for up to ~30s. Trafilatura now parses each document ONCE via `bare_extraction` (text + metadata together) — the old code ran `extract()` twice (the XML pass was computed and discarded) plus `bare_extraction`, three full parses per page. `extract_batch` caps full-page extractions per call (default 8; overflow URLs return unsuccessful so the web leg falls back to the search snippet), responses are capped at 5MB before parsing, and thread-pool queue depth is logged for backpressure visibility.
+- **Top-k enrichment with one batched lookup** - Phase 3 (`_enrich_with_related_dossiers`) ran a sequential Neo4j round-trip for EVERY fused result and the final trim then discarded most of the output. It now enriches only the results that can still reach the response (the Phase 4 rerank slice of 20 when reranking is enabled, otherwise `final_result_count`) and resolves all of them in ONE UNWIND-batched Cypher query (`GraphService.get_related_documents_batch`, tenant-scoped like the single-page variant, per-page ordering/limit preserved). Unenriched tail results carry an empty `related_dossiers` list as before.
+- **Search persistence off the hot path, one atomic transaction** - HybridRAG Phase 6 (`_persist_search_for_librarian`) no longer gates the `/query/hybrid` response: the `search_id` is generated up front and returned immediately while the Neo4j write runs as a background task (strong task references held so tasks are not GC'd mid-flight). The write itself collapsed from ~21+ sequential auto-commit queries (SearchQuery node + per-document FOUND links + per-web-result WebResult nodes) into ONE UNWIND-based `execute_write` transaction, so a mid-way failure can no longer leave a partial SearchQuery graph behind. The persisted shape (SearchQuery properties incl. `processed: false`, tenant labels, `FOUND` relationship properties, WebResult properties) is unchanged and pinned by `tests/test_search_persistence.py` against exactly what the consolidation service queries. `timing.persistence_ms` now reports 0 (no longer on the request path).
+- **Batched embeddings + delete-last reindex** - `OllamaClient.embed_batch` now sends ONE batched `/api/embed` request (verified against the live Ollama; the old "batch" looped one `/api/embeddings` call per chunk) with a per-text fallback preserving partial-success semantics. `VectorService.update_from_page` embeds all chunks in that single call and upserts them in one Qdrant batch, and the reindex order is reversed: new points are upserted BEFORE stale points are pruned (deterministic uuid5 chunk ids make the overwrite safe), so a mid-way failure can no longer leave a page with zero vectors — the old order deleted everything first. The summary now reports `status` (`success`/`partial`/`failed`) and `chunks_skipped` instead of unconditional `success=True`; a fully failed embedding pass keeps the old vectors and reports failure. Measured on a real 7-chunk page ingest as `llm_tester` against the local server: ~375ms → ~181ms median (3 runs each).
+- **Document sync indexing fixed** - `DocumentSyncService._index_vectors` now awaits `ensure_collection` (the coroutine was created but never ran, so fresh tenants had no collection at upsert time), filters out `None` entries from `embed_batch` so one failed chunk embedding no longer aborts the whole document upsert (all-failed still reports failure), and routes the raw `client.delete`/`client.upsert` calls through the async wrapper (`delete_by_filter` / new batch `upsert_points`). Offline unit tests added.
+- **Async Qdrant client** - `QdrantClientWrapper` now uses `AsyncQdrantClient` with an explicit timeout (`QDRANT_TIMEOUT`, default 30s). Every vector call previously ran on the synchronous client inside async wrapper methods, blocking the FastAPI event loop for the duration of each Qdrant round-trip. The wrapper API is unchanged (all methods were already `async`), so call sites only gained real awaits. The HybridRAG document leg was moved off the deprecated raw `client.search` onto the wrapper's `search_vectors` (fixing a latent `AttributeError`: it called the nonexistent `ollama.embed_text`, so the leg always reported `failed`), and the health check awaits `get_collections`.
+
+### Changed
+
+- **BREAKING: `user` is now required on every tenant-data endpoint** - The implicit `jpmschweitzer` default tenant (`DEFAULT_USER`) has been removed everywhere. All endpoints that read or write tenant data (`/query/*`, `/wiki/*`, `/vector/*`, `/graph/*`, `/ingest/*`, `/volatile/*`, `/documents/*`, `/stats`, `/rag/search`) now reject requests without an explicit, non-empty, non-whitespace `user` (HTTP 422), matching the existing `/maintenance/*` pattern. A shared validator (`require_user` dependency / `RequiredUser` model type) also rejects blank users. The Wiki.js change listener now skips changes whose notification email yields no user instead of attributing them to the production tenant. **Caller coordination required:** tatlock and the Scheduler ingest/prefetch/consolidation tasks must send an explicit `user` on every call — see the deploy checklist.
+
+### Fixed (security)
+
+- **Cross-tenant leaks in HybridRAG legs and ingestion closed** - A live probe as `user=llm_tester` returned `jpmschweitzer` pages. Root causes fixed:
+  - **Ingestion namespace enforcement**: `vector` and `graph` `update_from_page` now refuse pages whose wiki path is outside `users/{user}/` (previously any tenant could ingest any page id — including another tenant's — into its own collection/labels, which is how foreign content ended up in the vector leg). `/ingest/all` clamps `path_prefix` to the caller's namespace (400 on cross-tenant prefixes) and defaults to `users/{user}`.
+  - **Search persistence**: the `FOUND` link in HybridRAG phase 6 matched `(d:Document {page_id})` unscoped, attaching the caller's SearchQuery to other tenants' Document nodes; it now matches only `User_{Tenant}_Document` nodes.
+  - **Graph enrichment/consolidation queries scoped**: `_get_entity_mention_count`, entity-stub generation, orphan-entity find/purge, and `cleanup_broken_relationships` matched unscoped `Document`/`SearchQuery` nodes; all now use the tenant's labels. Entity-page existence checks list only the tenant's wiki namespace.
+  - **Volatile collections sanitized**: `volatile_{user}` collection names now use the sanitized user id (same scheme as document collections).
+  - **Namespace matching hardened**: `is_path_in_user_namespace` now enforces a path-segment boundary (`users/llm_tester2` is no longer inside `llm_tester`'s namespace) and compares sanitized tenant segments.
+  - Offline unit tests added per leg (vector, graph, volatile, documents, enrichment, persistence, ingestion) asserting the tenant-scoped collection/label/path is used.
+
+- **`/query/graph` and `/graph/query` hardened to read-only** - The documented "automatic user scoping" was a no-op (a live probe confirmed any user string could read the whole graph) and the client permitted writes. Raw Cypher queries are now (1) rejected with 400 when they contain write clauses (`CREATE`/`MERGE`/`DELETE`/`DETACH`/`SET`/`REMOVE`/`DROP`/`FOREACH`/`LOAD CSV`) or any `CALL` procedure (conservative denylist on the uppercased query), and (2) executed through a Neo4j session opened with `default_access_mode=READ_ACCESS` so the database itself refuses writes as a backstop. The endpoints are now honestly documented as **admin/debug, unscoped read-only**: results are not restricted to the caller's tenant labels — use `/graph/nodes` for tenant-scoped access.
+
+### Added
+
+- **Degradation signaling** - `HybridRAGResponse` now includes `source_status` (per-leg `'ok'`/`'failed'`/`'disabled'` for vector, graph, web, volatile, documents) and `degraded` (true when any enabled leg failed). Retrieval legs report errors instead of silently swallowing them; failed legs are logged at WARNING. Both fields are additive and optional, so clients that ignore them are unaffected.
+- **Hard-isolated test-tenant lifecycle for the test suite** - `tests/conftest.py` rewritten: the production host default (`192.168.86.149`) is gone (`TEST_HOST` env, safe `localhost` default; the API under test is the local wakeup server via `LIBRARY_DESK_URL`, never the production container on 8089). The suite is pinned to the reserved test tenant `llm_tester`; a session guard aborts the entire run if the effective tenant is `jpmschweitzer` or outside the reserved `llm_tester*` namespace. Integration tests are marked and only run with `RUN_INTEGRATION_TESTS=1` (plus a passing guard). A session-scoped teardown deletes ALL `llm_tester` artifacts created during the run — Qdrant `*_llm_tester` collections, Neo4j `User_Llm_Tester*`-labelled nodes, the `users/llm_tester` wiki subtree, and `llm_tester` Redis keys on the service DB — with hard tenant assertions before every delete. Legacy integration tests were pinned to the test tenant (no more production-namespace reads).
+- **Live tenant-isolation test** - `tests/test_tenant_isolation_live.py` (integration-marked, guard-gated): creates and ingests a wiki page as `llm_tester` against the local wakeup server + shared services, asserts `/query/hybrid` as `llm_tester` retrieves its own content with ZERO results from the `jpmschweitzer` tenant, and asserts a third nonexistent tenant (`llm_tester_void`, still inside the reserved namespace — nothing is ever written as `jpmschweitzer`) gets zero results entirely. Teardown removes everything it created.
+- **Test-residue purge tooling** - `scripts/purge_test_artifacts.py`: dry-run by DEFAULT (`--execute` required for real deletion), targets only confirmed test residue (Qdrant `library_desk_llm_tester` / `test_user` / `library_desk_test_user` / `memories_llm_tester` / `volatile_llm_tester` / `core_ai_user_test_*` / anything containing `llm_tester`; Neo4j `User_Llm_Tester*`-labelled nodes and legacy `users/llm*` Document nodes; `llm_tester` Redis keys on the service DB), prints counts per target, hard-aborts if any target rule ever matches a `jpmschweitzer`-namespaced identifier, and documents the snapshot prerequisite (Qdrant snapshot API + `neo4j-admin database dump`) in its docstring.
+- **Offline unit tests** - New mock-based tests (no live services) for model-name resolution under the env collision, per-leg failure signaling, the `/stats` page-count prefix, LLM-call timeouts, and Wiki.js listing pagination.
+
+### Fixed
+
+- **Ollama generation model env collision** - Renamed the generation-model setting `ollama_model` to `ollama_llm_model` (env: `OLLAMA_LLM_MODEL`, default `gemma4:e2b`). The container env `OLLAMA_MODEL=nomic-embed-text` (meant for embeddings) was shadowing the generation model, breaking Phase 0 keyword extraction and Phase 4 LLM re-ranking on every request. Startup now logs the resolved generation model.
+- **LLM call timeouts** - Phase 0 keyword extraction and Phase 4 re-ranking are wrapped in a 12s `asyncio.wait_for` with graceful fallback, so a hung Ollama call can no longer gate retrieval for the full 120s client timeout.
+- **`/stats` wiki page count** - The endpoint passed the bare user name as path prefix (matching nothing) and always reported 0 pages; it now counts pages under `users/{user}`.
+- **Wiki.js page listing** - `list_pages` applied the API-side `limit` before client-side path/tag filters, dropping matching pages that sort late; the limit now applies after filtering. `list_all_pages` replaced its fake pagination loop with a real limit-growth loop (Wiki.js 2.x `pages.list` has no offset argument) that fetches until the API returns fewer pages than requested.
+- **Consolidation loop silently drained its queue on LLM failure** - Root cause of the 30-minute knowledge-consolidation loop processing 0 searches ("No unprocessed searches found" in prod): the `OLLAMA_MODEL` env collision (see below) made every consolidation `/api/generate` call fail with HTTP 400 (`"nomic-embed-text" does not support generate` — confirmed in prod logs and by direct Ollama probe), classification returned empty, and the loop STILL marked every SearchQuery `processed: true` — permanently consuming the queue with zero pages ever created (live Neo4j: 197/200 SearchQuery nodes processed with no output). LLM-infrastructure failure now raises `ConsolidationLLMUnavailableError`: the affected searches stay unprocessed (retried next run), the batch aborts after the first failure, and the response reports `searches_deferred`. Every run now logs `searches_processed` and `duration_ms` (also new response fields). The lookback boundary is now timezone-aware UTC. Regression tests added.
+- **Wiki.js `update_page` without tags** - Wiki.js 2.x requires `tags` on the update mutation (the server unconditionally maps over it); every `update_page(page_id, content=...)` call without tags failed with `Cannot read properties of undefined (reading 'map')` — this silently broke the consolidation service's page-update path too. The client now preserves the page's current tags when the caller does not supply any.
+- **Wiki.js listing completeness under pre-filter limits** - Observed live: `pages.list(limit=100)` returned 43 pages while 140 existed (`limit=500` returned all) — Wiki.js applies the limit BEFORE its own visibility filtering, so "fewer pages than requested" does not mean the listing is complete and the limit-growth loop stopped early, silently truncating listings (page counts, cleanups, integrity scans). The loop now grows the limit until the returned count stops increasing (fixed point), at the cost of one confirming fetch.
+
+## [1.7.3] - 2026-01-07
+
+### Fixed
+
+- **Endpoint TTL defaults** - Updated all fetch endpoint defaults to match namespace TTLs (2x refresh interval)
+
+## [1.7.2] - 2026-01-07
+
+### Fixed
+
+- **Volatile TTL doubled** - TTL now 2x refresh interval to survive missed/delayed scheduler runs
+
+## [1.7.1] - 2026-01-07
+
+### Fixed
+
+- **CI workflow** - Updated Gitea Actions to trigger on tag push (matching core-api)
+
+## [1.7.0] - 2026-01-07
+
+### Added
+
+- **Combined Environment Endpoint** - `POST /volatile/fetch/environment/{city}`
+  - Fetches weather and air quality concurrently with `asyncio.gather()`
+  - Single geocode lookup shared between both API calls
+  - More efficient than calling weather and air_quality separately
+  - Reduces wall-clock time and eliminates redundant geocoding
+
+### Fixed
+
+- **Scheduler executor name** - Fixed `rest_api` → `rest_api_executor` in SchedulerTask model and register_volatile_fetch() to prevent "Executor module not found" errors
+
+## [1.6.2] - 2025-12-30
+
+### Added
+
+- **System Statistics Endpoint** - `GET /stats`
+  - Neo4j: node counts by type (Document, Entity, Collection, Search)
+  - Qdrant: collection counts, total vectors, per-collection breakdown
+  - Wiki.js: total page count
+  - Paperless: documents, tags, correspondents, document types
+
+- **Weather/Forecast Separation** - Split weather into two distinct namespaces
+  - `POST /volatile/fetch/weather/{city}` - Current conditions only (1hr TTL)
+  - `POST /volatile/fetch/forecast/{city}` - 7-day outlook (12hr TTL)
+  - Different update frequencies for efficient caching
+  - `FORECAST` namespace added to volatile namespaces
+
+### Changed
+
+- Weather namespace TTL changed from 30 minutes to 1 hour (current conditions)
+- Forecast data now stored separately with 12 hour TTL
+
 ## [1.6.1] - 2025-12-30
 
 ### Added

TODO.md

@@ -41,12 +41,3 @@ Check for duplicate or highly similar documents using vector similarity and grap
 3. Check graph relationships
 4. Return candidates with similarity scores
 
-## System Statistics
-
-#### `GET /stats`
-Get system statistics (wiki pages, neo4j nodes, qdrant vectors).
-
-**Implementation needed:**
-- Query Neo4j for node count
-- Query Qdrant for vector count
-- Query Wiki.js for page count

check_environment.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python3
+"""Quick script to check environmental data in Qdrant volatile cache."""
+
+import asyncio
+import os
+import time
+from dotenv import load_dotenv
+
+load_dotenv()
+
+async def main():
+    from qdrant_client import QdrantClient
+    from qdrant_client.models import Filter, FieldCondition, MatchValue
+    from src.config import get_settings
+
+    settings = get_settings()
+    qdrant = QdrantClient(url=settings.qdrant_url)
+
+    user = "jpmschweitzer"
+    collection = f"volatile_{user}"
+
+    print(f"\n{'='*60}")
+    print(f"Volatile Data in Qdrant ({collection})")
+    print(f"{'='*60}\n")
+
+    namespaces = ["weather", "air_quality", "forecast", "sun", "news"]
+
+    now_ms = int(time.time() * 1000)
+
+    for ns in namespaces:
+        try:
+            from qdrant_client.models import Range
+
+            # Filter by namespace AND not expired
+            results = qdrant.scroll(
+                collection_name=collection,
+                scroll_filter=Filter(
+                    must=[
+                        FieldCondition(key="namespace", match=MatchValue(value=ns)),
+                        FieldCondition(key="ttl_expiry", range=Range(gt=now_ms)),
+                    ]
+                ),
+                limit=10,
+                with_payload=True,
+                with_vectors=False,
+            )
+
+            points = results[0]
+            print(f"=== {ns.upper()} [namespace:{ns}] ({len(points)} records) ===")
+
+            if not points:
+                print("  (no data)")
+                print()
+                continue
+
+            for point in points:
+                payload = point.payload
+                raw_data = payload.get("raw_data", {})
+
+                if ns == "weather":
+                    print(f"  Temperature: {raw_data.get('temperature')}°C (feels like {raw_data.get('feels_like')}°C)")
+                    print(f"  Conditions: {raw_data.get('conditions')}")
+                    print(f"  Humidity: {raw_data.get('humidity')}%")
+                    print(f"  Wind: {raw_data.get('wind_speed')} km/h")
+                    print(f"  UV Index: {raw_data.get('uv_index')}")
+
+                elif ns == "air_quality":
+                    print(f"  European AQI: {raw_data.get('aqi_european')}")
+                    print(f"  US AQI: {raw_data.get('aqi_us')}")
+                    print(f"  PM2.5: {raw_data.get('pm2_5')} µg/m³")
+                    print(f"  PM10: {raw_data.get('pm10')} µg/m³")
+                    print(f"  Ozone: {raw_data.get('ozone')} µg/m³")
+                    print(f"  NO₂: {raw_data.get('nitrogen_dioxide')} µg/m³")
+
+                elif ns == "forecast":
+                    daily = raw_data.get("daily", [])
+                    for day in daily[:5]:
+                        print(f"  {day.get('day_name', 'N/A')[:3]}: {day.get('temp_low'):.0f}-{day.get('temp_high'):.0f}°C, {day.get('conditions')}")
+
+                elif ns == "sun":
+                    print(f"  Sunrise: {raw_data.get('sunrise')}")
+                    print(f"  Sunset: {raw_data.get('sunset')}")
+                    print(f"  Daylight: {raw_data.get('daylight_hours', 0):.1f} hours")
+
+                elif ns == "news":
+                    headlines = raw_data.get("headlines", [])
+                    print(f"  Category: {raw_data.get('category', 'general')}")
+                    print(f"  Headlines ({len(headlines)}):")
+                    for item in headlines[:5]:
+                        title = item.get("title", "")[:60]
+                        source = item.get("source", "")
+                        print(f"    - [{source}] {title}...")
+
+                # Show TTL info
+                ttl_expiry = payload.get("ttl_expiry")
+                if ttl_expiry:
+                    remaining = (ttl_expiry / 1000) - time.time()
+                    if remaining > 0:
+                        print(f"  TTL remaining: {int(remaining)}s ({int(remaining/60)} min)")
+                    else:
+                        print(f"  TTL: EXPIRED")
+
+            print()
+
+        except Exception as e:
+            print(f"  Error fetching {ns}: {e}")
+            print()
+
+if __name__ == "__main__":
+    asyncio.run(main())

docs/scheduler-tasks.md

@@ -0,0 +1,180 @@
+# Scheduler Task Definitions (Phase C deploy checklist)
+
+Production task payloads for the homelab's database-driven **Scheduler**
+service. These are **definitions only** — nothing in this repo registers
+them automatically. Register them as part of the deploy checklist, either
+via the Scheduler UI/API or with the helper script:
+
+```bash
+# Preview exactly what would be sent (default):
+SCHEDULER_URL=http://<scheduler-host>:8090 \
+    .venv/bin/python scripts/register_scheduler_tasks.py
+
+# Actually register/update the tasks (deploy checklist step):
+SCHEDULER_URL=http://<scheduler-host>:8090 \
+SCHEDULER_API_KEY=<scheduler-api-key> \
+    .venv/bin/python scripts/register_scheduler_tasks.py --execute
+```
+
+Conventions:
+
+- The Scheduler's task-management endpoints (`GET`/`POST /tasks`,
+  `PUT /tasks/{name}`) require `Authorization: Bearer $SCHEDULER_API_KEY`.
+  The registrar reads `SCHEDULER_API_KEY` from the environment for its
+  own HTTP calls (`--execute` refuses to run without it); the key is
+  never stored. This is separate from `LIBRARY_API_KEY` below, which the
+  Scheduler container needs at task **execution** time.
+
+- All tasks call the **production** library-desk container
+  (`http://library-desk:8089`) with the explicit production tenant
+  `user=jpmschweitzer` (there is no default tenant — Phase B).
+- `${LIBRARY_API_KEY}` is a literal placeholder stored in the task's
+  `auth.token` field. The Scheduler's `rest_api_executor` substitutes
+  `${ENV_VAR}` placeholders from **its own environment at execution
+  time** (it substitutes `url`/`payload`/`auth` — NOT plain `headers`),
+  so the raw key is never stored in the `scheduled_tasks.config` JSONB
+  column. The **Scheduler container** must have `LIBRARY_API_KEY` in its
+  environment. Never commit or register the real value.
+- The JSON body goes in `config.payload` (the executor ignores a `body`
+  key).
+- Schedule fields use the Scheduler's convention: `-1` = every,
+  `day_of_week`: `0` = Monday … `6` = Sunday.
+
+---
+
+## 1. Nightly integrity check — 04:30 daily
+
+Read-only report: pages without vectors, orphaned vectors, unexpected
+Qdrant collections, Document nodes without wiki pages. Caches its result
+in Redis for the weekly quality report.
+
+```json
+{
+  "task_name": "library_integrity_check",
+  "service": "library-desk",
+  "executor": "rest_api_executor",
+  "priority": 60,
+  "description": "Nightly read-only integrity check for the library (vectors/graph/wiki/collections)",
+  "enabled": true,
+  "max_retries": 2,
+  "timeout_seconds": 900,
+  "minute": 30,
+  "hour": 4,
+  "day_of_month": -1,
+  "month": -1,
+  "day_of_week": -1,
+  "config": {
+    "method": "POST",
+    "url": "http://library-desk:8089/maintenance/integrity-check",
+    "headers": {
+      "Content-Type": "application/json"
+    },
+    "payload": {
+      "user": "jpmschweitzer"
+    },
+    "auth": {
+      "type": "bearer",
+      "token": "${LIBRARY_API_KEY}"
+    }
+  }
+}
+```
+
+## 2. Weekly quality report — Sunday 03:00
+
+Runs the duplicate scan, flags stale/metadata-poor pages, folds in the
+latest integrity results, and writes the dated report page to
+`users/jpmschweitzer/system/quality-reports/YYYY-MM-DD`.
+
+```json
+{
+  "task_name": "library_quality_report",
+  "service": "library-desk",
+  "executor": "rest_api_executor",
+  "priority": 60,
+  "description": "Weekly library quality report (dedup, stale pages, missing metadata, integrity) written to the wiki",
+  "enabled": true,
+  "max_retries": 2,
+  "timeout_seconds": 1800,
+  "minute": 0,
+  "hour": 3,
+  "day_of_month": -1,
+  "month": -1,
+  "day_of_week": 6,
+  "config": {
+    "method": "POST",
+    "url": "http://library-desk:8089/maintenance/quality-report",
+    "headers": {
+      "Content-Type": "application/json"
+    },
+    "payload": {
+      "user": "jpmschweitzer",
+      "stale_days": 30,
+      "dedup_threshold": 0.9,
+      "write_page": true
+    },
+    "auth": {
+      "type": "bearer",
+      "token": "${LIBRARY_API_KEY}"
+    }
+  }
+}
+```
+
+## 3. Daily Paperless orphan cleanup — 05:00
+
+Hits the **existing** cleanup endpoint (query parameters, empty payload).
+`dry_run=false` deletes vectors/graph nodes for documents that were
+removed from Paperless-ngx.
+
+```json
+{
+  "task_name": "library_paperless_orphan_cleanup",
+  "service": "library-desk",
+  "executor": "rest_api_executor",
+  "priority": 60,
+  "description": "Daily cleanup of vectors/graph nodes for documents deleted from Paperless-ngx",
+  "enabled": true,
+  "max_retries": 2,
+  "timeout_seconds": 900,
+  "minute": 0,
+  "hour": 5,
+  "day_of_month": -1,
+  "month": -1,
+  "day_of_week": -1,
+  "config": {
+    "method": "POST",
+    "url": "http://library-desk:8089/maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false",
+    "payload": {},
+    "auth": {
+      "type": "bearer",
+      "token": "${LIBRARY_API_KEY}"
+    }
+  }
+}
+```
+
+## 4. Disable `test_example_task`
+
+Not a new task: the leftover example task must be **disabled** (not
+deleted, so its history is preserved).
+
+```
+PUT ${SCHEDULER_URL}/tasks/test_example_task
+Content-Type: application/json
+
+{"enabled": false}
+```
+
+---
+
+## Related (already registered / in-process)
+
+- `knowledge_consolidation` — every 30 minutes, POST
+  `/consolidate/knowledge` (already registered; after the Phase C
+  consolidation repair its runs log `searches_processed` and
+  `duration_ms`, and searches are no longer consumed while the LLM is
+  unavailable).
+- Redis job-set cleanup — runs **in-process** inside library-desk
+  (hourly `job_cleanup_loop` started at app startup); no Scheduler task
+  needed.

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "library-desk"
-version = "1.6.1"
+version = "1.9.1"
 description = "Coordination service for The Library system - HybridRAG queries, document ingestion, entity extraction, and knowledge consolidation"
 readme = "README.md"
 requires-python = ">=3.12"

scripts/purge_test_artifacts.py

@@ -0,0 +1,279 @@
+#!/usr/bin/env python3
+"""
+Purge test-tenant residue from the SHARED production services.
+
+Targets ONLY the confirmed test residue left behind by earlier test runs:
+
+  Qdrant collections
+    - library_desk_llm_tester, memories_llm_tester, volatile_llm_tester
+    - test_user, library_desk_test_user
+    - core_ai_user_test_*  (prefix)
+    - anything containing "llm_tester" / "llm-tester"
+
+  Neo4j
+    - all nodes carrying a label starting with "User_Llm_Tester"
+      (covers User_Llm_Tester, User_Llm_Tester_Document,
+       User_Llm_Tester_SearchQuery, User_Llm_Tester_WebResult and
+       sub-tenants like User_Llm_Tester_Void_*)
+    - legacy "llm-tester" Document nodes matched by path
+      (d.path STARTS WITH 'users/llm')
+
+  Redis (service DB from settings, default DB 4)
+    - keys matching *llm_tester* / *llm-tester*
+
+SAFETY
+======
+- DRY-RUN IS THE DEFAULT. Nothing is deleted unless --execute is passed.
+- The script REFUSES to touch anything namespaced to the production
+  tenant "jpmschweitzer": every candidate identifier is checked and the
+  script aborts (exit 2) if a production-namespaced identifier ever
+  matches a target rule.
+- Connection settings (hosts, credentials) come from the repo .env via
+  src.config.Settings; nothing is printed except identifiers and counts.
+
+SNAPSHOT PREREQUISITE (before any --execute run)
+================================================
+Take snapshots of both stores first so an erroneous deletion can be
+rolled back:
+
+  Qdrant  - full-storage snapshot via the snapshot API:
+              curl -X POST http://<qdrant-host>:6333/snapshots
+            (or per collection:
+              curl -X POST http://<qdrant-host>:6333/collections/<name>/snapshots)
+
+  Neo4j   - offline dump from inside the container:
+              docker exec <neo4j> neo4j-admin database dump neo4j \
+                  --to-path=/backups
+
+Only proceed with --execute after both snapshots completed successfully.
+
+USAGE
+=====
+  .venv/bin/python scripts/purge_test_artifacts.py            # dry run (default)
+  .venv/bin/python scripts/purge_test_artifacts.py --dry-run  # explicit dry run
+  .venv/bin/python scripts/purge_test_artifacts.py --execute  # REALLY delete
+"""
+
+import argparse
+import asyncio
+import sys
+from pathlib import Path
+
+# Allow running from the repo root or the scripts/ directory
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+PRODUCTION_TENANT = "jpmschweitzer"
+
+# Confirmed residue: exact Qdrant collection names
+QDRANT_EXACT_TARGETS = {
+    "library_desk_llm_tester",
+    "memories_llm_tester",
+    "volatile_llm_tester",
+    "test_user",
+    "library_desk_test_user",
+}
+# Confirmed residue: Qdrant collection name prefixes
+QDRANT_PREFIX_TARGETS = ("core_ai_user_test_",)
+# Reserved test-tenant substrings (any collection containing these is residue)
+QDRANT_SUBSTRING_TARGETS = ("llm_tester", "llm-tester")
+
+# Neo4j: tenant label prefix for the reserved test tenant
+NEO4J_TEST_LABEL_PREFIX = "User_Llm_Tester"
+# Neo4j: legacy Document nodes matched by wiki path (llm-tester / llm_tester)
+NEO4J_TEST_DOC_PATH_PREFIX = "users/llm"
+
+# Redis key patterns for the reserved test tenant
+REDIS_PATTERNS = ("*llm_tester*", "*llm-tester*")
+
+
+def guard_not_production(identifier: str) -> str:
+    """Abort the whole run if a production-namespaced identifier shows up."""
+    if PRODUCTION_TENANT.lower() in identifier.lower():
+        print(
+            f"FATAL: target rule matched production-namespaced identifier "
+            f"{identifier!r} - aborting without deleting anything.",
+            file=sys.stderr,
+        )
+        sys.exit(2)
+    return identifier
+
+
+def qdrant_is_target(name: str) -> bool:
+    if name in QDRANT_EXACT_TARGETS:
+        return True
+    if any(name.startswith(p) for p in QDRANT_PREFIX_TARGETS):
+        return True
+    if any(sub in name for sub in QDRANT_SUBSTRING_TARGETS):
+        return True
+    return False
+
+
+def purge_qdrant(settings, execute: bool) -> int:
+    from qdrant_client import QdrantClient
+
+    client = QdrantClient(url=settings.qdrant_url, timeout=15)
+    try:
+        collections = [c.name for c in client.get_collections().collections]
+        targets = []
+        for name in collections:
+            if qdrant_is_target(name):
+                guard_not_production(name)
+                targets.append(name)
+
+        print(f"\nQdrant ({settings.qdrant_url}): {len(targets)} target collection(s)")
+        for name in sorted(targets):
+            try:
+                points = client.get_collection(name).points_count or 0
+            except Exception:
+                points = "?"
+            print(f"  - {name} ({points} points)")
+            if execute:
+                client.delete_collection(name)
+                print(f"    DELETED {name}")
+        return len(targets)
+    finally:
+        client.close()
+
+
+async def purge_neo4j(settings, execute: bool) -> int:
+    from src.clients.neo4j_client import Neo4jClient
+
+    guard_not_production(NEO4J_TEST_LABEL_PREFIX)
+    guard_not_production(NEO4J_TEST_DOC_PATH_PREFIX)
+
+    client = Neo4jClient(
+        uri=settings.neo4j_uri,
+        user=settings.neo4j_user,
+        password=settings.neo4j_password,
+    )
+    try:
+        await client.connect()
+
+        label_count_q = """
+        MATCH (n)
+        WHERE any(l IN labels(n) WHERE l STARTS WITH $prefix)
+        RETURN count(n) AS c
+        """
+        doc_count_q = """
+        MATCH (d:Document)
+        WHERE d.path STARTS WITH $path_prefix
+          AND NOT any(l IN labels(d) WHERE l STARTS WITH $prefix)
+        RETURN count(d) AS c
+        """
+        params = {
+            "prefix": NEO4J_TEST_LABEL_PREFIX,
+            "path_prefix": NEO4J_TEST_DOC_PATH_PREFIX,
+        }
+
+        labelled = (await client.execute_read(label_count_q, params))[0]["c"]
+        legacy_docs = (await client.execute_read(doc_count_q, params))[0]["c"]
+
+        print(f"\nNeo4j ({settings.neo4j_uri}):")
+        print(f"  - {labelled} node(s) with label prefix {NEO4J_TEST_LABEL_PREFIX}*")
+        print(
+            f"  - {legacy_docs} legacy Document node(s) with path prefix "
+            f"'{NEO4J_TEST_DOC_PATH_PREFIX}' (no tenant label)"
+        )
+
+        if execute:
+            r1 = await client.execute_write(
+                """
+                MATCH (n)
+                WHERE any(l IN labels(n) WHERE l STARTS WITH $prefix)
+                DETACH DELETE n
+                RETURN count(n) AS c
+                """,
+                params,
+            )
+            r2 = await client.execute_write(
+                """
+                MATCH (d:Document)
+                WHERE d.path STARTS WITH $path_prefix
+                  AND NOT any(l IN labels(d) WHERE l STARTS WITH $prefix)
+                DETACH DELETE d
+                RETURN count(d) AS c
+                """,
+                params,
+            )
+            print(f"    DELETED {r1[0]['c']} labelled + {r2[0]['c']} legacy nodes")
+
+        return labelled + legacy_docs
+    finally:
+        await client.close()
+
+
+async def purge_redis(settings, execute: bool) -> int:
+    import redis.asyncio as aioredis
+
+    client = aioredis.from_url(
+        settings.redis_url, encoding="utf-8", decode_responses=True
+    )
+    try:
+        keys: set[str] = set()
+        for pattern in REDIS_PATTERNS:
+            async for key in client.scan_iter(match=pattern, count=500):
+                guard_not_production(key)
+                keys.add(key)
+
+        print(f"\nRedis ({settings.redis_url}): {len(keys)} target key(s)")
+        for key in sorted(keys):
+            print(f"  - {key}")
+            if execute:
+                await client.delete(key)
+                print(f"    DELETED {key}")
+        return len(keys)
+    finally:
+        await client.aclose()
+
+
+async def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=(
+            "Purge confirmed test-tenant residue from the shared Qdrant, "
+            "Neo4j, and Redis stores. DRY-RUN by default; refuses anything "
+            "namespaced to the production tenant."
+        ),
+        epilog="Read the module docstring for the snapshot prerequisite.",
+    )
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument(
+        "--dry-run",
+        action="store_true",
+        default=True,
+        help="List targets and counts without deleting (DEFAULT behaviour)",
+    )
+    group.add_argument(
+        "--execute",
+        action="store_true",
+        help=(
+            "REALLY delete the listed targets. Take Qdrant + Neo4j snapshots "
+            "first (see module docstring)."
+        ),
+    )
+    args = parser.parse_args()
+    execute = bool(args.execute)
+
+    from src.config import get_settings
+
+    settings = get_settings()
+
+    mode = "EXECUTE (deleting!)" if execute else "DRY-RUN (nothing is deleted)"
+    print(f"purge_test_artifacts: mode = {mode}")
+    print(f"production tenant guard: refusing anything containing "
+          f"'{PRODUCTION_TENANT}'")
+
+    totals = {}
+    totals["qdrant_collections"] = purge_qdrant(settings, execute)
+    totals["neo4j_nodes"] = await purge_neo4j(settings, execute)
+    totals["redis_keys"] = await purge_redis(settings, execute)
+
+    print("\n=== Summary ===")
+    for target, count in totals.items():
+        print(f"  {target}: {count}")
+    if not execute:
+        print("\nDry run only. Re-run with --execute (after snapshots) to delete.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(asyncio.run(main()))

scripts/register_scheduler_tasks.py

@@ -0,0 +1,250 @@
+#!/usr/bin/env python3
+"""
+Register the Phase C production Scheduler tasks (deploy-checklist helper).
+
+Defines the four task payloads from docs/scheduler-tasks.md:
+1. library_integrity_check          - nightly 04:30
+2. library_quality_report           - Sunday 03:00
+3. library_paperless_orphan_cleanup - daily 05:00
+4. test_example_task                - DISABLED (update, not create)
+
+SAFETY MODEL
+============
+- DRY-RUN BY DEFAULT: without --execute the script only prints the exact
+  payloads it would send. Nothing is contacted except (optionally) the
+  Scheduler health endpoint.
+- --execute performs the registration: create task if absent, update it
+  if present, and disable test_example_task.
+- The Scheduler API location comes from the environment (SCHEDULER_URL);
+  there is no hardcoded production default.
+- The Scheduler's task-management endpoints are themselves guarded by
+  Bearer auth (verify_api_key). --execute therefore requires
+  SCHEDULER_API_KEY in the environment; the registrar sends it as
+  ``Authorization: Bearer <key>`` on its own HTTP calls. It is read from
+  the environment only and never stored anywhere.
+- NO SECRET IS EVER STORED: the library-desk API key is referenced as the
+  literal placeholder ``${LIBRARY_API_KEY}`` inside the task's
+  ``auth.token`` field. The Scheduler's rest_api_executor substitutes
+  ``${ENV_VAR}`` placeholders from ITS OWN environment at execution time
+  (it substitutes url/payload/auth — NOT plain headers), so the raw token
+  never lands in the scheduled_tasks.config JSONB column. The Scheduler
+  container must therefore have LIBRARY_API_KEY in its environment.
+
+Usage:
+    # Preview (default)
+    SCHEDULER_URL=http://scheduler-host:8090 \
+        python scripts/register_scheduler_tasks.py
+
+    # Register for real (deploy checklist step)
+    SCHEDULER_URL=http://scheduler-host:8090 \
+    SCHEDULER_API_KEY=<scheduler-api-key> \
+        python scripts/register_scheduler_tasks.py --execute
+"""
+
+import argparse
+import json
+import os
+import sys
+
+import httpx
+
+API_KEY_PLACEHOLDER = "${LIBRARY_API_KEY}"
+PRODUCTION_TENANT = "jpmschweitzer"
+LIBRARY_BASE_URL = "http://library-desk:8089"
+
+#: Tasks to create-or-update (see docs/scheduler-tasks.md).
+TASKS = [
+    {
+        "task_name": "library_integrity_check",
+        "service": "library-desk",
+        "executor": "rest_api_executor",
+        "priority": 60,
+        "description": (
+            "Nightly read-only integrity check for the library "
+            "(vectors/graph/wiki/collections)"
+        ),
+        "enabled": True,
+        "max_retries": 2,
+        "timeout_seconds": 900,
+        "minute": 30,
+        "hour": 4,
+        "day_of_month": -1,
+        "month": -1,
+        "day_of_week": -1,
+        "config": {
+            "method": "POST",
+            "url": f"{LIBRARY_BASE_URL}/maintenance/integrity-check",
+            "headers": {"Content-Type": "application/json"},
+            # rest_api_executor sends config["payload"] as the JSON body and
+            # substitutes ${ENV_VAR} in auth.token from the Scheduler's env.
+            "payload": {"user": PRODUCTION_TENANT},
+            "auth": {"type": "bearer", "token": API_KEY_PLACEHOLDER},
+        },
+    },
+    {
+        "task_name": "library_quality_report",
+        "service": "library-desk",
+        "executor": "rest_api_executor",
+        "priority": 60,
+        "description": (
+            "Weekly library quality report (dedup, stale pages, missing "
+            "metadata, integrity) written to the wiki"
+        ),
+        "enabled": True,
+        "max_retries": 2,
+        "timeout_seconds": 1800,
+        "minute": 0,
+        "hour": 3,
+        "day_of_month": -1,
+        "month": -1,
+        "day_of_week": 6,  # Sunday (0 = Monday)
+        "config": {
+            "method": "POST",
+            "url": f"{LIBRARY_BASE_URL}/maintenance/quality-report",
+            "headers": {"Content-Type": "application/json"},
+            "payload": {
+                "user": PRODUCTION_TENANT,
+                "stale_days": 30,
+                "dedup_threshold": 0.9,
+                "write_page": True,
+            },
+            "auth": {"type": "bearer", "token": API_KEY_PLACEHOLDER},
+        },
+    },
+    {
+        "task_name": "library_paperless_orphan_cleanup",
+        "service": "library-desk",
+        "executor": "rest_api_executor",
+        "priority": 60,
+        "description": (
+            "Daily cleanup of vectors/graph nodes for documents deleted "
+            "from Paperless-ngx"
+        ),
+        "enabled": True,
+        "max_retries": 2,
+        "timeout_seconds": 900,
+        "minute": 0,
+        "hour": 5,
+        "day_of_month": -1,
+        "month": -1,
+        "day_of_week": -1,
+        "config": {
+            "method": "POST",
+            "url": (
+                f"{LIBRARY_BASE_URL}/maintenance/cleanup/paperless"
+                f"?user={PRODUCTION_TENANT}&dry_run=false"
+            ),
+            "payload": {},
+            "auth": {"type": "bearer", "token": API_KEY_PLACEHOLDER},
+        },
+    },
+]
+
+#: Existing tasks to update in place.
+TASK_UPDATES = [
+    {"task_name": "test_example_task", "updates": {"enabled": False}},
+]
+
+
+def dry_run(scheduler_url: str) -> None:
+    print("=" * 72)
+    print("DRY RUN - nothing will be sent. Re-run with --execute to register.")
+    print(f"Scheduler API: {scheduler_url or '(SCHEDULER_URL not set)'}")
+    print("=" * 72)
+    for task in TASKS:
+        print(f"\n--- create-or-update: POST {scheduler_url}/tasks "
+              f"(or PUT /tasks/{task['task_name']}) ---")
+        print(json.dumps(task, indent=2))
+    for update in TASK_UPDATES:
+        print(f"\n--- update: PUT {scheduler_url}/tasks/{update['task_name']} ---")
+        print(json.dumps(update["updates"], indent=2))
+    print("\nDry run complete: "
+          f"{len(TASKS)} task definition(s), {len(TASK_UPDATES)} update(s).")
+
+
+def execute(scheduler_url: str, scheduler_api_key: str) -> int:
+    failures = 0
+    # The Scheduler's task-management endpoints require Bearer auth
+    # (verify_api_key: 401 when missing, 403 when wrong). Without this
+    # header the existence probes 401 (misread as "task absent") and
+    # every POST/PUT fails.
+    auth_headers = {"Authorization": f"Bearer {scheduler_api_key}"}
+    with httpx.Client(
+        base_url=scheduler_url, timeout=30.0, headers=auth_headers
+    ) as client:
+        health = client.get("/health")
+        if health.status_code != 200:
+            print(f"ERROR: Scheduler health check failed: {health.status_code}")
+            return 1
+
+        for task in TASKS:
+            name = task["task_name"]
+            # Sent verbatim: the ${LIBRARY_API_KEY} placeholder is resolved
+            # by the Scheduler at execution time, never stored as a raw key.
+            payload = task
+            exists = client.get(f"/tasks/{name}").status_code == 200
+            if exists:
+                resp = client.put(f"/tasks/{name}", json=payload)
+                action = "updated"
+            else:
+                resp = client.post("/tasks", json=payload)
+                action = "created"
+            if resp.status_code == 200:
+                print(f"[ok] {action} {name}")
+            else:
+                failures += 1
+                print(f"[FAIL] {action} {name}: {resp.status_code} {resp.text[:200]}")
+
+        for update in TASK_UPDATES:
+            name = update["task_name"]
+            if client.get(f"/tasks/{name}").status_code != 200:
+                print(f"[skip] {name} does not exist - nothing to disable")
+                continue
+            resp = client.put(f"/tasks/{name}", json=update["updates"])
+            if resp.status_code == 200:
+                print(f"[ok] updated {name}: {update['updates']}")
+            else:
+                failures += 1
+                print(f"[FAIL] update {name}: {resp.status_code} {resp.text[:200]}")
+
+    return 1 if failures else 0
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description="Register library-desk production Scheduler tasks "
+                    "(dry-run by default)"
+    )
+    parser.add_argument(
+        "--execute",
+        action="store_true",
+        help="Actually register/update the tasks (default: dry-run print only)",
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Explicitly print payloads without sending (the default behavior)",
+    )
+    args = parser.parse_args()
+
+    scheduler_url = os.environ.get("SCHEDULER_URL", "").rstrip("/")
+
+    if not args.execute or args.dry_run:
+        dry_run(scheduler_url)
+        return 0
+
+    if not scheduler_url:
+        print("ERROR: SCHEDULER_URL must be set for --execute")
+        return 1
+
+    scheduler_api_key = os.environ.get("SCHEDULER_API_KEY", "")
+    if not scheduler_api_key:
+        print("ERROR: SCHEDULER_API_KEY must be set for --execute "
+              "(the Scheduler's task endpoints require Bearer auth)")
+        return 1
+
+    return execute(scheduler_url, scheduler_api_key)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

src/clients/content_extractor.py

@@ -5,6 +5,20 @@ A reusable Trafilatura wrapper that can be used throughout library-desk:
 - RAG search service (extract content from search results)
 - Ingestion service (extract content from URLs)
 - Standalone endpoint (ad-hoc content extraction)
+
+Hardening notes:
+- Pages are fetched with httpx.AsyncClient under real connect/read timeouts
+  on the event loop. Only the CPU-bound Trafilatura parse runs in the thread
+  pool, so a slow server can no longer pin a worker thread for the duration
+  of a blind blocking download (trafilatura.fetch_url had no caller-side
+  timeout control and kept downloading after asyncio.wait_for gave up).
+- Trafilatura runs ONCE per document (bare_extraction returns text and
+  metadata together); the old code ran extract() twice (the XML pass was
+  computed and thrown away) plus bare_extraction — three full parses.
+- extract_batch caps the number of full-page extractions per call; overflow
+  URLs are returned as unsuccessful results so callers fall back to the
+  search-engine snippet.
+- Thread-pool queue depth is logged so extraction backpressure is visible.
 """
 
 import asyncio
@@ -12,80 +26,156 @@ import logging
 from concurrent.futures import ThreadPoolExecutor
 from typing import List, Optional
 
+import httpx
 import trafilatura
 
 from src.models.content import ContentExtractionResult
 
 logger = logging.getLogger(__name__)
 
+# Modest but honest identification; some sites reject empty user agents.
+DEFAULT_HEADERS = {
+    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) LibraryDesk-ContentExtractor"
+}
+
+# Downloads are streamed and ABORTED past this many bytes (protects memory
+# and bandwidth during the fetch, and keeps Trafilatura from pinning a
+# worker thread on a multi-hundred-MB response).
+MAX_RESPONSE_BYTES = 5 * 1024 * 1024
+
 
 class ContentExtractor:
     """
     Generic content extraction client using Trafilatura.
 
-    Provides async wrappers around Trafilatura's synchronous extraction,
-    with support for parallel batch processing and configurable timeouts.
+    Fetches pages asynchronously via httpx and runs Trafilatura's
+    synchronous extraction in a thread pool, with support for parallel
+    batch processing and configurable timeouts.
     """
 
+    # Hard cap on full-page extractions per extract_batch call (e.g. one
+    # web-search leg). Overflow URLs get an unsuccessful result and callers
+    # fall back to the search snippet.
+    DEFAULT_MAX_URLS_PER_BATCH = 8
+
     def __init__(
         self,
         timeout: int = 5,
         max_length: int = 2000,
-        max_workers: int = 10
+        max_workers: int = 10,
+        connect_timeout: float = 3.0,
+        max_urls_per_batch: Optional[int] = None
     ):
         """
         Initialize ContentExtractor.
 
         Args:
-            timeout: Per-URL timeout in seconds
+            timeout: Per-URL read/parse timeout in seconds
             max_length: Maximum content length to return (truncated if longer)
             max_workers: Max concurrent extractions for batch operations
+            connect_timeout: TCP/TLS connect timeout in seconds
+            max_urls_per_batch: Cap on full-page extractions per
+                extract_batch call (None = DEFAULT_MAX_URLS_PER_BATCH)
         """
         self.timeout = timeout
         self.max_length = max_length
+        self.max_urls_per_batch = (
+            max_urls_per_batch
+            if max_urls_per_batch is not None
+            else self.DEFAULT_MAX_URLS_PER_BATCH
+        )
         self._executor = ThreadPoolExecutor(max_workers=max_workers)
+        self._http = httpx.AsyncClient(
+            timeout=httpx.Timeout(timeout, connect=connect_timeout),
+            follow_redirects=True,
+            headers=DEFAULT_HEADERS,
+            limits=httpx.Limits(
+                max_connections=max_workers,
+                max_keepalive_connections=max_workers
+            ),
+        )
         logger.info(
             f"Initialized ContentExtractor: timeout={timeout}s, "
-            f"max_length={max_length}, max_workers={max_workers}"
+            f"connect_timeout={connect_timeout}s, max_length={max_length}, "
+            f"max_workers={max_workers}, max_urls_per_batch={self.max_urls_per_batch}"
         )
 
-    def _extract_sync(
-        self,
-        url: str,
-        include_metadata: bool = True,
-        max_length: Optional[int] = None
-    ) -> ContentExtractionResult:
+    async def _fetch(self, url: str) -> Optional[str]:
         """
-        Synchronous extraction (runs in thread pool).
+        Fetch a URL asynchronously under real connect/read timeouts.
 
-        Args:
-            url: URL to extract content from
-            include_metadata: Whether to extract title, author, date
-            max_length: Override default max length
+        The body is STREAMED and the download is aborted as soon as
+        MAX_RESPONSE_BYTES have been received, so the cap bounds memory and
+        bandwidth during the download itself (the old implementation
+        buffered the entire response before truncating, so a
+        multi-hundred-MB URL was still fully downloaded).
 
         Returns:
-            ContentExtractionResult with extracted content or error
+            Response text (capped at MAX_RESPONSE_BYTES) or None when the
+            response is empty / not OK.
+
+        Raises:
+            httpx.HTTPError subclasses on timeout/network errors.
         """
-        effective_max_length = max_length or self.max_length
-
-        try:
-            # Fetch the URL
-            downloaded = trafilatura.fetch_url(url)
-            if not downloaded:
-                return ContentExtractionResult(
-                    url=url,
-                    content="",
-                    success=False,
-                    error="Failed to fetch URL"
+        async with self._http.stream("GET", url) as response:
+            if response.status_code != 200:
+                logger.debug(
+                    f"Fetch returned status {response.status_code} for {url}"
                 )
+                return None
 
-            # Extract content
-            content = trafilatura.extract(
-                downloaded,
+            chunks: List[bytes] = []
+            received = 0
+            truncated = False
+            async for chunk in response.aiter_bytes():
+                if received + len(chunk) >= MAX_RESPONSE_BYTES:
+                    chunks.append(chunk[: MAX_RESPONSE_BYTES - received])
+                    truncated = True
+                    break
+                chunks.append(chunk)
+                received += len(chunk)
+
+            body = b"".join(chunks)
+            if not body:
+                return None
+            if truncated:
+                logger.warning(
+                    f"Response for {url} exceeds {MAX_RESPONSE_BYTES} bytes; "
+                    "download aborted and body truncated"
+                )
+            # charset comes from the Content-Type header, available before
+            # the body is read.
+            encoding = response.charset_encoding or "utf-8"
+            return body.decode(encoding, errors="replace")
+
+    def _log_queue_depth(self, context: str) -> None:
+        """Log thread-pool queue depth so extraction backpressure is visible."""
+        depth = self._executor._work_queue.qsize()
+        if depth > 0:
+            logger.info(f"ContentExtractor thread-pool queue depth ({context}): {depth}")
+
+    @staticmethod
+    def _extract_html_sync(
+        html: str,
+        url: str,
+        include_metadata: bool,
+        max_length: int
+    ) -> ContentExtractionResult:
+        """
+        Synchronous Trafilatura pass (runs in the thread pool).
+
+        Runs bare_extraction ONCE — it returns text and metadata together
+        (the old implementation parsed the document three times).
+        """
+        try:
+            doc = trafilatura.bare_extraction(
+                html,
+                url=url or None,
                 include_comments=False,
                 include_tables=True,
-                output_format='txt'
+                with_metadata=include_metadata
             )
+            content = (doc or {}).get("text") or ""
 
             if not content:
                 return ContentExtractionResult(
@@ -96,44 +186,16 @@ class ContentExtractor:
                 )
 
             # Truncate if needed
-            if len(content) > effective_max_length:
-                content = content[:effective_max_length] + "..."
-
-            # Extract metadata if requested
-            title = None
-            author = None
-            date = None
-            language = None
-
-            if include_metadata:
-                metadata = trafilatura.extract(
-                    downloaded,
-                    output_format='xml',
-                    include_comments=False
-                )
-                # Parse metadata from XML if available
-                # trafilatura.extract with output_format='xml' returns XML with metadata
-                # For simplicity, we'll use bare_extraction which returns a dict
-                try:
-                    meta_dict = trafilatura.bare_extraction(
-                        downloaded,
-                        include_comments=False
-                    )
-                    if meta_dict:
-                        title = meta_dict.get('title')
-                        author = meta_dict.get('author')
-                        date = meta_dict.get('date')
-                        language = meta_dict.get('language')
-                except Exception as e:
-                    logger.debug(f"Metadata extraction failed for {url}: {e}")
+            if len(content) > max_length:
+                content = content[:max_length] + "..."
 
             return ContentExtractionResult(
                 url=url,
-                title=title,
+                title=doc.get("title") if include_metadata else None,
                 content=content,
-                author=author,
-                date=date,
-                language=language,
+                author=doc.get("author") if include_metadata else None,
+                date=doc.get("date") if include_metadata else None,
+                language=doc.get("language") if include_metadata else None,
                 success=True,
                 error=None
             )
@@ -156,6 +218,9 @@ class ContentExtractor:
         """
         Extract content from a single URL asynchronously.
 
+        The download happens on the event loop under httpx connect/read
+        timeouts; only the parse occupies a worker thread.
+
         Args:
             url: URL to extract content from
             include_metadata: Whether to extract title, author, date
@@ -164,60 +229,84 @@ class ContentExtractor:
         Returns:
             ContentExtractionResult with extracted content or error
         """
-        loop = asyncio.get_event_loop()
-
         try:
-            result = await asyncio.wait_for(
-                loop.run_in_executor(
-                    self._executor,
-                    self._extract_sync,
-                    url,
-                    include_metadata,
-                    max_length
-                ),
-                timeout=self.timeout
-            )
-            return result
-        except asyncio.TimeoutError:
-            logger.warning(f"Content extraction timed out for {url}")
+            html = await self._fetch(url)
+        except httpx.TimeoutException:
+            logger.warning(f"Fetch timed out for {url}")
             return ContentExtractionResult(
                 url=url,
                 content="",
                 success=False,
-                error=f"Extraction timed out after {self.timeout}s"
+                error=f"Fetch timed out after {self.timeout}s"
             )
-        except Exception as e:
-            logger.error(f"Unexpected error extracting {url}: {e}")
+        except httpx.HTTPError as e:
+            logger.warning(f"Fetch failed for {url}: {e}")
             return ContentExtractionResult(
                 url=url,
                 content="",
                 success=False,
-                error=str(e)
+                error=f"Failed to fetch URL: {e}"
             )
 
+        if not html:
+            return ContentExtractionResult(
+                url=url,
+                content="",
+                success=False,
+                error="Failed to fetch URL"
+            )
+
+        return await self.extract_from_html(html, url, include_metadata, max_length)
+
     async def extract_batch(
         self,
         urls: List[str],
         include_metadata: bool = True,
-        max_length: Optional[int] = None
+        max_length: Optional[int] = None,
+        max_urls: Optional[int] = None
     ) -> List[ContentExtractionResult]:
         """
         Extract content from multiple URLs in parallel.
 
+        At most max_urls (default: max_urls_per_batch) URLs get a full-page
+        extraction; the rest are returned unsuccessful so callers fall back
+        to their existing snippet.
+
         Args:
             urls: List of URLs to extract content from
             include_metadata: Whether to extract title, author, date
             max_length: Override default max length
+            max_urls: Override the per-call full-page extraction cap
 
         Returns:
             List of ContentExtractionResult in same order as input URLs
         """
+        cap = max_urls if max_urls is not None else self.max_urls_per_batch
+        fetch_urls = urls[:cap]
+        skipped_urls = urls[cap:]
+        if skipped_urls:
+            logger.info(
+                f"extract_batch capped at {cap} full-page extractions; "
+                f"skipping {len(skipped_urls)} of {len(urls)} URLs"
+            )
+        self._log_queue_depth("extract_batch")
+
         tasks = [
             self.extract(url, include_metadata, max_length)
-            for url in urls
+            for url in fetch_urls
         ]
-        results = await asyncio.gather(*tasks)
-        return list(results)
+        results = list(await asyncio.gather(*tasks))
+
+        results.extend(
+            ContentExtractionResult(
+                url=url,
+                content="",
+                success=False,
+                error=f"Skipped: per-call extraction cap ({cap}) reached"
+            )
+            for url in skipped_urls
+        )
+        return results
 
     async def extract_from_html(
         self,
@@ -238,73 +327,40 @@ class ContentExtractor:
         Returns:
             ContentExtractionResult with extracted content or error
         """
-        effective_max_length = max_length or self.max_length
-
-        def _extract():
-            try:
-                content = trafilatura.extract(
-                    html,
-                    include_comments=False,
-                    include_tables=True,
-                    output_format='txt'
-                )
-
-                if not content:
-                    return ContentExtractionResult(
-                        url=url,
-                        content="",
-                        success=False,
-                        error="No content extracted from HTML"
-                    )
-
-                # Truncate if needed
-                if len(content) > effective_max_length:
-                    content = content[:effective_max_length] + "..."
-
-                # Extract metadata
-                title = None
-                author = None
-                date = None
-                language = None
-
-                if include_metadata:
-                    try:
-                        meta_dict = trafilatura.bare_extraction(
-                            html,
-                            include_comments=False
-                        )
-                        if meta_dict:
-                            title = meta_dict.get('title')
-                            author = meta_dict.get('author')
-                            date = meta_dict.get('date')
-                            language = meta_dict.get('language')
-                    except Exception as e:
-                        logger.debug(f"Metadata extraction failed: {e}")
-
-                return ContentExtractionResult(
-                    url=url,
-                    title=title,
-                    content=content,
-                    author=author,
-                    date=date,
-                    language=language,
-                    success=True,
-                    error=None
-                )
-
-            except Exception as e:
-                logger.error(f"HTML content extraction failed: {e}")
-                return ContentExtractionResult(
-                    url=url,
-                    content="",
-                    success=False,
-                    error=str(e)
-                )
-
         loop = asyncio.get_event_loop()
-        return await loop.run_in_executor(self._executor, _extract)
+        self._log_queue_depth("extract_from_html")
+
+        try:
+            return await asyncio.wait_for(
+                loop.run_in_executor(
+                    self._executor,
+                    self._extract_html_sync,
+                    html,
+                    url,
+                    include_metadata,
+                    max_length or self.max_length
+                ),
+                timeout=self.timeout
+            )
+        except asyncio.TimeoutError:
+            logger.warning(f"Content extraction timed out for {url or '<raw html>'}")
+            return ContentExtractionResult(
+                url=url,
+                content="",
+                success=False,
+                error=f"Extraction timed out after {self.timeout}s"
+            )
+        except Exception as e:
+            logger.error(f"Unexpected error extracting {url or '<raw html>'}: {e}")
+            return ContentExtractionResult(
+                url=url,
+                content="",
+                success=False,
+                error=str(e)
+            )
 
     async def close(self):
-        """Shutdown the thread pool executor."""
+        """Shutdown the HTTP client and the thread pool executor."""
+        await self._http.aclose()
         self._executor.shutdown(wait=False)
         logger.info("ContentExtractor closed")

src/clients/neo4j_client.py

@@ -8,7 +8,7 @@ Provides async Neo4j operations with:
 - Automatic retry on transient failures
 """
 
-from neo4j import AsyncGraphDatabase, AsyncDriver, AsyncSession
+from neo4j import AsyncGraphDatabase, AsyncDriver, AsyncSession, READ_ACCESS
 from typing import Optional, List, Dict, Any
 import logging
 
@@ -97,6 +97,38 @@ class Neo4jClient:
             records = await result.data()
             return records
 
+    async def execute_read(
+        self,
+        cypher: str,
+        parameters: Optional[Dict[str, Any]] = None
+    ) -> List[Dict[str, Any]]:
+        """
+        Execute Cypher in a READ-ONLY session.
+
+        The session is opened with default_access_mode=READ_ACCESS, so the
+        database rejects any write attempt (CREATE/MERGE/DELETE/SET/...)
+        even if it slips past caller-side validation. Use this for any
+        query built from untrusted input (e.g. the /query/graph endpoint).
+
+        Args:
+            cypher: Cypher query string
+            parameters: Query parameters
+
+        Returns:
+            List of result records as dictionaries
+
+        Raises:
+            Exception: If driver not initialized, query fails, or the
+                query attempts a write (rejected by the read session)
+        """
+        if not self._driver:
+            await self.connect()
+
+        async with self._driver.session(default_access_mode=READ_ACCESS) as session:
+            result = await session.run(cypher, parameters or {})
+            records = await result.data()
+            return records
+
     async def execute_write(
         self,
         cypher: str,

src/clients/ollama_client.py

@@ -33,6 +33,7 @@ class OllamaClient:
         self.base_url = base_url.rstrip("/")
         self.model = model
         self.embeddings_url = f"{self.base_url}/api/embeddings"
+        self.embed_url = f"{self.base_url}/api/embed"
         self.generate_url = f"{self.base_url}/api/generate"
         self.tags_url = f"{self.base_url}/api/tags"
         self.client = httpx.AsyncClient(timeout=120.0)  # Embeddings can be slow
@@ -106,8 +107,37 @@ class OllamaClient:
             >>> len(embeddings)
             3
         """
-        embeddings = []
+        if not texts:
+            return []
 
+        # Single batched request via Ollama's /api/embed (the old
+        # implementation looped one /api/embeddings call per text).
+        try:
+            response = await self.client.post(
+                self.embed_url,
+                json={"model": self.model, "input": texts}
+            )
+            response.raise_for_status()
+            data = response.json()
+            embeddings = data.get("embeddings")
+
+            if embeddings is not None and len(embeddings) == len(texts):
+                if show_progress:
+                    logger.info(f"Batched embedding complete: {len(embeddings)}/{len(texts)}")
+                return embeddings
+
+            logger.warning(
+                f"Batched embed returned {len(embeddings or [])} vectors for "
+                f"{len(texts)} inputs, falling back to per-text embedding"
+            )
+        except Exception as e:
+            logger.warning(
+                f"Batched embed failed ({e}), falling back to per-text embedding"
+            )
+
+        # Fallback: per-text embedding preserves partial-success semantics
+        # (None entries for texts that failed to embed).
+        embeddings = []
         for i, text in enumerate(texts):
             if show_progress and i % 10 == 0:
                 logger.info(f"Embedding progress: {i}/{len(texts)}")

src/clients/qdrant_client.py

@@ -8,7 +8,7 @@ Provides async vector operations with:
 - Similarity queries
 """
 
-from qdrant_client import QdrantClient
+from qdrant_client import AsyncQdrantClient
 from qdrant_client.models import (
     Distance, VectorParams, PointStruct,
     Filter, FieldCondition, MatchValue, Range
@@ -30,17 +30,22 @@ class QdrantClientWrapper:
     Each user has isolated vector collection for their documents.
     """
 
-    def __init__(self, url: str, embedding_dim: int = 768):
+    def __init__(self, url: str, embedding_dim: int = 768, timeout: float = 30.0):
         """
         Initialize Qdrant client.
 
+        Uses AsyncQdrantClient so vector calls never block the FastAPI
+        event loop, with an explicit timeout so a hung Qdrant cannot
+        stall requests indefinitely.
+
         Args:
             url: Qdrant server URL (e.g., "http://qdrant:6333")
             embedding_dim: Vector embedding dimension (default 768 for nomic-embed-text)
+            timeout: Per-request timeout in seconds (default 30.0)
         """
-        self.client = QdrantClient(url=url)
+        self.client = AsyncQdrantClient(url=url, timeout=timeout)
         self.embedding_dim = embedding_dim
-        logger.info(f"Initialized Qdrant client: {url}")
+        logger.info(f"Initialized async Qdrant client: {url} (timeout={timeout}s)")
 
     def get_collection_name(self, user: str) -> str:
         """
@@ -62,12 +67,12 @@ class QdrantClientWrapper:
             collection_name: Collection name
         """
         try:
-            collections = self.client.get_collections()
+            collections = await self.client.get_collections()
             existing = [c.name for c in collections.collections]
 
             if collection_name not in existing:
                 logger.info(f"Creating Qdrant collection: {collection_name}")
-                self.client.create_collection(
+                await self.client.create_collection(
                     collection_name=collection_name,
                     vectors_config=VectorParams(
                         size=self.embedding_dim,
@@ -90,7 +95,7 @@ class QdrantClientWrapper:
             True if collection exists
         """
         try:
-            collections = self.client.get_collections()
+            collections = await self.client.get_collections()
             existing = [c.name for c in collections.collections]
             return collection_name in existing
         except Exception as e:
@@ -137,7 +142,9 @@ class QdrantClientWrapper:
             )
 
         collection_name = self.get_collection_name(user)
-        await self.ensure_collection(user)
+        # Ensure the tenant-scoped collection (passing the raw user here used
+        # to create a stray collection named after the bare user string).
+        await self.ensure_collection(collection_name)
 
         points = []
         for i, (chunk, embedding) in enumerate(zip(chunks, embeddings)):
@@ -159,7 +166,7 @@ class QdrantClientWrapper:
             ))
 
         try:
-            self.client.upsert(
+            await self.client.upsert(
                 collection_name=collection_name,
                 points=points
             )
@@ -211,7 +218,7 @@ class QdrantClientWrapper:
             query_filter = Filter(must=conditions)
 
         try:
-            response = self.client.query_points(
+            response = await self.client.query_points(
                 collection_name=collection_name,
                 query=query_vector,
                 limit=limit,
@@ -249,7 +256,7 @@ class QdrantClientWrapper:
         collection_name = self.get_collection_name(user)
 
         try:
-            self.client.delete(
+            await self.client.delete(
                 collection_name=collection_name,
                 points_selector=Filter(
                     must=[
@@ -285,7 +292,7 @@ class QdrantClientWrapper:
 
         try:
             # Scroll through points with doc_id filter
-            points, _ = self.client.scroll(
+            points, _ = await self.client.scroll(
                 collection_name=collection_name,
                 scroll_filter=Filter(
                     must=[
@@ -327,7 +334,7 @@ class QdrantClientWrapper:
 
         try:
             # Get collection info
-            collection_info = self.client.get_collection(collection_name)
+            collection_info = await self.client.get_collection(collection_name)
             # This gives total points, not unique docs
             # For unique docs, would need to aggregate by doc_id
             return collection_info.points_count
@@ -350,7 +357,7 @@ class QdrantClientWrapper:
         collection_name = self.get_collection_name(user)
 
         try:
-            self.client.delete_collection(collection_name)
+            await self.client.delete_collection(collection_name)
             logger.warning(f"Deleted collection: {collection_name}")
             return True
         except Exception as e:
@@ -380,7 +387,7 @@ class QdrantClientWrapper:
 
         try:
             # Get source document chunks with vectors
-            source_points, _ = self.client.scroll(
+            source_points, _ = await self.client.scroll(
                 collection_name=collection_name,
                 scroll_filter=Filter(
                     must=[
@@ -402,7 +409,7 @@ class QdrantClientWrapper:
             # (could aggregate multiple chunks for better results)
             first_vector = source_points[0].vector
 
-            response = self.client.query_points(
+            response = await self.client.query_points(
                 collection_name=collection_name,
                 query=first_vector,
                 limit=limit * 2,  # Get more to filter out same doc
@@ -447,7 +454,7 @@ class QdrantClientWrapper:
             True if successful
         """
         try:
-            self.client.upsert(
+            await self.client.upsert(
                 collection_name=collection_name,
                 points=[PointStruct(
                     id=vector_id,
@@ -460,6 +467,43 @@ class QdrantClientWrapper:
             logger.error(f"Failed to upsert vector: {e}", exc_info=True)
             return False
 
+    async def upsert_points(
+        self,
+        collection_name: str,
+        points: List[Dict[str, Any]]
+    ) -> int:
+        """
+        Upsert a batch of vector points in a single request.
+
+        Args:
+            collection_name: Collection name
+            points: List of {"id": str, "vector": List[float], "payload": dict}
+
+        Returns:
+            Number of points upserted
+
+        Raises:
+            Exception: If the upsert fails (callers decide how to degrade)
+        """
+        if not points:
+            return 0
+
+        structs = [
+            PointStruct(
+                id=p["id"],
+                vector=p["vector"],
+                payload=p.get("payload", {})
+            )
+            for p in points
+        ]
+
+        await self.client.upsert(
+            collection_name=collection_name,
+            points=structs
+        )
+        logger.info(f"Upserted {len(structs)} points into {collection_name}")
+        return len(structs)
+
     async def delete_by_filter(
         self,
         collection_name: str,
@@ -485,7 +529,7 @@ class QdrantClientWrapper:
             query_filter = Filter(must=conditions)
 
             # Delete points
-            result = self.client.delete(
+            result = await self.client.delete(
                 collection_name=collection_name,
                 points_selector=query_filter
             )
@@ -530,7 +574,7 @@ class QdrantClientWrapper:
             query_filter = Filter(must=conditions)
 
         try:
-            response = self.client.query_points(
+            response = await self.client.query_points(
                 collection_name=collection_name,
                 query=query_vector,
                 limit=limit,
@@ -587,7 +631,7 @@ class QdrantClientWrapper:
 
         try:
             while True:
-                points, next_offset = self.client.scroll(
+                points, next_offset = await self.client.scroll(
                     collection_name=collection_name,
                     scroll_filter=scroll_filter,
                     limit=batch_size,
@@ -597,10 +641,13 @@ class QdrantClientWrapper:
                 )
 
                 for point in points:
-                    all_points.append({
+                    entry = {
                         "id": str(point.id),
                         "payload": dict(point.payload) if point.payload else {}
-                    })
+                    }
+                    if with_vectors:
+                        entry["vector"] = point.vector
+                    all_points.append(entry)
 
                 if next_offset is None:
                     break
@@ -631,7 +678,7 @@ class QdrantClientWrapper:
             return 0
 
         try:
-            self.client.delete(
+            await self.client.delete(
                 collection_name=collection_name,
                 points_selector=point_ids
             )
@@ -650,13 +697,13 @@ class QdrantClientWrapper:
             List of collection info dictionaries
         """
         try:
-            collections = self.client.get_collections()
+            collections = await self.client.get_collections()
             result = []
 
             for coll in collections.collections:
                 # Get detailed collection info
                 try:
-                    info = self.client.get_collection(coll.name)
+                    info = await self.client.get_collection(coll.name)
                     result.append({
                         "name": coll.name,
                         "vectors_count": info.vectors_count or 0,
@@ -712,7 +759,7 @@ class QdrantClientWrapper:
         )
 
         try:
-            response = self.client.query_points(
+            response = await self.client.query_points(
                 collection_name=collection_name,
                 query=query_vector,
                 limit=limit,
@@ -763,7 +810,7 @@ class QdrantClientWrapper:
             count = 0
             offset = None
             while True:
-                points, next_offset = self.client.scroll(
+                points, next_offset = await self.client.scroll(
                     collection_name=collection_name,
                     scroll_filter=expiry_filter,
                     limit=100,
@@ -779,7 +826,7 @@ class QdrantClientWrapper:
                 return 0
 
             # Delete expired points
-            self.client.delete(
+            await self.client.delete(
                 collection_name=collection_name,
                 points_selector=expiry_filter
             )
@@ -799,7 +846,7 @@ class QdrantClientWrapper:
             List of volatile collection names
         """
         try:
-            collections = self.client.get_collections()
+            collections = await self.client.get_collections()
             return [
                 c.name for c in collections.collections
                 if c.name.startswith("volatile_")

src/clients/scheduler_client.py

@@ -8,17 +8,24 @@ Registers and manages scheduled tasks for prefetch operations
 import httpx
 import logging
 from typing import Optional, Any
+from urllib.parse import quote
 from pydantic import BaseModel, Field
 
 logger = logging.getLogger(__name__)
 
+#: Literal placeholder stored in task auth.token; the Scheduler's
+#: rest_api_executor substitutes ${ENV_VAR} from ITS OWN environment at
+#: execution time, so the raw library-desk key is never stored in the
+#: scheduled_tasks.config column.
+LIBRARY_API_KEY_PLACEHOLDER = "${LIBRARY_API_KEY}"
+
 
 class SchedulerTask(BaseModel):
     """Task definition for scheduler registration."""
 
     task_name: str = Field(..., description="Unique task identifier")
     service: str = Field(default="library-desk", description="Service that owns this task")
-    executor: str = Field(default="rest_api", description="Executor type")
+    executor: str = Field(default="rest_api_executor", description="Executor type")
     priority: int = Field(default=50, ge=1, le=100, description="Priority (lower = higher)")
     description: Optional[str] = Field(None, description="Human-readable description")
     enabled: bool = Field(default=True, description="Whether task is enabled")
@@ -39,24 +46,38 @@ class SchedulerTask(BaseModel):
 class SchedulerClient:
     """Client for external scheduler service."""
 
-    def __init__(self, base_url: str, timeout: float = 30.0):
+    def __init__(self, base_url: str, api_key: str = "", timeout: float = 30.0):
         """
         Initialize scheduler client.
 
         Args:
             base_url: Scheduler API base URL (e.g., "http://scheduler:8090")
+            api_key: Bearer key for the Scheduler's task-management API.
+                The /tasks endpoints are guarded by verify_api_key (401 when
+                the Authorization header is missing), so without this key
+                every registration call fails.
             timeout: HTTP request timeout in seconds
         """
         self.base_url = base_url.rstrip("/")
+        self.api_key = api_key
         self.timeout = timeout
         self._client: Optional[httpx.AsyncClient] = None
+        if not api_key:
+            logger.warning(
+                "SchedulerClient created without an API key; task-management "
+                "calls will be rejected by the Scheduler (401)"
+            )
 
     async def _get_client(self) -> httpx.AsyncClient:
-        """Get or create HTTP client."""
+        """Get or create HTTP client (with Scheduler API Bearer auth)."""
         if self._client is None or self._client.is_closed:
+            headers = (
+                {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
+            )
             self._client = httpx.AsyncClient(
                 base_url=self.base_url,
                 timeout=self.timeout,
+                headers=headers,
             )
         return self._client
 
@@ -291,7 +312,7 @@ class SchedulerClient:
         task = SchedulerTask(
             task_name=task_name,
             service="library-desk",
-            executor="rest_api",
+            executor="rest_api_executor",
             priority=60,  # Background maintenance priority
             description=description or f"Prefetch {namespace}/{key} for {user}",
             minute=schedule.get("minute", -1),
@@ -301,13 +322,25 @@ class SchedulerClient:
             day_of_week=schedule.get("day_of_week", -1),
             config={
                 "method": "POST",
-                "url": f"http://library-desk:8089/volatile/fetch/{namespace}/{key}",
+                # /volatile/fetch endpoints take user as a REQUIRED QUERY
+                # parameter (RequiredUserQuery) - a body user would 422.
+                "url": (
+                    f"http://library-desk:8089/volatile/fetch/"
+                    f"{namespace}/{key}?user={quote(user, safe='')}"
+                ),
                 "headers": {
                     "Content-Type": "application/json"
                 },
-                "body": {
-                    "user": user
-                }
+                # rest_api_executor only reads config["payload"] as the JSON
+                # body (a "body" key is silently ignored).
+                "payload": {},
+                # Substituted from the Scheduler's environment at execution
+                # time; without it the scheduled POST 401s against
+                # library-desk's verify_api_key.
+                "auth": {
+                    "type": "bearer",
+                    "token": LIBRARY_API_KEY_PLACEHOLDER,
+                },
             }
         )
 

src/clients/wikijs_client.py

@@ -96,24 +96,15 @@ class WikiJSClient:
             logger.error(f"GraphQL query failed: {e}", exc_info=True)
             raise
 
-    async def list_pages(
-        self,
-        path_prefix: str = "",
-        tags: Optional[List[str]] = None,
-        limit: int = 50
-    ) -> List[Dict[str, Any]]:
+    async def _fetch_pages(self, limit: int) -> List[Dict[str, Any]]:
         """
-        List pages with optional filtering.
-
-        Multi-tenancy: Use path_prefix to filter by user namespace.
+        Fetch a raw page listing from the GraphQL API (no client-side filtering).
 
         Args:
-            path_prefix: Filter by path prefix (e.g., "/users/jpmschweitzer")
-            tags: Filter by tags (e.g., ["projects"])
-            limit: Maximum results
+            limit: Maximum pages to request from the API
 
         Returns:
-            List of page objects
+            List of page objects with tags normalized to a list
         """
         query = """
         query ListPages($limit: Int, $orderBy: PageOrderBy) {
@@ -145,7 +136,47 @@ class WikiJSClient:
             if "tags" not in page or page["tags"] is None:
                 page["tags"] = []
 
-        # Filter by path prefix (client-side if API doesn't support)
+        return pages
+
+    async def _fetch_all_pages(self, initial_limit: int = 100) -> List[Dict[str, Any]]:
+        """
+        Fetch ALL pages from the GraphQL API.
+
+        The Wiki.js 2.x `pages.list` query only supports a `limit` argument
+        (no offset - verified via GraphQL introspection). Crucially, the
+        limit is applied BEFORE Wiki.js's own visibility filtering, so a
+        response with fewer pages than requested does NOT mean the listing
+        is complete (observed live: limit=100 -> 43 pages, limit=500 ->
+        140 pages). Exhaustive listing therefore grows the limit until the
+        returned page count stops increasing.
+
+        Args:
+            initial_limit: Page count for the first request
+
+        Returns:
+            Complete list of page objects
+        """
+        max_limit = 100_000  # Safety cap against pathological growth
+        limit = max(initial_limit, 1)
+        previous_count: Optional[int] = None
+
+        while True:
+            pages = await self._fetch_pages(limit)
+            # Complete when a grown limit yields no new pages (fixed point)
+            if previous_count is not None and len(pages) == previous_count:
+                return pages
+            if limit >= max_limit:
+                return pages
+            previous_count = len(pages)
+            limit = min(limit * 2, max_limit)
+
+    @staticmethod
+    def _filter_pages(
+        pages: List[Dict[str, Any]],
+        path_prefix: str = "",
+        tags: Optional[List[str]] = None
+    ) -> List[Dict[str, Any]]:
+        """Apply client-side path-prefix and tag filters to a page listing."""
         if path_prefix:
             # Normalize paths to have leading slash for consistent comparison
             normalized_prefix = "/" + path_prefix.lstrip("/")
@@ -163,6 +194,38 @@ class WikiJSClient:
 
         return pages
 
+    async def list_pages(
+        self,
+        path_prefix: str = "",
+        tags: Optional[List[str]] = None,
+        limit: int = 50
+    ) -> List[Dict[str, Any]]:
+        """
+        List pages with optional filtering.
+
+        Multi-tenancy: Use path_prefix to filter by user namespace.
+
+        Args:
+            path_prefix: Filter by path prefix (e.g., "/users/jpmschweitzer")
+            tags: Filter by tags (e.g., ["projects"])
+            limit: Maximum results (applied AFTER filtering)
+
+        Returns:
+            List of page objects
+        """
+        if path_prefix or tags:
+            # The API limit applies before our client-side filters, so a
+            # small limit would drop matching pages that sort late. Fetch
+            # everything, filter, then apply the limit.
+            pages = self._filter_pages(
+                await self._fetch_all_pages(),
+                path_prefix=path_prefix,
+                tags=tags
+            )
+            return pages[:limit]
+
+        return await self._fetch_pages(limit)
+
     async def list_all_pages(
         self,
         path_prefix: str = "",
@@ -172,34 +235,22 @@ class WikiJSClient:
         """
         List ALL pages with pagination support.
 
-        Fetches pages in batches until all are retrieved.
+        Fetches pages in growing batches until all are retrieved (Wiki.js
+        `pages.list` has no offset argument), then applies filters.
 
         Args:
             path_prefix: Filter by path prefix (e.g., "users/jpmschweitzer")
             tags: Filter by tags
-            batch_size: Number of pages per batch (max 100)
+            batch_size: Page count for the first request
 
         Returns:
             Complete list of page objects
         """
-        all_pages = []
-        offset = 0
-
-        while True:
-            # Wiki.js list doesn't support offset, but limit is enough
-            # since we filter client-side by path_prefix
-            # Just fetch a large batch
-            pages = await self.list_pages(
-                path_prefix=path_prefix,
-                tags=tags,
-                limit=1000  # Fetch up to 1000 at once
-            )
-
-            if not pages:
-                break
-
-            all_pages = pages
-            break  # Wiki.js list doesn't paginate, so one call is enough
+        all_pages = self._filter_pages(
+            await self._fetch_all_pages(initial_limit=batch_size),
+            path_prefix=path_prefix,
+            tags=tags
+        )
 
         logger.info(f"list_all_pages: found {len(all_pages)} pages (prefix: {path_prefix or 'all'})")
         return all_pages
@@ -449,15 +500,21 @@ class WikiJSClient:
         }
         """
 
-        variables = {"id": page_id}
+        # Wiki.js 2.x requires `tags` on the update mutation (the server
+        # unconditionally maps over it; omitting it fails with "Cannot read
+        # properties of undefined (reading 'map')"). Preserve the page's
+        # current tags when the caller does not supply any.
+        if tags is None:
+            current = await self.get_page(page_id)
+            tags = (current or {}).get("tags") or []
+
+        variables = {"id": page_id, "tags": tags}
         if content is not None:
             variables["content"] = content
         if title is not None:
             variables["title"] = title
         if description is not None:
             variables["description"] = description
-        if tags is not None:
-            variables["tags"] = tags
         if is_published is not None:
             variables["isPublished"] = is_published
 
@@ -537,9 +594,13 @@ class WikiJSClient:
         data = await self._execute_query(gql_query, {"query": query})
         results = data.get("pages", {}).get("search", {}).get("results", [])
 
-        # Filter by path prefix if provided
+        # Filter by path prefix if provided. Wiki.js returns paths WITHOUT a
+        # leading slash while get_wikijs_namespace() produces one WITH it, so
+        # compare slash-normalized (the mismatch made this filter reject every
+        # result, returning an empty search for every tenant).
         if path_prefix:
-            results = [r for r in results if r["path"].startswith(path_prefix)]
+            prefix = path_prefix.lstrip("/")
+            results = [r for r in results if r["path"].lstrip("/").startswith(prefix)]
 
         return results
 

src/config.py

@@ -40,6 +40,7 @@ class Settings(BaseSettings):
     # Qdrant Configuration
     qdrant_host: str = Field(default="qdrant", description="Qdrant host")
     qdrant_port: int = Field(default=6333, description="Qdrant port")
+    qdrant_timeout: int = Field(default=30, ge=1, le=300, description="Qdrant client timeout in seconds")
 
     # Wiki.js Configuration
     wikijs_url: str = Field(default="http://wiki:3000", description="Wiki.js URL")
@@ -66,7 +67,9 @@ class Settings(BaseSettings):
 
     # Ollama Configuration
     ollama_url: str = Field(default="http://ollama:11434", description="Ollama URL")
-    ollama_model: str = Field(default="mistral-nemo-large:latest", description="Ollama LLM model")
+    # Named ollama_llm_model (env: OLLAMA_LLM_MODEL) to avoid collision with the
+    # OLLAMA_MODEL container env var, which is used for the embedding model.
+    ollama_llm_model: str = Field(default="gemma4:e2b", description="Ollama LLM model for generation (keyword extraction, re-ranking, consolidation)")
     ollama_embedding_model: str = Field(default="nomic-embed-text", description="Ollama embedding model")
 
     # HybridRAG Configuration
@@ -92,6 +95,18 @@ class Settings(BaseSettings):
     app_version: str = Field(default=__version__, description="Application version")
     debug: bool = Field(default=False, description="Debug mode")
 
+    # CORS: comma-separated list of allowed browser origins. The default "*"
+    # is only acceptable because allow_credentials is disabled (see main.py).
+    cors_allow_origins: str = Field(
+        default="*",
+        description="Comma-separated CORS allowed origins (credentials are never allowed)"
+    )
+
+    @property
+    def cors_allow_origins_list(self) -> list[str]:
+        """cors_allow_origins parsed into a list for CORSMiddleware."""
+        return [o.strip() for o in self.cors_allow_origins.split(",") if o.strip()]
+
     # RAG Search Configuration
     search_cache_ttl: int = Field(default=300, ge=0, le=3600, description="Search cache TTL in seconds")
     search_timeout: int = Field(default=10, ge=1, le=60, description="SearXNG timeout in seconds")
@@ -130,6 +145,13 @@ class Settings(BaseSettings):
 
     # Scheduler Service
     scheduler_url: str = Field(default="http://scheduler:8090", description="Scheduler service URL")
+    scheduler_api_key: str = Field(
+        default="",
+        description=(
+            "Bearer key for the Scheduler's auth-guarded task-management API; "
+            "required for runtime prefetch task registration"
+        ),
+    )
 
     @property
     def qdrant_url(self) -> str:

src/core/dependencies.py

@@ -10,7 +10,7 @@ Provides FastAPI dependencies for service clients with:
 
 from functools import lru_cache
 from typing import Annotated
-from fastapi import Depends
+from fastapi import Depends, HTTPException, Query
 import logging
 
 import redis.asyncio as aioredis
@@ -38,6 +38,35 @@ logger = logging.getLogger(__name__)
 SettingsDep = Annotated[Settings, Depends(get_settings)]
 
 
+# Tenant user dependency
+def require_user(
+    user: str = Query(
+        ...,
+        description=(
+            "User identifier (tenant). Required — every operation is scoped to "
+            "this tenant's namespace (Qdrant collection, Neo4j labels, wiki path, "
+            "Redis keys). Requests without an explicit non-empty user are "
+            "rejected with 422. There is no default tenant."
+        ),
+    )
+) -> str:
+    """
+    FastAPI dependency: required tenant user query parameter.
+
+    Rejects missing (FastAPI returns 422 automatically), empty, and
+    whitespace-only user values. Use via the RequiredUserQuery alias.
+    """
+    from src.core.multi_tenancy import validate_required_user
+
+    try:
+        return validate_required_user(user)
+    except ValueError as e:
+        raise HTTPException(status_code=422, detail=str(e))
+
+
+RequiredUserQuery = Annotated[str, Depends(require_user)]
+
+
 # Client factory functions with @lru_cache for singletons
 @lru_cache
 def get_neo4j_client() -> Neo4jClient:
@@ -72,7 +101,8 @@ def get_qdrant_client() -> QdrantClientWrapper:
     settings = get_settings()
     client = QdrantClientWrapper(
         url=settings.qdrant_url,
-        embedding_dim=768  # nomic-embed-text default
+        embedding_dim=768,  # nomic-embed-text default
+        timeout=settings.qdrant_timeout
     )
     logger.debug("Created Qdrant client instance")
     return client
@@ -146,6 +176,21 @@ def get_redis_client() -> aioredis.Redis:
     return client
 
 
+@lru_cache
+def get_job_manager() -> "JobManager":
+    """
+    Get Redis-backed JobManager singleton.
+
+    Returns:
+        JobManager for background job tracking (connects lazily)
+    """
+    from src.jobs.job_manager import JobManager
+    settings = get_settings()
+    manager = JobManager(redis_url=settings.redis_url)
+    logger.debug(f"Created JobManager: {settings.redis_url}")
+    return manager
+
+
 @lru_cache
 def get_content_extractor() -> ContentExtractor:
     """
@@ -214,7 +259,10 @@ def get_scheduler_client() -> SchedulerClient:
     Note: Used for registering prefetch tasks discovered during HybridRAG searches
     """
     settings = get_settings()
-    client = SchedulerClient(base_url=settings.scheduler_url)
+    client = SchedulerClient(
+        base_url=settings.scheduler_url,
+        api_key=settings.scheduler_api_key,
+    )
     logger.debug(f"Created Scheduler client: {settings.scheduler_url}")
     return client
 
@@ -339,6 +387,9 @@ PaperlessDep = Annotated[PaperlessClient, Depends(get_paperless_client)]
 SettingsClientDep = Annotated[SettingsClient, Depends(get_settings_client)]
 SchedulerDep = Annotated[SchedulerClient, Depends(get_scheduler_client)]
 
+from src.jobs.job_manager import JobManager  # noqa: E402
+JobManagerDep = Annotated[JobManager, Depends(get_job_manager)]
+
 # External API provider dependencies
 WeatherProviderDep = Annotated[OpenMeteoProvider, Depends(get_weather_provider)]
 NewsProviderDep = Annotated[AggregatedNewsProvider, Depends(get_news_provider)]
@@ -498,6 +549,14 @@ async def shutdown_clients():
     except Exception as e:
         logger.error(f"Error closing scheduler client: {e}")
 
+    # Close job manager Redis connection
+    try:
+        job_manager = get_job_manager()
+        await job_manager.close()
+        logger.info("✓ JobManager closed")
+    except Exception as e:
+        logger.error(f"Error closing JobManager: {e}")
+
     logger.info("Service clients shutdown complete")
 
 
@@ -536,7 +595,7 @@ async def check_service_health() -> dict:
     try:
         qdrant = get_qdrant_client()
         # Check if we can list collections
-        collections = qdrant.client.get_collections()
+        await qdrant.client.get_collections()
         health["qdrant"] = True
     except Exception as e:
         logger.error(f"Qdrant health check failed: {e}")
@@ -662,7 +721,14 @@ def get_ingestion_service() -> "IngestionService":
 
 @lru_cache
 def get_hybrid_rag_service() -> "HybridRAGService":
-    """Get HybridRAGService singleton."""
+    """
+    Get HybridRAGService singleton.
+
+    The single wiring point for HybridRAG — routers must depend on this
+    instead of constructing their own instance (previous inline copies in
+    the /query/hybrid and /wiki/smart-create routers diverged on
+    volatile_service).
+    """
     from src.services.hybrid_rag_service import HybridRAGService
     return HybridRAGService(
         vector_service=get_vector_service(),
@@ -670,7 +736,8 @@ def get_hybrid_rag_service() -> "HybridRAGService":
         searxng_client=get_searxng_client(),
         ollama_client=get_ollama_client(),
         content_extractor=get_content_extractor(),
-        settings=get_settings()
+        settings=get_settings(),
+        volatile_service=get_volatile_cache_service()
     )
 
 
@@ -698,7 +765,8 @@ def get_volatile_cache_service() -> "VolatileCacheService":
 
 
 # Authentication
-from fastapi import Security, HTTPException
+import secrets
+from fastapi import Security, HTTPException, Request
 from fastapi.security import HTTPBearer
 
 security = HTTPBearer()
@@ -721,7 +789,7 @@ async def verify_api_key(
     Raises:
         HTTPException: If API key is invalid
     """
-    if credentials.credentials != settings.library_api_key:
+    if not secrets.compare_digest(credentials.credentials, settings.library_api_key):
         raise HTTPException(
             status_code=403,
             detail="Invalid API key"
@@ -729,6 +797,42 @@ async def verify_api_key(
     return credentials.credentials
 
 
+# Header set by NPM only on the authenticated /library-desk/ proxy location.
+# library-desk is bound to loopback (127.0.0.1:8089), so NPM is the only path
+# that can reach it and set this header — a client cannot forge it. NPM also
+# overwrites any client-supplied value via proxy_set_header.
+_PROXY_MARKER_HEADER = "x-library-desk-proxy"
+
+
+async def verify_browser_request(
+    request: Request,
+    settings: SettingsDep,
+) -> str:
+    """
+    Auth for browser-facing endpoints (the Wiki.js integration buttons).
+
+    Accepts the request when it arrives through the authenticated NPM proxy
+    location (Authentik session for external users, or the LAN bypass for
+    internal ones) — identified by the trusted proxy marker header. No secret
+    is carried in the browser. Machine callers may still authenticate with the
+    Bearer API key. Returns the acting user's identity.
+    """
+    if request.headers.get(_PROXY_MARKER_HEADER) == "1":
+        # Authentik injects the identity for externally-authenticated users;
+        # on the LAN bypass these are empty and the endpoint falls back to the
+        # user supplied in the request body.
+        return request.headers.get("x-authentik-email") or "lan"
+
+    # Fallback: server-to-server Bearer API key.
+    auth = request.headers.get("authorization", "")
+    if auth.startswith("Bearer ") and secrets.compare_digest(
+        auth[len("Bearer "):], settings.library_api_key
+    ):
+        return auth[len("Bearer "):]
+
+    raise HTTPException(status_code=401, detail="Unauthenticated")
+
+
 # Service type aliases for FastAPI endpoint dependencies
 # These are defined after the factory functions
 from src.services.vector_service import VectorService

src/core/hashing.py

@@ -0,0 +1,26 @@
+"""
+Content hashing helpers.
+
+A single canonical hash implementation is used everywhere page content is
+fingerprinted (Document nodes at ingestion time, /ingest/check-updates
+comparisons) so hashes computed at different times are comparable.
+"""
+
+import hashlib
+
+
+def compute_content_hash(content: str) -> str:
+    """
+    Compute the canonical content hash for wiki page content.
+
+    Args:
+        content: Raw page content (markdown). None-safe: treated as "".
+
+    Returns:
+        Hex-encoded SHA-256 digest of the UTF-8 encoded content.
+
+    Examples:
+        >>> compute_content_hash("hello")
+        '2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824'
+    """
+    return hashlib.sha256((content or "").encode("utf-8")).hexdigest()

src/core/multi_tenancy.py

@@ -8,9 +8,9 @@ Provides utilities for user namespace management across:
 """
 
 import re
+from typing import Annotated
 
-# Default user for all operations
-DEFAULT_USER = "jpmschweitzer"
+from pydantic import AfterValidator
 
 
 def sanitize_user_id(user_id: str) -> str:
@@ -186,6 +186,45 @@ def validate_user_id(user_id: str) -> bool:
     return True
 
 
+def validate_required_user(user: str) -> str:
+    """
+    Validate that a tenant user identifier is present and usable.
+
+    There is NO default tenant: every operation that touches tenant data
+    must receive an explicit user. Empty or whitespace-only values are
+    rejected, as are values that fail :func:`validate_user_id`.
+
+    Args:
+        user: Raw user identifier from a request
+
+    Returns:
+        The stripped user identifier
+
+    Raises:
+        ValueError: If the user is missing, blank, or invalid
+
+    Examples:
+        >>> validate_required_user("llm_tester")
+        'llm_tester'
+        >>> validate_required_user("  ")
+        Traceback (most recent call last):
+        ...
+        ValueError: user is required and must be a non-empty, non-whitespace string
+    """
+    if user is None or not str(user).strip():
+        raise ValueError(
+            "user is required and must be a non-empty, non-whitespace string"
+        )
+    stripped = str(user).strip()
+    if not validate_user_id(stripped):
+        raise ValueError(f"Invalid user identifier: {user!r}")
+    return stripped
+
+
+# Pydantic annotated type for request models: a required, validated tenant user.
+RequiredUser = Annotated[str, AfterValidator(validate_required_user)]
+
+
 def is_path_in_user_namespace(path: str, user_id: str) -> bool:
     """
     Check if a Wiki.js path belongs to user's namespace.
@@ -204,6 +243,16 @@ def is_path_in_user_namespace(path: str, user_id: str) -> bool:
         False
         >>> is_path_in_user_namespace("/public/docs", "jpmschweitzer")
         False
+        >>> is_path_in_user_namespace("/users/llm_tester2/x", "llm_tester")
+        False
+        >>> is_path_in_user_namespace("users/llm-tester/x", "llm_tester")
+        True
     """
-    namespace = get_wikijs_namespace(user_id)
-    return path.startswith(namespace)
+    # Compare the tenant path segment exactly (after sanitization, since
+    # canonical wiki namespaces use sanitized user ids). This enforces a
+    # segment boundary — "users/llm_tester2" is NOT in "llm_tester"'s
+    # namespace — and treats "llm-tester"/"llm_tester" as the same tenant.
+    parts = str(path).lstrip("/").split("/")
+    if len(parts) < 2 or parts[0] != "users":
+        return False
+    return sanitize_user_id(parts[1]) == sanitize_user_id(user_id)

src/jobs/job_manager.py

@@ -9,6 +9,7 @@ Provides background job management with:
 - User-scoped job queries
 """
 
+import asyncio
 import redis.asyncio as redis
 import json
 import uuid
@@ -424,3 +425,38 @@ class JobManager:
                 stats[status] += 1
 
         return stats
+
+
+async def job_cleanup_loop(
+    job_manager: JobManager,
+    interval_seconds: float = 3600,
+    max_iterations: Optional[int] = None
+) -> int:
+    """
+    Periodically clean up expired job-set memberships.
+
+    Redis auto-expires the job payloads (24h TTL) but set memberships
+    (library:active_jobs, library:user_jobs:{user}) need manual cleanup.
+    Started as an in-process background task at application startup.
+
+    Args:
+        job_manager: JobManager whose cleanup_expired_jobs is invoked
+        interval_seconds: Sleep between cleanup passes (default hourly)
+        max_iterations: Stop after N passes (None = run forever; used by tests)
+
+    Returns:
+        Number of completed cleanup passes (only reachable with max_iterations)
+    """
+    iterations = 0
+    while max_iterations is None or iterations < max_iterations:
+        try:
+            await asyncio.sleep(interval_seconds)
+            await job_manager.cleanup_expired_jobs()
+        except asyncio.CancelledError:
+            logger.info("Job cleanup loop cancelled")
+            raise
+        except Exception as e:
+            # Never let a transient Redis error kill the loop
+            logger.error(f"Job cleanup pass failed: {e}")
+        iterations += 1
+    return iterations

src/main.py

@@ -11,16 +11,17 @@ Following best practices:
 from fastapi import FastAPI, HTTPException, Depends, Query
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.staticfiles import StaticFiles
-from pydantic import BaseModel
+from pydantic import BaseModel, Field
 from typing import Dict, Any
 import logging
 from pathlib import Path
 
 from src.config import Settings, get_settings, __version__
 from src.core.dependencies import (
-    verify_api_key, QdrantDep, WikiJSDep, OllamaDep, Neo4jDep
+    verify_api_key, QdrantDep, WikiJSDep, OllamaDep, Neo4jDep, PaperlessDep,
+    RequiredUserQuery, JobManagerDep
 )
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.multi_tenancy import RequiredUser
 
 # Configure logging
 logging.basicConfig(
@@ -38,11 +39,18 @@ app = FastAPI(
     redoc_url="/redoc",
 )
 
-# CORS middleware
+# CORS middleware.
+# allow_credentials is deliberately False: combined with a wildcard origin it
+# would tell browsers to attach cookies/credentials for ANY site, which is the
+# classic CORS misconfiguration. All real callers (tatlock, the Scheduler) are
+# server-to-server and use the Authorization header, which wildcard-origin
+# CORS without credentials still permits. Origins can be restricted via the
+# CORS_ALLOW_ORIGINS env (comma-separated) once a cross-origin browser UI
+# exists; the bundled static UI is served same-origin and needs no CORS.
 app.add_middleware(
     CORSMiddleware,
-    allow_origins=["*"],  # Configure appropriately for production
-    allow_credentials=True,
+    allow_origins=get_settings().cors_allow_origins_list,
+    allow_credentials=False,
     allow_methods=["*"],
     allow_headers=["*"],
 )
@@ -85,6 +93,14 @@ class HealthResponse(BaseModel):
     services: Dict[str, Any]
 
 
+class StatsResponse(BaseModel):
+    """System statistics response model."""
+    neo4j: Dict[str, int]
+    qdrant: Dict[str, Any]
+    wiki_pages: int
+    paperless: Dict[str, Any]
+
+
 # Routes
 @app.get("/", tags=["Root"])
 async def root() -> Dict[str, str]:
@@ -107,10 +123,19 @@ async def health(settings: Settings = Depends(get_settings)) -> HealthResponse:
     # Check service connectivity
     service_health = await check_service_health()
 
-    # Overall status is healthy if at least Neo4j and Qdrant are up
+    # Overall status is healthy if at least Neo4j and Qdrant are up.
+    #
+    # The Wiki.js change listener is reported below but deliberately excluded
+    # from this decision. It supervises and reconnects itself, and a database
+    # restart would otherwise flip the container unhealthy for the duration of
+    # an outage it is already recovering from. It is reported so the state is
+    # observable at all — previously nothing anywhere exposed it, which is how a
+    # dead listener went unnoticed while this endpoint answered "healthy".
     all_healthy = service_health.get("neo4j", False) and service_health.get("qdrant", False)
     overall_status = "healthy" if all_healthy else "degraded"
 
+    wiki_listener = getattr(app.state, "wiki_listener", None)
+
     return HealthResponse(
         status=overall_status,
         app_name=settings.app_name,
@@ -134,68 +159,318 @@ async def health(settings: Settings = Depends(get_settings)) -> HealthResponse:
             },
             "ollama": {
                 "url": settings.ollama_url,
-                "model": settings.ollama_model,
+                "model": settings.ollama_llm_model,
                 "healthy": service_health.get("ollama", False)
+            },
+            "wiki_listener": {
+                "subscribed": bool(wiki_listener and wiki_listener.running),
+                "reconnects": getattr(wiki_listener, "reconnects", 0),
+                "last_gap_seconds": getattr(wiki_listener, "last_gap_seconds", None)
             }
         }
     )
 
 
+@app.get("/stats", response_model=StatsResponse, tags=["System"])
+async def stats(
+    user: RequiredUserQuery,
+    neo4j: Neo4jDep = None,
+    qdrant: QdrantDep = None,
+    wikijs: WikiJSDep = None,
+    paperless: PaperlessDep = None,
+    api_key: str = Depends(verify_api_key)
+) -> StatsResponse:
+    """
+    Get system statistics.
+
+    Returns counts for:
+    - Neo4j: nodes by type (Document, Entity, Collection, Search)
+    - Qdrant: vectors per collection
+    - Wiki.js: total page count
+    - Paperless: documents, tags, correspondents, document types
+    """
+    # Neo4j node counts by label
+    neo4j_stats = {}
+    try:
+        for label in ["Document", "Entity", "Collection", "Search"]:
+            result = await neo4j.execute_query(
+                f"MATCH (n:{label}) RETURN count(n) as count"
+            )
+            neo4j_stats[label.lower() + "_nodes"] = result[0]["count"] if result else 0
+    except Exception as e:
+        logger.error(f"Failed to get Neo4j stats: {e}")
+        neo4j_stats = {"error": str(e)}
+
+    # Qdrant collection stats
+    qdrant_stats = {}
+    try:
+        collections = await qdrant.list_collections()
+        qdrant_stats["collections"] = len(collections)
+        qdrant_stats["total_vectors"] = sum(c.get("vectors_count", 0) for c in collections)
+        qdrant_stats["by_collection"] = {
+            c["name"]: c["vectors_count"] for c in collections
+        }
+    except Exception as e:
+        logger.error(f"Failed to get Qdrant stats: {e}")
+        qdrant_stats = {"error": str(e)}
+
+    # Wiki.js page count (pages live under the user namespace, e.g. "users/jpmschweitzer/...")
+    wiki_pages = 0
+    try:
+        pages = await wikijs.list_all_pages(path_prefix=f"users/{user}")
+        wiki_pages = len(pages)
+    except Exception as e:
+        logger.warning(f"Failed to get Wiki.js stats: {e}")
+
+    # Paperless-ngx document stats
+    paperless_stats = {}
+    try:
+        # Get document count (page_size=1 for efficiency, we just need the count)
+        docs_result = await paperless.list_documents(page_size=1)
+        paperless_stats["documents"] = docs_result.get("count", 0)
+
+        # Get metadata counts
+        tags = await paperless.list_tags()
+        paperless_stats["tags"] = len(tags)
+
+        correspondents = await paperless.list_correspondents()
+        paperless_stats["correspondents"] = len(correspondents)
+
+        doc_types = await paperless.list_document_types()
+        paperless_stats["document_types"] = len(doc_types)
+    except Exception as e:
+        logger.warning(f"Failed to get Paperless stats: {e}")
+        paperless_stats = {"error": str(e)}
+
+    return StatsResponse(
+        neo4j=neo4j_stats,
+        qdrant=qdrant_stats,
+        wiki_pages=wiki_pages,
+        paperless=paperless_stats
+    )
+
+
+class CheckUpdatesRequest(BaseModel):
+    """Request body for /ingest/check-updates."""
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — only this tenant's namespace is compared."
+    )
+    path_prefix: str | None = Field(
+        default=None,
+        description="Optional sub-path inside the tenant namespace (e.g. 'technology')"
+    )
+
+
 @app.post("/ingest/check-updates", tags=["Ingestion"])
 async def check_updates(
-    documents: Dict[str, Any],
+    request: CheckUpdatesRequest,
+    neo4j: Neo4jDep = None,
+    wikijs: WikiJSDep = None,
     api_key: str = Depends(verify_api_key)
 ) -> Dict[str, Any]:
     """
-    Check which documents need updating based on content hashes.
-    Used by Scheduler to determine what changed since last sync.
+    Check which wiki pages need (re-)ingestion based on content hashes.
 
-    TODO: Implement update detection:
-    1. Query existing documents by path
-    2. Compare content hashes
-    3. Return list of updates needed
+    Compares the `content_hash` stored on the tenant's Neo4j Document nodes
+    (recorded at ingestion time) against the SHA-256 of the current Wiki.js
+    page content, in a single UNWIND Cypher query. Used by the Scheduler to
+    determine what changed since the last sync. Read-only.
+
+    Returns per-tenant lists:
+    - `changed`: page exists in wiki AND graph, but hashes differ (or the
+      stored hash predates hash tracking — flagged `stored_hash_missing`)
+    - `new`: wiki page with no Document node yet
+    - `deleted`: Document node whose wiki page no longer exists
     """
-    return {
-        "message": "Update checking not yet implemented",
-        "updates_needed": [],
-        "up_to_date": [],
-        "new_documents": []
-    }
+    import time as _time
+    from src.core.hashing import compute_content_hash
+    from src.core.multi_tenancy import get_neo4j_user_label, sanitize_user_id
+
+    start_time = _time.time()
+    user = request.user
+    tenant_prefix = f"users/{sanitize_user_id(user)}"
+    if request.path_prefix:
+        tenant_prefix = f"{tenant_prefix}/{request.path_prefix.strip('/')}"
+
+    try:
+        pages = await wikijs.list_all_pages(path_prefix=tenant_prefix)
+
+        # Auto-generated entity stubs are intentionally never ingested into
+        # the graph (see GraphService.update_from_page), so they would show
+        # up as perpetually "new". Exclude them.
+        pages = [
+            p for p in pages
+            if not ({"entity-stub", "auto-generated"} & set(p.get("tags") or []))
+        ]
+
+        page_hashes = []
+        for p in pages:
+            full_page = await wikijs.get_page(p["id"])
+            content = (full_page or {}).get("content", "")
+            page_hashes.append({
+                "page_id": p["id"],
+                "path": p.get("path", ""),
+                "title": p.get("title", ""),
+                "hash": compute_content_hash(content)
+            })
+
+        user_doc_label = get_neo4j_user_label(user)
+
+        if page_hashes:
+            # Single UNWIND query: compare every current page hash against the
+            # stored Document hash AND collect stale Document nodes whose wiki
+            # page is gone.
+            cypher = f"""
+            UNWIND $pages AS p
+            OPTIONAL MATCH (d:{user_doc_label}:Document {{page_id: p.page_id}})
+            WITH collect({{
+                    page_id: p.page_id,
+                    path: p.path,
+                    title: p.title,
+                    is_new: d IS NULL,
+                    changed: d IS NOT NULL AND (d.content_hash IS NULL OR d.content_hash <> p.hash),
+                    stored_hash_missing: d IS NOT NULL AND d.content_hash IS NULL
+                }}) AS checked,
+                collect(p.page_id) AS current_ids
+            OPTIONAL MATCH (stale:{user_doc_label}:Document)
+            WHERE stale.page_id IS NOT NULL AND NOT stale.page_id IN current_ids
+            RETURN checked,
+                   collect(CASE WHEN stale IS NULL THEN NULL ELSE {{
+                       page_id: stale.page_id, path: stale.path, title: stale.title
+                   }} END) AS deleted
+            """
+            rows = await neo4j.execute_query(cypher, {"pages": page_hashes})
+            checked = rows[0]["checked"] if rows else []
+            deleted = rows[0]["deleted"] if rows else []
+        else:
+            # No wiki pages under the prefix: every Document node is stale.
+            cypher = f"""
+            MATCH (stale:{user_doc_label}:Document)
+            WHERE stale.page_id IS NOT NULL
+            RETURN collect({{page_id: stale.page_id, path: stale.path, title: stale.title}}) AS deleted
+            """
+            rows = await neo4j.execute_query(cypher, {})
+            checked = []
+            deleted = rows[0]["deleted"] if rows else []
+
+        # Deleted detection is namespace-wide only for full-tenant scans; a
+        # sub-path scan must not flag documents outside its prefix.
+        if request.path_prefix:
+            deleted = [
+                d for d in deleted
+                if str(d.get("path", "")).lstrip("/").startswith(tenant_prefix)
+            ]
+
+        new_pages = [c for c in checked if c["is_new"]]
+        changed_pages = [c for c in checked if c["changed"]]
+        up_to_date = len(checked) - len(new_pages) - len(changed_pages)
+
+        duration_ms = (_time.time() - start_time) * 1000
+
+        return {
+            "user": user,
+            "path_prefix": tenant_prefix,
+            "total_wiki_pages": len(page_hashes),
+            "changed": [
+                {k: c[k] for k in ("page_id", "path", "title", "stored_hash_missing")}
+                for c in changed_pages
+            ],
+            "new": [
+                {k: c[k] for k in ("page_id", "path", "title")} for c in new_pages
+            ],
+            "deleted": deleted,
+            "counts": {
+                "changed": len(changed_pages),
+                "new": len(new_pages),
+                "deleted": len(deleted),
+                "up_to_date": up_to_date
+            },
+            "duration_ms": duration_ms
+        }
+
+    except Exception as e:
+        logger.error(f"check-updates failed for {user}: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Update check failed")
 
 
-@app.get("/ingest/status/{document_id}", tags=["Ingestion"])
+@app.get("/ingest/status/{job_id}", tags=["Ingestion"])
 async def get_ingestion_status(
-    document_id: str,
+    job_id: str,
+    user: RequiredUserQuery,
+    job_manager: JobManagerDep = None,
     api_key: str = Depends(verify_api_key)
 ) -> Dict[str, Any]:
     """
-    Get processing status for a document.
+    Get processing status for an ingestion job.
 
-    TODO: Implement status tracking
+    Backed by the Redis job store (`library:job:{job_id}`, 24h TTL). Job IDs
+    are returned by /ingest/page, /ingest/batch and /ingest/all. Jobs are
+    tenant-scoped: requesting another tenant's job returns 404.
     """
-    return {
-        "message": "Status tracking not yet implemented",
-        "document_id": document_id,
-        "status": "unknown"
-    }
+    job = await job_manager.get_job(job_id)
+    if not job or job.get("user") != user:
+        raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
+    return job
 
 
 @app.get("/ingest/repo-status/{repository}", tags=["Ingestion"])
 async def get_repo_status(
     repository: str,
+    user: RequiredUserQuery,
+    neo4j: Neo4jDep = None,
+    wikijs: WikiJSDep = None,
+    job_manager: JobManagerDep = None,
     api_key: str = Depends(verify_api_key)
 ) -> Dict[str, Any]:
     """
-    Get indexing status for an entire repository.
+    Get indexing status for a repository (a sub-path of the tenant namespace).
 
-    TODO: Implement repository-level statistics
+    `repository` is resolved as `users/{tenant}/{repository}`; use `_all` for
+    the whole tenant namespace. Reports how many wiki pages exist under the
+    path, how many have graph Document nodes (i.e. are indexed), and the
+    tenant's recent job statistics from the Redis job store.
     """
-    return {
-        "message": "Repository status not yet implemented",
-        "repository": repository,
-        "total_documents": 0,
-        "indexed_documents": 0
-    }
+    import time as _time
+    from src.core.multi_tenancy import get_neo4j_user_label, sanitize_user_id
+
+    start_time = _time.time()
+    tenant_root = f"users/{sanitize_user_id(user)}"
+    prefix = tenant_root if repository in ("_all", "all", "") else f"{tenant_root}/{repository.strip('/')}"
+
+    try:
+        pages = await wikijs.list_all_pages(path_prefix=prefix)
+        page_ids = [p["id"] for p in pages if p.get("id")]
+
+        indexed = 0
+        if page_ids:
+            user_doc_label = get_neo4j_user_label(user)
+            rows = await neo4j.execute_query(
+                f"""
+                MATCH (d:{user_doc_label}:Document)
+                WHERE d.page_id IN $page_ids
+                RETURN count(DISTINCT d.page_id) AS indexed
+                """,
+                {"page_ids": page_ids}
+            )
+            indexed = rows[0]["indexed"] if rows else 0
+
+        job_stats = await job_manager.get_job_stats(user=user)
+
+        return {
+            "repository": repository,
+            "user": user,
+            "path_prefix": prefix,
+            "total_documents": len(page_ids),
+            "indexed_documents": indexed,
+            "unindexed_documents": len(page_ids) - indexed,
+            "jobs": job_stats,
+            "duration_ms": (_time.time() - start_time) * 1000
+        }
+
+    except Exception as e:
+        logger.error(f"repo-status failed for {user}/{repository}: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Repository status failed")
 
 
 # Query endpoints
@@ -203,8 +478,8 @@ async def get_repo_status(
 
 @app.post("/query/semantic", tags=["Query"])
 async def semantic_query(
+    user: RequiredUserQuery,
     query: str = Query(..., min_length=1, description="Search query text"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     limit: int = Query(default=10, ge=1, le=100, description="Maximum results"),
     score_threshold: float = Query(default=0.5, ge=0.0, le=1.0, description="Minimum similarity score"),
     qdrant_client: QdrantDep = None,
@@ -244,24 +519,30 @@ async def semantic_query(
 
 @app.post("/query/graph", tags=["Query"])
 async def graph_query(
+    user: RequiredUserQuery,
     query: str = Query(..., description="Cypher query to execute"),
-    user: str = Query(default=DEFAULT_USER, description="User for scoping (auto-filters results)"),
     neo4j_client: Neo4jDep = None,
     wiki_client: WikiJSDep = None,
     api_key: str = Depends(verify_api_key)
 ):
     """
-    Execute a Cypher query against the Neo4j knowledge graph.
+    Execute a raw Cypher query against the Neo4j knowledge graph
+    (ADMIN/DEBUG — read-only, NOT tenant-scoped).
 
-    Queries are automatically scoped to the user's data for security.
-    Use this for custom graph traversals beyond what /graph/nodes provides.
+    **Security model:**
+    - Queries containing write clauses (CREATE/MERGE/DELETE/SET/REMOVE/DROP/
+      DETACH/FOREACH/LOAD CSV) or any CALL are rejected with 400.
+    - Execution happens in a read-only Neo4j session, so writes are refused
+      by the database even if validation is bypassed.
+    - Results are NOT automatically restricted to the requesting user's
+      tenant: an arbitrary query can read any tenant's nodes. Scope your
+      own patterns (e.g. `MATCH (d:User_<Tenant>_Document:Document) ...`).
+      For tenant-scoped access use /graph/nodes instead.
 
     **Example:**
     ```
-    POST /query/graph?query=MATCH%20(d:Document)-[:MENTIONS]->(p:Person)%20RETURN%20d,p&user=jpmschweitzer
+    POST /query/graph?query=MATCH%20(d:Document)-[:MENTIONS]->(p:Person)%20RETURN%20d,p&user=<tenant>
     ```
-
-    **Security:** All queries are user-scoped to prevent cross-user data access.
     """
     from src.services.graph_service import GraphService
 
@@ -280,42 +561,68 @@ async def graph_query(
 
 
 # Deduplication endpoints
+class DeduplicateCheckRequest(BaseModel):
+    """Request body for /deduplicate/check."""
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — only this tenant's collection is scanned."
+    )
+    similarity_threshold: float = Field(
+        default=0.9, ge=0.5, le=1.0,
+        description="Minimum cosine similarity for a chunk pair to count as duplicate"
+    )
+    max_pairs: int = Field(default=100, ge=1, le=500, description="Maximum page pairs returned")
+
+
 @app.post("/deduplicate/check", tags=["Deduplication"])
 async def check_duplicates(
-    request: Dict[str, Any],
+    request: DeduplicateCheckRequest,
+    qdrant_client: QdrantDep = None,
+    wiki_client: WikiJSDep = None,
+    ollama_client: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
 ) -> Dict[str, Any]:
     """
-    Check for duplicate or highly similar documents.
-    Uses vector similarity and graph analysis.
+    Check for duplicate or highly similar wiki pages (tenant-scoped, read-only).
 
-    Expected fields:
-    - document_id: str
-    - similarity_threshold: float (default 0.85)
-
-    TODO: Implement deduplication:
-    1. Get document embedding from Qdrant
-    2. Find similar vectors above threshold
-    3. Check graph relationships
-    4. Return candidates with similarity scores
+    Scans the tenant's own Qdrant collection: every wiki chunk vector is
+    queried against the same collection, and chunk pairs from different
+    pages scoring above the threshold (default 0.9 cosine) are grouped per
+    page pair with the best similarity and matching chunk-pair count.
     """
-    document_id = request.get("document_id")
-    threshold = request.get("similarity_threshold", 0.85)
+    import time as _time
+    from src.services.vector_service import VectorService
 
-    return {
-        "message": "Deduplication not yet implemented",
-        "document_id": document_id,
-        "threshold": threshold,
-        "duplicates": [],
-        "suggestions": None
-    }
+    start_time = _time.time()
+    vector_service = VectorService(qdrant_client, wiki_client, ollama_client)
+
+    try:
+        scan = await vector_service.find_duplicate_pairs(
+            user=request.user,
+            similarity_threshold=request.similarity_threshold,
+            max_pairs=request.max_pairs
+        )
+        return {
+            "user": request.user,
+            "similarity_threshold": request.similarity_threshold,
+            "chunks_scanned": scan["chunks_scanned"],
+            "duplicate_groups": scan["duplicate_groups"],
+            "duplicate_group_count": len(scan["duplicate_groups"]),
+            "duration_ms": (_time.time() - start_time) * 1000
+        }
+    except Exception as e:
+        logger.error(f"Deduplication check failed for {request.user}: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Deduplication check failed")
 
 
 # Application lifecycle
 @app.on_event("startup")
 async def startup_event():
     """Initialize connections and resources on startup."""
-    from src.core.dependencies import startup_clients
+    import asyncio
+
+    from src.core.dependencies import startup_clients, get_job_manager
+    from src.jobs.job_manager import job_cleanup_loop
     from src.services.wiki_change_listener import WikiChangeListener
 
     settings = get_settings()
@@ -325,10 +632,18 @@ async def startup_event():
     logger.info(f"Wiki.js: {settings.wikijs_url}")
     logger.info(f"SearXNG: {settings.searxng_url}")
     logger.info(f"Ollama: {settings.ollama_url}")
+    logger.info(f"Ollama generation model: {settings.ollama_llm_model} (embedding model: {settings.ollama_embedding_model})")
 
     # Initialize all service clients
     await startup_clients()
 
+    # Hourly in-process cleanup of expired Redis job-set memberships
+    # (job payloads auto-expire via TTL; set memberships do not)
+    app.state.job_cleanup_task = asyncio.create_task(
+        job_cleanup_loop(get_job_manager(), interval_seconds=3600)
+    )
+    logger.info("Job cleanup loop started (hourly)")
+
     # Start Wiki.js change listener (PostgreSQL NOTIFY/LISTEN)
     # This enables automatic processing of user-edited pages
     try:
@@ -349,6 +664,15 @@ async def shutdown_event():
 
     logger.info("Shutting down Library Desk API")
 
+    # Stop the job cleanup loop
+    if hasattr(app.state, "job_cleanup_task"):
+        app.state.job_cleanup_task.cancel()
+        try:
+            await app.state.job_cleanup_task
+        except Exception:
+            pass
+        logger.info("Job cleanup loop stopped")
+
     # Stop Wiki.js change listener if running
     if hasattr(app.state, "wiki_listener"):
         try:

src/models/consolidation.py

@@ -50,9 +50,14 @@ class ConsolidationResponse(BaseModel):
     volatile_cached: int = Field(default=0, description="Items cached to volatile storage")
     files_queued: int = Field(default=0, description="Files queued for Paperless")
     prefetch_registered: int = Field(default=0, description="Prefetch patterns registered")
+    searches_deferred: int = Field(
+        default=0,
+        description="Searches left unprocessed for the next run because the generation LLM was unavailable"
+    )
     errors: List[str] = Field(default=[], description="Error messages")
     results: List[ConsolidationResult] = Field(description="Per-search results")
     dry_run: bool = Field(description="Whether this was a dry run")
+    duration_ms: float = Field(default=0.0, description="Run duration in milliseconds")
 
 
 class MemoryRouteClassification(BaseModel):

src/models/graph.py

@@ -8,6 +8,8 @@ from pydantic import BaseModel, Field
 from typing import List, Dict, Any, Optional
 from datetime import datetime
 
+from src.core.multi_tenancy import RequiredUser
+
 
 class GraphNode(BaseModel):
     """Graph node representation."""
@@ -45,9 +47,13 @@ class CypherQueryRequest(BaseModel):
         default_factory=dict,
         description="Query parameters"
     )
-    user: str = Field(
-        default="jpmschweitzer",
-        description="User for filtering (automatically scopes query)"
+    user: RequiredUser = Field(
+        ...,
+        description=(
+            "User identifier (tenant). Required. NOTE: raw Cypher queries are NOT "
+            "automatically scoped to this tenant — the endpoint is read-only and "
+            "intended for admin/debug use. Results may span all tenants."
+        )
     )
 
 
@@ -61,9 +67,9 @@ class CypherQueryResponse(BaseModel):
 class UpdateFromPageRequest(BaseModel):
     """Request to update graph from a wiki page."""
     page_id: int = Field(..., description="Wiki page ID to process")
-    user: str = Field(
-        default="jpmschweitzer",
-        description="User identifier for namespace scoping"
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — graph writes are scoped to this tenant's labels."
     )
     force_refresh: bool = Field(
         default=False,

src/models/hybrid_rag.py

@@ -88,6 +88,11 @@ class HybridRAGResponse(BaseModel):
     timing: TimingBreakdown = Field(..., description="Performance breakdown")
     config_used: HybridRAGConfig = Field(..., description="Configuration used")
     search_id: Optional[str] = Field(None, description="Search ID for Librarian tracking")
+    source_status: Dict[str, str] = Field(
+        default={},
+        description="Per-leg retrieval status ('ok', 'failed', or 'disabled') keyed by: vector, graph, web, volatile, documents"
+    )
+    degraded: bool = Field(default=False, description="True when any enabled retrieval leg reported 'failed'")
 
 
 class HybridRAGRequest(BaseModel):

src/models/ingestion.py

@@ -5,11 +5,13 @@ from pydantic import BaseModel, Field
 from typing import Optional, List, Dict, Any
 from datetime import datetime
 
+from src.core.multi_tenancy import RequiredUser
+
 
 class IngestionRequest(BaseModel):
     """Request to ingest a wiki page."""
     page_id: int = Field(..., description="Wiki page ID to ingest")
-    user: str = Field(default="jpmschweitzer", description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — ingestion writes to this tenant's namespaces only.")
     force_refresh: bool = Field(
         default=False,
         description="Force re-ingestion even if page hasn't changed"
@@ -21,7 +23,7 @@ class IngestionRequest(BaseModel):
 class BatchIngestionRequest(BaseModel):
     """Request to ingest multiple wiki pages."""
     page_ids: List[int] = Field(..., description="List of wiki page IDs to ingest")
-    user: str = Field(default="jpmschweitzer", description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — ingestion writes to this tenant's namespaces only.")
     force_refresh: bool = Field(default=False)
     skip_vectors: bool = Field(default=False)
     skip_graph: bool = Field(default=False)
@@ -44,6 +46,10 @@ class IngestionResult(BaseModel):
     graph_entities_extracted: int = 0
     graph_relationships_created: int = 0
     processing_time_ms: float
+    job_id: Optional[str] = Field(
+        default=None,
+        description="Redis job-tracking ID (query via GET /ingest/status/{job_id})"
+    )
 
 
 class BatchIngestionResult(BaseModel):
@@ -53,6 +59,10 @@ class BatchIngestionResult(BaseModel):
     failed: int
     results: List[IngestionResult]
     total_processing_time_ms: float
+    job_id: Optional[str] = Field(
+        default=None,
+        description="Redis job-tracking ID (query via GET /ingest/status/{job_id})"
+    )
 
 
 class IngestionStatus(BaseModel):

src/models/rag_search.py

@@ -8,7 +8,7 @@ from enum import Enum
 from typing import Optional, List
 from pydantic import BaseModel, Field
 
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.multi_tenancy import RequiredUser
 
 
 class SearchType(str, Enum):
@@ -37,9 +37,9 @@ class RAGSearchRequest(BaseModel):
         le=20,
         description="Maximum number of results (1-20)"
     )
-    user: str = Field(
-        default=DEFAULT_USER,
-        description="User identifier for rate limiting/personalization"
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — used for rate limiting/personalization."
     )
 
 

src/models/vector.py

@@ -7,6 +7,8 @@ Provides models for semantic search, document chunks, and embeddings.
 from pydantic import BaseModel, Field
 from typing import List, Dict, Any, Optional
 
+from src.core.multi_tenancy import RequiredUser
+
 
 class DocumentChunk(BaseModel):
     """Document chunk with embedding."""
@@ -32,7 +34,7 @@ class SearchResult(BaseModel):
 class SearchRequest(BaseModel):
     """Semantic search request."""
     query: str = Field(..., min_length=1, description="Search query")
-    user: str = Field(default="jpmschweitzer", description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — search is scoped to this tenant's collection.")
     limit: int = Field(default=10, ge=1, le=100, description="Maximum results")
     score_threshold: float = Field(default=0.5, ge=0.0, le=1.0, description="Minimum similarity score")
 
@@ -48,9 +50,9 @@ class SearchResponse(BaseModel):
 class VectorUpdateRequest(BaseModel):
     """Request to update vectors from a wiki page."""
     page_id: int = Field(..., description="Wiki page ID to process")
-    user: str = Field(
-        default="jpmschweitzer",
-        description="User identifier for namespace scoping"
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — vectors are written to this tenant's collection."
     )
     force_refresh: bool = Field(
         default=False,
@@ -65,6 +67,8 @@ class VectorUpdateSummary(BaseModel):
     chunks_created: int = Field(default=0, description="New chunks created")
     chunks_updated: int = Field(default=0, description="Existing chunks updated")
     chunks_deleted: int = Field(default=0, description="Old chunks deleted")
+    chunks_skipped: int = Field(default=0, description="Chunks skipped (embedding failed)")
+    status: str = Field(default="success", description="'success', 'partial' (some chunks skipped), or 'failed'")
     total_chunks: int = Field(default=0, description="Total chunks for this page")
     embedding_dim: int = Field(default=768, description="Embedding dimensionality")
     processing_time_ms: float = Field(..., description="Processing time in milliseconds")
@@ -89,7 +93,7 @@ class CollectionListResponse(BaseModel):
 class DeletePageChunksRequest(BaseModel):
     """Request to delete all chunks for a page."""
     page_id: int = Field(..., description="Wiki page ID")
-    user: str = Field(default="jpmschweitzer", description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — deletion is scoped to this tenant's collection.")
 
 
 class DeletePageChunksResponse(BaseModel):

src/models/volatile.py

@@ -18,7 +18,8 @@ class VolatileNamespace(str, Enum):
     Each namespace can have different default TTLs and refresh schedules.
     """
     # Real-time external data
-    WEATHER = "weather"          # Current conditions, forecasts
+    WEATHER = "weather"          # Current conditions (temperature, humidity, wind)
+    FORECAST = "forecast"        # Multi-day weather outlook
     SUN = "sun"                  # Sunrise, sunset, daylight duration
     NEWS = "news"                # Headlines, breaking news
     FINANCIAL = "financial"      # Stock prices, exchange rates, crypto
@@ -37,19 +38,21 @@ class VolatileNamespace(str, Enum):
 
 
 # Default TTLs per namespace (in seconds)
+# TTL = 2x refresh interval to ensure data survives missed/delayed refreshes
 NAMESPACE_DEFAULT_TTL: Dict[str, int] = {
-    VolatileNamespace.WEATHER: 1800,       # 30 min - weather changes slowly
-    VolatileNamespace.SUN: 86400,          # 24 hours - sun times change daily
-    VolatileNamespace.NEWS: 3600,          # 1 hour - news cycles
-    VolatileNamespace.FINANCIAL: 300,      # 5 min - markets move fast
-    VolatileNamespace.TRANSIT: 300,        # 5 min - schedules update frequently
-    VolatileNamespace.TRAFFIC: 600,        # 10 min - traffic patterns
-    VolatileNamespace.AIR_QUALITY: 3600,   # 1 hour - air quality stable
-    VolatileNamespace.SPORTS: 60,          # 1 min - live scores
-    VolatileNamespace.SOCIAL: 600,         # 10 min - social notifications
-    VolatileNamespace.SYSTEM: 60,          # 1 min - system health
-    VolatileNamespace.CONTEXT: 3600,       # 1 hour - session context
-    VolatileNamespace.CUSTOM: 3600,        # 1 hour - default for custom
+    VolatileNamespace.WEATHER: 7200,       # 2 hours (hourly refresh)
+    VolatileNamespace.FORECAST: 86400,     # 24 hours (12hr refresh)
+    VolatileNamespace.SUN: 172800,         # 48 hours (daily refresh)
+    VolatileNamespace.NEWS: 7200,          # 2 hours (hourly refresh)
+    VolatileNamespace.FINANCIAL: 600,      # 10 min (5 min refresh)
+    VolatileNamespace.TRANSIT: 600,        # 10 min (5 min refresh)
+    VolatileNamespace.TRAFFIC: 1200,       # 20 min (10 min refresh)
+    VolatileNamespace.AIR_QUALITY: 7200,   # 2 hours (hourly refresh)
+    VolatileNamespace.SPORTS: 120,         # 2 min (1 min refresh)
+    VolatileNamespace.SOCIAL: 1200,        # 20 min (10 min refresh)
+    VolatileNamespace.SYSTEM: 120,         # 2 min (1 min refresh)
+    VolatileNamespace.CONTEXT: 7200,       # 2 hours - session context
+    VolatileNamespace.CUSTOM: 7200,        # 2 hours - default for custom
 }
 
 

src/models/wiki.py

@@ -11,6 +11,8 @@ from pydantic import BaseModel, Field, field_validator
 from typing import Optional, List, Dict, Any
 from datetime import datetime
 
+from src.core.multi_tenancy import RequiredUser
+
 
 # Base models
 class WikiPageBase(BaseModel):
@@ -35,7 +37,7 @@ class WikiPageCreate(WikiPageBase):
     content: str = Field(..., description="Page content (markdown)")
     path: str = Field(..., min_length=1, max_length=500, description="Page path (e.g., '/projects/library-desk')")
     editor: str = Field(default="markdown", description="Editor type")
-    user: Optional[str] = Field(None, description="User identifier (defaults to configured user)")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — the page is created inside this tenant's namespace.")
 
     @field_validator("path")
     @classmethod
@@ -106,7 +108,7 @@ class DossierCreate(BaseModel):
     title: str = Field(..., min_length=1, max_length=200, description="Human-readable title")
     description: str = Field(..., min_length=1, description="Dossier description")
     create_index_page: bool = Field(default=True, description="Create an index page for the dossier")
-    user: Optional[str] = Field(None, description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required.")
 
     @field_validator("name")
     @classmethod
@@ -198,7 +200,7 @@ class WikiSmartCreateRequest(BaseModel):
     topic: str = Field(..., min_length=1, max_length=500, description="Topic to research and create page about")
     path: Optional[str] = Field(None, description="Page path (auto-generated from topic if not provided)")
     tags: List[str] = Field(default_factory=list, description="Tags for the page")
-    user: Optional[str] = Field(None, description="User identifier")
+    user: RequiredUser = Field(..., description="User identifier (tenant). Required — research results and the created page are scoped to this tenant.")
     include_web_research: bool = Field(default=True, description="Include web search results")
     include_wiki_search: bool = Field(default=True, description="Include existing wiki knowledge")
 

src/routers/documents.py

@@ -29,7 +29,7 @@ from src.core.dependencies import (
     Neo4jDep,
     WikiJSDep,
 )
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.dependencies import RequiredUserQuery
 from src.config import get_settings
 
 logger = logging.getLogger(__name__)
@@ -50,7 +50,7 @@ async def receive_webhook(
     ollama: OllamaDep,
     neo4j: Neo4jDep,
     wiki: WikiJSDep,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
 ):
     """
     Receive webhook events from Paperless-ngx.
@@ -161,9 +161,9 @@ async def capture_webhook(request: Request):
 
 @router.post("/webhook-simple", response_model=WebhookResponse)
 async def receive_webhook_simple(
+    user: RequiredUserQuery,
     doc_url: str = Query(..., description="Paperless document URL containing ID"),
     title: str = Query(default="", description="Document title"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     paperless: PaperlessDep = None,
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
@@ -347,7 +347,7 @@ async def search_documents(
     request: DocumentSearchRequest,
     qdrant: QdrantDep,
     ollama: OllamaDep,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     api_key: str = Depends(verify_api_key),
 ):
     """

src/routers/entity_linking.py

@@ -18,7 +18,7 @@ from src.core.dependencies import (
     get_wiki_service,
     get_graph_service,
     get_ingestion_service,
-    verify_api_key
+    verify_browser_request,
 )
 from src.services.wiki_service import WikiService
 from src.services.graph_service import GraphService
@@ -57,7 +57,7 @@ async def link_entities_in_page(
     wiki_service: WikiService = Depends(get_wiki_service),
     graph_service: GraphService = Depends(get_graph_service),
     ingestion_service: IngestionService = Depends(get_ingestion_service),
-    api_key: str = Depends(verify_api_key)
+    actor: str = Depends(verify_browser_request)
 ) -> EntityLinkingResult:
     """
     Find and link entities mentioned in a wiki page.

src/routers/graph.py

@@ -15,8 +15,9 @@ from src.models.graph import (
     MindMapResponse
 )
 from src.services.graph_service import GraphService
-from src.core.dependencies import Neo4jDep, WikiJSDep, verify_api_key
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.dependencies import (
+    Neo4jDep, WikiJSDep, verify_api_key, RequiredUserQuery
+)
 
 logger = logging.getLogger(__name__)
 
@@ -39,21 +40,24 @@ async def execute_cypher_query(
     api_key: str = Depends(verify_api_key)
 ):
     """
-    Execute a user-scoped Cypher query.
+    Execute a raw Cypher query (ADMIN/DEBUG — read-only, NOT tenant-scoped).
 
-    The query is automatically scoped to the user's data for security.
-    This prevents users from accessing other users' graph data.
+    **Security model:**
+    - Write clauses (CREATE/MERGE/DELETE/SET/REMOVE/DROP/DETACH/FOREACH/
+      LOAD CSV) and CALL procedures are rejected with 400.
+    - Execution happens in a read-only Neo4j session as a hard backstop.
+    - Results are NOT restricted to the requesting user's tenant labels —
+      scope your own patterns (e.g. match `User_<Tenant>_Document`).
+      For tenant-scoped access use /graph/nodes instead.
 
     **Example Request:**
     ```json
     {
-        "query": "MATCH (d:Document) RETURN d LIMIT 10",
+        "query": "MATCH (d:User_Llm_Tester_Document:Document) RETURN d LIMIT 10",
         "parameters": {},
-        "user": "jpmschweitzer"
+        "user": "<tenant>"
     }
     ```
-
-    **Security:** Query is automatically scoped with user label.
     """
     try:
         return await graph_service.execute_query(
@@ -70,7 +74,7 @@ async def execute_cypher_query(
 
 @router.get("/nodes", response_model=NodeListResponse)
 async def list_nodes(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     node_type: Optional[str] = Query(default=None, description="Node type filter"),
     limit: int = Query(default=100, ge=1, le=500, description="Maximum nodes"),
     graph_service: GraphService = Depends(get_graph_service),
@@ -81,7 +85,7 @@ async def list_nodes(
 
     Optionally filter by node type (Document, Person, Project, Concept, etc.).
 
-    **Example:** `/graph/nodes?user=jpmschweitzer&node_type=Document&limit=50`
+    **Example:** `/graph/nodes?user=<tenant>&node_type=Document&limit=50`
     """
     try:
         return await graph_service.list_nodes(
@@ -97,7 +101,7 @@ async def list_nodes(
 @router.get("/nodes/{node_id}", response_model=GraphNodeDetail)
 async def get_node(
     node_id: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     graph_service: GraphService = Depends(get_graph_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -106,7 +110,7 @@ async def get_node(
 
     Returns the node, its relationships, and connected nodes.
 
-    **Example:** `/graph/nodes/4:abc123def:0?user=jpmschweitzer`
+    **Example:** `/graph/nodes/4:abc123def:0?user=<tenant>`
     """
     try:
         node = await graph_service.get_node(node_id, user)
@@ -123,7 +127,7 @@ async def get_node(
 @router.post("/update-from-page/{page_id}", response_model=GraphUpdateSummary)
 async def update_graph_from_page(
     page_id: int,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     force_refresh: bool = Query(default=False, description="Force re-extraction"),
     graph_service: GraphService = Depends(get_graph_service),
     api_key: str = Depends(verify_api_key)
@@ -143,7 +147,7 @@ async def update_graph_from_page(
     - Called manually by user/Librarian to refresh graph
     - Called by Scheduler for batch processing
 
-    **Example:** `POST /graph/update-from-page/4?user=jpmschweitzer`
+    **Example:** `POST /graph/update-from-page/4?user=<tenant>`
 
     **Returns:** Summary with nodes/relationships created and entities extracted
     """
@@ -171,8 +175,8 @@ async def update_graph_from_page(
 
 @router.post("/mindmap", response_model=MindMapResponse)
 async def generate_mindmap(
+    user: RequiredUserQuery,
     center_node_id: str = Query(..., description="Central node ID"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     depth: int = Query(default=2, ge=1, le=5, description="Traversal depth"),
     graph_service: GraphService = Depends(get_graph_service),
     api_key: str = Depends(verify_api_key)
@@ -205,7 +209,7 @@ async def generate_mindmap(
 
 @router.post("/generate-entity-pages")
 async def generate_entity_pages(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     min_mentions: int = Query(default=5, ge=1, le=100, description="Minimum mentions threshold"),
     entity_types: Optional[List[str]] = Query(default=None, description="Entity types to process"),
     graph_service: GraphService = Depends(get_graph_service),

src/routers/hybrid_rag.py

@@ -5,58 +5,24 @@ Provides endpoint for combining vector, graph, volatile cache, and web search
 with RRF fusion and LLM re-ranking.
 """
 
-from fastapi import APIRouter, HTTPException, Depends, Query
+from fastapi import APIRouter, HTTPException, Depends
 import logging
 
 from src.models.hybrid_rag import HybridRAGRequest, HybridRAGResponse
 from src.services.hybrid_rag_service import HybridRAGService
 from src.core.dependencies import (
-    Neo4jDep, WikiJSDep, QdrantDep, OllamaDep,
-    SearXNGDep, ContentExtractorDep, verify_api_key, get_settings
+    verify_api_key, get_hybrid_rag_service, RequiredUserQuery
 )
-from src.config import Settings
 
 logger = logging.getLogger(__name__)
 
 router = APIRouter(prefix="/query", tags=["HybridRAG"])
 
 
-# Dependency to get HybridRAG service
-def get_hybrid_rag_service(
-    neo4j_client: Neo4jDep,
-    wiki_client: WikiJSDep,
-    qdrant_client: QdrantDep,
-    ollama_client: OllamaDep,
-    searxng_client: SearXNGDep,
-    content_extractor: ContentExtractorDep,
-    settings: Settings = Depends(get_settings)
-) -> HybridRAGService:
-    """Get HybridRAG service instance with all dependencies."""
-    from src.services.vector_service import VectorService
-    from src.services.graph_service import GraphService
-    from src.services.volatile_service import VolatileCacheService
-
-    # Create component services
-    vector_service = VectorService(qdrant_client, wiki_client, ollama_client)
-    graph_service = GraphService(neo4j_client, wiki_client)
-    volatile_service = VolatileCacheService(qdrant_client, ollama_client, settings)
-
-    # Create HybridRAG service
-    return HybridRAGService(
-        vector_service=vector_service,
-        graph_service=graph_service,
-        searxng_client=searxng_client,
-        ollama_client=ollama_client,
-        content_extractor=content_extractor,
-        settings=settings,
-        volatile_service=volatile_service
-    )
-
-
 @router.post("/hybrid", response_model=HybridRAGResponse)
 async def hybrid_search(
     request: HybridRAGRequest,
-    user: str = Query(default="jpmschweitzer", description="User identifier for multi-tenancy"),
+    user: RequiredUserQuery,
     hybrid_rag_service: HybridRAGService = Depends(get_hybrid_rag_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -72,11 +38,13 @@ async def hybrid_search(
     6. **Context Formatting**: Format for LLM consumption
     7. **Persistence**: Store for Librarian knowledge consolidation
 
-    **Example Request:**
+    **Multi-tenancy:** the `user` query parameter is REQUIRED — all retrieval
+    legs and persistence are scoped to that tenant's namespaces.
+
+    **Example Request** (`POST /query/hybrid?user=<tenant>`):
     ```json
     {
         "query": "What's the weather in Rotterdam?",
-        "user": "jpmschweitzer",
         "config": {
             "vector_limit": 10,
             "graph_limit": 10,

src/routers/ingestion.py

@@ -5,6 +5,7 @@ Endpoints for ingesting wiki pages into the knowledge base (vectors + graph).
 """
 from fastapi import APIRouter, Depends, HTTPException, Query
 from typing import Optional
+import logging
 
 from src.services.ingestion_service import IngestionService
 from src.models.ingestion import (
@@ -13,16 +14,59 @@ from src.models.ingestion import (
     BatchIngestionRequest,
     BatchIngestionResult
 )
-from src.core.dependencies import get_ingestion_service, verify_api_key
+from src.core.dependencies import (
+    get_ingestion_service, verify_api_key, verify_browser_request,
+    RequiredUserQuery, JobManagerDep
+)
+from src.jobs.job_manager import JobManager, JobStatus, JobType
+
+logger = logging.getLogger(__name__)
 
 router = APIRouter(prefix="/ingest", tags=["Document Ingestion"])
 
 
+async def _track_job(
+    job_manager: JobManager,
+    job_type: JobType,
+    user: str,
+    parameters: dict
+) -> Optional[str]:
+    """Create a Redis job record; never fail the request over job tracking."""
+    try:
+        return await job_manager.create_job(job_type, user, parameters)
+    except Exception as e:
+        logger.warning(f"Job tracking unavailable ({job_type.value}): {e}")
+        return None
+
+
+async def _finish_job(
+    job_manager: JobManager,
+    job_id: Optional[str],
+    success: bool,
+    result: dict,
+    error: Optional[str] = None
+) -> None:
+    """Mark a tracked job completed/failed; never fail the request."""
+    if not job_id:
+        return
+    try:
+        await job_manager.update_job_status(
+            job_id,
+            JobStatus.COMPLETED if success else JobStatus.FAILED,
+            progress=100,
+            result=result,
+            error=error
+        )
+    except Exception as e:
+        logger.warning(f"Job tracking update failed for {job_id}: {e}")
+
+
 @router.post("/page", response_model=IngestionResult)
 async def ingest_page(
     request: IngestionRequest,
     ingestion: IngestionService = Depends(get_ingestion_service),
-    api_key: str = Depends(verify_api_key)
+    job_manager: JobManagerDep = None,
+    actor: str = Depends(verify_browser_request)
 ):
     """
     Ingest a single wiki page into the knowledge base.
@@ -52,11 +96,16 @@ async def ingest_page(
       -H "Content-Type: application/json" \
       -d '{
         "page_id": 19,
-        "user": "jpmschweitzer",
+        "user": "<tenant>",
         "force_refresh": false
       }'
     ```
     """
+    job_id = await _track_job(
+        job_manager, JobType.DOCUMENT_INGESTION, request.user,
+        {"page_id": request.page_id, "force_refresh": request.force_refresh}
+    )
+
     result = await ingestion.ingest_page(
         page_id=request.page_id,
         user=request.user,
@@ -64,6 +113,12 @@ async def ingest_page(
         skip_vectors=request.skip_vectors,
         skip_graph=request.skip_graph
     )
+    result.job_id = job_id
+
+    await _finish_job(
+        job_manager, job_id, result.success,
+        result=result.model_dump(mode="json"), error=result.error
+    )
 
     if not result.success:
         raise HTTPException(
@@ -78,6 +133,7 @@ async def ingest_page(
 async def ingest_batch(
     request: BatchIngestionRequest,
     ingestion: IngestionService = Depends(get_ingestion_service),
+    job_manager: JobManagerDep = None,
     api_key: str = Depends(verify_api_key)
 ):
     """
@@ -104,11 +160,16 @@ async def ingest_batch(
       -H "Content-Type: application/json" \
       -d '{
         "page_ids": [19, 20, 21, 22],
-        "user": "jpmschweitzer",
+        "user": "<tenant>",
         "max_concurrent": 3
       }'
     ```
     """
+    job_id = await _track_job(
+        job_manager, JobType.BATCH_INGESTION, request.user,
+        {"page_ids": request.page_ids, "force_refresh": request.force_refresh}
+    )
+
     result = await ingestion.ingest_batch(
         page_ids=request.page_ids,
         user=request.user,
@@ -117,17 +178,28 @@ async def ingest_batch(
         skip_graph=request.skip_graph,
         max_concurrent=request.max_concurrent
     )
+    result.job_id = job_id
+
+    await _finish_job(
+        job_manager, job_id, result.failed == 0,
+        result={
+            "total_pages": result.total_pages,
+            "successful": result.successful,
+            "failed": result.failed
+        }
+    )
 
     return result
 
 
 @router.post("/all", response_model=BatchIngestionResult)
 async def ingest_all_pages(
-    user: str = Query(default="jpmschweitzer", description="User identifier"),
-    path_prefix: Optional[str] = Query(None, description="Path prefix filter (e.g., 'users/jpmschweitzer/tech')"),
+    user: RequiredUserQuery,
+    path_prefix: Optional[str] = Query(None, description="Path prefix filter within the user's namespace (e.g., 'users/<tenant>/tech')"),
     force_refresh: bool = Query(False, description="Force re-ingestion of all pages"),
     max_concurrent: int = Query(3, ge=1, le=10, description="Maximum concurrent ingestion tasks"),
     ingestion: IngestionService = Depends(get_ingestion_service),
+    job_manager: JobManagerDep = None,
     api_key: str = Depends(verify_api_key)
 ):
     """
@@ -151,19 +223,38 @@ async def ingest_all_pages(
 
     ```bash
     # Ingest all pages for user
-    curl -X POST "http://192.168.86.149:8089/ingest/all?user=jpmschweitzer" \
+    curl -X POST "http://192.168.86.149:8089/ingest/all?user=<tenant>" \
       -H "Authorization: Bearer $API_KEY"
 
     # Ingest only tech docs
-    curl -X POST "http://192.168.86.149:8089/ingest/all?user=jpmschweitzer&path_prefix=users/jpmschweitzer/tech" \
+    curl -X POST "http://192.168.86.149:8089/ingest/all?user=<tenant>&path_prefix=users/<tenant>/tech" \
       -H "Authorization: Bearer $API_KEY"
     ```
     """
-    result = await ingestion.ingest_all_pages(
-        user=user,
-        path_prefix=path_prefix,
-        force_refresh=force_refresh,
-        max_concurrent=max_concurrent
+    job_id = await _track_job(
+        job_manager, JobType.BATCH_INGESTION, user,
+        {"path_prefix": path_prefix, "force_refresh": force_refresh, "scope": "all"}
+    )
+
+    try:
+        result = await ingestion.ingest_all_pages(
+            user=user,
+            path_prefix=path_prefix,
+            force_refresh=force_refresh,
+            max_concurrent=max_concurrent
+        )
+    except ValueError as e:
+        await _finish_job(job_manager, job_id, False, result={}, error=str(e))
+        raise HTTPException(status_code=400, detail=str(e))
+
+    result.job_id = job_id
+    await _finish_job(
+        job_manager, job_id, result.failed == 0,
+        result={
+            "total_pages": result.total_pages,
+            "successful": result.successful,
+            "failed": result.failed
+        }
     )
 
     return result

src/routers/maintenance.py

@@ -21,9 +21,9 @@ from src.core.dependencies import (
     VectorServiceDep, GraphServiceDep, WikiJSDep, RedisDep,
     QdrantDep, OllamaDep, PaperlessDep, verify_api_key
 )
+from src.core.multi_tenancy import RequiredUser, sanitize_user_id
 from src.config import get_settings
-from src.core.multi_tenancy import DEFAULT_USER
-from datetime import datetime, timezone
+from datetime import datetime, timedelta, timezone
 
 logger = logging.getLogger(__name__)
 
@@ -262,6 +262,14 @@ def _matches_test_user_path(path: str) -> bool:
     return any(path_lower.startswith(prefix) for prefix in TEST_USER_PATH_PREFIXES)
 
 
+def _tenant_from_path(path: str) -> str:
+    """Extract the tenant user from a 'users/{tenant}/...' wiki path."""
+    parts = path.lstrip("/").split("/")
+    if len(parts) >= 2 and parts[0] == "users":
+        return parts[1]
+    raise ValueError(f"Cannot derive tenant from path: {path}")
+
+
 # ========== Endpoints ==========
 
 @router.post("/cleanup/vectors", response_model=VectorCleanupResponse)
@@ -652,12 +660,14 @@ async def cleanup_test_data(
                 page_path = page["path"]
 
                 try:
-                    # Delete vector chunks for this page (using DEFAULT_USER collection)
-                    chunks_removed = await vector_service.delete_page_chunks(page_id, DEFAULT_USER)
+                    # Delete vector chunks/graph node scoped to the tenant that
+                    # owns the page (derived from its users/{tenant}/ path)
+                    tenant = _tenant_from_path(page_path)
+                    chunks_removed = await vector_service.delete_page_chunks(page_id, tenant)
                     vector_deleted += chunks_removed
 
                     # Delete graph node for this page (returns count, may be 0 if no node)
-                    graph_removed = await graph_service.delete_page(page_id, DEFAULT_USER)
+                    graph_removed = await graph_service.delete_page(page_id, tenant)
                     graph_deleted += graph_removed
 
                     # Delete wiki page (raises exception on failure, returns None on success)
@@ -1089,3 +1099,605 @@ async def reconcile_index(
     except Exception as e:
         logger.error(f"Reconcile-index failed: {e}", exc_info=True)
         raise HTTPException(status_code=500, detail=str(e))
+
+
+# ========== Integrity check (nightly, read-only) ==========
+
+# Redis key holding the latest integrity report per tenant (folded into the
+# weekly quality report).
+INTEGRITY_LATEST_KEY = "library:integrity:latest:{user}"
+INTEGRITY_LATEST_TTL = 86400 * 30  # 30 days
+
+#: Qdrant collection prefixes owned by library-desk.
+LIBRARY_COLLECTION_PREFIXES = ("library_desk_", "volatile_")
+
+
+def _looks_like_test_tenant(name: str) -> bool:
+    """Heuristic for test/probe residue in collection or tenant names."""
+    lowered = name.lower()
+    return (
+        "llm_tester" in lowered
+        or "llm-tester" in lowered
+        or "test" in lowered
+        or lowered.startswith("verify_probe")
+        or lowered.startswith("verify-probe")
+    )
+
+
+def classify_collection(name: str, known_tenants: set[str]) -> str:
+    """
+    Classify a Qdrant collection against known tenant patterns.
+
+    Returns one of:
+    - ``expected``: library-desk collection for a tenant with a wiki namespace
+    - ``test_residue``: library-desk collection for a test/probe tenant
+    - ``unknown_tenant``: library-desk collection for a tenant with no wiki
+      namespace (orphaned or mis-scoped)
+    - ``foreign_test_residue``: another service's collection that looks like
+      test residue (reported, but owned elsewhere)
+    - ``foreign``: another service's collection (informational only)
+    """
+    for prefix in LIBRARY_COLLECTION_PREFIXES:
+        if name.startswith(prefix):
+            tenant = name[len(prefix):]
+            if _looks_like_test_tenant(tenant):
+                return "test_residue"
+            if tenant in known_tenants:
+                return "expected"
+            return "unknown_tenant"
+    if _looks_like_test_tenant(name):
+        return "foreign_test_residue"
+    return "foreign"
+
+
+class IntegrityCheckRequest(BaseModel):
+    """Request body for /maintenance/integrity-check."""
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — the report is scoped to this tenant."
+    )
+
+
+class IntegrityCheckResponse(BaseModel):
+    """Read-only integrity report for one tenant."""
+    success: bool
+    user: str
+    generated_at: str
+    pages_without_vectors: List[Dict[str, Any]] = Field(
+        default_factory=list,
+        description="Wiki pages with ZERO vectors in Qdrant (silent-skip reindex victims)"
+    )
+    orphaned_vector_chunks: int = Field(
+        default=0, description="Vector chunks whose wiki page no longer exists"
+    )
+    orphaned_vector_page_ids: List[int] = Field(
+        default_factory=list, description="Distinct stale page ids referenced by orphaned chunks"
+    )
+    unexpected_collections: List[Dict[str, str]] = Field(
+        default_factory=list,
+        description="Qdrant collections flagged as test residue or unknown tenants"
+    )
+    foreign_collections: int = Field(
+        default=0, description="Collections owned by other services (informational)"
+    )
+    documents_without_wiki: List[Dict[str, Any]] = Field(
+        default_factory=list,
+        description="Neo4j Document nodes whose wiki page no longer exists"
+    )
+    counts: Dict[str, int] = Field(default_factory=dict)
+    duration_ms: float = 0.0
+
+
+async def run_integrity_check(
+    user: str,
+    vector_service: VectorService,
+    graph_service: GraphService,
+    wiki_client,
+    qdrant
+) -> IntegrityCheckResponse:
+    """
+    Run the read-only integrity check for one tenant.
+
+    Reports (never fixes):
+    1. Wiki pages with zero vectors in the tenant's Qdrant collection
+    2. Orphaned vectors whose wiki page no longer exists
+    3. Unexpected Qdrant collections (test residue / unknown tenants)
+    4. Neo4j Document nodes without wiki counterparts
+    """
+    start_time = time.time()
+    tenant_prefix = f"users/{sanitize_user_id(user)}"
+
+    # One unfiltered listing serves both the tenant scan and the
+    # known-tenant derivation for collection classification.
+    all_pages = await wiki_client.list_all_pages()
+    tenant_pages = [
+        p for p in all_pages
+        if ("/" + str(p.get("path", "")).lstrip("/")).startswith("/" + tenant_prefix)
+    ]
+
+    known_tenants = set()
+    for p in all_pages:
+        parts = str(p.get("path", "")).lstrip("/").split("/")
+        if len(parts) >= 2 and parts[0] == "users":
+            known_tenants.add(sanitize_user_id(parts[1]))
+
+    tenant_page_ids = {p["id"] for p in tenant_pages if p.get("id")}
+
+    # Vector side (tenant collection only)
+    chunk_refs = await vector_service.get_all_chunk_references(user)
+    wiki_chunk_refs = [r for r in chunk_refs if r.get("doc_type", "wiki") == "wiki"]
+    vectorized_page_ids = {r["page_id"] for r in wiki_chunk_refs if r.get("page_id")}
+
+    pages_without_vectors = [
+        {"page_id": p["id"], "path": p.get("path", ""), "title": p.get("title", "")}
+        for p in tenant_pages
+        if p.get("id") and p["id"] not in vectorized_page_ids
+    ]
+
+    orphaned_chunks = [
+        r for r in wiki_chunk_refs
+        if r.get("page_id") and r["page_id"] not in tenant_page_ids
+    ]
+    orphaned_page_ids = sorted({r["page_id"] for r in orphaned_chunks})
+
+    # Collection audit (global listing, read-only)
+    collections = await qdrant.list_collections()
+    unexpected = []
+    foreign_count = 0
+    for coll in collections:
+        category = classify_collection(coll["name"], known_tenants)
+        if category in ("test_residue", "unknown_tenant", "foreign_test_residue"):
+            unexpected.append({"name": coll["name"], "category": category})
+        elif category == "foreign":
+            foreign_count += 1
+
+    # Graph side (tenant labels only)
+    graph_docs = await graph_service.get_all_document_references(user)
+    documents_without_wiki = [
+        {"page_id": d.get("page_id"), "path": d.get("path", ""), "title": d.get("title", "")}
+        for d in graph_docs
+        if d.get("doc_type") == "wiki"
+        and d.get("page_id")
+        and d["page_id"] not in tenant_page_ids
+    ]
+
+    duration_ms = (time.time() - start_time) * 1000
+
+    return IntegrityCheckResponse(
+        success=True,
+        user=user,
+        generated_at=datetime.now(timezone.utc).isoformat(),
+        pages_without_vectors=pages_without_vectors,
+        orphaned_vector_chunks=len(orphaned_chunks),
+        orphaned_vector_page_ids=orphaned_page_ids,
+        unexpected_collections=unexpected,
+        foreign_collections=foreign_count,
+        documents_without_wiki=documents_without_wiki,
+        counts={
+            "tenant_wiki_pages": len(tenant_pages),
+            "tenant_vector_chunks": len(wiki_chunk_refs),
+            "tenant_graph_documents": len(graph_docs),
+            "pages_without_vectors": len(pages_without_vectors),
+            "orphaned_vector_chunks": len(orphaned_chunks),
+            "unexpected_collections": len(unexpected),
+            "documents_without_wiki": len(documents_without_wiki),
+        },
+        duration_ms=duration_ms
+    )
+
+
+@router.post("/integrity-check", response_model=IntegrityCheckResponse)
+async def integrity_check(
+    request: IntegrityCheckRequest,
+    vector_service: VectorServiceDep = None,
+    graph_service: GraphServiceDep = None,
+    wiki_client: WikiJSDep = None,
+    qdrant: QdrantDep = None,
+    redis: RedisDep = None,
+    api_key: str = Depends(verify_api_key)
+):
+    """
+    Nightly integrity check (READ-ONLY: reports, never auto-fixes).
+
+    Reports per tenant:
+    - Wiki pages with ZERO vectors in Qdrant (silent-skip reindex victims)
+    - Orphaned vectors whose wiki page no longer exists
+    - Unexpected Qdrant collections: anything not matching known tenant
+      patterns — flags test-tenant residue and unknown namespaces
+    - Neo4j Document nodes without wiki counterparts
+    - Counts and duration
+
+    The latest report is cached in Redis (30 days) so the weekly quality
+    report can fold it in without re-running the scan.
+
+    **Scheduler Task** — nightly at 04:30, see docs/scheduler-tasks.md.
+    """
+    try:
+        report = await run_integrity_check(
+            user=request.user,
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant
+        )
+
+        # Cache the latest report for the quality report (best-effort)
+        if redis:
+            try:
+                import json as _json
+                await redis.setex(
+                    INTEGRITY_LATEST_KEY.format(user=request.user),
+                    INTEGRITY_LATEST_TTL,
+                    _json.dumps(report.model_dump(mode="json"))
+                )
+            except Exception as e:
+                logger.warning(f"Failed to cache integrity report: {e}")
+
+        logger.info(
+            f"Integrity check for {request.user}: {report.counts} "
+            f"in {report.duration_ms:.0f}ms"
+        )
+        return report
+
+    except Exception as e:
+        logger.error(f"Integrity check failed for {request.user}: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Integrity check failed")
+
+
+# ========== Weekly quality report ==========
+
+QUALITY_REPORT_PATH_TEMPLATE = "users/{tenant}/system/quality-reports/{date}"
+
+
+class QualityReportRequest(BaseModel):
+    """Request body for /maintenance/quality-report."""
+    user: RequiredUser = Field(
+        ...,
+        description="User identifier (tenant). Required — the report is scoped to this tenant."
+    )
+    stale_days: int = Field(
+        default=30, ge=1, le=365,
+        description="Pages not updated in this many days are stale candidates"
+    )
+    max_search_hits: int = Field(
+        default=1, ge=0, le=100,
+        description="A stale candidate is only flagged when its SearchQuery hit count is <= this"
+    )
+    dedup_threshold: float = Field(
+        default=0.9, ge=0.5, le=1.0,
+        description="Cosine similarity threshold for the duplicate scan"
+    )
+    write_page: bool = Field(
+        default=True,
+        description="Write the dated report page to the tenant's wiki (users/{user}/system/quality-reports/YYYY-MM-DD)"
+    )
+
+
+class QualityReportResponse(BaseModel):
+    """Weekly quality report for one tenant."""
+    success: bool
+    user: str
+    generated_at: str
+    page_path: Optional[str] = Field(
+        default=None, description="Wiki path of the written report page (None when write_page=false)"
+    )
+    page_id: Optional[int] = None
+    report: str = Field(description="Full markdown report content")
+    duplicate_groups: List[Dict[str, Any]] = Field(default_factory=list)
+    stale_pages: List[Dict[str, Any]] = Field(default_factory=list)
+    pages_missing_metadata: List[Dict[str, Any]] = Field(default_factory=list)
+    integrity: Optional[Dict[str, Any]] = Field(
+        default=None, description="Latest integrity-check result (cached or run inline)"
+    )
+    counts: Dict[str, int] = Field(default_factory=dict)
+    duration_ms: float = 0.0
+
+
+def _parse_wiki_timestamp(value: Any) -> Optional[datetime]:
+    """Parse a Wiki.js ISO timestamp ('...Z' or offset) to aware UTC."""
+    if not value:
+        return None
+    try:
+        parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))
+        if parsed.tzinfo is None:
+            parsed = parsed.replace(tzinfo=timezone.utc)
+        return parsed.astimezone(timezone.utc)
+    except ValueError:
+        return None
+
+
+async def _get_search_hit_counts(graph_service: GraphService, user: str) -> Dict[int, int]:
+    """
+    Per-page SearchQuery FOUND-hit counts from the tenant's graph data.
+
+    Returns {page_id: hits} for every tenant Document node.
+    """
+    from src.core.multi_tenancy import get_neo4j_user_base_label, get_neo4j_user_label
+
+    base_label = get_neo4j_user_base_label(user)
+    doc_label = get_neo4j_user_label(user)
+
+    query = f"""
+    MATCH (d:{doc_label}:Document)
+    WHERE d.page_id IS NOT NULL
+    OPTIONAL MATCH (sq:{base_label}_SearchQuery:SearchQuery)-[f:FOUND]->(d)
+    RETURN d.page_id AS page_id, count(f) AS hits
+    """
+    try:
+        rows = await graph_service.neo4j.execute_query(query, {})
+        return {row["page_id"]: row["hits"] for row in rows}
+    except Exception as e:
+        logger.warning(f"Failed to get search hit counts for {user}: {e}")
+        return {}
+
+
+def _render_quality_report(
+    user: str,
+    generated_at: str,
+    duplicate_scan: Dict[str, Any],
+    stale_pages: List[Dict[str, Any]],
+    missing_metadata: List[Dict[str, Any]],
+    integrity: Optional[Dict[str, Any]],
+    stale_days: int,
+    max_search_hits: int,
+    dedup_threshold: float,
+) -> str:
+    """Render the markdown report page content."""
+    lines = [
+        f"Automated quality report for tenant `{user}`, generated {generated_at}.",
+        "",
+        "## Summary",
+        "",
+        "| Metric | Count |",
+        "|---|---|",
+        f"| Potential duplicate page pairs (cosine ≥ {dedup_threshold}) | {len(duplicate_scan.get('duplicate_groups', []))} |",
+        f"| Stale pages (> {stale_days}d old, ≤ {max_search_hits} search hits) | {len(stale_pages)} |",
+        f"| Pages missing tags/description | {len(missing_metadata)} |",
+    ]
+    if integrity:
+        counts = integrity.get("counts", {})
+        lines += [
+            f"| Pages without vectors (integrity) | {counts.get('pages_without_vectors', 0)} |",
+            f"| Orphaned vector chunks (integrity) | {counts.get('orphaned_vector_chunks', 0)} |",
+            f"| Documents without wiki page (integrity) | {counts.get('documents_without_wiki', 0)} |",
+            f"| Unexpected Qdrant collections (integrity) | {counts.get('unexpected_collections', 0)} |",
+        ]
+
+    lines += ["", "## Potential duplicates", ""]
+    groups = duplicate_scan.get("duplicate_groups", [])
+    if groups:
+        for g in groups:
+            pages = g.get("pages", [])
+            refs = " ↔ ".join(f"`{p.get('path')}` ({p.get('title')})" for p in pages)
+            lines.append(
+                f"- {refs} — similarity {g.get('max_similarity', 0):.3f}, "
+                f"{g.get('matching_chunk_pairs', 0)} matching chunk pair(s)"
+            )
+    else:
+        lines.append("_None found._")
+
+    lines += ["", f"## Stale pages (not updated in {stale_days} days, ≤ {max_search_hits} search hits)", ""]
+    if stale_pages:
+        for p in stale_pages:
+            lines.append(
+                f"- `{p['path']}` ({p['title']}) — last updated {p['updated_at']}, "
+                f"{p['search_hits']} search hit(s)"
+            )
+    else:
+        lines.append("_None found._")
+
+    lines += ["", "## Pages missing metadata", ""]
+    if missing_metadata:
+        for p in missing_metadata:
+            lines.append(f"- `{p['path']}` ({p['title']}) — missing: {', '.join(p['missing'])}")
+    else:
+        lines.append("_None found._")
+
+    lines += ["", "## Integrity check", ""]
+    if integrity:
+        lines.append(f"Source: {integrity.get('source', 'unknown')} (generated {integrity.get('generated_at', '?')})")
+        lines.append("")
+        for key, value in integrity.get("counts", {}).items():
+            lines.append(f"- {key}: {value}")
+        unexpected = integrity.get("unexpected_collections", [])
+        if unexpected:
+            lines.append("")
+            lines.append("Unexpected Qdrant collections:")
+            for c in unexpected:
+                lines.append(f"- `{c.get('name')}` ({c.get('category')})")
+    else:
+        lines.append("_No integrity data available._")
+
+    return "\n".join(lines) + "\n"
+
+
+@router.post("/quality-report", response_model=QualityReportResponse)
+async def quality_report(
+    request: QualityReportRequest,
+    vector_service: VectorServiceDep = None,
+    graph_service: GraphServiceDep = None,
+    wiki_client: WikiJSDep = None,
+    qdrant: QdrantDep = None,
+    redis: RedisDep = None,
+    api_key: str = Depends(verify_api_key)
+):
+    """
+    Weekly quality report for one tenant.
+
+    Runs the duplicate scan, flags stale pages (not updated in N days AND a
+    low SearchQuery hit count from the graph data), lists pages missing
+    tags/description, folds in the latest integrity-check results (cached in
+    Redis by /maintenance/integrity-check, or run inline when absent), and
+    writes a dated report page to the tenant's wiki under
+    `users/{user}/system/quality-reports/YYYY-MM-DD`.
+
+    **Scheduler Task** — weekly Sunday 03:00, see docs/scheduler-tasks.md.
+    """
+    import json as _json
+
+    start_time = time.time()
+    user = request.user
+    tenant = sanitize_user_id(user)
+    tenant_prefix = f"users/{tenant}"
+    system_prefix = f"{tenant_prefix}/system/"
+    now = datetime.now(timezone.utc)
+    generated_at = now.isoformat()
+
+    try:
+        # 1. Duplicate scan (tenant-scoped, read-only)
+        duplicate_scan = await vector_service.find_duplicate_pairs(
+            user=user,
+            similarity_threshold=request.dedup_threshold
+        )
+
+        # 2. Page inventory + search-hit counts
+        pages = await wiki_client.list_all_pages(path_prefix=tenant_prefix)
+        # The report subtree itself is exempt from quality checks
+        pages = [
+            p for p in pages
+            if not str(p.get("path", "")).lstrip("/").startswith(system_prefix)
+        ]
+        hit_counts = await _get_search_hit_counts(graph_service, user)
+
+        stale_cutoff = now - timedelta(days=request.stale_days)
+        stale_pages = []
+        missing_metadata = []
+
+        for p in pages:
+            path = p.get("path", "")
+            title = p.get("title", "")
+            updated_at = _parse_wiki_timestamp(p.get("updatedAt"))
+            hits = hit_counts.get(p.get("id"), 0)
+
+            if updated_at and updated_at < stale_cutoff and hits <= request.max_search_hits:
+                stale_pages.append({
+                    "page_id": p.get("id"),
+                    "path": path,
+                    "title": title,
+                    "updated_at": updated_at.date().isoformat(),
+                    "search_hits": hits,
+                })
+
+            missing = []
+            if not p.get("tags"):
+                missing.append("tags")
+            if not (p.get("description") or "").strip():
+                missing.append("description")
+            if missing:
+                missing_metadata.append({
+                    "page_id": p.get("id"),
+                    "path": path,
+                    "title": title,
+                    "missing": missing,
+                })
+
+        # 3. Integrity results: latest cached report, or run inline
+        integrity: Optional[Dict[str, Any]] = None
+        if redis:
+            try:
+                cached = await redis.get(INTEGRITY_LATEST_KEY.format(user=user))
+                if cached:
+                    integrity = _json.loads(cached)
+                    integrity["source"] = "cached"
+            except Exception as e:
+                logger.warning(f"Failed to read cached integrity report: {e}")
+        if integrity is None:
+            inline = await run_integrity_check(
+                user=user,
+                vector_service=vector_service,
+                graph_service=graph_service,
+                wiki_client=wiki_client,
+                qdrant=qdrant
+            )
+            integrity = inline.model_dump(mode="json")
+            integrity["source"] = "inline"
+
+        # 4. Render + write the dated report page
+        report_content = _render_quality_report(
+            user=user,
+            generated_at=generated_at,
+            duplicate_scan=duplicate_scan,
+            stale_pages=stale_pages,
+            missing_metadata=missing_metadata,
+            integrity=integrity,
+            stale_days=request.stale_days,
+            max_search_hits=request.max_search_hits,
+            dedup_threshold=request.dedup_threshold,
+        )
+
+        page_path = None
+        page_id = None
+        if request.write_page:
+            date_str = now.date().isoformat()
+            page_path = QUALITY_REPORT_PATH_TEMPLATE.format(tenant=tenant, date=date_str)
+            title = f"Quality Report {date_str}"
+
+            # Same-day reruns must UPDATE the existing page. The Wiki.js page
+            # listing updates asynchronously after creation, so the page id
+            # written today is remembered in Redis and used directly.
+            page_id_key = f"library:quality_report:page:{tenant}:{date_str}"
+            if redis:
+                try:
+                    cached_id = await redis.get(page_id_key)
+                    if cached_id:
+                        page_id = int(cached_id)
+                except Exception as e:
+                    logger.warning(f"Failed to read quality-report page id: {e}")
+
+            if page_id is None:
+                existing = await wiki_client.list_all_pages(path_prefix=page_path)
+                exact = [
+                    p for p in existing
+                    if str(p.get("path", "")).lstrip("/") == page_path
+                ]
+                if exact:
+                    page_id = exact[0]["id"]
+
+            if page_id is not None:
+                await wiki_client.update_page(page_id=page_id, content=report_content)
+                logger.info(f"Updated quality report page {page_path} (id={page_id})")
+            else:
+                created = await wiki_client.create_page(
+                    path=page_path,
+                    title=title,
+                    content=report_content,
+                    description=f"Automated weekly quality report for {user}",
+                    tags=["quality-report", "auto-generated"],
+                    is_published=True,
+                )
+                page_id = created.get("id") if created else None
+                logger.info(f"Created quality report page {page_path} (id={page_id})")
+
+            if redis and page_id:
+                try:
+                    await redis.setex(page_id_key, 86400 * 2, str(page_id))
+                except Exception as e:
+                    logger.warning(f"Failed to cache quality-report page id: {e}")
+
+        duration_ms = (time.time() - start_time) * 1000
+        counts = {
+            "duplicate_groups": len(duplicate_scan.get("duplicate_groups", [])),
+            "stale_pages": len(stale_pages),
+            "pages_missing_metadata": len(missing_metadata),
+            "pages_checked": len(pages),
+        }
+        logger.info(f"Quality report for {user}: {counts} in {duration_ms:.0f}ms")
+
+        return QualityReportResponse(
+            success=True,
+            user=user,
+            generated_at=generated_at,
+            page_path=page_path,
+            page_id=page_id,
+            report=report_content,
+            duplicate_groups=duplicate_scan.get("duplicate_groups", []),
+            stale_pages=stale_pages,
+            pages_missing_metadata=missing_metadata,
+            integrity=integrity,
+            counts=counts,
+            duration_ms=duration_ms
+        )
+
+    except Exception as e:
+        logger.error(f"Quality report failed for {user}: {e}", exc_info=True)
+        raise HTTPException(status_code=500, detail="Quality report failed")

src/routers/tools.py

@@ -25,10 +25,9 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer",
-                    example="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True,
+                    example="llm_tester"
                 ),
                 ToolParameter(
                     name="tag",
@@ -66,9 +65,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier for access control",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant) for access control. Required",
+                    required=True
                 ),
             ],
             returns="Complete page object with content",
@@ -119,9 +117,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
             ],
             returns="Created page object",
@@ -130,7 +127,7 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 "path": "/projects/my-project",
                 "content": "# My Project\n\nProject description here.",
                 "tags": ["projects"],
-                "user": "jpmschweitzer"
+                "user": "<tenant>"
             },
             fast=True
         ),
@@ -175,9 +172,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
             ],
             returns="Updated page object",
@@ -200,9 +196,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
             ],
             returns="Success confirmation",
@@ -225,9 +220,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
                 ToolParameter(
                     name="limit",
@@ -250,9 +244,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
             ],
             returns="List of dossiers with page counts",
@@ -275,9 +268,8 @@ def get_wiki_tools() -> list[ToolDefinition]:
                 ToolParameter(
                     name="user",
                     type=ParameterType.STRING,
-                    description="User identifier",
-                    required=False,
-                    default="jpmschweitzer"
+                    description="User identifier (tenant). Required - all operations are scoped to this tenant's namespace",
+                    required=True
                 ),
                 ToolParameter(
                     name="limit",

src/routers/vector.py

@@ -18,8 +18,9 @@ from src.services.vector_service import VectorService
 from src.clients.qdrant_client import QdrantClientWrapper
 from src.clients.wikijs_client import WikiJSClient
 from src.clients.ollama_client import OllamaClient
-from src.core.dependencies import QdrantDep, WikiJSDep, OllamaDep, verify_api_key
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.dependencies import (
+    QdrantDep, WikiJSDep, OllamaDep, verify_api_key, RequiredUserQuery
+)
 
 logger = logging.getLogger(__name__)
 
@@ -52,7 +53,7 @@ async def semantic_search(
     ```json
     {
         "query": "how to configure docker",
-        "user": "jpmschweitzer",
+        "user": "<tenant>",
         "limit": 10,
         "score_threshold": 0.5
     }
@@ -77,7 +78,7 @@ async def semantic_search(
 @router.post("/update-from-page/{page_id}", response_model=VectorUpdateSummary)
 async def update_vectors_from_page(
     page_id: int,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     force_refresh: bool = Query(default=False, description="Force re-embedding"),
     vector_service: VectorService = Depends(get_vector_service),
     api_key: str = Depends(verify_api_key)
@@ -96,7 +97,7 @@ async def update_vectors_from_page(
     - Called manually by user/Librarian to refresh vectors
     - Called by Scheduler for batch processing
 
-    **Example:** `POST /vector/update-from-page/5?user=jpmschweitzer`
+    **Example:** `POST /vector/update-from-page/5?user=<tenant> (user is REQUIRED)`
 
     **Returns:** Summary with chunks created and processing time
     """
@@ -125,7 +126,7 @@ async def update_vectors_from_page(
 @router.delete("/pages/{page_id}", response_model=DeletePageChunksResponse)
 async def delete_page_chunks(
     page_id: int,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     vector_service: VectorService = Depends(get_vector_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -134,7 +135,7 @@ async def delete_page_chunks(
 
     This is automatically called when a page is deleted from the wiki.
 
-    **Example:** `DELETE /vector/pages/5?user=jpmschweitzer`
+    **Example:** `DELETE /vector/pages/5?user=<tenant> (user is REQUIRED)`
     """
     try:
         deleted_count = await vector_service.delete_page_chunks(

src/routers/volatile.py

@@ -28,7 +28,7 @@ from src.core.dependencies import (
     get_news_provider,
     get_alphavantage_provider,
 )
-from src.core.multi_tenancy import DEFAULT_USER
+from src.core.dependencies import RequiredUserQuery
 from src.config import get_settings
 
 logger = logging.getLogger(__name__)
@@ -48,7 +48,7 @@ def get_volatile_service(qdrant: QdrantDep, ollama: OllamaDep) -> VolatileCacheS
 
 @router.get("/stats", response_model=VolatileStatsResponse)
 async def get_stats(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -72,7 +72,7 @@ async def get_stats(
 
 @router.get("/scheduled", response_model=VolatileScheduledResponse)
 async def get_scheduled(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -134,8 +134,8 @@ def _get_namespace_description(ns: VolatileNamespace) -> str:
 
 @router.get("/search")
 async def search_volatile(
+    user: RequiredUserQuery,
     q: str = Query(..., min_length=1, description="Search query"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     limit: int = Query(default=5, ge=1, le=20, description="Maximum results"),
     threshold: float = Query(default=0.75, ge=0.5, le=1.0, description="Minimum similarity score"),
     qdrant: QdrantDep = None,
@@ -166,10 +166,10 @@ async def search_volatile(
 
 @router.post("/store", response_model=VolatileRecordResponse)
 async def store_volatile(
+    user: RequiredUserQuery,
     namespace: str = Query(..., description="Data namespace (weather, news, etc.)"),
     key: str = Query(..., description="Record key (e.g., 'rotterdam', 'nos-headlines')"),
     request: VolatileRecordCreate = None,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -232,21 +232,21 @@ async def store_volatile(
 @router.post("/fetch/weather/{city}")
 async def fetch_weather(
     city: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
-    ttl: int = Query(default=86400, ge=60, le=604800, description="TTL in seconds"),
+    user: RequiredUserQuery,
+    ttl: int = Query(default=7200, ge=60, le=86400, description="TTL in seconds (default 2 hours)"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
 ):
     """
-    Fetch current weather for a city and store in volatile cache.
+    Fetch current weather conditions for a city and store in volatile cache.
 
-    Called by scheduler for prefetch or on-demand. Geocodes city name
-    and fetches weather from Open-Meteo API.
+    Stores temperature, humidity, wind, UV index. For forecasts use /fetch/forecast.
+    Called by scheduler for hourly prefetch or on-demand.
 
     **Example:**
     ```
-    POST /volatile/fetch/weather/amsterdam?user=jpmschweitzer
+    POST /volatile/fetch/weather/amsterdam?user=<tenant>
     ```
     """
     volatile_service = get_volatile_service(qdrant, ollama)
@@ -257,7 +257,49 @@ async def fetch_weather(
         weather_provider=weather_provider,
     )
 
-    result = await fetch_service.fetch_weather(user, city, ttl=ttl)
+    result = await fetch_service.fetch_current_weather(user, city, ttl=ttl)
+
+    if not result.success:
+        raise HTTPException(status_code=500, detail=result.error)
+
+    return {
+        "success": True,
+        "namespace": result.namespace,
+        "key": result.key,
+        "record": result.record,
+    }
+
+
+@router.post("/fetch/forecast/{city}")
+async def fetch_forecast(
+    city: str,
+    user: RequiredUserQuery,
+    days: int = Query(default=7, ge=1, le=16, description="Forecast days (1-16)"),
+    ttl: int = Query(default=86400, ge=60, le=604800, description="TTL in seconds (default 24 hours)"),
+    qdrant: QdrantDep = None,
+    ollama: OllamaDep = None,
+    api_key: str = Depends(verify_api_key)
+):
+    """
+    Fetch weather forecast for a city and store in volatile cache.
+
+    Stores multi-day outlook with highs/lows, precipitation, UV.
+    For current conditions use /fetch/weather.
+
+    **Example:**
+    ```
+    POST /volatile/fetch/forecast/amsterdam?user=<tenant>&days=7
+    ```
+    """
+    volatile_service = get_volatile_service(qdrant, ollama)
+    weather_provider = get_weather_provider()
+
+    fetch_service = VolatileFetchService(
+        volatile_service=volatile_service,
+        weather_provider=weather_provider,
+    )
+
+    result = await fetch_service.fetch_forecast(user, city, days=days, ttl=ttl)
 
     if not result.success:
         raise HTTPException(status_code=500, detail=result.error)
@@ -272,8 +314,8 @@ async def fetch_weather(
 
 @router.post("/fetch/news/{category}")
 async def fetch_news(
+    user: RequiredUserQuery,
     category: str = "general",
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     limit: int = Query(default=10, ge=1, le=50, description="Max headlines"),
     ttl: int = Query(default=7200, ge=60, le=86400, description="TTL in seconds"),
     qdrant: QdrantDep = None,
@@ -288,7 +330,7 @@ async def fetch_news(
 
     **Example:**
     ```
-    POST /volatile/fetch/news/tech?user=jpmschweitzer&limit=15
+    POST /volatile/fetch/news/tech?user=<tenant>&limit=15
     ```
     """
     volatile_service = get_volatile_service(qdrant, ollama)
@@ -317,8 +359,8 @@ async def fetch_news(
 @router.post("/fetch/stock/{symbol}")
 async def fetch_stock(
     symbol: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
-    ttl: int = Query(default=300, ge=60, le=3600, description="TTL in seconds"),
+    user: RequiredUserQuery,
+    ttl: int = Query(default=600, ge=60, le=3600, description="TTL in seconds (default 10 min)"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -330,7 +372,7 @@ async def fetch_stock(
 
     **Example:**
     ```
-    POST /volatile/fetch/stock/AAPL?user=jpmschweitzer
+    POST /volatile/fetch/stock/AAPL?user=<tenant>
     ```
     """
     volatile_service = get_volatile_service(qdrant, ollama)
@@ -365,9 +407,9 @@ async def fetch_stock(
 @router.post("/fetch/crypto/{symbol}")
 async def fetch_crypto(
     symbol: str,
+    user: RequiredUserQuery,
     market: str = Query(default="USD", description="Market currency"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
-    ttl: int = Query(default=300, ge=60, le=3600, description="TTL in seconds"),
+    ttl: int = Query(default=600, ge=60, le=3600, description="TTL in seconds (default 10 min)"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -379,7 +421,7 @@ async def fetch_crypto(
 
     **Example:**
     ```
-    POST /volatile/fetch/crypto/BTC?market=EUR&user=jpmschweitzer
+    POST /volatile/fetch/crypto/BTC?market=EUR&user=<tenant>
     ```
     """
     volatile_service = get_volatile_service(qdrant, ollama)
@@ -414,8 +456,8 @@ async def fetch_crypto(
 @router.post("/fetch/sun/{city}")
 async def fetch_sun_times(
     city: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
-    ttl: int = Query(default=86400, ge=60, le=604800, description="TTL in seconds"),
+    user: RequiredUserQuery,
+    ttl: int = Query(default=172800, ge=60, le=604800, description="TTL in seconds (default 48 hours)"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -427,7 +469,7 @@ async def fetch_sun_times(
 
     **Example:**
     ```
-    POST /volatile/fetch/sun/rotterdam?user=jpmschweitzer
+    POST /volatile/fetch/sun/rotterdam?user=<tenant>
     ```
 
     **Response data includes:**
@@ -460,8 +502,8 @@ async def fetch_sun_times(
 @router.post("/fetch/air_quality/{city}")
 async def fetch_air_quality(
     city: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
-    ttl: int = Query(default=3600, ge=60, le=86400, description="TTL in seconds"),
+    user: RequiredUserQuery,
+    ttl: int = Query(default=7200, ge=60, le=86400, description="TTL in seconds (default 2 hours)"),
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -473,7 +515,7 @@ async def fetch_air_quality(
 
     **Example:**
     ```
-    POST /volatile/fetch/air_quality/rotterdam?user=jpmschweitzer
+    POST /volatile/fetch/air_quality/rotterdam?user=<tenant>
     ```
 
     **Response data includes:**
@@ -503,11 +545,69 @@ async def fetch_air_quality(
     }
 
 
+@router.post("/fetch/environment/{city}")
+async def fetch_environment(
+    city: str,
+    user: RequiredUserQuery,
+    weather_ttl: int = Query(default=7200, ge=60, le=86400, description="Weather TTL in seconds (default 2 hours)"),
+    air_quality_ttl: int = Query(default=7200, ge=60, le=86400, description="Air quality TTL in seconds (default 2 hours)"),
+    qdrant: QdrantDep = None,
+    ollama: OllamaDep = None,
+    api_key: str = Depends(verify_api_key)
+):
+    """
+    Fetch weather and air quality concurrently for a city.
+
+    Performs a single geocode lookup and fetches both weather and air quality
+    data in parallel, storing both in volatile cache. More efficient than
+    calling /fetch/weather and /fetch/air_quality separately.
+
+    **Example:**
+    ```
+    POST /volatile/fetch/environment/rotterdam?user=<tenant>
+    ```
+
+    **Response includes:**
+    - weather: Current conditions (temperature, humidity, wind, UV)
+    - air_quality: AQI indices, pollutants, pollen data
+    """
+    volatile_service = get_volatile_service(qdrant, ollama)
+    weather_provider = get_weather_provider()
+
+    fetch_service = VolatileFetchService(
+        volatile_service=volatile_service,
+        weather_provider=weather_provider,
+    )
+
+    result = await fetch_service.fetch_environment(
+        user, city, weather_ttl=weather_ttl, air_quality_ttl=air_quality_ttl
+    )
+
+    if not result.success:
+        raise HTTPException(status_code=500, detail="; ".join(result.errors))
+
+    return {
+        "success": True,
+        "key": result.key,
+        "weather": {
+            "success": result.weather.success if result.weather else False,
+            "record": result.weather.record if result.weather else None,
+            "error": result.weather.error if result.weather else None,
+        },
+        "air_quality": {
+            "success": result.air_quality.success if result.air_quality else False,
+            "record": result.air_quality.record if result.air_quality else None,
+            "error": result.air_quality.error if result.air_quality else None,
+        },
+        "errors": result.errors,
+    }
+
+
 @router.get("/{namespace}/{key}", response_model=VolatileRecordResponse)
 async def get_record(
     namespace: str,
     key: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)
@@ -517,7 +617,7 @@ async def get_record(
 
     **Example:**
     ```
-    GET /volatile/weather/rotterdam?user=jpmschweitzer
+    GET /volatile/weather/rotterdam?user=<tenant>
     ```
     """
     service = get_volatile_service(qdrant, ollama)
@@ -536,7 +636,7 @@ async def get_record(
 async def delete_record(
     namespace: str,
     key: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     qdrant: QdrantDep = None,
     ollama: OllamaDep = None,
     api_key: str = Depends(verify_api_key)

src/routers/webhooks.py

@@ -320,7 +320,7 @@ async def process_page_rename(
         """
 
         try:
-            await neo4j.execute_query(update_query, {
+            await neo4j.execute_write(update_query, {
                 "page_id": page_id,
                 "new_path": new_path,
                 "new_title": new_title
@@ -441,7 +441,7 @@ async def cleanup_deleted_page(
         """
 
         try:
-            result = await neo4j.execute_query(delete_doc_query, {"page_id": page_id})
+            result = await neo4j.execute_write(delete_doc_query, {"page_id": page_id})
             deleted_count = result[0]["deleted_count"] if result else 0
             logger.info(f"Deleted {deleted_count} Document node(s) for page {page_id}")
         except Exception as e:
@@ -462,7 +462,7 @@ async def cleanup_deleted_page(
                 """
 
                 try:
-                    result = await neo4j.execute_query(delete_entity_query, {"entity_id": entity_id})
+                    result = await neo4j.execute_write(delete_entity_query, {"entity_id": entity_id})
                     deleted = result[0]["deleted_count"] if result else 0
                     if deleted > 0:
                         logger.info(f"Deleted orphaned entity: {entity_name}")
@@ -478,7 +478,7 @@ async def cleanup_deleted_page(
         """
 
         try:
-            result = await neo4j.execute_query(cleanup_search_query, {})
+            result = await neo4j.execute_write(cleanup_search_query, {})
             cleaned = result[0]["cleaned_count"] if result else 0
             if cleaned > 0:
                 logger.info(f"Cleaned up {cleaned} broken SearchQuery relationships")

src/routers/wiki.py

@@ -23,11 +23,10 @@ from src.clients.neo4j_client import Neo4jClient
 from src.clients.qdrant_client import QdrantClientWrapper
 from src.clients.ollama_client import OllamaClient
 from src.core.dependencies import (
-    WikiJSDep, Neo4jDep, QdrantDep, OllamaDep, SearXNGDep, ContentExtractorDep,
-    verify_api_key, get_settings, get_hybrid_rag_service, get_ingestion_service
+    WikiJSDep, Neo4jDep, QdrantDep, OllamaDep,
+    verify_api_key, get_settings, get_hybrid_rag_service, get_ingestion_service,
+    RequiredUserQuery
 )
-from src.core.multi_tenancy import DEFAULT_USER
-from src.services.hybrid_rag_service import HybridRAGService
 from src.services.wiki_page_writer import WikiPageWriter
 from src.services.entity_linking_utils import apply_bidirectional_entity_linking
 from src.config import Settings
@@ -62,7 +61,7 @@ def get_vector_service(
 # Page operations
 @router.get("/pages", response_model=WikiPageList)
 async def list_pages(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     tag: Optional[str] = Query(default=None, description="Filter by tag (dossier)"),
     limit: int = Query(default=50, ge=1, le=200, description="Maximum pages to return"),
     wiki_service: WikiService = Depends(get_wiki_service),
@@ -87,7 +86,7 @@ async def list_pages(
 @router.get("/pages/{page_id}", response_model=WikiPage)
 async def get_page(
     page_id: int,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     wiki_service: WikiService = Depends(get_wiki_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -139,14 +138,16 @@ async def create_page(
         "content": "# Architecture\\n\\nThis describes...",
         "description": "Architecture documentation",
         "tags": ["projects", "architecture"],
-        "user": "jpmschweitzer"
+        "user": "<tenant>"
     }
     ```
+
+    The `user` field is REQUIRED (no default tenant).
     """
     try:
         page = await wiki_service.create_page(page_data)
 
-        user = page_data.user or DEFAULT_USER
+        user = page_data.user
 
         # Schedule BOTH graph and vector updates in background (non-blocking)
         background_tasks.add_task(
@@ -178,8 +179,6 @@ async def smart_create_page(
     neo4j_client: Neo4jDep,
     qdrant_client: QdrantDep,
     ollama_client: OllamaDep,
-    searxng_client: SearXNGDep,
-    content_extractor: ContentExtractorDep,
     settings: Settings = Depends(get_settings),
     api_key: str = Depends(verify_api_key)
 ):
@@ -214,20 +213,15 @@ async def smart_create_page(
     - Entity linking statistics (forward/backward links)
     """
     try:
-        user = request.user or DEFAULT_USER
+        user = request.user
 
-        # Build services
+        # Build services. HybridRAG comes from the single wiring point in
+        # dependencies so it includes volatile_service (a previous inline
+        # copy here lacked it).
         wiki_service = WikiService(wiki_client)
         vector_service = VectorService(qdrant_client, wiki_client, ollama_client)
         graph_service = GraphService(neo4j_client, wiki_client)
-        hybrid_rag_service = HybridRAGService(
-            vector_service=vector_service,
-            graph_service=graph_service,
-            searxng_client=searxng_client,
-            ollama_client=ollama_client,
-            content_extractor=content_extractor,
-            settings=settings
-        )
+        hybrid_rag_service = get_hybrid_rag_service()
         wiki_page_writer = WikiPageWriter(ollama_client=ollama_client, settings=settings)
 
         # Step 1-5: Research + Generate + Create page
@@ -294,7 +288,7 @@ async def update_page(
     page_id: int,
     page_data: WikiPageUpdate,
     background_tasks: BackgroundTasks,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     wiki_service: WikiService = Depends(get_wiki_service),
     graph_service: GraphService = Depends(get_graph_service),
     vector_service: VectorService = Depends(get_vector_service),
@@ -353,7 +347,7 @@ async def update_page(
 async def delete_page(
     page_id: int,
     background_tasks: BackgroundTasks,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     wiki_service: WikiService = Depends(get_wiki_service),
     vector_service: VectorService = Depends(get_vector_service),
     graph_service: GraphService = Depends(get_graph_service),
@@ -402,7 +396,7 @@ async def delete_page(
 async def move_page(
     page_id: int,
     move_data: WikiPageMove,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     wiki_service: WikiService = Depends(get_wiki_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -445,8 +439,8 @@ async def move_page(
 # Search operations
 @router.get("/search", response_model=WikiSearchResponse)
 async def search_pages(
+    user: RequiredUserQuery,
     q: str = Query(..., min_length=1, description="Search query"),
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
     limit: int = Query(default=20, ge=1, le=100, description="Maximum results"),
     wiki_service: WikiService = Depends(get_wiki_service),
     api_key: str = Depends(verify_api_key)
@@ -487,7 +481,7 @@ async def search_pages(
 # Dossier operations
 @router.get("/dossiers", response_model=DossierList)
 async def list_dossiers(
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     wiki_service: WikiService = Depends(get_wiki_service),
     api_key: str = Depends(verify_api_key)
 ):
@@ -510,7 +504,7 @@ async def list_dossiers(
 @router.get("/dossiers/{dossier_name}/pages", response_model=WikiPageList)
 async def get_dossier_pages(
     dossier_name: str,
-    user: str = Query(default=DEFAULT_USER, description="User identifier"),
+    user: RequiredUserQuery,
     limit: int = Query(default=100, ge=1, le=500, description="Maximum pages"),
     wiki_service: WikiService = Depends(get_wiki_service),
     api_key: str = Depends(verify_api_key)

src/services/consolidation_service.py

@@ -13,7 +13,8 @@ This service:
 """
 import logging
 import json
-from datetime import datetime, timedelta
+import time
+from datetime import datetime, timedelta, timezone
 from typing import List, Dict, Any, Optional
 
 from src.clients.neo4j_client import Neo4jClient
@@ -32,6 +33,18 @@ from src.config import Settings
 logger = logging.getLogger(__name__)
 
 
+class ConsolidationLLMUnavailableError(Exception):
+    """
+    The generation LLM produced no output (infrastructure failure).
+
+    Raised instead of silently returning an empty classification so the
+    caller can leave the affected SearchQuery nodes UNPROCESSED for the
+    next run. Marking them processed on LLM failure permanently drains
+    the consolidation queue with zero output (the exact failure mode that
+    made every run log 'No unprocessed searches found' in production).
+    """
+
+
 class ConsolidationService:
     """
     Service for consolidating knowledge from search results.
@@ -77,7 +90,8 @@ class ConsolidationService:
         Returns:
             ConsolidationResponse with processing results
         """
-        logger.info(f"Starting knowledge consolidation")
+        run_start = time.time()
+        logger.info("Starting knowledge consolidation")
         logger.info(f"Limits: process={process_limit}, lookback={lookback_days}d, min_web={min_web_results}")
         if dry_run:
             logger.warning("DRY RUN MODE - will not create wiki pages")
@@ -86,7 +100,12 @@ class ConsolidationService:
         unprocessed = await self._find_unprocessed_searches(lookback_days, process_limit)
 
         if not unprocessed:
-            logger.info("No unprocessed searches found")
+            duration_ms = (time.time() - run_start) * 1000
+            logger.info(
+                f"Consolidation run complete: searches_processed=0 "
+                f"searches_deferred=0 duration_ms={duration_ms:.0f} "
+                f"(no unprocessed searches found)"
+            )
             return ConsolidationResponse(
                 total_found=0,
                 processed_count=0,
@@ -95,7 +114,8 @@ class ConsolidationService:
                 entities_added=0,
                 errors=[],
                 results=[],
-                dry_run=dry_run
+                dry_run=dry_run,
+                duration_ms=duration_ms
             )
 
         logger.info(f"Found {len(unprocessed)} unprocessed searches")
@@ -108,9 +128,10 @@ class ConsolidationService:
         total_volatile_cached = 0
         total_files_queued = 0
         total_prefetch_registered = 0
+        searches_deferred = 0
         errors: List[str] = []
 
-        for search in unprocessed:
+        for index, search in enumerate(unprocessed):
             try:
                 result = await self._process_search(
                     search=search,
@@ -132,6 +153,22 @@ class ConsolidationService:
                 if not dry_run:
                     await self._mark_search_processed(search['id'])
 
+            except ConsolidationLLMUnavailableError as e:
+                # Infrastructure failure: the generation LLM is unavailable.
+                # Do NOT consume the search - leave it (and the rest of this
+                # batch) unprocessed so the next run retries. Consuming
+                # searches here is what silently drained the queue in
+                # production ('No unprocessed searches found' with zero
+                # pages ever created).
+                searches_deferred = len(unprocessed) - index
+                error_msg = (
+                    f"Generation LLM unavailable ({e}); deferring "
+                    f"{searches_deferred} search(es) to the next run"
+                )
+                logger.error(error_msg)
+                errors.append(error_msg)
+                break
+
             except Exception as e:
                 error_msg = f"Search {search['id'][:8]}: {str(e)}"
                 logger.error(f"Failed to process search: {error_msg}", exc_info=True)
@@ -148,6 +185,7 @@ class ConsolidationService:
 
         # Build response
         processed_count = len([r for r in results if not r.error])
+        duration_ms = (time.time() - run_start) * 1000
 
         response = ConsolidationResponse(
             total_found=len(unprocessed),
@@ -158,13 +196,16 @@ class ConsolidationService:
             volatile_cached=total_volatile_cached,
             files_queued=total_files_queued,
             prefetch_registered=total_prefetch_registered,
+            searches_deferred=searches_deferred,
             errors=errors,
             results=results,
-            dry_run=dry_run
+            dry_run=dry_run,
+            duration_ms=duration_ms
         )
 
         logger.info(
-            f"Consolidation complete: {processed_count}/{len(unprocessed)} searches, "
+            f"Consolidation run complete: searches_processed={processed_count} "
+            f"searches_deferred={searches_deferred} duration_ms={duration_ms:.0f} | "
             f"{total_pages_created} pages created, {total_pages_updated} updated, "
             f"{total_entities_added} entities, {total_volatile_cached} volatile, "
             f"{total_files_queued} files, {total_prefetch_registered} prefetch"
@@ -180,7 +221,9 @@ class ConsolidationService:
         """
         Find unprocessed SearchQuery nodes from Neo4j.
         """
-        lookback_date = datetime.now() - timedelta(days=lookback_days)
+        # UTC-aware: sq.timestamp is stored via Neo4j datetime() (UTC), and a
+        # naive local isoformat would be misread as UTC by datetime($param).
+        lookback_date = datetime.now(timezone.utc) - timedelta(days=lookback_days)
 
         query = """
         MATCH (sq:SearchQuery {processed: false})
@@ -398,7 +441,7 @@ class ConsolidationService:
         query: str,
         web_results: List[Dict[str, Any]],
         keywords: List[str],
-        user: str = "jpmschweitzer"
+        user: str
     ) -> Optional[Dict[str, Any]]:
         """
         Analyze web results with Ollama for novel information.
@@ -505,7 +548,7 @@ JSON:"""
             # Call Ollama for analysis (temperature=0.0 for consistent classification)
             response = await self.ollama.generate_text(
                 prompt=prompt,
-                model=self.settings.ollama_model,
+                model=self.settings.ollama_llm_model,
                 stream=False,
                 temperature=0.0
             )
@@ -570,7 +613,7 @@ JSON:"""
         """
 
         try:
-            await self.neo4j.execute_query(query, {"search_id": search_id})
+            await self.neo4j.execute_write(query, {"search_id": search_id})
             logger.debug(f"Marked search {search_id} as processed")
         except Exception as e:
             logger.error(f"Failed to mark search as processed: {e}")
@@ -963,7 +1006,7 @@ JSON:"""
         """
 
         try:
-            await self.neo4j.execute_query(query, {
+            await self.neo4j.execute_write(query, {
                 "name": entity_name,
                 "description": description,
                 "search_id": source_search_id
@@ -977,7 +1020,7 @@ JSON:"""
         query: str,
         web_results: List[Dict[str, Any]],
         keywords: List[str],
-        user: str = "jpmschweitzer"
+        user: str
     ) -> MemoryRoutingResult:
         """
         Unified classification of web results for memory routing.
@@ -1079,14 +1122,21 @@ JSON:"""
         try:
             response = await self.ollama.generate_text(
                 prompt=prompt,
-                model=self.settings.ollama_model,
+                model=self.settings.ollama_llm_model,
                 stream=False,
                 temperature=0.0
             )
 
+            # generate_text returns None on any transport/HTTP failure (e.g.
+            # model missing, Ollama down) and Ollama never legitimately
+            # returns an empty completion for this prompt: both mean the LLM
+            # is unavailable, NOT that there is nothing to route. Raise so
+            # the search is retried next run instead of being consumed.
             if not response:
-                logger.warning("Empty response from Ollama for classification")
-                return MemoryRoutingResult()
+                raise ConsolidationLLMUnavailableError(
+                    f"no output from generation model "
+                    f"'{self.settings.ollama_llm_model}' for classification"
+                )
 
             # Extract JSON array from response
             response_clean = response.strip()
@@ -1140,7 +1190,12 @@ JSON:"""
             )
             return result
 
+        except ConsolidationLLMUnavailableError:
+            # Infrastructure failure: propagate so the search is NOT consumed
+            raise
         except json.JSONDecodeError as e:
+            # The model responded but with unparseable output: consume the
+            # search (empty routing) to avoid retrying a bad prompt forever.
             logger.error(f"Failed to parse classification response as JSON: {e}")
             return MemoryRoutingResult()
         except Exception as e:

src/services/document_sync_service.py

@@ -194,36 +194,34 @@ class DocumentSyncService:
     ) -> int:
         """Create vector embeddings for document content."""
         collection = get_qdrant_collection_name(user)
-        self.qdrant.ensure_collection(collection)
-
-        # Delete existing chunks for this document
-        try:
-            self.qdrant.client.delete(
-                collection_name=collection,
-                points_selector={
-                    "filter": {
-                        "must": [
-                            {"key": "doc_type", "match": {"value": "document"}},
-                            {"key": "paperless_id", "match": {"value": document_id}},
-                        ]
-                    }
-                }
-            )
-        except Exception as e:
-            logger.debug(f"No existing chunks to delete: {e}")
+        await self.qdrant.ensure_collection(collection)
 
         # Chunk content
         chunks = self._chunk_text(content)
         if not chunks:
             return 0
 
-        # Generate embeddings
+        # Generate embeddings (embed_batch returns None for failed chunks)
         embeddings = await self.ollama.embed_batch(chunks)
 
-        # Build points
+        # Build points, skipping chunks whose embedding failed. Previously a
+        # single None embedding poisoned the batch and aborted the whole
+        # document upsert.
         points = []
+        skipped = 0
         for i, (chunk, embedding) in enumerate(zip(chunks, embeddings)):
-            point_id = str(uuid.uuid4())
+            if embedding is None:
+                skipped += 1
+                logger.warning(
+                    f"Skipping chunk {i} of document {document_id}: embedding failed"
+                )
+                continue
+
+            # Deterministic id: re-upserting the same document overwrites
+            # its previous chunks in place (enables delete-last below).
+            point_id = str(
+                uuid.uuid5(uuid.NAMESPACE_DNS, f"document_{document_id}_chunk_{i}")
+            )
             content_hash = hashlib.md5(chunk.encode()).hexdigest()
 
             points.append({
@@ -240,13 +238,48 @@ class DocumentSyncService:
                 }
             })
 
-        # Upsert to Qdrant
+        if skipped and not points:
+            raise RuntimeError(
+                f"All {skipped} chunk embeddings failed for document {document_id}"
+            )
+        if skipped:
+            logger.warning(
+                f"Document {document_id}: {skipped}/{len(chunks)} chunks skipped "
+                f"(embedding failures); indexing the remaining {len(points)}"
+            )
+
+        # Upsert BEFORE pruning stale chunks (same order as the wiki
+        # reindex fix in VectorService.update_from_page): the old
+        # delete-first order left the document with ZERO vectors until the
+        # next successful sync whenever the embedding pass failed after the
+        # delete (e.g. Ollama down). Deterministic uuid5 ids make the
+        # in-place overwrite safe.
         if points:
-            self.qdrant.client.upsert(
+            await self.qdrant.upsert_points(
                 collection_name=collection,
                 points=points
             )
 
+            # Prune chunks left over from a previous version of the document
+            # (indexes beyond the new count, or legacy random-uuid4 points).
+            # Only prune after a successful upsert - a fully failed embedding
+            # pass must not wipe the old vectors.
+            new_ids = {p["id"] for p in points}
+            existing = await self.qdrant.scroll_all_points(
+                collection_name=collection,
+                filter_conditions={
+                    "doc_type": "document",
+                    "paperless_id": document_id,
+                },
+                with_payload=False,
+            )
+            stale_ids = [pt["id"] for pt in existing if pt["id"] not in new_ids]
+            if stale_ids:
+                await self.qdrant.delete_by_ids(
+                    collection_name=collection,
+                    point_ids=stale_ids,
+                )
+
         return len(points)
 
     async def _index_graph(
@@ -265,7 +298,7 @@ class DocumentSyncService:
             d.updated_at = datetime()
         RETURN d
         """
-        await self.neo4j.execute_query(
+        await self.neo4j.execute_write(
             query,
             {
                 "paperless_id": document_id,

src/services/graph_service.py

@@ -12,7 +12,7 @@ import logging
 
 from src.clients.neo4j_client import Neo4jClient
 from src.clients.wikijs_client import WikiJSClient
-from src.core.multi_tenancy import get_neo4j_user_label
+from src.core.multi_tenancy import get_neo4j_user_label, is_path_in_user_namespace
 from src.models.graph import (
     GraphNode, GraphRelationship, GraphNodeDetail,
     CypherQueryResponse, GraphUpdateSummary, EntityMention,
@@ -73,6 +73,15 @@ class GraphService:
         self.neo4j = neo4j_client
         self.wiki = wikijs_client
 
+    # Conservative denylist of Cypher write clauses / procedure calls.
+    # Matched as whole words against the uppercased query. CALL is rejected
+    # entirely (covers db.*/apoc.* write procedures and CALL {} subqueries)
+    # because reliably distinguishing read from write procedures would
+    # require a real Cypher parser.
+    _WRITE_CLAUSE_PATTERN = re.compile(
+        r"\b(CREATE|MERGE|DELETE|DETACH|SET|REMOVE|DROP|FOREACH|LOAD|CALL)\b"
+    )
+
     async def execute_query(
         self,
         query: str,
@@ -80,29 +89,39 @@ class GraphService:
         user: str
     ) -> CypherQueryResponse:
         """
-        Execute user-scoped Cypher query.
+        Execute a raw Cypher query — READ-ONLY, NOT tenant-scoped.
 
-        Automatically injects user label into query for security.
+        Security model (admin/debug endpoint):
+        - Queries containing write clauses (CREATE/MERGE/DELETE/SET/REMOVE/
+          DROP/DETACH/FOREACH/LOAD CSV) or any CALL are rejected up front.
+        - The query is executed through a session opened with
+          default_access_mode=READ_ACCESS, so the database itself refuses
+          writes even if the denylist is bypassed.
+        - Results are NOT automatically restricted to the user's tenant
+          labels: an arbitrary Cypher query can read any tenant's nodes.
+          Callers must scope patterns themselves (e.g. match on
+          `User_<Tenant>`/`User_<Tenant>_Document` labels).
 
         Args:
-            query: Cypher query
+            query: Cypher query (read-only)
             parameters: Query parameters
-            user: User identifier
+            user: Requesting user (audit logging only — does NOT scope
+                the query)
 
         Returns:
             Query results with metadata
+
+        Raises:
+            ValueError: If the query contains write clauses or fails
         """
         start_time = time.time()
 
-        # Get user-specific label
-        user_label = get_neo4j_user_label(user)
+        self._reject_write_clauses(query)
 
-        # Inject user label into query for scoping
-        # This ensures users can only query their own data
-        scoped_query = self._scope_query_to_user(query, user_label)
+        logger.info(f"Read-only Cypher query for user '{user}' (unscoped): {query[:200]}")
 
         try:
-            results = await self.neo4j.execute_query(scoped_query, parameters)
+            results = await self.neo4j.execute_read(query, parameters)
             query_time_ms = (time.time() - start_time) * 1000
 
             return CypherQueryResponse(
@@ -111,28 +130,33 @@ class GraphService:
                 query_time_ms=query_time_ms
             )
 
+        except ValueError:
+            raise
         except Exception as e:
             logger.error(f"Cypher query failed: {e}", exc_info=True)
             raise ValueError(f"Query execution failed: {str(e)}")
 
-    def _scope_query_to_user(self, query: str, user_label: str) -> str:
+    def _reject_write_clauses(self, query: str) -> None:
         """
-        Inject user label into Cypher query for multi-tenancy.
+        Reject Cypher queries containing write clauses or procedure calls.
 
-        Simple implementation: adds user label to node patterns.
-        Production version would use proper query parsing.
+        Conservative denylist on the uppercased query: false positives are
+        acceptable (e.g. the word SET in a string literal), false negatives
+        are not. The read-only session is the hard backstop.
 
         Args:
-            query: Original Cypher query
-            user_label: User-specific label
+            query: Raw Cypher query
 
-        Returns:
-            Scoped query
+        Raises:
+            ValueError: If a denylisted clause is found
         """
-        # For now, return query as-is
-        # TODO: Implement proper query scoping with label injection
-        logger.warning("Query scoping not yet implemented - returning unscoped query")
-        return query
+        match = self._WRITE_CLAUSE_PATTERN.search(query.upper())
+        if match:
+            raise ValueError(
+                f"Query rejected: '{match.group(1)}' is not allowed — "
+                "/query/graph and /graph/query are read-only (no write "
+                "clauses or CALL procedures)"
+            )
 
     async def list_nodes(
         self,
@@ -400,6 +424,14 @@ class GraphService:
             if not page:
                 raise ValueError(f"Page {page_id} not found")
 
+            # TENANT ISOLATION: only pages inside the user's own wiki
+            # namespace may be written into that user's graph labels.
+            if not is_path_in_user_namespace(page.get("path", ""), user):
+                raise ValueError(
+                    f"Page {page_id} (path: {page.get('path')!r}) is outside "
+                    f"user '{user}' namespace - refusing cross-tenant ingestion"
+                )
+
             # PROTECTION: Skip entity extraction on auto-generated entity stub pages
             tags = page.get("tags", [])
             if "entity-stub" in tags or "auto-generated" in tags:
@@ -420,23 +452,30 @@ class GraphService:
             user_base_label = get_neo4j_user_base_label(user)  # For entities
             user_doc_label = get_neo4j_user_label(user)  # For documents
 
-            # Create/update Document node
+            # Create/update Document node.
+            # content_hash records the fingerprint of the ingested content so
+            # /ingest/check-updates can detect changed pages without re-reading
+            # the graph's source content.
+            from src.core.hashing import compute_content_hash
+
             doc_query = f"""
             MERGE (d:{user_doc_label}:Document {{page_id: $page_id}})
             SET d.title = $title,
                 d.path = $path,
                 d.tags = $tags,
                 d.updated_at = datetime(),
-                d.content_length = $content_length
+                d.content_length = $content_length,
+                d.content_hash = $content_hash
             RETURN d
             """
 
-            await self.neo4j.execute_query(doc_query, {
+            await self.neo4j.execute_write(doc_query, {
                 "page_id": page_id,
                 "title": page.get("title"),
                 "path": page.get("path"),
                 "tags": tags,
-                "content_length": len(content)
+                "content_length": len(content),
+                "content_hash": compute_content_hash(content)
             })
 
             nodes_created = 1  # Document node
@@ -454,7 +493,7 @@ class GraphService:
                 RETURN e, r
                 """
 
-                result = await self.neo4j.execute_query(entity_query, {
+                result = await self.neo4j.execute_write(entity_query, {
                     "name": entity.text,
                     "page_id": page_id,
                     "confidence": entity.confidence
@@ -518,7 +557,7 @@ class GraphService:
         """
 
         try:
-            result = await self.neo4j.execute_query(
+            result = await self.neo4j.execute_write(
                 delete_query,
                 {"page_id": page_id}
             )
@@ -651,10 +690,11 @@ class GraphService:
         """
         from src.core.multi_tenancy import get_neo4j_user_base_label
         user_base_label = get_neo4j_user_base_label(user)
+        user_doc_label = get_neo4j_user_label(user)
 
         query = f"""
         MATCH (e:{user_base_label}:{entity_type} {{name: $name}})
-        MATCH (d:Document)-[:MENTIONS]->(e)
+        MATCH (d:{user_doc_label}:Document)-[:MENTIONS]->(e)
         RETURN count(distinct d) as mention_count
         """
 
@@ -690,8 +730,8 @@ class GraphService:
         entity_path = f"{user_namespace}/entities/{entity_type.lower()}/{entity_name.lower().replace(' ', '-')}"
 
         try:
-            # Search for page by path
-            pages = await self.wiki.list_pages(limit=1000)
+            # Search for page by path (scoped to the user's namespace)
+            pages = await self.wiki.list_pages(path_prefix=user_namespace, limit=1000)
             # list_pages returns a list directly, not a dict
             for page in pages:
                 if page.get("path", "") == entity_path:
@@ -798,11 +838,12 @@ Feel free to expand it with more details!
         try:
             from src.core.multi_tenancy import get_neo4j_user_base_label
             user_base_label = get_neo4j_user_base_label(user)
+            user_doc_label = get_neo4j_user_label(user)
 
-            # Get mentioning documents
+            # Get mentioning documents (scoped to this tenant's documents)
             mention_query = f"""
             MATCH (e:{user_base_label}:{entity_type} {{name: $name}})
-            MATCH (d:Document)-[:MENTIONS]->(e)
+            MATCH (d:{user_doc_label}:Document)-[:MENTIONS]->(e)
             RETURN d.title as title, d.path as path, d.page_id as page_id
             """
 
@@ -820,7 +861,7 @@ Feel free to expand it with more details!
             # Only match entity nodes (not Document nodes)
             related_query = f"""
             MATCH (e1:{user_base_label}:{entity_type} {{name: $name}})
-            MATCH (d:Document)-[:MENTIONS]->(e1)
+            MATCH (d:{user_doc_label}:Document)-[:MENTIONS]->(e1)
             MATCH (d)-[:MENTIONS]->(e2:{user_base_label})
             WHERE e2 <> e1 AND NOT (e2:Document)
             RETURN DISTINCT e2.name as name, labels(e2) as labels,
@@ -910,15 +951,16 @@ Feel free to expand it with more details!
 
         from src.core.multi_tenancy import get_neo4j_user_base_label
         user_base_label = get_neo4j_user_base_label(user)
+        user_doc_label = get_neo4j_user_label(user)
         pages_created = []
         pages_skipped = []
 
         try:
-            # Query for entities with sufficient mentions
+            # Query for entities with sufficient mentions (tenant-scoped)
             for entity_type in entity_types:
                 query = f"""
                 MATCH (e:{user_base_label}:{entity_type})
-                MATCH (d:Document)-[:MENTIONS]->(e)
+                MATCH (d:{user_doc_label}:Document)-[:MENTIONS]->(e)
                 WITH e, count(distinct d) as mention_count
                 WHERE mention_count >= $min_mentions
                 RETURN e.name as name, mention_count
@@ -1143,6 +1185,68 @@ Feel free to expand it with more details!
             logger.error(f"Failed to get related documents for page {page_id}: {e}", exc_info=True)
             return []
 
+    async def get_related_documents_batch(
+        self,
+        page_ids: List[int],
+        user: str,
+        limit_per_page: int = 5
+    ) -> Dict[int, List[Dict[str, Any]]]:
+        """
+        Batch variant of get_related_documents: ONE UNWIND query for all pages
+        instead of one round-trip per page.
+
+        Args:
+            page_ids: Page IDs to find related documents for
+            user: User identifier
+            limit_per_page: Maximum related documents per page
+
+        Returns:
+            Mapping of page_id -> related-document rows (same shape as
+            get_related_documents). Pages with no related documents are
+            absent from the mapping.
+        """
+        from src.core.multi_tenancy import get_neo4j_user_base_label
+
+        if not page_ids:
+            return {}
+
+        user_base_label = get_neo4j_user_base_label(user)
+        user_doc_label = get_neo4j_user_label(user)
+
+        # ORDER BY runs before collect() so each page's list is sorted by
+        # shared_entities descending; [..$limit] trims per page.
+        query = f"""
+        UNWIND $page_ids AS pid
+        MATCH (d1:{user_doc_label}:Document {{page_id: pid}})
+        MATCH (d1)-[:MENTIONS]->(e:{user_base_label})<-[:MENTIONS]-(d2:{user_doc_label}:Document)
+        WHERE d1 <> d2 AND NOT e:Document
+        WITH pid, d2, d2.tags as tags, count(DISTINCT e) as shared_entities
+        WHERE tags IS NOT NULL AND size(tags) > 0
+        WITH pid, d2, tags, shared_entities
+        ORDER BY shared_entities DESC
+        WITH pid, collect({{
+            page_id: d2.page_id,
+            title: d2.title,
+            path: d2.path,
+            tags: tags,
+            shared_entities: shared_entities
+        }})[..$limit] AS related
+        RETURN pid AS page_id, related
+        """
+
+        try:
+            rows = await self.neo4j.execute_query(
+                query,
+                {"page_ids": page_ids, "limit": limit_per_page}
+            )
+            return {row["page_id"]: row["related"] for row in rows}
+        except Exception as e:
+            logger.error(
+                f"Failed to get related documents for {len(page_ids)} pages: {e}",
+                exc_info=True
+            )
+            return {}
+
     async def get_all_entities(self, user: str) -> List[Dict[str, Any]]:
         """
         Get all entities from the knowledge graph for a user.
@@ -1249,7 +1353,7 @@ Feel free to expand it with more details!
         """
 
         try:
-            results = await self.neo4j.execute_query(
+            results = await self.neo4j.execute_write(
                 query,
                 {"page_id": page_id, "entity_names": names}
             )
@@ -1288,7 +1392,7 @@ Feel free to expand it with more details!
         """
 
         try:
-            result = await self.neo4j.execute_query(
+            result = await self.neo4j.execute_write(
                 delete_query,
                 {"document_id": document_id}
             )
@@ -1330,7 +1434,7 @@ Feel free to expand it with more details!
         """
 
         try:
-            result = await self.neo4j.execute_query(
+            result = await self.neo4j.execute_write(
                 delete_query,
                 {"paperless_id": paperless_id}
             )
@@ -1374,7 +1478,7 @@ Feel free to expand it with more details!
         """
 
         try:
-            result = await self.neo4j.execute_query(
+            result = await self.neo4j.execute_write(
                 delete_query,
                 {"collection_id": collection_id}
             )
@@ -1403,12 +1507,13 @@ Feel free to expand it with more details!
         from src.core.multi_tenancy import get_neo4j_user_base_label
 
         user_base_label = get_neo4j_user_base_label(user)
+        user_doc_label = get_neo4j_user_label(user)
 
         query = f"""
         MATCH (e:{user_base_label})
         WHERE NOT e:Document
         AND NOT e:DocumentCollection
-        AND NOT EXISTS {{ (d:Document)-[:MENTIONS]->(e) }}
+        AND NOT EXISTS {{ (d:{user_doc_label}:Document)-[:MENTIONS]->(e) }}
         RETURN elementId(e) as id, e.name as name, labels(e) as labels
         """
 
@@ -1451,18 +1556,19 @@ Feel free to expand it with more details!
         from src.core.multi_tenancy import get_neo4j_user_base_label
 
         user_base_label = get_neo4j_user_base_label(user)
+        user_doc_label = get_neo4j_user_label(user)
 
         query = f"""
         MATCH (e:{user_base_label})
         WHERE NOT e:Document
         AND NOT e:DocumentCollection
-        AND NOT EXISTS {{ (d:Document)-[:MENTIONS]->(e) }}
+        AND NOT EXISTS {{ (d:{user_doc_label}:Document)-[:MENTIONS]->(e) }}
         DETACH DELETE e
         RETURN count(e) as purged_count
         """
 
         try:
-            results = await self.neo4j.execute_query(query, {})
+            results = await self.neo4j.execute_write(query, {})
             purged_count = results[0]["purged_count"] if results else 0
 
             logger.info(f"Purged {purged_count} orphan entities for user {user}")
@@ -1545,7 +1651,7 @@ Feel free to expand it with more details!
                 DETACH DELETE d
                 RETURN count(d) as purged_count
                 """
-                results = await self.neo4j.execute_query(query, {"page_ids": page_ids})
+                results = await self.neo4j.execute_write(query, {"page_ids": page_ids})
                 count = results[0]["purged_count"] if results else 0
                 total_purged += count
                 logger.info(f"Purged {count} wiki Document nodes")
@@ -1558,7 +1664,7 @@ Feel free to expand it with more details!
                 DETACH DELETE d
                 RETURN count(d) as purged_count
                 """
-                results = await self.neo4j.execute_query(query, {"document_ids": document_ids})
+                results = await self.neo4j.execute_write(query, {"document_ids": document_ids})
                 count = results[0]["purged_count"] if results else 0
                 total_purged += count
                 logger.info(f"Purged {count} Document Store Document nodes")
@@ -1584,15 +1690,19 @@ Feel free to expand it with more details!
         Returns:
             Number of relationships cleaned
         """
-        query = """
-        MATCH (sq:SearchQuery)-[r:FOUND]->(d)
-        WHERE NOT EXISTS { (d) }
+        from src.core.multi_tenancy import get_neo4j_user_base_label
+
+        user_base_label = get_neo4j_user_base_label(user)
+
+        query = f"""
+        MATCH (sq:{user_base_label}_SearchQuery:SearchQuery)-[r:FOUND]->(d)
+        WHERE NOT EXISTS {{ (d) }}
         DELETE r
         RETURN count(r) as cleaned_count
         """
 
         try:
-            results = await self.neo4j.execute_query(query, {})
+            results = await self.neo4j.execute_write(query, {})
             cleaned_count = results[0]["cleaned_count"] if results else 0
 
             if cleaned_count > 0:

src/services/hybrid_rag_service.py

@@ -34,12 +34,29 @@ from src.core.multi_tenancy import get_neo4j_user_base_label, get_neo4j_user_lab
 
 logger = logging.getLogger(__name__)
 
+# Timeout for auxiliary LLM calls (keyword extraction, re-ranking).
+# A hung Ollama call must not gate retrieval for the full client timeout.
+LLM_CALL_TIMEOUT_SECONDS = 12.0
+
 
 class HybridRAGService:
     """
     Service for HybridRAG multi-source search with fusion and re-ranking.
     """
 
+    # Phase 4 reranking only ever considers this many fused results; results
+    # beyond the slice cannot reach the response when reranking is enabled.
+    RERANK_SLICE_SIZE = 20
+
+    # Maps internal retrieval leg names to source_status keys in the response
+    SOURCE_STATUS_KEYS = {
+        "vector": "vector",
+        "graph": "graph",
+        "web": "web",
+        "volatile": "volatile",
+        "document": "documents",
+    }
+
     def __init__(
         self,
         vector_service: VectorService,
@@ -69,7 +86,10 @@ class HybridRAGService:
         self.content_extractor = content_extractor
         self.settings = settings
         self.volatile = volatile_service
-        self.reranker_model = settings.ollama_model
+        self.reranker_model = settings.ollama_llm_model
+        # Strong references to fire-and-forget persistence tasks so they are
+        # not garbage-collected mid-flight (see Phase 6 in search()).
+        self._background_tasks: set = set()
 
     async def search(
         self,
@@ -132,10 +152,19 @@ class HybridRAGService:
         )
         timing["fusion_ms"] = (time.time() - phase2_start) * 1000
 
-        # Phase 3: Enrichment
+        # Phase 3: Enrichment — only for results that can still reach the
+        # response: the rerank slice (Phase 4 reorders within it) when
+        # reranking is on, otherwise just the final result count. Enriching
+        # the full fused set queried Neo4j per result and threw most of the
+        # output away at the final trim.
         phase3_start = time.time()
         if config.enable_enrichment:
-            enriched_results = await self._enrich_with_related_dossiers(fused_results, user)
+            enrich_top_k = config.final_result_count
+            if config.enable_reranking:
+                enrich_top_k = max(enrich_top_k, self.RERANK_SLICE_SIZE)
+            enriched_results = await self._enrich_with_related_dossiers(
+                fused_results, user, top_k=enrich_top_k
+            )
         else:
             enriched_results = fused_results
         timing["enrichment_ms"] = (time.time() - phase3_start) * 1000
@@ -143,7 +172,9 @@ class HybridRAGService:
         # Phase 4: LLM Re-ranking
         phase4_start = time.time()
         if config.enable_reranking and len(enriched_results) > 1:
-            reranked_results = await self._rerank_with_llm(enriched_results[:20], query)
+            reranked_results = await self._rerank_with_llm(
+                enriched_results[:self.RERANK_SLICE_SIZE], query
+            )
         else:
             reranked_results = enriched_results
         timing["reranking_ms"] = (time.time() - phase4_start) * 1000
@@ -169,17 +200,33 @@ class HybridRAGService:
 
         timing["total_ms"] = (time.time() - start_time) * 1000
 
-        # Phase 6: Persistence (async, non-blocking)
-        phase6_start = time.time()
-        search_id = await self._persist_search_for_librarian(
-            query=query,
-            user=user,
-            keywords_data=keywords_data,
-            raw_results=raw_results,
-            final_results=final_results,
-            timing=timing
+        # Phase 6: Persistence — genuinely off the hot path. The search_id is
+        # generated up front and returned immediately; the Neo4j write runs as
+        # a background task (one atomic transaction, see
+        # _persist_search_for_librarian) instead of gating the response.
+        search_id = str(uuid.uuid4())
+        timing["persistence_ms"] = 0.0  # not on the request path anymore
+        persist_task = asyncio.create_task(
+            self._persist_search_for_librarian(
+                search_id=search_id,
+                query=query,
+                user=user,
+                keywords_data=keywords_data,
+                raw_results=raw_results,
+                final_results=final_results,
+                timing=timing
+            )
         )
-        timing["persistence_ms"] = (time.time() - phase6_start) * 1000
+        self._background_tasks.add(persist_task)
+        persist_task.add_done_callback(self._background_tasks.discard)
+
+        # Degradation signaling: a failed leg contributes no results, but the
+        # response says so instead of silently pretending the leg was empty
+        source_status = raw_results.get("source_status", {})
+        degraded = any(status == "failed" for status in source_status.values())
+        if degraded:
+            failed_legs = [leg for leg, status in source_status.items() if status == "failed"]
+            logger.warning(f"HybridRAG search degraded: failed legs: {failed_legs}")
 
         # Build response
         return HybridRAGResponse(
@@ -191,7 +238,9 @@ class HybridRAGService:
             total_results=len(result_models),
             timing=TimingBreakdown(**timing),
             config_used=config,
-            search_id=search_id
+            search_id=search_id,
+            source_status=source_status,
+            degraded=degraded
         )
 
     async def _extract_keywords_and_synonyms(self, query: str) -> Dict[str, Any]:
@@ -224,10 +273,13 @@ Return format:
 JSON:"""
 
         try:
-            response = await self.ollama.generate_text(
-                prompt=prompt,
-                model=self.reranker_model,
-                temperature=0.0  # Deterministic for consistent extraction
+            response = await asyncio.wait_for(
+                self.ollama.generate_text(
+                    prompt=prompt,
+                    model=self.reranker_model,
+                    temperature=0.0  # Deterministic for consistent extraction
+                ),
+                timeout=LLM_CALL_TIMEOUT_SECONDS
             )
 
             # Parse JSON response (handle potential extra text)
@@ -261,6 +313,16 @@ JSON:"""
                 "synonyms": {},
                 "expansions": {}
             }
+        except asyncio.TimeoutError:
+            logger.warning(
+                f"Keyword extraction timed out after {LLM_CALL_TIMEOUT_SECONDS}s, using fallback"
+            )
+            return {
+                "core_keywords": query.split(),
+                "entities": [],
+                "synonyms": {},
+                "expansions": {}
+            }
         except Exception as e:
             logger.error(f"Keyword extraction failed: {e}", exc_info=True)
             return {
@@ -276,10 +338,14 @@ JSON:"""
         user: str,
         config: HybridRAGConfig,
         keywords_data: Dict[str, Any]
-    ) -> Dict[str, List]:
+    ) -> Dict[str, Any]:
         """
         Phase 1: Retrieve results from all sources in parallel.
 
+        Each retrieval leg returns (results, timing_ms, error) so that a
+        failed leg still contributes no results but is reported in
+        "source_status" instead of being silently swallowed.
+
         Args:
             query: Search query
             user: User identifier
@@ -287,7 +353,8 @@ JSON:"""
             keywords_data: Extracted keywords/synonyms
 
         Returns:
-            Dictionary with results from each source and timing
+            Dictionary with results from each source, timing, and per-leg
+            "source_status" ('ok', 'failed', or 'disabled')
         """
         tasks = {}
         timing = {}
@@ -314,10 +381,10 @@ JSON:"""
                         }
                         for r in response.results
                     ]
-                    return results, (time.time() - start) * 1000
+                    return results, (time.time() - start) * 1000, None
                 except Exception as e:
-                    logger.error(f"Vector search failed: {e}", exc_info=True)
-                    return [], (time.time() - start) * 1000
+                    logger.warning(f"Vector search failed: {e}", exc_info=True)
+                    return [], (time.time() - start) * 1000, e
 
             tasks["vector"] = vector_search()
 
@@ -352,10 +419,10 @@ JSON:"""
                         }
                         for r in results
                     ]
-                    return formatted, (time.time() - start) * 1000
+                    return formatted, (time.time() - start) * 1000, None
                 except Exception as e:
-                    logger.error(f"Graph search failed: {e}", exc_info=True)
-                    return [], (time.time() - start) * 1000
+                    logger.warning(f"Graph search failed: {e}", exc_info=True)
+                    return [], (time.time() - start) * 1000, e
 
             tasks["graph"] = graph_search()
 
@@ -391,10 +458,10 @@ JSON:"""
                         }
                         for r in results
                     ]
-                    return formatted, (time.time() - start) * 1000
+                    return formatted, (time.time() - start) * 1000, None
                 except Exception as e:
-                    logger.error(f"Web search failed: {e}", exc_info=True)
-                    return [], (time.time() - start) * 1000
+                    logger.warning(f"Web search failed: {e}", exc_info=True)
+                    return [], (time.time() - start) * 1000, e
 
             tasks["web"] = web_search()
 
@@ -422,10 +489,10 @@ JSON:"""
                         }
                         for r in results
                     ]
-                    return formatted, (time.time() - start) * 1000
+                    return formatted, (time.time() - start) * 1000, None
                 except Exception as e:
-                    logger.error(f"Volatile search failed: {e}", exc_info=True)
-                    return [], (time.time() - start) * 1000
+                    logger.warning(f"Volatile search failed: {e}", exc_info=True)
+                    return [], (time.time() - start) * 1000, e
 
             tasks["volatile"] = volatile_search()
 
@@ -441,37 +508,29 @@ JSON:"""
                     # Check if collection exists
                     exists = await self.vector.qdrant.collection_exists(collection_name)
                     if not exists:
-                        return [], (time.time() - start) * 1000
+                        return [], (time.time() - start) * 1000, None
 
                     # Get query embedding
-                    query_embedding = await self.vector.ollama.embed_text(query)
+                    query_embedding = await self.vector.ollama.embed(query)
 
-                    # Search with filter for doc_type=document
-                    from qdrant_client.models import Filter, FieldCondition, MatchValue
-                    search_results = self.vector.qdrant.client.search(
+                    # Search via the async wrapper with doc_type=document filter
+                    search_results = await self.vector.qdrant.search_vectors(
                         collection_name=collection_name,
                         query_vector=query_embedding,
                         limit=config.document_limit,
                         score_threshold=config.document_threshold,
-                        query_filter=Filter(
-                            must=[
-                                FieldCondition(
-                                    key="doc_type",
-                                    match=MatchValue(value="document")
-                                )
-                            ]
-                        )
+                        filter_conditions={"doc_type": "document"}
                     )
 
                     # Format results
                     formatted = []
                     for r in search_results:
-                        payload = r.payload or {}
+                        payload = r.get("payload") or {}
                         formatted.append({
                             "paperless_id": payload.get("paperless_id"),
                             "title": payload.get("title", "Untitled Document"),
                             "content": payload.get("chunk_text", ""),
-                            "score": r.score,
+                            "score": r["score"],
                             "correspondent": payload.get("correspondent"),
                             "document_type": payload.get("document_type"),
                             "tags": payload.get("tags", []),
@@ -479,22 +538,30 @@ JSON:"""
                             "source": "document"
                         })
 
-                    return formatted, (time.time() - start) * 1000
+                    return formatted, (time.time() - start) * 1000, None
                 except Exception as e:
-                    logger.error(f"Document search failed: {e}", exc_info=True)
-                    return [], (time.time() - start) * 1000
+                    logger.warning(f"Document search failed: {e}", exc_info=True)
+                    return [], (time.time() - start) * 1000, e
 
             tasks["document"] = document_search()
 
         # Execute all searches in parallel
+        # No return_exceptions needed: each leg captures its own exception
+        # and reports it via the (results, timing, error) tuple.
         results_dict = await asyncio.gather(*tasks.values())
 
-        # Combine results with timing
-        output = {"timing": {}}
+        # Combine results with timing and per-leg status
+        output = {"timing": {}, "source_status": {}}
         for i, source in enumerate(tasks.keys()):
-            results, source_timing = results_dict[i]
+            results, source_timing, error = results_dict[i]
             output[source] = results
             output["timing"][f"{source}_ms"] = source_timing
+            status_key = self.SOURCE_STATUS_KEYS[source]
+            output["source_status"][status_key] = "failed" if error is not None else "ok"
+
+        # Legs that were not attempted are reported as disabled
+        for status_key in self.SOURCE_STATUS_KEYS.values():
+            output["source_status"].setdefault(status_key, "disabled")
 
         logger.info(
             f"Parallel retrieval: vector={len(output.get('vector', []))}, "
@@ -684,49 +751,65 @@ JSON:"""
     async def _enrich_with_related_dossiers(
         self,
         results: List[Dict[str, Any]],
-        user: str
+        user: str,
+        top_k: Optional[int] = None
     ) -> List[Dict[str, Any]]:
         """
         Phase 3: Enrich results with related documents via shared entities.
 
+        Only the top_k results are enriched (the rest get an empty
+        related_dossiers list — they cannot survive the final trim anyway),
+        and all lookups go through ONE UNWIND-batched Neo4j query instead of
+        a sequential round-trip per result.
+
         Args:
             results: Fused results
             user: User identifier
+            top_k: How many leading results to enrich (None = all)
 
         Returns:
             Results with related_dossiers added
         """
+        enrich_slice = results if top_k is None else results[:top_k]
+
+        # Dedupe while preserving order; volatile/web results have no page_id
+        page_ids: List[int] = []
+        for result in enrich_slice:
+            page_id = result.get("result", {}).get("page_id")
+            if page_id and page_id not in page_ids:
+                page_ids.append(page_id)
+
+        try:
+            related_map = await self.graph.get_related_documents_batch(
+                page_ids=page_ids,
+                user=user,
+                limit_per_page=5
+            )
+        except Exception as e:
+            logger.warning(f"Failed batched related-docs lookup for {len(page_ids)} pages: {e}")
+            related_map = {}
+
         for result in results:
-            result_data = result.get("result", {})
-            page_id = result_data.get("page_id")
+            result["related_dossiers"] = []
 
-            if page_id:
-                try:
-                    related_docs = await self.graph.get_related_documents(
-                        page_id=page_id,
-                        user=user,
-                        limit=5
-                    )
+        for result in enrich_slice:
+            page_id = result.get("result", {}).get("page_id")
+            if not page_id:
+                continue
 
-                    # Convert to RelatedDossier format
-                    related_dossiers = []
-                    for doc in related_docs:
-                        for tag in doc.get("tags", [])[:3]:  # Max 3 tags per doc
-                            related_dossiers.append({
-                                "page_id": doc["page_id"],
-                                "title": doc["title"],
-                                "path": doc["path"],
-                                "tag": tag,
-                                "shared_entities": doc["shared_entities"]
-                            })
+            # Convert to RelatedDossier format
+            related_dossiers = []
+            for doc in related_map.get(page_id, []):
+                for tag in (doc.get("tags") or [])[:3]:  # Max 3 tags per doc
+                    related_dossiers.append({
+                        "page_id": doc["page_id"],
+                        "title": doc["title"],
+                        "path": doc["path"],
+                        "tag": tag,
+                        "shared_entities": doc["shared_entities"]
+                    })
 
-                    result["related_dossiers"] = related_dossiers[:5]  # Limit to 5 total
-
-                except Exception as e:
-                    logger.warning(f"Failed to get related docs for page {page_id}: {e}")
-                    result["related_dossiers"] = []
-            else:
-                result["related_dossiers"] = []
+            result["related_dossiers"] = related_dossiers[:5]  # Limit to 5 total
 
         return results
 
@@ -772,15 +855,24 @@ Example output: 3,1,5,2,4
 
 Ranking:"""
 
-            response = await self.ollama.generate_text(
-                prompt=prompt,
-                model=self.reranker_model,
-                temperature=0.0  # Deterministic for consistent rankings
+            response = await asyncio.wait_for(
+                self.ollama.generate_text(
+                    prompt=prompt,
+                    model=self.reranker_model,
+                    temperature=0.0  # Deterministic for consistent rankings
+                ),
+                timeout=LLM_CALL_TIMEOUT_SECONDS
             )
 
-            # Parse response: "3,1,5,2,4" → [2, 0, 4, 1, 3] (0-indexed)
+            # Parse response: "3,1,5,2,4" → [2, 0, 4, 1, 3] (0-indexed).
+            # Deduplicated preserving first occurrence: an LLM answer like
+            # "3,3,1" must not put the same result in the ranking twice.
             indices_str = response.strip().split('\n')[0]  # Take first line
-            indices = [int(x.strip()) - 1 for x in indices_str.split(",") if x.strip().isdigit()]
+            indices = list(dict.fromkeys(
+                int(x.strip()) - 1
+                for x in indices_str.split(",")
+                if x.strip().isdigit()
+            ))
 
             # Reorder results according to LLM ranking
             reranked = []
@@ -796,6 +888,11 @@ Ranking:"""
             logger.info(f"LLM re-ranking: reordered {len(reranked)} results")
             return reranked
 
+        except asyncio.TimeoutError:
+            logger.warning(
+                f"LLM re-ranking timed out after {LLM_CALL_TIMEOUT_SECONDS}s, using RRF order"
+            )
+            return results  # Fallback to RRF order
         except Exception as e:
             logger.warning(f"LLM re-ranking failed: {e}, using RRF order")
             return results  # Fallback to RRF order
@@ -833,6 +930,7 @@ Ranking:"""
 
     async def _persist_search_for_librarian(
         self,
+        search_id: str,
         query: str,
         user: str,
         keywords_data: Dict[str, Any],
@@ -843,10 +941,20 @@ Ranking:"""
         """
         Phase 6: Store search query and results for Librarian processing.
 
-        Creates SearchQuery node in Neo4j with relationships to found documents
-        and web results for offline knowledge consolidation.
+        Creates the SearchQuery node, FOUND links to this tenant's Document
+        nodes, and WebResult nodes in ONE UNWIND-based write transaction
+        (previously ~21+ sequential auto-commit queries), so a mid-way
+        failure can never leave a partial SearchQuery graph behind.
+
+        SHAPE CONTRACT: the consolidation service (consolidation_service.py)
+        consumes exactly this shape — SearchQuery {id, query, user,
+        timestamp, processed:false, total_results, web_count, keywords},
+        (sq)-[f:FOUND {rank, rrf_score}]->(wr:WebResult {url, title,
+        content}) — do not change it without updating both sides
+        (pinned by tests/test_search_persistence.py).
 
         Args:
+            search_id: Pre-generated search ID (already returned to the caller)
             query: Search query
             user: User identifier
             keywords_data: Extracted keywords/synonyms
@@ -855,14 +963,47 @@ Ranking:"""
             timing: Performance timing
 
         Returns:
-            Search ID for tracking
+            Search ID on success, None on failure
         """
         try:
             user_base_label = get_neo4j_user_base_label(user)
-            search_id = str(uuid.uuid4())
+            user_doc_label = get_neo4j_user_label(user)
 
-            # Create SearchQuery node
-            create_query = f"""
+            # Links to found wiki documents (top 20).
+            # TENANT ISOLATION: matched against this tenant's Document label
+            # only — an unscoped (d:Document {page_id}) match would attach
+            # FOUND relationships to other tenants' documents that share the
+            # same Wiki.js page id.
+            doc_links = []
+            for rank, result_data in enumerate(final_results[:20], start=1):
+                result = result_data.get("result", {})
+                page_id = result.get("page_id")
+                if page_id:
+                    doc_links.append({
+                        "page_id": page_id,
+                        "source": result_data.get("source_type", "unknown"),
+                        "rank": rank,
+                        "rrf_score": result_data.get("rrf_score", 0),
+                        "final_rank": result_data.get("final_rank", rank)
+                    })
+
+            # Web results as WebResult nodes (top 10)
+            web_links = []
+            web_results = [r for r in final_results[:10] if r.get("result", {}).get("url")]
+            for rank, result_data in enumerate(web_results, start=1):
+                result = result_data.get("result", {})
+                web_links.append({
+                    "url": result.get("url"),
+                    "title": result.get("title", ""),
+                    "content": result.get("content", "")[:1000],  # Truncate
+                    "rank": rank,
+                    "rrf_score": result_data.get("rrf_score", 0)
+                })
+
+            # Single atomic write: node + doc links + web results. The CALL
+            # subqueries aggregate so an empty UNWIND list cannot swallow the
+            # rest of the query.
+            persist_query = f"""
             CREATE (sq:{user_base_label}_SearchQuery:SearchQuery {{
                 id: $search_id,
                 query: $query,
@@ -877,10 +1018,39 @@ Ranking:"""
                 synonyms: $synonyms,
                 timing_ms: $timing_ms
             }})
-            RETURN sq.id as id
+            WITH sq
+            CALL {{
+                WITH sq
+                UNWIND $doc_links AS link
+                MATCH (d:{user_doc_label}:Document {{page_id: link.page_id}})
+                MERGE (sq)-[f:FOUND]->(d)
+                SET f.source = link.source,
+                    f.rank = link.rank,
+                    f.rrf_score = link.rrf_score,
+                    f.final_rank = link.final_rank
+                RETURN count(*) AS docs_linked
+            }}
+            CALL {{
+                WITH sq
+                UNWIND $web_links AS wl
+                CREATE (wr:{user_base_label}_WebResult:WebResult {{
+                    url: wl.url,
+                    title: wl.title,
+                    content: wl.content,
+                    search_id: $search_id,
+                    timestamp: datetime()
+                }})
+                CREATE (sq)-[:FOUND {{
+                    source: "web",
+                    rank: wl.rank,
+                    rrf_score: wl.rrf_score
+                }}]->(wr)
+                RETURN count(*) AS web_created
+            }}
+            RETURN sq.id AS id, docs_linked, web_created
             """
 
-            result = await self.graph.neo4j.execute_query(create_query, {
+            await self.graph.neo4j.execute_write(persist_query, {
                 "search_id": search_id,
                 "query": query,
                 "user": user,
@@ -890,63 +1060,11 @@ Ranking:"""
                 "web_count": len(raw_results.get("web", [])),
                 "keywords": keywords_data.get("core_keywords", []),
                 "synonyms": json.dumps(keywords_data.get("synonyms", {})),
-                "timing_ms": timing.get("total_ms", 0)
+                "timing_ms": timing.get("total_ms", 0),
+                "doc_links": doc_links,
+                "web_links": web_links
             })
 
-            # Link to found wiki documents (top 20)
-            for rank, result_data in enumerate(final_results[:20], start=1):
-                result = result_data.get("result", {})
-                page_id = result.get("page_id")
-
-                if page_id:
-                    link_doc_query = f"""
-                    MATCH (sq:{user_base_label}_SearchQuery:SearchQuery {{id: $search_id}})
-                    MATCH (d:Document {{page_id: $page_id}})
-                    MERGE (sq)-[f:FOUND]->(d)
-                    SET f.source = $source,
-                        f.rank = $rank,
-                        f.rrf_score = $rrf_score,
-                        f.final_rank = $final_rank
-                    """
-
-                    await self.graph.neo4j.execute_query(link_doc_query, {
-                        "search_id": search_id,
-                        "page_id": page_id,
-                        "source": result_data.get("source_type", "unknown"),
-                        "rank": rank,
-                        "rrf_score": result_data.get("rrf_score", 0),
-                        "final_rank": result_data.get("final_rank", rank)
-                    })
-
-            # Store web results as WebResult nodes (top 10)
-            web_results = [r for r in final_results[:10] if r.get("result", {}).get("url")]
-            for rank, result_data in enumerate(web_results, start=1):
-                result = result_data.get("result", {})
-                create_web_query = f"""
-                MATCH (sq:{user_base_label}_SearchQuery:SearchQuery {{id: $search_id}})
-                CREATE (wr:{user_base_label}_WebResult:WebResult {{
-                    url: $url,
-                    title: $title,
-                    content: $content,
-                    search_id: $search_id,
-                    timestamp: datetime()
-                }})
-                CREATE (sq)-[:FOUND {{
-                    source: "web",
-                    rank: $rank,
-                    rrf_score: $rrf_score
-                }}]->(wr)
-                """
-
-                await self.graph.neo4j.execute_query(create_web_query, {
-                    "search_id": search_id,
-                    "url": result.get("url"),
-                    "title": result.get("title", ""),
-                    "content": result.get("content", "")[:1000],  # Truncate
-                    "rank": rank,
-                    "rrf_score": result_data.get("rrf_score", 0)
-                })
-
             logger.info(f"Persisted search {search_id} for Librarian processing")
             return search_id
 

src/services/ingestion_service.py

@@ -386,12 +386,32 @@ class IngestionService:
         Returns:
             BatchIngestionResult
         """
-        logger.info(f"Finding all pages for user {user} (prefix: {path_prefix or 'all'})")
+        from src.core.multi_tenancy import sanitize_user_id
+
+        # TENANT ISOLATION: the listing prefix is clamped to the user's own
+        # wiki namespace. A caller-supplied prefix outside users/{user}/
+        # would otherwise ingest another tenant's pages into this tenant's
+        # collection and graph labels.
+        if path_prefix:
+            parts = path_prefix.strip("/").split("/")
+            if (
+                len(parts) < 2
+                or parts[0] != "users"
+                or sanitize_user_id(parts[1]) != sanitize_user_id(user)
+            ):
+                raise ValueError(
+                    f"path_prefix {path_prefix!r} is outside user '{user}' "
+                    f"namespace (users/{sanitize_user_id(user)}/) - refusing "
+                    "cross-tenant ingestion"
+                )
+            effective_prefix = path_prefix.strip("/")
+        else:
+            effective_prefix = f"users/{sanitize_user_id(user)}"
+
+        logger.info(f"Finding all pages for user {user} (prefix: {effective_prefix})")
 
         # List all pages (not search - search requires a query and may have stale index)
-        pages = await self.wiki.list_all_pages(
-            path_prefix=path_prefix or f"users/{user}"
-        )
+        pages = await self.wiki.list_all_pages(path_prefix=effective_prefix)
 
         if not pages:
             logger.warning(f"No pages found for user {user}")

src/services/vector_service.py

@@ -14,7 +14,7 @@ import logging
 from src.clients.qdrant_client import QdrantClientWrapper
 from src.clients.wikijs_client import WikiJSClient
 from src.clients.ollama_client import OllamaClient
-from src.core.multi_tenancy import get_qdrant_collection_name
+from src.core.multi_tenancy import get_qdrant_collection_name, is_path_in_user_namespace
 from src.models.vector import (
     SearchResult, SearchResponse, VectorUpdateSummary,
     DocumentChunk, CollectionInfo, CollectionListResponse
@@ -120,6 +120,16 @@ class VectorService:
             if not page:
                 raise ValueError(f"Page {page_id} not found")
 
+            # TENANT ISOLATION: only pages inside the user's own wiki
+            # namespace may be embedded into that user's collection.
+            # Without this check any tenant could ingest (and then read)
+            # another tenant's wiki content.
+            if not is_path_in_user_namespace(page.get("path", ""), user):
+                raise ValueError(
+                    f"Page {page_id} (path: {page.get('path')!r}) is outside "
+                    f"user '{user}' namespace - refusing cross-tenant ingestion"
+                )
+
             # Get collection name for user
             collection_name = get_qdrant_collection_name(user)
 
@@ -144,50 +154,79 @@ class VectorService:
             chunks = self._chunk_text(content)
             logger.info(f"Split page {page_id} into {len(chunks)} chunks")
 
-            # Delete existing chunks for this page
-            deleted_count = await self.qdrant.delete_by_filter(
-                collection_name=collection_name,
-                filter_conditions={"page_id": page_id}
-            )
+            # Generate ALL embeddings in one batched /api/embed call
+            # (previously one sequential Ollama round-trip per chunk)
+            embeddings = await self.ollama.embed_batch(chunks)
 
-            # Generate embeddings and upsert chunks
-            chunks_created = 0
-            for idx, chunk_text in enumerate(chunks):
-                # Generate deterministic UUID from page_id and chunk_index
-                chunk_id = str(uuid.uuid5(uuid.NAMESPACE_DNS, f"page_{page_id}_chunk_{idx}"))
-
-                # Generate embedding
-                embedding = await self.ollama.embed(chunk_text)
+            # Build points; deterministic uuid5 IDs mean re-upserting the
+            # same page overwrites its previous chunks in place.
+            points = []
+            chunks_skipped = 0
+            embedding_dim = 768
+            for idx, (chunk_text, embedding) in enumerate(zip(chunks, embeddings)):
                 if not embedding:
-                    logger.error(f"Failed to generate embedding for chunk {chunk_id}")
+                    chunks_skipped += 1
+                    logger.error(f"Failed to generate embedding for page {page_id} chunk {idx}")
                     continue
 
-                # Prepare metadata
-                metadata = {
-                    "page_id": page_id,
-                    "page_title": title,
-                    "page_path": path,
-                    "chunk_index": idx,
-                    "chunk_text": chunk_text,
-                    "user": user
-                }
+                embedding_dim = len(embedding)
+                chunk_id = str(uuid.uuid5(uuid.NAMESPACE_DNS, f"page_{page_id}_chunk_{idx}"))
+                points.append({
+                    "id": chunk_id,
+                    "vector": embedding,
+                    "payload": {
+                        "page_id": page_id,
+                        "page_title": title,
+                        "page_path": path,
+                        "chunk_index": idx,
+                        "chunk_text": chunk_text,
+                        "user": user
+                    }
+                })
 
-                # Upsert to Qdrant
-                success = await self.qdrant.upsert_vector(
+            # Upsert BEFORE deleting stale points. The old order (delete all,
+            # then embed+upsert one by one) left the page with ZERO vectors if
+            # anything failed mid-way; now old vectors survive until the new
+            # ones are safely stored.
+            chunks_created = 0
+            if points:
+                chunks_created = await self.qdrant.upsert_points(
                     collection_name=collection_name,
-                    vector_id=chunk_id,
-                    vector=embedding,
-                    payload=metadata
+                    points=points
                 )
 
-                if success:
-                    chunks_created += 1
+            # Prune stale points from a previous version of the page (chunk
+            # indexes beyond the new count, or indexes whose new embedding
+            # failed). Only prune if the new upsert actually stored points —
+            # a fully failed embedding pass must not wipe the old vectors.
+            deleted_count = 0
+            if points:
+                new_ids = {p["id"] for p in points}
+                existing = await self.qdrant.scroll_all_points(
+                    collection_name=collection_name,
+                    filter_conditions={"page_id": page_id},
+                    with_payload=False
+                )
+                stale_ids = [pt["id"] for pt in existing if pt["id"] not in new_ids]
+                if stale_ids:
+                    deleted_count = await self.qdrant.delete_by_ids(
+                        collection_name=collection_name,
+                        point_ids=stale_ids
+                    )
 
             processing_time_ms = (time.time() - start_time) * 1000
 
+            if chunks_created == 0:
+                status = "failed"
+            elif chunks_skipped > 0:
+                status = "partial"
+            else:
+                status = "success"
+
             logger.info(
                 f"Updated vectors for page {page_id}: "
-                f"{chunks_created} chunks created, {deleted_count} old chunks deleted"
+                f"{chunks_created} chunks created, {chunks_skipped} skipped, "
+                f"{deleted_count} stale chunks deleted ({status})"
             )
 
             return VectorUpdateSummary(
@@ -195,10 +234,16 @@ class VectorService:
                 page_title=title,
                 chunks_created=chunks_created,
                 chunks_deleted=deleted_count,
+                chunks_skipped=chunks_skipped,
                 total_chunks=chunks_created,
-                embedding_dim=len(embedding) if embedding else 768,
+                embedding_dim=embedding_dim,
                 processing_time_ms=processing_time_ms,
-                success=True
+                success=chunks_created > 0,
+                status=status,
+                error_message=(
+                    f"{chunks_skipped}/{len(chunks)} chunk embeddings failed"
+                    if chunks_skipped else None
+                )
             )
 
         except Exception as e:
@@ -533,6 +578,129 @@ class VectorService:
             logger.error(f"Failed to purge chunks: {e}", exc_info=True)
             return 0
 
+    async def find_duplicate_pairs(
+        self,
+        user: str,
+        similarity_threshold: float = 0.9,
+        max_chunks_scanned: int = 2000,
+        max_pairs: int = 100
+    ) -> Dict[str, Any]:
+        """
+        Tenant-scoped similarity scan for near-duplicate wiki pages.
+
+        Scrolls the tenant's own Qdrant collection (never another tenant's),
+        then queries each chunk's vector against the same collection. Chunk
+        pairs from DIFFERENT pages scoring above the threshold are grouped
+        per page pair with the best score and the number of matching chunk
+        pairs. Read-only: nothing is modified.
+
+        Args:
+            user: Tenant user identifier
+            similarity_threshold: Minimum cosine similarity (default 0.9)
+            max_chunks_scanned: Safety cap on chunks used as probes
+            max_pairs: Maximum page pairs returned (highest score first)
+
+        Returns:
+            {
+                "chunks_scanned": int,
+                "duplicate_groups": [
+                    {
+                        "pages": [{page_id, path, title}, {page_id, path, title}],
+                        "max_similarity": float,
+                        "matching_chunk_pairs": int
+                    }, ...
+                ]
+            }
+        """
+        collection_name = get_qdrant_collection_name(user)
+
+        exists = await self.qdrant.collection_exists(collection_name)
+        if not exists:
+            return {"chunks_scanned": 0, "duplicate_groups": []}
+
+        points = await self.qdrant.scroll_all_points(
+            collection_name=collection_name,
+            batch_size=100,
+            with_payload=True,
+            with_vectors=True
+        )
+
+        # Only wiki chunks participate (documents have their own dedup story)
+        wiki_points = [
+            p for p in points
+            if p.get("vector") is not None
+            and (p.get("payload") or {}).get("doc_type", "wiki") == "wiki"
+            and (p.get("payload") or {}).get("page_id")
+        ][:max_chunks_scanned]
+
+        page_meta: Dict[int, Dict[str, Any]] = {}
+        pair_stats: Dict[tuple, Dict[str, Any]] = {}
+        seen_chunk_pairs = set()
+
+        for point in wiki_points:
+            payload = point.get("payload") or {}
+            page_id = payload.get("page_id")
+            page_meta.setdefault(page_id, {
+                "page_id": page_id,
+                "path": payload.get("page_path", ""),
+                "title": payload.get("page_title", "")
+            })
+
+            hits = await self.qdrant.search_vectors(
+                collection_name=collection_name,
+                query_vector=point["vector"],
+                limit=10,
+                score_threshold=similarity_threshold
+            )
+
+            for hit in hits:
+                hit_payload = hit.get("payload") or {}
+                hit_page_id = hit_payload.get("page_id")
+                if not hit_page_id or hit_page_id == page_id:
+                    continue
+                if hit_payload.get("doc_type", "wiki") != "wiki":
+                    continue
+
+                # Deduplicate the A->B / B->A chunk pair directions
+                chunk_pair = tuple(sorted((point["id"], hit["id"])))
+                if chunk_pair in seen_chunk_pairs:
+                    continue
+                seen_chunk_pairs.add(chunk_pair)
+
+                page_meta.setdefault(hit_page_id, {
+                    "page_id": hit_page_id,
+                    "path": hit_payload.get("page_path", ""),
+                    "title": hit_payload.get("page_title", "")
+                })
+
+                page_pair = tuple(sorted((page_id, hit_page_id)))
+                stats = pair_stats.setdefault(page_pair, {
+                    "max_similarity": 0.0,
+                    "matching_chunk_pairs": 0
+                })
+                stats["max_similarity"] = max(stats["max_similarity"], hit["score"])
+                stats["matching_chunk_pairs"] += 1
+
+        groups = [
+            {
+                "pages": [page_meta[a], page_meta[b]],
+                "max_similarity": stats["max_similarity"],
+                "matching_chunk_pairs": stats["matching_chunk_pairs"]
+            }
+            for (a, b), stats in pair_stats.items()
+        ]
+        groups.sort(key=lambda g: g["max_similarity"], reverse=True)
+
+        logger.info(
+            f"Duplicate scan for {user}: {len(wiki_points)} chunks scanned, "
+            f"{len(groups)} page pairs above {similarity_threshold}"
+        )
+
+        return {
+            "chunks_scanned": len(wiki_points),
+            "duplicate_groups": groups[:max_pairs]
+        }
+
     def find_chunks_without_graph_nodes(
         self,
         chunk_references: List[Dict[str, Any]],

src/services/volatile_fetch_service.py

@@ -5,9 +5,10 @@ Orchestrates fetching data from external APIs and storing in volatile cache.
 Called by scheduler for prefetch or by HybridRAG for reactive caching.
 """
 
+import asyncio
 import logging
 from typing import Optional
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 
 from src.apis import (
     OpenMeteoProvider,
@@ -36,6 +37,16 @@ class FetchResult:
     error: Optional[str] = None
 
 
+@dataclass
+class EnvironmentFetchResult:
+    """Result of combined environment fetch (weather + air quality)."""
+    success: bool
+    key: str
+    weather: Optional[FetchResult] = None
+    air_quality: Optional[FetchResult] = None
+    errors: list[str] = field(default_factory=list)
+
+
 class VolatileFetchService:
     """
     Service to fetch external data and store in volatile cache.
@@ -67,12 +78,86 @@ class VolatileFetchService:
         self.news = news_provider
         self.financial = financial_provider
 
-    async def fetch_weather(
+    async def fetch_current_weather(
+        self,
+        user: str,
+        city: str,
+        ttl: int = 3600,  # 1 hour
+    ) -> FetchResult:
+        """
+        Fetch current weather conditions for a city and store in volatile cache.
+
+        Args:
+            user: User identifier
+            city: City name (will be geocoded)
+            ttl: Time-to-live in seconds (default 1 hour)
+
+        Returns:
+            FetchResult with success status and stored record
+        """
+        try:
+            # Geocode city and get current conditions
+            location = await self.weather.geocode(city)
+            if not location:
+                return FetchResult(
+                    success=False,
+                    namespace="weather",
+                    key=city.lower(),
+                    error=f"Could not geocode city: {city}"
+                )
+
+            current = await self.weather.get_current(location)
+
+            # Generate natural language summary
+            text = current.to_text()
+
+            # Convert to storage format
+            data = {
+                "temperature": current.temperature,
+                "feels_like": current.feels_like,
+                "humidity": current.humidity,
+                "wind_speed": current.wind_speed,
+                "wind_direction": current.wind_direction,
+                "conditions": current.condition_text,
+                "condition_code": current.condition.value,
+                "uv_index": current.uv_index,
+                "location": current.location,
+                "text": text,
+            }
+
+            # Store in volatile cache
+            record = await self.volatile.store(
+                user=user,
+                namespace=VolatileNamespace.WEATHER,
+                key=city.lower(),
+                data=data,
+                source="openmeteo",
+                ttl=ttl,
+            )
+
+            logger.info(f"Stored current weather for {city} (user={user})")
+            return FetchResult(
+                success=True,
+                namespace="weather",
+                key=city.lower(),
+                record=record
+            )
+
+        except Exception as e:
+            logger.error(f"Failed to fetch current weather for {city}: {e}")
+            return FetchResult(
+                success=False,
+                namespace="weather",
+                key=city.lower(),
+                error=str(e)
+            )
+
+    async def fetch_forecast(
         self,
         user: str,
         city: str,
         days: int = 7,
-        ttl: int = 86400,  # 24 hours
+        ttl: int = 43200,  # 12 hours
     ) -> FetchResult:
         """
         Fetch weather forecast for a city and store in volatile cache.
@@ -81,7 +166,7 @@ class VolatileFetchService:
             user: User identifier
             city: City name (will be geocoded)
             days: Number of forecast days (1-16)
-            ttl: Time-to-live in seconds
+            ttl: Time-to-live in seconds (default 12 hours)
 
         Returns:
             FetchResult with success status and stored record
@@ -92,13 +177,12 @@ class VolatileFetchService:
             if not location:
                 return FetchResult(
                     success=False,
-                    namespace="weather",
+                    namespace="forecast",
                     key=city.lower(),
                     error=f"Could not geocode city: {city}"
                 )
 
             forecast = await self.weather.get_forecast(location, days=days)
-            current = forecast.current
 
             # Build daily forecast array
             daily_forecasts = []
@@ -116,32 +200,23 @@ class VolatileFetchService:
                 })
 
             # Generate natural language summary
-            forecast_lines = [current.to_text()]
-            for day in forecast.daily[:5]:  # First 5 days
+            forecast_lines = [f"{city} {days}-day forecast:"]
+            for day in forecast.daily:
                 forecast_lines.append(day.to_text())
             text = "\n".join(forecast_lines)
 
             # Convert to storage format
             data = {
-                "current": {
-                    "temperature": current.temperature,
-                    "feels_like": current.feels_like,
-                    "humidity": current.humidity,
-                    "wind_speed": current.wind_speed,
-                    "wind_direction": current.wind_direction,
-                    "conditions": current.condition_text,
-                    "condition_code": current.condition.value,
-                    "uv_index": current.uv_index,
-                },
+                "days": days,
                 "daily": daily_forecasts,
-                "location": current.location,
+                "location": forecast.current.location,
                 "text": text,
             }
 
             # Store in volatile cache
             record = await self.volatile.store(
                 user=user,
-                namespace=VolatileNamespace.WEATHER,
+                namespace=VolatileNamespace.FORECAST,
                 key=city.lower(),
                 data=data,
                 source="openmeteo",
@@ -151,16 +226,16 @@ class VolatileFetchService:
             logger.info(f"Stored {days}-day forecast for {city} (user={user})")
             return FetchResult(
                 success=True,
-                namespace="weather",
+                namespace="forecast",
                 key=city.lower(),
                 record=record
             )
 
         except Exception as e:
-            logger.error(f"Failed to fetch weather for {city}: {e}")
+            logger.error(f"Failed to fetch forecast for {city}: {e}")
             return FetchResult(
                 success=False,
-                namespace="weather",
+                namespace="forecast",
                 key=city.lower(),
                 error=str(e)
             )
@@ -532,3 +607,130 @@ class VolatileFetchService:
                 key=city.lower(),
                 error=str(e)
             )
+
+    async def fetch_environment(
+        self,
+        user: str,
+        city: str,
+        weather_ttl: int = 3600,
+        air_quality_ttl: int = 3600,
+    ) -> EnvironmentFetchResult:
+        """
+        Fetch weather and air quality concurrently for a city.
+
+        Performs a single geocode lookup and fetches both weather and air quality
+        data in parallel, storing both in volatile cache.
+
+        Args:
+            user: User identifier
+            city: City name (will be geocoded once)
+            weather_ttl: TTL for weather data (default 1 hour)
+            air_quality_ttl: TTL for air quality data (default 1 hour)
+
+        Returns:
+            EnvironmentFetchResult with both weather and air quality results
+        """
+        errors: list[str] = []
+        key = city.lower()
+
+        # Single geocode lookup (shared by both fetches)
+        try:
+            location = await self.weather.geocode(city)
+            if not location:
+                return EnvironmentFetchResult(
+                    success=False,
+                    key=key,
+                    errors=[f"Could not geocode city: {city}"]
+                )
+        except Exception as e:
+            return EnvironmentFetchResult(
+                success=False,
+                key=key,
+                errors=[f"Geocoding failed: {e}"]
+            )
+
+        # Fetch weather and air quality concurrently
+        async def fetch_weather_data() -> FetchResult:
+            try:
+                current = await self.weather.get_current(location)
+                text = current.to_text()
+                data = {
+                    "temperature": current.temperature,
+                    "feels_like": current.feels_like,
+                    "humidity": current.humidity,
+                    "wind_speed": current.wind_speed,
+                    "wind_direction": current.wind_direction,
+                    "conditions": current.condition_text,
+                    "condition_code": current.condition.value,
+                    "uv_index": current.uv_index,
+                    "location": current.location,
+                    "text": text,
+                }
+                record = await self.volatile.store(
+                    user=user,
+                    namespace=VolatileNamespace.WEATHER,
+                    key=key,
+                    data=data,
+                    source="openmeteo",
+                    ttl=weather_ttl,
+                )
+                return FetchResult(success=True, namespace="weather", key=key, record=record)
+            except Exception as e:
+                return FetchResult(success=False, namespace="weather", key=key, error=str(e))
+
+        async def fetch_air_quality_data() -> FetchResult:
+            try:
+                air_quality = await self.weather.get_air_quality(location)
+                data = {
+                    "location": air_quality.location,
+                    "aqi_european": air_quality.aqi_european,
+                    "aqi_us": air_quality.aqi_us,
+                    "pm2_5": air_quality.pm2_5,
+                    "pm10": air_quality.pm10,
+                    "ozone": air_quality.ozone,
+                    "nitrogen_dioxide": air_quality.nitrogen_dioxide,
+                    "sulphur_dioxide": air_quality.sulphur_dioxide,
+                    "carbon_monoxide": air_quality.carbon_monoxide,
+                    "pollen_grass": air_quality.pollen_grass,
+                    "pollen_birch": air_quality.pollen_birch,
+                    "pollen_alder": air_quality.pollen_alder,
+                    "text": air_quality.to_text(),
+                }
+                record = await self.volatile.store(
+                    user=user,
+                    namespace=VolatileNamespace.AIR_QUALITY,
+                    key=key,
+                    data=data,
+                    source="openmeteo",
+                    ttl=air_quality_ttl,
+                )
+                return FetchResult(success=True, namespace="air_quality", key=key, record=record)
+            except Exception as e:
+                return FetchResult(success=False, namespace="air_quality", key=key, error=str(e))
+
+        # Run both fetches concurrently
+        weather_result, air_quality_result = await asyncio.gather(
+            fetch_weather_data(),
+            fetch_air_quality_data(),
+        )
+
+        # Collect any errors
+        if not weather_result.success:
+            errors.append(f"Weather: {weather_result.error}")
+        if not air_quality_result.success:
+            errors.append(f"Air quality: {air_quality_result.error}")
+
+        success = weather_result.success or air_quality_result.success
+        logger.info(
+            f"Environment fetch for {city} (user={user}): "
+            f"weather={'ok' if weather_result.success else 'failed'}, "
+            f"air_quality={'ok' if air_quality_result.success else 'failed'}"
+        )
+
+        return EnvironmentFetchResult(
+            success=success,
+            key=key,
+            weather=weather_result,
+            air_quality=air_quality_result,
+            errors=errors,
+        )

src/services/volatile_service.py

@@ -58,8 +58,15 @@ class VolatileCacheService:
         logger.info("Initialized VolatileCacheService (Qdrant backend)")
 
     def _collection_name(self, user: str) -> str:
-        """Get volatile collection name for user."""
-        return f"{self.COLLECTION_PREFIX}{user}"
+        """
+        Get volatile collection name for user.
+
+        The user id is sanitized (same rules as the document collections)
+        so raw identifiers cannot alias or escape the per-tenant
+        collection naming scheme.
+        """
+        from src.core.multi_tenancy import sanitize_user_id
+        return f"{self.COLLECTION_PREFIX}{sanitize_user_id(user)}"
 
     def _make_vector_id(self, namespace: str, key: str) -> str:
         """

src/services/wiki_change_listener.py

@@ -27,21 +27,59 @@ class WikiChangeListener:
     NOTIFY events on INSERT/UPDATE/DELETE to the pages table.
     """
 
+    CHANNEL = 'wiki_page_changes'
+
+    # Bounded exponential backoff between reconnect attempts.
+    BACKOFF_INITIAL_SECONDS = 1.0
+    BACKOFF_MAX_SECONDS = 60.0
+
     def __init__(self):
         self.settings = get_settings()
         self.connection: Optional[asyncpg.Connection] = None
-        self.running = False
 
         # Loop prevention: Track recently processed pages
         # Key: page_id, Value: timestamp of last processing
         self._recent_notifications = {}
         self._debounce_seconds = self.settings.wikijs_change_listener_debounce_seconds
 
-    async def start(self):
-        """Start listening to database changes."""
-        logger.info("Starting Wiki.js database change listener")
+        # Supervision state. A single LISTEN connection does not heal itself the
+        # way an asyncpg pool does, so the drop has to be detected and repaired
+        # explicitly — see _supervise().
+        self._stopping = False
+        self._disconnected = asyncio.Event()
+        self._supervisor_task: Optional[asyncio.Task] = None
+        self._disconnected_at: Optional[datetime] = None
+        self.reconnects = 0
+        self.last_gap_seconds: Optional[float] = None
 
-        # Connect to Wiki.js PostgreSQL database
+    @property
+    def running(self) -> bool:
+        """
+        Whether a live subscription actually exists.
+
+        Derived rather than assigned. The previous implementation set a flag once
+        in start() and never revisited it, so after the connection dropped the
+        listener reported itself as running while being deaf to every event.
+        """
+        return (
+            not self._stopping
+            and self.connection is not None
+            and not self.connection.is_closed()
+        )
+
+    async def start(self):
+        """Start listening to database changes, and keep listening."""
+        logger.info("Starting Wiki.js database change listener")
+        self._stopping = False
+        self._disconnected.clear()
+
+        await self._connect()
+
+        self._supervisor_task = asyncio.create_task(self._supervise())
+        logger.info("Listening for Wiki.js page changes via PostgreSQL NOTIFY")
+
+    async def _connect(self):
+        """Open a connection and subscribe. Raises if the database is unreachable."""
         self.connection = await asyncpg.connect(
             host=self.settings.wikijs_db_host,
             port=self.settings.wikijs_db_port,
@@ -50,18 +88,136 @@ class WikiChangeListener:
             database=self.settings.wikijs_db_name
         )
 
-        # Listen to the wiki_page_changes channel
-        await self.connection.add_listener('wiki_page_changes', self._handle_notification)
+        await self.connection.add_listener(self.CHANNEL, self._handle_notification)
 
-        self.running = True
-        logger.info("Listening for Wiki.js page changes via PostgreSQL NOTIFY")
+        # Must be re-registered on every connection: asyncpg clears its
+        # termination listeners as soon as it fires them, so this is one-shot.
+        self.connection.add_termination_listener(self._on_connection_lost)
+
+    def _on_connection_lost(self, connection):
+        """
+        Called by asyncpg when the connection terminates.
+
+        Dispatched through loop.call_soon, so it must stay synchronous — the work
+        of reconnecting belongs to _supervise(), which this only wakes.
+        """
+        if self._stopping:
+            return
+
+        self._disconnected_at = datetime.now()
+        logger.error(
+            "Wiki.js change listener lost its database connection — "
+            "page changes are NOT being processed until it reconnects"
+        )
+        self._disconnected.set()
+
+    async def _supervise(self, max_iterations: Optional[int] = None) -> int:
+        """
+        Reconnect whenever the subscription drops.
+
+        Args:
+            max_iterations: Stop after N reconnect cycles (None = run forever;
+                            used by tests)
+
+        Returns:
+            Number of completed reconnect cycles
+        """
+        iterations = 0
+        while max_iterations is None or iterations < max_iterations:
+            await self._disconnected.wait()
+            if self._stopping:
+                break
+
+            self._disconnected.clear()
+            await self._reconnect_with_backoff()
+            iterations += 1
+
+        return iterations
+
+    async def _reconnect_with_backoff(self, max_attempts: Optional[int] = None) -> bool:
+        """
+        Re-establish the subscription, backing off between failures.
+
+        Keeps trying indefinitely by default: a database that is down for
+        maintenance will come back, and giving up would recreate exactly the
+        silent-deafness this supervision exists to prevent.
+        """
+        delay = self.BACKOFF_INITIAL_SECONDS
+        attempts = 0
+
+        while not self._stopping and (max_attempts is None or attempts < max_attempts):
+            attempts += 1
+            await self._close_connection()
+
+            try:
+                await self._connect()
+            except Exception as e:
+                logger.warning(
+                    f"Wiki.js change listener reconnect attempt {attempts} failed: {e}; "
+                    f"retrying in {delay:.0f}s"
+                )
+                await asyncio.sleep(delay)
+                delay = min(delay * 2, self.BACKOFF_MAX_SECONDS)
+                continue
+
+            self.reconnects += 1
+            gap = None
+            if self._disconnected_at is not None:
+                gap = (datetime.now() - self._disconnected_at).total_seconds()
+                self.last_gap_seconds = gap
+            self._disconnected_at = None
+
+            # NOTIFY is fire-and-forget: anything emitted while we were gone was
+            # delivered to nobody and cannot be replayed. Say so, and say what
+            # closes the gap, rather than reporting a clean recovery.
+            outage = f" after {gap:.0f}s" if gap is not None else ""
+            logger.warning(
+                f"Wiki.js change listener reconnected{outage} "
+                f"(reconnect #{self.reconnects}). NOTIFY events emitted during the "
+                f"outage were lost and cannot be replayed — run "
+                f"POST /maintenance/integrity-check to reconcile pages that "
+                f"changed while the listener was down."
+            )
+            return True
+
+        return False
+
+    async def _close_connection(self):
+        """Drop the current connection, tolerating one that is already dead."""
+        if not self.connection:
+            return
+
+        try:
+            if not self.connection.is_closed():
+                await self.connection.remove_listener(
+                    self.CHANNEL, self._handle_notification
+                )
+                await self.connection.close()
+        except Exception as e:
+            # A terminated connection raises on both calls; that is expected here.
+            logger.debug(f"Error closing Wiki.js listener connection: {e}")
+        finally:
+            self.connection = None
 
     async def stop(self):
         """Stop listening and close connection."""
-        if self.connection:
-            await self.connection.remove_listener('wiki_page_changes', self._handle_notification)
-            await self.connection.close()
-        self.running = False
+        self._stopping = True
+
+        # Wake the supervisor so it observes _stopping and exits rather than
+        # racing us to reconnect the connection we are about to close.
+        self._disconnected.set()
+
+        if self._supervisor_task:
+            self._supervisor_task.cancel()
+            try:
+                await self._supervisor_task
+            except asyncio.CancelledError:
+                pass
+            except Exception as e:
+                logger.debug(f"Wiki.js listener supervisor ended with: {e}")
+            self._supervisor_task = None
+
+        await self._close_connection()
         logger.info("Stopped Wiki.js change listener")
 
     async def _handle_notification(self, connection, pid, channel, payload):
@@ -103,8 +259,17 @@ class WikiChangeListener:
             }
             event = event_map.get(operation, 'page.update')
 
-            # Extract user from email
-            user = user_email.split('@')[0] if '@' in user_email else 'jpmschweitzer'
+            # Extract user from email. There is NO default tenant: if no user
+            # can be derived from the notification, skip processing instead of
+            # attributing the change to an arbitrary tenant.
+            if '@' in user_email and user_email.split('@')[0].strip():
+                user = user_email.split('@')[0].strip()
+            else:
+                logger.warning(
+                    f"Skipping page {page_id} change: cannot derive tenant user "
+                    f"from notification email {user_email!r}"
+                )
+                return
 
             # Process the change
             await self._process_page_change(

src/services/wiki_page_writer.py

@@ -34,7 +34,7 @@ class WikiPageWriter:
             settings: Application settings
         """
         self.ollama = ollama_client
-        self.model = settings.ollama_model
+        self.model = settings.ollama_llm_model
 
     async def create_page(
         self,

src/services/wiki_service.py

@@ -12,7 +12,7 @@ from typing import List, Optional, Dict, Any
 import logging
 
 from src.clients.wikijs_client import WikiJSClient
-from src.core.multi_tenancy import get_wikijs_namespace, validate_user_id, DEFAULT_USER
+from src.core.multi_tenancy import get_wikijs_namespace, validate_user_id
 from src.models.wiki import (
     WikiPage, WikiPageSummary, WikiPageList,
     WikiPageCreate, WikiPageUpdate,
@@ -184,7 +184,7 @@ class WikiService:
         Raises:
             ValueError: If creation fails
         """
-        user = page_data.user or DEFAULT_USER
+        user = page_data.user
 
         # Ensure path is in user's namespace
         full_path = self._ensure_user_path(page_data.path, user)

static/wikijs-integration.js

@@ -2,21 +2,28 @@
  * Library Desk Integration for Wiki.js
  * Combined re-index and entity linking buttons
  *
- * Usage: Add to Wiki.js Code Injection:
- * <script src="http://192.168.86.149:8089/static/wikijs-integration.js"></script>
+ * Usage: Add to Wiki.js Code Injection (served same-origin behind Authentik):
+ * <script src="/library-desk/static/wikijs-integration.js"></script>
+ *
+ * Auth: none in the browser. Requests go same-origin through the NPM
+ * /library-desk/ location, which is gated by Authentik forward-auth with the
+ * LAN bypass — external users are authenticated, LAN users pass through, and
+ * library-desk trusts the proxy marker header. No API key is embedded here.
  */
 (function() {
   'use strict';
 
-  // Auto-detect Library Desk URL
+  // Same-origin base: the script is served from <origin>/library-desk/static/...,
+  // so strip '/static/...' to get the library-desk mount point on this origin.
   const scriptTag = document.currentScript;
   const scriptUrl = scriptTag ? scriptTag.src : '';
-  const libraryDeskUrl = scriptUrl ? scriptUrl.split('/static/')[0] : 'http://192.168.86.149:8089';
+  const libraryDeskUrl = scriptUrl
+    ? scriptUrl.replace(/^https?:\/\/[^/]+/, '').split('/static/')[0]
+    : '/library-desk';
 
   // Shared configuration
   const CONFIG = window.LIBRARY_DESK_CONFIG || {
     libraryDeskUrl: libraryDeskUrl,
-    apiKey: 'af88ed8f44bed81bdb20d0534f1c4547340b29e2aba4963f61a71b993d7eb6e5',
     user: 'jpmschweitzer',
     buttonPosition: 'toolbar', // 'toolbar' or 'floating'
     debug: true
@@ -229,8 +236,8 @@
       // Re-index directly
       const response = await fetch(CONFIG.libraryDeskUrl + '/ingest/page', {
         method: 'POST',
+        credentials: 'same-origin',
         headers: {
-          'Authorization': 'Bearer ' + CONFIG.apiKey,
           'Content-Type': 'application/json'
         },
         body: JSON.stringify({
@@ -334,8 +341,8 @@
       // Call entity linking endpoint
       const response = await fetch(CONFIG.libraryDeskUrl + '/entity-linking/link-page', {
         method: 'POST',
+        credentials: 'same-origin',
         headers: {
-          'Authorization': 'Bearer ' + CONFIG.apiKey,
           'Content-Type': 'application/json'
         },
         body: JSON.stringify({

tests/conftest.py

@@ -1,21 +1,308 @@
-"""Pytest configuration and shared fixtures for Library Desk tests."""
+"""
+Pytest configuration and shared fixtures for Library Desk tests.
 
+TENANT SAFETY MODEL
+===================
+There is no separate test infrastructure: integration tests run against the
+SHARED production services (Qdrant / Neo4j / Wiki.js / Redis / Wiki.js).
+Tenancy is the ONLY isolation wall, therefore:
+
+- The suite is pinned to the reserved test tenant ``llm_tester`` (env
+  ``TEST_TENANT`` may only select a tenant inside the reserved
+  ``llm_tester*`` namespace — anything else aborts the whole session).
+- The production tenant ``jpmschweitzer`` is NEVER written to. A session
+  guard hard-fails immediately if the effective tenant is the production
+  tenant or outside the reserved namespace.
+- Integration tests (marked ``integration``) only run when
+  ``RUN_INTEGRATION_TESTS=1`` is set; otherwise they are skipped. Offline
+  unit tests never contact the shared services.
+- A session-scoped teardown deletes ALL ``llm_tester`` artifacts created
+  during the run (Qdrant ``*_llm_tester`` collections, Neo4j nodes under
+  the ``User_Llm_Tester*`` labels, the ``users/llm_tester`` wiki subtree,
+  and ``llm_tester``-prefixed Redis keys on the service DB), with a hard
+  assertion on the tenant string before any delete.
+
+HOSTS
+=====
+``TEST_HOST`` selects where the shared services live. It defaults to
+``localhost`` (safe: nothing listens there unless you forwarded the
+services yourself). For live runs against the shared stack set it
+explicitly, e.g. ``TEST_HOST=192.168.86.149``.
+
+The API under test is the LOCAL wakeup server (``./wakeup.sh``, port 8778),
+selected via ``LIBRARY_DESK_URL`` (default ``http://localhost:8778``).
+NEVER point tests at the production container (port 8089).
+"""
+
+import asyncio
+import logging
 import os
-import pytest
-import pytest_asyncio
-from typing import AsyncGenerator
 
-# Test configuration
+import pytest
+
+from src.core.multi_tenancy import sanitize_user_id
+
+logger = logging.getLogger(__name__)
+
 pytest_plugins = ("pytest_asyncio",)
 
-# Use real host for tests (services available at this IP)
-TEST_HOST = os.environ.get("TEST_HOST", "192.168.86.149")
+# ---------------------------------------------------------------------------
+# Tenancy constants
+# ---------------------------------------------------------------------------
+
+#: The production tenant. No test may ever write under it.
+PRODUCTION_TENANT = "jpmschweitzer"
+
+#: The reserved test tenant namespace. The effective tenant must be
+#: exactly this or a sub-tenant of it (llm_tester_*).
+RESERVED_TEST_TENANT = "llm_tester"
+
+#: Effective tenant for the whole suite (guard-checked below).
+TEST_TENANT = os.environ.get("TEST_TENANT", RESERVED_TEST_TENANT)
+
+# ---------------------------------------------------------------------------
+# Hosts / URLs
+# ---------------------------------------------------------------------------
+
+#: Shared-services host. Default localhost — NOT the production host.
+TEST_HOST = os.environ.get("TEST_HOST", "localhost")
+
+#: Base URL of the local dev server under test (./wakeup.sh, port 8778).
+LIBRARY_DESK_URL = os.environ.get("LIBRARY_DESK_URL", "http://localhost:8778")
+
+RUN_INTEGRATION = os.environ.get("RUN_INTEGRATION_TESTS") == "1"
+
+
+def is_reserved_test_tenant(tenant: str) -> bool:
+    """True if tenant is inside the reserved llm_tester namespace."""
+    sanitized = sanitize_user_id(tenant)
+    return sanitized == RESERVED_TEST_TENANT or sanitized.startswith(
+        RESERVED_TEST_TENANT + "_"
+    )
+
+
+def assert_safe_test_tenant(tenant: str) -> str:
+    """
+    Hard assertion used before ANY destructive operation.
+
+    Raises AssertionError unless the tenant is inside the reserved test
+    namespace and is not the production tenant.
+    """
+    sanitized = sanitize_user_id(tenant)
+    assert sanitized != sanitize_user_id(PRODUCTION_TENANT), (
+        f"TENANT GUARD: refusing to touch production tenant {tenant!r}"
+    )
+    assert is_reserved_test_tenant(tenant), (
+        f"TENANT GUARD: {tenant!r} is not in the reserved test namespace "
+        f"({RESERVED_TEST_TENANT}*)"
+    )
+    return sanitized
+
+
+# ---------------------------------------------------------------------------
+# Session guard + integration gating
+# ---------------------------------------------------------------------------
+
+
+def pytest_collection_modifyitems(config, items):
+    """Skip integration-marked tests unless explicitly enabled AND safe."""
+    if RUN_INTEGRATION and is_reserved_test_tenant(TEST_TENANT):
+        return
+    reason = (
+        "integration tests disabled (set RUN_INTEGRATION_TESTS=1, TEST_HOST "
+        "and TEST_TENANT inside the reserved llm_tester namespace to run "
+        "against the shared services)"
+    )
+    skip_marker = pytest.mark.skip(reason=reason)
+    for item in items:
+        if "integration" in item.keywords:
+            item.add_marker(skip_marker)
+
+
+@pytest.fixture(scope="session", autouse=True)
+def tenant_guard():
+    """
+    Session guard: hard-fail the entire run if the effective tenant is the
+    production tenant or outside the reserved test namespace.
+    """
+    if sanitize_user_id(TEST_TENANT) == sanitize_user_id(PRODUCTION_TENANT):
+        pytest.exit(
+            f"TENANT GUARD: effective test tenant is the PRODUCTION tenant "
+            f"({TEST_TENANT!r}) - aborting the whole session.",
+            returncode=3,
+        )
+    if not is_reserved_test_tenant(TEST_TENANT):
+        pytest.exit(
+            f"TENANT GUARD: effective test tenant {TEST_TENANT!r} is not in "
+            f"the reserved test namespace ({RESERVED_TEST_TENANT}*) - "
+            f"aborting the whole session.",
+            returncode=3,
+        )
+    yield
+
+
+# ---------------------------------------------------------------------------
+# Session teardown: purge ALL llm_tester artifacts created during the run
+# ---------------------------------------------------------------------------
+
+
+async def _purge_qdrant_test_artifacts() -> None:
+    """Delete Qdrant collections belonging to the reserved test tenant."""
+    from qdrant_client import QdrantClient
+
+    tenant = assert_safe_test_tenant(TEST_TENANT)
+    client = QdrantClient(url=f"http://{TEST_HOST}:6333", timeout=10)
+    try:
+        for coll in client.get_collections().collections:
+            name = coll.name
+            # Only *_<tenant> style collections (library_desk_llm_tester,
+            # volatile_llm_tester, memories_llm_tester, ...).
+            if not (name.endswith(f"_{tenant}") or f"_{tenant}_" in name):
+                continue
+            assert PRODUCTION_TENANT not in name  # hard guard
+            assert tenant in name
+            client.delete_collection(name)
+            logger.info(f"[teardown] deleted Qdrant collection {name}")
+    finally:
+        client.close()
+
+
+async def _purge_neo4j_test_artifacts() -> None:
+    """Delete Neo4j nodes under the reserved test tenant's labels."""
+    from src.clients.neo4j_client import Neo4jClient
+    from src.config import get_settings
+    from src.core.multi_tenancy import get_neo4j_user_base_label
+
+    tenant = assert_safe_test_tenant(TEST_TENANT)
+    label_prefix = get_neo4j_user_base_label(tenant)  # e.g. User_Llm_Tester
+    assert "Jpmschweitzer" not in label_prefix  # hard guard
+    assert "Llm_Tester" in label_prefix
+
+    settings = get_settings()
+    client = Neo4jClient(
+        uri=f"bolt://{TEST_HOST}:7687",
+        user=settings.neo4j_user,
+        password=settings.neo4j_password,
+    )
+    try:
+        await client.connect()
+        result = await client.execute_write(
+            """
+            MATCH (n)
+            WHERE any(l IN labels(n) WHERE l STARTS WITH $prefix)
+            DETACH DELETE n
+            RETURN count(n) AS deleted
+            """,
+            {"prefix": label_prefix},
+        )
+        deleted = result[0]["deleted"] if result else 0
+        if deleted:
+            logger.info(f"[teardown] deleted {deleted} Neo4j {label_prefix}* nodes")
+    finally:
+        await client.close()
+
+
+async def _purge_wiki_test_artifacts() -> None:
+    """Delete the reserved test tenant's wiki subtree (users/llm_tester)."""
+    from src.clients.wikijs_client import WikiJSClient
+    from src.config import get_settings
+
+    tenant = assert_safe_test_tenant(TEST_TENANT)
+    settings = get_settings()
+    client = WikiJSClient(
+        base_url=settings.wikijs_url,
+        api_token=settings.wiki_graphql_api,
+    )
+    try:
+        for prefix in (f"users/{tenant}", f"users/{tenant.replace('_', '-')}"):
+            assert PRODUCTION_TENANT not in prefix  # hard guard
+            pages = await client.list_all_pages(path_prefix=prefix)
+            for page in pages:
+                path = page.get("path", "")
+                assert PRODUCTION_TENANT not in path  # hard guard
+                assert path.lstrip("/").startswith("users/llm")
+                await client.delete_page(page["id"])
+                logger.info(f"[teardown] deleted wiki page {path} (id={page['id']})")
+    finally:
+        await client.close()
+
+
+async def _purge_redis_test_artifacts() -> None:
+    """Delete llm_tester-prefixed keys on the Redis DB the service uses."""
+    import redis.asyncio as aioredis
+
+    from src.config import get_settings
+
+    tenant = assert_safe_test_tenant(TEST_TENANT)
+    settings = get_settings()
+    client = aioredis.from_url(
+        f"redis://{TEST_HOST}:6379/{settings.redis_db}",
+        encoding="utf-8",
+        decode_responses=True,
+    )
+    try:
+        deleted = 0
+        for pattern in (f"*{tenant}*", f"*{tenant.replace('_', '-')}*"):
+            async for key in client.scan_iter(match=pattern, count=200):
+                assert PRODUCTION_TENANT not in key  # hard guard
+                assert "llm" in key
+                await client.delete(key)
+                deleted += 1
+        if deleted:
+            logger.info(f"[teardown] deleted {deleted} Redis keys for {tenant}")
+    finally:
+        await client.aclose()
+
+
+@pytest.fixture(scope="session", autouse=True)
+def purge_test_tenant_artifacts(tenant_guard):
+    """
+    Session teardown: after the run, delete ALL artifacts under the reserved
+    test tenant on the shared services. Only active for integration runs
+    (offline unit-test runs never touch the shared services and must not
+    try to connect to them).
+    """
+    yield
+    if not RUN_INTEGRATION:
+        return
+
+    # Hard assertion before ANY delete.
+    assert_safe_test_tenant(TEST_TENANT)
+
+    async def _teardown():
+        for name, purge in (
+            ("qdrant", _purge_qdrant_test_artifacts),
+            ("neo4j", _purge_neo4j_test_artifacts),
+            ("wiki", _purge_wiki_test_artifacts),
+            ("redis", _purge_redis_test_artifacts),
+        ):
+            try:
+                await purge()
+            except AssertionError:
+                raise  # tenant-guard violations must never be swallowed
+            except Exception as e:
+                logger.warning(f"[teardown] {name} purge failed: {e}")
+
+    # Sync fixture + asyncio.run avoids the session/function loop-scope
+    # mismatch (see CLAUDE.md).
+    asyncio.run(_teardown())
+
+
+# ---------------------------------------------------------------------------
+# Shared fixtures
+# ---------------------------------------------------------------------------
 
 
 @pytest.fixture
 def test_user() -> str:
-    """Default test user."""
-    return "test_user"
+    """The reserved test tenant. Pinned for the whole suite."""
+    return TEST_TENANT
+
+
+@pytest.fixture
+def library_desk_url() -> str:
+    """Base URL of the LOCAL dev server under test (never production :8089)."""
+    return LIBRARY_DESK_URL
 
 
 @pytest.fixture
@@ -40,11 +327,11 @@ def qdrant_test_url() -> str:
 
 @pytest.fixture
 def wikijs_test_config() -> dict:
-    """Test Wiki.js configuration."""
+    """Test Wiki.js configuration (URL from settings, host env-overridable)."""
     from src.config import get_settings
     settings = get_settings()
     return {
-        "base_url": f"http://{TEST_HOST}:3000",
+        "base_url": settings.wikijs_url,
         "api_token": settings.wiki_graphql_api
     }
 
@@ -69,7 +356,9 @@ def ollama_test_config() -> dict:
 @pytest.fixture
 def redis_test_url() -> str:
     """Test Redis URL."""
-    return f"redis://{TEST_HOST}:6379/4"
+    from src.config import get_settings
+    settings = get_settings()
+    return f"redis://{TEST_HOST}:6379/{settings.redis_db}"
 
 
 @pytest.fixture
@@ -81,7 +370,7 @@ def sample_document() -> dict:
         "content": "This is a test document for unit testing.",
         "metadata": {
             "source": "test",
-            "author": "test_user"
+            "author": TEST_TENANT
         }
     }
 

tests/test_browser_auth.py

@@ -0,0 +1,57 @@
+"""
+Tests for verify_browser_request — the session/proxy auth used by the
+Wiki.js integration endpoints (no secret in the browser).
+"""
+import pytest
+from types import SimpleNamespace
+from fastapi import HTTPException
+from starlette.requests import Request
+
+from src.core.dependencies import verify_browser_request
+
+
+def _request(headers: dict) -> Request:
+    raw = [(k.lower().encode(), v.encode()) for k, v in headers.items()]
+    return Request({"type": "http", "method": "POST", "path": "/ingest/page", "headers": raw})
+
+
+SETTINGS = SimpleNamespace(library_api_key="server-secret-key")
+
+
+@pytest.mark.asyncio
+async def test_proxy_marker_with_authentik_identity_is_accepted():
+    req = _request({"X-Library-Desk-Proxy": "1", "X-Authentik-Email": "user@example.com"})
+    assert await verify_browser_request(req, SETTINGS) == "user@example.com"
+
+
+@pytest.mark.asyncio
+async def test_proxy_marker_on_lan_bypass_falls_back_to_lan():
+    # LAN bypass: proxy marker present, no Authentik identity headers.
+    req = _request({"X-Library-Desk-Proxy": "1"})
+    assert await verify_browser_request(req, SETTINGS) == "lan"
+
+
+@pytest.mark.asyncio
+async def test_valid_api_key_is_accepted_for_machine_callers():
+    req = _request({"Authorization": "Bearer server-secret-key"})
+    assert await verify_browser_request(req, SETTINGS) == "server-secret-key"
+
+
+@pytest.mark.asyncio
+async def test_no_marker_and_no_key_is_rejected():
+    with pytest.raises(HTTPException) as exc:
+        await verify_browser_request(_request({}), SETTINGS)
+    assert exc.value.status_code == 401
+
+
+@pytest.mark.asyncio
+async def test_forged_marker_value_is_rejected():
+    # Only the exact NPM-set value "1" is trusted.
+    with pytest.raises(HTTPException):
+        await verify_browser_request(_request({"X-Library-Desk-Proxy": "yes"}), SETTINGS)
+
+
+@pytest.mark.asyncio
+async def test_wrong_api_key_is_rejected():
+    with pytest.raises(HTTPException):
+        await verify_browser_request(_request({"Authorization": "Bearer wrong"}), SETTINGS)

tests/test_config.py

@@ -0,0 +1,62 @@
+"""
+Unit tests for application settings (offline).
+
+Covers the OLLAMA_MODEL env collision: the deployed container sets
+OLLAMA_MODEL=nomic-embed-text for embeddings, which must NOT shadow the
+generation model setting (ollama_llm_model / OLLAMA_LLM_MODEL).
+"""
+
+import pytest
+
+from src.config import Settings
+
+# Required fields so Settings can be constructed without a .env file
+REQUIRED = {
+    "library_api_key": "test-key",
+    "neo4j_password": "test-pass",
+    "wikijs_db_password": "test-pass",
+}
+
+
+@pytest.mark.unit
+class TestOllamaModelResolution:
+    """Generation model resolution must be immune to the OLLAMA_MODEL env var."""
+
+    def test_ollama_model_env_does_not_shadow_generation_model(self, monkeypatch):
+        """The container env OLLAMA_MODEL (embedding model) must not leak into ollama_llm_model."""
+        monkeypatch.setenv("OLLAMA_MODEL", "nomic-embed-text")
+
+        settings = Settings(_env_file=None, **REQUIRED)
+
+        assert settings.ollama_llm_model == "gemma4:e2b"
+        assert settings.ollama_llm_model != "nomic-embed-text"
+
+    def test_generation_model_default(self, monkeypatch):
+        monkeypatch.delenv("OLLAMA_LLM_MODEL", raising=False)
+
+        settings = Settings(_env_file=None, **REQUIRED)
+
+        assert settings.ollama_llm_model == "gemma4:e2b"
+
+    def test_generation_model_from_dedicated_env_var(self, monkeypatch):
+        """OLLAMA_LLM_MODEL is the dedicated env var for the generation model."""
+        monkeypatch.setenv("OLLAMA_MODEL", "nomic-embed-text")
+        monkeypatch.setenv("OLLAMA_LLM_MODEL", "mistral-nemo:latest")
+
+        settings = Settings(_env_file=None, **REQUIRED)
+
+        assert settings.ollama_llm_model == "mistral-nemo:latest"
+
+    def test_embedding_model_setting_untouched(self, monkeypatch):
+        """The embedding model keeps its own setting and env var."""
+        monkeypatch.setenv("OLLAMA_EMBEDDING_MODEL", "nomic-embed-text")
+
+        settings = Settings(_env_file=None, **REQUIRED)
+
+        assert settings.ollama_embedding_model == "nomic-embed-text"
+
+    def test_legacy_setting_name_removed(self):
+        """The old ollama_model attribute must be gone so nothing binds to OLLAMA_MODEL."""
+        settings = Settings(_env_file=None, **REQUIRED)
+
+        assert not hasattr(settings, "ollama_model")

tests/test_consolidation.py

@@ -38,7 +38,7 @@ def settings():
     """Get mocked application settings for testing."""
     mock_settings = MagicMock()
     mock_settings.reranker_model = "mistral-nemo"
-    mock_settings.ollama_model = "mistral-nemo"
+    mock_settings.ollama_llm_model = "mistral-nemo"
     return mock_settings
 
 
@@ -47,6 +47,7 @@ def mock_neo4j():
     """Mock Neo4j client."""
     mock = AsyncMock()
     mock.execute_query = AsyncMock()
+    mock.execute_write = AsyncMock()
     return mock
 
 
@@ -520,8 +521,8 @@ async def test_mark_search_processed(consolidation_service, mock_neo4j):
     """Test marking search as processed."""
     await consolidation_service._mark_search_processed(TEST_SEARCH_ID)
 
-    mock_neo4j.execute_query.assert_called_once()
-    call_args = mock_neo4j.execute_query.call_args
+    mock_neo4j.execute_write.assert_called_once()
+    call_args = mock_neo4j.execute_write.call_args
     assert TEST_SEARCH_ID in str(call_args)
 
 

tests/test_consolidation_repair.py

@@ -0,0 +1,225 @@
+"""
+Offline regression tests for the consolidation-loop repair (Phase C item 4).
+
+Production failure mode being locked in:
+- The generation LLM was unavailable (OLLAMA_MODEL env collision made every
+  /api/generate call 400), classification returned empty, and the loop
+  STILL marked every SearchQuery processed - permanently draining the queue
+  with zero output. Every subsequent 30-minute run then logged
+  'No unprocessed searches found'.
+
+The repaired behavior:
+- LLM-infrastructure failure raises ConsolidationLLMUnavailableError, the
+  affected searches stay UNPROCESSED (retried next run), and the run reports
+  searches_deferred.
+- Successful classification still consumes searches.
+- Every run logs searches_processed and duration_ms.
+
+All clients are mocked - no shared services are contacted.
+"""
+
+import json
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.services.consolidation_service import (
+    ConsolidationLLMUnavailableError,
+    ConsolidationService,
+)
+
+TEST_USER = "llm_tester"
+
+
+def _search_row(search_id: str, query: str = "test query", web_count: int = 3):
+    return {
+        "id": search_id,
+        "query": query,
+        "user": TEST_USER,
+        "timestamp": "2026-07-14T00:00:00Z",
+        "total_results": web_count,
+        "web_count": web_count,
+        "keywords": ["test"],
+    }
+
+
+def _web_result_row(url: str = "https://example.com/a"):
+    return {
+        "url": url,
+        "title": "Example",
+        "content": "Example content about the query.",
+        "rank": 1,
+        "rrf_score": 0.5,
+    }
+
+
+def _make_service(searches, ollama_response):
+    """ConsolidationService with a scripted Neo4j and Ollama."""
+    neo4j = AsyncMock()
+    executed = []
+
+    async def fake_query(cypher, params=None):
+        executed.append((cypher, params or {}))
+        if "processed: false" in cypher:
+            return searches
+        if "FOUND]->(wr:WebResult)" in cypher:
+            return [_web_result_row()]
+        return []
+
+    neo4j.execute_query = AsyncMock(side_effect=fake_query)
+    neo4j.execute_write = AsyncMock(side_effect=fake_query)
+
+    ollama = AsyncMock()
+    ollama.generate_text = AsyncMock(return_value=ollama_response)
+
+    wiki = AsyncMock()
+    wiki.get_taxonomy_structure = AsyncMock(return_value={})
+
+    settings = MagicMock()
+    settings.ollama_llm_model = "gemma4:e2b"
+
+    service = ConsolidationService(
+        neo4j=neo4j, ollama=ollama, wiki=wiki, settings=settings
+    )
+    return service, executed
+
+
+def _mark_processed_calls(executed):
+    return [(c, p) for c, p in executed if "SET sq.processed = true" in c]
+
+
+class TestLLMUnavailableDoesNotConsumeSearches:
+    @pytest.mark.asyncio
+    async def test_searches_stay_unprocessed_when_llm_returns_none(self):
+        """generate_text -> None (transport/HTTP failure): defer, don't consume."""
+        searches = [_search_row("aaaa1111"), _search_row("bbbb2222")]
+        service, executed = _make_service(searches, ollama_response=None)
+
+        response = await service.consolidate_knowledge()
+
+        assert response.total_found == 2
+        assert response.processed_count == 0
+        assert response.searches_deferred == 2
+        assert response.errors and "unavailable" in response.errors[0].lower()
+        # THE regression guard: no search was marked processed
+        assert _mark_processed_calls(executed) == []
+
+    @pytest.mark.asyncio
+    async def test_searches_stay_unprocessed_when_llm_returns_empty(self):
+        """An empty completion is an infra anomaly, not 'nothing to route'."""
+        service, executed = _make_service([_search_row("cccc3333")], "")
+
+        response = await service.consolidate_knowledge()
+
+        assert response.searches_deferred == 1
+        assert _mark_processed_calls(executed) == []
+
+    @pytest.mark.asyncio
+    async def test_classification_raises_on_no_output(self):
+        service, _ = _make_service([], None)
+        with pytest.raises(ConsolidationLLMUnavailableError):
+            await service._classify_web_results_unified(
+                query="q", web_results=[_web_result_row()], keywords=[], user=TEST_USER
+            )
+
+    @pytest.mark.asyncio
+    async def test_batch_aborts_after_first_llm_failure(self):
+        """When the LLM is down it is down for all searches: one probe, then stop."""
+        searches = [_search_row(f"id{i}", web_count=5) for i in range(5)]
+        service, executed = _make_service(searches, ollama_response=None)
+
+        response = await service.consolidate_knowledge()
+
+        assert response.searches_deferred == 5
+        # Only the first search's classification was attempted
+        assert service.ollama.generate_text.await_count == 1
+
+
+class TestSuccessfulRunsStillConsume:
+    @pytest.mark.asyncio
+    async def test_valid_classification_marks_processed(self):
+        classification = json.dumps([{
+            "url": "https://example.com/a",
+            "title": "Example",
+            "route_type": "skip",
+            "confidence": 0.9,
+            "reason": "low value",
+        }])
+        service, executed = _make_service([_search_row("dddd4444")], classification)
+
+        response = await service.consolidate_knowledge()
+
+        assert response.total_found == 1
+        assert response.processed_count == 1
+        assert response.searches_deferred == 0
+        marked = _mark_processed_calls(executed)
+        assert len(marked) == 1
+        assert marked[0][1]["search_id"] == "dddd4444"
+
+    @pytest.mark.asyncio
+    async def test_unparseable_output_still_consumes_search(self):
+        """Model responded with junk: consume (avoid retrying a bad prompt forever)."""
+        service, executed = _make_service(
+            [_search_row("eeee5555")], "not json at all"
+        )
+
+        response = await service.consolidate_knowledge()
+
+        assert response.searches_deferred == 0
+        assert len(_mark_processed_calls(executed)) == 1
+
+    @pytest.mark.asyncio
+    async def test_skipped_low_web_search_still_consumed(self):
+        """Insufficient web results: intentionally consumed (existing behavior)."""
+        service, executed = _make_service(
+            [_search_row("ffff6666", web_count=0)], None
+        )
+
+        response = await service.consolidate_knowledge()
+
+        assert response.searches_deferred == 0
+        assert len(_mark_processed_calls(executed)) == 1
+        # LLM never called for a skipped search
+        service.ollama.generate_text.assert_not_awaited()
+
+
+class TestRunLogging:
+    @pytest.mark.asyncio
+    async def test_logs_searches_processed_and_duration(self, caplog):
+        service, _ = _make_service([], None)
+
+        with caplog.at_level("INFO"):
+            response = await service.consolidate_knowledge()
+
+        assert response.duration_ms >= 0
+        run_logs = [r.message for r in caplog.records
+                    if "Consolidation run complete" in r.message]
+        assert run_logs, "every run must emit the run-complete log line"
+        assert "searches_processed=0" in run_logs[0]
+        assert "duration_ms=" in run_logs[0]
+
+    @pytest.mark.asyncio
+    async def test_logs_on_deferred_run(self, caplog):
+        service, _ = _make_service([_search_row("gggg7777")], None)
+
+        with caplog.at_level("INFO"):
+            response = await service.consolidate_knowledge()
+
+        assert response.duration_ms >= 0
+        run_logs = [r.message for r in caplog.records
+                    if "Consolidation run complete" in r.message]
+        assert run_logs
+        assert "searches_deferred=1" in run_logs[0]
+
+    @pytest.mark.asyncio
+    async def test_lookback_parameter_is_utc_aware(self):
+        """The lookback boundary must be timezone-aware UTC (Neo4j datetime()
+        interprets naive strings as UTC, shifting the window on CET hosts)."""
+        service, executed = _make_service([], None)
+
+        await service.consolidate_knowledge(lookback_days=7)
+
+        find_calls = [(c, p) for c, p in executed if "processed: false" in c]
+        assert find_calls
+        lookback = find_calls[0][1]["lookback_date"]
+        assert "+00:00" in lookback or lookback.endswith("Z")

tests/test_content_extractor.py

@@ -1,11 +1,26 @@
 """Tests for ContentExtractor client."""
 
 import pytest
-from unittest.mock import AsyncMock, MagicMock, patch
+from unittest.mock import AsyncMock, patch
+
+import httpx
 
 from src.clients.content_extractor import ContentExtractor
 from src.models.content import ContentExtractionResult
 
+TEST_HTML = "<html><body><article>Test</article></body></html>"
+
+
+def _doc(text, **meta):
+    """bare_extraction-style result dict."""
+    return {
+        "text": text,
+        "title": meta.get("title"),
+        "author": meta.get("author"),
+        "date": meta.get("date"),
+        "language": meta.get("language"),
+    }
+
 
 @pytest.fixture
 def content_extractor():
@@ -20,6 +35,9 @@ class TestContentExtractor:
         """Test ContentExtractor initialization."""
         assert content_extractor.timeout == 5
         assert content_extractor.max_length == 2000
+        assert content_extractor.max_urls_per_batch == (
+            ContentExtractor.DEFAULT_MAX_URLS_PER_BATCH
+        )
 
     @pytest.mark.asyncio
     async def test_extract_success(self, content_extractor):
@@ -27,31 +45,48 @@ class TestContentExtractor:
         test_url = "https://example.com/article"
         test_content = "This is the extracted article content."
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url.return_value = "<html><body>Test</body></html>"
-            mock_traf.extract.return_value = test_content
-            mock_traf.bare_extraction.return_value = {
-                "title": "Test Article",
-                "author": "John Doe",
-                "date": "2024-01-15",
-                "language": "en"
-            }
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc(
+                test_content,
+                title="Test Article",
+                author="John Doe",
+                date="2024-01-15",
+                language="en",
+            )
 
             result = await content_extractor.extract(test_url)
 
             assert result.success is True
             assert result.url == test_url
             assert result.content == test_content
+            assert result.title == "Test Article"
             assert result.error is None
 
+    @pytest.mark.asyncio
+    async def test_extraction_runs_once_per_url(self, content_extractor):
+        """Trafilatura must parse the document exactly ONCE (the old code
+        ran extract() twice plus bare_extraction — three full parses)."""
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc("content")
+
+            await content_extractor.extract("https://example.com/a")
+
+            assert mock_traf.bare_extraction.call_count == 1
+            mock_traf.extract.assert_not_called()
+            mock_traf.fetch_url.assert_not_called()  # httpx fetches now
+
     @pytest.mark.asyncio
     async def test_extract_fetch_failure(self, content_extractor):
         """Test extraction when URL fetch fails."""
         test_url = "https://example.com/nonexistent"
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url.return_value = None
-
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=None)
+        ):
             result = await content_extractor.extract(test_url)
 
             assert result.success is False
@@ -59,14 +94,44 @@ class TestContentExtractor:
             assert result.content == ""
             assert "Failed to fetch URL" in result.error
 
+    @pytest.mark.asyncio
+    async def test_extract_fetch_timeout(self, content_extractor):
+        """A slow server hits the httpx timeout instead of pinning a
+        worker thread on a blind download."""
+        test_url = "https://example.com/slow-server"
+
+        with patch.object(
+            content_extractor,
+            "_fetch",
+            AsyncMock(side_effect=httpx.ReadTimeout("read timeout")),
+        ):
+            result = await content_extractor.extract(test_url)
+
+            assert result.success is False
+            assert "timed out" in result.error.lower()
+
+    @pytest.mark.asyncio
+    async def test_extract_network_error(self, content_extractor):
+        """Connection errors return a failed result, not an exception."""
+        with patch.object(
+            content_extractor,
+            "_fetch",
+            AsyncMock(side_effect=httpx.ConnectError("refused")),
+        ):
+            result = await content_extractor.extract("https://example.com/down")
+
+            assert result.success is False
+            assert "Failed to fetch URL" in result.error
+
     @pytest.mark.asyncio
     async def test_extract_no_content(self, content_extractor):
         """Test extraction when page has no extractable content."""
         test_url = "https://example.com/empty"
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url.return_value = "<html><body></body></html>"
-            mock_traf.extract.return_value = None
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = None
 
             result = await content_extractor.extract(test_url)
 
@@ -80,10 +145,10 @@ class TestContentExtractor:
         # Content longer than max_length (2000)
         long_content = "x" * 3000
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url.return_value = "<html><body>Test</body></html>"
-            mock_traf.extract.return_value = long_content
-            mock_traf.bare_extraction.return_value = {}
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc(long_content)
 
             result = await content_extractor.extract(test_url)
 
@@ -97,13 +162,13 @@ class TestContentExtractor:
         test_urls = [
             "https://example.com/article1",
             "https://example.com/article2",
-            "https://example.com/article3"
+            "https://example.com/article3",
         ]
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url.return_value = "<html><body>Test</body></html>"
-            mock_traf.extract.return_value = "Extracted content"
-            mock_traf.bare_extraction.return_value = {}
+        with patch.object(
+            content_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc("Extracted content")
 
             results = await content_extractor.extract_batch(test_urls)
 
@@ -113,8 +178,32 @@ class TestContentExtractor:
                 assert result.success is True
 
     @pytest.mark.asyncio
-    async def test_extract_timeout(self):
-        """Test extraction timeout handling."""
+    async def test_extract_batch_caps_full_page_extractions(self):
+        """URLs beyond the per-call cap are skipped (callers fall back to
+        the search snippet) instead of fanning out unbounded downloads."""
+        extractor = ContentExtractor(timeout=5, max_length=2000, max_urls_per_batch=2)
+        test_urls = [f"https://example.com/{i}" for i in range(5)]
+
+        with patch.object(
+            extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ) as mock_fetch, patch(
+            "src.clients.content_extractor.trafilatura"
+        ) as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc("content")
+
+            results = await extractor.extract_batch(test_urls)
+
+            assert len(results) == 5
+            assert mock_fetch.await_count == 2
+            assert [r.url for r in results] == test_urls
+            assert all(r.success for r in results[:2])
+            for skipped in results[2:]:
+                assert skipped.success is False
+                assert "cap" in skipped.error
+
+    @pytest.mark.asyncio
+    async def test_extract_parse_timeout(self):
+        """Test extraction (parse) timeout handling."""
         import time
 
         test_url = "https://example.com/slow"
@@ -122,12 +211,14 @@ class TestContentExtractor:
         # Create an extractor with very short timeout
         fast_extractor = ContentExtractor(timeout=0.001, max_length=2000)
 
-        def slow_fetch(url):
+        def slow_parse(*args, **kwargs):
             time.sleep(1)  # Sleep synchronously (this runs in thread pool)
-            return "<html></html>"
+            return _doc("late content")
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.fetch_url = slow_fetch
+        with patch.object(
+            fast_extractor, "_fetch", AsyncMock(return_value=TEST_HTML)
+        ), patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction = slow_parse
 
             result = await fast_extractor.extract(test_url)
 
@@ -139,16 +230,103 @@ class TestContentExtractor:
         """Test extraction from raw HTML."""
         test_html = "<html><body><article>Article content here.</article></body></html>"
 
-        with patch('src.clients.content_extractor.trafilatura') as mock_traf:
-            mock_traf.extract.return_value = "Article content here."
-            mock_traf.bare_extraction.return_value = {"title": "Test"}
+        with patch("src.clients.content_extractor.trafilatura") as mock_traf:
+            mock_traf.bare_extraction.return_value = _doc(
+                "Article content here.", title="Test"
+            )
 
-            result = await content_extractor.extract_from_html(test_html, url="https://example.com")
+            result = await content_extractor.extract_from_html(
+                test_html, url="https://example.com"
+            )
 
             assert result.success is True
             assert result.content == "Article content here."
 
 
+class TestFetchStreamingCap:
+    """The 5MB cap must abort the DOWNLOAD, not just truncate after it."""
+
+    def _extractor_with_transport(self, handler):
+        extractor = ContentExtractor(timeout=5, max_length=2000)
+        extractor._http = httpx.AsyncClient(
+            transport=httpx.MockTransport(handler)
+        )
+        return extractor
+
+    @pytest.mark.asyncio
+    async def test_download_aborts_past_cap(self):
+        from src.clients.content_extractor import MAX_RESPONSE_BYTES
+
+        chunk = b"x" * (1024 * 1024)  # 1MB per chunk
+        chunks_produced = []
+
+        async def body():
+            for i in range(100):  # 100MB on offer
+                chunks_produced.append(i)
+                yield chunk
+
+        def handler(request):
+            return httpx.Response(
+                200,
+                content=body(),
+                headers={"Content-Type": "text/html; charset=utf-8"},
+            )
+
+        extractor = self._extractor_with_transport(handler)
+        try:
+            text = await extractor._fetch("https://example.com/huge")
+        finally:
+            await extractor.close()
+
+        assert text is not None
+        assert len(text.encode()) == MAX_RESPONSE_BYTES
+        # Streaming stopped at the cap instead of consuming all 100 chunks
+        assert len(chunks_produced) <= (MAX_RESPONSE_BYTES // len(chunk)) + 1
+
+    @pytest.mark.asyncio
+    async def test_small_response_returned_whole(self):
+        def handler(request):
+            return httpx.Response(
+                200,
+                content=TEST_HTML.encode(),
+                headers={"Content-Type": "text/html; charset=utf-8"},
+            )
+
+        extractor = self._extractor_with_transport(handler)
+        try:
+            text = await extractor._fetch("https://example.com/small")
+        finally:
+            await extractor.close()
+
+        assert text == TEST_HTML
+
+    @pytest.mark.asyncio
+    async def test_non_200_returns_none(self):
+        def handler(request):
+            return httpx.Response(404, content=b"not found")
+
+        extractor = self._extractor_with_transport(handler)
+        try:
+            text = await extractor._fetch("https://example.com/missing")
+        finally:
+            await extractor.close()
+
+        assert text is None
+
+    @pytest.mark.asyncio
+    async def test_empty_body_returns_none(self):
+        def handler(request):
+            return httpx.Response(200, content=b"")
+
+        extractor = self._extractor_with_transport(handler)
+        try:
+            text = await extractor._fetch("https://example.com/empty")
+        finally:
+            await extractor.close()
+
+        assert text is None
+
+
 class TestContentExtractionResult:
     """Tests for ContentExtractionResult model."""
 

tests/test_document_sync.py

@@ -0,0 +1,176 @@
+"""
+Offline unit tests for DocumentSyncService vector indexing fixes.
+
+Pins the Phase D fixes:
+- ensure_collection is actually awaited (it used to be a bare coroutine
+  that never ran, so fresh tenants had no collection at upsert time)
+- None entries from embed_batch are filtered out instead of poisoning
+  the whole batch upsert (one failed chunk aborted the document)
+- delete-LAST reindex order (same as the wiki fix): new points are
+  upserted with deterministic uuid5 ids BEFORE stale points are pruned,
+  so a failed embedding pass can no longer leave a document with zero
+  vectors (the old order ran delete_by_filter first)
+"""
+
+import uuid
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.services.document_sync_service import DocumentSyncService
+
+TENANT = "llm_tester"
+TENANT_COLLECTION = "library_desk_llm_tester"
+
+
+@pytest.fixture
+def mock_qdrant():
+    qdrant = MagicMock()
+    qdrant.ensure_collection = AsyncMock()
+    qdrant.upsert_points = AsyncMock(side_effect=lambda collection_name, points: len(points))
+    qdrant.scroll_all_points = AsyncMock(return_value=[])
+    qdrant.delete_by_ids = AsyncMock(side_effect=lambda collection_name, point_ids: len(point_ids))
+    return qdrant
+
+
+@pytest.fixture
+def mock_ollama():
+    ollama = MagicMock()
+    ollama.embed_batch = AsyncMock(
+        side_effect=lambda texts: [[0.1] * 768 for _ in texts]
+    )
+    return ollama
+
+
+@pytest.fixture
+def mock_neo4j():
+    neo4j = MagicMock()
+    neo4j.execute_query = AsyncMock(return_value=[])
+    neo4j.execute_write = AsyncMock(return_value=[])
+    return neo4j
+
+
+@pytest.fixture
+def mock_paperless():
+    paperless = MagicMock()
+    paperless.get_custom_field_by_name = AsyncMock(return_value=None)
+    return paperless
+
+
+@pytest.fixture
+def sync_service(mock_paperless, mock_qdrant, mock_ollama, mock_neo4j):
+    return DocumentSyncService(
+        paperless_client=mock_paperless,
+        qdrant_client=mock_qdrant,
+        ollama_client=mock_ollama,
+        neo4j_client=mock_neo4j,
+        wiki_client=MagicMock(),
+        settings=MagicMock(),
+    )
+
+
+@pytest.mark.unit
+class TestIndexVectors:
+    async def test_ensure_collection_is_awaited(self, sync_service, mock_qdrant):
+        result = await sync_service.index_document(
+            document_id=1, user=TENANT, content="hello world", title="Doc"
+        )
+
+        assert result.success is True
+        mock_qdrant.ensure_collection.assert_awaited_once_with(TENANT_COLLECTION)
+
+    async def test_point_ids_are_deterministic(self, sync_service, mock_qdrant):
+        await sync_service.index_document(
+            document_id=7, user=TENANT, content="hello world", title="Doc"
+        )
+
+        points = mock_qdrant.upsert_points.await_args.kwargs["points"]
+        expected = str(uuid.uuid5(uuid.NAMESPACE_DNS, "document_7_chunk_0"))
+        assert points[0]["id"] == expected
+
+    async def test_stale_chunks_pruned_after_upsert(self, sync_service, mock_qdrant):
+        """Old points not in the new set are deleted AFTER the new upsert."""
+        call_order = []
+        mock_qdrant.upsert_points = AsyncMock(
+            side_effect=lambda collection_name, points: (
+                call_order.append("upsert"), len(points))[1]
+        )
+        stale_id = str(uuid.uuid4())  # legacy random-uuid4 point
+        kept_id = str(uuid.uuid5(uuid.NAMESPACE_DNS, "document_7_chunk_0"))
+        mock_qdrant.scroll_all_points = AsyncMock(
+            return_value=[{"id": stale_id, "payload": {}},
+                          {"id": kept_id, "payload": {}}]
+        )
+        mock_qdrant.delete_by_ids = AsyncMock(
+            side_effect=lambda collection_name, point_ids: (
+                call_order.append("delete"), len(point_ids))[1]
+        )
+
+        await sync_service.index_document(
+            document_id=7, user=TENANT, content="hello world", title="Doc"
+        )
+
+        assert call_order == ["upsert", "delete"]
+        mock_qdrant.delete_by_ids.assert_awaited_once_with(
+            collection_name=TENANT_COLLECTION,
+            point_ids=[stale_id],
+        )
+        mock_qdrant.scroll_all_points.assert_awaited_once_with(
+            collection_name=TENANT_COLLECTION,
+            filter_conditions={"doc_type": "document", "paperless_id": 7},
+            with_payload=False,
+        )
+
+    async def test_upsert_routed_through_wrapper(self, sync_service, mock_qdrant):
+        result = await sync_service.index_document(
+            document_id=1, user=TENANT, content="hello world", title="Doc"
+        )
+
+        assert result.chunks_created == 1
+        kwargs = mock_qdrant.upsert_points.await_args.kwargs
+        assert kwargs["collection_name"] == TENANT_COLLECTION
+        payload = kwargs["points"][0]["payload"]
+        assert payload["doc_type"] == "document"
+        assert payload["paperless_id"] == 1
+
+    async def test_failed_chunk_embedding_is_skipped_not_fatal(
+        self, sync_service, mock_qdrant, mock_ollama
+    ):
+        # Three chunks; the middle embedding fails
+        long_content = " ".join(f"word{i}" for i in range(1200))
+        mock_ollama.embed_batch = AsyncMock(
+            side_effect=lambda texts: [
+                [0.1] * 768 if i != 1 else None for i in range(len(texts))
+            ]
+        )
+
+        result = await sync_service.index_document(
+            document_id=2, user=TENANT, content=long_content, title="Doc"
+        )
+
+        assert result.success is True
+        points = mock_qdrant.upsert_points.await_args.kwargs["points"]
+        assert result.chunks_created == len(points)
+        # The failed chunk (index 1) is absent, the others kept their index
+        indices = [p["payload"]["chunk_index"] for p in points]
+        assert 1 not in indices
+        assert len(indices) >= 2
+
+    async def test_all_embeddings_failed_reports_failure(
+        self, sync_service, mock_qdrant, mock_ollama
+    ):
+        mock_ollama.embed_batch = AsyncMock(
+            side_effect=lambda texts: [None for _ in texts]
+        )
+
+        result = await sync_service.index_document(
+            document_id=3, user=TENANT, content="hello world", title="Doc"
+        )
+
+        assert result.success is False
+        assert "embed" in (result.error or "").lower()
+        mock_qdrant.upsert_points.assert_not_awaited()
+        # Delete-last: a fully failed embedding pass must leave the old
+        # vectors untouched (previously delete_by_filter ran first, leaving
+        # the document with zero vectors until the next successful sync)
+        mock_qdrant.delete_by_ids.assert_not_awaited()

tests/test_enrichment_topk.py

@@ -0,0 +1,118 @@
+"""
+Offline tests for Phase 3 enrichment running only on the top-k slice with
+ONE batched related-documents lookup (perf: the old code ran a sequential
+Neo4j query per fused result and the final trim discarded most of it).
+"""
+
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.services.hybrid_rag_service import HybridRAGService
+
+TENANT = "llm_tester"
+
+
+@pytest.fixture
+def graph():
+    g = MagicMock()
+    g.get_related_documents_batch = AsyncMock(return_value={})
+    return g
+
+
+@pytest.fixture
+def service(graph):
+    settings = MagicMock()
+    settings.ollama_llm_model = "test-model"
+    return HybridRAGService(
+        vector_service=MagicMock(),
+        graph_service=graph,
+        searxng_client=MagicMock(),
+        ollama_client=MagicMock(),
+        content_extractor=MagicMock(),
+        settings=settings,
+    )
+
+
+def _fused(n):
+    return [
+        {
+            "result": {"page_id": i + 1, "title": f"page {i + 1}"},
+            "rrf_score": 1.0 / (i + 1),
+            "sources": ["vector"],
+        }
+        for i in range(n)
+    ]
+
+
+@pytest.mark.unit
+class TestTopKEnrichment:
+    async def test_only_top_k_enriched_with_single_batched_lookup(
+        self, service, graph
+    ):
+        results = _fused(30)
+
+        enriched = await service._enrich_with_related_dossiers(
+            results, user=TENANT, top_k=5
+        )
+
+        # ONE batched lookup, only for the top-k page ids
+        graph.get_related_documents_batch.assert_awaited_once()
+        kwargs = graph.get_related_documents_batch.await_args.kwargs
+        assert kwargs["page_ids"] == [1, 2, 3, 4, 5]
+        assert kwargs["user"] == TENANT
+        # The single-page method must not be used anymore
+        graph.get_related_documents.assert_not_called()
+
+        # Every result still carries the key (tail is empty)
+        assert all("related_dossiers" in r for r in enriched)
+        assert all(r["related_dossiers"] == [] for r in enriched[5:])
+
+    async def test_related_docs_mapped_onto_results(self, service, graph):
+        graph.get_related_documents_batch = AsyncMock(return_value={
+            1: [{
+                "page_id": 9, "title": "rel", "path": "p/rel",
+                "tags": ["docker", "infra", "misc", "extra-tag-ignored"],
+                "shared_entities": 3,
+            }],
+        })
+        results = _fused(3)
+
+        enriched = await service._enrich_with_related_dossiers(
+            results, user=TENANT, top_k=2
+        )
+
+        dossiers = enriched[0]["related_dossiers"]
+        assert len(dossiers) == 3  # max 3 tags per related doc
+        assert dossiers[0] == {
+            "page_id": 9, "title": "rel", "path": "p/rel",
+            "tag": "docker", "shared_entities": 3,
+        }
+        assert enriched[1]["related_dossiers"] == []
+        assert enriched[2]["related_dossiers"] == []
+
+    async def test_results_without_page_id_are_skipped(self, service, graph):
+        results = [
+            {"result": {"url": "http://x", "title": "web"}, "sources": ["web"]},
+            {"result": {"page_id": 7, "title": "wiki"}, "sources": ["vector"]},
+        ]
+
+        enriched = await service._enrich_with_related_dossiers(
+            results, user=TENANT, top_k=10
+        )
+
+        kwargs = graph.get_related_documents_batch.await_args.kwargs
+        assert kwargs["page_ids"] == [7]
+        assert enriched[0]["related_dossiers"] == []
+
+    async def test_batch_lookup_failure_degrades_gracefully(self, service, graph):
+        graph.get_related_documents_batch = AsyncMock(
+            side_effect=RuntimeError("neo4j down")
+        )
+        results = _fused(3)
+
+        enriched = await service._enrich_with_related_dossiers(
+            results, user=TENANT, top_k=3
+        )
+
+        assert all(r["related_dossiers"] == [] for r in enriched)

tests/test_graph_query_hardening.py

@@ -0,0 +1,112 @@
+"""
+Offline unit tests for /query/graph hardening.
+
+The raw Cypher endpoint must:
+- reject queries containing write clauses or CALL procedures (denylist),
+- execute allowed queries through the READ-ONLY client path
+  (Neo4jClient.execute_read), never the writable execute_query path.
+"""
+
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.services.graph_service import GraphService
+
+
+@pytest.fixture
+def mock_neo4j():
+    neo4j = MagicMock()
+    neo4j.execute_read = AsyncMock(return_value=[{"n": {"name": "x"}}])
+    neo4j.execute_query = AsyncMock(return_value=[{"n": {"name": "x"}}])
+    return neo4j
+
+
+@pytest.fixture
+def service(mock_neo4j):
+    return GraphService(neo4j_client=mock_neo4j, wikijs_client=MagicMock())
+
+
+WRITE_QUERIES = [
+    "CREATE (n:Evil) RETURN n",
+    "MATCH (n) DELETE n",
+    "MATCH (n) DETACH DELETE n",
+    "MERGE (n:Evil {name: 'x'}) RETURN n",
+    "MATCH (n) SET n.pwned = true RETURN n",
+    "MATCH (n) REMOVE n:Document RETURN n",
+    "DROP INDEX my_index",
+    "FOREACH (x IN [1] | CREATE (:Evil))",
+    "LOAD CSV FROM 'file:///etc/passwd' AS row RETURN row",
+    "CALL db.labels()",
+    "CALL apoc.periodic.iterate('MATCH (n) RETURN n', 'DELETE n', {})",
+    "call dbms.components()",
+    "match (n) detach delete n",  # lowercase
+    "MATCH (n)\nSET n.x = 1",  # multiline
+]
+
+
+@pytest.mark.unit
+class TestWriteClauseDenylist:
+    """Write clauses and procedure calls must be rejected before execution."""
+
+    @pytest.mark.parametrize("query", WRITE_QUERIES)
+    async def test_write_query_rejected(self, service, mock_neo4j, query):
+        with pytest.raises(ValueError, match="read-only"):
+            await service.execute_query(query, {}, user="llm_tester")
+
+        mock_neo4j.execute_read.assert_not_awaited()
+        mock_neo4j.execute_query.assert_not_awaited()
+
+    async def test_read_query_allowed(self, service):
+        response = await service.execute_query(
+            "MATCH (n:Document) RETURN n LIMIT 5", {}, user="llm_tester"
+        )
+        assert response.count == 1
+
+    async def test_word_boundary_no_false_positive(self, service):
+        """Words merely containing denylisted substrings must pass."""
+        response = await service.execute_query(
+            "MATCH (n:Document) WHERE n.title = 'dataSET dropped' RETURN n",
+            {},
+            user="llm_tester",
+        )
+        assert response.count == 1
+
+
+@pytest.mark.unit
+class TestReadOnlyExecution:
+    """Allowed queries must run through the read-only session path."""
+
+    async def test_uses_execute_read_not_execute_query(self, service, mock_neo4j):
+        await service.execute_query(
+            "MATCH (n) RETURN n LIMIT 1", {"p": 1}, user="llm_tester"
+        )
+
+        mock_neo4j.execute_read.assert_awaited_once_with(
+            "MATCH (n) RETURN n LIMIT 1", {"p": 1}
+        )
+        mock_neo4j.execute_query.assert_not_awaited()
+
+    async def test_neo4j_client_read_session_access_mode(self):
+        """Neo4jClient.execute_read must open the session with READ_ACCESS."""
+        from neo4j import READ_ACCESS
+        from src.clients.neo4j_client import Neo4jClient
+
+        client = Neo4jClient(uri="bolt://unused:7687", user="u", password="p")
+
+        session = MagicMock()
+        run_result = MagicMock()
+        run_result.data = AsyncMock(return_value=[{"ok": 1}])
+        session.run = AsyncMock(return_value=run_result)
+        session_cm = MagicMock()
+        session_cm.__aenter__ = AsyncMock(return_value=session)
+        session_cm.__aexit__ = AsyncMock(return_value=False)
+
+        driver = MagicMock()
+        driver.session = MagicMock(return_value=session_cm)
+        client._driver = driver
+
+        records = await client.execute_read("RETURN 1 AS ok")
+
+        assert records == [{"ok": 1}]
+        driver.session.assert_called_once_with(default_access_mode=READ_ACCESS)

tests/test_graph_service.py

@@ -27,6 +27,7 @@ def mock_neo4j():
     """Mock Neo4j client."""
     mock = AsyncMock()
     mock.execute_query = AsyncMock(return_value=[{"d": {"page_id": TEST_PAGE_ID}}])
+    mock.execute_write = AsyncMock(return_value=[{"d": {"page_id": TEST_PAGE_ID}}])
     return mock
 
 
@@ -89,12 +90,12 @@ class TestDocumentNodeCreation:
             user=TEST_USER
         )
 
-        # Verify execute_query was called
-        assert mock_neo4j.execute_query.called
+        # Verify the write transaction was used
+        assert mock_neo4j.execute_write.called
         assert result.success is True
 
         # Find the document creation query
-        calls = mock_neo4j.execute_query.call_args_list
+        calls = mock_neo4j.execute_write.call_args_list
         doc_creation_call = None
         for call in calls:
             query = call[0][0] if call[0] else ""
@@ -128,7 +129,7 @@ class TestDocumentNodeCreation:
         assert result.success is True
 
         # Find the document creation query
-        calls = mock_neo4j.execute_query.call_args_list
+        calls = mock_neo4j.execute_write.call_args_list
         doc_creation_call = None
         for call in calls:
             query = call[0][0] if call[0] else ""
@@ -171,6 +172,7 @@ class TestEntityStubSkipping:
         assert result.success is True
         # Neo4j should NOT be called for entity-stub pages
         assert mock_neo4j.execute_query.call_count == 0
+        assert mock_neo4j.execute_write.call_count == 0
 
     @pytest.mark.asyncio
     async def test_skip_auto_generated_pages(
@@ -196,6 +198,7 @@ class TestEntityStubSkipping:
 
         assert result.success is True
         assert mock_neo4j.execute_query.call_count == 0
+        assert mock_neo4j.execute_write.call_count == 0
 
 
 class TestPageNotFound:
@@ -242,7 +245,7 @@ class TestEntityExtraction:
 
         assert result.success is True
         # Should have called neo4j at least once (for document node)
-        assert mock_neo4j.execute_query.called
+        assert mock_neo4j.execute_write.called
 
 
 if __name__ == "__main__":

tests/test_hybrid_rag.py

@@ -506,7 +506,9 @@ class TestPhase6_Persistence:
 
         timing = {"total_ms": 1000}
 
+        import uuid as _uuid
         search_id = await hybrid_rag_service._persist_search_for_librarian(
+            search_id=str(_uuid.uuid4()),
             query="test query",
             user=TEST_USER,
             keywords_data=keywords_data,

tests/test_hybrid_rag_degradation.py

@@ -0,0 +1,232 @@
+"""
+Unit tests for HybridRAG degradation signaling (offline, all clients mocked).
+
+Covers:
+- Per-leg failure -> source_status reports 'failed', degraded=True
+- Disabled legs -> 'disabled', do not trigger degraded
+- Healthy legs -> 'ok', degraded=False
+- Reranker model resolution from settings.ollama_llm_model
+- Phase 0 keyword-extraction timeout falls back gracefully
+"""
+
+import asyncio
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+import src.services.hybrid_rag_service as hybrid_rag_module
+from src.services.hybrid_rag_service import HybridRAGService
+from src.models.hybrid_rag import HybridRAGConfig
+
+
+@pytest.fixture
+def settings():
+    settings = MagicMock()
+    settings.ollama_llm_model = "gemma4:e2b"
+    settings.vector_similarity_threshold = 0.7
+    return settings
+
+
+@pytest.fixture
+def vector_service():
+    """Vector service returning a single wiki hit."""
+    vector = MagicMock()
+    hit = MagicMock()
+    hit.page_id = 1
+    hit.page_title = "Test Page"
+    hit.content = "Test content"
+    hit.page_path = "users/jp/test"
+    hit.score = 0.9
+    response = MagicMock()
+    response.results = [hit]
+    vector.search = AsyncMock(return_value=response)
+    return vector
+
+
+@pytest.fixture
+def graph_service():
+    graph = MagicMock()
+    graph.search_documents = AsyncMock(return_value=[])
+    graph.get_related_documents = AsyncMock(return_value=[])
+    graph.neo4j.execute_query = AsyncMock(return_value=[{"id": "search-1"}])
+    graph.neo4j.execute_write = AsyncMock(return_value=[{"id": "search-1"}])
+    return graph
+
+
+@pytest.fixture
+def searxng_client():
+    searxng = MagicMock()
+    searxng.search_general = AsyncMock(return_value=[])
+    return searxng
+
+
+@pytest.fixture
+def ollama_client():
+    ollama = MagicMock()
+    ollama.generate_text = AsyncMock(
+        return_value='{"core_keywords": ["test"], "synonyms": {}}'
+    )
+    return ollama
+
+
+@pytest.fixture
+def content_extractor():
+    extractor = MagicMock()
+    extractor.extract_batch = AsyncMock(return_value=[])
+    return extractor
+
+
+@pytest.fixture
+def service(settings, vector_service, graph_service, searxng_client, ollama_client, content_extractor):
+    return HybridRAGService(
+        vector_service=vector_service,
+        graph_service=graph_service,
+        searxng_client=searxng_client,
+        ollama_client=ollama_client,
+        content_extractor=content_extractor,
+        settings=settings,
+        volatile_service=None,
+    )
+
+
+def make_config(**overrides):
+    """Config with documents disabled (leg needs a real Qdrant client) and fast phases."""
+    defaults = {
+        "enable_documents": False,
+        "enable_volatile": False,
+        "enable_reranking": False,
+        "enable_enrichment": False,
+    }
+    defaults.update(overrides)
+    return HybridRAGConfig(**defaults)
+
+
+@pytest.mark.unit
+class TestSourceStatus:
+    """source_status must report every leg as ok/failed/disabled."""
+
+    async def test_all_enabled_legs_ok(self, service):
+        response = await service.search("test query", "jp", make_config())
+
+        assert response.source_status == {
+            "vector": "ok",
+            "graph": "ok",
+            "web": "ok",
+            "volatile": "disabled",
+            "documents": "disabled",
+        }
+        assert response.degraded is False
+
+    async def test_failed_leg_reported_and_degraded(self, service, graph_service):
+        graph_service.search_documents = AsyncMock(side_effect=RuntimeError("neo4j down"))
+
+        response = await service.search("test query", "jp", make_config())
+
+        assert response.source_status["graph"] == "failed"
+        assert response.source_status["vector"] == "ok"
+        assert response.degraded is True
+
+    async def test_failed_leg_still_contributes_no_results(self, service, graph_service):
+        """Existing behavior preserved: failure -> empty leg, other legs still work."""
+        graph_service.search_documents = AsyncMock(side_effect=RuntimeError("neo4j down"))
+
+        response = await service.search("test query", "jp", make_config())
+
+        # The vector hit still comes through
+        assert response.total_results == 1
+        assert response.results[0].page_id == 1
+
+    async def test_multiple_failures(self, service, graph_service, searxng_client):
+        graph_service.search_documents = AsyncMock(side_effect=RuntimeError("neo4j down"))
+        searxng_client.search_general = AsyncMock(side_effect=OSError("searxng unreachable"))
+
+        response = await service.search("test query", "jp", make_config())
+
+        assert response.source_status["graph"] == "failed"
+        assert response.source_status["web"] == "failed"
+        assert response.degraded is True
+
+    async def test_disabled_legs_do_not_degrade(self, service):
+        config = make_config(enable_graph=False, enable_web=False)
+
+        response = await service.search("test query", "jp", config)
+
+        assert response.source_status["graph"] == "disabled"
+        assert response.source_status["web"] == "disabled"
+        assert response.degraded is False
+
+    async def test_volatile_without_service_is_disabled(self, service):
+        """enable_volatile=True but no volatile service wired -> disabled, not failed."""
+        response = await service.search("test query", "jp", make_config(enable_volatile=True))
+
+        assert response.source_status["volatile"] == "disabled"
+        assert response.degraded is False
+
+    async def test_status_fields_serialized(self, service, graph_service):
+        """Contract: fields present in the serialized response for tatlock to parse."""
+        graph_service.search_documents = AsyncMock(side_effect=RuntimeError("boom"))
+
+        response = await service.search("test query", "jp", make_config())
+        payload = response.model_dump()
+
+        assert set(payload["source_status"].keys()) == {"vector", "graph", "web", "volatile", "documents"}
+        assert payload["degraded"] is True
+
+
+@pytest.mark.unit
+class TestModelResolution:
+    """The reranker/keyword model must come from settings.ollama_llm_model."""
+
+    def test_reranker_model_from_llm_setting(self, service):
+        assert service.reranker_model == "gemma4:e2b"
+
+    async def test_generation_calls_use_llm_model(self, service, ollama_client):
+        await service.search("test query", "jp", make_config())
+
+        # Phase 0 keyword extraction ran with the generation model
+        assert ollama_client.generate_text.await_count >= 1
+        for call in ollama_client.generate_text.await_args_list:
+            assert call.kwargs["model"] == "gemma4:e2b"
+
+
+@pytest.mark.unit
+class TestLLMTimeout:
+    """A hung LLM call must not gate retrieval: 12s wait_for with fallback."""
+
+    async def test_keyword_extraction_timeout_falls_back(self, service, ollama_client, monkeypatch):
+        monkeypatch.setattr(hybrid_rag_module, "LLM_CALL_TIMEOUT_SECONDS", 0.05)
+
+        async def hang(*args, **kwargs):
+            await asyncio.sleep(5)
+
+        ollama_client.generate_text = AsyncMock(side_effect=hang)
+
+        response = await service.search("test query", "jp", make_config())
+
+        # Fallback: raw query words as keywords, retrieval still ran
+        assert response.keywords.core_keywords == ["test", "query"]
+        assert response.source_status["vector"] == "ok"
+
+    async def test_rerank_timeout_keeps_rrf_order(self, service, ollama_client, monkeypatch, searxng_client, content_extractor):
+        monkeypatch.setattr(hybrid_rag_module, "LLM_CALL_TIMEOUT_SECONDS", 0.05)
+
+        # Two web results so re-ranking actually runs (needs > 1 result)
+        searxng_client.search_general = AsyncMock(return_value=[
+            {"url": "http://a.test", "title": "A", "content": "a"},
+            {"url": "http://b.test", "title": "B", "content": "b"},
+        ])
+
+        keyword_json = '{"core_keywords": ["test"], "synonyms": {}}'
+
+        async def generate(prompt, **kwargs):
+            if "Rank these documents" in prompt:
+                await asyncio.sleep(5)  # Hang only the re-rank call
+            return keyword_json
+
+        ollama_client.generate_text = AsyncMock(side_effect=generate)
+
+        response = await service.search("test query", "jp", make_config(enable_reranking=True))
+
+        # RRF order preserved despite the hung re-rank call
+        assert response.total_results >= 2
+        assert response.degraded is False

tests/test_ingest_endpoints.py

@@ -0,0 +1,320 @@
+"""
+Offline unit tests for the implemented /ingest and /deduplicate endpoints.
+
+Covers (Phase C item 1):
+- /ingest/check-updates: content-hash comparison classification
+- /ingest/status/{job_id}: Redis job-manager backing + tenant scoping
+- /ingest/repo-status/{repository}: wiki vs graph counts + job stats
+- /deduplicate/check: tenant-scoped similarity scan grouping
+
+All external clients are mocked - no shared services are contacted.
+"""
+
+import pytest
+from unittest.mock import AsyncMock
+
+from fastapi import HTTPException
+
+from src.core.hashing import compute_content_hash
+from src.main import (
+    check_updates,
+    check_duplicates,
+    get_ingestion_status,
+    get_repo_status,
+    CheckUpdatesRequest,
+    DeduplicateCheckRequest,
+)
+from src.services.vector_service import VectorService
+
+TEST_USER = "llm_tester"
+
+
+# ---------------------------------------------------------------------------
+# /ingest/check-updates
+# ---------------------------------------------------------------------------
+
+
+class TestCheckUpdates:
+    @pytest.mark.asyncio
+    async def test_classifies_changed_new_and_deleted(self):
+        """Hash mismatch -> changed, no node -> new, stale node -> deleted."""
+        wikijs = AsyncMock()
+        wikijs.list_all_pages = AsyncMock(return_value=[
+            {"id": 1, "path": f"users/{TEST_USER}/a", "title": "A", "tags": []},
+            {"id": 2, "path": f"users/{TEST_USER}/b", "title": "B", "tags": []},
+        ])
+        wikijs.get_page = AsyncMock(side_effect=[
+            {"id": 1, "content": "content-a"},
+            {"id": 2, "content": "content-b"},
+        ])
+
+        neo4j = AsyncMock()
+
+        async def fake_query(cypher, params):
+            # The endpoint sends one UNWIND query with page hashes
+            pages = params["pages"]
+            assert pages[0]["hash"] == compute_content_hash("content-a")
+            return [{
+                "checked": [
+                    {  # page 1: node exists, hash matches -> up to date
+                        "page_id": 1, "path": pages[0]["path"], "title": "A",
+                        "is_new": False, "changed": False,
+                        "stored_hash_missing": False,
+                    },
+                    {  # page 2: node exists, hash differs -> changed
+                        "page_id": 2, "path": pages[1]["path"], "title": "B",
+                        "is_new": False, "changed": True,
+                        "stored_hash_missing": False,
+                    },
+                ],
+                "deleted": [
+                    {"page_id": 99, "path": f"users/{TEST_USER}/gone", "title": "Gone"},
+                ],
+            }]
+
+        neo4j.execute_query = AsyncMock(side_effect=fake_query)
+
+        result = await check_updates(
+            request=CheckUpdatesRequest(user=TEST_USER),
+            neo4j=neo4j,
+            wikijs=wikijs,
+            api_key="",
+        )
+
+        assert result["counts"] == {
+            "changed": 1, "new": 0, "deleted": 1, "up_to_date": 1
+        }
+        assert result["changed"][0]["page_id"] == 2
+        assert result["deleted"][0]["page_id"] == 99
+        assert result["duration_ms"] >= 0
+
+    @pytest.mark.asyncio
+    async def test_new_page_detected_and_stub_pages_excluded(self):
+        """Pages without Document nodes are new; entity stubs are skipped."""
+        wikijs = AsyncMock()
+        wikijs.list_all_pages = AsyncMock(return_value=[
+            {"id": 5, "path": f"users/{TEST_USER}/fresh", "title": "Fresh", "tags": []},
+            {"id": 6, "path": f"users/{TEST_USER}/stub", "title": "Stub",
+             "tags": ["entity-stub"]},
+        ])
+        wikijs.get_page = AsyncMock(return_value={"id": 5, "content": "x"})
+
+        neo4j = AsyncMock()
+        neo4j.execute_query = AsyncMock(return_value=[{
+            "checked": [{
+                "page_id": 5, "path": f"users/{TEST_USER}/fresh", "title": "Fresh",
+                "is_new": True, "changed": False, "stored_hash_missing": False,
+            }],
+            "deleted": [],
+        }])
+
+        result = await check_updates(
+            request=CheckUpdatesRequest(user=TEST_USER),
+            neo4j=neo4j,
+            wikijs=wikijs,
+            api_key="",
+        )
+
+        # Only page 5 was hashed (stub excluded -> get_page called once)
+        assert wikijs.get_page.await_count == 1
+        assert result["counts"]["new"] == 1
+        assert result["new"][0]["page_id"] == 5
+
+    @pytest.mark.asyncio
+    async def test_empty_wiki_reports_all_documents_deleted(self):
+        """With no wiki pages, every Document node is reported deleted."""
+        wikijs = AsyncMock()
+        wikijs.list_all_pages = AsyncMock(return_value=[])
+
+        neo4j = AsyncMock()
+        neo4j.execute_query = AsyncMock(return_value=[{
+            "deleted": [{"page_id": 7, "path": f"users/{TEST_USER}/x", "title": "X"}],
+        }])
+
+        result = await check_updates(
+            request=CheckUpdatesRequest(user=TEST_USER),
+            neo4j=neo4j,
+            wikijs=wikijs,
+            api_key="",
+        )
+
+        assert result["counts"] == {
+            "changed": 0, "new": 0, "deleted": 1, "up_to_date": 0
+        }
+
+    def test_requires_user(self):
+        """Empty user is rejected by the request model (Phase B rule)."""
+        with pytest.raises(Exception):
+            CheckUpdatesRequest(user="   ")
+
+
+# ---------------------------------------------------------------------------
+# /ingest/status/{job_id}
+# ---------------------------------------------------------------------------
+
+
+class TestIngestionStatus:
+    @pytest.mark.asyncio
+    async def test_returns_job_from_job_manager(self):
+        job_manager = AsyncMock()
+        job_manager.get_job = AsyncMock(return_value={
+            "job_id": "abc", "user": TEST_USER, "status": "completed",
+        })
+
+        job = await get_ingestion_status(
+            job_id="abc", user=TEST_USER, job_manager=job_manager, api_key=""
+        )
+        assert job["status"] == "completed"
+        job_manager.get_job.assert_awaited_once_with("abc")
+
+    @pytest.mark.asyncio
+    async def test_unknown_job_404(self):
+        job_manager = AsyncMock()
+        job_manager.get_job = AsyncMock(return_value=None)
+
+        with pytest.raises(HTTPException) as exc:
+            await get_ingestion_status(
+                job_id="missing", user=TEST_USER, job_manager=job_manager, api_key=""
+            )
+        assert exc.value.status_code == 404
+
+    @pytest.mark.asyncio
+    async def test_other_tenants_job_is_hidden(self):
+        """Tenant scoping: another user's job looks like a 404."""
+        job_manager = AsyncMock()
+        job_manager.get_job = AsyncMock(return_value={
+            "job_id": "abc", "user": "someone_else", "status": "completed",
+        })
+
+        with pytest.raises(HTTPException) as exc:
+            await get_ingestion_status(
+                job_id="abc", user=TEST_USER, job_manager=job_manager, api_key=""
+            )
+        assert exc.value.status_code == 404
+
+
+# ---------------------------------------------------------------------------
+# /ingest/repo-status/{repository}
+# ---------------------------------------------------------------------------
+
+
+class TestRepoStatus:
+    @pytest.mark.asyncio
+    async def test_counts_indexed_vs_total(self):
+        wikijs = AsyncMock()
+        wikijs.list_all_pages = AsyncMock(return_value=[
+            {"id": 1, "path": f"users/{TEST_USER}/tech/a"},
+            {"id": 2, "path": f"users/{TEST_USER}/tech/b"},
+            {"id": 3, "path": f"users/{TEST_USER}/tech/c"},
+        ])
+        neo4j = AsyncMock()
+        neo4j.execute_query = AsyncMock(return_value=[{"indexed": 2}])
+        job_manager = AsyncMock()
+        job_manager.get_job_stats = AsyncMock(return_value={"total": 4, "completed": 4})
+
+        result = await get_repo_status(
+            repository="tech", user=TEST_USER,
+            neo4j=neo4j, wikijs=wikijs, job_manager=job_manager, api_key=""
+        )
+
+        assert result["total_documents"] == 3
+        assert result["indexed_documents"] == 2
+        assert result["unindexed_documents"] == 1
+        assert result["jobs"]["total"] == 4
+        assert result["path_prefix"] == f"users/{TEST_USER}/tech"
+        # Wiki listing was scoped to the tenant namespace
+        wikijs.list_all_pages.assert_awaited_once_with(
+            path_prefix=f"users/{TEST_USER}/tech"
+        )
+
+
+# ---------------------------------------------------------------------------
+# /deduplicate/check + VectorService.find_duplicate_pairs
+# ---------------------------------------------------------------------------
+
+
+def _make_vector_service(points, search_results_by_id):
+    qdrant = AsyncMock()
+    qdrant.collection_exists = AsyncMock(return_value=True)
+    qdrant.scroll_all_points = AsyncMock(return_value=points)
+
+    async def fake_search(collection_name, query_vector, limit, score_threshold):
+        # Route on the probe vector's marker value
+        return search_results_by_id.get(query_vector[0], [])
+
+    qdrant.search_vectors = AsyncMock(side_effect=fake_search)
+    return VectorService(qdrant, AsyncMock(), AsyncMock()), qdrant
+
+
+class TestDeduplicateCheck:
+    @pytest.mark.asyncio
+    async def test_groups_pairs_by_page_and_dedupes_directions(self):
+        points = [
+            {"id": "c1", "vector": [1.0],
+             "payload": {"page_id": 10, "page_path": "users/llm_tester/a",
+                         "page_title": "A", "doc_type": "wiki"}},
+            {"id": "c2", "vector": [2.0],
+             "payload": {"page_id": 20, "page_path": "users/llm_tester/b",
+                         "page_title": "B", "doc_type": "wiki"}},
+        ]
+        search_results = {
+            1.0: [  # c1 finds c2 (cross-page) and itself (same page - ignored)
+                {"id": "c1", "score": 1.0, "payload": points[0]["payload"]},
+                {"id": "c2", "score": 0.95, "payload": points[1]["payload"]},
+            ],
+            2.0: [  # c2 finds c1 - reverse direction must NOT double count
+                {"id": "c1", "score": 0.95, "payload": points[0]["payload"]},
+            ],
+        }
+        service, _ = _make_vector_service(points, search_results)
+
+        scan = await service.find_duplicate_pairs(TEST_USER, similarity_threshold=0.9)
+
+        assert scan["chunks_scanned"] == 2
+        assert len(scan["duplicate_groups"]) == 1
+        group = scan["duplicate_groups"][0]
+        assert group["max_similarity"] == 0.95
+        assert group["matching_chunk_pairs"] == 1
+        assert {p["page_id"] for p in group["pages"]} == {10, 20}
+
+    @pytest.mark.asyncio
+    async def test_missing_collection_returns_empty(self):
+        qdrant = AsyncMock()
+        qdrant.collection_exists = AsyncMock(return_value=False)
+        service = VectorService(qdrant, AsyncMock(), AsyncMock())
+
+        scan = await service.find_duplicate_pairs(TEST_USER)
+        assert scan == {"chunks_scanned": 0, "duplicate_groups": []}
+        qdrant.scroll_all_points.assert_not_awaited()
+
+    @pytest.mark.asyncio
+    async def test_endpoint_shape(self):
+        service_qdrant = AsyncMock()
+        service_qdrant.collection_exists = AsyncMock(return_value=False)
+
+        result = await check_duplicates(
+            request=DeduplicateCheckRequest(user=TEST_USER),
+            qdrant_client=service_qdrant,
+            wiki_client=AsyncMock(),
+            ollama_client=AsyncMock(),
+            api_key="",
+        )
+        assert result["user"] == TEST_USER
+        assert result["similarity_threshold"] == 0.9
+        assert result["duplicate_groups"] == []
+        assert result["duplicate_group_count"] == 0
+
+    def test_requires_user(self):
+        with pytest.raises(Exception):
+            DeduplicateCheckRequest(user="")
+
+
+# ---------------------------------------------------------------------------
+# content hash canonicality
+# ---------------------------------------------------------------------------
+
+
+def test_content_hash_is_stable_and_none_safe():
+    assert compute_content_hash("abc") == compute_content_hash("abc")
+    assert compute_content_hash("abc") != compute_content_hash("abd")
+    assert compute_content_hash(None) == compute_content_hash("")

tests/test_integration.py

@@ -7,13 +7,22 @@ These tests require actual service connectivity:
 - SearXNG running at http://searxng:8080
 - Ollama running at http://ollama:11434
 
-Run with: pytest tests/test_integration.py -v
+These tests run against the SHARED production services under the reserved
+test tenant only. They are gated behind the session tenant guard and the
+RUN_INTEGRATION_TESTS=1 environment flag (see tests/conftest.py) and are
+skipped otherwise.
+
+Run with:
+    RUN_INTEGRATION_TESTS=1 TEST_HOST=<shared-host> \
+        .venv/bin/python -m pytest tests/test_integration.py -v
 """
 
 import pytest
 import pytest_asyncio
 from typing import AsyncGenerator
 
+from tests.conftest import TEST_TENANT, assert_safe_test_tenant
+
 from src.clients.neo4j_client import Neo4jClient
 from src.clients.qdrant_client import QdrantClientWrapper
 from src.clients.wikijs_client import WikiJSClient
@@ -22,6 +31,9 @@ from src.clients.ollama_client import OllamaClient
 from src.jobs.job_manager import JobManager, JobType, JobStatus
 from src.config import get_settings
 
+# Only run when the tenant guard passes and integration mode is enabled.
+pytestmark = pytest.mark.integration
+
 
 @pytest.fixture
 def settings():
@@ -100,6 +112,7 @@ class TestNeo4jIntegration:
     @pytest.mark.asyncio
     async def test_create_and_get_document(self, neo4j_client, test_user):
         """Test creating and retrieving a document node."""
+        assert_safe_test_tenant(test_user)
         doc_id = "test_doc_integration"
 
         # Create document
@@ -131,10 +144,11 @@ class TestQdrantIntegration:
     @pytest.mark.asyncio
     async def test_collection_creation(self, qdrant_client, test_user):
         """Test creating a collection."""
-        await qdrant_client.ensure_collection(test_user)
-
+        assert_safe_test_tenant(test_user)
         collection_name = qdrant_client.get_collection_name(test_user)
-        collections = qdrant_client.client.get_collections()
+        await qdrant_client.ensure_collection(collection_name)
+
+        collections = await qdrant_client.client.get_collections()
         collection_names = [c.name for c in collections.collections]
 
         assert collection_name in collection_names
@@ -142,7 +156,10 @@ class TestQdrantIntegration:
     @pytest.mark.asyncio
     async def test_upsert_and_search(self, qdrant_client, test_user):
         """Test upserting and searching chunks."""
-        await qdrant_client.ensure_collection(test_user)
+        assert_safe_test_tenant(test_user)
+        await qdrant_client.ensure_collection(
+            qdrant_client.get_collection_name(test_user)
+        )
 
         # Create test chunks with 768-dimensional embeddings
         chunks = [
@@ -190,7 +207,7 @@ class TestWikiJSIntegration:
     @pytest.mark.asyncio
     async def test_list_all_pages(self, wikijs_client):
         """Test listing all pages with pagination support."""
-        pages = await wikijs_client.list_all_pages(path_prefix="users/")
+        pages = await wikijs_client.list_all_pages(path_prefix=f"users/{TEST_TENANT}")
         assert isinstance(pages, list)
         # Verify each page has expected fields
         for page in pages[:5]:  # Check first 5
@@ -201,7 +218,7 @@ class TestWikiJSIntegration:
     @pytest.mark.asyncio
     async def test_get_taxonomy_structure(self, wikijs_client):
         """Test getting taxonomy structure for a user."""
-        taxonomy = await wikijs_client.get_taxonomy_structure("users/jpmschweitzer")
+        taxonomy = await wikijs_client.get_taxonomy_structure(f"users/{TEST_TENANT}")
         assert isinstance(taxonomy, dict)
         # Each key should be a category, value should be list of subcategories
         for category, subcategories in taxonomy.items():

tests/test_integrity_check.py

@@ -0,0 +1,186 @@
+"""
+Offline unit tests for the nightly integrity-check endpoint (Phase C item 2).
+
+All external clients are mocked - no shared services are contacted.
+The endpoint must be strictly read-only: these tests assert that no
+delete/purge/update methods are ever invoked.
+"""
+
+import pytest
+from unittest.mock import AsyncMock
+
+from src.routers.maintenance import (
+    IntegrityCheckRequest,
+    classify_collection,
+    integrity_check,
+    run_integrity_check,
+)
+
+TEST_USER = "llm_tester"
+TENANT_PREFIX = f"users/{TEST_USER}"
+
+
+def _mock_clients(
+    wiki_pages,
+    chunk_refs,
+    graph_docs,
+    collections,
+):
+    wiki_client = AsyncMock()
+    wiki_client.list_all_pages = AsyncMock(return_value=wiki_pages)
+
+    vector_service = AsyncMock()
+    vector_service.get_all_chunk_references = AsyncMock(return_value=chunk_refs)
+
+    graph_service = AsyncMock()
+    graph_service.get_all_document_references = AsyncMock(return_value=graph_docs)
+
+    qdrant = AsyncMock()
+    qdrant.list_collections = AsyncMock(return_value=collections)
+
+    return wiki_client, vector_service, graph_service, qdrant
+
+
+class TestRunIntegrityCheck:
+    @pytest.mark.asyncio
+    async def test_full_report(self):
+        wiki_pages = [
+            # tenant pages
+            {"id": 1, "path": f"{TENANT_PREFIX}/a", "title": "A"},
+            {"id": 2, "path": f"{TENANT_PREFIX}/b", "title": "B"},
+            # another tenant's page (defines a known tenant, out of scope here)
+            {"id": 50, "path": "users/jpmschweitzer/x", "title": "X"},
+        ]
+        chunk_refs = [
+            # page 1 has vectors; page 2 has none (silent-skip victim)
+            {"chunk_id": "c1", "page_id": 1, "doc_type": "wiki"},
+            # orphan: page 99 no longer exists in the wiki
+            {"chunk_id": "c9", "page_id": 99, "doc_type": "wiki"},
+            # non-wiki chunk is ignored by the wiki-side checks
+            {"chunk_id": "cd", "document_id": "d1", "doc_type": "document"},
+        ]
+        graph_docs = [
+            {"page_id": 1, "path": f"{TENANT_PREFIX}/a", "title": "A", "doc_type": "wiki"},
+            # stale Document node: wiki page 77 is gone
+            {"page_id": 77, "path": f"{TENANT_PREFIX}/old", "title": "Old", "doc_type": "wiki"},
+        ]
+        collections = [
+            {"name": "library_desk_jpmschweitzer", "vectors_count": 10},
+            {"name": "library_desk_llm_tester", "vectors_count": 2},
+            {"name": "library_desk_ghost_tenant", "vectors_count": 1},
+            {"name": "open-webui_files", "vectors_count": 5},
+        ]
+
+        wiki_client, vector_service, graph_service, qdrant = _mock_clients(
+            wiki_pages, chunk_refs, graph_docs, collections
+        )
+
+        report = await run_integrity_check(
+            TEST_USER, vector_service, graph_service, wiki_client, qdrant
+        )
+
+        assert report.success is True
+        assert report.user == TEST_USER
+
+        # Pages without vectors: page 2 only (page 1 has c1)
+        assert [p["page_id"] for p in report.pages_without_vectors] == [2]
+
+        # Orphaned vectors: c9 -> page 99
+        assert report.orphaned_vector_chunks == 1
+        assert report.orphaned_vector_page_ids == [99]
+
+        # Collections: llm_tester is test residue, ghost_tenant unknown,
+        # jpmschweitzer expected, open-webui foreign
+        flagged = {c["name"]: c["category"] for c in report.unexpected_collections}
+        assert flagged == {
+            "library_desk_llm_tester": "test_residue",
+            "library_desk_ghost_tenant": "unknown_tenant",
+        }
+        assert report.foreign_collections == 1
+
+        # Graph documents without wiki counterparts: page 77
+        assert [d["page_id"] for d in report.documents_without_wiki] == [77]
+
+        assert report.counts["tenant_wiki_pages"] == 2
+        assert report.counts["pages_without_vectors"] == 1
+        assert report.counts["documents_without_wiki"] == 1
+        assert report.duration_ms >= 0
+
+    @pytest.mark.asyncio
+    async def test_read_only_no_mutations(self):
+        """The integrity check must never call any destructive method."""
+        wiki_client, vector_service, graph_service, qdrant = _mock_clients(
+            [], [], [], []
+        )
+
+        await run_integrity_check(
+            TEST_USER, vector_service, graph_service, wiki_client, qdrant
+        )
+
+        for mock, destructive in (
+            (vector_service, ("purge_chunks_by_ids", "delete_page_chunks")),
+            (graph_service, ("purge_orphan_entities", "purge_stale_documents_by_ids",
+                             "delete_page", "cleanup_broken_relationships")),
+            (qdrant, ("delete_collection", "delete_by_ids", "delete_by_filter")),
+            (wiki_client, ("delete_page", "update_page", "create_page")),
+        ):
+            for name in destructive:
+                assert not getattr(mock, name).await_count, (
+                    f"integrity check must be read-only but called {name}"
+                )
+
+
+class TestIntegrityEndpoint:
+    @pytest.mark.asyncio
+    async def test_caches_latest_report_in_redis(self):
+        wiki_client, vector_service, graph_service, qdrant = _mock_clients(
+            [{"id": 1, "path": f"{TENANT_PREFIX}/a", "title": "A"}],
+            [{"chunk_id": "c1", "page_id": 1, "doc_type": "wiki"}],
+            [],
+            [],
+        )
+        redis = AsyncMock()
+
+        report = await integrity_check(
+            request=IntegrityCheckRequest(user=TEST_USER),
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant,
+            redis=redis,
+            api_key="",
+        )
+
+        assert report.success is True
+        redis.setex.assert_awaited_once()
+        key = redis.setex.await_args.args[0]
+        assert key == f"library:integrity:latest:{TEST_USER}"
+
+    def test_requires_user(self):
+        with pytest.raises(Exception):
+            IntegrityCheckRequest(user=" ")
+
+
+class TestClassifyCollection:
+    def test_expected_tenant(self):
+        assert classify_collection(
+            "library_desk_jpmschweitzer", {"jpmschweitzer"}
+        ) == "expected"
+
+    def test_test_residue_beats_known_tenant(self):
+        # Even if the test tenant has wiki pages during a run, its
+        # collections are still flagged as residue.
+        assert classify_collection(
+            "library_desk_llm_tester", {"llm_tester"}
+        ) == "test_residue"
+
+    def test_unknown_tenant(self):
+        assert classify_collection(
+            "volatile_mystery", {"jpmschweitzer"}
+        ) == "unknown_tenant"
+
+    def test_foreign_and_foreign_residue(self):
+        assert classify_collection("open-webui_files", set()) == "foreign"
+        assert classify_collection("core_ai_user_test_at_example_com", set()) == \
+            "foreign_test_residue"
+        assert classify_collection("test_user", set()) == "foreign_test_residue"

tests/test_job_plumbing.py

@@ -0,0 +1,181 @@
+"""
+Offline unit tests for job + Scheduler task plumbing (Phase C item 5).
+
+Covers:
+- job_cleanup_loop: invokes cleanup_expired_jobs per pass, survives
+  transient errors, honors cancellation
+- register_scheduler_tasks.py: dry-run default, payload contents
+  (explicit production user, auth placeholder, schedules)
+"""
+
+import asyncio
+from unittest.mock import AsyncMock
+
+import pytest
+
+from src.jobs.job_manager import job_cleanup_loop
+
+
+class TestJobCleanupLoop:
+    @pytest.mark.asyncio
+    async def test_invokes_cleanup_each_pass(self):
+        manager = AsyncMock()
+
+        passes = await job_cleanup_loop(manager, interval_seconds=0, max_iterations=3)
+
+        assert passes == 3
+        assert manager.cleanup_expired_jobs.await_count == 3
+
+    @pytest.mark.asyncio
+    async def test_transient_error_does_not_kill_loop(self):
+        manager = AsyncMock()
+        manager.cleanup_expired_jobs = AsyncMock(
+            side_effect=[RuntimeError("redis hiccup"), None]
+        )
+
+        passes = await job_cleanup_loop(manager, interval_seconds=0, max_iterations=2)
+
+        assert passes == 2
+        assert manager.cleanup_expired_jobs.await_count == 2
+
+    @pytest.mark.asyncio
+    async def test_cancellation_stops_loop(self):
+        manager = AsyncMock()
+
+        task = asyncio.create_task(job_cleanup_loop(manager, interval_seconds=60))
+        await asyncio.sleep(0)  # let it start sleeping
+        task.cancel()
+        with pytest.raises(asyncio.CancelledError):
+            await task
+
+
+class TestSchedulerTaskDefinitions:
+    def _load_module(self):
+        import importlib
+        return importlib.import_module("scripts.register_scheduler_tasks")
+
+    def test_four_production_payloads_defined(self):
+        mod = self._load_module()
+        names = {t["task_name"] for t in mod.TASKS}
+        assert names == {
+            "library_integrity_check",
+            "library_quality_report",
+            "library_paperless_orphan_cleanup",
+        }
+        updates = {u["task_name"]: u["updates"] for u in mod.TASK_UPDATES}
+        assert updates == {"test_example_task": {"enabled": False}}
+
+    def test_schedules(self):
+        mod = self._load_module()
+        by_name = {t["task_name"]: t for t in mod.TASKS}
+
+        integrity = by_name["library_integrity_check"]
+        assert (integrity["hour"], integrity["minute"], integrity["day_of_week"]) == (4, 30, -1)
+
+        quality = by_name["library_quality_report"]
+        # Sunday 03:00 (Scheduler: 0 = Monday .. 6 = Sunday)
+        assert (quality["hour"], quality["minute"], quality["day_of_week"]) == (3, 0, 6)
+
+        paperless = by_name["library_paperless_orphan_cleanup"]
+        assert (paperless["hour"], paperless["minute"], paperless["day_of_week"]) == (5, 0, -1)
+
+    def test_payloads_use_explicit_production_user_and_placeholder(self):
+        mod = self._load_module()
+        for task in mod.TASKS:
+            config = task["config"]
+            # Auth goes through the executor's auth block so the Scheduler
+            # substitutes ${LIBRARY_API_KEY} from ITS environment at
+            # execution time (plain headers are NOT substituted).
+            assert config["auth"] == {
+                "type": "bearer",
+                "token": mod.API_KEY_PLACEHOLDER,
+            }
+            assert "Authorization" not in config.get("headers", {})
+            # The executor sends config["payload"] as the JSON body ("body"
+            # would be silently ignored)
+            assert "body" not in config
+            # Explicit production tenant in payload or query string (Phase B)
+            payload_user = config.get("payload", {}).get("user")
+            assert payload_user == "jpmschweitzer" or "user=jpmschweitzer" in config["url"]
+
+    def test_paperless_task_hits_existing_endpoint(self):
+        mod = self._load_module()
+        task = next(t for t in mod.TASKS
+                    if t["task_name"] == "library_paperless_orphan_cleanup")
+        assert "/maintenance/cleanup/paperless" in task["config"]["url"]
+        assert "dry_run=false" in task["config"]["url"]
+
+    def test_no_client_side_key_substitution(self):
+        """The raw API key must never be resolved client-side — that would
+        store it hardcoded in the Scheduler's scheduled_tasks.config."""
+        mod = self._load_module()
+        assert not hasattr(mod, "substitute_api_key")
+        for task in mod.TASKS:
+            assert mod.API_KEY_PLACEHOLDER in task["config"]["auth"]["token"]
+
+    def test_dry_run_is_default_and_sends_nothing(self, capsys, monkeypatch):
+        mod = self._load_module()
+        monkeypatch.setattr("sys.argv", ["register_scheduler_tasks.py"])
+        monkeypatch.setenv("SCHEDULER_URL", "http://scheduler.test:8090")
+
+        def _boom(*args, **kwargs):  # any HTTP client construction = failure
+            raise AssertionError("dry-run must not contact the Scheduler")
+
+        monkeypatch.setattr(mod.httpx, "Client", _boom)
+
+        assert mod.main() == 0
+        out = capsys.readouterr().out
+        assert "DRY RUN" in out
+        assert "library_integrity_check" in out
+        assert mod.API_KEY_PLACEHOLDER in out  # placeholder, never a real key
+
+    def test_execute_requires_scheduler_api_key(self, capsys, monkeypatch):
+        """--execute must refuse to run without SCHEDULER_API_KEY (the
+        Scheduler's task endpoints are Bearer-guarded; without the key
+        every probe 401s and registration silently fails)."""
+        mod = self._load_module()
+        monkeypatch.setattr(
+            "sys.argv", ["register_scheduler_tasks.py", "--execute"]
+        )
+        monkeypatch.setenv("SCHEDULER_URL", "http://scheduler.test:8090")
+        monkeypatch.delenv("SCHEDULER_API_KEY", raising=False)
+
+        def _boom(*args, **kwargs):
+            raise AssertionError("must not contact the Scheduler without a key")
+
+        monkeypatch.setattr(mod.httpx, "Client", _boom)
+
+        assert mod.main() == 1
+        assert "SCHEDULER_API_KEY" in capsys.readouterr().out
+
+    def test_execute_sends_scheduler_bearer_auth(self, monkeypatch):
+        """The registrar's own HTTP client must carry
+        Authorization: Bearer $SCHEDULER_API_KEY on every call."""
+        import httpx as real_httpx
+
+        mod = self._load_module()
+        seen = {"auth_headers": [], "paths": []}
+
+        def handler(request: real_httpx.Request) -> real_httpx.Response:
+            seen["auth_headers"].append(request.headers.get("Authorization"))
+            path = request.url.path
+            seen["paths"].append(f"{request.method} {path}")
+            if path == "/health":
+                return real_httpx.Response(200, json={"status": "healthy"})
+            if request.method == "GET" and path.startswith("/tasks/"):
+                return real_httpx.Response(404)  # not registered yet
+            return real_httpx.Response(200, json={"ok": True})
+
+        real_client = real_httpx.Client
+
+        def client_factory(**kwargs):
+            kwargs["transport"] = real_httpx.MockTransport(handler)
+            return real_client(**kwargs)
+
+        monkeypatch.setattr(mod.httpx, "Client", client_factory)
+
+        assert mod.execute("http://scheduler.test:8090", "sched-key-123") == 0
+        assert seen["auth_headers"], "no HTTP calls were made"
+        assert all(h == "Bearer sched-key-123" for h in seen["auth_headers"])
+        # All three tasks created (404 probe -> POST /tasks)
+        assert seen["paths"].count("POST /tasks") == len(mod.TASKS)

tests/test_multi_tenancy.py

@@ -7,8 +7,8 @@ from src.core.multi_tenancy import (
     get_wikijs_namespace,
     get_neo4j_user_label,
     validate_user_id,
+    validate_required_user,
     is_path_in_user_namespace,
-    DEFAULT_USER
 )
 
 
@@ -46,8 +46,8 @@ class TestQdrantCollectionName:
     def test_email_user(self):
         assert get_qdrant_collection_name("john@example.com") == "library_desk_john_at_example_com"
 
-    def test_default_user(self):
-        assert get_qdrant_collection_name(DEFAULT_USER) == f"library_desk_{DEFAULT_USER}"
+    def test_test_tenant(self):
+        assert get_qdrant_collection_name("llm_tester") == "library_desk_llm_tester"
 
 
 class TestWikijsNamespace:
@@ -99,6 +99,41 @@ class TestValidateUserId:
         assert validate_user_id("___") is False
 
 
+class TestValidateRequiredUser:
+    """Test the required-user validator (no default tenant)."""
+
+    def test_no_default_user_constant(self):
+        """The DEFAULT_USER escape hatch must not exist anymore."""
+        import src.core.multi_tenancy as mt
+        assert not hasattr(mt, "DEFAULT_USER")
+
+    def test_valid_user_returned(self):
+        assert validate_required_user("llm_tester") == "llm_tester"
+
+    def test_valid_user_stripped(self):
+        assert validate_required_user("  llm_tester  ") == "llm_tester"
+
+    def test_empty_rejected(self):
+        with pytest.raises(ValueError):
+            validate_required_user("")
+
+    def test_whitespace_rejected(self):
+        with pytest.raises(ValueError):
+            validate_required_user("   ")
+
+    def test_none_rejected(self):
+        with pytest.raises(ValueError):
+            validate_required_user(None)
+
+    def test_no_alphanumeric_rejected(self):
+        with pytest.raises(ValueError):
+            validate_required_user("___")
+
+    def test_too_long_rejected(self):
+        with pytest.raises(ValueError):
+            validate_required_user("a" * 101)
+
+
 class TestPathInNamespace:
     """Test path namespace checking."""
 

tests/test_quality_report.py

@@ -0,0 +1,236 @@
+"""
+Offline unit tests for the weekly quality-report endpoint (Phase C item 3).
+
+All external clients are mocked - no shared services are contacted.
+Live verification against the local dev server runs in
+tests/test_quality_report_live.py (integration-marked).
+"""
+
+import json
+from datetime import datetime, timedelta, timezone
+from unittest.mock import AsyncMock
+
+import pytest
+
+from src.routers.maintenance import (
+    QualityReportRequest,
+    _parse_wiki_timestamp,
+    quality_report,
+)
+
+TEST_USER = "llm_tester"
+TENANT_PREFIX = f"users/{TEST_USER}"
+
+
+def _iso(days_ago: int) -> str:
+    return (datetime.now(timezone.utc) - timedelta(days=days_ago)).isoformat()
+
+
+def _mock_stack(
+    pages,
+    hit_rows,
+    duplicate_scan=None,
+    cached_integrity=None,
+):
+    vector_service = AsyncMock()
+    vector_service.find_duplicate_pairs = AsyncMock(
+        return_value=duplicate_scan or {"chunks_scanned": 0, "duplicate_groups": []}
+    )
+    # Used only when integrity runs inline
+    vector_service.get_all_chunk_references = AsyncMock(return_value=[])
+
+    graph_service = AsyncMock()
+    graph_service.neo4j = AsyncMock()
+    graph_service.neo4j.execute_query = AsyncMock(return_value=hit_rows)
+    graph_service.get_all_document_references = AsyncMock(return_value=[])
+
+    wiki_client = AsyncMock()
+    # First call: tenant listing; later calls: report-path existence check
+    wiki_client.list_all_pages = AsyncMock(side_effect=[pages, []])
+    wiki_client.create_page = AsyncMock(return_value={"id": 777})
+
+    qdrant = AsyncMock()
+    qdrant.list_collections = AsyncMock(return_value=[])
+
+    redis = AsyncMock()
+
+    async def _redis_get(key):
+        if key.startswith("library:integrity:latest:") and cached_integrity:
+            return json.dumps(cached_integrity)
+        return None
+
+    redis.get = AsyncMock(side_effect=_redis_get)
+
+    return vector_service, graph_service, wiki_client, qdrant, redis
+
+
+class TestQualityReport:
+    @pytest.mark.asyncio
+    async def test_full_report_flags_and_writes_page(self):
+        pages = [
+            # stale: 60 days old, 0 hits, missing tags+description
+            {"id": 1, "path": f"{TENANT_PREFIX}/old-page", "title": "Old",
+             "tags": [], "description": "", "updatedAt": _iso(60)},
+            # old but frequently found -> NOT stale; has metadata
+            {"id": 2, "path": f"{TENANT_PREFIX}/popular", "title": "Popular",
+             "tags": ["x"], "description": "d", "updatedAt": _iso(60)},
+            # fresh page missing description only
+            {"id": 3, "path": f"{TENANT_PREFIX}/fresh", "title": "Fresh",
+             "tags": ["y"], "description": "", "updatedAt": _iso(1)},
+            # system report page is exempt from all checks
+            {"id": 4, "path": f"{TENANT_PREFIX}/system/quality-reports/2026-07-07",
+             "title": "Old report", "tags": [], "description": "", "updatedAt": _iso(7)},
+        ]
+        hit_rows = [
+            {"page_id": 1, "hits": 0},
+            {"page_id": 2, "hits": 9},
+            {"page_id": 3, "hits": 0},
+        ]
+        duplicate_scan = {
+            "chunks_scanned": 10,
+            "duplicate_groups": [{
+                "pages": [
+                    {"page_id": 1, "path": f"{TENANT_PREFIX}/old-page", "title": "Old"},
+                    {"page_id": 2, "path": f"{TENANT_PREFIX}/popular", "title": "Popular"},
+                ],
+                "max_similarity": 0.93,
+                "matching_chunk_pairs": 2,
+            }],
+        }
+        cached_integrity = {
+            "generated_at": _iso(0),
+            "counts": {"pages_without_vectors": 1, "orphaned_vector_chunks": 0,
+                       "documents_without_wiki": 0, "unexpected_collections": 2},
+            "unexpected_collections": [
+                {"name": "library_desk_ghost", "category": "unknown_tenant"},
+            ],
+        }
+
+        vector_service, graph_service, wiki_client, qdrant, redis = _mock_stack(
+            pages, hit_rows, duplicate_scan, cached_integrity
+        )
+
+        result = await quality_report(
+            request=QualityReportRequest(user=TEST_USER, stale_days=30),
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant,
+            redis=redis,
+            api_key="",
+        )
+
+        assert result.success is True
+
+        # Stale: only page 1 (page 2 old but popular, page 3 fresh, page 4 exempt)
+        assert [p["page_id"] for p in result.stale_pages] == [1]
+
+        # Missing metadata: page 1 (tags+description), page 3 (description)
+        missing = {p["page_id"]: p["missing"] for p in result.pages_missing_metadata}
+        assert missing == {1: ["tags", "description"], 3: ["description"]}
+
+        # Dedup folded in
+        assert result.counts["duplicate_groups"] == 1
+
+        # Cached integrity used (no inline re-run needed)
+        assert result.integrity["source"] == "cached"
+        vector_service.get_all_chunk_references.assert_not_awaited()
+
+        # Page written under the dated report path
+        today = datetime.now(timezone.utc).date().isoformat()
+        assert result.page_path == f"{TENANT_PREFIX}/system/quality-reports/{today}"
+        assert result.page_id == 777
+        wiki_client.create_page.assert_awaited_once()
+        create_kwargs = wiki_client.create_page.await_args.kwargs
+        assert create_kwargs["path"] == result.page_path
+        assert "auto-generated" in create_kwargs["tags"]
+
+        # Report content mentions the key findings
+        assert "old-page" in result.report
+        assert "0.930" in result.report
+        assert "library_desk_ghost" in result.report
+
+    @pytest.mark.asyncio
+    async def test_runs_integrity_inline_when_no_cache(self):
+        vector_service, graph_service, wiki_client, qdrant, redis = _mock_stack(
+            pages=[], hit_rows=[], cached_integrity=None
+        )
+        # Inline integrity re-lists all pages: give the side_effect one more value
+        wiki_client.list_all_pages = AsyncMock(side_effect=[[], [], []])
+
+        result = await quality_report(
+            request=QualityReportRequest(user=TEST_USER),
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant,
+            redis=redis,
+            api_key="",
+        )
+
+        assert result.integrity["source"] == "inline"
+        vector_service.get_all_chunk_references.assert_awaited_once()
+
+    @pytest.mark.asyncio
+    async def test_write_page_false_skips_wiki_write(self):
+        vector_service, graph_service, wiki_client, qdrant, redis = _mock_stack(
+            pages=[], hit_rows=[],
+            cached_integrity={"generated_at": _iso(0), "counts": {},
+                              "unexpected_collections": []},
+        )
+
+        result = await quality_report(
+            request=QualityReportRequest(user=TEST_USER, write_page=False),
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant,
+            redis=redis,
+            api_key="",
+        )
+
+        assert result.page_path is None
+        assert result.page_id is None
+        wiki_client.create_page.assert_not_awaited()
+        wiki_client.update_page.assert_not_awaited()
+        assert result.report  # content still returned
+
+    @pytest.mark.asyncio
+    async def test_same_day_rerun_updates_existing_page(self):
+        today = datetime.now(timezone.utc).date().isoformat()
+        report_path = f"{TENANT_PREFIX}/system/quality-reports/{today}"
+
+        vector_service, graph_service, wiki_client, qdrant, redis = _mock_stack(
+            pages=[], hit_rows=[],
+            cached_integrity={"generated_at": _iso(0), "counts": {},
+                              "unexpected_collections": []},
+        )
+        wiki_client.list_all_pages = AsyncMock(side_effect=[
+            [],  # tenant listing
+            [{"id": 555, "path": report_path, "title": f"Quality Report {today}"}],
+        ])
+
+        result = await quality_report(
+            request=QualityReportRequest(user=TEST_USER),
+            vector_service=vector_service,
+            graph_service=graph_service,
+            wiki_client=wiki_client,
+            qdrant=qdrant,
+            redis=redis,
+            api_key="",
+        )
+
+        assert result.page_id == 555
+        wiki_client.update_page.assert_awaited_once()
+        wiki_client.create_page.assert_not_awaited()
+
+    def test_requires_user(self):
+        with pytest.raises(Exception):
+            QualityReportRequest(user="")
+
+
+def test_parse_wiki_timestamp():
+    parsed = _parse_wiki_timestamp("2026-07-14T09:39:45.244Z")
+    assert parsed is not None and parsed.tzinfo is not None
+    assert _parse_wiki_timestamp(None) is None
+    assert _parse_wiki_timestamp("not-a-date") is None

tests/test_quality_report_live.py

@@ -0,0 +1,161 @@
+"""
+Live end-to-end test for the weekly quality report (integration, guard-gated).
+
+Runs against the LOCAL wakeup server (./wakeup.sh, port 8778 — never the
+production container on 8089) with the shared backing services, entirely
+under the reserved llm_tester tenant namespace:
+
+1. Create + ingest a wiki page as ``llm_tester`` (deliberately without tags
+   so the report has something to flag).
+2. POST /maintenance/quality-report {user: llm_tester}: asserts the report
+   is generated, mentions the created page in the missing-metadata section,
+   folds in integrity results, and writes the dated report page under
+   users/llm_tester/system/quality-reports/YYYY-MM-DD.
+3. Fetches the written report page back from the wiki.
+4. Teardown deletes the created pages; the session-scoped teardown in
+   conftest purges every remaining llm_tester artifact.
+
+Run with:
+    RUN_INTEGRATION_TESTS=1 TEST_HOST=<shared-host> \\
+        LIBRARY_DESK_URL=http://localhost:8778 \\
+        .venv/bin/python -m pytest tests/test_quality_report_live.py -v
+"""
+
+import time
+import uuid
+from datetime import datetime, timezone
+
+import httpx
+import pytest
+
+from tests.conftest import (
+    LIBRARY_DESK_URL,
+    PRODUCTION_TENANT,
+    TEST_TENANT,
+    assert_safe_test_tenant,
+)
+
+pytestmark = pytest.mark.integration
+
+
+@pytest.fixture(scope="module")
+def api():
+    """HTTP client for the local dev server, with bearer auth."""
+    from src.config import get_settings
+
+    assert ":8089" not in LIBRARY_DESK_URL, (
+        "Refusing to run the live quality-report test against the "
+        "production container (port 8089); point LIBRARY_DESK_URL at ./wakeup.sh"
+    )
+    settings = get_settings()
+    client = httpx.Client(
+        base_url=LIBRARY_DESK_URL,
+        headers={"Authorization": f"Bearer {settings.library_api_key}"},
+        timeout=httpx.Timeout(300.0, connect=10.0),
+    )
+    yield client
+    client.close()
+
+
+@pytest.fixture(scope="module")
+def ingested_page(api):
+    """Create + ingest a metadata-poor wiki page as the test tenant."""
+    assert_safe_test_tenant(TEST_TENANT)
+
+    slug = f"quality-probe-{uuid.uuid4().hex[:8]}"
+    create_resp = api.post(
+        "/wiki/pages",
+        json={
+            "title": f"Quality Probe {slug}",
+            "path": f"/quality-tests/{slug}",
+            "content": (
+                "# Quality Probe\n\n"
+                "Ephemeral page used to verify the weekly quality report. "
+                "It intentionally has no tags so the report flags it."
+            ),
+            "description": "",
+            "tags": [],
+            "user": TEST_TENANT,
+        },
+    )
+    assert create_resp.status_code == 201, create_resp.text
+    page = create_resp.json()
+    page_id = page["id"]
+    assert page["path"].lstrip("/").startswith(f"users/{TEST_TENANT}")
+
+    ingest_resp = api.post(
+        "/ingest/page",
+        json={"page_id": page_id, "user": TEST_TENANT, "force_refresh": True},
+    )
+    assert ingest_resp.status_code == 200, ingest_resp.text
+
+    # Wiki.js updates its page-listing index asynchronously after creation;
+    # the quality report relies on that listing, so wait until the new page
+    # is visible (up to ~30s) before running the report.
+    deadline = time.time() + 30
+    while time.time() < deadline:
+        listing = api.get("/wiki/pages", params={"user": TEST_TENANT})
+        assert listing.status_code == 200, listing.text
+        if any(p["id"] == page_id for p in listing.json().get("pages", [])):
+            break
+        time.sleep(2)
+    else:
+        pytest.fail(f"Page {page_id} never appeared in the Wiki.js listing")
+
+    yield {"page_id": page_id, "path": page["path"]}
+
+    delete_resp = api.delete(f"/wiki/pages/{page_id}", params={"user": TEST_TENANT})
+    assert delete_resp.status_code == 200, delete_resp.text
+
+
+class TestLiveQualityReport:
+    def test_quality_report_end_to_end(self, api, ingested_page):
+        assert_safe_test_tenant(TEST_TENANT)
+
+        resp = api.post(
+            "/maintenance/quality-report",
+            json={"user": TEST_TENANT, "stale_days": 30},
+        )
+        assert resp.status_code == 200, resp.text
+        data = resp.json()
+
+        assert data["success"] is True
+        assert data["user"] == TEST_TENANT
+        assert data["duration_ms"] > 0
+
+        # The dated report page was written inside the tenant's namespace
+        today = datetime.now(timezone.utc).date().isoformat()
+        expected_path = f"users/{TEST_TENANT}/system/quality-reports/{today}"
+        assert data["page_path"] == expected_path
+        assert data["page_id"]
+        assert PRODUCTION_TENANT not in data["page_path"]
+
+        # The metadata-poor probe page is flagged
+        flagged_ids = {p["page_id"] for p in data["pages_missing_metadata"]}
+        assert ingested_page["page_id"] in flagged_ids
+
+        # Integrity results are folded in (cached or inline)
+        assert data["integrity"] is not None
+        assert data["integrity"]["source"] in ("cached", "inline")
+        assert "counts" in data["integrity"]
+
+        # Report content includes the summary + the probe page path
+        assert "## Summary" in data["report"]
+        assert ingested_page["path"].lstrip("/") in data["report"]
+
+        # The written page is retrievable from the wiki via the API
+        page_resp = api.get(
+            f"/wiki/pages/{data['page_id']}", params={"user": TEST_TENANT}
+        )
+        assert page_resp.status_code == 200, page_resp.text
+        page = page_resp.json()
+        assert page["path"].lstrip("/") == expected_path
+        assert "## Summary" in page["content"]
+
+        # Same-day rerun must update the same page, not create a duplicate
+        rerun = api.post(
+            "/maintenance/quality-report",
+            json={"user": TEST_TENANT, "stale_days": 30},
+        )
+        assert rerun.status_code == 200, rerun.text
+        assert rerun.json()["page_id"] == data["page_id"]

tests/test_rag_search.py

@@ -37,12 +37,19 @@ class TestRAGSearchModels:
     """Tests for RAG search Pydantic models."""
 
     def test_search_request_defaults(self):
-        """Test RAGSearchRequest with default values."""
-        request = RAGSearchRequest(query="test query")
+        """Test RAGSearchRequest defaults (user is required, no default tenant)."""
+        request = RAGSearchRequest(query="test query", user="llm_tester")
 
         assert request.query == "test query"
         assert request.search_type == SearchType.WEB
         assert request.limit == 10
+        assert request.user == "llm_tester"
+
+    def test_search_request_requires_user(self):
+        """A request without an explicit user must be rejected."""
+        import pytest
+        with pytest.raises(ValueError):
+            RAGSearchRequest(query="test query")
 
     def test_search_request_custom_values(self):
         """Test RAGSearchRequest with custom values."""

tests/test_required_user.py

@@ -0,0 +1,119 @@
+"""
+Offline unit tests: every tenant-data endpoint must REQUIRE an explicit user.
+
+A request without a user (query param or body field) must be rejected with
+422 before any service is touched. Empty/whitespace users are also rejected.
+
+No external services are contacted: validation failures short-circuit the
+request before the endpoint body executes.
+"""
+
+import pytest
+from fastapi.testclient import TestClient
+
+from src.main import app
+from src.core.dependencies import verify_api_key, verify_browser_request
+
+
+@pytest.fixture(scope="module")
+def client():
+    """TestClient with auth stubbed out (no lifespan startup)."""
+    app.dependency_overrides[verify_api_key] = lambda: "test-key"
+    app.dependency_overrides[verify_browser_request] = lambda: "test-user"
+    try:
+        # No context manager: startup/lifespan events are NOT triggered,
+        # so no connections to external services are attempted.
+        yield TestClient(app)
+    finally:
+        app.dependency_overrides.pop(verify_api_key, None)
+        app.dependency_overrides.pop(verify_browser_request, None)
+
+
+QUERY_PARAM_ENDPOINTS = [
+    ("GET", "/stats"),
+    ("POST", "/query/semantic?query=test"),
+    ("POST", "/query/graph?query=MATCH%20(n)%20RETURN%20n"),
+    ("GET", "/wiki/pages"),
+    ("GET", "/wiki/pages/1"),
+    ("PUT", "/wiki/pages/1"),
+    ("DELETE", "/wiki/pages/1"),
+    ("GET", "/wiki/search?q=test"),
+    ("GET", "/wiki/dossiers"),
+    ("POST", "/vector/update-from-page/1"),
+    ("DELETE", "/vector/pages/1"),
+    ("GET", "/graph/nodes"),
+    ("POST", "/graph/update-from-page/1"),
+    ("POST", "/graph/generate-entity-pages"),
+    ("POST", "/ingest/all"),
+    ("GET", "/volatile/stats"),
+    ("GET", "/volatile/search?q=test"),
+    ("POST", "/volatile/store?namespace=weather&key=test"),
+    ("GET", "/volatile/weather/rotterdam"),
+    ("DELETE", "/volatile/weather/rotterdam"),
+    ("POST", "/documents/webhook-simple?doc_url=http://x/documents/1/"),
+]
+
+
+@pytest.mark.unit
+class TestUserQueryParamRequired:
+    """Endpoints with a user query parameter must 422 without it."""
+
+    @pytest.mark.parametrize("method,path", QUERY_PARAM_ENDPOINTS)
+    def test_missing_user_is_422(self, client, method, path):
+        response = client.request(method, path, json={})
+        assert response.status_code == 422, (
+            f"{method} {path} returned {response.status_code}, expected 422"
+        )
+
+    @pytest.mark.parametrize("blank", ["", " ", "%20%20"])
+    def test_blank_user_is_422(self, client, blank):
+        response = client.get(f"/wiki/pages?user={blank}")
+        assert response.status_code == 422
+
+    def test_hybrid_query_missing_user_is_422(self, client):
+        response = client.post("/query/hybrid", json={"query": "test"})
+        assert response.status_code == 422
+
+    def test_hybrid_query_whitespace_user_is_422(self, client):
+        response = client.post("/query/hybrid?user=%20", json={"query": "test"})
+        assert response.status_code == 422
+
+
+@pytest.mark.unit
+class TestUserBodyFieldRequired:
+    """Request models with a user field must reject missing/blank values."""
+
+    def test_ingest_page_missing_user_is_422(self, client):
+        response = client.post("/ingest/page", json={"page_id": 1})
+        assert response.status_code == 422
+
+    def test_ingest_page_blank_user_is_422(self, client):
+        response = client.post("/ingest/page", json={"page_id": 1, "user": "  "})
+        assert response.status_code == 422
+
+    def test_ingest_batch_missing_user_is_422(self, client):
+        response = client.post("/ingest/batch", json={"page_ids": [1]})
+        assert response.status_code == 422
+
+    def test_wiki_create_page_missing_user_is_422(self, client):
+        response = client.post(
+            "/wiki/pages",
+            json={"title": "T", "path": "/t", "content": "c"},
+        )
+        assert response.status_code == 422
+
+    def test_wiki_smart_create_missing_user_is_422(self, client):
+        response = client.post("/wiki/pages/smart-create", json={"topic": "T"})
+        assert response.status_code == 422
+
+    def test_vector_search_missing_user_is_422(self, client):
+        response = client.post("/vector/search", json={"query": "test"})
+        assert response.status_code == 422
+
+    def test_graph_query_missing_user_is_422(self, client):
+        response = client.post("/graph/query", json={"query": "MATCH (n) RETURN n"})
+        assert response.status_code == 422
+
+    def test_rag_search_missing_user_is_422(self, client):
+        response = client.post("/rag/search", json={"query": "test"})
+        assert response.status_code == 422

tests/test_scheduler_client.py

@@ -0,0 +1,122 @@
+"""
+Offline unit tests for SchedulerClient (Phase D review batch).
+
+Pins the runtime prefetch-registration fixes:
+- SchedulerClient sends Authorization: Bearer <SCHEDULER_API_KEY> on its
+  own calls (the Scheduler's /tasks endpoints are auth-guarded; a bare
+  client 401s and registration fails silently at consolidation time)
+- register_volatile_fetch task config is actually executable by the
+  Scheduler's rest_api_executor:
+  * JSON body under config["payload"] (a "body" key is silently ignored)
+  * user as a QUERY parameter (the /volatile/fetch endpoints use
+    RequiredUserQuery; a body user would 422)
+  * an auth block with the ${LIBRARY_API_KEY} placeholder so the
+    scheduled POST passes library-desk's verify_api_key
+"""
+
+import httpx
+import pytest
+
+from src.clients.scheduler_client import (
+    LIBRARY_API_KEY_PLACEHOLDER,
+    SchedulerClient,
+)
+
+
+def _mock_transport(seen):
+    def handler(request: httpx.Request) -> httpx.Response:
+        seen.append(request)
+        path = request.url.path
+        if request.method == "GET" and path.startswith("/tasks/"):
+            return httpx.Response(404)  # task does not exist yet
+        if request.method == "POST" and path == "/tasks":
+            return httpx.Response(200, json={"ok": True})
+        return httpx.Response(200, json={})
+
+    return httpx.MockTransport(handler)
+
+
+@pytest.fixture
+def seen_requests():
+    return []
+
+
+@pytest.fixture
+def client(seen_requests, monkeypatch):
+    """SchedulerClient whose real _get_client builds against a MockTransport."""
+    import src.clients.scheduler_client as mod
+
+    real_async_client = httpx.AsyncClient
+
+    def client_factory(**kwargs):
+        kwargs["transport"] = _mock_transport(seen_requests)
+        return real_async_client(**kwargs)
+
+    monkeypatch.setattr(mod.httpx, "AsyncClient", client_factory)
+    return SchedulerClient(base_url="http://scheduler.test:8090", api_key="sched-key")
+
+
+@pytest.mark.unit
+class TestSchedulerApiAuth:
+    @pytest.mark.asyncio
+    async def test_bearer_auth_sent_on_all_calls(self, client, seen_requests):
+        ok = await client.register_volatile_fetch(
+            namespace="weather",
+            key="rotterdam",
+            user="llm_tester",
+            schedule={"minute": 0},
+        )
+
+        assert ok is True
+        assert seen_requests, "no HTTP calls were made"
+        for request in seen_requests:
+            assert request.headers.get("Authorization") == "Bearer sched-key"
+
+    def test_missing_api_key_is_flagged(self, caplog):
+        with caplog.at_level("WARNING"):
+            SchedulerClient(base_url="http://scheduler.test:8090")
+        assert any("without an API key" in r.message for r in caplog.records)
+
+
+@pytest.mark.unit
+class TestRegisterVolatileFetchConfig:
+    @pytest.mark.asyncio
+    async def _registered_config(self, client, seen_requests, user="llm_tester"):
+        ok = await client.register_volatile_fetch(
+            namespace="weather",
+            key="rotterdam",
+            user=user,
+            schedule={"minute": 0, "hour": -1},
+        )
+        assert ok is True
+        create = next(
+            r for r in seen_requests
+            if r.method == "POST" and r.url.path == "/tasks"
+        )
+        import json
+        return json.loads(create.content)["config"]
+
+    @pytest.mark.asyncio
+    async def test_body_key_not_used_payload_is(self, client, seen_requests):
+        config = await self._registered_config(client, seen_requests)
+        # rest_api_executor only reads config["payload"]; "body" is ignored
+        assert "body" not in config
+        assert config["payload"] == {}
+
+    @pytest.mark.asyncio
+    async def test_user_is_query_parameter(self, client, seen_requests):
+        config = await self._registered_config(client, seen_requests)
+        assert config["url"] == (
+            "http://library-desk:8089/volatile/fetch/weather/rotterdam"
+            "?user=llm_tester"
+        )
+
+    @pytest.mark.asyncio
+    async def test_auth_block_uses_placeholder(self, client, seen_requests):
+        config = await self._registered_config(client, seen_requests)
+        assert config["auth"] == {
+            "type": "bearer",
+            "token": LIBRARY_API_KEY_PLACEHOLDER,
+        }
+        # never the raw key, never in plain headers (not substituted there)
+        assert "Authorization" not in config.get("headers", {})

tests/test_search_persistence.py

@@ -0,0 +1,225 @@
+"""
+Offline tests for the Phase 6 persistence rewrite (single atomic write,
+off the hot path) and its SHAPE CONTRACT with the consolidation service.
+
+The consolidation repair loop (consolidation_service.py) consumes the
+persisted graph:
+
+- _find_unprocessed_searches:
+    MATCH (sq:SearchQuery {processed: false}) WHERE sq.timestamp > ...
+    RETURN sq.id, sq.query, sq.user, sq.timestamp, sq.total_results,
+           sq.web_count, sq.keywords
+- _get_web_results:
+    MATCH (sq:SearchQuery {id: $search_id})-[f:FOUND]->(wr:WebResult)
+    RETURN wr.url, wr.title, wr.content, f.rank, f.rrf_score
+- _mark_search_processed:
+    SET sq.processed = true
+
+These tests pin that the new UNWIND-based persistence still writes every
+node property, label, and relationship property that consolidation reads.
+"""
+
+import asyncio
+import json
+import uuid
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.models.hybrid_rag import HybridRAGConfig
+from src.services.hybrid_rag_service import HybridRAGService
+
+TENANT = "llm_tester"
+TENANT_BASE_LABEL = "User_Llm_Tester"
+TENANT_DOC_LABEL = "User_Llm_Tester_Document"
+
+
+@pytest.fixture
+def mock_neo4j():
+    neo4j = MagicMock()
+    neo4j.execute_query = AsyncMock(return_value=[])
+    neo4j.execute_write = AsyncMock(return_value=[{"id": "sid"}])
+    return neo4j
+
+
+@pytest.fixture
+def service(mock_neo4j):
+    vector = MagicMock()
+    response = MagicMock()
+    response.results = []
+    vector.search = AsyncMock(return_value=response)
+
+    graph = MagicMock()
+    graph.neo4j = mock_neo4j
+    graph.search_documents = AsyncMock(return_value=[])
+    graph.get_related_documents = AsyncMock(return_value=[])
+
+    ollama = MagicMock()
+    ollama.generate_text = AsyncMock(
+        return_value='{"core_keywords": ["q"], "synonyms": {}}'
+    )
+
+    settings = MagicMock()
+    settings.ollama_llm_model = "test-model"
+    settings.vector_similarity_threshold = 0.7
+
+    return HybridRAGService(
+        vector_service=vector,
+        graph_service=graph,
+        searxng_client=MagicMock(),
+        ollama_client=ollama,
+        content_extractor=MagicMock(),
+        settings=settings,
+    )
+
+
+FINAL_RESULTS = [
+    {
+        "result": {"page_id": 42, "title": "wiki hit"},
+        "source_type": "wiki", "sources": ["vector"],
+        "rrf_score": 0.5, "final_rank": 1,
+    },
+    {
+        "result": {
+            "url": "http://example.com",
+            "title": "web hit",
+            "content": "x" * 5000,  # must be truncated to 1000
+        },
+        "source_type": "web", "sources": ["web"],
+        "rrf_score": 0.4, "final_rank": 2,
+    },
+]
+
+
+async def _persist(service, **overrides):
+    kwargs = dict(
+        search_id="sid-123",
+        query="test query",
+        user=TENANT,
+        keywords_data={"core_keywords": ["docker"], "synonyms": {"k8s": ["kubernetes"]}},
+        raw_results={"vector": [1], "graph": [], "web": [1, 2]},
+        final_results=FINAL_RESULTS,
+        timing={"total_ms": 123.0},
+    )
+    kwargs.update(overrides)
+    return await service._persist_search_for_librarian(**kwargs)
+
+
+@pytest.mark.unit
+class TestPersistenceIsAtomic:
+    async def test_single_write_transaction_no_autocommit_queries(
+        self, service, mock_neo4j
+    ):
+        result = await _persist(service)
+
+        assert result == "sid-123"
+        mock_neo4j.execute_write.assert_awaited_once()
+        mock_neo4j.execute_query.assert_not_awaited()
+
+    async def test_failure_returns_none(self, service, mock_neo4j):
+        mock_neo4j.execute_write = AsyncMock(side_effect=RuntimeError("boom"))
+
+        assert await _persist(service) is None
+
+
+@pytest.mark.unit
+class TestConsolidationShapeContract:
+    """Every property/label/relationship consolidation reads must be written."""
+
+    async def test_searchquery_node_shape(self, service, mock_neo4j):
+        await _persist(service)
+        cypher, params = mock_neo4j.execute_write.await_args.args[:2]
+
+        # Interoperable label + tenant label (consolidation matches bare
+        # :SearchQuery, tenant scoping needs the prefixed label)
+        assert f":{TENANT_BASE_LABEL}_SearchQuery:SearchQuery" in cypher
+        # _find_unprocessed_searches filters on these
+        assert "processed: false" in cypher
+        assert "timestamp: datetime()" in cypher
+        # ... and returns these properties
+        for prop in ("id", "query", "user", "total_results", "web_count", "keywords"):
+            assert f"{prop}: ${'search_id' if prop == 'id' else prop}" in cypher, prop
+        assert params["search_id"] == "sid-123"
+        assert params["query"] == "test query"
+        assert params["user"] == TENANT
+        assert params["total_results"] == 2
+        assert params["web_count"] == 2
+        assert params["keywords"] == ["docker"]
+        assert json.loads(params["synonyms"]) == {"k8s": ["kubernetes"]}
+
+    async def test_webresult_shape_and_found_relationship(self, service, mock_neo4j):
+        await _persist(service)
+        cypher, params = mock_neo4j.execute_write.await_args.args[:2]
+
+        # _get_web_results traverses (sq)-[f:FOUND]->(wr:WebResult) and
+        # reads wr.url, wr.title, wr.content, f.rank, f.rrf_score
+        assert f":{TENANT_BASE_LABEL}_WebResult:WebResult" in cypher
+        for fragment in ("url: wl.url", "title: wl.title", "content: wl.content"):
+            assert fragment in cypher, fragment
+        assert "rank: wl.rank" in cypher
+        assert "rrf_score: wl.rrf_score" in cypher
+        assert 'source: "web"' in cypher
+
+        web = params["web_links"]
+        assert len(web) == 1
+        assert web[0]["url"] == "http://example.com"
+        assert web[0]["rank"] == 1
+        assert web[0]["rrf_score"] == 0.4
+        assert len(web[0]["content"]) == 1000  # truncation preserved
+
+    async def test_document_links_tenant_scoped(self, service, mock_neo4j):
+        await _persist(service)
+        cypher, params = mock_neo4j.execute_write.await_args.args[:2]
+
+        assert f"MATCH (d:{TENANT_DOC_LABEL}:Document {{page_id: link.page_id}})" in cypher
+        assert "MERGE (sq)-[f:FOUND]->(d)" in cypher
+        for fragment in ("f.source = link.source", "f.rank = link.rank",
+                         "f.rrf_score = link.rrf_score",
+                         "f.final_rank = link.final_rank"):
+            assert fragment in cypher, fragment
+
+        docs = params["doc_links"]
+        assert docs == [{
+            "page_id": 42, "source": "wiki", "rank": 1,
+            "rrf_score": 0.5, "final_rank": 1,
+        }]
+
+    async def test_empty_doc_links_cannot_swallow_web_results(
+        self, service, mock_neo4j
+    ):
+        """UNWIND [] yields no rows; the CALL subqueries must isolate that."""
+        await _persist(service, final_results=[FINAL_RESULTS[1]])
+        cypher, params = mock_neo4j.execute_write.await_args.args[:2]
+
+        assert params["doc_links"] == []
+        assert len(params["web_links"]) == 1
+        # Both UNWINDs live in aggregating CALL subqueries
+        assert cypher.count("CALL {") == 2
+
+
+@pytest.mark.unit
+class TestPersistenceOffHotPath:
+    async def test_search_returns_upfront_id_and_persists_in_background(
+        self, service, mock_neo4j
+    ):
+        config = HybridRAGConfig(
+            enable_vector=True, enable_graph=False, enable_web=False,
+            enable_volatile=False, enable_documents=False,
+            enable_reranking=False, enable_enrichment=False,
+        )
+
+        response = await service.search("q", TENANT, config)
+
+        # search_id is generated up front and returned immediately
+        assert response.search_id
+        uuid.UUID(response.search_id)  # valid uuid4
+        assert response.timing.persistence_ms == 0.0
+
+        # The write happens in a background task, not on the request path
+        pending = list(service._background_tasks)
+        assert len(pending) == 1
+        await asyncio.gather(*pending)
+
+        mock_neo4j.execute_write.assert_awaited_once()
+        params = mock_neo4j.execute_write.await_args.args[1]
+        assert params["search_id"] == response.search_id

tests/test_smart_create.py

@@ -26,15 +26,20 @@ class TestWikiSmartCreateRequest:
     """Tests for WikiSmartCreateRequest model validation."""
 
     def test_minimal_request(self):
-        """Test request with only required field."""
-        request = WikiSmartCreateRequest(topic="Docker containers")
+        """Test request with only required fields (topic AND user)."""
+        request = WikiSmartCreateRequest(topic="Docker containers", user="llm_tester")
         assert request.topic == "Docker containers"
         assert request.path is None
         assert request.tags == []
-        assert request.user is None
+        assert request.user == "llm_tester"
         assert request.include_web_research is True
         assert request.include_wiki_search is True
 
+    def test_user_is_required(self):
+        """A request without an explicit user must be rejected."""
+        with pytest.raises(ValueError):
+            WikiSmartCreateRequest(topic="Docker containers")
+
     def test_full_request(self):
         """Test request with all fields."""
         request = WikiSmartCreateRequest(
@@ -68,7 +73,8 @@ class TestWikiSmartCreateRequest:
         """Test that path without leading slash gets one added."""
         request = WikiSmartCreateRequest(
             topic="Test",
-            path="technology/test"
+            path="technology/test",
+            user="llm_tester"
         )
         assert request.path == "/technology/test"
 
@@ -76,7 +82,8 @@ class TestWikiSmartCreateRequest:
         """Test that trailing slash is removed."""
         request = WikiSmartCreateRequest(
             topic="Test",
-            path="/technology/test/"
+            path="/technology/test/",
+            user="llm_tester"
         )
         assert request.path == "/technology/test"
 
@@ -84,7 +91,8 @@ class TestWikiSmartCreateRequest:
         """Test that duplicate tags are removed."""
         request = WikiSmartCreateRequest(
             topic="Test",
-            tags=["devops", "devops", "containers", "devops"]
+            tags=["devops", "devops", "containers", "devops"],
+            user="llm_tester"
         )
         assert len(request.tags) == 2
         assert "devops" in request.tags
@@ -94,7 +102,8 @@ class TestWikiSmartCreateRequest:
         """Test that tag whitespace is cleaned."""
         request = WikiSmartCreateRequest(
             topic="Test",
-            tags=["  devops  ", "containers", "  ", ""]
+            tags=["  devops  ", "containers", "  ", ""],
+            user="llm_tester"
         )
         assert "devops" in request.tags
         assert "containers" in request.tags
@@ -536,7 +545,8 @@ class TestSmartCreateEndpoint:
         # For now, we test the model validation
         request = WikiSmartCreateRequest(
             topic="Test Topic",
-            tags=["test"]
+            tags=["test"],
+            user="llm_tester"
         )
         assert request.topic == "Test Topic"
 
@@ -546,8 +556,8 @@ class TestSmartCreateEndpoint:
             WikiSmartCreateRequest(topic="")
 
     def test_request_accepts_minimal_input(self):
-        """Test that only topic is required."""
-        request = WikiSmartCreateRequest(topic="Minimal test")
+        """Test that topic and user are the only required fields."""
+        request = WikiSmartCreateRequest(topic="Minimal test", user="llm_tester")
         assert request.topic == "Minimal test"
         assert request.include_web_research is True  # default
         assert request.include_wiki_search is True  # default

tests/test_stats.py

@@ -0,0 +1,99 @@
+"""
+Unit tests for the /stats endpoint (offline, all clients mocked).
+
+Covers the wiki page count fix: the endpoint must count pages under the
+user namespace ("users/{user}"), not pass the bare user name as prefix
+(which matched nothing and always reported 0 pages).
+"""
+
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.main import stats
+
+
+@pytest.fixture
+def neo4j_client():
+    neo4j = MagicMock()
+    neo4j.execute_query = AsyncMock(return_value=[{"count": 5}])
+    return neo4j
+
+
+@pytest.fixture
+def qdrant_client():
+    qdrant = MagicMock()
+    qdrant.list_collections = AsyncMock(return_value=[
+        {"name": "user_jp", "vectors_count": 42}
+    ])
+    return qdrant
+
+
+@pytest.fixture
+def wikijs_client():
+    wikijs = MagicMock()
+    wikijs.list_all_pages = AsyncMock(return_value=[
+        {"id": i, "path": f"users/jpmschweitzer/p{i}"} for i in range(138)
+    ])
+    return wikijs
+
+
+@pytest.fixture
+def paperless_client():
+    paperless = MagicMock()
+    paperless.list_documents = AsyncMock(return_value={"count": 3})
+    paperless.list_tags = AsyncMock(return_value=[])
+    paperless.list_correspondents = AsyncMock(return_value=[])
+    paperless.list_document_types = AsyncMock(return_value=[])
+    return paperless
+
+
+@pytest.mark.unit
+class TestStatsWikiPageCount:
+    """/stats must scope the wiki page count to the user namespace."""
+
+    async def test_uses_user_namespace_path_prefix(
+        self, neo4j_client, qdrant_client, wikijs_client, paperless_client
+    ):
+        await stats(
+            user="jpmschweitzer",
+            neo4j=neo4j_client,
+            qdrant=qdrant_client,
+            wikijs=wikijs_client,
+            paperless=paperless_client,
+            api_key="test-key",
+        )
+
+        wikijs_client.list_all_pages.assert_awaited_once_with(
+            path_prefix="users/jpmschweitzer"
+        )
+
+    async def test_reports_page_count(
+        self, neo4j_client, qdrant_client, wikijs_client, paperless_client
+    ):
+        response = await stats(
+            user="jpmschweitzer",
+            neo4j=neo4j_client,
+            qdrant=qdrant_client,
+            wikijs=wikijs_client,
+            paperless=paperless_client,
+            api_key="test-key",
+        )
+
+        assert response.wiki_pages == 138
+
+    async def test_wiki_failure_degrades_to_zero(
+        self, neo4j_client, qdrant_client, wikijs_client, paperless_client
+    ):
+        wikijs_client.list_all_pages = AsyncMock(side_effect=RuntimeError("wiki down"))
+
+        response = await stats(
+            user="jpmschweitzer",
+            neo4j=neo4j_client,
+            qdrant=qdrant_client,
+            wikijs=wikijs_client,
+            paperless=paperless_client,
+            api_key="test-key",
+        )
+
+        assert response.wiki_pages == 0

tests/test_tenant_isolation_live.py

@@ -0,0 +1,186 @@
+"""
+Live tenant-isolation test (integration, guard-gated).
+
+Runs against the LOCAL wakeup server (./wakeup.sh, port 8778 — never the
+production container on 8089) with the shared backing services, entirely
+under the reserved llm_tester tenant namespace:
+
+1. Create a wiki page + ingest it as ``llm_tester``.
+2. /query/hybrid as ``llm_tester``: the tenant's own content is reachable
+   and ZERO results come from the ``jpmschweitzer`` tenant.
+3. /query/hybrid as a THIRD, nonexistent tenant inside the reserved
+   namespace (``llm_tester_void``): ZERO results entirely — without ever
+   writing as ``jpmschweitzer``.
+4. Teardown deletes the created page; the session-scoped teardown in
+   conftest purges every remaining llm_tester artifact.
+
+Run with:
+    RUN_INTEGRATION_TESTS=1 TEST_HOST=<shared-host> \\
+        LIBRARY_DESK_URL=http://localhost:8778 \\
+        .venv/bin/python -m pytest tests/test_tenant_isolation_live.py -v
+"""
+
+import uuid
+
+import httpx
+import pytest
+
+from tests.conftest import (
+    LIBRARY_DESK_URL,
+    PRODUCTION_TENANT,
+    TEST_TENANT,
+    assert_safe_test_tenant,
+)
+
+pytestmark = pytest.mark.integration
+
+#: Third tenant: nonexistent, but still inside the reserved namespace so
+#: even its side effects (persisted SearchQuery nodes) stay in test space.
+GHOST_TENANT = f"{TEST_TENANT}_void"
+
+# Disable the web leg so "zero results" is meaningful, and re-ranking so
+# the test does not depend on LLM latency.
+HYBRID_CONFIG = {
+    "enable_web": False,
+    "enable_reranking": False,
+    "enable_vector": True,
+    "enable_graph": True,
+    "enable_volatile": True,
+    "enable_documents": True,
+    "final_result_count": 20,
+}
+
+
+@pytest.fixture(scope="module")
+def api():
+    """HTTP client for the local dev server, with bearer auth."""
+    from src.config import get_settings
+
+    assert ":8089" not in LIBRARY_DESK_URL, (
+        "Refusing to run the live isolation test against the production "
+        "container (port 8089); point LIBRARY_DESK_URL at ./wakeup.sh"
+    )
+    settings = get_settings()
+    client = httpx.Client(
+        base_url=LIBRARY_DESK_URL,
+        headers={"Authorization": f"Bearer {settings.library_api_key}"},
+        timeout=httpx.Timeout(180.0, connect=10.0),
+    )
+    yield client
+    client.close()
+
+
+@pytest.fixture(scope="module")
+def marker() -> str:
+    """Unique content marker for this run."""
+    return f"xylophone quantum walrus {uuid.uuid4().hex[:10]}"
+
+
+@pytest.fixture(scope="module")
+def ingested_page(api, marker):
+    """Create + ingest a wiki page as the test tenant; delete afterwards."""
+    assert_safe_test_tenant(TEST_TENANT)
+
+    page_slug = f"isolation-probe-{uuid.uuid4().hex[:8]}"
+    create_resp = api.post(
+        "/wiki/pages",
+        json={
+            "title": f"Tenant Isolation Probe {marker}",
+            "path": f"/isolation-tests/{page_slug}",
+            "content": (
+                f"# Tenant Isolation Probe\n\n"
+                f"The secret marker phrase is: {marker}.\n"
+                f"This page belongs exclusively to the {TEST_TENANT} tenant "
+                f"and is deleted by the test teardown."
+            ),
+            "description": "Ephemeral tenant-isolation test page",
+            "tags": ["isolation-test"],
+            "user": TEST_TENANT,
+        },
+    )
+    assert create_resp.status_code == 201, create_resp.text
+    page = create_resp.json()
+    page_id = page["id"]
+    assert page["path"].lstrip("/").startswith(f"users/{TEST_TENANT}")
+
+    # Deterministic ingestion (vectors + graph) as the test tenant.
+    ingest_resp = api.post(
+        "/ingest/page",
+        json={"page_id": page_id, "user": TEST_TENANT, "force_refresh": True},
+    )
+    assert ingest_resp.status_code == 200, ingest_resp.text
+    ingest = ingest_resp.json()
+    assert ingest["success"] is True
+    assert ingest["vector_chunks_created"] >= 1
+
+    yield {"page_id": page_id, "path": page["path"]}
+
+    delete_resp = api.delete(f"/wiki/pages/{page_id}", params={"user": TEST_TENANT})
+    assert delete_resp.status_code == 200, delete_resp.text
+
+
+def _hybrid(api, user: str, query: str) -> dict:
+    resp = api.post(
+        "/query/hybrid",
+        params={"user": user},
+        json={"query": query, "config": HYBRID_CONFIG},
+    )
+    assert resp.status_code == 200, resp.text
+    return resp.json()
+
+
+class TestLiveTenantIsolation:
+    def test_own_tenant_sees_own_content_and_nothing_from_production(
+        self, api, marker, ingested_page
+    ):
+        assert_safe_test_tenant(TEST_TENANT)
+        data = _hybrid(api, TEST_TENANT, f"secret marker phrase {marker}")
+
+        # 1) Sanity: the tenant's own freshly-ingested page is retrievable,
+        #    proving the pipeline works and the zero-assertions below are
+        #    meaningful.
+        own_hits = [
+            r for r in data["results"]
+            if r.get("page_id") == ingested_page["page_id"]
+        ]
+        assert own_hits, (
+            f"Expected the ingested page {ingested_page['page_id']} in "
+            f"results: {[(r.get('source_type'), r.get('title')) for r in data['results']]}"
+        )
+
+        # 2) ZERO results from the production tenant.
+        for result in data["results"]:
+            path = (result.get("page_path") or "")
+            assert PRODUCTION_TENANT not in path, (
+                f"PRODUCTION LEAK: result path {path!r} for tenant {TEST_TENANT}"
+            )
+            if result["source_type"] in ("wiki", "vector", "graph"):
+                assert path.lstrip("/").startswith(f"users/{TEST_TENANT}"), (
+                    f"Cross-tenant wiki result: {path!r}"
+                )
+
+        # 3) The formatted LLM context must not leak production paths either.
+        assert PRODUCTION_TENANT not in data["context"]
+
+    def test_nonexistent_tenant_gets_zero_results(self, api, marker, ingested_page):
+        """The inverse check without writing as jpmschweitzer: a third,
+        nonexistent tenant must see NOTHING - not llm_tester's page and
+        not jpmschweitzer's corpus."""
+        assert_safe_test_tenant(GHOST_TENANT)
+
+        data = _hybrid(api, GHOST_TENANT, f"secret marker phrase {marker}")
+
+        assert data["total_results"] == 0, (
+            f"Nonexistent tenant {GHOST_TENANT} got results: "
+            f"{[(r.get('source_type'), r.get('title'), r.get('page_path')) for r in data['results']]}"
+        )
+        assert data["results"] == []
+
+    def test_nonexistent_tenant_gets_zero_results_on_generic_query(self, api):
+        """Even a broad query over common homelab topics returns nothing
+        for a tenant with no data."""
+        assert_safe_test_tenant(GHOST_TENANT)
+
+        data = _hybrid(api, GHOST_TENANT, "docker kubernetes home server setup")
+
+        assert data["total_results"] == 0

tests/test_tenant_scoping.py

@@ -0,0 +1,439 @@
+"""
+Offline tenant-isolation unit tests for every HybridRAG leg (mocked clients).
+
+For each retrieval leg (vector, graph, volatile, documents) plus the
+enrichment and persistence phases, assert that the tenant-scoped
+collection / label / path is used and that cross-tenant access is refused.
+
+Live probes showed /query/hybrid as user=llm_tester returning jpmschweitzer
+pages: the root cause was unscoped ingestion (any tenant could ingest any
+page id / any path prefix into its own collection) and an unscoped
+Document match in search persistence. These tests pin the fixes.
+"""
+
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.models.hybrid_rag import HybridRAGConfig
+from src.services.graph_service import GraphService
+from src.services.hybrid_rag_service import HybridRAGService
+from src.services.ingestion_service import IngestionService
+from src.services.vector_service import VectorService
+from src.services.volatile_service import VolatileCacheService
+
+TENANT = "llm_tester"
+TENANT_COLLECTION = "library_desk_llm_tester"
+TENANT_BASE_LABEL = "User_Llm_Tester"
+TENANT_DOC_LABEL = "User_Llm_Tester_Document"
+
+
+# =============================================================================
+# Fixtures
+# =============================================================================
+
+
+@pytest.fixture
+def mock_qdrant():
+    qdrant = MagicMock()
+    qdrant.collection_exists = AsyncMock(return_value=True)
+    qdrant.ensure_collection = AsyncMock()
+    qdrant.search_vectors = AsyncMock(return_value=[])
+    qdrant.search_with_expiry_filter = AsyncMock(return_value=[])
+    qdrant.upsert_vector = AsyncMock(return_value=True)
+    qdrant.upsert_points = AsyncMock(side_effect=lambda collection_name, points: len(points))
+    qdrant.scroll_all_points = AsyncMock(return_value=[])
+    qdrant.delete_by_ids = AsyncMock(return_value=0)
+    qdrant.delete_by_filter = AsyncMock(return_value=0)
+    return qdrant
+
+
+@pytest.fixture
+def mock_ollama():
+    ollama = MagicMock()
+    ollama.embed = AsyncMock(return_value=[0.1] * 768)
+    ollama.embed_batch = AsyncMock(
+        side_effect=lambda texts, **kw: [[0.1] * 768 for _ in texts]
+    )
+    # NOTE: do not stub methods that don't exist on OllamaClient (a stale
+    # embed_text stub here previously hid a latent AttributeError in the
+    # HybridRAG document leg - see 406143e).
+    ollama.generate_text = AsyncMock(return_value="{}")
+    return ollama
+
+
+@pytest.fixture
+def mock_neo4j():
+    neo4j = MagicMock()
+    neo4j.execute_query = AsyncMock(return_value=[])
+    neo4j.execute_read = AsyncMock(return_value=[])
+    neo4j.execute_write = AsyncMock(return_value=[])
+    return neo4j
+
+
+@pytest.fixture
+def mock_wiki():
+    wiki = MagicMock()
+    wiki.get_page = AsyncMock(return_value=None)
+    wiki.list_all_pages = AsyncMock(return_value=[])
+    wiki.list_pages = AsyncMock(return_value=[])
+    return wiki
+
+
+@pytest.fixture
+def vector_service(mock_qdrant, mock_wiki, mock_ollama):
+    return VectorService(mock_qdrant, mock_wiki, mock_ollama)
+
+
+@pytest.fixture
+def graph_service(mock_neo4j, mock_wiki):
+    return GraphService(mock_neo4j, mock_wiki)
+
+
+@pytest.fixture
+def volatile_service(mock_qdrant, mock_ollama):
+    return VolatileCacheService(mock_qdrant, mock_ollama, MagicMock())
+
+
+@pytest.fixture
+def hybrid_service(vector_service, graph_service, volatile_service, mock_ollama):
+    settings = MagicMock()
+    settings.ollama_llm_model = "test-model"
+    settings.vector_similarity_threshold = 0.7
+    return HybridRAGService(
+        vector_service=vector_service,
+        graph_service=graph_service,
+        searxng_client=MagicMock(),
+        ollama_client=mock_ollama,
+        content_extractor=MagicMock(),
+        settings=settings,
+        volatile_service=volatile_service,
+    )
+
+
+def _all_cypher(mock_neo4j) -> str:
+    """Concatenate all Cypher sent to the mocked Neo4j client (reads + writes)."""
+    return "\n".join(
+        str(call.args[0])
+        for mock in (mock_neo4j.execute_query, mock_neo4j.execute_write)
+        for call in mock.await_args_list
+    )
+
+
+# =============================================================================
+# Vector leg
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestVectorLegScoping:
+    async def test_search_uses_tenant_collection(self, vector_service, mock_qdrant):
+        await vector_service.search(query="q", user=TENANT)
+
+        mock_qdrant.collection_exists.assert_awaited_with(TENANT_COLLECTION)
+        assert (
+            mock_qdrant.search_vectors.await_args.kwargs["collection_name"]
+            == TENANT_COLLECTION
+        )
+
+    async def test_delete_page_chunks_uses_tenant_collection(
+        self, vector_service, mock_qdrant
+    ):
+        await vector_service.delete_page_chunks(page_id=1, user=TENANT)
+
+        assert (
+            mock_qdrant.delete_by_filter.await_args.kwargs["collection_name"]
+            == TENANT_COLLECTION
+        )
+
+    async def test_update_from_page_rejects_foreign_namespace(
+        self, vector_service, mock_qdrant, mock_wiki
+    ):
+        """Ingesting another tenant's page into our collection must fail."""
+        mock_wiki.get_page = AsyncMock(return_value={
+            "id": 42, "title": "T", "path": "users/jpmschweitzer/secret",
+            "content": "secret content", "tags": [],
+        })
+
+        summary = await vector_service.update_from_page(page_id=42, user=TENANT)
+
+        assert summary.success is False
+        assert "outside user" in (summary.error_message or "")
+        mock_qdrant.upsert_points.assert_not_awaited()
+        mock_qdrant.delete_by_ids.assert_not_awaited()
+
+    async def test_update_from_page_rejects_sibling_prefix_namespace(
+        self, vector_service, mock_qdrant, mock_wiki
+    ):
+        """users/llm_tester2 is NOT inside llm_tester's namespace."""
+        mock_wiki.get_page = AsyncMock(return_value={
+            "id": 43, "title": "T", "path": "users/llm_tester2/page",
+            "content": "content", "tags": [],
+        })
+
+        summary = await vector_service.update_from_page(page_id=43, user=TENANT)
+
+        assert summary.success is False
+        mock_qdrant.upsert_points.assert_not_awaited()
+
+    async def test_update_from_page_accepts_own_namespace(
+        self, vector_service, mock_qdrant, mock_wiki
+    ):
+        mock_wiki.get_page = AsyncMock(return_value={
+            "id": 44, "title": "T", "path": "users/llm_tester/page",
+            "content": "hello world", "tags": [],
+        })
+
+        summary = await vector_service.update_from_page(page_id=44, user=TENANT)
+
+        assert summary.success is True
+        assert (
+            mock_qdrant.upsert_points.await_args.kwargs["collection_name"]
+            == TENANT_COLLECTION
+        )
+
+
+# =============================================================================
+# Graph leg
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestGraphLegScoping:
+    async def test_search_documents_uses_tenant_labels(
+        self, graph_service, mock_neo4j
+    ):
+        await graph_service.search_documents(
+            query="docker", user=TENANT, keywords_data={"core_keywords": ["docker"]}
+        )
+
+        cypher = _all_cypher(mock_neo4j)
+        assert f"(e:{TENANT_BASE_LABEL})" in cypher
+        assert f"(d:{TENANT_DOC_LABEL}:Document)" in cypher
+
+    async def test_update_from_page_rejects_foreign_namespace(
+        self, graph_service, mock_neo4j, mock_wiki
+    ):
+        mock_wiki.get_page = AsyncMock(return_value={
+            "id": 42, "title": "T", "path": "users/jpmschweitzer/secret",
+            "content": "secret", "tags": [],
+        })
+
+        summary = await graph_service.update_from_page(page_id=42, user=TENANT)
+
+        assert summary.success is False
+        assert "outside user" in (summary.error_message or "")
+        mock_neo4j.execute_query.assert_not_awaited()
+
+    async def test_update_from_page_writes_tenant_labels(
+        self, graph_service, mock_neo4j, mock_wiki
+    ):
+        mock_wiki.get_page = AsyncMock(return_value={
+            "id": 44, "title": "T", "path": "users/llm_tester/page",
+            "content": "Uses Docker daily", "tags": [],
+        })
+
+        summary = await graph_service.update_from_page(page_id=44, user=TENANT)
+
+        assert summary.success is True
+        cypher = _all_cypher(mock_neo4j)
+        assert TENANT_DOC_LABEL in cypher
+        # No unscoped Document writes
+        assert "MERGE (d:Document {" not in cypher
+
+    async def test_entity_mention_count_scoped(self, graph_service, mock_neo4j):
+        await graph_service._get_entity_mention_count("Docker", "Technology", TENANT)
+
+        cypher = _all_cypher(mock_neo4j)
+        assert f"(d:{TENANT_DOC_LABEL}:Document)" in cypher
+        assert "MATCH (d:Document)" not in cypher
+
+    async def test_orphan_queries_scoped(self, graph_service, mock_neo4j):
+        await graph_service.find_orphan_entities(TENANT)
+        await graph_service.purge_orphan_entities(TENANT)
+
+        cypher = _all_cypher(mock_neo4j)
+        assert f"(d:{TENANT_DOC_LABEL}:Document)" in cypher
+        assert "(d:Document)-[:MENTIONS]" not in cypher
+
+    async def test_cleanup_broken_relationships_scoped(
+        self, graph_service, mock_neo4j
+    ):
+        await graph_service.cleanup_broken_relationships(TENANT)
+
+        cypher = _all_cypher(mock_neo4j)
+        assert f"{TENANT_BASE_LABEL}_SearchQuery" in cypher
+        assert "MATCH (sq:SearchQuery)" not in cypher
+
+
+# =============================================================================
+# Volatile leg
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestVolatileLegScoping:
+    async def test_search_uses_tenant_collection(self, volatile_service, mock_qdrant):
+        await volatile_service.search(user=TENANT, query="weather")
+
+        mock_qdrant.collection_exists.assert_awaited_with("volatile_llm_tester")
+        assert (
+            mock_qdrant.search_with_expiry_filter.await_args.kwargs["collection_name"]
+            == "volatile_llm_tester"
+        )
+
+    async def test_store_uses_tenant_collection(self, volatile_service, mock_qdrant):
+        await volatile_service.store(
+            user=TENANT, namespace="weather", key="rotterdam", data={"t": 1}
+        )
+
+        assert (
+            mock_qdrant.upsert_vector.await_args.kwargs["collection_name"]
+            == "volatile_llm_tester"
+        )
+
+    def test_collection_name_is_sanitized(self, volatile_service):
+        """Raw user strings cannot alias/escape the collection scheme."""
+        assert volatile_service._collection_name("llm-tester") == "volatile_llm_tester"
+        assert volatile_service._collection_name("Evil User!") == "volatile_evil_user"
+
+
+# =============================================================================
+# Documents leg (Paperless chunks in the tenant collection)
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestDocumentLegScoping:
+    async def test_document_search_uses_tenant_collection(
+        self, hybrid_service, mock_qdrant
+    ):
+        config = HybridRAGConfig(
+            enable_vector=False, enable_graph=False, enable_web=False,
+            enable_volatile=False, enable_documents=True,
+        )
+
+        await hybrid_service._retrieve_parallel("q", TENANT, config, {})
+
+        assert (
+            mock_qdrant.search_vectors.await_args.kwargs["collection_name"]
+            == TENANT_COLLECTION
+        )
+        assert (
+            mock_qdrant.search_vectors.await_args.kwargs["filter_conditions"]
+            == {"doc_type": "document"}
+        )
+
+
+# =============================================================================
+# Enrichment phase
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestEnrichmentScoping:
+    async def test_related_documents_scoped_to_tenant(
+        self, graph_service, mock_neo4j
+    ):
+        await graph_service.get_related_documents(page_id=1, user=TENANT)
+
+        cypher = _all_cypher(mock_neo4j)
+        assert f"(d1:{TENANT_DOC_LABEL}:Document" in cypher
+        assert f"(d2:{TENANT_DOC_LABEL}:Document)" in cypher
+
+    async def test_batched_related_documents_scoped_to_tenant(
+        self, graph_service, mock_neo4j
+    ):
+        await graph_service.get_related_documents_batch(
+            page_ids=[1, 2], user=TENANT
+        )
+
+        cypher = _all_cypher(mock_neo4j)
+        assert "UNWIND $page_ids" in cypher
+        assert f"(d1:{TENANT_DOC_LABEL}:Document" in cypher
+        assert f"(d2:{TENANT_DOC_LABEL}:Document)" in cypher
+
+
+# =============================================================================
+# Persistence phase (SearchQuery / FOUND / WebResult)
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestPersistenceScoping:
+    async def test_persist_scopes_searchquery_links_and_webresults(
+        self, hybrid_service, mock_neo4j
+    ):
+        final_results = [
+            {
+                "result": {"page_id": 42, "title": "wiki hit"},
+                "source_type": "wiki", "sources": ["vector"],
+                "rrf_score": 0.5, "final_rank": 1,
+            },
+            {
+                "result": {"url": "http://example.com", "title": "web hit",
+                           "content": "c"},
+                "source_type": "web", "sources": ["web"],
+                "rrf_score": 0.4, "final_rank": 2,
+            },
+        ]
+
+        await hybrid_service._persist_search_for_librarian(
+            search_id="sid-1", query="q", user=TENANT, keywords_data={},
+            raw_results={}, final_results=final_results, timing={},
+        )
+
+        # Persistence is now ONE atomic write transaction
+        mock_neo4j.execute_write.assert_awaited_once()
+        cypher = str(mock_neo4j.execute_write.await_args.args[0])
+        # SearchQuery + WebResult nodes carry the tenant label
+        assert f"{TENANT_BASE_LABEL}_SearchQuery" in cypher
+        assert f"{TENANT_BASE_LABEL}_WebResult" in cypher
+        # FOUND link matches only this tenant's Document nodes
+        assert f"MATCH (d:{TENANT_DOC_LABEL}:Document {{page_id: link.page_id}})" in cypher
+        # The old cross-tenant match must be gone
+        assert "MATCH (d:Document {page_id:" not in cypher
+
+
+# =============================================================================
+# Ingestion (feeds the vector/graph legs)
+# =============================================================================
+
+
+@pytest.mark.unit
+class TestIngestAllPagesScoping:
+    @pytest.fixture
+    def ingestion_service(self, vector_service, graph_service, mock_wiki):
+        return IngestionService(vector_service, graph_service, mock_wiki)
+
+    async def test_defaults_to_own_namespace(self, ingestion_service, mock_wiki):
+        await ingestion_service.ingest_all_pages(user=TENANT)
+
+        mock_wiki.list_all_pages.assert_awaited_once_with(
+            path_prefix="users/llm_tester"
+        )
+
+    async def test_rejects_foreign_prefix(self, ingestion_service, mock_wiki):
+        with pytest.raises(ValueError, match="outside user"):
+            await ingestion_service.ingest_all_pages(
+                user=TENANT, path_prefix="users/jpmschweitzer"
+            )
+
+        mock_wiki.list_all_pages.assert_not_awaited()
+
+    async def test_rejects_sibling_prefix(self, ingestion_service, mock_wiki):
+        with pytest.raises(ValueError, match="outside user"):
+            await ingestion_service.ingest_all_pages(
+                user=TENANT, path_prefix="users/llm_tester2"
+            )
+
+    async def test_accepts_subtree_of_own_namespace(
+        self, ingestion_service, mock_wiki
+    ):
+        await ingestion_service.ingest_all_pages(
+            user=TENANT, path_prefix="users/llm_tester/tech"
+        )
+
+        mock_wiki.list_all_pages.assert_awaited_once_with(
+            path_prefix="users/llm_tester/tech"
+        )

tests/test_vector_reindex.py

@@ -0,0 +1,134 @@
+"""
+Offline unit tests for the batched, delete-last vector reindex path.
+
+Pins the Phase D fixes in VectorService.update_from_page:
+- embeddings come from ONE batched embed_batch call, not per-chunk embed()
+- new points are upserted BEFORE stale points are deleted (deterministic
+  uuid5 ids make the overwrite safe), so a failure can no longer leave
+  the page with zero vectors
+- the summary reports partial/failed status instead of unconditional
+  success=True when chunks are skipped
+"""
+
+import uuid
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+from src.services.vector_service import VectorService
+
+TENANT = "llm_tester"
+TENANT_COLLECTION = "library_desk_llm_tester"
+PAGE = {
+    "id": 44,
+    "title": "T",
+    "path": "users/llm_tester/page",
+    "content": " ".join(f"word{i}" for i in range(1200)),  # 3 chunks @ 500/50
+    "tags": [],
+}
+
+
+def _chunk_id(page_id: int, idx: int) -> str:
+    return str(uuid.uuid5(uuid.NAMESPACE_DNS, f"page_{page_id}_chunk_{idx}"))
+
+
+@pytest.fixture
+def mock_qdrant():
+    qdrant = MagicMock()
+    qdrant.collection_exists = AsyncMock(return_value=True)
+    qdrant.ensure_collection = AsyncMock()
+    qdrant.upsert_points = AsyncMock(side_effect=lambda collection_name, points: len(points))
+    qdrant.scroll_all_points = AsyncMock(return_value=[])
+    qdrant.delete_by_ids = AsyncMock(side_effect=lambda collection_name, point_ids: len(point_ids))
+    qdrant.delete_by_filter = AsyncMock(return_value=0)
+    return qdrant
+
+
+@pytest.fixture
+def mock_ollama():
+    ollama = MagicMock()
+    ollama.embed_batch = AsyncMock(
+        side_effect=lambda texts, **kw: [[0.1] * 768 for _ in texts]
+    )
+    return ollama
+
+
+@pytest.fixture
+def mock_wiki():
+    wiki = MagicMock()
+    wiki.get_page = AsyncMock(return_value=dict(PAGE))
+    return wiki
+
+
+@pytest.fixture
+def service(mock_qdrant, mock_wiki, mock_ollama):
+    return VectorService(mock_qdrant, mock_wiki, mock_ollama)
+
+
+@pytest.mark.unit
+class TestBatchedReindex:
+    async def test_single_batched_embed_and_single_upsert(
+        self, service, mock_qdrant, mock_ollama
+    ):
+        summary = await service.update_from_page(page_id=44, user=TENANT)
+
+        assert summary.success is True
+        assert summary.status == "success"
+        assert summary.chunks_created == 3
+        mock_ollama.embed_batch.assert_awaited_once()
+        mock_qdrant.upsert_points.assert_awaited_once()
+
+    async def test_upsert_happens_before_stale_delete(self, service, mock_qdrant):
+        order = []
+        mock_qdrant.upsert_points = AsyncMock(
+            side_effect=lambda collection_name, points: order.append("upsert") or len(points)
+        )
+        mock_qdrant.delete_by_ids = AsyncMock(
+            side_effect=lambda collection_name, point_ids: order.append("delete") or len(point_ids)
+        )
+        # One stale point from a previous, longer version of the page
+        mock_qdrant.scroll_all_points = AsyncMock(return_value=[
+            {"id": _chunk_id(44, i), "payload": {}} for i in range(4)
+        ])
+
+        summary = await service.update_from_page(page_id=44, user=TENANT)
+
+        assert order == ["upsert", "delete"]
+        assert summary.chunks_deleted == 1  # only the stale 4th chunk
+        deleted = mock_qdrant.delete_by_ids.await_args.kwargs["point_ids"]
+        assert deleted == [_chunk_id(44, 3)]
+
+    async def test_partial_embedding_failure_marks_partial(
+        self, service, mock_qdrant, mock_ollama
+    ):
+        mock_ollama.embed_batch = AsyncMock(
+            side_effect=lambda texts, **kw: [
+                [0.1] * 768 if i != 1 else None for i in range(len(texts))
+            ]
+        )
+
+        summary = await service.update_from_page(page_id=44, user=TENANT)
+
+        assert summary.status == "partial"
+        assert summary.success is True
+        assert summary.chunks_created == 2
+        assert summary.chunks_skipped == 1
+        assert "failed" in (summary.error_message or "")
+
+    async def test_total_embedding_failure_keeps_old_vectors(
+        self, service, mock_qdrant, mock_ollama
+    ):
+        mock_ollama.embed_batch = AsyncMock(
+            side_effect=lambda texts, **kw: [None for _ in texts]
+        )
+        mock_qdrant.scroll_all_points = AsyncMock(return_value=[
+            {"id": _chunk_id(44, 0), "payload": {}}
+        ])
+
+        summary = await service.update_from_page(page_id=44, user=TENANT)
+
+        assert summary.status == "failed"
+        assert summary.success is False
+        # Old vectors are NOT wiped when nothing new was stored
+        mock_qdrant.upsert_points.assert_not_awaited()
+        mock_qdrant.delete_by_ids.assert_not_awaited()

tests/test_volatile.py

@@ -98,19 +98,19 @@ class TestVolatileNamespaces:
 
     def test_weather_default_ttl(self):
         """Test weather namespace default TTL."""
-        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.WEATHER] == 1800  # 30 min
+        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.WEATHER] == 7200  # 2 hours (hourly refresh)
 
     def test_financial_default_ttl(self):
         """Test financial namespace default TTL."""
-        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.FINANCIAL] == 300  # 5 min
+        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.FINANCIAL] == 600  # 10 min (5 min refresh)
 
     def test_sports_default_ttl(self):
         """Test sports namespace default TTL (fast updates)."""
-        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.SPORTS] == 60  # 1 min
+        assert NAMESPACE_DEFAULT_TTL[VolatileNamespace.SPORTS] == 120  # 2 min (1 min refresh)
 
     def test_namespace_count(self):
         """Test we have the expected number of namespaces."""
-        assert len(VolatileNamespace) == 12  # Including SUN for sunrise/sunset
+        assert len(VolatileNamespace) == 13  # Including SUN, FORECAST
 
 
 class TestVolatileListResponse:
@@ -274,7 +274,7 @@ class TestVolatileService:
     def test_get_default_ttl_known_namespace(self, volatile_service):
         """Test default TTL for known namespace."""
         ttl = volatile_service._get_default_ttl("weather")
-        assert ttl == 1800  # Weather namespace default
+        assert ttl == 7200  # Weather namespace default (2 hours)
 
     def test_get_default_ttl_unknown_namespace(self, volatile_service):
         """Test default TTL for unknown namespace."""
@@ -366,7 +366,7 @@ class TestVolatileService:
             ttl=None  # Not specified
         )
 
-        assert result.ttl == 1800  # Weather default
+        assert result.ttl == 7200  # Weather default (2 hours)
 
     @pytest.mark.asyncio
     async def test_search_empty_collection(self, volatile_service, mock_qdrant, mock_ollama):

tests/test_wiki_change_listener.py

@@ -231,9 +231,8 @@ class TestWikiChangeListener:
                 'UPDATE:123:invaliduser'
             )
 
-            # Should use default user
-            call_args = mock_process.call_args[1]
-            assert call_args['user'] == 'jpmschweitzer'
+            # No default tenant: change must be skipped entirely
+            mock_process.assert_not_awaited()
 
     @pytest.mark.asyncio
     async def test_process_page_delete_calls_cleanup(self, listener):
@@ -309,6 +308,10 @@ class TestWikiChangeListener:
     async def test_connection_lifecycle(self, listener, mock_settings):
         """Test listener connection start and stop lifecycle."""
         mock_connection = AsyncMock()
+        # asyncpg's is_closed() is synchronous. Left as an AsyncMock it returns a
+        # coroutine, which is always truthy, so the connection would read as
+        # closed the moment `running` started deriving from it.
+        mock_connection.is_closed = MagicMock(return_value=False)
 
         with patch('src.services.wiki_change_listener.asyncpg.connect', return_value=mock_connection) as mock_connect:
             # Start listener
@@ -427,3 +430,168 @@ class TestWikiChangeListenerIntegration:
 
         # This would test actual pg_notify() calls from triggers
         # and verify the listener receives and processes them
+
+
+class TestWikiChangeListenerReconnect:
+    """
+    Supervision and reconnect behaviour.
+
+    Regression cover for 2026-08-08: redeploying postgres-shared dropped the
+    listener's connection and it never came back. The container kept reporting
+    healthy because nothing observed the subscription, so Wiki.js page edits
+    silently stopped being indexed until someone restarted the service.
+    """
+
+    @pytest.mark.asyncio
+    async def test_running_is_false_when_connection_closed(self, listener):
+        """running reflects the live connection, not a flag set once at startup."""
+        connection = MagicMock()
+        connection.is_closed.return_value = False
+        listener.connection = connection
+        assert listener.running is True
+
+        # The exact failure mode: connection dies, nothing reassigns a flag.
+        connection.is_closed.return_value = True
+        assert listener.running is False
+
+    @pytest.mark.asyncio
+    async def test_running_is_false_while_stopping(self, listener):
+        """A listener being torn down does not advertise itself as subscribed."""
+        connection = MagicMock()
+        connection.is_closed.return_value = False
+        listener.connection = connection
+        listener._stopping = True
+        assert listener.running is False
+
+    @pytest.mark.asyncio
+    async def test_termination_callback_wakes_supervisor(self, listener):
+        """asyncpg's termination callback signals the supervisor and records the time."""
+        assert not listener._disconnected.is_set()
+
+        listener._on_connection_lost(MagicMock())
+
+        assert listener._disconnected.is_set()
+        assert listener._disconnected_at is not None
+
+    @pytest.mark.asyncio
+    async def test_termination_callback_ignored_while_stopping(self, listener):
+        """A deliberate shutdown must not trigger a reconnect."""
+        listener._stopping = True
+
+        listener._on_connection_lost(MagicMock())
+
+        assert not listener._disconnected.is_set()
+
+    @pytest.mark.asyncio
+    async def test_supervisor_reconnects_after_drop(self, listener):
+        """One drop produces one reconnect cycle."""
+        connection = MagicMock()
+        connection.is_closed.return_value = True
+        listener.connection = connection
+
+        with patch.object(listener, '_connect', new=AsyncMock()) as connect:
+            listener._disconnected.set()
+            cycles = await listener._supervise(max_iterations=1)
+
+        assert cycles == 1
+        connect.assert_awaited_once()
+        assert listener.reconnects == 1
+
+    @pytest.mark.asyncio
+    async def test_supervisor_exits_without_reconnecting_when_stopping(self, listener):
+        """stop() wakes the supervisor to exit, not to re-establish the connection."""
+        with patch.object(listener, '_connect', new=AsyncMock()) as connect:
+            listener._stopping = True
+            listener._disconnected.set()
+            cycles = await listener._supervise(max_iterations=1)
+
+        assert cycles == 0
+        connect.assert_not_awaited()
+
+    @pytest.mark.asyncio
+    async def test_reconnect_backs_off_and_retries(self, listener):
+        """A database that is still down is retried, with growing delay."""
+        attempts = []
+
+        async def fail_twice_then_succeed():
+            attempts.append(1)
+            if len(attempts) < 3:
+                raise OSError("connection refused")
+
+        sleeps = []
+
+        async def fake_sleep(seconds):
+            sleeps.append(seconds)
+
+        with patch.object(listener, '_connect', new=AsyncMock(side_effect=fail_twice_then_succeed)), \
+             patch.object(listener, '_close_connection', new=AsyncMock()), \
+             patch('asyncio.sleep', new=fake_sleep):
+            ok = await listener._reconnect_with_backoff()
+
+        assert ok is True
+        assert len(attempts) == 3
+        assert sleeps == [1.0, 2.0]           # doubling
+        assert listener.reconnects == 1
+
+    @pytest.mark.asyncio
+    async def test_reconnect_backoff_is_capped(self, listener):
+        """Backoff does not grow without bound during a long outage."""
+        sleeps = []
+
+        async def fake_sleep(seconds):
+            sleeps.append(seconds)
+
+        with patch.object(listener, '_connect', new=AsyncMock(side_effect=OSError("down"))), \
+             patch.object(listener, '_close_connection', new=AsyncMock()), \
+             patch('asyncio.sleep', new=fake_sleep):
+            ok = await listener._reconnect_with_backoff(max_attempts=12)
+
+        assert ok is False
+        assert max(sleeps) == listener.BACKOFF_MAX_SECONDS
+        assert listener.reconnects == 0
+
+    @pytest.mark.asyncio
+    async def test_reconnect_records_outage_duration(self, listener):
+        """The gap is measured so the lost-notification window is reportable."""
+        listener._disconnected_at = datetime.now() - timedelta(seconds=30)
+
+        with patch.object(listener, '_connect', new=AsyncMock()), \
+             patch.object(listener, '_close_connection', new=AsyncMock()):
+            await listener._reconnect_with_backoff()
+
+        assert listener.last_gap_seconds is not None
+        assert 29 <= listener.last_gap_seconds <= 32
+        assert listener._disconnected_at is None
+
+    @pytest.mark.asyncio
+    async def test_close_connection_tolerates_dead_connection(self, listener):
+        """Cleaning up an already-terminated connection must not raise."""
+        connection = MagicMock()
+        connection.is_closed.return_value = False
+        connection.remove_listener = AsyncMock(side_effect=Exception("connection is closed"))
+        connection.close = AsyncMock()
+        listener.connection = connection
+
+        await listener._close_connection()
+
+        assert listener.connection is None
+
+    @pytest.mark.asyncio
+    async def test_start_registers_termination_listener(self, listener):
+        """
+        The termination listener is re-registered on every connection.
+
+        asyncpg clears its termination listeners as soon as it fires them, so a
+        registration that happened only once would survive exactly one drop.
+        """
+        connection = MagicMock()
+        connection.add_listener = AsyncMock()
+        connection.is_closed.return_value = False
+
+        with patch('asyncpg.connect', new=AsyncMock(return_value=connection)):
+            await listener._connect()
+
+        connection.add_listener.assert_awaited_once()
+        connection.add_termination_listener.assert_called_once_with(
+            listener._on_connection_lost
+        )

tests/test_wikijs_pagination.py

@@ -0,0 +1,147 @@
+"""
+Unit tests for WikiJSClient page listing and pagination (offline).
+
+Wiki.js 2.x `pages.list` supports only a `limit` argument (no offset), and
+the limit is applied BEFORE Wiki.js's own visibility filter — fewer pages
+than requested does NOT mean the listing is complete. Exhaustive listing
+therefore grows the limit until the returned count stops increasing.
+These tests mock the GraphQL layer.
+"""
+
+import pytest
+from unittest.mock import AsyncMock
+
+from src.clients.wikijs_client import WikiJSClient
+
+
+def make_pages(count_other: int, count_user: int):
+    """Build a fake TITLE-ordered page listing: 'other/' pages sort before 'users/jp/' pages."""
+    pages = [
+        {"id": i, "path": f"other/p{i:03d}", "title": f"A{i:03d}", "tags": []}
+        for i in range(count_other)
+    ]
+    pages += [
+        {"id": count_other + i, "path": f"users/jp/p{i:03d}", "title": f"Z{i:03d}", "tags": ["projects"]}
+        for i in range(count_user)
+    ]
+    return pages
+
+
+@pytest.fixture
+def client():
+    """WikiJSClient with a mocked GraphQL layer serving 138 pages (100 other + 38 user)."""
+    client = WikiJSClient("http://wiki.test", "")
+    client._all_pages = make_pages(100, 38)
+    client._requested_limits = []
+
+    async def fake_execute(query, variables=None):
+        limit = variables["limit"]
+        client._requested_limits.append(limit)
+        return {"pages": {"list": client._all_pages[:limit]}}
+
+    client._execute_query = fake_execute
+    return client
+
+
+@pytest.mark.unit
+class TestListAllPagesPagination:
+    """list_all_pages must exhaust the listing via a limit-growth loop."""
+
+    async def test_grows_limit_until_count_stabilizes(self, client):
+        pages = await client.list_all_pages(batch_size=50)
+
+        # 50 -> 50, 100 -> 100, 200 -> 138, 400 -> 138 (stable) done
+        assert client._requested_limits == [50, 100, 200, 400]
+        assert len(pages) == 138
+
+    async def test_confirms_completeness_with_second_fetch(self, client):
+        """A single not-full batch is NOT trusted (limit precedes Wiki.js's
+        visibility filter); a confirming fetch at a doubled limit runs."""
+        pages = await client.list_all_pages(batch_size=500)
+
+        assert client._requested_limits == [500, 1000]
+        assert len(pages) == 138
+
+    async def test_pre_filter_limit_does_not_truncate(self, client):
+        """Regression (observed live): Wiki.js applies `limit` before its
+        visibility filter, so limit=100 returned 43 pages while 140 existed.
+        The old `len < limit` stop condition silently dropped pages."""
+        all_pages = client._all_pages
+
+        async def fake_execute(query, variables=None):
+            limit = variables["limit"]
+            client._requested_limits.append(limit)
+            # Only ~half the pages within the limit window are visible
+            return {"pages": {"list": all_pages[: limit // 2]}}
+
+        client._execute_query = fake_execute
+
+        pages = await client.list_all_pages(batch_size=100)
+
+        # 100 -> 50 (< limit, but NOT complete), 200 -> 100, 400 -> 138,
+        # 800 -> 138 (stable) done
+        assert len(pages) == 138
+
+    async def test_path_prefix_filter_after_exhaustion(self, client):
+        """All 38 user pages are returned even though they sort last (beyond batch_size)."""
+        pages = await client.list_all_pages(path_prefix="users/jp", batch_size=50)
+
+        assert len(pages) == 38
+        assert all(p["path"].startswith("users/jp") for p in pages)
+
+    async def test_empty_wiki(self, client):
+        client._all_pages = []
+
+        pages = await client.list_all_pages(batch_size=100)
+
+        assert pages == []
+
+
+@pytest.mark.unit
+class TestListPagesLimitAfterFilter:
+    """list_pages must apply `limit` AFTER client-side filters, not before."""
+
+    async def test_prefix_filter_with_small_limit(self, client):
+        """Old defect: API limit=5 returned 5 'other/' pages, filter dropped all -> 0 results."""
+        pages = await client.list_pages(path_prefix="users/jp", limit=5)
+
+        assert len(pages) == 5
+        assert all(p["path"].startswith("users/jp") for p in pages)
+
+    async def test_tag_filter_with_small_limit(self, client):
+        pages = await client.list_pages(tags=["projects"], limit=10)
+
+        assert len(pages) == 10
+        assert all("projects" in p["tags"] for p in pages)
+
+    async def test_unfiltered_passes_limit_to_api(self, client):
+        pages = await client.list_pages(limit=7)
+
+        assert client._requested_limits == [7]
+        assert len(pages) == 7
+
+    async def test_tags_normalized_to_list(self, client):
+        client._all_pages = [{"id": 1, "path": "home", "title": "Home", "tags": None}]
+
+        pages = await client.list_pages(limit=10)
+
+        assert pages[0]["tags"] == []
+
+
+class TestSearchPagesPrefixNormalization:
+    """search_pages must match Wiki.js paths (no leading slash) against
+    get_wikijs_namespace prefixes (leading slash)."""
+
+    @pytest.mark.asyncio
+    async def test_slashed_prefix_matches_unslashed_paths(self):
+        client = WikiJSClient("http://wiki.test", "k")
+        client._execute_query = AsyncMock(return_value={
+            "pages": {"search": {"results": [
+                {"id": "399", "path": "users/llm_tester/docker-guide",
+                 "title": "Docker Guide", "description": ""},
+                {"id": "1", "path": "users/jpmschweitzer/other",
+                 "title": "Other", "description": ""},
+            ]}}
+        })
+        results = await client.search_pages("docker", path_prefix="/users/llm_tester")
+        assert [r["id"] for r in results] == ["399"]