Commit Graph
8 Commits
Author SHA1 Message Date
jpmschweitzerandClaude a687b770ef fix: clear ruff so the pre-push gate passes
105 findings to zero. Most were mechanical — 67 unused imports, and assorted
f-strings without placeholders. Three groups needed a decision.

The 15 F821 "undefined name" were forward references, not runtime errors. Each
annotation is quoted — `-> "WikiService"`, `Optional["IngestionService"]` — with
the real import inside the function body to break an import cycle. A quoted
annotation is never evaluated, so the code ran; the names were simply
unresolvable to any checker. They now have a TYPE_CHECKING block, which costs
nothing at import time and keeps the cycle broken.

The 6 E402 split two ways. `import secrets`, `Security`, `Request` and
`HTTPBearer` in dependencies.py had drifted below several hundred lines of
factory functions for no reason — stdlib and fastapi, no cycle to avoid — and
moved up. The other three are deliberate and now say so: the VectorService and
GraphService aliases import back into dependencies.py, and main.py's routers
expect a configured app, so both must stay put.

Bare `except:` narrowed to `except Exception:` in three places, which stops them
swallowing KeyboardInterrupt and SystemExit.

The 5 unused locals were all genuinely dead. One is worth naming rather than
fixing: qdrant_client.delete()'s return value was bound and never read, so a
failed delete is indistinguishable from a successful one — the assignment is
gone, but nothing checks the status either way and that has not changed here.
`timing = {}` in _retrieve_parallel looked like it might mean the reported
per-leg timings were always zero; traced, and they come from output["timing"],
so the local was only vestigial.

426 passed, 29 skipped, unchanged. The app imports and the service aliases still
resolve, which is the check that mattered after moving imports in
dependencies.py.

The gate still prints "not gated here yet: test (T-56)" — lint is green, tests
remain unwired, and that is left visible rather than silently absent.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 17:04:58 +02:00
jpmschweitzerandClaude Fable 5 c35d3c1fa9 fix: await ensure_collection and survive partial embed failures in document sync
- ensure_collection was called without await, so the coroutine never ran
  and fresh tenants had no collection when the upsert hit Qdrant
- a single None entry from embed_batch poisoned the point batch and
  aborted the whole document upsert; failed chunks are now skipped with
  a warning (all-failed raises and the IndexResult reports failure)
- raw client.delete/client.upsert calls now go through the async wrapper
  (delete_by_filter and the new batch upsert_points method)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 13:00:59 +02:00
jpmschweitzerandClaude Fable 5 406143e7ae perf: switch Qdrant to AsyncQdrantClient with explicit timeout
Every vector call ran on the sync QdrantClient inside async wrapper
methods, blocking the FastAPI event loop per Qdrant round-trip. The
wrapper now holds an AsyncQdrantClient (timeout via QDRANT_TIMEOUT,
default 30s) and awaits all client calls; the wrapper API is unchanged.

Call sites off the wrapper were fixed too: the HybridRAG document leg
now uses the async search_vectors wrapper instead of the deprecated raw
client.search (also fixing its call to the nonexistent ollama.embed_text
which made the leg permanently report 'failed'), the health check awaits
get_collections, and document_sync's raw delete/upsert calls are awaited
(routed through wrappers in the next commit).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 12:58:01 +02:00
jpmschweitzerandClaude Fable 5 86051d8022 feat: implement check-updates, job-backed ingest status, and dedup scan
Replace the four stub endpoints with real implementations, all requiring
an explicit tenant user (Phase B rule):

- /ingest/check-updates: GraphService now records a SHA-256 content_hash
  on every Document node at ingestion time; the endpoint compares those
  stored hashes against current Wiki.js page content in one UNWIND Cypher
  query per tenant and returns changed/new/deleted page lists (entity-stub
  pages excluded, pre-hash-tracking documents flagged stored_hash_missing).
- /ingest/status/{job_id}: backed by the Redis JobManager; jobs are
  tenant-scoped (foreign jobs 404). /ingest/page, /ingest/batch and
  /ingest/all now create job records and return job_id.
- /ingest/repo-status/{repository}: wiki page count vs indexed Document
  nodes under users/{tenant}/{repository} plus tenant job stats.
- /deduplicate/check: tenant-scoped Qdrant similarity scan; chunk pairs
  above ~0.9 cosine from different pages grouped per page pair with best
  score and page references (read-only).

Supporting changes: get_job_manager dependency (+ shutdown close),
scroll_all_points can return vectors, VectorService.find_duplicate_pairs,
src/core/hashing.compute_content_hash. 13 new offline unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 12:06:21 +02:00
jpmschweitzerandClaude Fable 5 33e62db6a2 fix: ensure the tenant-scoped collection in upsert_document_chunks
ensure_collection() was called with the raw user string while the
upsert targeted get_collection_name(user), creating stray bare-name
collections (e.g. 'test_user') and failing the actual upsert when the
scoped collection did not exist yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:17:36 +02:00
jpmschweitzerandClaude Opus 4.5 37f8e1819e feat: refactor volatile cache to vector storage with HybridRAG integration
Build and Push / build (release) Successful in 28s
- Migrate volatile backend from Redis to Qdrant for semantic search
- Add natural language conversion for structured data embedding
- Simplify API: /volatile/search, /volatile/store, /{namespace}/{key}
- Integrate volatile into HybridRAG with priority boost in RRF fusion
- Add POST /maintenance/cleanup/volatile for expiry purging
- Update tests for new Qdrant-based architecture (37/37 pass)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 20:03:37 +01:00
jpmschweitzerandClaude Opus 4.5 0e6c3619eb feat: add scroll and batch delete methods to Qdrant client
Add bulk operations needed for maintenance:
- scroll_all_points(): paginated iteration over all points
- delete_by_ids(): batch delete points by ID list

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 16:21:57 +01:00
jpmschweitzer 95852190ba Initial commit: library-desk service extraction from portainer-core
Build and Push / build (release) Successful in 36s
2025-12-11 17:28:23 +01:00