feat: implement check-updates, job-backed ingest status, and dedup scan

Replace the four stub endpoints with real implementations, all requiring
an explicit tenant user (Phase B rule):

- /ingest/check-updates: GraphService now records a SHA-256 content_hash
  on every Document node at ingestion time; the endpoint compares those
  stored hashes against current Wiki.js page content in one UNWIND Cypher
  query per tenant and returns changed/new/deleted page lists (entity-stub
  pages excluded, pre-hash-tracking documents flagged stored_hash_missing).
- /ingest/status/{job_id}: backed by the Redis JobManager; jobs are
  tenant-scoped (foreign jobs 404). /ingest/page, /ingest/batch and
  /ingest/all now create job records and return job_id.
- /ingest/repo-status/{repository}: wiki page count vs indexed Document
  nodes under users/{tenant}/{repository} plus tenant job stats.
- /deduplicate/check: tenant-scoped Qdrant similarity scan; chunk pairs
  above ~0.9 cosine from different pages grouped per page pair with best
  score and page references (read-only).

Supporting changes: get_job_manager dependency (+ shutdown close),
scroll_all_points can return vectors, VectorService.find_duplicate_pairs,
src/core/hashing.compute_content_hash. 13 new offline unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
This commit is contained in:
2026-07-14 12:06:21 +02:00
co-authored by Claude Fable 5
parent eae39aff3e
commit 86051d8022
10 changed files with 857 additions and 59 deletions
+8
View File
@@ -46,6 +46,10 @@ class IngestionResult(BaseModel):
graph_entities_extracted: int = 0
graph_relationships_created: int = 0
processing_time_ms: float
job_id: Optional[str] = Field(
default=None,
description="Redis job-tracking ID (query via GET /ingest/status/{job_id})"
)
class BatchIngestionResult(BaseModel):
@@ -55,6 +59,10 @@ class BatchIngestionResult(BaseModel):
failed: int
results: List[IngestionResult]
total_processing_time_ms: float
job_id: Optional[str] = Field(
default=None,
description="Redis job-tracking ID (query via GET /ingest/status/{job_id})"
)
class IngestionStatus(BaseModel):