feat: add maintenance system with index reconciliation

Complete maintenance subsystem for index health and cleanup:

Endpoints:
- GET /maintenance/health - lightweight (or detailed) health check
- POST /maintenance/cleanup/all - full orphan cleanup
- POST /maintenance/cleanup/vectors - purge orphan vector chunks
- POST /maintenance/cleanup/graph - purge orphan graph nodes
- POST /maintenance/reconcile-index - cleanup + reindex missing pages

Bidirectional orphan detection:
- find_documents_without_vectors() in GraphService
- find_chunks_without_graph_nodes() in VectorService

Redis integration:
- Tracks last_cleanup timestamp for scheduler visibility

Config additions:
- Document store, volatile cache, and maintenance settings
- VectorServiceDep and GraphServiceDep type aliases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2025-12-24 16:22:39 +01:00
co-authored by Claude Opus 4.5
parent 0e6c3619eb
commit f756ca9490
7 changed files with 1851 additions and 6 deletions
+19 -2
View File
@@ -43,8 +43,10 @@ class Settings(BaseSettings):
# Wiki.js Configuration
wikijs_url: str = Field(default="http://wiki:3000", description="Wiki.js URL")
wikijs_username: str = Field(..., description="Wiki.js username")
wikijs_password: str = Field(..., description="Wiki.js password")
wiki_graphql_api: str = Field(..., description="Wiki.js GraphQL API token (JWT)")
# Legacy auth fields - kept for backwards compatibility but deprecated
wikijs_username: str = Field(default="", description="Wiki.js username (deprecated, use wiki_graphql_api)")
wikijs_password: str = Field(default="", description="Wiki.js password (deprecated, use wiki_graphql_api)")
# Wiki.js Database Configuration (for change listener)
wikijs_db_host: str = Field(default="postgres-shared", description="Wiki.js PostgreSQL host")
@@ -99,6 +101,21 @@ class Settings(BaseSettings):
content_extraction_timeout: int = Field(default=5, ge=1, le=30, description="Trafilatura per-URL timeout in seconds")
content_max_length: int = Field(default=2000, ge=500, le=10000, description="Max extracted content length per result")
# Document Store Configuration
document_store_enabled: bool = Field(default=True, description="Enable document store feature")
document_catalog_path_prefix: str = Field(default="docs", description="Wiki path prefix for catalog pages")
# Volatile Cache Configuration
volatile_cache_enabled: bool = Field(default=True, description="Enable volatile cache feature")
volatile_default_ttl: int = Field(default=3600, ge=60, le=86400, description="Default TTL in seconds")
volatile_weather_ttl: int = Field(default=1800, ge=60, le=7200, description="Weather data TTL in seconds")
volatile_news_ttl: int = Field(default=7200, ge=300, le=86400, description="News data TTL in seconds")
volatile_financial_ttl: int = Field(default=300, ge=60, le=3600, description="Financial data TTL in seconds")
# Maintenance Configuration
maintenance_orphan_cleanup_enabled: bool = Field(default=True, description="Enable automatic orphan cleanup")
maintenance_cleanup_batch_size: int = Field(default=100, ge=10, le=1000, description="Cleanup batch size")
@property
def qdrant_url(self) -> str:
"""Computed Qdrant URL."""