feat: add unified memory routing to consolidation service

- Add MemoryRouteClassification and MemoryRoutingResult models
- Implement unified classifier (_classify_web_results_unified) that routes
  web results to: wiki, volatile, file (Paperless), prefetch, or skip
- Add routing methods: _route_to_volatile, _route_to_files, _register_prefetch
- Update _process_search to use unified classifier instead of separate analysis
- Add get_volatile_cache_service factory to dependencies
- Wire volatile_service and settings_client into ConsolidationService
- Update ConsolidationResult/Response with new routing counters

Test fixes:
- Fix WikiJSClient fixtures to use api_token instead of username/password
- Fix entity linking test assertions to expect full user-namespaced paths
- Add sample_unified_classification fixture for new classifier format

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2025-12-28 17:19:20 +01:00
co-authored by Claude Opus 4.5
parent ab892745fa
commit 1f47b052d8
9 changed files with 533 additions and 104 deletions
+50
View File
@@ -34,6 +34,9 @@ class ConsolidationResult(BaseModel):
pages_created: int = 0
pages_updated: int = 0
entities_added: int = 0
volatile_cached: int = 0
files_queued: int = 0
prefetch_registered: int = 0
error: Optional[str] = None
@@ -44,6 +47,53 @@ class ConsolidationResponse(BaseModel):
pages_created: int = Field(description="New wiki pages created")
pages_updated: int = Field(description="Existing pages updated")
entities_added: int = Field(description="New entities added to graph")
volatile_cached: int = Field(default=0, description="Items cached to volatile storage")
files_queued: int = Field(default=0, description="Files queued for Paperless")
prefetch_registered: int = Field(default=0, description="Prefetch patterns registered")
errors: List[str] = Field(default=[], description="Error messages")
results: List[ConsolidationResult] = Field(description="Per-search results")
dry_run: bool = Field(description="Whether this was a dry run")
class MemoryRouteClassification(BaseModel):
"""
Unified classification of a web result for memory routing.
Route types:
- wiki: Stable reference content → wiki page creation/update
- volatile: Ephemeral data (weather, news, prices) → volatile cache
- file: Downloadable file (PDF, doc, xls, images) → Paperless ingestion
- prefetch: Regularly updated source → scheduler registration
- skip: Low value, ads, errors → discard
"""
url: str
title: str
route_type: str = Field(description="One of: wiki, volatile, file, prefetch, skip")
# Wiki routing fields
wiki_action: Optional[str] = Field(default=None, description="create or update")
wiki_path: Optional[str] = Field(default=None, description="Wiki path for page")
wiki_summary: Optional[str] = Field(default=None, description="Summary for wiki page")
# Volatile routing fields
volatile_namespace: Optional[str] = Field(default=None, description="weather, news, financial, etc.")
volatile_key: Optional[str] = Field(default=None, description="Cache key")
volatile_ttl_hours: Optional[int] = Field(default=None, description="TTL in hours")
# Prefetch routing fields
prefetch_cron: Optional[str] = Field(default=None, description="Cron expression for refresh")
prefetch_endpoint: Optional[str] = Field(default=None, description="API endpoint to call")
# Classification metadata
confidence: float = Field(default=0.0, ge=0.0, le=1.0)
reason: str = Field(default="")
class MemoryRoutingResult(BaseModel):
"""Aggregated result of memory routing for a search."""
wiki_routed: int = 0
volatile_cached: int = 0
files_queued: int = 0
prefetch_registered: int = 0
skipped: int = 0
classifications: List[MemoryRouteClassification] = []