jpmschweitzer/library-desk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

feat: add Paperless document recall to HybridRAG

Browse Source 
Phase C of memory system: Documents are now a retrieval source alongside
wiki, volatile, and web search.

Changes:
- Add enable_documents, document_limit, document_threshold to HybridRAGConfig
- Add paperless_id field to HybridRAGResult
- Add document_ms timing to TimingBreakdown
- Add document search to parallel retrieval (filters doc_type=document)
- Update RRF fusion to include documents as fourth source
- Add document metadata (correspondent, document_type, tags) to results

HybridRAG now searches 4 sources in parallel:
- Wiki (vector + graph merged)
- Volatile cache (priority boost)
- Paperless documents (new)
- Web search

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-29 15:11:11 +01:00
co-authored by Claude Opus 4.5
parent 1f47b052d8
commit c01033505b
2 changed files with 105 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/models/hybrid_rag.py
+6 -1
View File
@@ -15,15 +15,18 @@ class HybridRAGConfig(BaseModel):
graph_limit: int = Field(default=10, ge=1, le=50, description="Max graph results")
web_limit: int = Field(default=5, ge=1, le=20, description="Max web results")
volatile_limit: int = Field(default=1, ge=1, le=5, description="Max volatile results (typically 1)")
document_limit: int = Field(default=5, ge=1, le=20, description="Max Paperless document results")
enable_vector: bool = Field(default=True, description="Enable vector search")
enable_graph: bool = Field(default=True, description="Enable graph search")
enable_web: bool = Field(default=True, description="Enable web search")
enable_volatile: bool = Field(default=True, description="Enable volatile cache search")
enable_documents: bool = Field(default=True, description="Enable Paperless document search")
enable_reranking: bool = Field(default=True, description="Enable LLM re-ranking")
enable_enrichment: bool = Field(default=True, description="Enable graph enrichment")
final_result_count: int = Field(default=10, ge=1, le=50, description="Final results to return")
rrf_k: int = Field(default=60, ge=1, le=100, description="RRF constant")
volatile_threshold: float = Field(default=0.8, ge=0.5, le=1.0, description="Volatile similarity threshold")
document_threshold: float = Field(default=0.6, ge=0.3, le=1.0, description="Document similarity threshold")
class RelatedDossier(BaseModel):
@@ -37,12 +40,13 @@ class RelatedDossier(BaseModel):
class HybridRAGResult(BaseModel):
"""Single result from HybridRAG query."""
source_type: str = Field(..., description="Source: 'wiki', 'web', 'volatile'")
source_type: str = Field(..., description="Source: 'wiki', 'web', 'volatile', 'document'")
title: str
content: str
url: Optional[str] = Field(None, description="URL for web results")
page_id: Optional[int] = Field(None, description="Page ID for wiki results")
page_path: Optional[str] = Field(None, description="Wiki page path")
paperless_id: Optional[int] = Field(None, description="Paperless document ID")
rrf_score: float = Field(..., description="Reciprocal Rank Fusion score")
final_rank: int = Field(..., description="Final rank after re-ranking")
sources: List[str] = Field(..., description="Which sources included this result")
@@ -57,6 +61,7 @@ class TimingBreakdown(BaseModel):
graph_ms: float = Field(..., description="Phase 1: Graph search")
web_ms: float = Field(..., description="Phase 1: Web search")
volatile_ms: float = Field(default=0, description="Phase 1: Volatile cache search")
document_ms: float = Field(default=0, description="Phase 1: Paperless document search")
fusion_ms: float = Field(..., description="Phase 2: RRF fusion")
enrichment_ms: float = Field(..., description="Phase 3: Graph enrichment")
reranking_ms: float = Field(..., description="Phase 4: LLM re-ranking")