library-desk/CHANGELOG.md

Changelog

All notable changes to Library Desk will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.3.1] - 2025-12-16

Fixed

• Smart create endpoint missing content_extractor dependency causing 500 errors on POST /wiki/pages/smart-create

[1.3.0] - 2025-12-15

Changed

• Two-Stage RRF Architecture - Major refactor to level the playing field between wiki and web results

  • Stage 1: Vector and graph results merged into single "wiki" ranking using mini-RRF
  • Stage 2: Final RRF between wiki (single source) and web (single source)
  • Wiki pages no longer get 2x advantage from appearing in both vector and graph searches
  • Multi-source confirmation still determines wiki internal ranking

• Skip synonyms in graph search - LLM-generated synonyms (e.g., "author") no longer match unrelated graph entities (e.g., "author2000")

  • Vector search still uses synonyms for semantic similarity
  • Graph search uses only core keywords for exact entity matching

Added

• VECTOR_SIMILARITY_THRESHOLD config setting (default: 0.7) to filter weak vector matches
• Deduplication in graph search to prevent same document appearing multiple times

Fixed

• Graph search duplicate entity bug where same document could appear twice if entity linked multiple times

[1.2.1] - 2025-12-15

Fixed

• HybridRAG router missing content_extractor dependency causing 500 errors on /query/hybrid endpoint

[1.2.0] - 2025-12-15

Added

• RAG Search Endpoint (POST /rag/search)

  • Web, news, and image search via SearXNG
  • Full content extraction using Trafilatura (F1 score 0.958)
  • Redis caching with configurable TTL
  • Markdown sources summary for LLM consumption
  • Returns both extracted content and original snippets

• Content Extraction Endpoints (/content/*)

  • POST /content/extract - Extract content from a single URL
  • POST /content/extract/batch - Batch extraction (up to 20 URLs)
  • Reusable ContentExtractor client for use across the codebase

• HybridRAG Content Extraction Enhancement

  • Web search results now include full extracted content via Trafilatura
  • Falls back to original snippets if extraction fails
  • Improves context quality for LLM re-ranking and consumption

Changed

• Added new configuration options:
  • SEARCH_CACHE_TTL - Search cache TTL in seconds (default: 300)
  • SEARCH_TIMEOUT - SearXNG timeout (default: 10s)
  • CONTENT_EXTRACTION_TIMEOUT - Per-URL extraction timeout (default: 5s)
  • CONTENT_MAX_LENGTH - Max extracted content length (default: 2000)
  • SEARCH_DEFAULT_LIMIT - Default search results (default: 10)

Dependencies

• Added trafilatura~=1.12.0 for content extraction

[1.1.3] - 2025-12-14

Added

• Watchtower update trigger in Gitea workflow after successful build

[1.1.2] - 2025-12-14

Fixed

• Updated registry login URL in Gitea workflow (git.schweitz.net → git.schweitz.internal)

[1.1.1] - 2025-12-14

Fixed

• Updated container registry tag URLs in Gitea workflow (git.schweitz.net → git.schweitz.internal)

Added

• Tests for Smart Page Creation feature (test_smart_create.py)
  • Model validation tests for WikiSmartCreateRequest/Response
  • WikiService.smart_create_page method tests
  • Bidirectional entity linking utility tests
  • Endpoint validation tests

[1.1.0] - 2025-12-11

Added

• Smart Page Creation Endpoint (POST /wiki/pages/smart-create)

  • Combines HybridRAG research with LLM content generation
  • Searches existing wiki, knowledge graph, and web for topic context
  • Uses WikiPageWriter to synthesize findings into structured wiki content
  • Auto-generates page path from topic if not provided
  • Returns research summary with source counts

• Bidirectional Entity Linking

  • New shared utility (entity_linking_utils.py) for reusable entity linking
  • Forward links: Links entities mentioned in new pages to existing entity pages
  • Backward links: Updates existing pages that mention the new entity
  • Runs automatically in background after smart page creation

• Version Management

  • Added pyproject.toml with project metadata and version
  • Version is now read from pyproject.toml (single source of truth)
  • Health check endpoint returns current version
  • FastAPI docs show current version

Changed

• Updated config.py to read version from pyproject.toml
• Updated main.py to use centralized version

[1.0.0] - 2025-12-10

Added

• Initial release extracted from portainer-core
• Wiki page management (/wiki/pages CRUD endpoints)
• HybridRAG search (/query/hybrid) with vector, graph, and web search
• Knowledge graph operations (/graph/*)
• Vector search operations (/vector/*)
• Knowledge consolidation from search results (/consolidate/knowledge)
• Entity linking and extraction
• Wiki.js change listener for auto-processing user edits
• Multi-tenant architecture with user namespace isolation