Commit Graph
10 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 4.5 910b289c9e feat: integrate external scheduler for prefetch task registration
Add SchedulerClient to communicate with external scheduler service for
registering volatile prefetch tasks discovered during HybridRAG searches.

- Add scheduler_client.py with full REST API for task CRUD operations
- Add scheduler_url config setting (default: http://scheduler:8090)
- Update consolidation service to use scheduler for prefetch registration
- Add scheduler health checks to startup/shutdown lifecycle

When HybridRAG classifies web content as prefetch-worthy, it now creates
scheduled tasks that periodically refresh the volatile cache via the
external scheduler service.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:42:37 +01:00
jpmschweitzerandClaude Opus 4.5 5d4a8dba95 feat: add external API providers and central settings database
- Add central settings database client (system_settings on postgres-shared)
  - User-scoped settings with global fallback
  - API config storage with enabled/disabled toggle
  - Per-source category filtering for news

- Add modular external API providers in src/apis/:
  - OpenMeteoProvider: weather with geocoding (free, no key)
  - NOSProvider: Dutch news RSS feeds
  - BBCProvider: English news RSS feeds
  - AggregatedNewsProvider: merges sources with category filtering
  - AlphaVantageProvider: financial quotes (API key from settings DB)

- Add provider dependencies and lifecycle management
- Add requirements-dev.txt with pip-audit for security auditing
- Add MEMORY_REMEMBER_PLAN.md documenting volatile/document memory architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-26 12:07:58 +01:00
jpmschweitzerandClaude Opus 4.5 4ff3fc4c7a feat: add Paperless-ngx document storage integration
- Add /documents router with webhook, upload, search, health endpoints
- Create DocumentSyncService for indexing documents to vectors/graph
- Add PaperlessClient for REST API integration
- Configure dependency injection for Paperless client
- Add document models for webhook payloads and responses
- Event-driven architecture via Paperless workflow webhooks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 14:16:40 +01:00
jpmschweitzerandClaude Opus 4.5 2552bfd1f9 fix: make Wiki.js API token optional for open GraphQL endpoints
Build and Push / build (release) Successful in 33s
The Wiki.js GraphQL API is accessible without authentication.
Make WIKI_GRAPHQL_API env var optional with empty default to fix
container startup failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 16:59:41 +01:00
jpmschweitzerandClaude Opus 4.5 f756ca9490 feat: add maintenance system with index reconciliation
Complete maintenance subsystem for index health and cleanup:

Endpoints:
- GET /maintenance/health - lightweight (or detailed) health check
- POST /maintenance/cleanup/all - full orphan cleanup
- POST /maintenance/cleanup/vectors - purge orphan vector chunks
- POST /maintenance/cleanup/graph - purge orphan graph nodes
- POST /maintenance/reconcile-index - cleanup + reindex missing pages

Bidirectional orphan detection:
- find_documents_without_vectors() in GraphService
- find_chunks_without_graph_nodes() in VectorService

Redis integration:
- Tracks last_cleanup timestamp for scheduler visibility

Config additions:
- Document store, volatile cache, and maintenance settings
- VectorServiceDep and GraphServiceDep type aliases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 16:22:39 +01:00
jpmschweitzerandClaude Opus 4.5 a1832e3245 refactor: consolidate Ollama model configuration
Build and Push / build (release) Successful in 27s
- Add OLLAMA_EMBEDDING_MODEL for embeddings (nomic-embed-text)
- OLLAMA_MODEL now used for all LLM operations (mistral-nemo-large:latest)
- Remove separate reranker_model setting
- Update WikiPageWriter to use settings instead of hardcoded model
- Improves VRAM efficiency by keeping one model hot

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 11:15:27 +01:00
jpmschweitzerandClaude Opus 4.5 c359fcbcd8 feat: two-stage RRF for fair wiki vs web ranking
Build and Push / build (release) Successful in 28s
- Merge vector+graph into single wiki source before RRF with web
- Wiki pages no longer get 2x advantage from dual retrieval
- Add vector similarity threshold (0.7 default)
- Skip synonyms in graph search to reduce noise
- Fix duplicate entity links bug in graph search

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 17:49:31 +01:00
jpmschweitzerandClaude Opus 4.5 61863ff597 feat: add RAG search endpoint with content extraction
Build and Push / build (release) Successful in 1m2s
- Add /rag/search endpoint for web, news, and image search via SearXNG
- Add /content/extract and /content/extract/batch endpoints
- Add ContentExtractor client using Trafilatura for content extraction
- Enhance HybridRAG web search with full content extraction
- Add Redis caching for search results
- Add new configuration options for search and extraction timeouts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 15:50:48 +01:00
jpmschweitzerandClaude Opus 4.5 3deed7cbcb chore: add version management and changelog
- Add pyproject.toml with project metadata and version (1.1.0)
- Update config.py to read version from pyproject.toml
- Update main.py to use centralized version in FastAPI app
- Add CHANGELOG.md documenting v1.0.0 and v1.1.0 changes

Version is now the single source of truth in pyproject.toml and is
displayed in the health check endpoint and API docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 20:23:56 +01:00
jpmschweitzer 95852190ba Initial commit: library-desk service extraction from portainer-core
Build and Push / build (release) Successful in 36s
2025-12-11 17:28:23 +01:00