- Fix WikiChangeListener: use get_page() instead of non-existent get_page_by_id()
- Fix webhooks: use delete_page_chunks() instead of non-existent delete_page_vectors()
- Fix Neo4j: replace deprecated id() with elementId() in entity query
- Change some warnings to debug level to reduce log noise
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add fuzzy_match_entity_to_document with confidence scoring
- Filter self-referential links (entity linking to current page)
- Fix path cleaning to preserve full wiki paths
- Add longest-first matching to prevent partial matches
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace WIKIJS_API_KEY with WIKIJS_USERNAME/PASSWORD and add
WIKIJS_DB_PASSWORD for PostgreSQL LISTEN/NOTIFY connection.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add WikiChangeListener service for PostgreSQL notifications
- Add webhooks router for HTTP webhook fallback
- Add asyncpg dependency for PostgreSQL async support
- Include setup scripts and documentation for triggers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
search_pages with empty query uses stale search index. list_all_pages
fetches actual page list from GraphQL for reliable bulk re-indexing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set d.tags = \$tags in MERGE query to prevent Neo4j warnings about
missing tags property in related documents query.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Pass ingestion_service to ConsolidationService constructor so pages
created/updated during consolidation are properly indexed in Qdrant
and Neo4j.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fetch existing wiki taxonomy structure before LLM analysis
- Include existing paths in prompt to prefer existing categories
- Add _format_taxonomy_for_prompt helper
- Mark searches as processed even when skipped/errored (prevents buildup)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- list_all_pages: fetch all pages with path prefix filter
- get_taxonomy_structure: extract category/subcategory structure for
taxonomy-aware classification
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use pages.singleByPath(path, locale) GraphQL query to get page ID
during initialization instead of searching through page list.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix create_entity_mentions to correctly count only newly created relationships,
not all relationships processed by MERGE.
**Problem**: count(r) was returning ALL relationships touched by MERGE (both
created and matched), breaking idempotency tests.
**Solution**: Use temporary flag 'just_created' set only ON CREATE, filter to
those relationships, count them, then remove the flag. This ensures the count
only includes new relationships.
Now properly returns:
- N on first run (N new relationships created)
- 0 on subsequent runs (no new relationships, all already exist)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two critical bug fixes for entity linking:
1. **Path cleaning**: Replace hardcoded "users/jpmschweitzer/" with regex pattern
to handle any user namespace. Now properly cleans paths for all users.
2. **Longest-first matching**: Move protected_ranges computation inside entity loop
to recompute after each entity is processed. Prevents nested links like
[[Machine](/machine) Learning](/ml) when processing multi-word entities.
These fixes ensure entity linking works correctly across all users and prevents
nested markdown links when entity names overlap.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cleanup Scripts:
- cleanup_graph.py - Clean up duplicate entities and orphaned nodes
- cleanup_wiki.py - Remove orphaned pages and fix broken links
- Utility scripts for database maintenance
- Not part of main application, run manually
Ingestion Router:
- POST /ingest/page - Index single page
- POST /ingest/batch - Batch indexing
- POST /ingest/all - Full knowledge base refresh
- Support vector and graph ingestion
Vector Router:
- POST /vector/search - Semantic search via Qdrant
- GET /vector/stats - Collection statistics
- DELETE /vector/page - Remove page embeddings
Wiki Router:
- GET /wiki/pages - List wiki pages
- GET /wiki/pages/{id} - Get page details
- PUT /wiki/pages/{id} - Update page
- POST /wiki/search - Search wiki content
- Full Wiki.js GraphQL integration
Tools Router:
- Expose library-desk capabilities as AI tool endpoints
- Support function calling for LLM agents
- Query wiki pages, search knowledge base
- Access graph entities and relationships
Tools Models:
- ToolDefinition for function schemas
- ToolParameter specifications
- ToolResponse format
- OpenAI function calling compatible
Graph Router:
- GET /graph/entities - List all entities for user
- GET /graph/relationships - Query entity relationships
- GET /graph/search - Search entities by name/type
- GET /graph/stats - Knowledge graph statistics
Graph Models:
- Entity, Relationship models
- GraphStats for analytics
- SearchFilters for queries
- Support multi-tenancy with user isolation
Configuration:
- Add HybridRAG settings (reranker model, search limits)
- Change Wiki.js auth from API key to username/password
- Configure vector, graph, and web search limits
Multi-tenancy:
- Add get_neo4j_user_base_label() for entity node labeling
- Support title-cased labels following Neo4j conventions
- Maintain namespace isolation for entities vs documents
Ollama Client:
- Improve model checking to handle :latest tag variants
- Match models with or without explicit tag
Qdrant Client:
- Add collection_exists() method for checking collection presence
- Refactor ensure_collection() to accept collection name directly
- Better separation of concerns
SearXNG Client:
- Add health_check() method for service monitoring
- Simple endpoint check without full search
- Used by health check endpoint
Vector Service:
- Manage document embeddings in Qdrant
- Update vectors from wiki pages
- Handle chunking and embedding generation
- Support force refresh and incremental updates
Vector Models:
- VectorSearchResult for search responses
- VectorUpdateSummary for indexing metrics
- Track chunks created/deleted
Used by ingestion_service for page embedding
- Add IngestionResult model for single page ingestion
- Add BatchIngestionResult for batch operations
- Track vector chunks, graph entities, and relationships
- Include processing time metrics
Used by ingestion_service for page indexing
- Add WikiPage model for page data
- Add WikiPageUpdate model for partial updates
- Add field validators for tags and descriptions
- Support optional fields for flexible updates
Used by wiki_service and entity_linking router
Main App:
- Mount /static directory for serving Wiki.js integration scripts
- Register entity_linking router
- Refactor API key verification to dependencies module
Dependencies:
- Add service factory functions for all services
- Add get_wiki_service() for wiki operations
- Add get_graph_service() for entity operations
- Add get_ingestion_service() for auto entity linking
- Improve health check for SearXNG
- Add combined integration script with both re-index and entity linking
- Add standalone entity linking button
- Add standalone re-index button
- Auto-detect Library Desk URL from script tag
- Support both toolbar and floating button positions
- Show real-time status updates and notifications
- Auto-reload page after successful entity linking
Usage: Inject via Wiki.js Code Injection settings
<script src="http://IP:8089/static/wikijs-integration.js"></script>
Graph Service:
- Add get_all_entities() to retrieve entities with wiki page paths
- Add create_entity_mentions() for MENTIONS relationship creation
- Support entity-to-document linking via title matching
Ingestion Service:
- Add _link_existing_entities() for automatic entity linking
- Auto-link entities during page ingestion
- Support skip_entity_linking parameter for granular control
- Add is_published parameter to WikiJS client update_page() method
- Update wiki_service to always pass is_published=True
- Prevents pages from being unpublished during entity linking updates
- Important for internal wikis where all pages should remain published
- Add /entity-linking/link-page endpoint to find and link entity mentions
- Creates both MENTIONS relationships in Neo4j and hyperlinks in wiki content
- Supports automatic re-indexing after linking
- Returns detailed statistics on entities found and linked
- Protects existing markdown links from being nested
- Idempotent: safe to run multiple times
Implements dual entity linking:
1. Graph relationships (MENTIONS) for knowledge graph traversal
2. Wiki content hyperlinks for user navigation
Add rest_api_executor as a universal executor that can call any REST API
endpoint across the system. This provides a standard way to trigger HTTP
operations from scheduled tasks.
Features:
- All HTTP methods: GET, POST, PUT, DELETE, PATCH
- Authentication: Bearer token, Basic auth, API key
- Environment variable substitution: ${VAR_NAME}
- JSONPath response extraction
- Configurable timeouts and SSL verification
- Sensitive data redaction in logs
- Custom headers support
This executor enables scheduler to call any service endpoint (Library Desk,
Core API, external webhooks) without needing service-specific executors.
Example usage:
{
"executor": "rest_api_executor",
"config": {
"url": "http://library-desk:8089/consolidate/knowledge",
"method": "POST",
"payload": {"process_limit": 10},
"auth": {"type": "bearer", "token": "${API_KEY}"}
}
}
The Qdrant client API changed from search() to query_points().
Updated both search() and find_similar_chunks() methods.
All integration tests now passing: 14/14 ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements comprehensive service client layer for Library Desk API to support
Librarian AI agent with multi-tenant knowledge management across Neo4j, Qdrant,
Wiki.js, SearXNG, and Ollama.
## Service Clients (src/clients/)
- Neo4j async client with connection pooling and user-scoped labels
- Qdrant vector store with collection-per-user multi-tenancy
- Wiki.js GraphQL API client for page/dossier management
- SearXNG client for web search integration
- Ollama client for text embeddings (nomic-embed-text)
## Core Infrastructure (src/core/)
- Multi-tenancy helpers for user namespace management
- Wiki.js: path-based namespaces (/users/{user})
- Neo4j: user-specific labels (User_{User}_Document)
- Qdrant: collection per user (library_desk_{user})
- Dependency injection with FastAPI Depends and @lru_cache singletons
- Lifecycle management (startup/shutdown) for all service connections
## Background Jobs (src/jobs/)
- Redis-based job manager for long-running operations
- Job status tracking with 24-hour TTL
- Support for queued, processing, completed, failed states
## Configuration
- Updated config.py with Redis DB 4 for library-desk jobs
- Updated docker-compose.yml: REDIS_DB from 2 to 4
- Added pytest and pytest-asyncio to requirements.txt
## Testing
- Unit tests: 25/25 passed (multi-tenancy helpers)
- Integration tests: 12/12 passed (all services verified)
- Neo4j connection and CRUD operations
- Qdrant vector operations with 768-dim embeddings
- Wiki.js GraphQL queries
- SearXNG web search
- Job Manager with Redis
- Dependency injection lifecycle
- pytest.ini configuration with asyncio support
## Health Monitoring
- Real-time service health checks via /health endpoint
- Connection status for all 5 external services
- Graceful degradation for partial service availability
## Architecture
- Follows async/await pattern throughout
- Connection pooling for Neo4j (singleton driver)
- HTTP client lifecycle management (httpx)
- Multi-tenancy enforced at client layer
- Default user: jpmschweitzer
Files changed: 26 files
- 5 new service clients (~1500 lines)
- 2 core modules (~500 lines)
- 1 job manager (~350 lines)
- 3 test files with 37 test cases
- Updated main.py with lifecycle hooks
All services tested and operational. Ready for Phase 2 (routers/services).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytest coverage output files to project-level .gitignore:
- coverage.json
- htmlcov/
- .pytest_cache/
These are generated artifacts from pytest-cov and should not be committed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add complete documentation for The Scheduler service.
README.md (500+ lines):
- Architecture overview with ASCII diagram
- Quick start guide
- Complete API reference with curl examples
- Task scheduling patterns and examples
- Priority system documentation
- Built-in executors documentation (example, doc_sync, config_backup)
- Custom executor development guide
- Current tasks table
- Database schema documentation
- Testing guide with coverage metrics
- Development and debugging information
- Monitoring and troubleshooting
- Security and performance notes
- API reference with response codes and filtering
CHANGELOG.md:
- Initial v1.0.0 release documentation
- Core features and architecture
- REST API endpoints
- Task executors and pre-configured tasks
- Testing infrastructure and metrics
- Technical details and dependencies
- Coverage metrics breakdown
- Planned features for future releases
Documentation covers:
- All API endpoints and authentication
- Scheduling examples (every minute, daily, monthly, etc.)
- Priority ranges and usage
- Executor configuration
- Test database setup
- Docker stack configuration
- Common issues and solutions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add hybrid APScheduler + PostgreSQL-based task scheduling system with minute-based execution and priority queue.
Core features:
- Minute-based scheduling with cron-like patterns (-1 = wildcard)
- Priority queue system (1-100, lower = higher priority)
- Concurrent execution (max 5 tasks simultaneously)
- Full REST API for task management (CRUD operations)
- Task execution tracking with audit trail
- API key authentication (Bearer token)
- Health checks and system statistics
Architecture:
- APScheduler runs every minute
- Queries PostgreSQL for tasks scheduled for current minute
- Executes tasks concurrently by priority
- Records execution history in database
Database schema:
- scheduled_tasks: Task definitions/templates
- task_executions: Individual execution records
Technical stack:
- FastAPI for REST API
- APScheduler for scheduling
- PostgreSQL for persistence
- Pydantic for configuration
Endpoints:
- POST/GET/PUT/DELETE /tasks - Task management
- POST /tasks/{name}/trigger - Manual execution
- GET /executions - Execution history
- GET /health - Health check
- GET /stats - System statistics
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fixed: Change `result.data` to `result.output` (correct PydanticAI API)
- Increased analysis_timeout from 3s to 10s (mistral-nemo needs more time)
**Status:** Steward now initializes correctly but there's a remaining issue
with the async generator merging logic in two_stage_agent.py causing
requests to hang. This needs further investigation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
jpmschweitzerandClaude Opus 4.5