Commit Graph
114 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 4.5 7f9e73a849 feat(npm): add internal domain aliases for all external hosts
Add *.schweitz.internal domains as HTTP-only alternatives to *.schweitz.net
domains for programmatic access without SSL or Authentik authentication.

- Configure 11 internal domain proxy hosts in NPM
- Document internal domain setup process in setup-new-host.md
- Add internal domains reference table to CONTAINERS.md
- Update external domains list with library and tatlock mappings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 11:51:13 +01:00
jpmschweitzerandClaude Opus 4.5 765818b4e9 feat(stack): add Paperless-ngx document management system
- Add paperless.yml stack (port 8091, documents.schweitz.net)
- Uses shared postgres (DB: paperless) and redis (DB 8)
- ClamAV installed on host for virus scanning (port 3310)
- Add Paperless integration to library-desk stack
- Update CONTAINERS.md with Paperless and ClamAV profiles
- Add Portainer and NPM API documentation to setup-new-host.md
- Update redis-shared.yml and postgres-shared.yml with Paperless refs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 00:36:15 +01:00
jpmschweitzer 7e06c915ad remove old webscraper code and add new home assistant settings to env for core-api.yaml 2025-12-24 19:38:51 +01:00
jpmschweitzerandClaude Opus 4.5 6a9000569a feat(stack): add Home Assistant smart home platform
- Add home-assistant.yml stack (port 8123, privileged mode for USB)
- External access via https://housekeeping.schweitz.net
- Health check using /manifest.json endpoint
- Update CONTAINERS.md with service profile
- Update stacks/README.md with port allocation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 22:32:13 +01:00
jpmschweitzer b3a30a5694 add qdrant env vars to tatlock 2025-12-14 15:49:57 +01:00
jpmschweitzerandClaude Opus 4.5 64ffeda928 feat: add Redis DB tracking and resolve DB conflicts
- Add Redis DB column to All Services table in CONTAINERS.md
- Move Nextcloud from DB 3 to DB 7 to resolve conflict with Scheduler
- Split Tatlock Redis usage: DB 1 (memory), DB 6 (benchmarks)
- Add missing services to table: Wiki.js, Tatlock, Library Desk

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 14:46:21 +01:00
jpmschweitzerandClaude Opus 4.5 5d0c0fbad9 chore: remove hardcoded APP_VERSION from stack files
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 13:36:03 +01:00
jpmschweitzerandClaude Opus 4.5 bde4386b90 feat: enable Watchtower HTTP API and use internal registry
- Add HTTP API to Watchtower for CI/CD triggered updates
- Update custom container images to use git.schweitz.internal
- Add GITEA_TOKEN env var to scheduler for release cleanup task

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 13:26:51 +01:00
jpmschweitzerandClaude Opus 4.5 9b2b32323c docs: add setup guide for SSL + Authentik protected hosts
Step-by-step instructions for adding new subdomains with:
- Let's Encrypt SSL via NPM
- Authentik forward authentication
- Troubleshooting common issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 12:06:05 +01:00
jpmschweitzerandClaude Opus 4.5 75c3950dd8 feat: add Docker healthchecks, remove Uptime Kuma
- Add healthcheck configurations to 13 stacks for Portainer status monitoring
- Remove Uptime Kuma service (replaced by Docker healthchecks)
- Clean up stale Heimdall references
- Update documentation and service counts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 12:05:05 +01:00
jpmschweitzer 8793ff1779 update with library desk api ENV vars 2025-12-11 19:20:40 +01:00
jpmschweitzerandClaude Opus 4.5 7c62577377 fix: use JSON array format for CORS_ORIGINS env var
Pydantic settings expects list[str] fields as JSON arrays, not plain strings.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 18:46:42 +01:00
jpmschweitzerandClaude Opus 4.5 235b694356 feat: add tatlock stack for homelab butler API
- OpenAI-compatible API server with LLM agent orchestration
- Port 8000, connects to docker-dataplane network
- Integrates with ollama, searxng, redis-shared services
- Watchtower auto-update enabled

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 18:28:04 +01:00
jpmschweitzerandClaude Opus 4.5 6581a743ac refactor: extract library-desk to standalone repository
- Move library-desk service to git.schweitz.net/jpmschweitzer/library-desk
- Update stacks/library-desk.yml to use container image
- Add library-desk to External Repositories in CONTAINERS.md
- Remove source code (now in external repo)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 17:46:48 +01:00
jpmschweitzer 98c92a1211 cleanup of docs 2025-12-11 16:32:37 +01:00
jpmschweitzerandClaude Opus 4.5 35077ed5ca chore: prune obsolete documentation and consolidate
- Remove obsolete plans/ directory (AI orchestrator moved to separate services)
- Remove docs/sessions/ (historical session notes)
- Remove docs/reference/ (duplicated in CONTAINERS.md)
- Remove docs/architecture/ (duplicated in CONTAINERS.md)
- Remove maintenance stack and backup-procedures guide (decommissioned)
- Remove scripts/ directory (unused)
- Fold EXTERNAL_SERVICES.md into CONTAINERS.md
- Update Nextcloud to reflect PostgreSQL shared (was MariaDB)
- Clean up README.md links

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 16:29:06 +01:00
jpmschweitzerandClaude Opus 4.5 0815f02529 refactor: extract core-api to standalone repository
Core-API service moved to git.schweitz.net/jpmschweitzer/core-api
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Documentation updated with external service reference
- Removed obsolete docs/services/core-api.md (docs now in core-api repo)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 16:10:17 +01:00
jpmschweitzerandClaude Opus 4.5 ca83bbffb5 refactor: extract scheduler to standalone repository
Scheduler service moved to git.schweitz.net/jpmschweitzer/scheduler
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Added Gitea Actions runner to gitea stack for CI/CD
- Documentation updated with external service reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 15:18:42 +01:00
jpmschweitzerandClaude Opus 4.5 9af231d72d fix(stacks): configure ollama for stability and persistent model loading
- OLLAMA_NUM_PARALLEL=1: Process requests sequentially to avoid batch
  overflow panics when multiple services call embeddings concurrently
- OLLAMA_KEEP_ALIVE=-1: Keep models loaded in VRAM indefinitely
- OLLAMA_MAX_LOADED_MODELS=2: Allow both mistral-nemo and nomic-embed-text
  to stay loaded simultaneously

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 11:01:03 +01:00
jpmschweitzerandClaude Opus 4.5 375913ec9d fix(library-desk): ensure page_id is int in get_page GraphQL call
GraphQL requires Int type, but page_id might be passed as string.
Explicit int() conversion prevents 400 Bad Request errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:31:55 +01:00
jpmschweitzerandClaude Opus 4.5 4cc2002fa6 fix(library-desk): reduce ollama health check to single API call
Previously called /api/tags twice (once for server check, once for
model check). Now reuses response from single call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:31:07 +01:00
jpmschweitzerandClaude Opus 4.5 c91f77bf28 chore(stacks): change wiki.js port to 3000 (internal = external)
Simplifies access by using the same port internally and externally.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:13:28 +01:00
jpmschweitzerandClaude Opus 4.5 a607d7a660 fix(library-desk): resolve warnings and errors in logs
- Fix WikiChangeListener: use get_page() instead of non-existent get_page_by_id()
- Fix webhooks: use delete_page_chunks() instead of non-existent delete_page_vectors()
- Fix Neo4j: replace deprecated id() with elementId() in entity query
- Change some warnings to debug level to reduce log noise

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:03:54 +01:00
jpmschweitzerandClaude Opus 4.5 40aafb0331 docs: update changelog for v0.12.0-library-desk-enhancements
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:44:20 +01:00
jpmschweitzerandClaude Opus 4.5 4b9092d631 feat(library-desk): improve entity linking with fuzzy matching
- Add fuzzy_match_entity_to_document with confidence scoring
- Filter self-referential links (entity linking to current page)
- Fix path cleaning to preserve full wiki paths
- Add longest-first matching to prevent partial matches

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:36:24 +01:00
jpmschweitzerandClaude Opus 4.5 651de9ac92 chore(stacks): update library-desk env vars for wiki change detection
Replace WIKIJS_API_KEY with WIKIJS_USERNAME/PASSWORD and add
WIKIJS_DB_PASSWORD for PostgreSQL LISTEN/NOTIFY connection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:36:02 +01:00
jpmschweitzerandClaude Opus 4.5 14d152bd64 test(library-desk): add test coverage for new features
- test_consolidation: taxonomy fetching, ingestion_service injection
- test_graph_service: Document node tags, entity-stub skipping
- test_ingestion: list_all_pages usage, batch ingestion
- test_integration: list_all_pages, get_taxonomy_structure
- test_wiki_change_listener: PostgreSQL notification handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:35:39 +01:00
jpmschweitzerandClaude Opus 4.5 b74b5bc2c8 feat(library-desk): add wiki change detection via PostgreSQL LISTEN/NOTIFY
- Add WikiChangeListener service for PostgreSQL notifications
- Add webhooks router for HTTP webhook fallback
- Add asyncpg dependency for PostgreSQL async support
- Include setup scripts and documentation for triggers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:35:18 +01:00
jpmschweitzerandClaude Opus 4.5 bce6b71f9b fix(library-desk): use list_all_pages instead of search_pages for bulk re-index
search_pages with empty query uses stale search index. list_all_pages
fetches actual page list from GraphQL for reliable bulk re-indexing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:34:40 +01:00
jpmschweitzerandClaude Opus 4.5 d100212da5 fix(library-desk): add tags property to Document nodes in graph_service
Set d.tags = \$tags in MERGE query to prevent Neo4j warnings about
missing tags property in related documents query.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:32:37 +01:00
jpmschweitzerandClaude Opus 4.5 aba981ff21 fix(library-desk): inject ingestion_service into ConsolidationService
Pass ingestion_service to ConsolidationService constructor so pages
created/updated during consolidation are properly indexed in Qdrant
and Neo4j.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:31:15 +01:00
jpmschweitzerandClaude Opus 4.5 6ddf2a1bd9 feat(library-desk): add taxonomy-aware classification to consolidation
- Fetch existing wiki taxonomy structure before LLM analysis
- Include existing paths in prompt to prefer existing categories
- Add _format_taxonomy_for_prompt helper
- Mark searches as processed even when skipped/errored (prevents buildup)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:58 +01:00
jpmschweitzerandClaude Opus 4.5 9baea7ee6c feat(library-desk): add list_all_pages and get_taxonomy_structure to wikijs client
- list_all_pages: fetch all pages with path prefix filter
- get_taxonomy_structure: extract category/subcategory structure for
  taxonomy-aware classification

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:47 +01:00
jpmschweitzerandClaude Opus 4.5 ed7792ef98 feat(library-desk): fetch page ID via GraphQL in wikijs-integration.js
Use pages.singleByPath(path, locale) GraphQL query to get page ID
during initialization instead of searching through page list.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:18 +01:00
jpmschweitzer d582f27098 more cleanup 2025-12-10 20:26:44 +01:00
jpmschweitzer 953bcc5388 cleanup 2025-12-10 20:26:06 +01:00
jpmschweitzer b80f43745a cleanup 2025-12-10 20:24:07 +01:00
jpmschweitzer c670b10630 obsolete 2025-12-10 20:23:57 +01:00
jpmschweitzer 32c4805a07 remove obsolete core-ai 2025-12-10 20:21:26 +01:00
jpmschweitzerandClaude Opus 4.5 65114cb477 fix(library-desk): fix idempotency in create_entity_mentions
Fix create_entity_mentions to correctly count only newly created relationships,
not all relationships processed by MERGE.

**Problem**: count(r) was returning ALL relationships touched by MERGE (both
created and matched), breaking idempotency tests.

**Solution**: Use temporary flag 'just_created' set only ON CREATE, filter to
those relationships, count them, then remove the flag. This ensures the count
only includes new relationships.

Now properly returns:
- N on first run (N new relationships created)
- 0 on subsequent runs (no new relationships, all already exist)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:10:31 +01:00
jpmschweitzerandClaude Opus 4.5 6ea6e4d0ad fix(library-desk): fix entity linking path cleaning and longest-first matching
Two critical bug fixes for entity linking:

1. **Path cleaning**: Replace hardcoded "users/jpmschweitzer/" with regex pattern
   to handle any user namespace. Now properly cleans paths for all users.

2. **Longest-first matching**: Move protected_ranges computation inside entity loop
   to recompute after each entity is processed. Prevents nested links like
   [[Machine](/machine) Learning](/ml) when processing multi-word entities.

These fixes ensure entity linking works correctly across all users and prevents
nested markdown links when entity names overlap.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:09:59 +01:00
jpmschweitzerandClaude Opus 4.5 60b0100861 test(library-desk): add comprehensive entity linking tests
Add test suite covering:
- Entity mention detection (case-insensitive, whole-word matching, sorting)
- Content link addition (protection of existing links, longest-first matching)
- Integration tests (get entities with paths, create relationships, idempotency)
- Multi-tenancy isolation tests

All 17 tests passing. Validates entity linking functionality end-to-end.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:09:38 +01:00
jpmschweitzer 1f0a894291 docs(library-desk): add architecture and API documentation
Documentation:
- Architecture overview and design decisions
- API endpoint reference
- Service interaction diagrams
- Configuration guide
- Multi-tenancy patterns
- Knowledge graph schema
2025-12-10 01:40:23 +01:00
jpmschweitzer ae8edf6111 chore(library-desk): add maintenance and cleanup scripts
Cleanup Scripts:
- cleanup_graph.py - Clean up duplicate entities and orphaned nodes
- cleanup_wiki.py - Remove orphaned pages and fix broken links
- Utility scripts for database maintenance
- Not part of main application, run manually
2025-12-10 01:40:10 +01:00
jpmschweitzer 9fe5faa3e7 feat(library-desk): add core API routers for wiki, vector, and ingestion
Ingestion Router:
- POST /ingest/page - Index single page
- POST /ingest/batch - Batch indexing
- POST /ingest/all - Full knowledge base refresh
- Support vector and graph ingestion

Vector Router:
- POST /vector/search - Semantic search via Qdrant
- GET /vector/stats - Collection statistics
- DELETE /vector/page - Remove page embeddings

Wiki Router:
- GET /wiki/pages - List wiki pages
- GET /wiki/pages/{id} - Get page details
- PUT /wiki/pages/{id} - Update page
- POST /wiki/search - Search wiki content
- Full Wiki.js GraphQL integration
2025-12-10 01:36:07 +01:00
jpmschweitzer 3ab44c2ab0 feat(library-desk): add AI agent tools router
Tools Router:
- Expose library-desk capabilities as AI tool endpoints
- Support function calling for LLM agents
- Query wiki pages, search knowledge base
- Access graph entities and relationships

Tools Models:
- ToolDefinition for function schemas
- ToolParameter specifications
- ToolResponse format
- OpenAI function calling compatible
2025-12-10 01:30:23 +01:00
jpmschweitzer dca8b63a50 feat(library-desk): add graph query router and models
Graph Router:
- GET /graph/entities - List all entities for user
- GET /graph/relationships - Query entity relationships
- GET /graph/search - Search entities by name/type
- GET /graph/stats - Knowledge graph statistics

Graph Models:
- Entity, Relationship models
- GraphStats for analytics
- SearchFilters for queries
- Support multi-tenancy with user isolation
2025-12-10 01:28:42 +01:00
jpmschweitzer 8f29bfe064 feat(library-desk): add knowledge consolidation system
Consolidation Service:
- Automated wiki page creation from research results
- Entity extraction and disambiguation
- Multi-source fact integration
- Confidence scoring and source citation
- Template-based page generation
- Schema.org taxonomy integration

Wiki Page Writer:
- Structured markdown generation
- Standard templates (person, organization, technology)
- Metadata formatting (tags, categories, timestamps)
- Citation and source linking

Router:
- POST /consolidate/research endpoint
- Batch consolidation support
- Manual and automated triggers

Models:
- ConsolidationRequest with source data
- ConsolidationResult with page details
- Entity resolution metadata

Tests:
- Page generation validation
- Entity extraction accuracy
- Multi-source merging logic
2025-12-10 01:28:29 +01:00
jpmschweitzer 15930a9600 feat(library-desk): implement HybridRAG query system
HybridRAG Service:
- Combine vector (Qdrant), graph (Neo4j), and web (SearXNG) search
- Reciprocal Rank Fusion (RRF) for result merging
- LLM re-ranking with mistral-nemo
- Graph enrichment with related dossiers
- Query enhancement with keyword/synonym extraction
- Search result persistence for offline processing

Router:
- POST /query/hybrid endpoint
- Configurable search limits per source
- Enable/disable individual sources
- Timing breakdown for performance monitoring

Models:
- HybridRAGRequest, HybridRAGResponse
- HybridRAGResult with source tracking
- KeywordExtraction for query analysis
- TimingBreakdown for performance metrics

Tests:
- End-to-end HybridRAG query tests
- RRF fusion algorithm validation
- Multi-source result merging
2025-12-10 01:28:13 +01:00
jpmschweitzer 8c0ced68eb feat(scheduler): add Pydantic models and improve task API
Scheduler API:
- Add Pydantic models for request/response validation
- Improve API documentation with examples
- Add detailed schedule pattern documentation
- Document priority levels and executor types

Models:
- TaskCreate, TaskUpdate, TaskResponse models
- Field validation and constraints
- Type safety for task operations

Documentation:
- Add TASK_REGISTRATION.md guide
- Document schedule patterns and executor configs
2025-12-10 01:26:21 +01:00