Files
portainer-core/stacks/library-desk.yml
T
jpmschweitzerandClaude Opus 4.5 1a41e5bb80 feat(library-desk): implement Phase 1 service clients and infrastructure
Implements comprehensive service client layer for Library Desk API to support
Librarian AI agent with multi-tenant knowledge management across Neo4j, Qdrant,
Wiki.js, SearXNG, and Ollama.

## Service Clients (src/clients/)
- Neo4j async client with connection pooling and user-scoped labels
- Qdrant vector store with collection-per-user multi-tenancy
- Wiki.js GraphQL API client for page/dossier management
- SearXNG client for web search integration
- Ollama client for text embeddings (nomic-embed-text)

## Core Infrastructure (src/core/)
- Multi-tenancy helpers for user namespace management
  - Wiki.js: path-based namespaces (/users/{user})
  - Neo4j: user-specific labels (User_{User}_Document)
  - Qdrant: collection per user (library_desk_{user})
- Dependency injection with FastAPI Depends and @lru_cache singletons
- Lifecycle management (startup/shutdown) for all service connections

## Background Jobs (src/jobs/)
- Redis-based job manager for long-running operations
- Job status tracking with 24-hour TTL
- Support for queued, processing, completed, failed states

## Configuration
- Updated config.py with Redis DB 4 for library-desk jobs
- Updated docker-compose.yml: REDIS_DB from 2 to 4
- Added pytest and pytest-asyncio to requirements.txt

## Testing
- Unit tests: 25/25 passed (multi-tenancy helpers)
- Integration tests: 12/12 passed (all services verified)
  - Neo4j connection and CRUD operations
  - Qdrant vector operations with 768-dim embeddings
  - Wiki.js GraphQL queries
  - SearXNG web search
  - Job Manager with Redis
  - Dependency injection lifecycle
- pytest.ini configuration with asyncio support

## Health Monitoring
- Real-time service health checks via /health endpoint
- Connection status for all 5 external services
- Graceful degradation for partial service availability

## Architecture
- Follows async/await pattern throughout
- Connection pooling for Neo4j (singleton driver)
- HTTP client lifecycle management (httpx)
- Multi-tenancy enforced at client layer
- Default user: jpmschweitzer

Files changed: 26 files
- 5 new service clients (~1500 lines)
- 2 core modules (~500 lines)
- 1 job manager (~350 lines)
- 3 test files with 37 test cases
- Updated main.py with lifecycle hooks

All services tested and operational. Ready for Phase 2 (routers/services).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-08 19:09:54 +01:00

152 lines
4.6 KiB
YAML

version: '3.8'
# Library - Front Desk API (Coordination Service)
# Application Layer
# Port: 8089 (HTTP)
# GPU: No
# Storage: SSD (venv, logs)
services:
library-desk:
image: python:3.12-slim
container_name: library-desk
restart: unless-stopped
ports:
- "8089:8089" # FastAPI HTTP
volumes:
# Mount source code for live editing
- /home/jpmschweitzer/Projects/portainer-core/services/library-desk:/app
# Persist container's venv for fast restarts
- /home/jpmschweitzer/docker-data/library-desk/venv:/venv
working_dir: /app
command: >
sh -c "
echo 'Installing system dependencies...' &&
apt-get update -qq &&
apt-get install -y --no-install-recommends curl >/dev/null 2>&1 &&
rm -rf /var/lib/apt/lists/* &&
echo 'Setting up Python environment...' &&
if [ ! -f /venv/bin/python ]; then
echo 'Initializing venv...' &&
python3 -m venv --clear /venv;
fi &&
echo 'Upgrading pip...' &&
/venv/bin/python -m pip install --upgrade pip --quiet &&
echo 'Installing dependencies from requirements.txt...' &&
/venv/bin/python -m pip install -r /app/requirements.txt --quiet &&
echo 'Starting Library Desk API...' &&
/venv/bin/python -m uvicorn src.main:app --host 0.0.0.0 --port 8089 --workers 2
"
environment:
# API Configuration
- LIBRARY_API_KEY=${LIBRARY_API_KEY}
# Neo4j Configuration
- NEO4J_URI=bolt://neo4j:7687
- NEO4J_USER=neo4j
- NEO4J_PASSWORD=${NEO4J_PASSWORD}
# Qdrant Configuration
- QDRANT_HOST=qdrant
- QDRANT_PORT=6333
# Wiki.js Configuration
- WIKIJS_URL=http://wiki:3000
- WIKIJS_API_KEY=${WIKIJS_API_KEY}
# SearXNG Configuration
- SEARXNG_URL=http://searxng:8080
# Ollama Configuration (for embeddings)
- OLLAMA_URL=http://ollama:11434
- OLLAMA_MODEL=nomic-embed-text
# Redis Configuration
- REDIS_HOST=redis-shared
- REDIS_PORT=6379
- REDIS_DB=4
# Python Configuration
- PYTHONUNBUFFERED=1
- TZ=${TZ:-Europe/Amsterdam}
networks:
- docker-dataplane
deploy:
resources:
reservations:
memory: 256M
limits:
memory: 768M
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8089/health')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
networks:
docker-dataplane:
external: true
name: docker-dataplane
# ⚠️ SECURITY WARNING:
# Set these in environment variables before deploying:
# - LIBRARY_API_KEY (generate with: openssl rand -hex 32)
# - NEO4J_PASSWORD (from library-neo4j deployment)
# - LIBRARY_DB_PASSWORD (from library-wiki deployment)
# - WIKIJS_API_KEY (from Wiki.js admin panel → API Access)
#
# Prerequisites:
# 1. Create /home/jpmschweitzer/Projects/portainer-core/services/library/services/front-desk/
# 2. Create requirements.txt (see DEPLOYMENT.md Phase 4.3)
# 3. Create main.py with FastAPI app (see DEPLOYMENT.md Phase 4.3)
#
# API Endpoints (once implemented):
# Query:
# POST /query/hybrid # HybridRAG (graph + vector + web)
# POST /query/semantic # Vector search only
# POST /query/graph # Graph traversal only
# GET /query/related/{id} # Find related content
#
# Content Management:
# POST /ingest/document # Index new document
# POST /ingest/wiki-page # Sync Wiki.js page
# POST /wiki/dossier # Create dossier (proxies to Wiki.js)
# PUT /wiki/dossier/{id} # Update dossier
# DELETE /wiki/dossier/{id} # Delete dossier
#
# Graph Operations:
# GET /graph/entities # List entities
# GET /graph/mindmap/{id} # Generate mind map for dossier
# POST /graph/query # Execute Cypher query
#
# Deduplication:
# POST /deduplicate/find # Find potential duplicates
# POST /deduplicate/merge # Merge duplicate entities
#
# System:
# GET /stats # System statistics
# GET /health # Health check
#
# API Documentation:
# - Interactive docs: http://192.168.86.149:8089/docs
# - OpenAPI spec: http://192.168.86.149:8089/openapi.json
#
# Features:
# - HybridRAG queries (Neo4j + Qdrant + SearXNG)
# - Document ingestion and indexing
# - Entity extraction and relationship mapping
# - Mind map generation
# - Deduplication detection
# - Wiki.js API proxy
#
# Dependencies:
# - Neo4j (knowledge graph)
# - Qdrant (vector search)
# - Wiki.js (wiki operations)
# - SearXNG (web search)
# - Ollama (embeddings)
# - Redis (caching)