100 Commits
Author SHA1 Message Date
jpmschweitzerandClaude 2b544ae6aa chore(stacks): remove the deprecated stack copies
This repo was merged into system-admin-toj/containers/ and the copies
here kept drifting. Stale infrastructure config is not inert: agents.yml
here still carried OLLAMA_DEFAULT_MODEL=mistral-nemo:latest months after
the live stack moved to gemma4:e2b, and that setting is what caused the
2026-08-07 outage — mistral-nemo held 9,262 MiB of an 11,264 MiB card,
Whisper got 7 MiB, and Speaches returned CUDA OOM for hours while
reporting healthy. Anyone deploying from this directory would have
reproduced it exactly.

The files stay in this repo's history; README.md explains how to read one
back and points at the live location.

Two had no counterpart in system-admin-toj and were never deployed:
appwrite.yml, added in the most recent commit, and penpot.yml, whose
service is decommissioned. Both are recoverable from history if wanted;
neither should come back to this directory.

Left alone deliberately: an uncommitted CONTAINERS.md edit, which is
someone's unfinished work and would not have survived a delete.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-08 17:00:48 +02:00
jpmschweitzerandClaude Opus 4.6 57139f0f54 feat(stack): add Appwrite BaaS stack for dev/testing
Self-contained stack with 25 containers: API server, console, realtime
WebSocket server, 10 workers, 3 schedulers, maintenance task, browser,
OpenRuntimes executor, plus internal MariaDB 10.11 and Redis 7.4.
Exposed on port 8093 via bundled Traefik. Watchtower disabled for
deliberate upgrades.

Also includes media.yml reformatting and Jellyfin GPU deploy block.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 12:09:42 +02:00
jpmschweitzerandClaude Opus 4.6 4520f627ba ops(scheduler): add GCS credentials mount for offsite backups
Mount secrets directory and GCS_CREDENTIALS_FILE env var to support
the new gcs_backup_executor in scheduler v1.2.0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:28:54 +02:00
jpmschweitzerandClaude Opus 4.6 73b70629bf docs(containers): remove obsolete Shell In A Box, update counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:39:18 +01:00
jpmschweitzerandClaude Opus 4.6 b7cca51d7a docs(stacks): update README for models stack and Matter Server
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:39:00 +01:00
jpmschweitzerandClaude Opus 4.6 a4379fe373 feat(stack): consolidate Ollama, Stable Audio, and TRELLIS into models stack
Merge three individual GPU service stacks into a unified models.yml.
All services share the RTX 2080 Ti and docker-dataplane network.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:38:52 +01:00
jpmschweitzerandClaude Opus 4.6 cfb6cea452 docs: add project handover notes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:35:09 +01:00
jpmschweitzerandClaude Opus 4.6 d0f06918ea docs(containers): add Stable Audio and TRELLIS, update service counts
- Add full profiles for Stable Audio and TRELLIS services
- Update quick reference table, GPU services, and storage distribution
- Add new stacks to README port allocation and GPU section
- Update total services to 37 across 23 stacks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:49 +01:00
jpmschweitzerandClaude Opus 4.6 bc1574b36e feat(stack): add Stable Audio and TRELLIS GPU service stacks
- Stable Audio Open: AI audio generation on port 11500 (~6GB VRAM)
- TRELLIS: 3D model generation on port 11510 (~6-8GB VRAM, low-VRAM fork)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:34 +01:00
jpmschweitzerandClaude Opus 4.6 827dffe164 fix(penpot): update assets storage config for Penpot 2.11+
Rename PENPOT_ASSETS_STORAGE_* to PENPOT_OBJECTS_STORAGE_* per upstream changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:27 +01:00
jpmschweitzerandClaude Opus 4.6 71cd913d05 fix(core-api): use IP address for Home Assistant URL
Container name resolution doesn't work for host-networked services.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:19 +01:00
jpmschweitzerandClaude Opus 4.6 aa54af8a87 config(agents): add Anthropic API config for Tatlock and Webber
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:14 +01:00
jpmschweitzerandClaude Opus 4.6 dd141099e1 config(ollama): reduce VRAM usage for GPU sharing with other services
Change keep-alive from infinite to 5m and max loaded models from 2 to 1,
freeing VRAM for Stable Audio, TRELLIS, and other GPU services.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:34:09 +01:00
jpmschweitzerandClaude Opus 4.6 ec8c3f5925 feat(home-assistant): switch to host networking and add Matter Server
- Switch HA from bridge to host networking for better device discovery
- Add python-matter-server container for Matter protocol support
- Both services share host network for mDNS/IPv6 multicast
- Update NPM forward hostname to use IP address
- Add Matter Server setup instructions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:33:49 +01:00
jpmschweitzerandClaude Opus 4.6 24a70d9527 fix(agents): correct RAM spec from 16GB to 64GB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:33:42 +01:00
jpmschweitzerandClaude Opus 4.6 9ac817bb6e chore(gitignore): ignore local Claude settings and Jupyter checkpoints
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 10:33:35 +01:00
jpmschweitzerandClaude Opus 4.5 cd3cdad11c docs(containers): add Shell In A Box host service
Add shellinabox web-based terminal emulator running on port 4200,
accessible via https://shell.schweitz.net through NPM reverse proxy.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 17:55:10 +01:00
jpmschweitzerandClaude Opus 4.5 27e7d9aff0 docs(home-assistant): add MCP server setup guide for Claude integration
Documents the configuration steps for connecting Claude.ai and Claude Desktop
to Home Assistant via the Model Context Protocol (MCP) server endpoint.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 09:12:35 +01:00
jpmschweitzerandClaude Opus 4.5 653cd2771e docs(penpot): update status and correct port/healthcheck info
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 12:43:23 +01:00
jpmschweitzerandClaude Opus 4.5 f54b6cff08 fix(penpot): correct port mappings, healthchecks, and feature flags
- Fix frontend port mapping (8080, not 80)
- Fix healthcheck endpoints to use /readyz
- Add secret key to exporter service
- Disable email verification (no SMTP configured)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 12:40:19 +01:00
jpmschweitzerandClaude Opus 4.5 7d61fab2b7 feat(stack): add Penpot design platform deployment
Add self-hosted Penpot (Figma alternative) with:
- Docker stack using shared PostgreSQL and Redis infrastructure
- Authentik SSO integration (OIDC, password login disabled)
- Three services: frontend, backend, exporter
- Port 9001 for web UI, external via penpot.schweitz.net

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 11:33:03 +01:00
jpmschweitzerandClaude Opus 4.5 39b604a0bd config(authentik): increase outpost memory limit to 1G
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 19:03:25 +01:00
jpmschweitzerandClaude Opus 4.5 bcae6055af config(agents): fix ALLOWED_PATHS array syntax in Webber
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 19:01:38 +01:00
jpmschweitzerandClaude Opus 4.5 fad6325b80 docs(amp): document AMP containerization migration
- AMP ADS controller now runs in Docker container (standalone, not Portainer stack)
- Uses host network mode for game server container communication
- Custom entrypoint wrapper handles Docker socket permissions
- Storage split: SSD (~/docker-data/amp/) for config, HDD (/mnt/media/amp/instances/) for game data
- Updated quick reference, service access matrix, storage distribution, and network architecture tables

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 18:59:15 +01:00
jpmschweitzerandClaude Opus 4.5 e8021307da feat(stack): create unified agents stack with Tatlock and Webber
Consolidate Tatlock API into new agents stack and add Webber LLM agent
orchestration service. Webber provides autonomous agent execution with
tool use, authenticating via Tatlock API.

Changes:
- Add stacks/agents.yml with Tatlock (port 8000) and Webber (port 8086)
- Remove stacks/tatlock.yml (merged into agents stack)
- Document Webber service in CONTAINERS.md (Redis DB 9)
- Add full Tatlock API documentation to CONTAINERS.md

Config updates (pre-existing):
- core-api: Enable OIDC, add host stats access, simplify model config
- gitea: Migrate from dedicated DB to postgres-shared
- media: Remove GPU reservation from Jellyfin

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 19:27:44 +01:00
jpmschweitzerandClaude Opus 4.5 3df367ff85 config(media): add SABnzbd host whitelist for web access
Adds HOST_WHITELIST env var to allow access from container name,
localhost, LAN IP, and hostname.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 18:42:46 +01:00
jpmschweitzerandClaude Opus 4.5 d3400fb622 config(authentik): optimize DB connections and worker scaling
- Replace USE_PGBOUNCER with CONN_MAX_AGE=0 and CONN_HEALTH_CHECKS
- Reduce web workers to 1 with 2 threads (homelab scale)
- Update worker config: CONCURRENCY=1, THREADS=2
- Remove redundant Redis config from worker service

Reduces overall PostgreSQL connection count.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:42:04 +01:00
jpmschweitzerandClaude Opus 4.5 6052f81ec1 feat(stack): add unified media stack with Sonarr, Radarr, Prowlarr, SABnzbd
Consolidates Jellyfin into a new media stack with full *arr automation:
- Sonarr (8989) - TV show management
- Radarr (7878) - Movie management
- Prowlarr (9696) - Indexer management
- SABnzbd (8880) - Usenet download client
- Jellyfin (8096) - Media streaming with GPU transcoding

Database: All *arr apps use PostgreSQL (postgres-shared) with media_user.
Databases created: sonarr, radarr, prowlarr

Also adds pg-connections.sh utility script for monitoring PostgreSQL connections.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 12:39:27 +01:00
jpmschweitzer dc1fb868fd host system stats access 2026-01-04 11:47:29 +01:00
jpmschweitzerandClaude Opus 4.5 47ceb31a18 config(authentik): upgrade to 2025.10.3, remove Redis dependency
Authentik 2025.10 no longer uses Redis for caching/sessions (now uses
Postgres). Updated stack to remove Redis config and renamed deprecated
WORKER__CONCURRENCY to WORKER__THREADS. Redis DB 0 now available.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 11:02:28 +01:00
jpmschweitzerandClaude Opus 4.5 ba0aa5ef3d feat(stack): replace Organizr with Tatlock UI, remove Netdata
Remove Organizr dashboard and Netdata monitoring:
- Delete stacks/organizr.yml and stacks/netdata.yml
- Delete organizr-widgets/ directory and npm forward-auth config
- Remove organizr database references from postgres-shared docs

Promote Tatlock UI as primary dashboard:
- Move from port 8092 to 9999 (Organizr's port)
- Enable external access at home.schweitz.net
- Update all documentation references

Update service counts: 26 containers across 20 stacks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 11:23:45 +01:00
jpmschweitzerandClaude Opus 4.5 d647a2e005 config(core-api): reorganize environment and add service integrations
- Disable debug mode for production
- Add Portainer, NPM, Postgres, Authentik API configurations
- Add SearXNG URL for search integration
- Use container name for Home Assistant URL
- Simplify healthcheck to /health endpoint
- Group environment variables by service

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 13:58:57 +01:00
jpmschweitzerandClaude Opus 4.5 4dc72b2128 feat(stack): add Tatlock UI home lab dashboard
Add Flutter-based dashboard to replace Organizr:
- Port 8092, stateless static web app via nginx
- Resource limits (128M/32M), wget-based healthcheck
- Documented in CONTAINERS.md with full service profile

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 13:58:26 +01:00
jpmschweitzerandClaude Opus 4.5 7b729aa285 docs: remove completed Paperless-ngx planning document
The NEW_CONTAINERS.md planning document for Paperless-ngx and ClamAV
has served its purpose - all information was already integrated into
CONTAINERS.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 20:52:10 +01:00
jpmschweitzerandClaude Opus 4.5 566afe0de5 config(library-desk): add system settings database and scheduler integration
Add environment variables for:
- Central settings database (postgres-shared/system_settings)
- Scheduler service URL for task coordination

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 00:22:17 +01:00
jpmschweitzerandClaude Opus 4.5 f3d29089d6 config(npm): reorganize tatlock and webui domain mappings
Fix naming conflict where tatlock.schweitz.net incorrectly pointed to
Open WebUI. Now correctly maps:
- tatlock.schweitz.net → Tatlock API (port 8000) with Authentik SSO
- webui.schweitz.net → Open WebUI (port 82) with Authentik SSO
- Corresponding .schweitz.internal domains for programmatic access

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 00:03:55 +01:00
jpmschweitzerandClaude Opus 4.5 facdd6a2ec feat(stack): add AdGuard Home DNS ad-blocking
Deploy AdGuard Home as network-wide DNS ad blocker with:
- Quad9 DoH upstream for encrypted, security-focused DNS
- Web UI on port 3053, DNS on 192.168.86.149:53
- Internal domain: dns.schweitz.internal

Includes setup guide (ADGUARD_SETUP.md) for completing wizard.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 14:09:38 +01:00
jpmschweitzerandClaude Opus 4.5 7f9e73a849 feat(npm): add internal domain aliases for all external hosts
Add *.schweitz.internal domains as HTTP-only alternatives to *.schweitz.net
domains for programmatic access without SSL or Authentik authentication.

- Configure 11 internal domain proxy hosts in NPM
- Document internal domain setup process in setup-new-host.md
- Add internal domains reference table to CONTAINERS.md
- Update external domains list with library and tatlock mappings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 11:51:13 +01:00
jpmschweitzerandClaude Opus 4.5 765818b4e9 feat(stack): add Paperless-ngx document management system
- Add paperless.yml stack (port 8091, documents.schweitz.net)
- Uses shared postgres (DB: paperless) and redis (DB 8)
- ClamAV installed on host for virus scanning (port 3310)
- Add Paperless integration to library-desk stack
- Update CONTAINERS.md with Paperless and ClamAV profiles
- Add Portainer and NPM API documentation to setup-new-host.md
- Update redis-shared.yml and postgres-shared.yml with Paperless refs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 00:36:15 +01:00
jpmschweitzer 7e06c915ad remove old webscraper code and add new home assistant settings to env for core-api.yaml 2025-12-24 19:38:51 +01:00
jpmschweitzerandClaude Opus 4.5 6a9000569a feat(stack): add Home Assistant smart home platform
- Add home-assistant.yml stack (port 8123, privileged mode for USB)
- External access via https://housekeeping.schweitz.net
- Health check using /manifest.json endpoint
- Update CONTAINERS.md with service profile
- Update stacks/README.md with port allocation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 22:32:13 +01:00
jpmschweitzer b3a30a5694 add qdrant env vars to tatlock 2025-12-14 15:49:57 +01:00
jpmschweitzerandClaude Opus 4.5 64ffeda928 feat: add Redis DB tracking and resolve DB conflicts
- Add Redis DB column to All Services table in CONTAINERS.md
- Move Nextcloud from DB 3 to DB 7 to resolve conflict with Scheduler
- Split Tatlock Redis usage: DB 1 (memory), DB 6 (benchmarks)
- Add missing services to table: Wiki.js, Tatlock, Library Desk

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 14:46:21 +01:00
jpmschweitzerandClaude Opus 4.5 5d0c0fbad9 chore: remove hardcoded APP_VERSION from stack files
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 13:36:03 +01:00
jpmschweitzerandClaude Opus 4.5 bde4386b90 feat: enable Watchtower HTTP API and use internal registry
- Add HTTP API to Watchtower for CI/CD triggered updates
- Update custom container images to use git.schweitz.internal
- Add GITEA_TOKEN env var to scheduler for release cleanup task

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 13:26:51 +01:00
jpmschweitzerandClaude Opus 4.5 9b2b32323c docs: add setup guide for SSL + Authentik protected hosts
Step-by-step instructions for adding new subdomains with:
- Let's Encrypt SSL via NPM
- Authentik forward authentication
- Troubleshooting common issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 12:06:05 +01:00
jpmschweitzerandClaude Opus 4.5 75c3950dd8 feat: add Docker healthchecks, remove Uptime Kuma
- Add healthcheck configurations to 13 stacks for Portainer status monitoring
- Remove Uptime Kuma service (replaced by Docker healthchecks)
- Clean up stale Heimdall references
- Update documentation and service counts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 12:05:05 +01:00
jpmschweitzer 8793ff1779 update with library desk api ENV vars 2025-12-11 19:20:40 +01:00
jpmschweitzerandClaude Opus 4.5 7c62577377 fix: use JSON array format for CORS_ORIGINS env var
Pydantic settings expects list[str] fields as JSON arrays, not plain strings.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 18:46:42 +01:00
jpmschweitzerandClaude Opus 4.5 235b694356 feat: add tatlock stack for homelab butler API
- OpenAI-compatible API server with LLM agent orchestration
- Port 8000, connects to docker-dataplane network
- Integrates with ollama, searxng, redis-shared services
- Watchtower auto-update enabled

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 18:28:04 +01:00
jpmschweitzerandClaude Opus 4.5 6581a743ac refactor: extract library-desk to standalone repository
- Move library-desk service to git.schweitz.net/jpmschweitzer/library-desk
- Update stacks/library-desk.yml to use container image
- Add library-desk to External Repositories in CONTAINERS.md
- Remove source code (now in external repo)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 17:46:48 +01:00
jpmschweitzer 98c92a1211 cleanup of docs 2025-12-11 16:32:37 +01:00
jpmschweitzerandClaude Opus 4.5 35077ed5ca chore: prune obsolete documentation and consolidate
- Remove obsolete plans/ directory (AI orchestrator moved to separate services)
- Remove docs/sessions/ (historical session notes)
- Remove docs/reference/ (duplicated in CONTAINERS.md)
- Remove docs/architecture/ (duplicated in CONTAINERS.md)
- Remove maintenance stack and backup-procedures guide (decommissioned)
- Remove scripts/ directory (unused)
- Fold EXTERNAL_SERVICES.md into CONTAINERS.md
- Update Nextcloud to reflect PostgreSQL shared (was MariaDB)
- Clean up README.md links

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 16:29:06 +01:00
jpmschweitzerandClaude Opus 4.5 0815f02529 refactor: extract core-api to standalone repository
Core-API service moved to git.schweitz.net/jpmschweitzer/core-api
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Documentation updated with external service reference
- Removed obsolete docs/services/core-api.md (docs now in core-api repo)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 16:10:17 +01:00
jpmschweitzerandClaude Opus 4.5 ca83bbffb5 refactor: extract scheduler to standalone repository
Scheduler service moved to git.schweitz.net/jpmschweitzer/scheduler
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Added Gitea Actions runner to gitea stack for CI/CD
- Documentation updated with external service reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 15:18:42 +01:00
jpmschweitzerandClaude Opus 4.5 9af231d72d fix(stacks): configure ollama for stability and persistent model loading
- OLLAMA_NUM_PARALLEL=1: Process requests sequentially to avoid batch
  overflow panics when multiple services call embeddings concurrently
- OLLAMA_KEEP_ALIVE=-1: Keep models loaded in VRAM indefinitely
- OLLAMA_MAX_LOADED_MODELS=2: Allow both mistral-nemo and nomic-embed-text
  to stay loaded simultaneously

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 11:01:03 +01:00
jpmschweitzerandClaude Opus 4.5 375913ec9d fix(library-desk): ensure page_id is int in get_page GraphQL call
GraphQL requires Int type, but page_id might be passed as string.
Explicit int() conversion prevents 400 Bad Request errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:31:55 +01:00
jpmschweitzerandClaude Opus 4.5 4cc2002fa6 fix(library-desk): reduce ollama health check to single API call
Previously called /api/tags twice (once for server check, once for
model check). Now reuses response from single call.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:31:07 +01:00
jpmschweitzerandClaude Opus 4.5 c91f77bf28 chore(stacks): change wiki.js port to 3000 (internal = external)
Simplifies access by using the same port internally and externally.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:13:28 +01:00
jpmschweitzerandClaude Opus 4.5 a607d7a660 fix(library-desk): resolve warnings and errors in logs
- Fix WikiChangeListener: use get_page() instead of non-existent get_page_by_id()
- Fix webhooks: use delete_page_chunks() instead of non-existent delete_page_vectors()
- Fix Neo4j: replace deprecated id() with elementId() in entity query
- Change some warnings to debug level to reduce log noise

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 22:03:54 +01:00
jpmschweitzerandClaude Opus 4.5 40aafb0331 docs: update changelog for v0.12.0-library-desk-enhancements
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:44:20 +01:00
jpmschweitzerandClaude Opus 4.5 4b9092d631 feat(library-desk): improve entity linking with fuzzy matching
- Add fuzzy_match_entity_to_document with confidence scoring
- Filter self-referential links (entity linking to current page)
- Fix path cleaning to preserve full wiki paths
- Add longest-first matching to prevent partial matches

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:36:24 +01:00
jpmschweitzerandClaude Opus 4.5 651de9ac92 chore(stacks): update library-desk env vars for wiki change detection
Replace WIKIJS_API_KEY with WIKIJS_USERNAME/PASSWORD and add
WIKIJS_DB_PASSWORD for PostgreSQL LISTEN/NOTIFY connection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:36:02 +01:00
jpmschweitzerandClaude Opus 4.5 14d152bd64 test(library-desk): add test coverage for new features
- test_consolidation: taxonomy fetching, ingestion_service injection
- test_graph_service: Document node tags, entity-stub skipping
- test_ingestion: list_all_pages usage, batch ingestion
- test_integration: list_all_pages, get_taxonomy_structure
- test_wiki_change_listener: PostgreSQL notification handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:35:39 +01:00
jpmschweitzerandClaude Opus 4.5 b74b5bc2c8 feat(library-desk): add wiki change detection via PostgreSQL LISTEN/NOTIFY
- Add WikiChangeListener service for PostgreSQL notifications
- Add webhooks router for HTTP webhook fallback
- Add asyncpg dependency for PostgreSQL async support
- Include setup scripts and documentation for triggers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:35:18 +01:00
jpmschweitzerandClaude Opus 4.5 bce6b71f9b fix(library-desk): use list_all_pages instead of search_pages for bulk re-index
search_pages with empty query uses stale search index. list_all_pages
fetches actual page list from GraphQL for reliable bulk re-indexing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:34:40 +01:00
jpmschweitzerandClaude Opus 4.5 d100212da5 fix(library-desk): add tags property to Document nodes in graph_service
Set d.tags = \$tags in MERGE query to prevent Neo4j warnings about
missing tags property in related documents query.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:32:37 +01:00
jpmschweitzerandClaude Opus 4.5 aba981ff21 fix(library-desk): inject ingestion_service into ConsolidationService
Pass ingestion_service to ConsolidationService constructor so pages
created/updated during consolidation are properly indexed in Qdrant
and Neo4j.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:31:15 +01:00
jpmschweitzerandClaude Opus 4.5 6ddf2a1bd9 feat(library-desk): add taxonomy-aware classification to consolidation
- Fetch existing wiki taxonomy structure before LLM analysis
- Include existing paths in prompt to prefer existing categories
- Add _format_taxonomy_for_prompt helper
- Mark searches as processed even when skipped/errored (prevents buildup)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:58 +01:00
jpmschweitzerandClaude Opus 4.5 9baea7ee6c feat(library-desk): add list_all_pages and get_taxonomy_structure to wikijs client
- list_all_pages: fetch all pages with path prefix filter
- get_taxonomy_structure: extract category/subcategory structure for
  taxonomy-aware classification

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:47 +01:00
jpmschweitzerandClaude Opus 4.5 ed7792ef98 feat(library-desk): fetch page ID via GraphQL in wikijs-integration.js
Use pages.singleByPath(path, locale) GraphQL query to get page ID
during initialization instead of searching through page list.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 21:30:18 +01:00
jpmschweitzer d582f27098 more cleanup 2025-12-10 20:26:44 +01:00
jpmschweitzer 953bcc5388 cleanup 2025-12-10 20:26:06 +01:00
jpmschweitzer b80f43745a cleanup 2025-12-10 20:24:07 +01:00
jpmschweitzer c670b10630 obsolete 2025-12-10 20:23:57 +01:00
jpmschweitzer 32c4805a07 remove obsolete core-ai 2025-12-10 20:21:26 +01:00
jpmschweitzerandClaude Opus 4.5 65114cb477 fix(library-desk): fix idempotency in create_entity_mentions
Fix create_entity_mentions to correctly count only newly created relationships,
not all relationships processed by MERGE.

**Problem**: count(r) was returning ALL relationships touched by MERGE (both
created and matched), breaking idempotency tests.

**Solution**: Use temporary flag 'just_created' set only ON CREATE, filter to
those relationships, count them, then remove the flag. This ensures the count
only includes new relationships.

Now properly returns:
- N on first run (N new relationships created)
- 0 on subsequent runs (no new relationships, all already exist)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:10:31 +01:00
jpmschweitzerandClaude Opus 4.5 6ea6e4d0ad fix(library-desk): fix entity linking path cleaning and longest-first matching
Two critical bug fixes for entity linking:

1. **Path cleaning**: Replace hardcoded "users/jpmschweitzer/" with regex pattern
   to handle any user namespace. Now properly cleans paths for all users.

2. **Longest-first matching**: Move protected_ranges computation inside entity loop
   to recompute after each entity is processed. Prevents nested links like
   [[Machine](/machine) Learning](/ml) when processing multi-word entities.

These fixes ensure entity linking works correctly across all users and prevents
nested markdown links when entity names overlap.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:09:59 +01:00
jpmschweitzerandClaude Opus 4.5 60b0100861 test(library-desk): add comprehensive entity linking tests
Add test suite covering:
- Entity mention detection (case-insensitive, whole-word matching, sorting)
- Content link addition (protection of existing links, longest-first matching)
- Integration tests (get entities with paths, create relationships, idempotency)
- Multi-tenancy isolation tests

All 17 tests passing. Validates entity linking functionality end-to-end.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 02:09:38 +01:00
jpmschweitzer 1f0a894291 docs(library-desk): add architecture and API documentation
Documentation:
- Architecture overview and design decisions
- API endpoint reference
- Service interaction diagrams
- Configuration guide
- Multi-tenancy patterns
- Knowledge graph schema
2025-12-10 01:40:23 +01:00
jpmschweitzer ae8edf6111 chore(library-desk): add maintenance and cleanup scripts
Cleanup Scripts:
- cleanup_graph.py - Clean up duplicate entities and orphaned nodes
- cleanup_wiki.py - Remove orphaned pages and fix broken links
- Utility scripts for database maintenance
- Not part of main application, run manually
2025-12-10 01:40:10 +01:00
jpmschweitzer 9fe5faa3e7 feat(library-desk): add core API routers for wiki, vector, and ingestion
Ingestion Router:
- POST /ingest/page - Index single page
- POST /ingest/batch - Batch indexing
- POST /ingest/all - Full knowledge base refresh
- Support vector and graph ingestion

Vector Router:
- POST /vector/search - Semantic search via Qdrant
- GET /vector/stats - Collection statistics
- DELETE /vector/page - Remove page embeddings

Wiki Router:
- GET /wiki/pages - List wiki pages
- GET /wiki/pages/{id} - Get page details
- PUT /wiki/pages/{id} - Update page
- POST /wiki/search - Search wiki content
- Full Wiki.js GraphQL integration
2025-12-10 01:36:07 +01:00
jpmschweitzer 3ab44c2ab0 feat(library-desk): add AI agent tools router
Tools Router:
- Expose library-desk capabilities as AI tool endpoints
- Support function calling for LLM agents
- Query wiki pages, search knowledge base
- Access graph entities and relationships

Tools Models:
- ToolDefinition for function schemas
- ToolParameter specifications
- ToolResponse format
- OpenAI function calling compatible
2025-12-10 01:30:23 +01:00
jpmschweitzer dca8b63a50 feat(library-desk): add graph query router and models
Graph Router:
- GET /graph/entities - List all entities for user
- GET /graph/relationships - Query entity relationships
- GET /graph/search - Search entities by name/type
- GET /graph/stats - Knowledge graph statistics

Graph Models:
- Entity, Relationship models
- GraphStats for analytics
- SearchFilters for queries
- Support multi-tenancy with user isolation
2025-12-10 01:28:42 +01:00
jpmschweitzer 8f29bfe064 feat(library-desk): add knowledge consolidation system
Consolidation Service:
- Automated wiki page creation from research results
- Entity extraction and disambiguation
- Multi-source fact integration
- Confidence scoring and source citation
- Template-based page generation
- Schema.org taxonomy integration

Wiki Page Writer:
- Structured markdown generation
- Standard templates (person, organization, technology)
- Metadata formatting (tags, categories, timestamps)
- Citation and source linking

Router:
- POST /consolidate/research endpoint
- Batch consolidation support
- Manual and automated triggers

Models:
- ConsolidationRequest with source data
- ConsolidationResult with page details
- Entity resolution metadata

Tests:
- Page generation validation
- Entity extraction accuracy
- Multi-source merging logic
2025-12-10 01:28:29 +01:00
jpmschweitzer 15930a9600 feat(library-desk): implement HybridRAG query system
HybridRAG Service:
- Combine vector (Qdrant), graph (Neo4j), and web (SearXNG) search
- Reciprocal Rank Fusion (RRF) for result merging
- LLM re-ranking with mistral-nemo
- Graph enrichment with related dossiers
- Query enhancement with keyword/synonym extraction
- Search result persistence for offline processing

Router:
- POST /query/hybrid endpoint
- Configurable search limits per source
- Enable/disable individual sources
- Timing breakdown for performance monitoring

Models:
- HybridRAGRequest, HybridRAGResponse
- HybridRAGResult with source tracking
- KeywordExtraction for query analysis
- TimingBreakdown for performance metrics

Tests:
- End-to-end HybridRAG query tests
- RRF fusion algorithm validation
- Multi-source result merging
2025-12-10 01:28:13 +01:00
jpmschweitzer 8c0ced68eb feat(scheduler): add Pydantic models and improve task API
Scheduler API:
- Add Pydantic models for request/response validation
- Improve API documentation with examples
- Add detailed schedule pattern documentation
- Document priority levels and executor types

Models:
- TaskCreate, TaskUpdate, TaskResponse models
- Field validation and constraints
- Type safety for task operations

Documentation:
- Add TASK_REGISTRATION.md guide
- Document schedule patterns and executor configs
2025-12-10 01:26:21 +01:00
jpmschweitzer 0bd4f8056a feat(library-desk): add configuration for HybridRAG and enhance multi-tenancy
Configuration:
- Add HybridRAG settings (reranker model, search limits)
- Change Wiki.js auth from API key to username/password
- Configure vector, graph, and web search limits

Multi-tenancy:
- Add get_neo4j_user_base_label() for entity node labeling
- Support title-cased labels following Neo4j conventions
- Maintain namespace isolation for entities vs documents
2025-12-10 01:25:40 +01:00
jpmschweitzer 12fe7e55c0 refactor(library-desk): improve client implementations
Ollama Client:
- Improve model checking to handle :latest tag variants
- Match models with or without explicit tag

Qdrant Client:
- Add collection_exists() method for checking collection presence
- Refactor ensure_collection() to accept collection name directly
- Better separation of concerns

SearXNG Client:
- Add health_check() method for service monitoring
- Simple endpoint check without full search
- Used by health check endpoint
2025-12-10 01:25:07 +01:00
jpmschweitzer 67a124af91 feat(library-desk): add vector service for semantic search
Vector Service:
- Manage document embeddings in Qdrant
- Update vectors from wiki pages
- Handle chunking and embedding generation
- Support force refresh and incremental updates

Vector Models:
- VectorSearchResult for search responses
- VectorUpdateSummary for indexing metrics
- Track chunks created/deleted

Used by ingestion_service for page embedding
2025-12-10 01:21:56 +01:00
jpmschweitzer c2faddf54a feat(library-desk): add ingestion models for page processing
- Add IngestionResult model for single page ingestion
- Add BatchIngestionResult for batch operations
- Track vector chunks, graph entities, and relationships
- Include processing time metrics

Used by ingestion_service for page indexing
2025-12-10 01:21:21 +01:00
jpmschweitzer b777098957 feat(library-desk): add wiki models for page operations
- Add WikiPage model for page data
- Add WikiPageUpdate model for partial updates
- Add field validators for tags and descriptions
- Support optional fields for flexible updates

Used by wiki_service and entity_linking router
2025-12-10 01:20:15 +01:00
jpmschweitzer 7354a66a3d feat(library-desk): integrate entity linking into main app
Main App:
- Mount /static directory for serving Wiki.js integration scripts
- Register entity_linking router
- Refactor API key verification to dependencies module

Dependencies:
- Add service factory functions for all services
- Add get_wiki_service() for wiki operations
- Add get_graph_service() for entity operations
- Add get_ingestion_service() for auto entity linking
- Improve health check for SearXNG
2025-12-10 01:18:40 +01:00
jpmschweitzer d80f063186 feat(library-desk): add Wiki.js integration buttons for entity linking
- Add combined integration script with both re-index and entity linking
- Add standalone entity linking button
- Add standalone re-index button
- Auto-detect Library Desk URL from script tag
- Support both toolbar and floating button positions
- Show real-time status updates and notifications
- Auto-reload page after successful entity linking

Usage: Inject via Wiki.js Code Injection settings
<script src="http://IP:8089/static/wikijs-integration.js"></script>
2025-12-10 01:16:46 +01:00
jpmschweitzer 30b9825365 feat(library-desk): add entity management to graph and ingestion services
Graph Service:
- Add get_all_entities() to retrieve entities with wiki page paths
- Add create_entity_mentions() for MENTIONS relationship creation
- Support entity-to-document linking via title matching

Ingestion Service:
- Add _link_existing_entities() for automatic entity linking
- Auto-link entities during page ingestion
- Support skip_entity_linking parameter for granular control
2025-12-10 01:16:24 +01:00
jpmschweitzer 5a4a203beb fix(library-desk): preserve published status when updating wiki pages
- Add is_published parameter to WikiJS client update_page() method
- Update wiki_service to always pass is_published=True
- Prevents pages from being unpublished during entity linking updates
- Important for internal wikis where all pages should remain published
2025-12-10 01:16:11 +01:00
jpmschweitzer 8638aad2d4 feat(library-desk): add entity linking endpoint
- Add /entity-linking/link-page endpoint to find and link entity mentions
- Creates both MENTIONS relationships in Neo4j and hyperlinks in wiki content
- Supports automatic re-indexing after linking
- Returns detailed statistics on entities found and linked
- Protects existing markdown links from being nested
- Idempotent: safe to run multiple times

Implements dual entity linking:
1. Graph relationships (MENTIONS) for knowledge graph traversal
2. Wiki content hyperlinks for user navigation
2025-12-10 01:15:38 +01:00
jpmschweitzer 65a91ab6f5 feat(scheduler): add generic REST API executor for universal HTTP task execution
Add rest_api_executor as a universal executor that can call any REST API
endpoint across the system. This provides a standard way to trigger HTTP
operations from scheduled tasks.

Features:
- All HTTP methods: GET, POST, PUT, DELETE, PATCH
- Authentication: Bearer token, Basic auth, API key
- Environment variable substitution: ${VAR_NAME}
- JSONPath response extraction
- Configurable timeouts and SSL verification
- Sensitive data redaction in logs
- Custom headers support

This executor enables scheduler to call any service endpoint (Library Desk,
Core API, external webhooks) without needing service-specific executors.

Example usage:
{
  "executor": "rest_api_executor",
  "config": {
    "url": "http://library-desk:8089/consolidate/knowledge",
    "method": "POST",
    "payload": {"process_limit": 10},
    "auth": {"type": "bearer", "token": "${API_KEY}"}
  }
}
2025-12-09 14:26:46 +01:00
jpmschweitzerandClaude Opus 4.5 5263523fcd fix(library-desk): update Qdrant client to use query_points API
The Qdrant client API changed from search() to query_points().
Updated both search() and find_similar_chunks() methods.

All integration tests now passing: 14/14 ✓

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-08 19:13:33 +01:00
jpmschweitzerandClaude Opus 4.5 1a41e5bb80 feat(library-desk): implement Phase 1 service clients and infrastructure
Implements comprehensive service client layer for Library Desk API to support
Librarian AI agent with multi-tenant knowledge management across Neo4j, Qdrant,
Wiki.js, SearXNG, and Ollama.

## Service Clients (src/clients/)
- Neo4j async client with connection pooling and user-scoped labels
- Qdrant vector store with collection-per-user multi-tenancy
- Wiki.js GraphQL API client for page/dossier management
- SearXNG client for web search integration
- Ollama client for text embeddings (nomic-embed-text)

## Core Infrastructure (src/core/)
- Multi-tenancy helpers for user namespace management
  - Wiki.js: path-based namespaces (/users/{user})
  - Neo4j: user-specific labels (User_{User}_Document)
  - Qdrant: collection per user (library_desk_{user})
- Dependency injection with FastAPI Depends and @lru_cache singletons
- Lifecycle management (startup/shutdown) for all service connections

## Background Jobs (src/jobs/)
- Redis-based job manager for long-running operations
- Job status tracking with 24-hour TTL
- Support for queued, processing, completed, failed states

## Configuration
- Updated config.py with Redis DB 4 for library-desk jobs
- Updated docker-compose.yml: REDIS_DB from 2 to 4
- Added pytest and pytest-asyncio to requirements.txt

## Testing
- Unit tests: 25/25 passed (multi-tenancy helpers)
- Integration tests: 12/12 passed (all services verified)
  - Neo4j connection and CRUD operations
  - Qdrant vector operations with 768-dim embeddings
  - Wiki.js GraphQL queries
  - SearXNG web search
  - Job Manager with Redis
  - Dependency injection lifecycle
- pytest.ini configuration with asyncio support

## Health Monitoring
- Real-time service health checks via /health endpoint
- Connection status for all 5 external services
- Graceful degradation for partial service availability

## Architecture
- Follows async/await pattern throughout
- Connection pooling for Neo4j (singleton driver)
- HTTP client lifecycle management (httpx)
- Multi-tenancy enforced at client layer
- Default user: jpmschweitzer

Files changed: 26 files
- 5 new service clients (~1500 lines)
- 2 core modules (~500 lines)
- 1 job manager (~350 lines)
- 3 test files with 37 test cases
- Updated main.py with lifecycle hooks

All services tested and operational. Ready for Phase 2 (routers/services).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-08 19:09:54 +01:00