diff --git a/CONTAINERS.md b/CONTAINERS.md index 3919803..0965425 100644 --- a/CONTAINERS.md +++ b/CONTAINERS.md @@ -1,6 +1,6 @@ # Container Reference - tower-of-joy Infrastructure -> **Last Updated:** 2025-12-30 +> **Last Updated:** 2026-01-01 > **Total Services:** 27 containers across 21 stacks > **System:** Intel i7-6700, RTX 2080 Ti (11GB VRAM), 16GB RAM, Zorin OS 16.3 diff --git a/NEW_CONTAINERS.md b/NEW_CONTAINERS.md deleted file mode 100644 index 8882e4d..0000000 --- a/NEW_CONTAINERS.md +++ /dev/null @@ -1,267 +0,0 @@ -# Plan: Add Paperless-ngx and ClamAV - -## Overview - -Add document management (Paperless-ngx) and host-level virus scanning (ClamAV) to the tower-of-joy infrastructure. - -## Decisions Made - -| Decision | Choice | -|----------|--------| -| Database | Use `postgres-shared` (new DB: `paperless`) | -| Redis | Use `redis-shared` DB 8 | -| ClamAV | Host OS installation (full server protection) | -| External Access | Yes, at `documents.schweitz.net` via NPM | -| Network | `docker-dataplane` | -| Stack | Separate `paperless.yml` stack | - -## Services - -| Service | Port | Location | Purpose | -|---------|------|----------|---------| -| Paperless-ngx | 8091 | Container | Document management, OCR | -| ClamAV | 3310 | Host OS | Virus scanning + server protection | - ---- - -## Implementation Steps - -### Phase 1: ClamAV Host Installation (requires sudo) - -**User must run these commands manually:** - -```bash -# Install ClamAV -sudo apt update -sudo apt install clamav clamav-daemon -y - -# Stop freshclam to update definitions -sudo systemctl stop clamav-freshclam -sudo freshclam -sudo systemctl start clamav-freshclam - -# Configure clamd to listen on network -sudo sed -i 's/^#TCPSocket.*/TCPSocket 3310/' /etc/clamav/clamd.conf -sudo sed -i 's/^#TCPAddr.*/TCPAddr 0.0.0.0/' /etc/clamav/clamd.conf - -# If TCPSocket/TCPAddr not present, add them -grep -q "^TCPSocket" /etc/clamav/clamd.conf || echo "TCPSocket 3310" | sudo tee -a /etc/clamav/clamd.conf -grep -q "^TCPAddr" /etc/clamav/clamd.conf || echo "TCPAddr 0.0.0.0" | sudo tee -a /etc/clamav/clamd.conf - -# Restart daemon -sudo systemctl restart clamav-daemon - -# Enable at boot -sudo systemctl enable clamav-daemon clamav-freshclam - -# Verify -clamdscan --ping 3 -``` - -### Phase 2: Create Storage Directories - -```bash -# SSD - configs and working data -mkdir -p ~/docker-data/paperless/{data,consume,export} - -# HDD - document storage -mkdir -p /mnt/media/paperless/media -``` - -### Phase 3: Create PostgreSQL Database - -```bash -docker exec -i postgres-shared psql -U postgres <<'EOF' --- Paperless database -CREATE DATABASE paperless; -CREATE USER paperless_user WITH PASSWORD ''; -GRANT ALL PRIVILEGES ON DATABASE paperless TO paperless_user; -\c paperless -GRANT ALL ON SCHEMA public TO paperless_user; -ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO paperless_user; -ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO paperless_user; -EOF -``` - -### Phase 4: Create Stack File - -**Create:** `stacks/paperless.yml` - -```yaml -version: '3.8' - -# Paperless-ngx - Document Management System -# Port: 8091 (HTTP) -# GPU: No -# External: documents.schweitz.net - -services: - paperless: - image: ghcr.io/paperless-ngx/paperless-ngx:latest - container_name: paperless - restart: unless-stopped - ports: - - "8091:8000" - volumes: - # SSD - config and working data - - /home/jpmschweitzer/docker-data/paperless/data:/usr/src/paperless/data - - /home/jpmschweitzer/docker-data/paperless/consume:/usr/src/paperless/consume - - /home/jpmschweitzer/docker-data/paperless/export:/usr/src/paperless/export - # HDD - document storage - - /mnt/media/paperless/media:/usr/src/paperless/media - environment: - # Database (shared PostgreSQL) - PAPERLESS_DBENGINE: postgresql - PAPERLESS_DBHOST: postgres-shared - PAPERLESS_DBPORT: 5432 - PAPERLESS_DBNAME: paperless - PAPERLESS_DBUSER: paperless_user - PAPERLESS_DBPASS: ${PAPERLESS_DB_PASSWORD} - - # Redis (shared, DB 8) - PAPERLESS_REDIS: redis://redis-shared:6379/8 - - # Security - PAPERLESS_SECRET_KEY: ${PAPERLESS_SECRET_KEY} - - # URLs - PAPERLESS_URL: https://documents.schweitz.net - PAPERLESS_ALLOWED_HOSTS: "*" - PAPERLESS_CORS_ALLOWED_HOSTS: "http://localhost:8091,https://documents.schweitz.net" - - # OCR Settings - PAPERLESS_OCR_LANGUAGE: eng+nld - PAPERLESS_OCR_MODE: skip - PAPERLESS_OCR_OUTPUT_TYPE: pdfa - - # Webhooks (for Library Desk integration) - PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS: "true" - - # Admin user (created on first run) - PAPERLESS_ADMIN_USER: admin - PAPERLESS_ADMIN_PASSWORD: ${PAPERLESS_ADMIN_PASSWORD} - - # Timezone - TZ: Europe/Amsterdam - labels: - - "com.centurylinklabs.watchtower.enable=true" - healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:8000"] - interval: 30s - timeout: 10s - retries: 5 - start_period: 60s - deploy: - resources: - limits: - memory: 2G - reservations: - memory: 512M - networks: - - docker-dataplane - -networks: - docker-dataplane: - external: true - name: docker-dataplane -``` - -### Phase 5: Environment Variables - -Add to Portainer stack environment or `.env`: - -```bash -# Generate these: -PAPERLESS_DB_PASSWORD= -PAPERLESS_SECRET_KEY= -PAPERLESS_ADMIN_PASSWORD= -``` - -### Phase 6: Deploy Stack - -Via Portainer or: -```bash -cd /mnt/media/Projects/portainer-core/stacks -docker-compose -f paperless.yml up -d -``` - -### Phase 7: Configure NPM Proxy - -Create proxy host in NPM: -- Domain: `documents.schweitz.net` -- Forward: `paperless:8000` (or `192.168.86.149:8091`) -- SSL: Request Let's Encrypt certificate -- Websockets: Enable -- Access List: As needed (Authentik SSO optional) - -### Phase 8: Post-Deployment Configuration - -1. **Access Paperless UI:** https://documents.schweitz.net -2. **Login** with admin credentials -3. **Create API Token:** My Profile → API Token → Create -4. **Create Custom Fields:** - - `source_url` (URL) - - `library_indexed` (Boolean) - - `library_doc_id` (Text) - - `collection` (Text) - -5. **Configure Webhook Workflow:** - - Trigger: Document Added - - Action: Webhook to `http://library-desk:8089/documents/webhook` - -### Phase 9: Update Library Desk Configuration - -Add to Library Desk environment: -```bash -PAPERLESS_URL=http://paperless:8000 -PAPERLESS_TOKEN= -CLAMAV_HOST=192.168.86.149 # Host IP (ClamAV on host) -CLAMAV_PORT=3310 -CLAMAV_ENABLED=true -``` - ---- - -## Files to Create/Modify - -| Action | File | Description | -|--------|------|-------------| -| Create | `stacks/paperless.yml` | Paperless-ngx stack | -| Update | `CONTAINERS.md` | Add Paperless profile, update tables | -| Update | `stacks/postgres-shared.yml` | Add paperless DB setup in comments | -| Update | `stacks/redis-shared.yml` | Update DB allocation (DB 8: Paperless) | - -## CONTAINERS.md Updates - -Add to Quick Reference table: -``` -| **Paperless-ngx** | 8091 | https://documents.schweitz.net | Internet | No | 8 | ✅ Running | -| **ClamAV** | 3310 | N/A (host service) | No | No | - | ✅ Running | -``` - -Add to External Domains: -``` -- **documents.schweitz.net** → Paperless-ngx -``` - -Update Redis DB allocation: -``` -DB 8: Paperless (task queue, cache) -``` - -Add to PostgreSQL databases: -``` -paperless (document storage) -``` - ---- - -## Verification Checklist - -- [ ] ClamAV responds: `clamdscan --ping 3` -- [ ] PostgreSQL database exists: `docker exec postgres-shared psql -U postgres -c '\l' | grep paperless` -- [ ] Paperless container running: `docker ps | grep paperless` -- [ ] Paperless health check passes -- [ ] External access works: https://documents.schweitz.net -- [ ] Library Desk can reach Paperless webhook endpoint -- [ ] ClamAV accessible from containers: `nc -zv 192.168.86.149 3310`