portainer-core/NEW_CONTAINERS.md

# Plan: Add Paperless-ngx and ClamAV

## Overview

Add document management (Paperless-ngx) and host-level virus scanning (ClamAV) to the tower-of-joy infrastructure.

## Decisions Made

| Decision | Choice |
|----------|--------|
| Database | Use `postgres-shared` (new DB: `paperless`) |
| Redis | Use `redis-shared` DB 8 |
| ClamAV | Host OS installation (full server protection) |
| External Access | Yes, at `documents.schweitz.net` via NPM |
| Network | `docker-dataplane` |
| Stack | Separate `paperless.yml` stack |

## Services

| Service | Port | Location | Purpose |
|---------|------|----------|---------|
| Paperless-ngx | 8091 | Container | Document management, OCR |
| ClamAV | 3310 | Host OS | Virus scanning + server protection |

---

## Implementation Steps

### Phase 1: ClamAV Host Installation (requires sudo)

**User must run these commands manually:**

```bash
# Install ClamAV
sudo apt update
sudo apt install clamav clamav-daemon -y

# Stop freshclam to update definitions
sudo systemctl stop clamav-freshclam
sudo freshclam
sudo systemctl start clamav-freshclam

# Configure clamd to listen on network
sudo sed -i 's/^#TCPSocket.*/TCPSocket 3310/' /etc/clamav/clamd.conf
sudo sed -i 's/^#TCPAddr.*/TCPAddr 0.0.0.0/' /etc/clamav/clamd.conf

# If TCPSocket/TCPAddr not present, add them
grep -q "^TCPSocket" /etc/clamav/clamd.conf || echo "TCPSocket 3310" | sudo tee -a /etc/clamav/clamd.conf
grep -q "^TCPAddr" /etc/clamav/clamd.conf || echo "TCPAddr 0.0.0.0" | sudo tee -a /etc/clamav/clamd.conf

# Restart daemon
sudo systemctl restart clamav-daemon

# Enable at boot
sudo systemctl enable clamav-daemon clamav-freshclam

# Verify
clamdscan --ping 3
```

### Phase 2: Create Storage Directories

```bash
# SSD - configs and working data
mkdir -p ~/docker-data/paperless/{data,consume,export}

# HDD - document storage
mkdir -p /mnt/media/paperless/media
```

### Phase 3: Create PostgreSQL Database

```bash
docker exec -i postgres-shared psql -U postgres <<'EOF'
-- Paperless database
CREATE DATABASE paperless;
CREATE USER paperless_user WITH PASSWORD '<generate-secure-password>';
GRANT ALL PRIVILEGES ON DATABASE paperless TO paperless_user;
\c paperless
GRANT ALL ON SCHEMA public TO paperless_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO paperless_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO paperless_user;
EOF
```

### Phase 4: Create Stack File

**Create:** `stacks/paperless.yml`

```yaml
version: '3.8'

# Paperless-ngx - Document Management System
# Port: 8091 (HTTP)
# GPU: No
# External: documents.schweitz.net

services:
  paperless:
    image: ghcr.io/paperless-ngx/paperless-ngx:latest
    container_name: paperless
    restart: unless-stopped
    ports:
      - "8091:8000"
    volumes:
      # SSD - config and working data
      - /home/jpmschweitzer/docker-data/paperless/data:/usr/src/paperless/data
      - /home/jpmschweitzer/docker-data/paperless/consume:/usr/src/paperless/consume
      - /home/jpmschweitzer/docker-data/paperless/export:/usr/src/paperless/export
      # HDD - document storage
      - /mnt/media/paperless/media:/usr/src/paperless/media
    environment:
      # Database (shared PostgreSQL)
      PAPERLESS_DBENGINE: postgresql
      PAPERLESS_DBHOST: postgres-shared
      PAPERLESS_DBPORT: 5432
      PAPERLESS_DBNAME: paperless
      PAPERLESS_DBUSER: paperless_user
      PAPERLESS_DBPASS: ${PAPERLESS_DB_PASSWORD}

      # Redis (shared, DB 8)
      PAPERLESS_REDIS: redis://redis-shared:6379/8

      # Security
      PAPERLESS_SECRET_KEY: ${PAPERLESS_SECRET_KEY}

      # URLs
      PAPERLESS_URL: https://documents.schweitz.net
      PAPERLESS_ALLOWED_HOSTS: "*"
      PAPERLESS_CORS_ALLOWED_HOSTS: "http://localhost:8091,https://documents.schweitz.net"

      # OCR Settings
      PAPERLESS_OCR_LANGUAGE: eng+nld
      PAPERLESS_OCR_MODE: skip
      PAPERLESS_OCR_OUTPUT_TYPE: pdfa

      # Webhooks (for Library Desk integration)
      PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS: "true"

      # Admin user (created on first run)
      PAPERLESS_ADMIN_USER: admin
      PAPERLESS_ADMIN_PASSWORD: ${PAPERLESS_ADMIN_PASSWORD}

      # Timezone
      TZ: Europe/Amsterdam
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 60s
    deploy:
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 512M
    networks:
      - docker-dataplane

networks:
  docker-dataplane:
    external: true
    name: docker-dataplane
```

### Phase 5: Environment Variables

Add to Portainer stack environment or `.env`:

```bash
# Generate these:
PAPERLESS_DB_PASSWORD=<openssl rand -hex 32>
PAPERLESS_SECRET_KEY=<openssl rand -base64 32>
PAPERLESS_ADMIN_PASSWORD=<your-admin-password>
```

### Phase 6: Deploy Stack

Via Portainer or:
```bash
cd /mnt/media/Projects/portainer-core/stacks
docker-compose -f paperless.yml up -d
```

### Phase 7: Configure NPM Proxy

Create proxy host in NPM:
- Domain: `documents.schweitz.net`
- Forward: `paperless:8000` (or `192.168.86.149:8091`)
- SSL: Request Let's Encrypt certificate
- Websockets: Enable
- Access List: As needed (Authentik SSO optional)

### Phase 8: Post-Deployment Configuration

1. **Access Paperless UI:** https://documents.schweitz.net
2. **Login** with admin credentials
3. **Create API Token:** My Profile → API Token → Create
4. **Create Custom Fields:**
   - `source_url` (URL)
   - `library_indexed` (Boolean)
   - `library_doc_id` (Text)
   - `collection` (Text)

5. **Configure Webhook Workflow:**
   - Trigger: Document Added
   - Action: Webhook to `http://library-desk:8089/documents/webhook`

### Phase 9: Update Library Desk Configuration

Add to Library Desk environment:
```bash
PAPERLESS_URL=http://paperless:8000
PAPERLESS_TOKEN=<api-token>
CLAMAV_HOST=192.168.86.149  # Host IP (ClamAV on host)
CLAMAV_PORT=3310
CLAMAV_ENABLED=true
```

---

## Files to Create/Modify

| Action | File | Description |
|--------|------|-------------|
| Create | `stacks/paperless.yml` | Paperless-ngx stack |
| Update | `CONTAINERS.md` | Add Paperless profile, update tables |
| Update | `stacks/postgres-shared.yml` | Add paperless DB setup in comments |
| Update | `stacks/redis-shared.yml` | Update DB allocation (DB 8: Paperless) |

## CONTAINERS.md Updates

Add to Quick Reference table:
```
| **Paperless-ngx** | 8091 | https://documents.schweitz.net | Internet | No | 8 | ✅ Running |
| **ClamAV** | 3310 | N/A (host service) | No | No | - | ✅ Running |
```

Add to External Domains:
```
- **documents.schweitz.net** → Paperless-ngx
```

Update Redis DB allocation:
```
DB 8: Paperless (task queue, cache)
```

Add to PostgreSQL databases:
```
paperless (document storage)
```

---

## Verification Checklist

- [ ] ClamAV responds: `clamdscan --ping 3`
- [ ] PostgreSQL database exists: `docker exec postgres-shared psql -U postgres -c '\l' | grep paperless`
- [ ] Paperless container running: `docker ps | grep paperless`
- [ ] Paperless health check passes
- [ ] External access works: https://documents.schweitz.net
- [ ] Library Desk can reach Paperless webhook endpoint
- [ ] ClamAV accessible from containers: `nc -zv 192.168.86.149 3310`