portainer-core/stacks/nextcloud-db-consolidation.md

# Nextcloud Database Consolidation Plan

**Goal**: Fresh Nextcloud installation using shared PostgreSQL + Redis infrastructure

**Date**: 2025-11-16
**Status**: APPROVED - Complete wipe and fresh start
**Approach**: No migration, no backups - complete fresh installation

---

## Current State

### Existing Setup
```yaml
nextcloud-db (MariaDB 10.11)
  ├─ Database: nextcloud
  ├─ User: nextcloud
  ├─ Data: /home/jpmschweitzer/docker-data/nextcloud/db
  └─ Network: docker-dataplane

nextcloud-redis (Redis Alpine)
  ├─ Standalone instance
  ├─ Data: In-memory only (no persistence configured)
  └─ Network: docker-dataplane

nextcloud (Nextcloud Stable)
  ├─ Config: /home/jpmschweitzer/docker-data/nextcloud/config
  ├─ Data: /mnt/media/nextcloud/data
  └─ Dependencies: nextcloud-db, nextcloud-redis
```

### Target Setup
```yaml
postgres-shared (PostgreSQL 16)
  ├─ New database: nextcloud
  ├─ New user: nextcloud_user
  └─ Database allocation: DB 3

redis-shared (Redis Alpine)
  ├─ Database allocation: DB 3 (Nextcloud)
  ├─ Existing DB 0: General cache
  ├─ Existing DB 1: Authentik
  └─ Existing DB 2: Gitea
```

---

## Migration Challenges

### Critical Issue: MariaDB → PostgreSQL
⚠️ **Nextcloud cannot simply switch database types!**

Nextcloud's database schema is different between MariaDB and PostgreSQL:
- Different data types (e.g., LONGTEXT vs TEXT)
- Different auto-increment handling
- Different JSON field types
- Different index structures

**Options:**

### Option A: Fresh Install + Data Migration (RECOMMENDED)
✅ **Pros:**
- Clean database schema
- Opportunity to optimize
- Lower risk of corruption
- Can test before switching

❌ **Cons:**
- Must recreate users/settings
- Requires careful data migration
- More complex process

### Option B: Database Conversion
✅ **Pros:**
- Preserves all settings
- Preserves user data

❌ **Cons:**
- Complex conversion process
- High risk of data loss
- Nextcloud doesn't officially support this
- May leave corrupted data

**RECOMMENDATION: Option A (Fresh Install)**

---

## Fresh Installation Plan

### Phase 1: Complete Cleanup - PURGE ALL DATA

**Estimated Time:** 2 minutes

⚠️ **DESTRUCTIVE OPERATION - REQUIRES EXPLICIT APPROVAL** ⚠️

The following will be PERMANENTLY DELETED:
- All Nextcloud containers (nextcloud, nextcloud-db, nextcloud-redis)
- All Nextcloud configuration (/home/jpmschweitzer/docker-data/nextcloud)
- All Nextcloud user files (/mnt/media/nextcloud)
- All Nextcloud database data

**APPROVAL REQUIRED BEFORE EACH DELETION STEP**

```bash
# Step 1: Stop and remove containers
# APPROVAL: Stop containers? (y/n)
docker stop nextcloud nextcloud-db nextcloud-redis 2>/dev/null || true
docker rm nextcloud nextcloud-db nextcloud-redis 2>/dev/null || true

# Step 2: Delete config directory
# APPROVAL: Delete /home/jpmschweitzer/docker-data/nextcloud? (y/n)
sudo rm -rf /home/jpmschweitzer/docker-data/nextcloud

# Step 3: Delete user data directory
# APPROVAL: Delete /mnt/media/nextcloud? (y/n)
sudo rm -rf /mnt/media/nextcloud

# Step 4: Verify complete removal
ls /home/jpmschweitzer/docker-data/ | grep nextcloud  # Should be empty
ls /mnt/media/ | grep nextcloud  # Should be empty
```

### Phase 2: Prepare Shared Infrastructure

**Estimated Time:** 5 minutes

```bash
# 1. Create Nextcloud database in postgres-shared
docker exec -i postgres-shared psql -U postgres <<'EOF'
-- Nextcloud database
CREATE DATABASE nextcloud;
CREATE USER nextcloud_user WITH PASSWORD 'GENERATE_NEW_PASSWORD_HERE';
GRANT ALL PRIVILEGES ON DATABASE nextcloud TO nextcloud_user;
\c nextcloud
GRANT ALL ON SCHEMA public TO nextcloud_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO nextcloud_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO nextcloud_user;
EOF

# 2. Update redis-shared documentation (already supports DB 3)
# No action needed - redis-shared already configured for multi-database
```

### Phase 3: Create Fresh Nextcloud Stack Configuration

**Estimated Time:** 5 minutes

Create new `nextcloud-shared.yml`:

```yaml
services:
  nextcloud:
    image: nextcloud:stable
    container_name: nextcloud
    restart: unless-stopped
    ports:
      - "8082:80"
    volumes:
      # Fresh config directory
      - /home/jpmschweitzer/docker-data/nextcloud/config:/var/www/html/config
      # Fresh user data directory
      - /mnt/media/nextcloud/data:/var/www/html/data
    environment:
      # PostgreSQL configuration
      - POSTGRES_HOST=postgres-shared
      - POSTGRES_DB=nextcloud
      - POSTGRES_USER=nextcloud_user
      - POSTGRES_PASSWORD=${NEXTCLOUD_DB_PASSWORD}

      # Redis configuration (Database 3)
      - REDIS_HOST=redis-shared
      - REDIS_HOST_PORT=6379
      - REDIS_DB_INDEX=3

      # Timezone
      - TZ=Europe/Amsterdam
    depends_on:
      - postgres-shared
      - redis-shared
    networks:
      - docker-dataplane
    deploy:
      resources:
        limits:
          memory: 1G

networks:
  docker-dataplane:
    external: true
    name: docker-dataplane
```

### Phase 5: Deploy Fresh Nextcloud

**Estimated Time:** 5 minutes

```bash
# 1. Create new config directory
mkdir -p /home/jpmschweitzer/docker-data/nextcloud-shared/config

# 2. Create .env file with database password
cat > /mnt/media/Projects/portainer-core/stacks/.env.nextcloud-shared <<EOF
NEXTCLOUD_DB_PASSWORD=<GENERATED_PASSWORD_FROM_PHASE2>
EOF

# 3. Deploy new stack
cd /mnt/media/Projects/portainer-core/stacks
docker compose -f nextcloud-shared.yml --env-file .env.nextcloud-shared up -d

# 4. Wait for initialization
docker logs -f nextcloud
```

### Phase 6: Initial Setup & Configuration

**Estimated Time:** 10 minutes

```bash
# 1. Access Nextcloud web interface
# Navigate to: http://localhost:8082 or https://cloud.schweitz.net

# 2. First-time setup wizard:
#    - Admin username: admin
#    - Admin password: <STRONG_PASSWORD>
#    - Data folder: /var/www/html/data (default)
#    - Database: PostgreSQL
#    - Database user: nextcloud_user
#    - Database password: <FROM_ENV_FILE>
#    - Database name: nextcloud
#    - Database host: postgres-shared

# 3. Wait for installation (2-3 minutes)

# 4. Configure trusted domains
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=cloud.schweitz.net
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 2 --value=192.168.86.149

# 5. Configure Redis caching
docker exec -u www-data nextcloud php occ config:system:set redis host --value=redis-shared
docker exec -u www-data nextcloud php occ config:system:set redis port --value=6379
docker exec -u www-data nextcloud php occ config:system:set redis dbindex --value=3
docker exec -u www-data nextcloud php occ config:system:set memcache.local --value='\\OC\\Memcache\\APCu'
docker exec -u www-data nextcloud php occ config:system:set memcache.distributed --value='\\OC\\Memcache\\Redis'
docker exec -u www-data nextcloud php occ config:system:set memcache.locking --value='\\OC\\Memcache\\Redis'

# 6. Optimize database
docker exec -u www-data nextcloud php occ db:add-missing-indices
docker exec -u www-data nextcloud php occ db:convert-filecache-bigint

# 7. Configure background jobs
docker exec -u www-data nextcloud php occ background:cron
```

### Phase 7: Verify Fresh Installation

**Estimated Time:** 5 minutes

```bash
# 1. Verify admin user can login via web interface
# Navigate to: https://cloud.schweitz.net

# 2. Check PostgreSQL connection
docker exec postgres-shared psql -U nextcloud_user -d nextcloud -c '\dt'

# 3. Check Redis caching
docker exec redis-shared redis-cli -n 3 DBSIZE

# 4. Verify storage location
docker exec -u www-data nextcloud php occ config:system:get datadirectory

# 5. Test file upload/download
# Upload a test file via web interface
# Download it back
# Delete it
```

### Phase 8: Final Cleanup & Documentation

**Estimated Time:** 2 minutes

```bash
# 1. Update postgres-shared.yml documentation
# Add Nextcloud to "Applications Using This Database" list

# 2. Update redis-shared.yml documentation
# Add "DB 3: Nextcloud (file locking, distributed cache)"

# 3. Rename stack file
cd /mnt/media/Projects/portainer-core/stacks
mv nextcloud.yml nextcloud-mariadb-archived.yml
mv nextcloud-shared.yml nextcloud.yml

# 4. Delete old archived stack (already purged data in Phase 1)
# All old containers and data already removed
```

---

## Rollback Plan

⚠️ **NO ROLLBACK POSSIBLE** ⚠️

Since all old data is purged in Phase 1, there is no rollback option.

If fresh installation fails:
1. Review error logs
2. Fix configuration issues
3. Retry fresh installation

This is acceptable since Nextcloud is not in production use.

---

## Testing Checklist

After fresh installation, verify:

- [ ] Admin login works
- [ ] File upload works
- [ ] File download works
- [ ] File delete works
- [ ] Redis caching active (`docker exec redis-shared redis-cli -n 3 DBSIZE` shows keys)
- [ ] PostgreSQL connection stable (`docker exec postgres-shared psql -U nextcloud_user -d nextcloud -c '\dt'` shows tables)
- [ ] Memory usage acceptable (<1GB for Nextcloud container)
- [ ] Nextcloud accessible via https://cloud.schweitz.net
- [ ] No errors in logs (`docker logs nextcloud`)

---

## Resource Savings

**Before Migration:**
- nextcloud-db (MariaDB): ~117 MB RAM
- nextcloud-redis: ~10 MB RAM
- **Total:** ~127 MB RAM + 2 containers

**After Migration:**
- Shared postgres-shared: Already running (minimal additional overhead for one more DB)
- Shared redis-shared: Already running (DB 3 uses ~5-10 MB additional)
- **Savings:** ~110-120 MB RAM + 2 fewer containers to manage

**Benefits:**
- Simplified infrastructure
- Centralized backups
- Better resource utilization
- Easier monitoring
- Consistent database management

---

## Risks & Mitigation

| Risk | Impact | Mitigation |
|------|--------|------------|
| Data loss during migration | HIGH | Full backups before starting, test on copy first |
| Incompatible plugins/apps | MEDIUM | Fresh install allows clean app selection |
| User resistance to re-setup | LOW | Minimal - same interface, same files |
| Extended downtime | MEDIUM | Plan migration during low-usage window |
| Redis DB conflict | LOW | Using dedicated DB 3, isolated from other apps |

---

## Timeline

**Total estimated time:** 20-30 minutes

- Phase 1: Purge all data: 2 min
- Phase 2: Prepare PostgreSQL/Redis: 5 min
- Phase 3: Create stack config: 2 min
- Phase 4: Create directories: 1 min
- Phase 5: Deploy Nextcloud: 3 min
- Phase 6: Initial setup & config: 10 min
- Phase 7: Testing: 5 min
- Phase 8: Documentation: 2 min

**Can be done anytime** - No production impact, no backups needed

---

## Approval Required

- [ ] Backup strategy approved
- [ ] Fresh install approach approved
- [ ] Downtime window approved
- [ ] Testing checklist reviewed
- [ ] Rollback plan understood
- [ ] Ready to proceed

---

## Notes

- **COMPLETE FRESH START** - All old data deleted
- Clean database, optimal performance from day one
- PostgreSQL generally faster than MariaDB for Nextcloud workloads
- Redis DB 3 dedicated to Nextcloud (isolated from other apps)
- No migration complexity - just a clean installation
- Ready for production use immediately after setup