ok... ok... I'll add it to git...

docs/backup-procedures.md

@@ -0,0 +1,219 @@
+# Backup & Restore Procedures
+
+## Overview
+
+The `maintenance` container runs scheduled backup tasks using cron. It's a simple, reliable, "set and forget" solution.
+
+**Current Backups:**
+- **Docker Configs:** Daily at 3 AM
+- **Retention:** 30 days
+- **Size:** ~94 MB per backup
+- **Location:** `/mnt/media/backups/docker-configs/`
+
+**What's Backed Up:**
+- ✅ All Docker container configurations
+- ✅ Nginx Proxy Manager configs & SSL certificates
+- ✅ Headscale database & config
+- ✅ All dashboard settings (Heimdall, Organizr, Uptime Kuma)
+- ✅ All service configs
+- ❌ Ollama models (re-downloadable)
+- ❌ Cache files
+- ❌ Log files
+
+## Automated Backups
+
+**Schedule:** Daily at 3:00 AM (configured in crontab)
+
+**View Backup Logs:**
+```bash
+# Real-time logs
+docker logs -f maintenance
+
+# Backup script logs
+cat ~/docker-data/maintenance/logs/backup-configs.log
+```
+
+**List Existing Backups:**
+```bash
+ls -lh /mnt/media/backups/docker-configs/
+```
+
+## Manual Backup
+
+Run a backup anytime:
+```bash
+docker exec maintenance /scripts/backup-configs.sh
+```
+
+## Restore from Backup
+
+### Full Restore
+
+1. **Stop all containers:**
+   ```bash
+   docker stop $(docker ps -aq)
+   ```
+
+2. **Backup current state (just in case):**
+   ```bash
+   mv ~/docker-data ~/docker-data.old
+   ```
+
+3. **Extract backup:**
+   ```bash
+   cd ~
+   tar -xzf /mnt/media/backups/docker-configs/docker-configs-YYYYMMDD-HHMMSS.tar.gz
+   ```
+
+4. **Restart containers:**
+   ```bash
+   docker start $(docker ps -aq)
+   ```
+
+5. **Verify services:**
+   ```bash
+   docker ps
+   ```
+
+### Selective Restore (Single Service)
+
+Restore only one service's config (example: Headscale):
+
+```bash
+# Extract only headscale directory
+tar -xzf /mnt/media/backups/docker-configs/docker-configs-20251111-221349.tar.gz \
+    --strip-components=2 \
+    -C ~/docker-data/ \
+    docker-data/headscale
+
+# Restart the service
+docker restart headscale
+```
+
+## Adding New Maintenance Tasks
+
+The maintenance container can run any scheduled task, not just backups.
+
+### 1. Create New Script
+
+```bash
+# Create script file
+nano ~/docker-data/maintenance/scripts/my-task.sh
+
+# Make it executable
+chmod +x ~/docker-data/maintenance/scripts/my-task.sh
+```
+
+### 2. Add to Crontab
+
+```bash
+# Edit crontab
+nano ~/docker-data/maintenance/crontab
+
+# Add your schedule (example: every Sunday at 4 AM)
+# 0 4 * * 0 /scripts/my-task.sh
+```
+
+### 3. Restart Container
+
+```bash
+docker restart maintenance
+```
+
+### Examples of Future Tasks
+
+- **Weekly cleanup:** Remove old Docker images
+- **Health checks:** Verify all services are responding
+- **Update checks:** Notify when container updates available
+- **Database optimization:** Compact/optimize databases
+- **SSL renewal checks:** Verify certificates are valid
+
+## Testing Backup Integrity
+
+Periodically test that backups can be restored:
+
+```bash
+# Create test directory
+mkdir -p /tmp/backup-test
+
+# Extract backup
+tar -xzf /mnt/media/backups/docker-configs/docker-configs-LATEST.tar.gz \
+    -C /tmp/backup-test
+
+# Verify contents
+ls -la /tmp/backup-test/docker-data/
+
+# Clean up
+rm -rf /tmp/backup-test
+```
+
+## Troubleshooting
+
+### Backup Not Running
+
+**Check if container is running:**
+```bash
+docker ps | grep maintenance
+```
+
+**Check cron logs:**
+```bash
+docker logs maintenance
+```
+
+**Manually run backup to test:**
+```bash
+docker exec maintenance /scripts/backup-configs.sh
+```
+
+### Backup Taking Too Long
+
+- Check if exclusions are working (Ollama models should be excluded)
+- Monitor disk I/O: `iostat -x 1`
+- Check HDD health: `sudo smartctl -a /dev/sdb`
+
+### Backup Disk Full
+
+- Old backups auto-delete after 30 days
+- Manually remove old backups if needed:
+  ```bash
+  # List backups by size
+  du -h /mnt/media/backups/docker-configs/*
+
+  # Remove specific backup
+  rm /mnt/media/backups/docker-configs/docker-configs-20251001-*.tar.gz
+  ```
+
+### Restore Failed
+
+1. Check backup file integrity:
+   ```bash
+   tar -tzf /mnt/media/backups/docker-configs/backup-file.tar.gz > /dev/null
+   ```
+
+2. If corrupted, try previous backup
+
+3. Check disk space before restoring:
+   ```bash
+   df -h ~/docker-data
+   ```
+
+## Backup Storage
+
+**Current Usage:**
+- ~94 MB per daily backup
+- 30 days retention = ~2.8 GB total
+- Stored on 3.6 TB HDD (plenty of space)
+
+**Offsite Backups (Recommended):**
+
+For extra protection, periodically copy backups to external drive:
+
+```bash
+# Copy last 7 days to external drive
+rsync -av --progress /mnt/media/backups/docker-configs/ /mnt/external-drive/backups/
+```
+
+---
+
+**Last Updated:** 2025-11-11