Files

T

jpmschweitzer 0e3fef20fc restructure documentation

2025-11-20 10:17:09 +01:00

66 KiB

Raw Blame History

Container Platform Implementation Plan

Implementation Date: 2025-11-11 System: tower-of-joy Solution: Portainer + Docker Compose + Nginx Proxy Manager + Headscale + Ollama Estimated Total Time: 8-20 hours for base infrastructure (applications in backlog)

Overview

This document provides a detailed, phase-by-phase implementation plan for setting up a home server using Portainer, Docker Compose, Nginx Proxy Manager (reverse proxy), Headscale (SDN), and Ollama (ML models). Each phase includes specific steps, commands, and functionality tests to verify successful completion before proceeding.

Key Features:

Web-based container management (Portainer)
Unified reverse proxy interface (Nginx Proxy Manager on port 8000)
GPU-accelerated ML model serving (Ollama/vLLM with RTX 2080 Ti)
Dual-disk storage strategy (489GB SSD + 3.7TB HDD)
Game server integration (AMP compatibility)
Secure remote access (Headscale/Tailscale SDN)

Note: This plan focuses on establishing the base infrastructure first. Application deployments (Jellyfin, Nextcloud, Samba) have been moved to the Phase Backlog section and will be implemented after the infrastructure is solid.

Dual-Disk Storage Strategy

This system has two disks with 4.2TB total capacity:

SSD (489GB): System drive (/dev/sda) - OS, Docker images, container configs, databases
HDD (3.7TB): Media drive (/dev/sdb) - User content, media files, Nextcloud data, backups

Phase 1 includes mounting the 4TB media drive to /mnt/media and configuring it for automatic mounting. All subsequent container deployments will use this dual-disk strategy:

Fast SSD for configs and performance-critical data
Large HDD for bulk storage and user content

Pre-Implementation Checklist

Before starting, verify the following:

System: tower-of-joy (Zorin OS 16.3)
Docker version: 28.1.1 or later
Available disk space: At least 50GB free
NVIDIA GPU: RTX 2080 Ti detected
Root/sudo access available
Internet connection stable
Backup of important data completed

Pre-Flight System Check

# Verify Docker installation
docker --version
docker ps

# Check NVIDIA GPU
nvidia-smi

# Check available disk space
df -h /

# Check system resources
free -h

Expected Results:

Docker version 28.1.1 or later
NVIDIA driver showing RTX 2080 Ti
At least 50GB free disk space
At least 12GB RAM available

Phase 1: Foundation Setup

Goal: Install NVIDIA Container Toolkit, deploy Portainer, configure dual-disk storage, set up reverse proxy, and prepare ML infrastructure Estimated Time: 2-4 hours Prerequisites: Docker installed, sudo access

Phase Overview:

Install NVIDIA Container Toolkit for GPU support
Deploy Portainer for container management
Configure GPU management in Portainer
Mount 4TB media drive for container user content
Configure AMP integration and game server storage
Deploy Nginx Proxy Manager for unified web interface
Prepare infrastructure for ML model containers (Ollama/vLLM)

Step 1.1: Install NVIDIA Container Toolkit

The NVIDIA Container Toolkit enables GPU access within Docker containers, essential for Jellyfin hardware transcoding.

Commands

# Add NVIDIA Docker repository
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update package list
sudo apt-get update

# Install NVIDIA Container Toolkit
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker to use NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker

# Restart Docker daemon
sudo systemctl restart docker

Functionality Test 1.1: Verify GPU Access in Docker

# Test GPU access in a container
docker run --rm --gpus all nvidia/cuda:11.4.0-base-ubuntu20.04 nvidia-smi

Expected Output:

NVIDIA-SMI output showing RTX 2080 Ti
Driver version: 470.256.02
CUDA Version: 11.4

Troubleshooting:

If --gpus flag not recognized: Docker daemon didn't restart properly, retry sudo systemctl restart docker
If GPU not visible: Check sudo nvidia-ctk runtime configure --runtime=docker output for errors
If CUDA version mismatch: Acceptable as long as GPU is detected

✅ Checkpoint 1.1: GPU successfully accessible in Docker containers

Step 1.2: Deploy Portainer

Portainer provides the web-based management interface for all our containers.

Commands

# Create Portainer volume for persistent data
docker volume create portainer_data

# Deploy Portainer CE (Community Edition)
docker run -d \
  -p 8080:9000 \
  -p 8443:9443 \
  --name=portainer \
  --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v portainer_data:/data \
  portainer/portainer-ce:latest

Port Configuration:

Port 8080 (HTTP) - Main Portainer web UI
Port 8443 (HTTPS) - Portainer HTTPS access
Port 8000 - Reserved for Nginx Proxy Manager (see Step 1.6)

Note: This uses port 8080 for Portainer. AMP currently uses 8080 and will need to be reconfigured to port 8081 (see Step 1.5).

Functionality Test 1.2: Access Portainer Web UI

# Check Portainer is running
docker ps | grep portainer

# Check Portainer logs
docker logs portainer

Access Portainer:

Open browser and navigate to: http://localhost:8080 or http://tower-of-joy:8080
First-time setup: Create admin account
- Username: (choose your username)
- Password: (minimum 12 characters, use strong password)
Click "Get Started"
Select "local" environment

Expected Results:

Portainer login page loads
Can create admin account
Dashboard shows "local" Docker environment
Can see existing Docker containers (including Portainer itself)

✅ Checkpoint 1.2: Portainer web UI accessible and showing local Docker environment

Step 1.3: Configure Portainer for GPU Support

Enable GPU device selection in Portainer settings.

Steps

In Portainer web UI, navigate to: Settings → Docker
Scroll to Available GPU Devices
Click Enable GPU Management
Save settings

Functionality Test 1.3: Verify GPU Visibility in Portainer

Go to Home → local → Containers
Click Add Container
Scroll to Runtime & Resources
Check if GPU toggle is available

Expected Results:

GPU toggle visible in container creation
Can enable/disable GPU access per container

✅ Checkpoint 1.3: Portainer configured for GPU management

Step 1.4: Mount the 4TB Media Drive

Mount the Seagate IronWolf 4TB drive for storing all container user content (media, Nextcloud data, game saves, etc.).

System Storage Strategy

Two-Disk Configuration:

SSD (/dev/sda): Container configs, databases, Docker images, OS
HDD (/dev/sdb): User content, media files, Nextcloud data, backups

Commands

# Create mount point
sudo mkdir -p /mnt/media

# Mount the media drive
sudo mount /dev/sdb /mnt/media

# Verify mount
df -h /mnt/media

Expected Output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb        3.7T  XXG  X.XTT  XX% /mnt/media

Add to /etc/fstab for Automatic Mounting

# Backup current fstab
sudo cp /etc/fstab /etc/fstab.backup

# Add media drive to fstab
echo "UUID=f4300e91-3f51-45a0-b038-03335c5bd792 /mnt/media ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab

# Test fstab configuration
sudo mount -a

# Verify all mounts
df -h

Set Permissions

# Set ownership to current user
sudo chown -R $USER:$USER /mnt/media

# Verify permissions
ls -la /mnt/media

Create Directory Structure for Container Data

# Create base directories for container content
mkdir -p /mnt/media/jellyfin
mkdir -p /mnt/media/nextcloud
mkdir -p /mnt/media/backups
mkdir -p /mnt/media/game-servers
mkdir -p /mnt/media/downloads

# Verify structure
tree -L 1 /mnt/media

Functionality Test 1.4: Verify Media Drive Setup

# Check mount point
df -h /mnt/media

# Test write permissions
touch /mnt/media/test-file.txt
echo "Media drive test" > /mnt/media/test-file.txt
cat /mnt/media/test-file.txt
rm /mnt/media/test-file.txt

# Verify directory structure
ls -la /mnt/media

Expected Results:

/mnt/media shows 3.7TB available
Can create and write files
Directory structure created successfully
Owned by current user
Drive will auto-mount on reboot (via fstab)

Storage Usage Policy Going Forward:

✅ ALL user content → /mnt/media/* (HDD)
✅ Container configs → ~/docker-data/* (SSD)
✅ Docker images → /var/lib/docker (SSD)

✅ Checkpoint 1.4: Media drive mounted and configured for container user content

Step 1.5: Configure AMP Integration and Game Server Storage

Your system already has AMP (CubeCoders Application Management Panel) running for game server management. This step ensures AMP and Portainer coexist peacefully and configures game servers to use the 4TB media drive for world data.

AMP + Portainer Compatibility Overview

Architecture:

┌─────────────────────────────────────────┐
│  AMP (Native Process)                   │
│  ├── Manages: Game Server Containers   │
│  ├── Features: Mod mgmt, RCON, updates │
│  └── UI: http://localhost:8081  (NEW)  │
│                                         │
│  Portainer (Container)                  │
│  ├── Manages: Infrastructure Containers│
│  ├── Features: Jellyfin, Nextcloud, etc│
│  ├── Visibility: ALL containers         │
│  └── UI: http://localhost:8080  (NEW)  │
│                                         │
│  Both share: /var/run/docker.sock       │
└─────────────────────────────────────────┘

Port Changes:

Portainer: Now on port 8080 (standard web port)
AMP: Needs to be reconfigured from 8080 → 8081 (we'll do this below)

Management Strategy:

AMP continues managing: Game servers (Minecraft, etc.)
Portainer manages: Infrastructure (Jellyfin, Nextcloud, monitoring)
Portainer shows: All containers (including AMP's) for unified visibility
No conflicts: Both tools can safely share Docker socket

Reconfigure AMP to Port 8081

Since Portainer now uses port 8080, we need to move AMP to 8081:

# Stop AMP instances
sudo systemctl stop 'amp*'

# Edit AMP instance manager configuration
# The main instance typically binds to port 8081 in the config already
# but the instances themselves need port changes

# Option 1: Via AMP CLI (recommended)
sudo -u amp ampinstmgr --port 8081

# Option 2: Manual edit (if needed)
# sudo nano /opt/cubecoders/amp/Instances/ADS01/ADS01.kvp
# Find: Core.Webserver.Port=8080
# Change to: Core.Webserver.Port=8081

# Restart AMP
sudo systemctl start ampinstmgr

Verify AMP Port Change:

# Check AMP is running on 8081
curl -I http://localhost:8081

# Or check in browser
# Open: http://localhost:8081

Expected Result: AMP UI loads on port 8081

Add User to Docker Group

Allow running docker commands without sudo (matching AMP's access):

# Add current user to docker group
sudo usermod -aG docker $USER

# Verify (requires logout/login to take effect)
# For now, check the change was made:
grep docker /etc/group

Expected Output: docker:x:998:amp,jpmschweitzer

Important: Log out and log back in for group membership to take effect.

Functionality Test 1.5A: Verify Docker Access

After logging out and back in:

# Test docker access without sudo
docker ps

# Should see AMP's containers
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"

Expected Results:

Can run docker ps without sudo
See AMP's game server containers listed
No "permission denied" errors

Configure AMP to Use Media Drive for Game Data

Currently, AMP stores everything in /home/amp/.ampdata/. We'll configure it to store large world data on the 4TB HDD while keeping configs on SSD for performance.

Storage Strategy for Game Servers:

AMP configs/databases → /home/amp/.ampdata/ (SSD - small, frequently accessed)
World data/backups → /mnt/media/game-servers/ (HDD - large files)

Option A: Symbolic Links (Recommended)

Move existing world data to HDD and symlink:

# Create game server directories on media drive
sudo mkdir -p /mnt/media/game-servers/amp-instances
sudo chown -R amp:amp /mnt/media/game-servers/

# For each AMP instance, move world data (example for one server)
# IMPORTANT: Stop the server in AMP first!
# Then run these commands:

# Example for instance 'justcreate01' - STOP SERVER FIRST IN AMP!
# sudo -u amp mv /home/amp/.ampdata/instances/justcreate01/Minecraft/world \
#   /mnt/media/game-servers/amp-instances/justcreate01-world
#
# sudo -u amp ln -s /mnt/media/game-servers/amp-instances/justcreate01-world \
#   /home/amp/.ampdata/instances/justcreate01/Minecraft/world

Option B: AMP Datastore Configuration (Alternative)

AMP supports configuring alternate data paths. This can be set per-instance in AMP's web UI:

Access AMP: http://localhost:8080
For each instance:
- Navigate to instance settings
- Look for "Data Directory" or "World Path" settings
- Point to /mnt/media/game-servers/amp-instances/<instance-name>/
Restart the instance

Recommendation: Use Option A (symlinks) as it's more transparent and doesn't require AMP reconfiguration.

Configure AMP Backups to Media Drive

# Create backup directory on media drive
sudo mkdir -p /mnt/media/backups/amp
sudo chown -R amp:amp /mnt/media/backups/amp

# In AMP web UI, configure backup location:
# Settings → Backup → Backup Path: /mnt/media/backups/amp

Create Directory Structure for Future Game Servers

# Organized structure for game servers on HDD
mkdir -p /mnt/media/game-servers/amp-instances
mkdir -p /mnt/media/game-servers/minecraft
mkdir -p /mnt/media/game-servers/other
mkdir -p /mnt/media/backups/amp/automated
mkdir -p /mnt/media/backups/amp/manual

# Set ownership for AMP
sudo chown -R amp:amp /mnt/media/game-servers/
sudo chown -R amp:amp /mnt/media/backups/amp/

Functionality Test 1.5B: Verify AMP Integration

# Check AMP can access media drive
sudo -u amp touch /mnt/media/game-servers/test-amp-access.txt
sudo -u amp ls -la /mnt/media/game-servers/

# Verify Portainer can see AMP containers
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}" | grep -E "Names|amp|minecraft"

# Check disk usage
df -h /mnt/media
df -h /home/amp/.ampdata

Expected Results:

AMP user can write to /mnt/media/game-servers/
AMP containers visible in docker ps
Media drive shows available space
AMP data directory on SSD still accessible

Best Practices for AMP + Portainer Coexistence

Container Naming Convention:

AMP containers: Keep AMP's naming (usually auto-generated)
Infrastructure containers: Use descriptive names (jellyfin, nextcloud, etc.)

Management Boundaries:

✅ Manage via AMP: Game server containers (start, stop, update, configure)
✅ Manage via Portainer: Infrastructure containers
✅ View in Portainer: All containers (for monitoring and unified visibility)
❌ Don't modify AMP containers via Portainer: Use AMP's UI instead

In Portainer UI:

AMP containers will appear in the container list
You can view logs and stats
Don't stop/restart AMP containers via Portainer - use AMP UI
Can tag AMP containers for organization: game-server, amp-managed

Update Phase 1 Completion Checklist

Add AMP-related items to the checklist below.

✅ Checkpoint 1.5: AMP integration configured and game server storage optimized

Step 1.6: Deploy Nginx Proxy Manager (Reverse Proxy)

Deploy Nginx Proxy Manager to create a unified web interface consolidating all web-facing services. This provides:

Single entry point for all services
Automatic SSL/TLS certificates (Let's Encrypt)
Easy subdomain/path-based routing
Web-based configuration interface

Why Nginx Proxy Manager over Headscale for this?

Headscale is an SDN (VPN mesh network) solution - it doesn't handle reverse proxying
NPM provides a visual interface for managing web traffic routing
NPM handles SSL certificates automatically
Both can coexist: Headscale for secure remote access, NPM for unified local/web interface

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: nginx-proxy-manager
Web editor: Paste the following configuration

Docker Compose Configuration

version: '3.8'

services:
  nginx-proxy-manager:
    image: jc21/nginx-proxy-manager:latest
    container_name: nginx-proxy-manager
    restart: unless-stopped
    ports:
      - "8000:81"    # Admin web interface (using port 8000)
      - "80:80"      # HTTP traffic
      - "443:443"    # HTTPS traffic
    volumes:
      - /home/jpmschweitzer/docker-data/nginx-proxy-manager/data:/data
      - /home/jpmschweitzer/docker-data/nginx-proxy-manager/letsencrypt:/etc/letsencrypt
    environment:
      - DB_SQLITE_FILE=/data/database.sqlite

Click Deploy the stack

Create Directory Structure

# Create directories for NPM
mkdir -p ~/docker-data/nginx-proxy-manager/data
mkdir -p ~/docker-data/nginx-proxy-manager/letsencrypt

# Set permissions
sudo chown -R $USER:$USER ~/docker-data/nginx-proxy-manager

Functionality Test 1.6A: Access NPM Web Interface

# Check NPM container
docker ps | grep nginx-proxy-manager

# Check NPM logs
docker logs nginx-proxy-manager

Open browser: http://localhost:8000
First-time login:
- Email: admin@example.com
- Password: changeme
IMPORTANT: Change admin email and password immediately
Dashboard loads showing no proxy hosts yet

Expected Results:

NPM container running
Can access admin interface on port 8000
Admin credentials changed

Functionality Test 1.6B: Configure Proxy Hosts

Example: Set up Portainer access through NPM

In NPM, click Proxy Hosts → Add Proxy Host
Configure:
- Domain Names: portainer.tower-of-joy.local (or use your local domain)
- Scheme: http
- Forward Hostname/IP: portainer
- Forward Port: 9000 (Portainer's internal port)
- SSL: Can enable later with Let's Encrypt
Click Save

Test access:

Add to /etc/hosts (if using .local): 127.0.0.1 portainer.tower-of-joy.local
Access: http://portainer.tower-of-joy.local
Should proxy to Portainer

Alternative Simple Setup (Path-based):

If you don't want domain-based routing, you can use path-based routing:

Keep services on their original ports (8080, 8081, 8096, etc.)
Access via NPM dashboard on port 8000 as a unified launcher
Add links in NPM custom HTML page or use Heimdall (Phase 3)

Configure Services Through NPM (Optional)

After deploying services in later phases, you can configure:

jellyfin.local → Jellyfin (port 8096)
nextcloud.local → Nextcloud (port 8082)
amp.local → AMP (port 8081)
monitor.local → Uptime Kuma (port 3001)

Or use path-based routing:

localhost/jellyfin → Jellyfin
localhost/nextcloud → Nextcloud
localhost/amp → AMP

Functionality Test 1.6C: Verify NPM Working

# Check NPM is responding
curl -I http://localhost:8000

# Expected: HTTP 200 OK with NPM login page

Expected Results:

NPM admin interface accessible on port 8000
Can create proxy hosts
Proxy routing works (test with one service)
Port 8000 now serves as unified web interface entry point

✅ Checkpoint 1.6: Nginx Proxy Manager deployed and ready for service routing

Step 1.7: Prepare ML Model Infrastructure (Ollama/vLLM)

Prepare infrastructure for running large language models (LLMs) using Ollama or vLLM with GPU acceleration.

Hardware Context:

GPU: NVIDIA RTX 2080 Ti (11GB VRAM, CUDA 11.4)
Suitable for: Models up to ~7B parameters (quantized), some 13B models
NVIDIA Container Toolkit: Already installed in Step 1.1

Ollama vs vLLM:

Ollama: Easier to use, built-in model management, good for general use
vLLM: More performant for serving, requires more manual setup, better for production APIs
Recommendation: Start with Ollama (simpler), can add vLLM later if needed

Option A: Deploy Ollama (Recommended)

Create directory for Ollama models:

# Create Ollama directories
mkdir -p ~/docker-data/ollama/models

# Note: Models can be large (4-15GB each)
# Consider moving to HDD if SSD space is limited
# Alternative: mkdir -p /mnt/media/ollama/models

Create Stack in Portainer:

Navigate to: Stacks → Add Stack
Name: ollama
Web editor:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"  # Ollama API
    volumes:
      - /home/jpmschweitzer/docker-data/ollama/models:/root/.ollama  # Model storage
      # Alternative for HDD storage (if SSD space limited):
      # - /mnt/media/ollama/models:/root/.ollama
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Click Deploy the stack

Functionality Test 1.7A: Verify Ollama GPU Access

# Check Ollama container
docker ps | grep ollama

# Check Ollama logs
docker logs ollama

# Verify GPU is accessible inside container
docker exec ollama nvidia-smi

Expected Results:

Container running
GPU visible inside container
Ollama API listening on port 11434

Functionality Test 1.7B: Pull and Test a Model

# Pull a small model for testing (e.g., Llama 3.2 3B - ~2GB)
docker exec ollama ollama pull llama3.2:3b

# List available models
docker exec ollama ollama list

# Test the model with a simple prompt
docker exec ollama ollama run llama3.2:3b "Hello, how are you?"

# Monitor GPU usage during inference
watch -n 1 nvidia-smi

Expected Results:

Model downloads successfully
Model responds to prompt
GPU shows utilization during inference
Response generated in a few seconds

Test Ollama API

# Test API endpoint
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

Expected: JSON response with generated text

Model Storage Considerations

VRAM Limits (11GB RTX 2080 Ti):

3B models: ~2-3GB VRAM (quantized) ✅ Comfortable
7B models: ~4-6GB VRAM (quantized) ✅ Works well
13B models: ~8-10GB VRAM (quantized) ⚠️ Possible, but tight
20B+ models: ❌ Exceeds VRAM, will be very slow

Disk Storage:

Models range from 2GB (3B) to 15GB+ (13B+)
Recommend starting with 3-7B models
Store on HDD if SSD space limited (slight load time increase, but inference speed unaffected)

Recommended Models to Start:

# Efficient models for RTX 2080 Ti (11GB VRAM)
docker exec ollama ollama pull llama3.2:3b        # 2GB - Fast, general purpose
docker exec ollama ollama pull mistral:7b         # 4GB - High quality, coding
docker exec ollama ollama pull codellama:7b       # 4GB - Code-specialized
docker exec ollama ollama pull phi3:mini          # 2GB - Fast, reasoning

Option B: vLLM (Advanced, Optional)

If you need higher performance serving (multiple concurrent requests), you can deploy vLLM:

version: '3.8'

services:
  vllm:
    image: vllm/vllm-openai:latest
    container_name: vllm
    restart: unless-stopped
    ports:
      - "8001:8000"  # OpenAI-compatible API
    volumes:
      - /home/jpmschweitzer/docker-data/vllm/models:/root/.cache/huggingface
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    command: >
      --model meta-llama/Meta-Llama-3-8B-Instruct
      --dtype float16
      --max-model-len 4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Note: vLLM requires manual model download from HuggingFace and more configuration. Recommend starting with Ollama.

Configure NPM Proxy for Ollama (Optional)

Add Ollama to NPM for easier access:

NPM → Proxy Hosts → Add
Domain: ollama.tower-of-joy.local (or path /ollama)
Forward to: ollama:11434
Save

Functionality Test 1.7C: Performance Validation

# Test inference speed
time docker exec ollama ollama run llama3.2:3b "Write a haiku about computers"

# Check GPU memory usage
docker exec ollama nvidia-smi

# Verify no CUDA errors in logs
docker logs ollama | grep -i error

Expected Results:

Response in 1-5 seconds (depending on model size)
GPU VRAM usage visible in nvidia-smi
No CUDA out-of-memory errors
Smooth inference performance

Storage Management for Models

# Check disk usage
du -sh ~/docker-data/ollama/models/

# If moving to HDD for more space:
# 1. Stop Ollama
# docker stop ollama
#
# 2. Move models
# sudo mv ~/docker-data/ollama/models /mnt/media/ollama/
#
# 3. Update docker-compose volume path
# 4. Restart stack in Portainer

✅ Checkpoint 1.7: ML model infrastructure ready with GPU acceleration

Phase 1 Completion Checklist

Infrastructure:

NVIDIA Container Toolkit installed
GPU accessible in test container
Portainer deployed and running
Portainer web UI accessible at port 8080
Admin account created
Local Docker environment connected
GPU management enabled in Portainer

Storage:

4TB media drive mounted at /mnt/media
Media drive added to /etc/fstab for auto-mount
Directory structure created for container content (/mnt/media/jellyfin/, /mnt/media/nextcloud/, etc.)
Permissions set correctly on media drive

AMP Integration:

AMP reconfigured to port 8081
User added to docker group (logout/login required)
Can run docker ps without sudo
AMP containers visible in Portainer
Game server directories created on media drive (/mnt/media/game-servers/)
AMP user has write access to media drive
AMP backup directory configured (/mnt/media/backups/amp/)
Understand AMP + Portainer management boundaries

Reverse Proxy:

Nginx Proxy Manager deployed and running
NPM web UI accessible at port 8000
Admin credentials changed from defaults
Can create proxy hosts
Port 8000 serving as unified web interface entry point

ML Infrastructure:

Ollama deployed with GPU support
Ollama API accessible at port 11434
GPU visible inside Ollama container
Test model pulled and working (llama3.2:3b or similar)
GPU acceleration confirmed during inference
Model storage configured (SSD or HDD based on space)

Phase 1 Status: Foundation established with container management, dual-disk storage, AMP integration, reverse proxy, and ML infrastructure

Phase 2: Networking & External Access

Goal: Deploy Headscale for secure external access Estimated Time: 2-6 hours Prerequisites: Phase 1 completed successfully

Step 2.1: Deploy Headscale

Headscale provides a self-hosted Tailscale control server for secure mesh networking.

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: headscale
Web editor: Paste the following configuration

Docker Compose Configuration

version: '3.8'

services:
  headscale:
    image: headscale/headscale:latest
    container_name: headscale
    restart: unless-stopped
    ports:
      - "8085:8080"  # Web/API port
      - "9090:9090"  # Metrics port (optional)
    volumes:
      - /home/jpmschweitzer/docker-data/headscale/config:/etc/headscale
      - /home/jpmschweitzer/docker-data/headscale/data:/var/lib/headscale
    command: headscale serve
    networks:
      - headscale-network

networks:
  headscale-network:
    driver: bridge

Click Deploy the stack

Functionality Test 3.1A: Verify Headscale Running

# Check Headscale container
docker ps | grep headscale

# Check Headscale logs
docker logs headscale

Expected Results:

Container running
Logs show Headscale server starting
Listening on port 8080 (inside container)

Step 2.1B: Initialize Headscale Configuration

# Create config directory
mkdir -p ~/docker-data/headscale/config
mkdir -p ~/docker-data/headscale/data

# Generate default config
docker exec headscale headscale config generate > ~/docker-data/headscale/config/config.yaml

# Edit config (important settings)
nano ~/docker-data/headscale/config/config.yaml

Key Configuration Changes:

# Change server URL to your server IP or hostname
server_url: http://tower-of-joy:8085

# Or use your local IP
server_url: http://192.168.x.x:8085

# Set database path
db_type: sqlite3
db_path: /var/lib/headscale/db.sqlite

# Enable private key generation
private_key_path: /var/lib/headscale/private.key

Save and restart Headscale:

# Restart Headscale to apply config
docker restart headscale

# Wait a few seconds and check logs
docker logs headscale

Functionality Test 3.1B: Verify Headscale API

# Test API access
curl http://localhost:8085/health

Expected Output: {"healthy":true} or similar health check response

✅ Checkpoint 2.1: Headscale server running and accessible

Step 2.2: Create Headscale User and Pre-Auth Key

Commands

# Create a user (namespace) in Headscale
docker exec headscale headscale users create homelab

# Generate a pre-authentication key (for easy device enrollment)
docker exec headscale headscale preauthkeys create --user homelab --expiration 24h

# Save this key - you'll need it for connecting devices

Expected Output: A pre-auth key like abc123def456...

Functionality Test 3.2: List Users

# Verify user created
docker exec headscale headscale users list

Expected Output: Shows homelab user

✅ Checkpoint 2.2: Headscale user and pre-auth key created

Step 2.3: Connect First Device (This Server)

Install Tailscale client on the host machine and connect to Headscale.

Commands

# Install Tailscale client
curl -fsSL https://tailscale.com/install.sh | sh

# Connect to Headscale server
sudo tailscale up --login-server=http://localhost:8085 --authkey=<your-preauthkey-here>

Replace <your-preauthkey-here> with the key from Step 3.2

Functionality Test 3.3: Verify Connection

# Check Tailscale status
tailscale status

# Get this machine's Tailscale IP
tailscale ip -4

# On Headscale, list nodes
docker exec headscale headscale nodes list

Expected Results:

Tailscale shows connected
Machine has Tailscale IP (e.g., 100.64.0.1)
Headscale nodes list shows this machine

✅ Checkpoint 2.3: Server connected to Headscale network

Step 2.4: Connect Additional Devices

Connect your laptop, phone, or other devices to access services remotely.

For Linux/Mac/Windows Desktop

Install Tailscale: https://tailscale.com/download
Run: tailscale up --login-server=http://tower-of-joy:8085
Browser opens to registration page

Generate pre-auth key on server:

docker exec headscale headscale preauthkeys create --user homelab --expiration 24h

Use the registration URL shown, or use the pre-auth key

For Mobile (iOS/Android)

Install Tailscale app from App Store/Play Store
Open app → Settings → Use custom control URL
Enter: http://<your-server-ip>:8085
Use pre-auth key or registration URL

Functionality Test 3.4: Test Device Connectivity

From the newly connected device:

# Ping the server's Tailscale IP
ping <server-tailscale-ip>

# Try accessing Jellyfin via Tailscale
# Open browser to: http://<server-tailscale-ip>:8096

Expected Results:

Can ping server via Tailscale IP
Can access Jellyfin web interface via Tailscale
Can access Nextcloud via Tailscale

✅ Checkpoint 2.4: Remote devices can access services via Headscale/Tailscale

Step 2.5: Configure Tailscale Serve (Optional)

Tailscale Serve provides HTTPS access to services with automatic certificates.

# Serve Jellyfin on HTTPS
sudo tailscale serve https / http://localhost:8096

# Or serve multiple services on different paths
sudo tailscale serve https /jellyfin http://localhost:8096
sudo tailscale serve https /nextcloud http://localhost:8080

Functionality Test 3.5: Access Services via Tailscale HTTPS

From any device on Tailscale network:

Get server's Tailscale hostname: tailscale status
Access in browser: https://<machine-name>.tailnet-name.ts.net/jellyfin

Expected Results:

HTTPS connection (valid certificate)
Service loads correctly
No certificate warnings

✅ Checkpoint 2.5: HTTPS access configured via Tailscale Serve

Phase 2 Completion Checklist

Headscale server deployed and running
Headscale user created
Pre-auth keys generated
Server connected to Headscale
Additional devices connected
Can access services remotely via Tailscale IPs
(Optional) Tailscale Serve configured for HTTPS

Phase 2 Status: Secure remote access operational

Phase 3: Monitoring & Management

Goal: Add monitoring, dashboards, and management tools Estimated Time: 2-4 hours Prerequisites: Phase 2 completed successfully

Step 3.1: Deploy Uptime Kuma (Service Monitoring)

Uptime Kuma monitors service availability and sends notifications.

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: uptime-kuma
Web editor:

version: '3.8'

services:
  uptime-kuma:
    image: louislam/uptime-kuma:latest
    container_name: uptime-kuma
    restart: unless-stopped
    ports:
      - "3001:3001"
    volumes:
      - /home/jpmschweitzer/docker-data/uptime-kuma:/app/data

Deploy the stack

Functionality Test 4.1: Configure Uptime Kuma

Open browser: http://localhost:3001
Create admin account
Add monitors for:
- Jellyfin: http://localhost:8096
- Nextcloud: http://localhost:8080
- Portainer: http://localhost:9000
- Headscale: http://localhost:8085/health
Set check intervals (e.g., 60 seconds)

Expected Results:

All monitors showing "Up" status
Dashboard displays service uptime

✅ Checkpoint 3.1: Service monitoring operational

Step 3.2: Deploy Netdata (System Monitoring)

Netdata provides real-time system performance monitoring.

Create Stack in Portainer

version: '3.8'

services:
  netdata:
    image: netdata/netdata:latest
    container_name: netdata
    restart: unless-stopped
    hostname: tower-of-joy
    ports:
      - "19999:19999"
    cap_add:
      - SYS_PTRACE
    security_opt:
      - apparmor:unconfined
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - NETDATA_CLAIM_TOKEN=  # Optional: claim token for Netdata Cloud
      - NETDATA_CLAIM_URL=https://app.netdata.cloud

Deploy the stack.

Functionality Test 4.2: Access Netdata Dashboard

Open browser: http://localhost:19999
Explore dashboard sections:
- CPU usage
- RAM usage
- Disk I/O
- Network traffic
- Docker containers
- GPU monitoring (if configured)

Expected Results:

Real-time graphs updating
All system metrics visible
Docker containers monitored

✅ Checkpoint 3.2: System monitoring operational

Step 3.3: Deploy Heimdall (Application Dashboard)

Heimdall provides a unified dashboard with links to all services.

Create Stack in Portainer

version: '3.8'

services:
  heimdall:
    image: linuxserver/heimdall:latest
    container_name: heimdall
    restart: unless-stopped
    ports:
      - "8888:80"
      - "8889:443"
    volumes:
      - /home/jpmschweitzer/docker-data/heimdall:/config
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Amsterdam

Deploy the stack.

Functionality Test 4.3: Configure Heimdall Dashboard

Open browser: http://localhost:8888
Add applications:
- Jellyfin: http://tower-of-joy:8096
- Nextcloud: http://tower-of-joy:8080
- Portainer: http://tower-of-joy:9000
- Uptime Kuma: http://tower-of-joy:3001
- Netdata: http://tower-of-joy:19999
Customize colors and icons

Expected Results:

Unified dashboard showing all services
One-click access to any service
Clean, organized interface

✅ Checkpoint 3.3: Application dashboard operational

Phase 3 Completion Checklist

Uptime Kuma monitoring all services
Netdata showing real-time system metrics
Heimdall dashboard with all service links
All dashboards accessible via Tailscale

Phase 3 Status: Monitoring and management tools operational

Phase 4: Optimization & Security

Goal: Harden security and optimize performance Estimated Time: 2-6 hours Prerequisites: Phase 3 completed successfully

Step 4.1: Configure Automatic Container Updates

Use Watchtower to automatically update containers.

Create Stack in Portainer

version: '3.8'

services:
  watchtower:
    image: containrrr/watchtower:latest
    container_name: watchtower
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WATCHTOWER_CLEANUP=true
      - WATCHTOWER_SCHEDULE=0 0 4 * * *  # 4 AM daily
      - WATCHTOWER_NOTIFICATIONS=shoutrrr
      - WATCHTOWER_NOTIFICATION_URL=  # Optional: notification URL

Deploy the stack.

Functionality Test 5.1: Verify Watchtower

# Check Watchtower logs
docker logs watchtower

# Force a scan
docker exec watchtower watchtower --run-once

Expected Results:

Watchtower checks all containers
Updates available containers (if any)
Logs show scan results

✅ Checkpoint 4.1: Automatic updates configured

Step 4.2: Configure Docker Log Rotation

Prevent logs from consuming disk space.

Commands

# Create Docker daemon config
sudo nano /etc/docker/daemon.json

Add the following:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Save and restart Docker:

sudo systemctl restart docker

Functionality Test 5.2: Verify Log Configuration

# Check Docker info
docker info | grep -A 5 "Logging Driver"

# Restart a container to apply new log settings
docker restart jellyfin

# Check log file sizes
sudo du -sh /var/lib/docker/containers/*/*-json.log

Expected Results:

Log driver shows json-file with size limits
Log files stay under 10MB

✅ Checkpoint 4.2: Log rotation configured

Step 4.3: Configure Firewall Rules

Set up UFW (Uncomplicated Firewall) to secure the server.

Commands

# Install UFW if not installed
sudo apt-get install -y ufw

# Allow SSH (IMPORTANT - do this first!)
sudo ufw allow 22/tcp

# Allow Tailscale
sudo ufw allow 41641/udp

# Allow Portainer
sudo ufw allow 9000/tcp

# Allow Jellyfin (only if you want LAN access without Tailscale)
sudo ufw allow 8096/tcp

# Allow Nextcloud (only if you want LAN access without Tailscale)
sudo ufw allow 8080/tcp

# Allow Samba (only if you want network shares)
sudo ufw allow 139/tcp
sudo ufw allow 445/tcp

# Enable firewall
sudo ufw enable

# Check status
sudo ufw status verbose

Functionality Test 5.3: Verify Firewall

# Check UFW status
sudo ufw status numbered

# Test accessing services from another device
# Should work: Portainer, Jellyfin, Nextcloud (if allowed)
# Should work via Tailscale regardless of firewall

Expected Results:

UFW enabled and active
Allowed services accessible
Tailscale connections work

✅ Checkpoint 4.3: Firewall configured and active

Step 4.4: Set Up Backup Strategy

Configure backups for critical data.

Deploy Duplicati (Backup Solution)

version: '3.8'

services:
  duplicati:
    image: linuxserver/duplicati:latest
    container_name: duplicati
    restart: unless-stopped
    ports:
      - "8200:8200"
    volumes:
      - /home/jpmschweitzer/docker-data/duplicati/config:/config
      - /home/jpmschweitzer/docker-data:/source/docker-data:ro
      - /path/to/backup/destination:/backups
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Amsterdam

Deploy and configure:

Access Duplicati: http://localhost:8200
Create backup job:
- Source: /source/docker-data
- Destination: Local folder, external drive, or cloud (S3, Google Drive, etc.)
- Schedule: Daily at 3 AM
- Retention: Keep 30 days
Run test backup

Functionality Test 5.4: Verify Backup

# Check Duplicati logs
docker logs duplicati

# Verify backup files created
ls -lh /path/to/backup/destination

Expected Results:

Backup job completes successfully
Backup files created
Can restore test file

✅ Checkpoint 4.4: Backup system operational

Step 4.5: Optimize Nextcloud Performance

Apply recommended Nextcloud optimizations.

Commands

# Install missing indices
docker exec -u www-data nextcloud php occ db:add-missing-indices

# Convert to bigint (if needed)
docker exec -u www-data nextcloud php occ db:convert-filecache-bigint

# Configure memory cache
docker exec -u www-data nextcloud php occ config:system:set memcache.local --value="\\OC\\Memcache\\Redis"
docker exec -u www-data nextcloud php occ config:system:set memcache.locking --value="\\OC\\Memcache\\Redis"
docker exec -u www-data nextcloud php occ config:system:set redis host --value="nextcloud-redis"
docker exec -u www-data nextcloud php occ config:system:set redis port --value=6379

# Set up cron instead of AJAX
docker exec -u www-data nextcloud php occ background:cron

# Add to crontab (on host)
echo "*/5 * * * * docker exec -u www-data nextcloud php cron.php" | sudo tee -a /etc/crontab

Functionality Test 5.5: Verify Optimizations

Access Nextcloud: Settings → Overview
Check for warnings/errors
Upload/download test files to check performance

Expected Results:

No warnings in Overview
Background job set to Cron
Improved performance

✅ Checkpoint 4.5: Nextcloud optimized

Phase 4 Completion Checklist

Watchtower configured for automatic updates
Docker log rotation enabled
Firewall rules configured
Backup solution deployed and tested
Nextcloud optimizations applied
Security best practices implemented

Phase 4 Status: System secured and optimized

Post-Implementation Verification

Complete System Test

Run these tests to verify the entire system:

1. Service Availability Test

# Check all containers running
docker ps

# Expected: All containers showing "Up" status

2. GPU Transcoding Test

Access Jellyfin
Start playing video that requires transcoding
Monitor GPU: watch nvidia-smi
Verify Video Engine usage

3. Remote Access Test

From device on Tailscale network
Access Jellyfin via Tailscale IP
Stream video remotely
Verify smooth playback

From another computer
Connect to \\tower-of-joy\Media
Copy file to share
Verify successful transfer

5. Nextcloud Sync Test

Install Nextcloud desktop client
Connect via Tailscale IP
Sync test folder
Verify files sync correctly

6. Monitoring Test

Access Uptime Kuma
Verify all services showing "Up"
Access Netdata
Verify system metrics displaying

7. Backup Test

Access Duplicati
Verify last backup successful
Test file restore
Confirm restore works

Final Checklist

All containers running
GPU transcoding working
Remote access via Tailscale functional
File sharing accessible
Nextcloud syncing
Monitoring operational
Backups running
Firewall active
Log rotation enabled
Automatic updates configured

Troubleshooting Guide

Common Issues and Solutions

Issue: GPU Not Detected in Container

Symptoms:

nvidia-smi fails in container
Jellyfin shows no hardware encoding options

Solutions:

# Verify NVIDIA runtime
docker info | grep -i runtime

# Restart Docker
sudo systemctl restart docker

# Verify GPU access
docker run --rm --gpus all nvidia/cuda:11.4.0-base-ubuntu20.04 nvidia-smi

Issue: Container Won't Start

Symptoms:

Container status: Exited
Service not accessible

Solutions:

# Check logs
docker logs <container-name>

# Check for port conflicts
sudo netstat -tulpn | grep <port>

# Verify volume permissions
ls -la ~/docker-data/<service>

Issue: Out of Disk Space

Symptoms:

Containers failing to start
df shows high usage

Solutions:

# Clean up unused containers/images
docker system prune -a

# Check Docker disk usage
docker system df

# Move media to external drive
# Update volume mounts in docker-compose

Issue: Can't Access via Tailscale

Symptoms:

Can't ping Tailscale IP
Services not accessible remotely

Solutions:

# Check Tailscale status
tailscale status

# Restart Tailscale
sudo systemctl restart tailscaled

# Check Headscale nodes
docker exec headscale headscale nodes list

# Verify firewall not blocking
sudo ufw status

Issue: Jellyfin Transcoding Using CPU Instead of GPU

Symptoms:

High CPU usage during playback
Dashboard shows software transcoding

Solutions:

Jellyfin Dashboard → Playback → Transcoding
Verify NVIDIA NVENC selected
Check all hardware decoding options enabled
Restart Jellyfin container
Test with compatible video format (H.264)

Issue: Nextcloud Slow Performance

Symptoms:

Slow file uploads
Sluggish web interface

Solutions:

# Verify Redis connected
docker logs nextcloud | grep -i redis

# Check cron job active
docker exec -u www-data nextcloud php occ config:system:get maintenance_window_start

# Add missing indices
docker exec -u www-data nextcloud php occ db:add-missing-indices

# Increase memory limits
docker exec -u www-data nextcloud php occ config:system:set memory_limit --value="512M"

Maintenance Tasks

Daily

Check Uptime Kuma dashboard for service status
Monitor disk space: df -h

Weekly

Review container logs for errors
Check backup completion in Duplicati
Review Netdata for unusual activity

Monthly

Update containers (if not using Watchtower)
Review and clean old Docker images: docker image prune -a
Test backup restoration
Review firewall logs: sudo ufw status verbose

Quarterly

Update system packages: sudo apt update && sudo apt upgrade
Review container performance and resource usage
Audit Tailscale/Headscale connected devices
Review and update documentation

Performance Optimization Tips

For Limited Storage (481GB):

Use external drives for media
- Mount external HDD/SSD
- Update Jellyfin volume mounts
- Keeps system drive lean
Enable aggressive log rotation
- Current config: 10MB × 3 files per container
- Can reduce to 5MB × 2 files if needed
Regular cleanup
- Schedule weekly: docker system prune -a --volumes --filter "until=168h"
- Removes unused images/volumes older than 7 days
Monitor disk usage
- Add Netdata disk space alerts
- Set threshold at 85% usage

For Better Performance:

Optimize container resource limits
- Add resource limits in docker-compose
- Prevent any container from consuming all resources
Use SSD for Docker data
- If possible, move /var/lib/docker to SSD
- Significantly improves container performance
Enable Docker experimental features
- Better caching
- Faster builds
Tune Jellyfin transcoding
- Limit concurrent transcodes
- Prefer direct play when possible
- Use hardware-compatible codecs (H.264/H.265)

Expansion Ideas

Future Services to Consider:

AI/ML Integration:

Open WebUI - Web interface for Ollama (like ChatGPT UI)
Stable Diffusion WebUI - GPU-accelerated image generation
ComfyUI - Node-based interface for AI workflows
Whisper - Speech-to-text transcription service
Text Generation WebUI - Alternative LLM interface

Productivity & Media: 6. Pi-hole - Network-wide ad blocking 7. Vaultwarden - Self-hosted password manager 8. Home Assistant - Home automation platform 9. Gitea - Self-hosted Git service 10. Paperless-ngx - Document management 11. Calibre-Web - Ebook library 12. Audiobookshelf - Audiobook server 13. Photoprism - Photo management 14. Grafana - Advanced metrics visualization

Hardware Upgrades:

UPS - Uninterruptible power supply for clean shutdowns and graceful shutdown on power loss
Network upgrade - 2.5GbE or 10GbE NIC for faster media streaming and transfers
More RAM - 32GB for running more simultaneous containers
Additional storage - Optional: More HDDs if 3.7TB isn't enough (current setup has plenty)
Backup drive - External USB drive for offsite backups

Documentation & Resources

Container Configuration Files

All Docker Compose files are managed in Portainer Stacks:

Portainer UI: http://tower-of-joy:9000
Stacks location: Portainer → Stacks
Export stacks regularly for backup

Important Paths

Storage Strategy:

SSD (/dev/sda): Configs, databases, cache (performance-critical)
HDD (/dev/sdb): User content, media, bulk storage (capacity-critical)

Service	Config Path (SSD)	Data Path (HDD)
Portainer	Docker volume: `portainer_data`	N/A
Nginx Proxy Manager	`~/docker-data/nginx-proxy-manager/data`	N/A
NPM SSL Certs	`~/docker-data/nginx-proxy-manager/letsencrypt`	N/A
Ollama Models	`~/docker-data/ollama/models`	Alternative: `/mnt/media/ollama/models`
AMP (Game Servers)	`/home/amp/.ampdata/`	`/mnt/media/game-servers/amp-instances/`
AMP Backups	N/A	`/mnt/media/backups/amp/`
Headscale	`~/docker-data/headscale/config`	`~/docker-data/headscale/data`
Jellyfin	`~/docker-data/jellyfin/config`	`/mnt/media/jellyfin/`
Nextcloud	`~/docker-data/nextcloud/config`	`/mnt/media/nextcloud/data/`
Nextcloud DB	`~/docker-data/nextcloud/db`	N/A (DB on SSD)
Samba	`~/docker-data/samba`	`/mnt/media/` (shares)
Backups	N/A	`/mnt/media/backups/`
Future Game Servers	`~/docker-data/<game>/config`	`/mnt/media/game-servers/<game>/`
vLLM Models (optional)	`~/docker-data/vllm/models`	Alternative: `/mnt/media/vllm/models`

Network Ports

Service	Port	Access	Notes
Nginx Proxy Manager	8000	HTTP	Unified web interface - Phase 1
Nginx Proxy Manager	80	HTTP	Reverse proxy traffic - Phase 1
Nginx Proxy Manager	443	HTTPS	Reverse proxy SSL - Phase 1
AMP (Game Servers)	8081	HTTP	Reconfigured from 8080 - Phase 1
Portainer	8080	HTTP	Container management - Phase 1
Portainer	8443	HTTPS	Portainer HTTPS - Phase 1
Ollama	11434	HTTP	ML model API - Phase 1
Headscale	8085	HTTP	SDN control server - Phase 2
Headscale	41641	UDP	Tailscale mesh - Phase 2
Uptime Kuma	3001	HTTP	Service monitoring - Phase 3
Netdata	19999	HTTP	System monitoring - Phase 3
Heimdall	8888	HTTP	Application dashboard - Phase 3
Duplicati	8200	HTTP	Backup management - Phase 4
Jellyfin	8096	HTTP	Media server - Backlog
Nextcloud	8082	HTTP	Cloud storage - Backlog
Samba	445	SMB	File sharing - Backlog
Minecraft Servers	25565+	TCP/UDP	Managed by AMP
vLLM (optional)	8001	HTTP	ML model API (OpenAI-compatible)

Support Resources

Infrastructure:

Portainer Docs: https://docs.portainer.io/
Nginx Proxy Manager: https://nginxproxymanager.com/guide/
Docker Docs: https://docs.docker.com/
NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/

ML/AI:

Ollama Docs: https://github.com/ollama/ollama
vLLM Docs: https://docs.vllm.ai/
Ollama Library: https://ollama.com/library (available models)

Networking:

Headscale Docs: https://headscale.net/
Tailscale KB: https://tailscale.com/kb/

Applications:

Jellyfin Docs: https://jellyfin.org/docs/
Nextcloud Admin Manual: https://docs.nextcloud.com/

Implementation Complete!

🎉 Congratulations! Your home server is now fully operational with:

Phase 1 - Foundation:

✅ Web-based container management (Portainer)
✅ Unified reverse proxy interface (Nginx Proxy Manager)
✅ GPU-accelerated ML models (Ollama/vLLM)
✅ Dual-disk storage strategy (SSD + 4TB HDD)
✅ AMP game server integration

Phase 2 - Networking:

✅ Secure remote access (Headscale/Tailscale)
✅ SDN mesh networking

Phase 3 - Monitoring:

✅ Service monitoring (Uptime Kuma)
✅ System monitoring (Netdata)
✅ Unified dashboard (Heimdall)

Phase 4 - Optimization:

✅ Automatic updates (Watchtower)
✅ Backup solution (Duplicati)
✅ Security hardening (UFW)
✅ Log rotation

Backlog - Applications:

✅ Media server with GPU transcoding (Jellyfin)
✅ Cloud storage with external access (Nextcloud)
✅ Network file sharing (Samba)

Total Services Running: 13+ containers Total Setup Time: 8-20 hours (including ML setup) System Status: Production-ready with ML capabilities

Next Steps:

Customize services to your preferences
Add media to Jellyfin libraries
Upload files to Nextcloud
Connect all your devices to Tailscale
Set up mobile apps
Explore additional services from expansion ideas
Enjoy your self-hosted home server!

Implementation Plan Version 1.0 - Last Updated: 2025-11-11

Phase Backlog: Application Deployment

Status: Deferred until base infrastructure (Phases 1-4) is complete

The following application deployments will be implemented after the base infrastructure is solid. This includes deploying Jellyfin media server with GPU transcoding, Nextcloud cloud storage, and Samba file sharing.

Phase 2: Core Services Deployment

Goal: Deploy Jellyfin, Nextcloud, and file sharing services Estimated Time: 2-4 hours Prerequisites: Base infrastructure completed (Phases 1-4)

Note: This phase is in the backlog and will be implemented after the base infrastructure is solid.

Step 2.1: Prepare Directory Structure

Create directories for container configurations and user content.

Storage Strategy Reminder:

Container configs → ~/docker-data/* (SSD - fast access)
User content → /mnt/media/* (4TB HDD - bulk storage)

Commands

# Create base directory for container configs on SSD
mkdir -p ~/docker-data

# Create Jellyfin config directories (SSD for configs, HDD for media)
mkdir -p ~/docker-data/jellyfin/config
mkdir -p ~/docker-data/jellyfin/cache

# Create Nextcloud config directory (SSD)
mkdir -p ~/docker-data/nextcloud/config
mkdir -p ~/docker-data/nextcloud/db

# Create Samba config directory (SSD)
mkdir -p ~/docker-data/samba/config

# Set permissions on SSD directories
sudo chown -R $USER:$USER ~/docker-data

# Verify media drive directories from Phase 1
# These should already exist from Step 1.4
ls -la /mnt/media

# If needed, create additional media subdirectories
mkdir -p /mnt/media/jellyfin/movies
mkdir -p /mnt/media/jellyfin/tv
mkdir -p /mnt/media/jellyfin/music
mkdir -p /mnt/media/nextcloud/data

Functionality Test 2.1: Verify Directory Structure

# Check SSD directories
tree -L 3 ~/docker-data

# Check HDD directories
tree -L 2 /mnt/media

# Verify permissions
ls -la ~/docker-data
ls -la /mnt/media

# Verify media drive is mounted
df -h /mnt/media

Expected Results:

SSD config directories created in ~/docker-data/
HDD content directories exist in /mnt/media/
All directories owned by current user
Media drive showing 3.7TB available
Write permissions enabled on both locations

✅ Checkpoint 2.1: Directory structure prepared with proper SSD/HDD separation

Step 2.2: Deploy Jellyfin Media Server with GPU

Deploy Jellyfin with NVIDIA GPU support for hardware transcoding.

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: jellyfin
Web editor: Paste the following Docker Compose configuration

Docker Compose Configuration

version: '3.8'

services:
  jellyfin:
    image: jellyfin/jellyfin:latest
    container_name: jellyfin
    user: 1000:1000  # Replace with your UID:GID (run `id` to find)
    network_mode: bridge
    ports:
      - 8096:8096      # HTTP web interface
      - 8920:8920      # HTTPS web interface (optional)
      - 7359:7359/udp  # Auto-discovery
      - 1900:1900/udp  # DLNA
    volumes:
      - /home/jpmschweitzer/docker-data/jellyfin/config:/config  # Config on SSD
      - /home/jpmschweitzer/docker-data/jellyfin/cache:/cache    # Cache on SSD
      - /mnt/media/jellyfin:/media:ro  # Media on 4TB HDD (read-only)
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu, video, compute, utility]

Click Deploy the stack

Functionality Test 2.2A: Verify Jellyfin Container Running

# Check Jellyfin container status
docker ps | grep jellyfin

# Check Jellyfin logs
docker logs jellyfin

# Verify GPU access inside container
docker exec jellyfin nvidia-smi

Expected Results:

Container status: "Up"
Logs show Jellyfin starting without errors
nvidia-smi inside container shows RTX 2080 Ti

Functionality Test 2.2B: Access Jellyfin Web Interface

Open browser: http://localhost:8096 or http://tower-of-joy:8096
Complete initial setup wizard:
- Choose language
- Create admin account
- Add media libraries (point to /media/movies, /media/tv, etc.)
Navigate to: Dashboard → Playback → Transcoding
Enable hardware acceleration:
- Hardware acceleration: NVIDIA NVENC
- Enable hardware decoding for: (select all applicable)
- Enable hardware encoding
- Enable NVENC
- Encoding preset: Auto or High Quality

Functionality Test 2.2C: Test GPU Transcoding

Upload a test video (or add existing media)
Start playback in browser
In Dashboard → Activity, watch transcoding session
On terminal, monitor GPU usage:

# Watch GPU utilization during transcoding
watch -n 1 nvidia-smi

Expected Results:

Video plays smoothly
Dashboard shows "(hw)" next to codec name during transcode
nvidia-smi shows GPU utilization (Video Engine usage)
CPU usage remains low during transcoding

✅ Checkpoint 2.2: Jellyfin deployed with working GPU hardware transcoding

Step 2.3: Deploy Nextcloud with Database

Deploy Nextcloud with MariaDB and Redis for optimal performance.

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: nextcloud
Web editor: Paste the following configuration

Docker Compose Configuration

version: '3.8'

services:
  nextcloud-db:
    image: mariadb:10.11
    container_name: nextcloud-db
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
    restart: unless-stopped
    volumes:
      - /home/jpmschweitzer/docker-data/nextcloud/db:/var/lib/mysql  # Database on SSD (performance-critical)
    environment:
      - MYSQL_ROOT_PASSWORD=changeme_root_password  # CHANGE THIS!
      - MYSQL_PASSWORD=changeme_nc_password         # CHANGE THIS!
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
    networks:
      - nextcloud-network

  nextcloud-redis:
    image: redis:alpine
    container_name: nextcloud-redis
    restart: unless-stopped
    networks:
      - nextcloud-network

  nextcloud:
    image: nextcloud:stable
    container_name: nextcloud
    restart: unless-stopped
    ports:
      - 8080:80
    volumes:
      - /home/jpmschweitzer/docker-data/nextcloud/config:/var/www/html  # App config on SSD
      - /mnt/media/nextcloud/data:/var/www/html/data  # User data on 4TB HDD
    environment:
      - MYSQL_HOST=nextcloud-db
      - MYSQL_PASSWORD=changeme_nc_password  # Match DB password
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - REDIS_HOST=nextcloud-redis
    depends_on:
      - nextcloud-db
      - nextcloud-redis
    networks:
      - nextcloud-network

networks:
  nextcloud-network:
    driver: bridge

IMPORTANT: Change the database passwords before deploying!

Click Deploy the stack

Functionality Test 2.3A: Verify Nextcloud Stack Running

# Check all Nextcloud containers
docker ps | grep nextcloud

# Check Nextcloud logs
docker logs nextcloud

# Check database logs
docker logs nextcloud-db

Expected Results:

All three containers running (nextcloud, nextcloud-db, nextcloud-redis)
No error messages in logs
Database initialized successfully

Functionality Test 2.3B: Complete Nextcloud Setup

Open browser: http://localhost:8080 or http://tower-of-joy:8080
First-time setup page appears
Create admin account:
- Username: (choose admin username)
- Password: (strong password)
Storage & database section:
- Data folder: Leave default /var/www/html/data
- Database: MySQL/MariaDB
- Database user: nextcloud
- Database password: (the one you set in compose file)
- Database name: nextcloud
- Database host: nextcloud-db
Click Install
Wait for installation (may take a few minutes)

Functionality Test 2.3C: Verify Nextcloud Functionality

After installation, log in to Nextcloud
Test file upload: Upload a test file
Test file download: Download the test file
Navigate to: Settings → Overview
Check for warnings/errors

Expected Results:

Can upload files successfully
Can download files
No critical errors in Overview page
Redis caching working (check Overview for confirmation)

Functionality Test 2.3D: Configure Trusted Domains

# Add tower-of-joy to trusted domains
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=tower-of-joy

# If using IP address, add it too
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 2 --value=192.168.x.x

Test: Access Nextcloud using hostname/IP to verify trusted domains

✅ Checkpoint 2.3: Nextcloud deployed and functional with database and Redis

Step 2.4: Deploy Samba File Server

Deploy Samba for Windows/Mac network file sharing.

Create Stack in Portainer

Navigate to: Stacks → Add Stack
Name: samba
Web editor: Paste the following configuration

Docker Compose Configuration

version: '3.8'

services:
  samba:
    image: dperson/samba
    container_name: samba
    restart: unless-stopped
    ports:
      - "139:139"
      - "445:445"
    environment:
      - TZ=Europe/Amsterdam
      - USERID=1000  # Your user ID (run `id -u`)
      - GROUPID=1000 # Your group ID (run `id -g`)
    volumes:
      - /home/jpmschweitzer/docker-data/samba:/share/config  # Config on SSD
      - /mnt/media/jellyfin:/share/media  # Media from 4TB HDD
      - /mnt/media/nextcloud/data:/share/nextcloud:ro  # Nextcloud data from 4TB HDD (read-only)
      - /mnt/media/downloads:/share/downloads  # Downloads folder from 4TB HDD
    command: '-s "Media;/share/media;yes;no;yes;all" -s "Nextcloud;/share/nextcloud;yes;no;yes;all" -s "Downloads;/share/downloads;yes;no;no;all" -u "jpmschweitzer;changeme_samba_password" -p'

Note: Replace changeme_samba_password with a strong password

Click Deploy the stack

Functionality Test 2.4A: Verify Samba Running

# Check Samba container
docker ps | grep samba

# Check Samba logs
docker logs samba

Expected Results:

Container running
SMB daemon started
Shares configured

From Windows:

Open File Explorer
Type in address bar: \\tower-of-joy\Media
Enter credentials: jpmschweitzer / (your password)
Verify you can browse files

From Mac:

Finder → Go → Connect to Server
Enter: smb://tower-of-joy/Media
Enter credentials
Verify access

From Linux:

# Install smbclient if needed
sudo apt-get install smbclient

# Test connection
smbclient -L tower-of-joy -U jpmschweitzer

Expected Results:

Can connect to shares
Can browse files
Can read/write (depending on share permissions)

✅ Checkpoint 2.4: Samba file sharing working from network devices

Phase 2 Completion Checklist

Directory structure created
Jellyfin deployed and accessible at port 8096
Jellyfin GPU transcoding working
Nextcloud deployed and accessible at port 8080
Nextcloud database and Redis running
Nextcloud file upload/download working
Samba file server deployed
Network shares accessible from other devices

Phase 2 Status: Core services operational

For questions or issues, refer to troubleshooting guide or consult service documentation

66 KiB Raw Blame History Unescape Escape

Container Platform Implementation Plan

Overview

Dual-Disk Storage Strategy

Pre-Implementation Checklist

Pre-Flight System Check

Phase 1: Foundation Setup

Step 1.1: Install NVIDIA Container Toolkit

Commands

Functionality Test 1.1: Verify GPU Access in Docker

Step 1.2: Deploy Portainer

Commands

Functionality Test 1.2: Access Portainer Web UI

Step 1.3: Configure Portainer for GPU Support

Steps

Functionality Test 1.3: Verify GPU Visibility in Portainer

Step 1.4: Mount the 4TB Media Drive

System Storage Strategy

Commands

Add to /etc/fstab for Automatic Mounting

Set Permissions

Create Directory Structure for Container Data

Functionality Test 1.4: Verify Media Drive Setup

Step 1.5: Configure AMP Integration and Game Server Storage

AMP + Portainer Compatibility Overview

Reconfigure AMP to Port 8081

Add User to Docker Group

Functionality Test 1.5A: Verify Docker Access

Configure AMP to Use Media Drive for Game Data

Configure AMP Backups to Media Drive

Create Directory Structure for Future Game Servers

Functionality Test 1.5B: Verify AMP Integration

Best Practices for AMP + Portainer Coexistence

Update Phase 1 Completion Checklist

Step 1.6: Deploy Nginx Proxy Manager (Reverse Proxy)

Create Stack in Portainer

Docker Compose Configuration

Create Directory Structure

Functionality Test 1.6A: Access NPM Web Interface

Functionality Test 1.6B: Configure Proxy Hosts

Configure Services Through NPM (Optional)

Functionality Test 1.6C: Verify NPM Working

Step 1.7: Prepare ML Model Infrastructure (Ollama/vLLM)

Option A: Deploy Ollama (Recommended)

Functionality Test 1.7A: Verify Ollama GPU Access

Functionality Test 1.7B: Pull and Test a Model

Test Ollama API

Model Storage Considerations

Option B: vLLM (Advanced, Optional)

Configure NPM Proxy for Ollama (Optional)

Functionality Test 1.7C: Performance Validation

Storage Management for Models

Phase 1 Completion Checklist

Phase 2: Networking & External Access

Step 2.1: Deploy Headscale

Create Stack in Portainer

Docker Compose Configuration

Functionality Test 3.1A: Verify Headscale Running

Step 2.1B: Initialize Headscale Configuration

Functionality Test 3.1B: Verify Headscale API

Step 2.2: Create Headscale User and Pre-Auth Key

Commands

Functionality Test 3.2: List Users

Step 2.3: Connect First Device (This Server)

Commands

Functionality Test 3.3: Verify Connection

Step 2.4: Connect Additional Devices

For Linux/Mac/Windows Desktop

For Mobile (iOS/Android)

Functionality Test 3.4: Test Device Connectivity

Step 2.5: Configure Tailscale Serve (Optional)

Functionality Test 3.5: Access Services via Tailscale HTTPS

Phase 2 Completion Checklist

Phase 3: Monitoring & Management

Step 3.1: Deploy Uptime Kuma (Service Monitoring)

Create Stack in Portainer

Functionality Test 4.1: Configure Uptime Kuma

Step 3.2: Deploy Netdata (System Monitoring)

Create Stack in Portainer

Functionality Test 4.2: Access Netdata Dashboard

66 KiB

Raw Blame History