portainer-core/docs/mesh-access-strategy.md

# Mesh Network Access Strategy (Option B)

> Hybrid approach: Public access for media/files, VPN-only for admin tools
> All VPN access uses Headscale mesh IPs (10.99.0.x)
> Created: 2025-11-11

## Core Principles

**RULE: All external/public access MUST route through NPM proxy**

**Why this rule is mandatory:**
- ✅ **Let's Encrypt SSL**: Automatic certificate management in one place
- ✅ **Unified logging**: All external access logged in NPM
- ✅ **Security headers**: Consistent security policy (HSTS, CSP, etc.)
- ✅ **Access control**: Single point to manage public access
- ✅ **DDoS protection**: Can add Cloudflare/rate limiting at proxy level
- ✅ **No port sprawl**: Only ports 80/443 exposed externally

**Access Patterns:**
- **Internal/VPN access**: Direct mesh IPs → `http://10.99.0.1:8096`
- **External/Public access**: Through NPM → `https://media.schweitz.net` → NPM forwards to mesh IP
- **NEVER**: Direct port forwarding to services (except NPM and Headscale)

## Architecture Overview

```
┌─────────────────────────────────────────────────────────┐
│                    Internet Users                        │
└──────────────────┬──────────────────┬───────────────────┘
                   │                  │
        ┌──────────▼────────┐  ┌─────▼──────────────────┐
        │   Public Access   │  │   Headscale VPN        │
        │   (Port 443)      │  │   (Port 8085)          │
        └──────────┬────────┘  └─────┬──────────────────┘
                   │                  │
                   │           ┌──────▼──────────────────┐
                   │           │  VPN Mesh Network       │
                   │           │  10.99.0.0/16           │
                   │           │                         │
                   │           │  tower-of-joy: 10.99.0.1│
                   │           │  laptop: 10.99.0.2      │
                   │           │  phone: 10.99.0.3       │
                   │           └──────┬──────────────────┘
                   │                  │
         ┌─────────▼──────────────────▼─────────────────┐
         │        tower-of-joy Services                  │
         │  ┌────────────────────────────────────────┐  │
         │  │ Public Services (via NPM)              │  │
         │  │ - Jellyfin (media)                     │  │
         │  │ - Nextcloud (files)                    │  │
         │  │ - Organizr (optional)                  │  │
         │  └────────────────────────────────────────┘  │
         │  ┌────────────────────────────────────────┐  │
         │  │ VPN-Only Services (mesh IPs)           │  │
         │  │ - Portainer: 10.99.0.1:8001            │  │
         │  │ - Netdata: 10.99.0.1:19999             │  │
         │  │ - Uptime Kuma: 10.99.0.1:3001          │  │
         │  │ - NPM Admin: 10.99.0.1:81              │  │
         │  │ - Heimdall: 10.99.0.1:8888             │  │
         │  └────────────────────────────────────────┘  │
         └───────────────────────────────────────────────┘
```

## Service Access Matrix

| Service | Mesh IP Access | Public Access | Use Case |
|---------|---------------|---------------|----------|
| **Organizr** | ✅ http://10.99.0.1:9999 | ✅ https://home.schweitz.net | Unified dashboard |
| **Portainer** | ✅ http://10.99.0.1:8001 | ❌ VPN ONLY | Container management |
| **Netdata** | ✅ http://10.99.0.1:19999 | ❌ VPN ONLY | System metrics |
| **Uptime Kuma** | ✅ http://10.99.0.1:3001 | ❌ VPN ONLY | Service monitoring |
| **Heimdall** | ✅ http://10.99.0.1:8888 | ❌ VPN ONLY | Alternative dashboard |
| **NPM Admin** | ✅ http://10.99.0.1:81 | ❌ NEVER | Proxy config |
| **Headscale** | ✅ http://10.99.0.1:8085 | ✅ Public :8085 | VPN control plane |
| **Jellyfin** | ✅ http://10.99.0.1:8096 | ✅ https://media.schweitz.net | Media streaming |
| **Nextcloud** | ✅ http://10.99.0.1:8082 | ✅ https://cloud.schweitz.net | File storage |
| **Ollama** | ✅ http://10.99.0.1:11434 | ❌ VPN ONLY | ML API |

**Note:** Mesh IP `10.99.0.1` is assumed for tower-of-joy. Actual IP will be assigned by Headscale.

## Implementation Steps

### Phase 1: Connect tower-of-joy to Headscale

**First, get the server onto its own VPN mesh:**

```bash
# Install Tailscale client on tower-of-joy
curl -fsSL https://tailscale.com/install.sh | sh

# Connect to your Headscale server
sudo tailscale up --login-server=http://192.168.86.149:8085 \
  --authkey=<your-preauth-key> \
  --hostname=tower-of-joy

# Verify connection
tailscale status
# Should show: tower-of-joy with mesh IP (e.g., 10.99.0.1)

# Get the mesh IP assigned to tower-of-joy
tailscale ip -4
# Note this IP - you'll use it in Organizr configuration
```

**Verify from Headscale:**
```bash
# List all nodes in mesh
docker exec headscale headscale nodes list

# Should show:
# ID | Name           | IP         | Last Seen
# 1  | tower-of-joy   | 10.99.0.1  | now
```

### Phase 2: Deploy Organizr

```bash
# Create config directory
mkdir -p ~/docker-data/organizr

# Deploy Organizr
docker compose -f stacks/organizr.yml up -d

# Verify running
docker ps | grep organizr
```

### Phase 3: Configure Organizr with Mesh IPs

**Access Organizr setup:**
- From local network: http://192.168.86.149:9999
- From VPN: http://10.99.0.1:9999

**Complete setup wizard:**
1. Choose installation type: "Personal"
2. Create admin user
3. Set timezone: Europe/Amsterdam
4. Complete setup

**Add tabs using mesh IPs:**

Navigate to: Settings → Tab Editor

#### Tab: Portainer
```
Tab Name: Portainer
Tab URL: http://10.99.0.1:8001
Tab Type: iframe
Category: Admin
Icon: docker
Enabled: Yes
Active: Yes
```

#### Tab: Netdata
```
Tab Name: Netdata
Tab URL: http://10.99.0.1:19999
Tab Type: iframe
Category: Monitoring
Icon: line-chart
Enabled: Yes
```

#### Tab: Uptime Kuma
```
Tab Name: Uptime
Tab URL: http://10.99.0.1:3001
Tab Type: iframe
Category: Monitoring
Icon: heartbeat
Enabled: Yes
```

#### Tab: Heimdall
```
Tab Name: Dashboard
Tab URL: http://10.99.0.1:8888
Tab Type: iframe
Category: Home
Icon: th
Enabled: Yes
```

#### Tab: Jellyfin (when deployed)
```
Tab Name: Media
Tab URL: http://10.99.0.1:8096
Tab Type: iframe
Category: Apps
Icon: film
Enabled: Yes
```

#### Tab: Nextcloud (when deployed)
```
Tab Name: Cloud
Tab URL: http://10.99.0.1:8082
Tab Type: iframe
Category: Apps
Icon: cloud
Enabled: Yes
```

### Phase 4: Configure NPM for Public Access

**Only expose these services publicly:**

Access NPM admin: http://10.99.0.1:81 (via VPN)

#### 1. Organizr (Public Dashboard)
```
Proxy Host Configuration:
Domain Names: home.schweitz.net
Scheme: http
Forward Hostname/IP: 10.99.0.1
Forward Port: 9999
✓ Block Common Exploits
✓ Websockets Support

SSL Tab:
✓ Force SSL
✓ HTTP/2 Support
✓ HSTS Enabled
Request New SSL Certificate (Let's Encrypt)
```

#### 2. Jellyfin (Public Media)
```
Proxy Host Configuration:
Domain Names: media.schweitz.net
Scheme: http
Forward Hostname/IP: 10.99.0.1
Forward Port: 8096
✓ Block Common Exploits
✓ Websockets Support

SSL Tab:
✓ Force SSL
✓ HTTP/2 Support
Request New SSL Certificate (Let's Encrypt)
```

#### 3. Nextcloud (Public Files)
```
Proxy Host Configuration:
Domain Names: cloud.schweitz.net
Scheme: http
Forward Hostname/IP: 10.99.0.1
Forward Port: 8082
✓ Block Common Exploits
✓ Websockets Support

SSL Tab:
✓ Force SSL
✓ HTTP/2 Support
Request New SSL Certificate (Let's Encrypt)

Custom Nginx Configuration:
client_max_body_size 10G;  # Allow large file uploads
proxy_request_buffering off;
```

### Phase 5: DNS Configuration

**Required DNS records:**
```
home.schweitz.net   A  <your-public-ip>
media.schweitz.net  A  <your-public-ip>
cloud.schweitz.net  A  <your-public-ip>
```

**Or use wildcard:**
```
*.schweitz.net      A  <your-public-ip>
```

### Phase 6: Router Port Forwarding

**CRITICAL: ONLY these ports exposed to internet:**
```
External Port 443  → 192.168.86.149:443  (NPM HTTPS - ALL public services)
External Port 80   → 192.168.86.149:80   (NPM HTTP redirect to HTTPS)
External Port 8085 → 192.168.86.149:8085 (Headscale VPN control plane)
```

**⚠️ NEVER forward service ports directly!**
- ❌ DO NOT forward port 8096 (Jellyfin)
- ❌ DO NOT forward port 8082 (Nextcloud)
- ❌ DO NOT forward port 9999 (Organizr)
- ❌ DO NOT forward ANY service port except NPM and Headscale

**Why?**
- All public services MUST go through NPM for SSL and logging
- Direct port forwards bypass centralized security and logging
- NPM provides unified Let's Encrypt management
- NPM logs all external access for audit trails

## Access Patterns

### Scenario 1: Working from Home (Local Network)

**Can access via:**
- Local IPs: http://192.168.86.149:9999
- Mesh IPs: http://10.99.0.1:9999 (if VPN connected)
- Public domains: https://home.schweitz.net

**Best practice:** Use mesh IPs consistently for uniform experience

### Scenario 2: Remote Work (Connected to Headscale VPN)

**From laptop/phone on VPN:**
```bash
# Verify VPN connection
tailscale status

# Access Organizr
http://10.99.0.1:9999

# All tabs work with mesh IPs:
- Portainer: http://10.99.0.1:8001
- Netdata: http://10.99.0.1:19999
- Uptime Kuma: http://10.99.0.1:3001
```

**Accessing public services:**
- Can still use: https://media.schweitz.net (Jellyfin)
- Or direct mesh: http://10.99.0.1:8096
- Choose whichever is more convenient

### Scenario 3: Sharing with Family/Friends (No VPN)

**Public access only:**
- Jellyfin: https://media.schweitz.net
- Nextcloud: https://cloud.schweitz.net
- Organizr: https://home.schweitz.net (if you want public dashboard)

**Cannot access:**
- Admin tools (Portainer, Netdata, NPM) - VPN required
- They need Headscale VPN for admin access

## Security Configuration

### Organizr Authentication

**Enable auth for public access:**

Settings → User Management
- Create user accounts for family/friends
- Configure access levels:
  - Admin: Full access to all tabs
  - User: Only media/cloud tabs visible
  - Guest: Read-only access

**Restrict admin tabs to admin users only:**
- Tab Editor → each admin tab → "Minimum Authentication" → Admin

### NPM Access Lists (Optional)

**For extra security on public services:**

Access Lists → Create "VPN Only"
```
Name: Headscale VPN Only
Allow: 10.99.0.0/16
Deny: all
```

Apply to sensitive proxy hosts if needed.

### Service-Level Authentication

**Each service maintains its own auth:**
- Portainer: Admin password
- Jellyfin: User accounts
- Nextcloud: User accounts
- Uptime Kuma: Admin password

**This is defense in depth:**
1. VPN layer (for admin tools)
2. Organizr layer (for organizing access)
3. Service layer (individual logins)

## Connecting Other Devices

### Laptop/Desktop

```bash
# Install Tailscale
curl -fsSL https://tailscale.com/install.sh | sh

# Connect to Headscale
sudo tailscale up --login-server=http://192.168.86.149:8085 \
  --authkey=<your-preauth-key> \
  --hostname=my-laptop

# Verify mesh access
curl http://10.99.0.1:9999
# Should load Organizr
```

### Phone (Android/iOS)

1. Install Tailscale app from store
2. In app settings:
   - Use custom control server
   - Server URL: http://<your-public-ip>:8085
   - OR: http://192.168.86.149:8085 (if on local network)
3. Authenticate with pre-auth key
4. Open browser: http://10.99.0.1:9999

### Work Computer (Can't Install Software)

**Use public access only:**
- https://home.schweitz.net (Organizr - only non-admin tabs)
- https://media.schweitz.net (Jellyfin)
- https://cloud.schweitz.net (Nextcloud)

**Cannot access admin tools without VPN**

## Testing Checklist

### Phase 1: Local Access
- [ ] tower-of-joy connected to Headscale
- [ ] Mesh IP assigned (10.99.0.x)
- [ ] Can access services via mesh IP from tower-of-joy itself

### Phase 2: VPN Access from Another Device
- [ ] Connect laptop/phone to Headscale
- [ ] Verify mesh connectivity: `ping 10.99.0.1`
- [ ] Access Organizr: http://10.99.0.1:9999
- [ ] All tabs load correctly with mesh IPs
- [ ] Portainer accessible via mesh
- [ ] Netdata accessible via mesh

### Phase 3: Public Access
- [ ] DNS configured correctly
- [ ] NPM proxy hosts configured
- [ ] SSL certificates generated (green padlock)
- [ ] Access from public network (phone on mobile data):
  - [ ] https://home.schweitz.net loads Organizr
  - [ ] https://media.schweitz.net loads Jellyfin
  - [ ] https://cloud.schweitz.net loads Nextcloud
- [ ] Admin tabs NOT accessible without VPN

### Phase 4: Security Validation
- [ ] Admin tools (Portainer, Netdata) not accessible from public internet
- [ ] Only exposed ports: 80, 443, 8085
- [ ] Organizr authentication working
- [ ] Service-level authentication working

## Advantages of This Architecture

### Mesh IP Benefits
✅ **Location independent:** Same IPs whether at home or remote
✅ **Secure by default:** Admin tools only via VPN
✅ **Simple routing:** No complex proxy rewrites
✅ **Flexible access:** Public and private services coexist
✅ **Future-proof:** Add devices easily, IPs don't change
✅ **No split-brain:** One set of URLs to remember

### NPM Proxy Benefits (For Public Access)
✅ **Centralized SSL:** All Let's Encrypt certs in one place
✅ **Unified logging:** All external access logged in NPM audit log
✅ **Security headers:** Consistent HSTS, CSP, X-Frame-Options
✅ **Access control:** Add rate limiting, IP blocking at proxy level
✅ **DDoS protection:** Can add Cloudflare in front of NPM
✅ **Port efficiency:** Only 2 ports exposed (80, 443)

### Compliance & Auditing
✅ **Audit trail:** NPM logs all external access attempts
✅ **SSL compliance:** Automatic certificate renewal
✅ **Security posture:** Single point to review/harden public access
✅ **Change management:** Proxy config changes tracked in one place

## Troubleshooting

### Can't connect to mesh IPs
**Check:**
```bash
# Verify Tailscale running
sudo systemctl status tailscaled

# Check mesh status
tailscale status

# Test connectivity
ping 10.99.0.1
```

### Organizr tabs not loading
**Issue:** Service blocking iframe embedding
**Solution:**
- Check browser console for errors
- Some services need `X-Frame-Options` configured
- Use "pseudo tab" mode (opens in new tab instead)

### Public access not working
**Check:**
1. DNS resolves to your public IP: `nslookup home.schweitz.net`
2. Router port forwarding configured
3. NPM proxy host using correct mesh IP (10.99.0.1)
4. SSL certificate valid

### Headscale connection fails
**Check:**
- Port 8085 accessible from internet
- Pre-auth key still valid
- Headscale service running: `docker logs headscale`

## Next Actions

1. **Connect tower-of-joy to Headscale** (get mesh IP)
2. **Deploy Organizr** (`make deploy-organizr`)
3. **Configure Organizr tabs** (using mesh IPs)
4. **Configure NPM** (public services only)
5. **Test VPN access** (from another device)
6. **Test public access** (from mobile data)

---

**This gives you the best of both worlds:**
- Secure admin access via VPN + mesh IPs
- Public access for media/files (family/friends)
- Single Organizr dashboard for everything
- No complex proxy rewrites
- Easy to add new devices