portainer-core/setup-new-host.md

# Setting Up a New SSL-Secured, Authentik-Protected Host

This guide documents how to add a new public-facing subdomain with SSL (Let's Encrypt) and Authentik forward authentication through Nginx Proxy Manager (NPM).

## Prerequisites

Before starting, ensure:

1. **DNS A Record**: Create an A record for `<subdomain>.schweitz.net` pointing to your public IP
2. **Container Running**: The target service container is running and accessible
3. **Network**: Container is on `docker-dataplane` network (or accessible from NPM)
4. **Authentik Running**: Both Authentik server and outpost(s) are operational

## Step 1: Authentik Configuration

### 1.1 Create a Proxy Provider

1. Go to **Authentik Admin** → **Applications** → **Providers**
2. Click **Create** → Select **Proxy Provider**
3. Configure:

| Setting | Value |
|---------|-------|
| Name | `<subdomain>.schweitz.net proxy` |
| Authorization flow | `default-provider-authorization-implicit-consent` |
| Forward auth (single application) | **Selected** |
| External host | `https://<subdomain>.schweitz.net` |

Leave other settings as defaults (token validity 24 hours is fine).

### 1.2 Create an Application

1. Go to **Applications** → **Applications**
2. Click **Create**
3. Configure:

| Setting | Value |
|---------|-------|
| Name | `<Descriptive Name>` (e.g., "Library Wiki") |
| Slug | `<subdomain>-schweitz-net` |
| Provider | Select the proxy provider created above |
| Launch URL | `https://<subdomain>.schweitz.net` |

### 1.3 Add Application to Outpost

1. Go to **Applications** → **Outposts**
2. Edit the **standalone outpost** (authentik-proxy on port 9443/9445)
3. In **Applications**, add your new application to the selected list
4. Click **Update**

> **Note**: If using both embedded and standalone outposts, add the application to whichever outpost your forward auth config points to (standalone at `localhost:9445` in our setup).

## Step 2: NPM Configuration

### 2.1 Create Proxy Host

In NPM Admin (port 8000 or via API):

**Details Tab:**

| Setting | Value |
|---------|-------|
| Domain Names | `<subdomain>.schweitz.net` |
| Scheme | `http` (usually) |
| Forward Hostname/IP | Container name or IP (e.g., `wiki` or `192.168.86.149`) |
| Forward Port | Container's internal port (e.g., `3000`) |
| Cache Assets | Off (unless needed) |
| Block Common Exploits | On |
| Websockets Support | On |

**SSL Tab:**

| Setting | Value |
|---------|-------|
| SSL Certificate | Request a new SSL Certificate |
| Force SSL | On |
| HTTP/2 Support | On |
| HSTS Enabled | On |
| Email for Let's Encrypt | `jpmschweitzer@gmail.com` |
| Agree to TOS | Yes |

**Advanced Tab:**

Paste the following configuration:

```nginx
# Increase buffer size for large headers from Authentik
proxy_buffers 8 16k;
proxy_buffer_size 32k;

# Forward authentication via standalone outpost
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;

# Capture auth response headers
auth_request_set $auth_cookie $upstream_http_set_cookie;
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
auth_request_set $authentik_name $upstream_http_x_authentik_name;
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;

# Forward auth headers to application
add_header Set-Cookie $auth_cookie;
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-groups $authentik_groups;
proxy_set_header X-authentik-email $authentik_email;
proxy_set_header X-authentik-name $authentik_name;
proxy_set_header X-authentik-uid $authentik_uid;

# Outpost proxy location
location /outpost.goauthentik.io {
    proxy_pass https://localhost:9445/outpost.goauthentik.io;
    proxy_set_header Host $host;
    proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_pass_request_body off;
    proxy_set_header Content-Length "";
}

# Signin redirect handler - MUST use relative path, not absolute URL
location @goauthentik_proxy_signin {
    internal;
    add_header Set-Cookie $auth_cookie;
    return 302 /outpost.goauthentik.io/start?rd=$request_uri;
}
```

> **Critical**: The redirect in `@goauthentik_proxy_signin` MUST use a relative path (`/outpost.goauthentik.io/start`), NOT an absolute URL. Using `https://auth.schweitz.net/outpost.goauthentik.io/start` will cause a 404 error due to host header mismatch.

### 2.2 Save and Verify

1. Click **Save**
2. Verify nginx status shows online (no errors)
3. If errors occur, check the Advanced config syntax

## Step 3: Testing

### 3.1 Initial Test

1. Open an **incognito/private browser window**
2. Navigate to `https://<subdomain>.schweitz.net`
3. You should be redirected to Authentik login (Google OAuth or local)
4. After login, you should see your application

### 3.2 SSO Cookie Test

If already logged into another `.schweitz.net` service:
1. Open a new tab (same browser, not incognito)
2. Navigate to the new subdomain
3. You should access the application without re-authenticating (SSO cookie shared across `.schweitz.net`)

## Troubleshooting

### Authentik 404 "Not Found" Error

**Symptom**: Redirected to `auth.schweitz.net/outpost.goauthentik.io/start?rd=...` but see 404

**Causes & Fixes**:

1. **Absolute URL in redirect** (most common)
   - Fix: Change `return 302 https://auth.schweitz.net/outpost.goauthentik.io/start...` to `return 302 /outpost.goauthentik.io/start...`

2. **Application not added to outpost**
   - Fix: Go to Authentik → Outposts → Edit standalone outpost → Add application

3. **Provider not linked to application**
   - Fix: Edit application → Select the correct proxy provider

### Nginx Config Error / nginx_online: false

**Symptom**: NPM shows nginx error when saving

**Causes & Fixes**:

1. **Undefined variable `$connection_upgrade`**
   - Fix: Remove WebSocket lines (`proxy_http_version`, `Upgrade`, `Connection` headers) from advanced config. NPM handles WebSockets via the "Websockets Support" toggle.

2. **Syntax error in advanced config**
   - Fix: Verify all brackets are closed, semicolons present

### 502 Bad Gateway

**Symptom**: Auth works but application shows 502

**Causes & Fixes**:

1. **Wrong forward host/port**
   - Fix: Verify container name and port in NPM proxy host settings
   - Test: `curl http://<forward_host>:<forward_port>` from NPM container

2. **Container not on correct network**
   - Fix: Ensure container is on `docker-dataplane` network

3. **Outpost not running**
   - Check: `docker ps | grep authentik-proxy`
   - Check logs: `docker logs authentik-proxy`

### SSL Certificate Not Provisioning

**Symptom**: Certificate request fails

**Causes & Fixes**:

1. **DNS not propagated**
   - Fix: Wait for DNS propagation, verify with `dig <subdomain>.schweitz.net`

2. **Port 80 blocked**
   - Fix: Ensure port 80 is open for Let's Encrypt HTTP-01 challenge

### Redirect Loop

**Symptom**: Browser shows "too many redirects"

**Causes & Fixes**:

1. **Auth domain has forward auth enabled**
   - Fix: `auth.schweitz.net` must NOT have forward auth (would create loop)

2. **Cookie domain mismatch**
   - Fix: Verify Authentik cookie domain is `.schweitz.net`

## Reference Files

| File | Purpose |
|------|---------|
| `stacks/authentik.yml` | Authentik server, worker, and standalone outpost |
| `stacks/wiki.yml` | Example: Wiki.js container configuration |

## Architecture Overview

```
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   User Browser  │────▶│  NPM (port 80/  │────▶│  Target Service │
│                 │     │  443)           │     │  (e.g., Wiki.js)│
└─────────────────┘     └────────┬────────┘     └─────────────────┘
                                 │
                                 │ auth_request
                                 ▼
                        ┌─────────────────┐     ┌─────────────────┐
                        │ Authentik       │────▶│ Authentik       │
                        │ Standalone      │     │ Server          │
                        │ Outpost (9445)  │     │ (auth.schweitz  │
                        └─────────────────┘     │ .net)           │
                                                └─────────────────┘
```

**Flow**:
1. User requests `https://<subdomain>.schweitz.net`
2. NPM's `auth_request` directive checks with Authentik outpost
3. If not authenticated: redirect to Authentik login
4. User authenticates (Google OAuth, etc.)
5. Authentik sets session cookie for `.schweitz.net`
6. User redirected back, `auth_request` succeeds
7. NPM proxies request to target service

---

## Setting Up an Internal Domain (*.schweitz.internal)

Internal domains provide LAN-accessible URLs without SSL or Authentik authentication, ideal for:
- Programmatic API access (scripts, automation, healthchecks)
- Local development and testing
- Container-to-container communication via DNS
- Avoiding port number confusion on LAN

### Pattern

Every external `<service>.schweitz.net` domain has a corresponding `<service>.schweitz.internal` domain that:
- Uses HTTP only (no SSL certificates)
- Has NO Authentik forward auth
- Points to the same backend service
- Resolves via `/etc/hosts` on the server

### Current Internal Domains

| Internal Domain | Backend | External Equivalent |
|-----------------|---------|---------------------|
| home.schweitz.internal | localhost:9999 | home.schweitz.net |
| media.schweitz.internal | localhost:8096 | media.schweitz.net |
| cloud.schweitz.internal | localhost:8082 | cloud.schweitz.net |
| api.schweitz.internal | localhost:8083 | api.schweitz.net |
| code.schweitz.internal | localhost:8084 | code.schweitz.net |
| amp.schweitz.internal | localhost:8080 | amp.schweitz.net |
| housekeeping.schweitz.internal | localhost:8123 | housekeeping.schweitz.net |
| documents.schweitz.internal | 192.168.86.149:8091 | documents.schweitz.net |
| git.schweitz.internal | localhost:3002 | git.schweitz.net |
| library.schweitz.internal | localhost:8088 | library.schweitz.net |
| tatlock.schweitz.internal | localhost:8000 | tatlock.schweitz.net |
| webui.schweitz.internal | localhost:82 | webui.schweitz.net |

### Step 1: Add DNS Entry

Add the internal domain to `/etc/hosts` on the tower-of-joy server:

```bash
# Add to /etc/hosts
192.168.86.149 <service>.schweitz.internal
```

### Step 2: Create NPM Proxy Host

**Via NPM UI (port 8000):**

**Details Tab:**

| Setting | Value |
|---------|-------|
| Domain Names | `<service>.schweitz.internal` |
| Scheme | `http` |
| Forward Hostname/IP | Same as external domain (e.g., `localhost`) |
| Forward Port | Same as external domain |
| Cache Assets | On (optional) |
| Block Common Exploits | On |
| Websockets Support | On (if needed) |

**SSL Tab:** Leave empty (no SSL for internal domains)

**Advanced Tab:** Leave empty (no Authentik forward auth)

**Via NPM API:**

```bash
# Get token
TOKEN=$(curl -s -X POST http://localhost:81/api/tokens \
  -H "Content-Type: application/json" \
  -d '{"identity": "<email>", "secret": "<password>"}' | python3 -c "import sys,json; print(json.load(sys.stdin)['token'])")

# Create internal proxy host
curl -s -X POST http://localhost:81/api/nginx/proxy-hosts \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "domain_names": ["<service>.schweitz.internal"],
    "forward_scheme": "http",
    "forward_host": "localhost",
    "forward_port": <port>,
    "access_list_id": 0,
    "certificate_id": 0,
    "ssl_forced": false,
    "caching_enabled": true,
    "block_exploits": true,
    "advanced_config": "",
    "allow_websocket_upgrade": true,
    "http2_support": false,
    "hsts_enabled": false,
    "hsts_subdomains": false,
    "enabled": true,
    "locations": []
  }'
```

### Step 3: Test

```bash
curl -s -o /dev/null -w "%{http_code}" http://<service>.schweitz.internal/
```

Expected: `200`, `302` (redirect), or service-specific response.

### Usage Examples

```bash
# API calls without auth
curl http://api.schweitz.internal/health

# Git operations (Docker image pulls)
docker pull git.schweitz.internal/jpmschweitzer/core-api:latest

# Healthchecks in Docker Compose
healthcheck:
  test: ["CMD-SHELL", "curl -fSs http://git.schweitz.internal/api/healthz"]
```

---

## Appendix: Portainer API for Remote Stack Deployment

When you cannot access the Portainer web UI (e.g., outside home network), you can deploy stacks via the API.

### Authentication

```bash
# Get JWT token (valid for 8 hours)
http --ignore-stdin POST http://192.168.86.149:8001/api/auth \
  username=admin password=<password>

# Response: {"jwt":"eyJ..."}
```

### List Stacks

```bash
http --ignore-stdin GET http://192.168.86.149:8001/api/stacks \
  "Authorization:Bearer <token>"
```

### Read Stack File

```bash
http --ignore-stdin GET "http://192.168.86.149:8001/api/stacks/<stack_id>/file" \
  "Authorization:Bearer <token>"
```

### Create Stack

**Endpoint**: `POST /api/stacks/create/standalone/string?endpointId=3`

**Payload format**:
```json
{
  "name": "stack-name",
  "stackFileContent": "version: '3.8'\nservices:\n  ...",
  "env": [
    {"name": "VAR_NAME", "value": "var_value"},
    {"name": "SECRET_KEY", "value": "secret_value"}
  ]
}
```

**Example deployment script**:
```bash
#!/bin/bash
# Get token
TOKEN=$(http --ignore-stdin POST http://192.168.86.149:8001/api/auth \
  username=admin password=<password> | jq -r '.jwt')

# Read stack file and create JSON payload
STACK_CONTENT=$(cat /path/to/stack.yml)
jq -n \
  --arg name "my-stack" \
  --arg content "$STACK_CONTENT" \
  '{
    name: $name,
    stackFileContent: $content,
    env: [
      {name: "DB_PASSWORD", value: "secret123"},
      {name: "API_KEY", value: "key456"}
    ]
  }' > /tmp/payload.json

# Deploy
curl -s -X POST "http://192.168.86.149:8001/api/stacks/create/standalone/string?endpointId=3" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @/tmp/payload.json
```

### Update Stack

```bash
curl -s -X PUT "http://192.168.86.149:8001/api/stacks/<stack_id>?endpointId=3" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @/tmp/payload.json
```

### Delete Stack

```bash
http --ignore-stdin DELETE "http://192.168.86.149:8001/api/stacks/<stack_id>?endpointId=3" \
  "Authorization:Bearer <token>"
```

### Key Notes

- **Endpoint ID**: Use `3` for the local Docker environment (verify with `GET /api/endpoints`)
- **Stack Type**: Use `standalone/string` for Docker Compose stacks (not Swarm)
- **Environment Variables**: Passed as array of `{name, value}` objects, referenced in compose as `${VAR_NAME}`
- **Token Expiry**: JWT tokens expire after 8 hours; re-authenticate if needed

---

## Appendix: NPM API for Remote Proxy Configuration

When you cannot access the NPM web UI, you can configure proxy hosts via the API (port 81).

### Authentication

```bash
http --ignore-stdin POST http://192.168.86.149:81/api/tokens \
  identity=<email> secret=<password>

# Response: {"token":"eyJ...", "expires":"..."}
```

### List Proxy Hosts

```bash
http --ignore-stdin GET http://192.168.86.149:81/api/nginx/proxy-hosts \
  "Authorization:Bearer <token>"
```

### Create Proxy Host (without SSL)

```bash
curl -s -X POST http://192.168.86.149:81/api/nginx/proxy-hosts \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "domain_names": ["subdomain.schweitz.net"],
    "forward_scheme": "http",
    "forward_host": "192.168.86.149",
    "forward_port": 8091,
    "access_list_id": 0,
    "certificate_id": 0,
    "ssl_forced": false,
    "caching_enabled": false,
    "block_exploits": true,
    "advanced_config": "",
    "allow_websocket_upgrade": true,
    "http2_support": false,
    "hsts_enabled": false,
    "hsts_subdomains": false,
    "enabled": true,
    "locations": []
  }'
```

### Request Let's Encrypt Certificate

```bash
curl -s -X POST http://192.168.86.149:81/api/nginx/certificates \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "domain_names": ["subdomain.schweitz.net"],
    "meta": {"dns_challenge": false},
    "provider": "letsencrypt"
  }'

# Response includes certificate ID
```

### Update Proxy Host with SSL

```bash
curl -s -X PUT http://192.168.86.149:81/api/nginx/proxy-hosts/<proxy_id> \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "domain_names": ["subdomain.schweitz.net"],
    "forward_scheme": "http",
    "forward_host": "192.168.86.149",
    "forward_port": 8091,
    "access_list_id": 0,
    "certificate_id": <cert_id>,
    "ssl_forced": true,
    "caching_enabled": false,
    "block_exploits": true,
    "advanced_config": "",
    "allow_websocket_upgrade": true,
    "http2_support": true,
    "hsts_enabled": true,
    "hsts_subdomains": false,
    "enabled": true,
    "locations": []
  }'
```

### Key Notes

- **API Port**: NPM API is on port 81, not 80/443
- **Token Expiry**: Tokens expire after 1 day
- **SSL Flow**: Create proxy host → Request certificate → Update proxy host with certificate_id
- **forward_host**: Use container name (if on same network) or host IP
- **Verify**: Check `meta.nginx_online: true` in response

---

**Last Updated**: 2025-12-30
**Based on**: library.schweitz.net setup session, Paperless-ngx API deployment, Internal domain consolidation, Tatlock/WebUI domain reorganization