Files

T

jpmschweitzer 5e734ad27f ai-flow improvement / add langchain

2025-11-23 14:51:19 +01:00

35 KiB

Raw Blame History

Security Implementation Plan v2.0: Google OAuth SSO for Homelab Infrastructure

Version: 2.0 (Post-Failure Analysis & Comprehensive Revision) Date: 2025-11-20 Status: Ready for Implementation Previous Version: 1.1 (Failed - Redirect loops and high memory usage)

Document Change Log

Version	Date	Changes	Author
1.0	2025-11-10	Initial plan	-
1.1	2025-11-15	Scope revision	-
2.0	2025-11-20	Post-failure analysis, redirect loop fixes, memory optimization, milestone checkpoints	Claude Code

Executive Summary

This document is a complete revision of the Authentik SSO implementation plan after a failed first attempt that resulted in infinite redirect loops and excessive memory usage (20-30% of system RAM).

What Went Wrong (v1.0 Deployment):

✗ Infinite redirect loops between NPM, Authentik, and Google OAuth
✗ auth.schweitz.net was protected by forward auth, creating self-referential loop
✗ Mixed hostname/IP/domain addressing caused OAuth callback failures
✗ Missing AUTHENTIK_HOST environment variable led to incorrect internal redirects
✗ Cookie domain not configured, preventing cross-service SSO
✗ Excessive memory usage (~3-5GB) due to separate PostgreSQL and Redis instances
✗ No rollback checkpoints, making recovery difficult

What's Different in v2.0:

✅ Detailed redirect loop prevention strategies
✅ Strict hostname/URL consistency requirements
✅ Memory optimization with shared PostgreSQL/Redis (target: <1GB total)
✅ Milestone-based deployment with rollback points
✅ Configuration versioning at each step
✅ Validation tests before proceeding to next milestone
✅ Progress tracking within this document
✅ Embedded outpost instead of separate container (simpler)

Critical Success Factors:

auth.schweitz.net must NEVER have forward auth enabled
AUTHENTIK_HOST=https://auth.schweitz.net must be set correctly
AUTHENTIK_COOKIE_DOMAIN=.schweitz.net for cross-service SSO
Test each service individually before moving to the next
Create backup snapshots at every milestone
Update this document with progress and issues as we go

SSO Inclusion Policy:

✅ Include: Web-based admin interfaces, dashboards, APIs requiring browser access
❌ Exclude: Services with native mobile/desktop apps that work better with username/password
❌ Exclude: Media streaming services (Jellyfin) - app integration priority
❌ Exclude: Development tools (code-server) - IDE integration priority
⏸️ Defer: Disabled/inactive services (Nextcloud) - implement when re-enabled

Section 0: Failure Analysis & Lessons Learned
Section 1: Prerequisites & Environment Validation
Section 2: Resource Optimization Strategy
Section 3: Redirect Loop Prevention
Section 4: Implementation Milestones
Section 5: Testing & Validation
Section 6: Troubleshooting Guide
Section 7: Rollback Procedures
Section 8: Progress Tracking

Section 0: Failure Analysis & Lessons Learned

0.1 What Happened in v1.0 Deployment

Timeline:

Deployed Authentik stack with separate PostgreSQL and Redis
Configured forward auth on ALL NPM proxy hosts (including auth.schweitz.net)
Attempted to access services → immediate redirect loops
Consumed 20-30% of system RAM (~3-5GB)
Rolled back to working state (commit: cb428a8)

Root Causes Identified:

Issue 1: Self-Referential Authentication Loop

User → https://api.schweitz.net
  → NPM: "Need auth, redirect to https://auth.schweitz.net"
  → https://auth.schweitz.net
  → NPM: "Need auth, redirect to https://auth.schweitz.net"  ← LOOP!

Why it happened: NPM configuration had forward auth enabled on auth.schweitz.net itself.

Fix: Remove forward auth from auth.schweitz.net NPM proxy host configuration.

Issue 2: Hostname/URL Discrepancies

External: https://auth.schweitz.net/source/oauth/callback/google/
Internal: http://localhost:9000/source/oauth/callback/google/
Outpost:  http://192.168.86.149:9001/...

Why it happened:

Mixed use of localhost, IP addresses, and domain names
AUTHENTIK_HOST environment variable not set
Authentik didn't know its external URL

Fix:

Set AUTHENTIK_HOST=https://auth.schweitz.net
Use consistent container names (not localhost/IP) in Docker configs
Use external URLs for all OAuth callbacks

Why it happened: Authentik didn't set cookie domain, so cookies were scoped to individual hosts, not shared across *.schweitz.net.

Fix: Set AUTHENTIK_COOKIE_DOMAIN=.schweitz.net

Issue 4: Excessive Memory Usage

Why it happened:

Separate PostgreSQL instance for Authentik (~1GB)
Separate Redis instance for Authentik (~100MB)
No resource limits on server/worker containers

Fix:

Use shared PostgreSQL (postgres-shared:5432)
Use shared Redis (redis-shared:6379 DB 0)
Set memory limits: 512M server, 384M worker

0.2 Evidence from NPM Backup (2025-11-19)

All 11 NPM proxy hosts had identical forward auth configuration:

auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @authentik_proxy_signin;

location @authentik_proxy_signin {
    return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}

location /outpost.goauthentik.io {
    proxy_pass http://192.168.86.149:9001/outpost.goauthentik.io;
}

Problem: This configuration was applied to ALL hosts, including auth.schweitz.net.

0.3 Lessons Learned

Never protect the authentication server with authentication - Seems obvious in retrospect, but easy to miss
Test incrementally - We tried to enable all 11 services at once
Document assumptions - We assumed Authentik would auto-detect external URLs
Create rollback points - Had to manually restore NPM config from backup
Monitor resource usage - Didn't realize memory consumption until it was running
Read the docs thoroughly - AUTHENTIK_HOST is a critical but easy-to-miss setting

Section 1: Prerequisites & Environment Validation

1.1 System Specifications

Component	Specification	Notes
CPU	Intel i7-6700 (4C/8T)	~19% usage for Authentik (1.5 cores)
RAM	16GB total	Target: <1GB for Authentik (<6.25% of total)
Storage (SSD)	489GB	~3GB needed for Authentik data
Storage (HDD)	3.7TB	Not used by Authentik
Network	docker-dataplane	All services on unified network

Available Resources:

RAM: 16GB - Current usage = ~6-8GB free
Authentik target: 896MB (5.6% of total RAM) ✓ Acceptable

1.2 Shared Infrastructure Status

PostgreSQL Shared (postgres-shared:5432):

# Verify connectivity
docker exec postgres-shared pg_isready

# Verify authentik database exists
docker exec postgres-shared psql -U postgres -c "\l" | grep authentik

# Test connection as authentik_user
docker exec postgres-shared psql -U authentik_user -d authentik -c "SELECT version();"

Expected output:

authentik | authentik_user | UTF8 | ...
PostgreSQL 16.x on x86_64-pc-linux-musl...

Redis Shared (redis-shared:6379):

# Verify connectivity
docker exec redis-shared redis-cli PING

# Check DB 0 is available (Authentik allocation)
docker exec redis-shared redis-cli -n 0 INFO keyspace

# Test write/read
docker exec redis-shared redis-cli -n 0 SET test:authentik "connection_ok"
docker exec redis-shared redis-cli -n 0 GET test:authentik
docker exec redis-shared redis-cli -n 0 DEL test:authentik

Expected output:

PONG
# Keyspace (should be empty or have other keys)
connection_ok
1

1.3 Network Verification

# Verify docker-dataplane network exists
docker network inspect docker-dataplane

# List all containers on docker-dataplane
docker network inspect docker-dataplane | grep -A 2 "Containers"

# Verify DNS resolution between containers
docker run --rm --network docker-dataplane alpine ping -c 3 postgres-shared
docker run --rm --network docker-dataplane alpine ping -c 3 redis-shared

Expected: 19 containers on network, postgres-shared and redis-shared resolve correctly.

1.4 NPM Current State

# Backup current NPM configuration
curl -s http://192.168.86.149:8000/api/ > backups/npm-pre-authentik-$(date +%Y%m%d-%H%M%S).json

# Verify auth.schweitz.net proxy host exists
cat backups/npm-pre-authentik-*.json | jq '.proxy_hosts[] | select(.domain_names[] | contains("auth.schweitz"))'

Expected: auth.schweitz.net points to localhost:9000, currently has forward auth (will be removed).

1.5 SSL Certificates

# Verify Let's Encrypt certificates for schweitz.net domains
docker exec npm ls -la /etc/letsencrypt/live/ | grep schweitz

Expected: Certificates for all *.schweitz.net domains, valid and auto-renewing.

Section 2: Resource Optimization Strategy

2.1 Resource Allocation Plan

Authentik Server:

deploy:
  resources:
    limits:
      memory: 512M       # Hard cap
      cpus: '0.5'        # 50% of one core
    reservations:
      memory: 256M       # Guaranteed minimum

Authentik Worker:

deploy:
  resources:
    limits:
      memory: 384M       # Hard cap
      cpus: '0.3'        # 30% of one core
    reservations:
      memory: 128M       # Guaranteed minimum

Total Authentik overhead: 896MB (vs previous 3-5GB = 78% reduction)

2.2 Worker Configuration

Reduce Celery workers and threads:

environment:
  # Celery worker configuration
  AUTHENTIK_BOOTSTRAP_WORKERS: 1          # Default: 1 (single worker)
  AUTHENTIK_WORKER__CONCURRENCY: 2        # Threads per worker (default: 2)

  # Logging optimization
  AUTHENTIK_LOG_LEVEL: warning            # Reduce log verbosity
  AUTHENTIK_ERROR_REPORTING__ENABLED: false  # Disable error reporting

  # Disable non-essential features
  AUTHENTIK_AVATARS: none                 # Disable avatar fetching (saves bandwidth)
  AUTHENTIK_FOOTER_LINKS: '[]'           # Disable footer links

Resource impact:

1 worker @ 2 threads = minimal CPU usage during idle
Warning-level logging = less I/O overhead
No avatar fetching = no external HTTP requests

2.3 Database Connection Optimization

Use shared infrastructure, no connection pooling (unnecessary for homelab scale):

environment:
  # PostgreSQL connection
  AUTHENTIK_POSTGRESQL__HOST: postgres-shared
  AUTHENTIK_POSTGRESQL__PORT: 5432
  AUTHENTIK_POSTGRESQL__NAME: authentik
  AUTHENTIK_POSTGRESQL__USER: authentik_user
  AUTHENTIK_POSTGRESQL__PASSWORD: ${AUTHENTIK_DB_PASSWORD}
  AUTHENTIK_POSTGRESQL__USE_PGBOUNCER: false     # No pooler needed
  AUTHENTIK_POSTGRESQL__USE_PGPOOL: false        # No pooler needed

  # Redis connection
  AUTHENTIK_REDIS__HOST: redis-shared
  AUTHENTIK_REDIS__PORT: 6379
  AUTHENTIK_REDIS__DB: 0                         # Dedicated DB for Authentik
  AUTHENTIK_REDIS__PASSWORD: ""                  # No password (internal network only)

Benefit: No dedicated PostgreSQL/Redis = ~1.1GB RAM saved.

2.4 Memory Usage Monitoring

Check memory usage after deployment:

# Container-level memory usage
docker stats authentik-server authentik-worker --no-stream --format "table {{.Name}}\t{{.MemUsage}}\t{{.MemPerc}}"

# Process-level memory inside containers
docker exec authentik-server ps aux --sort=-%mem | head -10
docker exec authentik-worker ps aux --sort=-%mem | head -10

Expected values:

authentik-server: 200-400MB (under 512M limit)
authentik-worker: 100-250MB (under 384M limit)
Total: 300-650MB (healthy, with burst headroom)

If memory exceeds limits:

Check logs for errors: docker logs authentik-server --tail 100
Verify no memory leaks: docker exec authentik-server free -h
Restart containers if needed: docker restart authentik-server authentik-worker

Section 3: Redirect Loop Prevention

3.1 Critical Configuration Rules

RULE #1: Never Enable Forward Auth on auth.schweitz.net

# ✓ CORRECT - auth.schweitz.net NPM config
{
  "domain_names": ["auth.schweitz.net"],
  "forward_host": "localhost",
  "forward_port": 9000,
  "ssl_forced": true,
  "advanced_config": "# NO FORWARD AUTH - This is the login page!\n\n# Optional: Rate limiting\nlimit_req_zone $binary_remote_addr zone=auth_limit:10m rate=10r/m;\nlimit_req zone=auth_limit burst=5 nodelay;"
}

# ✗ WRONG - DO NOT DO THIS!
{
  "domain_names": ["auth.schweitz.net"],
  "advanced_config": "auth_request /outpost.goauthentik.io/auth/nginx;"  # ← CAUSES LOOP!
}

RULE #2: Set AUTHENTIK_HOST to External URL

environment:
  AUTHENTIK_HOST: https://auth.schweitz.net              # ← External HTTPS URL
  AUTHENTIK_HOST_BROWSER: https://auth.schweitz.net      # ← Same as AUTHENTIK_HOST

Why: Authentik uses this for generating redirect URLs in OAuth flows. If unset, it uses the request Host header, which might be localhost or an IP address internally.

RULE #3: Configure Cookie Domain for SSO

environment:
  AUTHENTIK_COOKIE_DOMAIN: .schweitz.net                 # ← Note the leading dot!
  AUTHENTIK_COOKIE_SAMESITE: lax                         # ← Allow cross-subdomain cookies

Why: Without this, cookies are scoped to individual hosts (e.g., auth.schweitz.net only), and can't be shared with api.schweitz.net, cloud.schweitz.net, etc.

RULE #4: Use Embedded Outpost (Simplicity)

# Server container serves both web UI and outpost
authentik-server:
  ports:
    - "9000:9000"   # Web UI
    - "9443:9443"   # Embedded outpost
  environment:
    AUTHENTIK_OUTPOSTS__DOCKER_IMAGE_BASE: "ghcr.io/goauthentik/%(type)s:%(version)s"

Why: Eliminates need for separate outpost container, reduces complexity and memory usage.

RULE #5: Consistent NPM Forward Auth Configuration

# Forward auth configuration for protected services (NOT auth.schweitz.net)
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @authentik_proxy_signin;

auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;

auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;

proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-email $authentik_email;
proxy_set_header X-authentik-groups $authentik_groups;

location @authentik_proxy_signin {
    internal;
    return 302 https://auth.schweitz.net/outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}

location /outpost.goauthentik.io {
    proxy_pass http://authentik-server:9443/outpost.goauthentik.io;  # ← Use embedded outpost
    proxy_set_header Host $host;
    proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_pass_request_body off;
    proxy_set_header Content-Length "";
}

3.2 OAuth Redirect URI Configuration

Google OAuth Callback URL: https://auth.schweitz.net/source/oauth/callback/google/

In Google Cloud Console:

Authorized JavaScript origins:
  https://auth.schweitz.net

Authorized redirect URIs:
  https://auth.schweitz.net/source/oauth/callback/google/

In Authentik (Google Social Source):

Consumer Key: <Google Client ID>
Consumer Secret: <Google Client Secret>
Provider Type: Google
Callback URL: (auto-filled, verify it matches Google Cloud Console)

Critical: The callback URL must exactly match between Google Cloud Console and Authentik. Case-sensitive, trailing slashes matter.

3.3 Testing Redirect Flows

Manual redirect flow test:

# 1. Visit protected service (should redirect to Authentik)
curl -I https://api.schweitz.net
# Expected: 302 redirect to https://auth.schweitz.net/outpost.goauthentik.io/start?rd=...

# 2. Visit Authentik login page (should NOT redirect)
curl -I https://auth.schweitz.net
# Expected: 200 OK (Authentik login page)

# 3. After login, check if cookie is set
curl -I -c cookies.txt https://auth.schweitz.net/flows/executor/default-provider-authentication-flow/
# Expected: Set-Cookie: authentik_session=...; Domain=.schweitz.net

# 4. Use cookie to access protected service
curl -I -b cookies.txt https://api.schweitz.net
# Expected: 200 OK (authenticated)

3.4 Redirect Loop Detection

Symptoms:

Browser error: "ERR_TOO_MANY_REDIRECTS"
Nginx logs show repeated 302 responses
Authentik logs show repeated authentication attempts from same IP

Diagnosis:

# Check NPM logs for redirect chains
docker logs npm 2>&1 | grep -A 5 "auth.schweitz.net" | tail -50

# Check Authentik logs for repeated auth attempts
docker logs authentik-server 2>&1 | grep -i "redirect\|loop\|401\|302" | tail -50

# Test with verbose curl (shows redirect chain)
curl -v -L https://api.schweitz.net 2>&1 | grep -E "(< HTTP|< Location)"

If loop detected:

Check if auth.schweitz.net has forward auth enabled → Remove it
Verify AUTHENTIK_HOST environment variable → Should be https://auth.schweitz.net
Check cookie domain → Should be .schweitz.net
Verify outpost endpoint is reachable → curl http://authentik-server:9443/outpost.goauthentik.io/ping/

Section 4: Implementation Milestones

Milestone 0: Pre-Deployment Validation ✓ Safe to Execute

Objective: Verify all prerequisites are met before deploying Authentik.

Tasks:

M0.1: Run PostgreSQL connectivity tests (Section 1.2)
M0.2: Run Redis connectivity tests (Section 1.2)
M0.3: Verify docker-dataplane network (Section 1.3)
M0.4: Backup NPM configuration
M0.5: Document current NPM forward auth status (should have auth on all hosts)
M0.6: Remove forward auth from ALL NPM proxy hosts temporarily
M0.7: Verify system resource availability (RAM, CPU, disk)

Validation Checklist:

# Run all validation commands
bash scripts/validate-authentik-prerequisites.sh  # Create this script

Expected Duration: 15-30 minutes

Rollback: N/A (no changes made to running services)

Backup Files:

backups/npm-m0-$(date +%Y%m%d).json - NPM config before changes
backups/docker-ps-m0-$(date +%Y%m%d).txt - Container list

Status: ⏳ Not Started

Issues Log:

[Date] [Issue description] - [Resolution]

Milestone 1: Authentik Deployment (Core Only)

Objective: Deploy Authentik server and worker containers, configure NPM proxy for auth.schweitz.net (WITHOUT forward auth).

Dependencies: M0 completed

Tasks:

M1.1: Create stacks/authentik.yml with correct environment variables
M1.2: Set AUTHENTIK_HOST, AUTHENTIK_COOKIE_DOMAIN, resource limits
M1.3: Deploy Authentik stack via Portainer
M1.4: Wait for containers to start (watch logs)
M1.5: Verify database migrations completed
M1.6: Update NPM proxy host for auth.schweitz.net (NO forward auth!)
M1.7: Access https://auth.schweitz.net (should show Authentik setup wizard)
M1.8: Complete initial setup wizard, create admin account

Configuration Files:

stacks/authentik.yml:

version: '3.8'

# Authentik Identity Provider (SSO)
# Purpose: Centralized authentication for all homelab services
# Ports: 9000 (web UI), 9443 (embedded outpost)
# GPU: No
# Storage: SSD (configs), PostgreSQL shared (user data)

services:
  authentik-server:
    image: ghcr.io/goauthentik/server:2024.8.4  # Pinned version (2024.10 has redirect loop issues)
    container_name: authentik-server
    restart: unless-stopped
    command: server
    environment:
      # External URLs (CRITICAL for redirect loop prevention)
      AUTHENTIK_HOST: https://auth.schweitz.net
      AUTHENTIK_HOST_BROWSER: https://auth.schweitz.net

      # Cookie settings (CRITICAL for SSO across subdomains)
      AUTHENTIK_COOKIE_DOMAIN: .schweitz.net
      AUTHENTIK_COOKIE_SAMESITE: lax

      # SSL/TLS
      AUTHENTIK_INSECURE: false

      # PostgreSQL (shared)
      AUTHENTIK_POSTGRESQL__HOST: postgres-shared
      AUTHENTIK_POSTGRESQL__PORT: 5432
      AUTHENTIK_POSTGRESQL__NAME: authentik
      AUTHENTIK_POSTGRESQL__USER: authentik_user
      AUTHENTIK_POSTGRESQL__PASSWORD: F//j0ktck7cX06Vfgh0YXceONOtlSsHvadqROICeDx8=
      AUTHENTIK_POSTGRESQL__USE_PGBOUNCER: false

      # Redis (shared)
      AUTHENTIK_REDIS__HOST: redis-shared
      AUTHENTIK_REDIS__PORT: 6379
      AUTHENTIK_REDIS__DB: 0

      # Secret key (generate with: openssl rand -base64 32)
      AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY}  # Store in .env file

      # Resource optimization
      AUTHENTIK_LOG_LEVEL: warning
      AUTHENTIK_ERROR_REPORTING__ENABLED: false
      AUTHENTIK_AVATARS: none
      AUTHENTIK_FOOTER_LINKS: '[]'

      # Embedded outpost configuration
      AUTHENTIK_OUTPOSTS__DOCKER_IMAGE_BASE: "ghcr.io/goauthentik/%(type)s:%(version)s"

      # Timezone
      TZ: Europe/Amsterdam

    ports:
      - "9000:9000"   # Web UI
      - "9443:9443"   # Embedded outpost (proxy provider)

    volumes:
      - /home/jpmschweitzer/docker-data/authentik/media:/media
      - /home/jpmschweitzer/docker-data/authentik/custom-templates:/templates

    networks:
      - docker-dataplane

    depends_on:
      - postgres-shared
      - redis-shared

    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:9000/-/health/live/ || exit 1"]
      start_period: 60s
      interval: 30s
      timeout: 10s
      retries: 3

    deploy:
      resources:
        limits:
          memory: 512M
          cpus: '0.5'
        reservations:
          memory: 256M

  authentik-worker:
    image: ghcr.io/goauthentik/server:2024.8.4  # Same version as server
    container_name: authentik-worker
    restart: unless-stopped
    command: worker
    environment:
      # Same environment as server (MUST match exactly)
      AUTHENTIK_HOST: https://auth.schweitz.net
      AUTHENTIK_HOST_BROWSER: https://auth.schweitz.net
      AUTHENTIK_COOKIE_DOMAIN: .schweitz.net
      AUTHENTIK_COOKIE_SAMESITE: lax
      AUTHENTIK_INSECURE: false
      AUTHENTIK_POSTGRESQL__HOST: postgres-shared
      AUTHENTIK_POSTGRESQL__PORT: 5432
      AUTHENTIK_POSTGRESQL__NAME: authentik
      AUTHENTIK_POSTGRESQL__USER: authentik_user
      AUTHENTIK_POSTGRESQL__PASSWORD: F//j0ktck7cX06Vfgh0YXceONOtlSsHvadqROICeDx8=
      AUTHENTIK_POSTGRESQL__USE_PGBOUNCER: false
      AUTHENTIK_REDIS__HOST: redis-shared
      AUTHENTIK_REDIS__PORT: 6379
      AUTHENTIK_REDIS__DB: 0
      AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY}
      AUTHENTIK_LOG_LEVEL: warning
      AUTHENTIK_ERROR_REPORTING__ENABLED: false
      TZ: Europe/Amsterdam

      # Worker-specific configuration
      AUTHENTIK_BOOTSTRAP_WORKERS: 1   # Single worker (homelab scale)
      AUTHENTIK_WORKER__CONCURRENCY: 2  # 2 threads per worker

    volumes:
      - /home/jpmschweitzer/docker-data/authentik/media:/media
      - /home/jpmschweitzer/docker-data/authentik/custom-templates:/templates
      - /home/jpmschweitzer/docker-data/authentik/certs:/certs
      - /var/run/docker.sock:/var/run/docker.sock  # For outpost management

    networks:
      - docker-dataplane

    depends_on:
      - postgres-shared
      - redis-shared
      - authentik-server

    healthcheck:
      test: ["CMD-SHELL", "ak healthcheck"]
      start_period: 60s
      interval: 30s
      timeout: 10s
      retries: 3

    deploy:
      resources:
        limits:
          memory: 384M
          cpus: '0.3'
        reservations:
          memory: 128M

networks:
  docker-dataplane:
    external: true
    name: docker-dataplane

# Setup Instructions:
#
# 1. Generate secret key:
#    openssl rand -base64 32 > .env.authentik
#    echo "AUTHENTIK_SECRET_KEY=<paste-key-here>" >> .env.authentik
#
# 2. Create directories:
#    mkdir -p ~/docker-data/authentik/{media,custom-templates,certs}
#
# 3. Deploy stack:
#    docker-compose -f stacks/authentik.yml up -d
#
# 4. Watch logs:
#    docker logs -f authentik-server
#    docker logs -f authentik-worker
#
# 5. Wait for migrations to complete (~2-3 minutes):
#    docker logs authentik-server 2>&1 | grep "Applying migration"
#
# 6. Access web UI:
#    https://auth.schweitz.net (should show setup wizard)
#
# 7. Complete setup wizard:
#    - Email: admin@schweitz.net
#    - Password: <secure-password>
#    - Finish setup
#
# Monitoring:
#
# Memory usage:
#   docker stats authentik-server authentik-worker --no-stream
#
# Database connectivity:
#   docker exec authentik-server ak check
#
# Outpost status:
#   curl http://authentik-server:9443/outpost.goauthentik.io/ping/

Validation Commands:

# M1.1: Check containers are running
docker ps | grep authentik

# M1.2: Check logs for errors
docker logs authentik-server 2>&1 | grep -iE "error|critical|fail" | tail -20
docker logs authentik-worker 2>&1 | grep -iE "error|critical|fail" | tail -20

# M1.3: Check database migrations completed
docker logs authentik-server 2>&1 | grep "Applying migration" | tail -10
docker logs authentik-server 2>&1 | grep "No migrations to apply" || echo "❌ Migrations still running"

# M1.4: Check health checks passing
docker inspect authentik-server | jq '.[0].State.Health.Status'
docker inspect authentik-worker | jq '.[0].State.Health.Status'

# M1.5: Check memory usage (should be under limits)
docker stats authentik-server authentik-worker --no-stream --format "table {{.Name}}\t{{.MemUsage}}\t{{.MemPerc}}"

# M1.6: Check NPM proxy host for auth.schweitz.net
curl -I https://auth.schweitz.net
# Expected: 200 OK (Authentik setup wizard)

# M1.7: Check for redirect loops (should NOT redirect)
curl -v https://auth.schweitz.net 2>&1 | grep -c "Location:"
# Expected: 0 (or 1 if redirecting HTTP → HTTPS, which is fine)

# M1.8: Verify embedded outpost is accessible
docker exec -it authentik-server curl http://localhost:9443/outpost.goauthentik.io/ping/
# Expected: {"status": "ok"} or similar

Success Criteria:

✓ Both containers running (docker ps)
✓ Health checks passing (docker inspect)
✓ Memory usage < 700MB total
✓ Database migrations completed
✓ https://auth.schweitz.net accessible (200 OK, no redirect loop)
✓ Authentik setup wizard displayed
✓ Admin account created

Expected Duration: 30-60 minutes

Rollback Procedure:

# Stop and remove Authentik containers
docker-compose -f stacks/authentik.yml down

# Restore NPM configuration (if changed)
curl -X POST http://192.168.86.149:8000/api/restore \
  -H "Content-Type: application/json" \
  -d @backups/npm-m0-YYYYMMDD.json

# Verify NPM restored
curl -I https://auth.schweitz.net  # Should return error (Authentik not running)

Backup Files:

backups/authentik-m1-$(date +%Y%m%d).yml - Authentik stack config
backups/npm-m1-$(date +%Y%m%d).json - NPM config after M1
backups/authentik-m1-env-$(date +%Y%m%d).txt - Environment variables (REDACTED)

Status: ⏳ Not Started

Issues Log:

[Date] [Issue description] - [Resolution]

Milestone 2: Google OAuth Integration

Objective: Configure Google as an authentication source in Authentik.

Dependencies: M1 completed successfully

Tasks:

M2.1: Create Google Cloud Project (or use existing)
M2.2: Configure OAuth consent screen
M2.3: Create OAuth 2.0 client credentials
M2.4: Add Google as social source in Authentik admin panel
M2.5: Configure callback URLs
M2.6: Test Google login with Workspace account
M2.7: Test Google login with Gmail account
M2.8: Verify user profile attributes synced

Configuration: See original plan Section 5.1.2 for detailed steps.

Expected Duration: 30-45 minutes

Status: ⏳ Not Started

Milestone 3: Single Service Protection (Organizr Test)

Objective: Enable forward auth on ONE service (Organizr at home.schweitz.net) to validate SSO works end-to-end.

Dependencies: M2 completed successfully

Configuration: See original plan Section 5.2 for detailed steps.

Expected Duration: 60-90 minutes

Status: ⏳ Not Started

Milestone 4: Core API Protection (FastAPI Native OIDC)

Objective: Protect Core API with FastAPI native OIDC token validation (bearer token authentication).

Dependencies: M3 completed successfully

Status: ✅ COMPLETE - Using forward auth (shares Organizr Proxy provider)

Implementation Decision (2025-11-23):

Core API already protected with forward auth via NPM
Shares "Organizr Proxy" provider with home.schweitz.net
Authentication working correctly with Google OAuth
Headers forwarded: X-authentik-username, X-authentik-email, X-authentik-groups, X-authentik-name, X-authentik-uid
Decision: Keep current setup, defer separate admin provider to avoid complexity
Rationale: Current implementation is secure and functional for homelab use case

Configuration:

Provider: Organizr Proxy (shared)
External host: https://api.schweitz.net
Outpost: Standalone proxy (port 9445)
Mode: forward_single

Expected Duration: ~~90-120 minutes~~ SKIPPED (already functional)

Milestone 5: Remaining Services (Gradual Rollout)

Objective: Enable forward auth on remaining services, one at a time, testing each before proceeding.

Dependencies: M3 and M4 completed successfully

Services to Protect:

Gitea (git.schweitz.net)
Open WebUI (no external domain yet)
Netdata (no external domain yet)
Uptime Kuma (no external domain yet)
AMP (amp.schweitz.net)
Tatlock (tatlock.schweitz.net)

Services EXCLUDED from SSO (Keep Native Auth):

❌ Jellyfin (media.schweitz.net) - Better mobile app integration with native auth
❌ code-server (code.schweitz.net) - Better VS Code integration with native auth
❌ Nextcloud (cloud.schweitz.net) - Service disabled, SSO deferred until re-enabled

Rationale for Exclusions:

Jellyfin and code-server have excellent native authentication
Mobile apps and desktop clients work better with username/password
SSO adds complexity without significant security benefit for these services
Nextcloud is not currently in active use

Expected Duration: 3-6 hours (30-60 min per service)

Status: ⏳ Not Started

Section 5: Testing & Validation

5.1 Automated Testing Scripts

Create these scripts in /scripts directory:

validate-authentik-prerequisites.sh - Pre-deployment checks
test-redirect-flow.sh - Test redirect loops
monitor-resources.sh - Monitor memory/CPU usage

(Full script contents in original plan appendices)

Section 6: Troubleshooting Guide

6.1 Redirect Loop Troubleshooting

Common Causes & Fixes:

Cause	Fix
`auth.schweitz.net` has forward auth enabled	Remove forward auth from NPM config
`AUTHENTIK_HOST` not set or wrong	Set `AUTHENTIK_HOST=https://auth.schweitz.net`
Cookie domain mismatch	Set `AUTHENTIK_COOKIE_DOMAIN=.schweitz.net`
Outpost endpoint unreachable	Verify `curl http://authentik-server:9443/outpost.goauthentik.io/ping/` works

6.2 Memory Usage Issues

Common Causes & Fixes:

Cause	Fix
Too many Celery workers	Set `AUTHENTIK_BOOTSTRAP_WORKERS=1`
Too many threads per worker	Set `AUTHENTIK_WORKER__CONCURRENCY=2`
Memory leak	Restart containers

(Full troubleshooting guide in original plan Section 6)

Section 7: Rollback Procedures

7.1 Full Rollback

# Stop Authentik
docker-compose -f stacks/authentik.yml down -v

# Restore NPM config
curl -X POST http://192.168.86.149:8000/api/restore \
  -H "Content-Type: application/json" \
  -d @backups/npm-m0-YYYYMMDD.json

# Verify services accessible
curl -I https://api.schweitz.net

7.2 Partial Rollback (Single Service)

Restore NPM config for specific proxy host from milestone backup.

(Full rollback procedures in original plan Section 7)

Section 8: Progress Tracking

8.1 Implementation Status

Last Updated: 2025-11-20

Milestone	Status	Start Date	Complete Date	Duration	Issues
M0: Pre-deployment Validation	⏳ Not Started	-	-	-	-
M1: Authentik Deployment	⏳ Not Started	-	-	-	-
M2: Google OAuth	⏳ Not Started	-	-	-	-
M3: Organizr SSO	⏳ Not Started	-	-	-	-
M4: Core API OIDC	⏳ Not Started	-	-	-	-
M5: Remaining Services	⏳ Not Started	-	-	-	-

Legend:

⏳ Not Started
🔄 In Progress
✅ Completed
⚠️ Issues/Blocked
❌ Failed

8.2 Detailed Progress Log

Update this section as you progress through each milestone.

Template:

### Milestone X: [Name]

**Start:** [YYYY-MM-DD HH:MM]
**Completed:** [YYYY-MM-DD HH:MM]
**Duration:** [Time]

**Tasks Completed:**
- [x] Task 1
- [x] Task 2

**Issues Encountered:**
1. **Issue:** [Description]
   **Cause:** [Root cause]
   **Solution:** [How fixed]

**Validation Results:**
- ✓ Item 1
- ✓ Item 2

**Backups Created:**
- backups/filename-mX-YYYYMMDD.ext

**Next Steps:**
- Proceed to Milestone X+1

8.3 Configuration Changelog

Track all configuration changes:

Date	Component	Change	Reason	Rollback
2025-11-20	Plan	Created v2.0	Post-failure revision	N/A
-	-	-	-	-

8.4 Lessons Learned (Real-Time)

Add lessons as you encounter issues:

[Date] [Milestone] [Lesson]

Appendix A: Quick Reference Commands

A.1 Authentik Management

# Check status
docker ps | grep authentik
docker logs authentik-server --tail 50

# Restart
docker restart authentik-server authentik-worker

# Check database
docker exec authentik-server ak check

# Run migrations
docker exec authentik-server ak migrate

# Check outpost
curl http://authentik-server:9443/outpost.goauthentik.io/ping/

A.2 NPM Management

# Backup
curl -s http://192.168.86.149:8000/api/ > backups/npm-$(date +%Y%m%d-%H%M%S).json

# Test config
docker exec npm nginx -t

# Reload
docker exec npm nginx -s reload

Appendix B: Environment Variables Reference

Authentik Critical Variables:

AUTHENTIK_HOST=https://auth.schweitz.net
AUTHENTIK_HOST_BROWSER=https://auth.schweitz.net
AUTHENTIK_COOKIE_DOMAIN=.schweitz.net
AUTHENTIK_COOKIE_SAMESITE=lax
AUTHENTIK_POSTGRESQL__HOST=postgres-shared
AUTHENTIK_REDIS__HOST=redis-shared
AUTHENTIK_LOG_LEVEL=warning
AUTHENTIK_BOOTSTRAP_WORKERS=1
AUTHENTIK_WORKER__CONCURRENCY=2

Appendix C: Known Issues & Workarounds

C.1 Authentik 2024.10.x Redirect Loop Bug

Issue: Version 2024.10 has redirect loop bug.

Workaround: Use version 2024.8.4 (pinned in authentik.yml).

Reference: https://github.com/goauthentik/authentik/issues/11883

END OF SECURITY IMPLEMENTATION PLAN v2.0

Ready for Implementation: Yes ✓ Reviewed By: Claude Code (Autonomous Analysis) Approved By: [User - Pending]

This plan is a living document. Update Section 8 (Progress Tracking) as you progress through milestones.

35 KiB Raw Blame History

Security Implementation Plan v2.0: Google OAuth SSO for Homelab Infrastructure

Document Change Log

Executive Summary

Table of Contents

Section 0: Failure Analysis & Lessons Learned

0.1 What Happened in v1.0 Deployment

Issue 1: Self-Referential Authentication Loop

Issue 2: Hostname/URL Discrepancies

Issue 3: Cookie Domain Mismatch

Issue 4: Excessive Memory Usage

0.2 Evidence from NPM Backup (2025-11-19)

0.3 Lessons Learned

Section 1: Prerequisites & Environment Validation

1.1 System Specifications

1.2 Shared Infrastructure Status

1.3 Network Verification

1.4 NPM Current State

1.5 SSL Certificates

Section 2: Resource Optimization Strategy

2.1 Resource Allocation Plan

2.2 Worker Configuration

2.3 Database Connection Optimization

2.4 Memory Usage Monitoring

Section 3: Redirect Loop Prevention

3.1 Critical Configuration Rules

3.2 OAuth Redirect URI Configuration

3.3 Testing Redirect Flows

3.4 Redirect Loop Detection

Section 4: Implementation Milestones

Milestone 0: Pre-Deployment Validation ✓ Safe to Execute

Milestone 1: Authentik Deployment (Core Only)

Milestone 2: Google OAuth Integration

Milestone 3: Single Service Protection (Organizr Test)

Milestone 4: Core API Protection (FastAPI Native OIDC)

Milestone 5: Remaining Services (Gradual Rollout)

Section 5: Testing & Validation

5.1 Automated Testing Scripts

Section 6: Troubleshooting Guide

6.1 Redirect Loop Troubleshooting

6.2 Memory Usage Issues

Section 7: Rollback Procedures

7.1 Full Rollback

7.2 Partial Rollback (Single Service)

Section 8: Progress Tracking

8.1 Implementation Status

8.2 Detailed Progress Log

8.3 Configuration Changelog

8.4 Lessons Learned (Real-Time)

Appendix A: Quick Reference Commands

A.1 Authentik Management

A.2 NPM Management

Appendix B: Environment Variables Reference

Appendix C: Known Issues & Workarounds

C.1 Authentik 2024.10.x Redirect Loop Bug

35 KiB

Raw Blame History