jpmschweitzer/portainer-core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4cc2002fa65094a6b8ebfdd32d4987686e9efdf8
portainer-core/docs/sessions/2025-11-20-authentik-deployment.md
T

13 KiB
Raw Blame History

Authentik SSO Deployment Session

Date: 2025-11-20 Duration: ~4 hours Status: Milestone 2/5 Complete (Google OAuth Working) Version: 0.8.0-authentik-sso

Session Overview

Successfully deployed Authentik identity provider with Google OAuth integration and optimized memory usage. Forward authentication configuration blocked on embedded outpost initialization issue.

Accomplishments

✅ Milestone 1: Authentik Deployment (COMPLETE)

Infrastructure Setup:

  • Deployed Authentik server and worker containers (version 2024.8.4)
  • Configured shared PostgreSQL: authentik database with authentik_user
  • Configured shared Redis: Database 0
  • Network: Connected to docker-dataplane

Configuration Highlights:

Memory Limits:
  - Server: 512M limit, 256M reservation
  - Worker: 384M limit, 128M reservation
  - Total: 563MB actual usage (vs 3-5GB previous attempt = 80-90% reduction!)

Ports:
  - 9000: Web UI
  - 9444: Embedded outpost (mapped from container 9443)

Environment:
  - AUTHENTIK_HOST: https://auth.schweitz.net
  - AUTHENTIK_COOKIE_DOMAIN: .schweitz.net
  - PostgreSQL: postgres-shared:5432/authentik
  - Redis: redis-shared:6379/0

Issues Resolved:

  1. Health check failure - Container didn't have wget/curl
    • Solution: Used Python's urllib.request for health checks
  2. Database user didn't exist - authentik_user not created by init script
    • Solution: Manually created user with proper grants
  3. Port conflict - 9443 already in use
    • Solution: Mapped to 9444 on host
  4. NPM proxy missing - auth.schweitz.net not visible in UI
    • Solution: Entry was marked as deleted (is_deleted=1), recreated via UI

NPM Configuration:

  • Created proxy host for auth.schweitz.net
  • Forward to: http://localhost:9000
  • SSL: Let's Encrypt (enforced, HSTS enabled)
  • Critical: NO forward auth on auth.schweitz.net (prevents redirect loops)

✅ Milestone 2: Google OAuth Integration (COMPLETE)

Google Cloud Console Setup:

  • Created OAuth credentials:
    • Client ID: 59195574918-813nsfslhjduqto8nc4a3ejg2lj133il.apps.googleusercontent.com
    • Client Secret: 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
    • Authorized redirect URI: https://auth.schweitz.net/source/oauth/callback/google/

Authentik Configuration (via API):

# Created Google OAuth source
Source: "Google"
Slug: "google"
Provider: "google"
Consumer Key: [Google Client ID]
Consumer Secret: [Google Client Secret]
Enrollment Flow: default-source-enrollment
Authentication Flow: default-source-authentication

Login Flow Configuration:

  • Updated default-authentication-identification stage
  • Enabled "Show sources' labels"
  • Added Google source to sources list
  • Result: Google login button now appears on login page

Testing Results:

  • ✅ Google login button visible on auth.schweitz.net
  • ✅ OAuth redirect to Google works
  • ✅ User created successfully: jpmschweitzer@gmail.com
  • ✅ User type: external (correct for OAuth users)
  • ⚠️ External users blocked from admin interface (expected behavior)
  • ✅ Admin access via akadmin recovery key

Enrollment Flow Issue & Resolution:

  • Initial error: "Flow does not apply to current user"
  • Root cause: Browser session had conflicting flow plan cached
  • Solution: Cleared cookies, used incognito window
  • Policy check: default-source-enrollment-if-sso working correctly

🚧 Milestone 3: Forward Auth for Organizr (BLOCKED)

Progress:

  • ✅ Created Proxy Provider "Organizr Proxy" via API
    • Mode: forward_single
    • External host: https://home.schweitz.net
    • Authorization flow: default-provider-authorization-implicit-consent
  • ✅ Created Application "Organizr" via API
    • Slug: organizr
    • Provider: Organizr Proxy
    • Launch URL: https://home.schweitz.net
  • ✅ Assigned provider to embedded outpost
  • ✅ Embedded outpost responding on port 9444
    • Ping endpoint works: https://localhost:9444/outpost.goauthentik.io/ping

Current Blocker:

Issue: Auth endpoint returns 404
Endpoint: https://localhost:9444/outpost.goauthentik.io/auth/nginx
Status: 404 Not Found
Expected: 200 OK or 401/302 for unauthenticated requests

NPM Error Logs:
auth request unexpected status: 404 while sending to client

Analysis:

  • Outpost is running and healthy
  • Ping endpoint responds correctly
  • Auth endpoint not being exposed by outpost
  • Possible causes:
    1. Provider mode issue (forward_single vs forward_domain)
    2. Outpost not loading provider configuration
    3. Auth endpoint path incorrect for Authentik 2024.8.4
    4. Embedded outpost initialization incomplete

Forward Auth Config Attempted:

# NPM advanced config for home.schweitz.net
auth_request /outpost.goauthentik.io/auth/nginx;
error_page 401 = @goauthentik_proxy_signin;

location /outpost.goauthentik.io {
    proxy_pass https://localhost:9444/outpost.goauthentik.io;
    proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
    # ... (additional headers)
}

location @goauthentik_proxy_signin {
    internal;
    return 302 /outpost.goauthentik.io/start?rd=$request_uri;
}

Config Reverted:

  • Restored original NPM config for home.schweitz.net
  • Organizr accessible without SSO (for now)
  • Backup saved: /data/nginx/proxy_host/2.conf.backup

Technical Details

API Usage

Successfully used Authentik's REST API for automation:

# Created temporary API token
Token: dbc4eda544fd141a015b1ad1ec42955a4f6666fd22456a88c6f6402afa3107d1
Duration: 1 hour
User: akadmin

# API Endpoints Used:
POST /api/v3/providers/proxy/          # Create provider
POST /api/v3/core/applications/        # Create application
PATCH /api/v3/outposts/instances/{id}/ # Assign provider to outpost
GET /api/v3/flows/instances/           # List flows

Database Operations

-- Created authentik database and user
CREATE DATABASE authentik;
CREATE USER authentik_user WITH PASSWORD 'F//j0ktck7cX06Vfgh0YXceONOtlSsHvadqROICeDx8=';
GRANT ALL PRIVILEGES ON DATABASE authentik TO authentik_user;
GRANT ALL ON SCHEMA public TO authentik_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO authentik_user;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO authentik_user;

-- Verified user creation
SELECT id, username, email, is_active, type
FROM authentik_core_user
WHERE email = 'jpmschweitzer@gmail.com';
-- Result: id=5, type=external, is_active=t

-- Checked OAuth source
SELECT slug, name, enabled, provider_type
FROM authentik_core_source s
LEFT JOIN authentik_sources_oauth_oauthsource o
ON s.policybindingmodel_ptr_id = o.source_ptr_id;
-- Result: slug=google, enabled=t, provider_type=google

Memory Optimization Success

Previous Failed Deployment:

  • Memory usage: 3-5GB
  • Separate PostgreSQL instance: ~1GB
  • Separate Redis instance: ~100MB
  • No resource limits

Current Deployment:

$ docker stats authentik-server authentik-worker --no-stream
NAME               CPU %     MEM USAGE / LIMIT   MEM %
authentik-server   0.52%     291.1MiB / 512MiB   56.85%
authentik-worker   2.87%     271.9MiB / 384MiB   70.80%
Total: ~563MB

Savings: 82-88% reduction
Strategy:
  - Shared PostgreSQL (no dedicated instance)
  - Shared Redis (no dedicated instance)
  - Resource limits enforced
  - Single worker with 2 threads
  - Disabled: avatars, error reporting, footer links
  - Log level: warning

Files Modified

  1. stacks/authentik.yml - Created

    • Authentik server and worker configuration
    • Shared infrastructure connections
    • Resource limits and health checks
    • Port mappings: 9000, 9444

  2. NPM Database - Modified

    • Created proxy host for auth.schweitz.net
    • Attempted forward auth config (reverted)

  3. PostgreSQL - Modified

    • Created authentik database
    • Created authentik_user with grants

  4. STATUS.md - Updated

    • Version: 0.8.0-authentik-sso
    • Active work: Security & SSO Implementation
    • Added Milestone 1 & 2 accomplishments
    • Documented Milestone 3 blocker

Known Issues

1. Embedded Outpost Auth Endpoint Not Working

Symptom:

curl -k https://localhost:9444/outpost.goauthentik.io/auth/nginx
HTTP/1.1 404 Not Found

Impact:

  • Cannot configure forward authentication for applications
  • NPM forward auth results in 500 errors
  • Applications remain unprotected

Possible Solutions:

  1. Change provider mode:

    # Update via Authentik UI: Applications → Providers → Organizr Proxy
mode: "forward_domain"  # instead of "forward_single"
cookie_domain: "schweitz.net"

  2. Deploy standalone outpost:

    # Add to authentik.yml or separate stack
authentik-proxy:
  image: ghcr.io/goauthentik/proxy:2024.8.4
  environment:
    AUTHENTIK_HOST: https://auth.schweitz.net
    AUTHENTIK_TOKEN: <outpost-token>
  ports:
    - "9443:9443"

  3. Wait for full initialization:

    • Monitor logs: docker logs -f authentik-server
    • Check outpost status in Authentik UI: System → Outposts
    • Verify provider assignment

  4. Investigate version compatibility:

    • Authentik 2024.8.4 embedded outpost behavior
    • Check if auth endpoint requires specific configuration
    • Review Authentik documentation for forward auth setup

2. NPM Configuration Persistence

Issue:

  • Database updates don't trigger nginx config regeneration
  • Manual nginx file editing required
  • Changes lost on NPM restart/update

Workaround:

  • Update via NPM UI instead of database direct modification
  • Keep backup of custom nginx configs
  • Document config in code/scripts for reproducibility

Next Steps

Immediate (Milestone 3 Completion)

  1. Investigate Outpost Configuration:

    • Check Authentik UI: System → Outposts → authentik Embedded Outpost
    • Verify provider is assigned and status is healthy
    • Review outpost logs for errors

  2. Try Provider Mode Change:

    • Update Organizr Proxy provider to forward_domain mode
    • Add cookie_domain: schweitz.net
    • Restart Authentik containers
    • Test auth endpoint again

  3. Alternative: Deploy Standalone Outpost:

    • Create outpost stack configuration
    • Generate outpost token in Authentik UI
    • Deploy container and test auth endpoint

  4. Test Forward Auth:

    • Once auth endpoint works, apply NPM config
    • Test redirect to Authentik login
    • Verify SSO session persistence
    • Check for redirect loops

Future Milestones (from security-implementation-plan.md)

  • M4: Protect Core API with OIDC
  • M5: Protect remaining services (9 services)
    • Jellyfin, Nextcloud, Gitea, Portainer, NPM, Uptime Kuma, Open WebUI, Netdata, Headscale
  • M6: Documentation and rollback procedures

Lessons Learned

What Went Well

  1. Shared Infrastructure Approach:

    • Massive memory savings (80-90% reduction)
    • Easier management (single PostgreSQL/Redis)
    • Successful from day 1

  2. API-Driven Configuration:

    • Faster than UI clicks
    • Reproducible and documentable
    • Can be scripted for future deployments

  3. Incremental Testing:

    • Validated each component before moving forward
    • Caught issues early (health checks, database permissions)
    • Easy to rollback when issues encountered

  4. Documentation During Implementation:

    • Captured decisions and solutions in real-time
    • Easier to resume work later
    • Helpful for troubleshooting

What Could Be Improved

  1. Version Research:

    • Should have checked Authentik 2024.8.4 embedded outpost capabilities first
    • Version 2024.10+ has redirect loop issues (documented in security plan)
    • Tradeoff: stability vs features

  2. NPM Configuration Method:

    • Direct database edits don't trigger config regeneration
    • Should have used NPM UI from start
    • Need better automation for NPM config management

  3. Testing Approach:

    • Should have tested outpost endpoints before configuring NPM
    • Could have saved time on troubleshooting
    • Need outpost validation checklist

  4. Initialization Timing:

    • Didn't account for embedded outpost startup delay
    • Should wait for full health before testing endpoints
    • Need patience with complex distributed systems

References

Session End Status:

  • ✅ Authentik deployed and accessible
  • ✅ Google OAuth fully functional
  • ⚠️ Forward auth blocked on outpost initialization
  • 🔄 Investigation continuing in next session
Reference in New Issue View Git Blame Copy Permalink