jpmschweitzer/portainer-core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

security rework and memory optimilizations.

Browse Source
This commit is contained in:
jpmschweitzer
2025-11-17 08:48:00 +01:00
parent 72a653f494
commit e8eb2e954c
55 changed files with 8301 additions and 245 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/AUTHENTIK_FORWARD_AUTH_DEPLOYMENT.md
+373
View File
@@ -0,0 +1,373 @@
# Authentik Forward Authentication Deployment Summary
## Overview
Successfully deployed Authentik forward authentication to all NPM-managed domains. All protected sites now require SSO authentication via Authentik before access is granted.
## Deployment Date
2025-11-16
## Architecture
### Components
1. **Authentik Server** (`authentik-server`)
- Port: 9000
- Container: `authentik-server`
- Purpose: Main authentication server
2. **Authentik Proxy Outpost** (`authentik-proxy-outpost`)
- Port: 9001 (HTTP), 9300 (Metrics)
- Container: `authentik-proxy-outpost`
- Purpose: Forward authentication endpoint for NPM
- Network: host mode (to match NPM)
- Image: `ghcr.io/goauthentik/proxy:latest`
3. **Nginx Proxy Manager** (`nginx-proxy-manager`)
- Ports: 80, 81, 443
- Purpose: Reverse proxy with SSL termination
- Forward auth endpoint: `http://192.168.86.149:9001/outpost.goauthentik.io/auth/nginx`
### Authentication Flow
```
1. User → https://home.schweitz.net
2. NPM → auth_request to http://192.168.86.149:9001/outpost.goauthentik.io/auth/nginx
3. Outpost → validates session with Authentik Server (port 9000)
4. If not authenticated:
a. NPM → redirects to /outpost.goauthentik.io/start?rd=https://home.schweitz.net/
b. Outpost → redirects to https://auth.schweitz.net/if/flow/...
c. User → logs in via Google OAuth
d. Authentik → sets session cookie
e. Redirect → back to https://home.schweitz.net/
5. If authenticated:
a. Request passes through with user headers
```
## Protected Domains
All NPM proxy hosts now require authentication:
1. **192.168.86.149** - Direct IP access
2. **amp.schweitz.net** - AMP Server
3. **cloud.schweitz.net** - Nextcloud
4. **code.schweitz.net** - VS Code Server
5. **git.schweitz.net** - Gitea
6. **home.schweitz.net** - Home Assistant/Dashboard
7. **media.schweitz.net** - Media Server
8. **tatlock.schweitz.net** - Tatlock Services
9. **tower-of-joy** - Tower of Joy Services
### Excluded Domains
- **auth.schweitz.net** - Authentik itself (cannot protect the auth provider)
## Configuration Details
### Authentik Proxy Provider
- **Name**: `npm-forward-auth-provider`
- **ID**: 2
- **Mode**: `forward_single`
- **External Host**: `https://auth.schweitz.net`
- **Token Validity**: 480 minutes (8 hours)
- **Session Duration**: 480 minutes (8 hours)
- **Rolling Expiration**: Active (sessions extend on use)
### Authentik Outpost
- **Name**: `npm-forward-auth-outpost`
- **ID**: `5fbba1f4-e50f-4d3f-94e8-8d9f30bf7a51`
- **Type**: `proxy`
- **Provider**: `npm-forward-auth-provider` (ID: 2)
- **Token**: `aVJXnn3I44NhTYRYxGtbP3kJLEKQrTEMwu9IPVrhzN3NZV4vakgztcEf2L44`
- **Container**: `authentik-proxy-outpost`
- **Listen**: `0.0.0.0:9001` (HTTP), `0.0.0.0:9300` (metrics)
### NPM Configuration
Each protected proxy host has this advanced nginx configuration:
```nginx
# Authentik Forward Authentication
# Send authentication requests to Authentik
auth_request /outpost.goauthentik.io/auth/nginx;
# Preserve authentication cookies
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# Get user information from Authentik
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
auth_request_set $authentik_name $upstream_http_x_authentik_name;
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;
# Pass user info to backend
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-groups $authentik_groups;
proxy_set_header X-authentik-email $authentik_email;
proxy_set_header X-authentik-name $authentik_name;
proxy_set_header X-authentik-uid $authentik_uid;
# On authentication failure, redirect to Authentik login
error_page 401 = @authentik_proxy_signin;
location @authentik_proxy_signin {
internal;
add_header Set-Cookie $auth_cookie;
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}
# Authentik authentication endpoint
location /outpost.goauthentik.io {
proxy_pass http://192.168.86.149:9001/outpost.goauthentik.io;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header Host $host;
}
```
## Deployment Steps Taken
### 1. Created Proxy Provider
```bash
docker exec core-api /venv/bin/python /app/setup_authentik_forward_auth.py
```
- Created `npm-forward-auth-provider` proxy provider (ID: 2)
- Created `NPM Forward Auth` application (slug: `npm-forward-auth`)
- Updated embedded outpost to include provider
### 2. Enabled Forward Auth on NPM Hosts
```bash
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
```
- Added forward auth configuration to all 9 NPM proxy hosts
- Initially pointed to `http://192.168.86.149:9000` (incorrect)
### 3. Created Proxy Outpost
```bash
python3 -c "..." # Created outpost via Authentik API
```
- Created `npm-forward-auth-outpost` (ID: `5fbba1f4-e50f-4d3f-94e8-8d9f30bf7a51`)
- Authentik auto-created container via Docker service connection
- Retrieved token from auto-created container environment variables
### 4. Deployed Proxy Outpost Container
```bash
docker run -d \
--name authentik-proxy-outpost \
--restart unless-stopped \
--network host \
-e AUTHENTIK_HOST=http://192.168.86.149:9000 \
-e AUTHENTIK_TOKEN=aVJXnn3I44NhTYRYxGtbP3kJLEKQrTEMwu9IPVrhzN3NZV4vakgztcEf2L44 \
-e AUTHENTIK_LISTEN__HTTP=0.0.0.0:9001 \
-e AUTHENTIK_LISTEN__METRICS=0.0.0.0:9300 \
ghcr.io/goauthentik/proxy:latest
```
### 5. Updated NPM Hosts to Port 9001
```bash
docker exec core-api /venv/bin/python /app/fix_port_to_9001.py
```
- Updated all 9 hosts to point to `http://192.168.86.149:9001` instead of port 9000
## Verification
### Testing Forward Auth
```bash
# Test ping endpoint
curl http://192.168.86.149:9001/outpost.goauthentik.io/ping
# Expected: 204 No Content
# Test unauthenticated request
curl -I http://home.schweitz.net
# Expected: 302 redirect to /outpost.goauthentik.io/start?rd=...
```
### Logs
```bash
# Outpost logs
docker logs authentik-proxy-outpost
# Authentik server logs
docker logs authentik-server
# NPM logs
docker logs nginx-proxy-manager
```
## Token & Session Management
### Token Validity
- **Initial validity**: 480 minutes (8 hours)
- **Rolling expiration**: Token stays active on use
- **Idle timeout**: 8 hours after last activity
- **Explicit logout**: Token invalidated immediately at https://auth.schweitz.net/if/user/#/settings
### Session Behavior
- **Single login** protects all domains under `*.schweitz.net`
- **Cookie domain**: Shared across all sites (set by individual domain)
- **Persistent**: Survives browser restarts until expiration/logout
- **Secure**: HTTPS-only, HttpOnly flag set
## User Management
### Adding Users
1. Go to https://auth.schweitz.net/if/admin/
2. Navigate to **Directory** → **Users**
3. Click **Create** → **Create and enroll user**
4. Enter user details
5. Send enrollment invite
### Current Access Control
**All authenticated users** can access all protected sites. To restrict:
1. Go to https://auth.schweitz.net/if/admin/#/core/applications
2. Select **NPM Forward Auth** application
3. Go to **Policy / Group / User Bindings**
4. Add specific users or groups
## Troubleshooting
### Check Outpost Status
```bash
docker ps | grep authentik-proxy-outpost
docker logs authentik-proxy-outpost --tail 50
```
### Check Authentik Server
```bash
docker logs authentik-server --tail 50 | grep -i outpost
```
### Test Endpoints
```bash
# Ping (should return 204)
curl -I http://192.168.86.149:9001/outpost.goauthentik.io/ping
# Auth endpoint (needs proper headers from nginx)
curl -I http://home.schweitz.net
```
### Common Issues
1. **502 Bad Gateway**: Outpost not running or not reachable
- Check: `docker ps | grep authentik-proxy-outpost`
- Fix: Restart outpost container
2. **Redirect Loop**: Misconfigured nginx or outpost
- Check: NPM advanced config points to port 9001
- Check: Outpost logs for errors
3. **Session Expires Too Quickly**: Token validity too short
- Check: Provider token_validity setting (should be 480 minutes)
## Automation Scripts
### Enable Forward Auth on New Sites
```python
from src.clients.npm_client import get_npm_client
npm = get_npm_client()
proxy = await npm.create_proxy_host(
domain_names=["newsite.schweitz.net"],
forward_host="backend-container",
forward_port=8080,
ssl_enabled=True
)
await npm.enable_authentik_forward_auth(proxy["id"])
```
### Bulk Operations
```bash
# Enable on all hosts
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
# Update to port 9001
docker exec core-api /venv/bin/python /app/fix_port_to_9001.py
```
## Security Considerations
1. ✓ **HTTPS Required**: All sites use SSL termination at NPM
2. ✓ **Secure Cookies**: HttpOnly and Secure flags prevent XSS/MITM
3. ✓ **Token Rotation**: Tokens extend on activity for security
4. ✓ **Audit Logging**: Authentik logs all authentication events
5. ⚠ **MFA**: Not yet enabled (future enhancement)
## Monitoring
### Health Checks
```bash
# Authentik server
curl http://192.168.86.149:9000/-/health/live/
# Proxy outpost
curl http://192.168.86.149:9001/outpost.goauthentik.io/ping
# Metrics
curl http://192.168.86.149:9300/metrics
```
### Authentik Admin Dashboard
- **URL**: https://auth.schweitz.net/if/admin/
- **Events**: System → Events (view login attempts, failures)
- **Active Sessions**: System → Tokens
- **Outpost Status**: System → Outposts
## Future Enhancements
1. **Multi-Factor Authentication**: Enable TOTP/SMS for high-security sites
2. **Per-Site Access Control**: Different user groups for different domains
3. **Rate Limiting**: Prevent brute force attacks
4. **IP Whitelisting**: Allow certain IPs without auth
5. **API Key Support**: Service-to-service authentication
6. **Integration with Core API**: Expose auth status via REST API
## Related Documentation
- **Main Documentation**: `FORWARD_AUTH_CONFIGURATION.md`
- **SSO Progress**: `SSO_IMPLEMENTATION_PROGRESS.md`
- **Security Plan**: `security-implementation-plan.md`
- **Authentik Docs**: https://docs.goauthentik.io/docs/providers/proxy/
## Notes
- The embedded Authentik outpost (ID: `fbbefe20-b2e1-4706-8533-7bfece51989c`) is for Authentik's own web UI, not for external forward auth
- Forward auth requires a separate proxy outpost container listening on its own port (9001)
- The outpost automatically created by Authentik via Docker service connection had the wrong port configuration
- We manually deployed the outpost with correct settings using `docker run`
- NPM uses host network mode, so the outpost must also use host mode or be accessible via host IP:port
- Token keys are only visible once during creation in the Authentik UI; we retrieved it from the auto-created container's environment variables
## Completion Status
✅ **COMPLETE** - All protected sites now require Authentik authentication
**Deployment successful as of 2025-11-16 15:41 UTC**
docs/FORWARD_AUTH_CONFIGURATION.md
+259
View File
@@ -0,0 +1,259 @@
# Authentik Forward Authentication Configuration
## Overview
All NPM-managed domains now use Authentik for centralized Single Sign-On (SSO) authentication. When users access any protected site, they are automatically redirected to Authentik for login, then returned to their original destination.
## Authentication Flow
1. **User accesses protected site** (e.g., https://portainer.schweitz.net)
2. **NPM checks authentication** via Authentik forward auth
3. **If not authenticated**: Redirect to https://auth.schweitz.net/outpost.goauthentik.io/start
4. **User logs in** with Google (via Authentik)
5. **Authentik sets session cookie** (8-hour validity)
6. **Redirect back** to originally requested URL
7. **Subsequent requests**: Automatically authenticated (no login required)
## Token & Session Configuration
### Token Validity
- **Initial validity**: 8 hours (480 minutes)
- **Activity-based extension**: Token stays active on use (rolling expiration)
- **Idle timeout**: 8 hours after last activity
- **Explicit logout**: Token invalidated immediately
### Session Behavior
- **Single login** protects all domains under `*.schweitz.net`
- **Cookie domain**: Shared across all sites
- **Persistent**: Survives browser restarts (until expiration/logout)
- **Secure**: HTTPS-only, HttpOnly flag set
## Protected Domains
The following domains are protected with Authentik forward auth:
1. **192.168.86.149** - Direct IP access
2. **amp.schweitz.net** - AMP Server
3. **cloud.schweitz.net** - Nextcloud
4. **code.schweitz.net** - VS Code Server
5. **git.schweitz.net** - Gitea
6. **home.schweitz.net** - Home Assistant/Dashboard
7. **media.schweitz.net** - Media Server
8. **tatlock.schweitz.net** - Tatlock Services
9. **tower-of-joy** - Tower of Joy Services
### Unprotected Domains
- **auth.schweitz.net** - Authentik itself (cannot protect the auth provider)
## User Management
### Adding Users
1. Go to https://auth.schweitz.net/if/admin/
2. Navigate to **Directory** → **Users**
3. Click **Create** → **Create and enroll user**
4. Enter user details
5. Send enrollment invite (they'll set up Google OAuth)
### User Access Control
Currently, **all authenticated users** can access protected sites. To restrict access:
1. Go to https://auth.schweitz.net/if/admin/#/core/applications
2. Select **NPM Forward Auth** application
3. Go to **Policy / Group / User Bindings**
4. Add specific users or groups
### Group-Based Access (Future)
You can create groups and assign different access levels:
- `admin` - Full access to all sites
- `family` - Access to media, home
- `developers` - Access to code, git
## Logout
Users can log out at: https://auth.schweitz.net/if/user/#/settings
Click **Sign Out** to invalidate the session across all protected sites.
## Technical Implementation
### Authentik Components
1. **Proxy Provider** (`npm-forward-auth-provider`)
- Mode: `forward_single`
- External Host: `https://auth.schweitz.net`
- Token Validity: 480 minutes
2. **Application** (`NPM Forward Auth`)
- Links provider to user interface
- Accessible at: https://auth.schweitz.net
3. **Outpost** (`authentik Embedded Outpost`)
- Handles authentication requests from NPM
- Endpoint: `http://authentik-server:9000/outpost.goauthentik.io`
### NPM Configuration
Each proxy host has advanced nginx configuration:
```nginx
# Forward auth to Authentik
auth_request /outpost.goauthentik.io/auth/nginx;
# Preserve cookies
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# Extract user info
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
# Pass to backend
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-email $authentik_email;
# Redirect on auth failure
error_page 401 = @authentik_proxy_signin;
location @authentik_proxy_signin {
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}
# Auth endpoint
location /outpost.goauthentik.io {
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_pass_request_body off;
}
```
## Backend Application Integration
Protected applications receive user information via headers:
- `X-authentik-username` - Username
- `X-authentik-email` - Email address
- `X-authentik-groups` - Comma-separated groups
- `X-authentik-name` - Full name
- `X-authentik-uid` - Unique user ID
Applications can use these headers for:
- Displaying user info
- Audit logging
- Role-based access control
- Personalization
## Troubleshooting
### User Can't Log In
1. **Check Authentik status**: `docker ps | grep authentik`
2. **Check Authentik logs**: `docker logs authentik-server`
3. **Verify Google OAuth**:
- Go to https://auth.schweitz.net/if/admin/#/core/sources
- Ensure Google source is enabled
4. **Check user exists**:
- Go to https://auth.schweitz.net/if/admin/#/identity/users
- Verify user account is active
### Redirect Loop
If users get stuck in a redirect loop:
1. **Clear browser cookies** for `*.schweitz.net`
2. **Check NPM config**: Ensure `/outpost.goauthentik.io` location exists
3. **Restart NPM**: `docker restart npm`
4. **Check outpost**: Verify provider is assigned to outpost
### 502 Bad Gateway
If auth requests fail:
1. **Check Authentik container**: `docker ps | grep authentik`
2. **Verify network**: Ensure NPM can reach `authentik-server:9000`
3. **Check NPM logs**: `docker logs npm`
### Session Expires Too Quickly
If users are logged out unexpectedly:
1. **Check token validity**: Should be 480 minutes (8 hours)
2. **Verify rolling expiration**: Active use should extend session
3. **Check system time**: Ensure Docker host time is correct
## Automation
### Adding Forward Auth to New Sites
When creating new proxy hosts via core-api:
```python
from src.clients.npm_client import get_npm_client
npm = get_npm_client()
# Create proxy host
proxy = await npm.create_proxy_host(
domain_names=["newsite.schweitz.net"],
forward_host="backend-container",
forward_port=8080,
ssl_enabled=True
)
# Enable forward auth
await npm.enable_authentik_forward_auth(proxy["id"])
```
### Bulk Enable/Disable
To enable on all hosts:
```bash
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
```
## Security Considerations
1. **HTTPS Required**: Forward auth should only be used with HTTPS
2. **Secure Cookies**: HttpOnly and Secure flags prevent XSS/MITM
3. **Token Rotation**: Tokens are rotated on activity for security
4. **Audit Logging**: Authentik logs all authentication events
5. **MFA Support**: Can be enabled in Authentik for additional security
## Monitoring
### Check Authentication Status
```bash
# Check Authentik health
curl http://authentik-server:9000/-/health/live/
# Check active sessions (in Authentik admin)
# https://auth.schweitz.net/if/admin/#/events/log
```
### Monitor Failed Attempts
Go to **System** → **Events** in Authentik admin to see:
- Failed login attempts
- Successful authentications
- Token expirations
- Policy violations
## Future Enhancements
1. **Per-Site Access Control**: Different user groups for different domains
2. **Multi-Factor Authentication**: SMS/TOTP for high-security sites
3. **Rate Limiting**: Prevent brute force attacks
4. **IP Whitelisting**: Allow certain IPs without auth
5. **API Key Support**: Service-to-service authentication
## Related Documentation
- **Security Implementation Plan**: `security-implementation-plan.md`
- **SSO Progress**: `SSO_IMPLEMENTATION_PROGRESS.md`
- **OIDC Configuration**: `OIDC_CONFIGURATION.md`
- **Authentik Docs**: https://docs.goauthentik.io/docs/providers/proxy/
docs/OIDC_CONFIGURATION.md
+167
View File
@@ -0,0 +1,167 @@
# OIDC Authentication Configuration
## Overview
Core-API has been configured with OIDC authentication support using Authentik as the identity provider. This provides secure authentication for infrastructure management endpoints.
## Current Status
**OIDC is currently DISABLED** (`oidc_enabled=false` in config.py)
This allows:
- Internal services to access the API without authentication
- Direct API access from the Docker network
- Backward compatibility with existing integrations
## Authentik Configuration
### Provider Details
- **Provider Name**: `core-api-provider`
- **Provider ID**: `1`
- **Client ID**: `core-api`
- **Client Secret**: `WfsY0iIVOmO1wvXn1u8jwa08eiQsn2yVf9toBwVyEqps3M98nOZwJMgDOqH5PNZGM6wIxKTwlwYemtaOUg9u5bocd0EqShyoe4yhBQq4SDd0svyArILHZeGHFVEqUi4d`
- **Issuer**: `https://auth.schweitz.net/application/o/core-api/`
### Application Details
- **Application Name**: `Core API`
- **Slug**: `core-api`
- **Launch URL**: `http://localhost:8083/docs`
### Redirect URIs
- `http://localhost:8083/docs/oauth2-redirect`
- `https://core-api.schweitz.net/docs/oauth2-redirect`
- `http://192.168.86.149:8083/docs/oauth2-redirect`
## Enabling OIDC Authentication
When ready to enable OIDC authentication for external access:
### 1. Update core-api Configuration
In `/home/jpmschweitzer/Projects/portainer-core/services/core-api/src/config.py`:
```python
# OIDC Authentication (Authentik)
oidc_enabled: bool = True # Change from False to True
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
oidc_client_secret: str = "WfsY0iIVOmO1wvXn1u8jwa08eiQsn2yVf9toBwVyEqps3M98nOZwJMgDOqH5PNZGM6wIxKTwlwYemtaOUg9u5bocd0EqShyoe4yhBQq4SDd0svyArILHZeGHFVEqUi4d"
```
### 2. Restart core-api
```bash
docker restart core-api
```
### 3. Test Authentication
1. Visit: http://localhost:8083/docs
2. Click the "Authorize" button
3. Log in with Google via Authentik
4. Access protected endpoints
## Protected Endpoints
When OIDC is enabled, the following endpoints require admin authentication:
### Infrastructure Management (Write Operations)
- `POST /infrastructure/services` - Deploy stack
- `PUT /infrastructure/services/{name}` - Update stack
- `DELETE /infrastructure/services/{name}` - Delete stack
- `POST /infrastructure/proxy` - Create proxy host
- `POST /infrastructure/services/{name}/stop` - Stop service
- `POST /infrastructure/services/{name}/start` - Start service
### Monitoring Management
- `POST /infrastructure/monitors` - Create monitor
- `PUT /infrastructure/monitors/{monitor_id}` - Update monitor
- `DELETE /infrastructure/monitors/{monitor_id}` - Delete monitor
### Read Endpoints (Public)
All GET endpoints remain publicly accessible:
- `/health` - Health check
- `/infrastructure/health` - Infrastructure health
- `/infrastructure/services` - List services
- `/infrastructure/ports` - List ports
- `/infrastructure/domains` - List domains
- `/infrastructure/monitors` - List monitors
## Internal Network Access
**Important**: After enabling OIDC for external users, internal services still need unrestricted access.
### Future Enhancement Options
1. **Network-based Authentication**
- Check if request originates from `docker-dataplane` network
- Allow requests from internal IPs without auth
- Require OIDC only for external requests
2. **Service Accounts**
- Create machine tokens for internal services
- Use Bearer token authentication for service-to-service calls
- Keep OIDC for user authentication
3. **NPM Proxy Layer**
- External domain (https://core-api.schweitz.net) → Requires Authentik SSO
- Internal access (http://core-api:8083) → No authentication
## Admin Groups
Users must be members of one of these Authentik groups to access protected endpoints:
- `admin`
- `authentik Admins`
Configure user group membership in Authentik admin panel:
https://auth.schweitz.net/if/admin/#/identity/users
## Authentik Management
- **Admin Panel**: https://auth.schweitz.net/if/admin/
- **Applications**: https://auth.schweitz.net/if/admin/#/core/applications
- **Providers**: https://auth.schweitz.net/if/admin/#/core/providers
## API Token Management
The Authentik API token used for automation is stored in:
`/home/jpmschweitzer/Projects/portainer-core/services/core-api/src/credentials.py`
```python
AUTHENTIK_CORE_API_TOKEN = "peXM0EzDv2Wiwycbfm3cE5O44IMAOR8ntUwKVP977yFvVopCzDlKY8tymlMM"
```
To create additional tokens: https://auth.schweitz.net/if/admin/#/identity/tokens
## Security Considerations
1. **Client Secret**: Stored in config.py (gitignored), consider moving to environment variable
2. **Token Validation**: Tokens are validated using JWKS from Authentik
3. **Token Expiry**: Access tokens valid for 60 minutes, refresh tokens for 30 days
4. **SSL**: All production endpoints should use HTTPS (via NPM)
5. **Admin Access**: Restrict admin group membership carefully
## Troubleshooting
### Login Issues
- Check Authentik service status: `docker ps | grep authentik`
- Check Authentik logs: `docker logs authentik-server`
- Verify redirect URIs match exactly in Authentik provider config
### Token Validation Errors
- Verify OIDC issuer URL is correct
- Check core-api logs: `docker logs core-api`
- Ensure Authentik is accessible from core-api container
### Permission Denied (403)
- Verify user is member of `admin` group in Authentik
- Check user claims in JWT token at https://jwt.io
## Related Documentation
- **Security Implementation Plan**: `/home/jpmschweitzer/Projects/portainer-core/security-implementation-plan.md`
- **SSO Progress**: `/home/jpmschweitzer/Projects/portainer-core/SSO_IMPLEMENTATION_PROGRESS.md`
- **Authentik Docs**: https://docs.goauthentik.io/
docs/SHARED_INFRASTRUCTURE_ARCHITECTURE.md
+292
View File
@@ -0,0 +1,292 @@
# Shared Infrastructure Architecture
**Purpose:** Centralized PostgreSQL and Redis services for all homelab stacks
**Benefits:** Resource efficiency, easier maintenance, unified backups, centralized monitoring
---
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────┐
│ Application Stacks │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │Authentik │ │ Gitea │ │ Future │ │ Future │ │
│ │ │ │ │ │ Stack │ │ Stack │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
└───────┼─────────────┼──────────────┼──────────────┼─────────┘
│ │ │ │
└─────────────┴──────────────┴──────────────┘
│
┌─────────────▼──────────────────────────────┐
│ Unified Data Plane Network │
│ (docker-dataplane) │
└─────────────┬──────────────────────────────┘
│
┌─────────────┴──────────────┐
│ │
┌────▼──────┐ ┌────────▼────┐
│PostgreSQL │ │ Redis │
│ Shared │ │ Shared │
│ │ │ │
│ Databases:│ │ DB 0: Cache │
│ - auth │ │ DB 1: Auth │
│ - gitea │ │ DB 2: Gitea │
│ - future │ │ DB 3-15: .. │
└───────────┘ └─────────────┘
```
---
## Design Principles
### 1. **Database Isolation**
- Each application gets its own PostgreSQL database within the shared instance
- Each application gets its own Redis database number (0-15)
- Separate credentials per application for security
### 2. **Network Architecture**
- **Unified network:** `docker-dataplane` (external, bridge)
- All application containers connect to this single network
- Simplified connectivity: services discover each other by container name
- Replaces per-stack networks (ai-dataplane, nextcloud-network, etc.)
### 3. **Resource Allocation**
- PostgreSQL: No hard limits (homelab resource availability)
- Redis: No hard limits (lightweight Alpine image)
- Shared instances more efficient than per-stack deployments
### 4. **Backup Strategy**
- Single PostgreSQL backup covers all databases
- Automated pg_dumpall for disaster recovery
- Redis persistence: AOF + RDB snapshots
### 5. **Security Model**
- Each app has dedicated PostgreSQL user with access only to its database
- Redis AUTH with per-database passwords (optional)
- Network-level isolation via Docker networks
---
## Database Allocation Plan
### PostgreSQL Databases
| Database Name | Application | User | Purpose |
|---------------|-------------|------|---------|
| `authentik` | Authentik | `authentik_user` | User/group/policy storage |
| `gitea` | Gitea | `gitea_user` | Git repos, users, issues |
| `future_app1` | TBD | `app1_user` | Reserved |
| `future_app2` | TBD | `app2_user` | Reserved |
**Note:** Existing services stay as-is:
- Nextcloud: MariaDB (existing, not migrated)
- Others can migrate over time if beneficial
### Redis Database Numbers
| DB# | Application | Purpose |
|-----|-------------|---------|
| 0 | Authentik | Sessions, cache, message queue |
| 1 | Available | Reserved for future applications |
| 2 | Available | Reserved for future applications |
| 3-15 | Available | Reserved for future applications |
**Note:** Each application uses a dedicated DB number to prevent key collisions while sharing the same Redis instance.
---
## Connection Configuration
### PostgreSQL Connection Strings
**From Docker containers:**
```
Host: postgres-shared
Port: 5432
Database: authentik
User: authentik_user
Password: <app-specific-password>
```
**From host:**
```
Host: localhost
Port: 5432
Database: authentik
User: authentik_user
Password: <app-specific-password>
```
### Redis Connection Strings
**From Docker containers:**
```
redis://redis-shared:6379/0 (for Authentik, DB 0)
redis://redis-shared:6379/1 (for future apps, DB 1)
redis://redis-shared:6379/2 (for future apps, DB 2)
```
**From host:**
```
redis://localhost:6379/0
```
---
## Migration Strategy
### Phase 1: Deploy Shared Infrastructure ✅ **COMPLETE**
1. ✅ Deployed `postgres-shared.yml` and `redis-shared.yml` via Portainer
2. ✅ Verified PostgreSQL 17 and Redis 7 running on docker-dataplane
3. ✅ Created initial databases and users (authentik, gitea)
4. ✅ Both services monitored via Uptime Kuma
### Phase 2: New Services (Authentik) 🚧 **IN PROGRESS**
1. ⏳ Deploy Authentik pointing to shared services
2. ⏳ Test thoroughly
3. ⏳ Validate no performance degradation
### Phase 3: Network Consolidation ✅ **COMPLETE**
1. ✅ All services migrated to docker-dataplane network
2. ✅ Removed 7 obsolete Docker networks
3. ✅ 18 containers on unified network for service discovery
### Phase 4: Migrate Existing Services (Optional)
1. **Gitea**: Already uses PostgreSQL
- Export existing database
- Create gitea database in shared PostgreSQL
- Import data
- Update Gitea stack to use shared PostgreSQL
- Remove old gitea-db container
2. **Other services**: Evaluate case-by-case
- Nextcloud: Keep MariaDB (complex migration, low benefit)
- Future services: Use shared from day 1
---
## Advantages
✅ **Resource Efficiency**
- One PostgreSQL instance: ~1GB RAM vs ~300MB per instance
- Saves ~700MB RAM per additional service using PostgreSQL
✅ **Operational Simplicity**
- Single backup process for all PostgreSQL databases
- Centralized monitoring and health checks
- Easier version upgrades (upgrade once, affects all)
✅ **Performance**
- Shared connection pooling
- Better resource utilization
- Optimized caching with shared Redis
✅ **Scalability**
- Add new applications without deploying new database instances
- Up to 15 Redis databases (more than enough for homelab)
---
## Disadvantages & Mitigations
⚠️ **Single Point of Failure**
- **Mitigation:** Health checks, automated restarts, regular backups
- **Acceptable for homelab:** VPN access ensures admin can fix issues
⚠️ **Resource Contention**
- **Mitigation:** PostgreSQL connection limits per database
- **Mitigation:** Redis max memory policy (LRU eviction)
- **Monitoring:** Track per-database usage
⚠️ **Version Lock-In**
- **Mitigation:** Use latest stable PostgreSQL version (17)
- **Mitigation:** Test upgrades in staging before production deployment
---
## Monitoring & Maintenance
### Health Checks
- PostgreSQL: `pg_isready` every 30s
- Redis: `redis-cli ping` every 30s
- Application connectivity tests
### Uptime Kuma Integration ✅ **DEPLOYED**
Both shared services are monitored via Uptime Kuma with automatic monitor creation through the Core API:
**PostgreSQL Monitor** (ID 20):
```bash
curl -X POST http://192.168.86.149:8083/infrastructure/monitors \
-H "Content-Type: application/json" \
-d '{
"type": "postgres",
"name": "PostgreSQL Shared",
"interval": 60,
"retryInterval": 60,
"maxretries": 3,
"notificationIDList": [],
"accepted_statuscodes": ["200-299"],
"databaseConnectionString": "postgres://postgres:<url-encoded-password>@postgres-shared:5432/postgres"
}'
```
**Redis Monitor** (ID 18):
```bash
curl -X POST http://192.168.86.149:8083/infrastructure/monitors \
-H "Content-Type: application/json" \
-d '{
"type": "port",
"name": "Redis Shared - Port Check",
"hostname": "redis-shared",
"port": 6379,
"interval": 60,
"retryInterval": 60,
"maxretries": 3,
"notificationIDList": [],
"accepted_statuscodes": ["200-299"]
}'
```
**Note:** When monitoring PostgreSQL with passwords containing special characters, URL-encode them (`/` → `%2F`, `=` → `%3D`).
### Backup Schedule
- **PostgreSQL:** Manual pg_dump to `/backups/` volume (automated backups pending)
- **Redis:** AOF persistence (real-time) enabled via `--appendonly yes`
### Performance Monitoring
- Query: `SELECT datname, numbackends FROM pg_stat_database;` (active connections)
- Redis: `INFO stats` (keyspace usage per database)
- Uptime Kuma dashboard: Real-time availability tracking
### Upgrade Path
1. Backup all databases
2. Test upgrade with docker-compose override
3. Deploy new version
4. Verify all applications connect successfully
5. Rollback if issues detected
---
## Implementation Status
1. ✅ Review architecture design
2. ✅ Create `postgres-shared.yml` and `redis-shared.yml` stacks
3. ✅ Deploy shared PostgreSQL 17 and Redis 7 via Portainer
4. ✅ Create initial databases (authentik, gitea)
5. ✅ Consolidate all services to docker-dataplane network
6. ✅ Implement Uptime Kuma monitoring via Core API
7. ✅ Document connection patterns and deployment procedures
8. ⏳ Update `authentik.yml` to use shared services (pending)
9. ⏳ Test Authentik with shared infrastructure (pending)
---
## Future Enhancements
- **PostgreSQL Read Replicas** (if needed for heavy read workloads)
- **Redis Sentinel** (high availability, probably overkill for homelab)
- **PgBouncer** (connection pooling if >100 connections needed)
- **Prometheus + Grafana** (metrics visualization)
docs/SSO_IMPLEMENTATION_PROGRESS.md
+115
View File
@@ -0,0 +1,115 @@
# SSO Implementation Progress Tracker
**Project:** Google OAuth SSO for Homelab Infrastructure
**Started:** 2025-11-15
**Status:** 🚧 Phase 1 - In Progress
---
## Overview
Implementing Single Sign-On (SSO) using:
- **Identity Provider:** Authentik
- **Authentication Source:** Google OAuth (Workspace + Gmail)
- **In-Scope Services:** core-api, Nextcloud, Jellyfin, Gitea, Open WebUI, Organizr, code-server
---
## Phase 1: Foundation (Week 1)
**Goal:** Deploy Authentik, configure Google OAuth, protect core-api
### Task Checklist
- [x] **1.1 Deploy Authentik Stack** *(In Progress)*
- [x] Create `stacks/authentik.yml`
- [x] Generate secrets (PostgreSQL password, Authentik secret key)
- [x] Create `.env.authentik` file
- [x] Create data directories
- [ ] Deploy via Portainer
- [ ] Verify services running (4/4: server, worker, postgresql, redis)
- [ ] Complete initial setup wizard
- [ ] Access admin portal
- [ ] **1.2 Configure NPM Proxy**
- [ ] Create proxy host: `auth.schweitz.net` → `authentik-server:9000`
- [ ] Enable SSL with Let's Encrypt
- [ ] Test HTTPS access
- [ ] Verify health endpoint
- [ ] **1.3 Google OAuth Setup**
- [ ] Create/configure Google Cloud Project
- [ ] Set up OAuth consent screen
- [ ] Create OAuth 2.0 credentials
- [ ] Note Client ID and Client Secret
- [ ] Configure authorized redirect URIs
- [ ] **1.4 Configure Google Source in Authentik**
- [ ] Add Google OAuth source
- [ ] Configure scopes: openid, email, profile
- [ ] Test login with Google Workspace account
- [ ] Test login with Gmail account
- [ ] Verify user profile synced
- [ ] **1.5 Implement core-api OIDC Authentication**
- [ ] Add dependencies: PyJWT, python-jose
- [ ] Create `src/auth/oidc.py` module
- [ ] Update `src/config.py` with OIDC settings
- [ ] Create Authentik OIDC provider for core-api
- [ ] Protect infrastructure endpoints
- [ ] Update OpenAPI docs with security scheme
- [ ] **1.6 Testing & Validation**
- [ ] Test unauthenticated API request (expect 401)
- [ ] Test authenticated API request with valid token
- [ ] Verify user claims available in endpoints
- [ ] Test token expiration handling
- [ ] Test admin-only endpoints
- [ ] Update widget for OAuth flow
---
## Progress Log
### 2025-11-15 - 21:20 CET
**[Completed]** Shared Infrastructure Architecture
- ✅ Designed shared PostgreSQL + Redis architecture
- ✅ Created `SHARED_INFRASTRUCTURE_ARCHITECTURE.md` documentation
- ✅ Created separate stacks for modularity:
- `postgres-shared.yml` (centralized database)
- `redis-shared.yml` (centralized cache)
- `authentik-shared.yml` (using shared backends)
- ✅ Generated secure credentials (all passwords 32-byte random)
- ✅ Created unified `docker-dataplane` network
- ✅ Created PostgreSQL init script for multi-database setup
- ✅ Created environment files:
- `.env.postgres` (PostgreSQL + app DB passwords)
- `.env.authentik-shared` (Authentik config)
**Architecture Benefits:**
- Resource savings: ~400MB RAM per service using shared infrastructure
- Centralized backups and monitoring
- Easier maintenance and upgrades
- Modular deployment (PostgreSQL and Redis as separate stacks)
**[Next]** Deploy shared infrastructure, then Authentik
---
## Next: Create Authentik Stack
Creating `stacks/authentik.yml` with:
- authentik-server
- authentik-worker
- PostgreSQL database
- Redis cache
Expected resources: ~500MB RAM, 1.5 CPU, 5GB storage
---
## Notes
- **Architecture Decision:** FastAPI native OIDC for core-api (not NPM forward auth)
- **Out of Scope:** Infrastructure tools, data providers, local services
- **Security:** All passwords/secrets via environment variables, not committed to git