roll back authentik login. removed and restore working state.
This commit is contained in:
@@ -1,373 +0,0 @@
|
||||
# Authentik Forward Authentication Deployment Summary
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully deployed Authentik forward authentication to all NPM-managed domains. All protected sites now require SSO authentication via Authentik before access is granted.
|
||||
|
||||
## Deployment Date
|
||||
|
||||
2025-11-16
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
1. **Authentik Server** (`authentik-server`)
|
||||
- Port: 9000
|
||||
- Container: `authentik-server`
|
||||
- Purpose: Main authentication server
|
||||
|
||||
2. **Authentik Proxy Outpost** (`authentik-proxy-outpost`)
|
||||
- Port: 9001 (HTTP), 9300 (Metrics)
|
||||
- Container: `authentik-proxy-outpost`
|
||||
- Purpose: Forward authentication endpoint for NPM
|
||||
- Network: host mode (to match NPM)
|
||||
- Image: `ghcr.io/goauthentik/proxy:latest`
|
||||
|
||||
3. **Nginx Proxy Manager** (`nginx-proxy-manager`)
|
||||
- Ports: 80, 81, 443
|
||||
- Purpose: Reverse proxy with SSL termination
|
||||
- Forward auth endpoint: `http://192.168.86.149:9001/outpost.goauthentik.io/auth/nginx`
|
||||
|
||||
### Authentication Flow
|
||||
|
||||
```
|
||||
1. User → https://home.schweitz.net
|
||||
2. NPM → auth_request to http://192.168.86.149:9001/outpost.goauthentik.io/auth/nginx
|
||||
3. Outpost → validates session with Authentik Server (port 9000)
|
||||
4. If not authenticated:
|
||||
a. NPM → redirects to /outpost.goauthentik.io/start?rd=https://home.schweitz.net/
|
||||
b. Outpost → redirects to https://auth.schweitz.net/if/flow/...
|
||||
c. User → logs in via Google OAuth
|
||||
d. Authentik → sets session cookie
|
||||
e. Redirect → back to https://home.schweitz.net/
|
||||
5. If authenticated:
|
||||
a. Request passes through with user headers
|
||||
```
|
||||
|
||||
## Protected Domains
|
||||
|
||||
All NPM proxy hosts now require authentication:
|
||||
|
||||
1. **192.168.86.149** - Direct IP access
|
||||
2. **amp.schweitz.net** - AMP Server
|
||||
3. **cloud.schweitz.net** - Nextcloud
|
||||
4. **code.schweitz.net** - VS Code Server
|
||||
5. **git.schweitz.net** - Gitea
|
||||
6. **home.schweitz.net** - Home Assistant/Dashboard
|
||||
7. **media.schweitz.net** - Media Server
|
||||
8. **tatlock.schweitz.net** - Tatlock Services
|
||||
9. **tower-of-joy** - Tower of Joy Services
|
||||
|
||||
### Excluded Domains
|
||||
|
||||
- **auth.schweitz.net** - Authentik itself (cannot protect the auth provider)
|
||||
|
||||
## Configuration Details
|
||||
|
||||
### Authentik Proxy Provider
|
||||
|
||||
- **Name**: `npm-forward-auth-provider`
|
||||
- **ID**: 2
|
||||
- **Mode**: `forward_single`
|
||||
- **External Host**: `https://auth.schweitz.net`
|
||||
- **Token Validity**: 480 minutes (8 hours)
|
||||
- **Session Duration**: 480 minutes (8 hours)
|
||||
- **Rolling Expiration**: Active (sessions extend on use)
|
||||
|
||||
### Authentik Outpost
|
||||
|
||||
- **Name**: `npm-forward-auth-outpost`
|
||||
- **ID**: `5fbba1f4-e50f-4d3f-94e8-8d9f30bf7a51`
|
||||
- **Type**: `proxy`
|
||||
- **Provider**: `npm-forward-auth-provider` (ID: 2)
|
||||
- **Token**: `aVJXnn3I44NhTYRYxGtbP3kJLEKQrTEMwu9IPVrhzN3NZV4vakgztcEf2L44`
|
||||
- **Container**: `authentik-proxy-outpost`
|
||||
- **Listen**: `0.0.0.0:9001` (HTTP), `0.0.0.0:9300` (metrics)
|
||||
|
||||
### NPM Configuration
|
||||
|
||||
Each protected proxy host has this advanced nginx configuration:
|
||||
|
||||
```nginx
|
||||
# Authentik Forward Authentication
|
||||
# Send authentication requests to Authentik
|
||||
auth_request /outpost.goauthentik.io/auth/nginx;
|
||||
|
||||
# Preserve authentication cookies
|
||||
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
||||
add_header Set-Cookie $auth_cookie;
|
||||
|
||||
# Get user information from Authentik
|
||||
auth_request_set $authentik_username $upstream_http_x_authentik_username;
|
||||
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
|
||||
auth_request_set $authentik_email $upstream_http_x_authentik_email;
|
||||
auth_request_set $authentik_name $upstream_http_x_authentik_name;
|
||||
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;
|
||||
|
||||
# Pass user info to backend
|
||||
proxy_set_header X-authentik-username $authentik_username;
|
||||
proxy_set_header X-authentik-groups $authentik_groups;
|
||||
proxy_set_header X-authentik-email $authentik_email;
|
||||
proxy_set_header X-authentik-name $authentik_name;
|
||||
proxy_set_header X-authentik-uid $authentik_uid;
|
||||
|
||||
# On authentication failure, redirect to Authentik login
|
||||
error_page 401 = @authentik_proxy_signin;
|
||||
|
||||
location @authentik_proxy_signin {
|
||||
internal;
|
||||
add_header Set-Cookie $auth_cookie;
|
||||
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
|
||||
}
|
||||
|
||||
# Authentik authentication endpoint
|
||||
location /outpost.goauthentik.io {
|
||||
proxy_pass http://192.168.86.149:9001/outpost.goauthentik.io;
|
||||
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
```
|
||||
|
||||
## Deployment Steps Taken
|
||||
|
||||
### 1. Created Proxy Provider
|
||||
|
||||
```bash
|
||||
docker exec core-api /venv/bin/python /app/setup_authentik_forward_auth.py
|
||||
```
|
||||
|
||||
- Created `npm-forward-auth-provider` proxy provider (ID: 2)
|
||||
- Created `NPM Forward Auth` application (slug: `npm-forward-auth`)
|
||||
- Updated embedded outpost to include provider
|
||||
|
||||
### 2. Enabled Forward Auth on NPM Hosts
|
||||
|
||||
```bash
|
||||
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
|
||||
```
|
||||
|
||||
- Added forward auth configuration to all 9 NPM proxy hosts
|
||||
- Initially pointed to `http://192.168.86.149:9000` (incorrect)
|
||||
|
||||
### 3. Created Proxy Outpost
|
||||
|
||||
```bash
|
||||
python3 -c "..." # Created outpost via Authentik API
|
||||
```
|
||||
|
||||
- Created `npm-forward-auth-outpost` (ID: `5fbba1f4-e50f-4d3f-94e8-8d9f30bf7a51`)
|
||||
- Authentik auto-created container via Docker service connection
|
||||
- Retrieved token from auto-created container environment variables
|
||||
|
||||
### 4. Deployed Proxy Outpost Container
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name authentik-proxy-outpost \
|
||||
--restart unless-stopped \
|
||||
--network host \
|
||||
-e AUTHENTIK_HOST=http://192.168.86.149:9000 \
|
||||
-e AUTHENTIK_TOKEN=aVJXnn3I44NhTYRYxGtbP3kJLEKQrTEMwu9IPVrhzN3NZV4vakgztcEf2L44 \
|
||||
-e AUTHENTIK_LISTEN__HTTP=0.0.0.0:9001 \
|
||||
-e AUTHENTIK_LISTEN__METRICS=0.0.0.0:9300 \
|
||||
ghcr.io/goauthentik/proxy:latest
|
||||
```
|
||||
|
||||
### 5. Updated NPM Hosts to Port 9001
|
||||
|
||||
```bash
|
||||
docker exec core-api /venv/bin/python /app/fix_port_to_9001.py
|
||||
```
|
||||
|
||||
- Updated all 9 hosts to point to `http://192.168.86.149:9001` instead of port 9000
|
||||
|
||||
## Verification
|
||||
|
||||
### Testing Forward Auth
|
||||
|
||||
```bash
|
||||
# Test ping endpoint
|
||||
curl http://192.168.86.149:9001/outpost.goauthentik.io/ping
|
||||
# Expected: 204 No Content
|
||||
|
||||
# Test unauthenticated request
|
||||
curl -I http://home.schweitz.net
|
||||
# Expected: 302 redirect to /outpost.goauthentik.io/start?rd=...
|
||||
```
|
||||
|
||||
### Logs
|
||||
|
||||
```bash
|
||||
# Outpost logs
|
||||
docker logs authentik-proxy-outpost
|
||||
|
||||
# Authentik server logs
|
||||
docker logs authentik-server
|
||||
|
||||
# NPM logs
|
||||
docker logs nginx-proxy-manager
|
||||
```
|
||||
|
||||
## Token & Session Management
|
||||
|
||||
### Token Validity
|
||||
|
||||
- **Initial validity**: 480 minutes (8 hours)
|
||||
- **Rolling expiration**: Token stays active on use
|
||||
- **Idle timeout**: 8 hours after last activity
|
||||
- **Explicit logout**: Token invalidated immediately at https://auth.schweitz.net/if/user/#/settings
|
||||
|
||||
### Session Behavior
|
||||
|
||||
- **Single login** protects all domains under `*.schweitz.net`
|
||||
- **Cookie domain**: Shared across all sites (set by individual domain)
|
||||
- **Persistent**: Survives browser restarts until expiration/logout
|
||||
- **Secure**: HTTPS-only, HttpOnly flag set
|
||||
|
||||
## User Management
|
||||
|
||||
### Adding Users
|
||||
|
||||
1. Go to https://auth.schweitz.net/if/admin/
|
||||
2. Navigate to **Directory** → **Users**
|
||||
3. Click **Create** → **Create and enroll user**
|
||||
4. Enter user details
|
||||
5. Send enrollment invite
|
||||
|
||||
### Current Access Control
|
||||
|
||||
**All authenticated users** can access all protected sites. To restrict:
|
||||
|
||||
1. Go to https://auth.schweitz.net/if/admin/#/core/applications
|
||||
2. Select **NPM Forward Auth** application
|
||||
3. Go to **Policy / Group / User Bindings**
|
||||
4. Add specific users or groups
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Check Outpost Status
|
||||
|
||||
```bash
|
||||
docker ps | grep authentik-proxy-outpost
|
||||
docker logs authentik-proxy-outpost --tail 50
|
||||
```
|
||||
|
||||
### Check Authentik Server
|
||||
|
||||
```bash
|
||||
docker logs authentik-server --tail 50 | grep -i outpost
|
||||
```
|
||||
|
||||
### Test Endpoints
|
||||
|
||||
```bash
|
||||
# Ping (should return 204)
|
||||
curl -I http://192.168.86.149:9001/outpost.goauthentik.io/ping
|
||||
|
||||
# Auth endpoint (needs proper headers from nginx)
|
||||
curl -I http://home.schweitz.net
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **502 Bad Gateway**: Outpost not running or not reachable
|
||||
- Check: `docker ps | grep authentik-proxy-outpost`
|
||||
- Fix: Restart outpost container
|
||||
|
||||
2. **Redirect Loop**: Misconfigured nginx or outpost
|
||||
- Check: NPM advanced config points to port 9001
|
||||
- Check: Outpost logs for errors
|
||||
|
||||
3. **Session Expires Too Quickly**: Token validity too short
|
||||
- Check: Provider token_validity setting (should be 480 minutes)
|
||||
|
||||
## Automation Scripts
|
||||
|
||||
### Enable Forward Auth on New Sites
|
||||
|
||||
```python
|
||||
from src.clients.npm_client import get_npm_client
|
||||
|
||||
npm = get_npm_client()
|
||||
proxy = await npm.create_proxy_host(
|
||||
domain_names=["newsite.schweitz.net"],
|
||||
forward_host="backend-container",
|
||||
forward_port=8080,
|
||||
ssl_enabled=True
|
||||
)
|
||||
await npm.enable_authentik_forward_auth(proxy["id"])
|
||||
```
|
||||
|
||||
### Bulk Operations
|
||||
|
||||
```bash
|
||||
# Enable on all hosts
|
||||
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
|
||||
|
||||
# Update to port 9001
|
||||
docker exec core-api /venv/bin/python /app/fix_port_to_9001.py
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. ✓ **HTTPS Required**: All sites use SSL termination at NPM
|
||||
2. ✓ **Secure Cookies**: HttpOnly and Secure flags prevent XSS/MITM
|
||||
3. ✓ **Token Rotation**: Tokens extend on activity for security
|
||||
4. ✓ **Audit Logging**: Authentik logs all authentication events
|
||||
5. ⚠ **MFA**: Not yet enabled (future enhancement)
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Authentik server
|
||||
curl http://192.168.86.149:9000/-/health/live/
|
||||
|
||||
# Proxy outpost
|
||||
curl http://192.168.86.149:9001/outpost.goauthentik.io/ping
|
||||
|
||||
# Metrics
|
||||
curl http://192.168.86.149:9300/metrics
|
||||
```
|
||||
|
||||
### Authentik Admin Dashboard
|
||||
|
||||
- **URL**: https://auth.schweitz.net/if/admin/
|
||||
- **Events**: System → Events (view login attempts, failures)
|
||||
- **Active Sessions**: System → Tokens
|
||||
- **Outpost Status**: System → Outposts
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Multi-Factor Authentication**: Enable TOTP/SMS for high-security sites
|
||||
2. **Per-Site Access Control**: Different user groups for different domains
|
||||
3. **Rate Limiting**: Prevent brute force attacks
|
||||
4. **IP Whitelisting**: Allow certain IPs without auth
|
||||
5. **API Key Support**: Service-to-service authentication
|
||||
6. **Integration with Core API**: Expose auth status via REST API
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Main Documentation**: `FORWARD_AUTH_CONFIGURATION.md`
|
||||
- **SSO Progress**: `SSO_IMPLEMENTATION_PROGRESS.md`
|
||||
- **Security Plan**: `security-implementation-plan.md`
|
||||
- **Authentik Docs**: https://docs.goauthentik.io/docs/providers/proxy/
|
||||
|
||||
## Notes
|
||||
|
||||
- The embedded Authentik outpost (ID: `fbbefe20-b2e1-4706-8533-7bfece51989c`) is for Authentik's own web UI, not for external forward auth
|
||||
- Forward auth requires a separate proxy outpost container listening on its own port (9001)
|
||||
- The outpost automatically created by Authentik via Docker service connection had the wrong port configuration
|
||||
- We manually deployed the outpost with correct settings using `docker run`
|
||||
- NPM uses host network mode, so the outpost must also use host mode or be accessible via host IP:port
|
||||
- Token keys are only visible once during creation in the Authentik UI; we retrieved it from the auto-created container's environment variables
|
||||
|
||||
## Completion Status
|
||||
|
||||
✅ **COMPLETE** - All protected sites now require Authentik authentication
|
||||
|
||||
**Deployment successful as of 2025-11-16 15:41 UTC**
|
||||
@@ -1,259 +0,0 @@
|
||||
# Authentik Forward Authentication Configuration
|
||||
|
||||
## Overview
|
||||
|
||||
All NPM-managed domains now use Authentik for centralized Single Sign-On (SSO) authentication. When users access any protected site, they are automatically redirected to Authentik for login, then returned to their original destination.
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
1. **User accesses protected site** (e.g., https://portainer.schweitz.net)
|
||||
2. **NPM checks authentication** via Authentik forward auth
|
||||
3. **If not authenticated**: Redirect to https://auth.schweitz.net/outpost.goauthentik.io/start
|
||||
4. **User logs in** with Google (via Authentik)
|
||||
5. **Authentik sets session cookie** (8-hour validity)
|
||||
6. **Redirect back** to originally requested URL
|
||||
7. **Subsequent requests**: Automatically authenticated (no login required)
|
||||
|
||||
## Token & Session Configuration
|
||||
|
||||
### Token Validity
|
||||
- **Initial validity**: 8 hours (480 minutes)
|
||||
- **Activity-based extension**: Token stays active on use (rolling expiration)
|
||||
- **Idle timeout**: 8 hours after last activity
|
||||
- **Explicit logout**: Token invalidated immediately
|
||||
|
||||
### Session Behavior
|
||||
- **Single login** protects all domains under `*.schweitz.net`
|
||||
- **Cookie domain**: Shared across all sites
|
||||
- **Persistent**: Survives browser restarts (until expiration/logout)
|
||||
- **Secure**: HTTPS-only, HttpOnly flag set
|
||||
|
||||
## Protected Domains
|
||||
|
||||
The following domains are protected with Authentik forward auth:
|
||||
|
||||
1. **192.168.86.149** - Direct IP access
|
||||
2. **amp.schweitz.net** - AMP Server
|
||||
3. **cloud.schweitz.net** - Nextcloud
|
||||
4. **code.schweitz.net** - VS Code Server
|
||||
5. **git.schweitz.net** - Gitea
|
||||
6. **home.schweitz.net** - Home Assistant/Dashboard
|
||||
7. **media.schweitz.net** - Media Server
|
||||
8. **tatlock.schweitz.net** - Tatlock Services
|
||||
9. **tower-of-joy** - Tower of Joy Services
|
||||
|
||||
### Unprotected Domains
|
||||
|
||||
- **auth.schweitz.net** - Authentik itself (cannot protect the auth provider)
|
||||
|
||||
## User Management
|
||||
|
||||
### Adding Users
|
||||
|
||||
1. Go to https://auth.schweitz.net/if/admin/
|
||||
2. Navigate to **Directory** → **Users**
|
||||
3. Click **Create** → **Create and enroll user**
|
||||
4. Enter user details
|
||||
5. Send enrollment invite (they'll set up Google OAuth)
|
||||
|
||||
### User Access Control
|
||||
|
||||
Currently, **all authenticated users** can access protected sites. To restrict access:
|
||||
|
||||
1. Go to https://auth.schweitz.net/if/admin/#/core/applications
|
||||
2. Select **NPM Forward Auth** application
|
||||
3. Go to **Policy / Group / User Bindings**
|
||||
4. Add specific users or groups
|
||||
|
||||
### Group-Based Access (Future)
|
||||
|
||||
You can create groups and assign different access levels:
|
||||
- `admin` - Full access to all sites
|
||||
- `family` - Access to media, home
|
||||
- `developers` - Access to code, git
|
||||
|
||||
## Logout
|
||||
|
||||
Users can log out at: https://auth.schweitz.net/if/user/#/settings
|
||||
|
||||
Click **Sign Out** to invalidate the session across all protected sites.
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### Authentik Components
|
||||
|
||||
1. **Proxy Provider** (`npm-forward-auth-provider`)
|
||||
- Mode: `forward_single`
|
||||
- External Host: `https://auth.schweitz.net`
|
||||
- Token Validity: 480 minutes
|
||||
|
||||
2. **Application** (`NPM Forward Auth`)
|
||||
- Links provider to user interface
|
||||
- Accessible at: https://auth.schweitz.net
|
||||
|
||||
3. **Outpost** (`authentik Embedded Outpost`)
|
||||
- Handles authentication requests from NPM
|
||||
- Endpoint: `http://authentik-server:9000/outpost.goauthentik.io`
|
||||
|
||||
### NPM Configuration
|
||||
|
||||
Each proxy host has advanced nginx configuration:
|
||||
|
||||
```nginx
|
||||
# Forward auth to Authentik
|
||||
auth_request /outpost.goauthentik.io/auth/nginx;
|
||||
|
||||
# Preserve cookies
|
||||
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
||||
add_header Set-Cookie $auth_cookie;
|
||||
|
||||
# Extract user info
|
||||
auth_request_set $authentik_username $upstream_http_x_authentik_username;
|
||||
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
|
||||
auth_request_set $authentik_email $upstream_http_x_authentik_email;
|
||||
|
||||
# Pass to backend
|
||||
proxy_set_header X-authentik-username $authentik_username;
|
||||
proxy_set_header X-authentik-email $authentik_email;
|
||||
|
||||
# Redirect on auth failure
|
||||
error_page 401 = @authentik_proxy_signin;
|
||||
|
||||
location @authentik_proxy_signin {
|
||||
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
|
||||
}
|
||||
|
||||
# Auth endpoint
|
||||
location /outpost.goauthentik.io {
|
||||
proxy_pass http://authentik-server:9000/outpost.goauthentik.io;
|
||||
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||
proxy_pass_request_body off;
|
||||
}
|
||||
```
|
||||
|
||||
## Backend Application Integration
|
||||
|
||||
Protected applications receive user information via headers:
|
||||
|
||||
- `X-authentik-username` - Username
|
||||
- `X-authentik-email` - Email address
|
||||
- `X-authentik-groups` - Comma-separated groups
|
||||
- `X-authentik-name` - Full name
|
||||
- `X-authentik-uid` - Unique user ID
|
||||
|
||||
Applications can use these headers for:
|
||||
- Displaying user info
|
||||
- Audit logging
|
||||
- Role-based access control
|
||||
- Personalization
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### User Can't Log In
|
||||
|
||||
1. **Check Authentik status**: `docker ps | grep authentik`
|
||||
2. **Check Authentik logs**: `docker logs authentik-server`
|
||||
3. **Verify Google OAuth**:
|
||||
- Go to https://auth.schweitz.net/if/admin/#/core/sources
|
||||
- Ensure Google source is enabled
|
||||
4. **Check user exists**:
|
||||
- Go to https://auth.schweitz.net/if/admin/#/identity/users
|
||||
- Verify user account is active
|
||||
|
||||
### Redirect Loop
|
||||
|
||||
If users get stuck in a redirect loop:
|
||||
|
||||
1. **Clear browser cookies** for `*.schweitz.net`
|
||||
2. **Check NPM config**: Ensure `/outpost.goauthentik.io` location exists
|
||||
3. **Restart NPM**: `docker restart npm`
|
||||
4. **Check outpost**: Verify provider is assigned to outpost
|
||||
|
||||
### 502 Bad Gateway
|
||||
|
||||
If auth requests fail:
|
||||
|
||||
1. **Check Authentik container**: `docker ps | grep authentik`
|
||||
2. **Verify network**: Ensure NPM can reach `authentik-server:9000`
|
||||
3. **Check NPM logs**: `docker logs npm`
|
||||
|
||||
### Session Expires Too Quickly
|
||||
|
||||
If users are logged out unexpectedly:
|
||||
|
||||
1. **Check token validity**: Should be 480 minutes (8 hours)
|
||||
2. **Verify rolling expiration**: Active use should extend session
|
||||
3. **Check system time**: Ensure Docker host time is correct
|
||||
|
||||
## Automation
|
||||
|
||||
### Adding Forward Auth to New Sites
|
||||
|
||||
When creating new proxy hosts via core-api:
|
||||
|
||||
```python
|
||||
from src.clients.npm_client import get_npm_client
|
||||
|
||||
npm = get_npm_client()
|
||||
|
||||
# Create proxy host
|
||||
proxy = await npm.create_proxy_host(
|
||||
domain_names=["newsite.schweitz.net"],
|
||||
forward_host="backend-container",
|
||||
forward_port=8080,
|
||||
ssl_enabled=True
|
||||
)
|
||||
|
||||
# Enable forward auth
|
||||
await npm.enable_authentik_forward_auth(proxy["id"])
|
||||
```
|
||||
|
||||
### Bulk Enable/Disable
|
||||
|
||||
To enable on all hosts:
|
||||
```bash
|
||||
docker exec core-api /venv/bin/python /app/enable_npm_forward_auth.py
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **HTTPS Required**: Forward auth should only be used with HTTPS
|
||||
2. **Secure Cookies**: HttpOnly and Secure flags prevent XSS/MITM
|
||||
3. **Token Rotation**: Tokens are rotated on activity for security
|
||||
4. **Audit Logging**: Authentik logs all authentication events
|
||||
5. **MFA Support**: Can be enabled in Authentik for additional security
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Check Authentication Status
|
||||
|
||||
```bash
|
||||
# Check Authentik health
|
||||
curl http://authentik-server:9000/-/health/live/
|
||||
|
||||
# Check active sessions (in Authentik admin)
|
||||
# https://auth.schweitz.net/if/admin/#/events/log
|
||||
```
|
||||
|
||||
### Monitor Failed Attempts
|
||||
|
||||
Go to **System** → **Events** in Authentik admin to see:
|
||||
- Failed login attempts
|
||||
- Successful authentications
|
||||
- Token expirations
|
||||
- Policy violations
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Per-Site Access Control**: Different user groups for different domains
|
||||
2. **Multi-Factor Authentication**: SMS/TOTP for high-security sites
|
||||
3. **Rate Limiting**: Prevent brute force attacks
|
||||
4. **IP Whitelisting**: Allow certain IPs without auth
|
||||
5. **API Key Support**: Service-to-service authentication
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Security Implementation Plan**: `security-implementation-plan.md`
|
||||
- **SSO Progress**: `SSO_IMPLEMENTATION_PROGRESS.md`
|
||||
- **OIDC Configuration**: `OIDC_CONFIGURATION.md`
|
||||
- **Authentik Docs**: https://docs.goauthentik.io/docs/providers/proxy/
|
||||
@@ -1,167 +0,0 @@
|
||||
# OIDC Authentication Configuration
|
||||
|
||||
## Overview
|
||||
|
||||
Core-API has been configured with OIDC authentication support using Authentik as the identity provider. This provides secure authentication for infrastructure management endpoints.
|
||||
|
||||
## Current Status
|
||||
|
||||
**OIDC is currently DISABLED** (`oidc_enabled=false` in config.py)
|
||||
|
||||
This allows:
|
||||
- Internal services to access the API without authentication
|
||||
- Direct API access from the Docker network
|
||||
- Backward compatibility with existing integrations
|
||||
|
||||
## Authentik Configuration
|
||||
|
||||
### Provider Details
|
||||
|
||||
- **Provider Name**: `core-api-provider`
|
||||
- **Provider ID**: `1`
|
||||
- **Client ID**: `core-api`
|
||||
- **Client Secret**: `WfsY0iIVOmO1wvXn1u8jwa08eiQsn2yVf9toBwVyEqps3M98nOZwJMgDOqH5PNZGM6wIxKTwlwYemtaOUg9u5bocd0EqShyoe4yhBQq4SDd0svyArILHZeGHFVEqUi4d`
|
||||
- **Issuer**: `https://auth.schweitz.net/application/o/core-api/`
|
||||
|
||||
### Application Details
|
||||
|
||||
- **Application Name**: `Core API`
|
||||
- **Slug**: `core-api`
|
||||
- **Launch URL**: `http://localhost:8083/docs`
|
||||
|
||||
### Redirect URIs
|
||||
|
||||
- `http://localhost:8083/docs/oauth2-redirect`
|
||||
- `https://core-api.schweitz.net/docs/oauth2-redirect`
|
||||
- `http://192.168.86.149:8083/docs/oauth2-redirect`
|
||||
|
||||
## Enabling OIDC Authentication
|
||||
|
||||
When ready to enable OIDC authentication for external access:
|
||||
|
||||
### 1. Update core-api Configuration
|
||||
|
||||
In `/home/jpmschweitzer/Projects/portainer-core/services/core-api/src/config.py`:
|
||||
|
||||
```python
|
||||
# OIDC Authentication (Authentik)
|
||||
oidc_enabled: bool = True # Change from False to True
|
||||
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
|
||||
oidc_audience: str = "core-api"
|
||||
oidc_client_secret: str = "WfsY0iIVOmO1wvXn1u8jwa08eiQsn2yVf9toBwVyEqps3M98nOZwJMgDOqH5PNZGM6wIxKTwlwYemtaOUg9u5bocd0EqShyoe4yhBQq4SDd0svyArILHZeGHFVEqUi4d"
|
||||
```
|
||||
|
||||
### 2. Restart core-api
|
||||
|
||||
```bash
|
||||
docker restart core-api
|
||||
```
|
||||
|
||||
### 3. Test Authentication
|
||||
|
||||
1. Visit: http://localhost:8083/docs
|
||||
2. Click the "Authorize" button
|
||||
3. Log in with Google via Authentik
|
||||
4. Access protected endpoints
|
||||
|
||||
## Protected Endpoints
|
||||
|
||||
When OIDC is enabled, the following endpoints require admin authentication:
|
||||
|
||||
### Infrastructure Management (Write Operations)
|
||||
- `POST /infrastructure/services` - Deploy stack
|
||||
- `PUT /infrastructure/services/{name}` - Update stack
|
||||
- `DELETE /infrastructure/services/{name}` - Delete stack
|
||||
- `POST /infrastructure/proxy` - Create proxy host
|
||||
- `POST /infrastructure/services/{name}/stop` - Stop service
|
||||
- `POST /infrastructure/services/{name}/start` - Start service
|
||||
|
||||
### Monitoring Management
|
||||
- `POST /infrastructure/monitors` - Create monitor
|
||||
- `PUT /infrastructure/monitors/{monitor_id}` - Update monitor
|
||||
- `DELETE /infrastructure/monitors/{monitor_id}` - Delete monitor
|
||||
|
||||
### Read Endpoints (Public)
|
||||
All GET endpoints remain publicly accessible:
|
||||
- `/health` - Health check
|
||||
- `/infrastructure/health` - Infrastructure health
|
||||
- `/infrastructure/services` - List services
|
||||
- `/infrastructure/ports` - List ports
|
||||
- `/infrastructure/domains` - List domains
|
||||
- `/infrastructure/monitors` - List monitors
|
||||
|
||||
## Internal Network Access
|
||||
|
||||
**Important**: After enabling OIDC for external users, internal services still need unrestricted access.
|
||||
|
||||
### Future Enhancement Options
|
||||
|
||||
1. **Network-based Authentication**
|
||||
- Check if request originates from `docker-dataplane` network
|
||||
- Allow requests from internal IPs without auth
|
||||
- Require OIDC only for external requests
|
||||
|
||||
2. **Service Accounts**
|
||||
- Create machine tokens for internal services
|
||||
- Use Bearer token authentication for service-to-service calls
|
||||
- Keep OIDC for user authentication
|
||||
|
||||
3. **NPM Proxy Layer**
|
||||
- External domain (https://core-api.schweitz.net) → Requires Authentik SSO
|
||||
- Internal access (http://core-api:8083) → No authentication
|
||||
|
||||
## Admin Groups
|
||||
|
||||
Users must be members of one of these Authentik groups to access protected endpoints:
|
||||
- `admin`
|
||||
- `authentik Admins`
|
||||
|
||||
Configure user group membership in Authentik admin panel:
|
||||
https://auth.schweitz.net/if/admin/#/identity/users
|
||||
|
||||
## Authentik Management
|
||||
|
||||
- **Admin Panel**: https://auth.schweitz.net/if/admin/
|
||||
- **Applications**: https://auth.schweitz.net/if/admin/#/core/applications
|
||||
- **Providers**: https://auth.schweitz.net/if/admin/#/core/providers
|
||||
|
||||
## API Token Management
|
||||
|
||||
The Authentik API token used for automation is stored in:
|
||||
`/home/jpmschweitzer/Projects/portainer-core/services/core-api/src/credentials.py`
|
||||
|
||||
```python
|
||||
AUTHENTIK_CORE_API_TOKEN = "peXM0EzDv2Wiwycbfm3cE5O44IMAOR8ntUwKVP977yFvVopCzDlKY8tymlMM"
|
||||
```
|
||||
|
||||
To create additional tokens: https://auth.schweitz.net/if/admin/#/identity/tokens
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Client Secret**: Stored in config.py (gitignored), consider moving to environment variable
|
||||
2. **Token Validation**: Tokens are validated using JWKS from Authentik
|
||||
3. **Token Expiry**: Access tokens valid for 60 minutes, refresh tokens for 30 days
|
||||
4. **SSL**: All production endpoints should use HTTPS (via NPM)
|
||||
5. **Admin Access**: Restrict admin group membership carefully
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Login Issues
|
||||
- Check Authentik service status: `docker ps | grep authentik`
|
||||
- Check Authentik logs: `docker logs authentik-server`
|
||||
- Verify redirect URIs match exactly in Authentik provider config
|
||||
|
||||
### Token Validation Errors
|
||||
- Verify OIDC issuer URL is correct
|
||||
- Check core-api logs: `docker logs core-api`
|
||||
- Ensure Authentik is accessible from core-api container
|
||||
|
||||
### Permission Denied (403)
|
||||
- Verify user is member of `admin` group in Authentik
|
||||
- Check user claims in JWT token at https://jwt.io
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Security Implementation Plan**: `/home/jpmschweitzer/Projects/portainer-core/security-implementation-plan.md`
|
||||
- **SSO Progress**: `/home/jpmschweitzer/Projects/portainer-core/SSO_IMPLEMENTATION_PROGRESS.md`
|
||||
- **Authentik Docs**: https://docs.goauthentik.io/
|
||||
@@ -1,115 +0,0 @@
|
||||
# SSO Implementation Progress Tracker
|
||||
|
||||
**Project:** Google OAuth SSO for Homelab Infrastructure
|
||||
**Started:** 2025-11-15
|
||||
**Status:** 🚧 Phase 1 - In Progress
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Implementing Single Sign-On (SSO) using:
|
||||
- **Identity Provider:** Authentik
|
||||
- **Authentication Source:** Google OAuth (Workspace + Gmail)
|
||||
- **In-Scope Services:** core-api, Nextcloud, Jellyfin, Gitea, Open WebUI, Organizr, code-server
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foundation (Week 1)
|
||||
**Goal:** Deploy Authentik, configure Google OAuth, protect core-api
|
||||
|
||||
### Task Checklist
|
||||
|
||||
- [x] **1.1 Deploy Authentik Stack** *(In Progress)*
|
||||
- [x] Create `stacks/authentik.yml`
|
||||
- [x] Generate secrets (PostgreSQL password, Authentik secret key)
|
||||
- [x] Create `.env.authentik` file
|
||||
- [x] Create data directories
|
||||
- [ ] Deploy via Portainer
|
||||
- [ ] Verify services running (4/4: server, worker, postgresql, redis)
|
||||
- [ ] Complete initial setup wizard
|
||||
- [ ] Access admin portal
|
||||
|
||||
- [ ] **1.2 Configure NPM Proxy**
|
||||
- [ ] Create proxy host: `auth.schweitz.net` → `authentik-server:9000`
|
||||
- [ ] Enable SSL with Let's Encrypt
|
||||
- [ ] Test HTTPS access
|
||||
- [ ] Verify health endpoint
|
||||
|
||||
- [ ] **1.3 Google OAuth Setup**
|
||||
- [ ] Create/configure Google Cloud Project
|
||||
- [ ] Set up OAuth consent screen
|
||||
- [ ] Create OAuth 2.0 credentials
|
||||
- [ ] Note Client ID and Client Secret
|
||||
- [ ] Configure authorized redirect URIs
|
||||
|
||||
- [ ] **1.4 Configure Google Source in Authentik**
|
||||
- [ ] Add Google OAuth source
|
||||
- [ ] Configure scopes: openid, email, profile
|
||||
- [ ] Test login with Google Workspace account
|
||||
- [ ] Test login with Gmail account
|
||||
- [ ] Verify user profile synced
|
||||
|
||||
- [ ] **1.5 Implement core-api OIDC Authentication**
|
||||
- [ ] Add dependencies: PyJWT, python-jose
|
||||
- [ ] Create `src/auth/oidc.py` module
|
||||
- [ ] Update `src/config.py` with OIDC settings
|
||||
- [ ] Create Authentik OIDC provider for core-api
|
||||
- [ ] Protect infrastructure endpoints
|
||||
- [ ] Update OpenAPI docs with security scheme
|
||||
|
||||
- [ ] **1.6 Testing & Validation**
|
||||
- [ ] Test unauthenticated API request (expect 401)
|
||||
- [ ] Test authenticated API request with valid token
|
||||
- [ ] Verify user claims available in endpoints
|
||||
- [ ] Test token expiration handling
|
||||
- [ ] Test admin-only endpoints
|
||||
- [ ] Update widget for OAuth flow
|
||||
|
||||
---
|
||||
|
||||
## Progress Log
|
||||
|
||||
### 2025-11-15 - 21:20 CET
|
||||
|
||||
**[Completed]** Shared Infrastructure Architecture
|
||||
- ✅ Designed shared PostgreSQL + Redis architecture
|
||||
- ✅ Created `SHARED_INFRASTRUCTURE_ARCHITECTURE.md` documentation
|
||||
- ✅ Created separate stacks for modularity:
|
||||
- `postgres-shared.yml` (centralized database)
|
||||
- `redis-shared.yml` (centralized cache)
|
||||
- `authentik-shared.yml` (using shared backends)
|
||||
- ✅ Generated secure credentials (all passwords 32-byte random)
|
||||
- ✅ Created unified `docker-dataplane` network
|
||||
- ✅ Created PostgreSQL init script for multi-database setup
|
||||
- ✅ Created environment files:
|
||||
- `.env.postgres` (PostgreSQL + app DB passwords)
|
||||
- `.env.authentik-shared` (Authentik config)
|
||||
|
||||
**Architecture Benefits:**
|
||||
- Resource savings: ~400MB RAM per service using shared infrastructure
|
||||
- Centralized backups and monitoring
|
||||
- Easier maintenance and upgrades
|
||||
- Modular deployment (PostgreSQL and Redis as separate stacks)
|
||||
|
||||
**[Next]** Deploy shared infrastructure, then Authentik
|
||||
|
||||
---
|
||||
|
||||
## Next: Create Authentik Stack
|
||||
|
||||
Creating `stacks/authentik.yml` with:
|
||||
- authentik-server
|
||||
- authentik-worker
|
||||
- PostgreSQL database
|
||||
- Redis cache
|
||||
|
||||
Expected resources: ~500MB RAM, 1.5 CPU, 5GB storage
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- **Architecture Decision:** FastAPI native OIDC for core-api (not NPM forward auth)
|
||||
- **Out of Scope:** Infrastructure tools, data providers, local services
|
||||
- **Security:** All passwords/secrets via environment variables, not committed to git
|
||||
@@ -162,7 +162,7 @@ cors_origins: list[str] = ["http://your-organizr-domain.com"]
|
||||
The following service groups are defined (stopping one stops all in group):
|
||||
|
||||
- **jellyfin**: jellyfin
|
||||
- **nextcloud**: nextcloud, nextcloud-db, nextcloud-redis
|
||||
- **nextcloud**: nextcloud (uses shared postgres-shared + redis-shared)
|
||||
- **gitea**: gitea, gitea-db
|
||||
- **ai-stack**: open-webui, ollama, qdrant
|
||||
- **samba**: samba
|
||||
@@ -179,6 +179,8 @@ These infrastructure services are protected:
|
||||
- watchtower
|
||||
- netdata
|
||||
- maintenance
|
||||
- postgres-shared (shared database infrastructure)
|
||||
- redis-shared (shared cache infrastructure)
|
||||
|
||||
## Advanced: Customizing the UI
|
||||
|
||||
|
||||
@@ -1,332 +0,0 @@
|
||||
# Core-API Refactoring Plan
|
||||
|
||||
**Date:** 2025-11-14
|
||||
**Goal:** Restructure Core-API into controller-based architecture and add Infrastructure Management API
|
||||
|
||||
## Current Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── api/
|
||||
│ └── v1/
|
||||
│ ├── chat.py # AI chat completions
|
||||
│ ├── models.py # Model listing
|
||||
│ ├── conversations.py # Conversation memory
|
||||
│ └── schemas.py # Pydantic schemas
|
||||
├── web_scraper/
|
||||
│ ├── router.py # Webscraper endpoints
|
||||
│ ├── service.py
|
||||
│ └── schemas.py
|
||||
├── models/
|
||||
│ ├── ollama_client.py # Ollama HTTP client
|
||||
│ └── embeddings.py
|
||||
├── memory/ # Memory tier system
|
||||
├── config.py # Global settings
|
||||
└── main.py # FastAPI app
|
||||
|
||||
```
|
||||
|
||||
## Target Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── controllers/ # NEW: Controller-based routing
|
||||
│ ├── __init__.py
|
||||
│ ├── base.py # Base controller class
|
||||
│ ├── ai_controller.py # AI Orchestrator (chat, models, conversations)
|
||||
│ ├── tools_controller.py # Utility tools (webscraper, etc.)
|
||||
│ ├── health_controller.py # Health & monitoring
|
||||
│ └── infrastructure_controller.py # Infrastructure automation
|
||||
├── clients/ # NEW: External API clients
|
||||
│ ├── __init__.py
|
||||
│ ├── portainer_client.py # Portainer API
|
||||
│ ├── npm_client.py # Nginx Proxy Manager API
|
||||
│ └── kuma_client.py # Uptime Kuma Socket.IO API
|
||||
├── api/v1/ # Keep existing for backward compat
|
||||
├── web_scraper/ # Keep as-is for now
|
||||
├── models/ # Keep as-is
|
||||
├── memory/ # Keep as-is
|
||||
├── config.py # Enhanced with infrastructure settings
|
||||
└── main.py # Updated routing
|
||||
|
||||
```
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Infrastructure Setup ✅ COMPLETE
|
||||
- [x] Research API authentication methods
|
||||
- [x] Add infrastructure settings to config.py
|
||||
- [x] Create credentials.py for sensitive data (gitignored)
|
||||
- [x] Create credentials.example.py as template
|
||||
- [x] Update .gitignore to exclude credentials.py
|
||||
- [x] Update config.py to import from credentials module
|
||||
- [x] Create /controllers directory structure
|
||||
- [x] Create /clients directory structure
|
||||
- [x] Create base controller class
|
||||
|
||||
### Phase 2: API Clients ✅ COMPLETE (Portainer & NPM)
|
||||
- [x] Implement Portainer API client (access token auth)
|
||||
- [x] Implement NPM API client (JWT with refresh)
|
||||
- [x] Add token storage/refresh mechanisms
|
||||
- [ ] Implement Uptime Kuma Socket.IO client (DEFERRED - WebSocket complexity)
|
||||
|
||||
### Phase 3: Infrastructure Controller ✅ COMPLETE
|
||||
- [x] GET /infrastructure/health - Check connectivity ✅ TESTED
|
||||
- [x] GET /infrastructure/services - List all services ✅ TESTED
|
||||
- [x] GET /infrastructure/services/{name} - Get service details ✅ TESTED
|
||||
- [x] GET /infrastructure/ports - List allocated ports ✅ IMPLEMENTED & TESTED
|
||||
- [x] GET /infrastructure/domains - List configured domains ✅ TESTED
|
||||
- [x] Integrate with main.py routing ✅ TESTED
|
||||
- [x] Fix Pydantic validation issues (status field type conversion)
|
||||
- [x] POST /infrastructure/services - Deploy new service ✅ TESTED
|
||||
- [x] PUT /infrastructure/services/{name} - Update service ✅ TESTED
|
||||
- [x] DELETE /infrastructure/services/{name} - Remove service ✅ TESTED
|
||||
- [x] POST /infrastructure/proxy - Create NPM proxy host with optional SSL ✅ IMPLEMENTED
|
||||
- [ ] POST /infrastructure/monitoring/add - Auto-add Kuma monitor (DEFERRED - Socket.IO complexity)
|
||||
|
||||
### Phase 4: Refactor Existing Controllers ✅ COMPLETE
|
||||
- [x] Move AI endpoints to ai_controller.py ✅ COMPLETE
|
||||
- [x] Move webscraper to tools_controller.py ✅ COMPLETE
|
||||
- [x] Move health check to health_controller.py ✅ COMPLETE
|
||||
- [x] Update main.py imports and routing ✅ COMPLETE
|
||||
- [x] Test all refactored endpoints ✅ ALL WORKING
|
||||
|
||||
### Phase 5: Testing & Documentation ✅ COMPLETE
|
||||
- [x] Test all refactored endpoints ✅ ALL WORKING
|
||||
- [x] Update API documentation (OpenAPI spec auto-generated and validated)
|
||||
- [~] Create CLI wrapper scripts (SKIPPED - LLMs consume OpenAPI spec directly)
|
||||
- [~] Remove old shell scripts (DEFERRED - not blocking)
|
||||
|
||||
### Phase 6: Infrastructure Improvements 📋 FUTURE
|
||||
- [ ] Consolidate Docker network topology into single `docker-dataplane` network
|
||||
- Currently each stack has its own network (172.22.0.x, 172.25.0.x, 172.20.0.x, etc.)
|
||||
- Error-prone and unnecessarily complex
|
||||
- Single shared network simplifies inter-service communication
|
||||
- Reduces subnet conflicts and improves service discovery
|
||||
- Update all compose files to use: `networks: [docker-dataplane]`
|
||||
- Create network once: `docker network create docker-dataplane`
|
||||
|
||||
---
|
||||
|
||||
## Progress Notes (2025-11-14)
|
||||
|
||||
### Session 1: Foundation & Read Endpoints
|
||||
**Completed:**
|
||||
- Created controller and client architecture
|
||||
- Implemented Portainer client with full CRUD operations for stacks
|
||||
- Implemented NPM client with JWT refresh and proxy/certificate management
|
||||
- Built infrastructure controller with 5 read/list endpoints
|
||||
- Added infrastructure settings to config.py
|
||||
|
||||
**Files Created:**
|
||||
- `src/controllers/__init__.py`
|
||||
- `src/controllers/base.py`
|
||||
- `src/controllers/infrastructure_controller.py`
|
||||
- `src/clients/__init__.py`
|
||||
- `src/clients/portainer_client.py`
|
||||
- `src/clients/npm_client.py`
|
||||
- `REFACTORING_PLAN.md` (this file)
|
||||
|
||||
### Session 2: Credentials & Testing (2025-11-14 Evening)
|
||||
**Completed:**
|
||||
- Created credentials management system (credentials.py gitignored, credentials.example.py committed)
|
||||
- Updated config.py to import from credentials module with fallback
|
||||
- Generated Portainer API token programmatically via API
|
||||
- Integrated infrastructure controller into main.py
|
||||
- Fixed Pydantic validation bug (status field int→str conversion)
|
||||
- Tested all read endpoints with live Portainer/NPM infrastructure
|
||||
- Verified 8 stacks detected, domains with SSL status working
|
||||
|
||||
**Test Results:**
|
||||
- ✅ GET /infrastructure/health - Portainer connected, NPM accessible
|
||||
- ✅ GET /infrastructure/services - Returns 8 active stacks
|
||||
- ✅ GET /infrastructure/services/{name} - Service lookup working
|
||||
- ✅ GET /infrastructure/domains - Returns proxy hosts with SSL status
|
||||
- ✅ NPM health check fixed (now accepts 2xx/3xx status codes and follows redirects)
|
||||
|
||||
### Session 3: Write Endpoints (2025-11-14 Evening)
|
||||
**Completed:**
|
||||
- Created request/response models for write operations (DeployServiceRequest, UpdateServiceRequest, CreateProxyRequest, OperationResult)
|
||||
- Implemented POST /infrastructure/services - Deploy new service from compose YAML
|
||||
- Implemented PUT /infrastructure/services/{name} - Update existing service configuration
|
||||
- Implemented DELETE /infrastructure/services/{name} - Remove service and stack
|
||||
- Implemented POST /infrastructure/proxy - Create NPM proxy host with optional SSL certificate
|
||||
- Updated main.py API description with write endpoints
|
||||
- Tested all service management endpoints (POST/PUT/DELETE) with live Portainer instance
|
||||
|
||||
**Test Results:**
|
||||
- ✅ POST /infrastructure/services - Created test-nginx stack (ID: 30)
|
||||
- ✅ PUT /infrastructure/services/test-nginx - Updated compose with environment variable
|
||||
- ✅ DELETE /infrastructure/services/test-nginx - Removed stack successfully
|
||||
- ✅ POST /infrastructure/proxy - Implemented (not tested to avoid production interference)
|
||||
|
||||
**Next Steps:**
|
||||
1. ~~Refactor existing AI/tools/health endpoints into separate controllers (Phase 4)~~ ✅ DONE (2025-11-14)
|
||||
2. ~~Fix NPM health check to handle redirects~~ ✅ DONE (2025-11-14)
|
||||
3. ~~Implement port allocation detection logic~~ ✅ DONE (2025-11-14)
|
||||
4. ~~Create CLI wrappers for common operations~~ ⊘ SKIPPED (LLMs use OpenAPI)
|
||||
5. (OPTIONAL) Consolidate Docker networks into `docker-dataplane` (Phase 6)
|
||||
|
||||
### Session 4: NPM Health Check & Port Detection (2025-11-14 Afternoon)
|
||||
**Completed:**
|
||||
- Fixed NPM health check to handle redirects properly
|
||||
- Updated `npm_client.py` to accept 2xx/3xx status codes as healthy
|
||||
- Enabled explicit redirect following in httpx client
|
||||
- Verified fix with live NPM instance (now shows 9 proxy hosts)
|
||||
- Implemented comprehensive port detection in `GET /infrastructure/ports` endpoint
|
||||
- Added `get_containers()` and `get_container()` methods to PortainerClient
|
||||
- Enhanced PortInfo model with internal/external hostname and IP fields
|
||||
- Implemented domain mapping from NPM proxy hosts to services
|
||||
- Added deduplication logic for port entries (Docker returns duplicates per bind address)
|
||||
|
||||
**Port Detection Features:**
|
||||
- Scans all running containers across all Portainer endpoints
|
||||
- Extracts internal port, host port, and protocol for each container
|
||||
- Maps container names to service names via Docker Compose labels
|
||||
- Retrieves internal Docker hostnames and IP addresses per network
|
||||
- Cross-references NPM proxy hosts to identify external domains
|
||||
- Returns 22 unique port mappings with complete metadata
|
||||
|
||||
**Technical Details:**
|
||||
|
||||
*NPM Health Check:*
|
||||
- Issue: NPM's `/api` endpoint returns 302 redirect, old code only accepted 200
|
||||
- Solution: Accept `200 <= status_code < 400` as healthy response
|
||||
- Result: NPM health check now returns `true` and proxy hosts are enumerated correctly
|
||||
|
||||
*Port Detection:*
|
||||
- Queries Portainer Docker API for container list and port mappings
|
||||
- Extracts NetworkSettings for internal IPs and hostnames
|
||||
- Builds port→domain map from NPM proxy hosts configuration
|
||||
- Matches services to external domains using multiple strategies:
|
||||
- By container name + port
|
||||
- By internal IP + port
|
||||
- By host address + host port (localhost, 127.0.0.1, server IP)
|
||||
- Deduplicates based on (port, container_name, protocol) tuple
|
||||
- Example output: Nextcloud port 80 → internal IP 172.25.0.3 → external domain cloud.schweitz.net
|
||||
|
||||
### Session 5: Controller Architecture Refactoring (2025-11-14 Evening)
|
||||
**Completed:**
|
||||
- Created `ai_controller.py` consolidating chat, models, and conversations endpoints
|
||||
- Created `tools_controller.py` for web scraper functionality
|
||||
- Created `health_controller.py` for service health and info endpoints
|
||||
- Updated `main.py` to use new controller-based architecture
|
||||
- Removed legacy router imports and inline endpoint definitions
|
||||
- Tested all refactored endpoints - 16 endpoints working correctly
|
||||
|
||||
**Architecture Changes:**
|
||||
- All endpoints now follow consistent controller pattern inheriting from `BaseController`
|
||||
- Controllers use `create_router()` method for FastAPI router configuration
|
||||
- Clean separation of concerns:
|
||||
- `ai_controller.py` - AI orchestration and conversation memory (7 endpoints)
|
||||
- `tools_controller.py` - Utility tools like web scraper (1 endpoint)
|
||||
- `health_controller.py` - Service status and info (2 endpoints)
|
||||
- `infrastructure_controller.py` - Infrastructure management (6 endpoints)
|
||||
- Simplified `main.py` from 220 lines to 152 lines
|
||||
- Backward compatible - all existing endpoints work identically
|
||||
|
||||
**Test Results:**
|
||||
- ✅ GET / - Service information
|
||||
- ✅ GET /health - Health check with Ollama status
|
||||
- ✅ GET /v1/models - Model listing
|
||||
- ✅ POST /v1/chat/completions - Chat completions
|
||||
- ✅ GET /v1/conversations/{id} - Conversation history
|
||||
- ✅ GET /infrastructure/health - Infrastructure health
|
||||
- ✅ POST /web-scraper/scrape - Web scraping
|
||||
- ✅ OpenAPI spec generation - 16 endpoints documented
|
||||
|
||||
## API Authentication Strategy
|
||||
|
||||
### Portainer
|
||||
- **Method:** Access Token (X-API-Key header)
|
||||
- **Setup:** Manual creation in UI, store in config/env
|
||||
- **Duration:** Long-lived
|
||||
- **Storage:** Environment variable `PORTAINER_API_KEY`
|
||||
|
||||
### Nginx Proxy Manager
|
||||
- **Method:** JWT Bearer Token
|
||||
- **Setup:** Login via `/api/tokens` with credentials
|
||||
- **Duration:** ~24 hours
|
||||
- **Strategy:** Auto-refresh with stored credentials
|
||||
- **Storage:** `NPM_EMAIL` and `NPM_PASSWORD` in env
|
||||
|
||||
### Uptime Kuma
|
||||
- **Method:** Socket.IO WebSocket
|
||||
- **Setup:** Login via Socket.IO `login` event
|
||||
- **Duration:** Session-based
|
||||
- **Strategy:** Maintain persistent connection or re-auth per request
|
||||
- **Storage:** `KUMA_USERNAME` and `KUMA_PASSWORD` in env
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
### Credentials Management Strategy
|
||||
|
||||
**Use `credentials.py` for sensitive data** (added to `.gitignore`):
|
||||
- Keeps secrets out of version control
|
||||
- Easy terminal-based management with editor
|
||||
- Python format for type safety and autocomplete
|
||||
- Separate from config for security isolation
|
||||
|
||||
**Implementation:**
|
||||
1. Create `src/credentials.py` with credentials (gitignored)
|
||||
2. Create `src/credentials.example.py` as template (committed)
|
||||
3. Update `config.py` to import from credentials module
|
||||
4. Add `credentials.py` to `.gitignore`
|
||||
|
||||
**Example `src/credentials.py`:**
|
||||
```python
|
||||
"""
|
||||
Infrastructure credentials (GITIGNORED)
|
||||
Copy from credentials.example.py and fill in real values
|
||||
"""
|
||||
|
||||
# Portainer
|
||||
PORTAINER_URL = "http://localhost:8001"
|
||||
PORTAINER_API_KEY = "ptr_your_actual_token_here"
|
||||
|
||||
# Nginx Proxy Manager
|
||||
NPM_URL = "http://localhost:81"
|
||||
NPM_EMAIL = "jpmschweitzer@gmail.com"
|
||||
NPM_PASSWORD = "your_actual_password"
|
||||
|
||||
# Uptime Kuma
|
||||
KUMA_URL = "http://localhost:3001"
|
||||
KUMA_USERNAME = "admin"
|
||||
KUMA_PASSWORD = "your_actual_password"
|
||||
```
|
||||
|
||||
**Updated `config.py` to use credentials:**
|
||||
```python
|
||||
from src.credentials import (
|
||||
PORTAINER_URL, PORTAINER_API_KEY,
|
||||
NPM_URL, NPM_EMAIL, NPM_PASSWORD,
|
||||
KUMA_URL, KUMA_USERNAME, KUMA_PASSWORD
|
||||
)
|
||||
|
||||
class Settings(BaseSettings):
|
||||
# Infrastructure Management (from credentials.py)
|
||||
portainer_url: str = PORTAINER_URL
|
||||
portainer_api_key: str = PORTAINER_API_KEY
|
||||
|
||||
npm_url: str = NPM_URL
|
||||
npm_email: str = NPM_EMAIL
|
||||
npm_password: str = NPM_PASSWORD
|
||||
|
||||
kuma_url: str = KUMA_URL
|
||||
kuma_username: str = KUMA_USERNAME
|
||||
kuma_password: str = KUMA_PASSWORD
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Cleaner Code:** Separation of concerns, easier to maintain
|
||||
2. **Automation:** Programmatic service deployment and configuration
|
||||
3. **Elimination of Shell Scripts:** Replace ad-hoc scripts with proper API
|
||||
4. **Service Discovery:** Auto-detect running services and configurations
|
||||
5. **Self-Managing Homelab:** Foundation for autonomous infrastructure
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- Existing `/v1/` endpoints remain unchanged for backward compatibility
|
||||
- Web scraper endpoints stay at `/web-scraper/` initially
|
||||
- Old shell scripts in `/stacks/` will be replaced with CLI wrappers
|
||||
@@ -1,152 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Create Authentik Outpost for NPM Forward Authentication
|
||||
|
||||
Creates a dedicated outpost and retrieves its token for deployment.
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
sys.path.insert(0, '/home/jpmschweitzer/Projects/portainer-core/services/core-api')
|
||||
|
||||
from src.clients.authentik_client import get_authentik_client
|
||||
|
||||
|
||||
async def create_npm_outpost():
|
||||
"""Create outpost for NPM forward auth and get token"""
|
||||
authentik = get_authentik_client()
|
||||
|
||||
try:
|
||||
print("=" * 60)
|
||||
print("Creating NPM Forward Auth Outpost")
|
||||
print("=" * 60)
|
||||
|
||||
# Check if provider exists
|
||||
print("\n1. Checking for proxy provider...")
|
||||
provider = await authentik.get_provider_by_name_proxy("npm-forward-auth-provider")
|
||||
|
||||
if not provider:
|
||||
print("❌ Proxy provider not found. Run setup_authentik_forward_auth.py first.")
|
||||
return False
|
||||
|
||||
provider_id = provider["pk"]
|
||||
print(f"✓ Found provider (ID: {provider_id})")
|
||||
|
||||
# Check if outpost already exists
|
||||
print("\n2. Checking for existing NPM outpost...")
|
||||
try:
|
||||
existing = await authentik.get_outpost_by_name("npm-forward-auth-outpost")
|
||||
if existing:
|
||||
print(f"✓ Outpost already exists (ID: {existing['pk']})")
|
||||
outpost_id = existing["pk"]
|
||||
|
||||
# Update it to ensure provider is assigned
|
||||
print("\n3. Updating outpost configuration...")
|
||||
await authentik.update_outpost(
|
||||
outpost_id=outpost_id,
|
||||
providers=[provider_id]
|
||||
)
|
||||
print("✓ Outpost updated with provider")
|
||||
except Exception:
|
||||
# Outpost doesn't exist, create it
|
||||
print("⊘ Outpost doesn't exist, creating new one...")
|
||||
print("\n3. Creating NPM outpost...")
|
||||
|
||||
outpost = await authentik.create_outpost(
|
||||
name="npm-forward-auth-outpost",
|
||||
type="proxy",
|
||||
providers=[provider_id],
|
||||
config={
|
||||
"authentik_host": "http://192.168.86.149:9000",
|
||||
"authentik_host_insecure": False,
|
||||
"log_level": "info",
|
||||
"docker_labels": None,
|
||||
"docker_network": None,
|
||||
"docker_map_ports": True,
|
||||
"container_image": None,
|
||||
"kubernetes_replicas": 1,
|
||||
"kubernetes_namespace": "default"
|
||||
}
|
||||
)
|
||||
outpost_id = outpost["pk"]
|
||||
print(f"✓ Created outpost (ID: {outpost_id})")
|
||||
|
||||
# Try to get the service connection token
|
||||
print("\n4. Retrieving outpost token...")
|
||||
print("\nNote: Authentik creates service accounts for outposts automatically.")
|
||||
print("The token format is: ak-outpost-<outpost_uuid>-api")
|
||||
|
||||
# Get outpost details to find its service account
|
||||
outpost_details = await authentik._request("GET", f"outposts/instances/{outpost_id}/")
|
||||
|
||||
print(f"\nOutpost Details:")
|
||||
print(f" Name: {outpost_details.get('name')}")
|
||||
print(f" ID: {outpost_details.get('pk')}")
|
||||
print(f" Type: {outpost_details.get('type')}")
|
||||
print(f" Providers: {outpost_details.get('providers')}")
|
||||
|
||||
# The outpost service connection details
|
||||
if 'service_connection' in outpost_details:
|
||||
print(f" Service Connection: {outpost_details.get('service_connection')}")
|
||||
|
||||
# List tokens to find the one for this outpost
|
||||
print("\n5. Looking for outpost service token...")
|
||||
tokens = await authentik.list_tokens()
|
||||
|
||||
outpost_uuid = outpost_details.get('pk')
|
||||
token_identifier = f"ak-outpost-{outpost_uuid}-api"
|
||||
|
||||
matching_token = None
|
||||
for token in tokens:
|
||||
if token.get('identifier') == token_identifier:
|
||||
matching_token = token
|
||||
break
|
||||
|
||||
if matching_token:
|
||||
print(f"✓ Found token: {matching_token.get('identifier')}")
|
||||
print(f"\n{'=' * 60}")
|
||||
print("IMPORTANT: Token Key Required")
|
||||
print("=" * 60)
|
||||
print("\nAuthentik does not expose token keys via API after creation.")
|
||||
print("\nTo get the token key:")
|
||||
print("1. Go to: https://auth.schweitz.net/if/admin/#/core/tokens")
|
||||
print(f"2. Find token: {token_identifier}")
|
||||
print("3. Click 'View Token Key' or regenerate the token")
|
||||
print("4. Copy the token key")
|
||||
print("\nAlternatively, you can:")
|
||||
print("1. Delete the existing outpost via UI")
|
||||
print("2. Create a new outpost via UI")
|
||||
print("3. Copy the token key when it's displayed")
|
||||
print("\n" + "=" * 60)
|
||||
else:
|
||||
print("\n⚠ No automatic token found.")
|
||||
print("You may need to manually create a token for the outpost.")
|
||||
print("\nManual token creation:")
|
||||
print("1. Go to: https://auth.schweitz.net/if/admin/#/core/tokens")
|
||||
print("2. Click 'Create'")
|
||||
print(f"3. Identifier: npm-outpost-token")
|
||||
print("4. User: Select the outpost service account")
|
||||
print("5. Intent: API")
|
||||
print("6. Copy the token key when displayed")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("Next Steps")
|
||||
print("=" * 60)
|
||||
print("\n1. Obtain the outpost token key (see above)")
|
||||
print("2. Create/update .env.authentik-shared file:")
|
||||
print(" AUTHENTIK_OUTPOST_TOKEN=<your_token_key>")
|
||||
print("3. Deploy the stack:")
|
||||
print(" docker-compose -f stacks/authentik-shared.yml --env-file .env.authentik-shared up -d")
|
||||
print("4. Update NPM hosts to point to port 9001")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(create_npm_outpost())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -1,118 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Setup Authentik Forward Authentication for NPM
|
||||
|
||||
This script creates a proxy provider and outpost for forward authentication
|
||||
across all NPM-managed domains.
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
sys.path.insert(0, '/home/jpmschweitzer/Projects/portainer-core/services/core-api')
|
||||
|
||||
from src.clients.authentik_client import get_authentik_client
|
||||
|
||||
|
||||
async def setup_forward_auth():
|
||||
"""Create proxy provider, application, and outpost for forward auth"""
|
||||
client = get_authentik_client()
|
||||
|
||||
try:
|
||||
# Check if Authentik is accessible
|
||||
print("Checking Authentik connectivity...")
|
||||
if not await client.health_check():
|
||||
print("❌ Authentik is not accessible")
|
||||
return False
|
||||
print("✓ Authentik is accessible")
|
||||
|
||||
# Check if proxy provider already exists
|
||||
print("\nChecking for existing proxy provider...")
|
||||
existing_provider = await client.get_provider_by_name_proxy("npm-forward-auth-provider")
|
||||
|
||||
if existing_provider:
|
||||
print(f"✓ Proxy provider already exists (ID: {existing_provider.get('pk')})")
|
||||
provider = existing_provider
|
||||
else:
|
||||
# Create Proxy provider for forward auth
|
||||
print("\nCreating Proxy provider for forward authentication...")
|
||||
provider = await client.create_proxy_provider(
|
||||
name="npm-forward-auth-provider",
|
||||
external_host="https://auth.schweitz.net",
|
||||
mode="forward_single",
|
||||
token_validity=480 # 8 hours
|
||||
)
|
||||
print(f"✓ Created proxy provider (ID: {provider.get('pk')})")
|
||||
|
||||
# Display provider details
|
||||
print("\n" + "="*60)
|
||||
print("Proxy Provider Details:")
|
||||
print("="*60)
|
||||
print(f"Provider ID: {provider.get('pk')}")
|
||||
print(f"Mode: {provider.get('mode')}")
|
||||
print(f"External Host: {provider.get('external_host')}")
|
||||
print(f"Token Validity: {provider.get('access_token_validity')}")
|
||||
print(f"Session Duration: {provider.get('session_duration')}")
|
||||
print("="*60)
|
||||
|
||||
# Check if application already exists
|
||||
print("\nChecking for existing application...")
|
||||
existing_app = await client.get_application_by_slug("npm-forward-auth")
|
||||
|
||||
if existing_app:
|
||||
print(f"✓ Application already exists (slug: {existing_app.get('slug')})")
|
||||
app = existing_app
|
||||
else:
|
||||
# Create application
|
||||
print("\nCreating application...")
|
||||
app = await client.create_application(
|
||||
name="NPM Forward Auth",
|
||||
slug="npm-forward-auth",
|
||||
provider_pk=provider.get("pk"),
|
||||
launch_url="https://auth.schweitz.net"
|
||||
)
|
||||
print(f"✓ Created application (slug: {app.get('slug')})")
|
||||
|
||||
# Check if outpost already exists
|
||||
print("\nChecking for existing outpost...")
|
||||
existing_outpost = await client.get_outpost_by_name("npm-forward-auth-outpost")
|
||||
|
||||
if existing_outpost:
|
||||
print(f"✓ Outpost already exists (ID: {existing_outpost.get('pk')})")
|
||||
outpost = existing_outpost
|
||||
else:
|
||||
# Create outpost
|
||||
print("\nCreating outpost...")
|
||||
outpost = await client.create_outpost(
|
||||
name="npm-forward-auth-outpost",
|
||||
type="proxy",
|
||||
providers=[provider.get("pk")],
|
||||
config={
|
||||
"authentik_host": "https://auth.schweitz.net",
|
||||
"authentik_host_insecure": False,
|
||||
"log_level": "info"
|
||||
}
|
||||
)
|
||||
print(f"✓ Created outpost (ID: {outpost.get('pk')})")
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("Setup Complete!")
|
||||
print("="*60)
|
||||
print("\nNext steps:")
|
||||
print("1. Deploy the Authentik outpost container")
|
||||
print("2. Configure NPM proxy hosts with forward auth")
|
||||
print("3. Test the SSO flow")
|
||||
print("\nOutpost deployment command will be generated...")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
finally:
|
||||
await client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(setup_forward_auth())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -4,7 +4,7 @@ OIDC Authentication Module
|
||||
Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP.
|
||||
Implements bearer token authentication with JWT verification.
|
||||
"""
|
||||
from fastapi import Depends, HTTPException, Security
|
||||
from fastapi import Depends, HTTPException, Security, Request
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
from jose import jwt, JWTError
|
||||
import httpx
|
||||
@@ -233,3 +233,104 @@ async def get_optional_user(
|
||||
except HTTPException:
|
||||
# Invalid token - return None instead of raising
|
||||
return None
|
||||
|
||||
|
||||
async def get_forward_auth_user(
|
||||
request: Request
|
||||
) -> Optional[Dict]:
|
||||
"""
|
||||
Authentik Forward Auth authentication for external access via NPM
|
||||
|
||||
This dependency allows:
|
||||
- External access through api.schweitz.net (with Authentik forward auth headers) - REQUIRES authentication
|
||||
- Internal direct access (no forward auth headers) - ALLOWED without authentication
|
||||
|
||||
When accessing through NPM with Authentik forward auth enabled, NPM adds headers like:
|
||||
- X-authentik-username
|
||||
- X-authentik-email
|
||||
- X-authentik-groups
|
||||
- X-authentik-name
|
||||
- X-authentik-uid
|
||||
|
||||
Args:
|
||||
request: FastAPI request object containing headers
|
||||
|
||||
Returns:
|
||||
User info dict if authenticated via forward auth headers
|
||||
None if accessed internally (no forward auth headers)
|
||||
|
||||
Raises:
|
||||
HTTPException 401: If forward auth headers present but invalid/incomplete
|
||||
"""
|
||||
# Check for Authentik forward auth headers
|
||||
username = request.headers.get("x-authentik-username")
|
||||
email = request.headers.get("x-authentik-email")
|
||||
groups = request.headers.get("x-authentik-groups")
|
||||
name = request.headers.get("x-authentik-name")
|
||||
uid = request.headers.get("x-authentik-uid")
|
||||
|
||||
# If NO forward auth headers present, this is internal access - allow it
|
||||
if not username and not email:
|
||||
logger.debug("No forward auth headers - allowing internal access")
|
||||
return None
|
||||
|
||||
# Forward auth headers present (external access via api.schweitz.net)
|
||||
# Validate authentication
|
||||
if not username or not email:
|
||||
logger.warning("Incomplete forward auth headers detected")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required - incomplete forward auth headers"
|
||||
)
|
||||
|
||||
# Parse groups (comma-separated string to list)
|
||||
groups_list = [g.strip() for g in groups.split(",")] if groups else []
|
||||
|
||||
user_info = {
|
||||
"username": username,
|
||||
"email": email,
|
||||
"name": name or username,
|
||||
"groups": groups_list,
|
||||
"uid": uid,
|
||||
"auth_method": "forward_auth"
|
||||
}
|
||||
|
||||
logger.info(f"Authenticated via forward auth: {email} (groups: {groups_list})")
|
||||
return user_info
|
||||
|
||||
|
||||
async def get_forward_auth_admin(
|
||||
user: Optional[Dict] = Depends(get_forward_auth_user)
|
||||
) -> Dict:
|
||||
"""
|
||||
Require admin access for external requests, allow all internal requests
|
||||
|
||||
Use this dependency for endpoints that require admin access when accessed
|
||||
externally through api.schweitz.net, but allow unrestricted internal access.
|
||||
|
||||
Args:
|
||||
user: User info from get_forward_auth_user
|
||||
|
||||
Returns:
|
||||
User info dict if user is admin or if accessed internally
|
||||
|
||||
Raises:
|
||||
HTTPException 403: If external user is not in admin/authentik Admins group
|
||||
"""
|
||||
# Internal access (no forward auth headers) - allow all
|
||||
if user is None:
|
||||
logger.debug("Internal access - allowing without admin check")
|
||||
return {"email": "internal", "groups": ["admin"], "auth_method": "internal"}
|
||||
|
||||
# External access - check admin group membership
|
||||
groups = user.get("groups", [])
|
||||
|
||||
if "admin" not in groups and "authentik Admins" not in groups:
|
||||
user_email = user.get("email", "unknown")
|
||||
logger.warning(f"User {user_email} attempted admin access (groups: {groups})")
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="Admin access required"
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
@@ -2,9 +2,12 @@
|
||||
Uptime Kuma Socket.IO Client
|
||||
|
||||
Provides interface to Uptime Kuma via Socket.IO for monitor management.
|
||||
Also provides metrics API access for real-time status data.
|
||||
"""
|
||||
import socketio
|
||||
import asyncio
|
||||
import httpx
|
||||
import re
|
||||
from typing import Optional, Dict, List, Any
|
||||
from src.logging_config import get_logger
|
||||
from src.config import get_settings
|
||||
@@ -125,31 +128,76 @@ class KumaClient:
|
||||
|
||||
async def get_monitors(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
List all monitors
|
||||
List all monitors with uptime data
|
||||
|
||||
Returns:
|
||||
List of monitor configurations
|
||||
List of monitor configurations with uptime_24h field
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Get monitor list
|
||||
response = await self.sio.call('getMonitorList', timeout=self.timeout)
|
||||
# Storage for monitor list and uptime data received via events
|
||||
monitor_list_data = {}
|
||||
uptime_list_data = {}
|
||||
monitor_event_received = asyncio.Event()
|
||||
uptime_event_received = asyncio.Event()
|
||||
|
||||
if response and isinstance(response, dict):
|
||||
# Uptime Kuma returns monitors as a dict with monitor IDs as keys
|
||||
# Register event handler for monitorList
|
||||
@self.sio.event
|
||||
async def monitorList(data):
|
||||
nonlocal monitor_list_data
|
||||
monitor_list_data = data
|
||||
monitor_event_received.set()
|
||||
|
||||
# Register event handler for uptimeList (24h uptime percentages)
|
||||
@self.sio.event
|
||||
async def uptimeList(monitor_id, uptime_data):
|
||||
nonlocal uptime_list_data
|
||||
# uptime_data is typically a dict with time periods: {"24": 99.5, "720": 98.2, ...}
|
||||
uptime_list_data[str(monitor_id)] = uptime_data
|
||||
# Don't set event here as we'll get multiple calls
|
||||
|
||||
# Request monitor list - this triggers the server to send monitorList event
|
||||
response = await self.sio.call('getMonitorList', timeout=self.timeout)
|
||||
logger.info(f"getMonitorList call response: {response}")
|
||||
|
||||
# Wait for the monitorList event (with timeout)
|
||||
try:
|
||||
await asyncio.wait_for(monitor_event_received.wait(), timeout=5.0)
|
||||
logger.info(f"Received monitorList event with {len(monitor_list_data)} items")
|
||||
|
||||
# Give time for uptimeList events to arrive
|
||||
await asyncio.sleep(0.5)
|
||||
logger.info(f"Received uptime data for {len(uptime_list_data)} monitors")
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("Timeout waiting for monitorList event")
|
||||
|
||||
# Process the monitor list data
|
||||
if monitor_list_data and isinstance(monitor_list_data, dict):
|
||||
monitors = []
|
||||
for monitor_id, monitor_data in response.items():
|
||||
for monitor_id, monitor_data in monitor_list_data.items():
|
||||
if isinstance(monitor_data, dict):
|
||||
monitor_data['id'] = int(monitor_id)
|
||||
|
||||
# Add uptime data if available
|
||||
uptime_info = uptime_list_data.get(str(monitor_id), {})
|
||||
if isinstance(uptime_info, dict):
|
||||
# Uptime Kuma provides 24h uptime as key "24"
|
||||
monitor_data['uptime_24h'] = float(uptime_info.get('24', 0))
|
||||
else:
|
||||
monitor_data['uptime_24h'] = 0.0
|
||||
|
||||
monitors.append(monitor_data)
|
||||
self._monitors_cache[int(monitor_id)] = monitor_data
|
||||
|
||||
logger.info(f"Found {len(monitors)} monitors total")
|
||||
return monitors
|
||||
|
||||
logger.warning(f"No valid monitor data received")
|
||||
return []
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get monitors: {e}")
|
||||
logger.error(f"Failed to get monitors: {e}", exc_info=True)
|
||||
raise
|
||||
|
||||
async def get_monitor(self, monitor_id: int) -> Dict[str, Any]:
|
||||
@@ -419,6 +467,78 @@ class KumaClient:
|
||||
await self.delete_monitor(monitor["id"])
|
||||
return True
|
||||
|
||||
async def get_metrics_status(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""
|
||||
Get monitor status from Prometheus metrics endpoint
|
||||
|
||||
This is simpler and more reliable than Socket.IO for getting current status.
|
||||
Returns real-time UP/DOWN status but not historical uptime percentages.
|
||||
|
||||
Returns:
|
||||
Dict mapping monitor names to status info:
|
||||
{
|
||||
"Portainer": {
|
||||
"status": 1, # 1=UP, 0=DOWN, 2=PENDING, 3=MAINTENANCE
|
||||
"response_time": 5, # ms
|
||||
"monitor_type": "http",
|
||||
"url": "http://192.168.86.149:8001"
|
||||
},
|
||||
...
|
||||
}
|
||||
"""
|
||||
try:
|
||||
# Use API key authentication
|
||||
api_key = settings.kuma_api_key
|
||||
if not api_key:
|
||||
logger.warning("Kuma API key not configured")
|
||||
return {}
|
||||
|
||||
# Fetch metrics with HTTP Basic Auth (empty username, API key as password)
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
response = await client.get(
|
||||
f"{self.base_url}/metrics",
|
||||
auth=("", api_key)
|
||||
)
|
||||
response.raise_for_status()
|
||||
metrics_text = response.text
|
||||
|
||||
# Parse Prometheus format metrics
|
||||
# Format: metric_name{label1="value1",label2="value2"} value
|
||||
monitor_data = {}
|
||||
|
||||
# Parse monitor_status lines
|
||||
status_pattern = r'monitor_status\{monitor_name="([^"]+)",.*?\} (\d+)'
|
||||
for match in re.finditer(status_pattern, metrics_text):
|
||||
monitor_name = match.group(1)
|
||||
status = int(match.group(2))
|
||||
|
||||
if monitor_name not in monitor_data:
|
||||
monitor_data[monitor_name] = {}
|
||||
monitor_data[monitor_name]['status'] = status
|
||||
|
||||
# Parse monitor_response_time lines
|
||||
response_pattern = r'monitor_response_time\{monitor_name="([^"]+)",monitor_type="([^"]+)",monitor_url="([^"]+)",.*?\} ([\d.]+)'
|
||||
for match in re.finditer(response_pattern, metrics_text):
|
||||
monitor_name = match.group(1)
|
||||
monitor_type = match.group(2)
|
||||
monitor_url = match.group(3)
|
||||
response_time = float(match.group(4))
|
||||
|
||||
if monitor_name not in monitor_data:
|
||||
monitor_data[monitor_name] = {}
|
||||
monitor_data[monitor_name].update({
|
||||
'response_time': response_time,
|
||||
'monitor_type': monitor_type,
|
||||
'url': monitor_url
|
||||
})
|
||||
|
||||
logger.info(f"Fetched metrics for {len(monitor_data)} monitors")
|
||||
return monitor_data
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch metrics: {e}")
|
||||
return {}
|
||||
|
||||
async def __aenter__(self):
|
||||
"""Async context manager entry"""
|
||||
await self._ensure_connected()
|
||||
|
||||
@@ -9,7 +9,7 @@ try:
|
||||
from src.credentials import (
|
||||
PORTAINER_URL, PORTAINER_API_KEY,
|
||||
NPM_URL, NPM_EMAIL, NPM_PASSWORD,
|
||||
KUMA_URL, KUMA_USERNAME, KUMA_PASSWORD
|
||||
KUMA_URL, KUMA_USERNAME, KUMA_PASSWORD, KUMA_API_KEY
|
||||
)
|
||||
except ImportError:
|
||||
# Fallback to empty strings if credentials.py doesn't exist
|
||||
@@ -22,6 +22,7 @@ except ImportError:
|
||||
KUMA_URL = "http://localhost:3001"
|
||||
KUMA_USERNAME = ""
|
||||
KUMA_PASSWORD = ""
|
||||
KUMA_API_KEY = ""
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
@@ -43,7 +44,7 @@ class Settings(BaseSettings):
|
||||
cors_headers: list[str] = ["*"]
|
||||
|
||||
# Logging
|
||||
log_level: str = "INFO"
|
||||
log_level: str = "DEBUG"
|
||||
|
||||
# Ollama Configuration (for AI orchestration)
|
||||
ollama_base_url: str = "http://ollama:11434"
|
||||
@@ -88,6 +89,7 @@ class Settings(BaseSettings):
|
||||
kuma_url: str = KUMA_URL
|
||||
kuma_username: str = KUMA_USERNAME
|
||||
kuma_password: str = KUMA_PASSWORD
|
||||
kuma_api_key: str = KUMA_API_KEY
|
||||
|
||||
# OIDC Authentication (Authentik)
|
||||
oidc_enabled: bool = False # Set to True to require authentication
|
||||
|
||||
@@ -14,7 +14,7 @@ from src.clients.npm_client import get_npm_client
|
||||
from src.clients.kuma_client import get_kuma_client
|
||||
from src.logging_config import get_logger
|
||||
from src import service_groups
|
||||
from src.auth.oidc import get_admin_user
|
||||
from src.auth.oidc import get_admin_user, get_forward_auth_admin
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -751,11 +751,11 @@ class InfrastructureController(BaseController):
|
||||
"/services/{name}/stop",
|
||||
response_model=OperationResult,
|
||||
summary="Stop a service or service group",
|
||||
description="Stop a service or service group by pausing monitors and stopping containers. Requires admin authentication."
|
||||
description="Stop a service or service group by pausing monitors and stopping containers. Requires admin authentication when accessed externally via api.schweitz.net."
|
||||
)
|
||||
async def stop_service(
|
||||
name: str,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
user: Dict = Depends(get_forward_auth_admin)
|
||||
):
|
||||
"""
|
||||
Stop a service or service group
|
||||
@@ -866,11 +866,11 @@ class InfrastructureController(BaseController):
|
||||
"/services/{name}/start",
|
||||
response_model=OperationResult,
|
||||
summary="Start a service or service group",
|
||||
description="Start a service or service group by starting containers and resuming monitors. Requires admin authentication."
|
||||
description="Start a service or service group by starting containers and resuming monitors. Requires admin authentication when accessed externally via api.schweitz.net."
|
||||
)
|
||||
async def start_service(
|
||||
name: str,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
user: Dict = Depends(get_forward_auth_admin)
|
||||
):
|
||||
"""
|
||||
Start a service or service group
|
||||
@@ -971,6 +971,118 @@ class InfrastructureController(BaseController):
|
||||
|
||||
# ===== Monitoring Endpoints =====
|
||||
|
||||
@router.get(
|
||||
"/widget-data",
|
||||
summary="Get combined data for service control widget",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def get_widget_data():
|
||||
"""
|
||||
Get combined service and monitor data for the widget
|
||||
|
||||
Returns all data needed by service-control widget in a single call:
|
||||
- Service list with status and container counts
|
||||
- Monitor list with uptime percentages
|
||||
- Service groups and always-on list
|
||||
|
||||
This endpoint is designed for browser-based widgets to avoid
|
||||
multiple API calls and cross-origin issues.
|
||||
"""
|
||||
try:
|
||||
portainer = get_portainer_client()
|
||||
kuma = get_kuma_client()
|
||||
npm = get_npm_client()
|
||||
|
||||
# Fetch services (same logic as /services endpoint)
|
||||
stacks = await portainer.get_stacks()
|
||||
proxy_hosts = await npm.get_proxy_hosts()
|
||||
|
||||
# Build domain mapping
|
||||
domain_map = {}
|
||||
for proxy in proxy_hosts:
|
||||
for domain in proxy.get("domain_names", []):
|
||||
forward_host = proxy.get("forward_host", "")
|
||||
domain_map[domain] = forward_host
|
||||
|
||||
services = []
|
||||
for stack in stacks:
|
||||
stack_name = stack.get("Name", "")
|
||||
endpoint_id = stack.get("EndpointId")
|
||||
domains = [
|
||||
domain for domain, host in domain_map.items()
|
||||
if stack_name in host or host in stack_name
|
||||
]
|
||||
|
||||
# Get container status
|
||||
containers_running = 0
|
||||
containers_total = 0
|
||||
try:
|
||||
all_containers = await portainer.get_containers(endpoint_id, all_containers=True)
|
||||
for container in all_containers:
|
||||
labels = container.get("Labels", {})
|
||||
container_stack = labels.get("com.docker.compose.project", "")
|
||||
|
||||
if container_stack.lower() == stack_name.lower():
|
||||
containers_total += 1
|
||||
if container.get("State", "") == "running":
|
||||
containers_running += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get container status for {stack_name}: {e}")
|
||||
|
||||
services.append({
|
||||
"name": stack_name,
|
||||
"stack_id": stack.get("Id"),
|
||||
"status": "active" if stack.get("Status") == 1 else "inactive",
|
||||
"endpoint_id": endpoint_id,
|
||||
"domains": domains,
|
||||
"running": containers_running > 0,
|
||||
"containers_running": containers_running,
|
||||
"containers_total": containers_total
|
||||
})
|
||||
|
||||
# Fetch monitors with real-time status from metrics endpoint
|
||||
monitors_list = []
|
||||
try:
|
||||
# Get real-time status from Prometheus metrics
|
||||
metrics_data = await kuma.get_metrics_status()
|
||||
|
||||
for monitor_name, monitor_info in metrics_data.items():
|
||||
# Status: 1=UP, 0=DOWN, 2=PENDING, 3=MAINTENANCE
|
||||
status = monitor_info.get('status', 0)
|
||||
|
||||
# Convert status to simple up/down for widget
|
||||
# Treat UP (1) as 100%, anything else as 0%
|
||||
status_percentage = 100.0 if status == 1 else 0.0
|
||||
|
||||
monitors_list.append({
|
||||
"id": None, # Not available from metrics
|
||||
"name": monitor_name,
|
||||
"uptime_24h": status_percentage, # Current status as percentage
|
||||
"active": True, # Assume active if in metrics
|
||||
"status": status, # 1=UP, 0=DOWN, 2=PENDING, 3=MAINTENANCE
|
||||
"response_time": monitor_info.get('response_time', 0)
|
||||
})
|
||||
|
||||
logger.info(f"Fetched status for {len(monitors_list)} monitors from metrics")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to fetch monitors: {e}")
|
||||
# Continue without monitor data rather than failing
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"services": services,
|
||||
"monitors": monitors_list,
|
||||
"service_groups": {
|
||||
"groups": service_groups.list_service_groups(),
|
||||
"always_on": list(service_groups.ALWAYS_ON_SERVICES),
|
||||
"stoppable": service_groups.list_stoppable_services()
|
||||
}
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch widget data: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to fetch widget data: {str(e)}")
|
||||
|
||||
@router.get(
|
||||
"/monitors",
|
||||
summary="List all monitors",
|
||||
|
||||
@@ -16,6 +16,9 @@ ALWAYS_ON_SERVICES: Set[str] = {
|
||||
"watchtower",
|
||||
"netdata",
|
||||
"maintenance",
|
||||
"postgres-shared",
|
||||
"redis-shared",
|
||||
"authentik",
|
||||
}
|
||||
|
||||
# Service groups - services that should be started/stopped together
|
||||
@@ -25,8 +28,6 @@ SERVICE_GROUPS: Dict[str, List[str]] = {
|
||||
],
|
||||
"nextcloud": [
|
||||
"nextcloud",
|
||||
"nextcloud-db",
|
||||
"nextcloud-redis",
|
||||
],
|
||||
"gitea": [
|
||||
"gitea",
|
||||
|
||||
@@ -15,106 +15,163 @@
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: transparent;
|
||||
color: #e0e0e0;
|
||||
padding: 10px;
|
||||
padding: 15px;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
max-width: 100%;
|
||||
}
|
||||
|
||||
h2 {
|
||||
.section {
|
||||
margin-bottom: 30px;
|
||||
}
|
||||
|
||||
.section-header {
|
||||
color: #fff;
|
||||
margin-bottom: 15px;
|
||||
font-size: 20px;
|
||||
font-weight: 500;
|
||||
font-size: 18px;
|
||||
font-weight: 600;
|
||||
margin-bottom: 12px;
|
||||
padding-bottom: 8px;
|
||||
border-bottom: 2px solid rgba(255, 255, 255, 0.1);
|
||||
}
|
||||
|
||||
.service-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
.service-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.service-row {
|
||||
background: rgba(40, 40, 40, 0.95);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 6px;
|
||||
padding: 12px 16px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
transition: all 0.2s ease;
|
||||
gap: 15px;
|
||||
}
|
||||
|
||||
.service-card {
|
||||
background: rgba(40, 40, 40, 0.95);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
padding: 15px;
|
||||
transition: all 0.3s ease;
|
||||
.service-row:hover {
|
||||
border-color: rgba(66, 153, 225, 0.4);
|
||||
background: rgba(45, 45, 45, 0.95);
|
||||
}
|
||||
|
||||
.service-card:hover {
|
||||
border-color: rgba(66, 153, 225, 0.5);
|
||||
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
.service-header {
|
||||
.service-left {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 12px;
|
||||
gap: 15px;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.service-name {
|
||||
font-size: 16px;
|
||||
font-size: 15px;
|
||||
font-weight: 600;
|
||||
color: #fff;
|
||||
text-transform: capitalize;
|
||||
min-width: 300px;
|
||||
}
|
||||
|
||||
.status-badge {
|
||||
padding: 4px 12px;
|
||||
border-radius: 12px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
.service-status {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
min-width: 120px;
|
||||
}
|
||||
|
||||
.status-running {
|
||||
background: rgba(72, 187, 120, 0.2);
|
||||
color: #48bb78;
|
||||
border: 1px solid rgba(72, 187, 120, 0.4);
|
||||
.status-indicator {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.status-stopped {
|
||||
background: rgba(245, 101, 101, 0.2);
|
||||
color: #f56565;
|
||||
border: 1px solid rgba(245, 101, 101, 0.4);
|
||||
.status-indicator.running {
|
||||
background: #48bb78;
|
||||
box-shadow: 0 0 8px rgba(72, 187, 120, 0.6);
|
||||
}
|
||||
|
||||
.status-loading {
|
||||
background: rgba(237, 137, 54, 0.2);
|
||||
color: #ed8936;
|
||||
border: 1px solid rgba(237, 137, 54, 0.4);
|
||||
.status-indicator.stopped {
|
||||
background: #f56565;
|
||||
box-shadow: 0 0 8px rgba(245, 101, 101, 0.6);
|
||||
}
|
||||
|
||||
.service-info {
|
||||
.status-text {
|
||||
font-size: 13px;
|
||||
color: #a0a0a0;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.service-actions {
|
||||
.uptime-status {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
min-width: 150px;
|
||||
padding: 4px 10px;
|
||||
background: rgba(0, 0, 0, 0.2);
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
transition: background 0.2s;
|
||||
text-decoration: none;
|
||||
color: inherit;
|
||||
}
|
||||
|
||||
.uptime-status:hover {
|
||||
background: rgba(0, 0, 0, 0.4);
|
||||
}
|
||||
|
||||
.uptime-percentage {
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.uptime-percentage.excellent {
|
||||
color: #48bb78;
|
||||
}
|
||||
|
||||
.uptime-percentage.good {
|
||||
color: #68d391;
|
||||
}
|
||||
|
||||
.uptime-percentage.warning {
|
||||
color: #ed8936;
|
||||
}
|
||||
|
||||
.uptime-percentage.critical {
|
||||
color: #f56565;
|
||||
}
|
||||
|
||||
.uptime-percentage.unknown {
|
||||
color: #718096;
|
||||
}
|
||||
|
||||
.uptime-icon {
|
||||
font-size: 11px;
|
||||
color: #a0a0a0;
|
||||
}
|
||||
|
||||
.service-right {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.btn {
|
||||
flex: 1;
|
||||
padding: 8px 12px;
|
||||
padding: 6px 16px;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 13px;
|
||||
border-radius: 4px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
min-width: 70px;
|
||||
}
|
||||
|
||||
.btn:disabled {
|
||||
opacity: 0.5;
|
||||
opacity: 0.3;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
@@ -126,6 +183,7 @@
|
||||
.btn-start:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #38a169 0%, #2f855a 100%);
|
||||
transform: translateY(-1px);
|
||||
box-shadow: 0 2px 8px rgba(72, 187, 120, 0.3);
|
||||
}
|
||||
|
||||
.btn-stop {
|
||||
@@ -136,21 +194,12 @@
|
||||
.btn-stop:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #e53e3e 0%, #c53030 100%);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.btn-restart {
|
||||
background: linear-gradient(135deg, #4299e1 0%, #3182ce 100%);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.btn-restart:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #3182ce 0%, #2c5282 100%);
|
||||
transform: translateY(-1px);
|
||||
box-shadow: 0 2px 8px rgba(245, 101, 101, 0.3);
|
||||
}
|
||||
|
||||
.loading {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
padding: 30px;
|
||||
color: #a0a0a0;
|
||||
}
|
||||
|
||||
@@ -164,14 +213,14 @@
|
||||
}
|
||||
|
||||
.always-on-badge {
|
||||
display: inline-block;
|
||||
padding: 2px 8px;
|
||||
background: rgba(66, 153, 225, 0.2);
|
||||
font-size: 10px;
|
||||
color: #4299e1;
|
||||
border: 1px solid rgba(66, 153, 225, 0.4);
|
||||
border-radius: 10px;
|
||||
font-size: 11px;
|
||||
background: rgba(66, 153, 225, 0.15);
|
||||
padding: 2px 6px;
|
||||
border-radius: 3px;
|
||||
margin-left: 8px;
|
||||
text-transform: uppercase;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
@@ -180,45 +229,102 @@
|
||||
|
||||
.spinner {
|
||||
display: inline-block;
|
||||
width: 14px;
|
||||
height: 14px;
|
||||
width: 12px;
|
||||
height: 12px;
|
||||
border: 2px solid rgba(255, 255, 255, 0.3);
|
||||
border-top-color: #fff;
|
||||
border-radius: 50%;
|
||||
animation: spin 0.6s linear infinite;
|
||||
margin-right: 6px;
|
||||
}
|
||||
|
||||
/* Responsive design */
|
||||
@media (max-width: 768px) {
|
||||
.service-row {
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.service-name {
|
||||
min-width: 100px;
|
||||
}
|
||||
|
||||
.uptime-status {
|
||||
min-width: 100px;
|
||||
}
|
||||
|
||||
.service-right {
|
||||
width: 100%;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h2>🎛️ On-Demand Services</h2>
|
||||
<div id="error-container"></div>
|
||||
<div id="service-container" class="loading">Loading services...</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">🎛️ Stoppable Services</div>
|
||||
<div id="stoppable-container" class="loading">Loading services...</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<div class="section-header">🔒 Always-On Infrastructure</div>
|
||||
<div id="always-on-container" class="loading">Loading infrastructure...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Auto-detect API base from current domain (works with NPM proxy)
|
||||
const API_BASE = window.location.origin;
|
||||
// Use relative URL to work in any context (iframe, direct access, etc.)
|
||||
const API_BASE = '';
|
||||
const KUMA_BASE = window.location.protocol + '//' + window.location.hostname + ':3001';
|
||||
|
||||
let services = [];
|
||||
let alwaysOnServices = [];
|
||||
let monitors = {};
|
||||
|
||||
async function fetchServices() {
|
||||
async function fetchData() {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/infrastructure/services`);
|
||||
if (!response.ok) throw new Error('Failed to fetch services');
|
||||
services = await response.json();
|
||||
// Single API call to get all data
|
||||
const response = await fetch(`${API_BASE}/infrastructure/widget-data`);
|
||||
|
||||
const groupsResponse = await fetch(`${API_BASE}/infrastructure/service-groups`);
|
||||
if (groupsResponse.ok) {
|
||||
const groupsData = await groupsResponse.json();
|
||||
alwaysOnServices = groupsData.always_on || [];
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!data.success) {
|
||||
throw new Error('API returned unsuccessful response');
|
||||
}
|
||||
|
||||
// Update services
|
||||
services = data.services || [];
|
||||
|
||||
// Update always-on services list
|
||||
if (data.service_groups && data.service_groups.always_on) {
|
||||
alwaysOnServices = data.service_groups.always_on;
|
||||
}
|
||||
|
||||
// Build monitors map
|
||||
const monitorsMap = {};
|
||||
if (data.monitors) {
|
||||
data.monitors.forEach(monitor => {
|
||||
const name = monitor.name.toLowerCase().replace(/[^a-z0-9]/g, '-');
|
||||
monitorsMap[name] = {
|
||||
id: monitor.id,
|
||||
uptime_24h: monitor.uptime_24h || 0,
|
||||
active: monitor.active !== false
|
||||
};
|
||||
});
|
||||
}
|
||||
monitors = monitorsMap;
|
||||
|
||||
renderServices();
|
||||
document.getElementById('error-container').innerHTML = '';
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error fetching services:', error);
|
||||
console.error('Error fetching data:', error);
|
||||
document.getElementById('error-container').innerHTML =
|
||||
`<div class="error">❌ Failed to connect to API: ${error.message}</div>`;
|
||||
}
|
||||
@@ -228,80 +334,118 @@
|
||||
return alwaysOnServices.includes(serviceName.toLowerCase());
|
||||
}
|
||||
|
||||
function getServiceStatus(service) {
|
||||
if (service.containers_running > 0) {
|
||||
function getUptimeInfo(serviceName) {
|
||||
const monitorKey = serviceName.toLowerCase().replace(/[^a-z0-9]/g, '-');
|
||||
const monitor = monitors[monitorKey];
|
||||
|
||||
if (!monitor) {
|
||||
return {
|
||||
class: 'status-running',
|
||||
text: `Running (${service.containers_running}/${service.containers_total})`
|
||||
};
|
||||
} else if (service.containers_total > 0) {
|
||||
return {
|
||||
class: 'status-stopped',
|
||||
text: 'Stopped'
|
||||
};
|
||||
} else {
|
||||
return {
|
||||
class: 'status-stopped',
|
||||
text: 'No containers'
|
||||
percentage: 0,
|
||||
class: 'unknown',
|
||||
text: 'No monitor',
|
||||
id: null
|
||||
};
|
||||
}
|
||||
|
||||
const uptime = monitor.uptime_24h;
|
||||
let className = 'unknown';
|
||||
|
||||
if (uptime >= 99.5) className = 'excellent';
|
||||
else if (uptime >= 95) className = 'good';
|
||||
else if (uptime >= 90) className = 'warning';
|
||||
else if (uptime > 0) className = 'critical';
|
||||
|
||||
return {
|
||||
percentage: uptime,
|
||||
class: className,
|
||||
text: uptime > 0 ? `${uptime.toFixed(1)}% ↑` : 'Down',
|
||||
id: monitor.id
|
||||
};
|
||||
}
|
||||
|
||||
function renderServiceRow(service) {
|
||||
const isRunning = service.containers_running > 0;
|
||||
const alwaysOn = isAlwaysOn(service.name);
|
||||
const uptime = getUptimeInfo(service.name);
|
||||
|
||||
const kumaLink = uptime.id ?
|
||||
`${KUMA_BASE}/dashboard/${uptime.id}` :
|
||||
KUMA_BASE;
|
||||
|
||||
return `
|
||||
<div class="service-row" data-service="${service.name}">
|
||||
<div class="service-left">
|
||||
<div class="service-name">
|
||||
${service.name}
|
||||
${alwaysOn ? '<span class="always-on-badge">Protected</span>' : ''}
|
||||
</div>
|
||||
<div class="service-status">
|
||||
<div class="status-indicator ${isRunning ? 'running' : 'stopped'}"></div>
|
||||
<span class="status-text">
|
||||
${isRunning ? `Running (${service.containers_running}/${service.containers_total})` : 'Stopped'}
|
||||
</span>
|
||||
</div>
|
||||
<a href="${kumaLink}" target="_blank" class="uptime-status" title="View in Uptime Kuma">
|
||||
<span class="uptime-icon">📊</span>
|
||||
<span class="uptime-percentage ${uptime.class}">${uptime.text}</span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="service-right">
|
||||
<button class="btn btn-start"
|
||||
onclick="controlService('${service.name}', 'start')"
|
||||
${isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Start
|
||||
</button>
|
||||
<button class="btn btn-stop"
|
||||
onclick="controlService('${service.name}', 'stop')"
|
||||
${!isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Stop
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
function renderServices() {
|
||||
const container = document.getElementById('service-container');
|
||||
const stoppableContainer = document.getElementById('stoppable-container');
|
||||
const alwaysOnContainer = document.getElementById('always-on-container');
|
||||
|
||||
// Filter to only show stoppable services
|
||||
// Stoppable services
|
||||
const stoppableServices = services.filter(s => !isAlwaysOn(s.name));
|
||||
|
||||
if (stoppableServices.length === 0) {
|
||||
container.innerHTML = '<div class="loading">No stoppable services found</div>';
|
||||
return;
|
||||
stoppableContainer.innerHTML = '<div class="loading">No stoppable services found</div>';
|
||||
} else {
|
||||
stoppableContainer.className = 'service-list';
|
||||
stoppableContainer.innerHTML = stoppableServices
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
.map(service => renderServiceRow(service))
|
||||
.join('');
|
||||
}
|
||||
|
||||
container.className = 'service-grid';
|
||||
container.innerHTML = stoppableServices.map(service => {
|
||||
const status = getServiceStatus(service);
|
||||
const isRunning = service.containers_running > 0;
|
||||
const alwaysOn = isAlwaysOn(service.name);
|
||||
// Always-on services
|
||||
const alwaysOnServicesList = services.filter(s => isAlwaysOn(s.name));
|
||||
|
||||
return `
|
||||
<div class="service-card" data-service="${service.name}">
|
||||
<div class="service-header">
|
||||
<span class="service-name">
|
||||
${service.name}
|
||||
${alwaysOn ? '<span class="always-on-badge">ALWAYS ON</span>' : ''}
|
||||
</span>
|
||||
<span class="status-badge ${status.class}">${status.text}</span>
|
||||
</div>
|
||||
<div class="service-info">
|
||||
Stack ID: ${service.stack_id || 'N/A'}
|
||||
</div>
|
||||
<div class="service-actions">
|
||||
<button class="btn btn-start"
|
||||
onclick="controlService('${service.name}', 'start')"
|
||||
${isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Start
|
||||
</button>
|
||||
<button class="btn btn-stop"
|
||||
onclick="controlService('${service.name}', 'stop')"
|
||||
${!isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Stop
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
if (alwaysOnServicesList.length === 0) {
|
||||
alwaysOnContainer.innerHTML = '<div class="loading">No infrastructure services found</div>';
|
||||
} else {
|
||||
alwaysOnContainer.className = 'service-list';
|
||||
alwaysOnContainer.innerHTML = alwaysOnServicesList
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
.map(service => renderServiceRow(service))
|
||||
.join('');
|
||||
}
|
||||
}
|
||||
|
||||
async function controlService(serviceName, action) {
|
||||
const card = document.querySelector(`[data-service="${serviceName}"]`);
|
||||
const buttons = card.querySelectorAll('button');
|
||||
const row = document.querySelector(`[data-service="${serviceName}"]`);
|
||||
const buttons = row.querySelectorAll('button');
|
||||
|
||||
// Disable all buttons and show loading
|
||||
buttons.forEach(btn => {
|
||||
btn.disabled = true;
|
||||
if (btn.textContent.toLowerCase().includes(action)) {
|
||||
btn.innerHTML = `<span class="spinner"></span>${action.toUpperCase()}...`;
|
||||
btn.innerHTML = `<span class="spinner"></span>${action}`;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -318,26 +462,26 @@
|
||||
|
||||
console.log(`${action} ${serviceName}:`, result);
|
||||
|
||||
// Wait a bit for containers to start/stop
|
||||
// Wait for containers to start/stop
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
|
||||
// Refresh service list
|
||||
await fetchServices();
|
||||
await fetchData();
|
||||
|
||||
} catch (error) {
|
||||
console.error(`Error ${action}ing ${serviceName}:`, error);
|
||||
alert(`Failed to ${action} ${serviceName}: ${error.message}`);
|
||||
|
||||
// Re-enable buttons on error
|
||||
await fetchServices();
|
||||
await fetchData();
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-refresh every 10 seconds
|
||||
setInterval(fetchServices, 10000);
|
||||
setInterval(fetchData, 10000);
|
||||
|
||||
// Initial load
|
||||
fetchServices();
|
||||
fetchData();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Test what headers are being sent to Organizr
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
from src.clients.npm_client import get_npm_client
|
||||
|
||||
|
||||
async def main():
|
||||
npm = get_npm_client()
|
||||
|
||||
# Find home.schweitz.net
|
||||
hosts = await npm.get_proxy_hosts()
|
||||
|
||||
for host in hosts:
|
||||
if 'home.schweitz.net' in host.get('domain_names', []):
|
||||
print(f"Found: {', '.join(host.get('domain_names', []))}")
|
||||
print(f"Forward to: {host.get('forward_scheme')}://{host.get('forward_host')}:{host.get('forward_port')}")
|
||||
print()
|
||||
|
||||
config = host.get('advanced_config', '')
|
||||
|
||||
print("Checking for Authentik headers in nginx config:")
|
||||
print("=" * 60)
|
||||
|
||||
headers_to_check = [
|
||||
'X-authentik-username',
|
||||
'X-authentik-email',
|
||||
'X-authentik-groups',
|
||||
'X-authentik-name',
|
||||
'X-authentik-uid'
|
||||
]
|
||||
|
||||
for header in headers_to_check:
|
||||
if f'proxy_set_header {header}' in config:
|
||||
print(f"✓ {header} is configured")
|
||||
else:
|
||||
print(f"✗ {header} is NOT configured")
|
||||
|
||||
print()
|
||||
print("Full advanced config:")
|
||||
print("=" * 60)
|
||||
print(config)
|
||||
|
||||
break
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -1,115 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update all NPM proxy hosts to use port 9001 for Authentik forward auth
|
||||
"""
|
||||
import asyncio
|
||||
import httpx
|
||||
import os
|
||||
|
||||
NPM_URL = os.getenv("NPM_URL", "http://192.168.86.149:81")
|
||||
NPM_EMAIL = os.getenv("NPM_EMAIL", "admin@example.com")
|
||||
NPM_PASSWORD = os.getenv("NPM_PASSWORD", "changeme")
|
||||
|
||||
|
||||
async def get_npm_token():
|
||||
"""Get NPM authentication token"""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
f"{NPM_URL}/api/tokens",
|
||||
json={"identity": NPM_EMAIL, "secret": NPM_PASSWORD}
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()["token"]
|
||||
|
||||
|
||||
async def get_proxy_hosts(token):
|
||||
"""Get all proxy hosts"""
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{NPM_URL}/api/nginx/proxy-hosts",
|
||||
headers=headers
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def update_proxy_host(token, host_id, config):
|
||||
"""Update a proxy host"""
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.put(
|
||||
f"{NPM_URL}/api/nginx/proxy-hosts/{host_id}",
|
||||
headers=headers,
|
||||
json=config
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def main():
|
||||
print("Updating NPM proxy hosts to use port 9001...\n")
|
||||
|
||||
# Get token
|
||||
token = await get_npm_token()
|
||||
|
||||
# Get all proxy hosts
|
||||
hosts = await get_proxy_hosts(token)
|
||||
|
||||
updated = []
|
||||
skipped = []
|
||||
|
||||
for host in hosts:
|
||||
host_id = host.get("id")
|
||||
domain_names = host.get("domain_names", [])
|
||||
domain_str = ", ".join(domain_names)
|
||||
advanced_config = host.get("advanced_config", "")
|
||||
|
||||
# Skip if no authentik config
|
||||
if "authentik" not in advanced_config.lower():
|
||||
continue
|
||||
|
||||
# Skip if already port 9001
|
||||
if ":9001" in advanced_config:
|
||||
print(f"⊘ {domain_str} - Already using port 9001")
|
||||
skipped.append(domain_str)
|
||||
continue
|
||||
|
||||
# Update 9000 to 9001
|
||||
if ":9000" in advanced_config:
|
||||
print(f"⟳ {domain_str} - Updating to port 9001...", end=" ")
|
||||
|
||||
new_config = advanced_config.replace(":9000", ":9001")
|
||||
|
||||
# Clean config
|
||||
readonly_fields = [
|
||||
"id", "created_on", "modified_on", "owner", "owner_user_id",
|
||||
"certificate", "use_default_location", "ipv6", "meta",
|
||||
"nginx_online", "nginx_err", "access_list", "certificate_id"
|
||||
]
|
||||
|
||||
clean_host = {k: v for k, v in host.items() if k not in readonly_fields}
|
||||
clean_host["advanced_config"] = new_config
|
||||
|
||||
if "locations" not in clean_host or clean_host["locations"] is None:
|
||||
clean_host["locations"] = []
|
||||
|
||||
try:
|
||||
await update_proxy_host(token, host_id, clean_host)
|
||||
print("✓")
|
||||
updated.append(domain_str)
|
||||
except Exception as e:
|
||||
print(f"✗ Error: {e}")
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Updated: {len(updated)} hosts")
|
||||
print(f"Skipped: {len(skipped)} hosts")
|
||||
|
||||
if updated:
|
||||
print("\nUpdated hosts:")
|
||||
for d in updated:
|
||||
print(f" • {d}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -1,194 +0,0 @@
|
||||
version: '3.8'
|
||||
|
||||
# Authentik - Identity Provider for SSO (Using Shared Infrastructure)
|
||||
# Phase 1: Foundation - Google OAuth Integration
|
||||
# Ports: 9000 (HTTP), 9443 (HTTPS)
|
||||
# GPU: No
|
||||
# Dependencies: postgres-shared, redis-shared
|
||||
|
||||
services:
|
||||
authentik-server:
|
||||
image: ghcr.io/goauthentik/server:latest
|
||||
container_name: authentik-server
|
||||
restart: unless-stopped
|
||||
command: server
|
||||
ports:
|
||||
- "9000:9000"
|
||||
# Port 9443 removed - use NPM for HTTPS termination
|
||||
environment:
|
||||
# Database configuration (shared PostgreSQL)
|
||||
AUTHENTIK_POSTGRESQL__HOST: postgres-shared
|
||||
AUTHENTIK_POSTGRESQL__PORT: 5432
|
||||
AUTHENTIK_POSTGRESQL__NAME: authentik
|
||||
AUTHENTIK_POSTGRESQL__USER: authentik_user
|
||||
AUTHENTIK_POSTGRESQL__PASSWORD: ${AUTHENTIK_DB_PASSWORD:?database password required}
|
||||
|
||||
# Cache configuration (shared Redis, database 1)
|
||||
AUTHENTIK_REDIS__HOST: redis-shared
|
||||
AUTHENTIK_REDIS__PORT: 6379
|
||||
AUTHENTIK_REDIS__DB: 1
|
||||
|
||||
# Authentik secret key
|
||||
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?secret key required}
|
||||
|
||||
# Error reporting (disabled)
|
||||
AUTHENTIK_ERROR_REPORTING__ENABLED: "false"
|
||||
|
||||
# Performance tuning for home use
|
||||
WORKERS: 2
|
||||
|
||||
# Email configuration (optional - configure later if needed)
|
||||
# AUTHENTIK_EMAIL__HOST: smtp.gmail.com
|
||||
# AUTHENTIK_EMAIL__PORT: 587
|
||||
# AUTHENTIK_EMAIL__USERNAME: your-email@gmail.com
|
||||
# AUTHENTIK_EMAIL__PASSWORD: your-app-password
|
||||
# AUTHENTIK_EMAIL__USE_TLS: "true"
|
||||
# AUTHENTIK_EMAIL__FROM: authentik@schweitz.net
|
||||
|
||||
# Timezone
|
||||
TZ: Europe/Amsterdam
|
||||
volumes:
|
||||
- /home/jpmschweitzer/docker-data/authentik/media:/media
|
||||
- /home/jpmschweitzer/docker-data/authentik/custom-templates:/templates
|
||||
networks:
|
||||
- docker-dataplane
|
||||
depends_on:
|
||||
- postgres-shared
|
||||
- redis-shared
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 256M
|
||||
|
||||
authentik-worker:
|
||||
image: ghcr.io/goauthentik/server:latest
|
||||
container_name: authentik-worker
|
||||
restart: unless-stopped
|
||||
command: worker
|
||||
environment:
|
||||
# Database configuration (shared PostgreSQL)
|
||||
AUTHENTIK_POSTGRESQL__HOST: postgres-shared
|
||||
AUTHENTIK_POSTGRESQL__PORT: 5432
|
||||
AUTHENTIK_POSTGRESQL__NAME: authentik
|
||||
AUTHENTIK_POSTGRESQL__USER: authentik_user
|
||||
AUTHENTIK_POSTGRESQL__PASSWORD: ${AUTHENTIK_DB_PASSWORD}
|
||||
|
||||
# Cache configuration (shared Redis, database 1)
|
||||
AUTHENTIK_REDIS__HOST: redis-shared
|
||||
AUTHENTIK_REDIS__PORT: 6379
|
||||
AUTHENTIK_REDIS__DB: 1
|
||||
|
||||
# Authentik secret key
|
||||
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY}
|
||||
|
||||
# Error reporting (disabled)
|
||||
AUTHENTIK_ERROR_REPORTING__ENABLED: "false"
|
||||
|
||||
# Timezone
|
||||
TZ: Europe/Amsterdam
|
||||
user: root
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- /home/jpmschweitzer/docker-data/authentik/media:/media
|
||||
- /home/jpmschweitzer/docker-data/authentik/certs:/certs
|
||||
- /home/jpmschweitzer/docker-data/authentik/custom-templates:/templates
|
||||
networks:
|
||||
- docker-dataplane
|
||||
depends_on:
|
||||
- postgres-shared
|
||||
- redis-shared
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 192M
|
||||
|
||||
authentik-proxy-outpost:
|
||||
image: ghcr.io/goauthentik/proxy:latest
|
||||
container_name: authentik-proxy-outpost
|
||||
restart: unless-stopped
|
||||
network_mode: host
|
||||
environment:
|
||||
# Authentik connection
|
||||
AUTHENTIK_HOST: http://192.168.86.149:9000
|
||||
AUTHENTIK_INSECURE: "false"
|
||||
AUTHENTIK_TOKEN: ${AUTHENTIK_OUTPOST_TOKEN:?outpost token required}
|
||||
|
||||
# Logging
|
||||
AUTHENTIK_LOG_LEVEL: info
|
||||
|
||||
# Port configuration
|
||||
AUTHENTIK_LISTEN__HTTP: 0.0.0.0:9001
|
||||
AUTHENTIK_LISTEN__METRICS: 0.0.0.0:9300
|
||||
depends_on:
|
||||
- authentik-server
|
||||
labels:
|
||||
- "com.centurylinklabs.watchtower.enable=true"
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://localhost:9001/outpost.goauthentik.io/ping"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
|
||||
networks:
|
||||
docker-dataplane:
|
||||
external: true
|
||||
name: docker-dataplane
|
||||
|
||||
# Prerequisites:
|
||||
#
|
||||
# 1. Deploy shared-infrastructure stack first!
|
||||
# docker-compose -f shared-infrastructure.yml up -d
|
||||
#
|
||||
# 2. Verify shared services are running:
|
||||
# docker ps | grep -E 'postgres-shared|redis-shared'
|
||||
#
|
||||
# 3. Create Authentik data directories (if not exists):
|
||||
# mkdir -p ~/docker-data/authentik/{media,certs,custom-templates}
|
||||
#
|
||||
# 4. Create .env file with:
|
||||
# AUTHENTIK_DB_PASSWORD=<from shared-infrastructure .env>
|
||||
# AUTHENTIK_SECRET_KEY=<generate with: openssl rand -base64 60>
|
||||
#
|
||||
# 5. Deploy this stack:
|
||||
# docker-compose -f authentik-shared.yml --env-file .env.authentik-shared up -d
|
||||
#
|
||||
# After Deployment:
|
||||
#
|
||||
# 1. Wait for containers to start (may take 30-60 seconds for DB migrations)
|
||||
#
|
||||
# 2. Check logs:
|
||||
# docker logs authentik-server
|
||||
# docker logs authentik-worker
|
||||
#
|
||||
# 3. Access initial setup: http://localhost:9000/if/flow/initial-setup/
|
||||
# - Create admin account (akadmin recommended)
|
||||
# - Set strong password
|
||||
#
|
||||
# 4. Configure NPM reverse proxy:
|
||||
# - Domain: auth.schweitz.net
|
||||
# - Forward to: authentik-server:9000
|
||||
# - SSL: Let's Encrypt
|
||||
# - Websockets: Enabled
|
||||
#
|
||||
# 5. Access admin interface: https://auth.schweitz.net/if/admin/
|
||||
#
|
||||
# Connection Details:
|
||||
#
|
||||
# Database:
|
||||
# - Host: postgres-shared (from containers) / localhost (from host)
|
||||
# - Port: 5432
|
||||
# - Database: authentik
|
||||
# - User: authentik_user
|
||||
#
|
||||
# Cache:
|
||||
# - Host: redis-shared (from containers) / localhost (from host)
|
||||
# - Port: 6379
|
||||
# - Database: 1
|
||||
#
|
||||
# Resource Usage (optimized for home use):
|
||||
# - Server: 256MB RAM limit (WORKERS=2 reduces Gunicorn processes)
|
||||
# - Worker: 192MB RAM limit
|
||||
# - Proxy Outpost: ~32MB RAM
|
||||
# - Total Authentik: ~480MB max (vs ~700MB with dedicated PostgreSQL/Redis)
|
||||
# - Savings: ~400MB by using shared infrastructure!
|
||||
+1
-1
@@ -29,7 +29,7 @@ services:
|
||||
/venv/bin/uvicorn src.main:app
|
||||
--host 0.0.0.0
|
||||
--port 8083
|
||||
--workers 2
|
||||
--workers 1
|
||||
"
|
||||
|
||||
ports:
|
||||
|
||||
@@ -1,398 +0,0 @@
|
||||
# Nextcloud Database Consolidation Plan
|
||||
|
||||
**Goal**: Fresh Nextcloud installation using shared PostgreSQL + Redis infrastructure
|
||||
|
||||
**Date**: 2025-11-16
|
||||
**Status**: APPROVED - Complete wipe and fresh start
|
||||
**Approach**: No migration, no backups - complete fresh installation
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
### Existing Setup
|
||||
```yaml
|
||||
nextcloud-db (MariaDB 10.11)
|
||||
├─ Database: nextcloud
|
||||
├─ User: nextcloud
|
||||
├─ Data: /home/jpmschweitzer/docker-data/nextcloud/db
|
||||
└─ Network: docker-dataplane
|
||||
|
||||
nextcloud-redis (Redis Alpine)
|
||||
├─ Standalone instance
|
||||
├─ Data: In-memory only (no persistence configured)
|
||||
└─ Network: docker-dataplane
|
||||
|
||||
nextcloud (Nextcloud Stable)
|
||||
├─ Config: /home/jpmschweitzer/docker-data/nextcloud/config
|
||||
├─ Data: /mnt/media/nextcloud/data
|
||||
└─ Dependencies: nextcloud-db, nextcloud-redis
|
||||
```
|
||||
|
||||
### Target Setup
|
||||
```yaml
|
||||
postgres-shared (PostgreSQL 16)
|
||||
├─ New database: nextcloud
|
||||
├─ New user: nextcloud_user
|
||||
└─ Database allocation: DB 3
|
||||
|
||||
redis-shared (Redis Alpine)
|
||||
├─ Database allocation: DB 3 (Nextcloud)
|
||||
├─ Existing DB 0: General cache
|
||||
├─ Existing DB 1: Authentik
|
||||
└─ Existing DB 2: Gitea
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Challenges
|
||||
|
||||
### Critical Issue: MariaDB → PostgreSQL
|
||||
⚠️ **Nextcloud cannot simply switch database types!**
|
||||
|
||||
Nextcloud's database schema is different between MariaDB and PostgreSQL:
|
||||
- Different data types (e.g., LONGTEXT vs TEXT)
|
||||
- Different auto-increment handling
|
||||
- Different JSON field types
|
||||
- Different index structures
|
||||
|
||||
**Options:**
|
||||
|
||||
### Option A: Fresh Install + Data Migration (RECOMMENDED)
|
||||
✅ **Pros:**
|
||||
- Clean database schema
|
||||
- Opportunity to optimize
|
||||
- Lower risk of corruption
|
||||
- Can test before switching
|
||||
|
||||
❌ **Cons:**
|
||||
- Must recreate users/settings
|
||||
- Requires careful data migration
|
||||
- More complex process
|
||||
|
||||
### Option B: Database Conversion
|
||||
✅ **Pros:**
|
||||
- Preserves all settings
|
||||
- Preserves user data
|
||||
|
||||
❌ **Cons:**
|
||||
- Complex conversion process
|
||||
- High risk of data loss
|
||||
- Nextcloud doesn't officially support this
|
||||
- May leave corrupted data
|
||||
|
||||
**RECOMMENDATION: Option A (Fresh Install)**
|
||||
|
||||
---
|
||||
|
||||
## Fresh Installation Plan
|
||||
|
||||
### Phase 1: Complete Cleanup - PURGE ALL DATA
|
||||
|
||||
**Estimated Time:** 2 minutes
|
||||
|
||||
⚠️ **DESTRUCTIVE OPERATION - REQUIRES EXPLICIT APPROVAL** ⚠️
|
||||
|
||||
The following will be PERMANENTLY DELETED:
|
||||
- All Nextcloud containers (nextcloud, nextcloud-db, nextcloud-redis)
|
||||
- All Nextcloud configuration (/home/jpmschweitzer/docker-data/nextcloud)
|
||||
- All Nextcloud user files (/mnt/media/nextcloud)
|
||||
- All Nextcloud database data
|
||||
|
||||
**APPROVAL REQUIRED BEFORE EACH DELETION STEP**
|
||||
|
||||
```bash
|
||||
# Step 1: Stop and remove containers
|
||||
# APPROVAL: Stop containers? (y/n)
|
||||
docker stop nextcloud nextcloud-db nextcloud-redis 2>/dev/null || true
|
||||
docker rm nextcloud nextcloud-db nextcloud-redis 2>/dev/null || true
|
||||
|
||||
# Step 2: Delete config directory
|
||||
# APPROVAL: Delete /home/jpmschweitzer/docker-data/nextcloud? (y/n)
|
||||
sudo rm -rf /home/jpmschweitzer/docker-data/nextcloud
|
||||
|
||||
# Step 3: Delete user data directory
|
||||
# APPROVAL: Delete /mnt/media/nextcloud? (y/n)
|
||||
sudo rm -rf /mnt/media/nextcloud
|
||||
|
||||
# Step 4: Verify complete removal
|
||||
ls /home/jpmschweitzer/docker-data/ | grep nextcloud # Should be empty
|
||||
ls /mnt/media/ | grep nextcloud # Should be empty
|
||||
```
|
||||
|
||||
### Phase 2: Prepare Shared Infrastructure
|
||||
|
||||
**Estimated Time:** 5 minutes
|
||||
|
||||
```bash
|
||||
# 1. Create Nextcloud database in postgres-shared
|
||||
docker exec -i postgres-shared psql -U postgres <<'EOF'
|
||||
-- Nextcloud database
|
||||
CREATE DATABASE nextcloud;
|
||||
CREATE USER nextcloud_user WITH PASSWORD 'GENERATE_NEW_PASSWORD_HERE';
|
||||
GRANT ALL PRIVILEGES ON DATABASE nextcloud TO nextcloud_user;
|
||||
\c nextcloud
|
||||
GRANT ALL ON SCHEMA public TO nextcloud_user;
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO nextcloud_user;
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO nextcloud_user;
|
||||
EOF
|
||||
|
||||
# 2. Update redis-shared documentation (already supports DB 3)
|
||||
# No action needed - redis-shared already configured for multi-database
|
||||
```
|
||||
|
||||
### Phase 3: Create Fresh Nextcloud Stack Configuration
|
||||
|
||||
**Estimated Time:** 5 minutes
|
||||
|
||||
Create new `nextcloud-shared.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
nextcloud:
|
||||
image: nextcloud:stable
|
||||
container_name: nextcloud
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8082:80"
|
||||
volumes:
|
||||
# Fresh config directory
|
||||
- /home/jpmschweitzer/docker-data/nextcloud/config:/var/www/html/config
|
||||
# Fresh user data directory
|
||||
- /mnt/media/nextcloud/data:/var/www/html/data
|
||||
environment:
|
||||
# PostgreSQL configuration
|
||||
- POSTGRES_HOST=postgres-shared
|
||||
- POSTGRES_DB=nextcloud
|
||||
- POSTGRES_USER=nextcloud_user
|
||||
- POSTGRES_PASSWORD=${NEXTCLOUD_DB_PASSWORD}
|
||||
|
||||
# Redis configuration (Database 3)
|
||||
- REDIS_HOST=redis-shared
|
||||
- REDIS_HOST_PORT=6379
|
||||
- REDIS_DB_INDEX=3
|
||||
|
||||
# Timezone
|
||||
- TZ=Europe/Amsterdam
|
||||
depends_on:
|
||||
- postgres-shared
|
||||
- redis-shared
|
||||
networks:
|
||||
- docker-dataplane
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
|
||||
networks:
|
||||
docker-dataplane:
|
||||
external: true
|
||||
name: docker-dataplane
|
||||
```
|
||||
|
||||
### Phase 5: Deploy Fresh Nextcloud
|
||||
|
||||
**Estimated Time:** 5 minutes
|
||||
|
||||
```bash
|
||||
# 1. Create new config directory
|
||||
mkdir -p /home/jpmschweitzer/docker-data/nextcloud-shared/config
|
||||
|
||||
# 2. Create .env file with database password
|
||||
cat > /mnt/media/Projects/portainer-core/stacks/.env.nextcloud-shared <<EOF
|
||||
NEXTCLOUD_DB_PASSWORD=<GENERATED_PASSWORD_FROM_PHASE2>
|
||||
EOF
|
||||
|
||||
# 3. Deploy new stack
|
||||
cd /mnt/media/Projects/portainer-core/stacks
|
||||
docker compose -f nextcloud-shared.yml --env-file .env.nextcloud-shared up -d
|
||||
|
||||
# 4. Wait for initialization
|
||||
docker logs -f nextcloud
|
||||
```
|
||||
|
||||
### Phase 6: Initial Setup & Configuration
|
||||
|
||||
**Estimated Time:** 10 minutes
|
||||
|
||||
```bash
|
||||
# 1. Access Nextcloud web interface
|
||||
# Navigate to: http://localhost:8082 or https://cloud.schweitz.net
|
||||
|
||||
# 2. First-time setup wizard:
|
||||
# - Admin username: admin
|
||||
# - Admin password: <STRONG_PASSWORD>
|
||||
# - Data folder: /var/www/html/data (default)
|
||||
# - Database: PostgreSQL
|
||||
# - Database user: nextcloud_user
|
||||
# - Database password: <FROM_ENV_FILE>
|
||||
# - Database name: nextcloud
|
||||
# - Database host: postgres-shared
|
||||
|
||||
# 3. Wait for installation (2-3 minutes)
|
||||
|
||||
# 4. Configure trusted domains
|
||||
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=cloud.schweitz.net
|
||||
docker exec -u www-data nextcloud php occ config:system:set trusted_domains 2 --value=192.168.86.149
|
||||
|
||||
# 5. Configure Redis caching
|
||||
docker exec -u www-data nextcloud php occ config:system:set redis host --value=redis-shared
|
||||
docker exec -u www-data nextcloud php occ config:system:set redis port --value=6379
|
||||
docker exec -u www-data nextcloud php occ config:system:set redis dbindex --value=3
|
||||
docker exec -u www-data nextcloud php occ config:system:set memcache.local --value='\\OC\\Memcache\\APCu'
|
||||
docker exec -u www-data nextcloud php occ config:system:set memcache.distributed --value='\\OC\\Memcache\\Redis'
|
||||
docker exec -u www-data nextcloud php occ config:system:set memcache.locking --value='\\OC\\Memcache\\Redis'
|
||||
|
||||
# 6. Optimize database
|
||||
docker exec -u www-data nextcloud php occ db:add-missing-indices
|
||||
docker exec -u www-data nextcloud php occ db:convert-filecache-bigint
|
||||
|
||||
# 7. Configure background jobs
|
||||
docker exec -u www-data nextcloud php occ background:cron
|
||||
```
|
||||
|
||||
### Phase 7: Verify Fresh Installation
|
||||
|
||||
**Estimated Time:** 5 minutes
|
||||
|
||||
```bash
|
||||
# 1. Verify admin user can login via web interface
|
||||
# Navigate to: https://cloud.schweitz.net
|
||||
|
||||
# 2. Check PostgreSQL connection
|
||||
docker exec postgres-shared psql -U nextcloud_user -d nextcloud -c '\dt'
|
||||
|
||||
# 3. Check Redis caching
|
||||
docker exec redis-shared redis-cli -n 3 DBSIZE
|
||||
|
||||
# 4. Verify storage location
|
||||
docker exec -u www-data nextcloud php occ config:system:get datadirectory
|
||||
|
||||
# 5. Test file upload/download
|
||||
# Upload a test file via web interface
|
||||
# Download it back
|
||||
# Delete it
|
||||
```
|
||||
|
||||
### Phase 8: Final Cleanup & Documentation
|
||||
|
||||
**Estimated Time:** 2 minutes
|
||||
|
||||
```bash
|
||||
# 1. Update postgres-shared.yml documentation
|
||||
# Add Nextcloud to "Applications Using This Database" list
|
||||
|
||||
# 2. Update redis-shared.yml documentation
|
||||
# Add "DB 3: Nextcloud (file locking, distributed cache)"
|
||||
|
||||
# 3. Rename stack file
|
||||
cd /mnt/media/Projects/portainer-core/stacks
|
||||
mv nextcloud.yml nextcloud-mariadb-archived.yml
|
||||
mv nextcloud-shared.yml nextcloud.yml
|
||||
|
||||
# 4. Delete old archived stack (already purged data in Phase 1)
|
||||
# All old containers and data already removed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
⚠️ **NO ROLLBACK POSSIBLE** ⚠️
|
||||
|
||||
Since all old data is purged in Phase 1, there is no rollback option.
|
||||
|
||||
If fresh installation fails:
|
||||
1. Review error logs
|
||||
2. Fix configuration issues
|
||||
3. Retry fresh installation
|
||||
|
||||
This is acceptable since Nextcloud is not in production use.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
After fresh installation, verify:
|
||||
|
||||
- [ ] Admin login works
|
||||
- [ ] File upload works
|
||||
- [ ] File download works
|
||||
- [ ] File delete works
|
||||
- [ ] Redis caching active (`docker exec redis-shared redis-cli -n 3 DBSIZE` shows keys)
|
||||
- [ ] PostgreSQL connection stable (`docker exec postgres-shared psql -U nextcloud_user -d nextcloud -c '\dt'` shows tables)
|
||||
- [ ] Memory usage acceptable (<1GB for Nextcloud container)
|
||||
- [ ] Nextcloud accessible via https://cloud.schweitz.net
|
||||
- [ ] No errors in logs (`docker logs nextcloud`)
|
||||
|
||||
---
|
||||
|
||||
## Resource Savings
|
||||
|
||||
**Before Migration:**
|
||||
- nextcloud-db (MariaDB): ~117 MB RAM
|
||||
- nextcloud-redis: ~10 MB RAM
|
||||
- **Total:** ~127 MB RAM + 2 containers
|
||||
|
||||
**After Migration:**
|
||||
- Shared postgres-shared: Already running (minimal additional overhead for one more DB)
|
||||
- Shared redis-shared: Already running (DB 3 uses ~5-10 MB additional)
|
||||
- **Savings:** ~110-120 MB RAM + 2 fewer containers to manage
|
||||
|
||||
**Benefits:**
|
||||
- Simplified infrastructure
|
||||
- Centralized backups
|
||||
- Better resource utilization
|
||||
- Easier monitoring
|
||||
- Consistent database management
|
||||
|
||||
---
|
||||
|
||||
## Risks & Mitigation
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|------|--------|------------|
|
||||
| Data loss during migration | HIGH | Full backups before starting, test on copy first |
|
||||
| Incompatible plugins/apps | MEDIUM | Fresh install allows clean app selection |
|
||||
| User resistance to re-setup | LOW | Minimal - same interface, same files |
|
||||
| Extended downtime | MEDIUM | Plan migration during low-usage window |
|
||||
| Redis DB conflict | LOW | Using dedicated DB 3, isolated from other apps |
|
||||
|
||||
---
|
||||
|
||||
## Timeline
|
||||
|
||||
**Total estimated time:** 20-30 minutes
|
||||
|
||||
- Phase 1: Purge all data: 2 min
|
||||
- Phase 2: Prepare PostgreSQL/Redis: 5 min
|
||||
- Phase 3: Create stack config: 2 min
|
||||
- Phase 4: Create directories: 1 min
|
||||
- Phase 5: Deploy Nextcloud: 3 min
|
||||
- Phase 6: Initial setup & config: 10 min
|
||||
- Phase 7: Testing: 5 min
|
||||
- Phase 8: Documentation: 2 min
|
||||
|
||||
**Can be done anytime** - No production impact, no backups needed
|
||||
|
||||
---
|
||||
|
||||
## Approval Required
|
||||
|
||||
- [ ] Backup strategy approved
|
||||
- [ ] Fresh install approach approved
|
||||
- [ ] Downtime window approved
|
||||
- [ ] Testing checklist reviewed
|
||||
- [ ] Rollback plan understood
|
||||
- [ ] Ready to proceed
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- **COMPLETE FRESH START** - All old data deleted
|
||||
- Clean database, optimal performance from day one
|
||||
- PostgreSQL generally faster than MariaDB for Nextcloud workloads
|
||||
- Redis DB 3 dedicated to Nextcloud (isolated from other apps)
|
||||
- No migration complexity - just a clean installation
|
||||
- Ready for production use immediately after setup
|
||||
+84
-73
@@ -1,38 +1,4 @@
|
||||
version: '3.8'
|
||||
|
||||
# Nextcloud - Cloud Storage with Database and Redis
|
||||
# Backlog: Application Deployment
|
||||
# Ports: 8082
|
||||
# GPU: No
|
||||
# Storage: SSD (config/database), HDD (user data)
|
||||
|
||||
services:
|
||||
nextcloud-db:
|
||||
image: mariadb:10.11
|
||||
container_name: nextcloud-db
|
||||
command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
# Database on SSD for performance
|
||||
- /home/jpmschweitzer/docker-data/nextcloud/db:/var/lib/mysql
|
||||
environment:
|
||||
- MYSQL_ROOT_PASSWORD=xDgobrmzXOl+GgvBdXC9+z5v0OrWb29t
|
||||
- MYSQL_PASSWORD=maF91Sw9is6Zb57JVxU/gPGP8O/DsxFq
|
||||
- MYSQL_DATABASE=nextcloud
|
||||
- MYSQL_USER=nextcloud
|
||||
- TZ=Europe/Amsterdam
|
||||
networks:
|
||||
- docker-dataplane
|
||||
|
||||
nextcloud-redis:
|
||||
image: redis:alpine
|
||||
container_name: nextcloud-redis
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- TZ=Europe/Amsterdam
|
||||
networks:
|
||||
- docker-dataplane
|
||||
|
||||
nextcloud:
|
||||
image: nextcloud:stable
|
||||
container_name: nextcloud
|
||||
@@ -40,61 +6,106 @@ services:
|
||||
ports:
|
||||
- "8082:80"
|
||||
volumes:
|
||||
# App config on SSD
|
||||
- /home/jpmschweitzer/docker-data/nextcloud/config:/var/www/html
|
||||
|
||||
# User data on HDD (large files)
|
||||
# Fresh config directory
|
||||
- /home/jpmschweitzer/docker-data/nextcloud/config:/var/www/html/config
|
||||
# Fresh user data directory
|
||||
- /mnt/media/nextcloud/data:/var/www/html/data
|
||||
environment:
|
||||
- MYSQL_HOST=nextcloud-db
|
||||
- MYSQL_PASSWORD=maF91Sw9is6Zb57JVxU/gPGP8O/DsxFq
|
||||
- MYSQL_DATABASE=nextcloud
|
||||
- MYSQL_USER=nextcloud
|
||||
- REDIS_HOST=nextcloud-redis
|
||||
# PostgreSQL configuration
|
||||
- POSTGRES_HOST=postgres-shared
|
||||
- POSTGRES_DB=nextcloud
|
||||
- POSTGRES_USER=nextcloud_user
|
||||
- POSTGRES_PASSWORD=${NEXTCLOUD_DB_PASSWORD}
|
||||
|
||||
# Redis configuration (Database 3)
|
||||
- REDIS_HOST=redis-shared
|
||||
- REDIS_HOST_PORT=6379
|
||||
- REDIS_DB_INDEX=3
|
||||
|
||||
# Timezone
|
||||
- TZ=Europe/Amsterdam
|
||||
depends_on:
|
||||
- nextcloud-db
|
||||
- nextcloud-redis
|
||||
networks:
|
||||
- docker-dataplane
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
|
||||
networks:
|
||||
docker-dataplane:
|
||||
external: true
|
||||
name: docker-dataplane
|
||||
|
||||
# ⚠️ SECURITY WARNING:
|
||||
# Change MYSQL_ROOT_PASSWORD and MYSQL_PASSWORD before deploying!
|
||||
# Use strong, unique passwords.
|
||||
# Nextcloud - Personal Cloud Storage (Using Shared Infrastructure)
|
||||
# Port: 8082
|
||||
# GPU: No
|
||||
# Dependencies: postgres-shared, redis-shared
|
||||
#
|
||||
# Prerequisites:
|
||||
#
|
||||
# 1. Shared infrastructure must be running:
|
||||
# docker ps | grep -E 'postgres-shared|redis-shared'
|
||||
#
|
||||
# 2. Database and user already created in postgres-shared:
|
||||
# - Database: nextcloud
|
||||
# - User: nextcloud_user
|
||||
# - Redis DB: 3
|
||||
#
|
||||
# 3. Create .env file with:
|
||||
# NEXTCLOUD_DB_PASSWORD=<password from shared infrastructure setup>
|
||||
#
|
||||
# 4. Deploy this stack:
|
||||
# cd /mnt/media/Projects/portainer-core/stacks
|
||||
# docker compose -f nextcloud-shared.yml --env-file .env.nextcloud-shared up -d
|
||||
#
|
||||
# After Deployment:
|
||||
# 1. Access http://localhost:8082
|
||||
# 2. First-time setup:
|
||||
# - Create admin account (strong password!)
|
||||
#
|
||||
# 1. Wait for initialization (2-3 minutes)
|
||||
#
|
||||
# 2. Access web interface: http://localhost:8082 or https://cloud.schweitz.net
|
||||
#
|
||||
# 3. First-time setup wizard:
|
||||
# - Admin username: admin
|
||||
# - Admin password: <STRONG_PASSWORD>
|
||||
# - Data folder: /var/www/html/data (default)
|
||||
# - Database: MySQL/MariaDB
|
||||
# - Database user: nextcloud
|
||||
# - Database password: (the one you set above)
|
||||
# - Database: PostgreSQL
|
||||
# - Database user: nextcloud_user
|
||||
# - Database password: <FROM_ENV_FILE>
|
||||
# - Database name: nextcloud
|
||||
# - Database host: nextcloud-db
|
||||
# 3. Wait for installation (may take a few minutes)
|
||||
# - Database host: postgres-shared
|
||||
#
|
||||
# 4. Configure trusted domains:
|
||||
# docker exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=tower-of-joy
|
||||
# docker exec -u www-data nextcloud php occ config:system:set trusted_domains 2 --value=192.168.x.x
|
||||
# docker exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=cloud.schweitz.net
|
||||
# docker exec -u www-data nextcloud php occ config:system:set trusted_domains 2 --value=192.168.86.149
|
||||
#
|
||||
# Optimization (recommended):
|
||||
# docker exec -u www-data nextcloud php occ db:add-missing-indices
|
||||
# docker exec -u www-data nextcloud php occ db:convert-filecache-bigint
|
||||
# docker exec -u www-data nextcloud php occ background:cron
|
||||
# 5. Configure Redis caching:
|
||||
# docker exec -u www-data nextcloud php occ config:system:set redis host --value=redis-shared
|
||||
# docker exec -u www-data nextcloud php occ config:system:set redis port --value=6379
|
||||
# docker exec -u www-data nextcloud php occ config:system:set redis dbindex --value=3
|
||||
# docker exec -u www-data nextcloud php occ config:system:set memcache.local --value='\\OC\\Memcache\\APCu'
|
||||
# docker exec -u www-data nextcloud php occ config:system:set memcache.distributed --value='\\OC\\Memcache\\Redis'
|
||||
# docker exec -u www-data nextcloud php occ config:system:set memcache.locking --value='\\OC\\Memcache\\Redis'
|
||||
#
|
||||
# Add cron job for background tasks:
|
||||
# echo "*/5 * * * * docker exec -u www-data nextcloud php cron.php" | sudo tee -a /etc/crontab
|
||||
# 6. Optimize database:
|
||||
# docker exec -u www-data nextcloud php occ db:add-missing-indices
|
||||
# docker exec -u www-data nextcloud php occ db:convert-filecache-bigint
|
||||
#
|
||||
# Features:
|
||||
# - File sync and share
|
||||
# - Calendar and contacts
|
||||
# - Collaborative editing
|
||||
# - Photo gallery
|
||||
# - Mobile apps (iOS/Android)
|
||||
# - Desktop sync client
|
||||
# - External storage support
|
||||
# 7. Configure background jobs:
|
||||
# docker exec -u www-data nextcloud php occ background:cron
|
||||
#
|
||||
# Connection Details:
|
||||
#
|
||||
# Database:
|
||||
# - Host: postgres-shared (from containers) / localhost (from host)
|
||||
# - Port: 5432
|
||||
# - Database: nextcloud
|
||||
# - User: nextcloud_user
|
||||
#
|
||||
# Cache:
|
||||
# - Host: redis-shared (from containers) / localhost (from host)
|
||||
# - Port: 6379
|
||||
# - Database: 3
|
||||
#
|
||||
# Resource Usage:
|
||||
# - Nextcloud: 1GB RAM limit
|
||||
# - Savings: ~110-120 MB RAM + 2 fewer containers (MariaDB + Redis removed)
|
||||
|
||||
@@ -132,4 +132,5 @@ networks:
|
||||
# Applications Using This Database:
|
||||
# - Authentik (identity provider)
|
||||
# - Gitea (git hosting) - migrated from dedicated instance
|
||||
# - Nextcloud (personal cloud storage) - migrated from MariaDB
|
||||
# - Future applications as needed
|
||||
|
||||
@@ -66,7 +66,7 @@ networks:
|
||||
# DB 0: General cache (default, shared lightweight caching)
|
||||
# DB 1: Authentik (sessions, cache, message queue)
|
||||
# DB 2: Gitea (cache, sessions)
|
||||
# DB 3: Open WebUI (cache, if needed)
|
||||
# DB 3: Nextcloud (file locking, distributed cache, sessions)
|
||||
# DB 4-15: Reserved for future applications
|
||||
#
|
||||
# Connection Examples:
|
||||
@@ -144,6 +144,7 @@ networks:
|
||||
# Applications Using This Cache:
|
||||
# - Authentik (sessions, policies, background tasks)
|
||||
# - Gitea (sessions, cache, queues) - if migrated
|
||||
# - Nextcloud (file locking, distributed cache, sessions)
|
||||
# - Future applications as needed
|
||||
#
|
||||
# Performance Tips:
|
||||
|
||||
Reference in New Issue
Block a user