security rework and memory optimilizations.
This commit is contained in:
@@ -74,7 +74,7 @@ src/
|
||||
- [x] GET /infrastructure/health - Check connectivity ✅ TESTED
|
||||
- [x] GET /infrastructure/services - List all services ✅ TESTED
|
||||
- [x] GET /infrastructure/services/{name} - Get service details ✅ TESTED
|
||||
- [x] GET /infrastructure/ports - List allocated ports (skeleton)
|
||||
- [x] GET /infrastructure/ports - List allocated ports ✅ IMPLEMENTED & TESTED
|
||||
- [x] GET /infrastructure/domains - List configured domains ✅ TESTED
|
||||
- [x] Integrate with main.py routing ✅ TESTED
|
||||
- [x] Fix Pydantic validation issues (status field type conversion)
|
||||
@@ -84,17 +84,27 @@ src/
|
||||
- [x] POST /infrastructure/proxy - Create NPM proxy host with optional SSL ✅ IMPLEMENTED
|
||||
- [ ] POST /infrastructure/monitoring/add - Auto-add Kuma monitor (DEFERRED - Socket.IO complexity)
|
||||
|
||||
### Phase 4: Refactor Existing Controllers 📋 PENDING
|
||||
- [ ] Move AI endpoints to ai_controller.py
|
||||
- [ ] Move webscraper to tools_controller.py
|
||||
- [ ] Move health check to health_controller.py
|
||||
- [ ] Update main.py imports and routing
|
||||
### Phase 4: Refactor Existing Controllers ✅ COMPLETE
|
||||
- [x] Move AI endpoints to ai_controller.py ✅ COMPLETE
|
||||
- [x] Move webscraper to tools_controller.py ✅ COMPLETE
|
||||
- [x] Move health check to health_controller.py ✅ COMPLETE
|
||||
- [x] Update main.py imports and routing ✅ COMPLETE
|
||||
- [x] Test all refactored endpoints ✅ ALL WORKING
|
||||
|
||||
### Phase 5: Testing & Documentation 📋 PENDING
|
||||
- [ ] Test all refactored endpoints
|
||||
- [ ] Update API documentation
|
||||
- [ ] Create CLI wrapper scripts
|
||||
- [ ] Remove old shell scripts
|
||||
### Phase 5: Testing & Documentation ✅ COMPLETE
|
||||
- [x] Test all refactored endpoints ✅ ALL WORKING
|
||||
- [x] Update API documentation (OpenAPI spec auto-generated and validated)
|
||||
- [~] Create CLI wrapper scripts (SKIPPED - LLMs consume OpenAPI spec directly)
|
||||
- [~] Remove old shell scripts (DEFERRED - not blocking)
|
||||
|
||||
### Phase 6: Infrastructure Improvements 📋 FUTURE
|
||||
- [ ] Consolidate Docker network topology into single `docker-dataplane` network
|
||||
- Currently each stack has its own network (172.22.0.x, 172.25.0.x, 172.20.0.x, etc.)
|
||||
- Error-prone and unnecessarily complex
|
||||
- Single shared network simplifies inter-service communication
|
||||
- Reduces subnet conflicts and improves service discovery
|
||||
- Update all compose files to use: `networks: [docker-dataplane]`
|
||||
- Create network once: `docker network create docker-dataplane`
|
||||
|
||||
---
|
||||
|
||||
@@ -132,7 +142,7 @@ src/
|
||||
- ✅ GET /infrastructure/services - Returns 8 active stacks
|
||||
- ✅ GET /infrastructure/services/{name} - Service lookup working
|
||||
- ✅ GET /infrastructure/domains - Returns proxy hosts with SSL status
|
||||
- ⚠️ NPM health check shows `false` (returns 302 redirect instead of 200)
|
||||
- ✅ NPM health check fixed (now accepts 2xx/3xx status codes and follows redirects)
|
||||
|
||||
### Session 3: Write Endpoints (2025-11-14 Evening)
|
||||
**Completed:**
|
||||
@@ -151,10 +161,79 @@ src/
|
||||
- ✅ POST /infrastructure/proxy - Implemented (not tested to avoid production interference)
|
||||
|
||||
**Next Steps:**
|
||||
1. Refactor existing AI/tools/health endpoints into separate controllers (Phase 4)
|
||||
2. Fix NPM health check to handle redirects
|
||||
3. Implement port allocation detection logic
|
||||
4. Create CLI wrappers for common operations
|
||||
1. ~~Refactor existing AI/tools/health endpoints into separate controllers (Phase 4)~~ ✅ DONE (2025-11-14)
|
||||
2. ~~Fix NPM health check to handle redirects~~ ✅ DONE (2025-11-14)
|
||||
3. ~~Implement port allocation detection logic~~ ✅ DONE (2025-11-14)
|
||||
4. ~~Create CLI wrappers for common operations~~ ⊘ SKIPPED (LLMs use OpenAPI)
|
||||
5. (OPTIONAL) Consolidate Docker networks into `docker-dataplane` (Phase 6)
|
||||
|
||||
### Session 4: NPM Health Check & Port Detection (2025-11-14 Afternoon)
|
||||
**Completed:**
|
||||
- Fixed NPM health check to handle redirects properly
|
||||
- Updated `npm_client.py` to accept 2xx/3xx status codes as healthy
|
||||
- Enabled explicit redirect following in httpx client
|
||||
- Verified fix with live NPM instance (now shows 9 proxy hosts)
|
||||
- Implemented comprehensive port detection in `GET /infrastructure/ports` endpoint
|
||||
- Added `get_containers()` and `get_container()` methods to PortainerClient
|
||||
- Enhanced PortInfo model with internal/external hostname and IP fields
|
||||
- Implemented domain mapping from NPM proxy hosts to services
|
||||
- Added deduplication logic for port entries (Docker returns duplicates per bind address)
|
||||
|
||||
**Port Detection Features:**
|
||||
- Scans all running containers across all Portainer endpoints
|
||||
- Extracts internal port, host port, and protocol for each container
|
||||
- Maps container names to service names via Docker Compose labels
|
||||
- Retrieves internal Docker hostnames and IP addresses per network
|
||||
- Cross-references NPM proxy hosts to identify external domains
|
||||
- Returns 22 unique port mappings with complete metadata
|
||||
|
||||
**Technical Details:**
|
||||
|
||||
*NPM Health Check:*
|
||||
- Issue: NPM's `/api` endpoint returns 302 redirect, old code only accepted 200
|
||||
- Solution: Accept `200 <= status_code < 400` as healthy response
|
||||
- Result: NPM health check now returns `true` and proxy hosts are enumerated correctly
|
||||
|
||||
*Port Detection:*
|
||||
- Queries Portainer Docker API for container list and port mappings
|
||||
- Extracts NetworkSettings for internal IPs and hostnames
|
||||
- Builds port→domain map from NPM proxy hosts configuration
|
||||
- Matches services to external domains using multiple strategies:
|
||||
- By container name + port
|
||||
- By internal IP + port
|
||||
- By host address + host port (localhost, 127.0.0.1, server IP)
|
||||
- Deduplicates based on (port, container_name, protocol) tuple
|
||||
- Example output: Nextcloud port 80 → internal IP 172.25.0.3 → external domain cloud.schweitz.net
|
||||
|
||||
### Session 5: Controller Architecture Refactoring (2025-11-14 Evening)
|
||||
**Completed:**
|
||||
- Created `ai_controller.py` consolidating chat, models, and conversations endpoints
|
||||
- Created `tools_controller.py` for web scraper functionality
|
||||
- Created `health_controller.py` for service health and info endpoints
|
||||
- Updated `main.py` to use new controller-based architecture
|
||||
- Removed legacy router imports and inline endpoint definitions
|
||||
- Tested all refactored endpoints - 16 endpoints working correctly
|
||||
|
||||
**Architecture Changes:**
|
||||
- All endpoints now follow consistent controller pattern inheriting from `BaseController`
|
||||
- Controllers use `create_router()` method for FastAPI router configuration
|
||||
- Clean separation of concerns:
|
||||
- `ai_controller.py` - AI orchestration and conversation memory (7 endpoints)
|
||||
- `tools_controller.py` - Utility tools like web scraper (1 endpoint)
|
||||
- `health_controller.py` - Service status and info (2 endpoints)
|
||||
- `infrastructure_controller.py` - Infrastructure management (6 endpoints)
|
||||
- Simplified `main.py` from 220 lines to 152 lines
|
||||
- Backward compatible - all existing endpoints work identically
|
||||
|
||||
**Test Results:**
|
||||
- ✅ GET / - Service information
|
||||
- ✅ GET /health - Health check with Ollama status
|
||||
- ✅ GET /v1/models - Model listing
|
||||
- ✅ POST /v1/chat/completions - Chat completions
|
||||
- ✅ GET /v1/conversations/{id} - Conversation history
|
||||
- ✅ GET /infrastructure/health - Infrastructure health
|
||||
- ✅ POST /web-scraper/scrape - Web scraping
|
||||
- ✅ OpenAPI spec generation - 16 endpoints documented
|
||||
|
||||
## API Authentication Strategy
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Create Authentik Outpost for NPM Forward Authentication
|
||||
|
||||
Creates a dedicated outpost and retrieves its token for deployment.
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
sys.path.insert(0, '/home/jpmschweitzer/Projects/portainer-core/services/core-api')
|
||||
|
||||
from src.clients.authentik_client import get_authentik_client
|
||||
|
||||
|
||||
async def create_npm_outpost():
|
||||
"""Create outpost for NPM forward auth and get token"""
|
||||
authentik = get_authentik_client()
|
||||
|
||||
try:
|
||||
print("=" * 60)
|
||||
print("Creating NPM Forward Auth Outpost")
|
||||
print("=" * 60)
|
||||
|
||||
# Check if provider exists
|
||||
print("\n1. Checking for proxy provider...")
|
||||
provider = await authentik.get_provider_by_name_proxy("npm-forward-auth-provider")
|
||||
|
||||
if not provider:
|
||||
print("❌ Proxy provider not found. Run setup_authentik_forward_auth.py first.")
|
||||
return False
|
||||
|
||||
provider_id = provider["pk"]
|
||||
print(f"✓ Found provider (ID: {provider_id})")
|
||||
|
||||
# Check if outpost already exists
|
||||
print("\n2. Checking for existing NPM outpost...")
|
||||
try:
|
||||
existing = await authentik.get_outpost_by_name("npm-forward-auth-outpost")
|
||||
if existing:
|
||||
print(f"✓ Outpost already exists (ID: {existing['pk']})")
|
||||
outpost_id = existing["pk"]
|
||||
|
||||
# Update it to ensure provider is assigned
|
||||
print("\n3. Updating outpost configuration...")
|
||||
await authentik.update_outpost(
|
||||
outpost_id=outpost_id,
|
||||
providers=[provider_id]
|
||||
)
|
||||
print("✓ Outpost updated with provider")
|
||||
except Exception:
|
||||
# Outpost doesn't exist, create it
|
||||
print("⊘ Outpost doesn't exist, creating new one...")
|
||||
print("\n3. Creating NPM outpost...")
|
||||
|
||||
outpost = await authentik.create_outpost(
|
||||
name="npm-forward-auth-outpost",
|
||||
type="proxy",
|
||||
providers=[provider_id],
|
||||
config={
|
||||
"authentik_host": "http://192.168.86.149:9000",
|
||||
"authentik_host_insecure": False,
|
||||
"log_level": "info",
|
||||
"docker_labels": None,
|
||||
"docker_network": None,
|
||||
"docker_map_ports": True,
|
||||
"container_image": None,
|
||||
"kubernetes_replicas": 1,
|
||||
"kubernetes_namespace": "default"
|
||||
}
|
||||
)
|
||||
outpost_id = outpost["pk"]
|
||||
print(f"✓ Created outpost (ID: {outpost_id})")
|
||||
|
||||
# Try to get the service connection token
|
||||
print("\n4. Retrieving outpost token...")
|
||||
print("\nNote: Authentik creates service accounts for outposts automatically.")
|
||||
print("The token format is: ak-outpost-<outpost_uuid>-api")
|
||||
|
||||
# Get outpost details to find its service account
|
||||
outpost_details = await authentik._request("GET", f"outposts/instances/{outpost_id}/")
|
||||
|
||||
print(f"\nOutpost Details:")
|
||||
print(f" Name: {outpost_details.get('name')}")
|
||||
print(f" ID: {outpost_details.get('pk')}")
|
||||
print(f" Type: {outpost_details.get('type')}")
|
||||
print(f" Providers: {outpost_details.get('providers')}")
|
||||
|
||||
# The outpost service connection details
|
||||
if 'service_connection' in outpost_details:
|
||||
print(f" Service Connection: {outpost_details.get('service_connection')}")
|
||||
|
||||
# List tokens to find the one for this outpost
|
||||
print("\n5. Looking for outpost service token...")
|
||||
tokens = await authentik.list_tokens()
|
||||
|
||||
outpost_uuid = outpost_details.get('pk')
|
||||
token_identifier = f"ak-outpost-{outpost_uuid}-api"
|
||||
|
||||
matching_token = None
|
||||
for token in tokens:
|
||||
if token.get('identifier') == token_identifier:
|
||||
matching_token = token
|
||||
break
|
||||
|
||||
if matching_token:
|
||||
print(f"✓ Found token: {matching_token.get('identifier')}")
|
||||
print(f"\n{'=' * 60}")
|
||||
print("IMPORTANT: Token Key Required")
|
||||
print("=" * 60)
|
||||
print("\nAuthentik does not expose token keys via API after creation.")
|
||||
print("\nTo get the token key:")
|
||||
print("1. Go to: https://auth.schweitz.net/if/admin/#/core/tokens")
|
||||
print(f"2. Find token: {token_identifier}")
|
||||
print("3. Click 'View Token Key' or regenerate the token")
|
||||
print("4. Copy the token key")
|
||||
print("\nAlternatively, you can:")
|
||||
print("1. Delete the existing outpost via UI")
|
||||
print("2. Create a new outpost via UI")
|
||||
print("3. Copy the token key when it's displayed")
|
||||
print("\n" + "=" * 60)
|
||||
else:
|
||||
print("\n⚠ No automatic token found.")
|
||||
print("You may need to manually create a token for the outpost.")
|
||||
print("\nManual token creation:")
|
||||
print("1. Go to: https://auth.schweitz.net/if/admin/#/core/tokens")
|
||||
print("2. Click 'Create'")
|
||||
print(f"3. Identifier: npm-outpost-token")
|
||||
print("4. User: Select the outpost service account")
|
||||
print("5. Intent: API")
|
||||
print("6. Copy the token key when displayed")
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("Next Steps")
|
||||
print("=" * 60)
|
||||
print("\n1. Obtain the outpost token key (see above)")
|
||||
print("2. Create/update .env.authentik-shared file:")
|
||||
print(" AUTHENTIK_OUTPOST_TOKEN=<your_token_key>")
|
||||
print("3. Deploy the stack:")
|
||||
print(" docker-compose -f stacks/authentik-shared.yml --env-file .env.authentik-shared up -d")
|
||||
print("4. Update NPM hosts to point to port 9001")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(create_npm_outpost())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -6,6 +6,7 @@ pydantic-settings==2.7.0
|
||||
|
||||
# HTTP client
|
||||
httpx==0.28.1
|
||||
python-socketio[asyncio_client]==5.11.4
|
||||
|
||||
# Web scraping
|
||||
beautifulsoup4==4.12.3
|
||||
@@ -17,6 +18,11 @@ python-multipart==0.0.12
|
||||
python-dotenv==1.0.1
|
||||
python-json-logger==2.0.7
|
||||
|
||||
# Authentication & Security
|
||||
PyJWT[crypto]==2.9.0
|
||||
python-jose[cryptography]==3.3.0
|
||||
cryptography==43.0.3
|
||||
|
||||
# Memory & Embeddings
|
||||
qdrant-client==1.11.3
|
||||
sentence-transformers==3.3.1
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Setup Authentik Forward Authentication for NPM
|
||||
|
||||
This script creates a proxy provider and outpost for forward authentication
|
||||
across all NPM-managed domains.
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
sys.path.insert(0, '/home/jpmschweitzer/Projects/portainer-core/services/core-api')
|
||||
|
||||
from src.clients.authentik_client import get_authentik_client
|
||||
|
||||
|
||||
async def setup_forward_auth():
|
||||
"""Create proxy provider, application, and outpost for forward auth"""
|
||||
client = get_authentik_client()
|
||||
|
||||
try:
|
||||
# Check if Authentik is accessible
|
||||
print("Checking Authentik connectivity...")
|
||||
if not await client.health_check():
|
||||
print("❌ Authentik is not accessible")
|
||||
return False
|
||||
print("✓ Authentik is accessible")
|
||||
|
||||
# Check if proxy provider already exists
|
||||
print("\nChecking for existing proxy provider...")
|
||||
existing_provider = await client.get_provider_by_name_proxy("npm-forward-auth-provider")
|
||||
|
||||
if existing_provider:
|
||||
print(f"✓ Proxy provider already exists (ID: {existing_provider.get('pk')})")
|
||||
provider = existing_provider
|
||||
else:
|
||||
# Create Proxy provider for forward auth
|
||||
print("\nCreating Proxy provider for forward authentication...")
|
||||
provider = await client.create_proxy_provider(
|
||||
name="npm-forward-auth-provider",
|
||||
external_host="https://auth.schweitz.net",
|
||||
mode="forward_single",
|
||||
token_validity=480 # 8 hours
|
||||
)
|
||||
print(f"✓ Created proxy provider (ID: {provider.get('pk')})")
|
||||
|
||||
# Display provider details
|
||||
print("\n" + "="*60)
|
||||
print("Proxy Provider Details:")
|
||||
print("="*60)
|
||||
print(f"Provider ID: {provider.get('pk')}")
|
||||
print(f"Mode: {provider.get('mode')}")
|
||||
print(f"External Host: {provider.get('external_host')}")
|
||||
print(f"Token Validity: {provider.get('access_token_validity')}")
|
||||
print(f"Session Duration: {provider.get('session_duration')}")
|
||||
print("="*60)
|
||||
|
||||
# Check if application already exists
|
||||
print("\nChecking for existing application...")
|
||||
existing_app = await client.get_application_by_slug("npm-forward-auth")
|
||||
|
||||
if existing_app:
|
||||
print(f"✓ Application already exists (slug: {existing_app.get('slug')})")
|
||||
app = existing_app
|
||||
else:
|
||||
# Create application
|
||||
print("\nCreating application...")
|
||||
app = await client.create_application(
|
||||
name="NPM Forward Auth",
|
||||
slug="npm-forward-auth",
|
||||
provider_pk=provider.get("pk"),
|
||||
launch_url="https://auth.schweitz.net"
|
||||
)
|
||||
print(f"✓ Created application (slug: {app.get('slug')})")
|
||||
|
||||
# Check if outpost already exists
|
||||
print("\nChecking for existing outpost...")
|
||||
existing_outpost = await client.get_outpost_by_name("npm-forward-auth-outpost")
|
||||
|
||||
if existing_outpost:
|
||||
print(f"✓ Outpost already exists (ID: {existing_outpost.get('pk')})")
|
||||
outpost = existing_outpost
|
||||
else:
|
||||
# Create outpost
|
||||
print("\nCreating outpost...")
|
||||
outpost = await client.create_outpost(
|
||||
name="npm-forward-auth-outpost",
|
||||
type="proxy",
|
||||
providers=[provider.get("pk")],
|
||||
config={
|
||||
"authentik_host": "https://auth.schweitz.net",
|
||||
"authentik_host_insecure": False,
|
||||
"log_level": "info"
|
||||
}
|
||||
)
|
||||
print(f"✓ Created outpost (ID: {outpost.get('pk')})")
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("Setup Complete!")
|
||||
print("="*60)
|
||||
print("\nNext steps:")
|
||||
print("1. Deploy the Authentik outpost container")
|
||||
print("2. Configure NPM proxy hosts with forward auth")
|
||||
print("3. Test the SSO flow")
|
||||
print("\nOutpost deployment command will be generated...")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n❌ Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
finally:
|
||||
await client.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(setup_forward_auth())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -0,0 +1,5 @@
|
||||
"""
|
||||
Authentication module for core-api
|
||||
|
||||
Provides OIDC/OAuth2 authentication via Authentik
|
||||
"""
|
||||
@@ -0,0 +1,235 @@
|
||||
"""
|
||||
OIDC Authentication Module
|
||||
|
||||
Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP.
|
||||
Implements bearer token authentication with JWT verification.
|
||||
"""
|
||||
from fastapi import Depends, HTTPException, Security
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
from jose import jwt, JWTError
|
||||
import httpx
|
||||
from functools import lru_cache
|
||||
from typing import Dict, Optional
|
||||
from src.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
security = HTTPBearer(auto_error=False)
|
||||
|
||||
|
||||
class OIDCConfig:
|
||||
"""OIDC configuration from environment"""
|
||||
|
||||
def __init__(self):
|
||||
# These will be set from environment variables in config.py
|
||||
self.enabled = False
|
||||
self.issuer = ""
|
||||
self.audience = ""
|
||||
self.jwks_uri = ""
|
||||
|
||||
def configure(self, enabled: bool, issuer: str, audience: str):
|
||||
"""Configure OIDC settings"""
|
||||
self.enabled = enabled
|
||||
self.issuer = issuer
|
||||
self.audience = audience
|
||||
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
|
||||
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
|
||||
|
||||
|
||||
# Global OIDC config instance
|
||||
oidc_config = OIDCConfig()
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def get_jwks() -> Dict:
|
||||
"""
|
||||
Fetch JSON Web Key Set (JWKS) from Authentik
|
||||
|
||||
Cached to avoid repeated requests. Cache is cleared on server restart.
|
||||
|
||||
Returns:
|
||||
JWKS dictionary containing public keys for token verification
|
||||
|
||||
Raises:
|
||||
HTTPException: If JWKS fetch fails
|
||||
"""
|
||||
if not oidc_config.enabled:
|
||||
return {}
|
||||
|
||||
try:
|
||||
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}")
|
||||
response = httpx.get(oidc_config.jwks_uri, timeout=10.0)
|
||||
response.raise_for_status()
|
||||
jwks = response.json()
|
||||
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)")
|
||||
return jwks
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch JWKS: {e}")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Authentication service unavailable"
|
||||
)
|
||||
|
||||
|
||||
async def get_current_user(
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
|
||||
) -> Optional[Dict]:
|
||||
"""
|
||||
Validate OIDC token from Authorization: Bearer header
|
||||
|
||||
Extracts and validates JWT token from request header. Verifies:
|
||||
- Token signature using JWKS
|
||||
- Token expiration
|
||||
- Issuer matches Authentik
|
||||
- Audience matches core-api
|
||||
|
||||
Args:
|
||||
credentials: HTTP Bearer token from Authorization header
|
||||
|
||||
Returns:
|
||||
User claims dictionary containing email, name, groups, etc.
|
||||
Returns None if OIDC is disabled (allows unauthenticated access)
|
||||
|
||||
Raises:
|
||||
HTTPException 401: If token is invalid, expired, or missing when OIDC enabled
|
||||
"""
|
||||
# If OIDC is disabled, allow all requests (no authentication)
|
||||
if not oidc_config.enabled:
|
||||
logger.debug("OIDC disabled - allowing unauthenticated access")
|
||||
return None
|
||||
|
||||
# OIDC enabled - token required
|
||||
if not credentials:
|
||||
logger.warning("Authentication required but no token provided")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
|
||||
token = credentials.credentials
|
||||
|
||||
try:
|
||||
# Decode token header to get key ID
|
||||
unverified_header = jwt.get_unverified_header(token)
|
||||
kid = unverified_header.get("kid")
|
||||
|
||||
if not kid:
|
||||
raise HTTPException(status_code=401, detail="Invalid token format")
|
||||
|
||||
# Find matching key in JWKS
|
||||
jwks = get_jwks()
|
||||
rsa_key = None
|
||||
|
||||
for key in jwks.get("keys", []):
|
||||
if key.get("kid") == kid:
|
||||
rsa_key = key
|
||||
break
|
||||
|
||||
if not rsa_key:
|
||||
logger.warning(f"No matching key found for kid: {kid}")
|
||||
raise HTTPException(status_code=401, detail="Invalid token key")
|
||||
|
||||
# Verify and decode token
|
||||
payload = jwt.decode(
|
||||
token,
|
||||
rsa_key,
|
||||
algorithms=["RS256"],
|
||||
audience=oidc_config.audience,
|
||||
issuer=oidc_config.issuer,
|
||||
)
|
||||
|
||||
user_email = payload.get("email", "unknown")
|
||||
logger.info(f"Authenticated user: {user_email}")
|
||||
|
||||
return payload
|
||||
|
||||
except jwt.ExpiredSignatureError:
|
||||
logger.warning("Token expired")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Token expired",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
except jwt.JWTClaimsError as e:
|
||||
logger.warning(f"Invalid token claims: {e}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid token claims",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
except JWTError as e:
|
||||
logger.error(f"JWT validation error: {e}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid authentication token",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected authentication error: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Authentication error",
|
||||
)
|
||||
|
||||
|
||||
async def get_admin_user(
|
||||
user: Optional[Dict] = Depends(get_current_user)
|
||||
) -> Dict:
|
||||
"""
|
||||
Require admin group membership
|
||||
|
||||
Use this dependency for endpoints that require admin access.
|
||||
Checks if user is member of 'admin' group in Authentik.
|
||||
|
||||
Args:
|
||||
user: User claims from get_current_user
|
||||
|
||||
Returns:
|
||||
User claims dictionary if user is admin
|
||||
|
||||
Raises:
|
||||
HTTPException 403: If user is not in admin group
|
||||
HTTPException 401: If OIDC enabled but user not authenticated
|
||||
"""
|
||||
# If OIDC disabled, allow all (backward compatibility)
|
||||
if not oidc_config.enabled or user is None:
|
||||
logger.debug("OIDC disabled - allowing admin access")
|
||||
return {"email": "unauthenticated", "groups": ["admin"]}
|
||||
|
||||
# Check admin group membership
|
||||
groups = user.get("groups", [])
|
||||
|
||||
if "admin" not in groups and "authentik Admins" not in groups:
|
||||
user_email = user.get("email", "unknown")
|
||||
logger.warning(f"User {user_email} attempted admin access (groups: {groups})")
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="Admin access required"
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
|
||||
async def get_optional_user(
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
|
||||
) -> Optional[Dict]:
|
||||
"""
|
||||
Optional authentication - allows both authenticated and unauthenticated access
|
||||
|
||||
Use for endpoints that should be accessible to everyone but can provide
|
||||
enhanced functionality for authenticated users.
|
||||
|
||||
Args:
|
||||
credentials: HTTP Bearer token from Authorization header
|
||||
|
||||
Returns:
|
||||
User claims if valid token provided, None otherwise
|
||||
"""
|
||||
if not credentials or not oidc_config.enabled:
|
||||
return None
|
||||
|
||||
try:
|
||||
return await get_current_user(credentials)
|
||||
except HTTPException:
|
||||
# Invalid token - return None instead of raising
|
||||
return None
|
||||
@@ -0,0 +1,302 @@
|
||||
"""
|
||||
Authentik API Client
|
||||
|
||||
Provides methods for interacting with Authentik Identity Provider API.
|
||||
Used for managing applications, providers, and authentication flows.
|
||||
"""
|
||||
import httpx
|
||||
from typing import Dict, List, Any, Optional
|
||||
from functools import lru_cache
|
||||
from src.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class AuthentikClient:
|
||||
"""Client for Authentik API operations"""
|
||||
|
||||
def __init__(self, base_url: str, api_token: str):
|
||||
"""
|
||||
Initialize Authentik client
|
||||
|
||||
Args:
|
||||
base_url: Authentik base URL (e.g., http://authentik-server:9000)
|
||||
api_token: API token for authentication
|
||||
"""
|
||||
self.base_url = base_url.rstrip('/')
|
||||
self.api_token = api_token
|
||||
self.client = httpx.AsyncClient(timeout=30.0)
|
||||
|
||||
async def _request(self, method: str, endpoint: str, **kwargs) -> Dict:
|
||||
"""Make authenticated API request using token auth"""
|
||||
headers = kwargs.pop("headers", {})
|
||||
headers["Authorization"] = f"Bearer {self.api_token}"
|
||||
|
||||
response = await self.client.request(
|
||||
method,
|
||||
f"{self.base_url}/api/v3/{endpoint.lstrip('/')}",
|
||||
headers=headers,
|
||||
**kwargs
|
||||
)
|
||||
|
||||
if not response.is_success:
|
||||
logger.error(f"API request failed: {response.status_code}")
|
||||
logger.error(f"Response body: {response.text}")
|
||||
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def health_check(self) -> bool:
|
||||
"""Check if Authentik is accessible"""
|
||||
try:
|
||||
response = await self.client.get(f"{self.base_url}/-/health/live/")
|
||||
return response.status_code == 200
|
||||
except Exception as e:
|
||||
logger.error(f"Authentik health check failed: {e}")
|
||||
return False
|
||||
|
||||
async def create_oauth2_provider(
|
||||
self,
|
||||
name: str,
|
||||
client_id: str,
|
||||
redirect_uris: List[str],
|
||||
authorization_flow_slug: str = "default-provider-authorization-implicit-consent",
|
||||
signing_key: Optional[str] = None
|
||||
) -> Dict:
|
||||
"""
|
||||
Create an OAuth2/OIDC provider
|
||||
|
||||
Args:
|
||||
name: Provider name
|
||||
client_id: OAuth2 client ID
|
||||
redirect_uris: List of allowed redirect URIs
|
||||
authorization_flow_slug: Authorization flow slug (will be resolved to UUID)
|
||||
signing_key: Signing key UUID (defaults to auto-selected)
|
||||
|
||||
Returns:
|
||||
Created provider data including client_secret
|
||||
"""
|
||||
# Get authorization flow UUID from slug
|
||||
flows = await self.list_flows()
|
||||
auth_flow_uuid = None
|
||||
invalidation_flow_uuid = None
|
||||
|
||||
for flow in flows:
|
||||
if flow.get("slug") == authorization_flow_slug:
|
||||
auth_flow_uuid = flow.get("pk")
|
||||
if flow.get("slug") == "default-provider-invalidation-flow":
|
||||
invalidation_flow_uuid = flow.get("pk")
|
||||
|
||||
if not auth_flow_uuid:
|
||||
raise ValueError(f"Authorization flow '{authorization_flow_slug}' not found")
|
||||
if not invalidation_flow_uuid:
|
||||
raise ValueError("Invalidation flow not found")
|
||||
|
||||
# Get signing key if not provided
|
||||
if not signing_key:
|
||||
keys = await self._request("GET", "crypto/certificatekeypairs/")
|
||||
# Find the self-signed cert
|
||||
for key in keys.get("results", []):
|
||||
if "authentik" in key.get("name", "").lower():
|
||||
signing_key = key.get("pk")
|
||||
break
|
||||
|
||||
if not signing_key and keys.get("results"):
|
||||
signing_key = keys["results"][0]["pk"]
|
||||
|
||||
# Format redirect URIs as objects with matching_mode
|
||||
formatted_redirect_uris = [
|
||||
{"url": uri, "matching_mode": "strict"}
|
||||
for uri in redirect_uris
|
||||
]
|
||||
|
||||
provider_data = {
|
||||
"name": name,
|
||||
"authorization_flow": auth_flow_uuid,
|
||||
"invalidation_flow": invalidation_flow_uuid,
|
||||
"client_type": "confidential",
|
||||
"client_id": client_id,
|
||||
"redirect_uris": formatted_redirect_uris,
|
||||
"signing_key": signing_key,
|
||||
"sub_mode": "hashed_user_id",
|
||||
"include_claims_in_id_token": True,
|
||||
"issuer_mode": "per_provider",
|
||||
"access_token_validity": "minutes=60",
|
||||
"refresh_token_validity": "days=30",
|
||||
"property_mappings": [] # Will use default mappings
|
||||
}
|
||||
|
||||
result = await self._request("POST", "providers/oauth2/", json=provider_data)
|
||||
logger.info(f"Created OAuth2 provider: {name} (ID: {result.get('pk')})")
|
||||
return result
|
||||
|
||||
async def create_application(
|
||||
self,
|
||||
name: str,
|
||||
slug: str,
|
||||
provider_pk: int,
|
||||
launch_url: Optional[str] = None,
|
||||
icon_url: Optional[str] = None
|
||||
) -> Dict:
|
||||
"""
|
||||
Create an application
|
||||
|
||||
Args:
|
||||
name: Application display name
|
||||
slug: Application slug (URL-safe identifier)
|
||||
provider_pk: Primary key of the provider to use
|
||||
launch_url: Optional launch URL
|
||||
icon_url: Optional icon URL
|
||||
|
||||
Returns:
|
||||
Created application data
|
||||
"""
|
||||
app_data = {
|
||||
"name": name,
|
||||
"slug": slug,
|
||||
"provider": provider_pk,
|
||||
"meta_launch_url": launch_url or "",
|
||||
"meta_icon": icon_url or "",
|
||||
"policy_engine_mode": "any",
|
||||
"open_in_new_tab": False
|
||||
}
|
||||
|
||||
result = await self._request("POST", "core/applications/", json=app_data)
|
||||
logger.info(f"Created application: {name} (slug: {slug})")
|
||||
return result
|
||||
|
||||
async def get_provider_by_name(self, name: str) -> Optional[Dict]:
|
||||
"""Get OAuth2 provider by name"""
|
||||
providers = await self._request("GET", "providers/oauth2/", params={"name": name})
|
||||
results = providers.get("results", [])
|
||||
return results[0] if results else None
|
||||
|
||||
async def get_application_by_slug(self, slug: str) -> Optional[Dict]:
|
||||
"""Get application by slug"""
|
||||
apps = await self._request("GET", "core/applications/", params={"slug": slug})
|
||||
results = apps.get("results", [])
|
||||
return results[0] if results else None
|
||||
|
||||
async def list_flows(self) -> List[Dict]:
|
||||
"""List all authentication flows"""
|
||||
result = await self._request("GET", "flows/instances/")
|
||||
return result.get("results", [])
|
||||
|
||||
async def create_proxy_provider(
|
||||
self,
|
||||
name: str,
|
||||
external_host: str,
|
||||
authorization_flow_slug: str = "default-provider-authorization-implicit-consent",
|
||||
mode: str = "forward_single",
|
||||
token_validity: int = 480 # 8 hours in minutes
|
||||
) -> Dict:
|
||||
"""
|
||||
Create a Proxy Provider for forward authentication
|
||||
|
||||
Args:
|
||||
name: Provider name
|
||||
external_host: External URL (e.g., https://auth.schweitz.net)
|
||||
authorization_flow_slug: Authorization flow slug
|
||||
mode: Proxy mode (forward_single for forward auth)
|
||||
token_validity: Token validity in minutes (default: 480 = 8 hours)
|
||||
|
||||
Returns:
|
||||
Created provider data
|
||||
"""
|
||||
# Get authorization flow UUID from slug
|
||||
flows = await self.list_flows()
|
||||
auth_flow_uuid = None
|
||||
invalidation_flow_uuid = None
|
||||
|
||||
for flow in flows:
|
||||
if flow.get("slug") == authorization_flow_slug:
|
||||
auth_flow_uuid = flow.get("pk")
|
||||
if flow.get("slug") == "default-provider-invalidation-flow":
|
||||
invalidation_flow_uuid = flow.get("pk")
|
||||
|
||||
if not auth_flow_uuid:
|
||||
raise ValueError(f"Authorization flow '{authorization_flow_slug}' not found")
|
||||
if not invalidation_flow_uuid:
|
||||
raise ValueError("Invalidation flow not found")
|
||||
|
||||
provider_data = {
|
||||
"name": name,
|
||||
"authorization_flow": auth_flow_uuid,
|
||||
"invalidation_flow": invalidation_flow_uuid,
|
||||
"mode": mode,
|
||||
"external_host": external_host,
|
||||
"access_token_validity": f"minutes={token_validity}",
|
||||
"refresh_token_validity": f"minutes={token_validity}",
|
||||
"session_duration": f"seconds={token_validity * 60}",
|
||||
"cookie_domain": "", # Will use the domain of each proxied site
|
||||
"property_mappings": []
|
||||
}
|
||||
|
||||
result = await self._request("POST", "providers/proxy/", json=provider_data)
|
||||
logger.info(f"Created Proxy provider: {name} (ID: {result.get('pk')})")
|
||||
return result
|
||||
|
||||
async def get_provider_by_name_proxy(self, name: str) -> Optional[Dict]:
|
||||
"""Get Proxy provider by name"""
|
||||
providers = await self._request("GET", "providers/proxy/", params={"name": name})
|
||||
results = providers.get("results", [])
|
||||
return results[0] if results else None
|
||||
|
||||
async def create_outpost(
|
||||
self,
|
||||
name: str,
|
||||
type: str,
|
||||
providers: List[int],
|
||||
config: Optional[Dict] = None
|
||||
) -> Dict:
|
||||
"""
|
||||
Create an Authentik Outpost
|
||||
|
||||
Args:
|
||||
name: Outpost name
|
||||
type: Outpost type (e.g., "proxy")
|
||||
providers: List of provider PKs
|
||||
config: Optional configuration overrides
|
||||
|
||||
Returns:
|
||||
Created outpost data
|
||||
"""
|
||||
outpost_data = {
|
||||
"name": name,
|
||||
"type": type,
|
||||
"providers": providers,
|
||||
"config": config or {},
|
||||
"service_connection": None # Will use local Docker
|
||||
}
|
||||
|
||||
result = await self._request("POST", "outposts/instances/", json=outpost_data)
|
||||
logger.info(f"Created outpost: {name} (ID: {result.get('pk')})")
|
||||
return result
|
||||
|
||||
async def get_outpost_by_name(self, name: str) -> Optional[Dict]:
|
||||
"""Get outpost by name"""
|
||||
outposts = await self._request("GET", "outposts/instances/", params={"name": name})
|
||||
results = outposts.get("results", [])
|
||||
return results[0] if results else None
|
||||
|
||||
async def close(self):
|
||||
"""Close HTTP client"""
|
||||
await self.client.aclose()
|
||||
|
||||
|
||||
@lru_cache()
|
||||
def get_authentik_client() -> AuthentikClient:
|
||||
"""Get cached Authentik client instance"""
|
||||
# Import credentials from gitignored module
|
||||
try:
|
||||
from src.credentials import AUTHENTIK_URL, AUTHENTIK_CORE_API_TOKEN
|
||||
except ImportError:
|
||||
# Fallback to environment variables if credentials.py doesn't exist
|
||||
import os
|
||||
AUTHENTIK_URL = os.getenv("AUTHENTIK_URL", "http://authentik-server:9000")
|
||||
AUTHENTIK_CORE_API_TOKEN = os.getenv("AUTHENTIK_API_TOKEN", "")
|
||||
|
||||
return AuthentikClient(
|
||||
base_url=AUTHENTIK_URL,
|
||||
api_token=AUTHENTIK_CORE_API_TOKEN
|
||||
)
|
||||
@@ -0,0 +1,441 @@
|
||||
"""
|
||||
Uptime Kuma Socket.IO Client
|
||||
|
||||
Provides interface to Uptime Kuma via Socket.IO for monitor management.
|
||||
"""
|
||||
import socketio
|
||||
import asyncio
|
||||
from typing import Optional, Dict, List, Any
|
||||
from src.logging_config import get_logger
|
||||
from src.config import get_settings
|
||||
|
||||
logger = get_logger(__name__)
|
||||
settings = get_settings()
|
||||
|
||||
|
||||
class KumaClient:
|
||||
"""
|
||||
Socket.IO client for Uptime Kuma
|
||||
|
||||
Uses Socket.IO for real-time communication with Uptime Kuma.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: Optional[str] = None,
|
||||
username: Optional[str] = None,
|
||||
password: Optional[str] = None,
|
||||
timeout: int = 30
|
||||
):
|
||||
"""
|
||||
Initialize Kuma client
|
||||
|
||||
Args:
|
||||
base_url: Kuma base URL (default from settings)
|
||||
username: Kuma username (default from settings)
|
||||
password: Kuma password (default from settings)
|
||||
timeout: Request timeout in seconds
|
||||
"""
|
||||
self.base_url = (base_url or settings.kuma_url).rstrip("/")
|
||||
self.username = username or settings.kuma_username
|
||||
self.password = password or settings.kuma_password
|
||||
self.timeout = timeout
|
||||
|
||||
self.sio = socketio.AsyncClient(
|
||||
reconnection=True,
|
||||
reconnection_attempts=3,
|
||||
reconnection_delay=1,
|
||||
)
|
||||
self._connected = False
|
||||
self._authenticated = False
|
||||
self._monitors_cache: Dict[int, Dict[str, Any]] = {}
|
||||
|
||||
if not self.username or not self.password:
|
||||
logger.warning("Uptime Kuma credentials not configured")
|
||||
|
||||
async def _ensure_connected(self):
|
||||
"""Ensure we have an active connection and authentication"""
|
||||
if not self._connected:
|
||||
await self.connect()
|
||||
if not self._authenticated:
|
||||
await self.login()
|
||||
|
||||
async def connect(self):
|
||||
"""Connect to Uptime Kuma Socket.IO server"""
|
||||
if self._connected:
|
||||
return
|
||||
|
||||
try:
|
||||
await self.sio.connect(self.base_url, transports=['websocket'])
|
||||
self._connected = True
|
||||
logger.info(f"Connected to Uptime Kuma at {self.base_url}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to connect to Uptime Kuma: {e}")
|
||||
raise
|
||||
|
||||
async def disconnect(self):
|
||||
"""Disconnect from Uptime Kuma"""
|
||||
if self._connected:
|
||||
await self.sio.disconnect()
|
||||
self._connected = False
|
||||
self._authenticated = False
|
||||
logger.info("Disconnected from Uptime Kuma")
|
||||
|
||||
async def login(self):
|
||||
"""Authenticate with Uptime Kuma"""
|
||||
if not self._connected:
|
||||
await self.connect()
|
||||
|
||||
try:
|
||||
# Uptime Kuma login event
|
||||
login_response = await self.sio.call(
|
||||
'login',
|
||||
{
|
||||
'username': self.username,
|
||||
'password': self.password,
|
||||
'token': None
|
||||
},
|
||||
timeout=self.timeout
|
||||
)
|
||||
|
||||
if login_response and login_response.get('ok'):
|
||||
self._authenticated = True
|
||||
logger.info("Successfully authenticated with Uptime Kuma")
|
||||
else:
|
||||
error_msg = login_response.get('msg', 'Unknown error') if login_response else 'No response'
|
||||
raise Exception(f"Login failed: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to authenticate with Uptime Kuma: {e}")
|
||||
raise
|
||||
|
||||
async def health_check(self) -> bool:
|
||||
"""
|
||||
Check if Uptime Kuma is accessible
|
||||
|
||||
Returns:
|
||||
True if accessible, False otherwise
|
||||
"""
|
||||
try:
|
||||
await self._ensure_connected()
|
||||
return self._authenticated
|
||||
except Exception as e:
|
||||
logger.error(f"Uptime Kuma health check failed: {e}")
|
||||
return False
|
||||
|
||||
async def get_monitors(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
List all monitors
|
||||
|
||||
Returns:
|
||||
List of monitor configurations
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Get monitor list
|
||||
response = await self.sio.call('getMonitorList', timeout=self.timeout)
|
||||
|
||||
if response and isinstance(response, dict):
|
||||
# Uptime Kuma returns monitors as a dict with monitor IDs as keys
|
||||
monitors = []
|
||||
for monitor_id, monitor_data in response.items():
|
||||
if isinstance(monitor_data, dict):
|
||||
monitor_data['id'] = int(monitor_id)
|
||||
monitors.append(monitor_data)
|
||||
self._monitors_cache[int(monitor_id)] = monitor_data
|
||||
|
||||
return monitors
|
||||
return []
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get monitors: {e}")
|
||||
raise
|
||||
|
||||
async def get_monitor(self, monitor_id: int) -> Dict[str, Any]:
|
||||
"""
|
||||
Get details of a specific monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
Monitor configuration details
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
response = await self.sio.call('getMonitor', monitor_id, timeout=self.timeout)
|
||||
|
||||
if response:
|
||||
self._monitors_cache[monitor_id] = response
|
||||
return response
|
||||
|
||||
raise Exception(f"Monitor {monitor_id} not found")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get monitor {monitor_id}: {e}")
|
||||
raise
|
||||
|
||||
async def find_monitor_by_name(self, name: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Find a monitor by its name (case-insensitive)
|
||||
|
||||
Args:
|
||||
name: Monitor name to search for
|
||||
|
||||
Returns:
|
||||
Monitor object if found, None otherwise
|
||||
"""
|
||||
monitors = await self.get_monitors()
|
||||
name_lower = name.lower()
|
||||
|
||||
for monitor in monitors:
|
||||
if monitor.get("name", "").lower() == name_lower:
|
||||
return monitor
|
||||
|
||||
return None
|
||||
|
||||
async def find_monitors_by_tag(self, tag: str) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Find all monitors with a specific tag
|
||||
|
||||
Args:
|
||||
tag: Tag name to search for
|
||||
|
||||
Returns:
|
||||
List of monitors with the tag
|
||||
"""
|
||||
monitors = await self.get_monitors()
|
||||
tagged_monitors = []
|
||||
|
||||
for monitor in monitors:
|
||||
monitor_tags = monitor.get("tags", [])
|
||||
if any(t.get("name", "").lower() == tag.lower() for t in monitor_tags):
|
||||
tagged_monitors.append(monitor)
|
||||
|
||||
return tagged_monitors
|
||||
|
||||
async def pause_monitor(self, monitor_id: int) -> bool:
|
||||
"""
|
||||
Pause a monitor (disable monitoring)
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
True if successful
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Uptime Kuma pause event
|
||||
response = await self.sio.call('pauseMonitor', monitor_id, timeout=self.timeout)
|
||||
|
||||
if response and response.get('ok'):
|
||||
logger.info(f"Paused monitor {monitor_id}")
|
||||
return True
|
||||
|
||||
error_msg = response.get('msg', 'Unknown error') if response else 'No response'
|
||||
raise Exception(f"Failed to pause monitor: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to pause monitor {monitor_id}: {e}")
|
||||
raise
|
||||
|
||||
async def resume_monitor(self, monitor_id: int) -> bool:
|
||||
"""
|
||||
Resume a monitor (enable monitoring)
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
True if successful
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Uptime Kuma resume event
|
||||
response = await self.sio.call('resumeMonitor', monitor_id, timeout=self.timeout)
|
||||
|
||||
if response and response.get('ok'):
|
||||
logger.info(f"Resumed monitor {monitor_id}")
|
||||
return True
|
||||
|
||||
error_msg = response.get('msg', 'Unknown error') if response else 'No response'
|
||||
raise Exception(f"Failed to resume monitor: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to resume monitor {monitor_id}: {e}")
|
||||
raise
|
||||
|
||||
async def pause_monitor_by_name(self, name: str) -> bool:
|
||||
"""
|
||||
Pause a monitor by its name
|
||||
|
||||
Args:
|
||||
name: Monitor name
|
||||
|
||||
Returns:
|
||||
True if successful, False if monitor not found
|
||||
"""
|
||||
monitor = await self.find_monitor_by_name(name)
|
||||
if not monitor:
|
||||
logger.warning(f"Monitor '{name}' not found")
|
||||
return False
|
||||
|
||||
await self.pause_monitor(monitor["id"])
|
||||
return True
|
||||
|
||||
async def resume_monitor_by_name(self, name: str) -> bool:
|
||||
"""
|
||||
Resume a monitor by its name
|
||||
|
||||
Args:
|
||||
name: Monitor name
|
||||
|
||||
Returns:
|
||||
True if successful, False if monitor not found
|
||||
"""
|
||||
monitor = await self.find_monitor_by_name(name)
|
||||
if not monitor:
|
||||
logger.warning(f"Monitor '{name}' not found")
|
||||
return False
|
||||
|
||||
await self.resume_monitor(monitor["id"])
|
||||
return True
|
||||
|
||||
async def add_monitor(self, monitor_config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Create a new monitor
|
||||
|
||||
Args:
|
||||
monitor_config: Monitor configuration dict
|
||||
|
||||
Returns:
|
||||
Created monitor details including ID
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Uptime Kuma add monitor event
|
||||
response = await self.sio.call('add', monitor_config, timeout=self.timeout)
|
||||
|
||||
if response and response.get('ok'):
|
||||
monitor_id = response.get('monitorID')
|
||||
logger.info(f"Created monitor '{monitor_config.get('name')}' with ID {monitor_id}")
|
||||
|
||||
# Get full monitor details
|
||||
monitor = await self.get_monitor(monitor_id)
|
||||
return monitor
|
||||
|
||||
error_msg = response.get('msg', 'Unknown error') if response else 'No response'
|
||||
raise Exception(f"Failed to create monitor: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to create monitor '{monitor_config.get('name')}': {e}")
|
||||
raise
|
||||
|
||||
async def update_monitor(self, monitor_id: int, monitor_config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Update an existing monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
monitor_config: Updated monitor configuration
|
||||
|
||||
Returns:
|
||||
Updated monitor details
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Ensure ID is in the config
|
||||
monitor_config['id'] = monitor_id
|
||||
|
||||
# Uptime Kuma edit monitor event
|
||||
response = await self.sio.call('editMonitor', monitor_config, timeout=self.timeout)
|
||||
|
||||
if response and response.get('ok'):
|
||||
logger.info(f"Updated monitor {monitor_id}")
|
||||
|
||||
# Get updated monitor details
|
||||
monitor = await self.get_monitor(monitor_id)
|
||||
return monitor
|
||||
|
||||
error_msg = response.get('msg', 'Unknown error') if response else 'No response'
|
||||
raise Exception(f"Failed to update monitor: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to update monitor {monitor_id}: {e}")
|
||||
raise
|
||||
|
||||
async def delete_monitor(self, monitor_id: int) -> bool:
|
||||
"""
|
||||
Delete a monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
True if successful
|
||||
"""
|
||||
await self._ensure_connected()
|
||||
|
||||
try:
|
||||
# Uptime Kuma delete monitor event
|
||||
response = await self.sio.call('deleteMonitor', monitor_id, timeout=self.timeout)
|
||||
|
||||
if response and response.get('ok'):
|
||||
logger.info(f"Deleted monitor {monitor_id}")
|
||||
|
||||
# Remove from cache
|
||||
self._monitors_cache.pop(monitor_id, None)
|
||||
return True
|
||||
|
||||
error_msg = response.get('msg', 'Unknown error') if response else 'No response'
|
||||
raise Exception(f"Failed to delete monitor: {error_msg}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete monitor {monitor_id}: {e}")
|
||||
raise
|
||||
|
||||
async def delete_monitor_by_name(self, name: str) -> bool:
|
||||
"""
|
||||
Delete a monitor by its name
|
||||
|
||||
Args:
|
||||
name: Monitor name
|
||||
|
||||
Returns:
|
||||
True if successful, False if monitor not found
|
||||
"""
|
||||
monitor = await self.find_monitor_by_name(name)
|
||||
if not monitor:
|
||||
logger.warning(f"Monitor '{name}' not found")
|
||||
return False
|
||||
|
||||
await self.delete_monitor(monitor["id"])
|
||||
return True
|
||||
|
||||
async def __aenter__(self):
|
||||
"""Async context manager entry"""
|
||||
await self._ensure_connected()
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
"""Async context manager exit"""
|
||||
await self.disconnect()
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_kuma_client: Optional[KumaClient] = None
|
||||
|
||||
|
||||
def get_kuma_client() -> KumaClient:
|
||||
"""Get singleton Kuma client instance"""
|
||||
global _kuma_client
|
||||
if _kuma_client is None:
|
||||
_kuma_client = KumaClient()
|
||||
return _kuma_client
|
||||
@@ -99,9 +99,11 @@ class NPMClient:
|
||||
True if accessible, False otherwise
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
async with httpx.AsyncClient(timeout=self.timeout, follow_redirects=True) as client:
|
||||
response = await client.get(f"{self.base_url}/api")
|
||||
return response.status_code == 200
|
||||
# Accept any successful response (2xx) or redirect (3xx) as healthy
|
||||
# A redirect indicates the service is up and responding
|
||||
return 200 <= response.status_code < 400
|
||||
except Exception as e:
|
||||
logger.error(f"NPM health check failed: {e}")
|
||||
return False
|
||||
@@ -207,6 +209,116 @@ class NPMClient:
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def update_proxy_host(
|
||||
self,
|
||||
proxy_id: int,
|
||||
config: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Update an existing proxy host configuration
|
||||
|
||||
Args:
|
||||
proxy_id: Proxy host ID to update
|
||||
config: Full proxy host configuration (get from get_proxy_host, modify, then update)
|
||||
|
||||
Returns:
|
||||
Updated proxy host details
|
||||
"""
|
||||
await self._ensure_token()
|
||||
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
response = await client.put(
|
||||
f"{self.base_url}/api/nginx/proxy-hosts/{proxy_id}",
|
||||
headers=self._get_headers(),
|
||||
json=config
|
||||
)
|
||||
|
||||
if not response.is_success:
|
||||
logger.error(f"Update failed: {response.status_code}")
|
||||
logger.error(f"Response: {response.text}")
|
||||
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def enable_authentik_forward_auth(
|
||||
self,
|
||||
proxy_id: int,
|
||||
authentik_url: str = "http://authentik-server:9000"
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Enable Authentik forward authentication on a proxy host
|
||||
|
||||
Args:
|
||||
proxy_id: Proxy host ID to update
|
||||
authentik_url: Authentik server URL (default: http://authentik-server:9000)
|
||||
|
||||
Returns:
|
||||
Updated proxy host details
|
||||
"""
|
||||
# Get current config
|
||||
proxy_host = await self.get_proxy_host(proxy_id)
|
||||
|
||||
# Authentik forward auth configuration
|
||||
auth_config = f"""# Authentik Forward Authentication
|
||||
# Send authentication requests to Authentik
|
||||
auth_request /outpost.goauthentik.io/auth/nginx;
|
||||
|
||||
# Preserve authentication cookies
|
||||
auth_request_set $auth_cookie $upstream_http_set_cookie;
|
||||
add_header Set-Cookie $auth_cookie;
|
||||
|
||||
# Get user information from Authentik
|
||||
auth_request_set $authentik_username $upstream_http_x_authentik_username;
|
||||
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
|
||||
auth_request_set $authentik_email $upstream_http_x_authentik_email;
|
||||
auth_request_set $authentik_name $upstream_http_x_authentik_name;
|
||||
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;
|
||||
|
||||
# Pass user info to backend
|
||||
proxy_set_header X-authentik-username $authentik_username;
|
||||
proxy_set_header X-authentik-groups $authentik_groups;
|
||||
proxy_set_header X-authentik-email $authentik_email;
|
||||
proxy_set_header X-authentik-name $authentik_name;
|
||||
proxy_set_header X-authentik-uid $authentik_uid;
|
||||
|
||||
# On authentication failure, redirect to Authentik login
|
||||
error_page 401 = @authentik_proxy_signin;
|
||||
|
||||
location @authentik_proxy_signin {{
|
||||
internal;
|
||||
add_header Set-Cookie $auth_cookie;
|
||||
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
|
||||
}}
|
||||
|
||||
# Authentik authentication endpoint
|
||||
location /outpost.goauthentik.io {{
|
||||
proxy_pass {authentik_url}/outpost.goauthentik.io;
|
||||
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header Host $host;
|
||||
}}
|
||||
"""
|
||||
|
||||
# Update the advanced config
|
||||
proxy_host["advanced_config"] = auth_config
|
||||
|
||||
# Remove read-only fields that NPM doesn't accept in updates
|
||||
readonly_fields = [
|
||||
"id", "created_on", "modified_on", "owner", "owner_user_id",
|
||||
"certificate", "use_default_location", "ipv6", "meta", "nginx_online",
|
||||
"nginx_err", "access_list", "certificate_id"
|
||||
]
|
||||
|
||||
clean_config = {k: v for k, v in proxy_host.items() if k not in readonly_fields}
|
||||
|
||||
# Ensure locations is an array (required field)
|
||||
if "locations" not in clean_config or clean_config["locations"] is None:
|
||||
clean_config["locations"] = []
|
||||
|
||||
# Update the proxy host
|
||||
return await self.update_proxy_host(proxy_id, clean_config)
|
||||
|
||||
async def get_certificates(self) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
List all SSL certificates
|
||||
|
||||
@@ -208,6 +208,87 @@ class PortainerClient:
|
||||
response.raise_for_status()
|
||||
return True
|
||||
|
||||
async def get_containers(self, endpoint_id: int, all_containers: bool = True) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
List containers on a specific endpoint
|
||||
|
||||
Args:
|
||||
endpoint_id: Portainer endpoint identifier
|
||||
all_containers: Include stopped containers (default: True)
|
||||
|
||||
Returns:
|
||||
List of container details
|
||||
"""
|
||||
params = {"all": 1 if all_containers else 0}
|
||||
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
response = await client.get(
|
||||
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/json",
|
||||
headers=self._get_headers(),
|
||||
params=params
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def get_container(self, endpoint_id: int, container_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get detailed information about a specific container
|
||||
|
||||
Args:
|
||||
endpoint_id: Portainer endpoint identifier
|
||||
container_id: Container ID or name
|
||||
|
||||
Returns:
|
||||
Container details including network and port information
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
response = await client.get(
|
||||
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/json",
|
||||
headers=self._get_headers()
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
async def stop_container(self, endpoint_id: int, container_id: str) -> bool:
|
||||
"""
|
||||
Stop a container
|
||||
|
||||
Args:
|
||||
endpoint_id: Portainer endpoint identifier
|
||||
container_id: Container ID or name
|
||||
|
||||
Returns:
|
||||
True if successful
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
response = await client.post(
|
||||
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/stop",
|
||||
headers=self._get_headers()
|
||||
)
|
||||
response.raise_for_status()
|
||||
logger.info(f"Stopped container {container_id}")
|
||||
return True
|
||||
|
||||
async def start_container(self, endpoint_id: int, container_id: str) -> bool:
|
||||
"""
|
||||
Start a container
|
||||
|
||||
Args:
|
||||
endpoint_id: Portainer endpoint identifier
|
||||
container_id: Container ID or name
|
||||
|
||||
Returns:
|
||||
True if successful
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||
response = await client.post(
|
||||
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/start",
|
||||
headers=self._get_headers()
|
||||
)
|
||||
response.raise_for_status()
|
||||
logger.info(f"Started container {container_id}")
|
||||
return True
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_portainer_client: Optional[PortainerClient] = None
|
||||
|
||||
@@ -89,6 +89,11 @@ class Settings(BaseSettings):
|
||||
kuma_username: str = KUMA_USERNAME
|
||||
kuma_password: str = KUMA_PASSWORD
|
||||
|
||||
# OIDC Authentication (Authentik)
|
||||
oidc_enabled: bool = False # Set to True to require authentication
|
||||
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
|
||||
oidc_audience: str = "core-api"
|
||||
|
||||
@property
|
||||
def model_aliases(self) -> dict:
|
||||
"""Computed property for model aliases"""
|
||||
|
||||
@@ -0,0 +1,671 @@
|
||||
"""
|
||||
AI Controller
|
||||
|
||||
Provides AI orchestration endpoints including:
|
||||
- OpenAI-compatible chat completions
|
||||
- Model listing
|
||||
- Conversation memory management
|
||||
"""
|
||||
import time
|
||||
import logging
|
||||
import uuid
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from fastapi.responses import StreamingResponse
|
||||
from typing import AsyncIterator, List, Optional
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from src.controllers.base import BaseController
|
||||
from src.api.v1.schemas import (
|
||||
ChatCompletionRequest,
|
||||
ChatCompletionResponse,
|
||||
ChatCompletionChoice,
|
||||
ChatMessageResponse,
|
||||
UsageInfo,
|
||||
ChatCompletionStreamResponse,
|
||||
ChatCompletionStreamChoice,
|
||||
DeltaMessage,
|
||||
ModelsListResponse,
|
||||
ModelInfo,
|
||||
)
|
||||
from src.models.ollama_client import get_ollama_client
|
||||
from src.memory import get_memory_manager, MessageRole as MemoryMessageRole, TokenUsage
|
||||
from src.config import get_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Request/Response Models for Conversations
|
||||
class SearchRequest(BaseModel):
|
||||
"""Request model for semantic search"""
|
||||
query: str = Field(..., description="Search query")
|
||||
limit: int = Field(5, ge=1, le=50, description="Maximum number of results")
|
||||
|
||||
|
||||
class ConversationTurnResponse(BaseModel):
|
||||
"""Response model for a conversation turn"""
|
||||
turn_number: int
|
||||
role: str
|
||||
content: str
|
||||
timestamp: str
|
||||
tokens_prompt: Optional[int] = None
|
||||
tokens_completion: Optional[int] = None
|
||||
tokens_total: Optional[int] = None
|
||||
metadata: dict = Field(default_factory=dict)
|
||||
|
||||
|
||||
class ConversationHistoryResponse(BaseModel):
|
||||
"""Response model for conversation history"""
|
||||
conversation_id: str
|
||||
turn_count: int
|
||||
total_tokens: int
|
||||
turns: List[ConversationTurnResponse]
|
||||
|
||||
|
||||
class SearchResultResponse(BaseModel):
|
||||
"""Response model for a single search result"""
|
||||
conversation_id: str
|
||||
turn_number: int
|
||||
role: str
|
||||
content: str
|
||||
timestamp: str
|
||||
score: float
|
||||
|
||||
|
||||
class SearchResponse(BaseModel):
|
||||
"""Response model for search results"""
|
||||
query: str
|
||||
results: List[SearchResultResponse]
|
||||
count: int
|
||||
|
||||
|
||||
class ConversationStatsResponse(BaseModel):
|
||||
"""Response model for conversation statistics"""
|
||||
conversation_id: str
|
||||
buffer_turns: int
|
||||
buffer_tokens: int
|
||||
qdrant_turns: int
|
||||
qdrant_tokens: int
|
||||
exists_in_buffer: bool
|
||||
exists_in_qdrant: bool
|
||||
|
||||
|
||||
class DeleteResponse(BaseModel):
|
||||
"""Response model for delete operation"""
|
||||
conversation_id: str
|
||||
deleted: bool
|
||||
message: str
|
||||
|
||||
|
||||
# Helper functions
|
||||
def build_prompt_from_messages(messages: list) -> str:
|
||||
"""
|
||||
Convert message list to a prompt string.
|
||||
"""
|
||||
prompt_parts = []
|
||||
|
||||
for msg in messages:
|
||||
role = msg.role.value if hasattr(msg.role, 'value') else msg.role
|
||||
content = msg.content
|
||||
|
||||
if role == "system":
|
||||
prompt_parts.append(f"System: {content}")
|
||||
elif role == "user":
|
||||
prompt_parts.append(f"User: {content}")
|
||||
elif role == "assistant":
|
||||
prompt_parts.append(f"Assistant: {content}")
|
||||
|
||||
prompt_parts.append("Assistant:")
|
||||
return "\n\n".join(prompt_parts)
|
||||
|
||||
|
||||
async def stream_chat_completion(
|
||||
request_id: str,
|
||||
model: str,
|
||||
prompt: str,
|
||||
temperature: float,
|
||||
max_tokens: int | None
|
||||
) -> AsyncIterator[str]:
|
||||
"""
|
||||
Stream chat completion in OpenAI SSE format.
|
||||
|
||||
Yields:
|
||||
Server-Sent Events formatted strings
|
||||
"""
|
||||
created = int(time.time())
|
||||
ollama_client = get_ollama_client()
|
||||
|
||||
# First chunk with role
|
||||
first_chunk = ChatCompletionStreamResponse(
|
||||
id=request_id,
|
||||
created=created,
|
||||
model=model,
|
||||
choices=[
|
||||
ChatCompletionStreamChoice(
|
||||
index=0,
|
||||
delta=DeltaMessage(role="assistant"),
|
||||
finish_reason=None
|
||||
)
|
||||
]
|
||||
)
|
||||
yield f"data: {first_chunk.model_dump_json()}\n\n"
|
||||
|
||||
# Stream tokens
|
||||
try:
|
||||
async for token in ollama_client.generate_streaming(
|
||||
model=model,
|
||||
prompt=prompt,
|
||||
temperature=temperature,
|
||||
max_tokens=max_tokens
|
||||
):
|
||||
chunk = ChatCompletionStreamResponse(
|
||||
id=request_id,
|
||||
created=created,
|
||||
model=model,
|
||||
choices=[
|
||||
ChatCompletionStreamChoice(
|
||||
index=0,
|
||||
delta=DeltaMessage(content=token),
|
||||
finish_reason=None
|
||||
)
|
||||
]
|
||||
)
|
||||
yield f"data: {chunk.model_dump_json()}\n\n"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Streaming error: {e}")
|
||||
# Send error in OpenAI format
|
||||
import json
|
||||
error_chunk = {
|
||||
"error": {
|
||||
"message": str(e),
|
||||
"type": "server_error"
|
||||
}
|
||||
}
|
||||
yield f"data: {json.dumps(error_chunk)}\n\n"
|
||||
return
|
||||
|
||||
# Final chunk
|
||||
final_chunk = ChatCompletionStreamResponse(
|
||||
id=request_id,
|
||||
created=created,
|
||||
model=model,
|
||||
choices=[
|
||||
ChatCompletionStreamChoice(
|
||||
index=0,
|
||||
delta=DeltaMessage(),
|
||||
finish_reason="stop"
|
||||
)
|
||||
]
|
||||
)
|
||||
yield f"data: {final_chunk.model_dump_json()}\n\n"
|
||||
yield "data: [DONE]\n\n"
|
||||
|
||||
|
||||
async def store_conversation_turn(
|
||||
conversation_id: str,
|
||||
role: str,
|
||||
content: str,
|
||||
tokens: dict = None
|
||||
):
|
||||
"""
|
||||
Store a conversation turn in memory
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
role: Message role (user, assistant, system)
|
||||
content: Message content
|
||||
tokens: Optional token usage dict
|
||||
"""
|
||||
try:
|
||||
memory_manager = get_memory_manager()
|
||||
|
||||
# Convert role string to MemoryMessageRole
|
||||
if role == "user":
|
||||
memory_role = MemoryMessageRole.USER
|
||||
elif role == "assistant":
|
||||
memory_role = MemoryMessageRole.ASSISTANT
|
||||
elif role == "system":
|
||||
memory_role = MemoryMessageRole.SYSTEM
|
||||
else:
|
||||
memory_role = MemoryMessageRole.USER # Default fallback
|
||||
|
||||
# Create TokenUsage if provided
|
||||
token_usage = None
|
||||
if tokens:
|
||||
token_usage = TokenUsage(
|
||||
prompt=tokens.get("prompt", 0),
|
||||
completion=tokens.get("completion", 0),
|
||||
total=tokens.get("total", 0)
|
||||
)
|
||||
|
||||
# Store in memory
|
||||
await memory_manager.add_turn(
|
||||
conversation_id=conversation_id,
|
||||
role=memory_role,
|
||||
content=content,
|
||||
tokens=token_usage
|
||||
)
|
||||
|
||||
logger.debug(f"Stored {role} turn in memory for conversation {conversation_id}")
|
||||
|
||||
except Exception as e:
|
||||
# Log error but don't fail the request
|
||||
logger.error(f"Failed to store turn in memory: {e}")
|
||||
|
||||
|
||||
class AIController(BaseController):
|
||||
"""
|
||||
Controller for AI orchestration operations
|
||||
|
||||
Provides endpoints for:
|
||||
- OpenAI-compatible chat completions (streaming and non-streaming)
|
||||
- Model listing
|
||||
- Conversation memory management
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(prefix="/v1", tags=["AI"])
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the router"""
|
||||
router = APIRouter()
|
||||
settings = get_settings()
|
||||
|
||||
# Chat Completions Endpoint
|
||||
@router.post(
|
||||
"/v1/chat/completions",
|
||||
tags=["AI"]
|
||||
)
|
||||
async def chat_completions(request: ChatCompletionRequest):
|
||||
"""
|
||||
OpenAI-compatible chat completions endpoint.
|
||||
Supports both streaming and non-streaming.
|
||||
|
||||
Automatically stores conversations in memory system if enabled.
|
||||
"""
|
||||
request_id = f"chatcmpl-{int(time.time() * 1000)}"
|
||||
|
||||
# Generate or use provided conversation_id
|
||||
conversation_id = request.conversation_id or f"conv_{uuid.uuid4().hex[:16]}"
|
||||
|
||||
logger.info(
|
||||
f"Chat request: id={request_id}, model={request.model}, "
|
||||
f"messages={len(request.messages)}, stream={request.stream}, "
|
||||
f"conversation_id={conversation_id}, store_in_memory={request.store_in_memory}"
|
||||
)
|
||||
|
||||
# Store user messages in memory (if enabled)
|
||||
if request.store_in_memory:
|
||||
for msg in request.messages:
|
||||
role = msg.role.value if hasattr(msg.role, 'value') else msg.role
|
||||
if role == "user": # Store latest user message
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role=role,
|
||||
content=msg.content
|
||||
)
|
||||
|
||||
# Build prompt from messages
|
||||
prompt = build_prompt_from_messages(request.messages)
|
||||
|
||||
# Streaming response
|
||||
if request.stream:
|
||||
return StreamingResponse(
|
||||
stream_chat_completion(
|
||||
request_id=request_id,
|
||||
model=request.model,
|
||||
prompt=prompt,
|
||||
temperature=request.temperature,
|
||||
max_tokens=request.max_tokens
|
||||
),
|
||||
media_type="text/event-stream"
|
||||
)
|
||||
|
||||
# Non-streaming response
|
||||
try:
|
||||
ollama_client = get_ollama_client()
|
||||
result = await ollama_client.generate_non_streaming(
|
||||
model=request.model,
|
||||
prompt=prompt,
|
||||
temperature=request.temperature,
|
||||
max_tokens=request.max_tokens
|
||||
)
|
||||
|
||||
assistant_content = result["response"]
|
||||
|
||||
# Store assistant response in memory (if enabled)
|
||||
if request.store_in_memory:
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role="assistant",
|
||||
content=assistant_content,
|
||||
tokens=result["tokens"]
|
||||
)
|
||||
|
||||
response = ChatCompletionResponse(
|
||||
id=request_id,
|
||||
created=int(time.time()),
|
||||
model=request.model,
|
||||
choices=[
|
||||
ChatCompletionChoice(
|
||||
index=0,
|
||||
message=ChatMessageResponse(
|
||||
role="assistant",
|
||||
content=assistant_content
|
||||
),
|
||||
finish_reason="stop"
|
||||
)
|
||||
],
|
||||
usage=UsageInfo(
|
||||
prompt_tokens=result["tokens"]["prompt"],
|
||||
completion_tokens=result["tokens"]["completion"],
|
||||
total_tokens=result["tokens"]["total"]
|
||||
)
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"Chat response: id={request_id}, "
|
||||
f"tokens={result['tokens']['total']}, "
|
||||
f"conversation_id={conversation_id}"
|
||||
)
|
||||
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Chat completion error: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to generate completion: {str(e)}"
|
||||
)
|
||||
|
||||
# Models Endpoint
|
||||
@router.get(
|
||||
"/v1/models",
|
||||
tags=["AI"]
|
||||
)
|
||||
async def list_models():
|
||||
"""List available models in OpenAI format."""
|
||||
|
||||
models = []
|
||||
|
||||
# Add OpenAI-style aliases
|
||||
for alias in settings.model_aliases.keys():
|
||||
models.append(ModelInfo(id=alias, owned_by="tatlock"))
|
||||
|
||||
# Add actual local models
|
||||
for model_list in [
|
||||
settings.get_lightweight_models(),
|
||||
settings.get_heavy_models(),
|
||||
settings.get_code_models()
|
||||
]:
|
||||
for model in model_list:
|
||||
# Avoid duplicates
|
||||
if model not in [m.id for m in models]:
|
||||
models.append(ModelInfo(id=model, owned_by="tatlock"))
|
||||
|
||||
return ModelsListResponse(data=models)
|
||||
|
||||
# Conversation Endpoints
|
||||
@router.get(
|
||||
"/v1/conversations/{conversation_id}",
|
||||
response_model=ConversationHistoryResponse,
|
||||
tags=["Conversations"],
|
||||
summary="Get conversation history",
|
||||
description="Retrieve complete conversation history including all turns"
|
||||
)
|
||||
async def get_conversation(
|
||||
conversation_id: str,
|
||||
include_buffer: bool = Query(
|
||||
True,
|
||||
description="Include recent turns from buffer that haven't been consolidated yet"
|
||||
)
|
||||
):
|
||||
"""
|
||||
Get complete conversation history
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
include_buffer: Include recent buffer turns not yet consolidated
|
||||
|
||||
Returns:
|
||||
Complete conversation history with all turns
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
|
||||
# Get full history
|
||||
turns = await manager.get_full_history(conversation_id, include_buffer=include_buffer)
|
||||
|
||||
if not turns:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Conversation {conversation_id} not found"
|
||||
)
|
||||
|
||||
# Convert to response format
|
||||
turn_responses = []
|
||||
total_tokens = 0
|
||||
|
||||
for turn in turns:
|
||||
turn_response = ConversationTurnResponse(
|
||||
turn_number=turn.turn_number,
|
||||
role=turn.role.value if isinstance(turn.role, MemoryMessageRole) else turn.role,
|
||||
content=turn.content,
|
||||
timestamp=turn.timestamp.isoformat(),
|
||||
metadata=turn.metadata
|
||||
)
|
||||
|
||||
if turn.tokens:
|
||||
turn_response.tokens_prompt = turn.tokens.prompt
|
||||
turn_response.tokens_completion = turn.tokens.completion
|
||||
turn_response.tokens_total = turn.tokens.total
|
||||
total_tokens += turn.tokens.total
|
||||
|
||||
turn_responses.append(turn_response)
|
||||
|
||||
return ConversationHistoryResponse(
|
||||
conversation_id=conversation_id,
|
||||
turn_count=len(turns),
|
||||
total_tokens=total_tokens,
|
||||
turns=turn_responses
|
||||
)
|
||||
|
||||
@router.get(
|
||||
"/v1/conversations/{conversation_id}/stats",
|
||||
response_model=ConversationStatsResponse,
|
||||
tags=["Conversations"],
|
||||
summary="Get conversation statistics",
|
||||
description="Get detailed statistics about a conversation across all storage tiers"
|
||||
)
|
||||
async def get_conversation_stats(conversation_id: str):
|
||||
"""
|
||||
Get conversation statistics
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
|
||||
Returns:
|
||||
Statistics including turn counts and token usage across tiers
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
stats = await manager.get_conversation_stats(conversation_id)
|
||||
|
||||
return ConversationStatsResponse(**stats)
|
||||
|
||||
@router.post(
|
||||
"/v1/conversations/{conversation_id}/search",
|
||||
response_model=SearchResponse,
|
||||
tags=["Conversations"],
|
||||
summary="Search conversation semantically",
|
||||
description="Search for relevant turns within a conversation using semantic similarity"
|
||||
)
|
||||
async def search_conversation(
|
||||
conversation_id: str,
|
||||
search_request: SearchRequest
|
||||
):
|
||||
"""
|
||||
Semantic search within a conversation
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
search_request: Search query and parameters
|
||||
|
||||
Returns:
|
||||
Relevant conversation turns ranked by semantic similarity
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
|
||||
# Perform semantic search
|
||||
results = await manager.search_conversations(
|
||||
query=search_request.query,
|
||||
conversation_id=conversation_id,
|
||||
limit=search_request.limit
|
||||
)
|
||||
|
||||
# Convert to response format
|
||||
search_results = [
|
||||
SearchResultResponse(
|
||||
conversation_id=result["conversation_id"],
|
||||
turn_number=result["turn_number"],
|
||||
role=result["role"],
|
||||
content=result["content"],
|
||||
timestamp=result["timestamp"],
|
||||
score=result["score"]
|
||||
)
|
||||
for result in results
|
||||
]
|
||||
|
||||
return SearchResponse(
|
||||
query=search_request.query,
|
||||
results=search_results,
|
||||
count=len(search_results)
|
||||
)
|
||||
|
||||
@router.post(
|
||||
"/v1/conversations/search",
|
||||
response_model=SearchResponse,
|
||||
tags=["Conversations"],
|
||||
summary="Search all conversations",
|
||||
description="Search across all conversations using semantic similarity"
|
||||
)
|
||||
async def search_all_conversations(search_request: SearchRequest):
|
||||
"""
|
||||
Semantic search across all conversations
|
||||
|
||||
Args:
|
||||
search_request: Search query and parameters
|
||||
|
||||
Returns:
|
||||
Relevant turns from any conversation ranked by semantic similarity
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
|
||||
# Perform semantic search across all conversations
|
||||
results = await manager.search_conversations(
|
||||
query=search_request.query,
|
||||
conversation_id=None, # Search all conversations
|
||||
limit=search_request.limit
|
||||
)
|
||||
|
||||
# Convert to response format
|
||||
search_results = [
|
||||
SearchResultResponse(
|
||||
conversation_id=result["conversation_id"],
|
||||
turn_number=result["turn_number"],
|
||||
role=result["role"],
|
||||
content=result["content"],
|
||||
timestamp=result["timestamp"],
|
||||
score=result["score"]
|
||||
)
|
||||
for result in results
|
||||
]
|
||||
|
||||
return SearchResponse(
|
||||
query=search_request.query,
|
||||
results=search_results,
|
||||
count=len(search_results)
|
||||
)
|
||||
|
||||
@router.delete(
|
||||
"/v1/conversations/{conversation_id}",
|
||||
response_model=DeleteResponse,
|
||||
tags=["Conversations"],
|
||||
summary="Delete conversation",
|
||||
description="Delete a conversation from all storage tiers"
|
||||
)
|
||||
async def delete_conversation(
|
||||
conversation_id: str,
|
||||
clear_buffer: bool = Query(True, description="Clear from buffer (Tier 1)"),
|
||||
clear_qdrant: bool = Query(True, description="Clear from Qdrant (Tier 2/3)")
|
||||
):
|
||||
"""
|
||||
Delete a conversation
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
clear_buffer: Clear from Tier 1 buffer
|
||||
clear_qdrant: Clear from Tier 2/3 Qdrant
|
||||
|
||||
Returns:
|
||||
Deletion confirmation
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
|
||||
try:
|
||||
await manager.clear_conversation(
|
||||
conversation_id,
|
||||
clear_buffer=clear_buffer,
|
||||
clear_qdrant=clear_qdrant
|
||||
)
|
||||
|
||||
return DeleteResponse(
|
||||
conversation_id=conversation_id,
|
||||
deleted=True,
|
||||
message=f"Conversation {conversation_id} deleted successfully"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error deleting conversation: {str(e)}"
|
||||
)
|
||||
|
||||
@router.post(
|
||||
"/v1/conversations/{conversation_id}/consolidate",
|
||||
tags=["Conversations"],
|
||||
summary="Consolidate conversation",
|
||||
description="Manually trigger consolidation from buffer to persistent storage"
|
||||
)
|
||||
async def consolidate_conversation(conversation_id: str):
|
||||
"""
|
||||
Manually consolidate a conversation
|
||||
|
||||
Moves all buffer turns to Qdrant persistent storage.
|
||||
|
||||
Args:
|
||||
conversation_id: Unique conversation identifier
|
||||
|
||||
Returns:
|
||||
Number of turns consolidated
|
||||
"""
|
||||
manager = get_memory_manager()
|
||||
|
||||
try:
|
||||
count = await manager.consolidate(conversation_id)
|
||||
|
||||
return {
|
||||
"conversation_id": conversation_id,
|
||||
"consolidated_turns": count,
|
||||
"message": f"Successfully consolidated {count} turns to persistent storage"
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Error consolidating conversation: {str(e)}"
|
||||
)
|
||||
|
||||
return router
|
||||
|
||||
|
||||
# Create controller instance
|
||||
ai_controller = AIController()
|
||||
@@ -0,0 +1,89 @@
|
||||
"""
|
||||
Health Controller
|
||||
|
||||
Provides service health and information endpoints
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from src.controllers.base import BaseController
|
||||
from src.config import get_settings
|
||||
from src.logging_config import get_logger
|
||||
from src.models.ollama_client import get_ollama_client
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class HealthController(BaseController):
|
||||
"""
|
||||
Controller for service health and information
|
||||
|
||||
Provides endpoints for:
|
||||
- Service information and status
|
||||
- Health checks
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(prefix="", tags=["Health"])
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the router"""
|
||||
router = APIRouter(tags=self.tags)
|
||||
settings = get_settings()
|
||||
|
||||
@router.get(
|
||||
"/",
|
||||
summary="Service information",
|
||||
response_class=JSONResponse
|
||||
)
|
||||
async def root():
|
||||
"""
|
||||
Get service information and health status
|
||||
|
||||
Returns basic information about the API service and available endpoints.
|
||||
"""
|
||||
logger.debug("Root endpoint accessed")
|
||||
return {
|
||||
"service": settings.app_name,
|
||||
"version": settings.app_version,
|
||||
"status": "healthy",
|
||||
"documentation": {
|
||||
"swagger_ui": "/docs",
|
||||
"redoc": "/redoc",
|
||||
"openapi_spec": "/openapi.json"
|
||||
},
|
||||
"endpoints": {
|
||||
"chat_completions": "/v1/chat/completions",
|
||||
"models": "/v1/models",
|
||||
"conversations": "/v1/conversations",
|
||||
"web_scraper": "/web-scraper/scrape",
|
||||
"infrastructure": "/infrastructure",
|
||||
"health": "/health"
|
||||
}
|
||||
}
|
||||
|
||||
@router.get(
|
||||
"/health",
|
||||
summary="Health check",
|
||||
response_class=JSONResponse
|
||||
)
|
||||
async def health_check():
|
||||
"""
|
||||
Simple health check endpoint for container orchestration
|
||||
|
||||
Returns a 200 OK status when the service is running properly.
|
||||
Used by Docker, Kubernetes, and load balancers.
|
||||
"""
|
||||
ollama_client = get_ollama_client()
|
||||
ollama_healthy = await ollama_client.health_check()
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"ollama_connected": ollama_healthy
|
||||
}
|
||||
|
||||
return router
|
||||
|
||||
|
||||
# Create controller instance
|
||||
health_controller = HealthController()
|
||||
@@ -4,14 +4,17 @@ Infrastructure Management Controller
|
||||
Provides API endpoints for automated infrastructure management,
|
||||
including service deployment, configuration, and monitoring setup.
|
||||
"""
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi import APIRouter, HTTPException, Depends
|
||||
from typing import List, Dict, Any, Optional, Union
|
||||
from pydantic import BaseModel, field_validator
|
||||
|
||||
from src.controllers.base import BaseController
|
||||
from src.clients.portainer_client import get_portainer_client
|
||||
from src.clients.npm_client import get_npm_client
|
||||
from src.clients.kuma_client import get_kuma_client
|
||||
from src.logging_config import get_logger
|
||||
from src import service_groups
|
||||
from src.auth.oidc import get_admin_user
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -25,6 +28,9 @@ class ServiceInfo(BaseModel):
|
||||
endpoint_id: Optional[int]
|
||||
ports: List[int] = []
|
||||
domains: List[str] = []
|
||||
running: bool = False # True if at least one container is running
|
||||
containers_running: int = 0 # Number of running containers
|
||||
containers_total: int = 0 # Total number of containers
|
||||
|
||||
@field_validator('status', mode='before')
|
||||
@classmethod
|
||||
@@ -40,7 +46,12 @@ class PortInfo(BaseModel):
|
||||
"""Information about an allocated port"""
|
||||
port: int
|
||||
service: str
|
||||
container_name: Optional[str] = None
|
||||
protocol: str = "tcp"
|
||||
internal_hostname: Optional[str] = None
|
||||
internal_ip: Optional[str] = None
|
||||
external_domains: List[str] = []
|
||||
host_port: Optional[int] = None # Port exposed on host, if different from internal
|
||||
description: str = ""
|
||||
|
||||
|
||||
@@ -158,17 +169,19 @@ class InfrastructureController(BaseController):
|
||||
@router.get(
|
||||
"/services",
|
||||
response_model=List[ServiceInfo],
|
||||
summary="List all deployed services"
|
||||
summary="List all deployed services (Docker Compose stacks)",
|
||||
description="List all services deployed via Portainer stacks. Each 'service' represents a Docker Compose stack."
|
||||
)
|
||||
async def list_services():
|
||||
"""
|
||||
List all deployed services from Portainer stacks
|
||||
List all deployed Docker Compose stacks from Portainer
|
||||
|
||||
Returns comprehensive service information including:
|
||||
- Stack/service name
|
||||
- Status
|
||||
Returns comprehensive stack/service information including:
|
||||
- Stack name (the service name)
|
||||
- Status (active/inactive)
|
||||
- Exposed ports
|
||||
- Configured domains
|
||||
- Configured domains (from NPM reverse proxy)
|
||||
- Running container count
|
||||
"""
|
||||
portainer = get_portainer_client()
|
||||
npm = get_npm_client()
|
||||
@@ -189,18 +202,38 @@ class InfrastructureController(BaseController):
|
||||
for stack in stacks:
|
||||
# Find domains for this stack
|
||||
stack_name = stack.get("Name", "")
|
||||
endpoint_id = stack.get("EndpointId")
|
||||
domains = [
|
||||
domain for domain, host in domain_map.items()
|
||||
if stack_name in host or host in stack_name
|
||||
]
|
||||
|
||||
# Get container status for this stack
|
||||
containers_running = 0
|
||||
containers_total = 0
|
||||
try:
|
||||
all_containers = await portainer.get_containers(endpoint_id, all_containers=True)
|
||||
for container in all_containers:
|
||||
labels = container.get("Labels", {})
|
||||
container_stack = labels.get("com.docker.compose.project", "")
|
||||
|
||||
if container_stack.lower() == stack_name.lower():
|
||||
containers_total += 1
|
||||
if container.get("State", "") == "running":
|
||||
containers_running += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get container status for {stack_name}: {e}")
|
||||
|
||||
service_info = ServiceInfo(
|
||||
name=stack_name,
|
||||
stack_id=stack.get("Id"),
|
||||
status=stack.get("Status", "unknown"),
|
||||
endpoint_id=stack.get("EndpointId"),
|
||||
endpoint_id=endpoint_id,
|
||||
ports=[], # TODO: Extract from stack file
|
||||
domains=domains
|
||||
domains=domains,
|
||||
running=containers_running > 0,
|
||||
containers_running=containers_running,
|
||||
containers_total=containers_total
|
||||
)
|
||||
services.append(service_info)
|
||||
|
||||
@@ -263,12 +296,133 @@ class InfrastructureController(BaseController):
|
||||
"""
|
||||
List all currently allocated ports
|
||||
|
||||
Scans services and proxy configurations to build
|
||||
a comprehensive port allocation map.
|
||||
Scans all running containers to extract:
|
||||
- Internal and external port mappings
|
||||
- Container internal hostnames and IPs
|
||||
- External domain names (from NPM proxy configuration)
|
||||
"""
|
||||
# TODO: Implement port scanning from containers and proxy configs
|
||||
# For now, return a placeholder
|
||||
return []
|
||||
portainer = get_portainer_client()
|
||||
npm = get_npm_client()
|
||||
|
||||
try:
|
||||
# Get all endpoints (Docker environments)
|
||||
endpoints = await portainer.get_endpoints()
|
||||
|
||||
# Get proxy hosts for external domain mapping
|
||||
proxy_hosts = await npm.get_proxy_hosts()
|
||||
|
||||
# Build mapping of forward_host:forward_port -> domains
|
||||
port_domain_map = {}
|
||||
for proxy in proxy_hosts:
|
||||
forward_host = proxy.get("forward_host", "")
|
||||
forward_port = proxy.get("forward_port", 0)
|
||||
domains = proxy.get("domain_names", [])
|
||||
key = f"{forward_host}:{forward_port}"
|
||||
if key not in port_domain_map:
|
||||
port_domain_map[key] = []
|
||||
port_domain_map[key].extend(domains)
|
||||
|
||||
ports = []
|
||||
|
||||
# Scan containers on each endpoint
|
||||
for endpoint in endpoints:
|
||||
endpoint_id = endpoint.get("Id")
|
||||
|
||||
try:
|
||||
containers = await portainer.get_containers(endpoint_id, all_containers=False)
|
||||
|
||||
for container in containers:
|
||||
container_name = container.get("Names", ["unknown"])[0].lstrip("/")
|
||||
state = container.get("State", "")
|
||||
|
||||
# Skip non-running containers
|
||||
if state != "running":
|
||||
continue
|
||||
|
||||
# Extract network information
|
||||
networks = container.get("NetworkSettings", {}).get("Networks", {})
|
||||
internal_hostname = container_name
|
||||
internal_ip = None
|
||||
|
||||
# Get first network IP
|
||||
for network_name, network_info in networks.items():
|
||||
if network_info.get("IPAddress"):
|
||||
internal_ip = network_info.get("IPAddress")
|
||||
break
|
||||
|
||||
# Extract port mappings
|
||||
port_mappings = container.get("Ports", [])
|
||||
|
||||
for port_mapping in port_mappings:
|
||||
internal_port = port_mapping.get("PrivatePort")
|
||||
host_port = port_mapping.get("PublicPort")
|
||||
protocol = port_mapping.get("Type", "tcp")
|
||||
|
||||
if not internal_port:
|
||||
continue
|
||||
|
||||
# Find external domains for this port
|
||||
external_domains = []
|
||||
|
||||
# Try matching by container name and port
|
||||
key_by_name = f"{container_name}:{internal_port}"
|
||||
if key_by_name in port_domain_map:
|
||||
external_domains.extend(port_domain_map[key_by_name])
|
||||
|
||||
# Try matching by internal IP and port
|
||||
if internal_ip:
|
||||
key_by_ip = f"{internal_ip}:{internal_port}"
|
||||
if key_by_ip in port_domain_map:
|
||||
external_domains.extend(port_domain_map[key_by_ip])
|
||||
|
||||
# Try matching by localhost and host port
|
||||
if host_port:
|
||||
for localhost_variant in ["localhost", "127.0.0.1", "192.168.86.149"]:
|
||||
key_by_host = f"{localhost_variant}:{host_port}"
|
||||
if key_by_host in port_domain_map:
|
||||
external_domains.extend(port_domain_map[key_by_host])
|
||||
|
||||
# Deduplicate external domains
|
||||
external_domains = list(set(external_domains))
|
||||
|
||||
# Get service name from stack label or container name
|
||||
labels = container.get("Labels", {})
|
||||
service_name = labels.get("com.docker.compose.service", container_name)
|
||||
|
||||
port_info = PortInfo(
|
||||
port=internal_port,
|
||||
service=service_name,
|
||||
container_name=container_name,
|
||||
protocol=protocol,
|
||||
internal_hostname=internal_hostname,
|
||||
internal_ip=internal_ip,
|
||||
external_domains=external_domains,
|
||||
host_port=host_port,
|
||||
description=f"{container_name} on {endpoint.get('Name', 'unknown')}"
|
||||
)
|
||||
ports.append(port_info)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to scan containers on endpoint {endpoint_id}: {e}")
|
||||
continue
|
||||
|
||||
# Deduplicate ports based on (port, container_name, protocol)
|
||||
seen = set()
|
||||
unique_ports = []
|
||||
for port_info in ports:
|
||||
key = (port_info.port, port_info.container_name, port_info.protocol)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
unique_ports.append(port_info)
|
||||
|
||||
# Sort by port number
|
||||
unique_ports.sort(key=lambda p: p.port)
|
||||
|
||||
return unique_ports
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to list ports: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@router.get(
|
||||
"/domains",
|
||||
@@ -311,18 +465,32 @@ class InfrastructureController(BaseController):
|
||||
@router.post(
|
||||
"/services",
|
||||
response_model=OperationResult,
|
||||
summary="Deploy a new service",
|
||||
summary="Deploy a new Docker Compose stack",
|
||||
description="Deploy a new service by creating a Docker Compose stack in Portainer. Provide the stack name and compose file content. Requires admin authentication.",
|
||||
status_code=201
|
||||
)
|
||||
async def deploy_service(request: DeployServiceRequest):
|
||||
async def deploy_service(
|
||||
request: DeployServiceRequest,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Deploy a new service via Portainer stack
|
||||
Deploy a new Docker Compose stack via Portainer
|
||||
|
||||
Creates a new Portainer stack from the provided Docker Compose content.
|
||||
This is equivalent to deploying a stack through the Portainer UI.
|
||||
|
||||
Args:
|
||||
request: Service deployment configuration
|
||||
request: Stack deployment configuration containing:
|
||||
- name: Stack name (must be unique)
|
||||
- compose_content: Full docker-compose.yml content as string
|
||||
- endpoint_id: Portainer endpoint ID (default: 3 for local)
|
||||
|
||||
Returns:
|
||||
Operation result with stack details
|
||||
Operation result with created stack details including stack ID
|
||||
|
||||
Raises:
|
||||
409: If a stack with the same name already exists
|
||||
500: If deployment fails
|
||||
"""
|
||||
portainer = get_portainer_client()
|
||||
|
||||
@@ -360,11 +528,19 @@ class InfrastructureController(BaseController):
|
||||
@router.put(
|
||||
"/services/{name}",
|
||||
response_model=OperationResult,
|
||||
summary="Update an existing service"
|
||||
summary="Update an existing Docker Compose stack",
|
||||
description="Update a deployed stack's Docker Compose configuration. This redeploys the stack with the new configuration. Requires admin authentication."
|
||||
)
|
||||
async def update_service(name: str, request: UpdateServiceRequest):
|
||||
async def update_service(
|
||||
name: str,
|
||||
request: UpdateServiceRequest,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Update an existing service's compose configuration
|
||||
Update an existing Docker Compose stack's configuration
|
||||
|
||||
Updates the stack's compose file and redeploys it. This is equivalent
|
||||
to updating a stack through the Portainer UI.
|
||||
|
||||
Args:
|
||||
name: Service/stack name
|
||||
@@ -415,9 +591,13 @@ class InfrastructureController(BaseController):
|
||||
@router.delete(
|
||||
"/services/{name}",
|
||||
response_model=OperationResult,
|
||||
summary="Delete a service"
|
||||
summary="Delete a service",
|
||||
description="Delete a service and remove its stack. Requires admin authentication."
|
||||
)
|
||||
async def delete_service(name: str):
|
||||
async def delete_service(
|
||||
name: str,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Delete a service and remove its stack
|
||||
|
||||
@@ -463,13 +643,40 @@ class InfrastructureController(BaseController):
|
||||
logger.error(f"Failed to delete service '{name}': {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@router.get(
|
||||
"/proxy/{proxy_id}",
|
||||
summary="Get proxy host details"
|
||||
)
|
||||
async def get_proxy_host(proxy_id: int):
|
||||
"""
|
||||
Get detailed configuration of a specific proxy host
|
||||
|
||||
Args:
|
||||
proxy_id: NPM proxy host ID
|
||||
|
||||
Returns:
|
||||
Complete proxy host configuration including locations
|
||||
"""
|
||||
npm = get_npm_client()
|
||||
|
||||
try:
|
||||
proxy_host = await npm.get_proxy_host(proxy_id)
|
||||
return proxy_host
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get proxy host {proxy_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@router.post(
|
||||
"/proxy",
|
||||
response_model=OperationResult,
|
||||
summary="Create a new proxy host",
|
||||
description="Create a new Nginx Proxy Manager proxy host with optional SSL certificate. Requires admin authentication.",
|
||||
status_code=201
|
||||
)
|
||||
async def create_proxy(request: CreateProxyRequest):
|
||||
async def create_proxy(
|
||||
request: CreateProxyRequest,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Create a new Nginx Proxy Manager proxy host
|
||||
|
||||
@@ -523,6 +730,410 @@ class InfrastructureController(BaseController):
|
||||
logger.error(f"Failed to create proxy host: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
# Service Control Endpoints
|
||||
@router.get(
|
||||
"/service-groups",
|
||||
summary="List service groups"
|
||||
)
|
||||
async def list_service_groups():
|
||||
"""
|
||||
List all defined service groups
|
||||
|
||||
Returns service groups with their member services and status.
|
||||
"""
|
||||
return {
|
||||
"groups": service_groups.list_service_groups(),
|
||||
"always_on": list(service_groups.ALWAYS_ON_SERVICES),
|
||||
"stoppable": service_groups.list_stoppable_services()
|
||||
}
|
||||
|
||||
@router.post(
|
||||
"/services/{name}/stop",
|
||||
response_model=OperationResult,
|
||||
summary="Stop a service or service group",
|
||||
description="Stop a service or service group by pausing monitors and stopping containers. Requires admin authentication."
|
||||
)
|
||||
async def stop_service(
|
||||
name: str,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Stop a service or service group
|
||||
|
||||
This will:
|
||||
1. Validate service can be stopped (not always-on)
|
||||
2. Pause Uptime Kuma monitors for all services in group
|
||||
3. Stop the Portainer stack(s)
|
||||
|
||||
Args:
|
||||
name: Service or group name
|
||||
|
||||
Returns:
|
||||
Operation result with details
|
||||
"""
|
||||
portainer = get_portainer_client()
|
||||
kuma = get_kuma_client()
|
||||
|
||||
try:
|
||||
# Get all services in the group
|
||||
services = service_groups.get_service_group(name)
|
||||
|
||||
# Validate none are always-on
|
||||
is_valid, error_msg = service_groups.validate_stop_request(services)
|
||||
if not is_valid:
|
||||
raise HTTPException(status_code=403, detail=error_msg)
|
||||
|
||||
results = {
|
||||
"stopped_services": [],
|
||||
"paused_monitors": [],
|
||||
"errors": []
|
||||
}
|
||||
|
||||
# Stop each service
|
||||
for service_name in services:
|
||||
try:
|
||||
# 1. Pause Uptime Kuma monitor
|
||||
try:
|
||||
monitor_paused = await kuma.pause_monitor_by_name(service_name)
|
||||
if monitor_paused:
|
||||
results["paused_monitors"].append(service_name)
|
||||
logger.info(f"Paused Kuma monitor for {service_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to pause Kuma monitor for {service_name}: {e}")
|
||||
results["errors"].append(f"Kuma pause failed for {service_name}: {str(e)}")
|
||||
|
||||
# 2. Stop Portainer stack
|
||||
stacks = await portainer.get_stacks()
|
||||
stack = next(
|
||||
(s for s in stacks if s.get("Name", "").lower() == service_name.lower()),
|
||||
None
|
||||
)
|
||||
|
||||
if stack:
|
||||
stack_id = stack.get("Id")
|
||||
endpoint_id = stack.get("EndpointId")
|
||||
|
||||
# Stop stack by deleting it (Portainer doesn't have a "stop" operation)
|
||||
# Note: This is destructive. For a gentler approach, we'd need to use docker compose stop
|
||||
# Let's use docker API instead
|
||||
logger.info(f"Stopping containers for stack: {service_name}")
|
||||
|
||||
# Get containers for this stack
|
||||
containers = await portainer.get_containers(endpoint_id, all_containers=False)
|
||||
stopped_containers = []
|
||||
|
||||
for container in containers:
|
||||
labels = container.get("Labels", {})
|
||||
container_stack = labels.get("com.docker.compose.project", "")
|
||||
|
||||
if container_stack.lower() == service_name.lower():
|
||||
container_id = container.get("Id")
|
||||
# Stop container via Portainer Docker API
|
||||
await portainer.stop_container(endpoint_id, container_id)
|
||||
stopped_containers.append(container.get("Names", ["unknown"])[0])
|
||||
|
||||
results["stopped_services"].append({
|
||||
"service": service_name,
|
||||
"stack_id": stack_id,
|
||||
"containers": stopped_containers
|
||||
})
|
||||
logger.info(f"Stopped service: {service_name}")
|
||||
else:
|
||||
results["errors"].append(f"Stack not found: {service_name}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to stop service {service_name}: {e}")
|
||||
results["errors"].append(f"{service_name}: {str(e)}")
|
||||
|
||||
success = len(results["stopped_services"]) > 0
|
||||
message = f"Stopped {len(results['stopped_services'])} service(s)"
|
||||
if results["errors"]:
|
||||
message += f" with {len(results['errors'])} error(s)"
|
||||
|
||||
return OperationResult(
|
||||
success=success,
|
||||
message=message,
|
||||
details=results
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to stop service group '{name}': {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@router.post(
|
||||
"/services/{name}/start",
|
||||
response_model=OperationResult,
|
||||
summary="Start a service or service group",
|
||||
description="Start a service or service group by starting containers and resuming monitors. Requires admin authentication."
|
||||
)
|
||||
async def start_service(
|
||||
name: str,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Start a service or service group
|
||||
|
||||
This will:
|
||||
1. Start the Portainer stack(s)
|
||||
2. Resume Uptime Kuma monitors for all services in group
|
||||
|
||||
Args:
|
||||
name: Service or group name
|
||||
|
||||
Returns:
|
||||
Operation result with details
|
||||
"""
|
||||
portainer = get_portainer_client()
|
||||
kuma = get_kuma_client()
|
||||
|
||||
try:
|
||||
# Get all services in the group
|
||||
services = service_groups.get_service_group(name)
|
||||
|
||||
results = {
|
||||
"started_services": [],
|
||||
"resumed_monitors": [],
|
||||
"errors": []
|
||||
}
|
||||
|
||||
# Start each service
|
||||
for service_name in services:
|
||||
try:
|
||||
# 1. Start Portainer stack (start containers)
|
||||
stacks = await portainer.get_stacks()
|
||||
stack = next(
|
||||
(s for s in stacks if s.get("Name", "").lower() == service_name.lower()),
|
||||
None
|
||||
)
|
||||
|
||||
if stack:
|
||||
stack_id = stack.get("Id")
|
||||
endpoint_id = stack.get("EndpointId")
|
||||
|
||||
logger.info(f"Starting containers for stack: {service_name}")
|
||||
|
||||
# Get containers for this stack
|
||||
containers = await portainer.get_containers(endpoint_id, all_containers=True)
|
||||
started_containers = []
|
||||
|
||||
for container in containers:
|
||||
labels = container.get("Labels", {})
|
||||
container_stack = labels.get("com.docker.compose.project", "")
|
||||
|
||||
if container_stack.lower() == service_name.lower():
|
||||
container_id = container.get("Id")
|
||||
# Start container via Portainer Docker API
|
||||
await portainer.start_container(endpoint_id, container_id)
|
||||
started_containers.append(container.get("Names", ["unknown"])[0])
|
||||
|
||||
results["started_services"].append({
|
||||
"service": service_name,
|
||||
"stack_id": stack_id,
|
||||
"containers": started_containers
|
||||
})
|
||||
logger.info(f"Started service: {service_name}")
|
||||
|
||||
# 2. Resume Uptime Kuma monitor
|
||||
try:
|
||||
monitor_resumed = await kuma.resume_monitor_by_name(service_name)
|
||||
if monitor_resumed:
|
||||
results["resumed_monitors"].append(service_name)
|
||||
logger.info(f"Resumed Kuma monitor for {service_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to resume Kuma monitor for {service_name}: {e}")
|
||||
results["errors"].append(f"Kuma resume failed for {service_name}: {str(e)}")
|
||||
|
||||
else:
|
||||
results["errors"].append(f"Stack not found: {service_name}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start service {service_name}: {e}")
|
||||
results["errors"].append(f"{service_name}: {str(e)}")
|
||||
|
||||
success = len(results["started_services"]) > 0
|
||||
message = f"Started {len(results['started_services'])} service(s)"
|
||||
if results["errors"]:
|
||||
message += f" with {len(results['errors'])} error(s)"
|
||||
|
||||
return OperationResult(
|
||||
success=success,
|
||||
message=message,
|
||||
details=results
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start service group '{name}': {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
# ===== Monitoring Endpoints =====
|
||||
|
||||
@router.get(
|
||||
"/monitors",
|
||||
summary="List all monitors",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def list_monitors():
|
||||
"""
|
||||
List all Uptime Kuma monitors
|
||||
|
||||
Returns:
|
||||
List of monitors with their configurations
|
||||
"""
|
||||
try:
|
||||
kuma = get_kuma_client()
|
||||
monitors = await kuma.get_monitors()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"monitors": monitors,
|
||||
"total": len(monitors)
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to list monitors: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to list monitors: {str(e)}")
|
||||
|
||||
@router.post(
|
||||
"/monitors",
|
||||
summary="Create a new monitor",
|
||||
description="Create a new Uptime Kuma monitor. Requires admin authentication.",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def create_monitor(
|
||||
monitor_config: Dict[str, Any],
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Create a new Uptime Kuma monitor
|
||||
|
||||
Args:
|
||||
monitor_config: Monitor configuration (name, type, hostname, port, etc.)
|
||||
|
||||
Returns:
|
||||
Created monitor details including ID
|
||||
"""
|
||||
try:
|
||||
kuma = get_kuma_client()
|
||||
created_monitor = await kuma.add_monitor(monitor_config)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Monitor '{monitor_config.get('name')}' created successfully",
|
||||
"monitor": created_monitor
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to create monitor: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to create monitor: {str(e)}")
|
||||
|
||||
@router.get(
|
||||
"/monitors/{monitor_id}",
|
||||
summary="Get monitor details",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def get_monitor(monitor_id: int):
|
||||
"""
|
||||
Get details of a specific monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
Monitor configuration and status
|
||||
"""
|
||||
try:
|
||||
kuma = get_kuma_client()
|
||||
monitor = await kuma.get_monitor(monitor_id)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"monitor": monitor
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get monitor {monitor_id}: {e}")
|
||||
raise HTTPException(status_code=404, detail=f"Monitor {monitor_id} not found: {str(e)}")
|
||||
|
||||
@router.put(
|
||||
"/monitors/{monitor_id}",
|
||||
summary="Update a monitor",
|
||||
description="Update an existing Uptime Kuma monitor. Requires admin authentication.",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def update_monitor(
|
||||
monitor_id: int,
|
||||
updates: Dict[str, Any],
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Update an existing monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
updates: Fields to update
|
||||
|
||||
Returns:
|
||||
Updated monitor details
|
||||
"""
|
||||
try:
|
||||
kuma = get_kuma_client()
|
||||
|
||||
# Get existing monitor
|
||||
existing = await kuma.get_monitor(monitor_id)
|
||||
|
||||
# Merge updates
|
||||
monitor_config = existing.copy()
|
||||
monitor_config.update(updates)
|
||||
|
||||
# Update monitor
|
||||
updated_monitor = await kuma.update_monitor(monitor_id, monitor_config)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Monitor {monitor_id} updated successfully",
|
||||
"monitor": updated_monitor
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to update monitor {monitor_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to update monitor: {str(e)}")
|
||||
|
||||
@router.delete(
|
||||
"/monitors/{monitor_id}",
|
||||
summary="Delete a monitor",
|
||||
description="Delete an Uptime Kuma monitor. Requires admin authentication.",
|
||||
response_model=Dict[str, Any]
|
||||
)
|
||||
async def delete_monitor(
|
||||
monitor_id: int,
|
||||
user: Dict = Depends(get_admin_user)
|
||||
):
|
||||
"""
|
||||
Delete a monitor
|
||||
|
||||
Args:
|
||||
monitor_id: Monitor identifier
|
||||
|
||||
Returns:
|
||||
Success confirmation
|
||||
"""
|
||||
try:
|
||||
kuma = get_kuma_client()
|
||||
await kuma.delete_monitor(monitor_id)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Monitor {monitor_id} deleted successfully"
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete monitor {monitor_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to delete monitor: {str(e)}")
|
||||
|
||||
return router
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""
|
||||
Static Files Controller
|
||||
|
||||
Serves static files for widgets and other frontend assets.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
from fastapi.responses import FileResponse, HTMLResponse
|
||||
from pathlib import Path
|
||||
import os
|
||||
|
||||
from src.controllers.base import BaseController
|
||||
from src.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class StaticController(BaseController):
|
||||
"""
|
||||
Controller for serving static files
|
||||
|
||||
Provides endpoints for:
|
||||
- Organizr widgets
|
||||
- Other static assets
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(prefix="/static", tags=["Static"])
|
||||
self.static_dir = Path(__file__).parent.parent.parent / "static"
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the router"""
|
||||
router = APIRouter(prefix=self.prefix, tags=self.tags)
|
||||
|
||||
@router.get(
|
||||
"/widgets/{filename}",
|
||||
response_class=HTMLResponse,
|
||||
summary="Get widget file"
|
||||
)
|
||||
async def get_widget(filename: str):
|
||||
"""
|
||||
Serve widget HTML files
|
||||
|
||||
Args:
|
||||
filename: Widget filename (e.g., service-control.html)
|
||||
|
||||
Returns:
|
||||
HTML file content
|
||||
"""
|
||||
widget_path = self.static_dir / "widgets" / filename
|
||||
|
||||
if not widget_path.exists():
|
||||
return HTMLResponse(
|
||||
content=f"<h1>404 - Widget not found</h1><p>{filename}</p>",
|
||||
status_code=404
|
||||
)
|
||||
|
||||
if not widget_path.is_file():
|
||||
return HTMLResponse(
|
||||
content=f"<h1>400 - Not a file</h1>",
|
||||
status_code=400
|
||||
)
|
||||
|
||||
# Security: Ensure the path is within the static directory
|
||||
try:
|
||||
widget_path.resolve().relative_to(self.static_dir.resolve())
|
||||
except ValueError:
|
||||
return HTMLResponse(
|
||||
content=f"<h1>403 - Forbidden</h1>",
|
||||
status_code=403
|
||||
)
|
||||
|
||||
logger.info(f"Serving widget: {filename}")
|
||||
return FileResponse(
|
||||
widget_path,
|
||||
media_type="text/html",
|
||||
headers={
|
||||
"Cache-Control": "no-cache, no-store, must-revalidate",
|
||||
"Pragma": "no-cache",
|
||||
"Expires": "0"
|
||||
}
|
||||
)
|
||||
|
||||
@router.get(
|
||||
"/widgets",
|
||||
summary="List available widgets"
|
||||
)
|
||||
async def list_widgets():
|
||||
"""
|
||||
List all available widget files
|
||||
|
||||
Returns:
|
||||
List of widget filenames
|
||||
"""
|
||||
widgets_dir = self.static_dir / "widgets"
|
||||
|
||||
if not widgets_dir.exists():
|
||||
return {"widgets": [], "message": "Widgets directory not found"}
|
||||
|
||||
widgets = []
|
||||
for file in widgets_dir.glob("*.html"):
|
||||
widgets.append({
|
||||
"name": file.name,
|
||||
"url": f"/static/widgets/{file.name}",
|
||||
"size": file.stat().st_size
|
||||
})
|
||||
|
||||
return {
|
||||
"widgets": widgets,
|
||||
"count": len(widgets)
|
||||
}
|
||||
|
||||
return router
|
||||
|
||||
|
||||
# Create controller instance
|
||||
static_controller = StaticController()
|
||||
@@ -0,0 +1,98 @@
|
||||
"""
|
||||
Tools Controller
|
||||
|
||||
Provides utility tool endpoints including:
|
||||
- Web scraping and content extraction
|
||||
"""
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
|
||||
from src.controllers.base import BaseController
|
||||
from src.logging_config import get_logger
|
||||
from src.web_scraper.schemas import WebScraperRequest, WebScraperResponse
|
||||
from src.web_scraper.service import WebScraperService
|
||||
from src.web_scraper.exceptions import FetchError, ScrapingError
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class ToolsController(BaseController):
|
||||
"""
|
||||
Controller for utility tools
|
||||
|
||||
Provides endpoints for:
|
||||
- Web scraping and content extraction
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(prefix="/web-scraper", tags=["Tools"])
|
||||
# Initialize service (could be dependency injected for testing)
|
||||
self.scraper_service = WebScraperService()
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the router"""
|
||||
router = APIRouter(prefix=self.prefix, tags=self.tags)
|
||||
|
||||
@router.post(
|
||||
"/scrape",
|
||||
response_model=WebScraperResponse,
|
||||
status_code=status.HTTP_200_OK,
|
||||
summary="Scrape website content",
|
||||
description="""
|
||||
Scrape and extract main content from a website.
|
||||
|
||||
Uses trafilatura for intelligent content extraction (articles, blog posts, documentation),
|
||||
with BeautifulSoup as fallback. Perfect for feeding webpage content to LLMs.
|
||||
|
||||
**Features:**
|
||||
- Intelligent main content extraction
|
||||
- Removes navigation, ads, footers
|
||||
- Optional link extraction
|
||||
- Configurable content length limits
|
||||
|
||||
**Rate Limiting:** None (internal network use only)
|
||||
"""
|
||||
)
|
||||
async def scrape_website(request: WebScraperRequest) -> WebScraperResponse:
|
||||
"""
|
||||
Scrape a website and extract its main content
|
||||
|
||||
Args:
|
||||
request: Scraping request with URL and options
|
||||
|
||||
Returns:
|
||||
Extracted content with metadata
|
||||
|
||||
Raises:
|
||||
HTTPException: 400 for fetch errors, 500 for processing errors
|
||||
"""
|
||||
try:
|
||||
logger.info(f"Received scrape request for: {request.url}")
|
||||
result = await self.scraper_service.scrape_url(request)
|
||||
return result
|
||||
|
||||
except FetchError as e:
|
||||
logger.warning(f"Fetch failed: {str(e)}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail=f"Failed to fetch URL: {str(e)}"
|
||||
)
|
||||
|
||||
except ScrapingError as e:
|
||||
logger.error(f"Scraping failed: {str(e)}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Failed to extract content: {str(e)}"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error: {str(e)}", exc_info=True)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="An unexpected error occurred"
|
||||
)
|
||||
|
||||
return router
|
||||
|
||||
|
||||
# Create controller instance
|
||||
tools_controller = ToolsController()
|
||||
@@ -8,12 +8,13 @@ from contextlib import asynccontextmanager
|
||||
|
||||
from src.config import get_settings
|
||||
from src.logging_config import setup_logging, get_logger
|
||||
from src.web_scraper import router as web_scraper_router
|
||||
from src.api.v1.chat import router as chat_router
|
||||
from src.api.v1.models import router as models_router
|
||||
from src.api.v1.conversations import router as conversations_router
|
||||
from src.models.ollama_client import get_ollama_client, close_ollama_client
|
||||
from src.controllers.infrastructure_controller import infrastructure_controller
|
||||
from src.controllers.ai_controller import ai_controller
|
||||
from src.controllers.tools_controller import tools_controller
|
||||
from src.controllers.health_controller import health_controller
|
||||
from src.controllers.static_controller import static_controller
|
||||
from src.security import initialize_oidc
|
||||
|
||||
# Initialize settings
|
||||
settings = get_settings()
|
||||
@@ -47,6 +48,9 @@ async def lifespan(app: FastAPI):
|
||||
else:
|
||||
logger.warning("✗ Ollama connection failed - AI features may not work")
|
||||
|
||||
# Initialize security (OIDC authentication)
|
||||
initialize_oidc(settings)
|
||||
|
||||
yield
|
||||
|
||||
# Shutdown
|
||||
@@ -88,7 +92,7 @@ app = FastAPI(
|
||||
- `GET /infrastructure/ports` - List allocated ports
|
||||
- `GET /infrastructure/domains` - List configured domains
|
||||
|
||||
**Write Endpoints:**
|
||||
**Write Endpoints (Admin Only):**
|
||||
- `POST /infrastructure/services` - Deploy new service from compose YAML
|
||||
- `PUT /infrastructure/services/{name}` - Update existing service
|
||||
- `DELETE /infrastructure/services/{name}` - Remove service and stack
|
||||
@@ -100,6 +104,13 @@ app = FastAPI(
|
||||
Intelligent web scraping with main content extraction.
|
||||
Perfect for extracting articles, documentation, and blog posts for LLM consumption.
|
||||
|
||||
## Authentication
|
||||
|
||||
When OIDC authentication is enabled (oidc_enabled=true in config):
|
||||
- Infrastructure write endpoints require authentication
|
||||
- Use OAuth2/OIDC bearer token from Authentik
|
||||
- Admin group membership required for infrastructure operations
|
||||
|
||||
## Integration
|
||||
|
||||
This API is designed to integrate with:
|
||||
@@ -118,7 +129,11 @@ app = FastAPI(
|
||||
redoc_url="/redoc",
|
||||
openapi_url="/openapi.json",
|
||||
lifespan=lifespan,
|
||||
debug=settings.debug
|
||||
debug=settings.debug,
|
||||
swagger_ui_init_oauth={
|
||||
"clientId": settings.oidc_audience,
|
||||
"usePkceWithAuthorizationCodeGrant": True,
|
||||
} if settings.oidc_enabled else None
|
||||
)
|
||||
|
||||
# Add CORS middleware
|
||||
@@ -131,69 +146,12 @@ app.add_middleware(
|
||||
)
|
||||
|
||||
|
||||
# Root endpoint
|
||||
@app.get(
|
||||
"/",
|
||||
tags=["Health"],
|
||||
summary="Service information",
|
||||
response_class=JSONResponse
|
||||
)
|
||||
async def root():
|
||||
"""
|
||||
Get service information and health status
|
||||
|
||||
Returns basic information about the API service and available endpoints.
|
||||
"""
|
||||
logger.debug("Root endpoint accessed")
|
||||
return {
|
||||
"service": settings.app_name,
|
||||
"version": settings.app_version,
|
||||
"status": "healthy",
|
||||
"documentation": {
|
||||
"swagger_ui": "/docs",
|
||||
"redoc": "/redoc",
|
||||
"openapi_spec": "/openapi.json"
|
||||
},
|
||||
"endpoints": {
|
||||
"chat_completions": "/v1/chat/completions",
|
||||
"models": "/v1/models",
|
||||
"conversations": "/v1/conversations",
|
||||
"web_scraper": "/web-scraper/scrape",
|
||||
"infrastructure": "/infrastructure",
|
||||
"health": "/health"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
# Health check endpoint
|
||||
@app.get(
|
||||
"/health",
|
||||
tags=["Health"],
|
||||
summary="Health check",
|
||||
response_class=JSONResponse
|
||||
)
|
||||
async def health_check():
|
||||
"""
|
||||
Simple health check endpoint for container orchestration
|
||||
|
||||
Returns a 200 OK status when the service is running properly.
|
||||
Used by Docker, Kubernetes, and load balancers.
|
||||
"""
|
||||
ollama_client = get_ollama_client()
|
||||
ollama_healthy = await ollama_client.health_check()
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"ollama_connected": ollama_healthy
|
||||
}
|
||||
|
||||
|
||||
# Include routers
|
||||
app.include_router(chat_router) # /v1/chat/completions
|
||||
app.include_router(models_router) # /v1/models
|
||||
app.include_router(conversations_router) # /v1/conversations
|
||||
app.include_router(web_scraper_router) # /web-scraper/scrape
|
||||
app.include_router(infrastructure_controller.create_router()) # /infrastructure/*
|
||||
# Include controller routers
|
||||
app.include_router(health_controller.router) # / and /health
|
||||
app.include_router(ai_controller.router) # /v1/chat, /v1/models, /v1/conversations
|
||||
app.include_router(tools_controller.router) # /web-scraper/scrape
|
||||
app.include_router(infrastructure_controller.router) # /infrastructure/*
|
||||
app.include_router(static_controller.router) # /static/*
|
||||
|
||||
|
||||
# Global exception handler
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
"""
|
||||
Security initialization module
|
||||
|
||||
Handles OIDC configuration and authentication setup
|
||||
"""
|
||||
from src.config import Settings
|
||||
from src.auth.oidc import oidc_config
|
||||
from src.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def initialize_oidc(settings: Settings) -> None:
|
||||
"""
|
||||
Initialize OIDC authentication configuration
|
||||
|
||||
Configures the global oidc_config instance with settings from environment.
|
||||
If OIDC is enabled, logs the issuer URL for verification.
|
||||
|
||||
Args:
|
||||
settings: Application settings containing OIDC configuration
|
||||
"""
|
||||
oidc_config.configure(
|
||||
enabled=settings.oidc_enabled,
|
||||
issuer=settings.oidc_issuer,
|
||||
audience=settings.oidc_audience
|
||||
)
|
||||
|
||||
if settings.oidc_enabled:
|
||||
logger.info(f"✓ OIDC authentication enabled (issuer: {settings.oidc_issuer})")
|
||||
else:
|
||||
logger.info("○ OIDC authentication disabled - API is publicly accessible")
|
||||
@@ -0,0 +1,135 @@
|
||||
"""
|
||||
Service Groups and Safety Configuration
|
||||
|
||||
Defines service groups, dependencies, and always-on infrastructure services.
|
||||
"""
|
||||
from typing import List, Dict, Set
|
||||
|
||||
# Always-on infrastructure services (CANNOT be stopped via API)
|
||||
ALWAYS_ON_SERVICES: Set[str] = {
|
||||
"portainer",
|
||||
"nginx-proxy-manager",
|
||||
"core-api",
|
||||
"uptime-kuma",
|
||||
"organizr",
|
||||
"headscale",
|
||||
"watchtower",
|
||||
"netdata",
|
||||
"maintenance",
|
||||
}
|
||||
|
||||
# Service groups - services that should be started/stopped together
|
||||
SERVICE_GROUPS: Dict[str, List[str]] = {
|
||||
"jellyfin": [
|
||||
"jellyfin",
|
||||
],
|
||||
"nextcloud": [
|
||||
"nextcloud",
|
||||
"nextcloud-db",
|
||||
"nextcloud-redis",
|
||||
],
|
||||
"gitea": [
|
||||
"gitea",
|
||||
"gitea-db",
|
||||
],
|
||||
"ai-stack": [
|
||||
"open-webui",
|
||||
"ollama",
|
||||
"qdrant",
|
||||
],
|
||||
"samba": [
|
||||
"samba",
|
||||
],
|
||||
}
|
||||
|
||||
# Reverse mapping: service name -> group name
|
||||
SERVICE_TO_GROUP: Dict[str, str] = {}
|
||||
for group, services in SERVICE_GROUPS.items():
|
||||
for service in services:
|
||||
SERVICE_TO_GROUP[service] = group
|
||||
|
||||
|
||||
def is_always_on(service_name: str) -> bool:
|
||||
"""
|
||||
Check if a service is marked as always-on (infrastructure)
|
||||
|
||||
Args:
|
||||
service_name: Name of the service
|
||||
|
||||
Returns:
|
||||
True if service cannot be stopped, False otherwise
|
||||
"""
|
||||
return service_name.lower() in ALWAYS_ON_SERVICES
|
||||
|
||||
|
||||
def get_service_group(service_name: str) -> List[str]:
|
||||
"""
|
||||
Get all services in the same group as the given service
|
||||
|
||||
Args:
|
||||
service_name: Name of the service
|
||||
|
||||
Returns:
|
||||
List of service names in the group (including the service itself)
|
||||
Returns [service_name] if not part of a group
|
||||
"""
|
||||
group = SERVICE_TO_GROUP.get(service_name.lower())
|
||||
if group:
|
||||
return SERVICE_GROUPS[group].copy()
|
||||
return [service_name]
|
||||
|
||||
|
||||
def get_group_name(service_name: str) -> str:
|
||||
"""
|
||||
Get the group name for a service
|
||||
|
||||
Args:
|
||||
service_name: Name of the service
|
||||
|
||||
Returns:
|
||||
Group name or the service name if not in a group
|
||||
"""
|
||||
return SERVICE_TO_GROUP.get(service_name.lower(), service_name)
|
||||
|
||||
|
||||
def list_service_groups() -> Dict[str, List[str]]:
|
||||
"""
|
||||
Get all defined service groups
|
||||
|
||||
Returns:
|
||||
Dictionary of group names to service lists
|
||||
"""
|
||||
return SERVICE_GROUPS.copy()
|
||||
|
||||
|
||||
def list_stoppable_services() -> List[str]:
|
||||
"""
|
||||
Get list of all services that can be stopped
|
||||
|
||||
Returns:
|
||||
List of service names that are not always-on
|
||||
"""
|
||||
stoppable = []
|
||||
for services in SERVICE_GROUPS.values():
|
||||
stoppable.extend(services)
|
||||
|
||||
# Remove any always-on services (shouldn't be in groups, but safety check)
|
||||
return [s for s in stoppable if not is_always_on(s)]
|
||||
|
||||
|
||||
def validate_stop_request(service_names: List[str]) -> tuple[bool, str]:
|
||||
"""
|
||||
Validate that a list of services can be stopped
|
||||
|
||||
Args:
|
||||
service_names: List of service names to check
|
||||
|
||||
Returns:
|
||||
Tuple of (is_valid, error_message)
|
||||
error_message is empty string if valid
|
||||
"""
|
||||
for service in service_names:
|
||||
if is_always_on(service):
|
||||
return False, f"Cannot stop always-on service: {service}"
|
||||
|
||||
return True, ""
|
||||
@@ -0,0 +1,343 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Service Control</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
background: transparent;
|
||||
color: #e0e0e0;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
h2 {
|
||||
color: #fff;
|
||||
margin-bottom: 15px;
|
||||
font-size: 20px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.service-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
gap: 15px;
|
||||
}
|
||||
|
||||
.service-card {
|
||||
background: rgba(40, 40, 40, 0.95);
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
border-radius: 8px;
|
||||
padding: 15px;
|
||||
transition: all 0.3s ease;
|
||||
}
|
||||
|
||||
.service-card:hover {
|
||||
border-color: rgba(66, 153, 225, 0.5);
|
||||
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
.service-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.service-name {
|
||||
font-size: 16px;
|
||||
font-weight: 600;
|
||||
color: #fff;
|
||||
text-transform: capitalize;
|
||||
}
|
||||
|
||||
.status-badge {
|
||||
padding: 4px 12px;
|
||||
border-radius: 12px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.status-running {
|
||||
background: rgba(72, 187, 120, 0.2);
|
||||
color: #48bb78;
|
||||
border: 1px solid rgba(72, 187, 120, 0.4);
|
||||
}
|
||||
|
||||
.status-stopped {
|
||||
background: rgba(245, 101, 101, 0.2);
|
||||
color: #f56565;
|
||||
border: 1px solid rgba(245, 101, 101, 0.4);
|
||||
}
|
||||
|
||||
.status-loading {
|
||||
background: rgba(237, 137, 54, 0.2);
|
||||
color: #ed8936;
|
||||
border: 1px solid rgba(237, 137, 54, 0.4);
|
||||
}
|
||||
|
||||
.service-info {
|
||||
font-size: 13px;
|
||||
color: #a0a0a0;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.service-actions {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.btn {
|
||||
flex: 1;
|
||||
padding: 8px 12px;
|
||||
border: none;
|
||||
border-radius: 6px;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s ease;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
|
||||
.btn:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.btn-start {
|
||||
background: linear-gradient(135deg, #48bb78 0%, #38a169 100%);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.btn-start:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #38a169 0%, #2f855a 100%);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.btn-stop {
|
||||
background: linear-gradient(135deg, #f56565 0%, #e53e3e 100%);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.btn-stop:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #e53e3e 0%, #c53030 100%);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.btn-restart {
|
||||
background: linear-gradient(135deg, #4299e1 0%, #3182ce 100%);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.btn-restart:hover:not(:disabled) {
|
||||
background: linear-gradient(135deg, #3182ce 0%, #2c5282 100%);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.loading {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #a0a0a0;
|
||||
}
|
||||
|
||||
.error {
|
||||
background: rgba(245, 101, 101, 0.1);
|
||||
border: 1px solid rgba(245, 101, 101, 0.4);
|
||||
color: #f56565;
|
||||
padding: 12px;
|
||||
border-radius: 6px;
|
||||
margin-bottom: 15px;
|
||||
}
|
||||
|
||||
.always-on-badge {
|
||||
display: inline-block;
|
||||
padding: 2px 8px;
|
||||
background: rgba(66, 153, 225, 0.2);
|
||||
color: #4299e1;
|
||||
border: 1px solid rgba(66, 153, 225, 0.4);
|
||||
border-radius: 10px;
|
||||
font-size: 11px;
|
||||
margin-left: 8px;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
to { transform: rotate(360deg); }
|
||||
}
|
||||
|
||||
.spinner {
|
||||
display: inline-block;
|
||||
width: 14px;
|
||||
height: 14px;
|
||||
border: 2px solid rgba(255, 255, 255, 0.3);
|
||||
border-top-color: #fff;
|
||||
border-radius: 50%;
|
||||
animation: spin 0.6s linear infinite;
|
||||
margin-right: 6px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h2>🎛️ On-Demand Services</h2>
|
||||
<div id="error-container"></div>
|
||||
<div id="service-container" class="loading">Loading services...</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Auto-detect API base from current domain (works with NPM proxy)
|
||||
const API_BASE = window.location.origin;
|
||||
let services = [];
|
||||
let alwaysOnServices = [];
|
||||
|
||||
async function fetchServices() {
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/infrastructure/services`);
|
||||
if (!response.ok) throw new Error('Failed to fetch services');
|
||||
services = await response.json();
|
||||
|
||||
const groupsResponse = await fetch(`${API_BASE}/infrastructure/service-groups`);
|
||||
if (groupsResponse.ok) {
|
||||
const groupsData = await groupsResponse.json();
|
||||
alwaysOnServices = groupsData.always_on || [];
|
||||
}
|
||||
|
||||
renderServices();
|
||||
document.getElementById('error-container').innerHTML = '';
|
||||
} catch (error) {
|
||||
console.error('Error fetching services:', error);
|
||||
document.getElementById('error-container').innerHTML =
|
||||
`<div class="error">❌ Failed to connect to API: ${error.message}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
function isAlwaysOn(serviceName) {
|
||||
return alwaysOnServices.includes(serviceName.toLowerCase());
|
||||
}
|
||||
|
||||
function getServiceStatus(service) {
|
||||
if (service.containers_running > 0) {
|
||||
return {
|
||||
class: 'status-running',
|
||||
text: `Running (${service.containers_running}/${service.containers_total})`
|
||||
};
|
||||
} else if (service.containers_total > 0) {
|
||||
return {
|
||||
class: 'status-stopped',
|
||||
text: 'Stopped'
|
||||
};
|
||||
} else {
|
||||
return {
|
||||
class: 'status-stopped',
|
||||
text: 'No containers'
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function renderServices() {
|
||||
const container = document.getElementById('service-container');
|
||||
|
||||
// Filter to only show stoppable services
|
||||
const stoppableServices = services.filter(s => !isAlwaysOn(s.name));
|
||||
|
||||
if (stoppableServices.length === 0) {
|
||||
container.innerHTML = '<div class="loading">No stoppable services found</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
container.className = 'service-grid';
|
||||
container.innerHTML = stoppableServices.map(service => {
|
||||
const status = getServiceStatus(service);
|
||||
const isRunning = service.containers_running > 0;
|
||||
const alwaysOn = isAlwaysOn(service.name);
|
||||
|
||||
return `
|
||||
<div class="service-card" data-service="${service.name}">
|
||||
<div class="service-header">
|
||||
<span class="service-name">
|
||||
${service.name}
|
||||
${alwaysOn ? '<span class="always-on-badge">ALWAYS ON</span>' : ''}
|
||||
</span>
|
||||
<span class="status-badge ${status.class}">${status.text}</span>
|
||||
</div>
|
||||
<div class="service-info">
|
||||
Stack ID: ${service.stack_id || 'N/A'}
|
||||
</div>
|
||||
<div class="service-actions">
|
||||
<button class="btn btn-start"
|
||||
onclick="controlService('${service.name}', 'start')"
|
||||
${isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Start
|
||||
</button>
|
||||
<button class="btn btn-stop"
|
||||
onclick="controlService('${service.name}', 'stop')"
|
||||
${!isRunning || alwaysOn ? 'disabled' : ''}>
|
||||
Stop
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
async function controlService(serviceName, action) {
|
||||
const card = document.querySelector(`[data-service="${serviceName}"]`);
|
||||
const buttons = card.querySelectorAll('button');
|
||||
|
||||
// Disable all buttons and show loading
|
||||
buttons.forEach(btn => {
|
||||
btn.disabled = true;
|
||||
if (btn.textContent.toLowerCase().includes(action)) {
|
||||
btn.innerHTML = `<span class="spinner"></span>${action.toUpperCase()}...`;
|
||||
}
|
||||
});
|
||||
|
||||
try {
|
||||
const response = await fetch(`${API_BASE}/infrastructure/services/${serviceName}/${action}`, {
|
||||
method: 'POST'
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
|
||||
if (!response.ok || !result.success) {
|
||||
throw new Error(result.message || result.detail || 'Operation failed');
|
||||
}
|
||||
|
||||
console.log(`${action} ${serviceName}:`, result);
|
||||
|
||||
// Wait a bit for containers to start/stop
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
|
||||
// Refresh service list
|
||||
await fetchServices();
|
||||
|
||||
} catch (error) {
|
||||
console.error(`Error ${action}ing ${serviceName}:`, error);
|
||||
alert(`Failed to ${action} ${serviceName}: ${error.message}`);
|
||||
|
||||
// Re-enable buttons on error
|
||||
await fetchServices();
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-refresh every 10 seconds
|
||||
setInterval(fetchServices, 10000);
|
||||
|
||||
// Initial load
|
||||
fetchServices();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Test what headers are being sent to Organizr
|
||||
"""
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
from src.clients.npm_client import get_npm_client
|
||||
|
||||
|
||||
async def main():
|
||||
npm = get_npm_client()
|
||||
|
||||
# Find home.schweitz.net
|
||||
hosts = await npm.get_proxy_hosts()
|
||||
|
||||
for host in hosts:
|
||||
if 'home.schweitz.net' in host.get('domain_names', []):
|
||||
print(f"Found: {', '.join(host.get('domain_names', []))}")
|
||||
print(f"Forward to: {host.get('forward_scheme')}://{host.get('forward_host')}:{host.get('forward_port')}")
|
||||
print()
|
||||
|
||||
config = host.get('advanced_config', '')
|
||||
|
||||
print("Checking for Authentik headers in nginx config:")
|
||||
print("=" * 60)
|
||||
|
||||
headers_to_check = [
|
||||
'X-authentik-username',
|
||||
'X-authentik-email',
|
||||
'X-authentik-groups',
|
||||
'X-authentik-name',
|
||||
'X-authentik-uid'
|
||||
]
|
||||
|
||||
for header in headers_to_check:
|
||||
if f'proxy_set_header {header}' in config:
|
||||
print(f"✓ {header} is configured")
|
||||
else:
|
||||
print(f"✗ {header} is NOT configured")
|
||||
|
||||
print()
|
||||
print("Full advanced config:")
|
||||
print("=" * 60)
|
||||
print(config)
|
||||
|
||||
break
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update all NPM proxy hosts to use port 9001 for Authentik forward auth
|
||||
"""
|
||||
import asyncio
|
||||
import httpx
|
||||
import os
|
||||
|
||||
NPM_URL = os.getenv("NPM_URL", "http://192.168.86.149:81")
|
||||
NPM_EMAIL = os.getenv("NPM_EMAIL", "admin@example.com")
|
||||
NPM_PASSWORD = os.getenv("NPM_PASSWORD", "changeme")
|
||||
|
||||
|
||||
async def get_npm_token():
|
||||
"""Get NPM authentication token"""
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
f"{NPM_URL}/api/tokens",
|
||||
json={"identity": NPM_EMAIL, "secret": NPM_PASSWORD}
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()["token"]
|
||||
|
||||
|
||||
async def get_proxy_hosts(token):
|
||||
"""Get all proxy hosts"""
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{NPM_URL}/api/nginx/proxy-hosts",
|
||||
headers=headers
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def update_proxy_host(token, host_id, config):
|
||||
"""Update a proxy host"""
|
||||
headers = {"Authorization": f"Bearer {token}"}
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.put(
|
||||
f"{NPM_URL}/api/nginx/proxy-hosts/{host_id}",
|
||||
headers=headers,
|
||||
json=config
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
async def main():
|
||||
print("Updating NPM proxy hosts to use port 9001...\n")
|
||||
|
||||
# Get token
|
||||
token = await get_npm_token()
|
||||
|
||||
# Get all proxy hosts
|
||||
hosts = await get_proxy_hosts(token)
|
||||
|
||||
updated = []
|
||||
skipped = []
|
||||
|
||||
for host in hosts:
|
||||
host_id = host.get("id")
|
||||
domain_names = host.get("domain_names", [])
|
||||
domain_str = ", ".join(domain_names)
|
||||
advanced_config = host.get("advanced_config", "")
|
||||
|
||||
# Skip if no authentik config
|
||||
if "authentik" not in advanced_config.lower():
|
||||
continue
|
||||
|
||||
# Skip if already port 9001
|
||||
if ":9001" in advanced_config:
|
||||
print(f"⊘ {domain_str} - Already using port 9001")
|
||||
skipped.append(domain_str)
|
||||
continue
|
||||
|
||||
# Update 9000 to 9001
|
||||
if ":9000" in advanced_config:
|
||||
print(f"⟳ {domain_str} - Updating to port 9001...", end=" ")
|
||||
|
||||
new_config = advanced_config.replace(":9000", ":9001")
|
||||
|
||||
# Clean config
|
||||
readonly_fields = [
|
||||
"id", "created_on", "modified_on", "owner", "owner_user_id",
|
||||
"certificate", "use_default_location", "ipv6", "meta",
|
||||
"nginx_online", "nginx_err", "access_list", "certificate_id"
|
||||
]
|
||||
|
||||
clean_host = {k: v for k, v in host.items() if k not in readonly_fields}
|
||||
clean_host["advanced_config"] = new_config
|
||||
|
||||
if "locations" not in clean_host or clean_host["locations"] is None:
|
||||
clean_host["locations"] = []
|
||||
|
||||
try:
|
||||
await update_proxy_host(token, host_id, clean_host)
|
||||
print("✓")
|
||||
updated.append(domain_str)
|
||||
except Exception as e:
|
||||
print(f"✗ Error: {e}")
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Updated: {len(updated)} hosts")
|
||||
print(f"Skipped: {len(skipped)} hosts")
|
||||
|
||||
if updated:
|
||||
print("\nUpdated hosts:")
|
||||
for d in updated:
|
||||
print(f" • {d}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
Reference in New Issue
Block a user