Compare commits

...
10 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 4.5 cdb6344013 feat: add PostgreSQL database configuration
- Add POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD settings
- Add database_url property for connection string construction
- Database: core_api on shared postgres instance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:43:44 +01:00
jpmschweitzerandClaude Opus 4.5 0d2926dc6b fix: remove hardcoded service URLs, require ENV config
Build and Push / build (release) Successful in 29s
- Remove hardcoded default URLs (portainer, npm, ollama, etc.)
- All external service URLs now required via ENV vars
- Remove obsolete ai_client and ai_controller (core-ai proxy)
- Clean up health_controller obsolete core-ai references
- App fails fast at startup if required config missing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:59:46 +01:00
jpmschweitzerandClaude Opus 4.5 31370f053c fix: clean up root endpoint and disable redoc
Build and Push / build (release) Successful in 28s
- Simplify root endpoint to return minimal status info
- Remove obsolete endpoint listings (moved to /docs)
- Clean up verbose app description
- Disable redoc UI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:46:26 +01:00
jpmschweitzerandClaude Opus 4.5 9370c70311 fix: correct Portainer default port to 8001
Port 9000 is Authentik, Portainer runs on 8001

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:29:18 +01:00
jpmschweitzerandClaude Opus 4.5 68909e6576 feat: add development server startup script
Build and Push / build (release) Successful in 29s
Add wakeup.sh for local development with:
- Port conflict detection
- Virtual environment activation
- Log file management
- Colored output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:15:17 +01:00
jpmschweitzerandClaude Opus 4.5 435a3faeed chore: clean up .gitignore
Remove duplicate credentials.py entry

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:15:01 +01:00
jpmschweitzerandClaude Opus 4.5 06cb4c17c8 chore: bump version to 1.3.1
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:14:27 +01:00
jpmschweitzerandClaude Opus 4.5 ffa075528b test: update tests for Portainer client without socket fallback
- Remove TestPortainerClientDockerSocketFallback class
- Update wrapper method tests to expect RuntimeError on missing endpoints
- Remove all socket fallback related test cases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:14:04 +01:00
jpmschweitzerandClaude Opus 4.5 dbbb92d292 refactor: remove Docker socket fallback from Portainer client
- Remove _list_containers_via_socket method
- Remove _inspect_container_via_socket method
- Simplify list_containers and inspect_container to use Portainer API only
- Raise RuntimeError when no Portainer endpoints available

BREAKING: Portainer API configuration is now required for all container
and stack operations. Set PORTAINER_URL and PORTAINER_API_KEY env vars.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:13:46 +01:00
jpmschweitzerandClaude Opus 4.5 3ecfb91cc2 refactor: simplify configuration to use only environment variables
- Remove credentials.py import, use pydantic-settings .env support
- Remove unused search API keys (Brave, Google)
- Update default search provider to SearXNG
- Add extra="ignore" to allow flexible env var usage
- Update .env.example with organized sections

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:13:26 +01:00
13 changed files with 202 additions and 830 deletions
+55 -5
View File
@@ -1,15 +1,65 @@
# Core Code API Configuration # Core Code API Configuration
# Copy to .env and fill in real values
# Application settings # =============================================================================
# Application
# =============================================================================
APP_NAME="Core Code API" APP_NAME="Core Code API"
DEBUG=false DEBUG=false
LOG_LEVEL=INFO
# Server settings # =============================================================================
# Server
# =============================================================================
HOST=0.0.0.0 HOST=0.0.0.0
PORT=8083 PORT=8083
# CORS settings (default allows all origins for internal use) # CORS (default allows all origins for internal use)
# CORS_ORIGINS=["http://192.168.86.149:82"] # CORS_ORIGINS=["http://192.168.86.149:82"]
# Logging # =============================================================================
LOG_LEVEL=INFO # Infrastructure Services
# =============================================================================
# Portainer API (required for container/stack management)
PORTAINER_URL=http://localhost:8001
PORTAINER_API_KEY=ptr_your-api-key-here
# Nginx Proxy Manager API
NPM_URL=http://localhost:81
NPM_EMAIL=admin@example.com
NPM_PASSWORD=your-npm-password
# =============================================================================
# Home Automation
# =============================================================================
# Home Assistant API
HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token
# =============================================================================
# AI Services
# =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080
# =============================================================================
# Database (PostgreSQL)
# =============================================================================
POSTGRES_HOST=localhost:5432
POSTGRES_USER=core_api
POSTGRES_PASSWORD=your-password
# =============================================================================
# Vector Database
# =============================================================================
# Qdrant
QDRANT_HOST=qdrant
QDRANT_PORT=6333
-1
View File
@@ -105,7 +105,6 @@ data/
# Credentials and secrets # Credentials and secrets
src/credentials.py src/credentials.py
credentials.py
*.pem *.pem
*.key *.key
secrets/ secrets/
+14
View File
@@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.3.1] - 2025-12-31
### Changed
- Simplified configuration: all settings now read from environment variables/.env only
- Removed Docker socket fallback for container operations (Portainer API is now required)
- Updated default search provider to SearXNG
### Removed
- `src/credentials.py` - credentials now managed via environment variables
- Docker socket fallback methods from Portainer client
- Unused search API settings (Brave, Google)
## [1.3.0] - 2025-12-31 ## [1.3.0] - 2025-12-31
### Added ### Added
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "core-api" name = "core-api"
version = "1.3.0" version = "1.3.4"
description = "Core Code API - Infrastructure management and tools API" description = "Core Code API - Infrastructure management and tools API"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
-197
View File
@@ -1,197 +0,0 @@
"""
Core-AI HTTP Client
Provides interface to Core-AI service for AI performance metrics.
"""
import httpx
from typing import Optional, Dict, List, Any
from src.logging_config import get_logger
from src.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class CoreAIClient:
"""
HTTP client for Core-AI service
Provides access to AI performance metrics, tool execution stats,
and memory system monitoring.
"""
def __init__(
self,
base_url: Optional[str] = None,
timeout: int = 10
):
"""
Initialize Core-AI client
Args:
base_url: Core-AI base URL (default from settings)
timeout: Request timeout in seconds
"""
self.base_url = (base_url or getattr(settings, 'core_ai_base_url', 'http://core-ai:8086')).rstrip("/")
self.timeout = timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
async def close(self):
"""Close the HTTP client"""
await self.client.aclose()
async def health_check(self) -> bool:
"""
Check if Core-AI service is accessible
Returns:
True if accessible, False otherwise
"""
try:
response = await self.client.get(f"{self.base_url}/health")
return response.status_code == 200
except Exception as e:
logger.error(f"Core-AI health check failed: {e}")
return False
async def get_metrics(self) -> Dict[str, Any]:
"""
Get comprehensive AI performance metrics
Returns:
Dict with agent performance, tool execution, memory stats
Example:
{
"uptime_seconds": 3600,
"timestamp": "2025-12-03T20:00:00Z",
"agent": {
"total_requests": 100,
"avg_response_time_ms": 1250.5,
"p95_response_time_ms": 3200.0,
...
},
"tools": {
"total_calls": 250,
"success_rate": 0.98,
"top_tools": {...}
},
"memory": {
"tier1_hit_rate": 0.85,
...
},
...
}
"""
try:
response = await self.client.get(f"{self.base_url}/metrics")
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
logger.error(f"Failed to get metrics: HTTP {e.response.status_code}")
raise
except Exception as e:
logger.error(f"Failed to get metrics: {e}")
raise
async def get_recent_errors(self, limit: int = 20) -> List[Dict[str, Any]]:
"""
Get recent request errors
Args:
limit: Maximum number of errors to return
Returns:
List of error records with timestamps
Example:
[
{
"timestamp": "2025-12-03T19:45:12Z",
"agent_type": "pydantic",
"error": "Connection timeout",
"duration_ms": 5000
},
...
]
"""
try:
response = await self.client.get(
f"{self.base_url}/metrics/errors",
params={"limit": limit}
)
response.raise_for_status()
data = response.json()
return data.get("errors", [])
except Exception as e:
logger.error(f"Failed to get recent errors: {e}")
raise
async def get_tool_failures(self, limit: int = 20) -> List[Dict[str, Any]]:
"""
Get recent tool execution failures
Args:
limit: Maximum number of failures to return
Returns:
List of tool failure records
Example:
[
{
"timestamp": "2025-12-03T19:50:30Z",
"tool_name": "list_containers",
"error": "Connection refused",
"duration_ms": 150
},
...
]
"""
try:
response = await self.client.get(
f"{self.base_url}/metrics/tool-failures",
params={"limit": limit}
)
response.raise_for_status()
data = response.json()
return data.get("failures", [])
except Exception as e:
logger.error(f"Failed to get tool failures: {e}")
raise
async def reset_metrics(self) -> bool:
"""
Reset all metrics (admin operation)
Returns:
True if successful
"""
try:
response = await self.client.post(f"{self.base_url}/metrics/reset")
response.raise_for_status()
logger.info("Successfully reset Core-AI metrics")
return True
except Exception as e:
logger.error(f"Failed to reset metrics: {e}")
raise
async def __aenter__(self):
"""Async context manager entry"""
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""Async context manager exit"""
await self.close()
# Singleton instance
_ai_client: Optional[CoreAIClient] = None
def get_ai_client() -> CoreAIClient:
"""Get singleton Core-AI client instance"""
global _ai_client
if _ai_client is None:
_ai_client = CoreAIClient()
return _ai_client
+22 -112
View File
@@ -438,70 +438,14 @@ class PortainerClient:
return True return True
# ======================================================================== # ========================================================================
# Docker Socket Fallback (for containers not managed by Portainer) # Helper methods for agent tools (auto-detect endpoint)
# ========================================================================
async def _list_containers_via_socket(self, all_containers: bool = True) -> List[Dict[str, Any]]:
"""
Fallback: List containers directly via Docker socket
Used when Portainer API doesn't return complete data (e.g., containers
started outside Portainer, AMP game servers, etc.)
Args:
all_containers: Include stopped containers
Returns:
List of container details in Docker API format
"""
try:
# Docker socket is mounted at /var/run/docker.sock
# Use httpx with unix socket transport
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
params = {"all": 1 if all_containers else 0}
response = await client.get(
"http://localhost/v1.41/containers/json",
params=params
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.warning(f"Docker socket fallback failed: {e}")
return []
async def _inspect_container_via_socket(self, container_id_or_name: str) -> Optional[Dict[str, Any]]:
"""
Fallback: Inspect container directly via Docker socket
Args:
container_id_or_name: Container ID or name
Returns:
Container details or None
"""
try:
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
response = await client.get(
f"http://localhost/v1.41/containers/{container_id_or_name}/json"
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.warning(f"Docker socket inspect fallback failed for '{container_id_or_name}': {e}")
return None
# ========================================================================
# Helper methods for agent tools (auto-detect endpoint + fallback)
# ======================================================================== # ========================================================================
async def list_containers(self, all_containers: bool = True) -> List[Dict[str, Any]]: async def list_containers(self, all_containers: bool = True) -> List[Dict[str, Any]]:
""" """
List containers using auto-detected endpoint with Docker socket fallback List containers using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint. This is a convenience wrapper that automatically uses the first/default endpoint.
If Portainer doesn't have complete data, falls back to Docker socket.
Args: Args:
all_containers: Include stopped containers (default: True) all_containers: Include stopped containers (default: True)
@@ -509,34 +453,18 @@ class PortainerClient:
Returns: Returns:
List of container details List of container details
""" """
try: endpoints = await self.get_endpoints()
# Try Portainer first if not endpoints:
endpoints = await self.get_endpoints() raise RuntimeError("No Portainer endpoints available")
if endpoints:
endpoint_id = endpoints[0]["Id"]
containers = await self.get_containers(endpoint_id, all_containers)
if containers:
return containers
# Fallback to Docker socket endpoint_id = endpoints[0]["Id"]
logger.info("Portainer returned no containers, trying Docker socket fallback...") return await self.get_containers(endpoint_id, all_containers)
return await self._list_containers_via_socket(all_containers)
except Exception as e:
logger.error(f"Error listing containers: {e}")
# Try fallback even on exception
try:
return await self._list_containers_via_socket(all_containers)
except Exception as fallback_error:
logger.error(f"Fallback also failed: {fallback_error}")
return []
async def inspect_container(self, container_name: str) -> Optional[Dict[str, Any]]: async def inspect_container(self, container_name: str) -> Optional[Dict[str, Any]]:
""" """
Inspect a container by name using auto-detected endpoint with Docker socket fallback Inspect a container by name using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint. This is a convenience wrapper that automatically uses the first/default endpoint.
If Portainer doesn't find the container, falls back to Docker socket.
Args: Args:
container_name: Container name (e.g., "jellyfin", "ollama") container_name: Container name (e.g., "jellyfin", "ollama")
@@ -544,44 +472,26 @@ class PortainerClient:
Returns: Returns:
Container details or None if not found Container details or None if not found
""" """
try: endpoints = await self.get_endpoints()
# Try Portainer first if not endpoints:
endpoints = await self.get_endpoints() raise RuntimeError("No Portainer endpoints available")
if endpoints:
endpoint_id = endpoints[0]["Id"]
# First list all containers to find the one matching the name endpoint_id = endpoints[0]["Id"]
all_containers = await self.get_containers(endpoint_id, all_containers=True)
matching_container = None # List all containers to find the one matching the name
for container in all_containers: all_containers = await self.get_containers(endpoint_id, all_containers=True)
# Container names come as array like ['/jellyfin']
names = container.get('Names', [])
for name in names:
clean_name = name.lstrip('/')
if clean_name == container_name or clean_name.lower() == container_name.lower():
matching_container = container
break
if matching_container:
break
if matching_container: for container in all_containers:
# Container names come as array like ['/jellyfin']
names = container.get('Names', [])
for name in names:
clean_name = name.lstrip('/')
if clean_name == container_name or clean_name.lower() == container_name.lower():
# Get detailed info using container ID # Get detailed info using container ID
container_id = matching_container['Id'] container_id = container['Id']
return await self.get_container(endpoint_id, container_id) return await self.get_container(endpoint_id, container_id)
# Not found in Portainer, try Docker socket fallback return None
logger.info(f"Container '{container_name}' not found in Portainer, trying Docker socket fallback...")
return await self._inspect_container_via_socket(container_name)
except Exception as e:
logger.error(f"Error inspecting container '{container_name}': {e}")
# Try fallback even on exception
try:
return await self._inspect_container_via_socket(container_name)
except Exception as fallback_error:
logger.error(f"Fallback also failed: {fallback_error}")
return None
# Singleton instance # Singleton instance
+26 -41
View File
@@ -1,5 +1,8 @@
""" """
Global configuration for Core Code API Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
""" """
import tomllib import tomllib
from pathlib import Path from pathlib import Path
@@ -20,29 +23,6 @@ def _get_version_from_pyproject() -> str:
__version__ = _get_version_from_pyproject() __version__ = _get_version_from_pyproject()
# Import infrastructure credentials from gitignored module
try:
from src.credentials import (
PORTAINER_URL, PORTAINER_API_KEY,
NPM_URL, NPM_EMAIL, NPM_PASSWORD,
BRAVE_SEARCH_API_KEY,
GOOGLE_SEARCH_API_KEY, GOOGLE_SEARCH_ENGINE_ID,
HOMEASSISTANT_URL, HOMEASSISTANT_TOKEN
)
except ImportError:
# Fallback to empty strings if credentials.py doesn't exist
# (e.g., fresh clone before credentials setup)
PORTAINER_URL = "http://localhost:8001"
PORTAINER_API_KEY = ""
NPM_URL = "http://localhost:81"
NPM_EMAIL = ""
NPM_PASSWORD = ""
BRAVE_SEARCH_API_KEY = ""
GOOGLE_SEARCH_API_KEY = ""
GOOGLE_SEARCH_ENGINE_ID = ""
HOMEASSISTANT_URL = "http://localhost:8123"
HOMEASSISTANT_TOKEN = ""
class Settings(BaseSettings): class Settings(BaseSettings):
"""Global application settings""" """Global application settings"""
@@ -66,7 +46,7 @@ class Settings(BaseSettings):
log_level: str = "DEBUG" log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration) # Ollama Configuration (for AI orchestration)
ollama_base_url: str = "http://ollama:11434" ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes ollama_timeout: int = 300 # 5 minutes
# Model Configuration # Model Configuration
@@ -109,29 +89,33 @@ class Settings(BaseSettings):
embedding_batch_size: int = 32 embedding_batch_size: int = 32
# Search Configuration # Search Configuration
search_provider: str = "google" # Options: google, brave, searxng, duckduckgo search_provider: str = "searxng"
searxng_url: str = "http://searxng:8080" # For future self-hosted SearxNG searxng_url: str # Required - set SEARXNG_URL in .env
# Search API Keys (from credentials.py) # Infrastructure Management (Portainer)
brave_search_api_key: str = BRAVE_SEARCH_API_KEY # https://brave.com/search/api/ portainer_url: str # Required - set PORTAINER_URL in .env
google_search_api_key: str = GOOGLE_SEARCH_API_KEY # https://console.cloud.google.com/ portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
google_search_engine_id: str = GOOGLE_SEARCH_ENGINE_ID # Custom Search Engine ID
# Infrastructure Management (from credentials.py) # Infrastructure Management (Nginx Proxy Manager)
portainer_url: str = PORTAINER_URL npm_url: str # Required - set NPM_URL in .env
portainer_api_key: str = PORTAINER_API_KEY npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
npm_url: str = NPM_URL
npm_email: str = NPM_EMAIL
npm_password: str = NPM_PASSWORD
# Home Assistant Configuration # Home Assistant Configuration
homeassistant_url: str = HOMEASSISTANT_URL homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str = HOMEASSISTANT_TOKEN homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30 homeassistant_timeout: int = 30
# Core-AI Service (AI performance metrics) # PostgreSQL Database
core_ai_base_url: str = "http://core-ai:8086" postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik) # OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication oidc_enabled: bool = False # Set to True to require authentication
@@ -163,6 +147,7 @@ class Settings(BaseSettings):
class Config: class Config:
env_file = ".env" env_file = ".env"
case_sensitive = False case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache() @lru_cache()
-219
View File
@@ -1,219 +0,0 @@
"""
AI Metrics Proxy Controller
Provides proxy endpoints to Core-AI service metrics.
Allows external access to AI performance stats via core-api.
"""
from fastapi import APIRouter, HTTPException
from typing import Dict, List, Any
from src.clients.ai_client import get_ai_client
from src.logging_config import get_logger
logger = get_logger(__name__)
# Create router
router = APIRouter(
prefix="/ai",
tags=["AI Metrics"]
)
@router.get(
"/health",
summary="Check Core-AI service health",
description="Verify that the Core-AI service is accessible and responding"
)
async def ai_health_check():
"""
Check if Core-AI service is healthy
Returns:
Health status and availability
"""
try:
ai_client = get_ai_client()
is_healthy = await ai_client.health_check()
return {
"service": "core-ai",
"status": "healthy" if is_healthy else "unhealthy",
"accessible": is_healthy
}
except Exception as e:
logger.error(f"AI health check failed: {e}")
return {
"service": "core-ai",
"status": "error",
"accessible": False,
"error": str(e)
}
@router.get(
"/metrics",
response_model=Dict[str, Any],
summary="Get comprehensive AI performance metrics",
description="Returns detailed metrics including agent performance, tool execution stats, memory system metrics, and user activity"
)
async def get_ai_metrics():
"""
Proxy endpoint for Core-AI metrics
Returns comprehensive AI performance data:
- Agent request statistics (total, by type, response times)
- Response time percentiles (p50, p95, p99)
- Tool execution metrics (calls, success rates, durations)
- Memory system statistics (cache hits, consolidations)
- User activity tracking
- Concurrency metrics
Returns:
Dict with all collected metrics
Raises:
HTTPException: If Core-AI is unreachable or returns error
"""
try:
ai_client = get_ai_client()
metrics = await ai_client.get_metrics()
return metrics
except Exception as e:
logger.error(f"Failed to fetch AI metrics: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.get(
"/metrics/errors",
response_model=Dict[str, Any],
summary="Get recent request errors",
description="Returns recent AI agent request errors with timestamps and details"
)
async def get_ai_errors(limit: int = 20):
"""
Get recent AI request errors
Args:
limit: Maximum number of errors to return (default: 20)
Returns:
Dict with error list and total count
Example response:
{
"errors": [
{
"timestamp": "2025-12-03T19:45:12Z",
"agent_type": "pydantic",
"error": "Connection timeout",
"duration_ms": 5000
}
],
"total": 1
}
"""
try:
ai_client = get_ai_client()
errors = await ai_client.get_recent_errors(limit=limit)
return {
"errors": errors,
"total": len(errors)
}
except Exception as e:
logger.error(f"Failed to fetch AI errors: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.get(
"/metrics/tool-failures",
response_model=Dict[str, Any],
summary="Get recent tool execution failures",
description="Returns recent tool execution failures with error details"
)
async def get_ai_tool_failures(limit: int = 20):
"""
Get recent tool execution failures
Args:
limit: Maximum number of failures to return (default: 20)
Returns:
Dict with failure list and total count
Example response:
{
"failures": [
{
"timestamp": "2025-12-03T19:50:30Z",
"tool_name": "list_containers",
"error": "Connection refused",
"duration_ms": 150
}
],
"total": 1
}
"""
try:
ai_client = get_ai_client()
failures = await ai_client.get_tool_failures(limit=limit)
return {
"failures": failures,
"total": len(failures)
}
except Exception as e:
logger.error(f"Failed to fetch tool failures: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.post(
"/metrics/reset",
summary="Reset all AI metrics (admin)",
description="Clear all collected metrics. This is an administrative operation that resets all counters and history."
)
async def reset_ai_metrics():
"""
Reset all AI metrics (admin operation)
Clears all collected metrics including:
- Request history
- Tool execution stats
- Memory system metrics
- Error logs
Returns:
Success confirmation
Note:
This is an administrative operation that should be used carefully.
All historical data will be lost.
"""
try:
ai_client = get_ai_client()
await ai_client.reset_metrics()
logger.info("AI metrics reset successfully")
return {
"success": True,
"message": "AI metrics reset successfully"
}
except Exception as e:
logger.error(f"Failed to reset AI metrics: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
+3 -38
View File
@@ -11,9 +11,6 @@ from src.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client from src.models.ollama_client import get_ollama_client
# Note: Agent functionality moved to separate core-ai service (Dec 2025)
# This service (core-api) only provides infrastructure management and tools
AGENT_AVAILABLE = False
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -52,20 +49,7 @@ class HealthController(BaseController):
"service": settings.app_name, "service": settings.app_name,
"version": settings.app_version, "version": settings.app_version,
"status": "healthy", "status": "healthy",
"documentation": { "docs": "/docs"
"swagger_ui": "/docs",
"redoc": "/redoc",
"openapi_spec": "/openapi.json"
},
"endpoints": {
"chat_completions": "/v1/chat/completions",
"models": "/v1/models",
"conversations": "/v1/conversations",
"dns_lookup": "/tools/dns/lookup",
"infrastructure": "/infrastructure",
"health": "/health",
"health_full": "/health/full"
}
} }
@router.get( @router.get(
@@ -147,11 +131,6 @@ class HealthController(BaseController):
ollama_error = str(e) ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}") logger.warning(f"Ollama health check failed: {ollama_error}")
# Note: Agent functionality moved to separate core-ai service
# This service only needs Ollama for embeddings (infrastructure tools)
# Agent health is checked separately in core-ai service
# Determine overall status (only Ollama required for core-api)
is_healthy = ollama_healthy is_healthy = ollama_healthy
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
@@ -170,8 +149,7 @@ class HealthController(BaseController):
"available": False "available": False
}, },
"error": ollama_error "error": ollama_error
}, }
"note": "AI agent functionality available in separate core-ai service (port 8086)"
} }
} }
@@ -217,20 +195,7 @@ class HealthController(BaseController):
"error": str(e) "error": str(e)
} }
# 2. Agent Stack - Moved to separate core-ai service # 2. Configuration
diagnostics["components"]["agent"] = {
"status": "N/A",
"note": "AI agent functionality moved to separate core-ai service (port 8086)",
"check_url": "http://core-ai:8086/health"
}
# 3. Memory System (Qdrant) - Moved to core-ai service
diagnostics["components"]["qdrant"] = {
"status": "N/A",
"note": "Memory system managed by core-ai service (port 8086)"
}
# 4. Configuration
diagnostics["configuration"] = { diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled, "agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns, "memory_tier1_max_turns": settings.memory_tier1_max_turns,
-23
View File
@@ -1,23 +0,0 @@
"""
Infrastructure Credentials Template
INSTRUCTIONS:
1. Copy this file to credentials.py
2. Fill in your actual credentials
3. DO NOT commit credentials.py to version control (it's in .gitignore)
This file should be committed to the repository as a template.
"""
# Portainer Configuration
PORTAINER_URL = "http://localhost:8001"
PORTAINER_API_KEY = "ptr_your_api_token_here" # Create in Portainer UI: User menu → My account → Access tokens
# Nginx Proxy Manager Configuration
NPM_URL = "http://localhost:81"
NPM_EMAIL = "admin@example.com"
NPM_PASSWORD = "your_password_here"
# Home Assistant Configuration
HOMEASSISTANT_URL = "http://192.168.86.149:8123" # Or http://home-assistant:8123 in Docker
HOMEASSISTANT_TOKEN = "your_long_lived_access_token_here" # Create in HA: Profile → Long-Lived Access Tokens
+7 -80
View File
@@ -13,7 +13,6 @@ from src.controllers.infrastructure_controller import infrastructure_controller
from src.controllers.tools_controller import tools_controller from src.controllers.tools_controller import tools_controller
from src.controllers.health_controller import health_controller from src.controllers.health_controller import health_controller
from src.controllers.static_controller import static_controller from src.controllers.static_controller import static_controller
from src.controllers.ai_controller import router as ai_router
from src.controllers.housekeeping_controller import housekeeping_controller from src.controllers.housekeeping_controller import housekeeping_controller
from src.security import initialize_oidc from src.security import initialize_oidc
@@ -64,89 +63,18 @@ app = FastAPI(
title=settings.app_name, title=settings.app_name,
version=settings.app_version, version=settings.app_version,
description=""" description="""
Core Code API provides OpenAPI-compatible functions and AI orchestration for Open WebUI. Core Code API - Infrastructure management and home automation API.
## Features ## Features
### OpenAI-Compatible API (v1) - **Infrastructure Management** - Container and stack management via Portainer
- `/v1/chat/completions` - Chat completions with streaming support - **Home Automation** - Device control via Home Assistant
- `/v1/models` - List available models - **Tools** - DNS lookup and utilities
Compatible with OpenAI client libraries and Open WebUI.
### Conversation Memory (Phase 2) See `/docs` for the full API reference.
- `/v1/conversations/{id}` - Get conversation history
- `/v1/conversations/{id}/search` - Semantic search within conversation
- `/v1/conversations/search` - Search across all conversations
- `/v1/conversations/{id}/stats` - Get conversation statistics
- `/v1/conversations/{id}/consolidate` - Manual consolidation
- `DELETE /v1/conversations/{id}` - Delete conversation
Multi-tier memory system:
- **Tier 1**: Fast in-memory buffer (last 10 turns)
- **Tier 2/3**: Unified Qdrant storage (persistent + semantic search)
### Infrastructure Management
**Read Endpoints:**
- `GET /infrastructure/health` - Check Portainer & NPM connectivity
- `GET /infrastructure/services` - List all deployed services
- `GET /infrastructure/services/{name}` - Get service details
- `GET /infrastructure/ports` - List allocated ports
- `GET /infrastructure/domains` - List configured domains
**Write Endpoints (Admin Only):**
- `POST /infrastructure/services` - Deploy new service from compose YAML
- `PUT /infrastructure/services/{name}` - Update existing service
- `DELETE /infrastructure/services/{name}` - Remove service and stack
- `POST /infrastructure/proxy` - Create proxy host with optional SSL
Automates infrastructure operations via Portainer and Nginx Proxy Manager APIs.
### Home Automation (Housekeeping)
**Read Endpoints:**
- `GET /housekeeping/health` - Check Home Assistant connectivity
- `GET /housekeeping/devices` - List devices (filter by domain, area)
- `GET /housekeeping/devices/{entity_id}` - Get device details
- `GET /housekeeping/areas` - List areas/rooms
- `GET /housekeeping/scenes` - List scenes
- `GET /housekeeping/scripts` - List scripts
- `GET /housekeeping/automations` - List automations
- `GET /housekeeping/history` - Get state history
**Write Endpoints (Admin Only):**
- `POST /housekeeping/devices/{entity_id}/control` - Control device
- `POST /housekeeping/scenes/{scene_id}/activate` - Activate scene
- `POST /housekeeping/scripts/{script_id}/run` - Run script
- `POST /housekeeping/automations/{automation_id}/toggle` - Enable/disable automation
Abstracts Home Assistant for the Tatlock Housekeeper agent and other consumers.
### Web Scraper
Intelligent web scraping with main content extraction.
Perfect for extracting articles, documentation, and blog posts for LLM consumption.
## Authentication
When OIDC authentication is enabled (oidc_enabled=true in config):
- Infrastructure write endpoints require authentication
- Use OAuth2/OIDC bearer token from Authentik
- Admin group membership required for infrastructure operations
## Integration
This API is designed to integrate with:
- **Open WebUI**: Direct OpenAI API compatibility
- **Open WebUI Functions**: Import via OpenAPI spec
- **Open WebUI Pipelines**: Use as data source
- **LangChain**: Compatible with standard HTTP tools
## Documentation
- **OpenAPI Spec**: `/openapi.json`
- **Swagger UI**: `/docs`
- **ReDoc**: `/redoc`
""", """,
docs_url="/docs", docs_url="/docs",
redoc_url="/redoc", redoc_url=None,
openapi_url="/openapi.json", openapi_url="/openapi.json",
lifespan=lifespan, lifespan=lifespan,
debug=settings.debug, debug=settings.debug,
@@ -172,7 +100,6 @@ app.include_router(tools_controller.router) # /tools/*
app.include_router(infrastructure_controller.router) # /infrastructure/* app.include_router(infrastructure_controller.router) # /infrastructure/*
app.include_router(housekeeping_controller.router) # /housekeeping/* app.include_router(housekeeping_controller.router) # /housekeeping/*
app.include_router(static_controller.router) # /static/* app.include_router(static_controller.router) # /static/*
app.include_router(ai_router) # /ai/*
# Global exception handler # Global exception handler
+24 -113
View File
@@ -391,40 +391,12 @@ class TestPortainerClientContainers:
assert result is True assert result is True
class TestPortainerClientDockerSocketFallback:
"""Test Docker socket fallback methods."""
@pytest.mark.asyncio
async def test_list_containers_via_socket_returns_empty_on_error(self):
"""_list_containers_via_socket should return empty list on error."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch("httpx.AsyncHTTPTransport") as mock_transport:
mock_transport.side_effect = Exception("Socket not available")
result = await client._list_containers_via_socket()
assert result == []
@pytest.mark.asyncio
async def test_inspect_container_via_socket_returns_none_on_error(self):
"""_inspect_container_via_socket should return None on error."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch("httpx.AsyncHTTPTransport") as mock_transport:
mock_transport.side_effect = Exception("Socket not available")
result = await client._inspect_container_via_socket("container_name")
assert result is None
class TestPortainerClientWrapperMethods: class TestPortainerClientWrapperMethods:
"""Test convenience wrapper methods.""" """Test convenience wrapper methods."""
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_list_containers_uses_portainer_first(self): async def test_list_containers_uses_portainer(self):
"""list_containers should try Portainer first.""" """list_containers should use Portainer API."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key") client = PortainerClient(base_url="http://portainer:9000", api_key="key")
containers = [{"Id": "abc123", "Names": ["/test"]}] containers = [{"Id": "abc123", "Names": ["/test"]}]
@@ -442,57 +414,19 @@ class TestPortainerClientWrapperMethods:
mock_containers.assert_called_once() mock_containers.assert_called_once()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_list_containers_falls_back_to_socket(self): async def test_list_containers_raises_when_no_endpoints(self):
"""list_containers should fallback to Docker socket if Portainer returns empty.""" """list_containers should raise RuntimeError when no endpoints available."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key") client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints: with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = [{"Id": 1}] mock_endpoints.return_value = []
with patch.object(client, "get_containers", new_callable=AsyncMock) as mock_containers: with pytest.raises(RuntimeError, match="No Portainer endpoints available"):
mock_containers.return_value = [] await client.list_containers()
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = [{"Id": "from_socket"}]
result = await client.list_containers()
assert result == [{"Id": "from_socket"}]
mock_socket.assert_called_once()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_list_containers_handles_exception_with_fallback(self): async def test_inspect_container_uses_portainer(self):
"""list_containers should try fallback even on exception.""" """inspect_container should use Portainer API."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = [{"Id": "fallback"}]
result = await client.list_containers()
assert result == [{"Id": "fallback"}]
@pytest.mark.asyncio
async def test_list_containers_returns_empty_when_all_fails(self):
"""list_containers should return empty list when everything fails."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.side_effect = Exception("Socket error")
result = await client.list_containers()
assert result == []
@pytest.mark.asyncio
async def test_inspect_container_uses_portainer_first(self):
"""inspect_container should try Portainer first."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key") client = PortainerClient(base_url="http://portainer:9000", api_key="key")
container_list = [{"Id": "abc123", "Names": ["/mycontainer"]}] container_list = [{"Id": "abc123", "Names": ["/mycontainer"]}]
@@ -512,54 +446,31 @@ class TestPortainerClientWrapperMethods:
assert result == container_detail assert result == container_detail
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_inspect_container_falls_back_to_socket(self): async def test_inspect_container_returns_none_when_not_found(self):
"""inspect_container should fallback if not found in Portainer.""" """inspect_container should return None if container not found."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key") client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints: with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = [{"Id": 1}] mock_endpoints.return_value = [{"Id": 1}]
with patch.object(client, "get_containers", new_callable=AsyncMock) as mock_list: with patch.object(client, "get_containers", new_callable=AsyncMock) as mock_list:
mock_list.return_value = [] # Container not found mock_list.return_value = [] # No containers
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket: result = await client.inspect_container("missing_container")
mock_socket.return_value = {"Id": "from_socket"}
result = await client.inspect_container("missing_container")
assert result == {"Id": "from_socket"}
mock_socket.assert_called_once()
@pytest.mark.asyncio
async def test_inspect_container_handles_exception_with_fallback(self):
"""inspect_container should try fallback even on exception."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = {"Id": "fallback"}
result = await client.inspect_container("container")
assert result == {"Id": "fallback"}
@pytest.mark.asyncio
async def test_inspect_container_returns_none_when_all_fails(self):
"""inspect_container should return None when everything fails."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.side_effect = Exception("Socket error")
result = await client.inspect_container("container")
assert result is None assert result is None
@pytest.mark.asyncio
async def test_inspect_container_raises_when_no_endpoints(self):
"""inspect_container should raise RuntimeError when no endpoints available."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = []
with pytest.raises(RuntimeError, match="No Portainer endpoints available"):
await client.inspect_container("container")
class TestPortainerClientStackFile: class TestPortainerClientStackFile:
"""Test stack file operations.""" """Test stack file operations."""
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
# Core-API Server Startup Script
set -e
# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
echo -e "${GREEN}Starting Core-API server...${NC}"
# Check if port 8788 is already in use
if lsof -Pi :8788 -sTCP:LISTEN -t >/dev/null 2>&1 ; then
echo -e "${RED}Error: Port 8788 is already in use${NC}"
echo "Run: lsof -i :8788 to see what's using it"
echo "Or run: kill \$(lsof -t -i:8788) to stop it"
exit 1
fi
# Activate virtual environment if not already activated
if [ -z "$VIRTUAL_ENV" ]; then
if [ -d ".venv" ]; then
echo -e "${YELLOW}Activating virtual environment...${NC}"
source .venv/bin/activate
else
echo -e "${RED}Error: Virtual environment not found${NC}"
echo "Run: python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt"
exit 1
fi
fi
# Create logs directory if it doesn't exist
LOGS_DIR="logs"
mkdir -p "$LOGS_DIR"
# Clear/create log file
LOG_FILE="$LOGS_DIR/server.log"
> "$LOG_FILE"
echo -e "${YELLOW}Logs will be written to: ${LOG_FILE}${NC}"
# Start the server
echo -e "${GREEN}Starting uvicorn server on http://tower-of-joy:8788${NC}"
echo -e "${YELLOW}Press Ctrl+C to stop the server${NC}"
echo ""
uvicorn src.main:app --reload --host 0.0.0.0 --port 8788 2>&1 | tee "$LOG_FILE"