Compare commits

...
11 Commits
Author SHA1 Message Date
Jeroen SchweitzerandClaude Opus 4.5 4f45f9bf37 feat: add authentication and user management with Authentik integration
Build and Push / build (release) Successful in 1m16s
- Add PostgreSQL database with async SQLAlchemy
- Add Alembic migrations for schema management
- Add User, Role, UserPreferences, ApiKey models
- Add auth endpoints: /auth/me, /auth/users, /auth/users/sync-from-authentik
- Add token validation via Authentik userinfo endpoint
- Add bulk user sync from Authentik admin API
- Add database health check to diagnostics

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 20:25:03 +01:00
jpmschweitzerandClaude Opus 4.5 cdb6344013 feat: add PostgreSQL database configuration
- Add POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD settings
- Add database_url property for connection string construction
- Database: core_api on shared postgres instance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 15:43:44 +01:00
jpmschweitzerandClaude Opus 4.5 0d2926dc6b fix: remove hardcoded service URLs, require ENV config
Build and Push / build (release) Successful in 29s
- Remove hardcoded default URLs (portainer, npm, ollama, etc.)
- All external service URLs now required via ENV vars
- Remove obsolete ai_client and ai_controller (core-ai proxy)
- Clean up health_controller obsolete core-ai references
- App fails fast at startup if required config missing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:59:46 +01:00
jpmschweitzerandClaude Opus 4.5 31370f053c fix: clean up root endpoint and disable redoc
Build and Push / build (release) Successful in 28s
- Simplify root endpoint to return minimal status info
- Remove obsolete endpoint listings (moved to /docs)
- Clean up verbose app description
- Disable redoc UI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:46:26 +01:00
jpmschweitzerandClaude Opus 4.5 9370c70311 fix: correct Portainer default port to 8001
Port 9000 is Authentik, Portainer runs on 8001

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:29:18 +01:00
jpmschweitzerandClaude Opus 4.5 68909e6576 feat: add development server startup script
Build and Push / build (release) Successful in 29s
Add wakeup.sh for local development with:
- Port conflict detection
- Virtual environment activation
- Log file management
- Colored output

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:15:17 +01:00
jpmschweitzerandClaude Opus 4.5 435a3faeed chore: clean up .gitignore
Remove duplicate credentials.py entry

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:15:01 +01:00
jpmschweitzerandClaude Opus 4.5 06cb4c17c8 chore: bump version to 1.3.1
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:14:27 +01:00
jpmschweitzerandClaude Opus 4.5 ffa075528b test: update tests for Portainer client without socket fallback
- Remove TestPortainerClientDockerSocketFallback class
- Update wrapper method tests to expect RuntimeError on missing endpoints
- Remove all socket fallback related test cases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:14:04 +01:00
jpmschweitzerandClaude Opus 4.5 dbbb92d292 refactor: remove Docker socket fallback from Portainer client
- Remove _list_containers_via_socket method
- Remove _inspect_container_via_socket method
- Simplify list_containers and inspect_container to use Portainer API only
- Raise RuntimeError when no Portainer endpoints available

BREAKING: Portainer API configuration is now required for all container
and stack operations. Set PORTAINER_URL and PORTAINER_API_KEY env vars.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:13:46 +01:00
jpmschweitzerandClaude Opus 4.5 3ecfb91cc2 refactor: simplify configuration to use only environment variables
- Remove credentials.py import, use pydantic-settings .env support
- Remove unused search API keys (Brave, Google)
- Update default search provider to SearXNG
- Add extra="ignore" to allow flexible env var usage
- Update .env.example with organized sections

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 11:13:26 +01:00
30 changed files with 1952 additions and 850 deletions
+55 -5
View File
@@ -1,15 +1,65 @@
# Core Code API Configuration
# Copy to .env and fill in real values
# Application settings
# =============================================================================
# Application
# =============================================================================
APP_NAME="Core Code API"
DEBUG=false
LOG_LEVEL=INFO
# Server settings
# =============================================================================
# Server
# =============================================================================
HOST=0.0.0.0
PORT=8083
# CORS settings (default allows all origins for internal use)
# CORS (default allows all origins for internal use)
# CORS_ORIGINS=["http://192.168.86.149:82"]
# Logging
LOG_LEVEL=INFO
# =============================================================================
# Infrastructure Services
# =============================================================================
# Portainer API (required for container/stack management)
PORTAINER_URL=http://localhost:8001
PORTAINER_API_KEY=ptr_your-api-key-here
# Nginx Proxy Manager API
NPM_URL=http://localhost:81
NPM_EMAIL=admin@example.com
NPM_PASSWORD=your-npm-password
# =============================================================================
# Home Automation
# =============================================================================
# Home Assistant API
HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token
# =============================================================================
# AI Services
# =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080
# =============================================================================
# Database (PostgreSQL)
# =============================================================================
POSTGRES_HOST=localhost:5432
POSTGRES_USER=core_api
POSTGRES_PASSWORD=your-password
# =============================================================================
# Vector Database
# =============================================================================
# Qdrant
QDRANT_HOST=qdrant
QDRANT_PORT=6333
-1
View File
@@ -105,7 +105,6 @@ data/
# Credentials and secrets
src/credentials.py
credentials.py
*.pem
*.key
secrets/
+34
View File
@@ -5,6 +5,40 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.4.0] - 2026-01-01
### Added
- **Authentication & User Management** - Authentik integration for user synchronization
- `GET /auth/me` - Get current authenticated user info
- `GET /auth/users` - List all users with search and pagination
- `POST /auth/users/sync-from-authentik` - Bulk sync users from Authentik admin API
- PostgreSQL database integration with async SQLAlchemy
- Alembic database migrations for schema management
- Database models: User, Role, UserPreferences, ApiKey
- Token validation via Authentik userinfo endpoint
- Role synchronization from Authentik groups
- Health check now includes database connectivity status
### Changed
- Authentik configuration now uses username/password for admin API access
- Health endpoint includes database status in diagnostics
## [1.3.1] - 2025-12-31
### Changed
- Simplified configuration: all settings now read from environment variables/.env only
- Removed Docker socket fallback for container operations (Portainer API is now required)
- Updated default search provider to SearXNG
### Removed
- `src/credentials.py` - credentials now managed via environment variables
- Docker socket fallback methods from Portainer client
- Unused search API settings (Brave, Google)
## [1.3.0] - 2025-12-31
### Added
+68
View File
@@ -0,0 +1,68 @@
# Alembic Configuration for Core-API
#
# Database migrations using async SQLAlchemy
[alembic]
# Path to migration scripts
script_location = alembic
# Template for migration file names
file_template = %%(year)d%%(month).2d%%(day).2d_%%(hour).2d%%(minute).2d_%%(rev)s_%%(slug)s
# Prepend sys.path with the project root
prepend_sys_path = .
# Timezone for revision creation date
timezone = UTC
# Max length of revision identifiers
truncate_slug_length = 40
# Set to 'true' to run in offline mode
revision_environment = false
# Set to 'true' for sqlalchemy.url to be from the environment
# We use env.py to get the URL from config.py instead
sqlalchemy.url =
[post_write_hooks]
# Black formatting on generated migration files
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 100 REVISION_SCRIPT_FILENAME
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
+21
View File
@@ -0,0 +1,21 @@
Alembic Migrations for Core-API
This directory contains database migrations managed by Alembic.
Commands:
# Generate a new migration (after changing models)
alembic revision --autogenerate -m "description"
# Apply all pending migrations
alembic upgrade head
# Rollback last migration
alembic downgrade -1
# View migration history
alembic history
# View current revision
alembic current
See https://alembic.sqlalchemy.org for more documentation.
+98
View File
@@ -0,0 +1,98 @@
"""
Alembic Environment Configuration
Async migration environment for SQLAlchemy 2.0 with asyncpg.
"""
import asyncio
from logging.config import fileConfig
from sqlalchemy import pool
from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context
# Import our models and config
from src.config import get_settings
from src.db.database import Base
# Import all models to ensure they're registered with Base.metadata
from src.db.models import User, Role, UserRole, UserPreferences, ApiKey # noqa: F401
# Alembic Config object
config = context.config
# Interpret the config file for Python logging
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# Model metadata for autogenerate support
target_metadata = Base.metadata
# Get database URL from our settings
settings = get_settings()
db_url = settings.database_url
if db_url.startswith("postgresql://"):
db_url = db_url.replace("postgresql://", "postgresql+asyncpg://", 1)
def run_migrations_offline() -> None:
"""
Run migrations in 'offline' mode.
Generates SQL script without connecting to the database.
"""
context.configure(
url=db_url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection: Connection) -> None:
"""
Run migrations with the given connection.
"""
context.configure(
connection=connection,
target_metadata=target_metadata,
)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations() -> None:
"""
Run migrations in 'online' mode with async engine.
"""
configuration = config.get_section(config.config_ini_section) or {}
configuration["sqlalchemy.url"] = db_url
connectable = async_engine_from_config(
configuration,
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
"""
Run migrations in 'online' mode.
"""
asyncio.run(run_async_migrations())
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
+25
View File
@@ -0,0 +1,25 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}
@@ -0,0 +1,138 @@
"""Create auth tables
Revision ID: 001
Revises:
Create Date: 2026-01-01
Creates the initial authentication and authorization tables:
- users: User accounts synced from Authentik
- roles: Domain-scoped permission roles
- user_roles: User-Role association table
- user_preferences: User settings and preferences
- api_keys: API key authentication
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "001"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Users table
op.create_table(
"users",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("authentik_id", postgresql.UUID(as_uuid=True), nullable=False),
sa.Column("email", sa.String(255), nullable=False),
sa.Column("name", sa.String(255), nullable=False),
sa.Column("avatar_url", sa.String(500), nullable=True),
sa.Column("api_keys_enabled", sa.Boolean(), nullable=False, server_default="true"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("last_login", sa.DateTime(timezone=True), nullable=True),
)
op.create_index("ix_users_authentik_id", "users", ["authentik_id"], unique=True)
op.create_index("ix_users_email", "users", ["email"], unique=True)
# Roles table
op.create_table(
"roles",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("name", sa.String(100), nullable=False, comment="Role name in format domain:action"),
sa.Column("domain", sa.String(50), nullable=False, comment="Permission domain"),
sa.Column("action", sa.String(20), nullable=False, comment="Permission action"),
sa.Column("authentik_group", sa.String(255), nullable=True, comment="Corresponding Authentik group name"),
)
op.create_index("ix_roles_name", "roles", ["name"], unique=True)
op.create_index("ix_roles_domain", "roles", ["domain"])
op.create_index("ix_roles_authentik_group", "roles", ["authentik_group"], unique=True)
# User-Role association table
op.create_table(
"user_roles",
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
sa.Column("role_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("roles.id", ondelete="CASCADE"), primary_key=True),
)
# User preferences table
op.create_table(
"user_preferences",
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
sa.Column("theme", sa.String(20), nullable=False, server_default="system", comment="Theme preference"),
sa.Column("default_room", sa.String(50), nullable=False, server_default="front-hall", comment="Default room"),
sa.Column("preferences_json", postgresql.JSONB(), nullable=False, server_default="{}"),
)
# API keys table
op.create_table(
"api_keys",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("user_id", postgresql.UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
sa.Column("name", sa.String(100), nullable=False, comment="Human-readable key name"),
sa.Column("key_hash", sa.String(255), nullable=False, comment="SHA-256 hash of the API key"),
sa.Column("key_prefix", sa.String(8), nullable=False, comment="First 8 chars for identification"),
sa.Column("scopes", postgresql.ARRAY(sa.String()), nullable=True, comment="Optional scope restriction"),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_api_keys_user_id", "api_keys", ["user_id"])
# Seed initial roles (domain:action combinations)
roles_table = sa.table(
"roles",
sa.column("id", postgresql.UUID),
sa.column("name", sa.String),
sa.column("domain", sa.String),
sa.column("action", sa.String),
sa.column("authentik_group", sa.String),
)
domains = [
"control-room",
"library",
"media",
"ai",
"housekeeper",
"developer",
"documents",
"gaming",
"admin",
]
actions = ["viewer", "user", "editor", "admin"]
roles_data = []
for domain in domains:
for action in actions:
role_name = f"{domain}:{action}"
authentik_group = f"tatlock-{domain}-{action}"
roles_data.append({
"id": sa.text("gen_random_uuid()"),
"name": role_name,
"domain": domain,
"action": action,
"authentik_group": authentik_group,
})
# Insert roles using raw SQL for UUID generation
for role in roles_data:
op.execute(
f"""
INSERT INTO roles (id, name, domain, action, authentik_group)
VALUES (gen_random_uuid(), '{role["name"]}', '{role["domain"]}', '{role["action"]}', '{role["authentik_group"]}')
"""
)
def downgrade() -> None:
op.drop_table("api_keys")
op.drop_table("user_preferences")
op.drop_table("user_roles")
op.drop_table("roles")
op.drop_table("users")
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "core-api"
version = "1.3.0"
version = "1.4.0"
description = "Core Code API - Infrastructure management and tools API"
readme = "README.md"
requires-python = ">=3.12"
+14 -8
View File
@@ -1,12 +1,13 @@
# FastAPI and ASGI server
fastapi~=0.115.0
uvicorn[standard]>=0.34.0 # Updated for google-adk compatibility
pydantic>=2.11.1,<3.0.0 # Required for google-cloud-aiplatform[agent-engines]
fastapi>=0.115.0
starlette>=0.49.1 # CVE-2025-54121, CVE-2025-62727
uvicorn[standard]>=0.34.0
pydantic>=2.11.1,<3.0.0
pydantic-settings>=2.10.1
# HTTP client
httpx>=0.28.0 # Required for google-adk
python-socketio[asyncio_client]~=5.11.0
httpx>=0.28.0
python-socketio[asyncio_client]>=5.14.0 # CVE-2025-61765
# Web scraping
beautifulsoup4~=4.12.0
@@ -22,6 +23,11 @@ pytz~=2024.1
dnspython~=2.7.0
# Authentication & Security
PyJWT[crypto]~=2.9.0
python-jose[cryptography]~=3.3.0
cryptography~=43.0.0
PyJWT[crypto]>=2.9.0
python-jose[cryptography]>=3.4.0 # CVE PYSEC-2024-232, PYSEC-2024-233
cryptography>=44.0.1 # CVE-2024-12797
# Database
sqlalchemy[asyncio]~=2.0.0
asyncpg>=0.30.0
alembic~=1.13.0
+22
View File
@@ -3,3 +3,25 @@ Authentication module for core-api
Provides OIDC/OAuth2 authentication via Authentik
"""
from src.auth.oidc import (
get_current_user,
get_admin_user,
get_optional_user,
get_forward_auth_user,
get_forward_auth_admin,
oidc_config,
)
from src.auth.service import AuthService, get_auth_service
from src.auth.controller import auth_controller
__all__ = [
"get_current_user",
"get_admin_user",
"get_optional_user",
"get_forward_auth_user",
"get_forward_auth_admin",
"oidc_config",
"AuthService",
"get_auth_service",
"auth_controller",
]
+183
View File
@@ -0,0 +1,183 @@
"""
Authentication Controller
Provides authentication endpoints for OIDC token sync and user management.
"""
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi.responses import JSONResponse
from sqlalchemy.ext.asyncio import AsyncSession
from src.controllers.base import BaseController
from src.logging_config import get_logger
from src.db import get_async_session
from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema
from src.auth.service import AuthService
logger = get_logger(__name__)
class AuthController(BaseController):
"""
Controller for authentication operations
Provides endpoints for:
- Token synchronization (login)
- User profile retrieval
"""
def __init__(self):
super().__init__(prefix="/auth", tags=["Authentication"])
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
@router.post(
"/sync",
summary="Sync user from OIDC token",
response_model=AuthSyncResponse,
responses={
200: {"description": "User synced successfully"},
401: {"description": "Invalid or expired token"},
503: {"description": "Authentication service unavailable"},
},
)
async def sync_user(
request: AuthSyncRequest,
session: AsyncSession = Depends(get_async_session),
) -> AuthSyncResponse:
"""
Synchronize user from OIDC access token
This endpoint should be called after the client obtains an access token
from Authentik. It:
1. Validates the token via Authentik's userinfo endpoint
2. Creates or updates the user in the database
3. Syncs roles from Authentik groups
4. Returns the user profile with roles and preferences
The client should store the returned user info for local use.
"""
service = AuthService(session)
try:
# Validate token with Authentik
token_info = await service.validate_token(request.access_token)
except ValueError as e:
logger.warning(f"Token validation failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
# Sync user to database
user, is_new = await service.sync_user(token_info)
# Sync roles from groups
roles = await service.sync_roles(user, token_info.groups)
# Commit the transaction
await session.commit()
# Refresh to get relationships
await session.refresh(user, ["preferences"])
# Build response
return AuthSyncResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(roles),
preferences=service.preferences_to_schema(user.preferences),
is_new_user=is_new,
)
@router.get(
"/users",
summary="List all users",
response_model=UsersListResponse,
responses={
200: {"description": "List of users"},
},
)
async def list_users(
search: Optional[str] = Query(None, description="Search by name or email"),
offset: int = Query(0, ge=0, description="Number of records to skip"),
limit: int = Query(50, ge=1, le=100, description="Maximum records to return"),
session: AsyncSession = Depends(get_async_session),
) -> UsersListResponse:
"""
List all users who have logged in via Authentik
Returns paginated list of users with their roles.
Supports search filtering by name or email.
"""
service = AuthService(session)
items, total = await service.list_users(
search=search,
offset=offset,
limit=limit,
)
return UsersListResponse(items=items, total=total)
@router.post(
"/users/sync-from-authentik",
summary="Bulk sync users from Authentik",
response_model=BulkSyncResultSchema,
responses={
200: {"description": "Sync completed"},
401: {"description": "Authentik API token invalid"},
503: {"description": "Authentik service unavailable"},
},
)
async def sync_users_from_authentik(
session: AsyncSession = Depends(get_async_session),
) -> BulkSyncResultSchema:
"""
Fetch all users from Authentik and sync to local database
This endpoint uses the Authentik admin API to fetch all users
and create/update them in the local database. Requires
AUTHENTIK_CORE_API_TOKEN to be configured.
Use this to initially populate users or to re-sync after
changes in Authentik.
"""
service = AuthService(session)
try:
result = await service.bulk_sync_from_authentik()
logger.info(
f"Bulk sync completed: {result.created} created, "
f"{result.updated} updated, {result.failed} failed"
)
return result
except ValueError as e:
logger.error(f"Bulk sync failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
@router.get(
"/me",
summary="Get current user profile",
response_model=AuthSyncResponse,
responses={
200: {"description": "User profile"},
401: {"description": "Not authenticated"},
},
)
async def get_me(
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
"""
Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key.
For now, returns 501 Not Implemented until session management is added.
"""
# TODO: Implement with get_current_user dependency
raise HTTPException(
status_code=501,
detail="Not implemented - use /auth/sync with access token",
)
return router
# Create controller instance
auth_controller = AuthController()
+110
View File
@@ -0,0 +1,110 @@
"""
Authentication Schemas
Pydantic models for auth request/response payloads.
"""
import uuid
from datetime import datetime
from typing import Optional
from pydantic import Field
from src.base_schema import BaseSchema
class AuthSyncRequest(BaseSchema):
"""
Request payload for POST /auth/sync
The client sends this after obtaining an OIDC token from Authentik.
The access_token is validated against Authentik's userinfo endpoint.
"""
access_token: str = Field(
...,
description="OIDC access token from Authentik",
)
class RoleSchema(BaseSchema):
"""Role information in domain:action format"""
name: str = Field(..., description="Role name (e.g., 'control-room:admin')")
domain: str = Field(..., description="Permission domain (e.g., 'control-room')")
action: str = Field(..., description="Permission action (e.g., 'admin')")
class UserPreferencesSchema(BaseSchema):
"""User preferences"""
theme: str = Field(default="system", description="Theme preference: system, light, dark")
default_room: str = Field(default="front-hall", description="Default room for housekeeping")
preferences_json: dict = Field(default_factory=dict, description="Extended preferences")
class UserSchema(BaseSchema):
"""User information returned from sync"""
id: uuid.UUID = Field(..., description="Internal user ID")
authentik_id: uuid.UUID = Field(..., description="Authentik user ID")
email: str = Field(..., description="User email")
name: str = Field(..., description="Display name")
avatar_url: Optional[str] = Field(None, description="Profile picture URL")
created_at: datetime = Field(..., description="Account creation timestamp")
last_login: Optional[datetime] = Field(None, description="Last login timestamp")
class AuthSyncResponse(BaseSchema):
"""
Response from POST /auth/sync
Contains the synced user profile, roles, and preferences.
"""
user: UserSchema = Field(..., description="User profile")
roles: list[RoleSchema] = Field(..., description="User's permission roles")
preferences: UserPreferencesSchema = Field(..., description="User preferences")
is_new_user: bool = Field(..., description="True if user was just created")
class TokenInfoSchema(BaseSchema):
"""
Token information from Authentik userinfo endpoint
This is what Authentik returns when validating an access token.
"""
sub: str = Field(..., description="Subject (Authentik user ID)")
email: str = Field(..., description="User email")
name: Optional[str] = Field(None, description="Display name")
preferred_username: Optional[str] = Field(None, description="Username")
groups: list[str] = Field(default_factory=list, description="Group memberships")
picture: Optional[str] = Field(None, description="Profile picture URL")
class UserListItemSchema(BaseSchema):
"""User item for list display"""
id: uuid.UUID = Field(..., description="Internal user ID")
email: str = Field(..., description="User email")
name: str = Field(..., description="Display name")
avatar_url: Optional[str] = Field(None, description="Profile picture URL")
created_at: datetime = Field(..., description="Account creation timestamp")
last_login: Optional[datetime] = Field(None, description="Last login timestamp")
roles: list[str] = Field(default_factory=list, description="Role names")
class UsersListResponse(BaseSchema):
"""Response from GET /auth/users"""
items: list[UserListItemSchema] = Field(..., description="List of users")
total: int = Field(..., description="Total count of users")
class BulkSyncResultSchema(BaseSchema):
"""Result from bulk sync operation"""
created: int = Field(..., description="Number of users created")
updated: int = Field(..., description="Number of users updated")
failed: int = Field(..., description="Number of users that failed to sync")
total_in_authentik: int = Field(..., description="Total users in Authentik")
errors: list[str] = Field(default_factory=list, description="Error messages for failed syncs")
+453
View File
@@ -0,0 +1,453 @@
"""
Authentication Service
Business logic for user synchronization from Authentik.
"""
import uuid
from datetime import datetime, timezone
from typing import Optional
import httpx
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from src.config import get_settings
from src.logging_config import get_logger
from src.db.models import User, Role, UserPreferences
from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema
logger = get_logger(__name__)
settings = get_settings()
class AuthService:
"""
Service for authentication and user synchronization
Handles:
- Token validation via Authentik userinfo endpoint
- User creation/update from OIDC claims
- Role synchronization from Authentik groups
"""
def __init__(self, session: AsyncSession):
"""
Initialize auth service
Args:
session: Async database session
"""
self.session = session
self.userinfo_url = f"{settings.authentik_url}/application/o/userinfo/"
async def validate_token(self, access_token: str) -> TokenInfoSchema:
"""
Validate access token via Authentik userinfo endpoint
Args:
access_token: OIDC access token
Returns:
Token info containing user claims
Raises:
ValueError: If token is invalid or expired
"""
try:
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.get(
self.userinfo_url,
headers={"Authorization": f"Bearer {access_token}"},
)
if response.status_code == 401:
raise ValueError("Invalid or expired token")
response.raise_for_status()
data = response.json()
logger.debug(f"Userinfo response: {data}")
return TokenInfoSchema(
sub=data.get("sub"),
email=data.get("email"),
name=data.get("name") or data.get("preferred_username"),
preferred_username=data.get("preferred_username"),
groups=data.get("groups", []),
picture=data.get("picture"),
)
except httpx.HTTPStatusError as e:
logger.error(f"Authentik userinfo request failed: {e}")
raise ValueError(f"Token validation failed: {e.response.status_code}")
except httpx.RequestError as e:
logger.error(f"Authentik userinfo request error: {e}")
raise ValueError("Authentication service unavailable")
async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]:
"""
Create or update user from OIDC token info
Args:
token_info: Validated token information
Returns:
Tuple of (User, is_new_user)
"""
authentik_id = uuid.UUID(token_info.sub)
# Try to find existing user
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
user = result.scalar_one_or_none()
is_new = user is None
if is_new:
# Create new user
user = User(
authentik_id=authentik_id,
email=token_info.email,
name=token_info.name or token_info.email,
avatar_url=token_info.picture,
last_login=datetime.now(timezone.utc),
)
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
preferences = UserPreferences(user_id=user.id)
self.session.add(preferences)
logger.info(f"Created new user: {token_info.email}")
else:
# Update existing user
user.email = token_info.email
user.name = token_info.name or token_info.email
user.avatar_url = token_info.picture
user.last_login = datetime.now(timezone.utc)
logger.info(f"Updated existing user: {token_info.email}")
await self.session.flush()
return user, is_new
async def sync_roles(self, user: User, groups: list[str]) -> list[Role]:
"""
Synchronize user roles from Authentik groups
Maps Authentik groups (e.g., 'tatlock-control-room-admin')
to application roles (e.g., 'control-room:admin').
Args:
user: User to sync roles for
groups: List of Authentik group names
Returns:
List of synced Role objects
"""
# Get all roles that match the user's Authentik groups
stmt = select(Role).where(Role.authentik_group.in_(groups))
result = await self.session.execute(stmt)
matching_roles = list(result.scalars().all())
# Clear existing roles and set new ones
user.roles = matching_roles
role_names = [r.name for r in matching_roles]
logger.info(f"Synced roles for {user.email}: {role_names}")
return matching_roles
def user_to_schema(self, user: User) -> UserSchema:
"""Convert User model to schema"""
return UserSchema(
id=user.id,
authentik_id=user.authentik_id,
email=user.email,
name=user.name,
avatar_url=user.avatar_url,
created_at=user.created_at,
last_login=user.last_login,
)
def roles_to_schema(self, roles: list[Role]) -> list[RoleSchema]:
"""Convert Role models to schemas"""
return [
RoleSchema(name=r.name, domain=r.domain, action=r.action)
for r in roles
]
def preferences_to_schema(self, preferences: Optional[UserPreferences]) -> UserPreferencesSchema:
"""Convert UserPreferences model to schema"""
if preferences is None:
return UserPreferencesSchema()
return UserPreferencesSchema(
theme=preferences.theme,
default_room=preferences.default_room,
preferences_json=preferences.preferences_json or {},
)
async def list_users(
self,
search: Optional[str] = None,
offset: int = 0,
limit: int = 50,
) -> tuple[list[UserListItemSchema], int]:
"""
List all users with optional search and pagination
Args:
search: Optional search query (matches name or email)
offset: Number of records to skip
limit: Maximum number of records to return
Returns:
Tuple of (list of user schemas, total count)
"""
from sqlalchemy import func
# Base query with roles loaded
base_query = select(User).options(selectinload(User.roles))
# Apply search filter if provided
if search:
search_filter = f"%{search}%"
base_query = base_query.where(
(User.name.ilike(search_filter)) | (User.email.ilike(search_filter))
)
# Get total count
count_query = select(func.count()).select_from(base_query.subquery())
total_result = await self.session.execute(count_query)
total = total_result.scalar() or 0
# Apply pagination and ordering
query = base_query.order_by(User.name).offset(offset).limit(limit)
result = await self.session.execute(query)
users = list(result.scalars().all())
# Convert to schemas
items = [
UserListItemSchema(
id=user.id,
email=user.email,
name=user.name,
avatar_url=user.avatar_url,
created_at=user.created_at,
last_login=user.last_login,
roles=[role.name for role in user.roles],
)
for user in users
]
return items, total
def _get_csrf_token(self, client: httpx.AsyncClient) -> str:
"""Extract CSRF token from cookies"""
for cookie in client.cookies.jar:
if cookie.name == "authentik_csrf":
return cookie.value
return ""
async def _authentik_session_login(self, client: httpx.AsyncClient) -> None:
"""
Authenticate with Authentik using the flow API to establish a session
Authentik's flow API requires:
1. Cookie persistence between requests
2. X-authentik-CSRF header set to the authentik_csrf cookie value
3. Multi-stage flow handling (identification -> password -> done)
Args:
client: httpx client with cookie persistence
Raises:
ValueError: If authentication fails
"""
flow_url = f"{settings.authentik_url}/api/v3/flows/executor/default-authentication-flow/"
# Step 1: Get the initial flow challenge (this sets the session and csrf cookies)
resp = await client.get(flow_url, headers={"Accept": "application/json"})
resp.raise_for_status()
data = resp.json()
logger.debug(f"Flow initial response: component={data.get('component')}, type={data.get('type')}")
# Get CSRF token for subsequent requests
csrf_token = self._get_csrf_token(client)
logger.debug(f"CSRF token obtained: {bool(csrf_token)}")
# Build headers with CSRF token
headers = {
"Accept": "application/json",
"Content-Type": "application/json",
}
if csrf_token:
headers["X-authentik-CSRF"] = csrf_token
# Step 2: Handle identification stage - submit username
if data.get("component") == "ak-stage-identification":
resp = await client.post(
flow_url,
json={"uid_field": settings.authentik_admin_user},
headers=headers,
)
resp.raise_for_status()
data = resp.json()
logger.debug(f"After username: component={data.get('component')}, type={data.get('type')}")
# Update CSRF token (might change between stages)
csrf_token = self._get_csrf_token(client)
if csrf_token:
headers["X-authentik-CSRF"] = csrf_token
# Step 3: Handle password stage if required
if data.get("component") == "ak-stage-password":
resp = await client.post(
flow_url,
json={"password": settings.authentik_admin_password},
headers=headers,
)
resp.raise_for_status()
data = resp.json()
logger.debug(f"After password: component={data.get('component')}, type={data.get('type')}")
# Check for access denied
if data.get("component") == "ak-stage-access-denied":
raise ValueError("Authentik authentication failed: access denied")
# Check for redirect (successful auth)
if data.get("type") == "redirect" or data.get("to"):
logger.info("Successfully authenticated with Authentik via flow")
return
# If we're still in identification stage, the username might be wrong
if data.get("component") == "ak-stage-identification":
response_errors = data.get("response_errors", {})
raise ValueError(f"Authentication stuck at identification stage: {response_errors}")
logger.info(f"Authentik flow completed with component: {data.get('component')}")
async def bulk_sync_from_authentik(self) -> BulkSyncResultSchema:
"""
Fetch all users from Authentik admin API and sync to local database
Returns:
BulkSyncResultSchema with counts of created/updated/failed users
"""
if not settings.authentik_admin_user or not settings.authentik_admin_password:
raise ValueError("AUTHENTIK_ADMIN_USER and AUTHENTIK_ADMIN_PASSWORD must be configured")
created = 0
updated = 0
failed = 0
errors = []
total_in_authentik = 0
try:
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
# Authenticate with Authentik to get session
await self._authentik_session_login(client)
# Fetch users from Authentik admin API using session
response = await client.get(
f"{settings.authentik_url}/api/v3/core/users/",
params={"page_size": 500},
headers={"Accept": "application/json"},
)
if response.status_code == 401:
raise ValueError("Authentik API token is invalid or expired")
response.raise_for_status()
data = response.json()
authentik_users = data.get("results", [])
total_in_authentik = data.get("pagination", {}).get("count", len(authentik_users))
for auth_user in authentik_users:
try:
# Skip service accounts and inactive users
if auth_user.get("type") == "service_account":
continue
if not auth_user.get("is_active", True):
continue
# Extract user data from Authentik
authentik_id = uuid.UUID(auth_user["pk"])
email = auth_user.get("email") or f"{auth_user['username']}@local"
name = auth_user.get("name") or auth_user.get("username", "Unknown")
avatar_url = auth_user.get("avatar")
# Get user's groups for role mapping
groups = []
groups_summary = auth_user.get("groups_obj", [])
for group in groups_summary:
groups.append(group.get("name", ""))
# Check if user exists
stmt = select(User).where(User.authentik_id == authentik_id)
result = await self.session.execute(stmt)
user = result.scalar_one_or_none()
if user is None:
# Create new user
user = User(
authentik_id=authentik_id,
email=email,
name=name,
avatar_url=avatar_url,
)
self.session.add(user)
await self.session.flush()
# Create default preferences
preferences = UserPreferences(user_id=user.id)
self.session.add(preferences)
created += 1
logger.info(f"Created user from Authentik: {email}")
else:
# Update existing user
user.email = email
user.name = name
user.avatar_url = avatar_url
updated += 1
logger.info(f"Updated user from Authentik: {email}")
# Sync roles from groups
await self.sync_roles(user, groups)
except Exception as e:
failed += 1
error_msg = f"Failed to sync user {auth_user.get('username', 'unknown')}: {str(e)}"
errors.append(error_msg)
logger.warning(error_msg)
# Commit all changes
await self.session.commit()
except httpx.HTTPStatusError as e:
raise ValueError(f"Authentik API error: {e.response.status_code}")
except httpx.RequestError as e:
raise ValueError(f"Failed to connect to Authentik: {str(e)}")
return BulkSyncResultSchema(
created=created,
updated=updated,
failed=failed,
total_in_authentik=total_in_authentik,
errors=errors,
)
# Factory function for dependency injection
def get_auth_service(session: AsyncSession) -> AuthService:
"""Create AuthService instance with database session"""
return AuthService(session)
-197
View File
@@ -1,197 +0,0 @@
"""
Core-AI HTTP Client
Provides interface to Core-AI service for AI performance metrics.
"""
import httpx
from typing import Optional, Dict, List, Any
from src.logging_config import get_logger
from src.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class CoreAIClient:
"""
HTTP client for Core-AI service
Provides access to AI performance metrics, tool execution stats,
and memory system monitoring.
"""
def __init__(
self,
base_url: Optional[str] = None,
timeout: int = 10
):
"""
Initialize Core-AI client
Args:
base_url: Core-AI base URL (default from settings)
timeout: Request timeout in seconds
"""
self.base_url = (base_url or getattr(settings, 'core_ai_base_url', 'http://core-ai:8086')).rstrip("/")
self.timeout = timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
async def close(self):
"""Close the HTTP client"""
await self.client.aclose()
async def health_check(self) -> bool:
"""
Check if Core-AI service is accessible
Returns:
True if accessible, False otherwise
"""
try:
response = await self.client.get(f"{self.base_url}/health")
return response.status_code == 200
except Exception as e:
logger.error(f"Core-AI health check failed: {e}")
return False
async def get_metrics(self) -> Dict[str, Any]:
"""
Get comprehensive AI performance metrics
Returns:
Dict with agent performance, tool execution, memory stats
Example:
{
"uptime_seconds": 3600,
"timestamp": "2025-12-03T20:00:00Z",
"agent": {
"total_requests": 100,
"avg_response_time_ms": 1250.5,
"p95_response_time_ms": 3200.0,
...
},
"tools": {
"total_calls": 250,
"success_rate": 0.98,
"top_tools": {...}
},
"memory": {
"tier1_hit_rate": 0.85,
...
},
...
}
"""
try:
response = await self.client.get(f"{self.base_url}/metrics")
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
logger.error(f"Failed to get metrics: HTTP {e.response.status_code}")
raise
except Exception as e:
logger.error(f"Failed to get metrics: {e}")
raise
async def get_recent_errors(self, limit: int = 20) -> List[Dict[str, Any]]:
"""
Get recent request errors
Args:
limit: Maximum number of errors to return
Returns:
List of error records with timestamps
Example:
[
{
"timestamp": "2025-12-03T19:45:12Z",
"agent_type": "pydantic",
"error": "Connection timeout",
"duration_ms": 5000
},
...
]
"""
try:
response = await self.client.get(
f"{self.base_url}/metrics/errors",
params={"limit": limit}
)
response.raise_for_status()
data = response.json()
return data.get("errors", [])
except Exception as e:
logger.error(f"Failed to get recent errors: {e}")
raise
async def get_tool_failures(self, limit: int = 20) -> List[Dict[str, Any]]:
"""
Get recent tool execution failures
Args:
limit: Maximum number of failures to return
Returns:
List of tool failure records
Example:
[
{
"timestamp": "2025-12-03T19:50:30Z",
"tool_name": "list_containers",
"error": "Connection refused",
"duration_ms": 150
},
...
]
"""
try:
response = await self.client.get(
f"{self.base_url}/metrics/tool-failures",
params={"limit": limit}
)
response.raise_for_status()
data = response.json()
return data.get("failures", [])
except Exception as e:
logger.error(f"Failed to get tool failures: {e}")
raise
async def reset_metrics(self) -> bool:
"""
Reset all metrics (admin operation)
Returns:
True if successful
"""
try:
response = await self.client.post(f"{self.base_url}/metrics/reset")
response.raise_for_status()
logger.info("Successfully reset Core-AI metrics")
return True
except Exception as e:
logger.error(f"Failed to reset metrics: {e}")
raise
async def __aenter__(self):
"""Async context manager entry"""
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""Async context manager exit"""
await self.close()
# Singleton instance
_ai_client: Optional[CoreAIClient] = None
def get_ai_client() -> CoreAIClient:
"""Get singleton Core-AI client instance"""
global _ai_client
if _ai_client is None:
_ai_client = CoreAIClient()
return _ai_client
+12 -102
View File
@@ -438,70 +438,14 @@ class PortainerClient:
return True
# ========================================================================
# Docker Socket Fallback (for containers not managed by Portainer)
# ========================================================================
async def _list_containers_via_socket(self, all_containers: bool = True) -> List[Dict[str, Any]]:
"""
Fallback: List containers directly via Docker socket
Used when Portainer API doesn't return complete data (e.g., containers
started outside Portainer, AMP game servers, etc.)
Args:
all_containers: Include stopped containers
Returns:
List of container details in Docker API format
"""
try:
# Docker socket is mounted at /var/run/docker.sock
# Use httpx with unix socket transport
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
params = {"all": 1 if all_containers else 0}
response = await client.get(
"http://localhost/v1.41/containers/json",
params=params
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.warning(f"Docker socket fallback failed: {e}")
return []
async def _inspect_container_via_socket(self, container_id_or_name: str) -> Optional[Dict[str, Any]]:
"""
Fallback: Inspect container directly via Docker socket
Args:
container_id_or_name: Container ID or name
Returns:
Container details or None
"""
try:
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
response = await client.get(
f"http://localhost/v1.41/containers/{container_id_or_name}/json"
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.warning(f"Docker socket inspect fallback failed for '{container_id_or_name}': {e}")
return None
# ========================================================================
# Helper methods for agent tools (auto-detect endpoint + fallback)
# Helper methods for agent tools (auto-detect endpoint)
# ========================================================================
async def list_containers(self, all_containers: bool = True) -> List[Dict[str, Any]]:
"""
List containers using auto-detected endpoint with Docker socket fallback
List containers using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint.
If Portainer doesn't have complete data, falls back to Docker socket.
Args:
all_containers: Include stopped containers (default: True)
@@ -509,34 +453,18 @@ class PortainerClient:
Returns:
List of container details
"""
try:
# Try Portainer first
endpoints = await self.get_endpoints()
if endpoints:
if not endpoints:
raise RuntimeError("No Portainer endpoints available")
endpoint_id = endpoints[0]["Id"]
containers = await self.get_containers(endpoint_id, all_containers)
if containers:
return containers
# Fallback to Docker socket
logger.info("Portainer returned no containers, trying Docker socket fallback...")
return await self._list_containers_via_socket(all_containers)
except Exception as e:
logger.error(f"Error listing containers: {e}")
# Try fallback even on exception
try:
return await self._list_containers_via_socket(all_containers)
except Exception as fallback_error:
logger.error(f"Fallback also failed: {fallback_error}")
return []
return await self.get_containers(endpoint_id, all_containers)
async def inspect_container(self, container_name: str) -> Optional[Dict[str, Any]]:
"""
Inspect a container by name using auto-detected endpoint with Docker socket fallback
Inspect a container by name using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint.
If Portainer doesn't find the container, falls back to Docker socket.
Args:
container_name: Container name (e.g., "jellyfin", "ollama")
@@ -544,43 +472,25 @@ class PortainerClient:
Returns:
Container details or None if not found
"""
try:
# Try Portainer first
endpoints = await self.get_endpoints()
if endpoints:
if not endpoints:
raise RuntimeError("No Portainer endpoints available")
endpoint_id = endpoints[0]["Id"]
# First list all containers to find the one matching the name
# List all containers to find the one matching the name
all_containers = await self.get_containers(endpoint_id, all_containers=True)
matching_container = None
for container in all_containers:
# Container names come as array like ['/jellyfin']
names = container.get('Names', [])
for name in names:
clean_name = name.lstrip('/')
if clean_name == container_name or clean_name.lower() == container_name.lower():
matching_container = container
break
if matching_container:
break
if matching_container:
# Get detailed info using container ID
container_id = matching_container['Id']
container_id = container['Id']
return await self.get_container(endpoint_id, container_id)
# Not found in Portainer, try Docker socket fallback
logger.info(f"Container '{container_name}' not found in Portainer, trying Docker socket fallback...")
return await self._inspect_container_via_socket(container_name)
except Exception as e:
logger.error(f"Error inspecting container '{container_name}': {e}")
# Try fallback even on exception
try:
return await self._inspect_container_via_socket(container_name)
except Exception as fallback_error:
logger.error(f"Fallback also failed: {fallback_error}")
return None
+34 -46
View File
@@ -1,5 +1,8 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
@@ -20,29 +23,6 @@ def _get_version_from_pyproject() -> str:
__version__ = _get_version_from_pyproject()
# Import infrastructure credentials from gitignored module
try:
from src.credentials import (
PORTAINER_URL, PORTAINER_API_KEY,
NPM_URL, NPM_EMAIL, NPM_PASSWORD,
BRAVE_SEARCH_API_KEY,
GOOGLE_SEARCH_API_KEY, GOOGLE_SEARCH_ENGINE_ID,
HOMEASSISTANT_URL, HOMEASSISTANT_TOKEN
)
except ImportError:
# Fallback to empty strings if credentials.py doesn't exist
# (e.g., fresh clone before credentials setup)
PORTAINER_URL = "http://localhost:8001"
PORTAINER_API_KEY = ""
NPM_URL = "http://localhost:81"
NPM_EMAIL = ""
NPM_PASSWORD = ""
BRAVE_SEARCH_API_KEY = ""
GOOGLE_SEARCH_API_KEY = ""
GOOGLE_SEARCH_ENGINE_ID = ""
HOMEASSISTANT_URL = "http://localhost:8123"
HOMEASSISTANT_TOKEN = ""
class Settings(BaseSettings):
"""Global application settings"""
@@ -66,15 +46,13 @@ class Settings(BaseSettings):
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str = "http://ollama:11434"
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-tools:7b"
agent_model: str = "gemma2:9b-instruct-q5_K_M" # Must support tool calling with ADK (~4GB VRAM)
lightweight_models: str = "gemma3-tools:1b,phi3:mini"
heavy_models: str = "mistral:7b,gemma2:9b,gemma3:12b,mixtral:8x7b"
code_models: str = "codestral:latest,codegemma:latest"
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
@@ -109,35 +87,44 @@ class Settings(BaseSettings):
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "google" # Options: google, brave, searxng, duckduckgo
searxng_url: str = "http://searxng:8080" # For future self-hosted SearxNG
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Search API Keys (from credentials.py)
brave_search_api_key: str = BRAVE_SEARCH_API_KEY # https://brave.com/search/api/
google_search_api_key: str = GOOGLE_SEARCH_API_KEY # https://console.cloud.google.com/
google_search_engine_id: str = GOOGLE_SEARCH_ENGINE_ID # Custom Search Engine ID
# Infrastructure Management (Portainer)
portainer_url: str # Required - set PORTAINER_URL in .env
portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
# Infrastructure Management (from credentials.py)
portainer_url: str = PORTAINER_URL
portainer_api_key: str = PORTAINER_API_KEY
npm_url: str = NPM_URL
npm_email: str = NPM_EMAIL
npm_password: str = NPM_PASSWORD
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required - set NPM_URL in .env
npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
# Home Assistant Configuration
homeassistant_url: str = HOMEASSISTANT_URL
homeassistant_token: str = HOMEASSISTANT_TOKEN
homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30
# Core-AI Service (AI performance metrics)
core_ai_base_url: str = "http://core-ai:8086"
# PostgreSQL Database
postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Authentik API (for token validation and user management)
authentik_url: str = "http://192.168.86.149:9000" # Authentik base URL
authentik_admin_user: str = "" # Admin username for API access
authentik_admin_password: str = "" # Admin password for API access
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
@@ -163,6 +150,7 @@ class Settings(BaseSettings):
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache()
-219
View File
@@ -1,219 +0,0 @@
"""
AI Metrics Proxy Controller
Provides proxy endpoints to Core-AI service metrics.
Allows external access to AI performance stats via core-api.
"""
from fastapi import APIRouter, HTTPException
from typing import Dict, List, Any
from src.clients.ai_client import get_ai_client
from src.logging_config import get_logger
logger = get_logger(__name__)
# Create router
router = APIRouter(
prefix="/ai",
tags=["AI Metrics"]
)
@router.get(
"/health",
summary="Check Core-AI service health",
description="Verify that the Core-AI service is accessible and responding"
)
async def ai_health_check():
"""
Check if Core-AI service is healthy
Returns:
Health status and availability
"""
try:
ai_client = get_ai_client()
is_healthy = await ai_client.health_check()
return {
"service": "core-ai",
"status": "healthy" if is_healthy else "unhealthy",
"accessible": is_healthy
}
except Exception as e:
logger.error(f"AI health check failed: {e}")
return {
"service": "core-ai",
"status": "error",
"accessible": False,
"error": str(e)
}
@router.get(
"/metrics",
response_model=Dict[str, Any],
summary="Get comprehensive AI performance metrics",
description="Returns detailed metrics including agent performance, tool execution stats, memory system metrics, and user activity"
)
async def get_ai_metrics():
"""
Proxy endpoint for Core-AI metrics
Returns comprehensive AI performance data:
- Agent request statistics (total, by type, response times)
- Response time percentiles (p50, p95, p99)
- Tool execution metrics (calls, success rates, durations)
- Memory system statistics (cache hits, consolidations)
- User activity tracking
- Concurrency metrics
Returns:
Dict with all collected metrics
Raises:
HTTPException: If Core-AI is unreachable or returns error
"""
try:
ai_client = get_ai_client()
metrics = await ai_client.get_metrics()
return metrics
except Exception as e:
logger.error(f"Failed to fetch AI metrics: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.get(
"/metrics/errors",
response_model=Dict[str, Any],
summary="Get recent request errors",
description="Returns recent AI agent request errors with timestamps and details"
)
async def get_ai_errors(limit: int = 20):
"""
Get recent AI request errors
Args:
limit: Maximum number of errors to return (default: 20)
Returns:
Dict with error list and total count
Example response:
{
"errors": [
{
"timestamp": "2025-12-03T19:45:12Z",
"agent_type": "pydantic",
"error": "Connection timeout",
"duration_ms": 5000
}
],
"total": 1
}
"""
try:
ai_client = get_ai_client()
errors = await ai_client.get_recent_errors(limit=limit)
return {
"errors": errors,
"total": len(errors)
}
except Exception as e:
logger.error(f"Failed to fetch AI errors: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.get(
"/metrics/tool-failures",
response_model=Dict[str, Any],
summary="Get recent tool execution failures",
description="Returns recent tool execution failures with error details"
)
async def get_ai_tool_failures(limit: int = 20):
"""
Get recent tool execution failures
Args:
limit: Maximum number of failures to return (default: 20)
Returns:
Dict with failure list and total count
Example response:
{
"failures": [
{
"timestamp": "2025-12-03T19:50:30Z",
"tool_name": "list_containers",
"error": "Connection refused",
"duration_ms": 150
}
],
"total": 1
}
"""
try:
ai_client = get_ai_client()
failures = await ai_client.get_tool_failures(limit=limit)
return {
"failures": failures,
"total": len(failures)
}
except Exception as e:
logger.error(f"Failed to fetch tool failures: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
@router.post(
"/metrics/reset",
summary="Reset all AI metrics (admin)",
description="Clear all collected metrics. This is an administrative operation that resets all counters and history."
)
async def reset_ai_metrics():
"""
Reset all AI metrics (admin operation)
Clears all collected metrics including:
- Request history
- Tool execution stats
- Memory system metrics
- Error logs
Returns:
Success confirmation
Note:
This is an administrative operation that should be used carefully.
All historical data will be lost.
"""
try:
ai_client = get_ai_client()
await ai_client.reset_metrics()
logger.info("AI metrics reset successfully")
return {
"success": True,
"message": "AI metrics reset successfully"
}
except Exception as e:
logger.error(f"Failed to reset AI metrics: {e}")
raise HTTPException(
status_code=503,
detail=f"Core-AI service unavailable: {str(e)}"
)
+23 -45
View File
@@ -10,10 +10,8 @@ from src.controllers.base import BaseController
from src.config import get_settings
from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database
# Note: Agent functionality moved to separate core-ai service (Dec 2025)
# This service (core-api) only provides infrastructure management and tools
AGENT_AVAILABLE = False
logger = get_logger(__name__)
@@ -52,20 +50,7 @@ class HealthController(BaseController):
"service": settings.app_name,
"version": settings.app_version,
"status": "healthy",
"documentation": {
"swagger_ui": "/docs",
"redoc": "/redoc",
"openapi_spec": "/openapi.json"
},
"endpoints": {
"chat_completions": "/v1/chat/completions",
"models": "/v1/models",
"conversations": "/v1/conversations",
"dns_lookup": "/tools/dns/lookup",
"infrastructure": "/infrastructure",
"health": "/health",
"health_full": "/health/full"
}
"docs": "/docs"
}
@router.get(
@@ -75,18 +60,15 @@ class HealthController(BaseController):
)
async def health_check():
"""
Simple health check endpoint for container orchestration
Fast health check endpoint for container orchestration
Returns a 200 OK status when the service is running properly.
Used by Docker, Kubernetes, and load balancers.
Returns a 200 OK immediately if the service is running.
Does NOT check backend connectivity (use /health/full for that).
Used by Docker, Kubernetes, and load balancers for liveness probes.
"""
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
return {
"status": "healthy",
"version": settings.app_version,
"ollama_connected": ollama_healthy
"version": settings.app_version
}
@router.get(
@@ -147,12 +129,18 @@ class HealthController(BaseController):
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Note: Agent functionality moved to separate core-ai service
# This service only needs Ollama for embeddings (infrastructure tools)
# Agent health is checked separately in core-ai service
# Check 2: Database connection
database = get_database()
db_healthy = False
db_error = None
# Determine overall status (only Ollama required for core-api)
is_healthy = ollama_healthy
try:
db_healthy = await database.health_check()
except Exception as e:
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
@@ -171,7 +159,10 @@ class HealthController(BaseController):
},
"error": ollama_error
},
"note": "AI agent functionality available in separate core-ai service (port 8086)"
"database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy",
"error": db_error
}
}
}
@@ -217,20 +208,7 @@ class HealthController(BaseController):
"error": str(e)
}
# 2. Agent Stack - Moved to separate core-ai service
diagnostics["components"]["agent"] = {
"status": "N/A",
"note": "AI agent functionality moved to separate core-ai service (port 8086)",
"check_url": "http://core-ai:8086/health"
}
# 3. Memory System (Qdrant) - Moved to core-ai service
diagnostics["components"]["qdrant"] = {
"status": "N/A",
"note": "Memory system managed by core-ai service (port 8086)"
}
# 4. Configuration
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
-23
View File
@@ -1,23 +0,0 @@
"""
Infrastructure Credentials Template
INSTRUCTIONS:
1. Copy this file to credentials.py
2. Fill in your actual credentials
3. DO NOT commit credentials.py to version control (it's in .gitignore)
This file should be committed to the repository as a template.
"""
# Portainer Configuration
PORTAINER_URL = "http://localhost:8001"
PORTAINER_API_KEY = "ptr_your_api_token_here" # Create in Portainer UI: User menu → My account → Access tokens
# Nginx Proxy Manager Configuration
NPM_URL = "http://localhost:81"
NPM_EMAIL = "admin@example.com"
NPM_PASSWORD = "your_password_here"
# Home Assistant Configuration
HOMEASSISTANT_URL = "http://192.168.86.149:8123" # Or http://home-assistant:8123 in Docker
HOMEASSISTANT_TOKEN = "your_long_lived_access_token_here" # Create in HA: Profile → Long-Lived Access Tokens
+16
View File
@@ -0,0 +1,16 @@
"""
Database package for Core-API
Provides async PostgreSQL database connectivity using SQLAlchemy 2.0.
"""
from src.db.database import (
get_async_session,
get_database,
Database,
)
__all__ = [
"get_async_session",
"get_database",
"Database",
]
+179
View File
@@ -0,0 +1,179 @@
"""
Database Connection Module
Provides async PostgreSQL connectivity using SQLAlchemy 2.0 with asyncpg driver.
Follows the existing singleton pattern used throughout core-api.
"""
from typing import AsyncGenerator, Optional
from sqlalchemy import text
from sqlalchemy.ext.asyncio import (
AsyncSession,
AsyncEngine,
create_async_engine,
async_sessionmaker,
)
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool
from src.config import get_settings
from src.logging_config import get_logger
logger = get_logger(__name__)
settings = get_settings()
class Base(DeclarativeBase):
"""
SQLAlchemy declarative base for all models
All database models should inherit from this class.
"""
pass
class Database:
"""
Async database connection manager
Provides async engine and session factory for PostgreSQL connections.
Uses asyncpg driver for optimal async performance.
"""
def __init__(self, database_url: Optional[str] = None):
"""
Initialize database connection manager
Args:
database_url: PostgreSQL connection URL (default from settings)
"""
# Convert postgresql:// to postgresql+asyncpg:// for async driver
url = database_url or settings.database_url
if url.startswith("postgresql://"):
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
self._url = url
self._engine: Optional[AsyncEngine] = None
self._session_factory: Optional[async_sessionmaker[AsyncSession]] = None
@property
def engine(self) -> AsyncEngine:
"""
Get or create the async database engine
Returns:
AsyncEngine instance
"""
if self._engine is None:
self._engine = create_async_engine(
self._url,
echo=settings.debug, # Log SQL in debug mode
poolclass=NullPool, # Disable connection pooling for serverless compatibility
)
logger.info(f"Database engine created for {self._url.split('@')[-1]}")
return self._engine
@property
def session_factory(self) -> async_sessionmaker[AsyncSession]:
"""
Get or create the async session factory
Returns:
Session factory for creating database sessions
"""
if self._session_factory is None:
self._session_factory = async_sessionmaker(
bind=self.engine,
class_=AsyncSession,
expire_on_commit=False,
autocommit=False,
autoflush=False,
)
return self._session_factory
async def create_tables(self) -> None:
"""
Create all database tables
Should only be used for development/testing.
Use Alembic migrations for production.
"""
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
logger.info("Database tables created")
async def drop_tables(self) -> None:
"""
Drop all database tables
WARNING: Destroys all data. Use with caution.
"""
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all)
logger.warning("Database tables dropped")
async def health_check(self) -> bool:
"""
Check if database connection is healthy
Returns:
True if connection successful, False otherwise
"""
try:
async with self.session_factory() as session:
await session.execute(text("SELECT 1"))
return True
except Exception as e:
logger.error(f"Database health check failed: {e}")
return False
async def close(self) -> None:
"""
Close database connections and dispose of engine
"""
if self._engine is not None:
await self._engine.dispose()
self._engine = None
self._session_factory = None
logger.info("Database connections closed")
# Singleton instance
_database: Optional[Database] = None
def get_database() -> Database:
"""
Get singleton database instance
Returns:
Database instance
"""
global _database
if _database is None:
_database = Database()
return _database
async def get_async_session() -> AsyncGenerator[AsyncSession, None]:
"""
FastAPI dependency for database sessions
Yields an async session that is automatically closed after the request.
Usage:
@router.get("/items")
async def get_items(session: AsyncSession = Depends(get_async_session)):
result = await session.execute(select(Item))
return result.scalars().all()
Yields:
AsyncSession instance
"""
database = get_database()
async with database.session_factory() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
+17
View File
@@ -0,0 +1,17 @@
"""
SQLAlchemy Models for Core-API
Database models for authentication, authorization, and user management.
"""
from src.db.models.user import User
from src.db.models.role import Role, UserRole
from src.db.models.user_preferences import UserPreferences
from src.db.models.api_key import ApiKey
__all__ = [
"User",
"Role",
"UserRole",
"UserPreferences",
"ApiKey",
]
+98
View File
@@ -0,0 +1,98 @@
"""
API Key Model
Provides API key authentication as fallback for OIDC.
Keys are tied to user accounts and inherit user permissions.
"""
import uuid
from datetime import datetime
from sqlalchemy import String, ForeignKey, DateTime, func
from sqlalchemy.dialects.postgresql import UUID, ARRAY
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.database import Base
from typing import TYPE_CHECKING, List
if TYPE_CHECKING:
from src.db.models.user import User
class ApiKey(Base):
"""
API Key model for programmatic access
API keys provide an alternative to OIDC for:
- Local development without SSO
- Service-to-service communication
- Scripts and automation
Keys inherit the user's roles but can optionally
be restricted to a subset of scopes.
"""
__tablename__ = "api_keys"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
name: Mapped[str] = mapped_column(
String(100),
nullable=False,
comment="Human-readable key name (e.g., 'Dev Laptop', 'CI/CD')",
)
key_hash: Mapped[str] = mapped_column(
String(255),
nullable=False,
comment="SHA-256 hash of the API key",
)
key_prefix: Mapped[str] = mapped_column(
String(8),
nullable=False,
comment="First 8 chars of key for identification (e.g., 'cak_abc1')",
)
scopes: Mapped[List[str] | None] = mapped_column(
ARRAY(String),
nullable=True,
comment="Optional scope restriction (subset of user roles)",
)
expires_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="Optional expiration timestamp",
)
last_used_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="Last time this key was used",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
# Relationships
user: Mapped["User"] = relationship(
"User",
back_populates="api_keys",
)
def __repr__(self) -> str:
return f"<ApiKey {self.key_prefix}... ({self.name})>"
@property
def is_expired(self) -> bool:
"""Check if the API key has expired"""
if self.expires_at is None:
return False
return datetime.now(self.expires_at.tzinfo) > self.expires_at
+93
View File
@@ -0,0 +1,93 @@
"""
Role Models
Defines domain-scoped permissions mapped from Authentik groups.
Format: {domain}:{action} (e.g., control-room:admin, media:viewer)
"""
import uuid
from typing import TYPE_CHECKING, List
from sqlalchemy import String, ForeignKey
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.database import Base
if TYPE_CHECKING:
from src.db.models.user import User
class Role(Base):
"""
Role model for domain-scoped permissions
Roles are seeded from configuration, not user-editable.
Each role maps to an Authentik group (e.g., tatlock-control-room-admin).
Domains: control-room, library, media, ai, housekeeper, developer, documents, gaming, admin
Actions: viewer, user, editor, admin (hierarchical)
"""
__tablename__ = "roles"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
name: Mapped[str] = mapped_column(
String(100),
unique=True,
nullable=False,
index=True,
comment="Role name in format domain:action (e.g., control-room:admin)",
)
domain: Mapped[str] = mapped_column(
String(50),
nullable=False,
index=True,
comment="Permission domain (e.g., control-room, media, ai)",
)
action: Mapped[str] = mapped_column(
String(20),
nullable=False,
comment="Permission action (viewer, user, editor, admin)",
)
authentik_group: Mapped[str | None] = mapped_column(
String(255),
nullable=True,
unique=True,
comment="Corresponding Authentik group name (e.g., tatlock-control-room-admin)",
)
# Relationships
users: Mapped[List["User"]] = relationship(
"User",
secondary="user_roles",
back_populates="roles",
lazy="selectin",
)
def __repr__(self) -> str:
return f"<Role {self.name}>"
class UserRole(Base):
"""
Association table for User-Role many-to-many relationship
Synced from Authentik groups during user authentication.
"""
__tablename__ = "user_roles"
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
)
role_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("roles.id", ondelete="CASCADE"),
primary_key=True,
)
+94
View File
@@ -0,0 +1,94 @@
"""
User Model
Represents users synced from Authentik SSO.
"""
import uuid
from datetime import datetime
from typing import TYPE_CHECKING, List
from sqlalchemy import String, Boolean, DateTime, func
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.database import Base
if TYPE_CHECKING:
from src.db.models.role import Role
from src.db.models.user_preferences import UserPreferences
from src.db.models.api_key import ApiKey
class User(Base):
"""
User model synced from Authentik
Users are created/updated when they authenticate via OIDC.
The authentik_id links to the Authentik user record.
"""
__tablename__ = "users"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
authentik_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
unique=True,
nullable=False,
index=True,
)
email: Mapped[str] = mapped_column(
String(255),
unique=True,
nullable=False,
index=True,
)
name: Mapped[str] = mapped_column(
String(255),
nullable=False,
)
avatar_url: Mapped[str | None] = mapped_column(
String(500),
nullable=True,
)
api_keys_enabled: Mapped[bool] = mapped_column(
Boolean,
default=True,
nullable=False,
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
last_login: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
# Relationships
roles: Mapped[List["Role"]] = relationship(
"Role",
secondary="user_roles",
back_populates="users",
lazy="selectin",
)
preferences: Mapped["UserPreferences"] = relationship(
"UserPreferences",
back_populates="user",
uselist=False,
lazy="selectin",
cascade="all, delete-orphan",
)
api_keys: Mapped[List["ApiKey"]] = relationship(
"ApiKey",
back_populates="user",
lazy="selectin",
cascade="all, delete-orphan",
)
def __repr__(self) -> str:
return f"<User {self.email}>"
+61
View File
@@ -0,0 +1,61 @@
"""
User Preferences Model
Stores user-specific settings like theme and default room.
"""
import uuid
from sqlalchemy import String, ForeignKey
from sqlalchemy.dialects.postgresql import UUID, JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.database import Base
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from src.db.models.user import User
class UserPreferences(Base):
"""
User preferences model
Stores user-specific settings that persist across sessions.
Extended settings stored in preferences_json for flexibility.
"""
__tablename__ = "user_preferences"
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
)
theme: Mapped[str] = mapped_column(
String(20),
default="system",
nullable=False,
comment="Theme preference: system, light, dark",
)
default_room: Mapped[str] = mapped_column(
String(50),
default="front-hall",
nullable=False,
comment="Default room for housekeeping features",
)
preferences_json: Mapped[dict] = mapped_column(
JSONB,
default=dict,
nullable=False,
comment="Extended preferences as JSON",
)
# Relationships
user: Mapped["User"] = relationship(
"User",
back_populates="preferences",
)
def __repr__(self) -> str:
return f"<UserPreferences user_id={self.user_id}>"
+18 -79
View File
@@ -9,12 +9,13 @@ from contextlib import asynccontextmanager
from src.config import get_settings
from src.logging_config import setup_logging, get_logger
from src.models.ollama_client import get_ollama_client, close_ollama_client
from src.db import get_database
from src.controllers.infrastructure_controller import infrastructure_controller
from src.controllers.tools_controller import tools_controller
from src.controllers.health_controller import health_controller
from src.controllers.static_controller import static_controller
from src.controllers.ai_controller import router as ai_router
from src.controllers.housekeeping_controller import housekeeping_controller
from src.auth.controller import auth_controller
from src.security import initialize_oidc
# Initialize settings
@@ -49,6 +50,14 @@ async def lifespan(app: FastAPI):
else:
logger.warning("✗ Ollama connection failed - AI features may not work")
# Check database connectivity
database = get_database()
db_healthy = await database.health_check()
if db_healthy:
logger.info("✓ Database connection successful")
else:
logger.warning("✗ Database connection failed - auth features may not work")
# Initialize security (OIDC authentication)
initialize_oidc(settings)
@@ -57,6 +66,7 @@ async def lifespan(app: FastAPI):
# Shutdown
logger.info("Shutting down application")
await close_ollama_client()
await database.close()
# Create FastAPI application
@@ -64,89 +74,18 @@ app = FastAPI(
title=settings.app_name,
version=settings.app_version,
description="""
Core Code API provides OpenAPI-compatible functions and AI orchestration for Open WebUI.
Core Code API - Infrastructure management and home automation API.
## Features
### OpenAI-Compatible API (v1)
- `/v1/chat/completions` - Chat completions with streaming support
- `/v1/models` - List available models
Compatible with OpenAI client libraries and Open WebUI.
- **Infrastructure Management** - Container and stack management via Portainer
- **Home Automation** - Device control via Home Assistant
- **Tools** - DNS lookup and utilities
### Conversation Memory (Phase 2)
- `/v1/conversations/{id}` - Get conversation history
- `/v1/conversations/{id}/search` - Semantic search within conversation
- `/v1/conversations/search` - Search across all conversations
- `/v1/conversations/{id}/stats` - Get conversation statistics
- `/v1/conversations/{id}/consolidate` - Manual consolidation
- `DELETE /v1/conversations/{id}` - Delete conversation
Multi-tier memory system:
- **Tier 1**: Fast in-memory buffer (last 10 turns)
- **Tier 2/3**: Unified Qdrant storage (persistent + semantic search)
### Infrastructure Management
**Read Endpoints:**
- `GET /infrastructure/health` - Check Portainer & NPM connectivity
- `GET /infrastructure/services` - List all deployed services
- `GET /infrastructure/services/{name}` - Get service details
- `GET /infrastructure/ports` - List allocated ports
- `GET /infrastructure/domains` - List configured domains
**Write Endpoints (Admin Only):**
- `POST /infrastructure/services` - Deploy new service from compose YAML
- `PUT /infrastructure/services/{name}` - Update existing service
- `DELETE /infrastructure/services/{name}` - Remove service and stack
- `POST /infrastructure/proxy` - Create proxy host with optional SSL
Automates infrastructure operations via Portainer and Nginx Proxy Manager APIs.
### Home Automation (Housekeeping)
**Read Endpoints:**
- `GET /housekeeping/health` - Check Home Assistant connectivity
- `GET /housekeeping/devices` - List devices (filter by domain, area)
- `GET /housekeeping/devices/{entity_id}` - Get device details
- `GET /housekeeping/areas` - List areas/rooms
- `GET /housekeeping/scenes` - List scenes
- `GET /housekeeping/scripts` - List scripts
- `GET /housekeeping/automations` - List automations
- `GET /housekeeping/history` - Get state history
**Write Endpoints (Admin Only):**
- `POST /housekeeping/devices/{entity_id}/control` - Control device
- `POST /housekeeping/scenes/{scene_id}/activate` - Activate scene
- `POST /housekeeping/scripts/{script_id}/run` - Run script
- `POST /housekeeping/automations/{automation_id}/toggle` - Enable/disable automation
Abstracts Home Assistant for the Tatlock Housekeeper agent and other consumers.
### Web Scraper
Intelligent web scraping with main content extraction.
Perfect for extracting articles, documentation, and blog posts for LLM consumption.
## Authentication
When OIDC authentication is enabled (oidc_enabled=true in config):
- Infrastructure write endpoints require authentication
- Use OAuth2/OIDC bearer token from Authentik
- Admin group membership required for infrastructure operations
## Integration
This API is designed to integrate with:
- **Open WebUI**: Direct OpenAI API compatibility
- **Open WebUI Functions**: Import via OpenAPI spec
- **Open WebUI Pipelines**: Use as data source
- **LangChain**: Compatible with standard HTTP tools
## Documentation
- **OpenAPI Spec**: `/openapi.json`
- **Swagger UI**: `/docs`
- **ReDoc**: `/redoc`
See `/docs` for the full API reference.
""",
docs_url="/docs",
redoc_url="/redoc",
redoc_url=None,
openapi_url="/openapi.json",
lifespan=lifespan,
debug=settings.debug,
@@ -168,11 +107,11 @@ app.add_middleware(
# Include controller routers
app.include_router(health_controller.router) # / and /health
app.include_router(auth_controller.router) # /auth/*
app.include_router(tools_controller.router) # /tools/*
app.include_router(infrastructure_controller.router) # /infrastructure/*
app.include_router(housekeeping_controller.router) # /housekeeping/*
app.include_router(static_controller.router) # /static/*
app.include_router(ai_router) # /ai/*
# Global exception handler
+23 -112
View File
@@ -391,40 +391,12 @@ class TestPortainerClientContainers:
assert result is True
class TestPortainerClientDockerSocketFallback:
"""Test Docker socket fallback methods."""
@pytest.mark.asyncio
async def test_list_containers_via_socket_returns_empty_on_error(self):
"""_list_containers_via_socket should return empty list on error."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch("httpx.AsyncHTTPTransport") as mock_transport:
mock_transport.side_effect = Exception("Socket not available")
result = await client._list_containers_via_socket()
assert result == []
@pytest.mark.asyncio
async def test_inspect_container_via_socket_returns_none_on_error(self):
"""_inspect_container_via_socket should return None on error."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch("httpx.AsyncHTTPTransport") as mock_transport:
mock_transport.side_effect = Exception("Socket not available")
result = await client._inspect_container_via_socket("container_name")
assert result is None
class TestPortainerClientWrapperMethods:
"""Test convenience wrapper methods."""
@pytest.mark.asyncio
async def test_list_containers_uses_portainer_first(self):
"""list_containers should try Portainer first."""
async def test_list_containers_uses_portainer(self):
"""list_containers should use Portainer API."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
containers = [{"Id": "abc123", "Names": ["/test"]}]
@@ -442,57 +414,19 @@ class TestPortainerClientWrapperMethods:
mock_containers.assert_called_once()
@pytest.mark.asyncio
async def test_list_containers_falls_back_to_socket(self):
"""list_containers should fallback to Docker socket if Portainer returns empty."""
async def test_list_containers_raises_when_no_endpoints(self):
"""list_containers should raise RuntimeError when no endpoints available."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = [{"Id": 1}]
mock_endpoints.return_value = []
with patch.object(client, "get_containers", new_callable=AsyncMock) as mock_containers:
mock_containers.return_value = []
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = [{"Id": "from_socket"}]
result = await client.list_containers()
assert result == [{"Id": "from_socket"}]
mock_socket.assert_called_once()
with pytest.raises(RuntimeError, match="No Portainer endpoints available"):
await client.list_containers()
@pytest.mark.asyncio
async def test_list_containers_handles_exception_with_fallback(self):
"""list_containers should try fallback even on exception."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = [{"Id": "fallback"}]
result = await client.list_containers()
assert result == [{"Id": "fallback"}]
@pytest.mark.asyncio
async def test_list_containers_returns_empty_when_all_fails(self):
"""list_containers should return empty list when everything fails."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_list_containers_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.side_effect = Exception("Socket error")
result = await client.list_containers()
assert result == []
@pytest.mark.asyncio
async def test_inspect_container_uses_portainer_first(self):
"""inspect_container should try Portainer first."""
async def test_inspect_container_uses_portainer(self):
"""inspect_container should use Portainer API."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
container_list = [{"Id": "abc123", "Names": ["/mycontainer"]}]
@@ -512,54 +446,31 @@ class TestPortainerClientWrapperMethods:
assert result == container_detail
@pytest.mark.asyncio
async def test_inspect_container_falls_back_to_socket(self):
"""inspect_container should fallback if not found in Portainer."""
async def test_inspect_container_returns_none_when_not_found(self):
"""inspect_container should return None if container not found."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = [{"Id": 1}]
with patch.object(client, "get_containers", new_callable=AsyncMock) as mock_list:
mock_list.return_value = [] # Container not found
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = {"Id": "from_socket"}
mock_list.return_value = [] # No containers
result = await client.inspect_container("missing_container")
assert result == {"Id": "from_socket"}
mock_socket.assert_called_once()
@pytest.mark.asyncio
async def test_inspect_container_handles_exception_with_fallback(self):
"""inspect_container should try fallback even on exception."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.return_value = {"Id": "fallback"}
result = await client.inspect_container("container")
assert result == {"Id": "fallback"}
@pytest.mark.asyncio
async def test_inspect_container_returns_none_when_all_fails(self):
"""inspect_container should return None when everything fails."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.side_effect = Exception("API error")
with patch.object(client, "_inspect_container_via_socket", new_callable=AsyncMock) as mock_socket:
mock_socket.side_effect = Exception("Socket error")
result = await client.inspect_container("container")
assert result is None
@pytest.mark.asyncio
async def test_inspect_container_raises_when_no_endpoints(self):
"""inspect_container should raise RuntimeError when no endpoints available."""
client = PortainerClient(base_url="http://portainer:9000", api_key="key")
with patch.object(client, "get_endpoints", new_callable=AsyncMock) as mock_endpoints:
mock_endpoints.return_value = []
with pytest.raises(RuntimeError, match="No Portainer endpoints available"):
await client.inspect_container("container")
class TestPortainerClientStackFile:
"""Test stack file operations."""
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
# Core-API Server Startup Script
set -e
# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
echo -e "${GREEN}Starting Core-API server...${NC}"
# Check if port 8788 is already in use
if lsof -Pi :8788 -sTCP:LISTEN -t >/dev/null 2>&1 ; then
echo -e "${RED}Error: Port 8788 is already in use${NC}"
echo "Run: lsof -i :8788 to see what's using it"
echo "Or run: kill \$(lsof -t -i:8788) to stop it"
exit 1
fi
# Activate virtual environment if not already activated
if [ -z "$VIRTUAL_ENV" ]; then
if [ -d ".venv" ]; then
echo -e "${YELLOW}Activating virtual environment...${NC}"
source .venv/bin/activate
else
echo -e "${RED}Error: Virtual environment not found${NC}"
echo "Run: python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt"
exit 1
fi
fi
# Create logs directory if it doesn't exist
LOGS_DIR="logs"
mkdir -p "$LOGS_DIR"
# Clear/create log file
LOG_FILE="$LOGS_DIR/server.log"
> "$LOG_FILE"
echo -e "${YELLOW}Logs will be written to: ${LOG_FILE}${NC}"
# Start the server
echo -e "${GREEN}Starting uvicorn server on http://tower-of-joy:8788${NC}"
echo -e "${YELLOW}Press Ctrl+C to stop the server${NC}"
echo ""
uvicorn src.main:app --reload --host 0.0.0.0 --port 8788 2>&1 | tee "$LOG_FILE"