Compare commits

...
10 Commits
Author SHA1 Message Date
Jeroen SchweitzerandClaude Opus 4.5 bb438e22d6 fix: strip email domain from user identifier for environment endpoint
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
If preferred_username is an email (user@domain.com), extract just the
username part to match Qdrant collection naming (volatile_user).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:46:16 +01:00
Jeroen SchweitzerandClaude Opus 4.5 3bb3b01dbd fix: OIDC audience validation - use string not list
Build and Push / build (push) Successful in 1m15s
Build and Push / release (push) Successful in 3s
python-jose jwt.decode() requires audience as string or None, not list.
Now extract and validate audience from unverified claims first,
then use token's actual audience for JWT decode.

Fixes "audience must be a string or None" error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:35:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 ce761a9d2c debug: add logging for OIDC token validation
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:58:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6243f29aae feat: multi-issuer OIDC support with config consolidation
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m15s
- Support multiple OAuth providers (core-api, tatlock-ui, tatlock)
- Changed oidc_issuer (string) to oidc_issuers (list)
- Per-issuer JWKS caching
- Validates token issuer against allowed list
- Consolidated config files (removed deprecated src/config.py, src/security.py)
- Updated imports to use src/shared/config and src/shared/security

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:09:29 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c4d32952db fix: OIDC multi-audience support and Swagger UI fix
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m13s
- Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
- Fixed main.py to use oidc_audiences[0] for Swagger UI OAuth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:17:16 +01:00
Jeroen Schweitzer 4c45f139d9 correct documentation 2026-01-08 12:11:01 +01:00
Jeroen Schweitzer 67b33314fe fix: update main.py to use oidc_audiences list
Build and Push / release (push) Failing after 3s
Build and Push / build (push) Has been skipped
2026-01-07 19:34:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6ce34cc016 fix: accept multiple OIDC audiences for cross-client auth
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Changed oidc_audience (string) to oidc_audiences (list)
- Now accepts tokens with audience: core-api, tatlock-ui, tatlock
- Fixes environment endpoint returning "default" user when using
  tatlock-ui token (audience mismatch was causing JWT claims error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:01:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c1f16d44e5 refactor: remove Ollama integration and unused AI configuration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m19s
- Remove src/models/ollama_client.py, embeddings.py, embeddings_ollama.py
- Remove model aliases and AI config from settings (both config.py files)
- Update health endpoints to only check database connectivity
- Update tests to reflect database-only health checks
- Update README, .env.example, and OIDC docstrings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:48 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4df5cfc106 fix: initialize OIDC config for both auth modules
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m30s
The domains.auth.oidc module had its own oidc_config instance that
wasn't being configured, causing environment endpoint to always use
hardcoded "local" user instead of authenticated user.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:00:14 +01:00
25 changed files with 279 additions and 1407 deletions
+1 -4
View File
@@ -39,12 +39,9 @@ HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token
# =============================================================================
# AI Services
# Search
# =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080
+71
View File
@@ -5,6 +5,77 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.10.9] - 2026-01-08
### Fixed
- **Environment user ID cleanup** - Strip email domain from user identifier
- If `preferred_username` is an email, extract just the username part
- Ensures Qdrant collection name matches (e.g., `volatile_jpmschweitzer` not `volatile_jpmschweitzer@gmail.com`)
## [1.10.8] - 2026-01-08
### Fixed
- **OIDC audience validation** - python-jose requires string audience, not list
- Extract and validate audience from unverified claims first
- Use token's actual audience for JWT decode (after validating it's allowed)
- Fixes "audience must be a string or None" error
## [1.10.7] - 2026-01-08
### Added
- **Multi-issuer OIDC support** - Accept tokens from multiple OAuth providers
- Changed `oidc_issuer` (string) to `oidc_issuers` (list)
- Each issuer has its own JWKS endpoint, now cached per-issuer
- Validates token issuer against allowed list before fetching JWKS
- Supports tokens from: `core-api`, `tatlock-ui`, `tatlock` OAuth applications
- Completes fix for environment endpoint user resolution
### Removed
- Deprecated `src/config.py` - consolidated to `src/shared/config.py`
- Deprecated `src/security.py` - consolidated to `src/shared/security.py`
## [1.10.6] - 2026-01-08
### Fixed
- **OIDC audience mismatch** - Accept tokens from multiple clients
- Changed `oidc_audience` (string) to `oidc_audiences` (list)
- Now accepts tokens with audience: `core-api`, `tatlock-ui`, or `tatlock`
- Fixes environment endpoint returning "default" user instead of authenticated username
- Fixed main.py to use `oidc_audiences[0]` for Swagger UI OAuth client
### Changed
- Documentation cleanup in README
## [1.10.4] - 2026-01-07
### Removed
- **Ollama integration removed** - AI inference is no longer handled by this API
- Removed `src/models/ollama_client.py` and all Ollama-related configuration
- Removed `src/models/embeddings.py` and `src/models/embeddings_ollama.py`
- Removed model aliases and AI configuration from settings
- Health endpoints no longer check Ollama status
- Tests updated to reflect database-only health checks
### Changed
- Health check `/health/full` now only checks database connectivity
- Diagnostics endpoint simplified (removed Ollama component info)
## [1.10.3] - 2026-01-07
### Fixed
- **OIDC config not applied to domains module** - Both `src.auth.oidc` and `src.domains.auth.oidc` configs are now initialized
- Previously only `src.auth.oidc` was configured, leaving domains tools using hardcoded "local" user
- Environment endpoint now correctly uses authenticated user from OIDC token
## [1.10.2] - 2026-01-07
### Changed
+6 -17
View File
@@ -19,8 +19,7 @@ Central API service providing infrastructure management, home automation, and ut
### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
- **Health Checks**: Service health monitoring with database connectivity status
## Architecture
@@ -35,10 +34,8 @@ src/
├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client
├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management
@@ -88,9 +85,6 @@ src/
### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development
### Requirements
@@ -103,10 +97,6 @@ src/
# Install dependencies
pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
```
@@ -169,10 +159,9 @@ docker run -p 8083:8083 core-code:latest
| `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - |
| `OIDC_AUDIENCE` | OIDC audience | - |
| `OIDC_ISSUERS` | OIDC issuer URLs (comma-separated) | See config.py |
| `OIDC_AUDIENCES` | OIDC audiences (comma-separated) | See config.py |
## API Documentation
@@ -183,9 +172,9 @@ Once deployed, access documentation at:
## Health Checks
- **Basic**: `GET /health` - Returns status and Ollama connection
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information
- **Basic**: `GET /health` - Fast liveness check for container orchestration
- **Full**: `GET /health/full` - Returns database status (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Service info and configuration
## Security
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "core-api"
version = "1.10.2"
version = "1.10.9"
description = "Core Code API - Infrastructure management and tools API"
readme = "README.md"
requires-python = ">=3.12"
+60 -23
View File
@@ -22,29 +22,42 @@ class OIDCConfig:
def __init__(self):
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.jwks_uri = ""
self.issuers: list[str] = []
self.audiences: list[str] = []
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audiences = audiences
logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance
oidc_config = OIDCConfig()
@lru_cache(maxsize=1)
def get_jwks() -> Dict:
"""
Fetch JSON Web Key Set (JWKS) from Authentik
# Per-issuer JWKS cache
_jwks_cache: Dict[str, Dict] = {}
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns:
JWKS dictionary containing public keys for token verification
@@ -55,15 +68,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled:
return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0)
logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status()
jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)")
logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks
except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}")
logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException(
status_code=503,
detail="Authentication service unavailable"
@@ -109,6 +131,21 @@ async def get_current_user(
token = credentials.credentials
try:
# First, extract issuer and audience from unverified claims
unverified_claims = jwt.get_unverified_claims(token)
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
# Validate issuer is in our allowed list
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer}")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in our allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience}")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Decode token header to get key ID
unverified_header = jwt.get_unverified_header(token)
kid = unverified_header.get("kid")
@@ -116,8 +153,8 @@ async def get_current_user(
if not kid:
raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS
jwks = get_jwks()
# Find matching key in JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None
for key in jwks.get("keys", []):
@@ -129,17 +166,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token using the token's actual issuer and audience
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
issuer=oidc_config.issuer,
audience=token_audience, # Use the token's audience (already validated)
issuer=token_issuer, # Use the token's issuer (already validated)
)
user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}")
logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload
+1 -1
View File
@@ -13,7 +13,7 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
from src.db.models import User, Role, UserPreferences, Group
from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema
+1 -1
View File
@@ -10,7 +10,7 @@ import json
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta, timezone
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
+1 -1
View File
@@ -7,7 +7,7 @@ import httpx
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
+1 -1
View File
@@ -8,7 +8,7 @@ import httpx
import json
from typing import Optional, Dict, List, Any
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
-160
View File
@@ -1,160 +0,0 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
from pydantic_settings import BaseSettings
from functools import lru_cache
def _get_version_from_pyproject() -> str:
"""Load version from pyproject.toml"""
pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
try:
with open(pyproject_path, "rb") as f:
data = tomllib.load(f)
return data.get("project", {}).get("version", "0.0.0")
except FileNotFoundError:
return "0.0.0"
__version__ = _get_version_from_pyproject()
class Settings(BaseSettings):
"""Global application settings"""
# Application
app_name: str = "Core Code API"
app_version: str = __version__
debug: bool = False
# Server
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
# System Prompt Variant (for A/B testing)
# Options: v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion, v5_adk_optimized, v7_adk_best_practice, v8_holistic
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
qdrant_collection_conversations: str = "core_api_conversations"
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama - no local models needed)
embedding_model: str = "nomic-embed-text" # Ollama embedding model
embedding_dimension: int = 768 # nomic-embed-text dimension
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Infrastructure Management (Portainer)
portainer_url: str # Required - set PORTAINER_URL in .env
portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required - set NPM_URL in .env
npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
# Home Assistant Configuration
homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30
# PostgreSQL Database
postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Authentik API (for token validation and user management)
# Must use domain name (not IP) when AUTHENTIK_COOKIE_DOMAIN is set
authentik_url: str = "https://auth.schweitz.net" # Authentik base URL
authentik_username: str = "" # Admin username for API access (AUTHENTIK_USERNAME env var)
authentik_password: str = "" # Admin password for API access (AUTHENTIK_PASSWORD env var)
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_lightweight_models(self) -> list[str]:
"""Parse comma-separated lightweight models"""
return [m.strip().strip('"').strip("'") for m in self.lightweight_models.split(",") if m.strip()]
def get_heavy_models(self) -> list[str]:
"""Parse comma-separated heavy models"""
return [m.strip().strip('"').strip("'") for m in self.heavy_models.split(",") if m.strip()]
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache()
def get_settings() -> Settings:
"""Cached settings instance"""
return Settings()
+13 -96
View File
@@ -7,13 +7,11 @@ from fastapi import APIRouter, Response
from fastapi.responses import JSONResponse
from src.controllers.base import BaseController
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database
logger = get_logger(__name__)
@@ -73,63 +71,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,27 +93,17 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "✅ healthy" if ollama_healthy else "❌ unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
}
}
@@ -170,18 +113,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -189,30 +127,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "✅ connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "❌ error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+1 -1
View File
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import (
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
logger = get_logger(__name__)
+66 -28
View File
@@ -12,7 +12,6 @@ Permission Format: domain.category:action
Examples:
- control-room.general:admin - Full access to Control Room
- media.general:viewer - View-only access to Media area
- ai.ollama:user - User-level access to Ollama specifically (future)
Action Hierarchy (higher implies lower):
- admin > editor > user > viewer
@@ -64,29 +63,42 @@ class OIDCConfig:
def __init__(self):
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.jwks_uri = ""
self.issuers: list[str] = []
self.audiences: list[str] = []
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audiences = audiences
logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance
oidc_config = OIDCConfig()
@lru_cache(maxsize=1)
def get_jwks() -> Dict:
"""
Fetch JSON Web Key Set (JWKS) from Authentik
# Per-issuer JWKS cache
_jwks_cache: Dict[str, Dict] = {}
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns:
JWKS dictionary containing public keys for token verification
@@ -97,15 +109,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled:
return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0)
logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status()
jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)")
logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks
except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}")
logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException(
status_code=503,
detail="Authentication service unavailable"
@@ -158,15 +179,30 @@ async def get_current_user(
token = credentials.credentials
try:
# Decode token header to get key ID
# First, decode token without verification to get issuer and key ID
unverified_header = jwt.get_unverified_header(token)
unverified_claims = jwt.get_unverified_claims(token)
kid = unverified_header.get("kid")
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
if not kid:
raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS
jwks = get_jwks()
# Validate issuer is in allowed list
logger.debug(f"Token issuer: {token_issuer}, allowed issuers: {oidc_config.issuers}")
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer} (allowed: {oidc_config.issuers})")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience} (allowed: {oidc_config.audiences})")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Get JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None
for key in jwks.get("keys", []):
@@ -178,17 +214,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token using the token's actual issuer and audience
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
issuer=oidc_config.issuer,
audience=token_audience, # Use the token's audience (already validated)
issuer=token_issuer, # Use the token's issuer (already validated)
)
user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}")
logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload
@@ -286,12 +322,14 @@ async def get_optional_user(
}
if not credentials:
logger.debug("No credentials provided for optional auth")
return None
try:
return await get_current_user(credentials)
except HTTPException:
# Invalid token - return None instead of raising
except HTTPException as e:
# Invalid token - log and return None instead of raising
logger.warning(f"Optional auth failed: {e.detail}")
return None
@@ -554,7 +592,7 @@ def _extract_permissions_from_groups(groups: List[str]) -> List[str]:
Examples:
- tatlock-control-room-general-admin -> control-room.general:admin
- tatlock-media-viewer -> media.general:viewer (shorthand)
- tatlock-ai-ollama-user -> ai.ollama:user
- tatlock-tools-dns-user -> tools.dns:user
Args:
groups: List of Authentik group names
+11 -96
View File
@@ -70,66 +70,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Import here to avoid circular imports
from src.models.ollama_client import get_ollama_client
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,25 +92,15 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "healthy" if ollama_healthy else "unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
@@ -170,19 +112,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
from src.models.ollama_client import get_ollama_client
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -190,30 +126,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+3
View File
@@ -198,6 +198,9 @@ class ToolsController(BaseController):
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching environment data for user: {user_id}")
result = await self.environment_service.get_current(user_id)
+1 -12
View File
@@ -10,7 +10,6 @@ from src.shared.config import get_settings
from src.shared.logging import setup_logging, get_logger
from src.shared.database import get_database
from src.shared.security import initialize_oidc
from src.models.ollama_client import get_ollama_client, close_ollama_client
# Import domain controllers
from src.domains.health import health_controller
@@ -42,17 +41,8 @@ async def lifespan(app: FastAPI):
logger.info(f"Starting {settings.app_name} v{settings.app_version}")
logger.info(f"Debug mode: {settings.debug}")
logger.info(f"Log level: {settings.log_level}")
logger.info(f"Ollama URL: {settings.ollama_base_url}")
logger.info("=" * 60)
# Check Ollama connectivity
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
if ollama_healthy:
logger.info("Ollama connection successful")
else:
logger.warning("Ollama connection failed - AI features may not work")
# Check database connectivity
database = get_database()
db_healthy = await database.health_check()
@@ -68,7 +58,6 @@ async def lifespan(app: FastAPI):
# Shutdown
logger.info("Shutting down application")
await close_ollama_client()
await database.close()
@@ -94,7 +83,7 @@ See `/docs` for the full API reference.
lifespan=lifespan,
debug=settings.debug,
swagger_ui_init_oauth={
"clientId": settings.oidc_audience,
"clientId": settings.oidc_audiences[0] if settings.oidc_audiences else "core-api",
"usePkceWithAuthorizationCodeGrant": True,
} if settings.oidc_enabled else None
)
-128
View File
@@ -1,128 +0,0 @@
"""
Embedding model client for text vectorization
Uses sentence-transformers for generating embeddings.
"""
import logging
from typing import List, Optional
from sentence_transformers import SentenceTransformer
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class EmbeddingClient:
"""Client for generating text embeddings"""
def __init__(self, model_name: Optional[str] = None):
"""
Initialize embedding client
Args:
model_name: Optional model name, defaults to config
"""
self.model_name = model_name or settings.embedding_model
self.dimension = settings.embedding_dimension
self._model: Optional[SentenceTransformer] = None
logger.info(f"Initializing EmbeddingClient with model: {self.model_name}")
def _load_model(self) -> SentenceTransformer:
"""
Lazy load the embedding model
Returns:
Loaded SentenceTransformer model
"""
if self._model is None:
logger.info(f"Loading embedding model: {self.model_name}")
self._model = SentenceTransformer(self.model_name)
logger.info(f"Model loaded successfully. Embedding dimension: {self.dimension}")
return self._model
def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
model = self._load_model()
embedding = model.encode(text, convert_to_numpy=True)
return embedding.tolist()
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
model = self._load_model()
embeddings = model.encode(
texts,
batch_size=settings.embedding_batch_size,
convert_to_numpy=True,
show_progress_bar=False
)
return embeddings.tolist()
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[EmbeddingClient] = None
def get_embedding_client() -> EmbeddingClient:
"""
Get or create global embedding client instance
Returns:
EmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = EmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return client.embed_batch(texts)
-136
View File
@@ -1,136 +0,0 @@
"""
Ollama-based embedding client for text vectorization
Uses Ollama's embedding API instead of local sentence-transformers.
This eliminates the need for PyTorch and heavy ML dependencies.
"""
import logging
import httpx
from typing import List, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaEmbeddingClient:
"""Client for generating text embeddings using Ollama"""
def __init__(
self,
model_name: Optional[str] = None,
base_url: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Ollama embedding client
Args:
model_name: Embedding model name (default: nomic-embed-text)
base_url: Ollama base URL (default from settings)
timeout: Request timeout in seconds
"""
self.model_name = model_name or settings.embedding_model
self.base_url = (base_url or settings.ollama_base_url).rstrip("/")
self.timeout = timeout
self.dimension = settings.embedding_dimension
logger.info(f"Initializing OllamaEmbeddingClient with model: {self.model_name}")
logger.info(f"Ollama URL: {self.base_url}")
async def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text using Ollama
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/embeddings",
json={
"model": self.model_name,
"prompt": text
}
)
response.raise_for_status()
result = response.json()
return result["embedding"]
except Exception as e:
logger.error(f"Error generating embedding via Ollama: {e}")
raise
async def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
embeddings = []
for text in texts:
embedding = await self.embed_text(text)
embeddings.append(embedding)
return embeddings
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[OllamaEmbeddingClient] = None
def get_embedding_client() -> OllamaEmbeddingClient:
"""
Get or create global Ollama embedding client instance
Returns:
OllamaEmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = OllamaEmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return await client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return await client.embed_batch(texts)
-223
View File
@@ -1,223 +0,0 @@
"""
Ollama client for model inference.
Handles both streaming and non-streaming requests.
"""
import httpx
import json
import logging
from typing import AsyncIterator, Dict, Any, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaClient:
"""Client for interacting with Ollama API."""
def __init__(self):
self.base_url = settings.ollama_base_url
self.timeout = settings.ollama_timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
logger.info(f"Initialized Ollama client: {self.base_url}")
async def close(self):
"""Close the HTTP client."""
await self.client.aclose()
def resolve_model(self, model_name: str) -> str:
"""
Resolve model alias to actual Ollama model.
Args:
model_name: Requested model name (e.g., "gpt-3.5-turbo")
Returns:
Actual Ollama model name (e.g., "gemma:7b")
"""
resolved = settings.model_aliases.get(model_name, model_name)
if resolved != model_name:
logger.info(f"Model resolution: {model_name}{resolved}")
return resolved
async def generate_non_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> Dict[str, Any]:
"""
Generate non-streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Returns:
Dict with 'response' and 'tokens' keys
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama request to {actual_model}")
try:
response = await self.client.post(
f"{self.base_url}/api/chat",
json=payload
)
response.raise_for_status()
result = response.json()
return {
"response": result.get("message", {}).get("content", ""),
"tokens": {
"prompt": result.get("prompt_eval_count", 0),
"completion": result.get("eval_count", 0),
"total": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
}
}
except httpx.HTTPError as e:
logger.error(f"Ollama request failed: {e}")
raise
async def generate_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> AsyncIterator[str]:
"""
Generate streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Yields:
Token strings
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": True,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama streaming request to {actual_model}")
try:
async with self.client.stream(
"POST",
f"{self.base_url}/api/chat",
json=payload
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line:
continue
try:
chunk = json.loads(line)
if "message" in chunk:
content = chunk["message"].get("content", "")
if content:
yield content
# Check if done
if chunk.get("done", False):
break
except json.JSONDecodeError:
logger.warning(f"Failed to parse JSON: {line}")
continue
except httpx.HTTPError as e:
logger.error(f"Ollama streaming request failed: {e}")
raise
async def health_check(self) -> bool:
"""
Check if Ollama is healthy.
Returns:
True if healthy, False otherwise
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
return response.status_code == 200
except Exception as e:
logger.error(f"Ollama health check failed: {e}")
return False
async def list_models(self) -> Dict[str, Any]:
"""
List all available models in Ollama.
Returns:
Dict with 'models' key containing list of model info
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.error(f"Failed to list Ollama models: {e}")
raise
# Global client instance
_ollama_client: Optional[OllamaClient] = None
def get_ollama_client() -> OllamaClient:
"""Get or create the global Ollama client instance."""
global _ollama_client
if _ollama_client is None:
_ollama_client = OllamaClient()
return _ollama_client
async def close_ollama_client():
"""Close the global Ollama client."""
global _ollama_client
if _ollama_client is not None:
await _ollama_client.close()
_ollama_client = None
-32
View File
@@ -1,32 +0,0 @@
"""
Security initialization module
Handles OIDC configuration and authentication setup
"""
from src.config import Settings
from src.auth.oidc import oidc_config
from src.logging_config import get_logger
logger = get_logger(__name__)
def initialize_oidc(settings: Settings) -> None:
"""
Initialize OIDC authentication configuration
Configures the global oidc_config instance with settings from environment.
If OIDC is enabled, logs the issuer URL for verification.
Args:
settings: Application settings containing OIDC configuration
"""
oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
)
if settings.oidc_enabled:
logger.info(f"✓ OIDC authentication enabled (issuer: {settings.oidc_issuer})")
else:
logger.info("○ OIDC authentication disabled - API is publicly accessible")
+8 -46
View File
@@ -52,31 +52,6 @@ class Settings(BaseSettings):
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest"
code_models: str = "mistral-nemo-large:latest"
# System Prompt Variant (for A/B testing)
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
@@ -84,11 +59,6 @@ class Settings(BaseSettings):
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama)
embedding_model: str = "nomic-embed-text"
embedding_dimension: int = 768
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
@@ -120,28 +90,20 @@ class Settings(BaseSettings):
# OIDC Authentication (Authentik)
oidc_enabled: bool = False
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Accept tokens from multiple OAuth providers (each has its own issuer/JWKS)
oidc_issuers: list[str] = [
"https://auth.schweitz.net/application/o/core-api/",
"https://auth.schweitz.net/application/o/tatlock-ui/",
"https://auth.schweitz.net/application/o/tatlock/",
]
# Accept tokens from multiple clients
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management)
authentik_url: str = "https://auth.schweitz.net"
authentik_username: str = ""
authentik_password: str = ""
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
+13 -5
View File
@@ -17,15 +17,23 @@ def initialize_oidc(settings: Settings) -> None:
settings: Application settings containing OIDC configuration
"""
# Import here to avoid circular imports
from src.auth.oidc import oidc_config
# Configure BOTH oidc modules (src.auth and src.domains.auth)
from src.auth.oidc import oidc_config as auth_oidc_config
from src.domains.auth.oidc import oidc_config as domains_oidc_config
oidc_config.configure(
auth_oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
domains_oidc_config.configure(
enabled=settings.oidc_enabled,
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
if settings.oidc_enabled:
logger.info(f"OIDC authentication enabled (issuer: {settings.oidc_issuer})")
logger.info(f"OIDC authentication enabled (issuers: {settings.oidc_issuers})")
else:
logger.info("OIDC authentication disabled - API is publicly accessible")
+1 -1
View File
@@ -1,6 +1,6 @@
"""Tests for config module."""
import pytest
from src.config import (
from src.shared.config import (
__version__,
Settings,
get_settings,
+19 -87
View File
@@ -80,24 +80,24 @@ class TestFullHealthCheck:
"""Test /health/full endpoint."""
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_503_when_unhealthy(self, mock_get_ollama, mock_db_health, client):
"""Full health should return 503 when Ollama unhealthy."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
def test_full_health_returns_200_when_healthy(self, mock_db_health, client):
"""Full health should return 200 when database is healthy."""
mock_db_health.return_value = True
response = client.get("/health/full")
assert response.status_code == 200
@patch("src.shared.database.Database.health_check")
def test_full_health_returns_503_when_unhealthy(self, mock_db_health, client):
"""Full health should return 503 when database is unhealthy."""
mock_db_health.return_value = False
response = client.get("/health/full")
assert response.status_code == 503
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_components_status(self, mock_get_ollama, mock_db_health, client):
def test_full_health_returns_components_status(self, mock_db_health, client):
"""Full health should return component status."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
@@ -105,63 +105,32 @@ class TestFullHealthCheck:
assert "status" in data
assert "components" in data
assert "ollama" in data["components"]
assert "database" in data["components"]
assert "response_time_ms" in data
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_list_models_error(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle list_models errors."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_client.list_models.side_effect = Exception("Connection error")
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
def test_full_health_handles_database_error(self, mock_db_health, client):
"""Full health should handle database errors gracefully."""
mock_db_health.side_effect = Exception("Connection error")
response = client.get("/health/full")
data = response.json()
# Should report error in component status
assert "ollama" in data["components"]
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_health_check_exception(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle health check exceptions gracefully."""
mock_client = AsyncMock()
# Return False instead of raising exception to test unhealthy path
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
# Should return 503 for unhealthy
assert response.status_code == 503
data = response.json()
assert data["status"] == "unhealthy"
assert "error" in data["components"]["database"]
class TestDiagnosticsEndpoint:
"""Test /health/diagnostics endpoint."""
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_200(self, mock_get_ollama, client):
def test_diagnostics_returns_200(self, client):
"""Diagnostics should return 200."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
assert response.status_code == 200
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_service_info(self, mock_get_ollama, client):
def test_diagnostics_returns_service_info(self, client):
"""Diagnostics should return service information."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
@@ -169,54 +138,17 @@ class TestDiagnosticsEndpoint:
assert "name" in data["service"]
assert "version" in data["service"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_components(self, mock_get_ollama, client):
"""Diagnostics should return component details."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "components" in data
assert "ollama" in data["components"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_configuration(self, mock_get_ollama, client):
def test_diagnostics_returns_configuration(self, client):
"""Diagnostics should return configuration info."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "configuration" in data
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_response_time(self, mock_get_ollama, client):
def test_diagnostics_returns_response_time(self, client):
"""Diagnostics should return response time."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "response_time_ms" in data
assert isinstance(data["response_time_ms"], int)
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_handles_ollama_error(self, mock_get_ollama, client):
"""Diagnostics should handle Ollama connection errors."""
mock_client = AsyncMock()
mock_client.health_check.side_effect = Exception("Connection refused")
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
# Should still return 200 with error info
assert response.status_code == 200
assert "error" in data["components"]["ollama"]
-307
View File
@@ -1,307 +0,0 @@
"""Tests for Ollama client."""
import pytest
from unittest.mock import patch, AsyncMock, MagicMock
import json
from src.models.ollama_client import OllamaClient, get_ollama_client, close_ollama_client
class TestOllamaClientInit:
"""Test OllamaClient initialization."""
@patch("src.models.ollama_client.settings")
def test_uses_settings_defaults(self, mock_settings):
"""Client should use settings for defaults."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 60
client = OllamaClient()
assert client.base_url == "http://ollama:11434"
assert client.timeout == 60
@patch("src.models.ollama_client.settings")
def test_creates_http_client(self, mock_settings):
"""Client should create httpx AsyncClient."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
assert client.client is not None
class TestOllamaClientClose:
"""Test client close functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_closes_client(self, mock_settings):
"""close should close the HTTP client."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "aclose", new_callable=AsyncMock) as mock_close:
await client.close()
mock_close.assert_called_once()
class TestOllamaClientResolveModel:
"""Test model resolution."""
@patch("src.models.ollama_client.settings")
def test_resolves_aliased_model(self, mock_settings):
"""resolve_model should map alias to actual model."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {"gpt-3.5-turbo": "gemma:7b"}
client = OllamaClient()
result = client.resolve_model("gpt-3.5-turbo")
assert result == "gemma:7b"
@patch("src.models.ollama_client.settings")
def test_returns_original_if_no_alias(self, mock_settings):
"""resolve_model should return original if no alias found."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
result = client.resolve_model("llama2")
assert result == "llama2"
class TestOllamaClientHealthCheck:
"""Test health check functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_true_on_200(self, mock_settings):
"""Health check should return True when Ollama responds 200."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is True
mock_get.assert_called_once()
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_error(self, mock_settings):
"""Health check should return False on connection error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection refused")
result = await client.health_check()
assert result is False
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_non_200(self, mock_settings):
"""Health check should return False on non-200 status."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 500
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is False
class TestOllamaClientListModels:
"""Test list models functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_returns_dict(self, mock_settings):
"""list_models should return dict with models."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
models_data = {
"models": [
{"name": "llama2", "size": 1000000},
{"name": "gemma:7b", "size": 2000000}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = models_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.list_models()
assert result == models_data
assert len(result["models"]) == 2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_raises_on_error(self, mock_settings):
"""list_models should raise on error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection error")
with pytest.raises(Exception):
await client.list_models()
class TestOllamaClientGenerateNonStreaming:
"""Test non-streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_returns_response(self, mock_settings):
"""generate_non_streaming should return response dict."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
response_data = {
"message": {"content": "Hello! How can I help?"},
"prompt_eval_count": 10,
"eval_count": 20
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = response_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
result = await client.generate_non_streaming("llama2", "Hello")
assert result["response"] == "Hello! How can I help?"
assert result["tokens"]["prompt"] == 10
assert result["tokens"]["completion"] == 20
assert result["tokens"]["total"] == 30
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_includes_max_tokens(self, mock_settings):
"""generate_non_streaming should include max_tokens in payload."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"message": {"content": "Hi"}}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
await client.generate_non_streaming("llama2", "Hello", max_tokens=100)
call_args = mock_post.call_args
assert call_args[1]["json"]["options"]["num_predict"] == 100
class TestOllamaClientGenerateStreaming:
"""Test streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_streaming_yields_content(self, mock_settings):
"""generate_streaming should yield content chunks."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
# Create mock streaming response
async def mock_aiter_lines():
yield json.dumps({"message": {"content": "Hello"}})
yield json.dumps({"message": {"content": " world"}})
yield json.dumps({"done": True})
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_response.aiter_lines = mock_aiter_lines
mock_stream_context = AsyncMock()
mock_stream_context.__aenter__.return_value = mock_response
mock_stream_context.__aexit__.return_value = None
with patch.object(client.client, "stream", return_value=mock_stream_context):
chunks = []
async for chunk in client.generate_streaming("llama2", "Hi"):
chunks.append(chunk)
assert "Hello" in chunks
assert " world" in chunks
class TestOllamaClientSingleton:
"""Test singleton pattern."""
@patch("src.models.ollama_client.settings")
def test_get_ollama_client_returns_same_instance(self, mock_settings):
"""get_ollama_client should return singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client1 = get_ollama_client()
client2 = get_ollama_client()
assert client1 is client2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_ollama_client_clears_singleton(self, mock_settings):
"""close_ollama_client should clear the singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client = get_ollama_client()
with patch.object(client.client, "aclose", new_callable=AsyncMock):
await close_ollama_client()
assert module._ollama_client is None