Compare commits

..
16 Commits
Author SHA1 Message Date
Jeroen SchweitzerandClaude Opus 4.5 c4d32952db fix: OIDC multi-audience support and Swagger UI fix
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m13s
- Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
- Fixed main.py to use oidc_audiences[0] for Swagger UI OAuth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:17:16 +01:00
Jeroen Schweitzer 4c45f139d9 correct documentation 2026-01-08 12:11:01 +01:00
Jeroen Schweitzer 67b33314fe fix: update main.py to use oidc_audiences list
Build and Push / release (push) Failing after 3s
Build and Push / build (push) Has been skipped
2026-01-07 19:34:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6ce34cc016 fix: accept multiple OIDC audiences for cross-client auth
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Changed oidc_audience (string) to oidc_audiences (list)
- Now accepts tokens with audience: core-api, tatlock-ui, tatlock
- Fixes environment endpoint returning "default" user when using
  tatlock-ui token (audience mismatch was causing JWT claims error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:01:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c1f16d44e5 refactor: remove Ollama integration and unused AI configuration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m19s
- Remove src/models/ollama_client.py, embeddings.py, embeddings_ollama.py
- Remove model aliases and AI config from settings (both config.py files)
- Update health endpoints to only check database connectivity
- Update tests to reflect database-only health checks
- Update README, .env.example, and OIDC docstrings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:48 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4df5cfc106 fix: initialize OIDC config for both auth modules
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m30s
The domains.auth.oidc module had its own oidc_config instance that
wasn't being configured, causing environment endpoint to always use
hardcoded "local" user instead of authenticated user.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:00:14 +01:00
Jeroen SchweitzerandClaude Opus 4.5 0a16688cc8 chore: release v1.10.2
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m7s
Enhanced environment endpoint logging for debugging user resolution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 15:32:53 +01:00
Jeroen SchweitzerandClaude Opus 4.5 a7535fe8ea fix: correct OIDC import path in tools controller
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m14s
The environment endpoint was importing from non-existent path
`src.oidc.dependencies` instead of `src.auth.oidc`, causing
authentication to fail and queries to go to wrong Qdrant collection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:36:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6045c6ac6a feat: add environment endpoint for weather, forecast, and sun data
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m14s
Add GET /tools/environment endpoint that fetches weather, forecast, sun times,
and air quality data from user's volatile Qdrant collection.

- Add qdrant-client dependency
- Create QdrantReadClient wrapper for read-only queries
- Add environment schemas and service in tools domain
- Parse weather, forecast, sun times, and air quality from Qdrant payloads
- Support user-specific collections via preferred_username from OIDC
- Add comprehensive service tests (13 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:04:59 +01:00
Jeroen Schweitzer b8fca7060f npm config 2026-01-04 21:51:21 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e3a49c800a fix: SQLAlchemy async lazy loading for new users
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Initialize user.roles=[] and user.preferences on new user creation
to avoid MissingGreenlet error in async context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:05:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7ab9f73a1d chore: release v1.9.3
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:49:26 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd5b794de4 fix: handle non-UUID sub claim in auth sync
Authentik JWT sub claim may not be a valid UUID.
Now derives a deterministic UUID from the sub string if parsing fails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:45:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd0997679f fix: /auth/users/me now supports NPM forward auth headers
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Added get_current_user_or_forward_auth() combined dependency that:
- First checks for X-authentik-* headers from NPM forward auth (web)
- Falls back to JWT Bearer token validation (mobile/native)

This fixes web authentication where browsers don't send Bearer tokens
but rely on NPM's forward auth proxy to pass user info via headers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 00:24:41 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e2226cd923 fix(build): add production API URLs to Docker build
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m8s
Dockerfile now passes --dart-define flags for CORE_API_URL and
TATLOCK_API_URL pointing to schweitz.net domains. This enables
requiresAuth=true, fixing auth being completely skipped in production.

Also added service port reference table to AGENTS.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:18:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7bf3c76a1b fix(cors): use explicit origins instead of wildcard
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
When allow_credentials=True, browsers reject wildcard (*) origins.
Added specific allowed origins for Tatlock domains.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:09:47 +01:00
30 changed files with 1363 additions and 1239 deletions
+1 -4
View File
@@ -39,12 +39,9 @@ HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token
# =============================================================================
# AI Services
# Search
# =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080
+1 -1
View File
@@ -47,7 +47,7 @@ When changes are ready for deployment:
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8000/health`
- Verify deployment: `curl http://192.168.86.149:8083/health`
---
+102
View File
@@ -5,6 +5,108 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.10.6] - 2026-01-08
### Fixed
- **OIDC audience mismatch** - Accept tokens from multiple clients
- Changed `oidc_audience` (string) to `oidc_audiences` (list)
- Now accepts tokens with audience: `core-api`, `tatlock-ui`, or `tatlock`
- Fixes environment endpoint returning "default" user instead of authenticated username
- Fixed main.py to use `oidc_audiences[0]` for Swagger UI OAuth client
### Changed
- Documentation cleanup in README
## [1.10.4] - 2026-01-07
### Removed
- **Ollama integration removed** - AI inference is no longer handled by this API
- Removed `src/models/ollama_client.py` and all Ollama-related configuration
- Removed `src/models/embeddings.py` and `src/models/embeddings_ollama.py`
- Removed model aliases and AI configuration from settings
- Health endpoints no longer check Ollama status
- Tests updated to reflect database-only health checks
### Changed
- Health check `/health/full` now only checks database connectivity
- Diagnostics endpoint simplified (removed Ollama component info)
## [1.10.3] - 2026-01-07
### Fixed
- **OIDC config not applied to domains module** - Both `src.auth.oidc` and `src.domains.auth.oidc` configs are now initialized
- Previously only `src.auth.oidc` was configured, leaving domains tools using hardcoded "local" user
- Environment endpoint now correctly uses authenticated user from OIDC token
## [1.10.2] - 2026-01-07
### Changed
- **Enhanced environment endpoint logging** - Added detailed user claim logging for debugging
- Logs both `preferred_username` and `sub` claims when resolving user
- Distinguishes between authenticated and unauthenticated requests
## [1.10.1] - 2026-01-06
### Fixed
- Fix OIDC import path in tools controller (`src.oidc.dependencies``src.auth.oidc`)
- Environment endpoint was returning `user: "local"` instead of authenticated username
- Caused queries to wrong Qdrant collection (`volatile_local` vs `volatile_{username}`)
## [1.10.0] - 2026-01-06
### Added
- **Environment Data API** - Qdrant-backed endpoint for weather, forecast, and sun position data
- `GET /tools/environment` - Fetch environment data from user's volatile collection
- Weather: current temperature, conditions, humidity, wind speed
- Forecast: multi-day outlook with high/low temperatures
- Sun times: sunrise, sunset, daylight duration
- Air quality: AQI and quality level (when available)
- Data sourced from `volatile_{user}` Qdrant collection
- Uses `preferred_username` from OIDC, falls back to `default`
- `qdrant-client` dependency for vector database access
- `QdrantReadClient` wrapper for read-only collection queries
- Comprehensive test suite for environment service parsing
## [1.9.4] - 2026-01-04
### Fixed
- Fix SQLAlchemy async lazy loading error for new users in `/auth/sync`
- Initialize `user.roles = []` and `user.preferences` to avoid greenlet error
- Was causing "MissingGreenlet: greenlet_spawn has not been called" on new user creation
## [1.9.3] - 2026-01-04
### Fixed
- Handle non-UUID `sub` claim in `/auth/sync` - Authentik JWT may return non-UUID subject identifiers
- Now derives deterministic UUID from sub string if direct parsing fails
## [1.9.2] - 2026-01-04
### Fixed
- `/auth/users/me` endpoint now supports both NPM forward auth headers AND JWT Bearer tokens
- Added `get_current_user_or_forward_auth()` combined auth dependency
- Fixes web authentication where NPM passes `X-authentik-*` headers instead of JWT
- Mobile/native clients continue to use JWT Bearer tokens as before
## [1.9.1] - 2026-01-03
### Fixed
- CORS configuration now uses explicit origins instead of `"*"`
- When `allow_credentials=True`, wildcard origins are rejected by browsers
- Added `home.schweitz.net`, `tatlock.schweitz.net`, and localhost origins
## [1.9.0] - 2026-01-03
### Added
+4 -15
View File
@@ -19,8 +19,7 @@ Central API service providing infrastructure management, home automation, and ut
### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
- **Health Checks**: Service health monitoring with database connectivity status
## Architecture
@@ -35,10 +34,8 @@ src/
├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client
├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management
@@ -88,9 +85,6 @@ src/
### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development
### Requirements
@@ -103,10 +97,6 @@ src/
# Install dependencies
pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
```
@@ -169,7 +159,6 @@ docker run -p 8083:8083 core-code:latest
| `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - |
| `OIDC_AUDIENCE` | OIDC audience | - |
@@ -183,9 +172,9 @@ Once deployed, access documentation at:
## Health Checks
- **Basic**: `GET /health` - Returns status and Ollama connection
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information
- **Basic**: `GET /health` - Fast liveness check for container orchestration
- **Full**: `GET /health/full` - Returns database status (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Service info and configuration
## Security
+12
View File
@@ -0,0 +1,12 @@
proxy_buffers 8 16k;
proxy_buffer_size 32k;
# CORS headers for Flutter web
add_header Access-Control-Allow-Origin "https://home.schweitz.net" always;
add_header Access-Control-Allow-Credentials true always;
add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, PATCH, OPTIONS" always;
add_header Access-Control-Allow-Headers "Content-Type, Authorization" always;
if ($request_method = OPTIONS) {
return 204;
}
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "core-api"
version = "1.9.0"
version = "1.10.6"
description = "Core Code API - Infrastructure management and tools API"
readme = "README.md"
requires-python = ">=3.12"
+3
View File
@@ -32,3 +32,6 @@ cryptography>=44.0.1 # CVE-2024-12797
sqlalchemy[asyncio]~=2.0.0
asyncpg>=0.30.0
alembic~=1.13.0
# Vector Database
qdrant-client>=1.9.0
+6 -6
View File
@@ -23,16 +23,16 @@ class OIDCConfig:
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.audiences: list[str] = []
self.jwks_uri = ""
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuer: str, audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.audiences = audiences
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}, audiences={audiences}")
# Global OIDC config instance
@@ -129,12 +129,12 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token (accepts any of the configured audiences)
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
audience=oidc_config.audiences,
issuer=oidc_config.issuer,
)
+2 -57
View File
@@ -45,35 +45,6 @@ class Settings(BaseSettings):
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
# System Prompt Variant (for A/B testing)
# Options: v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion, v5_adk_optimized, v7_adk_best_practice, v8_holistic
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
@@ -81,11 +52,6 @@ class Settings(BaseSettings):
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama - no local models needed)
embedding_model: str = "nomic-embed-text" # Ollama embedding model
embedding_dimension: int = 768 # nomic-embed-text dimension
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
@@ -118,7 +84,8 @@ class Settings(BaseSettings):
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management)
# Must use domain name (not IP) when AUTHENTIK_COOKIE_DOMAIN is set
@@ -126,28 +93,6 @@ class Settings(BaseSettings):
authentik_username: str = "" # Admin username for API access (AUTHENTIK_USERNAME env var)
authentik_password: str = "" # Admin password for API access (AUTHENTIK_PASSWORD env var)
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_lightweight_models(self) -> list[str]:
"""Parse comma-separated lightweight models"""
return [m.strip().strip('"').strip("'") for m in self.lightweight_models.split(",") if m.strip()]
def get_heavy_models(self) -> list[str]:
"""Parse comma-separated heavy models"""
return [m.strip().strip('"').strip("'") for m in self.heavy_models.split(",") if m.strip()]
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
+12 -95
View File
@@ -9,11 +9,9 @@ from fastapi.responses import JSONResponse
from src.controllers.base import BaseController
from src.config import get_settings
from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database
logger = get_logger(__name__)
@@ -73,63 +71,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,27 +93,17 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "✅ healthy" if ollama_healthy else "❌ unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
}
}
@@ -170,18 +113,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -189,30 +127,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "✅ connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "❌ error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+62 -1
View File
@@ -3,14 +3,20 @@ Tools Controller
Provides utility tool endpoints including:
- DNS lookups
- Environment data (weather, forecast, sun times, air quality)
"""
from fastapi import APIRouter, HTTPException, status
from typing import Dict, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from src.controllers.base import BaseController
from src.logging_config import get_logger
from src.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.dns.service import DNSService
from src.dns.exceptions import DNSQueryError
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import get_environment_service
from src.auth.oidc import get_optional_user
logger = get_logger(__name__)
@@ -26,6 +32,7 @@ class ToolsController(BaseController):
def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService()
self.environment_service = get_environment_service()
def create_router(self) -> APIRouter:
"""Create and configure the router"""
@@ -95,6 +102,60 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Fetch current environment data including weather, forecast, sun times,
and optionally air quality.
Data is retrieved from the user's volatile Qdrant collection which is
populated by background data collectors.
**Data Sources:**
- Weather: Current temperature, conditions, humidity, wind
- Forecast: Multi-day weather outlook
- Sun Times: Sunrise, sunset, daylight duration
- Air Quality: AQI and pollutant levels (when available)
**Authentication:**
- Uses authenticated user's `preferred_username` if available
- Falls back to 'default' for unauthenticated requests
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data.
Args:
user: Optional authenticated user info
Returns:
Environment data with weather, forecast, sun times, and air quality
"""
try:
# Determine user identifier
user_id = "default"
if user:
logger.debug(f"User claims: {user}")
user_id = user.get("preferred_username") or user.get("sub", "default")
logger.info(f"Fetching environment data for user: {user_id} (preferred_username={user.get('preferred_username')}, sub={user.get('sub')})")
else:
logger.info(f"Fetching environment data for user: {user_id} (no auth)")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Error fetching environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to fetch environment data"
)
return router
+29 -9
View File
@@ -20,7 +20,7 @@ from src.domains.auth.schemas import (
ApiKeysListResponse,
)
from src.domains.auth.service import AuthService
from src.domains.auth.oidc import get_current_user
from src.domains.auth.oidc import get_current_user, get_current_user_or_forward_auth
logger = get_logger(__name__)
@@ -322,22 +322,26 @@ class AuthController(BaseController):
responses={
200: {"description": "User profile with roles and preferences"},
401: {"description": "Not authenticated"},
404: {"description": "User not found in database"},
},
)
async def get_current_user_profile(
user_claims: dict = Depends(get_current_user),
user_claims: dict = Depends(get_current_user_or_forward_auth),
session: AsyncSession = Depends(get_async_session),
) -> UserProfileResponse:
"""
Get the current authenticated user's profile
Returns the user's profile, roles, and preferences.
Requires authentication via Bearer token or API key.
Supports both:
- Bearer token (mobile/native clients)
- NPM forward auth headers (web clients via proxy)
For forward auth users, auto-creates the user in the database
if they don't exist yet (first login via web).
"""
service = AuthService(session)
# Get authentik_id from claims (JWT 'sub' field)
# Get authentik_id from claims (JWT 'sub' field or forward auth 'uid')
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
@@ -348,11 +352,27 @@ class AuthController(BaseController):
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
# If user not found and using forward auth, auto-create them
if user is None:
raise HTTPException(
status_code=404,
detail="User not found - please sync via /auth/sync first",
)
auth_method = user_claims.get("auth_method")
if auth_method == "forward_auth":
# Auto-sync user from forward auth headers
logger.info(f"Auto-creating user from forward auth: {user_claims.get('email')}")
user, is_new = await service.sync_user_from_claims(
authentik_id=authentik_id,
email=user_claims.get("email", ""),
name=user_claims.get("name", user_claims.get("preferred_username", "")),
groups=user_claims.get("groups", []),
)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
else:
# JWT auth but user not in DB - they need to sync first
raise HTTPException(
status_code=404,
detail="User not found - please sync via /auth/sync first",
)
return UserProfileResponse(
user=service.user_to_schema(user),
+57 -8
View File
@@ -12,7 +12,6 @@ Permission Format: domain.category:action
Examples:
- control-room.general:admin - Full access to Control Room
- media.general:viewer - View-only access to Media area
- ai.ollama:user - User-level access to Ollama specifically (future)
Action Hierarchy (higher implies lower):
- admin > editor > user > viewer
@@ -65,16 +64,16 @@ class OIDCConfig:
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.audiences: list[str] = []
self.jwks_uri = ""
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuer: str, audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.audiences = audiences
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}, audiences={audiences}")
# Global OIDC config instance
@@ -178,12 +177,12 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token (accepts any of the configured audiences)
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
audience=oidc_config.audiences,
issuer=oidc_config.issuer,
)
@@ -396,6 +395,56 @@ async def get_forward_auth_admin(
return user
async def get_current_user_or_forward_auth(
request: Request,
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Dict:
"""
Combined auth: Try forward auth headers first, then JWT Bearer token.
Supports both:
- Web clients via NPM forward auth (X-authentik-* headers from proxy)
- Mobile/native clients via OIDC JWT Bearer tokens
This is the preferred dependency for /auth/users/me and similar endpoints
that need to work with both web (cookie-based via NPM) and mobile (token-based).
Args:
request: FastAPI request object containing headers
credentials: HTTP Bearer token from Authorization header
Returns:
User claims dictionary with at minimum: sub, email, name, groups, auth_method
Raises:
HTTPException 401: If neither forward auth headers nor valid JWT provided
"""
# 1. Try forward auth headers first (web via NPM)
username = request.headers.get("x-authentik-username")
email = request.headers.get("x-authentik-email")
if username and email:
# Forward auth headers present - use them
groups = request.headers.get("x-authentik-groups", "")
name = request.headers.get("x-authentik-name", username)
uid = request.headers.get("x-authentik-uid")
user_info = {
"sub": uid, # Use authentik UID as subject (for user lookup)
"email": email,
"preferred_username": username,
"name": name,
"groups": [g.strip() for g in groups.split(",")] if groups else [],
"auth_method": "forward_auth"
}
logger.info(f"Authenticated via forward auth: {email}")
return user_info
# 2. Fall back to JWT Bearer token (mobile/native)
return await get_current_user(credentials)
# =============================================================================
# Permission-Based Access Control
# =============================================================================
@@ -504,7 +553,7 @@ def _extract_permissions_from_groups(groups: List[str]) -> List[str]:
Examples:
- tatlock-control-room-general-admin -> control-room.general:admin
- tatlock-media-viewer -> media.general:viewer (shorthand)
- tatlock-ai-ollama-user -> ai.ollama:user
- tatlock-tools-dns-user -> tools.dns:user
Args:
groups: List of Authentik group names
+79 -2
View File
@@ -102,7 +102,13 @@ class AuthService:
Returns:
Tuple of (User, is_new_user)
"""
authentik_id = uuid.UUID(token_info.sub)
# Parse authentik_id - may be UUID or other format
try:
authentik_id = uuid.UUID(token_info.sub)
except ValueError:
# If sub is not a valid UUID, derive one deterministically
logger.warning(f"sub claim is not a UUID: {token_info.sub}, deriving UUID")
authentik_id = uuid.uuid5(uuid.NAMESPACE_OID, token_info.sub)
# Try to find existing user
stmt = (
@@ -124,11 +130,14 @@ class AuthService:
avatar_url=token_info.picture,
last_login=datetime.now(timezone.utc),
)
# Initialize relationships to avoid lazy loading issues in async
user.roles = []
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
# Create default preferences and attach to user
preferences = UserPreferences(user_id=user.id)
user.preferences = preferences
self.session.add(preferences)
logger.info(f"Created new user: {token_info.email}")
@@ -144,6 +153,74 @@ class AuthService:
await self.session.flush()
return user, is_new
async def sync_user_from_claims(
self,
authentik_id: uuid.UUID,
email: str,
name: str,
groups: list[str],
avatar_url: Optional[str] = None,
) -> tuple[User, bool]:
"""
Create or update user from forward auth claims (NPM X-authentik-* headers)
This is similar to sync_user() but works with raw claims instead of
TokenInfoSchema. Used for auto-syncing users on first web login via NPM.
Args:
authentik_id: The Authentik user UUID (from X-authentik-uid)
email: User email (from X-authentik-email)
name: User display name (from X-authentik-name)
groups: List of group names (from X-authentik-groups)
avatar_url: Optional avatar URL
Returns:
Tuple of (User, is_new_user)
"""
# Try to find existing user
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
user = result.scalar_one_or_none()
is_new = user is None
if is_new:
# Create new user
user = User(
authentik_id=authentik_id,
email=email,
name=name or email,
avatar_url=avatar_url,
last_login=datetime.now(timezone.utc),
)
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
preferences = UserPreferences(user_id=user.id)
self.session.add(preferences)
logger.info(f"Created new user from forward auth: {email}")
else:
# Update existing user
user.email = email
user.name = name or email
if avatar_url:
user.avatar_url = avatar_url
user.last_login = datetime.now(timezone.utc)
logger.info(f"Updated existing user from forward auth: {email}")
# Sync roles from groups
await self.sync_roles(user, groups)
await self.session.flush()
return user, is_new
async def sync_roles(self, user: User, group_names: list[str]) -> list[Role]:
"""
Synchronize user roles from Authentik groups via group_roles mapping
+11 -96
View File
@@ -70,66 +70,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Import here to avoid circular imports
from src.models.ollama_client import get_ollama_client
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,25 +92,15 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "healthy" if ollama_healthy else "unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
@@ -170,19 +112,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
from src.models.ollama_client import get_ollama_client
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -190,30 +126,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+65 -1
View File
@@ -4,8 +4,10 @@ Tools Controller
Provides utility tool endpoints including:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
from fastapi import APIRouter, HTTPException, status
from typing import Dict, Optional
from fastapi import APIRouter, HTTPException, status, Depends
from src.shared.base import BaseController
from src.shared.logging import get_logger
@@ -14,6 +16,9 @@ from src.domains.tools.dns.service import DNSService
from src.domains.tools.dns.exceptions import DNSQueryError
from src.domains.tools.system.schemas import SystemStatsResponse
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import EnvironmentService
from src.domains.auth.oidc import get_optional_user
logger = get_logger(__name__)
@@ -25,12 +30,14 @@ class ToolsController(BaseController):
Provides endpoints for:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService()
self.system_stats_service = SystemStatsService()
self.environment_service = EnvironmentService()
def create_router(self) -> APIRouter:
"""Create and configure the router"""
@@ -146,6 +153,63 @@ class ToolsController(BaseController):
detail=f"Failed to collect system stats: {str(e)}"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Get current environment data including weather, forecast, and sun times.
Fetches data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Weather:** Current temperature, conditions, humidity, wind
- **Forecast:** Multi-day weather outlook
- **Sun Times:** Sunrise, sunset, daylight duration
- **Air Quality:** AQI and pollutant levels (if available)
**Data Source:** Qdrant volatile_{user} collection
**Use Cases:**
- Dashboard environment widgets
- Home automation context
- Weather-based automations
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data
Args:
user: Optional authenticated user from OIDC
Returns:
Environment data including weather, forecast, sun times
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
logger.info(f"Fetching environment data for user: {user_id}")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Failed to get environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch environment data: {str(e)}"
)
return router
+23
View File
@@ -0,0 +1,23 @@
"""
Environment data module for Tools domain.
Provides access to weather, forecast, sun times, and air quality data
from the Qdrant volatile collection.
"""
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
from src.domains.tools.environment.service import EnvironmentService
__all__ = [
"WeatherData",
"ForecastDay",
"SunTimesData",
"AirQualityData",
"EnvironmentResponse",
"EnvironmentService",
]
+185
View File
@@ -0,0 +1,185 @@
"""
Environment data schemas for Tools domain.
Provides Pydantic models for weather, forecast, sun times, and air quality data
retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List, Any
from pydantic import Field
from src.shared.base import BaseSchema
class WeatherData(BaseSchema):
"""Current weather conditions."""
temperature: Optional[float] = Field(
None,
description="Current temperature in Celsius"
)
feels_like: Optional[float] = Field(
None,
description="Feels-like temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description (e.g., 'Partly Cloudy')"
)
humidity: Optional[int] = Field(
None,
ge=0,
le=100,
description="Humidity percentage"
)
wind_speed: Optional[float] = Field(
None,
description="Wind speed in km/h"
)
wind_direction: Optional[str] = Field(
None,
description="Wind direction (e.g., 'NW')"
)
pressure: Optional[float] = Field(
None,
description="Atmospheric pressure in hPa"
)
visibility: Optional[float] = Field(
None,
description="Visibility in km"
)
uv_index: Optional[float] = Field(
None,
description="UV index"
)
location: Optional[str] = Field(
None,
description="Location name"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class ForecastDay(BaseSchema):
"""Single day forecast data."""
date: str = Field(
...,
description="Date string (e.g., '2025-01-07')"
)
high: Optional[float] = Field(
None,
description="High temperature in Celsius"
)
low: Optional[float] = Field(
None,
description="Low temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description"
)
precipitation_chance: Optional[int] = Field(
None,
ge=0,
le=100,
description="Chance of precipitation percentage"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class SunTimesData(BaseSchema):
"""Sunrise and sunset times."""
sunrise: Optional[datetime] = Field(
None,
description="Sunrise time"
)
sunset: Optional[datetime] = Field(
None,
description="Sunset time"
)
daylight_minutes: Optional[int] = Field(
None,
description="Total daylight duration in minutes"
)
solar_noon: Optional[datetime] = Field(
None,
description="Solar noon time"
)
dawn: Optional[datetime] = Field(
None,
description="Civil dawn time"
)
dusk: Optional[datetime] = Field(
None,
description="Civil dusk time"
)
class AirQualityData(BaseSchema):
"""Air quality information."""
aqi: Optional[int] = Field(
None,
ge=0,
description="Air Quality Index"
)
quality: Optional[str] = Field(
None,
description="Quality category (Good, Moderate, Unhealthy, etc.)"
)
pm25: Optional[float] = Field(
None,
description="PM2.5 concentration in microg/m3"
)
pm10: Optional[float] = Field(
None,
description="PM10 concentration in microg/m3"
)
o3: Optional[float] = Field(
None,
description="Ozone concentration in ppb"
)
no2: Optional[float] = Field(
None,
description="Nitrogen dioxide concentration in ppb"
)
location: Optional[str] = Field(
None,
description="Location name"
)
class EnvironmentResponse(BaseSchema):
"""Combined environment data response."""
weather: Optional[WeatherData] = Field(
None,
description="Current weather conditions"
)
forecast: Optional[List[ForecastDay]] = Field(
None,
description="Multi-day weather forecast"
)
sun_times: Optional[SunTimesData] = Field(
None,
description="Sunrise/sunset times"
)
air_quality: Optional[AirQualityData] = Field(
None,
description="Air quality data (None if not available)"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+246
View File
@@ -0,0 +1,246 @@
"""
Environment data service for Tools domain.
Fetches weather, forecast, sun times, and air quality data from
the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
logger = get_logger(__name__)
class EnvironmentService:
"""
Service for fetching environment data from Qdrant volatile collection.
Retrieves weather, forecast, sun times, and optionally air quality
data for a specific user.
"""
def __init__(self):
"""Initialize environment service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_weather(self, raw_data: Optional[Dict[str, Any]]) -> Optional[WeatherData]:
"""
Parse raw weather data into WeatherData schema.
Handles various field naming conventions that might come from
different weather APIs.
"""
if not raw_data:
return None
try:
return WeatherData(
temperature=raw_data.get("temperature") or raw_data.get("temp"),
feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"),
conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"),
humidity=raw_data.get("humidity"),
wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"),
wind_direction=raw_data.get("wind_direction") or raw_data.get("wind_dir"),
pressure=raw_data.get("pressure"),
visibility=raw_data.get("visibility"),
uv_index=raw_data.get("uv_index") or raw_data.get("uv"),
location=raw_data.get("location") or raw_data.get("city"),
icon=raw_data.get("icon") or raw_data.get("icon_url"),
)
except Exception as e:
logger.warning(f"Failed to parse weather data: {e}")
return None
def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]:
"""
Parse raw forecast data into list of ForecastDay schemas.
Handles both list format and dict with nested list.
"""
if not raw_data:
return None
try:
# Normalize to list
forecast_list = raw_data
if isinstance(raw_data, dict):
forecast_list = raw_data.get("days") or raw_data.get("forecast") or []
if not isinstance(forecast_list, list):
return None
days = []
for day in forecast_list:
if isinstance(day, dict):
days.append(ForecastDay(
date=day.get("date", ""),
high=day.get("high") or day.get("maxtemp") or day.get("temp_max"),
low=day.get("low") or day.get("mintemp") or day.get("temp_min"),
conditions=day.get("conditions") or day.get("weather") or day.get("description"),
precipitation_chance=day.get("precipitation_chance") or day.get("pop") or day.get("precip"),
icon=day.get("icon"),
))
return days if days else None
except Exception as e:
logger.warning(f"Failed to parse forecast data: {e}")
return None
def _parse_sun_times(self, raw_data: Optional[Dict[str, Any]]) -> Optional[SunTimesData]:
"""
Parse raw sun times data into SunTimesData schema.
Handles datetime strings and calculates daylight minutes if not provided.
"""
if not raw_data:
return None
try:
sunrise = raw_data.get("sunrise")
sunset = raw_data.get("sunset")
# Parse datetime strings if needed
if isinstance(sunrise, str):
sunrise = datetime.fromisoformat(sunrise.replace("Z", "+00:00"))
if isinstance(sunset, str):
sunset = datetime.fromisoformat(sunset.replace("Z", "+00:00"))
# Calculate daylight minutes if not provided
daylight_minutes = raw_data.get("daylight_minutes") or raw_data.get("daylight")
if daylight_minutes is None and sunrise and sunset:
daylight_minutes = int((sunset - sunrise).total_seconds() / 60)
# Parse optional fields
solar_noon = raw_data.get("solar_noon")
if isinstance(solar_noon, str):
solar_noon = datetime.fromisoformat(solar_noon.replace("Z", "+00:00"))
dawn = raw_data.get("dawn") or raw_data.get("civil_dawn")
if isinstance(dawn, str):
dawn = datetime.fromisoformat(dawn.replace("Z", "+00:00"))
dusk = raw_data.get("dusk") or raw_data.get("civil_dusk")
if isinstance(dusk, str):
dusk = datetime.fromisoformat(dusk.replace("Z", "+00:00"))
return SunTimesData(
sunrise=sunrise,
sunset=sunset,
daylight_minutes=daylight_minutes,
solar_noon=solar_noon,
dawn=dawn,
dusk=dusk,
)
except Exception as e:
logger.warning(f"Failed to parse sun times data: {e}")
return None
def _parse_air_quality(self, raw_data: Any) -> Optional[AirQualityData]:
"""
Parse raw air quality data into AirQualityData schema.
Handles both dict format and simple integer AQI value.
"""
if raw_data is None:
return None
try:
# Handle simple integer AQI
if isinstance(raw_data, (int, float)):
aqi = int(raw_data)
return AirQualityData(
aqi=aqi,
quality=self._aqi_to_quality(aqi),
)
if not isinstance(raw_data, dict):
return None
aqi = raw_data.get("aqi") or raw_data.get("index")
if isinstance(aqi, (int, float)):
aqi = int(aqi)
return AirQualityData(
aqi=aqi,
quality=raw_data.get("quality") or (self._aqi_to_quality(aqi) if aqi else None),
pm25=raw_data.get("pm25") or raw_data.get("pm2_5"),
pm10=raw_data.get("pm10"),
o3=raw_data.get("o3") or raw_data.get("ozone"),
no2=raw_data.get("no2"),
location=raw_data.get("location"),
)
except Exception as e:
logger.warning(f"Failed to parse air quality data: {e}")
return None
def _aqi_to_quality(self, aqi: int) -> str:
"""Convert AQI value to quality category string."""
if aqi <= 50:
return "Good"
elif aqi <= 100:
return "Moderate"
elif aqi <= 150:
return "Unhealthy for Sensitive Groups"
elif aqi <= 200:
return "Unhealthy"
elif aqi <= 300:
return "Very Unhealthy"
else:
return "Hazardous"
async def get_current(self, user: str = "default") -> EnvironmentResponse:
"""
Get current environment data for a user.
Fetches weather, forecast, sun times, and air quality from
the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
EnvironmentResponse with all available data
"""
logger.info(f"Fetching environment data for user: {user}")
# Get raw data from Qdrant
raw_data = await self.qdrant.get_environment_data(user)
# Parse each data type
weather = self._parse_weather(raw_data.get("weather"))
forecast = self._parse_forecast(raw_data.get("forecast"))
sun_times = self._parse_sun_times(raw_data.get("sun_times"))
air_quality = self._parse_air_quality(raw_data.get("air_quality"))
return EnvironmentResponse(
weather=weather,
forecast=forecast,
sun_times=sun_times,
air_quality=air_quality,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_environment_service: Optional[EnvironmentService] = None
def get_environment_service() -> EnvironmentService:
"""Get or create singleton environment service instance."""
global _environment_service
if _environment_service is None:
_environment_service = EnvironmentService()
return _environment_service
+1 -12
View File
@@ -10,7 +10,6 @@ from src.shared.config import get_settings
from src.shared.logging import setup_logging, get_logger
from src.shared.database import get_database
from src.shared.security import initialize_oidc
from src.models.ollama_client import get_ollama_client, close_ollama_client
# Import domain controllers
from src.domains.health import health_controller
@@ -42,17 +41,8 @@ async def lifespan(app: FastAPI):
logger.info(f"Starting {settings.app_name} v{settings.app_version}")
logger.info(f"Debug mode: {settings.debug}")
logger.info(f"Log level: {settings.log_level}")
logger.info(f"Ollama URL: {settings.ollama_base_url}")
logger.info("=" * 60)
# Check Ollama connectivity
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
if ollama_healthy:
logger.info("Ollama connection successful")
else:
logger.warning("Ollama connection failed - AI features may not work")
# Check database connectivity
database = get_database()
db_healthy = await database.health_check()
@@ -68,7 +58,6 @@ async def lifespan(app: FastAPI):
# Shutdown
logger.info("Shutting down application")
await close_ollama_client()
await database.close()
@@ -94,7 +83,7 @@ See `/docs` for the full API reference.
lifespan=lifespan,
debug=settings.debug,
swagger_ui_init_oauth={
"clientId": settings.oidc_audience,
"clientId": settings.oidc_audiences[0] if settings.oidc_audiences else "core-api",
"usePkceWithAuthorizationCodeGrant": True,
} if settings.oidc_enabled else None
)
-128
View File
@@ -1,128 +0,0 @@
"""
Embedding model client for text vectorization
Uses sentence-transformers for generating embeddings.
"""
import logging
from typing import List, Optional
from sentence_transformers import SentenceTransformer
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class EmbeddingClient:
"""Client for generating text embeddings"""
def __init__(self, model_name: Optional[str] = None):
"""
Initialize embedding client
Args:
model_name: Optional model name, defaults to config
"""
self.model_name = model_name or settings.embedding_model
self.dimension = settings.embedding_dimension
self._model: Optional[SentenceTransformer] = None
logger.info(f"Initializing EmbeddingClient with model: {self.model_name}")
def _load_model(self) -> SentenceTransformer:
"""
Lazy load the embedding model
Returns:
Loaded SentenceTransformer model
"""
if self._model is None:
logger.info(f"Loading embedding model: {self.model_name}")
self._model = SentenceTransformer(self.model_name)
logger.info(f"Model loaded successfully. Embedding dimension: {self.dimension}")
return self._model
def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
model = self._load_model()
embedding = model.encode(text, convert_to_numpy=True)
return embedding.tolist()
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
model = self._load_model()
embeddings = model.encode(
texts,
batch_size=settings.embedding_batch_size,
convert_to_numpy=True,
show_progress_bar=False
)
return embeddings.tolist()
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[EmbeddingClient] = None
def get_embedding_client() -> EmbeddingClient:
"""
Get or create global embedding client instance
Returns:
EmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = EmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return client.embed_batch(texts)
-136
View File
@@ -1,136 +0,0 @@
"""
Ollama-based embedding client for text vectorization
Uses Ollama's embedding API instead of local sentence-transformers.
This eliminates the need for PyTorch and heavy ML dependencies.
"""
import logging
import httpx
from typing import List, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaEmbeddingClient:
"""Client for generating text embeddings using Ollama"""
def __init__(
self,
model_name: Optional[str] = None,
base_url: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Ollama embedding client
Args:
model_name: Embedding model name (default: nomic-embed-text)
base_url: Ollama base URL (default from settings)
timeout: Request timeout in seconds
"""
self.model_name = model_name or settings.embedding_model
self.base_url = (base_url or settings.ollama_base_url).rstrip("/")
self.timeout = timeout
self.dimension = settings.embedding_dimension
logger.info(f"Initializing OllamaEmbeddingClient with model: {self.model_name}")
logger.info(f"Ollama URL: {self.base_url}")
async def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text using Ollama
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/embeddings",
json={
"model": self.model_name,
"prompt": text
}
)
response.raise_for_status()
result = response.json()
return result["embedding"]
except Exception as e:
logger.error(f"Error generating embedding via Ollama: {e}")
raise
async def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
embeddings = []
for text in texts:
embedding = await self.embed_text(text)
embeddings.append(embedding)
return embeddings
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[OllamaEmbeddingClient] = None
def get_embedding_client() -> OllamaEmbeddingClient:
"""
Get or create global Ollama embedding client instance
Returns:
OllamaEmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = OllamaEmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return await client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return await client.embed_batch(texts)
-223
View File
@@ -1,223 +0,0 @@
"""
Ollama client for model inference.
Handles both streaming and non-streaming requests.
"""
import httpx
import json
import logging
from typing import AsyncIterator, Dict, Any, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaClient:
"""Client for interacting with Ollama API."""
def __init__(self):
self.base_url = settings.ollama_base_url
self.timeout = settings.ollama_timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
logger.info(f"Initialized Ollama client: {self.base_url}")
async def close(self):
"""Close the HTTP client."""
await self.client.aclose()
def resolve_model(self, model_name: str) -> str:
"""
Resolve model alias to actual Ollama model.
Args:
model_name: Requested model name (e.g., "gpt-3.5-turbo")
Returns:
Actual Ollama model name (e.g., "gemma:7b")
"""
resolved = settings.model_aliases.get(model_name, model_name)
if resolved != model_name:
logger.info(f"Model resolution: {model_name}{resolved}")
return resolved
async def generate_non_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> Dict[str, Any]:
"""
Generate non-streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Returns:
Dict with 'response' and 'tokens' keys
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama request to {actual_model}")
try:
response = await self.client.post(
f"{self.base_url}/api/chat",
json=payload
)
response.raise_for_status()
result = response.json()
return {
"response": result.get("message", {}).get("content", ""),
"tokens": {
"prompt": result.get("prompt_eval_count", 0),
"completion": result.get("eval_count", 0),
"total": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
}
}
except httpx.HTTPError as e:
logger.error(f"Ollama request failed: {e}")
raise
async def generate_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> AsyncIterator[str]:
"""
Generate streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Yields:
Token strings
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": True,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama streaming request to {actual_model}")
try:
async with self.client.stream(
"POST",
f"{self.base_url}/api/chat",
json=payload
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line:
continue
try:
chunk = json.loads(line)
if "message" in chunk:
content = chunk["message"].get("content", "")
if content:
yield content
# Check if done
if chunk.get("done", False):
break
except json.JSONDecodeError:
logger.warning(f"Failed to parse JSON: {line}")
continue
except httpx.HTTPError as e:
logger.error(f"Ollama streaming request failed: {e}")
raise
async def health_check(self) -> bool:
"""
Check if Ollama is healthy.
Returns:
True if healthy, False otherwise
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
return response.status_code == 200
except Exception as e:
logger.error(f"Ollama health check failed: {e}")
return False
async def list_models(self) -> Dict[str, Any]:
"""
List all available models in Ollama.
Returns:
Dict with 'models' key containing list of model info
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.error(f"Failed to list Ollama models: {e}")
raise
# Global client instance
_ollama_client: Optional[OllamaClient] = None
def get_ollama_client() -> OllamaClient:
"""Get or create the global Ollama client instance."""
global _ollama_client
if _ollama_client is None:
_ollama_client = OllamaClient()
return _ollama_client
async def close_ollama_client():
"""Close the global Ollama client."""
global _ollama_client
if _ollama_client is not None:
await _ollama_client.close()
_ollama_client = None
+3
View File
@@ -7,6 +7,7 @@ from src.shared.clients.portainer_client import PortainerClient, get_portainer_c
from src.shared.clients.npm_client import NPMClient, get_npm_client
from src.shared.clients.homeassistant_client import HomeAssistantClient, get_homeassistant_client
from src.shared.clients.authentik_client import AuthentikClient, get_authentik_client
from src.shared.clients.qdrant_client import QdrantReadClient, get_qdrant_client
__all__ = [
"PortainerClient",
@@ -17,4 +18,6 @@ __all__ = [
"get_homeassistant_client",
"AuthentikClient",
"get_authentik_client",
"QdrantReadClient",
"get_qdrant_client",
]
+258
View File
@@ -0,0 +1,258 @@
"""
Qdrant Vector Database Client (Read-Only)
Provides read-only access to Qdrant collections for querying volatile data.
Used to fetch weather, forecast, and sun times from the volatile_{user} collection.
"""
import time
from typing import List, Dict, Any, Optional
from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class QdrantReadClient:
"""
Read-only Qdrant client for accessing volatile data.
Connects to Qdrant and provides methods to query collections
with filtering by namespace and TTL expiry.
"""
VOLATILE_COLLECTION_PREFIX = "volatile_"
def __init__(
self,
host: Optional[str] = None,
port: Optional[int] = None,
):
"""
Initialize Qdrant read client.
Args:
host: Qdrant server host (default from settings)
port: Qdrant server port (default from settings)
"""
self.host = host or settings.qdrant_host
self.port = port or settings.qdrant_port
self._client: Optional[QdrantClient] = None
logger.info(f"Initialized QdrantReadClient: {self.host}:{self.port}")
@property
def client(self) -> QdrantClient:
"""Lazy-load Qdrant client connection."""
if self._client is None:
self._client = QdrantClient(
host=self.host,
port=self.port,
)
return self._client
def _get_volatile_collection(self, user: str) -> str:
"""Get volatile collection name for user."""
return f"{self.VOLATILE_COLLECTION_PREFIX}{user}"
def _current_timestamp_ms(self) -> int:
"""Get current timestamp in milliseconds."""
return int(time.time() * 1000)
async def collection_exists(self, collection_name: str) -> bool:
"""
Check if a collection exists.
Args:
collection_name: Name of collection to check
Returns:
True if collection exists
"""
try:
collections = self.client.get_collections()
existing = [c.name for c in collections.collections]
return collection_name in existing
except Exception as e:
logger.error(f"Error checking collection existence: {e}")
return False
async def get_by_namespace(
self,
user: str,
namespace: str,
include_expired: bool = False
) -> List[Dict[str, Any]]:
"""
Get all records for a specific namespace from user's volatile collection.
Args:
user: User identifier (e.g., 'jpmschweitzer' or 'default')
namespace: Namespace to filter (e.g., 'weather', 'forecast', 'sun')
include_expired: Whether to include expired records (default False)
Returns:
List of records with payload data
"""
collection_name = self._get_volatile_collection(user)
if not await self.collection_exists(collection_name):
logger.debug(f"Collection {collection_name} does not exist")
return []
# Build filter conditions
conditions = [
FieldCondition(
key="namespace",
match=MatchValue(value=namespace)
)
]
# Add TTL expiry filter unless including expired
if not include_expired:
now_ms = self._current_timestamp_ms()
conditions.append(
FieldCondition(
key="ttl_expiry",
range=Range(gt=now_ms)
)
)
query_filter = Filter(must=conditions)
try:
# Scroll through matching records
points, _ = self.client.scroll(
collection_name=collection_name,
scroll_filter=query_filter,
limit=100,
with_payload=True,
with_vectors=False
)
results = []
for point in points:
payload = dict(point.payload) if point.payload else {}
results.append({
"id": str(point.id),
"namespace": payload.get("namespace"),
"key": payload.get("key"),
"raw_data": payload.get("raw_data", {}),
"source": payload.get("source"),
"ttl_expiry": payload.get("ttl_expiry"),
"updated_at": payload.get("updated_at"),
})
logger.debug(
f"Found {len(results)} records in {collection_name}/{namespace}"
)
return results
except Exception as e:
logger.error(f"Error fetching from {collection_name}/{namespace}: {e}")
return []
async def get_environment_data(
self,
user: str
) -> Dict[str, Any]:
"""
Get all environment data (weather, forecast, sun times) for a user.
Convenience method that fetches all environment-related namespaces
in a single call.
Args:
user: User identifier
Returns:
Dict with 'weather', 'forecast', 'sun_times', 'air_quality' keys
(each may be None if no data found)
"""
result = {
"weather": None,
"forecast": None,
"sun_times": None,
"air_quality": None,
}
# Fetch weather data
weather_records = await self.get_by_namespace(user, "weather")
if weather_records:
# Get the first/most recent weather record
result["weather"] = weather_records[0].get("raw_data")
# Check if air quality is embedded in weather data
if result["weather"]:
aqi = result["weather"].get("aqi") or result["weather"].get("air_quality")
if aqi:
result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi}
# Fetch forecast data
forecast_records = await self.get_by_namespace(user, "forecast")
if forecast_records:
# Forecast might be a single record with list or multiple records
first_record = forecast_records[0].get("raw_data")
if isinstance(first_record, list):
result["forecast"] = first_record
elif isinstance(first_record, dict):
# Could be a dict with 'days' or 'forecast' key
result["forecast"] = first_record.get(
"days",
first_record.get("forecast", [first_record])
)
# Fetch sun times data
sun_records = await self.get_by_namespace(user, "sun")
if sun_records:
result["sun_times"] = sun_records[0].get("raw_data")
# Check for separate air quality namespace if not embedded
if result["air_quality"] is None:
aq_records = await self.get_by_namespace(user, "air_quality")
if aq_records:
result["air_quality"] = aq_records[0].get("raw_data")
return result
async def health_check(self) -> Dict[str, Any]:
"""
Check Qdrant connectivity.
Returns:
Dict with connection status and info
"""
try:
collections = self.client.get_collections()
volatile_collections = [
c.name for c in collections.collections
if c.name.startswith(self.VOLATILE_COLLECTION_PREFIX)
]
return {
"status": "healthy",
"connected": True,
"host": f"{self.host}:{self.port}",
"volatile_collections": volatile_collections,
}
except Exception as e:
logger.error(f"Qdrant health check failed: {e}")
return {
"status": "unhealthy",
"connected": False,
"host": f"{self.host}:{self.port}",
"error": str(e),
}
# Singleton instance for reuse
_qdrant_client: Optional[QdrantReadClient] = None
def get_qdrant_client() -> QdrantReadClient:
"""Get or create singleton Qdrant client instance."""
global _qdrant_client
if _qdrant_client is None:
_qdrant_client = QdrantReadClient()
return _qdrant_client
+11 -47
View File
@@ -36,8 +36,15 @@ class Settings(BaseSettings):
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
# CORS - Note: When cors_credentials is True, cannot use "*" for origins
# Set CORS_ORIGINS env var to override (comma-separated list)
cors_origins: list[str] = [
"https://home.schweitz.net",
"https://tatlock.schweitz.net",
"http://localhost:8080",
"http://localhost:3000",
"http://127.0.0.1:8080",
]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
@@ -45,31 +52,6 @@ class Settings(BaseSettings):
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest"
code_models: str = "mistral-nemo-large:latest"
# System Prompt Variant (for A/B testing)
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
@@ -77,11 +59,6 @@ class Settings(BaseSettings):
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama)
embedding_model: str = "nomic-embed-text"
embedding_dimension: int = 768
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
@@ -114,27 +91,14 @@ class Settings(BaseSettings):
# OIDC Authentication (Authentik)
oidc_enabled: bool = False
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management)
authentik_url: str = "https://auth.schweitz.net"
authentik_username: str = ""
authentik_password: str = ""
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
+11 -3
View File
@@ -17,12 +17,20 @@ def initialize_oidc(settings: Settings) -> None:
settings: Application settings containing OIDC configuration
"""
# Import here to avoid circular imports
from src.auth.oidc import oidc_config
# Configure BOTH oidc modules (src.auth and src.domains.auth)
from src.auth.oidc import oidc_config as auth_oidc_config
from src.domains.auth.oidc import oidc_config as domains_oidc_config
oidc_config.configure(
auth_oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
audiences=settings.oidc_audiences
)
domains_oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audiences=settings.oidc_audiences
)
if settings.oidc_enabled:
+159
View File
@@ -0,0 +1,159 @@
"""Tests for environment service and schemas."""
import pytest
from datetime import datetime
class TestEnvironmentService:
"""Test EnvironmentService methods."""
def test_aqi_to_quality_good(self):
"""AQI 0-50 should return Good."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(0) == "Good"
assert service._aqi_to_quality(25) == "Good"
assert service._aqi_to_quality(50) == "Good"
def test_aqi_to_quality_moderate(self):
"""AQI 51-100 should return Moderate."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(51) == "Moderate"
assert service._aqi_to_quality(75) == "Moderate"
assert service._aqi_to_quality(100) == "Moderate"
def test_aqi_to_quality_unhealthy_sensitive(self):
"""AQI 101-150 should return Unhealthy for Sensitive Groups."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(101) == "Unhealthy for Sensitive Groups"
assert service._aqi_to_quality(150) == "Unhealthy for Sensitive Groups"
def test_aqi_to_quality_unhealthy(self):
"""AQI 151-200 should return Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(151) == "Unhealthy"
assert service._aqi_to_quality(200) == "Unhealthy"
def test_aqi_to_quality_very_unhealthy(self):
"""AQI 201-300 should return Very Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(201) == "Very Unhealthy"
assert service._aqi_to_quality(300) == "Very Unhealthy"
def test_aqi_to_quality_hazardous(self):
"""AQI >300 should return Hazardous."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(301) == "Hazardous"
assert service._aqi_to_quality(500) == "Hazardous"
def test_parse_weather_with_valid_data(self):
"""Parse weather should return WeatherData for valid input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"temperature": 15.5,
"conditions": "Cloudy",
"humidity": 72,
"location": "Rotterdam",
}
result = service._parse_weather(raw)
assert result is not None
assert result.temperature == 15.5
assert result.conditions == "Cloudy"
assert result.humidity == 72
def test_parse_weather_with_none(self):
"""Parse weather should return None for None input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_weather(None)
assert result is None
def test_parse_forecast_with_list(self):
"""Parse forecast should handle list format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
{"date": "2026-01-07", "high": 14, "low": 6, "conditions": "Sunny"},
]
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 2
assert result[0].date == "2026-01-06"
assert result[0].high == 12
def test_parse_forecast_with_dict(self):
"""Parse forecast should handle dict with days key."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"days": [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
]
}
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 1
def test_parse_sun_times_with_strings(self):
"""Parse sun times should handle ISO datetime strings."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"sunrise": "2026-01-06T08:45:00",
"sunset": "2026-01-06T16:50:00",
}
result = service._parse_sun_times(raw)
assert result is not None
assert result.sunrise.hour == 8
assert result.sunrise.minute == 45
assert result.sunset.hour == 16
assert result.daylight_minutes == 485
def test_parse_air_quality_with_int(self):
"""Parse air quality should handle simple integer AQI."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_air_quality(42)
assert result is not None
assert result.aqi == 42
assert result.quality == "Good"
def test_parse_air_quality_with_dict(self):
"""Parse air quality should handle dict format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"aqi": 75,
"pm25": 8.5,
"pm10": 15,
}
result = service._parse_air_quality(raw)
assert result is not None
assert result.aqi == 75
assert result.quality == "Moderate"
assert result.pm25 == 8.5
+19 -87
View File
@@ -80,24 +80,24 @@ class TestFullHealthCheck:
"""Test /health/full endpoint."""
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_503_when_unhealthy(self, mock_get_ollama, mock_db_health, client):
"""Full health should return 503 when Ollama unhealthy."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
def test_full_health_returns_200_when_healthy(self, mock_db_health, client):
"""Full health should return 200 when database is healthy."""
mock_db_health.return_value = True
response = client.get("/health/full")
assert response.status_code == 200
@patch("src.shared.database.Database.health_check")
def test_full_health_returns_503_when_unhealthy(self, mock_db_health, client):
"""Full health should return 503 when database is unhealthy."""
mock_db_health.return_value = False
response = client.get("/health/full")
assert response.status_code == 503
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_components_status(self, mock_get_ollama, mock_db_health, client):
def test_full_health_returns_components_status(self, mock_db_health, client):
"""Full health should return component status."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
@@ -105,63 +105,32 @@ class TestFullHealthCheck:
assert "status" in data
assert "components" in data
assert "ollama" in data["components"]
assert "database" in data["components"]
assert "response_time_ms" in data
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_list_models_error(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle list_models errors."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_client.list_models.side_effect = Exception("Connection error")
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
def test_full_health_handles_database_error(self, mock_db_health, client):
"""Full health should handle database errors gracefully."""
mock_db_health.side_effect = Exception("Connection error")
response = client.get("/health/full")
data = response.json()
# Should report error in component status
assert "ollama" in data["components"]
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_health_check_exception(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle health check exceptions gracefully."""
mock_client = AsyncMock()
# Return False instead of raising exception to test unhealthy path
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
# Should return 503 for unhealthy
assert response.status_code == 503
data = response.json()
assert data["status"] == "unhealthy"
assert "error" in data["components"]["database"]
class TestDiagnosticsEndpoint:
"""Test /health/diagnostics endpoint."""
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_200(self, mock_get_ollama, client):
def test_diagnostics_returns_200(self, client):
"""Diagnostics should return 200."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
assert response.status_code == 200
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_service_info(self, mock_get_ollama, client):
def test_diagnostics_returns_service_info(self, client):
"""Diagnostics should return service information."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
@@ -169,54 +138,17 @@ class TestDiagnosticsEndpoint:
assert "name" in data["service"]
assert "version" in data["service"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_components(self, mock_get_ollama, client):
"""Diagnostics should return component details."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "components" in data
assert "ollama" in data["components"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_configuration(self, mock_get_ollama, client):
def test_diagnostics_returns_configuration(self, client):
"""Diagnostics should return configuration info."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "configuration" in data
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_response_time(self, mock_get_ollama, client):
def test_diagnostics_returns_response_time(self, client):
"""Diagnostics should return response time."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "response_time_ms" in data
assert isinstance(data["response_time_ms"], int)
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_handles_ollama_error(self, mock_get_ollama, client):
"""Diagnostics should handle Ollama connection errors."""
mock_client = AsyncMock()
mock_client.health_check.side_effect = Exception("Connection refused")
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
# Should still return 200 with error info
assert response.status_code == 200
assert "error" in data["components"]["ollama"]
-307
View File
@@ -1,307 +0,0 @@
"""Tests for Ollama client."""
import pytest
from unittest.mock import patch, AsyncMock, MagicMock
import json
from src.models.ollama_client import OllamaClient, get_ollama_client, close_ollama_client
class TestOllamaClientInit:
"""Test OllamaClient initialization."""
@patch("src.models.ollama_client.settings")
def test_uses_settings_defaults(self, mock_settings):
"""Client should use settings for defaults."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 60
client = OllamaClient()
assert client.base_url == "http://ollama:11434"
assert client.timeout == 60
@patch("src.models.ollama_client.settings")
def test_creates_http_client(self, mock_settings):
"""Client should create httpx AsyncClient."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
assert client.client is not None
class TestOllamaClientClose:
"""Test client close functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_closes_client(self, mock_settings):
"""close should close the HTTP client."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "aclose", new_callable=AsyncMock) as mock_close:
await client.close()
mock_close.assert_called_once()
class TestOllamaClientResolveModel:
"""Test model resolution."""
@patch("src.models.ollama_client.settings")
def test_resolves_aliased_model(self, mock_settings):
"""resolve_model should map alias to actual model."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {"gpt-3.5-turbo": "gemma:7b"}
client = OllamaClient()
result = client.resolve_model("gpt-3.5-turbo")
assert result == "gemma:7b"
@patch("src.models.ollama_client.settings")
def test_returns_original_if_no_alias(self, mock_settings):
"""resolve_model should return original if no alias found."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
result = client.resolve_model("llama2")
assert result == "llama2"
class TestOllamaClientHealthCheck:
"""Test health check functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_true_on_200(self, mock_settings):
"""Health check should return True when Ollama responds 200."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is True
mock_get.assert_called_once()
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_error(self, mock_settings):
"""Health check should return False on connection error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection refused")
result = await client.health_check()
assert result is False
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_non_200(self, mock_settings):
"""Health check should return False on non-200 status."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 500
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is False
class TestOllamaClientListModels:
"""Test list models functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_returns_dict(self, mock_settings):
"""list_models should return dict with models."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
models_data = {
"models": [
{"name": "llama2", "size": 1000000},
{"name": "gemma:7b", "size": 2000000}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = models_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.list_models()
assert result == models_data
assert len(result["models"]) == 2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_raises_on_error(self, mock_settings):
"""list_models should raise on error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection error")
with pytest.raises(Exception):
await client.list_models()
class TestOllamaClientGenerateNonStreaming:
"""Test non-streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_returns_response(self, mock_settings):
"""generate_non_streaming should return response dict."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
response_data = {
"message": {"content": "Hello! How can I help?"},
"prompt_eval_count": 10,
"eval_count": 20
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = response_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
result = await client.generate_non_streaming("llama2", "Hello")
assert result["response"] == "Hello! How can I help?"
assert result["tokens"]["prompt"] == 10
assert result["tokens"]["completion"] == 20
assert result["tokens"]["total"] == 30
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_includes_max_tokens(self, mock_settings):
"""generate_non_streaming should include max_tokens in payload."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"message": {"content": "Hi"}}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
await client.generate_non_streaming("llama2", "Hello", max_tokens=100)
call_args = mock_post.call_args
assert call_args[1]["json"]["options"]["num_predict"] == 100
class TestOllamaClientGenerateStreaming:
"""Test streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_streaming_yields_content(self, mock_settings):
"""generate_streaming should yield content chunks."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
# Create mock streaming response
async def mock_aiter_lines():
yield json.dumps({"message": {"content": "Hello"}})
yield json.dumps({"message": {"content": " world"}})
yield json.dumps({"done": True})
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_response.aiter_lines = mock_aiter_lines
mock_stream_context = AsyncMock()
mock_stream_context.__aenter__.return_value = mock_response
mock_stream_context.__aexit__.return_value = None
with patch.object(client.client, "stream", return_value=mock_stream_context):
chunks = []
async for chunk in client.generate_streaming("llama2", "Hi"):
chunks.append(chunk)
assert "Hello" in chunks
assert " world" in chunks
class TestOllamaClientSingleton:
"""Test singleton pattern."""
@patch("src.models.ollama_client.settings")
def test_get_ollama_client_returns_same_instance(self, mock_settings):
"""get_ollama_client should return singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client1 = get_ollama_client()
client2 = get_ollama_client()
assert client1 is client2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_ollama_client_clears_singleton(self, mock_settings):
"""close_ollama_client should clear the singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client = get_ollama_client()
with patch.object(client.client, "aclose", new_callable=AsyncMock):
await close_ollama_client()
assert module._ollama_client is None