feat: initial FastAPI boilerplate setup
Set up Webber - multi-agent AI development system with: - Domain-based project structure (src/domains/, src/shared/) - BaseController pattern with lazy router instantiation - Pydantic Settings configuration with env file support - Logger decorator with temporal benchmarking and trace IDs - UserProvider singleton for request-scoped context - Custom exception hierarchy - Health endpoints (/, /health) Dependencies (CVE checked 2026-01-09): - FastAPI 0.128.0, Starlette 0.50.0, Uvicorn 0.40.0 - Pydantic 2.12.4, PydanticAI 1.40.0 - All packages at latest safe versions Placeholder domains for future implementation: - agents/ (explore, plan, task) - tools/ (file, shell, search) - auth/ (tatlock integration) Port: 8086 (per CONTAINERS.md allocation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
# Webber Configuration
|
||||
# Copy to .env and customize
|
||||
|
||||
# Application
|
||||
DEBUG=true
|
||||
LOG_LEVEL=DEBUG
|
||||
|
||||
# Server
|
||||
HOST=0.0.0.0
|
||||
PORT=8086
|
||||
|
||||
# CORS (comma-separated)
|
||||
CORS_ORIGINS=["http://localhost:3000","http://localhost:8080"]
|
||||
|
||||
# LLM - Ollama (tower-of-joy)
|
||||
OLLAMA_URL=http://192.168.86.149:11434
|
||||
OLLAMA_AGENT_MODEL=mistral-nemo-large:latest
|
||||
OLLAMA_EMBED_MODEL=nomic-embed-text:latest
|
||||
|
||||
# Auth - Tatlock integration (optional)
|
||||
# TATLOCK_API_URL=http://192.168.86.149:8000
|
||||
# INTERNAL_API_KEY=your-internal-key
|
||||
|
||||
# Tool execution
|
||||
TOOL_TIMEOUT_SECONDS=120
|
||||
SANDBOX_ENABLED=true
|
||||
# ALLOWED_PATHS=["/home/user/projects","/tmp/webber"]
|
||||
|
||||
# Sessions
|
||||
SESSION_TTL_HOURS=24
|
||||
MAX_CONTEXT_TOKENS=128000
|
||||
+66
@@ -0,0 +1,66 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
ENV/
|
||||
env/
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Testing
|
||||
.tox/
|
||||
.nox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
htmlcov/
|
||||
.pytest_cache/
|
||||
.hypothesis/
|
||||
|
||||
# MyPy / Type checking
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
|
||||
# Environment variables
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
*.env
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
logs/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Project specific
|
||||
*.db
|
||||
*.sqlite3
|
||||
@@ -0,0 +1,115 @@
|
||||
|
||||
# AGENTS.md
|
||||
|
||||
> **Start every session by reading this file.**
|
||||
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
|
||||
|
||||
## 1. Agent Operational Protocols
|
||||
|
||||
### 🧠 Work Patterns (Plan-Act-Reflect)
|
||||
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
|
||||
* **Act:** Execute the changes in small, atomic steps.
|
||||
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
|
||||
|
||||
### 🛡️ Git Discipline
|
||||
* **ALWAYS add the relevant tests for the added code** Make sure to keep the test coverage up as we go, and run tests before commiting.
|
||||
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
|
||||
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
|
||||
* `feat: add user login endpoint`
|
||||
* `fix: resolve database connection timeout`
|
||||
* `refactor: split monolith dependency file`
|
||||
* **Atomic Commits:** Keep commits small. One logical change = one commit.
|
||||
|
||||
### 📝 Changelog Maintenance
|
||||
* **Update `CHANGELOG.md`** with every user-facing change.
|
||||
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
|
||||
|
||||
### 🚀 Release Flow
|
||||
When changes are ready for deployment:
|
||||
|
||||
1. **Ask user if deploy cycle is desired **
|
||||
|
||||
2. **Update version** in `pyproject.toml`:
|
||||
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
|
||||
- New features: bump minor version (1.8.4 → 1.9.0)
|
||||
|
||||
3. **Update CHANGELOG.md**:
|
||||
- Move items from `[Unreleased]` to new version section
|
||||
- Add release date: `## [1.8.4] - 2025-12-16`
|
||||
|
||||
4. **Commit and tag**:
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: description of changes"
|
||||
git tag v1.8.4
|
||||
git push origin main --tags
|
||||
```
|
||||
|
||||
5. **CI/CD triggers automatically**:
|
||||
- Gitea CI builds Docker image on new version tag (starts with "v")
|
||||
- Watchtower pulls and deploys to production
|
||||
|
||||
---
|
||||
|
||||
### 🧪 Local Development Setup
|
||||
|
||||
* **Always test locally first** before committing and deploying. The build-deploy loop is slow.
|
||||
* **Only deploy** when a phase or feature is complete and tested locally
|
||||
* **Environment**: Copy `.env.example` to `.env` and configure for your local setup
|
||||
|
||||
#### ⚠️ CRITICAL: Starting the Local Server
|
||||
|
||||
**ALWAYS use `./wakeup.sh` to start the local server. NEVER use raw uvicorn commands.**
|
||||
|
||||
```bash
|
||||
./wakeup.sh
|
||||
```
|
||||
|
||||
The wakeup script provides:
|
||||
- **Port conflict detection** - Warns if port 8086 is already in use
|
||||
- **Virtual environment activation** - Ensures correct Python environment
|
||||
- **Centralized logging** - All logs written to `logs/server.log` for easy tailing
|
||||
- **Auto-reload** - Code changes picked up automatically (except requirements.txt changes)
|
||||
- **Consistent configuration** - Same startup every time
|
||||
|
||||
To monitor logs in another terminal:
|
||||
```bash
|
||||
tail -f logs/server.log
|
||||
```
|
||||
|
||||
To stop the server: Press `Ctrl+C`
|
||||
|
||||
To kill a stuck server:
|
||||
```bash
|
||||
pkill -f "uvicorn src.main:app"
|
||||
# or
|
||||
kill $(lsof -t -i:8086)
|
||||
```
|
||||
|
||||
#### Testing
|
||||
|
||||
**Test REST endpoints** against `http://localhost:8086`:
|
||||
```bash
|
||||
curl http://localhost:8086/health
|
||||
curl http://localhost:8086/
|
||||
curl http://localhost:8086/docs # Swagger UI
|
||||
```
|
||||
|
||||
**Running tests**: Always use the venv explicitly to avoid environment mismatches:
|
||||
```bash
|
||||
.venv/bin/python -m pytest tests/ # All tests
|
||||
.venv/bin/python -m pytest tests/ -v # Verbose output
|
||||
.venv/bin/python -m pytest tests/ --cov # With coverage
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. FastAPI Architecture & Best Practices
|
||||
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
|
||||
|
||||
### 📂 Project Structure (Directory-based, NOT File-type based)
|
||||
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
|
||||
|
||||
**Correct Structure:**
|
||||
```text
|
||||
to be determined
|
||||
@@ -0,0 +1,868 @@
|
||||
# FastAPI Best Practices
|
||||
|
||||
> **A comprehensive guide for building production-grade FastAPI applications.**
|
||||
> Based on patterns from [zhanymkanov/fastapi-best-practices](https://github.com/zhanymkanov/fastapi-best-practices) with additional patterns for logging, authentication, and scalable architecture.
|
||||
|
||||
---
|
||||
|
||||
## 1. Project Structure
|
||||
|
||||
### Domain-Based Organization (NOT File-Type Based)
|
||||
|
||||
**Do NOT** group files by type (e.g., one huge `routers/` folder). Group by **domain/module** inside a `src/` directory.
|
||||
|
||||
```
|
||||
project/
|
||||
├── AGENTS.md # AI/developer guidelines
|
||||
├── README.md # Project overview
|
||||
├── CHANGELOG.md # Version history
|
||||
├── pyproject.toml # Package metadata
|
||||
├── requirements.txt # Production dependencies only
|
||||
├── requirements-dev.txt # Dev/test dependencies
|
||||
├── .env.example # Environment template
|
||||
│
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── main.py # FastAPI app, lifespan (NO routes here)
|
||||
│ │
|
||||
│ ├── shared/ # Cross-cutting concerns
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── base.py # BaseController, BaseSchema
|
||||
│ │ ├── config.py # Pydantic BaseSettings
|
||||
│ │ ├── logging.py # Logger decorator + centralized setup
|
||||
│ │ ├── exceptions.py # Custom exception hierarchy
|
||||
│ │ ├── auth.py # Authentication utilities
|
||||
│ │ └── context.py # Request context (user provider, etc.)
|
||||
│ │
|
||||
│ └── domains/ # Feature domains
|
||||
│ ├── __init__.py
|
||||
│ ├── router.py # Root router - composes all domain routers
|
||||
│ │
|
||||
│ ├── health/ # Health check domain
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── router.py # Routes
|
||||
│ │ └── controller.py # Business logic
|
||||
│ │
|
||||
│ └── {domain}/ # Each feature domain
|
||||
│ ├── __init__.py
|
||||
│ ├── router.py # Domain routes
|
||||
│ ├── controller.py # Business logic
|
||||
│ ├── schemas.py # Pydantic models
|
||||
│ ├── service.py # External service calls (optional)
|
||||
│ └── exceptions.py # Domain-specific exceptions (optional)
|
||||
│
|
||||
├── tests/
|
||||
│ ├── __init__.py
|
||||
│ ├── conftest.py # Pytest fixtures
|
||||
│ └── test_{domain}.py
|
||||
│
|
||||
└── docs/
|
||||
└── architecture.md
|
||||
```
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **Clean main.py**: Only app creation, middleware, lifespan. NO routes.
|
||||
2. **Domain routers**: Each domain has `router.py`. Root `domains/router.py` composes them.
|
||||
3. **Shared utilities**: Cross-cutting concerns in `shared/` - import from there, not across domains.
|
||||
4. **Self-contained domains**: Each domain can be understood in isolation.
|
||||
|
||||
---
|
||||
|
||||
## 2. Base Patterns
|
||||
|
||||
### BaseController
|
||||
|
||||
```python
|
||||
from abc import ABC, abstractmethod
|
||||
from fastapi import APIRouter
|
||||
|
||||
|
||||
class BaseController(ABC):
|
||||
"""
|
||||
Base controller with lazy router instantiation.
|
||||
|
||||
All domain controllers inherit from this and implement create_router().
|
||||
"""
|
||||
|
||||
def __init__(self, prefix: str, tags: list[str]):
|
||||
self.prefix = prefix
|
||||
self.tags = tags
|
||||
self._router = None
|
||||
|
||||
@abstractmethod
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the FastAPI router with all routes."""
|
||||
pass
|
||||
|
||||
@property
|
||||
def router(self) -> APIRouter:
|
||||
"""Lazy router instantiation."""
|
||||
if self._router is None:
|
||||
self._router = self.create_router()
|
||||
return self._router
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
# src/domains/health/controller.py
|
||||
class HealthController(BaseController):
|
||||
def __init__(self):
|
||||
super().__init__(prefix="", tags=["Health"])
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
router = APIRouter(tags=self.tags)
|
||||
|
||||
@router.get("/health")
|
||||
async def health_check():
|
||||
return {"status": "healthy"}
|
||||
|
||||
return router
|
||||
|
||||
health_controller = HealthController()
|
||||
```
|
||||
|
||||
### BaseSchema
|
||||
|
||||
```python
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class BaseSchema(BaseModel):
|
||||
"""Base Pydantic model with standardized configuration."""
|
||||
|
||||
model_config = ConfigDict(
|
||||
strict=False,
|
||||
populate_by_name=True,
|
||||
use_enum_values=True,
|
||||
validate_assignment=True,
|
||||
json_encoders={
|
||||
datetime: lambda v: v.isoformat() if v else None
|
||||
}
|
||||
)
|
||||
|
||||
def dict_without_none(self) -> dict[str, Any]:
|
||||
"""Return model as dict, excluding None values."""
|
||||
return {k: v for k, v in self.model_dump().items() if v is not None}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Configuration
|
||||
|
||||
### Pydantic Settings
|
||||
|
||||
```python
|
||||
# src/shared/config.py
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from functools import lru_cache
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
def _get_version() -> str:
|
||||
"""Load version from pyproject.toml."""
|
||||
try:
|
||||
with open(Path(__file__).parent.parent.parent / "pyproject.toml", "rb") as f:
|
||||
return tomllib.load(f).get("project", {}).get("version", "0.0.0")
|
||||
except FileNotFoundError:
|
||||
return "0.0.0"
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Application settings loaded from environment."""
|
||||
|
||||
# Application
|
||||
app_name: str = "MyApp"
|
||||
app_version: str = _get_version()
|
||||
debug: bool = False
|
||||
|
||||
# Server
|
||||
host: str = "0.0.0.0"
|
||||
port: int = 8000
|
||||
|
||||
# Logging
|
||||
log_level: str = "INFO"
|
||||
|
||||
# CORS
|
||||
cors_origins: list[str] = ["http://localhost:3000"]
|
||||
cors_credentials: bool = True
|
||||
cors_methods: list[str] = ["*"]
|
||||
cors_headers: list[str] = ["*"]
|
||||
|
||||
class Config:
|
||||
env_file = ".env"
|
||||
case_sensitive = False
|
||||
extra = "ignore"
|
||||
|
||||
|
||||
@lru_cache()
|
||||
def get_settings() -> Settings:
|
||||
"""Cached settings singleton."""
|
||||
return Settings()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Logging with Temporal Benchmarking
|
||||
|
||||
### Logger Decorator
|
||||
|
||||
```python
|
||||
# src/shared/logging.py
|
||||
import functools
|
||||
import asyncio
|
||||
import time
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
from contextvars import ContextVar
|
||||
from dataclasses import dataclass, field
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
# === Trace Context ===
|
||||
|
||||
@dataclass
|
||||
class TraceSpan:
|
||||
"""Represents a timed execution span."""
|
||||
name: str
|
||||
trace_id: str
|
||||
parent_id: Optional[str] = None
|
||||
span_id: str = field(default_factory=lambda: uuid4().hex[:8])
|
||||
start_time: float = field(default_factory=time.perf_counter)
|
||||
end_time: Optional[float] = None
|
||||
|
||||
@property
|
||||
def duration_ms(self) -> float:
|
||||
end = self.end_time or time.perf_counter()
|
||||
return (end - self.start_time) * 1000
|
||||
|
||||
|
||||
_current_span: ContextVar[Optional[TraceSpan]] = ContextVar('current_span', default=None)
|
||||
_trace_id: ContextVar[Optional[str]] = ContextVar('trace_id', default=None)
|
||||
|
||||
|
||||
def get_current_trace_id() -> Optional[str]:
|
||||
"""Get current trace ID for log correlation."""
|
||||
return _trace_id.get()
|
||||
|
||||
|
||||
# === Setup ===
|
||||
|
||||
def setup_logging(log_level: str = "INFO") -> None:
|
||||
"""Configure application logging."""
|
||||
log_dir = Path("logs")
|
||||
log_dir.mkdir(exist_ok=True)
|
||||
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, log_level.upper()),
|
||||
format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
handlers=[
|
||||
logging.StreamHandler(sys.stdout),
|
||||
logging.FileHandler(log_dir / "app.log", encoding="utf-8")
|
||||
]
|
||||
)
|
||||
|
||||
# Quiet noisy libraries
|
||||
for name in ["httpx", "httpcore", "uvicorn.access"]:
|
||||
logging.getLogger(name).setLevel(logging.WARNING)
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""Get a logger instance."""
|
||||
return logging.getLogger(name)
|
||||
|
||||
|
||||
# === Decorator ===
|
||||
|
||||
def logged(
|
||||
logger: logging.Logger = None,
|
||||
slow_threshold_ms: float = 100.0,
|
||||
warn_threshold_ms: float = 500.0,
|
||||
include_args: bool = False,
|
||||
):
|
||||
"""
|
||||
Decorator for automatic function logging with temporal benchmarking.
|
||||
|
||||
Args:
|
||||
logger: Logger instance (defaults to module logger)
|
||||
slow_threshold_ms: Log INFO if execution exceeds this (default 100ms)
|
||||
warn_threshold_ms: Log WARNING if execution exceeds this (default 500ms)
|
||||
include_args: Include function arguments in log (careful with sensitive data)
|
||||
|
||||
Usage:
|
||||
@logged()
|
||||
async def my_function(): ...
|
||||
|
||||
@logged(slow_threshold_ms=50, warn_threshold_ms=200)
|
||||
def critical_path(): ...
|
||||
"""
|
||||
def decorator(func: Callable):
|
||||
nonlocal logger
|
||||
if logger is None:
|
||||
logger = logging.getLogger(func.__module__)
|
||||
|
||||
func_name = f"{func.__module__}.{func.__qualname__}"
|
||||
|
||||
def _create_span() -> TraceSpan:
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
return TraceSpan(
|
||||
name=func_name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
|
||||
def _log_completion(span: TraceSpan, error: Exception = None):
|
||||
span.end_time = time.perf_counter()
|
||||
duration = span.duration_ms
|
||||
tid = span.trace_id[:8]
|
||||
|
||||
if error:
|
||||
logger.error(f"[{tid}] {func_name} FAILED after {duration:.2f}ms: {error}", exc_info=True)
|
||||
elif duration >= warn_threshold_ms:
|
||||
logger.warning(f"[{tid}] {func_name} SLOW: {duration:.2f}ms (threshold: {warn_threshold_ms}ms)")
|
||||
elif duration >= slow_threshold_ms:
|
||||
logger.info(f"[{tid}] {func_name} completed in {duration:.2f}ms")
|
||||
else:
|
||||
logger.debug(f"[{tid}] {func_name} completed in {duration:.2f}ms")
|
||||
|
||||
@functools.wraps(func)
|
||||
async def async_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
try:
|
||||
result = await func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
@functools.wraps(func)
|
||||
def sync_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
try:
|
||||
result = func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
return async_wrapper if asyncio.iscoroutinefunction(func) else sync_wrapper
|
||||
return decorator
|
||||
|
||||
|
||||
# === Context Manager ===
|
||||
|
||||
class trace_span:
|
||||
"""
|
||||
Context manager for manual span creation.
|
||||
|
||||
Usage:
|
||||
with trace_span("database_query"):
|
||||
result = db.execute(query)
|
||||
|
||||
async with trace_span("external_api_call"):
|
||||
response = await client.get(url)
|
||||
"""
|
||||
def __init__(self, name: str, logger: logging.Logger = None):
|
||||
self.name = name
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
self.span: Optional[TraceSpan] = None
|
||||
self.token = None
|
||||
|
||||
def __enter__(self):
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
|
||||
self.span = TraceSpan(
|
||||
name=self.name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
self.token = _current_span.set(self.span)
|
||||
self.logger.debug(f"[{self.span.trace_id[:8]}] -> {self.name}")
|
||||
return self.span
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
if self.span:
|
||||
self.span.end_time = time.perf_counter()
|
||||
duration = self.span.duration_ms
|
||||
tid = self.span.trace_id[:8]
|
||||
if exc_val:
|
||||
self.logger.error(f"[{tid}] {self.name} FAILED: {duration:.2f}ms")
|
||||
else:
|
||||
self.logger.debug(f"[{tid}] {self.name}: {duration:.2f}ms")
|
||||
if self.token:
|
||||
_current_span.reset(self.token)
|
||||
return False
|
||||
|
||||
async def __aenter__(self):
|
||||
return self.__enter__()
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
return self.__exit__(exc_type, exc_val, exc_tb)
|
||||
```
|
||||
|
||||
**Example output:**
|
||||
```
|
||||
DEBUG [a1b2c3d4] -> src.domains.users.controller.get_user
|
||||
DEBUG [a1b2c3d4] -> src.domains.users.service.fetch_from_db
|
||||
DEBUG [a1b2c3d4] src.domains.users.service.fetch_from_db: 12.34ms
|
||||
DEBUG [a1b2c3d4] src.domains.users.controller.get_user completed in 15.67ms
|
||||
WARN [a1b2c3d4] src.domains.reports.controller.generate SLOW: 523.45ms
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Request Context (User Provider)
|
||||
|
||||
### Singleton Pattern with ContextVars
|
||||
|
||||
```python
|
||||
# src/shared/context.py
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
from contextvars import ContextVar
|
||||
|
||||
|
||||
@dataclass
|
||||
class User:
|
||||
"""Authenticated user context."""
|
||||
id: str
|
||||
email: str
|
||||
tenant_id: Optional[str] = None
|
||||
roles: list[str] = None
|
||||
|
||||
def __post_init__(self):
|
||||
if self.roles is None:
|
||||
self.roles = []
|
||||
|
||||
|
||||
_current_user: ContextVar[Optional[User]] = ContextVar('current_user', default=None)
|
||||
|
||||
|
||||
class UserProvider:
|
||||
"""
|
||||
Singleton for request-scoped user context.
|
||||
|
||||
Set once per request in middleware, accessible everywhere without
|
||||
passing user through function parameters.
|
||||
"""
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
return cls._instance
|
||||
|
||||
def set_user(self, user: User) -> None:
|
||||
_current_user.set(user)
|
||||
|
||||
def get_user(self) -> Optional[User]:
|
||||
return _current_user.get()
|
||||
|
||||
def clear_user(self) -> None:
|
||||
_current_user.set(None)
|
||||
|
||||
@property
|
||||
def current_user(self) -> Optional[User]:
|
||||
return self.get_user()
|
||||
|
||||
|
||||
# Global singleton
|
||||
user_provider = UserProvider()
|
||||
|
||||
|
||||
def get_current_user() -> Optional[User]:
|
||||
"""Convenience function to get current user."""
|
||||
return user_provider.get_user()
|
||||
|
||||
|
||||
def require_user() -> User:
|
||||
"""Get current user or raise if not authenticated."""
|
||||
user = user_provider.get_user()
|
||||
if user is None:
|
||||
raise ValueError("No authenticated user in context")
|
||||
return user
|
||||
```
|
||||
|
||||
**Usage in middleware:**
|
||||
```python
|
||||
@app.middleware("http")
|
||||
async def auth_middleware(request: Request, call_next):
|
||||
# Validate token, get user...
|
||||
user = await validate_token(request)
|
||||
if user:
|
||||
user_provider.set_user(user)
|
||||
try:
|
||||
response = await call_next(request)
|
||||
return response
|
||||
finally:
|
||||
user_provider.clear_user()
|
||||
```
|
||||
|
||||
**Usage in any function:**
|
||||
```python
|
||||
from src.shared.context import get_current_user, require_user
|
||||
|
||||
async def some_business_logic():
|
||||
user = require_user() # Raises if not authenticated
|
||||
# Use user.id, user.tenant_id, etc.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Application Entry Point
|
||||
|
||||
### Clean main.py
|
||||
|
||||
```python
|
||||
# src/main.py
|
||||
from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from src.shared.config import get_settings
|
||||
from src.shared.logging import setup_logging, get_logger
|
||||
from src.domains.router import root_router
|
||||
|
||||
settings = get_settings()
|
||||
setup_logging(settings.log_level)
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Application startup and shutdown."""
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"Starting {settings.app_name} v{settings.app_version}")
|
||||
logger.info(f"Debug: {settings.debug}")
|
||||
logger.info("=" * 60)
|
||||
|
||||
# Initialize resources (DB connections, caches, etc.)
|
||||
|
||||
yield
|
||||
|
||||
# Cleanup resources
|
||||
logger.info("Shutting down")
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title=settings.app_name,
|
||||
version=settings.app_version,
|
||||
docs_url="/docs",
|
||||
redoc_url=None,
|
||||
lifespan=lifespan,
|
||||
debug=settings.debug,
|
||||
)
|
||||
|
||||
# CORS
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=settings.cors_origins,
|
||||
allow_credentials=settings.cors_credentials,
|
||||
allow_methods=settings.cors_methods,
|
||||
allow_headers=settings.cors_headers,
|
||||
)
|
||||
|
||||
# Include all routes from domain router
|
||||
app.include_router(root_router)
|
||||
|
||||
|
||||
# Global exception handler
|
||||
@app.exception_handler(Exception)
|
||||
async def global_exception_handler(request, exc):
|
||||
logger.error(f"Unhandled exception: {exc}", exc_info=True)
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={"detail": "Internal server error", "type": type(exc).__name__}
|
||||
)
|
||||
```
|
||||
|
||||
### Root Router
|
||||
|
||||
```python
|
||||
# src/domains/router.py
|
||||
from fastapi import APIRouter
|
||||
from src.domains.health.controller import health_controller
|
||||
# from src.domains.users.controller import users_controller
|
||||
# from src.domains.items.controller import items_controller
|
||||
|
||||
root_router = APIRouter()
|
||||
|
||||
# Include all domain routers
|
||||
root_router.include_router(health_controller.router)
|
||||
# root_router.include_router(users_controller.router, prefix="/users", tags=["Users"])
|
||||
# root_router.include_router(items_controller.router, prefix="/items", tags=["Items"])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Dependency Management
|
||||
|
||||
### CRITICAL: Always Start with Latest Safe Versions
|
||||
|
||||
**When setting up a new project, ALWAYS search for the latest version of each package and verify it has no known CVEs.**
|
||||
|
||||
Do NOT copy version numbers from old projects or templates. Package versions in documentation become outdated quickly.
|
||||
|
||||
**Process for each dependency:**
|
||||
1. Search: `"{package} pypi latest version {current_year}"`
|
||||
2. Check PyPI directly: `https://pypi.org/project/{package}/`
|
||||
3. Search: `"{package} CVE vulnerability {current_year}"`
|
||||
4. Verify no unpatched CVEs affect the latest version
|
||||
5. Pin to the verified latest safe version
|
||||
|
||||
**Example search queries:**
|
||||
```
|
||||
"fastapi pypi latest version 2026"
|
||||
"starlette CVE vulnerability 2026"
|
||||
"pydantic-ai pypi latest version 2026"
|
||||
```
|
||||
|
||||
### requirements.txt (Production)
|
||||
|
||||
```
|
||||
# Production Dependencies
|
||||
# Minor version pinning (~=) allows patch updates for security fixes
|
||||
# CVE check date: YYYY-MM-DD <-- UPDATE THIS DATE
|
||||
# CVE sources: PyPI, GitHub Advisories, Snyk, NVD
|
||||
|
||||
# Core - VERIFY LATEST VERSIONS BEFORE USE
|
||||
fastapi~=0.128.0
|
||||
starlette~=0.50.0 # CVE-2025-62727, CVE-2025-54121 fixed
|
||||
uvicorn[standard]~=0.40.0
|
||||
pydantic~=2.12.4 # CVE-2024-3772 (ReDoS) fixed in 2.4.0+
|
||||
pydantic-settings~=2.12.0
|
||||
|
||||
# HTTP client
|
||||
httpx~=0.28.1
|
||||
aiofiles~=25.1.0
|
||||
|
||||
# Utilities
|
||||
python-multipart~=0.0.21
|
||||
python-dotenv~=1.2.1
|
||||
```
|
||||
|
||||
### requirements-dev.txt (Development)
|
||||
|
||||
```
|
||||
# Development Dependencies
|
||||
# NOT included in production Docker image
|
||||
# CVE check date: YYYY-MM-DD <-- UPDATE THIS DATE
|
||||
|
||||
-r requirements.txt
|
||||
|
||||
# Testing - VERIFY LATEST VERSIONS BEFORE USE
|
||||
pytest~=9.0.2
|
||||
anyio~=4.12.1 # Includes pytest-anyio plugin
|
||||
pytest-cov~=7.0.0
|
||||
|
||||
# Security auditing
|
||||
pip-audit~=2.9.0
|
||||
|
||||
# Type checking
|
||||
mypy~=1.19.1
|
||||
|
||||
# Linting (optional)
|
||||
# ruff~=0.9.0
|
||||
```
|
||||
|
||||
### CVE Check Process
|
||||
|
||||
Before adding or updating dependencies:
|
||||
|
||||
1. **Check PyPI** for security advisories: `https://pypi.org/project/{package}/`
|
||||
2. **GitHub Security Advisories**: `https://github.com/advisories`
|
||||
3. **Snyk vulnerability database**: `https://snyk.io/vuln`
|
||||
4. **NVD**: `https://nvd.nist.gov/vuln/search`
|
||||
5. **Run pip-audit**: `pip-audit` before releases
|
||||
|
||||
Document decisions in requirements.txt:
|
||||
```
|
||||
package~=1.2.0 # CVE-YYYY-XXXXX: pinned due to vulnerability in < 1.2.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Exception Handling
|
||||
|
||||
### Custom Exception Hierarchy
|
||||
|
||||
```python
|
||||
# src/shared/exceptions.py
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
class AppException(Exception):
|
||||
"""Base exception for application errors."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
status_code: int = 500,
|
||||
error_code: Optional[str] = None,
|
||||
details: Optional[dict[str, Any]] = None,
|
||||
):
|
||||
self.message = message
|
||||
self.status_code = status_code
|
||||
self.error_code = error_code or self.__class__.__name__
|
||||
self.details = details or {}
|
||||
super().__init__(message)
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"error": self.error_code,
|
||||
"message": self.message,
|
||||
"details": self.details,
|
||||
}
|
||||
|
||||
|
||||
class NotFoundError(AppException):
|
||||
def __init__(self, resource: str, identifier: Any):
|
||||
super().__init__(
|
||||
message=f"{resource} not found: {identifier}",
|
||||
status_code=404,
|
||||
details={"resource": resource, "identifier": str(identifier)},
|
||||
)
|
||||
|
||||
|
||||
class ValidationError(AppException):
|
||||
def __init__(self, message: str, field: Optional[str] = None):
|
||||
super().__init__(
|
||||
message=message,
|
||||
status_code=422,
|
||||
details={"field": field} if field else {},
|
||||
)
|
||||
|
||||
|
||||
class AuthenticationError(AppException):
|
||||
def __init__(self, message: str = "Authentication required"):
|
||||
super().__init__(message=message, status_code=401)
|
||||
|
||||
|
||||
class AuthorizationError(AppException):
|
||||
def __init__(self, message: str = "Permission denied"):
|
||||
super().__init__(message=message, status_code=403)
|
||||
|
||||
|
||||
class ConflictError(AppException):
|
||||
def __init__(self, message: str):
|
||||
super().__init__(message=message, status_code=409)
|
||||
|
||||
|
||||
class RateLimitError(AppException):
|
||||
def __init__(self, retry_after: int = 60):
|
||||
super().__init__(
|
||||
message="Rate limit exceeded",
|
||||
status_code=429,
|
||||
details={"retry_after": retry_after},
|
||||
)
|
||||
```
|
||||
|
||||
### Exception Handler
|
||||
|
||||
```python
|
||||
# In main.py
|
||||
from src.shared.exceptions import AppException
|
||||
|
||||
@app.exception_handler(AppException)
|
||||
async def app_exception_handler(request, exc: AppException):
|
||||
return JSONResponse(
|
||||
status_code=exc.status_code,
|
||||
content=exc.to_dict(),
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Testing
|
||||
|
||||
### conftest.py
|
||||
|
||||
```python
|
||||
# tests/conftest.py
|
||||
import pytest
|
||||
from httpx import AsyncClient, ASGITransport
|
||||
from src.main import app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anyio_backend():
|
||||
return "asyncio"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def client():
|
||||
"""Async test client."""
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test"
|
||||
) as ac:
|
||||
yield ac
|
||||
```
|
||||
|
||||
### Example Test
|
||||
|
||||
```python
|
||||
# tests/test_health.py
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_health_check(client):
|
||||
response = await client.get("/health")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "healthy"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Summary Checklist
|
||||
|
||||
### Project Setup
|
||||
- [ ] Domain-based directory structure (`src/domains/`)
|
||||
- [ ] Shared utilities in `src/shared/`
|
||||
- [ ] Clean `main.py` (no routes, only app setup)
|
||||
- [ ] Root router composing domain routers
|
||||
|
||||
### Patterns
|
||||
- [ ] BaseController with lazy router
|
||||
- [ ] BaseSchema with standard config
|
||||
- [ ] Pydantic Settings for configuration
|
||||
- [ ] Logger decorator with temporal benchmarking
|
||||
- [ ] UserProvider singleton for request context
|
||||
- [ ] Custom exception hierarchy
|
||||
|
||||
### Dependencies
|
||||
- [ ] Minor version pinning (`~=`)
|
||||
- [ ] Separate prod/dev requirements
|
||||
- [ ] CVE check before updates
|
||||
- [ ] pip-audit in CI/CD
|
||||
|
||||
### Quality
|
||||
- [ ] Pytest with async support
|
||||
- [ ] Type hints throughout
|
||||
- [ ] Docstrings on public APIs
|
||||
- [ ] CHANGELOG.md maintained
|
||||
@@ -0,0 +1,628 @@
|
||||
# Webber FastAPI Boilerplate Plan
|
||||
|
||||
## Overview
|
||||
Set up FastAPI boilerplate for "Webber" - a multi-agent AI development system (similar to Claude Code, but local with different models). Follows core-api patterns with defensive coding practices.
|
||||
|
||||
**Key Decision: PydanticAI Framework**
|
||||
After research, [PydanticAI](https://ai.pydantic.dev/) is the recommended agent coordination framework:
|
||||
- Model-agnostic: supports Ollama, OpenAI, Anthropic, and 20+ providers
|
||||
- Type-safe with Pydantic validation (same ecosystem as FastAPI)
|
||||
- Built-in tool/function calling with automatic schema generation
|
||||
- Multi-agent support for complex workflows
|
||||
- Maintained by Pydantic team (285M+ monthly downloads)
|
||||
|
||||
**Port: 8086** (next available slot after Headscale 8085 per CONTAINERS.md)
|
||||
|
||||
**Default Models (always hot in VRAM on tower-of-joy):**
|
||||
- Agent reasoning: `mistral-nemo-large:latest`
|
||||
- Embeddings: `nomic-embed-text:latest`
|
||||
|
||||
**Target Clients:**
|
||||
- **Tatlock Butler**: External advisor integration for coding/software guidance
|
||||
- **CLI Interface**: TBD - command-line interface for local development
|
||||
|
||||
**Multi-tenancy:** API key authentication integrated with tatlock-ui/core-api user management
|
||||
|
||||
---
|
||||
|
||||
## 1. Directory Structure
|
||||
|
||||
```
|
||||
webber/
|
||||
├── AGENTS.md # Expanded with defensive LLM guidelines
|
||||
├── README.md # Project overview
|
||||
├── CHANGELOG.md # Version history
|
||||
├── pyproject.toml # Package metadata
|
||||
├── requirements.txt # Production dependencies only (~= pinned)
|
||||
├── requirements-dev.txt # Dev/test dependencies (pytest, pip-audit, etc.)
|
||||
├── .env.example # Environment template
|
||||
├── wakeup.sh # Dev startup (update port to 8086)
|
||||
│
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── main.py # FastAPI app, lifespan, user provider init
|
||||
│ │ # NO routes here - delegates to domain routers
|
||||
│ │
|
||||
│ ├── shared/ # Cross-cutting concerns
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── base.py # BaseController, BaseSchema
|
||||
│ │ ├── config.py # Pydantic BaseSettings
|
||||
│ │ ├── logging.py # Logger decorator + centralized setup
|
||||
│ │ ├── exceptions.py # Custom exception hierarchy
|
||||
│ │ ├── auth.py # API key validation, multi-tenant support
|
||||
│ │ └── context.py # UserProvider singleton, request context
|
||||
│ │
|
||||
│ └── domains/ # Feature domains (each with router.py)
|
||||
│ ├── __init__.py
|
||||
│ ├── router.py # Root router - includes all domain routers
|
||||
│ │
|
||||
│ ├── health/ # Health endpoints
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── router.py # Health routes
|
||||
│ │ └── controller.py # Health logic
|
||||
│ │
|
||||
│ ├── auth/ # Authentication domain
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── router.py # Auth routes (API key mgmt)
|
||||
│ │ ├── controller.py
|
||||
│ │ └── schemas.py
|
||||
│ │
|
||||
│ │── agents/ # Agent domain container
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── router.py # Agent routes (lists agents, runs them)
|
||||
│ │ ├── controller.py # Agent orchestration logic
|
||||
│ │ ├── schemas.py
|
||||
│ │ │
|
||||
│ │ ├── explore/ # Explore agent (codebase navigation)
|
||||
│ │ │ ├── __init__.py
|
||||
│ │ │ ├── agent.py # PydanticAI agent definition
|
||||
│ │ │ └── prompts.py # System prompts
|
||||
│ │ │
|
||||
│ │ ├── plan/ # Plan agent (implementation design)
|
||||
│ │ │ ├── __init__.py
|
||||
│ │ │ ├── agent.py
|
||||
│ │ │ └── prompts.py
|
||||
│ │ │
|
||||
│ │ └── task/ # Task agent (execution)
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── agent.py
|
||||
│ │ └── prompts.py
|
||||
│ │
|
||||
│ └── tools/ # Tool domain container
|
||||
│ ├── __init__.py
|
||||
│ ├── router.py # Tool routes (list tools, execute)
|
||||
│ ├── controller.py # Tool orchestration
|
||||
│ ├── schemas.py
|
||||
│ │
|
||||
│ ├── file/ # File operation tools
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── read.py
|
||||
│ │ ├── write.py
|
||||
│ │ └── glob.py
|
||||
│ │
|
||||
│ ├── shell/ # Shell execution tools
|
||||
│ │ ├── __init__.py
|
||||
│ │ └── bash.py
|
||||
│ │
|
||||
│ └── search/ # Search tools
|
||||
│ ├── __init__.py
|
||||
│ ├── grep.py
|
||||
│ └── web.py
|
||||
│
|
||||
├── tests/
|
||||
│ ├── __init__.py
|
||||
│ ├── conftest.py
|
||||
│ └── test_health.py
|
||||
│
|
||||
└── docs/
|
||||
└── architecture.md
|
||||
```
|
||||
|
||||
### Key Architectural Decisions
|
||||
|
||||
1. **Clean main.py**: Only app creation, lifespan, and UserProvider init. All routes in domain routers.
|
||||
2. **Domain routers**: Each domain has `router.py` that defines routes. Root `domains/router.py` composes them.
|
||||
3. **Separate agent domains**: Each agent type (explore, plan, task) in its own subdir under `agents/`.
|
||||
4. **Separate tool domains**: Each tool category (file, shell, search) in its own subdir under `tools/`.
|
||||
5. **UserProvider singleton**: Set once in main.py lifespan, accessible everywhere via `shared/context.py`.
|
||||
6. **Multi-tenant auth**: API key validation in `shared/auth.py`, integrates with tatlock-ui/core-api.
|
||||
|
||||
---
|
||||
|
||||
## 2. Key Files to Create
|
||||
|
||||
### Phase 1: Foundation (fully implemented)
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/shared/base.py` | BaseController, BaseSchema |
|
||||
| `src/shared/config.py` | Settings via Pydantic BaseSettings |
|
||||
| `src/shared/logging.py` | Logger decorator + centralized setup |
|
||||
| `src/shared/exceptions.py` | Custom exception hierarchy |
|
||||
| `src/shared/auth.py` | API key validation, tatlock integration stub |
|
||||
| `src/shared/context.py` | UserProvider singleton pattern |
|
||||
| `src/main.py` | FastAPI app, lifespan, UserProvider init (no routes!) |
|
||||
| `src/domains/router.py` | Root router composing all domain routers |
|
||||
| `src/domains/health/router.py` | Health routes |
|
||||
| `src/domains/health/controller.py` | Health logic |
|
||||
| `pyproject.toml` | Package metadata, pytest config |
|
||||
| `requirements.txt` | Production deps (~= pinned) |
|
||||
| `requirements-dev.txt` | Dev/test deps (pytest, pip-audit) |
|
||||
| `.env.example` | Environment variable template |
|
||||
| `tests/conftest.py` | Pytest fixtures |
|
||||
| `tests/test_health.py` | Basic endpoint tests |
|
||||
|
||||
### Phase 2: Placeholders (structure + README docs)
|
||||
| Directory | Purpose |
|
||||
|-----------|---------|
|
||||
| `src/domains/auth/` | API key management (stub) |
|
||||
| `src/domains/agents/` | Agent container with explore/plan/task subdirs |
|
||||
| `src/domains/tools/` | Tool container with file/shell/search subdirs |
|
||||
| `docs/architecture.md` | System design documentation |
|
||||
|
||||
---
|
||||
|
||||
## 3. Dependency Management
|
||||
|
||||
### requirements.txt (Production - baked into Docker)
|
||||
```
|
||||
# Webber Production Dependencies
|
||||
# Minor version pinning (~=) for security patches
|
||||
# CVE check date: 2026-01-09
|
||||
# CVE check sources: PyPI, GitHub Advisories, Snyk, NVD
|
||||
|
||||
# Core FastAPI
|
||||
fastapi~=0.115.0
|
||||
starlette~=0.45.0
|
||||
uvicorn[standard]~=0.34.0
|
||||
pydantic~=2.11.0
|
||||
pydantic-settings~=2.7.0
|
||||
|
||||
# Agent Framework
|
||||
pydantic-ai~=0.0.39 # Multi-agent LLM orchestration
|
||||
|
||||
# HTTP
|
||||
httpx~=0.28.0
|
||||
aiofiles~=24.1.0
|
||||
|
||||
# Utilities
|
||||
python-multipart~=0.0.18
|
||||
python-dotenv~=1.0.0
|
||||
```
|
||||
|
||||
### requirements-dev.txt (Dev/Test only - NOT in Docker)
|
||||
```
|
||||
# Webber Development Dependencies
|
||||
# Install with: pip install -r requirements-dev.txt
|
||||
|
||||
-r requirements.txt # Include production deps
|
||||
|
||||
# Testing
|
||||
pytest~=8.3.0
|
||||
pytest-asyncio~=0.24.0
|
||||
pytest-cov~=6.0.0
|
||||
|
||||
# Security auditing
|
||||
pip-audit~=2.7.0 # Run before releases: pip-audit
|
||||
|
||||
# Type checking
|
||||
mypy~=1.13.0
|
||||
|
||||
# Code formatting (optional)
|
||||
# ruff~=0.8.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. AGENTS.md Additions
|
||||
|
||||
Add these new sections:
|
||||
|
||||
### Section 3: Defensive LLM Coding Practices
|
||||
- Input validation requirements
|
||||
- Output parsing guidelines (expect malformed responses)
|
||||
- Timeout and retry policies
|
||||
- Security: no secrets in prompts, sandbox execution
|
||||
|
||||
### Section 4: Pattern Reuse Requirements
|
||||
- Search existing code before writing new
|
||||
- Check `src/shared/` for base classes
|
||||
- Follow domain structure template
|
||||
- Code review checklist
|
||||
|
||||
### Section 5: CVE Check Process
|
||||
- Check PyPI, GitHub Advisories, Snyk, NVD before adding deps
|
||||
- Document CVE decisions in requirements.txt
|
||||
- Run `pip-audit` before releases
|
||||
|
||||
### Section 6: Mandatory Documentation
|
||||
- Required reading before work: AGENTS.md, docs/architecture.md, src/shared/base.py
|
||||
- Changelog and docstring requirements
|
||||
|
||||
### Section 7: Project Structure Reference
|
||||
- Directory tree with explanations
|
||||
- Domain structure template
|
||||
|
||||
---
|
||||
|
||||
## 5. Configuration (Settings)
|
||||
|
||||
Environment variables for:
|
||||
- **App**: DEBUG, LOG_LEVEL
|
||||
- **Server**: HOST, PORT (default **8086** per CONTAINERS.md allocation)
|
||||
- **CORS**: origins, methods, headers
|
||||
- **LLM Models** (hot in VRAM on tower-of-joy):
|
||||
- OLLAMA_URL (default: http://192.168.86.149:11434)
|
||||
- OLLAMA_AGENT_MODEL (default: mistral-nemo-large:latest)
|
||||
- OLLAMA_EMBED_MODEL (default: nomic-embed-text:latest)
|
||||
- **Auth**:
|
||||
- TATLOCK_API_URL (default: http://192.168.86.149:8000)
|
||||
- Internal API key for tatlock user validation
|
||||
- **Tools**: TOOL_TIMEOUT_SECONDS, SANDBOX_ENABLED, ALLOWED_PATHS
|
||||
- **Sessions**: SESSION_TTL_HOURS, MAX_CONTEXT_TOKENS
|
||||
|
||||
---
|
||||
|
||||
## 6. Core Patterns
|
||||
|
||||
### Logger Decorator with Temporal Benchmarking (shared/logging.py)
|
||||
```python
|
||||
import functools
|
||||
import asyncio
|
||||
import time
|
||||
import logging
|
||||
from typing import Callable, Optional
|
||||
from contextvars import ContextVar
|
||||
from dataclasses import dataclass, field
|
||||
from uuid import uuid4
|
||||
|
||||
# Trace context for nested timing
|
||||
@dataclass
|
||||
class TraceSpan:
|
||||
name: str
|
||||
trace_id: str
|
||||
parent_id: Optional[str] = None
|
||||
span_id: str = field(default_factory=lambda: uuid4().hex[:8])
|
||||
start_time: float = field(default_factory=time.perf_counter)
|
||||
end_time: Optional[float] = None
|
||||
|
||||
@property
|
||||
def duration_ms(self) -> float:
|
||||
if self.end_time is None:
|
||||
return (time.perf_counter() - self.start_time) * 1000
|
||||
return (self.end_time - self.start_time) * 1000
|
||||
|
||||
# Context variable for trace propagation
|
||||
_current_span: ContextVar[Optional[TraceSpan]] = ContextVar('current_span', default=None)
|
||||
_trace_id: ContextVar[Optional[str]] = ContextVar('trace_id', default=None)
|
||||
|
||||
def get_current_trace_id() -> Optional[str]:
|
||||
"""Get current trace ID for correlation."""
|
||||
return _trace_id.get()
|
||||
|
||||
def logged(
|
||||
logger: logging.Logger = None,
|
||||
slow_threshold_ms: float = 100.0,
|
||||
warn_threshold_ms: float = 500.0,
|
||||
include_args: bool = False,
|
||||
):
|
||||
"""
|
||||
Decorator for automatic function logging with temporal benchmarking.
|
||||
|
||||
Args:
|
||||
logger: Logger instance (defaults to module logger)
|
||||
slow_threshold_ms: Log INFO if execution exceeds this (default 100ms)
|
||||
warn_threshold_ms: Log WARNING if execution exceeds this (default 500ms)
|
||||
include_args: Include function arguments in log (careful with sensitive data)
|
||||
|
||||
Usage:
|
||||
@logged()
|
||||
async def my_function(): ...
|
||||
|
||||
@logged(slow_threshold_ms=50, warn_threshold_ms=200)
|
||||
def critical_path(): ...
|
||||
"""
|
||||
def decorator(func: Callable):
|
||||
nonlocal logger
|
||||
if logger is None:
|
||||
logger = logging.getLogger(func.__module__)
|
||||
|
||||
func_name = f"{func.__module__}.{func.__qualname__}"
|
||||
|
||||
def _create_span() -> TraceSpan:
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
return TraceSpan(
|
||||
name=func_name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
|
||||
def _log_completion(span: TraceSpan, error: Exception = None):
|
||||
span.end_time = time.perf_counter()
|
||||
duration = span.duration_ms
|
||||
|
||||
# Build log context
|
||||
ctx = {
|
||||
"trace_id": span.trace_id,
|
||||
"span_id": span.span_id,
|
||||
"duration_ms": round(duration, 2),
|
||||
"func": func_name,
|
||||
}
|
||||
if span.parent_id:
|
||||
ctx["parent_id"] = span.parent_id
|
||||
|
||||
if error:
|
||||
logger.error(
|
||||
f"[{span.trace_id[:8]}] {func_name} FAILED after {duration:.2f}ms: {error}",
|
||||
extra=ctx,
|
||||
exc_info=True
|
||||
)
|
||||
elif duration >= warn_threshold_ms:
|
||||
logger.warning(
|
||||
f"[{span.trace_id[:8]}] {func_name} SLOW: {duration:.2f}ms (threshold: {warn_threshold_ms}ms)",
|
||||
extra=ctx
|
||||
)
|
||||
elif duration >= slow_threshold_ms:
|
||||
logger.info(
|
||||
f"[{span.trace_id[:8]}] {func_name} completed in {duration:.2f}ms",
|
||||
extra=ctx
|
||||
)
|
||||
else:
|
||||
logger.debug(
|
||||
f"[{span.trace_id[:8]}] {func_name} completed in {duration:.2f}ms",
|
||||
extra=ctx
|
||||
)
|
||||
|
||||
@functools.wraps(func)
|
||||
async def async_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
|
||||
if include_args:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}({args}, {kwargs})")
|
||||
else:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
|
||||
try:
|
||||
result = await func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
@functools.wraps(func)
|
||||
def sync_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
|
||||
if include_args:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}({args}, {kwargs})")
|
||||
else:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
|
||||
try:
|
||||
result = func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
return async_wrapper if asyncio.iscoroutinefunction(func) else sync_wrapper
|
||||
return decorator
|
||||
|
||||
|
||||
# Convenience for manual span creation (context manager)
|
||||
class trace_span:
|
||||
"""
|
||||
Context manager for manual span creation.
|
||||
|
||||
Usage:
|
||||
with trace_span("database_query"):
|
||||
result = await db.execute(query)
|
||||
|
||||
async with trace_span("llm_call"):
|
||||
response = await agent.run(prompt)
|
||||
"""
|
||||
def __init__(self, name: str, logger: logging.Logger = None):
|
||||
self.name = name
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
self.span: Optional[TraceSpan] = None
|
||||
self.token = None
|
||||
|
||||
def __enter__(self):
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
|
||||
self.span = TraceSpan(
|
||||
name=self.name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
self.token = _current_span.set(self.span)
|
||||
self.logger.debug(f"[{self.span.trace_id[:8]}] -> {self.name}")
|
||||
return self.span
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
if self.span:
|
||||
self.span.end_time = time.perf_counter()
|
||||
duration = self.span.duration_ms
|
||||
if exc_val:
|
||||
self.logger.error(f"[{self.span.trace_id[:8]}] {self.name} FAILED: {duration:.2f}ms")
|
||||
else:
|
||||
self.logger.debug(f"[{self.span.trace_id[:8]}] {self.name}: {duration:.2f}ms")
|
||||
if self.token:
|
||||
_current_span.reset(self.token)
|
||||
return False
|
||||
|
||||
async def __aenter__(self):
|
||||
return self.__enter__()
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
return self.__exit__(exc_type, exc_val, exc_tb)
|
||||
```
|
||||
|
||||
**Example output:**
|
||||
```
|
||||
DEBUG [a1b2c3d4] -> src.domains.agents.controller.run_agent
|
||||
DEBUG [a1b2c3d4] -> src.domains.llm.service.call_ollama
|
||||
DEBUG [a1b2c3d4] src.domains.llm.service.call_ollama: 45.23ms
|
||||
INFO [a1b2c3d4] src.domains.agents.controller.run_agent completed in 156.78ms
|
||||
WARN [a1b2c3d4] src.domains.tools.file.read.read_file SLOW: 523.45ms (threshold: 500ms)
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- **Trace IDs**: Correlate logs across nested calls
|
||||
- **Parent/child spans**: Track call hierarchy
|
||||
- **Configurable thresholds**: `slow_threshold_ms` (INFO), `warn_threshold_ms` (WARNING)
|
||||
- **Context manager**: `trace_span()` for manual instrumentation of code blocks
|
||||
- **Zero overhead path**: Fast path for sub-threshold calls (DEBUG only)
|
||||
|
||||
### UserProvider Singleton (shared/context.py)
|
||||
```python
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
from contextvars import ContextVar
|
||||
|
||||
@dataclass
|
||||
class User:
|
||||
id: str
|
||||
email: str
|
||||
api_key: str
|
||||
tenant_id: Optional[str] = None
|
||||
|
||||
# Context variable for request-scoped user
|
||||
_current_user: ContextVar[Optional[User]] = ContextVar('current_user', default=None)
|
||||
|
||||
class UserProvider:
|
||||
"""Singleton for user context management."""
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
return cls._instance
|
||||
|
||||
def set_user(self, user: User) -> None:
|
||||
_current_user.set(user)
|
||||
|
||||
def get_user(self) -> Optional[User]:
|
||||
return _current_user.get()
|
||||
|
||||
def clear_user(self) -> None:
|
||||
_current_user.set(None)
|
||||
|
||||
# Global singleton
|
||||
user_provider = UserProvider()
|
||||
```
|
||||
|
||||
### BaseController (from core-api)
|
||||
```python
|
||||
class BaseController(ABC):
|
||||
def __init__(self, prefix: str, tags: list[str]):
|
||||
self.prefix = prefix
|
||||
self.tags = tags
|
||||
self._router = None
|
||||
|
||||
@abstractmethod
|
||||
def create_router(self) -> APIRouter: pass
|
||||
|
||||
@property
|
||||
def router(self) -> APIRouter:
|
||||
if self._router is None:
|
||||
self._router = self.create_router()
|
||||
return self._router
|
||||
```
|
||||
|
||||
### PydanticAI Agent Pattern (placeholder for future)
|
||||
```python
|
||||
from pydantic_ai import Agent
|
||||
from pydantic_ai.models.ollama import OllamaModel
|
||||
|
||||
# Use the hot model from VRAM
|
||||
agent = Agent(
|
||||
OllamaModel('mistral-nemo-large:latest'),
|
||||
system_prompt='You are a helpful assistant.',
|
||||
)
|
||||
|
||||
@agent.tool
|
||||
async def search_files(ctx, pattern: str) -> str:
|
||||
"""Search for files matching pattern."""
|
||||
pass # Implementation in tools/search/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Implementation Order
|
||||
|
||||
1. **Create directory structure** (`src/`, `src/shared/`, `src/domains/`)
|
||||
2. **Implement shared modules** (base.py, config.py, logging.py, exceptions.py)
|
||||
3. **Create main.py** with FastAPI app and lifespan
|
||||
4. **Add health domain** as working example
|
||||
5. **Set up tests** (conftest.py, test_health.py)
|
||||
6. **Create placeholder domains** (llm, agents, tools - structure only)
|
||||
7. **Update AGENTS.md** with new sections
|
||||
8. **Create supporting files** (pyproject.toml, requirements.txt, .env.example)
|
||||
9. **Add docs/architecture.md**
|
||||
|
||||
---
|
||||
|
||||
## 8. Verification
|
||||
|
||||
After implementation:
|
||||
1. `./wakeup.sh` starts server without errors
|
||||
2. `curl http://localhost:8086/health` returns healthy
|
||||
3. `http://localhost:8086/docs` shows API documentation
|
||||
4. `.venv/bin/python -m pytest tests/ -v` passes
|
||||
5. Code follows patterns in AGENTS.md
|
||||
|
||||
---
|
||||
|
||||
## 9. Critical Reference Files
|
||||
|
||||
- `/mnt/media/Projects/core-api/src/shared/base.py` - BaseController pattern
|
||||
- `/mnt/media/Projects/core-api/src/shared/config.py` - Settings pattern
|
||||
- `/mnt/media/Projects/core-api/src/domains/health/controller.py` - Controller example
|
||||
- https://ai.pydantic.dev/ - PydanticAI documentation
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**What will be created:**
|
||||
- Complete FastAPI project structure following core-api patterns
|
||||
- Working health endpoint at `http://localhost:8086/health`
|
||||
- **Clean main.py** - no routes, just app init and UserProvider setup
|
||||
- **Domain routers** - each domain has router.py, composed by root router
|
||||
- **Logger decorator** - centralized logging via `@logged` decorator
|
||||
- **UserProvider singleton** - request-scoped user context, no parameter passing
|
||||
- **Multi-tenant auth stub** - API key validation ready for tatlock integration
|
||||
- Separate **requirements.txt** (prod) and **requirements-dev.txt** (dev/test)
|
||||
- Placeholder domains with agent/tool subdirectories (explore, plan, task / file, shell, search)
|
||||
- Comprehensive AGENTS.md with defensive LLM coding practices, CVE checks, pattern reuse
|
||||
- Test infrastructure with pytest
|
||||
- docs/architecture.md explaining the system design
|
||||
|
||||
**What will NOT be created (deferred):**
|
||||
- Database layer (add when needed)
|
||||
- Full agent/tool implementations (PydanticAI patterns documented for future work)
|
||||
- Docker/deployment configuration (can add later)
|
||||
- CLI interface (TBD)
|
||||
|
||||
**Key decisions:**
|
||||
- Port: **8086**
|
||||
- Agent framework: **PydanticAI**
|
||||
- Default model: **mistral-nemo-large:latest** (hot in VRAM)
|
||||
- Embeddings: **nomic-embed-text:latest** (hot in VRAM)
|
||||
- No database initially
|
||||
- Separate prod/dev requirements
|
||||
- UserProvider singleton pattern for multi-tenancy
|
||||
@@ -0,0 +1,37 @@
|
||||
[project]
|
||||
name = "webber"
|
||||
version = "0.1.0"
|
||||
description = "Mrs. Webber - Multi-Agent AI Development System"
|
||||
authors = [
|
||||
{name = "jpmschweitzer"}
|
||||
]
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = {text = "MIT"}
|
||||
classifiers = [
|
||||
"Development Status :: 3 - Alpha",
|
||||
"Framework :: FastAPI",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: Software Development :: Code Generators",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=75.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["src*"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
python_files = ["test_*.py"]
|
||||
python_functions = ["test_*"]
|
||||
addopts = "-v"
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.12"
|
||||
warn_return_any = true
|
||||
warn_unused_ignores = true
|
||||
strict = false
|
||||
ignore_missing_imports = true
|
||||
@@ -0,0 +1,17 @@
|
||||
# Webber Development Dependencies
|
||||
# Install with: pip install -r requirements-dev.txt
|
||||
# NOT included in production Docker image
|
||||
# CVE check date: 2026-01-09
|
||||
|
||||
-r requirements.txt
|
||||
|
||||
# Testing
|
||||
pytest~=9.0.2
|
||||
anyio~=4.12.1 # Includes pytest-anyio plugin
|
||||
pytest-cov~=7.0.0
|
||||
|
||||
# Security auditing - run before releases: pip-audit
|
||||
pip-audit~=2.9.0
|
||||
|
||||
# Type checking
|
||||
mypy~=1.19.1
|
||||
@@ -0,0 +1,22 @@
|
||||
# Webber Production Dependencies
|
||||
# Minor version pinning (~=) allows patch updates for security fixes
|
||||
# CVE check date: 2026-01-09
|
||||
# CVE check sources: PyPI, GitHub Advisories, Snyk, NVD
|
||||
|
||||
# Core FastAPI
|
||||
fastapi~=0.128.0
|
||||
starlette~=0.50.0 # CVE-2025-62727, CVE-2025-54121 fixed
|
||||
uvicorn[standard]~=0.40.0
|
||||
pydantic~=2.12.4 # CVE-2024-3772 (ReDoS) fixed in 2.4.0+
|
||||
pydantic-settings~=2.12.0
|
||||
|
||||
# Agent Framework
|
||||
pydantic-ai~=1.40.0
|
||||
|
||||
# HTTP client
|
||||
httpx~=0.28.1
|
||||
aiofiles~=25.1.0
|
||||
|
||||
# Utilities
|
||||
python-multipart~=0.0.21
|
||||
python-dotenv~=1.2.1
|
||||
@@ -0,0 +1,53 @@
|
||||
# Agents Domain
|
||||
|
||||
This domain contains PydanticAI agent definitions and orchestration.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
agents/
|
||||
├── router.py # Agent routes (list, run)
|
||||
├── controller.py # Agent orchestration logic
|
||||
├── schemas.py # Request/response models
|
||||
│
|
||||
├── explore/ # Explore agent - codebase navigation
|
||||
│ ├── agent.py # PydanticAI agent definition
|
||||
│ └── prompts.py # System prompts
|
||||
│
|
||||
├── plan/ # Plan agent - implementation design
|
||||
│ ├── agent.py
|
||||
│ └── prompts.py
|
||||
│
|
||||
└── task/ # Task agent - execution
|
||||
├── agent.py
|
||||
└── prompts.py
|
||||
```
|
||||
|
||||
## PydanticAI Pattern
|
||||
|
||||
```python
|
||||
from pydantic_ai import Agent
|
||||
from pydantic_ai.models.ollama import OllamaModel
|
||||
from src.shared.config import get_settings
|
||||
|
||||
settings = get_settings()
|
||||
|
||||
explore_agent = Agent(
|
||||
OllamaModel(settings.ollama_agent_model, base_url=settings.ollama_url),
|
||||
system_prompt='You are a code exploration assistant...',
|
||||
)
|
||||
|
||||
@explore_agent.tool
|
||||
async def search_files(ctx, pattern: str) -> str:
|
||||
"""Search for files matching pattern."""
|
||||
# Implementation uses tools from src/domains/tools/
|
||||
pass
|
||||
```
|
||||
|
||||
## Adding a New Agent
|
||||
|
||||
1. Create a new directory under `agents/` (e.g., `agents/review/`)
|
||||
2. Create `agent.py` with PydanticAI Agent definition
|
||||
3. Create `prompts.py` with system prompts
|
||||
4. Register in `controller.py`
|
||||
5. Add tests in `tests/domains/test_agents/`
|
||||
@@ -0,0 +1,43 @@
|
||||
# Auth Domain
|
||||
|
||||
This domain handles API key management and authentication.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
auth/
|
||||
├── router.py # Auth routes
|
||||
├── controller.py # Auth logic
|
||||
└── schemas.py # Auth models
|
||||
```
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
1. Client sends `X-API-Key` header
|
||||
2. Middleware validates key (via `shared/auth.py`)
|
||||
3. User context set in `shared/context.py`
|
||||
4. Routes use `Depends(require_auth)` for protected endpoints
|
||||
|
||||
## Integration with Tatlock
|
||||
|
||||
API keys are validated against the tatlock-ui/core-api user management system.
|
||||
|
||||
```python
|
||||
# In shared/auth.py
|
||||
async def validate_api_key(api_key: str) -> Optional[User]:
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(
|
||||
f"{settings.tatlock_api_url}/auth/validate",
|
||||
headers={"X-API-Key": api_key}
|
||||
)
|
||||
if response.status_code == 200:
|
||||
return User(**response.json())
|
||||
return None
|
||||
```
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] Implement tatlock API key validation
|
||||
- [ ] Add API key generation endpoint
|
||||
- [ ] Add rate limiting per API key
|
||||
- [ ] Add usage tracking
|
||||
@@ -0,0 +1,71 @@
|
||||
"""
|
||||
Health check controller.
|
||||
|
||||
Provides service health and information endpoints.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
from pydantic import BaseModel
|
||||
|
||||
from src.shared.base import BaseController
|
||||
from src.shared.config import get_settings
|
||||
from src.shared.logging import get_logger, logged
|
||||
|
||||
logger = get_logger(__name__)
|
||||
settings = get_settings()
|
||||
|
||||
|
||||
class HealthResponse(BaseModel):
|
||||
"""Health check response."""
|
||||
status: str
|
||||
version: str
|
||||
service: str
|
||||
|
||||
|
||||
class InfoResponse(BaseModel):
|
||||
"""Service info response."""
|
||||
service: str
|
||||
version: str
|
||||
status: str
|
||||
docs: str
|
||||
debug: bool
|
||||
|
||||
|
||||
class HealthController(BaseController):
|
||||
"""Controller for health and info endpoints."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(prefix="", tags=["Health"])
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
router = APIRouter(tags=self.tags)
|
||||
|
||||
@router.get("/", response_model=InfoResponse, summary="Service information")
|
||||
@logged()
|
||||
async def root():
|
||||
"""Get service information."""
|
||||
return InfoResponse(
|
||||
service=settings.app_name,
|
||||
version=settings.app_version,
|
||||
status="healthy",
|
||||
docs="/docs",
|
||||
debug=settings.debug,
|
||||
)
|
||||
|
||||
@router.get("/health", response_model=HealthResponse, summary="Health check")
|
||||
@logged()
|
||||
async def health_check():
|
||||
"""
|
||||
Health check endpoint.
|
||||
|
||||
Returns service health status for monitoring and load balancers.
|
||||
"""
|
||||
return HealthResponse(
|
||||
status="healthy",
|
||||
version=settings.app_version,
|
||||
service=settings.app_name,
|
||||
)
|
||||
|
||||
return router
|
||||
|
||||
|
||||
health_controller = HealthController()
|
||||
@@ -0,0 +1,8 @@
|
||||
"""
|
||||
Health check routes.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
|
||||
from src.domains.health.controller import health_controller
|
||||
|
||||
router = health_controller.router
|
||||
@@ -0,0 +1,26 @@
|
||||
"""
|
||||
Root router - composes all domain routers.
|
||||
|
||||
Import and include domain routers here.
|
||||
main.py only includes this root_router.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
|
||||
from src.domains.health.router import router as health_router
|
||||
# from src.domains.auth.router import router as auth_router
|
||||
# from src.domains.agents.router import router as agents_router
|
||||
# from src.domains.tools.router import router as tools_router
|
||||
|
||||
root_router = APIRouter()
|
||||
|
||||
# Health (no prefix - root level)
|
||||
root_router.include_router(health_router)
|
||||
|
||||
# Auth domain
|
||||
# root_router.include_router(auth_router, prefix="/auth", tags=["Auth"])
|
||||
|
||||
# Agents domain
|
||||
# root_router.include_router(agents_router, prefix="/agents", tags=["Agents"])
|
||||
|
||||
# Tools domain
|
||||
# root_router.include_router(tools_router, prefix="/tools", tags=["Tools"])
|
||||
@@ -0,0 +1,70 @@
|
||||
# Tools Domain
|
||||
|
||||
This domain contains tool implementations for agent use.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
tools/
|
||||
├── router.py # Tool routes (list, execute)
|
||||
├── controller.py # Tool orchestration
|
||||
├── schemas.py # Tool request/response models
|
||||
│
|
||||
├── file/ # File operation tools
|
||||
│ ├── read.py # Read file contents
|
||||
│ ├── write.py # Write file contents
|
||||
│ └── glob.py # Find files by pattern
|
||||
│
|
||||
├── shell/ # Shell execution tools
|
||||
│ └── bash.py # Execute bash commands
|
||||
│
|
||||
└── search/ # Search tools
|
||||
├── grep.py # Search file contents
|
||||
└── web.py # Web search
|
||||
```
|
||||
|
||||
## Tool Pattern
|
||||
|
||||
Tools are registered with PydanticAI agents via the `@agent.tool` decorator.
|
||||
Each tool should:
|
||||
|
||||
1. Have clear input/output types
|
||||
2. Include a docstring (used by LLM)
|
||||
3. Handle errors gracefully
|
||||
4. Respect sandbox settings
|
||||
|
||||
```python
|
||||
from src.shared.config import get_settings
|
||||
from src.shared.logging import logged
|
||||
|
||||
settings = get_settings()
|
||||
|
||||
@logged()
|
||||
async def read_file(file_path: str, limit: int = 2000) -> str:
|
||||
"""
|
||||
Read contents of a file.
|
||||
|
||||
Args:
|
||||
file_path: Absolute path to the file
|
||||
limit: Maximum lines to read
|
||||
|
||||
Returns:
|
||||
File contents as string
|
||||
"""
|
||||
# Check path is allowed
|
||||
if settings.sandbox_enabled:
|
||||
# Validate against allowed_paths
|
||||
pass
|
||||
|
||||
# Read and return
|
||||
pass
|
||||
```
|
||||
|
||||
## Adding a New Tool
|
||||
|
||||
1. Create a new file in appropriate category (file/, shell/, search/)
|
||||
2. Implement the tool function with proper types and docstring
|
||||
3. Add `@logged()` decorator for timing
|
||||
4. Handle sandbox restrictions
|
||||
5. Register with agents that need it
|
||||
6. Add tests
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
"""
|
||||
Webber - Multi-Agent AI Development System
|
||||
|
||||
FastAPI application entry point.
|
||||
NO routes here - all routes delegated to domain routers.
|
||||
"""
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from src.shared.config import get_settings
|
||||
from src.shared.logging import setup_logging, get_logger
|
||||
from src.shared.exceptions import AppException
|
||||
from src.shared.context import user_provider
|
||||
from src.shared.auth import validate_api_key
|
||||
from src.domains.router import root_router
|
||||
|
||||
settings = get_settings()
|
||||
setup_logging(settings.log_level)
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Application startup and shutdown."""
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"Starting {settings.app_name} v{settings.app_version}")
|
||||
logger.info(f"Debug: {settings.debug}")
|
||||
logger.info(f"Port: {settings.port}")
|
||||
logger.info(f"Ollama: {settings.ollama_url}")
|
||||
logger.info(f"Agent model: {settings.ollama_agent_model}")
|
||||
logger.info("=" * 60)
|
||||
|
||||
# TODO: Initialize resources (LLM clients, etc.)
|
||||
|
||||
yield
|
||||
|
||||
# Cleanup
|
||||
logger.info("Shutting down")
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title=settings.app_name,
|
||||
version=settings.app_version,
|
||||
description="Multi-Agent AI Development System",
|
||||
docs_url="/docs",
|
||||
redoc_url=None,
|
||||
openapi_url="/openapi.json",
|
||||
lifespan=lifespan,
|
||||
debug=settings.debug,
|
||||
)
|
||||
|
||||
|
||||
# === Middleware ===
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=settings.cors_origins,
|
||||
allow_credentials=settings.cors_credentials,
|
||||
allow_methods=settings.cors_methods,
|
||||
allow_headers=settings.cors_headers,
|
||||
)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def auth_middleware(request: Request, call_next):
|
||||
"""
|
||||
Extract and validate API key, set user context.
|
||||
|
||||
Allows unauthenticated requests - individual routes decide if auth is required.
|
||||
"""
|
||||
api_key = request.headers.get("X-API-Key")
|
||||
|
||||
if api_key:
|
||||
user = await validate_api_key(api_key)
|
||||
if user:
|
||||
user_provider.set_user(user)
|
||||
|
||||
try:
|
||||
response = await call_next(request)
|
||||
return response
|
||||
finally:
|
||||
user_provider.clear_user()
|
||||
|
||||
|
||||
# === Exception Handlers ===
|
||||
|
||||
@app.exception_handler(AppException)
|
||||
async def app_exception_handler(request: Request, exc: AppException):
|
||||
"""Handle application exceptions."""
|
||||
logger.warning(f"AppException: {exc.error_code} - {exc.message}")
|
||||
return JSONResponse(
|
||||
status_code=exc.status_code,
|
||||
content=exc.to_dict(),
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def global_exception_handler(request: Request, exc: Exception):
|
||||
"""Catch-all exception handler."""
|
||||
logger.error(f"Unhandled exception: {exc}", exc_info=True)
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={
|
||||
"error": "InternalServerError",
|
||||
"message": "Internal server error",
|
||||
"details": {"type": type(exc).__name__},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# === Routes ===
|
||||
|
||||
app.include_router(root_router)
|
||||
@@ -0,0 +1,106 @@
|
||||
"""
|
||||
Authentication utilities.
|
||||
|
||||
Provides API key validation and integration with external auth services.
|
||||
"""
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import Request, HTTPException, Depends
|
||||
from fastapi.security import APIKeyHeader
|
||||
|
||||
from src.shared.config import get_settings
|
||||
from src.shared.context import User, user_provider
|
||||
from src.shared.logging import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
settings = get_settings()
|
||||
|
||||
# API key header scheme
|
||||
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
|
||||
|
||||
|
||||
async def validate_api_key(api_key: str) -> Optional[User]:
|
||||
"""
|
||||
Validate API key and return User if valid.
|
||||
|
||||
TODO: Integrate with tatlock-ui/core-api for real validation.
|
||||
Currently accepts any non-empty key for development.
|
||||
"""
|
||||
if not api_key:
|
||||
return None
|
||||
|
||||
# Development mode: accept any key
|
||||
if settings.debug:
|
||||
logger.debug(f"Debug mode: accepting API key {api_key[:8]}...")
|
||||
return User(
|
||||
id="dev-user",
|
||||
email="dev@localhost",
|
||||
api_key=api_key,
|
||||
tenant_id="dev-tenant",
|
||||
roles=["admin"],
|
||||
)
|
||||
|
||||
# TODO: Production mode - validate against tatlock API
|
||||
# async with httpx.AsyncClient() as client:
|
||||
# response = await client.get(
|
||||
# f"{settings.tatlock_api_url}/auth/validate",
|
||||
# headers={"X-API-Key": api_key}
|
||||
# )
|
||||
# if response.status_code == 200:
|
||||
# data = response.json()
|
||||
# return User(**data)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def get_api_key(
|
||||
api_key: Optional[str] = Depends(api_key_header),
|
||||
) -> Optional[str]:
|
||||
"""FastAPI dependency to extract API key from header."""
|
||||
return api_key
|
||||
|
||||
|
||||
async def get_current_user_dep(
|
||||
api_key: Optional[str] = Depends(get_api_key),
|
||||
) -> Optional[User]:
|
||||
"""
|
||||
FastAPI dependency to get current user from API key.
|
||||
|
||||
Returns None if no valid API key provided.
|
||||
"""
|
||||
if not api_key:
|
||||
return None
|
||||
return await validate_api_key(api_key)
|
||||
|
||||
|
||||
async def require_auth(
|
||||
user: Optional[User] = Depends(get_current_user_dep),
|
||||
) -> User:
|
||||
"""
|
||||
FastAPI dependency that requires authentication.
|
||||
|
||||
Raises 401 if no valid API key provided.
|
||||
"""
|
||||
if user is None:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid or missing API key",
|
||||
headers={"WWW-Authenticate": "ApiKey"},
|
||||
)
|
||||
return user
|
||||
|
||||
|
||||
async def require_admin(
|
||||
user: User = Depends(require_auth),
|
||||
) -> User:
|
||||
"""
|
||||
FastAPI dependency that requires admin role.
|
||||
|
||||
Raises 403 if user is not admin.
|
||||
"""
|
||||
if "admin" not in user.roles:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="Admin access required",
|
||||
)
|
||||
return user
|
||||
@@ -0,0 +1,74 @@
|
||||
"""
|
||||
Base classes for controllers and schemas.
|
||||
|
||||
All domain controllers and Pydantic models should inherit from these.
|
||||
"""
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class BaseController(ABC):
|
||||
"""
|
||||
Base controller with lazy router instantiation.
|
||||
|
||||
All domain controllers inherit from this and implement create_router().
|
||||
|
||||
Usage:
|
||||
class UsersController(BaseController):
|
||||
def __init__(self):
|
||||
super().__init__(prefix="/users", tags=["Users"])
|
||||
|
||||
def create_router(self) -> APIRouter:
|
||||
router = APIRouter(prefix=self.prefix, tags=self.tags)
|
||||
|
||||
@router.get("/")
|
||||
async def list_users():
|
||||
return []
|
||||
|
||||
return router
|
||||
|
||||
users_controller = UsersController()
|
||||
"""
|
||||
|
||||
def __init__(self, prefix: str, tags: list[str]):
|
||||
self.prefix = prefix
|
||||
self.tags = tags
|
||||
self._router = None
|
||||
|
||||
@abstractmethod
|
||||
def create_router(self) -> APIRouter:
|
||||
"""Create and configure the FastAPI router with all routes."""
|
||||
pass
|
||||
|
||||
@property
|
||||
def router(self) -> APIRouter:
|
||||
"""Lazy router instantiation."""
|
||||
if self._router is None:
|
||||
self._router = self.create_router()
|
||||
return self._router
|
||||
|
||||
|
||||
class BaseSchema(BaseModel):
|
||||
"""
|
||||
Base Pydantic model with standardized configuration.
|
||||
|
||||
All domain schemas should inherit from this.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(
|
||||
strict=False,
|
||||
populate_by_name=True,
|
||||
use_enum_values=True,
|
||||
validate_assignment=True,
|
||||
json_encoders={
|
||||
datetime: lambda v: v.isoformat() if v else None
|
||||
}
|
||||
)
|
||||
|
||||
def dict_without_none(self) -> dict[str, Any]:
|
||||
"""Return model as dict, excluding None values."""
|
||||
return {k: v for k, v in self.model_dump().items() if v is not None}
|
||||
@@ -0,0 +1,76 @@
|
||||
"""
|
||||
Application configuration via Pydantic Settings.
|
||||
|
||||
All settings loaded from environment variables or .env file.
|
||||
"""
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from functools import lru_cache
|
||||
from typing import Optional
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
def _get_version() -> str:
|
||||
"""Load version from pyproject.toml."""
|
||||
pyproject_path = Path(__file__).parent.parent.parent / "pyproject.toml"
|
||||
try:
|
||||
with open(pyproject_path, "rb") as f:
|
||||
return tomllib.load(f).get("project", {}).get("version", "0.0.0")
|
||||
except FileNotFoundError:
|
||||
return "0.0.0"
|
||||
|
||||
|
||||
__version__ = _get_version()
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Application settings loaded from environment."""
|
||||
|
||||
# Application
|
||||
app_name: str = "Webber"
|
||||
app_version: str = __version__
|
||||
debug: bool = False
|
||||
|
||||
# Server
|
||||
host: str = "0.0.0.0"
|
||||
port: int = 8086
|
||||
|
||||
# Logging
|
||||
log_level: str = "INFO"
|
||||
|
||||
# CORS
|
||||
cors_origins: list[str] = ["http://localhost:3000", "http://localhost:8080"]
|
||||
cors_credentials: bool = True
|
||||
cors_methods: list[str] = ["*"]
|
||||
cors_headers: list[str] = ["*"]
|
||||
|
||||
# LLM - Ollama (always hot in VRAM on tower-of-joy)
|
||||
ollama_url: str = "http://192.168.86.149:11434"
|
||||
ollama_agent_model: str = "mistral-nemo-large:latest"
|
||||
ollama_embed_model: str = "nomic-embed-text:latest"
|
||||
|
||||
# Auth - Tatlock integration
|
||||
tatlock_api_url: Optional[str] = "http://192.168.86.149:8000"
|
||||
internal_api_key: Optional[str] = None
|
||||
|
||||
# Tool execution
|
||||
tool_timeout_seconds: int = 120
|
||||
sandbox_enabled: bool = True
|
||||
allowed_paths: list[str] = []
|
||||
|
||||
# Sessions
|
||||
session_ttl_hours: int = 24
|
||||
max_context_tokens: int = 128000
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_file=".env",
|
||||
case_sensitive=False,
|
||||
extra="ignore",
|
||||
)
|
||||
|
||||
|
||||
@lru_cache()
|
||||
def get_settings() -> Settings:
|
||||
"""Cached settings singleton."""
|
||||
return Settings()
|
||||
@@ -0,0 +1,86 @@
|
||||
"""
|
||||
Request context management.
|
||||
|
||||
Provides UserProvider singleton for request-scoped user context.
|
||||
Set once per request in middleware, accessible everywhere without
|
||||
passing user through function parameters.
|
||||
"""
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
from contextvars import ContextVar
|
||||
|
||||
|
||||
@dataclass
|
||||
class User:
|
||||
"""Authenticated user context."""
|
||||
id: str
|
||||
email: str
|
||||
api_key: str
|
||||
tenant_id: Optional[str] = None
|
||||
roles: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
_current_user: ContextVar[Optional[User]] = ContextVar('current_user', default=None)
|
||||
|
||||
|
||||
class UserProvider:
|
||||
"""
|
||||
Singleton for request-scoped user context.
|
||||
|
||||
Set once per request in middleware, accessible everywhere without
|
||||
passing user through function parameters.
|
||||
|
||||
Usage in middleware:
|
||||
user = await validate_api_key(request)
|
||||
user_provider.set_user(user)
|
||||
try:
|
||||
response = await call_next(request)
|
||||
finally:
|
||||
user_provider.clear_user()
|
||||
|
||||
Usage anywhere:
|
||||
from src.shared.context import get_current_user, require_user
|
||||
|
||||
user = get_current_user() # Returns None if not authenticated
|
||||
user = require_user() # Raises if not authenticated
|
||||
"""
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
return cls._instance
|
||||
|
||||
def set_user(self, user: User) -> None:
|
||||
"""Set current user for this request context."""
|
||||
_current_user.set(user)
|
||||
|
||||
def get_user(self) -> Optional[User]:
|
||||
"""Get current user (may be None)."""
|
||||
return _current_user.get()
|
||||
|
||||
def clear_user(self) -> None:
|
||||
"""Clear current user (call in finally block)."""
|
||||
_current_user.set(None)
|
||||
|
||||
@property
|
||||
def current_user(self) -> Optional[User]:
|
||||
"""Property access to current user."""
|
||||
return self.get_user()
|
||||
|
||||
|
||||
# Global singleton
|
||||
user_provider = UserProvider()
|
||||
|
||||
|
||||
def get_current_user() -> Optional[User]:
|
||||
"""Get current user or None."""
|
||||
return user_provider.get_user()
|
||||
|
||||
|
||||
def require_user() -> User:
|
||||
"""Get current user or raise ValueError."""
|
||||
user = user_provider.get_user()
|
||||
if user is None:
|
||||
raise ValueError("No authenticated user in context")
|
||||
return user
|
||||
@@ -0,0 +1,107 @@
|
||||
"""
|
||||
Custom exception hierarchy.
|
||||
|
||||
All application exceptions inherit from AppException.
|
||||
"""
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
class AppException(Exception):
|
||||
"""Base exception for application errors."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
status_code: int = 500,
|
||||
error_code: Optional[str] = None,
|
||||
details: Optional[dict[str, Any]] = None,
|
||||
):
|
||||
self.message = message
|
||||
self.status_code = status_code
|
||||
self.error_code = error_code or self.__class__.__name__
|
||||
self.details = details or {}
|
||||
super().__init__(message)
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
"""Convert to JSON-serializable dict."""
|
||||
return {
|
||||
"error": self.error_code,
|
||||
"message": self.message,
|
||||
"details": self.details,
|
||||
}
|
||||
|
||||
|
||||
class NotFoundError(AppException):
|
||||
"""Resource not found."""
|
||||
|
||||
def __init__(self, resource: str, identifier: Any):
|
||||
super().__init__(
|
||||
message=f"{resource} not found: {identifier}",
|
||||
status_code=404,
|
||||
details={"resource": resource, "identifier": str(identifier)},
|
||||
)
|
||||
|
||||
|
||||
class ValidationError(AppException):
|
||||
"""Input validation failed."""
|
||||
|
||||
def __init__(self, message: str, field: Optional[str] = None):
|
||||
super().__init__(
|
||||
message=message,
|
||||
status_code=422,
|
||||
details={"field": field} if field else {},
|
||||
)
|
||||
|
||||
|
||||
class AuthenticationError(AppException):
|
||||
"""Authentication required or failed."""
|
||||
|
||||
def __init__(self, message: str = "Authentication required"):
|
||||
super().__init__(message=message, status_code=401)
|
||||
|
||||
|
||||
class AuthorizationError(AppException):
|
||||
"""Permission denied."""
|
||||
|
||||
def __init__(self, message: str = "Permission denied"):
|
||||
super().__init__(message=message, status_code=403)
|
||||
|
||||
|
||||
class ConflictError(AppException):
|
||||
"""Resource conflict (e.g., duplicate)."""
|
||||
|
||||
def __init__(self, message: str):
|
||||
super().__init__(message=message, status_code=409)
|
||||
|
||||
|
||||
class RateLimitError(AppException):
|
||||
"""Rate limit exceeded."""
|
||||
|
||||
def __init__(self, retry_after: int = 60):
|
||||
super().__init__(
|
||||
message="Rate limit exceeded",
|
||||
status_code=429,
|
||||
details={"retry_after": retry_after},
|
||||
)
|
||||
|
||||
|
||||
class ServiceUnavailableError(AppException):
|
||||
"""External service unavailable."""
|
||||
|
||||
def __init__(self, service: str, message: Optional[str] = None):
|
||||
super().__init__(
|
||||
message=message or f"Service unavailable: {service}",
|
||||
status_code=503,
|
||||
details={"service": service},
|
||||
)
|
||||
|
||||
|
||||
class TimeoutError(AppException):
|
||||
"""Operation timed out."""
|
||||
|
||||
def __init__(self, operation: str, timeout_seconds: float):
|
||||
super().__init__(
|
||||
message=f"Operation timed out: {operation}",
|
||||
status_code=504,
|
||||
details={"operation": operation, "timeout_seconds": timeout_seconds},
|
||||
)
|
||||
@@ -0,0 +1,239 @@
|
||||
"""
|
||||
Centralized logging with temporal benchmarking.
|
||||
|
||||
Provides:
|
||||
- @logged() decorator for automatic function timing
|
||||
- trace_span() context manager for manual instrumentation
|
||||
- Trace ID correlation across nested calls
|
||||
- Configurable slow/warn thresholds
|
||||
"""
|
||||
import functools
|
||||
import asyncio
|
||||
import time
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
from contextvars import ContextVar
|
||||
from dataclasses import dataclass, field
|
||||
from uuid import uuid4
|
||||
|
||||
|
||||
# === Trace Context ===
|
||||
|
||||
@dataclass
|
||||
class TraceSpan:
|
||||
"""Represents a timed execution span."""
|
||||
name: str
|
||||
trace_id: str
|
||||
parent_id: Optional[str] = None
|
||||
span_id: str = field(default_factory=lambda: uuid4().hex[:8])
|
||||
start_time: float = field(default_factory=time.perf_counter)
|
||||
end_time: Optional[float] = None
|
||||
|
||||
@property
|
||||
def duration_ms(self) -> float:
|
||||
"""Get duration in milliseconds."""
|
||||
end = self.end_time or time.perf_counter()
|
||||
return (end - self.start_time) * 1000
|
||||
|
||||
|
||||
_current_span: ContextVar[Optional[TraceSpan]] = ContextVar('current_span', default=None)
|
||||
_trace_id: ContextVar[Optional[str]] = ContextVar('trace_id', default=None)
|
||||
|
||||
|
||||
def get_current_trace_id() -> Optional[str]:
|
||||
"""Get current trace ID for log correlation."""
|
||||
return _trace_id.get()
|
||||
|
||||
|
||||
def get_current_span() -> Optional[TraceSpan]:
|
||||
"""Get current trace span."""
|
||||
return _current_span.get()
|
||||
|
||||
|
||||
# === Setup ===
|
||||
|
||||
def setup_logging(log_level: str = "INFO") -> None:
|
||||
"""Configure application logging."""
|
||||
log_dir = Path("logs")
|
||||
log_dir.mkdir(exist_ok=True)
|
||||
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, log_level.upper()),
|
||||
format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
handlers=[
|
||||
logging.StreamHandler(sys.stdout),
|
||||
logging.FileHandler(log_dir / "webber.log", encoding="utf-8")
|
||||
]
|
||||
)
|
||||
|
||||
# Quiet noisy libraries
|
||||
for name in ["httpx", "httpcore", "uvicorn.access", "uvicorn.error"]:
|
||||
logging.getLogger(name).setLevel(logging.WARNING)
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""Get a logger instance."""
|
||||
return logging.getLogger(name)
|
||||
|
||||
|
||||
# === Decorator ===
|
||||
|
||||
def logged(
|
||||
logger: logging.Logger = None,
|
||||
slow_threshold_ms: float = 100.0,
|
||||
warn_threshold_ms: float = 500.0,
|
||||
include_args: bool = False,
|
||||
):
|
||||
"""
|
||||
Decorator for automatic function logging with temporal benchmarking.
|
||||
|
||||
Args:
|
||||
logger: Logger instance (defaults to module logger)
|
||||
slow_threshold_ms: Log INFO if execution exceeds this (default 100ms)
|
||||
warn_threshold_ms: Log WARNING if execution exceeds this (default 500ms)
|
||||
include_args: Include function arguments in log (careful with sensitive data)
|
||||
|
||||
Usage:
|
||||
@logged()
|
||||
async def my_function(): ...
|
||||
|
||||
@logged(slow_threshold_ms=50, warn_threshold_ms=200)
|
||||
def critical_path(): ...
|
||||
"""
|
||||
def decorator(func: Callable):
|
||||
nonlocal logger
|
||||
if logger is None:
|
||||
logger = logging.getLogger(func.__module__)
|
||||
|
||||
func_name = f"{func.__module__}.{func.__qualname__}"
|
||||
|
||||
def _create_span() -> TraceSpan:
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
return TraceSpan(
|
||||
name=func_name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
|
||||
def _log_completion(span: TraceSpan, error: Exception = None):
|
||||
span.end_time = time.perf_counter()
|
||||
duration = span.duration_ms
|
||||
tid = span.trace_id[:8]
|
||||
|
||||
if error:
|
||||
logger.error(
|
||||
f"[{tid}] {func_name} FAILED after {duration:.2f}ms: {error}",
|
||||
exc_info=True
|
||||
)
|
||||
elif duration >= warn_threshold_ms:
|
||||
logger.warning(
|
||||
f"[{tid}] {func_name} SLOW: {duration:.2f}ms (threshold: {warn_threshold_ms}ms)"
|
||||
)
|
||||
elif duration >= slow_threshold_ms:
|
||||
logger.info(f"[{tid}] {func_name} completed in {duration:.2f}ms")
|
||||
else:
|
||||
logger.debug(f"[{tid}] {func_name} completed in {duration:.2f}ms")
|
||||
|
||||
@functools.wraps(func)
|
||||
async def async_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
|
||||
if include_args:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}({args}, {kwargs})")
|
||||
else:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
|
||||
try:
|
||||
result = await func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
@functools.wraps(func)
|
||||
def sync_wrapper(*args, **kwargs):
|
||||
span = _create_span()
|
||||
token = _current_span.set(span)
|
||||
|
||||
if include_args:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}({args}, {kwargs})")
|
||||
else:
|
||||
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
|
||||
|
||||
try:
|
||||
result = func(*args, **kwargs)
|
||||
_log_completion(span)
|
||||
return result
|
||||
except Exception as e:
|
||||
_log_completion(span, error=e)
|
||||
raise
|
||||
finally:
|
||||
_current_span.reset(token)
|
||||
|
||||
return async_wrapper if asyncio.iscoroutinefunction(func) else sync_wrapper
|
||||
return decorator
|
||||
|
||||
|
||||
# === Context Manager ===
|
||||
|
||||
class trace_span:
|
||||
"""
|
||||
Context manager for manual span creation.
|
||||
|
||||
Usage:
|
||||
with trace_span("database_query"):
|
||||
result = db.execute(query)
|
||||
|
||||
async with trace_span("llm_call"):
|
||||
response = await agent.run(prompt)
|
||||
"""
|
||||
|
||||
def __init__(self, name: str, logger: logging.Logger = None):
|
||||
self.name = name
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
self.span: Optional[TraceSpan] = None
|
||||
self.token = None
|
||||
|
||||
def __enter__(self) -> TraceSpan:
|
||||
parent = _current_span.get()
|
||||
trace_id = _trace_id.get() or uuid4().hex[:16]
|
||||
if _trace_id.get() is None:
|
||||
_trace_id.set(trace_id)
|
||||
|
||||
self.span = TraceSpan(
|
||||
name=self.name,
|
||||
trace_id=trace_id,
|
||||
parent_id=parent.span_id if parent else None,
|
||||
)
|
||||
self.token = _current_span.set(self.span)
|
||||
self.logger.debug(f"[{self.span.trace_id[:8]}] -> {self.name}")
|
||||
return self.span
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
if self.span:
|
||||
self.span.end_time = time.perf_counter()
|
||||
duration = self.span.duration_ms
|
||||
tid = self.span.trace_id[:8]
|
||||
if exc_val:
|
||||
self.logger.error(f"[{tid}] {self.name} FAILED: {duration:.2f}ms")
|
||||
else:
|
||||
self.logger.debug(f"[{tid}] {self.name}: {duration:.2f}ms")
|
||||
if self.token:
|
||||
_current_span.reset(self.token)
|
||||
return False
|
||||
|
||||
async def __aenter__(self) -> TraceSpan:
|
||||
return self.__enter__()
|
||||
|
||||
async def __aexit__(self, exc_type, exc_val, exc_tb):
|
||||
return self.__exit__(exc_type, exc_val, exc_tb)
|
||||
@@ -0,0 +1,34 @@
|
||||
"""
|
||||
Pytest configuration and fixtures.
|
||||
"""
|
||||
import pytest
|
||||
from httpx import AsyncClient, ASGITransport
|
||||
|
||||
from src.main import app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anyio_backend():
|
||||
"""Use asyncio for async tests."""
|
||||
return "asyncio"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def client():
|
||||
"""Async HTTP client for testing."""
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test"
|
||||
) as ac:
|
||||
yield ac
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def auth_client():
|
||||
"""Async HTTP client with API key for authenticated requests."""
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"X-API-Key": "test-api-key"}
|
||||
) as ac:
|
||||
yield ac
|
||||
@@ -0,0 +1,29 @@
|
||||
"""
|
||||
Health endpoint tests.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_root(client):
|
||||
"""Test root endpoint returns service info."""
|
||||
response = await client.get("/")
|
||||
assert response.status_code == 200
|
||||
|
||||
data = response.json()
|
||||
assert "Webber" in data["service"]
|
||||
assert data["status"] == "healthy"
|
||||
assert "version" in data
|
||||
assert data["docs"] == "/docs"
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_health_check(client):
|
||||
"""Test health check endpoint."""
|
||||
response = await client.get("/health")
|
||||
assert response.status_code == 200
|
||||
|
||||
data = response.json()
|
||||
assert data["status"] == "healthy"
|
||||
assert "Webber" in data["service"]
|
||||
assert "version" in data
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/bin/bash
|
||||
# Webber Server Startup Script
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
RED='\033[0;31m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
echo -e "${GREEN}Starting Webber...${NC}"
|
||||
|
||||
# Check if port 8086 is already in use
|
||||
if lsof -Pi :8086 -sTCP:LISTEN -t >/dev/null 2>&1 ; then
|
||||
echo -e "${RED}Error: Port 8086 is already in use${NC}"
|
||||
echo "Run: lsof -i :8086 to see what's using it"
|
||||
echo "Or run: kill \$(lsof -t -i:8086) to stop it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Activate virtual environment if not already activated
|
||||
if [ -z "$VIRTUAL_ENV" ]; then
|
||||
if [ -d ".venv" ]; then
|
||||
echo -e "${YELLOW}Activating virtual environment...${NC}"
|
||||
source .venv/bin/activate
|
||||
else
|
||||
echo -e "${RED}Error: Virtual environment not found${NC}"
|
||||
echo "Run: python -m venv .venv && source .venv/bin/activate && pip install -r requirements-dev.txt"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create logs directory if it doesn't exist
|
||||
LOGS_DIR="logs"
|
||||
mkdir -p "$LOGS_DIR"
|
||||
|
||||
# Clear/create log file
|
||||
LOG_FILE="$LOGS_DIR/server.log"
|
||||
> "$LOG_FILE"
|
||||
echo -e "${YELLOW}Logs will be written to: ${LOG_FILE}${NC}"
|
||||
|
||||
# Start the server
|
||||
echo -e "${GREEN}Starting uvicorn server on http://localhost:8086${NC}"
|
||||
echo -e "${YELLOW}Press Ctrl+C to stop the server${NC}"
|
||||
echo ""
|
||||
|
||||
uvicorn src.main:app --reload --host 0.0.0.0 --port 8086 2>&1 | tee "$LOG_FILE"
|
||||
Reference in New Issue
Block a user