diff --git a/AGENTS.md b/AGENTS.md index c9bf1f9..abdfa78 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -3,542 +3,48 @@ This document contains instructions and documentation references for AI assistants working with this codebase. > **📖 Important**: Before working on this project, read [PHILOSOPHY.md](PHILOSOPHY.md) to understand the system vision, architectural patterns, and design goals. All development should work towards realizing those patterns. +# AGENTS.md -## Project Overview +> **Start every session by reading this file.** +> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project. -This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer. +## 1. Agent Operational Protocols -### Architecture Pattern +### 🧠 Work Patterns (Plan-Act-Reflect) +* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be. +* **Act:** Execute the changes in small, atomic steps. +* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests? -The **Orchestrator** infrastructure layer with hybrid API architecture: +### 🛡️ Git Discipline +* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`. +* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format. + * `feat: add user login endpoint` + * `fix: resolve database connection timeout` + * `refactor: split monolith dependency file` +* **Atomic Commits:** Keep commits small. One logical change = one commit. -``` -Client (Open WebUI) - ↓ -Chat Completions (/v1/chat/completions) → Wrapper - ↓ -Responses API (/v1/responses) → Primary - ↓ -Agent Interface (lorem-tester, Tatlock) - ↓ -Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.) -``` +### 📝 Changelog Maintenance +* **Update `CHANGELOG.md`** with every user-facing change. +* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`. -**Architectural Layers:** +--- -1. **The Orchestrator** (Current Implementation) - - FastAPI application providing the infrastructure - - HTTP/SSE endpoints, streaming coordination - - Conversation history and context management - - OpenAI-compatible API surface +## 2. FastAPI Architecture & Best Practices +*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)* -2. **Future: The Household** (Phases 1-4) - - **Steward**: First-tier LLM for request analysis (PydanticAI agent) - - **Tatlock**: Second-tier LLM with butler personality (PydanticAI agent) - - **Expert Agents**: Domain specialists (Librarian, Developer, Handyman, etc.) +### 📂 Project Structure (Directory-based, NOT File-type based) +Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory. -**Key Architectural Decisions:** - -1. **Single Source of Truth**: All response generation happens in the Responses API - - Structured output with reasoning, function_call, and message items - - Real-time stop sequence and max tokens enforcement - - Conversation history tracking - - Context window management - -2. **Chat Completions Wrapper**: Provides compatibility without duplicating logic - - Calls Responses API internally - - Automatically enables reasoning generation - - Converts reasoning items to `` tags for Open WebUI - - Maintains OpenAI-compatible format - -3. **Agent Interface**: Clean abstraction for multiple models - - **lorem-tester**: Full-featured mock agent with realistic behavior - - Reasoning summaries (adjustable effort levels) - - Random tool/function calls - - Error triggers for testing - - Temperature variation - - **Tatlock**: Advertised model name (currently mock, future: PydanticAI Butler agent) - -4. **Hybrid Conversation History**: - - Client MUST send full context in `input` array (OpenAI compatible) - - Server optionally tracks via `metadata.conversation_id` - - Auto-generates deterministic IDs from first message - - Supports future vector memory integration (Qdrant) - -**Why This Architecture?** - -- **Open WebUI Compatibility**: Native Responses API support not yet in stable release -- **Future-Proof**: Easy migration when Open WebUI adds native support -- **Testability**: Full-featured mock agent (lorem-tester) for integration testing -- **Clean Separation**: Responses API as stable core, wrappers can change - -### Components - -- **FastAPI**: Web framework for the API layer -- **SSE-Starlette**: Server-Sent Events for streaming responses -- **Pydantic**: Request/response validation with field validators -- **Agent Interface**: Abstract base class for model implementations -- **Conversation History**: Server-side tracking with configurable max turns -- **Context Window**: Token counting and management -- **PydanticAI**: Integrated with Tatlock agent (Ollama backend) -- **Agent Tools**: Permanent tools module (`src/agents/tools.py`) - - Calculator: Safe mathematical expression evaluation - - Date/Time toolkit: Current time, relative dates, time differences - - Web Search: SearXNG integration for privacy-preserving search - -## Documentation References - -### Core Framework Documentation - -#### FastAPI -- **Official Documentation**: https://fastapi.tiangolo.com/ -- **Version**: 0.123.9 (Dec 2025) -- **Key Topics**: - - Path operations and routing - - Request/response models with Pydantic - - Dependency injection - - Background tasks - - WebSocket and streaming support -- **PyPI**: https://pypi.org/project/fastapi/ - -#### Uvicorn -- **Official Documentation**: https://www.uvicorn.org/ -- **Version**: 0.38.0 (Oct 2025) -- **Key Topics**: - - ASGI server configuration - - Deployment settings - - Logging and monitoring - - SSL/TLS configuration - -### AI/LLM Integration - -#### PydanticAI -- **Official Documentation**: https://ai.pydantic.dev/ -- **Version**: 1.27.0 (Dec 2025) -- **Status**: Dependency installed, ready for future integration -- **Key Topics** (for future implementation): - - Agent creation and configuration - - LLM provider integration (Ollama support) - - Structured outputs with Pydantic - - Streaming responses - - Tool/function calling - - RunContext and dynamic configuration - - MCP server integration -- **GitHub**: https://github.com/pydantic/pydantic-ai -- **PyPI**: https://pypi.org/project/pydantic-ai/ - -#### Pydantic -- **Official Documentation**: https://docs.pydantic.dev/latest/ -- **Version**: 2.11+ (Required for PydanticAI, currently using >=2.11,<2.13) -- **Key Topics**: - - Data validation and serialization - - Field types and validators - - Model configuration - - JSON schema generation - -### HTTP and Streaming - -#### HTTPX -- **Official Documentation**: https://www.python-httpx.org/ -- **Version**: 0.28.1 -- **Key Topics**: - - Async HTTP client for Ollama communication - - Streaming responses - - Timeout configuration - - Connection pooling - -#### SSE-Starlette -- **GitHub**: https://github.com/sysid/sse-starlette -- **Version**: 3.0.2 (Oct 2025) -- **Key Topics**: - - Server-Sent Events implementation - - Streaming event responses - - Integration with FastAPI/Starlette - -### Ollama Integration - -#### Ollama API -- **Official Documentation**: https://github.com/ollama/ollama/blob/main/docs/api.md -- **Status**: Async client implemented in `src/ollama/client.py`, ready for future integration -- **Key Topics** (for future implementation): - - REST API endpoints - - Streaming responses - - Model management - - Generate and chat endpoints - - Model configuration -- **Current Model Target**: mistral-nemo:latest - -### OpenAI API Compatibility - -#### OpenAI API Reference -- **Official Documentation**: https://platform.openai.com/docs/api-reference -- **Key API Endpoints**: - - `/v1/responses` - Responses API (PRIMARY) with structured output - - `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper - - `/v1/models` - List available models - -- **Key Features for Development**: - - **Responses API Format**: Structured output with reasoning, function_call, and message items - - **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences - - **Conversation History**: Hybrid client/server approach with auto-generated IDs - - **Context Management**: Token counting and window trimming - - **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement - - **Error Handling**: Custom exception types (RateLimitError, ContextLengthError) - - **Tool Calling**: PydanticAI tool integration with permanent tools - - **Testing**: Comprehensive test suite with mocks and real Ollama integration - -## FastAPI Best Practices - -This project follows best practices from [github.com/zhanymkanov/fastapi-best-practices](https://github.com/zhanymkanov/fastapi-best-practices) - -### Project Structure - -**Domain-Based Organization**: Code is organized by domain/feature rather than by file type: - -``` +**Correct Structure:** +```text src/ -├── agents/ # Agent interface and implementations -│ ├── base.py # Abstract AgentInterface -│ ├── lorem_tester.py # Full-featured mock agent -│ ├── tatlock.py # Placeholder for real agent -│ └── registry.py # ModelRegistry for agent management -├── responses/ # Responses API domain (PRIMARY) -│ ├── router.py # POST /v1/responses endpoint -│ ├── schemas.py # Request/response models with validators -│ ├── service.py # Response generation logic -│ ├── streaming.py # SSE streaming coordinator -│ ├── history.py # Conversation history management -│ └── context.py # Context window and token management -├── chat/ # Chat Completions domain (WRAPPER) -│ ├── router.py # POST /v1/chat/completions endpoint -│ ├── schemas.py # Chat request/response models -│ ├── service.py # Wraps Responses API, converts to tags -│ ├── constants.py # Chat constants (roles, finish reasons) -│ └── __init__.py -├── models/ # Models listing domain -│ ├── router.py # GET /v1/models endpoint -│ ├── schemas.py # Model schemas -│ ├── service.py # Accesses ModelRegistry -│ └── __init__.py -├── core/ # Shared utilities -│ ├── config.py # Global configuration (BaseSettings) -│ ├── models.py # Custom base Pydantic models -│ ├── exceptions.py # Custom exceptions (RateLimitError, etc.) -│ ├── dependencies.py # Shared dependencies -│ └── router.py # Core routes (health, root) -├── ollama/ # Ollama client layer (not yet integrated) -│ ├── client.py # Async Ollama HTTP client -│ └── schemas.py # Ollama API models -└── main.py # Application factory & configuration -``` - -**Key Architectural Principles**: -- **Single Source of Truth**: Responses API handles all generation logic -- **Wrapper Pattern**: Chat Completions wraps Responses API without duplicating code -- **Agent Abstraction**: AgentInterface defines contract for all models -- **Domain Separation**: Each domain has its own router, schemas, service -- **Service Layer**: Business logic in services, not routers -- **Type Safety**: Pydantic models for ALL request/response validation -- **Async First**: All I/O operations use async/await - -### Async/Await Best Practices - -**Critical Understanding**: FastAPI handles sync and async routes differently: - -- **Async routes** (`async def`): Called directly in event loop - - Use ONLY for non-blocking operations - - Perfect for `await httpx.get()`, database queries, file I/O - - **NEVER** use blocking calls like `time.sleep()` - this blocks entire server - -- **Sync routes** (`def`): Run in thread pool - - Use for CPU-intensive work or blocking SDKs - - Blocking I/O won't freeze the event loop - - Example: `time.sleep(10)` is safe here - -**Example**: -```python -@router.get("/terrible") -async def terrible(): - time.sleep(10) # ❌ BLOCKS ENTIRE SERVER - -@router.get("/good") -def good(): - time.sleep(10) # ✅ Runs in thread pool - -@router.get("/perfect") -async def perfect(): - await asyncio.sleep(10) # ✅ Non-blocking async -``` - -**For CPU-intensive tasks**: Use separate worker processes (not threads) due to Python's GIL. - -### Pydantic Configuration - -**Custom Base Model**: All schemas inherit from `CustomBaseModel` for consistent behavior: - -```python -# src/core/models.py -class CustomBaseModel(BaseModel): - model_config = ConfigDict( - json_encoders={datetime: datetime_to_iso_str}, - populate_by_name=True, - use_enum_values=True, - validate_assignment=True, - ) - - def serializable_dict(self, **kwargs): - """Return dict with only JSON-serializable fields.""" - return jsonable_encoder(self.model_dump(**kwargs)) -``` - -**Benefits**: -- Consistent datetime serialization across all responses -- Alias support for field name flexibility -- Easy JSON encoding for logging/debugging - -**Decoupled Settings**: Split configuration by domain instead of one monolithic file: - -```python -# src/core/config.py - Global settings -class Config(BaseSettings): - DATABASE_URL: PostgresDsn - ENVIRONMENT: Environment - -# src/chat/config.py - Chat-specific settings -class ChatConfig(BaseSettings): - MAX_TOKENS: int - DEFAULT_TEMPERATURE: float -``` - -### Dependency Injection Patterns - -**Validation with Dependencies**: Use dependencies for complex validations: - -```python -async def valid_post_id(post_id: UUID4) -> dict: - """Validate post exists in database.""" - post = await service.get_by_id(post_id) - if not post: - raise PostNotFound() - return post - -@router.get("/posts/{post_id}") -async def get_post(post: dict = Depends(valid_post_id)): - return post # Already validated! -``` - -**Chaining Dependencies**: Build reusable validation layers: - -```python -async def valid_owned_post( - post: dict = Depends(valid_post_id), - token_data: dict = Depends(parse_jwt_data), -) -> dict: - if post["creator_id"] != token_data["user_id"]: - raise UserNotOwner() - return post -``` - -**Dependency Caching**: Dependencies are cached within request scope - FastAPI only executes each dependency once per request, even if used multiple times. - -### Application Factory Pattern - -Main.py uses factory pattern for testability and configuration: - -```python -def create_application() -> FastAPI: - """Create and configure FastAPI app.""" - app = FastAPI(title=config.APP_NAME) - - # Add middleware - app.add_middleware(CORSMiddleware, ...) - - # Register exception handlers - register_exception_handlers(app) - - # Include routers - app.include_router(chat_router, prefix="/v1") - - return app - -app = create_application() -``` - -## Development Guidelines - -### Git Workflow - -**IMPORTANT**: Do NOT handle git commits or pushes automatically. Wait for explicit user instruction before: -- Running `git add` -- Running `git commit` -- Running `git push` -- Creating or pushing tags - -The user will manage git operations themselves unless they specifically request assistance. - -### Server Logs and Debugging - -**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is: -- Cleared on each server startup (fresh logs every time) -- Written in real-time as the server runs -- Already gitignored (won't be committed) - -**Accessing Logs**: You can read the log file at any time while the server is running: -```bash -# View current logs -cat logs/server.log - -# Follow logs in real-time -tail -f logs/server.log - -# Search logs -grep "ERROR" logs/server.log -``` - -This is useful for debugging issues, monitoring API calls, and understanding server behavior during development. - -### Code Structure Guidelines -- Use async/await for ALL I/O operations (database, HTTP, file access) -- Use sync (def) for blocking SDKs or CPU-intensive work -- Implement proper error handling and logging -- Follow dependency injection for validation and shared resources -- Use Pydantic models for ALL request/response validation -- Keep business logic in service modules, not routers -- Domain-based project structure (not file-type based) - -### Security Considerations -- Validate all inputs using Pydantic models -- Use environment variables for sensitive configuration -- Keep dependencies updated and CVE-checked -- Minor version locking for supply chain protection -- Consider rate limiting for production deployment -- Plan for authentication/API keys when needed - -### Testing Approach -- Write integration tests for API endpoints -- Test streaming functionality with appropriate timeouts -- Use pytest-asyncio for async test support -- Validate OpenAI API compatibility in tests -- Test both mock and real LLM integrations -- Cover main application (CORS, exception handlers, lifespan) -- Test wrapper layers (chat completions, etc.) -- Include tool functionality tests - -### Configuration Management -- Use `.env` files for local development -- Document all environment variables in README -- Provide sensible defaults where possible -- Use BaseSettings from pydantic-settings -- Support both local and container-based configuration - -## Common Patterns - -### Streaming Response Pattern - -Example from `src/chat/router.py`: - -```python -from sse_starlette.sse import EventSourceResponse -from fastapi import FastAPI - -async def event_generator(): - # Currently yields mock lorem ipsum chunks - # Future: Stream from Ollama/PydanticAI - yield {"data": chunk.model_dump_json()} - yield {"data": "[DONE]"} - -@app.post("/stream") -async def stream(): - return EventSourceResponse(event_generator()) -``` - -### PydanticAI Agent Pattern - -When implementing agents with PydanticAI and Ollama: - -```python -from pydantic_ai import Agent - -agent = Agent( - 'ollama:mistral-nemo', # Target model - # Configuration here -) - -# Use the agent -result = await agent.run('Your prompt') -``` - -### OpenAI-Compatible Response Format - -Example schema from `src/chat/schemas.py`: - -```python -{ - "id": "chatcmpl-123", - "object": "chat.completion.chunk", - "created": 1234567890, - "model": "mistral-nemo:latest", - "choices": [{ - "index": 0, - "delta": {"content": "response"}, - "finish_reason": None - }] -} -``` - -### PydanticAI Tool Registration Pattern - -Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples: - -```python -from pydantic_ai import Agent, RunContext - -# After creating the agent -@agent.tool -def tool_name(ctx: RunContext[None], param: str) -> str: - """ - Tool description that the LLM sees. - - Args: - param: Parameter description - - Returns: - Result description - """ - return result -``` - -**Tool Implementation Guidelines**: -- Keep tools in `src/agents/tools.py` for reusability -- Use clear, descriptive docstrings (LLM reads these) -- Include parameter descriptions in docstrings -- Handle errors gracefully and return error messages as strings -- For async operations, declare the tool function as `async def` -- Test tools independently before integration - -**Example Tool Module** (`src/agents/tools.py`): -```python -def calculate(expression: str) -> str: - """Safe calculator implementation.""" - try: - # Implementation - return str(result) - except Exception as e: - return f"Error: {str(e)}" - -async def search_web(query: str) -> str: - """Web search via SearXNG.""" - async with httpx.AsyncClient() as client: - # Implementation - return formatted_results -``` - -## Update Policy - -This document should be updated when: -- New development patterns are established -- Package versions are upgraded -- Major architectural changes occur -- New best practices are identified - -Last updated: 2025-12-06 (Tools integration) +├── auth/ +│ ├── router.py # Endpoints +│ ├── schemas.py # Pydantic models +│ ├── service.py # Business logic (CRUD, etc.) +│ ├── dependencies.py# Module-specific dependencies +│ └── config.py # Module-specific settings +├── posts/ +│ ├── router.py +│ └── ... +└── main.py # App entry point \ No newline at end of file