From fd459e9ffbdec1dd0552a5103b39f8ca8a47f484 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Sun, 7 Dec 2025 00:13:41 +0100 Subject: [PATCH] docs: update documentation for v0.2.0 tools release README.md: - Add Tatlock agent capabilities and tool descriptions - Add requirements section (Ollama, SearXNG setup) - Add configuration examples for external services - Add tool usage examples and philosophy - Add troubleshooting for Ollama and SearXNG - Update test statistics AGENTS.md: - Refactor for LLM development focus - Add PydanticAI tool registration pattern - Add tool implementation guidelines - Remove project status, focus on development instructions IMPLEMENTATION_ROADMAP.md: - Mark Phase 1 as "MOSTLY COMPLETE" - Update detailed completion status - Update current state summary --- AGENTS.md | 227 ++++++++++++++++++++++---------------- IMPLEMENTATION_ROADMAP.md | 87 +++++++++------ README.md | 103 +++++++++++++++-- 3 files changed, 272 insertions(+), 145 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ec4801b..c9bf1f9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,14 +6,11 @@ This document contains instructions and documentation references for AI assistan ## Project Overview -This project implements an OpenAI-compatible API endpoint using FastAPI, with streaming support. Currently returns mock responses - infrastructure prepared for future Ollama/PydanticAI integration. +This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer. -**Current State**: Production-ready testing API with Responses API and Open WebUI integration -**Future Integration**: PydanticAI for real LLM agents (tatlock model placeholder ready) +### Architecture Pattern -### Current Architecture (As of 2025-12-06) - -This project implements the **Orchestrator** infrastructure layer with a hybrid API architecture: +The **Orchestrator** infrastructure layer with hybrid API architecture: ``` Client (Open WebUI) @@ -83,7 +80,11 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.) - **Agent Interface**: Abstract base class for model implementations - **Conversation History**: Server-side tracking with configurable max turns - **Context Window**: Token counting and management -- **PydanticAI**: Dependency installed, ready for tatlock agent implementation +- **PydanticAI**: Integrated with Tatlock agent (Ollama backend) +- **Agent Tools**: Permanent tools module (`src/agents/tools.py`) + - Calculator: Safe mathematical expression evaluation + - Date/Time toolkit: Current time, relative dates, time differences + - Web Search: SearXNG integration for privacy-preserving search ## Documentation References @@ -171,58 +172,20 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.) #### OpenAI API Reference - **Official Documentation**: https://platform.openai.com/docs/api-reference -- **Implemented Endpoints**: - - ✅ `/v1/responses` - **Responses API (PRIMARY)** with structured output - - Reasoning items (thinking summaries) - - Function call items (tool execution) - - Message items (assistant responses) - - Full streaming support with SSE - - Stop sequence detection - - Max tokens enforcement - - Conversation history tracking - - ✅ `/v1/chat/completions` - **Compatibility wrapper** around Responses API - - Converts reasoning to `` tags for Open WebUI - - Automatically enables reasoning generation - - Maintains OpenAI-compatible format - - Supports streaming and non-streaming - - ✅ `/v1/models` - List available models (lorem-tester, tatlock) -- **Future Endpoints**: - - 🚧 `/v1/completions` - Text completion (legacy) - - 🚧 `/v1/embeddings` - Text embeddings -- **Implemented Features**: - - ✅ **Responses API Format**: - - Structured output items (reasoning, function_call, message) - - Extended thinking support - - Tool/function calling support - - Streaming with multiple event types - - ✅ **Advanced Parameter Validation**: - - Temperature: 0.0-2.0 with Pydantic validators - - Reasoning effort: none, minimal, low, medium, high, xhigh - - Max output tokens: positive integer enforcement - - Stop sequences: up to 4, non-empty strings - - ✅ **Conversation History**: - - Hybrid client/server approach - - Auto-generated conversation IDs - - Configurable max turns (default: 20) - - Placeholder for vector memory - - ✅ **Context Management**: - - Approximate token counting (~4 chars/token) - - Context window trimming - - Usage statistics - - ✅ **Streaming Enforcement**: - - Real-time stop sequence detection - - Real-time max tokens enforcement - - Word-by-word streaming with delays - - ✅ **Error Handling**: - - Custom exception types (RateLimitError, ContextLengthError) - - OpenAI-compatible error format - - Error triggers in lorem-tester for testing - - ✅ **Testing Infrastructure**: - - 95 tests (78.95% coverage) - - Unit tests for all components - - Integration tests for API endpoints - - Streaming tests for SSE functionality - - Main application and wrapper tests +- **Key API Endpoints**: + - `/v1/responses` - Responses API (PRIMARY) with structured output + - `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper + - `/v1/models` - List available models + +- **Key Features for Development**: + - **Responses API Format**: Structured output with reasoning, function_call, and message items + - **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences + - **Conversation History**: Hybrid client/server approach with auto-generated IDs + - **Context Management**: Token counting and window trimming + - **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement + - **Error Handling**: Custom exception types (RateLimitError, ContextLengthError) + - **Tool Calling**: PydanticAI tool integration with permanent tools + - **Testing**: Comprehensive test suite with mocks and real Ollama integration ## FastAPI Best Practices @@ -413,45 +376,66 @@ app = create_application() The user will manage git operations themselves unless they specifically request assistance. -### Code Structure (Current Implementation) -- ✅ Use async/await for ALL I/O operations (database, HTTP, file access) -- ✅ Use sync (def) for blocking SDKs or CPU-intensive work -- ✅ Implement proper error handling and logging -- ✅ Follow dependency injection for validation and shared resources -- ✅ Use Pydantic models for ALL request/response validation -- ✅ Keep business logic in service modules, not routers -- ✅ Domain-based project structure (not file-type based) +### Server Logs and Debugging + +**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is: +- Cleared on each server startup (fresh logs every time) +- Written in real-time as the server runs +- Already gitignored (won't be committed) + +**Accessing Logs**: You can read the log file at any time while the server is running: +```bash +# View current logs +cat logs/server.log + +# Follow logs in real-time +tail -f logs/server.log + +# Search logs +grep "ERROR" logs/server.log +``` + +This is useful for debugging issues, monitoring API calls, and understanding server behavior during development. + +### Code Structure Guidelines +- Use async/await for ALL I/O operations (database, HTTP, file access) +- Use sync (def) for blocking SDKs or CPU-intensive work +- Implement proper error handling and logging +- Follow dependency injection for validation and shared resources +- Use Pydantic models for ALL request/response validation +- Keep business logic in service modules, not routers +- Domain-based project structure (not file-type based) ### Security Considerations -- ✅ Validate all inputs using Pydantic models -- ✅ Use environment variables for sensitive configuration -- ✅ Keep dependencies updated (all CVE-checked as of 2025-12-06) -- ✅ Minor version locking for supply chain protection -- 🚧 Implement rate limiting for API endpoints (future) -- 🚧 Add authentication/API keys (future) +- Validate all inputs using Pydantic models +- Use environment variables for sensitive configuration +- Keep dependencies updated and CVE-checked +- Minor version locking for supply chain protection +- Consider rate limiting for production deployment +- Plan for authentication/API keys when needed -### Testing (Current Coverage: 78.95%, 95 tests) -- ✅ Integration tests for API endpoints -- ✅ Streaming functionality with 20s timeout protection -- ✅ Async test support with pytest-asyncio -- ✅ Validate OpenAI API compatibility -- ✅ Mock responses for all endpoints -- ✅ Main application tests (CORS, exception handlers, lifespan) -- ✅ Chat streaming wrapper tests -- 🚧 Future: Mock Ollama responses when integrated +### Testing Approach +- Write integration tests for API endpoints +- Test streaming functionality with appropriate timeouts +- Use pytest-asyncio for async test support +- Validate OpenAI API compatibility in tests +- Test both mock and real LLM integrations +- Cover main application (CORS, exception handlers, lifespan) +- Test wrapper layers (chat completions, etc.) +- Include tool functionality tests -### Configuration -- ✅ Use `.env` files for local development -- ✅ Document all environment variables in README -- ✅ Provide sensible defaults where possible -- ✅ BaseSettings from pydantic-settings -- 🚧 Support container-based configuration (future) +### Configuration Management +- Use `.env` files for local development +- Document all environment variables in README +- Provide sensible defaults where possible +- Use BaseSettings from pydantic-settings +- Support both local and container-based configuration ## Common Patterns -### Streaming Response Pattern (✅ Implemented) +### Streaming Response Pattern -See `src/chat/router.py` for the current implementation: +Example from `src/chat/router.py`: ```python from sse_starlette.sse import EventSourceResponse @@ -468,9 +452,9 @@ async def stream(): return EventSourceResponse(event_generator()) ``` -### PydanticAI Agent Pattern (🚧 Future Reference) +### PydanticAI Agent Pattern -For future integration when connecting to Ollama: +When implementing agents with PydanticAI and Ollama: ```python from pydantic_ai import Agent @@ -484,9 +468,9 @@ agent = Agent( result = await agent.run('Your prompt') ``` -### OpenAI-Compatible Response Format (✅ Implemented) +### OpenAI-Compatible Response Format -Current implementation in `src/chat/schemas.py`: +Example schema from `src/chat/schemas.py`: ```python { @@ -502,12 +486,59 @@ Current implementation in `src/chat/schemas.py`: } ``` +### PydanticAI Tool Registration Pattern + +Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples: + +```python +from pydantic_ai import Agent, RunContext + +# After creating the agent +@agent.tool +def tool_name(ctx: RunContext[None], param: str) -> str: + """ + Tool description that the LLM sees. + + Args: + param: Parameter description + + Returns: + Result description + """ + return result +``` + +**Tool Implementation Guidelines**: +- Keep tools in `src/agents/tools.py` for reusability +- Use clear, descriptive docstrings (LLM reads these) +- Include parameter descriptions in docstrings +- Handle errors gracefully and return error messages as strings +- For async operations, declare the tool function as `async def` +- Test tools independently before integration + +**Example Tool Module** (`src/agents/tools.py`): +```python +def calculate(expression: str) -> str: + """Safe calculator implementation.""" + try: + # Implementation + return str(result) + except Exception as e: + return f"Error: {str(e)}" + +async def search_web(query: str) -> str: + """Web search via SearXNG.""" + async with httpx.AsyncClient() as client: + # Implementation + return formatted_results +``` + ## Update Policy This document should be updated when: +- New development patterns are established - Package versions are upgraded -- New major features are added -- Breaking API changes occur -- Security vulnerabilities are discovered +- Major architectural changes occur +- New best practices are identified -Last updated: 2025-12-06 +Last updated: 2025-12-06 (Tools integration) diff --git a/IMPLEMENTATION_ROADMAP.md b/IMPLEMENTATION_ROADMAP.md index 6be1122..ad39ee4 100644 --- a/IMPLEMENTATION_ROADMAP.md +++ b/IMPLEMENTATION_ROADMAP.md @@ -4,69 +4,84 @@ This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system. -## Current State (v0.1.1 - Basic Setup Complete) +## Current State (v0.1.1+ - Phase 1 Mostly Complete) **What we have**: - ✅ **The Orchestrator** - FastAPI infrastructure layer - OpenAI-compatible API endpoints (Responses API + Chat Completions) - Streaming coordination and conversation management - Response format with reasoning support - - Test infrastructure (95 tests, 78.95% coverage) -- ✅ Mock agents (lorem-tester for testing, Tatlock placeholder) -- ✅ Agent interface abstraction ready for PydanticAI + - Test infrastructure (131 tests, 81.78% coverage) +- ✅ **Tatlock Agent** - Real PydanticAI integration + - Connected to Ollama (mistral-nemo:latest) + - British butler personality with research mindset + - Streaming responses with reasoning + - Tool calling framework functional +- ✅ **Permanent Tools** + - Calculator (safe mathematical expressions) + - Date/Time toolkit (current time, relative dates, time differences) + - Web search (SearXNG integration) +- ✅ Mock agent (lorem-tester for testing) +- ✅ Agent interface abstraction **What we need**: -- **The Household** - PydanticAI agent implementations: +- **The Household** - Full multi-agent coordination: - The Steward (first-tier request analysis) - - Tatlock Butler (second-tier coordination with personality) + - Tatlock coordination layer (expert agent delegation) - Expert household staff agents (Librarian, Developer, Handyman, etc.) -- Real LLM integration (PydanticAI + Ollama) - Multi-tenant database architecture - Containerized service ecosystem - MCP (Model Context Protocol) integration +- Dynamic model switching for specialized tasks --- -## Phase 1: Real LLM Integration - Ollama + PydanticAI +## Phase 1: Real LLM Integration - PydanticAI + Tools **Goal**: Connect to actual language models and establish the base plumbing +**Note**: Ollama is an external service dependency (already running separately) + ### Deliverables -1. **Ollama Setup** - - Docker compose configuration for Ollama - - Model download and management - - Base model selection (single model in VRAM) - - Health checking and monitoring +1. **PydanticAI Integration** ✅ + - PydanticAI → Ollama connection ✅ + - Agent creation patterns ✅ + - Streaming response handling ✅ + - Error handling and retries ✅ -2. **PydanticAI Integration** - - PydanticAI → Ollama connection - - Agent creation patterns - - Streaming response handling - - Error handling and retries +2. **Convert Tatlock Agent** ✅ + - Convert Tatlock agent from mock to PydanticAI ✅ + - British butler personality prompt ✅ + - Research-oriented mindset ✅ + - Streaming to reasoning output ✅ + - Tool calling framework setup ✅ -3. **Replace Mock Agents** - - Convert Tatlock agent from mock to PydanticAI - - Basic personality prompt - - Streaming to reasoning output - - Tool calling framework setup +3. **Permanent Tools** ✅ + - Calculator: Safe mathematical expression evaluation ✅ + - Date/Time toolkit: Current time, relative dates, time differences ✅ + - Web search: SearXNG integration (external service) ✅ + - Tool registration with PydanticAI ✅ -4. **Testing Infrastructure** - - Integration tests with real LLM - - Prompt testing utilities - - Response quality validation - - Performance benchmarking +4. **Testing Infrastructure** ✅ + - Integration tests with real LLM ✅ + - Tool functionality tests ✅ + - Response quality validation ✅ + - 131 tests, 81.78% coverage ✅ ### Success Criteria -- [ ] Ollama running in Docker -- [ ] Base model loaded and responding -- [ ] PydanticAI agents can call Ollama -- [ ] Streaming works end-to-end -- [ ] Can switch models (e.g., Codestral for code) -- [ ] Tests pass with real LLM +- [x] **PydanticAI agents can call Ollama** (mistral-nemo:latest) +- [x] **Streaming works end-to-end** +- [x] **Tool calling framework functional** +- [x] **Permanent tools working** (calculator, date/time, search) +- [x] **Tests pass with real LLM** +- [ ] Can switch models dynamically (e.g., Codestral for code) -### Estimated Effort -**2-3 weeks** - Critical foundation for everything else +### Status +**✅ MOSTLY COMPLETE** - Tatlock agent functional with permanent tools + +### Remaining Work +- Dynamic model switching for specialized tasks (e.g., Codestral for coding) ### Why First? Without real LLM integration, we can't meaningfully implement the Steward/Butler pattern. Everything else depends on having actual AI agents working. diff --git a/README.md b/README.md index 61cbe42..95ee5ab 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,9 @@ A privacy-first, offline-capable personal assistant system that coordinates spec - ✅ **Production-ready testing API** with OpenAI Responses API format - ✅ **Open WebUI integration** with reasoning bubbles (`` tags) - ✅ **Conversation history** with auto-generated IDs and context management -- ✅ **Comprehensive testing** - 95 tests, 78.95% coverage -- 🚧 **PydanticAI integration** prepared for future real LLM connection +- ✅ **Tatlock PydanticAI Agent** - Real LLM integration with Ollama + permanent tools +- ✅ **Permanent Tools** - Calculator, date/time toolkit, web search (SearXNG) +- ✅ **Comprehensive testing** - 131 tests, 81.78% coverage ## Features @@ -36,20 +37,32 @@ A privacy-first, offline-capable personal assistant system that coordinates spec - **Parameter Validation**: Temperature (0.0-2.0), reasoning effort levels, max tokens, stop sequences - **Real-time Enforcement**: Stop sequence detection and max token limits during streaming -### Testing Models +### Available Models - **lorem-tester**: Full-featured mock agent with realistic behavior - Configurable reasoning effort levels - Random tool/function calls - Error triggers for testing (rate_limit, context_overflow) -- **tatlock**: Placeholder for future PydanticAI agent +- **Tatlock**: Real PydanticAI agent with butler personality + - **LLM Backend**: Ollama (mistral-nemo:latest) + - **Personality**: Witty British butler, research-oriented + - **Permanent Tools**: + - **Calculator**: Safe mathematical expression evaluation (arithmetic, algebra, trigonometry, logarithms) + - **Date/Time Toolkit**: Current time, relative dates ("1 week ago"), time differences + - **Web Search**: Privacy-preserving search via SearXNG + - **Capabilities**: Streaming, reasoning, tool calling + - **Phase**: Phase 1 - Basic Integration (full household coordination coming in future phases) ## Requirements - Python 3.12+ (Python 3.12.11 recommended) -- No external dependencies for mock API -- (Future: Network access for PydanticAI integration) +- **Ollama** (for Tatlock agent): Running locally or network-accessible + - Download: https://ollama.ai/ + - Model: `ollama pull mistral-nemo:latest` +- **SearXNG** (for web search tool): Optional but recommended + - Docker: `docker run -d -p 8087:8080 searxng/searxng` + - Or use public instance (less private) ## Quick Start @@ -164,6 +177,42 @@ curl http://localhost:8000/v1/responses \ **Note**: Client must send full conversation history in `input` array (OpenAI compatible). Server optionally tracks via `metadata.conversation_id` for future features. +### Using Tatlock with Tools + +Tatlock automatically uses his permanent tools when appropriate: + +```bash +# Mathematical calculation +curl http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Tatlock", + "messages": [{"role": "user", "content": "What is sqrt(144) + 25?"}] + }' + +# Date/time queries +curl http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Tatlock", + "messages": [{"role": "user", "content": "What was the date 2 weeks ago?"}] + }' + +# Web search for current information +curl http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Tatlock", + "messages": [{"role": "user", "content": "Search for recent Python 3.12 features"}] + }' +``` + +**Tatlock's Tool Usage Philosophy:** +- Uses calculator for ALL mathematics (even simple arithmetic) +- Uses date/time tools instead of guessing dates +- Searches for current/volatile information to verify facts +- Maintains a researcher's mindset with tool-assisted verification + ## Open WebUI Integration ### Connection @@ -205,9 +254,14 @@ pytest # Run with coverage pytest --cov=src --cov-report=term-missing -# Current: 95 tests, 78.95% coverage +# Current: 131 tests, 81.78% coverage ``` +**Test Categories:** +- Unit tests: Agent tools, streaming, schemas +- Integration tests: Full API stack with real Ollama calls +- End-to-end tests: Chat completions, responses API + ## Deployment ### Production Server @@ -234,15 +288,24 @@ Create a `.env` file for custom configuration: API_HOST=0.0.0.0 API_PORT=8000 +# Ollama Configuration +OLLAMA_HOST=http://localhost:11434 +OLLAMA_DEFAULT_MODEL=mistral-nemo:latest +OLLAMA_TIMEOUT=120 + +# SearXNG Configuration (for web search tool) +SEARXNG_HOST=http://localhost:8087 +SEARXNG_TIMEOUT=30 + # Logging LOG_LEVEL=INFO # CORS (default: allow all) CORS_ORIGINS=["*"] - -# Future: LLM configuration ``` +See `.env.example` for full configuration options. + ## Troubleshooting ### Streaming not working @@ -260,19 +323,37 @@ CORS_ORIGINS=["*"] - Or manually enable in Responses API: `"reasoning": {"effort": "medium", "summary": "auto"}` - Check Open WebUI version supports `` tags +### Tatlock agent errors +- Verify Ollama is running: `curl http://localhost:11434/api/tags` +- Check model is downloaded: `ollama list` +- Review environment variables: `OLLAMA_HOST`, `OLLAMA_DEFAULT_MODEL` +- Check logs: `tail -f logs/server.log` + +### Web search not working +- Verify SearXNG is running: `curl http://localhost:8087/` +- Check `SEARXNG_HOST` environment variable +- SearXNG is optional - Tatlock will note if search is unavailable + ## Project Structure ``` tatlock/ ├── src/ │ ├── agents/ # Agent interface and implementations +│ │ ├── base.py # AgentInterface abstract class +│ │ ├── lorem_tester.py # Mock agent for testing +│ │ ├── tatlock.py # Real PydanticAI butler agent +│ │ ├── tools.py # Permanent tools (calculator, date/time, search) +│ │ └── registry.py # Model registry │ ├── responses/ # Responses API (primary endpoint) │ ├── chat/ # Chat Completions wrapper │ ├── models/ # Models listing │ ├── core/ # Shared utilities and config │ └── main.py # Application entry point -├── tests/ # Comprehensive test suite +├── tests/ # Comprehensive test suite (131 tests) ├── AGENTS.md # LLM agent development guidelines +├── PHILOSOPHY.md # System vision and architecture +├── IMPLEMENTATION_ROADMAP.md # Development phases ├── CHANGELOG.md # Version history └── README.md # This file ``` @@ -307,7 +388,7 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md ## Version -Current version: **0.1.1** - Basic setup complete +Current version: **0.2.0** - PydanticAI Integration with Permanent Tools ---