diff --git a/AGENTS.md b/AGENTS.md index 096e3fd..ec4801b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,7 +13,7 @@ This project implements an OpenAI-compatible API endpoint using FastAPI, with st ### Current Architecture (As of 2025-12-06) -This project implements a **hybrid architecture** with the Responses API as the primary endpoint and Chat Completions as a compatibility wrapper: +This project implements the **Orchestrator** infrastructure layer with a hybrid API architecture: ``` Client (Open WebUI) @@ -22,9 +22,24 @@ Chat Completions (/v1/chat/completions) → Wrapper ↓ Responses API (/v1/responses) → Primary ↓ -Agent Interface (lorem-tester, tatlock) +Agent Interface (lorem-tester, Tatlock) + ↓ +Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.) ``` +**Architectural Layers:** + +1. **The Orchestrator** (Current Implementation) + - FastAPI application providing the infrastructure + - HTTP/SSE endpoints, streaming coordination + - Conversation history and context management + - OpenAI-compatible API surface + +2. **Future: The Household** (Phases 1-4) + - **Steward**: First-tier LLM for request analysis (PydanticAI agent) + - **Tatlock**: Second-tier LLM with butler personality (PydanticAI agent) + - **Expert Agents**: Domain specialists (Librarian, Developer, Handyman, etc.) + **Key Architectural Decisions:** 1. **Single Source of Truth**: All response generation happens in the Responses API @@ -45,7 +60,7 @@ Agent Interface (lorem-tester, tatlock) - Random tool/function calls - Error triggers for testing - Temperature variation - - **tatlock**: Placeholder for future PydanticAI agent + - **Tatlock**: Advertised model name (currently mock, future: PydanticAI Butler agent) 4. **Hybrid Conversation History**: - Client MUST send full context in `input` array (OpenAI compatible) diff --git a/IMPLEMENTATION_ROADMAP.md b/IMPLEMENTATION_ROADMAP.md new file mode 100644 index 0000000..6be1122 --- /dev/null +++ b/IMPLEMENTATION_ROADMAP.md @@ -0,0 +1,584 @@ +# Tatlock Implementation Roadmap + +> **Reference**: See [PHILOSOPHY.md](PHILOSOPHY.md) for the target architecture and vision + +This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system. + +## Current State (v0.1.1 - Basic Setup Complete) + +**What we have**: +- ✅ **The Orchestrator** - FastAPI infrastructure layer + - OpenAI-compatible API endpoints (Responses API + Chat Completions) + - Streaming coordination and conversation management + - Response format with reasoning support + - Test infrastructure (95 tests, 78.95% coverage) +- ✅ Mock agents (lorem-tester for testing, Tatlock placeholder) +- ✅ Agent interface abstraction ready for PydanticAI + +**What we need**: +- **The Household** - PydanticAI agent implementations: + - The Steward (first-tier request analysis) + - Tatlock Butler (second-tier coordination with personality) + - Expert household staff agents (Librarian, Developer, Handyman, etc.) +- Real LLM integration (PydanticAI + Ollama) +- Multi-tenant database architecture +- Containerized service ecosystem +- MCP (Model Context Protocol) integration + +--- + +## Phase 1: Real LLM Integration - Ollama + PydanticAI + +**Goal**: Connect to actual language models and establish the base plumbing + +### Deliverables + +1. **Ollama Setup** + - Docker compose configuration for Ollama + - Model download and management + - Base model selection (single model in VRAM) + - Health checking and monitoring + +2. **PydanticAI Integration** + - PydanticAI → Ollama connection + - Agent creation patterns + - Streaming response handling + - Error handling and retries + +3. **Replace Mock Agents** + - Convert Tatlock agent from mock to PydanticAI + - Basic personality prompt + - Streaming to reasoning output + - Tool calling framework setup + +4. **Testing Infrastructure** + - Integration tests with real LLM + - Prompt testing utilities + - Response quality validation + - Performance benchmarking + +### Success Criteria +- [ ] Ollama running in Docker +- [ ] Base model loaded and responding +- [ ] PydanticAI agents can call Ollama +- [ ] Streaming works end-to-end +- [ ] Can switch models (e.g., Codestral for code) +- [ ] Tests pass with real LLM + +### Estimated Effort +**2-3 weeks** - Critical foundation for everything else + +### Why First? +Without real LLM integration, we can't meaningfully implement the Steward/Butler pattern. Everything else depends on having actual AI agents working. + +--- + +## Phase 2: Orchestration Layer - The Steward + +**Goal**: Implement the first-tier LLM call for tool/agent selection + +### Deliverables + +1. **Orchestrator Framework** + - Python orchestrator service/module + - Request preprocessing pipeline + - Tool/agent registry system + - Recommendation format definition + +2. **Steward Agent Implementation** + - Steward prompt engineering + - Tool selection logic + - Agent recommendation generation + - Output format (note to Butler) + +3. **Tool Registry** + - Available tools catalog + - Tool capability descriptions + - Tool category organization + - Dynamic tool loading + +### Success Criteria +- [ ] Steward analyzes incoming requests +- [ ] Produces tool/agent recommendations +- [ ] Recommendations formatted as prepended note +- [ ] Tool registry is queryable and extensible +- [ ] Steward output visible in reasoning stream + +### Estimated Effort +**3-4 weeks** - Core intelligence routing + +--- + +## Phase 3: The Butler - Tatlock Agent + +**Goal**: Implement the second-tier coordinator with personality within the existing Orchestrator infrastructure + +**Context**: The Orchestrator (FastAPI infrastructure) already exists. This phase implements the real Tatlock PydanticAI agent to replace the current mock agent. + +### Deliverables + +1. **Butler Agent (Tatlock)** + - PydanticAI agent implementation within Orchestrator + - Personality prompt engineering (witty British butler) + - Tool calling framework + - Multi-agent coordination logic + +2. **Scoped Tool Access** + - Filter tools based on Steward recommendations + - Dynamic tool loading for Butler context + - Tool execution framework + - Result aggregation + +3. **Real-Time Reasoning Output** + - Stream all Butler activities to reasoning output + - Tool call progress indicators + - Expert agent consultation messages + - Wait time transparency + +### Success Criteria +- [ ] Tatlock receives enriched requests (user + Steward notes) +- [ ] Only recommended tools are available +- [ ] Tatlock coordinates multiple tool calls +- [ ] All actions streamed to reasoning output +- [ ] Responses have consistent personality +- [ ] Synthesizes multi-source results coherently + +### Estimated Effort +**4-5 weeks** - Complex coordination logic + +--- + +## Phase 4: Expert Household Staff - Core Agents + +**Goal**: Implement the initial set of domain-specific expert agents + +### Priority Expert Agents + +1. **The Librarian** (Research & Knowledge Management) ⭐ **Priority** + - Research assistance and synthesis + - Automatic research dossier generation + - Knowledge base queries and organization + - Reference management + - Wiki integration (future: dedicated wiki container) + - Mind map maintenance (future) + - *Rationale: Helps guide development priorities through better research* + +2. **The Developer** (Software Development) + - Code generation assistance + - Debugging support + - Documentation generation + - Architecture guidance + - *Rationale: Directly supports building the system itself* + +3. **The Handyman** (System Maintenance) + - System status queries + - Log analysis + - Basic troubleshooting + - Infrastructure monitoring + +4. **The Secretary** (Scheduling & Organization) + - Calendar integration (placeholder) + - Task management (placeholder) + - Reminder system + - Schedule conflict detection + +5. **The Housekeeper** (Home Automation) + - Device control interface + - Status queries + - Automation triggers + - Environmental monitoring + +### Each Agent Includes +- Specialized prompt and personality +- Domain-specific tools +- MCP integration points (where applicable) +- Integration with Butler orchestration + +### Success Criteria +- [ ] Each agent implemented as separate module +- [ ] Agents callable via tool framework +- [ ] Agents use specialized prompts +- [ ] Results integrate cleanly with Butler +- [ ] Can invoke specialized models (e.g., Codestral for Developer) + +### Estimated Effort +**6-8 weeks** - Parallel development possible + +--- + +## Phase 5: Persistence Layer - Database & Multi-Tenancy + +**Goal**: Add persistent storage and multi-user support when needed + +### Deliverables + +1. **PostgreSQL Integration** + - Docker compose configuration for PostgreSQL + - Database schema design with tenant isolation + - Alembic migrations setup + - SQLAlchemy models + +2. **Multi-Tenant Architecture** + - Tenant identification middleware + - Tenant-scoped database sessions + - User authentication system (basic) + - Per-tenant data isolation + +3. **Core Data Models** + - Users and tenants + - Conversations and messages (migrate from in-memory) + - Agent interactions log + - System configuration and preferences + +4. **Migration Strategy** + - Gradual migration from in-memory to database + - Backward compatibility during transition + - Data export/import utilities + +### Success Criteria +- [ ] PostgreSQL container running +- [ ] Multiple users can authenticate separately +- [ ] Each user sees only their own data +- [ ] Conversations persist across restarts +- [ ] Database migrations work correctly +- [ ] Tests verify tenant isolation + +### Estimated Effort +**3-4 weeks** - Data layer foundation + +### Why Later? +The core orchestration (Steward → Butler → Experts) can work entirely with in-memory state. We only need database persistence when we want conversations to survive restarts and multiple users to have isolated experiences. + +--- + +## Phase 6: Extended Services Integration + +**Goal**: Connect to additional supporting services + +### Services to Integrate + +1. **Redis (Memory & Caching)** + - Docker compose setup + - Conversation cache + - Short-term memory + - Session management + +3. **Qdrant (Vector Storage)** + - Docker compose setup + - Long-term memory embeddings + - Semantic search + - Conversation history vectors + +4. **SearxNG (Web Search)** + - Docker compose setup + - Search tool integration + - Result processing + - Privacy-preserving queries + +### Success Criteria +- [ ] All services defined in docker-compose.yml +- [ ] Services communicate correctly +- [ ] Tatlock can invoke web search +- [ ] Redis used for session data +- [ ] Qdrant stores conversation embeddings +- [ ] Ollama serves the base model + +### Estimated Effort +**3-4 weeks** - Infrastructure setup + +--- + +## Phase 7: MCP (Model Context Protocol) Integration + +**Goal**: Enable rich tool integrations via MCP + +### Deliverables + +1. **MCP Server Framework** + - MCP server implementation + - Tool registration via MCP + - Schema validation + - Error handling + +2. **MCP Client in Agents** + - PydanticAI MCP integration + - Tool discovery from MCP servers + - Dynamic tool loading + - Result processing + +3. **Initial MCP Tools** + - File system operations + - Database queries + - API integrations + - System commands + +### Success Criteria +- [ ] MCP server running +- [ ] Tools exposed via MCP protocol +- [ ] Agents can discover and use MCP tools +- [ ] New tools addable without code changes +- [ ] MCP tools visible in Steward recommendations + +### Estimated Effort +**3-4 weeks** - Standards-based integration + +--- + +## Phase 8: Advanced Memory & Context + +**Goal**: Implement sophisticated memory and context management + +### Deliverables + +1. **Long-Term Memory** + - Conversation embedding pipeline + - Semantic search over history + - Memory consolidation + - Relevance ranking + +2. **Context Management** + - Smart context window trimming + - Conversation branching + - Topic tracking + - Memory retrieval integration + +3. **Personalization** + - User preference learning + - Interaction pattern analysis + - Adaptive responses + - Custom agent personalities per user + +### Success Criteria +- [ ] Conversations automatically embedded to Qdrant +- [ ] Relevant history retrieved for new requests +- [ ] Context stays within model limits +- [ ] User preferences affect responses +- [ ] Memory improves over time + +### Estimated Effort +**4-5 weeks** - AI/ML heavy + +--- + +## Phase 9: Extended Household Staff + +**Goal**: Add specialized agents for additional domains + +### Future Agents + +1. **The Librarian** (Knowledge Management) + - Personal documentation indexing + - Research assistance + - Knowledge base queries + - Reference management + +2. **The Accountant** (Financial Tracking) + - Expense tracking + - Budget monitoring + - Financial reports + - Transaction categorization + +3. **The Chef** (Meal Planning) + - Recipe management + - Meal planning + - Nutrition tracking + - Grocery lists + +4. **Others as Needed** + - Domain-specific as requirements emerge + +### Success Criteria +- [ ] Each new agent follows household pattern +- [ ] Integrates with Steward/Butler flow +- [ ] Has appropriate specialized tools +- [ ] Documented in PHILOSOPHY.md updates + +### Estimated Effort +**Ongoing** - Add as needed + +--- + +## Phase 10: User Experience Refinement + +**Goal**: Polish the interaction experience + +### Deliverables + +1. **Personality Tuning** + - Refine Tatlock's wit and tone + - Consistent household character + - Cultural references appropriate + - Humor that doesn't annoy + +2. **Transparency Improvements** + - Better progress indicators + - Clearer reasoning explanations + - Informative wait messages + - Error message clarity + +3. **Performance Optimization** + - Response time improvements + - Model loading optimization + - Caching strategies + - Streaming smoothness + +### Success Criteria +- [ ] Users find Tatlock engaging +- [ ] Wait times feel reasonable +- [ ] Errors are understandable +- [ ] System feels responsive + +### Estimated Effort +**Ongoing** - Continuous improvement + +--- + +## Phase 11: Production Hardening + +**Goal**: Make the system production-ready for homelab deployment + +### Deliverables + +1. **Deployment** + - Complete docker-compose stack + - Environment configuration + - Backup strategies + - Update procedures + +2. **Monitoring** + - Health checks + - Performance metrics + - Error tracking + - Usage analytics + +3. **Security** + - Authentication hardening + - Rate limiting + - Input validation + - Audit logging + +4. **Documentation** + - Installation guide + - Configuration reference + - Troubleshooting guide + - Architecture documentation + +### Success Criteria +- [ ] One-command deployment +- [ ] System health is monitorable +- [ ] Secure for homelab use +- [ ] Well documented + +### Estimated Effort +**3-4 weeks** - Production polish + +--- + +## Dependencies Between Phases + +``` +Phase 1 (Ollama + PydanticAI) ← Foundation for all AI + ↓ +Phase 2 (Steward) + ↓ +Phase 3 (Butler/Tatlock) + ↓ +Phase 4 (Expert Agents) ← Phase 7 (MCP) can enhance + ↓ +Phase 5 (Database/Multi-Tenancy) ← Can be deferred + ↓ +Phase 6 (Extended Services) → Phase 8 (Advanced Memory) + ↓ +Phase 9 (Extended Staff) → Phase 10 (UX) → Phase 11 (Production) +``` + +**Critical Path**: Phases 1 → 2 → 3 → 4 must be sequential +**Can Be Deferred**: Phase 5 (Database) until you need persistence +**Parallel Opportunities**: Phase 6 and 7 can overlap; Phase 9 and 10 ongoing + +--- + +## Overall Timeline Estimate + +**Minimum Viable Household** (Phases 1-4): **15-20 weeks** +- Working Steward → Butler → Expert Agents with real LLM +- In-memory state (no persistence needed yet) +- Core household functional + +**With Persistence** (Phases 1-5): **18-24 weeks** +- Add database and multi-tenancy +- Conversations survive restarts +- Multiple users supported + +**Full-Featured System** (Phases 1-9): **35-45 weeks** +- All services integrated +- Advanced memory and context +- Extended household staff + +**Production-Ready** (All phases): **40-50 weeks** +- Polished UX +- Hardened for homelab deployment +- Fully documented + +*Note: Timeline assumes consistent part-time development effort* + +--- + +## Success Metrics + +### Technical +- System implements PHILOSOPHY.md patterns +- All household roles functional +- Multi-tenant isolation verified +- Real-time reasoning transparency working +- MCP integration complete + +### User Experience +- Tatlock feels like interacting with a butler +- Wait times are transparent and acceptable +- Expert agents provide value in their domains +- System is reliable and trustworthy + +### Architecture +- Clean separation between household roles +- Easy to add new agents/tools +- Model efficiency (base model stays loaded) +- Scales to household + friends usage + +--- + +## Risk Management + +### High Risk Items +1. **PydanticAI + Ollama integration complexity** + - Mitigation: Prototype early, iterate on connection layer + +2. **Multi-agent coordination complexity** + - Mitigation: Start simple, add coordination gradually + +3. **Model performance on homelab hardware** + - Mitigation: Model selection, quantization, optimization + +4. **Prompt engineering for personality consistency** + - Mitigation: Extensive testing, user feedback, iteration + +### Medium Risk Items +- MCP protocol adoption and tooling maturity +- Vector embedding quality for memory +- Home automation integration variability +- User authentication security + +--- + +## Next Steps + +1. **Immediate**: Commit model name fix (Tatlock) +2. **Week 1-2**: Begin Phase 1 (PostgreSQL + multi-tenancy design) +3. **Week 3**: Parallel prototype of Steward agent +4. **Ongoing**: Update this roadmap as we learn + +--- + +**Document Status**: Active planning document +**Created**: 2025-12-06 +**Last Updated**: 2025-12-06 diff --git a/PHILOSOPHY.md b/PHILOSOPHY.md index 2f931de..66b2122 100644 --- a/PHILOSOPHY.md +++ b/PHILOSOPHY.md @@ -112,6 +112,33 @@ The system should feel less like "using a tool" and more like "asking a capable ## The Household Architecture +### System Layers + +The Tatlock system consists of two distinct architectural layers: + +#### The Orchestrator (Infrastructure Layer) + +The **Orchestrator** is the FastAPI application that provides the technical infrastructure: +- HTTP/SSE endpoints (`/v1/responses`, `/v1/chat/completions`) +- Streaming coordination and conversation management +- Token counting and context window management +- Integration with Open WebUI and other clients +- Request/response lifecycle management + +This is the "plumbing" layer that exists now and handles all the technical concerns of running an OpenAI-compatible API. + +#### Tatlock - The Butler (Agent Layer) + +**Tatlock** is the PydanticAI agent that provides the intelligence and personality: +- The witty British butler persona +- Coordination with the Steward and household staff +- Multi-agent orchestration and synthesis +- Context-aware, personalized responses + +The Orchestrator hosts Tatlock—users interact with "Tatlock" (the advertised model name), but technically they're talking to the Orchestrator infrastructure which routes requests through the Tatlock agent. + +**Current State**: The Orchestrator exists and uses mock agents. Phase 1-3 of the implementation roadmap will integrate the real Tatlock agent using PydanticAI. + ### The British Household Metaphor Tatlock adopts the organizational structure of a traditional British estate household, where specialized staff members handle distinct domains of responsibility under the coordination of a capable butler. This metaphor is not merely aesthetic—it reflects a deliberate architectural pattern that enables focused expertise, clear separation of concerns, and efficient coordination. @@ -170,8 +197,8 @@ The household operates through a carefully orchestrated two-tier process: #### Tier 1: The Steward's Preparation -1. **User request arrives** in the system -2. **Steward receives the raw request** for analysis +1. **User request arrives** at the Orchestrator (via HTTP API) +2. **Orchestrator routes** the raw request to the Steward for analysis 3. **Steward determines** which tools and household staff are relevant 4. **Steward prepares recommendations**, written as a note to Tatlock 5. **Recommendations are prepended** to the user's request