tatlock/IMPLEMENTATION_ROADMAP.md

# Tatlock Implementation Roadmap

> **Reference**: See [PHILOSOPHY.md](PHILOSOPHY.md) for the target architecture and vision

This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system.

## Current State (v0.1.1+ - Phase 1 Mostly Complete)

**What we have**:
- ✅ **The Orchestrator** - FastAPI infrastructure layer
  - OpenAI-compatible API endpoints (Responses API + Chat Completions)
  - Streaming coordination and conversation management
  - Response format with reasoning support
  - Test infrastructure (131 tests, 81.78% coverage)
- ✅ **Tatlock Agent** - Real PydanticAI integration
  - Connected to Ollama (mistral-nemo:latest)
  - British butler personality with research mindset
  - Streaming responses with reasoning
  - Tool calling framework functional
- ✅ **Permanent Tools**
  - Calculator (safe mathematical expressions)
  - Date/Time toolkit (current time, relative dates, time differences)
  - Web search (SearXNG integration)
- ✅ Mock agent (lorem-tester for testing)
- ✅ Agent interface abstraction

**What we need**:
- **The Household** - Full multi-agent coordination:
  - The Steward (first-tier request analysis)
  - Tatlock coordination layer (expert agent delegation)
  - Expert household staff agents (Librarian, Developer, Handyman, etc.)
- Multi-tenant database architecture
- Containerized service ecosystem
- MCP (Model Context Protocol) integration
- Dynamic model switching for specialized tasks

---

## Phase 1: Real LLM Integration - PydanticAI + Tools

**Goal**: Connect to actual language models and establish the base plumbing

**Note**: Ollama is an external service dependency (already running separately)

### Deliverables

1. **PydanticAI Integration** ✅
   - PydanticAI → Ollama connection ✅
   - Agent creation patterns ✅
   - Streaming response handling ✅
   - Error handling and retries ✅

2. **Convert Tatlock Agent** ✅
   - Convert Tatlock agent from mock to PydanticAI ✅
   - British butler personality prompt ✅
   - Research-oriented mindset ✅
   - Streaming to reasoning output ✅
   - Tool calling framework setup ✅

3. **Permanent Tools** ✅
   - Calculator: Safe mathematical expression evaluation ✅
   - Date/Time toolkit: Current time, relative dates, time differences ✅
   - Web search: SearXNG integration (external service) ✅
   - Tool registration with PydanticAI ✅

4. **Testing Infrastructure** ✅
   - Integration tests with real LLM ✅
   - Tool functionality tests ✅
   - Response quality validation ✅
   - 131 tests, 81.78% coverage ✅

### Success Criteria
- [x] **PydanticAI agents can call Ollama** (mistral-nemo:latest)
- [x] **Streaming works end-to-end**
- [x] **Tool calling framework functional**
- [x] **Permanent tools working** (calculator, date/time, search)
- [x] **Tests pass with real LLM**
- [ ] Can switch models dynamically (e.g., Codestral for code)

### Status
**✅ MOSTLY COMPLETE** - Tatlock agent functional with permanent tools

### Remaining Work
- Dynamic model switching for specialized tasks (e.g., Codestral for coding)

### Why First?
Without real LLM integration, we can't meaningfully implement the Steward/Butler pattern. Everything else depends on having actual AI agents working.

---

## Phase 2: Orchestration Layer - The Steward

**Goal**: Implement the first-tier LLM call for tool/agent selection

### Deliverables

1. **Orchestrator Framework**
   - Python orchestrator service/module
   - Request preprocessing pipeline
   - Tool/agent registry system
   - Recommendation format definition

2. **Steward Agent Implementation**
   - Steward prompt engineering
   - Tool selection logic
   - Agent recommendation generation
   - Output format (note to Butler)

3. **Tool Registry**
   - Available tools catalog
   - Tool capability descriptions
   - Tool category organization
   - Dynamic tool loading

### Success Criteria
- [ ] Steward analyzes incoming requests
- [ ] Produces tool/agent recommendations
- [ ] Recommendations formatted as prepended note
- [ ] Tool registry is queryable and extensible
- [ ] Steward output visible in reasoning stream

### Estimated Effort
**3-4 weeks** - Core intelligence routing

---

## Phase 3: The Butler - Tatlock Agent

**Goal**: Implement the second-tier coordinator with personality within the existing Orchestrator infrastructure

**Context**: The Orchestrator (FastAPI infrastructure) already exists. This phase implements the real Tatlock PydanticAI agent to replace the current mock agent.

### Deliverables

1. **Butler Agent (Tatlock)**
   - PydanticAI agent implementation within Orchestrator
   - Personality prompt engineering (witty British butler)
   - Tool calling framework
   - Multi-agent coordination logic

2. **Scoped Tool Access**
   - Filter tools based on Steward recommendations
   - Dynamic tool loading for Butler context
   - Tool execution framework
   - Result aggregation

3. **Real-Time Reasoning Output**
   - Stream all Butler activities to reasoning output
   - Tool call progress indicators
   - Expert agent consultation messages
   - Wait time transparency

### Success Criteria
- [ ] Tatlock receives enriched requests (user + Steward notes)
- [ ] Only recommended tools are available
- [ ] Tatlock coordinates multiple tool calls
- [ ] All actions streamed to reasoning output
- [ ] Responses have consistent personality
- [ ] Synthesizes multi-source results coherently

### Estimated Effort
**4-5 weeks** - Complex coordination logic

---

## Phase 4: Expert Household Staff - Core Agents

**Goal**: Implement the initial set of domain-specific expert agents

### Priority Expert Agents

1. **The Librarian** (Research & Knowledge Management) ⭐ **Priority**
   - Research assistance and synthesis
   - Automatic research dossier generation
   - Knowledge base queries and organization
   - Reference management
   - Wiki integration (future: dedicated wiki container)
   - Mind map maintenance (future)
   - *Rationale: Helps guide development priorities through better research*

2. **The Developer** (Software Development)
   - Code generation assistance
   - Debugging support
   - Documentation generation
   - Architecture guidance
   - *Rationale: Directly supports building the system itself*

3. **The Handyman** (System Maintenance)
   - System status queries
   - Log analysis
   - Basic troubleshooting
   - Infrastructure monitoring

4. **The Secretary** (Scheduling & Organization)
   - Calendar integration (placeholder)
   - Task management (placeholder)
   - Reminder system
   - Schedule conflict detection

5. **The Housekeeper** (Home Automation)
   - Device control interface
   - Status queries
   - Automation triggers
   - Environmental monitoring

### Each Agent Includes
- Specialized prompt and personality
- Domain-specific tools
- MCP integration points (where applicable)
- Integration with Butler orchestration

### Success Criteria
- [ ] Each agent implemented as separate module
- [ ] Agents callable via tool framework
- [ ] Agents use specialized prompts
- [ ] Results integrate cleanly with Butler
- [ ] Can invoke specialized models (e.g., Codestral for Developer)

### Estimated Effort
**6-8 weeks** - Parallel development possible

---

## Phase 5: Persistence Layer - Database & Multi-Tenancy

**Goal**: Add persistent storage and multi-user support when needed

### Deliverables

1. **PostgreSQL Integration**
   - Docker compose configuration for PostgreSQL
   - Database schema design with tenant isolation
   - Alembic migrations setup
   - SQLAlchemy models

2. **Multi-Tenant Architecture**
   - Tenant identification middleware
   - Tenant-scoped database sessions
   - User authentication system (basic)
   - Per-tenant data isolation

3. **Core Data Models**
   - Users and tenants
   - Conversations and messages (migrate from in-memory)
   - Agent interactions log
   - System configuration and preferences

4. **Migration Strategy**
   - Gradual migration from in-memory to database
   - Backward compatibility during transition
   - Data export/import utilities

### Success Criteria
- [ ] PostgreSQL container running
- [ ] Multiple users can authenticate separately
- [ ] Each user sees only their own data
- [ ] Conversations persist across restarts
- [ ] Database migrations work correctly
- [ ] Tests verify tenant isolation

### Estimated Effort
**3-4 weeks** - Data layer foundation

### Why Later?
The core orchestration (Steward → Butler → Experts) can work entirely with in-memory state. We only need database persistence when we want conversations to survive restarts and multiple users to have isolated experiences.

---

## Phase 6: Extended Services Integration

**Goal**: Connect to additional supporting services

### Services to Integrate

1. **Redis (Memory & Caching)**
   - Docker compose setup
   - Conversation cache
   - Short-term memory
   - Session management

3. **Qdrant (Vector Storage)**
   - Docker compose setup
   - Long-term memory embeddings
   - Semantic search
   - Conversation history vectors

4. **SearxNG (Web Search)**
   - Docker compose setup
   - Search tool integration
   - Result processing
   - Privacy-preserving queries

### Success Criteria
- [ ] All services defined in docker-compose.yml
- [ ] Services communicate correctly
- [ ] Tatlock can invoke web search
- [ ] Redis used for session data
- [ ] Qdrant stores conversation embeddings
- [ ] Ollama serves the base model

### Estimated Effort
**3-4 weeks** - Infrastructure setup

---

## Phase 7: MCP (Model Context Protocol) Integration

**Goal**: Enable rich tool integrations via MCP

### Deliverables

1. **MCP Server Framework**
   - MCP server implementation
   - Tool registration via MCP
   - Schema validation
   - Error handling

2. **MCP Client in Agents**
   - PydanticAI MCP integration
   - Tool discovery from MCP servers
   - Dynamic tool loading
   - Result processing

3. **Initial MCP Tools**
   - File system operations
   - Database queries
   - API integrations
   - System commands

### Success Criteria
- [ ] MCP server running
- [ ] Tools exposed via MCP protocol
- [ ] Agents can discover and use MCP tools
- [ ] New tools addable without code changes
- [ ] MCP tools visible in Steward recommendations

### Estimated Effort
**3-4 weeks** - Standards-based integration

---

## Phase 8: Advanced Memory & Context

**Goal**: Implement sophisticated memory and context management

### Deliverables

1. **Long-Term Memory**
   - Conversation embedding pipeline
   - Semantic search over history
   - Memory consolidation
   - Relevance ranking

2. **Context Management**
   - Smart context window trimming
   - Conversation branching
   - Topic tracking
   - Memory retrieval integration

3. **Personalization**
   - User preference learning
   - Interaction pattern analysis
   - Adaptive responses
   - Custom agent personalities per user

### Success Criteria
- [ ] Conversations automatically embedded to Qdrant
- [ ] Relevant history retrieved for new requests
- [ ] Context stays within model limits
- [ ] User preferences affect responses
- [ ] Memory improves over time

### Estimated Effort
**4-5 weeks** - AI/ML heavy

---

## Phase 9: Extended Household Staff

**Goal**: Add specialized agents for additional domains

### Future Agents

1. **The Librarian** (Knowledge Management)
   - Personal documentation indexing
   - Research assistance
   - Knowledge base queries
   - Reference management

2. **The Accountant** (Financial Tracking)
   - Expense tracking
   - Budget monitoring
   - Financial reports
   - Transaction categorization

3. **The Chef** (Meal Planning)
   - Recipe management
   - Meal planning
   - Nutrition tracking
   - Grocery lists

4. **Others as Needed**
   - Domain-specific as requirements emerge

### Success Criteria
- [ ] Each new agent follows household pattern
- [ ] Integrates with Steward/Butler flow
- [ ] Has appropriate specialized tools
- [ ] Documented in PHILOSOPHY.md updates

### Estimated Effort
**Ongoing** - Add as needed

---

## Phase 10: User Experience Refinement

**Goal**: Polish the interaction experience

### Deliverables

1. **Personality Tuning**
   - Refine Tatlock's wit and tone
   - Consistent household character
   - Cultural references appropriate
   - Humor that doesn't annoy

2. **Transparency Improvements**
   - Better progress indicators
   - Clearer reasoning explanations
   - Informative wait messages
   - Error message clarity

3. **Performance Optimization**
   - Response time improvements
   - Model loading optimization
   - Caching strategies
   - Streaming smoothness

### Success Criteria
- [ ] Users find Tatlock engaging
- [ ] Wait times feel reasonable
- [ ] Errors are understandable
- [ ] System feels responsive

### Estimated Effort
**Ongoing** - Continuous improvement

---

## Phase 11: Production Hardening

**Goal**: Make the system production-ready for homelab deployment

### Deliverables

1. **Deployment**
   - Complete docker-compose stack
   - Environment configuration
   - Backup strategies
   - Update procedures

2. **Monitoring**
   - Health checks
   - Performance metrics
   - Error tracking
   - Usage analytics

3. **Security**
   - Authentication hardening
   - Rate limiting
   - Input validation
   - Audit logging

4. **Documentation**
   - Installation guide
   - Configuration reference
   - Troubleshooting guide
   - Architecture documentation

### Success Criteria
- [ ] One-command deployment
- [ ] System health is monitorable
- [ ] Secure for homelab use
- [ ] Well documented

### Estimated Effort
**3-4 weeks** - Production polish

---

## Dependencies Between Phases

```
Phase 1 (Ollama + PydanticAI) ← Foundation for all AI
    ↓
Phase 2 (Steward)
    ↓
Phase 3 (Butler/Tatlock)
    ↓
Phase 4 (Expert Agents) ← Phase 7 (MCP) can enhance
    ↓
Phase 5 (Database/Multi-Tenancy) ← Can be deferred
    ↓
Phase 6 (Extended Services) → Phase 8 (Advanced Memory)
    ↓
Phase 9 (Extended Staff) → Phase 10 (UX) → Phase 11 (Production)
```

**Critical Path**: Phases 1 → 2 → 3 → 4 must be sequential
**Can Be Deferred**: Phase 5 (Database) until you need persistence
**Parallel Opportunities**: Phase 6 and 7 can overlap; Phase 9 and 10 ongoing

---

## Overall Timeline Estimate

**Minimum Viable Household** (Phases 1-4): **15-20 weeks**
- Working Steward → Butler → Expert Agents with real LLM
- In-memory state (no persistence needed yet)
- Core household functional

**With Persistence** (Phases 1-5): **18-24 weeks**
- Add database and multi-tenancy
- Conversations survive restarts
- Multiple users supported

**Full-Featured System** (Phases 1-9): **35-45 weeks**
- All services integrated
- Advanced memory and context
- Extended household staff

**Production-Ready** (All phases): **40-50 weeks**
- Polished UX
- Hardened for homelab deployment
- Fully documented

*Note: Timeline assumes consistent part-time development effort*

---

## Success Metrics

### Technical
- System implements PHILOSOPHY.md patterns
- All household roles functional
- Multi-tenant isolation verified
- Real-time reasoning transparency working
- MCP integration complete

### User Experience
- Tatlock feels like interacting with a butler
- Wait times are transparent and acceptable
- Expert agents provide value in their domains
- System is reliable and trustworthy

### Architecture
- Clean separation between household roles
- Easy to add new agents/tools
- Model efficiency (base model stays loaded)
- Scales to household + friends usage

---

## Risk Management

### High Risk Items
1. **PydanticAI + Ollama integration complexity**
   - Mitigation: Prototype early, iterate on connection layer

2. **Multi-agent coordination complexity**
   - Mitigation: Start simple, add coordination gradually

3. **Model performance on homelab hardware**
   - Mitigation: Model selection, quantization, optimization

4. **Prompt engineering for personality consistency**
   - Mitigation: Extensive testing, user feedback, iteration

### Medium Risk Items
- MCP protocol adoption and tooling maturity
- Vector embedding quality for memory
- Home automation integration variability
- User authentication security

---

## Next Steps

1. **Immediate**: Commit model name fix (Tatlock)
2. **Week 1-2**: Begin Phase 1 (PostgreSQL + multi-tenancy design)
3. **Week 3**: Parallel prototype of Steward agent
4. **Ongoing**: Update this roadmap as we learn

---

**Document Status**: Active planning document
**Created**: 2025-12-06
**Last Updated**: 2025-12-06