jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

docs: update README and roadmap for v1.2.0
Build and Push / build (release) Failing after 1m47s
Details

Browse Source 
README.md:
- Add household staff table with current status
- Update requirements to list external services
- Add Redis, Qdrant to configuration section
- Update project structure with new modules
- Update version to 1.2.0

IMPLEMENTATION_ROADMAP.md:
- Update current state to v1.2.0
- Mark Phase 2 (Steward) as complete
- Mark Phase 3 (Butler coordination) as complete
- Update Phase 4 with Librarian and Biographer complete
- Mark Phase 6 (Services) as complete
- Update Phase 8 (Memory) with completed items
- Update next steps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-13 19:29:30 +01:00
co-authored by Claude Opus 4.5
parent 7426dd1ac3
commit 7dd2c20e76
2 changed files with 208 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_ROADMAP.md
+124 -87
View File
@@ -4,35 +4,37 @@
This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system.
## Current State (v0.1.1+ - Phase 1 Mostly Complete)
## Current State (v1.2.0 - Phase F Complete)
**What we have**:
- ✅ **The Orchestrator** - FastAPI infrastructure layer
- OpenAI-compatible API endpoints (Responses API + Chat Completions)
- Streaming coordination and conversation management
- Response format with reasoning support
- Test infrastructure (131 tests, 81.78% coverage)
- ✅ **Tatlock Agent** - Real PydanticAI integration
- Connected to Ollama (mistral-nemo:latest)
- British butler personality with research mindset
- Streaming responses with reasoning
- Tool calling framework functional
- ✅ **Permanent Tools**
- Calculator (safe mathematical expressions)
- Date/Time toolkit (current time, relative dates, time differences)
- Web search (SearXNG integration)
- Test infrastructure (~400 tests)
- ✅ **Two-Tier Architecture**
- The Steward analyzes requests and recommends capabilities
- Tatlock coordinates execution with scoped tools
- Real-time streaming of analysis and reasoning
- ✅ **Household Staff**
- **Tatlock** (Butler): Primary interface with witty personality
- **The Steward**: Request analysis and capability recommendation
- **The Librarian**: Research via library-desk HybridRAG + wiki
- **The Biographer**: User memory, profiles, preferences, semantic recall
- ✅ **Core Tools**
- Calculator, Date/Time toolkit, Web search (SearXNG)
- ✅ **Memory System**
- Direct access layer (memory_service) for fast lookups
- Vector storage (Qdrant) for semantic recall
- Session cache (Redis) with 24h TTL
- Multi-tenancy via ContextVar
- ✅ Mock agent (lorem-tester for testing)
- ✅ Agent interface abstraction
**What we need**:
- **The Household** - Full multi-agent coordination:
- The Steward (first-tier request analysis)
- Tatlock coordination layer (expert agent delegation)
- Expert household staff agents (Librarian, Developer, Handyman, etc.)
- Multi-tenant database architecture
- Containerized service ecosystem
- More household staff (Developer, Secretary, Handyman, Housekeeper)
- MCP (Model Context Protocol) integration
- Dynamic model switching for specialized tasks
- Full multi-tenant database (PostgreSQL)
---
@@ -359,15 +361,18 @@ User Request → Orchestrator → Steward Analysis → Recommendations → Tatlo
### Success Criteria
- [ ] **Steward analyzes incoming requests** using PydanticAI agent
- [ ] **Produces structured recommendations** (tools, agents, reasoning)
- [ ] **Recommendations formatted as prepended note** to Tatlock
- [ ] **Tool registry is queryable and extensible** via clean API
- [ ] **Steward output visible in reasoning stream** for transparency
- [ ] **Only recommended tools available** to Tatlock (scoped context)
- [ ] **Base model stays loaded** between Steward and Tatlock calls
- [ ] **Recommendations are accurate** (not over/under-inclusive)
- [ ] **Integration tests pass** for full Steward → Tatlock flow
- [x] **Steward analyzes incoming requests** using PydanticAI agent
- [x] **Produces structured recommendations** (tools, agents, reasoning)
- [x] **Recommendations formatted as prepended note** to Tatlock
- [x] **Tool registry is queryable and extensible** via clean API
- [x] **Steward output visible in reasoning stream** for transparency
- [x] **Only recommended tools available** to Tatlock (scoped context)
- [x] **Base model stays loaded** between Steward and Tatlock calls
- [x] **Recommendations are accurate** (not over/under-inclusive)
- [x] **Integration tests pass** for full Steward → Tatlock flow
### Status
**✅ COMPLETE** (v0.2.5)
### Performance Targets
@@ -435,12 +440,15 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
- Wait time transparency
### Success Criteria
- [ ] Tatlock receives enriched requests (user + Steward notes)
- [ ] Only recommended tools are available
- [ ] Tatlock coordinates multiple tool calls
- [ ] All actions streamed to reasoning output
- [ ] Responses have consistent personality
- [ ] Synthesizes multi-source results coherently
- [x] Tatlock receives enriched requests (user + Steward notes)
- [x] Only recommended tools are available
- [x] Tatlock coordinates multiple tool calls
- [x] All actions streamed to reasoning output
- [x] Responses have consistent personality
- [x] Synthesizes multi-source results coherently
### Status
**✅ COMPLETE** (v1.1.0)
### Estimated Effort
**4-5 weeks** - Complex coordination logic
@@ -453,39 +461,44 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
### Priority Expert Agents
1. **The Librarian** (Research & Knowledge Management) ⭐ **Priority**
- Research assistance and synthesis
- Automatic research dossier generation
- Knowledge base queries and organization
- Reference management
- Wiki integration (future: dedicated wiki container)
- Mind map maintenance (future)
- *Rationale: Helps guide development priorities through better research*
1. **The Librarian** (Research & Knowledge Management) ✅ **COMPLETE** (v1.1.0)
- Research assistance via library-desk HybridRAG
- Wiki page management (search, create, update)
- Semantic vector search
- Knowledge graph queries
- Dossier browsing
2. **The Developer** (Software Development)
2. **The Biographer** (User Memory) ✅ **COMPLETE** (v1.2.0)
- User profile management (name, location, timezone)
- Preference storage (units, theme)
- Semantic memory recall ("What car do I drive?")
- Fact storage from conversations
- Session context caching
3. **The Developer** (Software Development) 🔜 **Planned**
- Code generation assistance
- Debugging support
- Documentation generation
- Architecture guidance
- *Rationale: Directly supports building the system itself*
3. **The Handyman** (System Maintenance)
4. **The Handyman** (System Maintenance) 🔜 **Planned**
- System status queries
- Log analysis
- Basic troubleshooting
- Infrastructure monitoring
4. **The Secretary** (Scheduling & Organization)
- Calendar integration (placeholder)
- Task management (placeholder)
5. **The Secretary** (Scheduling & Organization) 🔜 **Planned**
- Calendar integration
- Task management
- Reminder system
- Schedule conflict detection
5. **The Housekeeper** (Home Automation)
6. **The Housekeeper** (Home Automation) 🔜 **Planned**
- Home Assistant integration
- Device control interface
- Status queries
- Automation triggers
- Environmental monitoring
### Each Agent Includes
- Specialized prompt and personality
@@ -494,12 +507,15 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
- Integration with Butler orchestration
### Success Criteria
- [ ] Each agent implemented as separate module
- [ ] Agents callable via tool framework
- [ ] Agents use specialized prompts
- [ ] Results integrate cleanly with Butler
- [x] Each agent implemented as separate module
- [x] Agents callable via tool framework
- [x] Agents use specialized prompts
- [x] Results integrate cleanly with Butler
- [ ] Can invoke specialized models (e.g., Codestral for Developer)
### Status
**🔶 PARTIAL** - Librarian and Biographer complete, others planned
### Estimated Effort
**6-8 weeks** - Parallel development possible
@@ -556,31 +572,37 @@ The core orchestration (Steward → Butler → Experts) can work entirely with i
### Services to Integrate
1. **Redis (Memory & Caching)**
- Docker compose setup
- Conversation cache
- Short-term memory
- Session management
1. **Redis (Memory & Caching)** ✅ **COMPLETE** (v1.2.0)
- Benchmark storage (db=1)
- Memory cache for sessions (db=2)
- 24h TTL for session context
- Recent entities tracking
3. **Qdrant (Vector Storage)**
- Docker compose setup
- Long-term memory embeddings
- Semantic search
- Conversation history vectors
2. **Qdrant (Vector Storage)** ✅ **COMPLETE** (v1.2.0)
- Per-user memory collections
- 768-dim nomic-embed-text vectors
- Semantic search for recall
- Type-based filtering
4. **SearxNG (Web Search)**
- Docker compose setup
3. **SearxNG (Web Search)** ✅ **COMPLETE** (v0.2.0)
- Search tool integration
- Result processing
- Privacy-preserving queries
4. **library-desk (Research API)** ✅ **COMPLETE** (v1.1.0)
- HybridRAG search
- Wiki management
- Knowledge graph queries
### Success Criteria
- [ ] All services defined in docker-compose.yml
- [ ] Services communicate correctly
- [ ] Tatlock can invoke web search
- [ ] Redis used for session data
- [ ] Qdrant stores conversation embeddings
- [ ] Ollama serves the base model
- [x] Services communicate correctly
- [x] Tatlock can invoke web search
- [x] Redis used for session data
- [x] Qdrant stores user memories
- [x] Ollama serves the base model
### Status
**✅ COMPLETE** - All core services integrated
### Estimated Effort
**3-4 weeks** - Infrastructure setup
@@ -629,33 +651,48 @@ The core orchestration (Steward → Butler → Experts) can work entirely with i
### Deliverables
1. **Long-Term Memory**
- Conversation embedding pipeline
- Semantic search over history
- Memory consolidation
- Relevance ranking
1. **Long-Term Memory** ✅ **COMPLETE** (v1.2.0 - Phase F)
- Memory service for direct key-based access
- Qdrant vector storage for semantic recall
- Embedding via nomic-embed-text
- The Biographer agent for memory management
2. **Context Management**
2. **Session Memory** ✅ **COMPLETE** (v1.2.0)
- Redis session cache with 24h TTL
- Recent entities tracking
- Conversation context preservation
- Multi-tenancy via ContextVar
3. **Steward Integration** ✅ **COMPLETE** (v1.2.0)
- Memory pre-fetch during request analysis
- Profile/preferences included in context
- Keyword-based context determination
4. **Context Management** 🔜 **Future**
- Smart context window trimming
- Conversation branching
- Topic tracking
- Memory retrieval integration
3. **Personalization**
5. **Personalization** 🔜 **Future**
- User preference learning
- Interaction pattern analysis
- Adaptive responses
- Custom agent personalities per user
### Success Criteria
- [x] User facts stored in Qdrant with semantic search
- [x] Profile and preferences accessible via memory_service
- [x] Session context cached in Redis
- [x] User preferences affect responses (via Steward pre-fetch)
- [ ] Conversations automatically embedded to Qdrant
- [ ] Relevant history retrieved for new requests
- [ ] Context stays within model limits
- [ ] User preferences affect responses
- [ ] Memory improves over time
- [ ] Memory improves over time (learning from interactions)
### Status
**🔶 PARTIAL** - Core memory system complete, advanced features planned
### Estimated Effort
**4-5 weeks** - AI/ML heavy
**4-5 weeks** - AI/ML heavy (remaining work)
---
@@ -871,13 +908,13 @@ Phase 9 (Extended Staff) → Phase 10 (UX) → Phase 11 (Production)
## Next Steps
1. **Immediate**: Commit model name fix (Tatlock)
2. **Week 1-2**: Begin Phase 1 (PostgreSQL + multi-tenancy design)
3. **Week 3**: Parallel prototype of Steward agent
4. **Ongoing**: Update this roadmap as we learn
1. **Priority**: Implement The Developer agent for code assistance
2. **Integration**: Add Home Assistant integration for The Housekeeper
3. **Calendar**: Integrate scheduling service for The Secretary
4. **Ongoing**: Add more household staff as needed
---
**Document Status**: Active planning document
**Created**: 2025-12-06
**Last Updated**: 2025-12-06
**Last Updated**: 2025-12-13
README.md
+84 -40
View File
@@ -6,12 +6,25 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
## Current Status
- ✅ **Production-ready testing API** with OpenAI Responses API format
- ✅ **Production-ready API** with OpenAI Responses API format
- ✅ **Open WebUI integration** with reasoning bubbles (`<think>` tags)
- ✅ **Conversation history** with auto-generated IDs and context management
- ✅ **Tatlock PydanticAI Agent** - Real LLM integration with Ollama + permanent tools
- ✅ **Permanent Tools** - Calculator, date/time toolkit, web search (SearXNG)
- ✅ **Comprehensive testing** - 131 tests, 81.78% coverage
- ✅ **Two-tier architecture** - The Steward analyzes requests, Tatlock coordinates execution
- ✅ **Multi-agent coordination** - Expert household staff for specialized tasks
- ✅ **Memory system** - User profile, preferences, and semantic recall
- ✅ **Comprehensive testing** - 399 tests with good coverage
### The Household Staff
| Agent | Role | Status |
|-------|------|--------|
| **Tatlock** | The Butler - Primary interface with witty personality | ✅ Active |
| **The Steward** | Request analysis and capability recommendation | ✅ Active |
| **The Librarian** | Research, wiki management, knowledge synthesis | ✅ Active |
| **The Biographer** | User memory - profiles, preferences, facts | ✅ Active |
| **The Developer** | Code assistance, debugging, architecture | 🔜 Planned |
| **The Secretary** | Scheduling, calendars, reminders | 🔜 Planned |
| **The Handyman** | System administration, monitoring | 🔜 Planned |
| **The Housekeeper** | Home automation (Home Assistant) | 🔜 Planned |
## Features
@@ -45,24 +58,27 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
- Error triggers for testing (rate_limit, context_overflow)
- **Tatlock**: Real PydanticAI agent with butler personality
- **LLM Backend**: Ollama (mistral-nemo:latest)
- **LLM Backend**: Ollama (mistral-nemo:latest by default)
- **Personality**: Witty British butler, research-oriented
- **Permanent Tools**:
- **Calculator**: Safe mathematical expression evaluation (arithmetic, algebra, trigonometry, logarithms)
- **Date/Time Toolkit**: Current time, relative dates ("1 week ago"), time differences
- **Core Tools**:
- **Calculator**: Safe mathematical expression evaluation
- **Date/Time Toolkit**: Current time, relative dates, time differences
- **Web Search**: Privacy-preserving search via SearXNG
- **Capabilities**: Streaming, reasoning, tool calling
- **Phase**: Phase 1 - Basic Integration (full household coordination coming in future phases)
- **Household Coordination**:
- **The Steward**: Analyzes requests and recommends capabilities
- **The Librarian**: Research via library-desk HybridRAG + wiki
- **The Biographer**: User memory and preference management
- **Capabilities**: Streaming, reasoning, tool calling, multi-agent delegation
## Requirements
- Python 3.12+ (Python 3.12.11 recommended)
- **Ollama** (for Tatlock agent): Running locally or network-accessible
- Download: https://ollama.ai/
- Model: `ollama pull mistral-nemo:latest`
- **SearXNG** (for web search tool): Optional but recommended
- Docker: `docker run -d -p 8087:8080 searxng/searxng`
- Or use public instance (less private)
- **External Services** (must be running separately):
- **Ollama**: LLM inference (mistral-nemo:latest, nomic-embed-text)
- **Redis**: Caching and session memory
- **Qdrant**: Vector storage for The Biographer's memory
- **SearXNG**: Web search (optional)
- **library-desk**: Research API for The Librarian (optional)
## Quick Start
@@ -251,15 +267,18 @@ Interactive documentation available at:
# Run all tests
pytest
# Run unit tests only (no external services needed)
pytest --ignore=tests/e2e --ignore=tests/integration
# Run with coverage
pytest --cov=src --cov-report=term-missing
# Current: 131 tests, 81.78% coverage
# Current: ~400 tests
```
**Test Categories:**
- Unit tests: Agent tools, streaming, schemas
- Integration tests: Full API stack with real Ollama calls
- Unit tests: Agent tools, capabilities, schemas, memory service
- Integration tests: Full API stack with real Ollama
- End-to-end tests: Chat completions, responses API
## Deployment
@@ -291,9 +310,25 @@ API_PORT=8000
# Ollama Configuration
OLLAMA_HOST=http://localhost:11434
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
OLLAMA_TIMEOUT=120
# SearXNG Configuration (for web search tool)
# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_MEMORY_DB=2
REDIS_MEMORY_TTL_HOURS=24
# Qdrant Configuration (for memory)
QDRANT_HOST=localhost
QDRANT_PORT=6333
QDRANT_EMBEDDING_DIM=768
# Library-desk Configuration (for The Librarian)
LIBRARY_DESK_HOST=http://localhost:8089
LIBRARY_DESK_TIMEOUT=60
# SearXNG Configuration (for web search)
SEARXNG_HOST=http://localhost:8087
SEARXNG_TIMEOUT=30
@@ -339,23 +374,32 @@ See `.env.example` for full configuration options.
```
tatlock/
├── src/
│ ├── agents/ # Agent interface and implementations
│ │ ├── base.py # AgentInterface abstract class
│ │ ├── lorem_tester.py # Mock agent for testing
│ │ ├── tatlock.py # Real PydanticAI butler agent
│ │ ├── tools.py # Permanent tools (calculator, date/time, search)
│ │ └── registry.py # Model registry
│ ├── responses/ # Responses API (primary endpoint)
│ ├── chat/ # Chat Completions wrapper
│ ├── models/ # Models listing
│ ├── core/ # Shared utilities and config
│ └── main.py # Application entry point
├── tests/ # Comprehensive test suite (131 tests)
├── AGENTS.md # LLM agent development guidelines
├── PHILOSOPHY.md # System vision and architecture
├── IMPLEMENTATION_ROADMAP.md # Development phases
├── CHANGELOG.md # Version history
└── README.md # This file
│ ├── agents/ # Agent implementations
│ │ ├── biographer/ # The Biographer - memory management
│ │ ├── librarian/ # The Librarian - research & wiki
│ │ ├── steward/ # The Steward - request analysis
│ │ ├── tatlock_core/ # Core butler tools
│ │ ├── tatlock.py # Tatlock PydanticAI agent
│ │ ├── coordination.py # Multi-agent coordination
│ │ ├── delegation.py # Expert delegation wrappers
│ │ └── protocol.py # Agent communication protocol
│ ├── responses/ # Responses API (primary endpoint)
│ ├── chat/ # Chat Completions wrapper
│ ├── models/ # Models listing
│ ├── core/ # Shared infrastructure
│ │ ├── config.py # Configuration management
│ │ ├── context.py # Request context (ContextVar)
│ │ ├── memory_service.py # Direct memory access
│ │ ├── memory_cache.py # Redis session cache
│ │ ├── embeddings.py # Ollama embedding client
│ │ ├── qdrant.py # Vector database client
│ │ └── multi_tenancy.py # User isolation utilities
│ └── main.py # Application entry point
├── tests/ # Comprehensive test suite
├── PHILOSOPHY.md # System vision and architecture
├── IMPLEMENTATION_ROADMAP.md # Development phases
├── CHANGELOG.md # Version history
└── README.md # This file
```
## Development
@@ -388,8 +432,8 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md
## Version
Current version: **0.2.5** - Phase 2: The Steward (Two-Tier Architecture)
Current version: **1.2.0** - Phase F: Memory System (The Biographer)
---
**Note**: This is a production-ready testing API with mock responses. The architecture is designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).
**Note**: Tatlock is a production-ready homelab butler. All household staff use PydanticAI with Ollama for local LLM inference.