docs: update README and roadmap for v1.2.0

README.md:
- Add household staff table with current status
- Update requirements to list external services
- Add Redis, Qdrant to configuration section
- Update project structure with new modules
- Update version to 1.2.0

IMPLEMENTATION_ROADMAP.md:
- Update current state to v1.2.0
- Mark Phase 2 (Steward) as complete
- Mark Phase 3 (Butler coordination) as complete
- Update Phase 4 with Librarian and Biographer complete
- Mark Phase 6 (Services) as complete
- Update Phase 8 (Memory) with completed items
- Update next steps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

IMPLEMENTATION_ROADMAP.md

@@ -4,35 +4,37 @@
 
 This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system.
 
-## Current State (v0.1.1+ - Phase 1 Mostly Complete)
+## Current State (v1.2.0 - Phase F Complete)
 
 **What we have**:
 - ✅ **The Orchestrator** - FastAPI infrastructure layer
   - OpenAI-compatible API endpoints (Responses API + Chat Completions)
   - Streaming coordination and conversation management
   - Response format with reasoning support
-  - Test infrastructure (131 tests, 81.78% coverage)
+  - Test infrastructure (~400 tests)
-- ✅ **Tatlock Agent** - Real PydanticAI integration
+- ✅ **Two-Tier Architecture**
-  - Connected to Ollama (mistral-nemo:latest)
+  - The Steward analyzes requests and recommends capabilities
-  - British butler personality with research mindset
+  - Tatlock coordinates execution with scoped tools
-  - Streaming responses with reasoning
+  - Real-time streaming of analysis and reasoning
-  - Tool calling framework functional
+- ✅ **Household Staff**
-- ✅ **Permanent Tools**
+  - **Tatlock** (Butler): Primary interface with witty personality
-  - Calculator (safe mathematical expressions)
+  - **The Steward**: Request analysis and capability recommendation
-  - Date/Time toolkit (current time, relative dates, time differences)
+  - **The Librarian**: Research via library-desk HybridRAG + wiki
-  - Web search (SearXNG integration)
+  - **The Biographer**: User memory, profiles, preferences, semantic recall
+- ✅ **Core Tools**
+  - Calculator, Date/Time toolkit, Web search (SearXNG)
+- ✅ **Memory System**
+  - Direct access layer (memory_service) for fast lookups
+  - Vector storage (Qdrant) for semantic recall
+  - Session cache (Redis) with 24h TTL
+  - Multi-tenancy via ContextVar
 - ✅ Mock agent (lorem-tester for testing)
-- ✅ Agent interface abstraction
 
 **What we need**:
-- **The Household** - Full multi-agent coordination:
+- More household staff (Developer, Secretary, Handyman, Housekeeper)
-  - The Steward (first-tier request analysis)
-  - Tatlock coordination layer (expert agent delegation)
-  - Expert household staff agents (Librarian, Developer, Handyman, etc.)
-- Multi-tenant database architecture
-- Containerized service ecosystem
 - MCP (Model Context Protocol) integration
 - Dynamic model switching for specialized tasks
+- Full multi-tenant database (PostgreSQL)
 
 ---
 
@@ -359,15 +361,18 @@ User Request → Orchestrator → Steward Analysis → Recommendations → Tatlo
 
 ### Success Criteria
 
-- [ ] **Steward analyzes incoming requests** using PydanticAI agent
+- [x] **Steward analyzes incoming requests** using PydanticAI agent
-- [ ] **Produces structured recommendations** (tools, agents, reasoning)
+- [x] **Produces structured recommendations** (tools, agents, reasoning)
-- [ ] **Recommendations formatted as prepended note** to Tatlock
+- [x] **Recommendations formatted as prepended note** to Tatlock
-- [ ] **Tool registry is queryable and extensible** via clean API
+- [x] **Tool registry is queryable and extensible** via clean API
-- [ ] **Steward output visible in reasoning stream** for transparency
+- [x] **Steward output visible in reasoning stream** for transparency
-- [ ] **Only recommended tools available** to Tatlock (scoped context)
+- [x] **Only recommended tools available** to Tatlock (scoped context)
-- [ ] **Base model stays loaded** between Steward and Tatlock calls
+- [x] **Base model stays loaded** between Steward and Tatlock calls
-- [ ] **Recommendations are accurate** (not over/under-inclusive)
+- [x] **Recommendations are accurate** (not over/under-inclusive)
-- [ ] **Integration tests pass** for full Steward → Tatlock flow
+- [x] **Integration tests pass** for full Steward → Tatlock flow
+
+### Status
+**✅ COMPLETE** (v0.2.5)
 
 ### Performance Targets
 
@@ -435,12 +440,15 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
    - Wait time transparency
 
 ### Success Criteria
-- [ ] Tatlock receives enriched requests (user + Steward notes)
+- [x] Tatlock receives enriched requests (user + Steward notes)
-- [ ] Only recommended tools are available
+- [x] Only recommended tools are available
-- [ ] Tatlock coordinates multiple tool calls
+- [x] Tatlock coordinates multiple tool calls
-- [ ] All actions streamed to reasoning output
+- [x] All actions streamed to reasoning output
-- [ ] Responses have consistent personality
+- [x] Responses have consistent personality
-- [ ] Synthesizes multi-source results coherently
+- [x] Synthesizes multi-source results coherently
+
+### Status
+**✅ COMPLETE** (v1.1.0)
 
 ### Estimated Effort
 **4-5 weeks** - Complex coordination logic
@@ -453,39 +461,44 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
 
 ### Priority Expert Agents
 
-1. **The Librarian** (Research & Knowledge Management) ⭐ **Priority**
+1. **The Librarian** (Research & Knowledge Management) ✅ **COMPLETE** (v1.1.0)
-   - Research assistance and synthesis
+   - Research assistance via library-desk HybridRAG
-   - Automatic research dossier generation
+   - Wiki page management (search, create, update)
-   - Knowledge base queries and organization
+   - Semantic vector search
-   - Reference management
+   - Knowledge graph queries
-   - Wiki integration (future: dedicated wiki container)
+   - Dossier browsing
-   - Mind map maintenance (future)
-   - *Rationale: Helps guide development priorities through better research*
 
-2. **The Developer** (Software Development)
+2. **The Biographer** (User Memory) ✅ **COMPLETE** (v1.2.0)
+   - User profile management (name, location, timezone)
+   - Preference storage (units, theme)
+   - Semantic memory recall ("What car do I drive?")
+   - Fact storage from conversations
+   - Session context caching
+
+3. **The Developer** (Software Development) 🔜 **Planned**
    - Code generation assistance
    - Debugging support
    - Documentation generation
    - Architecture guidance
    - *Rationale: Directly supports building the system itself*
 
-3. **The Handyman** (System Maintenance)
+4. **The Handyman** (System Maintenance) 🔜 **Planned**
    - System status queries
    - Log analysis
    - Basic troubleshooting
    - Infrastructure monitoring
 
-4. **The Secretary** (Scheduling & Organization)
+5. **The Secretary** (Scheduling & Organization) 🔜 **Planned**
-   - Calendar integration (placeholder)
+   - Calendar integration
-   - Task management (placeholder)
+   - Task management
    - Reminder system
    - Schedule conflict detection
 
-5. **The Housekeeper** (Home Automation)
+6. **The Housekeeper** (Home Automation) 🔜 **Planned**
+   - Home Assistant integration
    - Device control interface
    - Status queries
    - Automation triggers
-   - Environmental monitoring
 
 ### Each Agent Includes
 - Specialized prompt and personality
@@ -494,12 +507,15 @@ The Steward is the foundation of the household architecture. Without it, we'd ne
 - Integration with Butler orchestration
 
 ### Success Criteria
-- [ ] Each agent implemented as separate module
+- [x] Each agent implemented as separate module
-- [ ] Agents callable via tool framework
+- [x] Agents callable via tool framework
-- [ ] Agents use specialized prompts
+- [x] Agents use specialized prompts
-- [ ] Results integrate cleanly with Butler
+- [x] Results integrate cleanly with Butler
 - [ ] Can invoke specialized models (e.g., Codestral for Developer)
 
+### Status
+**🔶 PARTIAL** - Librarian and Biographer complete, others planned
+
 ### Estimated Effort
 **6-8 weeks** - Parallel development possible
 
@@ -556,31 +572,37 @@ The core orchestration (Steward → Butler → Experts) can work entirely with i
 
 ### Services to Integrate
 
-1. **Redis (Memory & Caching)**
+1. **Redis (Memory & Caching)** ✅ **COMPLETE** (v1.2.0)
-   - Docker compose setup
+   - Benchmark storage (db=1)
-   - Conversation cache
+   - Memory cache for sessions (db=2)
-   - Short-term memory
+   - 24h TTL for session context
-   - Session management
+   - Recent entities tracking
 
-3. **Qdrant (Vector Storage)**
+2. **Qdrant (Vector Storage)** ✅ **COMPLETE** (v1.2.0)
-   - Docker compose setup
+   - Per-user memory collections
-   - Long-term memory embeddings
+   - 768-dim nomic-embed-text vectors
-   - Semantic search
+   - Semantic search for recall
-   - Conversation history vectors
+   - Type-based filtering
 
-4. **SearxNG (Web Search)**
+3. **SearxNG (Web Search)** ✅ **COMPLETE** (v0.2.0)
-   - Docker compose setup
    - Search tool integration
    - Result processing
    - Privacy-preserving queries
 
+4. **library-desk (Research API)** ✅ **COMPLETE** (v1.1.0)
+   - HybridRAG search
+   - Wiki management
+   - Knowledge graph queries
+
 ### Success Criteria
-- [ ] All services defined in docker-compose.yml
+- [x] Services communicate correctly
-- [ ] Services communicate correctly
+- [x] Tatlock can invoke web search
-- [ ] Tatlock can invoke web search
+- [x] Redis used for session data
-- [ ] Redis used for session data
+- [x] Qdrant stores user memories
-- [ ] Qdrant stores conversation embeddings
+- [x] Ollama serves the base model
-- [ ] Ollama serves the base model
+
+### Status
+**✅ COMPLETE** - All core services integrated
 
 ### Estimated Effort
 **3-4 weeks** - Infrastructure setup
@@ -629,33 +651,48 @@ The core orchestration (Steward → Butler → Experts) can work entirely with i
 
 ### Deliverables
 
-1. **Long-Term Memory**
+1. **Long-Term Memory** ✅ **COMPLETE** (v1.2.0 - Phase F)
-   - Conversation embedding pipeline
+   - Memory service for direct key-based access
-   - Semantic search over history
+   - Qdrant vector storage for semantic recall
-   - Memory consolidation
+   - Embedding via nomic-embed-text
-   - Relevance ranking
+   - The Biographer agent for memory management
 
-2. **Context Management**
+2. **Session Memory** ✅ **COMPLETE** (v1.2.0)
+   - Redis session cache with 24h TTL
+   - Recent entities tracking
+   - Conversation context preservation
+   - Multi-tenancy via ContextVar
+
+3. **Steward Integration** ✅ **COMPLETE** (v1.2.0)
+   - Memory pre-fetch during request analysis
+   - Profile/preferences included in context
+   - Keyword-based context determination
+
+4. **Context Management** 🔜 **Future**
    - Smart context window trimming
    - Conversation branching
    - Topic tracking
    - Memory retrieval integration
 
-3. **Personalization**
+5. **Personalization** 🔜 **Future**
    - User preference learning
    - Interaction pattern analysis
    - Adaptive responses
    - Custom agent personalities per user
 
 ### Success Criteria
+- [x] User facts stored in Qdrant with semantic search
+- [x] Profile and preferences accessible via memory_service
+- [x] Session context cached in Redis
+- [x] User preferences affect responses (via Steward pre-fetch)
 - [ ] Conversations automatically embedded to Qdrant
-- [ ] Relevant history retrieved for new requests
+- [ ] Memory improves over time (learning from interactions)
-- [ ] Context stays within model limits
+
-- [ ] User preferences affect responses
+### Status
-- [ ] Memory improves over time
+**🔶 PARTIAL** - Core memory system complete, advanced features planned
 
 ### Estimated Effort
-**4-5 weeks** - AI/ML heavy
+**4-5 weeks** - AI/ML heavy (remaining work)
 
 ---
 
@@ -871,13 +908,13 @@ Phase 9 (Extended Staff) → Phase 10 (UX) → Phase 11 (Production)
 
 ## Next Steps
 
-1. **Immediate**: Commit model name fix (Tatlock)
+1. **Priority**: Implement The Developer agent for code assistance
-2. **Week 1-2**: Begin Phase 1 (PostgreSQL + multi-tenancy design)
+2. **Integration**: Add Home Assistant integration for The Housekeeper
-3. **Week 3**: Parallel prototype of Steward agent
+3. **Calendar**: Integrate scheduling service for The Secretary
-4. **Ongoing**: Update this roadmap as we learn
+4. **Ongoing**: Add more household staff as needed
 
 ---
 
 **Document Status**: Active planning document
 **Created**: 2025-12-06
-**Last Updated**: 2025-12-06
+**Last Updated**: 2025-12-13

README.md

@@ -6,12 +6,25 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
 
 ## Current Status
 
-- ✅ **Production-ready testing API** with OpenAI Responses API format
+- ✅ **Production-ready API** with OpenAI Responses API format
 - ✅ **Open WebUI integration** with reasoning bubbles (`<think>` tags)
-- ✅ **Conversation history** with auto-generated IDs and context management
+- ✅ **Two-tier architecture** - The Steward analyzes requests, Tatlock coordinates execution
-- ✅ **Tatlock PydanticAI Agent** - Real LLM integration with Ollama + permanent tools
+- ✅ **Multi-agent coordination** - Expert household staff for specialized tasks
-- ✅ **Permanent Tools** - Calculator, date/time toolkit, web search (SearXNG)
+- ✅ **Memory system** - User profile, preferences, and semantic recall
-- ✅ **Comprehensive testing** - 131 tests, 81.78% coverage
+- ✅ **Comprehensive testing** - 399 tests with good coverage
+
+### The Household Staff
+
+| Agent | Role | Status |
+|-------|------|--------|
+| **Tatlock** | The Butler - Primary interface with witty personality | ✅ Active |
+| **The Steward** | Request analysis and capability recommendation | ✅ Active |
+| **The Librarian** | Research, wiki management, knowledge synthesis | ✅ Active |
+| **The Biographer** | User memory - profiles, preferences, facts | ✅ Active |
+| **The Developer** | Code assistance, debugging, architecture | 🔜 Planned |
+| **The Secretary** | Scheduling, calendars, reminders | 🔜 Planned |
+| **The Handyman** | System administration, monitoring | 🔜 Planned |
+| **The Housekeeper** | Home automation (Home Assistant) | 🔜 Planned |
 
 ## Features
 
@@ -45,24 +58,27 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
   - Error triggers for testing (rate_limit, context_overflow)
 
 - **Tatlock**: Real PydanticAI agent with butler personality
-  - **LLM Backend**: Ollama (mistral-nemo:latest)
+  - **LLM Backend**: Ollama (mistral-nemo:latest by default)
   - **Personality**: Witty British butler, research-oriented
-  - **Permanent Tools**:
+  - **Core Tools**:
-    - **Calculator**: Safe mathematical expression evaluation (arithmetic, algebra, trigonometry, logarithms)
+    - **Calculator**: Safe mathematical expression evaluation
-    - **Date/Time Toolkit**: Current time, relative dates ("1 week ago"), time differences
+    - **Date/Time Toolkit**: Current time, relative dates, time differences
     - **Web Search**: Privacy-preserving search via SearXNG
-  - **Capabilities**: Streaming, reasoning, tool calling
+  - **Household Coordination**:
-  - **Phase**: Phase 1 - Basic Integration (full household coordination coming in future phases)
+    - **The Steward**: Analyzes requests and recommends capabilities
+    - **The Librarian**: Research via library-desk HybridRAG + wiki
+    - **The Biographer**: User memory and preference management
+  - **Capabilities**: Streaming, reasoning, tool calling, multi-agent delegation
 
 ## Requirements
 
 - Python 3.12+ (Python 3.12.11 recommended)
-- **Ollama** (for Tatlock agent): Running locally or network-accessible
+- **External Services** (must be running separately):
-  - Download: https://ollama.ai/
+  - **Ollama**: LLM inference (mistral-nemo:latest, nomic-embed-text)
-  - Model: `ollama pull mistral-nemo:latest`
+  - **Redis**: Caching and session memory
-- **SearXNG** (for web search tool): Optional but recommended
+  - **Qdrant**: Vector storage for The Biographer's memory
-  - Docker: `docker run -d -p 8087:8080 searxng/searxng`
+  - **SearXNG**: Web search (optional)
-  - Or use public instance (less private)
+  - **library-desk**: Research API for The Librarian (optional)
 
 ## Quick Start
 
@@ -251,15 +267,18 @@ Interactive documentation available at:
 # Run all tests
 pytest
 
+# Run unit tests only (no external services needed)
+pytest --ignore=tests/e2e --ignore=tests/integration
+
 # Run with coverage
 pytest --cov=src --cov-report=term-missing
 
-# Current: 131 tests, 81.78% coverage
+# Current: ~400 tests
 ```
 
 **Test Categories:**
-- Unit tests: Agent tools, streaming, schemas
+- Unit tests: Agent tools, capabilities, schemas, memory service
-- Integration tests: Full API stack with real Ollama calls
+- Integration tests: Full API stack with real Ollama
 - End-to-end tests: Chat completions, responses API
 
 ## Deployment
@@ -291,9 +310,25 @@ API_PORT=8000
 # Ollama Configuration
 OLLAMA_HOST=http://localhost:11434
 OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
+OLLAMA_EMBEDDING_MODEL=nomic-embed-text
 OLLAMA_TIMEOUT=120
 
-# SearXNG Configuration (for web search tool)
+# Redis Configuration
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDIS_MEMORY_DB=2
+REDIS_MEMORY_TTL_HOURS=24
+
+# Qdrant Configuration (for memory)
+QDRANT_HOST=localhost
+QDRANT_PORT=6333
+QDRANT_EMBEDDING_DIM=768
+
+# Library-desk Configuration (for The Librarian)
+LIBRARY_DESK_HOST=http://localhost:8089
+LIBRARY_DESK_TIMEOUT=60
+
+# SearXNG Configuration (for web search)
 SEARXNG_HOST=http://localhost:8087
 SEARXNG_TIMEOUT=30
 
@@ -339,23 +374,32 @@ See `.env.example` for full configuration options.
 ```
 tatlock/
 ├── src/
-│   ├── agents/          # Agent interface and implementations
+│   ├── agents/              # Agent implementations
-│   │   ├── base.py      # AgentInterface abstract class
+│   │   ├── biographer/      # The Biographer - memory management
-│   │   ├── lorem_tester.py  # Mock agent for testing
+│   │   ├── librarian/       # The Librarian - research & wiki
-│   │   ├── tatlock.py   # Real PydanticAI butler agent
+│   │   ├── steward/         # The Steward - request analysis
-│   │   ├── tools.py     # Permanent tools (calculator, date/time, search)
+│   │   ├── tatlock_core/    # Core butler tools
-│   │   └── registry.py  # Model registry
+│   │   ├── tatlock.py       # Tatlock PydanticAI agent
-│   ├── responses/       # Responses API (primary endpoint)
+│   │   ├── coordination.py  # Multi-agent coordination
-│   ├── chat/            # Chat Completions wrapper
+│   │   ├── delegation.py    # Expert delegation wrappers
-│   ├── models/          # Models listing
+│   │   └── protocol.py      # Agent communication protocol
-│   ├── core/            # Shared utilities and config
+│   ├── responses/           # Responses API (primary endpoint)
-│   └── main.py          # Application entry point
+│   ├── chat/                # Chat Completions wrapper
-├── tests/               # Comprehensive test suite (131 tests)
+│   ├── models/              # Models listing
-├── AGENTS.md            # LLM agent development guidelines
+│   ├── core/                # Shared infrastructure
-├── PHILOSOPHY.md        # System vision and architecture
+│   │   ├── config.py        # Configuration management
-├── IMPLEMENTATION_ROADMAP.md  # Development phases
+│   │   ├── context.py       # Request context (ContextVar)
-├── CHANGELOG.md         # Version history
+│   │   ├── memory_service.py # Direct memory access
-└── README.md            # This file
+│   │   ├── memory_cache.py  # Redis session cache
+│   │   ├── embeddings.py    # Ollama embedding client
+│   │   ├── qdrant.py        # Vector database client
+│   │   └── multi_tenancy.py # User isolation utilities
+│   └── main.py              # Application entry point
+├── tests/                   # Comprehensive test suite
+├── PHILOSOPHY.md            # System vision and architecture
+├── IMPLEMENTATION_ROADMAP.md # Development phases
+├── CHANGELOG.md             # Version history
+└── README.md                # This file
 ```
 
 ## Development
@@ -388,8 +432,8 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md
 
 ## Version
 
-Current version: **0.2.5** - Phase 2: The Steward (Two-Tier Architecture)
+Current version: **1.2.0** - Phase F: Memory System (The Biographer)
 
 ---
 
-**Note**: This is a production-ready testing API with mock responses. The architecture is designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).
+**Note**: Tatlock is a production-ready homelab butler. All household staff use PydanticAI with Ollama for local LLM inference.