# Tatlock Enhancement Plan: Bidirectional Claude Integration

## Executive Summary

Implement a **bidirectional architecture** that:
1. **Superpowers Tatlock** by swapping Ollama→Claude backend (200k context, better reasoning, same butler personality)
2. **Exposes Tatlock as MCP server** for Claude instances on any device (phone, browser, desktop)

This gives you the flexibility to use whichever AI is best/most accessible at any moment.

## Key Insight: Blanket Backend Swap (Simpler Than Sidecar)

Instead of adding a Claude "Analyst" sidecar agent, **swap the underlying model for ALL agents**:

```
CURRENT:  TatlockAgent → OpenAIChatModel → OllamaProvider → Ollama (mistral-nemo)
PROPOSED: TatlockAgent → AnthropicModel → AnthropicProvider → Claude API
                      ↘ (fallback when offline) → OllamaProvider → Ollama
```

**Why this works:**
- PydanticAI natively supports Anthropic via `AnthropicModel` + `AnthropicProvider`
- The same `TATLOCK_SYSTEM_PROMPT` is passed to Claude - butler personality preserved
- Claude is **better** at following system prompts than mistral-nemo
- 200k context for ALL queries, not just "complex" ones
- Simpler architecture: no routing logic, no sidecar delegation

---

## Research Findings

### Industry Best Practices (2025-2026)

**MCP Protocol Updates** ([MCP Spec Updates June 2025](https://auth0.com/blog/mcp-specs-update-all-about-auth/)):
- Streamable HTTP replaced SSE (March 2025) - better for cloud deployment
- OAuth 2.0 required for remote servers - MCP servers are OAuth Resource Servers
- Tool Output Schemas now available - better structured data handling
- MCP Registry launched (Sept 2025) - community server discovery

**Community Patterns** ([Claude Code Router](https://github.com/musistudio/claude-code-router)):
- Task-based routing is becoming standard: route simple→local, complex→cloud
- Translation proxies bridge Anthropic Messages API ↔ OpenAI format
- Cost savings of up to 98% reported with smart routing

**Home Automation MCP** ([ha-mcp](https://github.com/homeassistant-ai/ha-mcp)):
- Production-ready MCP servers exist for Home Assistant
- Support Claude Code, Gemini CLI, Open WebUI, VSCode, Cursor
- Pattern: expose local tools securely to remote AI clients

**Remote MCP Access** ([mcp-remote](https://www.npmjs.com/package/mcp-remote)):
- Bridge local MCP servers to Claude Desktop/Browser via proxy
- Supports authentication headers for security
- Works with ngrok/Cloudflare Tunnel for HTTPS

---

## Recommended Architecture

```
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                    BIDIRECTIONAL TATLOCK-CLAUDE ARCHITECTURE                        │
├─────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                     │
│  ╔═══════════════════════════════════════════════════════════════════════════════╗  │
│  ║  SCENARIO A: Using Tatlock (Open WebUI, local apps)                           ║  │
│  ║  ─────────────────────────────────────────────────                            ║  │
│  ║                                                                               ║  │
│  ║    Request → Steward → Tatlock → Tools + Expert Delegation                    ║  │
│  ║                            │                                                  ║  │
│  ║                            ├─→ Librarian (Claude) → research, wiki, RAG       ║  │
│  ║                            ├─→ Biographer (Claude) → memory, preferences      ║  │
│  ║                            ├─→ Housekeeper (Claude) → home automation         ║  │
│  ║                            └─→ All powered by Claude with Ollama fallback     ║  │
│  ║                                                                               ║  │
│  ║    Butler personality preserved, 200k context for all queries                 ║  │
│  ╚═══════════════════════════════════════════════════════════════════════════════╝  │
│                                                                                     │
│  ╔═══════════════════════════════════════════════════════════════════════════════╗  │
│  ║  SCENARIO B: Using Claude.ai / Claude Desktop / Phone                         ║  │
│  ║  ────────────────────────────────────────────────────                         ║  │
│  ║                                                                               ║  │
│  ║    Claude ──[MCP over HTTPS]──► Tatlock MCP Server → Household Tools          ║  │
│  ║                                        │                                      ║  │
│  ║                                        ├─→ calculator, datetime               ║  │
│  ║                                        ├─→ web_search, wiki_search            ║  │
│  ║                                        ├─→ hybrid_search (RAG)                ║  │
│  ║                                        ├─→ memory_recall, store_insight       ║  │
│  ║                                        └─→ home_control (lights, climate)     ║  │
│  ║                                                                               ║  │
│  ║    Full 200k context, your local tools accessible from anywhere               ║  │
│  ╚═══════════════════════════════════════════════════════════════════════════════╝  │
│                                                                                     │
│  ╔═══════════════════════════════════════════════════════════════════════════════╗  │
│  ║  SCENARIO C: Offline (internet down)                                          ║  │
│  ║  ──────────────────────────────────                                           ║  │
│  ║                                                                               ║  │
│  ║    Tatlock operates fully locally with Ollama                                 ║  │
│  ║    • All tools work (except web search)                                       ║  │
│  ║    • Graceful degradation with same butler personality                        ║  │
│  ╚═══════════════════════════════════════════════════════════════════════════════╝  │
│                                                                                     │
└─────────────────────────────────────────────────────────────────────────────────────┘
```

---

## Implementation Plan

### Phase 1: Blanket Backend Swap (Claude for All Agents)

Replace Ollama with Claude as the default backend for all PydanticAI agents, with automatic offline fallback.

**New Files:**
```
src/anthropic/
├── __init__.py
├── provider.py       # Claude provider with health check
└── model_selector.py # Chooses Claude or Ollama based on availability
```

**Key Implementation (`src/anthropic/provider.py`):**
```python
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.providers.anthropic import AnthropicProvider
from pydantic_ai.models.openai import OpenAIChatModel
from src.ollama.provider import get_ollama_provider
from src.core.config import config

_anthropic_available: bool | None = None

async def check_anthropic_health() -> bool:
    """Check if Anthropic API is reachable."""
    global _anthropic_available
    try:
        from anthropic import AsyncAnthropic
        client = AsyncAnthropic(api_key=config.ANTHROPIC_API_KEY)
        await client.messages.create(
            model=config.ANTHROPIC_MODEL,
            max_tokens=1,
            messages=[{"role": "user", "content": "hi"}]
        )
        _anthropic_available = True
    except Exception:
        _anthropic_available = False
    return _anthropic_available

def get_model(prefer_cloud: bool = True):
    """Get the best available model. Returns Claude if available, otherwise Ollama."""
    if prefer_cloud and config.ANTHROPIC_API_KEY and _anthropic_available:
        provider = AnthropicProvider(api_key=config.ANTHROPIC_API_KEY)
        return AnthropicModel(
            model_name=config.ANTHROPIC_MODEL,
            provider=provider,
        )
    else:
        return OpenAIChatModel(
            model_name=config.OLLAMA_DEFAULT_MODEL,
            provider=get_ollama_provider()
        )
```

**Modify TatlockAgent (`src/agents/tatlock.py`):**
```python
def _ensure_agent(self):
    if self._agent is not None:
        return

    from src.anthropic.model_selector import get_model

    model = get_model(prefer_cloud=True)

    self._agent = Agent(
        model,
        system_prompt=TATLOCK_SYSTEM_PROMPT,  # Same butler personality!
    )
    self._register_tools()
```

---

### Phase 2: MCP Server (Expose Tools to Claude)

Create an MCP server that exposes Tatlock's household tools to external Claude instances.

**New Files:**
```
src/mcp/
├── __init__.py
├── server.py         # MCP server using mcp Python SDK
├── tool_adapters.py  # Convert PydanticAI tools → MCP schemas
├── auth.py           # API key authentication
└── transport.py      # Streamable HTTP transport
```

**Docker Stack Addition (`stacks/agents.yml`):**
```yaml
tatlock-mcp:
  image: git.schweitz.internal/jpmschweitzer/tatlock:latest
  command: ["python", "-m", "src.mcp.server"]
  ports:
    - "8002:8002"
  environment:
    - MCP_AUTH_TOKEN=${MCP_AUTH_TOKEN}
  networks:
    - docker-dataplane
```

**Claude Desktop Configuration:**
```json
{
  "mcpServers": {
    "tatlock": {
      "command": "npx",
      "args": ["mcp-remote", "https://mcp.schweitz.net/sse", "--header", "Authorization: Bearer ${MCP_AUTH_TOKEN}"]
    }
  }
}
```

---

## Files to Modify

### Phase 1 - Backend Swap

**New Files:**
| File | Purpose |
|------|---------|
| `src/anthropic/__init__.py` | Package init |
| `src/anthropic/provider.py` | Claude provider with health check |
| `src/anthropic/model_selector.py` | Choose Claude or Ollama based on availability |

**Modified Files:**
| File | Changes |
|------|---------|
| `src/core/config.py` | Add `ANTHROPIC_API_KEY`, `ANTHROPIC_MODEL`, `PREFER_CLOUD_BACKEND` |
| `src/agents/tatlock.py` | Use `get_model()` instead of hardcoded Ollama |
| `src/agents/librarian/agent.py` | Use `get_model()` instead of hardcoded Ollama |
| `src/agents/biographer/agent.py` | Use `get_model()` instead of hardcoded Ollama |
| `src/agents/steward/agent.py` | Convert to PydanticAI or add Anthropic API support |
| `src/core/startup.py` | Add Anthropic health check on startup |
| `requirements.txt` | Add `anthropic>=0.40.0` |
| `.env.example` | Document new environment variables |

### Phase 2 - MCP Server

**New Files:**
| File | Purpose |
|------|---------|
| `src/mcp/__init__.py` | Package init |
| `src/mcp/server.py` | MCP server implementation |
| `src/mcp/tool_adapters.py` | PydanticAI → MCP schema conversion |
| `src/mcp/auth.py` | Token-based authentication |

---

## Cost Analysis

- **Claude API**: $5-30/month (10-50 calls/day, ~2k input + 1k output tokens/call)
- **MCP via Claude Pro**: Included in subscription
- **Total**: ~$10-80/month for full bidirectional integration

---

## Verification Plan

### Phase 1 Testing
```bash
# 1. Run with Claude backend
ANTHROPIC_API_KEY=your-key docker-compose up -d tatlock

# 2. Verify Claude is being used
docker logs tatlock 2>&1 | grep -i "anthropic\|claude"

# 3. Test butler personality
curl -X POST http://tatlock.schweitz.internal:8000/v1/responses \
  -H "Content-Type: application/json" \
  -d '{"model": "Tatlock", "input": "Hello, who are you?"}'

# 4. Test offline fallback
ANTHROPIC_API_KEY="" docker-compose up -d tatlock
docker logs tatlock 2>&1 | grep -i "ollama\|fallback"
```

### Phase 2 Testing
```bash
# 1. Start MCP server
docker-compose up -d tatlock-mcp

# 2. Test MCP endpoint
curl -X POST https://mcp.schweitz.net/tools/list \
  -H "Authorization: Bearer $MCP_AUTH_TOKEN"
```

---

## Implementation Priority

1. **Phase 1: Backend Swap** (~1 week)
   - Immediate value: 200k context for ALL queries
   - Low risk: provider abstraction, graceful offline fallback

2. **Phase 2: MCP Server** (~2-3 weeks)
   - Enables cross-device access
   - Bidirectional: Tatlock superpowered by Claude AND accessible to Claude

---

## Future Phases (Optional)

- **Phase 3: LiteLLM Gateway** - Unified endpoint for all models, config-driven routing
- **Phase 4: Multi-Provider** - Add OpenAI, Vertex AI, etc.
- **Phase 5: Smart Routing** - Context-aware model selection, cost ceiling enforcement

---

## Offline Behavior

| Scenario | Behavior |
|----------|----------|
| No API key | Use Ollama exclusively |
| API unreachable | Use Ollama, log warning |
| API rate limited | Fallback to Ollama |

| Aspect | Claude | Ollama |
|--------|--------|--------|
| Context | 200k tokens | ~8k tokens |
| Latency | 1-3s (network) | 0.5-1s (local) |
| Personality | Preserved | Preserved |
| Tools | All work | All work |
| Cost | API charges | Free |

---

## Implementation Status

### Phase 1: Backend Swap - CODE COMPLETE (awaiting API access)

- [x] Add Anthropic config settings to `src/core/config.py`
- [x] Add `pydantic-ai-slim[openai,anthropic]` to requirements.txt
- [x] Create `src/anthropic/` module (model_selector.py)
- [x] Add Claude health check to startup.py
- [x] Refactor all PydanticAI agents to use `get_model()`
  - [x] Librarian
  - [x] Biographer
  - [x] Housekeeper
  - [x] Tatlock (6 locations)
- [x] Add Claude API path to Steward agent (direct API calls)
- [x] Update `.env.example` with new variables
- [x] Test Ollama fallback (working)
- [ ] Test with Claude API key (blocked: no API access currently)

**Note:** Implementation complete. Currently runs in Ollama-only mode. Will automatically use Claude when `ANTHROPIC_API_KEY` is configured.

### Phase 2: MCP Server - NOT STARTED

- [ ] Create `src/mcp/` module
- [ ] Tool adapters (PydanticAI → MCP schema)
- [ ] Authentication middleware
- [ ] Streamable HTTP transport
- [ ] Docker stack configuration

---

## Related Repository Handovers

Handover documents created in each repo: `PROJECT_CLAUDIFICATION_HANDOVER.md`

### library-desk - HANDOVER CREATED

- [x] Write handover document
- [ ] Review HybridRAG response size limits
- [ ] Review smart_create endpoint for Claude optimization
- [ ] Evaluate response formats for LLM consumption

### core-api - HANDOVER CREATED

- [x] Write handover document
- [ ] Review list_devices response format
- [ ] Review error messages for LLM consumption
- [ ] Evaluate rate limiting for faster Claude processing

### portainer-core - HANDOVER CREATED (blocking for production)

- [x] Write handover document
- [ ] Update stack with new environment variables
- [ ] Configure secrets management for API key
- [ ] Update CONTAINERS.md documentation

### webber - HANDOVER CREATED

- [x] Write handover document
- [ ] Review content truncation limits
- [ ] Evaluate extraction quality for LLM consumption

### tatlock-ui - HANDOVER CREATED

- [x] Write handover document
- [ ] Test streaming responses with Claude backend
- [ ] Test conversation history with larger context
- [ ] Verify tool call display and reasoning rendering