jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

docs: update documentation for v0.2.0 tools release

Browse Source 
README.md:
- Add Tatlock agent capabilities and tool descriptions
- Add requirements section (Ollama, SearXNG setup)
- Add configuration examples for external services
- Add tool usage examples and philosophy
- Add troubleshooting for Ollama and SearXNG
- Update test statistics

AGENTS.md:
- Refactor for LLM development focus
- Add PydanticAI tool registration pattern
- Add tool implementation guidelines
- Remove project status, focus on development instructions

IMPLEMENTATION_ROADMAP.md:
- Mark Phase 1 as "MOSTLY COMPLETE"
- Update detailed completion status
- Update current state summary
This commit is contained in:
jpmschweitzer
2025-12-07 00:13:41 +01:00
parent 958363d44e
commit fd459e9ffb
3 changed files with 272 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+129 -98
View File
@@ -6,14 +6,11 @@ This document contains instructions and documentation references for AI assistan
## Project Overview
This project implements an OpenAI-compatible API endpoint using FastAPI, with streaming support. Currently returns mock responses - infrastructure prepared for future Ollama/PydanticAI integration.
This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer.
**Current State**: Production-ready testing API with Responses API and Open WebUI integration
**Future Integration**: PydanticAI for real LLM agents (tatlock model placeholder ready)
### Architecture Pattern
### Current Architecture (As of 2025-12-06)
This project implements the **Orchestrator** infrastructure layer with a hybrid API architecture:
The **Orchestrator** infrastructure layer with hybrid API architecture:
```
Client (Open WebUI)
@@ -83,7 +80,11 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
- **Agent Interface**: Abstract base class for model implementations
- **Conversation History**: Server-side tracking with configurable max turns
- **Context Window**: Token counting and management
- **PydanticAI**: Dependency installed, ready for tatlock agent implementation
- **PydanticAI**: Integrated with Tatlock agent (Ollama backend)
- **Agent Tools**: Permanent tools module (`src/agents/tools.py`)
- Calculator: Safe mathematical expression evaluation
- Date/Time toolkit: Current time, relative dates, time differences
- Web Search: SearXNG integration for privacy-preserving search
## Documentation References
@@ -171,58 +172,20 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
#### OpenAI API Reference
- **Official Documentation**: https://platform.openai.com/docs/api-reference
- **Implemented Endpoints**:
- ✅ `/v1/responses` - **Responses API (PRIMARY)** with structured output
- Reasoning items (thinking summaries)
- Function call items (tool execution)
- Message items (assistant responses)
- Full streaming support with SSE
- Stop sequence detection
- Max tokens enforcement
- Conversation history tracking
- ✅ `/v1/chat/completions` - **Compatibility wrapper** around Responses API
- Converts reasoning to `<think>` tags for Open WebUI
- Automatically enables reasoning generation
- Maintains OpenAI-compatible format
- Supports streaming and non-streaming
- ✅ `/v1/models` - List available models (lorem-tester, tatlock)
- **Future Endpoints**:
- 🚧 `/v1/completions` - Text completion (legacy)
- 🚧 `/v1/embeddings` - Text embeddings
- **Implemented Features**:
- ✅ **Responses API Format**:
- Structured output items (reasoning, function_call, message)
- Extended thinking support
- Tool/function calling support
- Streaming with multiple event types
- ✅ **Advanced Parameter Validation**:
- Temperature: 0.0-2.0 with Pydantic validators
- Reasoning effort: none, minimal, low, medium, high, xhigh
- Max output tokens: positive integer enforcement
- Stop sequences: up to 4, non-empty strings
- ✅ **Conversation History**:
- Hybrid client/server approach
- Auto-generated conversation IDs
- Configurable max turns (default: 20)
- Placeholder for vector memory
- ✅ **Context Management**:
- Approximate token counting (~4 chars/token)
- Context window trimming
- Usage statistics
- ✅ **Streaming Enforcement**:
- Real-time stop sequence detection
- Real-time max tokens enforcement
- Word-by-word streaming with delays
- ✅ **Error Handling**:
- Custom exception types (RateLimitError, ContextLengthError)
- OpenAI-compatible error format
- Error triggers in lorem-tester for testing
- ✅ **Testing Infrastructure**:
- 95 tests (78.95% coverage)
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
- Main application and wrapper tests
- **Key API Endpoints**:
- `/v1/responses` - Responses API (PRIMARY) with structured output
- `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper
- `/v1/models` - List available models
- **Key Features for Development**:
- **Responses API Format**: Structured output with reasoning, function_call, and message items
- **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences
- **Conversation History**: Hybrid client/server approach with auto-generated IDs
- **Context Management**: Token counting and window trimming
- **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement
- **Error Handling**: Custom exception types (RateLimitError, ContextLengthError)
- **Tool Calling**: PydanticAI tool integration with permanent tools
- **Testing**: Comprehensive test suite with mocks and real Ollama integration
## FastAPI Best Practices
@@ -413,45 +376,66 @@ app = create_application()
The user will manage git operations themselves unless they specifically request assistance.
### Code Structure (Current Implementation)
- ✅ Use async/await for ALL I/O operations (database, HTTP, file access)
- ✅ Use sync (def) for blocking SDKs or CPU-intensive work
- ✅ Implement proper error handling and logging
- ✅ Follow dependency injection for validation and shared resources
- ✅ Use Pydantic models for ALL request/response validation
- ✅ Keep business logic in service modules, not routers
- ✅ Domain-based project structure (not file-type based)
### Server Logs and Debugging
**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is:
- Cleared on each server startup (fresh logs every time)
- Written in real-time as the server runs
- Already gitignored (won't be committed)
**Accessing Logs**: You can read the log file at any time while the server is running:
```bash
# View current logs
cat logs/server.log
# Follow logs in real-time
tail -f logs/server.log
# Search logs
grep "ERROR" logs/server.log
```
This is useful for debugging issues, monitoring API calls, and understanding server behavior during development.
### Code Structure Guidelines
- Use async/await for ALL I/O operations (database, HTTP, file access)
- Use sync (def) for blocking SDKs or CPU-intensive work
- Implement proper error handling and logging
- Follow dependency injection for validation and shared resources
- Use Pydantic models for ALL request/response validation
- Keep business logic in service modules, not routers
- Domain-based project structure (not file-type based)
### Security Considerations
- ✅ Validate all inputs using Pydantic models
- ✅ Use environment variables for sensitive configuration
- ✅ Keep dependencies updated (all CVE-checked as of 2025-12-06)
- ✅ Minor version locking for supply chain protection
- 🚧 Implement rate limiting for API endpoints (future)
- 🚧 Add authentication/API keys (future)
- Validate all inputs using Pydantic models
- Use environment variables for sensitive configuration
- Keep dependencies updated and CVE-checked
- Minor version locking for supply chain protection
- Consider rate limiting for production deployment
- Plan for authentication/API keys when needed
### Testing (Current Coverage: 78.95%, 95 tests)
- ✅ Integration tests for API endpoints
- ✅ Streaming functionality with 20s timeout protection
- ✅ Async test support with pytest-asyncio
- ✅ Validate OpenAI API compatibility
- ✅ Mock responses for all endpoints
- ✅ Main application tests (CORS, exception handlers, lifespan)
- ✅ Chat streaming wrapper tests
- 🚧 Future: Mock Ollama responses when integrated
### Testing Approach
- Write integration tests for API endpoints
- Test streaming functionality with appropriate timeouts
- Use pytest-asyncio for async test support
- Validate OpenAI API compatibility in tests
- Test both mock and real LLM integrations
- Cover main application (CORS, exception handlers, lifespan)
- Test wrapper layers (chat completions, etc.)
- Include tool functionality tests
### Configuration
- ✅ Use `.env` files for local development
- ✅ Document all environment variables in README
- ✅ Provide sensible defaults where possible
- ✅ BaseSettings from pydantic-settings
- 🚧 Support container-based configuration (future)
### Configuration Management
- Use `.env` files for local development
- Document all environment variables in README
- Provide sensible defaults where possible
- Use BaseSettings from pydantic-settings
- Support both local and container-based configuration
## Common Patterns
### Streaming Response Pattern (✅ Implemented)
### Streaming Response Pattern
See `src/chat/router.py` for the current implementation:
Example from `src/chat/router.py`:
```python
from sse_starlette.sse import EventSourceResponse
@@ -468,9 +452,9 @@ async def stream():
return EventSourceResponse(event_generator())
```
### PydanticAI Agent Pattern (🚧 Future Reference)
### PydanticAI Agent Pattern
For future integration when connecting to Ollama:
When implementing agents with PydanticAI and Ollama:
```python
from pydantic_ai import Agent
@@ -484,9 +468,9 @@ agent = Agent(
result = await agent.run('Your prompt')
```
### OpenAI-Compatible Response Format (✅ Implemented)
### OpenAI-Compatible Response Format
Current implementation in `src/chat/schemas.py`:
Example schema from `src/chat/schemas.py`:
```python
{
@@ -502,12 +486,59 @@ Current implementation in `src/chat/schemas.py`:
}
```
### PydanticAI Tool Registration Pattern
Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples:
```python
from pydantic_ai import Agent, RunContext
# After creating the agent
@agent.tool
def tool_name(ctx: RunContext[None], param: str) -> str:
"""
Tool description that the LLM sees.
Args:
param: Parameter description
Returns:
Result description
"""
return result
```
**Tool Implementation Guidelines**:
- Keep tools in `src/agents/tools.py` for reusability
- Use clear, descriptive docstrings (LLM reads these)
- Include parameter descriptions in docstrings
- Handle errors gracefully and return error messages as strings
- For async operations, declare the tool function as `async def`
- Test tools independently before integration
**Example Tool Module** (`src/agents/tools.py`):
```python
def calculate(expression: str) -> str:
"""Safe calculator implementation."""
try:
# Implementation
return str(result)
except Exception as e:
return f"Error: {str(e)}"
async def search_web(query: str) -> str:
"""Web search via SearXNG."""
async with httpx.AsyncClient() as client:
# Implementation
return formatted_results
```
## Update Policy
This document should be updated when:
- New development patterns are established
- Package versions are upgraded
- New major features are added
- Breaking API changes occur
- Security vulnerabilities are discovered
- Major architectural changes occur
- New best practices are identified
Last updated: 2025-12-06
Last updated: 2025-12-06 (Tools integration)
IMPLEMENTATION_ROADMAP.md
+51 -36
View File
@@ -4,69 +4,84 @@
This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system.
## Current State (v0.1.1 - Basic Setup Complete)
## Current State (v0.1.1+ - Phase 1 Mostly Complete)
**What we have**:
- ✅ **The Orchestrator** - FastAPI infrastructure layer
- OpenAI-compatible API endpoints (Responses API + Chat Completions)
- Streaming coordination and conversation management
- Response format with reasoning support
- Test infrastructure (95 tests, 78.95% coverage)
- ✅ Mock agents (lorem-tester for testing, Tatlock placeholder)
- ✅ Agent interface abstraction ready for PydanticAI
- Test infrastructure (131 tests, 81.78% coverage)
- ✅ **Tatlock Agent** - Real PydanticAI integration
- Connected to Ollama (mistral-nemo:latest)
- British butler personality with research mindset
- Streaming responses with reasoning
- Tool calling framework functional
- ✅ **Permanent Tools**
- Calculator (safe mathematical expressions)
- Date/Time toolkit (current time, relative dates, time differences)
- Web search (SearXNG integration)
- ✅ Mock agent (lorem-tester for testing)
- ✅ Agent interface abstraction
**What we need**:
- **The Household** - PydanticAI agent implementations:
- **The Household** - Full multi-agent coordination:
- The Steward (first-tier request analysis)
- Tatlock Butler (second-tier coordination with personality)
- Tatlock coordination layer (expert agent delegation)
- Expert household staff agents (Librarian, Developer, Handyman, etc.)
- Real LLM integration (PydanticAI + Ollama)
- Multi-tenant database architecture
- Containerized service ecosystem
- MCP (Model Context Protocol) integration
- Dynamic model switching for specialized tasks
---
## Phase 1: Real LLM Integration - Ollama + PydanticAI
## Phase 1: Real LLM Integration - PydanticAI + Tools
**Goal**: Connect to actual language models and establish the base plumbing
**Note**: Ollama is an external service dependency (already running separately)
### Deliverables
1. **Ollama Setup**
- Docker compose configuration for Ollama
- Model download and management
- Base model selection (single model in VRAM)
- Health checking and monitoring
1. **PydanticAI Integration** ✅
- PydanticAI → Ollama connection ✅
- Agent creation patterns ✅
- Streaming response handling ✅
- Error handling and retries ✅
2. **PydanticAI Integration**
- PydanticAI → Ollama connection
- Agent creation patterns
- Streaming response handling
- Error handling and retries
2. **Convert Tatlock Agent** ✅
- Convert Tatlock agent from mock to PydanticAI ✅
- British butler personality prompt ✅
- Research-oriented mindset ✅
- Streaming to reasoning output ✅
- Tool calling framework setup ✅
3. **Replace Mock Agents**
- Convert Tatlock agent from mock to PydanticAI
- Basic personality prompt
- Streaming to reasoning output
- Tool calling framework setup
3. **Permanent Tools** ✅
- Calculator: Safe mathematical expression evaluation ✅
- Date/Time toolkit: Current time, relative dates, time differences ✅
- Web search: SearXNG integration (external service) ✅
- Tool registration with PydanticAI ✅
4. **Testing Infrastructure**
- Integration tests with real LLM
- Prompt testing utilities
- Response quality validation
- Performance benchmarking
4. **Testing Infrastructure** ✅
- Integration tests with real LLM ✅
- Tool functionality tests ✅
- Response quality validation ✅
- 131 tests, 81.78% coverage ✅
### Success Criteria
- [ ] Ollama running in Docker
- [ ] Base model loaded and responding
- [ ] PydanticAI agents can call Ollama
- [ ] Streaming works end-to-end
- [ ] Can switch models (e.g., Codestral for code)
- [ ] Tests pass with real LLM
- [x] **PydanticAI agents can call Ollama** (mistral-nemo:latest)
- [x] **Streaming works end-to-end**
- [x] **Tool calling framework functional**
- [x] **Permanent tools working** (calculator, date/time, search)
- [x] **Tests pass with real LLM**
- [ ] Can switch models dynamically (e.g., Codestral for code)
### Estimated Effort
**2-3 weeks** - Critical foundation for everything else
### Status
**✅ MOSTLY COMPLETE** - Tatlock agent functional with permanent tools
### Remaining Work
- Dynamic model switching for specialized tasks (e.g., Codestral for coding)
### Why First?
Without real LLM integration, we can't meaningfully implement the Steward/Butler pattern. Everything else depends on having actual AI agents working.
README.md
+92 -11
View File
@@ -9,8 +9,9 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
- ✅ **Production-ready testing API** with OpenAI Responses API format
- ✅ **Open WebUI integration** with reasoning bubbles (`<think>` tags)
- ✅ **Conversation history** with auto-generated IDs and context management
- ✅ **Comprehensive testing** - 95 tests, 78.95% coverage
- 🚧 **PydanticAI integration** prepared for future real LLM connection
- ✅ **Tatlock PydanticAI Agent** - Real LLM integration with Ollama + permanent tools
- ✅ **Permanent Tools** - Calculator, date/time toolkit, web search (SearXNG)
- ✅ **Comprehensive testing** - 131 tests, 81.78% coverage
## Features
@@ -36,20 +37,32 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
- **Parameter Validation**: Temperature (0.0-2.0), reasoning effort levels, max tokens, stop sequences
- **Real-time Enforcement**: Stop sequence detection and max token limits during streaming
### Testing Models
### Available Models
- **lorem-tester**: Full-featured mock agent with realistic behavior
- Configurable reasoning effort levels
- Random tool/function calls
- Error triggers for testing (rate_limit, context_overflow)
- **tatlock**: Placeholder for future PydanticAI agent
- **Tatlock**: Real PydanticAI agent with butler personality
- **LLM Backend**: Ollama (mistral-nemo:latest)
- **Personality**: Witty British butler, research-oriented
- **Permanent Tools**:
- **Calculator**: Safe mathematical expression evaluation (arithmetic, algebra, trigonometry, logarithms)
- **Date/Time Toolkit**: Current time, relative dates ("1 week ago"), time differences
- **Web Search**: Privacy-preserving search via SearXNG
- **Capabilities**: Streaming, reasoning, tool calling
- **Phase**: Phase 1 - Basic Integration (full household coordination coming in future phases)
## Requirements
- Python 3.12+ (Python 3.12.11 recommended)
- No external dependencies for mock API
- (Future: Network access for PydanticAI integration)
- **Ollama** (for Tatlock agent): Running locally or network-accessible
- Download: https://ollama.ai/
- Model: `ollama pull mistral-nemo:latest`
- **SearXNG** (for web search tool): Optional but recommended
- Docker: `docker run -d -p 8087:8080 searxng/searxng`
- Or use public instance (less private)
## Quick Start
@@ -164,6 +177,42 @@ curl http://localhost:8000/v1/responses \
**Note**: Client must send full conversation history in `input` array (OpenAI compatible). Server optionally tracks via `metadata.conversation_id` for future features.
### Using Tatlock with Tools
Tatlock automatically uses his permanent tools when appropriate:
```bash
# Mathematical calculation
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Tatlock",
"messages": [{"role": "user", "content": "What is sqrt(144) + 25?"}]
}'
# Date/time queries
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Tatlock",
"messages": [{"role": "user", "content": "What was the date 2 weeks ago?"}]
}'
# Web search for current information
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Tatlock",
"messages": [{"role": "user", "content": "Search for recent Python 3.12 features"}]
}'
```
**Tatlock's Tool Usage Philosophy:**
- Uses calculator for ALL mathematics (even simple arithmetic)
- Uses date/time tools instead of guessing dates
- Searches for current/volatile information to verify facts
- Maintains a researcher's mindset with tool-assisted verification
## Open WebUI Integration
### Connection
@@ -205,9 +254,14 @@ pytest
# Run with coverage
pytest --cov=src --cov-report=term-missing
# Current: 95 tests, 78.95% coverage
# Current: 131 tests, 81.78% coverage
```
**Test Categories:**
- Unit tests: Agent tools, streaming, schemas
- Integration tests: Full API stack with real Ollama calls
- End-to-end tests: Chat completions, responses API
## Deployment
### Production Server
@@ -234,15 +288,24 @@ Create a `.env` file for custom configuration:
API_HOST=0.0.0.0
API_PORT=8000
# Ollama Configuration
OLLAMA_HOST=http://localhost:11434
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
OLLAMA_TIMEOUT=120
# SearXNG Configuration (for web search tool)
SEARXNG_HOST=http://localhost:8087
SEARXNG_TIMEOUT=30
# Logging
LOG_LEVEL=INFO
# CORS (default: allow all)
CORS_ORIGINS=["*"]
# Future: LLM configuration
```
See `.env.example` for full configuration options.
## Troubleshooting
### Streaming not working
@@ -260,19 +323,37 @@ CORS_ORIGINS=["*"]
- Or manually enable in Responses API: `"reasoning": {"effort": "medium", "summary": "auto"}`
- Check Open WebUI version supports `<think>` tags
### Tatlock agent errors
- Verify Ollama is running: `curl http://localhost:11434/api/tags`
- Check model is downloaded: `ollama list`
- Review environment variables: `OLLAMA_HOST`, `OLLAMA_DEFAULT_MODEL`
- Check logs: `tail -f logs/server.log`
### Web search not working
- Verify SearXNG is running: `curl http://localhost:8087/`
- Check `SEARXNG_HOST` environment variable
- SearXNG is optional - Tatlock will note if search is unavailable
## Project Structure
```
tatlock/
├── src/
│ ├── agents/ # Agent interface and implementations
│ │ ├── base.py # AgentInterface abstract class
│ │ ├── lorem_tester.py # Mock agent for testing
│ │ ├── tatlock.py # Real PydanticAI butler agent
│ │ ├── tools.py # Permanent tools (calculator, date/time, search)
│ │ └── registry.py # Model registry
│ ├── responses/ # Responses API (primary endpoint)
│ ├── chat/ # Chat Completions wrapper
│ ├── models/ # Models listing
│ ├── core/ # Shared utilities and config
│ └── main.py # Application entry point
├── tests/ # Comprehensive test suite
├── tests/ # Comprehensive test suite (131 tests)
├── AGENTS.md # LLM agent development guidelines
├── PHILOSOPHY.md # System vision and architecture
├── IMPLEMENTATION_ROADMAP.md # Development phases
├── CHANGELOG.md # Version history
└── README.md # This file
```
@@ -307,7 +388,7 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md
## Version
Current version: **0.1.1** - Basic setup complete
Current version: **0.2.0** - PydanticAI Integration with Permanent Tools
---