Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
426f9885fc | ||
|
|
199aa7228c | ||
|
|
fd459e9ffb | ||
|
|
958363d44e | ||
|
|
4216d89f12 | ||
|
|
67481515cc | ||
|
|
f3e2681a6c | ||
|
|
a1a0f6923b | ||
|
|
e85823ff18 | ||
|
|
5f4e93bf09 | ||
|
|
da9b4954be | ||
|
|
882347452f | ||
|
|
8d0618b647 |
@@ -14,6 +14,10 @@ OLLAMA_HOST=http://your-ollama-host:11434
|
||||
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
|
||||
OLLAMA_TIMEOUT=120
|
||||
|
||||
# SearXNG Configuration
|
||||
SEARXNG_HOST=http://searxng:8087
|
||||
SEARXNG_TIMEOUT=30
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
|
||||
@@ -2,16 +2,15 @@
|
||||
|
||||
This document contains instructions and documentation references for AI assistants working with this codebase.
|
||||
|
||||
> **📖 Important**: Before working on this project, read [PHILOSOPHY.md](PHILOSOPHY.md) to understand the system vision, architectural patterns, and design goals. All development should work towards realizing those patterns.
|
||||
|
||||
## Project Overview
|
||||
|
||||
This project implements an OpenAI-compatible API endpoint using FastAPI, with streaming support. Currently returns mock responses - infrastructure prepared for future Ollama/PydanticAI integration.
|
||||
This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer.
|
||||
|
||||
**Current State**: Production-ready testing API with Responses API and Open WebUI integration
|
||||
**Future Integration**: PydanticAI for real LLM agents (tatlock model placeholder ready)
|
||||
### Architecture Pattern
|
||||
|
||||
### Current Architecture (As of 2025-12-06)
|
||||
|
||||
This project implements a **hybrid architecture** with the Responses API as the primary endpoint and Chat Completions as a compatibility wrapper:
|
||||
The **Orchestrator** infrastructure layer with hybrid API architecture:
|
||||
|
||||
```
|
||||
Client (Open WebUI)
|
||||
@@ -20,9 +19,24 @@ Chat Completions (/v1/chat/completions) → Wrapper
|
||||
↓
|
||||
Responses API (/v1/responses) → Primary
|
||||
↓
|
||||
Agent Interface (lorem-tester, tatlock)
|
||||
Agent Interface (lorem-tester, Tatlock)
|
||||
↓
|
||||
Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
|
||||
```
|
||||
|
||||
**Architectural Layers:**
|
||||
|
||||
1. **The Orchestrator** (Current Implementation)
|
||||
- FastAPI application providing the infrastructure
|
||||
- HTTP/SSE endpoints, streaming coordination
|
||||
- Conversation history and context management
|
||||
- OpenAI-compatible API surface
|
||||
|
||||
2. **Future: The Household** (Phases 1-4)
|
||||
- **Steward**: First-tier LLM for request analysis (PydanticAI agent)
|
||||
- **Tatlock**: Second-tier LLM with butler personality (PydanticAI agent)
|
||||
- **Expert Agents**: Domain specialists (Librarian, Developer, Handyman, etc.)
|
||||
|
||||
**Key Architectural Decisions:**
|
||||
|
||||
1. **Single Source of Truth**: All response generation happens in the Responses API
|
||||
@@ -43,7 +57,7 @@ Agent Interface (lorem-tester, tatlock)
|
||||
- Random tool/function calls
|
||||
- Error triggers for testing
|
||||
- Temperature variation
|
||||
- **tatlock**: Placeholder for future PydanticAI agent
|
||||
- **Tatlock**: Advertised model name (currently mock, future: PydanticAI Butler agent)
|
||||
|
||||
4. **Hybrid Conversation History**:
|
||||
- Client MUST send full context in `input` array (OpenAI compatible)
|
||||
@@ -66,7 +80,11 @@ Agent Interface (lorem-tester, tatlock)
|
||||
- **Agent Interface**: Abstract base class for model implementations
|
||||
- **Conversation History**: Server-side tracking with configurable max turns
|
||||
- **Context Window**: Token counting and management
|
||||
- **PydanticAI**: Dependency installed, ready for tatlock agent implementation
|
||||
- **PydanticAI**: Integrated with Tatlock agent (Ollama backend)
|
||||
- **Agent Tools**: Permanent tools module (`src/agents/tools.py`)
|
||||
- Calculator: Safe mathematical expression evaluation
|
||||
- Date/Time toolkit: Current time, relative dates, time differences
|
||||
- Web Search: SearXNG integration for privacy-preserving search
|
||||
|
||||
## Documentation References
|
||||
|
||||
@@ -154,57 +172,20 @@ Agent Interface (lorem-tester, tatlock)
|
||||
|
||||
#### OpenAI API Reference
|
||||
- **Official Documentation**: https://platform.openai.com/docs/api-reference
|
||||
- **Implemented Endpoints**:
|
||||
- ✅ `/v1/responses` - **Responses API (PRIMARY)** with structured output
|
||||
- Reasoning items (thinking summaries)
|
||||
- Function call items (tool execution)
|
||||
- Message items (assistant responses)
|
||||
- Full streaming support with SSE
|
||||
- Stop sequence detection
|
||||
- Max tokens enforcement
|
||||
- Conversation history tracking
|
||||
- ✅ `/v1/chat/completions` - **Compatibility wrapper** around Responses API
|
||||
- Converts reasoning to `<think>` tags for Open WebUI
|
||||
- Automatically enables reasoning generation
|
||||
- Maintains OpenAI-compatible format
|
||||
- Supports streaming and non-streaming
|
||||
- ✅ `/v1/models` - List available models (lorem-tester, tatlock)
|
||||
- **Future Endpoints**:
|
||||
- 🚧 `/v1/completions` - Text completion (legacy)
|
||||
- 🚧 `/v1/embeddings` - Text embeddings
|
||||
- **Implemented Features**:
|
||||
- ✅ **Responses API Format**:
|
||||
- Structured output items (reasoning, function_call, message)
|
||||
- Extended thinking support
|
||||
- Tool/function calling support
|
||||
- Streaming with multiple event types
|
||||
- ✅ **Advanced Parameter Validation**:
|
||||
- Temperature: 0.0-2.0 with Pydantic validators
|
||||
- Reasoning effort: none, minimal, low, medium, high, xhigh
|
||||
- Max output tokens: positive integer enforcement
|
||||
- Stop sequences: up to 4, non-empty strings
|
||||
- ✅ **Conversation History**:
|
||||
- Hybrid client/server approach
|
||||
- Auto-generated conversation IDs
|
||||
- Configurable max turns (default: 20)
|
||||
- Placeholder for vector memory
|
||||
- ✅ **Context Management**:
|
||||
- Approximate token counting (~4 chars/token)
|
||||
- Context window trimming
|
||||
- Usage statistics
|
||||
- ✅ **Streaming Enforcement**:
|
||||
- Real-time stop sequence detection
|
||||
- Real-time max tokens enforcement
|
||||
- Word-by-word streaming with delays
|
||||
- ✅ **Error Handling**:
|
||||
- Custom exception types (RateLimitError, ContextLengthError)
|
||||
- OpenAI-compatible error format
|
||||
- Error triggers in lorem-tester for testing
|
||||
- ✅ **Testing Infrastructure**:
|
||||
- 75 tests (78.95% coverage)
|
||||
- Unit tests for all components
|
||||
- Integration tests for API endpoints
|
||||
- Streaming tests for SSE functionality
|
||||
- **Key API Endpoints**:
|
||||
- `/v1/responses` - Responses API (PRIMARY) with structured output
|
||||
- `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper
|
||||
- `/v1/models` - List available models
|
||||
|
||||
- **Key Features for Development**:
|
||||
- **Responses API Format**: Structured output with reasoning, function_call, and message items
|
||||
- **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences
|
||||
- **Conversation History**: Hybrid client/server approach with auto-generated IDs
|
||||
- **Context Management**: Token counting and window trimming
|
||||
- **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement
|
||||
- **Error Handling**: Custom exception types (RateLimitError, ContextLengthError)
|
||||
- **Tool Calling**: PydanticAI tool integration with permanent tools
|
||||
- **Testing**: Comprehensive test suite with mocks and real Ollama integration
|
||||
|
||||
## FastAPI Best Practices
|
||||
|
||||
@@ -385,43 +366,76 @@ app = create_application()
|
||||
|
||||
## Development Guidelines
|
||||
|
||||
### Code Structure (Current Implementation)
|
||||
- ✅ Use async/await for ALL I/O operations (database, HTTP, file access)
|
||||
- ✅ Use sync (def) for blocking SDKs or CPU-intensive work
|
||||
- ✅ Implement proper error handling and logging
|
||||
- ✅ Follow dependency injection for validation and shared resources
|
||||
- ✅ Use Pydantic models for ALL request/response validation
|
||||
- ✅ Keep business logic in service modules, not routers
|
||||
- ✅ Domain-based project structure (not file-type based)
|
||||
### Git Workflow
|
||||
|
||||
**IMPORTANT**: Do NOT handle git commits or pushes automatically. Wait for explicit user instruction before:
|
||||
- Running `git add`
|
||||
- Running `git commit`
|
||||
- Running `git push`
|
||||
- Creating or pushing tags
|
||||
|
||||
The user will manage git operations themselves unless they specifically request assistance.
|
||||
|
||||
### Server Logs and Debugging
|
||||
|
||||
**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is:
|
||||
- Cleared on each server startup (fresh logs every time)
|
||||
- Written in real-time as the server runs
|
||||
- Already gitignored (won't be committed)
|
||||
|
||||
**Accessing Logs**: You can read the log file at any time while the server is running:
|
||||
```bash
|
||||
# View current logs
|
||||
cat logs/server.log
|
||||
|
||||
# Follow logs in real-time
|
||||
tail -f logs/server.log
|
||||
|
||||
# Search logs
|
||||
grep "ERROR" logs/server.log
|
||||
```
|
||||
|
||||
This is useful for debugging issues, monitoring API calls, and understanding server behavior during development.
|
||||
|
||||
### Code Structure Guidelines
|
||||
- Use async/await for ALL I/O operations (database, HTTP, file access)
|
||||
- Use sync (def) for blocking SDKs or CPU-intensive work
|
||||
- Implement proper error handling and logging
|
||||
- Follow dependency injection for validation and shared resources
|
||||
- Use Pydantic models for ALL request/response validation
|
||||
- Keep business logic in service modules, not routers
|
||||
- Domain-based project structure (not file-type based)
|
||||
|
||||
### Security Considerations
|
||||
- ✅ Validate all inputs using Pydantic models
|
||||
- ✅ Use environment variables for sensitive configuration
|
||||
- ✅ Keep dependencies updated (all CVE-checked as of 2025-12-06)
|
||||
- ✅ Minor version locking for supply chain protection
|
||||
- 🚧 Implement rate limiting for API endpoints (future)
|
||||
- 🚧 Add authentication/API keys (future)
|
||||
- Validate all inputs using Pydantic models
|
||||
- Use environment variables for sensitive configuration
|
||||
- Keep dependencies updated and CVE-checked
|
||||
- Minor version locking for supply chain protection
|
||||
- Consider rate limiting for production deployment
|
||||
- Plan for authentication/API keys when needed
|
||||
|
||||
### Testing (Current Coverage: 62%)
|
||||
- ✅ Integration tests for API endpoints
|
||||
- ✅ Streaming functionality with 20s timeout protection
|
||||
- ✅ Async test support with pytest-asyncio
|
||||
- ✅ Validate OpenAI API compatibility
|
||||
- ✅ Mock responses for all endpoints
|
||||
- 🚧 Future: Mock Ollama responses when integrated
|
||||
### Testing Approach
|
||||
- Write integration tests for API endpoints
|
||||
- Test streaming functionality with appropriate timeouts
|
||||
- Use pytest-asyncio for async test support
|
||||
- Validate OpenAI API compatibility in tests
|
||||
- Test both mock and real LLM integrations
|
||||
- Cover main application (CORS, exception handlers, lifespan)
|
||||
- Test wrapper layers (chat completions, etc.)
|
||||
- Include tool functionality tests
|
||||
|
||||
### Configuration
|
||||
- ✅ Use `.env` files for local development
|
||||
- ✅ Document all environment variables in README
|
||||
- ✅ Provide sensible defaults where possible
|
||||
- ✅ BaseSettings from pydantic-settings
|
||||
- 🚧 Support container-based configuration (future)
|
||||
### Configuration Management
|
||||
- Use `.env` files for local development
|
||||
- Document all environment variables in README
|
||||
- Provide sensible defaults where possible
|
||||
- Use BaseSettings from pydantic-settings
|
||||
- Support both local and container-based configuration
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Streaming Response Pattern (✅ Implemented)
|
||||
### Streaming Response Pattern
|
||||
|
||||
See `src/chat/router.py` for the current implementation:
|
||||
Example from `src/chat/router.py`:
|
||||
|
||||
```python
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
@@ -438,9 +452,9 @@ async def stream():
|
||||
return EventSourceResponse(event_generator())
|
||||
```
|
||||
|
||||
### PydanticAI Agent Pattern (🚧 Future Reference)
|
||||
### PydanticAI Agent Pattern
|
||||
|
||||
For future integration when connecting to Ollama:
|
||||
When implementing agents with PydanticAI and Ollama:
|
||||
|
||||
```python
|
||||
from pydantic_ai import Agent
|
||||
@@ -454,9 +468,9 @@ agent = Agent(
|
||||
result = await agent.run('Your prompt')
|
||||
```
|
||||
|
||||
### OpenAI-Compatible Response Format (✅ Implemented)
|
||||
### OpenAI-Compatible Response Format
|
||||
|
||||
Current implementation in `src/chat/schemas.py`:
|
||||
Example schema from `src/chat/schemas.py`:
|
||||
|
||||
```python
|
||||
{
|
||||
@@ -472,12 +486,59 @@ Current implementation in `src/chat/schemas.py`:
|
||||
}
|
||||
```
|
||||
|
||||
### PydanticAI Tool Registration Pattern
|
||||
|
||||
Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples:
|
||||
|
||||
```python
|
||||
from pydantic_ai import Agent, RunContext
|
||||
|
||||
# After creating the agent
|
||||
@agent.tool
|
||||
def tool_name(ctx: RunContext[None], param: str) -> str:
|
||||
"""
|
||||
Tool description that the LLM sees.
|
||||
|
||||
Args:
|
||||
param: Parameter description
|
||||
|
||||
Returns:
|
||||
Result description
|
||||
"""
|
||||
return result
|
||||
```
|
||||
|
||||
**Tool Implementation Guidelines**:
|
||||
- Keep tools in `src/agents/tools.py` for reusability
|
||||
- Use clear, descriptive docstrings (LLM reads these)
|
||||
- Include parameter descriptions in docstrings
|
||||
- Handle errors gracefully and return error messages as strings
|
||||
- For async operations, declare the tool function as `async def`
|
||||
- Test tools independently before integration
|
||||
|
||||
**Example Tool Module** (`src/agents/tools.py`):
|
||||
```python
|
||||
def calculate(expression: str) -> str:
|
||||
"""Safe calculator implementation."""
|
||||
try:
|
||||
# Implementation
|
||||
return str(result)
|
||||
except Exception as e:
|
||||
return f"Error: {str(e)}"
|
||||
|
||||
async def search_web(query: str) -> str:
|
||||
"""Web search via SearXNG."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
# Implementation
|
||||
return formatted_results
|
||||
```
|
||||
|
||||
## Update Policy
|
||||
|
||||
This document should be updated when:
|
||||
- New development patterns are established
|
||||
- Package versions are upgraded
|
||||
- New major features are added
|
||||
- Breaking API changes occur
|
||||
- Security vulnerabilities are discovered
|
||||
- Major architectural changes occur
|
||||
- New best practices are identified
|
||||
|
||||
Last updated: 2025-12-06
|
||||
Last updated: 2025-12-06 (Tools integration)
|
||||
|
||||
+83
-1
@@ -7,6 +7,87 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.2.0] - 2025-12-06
|
||||
|
||||
### Added
|
||||
|
||||
#### PydanticAI Integration (Phase 1)
|
||||
- Real Tatlock agent using PydanticAI with Ollama backend (mistral-nemo:latest)
|
||||
- British butler personality with research-oriented mindset
|
||||
- Lazy agent initialization to avoid connection issues in tests
|
||||
- Streaming response integration with reasoning output
|
||||
- Error handling for PydanticAI-specific exceptions
|
||||
|
||||
#### Permanent Tools (Phase 1)
|
||||
- **Calculator tool** (`src/agents/tools.py`):
|
||||
- Safe mathematical expression evaluation using restricted namespace
|
||||
- Support for arithmetic, algebra, trigonometry, logarithms
|
||||
- Math functions: sqrt, sin, cos, tan, log, exp, etc.
|
||||
- Constants: pi, e
|
||||
- Integer result formatting (removes unnecessary decimals)
|
||||
- **Date/Time toolkit**:
|
||||
- `get_current_datetime`: Current date/time in multiple formats
|
||||
- `calculate_time_offset`: Relative date calculations ("1 week ago", "2 months from now")
|
||||
- `time_difference`: Human-readable time differences between dates
|
||||
- **Web Search tool**:
|
||||
- SearXNG integration for privacy-preserving web search
|
||||
- Automatic fallback from production to localhost in development
|
||||
- Formatted search results with titles, URLs, and snippets
|
||||
- Configurable result limits (max 10)
|
||||
|
||||
#### Tool Framework
|
||||
- PydanticAI tool registration with `@agent.tool` decorator
|
||||
- Tool descriptions visible to LLM for intelligent usage
|
||||
- Async tool support for I/O operations
|
||||
- Error handling with string-based error messages
|
||||
- Tool usage guidelines in system prompt
|
||||
|
||||
#### Configuration
|
||||
- SearXNG configuration in `src/core/config.py`:
|
||||
- `SEARXNG_HOST` with development fallback
|
||||
- `SEARXNG_TIMEOUT` setting
|
||||
- Updated `.env.example` with SearXNG configuration
|
||||
- Ollama configuration documentation
|
||||
|
||||
#### Testing
|
||||
- 26 new tool tests (`tests/agents/test_tools.py`):
|
||||
- 7 calculator tests (arithmetic, functions, error handling)
|
||||
- 14 date/time tests (current time, offsets, differences)
|
||||
- 5 web search tests (mocked HTTP client)
|
||||
- Updated registry tests for tools capability
|
||||
- Total: 131 tests, 81.78% coverage (up from 95 tests, 78.95%)
|
||||
|
||||
#### Documentation
|
||||
- Comprehensive README.md updates:
|
||||
- Tatlock agent capabilities and tool descriptions
|
||||
- Requirements section with Ollama and SearXNG setup
|
||||
- Configuration examples for external services
|
||||
- Tool usage examples and philosophy
|
||||
- Troubleshooting for Ollama and SearXNG
|
||||
- Updated test statistics
|
||||
- AGENTS.md refactored for LLM development:
|
||||
- PydanticAI tool registration pattern
|
||||
- Tool implementation guidelines
|
||||
- Removed project status, focused on development instructions
|
||||
- IMPLEMENTATION_ROADMAP.md updates:
|
||||
- Phase 1 marked as "MOSTLY COMPLETE"
|
||||
- Detailed completion status for each deliverable
|
||||
- Updated current state summary
|
||||
|
||||
### Changed
|
||||
- Tatlock agent converted from mock to real PydanticAI implementation
|
||||
- Tatlock capabilities updated: `tools: True`
|
||||
- Streaming coordination now handles chunk-based delivery (50 chars) to preserve markdown
|
||||
- Chat service streaming updated to preserve formatting
|
||||
- System prompt enhanced with tool usage guidelines and research mindset
|
||||
- Agent initialization changed to lazy pattern for better testability
|
||||
|
||||
### Fixed
|
||||
- Text duplication bug in streaming responses (proper delta calculation)
|
||||
- Markdown formatting preservation in streamed responses
|
||||
- GeneratorExit errors from async context managers in generators
|
||||
- PydanticAI API usage (`result.output` instead of `result.data`)
|
||||
|
||||
## [0.1.1] - 2025-12-06
|
||||
|
||||
### Added
|
||||
@@ -115,6 +196,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- CORS middleware
|
||||
- Exception handlers (OpenAI-compatible error format)
|
||||
|
||||
[Unreleased]: https://git.schweitz.net/jpmschweitzer/tatlock/compare/v0.1.1...main
|
||||
[Unreleased]: https://git.schweitz.net/jpmschweitzer/tatlock/compare/v0.2.0...main
|
||||
[0.2.0]: https://git.schweitz.net/jpmschweitzer/tatlock/compare/v0.1.1...v0.2.0
|
||||
[0.1.1]: https://git.schweitz.net/jpmschweitzer/tatlock/compare/v0.1.0...v0.1.1
|
||||
[0.1.0]: https://git.schweitz.net/jpmschweitzer/tatlock/releases/tag/v0.1.0
|
||||
|
||||
@@ -0,0 +1,599 @@
|
||||
# Tatlock Implementation Roadmap
|
||||
|
||||
> **Reference**: See [PHILOSOPHY.md](PHILOSOPHY.md) for the target architecture and vision
|
||||
|
||||
This document outlines the phased implementation plan to transform the current OpenAI-compatible API into the full Tatlock household butler system.
|
||||
|
||||
## Current State (v0.1.1+ - Phase 1 Mostly Complete)
|
||||
|
||||
**What we have**:
|
||||
- ✅ **The Orchestrator** - FastAPI infrastructure layer
|
||||
- OpenAI-compatible API endpoints (Responses API + Chat Completions)
|
||||
- Streaming coordination and conversation management
|
||||
- Response format with reasoning support
|
||||
- Test infrastructure (131 tests, 81.78% coverage)
|
||||
- ✅ **Tatlock Agent** - Real PydanticAI integration
|
||||
- Connected to Ollama (mistral-nemo:latest)
|
||||
- British butler personality with research mindset
|
||||
- Streaming responses with reasoning
|
||||
- Tool calling framework functional
|
||||
- ✅ **Permanent Tools**
|
||||
- Calculator (safe mathematical expressions)
|
||||
- Date/Time toolkit (current time, relative dates, time differences)
|
||||
- Web search (SearXNG integration)
|
||||
- ✅ Mock agent (lorem-tester for testing)
|
||||
- ✅ Agent interface abstraction
|
||||
|
||||
**What we need**:
|
||||
- **The Household** - Full multi-agent coordination:
|
||||
- The Steward (first-tier request analysis)
|
||||
- Tatlock coordination layer (expert agent delegation)
|
||||
- Expert household staff agents (Librarian, Developer, Handyman, etc.)
|
||||
- Multi-tenant database architecture
|
||||
- Containerized service ecosystem
|
||||
- MCP (Model Context Protocol) integration
|
||||
- Dynamic model switching for specialized tasks
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Real LLM Integration - PydanticAI + Tools
|
||||
|
||||
**Goal**: Connect to actual language models and establish the base plumbing
|
||||
|
||||
**Note**: Ollama is an external service dependency (already running separately)
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **PydanticAI Integration** ✅
|
||||
- PydanticAI → Ollama connection ✅
|
||||
- Agent creation patterns ✅
|
||||
- Streaming response handling ✅
|
||||
- Error handling and retries ✅
|
||||
|
||||
2. **Convert Tatlock Agent** ✅
|
||||
- Convert Tatlock agent from mock to PydanticAI ✅
|
||||
- British butler personality prompt ✅
|
||||
- Research-oriented mindset ✅
|
||||
- Streaming to reasoning output ✅
|
||||
- Tool calling framework setup ✅
|
||||
|
||||
3. **Permanent Tools** ✅
|
||||
- Calculator: Safe mathematical expression evaluation ✅
|
||||
- Date/Time toolkit: Current time, relative dates, time differences ✅
|
||||
- Web search: SearXNG integration (external service) ✅
|
||||
- Tool registration with PydanticAI ✅
|
||||
|
||||
4. **Testing Infrastructure** ✅
|
||||
- Integration tests with real LLM ✅
|
||||
- Tool functionality tests ✅
|
||||
- Response quality validation ✅
|
||||
- 131 tests, 81.78% coverage ✅
|
||||
|
||||
### Success Criteria
|
||||
- [x] **PydanticAI agents can call Ollama** (mistral-nemo:latest)
|
||||
- [x] **Streaming works end-to-end**
|
||||
- [x] **Tool calling framework functional**
|
||||
- [x] **Permanent tools working** (calculator, date/time, search)
|
||||
- [x] **Tests pass with real LLM**
|
||||
- [ ] Can switch models dynamically (e.g., Codestral for code)
|
||||
|
||||
### Status
|
||||
**✅ MOSTLY COMPLETE** - Tatlock agent functional with permanent tools
|
||||
|
||||
### Remaining Work
|
||||
- Dynamic model switching for specialized tasks (e.g., Codestral for coding)
|
||||
|
||||
### Why First?
|
||||
Without real LLM integration, we can't meaningfully implement the Steward/Butler pattern. Everything else depends on having actual AI agents working.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Orchestration Layer - The Steward
|
||||
|
||||
**Goal**: Implement the first-tier LLM call for tool/agent selection
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Orchestrator Framework**
|
||||
- Python orchestrator service/module
|
||||
- Request preprocessing pipeline
|
||||
- Tool/agent registry system
|
||||
- Recommendation format definition
|
||||
|
||||
2. **Steward Agent Implementation**
|
||||
- Steward prompt engineering
|
||||
- Tool selection logic
|
||||
- Agent recommendation generation
|
||||
- Output format (note to Butler)
|
||||
|
||||
3. **Tool Registry**
|
||||
- Available tools catalog
|
||||
- Tool capability descriptions
|
||||
- Tool category organization
|
||||
- Dynamic tool loading
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Steward analyzes incoming requests
|
||||
- [ ] Produces tool/agent recommendations
|
||||
- [ ] Recommendations formatted as prepended note
|
||||
- [ ] Tool registry is queryable and extensible
|
||||
- [ ] Steward output visible in reasoning stream
|
||||
|
||||
### Estimated Effort
|
||||
**3-4 weeks** - Core intelligence routing
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: The Butler - Tatlock Agent
|
||||
|
||||
**Goal**: Implement the second-tier coordinator with personality within the existing Orchestrator infrastructure
|
||||
|
||||
**Context**: The Orchestrator (FastAPI infrastructure) already exists. This phase implements the real Tatlock PydanticAI agent to replace the current mock agent.
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Butler Agent (Tatlock)**
|
||||
- PydanticAI agent implementation within Orchestrator
|
||||
- Personality prompt engineering (witty British butler)
|
||||
- Tool calling framework
|
||||
- Multi-agent coordination logic
|
||||
|
||||
2. **Scoped Tool Access**
|
||||
- Filter tools based on Steward recommendations
|
||||
- Dynamic tool loading for Butler context
|
||||
- Tool execution framework
|
||||
- Result aggregation
|
||||
|
||||
3. **Real-Time Reasoning Output**
|
||||
- Stream all Butler activities to reasoning output
|
||||
- Tool call progress indicators
|
||||
- Expert agent consultation messages
|
||||
- Wait time transparency
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Tatlock receives enriched requests (user + Steward notes)
|
||||
- [ ] Only recommended tools are available
|
||||
- [ ] Tatlock coordinates multiple tool calls
|
||||
- [ ] All actions streamed to reasoning output
|
||||
- [ ] Responses have consistent personality
|
||||
- [ ] Synthesizes multi-source results coherently
|
||||
|
||||
### Estimated Effort
|
||||
**4-5 weeks** - Complex coordination logic
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Expert Household Staff - Core Agents
|
||||
|
||||
**Goal**: Implement the initial set of domain-specific expert agents
|
||||
|
||||
### Priority Expert Agents
|
||||
|
||||
1. **The Librarian** (Research & Knowledge Management) ⭐ **Priority**
|
||||
- Research assistance and synthesis
|
||||
- Automatic research dossier generation
|
||||
- Knowledge base queries and organization
|
||||
- Reference management
|
||||
- Wiki integration (future: dedicated wiki container)
|
||||
- Mind map maintenance (future)
|
||||
- *Rationale: Helps guide development priorities through better research*
|
||||
|
||||
2. **The Developer** (Software Development)
|
||||
- Code generation assistance
|
||||
- Debugging support
|
||||
- Documentation generation
|
||||
- Architecture guidance
|
||||
- *Rationale: Directly supports building the system itself*
|
||||
|
||||
3. **The Handyman** (System Maintenance)
|
||||
- System status queries
|
||||
- Log analysis
|
||||
- Basic troubleshooting
|
||||
- Infrastructure monitoring
|
||||
|
||||
4. **The Secretary** (Scheduling & Organization)
|
||||
- Calendar integration (placeholder)
|
||||
- Task management (placeholder)
|
||||
- Reminder system
|
||||
- Schedule conflict detection
|
||||
|
||||
5. **The Housekeeper** (Home Automation)
|
||||
- Device control interface
|
||||
- Status queries
|
||||
- Automation triggers
|
||||
- Environmental monitoring
|
||||
|
||||
### Each Agent Includes
|
||||
- Specialized prompt and personality
|
||||
- Domain-specific tools
|
||||
- MCP integration points (where applicable)
|
||||
- Integration with Butler orchestration
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Each agent implemented as separate module
|
||||
- [ ] Agents callable via tool framework
|
||||
- [ ] Agents use specialized prompts
|
||||
- [ ] Results integrate cleanly with Butler
|
||||
- [ ] Can invoke specialized models (e.g., Codestral for Developer)
|
||||
|
||||
### Estimated Effort
|
||||
**6-8 weeks** - Parallel development possible
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Persistence Layer - Database & Multi-Tenancy
|
||||
|
||||
**Goal**: Add persistent storage and multi-user support when needed
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **PostgreSQL Integration**
|
||||
- Docker compose configuration for PostgreSQL
|
||||
- Database schema design with tenant isolation
|
||||
- Alembic migrations setup
|
||||
- SQLAlchemy models
|
||||
|
||||
2. **Multi-Tenant Architecture**
|
||||
- Tenant identification middleware
|
||||
- Tenant-scoped database sessions
|
||||
- User authentication system (basic)
|
||||
- Per-tenant data isolation
|
||||
|
||||
3. **Core Data Models**
|
||||
- Users and tenants
|
||||
- Conversations and messages (migrate from in-memory)
|
||||
- Agent interactions log
|
||||
- System configuration and preferences
|
||||
|
||||
4. **Migration Strategy**
|
||||
- Gradual migration from in-memory to database
|
||||
- Backward compatibility during transition
|
||||
- Data export/import utilities
|
||||
|
||||
### Success Criteria
|
||||
- [ ] PostgreSQL container running
|
||||
- [ ] Multiple users can authenticate separately
|
||||
- [ ] Each user sees only their own data
|
||||
- [ ] Conversations persist across restarts
|
||||
- [ ] Database migrations work correctly
|
||||
- [ ] Tests verify tenant isolation
|
||||
|
||||
### Estimated Effort
|
||||
**3-4 weeks** - Data layer foundation
|
||||
|
||||
### Why Later?
|
||||
The core orchestration (Steward → Butler → Experts) can work entirely with in-memory state. We only need database persistence when we want conversations to survive restarts and multiple users to have isolated experiences.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Extended Services Integration
|
||||
|
||||
**Goal**: Connect to additional supporting services
|
||||
|
||||
### Services to Integrate
|
||||
|
||||
1. **Redis (Memory & Caching)**
|
||||
- Docker compose setup
|
||||
- Conversation cache
|
||||
- Short-term memory
|
||||
- Session management
|
||||
|
||||
3. **Qdrant (Vector Storage)**
|
||||
- Docker compose setup
|
||||
- Long-term memory embeddings
|
||||
- Semantic search
|
||||
- Conversation history vectors
|
||||
|
||||
4. **SearxNG (Web Search)**
|
||||
- Docker compose setup
|
||||
- Search tool integration
|
||||
- Result processing
|
||||
- Privacy-preserving queries
|
||||
|
||||
### Success Criteria
|
||||
- [ ] All services defined in docker-compose.yml
|
||||
- [ ] Services communicate correctly
|
||||
- [ ] Tatlock can invoke web search
|
||||
- [ ] Redis used for session data
|
||||
- [ ] Qdrant stores conversation embeddings
|
||||
- [ ] Ollama serves the base model
|
||||
|
||||
### Estimated Effort
|
||||
**3-4 weeks** - Infrastructure setup
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: MCP (Model Context Protocol) Integration
|
||||
|
||||
**Goal**: Enable rich tool integrations via MCP
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **MCP Server Framework**
|
||||
- MCP server implementation
|
||||
- Tool registration via MCP
|
||||
- Schema validation
|
||||
- Error handling
|
||||
|
||||
2. **MCP Client in Agents**
|
||||
- PydanticAI MCP integration
|
||||
- Tool discovery from MCP servers
|
||||
- Dynamic tool loading
|
||||
- Result processing
|
||||
|
||||
3. **Initial MCP Tools**
|
||||
- File system operations
|
||||
- Database queries
|
||||
- API integrations
|
||||
- System commands
|
||||
|
||||
### Success Criteria
|
||||
- [ ] MCP server running
|
||||
- [ ] Tools exposed via MCP protocol
|
||||
- [ ] Agents can discover and use MCP tools
|
||||
- [ ] New tools addable without code changes
|
||||
- [ ] MCP tools visible in Steward recommendations
|
||||
|
||||
### Estimated Effort
|
||||
**3-4 weeks** - Standards-based integration
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Advanced Memory & Context
|
||||
|
||||
**Goal**: Implement sophisticated memory and context management
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Long-Term Memory**
|
||||
- Conversation embedding pipeline
|
||||
- Semantic search over history
|
||||
- Memory consolidation
|
||||
- Relevance ranking
|
||||
|
||||
2. **Context Management**
|
||||
- Smart context window trimming
|
||||
- Conversation branching
|
||||
- Topic tracking
|
||||
- Memory retrieval integration
|
||||
|
||||
3. **Personalization**
|
||||
- User preference learning
|
||||
- Interaction pattern analysis
|
||||
- Adaptive responses
|
||||
- Custom agent personalities per user
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Conversations automatically embedded to Qdrant
|
||||
- [ ] Relevant history retrieved for new requests
|
||||
- [ ] Context stays within model limits
|
||||
- [ ] User preferences affect responses
|
||||
- [ ] Memory improves over time
|
||||
|
||||
### Estimated Effort
|
||||
**4-5 weeks** - AI/ML heavy
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Extended Household Staff
|
||||
|
||||
**Goal**: Add specialized agents for additional domains
|
||||
|
||||
### Future Agents
|
||||
|
||||
1. **The Librarian** (Knowledge Management)
|
||||
- Personal documentation indexing
|
||||
- Research assistance
|
||||
- Knowledge base queries
|
||||
- Reference management
|
||||
|
||||
2. **The Accountant** (Financial Tracking)
|
||||
- Expense tracking
|
||||
- Budget monitoring
|
||||
- Financial reports
|
||||
- Transaction categorization
|
||||
|
||||
3. **The Chef** (Meal Planning)
|
||||
- Recipe management
|
||||
- Meal planning
|
||||
- Nutrition tracking
|
||||
- Grocery lists
|
||||
|
||||
4. **Others as Needed**
|
||||
- Domain-specific as requirements emerge
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Each new agent follows household pattern
|
||||
- [ ] Integrates with Steward/Butler flow
|
||||
- [ ] Has appropriate specialized tools
|
||||
- [ ] Documented in PHILOSOPHY.md updates
|
||||
|
||||
### Estimated Effort
|
||||
**Ongoing** - Add as needed
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: User Experience Refinement
|
||||
|
||||
**Goal**: Polish the interaction experience
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Personality Tuning**
|
||||
- Refine Tatlock's wit and tone
|
||||
- Consistent household character
|
||||
- Cultural references appropriate
|
||||
- Humor that doesn't annoy
|
||||
|
||||
2. **Transparency Improvements**
|
||||
- Better progress indicators
|
||||
- Clearer reasoning explanations
|
||||
- Informative wait messages
|
||||
- Error message clarity
|
||||
|
||||
3. **Performance Optimization**
|
||||
- Response time improvements
|
||||
- Model loading optimization
|
||||
- Caching strategies
|
||||
- Streaming smoothness
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Users find Tatlock engaging
|
||||
- [ ] Wait times feel reasonable
|
||||
- [ ] Errors are understandable
|
||||
- [ ] System feels responsive
|
||||
|
||||
### Estimated Effort
|
||||
**Ongoing** - Continuous improvement
|
||||
|
||||
---
|
||||
|
||||
## Phase 11: Production Hardening
|
||||
|
||||
**Goal**: Make the system production-ready for homelab deployment
|
||||
|
||||
### Deliverables
|
||||
|
||||
1. **Deployment**
|
||||
- Complete docker-compose stack
|
||||
- Environment configuration
|
||||
- Backup strategies
|
||||
- Update procedures
|
||||
|
||||
2. **Monitoring**
|
||||
- Health checks
|
||||
- Performance metrics
|
||||
- Error tracking
|
||||
- Usage analytics
|
||||
|
||||
3. **Security**
|
||||
- Authentication hardening
|
||||
- Rate limiting
|
||||
- Input validation
|
||||
- Audit logging
|
||||
|
||||
4. **Documentation**
|
||||
- Installation guide
|
||||
- Configuration reference
|
||||
- Troubleshooting guide
|
||||
- Architecture documentation
|
||||
|
||||
### Success Criteria
|
||||
- [ ] One-command deployment
|
||||
- [ ] System health is monitorable
|
||||
- [ ] Secure for homelab use
|
||||
- [ ] Well documented
|
||||
|
||||
### Estimated Effort
|
||||
**3-4 weeks** - Production polish
|
||||
|
||||
---
|
||||
|
||||
## Dependencies Between Phases
|
||||
|
||||
```
|
||||
Phase 1 (Ollama + PydanticAI) ← Foundation for all AI
|
||||
↓
|
||||
Phase 2 (Steward)
|
||||
↓
|
||||
Phase 3 (Butler/Tatlock)
|
||||
↓
|
||||
Phase 4 (Expert Agents) ← Phase 7 (MCP) can enhance
|
||||
↓
|
||||
Phase 5 (Database/Multi-Tenancy) ← Can be deferred
|
||||
↓
|
||||
Phase 6 (Extended Services) → Phase 8 (Advanced Memory)
|
||||
↓
|
||||
Phase 9 (Extended Staff) → Phase 10 (UX) → Phase 11 (Production)
|
||||
```
|
||||
|
||||
**Critical Path**: Phases 1 → 2 → 3 → 4 must be sequential
|
||||
**Can Be Deferred**: Phase 5 (Database) until you need persistence
|
||||
**Parallel Opportunities**: Phase 6 and 7 can overlap; Phase 9 and 10 ongoing
|
||||
|
||||
---
|
||||
|
||||
## Overall Timeline Estimate
|
||||
|
||||
**Minimum Viable Household** (Phases 1-4): **15-20 weeks**
|
||||
- Working Steward → Butler → Expert Agents with real LLM
|
||||
- In-memory state (no persistence needed yet)
|
||||
- Core household functional
|
||||
|
||||
**With Persistence** (Phases 1-5): **18-24 weeks**
|
||||
- Add database and multi-tenancy
|
||||
- Conversations survive restarts
|
||||
- Multiple users supported
|
||||
|
||||
**Full-Featured System** (Phases 1-9): **35-45 weeks**
|
||||
- All services integrated
|
||||
- Advanced memory and context
|
||||
- Extended household staff
|
||||
|
||||
**Production-Ready** (All phases): **40-50 weeks**
|
||||
- Polished UX
|
||||
- Hardened for homelab deployment
|
||||
- Fully documented
|
||||
|
||||
*Note: Timeline assumes consistent part-time development effort*
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Technical
|
||||
- System implements PHILOSOPHY.md patterns
|
||||
- All household roles functional
|
||||
- Multi-tenant isolation verified
|
||||
- Real-time reasoning transparency working
|
||||
- MCP integration complete
|
||||
|
||||
### User Experience
|
||||
- Tatlock feels like interacting with a butler
|
||||
- Wait times are transparent and acceptable
|
||||
- Expert agents provide value in their domains
|
||||
- System is reliable and trustworthy
|
||||
|
||||
### Architecture
|
||||
- Clean separation between household roles
|
||||
- Easy to add new agents/tools
|
||||
- Model efficiency (base model stays loaded)
|
||||
- Scales to household + friends usage
|
||||
|
||||
---
|
||||
|
||||
## Risk Management
|
||||
|
||||
### High Risk Items
|
||||
1. **PydanticAI + Ollama integration complexity**
|
||||
- Mitigation: Prototype early, iterate on connection layer
|
||||
|
||||
2. **Multi-agent coordination complexity**
|
||||
- Mitigation: Start simple, add coordination gradually
|
||||
|
||||
3. **Model performance on homelab hardware**
|
||||
- Mitigation: Model selection, quantization, optimization
|
||||
|
||||
4. **Prompt engineering for personality consistency**
|
||||
- Mitigation: Extensive testing, user feedback, iteration
|
||||
|
||||
### Medium Risk Items
|
||||
- MCP protocol adoption and tooling maturity
|
||||
- Vector embedding quality for memory
|
||||
- Home automation integration variability
|
||||
- User authentication security
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Immediate**: Commit model name fix (Tatlock)
|
||||
2. **Week 1-2**: Begin Phase 1 (PostgreSQL + multi-tenancy design)
|
||||
3. **Week 3**: Parallel prototype of Steward agent
|
||||
4. **Ongoing**: Update this roadmap as we learn
|
||||
|
||||
---
|
||||
|
||||
**Document Status**: Active planning document
|
||||
**Created**: 2025-12-06
|
||||
**Last Updated**: 2025-12-06
|
||||
+278
@@ -0,0 +1,278 @@
|
||||
# Tatlock - System Philosophy and Architecture
|
||||
|
||||
## Document Purpose
|
||||
|
||||
This document establishes the foundational philosophy and architectural patterns for the Tatlock system. It represents the **target design** that all development should work towards.
|
||||
|
||||
**When to modify this document**:
|
||||
- When there is a deliberate decision to deviate from these established patterns
|
||||
- When fundamental assumptions about the system's purpose change
|
||||
- When new architectural insights require rethinking core principles
|
||||
|
||||
**When NOT to modify this document**:
|
||||
- During implementation of these patterns (use README.md, AGENTS.md, or code comments for technical details)
|
||||
- For adding new household members or capabilities within the existing pattern
|
||||
- For tactical decisions about specific technologies or tools
|
||||
|
||||
This document should remain stable, serving as the north star for development decisions.
|
||||
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
### Vision
|
||||
|
||||
Tatlock is a comprehensive homelab butler and personal assistant system designed to augment personal and household productivity through intelligent automation, knowledge management, and contextual assistance. Named after a traditional British butler, Tatlock embodies the wit, competence, and organizational skill of a well-run household staff, coordinating a team of specialized expert agents to serve the needs of its users.
|
||||
|
||||
Unlike cloud-dependent AI assistants, Tatlock is built to operate primarily offline, maintaining privacy and control while providing sophisticated assistance across multiple domains of daily life.
|
||||
|
||||
### Purpose
|
||||
|
||||
The system serves as a unified intelligent interface for:
|
||||
|
||||
- **Knowledge Work**: Research assistance, information synthesis, general knowledge queries
|
||||
- **Technical Work**: Software development support, systems administration tasks
|
||||
- **Home Management**: Home automation control and monitoring
|
||||
- **Personal Organization**: Calendaring, scheduling, task management, list keeping
|
||||
- **Information Management**: Personal documentation, note-taking, knowledge base maintenance
|
||||
|
||||
### Core Philosophy
|
||||
|
||||
Tatlock is built on three fundamental principles:
|
||||
|
||||
1. **Privacy-First Architecture**: All processing occurs locally within your homelab environment. Your data, conversations, and personal information never leave your infrastructure unless you explicitly direct it to do so.
|
||||
|
||||
2. **Offline-Capable Operation**: While the system can leverage internet resources when available, core functionality remains operational without external connectivity. This ensures reliability and independence from third-party services.
|
||||
|
||||
3. **Multi-Tenant by Design**: Though primarily intended for personal use (yourself, household members, and close friends), the system architecture supports multiple users with complete data isolation, personalized experiences, and individual preferences.
|
||||
|
||||
### Scope
|
||||
|
||||
**Current Focus**: The initial implementation establishes the foundational architecture with OpenAI-compatible API interfaces, structured response formats, and reasoning transparency. This phase prioritizes:
|
||||
- Core API infrastructure
|
||||
- Response streaming and formatting
|
||||
- Basic conversation management
|
||||
- Testing and validation framework
|
||||
|
||||
**Future Expansion**: The system will evolve into a comprehensive personal assistant platform by integrating:
|
||||
- Specialized containerized services (machine learning, search, storage, memory)
|
||||
- Task and project management capabilities
|
||||
- Calendar and scheduling systems
|
||||
- Home automation integration
|
||||
- Personal knowledge management
|
||||
- Advanced multi-agent collaboration
|
||||
|
||||
### Deployment Model
|
||||
|
||||
Tatlock is designed for **single-instance, multi-user deployment** within a homelab environment:
|
||||
|
||||
- **Users**: Personal use for household members and trusted friends
|
||||
- **Infrastructure**: Self-hosted on your own hardware
|
||||
- **Architecture**: Containerized microservices on a single host
|
||||
- **Data Sovereignty**: Complete control over all data and processing
|
||||
|
||||
This deployment model balances simplicity of operation with the security and personalization needs of a small, trusted user base.
|
||||
|
||||
### System Context
|
||||
|
||||
Tatlock operates as the central orchestration layer within a broader ecosystem of containerized services:
|
||||
|
||||
#### Core Service Stack
|
||||
- **Language Models**: Ollama for local ML inference
|
||||
- **Search**: SearxNG for privacy-respecting web search
|
||||
- **Memory Systems**:
|
||||
- Redis for short-term memory and caching
|
||||
- Qdrant for long-term memory and vector storage
|
||||
- **Data Storage**: PostgreSQL for structured data and multi-tenant isolation
|
||||
- **Future Services**: Calendaring, scheduling, task management, documentation systems
|
||||
|
||||
#### Integration Approach
|
||||
Rather than building monolithic functionality, Tatlock acts as an intelligent coordinator, leveraging specialized services for specific capabilities while maintaining consistent interfaces and user experience.
|
||||
|
||||
### Design Goals
|
||||
|
||||
1. **Unified Experience**: Single point of interaction for diverse personal assistance needs
|
||||
2. **Contextual Intelligence**: Understanding across conversations, tasks, and time
|
||||
3. **Transparent Operation**: Visible reasoning and decision-making processes
|
||||
4. **Extensible Architecture**: Easy integration of new capabilities and services
|
||||
5. **Reliable Performance**: Consistent operation regardless of internet availability
|
||||
6. **User Privacy**: Zero data leakage to external parties
|
||||
7. **Multi-User Support**: Isolated experiences for different household members
|
||||
|
||||
### Success Criteria
|
||||
|
||||
Tatlock succeeds when it becomes the natural first point of interaction for:
|
||||
- Answering questions and conducting research
|
||||
- Managing daily tasks and schedules
|
||||
- Controlling home automation
|
||||
- Supporting development and technical work
|
||||
- Organizing personal information and knowledge
|
||||
|
||||
The system should feel less like "using a tool" and more like "asking a capable assistant" who understands your context, preferences, and needs.
|
||||
|
||||
## The Household Architecture
|
||||
|
||||
### System Layers
|
||||
|
||||
The Tatlock system consists of two distinct architectural layers:
|
||||
|
||||
#### The Orchestrator (Infrastructure Layer)
|
||||
|
||||
The **Orchestrator** is the FastAPI application that provides the technical infrastructure:
|
||||
- HTTP/SSE endpoints (`/v1/responses`, `/v1/chat/completions`)
|
||||
- Streaming coordination and conversation management
|
||||
- Token counting and context window management
|
||||
- Integration with Open WebUI and other clients
|
||||
- Request/response lifecycle management
|
||||
|
||||
This is the "plumbing" layer that exists now and handles all the technical concerns of running an OpenAI-compatible API.
|
||||
|
||||
#### Tatlock - The Butler (Agent Layer)
|
||||
|
||||
**Tatlock** is the PydanticAI agent that provides the intelligence and personality:
|
||||
- The witty British butler persona
|
||||
- Coordination with the Steward and household staff
|
||||
- Multi-agent orchestration and synthesis
|
||||
- Context-aware, personalized responses
|
||||
|
||||
The Orchestrator hosts Tatlock—users interact with "Tatlock" (the advertised model name), but technically they're talking to the Orchestrator infrastructure which routes requests through the Tatlock agent.
|
||||
|
||||
**Current State**: The Orchestrator exists and uses mock agents. Phase 1-3 of the implementation roadmap will integrate the real Tatlock agent using PydanticAI.
|
||||
|
||||
### The British Household Metaphor
|
||||
|
||||
Tatlock adopts the organizational structure of a traditional British estate household, where specialized staff members handle distinct domains of responsibility under the coordination of a capable butler. This metaphor is not merely aesthetic—it reflects a deliberate architectural pattern that enables focused expertise, clear separation of concerns, and efficient coordination.
|
||||
|
||||
### Household Roles
|
||||
|
||||
#### Tatlock - The Butler (Primary Interface)
|
||||
|
||||
**Character**: Witty, capable, and impeccably organized
|
||||
**Role**: Chief coordinator and primary point of contact with users
|
||||
|
||||
Tatlock serves as the face of the system, managing all user interactions with personality and competence. He understands the full context of requests, coordinates with appropriate household staff, synthesizes their contributions, and delivers coherent, thoughtful responses. His wit and personality make interactions engaging while maintaining professionalism.
|
||||
|
||||
**Responsibilities**:
|
||||
- Receiving and understanding user requests
|
||||
- Coordinating with household staff (expert agents)
|
||||
- Synthesizing multi-source information into coherent responses
|
||||
- Maintaining conversation context and user preferences
|
||||
- Presenting results with appropriate personality and tone
|
||||
|
||||
#### The Steward (Request Analysis)
|
||||
|
||||
**Role**: Initial request triage and resource planning
|
||||
|
||||
Before Tatlock engages with a request, the Steward performs crucial preparatory work. The Steward analyzes incoming requests to determine which tools, services, and household staff members will be needed, creating a curated recommendation that streamlines Tatlock's work.
|
||||
|
||||
**Responsibilities**:
|
||||
- Analyzing user requests for required capabilities
|
||||
- Identifying relevant tools and expert agents
|
||||
- Providing recommendations to focus Tatlock's attention
|
||||
- Reducing cognitive load on the Butler by pre-filtering options
|
||||
|
||||
#### Expert Household Staff (Domain Specialists)
|
||||
|
||||
**The Handyman** - System Maintenance and Technical Operations
|
||||
Handles system administration, server management, infrastructure monitoring, and technical troubleshooting.
|
||||
|
||||
**The Housekeeper** - Home Automation Management
|
||||
Controls and monitors home automation systems, environmental controls, security, and physical space management.
|
||||
|
||||
**The Secretary** - Scheduling and Organization
|
||||
Manages calendars, appointments, scheduling conflicts, reminders, and time-based coordination.
|
||||
|
||||
**The Developer** - Software Development Support
|
||||
Assists with code writing, debugging, architecture decisions, documentation, and development workflows.
|
||||
|
||||
**Additional Staff** (Future):
|
||||
- The Librarian - Knowledge management and research
|
||||
- The Accountant - Financial tracking and analysis
|
||||
- The Chef - Meal planning and nutrition
|
||||
- Others as needs emerge
|
||||
|
||||
### The Two-Tier Request Flow
|
||||
|
||||
The household operates through a carefully orchestrated two-tier process:
|
||||
|
||||
#### Tier 1: The Steward's Preparation
|
||||
|
||||
1. **User request arrives** at the Orchestrator (via HTTP API)
|
||||
2. **Orchestrator routes** the raw request to the Steward for analysis
|
||||
3. **Steward determines** which tools and household staff are relevant
|
||||
4. **Steward prepares recommendations**, written as a note to Tatlock
|
||||
5. **Recommendations are prepended** to the user's request
|
||||
|
||||
**Purpose**: This separation ensures that Tatlock isn't overwhelmed with the full universe of available tools and agents. The Steward narrows the scope to only relevant capabilities, making Tatlock's decision-making cleaner and more focused.
|
||||
|
||||
#### Tier 2: Tatlock's Orchestration
|
||||
|
||||
1. **Tatlock receives** the enriched request (original + Steward's notes)
|
||||
2. **Scope is limited** to recommended tools and staff only
|
||||
3. **Tatlock coordinates** with appropriate household members
|
||||
4. **Expert agents perform** their specialized tasks
|
||||
5. **All interactions are streamed** to the reasoning output in real-time
|
||||
6. **Tatlock synthesizes** results into a coherent response
|
||||
7. **User receives** a unified answer from Tatlock
|
||||
|
||||
**Purpose**: This tier focuses on execution and coordination. With a curated set of tools, Tatlock can efficiently orchestrate multiple expert agents, combine their outputs, and present a seamless response to the user.
|
||||
|
||||
**Real-Time Transparency**: Every interaction—whether Tatlock consulting the Handyman, waiting for a database query, or receiving results from the Secretary—is piped directly into the orchestrator's reasoning output. Users see the household at work in real-time, understanding what's happening even when operations take time. This transforms potentially frustrating wait times into engaging insight into the system's thought process.
|
||||
|
||||
### Why This Architecture Works
|
||||
|
||||
#### Focused Expertise
|
||||
Each household member (expert agent) receives highly specific prompts tailored to their domain. Rather than a single overly-broad prompt trying to do everything, specialized agents work within their areas of competence.
|
||||
|
||||
#### Cognitive Load Management
|
||||
By pre-filtering tools and agents, the Steward prevents Tatlock from being overwhelmed with options. This is analogous to how a real butler doesn't personally know every detail of every household operation—they know whom to ask.
|
||||
|
||||
#### Transparent Coordination
|
||||
The Steward's recommendations are visible in the thinking flow, keeping users informed about which household staff are being consulted. This transparency builds trust and understanding.
|
||||
|
||||
#### Composable Capabilities
|
||||
New expert agents can be added to the household without overwhelming the core system. The Steward learns about new staff members and includes them in recommendations when appropriate.
|
||||
|
||||
#### Model Efficiency
|
||||
Rather than requiring a single enormous context window containing all possible tools and capabilities, the system makes targeted calls with focused contexts. This is more efficient and produces better results.
|
||||
|
||||
**Unified Base Model**: All household members—the Steward, Tatlock, and expert agents—use the same base language model by default. This ensures the model stays loaded in VRAM, eliminating loading delays between calls and maximizing response speed.
|
||||
|
||||
**Specialized Models When Needed**: Individual household staff may invoke specialized models for domain-specific tasks when appropriate:
|
||||
- The Developer might use Codestral for complex code generation
|
||||
- Future visual agents might use vision-language models
|
||||
- Future audio agents might use speech-specific models
|
||||
|
||||
The decision to use a specialized model is made by the household member responsible for that domain, based on the specific requirements of their task. This balances efficiency (keeping the base model hot) with capability (accessing specialized models when they provide significant advantage).
|
||||
|
||||
### Personality and Interaction
|
||||
|
||||
While the underlying architecture is sophisticated, users interact solely with **Tatlock**, who maintains a consistent personality:
|
||||
|
||||
- **Witty but helpful**: Responses may include clever observations or light humor
|
||||
- **Competent and organized**: Always knows who to ask and how to coordinate
|
||||
- **Context-aware**: Remembers ongoing conversations and user preferences
|
||||
- **Transparent**: Explains which household staff are being consulted when relevant
|
||||
- **Professional**: Despite the wit, maintains respect and helpfulness
|
||||
|
||||
The user never directly interacts with the Steward or individual expert agents—those are internal household operations that Tatlock manages on their behalf.
|
||||
|
||||
---
|
||||
|
||||
## Document Metadata
|
||||
|
||||
**Document Type**: Architectural Philosophy (Stable)
|
||||
**Purpose**: Establish foundational patterns and guiding principles
|
||||
**Modification Policy**: Only update when deviating from or enhancing core architectural patterns
|
||||
**Version**: 1.0
|
||||
**Established**: 2025-12-06
|
||||
**Project Version**: 0.1.1
|
||||
|
||||
**Related Documents**:
|
||||
- **README.md**: User-facing documentation and usage guide
|
||||
- **AGENTS.md**: LLM agent development guidelines and technical patterns
|
||||
- **CHANGELOG.md**: Version history and implemented features
|
||||
|
||||
---
|
||||
|
||||
*All development should work towards realizing the patterns described in this document.*
|
||||
@@ -1,155 +1,87 @@
|
||||
# Tatlock - OpenAI-Compatible API with Responses API
|
||||
# Tatlock - Your Homelab Butler
|
||||
|
||||
A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. Features a hybrid architecture with chat completions as a compatibility wrapper around the Responses API.
|
||||
> **📖 For the complete system vision and architectural philosophy, see [PHILOSOPHY.md](PHILOSOPHY.md)**
|
||||
|
||||
A privacy-first, offline-capable personal assistant system that coordinates specialized AI agents to help with research, development, home automation, and daily organization.
|
||||
|
||||
## Current Status
|
||||
|
||||
**✅ Production-ready testing API** with OpenAI Responses API format
|
||||
**✅ Open WebUI integration** with reasoning bubbles (`<think>` tags)
|
||||
**✅ Conversation history** with hybrid client/server approach
|
||||
**🚧 PydanticAI integration** prepared for future real LLM connection
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Hybrid API Design
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Client (Open WebUI, etc.) │
|
||||
└────────┬────────────────────────────────┘
|
||||
│
|
||||
├──────────────────────────────────┐
|
||||
│ │
|
||||
v v
|
||||
┌────────────────────┐ ┌──────────────────────┐
|
||||
│ /v1/chat/ │ wrapper │ /v1/responses │
|
||||
│ completions ├─────────>│ (Primary API) │
|
||||
│ │ │ │
|
||||
│ • OpenAI compat │ │ • Reasoning items │
|
||||
│ • <think> tags │ │ • Function calls │
|
||||
│ • Legacy support │ │ • Message items │
|
||||
└────────────────────┘ └──────────┬───────────┘
|
||||
│
|
||||
v
|
||||
┌──────────────────────┐
|
||||
│ Agent Interface │
|
||||
│ │
|
||||
│ • lorem-tester │
|
||||
│ • tatlock (future) │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
**Key Architectural Decisions:**
|
||||
- **Single Source of Truth**: Responses API handles all generation logic
|
||||
- **Chat Completions Wrapper**: Converts Responses output to Chat format with `<think>` tags
|
||||
- **Agent Interface**: Clean abstraction for multiple models (mock and real)
|
||||
- **Hybrid History**: Client sends full context, server optionally tracks conversations
|
||||
- ✅ **Production-ready testing API** with OpenAI Responses API format
|
||||
- ✅ **Open WebUI integration** with reasoning bubbles (`<think>` tags)
|
||||
- ✅ **Conversation history** with auto-generated IDs and context management
|
||||
- ✅ **Tatlock PydanticAI Agent** - Real LLM integration with Ollama + permanent tools
|
||||
- ✅ **Permanent Tools** - Calculator, date/time toolkit, web search (SearXNG)
|
||||
- ✅ **Comprehensive testing** - 131 tests, 81.78% coverage
|
||||
|
||||
## Features
|
||||
|
||||
### Core API
|
||||
- ✅ **Responses API** (`/v1/responses`) - Primary endpoint with structured output
|
||||
- Reasoning items (thinking/extended thinking)
|
||||
- Function call items (tool execution)
|
||||
- Message items (assistant responses)
|
||||
- Streaming and non-streaming modes
|
||||
- ✅ **Chat Completions API** (`/v1/chat/completions`) - Compatibility wrapper
|
||||
- Converts reasoning to `<think>` tags for Open WebUI
|
||||
- Maintains OpenAI-compatible format
|
||||
- Wraps Responses API (single source of truth)
|
||||
- ✅ **Models API** (`/v1/models`) - Lists available models
|
||||
### API Endpoints
|
||||
|
||||
### Advanced Features
|
||||
- ✅ **Conversation History Management**
|
||||
- Hybrid approach: client maintains state, server tracks optionally
|
||||
- Auto-generated conversation IDs from first message hash
|
||||
- Configurable max turns (default: 20)
|
||||
- Placeholder for future vector memory (Qdrant)
|
||||
- ✅ **Context Window Management**
|
||||
- Approximate token counting (~4 chars/token)
|
||||
- Context trimming to fit model limits
|
||||
- Token usage statistics
|
||||
- ✅ **Parameter Validation**
|
||||
- Temperature: 0.0-2.0
|
||||
- Reasoning effort: none, minimal, low, medium, high, xhigh
|
||||
- Max output tokens enforcement
|
||||
- Stop sequences (up to 4)
|
||||
- ✅ **Stop Sequence Detection**
|
||||
- Real-time detection during streaming
|
||||
- Stops generation immediately when encountered
|
||||
- ✅ **Max Tokens Enforcement**
|
||||
- Real-time token counting during streaming
|
||||
- Stops when limit reached
|
||||
- **Responses API** (`/v1/responses`) - OpenAI Responses API format with structured output
|
||||
- Reasoning items for displaying thinking process
|
||||
- Function call items for tool execution
|
||||
- Message items for assistant responses
|
||||
- Streaming and non-streaming support
|
||||
|
||||
### Testing Models
|
||||
- ✅ **lorem-tester** - Full-featured mock agent
|
||||
- Realistic reasoning summaries
|
||||
- Random tool/function call generation
|
||||
- **Chat Completions** (`/v1/chat/completions`) - OpenAI Chat Completions compatibility
|
||||
- Automatic reasoning conversion to `<think>` tags for Open WebUI
|
||||
- Full OpenAI API compatibility
|
||||
- Streaming support
|
||||
|
||||
- **Models** (`/v1/models`) - List available models
|
||||
|
||||
### Advanced Capabilities
|
||||
|
||||
- **Conversation History**: Auto-generated IDs, configurable max turns (default: 20)
|
||||
- **Context Management**: Token counting, automatic trimming, usage statistics
|
||||
- **Parameter Validation**: Temperature (0.0-2.0), reasoning effort levels, max tokens, stop sequences
|
||||
- **Real-time Enforcement**: Stop sequence detection and max token limits during streaming
|
||||
|
||||
### Available Models
|
||||
|
||||
- **lorem-tester**: Full-featured mock agent with realistic behavior
|
||||
- Configurable reasoning effort levels
|
||||
- Random tool/function calls
|
||||
- Error triggers for testing (rate_limit, context_overflow)
|
||||
- Temperature variation
|
||||
- ✅ **tatlock** - Placeholder for real PydanticAI agent
|
||||
|
||||
### Open WebUI Integration
|
||||
- ✅ **Reasoning Display** - Thinking bubbles shown separately from responses
|
||||
- ✅ **Streaming Support** - Smooth word-by-word streaming
|
||||
- ✅ **Error Handling** - Graceful error display
|
||||
- ✅ **Model Selection** - Both models available in dropdown
|
||||
|
||||
## Components
|
||||
|
||||
- **FastAPI**: High-performance web framework
|
||||
- **SSE-Starlette**: Server-Sent Events for streaming
|
||||
- **Pydantic**: Type-safe request/response validation
|
||||
- **Agent Interface**: Abstraction for multiple model backends
|
||||
- **Conversation History**: Server-side tracking with hybrid approach
|
||||
- **Context Window**: Token management and trimming
|
||||
- **Tatlock**: Real PydanticAI agent with butler personality
|
||||
- **LLM Backend**: Ollama (mistral-nemo:latest)
|
||||
- **Personality**: Witty British butler, research-oriented
|
||||
- **Permanent Tools**:
|
||||
- **Calculator**: Safe mathematical expression evaluation (arithmetic, algebra, trigonometry, logarithms)
|
||||
- **Date/Time Toolkit**: Current time, relative dates ("1 week ago"), time differences
|
||||
- **Web Search**: Privacy-preserving search via SearXNG
|
||||
- **Capabilities**: Streaming, reasoning, tool calling
|
||||
- **Phase**: Phase 1 - Basic Integration (full household coordination coming in future phases)
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.12+ (Python 3.12.11 recommended)
|
||||
- No external dependencies for mock API
|
||||
- (Future: Network access for PydanticAI integration)
|
||||
- **Ollama** (for Tatlock agent): Running locally or network-accessible
|
||||
- Download: https://ollama.ai/
|
||||
- Model: `ollama pull mistral-nemo:latest`
|
||||
- **SearXNG** (for web search tool): Optional but recommended
|
||||
- Docker: `docker run -d -p 8087:8080 searxng/searxng`
|
||||
- Or use public instance (less private)
|
||||
|
||||
## Installation
|
||||
## Quick Start
|
||||
|
||||
### 1. Clone the repository
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
# Clone the repository
|
||||
git clone https://git.schweitz.net/jpmschweitzer/tatlock.git
|
||||
cd tatlock
|
||||
```
|
||||
|
||||
### 2. Create a virtual environment
|
||||
|
||||
```bash
|
||||
# Create virtual environment
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||||
```
|
||||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
|
||||
### 3. Install dependencies
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 4. Configure environment (Optional)
|
||||
|
||||
Create a `.env` file for custom configuration:
|
||||
|
||||
```env
|
||||
# API Configuration
|
||||
API_HOST=0.0.0.0
|
||||
API_PORT=8000
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
# Future: Add real LLM configuration here
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Start the server
|
||||
### Run the Server
|
||||
|
||||
```bash
|
||||
uvicorn src.main:app --reload
|
||||
@@ -157,11 +89,11 @@ uvicorn src.main:app --reload
|
||||
|
||||
API available at `http://localhost:8000`
|
||||
|
||||
### API Endpoints
|
||||
## Usage Examples
|
||||
|
||||
#### Responses API (Primary)
|
||||
### Responses API
|
||||
|
||||
OpenAI Responses API format with structured output:
|
||||
Generate a response with reasoning:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/responses \
|
||||
@@ -176,7 +108,6 @@ curl http://localhost:8000/v1/responses \
|
||||
"summary": "auto"
|
||||
},
|
||||
"max_output_tokens": 500,
|
||||
"stop": ["END"],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
@@ -192,22 +123,12 @@ curl http://localhost:8000/v1/responses \
|
||||
"output": [
|
||||
{
|
||||
"type": "reasoning",
|
||||
"id": "reasoning_xyz",
|
||||
"summary": [
|
||||
"Analyzing the user's request...",
|
||||
"Considering quantum mechanics principles..."
|
||||
]
|
||||
"summary": ["Analyzing the request...", "Considering quantum mechanics..."]
|
||||
},
|
||||
{
|
||||
"type": "message",
|
||||
"id": "msg_def456",
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{
|
||||
"type": "output_text",
|
||||
"text": "Quantum computing uses quantum mechanics..."
|
||||
}
|
||||
]
|
||||
"content": [{"type": "output_text", "text": "Quantum computing uses..."}]
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
@@ -219,9 +140,7 @@ curl http://localhost:8000/v1/responses \
|
||||
}
|
||||
```
|
||||
|
||||
#### Chat Completions (Compatibility)
|
||||
|
||||
OpenAI-compatible format with `<think>` tags:
|
||||
### Chat Completions (OpenAI-compatible)
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
@@ -236,66 +155,67 @@ curl http://localhost:8000/v1/chat/completions \
|
||||
}'
|
||||
```
|
||||
|
||||
**Note**: Chat Completions automatically enables reasoning and converts it to `<think>` tags for Open WebUI compatibility.
|
||||
|
||||
#### List Models
|
||||
### List Models
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/models
|
||||
```
|
||||
|
||||
Returns:
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "lorem-tester",
|
||||
"object": "model",
|
||||
"created": 1733529600,
|
||||
"owned_by": "tatlock"
|
||||
},
|
||||
{
|
||||
"id": "tatlock",
|
||||
"object": "model",
|
||||
"created": 1733529600,
|
||||
"owned_by": "tatlock"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Conversation History
|
||||
|
||||
Optional conversation tracking via metadata:
|
||||
Optionally track conversations using metadata:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/responses \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "lorem-tester",
|
||||
"input": [
|
||||
{"role": "user", "content": "Hello"}
|
||||
],
|
||||
"metadata": {
|
||||
"conversation_id": "conv_abc123"
|
||||
}
|
||||
"input": [{"role": "user", "content": "Hello"}],
|
||||
"metadata": {"conversation_id": "conv_abc123"}
|
||||
}'
|
||||
```
|
||||
|
||||
**Hybrid Approach:**
|
||||
- Client MUST send full conversation history in `input` array (OpenAI compatible)
|
||||
- Server optionally tracks via `metadata.conversation_id` (for analytics, future vector memory)
|
||||
- Auto-generates conversation ID from first message hash if not provided
|
||||
**Note**: Client must send full conversation history in `input` array (OpenAI compatible). Server optionally tracks via `metadata.conversation_id` for future features.
|
||||
|
||||
### Interactive Documentation
|
||||
### Using Tatlock with Tools
|
||||
|
||||
- **Swagger UI**: `http://localhost:8000/docs`
|
||||
- **ReDoc**: `http://localhost:8000/redoc`
|
||||
Tatlock automatically uses his permanent tools when appropriate:
|
||||
|
||||
```bash
|
||||
# Mathematical calculation
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": "What is sqrt(144) + 25?"}]
|
||||
}'
|
||||
|
||||
# Date/time queries
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": "What was the date 2 weeks ago?"}]
|
||||
}'
|
||||
|
||||
# Web search for current information
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": "Search for recent Python 3.12 features"}]
|
||||
}'
|
||||
```
|
||||
|
||||
**Tatlock's Tool Usage Philosophy:**
|
||||
- Uses calculator for ALL mathematics (even simple arithmetic)
|
||||
- Uses date/time tools instead of guessing dates
|
||||
- Searches for current/volatile information to verify facts
|
||||
- Maintains a researcher's mindset with tool-assisted verification
|
||||
|
||||
## Open WebUI Integration
|
||||
|
||||
### Docker Networking
|
||||
### Connection
|
||||
|
||||
If running Open WebUI in Docker and API on host:
|
||||
|
||||
@@ -306,71 +226,26 @@ http://172.17.0.1:8000/v1/chat/completions
|
||||
|
||||
### Reasoning Display
|
||||
|
||||
The Chat Completions wrapper automatically:
|
||||
The Chat Completions endpoint automatically:
|
||||
1. Enables reasoning generation
|
||||
2. Converts reasoning items to `<think>` tags
|
||||
3. Streams thinking before the actual response
|
||||
2. Converts reasoning to `<think>` tags
|
||||
3. Streams thinking before the response
|
||||
|
||||
Open WebUI displays this as:
|
||||
- **Thought bubble** showing reasoning steps
|
||||
- **Main response** showing the actual answer
|
||||
Open WebUI displays this as thought bubbles separate from the main response.
|
||||
|
||||
### Testing Error Handling
|
||||
|
||||
Lorem-tester supports error triggers:
|
||||
- **"trigger_rate_limit"** - Simulates rate limit error
|
||||
- **"trigger_context_overflow"** - Simulates context length error
|
||||
Use special triggers in user messages:
|
||||
- `"trigger_rate_limit"` - Simulates rate limit error
|
||||
- `"trigger_context_overflow"` - Simulates context length error
|
||||
|
||||
## Development
|
||||
## API Documentation
|
||||
|
||||
### Project Structure
|
||||
Interactive documentation available at:
|
||||
- **Swagger UI**: `http://localhost:8000/docs`
|
||||
- **ReDoc**: `http://localhost:8000/redoc`
|
||||
|
||||
Following FastAPI best practices with domain-based organization:
|
||||
|
||||
```
|
||||
tatlock/
|
||||
├── src/
|
||||
│ ├── agents/ # Agent interface and implementations
|
||||
│ │ ├── base.py # Abstract AgentInterface
|
||||
│ │ ├── lorem_tester.py # Full-featured mock agent
|
||||
│ │ ├── tatlock.py # Placeholder for real agent
|
||||
│ │ └── registry.py # Model registry
|
||||
│ ├── responses/ # Responses API domain (PRIMARY)
|
||||
│ │ ├── router.py # POST /v1/responses
|
||||
│ │ ├── schemas.py # Request/response models
|
||||
│ │ ├── service.py # Response generation logic
|
||||
│ │ ├── streaming.py # SSE streaming coordinator
|
||||
│ │ ├── history.py # Conversation history management
|
||||
│ │ └── context.py # Context window management
|
||||
│ ├── chat/ # Chat Completions domain (WRAPPER)
|
||||
│ │ ├── router.py # POST /v1/chat/completions
|
||||
│ │ ├── schemas.py # Chat request/response models
|
||||
│ │ ├── service.py # Wraps Responses API
|
||||
│ │ └── constants.py # Chat constants
|
||||
│ ├── models/ # Models listing domain
|
||||
│ │ ├── router.py # GET /v1/models
|
||||
│ │ ├── schemas.py # Model schemas
|
||||
│ │ └── service.py # Model registry access
|
||||
│ ├── core/ # Shared utilities
|
||||
│ │ ├── config.py # Configuration (BaseSettings)
|
||||
│ │ ├── models.py # Custom Pydantic base
|
||||
│ │ ├── exceptions.py # Custom exceptions
|
||||
│ │ └── router.py # Health check endpoints
|
||||
│ └── main.py # Application factory
|
||||
├── tests/ # Comprehensive test suite
|
||||
│ ├── agents/ # Agent tests
|
||||
│ ├── responses/ # Responses API tests
|
||||
│ ├── chat/ # Chat completions tests
|
||||
│ ├── models/ # Models API tests
|
||||
│ └── core/ # Core tests
|
||||
├── requirements.txt # Dependencies (pinned)
|
||||
├── .env # Environment variables
|
||||
├── AGENTS.md # Agent documentation
|
||||
├── CLEANUP_TODO.md # Architecture notes
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
### Testing
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
@@ -379,41 +254,13 @@ pytest
|
||||
# Run with coverage
|
||||
pytest --cov=src --cov-report=term-missing
|
||||
|
||||
# Current coverage: 78.95% (75 tests passing)
|
||||
# Current: 131 tests, 81.78% coverage
|
||||
```
|
||||
|
||||
**Test Organization:**
|
||||
- Unit tests for all components
|
||||
- Integration tests for API endpoints
|
||||
- Streaming tests for SSE functionality
|
||||
- Error handling tests
|
||||
- Advanced features tests (stop sequences, max tokens, validation)
|
||||
|
||||
### Code Style
|
||||
|
||||
- **Async-first**: All I/O operations use async/await
|
||||
- **Type hints**: All functions fully typed
|
||||
- **Pydantic validation**: All request/response validation
|
||||
- **Domain separation**: Clear boundaries between components
|
||||
- **Single responsibility**: Each module has one clear purpose
|
||||
|
||||
## Security
|
||||
|
||||
### Version Locking
|
||||
|
||||
Minor version locking (`>=X.Y,<X.(Y+1)`) for security:
|
||||
- Allows patch updates
|
||||
- Blocks potentially breaking minor updates
|
||||
- All dependencies checked for CVEs (2025-12-06)
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. Never commit `.env` files
|
||||
2. Use environment variables for sensitive config
|
||||
3. Keep dependencies updated monthly
|
||||
4. Validate all inputs with Pydantic
|
||||
5. Use HTTPS in production
|
||||
6. Implement rate limiting
|
||||
**Test Categories:**
|
||||
- Unit tests: Agent tools, streaming, schemas
|
||||
- Integration tests: Full API stack with real Ollama calls
|
||||
- End-to-end tests: Chat completions, responses API
|
||||
|
||||
## Deployment
|
||||
|
||||
@@ -424,51 +271,96 @@ Minor version locking (`>=X.Y,<X.(Y+1)`) for security:
|
||||
uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4
|
||||
```
|
||||
|
||||
### Considerations
|
||||
### Recommendations
|
||||
|
||||
- Use reverse proxy (nginx/caddy) for HTTPS
|
||||
- Enable rate limiting (SlowAPI or similar)
|
||||
- Enable rate limiting
|
||||
- Set up monitoring and logging
|
||||
- Configure resource limits
|
||||
- Use process manager (systemd/supervisor)
|
||||
|
||||
## Configuration
|
||||
|
||||
Create a `.env` file for custom configuration:
|
||||
|
||||
```env
|
||||
# API Configuration
|
||||
API_HOST=0.0.0.0
|
||||
API_PORT=8000
|
||||
|
||||
# Ollama Configuration
|
||||
OLLAMA_HOST=http://localhost:11434
|
||||
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
|
||||
OLLAMA_TIMEOUT=120
|
||||
|
||||
# SearXNG Configuration (for web search tool)
|
||||
SEARXNG_HOST=http://localhost:8087
|
||||
SEARXNG_TIMEOUT=30
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
# CORS (default: allow all)
|
||||
CORS_ORIGINS=["*"]
|
||||
```
|
||||
|
||||
See `.env.example` for full configuration options.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Streaming not working:**
|
||||
### Streaming not working
|
||||
- Verify SSE-Starlette is installed
|
||||
- Check client supports Server-Sent Events
|
||||
- Test with: `pytest tests/responses/ -k streaming`
|
||||
|
||||
**Open WebUI can't connect:**
|
||||
### Open WebUI can't connect
|
||||
- Use Docker bridge gateway IP: `172.17.0.1:8000`
|
||||
- Check firewall settings
|
||||
- Verify server is running on `0.0.0.0`
|
||||
|
||||
**Tests failing:**
|
||||
- Install test dependencies: `pip install -r requirements-dev.txt`
|
||||
- Activate virtual environment
|
||||
- Run with verbose: `pytest -v`
|
||||
|
||||
**Reasoning not showing:**
|
||||
### Reasoning not showing
|
||||
- Ensure using Chat Completions endpoint (auto-enables reasoning)
|
||||
- Or manually enable in Responses API: `"reasoning": {"effort": "medium", "summary": "auto"}`
|
||||
- Check Open WebUI version supports `<think>` tags
|
||||
|
||||
## Future Roadmap
|
||||
### Tatlock agent errors
|
||||
- Verify Ollama is running: `curl http://localhost:11434/api/tags`
|
||||
- Check model is downloaded: `ollama list`
|
||||
- Review environment variables: `OLLAMA_HOST`, `OLLAMA_DEFAULT_MODEL`
|
||||
- Check logs: `tail -f logs/server.log`
|
||||
|
||||
### Short-term
|
||||
- [ ] Connect tatlock model to real PydanticAI agent
|
||||
- [ ] Implement vector memory (Qdrant integration)
|
||||
- [ ] Add authentication/API keys
|
||||
- [ ] Rate limiting middleware
|
||||
### Web search not working
|
||||
- Verify SearXNG is running: `curl http://localhost:8087/`
|
||||
- Check `SEARXNG_HOST` environment variable
|
||||
- SearXNG is optional - Tatlock will note if search is unavailable
|
||||
|
||||
### Long-term
|
||||
- [ ] Multi-model support (OpenAI, Anthropic, etc.)
|
||||
- [ ] Advanced conversation memory
|
||||
- [ ] Tool/function calling integration
|
||||
- [ ] Usage tracking and analytics
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
tatlock/
|
||||
├── src/
|
||||
│ ├── agents/ # Agent interface and implementations
|
||||
│ │ ├── base.py # AgentInterface abstract class
|
||||
│ │ ├── lorem_tester.py # Mock agent for testing
|
||||
│ │ ├── tatlock.py # Real PydanticAI butler agent
|
||||
│ │ ├── tools.py # Permanent tools (calculator, date/time, search)
|
||||
│ │ └── registry.py # Model registry
|
||||
│ ├── responses/ # Responses API (primary endpoint)
|
||||
│ ├── chat/ # Chat Completions wrapper
|
||||
│ ├── models/ # Models listing
|
||||
│ ├── core/ # Shared utilities and config
|
||||
│ └── main.py # Application entry point
|
||||
├── tests/ # Comprehensive test suite (131 tests)
|
||||
├── AGENTS.md # LLM agent development guidelines
|
||||
├── PHILOSOPHY.md # System vision and architecture
|
||||
├── IMPLEMENTATION_ROADMAP.md # Development phases
|
||||
├── CHANGELOG.md # Version history
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
For LLM agent development guidelines and architectural decisions, see [AGENTS.md](AGENTS.md).
|
||||
|
||||
## Contributing
|
||||
|
||||
@@ -480,17 +372,24 @@ uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4
|
||||
|
||||
## Documentation
|
||||
|
||||
- **AGENTS.md**: Agent architecture and best practices
|
||||
- **CLEANUP_TODO.md**: Architecture decisions and future considerations
|
||||
- **CHANGELOG.md**: Version history
|
||||
- OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses
|
||||
- FastAPI: https://fastapi.tiangolo.com/
|
||||
- PydanticAI: https://ai.pydantic.dev/
|
||||
- **System Philosophy**: [PHILOSOPHY.md](PHILOSOPHY.md) - Vision, goals, and architectural patterns
|
||||
- **User Guide**: This file - Installation, usage, and examples
|
||||
- **Developer Guidelines**: [AGENTS.md](AGENTS.md) - LLM agent development patterns
|
||||
- **Version History**: [CHANGELOG.md](CHANGELOG.md) - Changes and releases
|
||||
|
||||
### External References
|
||||
- **OpenAI Responses API**: https://platform.openai.com/docs/api-reference/responses
|
||||
- **FastAPI**: https://fastapi.tiangolo.com/
|
||||
- **PydanticAI**: https://ai.pydantic.dev/
|
||||
|
||||
## License
|
||||
|
||||
[Add your license here]
|
||||
|
||||
## Version
|
||||
|
||||
Current version: **0.2.0** - PydanticAI Integration with Permanent Tools
|
||||
|
||||
---
|
||||
|
||||
**Note**: This is a testing/development API with mock responses. The architecture is production-ready and designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).
|
||||
**Note**: This is a production-ready testing API with mock responses. The architecture is designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).
|
||||
|
||||
@@ -37,9 +37,9 @@ class ModelRegistry:
|
||||
"owned_by": "tatlock",
|
||||
# Capabilities are retrieved from agent instance
|
||||
},
|
||||
"tatlock": {
|
||||
"Tatlock": {
|
||||
"agent_class": TatlockAgent,
|
||||
"description": "Tatlock reasoning agent (placeholder - not yet implemented)",
|
||||
"description": "Tatlock - Your homelab butler (British household coordinator)",
|
||||
"created": 1733529600, # 2025-12-06
|
||||
"owned_by": "tatlock",
|
||||
# Capabilities are retrieved from agent instance
|
||||
|
||||
+292
-34
@@ -1,17 +1,27 @@
|
||||
"""
|
||||
Tatlock agent - Placeholder for future real agent.
|
||||
Tatlock agent - The Butler (PydanticAI implementation).
|
||||
|
||||
This is a minimal placeholder implementation. In the future, this will
|
||||
be the production agent using PydanticAI and Ollama for real LLM inference.
|
||||
|
||||
For now, it returns a simple placeholder message to show up in the
|
||||
model list and allow basic testing.
|
||||
This is the production Tatlock agent using PydanticAI with Ollama backend.
|
||||
The agent embodies a witty, capable British butler personality.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import secrets
|
||||
from typing import AsyncGenerator, Any
|
||||
|
||||
from pydantic_ai import Agent, RunContext
|
||||
|
||||
from src.agents.base import AgentInterface, OutputItem
|
||||
from src.agents.tools import (
|
||||
calculate,
|
||||
get_current_datetime,
|
||||
calculate_time_offset,
|
||||
time_difference,
|
||||
search_web,
|
||||
)
|
||||
from src.core.config import config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def generate_id() -> str:
|
||||
@@ -19,17 +29,195 @@ def generate_id() -> str:
|
||||
return secrets.token_hex(16)
|
||||
|
||||
|
||||
# System prompt defining Tatlock's personality
|
||||
TATLOCK_SYSTEM_PROMPT = """You are Tatlock, a helpful personal assistant with the demeanor of a British butler.
|
||||
|
||||
Address users as "sir" and maintain a formal yet personable tone. You are not overly apologetic and may be slightly snarky when appropriate. If an opportunity for a pun presents itself, you cannot resist.
|
||||
|
||||
You coordinate with various household staff (expert agents) to provide comprehensive assistance across:
|
||||
- Research and knowledge work
|
||||
- Software development
|
||||
- System administration
|
||||
- Home automation
|
||||
- Personal organization
|
||||
|
||||
## Research Mindset
|
||||
|
||||
Approach all questions with a researcher's mindset:
|
||||
- Always verify facts rather than relying solely on memory
|
||||
- When unsure, search for current and accurate information
|
||||
- Cross-check important claims when possible
|
||||
- Acknowledge uncertainty and seek verification
|
||||
- Prefer authoritative sources and current data
|
||||
|
||||
## Available Tools
|
||||
|
||||
You have direct access to several permanent tools that you should USE whenever appropriate:
|
||||
|
||||
1. **Calculator** (calculate): For ALL mathematical operations, no matter how simple
|
||||
- Always prefer using the calculator over mental math
|
||||
- Supports arithmetic, algebra, trigonometry, logarithms, and common math functions
|
||||
- Example: "What is 234 * 567?" -> Use calculate("234 * 567")
|
||||
|
||||
2. **Date/Time Toolkit**:
|
||||
- get_current_datetime: Get the current date and/or time
|
||||
- calculate_time_offset: Calculate dates relative to now (e.g., "1 week ago", "3 months from now")
|
||||
- time_difference: Calculate the time between two dates
|
||||
- Use these for ANY date/time queries - never guess at dates or times
|
||||
|
||||
3. **Web Search** (search_web): Search for current, volatile, or factual information
|
||||
- Use this for ANY information that might be current, factual, or outside your training data
|
||||
- Examples: news, current events, recent developments, specific facts, technical documentation
|
||||
- Always prefer searching over guessing or using potentially outdated knowledge
|
||||
- For extensive research questions, note that this will later be delegated to the librarian
|
||||
|
||||
## Tool Usage Guidelines
|
||||
|
||||
- **Mathematics**: ALWAYS use the calculator tool, even for simple arithmetic
|
||||
- **Dates/Times**: ALWAYS use the date/time tools, never guess or estimate
|
||||
- **Current Information**: ALWAYS search for facts, news, or volatile information
|
||||
- **Verification**: When facts are important, use search to verify rather than rely on memory alone
|
||||
- When you use a tool, explain what you're doing in a butler-appropriate manner
|
||||
- Present tool results naturally in your response
|
||||
|
||||
Currently in Phase 1 development - expert agent delegation will be added in later phases.
|
||||
"""
|
||||
|
||||
|
||||
class TatlockAgent(AgentInterface):
|
||||
"""
|
||||
Placeholder for future Tatlock reasoning agent.
|
||||
Tatlock - The Butler agent using PydanticAI with Ollama.
|
||||
|
||||
TODO: Integrate PydanticAI and Ollama for real LLM inference
|
||||
TODO: Implement memory modules
|
||||
TODO: Implement expert modules
|
||||
TODO: Add reasoning/thinking capabilities
|
||||
TODO: Add tool/function calling
|
||||
This is the production implementation of the Tatlock personality,
|
||||
currently in Phase 1 (basic LLM integration without expert agents).
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Tatlock configuration (lazy agent creation)."""
|
||||
# Store Ollama configuration
|
||||
self.ollama_host = str(config.OLLAMA_HOST)
|
||||
self.model_name = config.OLLAMA_DEFAULT_MODEL
|
||||
self._agent = None # Lazy initialization
|
||||
|
||||
def _ensure_agent(self):
|
||||
"""Ensure the PydanticAI agent is initialized (lazy initialization)."""
|
||||
if self._agent is not None:
|
||||
return
|
||||
|
||||
logger.info(f"Initializing Tatlock agent with Ollama at {self.ollama_host}, model: {self.model_name}")
|
||||
|
||||
# Import required classes for Ollama configuration
|
||||
from pydantic_ai.models.openai import OpenAIChatModel
|
||||
from pydantic_ai.providers.ollama import OllamaProvider
|
||||
|
||||
# PydanticAI expects Ollama base URL to end with /v1
|
||||
# Remove trailing slash from ollama_host if present
|
||||
clean_host = self.ollama_host.rstrip('/')
|
||||
base_url = f"{clean_host}/v1"
|
||||
|
||||
# Create Ollama model with provider
|
||||
ollama_model = OpenAIChatModel(
|
||||
model_name=self.model_name,
|
||||
provider=OllamaProvider(base_url=base_url)
|
||||
)
|
||||
|
||||
# Create PydanticAI agent with Ollama model
|
||||
self._agent = Agent(
|
||||
ollama_model,
|
||||
system_prompt=TATLOCK_SYSTEM_PROMPT,
|
||||
)
|
||||
|
||||
# Register tools with the agent
|
||||
self._register_tools()
|
||||
|
||||
def _register_tools(self):
|
||||
"""Register permanent tools with the PydanticAI agent."""
|
||||
|
||||
# Calculator tool
|
||||
@self._agent.tool
|
||||
def calculate_math(ctx: RunContext[None], expression: str) -> str:
|
||||
"""
|
||||
Evaluate mathematical expressions safely.
|
||||
|
||||
Use this for ALL mathematical calculations, no matter how simple.
|
||||
|
||||
Args:
|
||||
expression: Mathematical expression (e.g., "2 + 2", "sqrt(16)", "pi * 2")
|
||||
|
||||
Returns:
|
||||
String result of the calculation
|
||||
"""
|
||||
return calculate(expression)
|
||||
|
||||
# Current date/time tool
|
||||
@self._agent.tool
|
||||
def get_current_time(ctx: RunContext[None], format_str: str = "full") -> str:
|
||||
"""
|
||||
Get the current date and time.
|
||||
|
||||
Args:
|
||||
format_str: Output format ("full", "date", "time", "iso", or custom strftime format)
|
||||
|
||||
Returns:
|
||||
Formatted current datetime string
|
||||
"""
|
||||
return get_current_datetime(format_str)
|
||||
|
||||
# Time offset calculator
|
||||
@self._agent.tool
|
||||
def calculate_date_offset(ctx: RunContext[None], offset_description: str) -> str:
|
||||
"""
|
||||
Calculate a date/time relative to now.
|
||||
|
||||
Args:
|
||||
offset_description: Natural language time offset (e.g., "1 week ago", "2 days from now")
|
||||
|
||||
Returns:
|
||||
Formatted datetime string (YYYY-MM-DD HH:MM:SS)
|
||||
"""
|
||||
return calculate_time_offset(offset_description)
|
||||
|
||||
# Time difference calculator
|
||||
@self._agent.tool
|
||||
def calculate_time_difference(ctx: RunContext[None], date1_str: str, date2_str: str = "now") -> str:
|
||||
"""
|
||||
Calculate the difference between two dates.
|
||||
|
||||
Args:
|
||||
date1_str: First date (YYYY-MM-DD or YYYY-MM-DD HH:MM:SS)
|
||||
date2_str: Second date or "now" for current time (default: "now")
|
||||
|
||||
Returns:
|
||||
Human-readable description of the time difference
|
||||
"""
|
||||
return time_difference(date1_str, date2_str)
|
||||
|
||||
# Web search tool
|
||||
@self._agent.tool
|
||||
async def web_search(ctx: RunContext[None], query: str, num_results: int = 5) -> str:
|
||||
"""
|
||||
Search the web using SearXNG for current information.
|
||||
|
||||
Use this tool for ANY information that might be:
|
||||
- Current or time-sensitive (news, events, recent developments)
|
||||
- Factual and verifiable (statistics, technical specs, definitions)
|
||||
- Outside your training data or knowledge cutoff
|
||||
|
||||
Args:
|
||||
query: Search query string
|
||||
num_results: Number of results to return (default: 5, max: 10)
|
||||
|
||||
Returns:
|
||||
Formatted search results with titles, URLs, and snippets
|
||||
"""
|
||||
return await search_web(query, num_results)
|
||||
|
||||
@property
|
||||
def agent(self):
|
||||
"""Get the PydanticAI agent, initializing it if needed."""
|
||||
self._ensure_agent()
|
||||
return self._agent
|
||||
|
||||
async def generate_response(
|
||||
self,
|
||||
messages: list[dict],
|
||||
@@ -41,38 +229,108 @@ class TatlockAgent(AgentInterface):
|
||||
**kwargs: Any
|
||||
) -> AsyncGenerator[OutputItem, None]:
|
||||
"""
|
||||
Generate minimal placeholder response.
|
||||
Generate response using PydanticAI with Ollama.
|
||||
|
||||
In the future, this will call PydanticAI with Ollama backend.
|
||||
Args:
|
||||
messages: Conversation history in OpenAI format
|
||||
reasoning: Reasoning configuration (if requested)
|
||||
tools: Available tools (not yet implemented)
|
||||
temperature: Sampling temperature
|
||||
max_tokens: Maximum tokens to generate
|
||||
stop: Stop sequences
|
||||
**kwargs: Additional parameters
|
||||
|
||||
Yields:
|
||||
OutputItem: Response items (reasoning, message)
|
||||
"""
|
||||
try:
|
||||
# Extract user message from messages
|
||||
# For now, use the last user message as the prompt
|
||||
user_message = ""
|
||||
for msg in reversed(messages):
|
||||
if msg.get("role") == "user":
|
||||
user_message = msg.get("content", "")
|
||||
break
|
||||
|
||||
# Simple placeholder message
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=f"msg_{generate_id()}",
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": "Tatlock agent is not yet implemented. Please use lorem-tester for testing.",
|
||||
"annotations": []
|
||||
}],
|
||||
status="completed"
|
||||
)
|
||||
if not user_message:
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=f"msg_{generate_id()}",
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": "I'm afraid I didn't receive a message, sir. How may I assist you?",
|
||||
"annotations": []
|
||||
}],
|
||||
status="completed"
|
||||
)
|
||||
return
|
||||
|
||||
# Generate reasoning output if requested
|
||||
if reasoning and reasoning.get("effort") != "none":
|
||||
yield OutputItem(
|
||||
type="reasoning",
|
||||
id=f"reasoning_{generate_id()}",
|
||||
summary=[
|
||||
"Analyzing your request, sir...",
|
||||
"Formulating response based on available knowledge..."
|
||||
],
|
||||
thinking="", # PydanticAI doesn't expose internal reasoning yet
|
||||
status="completed"
|
||||
)
|
||||
|
||||
# Stream the agent response token-by-token
|
||||
msg_id = f"msg_{generate_id()}"
|
||||
final_text = ""
|
||||
|
||||
# Use run() instead of run_stream() to avoid GeneratorExit issues
|
||||
# with async context managers inside generators
|
||||
# The StreamingCoordinator will handle word-by-word streaming
|
||||
result = await self.agent.run(user_message)
|
||||
final_text = result.output
|
||||
|
||||
# Yield the complete message
|
||||
# The StreamingCoordinator will break this into word-by-word deltas
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=msg_id,
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": final_text,
|
||||
"annotations": []
|
||||
}],
|
||||
status="completed"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error generating response: {e}", exc_info=True)
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=f"msg_{generate_id()}",
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": f"My apologies, sir. I encountered an error: {str(e)}",
|
||||
"annotations": []
|
||||
}],
|
||||
status="failed"
|
||||
)
|
||||
|
||||
async def supports_tools(self) -> bool:
|
||||
"""Tools not yet implemented."""
|
||||
return False
|
||||
"""Permanent tools now available."""
|
||||
return True
|
||||
|
||||
async def supports_reasoning(self) -> bool:
|
||||
"""Reasoning not yet implemented."""
|
||||
return False
|
||||
"""Basic reasoning support via summary."""
|
||||
return True
|
||||
|
||||
async def get_capabilities(self) -> dict:
|
||||
"""Return minimal capabilities."""
|
||||
"""Return current capabilities."""
|
||||
return {
|
||||
"streaming": True, # Basic streaming works
|
||||
"reasoning": False, # Not yet implemented
|
||||
"tools": False, # Not yet implemented
|
||||
"streaming": True, # Streaming implemented
|
||||
"reasoning": True, # Basic reasoning summaries
|
||||
"tools": True, # Permanent tools: calculator, date/time, search
|
||||
"vision": False, # Future
|
||||
"audio": False, # Future
|
||||
}
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
"""
|
||||
Tatlock's permanent tools.
|
||||
|
||||
These tools are always available to the butler agent:
|
||||
- Calculator: For all mathematical operations
|
||||
- Date/Time toolkit: For current time and time calculations
|
||||
- SearXNG search: For searching the web for current information
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from src.core.config import config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Calculator Tool
|
||||
# ============================================================================
|
||||
|
||||
def calculate(expression: str) -> str:
|
||||
"""
|
||||
Safely evaluate mathematical expressions.
|
||||
|
||||
Supports:
|
||||
- Basic arithmetic: +, -, *, /, //, %, **
|
||||
- Parentheses for grouping
|
||||
- Common math functions: sqrt, sin, cos, tan, log, exp, etc.
|
||||
- Constants: pi, e
|
||||
|
||||
Args:
|
||||
expression: Mathematical expression to evaluate (e.g., "2 + 2", "sqrt(16)", "pi * 2")
|
||||
|
||||
Returns:
|
||||
String result of the calculation or error message
|
||||
|
||||
Examples:
|
||||
calculate("2 + 2") -> "4"
|
||||
calculate("sqrt(16) + 10") -> "14.0"
|
||||
calculate("pi * 2") -> "6.283185307179586"
|
||||
"""
|
||||
try:
|
||||
# Clean the expression
|
||||
expression = expression.strip()
|
||||
|
||||
# Create safe namespace with math functions
|
||||
safe_dict = {
|
||||
# Basic math functions
|
||||
'sqrt': math.sqrt,
|
||||
'pow': math.pow,
|
||||
'abs': abs,
|
||||
'round': round,
|
||||
|
||||
# Trigonometric
|
||||
'sin': math.sin,
|
||||
'cos': math.cos,
|
||||
'tan': math.tan,
|
||||
'asin': math.asin,
|
||||
'acos': math.acos,
|
||||
'atan': math.atan,
|
||||
|
||||
# Logarithmic
|
||||
'log': math.log,
|
||||
'log10': math.log10,
|
||||
'log2': math.log2,
|
||||
'exp': math.exp,
|
||||
|
||||
# Other
|
||||
'ceil': math.ceil,
|
||||
'floor': math.floor,
|
||||
'factorial': math.factorial,
|
||||
|
||||
# Constants
|
||||
'pi': math.pi,
|
||||
'e': math.e,
|
||||
}
|
||||
|
||||
# Evaluate the expression safely
|
||||
result = eval(expression, {"__builtins__": {}}, safe_dict)
|
||||
|
||||
# Format result nicely
|
||||
if isinstance(result, float):
|
||||
# Remove unnecessary decimal places
|
||||
if result.is_integer():
|
||||
return str(int(result))
|
||||
return str(round(result, 10))
|
||||
|
||||
return str(result)
|
||||
|
||||
except ZeroDivisionError:
|
||||
return "Error: Division by zero"
|
||||
except Exception as e:
|
||||
return f"Error calculating '{expression}': {str(e)}"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Date/Time Toolkit
|
||||
# ============================================================================
|
||||
|
||||
def get_current_datetime(format_str: str = "full") -> str:
|
||||
"""
|
||||
Get the current date and time.
|
||||
|
||||
Args:
|
||||
format_str: Output format
|
||||
- "full": Full datetime with timezone (default)
|
||||
- "date": Just the date (YYYY-MM-DD)
|
||||
- "time": Just the time (HH:MM:SS)
|
||||
- "iso": ISO 8601 format
|
||||
- Custom strftime format string
|
||||
|
||||
Returns:
|
||||
Formatted current datetime string
|
||||
|
||||
Examples:
|
||||
get_current_datetime("full") -> "2024-01-15 14:30:45"
|
||||
get_current_datetime("date") -> "2024-01-15"
|
||||
get_current_datetime("time") -> "14:30:45"
|
||||
"""
|
||||
now = datetime.now()
|
||||
|
||||
if format_str == "full":
|
||||
return now.strftime("%Y-%m-%d %H:%M:%S")
|
||||
elif format_str == "date":
|
||||
return now.strftime("%Y-%m-%d")
|
||||
elif format_str == "time":
|
||||
return now.strftime("%H:%M:%S")
|
||||
elif format_str == "iso":
|
||||
return now.isoformat()
|
||||
else:
|
||||
# Custom format
|
||||
try:
|
||||
return now.strftime(format_str)
|
||||
except Exception as e:
|
||||
return f"Error formatting date: {str(e)}"
|
||||
|
||||
|
||||
def calculate_time_offset(offset_description: str) -> str:
|
||||
"""
|
||||
Calculate a date/time relative to now.
|
||||
|
||||
Args:
|
||||
offset_description: Natural language description of time offset
|
||||
Examples: "1 week ago", "2 days from now", "3 months ago",
|
||||
"1 year from now", "5 hours ago"
|
||||
|
||||
Returns:
|
||||
Formatted datetime string (YYYY-MM-DD HH:MM:SS) or error message
|
||||
|
||||
Examples:
|
||||
calculate_time_offset("1 week ago") -> "2024-01-08 14:30:45"
|
||||
calculate_time_offset("2 days from now") -> "2024-01-17 14:30:45"
|
||||
calculate_time_offset("3 months ago") -> "2023-10-15 14:30:45"
|
||||
"""
|
||||
try:
|
||||
now = datetime.now()
|
||||
|
||||
# Parse the offset description
|
||||
# Pattern: "N unit(s) ago/from now"
|
||||
pattern = r'(\d+)\s+(second|minute|hour|day|week|month|year)s?\s+(ago|from\s+now)'
|
||||
match = re.match(pattern, offset_description.lower().strip())
|
||||
|
||||
if not match:
|
||||
return f"Error: Cannot parse '{offset_description}'. Use format like '1 week ago' or '2 days from now'"
|
||||
|
||||
amount = int(match.group(1))
|
||||
unit = match.group(2)
|
||||
direction = match.group(3)
|
||||
|
||||
# Calculate the offset
|
||||
if direction == "ago":
|
||||
amount = -amount
|
||||
|
||||
if unit == "second":
|
||||
target = now + timedelta(seconds=amount)
|
||||
elif unit == "minute":
|
||||
target = now + timedelta(minutes=amount)
|
||||
elif unit == "hour":
|
||||
target = now + timedelta(hours=amount)
|
||||
elif unit == "day":
|
||||
target = now + timedelta(days=amount)
|
||||
elif unit == "week":
|
||||
target = now + timedelta(weeks=amount)
|
||||
elif unit == "month":
|
||||
# Approximate month as 30 days
|
||||
target = now + timedelta(days=amount * 30)
|
||||
elif unit == "year":
|
||||
# Approximate year as 365 days
|
||||
target = now + timedelta(days=amount * 365)
|
||||
else:
|
||||
return f"Error: Unknown time unit '{unit}'"
|
||||
|
||||
return target.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
except Exception as e:
|
||||
return f"Error calculating time offset: {str(e)}"
|
||||
|
||||
|
||||
def time_difference(date1_str: str, date2_str: str = "now") -> str:
|
||||
"""
|
||||
Calculate the difference between two dates.
|
||||
|
||||
Args:
|
||||
date1_str: First date (YYYY-MM-DD or YYYY-MM-DD HH:MM:SS)
|
||||
date2_str: Second date or "now" for current time (default: "now")
|
||||
|
||||
Returns:
|
||||
Human-readable description of the time difference
|
||||
|
||||
Examples:
|
||||
time_difference("2024-01-01", "now") -> "14 days, 14 hours"
|
||||
time_difference("2024-01-01", "2024-01-15") -> "14 days"
|
||||
"""
|
||||
try:
|
||||
# Parse date1
|
||||
if len(date1_str) == 10: # YYYY-MM-DD
|
||||
date1 = datetime.strptime(date1_str, "%Y-%m-%d")
|
||||
else:
|
||||
date1 = datetime.strptime(date1_str, "%Y-%m-%d %H:%M:%S")
|
||||
|
||||
# Parse date2
|
||||
if date2_str.lower() == "now":
|
||||
date2 = datetime.now()
|
||||
elif len(date2_str) == 10:
|
||||
date2 = datetime.strptime(date2_str, "%Y-%m-%d")
|
||||
else:
|
||||
date2 = datetime.strptime(date2_str, "%Y-%m-%d %H:%M:%S")
|
||||
|
||||
# Calculate difference
|
||||
diff = abs(date2 - date1)
|
||||
|
||||
# Format human-readable
|
||||
days = diff.days
|
||||
seconds = diff.seconds
|
||||
hours = seconds // 3600
|
||||
minutes = (seconds % 3600) // 60
|
||||
|
||||
parts = []
|
||||
if days > 0:
|
||||
parts.append(f"{days} day{'s' if days != 1 else ''}")
|
||||
if hours > 0:
|
||||
parts.append(f"{hours} hour{'s' if hours != 1 else ''}")
|
||||
if minutes > 0 and days == 0: # Only show minutes if less than a day
|
||||
parts.append(f"{minutes} minute{'s' if minutes != 1 else ''}")
|
||||
|
||||
if not parts:
|
||||
return "Less than a minute"
|
||||
|
||||
return ", ".join(parts)
|
||||
|
||||
except Exception as e:
|
||||
return f"Error calculating time difference: {str(e)}"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# SearXNG Search Tool
|
||||
# ============================================================================
|
||||
|
||||
async def search_web(query: str, num_results: int = 5) -> str:
|
||||
"""
|
||||
Search the web using SearXNG.
|
||||
|
||||
Args:
|
||||
query: Search query string
|
||||
num_results: Number of results to return (default: 5, max: 10)
|
||||
|
||||
Returns:
|
||||
Formatted search results as a string with titles, URLs, and snippets
|
||||
|
||||
Examples:
|
||||
search_web("Python async programming") -> "1. Title: ...\n URL: ...\n ..."
|
||||
"""
|
||||
try:
|
||||
# Limit results
|
||||
num_results = min(num_results, 10)
|
||||
|
||||
# Get SearXNG host with fallback logic
|
||||
searxng_host = str(config.SEARXNG_HOST)
|
||||
|
||||
# Try production host first, fall back to localhost in development
|
||||
hosts_to_try = [searxng_host]
|
||||
if config.ENVIRONMENT.value == "development" and "localhost" not in searxng_host:
|
||||
# Add localhost fallback for development
|
||||
hosts_to_try.append("http://localhost:8087")
|
||||
|
||||
last_error = None
|
||||
|
||||
for host in hosts_to_try:
|
||||
try:
|
||||
logger.info(f"Attempting SearXNG search at {host}")
|
||||
|
||||
async with httpx.AsyncClient(timeout=config.SEARXNG_TIMEOUT) as client:
|
||||
response = await client.get(
|
||||
f"{host}/search",
|
||||
params={
|
||||
"q": query,
|
||||
"format": "json",
|
||||
"pageno": 1,
|
||||
}
|
||||
)
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
results = data.get("results", [])
|
||||
|
||||
if not results:
|
||||
return f"No results found for '{query}'"
|
||||
|
||||
# Format results
|
||||
formatted_results = []
|
||||
for i, result in enumerate(results[:num_results], 1):
|
||||
title = result.get("title", "No title")
|
||||
url = result.get("url", "")
|
||||
content = result.get("content", "No description available")
|
||||
|
||||
formatted_results.append(
|
||||
f"{i}. {title}\n"
|
||||
f" URL: {url}\n"
|
||||
f" {content}\n"
|
||||
)
|
||||
|
||||
return "\n".join(formatted_results)
|
||||
else:
|
||||
last_error = f"SearXNG returned status {response.status_code}"
|
||||
|
||||
except httpx.ConnectError:
|
||||
last_error = f"Cannot connect to SearXNG at {host}"
|
||||
logger.warning(f"SearXNG connection failed at {host}, trying next host if available")
|
||||
continue
|
||||
except Exception as e:
|
||||
last_error = str(e)
|
||||
logger.warning(f"SearXNG error at {host}: {e}")
|
||||
continue
|
||||
|
||||
# All hosts failed
|
||||
return f"Error searching: {last_error}. Please check that SearXNG is running."
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error in search_web: {e}", exc_info=True)
|
||||
return f"Error searching: {str(e)}"
|
||||
+7
-4
@@ -214,9 +214,12 @@ async def create_chat_completion_stream(
|
||||
in_reasoning = False
|
||||
|
||||
elif item.type == "message":
|
||||
# Stream message content word by word
|
||||
# Stream message content in chunks (preserves newlines, markdown, etc.)
|
||||
text = item.data["content"][0]["text"]
|
||||
for word in text.split():
|
||||
chunk_size = 50 # characters per chunk
|
||||
|
||||
for i in range(0, len(text), chunk_size):
|
||||
chunk = text[i:i+chunk_size]
|
||||
yield ChatCompletionChunk(
|
||||
id=completion_id,
|
||||
object=constants.CHAT_COMPLETION_CHUNK_OBJECT,
|
||||
@@ -225,12 +228,12 @@ async def create_chat_completion_stream(
|
||||
choices=[
|
||||
ChatCompletionChunkChoice(
|
||||
index=0,
|
||||
delta=ChatCompletionChunkDelta(content=f"{word} "),
|
||||
delta=ChatCompletionChunkDelta(content=chunk),
|
||||
finish_reason=None,
|
||||
)
|
||||
],
|
||||
)
|
||||
await asyncio.sleep(0.05) # Simulate typing
|
||||
await asyncio.sleep(0.02) # Faster since chunks are larger
|
||||
|
||||
# Final chunk with finish_reason
|
||||
yield ChatCompletionChunk(
|
||||
|
||||
+11
-1
@@ -32,7 +32,7 @@ class Config(BaseSettings):
|
||||
|
||||
# Application
|
||||
APP_NAME: str = "OpenAI-Compatible API"
|
||||
APP_VERSION: str = "0.1.1"
|
||||
APP_VERSION: str = "0.2.0"
|
||||
ENVIRONMENT: Environment = Environment.DEVELOPMENT
|
||||
DEBUG: bool = Field(default=False, description="Debug mode")
|
||||
|
||||
@@ -59,6 +59,16 @@ class Config(BaseSettings):
|
||||
description="Timeout for each streaming turn in seconds"
|
||||
)
|
||||
|
||||
# SearXNG Configuration
|
||||
SEARXNG_HOST: HttpUrl = Field(
|
||||
default="http://localhost:8087",
|
||||
description="SearXNG server URL"
|
||||
)
|
||||
SEARXNG_TIMEOUT: int = Field(
|
||||
default=30,
|
||||
description="SearXNG request timeout in seconds"
|
||||
)
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL: str = Field(default="INFO", description="Logging level")
|
||||
|
||||
|
||||
+40
-33
@@ -140,6 +140,7 @@ class StreamingCoordinator:
|
||||
import asyncio
|
||||
|
||||
output_items = []
|
||||
last_message_text = "" # Track last streamed message text to compute deltas
|
||||
|
||||
try:
|
||||
# Strip pipeline prefix if present (e.g., "pipeline.model" -> "model")
|
||||
@@ -189,45 +190,51 @@ class StreamingCoordinator:
|
||||
yield FunctionCallDone()
|
||||
|
||||
elif item.type == "message":
|
||||
# Stream output text with stop sequence and max tokens enforcement
|
||||
text = item.data["content"][0]["text"]
|
||||
words = text.split()
|
||||
# Get current accumulated text from agent
|
||||
current_text = item.data["content"][0]["text"]
|
||||
|
||||
# Track accumulated text and tokens for enforcement
|
||||
accumulated_text = ""
|
||||
output_tokens = 0
|
||||
# Only stream the NEW text (delta) since last update
|
||||
if current_text.startswith(last_message_text):
|
||||
# Extract only the new portion
|
||||
delta_text = current_text[len(last_message_text):]
|
||||
|
||||
for word in words:
|
||||
# Add word to accumulated text
|
||||
word_with_space = f"{word} "
|
||||
accumulated_text += word_with_space
|
||||
if delta_text:
|
||||
# Stream the delta text in chunks while preserving formatting
|
||||
# (newlines, markdown, code blocks, etc.)
|
||||
chunk_size = 50 # characters per chunk
|
||||
|
||||
# Check stop sequences
|
||||
stop_found, text_before_stop = self._check_stop_sequence(
|
||||
accumulated_text,
|
||||
request.stop
|
||||
)
|
||||
for i in range(0, len(delta_text), chunk_size):
|
||||
chunk = delta_text[i:i+chunk_size]
|
||||
|
||||
if stop_found:
|
||||
# Emit final text before stop sequence
|
||||
remaining_text = text_before_stop[len(accumulated_text) - len(word_with_space):]
|
||||
if remaining_text:
|
||||
yield OutputTextDelta(delta=remaining_text)
|
||||
yield OutputTextDone()
|
||||
break
|
||||
# Check stop sequences on full accumulated text
|
||||
stop_found, text_before_stop = self._check_stop_sequence(
|
||||
current_text,
|
||||
request.stop
|
||||
)
|
||||
|
||||
# Check max tokens
|
||||
output_tokens = self._count_tokens_approx(accumulated_text)
|
||||
if self._check_max_tokens(output_tokens, request.max_output_tokens):
|
||||
# Max tokens reached - stop streaming
|
||||
yield OutputTextDone()
|
||||
break
|
||||
if stop_found:
|
||||
# Only emit remaining delta before stop
|
||||
remaining = text_before_stop[len(last_message_text):]
|
||||
if remaining:
|
||||
yield OutputTextDelta(delta=remaining)
|
||||
yield OutputTextDone()
|
||||
break
|
||||
|
||||
# Normal streaming
|
||||
yield OutputTextDelta(delta=word_with_space)
|
||||
await asyncio.sleep(0.05) # Simulate typing
|
||||
else:
|
||||
# Completed normally without stop/limit
|
||||
# Check max tokens on full text
|
||||
output_tokens = self._count_tokens_approx(current_text)
|
||||
if self._check_max_tokens(output_tokens, request.max_output_tokens):
|
||||
yield OutputTextDone()
|
||||
break
|
||||
|
||||
# Normal streaming of delta chunk (preserves all formatting)
|
||||
yield OutputTextDelta(delta=chunk)
|
||||
await asyncio.sleep(0.02) # Shorter delay since chunks are larger
|
||||
|
||||
# Update tracking variable
|
||||
last_message_text = current_text
|
||||
|
||||
# If this is the final message (status=completed), ensure we send done
|
||||
if item.data.get("status") == "completed":
|
||||
yield OutputTextDone()
|
||||
|
||||
# Final response.done event with complete response
|
||||
|
||||
@@ -22,7 +22,7 @@ async def test_list_models():
|
||||
# Check model IDs
|
||||
model_ids = [m["id"] for m in models]
|
||||
assert "lorem-tester" in model_ids
|
||||
assert "tatlock" in model_ids
|
||||
assert "Tatlock" in model_ids
|
||||
|
||||
# Check structure
|
||||
for model in models:
|
||||
@@ -57,16 +57,16 @@ async def test_lorem_tester_capabilities():
|
||||
async def test_tatlock_capabilities():
|
||||
"""Test tatlock model capabilities."""
|
||||
models = await ModelRegistry.list_models()
|
||||
tatlock_model = next(m for m in models if m["id"] == "tatlock")
|
||||
tatlock_model = next(m for m in models if m["id"] == "Tatlock")
|
||||
|
||||
capabilities = tatlock_model["capabilities"]
|
||||
|
||||
# Tatlock is placeholder - minimal capabilities
|
||||
# Tatlock Phase 1 - basic streaming, reasoning, and permanent tools
|
||||
assert capabilities["streaming"] is True
|
||||
assert capabilities["reasoning"] is False # Not yet
|
||||
assert capabilities["tools"] is False # Not yet
|
||||
assert capabilities["vision"] is False
|
||||
assert capabilities["audio"] is False
|
||||
assert capabilities["reasoning"] is True # Basic reasoning summaries
|
||||
assert capabilities["tools"] is True # Permanent tools: calculator, date/time, search
|
||||
assert capabilities["vision"] is False # Future
|
||||
assert capabilities["audio"] is False # Future
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
@@ -80,7 +80,7 @@ def test_get_agent_lorem_tester():
|
||||
@pytest.mark.unit
|
||||
def test_get_agent_tatlock():
|
||||
"""Test getting tatlock agent instance."""
|
||||
agent = ModelRegistry.get_agent("tatlock")
|
||||
agent = ModelRegistry.get_agent("Tatlock")
|
||||
|
||||
assert isinstance(agent, TatlockAgent)
|
||||
|
||||
@@ -98,7 +98,7 @@ def test_get_agent_not_found():
|
||||
def test_model_exists():
|
||||
"""Test checking if model exists."""
|
||||
assert ModelRegistry.model_exists("lorem-tester") is True
|
||||
assert ModelRegistry.model_exists("tatlock") is True
|
||||
assert ModelRegistry.model_exists("Tatlock") is True
|
||||
assert ModelRegistry.model_exists("nonexistent") is False
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,371 @@
|
||||
"""
|
||||
Tests for Tatlock's permanent tools (calculator, date/time, search).
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from datetime import datetime
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
from src.agents.tools import (
|
||||
calculate,
|
||||
get_current_datetime,
|
||||
calculate_time_offset,
|
||||
time_difference,
|
||||
search_web,
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Calculator Tests
|
||||
# ============================================================================
|
||||
|
||||
class TestCalculator:
|
||||
"""Tests for the calculator tool."""
|
||||
|
||||
def test_basic_arithmetic(self):
|
||||
"""Test basic arithmetic operations."""
|
||||
assert calculate("2 + 2") == "4"
|
||||
assert calculate("10 - 3") == "7"
|
||||
assert calculate("5 * 6") == "30"
|
||||
assert calculate("20 / 4") == "5" # Integer result, no decimal
|
||||
|
||||
def test_complex_expressions(self):
|
||||
"""Test complex mathematical expressions."""
|
||||
assert calculate("(2 + 3) * 4") == "20"
|
||||
assert calculate("10 ** 2") == "100"
|
||||
assert calculate("17 % 5") == "2"
|
||||
|
||||
def test_math_functions(self):
|
||||
"""Test mathematical functions."""
|
||||
assert calculate("sqrt(16)") == "4" # Integer result
|
||||
assert calculate("abs(-5)") == "5"
|
||||
assert calculate("round(3.7)") == "4"
|
||||
|
||||
# Test with constants
|
||||
result = calculate("pi * 2")
|
||||
assert "6.28" in result # Approximately 6.283...
|
||||
|
||||
def test_trigonometry(self):
|
||||
"""Test trigonometric functions."""
|
||||
result = calculate("sin(0)")
|
||||
assert result == "0" # Integer result
|
||||
|
||||
# cos(0) should be 1
|
||||
result = calculate("cos(0)")
|
||||
assert result == "1" # Integer result
|
||||
|
||||
def test_logarithms(self):
|
||||
"""Test logarithmic functions."""
|
||||
result = calculate("log10(100)")
|
||||
assert result == "2" # Integer result
|
||||
|
||||
result = calculate("exp(0)")
|
||||
assert result == "1" # Integer result
|
||||
|
||||
def test_error_handling(self):
|
||||
"""Test error handling for invalid expressions."""
|
||||
result = calculate("1 / 0")
|
||||
assert "Error: Division by zero" in result
|
||||
|
||||
result = calculate("invalid_function(5)")
|
||||
assert "Error calculating" in result
|
||||
|
||||
def test_integer_results(self):
|
||||
"""Test that integer results don't show unnecessary decimals."""
|
||||
assert calculate("4.0 + 6.0") == "10"
|
||||
assert calculate("sqrt(9)") == "3"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Date/Time Tests
|
||||
# ============================================================================
|
||||
|
||||
class TestDateTime:
|
||||
"""Tests for date/time toolkit."""
|
||||
|
||||
def test_get_current_datetime_full(self):
|
||||
"""Test getting full current datetime."""
|
||||
result = get_current_datetime("full")
|
||||
# Should match format YYYY-MM-DD HH:MM:SS
|
||||
assert len(result) == 19
|
||||
assert result[4] == "-"
|
||||
assert result[7] == "-"
|
||||
assert result[10] == " "
|
||||
assert result[13] == ":"
|
||||
assert result[16] == ":"
|
||||
|
||||
def test_get_current_datetime_date(self):
|
||||
"""Test getting current date only."""
|
||||
result = get_current_datetime("date")
|
||||
# Should match format YYYY-MM-DD
|
||||
assert len(result) == 10
|
||||
assert result[4] == "-"
|
||||
assert result[7] == "-"
|
||||
|
||||
# Verify it's a valid date
|
||||
datetime.strptime(result, "%Y-%m-%d")
|
||||
|
||||
def test_get_current_datetime_time(self):
|
||||
"""Test getting current time only."""
|
||||
result = get_current_datetime("time")
|
||||
# Should match format HH:MM:SS
|
||||
assert len(result) == 8
|
||||
assert result[2] == ":"
|
||||
assert result[5] == ":"
|
||||
|
||||
def test_get_current_datetime_iso(self):
|
||||
"""Test getting ISO format."""
|
||||
result = get_current_datetime("iso")
|
||||
# Should be parseable as ISO format
|
||||
datetime.fromisoformat(result)
|
||||
|
||||
def test_calculate_time_offset_days(self):
|
||||
"""Test calculating time offsets in days."""
|
||||
result = calculate_time_offset("1 day ago")
|
||||
assert len(result) == 19 # YYYY-MM-DD HH:MM:SS
|
||||
|
||||
result = calculate_time_offset("2 days from now")
|
||||
assert len(result) == 19
|
||||
|
||||
def test_calculate_time_offset_weeks(self):
|
||||
"""Test calculating time offsets in weeks."""
|
||||
result = calculate_time_offset("1 week ago")
|
||||
assert len(result) == 19
|
||||
|
||||
result = calculate_time_offset("2 weeks from now")
|
||||
assert len(result) == 19
|
||||
|
||||
def test_calculate_time_offset_months(self):
|
||||
"""Test calculating time offsets in months."""
|
||||
result = calculate_time_offset("1 month ago")
|
||||
assert len(result) == 19
|
||||
|
||||
result = calculate_time_offset("3 months from now")
|
||||
assert len(result) == 19
|
||||
|
||||
def test_calculate_time_offset_years(self):
|
||||
"""Test calculating time offsets in years."""
|
||||
result = calculate_time_offset("1 year ago")
|
||||
assert len(result) == 19
|
||||
|
||||
result = calculate_time_offset("2 years from now")
|
||||
assert len(result) == 19
|
||||
|
||||
def test_calculate_time_offset_hours(self):
|
||||
"""Test calculating time offsets in hours."""
|
||||
result = calculate_time_offset("5 hours ago")
|
||||
assert len(result) == 19
|
||||
|
||||
result = calculate_time_offset("3 hours from now")
|
||||
assert len(result) == 19
|
||||
|
||||
def test_calculate_time_offset_invalid(self):
|
||||
"""Test error handling for invalid time offsets."""
|
||||
result = calculate_time_offset("invalid input")
|
||||
assert "Error" in result
|
||||
assert "Cannot parse" in result
|
||||
|
||||
def test_time_difference(self):
|
||||
"""Test calculating time difference."""
|
||||
result = time_difference("2024-01-01", "2024-01-15")
|
||||
assert "14 day" in result
|
||||
|
||||
def test_time_difference_with_now(self):
|
||||
"""Test time difference with 'now'."""
|
||||
# Get today's date
|
||||
today = datetime.now().strftime("%Y-%m-%d")
|
||||
result = time_difference(today, "now")
|
||||
# Should be less than a day
|
||||
assert "Less than" in result or "hour" in result or "minute" in result
|
||||
|
||||
def test_time_difference_with_times(self):
|
||||
"""Test time difference with full timestamps."""
|
||||
result = time_difference("2024-01-01 10:00:00", "2024-01-01 14:30:00")
|
||||
assert "4 hour" in result
|
||||
assert "30 minute" in result
|
||||
|
||||
def test_time_difference_error(self):
|
||||
"""Test error handling for invalid dates."""
|
||||
result = time_difference("invalid-date", "now")
|
||||
assert "Error" in result
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Search Tests
|
||||
# ============================================================================
|
||||
|
||||
class TestSearch:
|
||||
"""Tests for web search tool."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_success(self):
|
||||
"""Test successful web search."""
|
||||
mock_response = {
|
||||
"results": [
|
||||
{
|
||||
"title": "Test Result 1",
|
||||
"url": "https://example.com/1",
|
||||
"content": "This is a test result"
|
||||
},
|
||||
{
|
||||
"title": "Test Result 2",
|
||||
"url": "https://example.com/2",
|
||||
"content": "Another test result"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
with patch("src.agents.tools.httpx.AsyncClient") as mock_client_class:
|
||||
# Create mock response
|
||||
mock_response_obj = type('MockResponse', (), {
|
||||
'status_code': 200,
|
||||
'json': lambda *args, **kwargs: mock_response
|
||||
})()
|
||||
|
||||
# Create mock client with async get method
|
||||
async def mock_get(*args, **kwargs):
|
||||
return mock_response_obj
|
||||
|
||||
mock_client_instance = type('MockClient', (), {
|
||||
'get': mock_get
|
||||
})()
|
||||
|
||||
# Setup async context manager
|
||||
async def mock_aenter(*args, **kwargs):
|
||||
return mock_client_instance
|
||||
|
||||
async def mock_aexit(*args, **kwargs):
|
||||
return None
|
||||
|
||||
mock_client_class.return_value.__aenter__ = mock_aenter
|
||||
mock_client_class.return_value.__aexit__ = mock_aexit
|
||||
|
||||
result = await search_web("test query", num_results=2)
|
||||
|
||||
assert "Test Result 1" in result
|
||||
assert "https://example.com/1" in result
|
||||
assert "Test Result 2" in result
|
||||
assert "https://example.com/2" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_no_results(self):
|
||||
"""Test web search with no results."""
|
||||
mock_response_data = {"results": []}
|
||||
|
||||
with patch("src.agents.tools.httpx.AsyncClient") as mock_client_class:
|
||||
mock_response_obj = type('MockResponse', (), {
|
||||
'status_code': 200,
|
||||
'json': lambda *args, **kwargs: mock_response_data
|
||||
})()
|
||||
|
||||
async def mock_get(*args, **kwargs):
|
||||
return mock_response_obj
|
||||
|
||||
mock_client_instance = type('MockClient', (), {
|
||||
'get': mock_get
|
||||
})()
|
||||
|
||||
async def mock_aenter(*args, **kwargs):
|
||||
return mock_client_instance
|
||||
|
||||
async def mock_aexit(*args, **kwargs):
|
||||
return None
|
||||
|
||||
mock_client_class.return_value.__aenter__ = mock_aenter
|
||||
mock_client_class.return_value.__aexit__ = mock_aexit
|
||||
|
||||
result = await search_web("test query")
|
||||
|
||||
assert "No results found" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_connection_error(self):
|
||||
"""Test web search with connection error."""
|
||||
with patch("httpx.AsyncClient") as mock_client:
|
||||
mock_client_instance = AsyncMock()
|
||||
mock_client_instance.get.side_effect = Exception("Connection failed")
|
||||
mock_client.return_value.__aenter__.return_value = mock_client_instance
|
||||
|
||||
result = await search_web("test query")
|
||||
|
||||
assert "Error searching" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_limits_results(self):
|
||||
"""Test that search limits results to max 10."""
|
||||
mock_response_data = {
|
||||
"results": [
|
||||
{"title": f"Result {i}", "url": f"https://example.com/{i}", "content": "Test"}
|
||||
for i in range(20)
|
||||
]
|
||||
}
|
||||
|
||||
with patch("src.agents.tools.httpx.AsyncClient") as mock_client_class:
|
||||
mock_response_obj = type('MockResponse', (), {
|
||||
'status_code': 200,
|
||||
'json': lambda *args, **kwargs: mock_response_data
|
||||
})()
|
||||
|
||||
async def mock_get(*args, **kwargs):
|
||||
return mock_response_obj
|
||||
|
||||
mock_client_instance = type('MockClient', (), {
|
||||
'get': mock_get
|
||||
})()
|
||||
|
||||
async def mock_aenter(*args, **kwargs):
|
||||
return mock_client_instance
|
||||
|
||||
async def mock_aexit(*args, **kwargs):
|
||||
return None
|
||||
|
||||
mock_client_class.return_value.__aenter__ = mock_aenter
|
||||
mock_client_class.return_value.__aexit__ = mock_aexit
|
||||
|
||||
result = await search_web("test query", num_results=15)
|
||||
|
||||
# Should only return 10 results (max limit)
|
||||
result_count = result.count("URL:")
|
||||
assert result_count == 10
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_formats_results(self):
|
||||
"""Test that search results are properly formatted."""
|
||||
mock_response_data = {
|
||||
"results": [
|
||||
{
|
||||
"title": "Test Title",
|
||||
"url": "https://example.com",
|
||||
"content": "Test content description"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
with patch("src.agents.tools.httpx.AsyncClient") as mock_client_class:
|
||||
mock_response_obj = type('MockResponse', (), {
|
||||
'status_code': 200,
|
||||
'json': lambda *args, **kwargs: mock_response_data
|
||||
})()
|
||||
|
||||
async def mock_get(*args, **kwargs):
|
||||
return mock_response_obj
|
||||
|
||||
mock_client_instance = type('MockClient', (), {
|
||||
'get': mock_get
|
||||
})()
|
||||
|
||||
async def mock_aenter(*args, **kwargs):
|
||||
return mock_client_instance
|
||||
|
||||
async def mock_aexit(*args, **kwargs):
|
||||
return None
|
||||
|
||||
mock_client_class.return_value.__aenter__ = mock_aenter
|
||||
mock_client_class.return_value.__aexit__ = mock_aexit
|
||||
|
||||
result = await search_web("test query")
|
||||
|
||||
# Check formatting
|
||||
assert "1. Test Title" in result
|
||||
assert "URL: https://example.com" in result
|
||||
assert "Test content description" in result
|
||||
@@ -46,7 +46,7 @@ def test_chat_completion_non_streaming(
|
||||
def test_chat_completion_validation_error(client: TestClient) -> None:
|
||||
"""Test chat completion with invalid request."""
|
||||
# Missing required field 'messages'
|
||||
invalid_request = {"model": "tatlock"}
|
||||
invalid_request = {"model": "Tatlock"}
|
||||
|
||||
response = client.post("/v1/chat/completions", json=invalid_request)
|
||||
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ async def async_client() -> AsyncClient:
|
||||
def mock_chat_request() -> dict:
|
||||
"""Standard chat completion request fixture."""
|
||||
return {
|
||||
"model": "tatlock",
|
||||
"model": "Tatlock",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, world!"}
|
||||
],
|
||||
|
||||
@@ -0,0 +1,410 @@
|
||||
"""
|
||||
Integration tests for Tatlock agent streaming through full API stack.
|
||||
|
||||
These tests verify the complete streaming flow from API endpoint through
|
||||
StreamingCoordinator to TatlockAgent, ensuring no text duplication and
|
||||
proper delta calculation.
|
||||
"""
|
||||
import json
|
||||
import pytest
|
||||
from httpx import AsyncClient
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.asyncio
|
||||
async def test_tatlock_streaming_no_duplication(async_client: AsyncClient):
|
||||
"""
|
||||
Integration test: Verify Tatlock streaming produces no text duplication.
|
||||
|
||||
This test catches the bug where accumulated text from PydanticAI was
|
||||
being re-streamed multiple times by the StreamingCoordinator.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Say hello"}],
|
||||
"stream": True
|
||||
}
|
||||
|
||||
collected_deltas = []
|
||||
|
||||
async with async_client.stream(
|
||||
"POST",
|
||||
"/v1/responses",
|
||||
json=request_data,
|
||||
timeout=30.0, # Give enough time for Ollama response
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
assert response.headers["content-type"] == "text/event-stream; charset=utf-8"
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
if line.startswith("event: "):
|
||||
event_type = line[7:].strip()
|
||||
elif line.startswith("data: "):
|
||||
data_str = line[6:].strip()
|
||||
if data_str != "[DONE]":
|
||||
try:
|
||||
chunk = json.loads(data_str)
|
||||
|
||||
# Collect output text deltas
|
||||
if chunk.get("event") == "response.output_text.delta":
|
||||
collected_deltas.append(chunk["delta"])
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Reconstruct full text from deltas
|
||||
full_text = "".join(collected_deltas)
|
||||
|
||||
# Verify we got some response
|
||||
assert len(full_text) > 0, "Should have received some text"
|
||||
|
||||
# Verify no obvious duplication patterns
|
||||
# Check that common words don't appear excessively repeated
|
||||
words = full_text.lower().split()
|
||||
if len(words) > 0:
|
||||
# Check for consecutive duplicate words (sign of duplication bug)
|
||||
consecutive_dupes = sum(
|
||||
1 for i in range(len(words) - 1)
|
||||
if words[i] == words[i + 1] and len(words[i]) > 3
|
||||
)
|
||||
# Allow a few duplicates (natural language), but not excessive
|
||||
assert consecutive_dupes < len(words) * 0.1, \
|
||||
f"Too many consecutive duplicate words: {consecutive_dupes}/{len(words)}"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.asyncio
|
||||
async def test_tatlock_chat_streaming_no_duplication(async_client: AsyncClient):
|
||||
"""
|
||||
Integration test: Verify Tatlock streaming through Chat Completions API.
|
||||
|
||||
Tests the full stack through the chat completions wrapper to ensure
|
||||
streaming works correctly without duplication.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"stream": True
|
||||
}
|
||||
|
||||
collected_content = []
|
||||
|
||||
async with async_client.stream(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
json=request_data,
|
||||
timeout=30.0,
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
if line.startswith("data: "):
|
||||
data_str = line[6:].strip()
|
||||
if data_str == "[DONE]":
|
||||
break
|
||||
|
||||
try:
|
||||
chunk = json.loads(data_str)
|
||||
|
||||
# Collect content deltas from choices
|
||||
if "choices" in chunk and len(chunk["choices"]) > 0:
|
||||
delta = chunk["choices"][0].get("delta", {})
|
||||
if "content" in delta and delta["content"]:
|
||||
collected_content.append(delta["content"])
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Reconstruct full response
|
||||
full_response = "".join(collected_content)
|
||||
|
||||
# Verify we got a response
|
||||
assert len(full_response) > 0, "Should have received response content"
|
||||
|
||||
# Check for duplication patterns
|
||||
words = full_response.lower().split()
|
||||
if len(words) > 0:
|
||||
consecutive_dupes = sum(
|
||||
1 for i in range(len(words) - 1)
|
||||
if words[i] == words[i + 1] and len(words[i]) > 3
|
||||
)
|
||||
assert consecutive_dupes < len(words) * 0.1, \
|
||||
f"Too many consecutive duplicate words in chat response: {consecutive_dupes}/{len(words)}"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_tatlock_non_streaming_responses_api(client: TestClient):
|
||||
"""
|
||||
Integration test: Verify Tatlock non-streaming through Responses API.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Say hello"}],
|
||||
"stream": False
|
||||
}
|
||||
|
||||
response = client.post("/v1/responses", json=request_data, timeout=30.0)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
|
||||
# Verify response structure
|
||||
assert data["status"] == "completed"
|
||||
assert "output" in data
|
||||
assert len(data["output"]) > 0
|
||||
|
||||
# Get the message content
|
||||
message_item = next((item for item in data["output"] if item["type"] == "message"), None)
|
||||
assert message_item is not None, "Should have a message output item"
|
||||
assert len(message_item["content"]) > 0
|
||||
|
||||
text = message_item["content"][0]["text"]
|
||||
assert len(text) > 0, "Should have response text"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_tatlock_non_streaming_chat_api(client: TestClient):
|
||||
"""
|
||||
Integration test: Verify Tatlock non-streaming through Chat Completions API.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"stream": False
|
||||
}
|
||||
|
||||
response = client.post("/v1/chat/completions", json=request_data, timeout=30.0)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
|
||||
# Verify OpenAI-compatible structure
|
||||
assert "id" in data
|
||||
assert data["object"] == "chat.completion"
|
||||
assert "choices" in data
|
||||
assert len(data["choices"]) > 0
|
||||
|
||||
# Verify content
|
||||
choice = data["choices"][0]
|
||||
assert choice["message"]["role"] == "assistant"
|
||||
assert len(choice["message"]["content"]) > 0
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.asyncio
|
||||
async def test_tatlock_streaming_delta_accumulation(async_client: AsyncClient):
|
||||
"""
|
||||
Integration test: Verify deltas accumulate correctly without duplication.
|
||||
|
||||
This test explicitly checks that when we accumulate all deltas,
|
||||
we get a coherent response without repeated text.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Count to three"}],
|
||||
"stream": True
|
||||
}
|
||||
|
||||
collected_deltas = []
|
||||
previous_full_text = ""
|
||||
|
||||
async with async_client.stream(
|
||||
"POST",
|
||||
"/v1/responses",
|
||||
json=request_data,
|
||||
timeout=30.0,
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
if line.startswith("data: "):
|
||||
data_str = line[6:].strip()
|
||||
if data_str != "[DONE]":
|
||||
try:
|
||||
chunk = json.loads(data_str)
|
||||
|
||||
if chunk.get("event") == "response.output_text.delta":
|
||||
delta = chunk["delta"]
|
||||
collected_deltas.append(delta)
|
||||
|
||||
# Verify each delta is new content
|
||||
current_full = "".join(collected_deltas)
|
||||
assert current_full.startswith(previous_full_text), \
|
||||
"Deltas should accumulate progressively"
|
||||
previous_full_text = current_full
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
full_text = "".join(collected_deltas)
|
||||
assert len(full_text) > 0
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.asyncio
|
||||
async def test_tatlock_with_reasoning(async_client: AsyncClient):
|
||||
"""
|
||||
Integration test: Verify Tatlock with reasoning enabled.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Hello"}],
|
||||
"reasoning": {"effort": "medium", "summary": "auto"},
|
||||
"stream": True
|
||||
}
|
||||
|
||||
has_reasoning = False
|
||||
has_output = False
|
||||
|
||||
async with async_client.stream(
|
||||
"POST",
|
||||
"/v1/responses",
|
||||
json=request_data,
|
||||
timeout=30.0,
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
if line.startswith("data: "):
|
||||
data_str = line[6:].strip()
|
||||
if data_str != "[DONE]":
|
||||
try:
|
||||
chunk = json.loads(data_str)
|
||||
|
||||
if chunk.get("event") == "response.reasoning_summary_text.delta":
|
||||
has_reasoning = True
|
||||
elif chunk.get("event") == "response.output_text.delta":
|
||||
has_output = True
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
assert has_reasoning, "Should have reasoning summary"
|
||||
assert has_output, "Should have output text"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.asyncio
|
||||
async def test_tatlock_markdown_formatting_preserved(async_client: AsyncClient):
|
||||
"""
|
||||
Integration test: Verify markdown formatting is preserved in responses.
|
||||
|
||||
Tests that code blocks, newlines, and other markdown formatting
|
||||
are properly preserved through the streaming pipeline.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Can you give me an HTML5 boilerplate template?"}],
|
||||
"stream": True
|
||||
}
|
||||
|
||||
collected_deltas = []
|
||||
|
||||
async with async_client.stream(
|
||||
"POST",
|
||||
"/v1/responses",
|
||||
json=request_data,
|
||||
timeout=45.0, # Give extra time for code generation
|
||||
) as response:
|
||||
assert response.status_code == 200
|
||||
|
||||
async for line in response.aiter_lines():
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
if line.startswith("data: "):
|
||||
data_str = line[6:].strip()
|
||||
if data_str != "[DONE]":
|
||||
try:
|
||||
chunk = json.loads(data_str)
|
||||
|
||||
if chunk.get("event") == "response.output_text.delta":
|
||||
collected_deltas.append(chunk["delta"])
|
||||
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Reconstruct full response
|
||||
full_response = "".join(collected_deltas)
|
||||
|
||||
# Always print the response for debugging
|
||||
print("\n" + "="*80)
|
||||
print("FULL RESPONSE (repr):")
|
||||
print("="*80)
|
||||
print(repr(full_response))
|
||||
print("\n" + "="*80)
|
||||
print("FULL RESPONSE (formatted):")
|
||||
print("="*80)
|
||||
print(full_response)
|
||||
print("="*80 + "\n")
|
||||
|
||||
# Verify we got a response
|
||||
assert len(full_response) > 100, "Should have a substantial response"
|
||||
|
||||
# Verify markdown code block is present
|
||||
assert "```" in full_response, "Response should contain markdown code blocks"
|
||||
|
||||
# Verify newlines are preserved (not all collapsed to spaces)
|
||||
newline_count = full_response.count('\n')
|
||||
assert newline_count > 5, f"Should have multiple newlines preserved, got {newline_count}"
|
||||
|
||||
# Verify code block markers are complete
|
||||
code_block_starts = full_response.count("```")
|
||||
# Should have at least opening and closing markers (even count)
|
||||
assert code_block_starts % 2 == 0, "Code blocks should have matching opening/closing markers"
|
||||
assert code_block_starts >= 2, "Should have at least one complete code block"
|
||||
|
||||
# Verify HTML tags are present (indicates code block content is preserved)
|
||||
assert "<!DOCTYPE html>" in full_response or "<html" in full_response, \
|
||||
"Should contain HTML5 boilerplate elements"
|
||||
|
||||
# Verify indentation is preserved (check for multiple spaces in a row)
|
||||
# This indicates that code formatting with indentation is maintained
|
||||
assert " " in full_response, "Should preserve indentation (multiple spaces)"
|
||||
|
||||
# Log the response for debugging if test fails
|
||||
if "```" not in full_response or newline_count < 5:
|
||||
print("\n=== Full Response ===")
|
||||
print(repr(full_response)) # Use repr to see escaped characters
|
||||
print("\n=== Newline count ===")
|
||||
print(f"Found {newline_count} newlines")
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
def test_tatlock_markdown_non_streaming(client: TestClient):
|
||||
"""
|
||||
Integration test: Verify markdown in non-streaming mode.
|
||||
"""
|
||||
request_data = {
|
||||
"model": "Tatlock",
|
||||
"input": [{"role": "user", "content": "Give me a simple Python hello world code"}],
|
||||
"stream": False
|
||||
}
|
||||
|
||||
response = client.post("/v1/responses", json=request_data, timeout=30.0)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
|
||||
# Get the message content
|
||||
message_item = next((item for item in data["output"] if item["type"] == "message"), None)
|
||||
assert message_item is not None
|
||||
|
||||
text = message_item["content"][0]["text"]
|
||||
|
||||
# Verify markdown code block
|
||||
assert "```" in text, "Should contain code block markers"
|
||||
assert "\n" in text, "Should contain newlines"
|
||||
@@ -24,7 +24,7 @@ def test_list_models(client: TestClient) -> None:
|
||||
# Check for expected model IDs
|
||||
model_ids = [m["id"] for m in data["data"]]
|
||||
assert "lorem-tester" in model_ids
|
||||
assert "tatlock" in model_ids
|
||||
assert "Tatlock" in model_ids
|
||||
|
||||
# Verify model structure
|
||||
for model in data["data"]:
|
||||
|
||||
@@ -376,3 +376,226 @@ def test_invalid_combined_parameters(client: TestClient):
|
||||
data = response.json()
|
||||
assert "error" in data
|
||||
assert data["error"]["type"] == "invalid_request_error"
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Streaming Delta Calculation Tests (No Duplication)
|
||||
# ============================================================================
|
||||
|
||||
@pytest.mark.unit
|
||||
@pytest.mark.asyncio
|
||||
async def test_streaming_delta_calculation_no_duplication():
|
||||
"""
|
||||
Test that StreamingCoordinator correctly calculates deltas when agent
|
||||
yields accumulated text multiple times (PydanticAI pattern).
|
||||
|
||||
This test prevents the duplication bug where the same text was
|
||||
streamed multiple times because we weren't computing deltas correctly.
|
||||
"""
|
||||
from src.agents.base import AgentInterface, OutputItem
|
||||
from typing import AsyncGenerator, Any
|
||||
|
||||
# Create a mock agent that simulates PydanticAI's behavior
|
||||
# (yielding accumulated text, not deltas)
|
||||
class MockStreamingAgent(AgentInterface):
|
||||
async def generate_response(
|
||||
self,
|
||||
messages: list[dict],
|
||||
reasoning: dict | None = None,
|
||||
tools: list[dict] | None = None,
|
||||
temperature: float = 1.0,
|
||||
max_tokens: int | None = None,
|
||||
stop: list[str] | None = None,
|
||||
**kwargs: Any
|
||||
) -> AsyncGenerator[OutputItem, None]:
|
||||
"""
|
||||
Simulate PydanticAI streaming behavior:
|
||||
- Yields accumulated text, not deltas
|
||||
- Multiple yields with status="in_progress"
|
||||
- Final yield with status="completed"
|
||||
"""
|
||||
msg_id = "msg_test_123"
|
||||
|
||||
# Simulate incremental accumulation like PydanticAI does
|
||||
accumulated_texts = [
|
||||
"Hello",
|
||||
"Hello world",
|
||||
"Hello world how",
|
||||
"Hello world how are",
|
||||
"Hello world how are you",
|
||||
]
|
||||
|
||||
for text in accumulated_texts:
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=msg_id,
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": text,
|
||||
"annotations": []
|
||||
}],
|
||||
status="in_progress"
|
||||
)
|
||||
|
||||
# Final message
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id=msg_id,
|
||||
role="assistant",
|
||||
content=[{
|
||||
"type": "output_text",
|
||||
"text": "Hello world how are you",
|
||||
"annotations": []
|
||||
}],
|
||||
status="completed"
|
||||
)
|
||||
|
||||
async def supports_tools(self) -> bool:
|
||||
return False
|
||||
|
||||
async def supports_reasoning(self) -> bool:
|
||||
return False
|
||||
|
||||
async def get_capabilities(self) -> dict:
|
||||
return {"streaming": True, "reasoning": False, "tools": False}
|
||||
|
||||
# Register the mock agent
|
||||
import time
|
||||
from src.agents.registry import ModelRegistry
|
||||
ModelRegistry.MODELS["mock-streaming"] = {
|
||||
"agent_class": MockStreamingAgent,
|
||||
"description": "Mock streaming agent for testing",
|
||||
"created": int(time.time()),
|
||||
"owned_by": "test",
|
||||
}
|
||||
|
||||
try:
|
||||
# Create a test request
|
||||
request = ResponseRequest(
|
||||
model="mock-streaming",
|
||||
input=[{"role": "user", "content": "Test"}],
|
||||
stream=True
|
||||
)
|
||||
|
||||
# Stream the response
|
||||
coordinator = StreamingCoordinator()
|
||||
collected_deltas = []
|
||||
|
||||
async for event in coordinator.stream_response(request):
|
||||
if event.event == "response.output_text.delta":
|
||||
collected_deltas.append(event.delta)
|
||||
|
||||
# Reconstruct the full text from deltas
|
||||
full_text = "".join(collected_deltas)
|
||||
|
||||
# Verify no duplication - the text should appear exactly once
|
||||
assert full_text.count("Hello") == 1, "Text 'Hello' should appear exactly once"
|
||||
assert full_text.count("world") == 1, "Text 'world' should appear exactly once"
|
||||
assert full_text.count("how") == 1, "Text 'how' should appear exactly once"
|
||||
assert full_text.count("are") == 1, "Text 'are' should appear exactly once"
|
||||
assert full_text.count("you") == 1, "Text 'you' should appear exactly once"
|
||||
|
||||
# Verify the reconstructed text is correct (no trailing space with chunk streaming)
|
||||
expected_text = "Hello world how are you"
|
||||
assert full_text == expected_text, f"Expected '{expected_text}', got '{full_text}'"
|
||||
|
||||
finally:
|
||||
# Clean up
|
||||
del ModelRegistry.MODELS["mock-streaming"]
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
@pytest.mark.asyncio
|
||||
async def test_streaming_with_multiple_message_items():
|
||||
"""
|
||||
Test that coordinator handles multiple message OutputItems correctly,
|
||||
only streaming the delta between each one.
|
||||
"""
|
||||
from src.agents.base import AgentInterface, OutputItem
|
||||
from typing import AsyncGenerator, Any
|
||||
|
||||
class MockMultiMessageAgent(AgentInterface):
|
||||
async def generate_response(
|
||||
self,
|
||||
messages: list[dict],
|
||||
reasoning: dict | None = None,
|
||||
tools: list[dict] | None = None,
|
||||
temperature: float = 1.0,
|
||||
max_tokens: int | None = None,
|
||||
stop: list[str] | None = None,
|
||||
**kwargs: Any
|
||||
) -> AsyncGenerator[OutputItem, None]:
|
||||
"""Yield multiple in_progress messages with accumulated text."""
|
||||
# First chunk
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id="msg_1",
|
||||
role="assistant",
|
||||
content=[{"type": "output_text", "text": "The answer is", "annotations": []}],
|
||||
status="in_progress"
|
||||
)
|
||||
|
||||
# Second chunk (more text accumulated)
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id="msg_1",
|
||||
role="assistant",
|
||||
content=[{"type": "output_text", "text": "The answer is 42", "annotations": []}],
|
||||
status="in_progress"
|
||||
)
|
||||
|
||||
# Final chunk
|
||||
yield OutputItem(
|
||||
type="message",
|
||||
id="msg_1",
|
||||
role="assistant",
|
||||
content=[{"type": "output_text", "text": "The answer is 42", "annotations": []}],
|
||||
status="completed"
|
||||
)
|
||||
|
||||
async def supports_tools(self) -> bool:
|
||||
return False
|
||||
|
||||
async def supports_reasoning(self) -> bool:
|
||||
return False
|
||||
|
||||
async def get_capabilities(self) -> dict:
|
||||
return {"streaming": True, "reasoning": False, "tools": False}
|
||||
|
||||
# Register mock agent
|
||||
import time
|
||||
from src.agents.registry import ModelRegistry
|
||||
ModelRegistry.MODELS["mock-multi"] = {
|
||||
"agent_class": MockMultiMessageAgent,
|
||||
"description": "Mock multi-message agent for testing",
|
||||
"created": int(time.time()),
|
||||
"owned_by": "test",
|
||||
}
|
||||
|
||||
try:
|
||||
request = ResponseRequest(
|
||||
model="mock-multi",
|
||||
input=[{"role": "user", "content": "What is the answer?"}],
|
||||
stream=True
|
||||
)
|
||||
|
||||
coordinator = StreamingCoordinator()
|
||||
collected_deltas = []
|
||||
|
||||
async for event in coordinator.stream_response(request):
|
||||
if event.event == "response.output_text.delta":
|
||||
collected_deltas.append(event.delta)
|
||||
|
||||
full_text = "".join(collected_deltas)
|
||||
|
||||
# Should only see "The answer is 42" once, not repeated
|
||||
assert "The answer is 42" in full_text
|
||||
# Count occurrences - should only appear once
|
||||
assert full_text.count("The") == 1
|
||||
assert full_text.count("answer") == 1
|
||||
assert full_text.count("42") == 1
|
||||
|
||||
finally:
|
||||
# Clean up
|
||||
del ModelRegistry.MODELS["mock-multi"]
|
||||
|
||||
+7
-3
@@ -20,7 +20,8 @@ def test_app_creation():
|
||||
|
||||
assert isinstance(app, FastAPI)
|
||||
assert app.title == "OpenAI-Compatible API"
|
||||
assert app.version == "0.1.0"
|
||||
# Version testing is brittle - just verify it's set
|
||||
assert app.version is not None
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
@@ -205,7 +206,8 @@ def test_app_metadata():
|
||||
from src.main import app
|
||||
|
||||
assert app.title == "OpenAI-Compatible API"
|
||||
assert app.version == "0.1.0"
|
||||
# Version testing is brittle - just verify it's set
|
||||
assert app.version is not None
|
||||
# Description is not set in main.py, so it will be empty
|
||||
# We just verify the important metadata is present
|
||||
assert app.debug is not None # Debug flag should be set
|
||||
@@ -222,7 +224,9 @@ def test_app_contact_info():
|
||||
|
||||
# Title and version should be set
|
||||
assert schema["info"]["title"] == "OpenAI-Compatible API"
|
||||
assert schema["info"]["version"] == "0.1.0"
|
||||
# Version testing is brittle - just verify it exists
|
||||
assert "version" in schema["info"]
|
||||
assert schema["info"]["version"] is not None
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
|
||||
|
||||
#!/bin/bash
|
||||
# Tatlock Server Startup Script
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
RED='\033[0;31m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
echo -e "${GREEN}Starting Tatlock server...${NC}"
|
||||
|
||||
# Check if port 8000 is already in use
|
||||
if lsof -Pi :8000 -sTCP:LISTEN -t >/dev/null 2>&1 ; then
|
||||
echo -e "${RED}Error: Port 8000 is already in use${NC}"
|
||||
echo "Run: lsof -i :8000 to see what's using it"
|
||||
echo "Or run: kill \$(lsof -t -i:8000) to stop it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Activate virtual environment if not already activated
|
||||
if [ -z "$VIRTUAL_ENV" ]; then
|
||||
if [ -d ".venv" ]; then
|
||||
echo -e "${YELLOW}Activating virtual environment...${NC}"
|
||||
source .venv/bin/activate
|
||||
else
|
||||
echo -e "${RED}Error: Virtual environment not found${NC}"
|
||||
echo "Run: python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create logs directory if it doesn't exist
|
||||
LOGS_DIR="logs"
|
||||
mkdir -p "$LOGS_DIR"
|
||||
|
||||
# Clear/create log file
|
||||
LOG_FILE="$LOGS_DIR/server.log"
|
||||
> "$LOG_FILE"
|
||||
echo -e "${YELLOW}Logs will be written to: ${LOG_FILE}${NC}"
|
||||
|
||||
# Start the server
|
||||
echo -e "${GREEN}Starting uvicorn server on http://localhost:8000${NC}"
|
||||
echo -e "${YELLOW}Press Ctrl+C to stop the server${NC}"
|
||||
echo ""
|
||||
|
||||
uvicorn src.main:app --reload --host 0.0.0.0 --port 8000 2>&1 | tee "$LOG_FILE"
|
||||
Reference in New Issue
Block a user