From 8d0618b64718b6a37446346851b9e2220fc63204 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Sat, 6 Dec 2025 20:01:49 +0100 Subject: [PATCH] Deduplicate and refocus documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restructured README.md and AGENTS.md to eliminate duplication: README.md (user-focused): - Simplified to focus on project description and usage - Quick start guide with installation steps - API usage examples with curl commands - Open WebUI integration guide - Troubleshooting section - Deployment recommendations - Removed internal architectural details AGENTS.md (LLM agent instructions): - Retained detailed architectural decisions and rationale - FastAPI best practices and patterns - Development guidelines and code structure - Documentation references for frameworks - Testing strategy and coverage details - Updated test coverage: 78.95% (95 tests) - Common implementation patterns Changes: - README.md: Streamlined from 497 to 310 lines - AGENTS.md: Updated test coverage numbers - Clear separation: README for users, AGENTS for AI developers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- AGENTS.md | 7 +- README.md | 419 +++++++++++++++--------------------------------------- 2 files changed, 121 insertions(+), 305 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8926e74..96be9c9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -201,10 +201,11 @@ Agent Interface (lorem-tester, tatlock) - OpenAI-compatible error format - Error triggers in lorem-tester for testing - ✅ **Testing Infrastructure**: - - 75 tests (78.95% coverage) + - 95 tests (78.95% coverage) - Unit tests for all components - Integration tests for API endpoints - Streaming tests for SSE functionality + - Main application and wrapper tests ## FastAPI Best Practices @@ -402,12 +403,14 @@ app = create_application() - 🚧 Implement rate limiting for API endpoints (future) - 🚧 Add authentication/API keys (future) -### Testing (Current Coverage: 62%) +### Testing (Current Coverage: 78.95%, 95 tests) - ✅ Integration tests for API endpoints - ✅ Streaming functionality with 20s timeout protection - ✅ Async test support with pytest-asyncio - ✅ Validate OpenAI API compatibility - ✅ Mock responses for all endpoints +- ✅ Main application tests (CORS, exception handlers, lifespan) +- ✅ Chat streaming wrapper tests - 🚧 Future: Mock Ollama responses when integrated ### Configuration diff --git a/README.md b/README.md index b32bf95..5c7ff6e 100644 --- a/README.md +++ b/README.md @@ -1,108 +1,47 @@ # Tatlock - OpenAI-Compatible API with Responses API -A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. Features a hybrid architecture with chat completions as a compatibility wrapper around the Responses API. +A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. ## Current Status -**✅ Production-ready testing API** with OpenAI Responses API format -**✅ Open WebUI integration** with reasoning bubbles (`` tags) -**✅ Conversation history** with hybrid client/server approach -**🚧 PydanticAI integration** prepared for future real LLM connection - -## Architecture Overview - -### Hybrid API Design - -``` -┌─────────────────────────────────────────┐ -│ Client (Open WebUI, etc.) │ -└────────┬────────────────────────────────┘ - │ - ├──────────────────────────────────┐ - │ │ - v v -┌────────────────────┐ ┌──────────────────────┐ -│ /v1/chat/ │ wrapper │ /v1/responses │ -│ completions ├─────────>│ (Primary API) │ -│ │ │ │ -│ • OpenAI compat │ │ • Reasoning items │ -│ • tags │ │ • Function calls │ -│ • Legacy support │ │ • Message items │ -└────────────────────┘ └──────────┬───────────┘ - │ - v - ┌──────────────────────┐ - │ Agent Interface │ - │ │ - │ • lorem-tester │ - │ • tatlock (future) │ - └──────────────────────┘ -``` - -**Key Architectural Decisions:** -- **Single Source of Truth**: Responses API handles all generation logic -- **Chat Completions Wrapper**: Converts Responses output to Chat format with `` tags -- **Agent Interface**: Clean abstraction for multiple models (mock and real) -- **Hybrid History**: Client sends full context, server optionally tracks conversations +- ✅ **Production-ready testing API** with OpenAI Responses API format +- ✅ **Open WebUI integration** with reasoning bubbles (`` tags) +- ✅ **Conversation history** with auto-generated IDs and context management +- ✅ **Comprehensive testing** - 95 tests, 78.95% coverage +- 🚧 **PydanticAI integration** prepared for future real LLM connection ## Features -### Core API -- ✅ **Responses API** (`/v1/responses`) - Primary endpoint with structured output - - Reasoning items (thinking/extended thinking) - - Function call items (tool execution) - - Message items (assistant responses) - - Streaming and non-streaming modes -- ✅ **Chat Completions API** (`/v1/chat/completions`) - Compatibility wrapper - - Converts reasoning to `` tags for Open WebUI - - Maintains OpenAI-compatible format - - Wraps Responses API (single source of truth) -- ✅ **Models API** (`/v1/models`) - Lists available models +### API Endpoints -### Advanced Features -- ✅ **Conversation History Management** - - Hybrid approach: client maintains state, server tracks optionally - - Auto-generated conversation IDs from first message hash - - Configurable max turns (default: 20) - - Placeholder for future vector memory (Qdrant) -- ✅ **Context Window Management** - - Approximate token counting (~4 chars/token) - - Context trimming to fit model limits - - Token usage statistics -- ✅ **Parameter Validation** - - Temperature: 0.0-2.0 - - Reasoning effort: none, minimal, low, medium, high, xhigh - - Max output tokens enforcement - - Stop sequences (up to 4) -- ✅ **Stop Sequence Detection** - - Real-time detection during streaming - - Stops generation immediately when encountered -- ✅ **Max Tokens Enforcement** - - Real-time token counting during streaming - - Stops when limit reached +- **Responses API** (`/v1/responses`) - OpenAI Responses API format with structured output + - Reasoning items for displaying thinking process + - Function call items for tool execution + - Message items for assistant responses + - Streaming and non-streaming support + +- **Chat Completions** (`/v1/chat/completions`) - OpenAI Chat Completions compatibility + - Automatic reasoning conversion to `` tags for Open WebUI + - Full OpenAI API compatibility + - Streaming support + +- **Models** (`/v1/models`) - List available models + +### Advanced Capabilities + +- **Conversation History**: Auto-generated IDs, configurable max turns (default: 20) +- **Context Management**: Token counting, automatic trimming, usage statistics +- **Parameter Validation**: Temperature (0.0-2.0), reasoning effort levels, max tokens, stop sequences +- **Real-time Enforcement**: Stop sequence detection and max token limits during streaming ### Testing Models -- ✅ **lorem-tester** - Full-featured mock agent - - Realistic reasoning summaries - - Random tool/function call generation + +- **lorem-tester**: Full-featured mock agent with realistic behavior + - Configurable reasoning effort levels + - Random tool/function calls - Error triggers for testing (rate_limit, context_overflow) - - Temperature variation -- ✅ **tatlock** - Placeholder for real PydanticAI agent -### Open WebUI Integration -- ✅ **Reasoning Display** - Thinking bubbles shown separately from responses -- ✅ **Streaming Support** - Smooth word-by-word streaming -- ✅ **Error Handling** - Graceful error display -- ✅ **Model Selection** - Both models available in dropdown - -## Components - -- **FastAPI**: High-performance web framework -- **SSE-Starlette**: Server-Sent Events for streaming -- **Pydantic**: Type-safe request/response validation -- **Agent Interface**: Abstraction for multiple model backends -- **Conversation History**: Server-side tracking with hybrid approach -- **Context Window**: Token management and trimming +- **tatlock**: Placeholder for future PydanticAI agent ## Requirements @@ -110,46 +49,24 @@ A FastAPI-based service providing OpenAI-compatible API endpoints with full Resp - No external dependencies for mock API - (Future: Network access for PydanticAI integration) -## Installation +## Quick Start -### 1. Clone the repository +### Installation ```bash -git clone +# Clone the repository +git clone https://git.schweitz.net/jpmschweitzer/tatlock.git cd tatlock -``` -### 2. Create a virtual environment - -```bash +# Create virtual environment python -m venv .venv -source .venv/bin/activate # On Windows: .venv\Scripts\activate -``` +source .venv/bin/activate # Windows: .venv\Scripts\activate -### 3. Install dependencies - -```bash +# Install dependencies pip install -r requirements.txt ``` -### 4. Configure environment (Optional) - -Create a `.env` file for custom configuration: - -```env -# API Configuration -API_HOST=0.0.0.0 -API_PORT=8000 - -# Logging -LOG_LEVEL=INFO - -# Future: Add real LLM configuration here -``` - -## Usage - -### Start the server +### Run the Server ```bash uvicorn src.main:app --reload @@ -157,11 +74,11 @@ uvicorn src.main:app --reload API available at `http://localhost:8000` -### API Endpoints +## Usage Examples -#### Responses API (Primary) +### Responses API -OpenAI Responses API format with structured output: +Generate a response with reasoning: ```bash curl http://localhost:8000/v1/responses \ @@ -176,7 +93,6 @@ curl http://localhost:8000/v1/responses \ "summary": "auto" }, "max_output_tokens": 500, - "stop": ["END"], "stream": false }' ``` @@ -192,22 +108,12 @@ curl http://localhost:8000/v1/responses \ "output": [ { "type": "reasoning", - "id": "reasoning_xyz", - "summary": [ - "Analyzing the user's request...", - "Considering quantum mechanics principles..." - ] + "summary": ["Analyzing the request...", "Considering quantum mechanics..."] }, { "type": "message", - "id": "msg_def456", "role": "assistant", - "content": [ - { - "type": "output_text", - "text": "Quantum computing uses quantum mechanics..." - } - ] + "content": [{"type": "output_text", "text": "Quantum computing uses..."}] } ], "usage": { @@ -219,9 +125,7 @@ curl http://localhost:8000/v1/responses \ } ``` -#### Chat Completions (Compatibility) - -OpenAI-compatible format with `` tags: +### Chat Completions (OpenAI-compatible) ```bash curl http://localhost:8000/v1/chat/completions \ @@ -236,66 +140,31 @@ curl http://localhost:8000/v1/chat/completions \ }' ``` -**Note**: Chat Completions automatically enables reasoning and converts it to `` tags for Open WebUI compatibility. - -#### List Models +### List Models ```bash curl http://localhost:8000/v1/models ``` -Returns: -```json -{ - "object": "list", - "data": [ - { - "id": "lorem-tester", - "object": "model", - "created": 1733529600, - "owned_by": "tatlock" - }, - { - "id": "tatlock", - "object": "model", - "created": 1733529600, - "owned_by": "tatlock" - } - ] -} -``` - ### Conversation History -Optional conversation tracking via metadata: +Optionally track conversations using metadata: ```bash curl http://localhost:8000/v1/responses \ -H "Content-Type: application/json" \ -d '{ "model": "lorem-tester", - "input": [ - {"role": "user", "content": "Hello"} - ], - "metadata": { - "conversation_id": "conv_abc123" - } + "input": [{"role": "user", "content": "Hello"}], + "metadata": {"conversation_id": "conv_abc123"} }' ``` -**Hybrid Approach:** -- Client MUST send full conversation history in `input` array (OpenAI compatible) -- Server optionally tracks via `metadata.conversation_id` (for analytics, future vector memory) -- Auto-generates conversation ID from first message hash if not provided - -### Interactive Documentation - -- **Swagger UI**: `http://localhost:8000/docs` -- **ReDoc**: `http://localhost:8000/redoc` +**Note**: Client must send full conversation history in `input` array (OpenAI compatible). Server optionally tracks via `metadata.conversation_id` for future features. ## Open WebUI Integration -### Docker Networking +### Connection If running Open WebUI in Docker and API on host: @@ -306,71 +175,26 @@ http://172.17.0.1:8000/v1/chat/completions ### Reasoning Display -The Chat Completions wrapper automatically: +The Chat Completions endpoint automatically: 1. Enables reasoning generation -2. Converts reasoning items to `` tags -3. Streams thinking before the actual response +2. Converts reasoning to `` tags +3. Streams thinking before the response -Open WebUI displays this as: -- **Thought bubble** showing reasoning steps -- **Main response** showing the actual answer +Open WebUI displays this as thought bubbles separate from the main response. ### Testing Error Handling -Lorem-tester supports error triggers: -- **"trigger_rate_limit"** - Simulates rate limit error -- **"trigger_context_overflow"** - Simulates context length error +Use special triggers in user messages: +- `"trigger_rate_limit"` - Simulates rate limit error +- `"trigger_context_overflow"` - Simulates context length error -## Development +## API Documentation -### Project Structure +Interactive documentation available at: +- **Swagger UI**: `http://localhost:8000/docs` +- **ReDoc**: `http://localhost:8000/redoc` -Following FastAPI best practices with domain-based organization: - -``` -tatlock/ -├── src/ -│ ├── agents/ # Agent interface and implementations -│ │ ├── base.py # Abstract AgentInterface -│ │ ├── lorem_tester.py # Full-featured mock agent -│ │ ├── tatlock.py # Placeholder for real agent -│ │ └── registry.py # Model registry -│ ├── responses/ # Responses API domain (PRIMARY) -│ │ ├── router.py # POST /v1/responses -│ │ ├── schemas.py # Request/response models -│ │ ├── service.py # Response generation logic -│ │ ├── streaming.py # SSE streaming coordinator -│ │ ├── history.py # Conversation history management -│ │ └── context.py # Context window management -│ ├── chat/ # Chat Completions domain (WRAPPER) -│ │ ├── router.py # POST /v1/chat/completions -│ │ ├── schemas.py # Chat request/response models -│ │ ├── service.py # Wraps Responses API -│ │ └── constants.py # Chat constants -│ ├── models/ # Models listing domain -│ │ ├── router.py # GET /v1/models -│ │ ├── schemas.py # Model schemas -│ │ └── service.py # Model registry access -│ ├── core/ # Shared utilities -│ │ ├── config.py # Configuration (BaseSettings) -│ │ ├── models.py # Custom Pydantic base -│ │ ├── exceptions.py # Custom exceptions -│ │ └── router.py # Health check endpoints -│ └── main.py # Application factory -├── tests/ # Comprehensive test suite -│ ├── agents/ # Agent tests -│ ├── responses/ # Responses API tests -│ ├── chat/ # Chat completions tests -│ ├── models/ # Models API tests -│ └── core/ # Core tests -├── requirements.txt # Dependencies (pinned) -├── .env # Environment variables -├── AGENTS.md # Agent documentation -├── CLEANUP_TODO.md # Architecture notes -└── README.md # This file -``` - -### Testing +## Testing ```bash # Run all tests @@ -379,42 +203,9 @@ pytest # Run with coverage pytest --cov=src --cov-report=term-missing -# Current coverage: 78.95% (75 tests passing) +# Current: 95 tests, 78.95% coverage ``` -**Test Organization:** -- Unit tests for all components -- Integration tests for API endpoints -- Streaming tests for SSE functionality -- Error handling tests -- Advanced features tests (stop sequences, max tokens, validation) - -### Code Style - -- **Async-first**: All I/O operations use async/await -- **Type hints**: All functions fully typed -- **Pydantic validation**: All request/response validation -- **Domain separation**: Clear boundaries between components -- **Single responsibility**: Each module has one clear purpose - -## Security - -### Version Locking - -Minor version locking (`>=X.Y,=X.Y,` tags -## Future Roadmap +## Project Structure -### Short-term -- [ ] Connect tatlock model to real PydanticAI agent -- [ ] Implement vector memory (Qdrant integration) -- [ ] Add authentication/API keys -- [ ] Rate limiting middleware +``` +tatlock/ +├── src/ +│ ├── agents/ # Agent interface and implementations +│ ├── responses/ # Responses API (primary endpoint) +│ ├── chat/ # Chat Completions wrapper +│ ├── models/ # Models listing +│ ├── core/ # Shared utilities and config +│ └── main.py # Application entry point +├── tests/ # Comprehensive test suite +├── AGENTS.md # LLM agent development guidelines +├── CHANGELOG.md # Version history +└── README.md # This file +``` -### Long-term -- [ ] Multi-model support (OpenAI, Anthropic, etc.) -- [ ] Advanced conversation memory -- [ ] Tool/function calling integration -- [ ] Usage tracking and analytics +## Development + +For LLM agent development guidelines and architectural decisions, see [AGENTS.md](AGENTS.md). ## Contributing @@ -480,17 +289,21 @@ uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4 ## Documentation -- **AGENTS.md**: Agent architecture and best practices -- **CLEANUP_TODO.md**: Architecture decisions and future considerations -- **CHANGELOG.md**: Version history -- OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses -- FastAPI: https://fastapi.tiangolo.com/ -- PydanticAI: https://ai.pydantic.dev/ +- **Project Documentation**: This file +- **Agent Guidelines**: [AGENTS.md](AGENTS.md) +- **Version History**: [CHANGELOG.md](CHANGELOG.md) +- **OpenAI Responses API**: https://platform.openai.com/docs/api-reference/responses +- **FastAPI**: https://fastapi.tiangolo.com/ +- **PydanticAI**: https://ai.pydantic.dev/ ## License [Add your license here] +## Version + +Current version: **0.1.1** - Basic setup complete + --- -**Note**: This is a testing/development API with mock responses. The architecture is production-ready and designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.). +**Note**: This is a production-ready testing API with mock responses. The architecture is designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).