# Tatlock - OpenAI-Compatible API with Responses API A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. Features a hybrid architecture with chat completions as a compatibility wrapper around the Responses API. ## Current Status **✅ Production-ready testing API** with OpenAI Responses API format **✅ Open WebUI integration** with reasoning bubbles (`` tags) **✅ Conversation history** with hybrid client/server approach **🚧 PydanticAI integration** prepared for future real LLM connection ## Architecture Overview ### Hybrid API Design ``` ┌─────────────────────────────────────────┐ │ Client (Open WebUI, etc.) │ └────────┬────────────────────────────────┘ │ ├──────────────────────────────────┐ │ │ v v ┌────────────────────┐ ┌──────────────────────┐ │ /v1/chat/ │ wrapper │ /v1/responses │ │ completions ├─────────>│ (Primary API) │ │ │ │ │ │ • OpenAI compat │ │ • Reasoning items │ │ • tags │ │ • Function calls │ │ • Legacy support │ │ • Message items │ └────────────────────┘ └──────────┬───────────┘ │ v ┌──────────────────────┐ │ Agent Interface │ │ │ │ • lorem-tester │ │ • tatlock (future) │ └──────────────────────┘ ``` **Key Architectural Decisions:** - **Single Source of Truth**: Responses API handles all generation logic - **Chat Completions Wrapper**: Converts Responses output to Chat format with `` tags - **Agent Interface**: Clean abstraction for multiple models (mock and real) - **Hybrid History**: Client sends full context, server optionally tracks conversations ## Features ### Core API - ✅ **Responses API** (`/v1/responses`) - Primary endpoint with structured output - Reasoning items (thinking/extended thinking) - Function call items (tool execution) - Message items (assistant responses) - Streaming and non-streaming modes - ✅ **Chat Completions API** (`/v1/chat/completions`) - Compatibility wrapper - Converts reasoning to `` tags for Open WebUI - Maintains OpenAI-compatible format - Wraps Responses API (single source of truth) - ✅ **Models API** (`/v1/models`) - Lists available models ### Advanced Features - ✅ **Conversation History Management** - Hybrid approach: client maintains state, server tracks optionally - Auto-generated conversation IDs from first message hash - Configurable max turns (default: 20) - Placeholder for future vector memory (Qdrant) - ✅ **Context Window Management** - Approximate token counting (~4 chars/token) - Context trimming to fit model limits - Token usage statistics - ✅ **Parameter Validation** - Temperature: 0.0-2.0 - Reasoning effort: none, minimal, low, medium, high, xhigh - Max output tokens enforcement - Stop sequences (up to 4) - ✅ **Stop Sequence Detection** - Real-time detection during streaming - Stops generation immediately when encountered - ✅ **Max Tokens Enforcement** - Real-time token counting during streaming - Stops when limit reached ### Testing Models - ✅ **lorem-tester** - Full-featured mock agent - Realistic reasoning summaries - Random tool/function call generation - Error triggers for testing (rate_limit, context_overflow) - Temperature variation - ✅ **tatlock** - Placeholder for real PydanticAI agent ### Open WebUI Integration - ✅ **Reasoning Display** - Thinking bubbles shown separately from responses - ✅ **Streaming Support** - Smooth word-by-word streaming - ✅ **Error Handling** - Graceful error display - ✅ **Model Selection** - Both models available in dropdown ## Components - **FastAPI**: High-performance web framework - **SSE-Starlette**: Server-Sent Events for streaming - **Pydantic**: Type-safe request/response validation - **Agent Interface**: Abstraction for multiple model backends - **Conversation History**: Server-side tracking with hybrid approach - **Context Window**: Token management and trimming ## Requirements - Python 3.12+ (Python 3.12.11 recommended) - No external dependencies for mock API - (Future: Network access for PydanticAI integration) ## Installation ### 1. Clone the repository ```bash git clone cd tatlock ``` ### 2. Create a virtual environment ```bash python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate ``` ### 3. Install dependencies ```bash pip install -r requirements.txt ``` ### 4. Configure environment (Optional) Create a `.env` file for custom configuration: ```env # API Configuration API_HOST=0.0.0.0 API_PORT=8000 # Logging LOG_LEVEL=INFO # Future: Add real LLM configuration here ``` ## Usage ### Start the server ```bash uvicorn src.main:app --reload ``` API available at `http://localhost:8000` ### API Endpoints #### Responses API (Primary) OpenAI Responses API format with structured output: ```bash curl http://localhost:8000/v1/responses \ -H "Content-Type: application/json" \ -d '{ "model": "lorem-tester", "input": [ {"role": "user", "content": "Explain quantum computing"} ], "reasoning": { "effort": "medium", "summary": "auto" }, "max_output_tokens": 500, "stop": ["END"], "stream": false }' ``` **Response Structure:** ```json { "id": "resp_abc123", "object": "response", "created_at": 1733529600, "model": "lorem-tester", "status": "completed", "output": [ { "type": "reasoning", "id": "reasoning_xyz", "summary": [ "Analyzing the user's request...", "Considering quantum mechanics principles..." ] }, { "type": "message", "id": "msg_def456", "role": "assistant", "content": [ { "type": "output_text", "text": "Quantum computing uses quantum mechanics..." } ] } ], "usage": { "input_tokens": 10, "output_tokens": 50, "reasoning_tokens": 20, "total_tokens": 80 } } ``` #### Chat Completions (Compatibility) OpenAI-compatible format with `` tags: ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "lorem-tester", "messages": [ {"role": "user", "content": "Hello!"} ], "temperature": 0.7, "stream": true }' ``` **Note**: Chat Completions automatically enables reasoning and converts it to `` tags for Open WebUI compatibility. #### List Models ```bash curl http://localhost:8000/v1/models ``` Returns: ```json { "object": "list", "data": [ { "id": "lorem-tester", "object": "model", "created": 1733529600, "owned_by": "tatlock" }, { "id": "tatlock", "object": "model", "created": 1733529600, "owned_by": "tatlock" } ] } ``` ### Conversation History Optional conversation tracking via metadata: ```bash curl http://localhost:8000/v1/responses \ -H "Content-Type: application/json" \ -d '{ "model": "lorem-tester", "input": [ {"role": "user", "content": "Hello"} ], "metadata": { "conversation_id": "conv_abc123" } }' ``` **Hybrid Approach:** - Client MUST send full conversation history in `input` array (OpenAI compatible) - Server optionally tracks via `metadata.conversation_id` (for analytics, future vector memory) - Auto-generates conversation ID from first message hash if not provided ### Interactive Documentation - **Swagger UI**: `http://localhost:8000/docs` - **ReDoc**: `http://localhost:8000/redoc` ## Open WebUI Integration ### Docker Networking If running Open WebUI in Docker and API on host: ```bash # Use Docker bridge gateway IP http://172.17.0.1:8000/v1/chat/completions ``` ### Reasoning Display The Chat Completions wrapper automatically: 1. Enables reasoning generation 2. Converts reasoning items to `` tags 3. Streams thinking before the actual response Open WebUI displays this as: - **Thought bubble** showing reasoning steps - **Main response** showing the actual answer ### Testing Error Handling Lorem-tester supports error triggers: - **"trigger_rate_limit"** - Simulates rate limit error - **"trigger_context_overflow"** - Simulates context length error ## Development ### Project Structure Following FastAPI best practices with domain-based organization: ``` tatlock/ ├── src/ │ ├── agents/ # Agent interface and implementations │ │ ├── base.py # Abstract AgentInterface │ │ ├── lorem_tester.py # Full-featured mock agent │ │ ├── tatlock.py # Placeholder for real agent │ │ └── registry.py # Model registry │ ├── responses/ # Responses API domain (PRIMARY) │ │ ├── router.py # POST /v1/responses │ │ ├── schemas.py # Request/response models │ │ ├── service.py # Response generation logic │ │ ├── streaming.py # SSE streaming coordinator │ │ ├── history.py # Conversation history management │ │ └── context.py # Context window management │ ├── chat/ # Chat Completions domain (WRAPPER) │ │ ├── router.py # POST /v1/chat/completions │ │ ├── schemas.py # Chat request/response models │ │ ├── service.py # Wraps Responses API │ │ └── constants.py # Chat constants │ ├── models/ # Models listing domain │ │ ├── router.py # GET /v1/models │ │ ├── schemas.py # Model schemas │ │ └── service.py # Model registry access │ ├── core/ # Shared utilities │ │ ├── config.py # Configuration (BaseSettings) │ │ ├── models.py # Custom Pydantic base │ │ ├── exceptions.py # Custom exceptions │ │ └── router.py # Health check endpoints │ └── main.py # Application factory ├── tests/ # Comprehensive test suite │ ├── agents/ # Agent tests │ ├── responses/ # Responses API tests │ ├── chat/ # Chat completions tests │ ├── models/ # Models API tests │ └── core/ # Core tests ├── requirements.txt # Dependencies (pinned) ├── .env # Environment variables ├── AGENTS.md # Agent documentation ├── CLEANUP_TODO.md # Architecture notes └── README.md # This file ``` ### Testing ```bash # Run all tests pytest # Run with coverage pytest --cov=src --cov-report=term-missing # Current coverage: 78.95% (75 tests passing) ``` **Test Organization:** - Unit tests for all components - Integration tests for API endpoints - Streaming tests for SSE functionality - Error handling tests - Advanced features tests (stop sequences, max tokens, validation) ### Code Style - **Async-first**: All I/O operations use async/await - **Type hints**: All functions fully typed - **Pydantic validation**: All request/response validation - **Domain separation**: Clear boundaries between components - **Single responsibility**: Each module has one clear purpose ## Security ### Version Locking Minor version locking (`>=X.Y,` tags ## Future Roadmap ### Short-term - [ ] Connect tatlock model to real PydanticAI agent - [ ] Implement vector memory (Qdrant integration) - [ ] Add authentication/API keys - [ ] Rate limiting middleware ### Long-term - [ ] Multi-model support (OpenAI, Anthropic, etc.) - [ ] Advanced conversation memory - [ ] Tool/function calling integration - [ ] Usage tracking and analytics ## Contributing 1. Fork the repository 2. Create a feature branch 3. Make changes with tests 4. Ensure tests pass: `pytest` 5. Submit pull request ## Documentation - **AGENTS.md**: Agent architecture and best practices - **CLEANUP_TODO.md**: Architecture decisions and future considerations - **CHANGELOG.md**: Version history - OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses - FastAPI: https://fastapi.tiangolo.com/ - PydanticAI: https://ai.pydantic.dev/ ## License [Add your license here] --- **Note**: This is a testing/development API with mock responses. The architecture is production-ready and designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).