jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

Update documentation with hybrid architecture

Browse Source 
README.md:
- Complete rewrite with hybrid architecture documentation
- Architecture diagram showing wrapper pattern
- Detailed feature list for all implemented phases
- API usage examples for Responses and Chat Completions
- Conversation history usage guide
- Open WebUI integration instructions
- Comprehensive troubleshooting section
- Updated project structure
- Deployment considerations

AGENTS.md:
- Current architecture section (as of 2025-12-06)
- Hybrid architecture explanation
- Key architectural decisions documented
- Agent interface design patterns
- Conversation history approach
- Context window management
- Testing infrastructure details
- Implementation status updates
- Coverage statistics (78.95%, 95 tests)

Key Documentation Themes:
- Single source of truth: Responses API
- Wrapper pattern for Chat Completions
- Hybrid conversation history approach
- Clean agent abstraction
- Production-ready testing infrastructure

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-06 19:39:32 +01:00
co-authored by Claude
parent 8b6cff2920
commit 661db0a672
2 changed files with 490 additions and 194 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+134 -32
View File
@@ -6,16 +6,67 @@ This document contains instructions and documentation references for AI assistan
This project implements an OpenAI-compatible API endpoint using FastAPI, with streaming support. Currently returns mock responses - infrastructure prepared for future Ollama/PydanticAI integration.
**Current State**: Production-ready mock API with OpenAI-compatible format
**Future Integration**: Ollama and PydanticAI (client code ready, not connected)
**Current State**: Production-ready testing API with Responses API and Open WebUI integration
**Future Integration**: PydanticAI for real LLM agents (tatlock model placeholder ready)
### Current Architecture (As of 2025-12-06)
This project implements a **hybrid architecture** with the Responses API as the primary endpoint and Chat Completions as a compatibility wrapper:
```
Client (Open WebUI)
↓
Chat Completions (/v1/chat/completions) → Wrapper
↓
Responses API (/v1/responses) → Primary
↓
Agent Interface (lorem-tester, tatlock)
```
**Key Architectural Decisions:**
1. **Single Source of Truth**: All response generation happens in the Responses API
- Structured output with reasoning, function_call, and message items
- Real-time stop sequence and max tokens enforcement
- Conversation history tracking
- Context window management
2. **Chat Completions Wrapper**: Provides compatibility without duplicating logic
- Calls Responses API internally
- Automatically enables reasoning generation
- Converts reasoning items to `<think>` tags for Open WebUI
- Maintains OpenAI-compatible format
3. **Agent Interface**: Clean abstraction for multiple models
- **lorem-tester**: Full-featured mock agent with realistic behavior
- Reasoning summaries (adjustable effort levels)
- Random tool/function calls
- Error triggers for testing
- Temperature variation
- **tatlock**: Placeholder for future PydanticAI agent
4. **Hybrid Conversation History**:
- Client MUST send full context in `input` array (OpenAI compatible)
- Server optionally tracks via `metadata.conversation_id`
- Auto-generates deterministic IDs from first message
- Supports future vector memory integration (Qdrant)
**Why This Architecture?**
- **Open WebUI Compatibility**: Native Responses API support not yet in stable release
- **Future-Proof**: Easy migration when Open WebUI adds native support
- **Testability**: Full-featured mock agent (lorem-tester) for integration testing
- **Clean Separation**: Responses API as stable core, wrappers can change
### Components
- **FastAPI**: Web framework for the API layer
- **SSE-Starlette**: Server-Sent Events for streaming responses
- **Pydantic**: Request/response validation
- **PydanticAI**: Dependency installed, ready for future LLM integration
- **Ollama**: Async client implemented, ready for future connection
- **Pydantic**: Request/response validation with field validators
- **Agent Interface**: Abstract base class for model implementations
- **Conversation History**: Server-side tracking with configurable max turns
- **Context Window**: Token counting and management
- **PydanticAI**: Dependency installed, ready for tatlock agent implementation
## Documentation References
@@ -104,17 +155,56 @@ This project implements an OpenAI-compatible API endpoint using FastAPI, with st
#### OpenAI API Reference
- **Official Documentation**: https://platform.openai.com/docs/api-reference
- **Implemented Endpoints**:
- ✅ `/v1/chat/completions` - Chat completion with streaming (mock responses)
- ✅ `/v1/models` - List available models (mock listing)
- ✅ `/v1/responses` - **Responses API (PRIMARY)** with structured output
- Reasoning items (thinking summaries)
- Function call items (tool execution)
- Message items (assistant responses)
- Full streaming support with SSE
- Stop sequence detection
- Max tokens enforcement
- Conversation history tracking
- ✅ `/v1/chat/completions` - **Compatibility wrapper** around Responses API
- Converts reasoning to `<think>` tags for Open WebUI
- Automatically enables reasoning generation
- Maintains OpenAI-compatible format
- Supports streaming and non-streaming
- ✅ `/v1/models` - List available models (lorem-tester, tatlock)
- **Future Endpoints**:
- 🚧 `/v1/completions` - Text completion (legacy)
- 🚧 `/v1/embeddings` - Text embeddings
- **Implemented Features**:
- ✅ Streaming with Server-Sent Events
- ✅ Message format compatibility
- ✅ Response structure compatibility
- ✅ OpenAI error format
- ✅ Request validation with Pydantic
- ✅ **Responses API Format**:
- Structured output items (reasoning, function_call, message)
- Extended thinking support
- Tool/function calling support
- Streaming with multiple event types
- ✅ **Advanced Parameter Validation**:
- Temperature: 0.0-2.0 with Pydantic validators
- Reasoning effort: none, minimal, low, medium, high, xhigh
- Max output tokens: positive integer enforcement
- Stop sequences: up to 4, non-empty strings
- ✅ **Conversation History**:
- Hybrid client/server approach
- Auto-generated conversation IDs
- Configurable max turns (default: 20)
- Placeholder for vector memory
- ✅ **Context Management**:
- Approximate token counting (~4 chars/token)
- Context window trimming
- Usage statistics
- ✅ **Streaming Enforcement**:
- Real-time stop sequence detection
- Real-time max tokens enforcement
- Word-by-word streaming with delays
- ✅ **Error Handling**:
- Custom exception types (RateLimitError, ContextLengthError)
- OpenAI-compatible error format
- Error triggers in lorem-tester for testing
- ✅ **Testing Infrastructure**:
- 75 tests (78.95% coverage)
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
## FastAPI Best Practices
@@ -126,37 +216,49 @@ This project follows best practices from [github.com/zhanymkanov/fastapi-best-pr
```
src/
├── chat/ # Chat completions domain
│ ├── router.py # FastAPI routes
│ ├── schemas.py # Pydantic request/response models
│ ├── service.py # Business logic
│ ├── dependencies.py # Domain-specific dependencies
│ ├── constants.py # Domain constants
├── agents/ # Agent interface and implementations
│ ├── base.py # Abstract AgentInterface
│ ├── lorem_tester.py # Full-featured mock agent
│ ├── tatlock.py # Placeholder for real agent
│ └── registry.py # ModelRegistry for agent management
├── responses/ # Responses API domain (PRIMARY)
│ ├── router.py # POST /v1/responses endpoint
│ ├── schemas.py # Request/response models with validators
│ ├── service.py # Response generation logic
│ ├── streaming.py # SSE streaming coordinator
│ ├── history.py # Conversation history management
│ └── context.py # Context window and token management
├── chat/ # Chat Completions domain (WRAPPER)
│ ├── router.py # POST /v1/chat/completions endpoint
│ ├── schemas.py # Chat request/response models
│ ├── service.py # Wraps Responses API, converts to <think> tags
│ ├── constants.py # Chat constants (roles, finish reasons)
│ └── __init__.py
├── models/ # Models listing domain
│ ├── router.py
│ ├── schemas.py
│ ├── service.py
│ ├── router.py # GET /v1/models endpoint
│ ├── schemas.py # Model schemas
│ ├── service.py # Accesses ModelRegistry
│ └── __init__.py
├── core/ # Shared utilities
│ ├── config.py # Global configuration
│ ├── config.py # Global configuration (BaseSettings)
│ ├── models.py # Custom base Pydantic models
│ ├── exceptions.py # Global exceptions
│ ├── exceptions.py # Custom exceptions (RateLimitError, etc.)
│ ├── dependencies.py # Shared dependencies
│ └── router.py # Core routes (health, root)
├── ollama/ # Ollama client layer
├── ollama/ # Ollama client layer (not yet integrated)
│ ├── client.py # Async Ollama HTTP client
│ ├── schemas.py # Ollama API models
│ └── __init__.py
│ └── schemas.py # Ollama API models
└── main.py # Application factory & configuration
```
**Key Principles**:
- Each domain has its own router, schemas, models, service, etc.
- Cross-domain imports use explicit naming: `from src.auth import constants as auth_constants`
- Main.py focuses on configuration, middleware, and exception handlers
- Business logic stays in service modules
- Routes delegate to services for all business logic
**Key Architectural Principles**:
- **Single Source of Truth**: Responses API handles all generation logic
- **Wrapper Pattern**: Chat Completions wraps Responses API without duplicating code
- **Agent Abstraction**: AgentInterface defines contract for all models
- **Domain Separation**: Each domain has its own router, schemas, service
- **Service Layer**: Business logic in services, not routers
- **Type Safety**: Pydantic models for ALL request/response validation
- **Async First**: All I/O operations use async/await
### Async/Await Best Practices
README.md
+356 -162
View File
@@ -1,36 +1,114 @@
# OpenAI-Compatible API
# Tatlock - OpenAI-Compatible API with Responses API
A FastAPI-based service that provides OpenAI-compatible API endpoints with streaming support. Currently returns mock responses - ready for future Ollama/PydanticAI integration.
A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. Features a hybrid architecture with chat completions as a compatibility wrapper around the Responses API.
## Current Status
**✅ Production-ready mock API** with OpenAI-compatible format
**🚧 Ollama/PydanticAI integration** prepared but not connected
**✅ Production-ready testing API** with OpenAI Responses API format
**✅ Open WebUI integration** with reasoning bubbles (`<think>` tags)
**✅ Conversation history** with hybrid client/server approach
**🚧 PydanticAI integration** prepared for future real LLM connection
## Components
## Architecture Overview
- **FastAPI**: High-performance web framework providing the API layer
- **SSE-Starlette**: Server-Sent Events for streaming responses
- **Pydantic**: Type-safe request/response handling and validation
- **Ollama Client**: Async HTTP client prepared for future integration
- **PydanticAI**: Ready for LLM integration (not yet connected)
### Hybrid API Design
```
┌─────────────────────────────────────────┐
│ Client (Open WebUI, etc.) │
└────────┬────────────────────────────────┘
│
├──────────────────────────────────┐
│ │
v v
┌────────────────────┐ ┌──────────────────────┐
│ /v1/chat/ │ wrapper │ /v1/responses │
│ completions ├─────────>│ (Primary API) │
│ │ │ │
│ • OpenAI compat │ │ • Reasoning items │
│ • <think> tags │ │ • Function calls │
│ • Legacy support │ │ • Message items │
└────────────────────┘ └──────────┬───────────┘
│
v
┌──────────────────────┐
│ Agent Interface │
│ │
│ • lorem-tester │
│ • tatlock (future) │
└──────────────────────┘
```
**Key Architectural Decisions:**
- **Single Source of Truth**: Responses API handles all generation logic
- **Chat Completions Wrapper**: Converts Responses output to Chat format with `<think>` tags
- **Agent Interface**: Clean abstraction for multiple models (mock and real)
- **Hybrid History**: Client sends full context, server optionally tracks conversations
## Features
- ✅ OpenAI-compatible API endpoints (`/v1/chat/completions`, `/v1/models`)
- ✅ Streaming responses with Server-Sent Events (SSE)
- ✅ Type-safe request/response handling with Pydantic
- ✅ Async/await throughout for optimal performance
- ✅ Comprehensive test suite (62% coverage)
- ✅ Domain-based architecture following FastAPI best practices
- 🚧 Ollama integration (client ready, not connected)
- 🚧 PydanticAI integration (dependency installed, not connected)
### Core API
- ✅ **Responses API** (`/v1/responses`) - Primary endpoint with structured output
- Reasoning items (thinking/extended thinking)
- Function call items (tool execution)
- Message items (assistant responses)
- Streaming and non-streaming modes
- ✅ **Chat Completions API** (`/v1/chat/completions`) - Compatibility wrapper
- Converts reasoning to `<think>` tags for Open WebUI
- Maintains OpenAI-compatible format
- Wraps Responses API (single source of truth)
- ✅ **Models API** (`/v1/models`) - Lists available models
### Advanced Features
- ✅ **Conversation History Management**
- Hybrid approach: client maintains state, server tracks optionally
- Auto-generated conversation IDs from first message hash
- Configurable max turns (default: 20)
- Placeholder for future vector memory (Qdrant)
- ✅ **Context Window Management**
- Approximate token counting (~4 chars/token)
- Context trimming to fit model limits
- Token usage statistics
- ✅ **Parameter Validation**
- Temperature: 0.0-2.0
- Reasoning effort: none, minimal, low, medium, high, xhigh
- Max output tokens enforcement
- Stop sequences (up to 4)
- ✅ **Stop Sequence Detection**
- Real-time detection during streaming
- Stops generation immediately when encountered
- ✅ **Max Tokens Enforcement**
- Real-time token counting during streaming
- Stops when limit reached
### Testing Models
- ✅ **lorem-tester** - Full-featured mock agent
- Realistic reasoning summaries
- Random tool/function call generation
- Error triggers for testing (rate_limit, context_overflow)
- Temperature variation
- ✅ **tatlock** - Placeholder for real PydanticAI agent
### Open WebUI Integration
- ✅ **Reasoning Display** - Thinking bubbles shown separately from responses
- ✅ **Streaming Support** - Smooth word-by-word streaming
- ✅ **Error Handling** - Graceful error display
- ✅ **Model Selection** - Both models available in dropdown
## Components
- **FastAPI**: High-performance web framework
- **SSE-Starlette**: Server-Sent Events for streaming
- **Pydantic**: Type-safe request/response validation
- **Agent Interface**: Abstraction for multiple model backends
- **Conversation History**: Server-side tracking with hybrid approach
- **Context Window**: Token management and trimming
## Requirements
- Python 3.12+ (Python 3.12.11 recommended for security)
- No external dependencies required for mock API
- (Future: Network access to Ollama instance for LLM integration)
- Python 3.12+ (Python 3.12.11 recommended)
- No external dependencies for mock API
- (Future: Network access for PydanticAI integration)
## Installation
@@ -44,8 +122,8 @@ cd tatlock
### 2. Create a virtual environment
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
### 3. Install dependencies
@@ -54,9 +132,9 @@ source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```
### 4. Configure environment variables (Optional)
### 4. Configure environment (Optional)
Create a `.env` file in the project root for custom configuration:
Create a `.env` file for custom configuration:
```env
# API Configuration
@@ -66,231 +144,348 @@ API_PORT=8000
# Logging
LOG_LEVEL=INFO
# Future Ollama Configuration (not yet integrated)
# OLLAMA_HOST=http://localhost:11434
# OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
# OLLAMA_TIMEOUT=120
# Future: Add real LLM configuration here
```
**Note**: The API works with defaults. Environment variables are optional for customization. Ollama configuration is prepared but not currently used.
## Usage
### Start the development server
### Start the server
```bash
uvicorn src.main:app --reload
```
The API will be available at `http://localhost:8000`
API available at `http://localhost:8000`
### API Endpoints
#### Chat Completions (OpenAI-compatible)
#### Responses API (Primary)
Returns mock lorem ipsum responses:
OpenAI Responses API format with structured output:
```bash
curl http://localhost:8000/v1/responses \
-H "Content-Type: application/json" \
-d '{
"model": "lorem-tester",
"input": [
{"role": "user", "content": "Explain quantum computing"}
],
"reasoning": {
"effort": "medium",
"summary": "auto"
},
"max_output_tokens": 500,
"stop": ["END"],
"stream": false
}'
```
**Response Structure:**
```json
{
"id": "resp_abc123",
"object": "response",
"created_at": 1733529600,
"model": "lorem-tester",
"status": "completed",
"output": [
{
"type": "reasoning",
"id": "reasoning_xyz",
"summary": [
"Analyzing the user's request...",
"Considering quantum mechanics principles..."
]
},
{
"type": "message",
"id": "msg_def456",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Quantum computing uses quantum mechanics..."
}
]
}
],
"usage": {
"input_tokens": 10,
"output_tokens": 50,
"reasoning_tokens": 20,
"total_tokens": 80
}
}
```
#### Chat Completions (Compatibility)
OpenAI-compatible format with `<think>` tags:
```bash
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "mistral-nemo:latest",
"model": "lorem-tester",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
{"role": "user", "content": "Hello!"}
],
"temperature": 0.7,
"stream": true
}'
```
#### List Models
**Note**: Chat Completions automatically enables reasoning and converts it to `<think>` tags for Open WebUI compatibility.
Returns mock model listing:
#### List Models
```bash
curl http://localhost:8000/v1/models
```
Response: `{"object": "list", "data": [{"id": "mistral-nemo:latest", ...}]}`
Returns:
```json
{
"object": "list",
"data": [
{
"id": "lorem-tester",
"object": "model",
"created": 1733529600,
"owned_by": "tatlock"
},
{
"id": "tatlock",
"object": "model",
"created": 1733529600,
"owned_by": "tatlock"
}
]
}
```
### Interactive API Documentation
### Conversation History
- Swagger UI: `http://localhost:8000/docs`
- ReDoc: `http://localhost:8000/redoc`
Optional conversation tracking via metadata:
## Security
```bash
curl http://localhost:8000/v1/responses \
-H "Content-Type: application/json" \
-d '{
"model": "lorem-tester",
"input": [
{"role": "user", "content": "Hello"}
],
"metadata": {
"conversation_id": "conv_abc123"
}
}'
```
### Version Locking Strategy
**Hybrid Approach:**
- Client MUST send full conversation history in `input` array (OpenAI compatible)
- Server optionally tracks via `metadata.conversation_id` (for analytics, future vector memory)
- Auto-generates conversation ID from first message hash if not provided
This project uses minor version locking (`>=X.Y,<X.(Y+1)`) to protect against supply chain attacks while allowing patch updates. All dependencies have been:
### Interactive Documentation
- Checked for known CVEs (as of 2025-12-06)
- Pinned to secure minor versions
- Documented with version rationale in `requirements.txt`
- **Swagger UI**: `http://localhost:8000/docs`
- **ReDoc**: `http://localhost:8000/redoc`
### CVE Status (2025-12-06)
## Open WebUI Integration
- **FastAPI 0.123.9**: No known vulnerabilities
- **Uvicorn 0.38.0**: No known vulnerabilities
- **PydanticAI 1.27.0**: No known vulnerabilities
- **HTTPX 0.28.1**: No known vulnerabilities
- **SSE-Starlette 3.0.2**: No known vulnerabilities
### Docker Networking
Regular security updates are recommended. Check for new versions monthly.
If running Open WebUI in Docker and API on host:
### Security Best Practices
```bash
# Use Docker bridge gateway IP
http://172.17.0.1:8000/v1/chat/completions
```
1. Never commit `.env` files
2. Use environment variables for sensitive configuration
3. Keep dependencies updated
4. Implement rate limiting in production
5. Use HTTPS in production environments
6. Validate all inputs with Pydantic models
### Reasoning Display
The Chat Completions wrapper automatically:
1. Enables reasoning generation
2. Converts reasoning items to `<think>` tags
3. Streams thinking before the actual response
Open WebUI displays this as:
- **Thought bubble** showing reasoning steps
- **Main response** showing the actual answer
### Testing Error Handling
Lorem-tester supports error triggers:
- **"trigger_rate_limit"** - Simulates rate limit error
- **"trigger_context_overflow"** - Simulates context length error
## Development
### Project Structure
Following [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices) with domain-based organization:
Following FastAPI best practices with domain-based organization:
```
tatlock/
├── src/
│ ├── chat/ # Chat completions domain
│ │ ├── router.py # OpenAI-compatible /v1/chat/completions
│ ├── agents/ # Agent interface and implementations
│ │ ├── base.py # Abstract AgentInterface
│ │ ├── lorem_tester.py # Full-featured mock agent
│ │ ├── tatlock.py # Placeholder for real agent
│ │ └── registry.py # Model registry
│ ├── responses/ # Responses API domain (PRIMARY)
│ │ ├── router.py # POST /v1/responses
│ │ ├── schemas.py # Request/response models
│ │ ├── service.py # Business logic (currently mock)
│ │ ├── dependencies.py # Route dependencies
│ │ └── constants.py # Domain constants
│ │ ├── service.py # Response generation logic
│ │ ├── streaming.py # SSE streaming coordinator
│ │ ├── history.py # Conversation history management
│ │ └── context.py # Context window management
│ ├── chat/ # Chat Completions domain (WRAPPER)
│ │ ├── router.py # POST /v1/chat/completions
│ │ ├── schemas.py # Chat request/response models
│ │ ├── service.py # Wraps Responses API
│ │ └── constants.py # Chat constants
│ ├── models/ # Models listing domain
│ │ ├── router.py # OpenAI-compatible /v1/models
│ │ ├── router.py # GET /v1/models
│ │ ├── schemas.py # Model schemas
│ │ └── service.py # Model list service (currently mock)
│ │ └── service.py # Model registry access
│ ├── core/ # Shared utilities
│ │ ├── config.py # Global configuration (BaseSettings)
│ │ ├── models.py # Custom Pydantic base models
│ │ ├── config.py # Configuration (BaseSettings)
│ │ ├── models.py # Custom Pydantic base
│ │ ├── exceptions.py # Custom exceptions
│ │ ├── router.py # Health check & root endpoints
│ │ └── dependencies.py # Shared dependencies
│ ├── ollama/ # Ollama client (ready, not integrated yet)
│ │ ├── client.py # Async HTTP client
│ │ └── schemas.py # Ollama API models
│ └── main.py # Application factory & configuration
├── requirements.txt # Python dependencies (pinned)
├── .env # Environment variables (git-ignored)
├── .gitignore # Git ignore rules
├── AGENTS.md # LLM agent documentation + best practices
└── README.md # This file
│ │ └── router.py # Health check endpoints
│ └── main.py # Application factory
├── tests/ # Comprehensive test suite
│ ├── agents/ # Agent tests
│ ├── responses/ # Responses API tests
│ ├── chat/ # Chat completions tests
│ ├── models/ # Models API tests
│ └── core/ # Core tests
├── requirements.txt # Dependencies (pinned)
├── .env # Environment variables
├── AGENTS.md # Agent documentation
├── CLEANUP_TODO.md # Architecture notes
└── README.md # This file
```
**Key Architectural Decisions**:
- **Domain-based** structure (not file-type based)
- **Separation of concerns**: Routers → Services → Clients
- **Factory pattern** in main.py for testability
- **Custom base models** for consistent serialization
- **Async-first** for all I/O operations
### Code Style
Following FastAPI best practices:
- **Async routes** for ALL I/O operations (HTTP, database, file access)
- **Sync routes** only for CPU-intensive work or blocking SDKs
- **Type hints** on all functions and class attributes
- **Pydantic models** for ALL request/response validation
- **Dependency injection** for validation and shared resources
- **Business logic** in service modules, NOT in routers
- Follow PEP 8 style guidelines
- Document complex logic with docstrings
### Current Implementation Status
The API is currently set up with **mock responses** for development:
**✅ Implemented**:
- OpenAI-compatible API structure
- `/v1/chat/completions` endpoint (returns lorem ipsum)
- `/v1/models` endpoint (returns mistral-nemo:latest)
- `/health` and `/` endpoints
- Streaming support with SSE
- Exception handling
- Configuration management
- Async Ollama client (ready, not connected)
**🚧 TODO** (future integration):
- Connect chat completions to Ollama/PydanticAI
- Implement actual model listing from Ollama
- Add authentication/API keys
- Rate limiting
- Usage tracking
- More OpenAI-compatible endpoints
### Testing
```bash
# Install test dependencies
pip install -r requirements-dev.txt
# Run tests
# Run all tests
pytest
# Run tests with coverage
# Run with coverage
pytest --cov=src --cov-report=term-missing
# Current coverage: 62%
# Current coverage: 78.95% (75 tests passing)
```
## Documentation
**Test Organization:**
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
- Error handling tests
- Advanced features tests (stop sequences, max tokens, validation)
- See `AGENTS.md` for LLM agent instructions and package documentation
- FastAPI docs: https://fastapi.tiangolo.com/
- PydanticAI docs: https://ai.pydantic.dev/
- Ollama API: https://github.com/ollama/ollama/blob/main/docs/api.md
### Code Style
- **Async-first**: All I/O operations use async/await
- **Type hints**: All functions fully typed
- **Pydantic validation**: All request/response validation
- **Domain separation**: Clear boundaries between components
- **Single responsibility**: Each module has one clear purpose
## Security
### Version Locking
Minor version locking (`>=X.Y,<X.(Y+1)`) for security:
- Allows patch updates
- Blocks potentially breaking minor updates
- All dependencies checked for CVEs (2025-12-06)
### Best Practices
1. Never commit `.env` files
2. Use environment variables for sensitive config
3. Keep dependencies updated monthly
4. Validate all inputs with Pydantic
5. Use HTTPS in production
6. Implement rate limiting
## Deployment
### Docker Deployment (Coming Soon)
### Production Server
```bash
docker-compose up -d
# Multiple workers for production
uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4
```
### Production Considerations
### Considerations
- Use a production ASGI server (uvicorn with multiple workers)
- Enable HTTPS with reverse proxy (nginx/caddy)
- Implement rate limiting
- Use reverse proxy (nginx/caddy) for HTTPS
- Enable rate limiting (SlowAPI or similar)
- Set up monitoring and logging
- Use a process manager (systemd/supervisor)
- Configure proper resource limits
- (Future) Ensure reliable network connectivity to Ollama instance
- (Future) Consider Ollama failover/redundancy strategies
- Configure resource limits
- Use process manager (systemd/supervisor)
## Troubleshooting
### Common Issues
**Issue**: Import errors
- Ensure virtual environment is activated
- Reinstall dependencies: `pip install -r requirements.txt`
- Verify Python 3.12+ is being used
**Issue**: Streaming not working
**Streaming not working:**
- Verify SSE-Starlette is installed
- Check client supports Server-Sent Events
- Review browser/tool compatibility
- Check test suite: `pytest tests/chat/test_router.py -k streaming`
- Test with: `pytest tests/responses/ -k streaming`
**Issue**: Tests failing
**Open WebUI can't connect:**
- Use Docker bridge gateway IP: `172.17.0.1:8000`
- Check firewall settings
- Verify server is running on `0.0.0.0`
**Tests failing:**
- Install test dependencies: `pip install -r requirements-dev.txt`
- Check async test configuration in `pytest.ini`
- Run with verbose output: `pytest -v`
- Activate virtual environment
- Run with verbose: `pytest -v`
**Reasoning not showing:**
- Ensure using Chat Completions endpoint (auto-enables reasoning)
- Or manually enable in Responses API: `"reasoning": {"effort": "medium", "summary": "auto"}`
- Check Open WebUI version supports `<think>` tags
## Future Roadmap
### Short-term
- [ ] Connect tatlock model to real PydanticAI agent
- [ ] Implement vector memory (Qdrant integration)
- [ ] Add authentication/API keys
- [ ] Rate limiting middleware
### Long-term
- [ ] Multi-model support (OpenAI, Anthropic, etc.)
- [ ] Advanced conversation memory
- [ ] Tool/function calling integration
- [ ] Usage tracking and analytics
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request
3. Make changes with tests
4. Ensure tests pass: `pytest`
5. Submit pull request
## Documentation
- **AGENTS.md**: Agent architecture and best practices
- **CLEANUP_TODO.md**: Architecture decisions and future considerations
- **CHANGELOG.md**: Version history
- OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses
- FastAPI: https://fastapi.tiangolo.com/
- PydanticAI: https://ai.pydantic.dev/
## License
@@ -298,5 +493,4 @@ docker-compose up -d
---
For detailed changelog, see [CHANGELOG.md](CHANGELOG.md).
For AI assistant instructions and package documentation, see [AGENTS.md](AGENTS.md).
**Note**: This is a testing/development API with mock responses. The architecture is production-ready and designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).