jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

Deduplicate and refocus documentation

Browse Source 
Restructured README.md and AGENTS.md to eliminate duplication:

README.md (user-focused):
- Simplified to focus on project description and usage
- Quick start guide with installation steps
- API usage examples with curl commands
- Open WebUI integration guide
- Troubleshooting section
- Deployment recommendations
- Removed internal architectural details

AGENTS.md (LLM agent instructions):
- Retained detailed architectural decisions and rationale
- FastAPI best practices and patterns
- Development guidelines and code structure
- Documentation references for frameworks
- Testing strategy and coverage details
- Updated test coverage: 78.95% (95 tests)
- Common implementation patterns

Changes:
- README.md: Streamlined from 497 to 310 lines
- AGENTS.md: Updated test coverage numbers
- Clear separation: README for users, AGENTS for AI developers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-06 20:01:49 +01:00
co-authored by Claude
parent ab8e3b1566
commit 8d0618b647
2 changed files with 121 additions and 305 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+5 -2
View File
@@ -201,10 +201,11 @@ Agent Interface (lorem-tester, tatlock)
- OpenAI-compatible error format
- Error triggers in lorem-tester for testing
- ✅ **Testing Infrastructure**:
- 75 tests (78.95% coverage)
- 95 tests (78.95% coverage)
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
- Main application and wrapper tests
## FastAPI Best Practices
@@ -402,12 +403,14 @@ app = create_application()
- 🚧 Implement rate limiting for API endpoints (future)
- 🚧 Add authentication/API keys (future)
### Testing (Current Coverage: 62%)
### Testing (Current Coverage: 78.95%, 95 tests)
- ✅ Integration tests for API endpoints
- ✅ Streaming functionality with 20s timeout protection
- ✅ Async test support with pytest-asyncio
- ✅ Validate OpenAI API compatibility
- ✅ Mock responses for all endpoints
- ✅ Main application tests (CORS, exception handlers, lifespan)
- ✅ Chat streaming wrapper tests
- 🚧 Future: Mock Ollama responses when integrated
### Configuration
README.md
+116 -303
View File
@@ -1,108 +1,47 @@
# Tatlock - OpenAI-Compatible API with Responses API
A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming. Features a hybrid architecture with chat completions as a compatibility wrapper around the Responses API.
A FastAPI-based service providing OpenAI-compatible API endpoints with full Responses API support, reasoning display, and streaming.
## Current Status
**✅ Production-ready testing API** with OpenAI Responses API format
**✅ Open WebUI integration** with reasoning bubbles (`<think>` tags)
**✅ Conversation history** with hybrid client/server approach
**🚧 PydanticAI integration** prepared for future real LLM connection
## Architecture Overview
### Hybrid API Design
```
┌─────────────────────────────────────────┐
│ Client (Open WebUI, etc.) │
└────────┬────────────────────────────────┘
│
├──────────────────────────────────┐
│ │
v v
┌────────────────────┐ ┌──────────────────────┐
│ /v1/chat/ │ wrapper │ /v1/responses │
│ completions ├─────────>│ (Primary API) │
│ │ │ │
│ • OpenAI compat │ │ • Reasoning items │
│ • <think> tags │ │ • Function calls │
│ • Legacy support │ │ • Message items │
└────────────────────┘ └──────────┬───────────┘
│
v
┌──────────────────────┐
│ Agent Interface │
│ │
│ • lorem-tester │
│ • tatlock (future) │
└──────────────────────┘
```
**Key Architectural Decisions:**
- **Single Source of Truth**: Responses API handles all generation logic
- **Chat Completions Wrapper**: Converts Responses output to Chat format with `<think>` tags
- **Agent Interface**: Clean abstraction for multiple models (mock and real)
- **Hybrid History**: Client sends full context, server optionally tracks conversations
- ✅ **Production-ready testing API** with OpenAI Responses API format
- ✅ **Open WebUI integration** with reasoning bubbles (`<think>` tags)
- ✅ **Conversation history** with auto-generated IDs and context management
- ✅ **Comprehensive testing** - 95 tests, 78.95% coverage
- 🚧 **PydanticAI integration** prepared for future real LLM connection
## Features
### Core API
- ✅ **Responses API** (`/v1/responses`) - Primary endpoint with structured output
- Reasoning items (thinking/extended thinking)
- Function call items (tool execution)
- Message items (assistant responses)
- Streaming and non-streaming modes
- ✅ **Chat Completions API** (`/v1/chat/completions`) - Compatibility wrapper
- Converts reasoning to `<think>` tags for Open WebUI
- Maintains OpenAI-compatible format
- Wraps Responses API (single source of truth)
- ✅ **Models API** (`/v1/models`) - Lists available models
### API Endpoints
### Advanced Features
- ✅ **Conversation History Management**
- Hybrid approach: client maintains state, server tracks optionally
- Auto-generated conversation IDs from first message hash
- Configurable max turns (default: 20)
- Placeholder for future vector memory (Qdrant)
- ✅ **Context Window Management**
- Approximate token counting (~4 chars/token)
- Context trimming to fit model limits
- Token usage statistics
- ✅ **Parameter Validation**
- Temperature: 0.0-2.0
- Reasoning effort: none, minimal, low, medium, high, xhigh
- Max output tokens enforcement
- Stop sequences (up to 4)
- ✅ **Stop Sequence Detection**
- Real-time detection during streaming
- Stops generation immediately when encountered
- ✅ **Max Tokens Enforcement**
- Real-time token counting during streaming
- Stops when limit reached
- **Responses API** (`/v1/responses`) - OpenAI Responses API format with structured output
- Reasoning items for displaying thinking process
- Function call items for tool execution
- Message items for assistant responses
- Streaming and non-streaming support
- **Chat Completions** (`/v1/chat/completions`) - OpenAI Chat Completions compatibility
- Automatic reasoning conversion to `<think>` tags for Open WebUI
- Full OpenAI API compatibility
- Streaming support
- **Models** (`/v1/models`) - List available models
### Advanced Capabilities
- **Conversation History**: Auto-generated IDs, configurable max turns (default: 20)
- **Context Management**: Token counting, automatic trimming, usage statistics
- **Parameter Validation**: Temperature (0.0-2.0), reasoning effort levels, max tokens, stop sequences
- **Real-time Enforcement**: Stop sequence detection and max token limits during streaming
### Testing Models
- ✅ **lorem-tester** - Full-featured mock agent
- Realistic reasoning summaries
- Random tool/function call generation
- **lorem-tester**: Full-featured mock agent with realistic behavior
- Configurable reasoning effort levels
- Random tool/function calls
- Error triggers for testing (rate_limit, context_overflow)
- Temperature variation
- ✅ **tatlock** - Placeholder for real PydanticAI agent
### Open WebUI Integration
- ✅ **Reasoning Display** - Thinking bubbles shown separately from responses
- ✅ **Streaming Support** - Smooth word-by-word streaming
- ✅ **Error Handling** - Graceful error display
- ✅ **Model Selection** - Both models available in dropdown
## Components
- **FastAPI**: High-performance web framework
- **SSE-Starlette**: Server-Sent Events for streaming
- **Pydantic**: Type-safe request/response validation
- **Agent Interface**: Abstraction for multiple model backends
- **Conversation History**: Server-side tracking with hybrid approach
- **Context Window**: Token management and trimming
- **tatlock**: Placeholder for future PydanticAI agent
## Requirements
@@ -110,46 +49,24 @@ A FastAPI-based service providing OpenAI-compatible API endpoints with full Resp
- No external dependencies for mock API
- (Future: Network access for PydanticAI integration)
## Installation
## Quick Start
### 1. Clone the repository
### Installation
```bash
git clone <repository-url>
# Clone the repository
git clone https://git.schweitz.net/jpmschweitzer/tatlock.git
cd tatlock
```
### 2. Create a virtual environment
```bash
# Create virtual environment
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
source .venv/bin/activate # Windows: .venv\Scripts\activate
### 3. Install dependencies
```bash
# Install dependencies
pip install -r requirements.txt
```
### 4. Configure environment (Optional)
Create a `.env` file for custom configuration:
```env
# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
# Logging
LOG_LEVEL=INFO
# Future: Add real LLM configuration here
```
## Usage
### Start the server
### Run the Server
```bash
uvicorn src.main:app --reload
@@ -157,11 +74,11 @@ uvicorn src.main:app --reload
API available at `http://localhost:8000`
### API Endpoints
## Usage Examples
#### Responses API (Primary)
### Responses API
OpenAI Responses API format with structured output:
Generate a response with reasoning:
```bash
curl http://localhost:8000/v1/responses \
@@ -176,7 +93,6 @@ curl http://localhost:8000/v1/responses \
"summary": "auto"
},
"max_output_tokens": 500,
"stop": ["END"],
"stream": false
}'
```
@@ -192,22 +108,12 @@ curl http://localhost:8000/v1/responses \
"output": [
{
"type": "reasoning",
"id": "reasoning_xyz",
"summary": [
"Analyzing the user's request...",
"Considering quantum mechanics principles..."
]
"summary": ["Analyzing the request...", "Considering quantum mechanics..."]
},
{
"type": "message",
"id": "msg_def456",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Quantum computing uses quantum mechanics..."
}
]
"content": [{"type": "output_text", "text": "Quantum computing uses..."}]
}
],
"usage": {
@@ -219,9 +125,7 @@ curl http://localhost:8000/v1/responses \
}
```
#### Chat Completions (Compatibility)
OpenAI-compatible format with `<think>` tags:
### Chat Completions (OpenAI-compatible)
```bash
curl http://localhost:8000/v1/chat/completions \
@@ -236,66 +140,31 @@ curl http://localhost:8000/v1/chat/completions \
}'
```
**Note**: Chat Completions automatically enables reasoning and converts it to `<think>` tags for Open WebUI compatibility.
#### List Models
### List Models
```bash
curl http://localhost:8000/v1/models
```
Returns:
```json
{
"object": "list",
"data": [
{
"id": "lorem-tester",
"object": "model",
"created": 1733529600,
"owned_by": "tatlock"
},
{
"id": "tatlock",
"object": "model",
"created": 1733529600,
"owned_by": "tatlock"
}
]
}
```
### Conversation History
Optional conversation tracking via metadata:
Optionally track conversations using metadata:
```bash
curl http://localhost:8000/v1/responses \
-H "Content-Type: application/json" \
-d '{
"model": "lorem-tester",
"input": [
{"role": "user", "content": "Hello"}
],
"metadata": {
"conversation_id": "conv_abc123"
}
"input": [{"role": "user", "content": "Hello"}],
"metadata": {"conversation_id": "conv_abc123"}
}'
```
**Hybrid Approach:**
- Client MUST send full conversation history in `input` array (OpenAI compatible)
- Server optionally tracks via `metadata.conversation_id` (for analytics, future vector memory)
- Auto-generates conversation ID from first message hash if not provided
### Interactive Documentation
- **Swagger UI**: `http://localhost:8000/docs`
- **ReDoc**: `http://localhost:8000/redoc`
**Note**: Client must send full conversation history in `input` array (OpenAI compatible). Server optionally tracks via `metadata.conversation_id` for future features.
## Open WebUI Integration
### Docker Networking
### Connection
If running Open WebUI in Docker and API on host:
@@ -306,71 +175,26 @@ http://172.17.0.1:8000/v1/chat/completions
### Reasoning Display
The Chat Completions wrapper automatically:
The Chat Completions endpoint automatically:
1. Enables reasoning generation
2. Converts reasoning items to `<think>` tags
3. Streams thinking before the actual response
2. Converts reasoning to `<think>` tags
3. Streams thinking before the response
Open WebUI displays this as:
- **Thought bubble** showing reasoning steps
- **Main response** showing the actual answer
Open WebUI displays this as thought bubbles separate from the main response.
### Testing Error Handling
Lorem-tester supports error triggers:
- **"trigger_rate_limit"** - Simulates rate limit error
- **"trigger_context_overflow"** - Simulates context length error
Use special triggers in user messages:
- `"trigger_rate_limit"` - Simulates rate limit error
- `"trigger_context_overflow"` - Simulates context length error
## Development
## API Documentation
### Project Structure
Interactive documentation available at:
- **Swagger UI**: `http://localhost:8000/docs`
- **ReDoc**: `http://localhost:8000/redoc`
Following FastAPI best practices with domain-based organization:
```
tatlock/
├── src/
│ ├── agents/ # Agent interface and implementations
│ │ ├── base.py # Abstract AgentInterface
│ │ ├── lorem_tester.py # Full-featured mock agent
│ │ ├── tatlock.py # Placeholder for real agent
│ │ └── registry.py # Model registry
│ ├── responses/ # Responses API domain (PRIMARY)
│ │ ├── router.py # POST /v1/responses
│ │ ├── schemas.py # Request/response models
│ │ ├── service.py # Response generation logic
│ │ ├── streaming.py # SSE streaming coordinator
│ │ ├── history.py # Conversation history management
│ │ └── context.py # Context window management
│ ├── chat/ # Chat Completions domain (WRAPPER)
│ │ ├── router.py # POST /v1/chat/completions
│ │ ├── schemas.py # Chat request/response models
│ │ ├── service.py # Wraps Responses API
│ │ └── constants.py # Chat constants
│ ├── models/ # Models listing domain
│ │ ├── router.py # GET /v1/models
│ │ ├── schemas.py # Model schemas
│ │ └── service.py # Model registry access
│ ├── core/ # Shared utilities
│ │ ├── config.py # Configuration (BaseSettings)
│ │ ├── models.py # Custom Pydantic base
│ │ ├── exceptions.py # Custom exceptions
│ │ └── router.py # Health check endpoints
│ └── main.py # Application factory
├── tests/ # Comprehensive test suite
│ ├── agents/ # Agent tests
│ ├── responses/ # Responses API tests
│ ├── chat/ # Chat completions tests
│ ├── models/ # Models API tests
│ └── core/ # Core tests
├── requirements.txt # Dependencies (pinned)
├── .env # Environment variables
├── AGENTS.md # Agent documentation
├── CLEANUP_TODO.md # Architecture notes
└── README.md # This file
```
### Testing
## Testing
```bash
# Run all tests
@@ -379,42 +203,9 @@ pytest
# Run with coverage
pytest --cov=src --cov-report=term-missing
# Current coverage: 78.95% (75 tests passing)
# Current: 95 tests, 78.95% coverage
```
**Test Organization:**
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
- Error handling tests
- Advanced features tests (stop sequences, max tokens, validation)
### Code Style
- **Async-first**: All I/O operations use async/await
- **Type hints**: All functions fully typed
- **Pydantic validation**: All request/response validation
- **Domain separation**: Clear boundaries between components
- **Single responsibility**: Each module has one clear purpose
## Security
### Version Locking
Minor version locking (`>=X.Y,<X.(Y+1)`) for security:
- Allows patch updates
- Blocks potentially breaking minor updates
- All dependencies checked for CVEs (2025-12-06)
### Best Practices
1. Never commit `.env` files
2. Use environment variables for sensitive config
3. Keep dependencies updated monthly
4. Validate all inputs with Pydantic
5. Use HTTPS in production
6. Implement rate limiting
## Deployment
### Production Server
@@ -424,51 +215,69 @@ Minor version locking (`>=X.Y,<X.(Y+1)`) for security:
uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4
```
### Considerations
### Recommendations
- Use reverse proxy (nginx/caddy) for HTTPS
- Enable rate limiting (SlowAPI or similar)
- Enable rate limiting
- Set up monitoring and logging
- Configure resource limits
- Use process manager (systemd/supervisor)
## Configuration
Create a `.env` file for custom configuration:
```env
# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
# Logging
LOG_LEVEL=INFO
# CORS (default: allow all)
CORS_ORIGINS=["*"]
# Future: LLM configuration
```
## Troubleshooting
### Common Issues
**Streaming not working:**
### Streaming not working
- Verify SSE-Starlette is installed
- Check client supports Server-Sent Events
- Test with: `pytest tests/responses/ -k streaming`
**Open WebUI can't connect:**
### Open WebUI can't connect
- Use Docker bridge gateway IP: `172.17.0.1:8000`
- Check firewall settings
- Verify server is running on `0.0.0.0`
**Tests failing:**
- Install test dependencies: `pip install -r requirements-dev.txt`
- Activate virtual environment
- Run with verbose: `pytest -v`
**Reasoning not showing:**
### Reasoning not showing
- Ensure using Chat Completions endpoint (auto-enables reasoning)
- Or manually enable in Responses API: `"reasoning": {"effort": "medium", "summary": "auto"}`
- Check Open WebUI version supports `<think>` tags
## Future Roadmap
## Project Structure
### Short-term
- [ ] Connect tatlock model to real PydanticAI agent
- [ ] Implement vector memory (Qdrant integration)
- [ ] Add authentication/API keys
- [ ] Rate limiting middleware
```
tatlock/
├── src/
│ ├── agents/ # Agent interface and implementations
│ ├── responses/ # Responses API (primary endpoint)
│ ├── chat/ # Chat Completions wrapper
│ ├── models/ # Models listing
│ ├── core/ # Shared utilities and config
│ └── main.py # Application entry point
├── tests/ # Comprehensive test suite
├── AGENTS.md # LLM agent development guidelines
├── CHANGELOG.md # Version history
└── README.md # This file
```
### Long-term
- [ ] Multi-model support (OpenAI, Anthropic, etc.)
- [ ] Advanced conversation memory
- [ ] Tool/function calling integration
- [ ] Usage tracking and analytics
## Development
For LLM agent development guidelines and architectural decisions, see [AGENTS.md](AGENTS.md).
## Contributing
@@ -480,17 +289,21 @@ uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4
## Documentation
- **AGENTS.md**: Agent architecture and best practices
- **CLEANUP_TODO.md**: Architecture decisions and future considerations
- **CHANGELOG.md**: Version history
- OpenAI Responses API: https://platform.openai.com/docs/api-reference/responses
- FastAPI: https://fastapi.tiangolo.com/
- PydanticAI: https://ai.pydantic.dev/
- **Project Documentation**: This file
- **Agent Guidelines**: [AGENTS.md](AGENTS.md)
- **Version History**: [CHANGELOG.md](CHANGELOG.md)
- **OpenAI Responses API**: https://platform.openai.com/docs/api-reference/responses
- **FastAPI**: https://fastapi.tiangolo.com/
- **PydanticAI**: https://ai.pydantic.dev/
## License
[Add your license here]
## Version
Current version: **0.1.1** - Basic setup complete
---
**Note**: This is a testing/development API with mock responses. The architecture is production-ready and designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).
**Note**: This is a production-ready testing API with mock responses. The architecture is designed for easy integration with real LLM backends (PydanticAI, Ollama, OpenAI, etc.).