tatlock/README.md

# OpenAI-Compatible API with Ollama Backend

A FastAPI-based service that provides an OpenAI-compatible API endpoint, powered by PydanticAI and Ollama for LLM inference.

## Architecture

```
┌─────────────┐      ┌──────────────────┐      ┌─────────────┐
│   Client    │─────▶│  FastAPI Server  │─────▶│   Ollama    │
│             │◀─────│ (Stream Coord.)  │◀─────│  Container  │
└─────────────┘      └──────────────────┘      └─────────────┘
                              │
                              ▼
                        ┌──────────┐
                        │Pydantic  │
                        │   AI     │
                        └──────────┘
```

### Components

- **FastAPI**: High-performance web framework providing the API layer
- **Stream Coordinator**: Manages streaming responses in OpenAI-compatible format
- **PydanticAI**: Agent framework handling LLM integration and structured outputs
- **Ollama**: External LLM backend (networked, managed separately)
- **SSE-Starlette**: Server-Sent Events for streaming responses

## Features

- OpenAI-compatible API endpoints
- Streaming responses with Server-Sent Events
- PydanticAI integration for robust LLM interactions
- Networked Ollama support
- Type-safe request/response handling with Pydantic
- Async/await throughout for optimal performance

## Requirements

- Python 3.12+ (Python 3.12.11 recommended for security)
- Network access to an existing Ollama instance (managed externally)

## Installation

### 1. Clone the repository

```bash
git clone <repository-url>
cd tatlock
```

### 2. Create a virtual environment

```bash
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### 3. Install dependencies

```bash
pip install -r requirements.txt
```

### 4. Configure environment variables

Create a `.env` file in the project root:

```env
# Ollama Configuration (point to your existing Ollama instance)
OLLAMA_HOST=http://your-ollama-host:11434
OLLAMA_MODEL=llama2

# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
API_RELOAD=true

# Logging
LOG_LEVEL=info
```

**Note**: Update `OLLAMA_HOST` to point to your existing Ollama instance. Ensure the Ollama service is accessible from your network and has the required models installed.

## Usage

### Start the development server

```bash
uvicorn main:app --reload
```

The API will be available at `http://localhost:8000`

### API Endpoints

#### Chat Completions (OpenAI-compatible)

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama2",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "stream": true
  }'
```

#### List Models

```bash
curl http://localhost:8000/v1/models
```

### Interactive API Documentation

- Swagger UI: `http://localhost:8000/docs`
- ReDoc: `http://localhost:8000/redoc`

## Security

### Version Locking Strategy

This project uses minor version locking (`>=X.Y,<X.(Y+1)`) to protect against supply chain attacks while allowing patch updates. All dependencies have been:

- Checked for known CVEs (as of 2025-12-05)
- Pinned to secure minor versions
- Documented with version rationale in `requirements.txt`

### CVE Status (2025-12-05)

- **FastAPI 0.123.9**: No known vulnerabilities
- **Uvicorn 0.38.0**: No known vulnerabilities
- **PydanticAI 1.27.0**: No known vulnerabilities
- **HTTPX 0.28.1**: No known vulnerabilities
- **SSE-Starlette 3.0.2**: No known vulnerabilities

Regular security updates are recommended. Check for new versions monthly.

### Security Best Practices

1. Never commit `.env` files
2. Use environment variables for sensitive configuration
3. Keep dependencies updated
4. Implement rate limiting in production
5. Use HTTPS in production environments
6. Validate all inputs with Pydantic models

## Development

### Project Structure

Following [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices) with domain-based organization:

```
tatlock/
├── src/
│   ├── chat/                  # Chat completions domain
│   │   ├── router.py          # OpenAI-compatible /v1/chat/completions
│   │   ├── schemas.py         # Request/response models
│   │   ├── service.py         # Business logic (currently mock)
│   │   ├── dependencies.py    # Route dependencies
│   │   └── constants.py       # Domain constants
│   ├── models/                # Models listing domain
│   │   ├── router.py          # OpenAI-compatible /v1/models
│   │   ├── schemas.py         # Model schemas
│   │   └── service.py         # Model list service (currently mock)
│   ├── core/                  # Shared utilities
│   │   ├── config.py          # Global configuration (BaseSettings)
│   │   ├── models.py          # Custom Pydantic base models
│   │   ├── exceptions.py      # Custom exceptions
│   │   ├── router.py          # Health check & root endpoints
│   │   └── dependencies.py    # Shared dependencies
│   ├── ollama/                # Ollama client (ready, not integrated yet)
│   │   ├── client.py          # Async HTTP client
│   │   └── schemas.py         # Ollama API models
│   └── main.py                # Application factory & configuration
├── requirements.txt           # Python dependencies (pinned)
├── .env                       # Environment variables (git-ignored)
├── .gitignore                # Git ignore rules
├── AGENTS.md                 # LLM agent documentation + best practices
└── README.md                 # This file
```

**Key Architectural Decisions**:
- **Domain-based** structure (not file-type based)
- **Separation of concerns**: Routers → Services → Clients
- **Factory pattern** in main.py for testability
- **Custom base models** for consistent serialization
- **Async-first** for all I/O operations

### Code Style

Following FastAPI best practices:
- **Async routes** for ALL I/O operations (HTTP, database, file access)
- **Sync routes** only for CPU-intensive work or blocking SDKs
- **Type hints** on all functions and class attributes
- **Pydantic models** for ALL request/response validation
- **Dependency injection** for validation and shared resources
- **Business logic** in service modules, NOT in routers
- Follow PEP 8 style guidelines
- Document complex logic with docstrings

### Current Implementation Status

The API is currently set up with **mock responses** for development:

**✅ Implemented**:
- OpenAI-compatible API structure
- `/v1/chat/completions` endpoint (returns lorem ipsum)
- `/v1/models` endpoint (returns mistral-nemo:latest)
- `/health` and `/` endpoints
- Streaming support with SSE
- Exception handling
- Configuration management
- Async Ollama client (ready, not connected)

**🚧 TODO** (future integration):
- Connect chat completions to Ollama/PydanticAI
- Implement actual model listing from Ollama
- Add authentication/API keys
- Rate limiting
- Usage tracking
- More OpenAI-compatible endpoints

### Testing

```bash
# Install test dependencies
pip install pytest pytest-asyncio httpx

# Run tests
pytest
```

## Documentation

- See `AGENTS.md` for LLM agent instructions and package documentation
- FastAPI docs: https://fastapi.tiangolo.com/
- PydanticAI docs: https://ai.pydantic.dev/
- Ollama API: https://github.com/ollama/ollama/blob/main/docs/api.md

## Deployment

### Docker Deployment (Coming Soon)

```bash
docker-compose up -d
```

### Production Considerations

- Use a production ASGI server (uvicorn with multiple workers)
- Enable HTTPS with reverse proxy (nginx/caddy)
- Implement rate limiting
- Set up monitoring and logging
- Use a process manager (systemd/supervisor)
- Configure proper resource limits
- Ensure reliable network connectivity to Ollama instance
- Consider Ollama failover/redundancy strategies

## Troubleshooting

### Common Issues

**Issue**: Cannot connect to Ollama
- Verify `OLLAMA_HOST` in `.env` points to the correct address
- Ensure the Ollama instance is running and accessible
- Check network connectivity and firewall rules
- Test connectivity: `curl http://your-ollama-host:11434/api/tags`

**Issue**: Streaming not working
- Verify SSE-Starlette is installed
- Check client supports Server-Sent Events
- Review browser/tool compatibility

**Issue**: Import errors
- Ensure virtual environment is activated
- Reinstall dependencies: `pip install -r requirements.txt`

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

## License

[Add your license here]

## Changelog

### 2025-12-05 - Initial Setup
- Project structure created
- Dependencies configured with CVE checks
- Security: Minor version locking implemented
- Documentation added

---

For AI assistant instructions and package documentation, see [AGENTS.md](AGENTS.md).