OpenAI-Compatible API

A FastAPI-based service that provides OpenAI-compatible API endpoints with streaming support. Currently returns mock responses - ready for future Ollama/PydanticAI integration.

Current Status

✅ Production-ready mock API with OpenAI-compatible format 🚧 Ollama/PydanticAI integration prepared but not connected

Components

• FastAPI: High-performance web framework providing the API layer
• SSE-Starlette: Server-Sent Events for streaming responses
• Pydantic: Type-safe request/response handling and validation
• Ollama Client: Async HTTP client prepared for future integration
• PydanticAI: Ready for LLM integration (not yet connected)

Features

• ✅ OpenAI-compatible API endpoints (/v1/chat/completions, /v1/models)
• ✅ Streaming responses with Server-Sent Events (SSE)
• ✅ Type-safe request/response handling with Pydantic
• ✅ Async/await throughout for optimal performance
• ✅ Comprehensive test suite (62% coverage)
• ✅ Domain-based architecture following FastAPI best practices
• 🚧 Ollama integration (client ready, not connected)
• 🚧 PydanticAI integration (dependency installed, not connected)

Requirements

• Python 3.12+ (Python 3.12.11 recommended for security)
• No external dependencies required for mock API
• (Future: Network access to Ollama instance for LLM integration)

Installation

1. Clone the repository

git clone <repository-url>
cd tatlock

2. Create a virtual environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt

4. Configure environment variables (Optional)

Create a .env file in the project root for custom configuration:

# API Configuration
API_HOST=0.0.0.0
API_PORT=8000

# Logging
LOG_LEVEL=INFO

# Future Ollama Configuration (not yet integrated)
# OLLAMA_HOST=http://localhost:11434
# OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
# OLLAMA_TIMEOUT=120

Note: The API works with defaults. Environment variables are optional for customization. Ollama configuration is prepared but not currently used.

Usage

Start the development server

uvicorn src.main:app --reload

The API will be available at http://localhost:8000

API Endpoints

Chat Completions (OpenAI-compatible)

Returns mock lorem ipsum responses:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-nemo:latest",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "stream": true
  }'

List Models

Returns mock model listing:

curl http://localhost:8000/v1/models

Response: {"object": "list", "data": [{"id": "mistral-nemo:latest", ...}]}

Interactive API Documentation

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

Security

Version Locking Strategy

This project uses minor version locking (>=X.Y,<X.(Y+1)) to protect against supply chain attacks while allowing patch updates. All dependencies have been:

• Checked for known CVEs (as of 2025-12-06)
• Pinned to secure minor versions
• Documented with version rationale in requirements.txt

CVE Status (2025-12-06)

• FastAPI 0.123.9: No known vulnerabilities
• Uvicorn 0.38.0: No known vulnerabilities
• PydanticAI 1.27.0: No known vulnerabilities
• HTTPX 0.28.1: No known vulnerabilities
• SSE-Starlette 3.0.2: No known vulnerabilities

Regular security updates are recommended. Check for new versions monthly.

Security Best Practices

1. Never commit .env files
2. Use environment variables for sensitive configuration
3. Keep dependencies updated
4. Implement rate limiting in production
5. Use HTTPS in production environments
6. Validate all inputs with Pydantic models

Development

Project Structure

Following FastAPI best practices with domain-based organization:

tatlock/
├── src/
│   ├── chat/                  # Chat completions domain
│   │   ├── router.py          # OpenAI-compatible /v1/chat/completions
│   │   ├── schemas.py         # Request/response models
│   │   ├── service.py         # Business logic (currently mock)
│   │   ├── dependencies.py    # Route dependencies
│   │   └── constants.py       # Domain constants
│   ├── models/                # Models listing domain
│   │   ├── router.py          # OpenAI-compatible /v1/models
│   │   ├── schemas.py         # Model schemas
│   │   └── service.py         # Model list service (currently mock)
│   ├── core/                  # Shared utilities
│   │   ├── config.py          # Global configuration (BaseSettings)
│   │   ├── models.py          # Custom Pydantic base models
│   │   ├── exceptions.py      # Custom exceptions
│   │   ├── router.py          # Health check & root endpoints
│   │   └── dependencies.py    # Shared dependencies
│   ├── ollama/                # Ollama client (ready, not integrated yet)
│   │   ├── client.py          # Async HTTP client
│   │   └── schemas.py         # Ollama API models
│   └── main.py                # Application factory & configuration
├── requirements.txt           # Python dependencies (pinned)
├── .env                       # Environment variables (git-ignored)
├── .gitignore                # Git ignore rules
├── AGENTS.md                 # LLM agent documentation + best practices
└── README.md                 # This file

Key Architectural Decisions:

• Domain-based structure (not file-type based)
• Separation of concerns: Routers → Services → Clients
• Factory pattern in main.py for testability
• Custom base models for consistent serialization
• Async-first for all I/O operations

Code Style

Following FastAPI best practices:

• Async routes for ALL I/O operations (HTTP, database, file access)
• Sync routes only for CPU-intensive work or blocking SDKs
• Type hints on all functions and class attributes
• Pydantic models for ALL request/response validation
• Dependency injection for validation and shared resources
• Business logic in service modules, NOT in routers
• Follow PEP 8 style guidelines
• Document complex logic with docstrings

Current Implementation Status

The API is currently set up with mock responses for development:

✅ Implemented:

• OpenAI-compatible API structure
• /v1/chat/completions endpoint (returns lorem ipsum)
• /v1/models endpoint (returns mistral-nemo:latest)
• /health and / endpoints
• Streaming support with SSE
• Exception handling
• Configuration management
• Async Ollama client (ready, not connected)

🚧 TODO (future integration):

• Connect chat completions to Ollama/PydanticAI
• Implement actual model listing from Ollama
• Add authentication/API keys
• Rate limiting
• Usage tracking
• More OpenAI-compatible endpoints

Testing

# Install test dependencies
pip install -r requirements-dev.txt

# Run tests
pytest

# Run tests with coverage
pytest --cov=src --cov-report=term-missing

# Current coverage: 62%

Documentation

• See AGENTS.md for LLM agent instructions and package documentation
• FastAPI docs: https://fastapi.tiangolo.com/
• PydanticAI docs: https://ai.pydantic.dev/
• Ollama API: https://github.com/ollama/ollama/blob/main/docs/api.md

Deployment

Docker Deployment (Coming Soon)

docker-compose up -d

Production Considerations

• Use a production ASGI server (uvicorn with multiple workers)
• Enable HTTPS with reverse proxy (nginx/caddy)
• Implement rate limiting
• Set up monitoring and logging
• Use a process manager (systemd/supervisor)
• Configure proper resource limits
• (Future) Ensure reliable network connectivity to Ollama instance
• (Future) Consider Ollama failover/redundancy strategies

Troubleshooting

Common Issues

Issue: Import errors

• Ensure virtual environment is activated
• Reinstall dependencies: pip install -r requirements.txt
• Verify Python 3.12+ is being used

Issue: Streaming not working

• Verify SSE-Starlette is installed
• Check client supports Server-Sent Events
• Review browser/tool compatibility
• Check test suite: pytest tests/chat/test_router.py -k streaming

Issue: Tests failing

• Install test dependencies: pip install -r requirements-dev.txt
• Check async test configuration in pytest.ini
• Run with verbose output: pytest -v

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

License

[Add your license here]

---

For detailed changelog, see CHANGELOG.md. For AI assistant instructions and package documentation, see AGENTS.md.