jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

docs: streamline AGENTS.md for clarity

Browse Source 
Simplify development guidelines and operational protocols

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-11 21:37:35 +01:00
co-authored by Claude Opus 4.5
parent 27375cd6d2
commit a53fd67f4f
1 changed files with 35 additions and 529 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+35 -529
View File
@@ -3,542 +3,48 @@
This document contains instructions and documentation references for AI assistants working with this codebase.
> **📖 Important**: Before working on this project, read [PHILOSOPHY.md](PHILOSOPHY.md) to understand the system vision, architectural patterns, and design goals. All development should work towards realizing those patterns.
# AGENTS.md
## Project Overview
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer.
## 1. Agent Operational Protocols
### Architecture Pattern
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
The **Orchestrator** infrastructure layer with hybrid API architecture:
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
```
Client (Open WebUI)
↓
Chat Completions (/v1/chat/completions) → Wrapper
↓
Responses API (/v1/responses) → Primary
↓
Agent Interface (lorem-tester, Tatlock)
↓
Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
```
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
**Architectural Layers:**
---
1. **The Orchestrator** (Current Implementation)
- FastAPI application providing the infrastructure
- HTTP/SSE endpoints, streaming coordination
- Conversation history and context management
- OpenAI-compatible API surface
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
2. **Future: The Household** (Phases 1-4)
- **Steward**: First-tier LLM for request analysis (PydanticAI agent)
- **Tatlock**: Second-tier LLM with butler personality (PydanticAI agent)
- **Expert Agents**: Domain specialists (Librarian, Developer, Handyman, etc.)
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Key Architectural Decisions:**
1. **Single Source of Truth**: All response generation happens in the Responses API
- Structured output with reasoning, function_call, and message items
- Real-time stop sequence and max tokens enforcement
- Conversation history tracking
- Context window management
2. **Chat Completions Wrapper**: Provides compatibility without duplicating logic
- Calls Responses API internally
- Automatically enables reasoning generation
- Converts reasoning items to `<think>` tags for Open WebUI
- Maintains OpenAI-compatible format
3. **Agent Interface**: Clean abstraction for multiple models
- **lorem-tester**: Full-featured mock agent with realistic behavior
- Reasoning summaries (adjustable effort levels)
- Random tool/function calls
- Error triggers for testing
- Temperature variation
- **Tatlock**: Advertised model name (currently mock, future: PydanticAI Butler agent)
4. **Hybrid Conversation History**:
- Client MUST send full context in `input` array (OpenAI compatible)
- Server optionally tracks via `metadata.conversation_id`
- Auto-generates deterministic IDs from first message
- Supports future vector memory integration (Qdrant)
**Why This Architecture?**
- **Open WebUI Compatibility**: Native Responses API support not yet in stable release
- **Future-Proof**: Easy migration when Open WebUI adds native support
- **Testability**: Full-featured mock agent (lorem-tester) for integration testing
- **Clean Separation**: Responses API as stable core, wrappers can change
### Components
- **FastAPI**: Web framework for the API layer
- **SSE-Starlette**: Server-Sent Events for streaming responses
- **Pydantic**: Request/response validation with field validators
- **Agent Interface**: Abstract base class for model implementations
- **Conversation History**: Server-side tracking with configurable max turns
- **Context Window**: Token counting and management
- **PydanticAI**: Integrated with Tatlock agent (Ollama backend)
- **Agent Tools**: Permanent tools module (`src/agents/tools.py`)
- Calculator: Safe mathematical expression evaluation
- Date/Time toolkit: Current time, relative dates, time differences
- Web Search: SearXNG integration for privacy-preserving search
## Documentation References
### Core Framework Documentation
#### FastAPI
- **Official Documentation**: https://fastapi.tiangolo.com/
- **Version**: 0.123.9 (Dec 2025)
- **Key Topics**:
- Path operations and routing
- Request/response models with Pydantic
- Dependency injection
- Background tasks
- WebSocket and streaming support
- **PyPI**: https://pypi.org/project/fastapi/
#### Uvicorn
- **Official Documentation**: https://www.uvicorn.org/
- **Version**: 0.38.0 (Oct 2025)
- **Key Topics**:
- ASGI server configuration
- Deployment settings
- Logging and monitoring
- SSL/TLS configuration
### AI/LLM Integration
#### PydanticAI
- **Official Documentation**: https://ai.pydantic.dev/
- **Version**: 1.27.0 (Dec 2025)
- **Status**: Dependency installed, ready for future integration
- **Key Topics** (for future implementation):
- Agent creation and configuration
- LLM provider integration (Ollama support)
- Structured outputs with Pydantic
- Streaming responses
- Tool/function calling
- RunContext and dynamic configuration
- MCP server integration
- **GitHub**: https://github.com/pydantic/pydantic-ai
- **PyPI**: https://pypi.org/project/pydantic-ai/
#### Pydantic
- **Official Documentation**: https://docs.pydantic.dev/latest/
- **Version**: 2.11+ (Required for PydanticAI, currently using >=2.11,<2.13)
- **Key Topics**:
- Data validation and serialization
- Field types and validators
- Model configuration
- JSON schema generation
### HTTP and Streaming
#### HTTPX
- **Official Documentation**: https://www.python-httpx.org/
- **Version**: 0.28.1
- **Key Topics**:
- Async HTTP client for Ollama communication
- Streaming responses
- Timeout configuration
- Connection pooling
#### SSE-Starlette
- **GitHub**: https://github.com/sysid/sse-starlette
- **Version**: 3.0.2 (Oct 2025)
- **Key Topics**:
- Server-Sent Events implementation
- Streaming event responses
- Integration with FastAPI/Starlette
### Ollama Integration
#### Ollama API
- **Official Documentation**: https://github.com/ollama/ollama/blob/main/docs/api.md
- **Status**: Async client implemented in `src/ollama/client.py`, ready for future integration
- **Key Topics** (for future implementation):
- REST API endpoints
- Streaming responses
- Model management
- Generate and chat endpoints
- Model configuration
- **Current Model Target**: mistral-nemo:latest
### OpenAI API Compatibility
#### OpenAI API Reference
- **Official Documentation**: https://platform.openai.com/docs/api-reference
- **Key API Endpoints**:
- `/v1/responses` - Responses API (PRIMARY) with structured output
- `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper
- `/v1/models` - List available models
- **Key Features for Development**:
- **Responses API Format**: Structured output with reasoning, function_call, and message items
- **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences
- **Conversation History**: Hybrid client/server approach with auto-generated IDs
- **Context Management**: Token counting and window trimming
- **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement
- **Error Handling**: Custom exception types (RateLimitError, ContextLengthError)
- **Tool Calling**: PydanticAI tool integration with permanent tools
- **Testing**: Comprehensive test suite with mocks and real Ollama integration
## FastAPI Best Practices
This project follows best practices from [github.com/zhanymkanov/fastapi-best-practices](https://github.com/zhanymkanov/fastapi-best-practices)
### Project Structure
**Domain-Based Organization**: Code is organized by domain/feature rather than by file type:
```
**Correct Structure:**
```text
src/
├── agents/ # Agent interface and implementations
│ ├── base.py # Abstract AgentInterface
│ ├── lorem_tester.py # Full-featured mock agent
│ ├── tatlock.py # Placeholder for real agent
│ └── registry.py # ModelRegistry for agent management
├── responses/ # Responses API domain (PRIMARY)
│ ├── router.py # POST /v1/responses endpoint
│ ├── schemas.py # Request/response models with validators
│ ├── service.py # Response generation logic
│ ├── streaming.py # SSE streaming coordinator
│ ├── history.py # Conversation history management
│ └── context.py # Context window and token management
├── chat/ # Chat Completions domain (WRAPPER)
│ ├── router.py # POST /v1/chat/completions endpoint
│ ├── schemas.py # Chat request/response models
│ ├── service.py # Wraps Responses API, converts to <think> tags
│ ├── constants.py # Chat constants (roles, finish reasons)
│ └── __init__.py
├── models/ # Models listing domain
│ ├── router.py # GET /v1/models endpoint
│ ├── schemas.py # Model schemas
│ ├── service.py # Accesses ModelRegistry
│ └── __init__.py
├── core/ # Shared utilities
│ ├── config.py # Global configuration (BaseSettings)
│ ├── models.py # Custom base Pydantic models
│ ├── exceptions.py # Custom exceptions (RateLimitError, etc.)
│ ├── dependencies.py # Shared dependencies
│ └── router.py # Core routes (health, root)
├── ollama/ # Ollama client layer (not yet integrated)
│ ├── client.py # Async Ollama HTTP client
│ └── schemas.py # Ollama API models
└── main.py # Application factory & configuration
```
**Key Architectural Principles**:
- **Single Source of Truth**: Responses API handles all generation logic
- **Wrapper Pattern**: Chat Completions wraps Responses API without duplicating code
- **Agent Abstraction**: AgentInterface defines contract for all models
- **Domain Separation**: Each domain has its own router, schemas, service
- **Service Layer**: Business logic in services, not routers
- **Type Safety**: Pydantic models for ALL request/response validation
- **Async First**: All I/O operations use async/await
### Async/Await Best Practices
**Critical Understanding**: FastAPI handles sync and async routes differently:
- **Async routes** (`async def`): Called directly in event loop
- Use ONLY for non-blocking operations
- Perfect for `await httpx.get()`, database queries, file I/O
- **NEVER** use blocking calls like `time.sleep()` - this blocks entire server
- **Sync routes** (`def`): Run in thread pool
- Use for CPU-intensive work or blocking SDKs
- Blocking I/O won't freeze the event loop
- Example: `time.sleep(10)` is safe here
**Example**:
```python
@router.get("/terrible")
async def terrible():
time.sleep(10) # ❌ BLOCKS ENTIRE SERVER
@router.get("/good")
def good():
time.sleep(10) # ✅ Runs in thread pool
@router.get("/perfect")
async def perfect():
await asyncio.sleep(10) # ✅ Non-blocking async
```
**For CPU-intensive tasks**: Use separate worker processes (not threads) due to Python's GIL.
### Pydantic Configuration
**Custom Base Model**: All schemas inherit from `CustomBaseModel` for consistent behavior:
```python
# src/core/models.py
class CustomBaseModel(BaseModel):
model_config = ConfigDict(
json_encoders={datetime: datetime_to_iso_str},
populate_by_name=True,
use_enum_values=True,
validate_assignment=True,
)
def serializable_dict(self, **kwargs):
"""Return dict with only JSON-serializable fields."""
return jsonable_encoder(self.model_dump(**kwargs))
```
**Benefits**:
- Consistent datetime serialization across all responses
- Alias support for field name flexibility
- Easy JSON encoding for logging/debugging
**Decoupled Settings**: Split configuration by domain instead of one monolithic file:
```python
# src/core/config.py - Global settings
class Config(BaseSettings):
DATABASE_URL: PostgresDsn
ENVIRONMENT: Environment
# src/chat/config.py - Chat-specific settings
class ChatConfig(BaseSettings):
MAX_TOKENS: int
DEFAULT_TEMPERATURE: float
```
### Dependency Injection Patterns
**Validation with Dependencies**: Use dependencies for complex validations:
```python
async def valid_post_id(post_id: UUID4) -> dict:
"""Validate post exists in database."""
post = await service.get_by_id(post_id)
if not post:
raise PostNotFound()
return post
@router.get("/posts/{post_id}")
async def get_post(post: dict = Depends(valid_post_id)):
return post # Already validated!
```
**Chaining Dependencies**: Build reusable validation layers:
```python
async def valid_owned_post(
post: dict = Depends(valid_post_id),
token_data: dict = Depends(parse_jwt_data),
) -> dict:
if post["creator_id"] != token_data["user_id"]:
raise UserNotOwner()
return post
```
**Dependency Caching**: Dependencies are cached within request scope - FastAPI only executes each dependency once per request, even if used multiple times.
### Application Factory Pattern
Main.py uses factory pattern for testability and configuration:
```python
def create_application() -> FastAPI:
"""Create and configure FastAPI app."""
app = FastAPI(title=config.APP_NAME)
# Add middleware
app.add_middleware(CORSMiddleware, ...)
# Register exception handlers
register_exception_handlers(app)
# Include routers
app.include_router(chat_router, prefix="/v1")
return app
app = create_application()
```
## Development Guidelines
### Git Workflow
**IMPORTANT**: Do NOT handle git commits or pushes automatically. Wait for explicit user instruction before:
- Running `git add`
- Running `git commit`
- Running `git push`
- Creating or pushing tags
The user will manage git operations themselves unless they specifically request assistance.
### Server Logs and Debugging
**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is:
- Cleared on each server startup (fresh logs every time)
- Written in real-time as the server runs
- Already gitignored (won't be committed)
**Accessing Logs**: You can read the log file at any time while the server is running:
```bash
# View current logs
cat logs/server.log
# Follow logs in real-time
tail -f logs/server.log
# Search logs
grep "ERROR" logs/server.log
```
This is useful for debugging issues, monitoring API calls, and understanding server behavior during development.
### Code Structure Guidelines
- Use async/await for ALL I/O operations (database, HTTP, file access)
- Use sync (def) for blocking SDKs or CPU-intensive work
- Implement proper error handling and logging
- Follow dependency injection for validation and shared resources
- Use Pydantic models for ALL request/response validation
- Keep business logic in service modules, not routers
- Domain-based project structure (not file-type based)
### Security Considerations
- Validate all inputs using Pydantic models
- Use environment variables for sensitive configuration
- Keep dependencies updated and CVE-checked
- Minor version locking for supply chain protection
- Consider rate limiting for production deployment
- Plan for authentication/API keys when needed
### Testing Approach
- Write integration tests for API endpoints
- Test streaming functionality with appropriate timeouts
- Use pytest-asyncio for async test support
- Validate OpenAI API compatibility in tests
- Test both mock and real LLM integrations
- Cover main application (CORS, exception handlers, lifespan)
- Test wrapper layers (chat completions, etc.)
- Include tool functionality tests
### Configuration Management
- Use `.env` files for local development
- Document all environment variables in README
- Provide sensible defaults where possible
- Use BaseSettings from pydantic-settings
- Support both local and container-based configuration
## Common Patterns
### Streaming Response Pattern
Example from `src/chat/router.py`:
```python
from sse_starlette.sse import EventSourceResponse
from fastapi import FastAPI
async def event_generator():
# Currently yields mock lorem ipsum chunks
# Future: Stream from Ollama/PydanticAI
yield {"data": chunk.model_dump_json()}
yield {"data": "[DONE]"}
@app.post("/stream")
async def stream():
return EventSourceResponse(event_generator())
```
### PydanticAI Agent Pattern
When implementing agents with PydanticAI and Ollama:
```python
from pydantic_ai import Agent
agent = Agent(
'ollama:mistral-nemo', # Target model
# Configuration here
)
# Use the agent
result = await agent.run('Your prompt')
```
### OpenAI-Compatible Response Format
Example schema from `src/chat/schemas.py`:
```python
{
"id": "chatcmpl-123",
"object": "chat.completion.chunk",
"created": 1234567890,
"model": "mistral-nemo:latest",
"choices": [{
"index": 0,
"delta": {"content": "response"},
"finish_reason": None
}]
}
```
### PydanticAI Tool Registration Pattern
Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples:
```python
from pydantic_ai import Agent, RunContext
# After creating the agent
@agent.tool
def tool_name(ctx: RunContext[None], param: str) -> str:
"""
Tool description that the LLM sees.
Args:
param: Parameter description
Returns:
Result description
"""
return result
```
**Tool Implementation Guidelines**:
- Keep tools in `src/agents/tools.py` for reusability
- Use clear, descriptive docstrings (LLM reads these)
- Include parameter descriptions in docstrings
- Handle errors gracefully and return error messages as strings
- For async operations, declare the tool function as `async def`
- Test tools independently before integration
**Example Tool Module** (`src/agents/tools.py`):
```python
def calculate(expression: str) -> str:
"""Safe calculator implementation."""
try:
# Implementation
return str(result)
except Exception as e:
return f"Error: {str(e)}"
async def search_web(query: str) -> str:
"""Web search via SearXNG."""
async with httpx.AsyncClient() as client:
# Implementation
return formatted_results
```
## Update Policy
This document should be updated when:
- New development patterns are established
- Package versions are upgraded
- Major architectural changes occur
- New best practices are identified
Last updated: 2025-12-06 (Tools integration)
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point