jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

docs: update documentation for v0.2.0 tools release

Browse Source 
README.md:
- Add Tatlock agent capabilities and tool descriptions
- Add requirements section (Ollama, SearXNG setup)
- Add configuration examples for external services
- Add tool usage examples and philosophy
- Add troubleshooting for Ollama and SearXNG
- Update test statistics

AGENTS.md:
- Refactor for LLM development focus
- Add PydanticAI tool registration pattern
- Add tool implementation guidelines
- Remove project status, focus on development instructions

IMPLEMENTATION_ROADMAP.md:
- Mark Phase 1 as "MOSTLY COMPLETE"
- Update detailed completion status
- Update current state summary
This commit is contained in:
jpmschweitzer
2025-12-07 00:13:41 +01:00
parent 958363d44e
commit fd459e9ffb
3 changed files with 272 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+129 -98
View File
@@ -6,14 +6,11 @@ This document contains instructions and documentation references for AI assistan
## Project Overview
This project implements an OpenAI-compatible API endpoint using FastAPI, with streaming support. Currently returns mock responses - infrastructure prepared for future Ollama/PydanticAI integration.
This project implements an OpenAI-compatible API with FastAPI, featuring a hybrid architecture that provides both the OpenAI Responses API and Chat Completions compatibility layer.
**Current State**: Production-ready testing API with Responses API and Open WebUI integration
**Future Integration**: PydanticAI for real LLM agents (tatlock model placeholder ready)
### Architecture Pattern
### Current Architecture (As of 2025-12-06)
This project implements the **Orchestrator** infrastructure layer with a hybrid API architecture:
The **Orchestrator** infrastructure layer with hybrid API architecture:
```
Client (Open WebUI)
@@ -83,7 +80,11 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
- **Agent Interface**: Abstract base class for model implementations
- **Conversation History**: Server-side tracking with configurable max turns
- **Context Window**: Token counting and management
- **PydanticAI**: Dependency installed, ready for tatlock agent implementation
- **PydanticAI**: Integrated with Tatlock agent (Ollama backend)
- **Agent Tools**: Permanent tools module (`src/agents/tools.py`)
- Calculator: Safe mathematical expression evaluation
- Date/Time toolkit: Current time, relative dates, time differences
- Web Search: SearXNG integration for privacy-preserving search
## Documentation References
@@ -171,58 +172,20 @@ Mock Agents (lorem-tester) / Future: PydanticAI Agents (Tatlock, Steward, etc.)
#### OpenAI API Reference
- **Official Documentation**: https://platform.openai.com/docs/api-reference
- **Implemented Endpoints**:
- ✅ `/v1/responses` - **Responses API (PRIMARY)** with structured output
- Reasoning items (thinking summaries)
- Function call items (tool execution)
- Message items (assistant responses)
- Full streaming support with SSE
- Stop sequence detection
- Max tokens enforcement
- Conversation history tracking
- ✅ `/v1/chat/completions` - **Compatibility wrapper** around Responses API
- Converts reasoning to `<think>` tags for Open WebUI
- Automatically enables reasoning generation
- Maintains OpenAI-compatible format
- Supports streaming and non-streaming
- ✅ `/v1/models` - List available models (lorem-tester, tatlock)
- **Future Endpoints**:
- 🚧 `/v1/completions` - Text completion (legacy)
- 🚧 `/v1/embeddings` - Text embeddings
- **Implemented Features**:
- ✅ **Responses API Format**:
- Structured output items (reasoning, function_call, message)
- Extended thinking support
- Tool/function calling support
- Streaming with multiple event types
- ✅ **Advanced Parameter Validation**:
- Temperature: 0.0-2.0 with Pydantic validators
- Reasoning effort: none, minimal, low, medium, high, xhigh
- Max output tokens: positive integer enforcement
- Stop sequences: up to 4, non-empty strings
- ✅ **Conversation History**:
- Hybrid client/server approach
- Auto-generated conversation IDs
- Configurable max turns (default: 20)
- Placeholder for vector memory
- ✅ **Context Management**:
- Approximate token counting (~4 chars/token)
- Context window trimming
- Usage statistics
- ✅ **Streaming Enforcement**:
- Real-time stop sequence detection
- Real-time max tokens enforcement
- Word-by-word streaming with delays
- ✅ **Error Handling**:
- Custom exception types (RateLimitError, ContextLengthError)
- OpenAI-compatible error format
- Error triggers in lorem-tester for testing
- ✅ **Testing Infrastructure**:
- 95 tests (78.95% coverage)
- Unit tests for all components
- Integration tests for API endpoints
- Streaming tests for SSE functionality
- Main application and wrapper tests
- **Key API Endpoints**:
- `/v1/responses` - Responses API (PRIMARY) with structured output
- `/v1/chat/completions` - OpenAI Chat Completions compatibility wrapper
- `/v1/models` - List available models
- **Key Features for Development**:
- **Responses API Format**: Structured output with reasoning, function_call, and message items
- **Parameter Validation**: Temperature, reasoning effort levels, max tokens, stop sequences
- **Conversation History**: Hybrid client/server approach with auto-generated IDs
- **Context Management**: Token counting and window trimming
- **Streaming**: Real-time SSE streaming with stop sequence and max token enforcement
- **Error Handling**: Custom exception types (RateLimitError, ContextLengthError)
- **Tool Calling**: PydanticAI tool integration with permanent tools
- **Testing**: Comprehensive test suite with mocks and real Ollama integration
## FastAPI Best Practices
@@ -413,45 +376,66 @@ app = create_application()
The user will manage git operations themselves unless they specifically request assistance.
### Code Structure (Current Implementation)
- ✅ Use async/await for ALL I/O operations (database, HTTP, file access)
- ✅ Use sync (def) for blocking SDKs or CPU-intensive work
- ✅ Implement proper error handling and logging
- ✅ Follow dependency injection for validation and shared resources
- ✅ Use Pydantic models for ALL request/response validation
- ✅ Keep business logic in service modules, not routers
- ✅ Domain-based project structure (not file-type based)
### Server Logs and Debugging
**Development Mode Logging**: When the server is started using `./wakeup.sh`, logs are written to `logs/server.log`. This file is:
- Cleared on each server startup (fresh logs every time)
- Written in real-time as the server runs
- Already gitignored (won't be committed)
**Accessing Logs**: You can read the log file at any time while the server is running:
```bash
# View current logs
cat logs/server.log
# Follow logs in real-time
tail -f logs/server.log
# Search logs
grep "ERROR" logs/server.log
```
This is useful for debugging issues, monitoring API calls, and understanding server behavior during development.
### Code Structure Guidelines
- Use async/await for ALL I/O operations (database, HTTP, file access)
- Use sync (def) for blocking SDKs or CPU-intensive work
- Implement proper error handling and logging
- Follow dependency injection for validation and shared resources
- Use Pydantic models for ALL request/response validation
- Keep business logic in service modules, not routers
- Domain-based project structure (not file-type based)
### Security Considerations
- ✅ Validate all inputs using Pydantic models
- ✅ Use environment variables for sensitive configuration
- ✅ Keep dependencies updated (all CVE-checked as of 2025-12-06)
- ✅ Minor version locking for supply chain protection
- 🚧 Implement rate limiting for API endpoints (future)
- 🚧 Add authentication/API keys (future)
- Validate all inputs using Pydantic models
- Use environment variables for sensitive configuration
- Keep dependencies updated and CVE-checked
- Minor version locking for supply chain protection
- Consider rate limiting for production deployment
- Plan for authentication/API keys when needed
### Testing (Current Coverage: 78.95%, 95 tests)
- ✅ Integration tests for API endpoints
- ✅ Streaming functionality with 20s timeout protection
- ✅ Async test support with pytest-asyncio
- ✅ Validate OpenAI API compatibility
- ✅ Mock responses for all endpoints
- ✅ Main application tests (CORS, exception handlers, lifespan)
- ✅ Chat streaming wrapper tests
- 🚧 Future: Mock Ollama responses when integrated
### Testing Approach
- Write integration tests for API endpoints
- Test streaming functionality with appropriate timeouts
- Use pytest-asyncio for async test support
- Validate OpenAI API compatibility in tests
- Test both mock and real LLM integrations
- Cover main application (CORS, exception handlers, lifespan)
- Test wrapper layers (chat completions, etc.)
- Include tool functionality tests
### Configuration
- ✅ Use `.env` files for local development
- ✅ Document all environment variables in README
- ✅ Provide sensible defaults where possible
- ✅ BaseSettings from pydantic-settings
- 🚧 Support container-based configuration (future)
### Configuration Management
- Use `.env` files for local development
- Document all environment variables in README
- Provide sensible defaults where possible
- Use BaseSettings from pydantic-settings
- Support both local and container-based configuration
## Common Patterns
### Streaming Response Pattern (✅ Implemented)
### Streaming Response Pattern
See `src/chat/router.py` for the current implementation:
Example from `src/chat/router.py`:
```python
from sse_starlette.sse import EventSourceResponse
@@ -468,9 +452,9 @@ async def stream():
return EventSourceResponse(event_generator())
```
### PydanticAI Agent Pattern (🚧 Future Reference)
### PydanticAI Agent Pattern
For future integration when connecting to Ollama:
When implementing agents with PydanticAI and Ollama:
```python
from pydantic_ai import Agent
@@ -484,9 +468,9 @@ agent = Agent(
result = await agent.run('Your prompt')
```
### OpenAI-Compatible Response Format (✅ Implemented)
### OpenAI-Compatible Response Format
Current implementation in `src/chat/schemas.py`:
Example schema from `src/chat/schemas.py`:
```python
{
@@ -502,12 +486,59 @@ Current implementation in `src/chat/schemas.py`:
}
```
### PydanticAI Tool Registration Pattern
Tools are registered with PydanticAI agents using decorators. See `src/agents/tatlock.py` for examples:
```python
from pydantic_ai import Agent, RunContext
# After creating the agent
@agent.tool
def tool_name(ctx: RunContext[None], param: str) -> str:
"""
Tool description that the LLM sees.
Args:
param: Parameter description
Returns:
Result description
"""
return result
```
**Tool Implementation Guidelines**:
- Keep tools in `src/agents/tools.py` for reusability
- Use clear, descriptive docstrings (LLM reads these)
- Include parameter descriptions in docstrings
- Handle errors gracefully and return error messages as strings
- For async operations, declare the tool function as `async def`
- Test tools independently before integration
**Example Tool Module** (`src/agents/tools.py`):
```python
def calculate(expression: str) -> str:
"""Safe calculator implementation."""
try:
# Implementation
return str(result)
except Exception as e:
return f"Error: {str(e)}"
async def search_web(query: str) -> str:
"""Web search via SearXNG."""
async with httpx.AsyncClient() as client:
# Implementation
return formatted_results
```
## Update Policy
This document should be updated when:
- New development patterns are established
- Package versions are upgraded
- New major features are added
- Breaking API changes occur
- Security vulnerabilities are discovered
- Major architectural changes occur
- New best practices are identified
Last updated: 2025-12-06
Last updated: 2025-12-06 (Tools integration)