tatlock/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

[1.3.2] - 2025-12-14

Fixed

• Memory: Fix biographer tool type hints for Ollama compatibility (remove | None union types)

[1.3.1] - 2025-12-14

Fixed

• Memory: Add biographer to delegation wrappers (was returning raw tools causing Ollama error)
• Config: Add Qdrant host/port to .env.example

[1.3.0] - 2025-12-14

Fixed

• Memory: Update Qdrant client to use query_points API (qdrant-client >= 1.10)

Changed

• Config: Rename REDIS_DB to REDIS_BENCHMARK_DB for clarity
• Config: Update Redis defaults to match stack allocation (benchmark=6, memory=1)

[1.2.5] - 2025-12-14

Fixed

• Dependencies: Add missing pydantic-settings (not included in pydantic-ai-slim)

[1.2.4] - 2025-12-14

Added

• CI: Trigger Watchtower update after successful image push

[1.2.3] - 2025-12-14

Fixed

• CI: Upgrade to build-push-action@v6, disable provenance and sbom for Gitea registry

[1.2.2] - 2025-12-13

Fixed

• CI: Add provenance: false to docker/build-push-action to fix Gitea registry push

[1.2.1] - 2025-12-13

Changed

• Dependency slimming: Switched from pydantic-ai to pydantic-ai-slim[openai]
  • Removes unused LLM provider SDKs (anthropic, boto3, cohere, google-genai, groq, huggingface)
  • Production packages: 53 (down from ~158)
  • Production footprint: 178MB
  • Tatlock uses Ollama via OpenAI-compatible API, so only openai extra is needed
  • See DEPENDENCY_SLIM.md for rollback instructions

1.2.0 - 2025-12-13

Added

Phase F: Memory System (The Biographer)

• Memory Infrastructure (Phase F.1):

  • src/core/context.py: ContextVar-based request context for async-safe user/conversation tracking
    • get_user(), get_conversation_id() helpers
    • RequestContext manager for clean setup/teardown
  • src/core/multi_tenancy.py: User ID sanitization and collection naming
    • Per-user collection pattern: memories_{user}
    • Redis key patterns: session:{user}:{conv}, entities:{user}:{conv}
  • src/core/embeddings.py: Ollama embedding client
    • nomic-embed-text model (768 dimensions)
    • embed(), embed_batch(), health_check() methods
  • src/core/qdrant.py: Qdrant vector database client
    • ensure_collection(), upsert_memory(), search_memories(), delete_memory()
    • Type-based filtering for memory queries
  • src/core/memory_cache.py: Redis session memory cache
    • Session context with 24h TTL (db=2, separate from benchmarks)
    • Recent entities tracking per conversation

• Memory Service (Phase F.2a):

  • src/core/memory_service.py: Direct access layer for fast, LLM-free memory lookups
    • Profile methods: get_profile(), set_profile()
    • Preference methods: get_preference(), set_preference(), get_all_preferences()
    • Fact methods: store_fact(), get_fact()
    • Session context: get_session_context(), set_session_context(), update_session_context()
    • Steward integration: prefetch_context() for request preprocessing

• The Biographer Agent (Phase F.2b):

  • src/agents/biographer/: Household memory keeper agent
    • PydanticAI agent with discreet chronicler personality
    • System prompt emphasizes privacy and accurate recall
  • Biographer Tools (src/agents/biographer/tools.py):
    • recall_semantic: Semantic search for memories by meaning
    • list_memories: Browse stored memories by type
    • store_insight: Record new facts from conversation
    • update_profile: Update core profile fields (name, location, timezone)
    • update_preference: Update user preferences (units, theme)
    • forget_memory: Remove specific memories
  • Capability Registration:
    • BIOGRAPHER_CAPABILITY with context domain
    • Automatic registration on startup
    • Low cost (vector search, minimal LLM)

• Delegation Wrapper:

  • delegate_to_biographer() in src/agents/delegation.py
  • Async delegation with error handling

• Steward Memory Integration:

  • Memory context pre-fetch during request analysis
  • Profile and preferences included in Steward's note to Butler
  • Keyword-based context determination (weather → location, time → timezone)

• Configuration:

  • QDRANT_HOST, QDRANT_PORT, QDRANT_EMBEDDING_DIM (768)
  • OLLAMA_EMBEDDING_MODEL (nomic-embed-text)
  • REDIS_MEMORY_DB (2), REDIS_MEMORY_TTL_HOURS (24)

• Test Suite:

  • 34 new tests for memory system
  • Biographer capability tests (15 tests)
  • Memory service tests (19 tests)

• OpenAI Standard user Field:

  • Added user field to ResponseRequest schema
  • Request context set at API entry point
  • Propagates through async calls via ContextVar

Changed

• Application startup now registers The Biographer with Household Registry
• Steward analysis includes memory context pre-fetch
• Librarian client methods now use get_user() from context (12 methods updated)
• Request router sets user/conversation context at entry

1.1.0 - 2025-12-11

Added

Phase 3: Butler Orchestration (Multi-Agent Coordination)

• The Librarian Agent: Expert agent for research and knowledge management

  • PydanticAI agent with specialized research assistant personality
  • Connects to library-desk API for HybridRAG capabilities
  • System prompt emphasizes fetching wiki pages before summarizing
  • Streaming support via run_librarian_stream()

• Library-Desk API Client (src/agents/librarian/client.py):

  • Async HTTP client with httpx for library-desk API integration
  • HybridRAG search (vector + graph + web search)
  • Wiki operations (search, get, list, create, update pages)
  • Smart page creation with HybridRAG research (POST /wiki/pages/smart-create)
  • Semantic vector search
  • Knowledge graph queries (Cypher execution)
  • Dossier (tag collection) browsing
  • Health check endpoint

• Librarian Tools (src/agents/librarian/tools.py):

  • Research tools:
    • hybrid_search: Combined vector, graph, and web search
    • search_wiki: Full-text wiki page search
    • get_wiki_page: Fetch full wiki page content by ID
    • semantic_search: Vector similarity search
    • list_dossiers: Browse knowledge collections
    • get_dossier_pages: Get pages in a dossier
    • explore_knowledge_graph: Entity and relationship discovery
    • find_related_entities: Find connected concepts
  • Write tools:
    • smart_create_wiki_page: Create page with automatic HybridRAG research (PREFERRED for topic-based creation)
    • create_wiki_page: Create page with user-provided content
    • update_wiki_page: Update existing page (partial updates supported)

• Agent Communication Protocol (src/agents/protocol.py):

  • AgentRequest: Standardized task request with context and constraints
  • AgentResponse: Response with result, reasoning, tool calls, confidence
  • DelegationIntent: Routing intent with target agent and reason
  • CoordinationResult: Aggregated multi-agent results
  • DelegationReason enum: domain expertise, tool access, resource efficiency, user preference
  • Error types: AgentError, AgentTimeoutError, AgentUnavailableError

• Coordination Engine (src/agents/coordination.py):

  • CoordinationEngine: Multi-agent task orchestration
  • Routing tasks to appropriate expert agents
  • Sequential and parallel execution support
  • Result aggregation from multiple agents
  • Graceful error handling and degradation
  • Streaming delegation support
  • Convenience functions: delegate_to_librarian(), delegate_to_librarian_stream()

• Librarian Capability Registration:

  • LIBRARIAN_CAPABILITY definition with research domains
  • Automatic registration on application startup
  • Integration with Household Registry

• Configuration:

  • LIBRARY_DESK_HOST: Library-desk API URL (default: http://localhost:8089)
  • LIBRARY_DESK_API_KEY: Optional API key for authentication
  • LIBRARY_DESK_TIMEOUT: Request timeout in seconds (default: 60)

• Test Suite:

  • 78 new tests for Phase 3 components
  • Protocol model tests (requests, responses, intents, errors)
  • Coordination engine tests (delegation, streaming, multi-agent)
  • Library-desk client tests (all endpoints with mocked HTTP)
  • Wiki write operation tests (update, smart-create)
  • Capability registration tests

Changed

• Application startup now registers The Librarian with Household Registry
• Configuration expanded to support library-desk API integration
• Version loading: APP_VERSION now dynamically loaded from pyproject.toml

1.0.0a - 2025-12-11

Added

• CI/CD Pipeline: Release-triggered automated builds
  • Dockerfile for containerized deployment (Python 3.12-slim, port 8000)
  • Gitea Actions workflow triggered on release publish
  • Builds and pushes to git.schweitz.net registry with latest and version tags
  • Watchtower integration for automatic container updates
• Portainer Stack: Production deployment configuration
  • Connects to docker-dataplane network for service discovery
  • Integration with ollama, searxng, and redis-shared services
  • Health check endpoint monitoring
  • Resource limits (1 CPU, 1GB memory)

Changed

• Version bump to 1.0.0 marking production-ready release

0.2.5 - 2025-12-07

Added

Phase 2: The Steward (Two-Tier Architecture)

• The Steward Agent: First-tier LLM agent for request analysis and capability recommendation

  • Analyzes requests with full conversation context awareness
  • Recommends relevant household capabilities for each request
  • Detects missing capabilities and provides guidance
  • Estimates request complexity (simple/moderate/complex)
  • Uses same Ollama model as Tatlock for VRAM efficiency

• Household Registry: Centralized capability management system

  • HouseholdRegistry for registering capabilities and toolsets
  • HouseholdCapability executive summaries for coordination
  • HouseholdMember specifications with PydanticAI toolsets
  • Domain-based tool organization (e.g., src/agents/tatlock_core/)
  • Dynamic tool scoping per request

• Request Preprocessing Pipeline: Steward → Tatlock flow integration

  • preprocess_request() orchestrates Steward analysis
  • Creates scoped toolsets based on recommendations
  • Formats Steward notes for Butler (conversation context included)
  • Integrated with Responses API via create_response_with_steward()

• Tool Usage Tracking: Benchmarking and accuracy analysis

  • ToolCallTracker for monitoring recommended vs. actual tool usage
  • Tracks recommendation accuracy metrics
  • Records benchmarks to Redis for cross-session analysis
  • Supports precision/recall/F1 score calculation

• Streaming Transparency: Real-time Steward analysis visibility

  • Streams Steward's reasoning as reasoning summary deltas
  • Streams Tatlock's response as output text deltas
  • Full SSE support for Steward + Tatlock flow
  • Conversation context and missing capabilities visible in stream

• Structured Logging: Operation timing and metadata tracking

  • structlog-based JSON logging for machine parsing
  • Context managers for automatic operation timing
  • Metadata enrichment for debugging and analysis
  • Integrated with benchmark recording

• Redis Benchmark Storage: Performance metrics persistence

  • Cross-session benchmark storage with 30-day expiry
  • Time-series metrics for Steward analysis and tool calls
  • Queryable by operation, time range, and metadata
  • Support for recommendation accuracy tracking

• Benchmark Analysis Tools: Performance analysis CLI

  • scripts/benchmark_analysis.py for metric analysis
  • Steward performance statistics (latency, success rate, recommendations)
  • Tool recommendation accuracy analysis (precision, recall, F1)
  • Per-tool accuracy breakdown and duration statistics

• End-to-End Test Suite: Comprehensive API integration tests

  • 17 E2E tests making real HTTP requests to running server
  • Tests for Chat Completions, Responses API, and streaming endpoints
  • OpenAI API spec compliance verification (format validation)
  • Steward preprocessing integration verification
  • Error handling tests (404, 422 status codes)
  • Flexible assertions for LLM output variance
  • Tool usage indicators: 🧮 (calculator), 🔍 (search), 🕐 (datetime)
  • Full documentation in tests/e2e/README.md

Phase 1 Enhancements

• Conversation history support: Tatlock now remembers previous turns in multi-turn conversations
  • OpenAI-format messages converted to PydanticAI ModelRequest/ModelResponse objects
  • Full conversation context passed to agent via message_history parameter
  • Empty messages filtered to prevent Ollama errors
• Tool call logging to reasoning output: Users can see what tools are doing in real-time
  • ToolCallTracker dependency system for per-request tool usage logging
  • Web search queries appear with 🔍 emoji (e.g., "🔍 Searching for: 'Python 3.13'")
  • Calculator expressions appear with 🧮 emoji (e.g., "🧮 Calculating: sqrt(144) + 25")
  • Date/time operations appear with 🕐 emoji (e.g., "🕐 Calculating date offset: 2 weeks ago")
  • Tool usage visible in <think> tags in Open WebUI

Changed

• Architecture: Two-tier request flow (Steward analysis → Tatlock execution)
• Tool Organization: Tatlock core tools reorganized into domain directory
• Tool Scoping: Tatlock runs with dynamically scoped toolsets per request
• Responses API: Integrated Steward preprocessing for all Tatlock requests
• Streaming: Enhanced to include Steward reasoning transparency
• Enhanced Tatlock agent with conversation memory capabilities
• All tools now log their usage via RunContext dependencies
• Improved debug logging for message history construction

Fixed

• Streaming text repetition: Fixed text accumulation bug causing repetitive output in Open WebUI
  • Changed from accumulated text to delta mode (stream_text(delta=True))
  • Implemented proper run_with_scoped_tools_stream() using PydanticAI's run_stream()
  • Replaced artificial word-by-word chunking with real LLM deltas
• Broken tool execution in streaming: Tools now execute properly in streaming mode
  • Previously showed raw JSON function calls instead of executed results
  • Now properly streams tool execution results
• Invalid schema parameter: Removed invalid thinking parameter from ReasoningOutputItem
• Case sensitivity in model routing: Model comparison now case-insensitive (.lower())
• Conversation context now properly maintained across multiple turns
• Tool usage transparency - users can see exactly what queries/calculations are being performed
• Schema object handling in usage calculation (_calculate_usage reordered isinstance checks)

0.2.0 - 2025-12-06

Added

PydanticAI Integration (Phase 1)

• Real Tatlock agent using PydanticAI with Ollama backend (mistral-nemo:latest)
• British butler personality with research-oriented mindset
• Lazy agent initialization to avoid connection issues in tests
• Streaming response integration with reasoning output
• Error handling for PydanticAI-specific exceptions

Permanent Tools (Phase 1)

• Calculator tool (src/agents/tools.py):
  • Safe mathematical expression evaluation using restricted namespace
  • Support for arithmetic, algebra, trigonometry, logarithms
  • Math functions: sqrt, sin, cos, tan, log, exp, etc.
  • Constants: pi, e
  • Integer result formatting (removes unnecessary decimals)
• Date/Time toolkit:
  • get_current_datetime: Current date/time in multiple formats
  • calculate_time_offset: Relative date calculations ("1 week ago", "2 months from now")
  • time_difference: Human-readable time differences between dates
• Web Search tool:
  • SearXNG integration for privacy-preserving web search
  • Automatic fallback from production to localhost in development
  • Formatted search results with titles, URLs, and snippets
  • Configurable result limits (max 10)

Tool Framework

• PydanticAI tool registration with @agent.tool decorator
• Tool descriptions visible to LLM for intelligent usage
• Async tool support for I/O operations
• Error handling with string-based error messages
• Tool usage guidelines in system prompt

Configuration

• SearXNG configuration in src/core/config.py:
  • SEARXNG_HOST with development fallback
  • SEARXNG_TIMEOUT setting
• Updated .env.example with SearXNG configuration
• Ollama configuration documentation

Testing

• 26 new tool tests (tests/agents/test_tools.py):
  • 7 calculator tests (arithmetic, functions, error handling)
  • 14 date/time tests (current time, offsets, differences)
  • 5 web search tests (mocked HTTP client)
• Updated registry tests for tools capability
• Total: 131 tests, 81.78% coverage (up from 95 tests, 78.95%)

Documentation

• Comprehensive README.md updates:
  • Tatlock agent capabilities and tool descriptions
  • Requirements section with Ollama and SearXNG setup
  • Configuration examples for external services
  • Tool usage examples and philosophy
  • Troubleshooting for Ollama and SearXNG
  • Updated test statistics
• AGENTS.md refactored for LLM development:
  • PydanticAI tool registration pattern
  • Tool implementation guidelines
  • Removed project status, focused on development instructions
• IMPLEMENTATION_ROADMAP.md updates:
  • Phase 1 marked as "MOSTLY COMPLETE"
  • Detailed completion status for each deliverable
  • Updated current state summary

Changed

• Tatlock agent converted from mock to real PydanticAI implementation
• Tatlock capabilities updated: tools: True
• Streaming coordination now handles chunk-based delivery (50 chars) to preserve markdown
• Chat service streaming updated to preserve formatting
• System prompt enhanced with tool usage guidelines and research mindset
• Agent initialization changed to lazy pattern for better testability

Fixed

• Text duplication bug in streaming responses (proper delta calculation)
• Markdown formatting preservation in streamed responses
• GeneratorExit errors from async context managers in generators
• PydanticAI API usage (result.output instead of result.data)

0.1.1 - 2025-12-06

Added

Agent Interface (Phase 1)

• Abstract AgentInterface base class for model abstraction
• LoremTesterAgent: Full-featured mock agent with realistic behavior
  • Configurable reasoning effort levels (none, minimal, low, medium, high, xhigh)
  • Random tool/function call generation for testing
  • Error triggers: rate_limit, context_overflow, invalid_tool
  • Temperature-based response variation
• TatlockAgent: Placeholder for future PydanticAI integration
• ModelRegistry: Centralized model management and discovery
• 18 agent tests with comprehensive coverage

Responses API (Phases 2, 3, 6)

• OpenAI Responses API format with structured output (/v1/responses)
  • Reasoning items (thinking summaries with configurable effort)
  • Function call items (tool execution simulation)
  • Message items (assistant responses with output_text)
  • Streaming and non-streaming modes
• Real-time streaming with SSE-Starlette
• Conversation history management (Phase 3):
  • Hybrid client/server approach (client maintains state, server tracks)
  • Auto-generated deterministic conversation IDs from message hash
  • Configurable max turns with automatic trimming (default: 20)
  • Context window management with approximate token counting
  • Token usage statistics
  • Placeholder for future vector memory (Qdrant)
• Advanced features (Phase 6):
  • Parameter validation with Pydantic field validators
  • Temperature: 0.0-2.0 range enforcement
  • Reasoning effort: 6 levels validation
  • Max output tokens: positive integer enforcement
  • Stop sequences: up to 4, non-empty strings
  • Real-time stop sequence detection during streaming
  • Real-time max tokens enforcement with token counting
• 45 Responses API tests (router, error handling, history, advanced features)

Chat Completions Wrapper (Phase 5)

• OpenAI Chat Completions compatibility layer (/v1/chat/completions)
• Single source of truth architecture (wraps Responses API)
• Automatic reasoning generation
• Converts reasoning items to <think> tags for Open WebUI
• Pipeline prefix preservation
• System message support
• Enhanced error types (RateLimitError, ContextLengthError)
• 12 Chat Completions tests (router + streaming wrapper)

Application Infrastructure

• FastAPI application factory pattern
• CORS middleware with configurable origins
• Global exception handlers:
  • AppException handler for custom errors
  • RequestValidationError handler for Pydantic validation
  • General exception handler for unexpected errors
• Lifespan management for startup/shutdown
• OpenAPI schema with interactive documentation
• 16 main application tests

Testing Infrastructure

• Comprehensive test suite: 95 tests, 78.95% coverage (up from 62%)
• Async test support with pytest-asyncio
• Test fixtures for sync and async clients
• Integration tests for all API endpoints
• Streaming functionality tests
• Parameter validation tests
• Error handling tests
• Conversation history tests

Documentation

• Complete README.md rewrite with hybrid architecture
• Architecture diagrams and decision documentation
• AGENTS.md with technical implementation details
• API usage examples for all endpoints
• Conversation history guide
• Open WebUI integration instructions
• Troubleshooting section
• Implementation planning documents

Changed

• Hybrid architecture with Responses API as primary endpoint
• Chat Completions now wraps Responses API (no duplicate logic)
• Enhanced error handling with OpenAI-compatible format
• Improved streaming with word-by-word delivery
• Better test organization with domain-based structure

Security

• Minor version locking for all dependencies
• All packages CVE-checked (as of 2025-12-06)
• Environment variable protection via .gitignore
• No known vulnerabilities in dependency tree
• Input validation on all API endpoints

0.1.0 - 2025-12-06

Added

• Project initialization
• Python 3.12.11 environment
• FastAPI 0.123.9 web framework
• PydanticAI 1.27.0 dependency (ready for future integration)
• Mock chat completions (lorem ipsum responses)
• Mock model listing (mistral-nemo:latest)
• Testing infrastructure (pytest, coverage, ruff, mypy)
• Configuration management with pydantic-settings
• CORS middleware
• Exception handlers (OpenAI-compatible error format)