tatlock/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

[2.0.3] - 2026-02-05

Fixed

• Steward analysis leaking into responses - Removed internal routing analysis (DELEGATE: tatlock_core...) from user-visible reasoning in both streaming and non-streaming paths

[2.0.2] - 2026-02-05

Fixed

• tool_choice format incompatibility - Removed extra_body tool_choice hack for Claude backend; PydanticAI handles tool_choice natively for Anthropic, preventing infinite tool call loops
• CI trigger - Changed workflow trigger from release:published to push:tags:v[0-9]*

[2.0.1] - 2026-02-05

Fixed

• Expert agent registration failure - AnthropicModel does not accept api_key directly; now passes it via AnthropicProvider

2.0.0 - 2026-02-05

Added

• Claude backend support (Claudification Phase 1) - All agents now prefer Claude over Ollama

  • New src/anthropic/ module with model selector and health check
  • get_model() factory returns Claude if available, Ollama as fallback
  • Startup health check caches Claude API availability
  • Configuration: ANTHROPIC_API_KEY, ANTHROPIC_MODEL, PREFER_CLOUD_BACKEND
  • 200k token context when using Claude backend

• Steward dual-backend support - Direct API calls to Claude or Ollama

  • _call_claude(): Anthropic Messages API path
  • _call_ollama(): Existing Ollama generate API path (preserved)
  • Automatic fallback: if Claude call fails mid-request, retries with Ollama

• Claudification project tracking - PROJECT_CLAUDIFICATION.md with Phase 1/2 roadmap

Changed

• All PydanticAI agents refactored to use get_model():
  • Tatlock (6 instantiation locations)
  • Librarian
  • Biographer
  • Housekeeper
• initialize_application() is now async - Supports async Claude health check at startup
• Dependencies: pydantic-ai-slim[openai,anthropic] replaces pydantic-ai-slim[openai]
• Startup logging now includes backend selection info (claude/ollama)
• Agent creation logging now includes backend and model info

Removed

• Stale tests/core/test_benchmarks.py (benchmark system was removed in v1.10.0)

1.11.0 - 2025-12-30

Added

• Paperless document integration - HybridRAG now includes indexed PDFs and scanned documents from Paperless-ngx

  • New include_documents parameter in hybrid_search tool
  • 📑 icon for document sources in search results
  • Librarian prompt updated with document awareness

• Volatile cache integration - HybridRAG now includes pre-fetched real-time data

  • New include_volatile parameter in hybrid_search tool
  • ⚡ icon for volatile sources in search results
  • Supports weather, forecast, news, stock, crypto, sun, air_quality namespaces
  • Librarian prompt updated with volatile cache awareness (user-configured items only)

• Biographer routing in Steward - Personal memory queries now correctly route to The Biographer

  • Added explicit routing rules for "where do I live", "what car do I drive", etc.
  • Added biographer delegation examples to Steward prompt
  • Location keywords ("live", "where", "home") now trigger profile pre-fetch

Changed

• LibraryDeskClient.hybrid_search - Now passes full config including document_limit, volatile_limit, and enable flags
• Steward guidelines - Clarified that research queries about TOPICS go to Librarian, queries about USER go to Biographer

[1.10.1] - 2025-12-23

Fixed

• Tatlock's excessive apologizing - Strengthened personality prompt to prevent unnecessary apologies after successful Librarian delegations. Added explicit "do NOT apologize" instructions to both system prompt and synthesis prompt.

1.10.0 - 2025-12-22

Added

Lightweight Request Tracing

• JSON-based tracing system for local development debugging
  • Captures full request flow through multi-agent architecture
  • Trace and Span dataclasses with automatic timing and nesting
  • ContextVar-based propagation for async-safe tracing
  • trace_span async context manager for clean instrumentation
  • Traces written to logs/traces/{trace_id}.json
  • Enabled via DEBUG=true environment variable
• Trace Viewer UI (logs/traces/viewer.html)
  • Standalone HTML viewer with timeline visualization
  • Filter by status, search by request text
  • Expandable span details with prompts and responses
• Tracing REST API (/traces)
  • GET /traces - Serve trace viewer UI
  • GET /traces/list - List available traces with filtering
  • GET /traces/{trace_id} - Retrieve specific trace JSON
  • Only available when DEBUG=true
• Full pipeline instrumentation
  • Router-level trace start/end with context management
  • Steward analysis spans in preprocessing
  • Tatlock orchestrate/synthesize spans
  • Expert delegation spans (librarian/biographer/housekeeper)
  • Tool-level spans extracted from PydanticAI messages

Changed

• Replaced Redis benchmarks with file-based tracing - Simpler, more useful for debugging
• Context management moved to service layer - Router simplified, context set in response service
• Server binds to all interfaces - wakeup.sh now uses 0.0.0.0 for network access

Removed

• Redis benchmark system (src/core/benchmarks.py)
  • ENABLE_BENCHMARKS config setting
  • REDIS_BENCHMARK_DB config setting
  • redis_url property (kept redis_memory_url)
  • Benchmark recording in Steward service and tool tracking

Fixed

• Librarian fabrication prevention - Added explicit instructions to never invent data when tools fail or sources are unavailable

1.9.0 - 2025-12-18

Changed

• Housekeeper prompt optimization - Rewrote system prompt for Mistral-Nemo function calling with negative constraints, step-by-step process, and explicit entity ID format guidance
• Housekeeper temperature setting - Set temperature to 0.1 for deterministic tool calling behavior
• Device list room group priority - Room groups now appear first in list_devices output with [ROOM GROUP] marker to address positional bias
• Tool docstring improvements - Updated turn_on/turn_off/toggle with explicit entity_id= parameter examples

Added

• Housekeeper optimization findings - Added docs/housekeeper-optimization-findings.md documenting the experiment journey from 0% to 100% success rate
• Housekeeper test script - Added scripts/test_housekeeper.sh for room group detection regression testing

1.8.6 - 2025-12-17

Fixed

• Housekeeper API paths - Updated all client endpoints to use /housekeeping/ prefix to match core-api routes
• Housekeeper entity hallucination - Improved system prompt with critical rule requiring list_devices() before any control action to prevent guessing entity IDs

Added

• Housekeeping API spec - Added docs/housekeeping-api-spec.md documenting the core-api home automation interface

1.8.5 - 2025-12-16

Fixed

• Redis benchmark boolean storage - Convert booleans to strings for Redis hset (Redis doesn't accept bool type directly)
• Tool tracking capability matching - delegate_to_librarian now correctly recognized as using "librarian" capability when checking Steward recommendations
• E2E test fixture scope - Fixed pytest-asyncio ScopeMismatch error by using loop_scope="module" for module-scoped async fixtures

1.8.4 - 2025-12-16

Fixed

• Remove <think> wrappers from think messages - Messages in reasoning_content should be plain text
• Removed <think> wrappers from delegation.py household think messages
• Removed <think> wrappers from orchestration.py status messages
• Think messages now appear cleanly in Open WebUI's reasoning block

1.8.3 - 2025-12-16

Fixed

• Open WebUI streaming rendering - Use reasoning_content field for thinking (DeepSeek R1 format) instead of <think> tags in content
• Open WebUI now renders thinking as proper collapsible blocks instead of broken HTML

1.8.2 - 2025-12-16

Fixed

• HybridRAG keywords schema mismatch - library-desk now returns keywords as dict with core_keywords, client now handles both formats

1.8.1 - 2025-12-16

Fixed

Ollama Message Sanitization

• Fixed invalid message content type: <nil> error from Ollama
• Created custom TatlockOllamaProvider that sanitizes messages before sending to Ollama
• Ollama rejects assistant messages with content: null (tool-only messages from PydanticAI)
• Provider converts null content to empty string "" for compatibility
• Updated all agents (Librarian, Biographer, Housekeeper, Tatlock) to use sanitized provider
• Added src/ollama/provider.py with reusable provider pattern

Streaming Think Message Accumulation

• Fixed repeating think messages in frontend (e.g., 10x "The Librarian has compiled...")
• Frontend was accumulating ReasoningSummaryDelta events expecting concatenation
• Added ReasoningSummaryDone() signal after each think message to indicate completion
• Each think slug is now treated as a complete message, not a continuation

1.8.0 - 2025-12-15

Fixed

Steward Routing for Web Search

• Updated Steward guidelines to route web searches, weather, news → Librarian with search_web
• Added URL/article reading → Librarian with read_url to routing guidelines
• Added examples showing search_web and read_url tool usage

Librarian Agent Tool Registration

• Registered search_web, read_url, read_urls_batch tools with the Librarian PydanticAI agent
• Updated Librarian system prompt with Web Search & Content Extraction section
• Fixed tool count in agent logger (11 → 14 tools)

Query Enrichment Integration

• Fixed enriched query (with location/timezone context) not being passed to delegations
• Response service now uses enriched_query from Steward recommendation for all delegations
• Weather queries now automatically include user's stored location

Action Type Detection

• Added "read", "fetch", "url", "http" keywords to RESEARCH action type for Librarian
• Ensures proper think messages for URL reading tasks

1.7.0 - 2025-12-15

Added

Web Search Migration to Librarian

• search_web() tool in Librarian for web search via library-desk /rag/search endpoint
• read_url() tool for single URL content extraction via Trafilatura
• read_urls_batch() tool for parallel batch URL extraction (max 20 URLs)
• WebSearchResult, WebSearchResponse models in LibraryDeskClient
• ContentExtractionResult, BatchExtractionResponse models for content extraction
• search_web(), extract_content(), extract_content_batch() methods in LibraryDeskClient
• Comprehensive unit tests for new Librarian tools (tests/agents/librarian/test_tools.py)

Changed

• Librarian capability updated with web search domains: "web", "url", "internet"
• Tatlock system prompt now delegates web search to Librarian
• tatlock_core capability reduced to computation/datetime only (no longer requires network)

Removed

• search_web function from src/agents/tatlock_core/tools.py
• web_search_tool from tatlock_core_tools list
• search_web from legacy src/agents/tools.py
• Search tests from tests/agents/test_tools.py (moved to Librarian tests)

1.6.0 - 2025-12-15

Added

Two-Phase Tatlock Execution

• Phase 1: Orchestration - Executes tool calls and expert delegations, returns structured results
• Phase 2: Synthesis - Synthesizes butler-toned response from gathered results
• orchestrate_tool_calls() method in TatlockAgent for coordination phase
• synthesize_from_results() method in TatlockAgent for synthesis phase
• Guarantees butler personality in all responses by separating coordination from response generation

Automatic Think Slugs

• Deterministic butler-perspective messages during expert delegation (no LLM involved)
• ActionType enum: RETRIEVE, RESEARCH, CREATE, CONTROL, RECORD
• HOUSEHOLD_THINK_MESSAGES mapping with butler-perspective messages for all experts:
  • Librarian: "Allow me to consult the archives, sir." / "I'm having the Librarian prepare a new entry."
  • Biographer: "Let me consult the household records." / "I've asked the Biographer to take note, sir."
  • Housekeeper: "I'm instructing the household staff now, sir." / "Allow me to inquire with the household staff."
• _detect_action_type() function for keyword-based action detection
• get_think_message() helper for retrieving appropriate messages
• Streaming delegation wrappers: stream_delegate_to_librarian(), stream_delegate_to_biographer(), stream_delegate_to_housekeeper()
• STREAMING_DELEGATION_WRAPPERS mapping in delegation.py
• get_streaming_delegation_tools() method in HouseholdRegistry

Steward Query Enrichment

• Auto-fill user context (location, timezone) when not specified in query
• _build_enriched_query() function in steward service
• Regex word boundary matching for accurate location detection (avoids false positives)
• enriched_query field added to StewardRecommendation schema
• Automatic enrichment for weather queries (location), time queries (timezone), temperature preferences

Documentation

• ORCHESTRATION_SCENARIOS.md completely rewritten with:
  • Mermaid flow diagrams for two-phase execution
  • 4 new Housekeeper scenarios (light control, device status, parallel delegation)
  • Biographer memory recording scenario
  • Complete think slug reference tables
  • Action type detection tables
  • Updated architecture mindmap
• TESTING_IMPROVEMENTS.md - LLM testing best practices for future implementation

Changed

• create_response_with_steward() now uses two-phase execution
• _direct_delegation() routes through synthesis phase for consistent butler tone
• _execute_single_delegation() now supports housekeeper
• Streaming response handler integrated with think slug system
• All 326 unit tests passing

1.5.0 - 2025-12-15

Added

The Housekeeper Agent

• New home automation expert agent following the Librarian pattern
• CoreAPIClient for communicating with core-api service (Home Assistant wrapper)
• 13 tools for home automation:
  • Discovery: list_areas, list_devices, get_device_state
  • Control: turn_on, turn_off, toggle
  • Scenes: list_scenes, activate_scene
  • Scripts: list_scripts, run_script
  • Automations: list_automations, toggle_automation
  • History: get_history
• PydanticAI agent with system prompt for home automation tasks
• HouseholdCapability registration with domains: lights, switches, automation, home, smart home, scene, script, device, climate, fan, cover, blinds
• delegate_to_housekeeper() delegation wrapper
• Config settings: CORE_API_HOST, CORE_API_KEY, CORE_API_TIMEOUT

Development Port Change

• Dev server port changed from 8123 to 8777 to avoid conflict with Home Assistant default port
• Updated wakeup.sh, E2E tests, and documentation

Changed

• All unit tests pass (421 passed, 5 xfailed)
• Housekeeper registered on startup alongside Librarian and Biographer

1.4.0 - 2025-12-14

Added

Environment-Aware Configuration

• Auto-selected logging level: DEBUG for development, WARNING for production
• Auto-selected default user: llm_tester for development (isolated test scope), jpmschweitzer for production
• Properties effective_log_level and effective_default_user in config
• User context logging at request entry with INFO level

Direct Delegation Bypass

• Pure memory/librarian requests bypass Tatlock: When Steward recommends only biographer/librarian, skip Tatlock LLM call
• _direct_delegation() function for immediate expert agent execution
• Reduces latency for memory-only requests

Text-Based Delegation Fallback

• Parse text delegation patterns: Handle LLM outputs like [DELEGATE:biographer] task="..."
• Multiple pattern support for delegation parsing
• Sequential and parallel execution with [PARALLEL] prefix

Comprehensive E2E Test Suite

• 22 new orchestration tests in tests/e2e/test_orchestration_e2e.py
• QdrantVerifier helper class for data verification
• assert_llm_behavior() for flexible LLM output pattern matching
• Test classes covering:
  • Memory storage and recall
  • Steward delegation
  • Direct delegation bypass
  • User context isolation (llm_tester vs production)
  • Data verification in Qdrant
  • Integration health checks
  • Orchestration scenarios (weather, calculator, wiki, multi-expert)
  • Error handling
  • Evaluation reports
• Updated tests/e2e/README.md with comprehensive documentation

Fixed

• Unit test mocks: Updated Steward streaming tests to mock run_with_scoped_tools_stream (async generator)
• Temporal context in tests: Tests now account for _inject_temporal_context() appending timestamps
• LLM non-determinism: Integration tests use pytest.xfail() for LLM-dependent assertions
• Streaming test timeouts: Increased timeouts (60-90s) for LLM processing time

Changed

• All unit tests now pass (380 passed, 5 xfailed for LLM non-determinism)
• E2E tests use llm_tester user for isolation from production data

1.3.3 - 2025-12-14

Fixed

• Memory: Fix Qdrant point IDs - use UUID5 instead of arbitrary strings

1.3.2 - 2025-12-14

Fixed

• Memory: Fix biographer tool type hints for Ollama compatibility (remove | None union types)

1.3.1 - 2025-12-14

Fixed

• Memory: Add biographer to delegation wrappers (was returning raw tools causing Ollama error)
• Config: Add Qdrant host/port to .env.example

1.3.0 - 2025-12-14

Fixed

• Memory: Update Qdrant client to use query_points API (qdrant-client >= 1.10)

Changed

• Config: Rename REDIS_DB to REDIS_BENCHMARK_DB for clarity
• Config: Update Redis defaults to match stack allocation (benchmark=6, memory=1)

1.2.5 - 2025-12-14

Fixed

• Dependencies: Add missing pydantic-settings (not included in pydantic-ai-slim)

1.2.4 - 2025-12-14

Added

• CI: Trigger Watchtower update after successful image push

1.2.3 - 2025-12-14

Fixed

• CI: Upgrade to build-push-action@v6, disable provenance and sbom for Gitea registry

1.2.2 - 2025-12-13

Fixed

• CI: Add provenance: false to docker/build-push-action to fix Gitea registry push

1.2.1 - 2025-12-13

Changed

• Dependency slimming: Switched from pydantic-ai to pydantic-ai-slim[openai]
  • Removes unused LLM provider SDKs (anthropic, boto3, cohere, google-genai, groq, huggingface)
  • Production packages: 53 (down from ~158)
  • Production footprint: 178MB
  • Tatlock uses Ollama via OpenAI-compatible API, so only openai extra is needed
  • See DEPENDENCY_SLIM.md for rollback instructions

1.2.0 - 2025-12-13

Added

Phase F: Memory System (The Biographer)

• Memory Infrastructure (Phase F.1):

  • src/core/context.py: ContextVar-based request context for async-safe user/conversation tracking
    • get_user(), get_conversation_id() helpers
    • RequestContext manager for clean setup/teardown
  • src/core/multi_tenancy.py: User ID sanitization and collection naming
    • Per-user collection pattern: memories_{user}
    • Redis key patterns: session:{user}:{conv}, entities:{user}:{conv}
  • src/core/embeddings.py: Ollama embedding client
    • nomic-embed-text model (768 dimensions)
    • embed(), embed_batch(), health_check() methods
  • src/core/qdrant.py: Qdrant vector database client
    • ensure_collection(), upsert_memory(), search_memories(), delete_memory()
    • Type-based filtering for memory queries
  • src/core/memory_cache.py: Redis session memory cache
    • Session context with 24h TTL (db=2, separate from benchmarks)
    • Recent entities tracking per conversation

• Memory Service (Phase F.2a):

  • src/core/memory_service.py: Direct access layer for fast, LLM-free memory lookups
    • Profile methods: get_profile(), set_profile()
    • Preference methods: get_preference(), set_preference(), get_all_preferences()
    • Fact methods: store_fact(), get_fact()
    • Session context: get_session_context(), set_session_context(), update_session_context()
    • Steward integration: prefetch_context() for request preprocessing

• The Biographer Agent (Phase F.2b):

  • src/agents/biographer/: Household memory keeper agent
    • PydanticAI agent with discreet chronicler personality
    • System prompt emphasizes privacy and accurate recall
  • Biographer Tools (src/agents/biographer/tools.py):
    • recall_semantic: Semantic search for memories by meaning
    • list_memories: Browse stored memories by type
    • store_insight: Record new facts from conversation
    • update_profile: Update core profile fields (name, location, timezone)
    • update_preference: Update user preferences (units, theme)
    • forget_memory: Remove specific memories
  • Capability Registration:
    • BIOGRAPHER_CAPABILITY with context domain
    • Automatic registration on startup
    • Low cost (vector search, minimal LLM)

• Delegation Wrapper:

  • delegate_to_biographer() in src/agents/delegation.py
  • Async delegation with error handling

• Steward Memory Integration:

  • Memory context pre-fetch during request analysis
  • Profile and preferences included in Steward's note to Butler
  • Keyword-based context determination (weather → location, time → timezone)

• Configuration:

  • QDRANT_HOST, QDRANT_PORT, QDRANT_EMBEDDING_DIM (768)
  • OLLAMA_EMBEDDING_MODEL (nomic-embed-text)
  • REDIS_MEMORY_DB (2), REDIS_MEMORY_TTL_HOURS (24)

• Test Suite:

  • 34 new tests for memory system
  • Biographer capability tests (15 tests)
  • Memory service tests (19 tests)

• OpenAI Standard user Field:

  • Added user field to ResponseRequest schema
  • Request context set at API entry point
  • Propagates through async calls via ContextVar

Changed

• Application startup now registers The Biographer with Household Registry
• Steward analysis includes memory context pre-fetch
• Librarian client methods now use get_user() from context (12 methods updated)
• Request router sets user/conversation context at entry

1.1.0 - 2025-12-11

Added

Phase 3: Butler Orchestration (Multi-Agent Coordination)

• The Librarian Agent: Expert agent for research and knowledge management

  • PydanticAI agent with specialized research assistant personality
  • Connects to library-desk API for HybridRAG capabilities
  • System prompt emphasizes fetching wiki pages before summarizing
  • Streaming support via run_librarian_stream()

• Library-Desk API Client (src/agents/librarian/client.py):

  • Async HTTP client with httpx for library-desk API integration
  • HybridRAG search (vector + graph + web search)
  • Wiki operations (search, get, list, create, update pages)
  • Smart page creation with HybridRAG research (POST /wiki/pages/smart-create)
  • Semantic vector search
  • Knowledge graph queries (Cypher execution)
  • Dossier (tag collection) browsing
  • Health check endpoint

• Librarian Tools (src/agents/librarian/tools.py):

  • Research tools:
    • hybrid_search: Combined vector, graph, and web search
    • search_wiki: Full-text wiki page search
    • get_wiki_page: Fetch full wiki page content by ID
    • semantic_search: Vector similarity search
    • list_dossiers: Browse knowledge collections
    • get_dossier_pages: Get pages in a dossier
    • explore_knowledge_graph: Entity and relationship discovery
    • find_related_entities: Find connected concepts
  • Write tools:
    • smart_create_wiki_page: Create page with automatic HybridRAG research (PREFERRED for topic-based creation)
    • create_wiki_page: Create page with user-provided content
    • update_wiki_page: Update existing page (partial updates supported)

• Agent Communication Protocol (src/agents/protocol.py):

  • AgentRequest: Standardized task request with context and constraints
  • AgentResponse: Response with result, reasoning, tool calls, confidence
  • DelegationIntent: Routing intent with target agent and reason
  • CoordinationResult: Aggregated multi-agent results
  • DelegationReason enum: domain expertise, tool access, resource efficiency, user preference
  • Error types: AgentError, AgentTimeoutError, AgentUnavailableError

• Coordination Engine (src/agents/coordination.py):

  • CoordinationEngine: Multi-agent task orchestration
  • Routing tasks to appropriate expert agents
  • Sequential and parallel execution support
  • Result aggregation from multiple agents
  • Graceful error handling and degradation
  • Streaming delegation support
  • Convenience functions: delegate_to_librarian(), delegate_to_librarian_stream()

• Librarian Capability Registration:

  • LIBRARIAN_CAPABILITY definition with research domains
  • Automatic registration on application startup
  • Integration with Household Registry

• Configuration:

  • LIBRARY_DESK_HOST: Library-desk API URL (default: http://localhost:8089)
  • LIBRARY_DESK_API_KEY: Optional API key for authentication
  • LIBRARY_DESK_TIMEOUT: Request timeout in seconds (default: 60)

• Test Suite:

  • 78 new tests for Phase 3 components
  • Protocol model tests (requests, responses, intents, errors)
  • Coordination engine tests (delegation, streaming, multi-agent)
  • Library-desk client tests (all endpoints with mocked HTTP)
  • Wiki write operation tests (update, smart-create)
  • Capability registration tests

Changed

• Application startup now registers The Librarian with Household Registry
• Configuration expanded to support library-desk API integration
• Version loading: APP_VERSION now dynamically loaded from pyproject.toml

1.0.0a - 2025-12-11

Added

• CI/CD Pipeline: Release-triggered automated builds
  • Dockerfile for containerized deployment (Python 3.12-slim, port 8000)
  • Gitea Actions workflow triggered on release publish
  • Builds and pushes to git.schweitz.net registry with latest and version tags
  • Watchtower integration for automatic container updates
• Portainer Stack: Production deployment configuration
  • Connects to docker-dataplane network for service discovery
  • Integration with ollama, searxng, and redis-shared services
  • Health check endpoint monitoring
  • Resource limits (1 CPU, 1GB memory)

Changed

• Version bump to 1.0.0 marking production-ready release

0.2.5 - 2025-12-07

Added

Phase 2: The Steward (Two-Tier Architecture)

• The Steward Agent: First-tier LLM agent for request analysis and capability recommendation

  • Analyzes requests with full conversation context awareness
  • Recommends relevant household capabilities for each request
  • Detects missing capabilities and provides guidance
  • Estimates request complexity (simple/moderate/complex)
  • Uses same Ollama model as Tatlock for VRAM efficiency

• Household Registry: Centralized capability management system

  • HouseholdRegistry for registering capabilities and toolsets
  • HouseholdCapability executive summaries for coordination
  • HouseholdMember specifications with PydanticAI toolsets
  • Domain-based tool organization (e.g., src/agents/tatlock_core/)
  • Dynamic tool scoping per request

• Request Preprocessing Pipeline: Steward → Tatlock flow integration

  • preprocess_request() orchestrates Steward analysis
  • Creates scoped toolsets based on recommendations
  • Formats Steward notes for Butler (conversation context included)
  • Integrated with Responses API via create_response_with_steward()

• Tool Usage Tracking: Benchmarking and accuracy analysis

  • ToolCallTracker for monitoring recommended vs. actual tool usage
  • Tracks recommendation accuracy metrics
  • Records benchmarks to Redis for cross-session analysis
  • Supports precision/recall/F1 score calculation

• Streaming Transparency: Real-time Steward analysis visibility

  • Streams Steward's reasoning as reasoning summary deltas
  • Streams Tatlock's response as output text deltas
  • Full SSE support for Steward + Tatlock flow
  • Conversation context and missing capabilities visible in stream

• Structured Logging: Operation timing and metadata tracking

  • structlog-based JSON logging for machine parsing
  • Context managers for automatic operation timing
  • Metadata enrichment for debugging and analysis
  • Integrated with benchmark recording

• Redis Benchmark Storage: Performance metrics persistence

  • Cross-session benchmark storage with 30-day expiry
  • Time-series metrics for Steward analysis and tool calls
  • Queryable by operation, time range, and metadata
  • Support for recommendation accuracy tracking

• Benchmark Analysis Tools: Performance analysis CLI

  • scripts/benchmark_analysis.py for metric analysis
  • Steward performance statistics (latency, success rate, recommendations)
  • Tool recommendation accuracy analysis (precision, recall, F1)
  • Per-tool accuracy breakdown and duration statistics

• End-to-End Test Suite: Comprehensive API integration tests

  • 17 E2E tests making real HTTP requests to running server
  • Tests for Chat Completions, Responses API, and streaming endpoints
  • OpenAI API spec compliance verification (format validation)
  • Steward preprocessing integration verification
  • Error handling tests (404, 422 status codes)
  • Flexible assertions for LLM output variance
  • Tool usage indicators: 🧮 (calculator), 🔍 (search), 🕐 (datetime)
  • Full documentation in tests/e2e/README.md

Phase 1 Enhancements

• Conversation history support: Tatlock now remembers previous turns in multi-turn conversations
  • OpenAI-format messages converted to PydanticAI ModelRequest/ModelResponse objects
  • Full conversation context passed to agent via message_history parameter
  • Empty messages filtered to prevent Ollama errors
• Tool call logging to reasoning output: Users can see what tools are doing in real-time
  • ToolCallTracker dependency system for per-request tool usage logging
  • Web search queries appear with 🔍 emoji (e.g., "🔍 Searching for: 'Python 3.13'")
  • Calculator expressions appear with 🧮 emoji (e.g., "🧮 Calculating: sqrt(144) + 25")
  • Date/time operations appear with 🕐 emoji (e.g., "🕐 Calculating date offset: 2 weeks ago")
  • Tool usage visible in <think> tags in Open WebUI

Changed

• Architecture: Two-tier request flow (Steward analysis → Tatlock execution)
• Tool Organization: Tatlock core tools reorganized into domain directory
• Tool Scoping: Tatlock runs with dynamically scoped toolsets per request
• Responses API: Integrated Steward preprocessing for all Tatlock requests
• Streaming: Enhanced to include Steward reasoning transparency
• Enhanced Tatlock agent with conversation memory capabilities
• All tools now log their usage via RunContext dependencies
• Improved debug logging for message history construction

Fixed

• Streaming text repetition: Fixed text accumulation bug causing repetitive output in Open WebUI
  • Changed from accumulated text to delta mode (stream_text(delta=True))
  • Implemented proper run_with_scoped_tools_stream() using PydanticAI's run_stream()
  • Replaced artificial word-by-word chunking with real LLM deltas
• Broken tool execution in streaming: Tools now execute properly in streaming mode
  • Previously showed raw JSON function calls instead of executed results
  • Now properly streams tool execution results
• Invalid schema parameter: Removed invalid thinking parameter from ReasoningOutputItem
• Case sensitivity in model routing: Model comparison now case-insensitive (.lower())
• Conversation context now properly maintained across multiple turns
• Tool usage transparency - users can see exactly what queries/calculations are being performed
• Schema object handling in usage calculation (_calculate_usage reordered isinstance checks)

0.2.0 - 2025-12-06

Added

PydanticAI Integration (Phase 1)

• Real Tatlock agent using PydanticAI with Ollama backend (mistral-nemo:latest)
• British butler personality with research-oriented mindset
• Lazy agent initialization to avoid connection issues in tests
• Streaming response integration with reasoning output
• Error handling for PydanticAI-specific exceptions

Permanent Tools (Phase 1)

• Calculator tool (src/agents/tools.py):
  • Safe mathematical expression evaluation using restricted namespace
  • Support for arithmetic, algebra, trigonometry, logarithms
  • Math functions: sqrt, sin, cos, tan, log, exp, etc.
  • Constants: pi, e
  • Integer result formatting (removes unnecessary decimals)
• Date/Time toolkit:
  • get_current_datetime: Current date/time in multiple formats
  • calculate_time_offset: Relative date calculations ("1 week ago", "2 months from now")
  • time_difference: Human-readable time differences between dates
• Web Search tool:
  • SearXNG integration for privacy-preserving web search
  • Automatic fallback from production to localhost in development
  • Formatted search results with titles, URLs, and snippets
  • Configurable result limits (max 10)

Tool Framework

• PydanticAI tool registration with @agent.tool decorator
• Tool descriptions visible to LLM for intelligent usage
• Async tool support for I/O operations
• Error handling with string-based error messages
• Tool usage guidelines in system prompt

Configuration

• SearXNG configuration in src/core/config.py:
  • SEARXNG_HOST with development fallback
  • SEARXNG_TIMEOUT setting
• Updated .env.example with SearXNG configuration
• Ollama configuration documentation

Testing

• 26 new tool tests (tests/agents/test_tools.py):
  • 7 calculator tests (arithmetic, functions, error handling)
  • 14 date/time tests (current time, offsets, differences)
  • 5 web search tests (mocked HTTP client)
• Updated registry tests for tools capability
• Total: 131 tests, 81.78% coverage (up from 95 tests, 78.95%)

Documentation

• Comprehensive README.md updates:
  • Tatlock agent capabilities and tool descriptions
  • Requirements section with Ollama and SearXNG setup
  • Configuration examples for external services
  • Tool usage examples and philosophy
  • Troubleshooting for Ollama and SearXNG
  • Updated test statistics
• AGENTS.md refactored for LLM development:
  • PydanticAI tool registration pattern
  • Tool implementation guidelines
  • Removed project status, focused on development instructions
• IMPLEMENTATION_ROADMAP.md updates:
  • Phase 1 marked as "MOSTLY COMPLETE"
  • Detailed completion status for each deliverable
  • Updated current state summary

Changed

• Tatlock agent converted from mock to real PydanticAI implementation
• Tatlock capabilities updated: tools: True
• Streaming coordination now handles chunk-based delivery (50 chars) to preserve markdown
• Chat service streaming updated to preserve formatting
• System prompt enhanced with tool usage guidelines and research mindset
• Agent initialization changed to lazy pattern for better testability

Fixed

• Text duplication bug in streaming responses (proper delta calculation)
• Markdown formatting preservation in streamed responses
• GeneratorExit errors from async context managers in generators
• PydanticAI API usage (result.output instead of result.data)

0.1.1 - 2025-12-06

Added

Agent Interface (Phase 1)

• Abstract AgentInterface base class for model abstraction
• LoremTesterAgent: Full-featured mock agent with realistic behavior
  • Configurable reasoning effort levels (none, minimal, low, medium, high, xhigh)
  • Random tool/function call generation for testing
  • Error triggers: rate_limit, context_overflow, invalid_tool
  • Temperature-based response variation
• TatlockAgent: Placeholder for future PydanticAI integration
• ModelRegistry: Centralized model management and discovery
• 18 agent tests with comprehensive coverage

Responses API (Phases 2, 3, 6)

• OpenAI Responses API format with structured output (/v1/responses)
  • Reasoning items (thinking summaries with configurable effort)
  • Function call items (tool execution simulation)
  • Message items (assistant responses with output_text)
  • Streaming and non-streaming modes
• Real-time streaming with SSE-Starlette
• Conversation history management (Phase 3):
  • Hybrid client/server approach (client maintains state, server tracks)
  • Auto-generated deterministic conversation IDs from message hash
  • Configurable max turns with automatic trimming (default: 20)
  • Context window management with approximate token counting
  • Token usage statistics
  • Placeholder for future vector memory (Qdrant)
• Advanced features (Phase 6):
  • Parameter validation with Pydantic field validators
  • Temperature: 0.0-2.0 range enforcement
  • Reasoning effort: 6 levels validation
  • Max output tokens: positive integer enforcement
  • Stop sequences: up to 4, non-empty strings
  • Real-time stop sequence detection during streaming
  • Real-time max tokens enforcement with token counting
• 45 Responses API tests (router, error handling, history, advanced features)

Chat Completions Wrapper (Phase 5)

• OpenAI Chat Completions compatibility layer (/v1/chat/completions)
• Single source of truth architecture (wraps Responses API)
• Automatic reasoning generation
• Converts reasoning items to <think> tags for Open WebUI
• Pipeline prefix preservation
• System message support
• Enhanced error types (RateLimitError, ContextLengthError)
• 12 Chat Completions tests (router + streaming wrapper)

Application Infrastructure

• FastAPI application factory pattern
• CORS middleware with configurable origins
• Global exception handlers:
  • AppException handler for custom errors
  • RequestValidationError handler for Pydantic validation
  • General exception handler for unexpected errors
• Lifespan management for startup/shutdown
• OpenAPI schema with interactive documentation
• 16 main application tests

Testing Infrastructure

• Comprehensive test suite: 95 tests, 78.95% coverage (up from 62%)
• Async test support with pytest-asyncio
• Test fixtures for sync and async clients
• Integration tests for all API endpoints
• Streaming functionality tests
• Parameter validation tests
• Error handling tests
• Conversation history tests

Documentation

• Complete README.md rewrite with hybrid architecture
• Architecture diagrams and decision documentation
• AGENTS.md with technical implementation details
• API usage examples for all endpoints
• Conversation history guide
• Open WebUI integration instructions
• Troubleshooting section
• Implementation planning documents

Changed

• Hybrid architecture with Responses API as primary endpoint
• Chat Completions now wraps Responses API (no duplicate logic)
• Enhanced error handling with OpenAI-compatible format
• Improved streaming with word-by-word delivery
• Better test organization with domain-based structure

Security

• Minor version locking for all dependencies
• All packages CVE-checked (as of 2025-12-06)
• Environment variable protection via .gitignore
• No known vulnerabilities in dependency tree
• Input validation on all API endpoints

0.1.0 - 2025-12-06

Added

• Project initialization
• Python 3.12.11 environment
• FastAPI 0.123.9 web framework
• PydanticAI 1.27.0 dependency (ready for future integration)
• Mock chat completions (lorem ipsum responses)
• Mock model listing (mistral-nemo:latest)
• Testing infrastructure (pytest, coverage, ruff, mypy)
• Configuration management with pydantic-settings
• CORS middleware
• Exception handlers (OpenAI-compatible error format)