Commit Graph
23 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 4.5 100ebeae52 feat: migrate web search from tatlock_core to Librarian
Move web search functionality to The Librarian agent, integrating with
the library-desk /rag/search endpoint for enhanced search capabilities.

Changes:
- Add search_web, read_url, read_urls_batch tools to Librarian
- Add WebSearchResult, ContentExtractionResult models to client
- Add search_web, extract_content, extract_content_batch client methods
- Update Librarian capability with web/url/internet domains
- Remove search_web from tatlock_core tools and toolset
- Update Tatlock system prompt to delegate web search to Librarian
- Add comprehensive unit tests for new Librarian tools
- Clean up legacy src/agents/tools.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 18:30:16 +01:00
jpmschweitzerandClaude Opus 4.5 49f0da8068 feat: two-phase execution, think slugs, query enrichment (v1.6.0)
Build and Push / build (release) Successful in 1m14s
Two-Phase Tatlock Execution:
- orchestrate_tool_calls() for Phase 1 coordination
- synthesize_from_results() for Phase 2 butler-toned synthesis
- Guarantees butler personality in all responses

Automatic Think Slugs:
- Deterministic butler-perspective messages during expert delegation
- ActionType enum: RETRIEVE, RESEARCH, CREATE, CONTROL, RECORD
- HOUSEHOLD_THINK_MESSAGES mapping for all experts
- Streaming delegation wrappers with automatic think messages

Steward Query Enrichment:
- Auto-fill user context (location, timezone) when not specified
- _build_enriched_query() with regex word boundary matching
- enriched_query field in StewardRecommendation schema

Documentation:
- ORCHESTRATION_SCENARIOS.md rewritten with Mermaid diagrams
- New Housekeeper and Biographer scenarios
- TESTING_IMPROVEMENTS.md for future LLM testing patterns

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 14:00:32 +01:00
jpmschweitzerandClaude Opus 4.5 a1b8fe46e8 feat: add The Housekeeper agent for home automation
Implements The Housekeeper, a new expert agent for home automation
following the Librarian pattern. Communicates with core-api service
which wraps Home Assistant REST API.

New agent features:
- CoreAPIClient with 13 home automation methods
- 13 tools: list_areas, list_devices, get_device_state, turn_on,
  turn_off, toggle, list_scenes, activate_scene, list_scripts,
  run_script, list_automations, toggle_automation, get_history
- PydanticAI agent with butler-friendly system prompt
- HouseholdCapability registration for Steward coordination
- delegate_to_housekeeper() wrapper for orchestration

Also includes:
- Dev port changed from 8123 to 8777 (avoids Home Assistant conflict)
- Config: CORE_API_HOST, CORE_API_KEY, CORE_API_TIMEOUT
- 44 unit tests for client and capability

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 10:24:35 +01:00
jpmschweitzerandClaude Opus 4.5 64cad4500a feat: environment-aware config, direct delegation, E2E test suite (v1.4.0)
Build and Push / build (release) Successful in 52s
### Added
- Environment-aware configuration:
  - Auto-selected logging (DEBUG for dev, WARNING for prod)
  - Auto-selected default user (llm_tester for dev isolation)
  - User context logging at request entry
- Direct delegation bypass:
  - Pure memory/librarian requests skip Tatlock LLM
  - Reduces latency for memory-only requests
- Text-based delegation fallback:
  - Parse [DELEGATE:agent] patterns from LLM output
  - Sequential and parallel execution support
- Comprehensive E2E test suite:
  - 22 orchestration tests with QdrantVerifier
  - assert_llm_behavior() for flexible pattern matching
  - Tests for memory, delegation, isolation, scenarios

### Fixed
- Unit test mocks for streaming (async generator)
- Temporal context handling in tests
- LLM non-determinism with pytest.xfail()
- Streaming test timeouts increased

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 21:19:47 +01:00
jpmschweitzerandClaude Opus 4.5 9d7ce399c8 fix(memory): biographer tool type hints for Ollama (v1.3.2)
Build and Push / build (release) Successful in 51s
- Change `str | None` to `str` with empty default for memory_type
- Remove `keywords` parameter from store_insight (auto-generated anyway)
- Ollama's OpenAI API doesn't handle union types with None properly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 15:04:10 +01:00
jpmschweitzerandClaude Opus 4.5 7426dd1ac3 feat: add Phase F.2 - The Biographer (memory agent)
Add The Biographer household member for user memory management:

Memory Service (direct access layer):
- src/core/memory_service.py for fast, LLM-free lookups
- Profile, preference, and fact management
- Session context with Redis caching
- Steward integration via prefetch_context()

The Biographer Agent:
- src/agents/biographer/ package with PydanticAI agent
- Discreet chronicler personality for privacy
- Tools: recall_semantic, list_memories, store_insight,
  update_profile, update_preference, forget_memory
- Registered with Household Registry on startup

Steward Integration:
- Memory context pre-fetch during analysis
- Profile/preferences included in Butler note
- Keyword-based context determination

Also includes:
- delegate_to_biographer() wrapper
- 34 new tests (capability + memory service)
- Version bump to 1.2.0

Documentation cleanup:
- Removed obsolete PHASE2_COMPLETE.md, PHASE2_PLAN.md
- Removed docs/library-desk-requirements.md
- Moved ORCHESTRATION_SCENARIOS.md to project root

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 19:20:18 +01:00
jpmschweitzerandClaude Opus 4.5 4c6ac89808 feat: add Phase F.1 memory infrastructure
Add multi-tenancy support and memory storage infrastructure:

- Add ContextVar-based request context (src/core/context.py)
  - Async-safe user/conversation tracking via contextvars
  - RequestContext manager for clean setup/teardown
  - get_user(), get_conversation_id() helpers

- Add multi-tenancy utilities (src/core/multi_tenancy.py)
  - User ID sanitization for collection/key names
  - get_memory_collection_name(), get_session_key() helpers

- Add Ollama embedding client (src/core/embeddings.py)
  - nomic-embed-text model (768 dimensions)
  - embed(), embed_batch(), health_check() methods

- Add Qdrant client wrapper (src/core/qdrant.py)
  - Per-user collection pattern: memories_{user}
  - upsert_memory(), search_memories(), delete_memory()
  - Type-based filtering support

- Add Redis memory cache (src/core/memory_cache.py)
  - Session context with 24h TTL
  - Recent entities tracking
  - Separate from benchmarks (db=2)

- Update config with memory settings
  - QDRANT_HOST, QDRANT_PORT, QDRANT_EMBEDDING_DIM
  - OLLAMA_EMBEDDING_MODEL
  - REDIS_MEMORY_DB, REDIS_MEMORY_TTL_HOURS

- Add user field to ResponseRequest (OpenAI standard)
- Set context in router, reset in finally block
- Update librarian client to use get_user() (12 methods)

All 333 unit tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 17:27:51 +01:00
jpmschweitzerandClaude Opus 4.5 1e4bba2422 feat: add sequential multi-expert execution to orchestration
Adds multi-expert coordination infrastructure:
- ExecutionMode enum (SEQUENTIAL, PARALLEL)
- MultiExpertResult dataclass for aggregating results
- execute_sequential(): Tasks run one after another
- execute_parallel(): Tasks run concurrently via asyncio.gather
- orchestrate_multi_expert(): Streaming think updates during multi-expert work

Supports:
- Stop-on-failure mode for sequential execution
- Partial failure handling (some succeed, some fail)
- Result aggregation with combined output formatting
- Exception handling in parallel execution

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 13:22:11 +01:00
jpmschweitzerandClaude Opus 4.5 1970751b2f feat: add orchestration loop with think update streaming
Creates orchestration module for multi-expert coordination:
- parse_delegation_from_steward_note(): Extracts delegation task
- execute_delegation(): Routes to appropriate expert agent
- orchestrate_with_think_updates(): Streams <think> updates around
  delegation calls while using run() internally

This enables real-time user feedback while avoiding Ollama's
streaming+tool call bugs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 12:59:20 +01:00
jpmschweitzerandClaude Opus 4.5 6cc0bd78b2 feat: update Steward prompt for clearer delegation instructions
Updates Steward's output format to structured delegation format:
- DELEGATE: [capability] to [action] [task]
- REASON: [explanation]
- COMPLEXITY: [simple/moderate/complex]
- CONTEXT: [relevant history or "none"]

Also adds guidance for conversation memory queries (handled by
Tatlock directly, not delegated to Librarian).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 12:46:51 +01:00
jpmschweitzerandClaude Opus 4.5 54b6fcd7cc feat: add DelegationTask dataclass and delegate_to_librarian wrapper
Introduces agent-as-tool pattern infrastructure:
- DelegationTask: Structured representation of expert work
- DelegationResult: Typed result from expert delegation
- delegate_to_librarian(): Wrapper for Librarian agent calls

This implements PydanticAI's recommended delegation pattern where
parent agents call child agents via tool wrappers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 11:19:35 +01:00
jpmschweitzerandClaude Opus 4.5 b5ee1f3e44 fix: use run() instead of run_stream() for scoped tools to avoid Ollama 400 bug
PydanticAI + Ollama streaming with tool calls has known issues:
- Issue #1292: Streaming stops after tool call due to empty TextPart
- Issue #2256: Empty text part causes run to end prematurely

This change uses run() for the actual tool execution while still
yielding the response in chunks to maintain the streaming UX.
The orchestration loop can emit <think> updates between await calls.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 10:59:58 +01:00
jpmschweitzerandClaude Opus 4.5 4efa717796 fix: improve Steward delegation instructions for Librarian
Build and Push / build (release) Successful in 10s
- Update Librarian capability description to highlight CREATE/UPDATE/SEARCH
- Add specific Steward guidelines for wiki creation, updates, and research
- Add dynamic time injection to user prompts for temporal awareness
- Expand domains to include 'create', 'write', 'update'
- Update test to match new capability description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:55:17 +01:00
jpmschweitzerandClaude Opus 4.5 7ec6e03c65 feat(phase3): add multi-agent coordination engine
- CoordinationEngine for task orchestration between agents
- Routing tasks to appropriate expert agents
- Sequential and parallel execution support
- Result aggregation from multiple agents
- Graceful error handling and degradation
- Streaming delegation support
- Convenience functions: delegate_to_librarian(), delegate_to_librarian_stream()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:29:03 +01:00
jpmschweitzerandClaude Opus 4.5 f6f37b341b feat(phase3): add The Librarian agent with library-desk integration
Library-Desk API Client:
- Async HTTP client with httpx for library-desk API
- HybridRAG search (vector + graph + web)
- Wiki operations (search, get, list, create, update)
- Smart page creation with HybridRAG research
- Semantic vector search and knowledge graph queries
- Dossier browsing and health checks

Librarian Tools (11 total):
- Research: hybrid_search, search_wiki, get_wiki_page, semantic_search
- Browse: list_dossiers, get_dossier_pages, explore_knowledge_graph
- Graph: find_related_entities
- Write: create_wiki_page, update_wiki_page, smart_create_wiki_page

Agent:
- PydanticAI agent with research assistant personality
- System prompt with research and writing workflows
- Streaming support via run_librarian_stream()

Capability:
- LIBRARIAN_CAPABILITY definition for Household Registry
- Automatic registration on startup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:27:09 +01:00
jpmschweitzerandClaude Opus 4.5 92c0d5d770 feat(phase3): add agent communication protocol
- AgentRequest/AgentResponse for standardized inter-agent communication
- DelegationIntent for routing tasks to expert agents
- CoordinationResult for aggregated multi-agent results
- DelegationReason enum (domain expertise, tool access, etc.)
- Error types: AgentError, AgentTimeoutError, AgentUnavailableError

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:26:52 +01:00
jpmschweitzerandClaude Sonnet 4.5 4d109100fe fix: implement proper streaming with PydanticAI delta mode
Fix streaming issues that caused text repetition and broken tool execution
in Open WebUI. Implements real LLM streaming using PydanticAI's run_stream()
with delta=True instead of artificial word-by-word chunking.

**Fixed:**
- Text repetition in streaming output (was accumulating instead of deltas)
- Broken tool execution (tools now execute properly in streaming mode)
- Invalid 'thinking' parameter in ReasoningOutputItem schema

**Changes:**
- Add run_with_scoped_tools_stream() method to TatlockAgent
  - Uses PydanticAI's run_stream() with delta=True for real deltas
  - Properly streams LLM output with tool execution
- Update StreamingCoordinator.stream_response_with_steward()
  - Uses new streaming method instead of fake word-by-word streaming
  - Removes invalid thinking parameter from ReasoningOutputItem
- All streaming now uses actual LLM deltas, not accumulated text

Resolves streaming issues reported in Open WebUI where responses showed
repetitive text and tool calls appeared as raw JSON instead of executed results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:39:52 +01:00
jpmschweitzerandClaude Sonnet 4.5 6eed5f4d13 feat: implement Phase 2 two-tier architecture with Steward
Add comprehensive two-tier architecture where Steward analyzes requests
and Tatlock executes with scoped tools. Includes full infrastructure for
request preprocessing, tool tracking, benchmarking, and streaming.

**Added:**
- Steward agent for request analysis and capability recommendation
- Household Registry for centralized capability management
- Request preprocessing pipeline (Steward → Tatlock flow)
- Tool usage tracking and benchmarking system
- Streaming transparency (Steward reasoning visible in streams)
- Structured logging with operation timing
- Redis benchmark storage with 30-day expiry
- Benchmark analysis CLI tools

**Infrastructure:**
- src/agents/steward/ - Steward agent implementation
- src/agents/tatlock_core/ - Tatlock capability domain
- src/core/preprocessing.py - Request preprocessing pipeline
- src/core/tool_tracking.py - Tool call tracking
- src/core/benchmarks.py - Benchmark recording system
- src/core/household_registry.py - Capability registry
- src/core/startup.py - Application startup coordination
- src/core/logging_config.py - Structured logging setup

**Integration:**
- Responses API uses Steward for Tatlock requests
- Chat Completions wraps Responses API for OpenAI compatibility
- Streaming coordinator supports Steward + Tatlock flow
- Tool scoping per request based on Steward recommendations

**Testing:**
- Integration tests for Steward-Tatlock flow
- Benchmark and registry unit tests
- Steward streaming tests

See PHASE2_PLAN.md and PHASE2_COMPLETE.md for detailed documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:39:20 +01:00
jpmschweitzerandClaude Sonnet 4.5 38120696bf feat: add conversation history and tool call logging
Add two major features to enhance Tatlock's capabilities:

1. Conversation History Support:
   - Convert OpenAI-format messages to PydanticAI ModelRequest/ModelResponse
   - Pass full conversation context via message_history parameter
   - Filter empty messages to prevent Ollama errors
   - Add debug logging for message history construction
   - Tatlock now remembers previous turns in multi-turn conversations

2. Tool Call Logging:
   - Implement ToolCallTracker dependency for per-request tracking
   - Tools log usage via RunContext deps parameter
   - Web search: "🔍 Searching for: 'query'"
   - Calculator: "🧮 Calculating: expression"
   - Date/time: "🕐 Calculating date offset: description"
   - Tool logs appear in reasoning output as <think> tags in Open WebUI

Both features improve user experience by maintaining conversation context
and providing transparency into tool usage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 01:13:19 +01:00
jpmschweitzer 67481515cc feat: integrate Tatlock agent with PydanticAI and Ollama
Convert Tatlock from mock to real PydanticAI agent:
- Connect to Ollama backend (mistral-nemo:latest)
- British butler personality with research-oriented mindset
- Lazy initialization pattern for better testability
- Register permanent tools (calculator, date/time, search)
- Streaming response support with reasoning output
- Error handling for PydanticAI exceptions
- Update registry tests for tools capability
- Add integration test for streaming functionality
2025-12-07 00:12:39 +01:00
jpmschweitzer f3e2681a6c feat: implement permanent tools (calculator, date/time, search)
Add three permanent tools for Tatlock agent:
- Calculator: Safe math expression evaluation (arithmetic, algebra, trig, log)
- Date/Time toolkit: Current time, relative dates, time differences
- Web Search: SearXNG integration for privacy-preserving search

Tools use PydanticAI @agent.tool decorator pattern with:
- Clear docstrings visible to LLM
- Error handling with string-based messages
- Async support for I/O operations (web search)
- 26 comprehensive tool tests
2025-12-07 00:10:49 +01:00
jpmschweitzer da9b4954be rename to Tatlock 2025-12-06 20:59:15 +01:00
jpmschweitzerandClaude 4e6ca4466b Add agent interface and model implementations
Implements Phase 1: Agent abstraction layer with multiple model support

Features:
- Abstract AgentInterface base class with standard contract
- LoremTesterAgent: Full-featured mock agent with realistic behavior
  - Configurable reasoning effort levels (none to xhigh)
  - Random tool/function call generation
  - Error triggers for testing (rate_limit, context_overflow)
  - Temperature-based response variation
- TatlockAgent: Placeholder for future PydanticAI integration
- ModelRegistry: Centralized model management and discovery

Testing:
- 9 unit tests for lorem-tester agent behavior
- 9 unit tests for registry operations
- Coverage: Agent abstraction fully tested

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 19:36:53 +01:00