Commit Graph
27 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Fable 5 033a1c01e8 feat: make Ollama/gemma4 the primary backend with Claude as fallback
Rolls back the claudification backend preference: PREFER_CLOUD_BACKEND now
defaults to false, resolve_backend() picks Ollama first and uses Claude when
explicitly preferred or when the new Ollama startup health check fails. The
Steward retries mid-request failures on the other backend in both directions.

Also hardens the fallback itself: Anthropic SDK imports are lazy so a broken
anthropic package degrades to Ollama-only instead of crashing at import time
(root cause of the production outage since April), anthropic is pinned to a
pydantic-ai-1.27-compatible range, ANTHROPIC_MODEL defaults to claude-sonnet-5
(sonnet-4-20250514 retired 2026-06-15), sampling parameters are stripped from
Claude calls (Sonnet 5 rejects them), and the Steward timeout is configurable
(STEWARD_TIMEOUT, default 60s) since gemma4 needs ~35s warm for analysis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 18:35:53 +02:00
jpmschweitzerandClaude Opus 4.6 427ad311dc feat: switch default Ollama model to gemma4:e2b
Build and Push / release (push) Successful in 21s
Build and Push / build (push) Successful in 5m27s
gemma4:e2b has native function calling with dedicated tool tokens,
achieving 100% tool selection accuracy in benchmarks vs 67% for
mistral-nemo-large, with 5-8x faster response times (2-4s vs 15-20s)
and lower VRAM usage (8GB vs 9.2GB).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:44:27 +02:00
jpmschweitzerandClaude Opus 4.5 496f37a538 feat: add Claude backend with automatic Ollama fallback (Claudification Phase 1)
Build and Push / release (release) Failing after 6s
Build and Push / build (release) Successful in 3m5s
All agents now prefer Claude API when ANTHROPIC_API_KEY is configured,
with automatic fallback to Ollama when offline or unconfigured. New
src/anthropic/ module provides model selection via get_model() factory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 07:19:29 +01:00
jpmschweitzerandClaude Opus 4.5 87f2926db2 feat: integrate tracing throughout request pipeline
Instrument the full request flow with trace spans for debugging:

- Wrap expert delegations (librarian/biographer/housekeeper) in spans
- Add orchestrate and synthesize spans to TatlockAgent
- Trace Steward analysis in preprocessing
- Start/end traces in response service with context management
- Simplify router by moving context handling to service layer
- Include tracing router in debug mode
- Remove benchmark recording from tool_tracking and steward service

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 10:26:37 +01:00
jpmschweitzerandClaude Opus 4.5 2a9449bc81 refactor: remove Redis benchmark system
Remove the Redis-backed performance benchmarking in favor of the new
lightweight file-based tracing system which provides better debugging
capabilities for local development.

- Delete src/core/benchmarks.py
- Remove ENABLE_BENCHMARKS, REDIS_BENCHMARK_DB, redis_url from config
- Update memory_cache comment (now uses DB 1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 10:26:17 +01:00
jpmschweitzerandClaude Opus 4.5 60d84535c0 feat: add lightweight request tracing for debugging
Adds JSON-based tracing system for local development that captures
the full request flow through Tatlock's multi-agent architecture.

- Trace/Span dataclasses with automatic timing and nesting
- Context-var based propagation for async-safe tracing
- trace_span async context manager for clean instrumentation
- Traces written to logs/traces/{trace_id}.json
- REST API for listing and retrieving traces (/traces)
- Standalone HTML viewer with timeline visualization

Enabled via DEBUG=true environment variable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 10:25:56 +01:00
jpmschweitzerandClaude Opus 4.5 583c407edd fix: Redis bool storage, tool tracking matching, e2e fixture scope
Build and Push / build (release) Successful in 53s
- Convert booleans to strings for Redis hset (Redis doesn't accept bool)
- Extract capability from delegate_to_X tool names for tracking
- Use loop_scope="module" for pytest-asyncio module-scoped fixtures
- Add note about using venv for tests in AGENTS.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 14:53:51 +01:00
jpmschweitzerandClaude Opus 4.5 49f0da8068 feat: two-phase execution, think slugs, query enrichment (v1.6.0)
Build and Push / build (release) Successful in 1m14s
Two-Phase Tatlock Execution:
- orchestrate_tool_calls() for Phase 1 coordination
- synthesize_from_results() for Phase 2 butler-toned synthesis
- Guarantees butler personality in all responses

Automatic Think Slugs:
- Deterministic butler-perspective messages during expert delegation
- ActionType enum: RETRIEVE, RESEARCH, CREATE, CONTROL, RECORD
- HOUSEHOLD_THINK_MESSAGES mapping for all experts
- Streaming delegation wrappers with automatic think messages

Steward Query Enrichment:
- Auto-fill user context (location, timezone) when not specified
- _build_enriched_query() with regex word boundary matching
- enriched_query field in StewardRecommendation schema

Documentation:
- ORCHESTRATION_SCENARIOS.md rewritten with Mermaid diagrams
- New Housekeeper and Biographer scenarios
- TESTING_IMPROVEMENTS.md for future LLM testing patterns

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 14:00:32 +01:00
jpmschweitzerandClaude Opus 4.5 a1b8fe46e8 feat: add The Housekeeper agent for home automation
Implements The Housekeeper, a new expert agent for home automation
following the Librarian pattern. Communicates with core-api service
which wraps Home Assistant REST API.

New agent features:
- CoreAPIClient with 13 home automation methods
- 13 tools: list_areas, list_devices, get_device_state, turn_on,
  turn_off, toggle, list_scenes, activate_scene, list_scripts,
  run_script, list_automations, toggle_automation, get_history
- PydanticAI agent with butler-friendly system prompt
- HouseholdCapability registration for Steward coordination
- delegate_to_housekeeper() wrapper for orchestration

Also includes:
- Dev port changed from 8123 to 8777 (avoids Home Assistant conflict)
- Config: CORE_API_HOST, CORE_API_KEY, CORE_API_TIMEOUT
- 44 unit tests for client and capability

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 10:24:35 +01:00
jpmschweitzerandClaude Opus 4.5 64cad4500a feat: environment-aware config, direct delegation, E2E test suite (v1.4.0)
Build and Push / build (release) Successful in 52s
### Added
- Environment-aware configuration:
  - Auto-selected logging (DEBUG for dev, WARNING for prod)
  - Auto-selected default user (llm_tester for dev isolation)
  - User context logging at request entry
- Direct delegation bypass:
  - Pure memory/librarian requests skip Tatlock LLM
  - Reduces latency for memory-only requests
- Text-based delegation fallback:
  - Parse [DELEGATE:agent] patterns from LLM output
  - Sequential and parallel execution support
- Comprehensive E2E test suite:
  - 22 orchestration tests with QdrantVerifier
  - assert_llm_behavior() for flexible pattern matching
  - Tests for memory, delegation, isolation, scenarios

### Fixed
- Unit test mocks for streaming (async generator)
- Temporal context handling in tests
- LLM non-determinism with pytest.xfail()
- Streaming test timeouts increased

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 21:19:47 +01:00
jpmschweitzerandClaude Opus 4.5 8e38a568ef fix(memory): add biographer to delegation wrappers (v1.3.1)
Build and Push / build (release) Successful in 50s
- Add delegate_to_biographer to household registry delegation map
- Was returning raw tools which caused Ollama "invalid message content type: nil"
- Add Qdrant host/port to .env.example

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 14:56:32 +01:00
jpmschweitzerandClaude Opus 4.5 40663511b4 feat: memory system fixes and Redis config cleanup (v1.3.0)
Build and Push / build (release) Successful in 26s
- Fix Qdrant client to use query_points API (qdrant-client >= 1.10)
- Rename REDIS_DB to REDIS_BENCHMARK_DB for clarity
- Update Redis defaults to match stack allocation (benchmark=6, memory=1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 14:36:56 +01:00
jpmschweitzerandClaude Opus 4.5 7426dd1ac3 feat: add Phase F.2 - The Biographer (memory agent)
Add The Biographer household member for user memory management:

Memory Service (direct access layer):
- src/core/memory_service.py for fast, LLM-free lookups
- Profile, preference, and fact management
- Session context with Redis caching
- Steward integration via prefetch_context()

The Biographer Agent:
- src/agents/biographer/ package with PydanticAI agent
- Discreet chronicler personality for privacy
- Tools: recall_semantic, list_memories, store_insight,
  update_profile, update_preference, forget_memory
- Registered with Household Registry on startup

Steward Integration:
- Memory context pre-fetch during analysis
- Profile/preferences included in Butler note
- Keyword-based context determination

Also includes:
- delegate_to_biographer() wrapper
- 34 new tests (capability + memory service)
- Version bump to 1.2.0

Documentation cleanup:
- Removed obsolete PHASE2_COMPLETE.md, PHASE2_PLAN.md
- Removed docs/library-desk-requirements.md
- Moved ORCHESTRATION_SCENARIOS.md to project root

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 19:20:18 +01:00
jpmschweitzerandClaude Opus 4.5 4c6ac89808 feat: add Phase F.1 memory infrastructure
Add multi-tenancy support and memory storage infrastructure:

- Add ContextVar-based request context (src/core/context.py)
  - Async-safe user/conversation tracking via contextvars
  - RequestContext manager for clean setup/teardown
  - get_user(), get_conversation_id() helpers

- Add multi-tenancy utilities (src/core/multi_tenancy.py)
  - User ID sanitization for collection/key names
  - get_memory_collection_name(), get_session_key() helpers

- Add Ollama embedding client (src/core/embeddings.py)
  - nomic-embed-text model (768 dimensions)
  - embed(), embed_batch(), health_check() methods

- Add Qdrant client wrapper (src/core/qdrant.py)
  - Per-user collection pattern: memories_{user}
  - upsert_memory(), search_memories(), delete_memory()
  - Type-based filtering support

- Add Redis memory cache (src/core/memory_cache.py)
  - Session context with 24h TTL
  - Recent entities tracking
  - Separate from benchmarks (db=2)

- Update config with memory settings
  - QDRANT_HOST, QDRANT_PORT, QDRANT_EMBEDDING_DIM
  - OLLAMA_EMBEDDING_MODEL
  - REDIS_MEMORY_DB, REDIS_MEMORY_TTL_HOURS

- Add user field to ResponseRequest (OpenAI standard)
- Set context in router, reset in finally block
- Update librarian client to use get_user() (12 methods)

All 333 unit tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 17:27:51 +01:00
jpmschweitzerandClaude Opus 4.5 a077121b39 refactor: switch preprocessing to use delegation tools
Changes preprocessing to use get_delegation_tools() instead of
get_scoped_tools(). Expert agents now get delegation wrappers
(delegate_to_librarian) while core tools are returned directly.

This reduces Tatlock's cognitive load from 16+ tools to ~3-5.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 12:46:39 +01:00
jpmschweitzerandClaude Opus 4.5 1a2e6392d2 feat: add get_delegation_tools() to household registry
Implements the agent-as-tool pattern in the registry:
- For members WITH an agent: returns delegation wrapper function
- For members WITHOUT an agent: returns raw tools directly

This reduces Tatlock's tool count from 16+ to ~3-5, preventing
cognitive overload and improving Ollama reliability.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 11:23:10 +01:00
jpmschweitzerandClaude Opus 4.5 4efa717796 fix: improve Steward delegation instructions for Librarian
Build and Push / build (release) Successful in 10s
- Update Librarian capability description to highlight CREATE/UPDATE/SEARCH
- Add specific Steward guidelines for wiki creation, updates, and research
- Add dynamic time injection to user prompts for temporal awareness
- Expand domains to include 'create', 'write', 'update'
- Update test to match new capability description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:55:17 +01:00
jpmschweitzerandClaude Opus 4.5 09e468e7f8 feat: load version dynamically from pyproject.toml
- Add _get_version_from_pyproject() function to config.py
- APP_VERSION now uses default_factory to load from pyproject.toml
- Add pyproject.toml to Docker build for version detection
- Add LIBRARY_DESK configuration settings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:34:23 +01:00
jpmschweitzerandClaude Opus 4.5 27b46a9fe7 feat(phase3): register Librarian on application startup
- Add Librarian registration to household member registration
- Error handling to prevent startup failure if Librarian unavailable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:29:53 +01:00
jpmschweitzerandClaude Sonnet 4.5 eba46f7e9b chore: bump version to 0.2.5
Update version across all configuration files and documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:44:27 +01:00
jpmschweitzerandClaude Sonnet 4.5 6eed5f4d13 feat: implement Phase 2 two-tier architecture with Steward
Add comprehensive two-tier architecture where Steward analyzes requests
and Tatlock executes with scoped tools. Includes full infrastructure for
request preprocessing, tool tracking, benchmarking, and streaming.

**Added:**
- Steward agent for request analysis and capability recommendation
- Household Registry for centralized capability management
- Request preprocessing pipeline (Steward → Tatlock flow)
- Tool usage tracking and benchmarking system
- Streaming transparency (Steward reasoning visible in streams)
- Structured logging with operation timing
- Redis benchmark storage with 30-day expiry
- Benchmark analysis CLI tools

**Infrastructure:**
- src/agents/steward/ - Steward agent implementation
- src/agents/tatlock_core/ - Tatlock capability domain
- src/core/preprocessing.py - Request preprocessing pipeline
- src/core/tool_tracking.py - Tool call tracking
- src/core/benchmarks.py - Benchmark recording system
- src/core/household_registry.py - Capability registry
- src/core/startup.py - Application startup coordination
- src/core/logging_config.py - Structured logging setup

**Integration:**
- Responses API uses Steward for Tatlock requests
- Chat Completions wraps Responses API for OpenAI compatibility
- Streaming coordinator supports Steward + Tatlock flow
- Tool scoping per request based on Steward recommendations

**Testing:**
- Integration tests for Steward-Tatlock flow
- Benchmark and registry unit tests
- Steward streaming tests

See PHASE2_PLAN.md and PHASE2_COMPLETE.md for detailed documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:39:20 +01:00
jpmschweitzer a1a0f6923b feat: add SearXNG configuration for web search tool
- Add SEARXNG_HOST config with localhost:8087 default
- Add SEARXNG_TIMEOUT setting (30 seconds default)
- Update .env.example with SearXNG configuration
- Supports both local and production SearXNG instances
2025-12-07 00:10:30 +01:00
jpmschweitzerandClaude ab8e3b1566 Bump version to 0.1.1
Version increment to mark basic setup completion milestone.

Changes:
- Updated APP_VERSION to 0.1.1 in src/core/config.py
- Released CHANGELOG.md [Unreleased] section as [0.1.1]
- Updated version links to use git.schweitz.net repository

This version represents the completion of all core infrastructure:
- Agent interface and implementations
- Responses API with full feature set
- Chat Completions wrapper
- Comprehensive test coverage (95 tests, 78.95%)
- Production-ready architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 19:55:56 +01:00
jpmschweitzerandClaude 5e40704d91 Add Chat Completions wrapper with reasoning conversion
Implements Phase 5: OpenAI Chat Completions compatibility layer

Features:
- Wraps Responses API for single source of truth
- Automatically enables reasoning generation
- Converts reasoning items to <think> tags for Open WebUI
- Maintains OpenAI-compatible chat completion format
- Supports both streaming and non-streaming modes
- Pipeline prefix preservation for model names
- System message handling

Architecture:
- Service layer calls Responses API internally
- Streams word-by-word for smooth UX
- Reasoning displayed in thought bubbles (Open WebUI)
- Main response shown separately from thinking

Error Handling:
- Enhanced exception types (RateLimitError, ContextLengthError)
- OpenAI-compatible error format
- Graceful error propagation from Responses API

Testing:
- 6 unit tests for chat router functionality
- 6 unit tests for streaming wrapper behavior
- Total: 12 tests with comprehensive coverage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 19:38:35 +01:00
jpmschweitzerandClaude 1d15672ed8 Add core router and main application
Implement application factory pattern with clean main.py.
Separate routers for each domain, centralized exception handling.

Core Router (src/core/router.py):
- Root endpoint (/)
- Health check endpoint (/health)
- Simple status responses
- No prefix (mounted at root)

Main Application (src/main.py):
- create_application() factory function
- Clean configuration-focused main.py
- CORS middleware setup
- Exception handler registration
- Router registration with proper prefixes
- Global config integration

Application Architecture:
- Application factory pattern for testability
- Routers imported from separate controllers
- Exception handlers in dedicated function
- All routes cleanly separated by domain

Exception Handling:
- OpenAI-compatible error format
- Custom AppException handler
- Validation error handler (422)
- Generic exception handler (500)

Following Best Practices:
- Separation of concerns
- Factory pattern for DI
- Clean main.py (config only)
- Type hints throughout

Status: Production-ready structure

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 10:46:38 +01:00
jpmschweitzerandClaude 8474d8b21c Implement async Ollama client
Add production-ready async HTTP client for Ollama API communication
with proper error handling and dependency injection.

Ollama Client (src/ollama/client.py):
- Async context manager for connection lifecycle
- Non-streaming chat endpoint
- Streaming chat endpoint with async generator
- Model listing endpoint
- Health check endpoint
- Timeout configuration per request
- Comprehensive error handling with custom exceptions
- FastAPI dependency injection support

Ollama Schemas (src/ollama/schemas.py):
- OllamaMessage: Chat message format
- OllamaChatRequest: Request with model, messages, options
- OllamaChatResponse: Complete chat response
- OllamaModelInfo: Model metadata
- OllamaModelsResponse: Model list response

Features:
- Async/await throughout for non-blocking I/O
- Connection pooling via httpx.AsyncClient
- Configurable timeouts (default: 120s)
- Proper exception mapping (connection errors, timeouts)
- Ready for integration (currently not connected to routes)

Following Best Practices:
- Async context manager pattern
- Dependency injection for FastAPI routes
- Separation of concerns (client vs schemas)
- Type hints throughout
- Comprehensive logging

Status: Ready for integration (mock responses used in routes currently)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 10:32:10 +01:00
jpmschweitzerandClaude 0ed6c5086c Add core configuration and base models
Implement global application configuration and custom Pydantic models
following FastAPI best practices.

Core Configuration (src/core/config.py):
- BaseSettings with environment variable support
- Split configuration by domain (following best practices)
- Ollama connection settings (host, model, timeouts)
- API configuration (host, port, prefix)
- CORS settings
- 20-second streaming timeout per turn
- Cached configuration with @lru_cache

Custom Base Models (src/core/models.py):
- CustomBaseModel for consistent serialization
- ISO datetime formatting
- Alias population support
- Enum value serialization
- Validation on assignment
- serializable_dict() for logging/debugging

Exception Handling (src/core/exceptions.py):
- Base AppException with status codes
- OllamaConnectionError (503)
- OllamaTimeoutError (504)
- ModelNotFoundError (404)
- ValidationError (422)
- OpenAI-compatible error structure

Benefits:
- Consistent configuration across domains
- Type-safe settings with validation
- Easy environment override via .env
- Predictable error responses
- Better debugging with serializable models

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-06 10:31:36 +01:00