Commit Graph
52 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Fable 5 a8bc282576 fix(librarian): expose page_id in wiki search results
search_wiki printed ordinally numbered results with no page ID while
get_wiki_page demands 'the page ID from search results' - the model
passed the list position (page 1) and 404'd. Results now carry
page_id and drop the ordinals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 15:37:00 +02:00
jpmschweitzerandClaude Fable 5 fccbe65ecf fix(librarian): use terse tool-phase prompt so gemma4 calls tools
The scholarly persona prompt reproduced the exact pathology
TATLOCK_ORCHESTRATION_PROMPT fixed for the butler: gemma4 answered in
character ('please provide your request') without calling a single tool.
The research phase now uses a tool-discipline prompt; Tatlock's synthesis
supplies the voice. Anti-fabrication rules kept verbatim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 15:37:00 +02:00
jpmschweitzerandClaude Fable 5 5376a34645 refactor(agents): reduce protocol.py to the live AgentError
Post-coordination-removal sweep: the coordination wire protocol
(AgentRequest, AgentResponse, DelegationIntent, CoordinationResult,
DelegationReason, TaskComplexity, ToolCallRecord, AgentTimeoutError,
AgentUnavailableError, DelegationError) had zero importers left in
src/ - only its own test module. AgentError stays (raised by
run_librarian, mapped to user-safe failures by delegation.py).
Also drops the stale coordination.py line from the README tree.

Import-cycle sanity: python -c 'import src.main' passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 12:03:31 +02:00
jpmschweitzerandClaude Fable 5 8cf3609948 fix(librarian): quiet healthy-search coverage notes, tag clearing, strict result pairing
Phase A review minors:

- Coverage note: source_status (when present) is now used exclusively;
  the source_counts-absence fallback only considers the optional legs
  the request explicitly enabled (web/documents/volatile). library-desk
  computes source_counts from the final top-N fused results only, so
  absence of the always-on vector/graph legs is normal ranking behavior
  - the old heuristic warned on virtually every healthy search
- update_wiki_page: the empty-list tags sentinel (leave unchanged) made
  clearing all tags impossible; pass exactly ["__CLEAR__"] to send an
  empty tag list, documented in the docstring for the local model
- Text-delegation parallel fallback: zip(..., strict=True) with an
  explicit count-mismatch guard so results can never be silently
  attributed to the wrong agent

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 11:59:15 +02:00
jpmschweitzerandClaude Fable 5 31b948a748 refactor(agents): delete dead coordination/streaming delegation stack
One delegation implementation remains (src/agents/delegation.py).
Removed, after verifying zero live importers post-Phase-A/B:

- src/agents/coordination.py: CoordinationEngine, duplicate
  delegate_to_librarian, AGENT_EXECUTORS/AGENT_STREAM_EXECUTORS
  (only importer was its own test module)
- run_librarian_stream: documented-broken path (Ollama streaming +
  tool call bug, PydanticAI #1292/#2256), only called by the deleted
  coordination engine
- stream_delegate_to_* wrappers + STREAMING_DELEGATION_WRAPPERS and
  the never-parsed __DELEGATION_RESULT__ marker in delegation.py
- HouseholdRegistry.get_streaming_delegation_tools() (no callers)
- tests/agents/test_coordination.py and the wrapper/stream tests

Note: the STREAMING_DELEGATION_WRAPPERS import in
src/responses/streaming.py was already removed by Phase A (7ce1c1a);
nothing to delete there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 11:52:36 +02:00
jpmschweitzerandClaude Fable 5 c00224222b fix(librarian): apply tenant guard to explicit user args in client
LibraryDeskClient._resolve_user only enforced non-empty: an explicit
user argument to any tenant-scoped method bypassed tatlock's tenant
guard entirely and went straight to library-desk, and padded values
were sent un-stripped on the wire.

Route the explicit-arg path through the same apply_tenant_guard() used
by context resolution and strip whitespace before the empty check, so
a non-production environment can never send the production tenant (or
a sanitization-collision variant) to library-desk, regardless of how
the user was supplied. Defense in depth - no in-repo caller passes an
explicit user today.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:29:14 +02:00
jpmschweitzerandClaude Fable 5 b9eae38556 feat(librarian): send explicit non-empty user on every library-desk request
Library-desk is removing its server-side default user, so a request
without an explicit tenant will 422 after its next deploy:

- New client-level _resolve_user() resolves the tenant (explicit arg
  or request context) and raises ValueError on an empty/whitespace
  value BEFORE any bytes hit the wire; all 15 tenant-scoped methods
  use it
- extract_content / extract_content_batch now accept and send the
  user (query param), matching the rest of the API surface
- search_web no longer falls back to a phantom "tatlock-librarian"
  tenant; it sends the resolved user
- health_check stays user-less (public, not tenant-scoped)

Tests: parametrized sweep pins the wire contract (user present in
params or payload) for every tenant-scoped method, for both context
and explicit users; empty-tenant calls are asserted to fail without
any HTTP call; the recorded-fixture hybrid contract test now pins
user as an explicit query param.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:02:46 +02:00
jpmschweitzerandClaude Fable 5 7ce1c1a314 feat(responses): pass conversation context and stream thinks in real time
- delegate_to_* now receives a trimmed conversation history (last ~6
  turns, 500 chars/turn) as context on both live direct-delegation
  paths (streaming and steward non-streaming), via new
  build_delegation_context helper
- _stream_direct_delegation restructured as an async generator: the
  butler 'start' think message streams BEFORE the expert runs and the
  success/error message right after it finishes, instead of all
  messages arriving after the research completed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:30:34 +02:00
jpmschweitzerandClaude Fable 5 0708c759fc fix(librarian): replace nullable tool params with sentinel defaults
Ollama's OpenAI-compatible API mishandles anyOf[X, null] parameter
schemas. update_wiki_page (content/title/tags/description) and
smart_create_wiki_page (path) now use empty-string/empty-list
sentinels translated to None inside the tool, following the
biographer pattern from 9d7ce39.

Adds a snapshot test that walks every registered librarian tool's
emitted JSON schema and fails on any anyOf[..., null].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:26:55 +02:00
jpmschweitzerandClaude Fable 5 24fed8814f feat(librarian): bounded retries, timeout wiring, and client reuse
- 2-attempt short-backoff retry for GETs and the read-only
  POST /query/* and /rag/search endpoints only; wiki writes are never
  retried (duplicate-page risk)
- honor the defined-but-ignored LIBRARY_DESK_TIMEOUT config instead of
  hardcoded 60s/30s per-call values
- hold ONE shared httpx.AsyncClient per librarian run via
  library_client_session (contextvar), instead of constructing a
  client per tool call; nested sessions are no-ops and custom targets
  still get their own client
- read tools raise ModelRetry on transient HTTP errors (transport
  errors, 5xx, 429) so Agent(retries=2) engages; write tools keep
  returning safe failure messages

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:24:42 +02:00
jpmschweitzerandClaude Fable 5 18f2e0efbd feat(agents): enforce one librarian timeout budget
- add LIBRARIAN_TIMEOUT config (default 180s) and enforce it with
  asyncio.wait_for inside delegate_to_librarian, covering the live
  paths (steward direct delegation and SSE streaming) that had no cap
- timeouts fail honestly: success=False with a curated butler sentence,
  detail in logs
- set an explicit timeout on TatlockOllamaProvider's AsyncOpenAI client
  from OLLAMA_TIMEOUT instead of the SDK default (~600s per LLM call)
- remove the contradictory unused 60s default from
  AgentRequest.timeout_seconds; coordination falls back to the
  configured budget

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:19:16 +02:00
jpmschweitzerandClaude Fable 5 99e1fe33ca feat(librarian): signal degraded search coverage
- parse source_counts into HybridRAGResponse and additively parse the
  shared-contract source_status/degraded fields when present (absence
  tolerated, so deploy order between tatlock and library-desk never
  matters)
- hybrid_search appends a one-line coverage note when a leg reported
  'failed' (or degraded is set), falling back to inferring silent legs
  from source_counts on older library-desk versions

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:15:09 +02:00
jpmschweitzerandClaude Fable 5 f853db8ccc fix(agents): make librarian failures structured and user-safe
- run_librarian / run_librarian_stream raise AgentError instead of
  returning/yielding error text as normal output; detail stays in logs
- delegate_to_* wrappers now put a curated butler-toned sentence in
  DelegationResult.output on failure and never expose str(e), so
  streaming's error branch is reachable and honest
- _execute_single_delegation propagates success; direct delegation only
  records delegate_to_* as called when the expert actually succeeded
- librarian tools return user-safe messages instead of
  'Error searching: {e}' strings that leaked internal URLs into
  synthesis; coordination stream errors are curated as well
- ruff cleanups (TYPE_CHECKING forward refs, B904, unused locals) in
  the touched files to keep them lint-clean

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:12:30 +02:00
jpmschweitzerandClaude Fable 5 59f5b54ac9 fix(librarian): map live HybridRAG response fields correctly
The client parsed field names the live library-desk service never
returns, so every result rendered as "unknown (score: 0.00)":

- source_type/sources -> source + sources (icons key off sources values)
- rrf_score -> score
- context -> formatted_context
- related_dossiers are per-result; top level aggregates unique titles
- synonyms live inside the keywords dict as a {term: [synonyms]} map

Also stop sending zero limits (service 422s on limit < 1); disabled
legs now rely on the enable_* flags with limits clamped to >= 1.

Adds a recorded live response as a fixture plus contract tests that
pin the mapping (non-unknown sources, non-zero scores, icon coverage).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 10:00:10 +02:00
jpmschweitzerandClaude Fable 5 6ec77091b6 style(librarian): apply ruff autofixes to client and tools
Mechanical Optional[X] -> X | None and f-string cleanups so subsequent
librarian changes lint clean against the dirty baseline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:58:43 +02:00
jpmschweitzerandClaude Fable 5 f03d41c698 fix: use dedicated tool-phase prompt for gemma4 orchestration
With the butler persona prompt attached, gemma4 reasons about calling the
calculator and then answers from memory with a different wrong product every
run; tool_choice=required via extra_body is advisory at best on Ollama's
OpenAI-compat layer. orchestrate_tool_calls() now uses a terse
TATLOCK_ORCHESTRATION_PROMPT; synthesize_from_results() keeps the persona,
so the user-visible voice is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 18:35:53 +02:00
jpmschweitzerandClaude Fable 5 033a1c01e8 feat: make Ollama/gemma4 the primary backend with Claude as fallback
Rolls back the claudification backend preference: PREFER_CLOUD_BACKEND now
defaults to false, resolve_backend() picks Ollama first and uses Claude when
explicitly preferred or when the new Ollama startup health check fails. The
Steward retries mid-request failures on the other backend in both directions.

Also hardens the fallback itself: Anthropic SDK imports are lazy so a broken
anthropic package degrades to Ollama-only instead of crashing at import time
(root cause of the production outage since April), anthropic is pinned to a
pydantic-ai-1.27-compatible range, ANTHROPIC_MODEL defaults to claude-sonnet-5
(sonnet-4-20250514 retired 2026-06-15), sampling parameters are stripped from
Claude calls (Sonnet 5 rejects them), and the Steward timeout is configurable
(STEWARD_TIMEOUT, default 60s) since gemma4 needs ~35s warm for analysis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 18:35:53 +02:00
jpmschweitzerandClaude Opus 4.5 e15def607d fix: remove extra_body tool_choice hack for Claude backend
Build and Push / build (push) Successful in 1m57s
Build and Push / release (push) Successful in 3s
PydanticAI handles tool_choice natively for Anthropic. The extra_body
hack caused an infinite tool call loop where Claude kept calling the
same tool because tool_choice was forced to "any".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 11:51:42 +01:00
jpmschweitzerandClaude Opus 4.5 496f37a538 feat: add Claude backend with automatic Ollama fallback (Claudification Phase 1)
Build and Push / release (release) Failing after 6s
Build and Push / build (release) Successful in 3m5s
All agents now prefer Claude API when ANTHROPIC_API_KEY is configured,
with automatic fallback to Ollama when offline or unconfigured. New
src/anthropic/ module provides model selection via get_model() factory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 07:19:29 +01:00
jpmschweitzerandClaude Opus 4.5 62eac3eb61 feat: integrate Paperless documents and volatile cache into Librarian
Build and Push / build (release) Successful in 54s
- Add Paperless document search to HybridRAG pipeline
- Add volatile cache (weather, forecast, news, stocks) to HybridRAG
- Add include_documents and include_volatile params to hybrid_search
- Add 📑 and  icons for document/volatile sources
- Update Librarian prompt with new data source awareness
- Fix Biographer routing: personal memory queries now route correctly
- Add location keywords to Steward pre-fetch logic

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 13:31:23 +01:00
jpmschweitzerandClaude Opus 4.5 ba195e401a fix: reduce Tatlock's excessive apologizing
Build and Push / build (release) Successful in 53s
Strengthened personality prompt to prevent unnecessary apologies after
successful Librarian delegations. Added explicit "do NOT apologize"
instructions to both system prompt and synthesis prompt.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 17:52:19 +01:00
jpmschweitzerandClaude Opus 4.5 51fd59ce92 fix: prevent Librarian from fabricating information
Add explicit instructions to the Librarian system prompt to never
invent data when tools fail or data sources are unavailable.

- Report what failed specifically
- Never provide placeholder or made-up data
- Better to return no information than fabricated information

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 10:28:13 +01:00
jpmschweitzerandClaude Opus 4.5 87f2926db2 feat: integrate tracing throughout request pipeline
Instrument the full request flow with trace spans for debugging:

- Wrap expert delegations (librarian/biographer/housekeeper) in spans
- Add orchestrate and synthesize spans to TatlockAgent
- Trace Steward analysis in preprocessing
- Start/end traces in response service with context management
- Simplify router by moving context handling to service layer
- Include tracing router in debug mode
- Remove benchmark recording from tool_tracking and steward service

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 10:26:37 +01:00
jpmschweitzerandClaude Opus 4.5 363ab378af feat: optimize Housekeeper for Mistral-Nemo tool calling
- Rewrite system prompt with negative constraints and step-by-step process
- Set temperature to 0.1 for deterministic tool calling
- Sort room groups to top of device list (address positional bias)
- Add [ROOM GROUP] marker in list_devices output
- Update tool docstrings with explicit entity_id= parameter examples
- Add optimization findings doc (experiment log: 0% → 100% success)
- Add test script for room group detection regression testing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 20:58:39 +01:00
jpmschweitzerandClaude Opus 4.5 e5d50dda77 fix: housekeeper API paths and entity hallucination prevention
Build and Push / build (release) Successful in 56s
- Update all client endpoints to use /housekeeping/ prefix
- Add critical rule requiring list_devices() before control actions
- Add housekeeping API spec documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-17 20:43:46 +01:00
jpmschweitzerandClaude Opus 4.5 9980e4764c fix: remove <think> wrappers from think messages
Build and Push / build (release) Successful in 1m49s
Messages in reasoning_content should be plain text, not wrapped
in <think> tags. Removed wrappers from:
- delegation.py household think messages
- orchestration.py status messages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 09:12:17 +01:00
jpmschweitzerandClaude Opus 4.5 fb54887c03 fix: handle HybridRAG keywords schema change
Build and Push / build (release) Successful in 52s
library-desk now returns keywords as dict with core_keywords field.
Client now handles both list and dict formats for backwards compat.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 00:32:36 +01:00
jpmschweitzerandClaude Opus 4.5 d5e5fc1ad8 fix: Ollama message sanitization and streaming think slugs
Build and Push / build (release) Successful in 52s
- Fix `invalid message content type: <nil>` error from Ollama
- Create TatlockOllamaProvider that sanitizes messages (null → "")
- Update all agents to use sanitized provider
- Fix repeating think messages by adding ReasoningSummaryDone signal

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 00:19:03 +01:00
jpmschweitzerandClaude Opus 4.5 74f47097c2 fix: complete web search integration with query enrichment
Build and Push / build (release) Successful in 52s
Fixes several issues with the web search migration to Librarian:

- Update Steward routing guidelines for web search/weather → Librarian
- Register search_web, read_url, read_urls_batch tools with Librarian agent
- Update Librarian system prompt with web search documentation
- Fix query enrichment not being passed to delegations (location context)
- Add URL reading keywords to RESEARCH action type detection

Weather queries now automatically include user's stored location from
the Biographer, enabling location-aware search results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 22:51:18 +01:00
jpmschweitzerandClaude Opus 4.5 100ebeae52 feat: migrate web search from tatlock_core to Librarian
Move web search functionality to The Librarian agent, integrating with
the library-desk /rag/search endpoint for enhanced search capabilities.

Changes:
- Add search_web, read_url, read_urls_batch tools to Librarian
- Add WebSearchResult, ContentExtractionResult models to client
- Add search_web, extract_content, extract_content_batch client methods
- Update Librarian capability with web/url/internet domains
- Remove search_web from tatlock_core tools and toolset
- Update Tatlock system prompt to delegate web search to Librarian
- Add comprehensive unit tests for new Librarian tools
- Clean up legacy src/agents/tools.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 18:30:16 +01:00
jpmschweitzerandClaude Opus 4.5 49f0da8068 feat: two-phase execution, think slugs, query enrichment (v1.6.0)
Build and Push / build (release) Successful in 1m14s
Two-Phase Tatlock Execution:
- orchestrate_tool_calls() for Phase 1 coordination
- synthesize_from_results() for Phase 2 butler-toned synthesis
- Guarantees butler personality in all responses

Automatic Think Slugs:
- Deterministic butler-perspective messages during expert delegation
- ActionType enum: RETRIEVE, RESEARCH, CREATE, CONTROL, RECORD
- HOUSEHOLD_THINK_MESSAGES mapping for all experts
- Streaming delegation wrappers with automatic think messages

Steward Query Enrichment:
- Auto-fill user context (location, timezone) when not specified
- _build_enriched_query() with regex word boundary matching
- enriched_query field in StewardRecommendation schema

Documentation:
- ORCHESTRATION_SCENARIOS.md rewritten with Mermaid diagrams
- New Housekeeper and Biographer scenarios
- TESTING_IMPROVEMENTS.md for future LLM testing patterns

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 14:00:32 +01:00
jpmschweitzerandClaude Opus 4.5 a1b8fe46e8 feat: add The Housekeeper agent for home automation
Implements The Housekeeper, a new expert agent for home automation
following the Librarian pattern. Communicates with core-api service
which wraps Home Assistant REST API.

New agent features:
- CoreAPIClient with 13 home automation methods
- 13 tools: list_areas, list_devices, get_device_state, turn_on,
  turn_off, toggle, list_scenes, activate_scene, list_scripts,
  run_script, list_automations, toggle_automation, get_history
- PydanticAI agent with butler-friendly system prompt
- HouseholdCapability registration for Steward coordination
- delegate_to_housekeeper() wrapper for orchestration

Also includes:
- Dev port changed from 8123 to 8777 (avoids Home Assistant conflict)
- Config: CORE_API_HOST, CORE_API_KEY, CORE_API_TIMEOUT
- 44 unit tests for client and capability

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 10:24:35 +01:00
jpmschweitzerandClaude Opus 4.5 64cad4500a feat: environment-aware config, direct delegation, E2E test suite (v1.4.0)
Build and Push / build (release) Successful in 52s
### Added
- Environment-aware configuration:
  - Auto-selected logging (DEBUG for dev, WARNING for prod)
  - Auto-selected default user (llm_tester for dev isolation)
  - User context logging at request entry
- Direct delegation bypass:
  - Pure memory/librarian requests skip Tatlock LLM
  - Reduces latency for memory-only requests
- Text-based delegation fallback:
  - Parse [DELEGATE:agent] patterns from LLM output
  - Sequential and parallel execution support
- Comprehensive E2E test suite:
  - 22 orchestration tests with QdrantVerifier
  - assert_llm_behavior() for flexible pattern matching
  - Tests for memory, delegation, isolation, scenarios

### Fixed
- Unit test mocks for streaming (async generator)
- Temporal context handling in tests
- LLM non-determinism with pytest.xfail()
- Streaming test timeouts increased

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 21:19:47 +01:00
jpmschweitzerandClaude Opus 4.5 9d7ce399c8 fix(memory): biographer tool type hints for Ollama (v1.3.2)
Build and Push / build (release) Successful in 51s
- Change `str | None` to `str` with empty default for memory_type
- Remove `keywords` parameter from store_insight (auto-generated anyway)
- Ollama's OpenAI API doesn't handle union types with None properly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 15:04:10 +01:00
jpmschweitzerandClaude Opus 4.5 7426dd1ac3 feat: add Phase F.2 - The Biographer (memory agent)
Add The Biographer household member for user memory management:

Memory Service (direct access layer):
- src/core/memory_service.py for fast, LLM-free lookups
- Profile, preference, and fact management
- Session context with Redis caching
- Steward integration via prefetch_context()

The Biographer Agent:
- src/agents/biographer/ package with PydanticAI agent
- Discreet chronicler personality for privacy
- Tools: recall_semantic, list_memories, store_insight,
  update_profile, update_preference, forget_memory
- Registered with Household Registry on startup

Steward Integration:
- Memory context pre-fetch during analysis
- Profile/preferences included in Butler note
- Keyword-based context determination

Also includes:
- delegate_to_biographer() wrapper
- 34 new tests (capability + memory service)
- Version bump to 1.2.0

Documentation cleanup:
- Removed obsolete PHASE2_COMPLETE.md, PHASE2_PLAN.md
- Removed docs/library-desk-requirements.md
- Moved ORCHESTRATION_SCENARIOS.md to project root

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 19:20:18 +01:00
jpmschweitzerandClaude Opus 4.5 4c6ac89808 feat: add Phase F.1 memory infrastructure
Add multi-tenancy support and memory storage infrastructure:

- Add ContextVar-based request context (src/core/context.py)
  - Async-safe user/conversation tracking via contextvars
  - RequestContext manager for clean setup/teardown
  - get_user(), get_conversation_id() helpers

- Add multi-tenancy utilities (src/core/multi_tenancy.py)
  - User ID sanitization for collection/key names
  - get_memory_collection_name(), get_session_key() helpers

- Add Ollama embedding client (src/core/embeddings.py)
  - nomic-embed-text model (768 dimensions)
  - embed(), embed_batch(), health_check() methods

- Add Qdrant client wrapper (src/core/qdrant.py)
  - Per-user collection pattern: memories_{user}
  - upsert_memory(), search_memories(), delete_memory()
  - Type-based filtering support

- Add Redis memory cache (src/core/memory_cache.py)
  - Session context with 24h TTL
  - Recent entities tracking
  - Separate from benchmarks (db=2)

- Update config with memory settings
  - QDRANT_HOST, QDRANT_PORT, QDRANT_EMBEDDING_DIM
  - OLLAMA_EMBEDDING_MODEL
  - REDIS_MEMORY_DB, REDIS_MEMORY_TTL_HOURS

- Add user field to ResponseRequest (OpenAI standard)
- Set context in router, reset in finally block
- Update librarian client to use get_user() (12 methods)

All 333 unit tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 17:27:51 +01:00
jpmschweitzerandClaude Opus 4.5 1e4bba2422 feat: add sequential multi-expert execution to orchestration
Adds multi-expert coordination infrastructure:
- ExecutionMode enum (SEQUENTIAL, PARALLEL)
- MultiExpertResult dataclass for aggregating results
- execute_sequential(): Tasks run one after another
- execute_parallel(): Tasks run concurrently via asyncio.gather
- orchestrate_multi_expert(): Streaming think updates during multi-expert work

Supports:
- Stop-on-failure mode for sequential execution
- Partial failure handling (some succeed, some fail)
- Result aggregation with combined output formatting
- Exception handling in parallel execution

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 13:22:11 +01:00
jpmschweitzerandClaude Opus 4.5 1970751b2f feat: add orchestration loop with think update streaming
Creates orchestration module for multi-expert coordination:
- parse_delegation_from_steward_note(): Extracts delegation task
- execute_delegation(): Routes to appropriate expert agent
- orchestrate_with_think_updates(): Streams <think> updates around
  delegation calls while using run() internally

This enables real-time user feedback while avoiding Ollama's
streaming+tool call bugs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 12:59:20 +01:00
jpmschweitzerandClaude Opus 4.5 6cc0bd78b2 feat: update Steward prompt for clearer delegation instructions
Updates Steward's output format to structured delegation format:
- DELEGATE: [capability] to [action] [task]
- REASON: [explanation]
- COMPLEXITY: [simple/moderate/complex]
- CONTEXT: [relevant history or "none"]

Also adds guidance for conversation memory queries (handled by
Tatlock directly, not delegated to Librarian).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 12:46:51 +01:00
jpmschweitzerandClaude Opus 4.5 54b6fcd7cc feat: add DelegationTask dataclass and delegate_to_librarian wrapper
Introduces agent-as-tool pattern infrastructure:
- DelegationTask: Structured representation of expert work
- DelegationResult: Typed result from expert delegation
- delegate_to_librarian(): Wrapper for Librarian agent calls

This implements PydanticAI's recommended delegation pattern where
parent agents call child agents via tool wrappers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 11:19:35 +01:00
jpmschweitzerandClaude Opus 4.5 b5ee1f3e44 fix: use run() instead of run_stream() for scoped tools to avoid Ollama 400 bug
PydanticAI + Ollama streaming with tool calls has known issues:
- Issue #1292: Streaming stops after tool call due to empty TextPart
- Issue #2256: Empty text part causes run to end prematurely

This change uses run() for the actual tool execution while still
yielding the response in chunks to maintain the streaming UX.
The orchestration loop can emit <think> updates between await calls.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 10:59:58 +01:00
jpmschweitzerandClaude Opus 4.5 4efa717796 fix: improve Steward delegation instructions for Librarian
Build and Push / build (release) Successful in 10s
- Update Librarian capability description to highlight CREATE/UPDATE/SEARCH
- Add specific Steward guidelines for wiki creation, updates, and research
- Add dynamic time injection to user prompts for temporal awareness
- Expand domains to include 'create', 'write', 'update'
- Update test to match new capability description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:55:17 +01:00
jpmschweitzerandClaude Opus 4.5 7ec6e03c65 feat(phase3): add multi-agent coordination engine
- CoordinationEngine for task orchestration between agents
- Routing tasks to appropriate expert agents
- Sequential and parallel execution support
- Result aggregation from multiple agents
- Graceful error handling and degradation
- Streaming delegation support
- Convenience functions: delegate_to_librarian(), delegate_to_librarian_stream()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:29:03 +01:00
jpmschweitzerandClaude Opus 4.5 f6f37b341b feat(phase3): add The Librarian agent with library-desk integration
Library-Desk API Client:
- Async HTTP client with httpx for library-desk API
- HybridRAG search (vector + graph + web)
- Wiki operations (search, get, list, create, update)
- Smart page creation with HybridRAG research
- Semantic vector search and knowledge graph queries
- Dossier browsing and health checks

Librarian Tools (11 total):
- Research: hybrid_search, search_wiki, get_wiki_page, semantic_search
- Browse: list_dossiers, get_dossier_pages, explore_knowledge_graph
- Graph: find_related_entities
- Write: create_wiki_page, update_wiki_page, smart_create_wiki_page

Agent:
- PydanticAI agent with research assistant personality
- System prompt with research and writing workflows
- Streaming support via run_librarian_stream()

Capability:
- LIBRARIAN_CAPABILITY definition for Household Registry
- Automatic registration on startup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:27:09 +01:00
jpmschweitzerandClaude Opus 4.5 92c0d5d770 feat(phase3): add agent communication protocol
- AgentRequest/AgentResponse for standardized inter-agent communication
- DelegationIntent for routing tasks to expert agents
- CoordinationResult for aggregated multi-agent results
- DelegationReason enum (domain expertise, tool access, etc.)
- Error types: AgentError, AgentTimeoutError, AgentUnavailableError

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:26:52 +01:00
jpmschweitzerandClaude Sonnet 4.5 4d109100fe fix: implement proper streaming with PydanticAI delta mode
Fix streaming issues that caused text repetition and broken tool execution
in Open WebUI. Implements real LLM streaming using PydanticAI's run_stream()
with delta=True instead of artificial word-by-word chunking.

**Fixed:**
- Text repetition in streaming output (was accumulating instead of deltas)
- Broken tool execution (tools now execute properly in streaming mode)
- Invalid 'thinking' parameter in ReasoningOutputItem schema

**Changes:**
- Add run_with_scoped_tools_stream() method to TatlockAgent
  - Uses PydanticAI's run_stream() with delta=True for real deltas
  - Properly streams LLM output with tool execution
- Update StreamingCoordinator.stream_response_with_steward()
  - Uses new streaming method instead of fake word-by-word streaming
  - Removes invalid thinking parameter from ReasoningOutputItem
- All streaming now uses actual LLM deltas, not accumulated text

Resolves streaming issues reported in Open WebUI where responses showed
repetitive text and tool calls appeared as raw JSON instead of executed results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:39:52 +01:00
jpmschweitzerandClaude Sonnet 4.5 6eed5f4d13 feat: implement Phase 2 two-tier architecture with Steward
Add comprehensive two-tier architecture where Steward analyzes requests
and Tatlock executes with scoped tools. Includes full infrastructure for
request preprocessing, tool tracking, benchmarking, and streaming.

**Added:**
- Steward agent for request analysis and capability recommendation
- Household Registry for centralized capability management
- Request preprocessing pipeline (Steward → Tatlock flow)
- Tool usage tracking and benchmarking system
- Streaming transparency (Steward reasoning visible in streams)
- Structured logging with operation timing
- Redis benchmark storage with 30-day expiry
- Benchmark analysis CLI tools

**Infrastructure:**
- src/agents/steward/ - Steward agent implementation
- src/agents/tatlock_core/ - Tatlock capability domain
- src/core/preprocessing.py - Request preprocessing pipeline
- src/core/tool_tracking.py - Tool call tracking
- src/core/benchmarks.py - Benchmark recording system
- src/core/household_registry.py - Capability registry
- src/core/startup.py - Application startup coordination
- src/core/logging_config.py - Structured logging setup

**Integration:**
- Responses API uses Steward for Tatlock requests
- Chat Completions wraps Responses API for OpenAI compatibility
- Streaming coordinator supports Steward + Tatlock flow
- Tool scoping per request based on Steward recommendations

**Testing:**
- Integration tests for Steward-Tatlock flow
- Benchmark and registry unit tests
- Steward streaming tests

See PHASE2_PLAN.md and PHASE2_COMPLETE.md for detailed documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 15:39:20 +01:00
jpmschweitzerandClaude Sonnet 4.5 38120696bf feat: add conversation history and tool call logging
Add two major features to enhance Tatlock's capabilities:

1. Conversation History Support:
   - Convert OpenAI-format messages to PydanticAI ModelRequest/ModelResponse
   - Pass full conversation context via message_history parameter
   - Filter empty messages to prevent Ollama errors
   - Add debug logging for message history construction
   - Tatlock now remembers previous turns in multi-turn conversations

2. Tool Call Logging:
   - Implement ToolCallTracker dependency for per-request tracking
   - Tools log usage via RunContext deps parameter
   - Web search: "🔍 Searching for: 'query'"
   - Calculator: "🧮 Calculating: expression"
   - Date/time: "🕐 Calculating date offset: description"
   - Tool logs appear in reasoning output as <think> tags in Open WebUI

Both features improve user experience by maintaining conversation context
and providing transparency into tool usage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 01:13:19 +01:00
jpmschweitzer 67481515cc feat: integrate Tatlock agent with PydanticAI and Ollama
Convert Tatlock from mock to real PydanticAI agent:
- Connect to Ollama backend (mistral-nemo:latest)
- British butler personality with research-oriented mindset
- Lazy initialization pattern for better testability
- Register permanent tools (calculator, date/time, search)
- Streaming response support with reasoning output
- Error handling for PydanticAI exceptions
- Update registry tests for tools capability
- Add integration test for streaming functionality
2025-12-07 00:12:39 +01:00
jpmschweitzer f3e2681a6c feat: implement permanent tools (calculator, date/time, search)
Add three permanent tools for Tatlock agent:
- Calculator: Safe math expression evaluation (arithmetic, algebra, trig, log)
- Date/Time toolkit: Current time, relative dates, time differences
- Web Search: SearXNG integration for privacy-preserving search

Tools use PydanticAI @agent.tool decorator pattern with:
- Clear docstrings visible to LLM
- Error handling with string-based messages
- Async support for I/O operations (web search)
- 26 comprehensive tool tests
2025-12-07 00:10:49 +01:00