search_wiki printed ordinally numbered results with no page ID while
get_wiki_page demands 'the page ID from search results' - the model
passed the list position (page 1) and 404'd. Results now carry
page_id and drop the ordinals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gemma thinking-only assistant turns carry content: null with no
tool_calls, slipping past the tool-call-only sanitizer and 400ing the
whole agent run ('invalid message content type: <nil>'). Null content is
now blanked for any role.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The scholarly persona prompt reproduced the exact pathology
TATLOCK_ORCHESTRATION_PROMPT fixed for the butler: gemma4 answered in
character ('please provide your request') without calling a single tool.
The research phase now uses a tool-discipline prompt; Tatlock's synthesis
supplies the voice. Anti-fabrication rules kept verbatim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-coordination-removal sweep: the coordination wire protocol
(AgentRequest, AgentResponse, DelegationIntent, CoordinationResult,
DelegationReason, TaskComplexity, ToolCallRecord, AgentTimeoutError,
AgentUnavailableError, DelegationError) had zero importers left in
src/ - only its own test module. AgentError stays (raised by
run_librarian, mapped to user-safe failures by delegation.py).
Also drops the stale coordination.py line from the README tree.
Import-cycle sanity: python -c 'import src.main' passes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
Phase A review minors:
- Coverage note: source_status (when present) is now used exclusively;
the source_counts-absence fallback only considers the optional legs
the request explicitly enabled (web/documents/volatile). library-desk
computes source_counts from the final top-N fused results only, so
absence of the always-on vector/graph legs is normal ranking behavior
- the old heuristic warned on virtually every healthy search
- update_wiki_page: the empty-list tags sentinel (leave unchanged) made
clearing all tags impossible; pass exactly ["__CLEAR__"] to send an
empty tag list, documented in the docstring for the local model
- Text-delegation parallel fallback: zip(..., strict=True) with an
explicit count-mismatch guard so results can never be silently
attributed to the wrong agent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
One delegation implementation remains (src/agents/delegation.py).
Removed, after verifying zero live importers post-Phase-A/B:
- src/agents/coordination.py: CoordinationEngine, duplicate
delegate_to_librarian, AGENT_EXECUTORS/AGENT_STREAM_EXECUTORS
(only importer was its own test module)
- run_librarian_stream: documented-broken path (Ollama streaming +
tool call bug, PydanticAI #1292/#2256), only called by the deleted
coordination engine
- stream_delegate_to_* wrappers + STREAMING_DELEGATION_WRAPPERS and
the never-parsed __DELEGATION_RESULT__ marker in delegation.py
- HouseholdRegistry.get_streaming_delegation_tools() (no callers)
- tests/agents/test_coordination.py and the wrapper/stream tests
Note: the STREAMING_DELEGATION_WRAPPERS import in
src/responses/streaming.py was already removed by Phase A (7ce1c1a);
nothing to delete there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
LibraryDeskClient._resolve_user only enforced non-empty: an explicit
user argument to any tenant-scoped method bypassed tatlock's tenant
guard entirely and went straight to library-desk, and padded values
were sent un-stripped on the wire.
Route the explicit-arg path through the same apply_tenant_guard() used
by context resolution and strip whitespace before the empty check, so
a non-production environment can never send the production tenant (or
a sanitization-collision variant) to library-desk, regardless of how
the user was supplied. Defense in depth - no in-repo caller passes an
explicit user today.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The request-level tenant guard compared the raw user string exactly
(user == PRODUCTION_TENANT), but all local namespaces (Qdrant
collections, Redis keys) are derived through sanitize_user_id(), which
lowercases and strips/maps punctuation. Case or punctuation variants
("JPMSchweitzer", "jpmschweitzer.", " jpmschweitzer") therefore passed
the guard yet resolved to the production namespaces, letting a dev
instance on the shared services read/write production tenant data.
- context.py: compare sanitize_user_id(user) against the sanitized
production tenant; expose the guard as public apply_tenant_guard()
- config.py: startup refusal validator uses the same sanitized
comparison, so a colliding DEFAULT_USER refuses startup loudly
instead of relying on the allowlist fallback
- tests: variant matrix at both config and request-context level,
plus a non-colliding passthrough case
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Session-scoped autouse guard in tests/conftest.py refuses to run any
test (pytest.exit, returncode 1) when the effective tenant resolves
to the production tenant jpmschweitzer - the same guard library-desk
applies on its side. _initialize_app now depends on the guard so the
refusal happens before any initialization.
Suite-level assertions pin that the live session runs under the
llm_tester namespaces: Qdrant memories_llm_tester collection and
Redis session:llm_tester:* keys. The biographer/memory unit tests
already run fully mocked (no shared-service writes); the e2e
isolation tests already used llm_tester - their constants now derive
from the shared TEST_TENANT/PRODUCTION_TENANT config constants so a
drift fails loudly instead of silently splitting.
Verified: ENVIRONMENT=production pytest run exits 1 with the TENANT
GUARD message and zero tests executed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Library-desk is removing its server-side default user, so a request
without an explicit tenant will 422 after its next deploy:
- New client-level _resolve_user() resolves the tenant (explicit arg
or request context) and raises ValueError on an empty/whitespace
value BEFORE any bytes hit the wire; all 15 tenant-scoped methods
use it
- extract_content / extract_content_batch now accept and send the
user (query param), matching the rest of the API surface
- search_web no longer falls back to a phantom "tatlock-librarian"
tenant; it sends the resolved user
- health_check stays user-less (public, not tenant-scoped)
Tests: parametrized sweep pins the wire contract (user present in
params or payload) for every tenant-scoped method, for both context
and explicit users; empty-tenant calls are asserted to fail without
any HTTP call; the recorded-fixture hybrid contract test now pins
user as an explicit query param.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Non-production environments (development/testing) now force the
effective tenant to the reserved test tenant "llm_tester" (or a
test_-prefixed override) regardless of DEFAULT_USER misconfiguration:
- Config.effective_default_user only honors DEFAULT_USER outside
production when it is llm_tester or test_-prefixed; anything else
is forced to llm_tester (tenant_forced flags the override)
- Config refuses startup (validation error) when a non-production
environment is explicitly configured with the production tenant
jpmschweitzer
- get_user() applies the same guard at request-context resolution,
so an explicit request for the production tenant in dev/test is
forced to llm_tester with a warning log
- initialize_application() emits one loud startup log line
(tenant_guard_active / tenant_guard_production) stating the
effective tenant
Unit tests cover the dev/test/prod x default/explicit-user matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- delegate_to_* now receives a trimmed conversation history (last ~6
turns, 500 chars/turn) as context on both live direct-delegation
paths (streaming and steward non-streaming), via new
build_delegation_context helper
- _stream_direct_delegation restructured as an async generator: the
butler 'start' think message streams BEFORE the expert runs and the
success/error message right after it finishes, instead of all
messages arriving after the research completed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ollama's OpenAI-compatible API mishandles anyOf[X, null] parameter
schemas. update_wiki_page (content/title/tags/description) and
smart_create_wiki_page (path) now use empty-string/empty-list
sentinels translated to None inside the tool, following the
biographer pattern from 9d7ce39.
Adds a snapshot test that walks every registered librarian tool's
emitted JSON schema and fails on any anyOf[..., null].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- 2-attempt short-backoff retry for GETs and the read-only
POST /query/* and /rag/search endpoints only; wiki writes are never
retried (duplicate-page risk)
- honor the defined-but-ignored LIBRARY_DESK_TIMEOUT config instead of
hardcoded 60s/30s per-call values
- hold ONE shared httpx.AsyncClient per librarian run via
library_client_session (contextvar), instead of constructing a
client per tool call; nested sessions are no-ops and custom targets
still get their own client
- read tools raise ModelRetry on transient HTTP errors (transport
errors, 5xx, 429) so Agent(retries=2) engages; write tools keep
returning safe failure messages
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- add LIBRARIAN_TIMEOUT config (default 180s) and enforce it with
asyncio.wait_for inside delegate_to_librarian, covering the live
paths (steward direct delegation and SSE streaming) that had no cap
- timeouts fail honestly: success=False with a curated butler sentence,
detail in logs
- set an explicit timeout on TatlockOllamaProvider's AsyncOpenAI client
from OLLAMA_TIMEOUT instead of the SDK default (~600s per LLM call)
- remove the contradictory unused 60s default from
AgentRequest.timeout_seconds; coordination falls back to the
configured budget
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- parse source_counts into HybridRAGResponse and additively parse the
shared-contract source_status/degraded fields when present (absence
tolerated, so deploy order between tatlock and library-desk never
matters)
- hybrid_search appends a one-line coverage note when a leg reported
'failed' (or degraded is set), falling back to inferring silent legs
from source_counts on older library-desk versions
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- run_librarian / run_librarian_stream raise AgentError instead of
returning/yielding error text as normal output; detail stays in logs
- delegate_to_* wrappers now put a curated butler-toned sentence in
DelegationResult.output on failure and never expose str(e), so
streaming's error branch is reachable and honest
- _execute_single_delegation propagates success; direct delegation only
records delegate_to_* as called when the expert actually succeeded
- librarian tools return user-safe messages instead of
'Error searching: {e}' strings that leaked internal URLs into
synthesis; coordination stream errors are curated as well
- ruff cleanups (TYPE_CHECKING forward refs, B904, unused locals) in
the touched files to keep them lint-clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The client parsed field names the live library-desk service never
returns, so every result rendered as "unknown (score: 0.00)":
- source_type/sources -> source + sources (icons key off sources values)
- rrf_score -> score
- context -> formatted_context
- related_dossiers are per-result; top level aggregates unique titles
- synonyms live inside the keywords dict as a {term: [synonyms]} map
Also stop sending zero limits (service 422s on limit < 1); disabled
legs now rely on the enable_* flags with limits clamped to >= 1.
Adds a recorded live response as a fixture plus contract tests that
pin the mapping (non-unknown sources, non-zero scores, icon coverage).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mechanical Optional[X] -> X | None and f-string cleanups so subsequent
librarian changes lint clean against the dirty baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/contracts sends the raw requests the code sends to Ollama (native API
and OpenAI-compat tool calling), Anthropic (including the pinned Sonnet 5
temperature-rejection contract), Qdrant, SearXNG, library-desk, and Redis.
Unreachable services skip; wrong response shapes fail. Run via
make test-contracts; excluded from the unit suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With the butler persona prompt attached, gemma4 reasons about calling the
calculator and then answers from memory with a different wrong product every
run; tool_choice=required via extra_body is advisory at best on Ollama's
OpenAI-compat layer. orchestrate_tool_calls() now uses a terse
TATLOCK_ORCHESTRATION_PROMPT; synthesize_from_results() keeps the persona,
so the user-visible voice is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rolls back the claudification backend preference: PREFER_CLOUD_BACKEND now
defaults to false, resolve_backend() picks Ollama first and uses Claude when
explicitly preferred or when the new Ollama startup health check fails. The
Steward retries mid-request failures on the other backend in both directions.
Also hardens the fallback itself: Anthropic SDK imports are lazy so a broken
anthropic package degrades to Ollama-only instead of crashing at import time
(root cause of the production outage since April), anthropic is pinned to a
pydantic-ai-1.27-compatible range, ANTHROPIC_MODEL defaults to claude-sonnet-5
(sonnet-4-20250514 retired 2026-06-15), sampling parameters are stripped from
Claude calls (Sonnet 5 rejects them), and the Steward timeout is configurable
(STEWARD_TIMEOUT, default 60s) since gemma4 needs ~35s warm for analysis.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gemma4:e2b has native function calling with dedicated tool tokens,
achieving 100% tool selection accuracy in benchmarks vs 67% for
mistral-nemo-large, with 5-8x faster response times (2-4s vs 15-20s)
and lower VRAM usage (8GB vs 9.2GB).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests are run locally before tagging. Removes the slow CI test job
and its dependency gates on release and build jobs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
exclude_none was too aggressive — it stripped finish_reason: null from
intermediate chunks (which OpenAI includes). exclude_unset correctly
omits only fields never passed to the constructor (like reasoning_content
on content-only chunks) while preserving explicitly-set finish_reason: null.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Documents architecture, key file locations, test setup, and critical
gotchas discovered during development (ASGITransport lifespan, async
scope mismatch, Ollama fallback behavior, missing benchmark store).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove references to unimplemented get_benchmark_store from steward and
tool tracking tests
- Fix steward test fixture calling async initialize_application synchronously
by using sync register_household_members instead
- Rewrite tool tracking tests to assert actual logging behavior
- Change unit test fixture model from Tatlock to lorem-tester so unit tests
don't require external services
- Add session-scoped _initialize_app fixture to run Claude health check,
ensuring integration tests use Claude instead of falling back to Ollama
- Increase integration test timeouts from 30s to 120s to match OLLAMA_TIMEOUT
- Add Steward reasoning as ReasoningOutputItem in create_response_with_steward
so <think> tags appear in chat completion responses
- Add test_tatlock_ollama_fallback to verify Ollama fallback path works
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OpenAI's API omits null fields in streaming chunks, but Tatlock was
including them (content: null, reasoning_content: null). This caused
parsing issues in Open WebUI's streaming handler.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
sse_starlette's EventSourceResponse added \r\n line endings that
Open WebUI couldn't parse. Switched to plain StreamingResponse with
manual SSE formatting matching OpenAI's exact format.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Steward's internal routing analysis (DELEGATE, COMPLEXITY, etc.)
was being exposed in <think> blocks. This is implementation detail,
not useful reasoning for the user.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
PydanticAI handles tool_choice natively for Anthropic. The extra_body
hack caused an infinite tool call loop where Claude kept calling the
same tool because tool_choice was forced to "any".
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed workflow trigger from release:published to push:tags:v[0-9]*
so that pushing a version tag triggers the build pipeline.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
AnthropicModel doesn't accept api_key directly; it must be passed
through an AnthropicProvider instance.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All agents now prefer Claude API when ANTHROPIC_API_KEY is configured,
with automatic fallback to Ollama when offline or unconfigured. New
src/anthropic/ module provides model selection via get_model() factory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Paperless document search to HybridRAG pipeline
- Add volatile cache (weather, forecast, news, stocks) to HybridRAG
- Add include_documents and include_volatile params to hybrid_search
- Add 📑 and ⚡ icons for document/volatile sources
- Update Librarian prompt with new data source awareness
- Fix Biographer routing: personal memory queries now route correctly
- Add location keywords to Steward pre-fetch logic
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Strengthened personality prompt to prevent unnecessary apologies after
successful Librarian delegations. Added explicit "do NOT apologize"
instructions to both system prompt and synthesis prompt.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change uvicorn from localhost to 0.0.0.0 to allow connections
from other machines on the network.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add explicit instructions to the Librarian system prompt to never
invent data when tools fail or data sources are unavailable.
- Report what failed specifically
- Never provide placeholder or made-up data
- Better to return no information than fabricated information
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instrument the full request flow with trace spans for debugging:
- Wrap expert delegations (librarian/biographer/housekeeper) in spans
- Add orchestrate and synthesize spans to TatlockAgent
- Trace Steward analysis in preprocessing
- Start/end traces in response service with context management
- Simplify router by moving context handling to service layer
- Include tracing router in debug mode
- Remove benchmark recording from tool_tracking and steward service
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove the Redis-backed performance benchmarking in favor of the new
lightweight file-based tracing system which provides better debugging
capabilities for local development.
- Delete src/core/benchmarks.py
- Remove ENABLE_BENCHMARKS, REDIS_BENCHMARK_DB, redis_url from config
- Update memory_cache comment (now uses DB 1)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds JSON-based tracing system for local development that captures
the full request flow through Tatlock's multi-agent architecture.
- Trace/Span dataclasses with automatic timing and nesting
- Context-var based propagation for async-safe tracing
- trace_span async context manager for clean instrumentation
- Traces written to logs/traces/{trace_id}.json
- REST API for listing and retrieving traces (/traces)
- Standalone HTML viewer with timeline visualization
Enabled via DEBUG=true environment variable.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
