Compare commits
+11
-9
@@ -8,17 +8,19 @@ API_HOST=0.0.0.0
|
||||
API_PORT=8000
|
||||
API_PREFIX=/v1
|
||||
|
||||
# Anthropic Configuration (Claude - preferred backend)
|
||||
# Set ANTHROPIC_API_KEY to enable Claude as the default backend
|
||||
# Without an API key, Tatlock uses Ollama exclusively
|
||||
# ANTHROPIC_API_KEY=sk-ant-api03-your-key-here
|
||||
ANTHROPIC_MODEL=claude-sonnet-4-20250514
|
||||
PREFER_CLOUD_BACKEND=true
|
||||
|
||||
# Ollama Configuration (local fallback when Claude unavailable)
|
||||
# Ollama Configuration (local - primary backend)
|
||||
OLLAMA_HOST=http://localhost:11434
|
||||
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
|
||||
OLLAMA_DEFAULT_MODEL=gemma4:e2b
|
||||
OLLAMA_TIMEOUT=120
|
||||
STEWARD_TIMEOUT=60
|
||||
|
||||
# Anthropic Configuration (Claude - cloud fallback)
|
||||
# Set ANTHROPIC_API_KEY to keep the Claude fallback available: it is used
|
||||
# automatically when Ollama is down, or exclusively when PREFER_CLOUD_BACKEND=true
|
||||
# Without an API key, Tatlock uses Ollama only
|
||||
# ANTHROPIC_API_KEY=sk-ant-api03-your-key-here
|
||||
ANTHROPIC_MODEL=claude-sonnet-5
|
||||
PREFER_CLOUD_BACKEND=false
|
||||
|
||||
# SearXNG Configuration
|
||||
SEARXNG_HOST=http://localhost:8087
|
||||
|
||||
@@ -6,24 +6,7 @@ on:
|
||||
- 'v[0-9]*'
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install dependencies
|
||||
run: make setup
|
||||
|
||||
- name: Run unit tests
|
||||
run: make test
|
||||
|
||||
release:
|
||||
needs: test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Create Gitea Release
|
||||
@@ -35,7 +18,6 @@ jobs:
|
||||
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
|
||||
|
||||
build:
|
||||
needs: test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -43,7 +25,7 @@ jobs:
|
||||
- name: Login to Gitea Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: git.schweitz.internal
|
||||
registry: git.schweitz.net
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
|
||||
@@ -55,8 +37,8 @@ jobs:
|
||||
provenance: false
|
||||
sbom: false
|
||||
tags: |
|
||||
git.schweitz.internal/jpmschweitzer/tatlock:latest
|
||||
git.schweitz.internal/jpmschweitzer/tatlock:${{ github.ref_name }}
|
||||
git.schweitz.net/jpmschweitzer/tatlock:latest
|
||||
git.schweitz.net/jpmschweitzer/tatlock:${{ github.ref_name }}
|
||||
|
||||
- name: Trigger Watchtower update
|
||||
if: success()
|
||||
|
||||
@@ -7,6 +7,77 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.4.2] - 2026-07-19
|
||||
|
||||
### Fixed
|
||||
|
||||
- Container crash-loop on fresh builds: cap `opentelemetry-api` below 1.44,
|
||||
which removed the private `_events` module that pydantic-ai 1.27 imports
|
||||
|
||||
## [2.4.1] - 2026-07-19
|
||||
|
||||
### Changed
|
||||
|
||||
- **Container-name network defaults** - `SEARXNG_HOST`, `LIBRARY_DESK_HOST`, and `CORE_API_HOST` now default to docker container names on the docker-dataplane network (`http://searxng:8080`, `http://library-desk:8089`, `http://core-api:8083`) instead of host `localhost` ports, ahead of the loopback port rebinding; this also fixes `CORE_API_HOST` pointing at port 8090 (the Scheduler's host port) rather than Core-API's 8083. `scripts/test_housekeeper.sh` now reaches Core-API via `localhost:8083` instead of the LAN IP. Local development against host-published ports still works via `.env` overrides
|
||||
|
||||
## [2.4.0] - 2026-07-14
|
||||
|
||||
### Removed
|
||||
|
||||
- **Dead delegation stack** - deleted the duplicate, never-wired coordination layer so exactly ONE delegation implementation remains (`src/agents/delegation.py`): `src/agents/coordination.py` (`CoordinationEngine`, its own `delegate_to_librarian`, `AGENT_EXECUTORS`/`AGENT_STREAM_EXECUTORS`), the broken-by-design `run_librarian_stream` path it used (Ollama streaming + tool call bug), the `stream_delegate_to_*` wrappers with their never-parsed `__DELEGATION_RESULT__` marker, and `HouseholdRegistry.get_streaming_delegation_tools()` (no callers)
|
||||
- **Orphaned agent protocol models** - `src/agents/protocol.py` now contains only the live `AgentError`; the coordination wire protocol it carried (`AgentRequest`, `AgentResponse`, `DelegationIntent`, `CoordinationResult`, `DelegationReason`, `TaskComplexity`, `ToolCallRecord`, `AgentTimeoutError`, `AgentUnavailableError`, `DelegationError`) had no importer left outside its own tests after the coordination stack removal
|
||||
|
||||
### Added
|
||||
|
||||
- **Test-suite tenant guard** - `tests/conftest.py` hard-fails the whole pytest session (exit code 1, zero tests run) if the effective tenant resolves to the production tenant `jpmschweitzer`, mirroring the guard library-desk applies on its side. Suite-level assertions pin that the session runs under `llm_tester` namespaces (Qdrant `memories_llm_tester`, Redis `session:llm_tester:*`), and the e2e isolation constants now derive from the shared `TEST_TENANT`/`PRODUCTION_TENANT` config constants instead of string literals
|
||||
- **Explicit tenant on every library-desk request** - the librarian client now resolves and sends the `user` parameter explicitly on every request (library-desk is removing its server-side default; a missing user would 422). The content extraction endpoints now carry the tenant too, `search_web` no longer falls back to a phantom `tatlock-librarian` user, and a client-level assertion rejects an empty/whitespace tenant before any bytes hit the wire. A parametrized sweep pins the wire contract for all 15 tenant-scoped client methods
|
||||
- **Tenant isolation guard** - non-production environments (development/testing) now FORCE the effective tenant to the reserved test tenant `llm_tester` (only `llm_tester` itself or a `test_`-prefixed override is accepted), regardless of `DEFAULT_USER` misconfiguration, at both config resolution and request-context resolution (`get_user()`). Startup refuses (clear error) when a non-production environment is explicitly configured with the production tenant `jpmschweitzer`, and one loud startup log line states the effective/forced tenant
|
||||
|
||||
- **Conversation context for experts + real-time think messages** - direct delegation (streaming and non-streaming) now passes a trimmed conversation history (last 6 turns) as expert context, so follow-up questions keep their referent; `_stream_direct_delegation` is now an async generator, so butler think messages ("Allow me to consult the archives, sir.") stream BEFORE the research runs instead of after it completes
|
||||
- **Bounded retries and connection reuse for library-desk** - GETs and the read-only `POST /query/*` and `POST /rag/search` endpoints retry once (2 attempts, short backoff) on transport errors and retryable 5xx; wiki writes are never retried. The client now honors `LIBRARY_DESK_TIMEOUT` instead of hardcoded 60s/30s, a librarian run holds one shared HTTP connection instead of constructing a client per tool call, and read tools raise `ModelRetry` on transient HTTP errors so the agent's retry budget engages
|
||||
- **One librarian timeout budget** - new `LIBRARIAN_TIMEOUT` (default 180s) enforced with `asyncio.wait_for` inside `delegate_to_librarian`, capping the previously uncapped live paths (steward direct delegation and streaming). The Ollama provider's AsyncOpenAI client now carries an explicit `OLLAMA_TIMEOUT` instead of the SDK's ~600s default, and the contradictory unused 60s default in `AgentRequest.timeout_seconds` was removed (None defers to the configured budget)
|
||||
- **Search degradation signaling** - The librarian client parses `source_counts` (plus the additive `source_status`/`degraded` fields when a newer library-desk sends them; absence is tolerated), and `hybrid_search` appends a one-line coverage note when a search is degraded or an enabled source leg contributed nothing, so outages are visible to the model and the user. When `source_status` is present it is used exclusively; without it, count-absence is only inferred for the optional legs the request explicitly enabled (web/documents/volatile) - never the always-on vector/graph legs, whose absence from the top-N counts is normal ranking behavior, so healthy searches no longer emit warnings
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Clearing all wiki-page tags is possible again** - the Ollama-safe empty-list sentinel in `update_wiki_page` means "leave unchanged", which made it impossible to remove all tags; passing exactly `["__CLEAR__"]` now sends an empty tag list to library-desk (documented in the tool docstring for the local model)
|
||||
- **Text-delegation fallback pairs results strictly** - the parallel branch now verifies `asyncio.gather` returned one result per parsed delegation (`zip(..., strict=True)`); a count mismatch fails loudly with a curated apology instead of silently attributing outputs to the wrong agent
|
||||
- **Ollama-safe librarian tool schemas** - `update_wiki_page` and `smart_create_wiki_page` no longer use `X | None` parameters (Ollama's OpenAI-compatible API mishandles `anyOf[X, null]`); empty-string/empty-list sentinels are translated to `None` inside the tools, matching the biographer pattern. A snapshot test pins every librarian tool schema to contain no nullable `anyOf`
|
||||
- **Honest expert failures** - `run_librarian` now raises a structured `AgentError` instead of returning error text as if it were research output, so delegation correctly reports `success=False` and the streaming error branch is reachable. Failures surface to the user as curated butler-toned sentences; exception detail (including internal URLs) stays in the logs only. Librarian tool errors no longer leak `str(e)` into synthesis
|
||||
|
||||
- **HybridRAG response mapping** - The librarian client now parses the field names library-desk actually returns (`source_type`/`sources`, `rrf_score`, `context`, per-item `related_dossiers`, synonyms nested in the `keywords` dict); previously every result rendered as "unknown (score: 0.00)". Source icons now key off the per-item `sources` list. Requests no longer send zero limits (the service rejects them with 422); legs are disabled via `enable_*` flags. Pinned by a contract test against a recorded live response (`tests/agents/librarian/fixtures/`)
|
||||
|
||||
## [2.3.0] - 2026-07-13
|
||||
|
||||
### Changed
|
||||
|
||||
- **Local-first backend (claudification rollback)** - Ollama/gemma4 is now the primary backend; Claude remains as fallback. `PREFER_CLOUD_BACKEND` defaults to `false`, Claude is used automatically when the Ollama startup health check fails, and the Steward retries mid-request failures on the other backend in both directions
|
||||
- **Default Claude model `claude-sonnet-5`** - `claude-sonnet-4-20250514` was retired by Anthropic on 2026-06-15 and would 404, leaving the fallback dead
|
||||
- **Dedicated orchestration prompt** - `orchestrate_tool_calls()` now uses a terse tool-execution prompt (`TATLOCK_ORCHESTRATION_PROMPT`); the butler persona prompt suppressed gemma4 tool calling (the model reasoned about the calculator, then answered from memory with wrong arithmetic). Synthesis keeps the persona prompt, so user-visible voice is unchanged
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Startup crash with broken anthropic package** - Anthropic SDK imports in the model selector are now lazy, so an incompatible `anthropic` install degrades to Ollama-only operation instead of crashing the app at import time (root cause of the production outage since April)
|
||||
- **Claude Sonnet 5 rejects sampling parameters** - removed `temperature` from the Steward's direct Claude call and made the Housekeeper's temperature setting backend-conditional via `get_sampling_settings()`
|
||||
- **Pin `anthropic>=0.77,<1.0`** - the April image resolved an anthropic version incompatible with pydantic-ai 1.27
|
||||
- **Steward timeout configurable** - new `STEWARD_TIMEOUT` (default 60s) replaces the hardcoded 30s, which gemma4 chronically exceeded (~35s warm analysis), causing every request to fail or fall back
|
||||
|
||||
### Added
|
||||
|
||||
- **Ollama startup health check** - verifies the server is reachable and `OLLAMA_DEFAULT_MODEL` is pulled; feeds backend resolution and `get_model_info()`
|
||||
- **Contract tests** (`tests/contracts/`, `make test-contracts`) - wire-level tests that send the raw requests the code sends to Ollama (native + OpenAI-compat tool calling), Anthropic (including the pinned temperature-rejection contract), Qdrant, SearXNG, library-desk, and Redis; unreachable services skip, wrong response shapes fail
|
||||
- **Backend resolution unit tests** (`tests/anthropic/`)
|
||||
|
||||
## [2.2.0] - 2026-04-04
|
||||
|
||||
### Changed
|
||||
|
||||
- **Switch default Ollama model to gemma4:e2b** - Replaces mistral-nemo as the local LLM backend; gemma4:e2b has native function calling support, faster tool calling (2-4s vs 15-20s), better parameter accuracy on word problems, and uses less VRAM (8GB vs 9.2GB)
|
||||
|
||||
### Added
|
||||
|
||||
- **Tool calling benchmark script** (`scripts/benchmark_tool_calling.py`) - Compares tool calling accuracy and latency across Ollama models via the Tatlock API
|
||||
|
||||
## [2.1.0] - 2026-02-05
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -8,6 +8,7 @@ Claude Code-specific notes for this project. For general development instruction
|
||||
make setup # Create venv and install all dependencies
|
||||
make test # Unit tests (no external services)
|
||||
make test-integration # Integration tests (needs Claude/Ollama)
|
||||
make test-contracts # Wire-level contract tests against live service boundaries
|
||||
make run # Start dev server on port 8777
|
||||
make lint # Ruff linter + formatter check
|
||||
make typecheck # Mypy
|
||||
@@ -18,13 +19,15 @@ Dependencies are in `pyproject.toml` (`[project.dependencies]` and `[project.opt
|
||||
|
||||
## Critical Gotchas
|
||||
|
||||
**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app` fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`. Without this, `check_claude_health()` never runs and `_claude_available` stays `None`, causing all tests to silently fall back to Ollama.
|
||||
**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app` fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`. Without this, the Ollama/Claude health checks never run: `_ollama_available` stays `None` (treated as available, so requests go to Ollama) and `_claude_available` stays `None` (treated as unavailable, so the Claude fallback never engages).
|
||||
|
||||
**AsyncIO scope mismatch.** `asyncio_default_fixture_loop_scope = function` is set in `pyproject.toml`. Session-scoped async fixtures cause `ScopeMismatch` errors. The fix is to use a sync fixture with `asyncio.run()` for session-scoped initialization.
|
||||
|
||||
**Ollama is unreliable for tool calling.** `mistral-nemo` on Ollama often does mental math instead of calling calculator tools, and frequently gets wrong answers. Claude reliably calls tools. If integration tests give wrong math answers, check which backend is actually being used.
|
||||
**The butler persona prompt suppresses local-model tool calling.** With `TATLOCK_SYSTEM_PROMPT` attached, gemma4 reasons about calling the calculator, then answers from memory with wrong arithmetic (a different wrong product each run). `orchestrate_tool_calls()` therefore uses the terse `TATLOCK_ORCHESTRATION_PROMPT`; the persona is applied in `synthesize_from_results()`. Do not reattach the persona prompt to a tool-phase agent. `tool_choice: "required"` via extra_body does NOT force Ollama to call tools — it is advisory at best.
|
||||
|
||||
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config. Ollama on tower-of-joy can be slow, especially on first request.
|
||||
**Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return a 400. Use `get_sampling_settings()` from the model selector instead of passing `ModelSettings(temperature=...)` directly to agents that can run on the Claude fallback. The contract test suite pins this (`make test-contracts`).
|
||||
|
||||
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). Current GPU-resident numbers (2026-07-14, driver 570, gemma4:e2b at ~100 tok/s): Steward analysis ~6s warm, full Steward → orchestrate → synthesize flow 11–25s, librarian-routed queries ~20-25s. The old "~35s steward / ~2 min flow" figures were measured during the CPU-only era (driver mismatch, 13 tok/s) — do not plan against them. Cold start after 2h idle adds ~8s (`OLLAMA_KEEP_ALIVE=2h`). `STEWARD_TIMEOUT` defaults to 60s.
|
||||
|
||||
**`get_benchmark_store` does not exist.** The benchmarking module (`src/core/benchmarks.py`) was never implemented. `scripts/benchmark_analysis.py` also references it and is broken. Do not add mocks for it in tests.
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: help setup run test test-unit test-integration lint typecheck clean
|
||||
.PHONY: help setup run test test-unit test-integration test-contracts lint typecheck clean
|
||||
|
||||
VENV := .venv
|
||||
PYTHON := $(VENV)/bin/python
|
||||
@@ -29,13 +29,16 @@ run: ## Start the development server on port 8777
|
||||
$(UVICORN) src.main:app --reload --host $(HOST) --port $(PORT) 2>&1 | tee build/logs/server.log
|
||||
|
||||
test: ## Run unit tests (no external services needed)
|
||||
$(PYTEST) --ignore=tests/e2e --ignore=tests/integration
|
||||
$(PYTEST) --ignore=tests/e2e --ignore=tests/integration --ignore=tests/contracts
|
||||
|
||||
test-unit: test ## Alias for test
|
||||
|
||||
test-integration: ## Run integration tests (needs Claude/Ollama)
|
||||
$(PYTEST) tests/agents/test_tatlock_agent.py -v
|
||||
|
||||
test-contracts: ## Wire-level contract tests against live service boundaries
|
||||
$(PYTEST) tests/contracts -v --no-cov
|
||||
|
||||
lint: ## Run ruff linter and formatter check
|
||||
$(RUFF) check src tests
|
||||
$(RUFF) format --check src tests
|
||||
|
||||
@@ -58,7 +58,7 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
|
||||
- Error triggers for testing (rate_limit, context_overflow)
|
||||
|
||||
- **Tatlock**: Real PydanticAI agent with butler personality
|
||||
- **LLM Backend**: Ollama (mistral-nemo:latest by default)
|
||||
- **LLM Backend**: Ollama (gemma4:e2b by default, local-first) with optional Claude fallback
|
||||
- **Personality**: Witty British butler, research-oriented
|
||||
- **Core Tools**:
|
||||
- **Calculator**: Safe mathematical expression evaluation
|
||||
@@ -74,7 +74,7 @@ A privacy-first, offline-capable personal assistant system that coordinates spec
|
||||
|
||||
- Python 3.12+ (Python 3.12.11 recommended)
|
||||
- **External Services** (must be running separately):
|
||||
- **Ollama**: LLM inference (mistral-nemo:latest, nomic-embed-text)
|
||||
- **Ollama**: LLM inference (gemma4:e2b, nomic-embed-text)
|
||||
- **Redis**: Caching and session memory
|
||||
- **Qdrant**: Vector storage for The Biographer's memory
|
||||
- **SearXNG**: Web search (optional)
|
||||
@@ -264,7 +264,10 @@ Interactive documentation available at:
|
||||
pytest
|
||||
|
||||
# Run unit tests only (no external services needed)
|
||||
pytest --ignore=tests/e2e --ignore=tests/integration
|
||||
pytest --ignore=tests/e2e --ignore=tests/integration --ignore=tests/contracts
|
||||
|
||||
# Wire-level contract tests against live service boundaries
|
||||
make test-contracts
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=src --cov-report=term-missing
|
||||
@@ -303,12 +306,17 @@ Create a `.env` file for custom configuration:
|
||||
API_HOST=0.0.0.0
|
||||
API_PORT=8000
|
||||
|
||||
# Ollama Configuration
|
||||
# Ollama Configuration (primary backend)
|
||||
OLLAMA_HOST=http://localhost:11434
|
||||
OLLAMA_DEFAULT_MODEL=mistral-nemo:latest
|
||||
OLLAMA_DEFAULT_MODEL=gemma4:e2b
|
||||
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
|
||||
OLLAMA_TIMEOUT=120
|
||||
|
||||
# Claude fallback (optional; used when Ollama is down or PREFER_CLOUD_BACKEND=true)
|
||||
# ANTHROPIC_API_KEY=sk-ant-api03-your-key-here
|
||||
ANTHROPIC_MODEL=claude-sonnet-5
|
||||
PREFER_CLOUD_BACKEND=false
|
||||
|
||||
# Redis Configuration
|
||||
REDIS_HOST=localhost
|
||||
REDIS_PORT=6379
|
||||
@@ -376,9 +384,8 @@ tatlock/
|
||||
│ │ ├── steward/ # The Steward - request analysis
|
||||
│ │ ├── tatlock_core/ # Core butler tools
|
||||
│ │ ├── tatlock.py # Tatlock PydanticAI agent
|
||||
│ │ ├── coordination.py # Multi-agent coordination
|
||||
│ │ ├── delegation.py # Expert delegation wrappers
|
||||
│ │ └── protocol.py # Agent communication protocol
|
||||
│ │ └── protocol.py # Agent error protocol
|
||||
│ ├── responses/ # Responses API (primary endpoint)
|
||||
│ ├── chat/ # Chat Completions wrapper
|
||||
│ ├── models/ # Models listing
|
||||
@@ -427,8 +434,8 @@ For LLM agent development guidelines and architectural decisions, see [AGENTS.md
|
||||
|
||||
## Version
|
||||
|
||||
Current version: **1.3.2** - Biographer tool type hints fix
|
||||
Current version: see [CHANGELOG.md](CHANGELOG.md)
|
||||
|
||||
---
|
||||
|
||||
**Note**: Tatlock is a production-ready homelab butler. All household staff use PydanticAI with Ollama for local LLM inference.
|
||||
**Note**: Tatlock is a production-ready homelab butler. All household staff use PydanticAI with local Ollama inference (gemma4), with an optional Claude cloud fallback.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
## Overview
|
||||
|
||||
Tatlock uses a bidirectional Claude architecture:
|
||||
- **Scenario A**: Tatlock powered by Claude backend (with Ollama fallback) — **COMPLETE**
|
||||
- **Scenario A**: Tatlock powered by Claude backend (with Ollama fallback) — **COMPLETE**, then **rolled back to local-first**: Ollama/gemma4 is primary, Claude is retained as fallback (`PREFER_CLOUD_BACKEND=false`)
|
||||
- **Scenario B**: Tatlock exposed as MCP server for external Claude instances — **OPEN**
|
||||
- **Scenario C**: Offline operation via Ollama — **COMPLETE**
|
||||
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@ This document tracks open/planned work. Completed phases have been removed.
|
||||
- Household staff: Tatlock (Butler), Steward, Librarian, Biographer
|
||||
- Core tools: Calculator, Date/Time, Web search (SearXNG)
|
||||
- Memory system: Qdrant (vector), Redis (session cache), multi-tenancy via ContextVar
|
||||
- Dual backend: Claude (preferred) + Ollama (fallback)
|
||||
- Dual backend: Ollama/gemma4 (primary) + Claude (fallback)
|
||||
- 439 tests with good coverage
|
||||
|
||||
---
|
||||
|
||||
+6
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "tatlock"
|
||||
version = "2.1.0"
|
||||
version = "2.4.2"
|
||||
description = "OpenAI-compatible API with Ollama backend"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
@@ -13,6 +13,10 @@ dependencies = [
|
||||
"pydantic>=2.11,<2.13",
|
||||
"pydantic-settings>=2.12,<2.13",
|
||||
"pydantic-ai-slim[openai,anthropic]>=1.27,<1.28",
|
||||
# pydantic-ai 1.27 imports the private opentelemetry._events module,
|
||||
# removed in opentelemetry-api 1.44 — cap until pydantic-ai is bumped
|
||||
"opentelemetry-api>=1.30,<1.44",
|
||||
"anthropic>=0.77,<1.0",
|
||||
"httpx>=0.28,<0.29",
|
||||
"sse-starlette>=3.0,<3.1",
|
||||
"python-dotenv>=1.2,<1.3",
|
||||
@@ -46,6 +50,7 @@ markers = [
|
||||
"unit: Unit tests",
|
||||
"integration: Integration tests",
|
||||
"slow: Slow running tests",
|
||||
"contract: Wire-level contract tests against live service boundaries",
|
||||
]
|
||||
addopts = [
|
||||
"--verbose",
|
||||
|
||||
@@ -0,0 +1,542 @@
|
||||
"""
|
||||
Benchmark tool calling across different Ollama models via Tatlock API.
|
||||
|
||||
Sends test prompts through the full Tatlock pipeline (Steward -> Orchestration
|
||||
-> Synthesis) and records tool selection accuracy, latency, and response quality.
|
||||
|
||||
Between models, swaps OLLAMA_DEFAULT_MODEL in .env and waits for uvicorn
|
||||
auto-reload. Requires the server to be running via ./wakeup.sh.
|
||||
|
||||
Usage:
|
||||
.venv/bin/python scripts/benchmark_tool_calling.py
|
||||
.venv/bin/python scripts/benchmark_tool_calling.py --models "gemma4:e4b,gemma4:e2b"
|
||||
.venv/bin/python scripts/benchmark_tool_calling.py --iterations 3
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import statistics
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Configuration
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
API_BASE = "http://localhost:8777"
|
||||
CHAT_URL = f"{API_BASE}/v1/chat/completions"
|
||||
HEALTH_URL = f"{API_BASE}/health"
|
||||
OLLAMA_URL = "http://localhost:11434"
|
||||
ENV_PATH = Path(__file__).parent.parent / ".env"
|
||||
|
||||
DEFAULT_MODELS = ["mistral-nemo-large:latest", "gemma4:e4b", "gemma4:e2b"]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test scenarios
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class Scenario:
|
||||
name: str
|
||||
prompt: str
|
||||
expected_tool: str | None # None = no tool expected
|
||||
# Patterns to check in the response text for indirect tool-use evidence
|
||||
success_patterns: list[str] = field(default_factory=list)
|
||||
category: str = "basic"
|
||||
|
||||
|
||||
SCENARIOS = [
|
||||
# --- Should call calculate_math ---
|
||||
Scenario(
|
||||
name="Simple arithmetic",
|
||||
prompt="What is 144 divided by 12?",
|
||||
expected_tool="calculate_math",
|
||||
success_patterns=["12"],
|
||||
category="calculator",
|
||||
),
|
||||
Scenario(
|
||||
name="Square root",
|
||||
prompt="What's the square root of 256?",
|
||||
expected_tool="calculate_math",
|
||||
success_patterns=["16"],
|
||||
category="calculator",
|
||||
),
|
||||
Scenario(
|
||||
name="Complex math",
|
||||
prompt="Calculate pi times the square of 5",
|
||||
expected_tool="calculate_math",
|
||||
success_patterns=["78.5"], # pi * 25 ≈ 78.54
|
||||
category="calculator",
|
||||
),
|
||||
Scenario(
|
||||
name="Word problem",
|
||||
prompt="If I have 3 bags with 17 apples each and I eat 4, how many apples do I have?",
|
||||
expected_tool="calculate_math",
|
||||
success_patterns=["47"],
|
||||
category="calculator",
|
||||
),
|
||||
|
||||
# --- Should call get_current_time ---
|
||||
Scenario(
|
||||
name="Current date",
|
||||
prompt="What's today's date?",
|
||||
expected_tool="get_current_time",
|
||||
success_patterns=["2026"], # Should contain current year
|
||||
category="datetime",
|
||||
),
|
||||
Scenario(
|
||||
name="Current time",
|
||||
prompt="What time is it right now?",
|
||||
expected_tool="get_current_time",
|
||||
success_patterns=[":"], # Time format contains colons
|
||||
category="datetime",
|
||||
),
|
||||
|
||||
# --- Should call calculate_date_offset ---
|
||||
Scenario(
|
||||
name="Relative date past",
|
||||
prompt="What was the date 2 weeks ago?",
|
||||
expected_tool="calculate_date_offset",
|
||||
success_patterns=["2026"],
|
||||
category="datetime",
|
||||
),
|
||||
|
||||
# --- Should call calculate_time_difference ---
|
||||
Scenario(
|
||||
name="Date difference",
|
||||
prompt="How many days between January 1st 2025 and March 15th 2025?",
|
||||
expected_tool="calculate_time_difference",
|
||||
success_patterns=["73", "74"], # 73 or 74 days
|
||||
category="datetime",
|
||||
),
|
||||
|
||||
# --- Should NOT call any tool ---
|
||||
Scenario(
|
||||
name="Greeting",
|
||||
prompt="Hello! How are you?",
|
||||
expected_tool=None,
|
||||
success_patterns=["sir"], # Butler personality
|
||||
category="no_tool",
|
||||
),
|
||||
Scenario(
|
||||
name="Knowledge question",
|
||||
prompt="What is the capital of France?",
|
||||
expected_tool=None,
|
||||
success_patterns=["Paris"],
|
||||
category="no_tool",
|
||||
),
|
||||
Scenario(
|
||||
name="Opinion request",
|
||||
prompt="What do you think about rainy days?",
|
||||
expected_tool=None,
|
||||
category="no_tool",
|
||||
),
|
||||
]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Result tracking
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunResult:
|
||||
scenario: str
|
||||
model: str
|
||||
iteration: int
|
||||
latency: float
|
||||
response_text: str
|
||||
has_correct_answer: bool
|
||||
error: str | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ModelStats:
|
||||
model: str
|
||||
results: list[RunResult] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def total(self) -> int:
|
||||
return len(self.results)
|
||||
|
||||
@property
|
||||
def errors(self) -> int:
|
||||
return sum(1 for r in self.results if r.error)
|
||||
|
||||
@property
|
||||
def accuracy(self) -> float:
|
||||
valid = [r for r in self.results if not r.error]
|
||||
if not valid:
|
||||
return 0
|
||||
return sum(1 for r in valid if r.has_correct_answer) / len(valid) * 100
|
||||
|
||||
@property
|
||||
def avg_latency(self) -> float:
|
||||
lats = [r.latency for r in self.results if not r.error]
|
||||
return statistics.mean(lats) if lats else 0
|
||||
|
||||
@property
|
||||
def p95_latency(self) -> float:
|
||||
lats = sorted(r.latency for r in self.results if not r.error)
|
||||
if not lats:
|
||||
return 0
|
||||
return lats[min(int(len(lats) * 0.95), len(lats) - 1)]
|
||||
|
||||
@property
|
||||
def max_latency(self) -> float:
|
||||
lats = [r.latency for r in self.results if not r.error]
|
||||
return max(lats) if lats else 0
|
||||
|
||||
def category_accuracy(self, category: str) -> float:
|
||||
cat_scenarios = {s.name for s in SCENARIOS if s.category == category}
|
||||
valid = [r for r in self.results if not r.error and r.scenario in cat_scenarios]
|
||||
if not valid:
|
||||
return 0
|
||||
return sum(1 for r in valid if r.has_correct_answer) / len(valid) * 100
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# .env manipulation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def swap_model_in_env(model_name: str):
|
||||
"""Swap OLLAMA_DEFAULT_MODEL in .env file."""
|
||||
content = ENV_PATH.read_text()
|
||||
content = re.sub(
|
||||
r'^OLLAMA_DEFAULT_MODEL=.*$',
|
||||
f'OLLAMA_DEFAULT_MODEL={model_name}',
|
||||
content,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
ENV_PATH.write_text(content)
|
||||
print(f" .env updated: OLLAMA_DEFAULT_MODEL={model_name}")
|
||||
|
||||
|
||||
async def wait_for_server_reload(client: httpx.AsyncClient, timeout: float = 30):
|
||||
"""Wait for uvicorn to auto-reload after .env change."""
|
||||
# Give uvicorn a moment to detect the file change
|
||||
await asyncio.sleep(3)
|
||||
|
||||
# Poll health endpoint
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
r = await client.get(HEALTH_URL, timeout=5)
|
||||
if r.status_code == 200:
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
await asyncio.sleep(1)
|
||||
|
||||
raise TimeoutError("Server did not come back after reload")
|
||||
|
||||
|
||||
async def warm_up_ollama_model(client: httpx.AsyncClient, model_name: str):
|
||||
"""Send a throwaway request to load the model into VRAM."""
|
||||
print(f" Warming up {model_name} in Ollama...", end=" ", flush=True)
|
||||
try:
|
||||
r = await client.post(
|
||||
f"{OLLAMA_URL}/api/generate",
|
||||
json={"model": model_name, "prompt": "hi", "stream": False},
|
||||
timeout=120,
|
||||
)
|
||||
r.raise_for_status()
|
||||
duration = r.json().get("total_duration", 0) / 1e9
|
||||
print(f"OK ({duration:.1f}s)")
|
||||
except Exception as e:
|
||||
print(f"WARN: {e}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core benchmark logic
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def run_scenario(
|
||||
client: httpx.AsyncClient,
|
||||
scenario: Scenario,
|
||||
model: str,
|
||||
iteration: int,
|
||||
) -> RunResult:
|
||||
"""Run a single scenario through the Tatlock API."""
|
||||
payload = {
|
||||
"model": "Tatlock",
|
||||
"messages": [{"role": "user", "content": scenario.prompt}],
|
||||
}
|
||||
|
||||
start = time.monotonic()
|
||||
try:
|
||||
r = await client.post(CHAT_URL, json=payload, timeout=120)
|
||||
latency = time.monotonic() - start
|
||||
|
||||
if r.status_code != 200:
|
||||
return RunResult(
|
||||
scenario=scenario.name,
|
||||
model=model,
|
||||
iteration=iteration,
|
||||
latency=latency,
|
||||
response_text="",
|
||||
has_correct_answer=False,
|
||||
error=f"HTTP {r.status_code}: {r.text[:100]}",
|
||||
)
|
||||
|
||||
data = r.json()
|
||||
response_text = data["choices"][0]["message"]["content"]
|
||||
|
||||
# Check if the response contains expected patterns
|
||||
has_correct = True
|
||||
if scenario.success_patterns:
|
||||
has_correct = any(
|
||||
p.lower() in response_text.lower()
|
||||
for p in scenario.success_patterns
|
||||
)
|
||||
|
||||
return RunResult(
|
||||
scenario=scenario.name,
|
||||
model=model,
|
||||
iteration=iteration,
|
||||
latency=latency,
|
||||
response_text=response_text,
|
||||
has_correct_answer=has_correct,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
latency = time.monotonic() - start
|
||||
return RunResult(
|
||||
scenario=scenario.name,
|
||||
model=model,
|
||||
iteration=iteration,
|
||||
latency=latency,
|
||||
response_text="",
|
||||
has_correct_answer=False,
|
||||
error=str(e)[:200],
|
||||
)
|
||||
|
||||
|
||||
async def benchmark_model(
|
||||
client: httpx.AsyncClient,
|
||||
model_name: str,
|
||||
iterations: int,
|
||||
) -> ModelStats:
|
||||
"""Run all scenarios for a single model."""
|
||||
stats = ModelStats(model=model_name)
|
||||
|
||||
print(f"\n{'=' * 70}")
|
||||
print(f" Model: {model_name}")
|
||||
print(f"{'=' * 70}")
|
||||
|
||||
# Swap model in .env
|
||||
swap_model_in_env(model_name)
|
||||
|
||||
# Warm up model in Ollama BEFORE server reload picks it up
|
||||
await warm_up_ollama_model(client, model_name)
|
||||
|
||||
# Wait for server to reload with new model
|
||||
print(" Waiting for server reload...", end=" ", flush=True)
|
||||
await wait_for_server_reload(client)
|
||||
print("OK")
|
||||
|
||||
# Run a throwaway request through the full pipeline to warm up
|
||||
print(" Warming up pipeline...", end=" ", flush=True)
|
||||
try:
|
||||
await client.post(
|
||||
CHAT_URL,
|
||||
json={"model": "Tatlock", "messages": [{"role": "user", "content": "hi"}]},
|
||||
timeout=120,
|
||||
)
|
||||
print("OK")
|
||||
except Exception as e:
|
||||
print(f"WARN: {e}")
|
||||
|
||||
for iteration in range(iterations):
|
||||
if iterations > 1:
|
||||
print(f"\n --- Iteration {iteration + 1}/{iterations} ---")
|
||||
|
||||
for scenario in SCENARIOS:
|
||||
result = await run_scenario(client, scenario, model_name, iteration)
|
||||
stats.results.append(result)
|
||||
|
||||
# Display
|
||||
if result.error:
|
||||
print(
|
||||
f" [ERR ] {scenario.name:30s} {result.latency:5.1f}s "
|
||||
f"{result.error[:60]}"
|
||||
)
|
||||
elif result.has_correct_answer:
|
||||
preview = result.response_text[:60].replace("\n", " ")
|
||||
print(f" [OK ] {scenario.name:30s} {result.latency:5.1f}s {preview}")
|
||||
else:
|
||||
preview = result.response_text[:60].replace("\n", " ")
|
||||
print(f" [MISS] {scenario.name:30s} {result.latency:5.1f}s {preview}")
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
def print_comparison(all_stats: list[ModelStats]):
|
||||
"""Print side-by-side comparison table."""
|
||||
print("\n" + "=" * 80)
|
||||
print(" COMPARISON SUMMARY")
|
||||
print("=" * 80)
|
||||
|
||||
col_width = max(len(s.model) for s in all_stats) + 2
|
||||
label_width = 32
|
||||
|
||||
header = f"{'Metric':<{label_width}}"
|
||||
for s in all_stats:
|
||||
header += f" {s.model:>{col_width}}"
|
||||
print(f"\n{header}")
|
||||
print("-" * (label_width + (col_width + 2) * len(all_stats)))
|
||||
|
||||
# Answer accuracy
|
||||
row = f"{'Correct answer rate':<{label_width}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.accuracy:>{col_width - 1}.1f}%"
|
||||
print(row)
|
||||
|
||||
# Latency
|
||||
row = f"{'Avg latency':<{label_width}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.avg_latency:>{col_width - 1}.1f}s"
|
||||
print(row)
|
||||
|
||||
row = f"{'P95 latency':<{label_width}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.p95_latency:>{col_width - 1}.1f}s"
|
||||
print(row)
|
||||
|
||||
row = f"{'Max latency':<{label_width}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.max_latency:>{col_width - 1}.1f}s"
|
||||
print(row)
|
||||
|
||||
# Errors
|
||||
row = f"{'Errors':<{label_width}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.errors:>{col_width}}"
|
||||
print(row)
|
||||
|
||||
# Per-category
|
||||
categories = sorted(set(sc.category for sc in SCENARIOS))
|
||||
print(f"\n{'Per-category accuracy':<{label_width}}")
|
||||
print("-" * (label_width + (col_width + 2) * len(all_stats)))
|
||||
for cat in categories:
|
||||
row = f" {cat:<{label_width - 2}}"
|
||||
for s in all_stats:
|
||||
row += f" {s.category_accuracy(cat):>{col_width - 1}.1f}%"
|
||||
print(row)
|
||||
|
||||
# Mismatches
|
||||
print(f"\n{'Missed answers':<50}")
|
||||
print("-" * 80)
|
||||
any_miss = False
|
||||
for scenario in SCENARIOS:
|
||||
misses = []
|
||||
for s in all_stats:
|
||||
sc_results = [r for r in s.results if r.scenario == scenario.name]
|
||||
fails = [r for r in sc_results if not r.has_correct_answer and not r.error]
|
||||
if fails:
|
||||
preview = fails[0].response_text[:50].replace("\n", " ")
|
||||
misses.append(f"{s.model}: \"{preview}\"")
|
||||
if misses:
|
||||
any_miss = True
|
||||
print(f" {scenario.name}")
|
||||
for m in misses:
|
||||
print(f" {m}")
|
||||
|
||||
if not any_miss:
|
||||
print(" (none)")
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
|
||||
|
||||
def save_results(all_stats: list[ModelStats], output_path: Path):
|
||||
"""Save detailed results to JSON."""
|
||||
data = {}
|
||||
for stats in all_stats:
|
||||
data[stats.model] = {
|
||||
"summary": {
|
||||
"accuracy": stats.accuracy,
|
||||
"avg_latency": round(stats.avg_latency, 2),
|
||||
"p95_latency": round(stats.p95_latency, 2),
|
||||
"max_latency": round(stats.max_latency, 2),
|
||||
"errors": stats.errors,
|
||||
"total_runs": stats.total,
|
||||
},
|
||||
"runs": [
|
||||
{
|
||||
"scenario": r.scenario,
|
||||
"iteration": r.iteration,
|
||||
"latency": round(r.latency, 3),
|
||||
"has_correct_answer": r.has_correct_answer,
|
||||
"response_text": r.response_text,
|
||||
"error": r.error,
|
||||
}
|
||||
for r in stats.results
|
||||
],
|
||||
}
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_text(json.dumps(data, indent=2))
|
||||
print(f"\nDetailed results saved to: {output_path}")
|
||||
|
||||
|
||||
async def main():
|
||||
parser = argparse.ArgumentParser(description="Benchmark tool calling across Ollama models via Tatlock API")
|
||||
parser.add_argument(
|
||||
"--iterations", type=int, default=1,
|
||||
help="Iterations per model (default: 1)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--models", type=str, default=",".join(DEFAULT_MODELS),
|
||||
help=f"Comma-separated models (default: {','.join(DEFAULT_MODELS)})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output", type=str, default="logs/benchmark_results.json",
|
||||
help="JSON output path (default: logs/benchmark_results.json)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
models = [m.strip() for m in args.models.split(",")]
|
||||
|
||||
# Verify server is running
|
||||
async with httpx.AsyncClient() as client:
|
||||
try:
|
||||
r = await client.get(HEALTH_URL, timeout=5)
|
||||
r.raise_for_status()
|
||||
print("Server is running.")
|
||||
except Exception:
|
||||
print("ERROR: Server not running. Start it with ./wakeup.sh first.")
|
||||
return
|
||||
|
||||
print("=" * 70)
|
||||
print(" Tool Calling Benchmark (via Tatlock API)")
|
||||
print("=" * 70)
|
||||
print(f" Models: {', '.join(models)}")
|
||||
print(f" Scenarios: {len(SCENARIOS)}")
|
||||
print(f" Iterations: {args.iterations}")
|
||||
print(f" Total runs: {len(SCENARIOS) * args.iterations * len(models)}")
|
||||
|
||||
# Remember original model to restore after benchmark
|
||||
original_env = ENV_PATH.read_text()
|
||||
|
||||
all_stats = []
|
||||
async with httpx.AsyncClient() as client:
|
||||
for model in models:
|
||||
stats = await benchmark_model(client, model, args.iterations)
|
||||
all_stats.append(stats)
|
||||
|
||||
# Restore original .env
|
||||
ENV_PATH.write_text(original_env)
|
||||
print(f"\n .env restored to original")
|
||||
|
||||
print_comparison(all_stats)
|
||||
save_results(all_stats, Path(args.output))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -3,7 +3,7 @@
|
||||
# Verifies room groups are controlled by checking actual state changes
|
||||
|
||||
API_URL="http://localhost:8777/v1/chat/completions"
|
||||
CORE_API="http://192.168.86.149:8083"
|
||||
CORE_API="http://localhost:8083"
|
||||
RESULTS_FILE="/tmp/housekeeper_test_results.txt"
|
||||
|
||||
GREEN='\033[0;32m'
|
||||
|
||||
@@ -1,407 +0,0 @@
|
||||
"""
|
||||
Multi-agent coordination engine.
|
||||
|
||||
Orchestrates delegation from Tatlock to expert agents (Librarian, etc.)
|
||||
based on Steward recommendations. Handles:
|
||||
- Routing tasks to appropriate agents
|
||||
- Parallel and sequential execution
|
||||
- Result aggregation
|
||||
- Error handling and graceful degradation
|
||||
"""
|
||||
import asyncio
|
||||
import time
|
||||
from typing import Any, AsyncGenerator, Optional
|
||||
|
||||
from src.agents.librarian import run_librarian, run_librarian_stream
|
||||
from src.agents.protocol import (
|
||||
AgentError,
|
||||
AgentRequest,
|
||||
AgentResponse,
|
||||
AgentTimeoutError,
|
||||
AgentUnavailableError,
|
||||
CoordinationResult,
|
||||
DelegationIntent,
|
||||
DelegationReason,
|
||||
ToolCallRecord,
|
||||
)
|
||||
from src.core.household_registry import get_household_registry
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
# Agent execution functions registry
|
||||
AGENT_EXECUTORS: dict[str, Any] = {
|
||||
"librarian": run_librarian,
|
||||
}
|
||||
|
||||
AGENT_STREAM_EXECUTORS: dict[str, Any] = {
|
||||
"librarian": run_librarian_stream,
|
||||
}
|
||||
|
||||
|
||||
class CoordinationEngine:
|
||||
"""
|
||||
Coordinates multi-agent task execution.
|
||||
|
||||
Routes tasks from Tatlock to appropriate expert agents,
|
||||
handles execution, and aggregates results.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the coordination engine."""
|
||||
self.registry = get_household_registry()
|
||||
logger.info("coordination_engine_initialized")
|
||||
|
||||
def get_available_agents(self) -> list[str]:
|
||||
"""
|
||||
Get list of available expert agents.
|
||||
|
||||
Returns:
|
||||
List of agent names that can accept delegations
|
||||
"""
|
||||
available = []
|
||||
for name in self.registry.list_members():
|
||||
member = self.registry.get_member(name)
|
||||
if member and member.agent is not None:
|
||||
available.append(name)
|
||||
return available
|
||||
|
||||
def can_delegate_to(self, agent_name: str) -> bool:
|
||||
"""
|
||||
Check if delegation to an agent is possible.
|
||||
|
||||
Args:
|
||||
agent_name: Name of the target agent
|
||||
|
||||
Returns:
|
||||
True if agent is available and can accept tasks
|
||||
"""
|
||||
if agent_name not in AGENT_EXECUTORS:
|
||||
return False
|
||||
|
||||
member = self.registry.get_member(agent_name)
|
||||
return member is not None and member.agent is not None
|
||||
|
||||
async def execute_delegation(
|
||||
self,
|
||||
intent: DelegationIntent,
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
) -> AgentResponse:
|
||||
"""
|
||||
Execute a single delegation to an expert agent.
|
||||
|
||||
Args:
|
||||
intent: The delegation intent with task details
|
||||
context: Additional context for the agent
|
||||
message_history: Optional conversation history
|
||||
|
||||
Returns:
|
||||
AgentResponse with results
|
||||
|
||||
Raises:
|
||||
AgentUnavailableError: If agent is not available
|
||||
AgentTimeoutError: If execution times out
|
||||
AgentError: For other execution errors
|
||||
"""
|
||||
start_time = time.time()
|
||||
agent_name = intent.target_agent
|
||||
|
||||
logger.info(
|
||||
"delegation_started",
|
||||
agent=agent_name,
|
||||
task=intent.task[:100],
|
||||
reason=intent.reason.value,
|
||||
)
|
||||
|
||||
# Check if agent is available
|
||||
if not self.can_delegate_to(agent_name):
|
||||
raise AgentUnavailableError(
|
||||
f"Agent '{agent_name}' is not available for delegation",
|
||||
agent_name=agent_name,
|
||||
)
|
||||
|
||||
# Get the executor
|
||||
executor = AGENT_EXECUTORS.get(agent_name)
|
||||
if not executor:
|
||||
raise AgentUnavailableError(
|
||||
f"No executor found for agent '{agent_name}'",
|
||||
agent_name=agent_name,
|
||||
)
|
||||
|
||||
try:
|
||||
# Build the request
|
||||
request = AgentRequest(
|
||||
task=intent.task,
|
||||
context=context,
|
||||
delegation_reason=intent.reason,
|
||||
)
|
||||
|
||||
# Execute with timeout
|
||||
timeout = request.timeout_seconds or 60
|
||||
|
||||
result = await asyncio.wait_for(
|
||||
executor(
|
||||
task=request.task,
|
||||
context=request.context,
|
||||
message_history=message_history,
|
||||
),
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
logger.info(
|
||||
"delegation_completed",
|
||||
agent=agent_name,
|
||||
duration_ms=duration_ms,
|
||||
output_length=len(result),
|
||||
)
|
||||
|
||||
return AgentResponse(
|
||||
success=True,
|
||||
result=result,
|
||||
reasoning=f"Delegated to {agent_name}: {intent.expected_outcome}",
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.error(
|
||||
"delegation_timeout",
|
||||
agent=agent_name,
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
raise AgentTimeoutError(
|
||||
f"Agent '{agent_name}' timed out after {duration_ms}ms",
|
||||
agent_name=agent_name,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.error(
|
||||
"delegation_error",
|
||||
agent=agent_name,
|
||||
error=str(e),
|
||||
duration_ms=duration_ms,
|
||||
exc_info=True,
|
||||
)
|
||||
return AgentResponse(
|
||||
success=False,
|
||||
result="",
|
||||
error_message=str(e),
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
|
||||
async def execute_delegation_stream(
|
||||
self,
|
||||
intent: DelegationIntent,
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
) -> AsyncGenerator[str, None]:
|
||||
"""
|
||||
Execute a delegation with streaming output.
|
||||
|
||||
Args:
|
||||
intent: The delegation intent with task details
|
||||
context: Additional context for the agent
|
||||
message_history: Optional conversation history
|
||||
|
||||
Yields:
|
||||
Text deltas from the agent
|
||||
|
||||
Raises:
|
||||
AgentUnavailableError: If agent is not available
|
||||
"""
|
||||
agent_name = intent.target_agent
|
||||
|
||||
logger.info(
|
||||
"delegation_stream_started",
|
||||
agent=agent_name,
|
||||
task=intent.task[:100],
|
||||
)
|
||||
|
||||
# Check if agent is available
|
||||
if agent_name not in AGENT_STREAM_EXECUTORS:
|
||||
raise AgentUnavailableError(
|
||||
f"Agent '{agent_name}' does not support streaming",
|
||||
agent_name=agent_name,
|
||||
)
|
||||
|
||||
executor = AGENT_STREAM_EXECUTORS[agent_name]
|
||||
|
||||
try:
|
||||
async for delta in executor(
|
||||
task=intent.task,
|
||||
context=context,
|
||||
message_history=message_history,
|
||||
):
|
||||
yield delta
|
||||
|
||||
logger.info("delegation_stream_completed", agent=agent_name)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"delegation_stream_error",
|
||||
agent=agent_name,
|
||||
error=str(e),
|
||||
exc_info=True,
|
||||
)
|
||||
yield f"\n\n[Error from {agent_name}: {str(e)}]"
|
||||
|
||||
async def coordinate(
|
||||
self,
|
||||
intents: list[DelegationIntent],
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
) -> CoordinationResult:
|
||||
"""
|
||||
Coordinate execution of multiple delegations.
|
||||
|
||||
Handles parallel execution for independent tasks and
|
||||
sequential execution for dependent tasks.
|
||||
|
||||
Args:
|
||||
intents: List of delegation intents to execute
|
||||
context: Shared context for all agents
|
||||
message_history: Optional conversation history
|
||||
|
||||
Returns:
|
||||
CoordinationResult with aggregated results
|
||||
"""
|
||||
start_time = time.time()
|
||||
agent_responses: dict[str, AgentResponse] = {}
|
||||
agents_consulted: list[str] = []
|
||||
|
||||
logger.info(
|
||||
"coordination_started",
|
||||
intent_count=len(intents),
|
||||
agents=[i.target_agent for i in intents],
|
||||
)
|
||||
|
||||
# Sort by priority
|
||||
sorted_intents = sorted(intents, key=lambda x: x.priority)
|
||||
|
||||
# Group by dependencies (simple version: sequential for now)
|
||||
# TODO: Implement parallel execution for independent tasks
|
||||
for intent in sorted_intents:
|
||||
try:
|
||||
response = await self.execute_delegation(
|
||||
intent=intent,
|
||||
context=context,
|
||||
message_history=message_history,
|
||||
)
|
||||
agent_responses[intent.target_agent] = response
|
||||
if response.success:
|
||||
agents_consulted.append(intent.target_agent)
|
||||
|
||||
except AgentError as e:
|
||||
agent_responses[intent.target_agent] = AgentResponse(
|
||||
success=False,
|
||||
result="",
|
||||
error_message=str(e),
|
||||
)
|
||||
|
||||
# Aggregate results
|
||||
successful_results = [
|
||||
r.result for r in agent_responses.values() if r.success and r.result
|
||||
]
|
||||
|
||||
final_response = "\n\n---\n\n".join(successful_results) if successful_results else ""
|
||||
|
||||
total_duration = int((time.time() - start_time) * 1000)
|
||||
|
||||
logger.info(
|
||||
"coordination_completed",
|
||||
total_duration_ms=total_duration,
|
||||
agents_consulted=agents_consulted,
|
||||
success_count=len(successful_results),
|
||||
)
|
||||
|
||||
return CoordinationResult(
|
||||
final_response=final_response,
|
||||
agent_responses=agent_responses,
|
||||
delegation_intents=intents,
|
||||
total_duration_ms=total_duration,
|
||||
agents_consulted=agents_consulted,
|
||||
)
|
||||
|
||||
|
||||
# Global coordination engine instance
|
||||
_coordination_engine: Optional[CoordinationEngine] = None
|
||||
|
||||
|
||||
def get_coordination_engine() -> CoordinationEngine:
|
||||
"""Get the global coordination engine instance."""
|
||||
global _coordination_engine
|
||||
if _coordination_engine is None:
|
||||
_coordination_engine = CoordinationEngine()
|
||||
return _coordination_engine
|
||||
|
||||
|
||||
async def delegate_to_librarian(
|
||||
task: str,
|
||||
context: str = "",
|
||||
reason: DelegationReason = DelegationReason.DOMAIN_EXPERTISE,
|
||||
message_history: Optional[list[Any]] = None,
|
||||
) -> AgentResponse:
|
||||
"""
|
||||
Convenience function to delegate a task to The Librarian.
|
||||
|
||||
Args:
|
||||
task: Research task description
|
||||
context: Additional context
|
||||
reason: Why delegating to Librarian
|
||||
message_history: Optional conversation history
|
||||
|
||||
Returns:
|
||||
AgentResponse with research results
|
||||
"""
|
||||
engine = get_coordination_engine()
|
||||
|
||||
intent = DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task=task,
|
||||
reason=reason,
|
||||
expected_outcome="Research findings and relevant information",
|
||||
)
|
||||
|
||||
return await engine.execute_delegation(
|
||||
intent=intent,
|
||||
context=context,
|
||||
message_history=message_history,
|
||||
)
|
||||
|
||||
|
||||
async def delegate_to_librarian_stream(
|
||||
task: str,
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
) -> AsyncGenerator[str, None]:
|
||||
"""
|
||||
Convenience function to delegate to Librarian with streaming.
|
||||
|
||||
Args:
|
||||
task: Research task description
|
||||
context: Additional context
|
||||
message_history: Optional conversation history
|
||||
|
||||
Yields:
|
||||
Text deltas from The Librarian
|
||||
"""
|
||||
engine = get_coordination_engine()
|
||||
|
||||
intent = DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task=task,
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Research findings",
|
||||
)
|
||||
|
||||
async for delta in engine.execute_delegation_stream(
|
||||
intent=intent,
|
||||
context=context,
|
||||
message_history=message_history,
|
||||
):
|
||||
yield delta
|
||||
+96
-111
@@ -8,12 +8,13 @@ returns a structured result for synthesis.
|
||||
This implements the agent-as-tool pattern recommended by PydanticAI:
|
||||
agents call other agents via tool wrappers, keeping each agent focused.
|
||||
"""
|
||||
import asyncio
|
||||
from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
from typing import AsyncGenerator, Callable, Optional, Any
|
||||
|
||||
from src.core.config import config
|
||||
from src.core.logging_config import get_logger
|
||||
from src.core.tracing import trace_span, SpanType
|
||||
from src.core.tracing import SpanType, trace_span
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -126,6 +127,50 @@ def _detect_action_type(expert: str, task: str) -> ActionType:
|
||||
return ActionType.RETRIEVE
|
||||
|
||||
|
||||
def build_delegation_context(
|
||||
conversation_history: list[dict] | None,
|
||||
max_turns: int = 6,
|
||||
max_chars_per_turn: int = 500,
|
||||
) -> str:
|
||||
"""
|
||||
Format the most recent conversation turns as delegation context.
|
||||
|
||||
Experts accept a context string but the live paths never passed the
|
||||
in-scope conversation history; this trims it to the last few turns
|
||||
so follow-up questions ("and what about X?") keep their referent.
|
||||
|
||||
Args:
|
||||
conversation_history: Prior messages as {"role", "content"} dicts
|
||||
max_turns: How many trailing turns to include
|
||||
max_chars_per_turn: Truncation limit per turn
|
||||
|
||||
Returns:
|
||||
str: Newline-joined "role: content" lines ("" when no history)
|
||||
"""
|
||||
if not conversation_history:
|
||||
return ""
|
||||
|
||||
lines = []
|
||||
for msg in conversation_history[-max_turns:]:
|
||||
if not isinstance(msg, dict):
|
||||
continue
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
if isinstance(content, list):
|
||||
# Tolerate structured content parts
|
||||
content = " ".join(
|
||||
part.get("text", "") if isinstance(part, dict) else str(part)
|
||||
for part in content
|
||||
)
|
||||
content = str(content).strip()
|
||||
if content:
|
||||
lines.append(f"{role}: {content[:max_chars_per_turn]}")
|
||||
|
||||
if not lines:
|
||||
return ""
|
||||
return "Recent conversation:\n" + "\n".join(lines)
|
||||
|
||||
|
||||
def get_think_message(expert: str, task: str, phase: str) -> str:
|
||||
"""
|
||||
Get the appropriate think message for an expert delegation.
|
||||
@@ -167,7 +212,7 @@ class DelegationTask:
|
||||
action: str = ""
|
||||
priority: int = 0
|
||||
depends_on: list[str] = field(default_factory=list)
|
||||
result: Optional[str] = None
|
||||
result: str | None = None
|
||||
task_id: str = ""
|
||||
|
||||
def __post_init__(self):
|
||||
@@ -186,14 +231,16 @@ class DelegationResult:
|
||||
expert_name: Which expert handled the task
|
||||
task: Original task description
|
||||
success: Whether the delegation succeeded
|
||||
output: Expert's response/findings
|
||||
error: Error message if failed
|
||||
output: Expert's response/findings. On failure this holds a
|
||||
curated, user-safe butler sentence (never exception detail)
|
||||
error: Short user-safe error label if failed. Exception detail
|
||||
stays in the logs only
|
||||
"""
|
||||
expert_name: str
|
||||
task: str
|
||||
success: bool
|
||||
output: str
|
||||
error: Optional[str] = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
async def delegate_to_librarian(
|
||||
@@ -250,8 +297,14 @@ async def delegate_to_librarian(
|
||||
},
|
||||
) as span:
|
||||
try:
|
||||
# Use run() not run_stream() - avoids Ollama bug
|
||||
output = await run_librarian(task=task, context=context)
|
||||
# Use run() not run_stream() - avoids Ollama bug.
|
||||
# One timeout budget for the whole delegation - covers both
|
||||
# live paths (steward direct delegation and streaming), which
|
||||
# previously had no cap at all (SDK default ~600s per LLM call).
|
||||
output = await asyncio.wait_for(
|
||||
run_librarian(task=task, context=context),
|
||||
timeout=config.LIBRARIAN_TIMEOUT,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"delegation_to_librarian_completed",
|
||||
@@ -273,6 +326,30 @@ async def delegate_to_librarian(
|
||||
output=output,
|
||||
)
|
||||
|
||||
except TimeoutError:
|
||||
logger.error(
|
||||
"delegation_to_librarian_timeout",
|
||||
task=task[:50],
|
||||
timeout_seconds=config.LIBRARIAN_TIMEOUT,
|
||||
)
|
||||
|
||||
if span:
|
||||
span.metadata["success"] = False
|
||||
span.details["error"] = (
|
||||
f"timed out after {config.LIBRARIAN_TIMEOUT}s"
|
||||
)
|
||||
|
||||
return DelegationResult(
|
||||
expert_name="librarian",
|
||||
task=task,
|
||||
success=False,
|
||||
output=(
|
||||
"I'm afraid the research took longer than expected "
|
||||
"and had to be abandoned, sir."
|
||||
),
|
||||
error="The Librarian did not respond within the time budget.",
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"delegation_to_librarian_error",
|
||||
@@ -285,12 +362,15 @@ async def delegate_to_librarian(
|
||||
span.metadata["success"] = False
|
||||
span.details["error"] = str(e)
|
||||
|
||||
# Exception detail stays in the logs; the user-facing output
|
||||
# is a curated butler sentence so internals never leak into
|
||||
# synthesis.
|
||||
return DelegationResult(
|
||||
expert_name="librarian",
|
||||
task=task,
|
||||
success=False,
|
||||
output="",
|
||||
error=str(e),
|
||||
output=get_think_message("librarian", task, "error"),
|
||||
error="The Librarian was unable to complete the task.",
|
||||
)
|
||||
|
||||
|
||||
@@ -383,12 +463,13 @@ async def delegate_to_biographer(
|
||||
span.metadata["success"] = False
|
||||
span.details["error"] = str(e)
|
||||
|
||||
# Exception detail stays in the logs only.
|
||||
return DelegationResult(
|
||||
expert_name="biographer",
|
||||
task=task,
|
||||
success=False,
|
||||
output="",
|
||||
error=str(e),
|
||||
output=get_think_message("biographer", task, "error"),
|
||||
error="The Biographer was unable to complete the task.",
|
||||
)
|
||||
|
||||
|
||||
@@ -480,112 +561,16 @@ async def delegate_to_housekeeper(
|
||||
span.metadata["success"] = False
|
||||
span.details["error"] = str(e)
|
||||
|
||||
# Exception detail stays in the logs only.
|
||||
return DelegationResult(
|
||||
expert_name="housekeeper",
|
||||
task=task,
|
||||
success=False,
|
||||
output="",
|
||||
error=str(e),
|
||||
output=get_think_message("housekeeper", task, "error"),
|
||||
error="The Housekeeper was unable to complete the task.",
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Streaming Delegation Wrappers (with Think Messages)
|
||||
# =============================================================================
|
||||
|
||||
async def stream_delegate_to_librarian(
|
||||
task: str,
|
||||
context: str = "",
|
||||
) -> AsyncGenerator[str, None]:
|
||||
"""
|
||||
Stream delegation to Librarian with automatic think messages.
|
||||
|
||||
Yields butler-perspective think messages before and after the delegation,
|
||||
allowing the UI to show progress to the user.
|
||||
|
||||
Args:
|
||||
task: Task description
|
||||
context: Additional context
|
||||
|
||||
Yields:
|
||||
str: Think messages and final result marker
|
||||
"""
|
||||
# Yield start message (deterministic)
|
||||
yield get_think_message("librarian", task, "start") + "\n"
|
||||
|
||||
# Execute delegation
|
||||
result = await delegate_to_librarian(task, context)
|
||||
|
||||
# Yield completion message (deterministic)
|
||||
if result.success:
|
||||
yield get_think_message("librarian", task, "success") + "\n"
|
||||
else:
|
||||
yield get_think_message("librarian", task, "error") + "\n"
|
||||
|
||||
# Yield result marker for extraction
|
||||
yield f"__DELEGATION_RESULT__:librarian:{result.output}"
|
||||
|
||||
|
||||
async def stream_delegate_to_biographer(
|
||||
task: str,
|
||||
context: str = "",
|
||||
) -> AsyncGenerator[str, None]:
|
||||
"""
|
||||
Stream delegation to Biographer with automatic think messages.
|
||||
|
||||
Args:
|
||||
task: Task description
|
||||
context: Additional context
|
||||
|
||||
Yields:
|
||||
str: Think messages and final result marker
|
||||
"""
|
||||
yield get_think_message("biographer", task, "start") + "\n"
|
||||
|
||||
result = await delegate_to_biographer(task, context)
|
||||
|
||||
if result.success:
|
||||
yield get_think_message("biographer", task, "success") + "\n"
|
||||
else:
|
||||
yield get_think_message("biographer", task, "error") + "\n"
|
||||
|
||||
yield f"__DELEGATION_RESULT__:biographer:{result.output}"
|
||||
|
||||
|
||||
async def stream_delegate_to_housekeeper(
|
||||
task: str,
|
||||
context: str = "",
|
||||
) -> AsyncGenerator[str, None]:
|
||||
"""
|
||||
Stream delegation to Housekeeper with automatic think messages.
|
||||
|
||||
Args:
|
||||
task: Task description
|
||||
context: Additional context
|
||||
|
||||
Yields:
|
||||
str: Think messages and final result marker
|
||||
"""
|
||||
yield get_think_message("housekeeper", task, "start") + "\n"
|
||||
|
||||
result = await delegate_to_housekeeper(task, context)
|
||||
|
||||
if result.success:
|
||||
yield get_think_message("housekeeper", task, "success") + "\n"
|
||||
else:
|
||||
yield get_think_message("housekeeper", task, "error") + "\n"
|
||||
|
||||
yield f"__DELEGATION_RESULT__:housekeeper:{result.output}"
|
||||
|
||||
|
||||
# Mapping of streaming delegation wrappers
|
||||
STREAMING_DELEGATION_WRAPPERS = {
|
||||
"librarian": stream_delegate_to_librarian,
|
||||
"biographer": stream_delegate_to_biographer,
|
||||
"housekeeper": stream_delegate_to_housekeeper,
|
||||
}
|
||||
|
||||
|
||||
# Future expert delegation wrappers will be added here:
|
||||
# - delegate_to_developer(task, context) -> DelegationResult
|
||||
# - delegate_to_secretary(task, context) -> DelegationResult
|
||||
|
||||
@@ -204,13 +204,13 @@ async def run_housekeeper(
|
||||
)
|
||||
|
||||
try:
|
||||
# Use temperature 0.1 for slight exploration
|
||||
from pydantic_ai.settings import ModelSettings
|
||||
# Temperature 0.1 for slight exploration (skipped on Claude backend)
|
||||
from src.anthropic.model_selector import get_sampling_settings
|
||||
|
||||
result = await agent.run(
|
||||
prompt,
|
||||
message_history=message_history,
|
||||
model_settings=ModelSettings(temperature=0.1),
|
||||
model_settings=get_sampling_settings(0.1),
|
||||
)
|
||||
|
||||
logger.info(
|
||||
@@ -266,13 +266,13 @@ async def run_housekeeper_stream(
|
||||
)
|
||||
|
||||
try:
|
||||
# Use temperature 0.1 for slight exploration
|
||||
from pydantic_ai.settings import ModelSettings
|
||||
# Temperature 0.1 for slight exploration (skipped on Claude backend)
|
||||
from src.anthropic.model_selector import get_sampling_settings
|
||||
|
||||
async with agent.run_stream(
|
||||
prompt,
|
||||
message_history=message_history,
|
||||
model_settings=ModelSettings(temperature=0.1),
|
||||
model_settings=get_sampling_settings(0.1),
|
||||
) as response:
|
||||
async for delta in response.stream_text(delta=True):
|
||||
yield delta
|
||||
|
||||
@@ -10,7 +10,6 @@ Connects to the library-desk API to provide:
|
||||
from src.agents.librarian.agent import (
|
||||
get_librarian_agent,
|
||||
run_librarian,
|
||||
run_librarian_stream,
|
||||
)
|
||||
from src.agents.librarian.capability import (
|
||||
LIBRARIAN_CAPABILITY,
|
||||
@@ -26,5 +25,4 @@ __all__ = [
|
||||
"register_librarian",
|
||||
"unregister_librarian",
|
||||
"run_librarian",
|
||||
"run_librarian_stream",
|
||||
]
|
||||
|
||||
@@ -7,10 +7,11 @@ the library-desk API, offering:
|
||||
- Wiki and document management
|
||||
- Semantic search and knowledge graph exploration
|
||||
"""
|
||||
from typing import Any, Optional
|
||||
from typing import Any
|
||||
|
||||
from pydantic_ai import Agent
|
||||
|
||||
from src.agents.librarian.client import library_client_session
|
||||
from src.agents.librarian.tools import (
|
||||
create_wiki_page,
|
||||
explore_knowledge_graph,
|
||||
@@ -27,7 +28,7 @@ from src.agents.librarian.tools import (
|
||||
smart_create_wiki_page,
|
||||
update_wiki_page,
|
||||
)
|
||||
from src.core.config import config
|
||||
from src.agents.protocol import AgentError
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -137,8 +138,32 @@ If a tool fails or you cannot access a data source:
|
||||
- It is better to return no information than to return fabricated information
|
||||
"""
|
||||
|
||||
|
||||
# Tool-phase prompt actually used by the agent. The scholarly persona prompt
|
||||
# above suppresses tool calling on small local models (gemma4 answers in
|
||||
# character - "please provide your request" - without ever calling a tool),
|
||||
# the same pathology TATLOCK_ORCHESTRATION_PROMPT fixed for the butler.
|
||||
# Tatlock's synthesis phase supplies the user-facing voice, so the research
|
||||
# phase only needs tool discipline. Kept: the anti-fabrication rule.
|
||||
LIBRARIAN_TASK_PROMPT = """You are The Librarian, the research executor of the \
|
||||
Tatlock household. Your only job is to gather accurate findings by calling the \
|
||||
provided tools.
|
||||
|
||||
- ALWAYS use tools - never answer a research task from memory alone.
|
||||
- Research or wiki questions: call hybrid_search first; then search_wiki and \
|
||||
get_wiki_page to read specific pages BEFORE summarizing them.
|
||||
- Current or external information (weather, news, live facts): call search_web; \
|
||||
call read_url when given a specific URL.
|
||||
- Wiki writing: smart_create_wiki_page when asked for a page about a topic; \
|
||||
create_wiki_page only for user-provided verbatim content; update_wiki_page for \
|
||||
edits (search_wiki, then get_wiki_page, then update).
|
||||
- Reply with a concise factual summary of what the tools returned, citing page \
|
||||
titles and URLs. A later step writes the polished answer, so no personality.
|
||||
- NEVER fabricate. If a tool fails or returns nothing, state exactly what you \
|
||||
could not retrieve and stop."""
|
||||
|
||||
# Lazy initialization to avoid connection issues during imports
|
||||
_librarian_agent: Optional[Agent[None, str]] = None
|
||||
_librarian_agent: Agent[None, str] | None = None
|
||||
|
||||
|
||||
def _create_librarian_agent() -> Agent[None, str]:
|
||||
@@ -150,7 +175,7 @@ def _create_librarian_agent() -> Agent[None, str]:
|
||||
|
||||
agent: Agent[None, str] = Agent(
|
||||
model=model,
|
||||
system_prompt=LIBRARIAN_SYSTEM_PROMPT,
|
||||
system_prompt=LIBRARIAN_TASK_PROMPT,
|
||||
retries=2,
|
||||
)
|
||||
|
||||
@@ -204,7 +229,7 @@ def get_librarian_agent() -> Agent[None, str]:
|
||||
async def run_librarian(
|
||||
task: str,
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
message_history: list[Any] | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
Execute a research task with The Librarian.
|
||||
@@ -220,6 +245,10 @@ async def run_librarian(
|
||||
Returns:
|
||||
Research results and findings
|
||||
|
||||
Raises:
|
||||
AgentError: If the research task fails. Exception detail is
|
||||
logged here; callers map the failure to a user-safe message.
|
||||
|
||||
Example:
|
||||
result = await run_librarian(
|
||||
task="Find information about Docker networking",
|
||||
@@ -241,10 +270,12 @@ async def run_librarian(
|
||||
)
|
||||
|
||||
try:
|
||||
result = await agent.run(
|
||||
prompt,
|
||||
message_history=message_history,
|
||||
)
|
||||
# One shared library-desk connection for all tool calls in this run
|
||||
async with library_client_session():
|
||||
result = await agent.run(
|
||||
prompt,
|
||||
message_history=message_history,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"librarian_task_completed",
|
||||
@@ -255,64 +286,14 @@ async def run_librarian(
|
||||
return result.output
|
||||
|
||||
except Exception as e:
|
||||
# Full detail stays in the logs; callers receive a structured
|
||||
# failure instead of error text masquerading as research output.
|
||||
logger.error(
|
||||
"librarian_task_error",
|
||||
task=task[:50],
|
||||
error=str(e),
|
||||
exc_info=True,
|
||||
)
|
||||
return f"The Librarian encountered an error: {str(e)}"
|
||||
|
||||
|
||||
async def run_librarian_stream(
|
||||
task: str,
|
||||
context: str = "",
|
||||
message_history: Optional[list[Any]] = None,
|
||||
):
|
||||
"""
|
||||
Execute a research task with streaming output.
|
||||
|
||||
Yields text deltas as The Librarian generates the response.
|
||||
|
||||
Args:
|
||||
task: The research task or question
|
||||
context: Additional context from conversation
|
||||
message_history: Optional conversation history
|
||||
|
||||
Yields:
|
||||
str: Text deltas from the response
|
||||
|
||||
Example:
|
||||
async for delta in run_librarian_stream("Find Docker docs"):
|
||||
print(delta, end="", flush=True)
|
||||
"""
|
||||
agent = get_librarian_agent()
|
||||
|
||||
# Build prompt with context if provided
|
||||
prompt = task
|
||||
if context:
|
||||
prompt = f"Context: {context}\n\nTask: {task}"
|
||||
|
||||
logger.info(
|
||||
"librarian_stream_started",
|
||||
task=task[:100],
|
||||
)
|
||||
|
||||
try:
|
||||
async with agent.run_stream(
|
||||
prompt,
|
||||
message_history=message_history,
|
||||
) as response:
|
||||
async for delta in response.stream_text(delta=True):
|
||||
yield delta
|
||||
|
||||
logger.info("librarian_stream_completed", task=task[:50])
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"librarian_stream_error",
|
||||
task=task[:50],
|
||||
error=str(e),
|
||||
exc_info=True,
|
||||
)
|
||||
yield f"\n\nThe Librarian encountered an error: {str(e)}"
|
||||
raise AgentError(
|
||||
"Research task failed", agent_name="librarian"
|
||||
) from e
|
||||
|
||||
+289
-88
@@ -7,17 +7,32 @@ Provides async methods for all relevant library-desk endpoints:
|
||||
- Vector search
|
||||
- Knowledge graph queries
|
||||
"""
|
||||
from typing import Any, Optional
|
||||
import asyncio
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable
|
||||
from contextlib import asynccontextmanager
|
||||
from contextvars import ContextVar
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from src.core.config import config
|
||||
from src.core.context import get_user
|
||||
from src.core.context import apply_tenant_guard, get_user
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
# Retry policy for idempotent/read-only requests (GETs, POST /query/*,
|
||||
# POST /rag/search). Writes are never retried.
|
||||
_RETRY_ATTEMPTS = 2
|
||||
_RETRY_BACKOFF_SECONDS = 0.5
|
||||
_RETRYABLE_STATUS_CODES = {502, 503, 504}
|
||||
|
||||
# One shared HTTP connection per librarian run (see library_client_session)
|
||||
_shared_http_client: ContextVar[httpx.AsyncClient | None] = ContextVar(
|
||||
"library_desk_http_client", default=None
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Response Models
|
||||
@@ -28,11 +43,11 @@ class WikiPage(BaseModel):
|
||||
id: int
|
||||
path: str
|
||||
title: str
|
||||
description: Optional[str] = None
|
||||
content: Optional[str] = None
|
||||
description: str | None = None
|
||||
content: str | None = None
|
||||
tags: list[str] = Field(default_factory=list)
|
||||
created_at: Optional[str] = None
|
||||
updated_at: Optional[str] = None
|
||||
created_at: str | None = None
|
||||
updated_at: str | None = None
|
||||
|
||||
|
||||
class WikiSearchResult(BaseModel):
|
||||
@@ -40,8 +55,8 @@ class WikiSearchResult(BaseModel):
|
||||
id: int
|
||||
path: str
|
||||
title: str
|
||||
description: Optional[str] = None
|
||||
locale: Optional[str] = None
|
||||
description: str | None = None
|
||||
locale: str | None = None
|
||||
|
||||
|
||||
class VectorSearchResult(BaseModel):
|
||||
@@ -56,12 +71,14 @@ class VectorSearchResult(BaseModel):
|
||||
|
||||
class HybridSearchResult(BaseModel):
|
||||
"""Result from HybridRAG search."""
|
||||
source: str # "vector", "graph", "web"
|
||||
source: str # source_type: "wiki", "web", "volatile", "document"
|
||||
sources: list[str] = Field(default_factory=list) # legs that found it: "vector", "graph", "web", ...
|
||||
title: str
|
||||
content: str
|
||||
url: Optional[str] = None
|
||||
score: float
|
||||
page_id: Optional[int] = None
|
||||
url: str | None = None
|
||||
score: float # rrf_score from the live service
|
||||
page_id: int | None = None
|
||||
related_dossiers: list[dict[str, Any]] = Field(default_factory=list)
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
@@ -72,8 +89,15 @@ class HybridRAGResponse(BaseModel):
|
||||
synonyms: list[str] = Field(default_factory=list)
|
||||
related_dossiers: list[str] = Field(default_factory=list)
|
||||
formatted_context: str = ""
|
||||
search_id: Optional[str] = None
|
||||
search_id: str | None = None
|
||||
source_counts: dict[str, int] = Field(default_factory=dict)
|
||||
timing: dict[str, float] = Field(default_factory=dict)
|
||||
# Additive degradation contract - only newer library-desk versions
|
||||
# send these; absence means "no status reported", not "healthy".
|
||||
# Maps each leg (vector/graph/web/volatile/documents) to
|
||||
# "ok" | "failed" | "disabled".
|
||||
source_status: dict[str, str] = Field(default_factory=dict)
|
||||
degraded: bool = False
|
||||
|
||||
|
||||
class GraphNode(BaseModel):
|
||||
@@ -105,7 +129,7 @@ class WebSearchResult(BaseModel):
|
||||
content: str = "" # Full extracted text via Trafilatura
|
||||
snippet: str = "" # Original search engine snippet
|
||||
source: str = "" # Domain name
|
||||
published_date: Optional[str] = None
|
||||
published_date: str | None = None
|
||||
|
||||
|
||||
class WebSearchResponse(BaseModel):
|
||||
@@ -121,13 +145,13 @@ class WebSearchResponse(BaseModel):
|
||||
class ContentExtractionResult(BaseModel):
|
||||
"""Result from content extraction."""
|
||||
url: str
|
||||
title: Optional[str] = None
|
||||
title: str | None = None
|
||||
content: str = ""
|
||||
author: Optional[str] = None
|
||||
date: Optional[str] = None
|
||||
language: Optional[str] = None
|
||||
author: str | None = None
|
||||
date: str | None = None
|
||||
language: str | None = None
|
||||
success: bool = True
|
||||
error: Optional[str] = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class BatchExtractionResponse(BaseModel):
|
||||
@@ -151,7 +175,7 @@ class SmartCreateResponse(BaseModel):
|
||||
page: WikiPage
|
||||
research_summary: ResearchSummary = Field(default_factory=ResearchSummary)
|
||||
sources_used: int = 0
|
||||
search_id: Optional[str] = None
|
||||
search_id: str | None = None
|
||||
entity_linking: EntityLinking = Field(default_factory=EntityLinking)
|
||||
|
||||
|
||||
@@ -170,9 +194,9 @@ class LibraryDeskClient:
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: Optional[str] = None,
|
||||
api_key: Optional[str] = None,
|
||||
timeout: int = 60,
|
||||
base_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
timeout: int | None = None,
|
||||
):
|
||||
"""
|
||||
Initialize the client.
|
||||
@@ -181,30 +205,55 @@ class LibraryDeskClient:
|
||||
base_url: Library-desk API URL (defaults to config)
|
||||
api_key: API key for authentication (defaults to config)
|
||||
timeout: Request timeout in seconds
|
||||
(defaults to config.LIBRARY_DESK_TIMEOUT)
|
||||
"""
|
||||
self.base_url = base_url or str(config.LIBRARY_DESK_HOST)
|
||||
self.api_key = api_key or config.LIBRARY_DESK_API_KEY
|
||||
self.timeout = timeout
|
||||
self._client: Optional[httpx.AsyncClient] = None
|
||||
self.timeout = timeout if timeout is not None else config.LIBRARY_DESK_TIMEOUT
|
||||
self._client: httpx.AsyncClient | None = None
|
||||
self._owns_client = False
|
||||
|
||||
async def __aenter__(self) -> "LibraryDeskClient":
|
||||
"""Create HTTP client on context entry."""
|
||||
def _build_http_client(self) -> httpx.AsyncClient:
|
||||
"""Build a configured httpx client."""
|
||||
headers = {}
|
||||
if self.api_key:
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
|
||||
self._client = httpx.AsyncClient(
|
||||
return httpx.AsyncClient(
|
||||
base_url=self.base_url,
|
||||
headers=headers,
|
||||
timeout=self.timeout,
|
||||
)
|
||||
|
||||
def _uses_default_target(self) -> bool:
|
||||
"""Whether this client targets the configured library-desk instance."""
|
||||
return (
|
||||
self.base_url == str(config.LIBRARY_DESK_HOST)
|
||||
and self.api_key == config.LIBRARY_DESK_API_KEY
|
||||
)
|
||||
|
||||
async def __aenter__(self) -> "LibraryDeskClient":
|
||||
"""
|
||||
Acquire an HTTP client on context entry.
|
||||
|
||||
Reuses the run-level shared connection (see library_client_session)
|
||||
when one is active, instead of constructing a new client per call.
|
||||
"""
|
||||
shared = _shared_http_client.get()
|
||||
if shared is not None and not shared.is_closed and self._uses_default_target():
|
||||
self._client = shared
|
||||
self._owns_client = False
|
||||
else:
|
||||
self._client = self._build_http_client()
|
||||
self._owns_client = True
|
||||
return self
|
||||
|
||||
async def __aexit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
|
||||
"""Close HTTP client on context exit."""
|
||||
if self._client:
|
||||
"""Close HTTP client on context exit (only if we own it)."""
|
||||
if self._client and self._owns_client:
|
||||
await self._client.aclose()
|
||||
self._client = None
|
||||
self._client = None
|
||||
self._owns_client = False
|
||||
|
||||
def _ensure_client(self) -> httpx.AsyncClient:
|
||||
"""Ensure client is initialized."""
|
||||
@@ -214,6 +263,69 @@ class LibraryDeskClient:
|
||||
)
|
||||
return self._client
|
||||
|
||||
def _resolve_user(self, user: str | None) -> str:
|
||||
"""
|
||||
Resolve the effective tenant for a request and require it non-empty.
|
||||
|
||||
Library-desk is removing its server-side default user, so every
|
||||
request must carry an explicit tenant (a missing user will 422).
|
||||
An empty tenant is a programming or configuration error - fail
|
||||
loudly here, before any bytes hit the wire.
|
||||
|
||||
Explicit user arguments are stripped and routed through the same
|
||||
tenant guard as context resolution (get_user() already applies
|
||||
it), so a dev environment can never send the production tenant -
|
||||
or a sanitization-collision variant of it - to library-desk.
|
||||
"""
|
||||
effective = (user if user is not None else get_user()).strip()
|
||||
if not effective:
|
||||
raise ValueError(
|
||||
"library-desk request requires a non-empty user (tenant); "
|
||||
"got an empty value from the caller or request context"
|
||||
)
|
||||
return apply_tenant_guard(effective)
|
||||
|
||||
async def _request_with_retry(
|
||||
self,
|
||||
send: Callable[[], Awaitable[httpx.Response]],
|
||||
description: str,
|
||||
) -> httpx.Response:
|
||||
"""
|
||||
Send an idempotent/read-only request with a bounded retry.
|
||||
|
||||
Retries once (2 attempts total) with a short backoff on transport
|
||||
errors and retryable 5xx statuses. Only used for GETs and the
|
||||
read-only POST /query/* and /rag/search endpoints - never for
|
||||
wiki writes.
|
||||
"""
|
||||
for attempt in range(1, _RETRY_ATTEMPTS + 1):
|
||||
try:
|
||||
response = await send()
|
||||
except httpx.TransportError as e:
|
||||
if attempt >= _RETRY_ATTEMPTS:
|
||||
raise
|
||||
logger.warning(
|
||||
"library_desk_retry",
|
||||
request=description,
|
||||
error=str(e),
|
||||
attempt=attempt,
|
||||
)
|
||||
else:
|
||||
if (
|
||||
response.status_code not in _RETRYABLE_STATUS_CODES
|
||||
or attempt >= _RETRY_ATTEMPTS
|
||||
):
|
||||
return response
|
||||
logger.warning(
|
||||
"library_desk_retry",
|
||||
request=description,
|
||||
status_code=response.status_code,
|
||||
attempt=attempt,
|
||||
)
|
||||
await asyncio.sleep(_RETRY_BACKOFF_SECONDS * attempt)
|
||||
|
||||
raise RuntimeError("unreachable") # pragma: no cover
|
||||
|
||||
# ========================================================================
|
||||
# HybridRAG
|
||||
# ========================================================================
|
||||
@@ -247,17 +359,19 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
HybridRAGResponse with ranked results and context
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
# The live service requires all limits >= 1 (422 otherwise);
|
||||
# legs are disabled via the enable_* flags, not a zero limit.
|
||||
payload = {
|
||||
"query": query,
|
||||
"config": {
|
||||
"vector_limit": vector_limit,
|
||||
"graph_limit": graph_limit,
|
||||
"web_limit": web_limit,
|
||||
"document_limit": document_limit,
|
||||
"volatile_limit": volatile_limit,
|
||||
"vector_limit": max(vector_limit, 1),
|
||||
"graph_limit": max(graph_limit, 1),
|
||||
"web_limit": max(web_limit, 1),
|
||||
"document_limit": max(document_limit, 1),
|
||||
"volatile_limit": max(volatile_limit, 1),
|
||||
"enable_documents": document_limit > 0,
|
||||
"enable_volatile": volatile_limit > 0,
|
||||
"enable_web": web_limit > 0,
|
||||
@@ -268,43 +382,69 @@ class LibraryDeskClient:
|
||||
|
||||
logger.info("library_desk_hybrid_search", query=query, user=user)
|
||||
|
||||
response = await client.post(
|
||||
"/query/hybrid",
|
||||
json=payload,
|
||||
params={"user": user},
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.post(
|
||||
"/query/hybrid",
|
||||
json=payload,
|
||||
params={"user": user},
|
||||
),
|
||||
"POST /query/hybrid",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
|
||||
# Parse results
|
||||
# Parse results (live field names: source_type, sources, rrf_score,
|
||||
# related_dossiers; older names kept as fallbacks)
|
||||
results = []
|
||||
for r in data.get("results", []):
|
||||
results.append(HybridSearchResult(
|
||||
source=r.get("source", "unknown"),
|
||||
source=r.get("source_type") or r.get("source", "unknown"),
|
||||
sources=r.get("sources", []),
|
||||
title=r.get("title", ""),
|
||||
content=r.get("content", ""),
|
||||
url=r.get("url"),
|
||||
score=r.get("score", 0.0),
|
||||
score=r.get("rrf_score", r.get("score", 0.0)),
|
||||
page_id=r.get("page_id"),
|
||||
related_dossiers=r.get("related_dossiers", []),
|
||||
metadata=r.get("metadata", {}),
|
||||
))
|
||||
|
||||
# Handle keywords being either a list or a dict with core_keywords
|
||||
# Handle keywords being either a list or a dict with core_keywords;
|
||||
# the live service nests synonyms inside the keywords dict as a
|
||||
# {term: [synonyms]} map.
|
||||
raw_keywords = data.get("keywords", [])
|
||||
raw_synonyms: Any = data.get("synonyms", [])
|
||||
if isinstance(raw_keywords, dict):
|
||||
keywords = raw_keywords.get("core_keywords", [])
|
||||
raw_synonyms = raw_keywords.get("synonyms", {})
|
||||
else:
|
||||
keywords = raw_keywords
|
||||
if isinstance(raw_synonyms, dict):
|
||||
synonyms = [s for values in raw_synonyms.values() for s in values]
|
||||
else:
|
||||
synonyms = raw_synonyms
|
||||
|
||||
# Aggregate per-result related dossiers into unique top-level titles
|
||||
related_dossiers: list[str] = []
|
||||
for result in results:
|
||||
for dossier in result.related_dossiers:
|
||||
title = dossier.get("title", "")
|
||||
if title and title not in related_dossiers:
|
||||
related_dossiers.append(title)
|
||||
|
||||
return HybridRAGResponse(
|
||||
results=results,
|
||||
keywords=keywords,
|
||||
synonyms=data.get("synonyms", []),
|
||||
related_dossiers=data.get("related_dossiers", []),
|
||||
formatted_context=data.get("formatted_context", ""),
|
||||
synonyms=synonyms,
|
||||
related_dossiers=related_dossiers,
|
||||
formatted_context=data.get("context", data.get("formatted_context", "")),
|
||||
search_id=data.get("search_id"),
|
||||
source_counts=data.get("source_counts", {}),
|
||||
timing=data.get("timing", {}),
|
||||
# Additive fields - tolerate absence on older library-desk
|
||||
source_status=data.get("source_status") or {},
|
||||
degraded=bool(data.get("degraded", False)),
|
||||
)
|
||||
|
||||
# ========================================================================
|
||||
@@ -328,14 +468,17 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of matching wiki pages
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
logger.debug("library_desk_wiki_search", query=query, user=user)
|
||||
|
||||
response = await client.get(
|
||||
"/wiki/search",
|
||||
params={"q": query, "user": user, "limit": limit},
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get(
|
||||
"/wiki/search",
|
||||
params={"q": query, "user": user, "limit": limit},
|
||||
),
|
||||
"GET /wiki/search",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -357,12 +500,15 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
WikiPage with full content
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
response = await client.get(
|
||||
f"/wiki/pages/{page_id}",
|
||||
params={"user": user},
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get(
|
||||
f"/wiki/pages/{page_id}",
|
||||
params={"user": user},
|
||||
),
|
||||
f"GET /wiki/pages/{page_id}",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -371,7 +517,7 @@ class LibraryDeskClient:
|
||||
async def list_wiki_pages(
|
||||
self,
|
||||
user: str | None = None,
|
||||
tag: Optional[str] = None,
|
||||
tag: str | None = None,
|
||||
limit: int = 50,
|
||||
) -> list[WikiPage]:
|
||||
"""
|
||||
@@ -385,14 +531,17 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of wiki pages
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
params: dict[str, Any] = {"user": user, "limit": limit}
|
||||
if tag:
|
||||
params["tag"] = tag
|
||||
|
||||
response = await client.get("/wiki/pages", params=params)
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get("/wiki/pages", params=params),
|
||||
"GET /wiki/pages",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
@@ -405,7 +554,7 @@ class LibraryDeskClient:
|
||||
content: str,
|
||||
user: str | None = None,
|
||||
description: str = "",
|
||||
tags: Optional[list[str]] = None,
|
||||
tags: list[str] | None = None,
|
||||
) -> WikiPage:
|
||||
"""
|
||||
Create a new wiki page.
|
||||
@@ -421,7 +570,7 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
Created WikiPage
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
@@ -444,10 +593,10 @@ class LibraryDeskClient:
|
||||
self,
|
||||
page_id: int,
|
||||
user: str | None = None,
|
||||
content: Optional[str] = None,
|
||||
title: Optional[str] = None,
|
||||
tags: Optional[list[str]] = None,
|
||||
description: Optional[str] = None,
|
||||
content: str | None = None,
|
||||
title: str | None = None,
|
||||
tags: list[str] | None = None,
|
||||
description: str | None = None,
|
||||
) -> WikiPage:
|
||||
"""
|
||||
Update an existing wiki page.
|
||||
@@ -466,7 +615,7 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
Updated WikiPage
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
# Build update payload with only provided fields
|
||||
@@ -500,7 +649,7 @@ class LibraryDeskClient:
|
||||
topic: str,
|
||||
tags: list[str],
|
||||
user: str | None = None,
|
||||
path: Optional[str] = None,
|
||||
path: str | None = None,
|
||||
include_web_research: bool = True,
|
||||
include_wiki_search: bool = True,
|
||||
) -> SmartCreateResponse:
|
||||
@@ -524,7 +673,7 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
SmartCreateResponse with page and research metadata
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload: dict[str, Any] = {
|
||||
@@ -575,12 +724,15 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of dossiers with page counts
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
response = await client.get(
|
||||
"/wiki/dossiers",
|
||||
params={"user": user},
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get(
|
||||
"/wiki/dossiers",
|
||||
params={"user": user},
|
||||
),
|
||||
"GET /wiki/dossiers",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -610,7 +762,7 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of matching document chunks with scores
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
@@ -636,7 +788,7 @@ class LibraryDeskClient:
|
||||
self,
|
||||
cypher_query: str,
|
||||
user: str | None = None,
|
||||
parameters: Optional[dict[str, Any]] = None,
|
||||
parameters: dict[str, Any] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Execute a Cypher query on the knowledge graph.
|
||||
@@ -651,7 +803,7 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of result records
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
@@ -670,7 +822,7 @@ class LibraryDeskClient:
|
||||
async def list_graph_nodes(
|
||||
self,
|
||||
user: str | None = None,
|
||||
node_type: Optional[str] = None,
|
||||
node_type: str | None = None,
|
||||
limit: int = 100,
|
||||
) -> list[GraphNode]:
|
||||
"""
|
||||
@@ -684,14 +836,17 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
List of graph nodes
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
params: dict[str, Any] = {"user": user, "limit": limit}
|
||||
if node_type:
|
||||
params["node_type"] = node_type
|
||||
|
||||
response = await client.get("/graph/nodes", params=params)
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get("/graph/nodes", params=params),
|
||||
"GET /graph/nodes",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
@@ -712,12 +867,15 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
Node with relationships and connected nodes
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
response = await client.get(
|
||||
f"/graph/nodes/{node_id}",
|
||||
params={"user": user},
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get(
|
||||
f"/graph/nodes/{node_id}",
|
||||
params={"user": user},
|
||||
),
|
||||
f"GET /graph/nodes/{node_id}",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -736,7 +894,10 @@ class LibraryDeskClient:
|
||||
"""
|
||||
try:
|
||||
client = self._ensure_client()
|
||||
response = await client.get("/health")
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.get("/health"),
|
||||
"GET /health",
|
||||
)
|
||||
return response.status_code == 200
|
||||
except Exception as e:
|
||||
logger.warning("library_desk_health_check_failed", error=str(e))
|
||||
@@ -768,19 +929,22 @@ class LibraryDeskClient:
|
||||
Returns:
|
||||
WebSearchResponse with results and pre-formatted sources
|
||||
"""
|
||||
user = user or get_user()
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
"query": query,
|
||||
"search_type": search_type,
|
||||
"limit": limit,
|
||||
"user": user or "tatlock-librarian",
|
||||
"user": user,
|
||||
}
|
||||
|
||||
logger.info("library_desk_web_search", query=query, limit=limit)
|
||||
|
||||
response = await client.post("/rag/search", json=payload, timeout=30.0)
|
||||
response = await self._request_with_retry(
|
||||
lambda: client.post("/rag/search", json=payload),
|
||||
"POST /rag/search",
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
@@ -813,6 +977,7 @@ class LibraryDeskClient:
|
||||
async def extract_content(
|
||||
self,
|
||||
url: str,
|
||||
user: str | None = None,
|
||||
include_metadata: bool = True,
|
||||
max_length: int = 5000,
|
||||
) -> ContentExtractionResult:
|
||||
@@ -826,12 +991,14 @@ class LibraryDeskClient:
|
||||
|
||||
Args:
|
||||
url: URL to extract content from
|
||||
user: User identifier (defaults to request context)
|
||||
include_metadata: Whether to extract author, date, etc.
|
||||
max_length: Maximum content length
|
||||
|
||||
Returns:
|
||||
ContentExtractionResult (check .success and .error fields)
|
||||
"""
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
@@ -842,7 +1009,11 @@ class LibraryDeskClient:
|
||||
|
||||
logger.debug("library_desk_extract_content", url=url)
|
||||
|
||||
response = await client.post("/content/extract", json=payload, timeout=30.0)
|
||||
response = await client.post(
|
||||
"/content/extract",
|
||||
json=payload,
|
||||
params={"user": user},
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
data = response.json()
|
||||
@@ -862,6 +1033,7 @@ class LibraryDeskClient:
|
||||
async def extract_content_batch(
|
||||
self,
|
||||
urls: list[str],
|
||||
user: str | None = None,
|
||||
include_metadata: bool = True,
|
||||
max_length: int = 2000,
|
||||
) -> BatchExtractionResponse:
|
||||
@@ -875,12 +1047,14 @@ class LibraryDeskClient:
|
||||
|
||||
Args:
|
||||
urls: List of URLs to extract (max 20)
|
||||
user: User identifier (defaults to request context)
|
||||
include_metadata: Whether to extract author, date, etc.
|
||||
max_length: Maximum content length per URL
|
||||
|
||||
Returns:
|
||||
BatchExtractionResponse with results and stats
|
||||
"""
|
||||
user = self._resolve_user(user)
|
||||
client = self._ensure_client()
|
||||
|
||||
payload = {
|
||||
@@ -894,7 +1068,7 @@ class LibraryDeskClient:
|
||||
response = await client.post(
|
||||
"/content/extract/batch",
|
||||
json=payload,
|
||||
timeout=60.0, # Longer timeout for batch
|
||||
params={"user": user},
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
@@ -933,3 +1107,30 @@ async def get_library_client() -> LibraryDeskClient:
|
||||
results = await client.hybrid_search("query")
|
||||
"""
|
||||
return LibraryDeskClient()
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def library_client_session() -> AsyncIterator[None]:
|
||||
"""
|
||||
Hold ONE shared HTTP connection for the duration of a librarian run.
|
||||
|
||||
While the session is active, every LibraryDeskClient targeting the
|
||||
configured library-desk instance reuses the shared httpx client
|
||||
instead of constructing (and tearing down) a connection per tool
|
||||
call. Nested sessions are no-ops.
|
||||
|
||||
Usage:
|
||||
async with library_client_session():
|
||||
... # librarian tools reuse one connection
|
||||
"""
|
||||
if _shared_http_client.get() is not None:
|
||||
yield
|
||||
return
|
||||
|
||||
http_client = LibraryDeskClient()._build_http_client()
|
||||
token = _shared_http_client.set(http_client)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_shared_http_client.reset(token)
|
||||
await http_client.aclose()
|
||||
|
||||
+181
-46
@@ -4,12 +4,109 @@ Librarian tools for PydanticAI agent.
|
||||
These tools wrap the library-desk API and are registered with
|
||||
The Librarian agent for research and knowledge management tasks.
|
||||
"""
|
||||
from src.agents.librarian.client import LibraryDeskClient
|
||||
import httpx
|
||||
from pydantic_ai import ModelRetry
|
||||
|
||||
from src.agents.librarian.client import HybridRAGResponse, LibraryDeskClient
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def _retry_if_transient(e: Exception, what: str) -> None:
|
||||
"""
|
||||
Convert transient HTTP errors into ModelRetry so the agent's
|
||||
retry budget (Agent(retries=2)) engages instead of the tool
|
||||
swallowing the failure.
|
||||
|
||||
Only read tools call this - writes are never retried to avoid
|
||||
duplicate wiki pages.
|
||||
"""
|
||||
retryable = isinstance(e, httpx.TransportError)
|
||||
if isinstance(e, httpx.HTTPStatusError):
|
||||
status = e.response.status_code
|
||||
retryable = status >= 500 or status == 429
|
||||
if retryable:
|
||||
raise ModelRetry(
|
||||
f"{what} is temporarily unavailable; please retry."
|
||||
) from e
|
||||
|
||||
# Icons keyed by the values library-desk emits in each result's `sources`
|
||||
# list (search legs) and `source_type` (result origin).
|
||||
SOURCE_ICONS = {
|
||||
"vector": "📄",
|
||||
"graph": "🔗",
|
||||
"web": "🌐",
|
||||
"document": "📑",
|
||||
"documents": "📑",
|
||||
"volatile": "⚡",
|
||||
"wiki": "📄",
|
||||
}
|
||||
|
||||
|
||||
def _coverage_note(
|
||||
response: HybridRAGResponse,
|
||||
include_web: bool,
|
||||
include_documents: bool,
|
||||
include_volatile: bool,
|
||||
) -> str:
|
||||
"""
|
||||
Build a one-line coverage note when the search was degraded or an
|
||||
enabled source leg contributed nothing, so outages stay visible to
|
||||
the model and the user instead of silently narrowing results.
|
||||
|
||||
When the additive source_status/degraded contract is present it is
|
||||
authoritative and used EXCLUSIVELY - no count heuristics. Without
|
||||
it, absence from source_counts is only inferred for the optional
|
||||
legs this request explicitly enabled (web/documents/volatile);
|
||||
the always-on wiki legs (vector/graph) are never inferred, because
|
||||
source_counts only tallies the sources of the final top-N fused
|
||||
results, so their absence is normal ranking behavior, not an outage.
|
||||
"""
|
||||
if response.source_status:
|
||||
failed = sorted(
|
||||
leg
|
||||
for leg, status in response.source_status.items()
|
||||
if status == "failed"
|
||||
)
|
||||
if failed:
|
||||
return (
|
||||
"⚠️ *Coverage note: results are partial - "
|
||||
f"these sources failed: {', '.join(failed)}.*"
|
||||
)
|
||||
if response.degraded:
|
||||
return (
|
||||
"⚠️ *Coverage note: results are partial - "
|
||||
"one or more sources failed during this search.*"
|
||||
)
|
||||
return ""
|
||||
|
||||
if not response.source_counts:
|
||||
# Older library-desk without per-source reporting - nothing to infer
|
||||
return ""
|
||||
|
||||
# Only legs the request explicitly enabled; never vector/graph (their
|
||||
# absence from the top-N counts is healthy, see docstring)
|
||||
expected = set()
|
||||
if include_web:
|
||||
expected.add("web")
|
||||
if include_documents:
|
||||
expected.add("documents")
|
||||
if include_volatile:
|
||||
expected.add("volatile")
|
||||
|
||||
# Normalize count keys to leg names (document/documents)
|
||||
aliases = {"document": "documents"}
|
||||
reported = {aliases.get(key, key) for key in response.source_counts}
|
||||
missing = sorted(expected - reported)
|
||||
if missing:
|
||||
return (
|
||||
"⚠️ *Coverage note: no results came from: "
|
||||
f"{', '.join(missing)} (source unavailable or nothing found).*"
|
||||
)
|
||||
return ""
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# HybridRAG Search
|
||||
# ============================================================================
|
||||
@@ -76,13 +173,10 @@ async def hybrid_search(
|
||||
|
||||
# Add results
|
||||
for i, result in enumerate(response.results, 1):
|
||||
source_icon = {
|
||||
"vector": "📄",
|
||||
"graph": "🔗",
|
||||
"web": "🌐",
|
||||
"document": "📑",
|
||||
"volatile": "⚡",
|
||||
}.get(result.source, "•")
|
||||
source_keys = result.sources or [result.source]
|
||||
source_icon = "".join(
|
||||
dict.fromkeys(SOURCE_ICONS.get(key, "•") for key in source_keys)
|
||||
)
|
||||
|
||||
output_parts.append(
|
||||
f"{i}. {source_icon} **{result.title}** (score: {result.score:.2f})"
|
||||
@@ -92,17 +186,30 @@ async def hybrid_search(
|
||||
output_parts.append(f" {result.content[:300]}...")
|
||||
output_parts.append("")
|
||||
|
||||
# Surface degraded coverage so outages are visible downstream
|
||||
coverage_note = _coverage_note(
|
||||
response,
|
||||
include_web=include_web,
|
||||
include_documents=include_documents,
|
||||
include_volatile=include_volatile,
|
||||
)
|
||||
if coverage_note:
|
||||
output_parts.append(coverage_note)
|
||||
|
||||
logger.info(
|
||||
"librarian_hybrid_search",
|
||||
query=query,
|
||||
result_count=len(response.results),
|
||||
degraded=response.degraded,
|
||||
source_counts=response.source_counts,
|
||||
)
|
||||
|
||||
return "\n".join(output_parts)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_hybrid_search_error", error=str(e), query=query)
|
||||
return f"Error searching: {str(e)}"
|
||||
_retry_if_transient(e, "The knowledge archive")
|
||||
return "I was unable to search the knowledge archives; the search service did not respond properly."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -139,18 +246,22 @@ async def search_wiki(
|
||||
|
||||
output_parts = [f"## Wiki Search: {query}\n"]
|
||||
|
||||
for i, page in enumerate(results, 1):
|
||||
output_parts.append(f"{i}. **{page.title}**")
|
||||
output_parts.append(f" Path: {page.path}")
|
||||
# No ordinal numbering: small models pass the list position to
|
||||
# get_wiki_page instead of the page ID unless the ID is the only
|
||||
# number in sight.
|
||||
for page in results:
|
||||
output_parts.append(f"- **{page.title}** (page_id: {page.id})")
|
||||
output_parts.append(f" Path: {page.path}")
|
||||
if page.description:
|
||||
output_parts.append(f" {page.description}")
|
||||
output_parts.append(f" {page.description}")
|
||||
output_parts.append("")
|
||||
|
||||
return "\n".join(output_parts)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_wiki_search_error", error=str(e))
|
||||
return f"Error searching wiki: {str(e)}"
|
||||
_retry_if_transient(e, "The wiki search")
|
||||
return "I was unable to search the wiki at this time."
|
||||
|
||||
|
||||
async def get_wiki_page(
|
||||
@@ -193,7 +304,8 @@ async def get_wiki_page(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_get_page_error", error=str(e), page_id=page_id)
|
||||
return f"Error getting page {page_id}: {str(e)}"
|
||||
_retry_if_transient(e, "The wiki")
|
||||
return f"I was unable to retrieve wiki page {page_id}."
|
||||
|
||||
|
||||
async def list_dossiers() -> str:
|
||||
@@ -227,7 +339,8 @@ async def list_dossiers() -> str:
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_list_dossiers_error", error=str(e))
|
||||
return f"Error listing dossiers: {str(e)}"
|
||||
_retry_if_transient(e, "The dossier index")
|
||||
return "I was unable to retrieve the list of dossiers."
|
||||
|
||||
|
||||
async def get_dossier_pages(
|
||||
@@ -268,7 +381,8 @@ async def get_dossier_pages(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_get_dossier_error", error=str(e))
|
||||
return f"Error getting dossier: {str(e)}"
|
||||
_retry_if_transient(e, "The dossier index")
|
||||
return f"I was unable to retrieve the dossier '{dossier_name}'."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -317,7 +431,8 @@ async def semantic_search(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_semantic_search_error", error=str(e))
|
||||
return f"Error in semantic search: {str(e)}"
|
||||
_retry_if_transient(e, "The semantic search")
|
||||
return "I was unable to complete the semantic search."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -370,7 +485,8 @@ async def explore_knowledge_graph(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_explore_graph_error", error=str(e))
|
||||
return f"Error exploring knowledge graph: {str(e)}"
|
||||
_retry_if_transient(e, "The knowledge graph")
|
||||
return "I was unable to explore the knowledge graph."
|
||||
|
||||
|
||||
async def find_related_entities(
|
||||
@@ -441,7 +557,8 @@ async def find_related_entities(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_find_related_error", error=str(e))
|
||||
return f"Error finding related entities: {str(e)}"
|
||||
_retry_if_transient(e, "The knowledge graph")
|
||||
return f"I was unable to look up entities related to '{entity_name}'."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -524,7 +641,8 @@ async def search_web(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_web_search_error", error=str(e), query=query)
|
||||
return f"Error searching web: {str(e)}"
|
||||
_retry_if_transient(e, "The web search")
|
||||
return "I was unable to search the web at this time."
|
||||
|
||||
|
||||
async def read_url(
|
||||
@@ -600,7 +718,8 @@ async def read_url(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_read_url_error", error=str(e), url=url)
|
||||
return f"Error reading URL: {str(e)}"
|
||||
_retry_if_transient(e, "Content extraction")
|
||||
return f"I was unable to read the page at {url}."
|
||||
|
||||
|
||||
async def read_urls_batch(
|
||||
@@ -635,7 +754,7 @@ async def read_urls_batch(
|
||||
)
|
||||
|
||||
output_parts = [
|
||||
f"## Batch Content Extraction",
|
||||
"## Batch Content Extraction",
|
||||
f"*Extracted {response.successful}/{response.total_urls} URLs in {response.extraction_time_ms}ms*\n",
|
||||
]
|
||||
|
||||
@@ -674,19 +793,23 @@ async def read_urls_batch(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_read_urls_batch_error", error=str(e))
|
||||
return f"Error reading URLs: {str(e)}"
|
||||
_retry_if_transient(e, "Content extraction")
|
||||
return "I was unable to read the requested pages."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Wiki Write Operations
|
||||
# ============================================================================
|
||||
|
||||
CLEAR_TAGS_SENTINEL = "__CLEAR__"
|
||||
|
||||
|
||||
async def update_wiki_page(
|
||||
page_id: int,
|
||||
content: str | None = None,
|
||||
title: str | None = None,
|
||||
tags: list[str] | None = None,
|
||||
description: str | None = None,
|
||||
content: str = "",
|
||||
title: str = "",
|
||||
tags: list[str] = [], # noqa: B006 - sentinel, never mutated
|
||||
description: str = "",
|
||||
) -> str:
|
||||
"""
|
||||
Update an existing wiki page.
|
||||
@@ -700,12 +823,17 @@ async def update_wiki_page(
|
||||
- Updating tags to organize pages into dossiers
|
||||
- Fixing descriptions or titles
|
||||
|
||||
Note: empty values are sentinels for "leave unchanged" (Ollama's
|
||||
OpenAI-compatible API mishandles anyOf[X, null] parameter schemas).
|
||||
|
||||
Args:
|
||||
page_id: ID of the page to update (get from search_wiki results)
|
||||
content: New markdown content (optional - only if changing content)
|
||||
title: New title (optional - only if renaming)
|
||||
tags: New tag list (optional - replaces existing tags)
|
||||
description: New description (optional)
|
||||
content: New markdown content (empty = leave unchanged)
|
||||
title: New title (empty = leave unchanged)
|
||||
tags: New tag list, replaces existing tags (empty = leave unchanged).
|
||||
To remove ALL tags from a page, pass exactly ["__CLEAR__"]
|
||||
(an empty list means "leave unchanged", not "clear")
|
||||
description: New description (empty = leave unchanged)
|
||||
|
||||
Returns:
|
||||
Confirmation with updated page details
|
||||
@@ -713,27 +841,34 @@ async def update_wiki_page(
|
||||
Examples:
|
||||
update_wiki_page(42, content="# Updated Content\\n\\nNew information here")
|
||||
update_wiki_page(42, tags=["projects", "devops"]) # Add to dossiers
|
||||
update_wiki_page(42, tags=["__CLEAR__"]) # Remove all tags
|
||||
update_wiki_page(42, description="Updated description")
|
||||
"""
|
||||
# Empty list = leave unchanged; the explicit clear sentinel sends an
|
||||
# empty tag list to the service, which replaces (clears) all tags.
|
||||
clear_tags = tags == [CLEAR_TAGS_SENTINEL]
|
||||
|
||||
try:
|
||||
async with LibraryDeskClient() as client:
|
||||
page = await client.update_wiki_page(
|
||||
page_id=page_id,
|
||||
content=content,
|
||||
title=title,
|
||||
tags=tags,
|
||||
description=description,
|
||||
content=content if content else None,
|
||||
title=title if title else None,
|
||||
tags=[] if clear_tags else (tags if tags else None),
|
||||
description=description if description else None,
|
||||
)
|
||||
|
||||
# Build update summary
|
||||
updated_fields = []
|
||||
if content is not None:
|
||||
if content:
|
||||
updated_fields.append("content")
|
||||
if title is not None:
|
||||
if title:
|
||||
updated_fields.append("title")
|
||||
if tags is not None:
|
||||
if clear_tags:
|
||||
updated_fields.append("tags (cleared)")
|
||||
elif tags:
|
||||
updated_fields.append("tags")
|
||||
if description is not None:
|
||||
if description:
|
||||
updated_fields.append("description")
|
||||
|
||||
output_parts = [
|
||||
@@ -757,7 +892,7 @@ async def update_wiki_page(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_update_page_error", error=str(e), page_id=page_id)
|
||||
return f"Error updating page {page_id}: {str(e)}"
|
||||
return f"I was unable to update wiki page {page_id}."
|
||||
|
||||
|
||||
async def create_wiki_page(
|
||||
@@ -832,13 +967,13 @@ async def create_wiki_page(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_create_page_error", error=str(e), title=title)
|
||||
return f"Error creating page: {str(e)}"
|
||||
return f"I was unable to create the page '{title}'."
|
||||
|
||||
|
||||
async def smart_create_wiki_page(
|
||||
topic: str,
|
||||
tags: list[str],
|
||||
path: str | None = None,
|
||||
path: str = "",
|
||||
include_web_research: bool = True,
|
||||
include_wiki_search: bool = True,
|
||||
) -> str:
|
||||
@@ -860,7 +995,7 @@ async def smart_create_wiki_page(
|
||||
Args:
|
||||
topic: The topic to research and create a page about
|
||||
tags: List of tags/dossiers for categorization
|
||||
path: Optional custom path (auto-generated from topic if not provided)
|
||||
path: Optional custom path (empty = auto-generated from topic)
|
||||
include_web_research: Whether to search the web (default: True)
|
||||
include_wiki_search: Whether to search existing wiki (default: True)
|
||||
|
||||
@@ -876,7 +1011,7 @@ async def smart_create_wiki_page(
|
||||
response = await client.smart_create_wiki_page(
|
||||
topic=topic,
|
||||
tags=tags,
|
||||
path=path,
|
||||
path=path if path else None,
|
||||
include_web_research=include_web_research,
|
||||
include_wiki_search=include_wiki_search,
|
||||
)
|
||||
@@ -921,7 +1056,7 @@ async def smart_create_wiki_page(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("librarian_smart_create_error", error=str(e), topic=topic)
|
||||
return f"Error creating page about '{topic}': {str(e)}"
|
||||
return f"I was unable to create a page about '{topic}'."
|
||||
|
||||
|
||||
# ============================================================================
|
||||
|
||||
+5
-189
@@ -1,180 +1,11 @@
|
||||
"""
|
||||
Agent communication protocol for multi-agent coordination.
|
||||
Agent error protocol.
|
||||
|
||||
Defines standardized request/response formats for communication between:
|
||||
- Steward (request analysis) → Tatlock (coordination)
|
||||
- Tatlock (coordination) → Expert agents (Librarian, Developer, etc.)
|
||||
Structured exceptions raised by expert agents (e.g. The Librarian) so
|
||||
callers - the delegation wrappers in src/agents/delegation.py - can
|
||||
report success=False and map failures to curated user-safe messages
|
||||
while exception detail stays in the logs.
|
||||
"""
|
||||
from enum import Enum
|
||||
from typing import Any, Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class DelegationReason(str, Enum):
|
||||
"""Why a task is being delegated to an expert agent."""
|
||||
DOMAIN_EXPERTISE = "domain_expertise" # Expert has specialized knowledge
|
||||
TOOL_ACCESS = "tool_access" # Expert has required tools
|
||||
RESOURCE_EFFICIENCY = "resource_efficiency" # Better handled by specialist
|
||||
USER_PREFERENCE = "user_preference" # User requested specific agent
|
||||
|
||||
|
||||
class TaskComplexity(str, Enum):
|
||||
"""Complexity estimate for task execution."""
|
||||
SIMPLE = "simple" # Single tool call, fast
|
||||
MODERATE = "moderate" # Multiple steps, moderate time
|
||||
COMPLEX = "complex" # Multi-agent, significant processing
|
||||
|
||||
|
||||
class AgentRequest(BaseModel):
|
||||
"""
|
||||
Request to an expert agent.
|
||||
|
||||
Contains everything the agent needs to execute a task,
|
||||
including context from the conversation and delegation intent.
|
||||
"""
|
||||
task: str = Field(
|
||||
...,
|
||||
description="Clear description of what the agent should do"
|
||||
)
|
||||
context: str = Field(
|
||||
default="",
|
||||
description="Relevant context from conversation history"
|
||||
)
|
||||
constraints: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Any constraints or requirements for the task"
|
||||
)
|
||||
delegation_reason: DelegationReason = Field(
|
||||
default=DelegationReason.DOMAIN_EXPERTISE,
|
||||
description="Why this task was delegated to this agent"
|
||||
)
|
||||
user_id: str = Field(
|
||||
default="default",
|
||||
description="User identifier for multi-tenant operations"
|
||||
)
|
||||
max_tokens: Optional[int] = Field(
|
||||
default=None,
|
||||
description="Optional token limit for response"
|
||||
)
|
||||
timeout_seconds: Optional[int] = Field(
|
||||
default=60,
|
||||
description="Maximum time for task completion"
|
||||
)
|
||||
|
||||
|
||||
class ToolCallRecord(BaseModel):
|
||||
"""Record of a tool call made during execution."""
|
||||
tool_name: str
|
||||
arguments: dict[str, Any]
|
||||
result: str
|
||||
duration_ms: int
|
||||
|
||||
|
||||
class AgentResponse(BaseModel):
|
||||
"""
|
||||
Response from an expert agent.
|
||||
|
||||
Contains the result, reasoning, and metadata about execution.
|
||||
"""
|
||||
success: bool = Field(
|
||||
...,
|
||||
description="Whether the task completed successfully"
|
||||
)
|
||||
result: str = Field(
|
||||
...,
|
||||
description="The main output/answer from the agent"
|
||||
)
|
||||
reasoning: str = Field(
|
||||
default="",
|
||||
description="Agent's reasoning process (for transparency)"
|
||||
)
|
||||
tool_calls: list[ToolCallRecord] = Field(
|
||||
default_factory=list,
|
||||
description="Tools called during execution"
|
||||
)
|
||||
confidence: float = Field(
|
||||
default=1.0,
|
||||
ge=0.0,
|
||||
le=1.0,
|
||||
description="Agent's confidence in the result (0.0-1.0)"
|
||||
)
|
||||
sources: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Sources or references used"
|
||||
)
|
||||
error_message: Optional[str] = Field(
|
||||
default=None,
|
||||
description="Error details if success=False"
|
||||
)
|
||||
duration_ms: int = Field(
|
||||
default=0,
|
||||
description="Total execution time in milliseconds"
|
||||
)
|
||||
|
||||
|
||||
class DelegationIntent(BaseModel):
|
||||
"""
|
||||
Intent to delegate a task to an expert agent.
|
||||
|
||||
Created by Tatlock when deciding to delegate, based on
|
||||
Steward's recommendations.
|
||||
"""
|
||||
target_agent: str = Field(
|
||||
...,
|
||||
description="Name of the expert agent to delegate to"
|
||||
)
|
||||
task: str = Field(
|
||||
...,
|
||||
description="Task description for the agent"
|
||||
)
|
||||
reason: DelegationReason = Field(
|
||||
default=DelegationReason.DOMAIN_EXPERTISE,
|
||||
description="Why delegating to this agent"
|
||||
)
|
||||
expected_outcome: str = Field(
|
||||
default="",
|
||||
description="What we expect the agent to provide"
|
||||
)
|
||||
priority: int = Field(
|
||||
default=1,
|
||||
ge=1,
|
||||
le=10,
|
||||
description="Priority (1=highest, 10=lowest)"
|
||||
)
|
||||
depends_on: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Other delegation IDs this depends on (for sequencing)"
|
||||
)
|
||||
|
||||
|
||||
class CoordinationResult(BaseModel):
|
||||
"""
|
||||
Result of multi-agent coordination.
|
||||
|
||||
Aggregates results from multiple expert agents into
|
||||
a single coherent response.
|
||||
"""
|
||||
final_response: str = Field(
|
||||
...,
|
||||
description="Synthesized response from all agents"
|
||||
)
|
||||
agent_responses: dict[str, AgentResponse] = Field(
|
||||
default_factory=dict,
|
||||
description="Individual responses keyed by agent name"
|
||||
)
|
||||
delegation_intents: list[DelegationIntent] = Field(
|
||||
default_factory=list,
|
||||
description="All delegations that were executed"
|
||||
)
|
||||
total_duration_ms: int = Field(
|
||||
default=0,
|
||||
description="Total coordination time"
|
||||
)
|
||||
agents_consulted: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Names of agents that contributed"
|
||||
)
|
||||
|
||||
|
||||
class AgentError(Exception):
|
||||
@@ -184,18 +15,3 @@ class AgentError(Exception):
|
||||
self.message = message
|
||||
self.agent_name = agent_name
|
||||
super().__init__(f"[{agent_name}] {message}")
|
||||
|
||||
|
||||
class AgentTimeoutError(AgentError):
|
||||
"""Agent execution timed out."""
|
||||
pass
|
||||
|
||||
|
||||
class AgentUnavailableError(AgentError):
|
||||
"""Agent is not available or registered."""
|
||||
pass
|
||||
|
||||
|
||||
class DelegationError(AgentError):
|
||||
"""Error during task delegation."""
|
||||
pass
|
||||
|
||||
+28
-14
@@ -11,7 +11,7 @@ Uses plain text output (not JSON) for reliability. Supports both Claude
|
||||
import httpx
|
||||
from typing import Optional
|
||||
|
||||
from src.anthropic.model_selector import is_claude_available, get_model_info
|
||||
from src.anthropic.model_selector import get_model_info, is_claude_available, resolve_backend
|
||||
from src.core.config import config
|
||||
from src.core.household_registry import get_household_registry
|
||||
from src.core.logging_config import get_logger
|
||||
@@ -113,18 +113,19 @@ class StewardAgent:
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Steward with backend selection based on availability."""
|
||||
# Ollama config (fallback)
|
||||
# Ollama config (primary)
|
||||
self.ollama_host = str(config.OLLAMA_HOST).rstrip('/')
|
||||
self.ollama_model = config.OLLAMA_DEFAULT_MODEL
|
||||
|
||||
# Claude config (preferred)
|
||||
# Claude config (fallback)
|
||||
self.claude_model = config.ANTHROPIC_MODEL
|
||||
self._anthropic_client = None
|
||||
|
||||
# Determine which backend to use
|
||||
self._use_claude = config.PREFER_CLOUD_BACKEND and is_claude_available()
|
||||
# Determine which backend to use (Ollama-first, Claude when
|
||||
# preferred via config or when Ollama is down)
|
||||
self._use_claude = resolve_backend() == "claude"
|
||||
|
||||
self.timeout = 30.0 # 30 second timeout for analysis
|
||||
self.timeout = float(config.STEWARD_TIMEOUT)
|
||||
|
||||
model_info = get_model_info()
|
||||
logger.info(
|
||||
@@ -145,12 +146,12 @@ class StewardAgent:
|
||||
"""Call Claude API directly for plain text generation."""
|
||||
client = self._get_anthropic_client()
|
||||
|
||||
# No temperature: rejected by Claude Sonnet 5+ (sampling params deprecated)
|
||||
response = await client.messages.create(
|
||||
model=self.claude_model,
|
||||
max_tokens=1024,
|
||||
system=system_prompt,
|
||||
messages=[{"role": "user", "content": user_message}],
|
||||
temperature=0.3, # Lower = more consistent
|
||||
)
|
||||
|
||||
return response.content[0].text.strip()
|
||||
@@ -227,20 +228,33 @@ class StewardAgent:
|
||||
return analysis_text
|
||||
|
||||
except Exception as e:
|
||||
# If Claude fails, try Ollama as fallback
|
||||
# Mid-request fallback: retry on the other backend when possible
|
||||
if self._use_claude:
|
||||
logger.warning(
|
||||
"steward_claude_fallback",
|
||||
error=str(e),
|
||||
)
|
||||
analysis_text = await self._call_ollama(prompt)
|
||||
logger.debug(
|
||||
"steward_analysis_received",
|
||||
backend="ollama_fallback",
|
||||
text_preview=analysis_text[:150],
|
||||
fallback_backend = "ollama_fallback"
|
||||
elif is_claude_available():
|
||||
logger.warning(
|
||||
"steward_ollama_fallback",
|
||||
error=str(e),
|
||||
)
|
||||
return analysis_text
|
||||
raise
|
||||
analysis_text = await self._call_claude(
|
||||
system_prompt="You are the Steward of the household, advising the Butler (Tatlock) on which capabilities to use. Be concise and specific.",
|
||||
user_message=prompt,
|
||||
)
|
||||
fallback_backend = "claude_fallback"
|
||||
else:
|
||||
raise
|
||||
|
||||
logger.debug(
|
||||
"steward_analysis_received",
|
||||
backend=fallback_backend,
|
||||
text_preview=analysis_text[:150],
|
||||
)
|
||||
return analysis_text
|
||||
|
||||
|
||||
# Global Steward instance
|
||||
|
||||
+18
-2
@@ -129,6 +129,22 @@ or
|
||||
"""
|
||||
|
||||
|
||||
# Tool-phase prompt for orchestrate_tool_calls(). The butler personality prompt
|
||||
# suppresses tool calling on small local models (gemma4 reasons about the tool,
|
||||
# then answers from memory with wrong arithmetic), so the orchestration phase
|
||||
# uses a terse operator prompt; synthesize_from_results() applies the persona.
|
||||
TATLOCK_ORCHESTRATION_PROMPT = """You are the tool-execution phase of Tatlock, \
|
||||
a butler assistant. Your only job is to gather accurate results by calling the \
|
||||
provided tools.
|
||||
|
||||
- ALWAYS use tools for the task - never answer from memory and never do mental math.
|
||||
- Mathematics: call the calculate tool, even for trivial arithmetic.
|
||||
- Dates and times: call the date/time tools, never guess.
|
||||
- When the instructions say DELEGATE to an agent, call the matching delegate_to_* tool.
|
||||
- After the tool results arrive, reply with a one-line factual summary of the results. \
|
||||
A later step writes the polished reply, so do not add personality."""
|
||||
|
||||
|
||||
class TatlockAgent(AgentInterface):
|
||||
"""
|
||||
Tatlock - The Butler agent using PydanticAI with Ollama.
|
||||
@@ -654,10 +670,10 @@ class TatlockAgent(AgentInterface):
|
||||
# Create a fresh agent instance with scoped tools only
|
||||
model = get_model()
|
||||
|
||||
# Create agent with scoped tools
|
||||
# Create agent with scoped tools, using the tool-phase prompt
|
||||
scoped_agent = Agent(
|
||||
model,
|
||||
system_prompt=TATLOCK_SYSTEM_PROMPT,
|
||||
system_prompt=TATLOCK_ORCHESTRATION_PROMPT,
|
||||
tools=scoped_tools,
|
||||
)
|
||||
|
||||
|
||||
@@ -1,19 +1,26 @@
|
||||
"""
|
||||
Anthropic/Claude integration module.
|
||||
|
||||
Provides model selection with automatic fallback between Claude and Ollama.
|
||||
Provides model selection with Ollama as primary backend and Claude
|
||||
as the cloud fallback.
|
||||
"""
|
||||
|
||||
from src.anthropic.model_selector import (
|
||||
check_claude_health,
|
||||
check_ollama_health,
|
||||
get_model,
|
||||
get_tool_choice_settings,
|
||||
is_claude_available,
|
||||
is_ollama_available,
|
||||
resolve_backend,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"check_claude_health",
|
||||
"check_ollama_health",
|
||||
"get_model",
|
||||
"get_tool_choice_settings",
|
||||
"is_claude_available",
|
||||
"is_ollama_available",
|
||||
"resolve_backend",
|
||||
]
|
||||
|
||||
+155
-31
@@ -1,23 +1,84 @@
|
||||
"""
|
||||
Model selector for Claude/Ollama backend switching.
|
||||
Model selector for Ollama/Claude backend switching.
|
||||
|
||||
Provides automatic model selection with Claude as preferred backend
|
||||
and Ollama as offline fallback.
|
||||
Provides automatic model selection with Ollama as the primary local backend
|
||||
and Claude as the cloud fallback. Claude is used when PREFER_CLOUD_BACKEND
|
||||
is enabled, or automatically when Ollama is unavailable at startup.
|
||||
|
||||
The Anthropic SDK is imported lazily so a missing or broken `anthropic`
|
||||
package degrades to Ollama-only operation instead of crashing the app.
|
||||
"""
|
||||
|
||||
from typing import Union
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic_ai.models.anthropic import AnthropicModel
|
||||
from pydantic_ai.models.openai import OpenAIChatModel
|
||||
from pydantic_ai.providers.anthropic import AnthropicProvider
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import httpx
|
||||
|
||||
from src.core.config import config
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pydantic_ai.models.anthropic import AnthropicModel
|
||||
from pydantic_ai.models.openai import OpenAIChatModel
|
||||
from pydantic_ai.settings import ModelSettings
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
# Cached health check result (set once at startup)
|
||||
# Cached health check results (set once at startup)
|
||||
_claude_available: bool | None = None
|
||||
_ollama_available: bool | None = None
|
||||
|
||||
|
||||
async def check_ollama_health() -> bool:
|
||||
"""
|
||||
Check if the Ollama server is reachable and has the configured model.
|
||||
|
||||
This should be called once at application startup.
|
||||
The result is cached in `_ollama_available`.
|
||||
|
||||
Returns:
|
||||
True if Ollama is reachable and OLLAMA_DEFAULT_MODEL is pulled.
|
||||
"""
|
||||
global _ollama_available
|
||||
|
||||
host = str(config.OLLAMA_HOST).rstrip("/")
|
||||
model = config.OLLAMA_DEFAULT_MODEL
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
response = await client.get(f"{host}/api/tags")
|
||||
response.raise_for_status()
|
||||
names = [m.get("name", "") for m in response.json().get("models", [])]
|
||||
|
||||
if model in names or f"{model}:latest" in names:
|
||||
_ollama_available = True
|
||||
logger.info(
|
||||
"ollama_health_check_passed",
|
||||
host=host,
|
||||
model=model,
|
||||
)
|
||||
return True
|
||||
|
||||
_ollama_available = False
|
||||
logger.warning(
|
||||
"ollama_health_check_failed",
|
||||
reason="model_not_pulled",
|
||||
host=host,
|
||||
model=model,
|
||||
hint=f"run `ollama pull {model}`",
|
||||
)
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
_ollama_available = False
|
||||
logger.warning(
|
||||
"ollama_health_check_failed",
|
||||
reason="server_unreachable",
|
||||
host=host,
|
||||
error=str(e),
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
async def check_claude_health() -> bool:
|
||||
@@ -85,46 +146,93 @@ def is_claude_available() -> bool:
|
||||
return _claude_available is True
|
||||
|
||||
|
||||
def get_model(prefer_cloud: bool | None = None) -> Union[AnthropicModel, OpenAIChatModel]:
|
||||
def is_ollama_available() -> bool:
|
||||
"""
|
||||
Check if Ollama is available (from cached health check result).
|
||||
|
||||
Returns:
|
||||
False only if the startup health check confirmed Ollama is down.
|
||||
Unknown (check not run yet) counts as available so that contexts
|
||||
without lifespan events keep the local-first behavior.
|
||||
"""
|
||||
return _ollama_available is not False
|
||||
|
||||
|
||||
def resolve_backend(prefer_cloud: bool | None = None) -> str:
|
||||
"""
|
||||
Resolve which backend should serve requests.
|
||||
|
||||
Ollama is the primary backend. Claude is used when explicitly
|
||||
preferred via PREFER_CLOUD_BACKEND, or as automatic fallback
|
||||
when the startup health check found Ollama down.
|
||||
|
||||
Args:
|
||||
prefer_cloud: Override config.PREFER_CLOUD_BACKEND for this call.
|
||||
|
||||
Returns:
|
||||
"claude" or "ollama".
|
||||
"""
|
||||
use_cloud = prefer_cloud if prefer_cloud is not None else config.PREFER_CLOUD_BACKEND
|
||||
|
||||
if use_cloud and is_claude_available():
|
||||
return "claude"
|
||||
|
||||
if not is_ollama_available() and is_claude_available():
|
||||
logger.warning(
|
||||
"backend_fallback_to_claude",
|
||||
reason="ollama_unavailable",
|
||||
)
|
||||
return "claude"
|
||||
|
||||
return "ollama"
|
||||
|
||||
|
||||
def get_model(prefer_cloud: bool | None = None) -> AnthropicModel | OpenAIChatModel:
|
||||
"""
|
||||
Get the best available model.
|
||||
|
||||
Returns Claude if available and preferred, otherwise Ollama.
|
||||
Returns Ollama unless Claude is preferred (or Ollama is down).
|
||||
|
||||
Args:
|
||||
prefer_cloud: Override config.PREFER_CLOUD_BACKEND for this call.
|
||||
If None, uses the config value.
|
||||
|
||||
Returns:
|
||||
PydanticAI model instance (AnthropicModel or OpenAIChatModel).
|
||||
PydanticAI model instance (OpenAIChatModel or AnthropicModel).
|
||||
|
||||
Example:
|
||||
>>> model = get_model()
|
||||
>>> agent = Agent(model, system_prompt="...")
|
||||
"""
|
||||
# Determine preference
|
||||
use_cloud = prefer_cloud if prefer_cloud is not None else config.PREFER_CLOUD_BACKEND
|
||||
if resolve_backend(prefer_cloud) == "claude":
|
||||
try:
|
||||
from pydantic_ai.models.anthropic import AnthropicModel
|
||||
from pydantic_ai.providers.anthropic import AnthropicProvider
|
||||
|
||||
# Use Claude if available and preferred
|
||||
if use_cloud and is_claude_available():
|
||||
logger.debug(
|
||||
"model_selected",
|
||||
backend="claude",
|
||||
model=config.ANTHROPIC_MODEL,
|
||||
)
|
||||
return AnthropicModel(
|
||||
model_name=config.ANTHROPIC_MODEL,
|
||||
provider=AnthropicProvider(api_key=config.ANTHROPIC_API_KEY),
|
||||
)
|
||||
logger.debug(
|
||||
"model_selected",
|
||||
backend="claude",
|
||||
model=config.ANTHROPIC_MODEL,
|
||||
)
|
||||
return AnthropicModel(
|
||||
model_name=config.ANTHROPIC_MODEL,
|
||||
provider=AnthropicProvider(api_key=config.ANTHROPIC_API_KEY),
|
||||
)
|
||||
except ImportError as e:
|
||||
logger.error(
|
||||
"claude_backend_import_failed",
|
||||
error=str(e),
|
||||
hint="anthropic package missing or incompatible; using Ollama",
|
||||
)
|
||||
|
||||
from pydantic_ai.models.openai import OpenAIChatModel
|
||||
|
||||
# Fall back to Ollama
|
||||
from src.ollama.provider import get_ollama_provider
|
||||
|
||||
logger.debug(
|
||||
"model_selected",
|
||||
backend="ollama",
|
||||
model=config.OLLAMA_DEFAULT_MODEL,
|
||||
reason="fallback" if use_cloud else "preferred_local",
|
||||
)
|
||||
return OpenAIChatModel(
|
||||
model_name=config.OLLAMA_DEFAULT_MODEL,
|
||||
@@ -132,7 +240,7 @@ def get_model(prefer_cloud: bool | None = None) -> Union[AnthropicModel, OpenAIC
|
||||
)
|
||||
|
||||
|
||||
def get_tool_choice_settings() -> 'ModelSettings':
|
||||
def get_tool_choice_settings() -> ModelSettings:
|
||||
"""
|
||||
Get model_settings for forcing tool calls on the first request.
|
||||
|
||||
@@ -141,7 +249,7 @@ def get_tool_choice_settings() -> 'ModelSettings':
|
||||
"""
|
||||
from pydantic_ai.settings import ModelSettings
|
||||
|
||||
if is_claude_available() and config.PREFER_CLOUD_BACKEND:
|
||||
if resolve_backend() == "claude":
|
||||
# PydanticAI's Anthropic model handles tool_choice internally
|
||||
return ModelSettings()
|
||||
else:
|
||||
@@ -149,6 +257,20 @@ def get_tool_choice_settings() -> 'ModelSettings':
|
||||
return ModelSettings(extra_body={"tool_choice": "required"})
|
||||
|
||||
|
||||
def get_sampling_settings(temperature: float) -> ModelSettings:
|
||||
"""
|
||||
Get model_settings with a sampling temperature where the backend allows it.
|
||||
|
||||
Ollama accepts a temperature; Claude Sonnet 5+ rejects sampling
|
||||
parameters, so the Claude backend gets empty settings.
|
||||
"""
|
||||
from pydantic_ai.settings import ModelSettings
|
||||
|
||||
if resolve_backend() == "claude":
|
||||
return ModelSettings()
|
||||
return ModelSettings(temperature=temperature)
|
||||
|
||||
|
||||
def get_model_info() -> dict:
|
||||
"""
|
||||
Get information about the current model configuration.
|
||||
@@ -158,12 +280,14 @@ def get_model_info() -> dict:
|
||||
Returns:
|
||||
Dict with backend, model name, and availability info.
|
||||
"""
|
||||
use_cloud = config.PREFER_CLOUD_BACKEND and is_claude_available()
|
||||
backend = resolve_backend()
|
||||
|
||||
return {
|
||||
"backend": "claude" if use_cloud else "ollama",
|
||||
"model": config.ANTHROPIC_MODEL if use_cloud else config.OLLAMA_DEFAULT_MODEL,
|
||||
"backend": backend,
|
||||
"model": config.ANTHROPIC_MODEL if backend == "claude" else config.OLLAMA_DEFAULT_MODEL,
|
||||
"claude_available": is_claude_available(),
|
||||
"claude_configured": bool(config.ANTHROPIC_API_KEY),
|
||||
"ollama_available": is_ollama_available(),
|
||||
"ollama_model": config.OLLAMA_DEFAULT_MODEL,
|
||||
"prefer_cloud": config.PREFER_CLOUD_BACKEND,
|
||||
}
|
||||
|
||||
+91
-27
@@ -6,9 +6,17 @@ from enum import Enum
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import Field, HttpUrl
|
||||
from pydantic import Field, HttpUrl, model_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
# Tenant isolation constants (see docs: tenant-based isolation, no separate
|
||||
# test infrastructure). The production tenant owns real data in the shared
|
||||
# services (Qdrant/Neo4j/Wiki.js/Redis); everything non-production must run
|
||||
# under the reserved test tenant or an explicit test_-prefixed namespace.
|
||||
PRODUCTION_TENANT = "jpmschweitzer"
|
||||
TEST_TENANT = "llm_tester"
|
||||
TEST_TENANT_PREFIX = "test_"
|
||||
|
||||
|
||||
def _get_version_from_pyproject() -> str:
|
||||
"""
|
||||
@@ -42,7 +50,7 @@ class Environment(str, Enum):
|
||||
class Config(BaseSettings):
|
||||
"""
|
||||
Global application configuration.
|
||||
|
||||
|
||||
Loads from environment variables and .env file.
|
||||
Domain-specific configs should be in their respective modules.
|
||||
"""
|
||||
@@ -52,45 +60,49 @@ class Config(BaseSettings):
|
||||
case_sensitive=True,
|
||||
extra="ignore",
|
||||
)
|
||||
|
||||
|
||||
# Application
|
||||
APP_NAME: str = "OpenAI-Compatible API"
|
||||
APP_VERSION: str = Field(default_factory=_get_version_from_pyproject)
|
||||
ENVIRONMENT: Environment = Environment.DEVELOPMENT
|
||||
DEBUG: bool = Field(default=False, description="Debug mode")
|
||||
|
||||
|
||||
# API Configuration
|
||||
API_HOST: str = Field(default="0.0.0.0", description="API host")
|
||||
API_PORT: int = Field(default=8000, description="API port")
|
||||
API_PREFIX: str = Field(default="/v1", description="API route prefix")
|
||||
|
||||
# Anthropic Configuration (Claude - preferred backend)
|
||||
|
||||
# Anthropic Configuration (Claude - cloud fallback)
|
||||
ANTHROPIC_API_KEY: str | None = Field(
|
||||
default=None,
|
||||
description="Anthropic API key for Claude access"
|
||||
description="Anthropic API key for the Claude fallback backend"
|
||||
)
|
||||
ANTHROPIC_MODEL: str = Field(
|
||||
default="claude-sonnet-4-20250514",
|
||||
description="Claude model to use"
|
||||
default="claude-sonnet-5",
|
||||
description="Claude model for the fallback backend"
|
||||
)
|
||||
PREFER_CLOUD_BACKEND: bool = Field(
|
||||
default=True,
|
||||
description="Prefer Claude over Ollama when available"
|
||||
default=False,
|
||||
description="Prefer Claude over Ollama (default: local-first)"
|
||||
)
|
||||
|
||||
# Ollama Configuration (local fallback)
|
||||
# Ollama Configuration (local - primary backend)
|
||||
OLLAMA_HOST: HttpUrl = Field(
|
||||
default="http://localhost:11434",
|
||||
description="Ollama server URL"
|
||||
)
|
||||
OLLAMA_DEFAULT_MODEL: str = Field(
|
||||
default="mistral-nemo:latest",
|
||||
default="gemma4:e2b",
|
||||
description="Default Ollama model"
|
||||
)
|
||||
OLLAMA_TIMEOUT: int = Field(
|
||||
default=120,
|
||||
description="Ollama request timeout in seconds"
|
||||
)
|
||||
STEWARD_TIMEOUT: int = Field(
|
||||
default=60,
|
||||
description="Steward analysis timeout in seconds (gemma4 needs ~35s warm)"
|
||||
)
|
||||
STREAM_TIMEOUT: int = Field(
|
||||
default=20,
|
||||
description="Timeout for each streaming turn in seconds"
|
||||
@@ -98,8 +110,8 @@ class Config(BaseSettings):
|
||||
|
||||
# SearXNG Configuration
|
||||
SEARXNG_HOST: HttpUrl = Field(
|
||||
default="http://localhost:8087",
|
||||
description="SearXNG server URL"
|
||||
default="http://searxng:8080",
|
||||
description="SearXNG server URL (container name; internal port 8080)"
|
||||
)
|
||||
SEARXNG_TIMEOUT: int = Field(
|
||||
default=30,
|
||||
@@ -121,9 +133,13 @@ class Config(BaseSettings):
|
||||
)
|
||||
|
||||
# Library-Desk Configuration (The Librarian backend)
|
||||
LIBRARIAN_TIMEOUT: int = Field(
|
||||
default=180,
|
||||
description="Total time budget for a librarian delegation in seconds"
|
||||
)
|
||||
LIBRARY_DESK_HOST: HttpUrl = Field(
|
||||
default="http://localhost:8089",
|
||||
description="Library-Desk API URL"
|
||||
default="http://library-desk:8089",
|
||||
description="Library-Desk API URL (container name; internal port 8089)"
|
||||
)
|
||||
LIBRARY_DESK_API_KEY: str = Field(
|
||||
default="",
|
||||
@@ -136,8 +152,8 @@ class Config(BaseSettings):
|
||||
|
||||
# Core-API Configuration (The Housekeeper backend)
|
||||
CORE_API_HOST: HttpUrl = Field(
|
||||
default="http://localhost:8090",
|
||||
description="Core-API URL for Home Assistant integration"
|
||||
default="http://core-api:8083",
|
||||
description="Core-API URL for Home Assistant integration (container name; internal port 8083)"
|
||||
)
|
||||
CORE_API_KEY: str = Field(
|
||||
default="",
|
||||
@@ -199,6 +215,38 @@ class Config(BaseSettings):
|
||||
CORS_ALLOW_METHODS: list[str] = ["*"]
|
||||
CORS_ALLOW_HEADERS: list[str] = ["*"]
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _refuse_production_tenant_outside_production(self) -> "Config":
|
||||
"""
|
||||
Refuse startup when a non-production environment is explicitly
|
||||
configured with the production tenant.
|
||||
|
||||
This is the hard stop of the tenant isolation guard: a dev/test
|
||||
instance must never be able to read or write the production
|
||||
tenant's data in the shared services.
|
||||
|
||||
The comparison is on the sanitized form: namespaces are derived
|
||||
through sanitize_user_id(), so variants like "JPMSchweitzer" or
|
||||
"jpmschweitzer." collide with the production namespaces and are
|
||||
refused just as loudly.
|
||||
"""
|
||||
from src.core.multi_tenancy import sanitize_user_id
|
||||
|
||||
if (
|
||||
self.ENVIRONMENT != Environment.PRODUCTION
|
||||
and self.DEFAULT_USER is not None
|
||||
and sanitize_user_id(self.DEFAULT_USER)
|
||||
== sanitize_user_id(PRODUCTION_TENANT)
|
||||
):
|
||||
raise ValueError(
|
||||
f"Refusing to start: ENVIRONMENT={self.ENVIRONMENT.value} is "
|
||||
f"explicitly configured with the production tenant "
|
||||
f"'{PRODUCTION_TENANT}'. Non-production environments must use "
|
||||
f"'{TEST_TENANT}' or a '{TEST_TENANT_PREFIX}'-prefixed tenant. "
|
||||
f"Unset DEFAULT_USER or set ENVIRONMENT=production."
|
||||
)
|
||||
return self
|
||||
|
||||
@property
|
||||
def redis_memory_url(self) -> str:
|
||||
"""Construct Redis connection URL for memory cache."""
|
||||
@@ -239,23 +287,39 @@ class Config(BaseSettings):
|
||||
@property
|
||||
def effective_default_user(self) -> str:
|
||||
"""
|
||||
Get effective default user, auto-determining from environment if not set.
|
||||
Get effective default user (tenant), enforcing tenant isolation.
|
||||
|
||||
- development/testing: llm_tester (isolated test scope)
|
||||
- production: jpmschweitzer (real user)
|
||||
- production: DEFAULT_USER if set, else the production tenant
|
||||
- development/testing: FORCED to the reserved test tenant
|
||||
("llm_tester") - the only accepted overrides are the test tenant
|
||||
itself or a "test_"-prefixed namespace. Any other DEFAULT_USER
|
||||
value is treated as misconfiguration and ignored.
|
||||
"""
|
||||
if self.DEFAULT_USER is not None:
|
||||
return self.DEFAULT_USER
|
||||
if self.ENVIRONMENT == Environment.PRODUCTION:
|
||||
return "jpmschweitzer"
|
||||
return "llm_tester"
|
||||
return self.DEFAULT_USER or PRODUCTION_TENANT
|
||||
|
||||
if self.DEFAULT_USER is not None and (
|
||||
self.DEFAULT_USER == TEST_TENANT
|
||||
or self.DEFAULT_USER.startswith(TEST_TENANT_PREFIX)
|
||||
):
|
||||
return self.DEFAULT_USER
|
||||
return TEST_TENANT
|
||||
|
||||
@property
|
||||
def tenant_forced(self) -> bool:
|
||||
"""Whether the tenant guard overrode a misconfigured DEFAULT_USER."""
|
||||
return (
|
||||
self.ENVIRONMENT != Environment.PRODUCTION
|
||||
and self.DEFAULT_USER is not None
|
||||
and self.effective_default_user != self.DEFAULT_USER
|
||||
)
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_config() -> Config:
|
||||
"""
|
||||
Get cached configuration instance.
|
||||
|
||||
|
||||
Uses lru_cache to ensure config is loaded once and reused.
|
||||
"""
|
||||
return Config()
|
||||
|
||||
+38
-1
@@ -41,6 +41,41 @@ current_conversation: ContextVar[str | None] = ContextVar(
|
||||
)
|
||||
|
||||
|
||||
def apply_tenant_guard(user: str) -> str:
|
||||
"""
|
||||
Enforce tenant isolation at request-context resolution.
|
||||
|
||||
In non-production environments the production tenant must never be
|
||||
the effective user - a request that explicitly asks for it is forced
|
||||
to the reserved test tenant instead (with a loud log line).
|
||||
|
||||
Comparison happens on the *sanitized* form of the user: every local
|
||||
namespace (Qdrant collection, Redis key) is derived through
|
||||
sanitize_user_id(), so any raw variant that collides with the
|
||||
production tenant after sanitization ("JPMSchweitzer",
|
||||
"jpmschweitzer.", " jpmschweitzer", ...) would otherwise resolve to
|
||||
the production namespaces. Those variants are forced too.
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from src.core.config import PRODUCTION_TENANT, TEST_TENANT, Environment, config
|
||||
from src.core.multi_tenancy import sanitize_user_id
|
||||
|
||||
if (
|
||||
config.ENVIRONMENT != Environment.PRODUCTION
|
||||
and sanitize_user_id(user) == sanitize_user_id(PRODUCTION_TENANT)
|
||||
):
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
get_logger(__name__).warning(
|
||||
"tenant_guard_forced",
|
||||
environment=config.ENVIRONMENT.value,
|
||||
requested_tenant=user,
|
||||
forced_tenant=TEST_TENANT,
|
||||
)
|
||||
return TEST_TENANT
|
||||
return user
|
||||
|
||||
|
||||
def get_user() -> str:
|
||||
"""
|
||||
Get current user from request context.
|
||||
@@ -48,6 +83,8 @@ def get_user() -> str:
|
||||
Returns:
|
||||
User identifier for the current request.
|
||||
Falls back to environment-aware default if not set.
|
||||
In non-production environments the production tenant is never
|
||||
returned - the tenant guard forces the reserved test tenant.
|
||||
|
||||
Example:
|
||||
user = get_user() # "llm_tester" (dev) or "jpmschweitzer" (prod)
|
||||
@@ -55,7 +92,7 @@ def get_user() -> str:
|
||||
user = current_user.get()
|
||||
if user == _USER_NOT_SET:
|
||||
return get_default_user()
|
||||
return user
|
||||
return apply_tenant_guard(user)
|
||||
|
||||
|
||||
def get_conversation_id() -> str | None:
|
||||
|
||||
@@ -273,65 +273,6 @@ class HouseholdRegistry:
|
||||
|
||||
return tools
|
||||
|
||||
def get_streaming_delegation_tools(self, names: list[str]) -> list[Any]:
|
||||
"""
|
||||
Get streaming delegation wrapper tools for specified capabilities.
|
||||
|
||||
Similar to get_delegation_tools() but returns streaming wrappers
|
||||
that yield butler-perspective think messages during execution.
|
||||
|
||||
These wrappers emit think slugs like:
|
||||
- "Allow me to consult the archives, sir."
|
||||
- "The Librarian has compiled the relevant findings."
|
||||
|
||||
Args:
|
||||
names: List of member names to include
|
||||
|
||||
Returns:
|
||||
List of streaming delegation wrappers and/or raw tools
|
||||
|
||||
Example:
|
||||
>>> tools = registry.get_streaming_delegation_tools(["librarian"])
|
||||
>>> async for chunk in tools[0](task="Search for Docker"):
|
||||
... print(chunk) # Yields think messages then result
|
||||
"""
|
||||
from src.agents.delegation import STREAMING_DELEGATION_WRAPPERS
|
||||
|
||||
tools = []
|
||||
for name in names:
|
||||
member = self._members.get(name)
|
||||
if not member:
|
||||
logger.warning(
|
||||
"household_member_not_found",
|
||||
requested_name=name,
|
||||
available_names=list(self._members.keys()),
|
||||
)
|
||||
continue
|
||||
|
||||
# Check if this member has a streaming delegation wrapper
|
||||
if name in STREAMING_DELEGATION_WRAPPERS and member.agent is not None:
|
||||
tools.append(STREAMING_DELEGATION_WRAPPERS[name])
|
||||
logger.debug(
|
||||
"streaming_delegation_wrapper_added",
|
||||
member=name,
|
||||
)
|
||||
else:
|
||||
# No agent = direct tools (e.g., tatlock_core)
|
||||
tools.extend(member.tools)
|
||||
logger.debug(
|
||||
"raw_tools_added",
|
||||
member=name,
|
||||
tool_count=len(member.tools),
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"streaming_delegation_tools_created",
|
||||
requested_members=names,
|
||||
total_tools=len(tools),
|
||||
)
|
||||
|
||||
return tools
|
||||
|
||||
def list_members(self) -> list[str]:
|
||||
"""
|
||||
List all registered member names.
|
||||
|
||||
+38
-3
@@ -9,13 +9,43 @@ from src.agents.biographer import register_biographer
|
||||
from src.agents.housekeeper import register_housekeeper
|
||||
from src.agents.librarian import register_librarian
|
||||
from src.agents.tatlock_core import TATLOCK_CORE_CAPABILITY, tatlock_core_tools
|
||||
from src.anthropic.model_selector import check_claude_health, get_model_info
|
||||
from src.anthropic.model_selector import (
|
||||
check_claude_health,
|
||||
check_ollama_health,
|
||||
get_model_info,
|
||||
)
|
||||
from src.core.config import Environment, config
|
||||
from src.core.household_registry import get_household_registry
|
||||
from src.core.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def log_tenant_guard() -> None:
|
||||
"""
|
||||
Emit one loud startup log line stating the effective tenant.
|
||||
|
||||
In non-production environments the tenant guard forces the reserved
|
||||
test tenant regardless of DEFAULT_USER misconfiguration - this line
|
||||
makes that override visible at startup.
|
||||
"""
|
||||
if config.ENVIRONMENT == Environment.PRODUCTION:
|
||||
logger.info(
|
||||
"tenant_guard_production",
|
||||
environment=config.ENVIRONMENT.value,
|
||||
tenant=config.effective_default_user,
|
||||
)
|
||||
return
|
||||
|
||||
logger.warning(
|
||||
"tenant_guard_active",
|
||||
environment=config.ENVIRONMENT.value,
|
||||
forced_tenant=config.effective_default_user,
|
||||
default_user_overridden=config.tenant_forced,
|
||||
configured_default_user=config.DEFAULT_USER,
|
||||
)
|
||||
|
||||
|
||||
def register_household_members():
|
||||
"""
|
||||
Register all household members with the registry.
|
||||
@@ -87,7 +117,7 @@ async def initialize_application():
|
||||
Initialize the application.
|
||||
|
||||
Performs all startup tasks:
|
||||
1. Check Claude API health (for backend selection)
|
||||
1. Check Ollama (primary) and Claude (fallback) health for backend selection
|
||||
2. Register household members
|
||||
3. (Future) Initialize connections
|
||||
|
||||
@@ -95,13 +125,18 @@ async def initialize_application():
|
||||
"""
|
||||
logger.info("application_initialization_starting")
|
||||
|
||||
# Check Claude API health for backend selection
|
||||
# Tenant isolation guard: state the effective tenant loudly
|
||||
log_tenant_guard()
|
||||
|
||||
# Check backend health: Ollama is primary, Claude is the fallback
|
||||
await check_ollama_health()
|
||||
await check_claude_health()
|
||||
model_info = get_model_info()
|
||||
logger.info(
|
||||
"model_backend_configured",
|
||||
backend=model_info["backend"],
|
||||
model=model_info["model"],
|
||||
ollama_available=model_info["ollama_available"],
|
||||
claude_available=model_info["claude_available"],
|
||||
)
|
||||
|
||||
|
||||
+21
-10
@@ -40,14 +40,21 @@ class TatlockOllamaProvider(OllamaProvider):
|
||||
# Override the client with our sanitized version
|
||||
self._openai_client = _SanitizedAsyncOpenAI(base_url=base_url)
|
||||
|
||||
logger.debug("tatlock_ollama_provider_created", base_url=base_url)
|
||||
logger.debug(
|
||||
"tatlock_ollama_provider_created",
|
||||
base_url=base_url,
|
||||
timeout=config.OLLAMA_TIMEOUT,
|
||||
)
|
||||
|
||||
|
||||
class _SanitizedAsyncOpenAI(AsyncOpenAI):
|
||||
"""AsyncOpenAI client that sanitizes messages before sending."""
|
||||
|
||||
def __init__(self, **kwargs: Any):
|
||||
# Ollama doesn't need an API key
|
||||
# Ollama doesn't need an API key. Cap each LLM call at the
|
||||
# configured Ollama timeout instead of the SDK default (~600s),
|
||||
# so one stuck request cannot eat the whole delegation budget.
|
||||
kwargs.setdefault("timeout", float(config.OLLAMA_TIMEOUT))
|
||||
super().__init__(api_key="ollama", **kwargs)
|
||||
|
||||
@property
|
||||
@@ -106,14 +113,18 @@ def _sanitize_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
for msg in messages:
|
||||
msg_copy = dict(msg)
|
||||
|
||||
# Fix null content in assistant messages with tool calls
|
||||
if msg_copy.get("role") == "assistant":
|
||||
if msg_copy.get("content") is None and msg_copy.get("tool_calls"):
|
||||
msg_copy["content"] = ""
|
||||
logger.debug(
|
||||
"sanitized_null_content",
|
||||
tool_call_count=len(msg_copy["tool_calls"]),
|
||||
)
|
||||
# Fix null content in ANY message: Ollama rejects content: null with
|
||||
# "invalid message content type: <nil>". The tool-call-only assistant
|
||||
# case is the common one, but gemma thinking-only turns produce
|
||||
# assistant messages with null content and NO tool_calls, which
|
||||
# previously slipped through and 400'd the whole agent run.
|
||||
if "content" in msg_copy and msg_copy.get("content") is None:
|
||||
msg_copy["content"] = ""
|
||||
logger.debug(
|
||||
"sanitized_null_content",
|
||||
role=msg_copy.get("role"),
|
||||
tool_call_count=len(msg_copy.get("tool_calls") or []),
|
||||
)
|
||||
|
||||
sanitized.append(msg_copy)
|
||||
|
||||
|
||||
+91
-50
@@ -6,32 +6,32 @@ Tracks conversation history for analytics and future vector memory.
|
||||
Integrates with Steward preprocessing for Phase 2 two-tier architecture.
|
||||
"""
|
||||
|
||||
import time
|
||||
import asyncio
|
||||
import re
|
||||
import secrets
|
||||
from typing import AsyncGenerator
|
||||
import time
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
from src.agents.delegation import build_delegation_context, get_think_message
|
||||
from src.agents.registry import ModelRegistry
|
||||
from src.agents.steward.schemas import StewardRecommendation
|
||||
from src.core.context import current_conversation, current_user, get_default_user
|
||||
from src.core.logging_config import get_logger
|
||||
from src.core.preprocessing import preprocess_request
|
||||
from src.core.tool_tracking import ToolCallTracker
|
||||
from src.core.tracing import SpanType, end_trace, start_span, start_trace
|
||||
from src.responses.context import ContextWindow
|
||||
from src.responses.history import ConversationHistory
|
||||
from src.responses.schemas import (
|
||||
FunctionCallOutputItem,
|
||||
MessageOutputItem,
|
||||
OutputTextContent,
|
||||
ReasoningOutputItem,
|
||||
Response,
|
||||
ResponseRequest,
|
||||
ResponseUsage,
|
||||
MessageOutputItem,
|
||||
ReasoningOutputItem,
|
||||
FunctionCallOutputItem,
|
||||
OutputTextContent,
|
||||
)
|
||||
from src.responses.streaming import StreamingCoordinator
|
||||
from src.responses.history import ConversationHistory
|
||||
from src.responses.context import ContextWindow
|
||||
from src.core.preprocessing import preprocess_request
|
||||
from src.core.tool_tracking import ToolCallTracker
|
||||
from src.core.logging_config import get_logger
|
||||
from src.core.tracing import start_trace, end_trace, start_span, SpanType
|
||||
from src.core.context import current_user, current_conversation, get_default_user
|
||||
from src.agents.steward.schemas import StewardRecommendation
|
||||
|
||||
import re
|
||||
import asyncio
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -63,7 +63,8 @@ async def _execute_single_delegation(
|
||||
agent_name: str,
|
||||
task: str,
|
||||
tracker: "ToolCallTracker",
|
||||
) -> tuple[str, str]:
|
||||
context: str = "",
|
||||
) -> tuple[str, str, bool]:
|
||||
"""
|
||||
Execute a single delegation to an agent.
|
||||
|
||||
@@ -71,36 +72,38 @@ async def _execute_single_delegation(
|
||||
agent_name: Name of agent (biographer, librarian, housekeeper)
|
||||
task: Task description
|
||||
tracker: Tool call tracker
|
||||
context: Trimmed conversation context for the expert
|
||||
|
||||
Returns:
|
||||
tuple: (agent_name, result_summary)
|
||||
tuple: (agent_name, result_summary, success). On failure the
|
||||
result summary is a curated user-safe sentence.
|
||||
"""
|
||||
import time
|
||||
start_time = time.time()
|
||||
|
||||
if agent_name == "biographer":
|
||||
from src.agents.delegation import delegate_to_biographer
|
||||
result = await delegate_to_biographer(task=task)
|
||||
result = await delegate_to_biographer(task=task, context=context)
|
||||
duration = time.time() - start_time
|
||||
await tracker.track_call("delegate_to_biographer", duration)
|
||||
return (agent_name, result.output)
|
||||
return (agent_name, result.output, result.success)
|
||||
|
||||
elif agent_name == "librarian":
|
||||
from src.agents.delegation import delegate_to_librarian
|
||||
result = await delegate_to_librarian(task=task)
|
||||
result = await delegate_to_librarian(task=task, context=context)
|
||||
duration = time.time() - start_time
|
||||
await tracker.track_call("delegate_to_librarian", duration)
|
||||
return (agent_name, result.output)
|
||||
return (agent_name, result.output, result.success)
|
||||
|
||||
elif agent_name == "housekeeper":
|
||||
from src.agents.delegation import delegate_to_housekeeper
|
||||
result = await delegate_to_housekeeper(task=task)
|
||||
result = await delegate_to_housekeeper(task=task, context=context)
|
||||
duration = time.time() - start_time
|
||||
await tracker.track_call("delegate_to_housekeeper", duration)
|
||||
return (agent_name, result.output)
|
||||
return (agent_name, result.output, result.success)
|
||||
|
||||
else:
|
||||
return (agent_name, f"Unknown agent: {agent_name}")
|
||||
return (agent_name, f"Unknown agent: {agent_name}", False)
|
||||
|
||||
|
||||
async def _handle_text_delegation(
|
||||
@@ -175,13 +178,39 @@ async def _handle_text_delegation(
|
||||
]
|
||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
# Combine results
|
||||
# asyncio.gather returns exactly one item per task; a length
|
||||
# mismatch would mean results are attributed to the wrong
|
||||
# agent, so fail loudly instead of mispairing silently.
|
||||
if len(results) != len(matches):
|
||||
logger.error(
|
||||
"delegation_result_count_mismatch",
|
||||
expected=len(matches),
|
||||
got=len(results),
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
return (
|
||||
"I apologize, sir. I was unable to complete the "
|
||||
"requested delegations."
|
||||
)
|
||||
|
||||
# Combine results (failures carry curated user-safe sentences)
|
||||
summaries = []
|
||||
for agent_name, result in results:
|
||||
if isinstance(result, Exception):
|
||||
summaries.append(f"**{agent_name}**: Error - {result}")
|
||||
for (agent, task), item in zip(matches, results, strict=True):
|
||||
agent_name = agent.lower()
|
||||
if isinstance(item, BaseException):
|
||||
logger.error(
|
||||
"delegation_failed",
|
||||
agent=agent_name,
|
||||
error=str(item),
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
summaries.append(
|
||||
f"**{agent_name}**: "
|
||||
f"{get_think_message(agent_name, task, 'error')}"
|
||||
)
|
||||
else:
|
||||
summaries.append(f"**{agent_name}**: {result}")
|
||||
_, output, _ = item
|
||||
summaries.append(f"**{agent_name}**: {output}")
|
||||
|
||||
return "\n\n".join(summaries)
|
||||
|
||||
@@ -197,20 +226,19 @@ async def _handle_text_delegation(
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
try:
|
||||
_, result = await _execute_single_delegation(
|
||||
_, output, _ = await _execute_single_delegation(
|
||||
agent_name, task, tracker
|
||||
)
|
||||
summaries.append(result)
|
||||
summaries.append(output)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
"delegation_failed",
|
||||
agent=agent_name,
|
||||
error=str(e),
|
||||
conversation_id=conversation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
summaries.append(
|
||||
f"I apologize, sir. Delegation to {agent_name} failed: {e}"
|
||||
)
|
||||
summaries.append(get_think_message(agent_name, task, "error"))
|
||||
|
||||
return "\n\n".join(summaries)
|
||||
|
||||
@@ -219,8 +247,9 @@ async def _handle_text_delegation(
|
||||
"text_delegation_failed",
|
||||
error=str(e),
|
||||
conversation_id=conversation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return f"I apologize, sir. I encountered an error processing delegations: {e}"
|
||||
return "I apologize, sir. I was unable to complete the requested delegations."
|
||||
|
||||
|
||||
async def _direct_delegation(
|
||||
@@ -254,13 +283,14 @@ async def _direct_delegation(
|
||||
results = []
|
||||
for agent in recommendation.recommended_capabilities:
|
||||
try:
|
||||
agent_name, result = await _execute_single_delegation(
|
||||
agent_name, result, success = await _execute_single_delegation(
|
||||
agent, user_message, tracker
|
||||
)
|
||||
results.append(result)
|
||||
logger.info(
|
||||
"direct_delegation_complete",
|
||||
agent=agent_name,
|
||||
success=success,
|
||||
result_preview=result[:100] if result else "empty",
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
@@ -270,8 +300,9 @@ async def _direct_delegation(
|
||||
agent=agent,
|
||||
error=str(e),
|
||||
conversation_id=conversation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
results.append(f"I apologize, sir. Delegation to {agent} failed: {e}")
|
||||
results.append(get_think_message(agent, user_message, "error"))
|
||||
|
||||
return "\n\n".join(results) if results else "I apologize, sir. No delegation results available."
|
||||
|
||||
@@ -281,6 +312,7 @@ async def _direct_delegation_with_results(
|
||||
recommendation: "StewardRecommendation",
|
||||
tracker: "ToolCallTracker",
|
||||
conversation_id: str,
|
||||
conversation_history: list | None = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Directly delegate to expert agents and return structured results.
|
||||
@@ -294,6 +326,7 @@ async def _direct_delegation_with_results(
|
||||
recommendation: Steward's recommendation
|
||||
tracker: Tool call tracker
|
||||
conversation_id: Conversation ID
|
||||
conversation_history: Prior turns, trimmed into expert context
|
||||
|
||||
Returns:
|
||||
dict: Orchestration results with expert_results, tool_outputs, etc.
|
||||
@@ -306,18 +339,21 @@ async def _direct_delegation_with_results(
|
||||
|
||||
expert_results = {}
|
||||
tools_called = []
|
||||
context = build_delegation_context(conversation_history)
|
||||
|
||||
for agent in recommendation.recommended_capabilities:
|
||||
try:
|
||||
agent_name, result = await _execute_single_delegation(
|
||||
agent, user_message, tracker
|
||||
agent_name, result, success = await _execute_single_delegation(
|
||||
agent, user_message, tracker, context=context
|
||||
)
|
||||
expert_results[agent_name] = result
|
||||
tools_called.append(f"delegate_to_{agent_name}")
|
||||
if success:
|
||||
tools_called.append(f"delegate_to_{agent_name}")
|
||||
|
||||
logger.info(
|
||||
"direct_delegation_result",
|
||||
agent=agent_name,
|
||||
success=success,
|
||||
result_preview=result[:100] if result else "empty",
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
@@ -327,8 +363,9 @@ async def _direct_delegation_with_results(
|
||||
agent=agent,
|
||||
error=str(e),
|
||||
conversation_id=conversation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
expert_results[agent] = f"Error: {e}"
|
||||
expert_results[agent] = get_think_message(agent, user_message, "error")
|
||||
|
||||
return {
|
||||
"tools_called": tools_called,
|
||||
@@ -439,7 +476,7 @@ async def create_response(request: ResponseRequest) -> Response:
|
||||
user_input = _extract_user_input(request.input)
|
||||
|
||||
# Start trace
|
||||
trace = start_trace(
|
||||
start_trace(
|
||||
conversation_id=conversation_id,
|
||||
user=effective_user,
|
||||
request={
|
||||
@@ -451,7 +488,7 @@ async def create_response(request: ResponseRequest) -> Response:
|
||||
)
|
||||
|
||||
# Start service span
|
||||
service_span = start_span(
|
||||
start_span(
|
||||
"create_response",
|
||||
SpanType.ROUTER,
|
||||
metadata={"model": request.model, "user": effective_user},
|
||||
@@ -509,7 +546,7 @@ async def create_response(request: ResponseRequest) -> Response:
|
||||
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
except Exception:
|
||||
end_trace(status="error")
|
||||
raise
|
||||
|
||||
@@ -550,7 +587,7 @@ async def create_response_with_steward(request: ResponseRequest) -> Response:
|
||||
user_input = _extract_user_input(request.input)
|
||||
|
||||
# Start trace
|
||||
trace = start_trace(
|
||||
start_trace(
|
||||
conversation_id=conversation_id,
|
||||
user=effective_user,
|
||||
request={
|
||||
@@ -562,7 +599,7 @@ async def create_response_with_steward(request: ResponseRequest) -> Response:
|
||||
)
|
||||
|
||||
# Start service span
|
||||
service_span = start_span(
|
||||
start_span(
|
||||
"create_response_with_steward",
|
||||
SpanType.ROUTER,
|
||||
metadata={"model": request.model, "user": effective_user},
|
||||
@@ -617,7 +654,11 @@ async def create_response_with_steward(request: ResponseRequest) -> Response:
|
||||
if delegation_only:
|
||||
# Direct delegation path - collect results then synthesize
|
||||
orchestration_results = await _direct_delegation_with_results(
|
||||
effective_query, enriched.recommendation, tracker, conversation_id
|
||||
effective_query,
|
||||
enriched.recommendation,
|
||||
tracker,
|
||||
conversation_id,
|
||||
conversation_history=conversation_history,
|
||||
)
|
||||
else:
|
||||
# Phase 1: Orchestrate tool calls
|
||||
@@ -706,7 +747,7 @@ async def create_response_with_steward(request: ResponseRequest) -> Response:
|
||||
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
except Exception:
|
||||
end_trace(status="error")
|
||||
raise
|
||||
|
||||
|
||||
+95
-58
@@ -8,14 +8,22 @@ Handles Server-Sent Events (SSE) streaming with proper event types:
|
||||
- response.done
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
from typing import Literal, AsyncGenerator
|
||||
import json
|
||||
import time
|
||||
from collections.abc import AsyncGenerator
|
||||
from enum import Enum
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from src.agents.registry import ModelRegistry
|
||||
from src.core.logging_config import get_logger
|
||||
from src.core.models import CustomBaseModel
|
||||
from src.responses.schemas import Response
|
||||
from src.agents.registry import ModelRegistry
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from src.agents.steward.schemas import StewardRecommendation
|
||||
from src.core.tool_tracking import ToolCallTracker
|
||||
from src.responses.schemas import ResponseRequest
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -131,18 +139,17 @@ class StreamingCoordinator:
|
||||
Yields:
|
||||
StreamEvent: Stream of SSE events
|
||||
"""
|
||||
from src.responses.service import (
|
||||
_calculate_usage,
|
||||
generate_id,
|
||||
_conversation_history,
|
||||
_direct_delegation_with_results,
|
||||
)
|
||||
import asyncio
|
||||
|
||||
from src.agents.tatlock import TatlockAgent
|
||||
from src.core.preprocessing import preprocess_request
|
||||
from src.core.tool_tracking import ToolCallTracker
|
||||
from src.responses.schemas import MessageOutputItem, ReasoningOutputItem, OutputTextContent
|
||||
from src.agents.tatlock import TatlockAgent
|
||||
from src.agents.delegation import get_think_message, STREAMING_DELEGATION_WRAPPERS
|
||||
import asyncio
|
||||
from src.responses.schemas import MessageOutputItem, OutputTextContent
|
||||
from src.responses.service import (
|
||||
_calculate_usage,
|
||||
_conversation_history,
|
||||
generate_id,
|
||||
)
|
||||
|
||||
output_items = []
|
||||
|
||||
@@ -182,20 +189,18 @@ class StreamingCoordinator:
|
||||
tatlock = TatlockAgent()
|
||||
|
||||
if delegation_only:
|
||||
# Direct delegation path with streaming think slugs
|
||||
orchestration_results = await self._stream_direct_delegation(
|
||||
# Direct delegation path - think slugs stream in real time,
|
||||
# BEFORE and after each expert runs (not after the fact)
|
||||
orchestration_results: dict = {}
|
||||
async for event in self._stream_direct_delegation(
|
||||
user_message=user_message,
|
||||
recommendation=enriched.recommendation,
|
||||
tracker=tracker,
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
|
||||
# Stream think slugs that were collected during delegation
|
||||
# Each think message is complete, so we signal done after each
|
||||
for think_msg in orchestration_results.get("think_messages", []):
|
||||
yield ReasoningSummaryDelta(delta=think_msg)
|
||||
yield ReasoningSummaryDone()
|
||||
await asyncio.sleep(0.05)
|
||||
conversation_history=conversation_history,
|
||||
results=orchestration_results,
|
||||
):
|
||||
yield event
|
||||
|
||||
else:
|
||||
# Phase 1: Orchestrate tool calls
|
||||
@@ -265,47 +270,66 @@ class StreamingCoordinator:
|
||||
recommendation: "StewardRecommendation", # type: ignore
|
||||
tracker: "ToolCallTracker", # type: ignore
|
||||
conversation_id: str,
|
||||
) -> dict:
|
||||
conversation_history: list | None = None,
|
||||
results: dict | None = None,
|
||||
) -> AsyncGenerator[StreamEvent, None]:
|
||||
"""
|
||||
Execute direct delegation with streaming think messages.
|
||||
Execute direct delegation, streaming think messages in real time.
|
||||
|
||||
Collects think messages as delegations execute for streaming to client.
|
||||
An async generator: the "start" think message for each expert is
|
||||
yielded BEFORE its research runs (so the user sees 'Allow me to
|
||||
consult the archives, sir.' while waiting), and the success/error
|
||||
message right after it finishes.
|
||||
|
||||
Args:
|
||||
user_message: User's request
|
||||
recommendation: Steward's recommendation
|
||||
tracker: Tool call tracker
|
||||
conversation_id: Conversation ID
|
||||
conversation_history: Prior turns, trimmed into expert context
|
||||
results: Mutable dict populated with orchestration results
|
||||
(expert_results, tools_called, think_messages, ...)
|
||||
|
||||
Returns:
|
||||
dict: Orchestration results with think_messages list
|
||||
Yields:
|
||||
StreamEvent: Reasoning summary events as delegation progresses
|
||||
"""
|
||||
import time as time_module
|
||||
|
||||
from src.agents.delegation import (
|
||||
get_think_message,
|
||||
delegate_to_librarian,
|
||||
build_delegation_context,
|
||||
delegate_to_biographer,
|
||||
delegate_to_housekeeper,
|
||||
delegate_to_librarian,
|
||||
get_think_message,
|
||||
)
|
||||
import time as time_module
|
||||
|
||||
expert_results = {}
|
||||
tools_called = []
|
||||
think_messages = []
|
||||
context = build_delegation_context(conversation_history)
|
||||
|
||||
for agent in recommendation.recommended_capabilities:
|
||||
# Emit start think message
|
||||
# Emit start think message BEFORE the expert runs
|
||||
start_msg = get_think_message(agent, user_message, "start")
|
||||
think_messages.append(start_msg + "\n")
|
||||
yield ReasoningSummaryDelta(delta=start_msg + "\n")
|
||||
yield ReasoningSummaryDone()
|
||||
|
||||
start_time = time_module.time()
|
||||
try:
|
||||
# Execute delegation
|
||||
if agent == "librarian":
|
||||
result = await delegate_to_librarian(task=user_message)
|
||||
result = await delegate_to_librarian(
|
||||
task=user_message, context=context
|
||||
)
|
||||
elif agent == "biographer":
|
||||
result = await delegate_to_biographer(task=user_message)
|
||||
result = await delegate_to_biographer(
|
||||
task=user_message, context=context
|
||||
)
|
||||
elif agent == "housekeeper":
|
||||
result = await delegate_to_housekeeper(task=user_message)
|
||||
result = await delegate_to_housekeeper(
|
||||
task=user_message, context=context
|
||||
)
|
||||
else:
|
||||
result = None
|
||||
|
||||
@@ -316,27 +340,39 @@ class StreamingCoordinator:
|
||||
expert_results[agent] = result.output
|
||||
tools_called.append(f"delegate_to_{agent}")
|
||||
# Emit success think message
|
||||
success_msg = get_think_message(agent, user_message, "success")
|
||||
think_messages.append(success_msg + "\n")
|
||||
phase_msg = get_think_message(agent, user_message, "success")
|
||||
else:
|
||||
error_msg = result.error if result else "Unknown error"
|
||||
expert_results[agent] = f"Error: {error_msg}"
|
||||
# Emit error think message
|
||||
error_think = get_think_message(agent, user_message, "error")
|
||||
think_messages.append(error_think + "\n")
|
||||
# Failed delegations carry a curated user-safe sentence
|
||||
# in output; exception detail is already in the logs.
|
||||
phase_msg = get_think_message(agent, user_message, "error")
|
||||
if result and result.output:
|
||||
expert_results[agent] = result.output
|
||||
else:
|
||||
expert_results[agent] = phase_msg
|
||||
|
||||
except Exception as e:
|
||||
expert_results[agent] = f"Error: {e}"
|
||||
error_think = get_think_message(agent, user_message, "error")
|
||||
think_messages.append(error_think + "\n")
|
||||
logger.error(
|
||||
"direct_delegation_stream_error",
|
||||
agent=agent,
|
||||
error=str(e),
|
||||
conversation_id=conversation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
phase_msg = get_think_message(agent, user_message, "error")
|
||||
expert_results[agent] = phase_msg
|
||||
|
||||
return {
|
||||
"tools_called": tools_called,
|
||||
"expert_results": expert_results,
|
||||
"tool_outputs": {},
|
||||
"raw_output": "",
|
||||
"think_messages": think_messages,
|
||||
}
|
||||
think_messages.append(phase_msg + "\n")
|
||||
yield ReasoningSummaryDelta(delta=phase_msg + "\n")
|
||||
yield ReasoningSummaryDone()
|
||||
|
||||
if results is not None:
|
||||
results.update({
|
||||
"tools_called": tools_called,
|
||||
"expert_results": expert_results,
|
||||
"tool_outputs": {},
|
||||
"raw_output": "",
|
||||
"think_messages": think_messages,
|
||||
})
|
||||
|
||||
async def stream_response(
|
||||
self,
|
||||
@@ -361,9 +397,10 @@ class StreamingCoordinator:
|
||||
event: response.done
|
||||
data: {"response": {...}}
|
||||
"""
|
||||
from src.responses.service import _calculate_usage, generate_id
|
||||
import asyncio
|
||||
|
||||
from src.responses.service import _calculate_usage, generate_id
|
||||
|
||||
output_items = []
|
||||
last_message_text = "" # Track last streamed message text to compute deltas
|
||||
|
||||
@@ -488,10 +525,10 @@ class StreamingCoordinator:
|
||||
def _convert_output_items(self, items: list) -> list:
|
||||
"""Convert agent OutputItem objects to schema OutputItem objects."""
|
||||
from src.responses.schemas import (
|
||||
MessageOutputItem,
|
||||
ReasoningOutputItem,
|
||||
FunctionCallOutputItem,
|
||||
MessageOutputItem,
|
||||
OutputTextContent,
|
||||
ReasoningOutputItem,
|
||||
)
|
||||
|
||||
converted = []
|
||||
@@ -521,9 +558,9 @@ class StreamingCoordinator:
|
||||
def _create_error_event(self, error: Exception) -> ErrorEvent:
|
||||
"""Create error event from exception."""
|
||||
from src.core.exceptions import (
|
||||
RateLimitError,
|
||||
ContextLengthError,
|
||||
AppException,
|
||||
ContextLengthError,
|
||||
RateLimitError,
|
||||
)
|
||||
|
||||
if isinstance(error, RateLimitError):
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
{
|
||||
"query": "home server infrastructure",
|
||||
"keywords": {
|
||||
"core_keywords": [
|
||||
"home",
|
||||
"server",
|
||||
"infrastructure"
|
||||
],
|
||||
"entities": [],
|
||||
"synonyms": {},
|
||||
"expansions": {}
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"source_type": "wiki",
|
||||
"title": "Tower of Joy - AI Butler System",
|
||||
"content": "",
|
||||
"url": null,
|
||||
"page_id": 146,
|
||||
"page_path": "users/jpmschweitzer/projects/tower-of-joy",
|
||||
"paperless_id": null,
|
||||
"rrf_score": 0.01639344262295082,
|
||||
"final_rank": 1,
|
||||
"sources": [
|
||||
"graph"
|
||||
],
|
||||
"related_dossiers": [
|
||||
{
|
||||
"page_id": 161,
|
||||
"title": "Library Desk - Knowledge Management",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/applications/library-desk",
|
||||
"tag": "ai",
|
||||
"shared_entities": 13
|
||||
},
|
||||
{
|
||||
"page_id": 161,
|
||||
"title": "Library Desk - Knowledge Management",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/applications/library-desk",
|
||||
"tag": "dossier:tatlock",
|
||||
"shared_entities": 13
|
||||
},
|
||||
{
|
||||
"page_id": 161,
|
||||
"title": "Library Desk - Knowledge Management",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/applications/library-desk",
|
||||
"tag": "applications",
|
||||
"shared_entities": 13
|
||||
},
|
||||
{
|
||||
"page_id": 167,
|
||||
"title": "Qdrant - Vector Database",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/applications/qdrant",
|
||||
"tag": "vector",
|
||||
"shared_entities": 12
|
||||
},
|
||||
{
|
||||
"page_id": 167,
|
||||
"title": "Qdrant - Vector Database",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/applications/qdrant",
|
||||
"tag": "dossier:tatlock",
|
||||
"shared_entities": 12
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"entity_matches": 3,
|
||||
"matched_entities": [
|
||||
"Infrastructure Services",
|
||||
"Infrastructure Layer\nThe"
|
||||
],
|
||||
"engine": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"source_type": "web",
|
||||
"title": "What do I need to start a home server ? Can I go in almost blind",
|
||||
"content": "You don't need industrial grade hardware to be a server. You may get better reliability and management options from that, but they can all run\u00a0...",
|
||||
"url": "https://www.reddit.com/r/HomeServer/comments/1rx7udl/what_do_i_need_to_start_a_home_server_can_i_go_in/",
|
||||
"page_id": null,
|
||||
"page_path": null,
|
||||
"paperless_id": null,
|
||||
"rrf_score": 0.01639344262295082,
|
||||
"final_rank": 2,
|
||||
"sources": [
|
||||
"web"
|
||||
],
|
||||
"related_dossiers": [],
|
||||
"metadata": {
|
||||
"entity_matches": null,
|
||||
"matched_entities": null,
|
||||
"engine": "startpage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"source_type": "wiki",
|
||||
"title": "PostgreSQL Shared - Database Server",
|
||||
"content": "",
|
||||
"url": null,
|
||||
"page_id": 148,
|
||||
"page_path": "users/jpmschweitzer/projects/tower-of-joy/infrastructure/postgres",
|
||||
"paperless_id": null,
|
||||
"rrf_score": 0.016129032258064516,
|
||||
"final_rank": 3,
|
||||
"sources": [
|
||||
"graph"
|
||||
],
|
||||
"related_dossiers": [
|
||||
{
|
||||
"page_id": 149,
|
||||
"title": "Redis Shared - Cache and Session Store",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/infrastructure/redis",
|
||||
"tag": "infrastructure",
|
||||
"shared_entities": 5
|
||||
},
|
||||
{
|
||||
"page_id": 149,
|
||||
"title": "Redis Shared - Cache and Session Store",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/infrastructure/redis",
|
||||
"tag": "dossier:tatlock",
|
||||
"shared_entities": 5
|
||||
},
|
||||
{
|
||||
"page_id": 149,
|
||||
"title": "Redis Shared - Cache and Session Store",
|
||||
"path": "users/jpmschweitzer/projects/tower-of-joy/infrastructure/redis",
|
||||
"tag": "cache",
|
||||
"shared_entities": 5
|
||||
},
|
||||
{
|
||||
"page_id": 105,
|
||||
"title": "Google Cloud Platform",
|
||||
"path": "users/jpmschweitzer/technology/cloud-platforms/gcp",
|
||||
"tag": "technology",
|
||||
"shared_entities": 4
|
||||
},
|
||||
{
|
||||
"page_id": 105,
|
||||
"title": "Google Cloud Platform",
|
||||
"path": "users/jpmschweitzer/technology/cloud-platforms/gcp",
|
||||
"tag": "cloud-platforms",
|
||||
"shared_entities": 4
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"entity_matches": 1,
|
||||
"matched_entities": [
|
||||
"Database Server"
|
||||
],
|
||||
"engine": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"source_type": "web",
|
||||
"title": "25+ Must-Have Home Server Services for 2025 (Ultimate Guide)",
|
||||
"content": "I\u2019ve been running a home server setup for years now, and it\u2019s been an incredible journey of discovery, learning, and practical benefits.\nIf you\u2019re considering setting up a home server or looking to expand your existing home lab, you\u2019re in the right place.\nIn this comprehensive guide, I\u2019ll walk you through the essential services that can transform your home server from a simple file storage system into a powerful, versatile hub that enhances your digital life.\nFrom media streaming to home automation, security, productivity, and more \u2013 we\u2019ll cover it all.\nWhether you\u2019re a seasoned self-hosting veteran or just taking your first steps into home server territory, this guide will help you discover new possibilities and build a setup that perfectly suits your needs.\nFoundation Services: The Building Blocks\nBefore diving into specific applications, let\u2019s cover the fundamental services that form the backbone of any robust home server setup.\nThese core components provide the infrastructure for everything else to run smoothly.\nHypervisors & Virtualization Platforms\nHypervisors allow you to run multiple virtual machines on a single physical server, making them crucial for efficient resource utilization.\nProxmox VE: The Homelab Virtualization King\nProxmox is my top recommendation for home server virtualization.\nThis open-source solution combines KVM virtualization with LXC containers, a powerful web interface, and built-in features like clustering, backups, and storage management.\nProxmox gives you the ability to run both full virtual machines and lightweight containers on the same hardware.\nIt\u2019s been extremely reliable in my setup, and its active community provides excellent support.\nI\u2019ve been using Proxmox for about three years, after switching from ESXi.\nThe transition was straightforward, and I\u2019ve found it much more suitable for home lab use.\nYou can create clusters with multiple nodes, run HA setups, and even use distributed storage with Ceph.\nTrueNAS SCALE: Storage with Ad...",
|
||||
"url": "https://hostbor.com/25-must-have-home-server-services/",
|
||||
"page_id": null,
|
||||
"page_path": null,
|
||||
"paperless_id": null,
|
||||
"rrf_score": 0.016129032258064516,
|
||||
"final_rank": 4,
|
||||
"sources": [
|
||||
"web"
|
||||
],
|
||||
"related_dossiers": [],
|
||||
"metadata": {
|
||||
"entity_matches": null,
|
||||
"matched_entities": null,
|
||||
"engine": "duckduckgo"
|
||||
}
|
||||
},
|
||||
{
|
||||
"source_type": "wiki",
|
||||
"title": "Bazzite",
|
||||
"content": "",
|
||||
"url": null,
|
||||
"page_id": 108,
|
||||
"page_path": "users/jpmschweitzer/technology/linux_distributions/bazzite",
|
||||
"paperless_id": null,
|
||||
"rrf_score": 0.015873015873015872,
|
||||
"final_rank": 5,
|
||||
"sources": [
|
||||
"graph"
|
||||
],
|
||||
"related_dossiers": [
|
||||
{
|
||||
"page_id": 110,
|
||||
"title": "Zorin OS",
|
||||
"path": "users/jpmschweitzer/technology/linux_distributions/zorin_os",
|
||||
"tag": "technology",
|
||||
"shared_entities": 22
|
||||
},
|
||||
{
|
||||
"page_id": 110,
|
||||
"title": "Zorin OS",
|
||||
"path": "users/jpmschweitzer/technology/linux_distributions/zorin_os",
|
||||
"tag": "linux_distributions",
|
||||
"shared_entities": 22
|
||||
},
|
||||
{
|
||||
"page_id": 110,
|
||||
"title": "Zorin OS",
|
||||
"path": "users/jpmschweitzer/technology/linux_distributions/zorin_os",
|
||||
"tag": "zorin_os",
|
||||
"shared_entities": 22
|
||||
},
|
||||
{
|
||||
"page_id": 109,
|
||||
"title": "Linux",
|
||||
"path": "users/jpmschweitzer/technology/linux",
|
||||
"tag": "technology",
|
||||
"shared_entities": 21
|
||||
},
|
||||
{
|
||||
"page_id": 109,
|
||||
"title": "Linux",
|
||||
"path": "users/jpmschweitzer/technology/linux",
|
||||
"tag": "linux",
|
||||
"shared_entities": 21
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"entity_matches": 1,
|
||||
"matched_entities": [
|
||||
"Display Server"
|
||||
],
|
||||
"engine": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"context": "1. [WIKI] Tower of Joy - AI Butler System\n (no content)...\n Related research: ai, dossier:tatlock, applications\n\n2. [WEB] What do I need to start a home server ? Can I go in almost blind\n You don't need industrial grade hardware to be a server. You may get better reliability and management options from that, but they can all run\u00a0......\n\n3. [WIKI] PostgreSQL Shared - Database Server\n (no content)...\n Related research: infrastructure, dossier:tatlock, cache\n\n4. [WEB] 25+ Must-Have Home Server Services for 2025 (Ultimate Guide)\n I\u2019ve been running a home server setup for years now, and it\u2019s been an incredible journey of discovery, learning, and practical benefits.\nIf you\u2019re considering setting up a home server or looking to expand your existing home lab, you\u2019re in the right place.\nIn this comprehensive guide, I\u2019ll walk you t...\n\n5. [WIKI] Bazzite\n (no content)...\n Related research: technology, linux_distributions, zorin_os",
|
||||
"source_counts": {
|
||||
"graph": 3,
|
||||
"web": 2
|
||||
},
|
||||
"total_results": 5,
|
||||
"timing": {
|
||||
"query_enhancement_ms": 30.002593994140625,
|
||||
"vector_ms": 105.46708106994629,
|
||||
"graph_ms": 21.07977867126465,
|
||||
"web_ms": 1577.7764320373535,
|
||||
"volatile_ms": 0.0,
|
||||
"document_ms": 1.8155574798583984,
|
||||
"fusion_ms": 0.1761913299560547,
|
||||
"enrichment_ms": 14.33563232421875,
|
||||
"reranking_ms": 0.0002384185791015625,
|
||||
"persistence_ms": 33.80393981933594,
|
||||
"total_ms": 1624.767780303955
|
||||
},
|
||||
"config_used": {
|
||||
"vector_limit": 3,
|
||||
"graph_limit": 3,
|
||||
"web_limit": 2,
|
||||
"volatile_limit": 1,
|
||||
"document_limit": 2,
|
||||
"enable_vector": true,
|
||||
"enable_graph": true,
|
||||
"enable_web": true,
|
||||
"enable_volatile": false,
|
||||
"enable_documents": true,
|
||||
"enable_reranking": false,
|
||||
"enable_enrichment": true,
|
||||
"final_result_count": 6,
|
||||
"rrf_k": 60,
|
||||
"volatile_threshold": 0.8,
|
||||
"document_threshold": 0.6
|
||||
},
|
||||
"search_id": "01062bc7-ca65-4d9a-a210-e1a8f44b93c1"
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
"""
|
||||
Tests for structured failure behavior of The Librarian entry point.
|
||||
|
||||
run_librarian must raise AgentError on failure instead of returning
|
||||
error text as if it were research output, and the raised error must
|
||||
not leak exception detail (internal URLs etc.).
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.agents.librarian.agent import run_librarian
|
||||
from src.agents.protocol import AgentError
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestRunLibrarianFailures:
|
||||
"""run_librarian raises structured errors instead of returning text."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_librarian_raises_agent_error(self):
|
||||
"""Failures raise AgentError rather than returning error prose."""
|
||||
mock_agent = MagicMock()
|
||||
mock_agent.run = AsyncMock(
|
||||
side_effect=RuntimeError("Connection refused to http://internal:8089")
|
||||
)
|
||||
|
||||
with patch(
|
||||
"src.agents.librarian.agent.get_librarian_agent",
|
||||
return_value=mock_agent,
|
||||
):
|
||||
with pytest.raises(AgentError) as exc_info:
|
||||
await run_librarian(task="Find Docker docs")
|
||||
|
||||
assert exc_info.value.agent_name == "librarian"
|
||||
# Exception detail stays in logs only
|
||||
assert "internal" not in str(exc_info.value)
|
||||
assert "Connection refused" not in str(exc_info.value)
|
||||
@@ -2,22 +2,23 @@
|
||||
Tests for the Library-Desk HTTP client.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from src.agents.librarian.client import (
|
||||
LibraryDeskClient,
|
||||
Dossier,
|
||||
EntityLinking,
|
||||
GraphNode,
|
||||
HybridRAGResponse,
|
||||
HybridSearchResult,
|
||||
LibraryDeskClient,
|
||||
ResearchSummary,
|
||||
SmartCreateResponse,
|
||||
VectorSearchResult,
|
||||
WikiPage,
|
||||
WikiSearchResult,
|
||||
VectorSearchResult,
|
||||
GraphNode,
|
||||
Dossier,
|
||||
SmartCreateResponse,
|
||||
ResearchSummary,
|
||||
EntityLinking,
|
||||
)
|
||||
|
||||
|
||||
@@ -549,6 +550,181 @@ class TestSmartCreateWikiPage:
|
||||
assert result.research_summary.web_results == 0
|
||||
|
||||
|
||||
# All tenant-scoped client methods with minimal call kwargs. Used to sweep
|
||||
# the explicit-user contract: every library-desk request must carry a
|
||||
# non-empty user (the service is removing its server-side default).
|
||||
TENANT_SCOPED_METHODS = [
|
||||
("hybrid_search", {"query": "q"}),
|
||||
("search_wiki", {"query": "q"}),
|
||||
("get_wiki_page", {"page_id": 1}),
|
||||
("list_wiki_pages", {}),
|
||||
("create_wiki_page", {"title": "t", "path": "/p", "content": "c"}),
|
||||
("update_wiki_page", {"page_id": 1, "content": "c"}),
|
||||
("smart_create_wiki_page", {"topic": "t", "tags": ["x"]}),
|
||||
("list_dossiers", {}),
|
||||
("semantic_search", {"query": "q"}),
|
||||
("query_graph", {"cypher_query": "MATCH (n) RETURN n"}),
|
||||
("list_graph_nodes", {}),
|
||||
("get_graph_node", {"node_id": "n1"}),
|
||||
("search_web", {"query": "q"}),
|
||||
("extract_content", {"url": "http://example.com"}),
|
||||
("extract_content_batch", {"urls": ["http://example.com"]}),
|
||||
]
|
||||
|
||||
# One permissive response body that satisfies every method's parser
|
||||
# (extra keys are ignored by the pydantic models).
|
||||
UNIVERSAL_RESPONSE = {
|
||||
"id": 1,
|
||||
"path": "/p",
|
||||
"title": "T",
|
||||
"results": [],
|
||||
"pages": [],
|
||||
"dossiers": [],
|
||||
"records": [],
|
||||
"nodes": [],
|
||||
"keywords": [],
|
||||
"page": {"id": 1, "path": "/p", "title": "T"},
|
||||
"result": {"url": "http://example.com", "success": True},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestExplicitUserContract:
|
||||
"""Every library-desk request sends a non-empty user explicitly."""
|
||||
|
||||
def _wire_client(self):
|
||||
mock_response = MagicMock()
|
||||
mock_response.status_code = 200
|
||||
mock_response.json.return_value = UNIVERSAL_RESPONSE
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
|
||||
mock_httpx = AsyncMock(spec=httpx.AsyncClient)
|
||||
mock_httpx.get.return_value = mock_response
|
||||
mock_httpx.post.return_value = mock_response
|
||||
mock_httpx.put.return_value = mock_response
|
||||
|
||||
client = LibraryDeskClient(base_url="http://test:8089", api_key="k")
|
||||
client._client = mock_httpx
|
||||
return client, mock_httpx
|
||||
|
||||
def _sent_user(self, mock_httpx) -> str:
|
||||
"""Extract the user sent on the single outgoing request."""
|
||||
calls = (
|
||||
mock_httpx.get.call_args_list
|
||||
+ mock_httpx.post.call_args_list
|
||||
+ mock_httpx.put.call_args_list
|
||||
)
|
||||
assert len(calls) == 1, "expected exactly one outgoing request"
|
||||
kwargs = calls[0].kwargs
|
||||
params = kwargs.get("params") or {}
|
||||
payload = kwargs.get("json") or {}
|
||||
return params.get("user") or payload.get("user") or ""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("method_name,kwargs", TENANT_SCOPED_METHODS)
|
||||
async def test_user_from_context_is_sent_on_the_wire(
|
||||
self, method_name, kwargs
|
||||
):
|
||||
"""With no explicit user, the context user is resolved and sent."""
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
with patch(
|
||||
"src.agents.librarian.client.get_user", return_value="llm_tester"
|
||||
):
|
||||
await getattr(client, method_name)(**kwargs)
|
||||
|
||||
assert self._sent_user(mock_httpx) == "llm_tester"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("method_name,kwargs", TENANT_SCOPED_METHODS)
|
||||
async def test_explicit_user_is_sent_on_the_wire(self, method_name, kwargs):
|
||||
"""An explicitly passed user is sent unchanged."""
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
await getattr(client, method_name)(user="test_phase_b", **kwargs)
|
||||
|
||||
assert self._sent_user(mock_httpx) == "test_phase_b"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("method_name,kwargs", TENANT_SCOPED_METHODS)
|
||||
async def test_empty_context_user_fails_before_any_request(
|
||||
self, method_name, kwargs
|
||||
):
|
||||
"""An empty resolved user raises before any bytes hit the wire."""
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
with patch("src.agents.librarian.client.get_user", return_value=""):
|
||||
with pytest.raises(ValueError, match="non-empty user"):
|
||||
await getattr(client, method_name)(**kwargs)
|
||||
|
||||
mock_httpx.get.assert_not_called()
|
||||
mock_httpx.post.assert_not_called()
|
||||
mock_httpx.put.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_explicit_whitespace_user_is_rejected(self):
|
||||
"""A whitespace-only explicit user is rejected."""
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
with pytest.raises(ValueError, match="non-empty user"):
|
||||
await client.hybrid_search("q", user=" ")
|
||||
|
||||
mock_httpx.post.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_explicit_padded_user_is_stripped_on_the_wire(self):
|
||||
"""Padded explicit users are stripped, not sent verbatim."""
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
await client.hybrid_search("q", user=" llm_tester ")
|
||||
|
||||
assert self._sent_user(mock_httpx) == "llm_tester"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"explicit_user",
|
||||
["jpmschweitzer", "JPMSchweitzer", "jpmschweitzer.", " jpmschweitzer"],
|
||||
)
|
||||
@pytest.mark.parametrize("method_name,kwargs", TENANT_SCOPED_METHODS)
|
||||
async def test_explicit_production_tenant_is_guarded_in_dev(
|
||||
self, monkeypatch, method_name, kwargs, explicit_user
|
||||
):
|
||||
"""
|
||||
An explicit production-tenant argument (or a sanitization-collision
|
||||
variant) never reaches library-desk from a non-production
|
||||
environment - the client applies the same tenant guard as
|
||||
context resolution.
|
||||
"""
|
||||
from src.core import config as config_module
|
||||
from src.core.config import Environment
|
||||
|
||||
monkeypatch.setattr(
|
||||
config_module.config, "ENVIRONMENT", Environment.DEVELOPMENT
|
||||
)
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
await getattr(client, method_name)(user=explicit_user, **kwargs)
|
||||
|
||||
assert self._sent_user(mock_httpx) == "llm_tester"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_explicit_production_tenant_passes_through_in_prod(
|
||||
self, monkeypatch
|
||||
):
|
||||
"""In production the production tenant is sent unchanged."""
|
||||
from src.core import config as config_module
|
||||
from src.core.config import Environment
|
||||
|
||||
monkeypatch.setattr(
|
||||
config_module.config, "ENVIRONMENT", Environment.PRODUCTION
|
||||
)
|
||||
client, mock_httpx = self._wire_client()
|
||||
|
||||
await client.hybrid_search("q", user="jpmschweitzer")
|
||||
|
||||
assert self._sent_user(mock_httpx) == "jpmschweitzer"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestNewResponseModels:
|
||||
"""Tests for new response models."""
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
"""
|
||||
Tests for bounded retries, timeout wiring, and client reuse in
|
||||
LibraryDeskClient, plus ModelRetry escalation from the read tools.
|
||||
"""
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from pydantic_ai import ModelRetry
|
||||
|
||||
from src.agents.librarian.client import (
|
||||
LibraryDeskClient,
|
||||
library_client_session,
|
||||
)
|
||||
from src.agents.librarian.tools import (
|
||||
create_wiki_page,
|
||||
hybrid_search,
|
||||
search_wiki,
|
||||
)
|
||||
from src.core.config import config
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _no_backoff(monkeypatch):
|
||||
"""Skip the retry backoff sleep in tests."""
|
||||
monkeypatch.setattr("src.agents.librarian.client._RETRY_BACKOFF_SECONDS", 0)
|
||||
|
||||
|
||||
def _ok_response(payload: dict) -> MagicMock:
|
||||
response = MagicMock()
|
||||
response.status_code = 200
|
||||
response.json.return_value = payload
|
||||
response.raise_for_status = MagicMock()
|
||||
return response
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client_with_mock():
|
||||
client = LibraryDeskClient(base_url="http://test:8089", api_key="test-key")
|
||||
client._client = AsyncMock(spec=httpx.AsyncClient)
|
||||
return client
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestBoundedRetries:
|
||||
"""2-attempt retry for GETs and read-only POST /query/*, /rag/search."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_retries_once_on_transport_error(self, client_with_mock):
|
||||
mock_httpx = client_with_mock._client
|
||||
mock_httpx.get.side_effect = [
|
||||
httpx.ConnectError("Connection refused"),
|
||||
_ok_response({"results": []}),
|
||||
]
|
||||
|
||||
results = await client_with_mock.search_wiki("docker", user="u")
|
||||
|
||||
assert results == []
|
||||
assert mock_httpx.get.call_count == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_gives_up_after_two_attempts(self, client_with_mock):
|
||||
mock_httpx = client_with_mock._client
|
||||
mock_httpx.get.side_effect = httpx.ConnectError("Connection refused")
|
||||
|
||||
with pytest.raises(httpx.ConnectError):
|
||||
await client_with_mock.search_wiki("docker", user="u")
|
||||
|
||||
assert mock_httpx.get.call_count == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_query_hybrid_retries_on_503(self, client_with_mock):
|
||||
mock_httpx = client_with_mock._client
|
||||
bad = MagicMock()
|
||||
bad.status_code = 503
|
||||
mock_httpx.post.side_effect = [
|
||||
bad,
|
||||
_ok_response({"results": [], "keywords": {}, "context": ""}),
|
||||
]
|
||||
|
||||
response = await client_with_mock.hybrid_search("docker", user="u")
|
||||
|
||||
assert response.results == []
|
||||
assert mock_httpx.post.call_count == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wiki_write_is_never_retried(self, client_with_mock):
|
||||
"""POST /wiki/pages must not retry - it could duplicate pages."""
|
||||
mock_httpx = client_with_mock._client
|
||||
mock_httpx.post.side_effect = httpx.ConnectError("Connection refused")
|
||||
|
||||
with pytest.raises(httpx.ConnectError):
|
||||
await client_with_mock.create_wiki_page(
|
||||
title="T", path="/t", content="c", user="u"
|
||||
)
|
||||
|
||||
assert mock_httpx.post.call_count == 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_smart_create_is_never_retried(self, client_with_mock):
|
||||
mock_httpx = client_with_mock._client
|
||||
mock_httpx.post.side_effect = httpx.ConnectError("Connection refused")
|
||||
|
||||
with pytest.raises(httpx.ConnectError):
|
||||
await client_with_mock.smart_create_wiki_page(
|
||||
topic="T", tags=["x"], user="u"
|
||||
)
|
||||
|
||||
assert mock_httpx.post.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestTimeoutWiring:
|
||||
"""LIBRARY_DESK_TIMEOUT config replaces the hardcoded 60s/30s."""
|
||||
|
||||
def test_default_timeout_from_config(self):
|
||||
client = LibraryDeskClient()
|
||||
|
||||
assert client.timeout == config.LIBRARY_DESK_TIMEOUT
|
||||
|
||||
def test_explicit_timeout_wins(self):
|
||||
client = LibraryDeskClient(timeout=5)
|
||||
|
||||
assert client.timeout == 5
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestClientReuse:
|
||||
"""One shared HTTP connection per librarian run."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_clients_share_connection_inside_session(self):
|
||||
async with library_client_session():
|
||||
async with LibraryDeskClient() as c1:
|
||||
http1 = c1._client
|
||||
# shared connection survives client exit
|
||||
assert http1 is not None
|
||||
assert not http1.is_closed
|
||||
|
||||
async with LibraryDeskClient() as c2:
|
||||
assert c2._client is http1
|
||||
|
||||
# session close tears the shared connection down
|
||||
assert http1.is_closed
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_nested_sessions_are_noops(self):
|
||||
async with library_client_session():
|
||||
async with LibraryDeskClient() as c1:
|
||||
http1 = c1._client
|
||||
async with library_client_session():
|
||||
async with LibraryDeskClient() as c2:
|
||||
assert c2._client is http1
|
||||
# inner session exit must not close the shared connection
|
||||
assert not http1.is_closed
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_client_owns_connection_outside_session(self):
|
||||
async with LibraryDeskClient() as client:
|
||||
http_client = client._client
|
||||
|
||||
assert http_client.is_closed
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_custom_target_does_not_reuse_shared(self):
|
||||
async with library_client_session():
|
||||
async with LibraryDeskClient() as shared_client:
|
||||
shared_http = shared_client._client
|
||||
async with LibraryDeskClient(base_url="http://other:9999") as custom:
|
||||
assert custom._client is not shared_http
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestModelRetryEscalation:
|
||||
"""Read tools raise ModelRetry on transient errors so Agent(retries=2) engages."""
|
||||
|
||||
def _patched_client(self, mock_client):
|
||||
factory = MagicMock()
|
||||
factory.return_value.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
factory.return_value.__aexit__ = AsyncMock(return_value=None)
|
||||
return patch("src.agents.librarian.tools.LibraryDeskClient", factory)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_tool_raises_model_retry_on_transport_error(self):
|
||||
mock_client = AsyncMock()
|
||||
mock_client.hybrid_search.side_effect = httpx.ConnectError(
|
||||
"Connection refused"
|
||||
)
|
||||
|
||||
with self._patched_client(mock_client):
|
||||
with pytest.raises(ModelRetry):
|
||||
await hybrid_search("docker")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_tool_raises_model_retry_on_5xx(self):
|
||||
request = httpx.Request("GET", "http://test:8089/wiki/search")
|
||||
response = httpx.Response(502, request=request)
|
||||
mock_client = AsyncMock()
|
||||
mock_client.search_wiki.side_effect = httpx.HTTPStatusError(
|
||||
"bad gateway", request=request, response=response
|
||||
)
|
||||
|
||||
with self._patched_client(mock_client):
|
||||
with pytest.raises(ModelRetry):
|
||||
await search_wiki("docker")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_tool_returns_safe_message_on_non_transient(self):
|
||||
mock_client = AsyncMock()
|
||||
mock_client.hybrid_search.side_effect = ValueError("bad parse")
|
||||
|
||||
with self._patched_client(mock_client):
|
||||
result = await hybrid_search("docker")
|
||||
|
||||
assert "unable" in result
|
||||
assert "bad parse" not in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_tool_never_raises_model_retry(self):
|
||||
mock_client = AsyncMock()
|
||||
mock_client.create_wiki_page.side_effect = httpx.ConnectError(
|
||||
"Connection refused"
|
||||
)
|
||||
|
||||
with self._patched_client(mock_client):
|
||||
result = await create_wiki_page(
|
||||
title="T", path="/t", content="c", tags=["x"]
|
||||
)
|
||||
|
||||
assert "unable" in result
|
||||
assert "Connection refused" not in result
|
||||
@@ -0,0 +1,344 @@
|
||||
"""
|
||||
Contract tests for HybridRAG parsing against a recorded live response.
|
||||
|
||||
The fixture in fixtures/hybrid_query_recorded.json is a real (recorded)
|
||||
response from library-desk's POST /query/hybrid. These tests pin the
|
||||
field mapping (source_type/sources, rrf_score, context, per-item
|
||||
related_dossiers, keywords dict with nested synonyms) so a drift in
|
||||
either side shows up as a test failure instead of every result
|
||||
rendering as "unknown (score: 0.00)".
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from src.agents.librarian.client import HybridRAGResponse, LibraryDeskClient
|
||||
from src.agents.librarian.tools import SOURCE_ICONS, _coverage_note, hybrid_search
|
||||
|
||||
FIXTURE_PATH = Path(__file__).parent / "fixtures" / "hybrid_query_recorded.json"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def recorded_response() -> dict:
|
||||
"""Load the recorded /query/hybrid response."""
|
||||
return json.loads(FIXTURE_PATH.read_text())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client_with_recorded_response(recorded_response):
|
||||
"""LibraryDeskClient whose httpx client replays the recorded response."""
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = recorded_response
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
|
||||
mock_httpx = AsyncMock(spec=httpx.AsyncClient)
|
||||
mock_httpx.post.return_value = mock_response
|
||||
|
||||
client = LibraryDeskClient(base_url="http://test:8089", api_key="test-key")
|
||||
client._client = mock_httpx
|
||||
return client
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestHybridRAGContract:
|
||||
"""Contract tests for parsing the live /query/hybrid response shape."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sources_are_not_unknown(self, client_with_recorded_response):
|
||||
"""Every result maps source_type - nothing falls back to 'unknown'."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
assert isinstance(response, HybridRAGResponse)
|
||||
assert response.results, "recorded fixture must contain results"
|
||||
for result in response.results:
|
||||
assert result.source != "unknown"
|
||||
assert result.source in {"wiki", "web", "volatile", "document"}
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scores_are_non_zero(self, client_with_recorded_response):
|
||||
"""rrf_score maps to score - no silent 0.00 fallback."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
for result in response.results:
|
||||
assert result.score > 0.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sources_list_and_icons(self, client_with_recorded_response):
|
||||
"""Per-item sources list is parsed and every value has an icon."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
for result in response.results:
|
||||
assert result.sources, f"result '{result.title}' has empty sources"
|
||||
for source in result.sources:
|
||||
assert source in SOURCE_ICONS, f"no icon for source '{source}'"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_context_maps_to_formatted_context(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""Top-level 'context' field maps to formatted_context."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
assert response.formatted_context != ""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_keywords_and_synonyms_from_dict(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""keywords is a dict: core_keywords + nested synonyms map."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
assert response.keywords, "core_keywords should be extracted"
|
||||
assert all(isinstance(k, str) for k in response.keywords)
|
||||
# synonyms map in the fixture is empty, but must parse to a list
|
||||
assert isinstance(response.synonyms, list)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_per_item_related_dossiers(self, client_with_recorded_response):
|
||||
"""related_dossiers live per result and aggregate to unique titles."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
per_item = [d for r in response.results for d in r.related_dossiers]
|
||||
assert per_item, "recorded fixture contains per-item related_dossiers"
|
||||
for dossier in per_item:
|
||||
assert "title" in dossier
|
||||
assert "tag" in dossier
|
||||
|
||||
assert response.related_dossiers, "top-level titles are aggregated"
|
||||
assert len(response.related_dossiers) == len(set(response.related_dossiers))
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_payload_never_sends_zero_limits(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""The live service 422s on limits < 1; disabled legs use enable_* flags."""
|
||||
await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure",
|
||||
user="testuser",
|
||||
web_limit=0,
|
||||
document_limit=0,
|
||||
volatile_limit=0,
|
||||
)
|
||||
|
||||
payload = client_with_recorded_response._client.post.call_args.kwargs["json"]
|
||||
config = payload["config"]
|
||||
for key in (
|
||||
"vector_limit",
|
||||
"graph_limit",
|
||||
"web_limit",
|
||||
"document_limit",
|
||||
"volatile_limit",
|
||||
):
|
||||
assert config[key] >= 1
|
||||
assert config["enable_web"] is False
|
||||
assert config["enable_documents"] is False
|
||||
assert config["enable_volatile"] is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_user_always_sent_as_query_param(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""The tenant is always sent explicitly - library-desk is removing
|
||||
its server-side default, so a missing user would 422."""
|
||||
await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
params = client_with_recorded_response._client.post.call_args.kwargs[
|
||||
"params"
|
||||
]
|
||||
assert params["user"] == "testuser"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_counts_and_timing_parsed(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""source_counts and timing map into the response model."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
assert response.source_counts == {"graph": 3, "web": 2}
|
||||
assert response.timing.get("total_ms", 0) > 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_status_absent_is_tolerated(
|
||||
self, client_with_recorded_response
|
||||
):
|
||||
"""Recorded response predates source_status/degraded - defaults apply."""
|
||||
response = await client_with_recorded_response.hybrid_search(
|
||||
"home server infrastructure", user="testuser"
|
||||
)
|
||||
|
||||
assert response.source_status == {}
|
||||
assert response.degraded is False
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_source_status_parsed_when_present(self, recorded_response):
|
||||
"""Additive source_status/degraded fields parse when the service sends them."""
|
||||
enriched = dict(recorded_response)
|
||||
enriched["source_status"] = {
|
||||
"vector": "ok",
|
||||
"graph": "ok",
|
||||
"web": "failed",
|
||||
"volatile": "disabled",
|
||||
"documents": "ok",
|
||||
}
|
||||
enriched["degraded"] = True
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = enriched
|
||||
mock_response.raise_for_status = MagicMock()
|
||||
mock_httpx = AsyncMock(spec=httpx.AsyncClient)
|
||||
mock_httpx.post.return_value = mock_response
|
||||
|
||||
client = LibraryDeskClient(base_url="http://test:8089", api_key="test-key")
|
||||
client._client = mock_httpx
|
||||
|
||||
response = await client.hybrid_search("home server infrastructure", user="u")
|
||||
|
||||
assert response.degraded is True
|
||||
assert response.source_status["web"] == "failed"
|
||||
assert response.source_status["volatile"] == "disabled"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tool_renders_no_unknown_results(self, client_with_recorded_response, monkeypatch):
|
||||
"""The hybrid_search tool renders real sources and non-zero scores."""
|
||||
|
||||
class _Factory:
|
||||
def __call__(self):
|
||||
return self
|
||||
|
||||
async def __aenter__(self):
|
||||
return client_with_recorded_response
|
||||
|
||||
async def __aexit__(self, *args):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"src.agents.librarian.tools.LibraryDeskClient", _Factory()
|
||||
)
|
||||
|
||||
output = await hybrid_search("home server infrastructure")
|
||||
|
||||
assert "unknown" not in output
|
||||
assert "score: 0.00" not in output
|
||||
assert "•" not in output, "every source value should map to an icon"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestCoverageNote:
|
||||
"""Coverage note makes degraded searches visible to model and user."""
|
||||
|
||||
def _response(self, **kwargs) -> HybridRAGResponse:
|
||||
return HybridRAGResponse(**kwargs)
|
||||
|
||||
def test_no_note_when_all_legs_report(self):
|
||||
response = self._response(
|
||||
source_counts={
|
||||
"vector": 2,
|
||||
"graph": 1,
|
||||
"web": 2,
|
||||
"documents": 1,
|
||||
"volatile": 1,
|
||||
},
|
||||
)
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert note == ""
|
||||
|
||||
def test_note_when_enabled_leg_missing_from_counts(self):
|
||||
response = self._response(source_counts={"graph": 3, "web": 2})
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert "Coverage note" in note
|
||||
assert "documents" in note
|
||||
assert "volatile" in note
|
||||
# Always-on wiki legs are never inferred from count absence
|
||||
assert "vector" not in note
|
||||
assert "graph" not in note
|
||||
|
||||
def test_wiki_leg_absence_is_not_degradation(self):
|
||||
"""vector/graph missing from top-N counts is healthy ranking, not outage."""
|
||||
response = self._response(
|
||||
source_counts={"web": 2, "documents": 1, "volatile": 1}
|
||||
)
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert note == ""
|
||||
|
||||
def test_disabled_legs_are_not_reported_missing(self):
|
||||
response = self._response(source_counts={"vector": 2, "graph": 1})
|
||||
note = _coverage_note(
|
||||
response,
|
||||
include_web=False,
|
||||
include_documents=False,
|
||||
include_volatile=False,
|
||||
)
|
||||
assert note == ""
|
||||
|
||||
def test_note_prefers_source_status_failures(self):
|
||||
response = self._response(
|
||||
source_counts={"vector": 2, "graph": 1},
|
||||
source_status={
|
||||
"vector": "ok",
|
||||
"graph": "ok",
|
||||
"web": "failed",
|
||||
"volatile": "disabled",
|
||||
"documents": "ok",
|
||||
},
|
||||
degraded=True,
|
||||
)
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert "failed" in note
|
||||
assert "web" in note
|
||||
# disabled legs are not reported as failures
|
||||
assert "volatile" not in note
|
||||
|
||||
def test_no_note_when_status_all_ok(self):
|
||||
response = self._response(
|
||||
source_counts={"graph": 1},
|
||||
source_status={
|
||||
"vector": "ok",
|
||||
"graph": "ok",
|
||||
"web": "ok",
|
||||
"volatile": "ok",
|
||||
"documents": "ok",
|
||||
},
|
||||
degraded=False,
|
||||
)
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert note == ""
|
||||
|
||||
def test_degraded_without_named_failures(self):
|
||||
response = self._response(
|
||||
source_status={"vector": "ok", "graph": "ok"},
|
||||
degraded=True,
|
||||
)
|
||||
note = _coverage_note(
|
||||
response, include_web=True, include_documents=True, include_volatile=True
|
||||
)
|
||||
assert "partial" in note
|
||||
@@ -0,0 +1,49 @@
|
||||
"""
|
||||
Snapshot tests for the JSON schemas emitted for librarian tools.
|
||||
|
||||
Ollama's OpenAI-compatible API mishandles anyOf[X, null] parameter
|
||||
schemas, so tool parameters must use empty-string/empty-list sentinels
|
||||
translated to None inside the tool (same pattern as the biographer
|
||||
tools, commit 9d7ce39). This test fails if a X | None parameter ever
|
||||
creeps back in.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from pydantic_ai.tools import Tool
|
||||
|
||||
from src.agents.librarian.tools import LIBRARIAN_TOOLS
|
||||
|
||||
|
||||
def _nullable_anyof_paths(schema: object, path: str = "") -> list[str]:
|
||||
"""Recursively collect JSON-schema paths that are anyOf[..., null]."""
|
||||
offenders: list[str] = []
|
||||
if isinstance(schema, dict):
|
||||
any_of = schema.get("anyOf")
|
||||
if isinstance(any_of, list) and any(
|
||||
isinstance(sub, dict) and sub.get("type") == "null" for sub in any_of
|
||||
):
|
||||
offenders.append(path or "<root>")
|
||||
for key, value in schema.items():
|
||||
offenders.extend(_nullable_anyof_paths(value, f"{path}/{key}"))
|
||||
elif isinstance(schema, list):
|
||||
for i, item in enumerate(schema):
|
||||
offenders.extend(_nullable_anyof_paths(item, f"{path}[{i}]"))
|
||||
return offenders
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestLibrarianToolSchemas:
|
||||
"""All registered librarian tools emit Ollama-safe parameter schemas."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"tool_func", LIBRARIAN_TOOLS, ids=lambda f: f.__name__
|
||||
)
|
||||
def test_no_nullable_anyof_in_schema(self, tool_func):
|
||||
schema = Tool(tool_func).function_schema.json_schema
|
||||
|
||||
offenders = _nullable_anyof_paths(schema)
|
||||
|
||||
assert offenders == [], (
|
||||
f"{tool_func.__name__} emits anyOf[..., null] at {offenders}; "
|
||||
"use empty-string/empty-list sentinels instead of X | None"
|
||||
)
|
||||
@@ -5,21 +5,23 @@ Tests the tool functions that wrap the Library-Desk API,
|
||||
including the new web search and content extraction tools.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.agents.librarian.client import (
|
||||
BatchExtractionResponse,
|
||||
ContentExtractionResult,
|
||||
WebSearchResponse,
|
||||
WebSearchResult,
|
||||
WikiPage,
|
||||
)
|
||||
from src.agents.librarian.tools import (
|
||||
search_web,
|
||||
CLEAR_TAGS_SENTINEL,
|
||||
read_url,
|
||||
read_urls_batch,
|
||||
hybrid_search,
|
||||
search_wiki,
|
||||
)
|
||||
from src.agents.librarian.client import (
|
||||
WebSearchResult,
|
||||
WebSearchResponse,
|
||||
ContentExtractionResult,
|
||||
BatchExtractionResponse,
|
||||
search_web,
|
||||
update_wiki_page,
|
||||
)
|
||||
|
||||
|
||||
@@ -104,8 +106,10 @@ class TestSearchWeb:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_error_handling(self, mock_client):
|
||||
"""Test web search error handling."""
|
||||
mock_client.search_web.side_effect = Exception("Connection failed")
|
||||
"""Test web search errors return a user-safe message without internals."""
|
||||
mock_client.search_web.side_effect = Exception(
|
||||
"Connection failed to http://internal-host:8089"
|
||||
)
|
||||
|
||||
with patch(
|
||||
"src.agents.librarian.tools.LibraryDeskClient"
|
||||
@@ -115,8 +119,10 @@ class TestSearchWeb:
|
||||
|
||||
result = await search_web("test query")
|
||||
|
||||
assert "Error" in result
|
||||
assert "Connection failed" in result
|
||||
assert "unable to search the web" in result
|
||||
# Exception detail (internal URLs etc.) must not leak
|
||||
assert "Connection failed" not in result
|
||||
assert "internal-host" not in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_web_with_news_type(self, mock_client):
|
||||
@@ -227,7 +233,7 @@ class TestReadUrl:
|
||||
mock_client_class.return_value.__aenter__.return_value = mock_client
|
||||
mock_client_class.return_value.__aexit__.return_value = None
|
||||
|
||||
result = await read_url("https://example.com/long", max_length=2000)
|
||||
await read_url("https://example.com/long", max_length=2000)
|
||||
|
||||
mock_client.extract_content.assert_called_with(
|
||||
url="https://example.com/long",
|
||||
@@ -424,3 +430,59 @@ class TestWebSearchModels:
|
||||
assert response.total_urls == 2
|
||||
assert response.successful == 1
|
||||
assert response.failed == 1
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Wiki Update Tests (tag sentinel behavior)
|
||||
# ============================================================================
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestUpdateWikiPageTagSentinels:
|
||||
"""Empty list leaves tags unchanged; the clear sentinel empties them."""
|
||||
|
||||
def _page(self, tags: list[str]) -> WikiPage:
|
||||
return WikiPage(id=42, path="test/page", title="Test Page", tags=tags)
|
||||
|
||||
def _patched_client(self, mock_client):
|
||||
patcher = patch("src.agents.librarian.tools.LibraryDeskClient")
|
||||
mock_client_class = patcher.start()
|
||||
mock_client_class.return_value.__aenter__.return_value = mock_client
|
||||
mock_client_class.return_value.__aexit__.return_value = None
|
||||
return patcher
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_tags_means_leave_unchanged(self, mock_client):
|
||||
mock_client.update_wiki_page.return_value = self._page(["existing"])
|
||||
patcher = self._patched_client(mock_client)
|
||||
try:
|
||||
await update_wiki_page(42, content="new content")
|
||||
finally:
|
||||
patcher.stop()
|
||||
|
||||
_, kwargs = mock_client.update_wiki_page.call_args
|
||||
assert kwargs["tags"] is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_clear_sentinel_sends_empty_tag_list(self, mock_client):
|
||||
mock_client.update_wiki_page.return_value = self._page([])
|
||||
patcher = self._patched_client(mock_client)
|
||||
try:
|
||||
result = await update_wiki_page(42, tags=[CLEAR_TAGS_SENTINEL])
|
||||
finally:
|
||||
patcher.stop()
|
||||
|
||||
_, kwargs = mock_client.update_wiki_page.call_args
|
||||
assert kwargs["tags"] == []
|
||||
assert "tags (cleared)" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_real_tags_are_passed_through(self, mock_client):
|
||||
mock_client.update_wiki_page.return_value = self._page(["a", "b"])
|
||||
patcher = self._patched_client(mock_client)
|
||||
try:
|
||||
await update_wiki_page(42, tags=["a", "b"])
|
||||
finally:
|
||||
patcher.stop()
|
||||
|
||||
_, kwargs = mock_client.update_wiki_page.call_args
|
||||
assert kwargs["tags"] == ["a", "b"]
|
||||
|
||||
@@ -1,339 +0,0 @@
|
||||
"""
|
||||
Tests for multi-agent coordination engine.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
from src.agents.coordination import (
|
||||
CoordinationEngine,
|
||||
get_coordination_engine,
|
||||
delegate_to_librarian,
|
||||
)
|
||||
from src.agents.protocol import (
|
||||
AgentResponse,
|
||||
AgentUnavailableError,
|
||||
DelegationIntent,
|
||||
DelegationReason,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def coordination_engine():
|
||||
"""Create a fresh coordination engine for testing."""
|
||||
return CoordinationEngine()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_registry():
|
||||
"""Mock the household registry."""
|
||||
with patch("src.agents.coordination.get_household_registry") as mock:
|
||||
registry = MagicMock()
|
||||
mock.return_value = registry
|
||||
yield registry
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def librarian_intent():
|
||||
"""Create a standard librarian delegation intent."""
|
||||
return DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Find information about Docker networking",
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Documentation and examples",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestCoordinationEngine:
|
||||
"""Tests for CoordinationEngine class."""
|
||||
|
||||
def test_initialization(self, coordination_engine):
|
||||
"""Test engine initializes correctly."""
|
||||
assert coordination_engine is not None
|
||||
assert coordination_engine.registry is not None
|
||||
|
||||
def test_get_available_agents_empty(self, mock_registry):
|
||||
"""Test getting available agents when none have agents."""
|
||||
mock_registry.list_members.return_value = ["tatlock_core"]
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = None # No agent
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
engine = CoordinationEngine()
|
||||
available = engine.get_available_agents()
|
||||
|
||||
assert available == []
|
||||
|
||||
def test_get_available_agents_with_librarian(self, mock_registry):
|
||||
"""Test getting available agents with librarian registered."""
|
||||
mock_registry.list_members.return_value = ["tatlock_core", "librarian"]
|
||||
|
||||
# tatlock_core has no agent
|
||||
core_member = MagicMock()
|
||||
core_member.agent = None
|
||||
|
||||
# librarian has an agent
|
||||
librarian_member = MagicMock()
|
||||
librarian_member.agent = MagicMock()
|
||||
|
||||
def get_member_side_effect(name):
|
||||
if name == "tatlock_core":
|
||||
return core_member
|
||||
elif name == "librarian":
|
||||
return librarian_member
|
||||
return None
|
||||
|
||||
mock_registry.get_member.side_effect = get_member_side_effect
|
||||
|
||||
engine = CoordinationEngine()
|
||||
available = engine.get_available_agents()
|
||||
|
||||
assert "librarian" in available
|
||||
assert "tatlock_core" not in available
|
||||
|
||||
def test_can_delegate_to_unknown_agent(self, mock_registry):
|
||||
"""Test checking delegation to unknown agent."""
|
||||
mock_registry.get_member.return_value = None
|
||||
|
||||
engine = CoordinationEngine()
|
||||
|
||||
assert engine.can_delegate_to("unknown_agent") is False
|
||||
|
||||
def test_can_delegate_to_librarian(self, mock_registry):
|
||||
"""Test checking delegation to librarian."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock() # Has an agent
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
engine = CoordinationEngine()
|
||||
|
||||
assert engine.can_delegate_to("librarian") is True
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegationExecution:
|
||||
"""Tests for delegation execution."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_execute_delegation_unavailable_agent(
|
||||
self, mock_registry, librarian_intent
|
||||
):
|
||||
"""Test delegation fails for unavailable agent."""
|
||||
mock_registry.get_member.return_value = None
|
||||
|
||||
engine = CoordinationEngine()
|
||||
|
||||
with pytest.raises(AgentUnavailableError) as exc_info:
|
||||
await engine.execute_delegation(librarian_intent)
|
||||
|
||||
assert "librarian" in str(exc_info.value)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_execute_delegation_success(
|
||||
self, mock_registry, librarian_intent
|
||||
):
|
||||
"""Test successful delegation execution."""
|
||||
# Setup mock member with agent
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
# Mock the executor
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_EXECUTORS",
|
||||
{"librarian": AsyncMock(return_value="Research results here")},
|
||||
):
|
||||
engine = CoordinationEngine()
|
||||
response = await engine.execute_delegation(librarian_intent)
|
||||
|
||||
assert response.success is True
|
||||
assert response.result == "Research results here"
|
||||
# Duration might be 0 for very fast mock execution
|
||||
assert response.duration_ms >= 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_execute_delegation_error(
|
||||
self, mock_registry, librarian_intent
|
||||
):
|
||||
"""Test delegation handles executor errors."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
# Mock executor that raises
|
||||
async def failing_executor(**kwargs):
|
||||
raise ValueError("API connection failed")
|
||||
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_EXECUTORS",
|
||||
{"librarian": failing_executor},
|
||||
):
|
||||
engine = CoordinationEngine()
|
||||
response = await engine.execute_delegation(librarian_intent)
|
||||
|
||||
assert response.success is False
|
||||
assert "API connection failed" in response.error_message
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestCoordinate:
|
||||
"""Tests for multi-agent coordination."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_coordinate_single_intent(self, mock_registry, librarian_intent):
|
||||
"""Test coordinating a single delegation."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_EXECUTORS",
|
||||
{"librarian": AsyncMock(return_value="Found docs")},
|
||||
):
|
||||
engine = CoordinationEngine()
|
||||
result = await engine.coordinate([librarian_intent])
|
||||
|
||||
assert result.final_response == "Found docs"
|
||||
assert "librarian" in result.agents_consulted
|
||||
# Duration might be 0 for very fast mock execution
|
||||
assert result.total_duration_ms >= 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_coordinate_empty_intents(self, mock_registry):
|
||||
"""Test coordinating with no intents."""
|
||||
engine = CoordinationEngine()
|
||||
result = await engine.coordinate([])
|
||||
|
||||
assert result.final_response == ""
|
||||
assert result.agents_consulted == []
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_coordinate_multiple_intents(self, mock_registry):
|
||||
"""Test coordinating multiple delegations."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
intents = [
|
||||
DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Task 1",
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Result 1",
|
||||
priority=1,
|
||||
),
|
||||
DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Task 2",
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Result 2",
|
||||
priority=2,
|
||||
),
|
||||
]
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def mock_executor(**kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
return f"Result {call_count}"
|
||||
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_EXECUTORS",
|
||||
{"librarian": mock_executor},
|
||||
):
|
||||
engine = CoordinationEngine()
|
||||
result = await engine.coordinate(intents)
|
||||
|
||||
# Both intents were executed (check agents_consulted count)
|
||||
assert len(result.agents_consulted) == 2
|
||||
# Current implementation replaces same-agent responses in dict
|
||||
# So final_response has the last result (or combined if different agents)
|
||||
assert len(result.final_response) > 0
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegateToLibrarian:
|
||||
"""Tests for convenience delegation function."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delegate_to_librarian(self, mock_registry):
|
||||
"""Test the delegate_to_librarian helper."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_EXECUTORS",
|
||||
{"librarian": AsyncMock(return_value="Wiki search results")},
|
||||
):
|
||||
# Reset global engine
|
||||
with patch(
|
||||
"src.agents.coordination._coordination_engine",
|
||||
None,
|
||||
):
|
||||
response = await delegate_to_librarian(
|
||||
task="Search for Docker docs",
|
||||
context="Setting up homelab",
|
||||
)
|
||||
|
||||
assert response.success is True
|
||||
assert response.result == "Wiki search results"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestGetCoordinationEngine:
|
||||
"""Tests for engine singleton."""
|
||||
|
||||
def test_get_coordination_engine_singleton(self):
|
||||
"""Test engine is singleton."""
|
||||
with patch("src.agents.coordination._coordination_engine", None):
|
||||
engine1 = get_coordination_engine()
|
||||
engine2 = get_coordination_engine()
|
||||
|
||||
# Should be same instance
|
||||
assert engine1 is engine2
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegationStreaming:
|
||||
"""Tests for streaming delegation."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_execute_delegation_stream_unavailable(
|
||||
self, mock_registry, librarian_intent
|
||||
):
|
||||
"""Test streaming fails for unavailable agent."""
|
||||
engine = CoordinationEngine()
|
||||
|
||||
# Change target to an agent that doesn't have a stream executor
|
||||
librarian_intent.target_agent = "nonexistent_agent"
|
||||
|
||||
with pytest.raises(AgentUnavailableError):
|
||||
async for _ in engine.execute_delegation_stream(librarian_intent):
|
||||
pass
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_execute_delegation_stream_success(
|
||||
self, mock_registry, librarian_intent
|
||||
):
|
||||
"""Test successful streaming delegation."""
|
||||
mock_member = MagicMock()
|
||||
mock_member.agent = MagicMock()
|
||||
mock_registry.get_member.return_value = mock_member
|
||||
|
||||
async def mock_stream(**kwargs):
|
||||
yield "Hello "
|
||||
yield "world"
|
||||
|
||||
with patch(
|
||||
"src.agents.coordination.AGENT_STREAM_EXECUTORS",
|
||||
{"librarian": mock_stream},
|
||||
):
|
||||
engine = CoordinationEngine()
|
||||
chunks = []
|
||||
async for chunk in engine.execute_delegation_stream(librarian_intent):
|
||||
chunks.append(chunk)
|
||||
|
||||
assert chunks == ["Hello ", "world"]
|
||||
@@ -4,21 +4,79 @@ Tests for delegation infrastructure.
|
||||
Tests the DelegationTask dataclass and delegation wrapper functions
|
||||
that implement the agent-as-tool pattern.
|
||||
"""
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, patch, MagicMock
|
||||
|
||||
from src.agents.delegation import (
|
||||
ActionType,
|
||||
DelegationTask,
|
||||
DelegationResult,
|
||||
HOUSEHOLD_THINK_MESSAGES,
|
||||
STREAMING_DELEGATION_WRAPPERS,
|
||||
ActionType,
|
||||
DelegationResult,
|
||||
DelegationTask,
|
||||
_detect_action_type,
|
||||
build_delegation_context,
|
||||
delegate_to_librarian,
|
||||
get_think_message,
|
||||
_detect_action_type,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestBuildDelegationContext:
|
||||
"""Tests for trimming conversation history into expert context."""
|
||||
|
||||
def test_empty_history_returns_empty(self):
|
||||
assert build_delegation_context(None) == ""
|
||||
assert build_delegation_context([]) == ""
|
||||
|
||||
def test_recent_turns_are_formatted(self):
|
||||
history = [
|
||||
{"role": "user", "content": "Tell me about Docker"},
|
||||
{"role": "assistant", "content": "Docker is a container runtime."},
|
||||
]
|
||||
|
||||
context = build_delegation_context(history)
|
||||
|
||||
assert "Recent conversation:" in context
|
||||
assert "user: Tell me about Docker" in context
|
||||
assert "assistant: Docker is a container runtime." in context
|
||||
|
||||
def test_only_last_max_turns_kept(self):
|
||||
history = [
|
||||
{"role": "user", "content": f"message {i}"} for i in range(10)
|
||||
]
|
||||
|
||||
context = build_delegation_context(history, max_turns=6)
|
||||
|
||||
assert "message 3" not in context
|
||||
assert "message 4" in context
|
||||
assert "message 9" in context
|
||||
|
||||
def test_long_turns_are_truncated(self):
|
||||
history = [{"role": "user", "content": "x" * 2000}]
|
||||
|
||||
context = build_delegation_context(history, max_chars_per_turn=500)
|
||||
|
||||
assert "x" * 500 in context
|
||||
assert "x" * 501 not in context
|
||||
|
||||
def test_structured_content_parts_tolerated(self):
|
||||
history = [
|
||||
{"role": "user", "content": [{"type": "text", "text": "hello there"}]}
|
||||
]
|
||||
|
||||
context = build_delegation_context(history)
|
||||
|
||||
assert "hello there" in context
|
||||
|
||||
def test_non_dict_entries_skipped(self):
|
||||
history = ["garbage", {"role": "user", "content": "real message"}]
|
||||
|
||||
context = build_delegation_context(history)
|
||||
|
||||
assert "real message" in context
|
||||
assert "garbage" not in context
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegationTask:
|
||||
"""Tests for the DelegationTask dataclass."""
|
||||
@@ -170,11 +228,11 @@ class TestDelegateToLibrarian:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delegate_to_librarian_handles_error(self):
|
||||
"""Test delegation handles Librarian errors gracefully."""
|
||||
"""Test delegation maps Librarian errors to a user-safe result."""
|
||||
with patch(
|
||||
"src.agents.librarian.agent.run_librarian",
|
||||
new_callable=AsyncMock,
|
||||
side_effect=Exception("Connection refused"),
|
||||
side_effect=Exception("Connection refused to http://internal:8089"),
|
||||
):
|
||||
result = await delegate_to_librarian(
|
||||
task="Search for information",
|
||||
@@ -182,8 +240,39 @@ class TestDelegateToLibrarian:
|
||||
|
||||
assert isinstance(result, DelegationResult)
|
||||
assert result.success is False
|
||||
assert result.output == ""
|
||||
assert result.error == "Connection refused"
|
||||
# Output carries a curated butler-toned sentence
|
||||
assert result.output == get_think_message(
|
||||
"librarian", "Search for information", "error"
|
||||
)
|
||||
# Exception detail stays in logs only - never in the result
|
||||
assert "Connection refused" not in result.output
|
||||
assert result.error is not None
|
||||
assert "Connection refused" not in result.error
|
||||
assert "internal" not in result.error
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delegate_to_librarian_timeout(self, monkeypatch):
|
||||
"""Delegation is capped by LIBRARIAN_TIMEOUT and fails honestly."""
|
||||
import asyncio
|
||||
|
||||
from src.core.config import config
|
||||
|
||||
async def slow_run(task, context=""):
|
||||
await asyncio.sleep(5)
|
||||
return "too late"
|
||||
|
||||
monkeypatch.setattr(config, "LIBRARIAN_TIMEOUT", 0.05)
|
||||
|
||||
with patch(
|
||||
"src.agents.librarian.agent.run_librarian",
|
||||
new=slow_run,
|
||||
):
|
||||
result = await delegate_to_librarian(task="Search for information")
|
||||
|
||||
assert result.success is False
|
||||
assert "longer than expected" in result.output
|
||||
assert result.error is not None
|
||||
assert "time budget" in result.error
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delegate_to_librarian_preserves_task(self):
|
||||
@@ -340,20 +429,3 @@ class TestGetThinkMessage:
|
||||
msg = get_think_message("unknown_expert", "some task", "start")
|
||||
assert "<think>" not in msg
|
||||
assert "unknown_expert" in msg.lower()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestStreamingDelegationWrappers:
|
||||
"""Tests for streaming delegation wrapper mapping."""
|
||||
|
||||
def test_streaming_wrappers_exist(self):
|
||||
"""Test streaming wrappers mapping has all experts."""
|
||||
assert "librarian" in STREAMING_DELEGATION_WRAPPERS
|
||||
assert "biographer" in STREAMING_DELEGATION_WRAPPERS
|
||||
assert "housekeeper" in STREAMING_DELEGATION_WRAPPERS
|
||||
|
||||
def test_streaming_wrappers_are_async_generators(self):
|
||||
"""Test streaming wrappers are async generator functions."""
|
||||
import inspect
|
||||
for name, wrapper in STREAMING_DELEGATION_WRAPPERS.items():
|
||||
assert inspect.isasyncgenfunction(wrapper), f"{name} is not an async generator"
|
||||
|
||||
+15
-240
@@ -1,256 +1,31 @@
|
||||
"""
|
||||
Tests for agent communication protocol.
|
||||
Tests for the agent error protocol.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from src.agents.protocol import (
|
||||
AgentError,
|
||||
AgentRequest,
|
||||
AgentResponse,
|
||||
AgentTimeoutError,
|
||||
AgentUnavailableError,
|
||||
CoordinationResult,
|
||||
DelegationIntent,
|
||||
DelegationReason,
|
||||
ToolCallRecord,
|
||||
)
|
||||
from src.agents.protocol import AgentError
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestAgentRequest:
|
||||
"""Tests for AgentRequest model."""
|
||||
class TestAgentError:
|
||||
"""Tests for the AgentError exception."""
|
||||
|
||||
def test_basic_request(self):
|
||||
"""Test creating a basic agent request."""
|
||||
request = AgentRequest(task="Find information about Docker")
|
||||
|
||||
assert request.task == "Find information about Docker"
|
||||
assert request.context == ""
|
||||
assert request.timeout_seconds == 60
|
||||
|
||||
def test_request_with_context(self):
|
||||
"""Test request with additional context."""
|
||||
request = AgentRequest(
|
||||
task="Find Docker networking docs",
|
||||
context="User is setting up a homelab",
|
||||
delegation_reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
)
|
||||
|
||||
assert request.task == "Find Docker networking docs"
|
||||
assert request.context == "User is setting up a homelab"
|
||||
assert request.delegation_reason == DelegationReason.DOMAIN_EXPERTISE
|
||||
|
||||
def test_request_serialization(self):
|
||||
"""Test request can be serialized to dict."""
|
||||
request = AgentRequest(
|
||||
task="Research task",
|
||||
context="Some context",
|
||||
)
|
||||
|
||||
data = request.model_dump()
|
||||
|
||||
assert data["task"] == "Research task"
|
||||
assert data["context"] == "Some context"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestAgentResponse:
|
||||
"""Tests for AgentResponse model."""
|
||||
|
||||
def test_successful_response(self):
|
||||
"""Test creating a successful response."""
|
||||
response = AgentResponse(
|
||||
success=True,
|
||||
result="Here are the findings...",
|
||||
reasoning="Searched wiki and found relevant docs",
|
||||
duration_ms=1500,
|
||||
)
|
||||
|
||||
assert response.success is True
|
||||
assert response.result == "Here are the findings..."
|
||||
assert response.reasoning == "Searched wiki and found relevant docs"
|
||||
assert response.duration_ms == 1500
|
||||
assert response.error_message is None
|
||||
|
||||
def test_failed_response(self):
|
||||
"""Test creating a failed response."""
|
||||
response = AgentResponse(
|
||||
success=False,
|
||||
result="",
|
||||
error_message="Connection timeout",
|
||||
duration_ms=30000,
|
||||
)
|
||||
|
||||
assert response.success is False
|
||||
assert response.result == ""
|
||||
assert response.error_message == "Connection timeout"
|
||||
|
||||
def test_response_with_tool_calls(self):
|
||||
"""Test response tracking tool calls."""
|
||||
tool_call = ToolCallRecord(
|
||||
tool_name="hybrid_search",
|
||||
arguments={"query": "Docker networking"},
|
||||
result="Found 5 results",
|
||||
duration_ms=500,
|
||||
)
|
||||
|
||||
response = AgentResponse(
|
||||
success=True,
|
||||
result="Based on search...",
|
||||
tool_calls=[tool_call],
|
||||
)
|
||||
|
||||
assert len(response.tool_calls) == 1
|
||||
assert response.tool_calls[0].tool_name == "hybrid_search"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegationIntent:
|
||||
"""Tests for DelegationIntent model."""
|
||||
|
||||
def test_basic_intent(self):
|
||||
"""Test creating a basic delegation intent."""
|
||||
intent = DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Research Docker networking",
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Documentation and examples",
|
||||
)
|
||||
|
||||
assert intent.target_agent == "librarian"
|
||||
assert intent.task == "Research Docker networking"
|
||||
assert intent.reason == DelegationReason.DOMAIN_EXPERTISE
|
||||
assert intent.priority == 1 # Default
|
||||
|
||||
def test_intent_with_priority(self):
|
||||
"""Test intent with custom priority."""
|
||||
intent = DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Urgent research",
|
||||
reason=DelegationReason.RESOURCE_EFFICIENCY,
|
||||
expected_outcome="Quick answer",
|
||||
priority=1,
|
||||
)
|
||||
|
||||
assert intent.priority == 1
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestDelegationReason:
|
||||
"""Tests for DelegationReason enum."""
|
||||
|
||||
def test_all_reasons_have_values(self):
|
||||
"""Test all delegation reasons are defined."""
|
||||
reasons = list(DelegationReason)
|
||||
|
||||
assert DelegationReason.DOMAIN_EXPERTISE in reasons
|
||||
assert DelegationReason.TOOL_ACCESS in reasons
|
||||
assert DelegationReason.RESOURCE_EFFICIENCY in reasons
|
||||
assert DelegationReason.USER_PREFERENCE in reasons
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestCoordinationResult:
|
||||
"""Tests for CoordinationResult model."""
|
||||
|
||||
def test_single_agent_result(self):
|
||||
"""Test coordination with single agent."""
|
||||
agent_response = AgentResponse(
|
||||
success=True,
|
||||
result="Research findings",
|
||||
duration_ms=1000,
|
||||
)
|
||||
|
||||
intent = DelegationIntent(
|
||||
target_agent="librarian",
|
||||
task="Research task",
|
||||
reason=DelegationReason.DOMAIN_EXPERTISE,
|
||||
expected_outcome="Findings",
|
||||
)
|
||||
|
||||
result = CoordinationResult(
|
||||
final_response="Research findings",
|
||||
agent_responses={"librarian": agent_response},
|
||||
delegation_intents=[intent],
|
||||
total_duration_ms=1200,
|
||||
agents_consulted=["librarian"],
|
||||
)
|
||||
|
||||
assert result.final_response == "Research findings"
|
||||
assert len(result.agent_responses) == 1
|
||||
assert result.agents_consulted == ["librarian"]
|
||||
|
||||
def test_empty_result(self):
|
||||
"""Test coordination with no delegations."""
|
||||
result = CoordinationResult(
|
||||
final_response="",
|
||||
agent_responses={},
|
||||
delegation_intents=[],
|
||||
total_duration_ms=0,
|
||||
agents_consulted=[],
|
||||
)
|
||||
|
||||
assert result.final_response == ""
|
||||
assert len(result.agents_consulted) == 0
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestAgentErrors:
|
||||
"""Tests for agent error types."""
|
||||
|
||||
def test_agent_error(self):
|
||||
"""Test base AgentError."""
|
||||
def test_agent_error_defaults(self):
|
||||
"""Test base AgentError with default agent name."""
|
||||
error = AgentError("Something went wrong")
|
||||
|
||||
assert "Something went wrong" in str(error)
|
||||
assert error.agent_name == "unknown"
|
||||
|
||||
def test_agent_timeout_error(self):
|
||||
"""Test AgentTimeoutError."""
|
||||
error = AgentTimeoutError(
|
||||
"Timed out after 60s",
|
||||
agent_name="librarian",
|
||||
)
|
||||
def test_agent_error_carries_agent_name(self):
|
||||
"""Agent name is stored and prefixed into the message."""
|
||||
error = AgentError("Research task failed", agent_name="librarian")
|
||||
|
||||
assert "Timed out" in str(error)
|
||||
assert error.agent_name == "librarian"
|
||||
assert str(error) == "[librarian] Research task failed"
|
||||
assert error.message == "Research task failed"
|
||||
|
||||
def test_agent_unavailable_error(self):
|
||||
"""Test AgentUnavailableError."""
|
||||
error = AgentUnavailableError(
|
||||
"Agent not registered",
|
||||
agent_name="unknown_agent",
|
||||
)
|
||||
|
||||
assert "not registered" in str(error)
|
||||
assert error.agent_name == "unknown_agent"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestToolCallRecord:
|
||||
"""Tests for ToolCallRecord model."""
|
||||
|
||||
def test_tool_call_record(self):
|
||||
"""Test creating a tool call record."""
|
||||
record = ToolCallRecord(
|
||||
tool_name="semantic_search",
|
||||
arguments={"query": "networking concepts", "limit": 10},
|
||||
result="Found 10 relevant documents",
|
||||
duration_ms=250,
|
||||
)
|
||||
|
||||
assert record.tool_name == "semantic_search"
|
||||
assert record.arguments["query"] == "networking concepts"
|
||||
assert record.duration_ms == 250
|
||||
|
||||
def test_tool_call_with_empty_result(self):
|
||||
"""Test tool call with empty result."""
|
||||
record = ToolCallRecord(
|
||||
tool_name="query_graph",
|
||||
arguments={"cypher": "MATCH (n) RETURN n"},
|
||||
result="",
|
||||
duration_ms=100,
|
||||
)
|
||||
|
||||
assert record.result == ""
|
||||
def test_agent_error_is_catchable_as_exception(self):
|
||||
"""AgentError participates in normal exception handling."""
|
||||
with pytest.raises(AgentError):
|
||||
raise AgentError("boom", agent_name="librarian")
|
||||
|
||||
@@ -410,10 +410,12 @@ async def test_tatlock_ollama_fallback(async_client: AsyncClient):
|
||||
"stream": False
|
||||
}
|
||||
|
||||
# 300s: this test forbids the Claude rescue, and the full local
|
||||
# Steward -> orchestrate -> synthesize flow on gemma4 exceeds 120s
|
||||
response = await async_client.post(
|
||||
"/v1/chat/completions",
|
||||
json=request_data,
|
||||
timeout=120.0
|
||||
timeout=300.0
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
"""
|
||||
Unit tests for backend selection (Ollama primary, Claude fallback).
|
||||
|
||||
These tests set the cached health-check globals directly so they are
|
||||
deterministic regardless of which services are reachable.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
from src.anthropic import model_selector
|
||||
from src.core.config import config
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def local_first(monkeypatch):
|
||||
"""Baseline: local-first config, both backends healthy."""
|
||||
monkeypatch.setattr(config, "PREFER_CLOUD_BACKEND", False)
|
||||
monkeypatch.setattr(config, "ANTHROPIC_API_KEY", "sk-test-fake")
|
||||
monkeypatch.setattr(model_selector, "_claude_available", True)
|
||||
monkeypatch.setattr(model_selector, "_ollama_available", True)
|
||||
|
||||
|
||||
class TestResolveBackend:
|
||||
def test_default_is_ollama(self, local_first):
|
||||
assert model_selector.resolve_backend() == "ollama"
|
||||
|
||||
def test_prefer_cloud_config_selects_claude(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(config, "PREFER_CLOUD_BACKEND", True)
|
||||
assert model_selector.resolve_backend() == "claude"
|
||||
|
||||
def test_prefer_cloud_override_selects_claude(self, local_first):
|
||||
assert model_selector.resolve_backend(prefer_cloud=True) == "claude"
|
||||
|
||||
def test_prefer_cloud_without_claude_falls_back_to_ollama(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(config, "PREFER_CLOUD_BACKEND", True)
|
||||
monkeypatch.setattr(model_selector, "_claude_available", False)
|
||||
assert model_selector.resolve_backend() == "ollama"
|
||||
|
||||
def test_ollama_down_falls_back_to_claude(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(model_selector, "_ollama_available", False)
|
||||
assert model_selector.resolve_backend() == "claude"
|
||||
|
||||
def test_ollama_down_without_claude_stays_ollama(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(model_selector, "_ollama_available", False)
|
||||
monkeypatch.setattr(model_selector, "_claude_available", False)
|
||||
assert model_selector.resolve_backend() == "ollama"
|
||||
|
||||
def test_unknown_ollama_state_counts_as_available(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(model_selector, "_ollama_available", None)
|
||||
assert model_selector.resolve_backend() == "ollama"
|
||||
|
||||
|
||||
class TestGetModel:
|
||||
def test_ollama_backend_returns_openai_chat_model(self, local_first):
|
||||
from pydantic_ai.models.openai import OpenAIChatModel
|
||||
|
||||
model = model_selector.get_model()
|
||||
assert isinstance(model, OpenAIChatModel)
|
||||
assert model.model_name == config.OLLAMA_DEFAULT_MODEL
|
||||
|
||||
def test_claude_backend_returns_anthropic_model(self, local_first):
|
||||
from pydantic_ai.models.anthropic import AnthropicModel
|
||||
|
||||
model = model_selector.get_model(prefer_cloud=True)
|
||||
assert isinstance(model, AnthropicModel)
|
||||
assert model.model_name == config.ANTHROPIC_MODEL
|
||||
|
||||
|
||||
class TestToolChoiceSettings:
|
||||
def test_ollama_forces_tool_choice(self, local_first):
|
||||
settings = model_selector.get_tool_choice_settings()
|
||||
assert settings.get("extra_body") == {"tool_choice": "required"}
|
||||
|
||||
def test_claude_uses_native_tool_choice(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(config, "PREFER_CLOUD_BACKEND", True)
|
||||
settings = model_selector.get_tool_choice_settings()
|
||||
assert not settings.get("extra_body")
|
||||
|
||||
|
||||
class TestGetModelInfo:
|
||||
def test_reports_ollama_primary(self, local_first):
|
||||
info = model_selector.get_model_info()
|
||||
assert info["backend"] == "ollama"
|
||||
assert info["model"] == config.OLLAMA_DEFAULT_MODEL
|
||||
assert info["ollama_available"] is True
|
||||
assert info["claude_available"] is True
|
||||
assert info["prefer_cloud"] is False
|
||||
|
||||
def test_reports_claude_when_ollama_down(self, local_first, monkeypatch):
|
||||
monkeypatch.setattr(model_selector, "_ollama_available", False)
|
||||
info = model_selector.get_model_info()
|
||||
assert info["backend"] == "claude"
|
||||
assert info["model"] == config.ANTHROPIC_MODEL
|
||||
+41
-1
@@ -12,11 +12,51 @@ from src.main import app
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def _initialize_app():
|
||||
def _tenant_guard():
|
||||
"""
|
||||
Hard-fail the whole suite if the effective tenant resolves to the
|
||||
production tenant ("jpmschweitzer").
|
||||
|
||||
Isolation is tenant-based: tests that touch shared services
|
||||
(Qdrant memories collections, Wiki.js via library-desk, Neo4j,
|
||||
Redis) must run under the reserved test tenant "llm_tester" (or a
|
||||
test_-prefixed namespace). This mirrors the guard library-desk
|
||||
applies on its side.
|
||||
"""
|
||||
from src.core.config import PRODUCTION_TENANT, config
|
||||
from src.core.context import get_default_user
|
||||
from src.core.multi_tenancy import get_memory_collection_name
|
||||
|
||||
effective = get_default_user()
|
||||
if (
|
||||
effective == PRODUCTION_TENANT
|
||||
or config.effective_default_user == PRODUCTION_TENANT
|
||||
):
|
||||
pytest.exit(
|
||||
f"TENANT GUARD: refusing to run the test suite - the effective "
|
||||
f"tenant resolves to the production tenant '{PRODUCTION_TENANT}' "
|
||||
f"(ENVIRONMENT={config.ENVIRONMENT.value}, "
|
||||
f"DEFAULT_USER={config.DEFAULT_USER}). Tests must run under "
|
||||
f"'llm_tester' or a test_-prefixed tenant.",
|
||||
returncode=1,
|
||||
)
|
||||
|
||||
# The Qdrant memories namespace derived from the effective tenant
|
||||
# must never be the production collection.
|
||||
assert get_memory_collection_name(effective) != get_memory_collection_name(
|
||||
PRODUCTION_TENANT
|
||||
), "test suite would target the production memories collection"
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def _initialize_app(_tenant_guard):
|
||||
"""
|
||||
Run application lifespan (Claude health check, household registration, etc.)
|
||||
once per test session. ASGITransport doesn't trigger lifespan events,
|
||||
so we call it explicitly.
|
||||
|
||||
Depends on _tenant_guard so the suite refuses to start under the
|
||||
production tenant before any initialization happens.
|
||||
"""
|
||||
from src.core.startup import initialize_application
|
||||
asyncio.run(initialize_application())
|
||||
|
||||
@@ -0,0 +1,219 @@
|
||||
"""
|
||||
Wire-level contract tests for external service boundaries.
|
||||
|
||||
Each test sends the raw request the application code sends (no client
|
||||
wrappers, no mocks) and asserts on the response shape, so boundary
|
||||
breakage is caught directly instead of surfacing as agent misbehavior.
|
||||
|
||||
Semantics:
|
||||
- Service unreachable -> skip (an outage is not a contract violation)
|
||||
- Service reachable but wrong response shape -> fail
|
||||
|
||||
Run with: make test-contracts
|
||||
"""
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from src.core.config import config
|
||||
|
||||
OLLAMA = str(config.OLLAMA_HOST).rstrip("/")
|
||||
QDRANT = f"http://{config.QDRANT_HOST}:{config.QDRANT_PORT}"
|
||||
SEARXNG = str(config.SEARXNG_HOST).rstrip("/")
|
||||
|
||||
CALCULATOR_TOOL = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "calculator",
|
||||
"description": "Evaluate a math expression",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {"expression": {"type": "string"}},
|
||||
"required": ["expression"],
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
async def _get_or_skip(url: str, service: str, timeout: float = 5.0) -> httpx.Response:
|
||||
"""GET a URL, skipping the test if the service is unreachable."""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||
return await client.get(url)
|
||||
except httpx.TransportError as e:
|
||||
pytest.skip(f"{service} unreachable at {url}: {e}")
|
||||
|
||||
|
||||
async def _post_or_skip(
|
||||
url: str, service: str, payload: dict, timeout: float, headers: dict | None = None
|
||||
) -> httpx.Response:
|
||||
"""POST a payload, skipping the test if the service is unreachable."""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||
return await client.post(url, json=payload, headers=headers)
|
||||
except httpx.TransportError as e:
|
||||
pytest.skip(f"{service} unreachable at {url}: {e}")
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestOllamaContract:
|
||||
"""Boundary: Ollama native API and its OpenAI-compat layer."""
|
||||
|
||||
async def test_tags_lists_configured_model(self):
|
||||
# Mirrors check_ollama_health()
|
||||
response = await _get_or_skip(f"{OLLAMA}/api/tags", "ollama")
|
||||
assert response.status_code == 200
|
||||
names = [m["name"] for m in response.json()["models"]]
|
||||
model = config.OLLAMA_DEFAULT_MODEL
|
||||
assert model in names or f"{model}:latest" in names, (
|
||||
f"{model} not pulled; available: {names}"
|
||||
)
|
||||
|
||||
async def test_generate_returns_plain_text(self):
|
||||
# Mirrors StewardAgent._call_ollama()
|
||||
response = await _post_or_skip(
|
||||
f"{OLLAMA}/api/generate",
|
||||
"ollama",
|
||||
{
|
||||
"model": config.OLLAMA_DEFAULT_MODEL,
|
||||
"prompt": "Reply with the single word: pong",
|
||||
"stream": False,
|
||||
"options": {"temperature": 0.3, "top_p": 0.9},
|
||||
},
|
||||
timeout=config.OLLAMA_TIMEOUT,
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["response"].strip()
|
||||
|
||||
async def test_openai_compat_tool_calling(self):
|
||||
# Mirrors the request PydanticAI's OpenAIChatModel sends for the
|
||||
# orchestration phase, including the extra_body tool_choice.
|
||||
response = await _post_or_skip(
|
||||
f"{OLLAMA}/v1/chat/completions",
|
||||
"ollama",
|
||||
{
|
||||
"model": config.OLLAMA_DEFAULT_MODEL,
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is 6 * 7? Use the calculator."}
|
||||
],
|
||||
"tools": [CALCULATOR_TOOL],
|
||||
"tool_choice": "required",
|
||||
"stream": False,
|
||||
},
|
||||
timeout=config.OLLAMA_TIMEOUT,
|
||||
)
|
||||
assert response.status_code == 200
|
||||
message = response.json()["choices"][0]["message"]
|
||||
tool_calls = message.get("tool_calls")
|
||||
assert tool_calls, f"model answered in text instead of calling the tool: {message}"
|
||||
assert tool_calls[0]["function"]["name"] == "calculator"
|
||||
arguments = json.loads(tool_calls[0]["function"]["arguments"])
|
||||
assert "expression" in arguments
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestAnthropicContract:
|
||||
"""Boundary: Anthropic Messages API (the Claude fallback backend)."""
|
||||
|
||||
HEADERS_KEY = "anthropic-version"
|
||||
|
||||
def _headers(self) -> dict:
|
||||
if not config.ANTHROPIC_API_KEY:
|
||||
pytest.skip("ANTHROPIC_API_KEY not configured")
|
||||
return {
|
||||
"x-api-key": config.ANTHROPIC_API_KEY,
|
||||
"anthropic-version": "2023-06-01",
|
||||
}
|
||||
|
||||
async def test_minimal_message_accepted(self):
|
||||
# Mirrors check_claude_health(): tiny request, no sampling params
|
||||
response = await _post_or_skip(
|
||||
"https://api.anthropic.com/v1/messages",
|
||||
"anthropic",
|
||||
{
|
||||
"model": config.ANTHROPIC_MODEL,
|
||||
"max_tokens": 1,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
timeout=30.0,
|
||||
headers=self._headers(),
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
|
||||
async def test_temperature_rejected(self):
|
||||
# Pins the Claude Sonnet 5+ contract that broke the Steward:
|
||||
# sampling parameters are rejected with a 400 (and not billed).
|
||||
response = await _post_or_skip(
|
||||
"https://api.anthropic.com/v1/messages",
|
||||
"anthropic",
|
||||
{
|
||||
"model": config.ANTHROPIC_MODEL,
|
||||
"max_tokens": 1,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"temperature": 0.3,
|
||||
},
|
||||
timeout=30.0,
|
||||
headers=self._headers(),
|
||||
)
|
||||
assert response.status_code == 400
|
||||
assert "temperature" in response.text
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestQdrantContract:
|
||||
"""Boundary: Qdrant REST API (Biographer's vector memory)."""
|
||||
|
||||
async def test_collections_endpoint(self):
|
||||
response = await _get_or_skip(f"{QDRANT}/collections", "qdrant")
|
||||
assert response.status_code == 200
|
||||
assert "collections" in response.json()["result"]
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestSearxngContract:
|
||||
"""Boundary: SearXNG JSON search API (web search tool)."""
|
||||
|
||||
async def test_json_search(self):
|
||||
response = await _get_or_skip(
|
||||
f"{SEARXNG}/search?q=test&format=json", "searxng", timeout=config.SEARXNG_TIMEOUT
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert "results" in response.json()
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestLibraryDeskContract:
|
||||
"""Boundary: library-desk research API (the Librarian's backend)."""
|
||||
|
||||
async def test_health(self):
|
||||
host = getattr(config, "LIBRARY_DESK_HOST", None)
|
||||
if not host:
|
||||
pytest.skip("LIBRARY_DESK_HOST not configured")
|
||||
response = await _get_or_skip(f"{str(host).rstrip('/')}/health", "library-desk")
|
||||
assert response.status_code == 200
|
||||
|
||||
|
||||
@pytest.mark.contract
|
||||
class TestRedisContract:
|
||||
"""Boundary: Redis on the configured memory DB."""
|
||||
|
||||
async def test_roundtrip(self):
|
||||
import redis.asyncio as redis
|
||||
|
||||
client = redis.Redis(
|
||||
host=config.REDIS_HOST,
|
||||
port=config.REDIS_PORT,
|
||||
db=config.REDIS_MEMORY_DB,
|
||||
socket_connect_timeout=3,
|
||||
)
|
||||
try:
|
||||
await client.ping()
|
||||
except Exception as e:
|
||||
pytest.skip(f"redis unreachable: {e}")
|
||||
try:
|
||||
await client.set("contract-test-key", "ok", ex=30)
|
||||
assert await client.get("contract-test-key") == b"ok"
|
||||
await client.delete("contract-test-key")
|
||||
finally:
|
||||
await client.aclose()
|
||||
@@ -0,0 +1,300 @@
|
||||
"""
|
||||
Tests for the tenant isolation guard.
|
||||
|
||||
Isolation is tenant-based: the production tenant ("jpmschweitzer") owns
|
||||
real data in the shared services, and every non-production environment
|
||||
must run under the reserved test tenant ("llm_tester") or an explicit
|
||||
"test_"-prefixed namespace.
|
||||
|
||||
Guard matrix covered here: dev/test/prod x default/explicit user, at
|
||||
both config level (effective_default_user) and request-context
|
||||
resolution (get_user).
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from src.core.config import (
|
||||
PRODUCTION_TENANT,
|
||||
TEST_TENANT,
|
||||
Config,
|
||||
Environment,
|
||||
)
|
||||
from src.core.context import RequestContext, get_user
|
||||
|
||||
|
||||
def make_config(**overrides) -> Config:
|
||||
"""Build a Config isolated from the local .env file."""
|
||||
return Config(_env_file=None, **overrides)
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestEffectiveDefaultUserMatrix:
|
||||
"""Config-level guard: effective_default_user per environment."""
|
||||
|
||||
# --- development ---
|
||||
|
||||
def test_dev_without_default_user_forces_test_tenant(self):
|
||||
config = make_config(ENVIRONMENT=Environment.DEVELOPMENT)
|
||||
assert config.effective_default_user == TEST_TENANT
|
||||
assert config.tenant_forced is False
|
||||
|
||||
def test_dev_with_test_tenant_is_kept(self):
|
||||
config = make_config(ENVIRONMENT=Environment.DEVELOPMENT, DEFAULT_USER=TEST_TENANT)
|
||||
assert config.effective_default_user == TEST_TENANT
|
||||
assert config.tenant_forced is False
|
||||
|
||||
def test_dev_with_test_prefixed_override_is_kept(self):
|
||||
config = make_config(ENVIRONMENT=Environment.DEVELOPMENT, DEFAULT_USER="test_phase_b")
|
||||
assert config.effective_default_user == "test_phase_b"
|
||||
assert config.tenant_forced is False
|
||||
|
||||
def test_dev_with_misconfigured_user_is_forced_to_test_tenant(self):
|
||||
config = make_config(ENVIRONMENT=Environment.DEVELOPMENT, DEFAULT_USER="alice")
|
||||
assert config.effective_default_user == TEST_TENANT
|
||||
assert config.tenant_forced is True
|
||||
|
||||
def test_dev_with_production_tenant_refuses_startup(self):
|
||||
with pytest.raises(ValidationError) as exc_info:
|
||||
make_config(
|
||||
ENVIRONMENT=Environment.DEVELOPMENT,
|
||||
DEFAULT_USER=PRODUCTION_TENANT,
|
||||
)
|
||||
assert "Refusing to start" in str(exc_info.value)
|
||||
assert PRODUCTION_TENANT in str(exc_info.value)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"variant",
|
||||
[
|
||||
"JPMSchweitzer",
|
||||
"JPMSCHWEITZER",
|
||||
"jpmschweitzer.",
|
||||
" jpmschweitzer",
|
||||
"jpmschweitzer ",
|
||||
"_jpmschweitzer_",
|
||||
"jpmschweitzer!",
|
||||
],
|
||||
)
|
||||
def test_dev_with_production_tenant_variant_refuses_startup(self, variant):
|
||||
"""Sanitization collisions with the production tenant are refused too."""
|
||||
with pytest.raises(ValidationError, match="Refusing to start"):
|
||||
make_config(
|
||||
ENVIRONMENT=Environment.DEVELOPMENT,
|
||||
DEFAULT_USER=variant,
|
||||
)
|
||||
|
||||
# --- testing ---
|
||||
|
||||
def test_testing_without_default_user_forces_test_tenant(self):
|
||||
config = make_config(ENVIRONMENT=Environment.TESTING)
|
||||
assert config.effective_default_user == TEST_TENANT
|
||||
|
||||
def test_testing_with_misconfigured_user_is_forced_to_test_tenant(self):
|
||||
config = make_config(ENVIRONMENT=Environment.TESTING, DEFAULT_USER="bob")
|
||||
assert config.effective_default_user == TEST_TENANT
|
||||
assert config.tenant_forced is True
|
||||
|
||||
def test_testing_with_production_tenant_refuses_startup(self):
|
||||
with pytest.raises(ValidationError, match="Refusing to start"):
|
||||
make_config(
|
||||
ENVIRONMENT=Environment.TESTING,
|
||||
DEFAULT_USER=PRODUCTION_TENANT,
|
||||
)
|
||||
|
||||
def test_testing_with_test_prefixed_override_is_kept(self):
|
||||
config = make_config(ENVIRONMENT=Environment.TESTING, DEFAULT_USER="test_ci_run")
|
||||
assert config.effective_default_user == "test_ci_run"
|
||||
|
||||
# --- production ---
|
||||
|
||||
def test_prod_without_default_user_uses_production_tenant(self):
|
||||
config = make_config(ENVIRONMENT=Environment.PRODUCTION)
|
||||
assert config.effective_default_user == PRODUCTION_TENANT
|
||||
assert config.tenant_forced is False
|
||||
|
||||
def test_prod_with_explicit_production_tenant_is_kept(self):
|
||||
config = make_config(ENVIRONMENT=Environment.PRODUCTION, DEFAULT_USER=PRODUCTION_TENANT)
|
||||
assert config.effective_default_user == PRODUCTION_TENANT
|
||||
|
||||
def test_prod_with_explicit_other_user_is_kept(self):
|
||||
config = make_config(ENVIRONMENT=Environment.PRODUCTION, DEFAULT_USER="household_guest")
|
||||
assert config.effective_default_user == "household_guest"
|
||||
assert config.tenant_forced is False
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestRequestContextGuard:
|
||||
"""Request-context resolution guard: get_user() per environment."""
|
||||
|
||||
def _patch_environment(self, monkeypatch, environment: Environment):
|
||||
from src.core import config as config_module
|
||||
|
||||
monkeypatch.setattr(config_module.config, "ENVIRONMENT", environment)
|
||||
|
||||
def test_dev_default_resolution_is_test_tenant(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.DEVELOPMENT)
|
||||
assert get_user() == TEST_TENANT
|
||||
|
||||
def test_dev_explicit_production_tenant_is_forced(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.DEVELOPMENT)
|
||||
with RequestContext(user=PRODUCTION_TENANT):
|
||||
assert get_user() == TEST_TENANT
|
||||
|
||||
def test_testing_explicit_production_tenant_is_forced(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.TESTING)
|
||||
with RequestContext(user=PRODUCTION_TENANT):
|
||||
assert get_user() == TEST_TENANT
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"variant",
|
||||
[
|
||||
"JPMSchweitzer",
|
||||
"JPMSCHWEITZER",
|
||||
"jpmschweitzer.",
|
||||
" jpmschweitzer",
|
||||
"jpmschweitzer ",
|
||||
"_jpmschweitzer_",
|
||||
"jpmschweitzer!",
|
||||
],
|
||||
)
|
||||
@pytest.mark.parametrize(
|
||||
"environment", [Environment.DEVELOPMENT, Environment.TESTING]
|
||||
)
|
||||
def test_production_tenant_sanitization_variants_are_forced(
|
||||
self, monkeypatch, environment, variant
|
||||
):
|
||||
"""
|
||||
Any raw user that sanitizes to the production tenant would resolve
|
||||
to the production namespaces (memories_jpmschweitzer,
|
||||
session:jpmschweitzer:*) - the guard must force it to the test
|
||||
tenant in non-production environments.
|
||||
"""
|
||||
from src.core.multi_tenancy import get_memory_collection_name
|
||||
|
||||
self._patch_environment(monkeypatch, environment)
|
||||
with RequestContext(user=variant):
|
||||
effective = get_user()
|
||||
assert effective == TEST_TENANT
|
||||
assert (
|
||||
get_memory_collection_name(effective)
|
||||
!= get_memory_collection_name(PRODUCTION_TENANT)
|
||||
)
|
||||
|
||||
def test_dev_non_colliding_user_is_not_forced(self, monkeypatch):
|
||||
"""A user that sanitizes to a different namespace passes through."""
|
||||
self._patch_environment(monkeypatch, Environment.DEVELOPMENT)
|
||||
with RequestContext(user="jpm.schweitzer"):
|
||||
# sanitizes to jpm_schweitzer != jpmschweitzer
|
||||
assert get_user() == "jpm.schweitzer"
|
||||
|
||||
def test_dev_explicit_other_user_passes_through(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.DEVELOPMENT)
|
||||
with RequestContext(user="testuser"):
|
||||
assert get_user() == "testuser"
|
||||
|
||||
def test_prod_explicit_production_tenant_passes_through(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.PRODUCTION)
|
||||
with RequestContext(user=PRODUCTION_TENANT):
|
||||
assert get_user() == PRODUCTION_TENANT
|
||||
|
||||
def test_prod_explicit_other_user_passes_through(self, monkeypatch):
|
||||
self._patch_environment(monkeypatch, Environment.PRODUCTION)
|
||||
with RequestContext(user="alice"):
|
||||
assert get_user() == "alice"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestSuiteRunsUnderTestTenant:
|
||||
"""
|
||||
The live test session itself must resolve to the test tenant.
|
||||
|
||||
The session guard in tests/conftest.py hard-fails the suite when the
|
||||
effective tenant is the production tenant; these tests assert the
|
||||
namespaces every shared-service touch would use (Qdrant memories
|
||||
collection, Redis session keys) are the llm_tester ones.
|
||||
"""
|
||||
|
||||
def test_effective_tenant_is_not_production(self):
|
||||
from src.core.context import get_default_user
|
||||
|
||||
assert get_default_user() != PRODUCTION_TENANT
|
||||
|
||||
def test_effective_tenant_is_the_reserved_test_tenant(self):
|
||||
from src.core.context import get_default_user
|
||||
|
||||
assert get_default_user() == TEST_TENANT
|
||||
|
||||
def test_memories_collection_namespace_is_test_tenant(self):
|
||||
from src.core.context import get_default_user
|
||||
from src.core.multi_tenancy import get_memory_collection_name
|
||||
|
||||
assert (
|
||||
get_memory_collection_name(get_default_user())
|
||||
== f"memories_{TEST_TENANT}"
|
||||
)
|
||||
|
||||
def test_redis_session_namespace_is_test_tenant(self):
|
||||
from src.core.context import get_default_user
|
||||
from src.core.multi_tenancy import get_session_key
|
||||
|
||||
key = get_session_key(get_default_user(), "conv_test")
|
||||
assert key.startswith(f"session:{TEST_TENANT}:")
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestStartupTenantGuardLog:
|
||||
"""One loud startup log line states the effective tenant."""
|
||||
|
||||
def test_non_production_logs_forced_tenant(self, monkeypatch):
|
||||
from src.core import startup as startup_module
|
||||
|
||||
events = []
|
||||
|
||||
class _Recorder:
|
||||
def warning(self, event, **kw):
|
||||
events.append((event, kw))
|
||||
|
||||
def info(self, event, **kw):
|
||||
events.append((event, kw))
|
||||
|
||||
monkeypatch.setattr(startup_module, "logger", _Recorder())
|
||||
monkeypatch.setattr(startup_module.config, "ENVIRONMENT", Environment.DEVELOPMENT)
|
||||
|
||||
startup_module.log_tenant_guard()
|
||||
|
||||
assert events == [
|
||||
(
|
||||
"tenant_guard_active",
|
||||
{
|
||||
"environment": "development",
|
||||
"forced_tenant": TEST_TENANT,
|
||||
"default_user_overridden": startup_module.config.tenant_forced,
|
||||
"configured_default_user": startup_module.config.DEFAULT_USER,
|
||||
},
|
||||
)
|
||||
]
|
||||
|
||||
def test_production_logs_production_tenant(self, monkeypatch):
|
||||
from src.core import startup as startup_module
|
||||
|
||||
events = []
|
||||
|
||||
class _Recorder:
|
||||
def warning(self, event, **kw):
|
||||
events.append(("warning", event, kw))
|
||||
|
||||
def info(self, event, **kw):
|
||||
events.append(("info", event, kw))
|
||||
|
||||
monkeypatch.setattr(startup_module, "logger", _Recorder())
|
||||
monkeypatch.setattr(startup_module.config, "ENVIRONMENT", Environment.PRODUCTION)
|
||||
|
||||
startup_module.log_tenant_guard()
|
||||
|
||||
assert events == [
|
||||
(
|
||||
"info",
|
||||
"tenant_guard_production",
|
||||
{"environment": "production", "tenant": PRODUCTION_TENANT},
|
||||
)
|
||||
]
|
||||
@@ -26,12 +26,18 @@ from typing import AsyncGenerator
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
from src.core.config import PRODUCTION_TENANT, TEST_TENANT
|
||||
|
||||
# Test configuration
|
||||
BASE_URL = "http://localhost:8777"
|
||||
QDRANT_URL = "http://localhost:6333"
|
||||
API_TIMEOUT = 120.0 # LLM calls can be slow
|
||||
TEST_USER = "llm_tester"
|
||||
# All e2e writes to shared services go to the reserved test tenant's
|
||||
# namespaces (Qdrant memories_llm_tester, wiki llm_tester scope) - never
|
||||
# the production tenant's.
|
||||
TEST_USER = TEST_TENANT
|
||||
TEST_COLLECTION = f"memories_{TEST_USER}"
|
||||
assert TEST_USER != PRODUCTION_TENANT
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -986,7 +992,7 @@ class TestUserContextIsolation:
|
||||
in_test_collection = any(unique_value in v for v in test_values)
|
||||
|
||||
# Check production collection (should NOT be there)
|
||||
prod_collection = "memories_jpmschweitzer"
|
||||
prod_collection = f"memories_{PRODUCTION_TENANT}"
|
||||
if await qdrant.collection_exists(prod_collection):
|
||||
prod_points = await qdrant.scroll_points(prod_collection)
|
||||
prod_values = [str(p.get("payload", {})) for p in prod_points]
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
"""
|
||||
Tests for TatlockOllamaProvider configuration.
|
||||
|
||||
The AsyncOpenAI client must carry an explicit timeout from
|
||||
config.OLLAMA_TIMEOUT instead of the SDK default (~600s), so a stuck
|
||||
LLM call cannot consume the whole delegation budget.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from src.core.config import config
|
||||
from src.ollama.provider import TatlockOllamaProvider, _sanitize_messages
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestProviderTimeout:
|
||||
"""Timeout configuration on the underlying AsyncOpenAI client."""
|
||||
|
||||
def test_openai_client_timeout_from_config(self):
|
||||
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
|
||||
|
||||
assert provider._openai_client.timeout == float(config.OLLAMA_TIMEOUT)
|
||||
|
||||
def test_timeout_is_not_sdk_default(self):
|
||||
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
|
||||
|
||||
# The OpenAI SDK defaults to 600s; the configured cap must win
|
||||
assert provider._openai_client.timeout < 600
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestMessageSanitization:
|
||||
"""Null content sanitization for Ollama compatibility."""
|
||||
|
||||
def test_null_content_with_tool_calls_becomes_empty_string(self):
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [{"id": "call_1", "type": "function"}],
|
||||
}
|
||||
]
|
||||
|
||||
sanitized = _sanitize_messages(messages)
|
||||
|
||||
assert sanitized[0]["content"] == ""
|
||||
|
||||
def test_regular_messages_unchanged(self):
|
||||
messages = [
|
||||
{"role": "user", "content": "Hello"},
|
||||
{"role": "assistant", "content": "Good day, sir."},
|
||||
]
|
||||
|
||||
assert _sanitize_messages(messages) == messages
|
||||
@@ -0,0 +1,185 @@
|
||||
"""
|
||||
Tests for real-time think message streaming and context plumbing in
|
||||
the direct delegation paths.
|
||||
|
||||
_stream_direct_delegation must be an async generator that yields the
|
||||
"start" think message BEFORE the expert runs (so 'Allow me to consult
|
||||
the archives, sir.' streams while research is in flight), and both
|
||||
direct delegation paths must pass trimmed conversation history as
|
||||
expert context.
|
||||
"""
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.agents.delegation import DelegationResult
|
||||
from src.responses.streaming import (
|
||||
ReasoningSummaryDelta,
|
||||
ReasoningSummaryDone,
|
||||
StreamingCoordinator,
|
||||
)
|
||||
|
||||
HISTORY = [
|
||||
{"role": "user", "content": "Tell me about my homelab wiki"},
|
||||
{"role": "assistant", "content": "It documents your services, sir."},
|
||||
]
|
||||
|
||||
|
||||
def _librarian_result(output: str = "Findings.") -> DelegationResult:
|
||||
return DelegationResult(
|
||||
expert_name="librarian",
|
||||
task="task",
|
||||
success=True,
|
||||
output=output,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestStreamDirectDelegation:
|
||||
"""Real-time streaming behavior of _stream_direct_delegation."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_start_think_streams_before_research_runs(self):
|
||||
coordinator = StreamingCoordinator()
|
||||
tracker = AsyncMock()
|
||||
results: dict = {}
|
||||
|
||||
with patch(
|
||||
"src.agents.delegation.delegate_to_librarian",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_librarian_result(),
|
||||
) as mock_delegate:
|
||||
gen = coordinator._stream_direct_delegation(
|
||||
user_message="Search for Docker info",
|
||||
recommendation=SimpleNamespace(
|
||||
recommended_capabilities=["librarian"]
|
||||
),
|
||||
tracker=tracker,
|
||||
conversation_id="conv_1",
|
||||
conversation_history=HISTORY,
|
||||
results=results,
|
||||
)
|
||||
|
||||
# First event: the start think message, BEFORE any research
|
||||
first = await gen.__anext__()
|
||||
assert isinstance(first, ReasoningSummaryDelta)
|
||||
assert first.delta.strip() != ""
|
||||
assert mock_delegate.await_count == 0, (
|
||||
"start think message must stream before the expert runs"
|
||||
)
|
||||
|
||||
second = await gen.__anext__()
|
||||
assert isinstance(second, ReasoningSummaryDone)
|
||||
assert mock_delegate.await_count == 0
|
||||
|
||||
# Third event: completion message - research has now run
|
||||
third = await gen.__anext__()
|
||||
assert isinstance(third, ReasoningSummaryDelta)
|
||||
assert mock_delegate.await_count == 1
|
||||
|
||||
remaining = [event async for event in gen]
|
||||
assert any(isinstance(e, ReasoningSummaryDone) for e in remaining)
|
||||
|
||||
# Results dict is populated for Phase 2 synthesis
|
||||
assert results["expert_results"] == {"librarian": "Findings."}
|
||||
assert results["tools_called"] == ["delegate_to_librarian"]
|
||||
assert len(results["think_messages"]) == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_conversation_history_passed_as_context(self):
|
||||
coordinator = StreamingCoordinator()
|
||||
tracker = AsyncMock()
|
||||
|
||||
with patch(
|
||||
"src.agents.delegation.delegate_to_librarian",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_librarian_result(),
|
||||
) as mock_delegate:
|
||||
events = [
|
||||
event
|
||||
async for event in coordinator._stream_direct_delegation(
|
||||
user_message="And what services does it list?",
|
||||
recommendation=SimpleNamespace(
|
||||
recommended_capabilities=["librarian"]
|
||||
),
|
||||
tracker=tracker,
|
||||
conversation_id="conv_1",
|
||||
conversation_history=HISTORY,
|
||||
results={},
|
||||
)
|
||||
]
|
||||
|
||||
assert events, "generator must yield think events"
|
||||
context = mock_delegate.await_args.kwargs["context"]
|
||||
assert "Tell me about my homelab wiki" in context
|
||||
assert "It documents your services, sir." in context
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_failed_delegation_streams_error_think(self):
|
||||
coordinator = StreamingCoordinator()
|
||||
tracker = AsyncMock()
|
||||
results: dict = {}
|
||||
|
||||
failed = DelegationResult(
|
||||
expert_name="librarian",
|
||||
task="task",
|
||||
success=False,
|
||||
output="I'm afraid the archives proved difficult to access.",
|
||||
error="The Librarian was unable to complete the task.",
|
||||
)
|
||||
|
||||
with patch(
|
||||
"src.agents.delegation.delegate_to_librarian",
|
||||
new_callable=AsyncMock,
|
||||
return_value=failed,
|
||||
):
|
||||
events = [
|
||||
event
|
||||
async for event in coordinator._stream_direct_delegation(
|
||||
user_message="Search for Docker info",
|
||||
recommendation=SimpleNamespace(
|
||||
recommended_capabilities=["librarian"]
|
||||
),
|
||||
tracker=tracker,
|
||||
conversation_id="conv_1",
|
||||
results=results,
|
||||
)
|
||||
]
|
||||
|
||||
assert results["tools_called"] == []
|
||||
# Expert result carries the curated user-safe sentence
|
||||
assert "archives" in results["expert_results"]["librarian"]
|
||||
deltas = [e.delta for e in events if isinstance(e, ReasoningSummaryDelta)]
|
||||
assert len(deltas) == 2 # start + error think messages
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestServiceDelegationContext:
|
||||
"""The non-streaming direct delegation path passes trimmed history."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_direct_delegation_with_results_passes_context(self):
|
||||
from src.responses.service import _direct_delegation_with_results
|
||||
|
||||
tracker = AsyncMock()
|
||||
|
||||
with patch(
|
||||
"src.agents.delegation.delegate_to_librarian",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_librarian_result(),
|
||||
) as mock_delegate:
|
||||
results = await _direct_delegation_with_results(
|
||||
user_message="And what services does it list?",
|
||||
recommendation=SimpleNamespace(
|
||||
recommended_capabilities=["librarian"]
|
||||
),
|
||||
tracker=tracker,
|
||||
conversation_id="conv_1",
|
||||
conversation_history=HISTORY,
|
||||
)
|
||||
|
||||
context = mock_delegate.await_args.kwargs["context"]
|
||||
assert "Tell me about my homelab wiki" in context
|
||||
assert results["expert_results"]["librarian"] == "Findings."
|
||||
Reference in New Issue
Block a user