docs: add CLAUDE.md with development guide and testing gotchas

Documents architecture, key file locations, test setup, and critical
gotchas discovered during development (ASGITransport lifespan, async
scope mismatch, Ollama fallback behavior, missing benchmark store).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-05 20:22:06 +01:00
co-authored by Claude Opus 4.6
parent 334d313d17
commit 901a04825d
+86
View File
@@ -0,0 +1,86 @@
# CLAUDE.md - Development Guide for AI Assistants
This file documents key patterns, gotchas, and architecture notes for working on the Tatlock codebase.
## Project Overview
Tatlock is a homelab AI butler with an OpenAI-compatible API. Built with Python 3.12+, FastAPI, and PydanticAI. Dual backend: Claude (preferred) + Ollama (fallback).
- **Git remote**: `git.schweitz.net:jpmschweitzer/tatlock.git` (SSH port 2222, User `git`)
- **Services on `tower-of-joy`**: Ollama (`11434`), Redis (`6379`), Qdrant (`6333`/`6334`), SearXNG (`8080`)
## Architecture
### Two-Tier Request Flow
1. **Steward** analyzes the request via `preprocess_request()` → returns `EnrichedRequest`
- `steward_reasoning`: plain text analysis
- `scoped_tools`: tools from household registry based on recommendations
- `steward_note`: formatted note prepended to Tatlock's prompt
2. **Tatlock** executes in two phases:
- **Phase 1** (`orchestrate_tool_calls`): Scoped agent with tools runs
- **Phase 2** (`synthesize_from_results`): Synthesis agent (no tools) produces butler-toned response
3. Response wraps steward reasoning in `<think>` tags for Open WebUI reasoning bubbles
### Backend Selection (`src/anthropic/model_selector.py`)
- `check_claude_health()` runs at startup, caches `_claude_available`
- `get_model()`: returns Claude if available+preferred, else Ollama
- Config: `PREFER_CLOUD_BACKEND=true`, `ANTHROPIC_API_KEY` in `.env`
### Household Registry
- Global singleton at `src/core/household_registry.py`
- `register_household_members()` in `src/core/startup.py` populates it (sync)
- `initialize_application()` in `src/core/startup.py` runs health check + registration (async)
### Key Models
- Claude: `claude-sonnet-4-20250514` (reliable tool calling)
- Ollama: `mistral-nemo-large:latest` on tower-of-joy (unreliable tool calling)
## Key File Locations
| File | Purpose |
|------|---------|
| `src/core/config.py` | pydantic-settings Config class, loads `.env` |
| `src/anthropic/model_selector.py` | Claude/Ollama switching |
| `src/responses/service.py` | Response pipeline (steward → tatlock → output) |
| `src/agents/tatlock.py` | TatlockAgent with tool orchestration and synthesis |
| `src/agents/steward/service.py` | Steward analysis and capability extraction |
| `src/core/startup.py` | `initialize_application()` (async) and `register_household_members()` (sync) |
| `src/core/household_registry.py` | Global registry singleton |
| `src/chat/service.py` | `create_chat_completion`, wraps reasoning in `<think>` tags |
| `tests/conftest.py` | Shared test fixtures |
## Testing
### Running Tests
```bash
# Unit tests only (fast, no external services)
.venv/bin/pytest --ignore=tests/e2e --ignore=tests/integration
# Single integration test
.venv/bin/pytest tests/agents/test_tatlock_agent.py::test_tatlock_tool_call_logging_calculator -v
```
### Test Categories
- **Unit tests**: Use `lorem-tester` model (no external services needed)
- **Integration tests**: Use `Tatlock` model (needs Claude or Ollama), marked with `@pytest.mark.integration`
- **Ollama fallback test**: Patches `_claude_available = False` to force Ollama path
### Critical Gotchas
**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app` fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`. Without this, `check_claude_health()` never runs and `_claude_available` stays `None`, causing all tests to silently fall back to Ollama.
**AsyncIO scope mismatch.** `asyncio_default_fixture_loop_scope = function` is set in `pytest.ini`. Session-scoped async fixtures cause `ScopeMismatch` errors. The fix is to use a sync fixture with `asyncio.run()` for session-scoped initialization.
**Ollama is unreliable for tool calling.** `mistral-nemo` on Ollama often does mental math instead of calling calculator tools, and frequently gets wrong answers. Claude reliably calls tools. If integration tests give wrong math answers, check which backend is actually being used.
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config. Ollama on tower-of-joy can be slow, especially on first request.
**`get_benchmark_store` does not exist.** The benchmarking module (`src/core/benchmarks.py`) was never implemented. `scripts/benchmark_analysis.py` also references it and is broken. Do not add mocks for it in tests.
**Steward tests need household registry.** Use `register_household_members()` (sync) in fixtures, not `initialize_application()` (async). The steward extracts capabilities from the registry.