jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

docs: replace CPU-era latency figures with measured GPU numbers

Browse Source 
The ~35s steward / ~2 min flow figures dated from the driver-mismatch era
and were being inherited by downstream consumers (desklock architecture
doc) as planning baselines. Current measured: steward ~6s warm, full flow
11-25s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2026-07-14 17:17:13 +02:00
co-authored by Claude Fable 5
parent f0a08ede64
commit 99683357d2
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+1 -1
View File
@@ -27,7 +27,7 @@ Dependencies are in `pyproject.toml` (`[project.dependencies]` and `[project.opt
**Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return a 400. Use `get_sampling_settings()` from the model selector instead of passing `ModelSettings(temperature=...)` directly to agents that can run on the Claude fallback. The contract test suite pins this (`make test-contracts`).
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). The full local Steward → orchestrate → synthesize flow takes ~2 minutes on gemma4. Steward analysis alone needs ~35s warm — `STEWARD_TIMEOUT` defaults to 60s.
**Integration test timeouts.** Set to 120s to match `OLLAMA_TIMEOUT` config (300s for the pure-Ollama fallback test, which cannot be rescued by Claude). Current GPU-resident numbers (2026-07-14, driver 570, gemma4:e2b at ~100 tok/s): Steward analysis ~6s warm, full Steward → orchestrate → synthesize flow 11–25s, librarian-routed queries ~20-25s. The old "~35s steward / ~2 min flow" figures were measured during the CPU-only era (driver mismatch, 13 tok/s) — do not plan against them. Cold start after 2h idle adds ~8s (`OLLAMA_KEEP_ALIVE=2h`). `STEWARD_TIMEOUT` defaults to 60s.
**`get_benchmark_store` does not exist.** The benchmarking module (`src/core/benchmarks.py`) was never implemented. `scripts/benchmark_analysis.py` also references it and is broken. Do not add mocks for it in tests.