A scripted 8-turn conversation through the real pipeline (tool call,
memory recall, librarian route, small talk), run against any base URL so
the same instrument measures Ollama today and llama-server after the
cutover. Non-stream mode records wall time and usage; --stream records
time-to-first-token; a VRAM poller and an optional interleaved second
tenant capture the contention the serving plan calls out.
Planted facts carry a BENCH-MARKER prefix so anything the biographer
learns from a bench run stays recognizable in the memory stores.
First smoke against production: 9.7s mean per simple turn, and TTFT
equals wall time — the client sees nothing until the whole
steward-orchestrate-synthesize pipeline has finished. Time-to-first-word
for desklock is currently the full pipeline latency, which makes
streamed synthesis a first-class goal of the serving work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jpmschweitzerandClaude Fable 5
f578ebded8