feat(backend): BACKEND_SLOT_PINNING — per-phase engine slot ownership
Each pipeline phase owns one llama-server slot (steward 0, orchestrator 1, synthesizer 2), carried as id_slot in extra_body through the same mechanism tool_choice already uses, so the phase's stable prompt prefix stays in that slot's KV cache and a turn re-prefills only its new tokens. Off by default; a no-op on the Claude backend and ignored by Ollama, so the flag is safe on any backend and the cutover itself stays a pure env swap. The merge helper preserves existing extra_body keys — mutation-checked (dropping the merge fails exactly the test written for it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -147,6 +147,10 @@ class Config(BaseSettings):
|
||||
default=None,
|
||||
description="OpenAI-compatible embeddings host; falls back to OLLAMA_HOST so gen and embed can live on different servers",
|
||||
)
|
||||
BACKEND_SLOT_PINNING: bool = Field(
|
||||
default=False,
|
||||
description="Pin each pipeline phase to a llama-server slot (steward=0, orchestrator=1, synthesizer=2) so a phase's stable prompt prefix stays in that slot's KV cache; no-op on Claude, ignored by Ollama",
|
||||
)
|
||||
|
||||
# Redis Memory Database
|
||||
REDIS_MEMORY_DB: int = Field(default=1, description="Redis database number for memory cache")
|
||||
|
||||
Reference in New Issue
Block a user