release v2.6.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m54s

The cutover's first real lesson ships: tool_choice "required" was a
workaround for a backend that ignored it, and became an unbreakable
tool loop on a backend that obeys it every request (~80 s arithmetic
turns, observed). The health check now learns which server answers
behind OLLAMA_HOST (/props is llama-server's own surface) and only
Ollama gets the advisory nudge. Probed unforced on llama-server: 3/3
tool calls via the --jinja template.

Also carries BACKEND_SLOT_PINNING (default off) for the P3 pilot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-11 22:41:31 +02:00
co-authored by Claude Fable 5
parent da5ded48ab
commit 64b6a3d826
5 changed files with 65 additions and 12 deletions
+31 -7
View File
@@ -28,6 +28,11 @@ logger = get_logger(__name__)
# Cached health check results (set once at startup)
_claude_available: bool | None = None
_ollama_available: bool | None = None
# Which server answers behind OLLAMA_HOST: "llama-server" or "ollama".
# They disagree on tool_choice semantics (see get_tool_choice_settings),
# so the health check probes /props — served by llama-server only — and
# caches the answer here.
_local_flavor: str | None = None
async def check_ollama_health() -> bool:
@@ -40,7 +45,7 @@ async def check_ollama_health() -> bool:
Returns:
True if Ollama is reachable and OLLAMA_DEFAULT_MODEL is pulled.
"""
global _ollama_available
global _ollama_available, _local_flavor
host = str(config.OLLAMA_HOST).rstrip("/")
model = config.OLLAMA_DEFAULT_MODEL
@@ -53,12 +58,20 @@ async def check_ollama_health() -> bool:
response.raise_for_status()
names = [m.get("id", "") for m in response.json().get("data", [])]
# /props is llama-server's own surface; Ollama 404s it.
try:
props = await client.get(f"{host}/props")
_local_flavor = "llama-server" if props.status_code == 200 else "ollama"
except httpx.HTTPError:
_local_flavor = "ollama"
if model in names or f"{model}:latest" in names:
_ollama_available = True
logger.info(
"ollama_health_check_passed",
host=host,
model=model,
flavor=_local_flavor,
)
return True
@@ -244,19 +257,30 @@ def get_model(prefer_cloud: bool | None = None) -> AnthropicModel | OpenAIChatMo
def get_tool_choice_settings() -> ModelSettings:
"""
Get model_settings for forcing tool calls on the first request.
Get model_settings for tool calling on the orchestration phase.
For Claude: PydanticAI handles tool_choice natively, so no extra_body needed.
For Ollama: Pass tool_choice="required" via extra_body to force tool calling.
Claude: PydanticAI handles tool_choice natively no extra_body.
Ollama: tool_choice="required" via extra_body. Advisory there (Ollama
ignores it), but it nudges gemma4 to actually call tools, which the
persona-suppression gotcha made necessary.
llama-server: NO tool_choice. It enforces "required" on every request
in the run, so after a tool returns, the next generation is again
forced to call a tool — an unbreakable tool loop (~80 s turns,
observed at cutover). Its --jinja template renders tool definitions
the way gemma4 was trained, and the model calls tools reliably
unforced (probed 3/3).
"""
from pydantic_ai.settings import ModelSettings
if resolve_backend() == "claude":
# PydanticAI's Anthropic model handles tool_choice internally
return ModelSettings()
else:
# Ollama needs explicit tool_choice via extra_body
return ModelSettings(extra_body={"tool_choice": "required"})
if _local_flavor == "llama-server":
return ModelSettings()
# Ollama needs explicit tool_choice via extra_body
return ModelSettings(extra_body={"tool_choice": "required"})
def with_slot_pinning(settings: ModelSettings | None, slot: int) -> ModelSettings | None: