Compare commits

...
5 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 4.5 8092740fa4 fix: exclude null fields from streaming chunks for Open WebUI compatibility
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m24s
OpenAI's API omits null fields in streaming chunks, but Tatlock was
including them (content: null, reasoning_content: null). This caused
parsing issues in Open WebUI's streaming handler.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 15:36:37 +01:00
jpmschweitzerandClaude Opus 4.5 e469746f75 fix: use StreamingResponse for chat completions SSE
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m22s
sse_starlette's EventSourceResponse added \r\n line endings that
Open WebUI couldn't parse. Switched to plain StreamingResponse with
manual SSE formatting matching OpenAI's exact format.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 12:29:08 +01:00
jpmschweitzerandClaude Opus 4.5 31e7884d8f fix: remove Steward analysis from user-visible reasoning
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m21s
The Steward's internal routing analysis (DELEGATE, COMPLEXITY, etc.)
was being exposed in <think> blocks. This is implementation detail,
not useful reasoning for the user.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 12:19:05 +01:00
jpmschweitzerandClaude Opus 4.5 e15def607d fix: remove extra_body tool_choice hack for Claude backend
Build and Push / build (push) Successful in 1m57s
Build and Push / release (push) Successful in 3s
PydanticAI handles tool_choice natively for Anthropic. The extra_body
hack caused an infinite tool call loop where Claude kept calling the
same tool because tool_choice was forced to "any".

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 11:51:42 +01:00
jpmschweitzerandClaude Opus 4.5 6dd1c2e2a9 fix: trigger CI on version tag push instead of release event
Changed workflow trigger from release:published to push:tags:v[0-9]*
so that pushing a version tag triggers the build pipeline.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 09:52:43 +01:00
9 changed files with 75 additions and 56 deletions
+3 -2
View File
@@ -1,8 +1,9 @@
name: Build and Push
on:
release:
types: [published]
push:
tags:
- 'v[0-9]*'
jobs:
release:
+25
View File
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.0.5] - 2026-02-05
### Fixed
- **Streaming JSON compatibility** - Exclude null fields from streaming chunks using `exclude_none=True`; OpenAI's API omits null fields entirely, and including them (e.g., `content: null`, `reasoning_content: null`) caused parsing issues in Open WebUI
## [2.0.4] - 2026-02-05
### Fixed
- **Open WebUI streaming compatibility** - Replaced `sse_starlette` `EventSourceResponse` with plain `StreamingResponse` for chat completions; `sse_starlette` added `\r\n` line endings and extra SSE fields that Open WebUI couldn't parse
## [2.0.3] - 2026-02-05
### Fixed
- **Steward analysis leaking into responses** - Removed internal routing analysis (`DELEGATE: tatlock_core...`) from user-visible reasoning in both streaming and non-streaming paths
## [2.0.2] - 2026-02-05
### Fixed
- **tool_choice format incompatibility** - Removed `extra_body` tool_choice hack for Claude backend; PydanticAI handles tool_choice natively for Anthropic, preventing infinite tool call loops
- **CI trigger** - Changed workflow trigger from `release:published` to `push:tags:v[0-9]*`
## [2.0.1] - 2026-02-05
### Fixed
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "tatlock"
version = "2.0.1"
version = "2.0.5"
description = "OpenAI-compatible API with Ollama backend"
requires-python = ">=3.12"
dependencies = []
+5 -5
View File
@@ -494,13 +494,13 @@ class TatlockAgent(AgentInterface):
)
# Run with scoped tools and tracker
# Force tool_choice: required to make LLM actually call tools
from pydantic_ai.settings import ModelSettings
# Force tool_choice to make LLM actually call tools
from src.anthropic.model_selector import get_tool_choice_settings
result = await scoped_agent.run(
enriched_message,
message_history=pydantic_history if pydantic_history else None,
deps=tool_tracker,
model_settings=ModelSettings(extra_body={"tool_choice": "required"})
model_settings=get_tool_choice_settings(),
)
logger.info(
@@ -624,7 +624,6 @@ class TatlockAgent(AgentInterface):
- tool_outputs: Dict mapping tool names to their outputs
- raw_output: The agent's raw text output
"""
from pydantic_ai.settings import ModelSettings
from pydantic_ai.messages import (
ModelRequest,
ModelResponse,
@@ -684,11 +683,12 @@ class TatlockAgent(AgentInterface):
)
# Run with scoped tools and tracker
from src.anthropic.model_selector import get_tool_choice_settings
result = await scoped_agent.run(
enriched_message,
message_history=pydantic_history if pydantic_history else None,
deps=tool_tracker,
model_settings=ModelSettings(extra_body={"tool_choice": "required"})
model_settings=get_tool_choice_settings(),
)
# Extract tool calls and results from the agent's messages
+2
View File
@@ -7,11 +7,13 @@ Provides model selection with automatic fallback between Claude and Ollama.
from src.anthropic.model_selector import (
check_claude_health,
get_model,
get_tool_choice_settings,
is_claude_available,
)
__all__ = [
"check_claude_health",
"get_model",
"get_tool_choice_settings",
"is_claude_available",
]
+17
View File
@@ -132,6 +132,23 @@ def get_model(prefer_cloud: bool | None = None) -> Union[AnthropicModel, OpenAIC
)
def get_tool_choice_settings() -> 'ModelSettings':
"""
Get model_settings for forcing tool calls on the first request.
For Claude: PydanticAI handles tool_choice natively, so no extra_body needed.
For Ollama: Pass tool_choice="required" via extra_body to force tool calling.
"""
from pydantic_ai.settings import ModelSettings
if is_claude_available() and config.PREFER_CLOUD_BACKEND:
# PydanticAI's Anthropic model handles tool_choice internally
return ModelSettings()
else:
# Ollama needs explicit tool_choice via extra_body
return ModelSettings(extra_body={"tool_choice": "required"})
def get_model_info() -> dict:
"""
Get information about the current model configuration.
+22 -18
View File
@@ -7,7 +7,7 @@ import logging
from typing import AsyncGenerator
from fastapi import APIRouter
from sse_starlette.sse import EventSourceResponse
from starlette.responses import StreamingResponse
from src.chat import service
from src.chat.schemas import (
@@ -22,47 +22,51 @@ router = APIRouter(prefix="/chat", tags=["chat"])
async def _stream_response(
request: ChatCompletionRequest,
) -> AsyncGenerator[dict, None]:
) -> AsyncGenerator[str, None]:
"""
Generate SSE stream for chat completion.
EventSourceResponse adds "data: " prefix automatically.
We just yield the dict/string content.
Yields raw SSE-formatted strings matching OpenAI's format exactly:
data: {json}\n\n
"""
try:
async for chunk in service.create_chat_completion_stream(request):
# Yield dict - EventSourceResponse will format as SSE
yield {"data": chunk.model_dump_json()}
yield f"data: {chunk.model_dump_json(exclude_none=True)}\n\n"
# Send [DONE] message
yield {"data": "[DONE]"}
yield "data: [DONE]\n\n"
except Exception as e:
logger.error(f"Error in streaming response: {e}")
error_data = {"error": {"message": str(e), "type": "internal_error"}}
yield {"data": json.dumps(error_data)}
error_data = json.dumps({"error": {"message": str(e), "type": "internal_error"}})
yield f"data: {error_data}\n\n"
@router.post("/completions", response_model=ChatCompletionResponse)
async def create_chat_completion(
request: ChatCompletionRequest,
) -> ChatCompletionResponse | EventSourceResponse:
) -> ChatCompletionResponse | StreamingResponse:
"""
Create chat completion (OpenAI-compatible).
Supports both regular and streaming responses.
Currently returns mock lorem ipsum responses.
Args:
request: Chat completion request
Returns:
Chat completion response or SSE stream
"""
logger.info(f"Chat completion request for model: {request.model}")
if request.stream:
logger.info("Streaming response requested")
return EventSourceResponse(_stream_response(request))
return StreamingResponse(
_stream_response(request),
media_type="text/event-stream",
headers={
"Cache-Control": "no-store",
"X-Accel-Buffering": "no",
},
)
return await service.create_chat_completion(request)
-10
View File
@@ -651,16 +651,6 @@ async def create_response_with_steward(request: ResponseRequest) -> Response:
# Build response output items
output_items = []
# Add Steward reasoning as a reasoning output item
output_items.append(ReasoningOutputItem(
id=f"reasoning_{generate_id()}",
summary=[
"🎩 Steward's Analysis:",
enriched.steward_reasoning,
],
status="completed"
))
# Add Tatlock's message
output_items.append(MessageOutputItem(
id=f"msg_{generate_id()}",
-20
View File
@@ -166,26 +166,6 @@ class StreamingCoordinator:
conversation_id=conversation_id,
)
# Stream Steward's analysis as reasoning summary
steward_lines = enriched.steward_reasoning.split('\n')
for line in steward_lines:
if line.strip():
yield ReasoningSummaryDelta(delta=line + "\n")
await asyncio.sleep(0.05)
yield ReasoningSummaryDone()
# Add Steward reasoning to output items
reasoning_item = ReasoningOutputItem(
id=f"reasoning_{generate_id()}",
summary=[
"🎩 Steward's Analysis:",
enriched.steward_reasoning,
],
status="completed"
)
output_items.append(reasoning_item)
# Initialize tool tracker
tracker = ToolCallTracker(
recommended_capabilities=enriched.recommendation.recommended_capabilities,