jpmschweitzer/tatlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

refactor(backend): speak only OpenAI-compat /v1 to the local backend

Browse Source 
Three native-API touchpoints converted — steward /api/generate to
/v1/chat/completions, embeddings /api/embeddings to /v1/embeddings,
health /api/tags to /v1/models — so the backend behind OLLAMA_HOST is
swappable by env alone. This makes the serving plan's "tatlock needs
zero changes" claim true for the llama-server cutover and for forge
after it. EMBEDDING_HOST (default: OLLAMA_HOST) lets gen and embed
point at different servers, which the boilerroom stack needs.

The dead OllamaClient goes with it: a native-API client nothing
imported, whose presence would make the post-cutover "no native
endpoints" grep lie.

Contract tests rewritten to mirror the new requests and extended with
the embeddings shape (dim must match the configured Qdrant dimension).
All pass against live Ollama's /v1 — deployable before any cutover.
Both new assertions mutation-checked via env overrides (bogus model,
wrong dim: each fails). The Anthropic contract now skips on 401: the
configured key is deliberately revoked per workspace D-11, which is
"fallback disabled", not a boundary break. Embedding continuity across
backends was measured separately: same nomic bytes, cosine 1.0000.

663 unit tests pass; ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2026-09-11 21:35:48 +02:00
co-authored by Claude Fable 5
parent f578ebded8
commit 16b7edbc26
8 changed files with 110 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/core/config.py
+6 -2
View File
@@ -139,9 +139,13 @@ class Config(BaseSettings):
default=768, description="Embedding dimension (768 for nomic-embed-text)"
)
# Ollama Embedding Configuration
# Embedding Configuration
OLLAMA_EMBEDDING_MODEL: str = Field(
default="nomic-embed-text", description="Ollama model for embeddings"
default="nomic-embed-text", description="Model for embeddings"
)
EMBEDDING_HOST: HttpUrl | None = Field(
default=None,
description="OpenAI-compatible embeddings host; falls back to OLLAMA_HOST so gen and embed can live on different servers",
)
# Redis Memory Database
src/core/embeddings.py
+26 -18
View File
@@ -1,7 +1,8 @@
"""
Ollama client for embeddings generation.
Embeddings client for the local OpenAI-compatible backend.
Provides async embedding operations via Ollama API:
Provides async embedding operations via /v1/embeddings (served by Ollama
today and llama-server after the serving cutover):
- Text embedding generation
- Batch embedding support
- Health checks
@@ -21,10 +22,10 @@ logger = get_logger(__name__)
class OllamaEmbeddingClient:
"""
Ollama API client for embeddings.
Embeddings client for the local backend (name kept for config lineage).
Uses the Ollama embeddings endpoint to generate vector representations
of text using the nomic-embed-text model (768 dimensions).
Uses the OpenAI-compatible /v1/embeddings endpoint to generate vector
representations of text using the nomic-embed-text model (768 dimensions).
Usage:
client = OllamaEmbeddingClient()
@@ -46,14 +47,16 @@ class OllamaEmbeddingClient:
Initialize Ollama embedding client.
Args:
base_url: Ollama server URL (defaults to config.OLLAMA_HOST)
base_url: embeddings server URL (defaults to config.EMBEDDING_HOST,
falling back to config.OLLAMA_HOST)
model: Embedding model name (defaults to config.OLLAMA_EMBEDDING_MODEL)
timeout: Request timeout in seconds (embeddings can be slow)
"""
self.base_url = (base_url or str(config.OLLAMA_HOST)).rstrip("/")
default_host = config.EMBEDDING_HOST or config.OLLAMA_HOST
self.base_url = (base_url or str(default_host)).rstrip("/")
self.model = model or config.OLLAMA_EMBEDDING_MODEL
self.embeddings_url = f"{self.base_url}/api/embeddings"
self.tags_url = f"{self.base_url}/api/tags"
self.embeddings_url = f"{self.base_url}/v1/embeddings"
self.models_url = f"{self.base_url}/v1/models"
self._client: httpx.AsyncClient | None = None
self._timeout = timeout
@@ -109,14 +112,15 @@ class OllamaEmbeddingClient:
payload = {
"model": self.model,
"prompt": text,
"input": text,
}
response = await client.post(self.embeddings_url, json=payload)
response.raise_for_status()
data = response.json()
embedding = data.get("embedding")
rows = data.get("data") or []
embedding = rows[0].get("embedding") if rows else None
if not embedding:
logger.error("ollama_embed_no_embedding", response_data=data)
return None
@@ -142,8 +146,9 @@ class OllamaEmbeddingClient:
"""
Generate embeddings for multiple texts.
Note: Ollama doesn't support native batch embeddings, so this
sequentially calls embed() for each text.
Note: sequential by choice — /v1/embeddings accepts an array input
on both backends, so native batching is available when the call
volume ever justifies changing these semantics.
Args:
texts: List of texts to embed
@@ -223,22 +228,25 @@ class OllamaEmbeddingClient:
async def health_check(self) -> bool:
"""
Check if Ollama server is reachable and model is available.
Check if the embeddings server is reachable and serves the model.
Reads the OpenAI-compatible /v1/models surface, so it holds for any
backend behind EMBEDDING_HOST (Ollama, llama-server).
Returns:
True if healthy, False otherwise
"""
try:
client = await self._get_client()
response = await client.get(self.tags_url, timeout=5.0)
response = await client.get(self.models_url, timeout=5.0)
response.raise_for_status()
data = response.json()
models = data.get("models", [])
models = data.get("data", [])
# Check if our embedding model is available
model_found = False
for m in models:
name = m.get("name", "")
name = m.get("id", "")
if name == self.model or name.startswith(f"{self.model}:"):
model_found = True
break
@@ -247,7 +255,7 @@ class OllamaEmbeddingClient:
logger.warning(
"ollama_embedding_model_not_found",
model=self.model,
available=[m.get("name") for m in models],
available=[m.get("id") for m in models],
)
return False