Compare commits

...
13 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Fable 5 8808880145 fix(tests): T-7 — the unit suite is hermetic, the flake class is gone
Three flakes in one day, never a captured name — the name finally
came from a discriminating run instead of patience: pointing
OLLAMA_HOST at a dead port failed exactly eight tests, all in
tests/agents/test_tatlock_agent.py, all already marked integration,
all running in the unit gate anyway because make test excluded by
directory and they live outside the ignored directories. Eight tests
doing ~110 seconds of real LLM inference against the production
wrapper on every unit run, failing whenever the backend was
mid-deploy — which this week it constantly was.

The second coupling sat in the session init: conftest ran the real
health probes, an HTTP round trip to whatever answers behind
OLLAMA_HOST plus a real Anthropic API call whenever the dev .env
carries a key, so the cached backend globals followed the network of
the moment. The probes are stubbed to the deterministic local-first
state; tests needing other states patch the globals themselves, as
the selector tests always did.

make test now enforces -m "not integration" alongside the directory
ignores, and the acceptance is blunt: 679 passed in ~12 s, identical
against a dead backend and no API key — down from ~129 s of
infrastructure-coupled runtime. Both prongs mutation-checked: the
gate removed fails eight against a dead backend; the stub removed
fails the new session-globals test. T-8 files the orphaned
tests/integration directory that no make target runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 11:29:10 +02:00
jpmschweitzerandClaude Fable 5 b62a6f4672 release v2.7.2
Build and Push / release (push) Successful in 4s
Build and Push / build (push) Successful in 1m51s
Production logs INFO to stdout: docker logs tatlock becomes a real
diagnostic surface for the first time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 11:11:27 +02:00
jpmschweitzerandClaude Fable 5 aace128ef9 fix(logging): T-5 — production logs INFO to stdout
docker logs tatlock has been empty for months and the /app/logs
mount untouched since 2025-12. Not a broken handler: the stdout
StreamHandler was correct all along, but the production default
level was WARNING ("minimal noise") and a healthy service warns
roughly never — startup, backend flavor detection and the wrapper's
balancing/compaction signals were all suppressed, which is exactly
what made the serving cutover unverifiable from logs on 2026-09-12.

Production now defaults to INFO; LOG_LEVEL stays the override for
when noise is ever the real problem; development keeps DEBUG. The
mount was always decorative — no file handler exists, docker logs is
the log. Three tests pin the level table, the WARNING-revert
mutation shown to fail its test.

Also filed T-7: the full suite flaked once and passed on rerun for
the second time — next occurrence keeps the failing run's output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 11:00:50 +02:00
jpmschweitzerandClaude Fable 5 cbec26835d release v2.7.1
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m47s
The T-6 wiring fix reaches production: the sanitized client is in
the request path for the first time — null-content sanitizer,
OLLAMA_TIMEOUT cap and wrapper signal reading all become real.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 10:33:13 +02:00
jpmschweitzerandClaude Fable 5 1139d8ca38 fix(provider): T-6 — the sanitized client is wired for real
The provider assigned self._openai_client, an attribute nobody reads
— OllamaProvider.client serves self._client — so the null-content
sanitizer, the wrapper signal reading and the OLLAMA_TIMEOUT cap were
all silently dead in production (sessions unaffected; they ride
ModelSettings). Found via webber's identical bug on 2026-09-13.

Three fixes travel together, because wiring alone would have traded a
silent bypass for a loud crash: the client goes through the official
openai_client constructor parameter; the parent-chat lookup survives
openai's property/cached_property descriptor change (this venv's
2.11.0 is the latter — the old hardcoded .fget would have raised
AttributeError on the first wired completion; ported from webber's
2026-08-11 fix); and the balancing/compaction signals are read from
the response body's model_extra inside create() — the httpx
event-hook approach demonstrably never fires under the openai SDK.

The tests now assert through provider.client, the property
pydantic_ai actually reads. The old ones asserted on the dead
attribute and stayed green for the entire bypass — a check whose
subject is not the real thing cannot fail for the real reason
(workspace D-24's shape). Three mutations shown to fail their tests:
the dead-attribute wiring, the .fget-only lookup, and removing the
signal read from the request path. 683 tests green; a live dev turn
through the wired client answers normally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 10:27:21 +02:00
jpmschweitzerandClaude Fable 5 d36d2478c0 release v2.7.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m54s
Named sessions through the boilerroom wrapper, ranked by use:
pipeline phases 40, experts 35, librarian 30 — lower parks sooner.
Balancing and compaction signals read on every response.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 09:43:27 +02:00
jpmschweitzerandClaude Fable 5 fd2cd55747 feat(backend): every generation call carries a session name
The release audit swept every agent.run and raw chat call: the two
streaming orchestrate paths never carried a binding (they predate the
v2.6.0 pinning), and the biographer and housekeeper ran bare. Through
the wrapper a session-less call takes idle slots only and 503s once
four sessions are resident — so the streaming paths join
tatlock-orchestrate, and the experts share tatlock-experts at rank 35:
between the librarian and the phases, the two least-used consumers
trade the spare slot by rank instead of anyone hitting an empty pool.
Rank mutation shown to fail its test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-13 09:43:27 +02:00
jpmschweitzerandClaude Fable 5 5b141669ce feat(backend): T-4 — named sessions through the boilerroom wrapper
The flavor probe gains a third answer: the wrapper names itself on
/health, so behind OLLAMA_HOST tatlock now distinguishes boilerroom,
a bare llama-server, and Ollama. Through the wrapper each pipeline
phase is a named session with the decided eviction ranking —
tatlock-steward/-orchestrate/-synthesize at 40, librarian at 30,
lower parks sooner (webber will sit at 20; Open WebUI stays
session-less and can never evict anyone). Against a bare llama-server
the raw id_slot pins survive unchanged, Ollama gets neither, and an
unprobed flavor sends nothing rather than guessing — the backend
stays swappable by env alone. tool_choice through the wrapper follows
the llama-server rule, since that is who answers.

The wrapper's balancing and compaction-due signals are read
everywhere: an httpx response hook on the provider covers every
PydanticAI call, streams included, and the steward's raw call reads
the body extras. Acting on compaction_due is a future ticket — the
signal just must not pass silently.

Verified end to end against the live wrapper: the dev server probed
flavor=boilerroom, a full pipeline turn answered in 5.7 s, and
GET /sessions showed all three phase sessions resident at rank 40
with engine-reported occupancies. The demonstration also filled the
production slot map — session-less prod delegations would have 503d
— cleared by a wrapper restart and filed as boilerroom T-11 (sessions
need an exit). A latent test flaw surfaced too: the ollama
tool_choice test relied on the dev backend probing as ollama; it now
pins the flavor it claims to test.

27 selector tests (9 new), 677 total green; three mutations shown to
fail their tests (librarian rank, the wrapper branch, the no-nudge
set); the new wrapper contract class runs 10/10 against the live
boundary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-12 16:13:29 +02:00
jpmschweitzerandClaude Fable 5 a5681e9511 chore(pql): file T-4 (boilerroom session adoption, ranked) and T-5 (logs go nowhere)
T-4 carries the decided usage ranking — tatlock 40, librarian 30,
webber 20 (workspace T-137), Open WebUI session-less and deprecating.
T-5: docker logs is empty and the log mount untouched since 2025-12,
which made today's cutover verification blind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-12 10:21:04 +02:00
jpmschweitzerandClaude Fable 5 401533d0af chore(pql): file T-3 — stream the synthesis phase
TTFT equals wall time today: nothing streams until the three-phase
pipeline completes, so desklock's time-to-first-word is the full
pipeline (14.9 s mean, measured 2026-09-11). Workspace D-60 names
streaming synthesis a first-class goal of the serving rework.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-11 23:14:17 +02:00
jpmschweitzerandClaude Fable 5 64b6a3d826 release v2.6.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m54s
The cutover's first real lesson ships: tool_choice "required" was a
workaround for a backend that ignored it, and became an unbreakable
tool loop on a backend that obeys it every request (~80 s arithmetic
turns, observed). The health check now learns which server answers
behind OLLAMA_HOST (/props is llama-server's own surface) and only
Ollama gets the advisory nudge. Probed unforced on llama-server: 3/3
tool calls via the --jinja template.

Also carries BACKEND_SLOT_PINNING (default off) for the P3 pilot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-11 22:41:31 +02:00
jpmschweitzerandClaude Fable 5 da5ded48ab fix(ci): the release step tolerates a release that already exists
A re-fired tag hits the release POST with a 409 and curl -sf turns
"already exists" into a red job while the image jobs succeed — observed
on boilerroom's v0.1.0 re-fires tonight; this workflow fails the same
way. Check-then-create makes the step idempotent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-11 22:28:39 +02:00
jpmschweitzerandClaude Fable 5 23217bdb26 feat(backend): BACKEND_SLOT_PINNING — per-phase engine slot ownership
Each pipeline phase owns one llama-server slot (steward 0, orchestrator
1, synthesizer 2), carried as id_slot in extra_body through the same
mechanism tool_choice already uses, so the phase's stable prompt prefix
stays in that slot's KV cache and a turn re-prefills only its new
tokens. Off by default; a no-op on the Claude backend and ignored by
Ollama, so the flag is safe on any backend and the cutover itself stays
a pure env swap.

The merge helper preserves existing extra_body keys — mutation-checked
(dropping the merge fails exactly the test written for it).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-11 22:25:53 +02:00
22 changed files with 790 additions and 51 deletions
+6 -1
View File
@@ -8,11 +8,16 @@ API_HOST=0.0.0.0
API_PORT=8000 API_PORT=8000
API_PREFIX=/v1 API_PREFIX=/v1
# Ollama Configuration (local - primary backend) # Local backend (primary). The var name is legacy: any OpenAI-compatible
# server works — Ollama, a bare llama-server, or the boilerroom wrapper
# (http://127.0.0.1:8602 on tower-of-joy). The flavor is probed at
# startup; through the wrapper, BACKEND_SLOT_PINNING=true sends named
# sessions with the T-4 eviction ranking instead of raw slot pins.
OLLAMA_HOST=http://localhost:11434 OLLAMA_HOST=http://localhost:11434
OLLAMA_DEFAULT_MODEL=gemma4:e2b OLLAMA_DEFAULT_MODEL=gemma4:e2b
OLLAMA_TIMEOUT=120 OLLAMA_TIMEOUT=120
STEWARD_TIMEOUT=60 STEWARD_TIMEOUT=60
BACKEND_SLOT_PINNING=false
# Anthropic Configuration (Claude - cloud fallback) # Anthropic Configuration (Claude - cloud fallback)
# Set ANTHROPIC_API_KEY to keep the Claude fallback available: it is used # Set ANTHROPIC_API_KEY to keep the Claude fallback available: it is used
+10 -2
View File
@@ -10,12 +10,20 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Create Gitea Release - name: Create Gitea Release
# Idempotent: a re-fired tag finds its release already present and
# says so instead of failing on the 409.
run: | run: |
api="${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
auth='Authorization: token ${{ secrets.GITHUB_TOKEN }}'
if curl -sf -H "$auth" "$api/tags/${{ github.ref_name }}" > /dev/null; then
echo "release for ${{ github.ref_name }} already exists — nothing to do"
exit 0
fi
curl -sf -X POST \ curl -sf -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \ -H "$auth" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \ -d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases" "$api"
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
+14
View File
@@ -0,0 +1,14 @@
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'status', 'backlog', 'in_progress', NULL, '2026-09-12 13:50:48', '2026-09-12 13:50:48.289', '2026-09-12 13:50:48.289', NULL, '2e88d155325b3e704787f48b2aa97591', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'status', 'in_progress', 'done', NULL, '2026-09-12 14:13:29', '2026-09-12 14:13:29.027', '2026-09-12 14:13:29.027', NULL, '5b9dd9a62002db0fb4dc58f7f9ffcbb6', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'description', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.
Second finding 2026-09-13, from webber''s twin fix: an httpx event-hook on the client (the T-4 signal-reading approach) never fires under the openai SDK even when correctly wired — webber verified headers arriving while the hook stayed silent. When rewiring the client, read the signals from the parsed body''s model_extra in the sanitized create() instead (the wrapper injects balancing/compaction_due into non-streamed JSON bodies; openai pydantic models retain unknown fields). The steward''s raw-call body read is unaffected and already correct.', NULL, '2026-09-13 08:00:17', '2026-09-13 08:00:17.002', '2026-09-13 08:00:17.002', NULL, '6fb4cc1d7d7222e3d570cd8c7fed4fd9', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'status', 'backlog', 'in_progress', NULL, '2026-09-13 08:19:35', '2026-09-13 08:19:35.028', '2026-09-13 08:19:35.028', NULL, '5bcf19379361a2f766cd4787ff734611', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'status', 'in_progress', 'done', NULL, '2026-09-13 08:27:21', '2026-09-13 08:27:21.779', '2026-09-13 08:27:21.779', NULL, '1c59422336b88a76c6f1c3720916d7d5', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'status', 'backlog', 'in_progress', NULL, '2026-09-13 08:48:55', '2026-09-13 08:48:55.587', '2026-09-13 08:48:55.587', NULL, 'ca4f8fb42c961b0002aa87bb7b1d5af5', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'status', 'in_progress', 'done', NULL, '2026-09-13 09:00:50', '2026-09-13 09:00:50.006', '2026-09-13 09:00:50.006', NULL, '5ea799ee4815e42ede80c3df4c81b28c', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'description', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.
Third occurrence 2026-09-13, during the v2.7.2 release gate — and two process lessons captured the hard way: (1) the gate run was piped through grep, which swallowed make''s exit code, so the tag went out on a red run (release content was safe — version+changelog only on green-tested aace128 — but the gate did not gate); release runs must let make test''s exit code stand alone. (2) The failing name went uncaptured AGAIN because the tee capture only started on the rerun, which passed. The capture must wrap the FIRST gate run, every time, not begin after a failure.', NULL, '2026-09-13 09:14:10', '2026-09-13 09:14:10.345', '2026-09-13 09:14:10.345', NULL, '22909a81b656320b4a39405165f48612', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'status', 'backlog', 'in_progress', NULL, '2026-09-13 09:16:08', '2026-09-13 09:16:08.928', '2026-09-13 09:16:08.928', NULL, '34276614ec535d50d5fa6ddf9f5e0da6', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'status', 'in_progress', 'done', NULL, '2026-09-13 09:29:10', '2026-09-13 09:29:10.192', '2026-09-13 09:29:10.192', NULL, 'e040af9004e4510c4f8a9e9cacee9a8c', 2) ON CONFLICT(hash) DO NOTHING;
+6
View File
@@ -0,0 +1,6 @@
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G94KC9XM3PW91C6B7VJA6VRM', 'T-3', '2026-09-11 21:09:10.267', '2026-09-11 21:09:10.267', NULL, 'a1ccf4d118877c230dffb961a7531cb4', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'T-4', '2026-09-12 08:20:21.494', '2026-09-12 08:20:21.494', NULL, '5b28e07bdc3be1f9ed0ed71d2da60c72', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'T-5', '2026-09-12 08:20:21.625', '2026-09-12 08:20:21.625', NULL, '0a417012779f612a4dad1a7dbd0787e3', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'T-6', '2026-09-13 07:57:01.782', '2026-09-13 07:57:01.782', NULL, 'd6c184ba6d24e03c0dc77064bf71fead', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'T-7', '2026-09-13 09:00:32.255', '2026-09-13 09:00:32.255', NULL, 'b0b0a4c3ccaabeb58ad1958e7c2c2602', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9M68FC27YHQSP1R9SQQ87P0', 'T-8', '2026-09-13 09:28:48.737', '2026-09-13 09:28:48.737', NULL, 'd3a29143f5496ea4b4ee8a9a834c5290', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
+28
View File
@@ -0,0 +1,28 @@
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G94KC9XM3PW91C6B7VJA6VRM', 'story', NULL, 'Stream the synthesis phase — TTFT equals wall time today', 'Nothing streams until steward-orchestrate-synthesize completes, so desklock''s time-to-first-word is the full pipeline (TTFT mean 14.9s, max 26.2s — measured 2026-09-11, workspace note cladmin/agent/notes/2026-09-11-serving-baseline.md). Stream the synthesis phase out as it generates: a first-class goal of the serving rework independent of KV reuse (workspace D-60). The backend now streams /v1 SSE natively (llama-gen).', 'backlog', 'high', NULL, NULL, NULL, '2026-09-11 21:09:10.253', '2026-09-11 21:09:10.253', NULL, 'f7d31ca4fe747835a32744c310f51674', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'story', NULL, 'Adopt boilerroom named sessions with the usage ranking', 'Replace raw id_slot pinning (BACKEND_SLOT_PINNING) with the wrapper''s session extension fields, now that tatlock routes through boilerroom:8602 (since 2026-09-12). Ranking by likelihood of use, decided 2026-09-12 — lower eviction_order parks sooner: tatlock pipeline phases highest (eviction_order 40; e.g. sessions tatlock-steward/tatlock-orchestrate/tatlock-synthesize), librarian-routed calls as session librarian (30), webber will sit at 20 (workspace T-137), Open WebUI stays session-less by design (idle slots only, can never evict — and it is being deprecated for an own-webui build). Read the x-boilerroom-balancing and compaction_due signals; the compaction endpoint expects the client transcript (tatlock''s Redis/Qdrant context is the natural producer — boilerroom D-5 names tatlock the first producer).', 'backlog', 'high', NULL, NULL, NULL, '2026-09-12 08:20:21.493', '2026-09-12 08:20:21.493', NULL, '3fb8c350876fc056648a19e0f2b7718c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'bug', NULL, 'The container logs nowhere: stdout empty, /app/logs mount untouched', 'docker logs tatlock is 0 lines and the mounted ~/docker-data/tatlock/logs has been empty since 2025-12 — the running app writes logs to neither, which made the 2026-09-12 wrapper-cutover verification blind (flavor detection had to be proven behaviorally). Dev mode logs to build/logs/server.log via make run; production should log to stdout for docker logs at minimum.', 'backlog', 'medium', NULL, NULL, NULL, '2026-09-12 08:20:21.624', '2026-09-12 08:20:21.624', NULL, '3fd294efc9afa857aef15810d9003298', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'story', NULL, 'Adopt boilerroom named sessions with the usage ranking', 'Replace raw id_slot pinning (BACKEND_SLOT_PINNING) with the wrapper''s session extension fields, now that tatlock routes through boilerroom:8602 (since 2026-09-12). Ranking by likelihood of use, decided 2026-09-12 — lower eviction_order parks sooner: tatlock pipeline phases highest (eviction_order 40; e.g. sessions tatlock-steward/tatlock-orchestrate/tatlock-synthesize), librarian-routed calls as session librarian (30), webber will sit at 20 (workspace T-137), Open WebUI stays session-less by design (idle slots only, can never evict — and it is being deprecated for an own-webui build). Read the x-boilerroom-balancing and compaction_due signals; the compaction endpoint expects the client transcript (tatlock''s Redis/Qdrant context is the natural producer — boilerroom D-5 names tatlock the first producer).', 'in_progress', 'high', NULL, NULL, NULL, '2026-09-12 08:20:21.493', '2026-09-12 13:50:48.289', NULL, '62abeb4da2f1f7aedb8d92884e7e148c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07EM5DZWTS0WD9V8PSKM', 'story', NULL, 'Adopt boilerroom named sessions with the usage ranking', 'Replace raw id_slot pinning (BACKEND_SLOT_PINNING) with the wrapper''s session extension fields, now that tatlock routes through boilerroom:8602 (since 2026-09-12). Ranking by likelihood of use, decided 2026-09-12 — lower eviction_order parks sooner: tatlock pipeline phases highest (eviction_order 40; e.g. sessions tatlock-steward/tatlock-orchestrate/tatlock-synthesize), librarian-routed calls as session librarian (30), webber will sit at 20 (workspace T-137), Open WebUI stays session-less by design (idle slots only, can never evict — and it is being deprecated for an own-webui build). Read the x-boilerroom-balancing and compaction_due signals; the compaction endpoint expects the client transcript (tatlock''s Redis/Qdrant context is the natural producer — boilerroom D-5 names tatlock the first producer).', 'done', 'high', NULL, NULL, NULL, '2026-09-12 08:20:21.493', '2026-09-12 14:13:29.027', NULL, '89ce1be323ebb26f00c63569f197cab1', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'bug', NULL, 'The sanitized OpenAI client is never used: the override pokes an attribute nobody reads', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.', 'backlog', 'high', NULL, NULL, NULL, '2026-09-13 07:57:01.782', '2026-09-13 07:57:01.782', NULL, '5e8ea3bf2d18ade9d17f0044ae1ff96b', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'bug', NULL, 'The sanitized OpenAI client is never used: the override pokes an attribute nobody reads', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.
Second finding 2026-09-13, from webber''s twin fix: an httpx event-hook on the client (the T-4 signal-reading approach) never fires under the openai SDK even when correctly wired — webber verified headers arriving while the hook stayed silent. When rewiring the client, read the signals from the parsed body''s model_extra in the sanitized create() instead (the wrapper injects balancing/compaction_due into non-streamed JSON bodies; openai pydantic models retain unknown fields). The steward''s raw-call body read is unaffected and already correct.', 'backlog', 'high', NULL, NULL, NULL, '2026-09-13 07:57:01.782', '2026-09-13 08:00:17.002', NULL, '755abc3689e8ff4fc511deb164137fe5', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'bug', NULL, 'The sanitized OpenAI client is never used: the override pokes an attribute nobody reads', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.
Second finding 2026-09-13, from webber''s twin fix: an httpx event-hook on the client (the T-4 signal-reading approach) never fires under the openai SDK even when correctly wired — webber verified headers arriving while the hook stayed silent. When rewiring the client, read the signals from the parsed body''s model_extra in the sanitized create() instead (the wrapper injects balancing/compaction_due into non-streamed JSON bodies; openai pydantic models retain unknown fields). The steward''s raw-call body read is unaffected and already correct.', 'in_progress', 'high', NULL, NULL, NULL, '2026-09-13 07:57:01.782', '2026-09-13 08:19:35.028', NULL, '5af3c96930a30bbe4bb64d13ddbc509e', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KH87TRDGW45GMDWVDRPABR', 'bug', NULL, 'The sanitized OpenAI client is never used: the override pokes an attribute nobody reads', 'src/ollama/provider.py sets self._openai_client after super().__init__, but OllamaProvider.client serves self._client (pydantic_ai 1.27''s official openai_client= constructor param is the right wiring — webber hit the identical bug and fixed it 2026-09-13, exposed by the wrapper 503ing a session-less request). Silently dead in production as a result: the content-null sanitizer (moot on llama-server, live hazard if Ollama ever returns), the T-4 x-boilerroom-* signal hook (never logs), and the OLLAMA_TIMEOUT=120 cap (SDK default ~600s applies). Sessions were unaffected — they ride ModelSettings extra_body. Fix like webber: pass _SanitizedAsyncOpenAI through openai_client=, add a wiring test asserting provider.client is the sanitized type, then verify balancing/compaction logs actually appear.
Second finding 2026-09-13, from webber''s twin fix: an httpx event-hook on the client (the T-4 signal-reading approach) never fires under the openai SDK even when correctly wired — webber verified headers arriving while the hook stayed silent. When rewiring the client, read the signals from the parsed body''s model_extra in the sanitized create() instead (the wrapper injects balancing/compaction_due into non-streamed JSON bodies; openai pydantic models retain unknown fields). The steward''s raw-call body read is unaffected and already correct.', 'done', 'high', NULL, NULL, NULL, '2026-09-13 07:57:01.782', '2026-09-13 08:27:21.778', NULL, 'ce3151ff27c70a1028ba7c03b07119a2', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'bug', NULL, 'The container logs nowhere: stdout empty, /app/logs mount untouched', 'docker logs tatlock is 0 lines and the mounted ~/docker-data/tatlock/logs has been empty since 2025-12 — the running app writes logs to neither, which made the 2026-09-12 wrapper-cutover verification blind (flavor detection had to be proven behaviorally). Dev mode logs to build/logs/server.log via make run; production should log to stdout for docker logs at minimum.', 'in_progress', 'medium', NULL, NULL, NULL, '2026-09-12 08:20:21.624', '2026-09-13 08:48:55.586', NULL, '4770f6c6759cb82d825eb550dd4e42e2', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'bug', NULL, 'The full suite flakes: one run fails, the rerun passes, twice now', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.', 'backlog', 'medium', NULL, NULL, NULL, '2026-09-13 09:00:32.255', '2026-09-13 09:00:32.255', NULL, '36c6f97b5a309e169b7acb35f7ca46d4', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G99D07Z3FZY8EBT9PJ1RCGHW', 'bug', NULL, 'The container logs nowhere: stdout empty, /app/logs mount untouched', 'docker logs tatlock is 0 lines and the mounted ~/docker-data/tatlock/logs has been empty since 2025-12 — the running app writes logs to neither, which made the 2026-09-12 wrapper-cutover verification blind (flavor detection had to be proven behaviorally). Dev mode logs to build/logs/server.log via make run; production should log to stdout for docker logs at minimum.', 'done', 'medium', NULL, NULL, NULL, '2026-09-12 08:20:21.624', '2026-09-13 09:00:50.006', NULL, 'd04562943a55527d6036ec38c2657308', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'bug', NULL, 'The full suite flakes: one run fails, the rerun passes, twice now', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.
Third occurrence 2026-09-13, during the v2.7.2 release gate — and two process lessons captured the hard way: (1) the gate run was piped through grep, which swallowed make''s exit code, so the tag went out on a red run (release content was safe — version+changelog only on green-tested aace128 — but the gate did not gate); release runs must let make test''s exit code stand alone. (2) The failing name went uncaptured AGAIN because the tee capture only started on the rerun, which passed. The capture must wrap the FIRST gate run, every time, not begin after a failure.', 'backlog', 'medium', NULL, NULL, NULL, '2026-09-13 09:00:32.255', '2026-09-13 09:14:10.345', NULL, 'ba4c6ab575af14f50acd3a29ff5e7ef8', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'bug', NULL, 'The full suite flakes: one run fails, the rerun passes, twice now', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.
Third occurrence 2026-09-13, during the v2.7.2 release gate — and two process lessons captured the hard way: (1) the gate run was piped through grep, which swallowed make''s exit code, so the tag went out on a red run (release content was safe — version+changelog only on green-tested aace128 — but the gate did not gate); release runs must let make test''s exit code stand alone. (2) The failing name went uncaptured AGAIN because the tee capture only started on the rerun, which passed. The capture must wrap the FIRST gate run, every time, not begin after a failure.', 'in_progress', 'medium', NULL, NULL, NULL, '2026-09-13 09:00:32.255', '2026-09-13 09:16:08.928', NULL, '660f20aa80dccf32427c3c3e26c46c8c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9M68FC27YHQSP1R9SQQ87P0', 'task', NULL, 'tests/integration/ runs under no make target', 'Surfaced during T-7: make test ignores tests/integration by directory, and make test-integration runs only tests/agents/test_tatlock_agent.py — so the marked streaming tests in tests/integration/test_tatlock_streaming.py execute under no target at all. Fold them into test-integration (and check they still pass against the current wrapper topology) or retire them deliberately.', 'backlog', 'medium', NULL, NULL, NULL, '2026-09-13 09:28:48.737', '2026-09-13 09:28:48.737', NULL, '50b6e6d25ab009bd17a23544b08e722a', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G9KZSCFR79G4DFCT4E5W5CAM', 'bug', NULL, 'The full suite flakes: one run fails, the rerun passes, twice now', 'Observed 2026-09-13 twice: before the v2.7.0 release audit and again during T-5 — make test exits 1 with a failure that does not reproduce on the immediate rerun, and both times the failing test''s name went uncaptured because the diagnostic rerun passed. Next occurrence: keep the FULL output of the failing run (make test 2>&1 | tee /tmp/flake.log) before rerunning, then pin the named test. Suspects worth checking when a name exists: order-dependent state in the session-scoped app init (conftest asyncio.run + cached health globals), port/redis contention with the live containers, and the coverage plugin''s tmp handling.
Third occurrence 2026-09-13, during the v2.7.2 release gate — and two process lessons captured the hard way: (1) the gate run was piped through grep, which swallowed make''s exit code, so the tag went out on a red run (release content was safe — version+changelog only on green-tested aace128 — but the gate did not gate); release runs must let make test''s exit code stand alone. (2) The failing name went uncaptured AGAIN because the tee capture only started on the rerun, which passed. The capture must wrap the FIRST gate run, every time, not begin after a failure.', 'done', 'medium', NULL, NULL, NULL, '2026-09-13 09:00:32.255', '2026-09-13 09:29:10.191', NULL, 'b15a45c0eaa13b417afe5257236ed0ec', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
+94
View File
@@ -7,6 +7,100 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Fixed
- The unit suite is hermetic and the flake class is gone (T-7). Two
live couplings hid inside `make test`: the session init ran the real
backend health probes (an HTTP round trip to OLLAMA_HOST and a real
Anthropic API call whenever the dev .env carries a key), and the
eight integration-marked tests in `tests/agents/test_tatlock_agent.py`
ran in the unit gate because it excluded by directory while they
live outside the ignored directories — ~110 seconds of real LLM
inference per "unit" run, failing whenever the backend was
mid-deploy. The probes are now stubbed to the deterministic
local-first state (a session test pins it), the gate enforces
`-m "not integration"`, and the suite runs in ~12 s with identical
results against a dead backend. T-8 files the orphaned
`tests/integration/` directory that no make target runs.
## [2.7.2] - 2026-09-13
### Fixed
- Production logs exist now (T-5): the production default log level
was WARNING ("minimal noise"), and a healthy service warns roughly
never — so `docker logs tatlock` was empty for months, startup,
backend flavor detection and the wrapper's balancing/compaction
signals all suppressed, and the serving cutover could not be
verified from logs. The production default is INFO on stdout;
`LOG_LEVEL` remains the override for when noise is ever the real
problem. (The `/app/logs` mount was always decorative — no file
handler exists; `docker logs` is the log.)
## [2.7.1] - 2026-09-13
### Fixed
- The sanitized OpenAI client was never used (T-6): the provider
assigned `self._openai_client`, an attribute nobody reads —
`OllamaProvider.client` serves `self._client` — so the null-content
sanitizer, the wrapper signal reading and the `OLLAMA_TIMEOUT` cap
were all silently dead (sessions were unaffected; they ride
ModelSettings). Found via webber's identical bug. The client now
goes through the constructor's official `openai_client` parameter,
the parent-chat lookup survives openai's property/cached_property
descriptor change (ported from webber's 2026-08-11 fix — the old
`.fget` access would have crashed the first wired completion on
openai 2.11), and the wrapper's `balancing`/`compaction_due`
signals are read from the response body's `model_extra` in the
request path — the httpx event-hook approach demonstrably never
fires under the openai SDK. The provider tests now assert through
`provider.client`, the property pydantic_ai actually reads; the
old ones asserted on the dead attribute and stayed green
throughout.
## [2.7.0] - 2026-09-13
### Added
- Named sessions through the boilerroom wrapper (T-4): the backend
flavor probe now recognizes the wrapper by its `/health` service
name, and each pipeline phase sends `session`/`eviction_order`
extension fields instead of raw `id_slot``tatlock-steward`,
`tatlock-orchestrate`, `tatlock-synthesize` at rank 40 and the
librarian at 30, per the 2026-09-12 usage ranking (lower parks
sooner; webber will sit at 20, Open WebUI stays session-less).
Against a bare llama-server the old slot pins still apply, and
Ollama gets neither, so the backend remains swappable by env alone.
The wrapper's `x-boilerroom-balancing` and compaction-due signals
are read on every response (an httpx hook on the provider, body
extras on the steward's raw call) and logged — acting on
`compaction_due` is a future ticket. `get_model_info()` reports the
probed `local_flavor`. Wire-pinned by a new boilerroom wrapper
contract class in `make test-contracts`.
- Every generation call is named: the streaming orchestrate paths and
the biographer/housekeeper experts (shared session `tatlock-experts`,
rank 35) — through the wrapper a session-less call takes idle slots
only and would 503 once four sessions are resident, so the release
audit swept every `agent.run` and raw chat call under a session.
## [2.6.0] - 2026-09-11
### Added
- `BACKEND_SLOT_PINNING` (default off): pins each pipeline phase to a
llama-server slot (steward=0, orchestrator=1, synthesizer=2) so a
phase's stable prompt prefix stays in that slot's KV cache. No-op on
Claude; ignored by Ollama.
### Fixed
- Tool-calling no longer loops on llama-server backends: the health
check detects the server flavor, and `tool_choice: "required"` (an
advisory nudge on Ollama, an every-request mandate on llama-server)
is sent only to Ollama. On llama-server the model calls tools
unforced via its chat template.
## [2.5.0] - 2026-09-11 ## [2.5.0] - 2026-09-11
### Added ### Added
+1 -1
View File
@@ -40,7 +40,7 @@ run: ## Start the development server on port 8777
$(UVICORN) src.main:app --reload --host $(HOST) --port $(PORT) 2>&1 | tee build/logs/server.log $(UVICORN) src.main:app --reload --host $(HOST) --port $(PORT) 2>&1 | tee build/logs/server.log
test: ## Run unit tests (no external services needed) test: ## Run unit tests (no external services needed)
$(PYTEST) --ignore=tests/e2e --ignore=tests/integration --ignore=tests/contracts $(PYTEST) --ignore=tests/e2e --ignore=tests/integration --ignore=tests/contracts -m "not integration"
test-unit: test ## Alias for test test-unit: test ## Alias for test
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "tatlock" name = "tatlock"
version = "2.5.0" version = "2.7.2"
description = "OpenAI-compatible API with Ollama backend" description = "OpenAI-compatible API with Ollama backend"
requires-python = ">=3.12" requires-python = ">=3.12"
dependencies = [ dependencies = [
+3
View File
@@ -191,9 +191,12 @@ async def run_biographer(
) )
try: try:
from src.anthropic.model_selector import with_phase_binding
result = await agent.run( result = await agent.run(
prompt, prompt,
message_history=message_history, message_history=message_history,
model_settings=with_phase_binding(None, "experts"),
) )
logger.info( logger.info(
+2 -2
View File
@@ -206,12 +206,12 @@ async def run_housekeeper(
try: try:
# Temperature 0.1 for slight exploration (skipped on Claude backend) # Temperature 0.1 for slight exploration (skipped on Claude backend)
from src.anthropic.model_selector import get_sampling_settings from src.anthropic.model_selector import get_sampling_settings, with_phase_binding
result = await agent.run( result = await agent.run(
prompt, prompt,
message_history=message_history, message_history=message_history,
model_settings=get_sampling_settings(0.1), model_settings=with_phase_binding(get_sampling_settings(0.1), "experts"),
) )
logger.info( logger.info(
+3
View File
@@ -272,11 +272,14 @@ async def run_librarian(
) )
try: try:
from src.anthropic.model_selector import with_phase_binding
# One shared library-desk connection for all tool calls in this run # One shared library-desk connection for all tool calls in this run
async with library_client_session(): async with library_client_session():
result = await agent.run( result = await agent.run(
prompt, prompt,
message_history=message_history, message_history=message_history,
model_settings=with_phase_binding(None, "librarian"),
) )
logger.info( logger.info(
+24 -6
View File
@@ -162,19 +162,37 @@ class StewardAgent:
OpenAI-compatible server (Ollama, llama-server) can sit behind OpenAI-compatible server (Ollama, llama-server) can sit behind
OLLAMA_HOST without this method knowing which. OLLAMA_HOST without this method knowing which.
""" """
from src.anthropic.model_selector import phase_extra_body
payload: dict = {
"model": self.ollama_model,
"messages": [{"role": "user", "content": prompt}],
"temperature": 0.3, # Lower = more consistent
"top_p": 0.9,
}
# Session through the wrapper, id_slot 0 against a bare
# llama-server, nothing elsewhere (repo T-4).
payload.update(phase_extra_body("steward"))
async with httpx.AsyncClient(timeout=self.timeout) as client: async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post( response = await client.post(
f"{self.ollama_host}/v1/chat/completions", f"{self.ollama_host}/v1/chat/completions",
json={ json=payload,
"model": self.ollama_model,
"messages": [{"role": "user", "content": prompt}],
"temperature": 0.3, # Lower = more consistent
"top_p": 0.9,
},
) )
response.raise_for_status() response.raise_for_status()
result = response.json() result = response.json()
# The wrapper reports on session-carrying responses (its
# D-4/D-5); this raw call bypasses the provider hook, so
# the body extras are read here. Acting on compaction_due
# is a later ticket — the signal must not pass silently.
balancing = result.get("balancing")
if balancing:
logger.info("backend_balancing", session="tatlock-steward", moves=balancing)
if result.get("compaction_due"):
logger.warning("backend_compaction_due", session="tatlock-steward")
return result["choices"][0]["message"]["content"].strip() return result["choices"][0]["message"]["content"].strip()
async def analyze(self, query: str, conversation_history: list[dict] | None = None) -> str: async def analyze(self, query: str, conversation_history: list[dict] | None = None) -> str:
+13 -4
View File
@@ -404,10 +404,13 @@ class TatlockAgent(AgentInterface):
# with async context managers inside generators # with async context managers inside generators
# The StreamingCoordinator will handle word-by-word streaming # The StreamingCoordinator will handle word-by-word streaming
# Pass message_history to maintain conversation context and tracker for tool logging # Pass message_history to maintain conversation context and tracker for tool logging
from src.anthropic.model_selector import with_phase_binding
result = await self.agent.run( result = await self.agent.run(
user_message, user_message,
message_history=message_history if message_history else None, message_history=message_history if message_history else None,
deps=tracker, deps=tracker,
model_settings=with_phase_binding(None, "orchestrate"),
) )
final_text = result.output final_text = result.output
@@ -536,13 +539,13 @@ class TatlockAgent(AgentInterface):
# Run with scoped tools and tracker # Run with scoped tools and tracker
# Force tool_choice to make LLM actually call tools # Force tool_choice to make LLM actually call tools
from src.anthropic.model_selector import get_tool_choice_settings from src.anthropic.model_selector import get_tool_choice_settings, with_phase_binding
result = await scoped_agent.run( result = await scoped_agent.run(
enriched_message, enriched_message,
message_history=pydantic_history if pydantic_history else None, message_history=pydantic_history if pydantic_history else None,
deps=tool_tracker, deps=tool_tracker,
model_settings=get_tool_choice_settings(), model_settings=with_phase_binding(get_tool_choice_settings(), "orchestrate"),
) )
logger.info( logger.info(
@@ -625,10 +628,13 @@ class TatlockAgent(AgentInterface):
# Use run() instead of run_stream() to avoid Ollama 400 bug # Use run() instead of run_stream() to avoid Ollama 400 bug
# with streaming + tool calls (PydanticAI issues #1292, #2256) # with streaming + tool calls (PydanticAI issues #1292, #2256)
# We yield the final response in chunks to maintain streaming interface # We yield the final response in chunks to maintain streaming interface
from src.anthropic.model_selector import with_phase_binding
result = await scoped_agent.run( result = await scoped_agent.run(
enriched_message, enriched_message,
message_history=pydantic_history if pydantic_history else None, message_history=pydantic_history if pydantic_history else None,
deps=tool_tracker, deps=tool_tracker,
model_settings=with_phase_binding(None, "orchestrate"),
) )
# Stream the final response in chunks to maintain UX # Stream the final response in chunks to maintain UX
@@ -725,13 +731,13 @@ class TatlockAgent(AgentInterface):
pydantic_history.append(ModelResponse(parts=[TextPart(content=content)])) pydantic_history.append(ModelResponse(parts=[TextPart(content=content)]))
# Run with scoped tools and tracker # Run with scoped tools and tracker
from src.anthropic.model_selector import get_tool_choice_settings from src.anthropic.model_selector import get_tool_choice_settings, with_phase_binding
result = await scoped_agent.run( result = await scoped_agent.run(
enriched_message, enriched_message,
message_history=pydantic_history if pydantic_history else None, message_history=pydantic_history if pydantic_history else None,
deps=tool_tracker, deps=tool_tracker,
model_settings=get_tool_choice_settings(), model_settings=with_phase_binding(get_tool_choice_settings(), "orchestrate"),
) )
# Extract tool calls and results from the agent's messages # Extract tool calls and results from the agent's messages
@@ -888,9 +894,12 @@ class TatlockAgent(AgentInterface):
pydantic_history.append(ModelResponse(parts=[TextPart(content=content)])) pydantic_history.append(ModelResponse(parts=[TextPart(content=content)]))
# Run synthesis # Run synthesis
from src.anthropic.model_selector import with_phase_binding
result = await synthesis_agent.run( result = await synthesis_agent.run(
synthesis_prompt, synthesis_prompt,
message_history=pydantic_history if pydantic_history else None, message_history=pydantic_history if pydantic_history else None,
model_settings=with_phase_binding(None, "synthesize"),
) )
logger.info( logger.info(
+115 -7
View File
@@ -28,6 +28,13 @@ logger = get_logger(__name__)
# Cached health check results (set once at startup) # Cached health check results (set once at startup)
_claude_available: bool | None = None _claude_available: bool | None = None
_ollama_available: bool | None = None _ollama_available: bool | None = None
# Which server answers behind OLLAMA_HOST: "boilerroom", "llama-server"
# or "ollama". They disagree on tool_choice semantics and on which
# extension fields do anything (see get_tool_choice_settings and
# phase_extra_body), so the health check probes for the wrapper's name
# on /health, then for /props — served by llama-server only — and
# caches the answer here.
_local_flavor: str | None = None
async def check_ollama_health() -> bool: async def check_ollama_health() -> bool:
@@ -40,7 +47,7 @@ async def check_ollama_health() -> bool:
Returns: Returns:
True if Ollama is reachable and OLLAMA_DEFAULT_MODEL is pulled. True if Ollama is reachable and OLLAMA_DEFAULT_MODEL is pulled.
""" """
global _ollama_available global _ollama_available, _local_flavor
host = str(config.OLLAMA_HOST).rstrip("/") host = str(config.OLLAMA_HOST).rstrip("/")
model = config.OLLAMA_DEFAULT_MODEL model = config.OLLAMA_DEFAULT_MODEL
@@ -53,12 +60,34 @@ async def check_ollama_health() -> bool:
response.raise_for_status() response.raise_for_status()
names = [m.get("id", "") for m in response.json().get("data", [])] names = [m.get("id", "") for m in response.json().get("data", [])]
# The boilerroom wrapper names itself on /health; a bare
# llama-server answers /health without a service field and
# serves /props (its own surface); Ollama answers neither.
is_wrapper = False
try:
health = await client.get(f"{host}/health")
is_wrapper = (
health.status_code == 200 and health.json().get("service") == "boilerroom"
)
except (httpx.HTTPError, ValueError):
is_wrapper = False
if is_wrapper:
_local_flavor = "boilerroom"
else:
try:
props = await client.get(f"{host}/props")
_local_flavor = "llama-server" if props.status_code == 200 else "ollama"
except httpx.HTTPError:
_local_flavor = "ollama"
if model in names or f"{model}:latest" in names: if model in names or f"{model}:latest" in names:
_ollama_available = True _ollama_available = True
logger.info( logger.info(
"ollama_health_check_passed", "ollama_health_check_passed",
host=host, host=host,
model=model, model=model,
flavor=_local_flavor,
) )
return True return True
@@ -244,19 +273,97 @@ def get_model(prefer_cloud: bool | None = None) -> AnthropicModel | OpenAIChatMo
def get_tool_choice_settings() -> ModelSettings: def get_tool_choice_settings() -> ModelSettings:
""" """
Get model_settings for forcing tool calls on the first request. Get model_settings for tool calling on the orchestration phase.
For Claude: PydanticAI handles tool_choice natively, so no extra_body needed. Claude: PydanticAI handles tool_choice natively no extra_body.
For Ollama: Pass tool_choice="required" via extra_body to force tool calling.
Ollama: tool_choice="required" via extra_body. Advisory there (Ollama
ignores it), but it nudges gemma4 to actually call tools, which the
persona-suppression gotcha made necessary.
llama-server: NO tool_choice. It enforces "required" on every request
in the run, so after a tool returns, the next generation is again
forced to call a tool — an unbreakable tool loop (~80 s turns,
observed at cutover). Its --jinja template renders tool definitions
the way gemma4 was trained, and the model calls tools reliably
unforced (probed 3/3).
boilerroom: same as llama-server — the wrapper forwards to it, and
its /props passthrough is what lets this detection keep working.
""" """
from pydantic_ai.settings import ModelSettings from pydantic_ai.settings import ModelSettings
if resolve_backend() == "claude": if resolve_backend() == "claude":
# PydanticAI's Anthropic model handles tool_choice internally # PydanticAI's Anthropic model handles tool_choice internally
return ModelSettings() return ModelSettings()
else: if _local_flavor in ("llama-server", "boilerroom"):
# Ollama needs explicit tool_choice via extra_body return ModelSettings()
return ModelSettings(extra_body={"tool_choice": "required"}) # Ollama needs explicit tool_choice via extra_body
return ModelSettings(extra_body={"tool_choice": "required"})
# Phase bindings for the local backend (this repo's T-4, decided
# 2026-09-12): through the boilerroom wrapper each phase is a named
# session with an eviction rank — lower parks sooner, so the most-used
# context keeps its KV warm. The pipeline phases outrank the librarian,
# which outranks webber (20, filed in its own repo); Open WebUI stays
# session-less and can never evict anyone (wrapper D-4).
# phase -> (direct llama-server slot, wrapper session name, eviction rank)
_PHASE_BINDINGS: dict[str, tuple[int | None, str, int]] = {
"steward": (0, "tatlock-steward", 40),
"orchestrate": (1, "tatlock-orchestrate", 40),
"synthesize": (2, "tatlock-synthesize", 40),
# Experts (biographer, housekeeper) share one session between the
# librarian and the phases: the two least-used consumers trade the
# spare slot by rank instead of anyone hitting an empty pool — a
# session-less call 503s once four sessions are resident (D-4's
# idle-only rule), so every generation call here must carry a name.
"experts": (None, "tatlock-experts", 35),
"librarian": (None, "librarian", 30),
}
def phase_extra_body(phase: str) -> dict[str, str | int]:
"""
The extension fields a pipeline phase sends to the local backend.
Through the boilerroom wrapper: a named session with its eviction
rank — the wrapper owns the name-to-slot map and reports parks and
restores on the response. Against a bare llama-server: the raw
id_slot pin this replaces (the librarian floats there, as it always
did). Ollama and Claude get nothing, and an unprobed flavor sends
nothing rather than guessing. Gated by BACKEND_SLOT_PINNING like
the pinning it grew out of.
"""
if not config.BACKEND_SLOT_PINNING or resolve_backend() == "claude":
return {}
slot, session, rank = _PHASE_BINDINGS[phase]
if _local_flavor == "boilerroom":
return {"session": session, "eviction_order": rank}
if _local_flavor == "llama-server" and slot is not None:
return {"id_slot": slot}
return {}
def with_phase_binding(settings: ModelSettings | None, phase: str) -> ModelSettings | None:
"""
Merge a phase's local-backend binding into model settings.
Session fields through the wrapper, id_slot against a bare
llama-server, settings untouched everywhere else — one call site
stays correct on any backend (see phase_extra_body).
"""
extra = phase_extra_body(phase)
if not extra:
return settings
from pydantic_ai.settings import ModelSettings
merged = dict(settings or {})
extra_body = dict(merged.get("extra_body") or {})
extra_body.update(extra)
merged["extra_body"] = extra_body
return ModelSettings(**merged)
def get_sampling_settings(temperature: float) -> ModelSettings: def get_sampling_settings(temperature: float) -> ModelSettings:
@@ -291,5 +398,6 @@ def get_model_info() -> dict:
"claude_configured": bool(config.ANTHROPIC_API_KEY), "claude_configured": bool(config.ANTHROPIC_API_KEY),
"ollama_available": is_ollama_available(), "ollama_available": is_ollama_available(),
"ollama_model": config.OLLAMA_DEFAULT_MODEL, "ollama_model": config.OLLAMA_DEFAULT_MODEL,
"local_flavor": _local_flavor,
"prefer_cloud": config.PREFER_CLOUD_BACKEND, "prefer_cloud": config.PREFER_CLOUD_BACKEND,
} }
+13 -3
View File
@@ -147,6 +147,10 @@ class Config(BaseSettings):
default=None, default=None,
description="OpenAI-compatible embeddings host; falls back to OLLAMA_HOST so gen and embed can live on different servers", description="OpenAI-compatible embeddings host; falls back to OLLAMA_HOST so gen and embed can live on different servers",
) )
BACKEND_SLOT_PINNING: bool = Field(
default=False,
description="Pin each pipeline phase to a llama-server slot (steward=0, orchestrator=1, synthesizer=2) so a phase's stable prompt prefix stays in that slot's KV cache; no-op on Claude, ignored by Ollama",
)
# Redis Memory Database # Redis Memory Database
REDIS_MEMORY_DB: int = Field(default=1, description="Redis database number for memory cache") REDIS_MEMORY_DB: int = Field(default=1, description="Redis database number for memory cache")
@@ -226,15 +230,21 @@ class Config(BaseSettings):
Get effective log level, auto-determining from environment if not set. Get effective log level, auto-determining from environment if not set.
- development: DEBUG (maximum verbosity) - development: DEBUG (maximum verbosity)
- production: WARNING (minimal noise) - production: INFO
- testing: INFO - testing: INFO
Production ran at WARNING ("minimal noise") until T-5: a healthy
service warns roughly never, so `docker logs` was empty for
months — startup, backend flavor detection and the wrapper's
balancing/compaction signals all suppressed, which made the
2026-09-12 serving cutover unverifiable from logs. INFO on
stdout is the operational floor; LOG_LEVEL quiets it when noise
ever becomes the real problem.
""" """
if self.LOG_LEVEL is not None: if self.LOG_LEVEL is not None:
return self.LOG_LEVEL return self.LOG_LEVEL
if self.ENVIRONMENT == Environment.DEVELOPMENT: if self.ENVIRONMENT == Environment.DEVELOPMENT:
return "DEBUG" return "DEBUG"
if self.ENVIRONMENT == Environment.PRODUCTION:
return "WARNING"
return "INFO" return "INFO"
@property @property
+77 -8
View File
@@ -4,6 +4,17 @@ PydanticAI provider for Ollama with message sanitization.
Ollama's OpenAI-compatible API rejects messages with `content: null`, Ollama's OpenAI-compatible API rejects messages with `content: null`,
which PydanticAI sends for assistant messages that only contain tool calls. which PydanticAI sends for assistant messages that only contain tool calls.
This provider sanitizes messages to use empty strings instead of null. This provider sanitizes messages to use empty strings instead of null.
History worth keeping (repo T-6): until 2026-09-13 the sanitized client
was never in the request path — the provider assigned
`self._openai_client`, an attribute nobody reads, while
`OllamaProvider.client` serves `self._client`. The sanitizer, the
wrapper-signal reading and the OLLAMA_TIMEOUT cap were all silently
dead (sessions were unaffected — they ride ModelSettings). webber hit
the identical bug the same day; both now wire the client through the
constructor's official `openai_client` parameter, and the signals are
read from the response body — an httpx event-hook variant demonstrably
never fires under the openai SDK.
""" """
from typing import Any from typing import Any
@@ -17,6 +28,31 @@ from src.core.logging_config import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
def _log_wrapper_signals(response: Any) -> None:
"""
Surface the boilerroom wrapper's per-response signals (repo T-4).
balancing names the sessions parked or restored to serve this
request (wrapper D-4 — never silent); compaction-due says a session
crossed its authored context budget (wrapper D-5 — a signal, not an
interrupt). Read from the parsed body's extra fields: the wrapper
injects them into non-streamed JSON answers and openai's pydantic
models retain unknown fields in `model_extra`. Absent on any other
backend, so this costs nothing there. Streams carry the signals in
headers only and go unlogged here; the steward's raw call reads its
own body.
"""
extra = getattr(response, "model_extra", None) or {}
balancing = extra.get("balancing")
if balancing:
logger.info("backend_balancing", moves=balancing)
if extra.get("compaction_due"):
logger.warning(
"backend_compaction_due",
hint="GET /sessions on the wrapper names the session over budget",
)
class TatlockOllamaProvider(OllamaProvider): class TatlockOllamaProvider(OllamaProvider):
""" """
Custom OllamaProvider with message sanitization for Tatlock agents. Custom OllamaProvider with message sanitization for Tatlock agents.
@@ -36,10 +72,12 @@ class TatlockOllamaProvider(OllamaProvider):
clean_host = str(config.OLLAMA_HOST).rstrip("/") clean_host = str(config.OLLAMA_HOST).rstrip("/")
base_url = f"{clean_host}/v1" base_url = f"{clean_host}/v1"
super().__init__(base_url=base_url) # The sanitized client goes through the official constructor
# parameter: the provider's `.client` property serves `_client`,
# Override the client with our sanitized version # and poking a lookalike attribute after super().__init__ had
self._openai_client = _SanitizedAsyncOpenAI(base_url=base_url) # built its own client is how the sanitizer sat bypassed in
# production (T-6).
super().__init__(openai_client=_SanitizedAsyncOpenAI(base_url=base_url))
logger.debug( logger.debug(
"tatlock_ollama_provider_created", "tatlock_ollama_provider_created",
@@ -59,17 +97,46 @@ class _SanitizedAsyncOpenAI(AsyncOpenAI):
super().__init__(api_key="ollama", **kwargs) super().__init__(api_key="ollama", **kwargs)
@property @property
def chat(self) -> "_SanitizedChat": def chat(self) -> "_SanitizedChat": # type: ignore[override]
"""Return sanitized chat interface.""" """Return sanitized chat interface.
Deliberately incompatible with AsyncOpenAI.chat, which is a Chat
resource. Replacing it is the entire mechanism of this class;
typing it as the parent's Chat would describe an object this
class does not return. The suppression is on this member alone.
"""
return _SanitizedChat(self) return _SanitizedChat(self)
def _parent_chat(client: AsyncOpenAI) -> Any:
"""Get AsyncOpenAI's own `chat`, bypassing the subclass override.
openai has shipped `chat` as both a property (getter `.fget`) and a
functools.cached_property (getter `.func`) — this venv's 2.11.0 is
the latter, so the old hardcoded `.fget` access raised
AttributeError the moment anything touched `.chat`. It went
unnoticed here for the same reason the whole class did: the client
was never wired (T-6). Reading whichever getter the descriptor
exposes keeps this working across both shapes, and raises something
legible if openai adopts a third. Ported from webber's 2026-08-11
fix.
"""
descriptor = AsyncOpenAI.__dict__["chat"]
getter = getattr(descriptor, "func", None) or getattr(descriptor, "fget", None)
if getter is None: # pragma: no cover - defensive
raise TypeError(
f"AsyncOpenAI.chat is a {type(descriptor).__name__} with neither "
"'func' nor 'fget'; the sanitising wrapper needs updating"
)
return getter(client)
class _SanitizedChat: class _SanitizedChat:
"""Chat interface wrapper with sanitized completions.""" """Chat interface wrapper with sanitized completions."""
def __init__(self, client: _SanitizedAsyncOpenAI): def __init__(self, client: _SanitizedAsyncOpenAI):
self._client = client self._client = client
self._original_chat = AsyncOpenAI.chat.fget(client) # type: ignore self._original_chat = _parent_chat(client)
@property @property
def completions(self) -> "_SanitizedCompletions": def completions(self) -> "_SanitizedCompletions":
@@ -93,7 +160,9 @@ class _SanitizedCompletions:
if "messages" in kwargs: if "messages" in kwargs:
kwargs["messages"] = _sanitize_messages(kwargs["messages"]) kwargs["messages"] = _sanitize_messages(kwargs["messages"])
return await self._original.create(**kwargs) response = await self._original.create(**kwargs)
_log_wrapper_signals(response)
return response
def _sanitize_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]: def _sanitize_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
+115 -1
View File
@@ -67,7 +67,10 @@ class TestGetModel:
class TestToolChoiceSettings: class TestToolChoiceSettings:
def test_ollama_forces_tool_choice(self, local_first): def test_ollama_forces_tool_choice(self, local_first, monkeypatch):
# Pin the flavor: the session-scoped app init probes the real
# dev backend, and this test's answer must not depend on it.
monkeypatch.setattr(model_selector, "_local_flavor", "ollama")
settings = model_selector.get_tool_choice_settings() settings = model_selector.get_tool_choice_settings()
assert settings.get("extra_body") == {"tool_choice": "required"} assert settings.get("extra_body") == {"tool_choice": "required"}
@@ -91,3 +94,114 @@ class TestGetModelInfo:
info = model_selector.get_model_info() info = model_selector.get_model_info()
assert info["backend"] == "claude" assert info["backend"] == "claude"
assert info["model"] == config.ANTHROPIC_MODEL assert info["model"] == config.ANTHROPIC_MODEL
class TestPhaseBinding:
def test_disabled_returns_settings_unchanged(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", False)
base = model_selector.get_tool_choice_settings()
assert model_selector.with_phase_binding(base, "orchestrate") is base
assert model_selector.with_phase_binding(None, "synthesize") is None
def test_wrapper_flavor_sends_session_and_rank(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "boilerroom")
settings = model_selector.with_phase_binding(
model_selector.get_tool_choice_settings(), "orchestrate"
)
extra_body = settings["extra_body"]
assert extra_body["session"] == "tatlock-orchestrate"
assert extra_body["eviction_order"] == 40
assert "id_slot" not in extra_body
def test_wrapper_ranking_matches_the_decision(self, local_first, monkeypatch):
# Decided 2026-09-12: pipeline over librarian over webber (20,
# filed in its own repo); lower eviction_order parks sooner.
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "boilerroom")
ranks = {
phase: model_selector.phase_extra_body(phase)["eviction_order"]
for phase in ("steward", "orchestrate", "synthesize", "experts", "librarian")
}
assert ranks["steward"] == ranks["orchestrate"] == ranks["synthesize"] == 40
assert ranks["experts"] == 35
assert ranks["librarian"] == 30
assert ranks["librarian"] < ranks["experts"] < ranks["steward"]
assert ranks["librarian"] > 20 # webber's rank stays below
def test_wrapper_session_names_are_stable(self, local_first, monkeypatch):
# The wrapper's map keys on these; renaming one orphans its slot.
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "boilerroom")
names = {
phase: model_selector.phase_extra_body(phase)["session"]
for phase in ("steward", "orchestrate", "synthesize", "experts", "librarian")
}
assert names == {
"steward": "tatlock-steward",
"orchestrate": "tatlock-orchestrate",
"synthesize": "tatlock-synthesize",
"experts": "tatlock-experts",
"librarian": "librarian",
}
def test_direct_llama_server_keeps_id_slot(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "llama-server")
settings = model_selector.with_phase_binding(None, "synthesize")
assert settings["extra_body"] == {"id_slot": 2}
def test_librarian_floats_on_direct_llama_server(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "llama-server")
assert model_selector.phase_extra_body("librarian") == {}
def test_ollama_flavor_gets_no_fields(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "ollama")
assert model_selector.phase_extra_body("steward") == {}
def test_unprobed_flavor_sends_nothing_rather_than_guessing(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", None)
assert model_selector.phase_extra_body("orchestrate") == {}
def test_merge_preserves_existing_extra_body(self, local_first, monkeypatch):
from pydantic_ai.settings import ModelSettings
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(model_selector, "_local_flavor", "boilerroom")
base = ModelSettings(extra_body={"marker": 1})
merged = model_selector.with_phase_binding(base, "steward")
assert merged["extra_body"]["marker"] == 1
assert merged["extra_body"]["session"] == "tatlock-steward"
def test_claude_backend_never_bound(self, local_first, monkeypatch):
monkeypatch.setattr(config, "BACKEND_SLOT_PINNING", True)
monkeypatch.setattr(config, "PREFER_CLOUD_BACKEND", True)
base = model_selector.get_tool_choice_settings()
assert model_selector.with_phase_binding(base, "orchestrate") is base
class TestLocalFlavorToolChoice:
def test_llama_server_flavor_sends_no_tool_choice(self, local_first, monkeypatch):
monkeypatch.setattr(model_selector, "_local_flavor", "llama-server")
settings = model_selector.get_tool_choice_settings()
assert not settings.get("extra_body")
def test_boilerroom_flavor_sends_no_tool_choice(self, local_first, monkeypatch):
# The wrapper forwards to llama-server, which enforces
# tool_choice — the nudge through it is the tool-loop incident.
monkeypatch.setattr(model_selector, "_local_flavor", "boilerroom")
settings = model_selector.get_tool_choice_settings()
assert not settings.get("extra_body")
def test_ollama_flavor_keeps_advisory_required(self, local_first, monkeypatch):
monkeypatch.setattr(model_selector, "_local_flavor", "ollama")
settings = model_selector.get_tool_choice_settings()
assert settings["extra_body"] == {"tool_choice": "required"}
def test_unknown_flavor_defaults_to_ollama_semantics(self, local_first, monkeypatch):
monkeypatch.setattr(model_selector, "_local_flavor", None)
settings = model_selector.get_tool_choice_settings()
assert settings["extra_body"] == {"tool_choice": "required"}
+31 -3
View File
@@ -49,16 +49,44 @@ def _tenant_guard():
@pytest.fixture(scope="session", autouse=True) @pytest.fixture(scope="session", autouse=True)
def _initialize_app(_tenant_guard): def _initialize_app(_tenant_guard):
""" """
Run application lifespan (Claude health check, household registration, etc.) Run application lifespan (household registration, backend globals)
once per test session. ASGITransport doesn't trigger lifespan events, once per test session. ASGITransport doesn't trigger lifespan events,
so we call it explicitly. so we call it explicitly.
The backend health probes are stubbed (T-7): `make test` promises
"no external services", but the real probes coupled every unit run
to live infrastructure — an HTTP round trip to whatever answers
behind OLLAMA_HOST, and a real Anthropic API call whenever the dev
.env carries a key. The suite flaked whenever the local backend was
mid-deploy: the ollama probe failed, the cached globals flipped,
and any test that consults them unpatched changed behavior. The
stubs land the globals in the deterministic local-first state a
healthy startup produces; tests needing other states patch the
globals themselves, and `make test-integration` runs real probes.
Depends on _tenant_guard so the suite refuses to start under the Depends on _tenant_guard so the suite refuses to start under the
production tenant before any initialization happens. production tenant before any initialization happens.
""" """
from src.core.startup import initialize_application from unittest.mock import patch
asyncio.run(initialize_application()) from src.anthropic import model_selector
from src.core import startup
async def _healthy_local_backend() -> bool:
model_selector._ollama_available = True
model_selector._local_flavor = "boilerroom"
return True
async def _no_cloud_fallback() -> bool:
model_selector._claude_available = False
return False
# Patch the names startup actually calls (imported into its module).
with (
patch.object(startup, "check_ollama_health", _healthy_local_backend),
patch.object(startup, "check_claude_health", _no_cloud_fallback),
):
asyncio.run(startup.initialize_application())
@pytest.fixture @pytest.fixture
+65 -4
View File
@@ -114,16 +114,18 @@ class TestLocalBackendContract:
) )
async def test_openai_compat_tool_calling(self): async def test_openai_compat_tool_calling(self):
# Mirrors the request PydanticAI's OpenAIChatModel sends for the # Mirrors the orchestration-phase request on llama-server: tools
# orchestration phase, including the extra_body tool_choice. # attached, NO tool_choice. The backend must call the tool unforced
# — "required" is deliberately absent because llama-server enforces
# it on every request in a run, which turns the tool loop
# unbreakable (observed at cutover: ~80 s turns).
response = await _post_or_skip( response = await _post_or_skip(
f"{OLLAMA}/v1/chat/completions", f"{OLLAMA}/v1/chat/completions",
"ollama", "local-backend",
{ {
"model": config.OLLAMA_DEFAULT_MODEL, "model": config.OLLAMA_DEFAULT_MODEL,
"messages": [{"role": "user", "content": "What is 6 * 7? Use the calculator."}], "messages": [{"role": "user", "content": "What is 6 * 7? Use the calculator."}],
"tools": [CALCULATOR_TOOL], "tools": [CALCULATOR_TOOL],
"tool_choice": "required",
"stream": False, "stream": False,
}, },
timeout=config.OLLAMA_TIMEOUT, timeout=config.OLLAMA_TIMEOUT,
@@ -252,3 +254,62 @@ class TestRedisContract:
await client.delete("contract-test-key") await client.delete("contract-test-key")
finally: finally:
await client.aclose() await client.aclose()
@pytest.mark.contract
class TestBoilerroomWrapperContract:
"""Boundary: the boilerroom wrapper surface session adoption rides on
(repo T-4). Skips when OLLAMA_HOST is not the wrapper — a direct
engine or Ollama is a valid deployment, not a violation.
"""
async def _skip_unless_wrapper(self) -> None:
response = await _get_or_skip(f"{OLLAMA}/health", "local-backend")
try:
service = response.json().get("service")
except ValueError:
service = None
if service != "boilerroom":
pytest.skip("OLLAMA_HOST is not the boilerroom wrapper")
async def test_wrapper_names_itself_on_health(self):
# Mirrors check_ollama_health()'s wrapper probe: the flavor
# detection keys on this exact field.
await self._skip_unless_wrapper()
async def test_props_passes_through_for_flavor_detection(self):
# tatlock treats a non-200 /props as Ollama and applies the
# tool_choice nudge — enforced by llama-server into a tool
# loop. The wrapper passing /props through is what keeps that
# detection safe (boilerroom T-10).
await self._skip_unless_wrapper()
response = await _get_or_skip(f"{OLLAMA}/props", "local-backend")
assert response.status_code == 200
async def test_session_fields_answer_with_signals(self):
# Mirrors phase_extra_body() through the wrapper: a session-
# carrying request must come back with the balancing header and
# the compaction signal in the body (wrapper D-3/D-4/D-5).
await self._skip_unless_wrapper()
response = await _post_or_skip(
f"{OLLAMA}/v1/chat/completions",
"local-backend",
{
"model": config.OLLAMA_DEFAULT_MODEL,
# No max_tokens: gemma4 spends its first tokens thinking,
# and a tight cap returns empty content (not a wrapper
# fault — the sibling passthrough test runs uncapped too).
"messages": [{"role": "user", "content": "Reply with the single word: pong"}],
"session": "contract-probe",
"eviction_order": 1,
},
timeout=config.OLLAMA_TIMEOUT,
)
assert response.status_code == 200
assert "x-boilerroom-balancing" in response.headers
assert response.headers.get("x-boilerroom-compaction-due") in ("true", "false")
body = response.json()
assert isinstance(body.get("compaction_due"), bool)
assert isinstance(body.get("balancing"), list)
# The extension fields must not leak into the engine's answer
assert body["choices"][0]["message"]["content"].strip()
+35
View File
@@ -241,6 +241,41 @@ class TestLoggingConfiguration:
with patch.object(config, "ENVIRONMENT", Environment.PRODUCTION): with patch.object(config, "ENVIRONMENT", Environment.PRODUCTION):
assert config.log_format == "json" assert config.log_format == "json"
def test_production_default_level_is_info(self):
"""Production logs INFO to stdout (T-5).
The old WARNING default made `docker logs` empty for months — a
healthy service warns roughly never, so startup, flavor
detection and the wrapper signals were all suppressed and the
serving cutover could not be verified from logs.
"""
from src.core.config import Environment, config
with (
patch.object(config, "ENVIRONMENT", Environment.PRODUCTION),
patch.object(config, "LOG_LEVEL", None),
):
assert config.effective_log_level == "INFO"
def test_log_level_env_still_quiets_production(self):
"""LOG_LEVEL stays the override for when noise is the problem."""
from src.core.config import Environment, config
with (
patch.object(config, "ENVIRONMENT", Environment.PRODUCTION),
patch.object(config, "LOG_LEVEL", "WARNING"),
):
assert config.effective_log_level == "WARNING"
def test_development_default_stays_debug(self):
from src.core.config import Environment, config
with (
patch.object(config, "ENVIRONMENT", Environment.DEVELOPMENT),
patch.object(config, "LOG_LEVEL", None),
):
assert config.effective_log_level == "DEBUG"
def test_multiple_loggers_independent(self): def test_multiple_loggers_independent(self):
"""Test multiple loggers are independent.""" """Test multiple loggers are independent."""
logger1 = get_logger("test.logger1") logger1 = get_logger("test.logger1")
+19
View File
@@ -0,0 +1,19 @@
"""
The suite's backend globals are stubbed, not probed (T-7).
`make test` promises "no external services". The session init used to
run the real health probes — an HTTP round trip to whatever answers
behind OLLAMA_HOST and a real Anthropic API call whenever the dev .env
carries a key — so the cached globals depended on live infrastructure,
and the suite flaked whenever the backend was mid-deploy. This test
discriminates: under the stub the globals are always the local-first
healthy state; under real probes they follow the network of the moment.
"""
from src.anthropic import model_selector
def test_session_globals_are_deterministic():
assert model_selector._ollama_available is True
assert model_selector._local_flavor == "boilerroom"
assert model_selector._claude_available is False
+115 -8
View File
@@ -1,31 +1,73 @@
""" """
Tests for TatlockOllamaProvider configuration. Tests for TatlockOllamaProvider configuration and wiring.
The AsyncOpenAI client must carry an explicit timeout from Every assertion here goes through `provider.client` — the property
config.OLLAMA_TIMEOUT instead of the SDK default (~600s), so a stuck pydantic_ai actually reads. The previous version asserted on
LLM call cannot consume the whole delegation budget. `_openai_client`, a lookalike attribute nobody read, and stayed green
for the whole period the sanitizer was bypassed in production (T-6):
a test whose subject is not the real thing cannot fail for the real
reason.
""" """
import pytest import pytest
from src.core.config import config from src.core.config import config
from src.ollama.provider import TatlockOllamaProvider, _sanitize_messages from src.ollama import provider as provider_module
from src.ollama.provider import (
TatlockOllamaProvider,
_parent_chat,
_sanitize_messages,
_SanitizedAsyncOpenAI,
_SanitizedCompletions,
)
@pytest.mark.unit
class TestClientWiring:
"""The sanitized client must be the one the provider serves (T-6)."""
def test_provider_serves_the_sanitized_client(self):
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
assert isinstance(provider.client, _SanitizedAsyncOpenAI)
def test_the_full_chain_reaches_completions(self):
# Walks provider.client -> sanitized chat -> parent lookup ->
# completions. Under openai's cached_property `chat`, the old
# hardcoded `.fget` access raised AttributeError right here.
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
assert provider.client.chat.completions is not None
def test_parent_lookup_survives_either_descriptor_shape(self):
from openai import AsyncOpenAI
descriptor = AsyncOpenAI.__dict__["chat"]
assert hasattr(descriptor, "func") or hasattr(
descriptor, "fget"
), "AsyncOpenAI.chat exposes neither getter; _parent_chat needs updating"
client = _SanitizedAsyncOpenAI(base_url="http://localhost:11434/v1")
parent = _parent_chat(client)
assert parent is not None
assert (
type(parent).__name__ != "_SanitizedChat"
), "parent lookup must not recurse into the override"
@pytest.mark.unit @pytest.mark.unit
class TestProviderTimeout: class TestProviderTimeout:
"""Timeout configuration on the underlying AsyncOpenAI client.""" """Timeout configuration on the client the provider actually serves."""
def test_openai_client_timeout_from_config(self): def test_openai_client_timeout_from_config(self):
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1") provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
assert provider._openai_client.timeout == float(config.OLLAMA_TIMEOUT) assert provider.client.timeout == float(config.OLLAMA_TIMEOUT)
def test_timeout_is_not_sdk_default(self): def test_timeout_is_not_sdk_default(self):
provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1") provider = TatlockOllamaProvider(base_url="http://localhost:11434/v1")
# The OpenAI SDK defaults to 600s; the configured cap must win # The OpenAI SDK defaults to 600s; the configured cap must win
assert provider._openai_client.timeout < 600 assert provider.client.timeout < 600
@pytest.mark.unit @pytest.mark.unit
@@ -52,3 +94,68 @@ class TestMessageSanitization:
] ]
assert _sanitize_messages(messages) == messages assert _sanitize_messages(messages) == messages
class _StubLogger:
def __init__(self):
self.calls = []
def info(self, event, **kwargs):
self.calls.append(("info", event, kwargs))
def warning(self, event, **kwargs):
self.calls.append(("warning", event, kwargs))
class _BusyResponse:
def __init__(self):
self.model_extra = {
"balancing": [{"parked": "librarian"}],
"compaction_due": True,
}
class _QuietResponse:
def __init__(self):
self.model_extra = {"balancing": [], "compaction_due": False}
@pytest.mark.unit
class TestWrapperSignals:
"""The wrapper's body signals must not pass silently (T-4/T-6).
Read from the parsed body's model_extra: an httpx event-hook
variant demonstrably never fires under the openai SDK.
"""
def test_busy_response_logs_balancing_and_compaction(self, monkeypatch):
stub = _StubLogger()
monkeypatch.setattr(provider_module, "logger", stub)
provider_module._log_wrapper_signals(_BusyResponse())
assert any(kind == "info" and event == "backend_balancing" for kind, event, _ in stub.calls)
assert any(kind == "warning" for kind, _, _ in stub.calls)
def test_quiet_response_logs_nothing(self, monkeypatch):
stub = _StubLogger()
monkeypatch.setattr(provider_module, "logger", stub)
provider_module._log_wrapper_signals(_QuietResponse())
assert stub.calls == []
async def test_create_surfaces_the_signals(self, monkeypatch):
# The read must sit in the request path, not merely exist:
# removing the call from create() has to fail this test.
stub = _StubLogger()
monkeypatch.setattr(provider_module, "logger", stub)
class FakeOriginal:
async def create(self, **kwargs):
return _BusyResponse()
completions = _SanitizedCompletions(FakeOriginal())
await completions.create(messages=[{"role": "user", "content": "hi"}])
assert any(event == "backend_balancing" for _, event, _ in stub.calls)