fix(tests): T-7 — the unit suite is hermetic, the flake class is gone
Three flakes in one day, never a captured name — the name finally came from a discriminating run instead of patience: pointing OLLAMA_HOST at a dead port failed exactly eight tests, all in tests/agents/test_tatlock_agent.py, all already marked integration, all running in the unit gate anyway because make test excluded by directory and they live outside the ignored directories. Eight tests doing ~110 seconds of real LLM inference against the production wrapper on every unit run, failing whenever the backend was mid-deploy — which this week it constantly was. The second coupling sat in the session init: conftest ran the real health probes, an HTTP round trip to whatever answers behind OLLAMA_HOST plus a real Anthropic API call whenever the dev .env carries a key, so the cached backend globals followed the network of the moment. The probes are stubbed to the deterministic local-first state; tests needing other states patch the globals themselves, as the selector tests always did. make test now enforces -m "not integration" alongside the directory ignores, and the acceptance is blunt: 679 passed in ~12 s, identical against a dead backend and no API key — down from ~129 s of infrastructure-coupled runtime. Both prongs mutation-checked: the gate removed fails eight against a dead backend; the stub removed fails the new session-globals test. T-8 files the orphaned tests/integration directory that no make target runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+31
-3
@@ -49,16 +49,44 @@ def _tenant_guard():
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def _initialize_app(_tenant_guard):
|
||||
"""
|
||||
Run application lifespan (Claude health check, household registration, etc.)
|
||||
Run application lifespan (household registration, backend globals)
|
||||
once per test session. ASGITransport doesn't trigger lifespan events,
|
||||
so we call it explicitly.
|
||||
|
||||
The backend health probes are stubbed (T-7): `make test` promises
|
||||
"no external services", but the real probes coupled every unit run
|
||||
to live infrastructure — an HTTP round trip to whatever answers
|
||||
behind OLLAMA_HOST, and a real Anthropic API call whenever the dev
|
||||
.env carries a key. The suite flaked whenever the local backend was
|
||||
mid-deploy: the ollama probe failed, the cached globals flipped,
|
||||
and any test that consults them unpatched changed behavior. The
|
||||
stubs land the globals in the deterministic local-first state a
|
||||
healthy startup produces; tests needing other states patch the
|
||||
globals themselves, and `make test-integration` runs real probes.
|
||||
|
||||
Depends on _tenant_guard so the suite refuses to start under the
|
||||
production tenant before any initialization happens.
|
||||
"""
|
||||
from src.core.startup import initialize_application
|
||||
from unittest.mock import patch
|
||||
|
||||
asyncio.run(initialize_application())
|
||||
from src.anthropic import model_selector
|
||||
from src.core import startup
|
||||
|
||||
async def _healthy_local_backend() -> bool:
|
||||
model_selector._ollama_available = True
|
||||
model_selector._local_flavor = "boilerroom"
|
||||
return True
|
||||
|
||||
async def _no_cloud_fallback() -> bool:
|
||||
model_selector._claude_available = False
|
||||
return False
|
||||
|
||||
# Patch the names startup actually calls (imported into its module).
|
||||
with (
|
||||
patch.object(startup, "check_ollama_health", _healthy_local_backend),
|
||||
patch.object(startup, "check_claude_health", _no_cloud_fallback),
|
||||
):
|
||||
asyncio.run(startup.initialize_application())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
"""
|
||||
The suite's backend globals are stubbed, not probed (T-7).
|
||||
|
||||
`make test` promises "no external services". The session init used to
|
||||
run the real health probes — an HTTP round trip to whatever answers
|
||||
behind OLLAMA_HOST and a real Anthropic API call whenever the dev .env
|
||||
carries a key — so the cached globals depended on live infrastructure,
|
||||
and the suite flaked whenever the backend was mid-deploy. This test
|
||||
discriminates: under the stub the globals are always the local-first
|
||||
healthy state; under real probes they follow the network of the moment.
|
||||
"""
|
||||
|
||||
from src.anthropic import model_selector
|
||||
|
||||
|
||||
def test_session_globals_are_deterministic():
|
||||
assert model_selector._ollama_available is True
|
||||
assert model_selector._local_flavor == "boilerroom"
|
||||
assert model_selector._claude_available is False
|
||||
Reference in New Issue
Block a user