From b1cb3eb8995b4732949313d38df047a12a305cb6 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Tue, 18 Aug 2026 20:44:24 +0200 Subject: [PATCH] build(make): deselect the integration tests, and give them a target (T-55) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Twelve tests in test_database_integration.py carry @pytest.mark.integration and need a real Postgres. The marker was registered in pytest.ini and used correctly on the file — but `make test` never deselected it and no target ever selected it. So those tests neither ran nor passed: they errored on every invocation, and `make test` exited 2 permanently, which teaches a reader that the exit code means nothing. That is worse than either running them or not having them. `make test` is now hermetic per D-26 — 164 passed, 19 deselected, exit 0, verified inside an unprivileged network namespace as well as outside. WHY THE AUDIT MISSED THIS, which is the part worth keeping. T-55 measured each suite with and without a network and treated identical results as proof of no live dependency. These twelve fail identically both ways, because postgres-shared is a Docker-internal name that a host process cannot resolve in either condition. A namespace proves a test does not reach the network; it cannot distinguish that from a test whose dependency is unreachable regardless. The positive control was run against a host that WAS reachable, so it never covered this case. The new target refuses an empty selection: pytest exit 5 fails with its own message and a collection error gets a different one, so "nothing to run" can never be read as "everything passed". Mutation-checked — anchor asserted unique, marker renamed, target failed at exit 2 with the intended message, Makefile restored byte-identical. It reports honestly when Postgres is absent rather than skipping: 6 passed, 1 skipped, 12 errors from this host, which is the true state. Co-Authored-By: Claude --- Makefile | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 468bf99..5b91dfd 100644 --- a/Makefile +++ b/Makefile @@ -31,9 +31,38 @@ setup: ## Create the venv, install the test extra, and prove it actually works $(VENV)/bin/python -m pytest tests/ --collect-only -q .PHONY: test -test: ## Run the test suite +test: ## Run the test suite — hermetic, no live services (D-26) @test -x $(VENV)/bin/python || { echo "FAIL — no venv in this tree; run: make setup"; exit 69; } - $(VENV)/bin/python -m pytest tests/ + # Integration tests are deselected here, not skipped by accident. Twelve + # tests in test_database_integration.py carry @pytest.mark.integration and + # need a real Postgres; they errored on every run of this target because + # nothing deselected them, and the marker had no target to select it either. + # So they neither passed nor ran — they just made `make test` exit 2 forever, + # which trains a reader to ignore the exit code (T-55). + # + # They were invisible to the netns audit that found the rest of this: they + # fail identically with and without a network, because postgres-shared is a + # Docker-internal name a host process cannot resolve in either case. A + # namespace proves a test does not reach the network; it cannot tell that + # apart from a test whose dependency is unreachable anyway. + $(VENV)/bin/python -m pytest tests/ -m "not integration" + +.PHONY: test-integration +test-integration: ## Run only the tests that need live Postgres/Redis + @test -x $(VENV)/bin/python || { echo "FAIL — no venv in this tree; run: make setup"; exit 69; } + # Refuses an empty selection. A target that passes because it selected + # nothing is the defect this repo keeps meeting from the other side, so + # pytest's exit 5 (no tests collected) is a failure with its own message, + # and a collection error gets a different one — "nothing to run" must never + # read as "everything passed" (D-24). + @$(VENV)/bin/python -m pytest tests/ -m integration --collect-only -q >/dev/null 2>&1; \ + rc=$$?; \ + if [ $$rc -eq 5 ]; then \ + echo "FAIL test-integration — selected 0 tests (marker renamed, moved, or lost — this is a defect, not a pass)"; exit 1; \ + elif [ $$rc -ne 0 ]; then \ + echo "FAIL test-integration — collection errored (rc=$$rc)"; exit 1; \ + fi + $(VENV)/bin/python -m pytest tests/ -m integration -v # No `lint` target, deliberately. CLAUDE.md states it outright: no linter is # configured, no ruff or flake8 config, neither in the dependencies. Per D-27