Files
odysseus/tests/helpers/import_state.py
T
Léo fba6f73260 test: fail the test that leaks a bare src/core module stub
#25 fixed two sys.modules writes in test_auth_regressions.py that left empty
stub modules behind for the rest of the session, breaking 23 tests under one
collection order while the full suite stayed green. The class is wider than
that file, and an audit is the wrong answer to it: nothing stops the next one,
and the failure it causes lands on an unrelated test in a different file.

So this is a guard instead. An autouse fixture in the root conftest snapshots
which src.* / core.* names are bound to a bare ModuleType, and fails any test
that adds one. "Bare" is the same test the clear_fake_* helpers already use -
a plain types.ModuleType with no on-disk __file__. MagicMock stand-ins are out
of scope: they answer every attribute, so they fail at the point of use rather
than silently, and several files install them deliberately.

Three details that matter:

- It lives in the root conftest, so it is set up before any test-module
  fixture and torn down after all of them. A stub a test's own teardown
  removes is not reported.
- It drops the leaked entries as well as reporting them, so the failure stays
  on the test that introduced it instead of cascading through the rest of the
  run.
- It only reports stubs added during the test. Import state the session starts
  with, including this conftest's own src.database stub, is left alone.

It found one beyond #25 on the first full run: _stub_heavy in
test_scheduler_restart_doublefire.py leaks the same five src.* modules as the
test #25 fixed, via sys.modules.setdefault. It already receives monkeypatch,
so the fix is to register through it. Fixed here because the guard has to land
green.

Full suite, macOS, default collection order:

  this branch   10655 passed, 6 failed, 6 skipped   406s
  lab           10655 passed, 6 failed, 6 skipped   371s

Same six either way, which is the point - none of this is visible in the
default order. Four are pre-existing macOS environment failures:
test_glob_confined_e2e and the two test_code_nav_tools document cases resolve
/tmp to /private/tmp, and
test_real_socket_falls_back_from_dead_first_to_live_second is connect-refused
timing on real sockets. The other two are the rich-colour and ffmpeg items
from the same ledger, fixed on their own branches.

Not verified: Linux, and any collection order other than the default. The
guard is order-independent by construction - it compares before and after
within a single test - but I have only run the default order.
2026-09-30 13:02:34 +02:00

201 lines
8.5 KiB
Python

"""Shared helper for saving and restoring Python import state in tests.
Use ``preserve_import_state`` as a context manager around any block that needs
to mutate ``sys.modules`` or parent-package attributes temporarily. On exit
(normal or exception), every named module is restored to exactly the state it
had before the block — present, absent, or carrying a parent-package attribute.
Use ``clear_module`` to drop a single module from both ``sys.modules`` and its
parent-package attribute (e.g. before forcing a fresh import inside the block).
Use ``clear_fake_database_modules`` to evict a *stubbed* ``core.database`` (and
its companion ``src.database``) that another test left in import state, without
touching a real ``core.database`` loaded from disk.
Use ``clear_fake_endpoint_resolver_modules`` to evict a *stubbed*
``src.endpoint_resolver`` (and the route modules that imported it) that another
test left in import state, without touching a real ``src.endpoint_resolver``
loaded from disk.
Background: importing ``routes.session_routes`` also sets ``session_routes`` on
the parent ``routes`` package object. A ``from routes import session_routes``
or ``import routes.session_routes as X`` statement resolves through that parent
attribute, so restoring ``sys.modules`` alone is not sufficient — the parent
attribute must be restored too. This helper handles both.
Restoration in ``preserve_import_state`` is two-phased: all ``sys.modules``
entries are written back first, then all parent-package attributes. This means
parent-attr restoration always resolves the parent through the already-restored
``sys.modules``, so results are deterministic regardless of argument order —
safe for callers that pass both a parent package and a child module.
"""
import sys
import types
from contextlib import contextmanager
_ABSENT = object()
def _save_one(dotted_name):
saved_mod = sys.modules.get(dotted_name, _ABSENT)
pkg_name, _, attr = dotted_name.rpartition(".")
pkg = sys.modules.get(pkg_name)
saved_attr = getattr(pkg, attr, _ABSENT) if pkg is not None else _ABSENT
return saved_mod, saved_attr
def _restore_parent_attr(dotted_name, saved_attr):
pkg_name, _, attr = dotted_name.rpartition(".")
pkg = sys.modules.get(pkg_name)
if pkg is None:
return
if saved_attr is _ABSENT:
if hasattr(pkg, attr):
delattr(pkg, attr)
else:
setattr(pkg, attr, saved_attr)
def _restore_one(dotted_name, saved_mod, saved_attr):
if saved_mod is _ABSENT:
sys.modules.pop(dotted_name, None)
else:
sys.modules[dotted_name] = saved_mod
_restore_parent_attr(dotted_name, saved_attr)
def clear_module(dotted_name):
"""Remove a module from sys.modules and its parent-package attribute."""
_restore_one(dotted_name, _ABSENT, _ABSENT)
def clear_fake_database_modules():
"""Evict a *stubbed* ``core.database`` (and ``src.database``) from import state.
Test-only. Some tests install a fake ``core.database`` — a stub module with
no on-disk ``__file__`` — into ``sys.modules`` and onto the ``core`` package.
A later test that needs the real database module must evict that stub first,
or its ``import core.database`` resolves to the fake.
This is deliberately conservative and mirrors the per-file helpers it
replaces:
* It acts only when ``core.database`` is a fake/stub, detected by a missing
string ``__file__``. A real ``core.database`` loaded from disk is left
untouched, as is the case where nothing is cached.
* When it does act, it also drops the cached ``src.database`` entry.
* It removes the ``core.database`` parent-package attribute only when that
attribute is the same fake object being evicted.
"""
parent = sys.modules.get("core")
attr = getattr(parent, "database", None) if parent is not None else None
mod = sys.modules.get("core.database") or attr
if mod is None or isinstance(getattr(mod, "__file__", None), str):
return
sys.modules.pop("core.database", None)
sys.modules.pop("src.database", None)
if parent is not None and attr is mod:
delattr(parent, "database")
def clear_fake_endpoint_resolver_modules(*extra_modules):
"""Evict a *stubbed* ``src.endpoint_resolver`` (and dependent route modules).
Test-only. Several route tests need the *real* ``src.endpoint_resolver`` URL
helpers, but another test may have installed a fake — a stub module with no
on-disk ``__file__`` — into ``sys.modules`` and onto the ``src`` package
during collection. The route modules (``routes.model_routes`` and any extras
passed in, e.g. ``routes.chat_routes``) get cached against that fake on first
import, so they must be evicted too.
Conservative, mirroring ``clear_fake_database_modules`` and the per-file
guards it replaces:
* It acts only when ``src.endpoint_resolver`` is a fake/stub, detected by a
falsy ``__file__`` (missing, ``None``, or empty string) — exactly the
truthiness check the old inline guards used. A real resolver loaded from
disk carries a truthy ``__file__`` and is left untouched, as is the case
where nothing is cached. When the resolver is real, the dependent route
modules are left untouched too.
* When it does act, it drops ``routes.model_routes`` plus every name in
``extra_modules``.
* It removes the ``src.endpoint_resolver`` parent-package attribute only when
that attribute is the same fake object being evicted.
Behavior delta vs. the old bare ``sys.modules.pop(...)`` guards: dependent
modules are dropped via :func:`clear_module`, which also clears the parent
``routes`` package attribute (e.g. ``routes.model_routes``), not just the
``sys.modules`` entry. This prevents a stale parent attribute from shadowing
the fresh import — the same parent-attr handling the rest of this helper
family already applies.
"""
parent = sys.modules.get("src")
attr = getattr(parent, "endpoint_resolver", None) if parent is not None else None
mod = sys.modules.get("src.endpoint_resolver") or attr
if mod is None or getattr(mod, "__file__", None):
return
sys.modules.pop("src.endpoint_resolver", None)
if parent is not None and attr is mod:
delattr(parent, "endpoint_resolver")
clear_module("routes.model_routes")
for name in extra_modules:
clear_module(name)
@contextmanager
def preserve_import_state(*module_names):
"""Save and restore sys.modules entries and parent-package attributes.
Restoration is two-phased: sys.modules entries are written back first,
then parent-package attributes. This ensures parent-attr restoration always
sees the correctly restored parent in sys.modules, regardless of argument
order — safe for callers that pass both a parent and a child module.
On exit (normal or exception), each named module is restored to its state
before the block — whether present, absent, or carrying a parent attribute.
"""
saved = {name: _save_one(name) for name in module_names}
try:
yield
finally:
# Phase 1: restore all sys.modules entries.
for name, (saved_mod, _) in saved.items():
if saved_mod is _ABSENT:
sys.modules.pop(name, None)
else:
sys.modules[name] = saved_mod
# Phase 2: restore all parent-package attributes.
for name, (_, saved_attr) in saved.items():
_restore_parent_attr(name, saved_attr)
# Names under these prefixes are the ones a leaked stub actually breaks: a
# later test doing ``import src.x`` or ``import core.x`` silently gets the
# empty stub instead of the real module.
_GUARDED_PREFIXES = ("src.", "core.")
def bare_module_stubs():
"""Return the ``src.*``/``core.*`` names currently bound to a bare stub.
A bare stub is a plain :class:`types.ModuleType` with no on-disk
``__file__`` — the object ``types.ModuleType(name)`` produces. That is the
same "is this a fake?" test the ``clear_fake_*`` helpers above use, so a
module imported from disk is never reported.
``MagicMock`` stand-ins are deliberately out of scope: they answer every
attribute, so they fail loudly at use rather than silently, and several
test modules install them on purpose.
"""
found = set()
for name, mod in list(sys.modules.items()):
if not name.startswith(_GUARDED_PREFIXES):
continue
if type(mod) is not types.ModuleType:
continue
if getattr(mod, "__file__", None):
continue
found.add(name)
return found