fix(tests): resolve temp paths consistently on macOS

Three of the six recorded failures were the same test bug: an unresolved
/tmp path compared against a resolved /private/tmp one. macOS makes /tmp a
symlink, so a fixture built with tempfile.mkdtemp(dir="/tmp") and a code path
that resolves what it reports disagree about a file both found correctly.

test_code_nav_tools builds its fixture unresolved and compares it against the
reported path. One realpath fixes both of its failures.

test_glob_confined_e2e is the same cause through a longer route: it mixed
os.path.realpath(ws) with an unresolved secret directory, so relpath emitted
"../../../../tmp/<absolute path>" and the assertion that the absolute path was
absent from the output matched it as a substring. Resolving the secret
directory puts both sides in one tree and the relative path stays short.

macOS full suite goes from 6 failures to 3. The remaining three are an ffmpeg
build without a WebP encoder, a socket test that needs a fast connection
refusal, and the rich-text colour test that is still unexplained.

The ledger is updated in the same change so it does not describe failures that
no longer happen.
This commit is contained in:
Léo
2026-09-30 10:56:58 +02:00
parent b5d1505582
commit 32d9dbc267
3 changed files with 27 additions and 21 deletions
+16 -18
View File
@@ -9,10 +9,10 @@ This is that list. It is a record of observation, not a permission slip: a test
here is still a test that does not pass, and three of the six below are here is still a test that does not pass, and three of the six below are
defects someone should fix. defects someone should fix.
Last measured: `lab @ c499c01b`, macOS 15 on Apple Silicon, Python 3.11. Last measured: `lab @ c499c01b` plus the fixes in this change, macOS 15 on Apple Silicon, Python 3.11.
``` ```
6 failed, 10658 passed, 6 skipped 3 failed, 10658 passed, 6 skipped
``` ```
## Get the prerequisites right first ## Get the prerequisites right first
@@ -38,26 +38,24 @@ including one holding real data.
Miss `npm ci` and roughly 36 browser tests fail on `Cannot find package Miss `npm ci` and roughly 36 browser tests fail on `Cannot find package
'playwright'`. That is not a regression, it is the missing install. 'playwright'`. That is not a regression, it is the missing install.
## The six ## The three
### Test bugs: comparing an unresolved path against a resolved one ### Test bugs: fixed
- `tests/test_code_nav_tools.py::test_read_file_extracts_structured_documents` Three failures compared an unresolved `/tmp` path against a resolved
- `tests/test_code_nav_tools.py::test_read_file_extracts_legacy_word_documents` `/private/tmp` one, and are fixed rather than listed:
- `tests/test_workspace_confine.py::test_glob_confined_e2e`
``` - `tests/test_code_nav_tools.py` (two tests) built a fixture under
assert [('/private/tmp/codenav_.../report.docx', ...)] `tempfile.mkdtemp(dir="/tmp")` and compared it against the path the code
== [('/tmp/codenav_.../report.docx', ...)] reports, which it resolves.
``` - `tests/test_workspace_confine.py::test_glob_confined_e2e` mixed
`os.path.realpath(ws)` with an unresolved secret directory, so `relpath`
produced `../../../../tmp/<absolute path>` and the assertion that the
absolute path was absent matched it as a substring.
On macOS `/tmp` is a symlink to `/private/tmp`. The code under test resolves Both now resolve consistently. They are recorded here because the shape recurs:
the path and the assertion does not, so the two disagree about a file they both on macOS, mixing a resolved and an unresolved temp path is a test bug that
found. Nothing is wrong with the behaviour. looks like a platform failure.
**These are fixable and should be fixed**: resolve both sides before comparing.
They are listed as known rather than environmental because the platform is only
what exposes them.
### Optional dependency: ffmpeg without a WebP encoder ### Optional dependency: ffmpeg without a WebP encoder
+4 -1
View File
@@ -17,7 +17,10 @@ def _run(tool, content):
@pytest.fixture @pytest.fixture
def repo(): def repo():
# Built under /tmp, which is on the default tool-path allowlist. # Built under /tmp, which is on the default tool-path allowlist.
root = tempfile.mkdtemp(dir="/tmp", prefix="codenav_") # realpath because the code under test resolves the path it reports, and on
# macOS /tmp is a symlink to /private/tmp: comparing the unresolved path
# against the resolved one fails on a file both sides found correctly.
root = os.path.realpath(tempfile.mkdtemp(dir="/tmp", prefix="codenav_"))
try: try:
with open(os.path.join(root, "a.py"), "w") as f: with open(os.path.join(root, "a.py"), "w") as f:
f.write("import os\n# needle here\nprint('x')\n") f.write("import os\n# needle here\nprint('x')\n")
+7 -2
View File
@@ -225,8 +225,13 @@ async def test_glob_confined_e2e(ws, admin):
assert ws not in r["output"] assert ws not in r["output"]
assert "/workspace/found.py" in r["output"] assert "/workspace/found.py" in r["output"]
# a secret outside the workspace must not be discoverable via glob # a secret outside the workspace must not be discoverable via glob.
outside = tempfile.mkdtemp() # realpath so this directory and os.path.realpath(ws) below sit in the same
# resolved tree. On macOS /tmp is a symlink to /private/tmp, and mixing a
# resolved workspace with an unresolved secret makes relpath emit
# "../../../../tmp/<abs path>", which trivially contains the absolute path
# the assertion is checking for.
outside = os.path.realpath(tempfile.mkdtemp())
secret = os.path.join(outside, "secret.txt") secret = os.path.join(outside, "secret.txt")
with open(secret, "w") as f: with open(secret, "w") as f:
f.write("nope") f.write("nope")