fix(tests): resolve temp paths consistently on macOS

Three of the six recorded failures were the same test bug: an unresolved
/tmp path compared against a resolved /private/tmp one. macOS makes /tmp a
symlink, so a fixture built with tempfile.mkdtemp(dir="/tmp") and a code path
that resolves what it reports disagree about a file both found correctly.

test_code_nav_tools builds its fixture unresolved and compares it against the
reported path. One realpath fixes both of its failures.

test_glob_confined_e2e is the same cause through a longer route: it mixed
os.path.realpath(ws) with an unresolved secret directory, so relpath emitted
"../../../../tmp/<absolute path>" and the assertion that the absolute path was
absent from the output matched it as a substring. Resolving the secret
directory puts both sides in one tree and the relative path stays short.

macOS full suite goes from 6 failures to 3. The remaining three are an ffmpeg
build without a WebP encoder, a socket test that needs a fast connection
refusal, and the rich-text colour test that is still unexplained.

The ledger is updated in the same change so it does not describe failures that
no longer happen.
This commit is contained in:
Léo
2026-09-30 10:56:58 +02:00
parent b5d1505582
commit 32d9dbc267
3 changed files with 27 additions and 21 deletions
+16 -18
View File
@@ -9,10 +9,10 @@ This is that list. It is a record of observation, not a permission slip: a test
here is still a test that does not pass, and three of the six below are
defects someone should fix.
Last measured: `lab @ c499c01b`, macOS 15 on Apple Silicon, Python 3.11.
Last measured: `lab @ c499c01b` plus the fixes in this change, macOS 15 on Apple Silicon, Python 3.11.
```
6 failed, 10658 passed, 6 skipped
3 failed, 10658 passed, 6 skipped
```
## Get the prerequisites right first
@@ -38,26 +38,24 @@ including one holding real data.
Miss `npm ci` and roughly 36 browser tests fail on `Cannot find package
'playwright'`. That is not a regression, it is the missing install.
## The six
## The three
### Test bugs: comparing an unresolved path against a resolved one
### Test bugs: fixed
- `tests/test_code_nav_tools.py::test_read_file_extracts_structured_documents`
- `tests/test_code_nav_tools.py::test_read_file_extracts_legacy_word_documents`
- `tests/test_workspace_confine.py::test_glob_confined_e2e`
Three failures compared an unresolved `/tmp` path against a resolved
`/private/tmp` one, and are fixed rather than listed:
```
assert [('/private/tmp/codenav_.../report.docx', ...)]
== [('/tmp/codenav_.../report.docx', ...)]
```
- `tests/test_code_nav_tools.py` (two tests) built a fixture under
`tempfile.mkdtemp(dir="/tmp")` and compared it against the path the code
reports, which it resolves.
- `tests/test_workspace_confine.py::test_glob_confined_e2e` mixed
`os.path.realpath(ws)` with an unresolved secret directory, so `relpath`
produced `../../../../tmp/<absolute path>` and the assertion that the
absolute path was absent matched it as a substring.
On macOS `/tmp` is a symlink to `/private/tmp`. The code under test resolves
the path and the assertion does not, so the two disagree about a file they both
found. Nothing is wrong with the behaviour.
**These are fixable and should be fixed**: resolve both sides before comparing.
They are listed as known rather than environmental because the platform is only
what exposes them.
Both now resolve consistently. They are recorded here because the shape recurs:
on macOS, mixing a resolved and an unresolved temp path is a test bug that
looks like a platform failure.
### Optional dependency: ffmpeg without a WebP encoder
+4 -1
View File
@@ -17,7 +17,10 @@ def _run(tool, content):
@pytest.fixture
def repo():
# Built under /tmp, which is on the default tool-path allowlist.
root = tempfile.mkdtemp(dir="/tmp", prefix="codenav_")
# realpath because the code under test resolves the path it reports, and on
# macOS /tmp is a symlink to /private/tmp: comparing the unresolved path
# against the resolved one fails on a file both sides found correctly.
root = os.path.realpath(tempfile.mkdtemp(dir="/tmp", prefix="codenav_"))
try:
with open(os.path.join(root, "a.py"), "w") as f:
f.write("import os\n# needle here\nprint('x')\n")
+7 -2
View File
@@ -225,8 +225,13 @@ async def test_glob_confined_e2e(ws, admin):
assert ws not in r["output"]
assert "/workspace/found.py" in r["output"]
# a secret outside the workspace must not be discoverable via glob
outside = tempfile.mkdtemp()
# a secret outside the workspace must not be discoverable via glob.
# realpath so this directory and os.path.realpath(ws) below sit in the same
# resolved tree. On macOS /tmp is a symlink to /private/tmp, and mixing a
# resolved workspace with an unresolved secret makes relpath emit
# "../../../../tmp/<abs path>", which trivially contains the absolute path
# the assertion is checking for.
outside = os.path.realpath(tempfile.mkdtemp())
secret = os.path.join(outside, "secret.txt")
with open(secret, "w") as f:
f.write("nope")