jpmschweitzerandClaude Opus 5
a384ec0c7c
reach godot parse-sweep / cold-parse, reach visual diff / blank-check / thumbnail. Five scripts retired, and the callers rewired — tests/run-visual invoked three of them by path at four sites, which is a wider blast radius than the make targets were. The godot pair were grep pipelines encoding five hard-won lessons as comments nobody could test. They are Python filters now, with the reasons attached, and the engine invocation is a guarded exec. Verified on the real client: 229 scripts, clean. Their three not-ok states stay distinct, because only one is a verdict about the code. An engine that crashed or is missing is not a parse failure — reporting it as one blames the tree for a broken toolchain. A sweep that emitted no completion marker checked nothing, and zero errors from a check that never ran reads as clean, which is the false-green the sweep exists to close. The deliberate asymmetry between the two checks is preserved and documented: cold-parse filters "Cannot infer the type", the sweep does not, because that suppression is why cold-parse stayed silent about a helper that genuinely does not parse. All three visual scripts carried the same root bug as validate-checklist: Path(__file__).parent.parent, correct at tooling/ and two levels too deep at tooling/domains/visual. Fixed during the move rather than after, having learned that it fails silently — paths resolve to nothing, the work appears to have nothing to do, and the tool reports success. Three domains now where that would have shipped a false pass. Two bugs my own transformation introduced, both found by running rather than reading. Multi-line print(..., file=sys.stderr) became console.event(..., file=sys.stderr), and console puts unknown kwargs into the payload — a file object would have reached json.dumps at the exact moment something was already being reported as an error. And the replacement script wrote escaped quotes into three files. Mechanical transformations need mechanical verification. sys.exit removed from four sites: a service must not end the process. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
92 lines
3.6 KiB
Python
Executable File
92 lines
3.6 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Reject a screen capture in which the renderer drew nothing.
|
|
|
|
WHY THIS EXISTS (2026-08-06). `tests/run-visual` verified only that the
|
|
captured PNG was non-empty *as a file*. A capture of a blank screen is a
|
|
perfectly valid ~19 KB PNG, so it passed — and then, because the golden had
|
|
been recorded from an equally blank capture, `visual-diff` reported 0.0%
|
|
difference and the scenario PASSED. Two Atlas scenarios
|
|
(`atlas_GJ338Bd_Block`, `atlas_GJ445c-m1_Chunk`) sat green for months against
|
|
blank goldens. This is the same failure `e024cfb3f` recorded once already
|
|
("the Atlas Global goldens have been measuring nothing"); it kept recurring
|
|
because nothing checked the property, only the file.
|
|
|
|
A green comparison between two blank images is the worst kind of test result:
|
|
it is indistinguishable from success and it is load-bearing for exactly the
|
|
work it fails to cover.
|
|
|
|
THE METRIC. Fraction of the frame occupied by the single most common colour.
|
|
Measured on this project's real captures:
|
|
|
|
Global (real world map) 38.7% modal
|
|
Region (flat colour wash) 7.2% modal <- dither; least uniform of all
|
|
District 45.4% modal
|
|
Block / Chunk / Quarter 92.9-94.6% modal <- nothing drawn
|
|
|
|
Nothing falls between 45% and 93%, so the default 0.85 threshold sits in open
|
|
space rather than being tuned to a boundary case.
|
|
|
|
Note what this deliberately does NOT do: it does not judge whether a render is
|
|
GOOD. The Region wash is a real product gap (T-1213), and it scores 7.2% —
|
|
comfortably "content". The question here is only "did the renderer put a world
|
|
on the screen", never "is the world any good". Keep it that way; a check that
|
|
creeps toward aesthetics will start failing legitimate frames and get muted.
|
|
|
|
Usage:
|
|
tooling/visual-blank-check <PNG> [--max-modal 0.85] [--quiet]
|
|
|
|
Exit: 0 = has content, 1 = blank, 2 = error (unreadable/missing).
|
|
"""
|
|
|
|
from pathlib import Path
|
|
|
|
from tooling.core import console
|
|
|
|
DEFAULT_MAX_MODAL = 0.85
|
|
|
|
|
|
def modal_fraction(path: Path) -> tuple[float, int]:
|
|
"""Return (modal colour's share of the frame, distinct colour count)."""
|
|
try:
|
|
from PIL import Image
|
|
except ImportError:
|
|
console.event("visual-blank-check: Pillow not available", level="error")
|
|
raise SystemExit(2)
|
|
|
|
with Image.open(path) as im:
|
|
rgb = im.convert("RGB")
|
|
total = rgb.size[0] * rgb.size[1]
|
|
if total == 0:
|
|
return 1.0, 0
|
|
# maxcolors high enough that a real frame is never bucketed away;
|
|
# getcolors returns None if it overflows, which itself means the frame
|
|
# is richly coloured and therefore certainly not blank.
|
|
colors = rgb.getcolors(maxcolors=1 << 24)
|
|
if colors is None:
|
|
return 0.0, 1 << 24
|
|
top = max(colors)[0]
|
|
return top / total, len(colors)
|
|
|
|
|
|
def run(path, max_modal: float = DEFAULT_MAX_MODAL, quiet: bool = False) -> int:
|
|
"""Fail if a capture is overwhelmingly one colour. Returns 0, 1 or 2."""
|
|
if not path.is_file():
|
|
console.event(f"visual-blank-check: no such file: {path}", level="error")
|
|
return 2
|
|
|
|
modal, distinct = modal_fraction(path)
|
|
|
|
if modal > max_modal:
|
|
console.event(
|
|
f"BLANK: {path.name} is {modal:.1%} a single colour "
|
|
f"({distinct} distinct) — the renderer drew nothing but chrome. "
|
|
f"A golden recorded from this would pass against any other blank "
|
|
f"capture forever.",
|
|
level="error",
|
|
)
|
|
return 1
|
|
|
|
if not quiet:
|
|
console.event(f"content: {path.name} modal={modal:.1%} distinct={distinct}")
|
|
return 0
|