Commit Graph
5 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 5 6147529fe8 test(config): reject captures where the renderer drew nothing
run-visual verified only that the captured PNG was non-empty AS A FILE. A
blank screen is a perfectly valid ~19 KB PNG, so it passed — and once a blank
capture had been recorded as a golden, every later blank capture matched it at
0.0% and the scenario PASSED. atlas_GJ338Bd_Block and atlas_GJ445c-m1_Chunk
sat green against blank goldens while the suite's other 30 scenarios failed.

That is the worst kind of test result: indistinguishable from success, and
load-bearing for exactly the work it fails to cover. e024cfb3f recorded this
same failure once already ("the Atlas Global goldens have been measuring
nothing"); it recurred because nothing checked the property, only the file.

tooling/visual-blank-check measures the share of the frame taken by its single
most common colour. On this project's real captures the classes are far apart:

  Global (real world map)     38.7% modal
  Region (flat colour wash)    7.2% modal   <- dither; least uniform of all
  District                    45.4% modal
  Block / Chunk / Quarter     92.9-94.6% modal   <- nothing drawn

Nothing falls between 45% and 93%, so the 0.85 default sits in open space
rather than being tuned against a boundary case. Deliberately NOT an aesthetic
judgement: the Region wash is a real product gap (T-1213) and scores 7.2%,
comfortably "content". The question is only whether a world reached the
screen.

Wired into both paths, and the update path is the one that matters — refusing
to RECORD a blank golden is what stops the trap being re-armed. Ad-hoc
--screenshot only warns, since capturing a rung that renders nothing is a
legitimate thing to want to do; that is how the empty deep rungs were found.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 09:12:12 +02:00
jpmschweitzerandClaude Opus 5 8e69503e77 test(client): capture offscreen under gamescope, at the native 3440x1440
Two faults, one fix.

Captures were stealing the desktop. Only the golden path even tried to go
offscreen, via xvfb-run -- which is not installed here, so it took the
"using visible window" fallback; --screenshot and --movie never wrapped at
all. Every capture opened a Godot window on the machine Jeroen is working
and gaming on. Now a single wrapper covers all three paths.

gamescope, not the alternatives, for two independent reasons. It renders on
the real AMD GPU, and the goldens are pinned to this box's Mesa/AMD output
(T-1121 -- they do not port across rendering stacks), so xvfb-run's llvmpipe
would shift every pixel: offscreen must not silently mean a different
renderer. And it is the only installed option that lets the output size be
set. cage is also present and also GPU-backed, but it is a kiosk compositor
and forces its client to the headless output's default -- measured, a
960x540 request produced a 1280x720 PNG. A wrapper that quietly changes
resolution is worse than none here.

Worse, because resolution is not cosmetic on this map. D-255's extent
inversion makes the shorter viewport axis span exactly one cell of the rung,
so the viewport decides how much world a rung shows and at what cell count
-- a small capture is a DIFFERENT map, not a scaled one. 960x540 was also
16:9, so it never exercised the ultrawide aspect added in 21e263d0a, which
is the aspect actually in daily use. Raised to the panel's native 3440x1440.

It paid for itself immediately: at native, Ferrath Global reports
courses=375 drawn=0 -- every river culled, where the same build drew them at
960x540. Filed as T-1239. That is exactly the class of bug a too-small
capture hides.

Goldens are NOT regenerated here. They are stale across 15 commits already,
and blessing the current look before it has been reviewed is the trap this
suite just spent a day proving.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 16:47:13 +02:00
jpmschweitzerandClaude Opus 5 c91096a3c9 test(client): retire the top-down visual suite, aim the Atlas ladder at land
The visual gate had stopped measuring anything: 30 of 32 scenarios
failed, and the two that passed were the worst result of the lot.

Deleted the 15 top-down scenarios (fog, HUD, dialogue, NPC, minimap,
cursor) and their goldens. They all failed at a near-uniform ~12%, and
that uniformity across unrelated scenes is one global cause -- the
ultrawide UI stretch moved every element. They cover the renderer the
cascade freezes until Phase 5, which will need its own tests anyway, so
re-baselining would only have blessed a deprecated layer nobody is
reviewing. Jeroen's call.

The remaining problem was the goldens that PASSED. atlas_GJ338Bd_Block
and atlas_GJ445c-m1_Chunk matched at 0.0% because capture and golden
were both blank -- the same "goldens have been measuring nothing" trap
e024cfb3f caught at Global, still live at the bottom of the ladder. The
cause is that every below-Global golden descends at jump_to(ZERO), and
world-metre zero is merely the origin of the region grid, not anywhere
chosen. So _setup_atlas_golden_shot now takes an optional world_center
(default ZERO -- existing goldens are untouched), and a new
atlas_GJ820Bc_land_* set walks Region through Chunk at ONE land point,
so the rungs can be read as a descent instead of five unrelated frames.

aliveness_probe prints the placement's world metres alongside its pixel
and survey cell, since that is the coordinate the Atlas actually
navigates in.

Recorded because it will be asked again: the ladder is anchored via a
CityPlacement, but that is a match record -- a pixel, an archetype, an
orientation -- not built geography. No settlement exists anywhere yet
and none is due before T-1207, so the empty deep rungs are the expected
state. What the ladder judges is the nature layer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 16:33:57 +02:00
jpmschweitzerandClaude Opus 4.6 ac763fef97 feat(engine): live server visual tests and gauntlet snapshot replay
Add live server lifecycle to tests/run-visual (start/stop server per
scenario, parse LISTENING:{port}). Add MessagePack snapshot replay to
visual_capture.gd via Protocol.decode_snapshot() — exercises the full
client pipeline from wire bytes to rendered fog. Three replay scenarios
(hub_spawn, fog_theater, hub_after_movement) plus one live scenario
(fog_live_hub). Add gen_gauntlet_fixtures.rs to produce .msgpack fixtures
from the Gauntlet test world. Add max_diff_pct threshold to visual-diff.
Makefile: add fixtures-gauntlet target, fix build-client double-import,
preserve .godot cache in clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 08:41:09 +01:00
jpmschweitzerandClaude Opus 4.6 6bcdc48412 feat(client): add visual test harness with golden regression
Gives Claude eyes: `make screenshot` captures a rendered frame,
`make test-visual` compares against golden PNGs, `make visual-update`
regenerates goldens. Built to debug the Sprint 22 fog regression and
prevent future visual regressions across fog, HUD, dialogue, and UI.

Config-driven via tests/visual.json (11 scenarios, 2 flows).
Capture engine boots main.tscn with real GPU rendering (not --headless),
waits for NoiseTexture2D async gen, uses deterministic shader time.

Components:
- visual_capture.gd: SceneTree capture engine (scenario + movie modes)
- visual_scenarios.gd: per-scenario setup hooks
- tooling/visual-diff: pixel comparator (PIL primary, struct fallback)
- tooling/visual-thumbnail: contact sheet + crop tool
- tests/run-visual: suite script (xvfb wrapping, golden workflow)
- fog_state.gd: override_time for deterministic captures
- fog_shader.gd: fog_noise_ready signal for settle sequencing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 16:33:55 +01:00