Files
settled-reach/docs/workshops/test-architecture/stig-round2.md
T
jpmschweitzerandClaude Opus 4.6 a87c95a6eb docs(workshops): complete QA test architecture workshop
3-round workshop with 7 agents (Tyre, Dudley, Stig, Hoshe,
Justine, Gestalt, Ozzie) plus Qatux documenting. Produced:

- 59-item prioritized test backlog (60 tickets under epic #455)
- Gauntlet test world spec: 7 rooms + hub, 48 entities
- Test client binary spec (tooling/test-client/)
- Determinism fixes (3 patches, ~22 lines)
- Server --test-mode + --port 0 design
- Content cross-reference validation (9 checks)
- make pre-pr pipeline (6-step)
- 38 client tests prioritized
- Anti-tedium features (reset plate, hub teleport, WRONG button)
- Human tester walkthrough
- CI pipeline design (deferred but documented)

Sprint 8 scope: ~17.75 team-days across 26 tickets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 14:00:15 +01:00

23 KiB

Stig — Round 2: Client Assertions, Anti-Tedium UI, Cross-Review

OQ-1: Does the Godot client depend on visible_tiles Vec ordering?

No. The client is order-independent. Fix #2 (sort visible_tiles server-side) has zero client impact.

Audited every consumer of visible_tiles in the client:

Consumer File:Line How it uses visible_tiles Order-sensitive?
GameState.apply_snapshot() game_state.gd:62-68 Stores array as-is into visible_tiles No — just assignment
GameState.apply_snapshot() game_state.gd:119-131 Iterates to build visibility_sectors Dict and visible_positions Dict, keyed by Vector2i No — Dict keyed by position, insertion order irrelevant
FogState.update_from_state() fog_state.gd:52-99 Reads GameState.visible_positions (Dict) and GameState.visibility_sectors (Dict). Writes pixel values by position into byte arrays No — position-indexed writes, any order produces same texture
TileRenderer.update_tiles() tile_renderer.gd:63-81 Iterates tiles, calls set_cell(coords, ...) per tile Noset_cell() is idempotent per coordinate. Same tilemap regardless of call order
WorldRenderer.update_from_state() world_renderer.gd:35-36 Passes visible_tiles to tile_renderer.update_tiles() Passthrough — see above

Test fixtures that reference by index:

Two test files reference visible_tiles[0] by index: test_protocol.gd:299-301 and test_rendering.gd:66-69. These assert against fixture data, not server output. If server-side sorting changes the fixture ordering, these tests need fixture updates — but they're not runtime bugs, just test data alignment. Flag for Hoshe to include in fixture regeneration.

Verdict: Fix #2 is safe to ship. No client code changes needed.


OQ-6: Fog Byte Value Constants

Yes, promote to named constants. These magic numbers appear in fog_state.gd and are assertion targets for 3+ tests. Named constants improve readability and testability.

Proposed constants (add to fog_state.gd)

# Fog texture byte values — visibility and exploration layers.
# Used by fog shader to select visual treatment per tile.
# Assertable in tests: assert_that(vis_bytes[idx]).is_equal(FogState.VIS_FORWARD)
const VIS_HIDDEN: int = 0        # Not in LOS — fully fogged
const VIS_PERIPHERAL: int = 180  # In LOS, peripheral sector — light fog
const VIS_FORWARD: int = 255     # In LOS, forward sector — clear

const EXP_UNEXPLORED: int = 0   # Never seen — dark
const EXP_EXPLORED: int = 128   # Previously seen, now out of LOS — deep fog
const EXP_VISIBLE: int = 255    # Currently in LOS — clear (matches VIS_FORWARD)

Migration in fog_state.gd

Replace the hardcoded values:

Current code Replacement
_vis_bytes[...] = 255 if sector == "Forward" else 180 _vis_bytes[...] = VIS_FORWARD if sector == "Forward" else VIS_PERIPHERAL
if _exp_bytes[idx] > 128: / _exp_bytes[idx] = 128 if _exp_bytes[idx] > EXP_EXPLORED: / _exp_bytes[idx] = EXP_EXPLORED
_exp_bytes[...] = 255 _exp_bytes[...] = EXP_VISIBLE
_vis_bytes.fill(0) _vis_bytes.fill(VIS_HIDDEN) (clearer intent)

Tests then assert: assert_that(vis_byte).is_equal(FogState.VIS_FORWARD) instead of magic 255.


Anti-Tedium Client UI Specs

Lead approved Ozzie's full anti-tedium suite. Here's the client-side implementation for each feature.

1. Room Reset Trigger

What: Player steps on a marked floor plate at a room entrance. Everything in that room resets to tick-0 state.

Client implementation:

  • Visual: A 1-tile floor plate at each room entrance, rendered as a distinct tile type (reset_plate) in the TileRenderer. Color: subtle amber outline on floor tile (reuses INSERT_COLOR_HOVER amber at 30% alpha — visible but not distracting).
  • Interaction: When the player entity occupies the reset plate tile, the interaction list shows a single verb: "Reset Room". No auto-trigger — the player must press Interact. This prevents accidental resets while walking through.
  • Feedback on activation:
    1. Brief screen flash (0.15s amber overlay at 10% opacity on the Insert CanvasLayer — diegetic, like a system pulse)
    2. Monologue-style text: "Systems recalibrated." (1.5s duration, using existing MonologueDisplay)
    3. Server handles the actual reset — client just renders the new snapshot as normal
  • No new UI elements needed. The existing interaction list + monologue display handle everything. The reset plate is a tile type + an interaction verb.

Client code changes:

  • tile_renderer.gd: Add "reset_plate" to TILE_TYPE_MAP with atlas coords (4,0), amber-tinted floor tile
  • constants.gd: Add TILE_COLOR_RESET_PLATE: Color if needed for the atlas
  • No new scene nodes, no new scripts

2. Hub Teleport Hotkey

What: One key press teleports the player back to the Central Hub. No walking.

Key: Home — unmissable on standard keyboards, not used by any game action, mnemonic ("go home"). Alternative: Backtick (but that conflicts with console conventions).

Client implementation:

  • InputMapper addition: New action TELEPORT_HUB mapped to Home key. Client-only in non-Gauntlet contexts (InputMapper can gate on a gauntlet_mode flag).
  • Wire format: Sends PlayerInput { action: "TeleportHub" } to server. Server handles the actual teleport (sets player position to hub center, recomputes visibility).
  • Client feedback:
    1. Instant camera snap to hub position (disable smoothing for 1 frame, same pattern as the startup camera anchor in main.gd:34)
    2. Brief fade-to-black-and-back (0.3s total) via a CanvasLayer overlay — gives the teleport a sense of transition without being slow
    3. No monologue — teleporting is a meta action, not diegetic

Client code changes:

  • input_mapper.gd: Add TELEPORT_HUB enum value, map to Home key
  • sim_bridge.gd: Add "TeleportHub" to _action_enum_to_wire()
  • main.gd: On next snapshot after teleport, reset camera anchor (same as _camera_anchored = false pattern)
  • New scene: UILayer/TeleportFade — a ColorRect on the UI CanvasLayer, starts transparent, tweens to black and back

3. WRONG Button (F12) — One-Press Bug Report

What: Press F12. System captures full game state, opens a one-line prompt, saves everything to disk.

Client captures:

Data Source Format
Current ObserverSnapshot GameState.current_snapshot JSON (via JSON.stringify())
Client scene tree dump get_tree().root recursive dump Text: node paths, visibility, modulate, position
Last 60 ticks of input history New InputMapper.input_history ring buffer (60 entries) JSON array of {tick, action, timestamp}
Screenshot get_viewport().get_texture().get_image() PNG
Client console log (last 100 lines) Godot's log file or custom ring buffer Text
Player-entered description One-line text input Text

Client implementation:

  • Hotkey: F12 mapped in _unhandled_input() on a new BugReportCapture autoload. Not in InputMapper — this is meta, not gameplay.
  • Flow on F12 press:
    1. Game pauses immediately (sends Pause to server)
    2. All data captured in <100ms (snapshot is already in memory, screenshot is one API call, scene dump is a recursive traversal)
    3. A minimal text input appears center-screen on the Modal CanvasLayer: "What's wrong? (one line)" with a text field and [Save] / [Cancel] buttons
    4. On Save: writes all data to tests/bug-reports/gauntlet-{YYYYMMDD-HHmmss}/ as individual files (snapshot.json, scene_tree.txt, inputs.json, screenshot.png, console.txt, description.txt)
    5. On Save or Cancel: unpauses
    6. Brief monologue: "Noted." (1s) — confirms to the tester it worked

Client code changes:

  • New autoload: client/scripts/autoloads/bug_report.gd (~80 lines)
  • New scene node: ModalLayer/BugReportPrompt (Label + LineEdit + two Buttons)
  • input_mapper.gd: Add input_history: Array ring buffer, append on every queue_action()
  • New directory: tests/bug-reports/ (gitignored)

Scene tree dump function:

func _dump_scene_tree(node: Node, depth: int = 0) -> String:
    var indent := "  ".repeat(depth)
    var line := "%s%s" % [indent, node.name]
    if node is CanvasItem:
        line += " visible=%s modulate=%s" % [node.visible, node.modulate]
    if node is Node2D:
        line += " pos=%s" % node.position
    if node is Control:
        line += " pos=%s size=%s" % [node.position, node.size]
    var result := line + "\n"
    for child in node.get_children():
        result += _dump_scene_tree(child, depth + 1)
    return result

4. Room Timer + Progress Overlay

What: Small overlay showing current room, run count, elapsed time, and checklist progress.

Position: Top-right corner of the UI CanvasLayer. Small, semi-transparent, out of the way. Below the HUD's existing elements.

Layout:

┌─────────────────────────┐
│ OCCLUSION CORRIDOR      │  ← Room name (from player position + room bounds)
│ Run #12  ·  0:47        │  ← Run counter + elapsed since room entry
│ ████░░░  4/7            │  ← Checklist progress bar + fraction
└─────────────────────────┘

Visual style:

  • Background: Color(0.05, 0.05, 0.08, 0.7) — dark, semi-transparent, matching fog aesthetic
  • Text: INSERT_COLOR_TEXT (#c8d0e0) — consistent with diegetic insert UI
  • Progress bar: filled = INSERT_COLOR_ACTIVE (#6bc9a6), empty = Color(0.2, 0.2, 0.25)
  • Font size: small (12px), monospace
  • Only visible when gauntlet_mode is true

Data flow:

  • Room name: Server includes room metadata in the Gauntlet ObserverSnapshot (or client derives from player position + a room bounds lookup table loaded from the Gauntlet content pack)
  • Run counter: Client-local. Dictionary<room_name, int> incremented when entering a new room. Persisted to user://gauntlet_stats.json between sessions.
  • Timer: Client-local. Starts when player enters a room (position crosses room bounds). Resets on room entry or room reset trigger.
  • Checklist progress: Server-side. The Gauntlet tracks which assertions have been "witnessed" (e.g., "player stood at the right position and the correct entities were visible/hidden"). Comes in the ObserverSnapshot as gauntlet_progress: {room: str, checked: int, total: int} — or client-side if we prefer no server coupling.

Client code changes:

  • New scene node: UILayer/GauntletProgress (Panel with Labels + ProgressBar)
  • New script: client/scripts/ui/gauntlet_progress.gd (~60 lines)
  • Reads gauntlet_mode flag from a launch argument or environment variable
  • Hidden when gauntlet_mode == false

Preference: client-side progress tracking. The client knows which room the player is in and can track "did the player stand at the right position" locally. This avoids coupling the server to Gauntlet-specific UI state. The checklist YAML (see section 5) defines check conditions; the client evaluates them against GameState each tick.


Refined Test Functions (32 → 35)

Based on Tyre's cross-review and Ozzie's anti-tedium additions, three new tests added and some refined.

Camera System (7 tests — unchanged)

class_name TestCameraSystem extends GdUnitTestSuite

# Existing (already in test_camera_anchor.gd):
func test_camera_position_after_ready() -> void
func test_camera_anchored_flag_after_ready() -> void
func test_camera_smoothing_off_after_ready() -> void
func test_camera_smoothing_reenabled_after_process() -> void
func test_camera_follows_player_movement() -> void

# New:
func test_camera_static_during_pause() -> void
    # Apply snapshot with tick_rate "Paused", send MoveNorth
    # Assert camera.global_position unchanged

func test_camera_position_after_rapid_snapshots() -> void
    # Call receive_bytes() 3x (simulating server ticking faster than client)
    # Call _process() once — camera should be at LAST snapshot's player position

Entity Rendering (7 tests — unchanged)

class_name TestEntityRendering extends GdUnitTestSuite

func test_entity_peripheral_alpha() -> void
    # visibility="Peripheral" → modulate.a == Constants.PERIPHERAL_ALPHA (0.5)

func test_entity_forward_full_alpha() -> void
    # visibility="Forward" → modulate.a == 1.0

func test_entity_color_player() -> void
    # kind.variant="Player" → color == Constants.ENTITY_COLOR_PLAYER

func test_entity_color_npc_unknown() -> void
    # kind.variant="Npc" → color == Constants.ENTITY_COLOR_UNKNOWN (Phase 1)

func test_entity_color_object() -> void
    # kind.variant="Object" → color == Constants.ENTITY_COLOR_OBJECT

func test_entity_removed_when_leaving_visibility() -> void
    # update_entities with entity, then without → node removed, entity_nodes empty

func test_entity_created_on_first_appearance() -> void
    # Empty renderer, update_entities with 1 entity → 1 node in entity_nodes

Z-Layer Ordering (4 tests — unchanged)

class_name TestZLayerOrdering extends GdUnitTestSuite

func test_fog_overlay_z_index() -> void
    # FogOverlay node z_index == Constants.Z_FOG (900)

func test_fog_entities_z_index() -> void
    # FogEntities node z_index == Constants.Z_FOG_ENTITIES (950)

func test_insert_overlay_canvas_layer() -> void
    # InsertOverlay CanvasLayer.layer == Constants.CANVAS_INSERT (10)

func test_ui_layer_canvas_layer() -> void
    # UILayer CanvasLayer.layer == Constants.CANVAS_UI (20)

Fog Shader State (4 tests — +1 new, uses proposed constants)

class_name TestFogState extends GdUnitTestSuite

func test_visibility_texture_forward_tile() -> void
    # Set visible_positions with Forward sector
    # Assert vis_bytes at that position == FogState.VIS_FORWARD (255)

func test_visibility_texture_peripheral_tile() -> void
    # Set visible_positions with Peripheral sector
    # Assert vis_bytes at that position == FogState.VIS_PERIPHERAL (180)

func test_exploration_persistence_after_los_exit() -> void
    # Frame 1: tile visible (EXP_VISIBLE=255)
    # Frame 2: tile not visible
    # Assert exp_bytes at that position == FogState.EXP_EXPLORED (128)

# NEW: test the hidden state explicitly
func test_visibility_texture_hidden_tile() -> void
    # Tile not in visible_positions
    # Assert vis_bytes at that position == FogState.VIS_HIDDEN (0)

UI Elements (8 tests — unchanged)

class_name TestUIElements extends GdUnitTestSuite

# Monologue (bug #5 regression)
func test_monologue_consumed_once_per_tick() -> void
    # Set current_monologue, call _consume_monologue() twice with same tick
    # Assert monologue_display.show_monologue() called once

func test_monologue_carried_forward_on_overwrite() -> void
    # receive_bytes() with monologue, receive_bytes() without monologue
    # Assert _last_snapshot still has monologue (carry-forward logic)

# Interaction list (D-057)
func test_interaction_list_shows_verbs() -> void
    # Set nearby_interactions with 2 verbs, call update_from_state()
    # Assert interaction list visible, shows both verbs

func test_interaction_list_hidden_when_empty() -> void
    # Set nearby_interactions = [], call update_from_state()
    # Assert interaction list not visible

# Dialogue (D-061)
func test_dialogue_shows_on_snapshot() -> void
    # Set current_dialogue, call _consume_dialogue()
    # Assert dialogue_box.is_dialogue_active() == true

func test_dialogue_dismissed_sends_end_input() -> void
    # Emit dialogue_dismissed signal
    # Assert SimBridge received DialogueEnd input

func test_dialogue_option_sends_response() -> void
    # Emit option_selected(1, "text")
    # Assert SimBridge received DialogueResponse with index=1

# Stance indicator (D-053)
func test_stance_indicator_matches_game_state() -> void
    # Set player_stance = "Sprint", call update_from_state()
    # Assert indicator shows Sprint state

Entity Lerp (3 tests — unchanged)

class_name TestEntityLerp extends GdUnitTestSuite

func test_entity_lerp_target_set_on_update() -> void
    # update_entities with entity at (5,5)
    # Assert _entity_targets[id] == Vector2(5*32+4, 5*32+4) (TILE_SIZE*pos + ENTITY_OFFSET)

func test_entity_snap_on_first_appearance() -> void
    # New entity → position == target immediately (no lerp from origin)
    # Assert entity_nodes[id].position == _entity_targets[id]

func test_entity_lerp_converges() -> void
    # Set target to (10,10), call _process(0.016) 20 times
    # Assert position.distance_to(target) < 1.0 (converged within ~0.3s)

Anti-Tedium (2 new tests)

class_name TestGauntletUI extends GdUnitTestSuite

# NEW: Bug report capture
func test_bug_report_captures_snapshot() -> void
    # Set GameState with known snapshot
    # Call BugReport.capture()
    # Assert output directory created with snapshot.json containing current_tick

# NEW: Gauntlet progress overlay visibility
func test_gauntlet_progress_hidden_when_not_gauntlet() -> void
    # gauntlet_mode = false
    # Assert GauntletProgress node is not visible

Final count: 35 test functions across 7 categories.


Checklist Generation: make checklist Spec

YAML Schema (room definition checklist: block)

Each Gauntlet room YAML file includes a checklist section:

# content/gauntlet/rooms/occlusion_corridor.yaml
room:
  id: occlusion_corridor
  name: "Occlusion Corridor"
  systems: [LOS, shadowcasting, vision_cone, perception_modes]
  bounds:
    origin: {x: 40, y: 0}
    size: {x: 20, y: 15}

# ... entity placement, walls, etc ...

checklist:
  - step: "Stand at corridor entrance (45,3), face East"
    verify:
      - id: occ_guard_visible
        text: "guard-1 at (48,3) is visible, Forward sector, full alpha"
        condition: "entity guard-1 visible AND sector==Forward"
      - id: occ_hidden_blocked
        text: "hidden-1 at (50,7) is NOT in entity list (wall blocks LOS)"
        condition: "entity hidden-1 NOT visible"
      - id: occ_fog_corridor
        text: "Corridor tiles ahead are visible, room behind wall is unexplored"
        condition: "fog visible_count > 10"

  - step: "Walk south to (45,8), observe peripheral vision"
    verify:
      - id: occ_guard_peripheral
        text: "guard-1 modulate.a ≈ 0.5 (Peripheral sector)"
        condition: "entity guard-1 sector==Peripheral"
      - id: occ_fog_decrease
        text: "Fog visible_count decreases as corridor narrows"
        condition: "fog visible_count < prev.visible_count"

  - step: "Switch to Sensor perception mode"
    verify:
      - id: occ_hidden_sensor
        text: "hidden-1 appears in entity list with sensor-specific data"
        condition: "entity hidden-1 visible AND perception==Sensor"

YAML Fields

Field Type Required Description
step string yes Human-readable instruction for the tester
verify array yes List of verification items for this step
verify[].id string yes Unique identifier (used for progress tracking)
verify[].text string yes Human-readable description of what to check
verify[].condition string no Machine-parseable condition for auto-progress tracking (future)

The condition field is optional — it enables the client-side progress tracker (section 4 above) to automatically check off items. For v1, human testers check items manually. The conditions use a simple grammar:

condition := "entity" <name> ("visible" | "NOT visible") [AND clause]*
           | "fog" <field> <comparator> <value>
           | "inventory" <field> <comparator> <value>
clause    := field "==" value | field "!=" value
comparator := ">" | "<" | "==" | ">="

Not a full DSL — just enough for the progress tracker to evaluate against GameState. Conditions that can't be expressed are omitted (tester checks manually).

Markdown Output Format

make checklist generates docs/qa/gauntlet-checklist.md:

# Gauntlet QA Checklist
Generated from content/gauntlet/rooms/*.yaml
Date: {generation_date}

## Occlusion Corridor
Systems: LOS, shadowcasting, vision_cone, perception_modes

### Step 1: Stand at corridor entrance (45,3), face East
- [ ] guard-1 at (48,3) is visible, Forward sector, full alpha
- [ ] hidden-1 at (50,7) is NOT in entity list (wall blocks LOS)
- [ ] Corridor tiles ahead are visible, room behind wall is unexplored

### Step 2: Walk south to (45,8), observe peripheral vision
- [ ] guard-1 modulate.a ≈ 0.5 (Peripheral sector)
- [ ] Fog visible_count decreases as corridor narrows

### Step 3: Switch to Sensor perception mode
- [ ] hidden-1 appears in entity list with sensor-specific data

---

## Inventory Warehouse
Systems: pickup, CarriedBy, inventory_grid, 9_slot_limit

### Step 1: Enter warehouse, interact with crate_1
...

---

Total: {N} rooms, {M} steps, {K} verification items

make checklist Implementation

checklist:
	@echo "Generating Gauntlet QA checklist..."
	python3 tooling/gen_checklist.py \
		--rooms content/gauntlet/rooms/ \
		--output docs/qa/gauntlet-checklist.md
	@echo "Written to docs/qa/gauntlet-checklist.md"

The Python script (tooling/gen_checklist.py, ~50 lines):

  1. Glob content/gauntlet/rooms/*.yaml
  2. Sort alphabetically (deterministic output)
  3. Parse each YAML file, extract room.name, room.systems, checklist[]
  4. Emit markdown with - [ ] checkboxes per verify item
  5. Footer with totals

Summary

Deliverable Status
OQ-1: visible_tiles ordering No client dependency — Fix #2 safe to ship
OQ-6: Fog byte constants 6 constants proposed — VIS_HIDDEN/PERIPHERAL/FORWARD, EXP_UNEXPLORED/EXPLORED/VISIBLE
Room reset trigger Reuses existing interaction list + monologue. New tile type only
Hub teleport Home key, TeleportHub wire action, fade transition
WRONG button (F12) New autoload + modal prompt, captures 6 data types to disk
Room timer + progress Top-right overlay, client-local tracking, reads checklist YAML conditions
Test functions 35 total (32 original + 1 fog hidden state + 2 anti-tedium)
Checklist spec YAML schema with step/verify/condition, make checklist generates markdown

Cross-Review Notes for Other Agents

  • For Dudley: The room reset trigger needs a server-side handler for "ResetRoom" action — resets all entities in the room's bounds to tick-0 state. Also needs "TeleportHub" action support.
  • For Tyre: The bug report capture writes to disk from GDScript. Is DirAccess.make_dir_recursive() + FileAccess.store_string() sufficient, or should we use a dedicated output directory configured at launch?
  • For Hoshe: Two fixture-based tests (test_protocol.gd:299, test_rendering.gd:66) reference visible_tiles[0] by index. If Fix #2 changes fixture ordering, these need updates during fixture regeneration. Low risk — just a heads up.
  • For Ozzie: Your State Inspector Overlay (F3) is powerful but I'd defer it to a later sprint. The WRONG button captures the same data on demand. F3 as a real-time overlay requires per-frame string formatting of the entire ObserverSnapshot — measurable performance cost. Ship WRONG button first, F3 if testers ask for it.