settled-reach/docs/workshops/test-architecture/ozzie-round3.md

title, description, type, status, workshop, agent, round, created

title	description	type	status	workshop	agent	round	created
Ozzie — Round 3: Final Tester Workflow + UX Specs	Ozzie's final tester workflow and UX specifications for the Gauntlet test client	workshop	archived	test-architecture	ozzie	3	2026-02-17

Ozzie — Round 3: Final Tester Workflow + UX Specs

Workshop: QA Strategy & Test Architecture Track: 1 (Test World Design) — Human Tester Experience Date: 2026-02-17 Round: 3 (Prioritization) Inputs: All Round 2 outputs, round-2-notes.md

---

1. Human Tester Walkthrough — Final

This is the definitive step-by-step workflow. A human tester picks this up and knows exactly what to do from zero to bug report.

Prerequisites

• Server binary built: make build-server
• Test client binary built: make build-test-client (builds settled-reach-test-client)
• Gauntlet content pack exists at content/gauntlet/
• Checklist generated: make checklist (produces docs/qa/gauntlet-checklist.md)

Session Start

Terminal 1 — Server:

make test-world-headless
# Equivalent to: settled-reach-server --test-mode --port 0 --seed 42
# Server prints: LISTENING:54321
# Server waits for client connection

Terminal 2 — Test Client:

make test-client
# Equivalent to: settled-reach-test-client --connect 127.0.0.1:54321 --text
# Test client connects, receives first ObserverSnapshot
# Live terminal display appears

What the tester sees on connect:

╔══════════════════════════════════════════════════════════════════════╗
║ GAUNTLET TEST CLIENT v0.1            Tick: 0   TickRate: Full      ║
║ Room: Central Hub                    Seed: 42  Session: 00:00:01   ║
╠══════════════════════════════════════════════════════════════════════╣
║ PLAYER (50,50) → North | Walk | Inventory: 0/9                    ║
╠══════════════════════════════════════════════════════════════════════╣
║ ENTITIES (0 visible)                                               ║
╠══════════════════════════════════════════════════════════════════════╣
║ FOG  Clear:24 | Periph:8 | Deep:0 | Map:0 | Dark:468              ║
╠══════════════════════════════════════════════════════════════════════╣
║ EXITS: [N]Fog Theater [E]Occlusion [S]Dialogue [W]Inventory       ║
╠══════════════════════════════════════════════════════════════════════╣
║ MONOLOGUE: none | DIALOGUE: none                                   ║
╠══════════════════════════════════════════════════════════════════════╣
║ CHECKLIST: Central Hub — N/A                                       ║
║ [F12:WRONG] [Home:Hub] [R:Reset]                                   ║
╚══════════════════════════════════════════════════════════════════════╝

Phase 1: Navigate to a Room

Tester sends movement inputs (arrow keys or WASD via --interactive mode, or via scripted replay file). They walk East toward the Occlusion Corridor.

As they enter the corridor connecting Hub to Occlusion Corridor:

• Room name changes: Room: → Occlusion Corridor
• Timer starts: TIMER: 00:00
• Checklist loads: CHECKLIST: Occlusion Corridor — 0/7
• Entities appear as they enter the player's vision cone

Phase 2: Execute the Checklist

The tester has a printed copy of docs/qa/gauntlet-checklist.md taped to their monitor (or the test client shows checklist items in the status bar).

Step 1 from checklist: "Stand at corridor entrance (45,3), face East"

Tester moves to position. Terminal updates:

╠══════════════════════════════════════════════════════════════════════╣
║ ENTITIES (3)                                                       ║
║  ● npc:100   (48,3)  Fwd   Neutral(#4a9ebb)  VISIBLE   d=3       ║
║  ● obj:200   (46,2)  Fwd   n/a               VISIBLE   d=1       ║
║  ✕ npc:101   (50,7)  ---   ---               BLOCKED              ║
║    └─ wall at (49,4) blocks LOS                                    ║
╠══════════════════════════════════════════════════════════════════════╣

Tester checks against the checklist:

• npc:100 (guard-1) visible, Forward sector? YES
• npc:101 (hidden-1) NOT visible? YES — blocked by wall
• Corridor tiles visible, room behind wall unexplored? Check FOG line

Three items checked. Progress bar updates: 3/7 ██████░░░░░

Step 2: "Walk south to (45,8), observe peripheral vision"

Tester moves. guard-1 shifts from Forward to Peripheral:

║  ◐ npc:100   (48,3)  Periph  Neutral(#4a9ebb)  VISIBLE   d=6     ║

• guard-1 in Peripheral sector? YES — symbol changed from ● to ◐
• Fog visible_count decreased? Check FOG line: was 24, now 18

Step 3: "Switch to Sensor perception mode"

Tester sends TogglePerceptionMode input. If working correctly, hidden-1 should appear:

║  ⚡ npc:101   (50,7)  ---   Unknown(#4a9ebb)  DETECTED   d=7     ║

• hidden-1 appears in entity list with sensor data? YES

Checklist complete: 7/7. Timer shows 00:47. If personal best was 00:38, tester sees (PB: 00:38).

Phase 3: Encounter a Bug

What if step 3 fails? hidden-1 doesn't appear even in Sensor mode. The tester presses F12.

F12 Flow:

1. Game pauses immediately (Pause sent to server)
2. Terminal shows:

╔══════════════════════════════════════════════════════════════════╗
║ BUG REPORT — Tick 87 — Occlusion Corridor                      ║
╠══════════════════════════════════════════════════════════════════╣
║ What's wrong? (type, then Enter):                              ║
║ > Sensor mode doesn't detect NPC behind wall_                  ║
╚══════════════════════════════════════════════════════════════════╝

3. Tester types one sentence, presses Enter
4. Terminal shows:

Bug report saved: tests/bug-reports/gauntlet-t87-20260217-084213/
  report.md | snapshot.json | text_output.txt | description.txt
Resume testing. [Enter]

5. Game unpauses. Tester continues.

Phase 4: Reset and Re-Test

Tester walks back to the room entrance and steps on the reset plate (2x2 amber-striped floor tile). Interaction list shows [Reset Room]. Tester activates it.

Reset: Occlusion Corridor → tick-0 state. Timer reset.

All entities return to starting positions. Fog reverts. Timer restarts. Checklist resets to 0/7. Tester can re-run the checklist from step 1.

Phase 5: Move to Another Room

Tester presses Home → teleports to Central Hub. Walks to next room. Repeat Phase 2-4.

Phase 6: End Session

Tester presses Ctrl+C. Test client prints:

══════════════════════════════════════════════════════
SESSION SUMMARY — 2026-02-17 08:55
══════════════════════════════════════════════════════
Rooms tested:  4 / 14
Total time:    12:34

  Occlusion Corridor:   7/7  ███████████ 100%  00:47
  Fog Theater:          5/8  ████████░░░  63%  01:12
  Inventory Warehouse:  7/7  ███████████ 100%  00:25  ★ PB
  Crowd Plaza:          4/9  ██████░░░░░  44%  01:35

Bug reports filed: 1
  → tests/bug-reports/gauntlet-t87-20260217-084213/

Stats saved: tests/gauntlet-stats.json
══════════════════════════════════════════════════════

Done. Developer picks up the bug report folder, has everything they need to reproduce.

---

2. Test Client UX Final Spec

Merges my Round 2 terminal layout with Tyre's kind:entity_id labels and the lead-confirmed architecture decisions.

Display Architecture

• Binary: settled-reach-test-client at tooling/test-client/ (separate crate, imports shared types from server)
• Text renderer: tooling/test-client/src/text_renderer.rs (or shared module within the crate)
• Display mode: Live-updating terminal via crossterm ANSI escape codes. NOT a TUI framework. Fixed-layout sections that refresh every tick.
• Simultaneous logging: Every tick appended to gauntlet-session-{timestamp}.log for the WRONG button and post-session review.

Entity Label Format

Per Tyre's decision: kind:entity_id. Examples:

• npc:100 — NPC with wire entity_id 100
• obj:200 — Object with wire entity_id 200
• player:0 — Player entity

No display names on wire. Test assertions use Gauntlet constants: assert_entity_visible(&snapshot, GUARD_1.wire_id).

For human testers, the printed checklist maps IDs to names: "npc:100 = guard-1, npc:101 = hidden-1". This lives in the checklist header, not the terminal display.

Entity Visibility Symbols

Symbol	State	Meaning
●	VISIBLE	In clear vision cone (Forward or Peripheral)
◐	REMEMBERED	In fog, previously seen, entity persists in snapshot
◌	FOGGED	Detected in fog, NOT yet recognized (grey blob state)
✕	BLOCKED	Exists but LOS blocked by wall (debug info, requires blocked_entities field)
⚡	RECOGNIZING	Mid-cognitive-delay (D-060), transitioning from blob to recognized

Peripheral entities show ● but with the Periph sector label — the symbol indicates visibility state, the sector label indicates where in the cone.

Section Order (top to bottom)

1. Header — tick, tick rate, room name, seed, session timer
2. Player — position, facing, stance, inventory count
3. Entities — one line per entity, sorted by distance (nearest first per Tyre's format)
4. Fog — 5-layer tile counts (Clear/Periph/Deep/Map/Dark)
5. Sound — events this tick (MVP: omit if no SoundEvent in snapshot yet)
6. Cognition — active cognitive delays (MVP: derive from pending_recognitions)
7. Interactions — available verbs per entity
8. Monologue/Dialogue — exact text content
9. Inventory — slot map
10. Status — checklist progress, timer, PB, hotkey reminders

MVP vs Full Display

Sprint 8 MVP (what ships first):

Sections 1, 2, 3, 4, 7, 8, 9, 10. These derive entirely from the existing ObserverSnapshot fields. No server changes needed.

Sprint 9+ Full (requires new snapshot fields):

Add sections 5 (Sound) and 6 (Cognition). These need:

• Vec<SoundEvent> in ObserverSnapshot (source position, type, range)
• Enhanced pending_recognitions with elapsed/total timing
• blocked_entities for the ✕ BLOCKED display

Refresh Behavior

• Per-tick refresh. Every ObserverSnapshot received triggers a full terminal redraw.
• Cursor positioning. Use ANSI escape \x1b[H (cursor home) + section rewrites. No full clear (avoids flicker).
• Paused display. When TickRate is Paused, display shows [PAUSED] in header. Still refreshes on snapshot receipt (pause state changes are snapshots).

---

3. WRONG Button Final Spec

MVP — Sprint 8

The MINIMUM that makes bug reporting useful. Implementable without any server changes.

Hotkey: F12

What gets captured:

Data	Source	Format	Size
Current ObserverSnapshot	Last received snapshot (already in memory)	JSON	~2-5KB
Current text output	Last format_snapshot_text() result	Text	~1KB
Tick + room + seed	From snapshot + Gauntlet constants	Part of report.md	Trivial
Tester description	One-line text prompt	Text	~100 bytes

What does NOT ship in MVP:

• No snapshot history ring buffer (requires 60x snapshot storage)
• No input history (requires InputMapper ring buffer)
• No screenshot (test client is terminal-only)
• No world digest (requires server-side addition)

UX Flow (MVP):

1. F12 pressed
2. Test client sends Pause to server
3. Terminal shows one-line prompt: BUG: What's wrong? > _
4. Tester types, presses Enter
5. Test client writes files to tests/bug-reports/gauntlet-t{tick}-{timestamp}/:

tests/bug-reports/gauntlet-t87-20260217-084213/
├── report.md           # Human-readable summary (generated)
├── snapshot.json       # ObserverSnapshot as pretty-printed JSON
├── text_output.txt     # What the terminal was showing
└── description.txt     # Tester's one-line description

6. Test client sends Unpause
7. Testing resumes

report.md Format (MVP):

# Bug Report — Gauntlet
- **Tick:** 87
- **Room:** Occlusion Corridor
- **Seed:** 42
- **Date:** 2026-02-17 08:42:13
- **Description:** Sensor mode doesn't detect NPC behind wall

## Player State
Position: (45,8) facing East | Stance: Walk | Inventory: 0/9

## Entities at Time of Report
| ID | Position | Sector | Relationship | Visibility | Distance |
|----|----------|--------|-------------|------------|----------|
| npc:100 | (48,3) | Periph | Neutral | VISIBLE | 6 |
| npc:101 | (50,7) | --- | --- | BLOCKED | 7 |

## Fog State
Clear: 18 | Peripheral: 6 | Deep: 12 | Map: 0 | Dark: 464

## Checklist State
Occlusion Corridor: 5/7 — item 'occ_hidden_sensor' FAILED

## Reproduction
1. `make test-world-headless` (seed 42)
2. `make test-client`
3. Walk to Occlusion Corridor (45,8), face East
4. Switch to Sensor perception mode
5. Expected: npc:101 appears in entity list
6. Actual: npc:101 still BLOCKED

The reproduction steps are templated from the current checklist step + room position. A developer reads this and can reproduce in under a minute.

Full — Sprint 9+

Everything in MVP plus:

Addition	Source	Requires
Snapshot history (60 ticks)	Ring buffer in test client	~300KB memory, client-side only
Input history (60 inputs)	Ring buffer in test client	~12KB memory, client-side only
Replay seed + input file	Combine seed + full input log	Can reproduce entire session
Room metadata + expected state	From checklist YAML	Client loads checklist at startup

Additional files in Full mode:

tests/bug-reports/gauntlet-t87-20260217-084213/
├── report.md
├── snapshot.json
├── snapshot_history.jsonl    # Last 60 snapshots, one per line
├── input_history.jsonl       # Last 60 inputs, one per line
├── text_output.txt
├── description.txt
└── room_metadata.json        # Checklist items + expected state

The ring buffer is the key addition. 60 ticks = ~6 seconds at 10 tps. When the tester presses F12, the last 6 seconds of game state are preserved. A developer can replay those 6 seconds to see exactly what happened leading up to the bug.

Implementation Notes for Dudley (server) and Stig (client)

Server (Dudley): MVP requires ZERO server changes. The test client formats the existing ObserverSnapshot. Full mode also requires no server changes — the ring buffer and input history are client-side. The only server-side addition (deferred to Sprint 9+) is blocked_entities for the ✕ BLOCKED display.

Client (Stig): The Godot client gets its own F12 handler (bug_report.gd autoload per Stig's R2 spec). The Godot version captures a screenshot + scene tree dump that the test client can't. Both clients write to the same tests/bug-reports/ directory. Reports from either client are useful.

---

4. Quick-Test Developer Workflow

A developer just fixed the fog shader. They want to verify the Occlusion Corridor works. Total time target: under 2 minutes.

The Flow

# Step 1: Build (incremental, ~5-10s)
make build-server

# Step 2: Start server + client (split terminal or use tmux)
make test-world-headless &
# Wait for LISTENING:port output
make test-client

# Step 3: Teleport to target room (~2s)
# Press Home (if not already in Hub)
# Walk East to Occlusion Corridor (or type room shortcut if implemented)

# Step 4: Run the specific checklist items (~30-60s)
# Move to (45,3), face East — check entities
# Move to (45,8) — check peripheral
# Toggle Sensor mode — check hidden NPC

# Step 5: Done
# If all good: Ctrl+C, session summary confirms 7/7
# If bug found: F12, one sentence, resume or Ctrl+C

Total wall-clock: Build 10s + startup 3s + navigate 5s + test 45s + exit 2s = ~65 seconds.

Why This Is Fast

1. Incremental build — only recompiles changed files (~5-10s for a shader fix)
2. Hub-and-spoke layout — walk directly to target room, skip everything else
3. No server restart for re-test — room reset plate lets them retry without killing the server
4. No report overhead — F12 captures everything in one press, no manual note-taking

For Regression Verification After a Fix

The developer fixed the bug from the report. Now they want to verify:

# Option A: Manual verification (interactive)
make test-world-headless &
make test-client
# Navigate to the room, reproduce the steps from report.md
# Verify the bug is fixed

# Option B: Automated verification (replay file)
make test-world-headless &
settled-reach-test-client \
  --connect 127.0.0.1:54321 \
  --replay tests/bug-reports/gauntlet-t87-20260217-084213/input_history.jsonl \
  --ticks 90 \
  --text
# Watch the replay, verify the entity appears correctly at tick 87

Option B is available once the Full WRONG button ships (Sprint 9+) — it uses the captured input history as a replay file. The developer literally replays the exact sequence that triggered the bug and checks if it's fixed.

---

5. Anti-Tedium Priority for Sprint 8 MVP

Four features approved. Not all are equally important for Sprint 8. Here's the priority ranking.

Sprint 8: MUST SHIP (2 features)

Priority 1: Room Reset Triggers

Why first: Without room reset, a tester who wants to re-run a room's checklist must restart the server. That's 30+ seconds of downtime every retry. Over a sprint of testing, that's hours of wasted time. Room reset is the difference between "I'll re-test that" and "I'll skip it."

Implementation scope:

• Server: RoomResetTrigger component, RoomSnapshots resource, detect_room_reset + execute_room_reset systems, 10-tick debounce. ~150 lines (Dudley's R2 spec).
• Client: reset_plate tile type in TileRenderer, "Reset Room" interaction verb. ~20 lines (Stig's R2 spec).
• Test mode only: systems only registered with --test-mode.

Estimated effort: 1-1.5 days.

Priority 2: WRONG Button (MVP)

Why second: Without the WRONG button, a tester who finds a bug has to manually describe it — writing down the tick, room, entity positions, what they expected. That's slow, error-prone, and most testers won't bother. The WRONG button makes bug reporting a 5-second action.

Implementation scope:

• Test client: capture current snapshot + text output, one-line prompt, write to disk. ~100 lines.
• Godot client: bug_report.gd autoload, F12 handler, modal prompt, snapshot + screenshot + scene tree dump. ~80 lines (Stig's R2 spec).
• No server changes.

Estimated effort: 1 day.

Sprint 9: SHOULD SHIP (2 features)

Priority 3: Hub Teleport

Why deferred: Hub teleport saves walking time (~5-10 seconds per room transition). Useful, but the Gauntlet corridors are only ~10-15 tiles long. Walking is tolerable in Sprint 8. Hub teleport becomes more valuable as more rooms are added.

Implementation scope:

• Server: TeleportToHub PlayerAction variant, process in process_player_input. ~20 lines.
• Client: Home key mapping, TeleportHub wire action, camera snap + fade. ~30 lines.

Estimated effort: 0.5 days.

Priority 4: Room Timer + Auto-Checklist Progress

Why deferred: Timers and progress tracking are engagement features — they make testing more pleasant, not more possible. The Gauntlet works without them. They become valuable once the team is doing regular sprint-over-sprint testing.

Implementation scope:

• Test client: room detection from bounds, wall-clock timer, stats file, session summary. ~150 lines.
• Test client: checklist YAML loading, condition evaluation against snapshot, progress display. ~200 lines.
• Godot client: GauntletProgress overlay. ~60 lines (Stig's R2 spec).
• Stats persistence: tests/gauntlet-stats.json.

Estimated effort: 1.5-2 days.

Summary Table

Priority	Feature	Sprint	Effort	Why
P1	Room Reset	8	1-1.5d	Can't re-test without it
P2	WRONG Button (MVP)	8	1d	Can't report bugs without it
P3	Hub Teleport	9	0.5d	Convenience, not necessity
P4	Timer + Checklist	9	1.5-2d	Engagement, not necessity

Sprint 8 anti-tedium budget: ~2-2.5 days. This fits alongside the other Sprint 8 priorities (test client binary, determinism fixes, server flags).

What About F3 Debug Overlay?

Defer indefinitely. Stig is right — the WRONG button captures the same data on demand. F3 as a real-time overlay is a performance cost for a marginal convenience gain. If testers ask for it after using the WRONG button for a sprint, reconsider.

---

Summary of Round 3 Deliverables

#	Deliverable	Status
1	Human tester walkthrough — final (start to finish, 6 phases)	Complete
2	Test client UX final spec (labels, symbols, sections, MVP vs full)	Complete
3	WRONG button final spec (MVP Sprint 8 + Full Sprint 9+)	Complete
4	Quick-test developer workflow (65-second flow)	Complete
5	Anti-tedium Sprint 8 priority (Room Reset + WRONG Button MVP)	Complete

Build-Ready Specs

These specs are implementation-ready for the assigned developers:

• Dudley: Room reset server mechanism (Section 5, P1). Use your R2 RoomResetTrigger + RoomSnapshots design. No server changes needed for WRONG button MVP.
• Stig: Room reset tile type + WRONG button Godot client (Section 5, P1+P2). Use your R2 reset_plate tile + bug_report.gd autoload designs.
• Tyre: Test client binary text renderer + WRONG button integration (Section 2+3). Use your R2 test_client.rs design. Add the F12 capture flow from Section 3 MVP.

The human tester picks up docs/qa/gauntlet-checklist.md, opens two terminals, and goes. That's the goal. That's what we're shipping.