Files
settled-reach/docs/workshops/test-architecture/round-2-notes.md
T
jpmschweitzerandClaude Opus 4.6 a87c95a6eb docs(workshops): complete QA test architecture workshop
3-round workshop with 7 agents (Tyre, Dudley, Stig, Hoshe,
Justine, Gestalt, Ozzie) plus Qatux documenting. Produced:

- 59-item prioritized test backlog (60 tickets under epic #455)
- Gauntlet test world spec: 7 rooms + hub, 48 entities
- Test client binary spec (tooling/test-client/)
- Determinism fixes (3 patches, ~22 lines)
- Server --test-mode + --port 0 design
- Content cross-reference validation (9 checks)
- make pre-pr pipeline (6-step)
- 38 client tests prioritized
- Anti-tedium features (reset plate, hub teleport, WRONG button)
- Human tester walkthrough
- CI pipeline design (deferred but documented)

Sprint 8 scope: ~17.75 team-days across 26 tickets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 14:00:15 +01:00

624 lines
35 KiB
Markdown

# QA Strategy & Test Architecture Workshop — Round 2 Notes
**Workshop:** QA Strategy & Test Architecture
**Round:** 2 (Synthesis & Cross-Review)
**Date:** 2026-02-17
**Documenter:** Qatux
**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie
---
## Executive Summary
Round 2 produced implementation-ready specifications across all 5 tracks. The lead's overrule on WS-D5 (separate test client binary instead of server-side text renderer) was accepted constructively by Tyre, who designed the binary architecture at `server/src/bin/test_client.rs`. Dudley delivered concrete code changes for all 4 determinism fixes (one was already done — a self-correction from Round 1). The cross-review process identified real coverage gaps: Hoshe found ZERO direct test coverage for the pause guard (all 6 of Dudley's proposed tests are genuine gaps), and Tyre added 6 tests to Stig's 32 for a total of 38 client assertions. The anti-tedium suite is fully specified across Ozzie, Stig, Gestalt, and Dudley. Content cross-reference validation is specified with 8 concrete checks.
**Key convergence:** Hoshe and Justine independently designed compatible `make pre-pr` targets. Ozzie and Stig independently designed compatible checklist formats (with Ozzie identifying 4 additions). Tyre validates Hoshe's CI tiers with refinements. All cross-reviews confirmed prior proposals as feasible.
**One disagreement:** fixture staleness should be BLOCKER (Tyre) or WARNING (Justine). Tyre's argument is stronger — stale fixtures mean client tests run against outdated protocol data, making all passing results false positives.
---
## Lead Decisions — Agent Responses
The lead confirmed/overruled the 5 workshop decisions from Round 1. Here is how each was received and incorporated.
| R1 Decision | Lead Ruling | Agent Response |
|-------------|-------------|----------------|
| **WS-D1:** Commit to determinism | **ACCEPTED** | Dudley produced exact code changes for 4 fixes (see Section 2). Self-corrected Fix C (monologue ordering was already done). |
| **WS-D2:** Hybrid YAML + Rust inject | **ACCEPTED** | Dudley designed `GauntletRoom` constants module + room reset mechanism. Gestalt mapped 14 rooms to the layout. |
| **WS-D3:** Rust test macros | **ACCEPTED** | No further discussion needed. Assertion helpers referenced throughout all proposals. |
| **WS-D4:** ObserverSnapshot golden files | **ACCEPTED** | Justine designed JSON golden file format with field-by-field diff. Dudley confirmed multi-position golden files (one per room). |
| **WS-D5:** Text renderer | **OVERRULED** — separate test client binary | Tyre accepted: "The architectural purity argument wins." Designed `server/src/bin/test_client.rs` with full CLI, text format, Layer 3 integration. See Section 1. |
**Additional lead decisions incorporated:**
- No Gitea Actions for now — manual `make ci` stays. All agents adapted: Hoshe and Justine designed `make pre-pr` as the developer-facing tool. Tyre validated the CI tier design as "future-ready for when the lead greenlights CI."
- Anti-tedium full suite approved. Ozzie, Stig, Gestalt, and Dudley each contributed implementation specs.
- `--test-mode` + `--port 0` confirmed for Sprint 8. Dudley designed the server flag changes (Section 3).
- Cross-reference validation added to `make validate-content`. Hoshe specified 8 checks; Justine specified 7 checks (compatible sets).
---
## 1. Test Client Binary Architecture (Tyre)
The lead overruled the Round 1 consensus (WS-D5: server-side text renderer) in favor of a separate Rust binary. Tyre designed the architecture.
### Location
**`server/src/bin/test_client.rs`** — a second binary target in the server crate.
```toml
# server/Cargo.toml
[[bin]]
name = "settled-reach-test-client"
path = "src/bin/test_client.rs"
```
**Rationale:** Shares bridge types (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed`) with zero duplication. Rust compiles each binary independently — no server bloat. A separate crate would need to depend on the server crate anyway (or require premature extraction of a `protocol` crate).
### CLI Interface
```
settled-reach-test-client [OPTIONS]
Connection:
--connect <host:port> Connect to running server (default: 127.0.0.1:9876)
Input:
--replay <file.jsonl> Send inputs from file (one JSON PlayerInput per line)
--interactive Read inputs from stdin (future)
Output:
--text Render each snapshot as structured text to stdout
--json Dump each snapshot as JSON (for golden files)
--quiet No output (assertions only, for CI)
Assertions:
--golden <file.json> Compare final snapshot against golden file, exit 1 on diff
--ticks <N> Disconnect after N ticks (default: unlimited)
```
### Text Output Format
```
=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full ===
Game time: Day 0, 04:12 (Morning)
Entities (5):
npc:100 (18,10) Forward rel:Neutral vis:Visible
npc:101 (20,10) Forward rel:Unknown vis:Remembered
obj:200 (16,9) Forward rel:n/a vis:Visible
Tiles: 31 visible
Interactions (2):
npc:100 [Talk(1), ExamineNpc(2)] distance=3
Inventory: 2/9 [item:300(slot-0), item:301(slot-3)]
Monologue: "Something about this manifest doesn't add up."
===
```
Entities labeled as `kind:entity_id` (e.g., `npc:100`). No display names on wire — test assertions use Gauntlet constants module: `assert_entity_visible(&snapshot, GUARD_1.wire_id, GUARD_1.position)`.
### Text Renderer Lives in Library
`server/src/bridge/text_renderer.rs` — library code callable by the test client binary AND integration tests. The server binary never references it.
### Ozzie's Enhanced Terminal Layout
Ozzie proposed a richer live-updating terminal display for human testers (crossterm-based, ANSI escape codes), adding:
- Sound events section (D-018 three-range model)
- Cognition section (cognitive delay progress)
- Checklist progress bar
- Hotkey reminders (F12: WRONG, Home: Hub, R: Reset)
- Session timer
- Symbols per visibility state: `●` VISIBLE, `◐` REMEMBERED, `◌` FOGGED, `✕` BLOCKED, `⚡` RECOGNIZING
### Missing ObserverSnapshot Fields (Ozzie)
4 fields needed for the full test client experience, gated behind `--debug`/test-mode:
1. `blocked_entities: Vec<BlockedEntity>` — entities not visible with blocking wall position
2. Entity display names (or resolve via Gauntlet constants — Tyre recommends the latter)
3. Fog layer counts per type (5-layer breakdown)
4. `Vec<SoundEvent>` — source position, type, range classification
### Effort Estimate
~2-3 days total: binary scaffolding (0.5d), text renderer (0.5-1d), replay loading (0.5d), golden file comparison (0.5d), Layer 3 wiring (0.5d).
---
## 2. Determinism Fixes — Concrete Code Changes (Dudley)
Dudley produced exact code changes for all 4 fixes identified in Round 1. Total: ~40 lines changed across 4 files (revised up from Round 1's estimate of ~15 lines due to full context being included).
### Fix A: BTreeSet for visible_ids + sort visible_tiles
**Files:** `server/src/perception/query.rs`, `server/src/perception/observer/mod.rs`
Changes:
- `visible_positions: HashSet<(i32, i32)>``BTreeSet<(i32, i32)>` in `VisibilityGeometry`
- `visible_ids: HashSet<u64>``BTreeSet<u64>` in `filter_visible_entities`
- Add `visible_tiles.sort_by_key(|t| (t.x, t.y))` after collection in `NaturalVision::compute_geometry()`
- Update `collect_remembered_entities` signature to accept `&BTreeSet` references
`sector_lookup` HashMap stays — it's point-lookup only via `.get()`, never iterated.
### Fix B: Sort visible entities in snapshot
**File:** `server/src/perception/observer/mod.rs`, in `compute_observer_snapshot()`
Add after `collect_remembered_entities`:
```rust
entities.sort_by_key(|e| e.entity_id);
```
### Fix C: Pin monologue system ordering — ALREADY DONE
**Round 1 self-correction:** Dudley stated monologue systems lacked explicit ordering. After re-reading `bridge/mod.rs:168-175`, confirmed the ordering is already explicit via `.after()` constraints. **No change needed.**
### Fix D: Sort movers in validate_movement
**File:** `server/src/simulation/movement.rs`
Collect movers into Vec, sort by `entity.to_bits()`, then process:
```rust
let mut mover_list: Vec<_> = movers.iter_mut().collect();
mover_list.sort_by_key(|(entity, _, _, _)| entity.to_bits());
```
Note: `entity.to_bits()` provides stable ordering within a single run. For cross-session determinism (save/load), would need `registry.to_stable(entity)` — deferred to Sprint 10+.
### Determinism Fix Coverage (Hoshe cross-review)
Hoshe assessed that **none of the 4 fixes have direct test coverage today**. Each fix should ship with its own regression test:
- Fix A: Test observer with 2 equidistant contradicted NPCs, assert same one selected
- Fix A (tiles): Generate snapshot with random-order tiles, assert output sorted
- Fix B: Not identified separately — covered by golden file tests
- Fix D: Two entities at same distance to same tile, assert deterministic winner
---
## 3. Server `--test-mode` and `--port 0` Design (Dudley)
### `--test-mode` Flag
| Aspect | Behavior |
|--------|----------|
| Content | Loads Gauntlet content pack (`content/gauntlet/`), falls back to proof room |
| Seed | Fixed seed 42 |
| Stdout | Prints `LISTENING:{port}` after TCP bind, before accept |
| Shutdown | Exits after first client disconnect |
| Logging | Defaults to `warn` (override with `RUST_LOG`) |
### `--port 0` Support
Uses existing `TcpBridge::accept_on(listener)` (already in `tcp.rs:67`). The bind/accept split was added for test race condition prevention — exactly what `--port 0` needs. No TcpBridge changes required.
Port discovery protocol: server prints `LISTENING:{port}` to stdout, test client parses it.
---
## 4. Cross-Review Results
### 4.1 Hoshe validates Dudley's server proposals
**Pause guard:** All 6 of Dudley's proposed tests are genuine gaps. The pause guard has **ZERO direct test coverage today**. The existing test (`process_input_pause_sets_paused`) tests the Pause action side effect, not the guard itself. If someone deleted lines 102-105 (the guard), all existing tests would still pass.
Hoshe found 3 additional gaps:
- `set_tick_rate_while_paused` (P2) — SetTickRate(Half) while paused unconditionally sets rate, making `paused()` return false
- `perception_mode_while_paused` (P2) — currently no-op but should pass through guard
- `interact_take_while_paused` (P2) — documents expected behavior for Take during pause
**EntityRegistry:** 3 of 5 Dudley tests are genuine gaps. 2 are not applicable (concurrent access impossible in bevy, bulk performance is fine at 2000 entries). Hoshe found 2 additional gaps:
- `unregister_unknown_entity_is_noop` (P1) — code handles it but no test
- `register_with_pre_existing_stable_id_component` (P2) — component/registry divergence
**Bridge deserialization:** Hoshe recommends keeping batch-failure (current behavior) with a test documenting it: `malformed_input_in_batch_rejects_entire_batch`.
### 4.2 Tyre validates Stig's client proposals
All 32 tests **approved**. No redundancies. Tyre adds 6 more (38 total):
| # | Test | Category | Rationale |
|---|------|----------|-----------|
| 33 | Pending recognition blob rendering | Entity rendering | D-060 cognitive delay visual |
| 34 | Recognition transition animation | Entity lerp | Blob → full entity over ~0.3s |
| 35 | Tick rate HUD indicator | UI elements | Full/Half/Paused display |
| 36 | Inventory full visual state | UI elements | 9/9 feedback |
| 37 | Sprint interaction suppression | UI elements | D-055 interaction buffer empty during sprint |
| 38 | Entity modulate for Remembered | Entity rendering | Remembered vs Visible visual difference |
**Priority ranking:** P0 = monologue not lost on overwrite (Bug #5), camera static during pause (Bug #2). P1 = fog shader state (3 tests), entity lifecycle, pending recognition blob.
**Refinements:**
- Camera rapid snapshots test should verify convergence, not just "doesn't crash"
- Entity lifecycle test should verify node freed (not just hidden) — memory leak prevention
### 4.3 Tyre validates Hoshe's CI tiers
**Approved** with refinements:
- PR tier budget revised: <15 min (not <10 min) — clean cache server builds take 5-7 min
- Add content cross-reference validation to PR tier (<2s, negligible budget impact)
- Add content scaling stress test to Nightly tier
- **Fixture staleness should be BLOCKER, not WARNING** — stale fixtures make all client tests false positives
### 4.4 Dudley validates Hoshe's Layer 3 test
**Feasible** with 2 minor adjustments:
1. Use `rmp_serde::to_vec` (not `to_vec_named`) for input serialization — matches GDScript's encoding
2. Tolerate initial tick=0 snapshot before input is processed — server may send snapshot before reading client input
Hoshe's assertions on tick=0 snapshot are fine — they verify snapshot delivery, not input processing.
---
## 5. Resolved Open Questions
| ID | Question | Answer | Answered By |
|----|----------|--------|-------------|
| **OQ-01** | Does client depend on visible_tiles ordering? | **No.** All consumers are position-indexed (Dict keyed by Vector2i, set_cell() idempotent). Fix #2 safe to ship. Two fixture tests reference `visible_tiles[0]` by index — flag for fixture regeneration. | Stig |
| **OQ-02** | Does rmp_serde accept int_16 for u64? | **Yes.** Traced through rmp-serde 1.3.1: `Marker::I16 → visit_i16 → visit_i64 → u64::try_from`. GDScript encoding 256 as int_16 deserializes correctly. Negative values correctly rejected. | Dudley |
| **OQ-04** | WalkabilityMap HashMap → BTreeMap? | **No. Leave as HashMap.** Point-lookup only, never iterated. HashMap is faster in the hot path. Document as known-safe usage. | Tyre |
| **OQ-05** | Fixture staleness: git diff robust enough? | **Yes.** `git diff --exit-code` detects any byte-level change. Content-addressed hashing adds complexity for zero additional safety. Fixture generation is deterministic (verified — no timestamps/random values). | Justine |
| **OQ-06** | = OQ-02 (rmp_serde int_16→u64) | See OQ-02 | Dudley |
| **OQ-07** | Server --test-mode and --port 0? | **Designed.** See Section 3. Uses existing `accept_on()`. `LISTENING:{port}` signal to stdout. | Dudley |
| **OQ-08** | Room name from coordinates? | **Gauntlet coordinate bounds constants** in `server/src/test_world/constants.rs`. `room_at(player_pos)` returns room name. No server API needed. | Dudley |
| **OQ-10** | Fog byte value constants? | **Yes, promote to named constants.** `VIS_HIDDEN=0`, `VIS_PERIPHERAL=180`, `VIS_FORWARD=255`, `EXP_UNEXPLORED=0`, `EXP_EXPLORED=128`, `EXP_VISIBLE=255`. Replace magic numbers in `fog_state.gd`. | Stig |
| **OQ-12** | = OQ-04 (WalkabilityMap HashMap) | See OQ-04 | Tyre |
### Partially Resolved
| ID | Question | Status |
|----|----------|--------|
| **OQ-03** | Gitea Actions available? | Moot — lead confirmed no CI for now. Tyre notes: ~1 day effort when greenlighted. |
| **OQ-05** | Entity display_name on wire? | **Deferred.** Tyre: use `kind:entity_id` labels now. Add `display_name: Option<String>` when client needs name labels (Sprint 9-10 gameplay feature). |
| **OQ-09** | Text renderer from ObserverSnapshot? | **Yes**, confirmed. The text renderer formats ObserverSnapshot. Lives in server library, called by test client binary. |
| **OQ-11** | Client tests headless? | **Unresolved.** No agent tested this in Round 2. Remains a concern for CI. |
### Resolved Unresolved Architectural Questions (from Round 1)
| ID | Question | Resolution |
|----|----------|-----------|
| **UQ-01** | Bridge deserialization: skip-and-log vs batch failure? | **Keep batch failure.** Both sides are co-versioned (D-020). Add test documenting behavior: `malformed_input_in_batch_rejects_entire_batch`. (Hoshe) |
| **UQ-02** | Gauntlet geometry: YAML vs Rust? | **Not directly addressed in Round 2.** Dudley's code uses Rust constants for room bounds. Gestalt references YAML room definitions for checklists. The hybrid approach (WS-D2) remains — specific geometry format deferred to implementation. |
| **UQ-03** | Wall-blocking-LOS info? | **Optional `blocked_entities` field** gated behind `--debug`/test-mode. Ozzie specifies as a requirement for the test client's `✕ BLOCKED` display. Feasibility question to Dudley in Round 3. |
| **UQ-04** | Fixture path fragility? | **Addressed by `make pre-pr` fixture staleness check.** Regenerate + diff catches stale fixtures regardless of path. |
---
## 6. Content Cross-Reference Validation
Both Hoshe and Justine independently specified content validation extensions. Their proposals are compatible and complementary.
### Hoshe's 8 Checks
| Check | Severity | What | Target |
|-------|----------|------|--------|
| 1 | ERROR | `canonical_id` uniqueness across all NPCs | `npcs/*.yaml` |
| 2 | ERROR | Relationship `target` resolves to defined NPC | `npcs/*.yaml → relationships[].target` |
| 3 | ERROR | District `locations[]` slug matches location file | `district.yaml → locations/` |
| 4 | ERROR | Dialogue pool `location` matches district | `dialogue/**/*.yaml → location` |
| 5 | ERROR | `knowledge_grant.fact_id` validity | `dialogue/**/*.yaml → lines[]` |
| 6 | ERROR | NPC `triangle_membership` matches triangle file | `npcs/*.yaml` |
| 7 | WARNING | `npc_count` matches actual NPC file count | `district.yaml` |
| 8 | ERROR | Dialogue line ID uniqueness within pool | `dialogue/**/*.yaml → lines[].id` |
### Justine's 7 Checks
| Check | Severity | What |
|-------|----------|------|
| 1 | ERROR | NPC relationship targets resolve |
| 2 | ERROR | Triangle members resolve |
| 3 | ERROR | NPC triangle_membership matches triangles |
| 4 | ERROR | Dialogue pool location resolves |
| 5 | ERROR | Fact IDs resolve (absorb `check-fact-ids`) |
| 6 | ERROR | District location list matches files |
| 7 | WARNING | Bidirectional relationship consistency |
### Merged View
Hoshe and Justine agree on checks 1-6. Justine adds bidirectional relationship warnings (Check 7). Hoshe adds dialogue line ID uniqueness (Check 8) and `npc_count` accuracy (Check 7). Combined: **9 unique checks** (7 ERROR, 2 WARNING).
### Architecture Agreement
Both recommend extending `tooling/validate-content` (Python) with a second pass after schema validation. No Rust dependency — content authors validate without compiling the server. Phased rollout: NPC/triangle/district first, dialogue/monologue second, warnings third.
---
## 7. `make pre-pr` Target
Both Hoshe and Justine independently designed this target. Their proposals are compatible.
### Agreed Chain
```
pre-pr
├── 1. lint-server + lint-client (~15s)
├── 2. build-server + build-client (~30-90s)
├── 3. test-server + test-client (~15-30s)
├── 4. validate-content (~2-5s)
├── 5. check-fact-ids (~2s)
└── 6. fixture staleness check (~10-15s)
```
**Total: ~90-180s** (under 3 minutes for clean incremental build). Fast enough for every PR.
### Branch-Specific Variants (Hoshe)
- `make pre-pr-server` — lint-server, build-server, test-server, fixtures
- `make pre-pr-client` — lint-client, build-client, test-client
- `make pre-pr-content` — validate-content, check-fact-ids
### Fixture Staleness Check
```makefile
fixtures-check: fixtures
@if git diff --quiet client/tests/fixtures/; then \
echo "Fixtures: up to date"; \
else \
echo "FIXTURES STALE"; exit 1; \
fi
```
---
## 8. Anti-Tedium Suite — Full Specifications
All features approved by lead. Four agents contributed implementation specs.
### 8.1 Room Reset Trigger
| Aspect | Spec | Source |
|--------|------|--------|
| Trigger | Player steps on ResetPlate tile + presses Interact (not automatic) | Gestalt, Ozzie agree |
| Server mechanism | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system | Dudley |
| What resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles only), inventory items from room, dialogue state | Gestalt |
| What does NOT reset | Other rooms, player position (stays on plate), session timer, other-room checklist progress | Ozzie, Gestalt |
| Edge case | Items carried from room returned to tick-0 position, terminal shows "Items returned: keycard → crate_1" | Ozzie |
| Client visual | Distinct tile type (`reset_plate`), amber outline, interaction verb "Reset Room", 0.15s amber flash + monologue "Systems recalibrated." | Stig |
| Debounce | 10-tick cooldown prevents re-trigger while walking across plate | Dudley |
| Test mode only | `RoomResetTrigger` entities only added with `--test-mode` | Dudley |
### 8.2 Hub Teleport
| Aspect | Spec | Source |
|--------|------|--------|
| Hotkey | `Home` key | Ozzie, Stig agree |
| Wire format | `PlayerAction::TeleportToHub` | Gestalt |
| Server behavior | Move player entity to `GAUNTLET.hub_center`, clear dialogue/monologue/interaction buffer | Dudley, Ozzie |
| Does NOT affect | Room state, inventory, game time, knowledge graph | Ozzie, Gestalt |
| Client visual | Instant camera snap, 0.3s fade-to-black-and-back, no monologue (meta action) | Stig |
| Gauntlet-only | Server rejects `TeleportToHub` in non-Gauntlet maps | Gestalt |
### 8.3 WRONG Button (F12)
| Aspect | Spec | Source |
|--------|------|--------|
| MVP captures (Sprint 8) | ObserverSnapshot, tick + position, text render output, human description | Gestalt |
| Full captures (Sprint 9+) | + input history (60 ticks), snapshot history (60 ticks), world digest, replay seed | Ozzie |
| Output directory | `tests/bug-reports/gauntlet-{tick}-{timestamp}/` | Gestalt, Ozzie |
| Bug report format | `report.md` (human-readable), `snapshot.json`, `text_output.txt`, `description.txt` | Ozzie |
| Client implementation | `BugReportCapture` autoload, F12 hotkey in `_unhandled_input()`, modal prompt, 6 data captures | Stig |
| Ring buffer | 60 ticks (configurable via `--history-buffer`) | Ozzie |
### 8.4 Room Timer + Personal Bests
| Aspect | Spec | Source |
|--------|------|--------|
| Display | `TIMER: 00:47 (PB: 00:38)` in status bar/overlay | Ozzie |
| Start | Player enters room (crosses bounding box) | Ozzie |
| Reset | Room reset trigger resets timer | Ozzie |
| Persistence | `tests/gauntlet-stats.json` — local, not committed | Ozzie |
| Session summary | Printed on disconnect: rooms visited, coverage %, times, PBs, bug reports filed | Ozzie |
### 8.5 Auto-Checklist Progress (Ozzie + Stig merged)
Ozzie identified 4 gaps in Stig's per-room checklist proposal and proposed a merged format:
| Gap | Stig's Proposal | Ozzie's Addition |
|-----|----------------|-----------------|
| No machine-readable conditions | Human prose only | Add `condition:` field (structured data) for auto-tracking |
| No auto vs manual distinction | All items equal | Add `type: auto/manual` — auto-confirms from snapshot, manual requires tester input |
| No cross-room items | Per-room only | Add `cross_room_checks.yaml` at Gauntlet root |
| No failure guidance | Just "check X" | Add `if_wrong:` field with likely causes + file references |
**Merged checklist YAML format** (Ozzie):
```yaml
checks:
- id: occ_01_hidden_not_visible
description: "NPC behind wall is NOT visible"
type: auto
condition:
player_near: [15, 10]
entity: hidden-1
expected: blocked
if_wrong: |
LOS leaking through wall. Check shadowcast.rs.
```
`make checklist` generates markdown from YAML (Stig's `tooling/gen_checklist.py`). Test client loads structured conditions for auto-tracking.
### 8.6 Stig's Client-Side Anti-Tedium UI
- **Room reset:** Tile type `reset_plate` in TileRenderer, interaction verb "Reset Room", amber flash
- **Hub teleport:** `TELEPORT_HUB` in InputMapper → `Home` key, fade transition
- **WRONG button:** New `bug_report.gd` autoload, 60-entry `input_history` ring buffer
- **Progress overlay:** Top-right panel with room name, run counter, timer, checklist progress bar. Only visible when `gauntlet_mode == true`.
### 8.7 Deferred: F3 Debug Overlay
**Stig recommends deferring** the F3 State Inspector Overlay. The WRONG button captures the same data on demand. F3 as a real-time overlay requires per-frame string formatting of the entire ObserverSnapshot — measurable performance cost. Ship WRONG button first, F3 if testers ask for it.
---
## 9. Gauntlet Room Coverage (Gestalt)
### Final Room List: 14 Rooms
| # | Room | Primary Systems | Source |
|---|------|----------------|--------|
| 1 | Inventory Warehouse | Pickup, CarriedBy, 9-slot limit | Brief |
| 2 | Occlusion Corridor | LOS, shadowcasting, perception modes | Brief |
| 3 | Interaction Gallery | ObjectType verbs, sprint suppression | Brief |
| 4 | Crowd Plaza | Entity density, relationship colors, cognitive delay | Brief |
| 5 | Fog Theater | Fog transitions, peripheral dimming, exploration persistence | Brief |
| 6 | Dialogue Room | Trust tiers, contradiction, walk-away, monologue during dialogue | Brief |
| 7 | Pause Chamber | TickRate toggle, state transitions | Brief |
| 8 | Zone Gate | Zone transition (reserved, future contract) | Brief |
| 9 | Eavesdrop Alcove | ListeningFocus, zone ambient, Careful stance | Gestalt R1 |
| 10 | Confrontation Stage | Cognitive vulnerability, audio dip | Gestalt R1 |
| 11 | Sprint Gauntlet | Sprint suppression, anomaly survival, stance transitions | Gestalt R1 |
| 12 | Sound Lab | Three-range sound, sound pings, recognition chime | Gestalt R1 |
| 13 | Decay Observatory | Knowledge decay, stale state, fog degradation | Gestalt R1 |
| 14 | Shift Change | Stress test: all systems at density | Gestalt R1 |
### Coverage Matrix
Gestalt mapped 50+ system-to-room relationships across 6 pillars: Characters & Information, Perception, Movement & Interaction, Audio, Simulation & Architecture, Content Systems. Every system from confirmed decisions has at least one room exercising it.
**Coverage gaps (3 minor, all addressable without new rooms):**
1. Object-layer favorite colors (D-052) — add to Inventory Warehouse in v0.1.2+
2. POI navigation (D-013) — add to Hub as POI markers. v0.1 stretch.
3. Environmental neutrality (D-045) — assertion on Dialogue Room (pre/post confrontation CanvasModulate identical)
### 8 Transition Scenarios
All scenarios reference physically connected rooms via hub paths or cross-cuts:
| # | Scenario | Path | Key Test |
|---|----------|------|----------|
| T1 | Sprint Exit | Plaza → Occlusion Corridor | Buffer cleared during sprint, LOS recalculated |
| T2 | Fog into Dialogue | Fog Theater → Dialogue Room | Fog state preserved during dialogue, monologue above dialogue box |
| T3 | Full Inventory Interact | Inventory → Interaction Gallery | 9/9 inventory, Take still offered server-side, client greys out |
| T4 | Sprint into Interaction | Sprint Gauntlet → Interaction Gallery | Verbs repopulate within 1 tick after stance change |
| T5 | Confrontation to Eavesdrop | Confrontation → Eavesdrop | Audio dip release + ListeningFocus activation don't conflict |
| T6 | Pause Anywhere | Pause Chamber → Hub → any room | Pause-during-transition state corruption |
| T7 | Sound across Fog | Sound Lab → Fog Theater | Sound propagation through walls, cognitive delay from sound |
| T8 | Walk-away Sprint | Dialogue → Hub → Sprint Gauntlet | KG incompleteness recorded, sprint suppresses post-dialogue monologue |
### Top 10 Invariants for Sprint 8 (Gestalt)
| Rank | ID | Invariant | Bug Match |
|------|-----|-----------|-----------|
| 1 | INV-T04 | Pause coherence | Bug #3 |
| 2 | INV-T01 | Deterministic replay | Bug #1 class |
| 3 | INV-T03 | Snapshot delivery | Bug #1, #5 |
| 4 | INV-T05 | Input ordering | Bug #1 |
| 5 | INV-S01 | Player spawn reachable | Softlock prevention |
| 6 | INV-C03 | StableId uniqueness | Corruption prevention |
| 7 | INV-C07 | Dialogue pool non-empty | Player-facing failure |
| 8 | INV-T02 | Tick budget | Bug #6 class |
| 9 | INV-P02 | LOS symmetry | D-035 mandate |
| 10 | INV-S05 | No entity inside geometry | Content scaling safety |
**Bug catalogue mapping: every Sprint 6-7 bug is now covered** by at least one invariant + room + test type.
---
## 10. Encoding Asymmetry Tests (Hoshe)
Hoshe specified 4-direction cross-language testing for the encoding asymmetry between GDScript (int_16 for 256-32767) and Rust (uint_16):
| Direction | What | Test Location | When |
|-----------|------|---------------|------|
| Rust → GDScript (fixture) | Snapshots with overlap-zone ticks | `test_msgpack_boundaries.gd` | Every PR |
| GDScript → Rust (fixture) | Inputs with overlap-zone ticks | `serialization.rs` | Every PR |
| Rust → GDScript (raw bytes) | Hand-crafted uint_16/uint_32 bytes | `test_msgpack_boundaries.gd` | Every commit |
| GDScript → Rust (raw bytes) | Hand-crafted int_16/int_32 bytes | `serialization.rs` | Every commit |
New `make fixtures-client` target generates GDScript-encoded fixtures for Rust to verify. Reverse direction of existing `make fixtures`.
---
## 11. Performance & Golden File Tooling (Justine)
### Performance Baseline
- `tests/perf/baseline.json` — committed, records median/min/max from 5 runs
- `tooling/perf-measure` — builds release, runs benchmarks, compares against baseline
- Thresholds: <15% = PASS, 15-30% = WARNING, >30% = FAIL
- Machine tag prevents meaningless cross-machine comparisons
- `make perf-baseline` (compare) and `make perf-baseline-update` (update)
### Golden File Diff
- **Rust test, not separate tool** — golden file is an ObserverSnapshot, Rust code knows the structure
- JSON format with sorted keys, pretty-printed (`serde_json::to_string_pretty`)
- Field-by-field diff output on failure: changed fields, POSITION markers, added/removed entities
- `make golden-diff` (view diff) and `make golden-update` (regenerate)
- Why JSON not MessagePack: human-readable in `git diff`, sorted keys = deterministic output
---
## 12. Client Test Suite — Final Count
| Category | Round 1 (Stig) | Round 2 Additions | Total |
|----------|---------------|-------------------|-------|
| Camera system | 7 | — | 7 |
| Entity rendering | 7 | +2 (Tyre: blob, remembered modulate) | 9 |
| Z-layer ordering | 4 | — | 4 |
| Fog shader state | 3 | +1 (Stig: hidden state) | 4 |
| UI elements | 8 | +3 (Tyre: tick rate, inventory full, sprint suppression) | 11 |
| Entity lerp | 3 | +1 (Tyre: recognition transition) | 4 |
| Anti-tedium | — | +2 (Stig: bug report capture, progress hidden) | 2 |
| **Total** | **32** | **+9** (6 Tyre + 3 Stig) | **~38-41** |
Note: Stig counts 35 (32+3), Tyre counts 38 (32+6). Combined unique total depends on overlap — upper bound is 41.
---
## 13. Open Questions for Round 3
### New Questions (raised in Round 2)
| ID | From | To | Question |
|----|------|----|----------|
| R2-OQ-01 | Hoshe | Dudley | `SetTickRate(Half)` while paused — should this unpause? Current code sets rate unconditionally (input.rs:165-168). Intentional or bug? |
| R2-OQ-02 | Hoshe | Dudley | Entity respawn + registry — when bevy recycles Entity index, does registry handle old StableId not being unregistered? |
| R2-OQ-03 | Hoshe | Tyre | `make pre-pr` — should it include `make content-ron` (YAML→RON conversion)? |
| R2-OQ-04 | Hoshe | Justine | Fixture staleness in `make pre-pr` — separate `make pre-pr-full` to keep basic pre-PR fast? |
| R2-OQ-05 | Ozzie | Dudley | `blocked_entities` debug field on ObserverSnapshot — feasible? Cost per tick? |
| R2-OQ-06 | Ozzie | Stig | Can Godot client render checklist progress overlay, or test-client-only? |
| R2-OQ-07 | Ozzie | Tyre | Test client binary location: `server/src/bin/` or separate `tools/` crate? |
| R2-OQ-08 | Ozzie | Gestalt | Cross-room transition scenarios: where in YAML hierarchy? Own checklist section? |
| R2-OQ-09 | Gestalt | All | Room ordering in Gauntlet YAML — canonical ordering affects entity StableId assignment. |
| R2-OQ-10 | Gestalt | All | Per-room reset sufficient, or need "full server restart" command? |
| R2-OQ-11 | Gestalt | All | 4 new cross-cuts — too many? Consolidate Sound Lab into Occlusion Corridor sub-area? |
### Remaining Unresolved from Round 1
| ID | Question | Status |
|----|----------|--------|
| OQ-11 | Client tests headless stability | Unresolved — no agent tested in Round 2 |
---
## 14. Points of Agreement (consensus)
1. **Test client binary is the right call.** Tyre accepted the overrule and designed it properly. The binary shares types, exercises real TCP, and enables Layer 3 testing.
2. **Determinism fixes are small and well-understood.** Dudley's code changes are concrete. Hoshe's coverage assessment confirms no existing tests break.
3. **Pause guard has zero coverage.** Hoshe independently confirmed Dudley's gap analysis. All 6 tests are genuine needs.
4. **`make pre-pr` replaces CI discipline.** Both Hoshe and Justine converged on the same chain (lint → build → test → validate → fixtures).
5. **Content cross-reference validation extends Python validator.** Both Hoshe and Justine agree: no Rust dependency, two-pass architecture (schema then cross-refs).
6. **Anti-tedium is fully specified.** Room reset (Interact-triggered), hub teleport (Home key), WRONG button (F12 + MVP captures). No dissent on any feature.
7. **38 client tests are architecturally sound.** Tyre's cross-review found no redundancies in Stig's 32 and added 6 meaningful tests.
8. **Bug catalogue fully covered.** Gestalt's invariant mapping confirms every Sprint 6-7 bug class has a test + room + invariant.
9. **Checklist format merges Stig + Ozzie proposals.** YAML with structured conditions for auto-tracking AND human prose for markdown generation.
## 15. Points of Tension
1. **Fixture staleness: BLOCKER vs WARNING.** Tyre argues BLOCKER (stale fixtures = false positive client tests). Justine tagged it WARNING. Recommend resolving in Round 3 — Tyre's argument appears stronger.
2. **F3 debug overlay: defer or implement.** Ozzie wants it for the testing experience. Stig says defer due to per-frame performance cost. WRONG button captures same data on demand. Lean toward deferral.
3. **PR tier time budget: <10 min (Hoshe) vs <15 min (Tyre).** Tyre accounts for clean-cache builds. Recommend <15 min as the budget (covers worst case).
4. **Cross-room checklist location.** Ozzie identifies a real gap: cross-room transition scenarios (Gestalt's T1-T8) don't fit in per-room YAML. Needs `cross_room_checks.yaml` or equivalent.
5. **Number of Gauntlet cross-cuts.** Gestalt proposes 4 new cross-cuts. Potential scope concern. Could consolidate Sound Lab into Occlusion Corridor sub-area.
---
## 16. Gaps Remaining
1. **Test client binary doesn't exist yet.** Tyre estimated 2-3 days. Depends on `--test-mode` + `--port 0` (Dudley, Sprint 8).
2. **Gauntlet content doesn't exist yet.** Room YAML, entity placement, checklist definitions all need writing. No specific agent assigned.
3. **Client test headless stability unverified.** OQ-11 from Round 1 remains open. Must verify before making client tests a merge gate.
4. **Cross-room checklist format unresolved.** Ozzie identified the gap. Needs design in Round 3.
5. **Gauntlet room ordering (entity spawn order → StableId assignment).** Gestalt raised this. Affects determinism — canonical YAML ordering needed.
6. **Sound events not in ObserverSnapshot.** Ozzie requires `Vec<SoundEvent>` for full test client display. Not yet designed on server side.
7. **Cognitive delay visual not yet implemented.** Tyre's tests #33-34 (blob rendering, recognition transition) depend on D-060 implementation.