From a87c95a6eb6a3dfffef33f0fbc00527ae631b3c5 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Tue, 17 Feb 2026 14:00:15 +0100 Subject: [PATCH] docs(workshops): complete QA test architecture workshop 3-round workshop with 7 agents (Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie) plus Qatux documenting. Produced: - 59-item prioritized test backlog (60 tickets under epic #455) - Gauntlet test world spec: 7 rooms + hub, 48 entities - Test client binary spec (tooling/test-client/) - Determinism fixes (3 patches, ~22 lines) - Server --test-mode + --port 0 design - Content cross-reference validation (9 checks) - make pre-pr pipeline (6-step) - 38 client tests prioritized - Anti-tedium features (reset plate, hub teleport, WRONG button) - Human tester walkthrough - CI pipeline design (deferred but documented) Sprint 8 scope: ~17.75 team-days across 26 tickets. Co-Authored-By: Claude Opus 4.6 --- .../test-architecture/dudley-round2.md | 575 +++++++++ .../test-architecture/dudley-round3.md | 1075 +++++++++++++++++ .../test-architecture/gestalt-round2.md | 418 +++++++ .../test-architecture/gestalt-round3.md | 764 ++++++++++++ .../test-architecture/hoshe-round1.md | 747 ++++++++++++ .../test-architecture/hoshe-round2.md | 651 ++++++++++ .../test-architecture/hoshe-round3.md | 677 +++++++++++ .../test-architecture/justine-round2.md | 750 ++++++++++++ .../test-architecture/justine-round3.md | 996 +++++++++++++++ .../test-architecture/ozzie-round2.md | 562 +++++++++ .../test-architecture/ozzie-round3.md | 520 ++++++++ .../test-architecture/round-1-notes.md | 563 +++++++++ .../test-architecture/round-2-notes.md | 623 ++++++++++ .../test-architecture/round-3-notes.md | 590 +++++++++ .../test-architecture/stig-round2.md | 510 ++++++++ .../test-architecture/stig-round3.md | 637 ++++++++++ .../test-architecture-workshop-brief.md | 247 ++-- .../test-architecture/tyre-round2.md | 405 +++++++ .../test-architecture/tyre-round3.md | 956 +++++++++++++++ .../test-architecture/workshop-outcomes.md | 655 ++++++++++ 20 files changed, 12848 insertions(+), 73 deletions(-) create mode 100644 docs/workshops/test-architecture/dudley-round2.md create mode 100644 docs/workshops/test-architecture/dudley-round3.md create mode 100644 docs/workshops/test-architecture/gestalt-round2.md create mode 100644 docs/workshops/test-architecture/gestalt-round3.md create mode 100644 docs/workshops/test-architecture/hoshe-round1.md create mode 100644 docs/workshops/test-architecture/hoshe-round2.md create mode 100644 docs/workshops/test-architecture/hoshe-round3.md create mode 100644 docs/workshops/test-architecture/justine-round2.md create mode 100644 docs/workshops/test-architecture/justine-round3.md create mode 100644 docs/workshops/test-architecture/ozzie-round2.md create mode 100644 docs/workshops/test-architecture/ozzie-round3.md create mode 100644 docs/workshops/test-architecture/round-1-notes.md create mode 100644 docs/workshops/test-architecture/round-2-notes.md create mode 100644 docs/workshops/test-architecture/round-3-notes.md create mode 100644 docs/workshops/test-architecture/stig-round2.md create mode 100644 docs/workshops/test-architecture/stig-round3.md create mode 100644 docs/workshops/test-architecture/tyre-round2.md create mode 100644 docs/workshops/test-architecture/tyre-round3.md create mode 100644 docs/workshops/test-architecture/workshop-outcomes.md diff --git a/docs/workshops/test-architecture/dudley-round2.md b/docs/workshops/test-architecture/dudley-round2.md new file mode 100644 index 000000000..d4bdb3aa4 --- /dev/null +++ b/docs/workshops/test-architecture/dudley-round2.md @@ -0,0 +1,575 @@ +# Dudley — Round 2: Determinism Fixes, Server Flags, Cross-Review + +**Workshop:** QA Strategy & Test Architecture +**Tracks:** 2 (Determinism), 3 (Automation), 4 (Serialization) +**Date:** 2026-02-17 +**Round 2 scope:** Concrete code changes, server flag design, open question answers, cross-review + +--- + +## 1. Determinism Fixes — Concrete Code Changes + +Four fixes to achieve deterministic single-player, single-thread simulation. Estimated total: ~40 lines changed across 4 files. + +### Fix A: BTreeSet for visible_ids in observer snapshot + +**Problem:** `filter_visible_entities` in `observer/mod.rs:217-219` uses `HashSet` for `visible_ids`. The HashSet itself is only used for membership tests (`contains`), but the `entities: Vec` output is built by iterating `all_entities.iter()` — bevy query order is non-deterministic. Additionally, `collect_remembered_entities` iterates `visible_ids` for its exclusion check — safe (membership test), but `visible_positions` in `VisibilityGeometry` is also a HashSet and feeds into the same exclusion logic. + +However, the real non-determinism in visible_tiles comes from **upstream**: the shadowcasting algorithm outputs `HashSet<(i32, i32)>` (`shadowcast.rs:99-100`), which `apply_vision_cone` iterates via `fov.visible_tiles()` (`shadowcast.rs:52-53`). This `HashSet::iter()` order feeds directly into `VisibilityGeometry.visible_tiles`, making the entire tile list non-deterministically ordered. + +**Fix:** Sort `visible_tiles` in `NaturalVision::compute_geometry()` after collection. Change `visible_ids` to `BTreeSet` for consistency. + +**File: `server/src/perception/query.rs`** + +```rust +// Line 8 — BEFORE: +use std::collections::{HashMap, HashSet}; + +// AFTER: +use std::collections::{BTreeSet, HashMap, HashSet}; +``` + +```rust +// Line 24 — BEFORE: +pub visible_positions: HashSet<(i32, i32)>, + +// AFTER: +pub visible_positions: BTreeSet<(i32, i32)>, +``` + +**File: `server/src/perception/query.rs`, inside `NaturalVision::compute_geometry()`** + +```rust +// Lines 67-97 — BEFORE: +let visible_tiles = cone_tiles + .iter() + .map(|&(x, y, sector)| { + // ... + }) + .collect(); + +let visible_positions = cone_tiles.iter().map(|&(x, y, _)| (x, y)).collect(); + +let sector_lookup = cone_tiles + .iter() + .map(|&(x, y, sector)| ((x, y), sector)) + .collect(); + +// AFTER: +let mut visible_tiles: Vec<_> = cone_tiles + .iter() + .map(|&(x, y, sector)| { + let tile_kind = if walkability.can_move_to(&TilePosition::new(x, y, z)) { + TileKind::Floor + } else { + TileKind::Wall + }; + VisibleTile { + x, + y, + z, + visibility: sector, + tile_kind, + } + }) + .collect(); +// Sort for deterministic snapshot ordering +visible_tiles.sort_by_key(|t| (t.x, t.y)); + +let visible_positions: BTreeSet<(i32, i32)> = + cone_tiles.iter().map(|&(x, y, _)| (x, y)).collect(); + +let sector_lookup = cone_tiles + .iter() + .map(|&(x, y, sector)| ((x, y), sector)) + .collect(); +``` + +**File: `server/src/perception/observer/mod.rs`** + +```rust +// Line 10 — BEFORE: +use std::collections::HashSet; + +// AFTER: +use std::collections::{BTreeSet, HashSet}; +``` + +```rust +// Line 217-219 — BEFORE: +fn filter_visible_entities( + // ... +) -> (Vec, HashSet) { + let mut entities = Vec::new(); + let mut visible_ids: HashSet = HashSet::new(); + +// AFTER: +fn filter_visible_entities( + // ... +) -> (Vec, BTreeSet) { + let mut entities = Vec::new(); + let mut visible_ids: BTreeSet = BTreeSet::new(); +``` + +```rust +// Line 284-285 — BEFORE: +fn collect_remembered_entities( + // ... + visible_ids: &HashSet, + visible_positions: &HashSet<(i32, i32)>, + +// AFTER: +fn collect_remembered_entities( + // ... + visible_ids: &BTreeSet, + visible_positions: &BTreeSet<(i32, i32)>, +``` + +**Impact:** `visible_tiles` in the snapshot is now sorted by (x, y). `visible_ids` is a BTreeSet (deterministic iteration for any future use). `visible_positions` is a BTreeSet (deterministic membership tests — same result as HashSet, but consistent type choice). The `sector_lookup` HashMap stays — it's point-lookup only via `.get()`, never iterated. + +### Fix B: Sort visible entities in snapshot + +**Problem:** `filter_visible_entities` builds the `entities: Vec` list by iterating `all_entities.iter()` — bevy query iteration order is archetype-based and not stable across runs. The snapshot entity list order is therefore non-deterministic. + +**Fix:** Sort entities by `entity_id` after collection. + +**File: `server/src/perception/observer/mod.rs`, inside `compute_observer_snapshot()`** + +```rust +// After line 126 (after collect_remembered_entities call) — ADD: +entities.sort_by_key(|e| { + // Primary sort: entity_id for deterministic ordering + // Secondary sort not needed — entity_id is unique + e.entity_id +}); +``` + +Insert this between the `collect_remembered_entities` call and the sprint anomaly detection block. + +### Fix C: Pin monologue system order (CORRECTION — already done) + +**Round 1 correction:** In my Round 1 analysis, I stated that monologue systems lack explicit ordering. This was **wrong**. After reading `bridge/mod.rs:168-175`, I can see the ordering is already explicit: + +```rust +crate::simulation::monologue::trigger_monologue + .after(crate::simulation::movement::validate_movement), +crate::simulation::monologue::process_sprint_anomaly_monologue + .after(crate::simulation::monologue::trigger_monologue), +``` + +**No change needed.** The `BridgePlugin` already pins both monologue systems with `.after()` constraints relative to `validate_movement` and each other. The SimRng consumption order is deterministic. I apologize for the Round 1 error — I only checked `SimulationPlugin` in `simulation/mod.rs` and missed the ordering in `BridgePlugin`. + +### Fix D: Sort movers in validate_movement + +**Problem:** `validate_movement` in `movement.rs:288` iterates `movers.iter_mut()` — bevy query order is non-deterministic. When two entities move to the same tile in the same tick, which one "wins" depends on iteration order. The existing test (`movement.rs:556-590`) correctly tests that *exactly one* wins, but not *which one*. + +**Fix:** Collect movers into a Vec, sort by Entity bits, then process in order. + +**File: `server/src/simulation/movement.rs`** + +```rust +// Lines 288-316 — BEFORE: +for (entity, intent, mut position, presence) in movers.iter_mut() { + let target = &intent.target; + let layer = presence.copied().unwrap_or_default(); + let slot = (*target, layer); + + if !map.can_move_to(target) { + tracing::trace!("Entity {:?} blocked by terrain at {:?}", entity, target); + } else if occupied.contains_key(&slot) { + // ... + } else { + // ... move logic + } + commands.entity(entity).remove::(); +} + +// AFTER: +// Collect and sort for deterministic processing order (D-010 principle 4). +// Entity::to_bits() provides a stable ordering within a single run. +let mut mover_list: Vec<_> = movers.iter_mut().collect(); +mover_list.sort_by_key(|(entity, _, _, _)| entity.to_bits()); + +for (entity, intent, mut position, presence) in mover_list { + let target = &intent.target; + let layer = presence.copied().unwrap_or_default(); + let slot = (*target, layer); + + if !map.can_move_to(target) { + tracing::trace!("Entity {:?} blocked by terrain at {:?}", entity, target); + } else if occupied.contains_key(&slot) { + tracing::trace!( + "Entity {:?} blocked by entity at {:?} (layer {:?})", + entity, + target, + layer + ); + } else { + tracing::trace!( + "Entity {:?} moving from {:?} to {:?} (layer {:?})", + entity, + *position, + target, + layer + ); + occupied.remove(&(*position, layer)); + *position = *target; + occupied.insert(slot, entity); + } + commands.entity(entity).remove::(); +} +``` + +**Note on `entity.to_bits()`:** This provides stable ordering within a single run (same spawn order → same Entity bits → same processing order). Across save/load cycles, Entity bits may differ, but StableId lookup would be needed. For the Gauntlet (fixed spawn order, no save/load), `to_bits()` is sufficient. If cross-session determinism is needed later, sort by `registry.to_stable(entity).map(|s| s.0)` instead. + +**Test update:** The existing test `validate_movement_two_movers_same_target_first_wins` can now be strengthened to assert WHICH entity wins (the one with the lower Entity bits), making it a determinism regression test. + +--- + +## 2. Server `--test-mode` Flag Design + +**What it does:** + +| Aspect | Behavior | +|--------|----------| +| Content | Loads the Gauntlet content pack (`content/gauntlet/`) instead of campaign content. Falls back to the current hardcoded proof room if Gauntlet content doesn't exist yet. | +| Seed | Fixed seed (42) for deterministic replay. | +| Stdout | Prints `LISTENING:{port}` to stdout after binding the TCP socket, before accepting a connection. This is the signal the test client uses to discover the port. | +| Shutdown | Exits after first client disconnects (existing behavior — `ServerRunning` flag already handles this). | +| Logging | Defaults to `warn` level instead of `debug` to reduce noise in test output. Test can override with `RUST_LOG`. | + +**Minimum change to main.rs:** + +```rust +// server/src/main.rs — proposed changes + +fn main() { + // Parse CLI args + let args: Vec = std::env::args().collect(); + let test_mode = args.iter().any(|a| a == "--test-mode"); + let port = args.iter() + .position(|a| a == "--port") + .and_then(|i| args.get(i + 1)) + .map(|s| s.as_str()); + + // Initialize tracing — quieter in test mode + let default_filter = if test_mode { + "settled_reach_server=warn" + } else { + "settled_reach_server=debug" + }; + tracing_subscriber::registry() + .with( + tracing_subscriber::EnvFilter::try_from_default_env() + .unwrap_or_else(|_| default_filter.into()), + ) + .with(tracing_subscriber::fmt::layer()) + .init(); + + // Resolve address + let addr = if let Some(p) = port { + format!("127.0.0.1:{}", p) + } else { + std::env::args() + .nth(1) + .filter(|a| !a.starts_with("--")) // Don't treat flags as address + .or_else(|| std::env::var("SR_ADDR").ok()) + .unwrap_or_else(|| "127.0.0.1:9876".to_string()) + }; + + // Bind FIRST, print port, THEN accept. + // Splitting bind/accept is critical for --port 0: we need + // the OS-assigned port number before the client connects. + let listener = std::net::TcpListener::bind(&addr).unwrap_or_else(|e| { + eprintln!("Failed to bind {}: {}", addr, e); + std::process::exit(1); + }); + let actual_port = listener.local_addr().unwrap().port(); + + // Print the port signal for test client discovery. + // Flush to ensure the test client reads it before we block on accept(). + println!("LISTENING:{}", actual_port); + use std::io::Write; + std::io::stdout().flush().ok(); + + tracing::info!("Waiting for client connection on port {}", actual_port); + let bridge = TcpBridge::accept_on(listener).unwrap_or_else(|e| { + tracing::error!("Failed to accept: {}", e); + std::process::exit(1); + }); + + // ... rest of main.rs (app setup, game loop) ... + + // Seed: use fixed seed in test mode + let seed = if test_mode { 42 } else { 0 }; + // Replace: .insert_resource(rng::SimRng::new(0)) + // With: passed into SimulationPlugin or inserted directly +} +``` + +**Key design decisions:** +- `TcpBridge::accept_on(listener)` already exists (`tcp.rs:67-97`). This separates bind from accept, which is exactly what we need for port discovery. +- The `LISTENING:{port}` signal is printed to stdout (not stderr, not tracing) so the test client can parse it reliably without tracing noise. +- `--port 0` triggers OS port assignment. The actual port is read from `listener.local_addr()`. +- The first positional argument is currently treated as the address. With `--port`, we switch to flag-based parsing. Both modes coexist — `--port` takes precedence. + +--- + +## 3. `--port 0` Support + +**How it works with the current TCP bridge setup:** + +The current `main.rs:47` calls `TcpBridge::accept(&addr)` which both binds AND accepts in one call. For `--port 0`, we need to: + +1. Bind the listener (to get the OS-assigned port) +2. Print the port +3. Accept on the bound listener + +`TcpBridge::accept_on(listener)` already exists for exactly this purpose (it was added for test race condition prevention — `tcp.rs:67`, "Avoids race conditions in tests by separating bind from accept"). + +**No changes needed to TcpBridge.** The only change is in `main.rs` to call `TcpBridge::accept_on()` instead of `TcpBridge::accept()`. See the code above — the proposed main.rs already uses `accept_on`. + +**Interaction with the test client:** + +``` +Test client Server + | | + | bind("127.0.0.1:0") + | stdout: "LISTENING:54321" + | accept() — blocks + | | + | <-- read stdout, parse port | + | connect("127.0.0.1:54321") ------>| + | accept() returns + | <--- snapshot ---------- | + | --- input -------------> | + | ... | +``` + +--- + +## 4. OQ-2 Answer: Does rmp_serde accept int_16-encoded positive values for u64? + +**Yes.** Verified by tracing through the rmp-serde 1.3.1 source code. + +When `rmp_serde::Deserializer::deserialize_u64()` encounters a `Marker::I16` byte (0xd1), it calls `visitor.visit_i16(value)`. Serde's default `visit_i16` implementation for u64 deserialization performs `u64::try_from(value)`, which succeeds for any non-negative i16 value. + +**Chain of evidence:** +1. `deserialize_u64()` delegates to `any_num()` ([rmp-serde decode.rs](https://github.com/3Hren/msgpack-rust/blob/master/rmp-serde/src/decode.rs)) +2. `any_num()` matches on the marker: `Marker::I16 => visitor.visit_i16(rd.read_data_i16()?)` +3. Serde's `Visitor::visit_i16` for u64 calls `self.visit_i64(v as i64)` +4. Serde's `Visitor::visit_i64` for u64 calls `u64::try_from(v)` — succeeds for positive values + +**This means:** GDScript encoding 256 as int_16 (0xd1, 0x01, 0x00) will deserialize correctly into Rust's `u64` field. The asymmetric encoding between GDScript and Rust (Hoshe's observation in T4-H1) does NOT cause a deserialization failure. + +**However**, negative int_16 values (e.g., -129) will fail `u64::try_from()` and produce a serde error. This is the expected behavior — a negative tick is always a bug. + +**Recommendation:** No code change needed. The existing deserialization handles this correctly. Hoshe's cross-language boundary tests should explicitly cover values 256-32767 deserialized from int_16 into u64 fields to document this guarantee. + +--- + +## 5. OQ-7 Answer: Room Name Lookup + +**Question:** Where does the test client get room names — from Gauntlet coordinate bounds or content metadata? + +**Answer: Gauntlet coordinate bounds, defined as constants in a shared module.** + +The Gauntlet is a fixed-layout map where every room has known coordinate ranges. The test client doesn't need to query the server for room names — it knows the map layout at compile time. + +```rust +// server/src/test_world/constants.rs (also importable by test client binary) + +pub struct GauntletRoom { + pub name: &'static str, + pub bounds: (TilePosition, TilePosition), // top-left, bottom-right + pub observer_position: TilePosition, // fixed position for golden file snapshots +} + +pub const ROOMS: &[GauntletRoom] = &[ + GauntletRoom { + name: "inventory_warehouse", + bounds: (TilePosition::new(0, 0, 0), TilePosition::new(15, 15, 0)), + observer_position: TilePosition::new(8, 8, 0), + }, + GauntletRoom { + name: "occlusion_corridor", + bounds: (TilePosition::new(16, 0, 0), TilePosition::new(31, 15, 0)), + observer_position: TilePosition::new(20, 8, 0), + }, + // ... etc +]; + +/// Look up the room a position falls in. +pub fn room_at(pos: &TilePosition) -> Option<&'static GauntletRoom> { + ROOMS.iter().find(|r| { + pos.x >= r.bounds.0.x && pos.x <= r.bounds.1.x + && pos.y >= r.bounds.0.y && pos.y <= r.bounds.1.y + && pos.z >= r.bounds.0.z && pos.z <= r.bounds.1.z + }) +} +``` + +The test client uses `room_at(player_position)` to label rooms in text output. The server doesn't need a room-naming API — the coordinate bounds are the source of truth. + +**Why not content metadata?** The Gauntlet rooms aren't "locations" in the content system sense — they're mechanical test environments. Content locations (transit district, docking bay) use a different system (`content/locations/`). The Gauntlet constants module is purpose-built for the test world and shouldn't be conflated with game content. + +--- + +## 6. Room Reset Triggers (Anti-Tedium) + +**Use case:** A human tester walks through the Gauntlet, completes the Inventory Warehouse tests, then wants to re-test. Instead of restarting the server, they step on a trigger tile to reset the room to tick-0 state. + +### Server-Side Mechanism + +**Proposal: Reset trigger tiles that restore per-room entity state from a stored snapshot.** + +```rust +// server/src/test_world/reset.rs + +/// Component marking a tile as a room reset trigger. +#[derive(Component)] +pub struct RoomResetTrigger { + pub room_name: &'static str, +} + +/// Stored tick-0 state for a Gauntlet room's entities. +/// Captured immediately after setup, before any simulation ticks. +#[derive(Resource)] +pub struct RoomSnapshots { + /// room_name -> list of (StableId, component bundle snapshot) + snapshots: BTreeMap>, +} + +#[derive(Clone)] +struct EntitySnapshot { + stable_id: StableId, + position: TilePosition, + // All resettable component values captured at tick 0: + knowledge_state: Option, + relationship: Option, + confidence: Option, + inventory_items: Vec<(StableId, String, u8)>, // item_sid, name, slot + // Extend as needed per room type +} +``` + +### Reset Flow + +1. **At Gauntlet setup (tick 0):** After `setup_gauntlet_world()` completes, capture each room's entity state into `RoomSnapshots`. +2. **Trigger detection system:** Each tick, check if the player is on a `RoomResetTrigger` tile. (Simple position check, runs after `validate_movement`.) +3. **Reset execution:** When triggered: + - Move all entities in the room back to their tick-0 positions + - Restore knowledge graph entries to tick-0 state + - Remove all inventory items picked up from the room, re-place them + - Reset the player's KnowledgeGraph entries for entities in this room + - Reset MonologueState cooldowns for the room + +```rust +/// System: detect player stepping on reset trigger tiles. +pub fn detect_room_reset( + player_query: Query<&TilePosition, With>, + triggers: Query<(&TilePosition, &RoomResetTrigger)>, + mut reset_events: EventWriter, +) { + let Ok(player_pos) = player_query.single() else { return }; + for (trigger_pos, trigger) in triggers.iter() { + if player_pos == trigger_pos { + reset_events.send(RoomResetEvent { + room_name: trigger.room_name.to_string(), + }); + } + } +} + +/// System: execute room reset from stored snapshot. +pub fn execute_room_reset( + mut events: EventReader, + snapshots: Res, + registry: Res, + mut positions: Query<&mut TilePosition>, + // ... other component queries for full reset +) { + for event in events.read() { + let Some(room_snapshot) = snapshots.snapshots.get(&event.room_name) else { + tracing::warn!("No snapshot for room {}", event.room_name); + continue; + }; + for entity_snap in room_snapshot { + let Some(entity) = registry.to_entity(&entity_snap.stable_id) else { + continue; + }; + if let Ok(mut pos) = positions.get_mut(entity) { + *pos = entity_snap.position; + } + // Restore other components... + } + tracing::info!("Room {} reset to tick-0 state", event.room_name); + } +} +``` + +### ECS Implications + +1. **Despawned entities:** If the tester picked up an item (which removes TilePosition and adds CarriedBy), the reset must reverse this — remove CarriedBy/InventorySlot, re-add TilePosition. This is straightforward with `Commands`. + +2. **Knowledge graph:** The player's KG entries for room entities need selective reset. This requires knowing which StableIds belong to which room — the `GauntletRoom` constants provide this mapping. + +3. **No new entity spawning:** Reset restores existing entities, it doesn't despawn/respawn. This avoids Entity ID recycling issues and keeps the EntityRegistry stable. + +4. **Trigger debounce:** The reset trigger should have a cooldown (e.g., 10 ticks) to prevent re-triggering while the player walks across the trigger tile. + +5. **Test mode only:** `RoomResetTrigger` entities and the reset systems are only added in `--test-mode`. They don't exist in production. + +--- + +## 7. Cross-Review: Hoshe's Layer 3 Test + +**Hoshe's proposal** (from `hoshe-round1.md`, T4-H5): Launch the server binary as a child process with `--test-mode --port 0`, connect via TCP, send one input, receive one snapshot, assert fields. + +### Feasibility Assessment + +**Verdict: Feasible with the server flag changes from section 2. No architectural blockers.** + +Detailed review: + +| Aspect | Hoshe's Proposal | Feasibility | Notes | +|--------|-----------------|-------------|-------| +| Build server binary | `cargo build --bin settled-reach-server` | Works | Binary name is correct per `Cargo.toml` | +| Launch as subprocess | `Command::new("target/debug/settled-reach-server")` | Works | Standard Rust subprocess API | +| `--test-mode` flag | Args: `["--test-mode", "--port", "0"]` | Needs implementation | See section 2 above | +| Port discovery | Parse `LISTENING:{port}` from stdout | Works with section 2 changes | Current main.rs doesn't print this — section 2 adds it | +| TCP connect | `TcpStream::connect(format!("127.0.0.1:{}", port))` | Works | Standard TCP | +| Send PlayerInput | `write_framed` with `rmp_serde::to_vec_named` | Works | But note: Hoshe serializes `Vec`, which is correct — the bridge expects a Vec, not a single input | +| Receive snapshot | `read_framed` → `rmp_serde::from_slice::` | Works | The bridge sends snapshots as single ObserverSnapshot, not Vec | +| Assertions | version, tick, entity count, player entity | All valid | Snapshot at tick 0 will have the proof room entities | +| Cleanup | Drop connection, kill process | Works | `server.kill().ok()` is correct — graceful shutdown via `ServerRunning` flag also works when TCP disconnects | + +### Issues Found + +1. **Timing sensitivity.** Hoshe's test sends one input and expects one snapshot. But the server's game loop runs at 20 tps with `thread::sleep(50ms)`. After accepting the connection, the server enters the game loop. The first `app.update()` calls `receive_bridge_inputs` which reads the TCP socket (non-blocking). If the test client hasn't sent its input yet, the server produces a snapshot with tick=0 and no input processed. The client may receive this "empty" snapshot before its input is processed. + + **Fix:** The test should loop, reading snapshots until it gets one where its input has been processed (e.g., `snapshot.tick >= 1`), or accept that tick=0 snapshots are valid (they contain the player entity and visible NPCs even without input). + + Hoshe's assertion `snapshot.tick == 0` is actually fine for the first snapshot. The test doesn't need to verify input processing — it just needs to verify the snapshot arrives. Input processing is tested at Layer 2. + +2. **Serialization format mismatch.** Hoshe writes `rmp_serde::to_vec_named(&inputs)` but the bridge receives with `rmp_serde::from_slice::>`. The `to_vec_named` function produces MessagePack maps (with field names), while `to_vec` produces arrays. Since `PlayerInput` derives `Deserialize`, `from_slice` handles both formats — but the GDScript client sends `to_vec` (array) format. For consistency, the test should use `rmp_serde::to_vec(&inputs)` to match the real client's encoding. + +3. **read_framed import.** Hoshe references `read_framed` and `write_framed` from the bridge module. These are in `server/src/bridge/framing.rs` and take `&mut impl Read`/`&mut impl Write`. The test will need to import them via `settled_reach_server::bridge::framing::{read_framed, write_framed}` — this requires the bridge framing module to be public. Currently, `framing` is `pub mod framing` in `bridge/mod.rs:9`, so it's accessible. + +### Summary + +Hoshe's Layer 3 test design is sound and feasible once the `--test-mode` and `--port 0` server flags are implemented. The test correctly identifies what Layer 3 catches that Layer 2 doesn't (subprocess I/O blocking, binary startup issues). Two minor adjustments recommended: (1) use `to_vec` instead of `to_vec_named` for input serialization, (2) tolerate receiving an "initial" snapshot before the input is processed. + +--- + +## Summary of Round 2 Deliverables + +| # | Deliverable | Status | +|---|-------------|--------| +| 1 | 4 determinism fixes with exact code changes | Complete (Fix A-D; Fix C was already done) | +| 2 | `--test-mode` flag design + main.rs changes | Complete | +| 3 | `--port 0` support design | Complete (uses existing `accept_on`) | +| 4 | OQ-2: rmp_serde int_16→u64 | Answered: Yes, works correctly | +| 5 | OQ-7: Room name lookup | Answered: Gauntlet coordinate constants | +| 6 | Room reset trigger mechanism | Complete | +| 7 | Cross-review of Hoshe's Layer 3 test | Complete — feasible with 2 minor adjustments | diff --git a/docs/workshops/test-architecture/dudley-round3.md b/docs/workshops/test-architecture/dudley-round3.md new file mode 100644 index 000000000..fe341b31b --- /dev/null +++ b/docs/workshops/test-architecture/dudley-round3.md @@ -0,0 +1,1075 @@ +# Dudley — Round 3: Final Server Specs & Sprint 8 Deliverables + +**Workshop:** QA Strategy & Test Architecture +**Round:** 3 (Prioritization & Final Specs) +**Date:** 2026-02-17 +**Inputs:** All Round 2 outputs, round-2-notes.md, live codebase + +--- + +## 1. Determinism Fixes — Final Patch Spec + +Three fixes (A, B, D). Fix C was already done (corrected in Round 2). Each fix includes exact code, the affected lines, and a focused regression test. + +### Fix A: Deterministic visible_tiles ordering + BTreeSet for visible_positions + +**Files:** `server/src/perception/query.rs`, `server/src/perception/observer/mod.rs` + +**Problem:** `VisibilityGeometry.visible_positions` is `HashSet<(i32,i32)>` (line 24, query.rs). `visible_tiles` inherits non-deterministic ordering from `apply_vision_cone` which iterates the shadowcast's `HashSet`. The `visible_ids` in `filter_visible_entities` is also `HashSet`. These feed into the ObserverSnapshot and affect sprint anomaly detection order. + +#### Patch 1a: `server/src/perception/query.rs` + +```rust +// Line 8 — BEFORE: +use std::collections::{HashMap, HashSet}; + +// AFTER: +use std::collections::{BTreeSet, HashMap}; +``` + +```rust +// Line 24 — BEFORE: + pub visible_positions: HashSet<(i32, i32)>, + +// AFTER: + pub visible_positions: BTreeSet<(i32, i32)>, +``` + +```rust +// Lines 67-83 (inside NaturalVision::compute_geometry) — BEFORE: + let visible_tiles = cone_tiles + .iter() + .map(|&(x, y, sector)| { + let tile_kind = if walkability.can_move_to(&TilePosition::new(x, y, z)) { + TileKind::Floor + } else { + TileKind::Wall + }; + VisibleTile { + x, + y, + z, + visibility: sector, + tile_kind, + } + }) + .collect(); + + let visible_positions = cone_tiles.iter().map(|&(x, y, _)| (x, y)).collect(); + +// AFTER: + let mut visible_tiles: Vec<_> = cone_tiles + .iter() + .map(|&(x, y, sector)| { + let tile_kind = if walkability.can_move_to(&TilePosition::new(x, y, z)) { + TileKind::Floor + } else { + TileKind::Wall + }; + VisibleTile { + x, + y, + z, + visibility: sector, + tile_kind, + } + }) + .collect(); + // Deterministic tile ordering for snapshot stability (D-010 principle 4) + visible_tiles.sort_by_key(|t| (t.x, t.y)); + + let visible_positions: BTreeSet<(i32, i32)> = + cone_tiles.iter().map(|&(x, y, _)| (x, y)).collect(); +``` + +**Note:** `sector_lookup: HashMap<(i32,i32), VisibilitySector>` on line 25 stays as HashMap — point-lookup only via `.get()`, never iterated. Confirmed safe per Tyre OQ-4 answer. + +#### Patch 1b: `server/src/perception/observer/mod.rs` + +```rust +// Line 10 — BEFORE: +use std::collections::HashSet; + +// AFTER: +use std::collections::BTreeSet; +``` + +```rust +// Line 217 — BEFORE: +) -> (Vec, HashSet) { + let mut entities = Vec::new(); + let mut visible_ids: HashSet = HashSet::new(); + +// AFTER: +) -> (Vec, BTreeSet) { + let mut entities = Vec::new(); + let mut visible_ids: BTreeSet = BTreeSet::new(); +``` + +```rust +// Line 284-285 — BEFORE: +fn collect_remembered_entities( + observer_kg: &KnowledgeGraph, + visible_ids: &HashSet, + visible_positions: &HashSet<(i32, i32)>, + +// AFTER: +fn collect_remembered_entities( + observer_kg: &KnowledgeGraph, + visible_ids: &BTreeSet, + visible_positions: &BTreeSet<(i32, i32)>, +``` + +#### Fix A Regression Test + +```rust +// server/src/perception/observer/tests.rs (or new test file) + +/// Verify visible_tiles in snapshot are sorted by (x, y). +/// Regression test for determinism Fix A. +#[test] +fn snapshot_visible_tiles_are_sorted() { + // Setup: create world with WalkabilityMap, player, geometry + let mut world = bevy_ecs::world::World::new(); + // ... (standard test world setup with WalkabilityMap, player, geometry) ... + + // After compute_observer_snapshot: + let snapshot = world.resource::().snapshot.as_ref().unwrap(); + for window in snapshot.visible_tiles.windows(2) { + assert!( + (window[0].x, window[0].y) <= (window[1].x, window[1].y), + "visible_tiles not sorted: ({},{}) > ({},{})", + window[0].x, window[0].y, window[1].x, window[1].y, + ); + } +} + +/// Verify sprint anomaly detection picks deterministic entity +/// when multiple Contradicted entities are equidistant. +#[test] +fn sprint_anomaly_picks_lowest_stable_id() { + // Setup: player sprinting, 2 NPCs both Contradicted in KG, + // both visible. BTreeSet iterates in ascending order, + // so the NPC with the lower StableId wins. + // Assert: anomaly_queue contains the lower-ID NPC. +} +``` + +--- + +### Fix B: Sort visible entities in snapshot by entity_id + +**File:** `server/src/perception/observer/mod.rs` + +**Problem:** `filter_visible_entities` builds `entities: Vec` by iterating `all_entities.iter()` (line 221). Bevy query iteration order is archetype-based and not stable across runs. The entity list in ObserverSnapshot is therefore non-deterministic. + +#### Patch 2: `server/src/perception/observer/mod.rs` + +```rust +// After line 126 (after collect_remembered_entities call, before sprint anomaly block) +// In compute_observer_snapshot, INSERT: + + // Sort entities by wire ID for deterministic snapshot ordering (D-010 principle 4) + entities.sort_by_key(|e| e.entity_id); +``` + +Exact insertion point: between the `collect_remembered_entities(...)` call (line 119-126) and the sprint anomaly detection block (line 128 `if stance_opt.map(...)`). + +#### Fix B Regression Test + +```rust +/// Verify entities in snapshot are sorted by entity_id. +/// Regression test for determinism Fix B. +#[test] +fn snapshot_entities_sorted_by_id() { + // Setup: 3+ NPCs visible to player + // After compute_observer_snapshot: + let snapshot = world.resource::().snapshot.as_ref().unwrap(); + for window in snapshot.entities.windows(2) { + assert!( + window[0].entity_id <= window[1].entity_id, + "entities not sorted: {} > {}", + window[0].entity_id, window[1].entity_id, + ); + } +} +``` + +--- + +### Fix D: Sort movers in validate_movement for deterministic collision resolution + +**File:** `server/src/simulation/movement.rs` + +**Problem:** `validate_movement` iterates `movers.iter_mut()` (line 288). When two entities move to the same tile in the same tick, the "first-come-first-served" collision resolution depends on bevy query iteration order, which is non-deterministic. + +#### Patch 3: `server/src/simulation/movement.rs` + +```rust +// Lines 288-316 — BEFORE: + for (entity, intent, mut position, presence) in movers.iter_mut() { + let target = &intent.target; + let layer = presence.copied().unwrap_or_default(); + let slot = (*target, layer); + + if !map.can_move_to(target) { + tracing::trace!("Entity {:?} blocked by terrain at {:?}", entity, target); + } else if occupied.contains_key(&slot) { + tracing::trace!( + "Entity {:?} blocked by entity at {:?} (layer {:?})", + entity, + target, + layer + ); + } else { + tracing::trace!( + "Entity {:?} moving from {:?} to {:?} (layer {:?})", + entity, + *position, + target, + layer + ); + // Free old layer slot, claim new one + occupied.remove(&(*position, layer)); + *position = *target; + occupied.insert(slot, entity); + } + commands.entity(entity).remove::(); + } + +// AFTER: + // Collect and sort movers for deterministic collision resolution (D-010 principle 4). + // Entity::to_bits() provides stable ordering within a single run. + // For cross-session determinism (save/load), use registry.to_stable() instead. + let mut mover_list: Vec<_> = movers.iter_mut().collect(); + mover_list.sort_by_key(|(entity, _, _, _)| entity.to_bits()); + + for (entity, intent, mut position, presence) in mover_list { + let target = &intent.target; + let layer = presence.copied().unwrap_or_default(); + let slot = (*target, layer); + + if !map.can_move_to(target) { + tracing::trace!("Entity {:?} blocked by terrain at {:?}", entity, target); + } else if occupied.contains_key(&slot) { + tracing::trace!( + "Entity {:?} blocked by entity at {:?} (layer {:?})", + entity, + target, + layer + ); + } else { + tracing::trace!( + "Entity {:?} moving from {:?} to {:?} (layer {:?})", + entity, + *position, + target, + layer + ); + occupied.remove(&(*position, layer)); + *position = *target; + occupied.insert(slot, entity); + } + commands.entity(entity).remove::(); + } +``` + +#### Fix D Regression Test + +```rust +/// Verify that same-tile collision resolution is deterministic. +/// The entity with the lower Entity::to_bits() value wins. +/// Regression test for determinism Fix D. +#[test] +fn validate_movement_deterministic_collision_winner() { + let mut world = bevy_ecs::world::World::new(); + world.insert_resource(WalkabilityMap::new(10, 10, 1)); + + let entity_a = world + .spawn(( + TilePosition::new(5, 4, 0), + MoveIntent { + target: TilePosition::new(5, 5, 0), + }, + )) + .id(); + + let entity_b = world + .spawn(( + TilePosition::new(5, 6, 0), + MoveIntent { + target: TilePosition::new(5, 5, 0), + }, + )) + .id(); + + let mut schedule = bevy_ecs::schedule::Schedule::default(); + schedule.add_systems(validate_movement); + schedule.run(&mut world); + + let pos_a = *world.get::(entity_a).unwrap(); + let pos_b = *world.get::(entity_b).unwrap(); + + // The entity with lower to_bits() should win the tile + let expected_winner = if entity_a.to_bits() < entity_b.to_bits() { + entity_a + } else { + entity_b + }; + let expected_loser = if expected_winner == entity_a { + entity_b + } else { + entity_a + }; + + assert_eq!( + *world.get::(expected_winner).unwrap(), + TilePosition::new(5, 5, 0), + "lower-bits entity should win the contested tile" + ); + assert_ne!( + *world.get::(expected_loser).unwrap(), + TilePosition::new(5, 5, 0), + "higher-bits entity should remain at original position" + ); +} +``` + +**Strengthens existing test:** The current `validate_movement_two_movers_same_target_first_wins` (line 556) asserts "exactly one wins" but not which one. After Fix D, the winner is deterministic — the entity with lower `Entity::to_bits()` wins. The new test above replaces that weaker assertion. + +--- + +### Summary: All Determinism Fixes + +| Fix | File(s) | Lines changed | Regression test | +|-----|---------|---------------|-----------------| +| A | `query.rs`, `observer/mod.rs` | ~15 | `snapshot_visible_tiles_are_sorted`, `sprint_anomaly_picks_lowest_stable_id` | +| B | `observer/mod.rs` | 2 | `snapshot_entities_sorted_by_id` | +| C | (none — already done) | 0 | — | +| D | `movement.rs` | ~5 | `validate_movement_deterministic_collision_winner` | + +**Total: ~22 lines of production code, 4 regression tests.** + +--- + +## 2. Server `--test-mode` Final Spec + +### CLI Interface + +``` +settled-reach-server [OPTIONS] [ADDRESS] + +Options: + --test-mode Enable test mode (Gauntlet content, fixed seed, LISTENING signal) + --port Bind to specific port (0 = OS-assigned). Overrides ADDRESS. + --seed RNG seed (default: 0, test-mode default: 42) + +Legacy: + ADDRESS First positional arg (e.g., "127.0.0.1:9876"). Overridden by --port. + SR_ADDR Env var fallback. Overridden by --port and ADDRESS. + Default: 127.0.0.1:9876 +``` + +### Complete `main.rs` Replacement + +```rust +// server/src/main.rs — Sprint 8 version with --test-mode + --port support + +use bevy_app::prelude::*; +use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt}; + +use settled_reach_server::bridge::tcp::TcpBridge; +use settled_reach_server::bridge::{BridgePlugin, BridgeResource, ServerRunning}; +use settled_reach_server::knowledge::KnowledgePlugin; +use settled_reach_server::npc::NpcPlugin; +use settled_reach_server::simulation::SimulationPlugin; + +fn main() { + let args: Vec = std::env::args().collect(); + let test_mode = args.iter().any(|a| a == "--test-mode"); + let port_flag = args + .iter() + .position(|a| a == "--port") + .and_then(|i| args.get(i + 1)) + .and_then(|s| s.parse::().ok()); + let seed_flag = args + .iter() + .position(|a| a == "--seed") + .and_then(|i| args.get(i + 1)) + .and_then(|s| s.parse::().ok()); + + // Tracing: quieter in test mode to reduce stdout noise + let default_filter = if test_mode { + "settled_reach_server=warn" + } else { + "settled_reach_server=debug" + }; + tracing_subscriber::registry() + .with( + tracing_subscriber::EnvFilter::try_from_default_env() + .unwrap_or_else(|_| default_filter.into()), + ) + .with(tracing_subscriber::fmt::layer().with_writer(std::io::stderr)) + .init(); + + // Resolve bind address + let addr = if let Some(port) = port_flag { + format!("127.0.0.1:{}", port) + } else { + args.iter() + .skip(1) + .find(|a| !a.starts_with("--")) + .cloned() + .or_else(|| std::env::var("SR_ADDR").ok()) + .unwrap_or_else(|| "127.0.0.1:9876".to_string()) + }; + + // Bind FIRST, print port, THEN accept. + // Critical for --port 0: the OS assigns a random port at bind time. + // The LISTENING:{port} line is the handshake signal for the test client. + let listener = std::net::TcpListener::bind(&addr).unwrap_or_else(|e| { + eprintln!("Failed to bind {}: {}", addr, e); + std::process::exit(1); + }); + let actual_port = listener.local_addr().unwrap().port(); + + // LISTENING signal to stdout. The test client parses this to discover the port. + // All tracing goes to stderr (see .with_writer above), so stdout is clean. + println!("LISTENING:{}", actual_port); + use std::io::Write; + std::io::stdout().flush().ok(); + + tracing::info!("Waiting for client connection on port {}", actual_port); + let bridge = TcpBridge::accept_on(listener).unwrap_or_else(|e| { + tracing::error!("Failed to accept: {}", e); + std::process::exit(1); + }); + tracing::info!("Client connected, initializing simulation"); + + // RNG seed: test-mode defaults to 42 for deterministic replay + let seed = seed_flag.unwrap_or(if test_mode { 42 } else { 0 }); + + let mut app = App::new(); + app.add_plugins(SimulationPlugin); + app.add_plugins(BridgePlugin); + app.add_plugins(KnowledgePlugin); + app.add_plugins(NpcPlugin); + app.add_plugins(settled_reach_server::content::ContentPlugin); + app.insert_resource(BridgeResource::new(bridge)); + + if test_mode { + // Gauntlet content: deferred until Gauntlet loader exists. + // For now, fall back to the proof room setup. + setup_proof_room(&mut app, seed); + } else { + setup_proof_room(&mut app, seed); + } + + tracing::info!("Simulation initialized (seed={}, test_mode={})", seed, test_mode); + + let target_frame_time = std::time::Duration::from_millis(50); + loop { + let frame_start = std::time::Instant::now(); + app.update(); + if !app.world().resource::().0 { + break; + } + let elapsed = frame_start.elapsed(); + if elapsed < target_frame_time { + std::thread::sleep(target_frame_time - elapsed); + } + } + + tracing::info!("Simulation server shutting down"); +} + +/// Proof room: 32x32 map, wall at (16,14), player at (16,16), 3 NPCs. +/// Extracted from current inline setup for reuse by both test-mode and normal mode. +fn setup_proof_room(app: &mut App, seed: u64) { + use settled_reach_server::knowledge::registry::EntityRegistry; + use settled_reach_server::knowledge::KnowledgeGraph; + use settled_reach_server::npc::relationships::{RelationshipEdge, RelationshipGraph}; + use settled_reach_server::npc::*; + use settled_reach_server::perception::cognitive_delay::CognitiveDelay; + use settled_reach_server::perception::vision_cone::Facing; + use settled_reach_server::simulation::interaction::{Interactable, NearbyInteractionBuffer}; + use settled_reach_server::simulation::listening::ListeningFocus; + use settled_reach_server::simulation::monologue::{ + MonologueBuffer, MonologueState, SprintAnomalyQueue, + }; + use settled_reach_server::simulation::movement::{ + PlayerCharacter, TilePosition, WalkabilityMap, + }; + use settled_reach_server::simulation::path_follow::MovementSpeed; + use settled_reach_server::simulation::stance::{MovementProfile, PlayerMoveCooldown}; + use settled_reach_server::simulation::time::DayPhase; + + app.insert_resource(WalkabilityMap::new(32, 32, 1)); + { + let mut wm = app.world_mut().resource_mut::(); + wm.set_walkable(&TilePosition::new(16, 14, 0), false); + } + + let mut registry = EntityRegistry::new(0); + // ... (existing player + 3 NPC spawn code, unchanged from current main.rs lines 72-225) + // Omitted here for brevity — the extraction is mechanical. + // The seed parameter will be passed to SimRng::new(seed) once that resource + // is inserted by SimulationPlugin (currently hardcoded to 0 in SimulationPlugin). + + app.insert_resource(registry); +} +``` + +### Key Design Decisions + +| Decision | Rationale | +|----------|-----------| +| Tracing to stderr, LISTENING to stdout | Test client parses stdout cleanly without tracing noise | +| `accept_on(listener)` instead of `accept(&addr)` | Already exists at `tcp.rs:67`. Separates bind from accept for port discovery | +| `--port` overrides positional arg | Flag-based parsing is unambiguous. Positional arg preserved for backward compat | +| `setup_proof_room()` extracted | Reusable by both modes. Gauntlet content loads into the same slot when ready | +| `--seed` flag exposed | Test client can verify determinism with different seeds | + +### Test Client Port Discovery Protocol + +``` +Server stdout: LISTENING:54321\n +Test client: parse port → connect("127.0.0.1:54321") +``` + +The test client reads lines from the server's stdout pipe until it sees `LISTENING:(\d+)`. Timeout: 5 seconds. If no LISTENING line arrives, the test fails with the server's stderr output for debugging. + +--- + +## 3. Gauntlet Loader MVP Spec (Sprint 8 Scope) + +### Sprint 8 Rooms: 5 of 14 + +Based on Gestalt's priority ranking (INV-T04, INV-T01, INV-T03, INV-T05) and the systems we can test NOW: + +| # | Room | Why Sprint 8 | Systems tested | +|---|------|-------------|----------------| +| 1 | **Hub** | Central teleport target, room naming, corridor flow | Room naming, coordinates, basic spawning | +| 2 | **Pause Chamber** | Bug #3 regression, INV-T04 (rank #1) | Pause guard, state transitions, TickRate | +| 3 | **Inventory Warehouse** | D-065 Take/Place, 9-slot limit | Inventory, pickup, CarriedBy, slot assignment | +| 4 | **Occlusion Corridor** | D-035 LOS, D-015 vision cone, shadowcasting | Visibility, perception, fog | +| 5 | **Interaction Gallery** | D-057/D-060 verb system, sprint suppression | Phase 1/2 verbs, ObjectType, interaction buffer | + +**Deferred to Sprint 9:** Crowd Plaza (needs cognitive delay visual), Fog Theater (needs 5-layer fog), Dialogue Room (needs dialogue dispatch), Sound Lab/Eavesdrop/Confrontation (needs audio systems), Decay Observatory (needs decay system), Sprint Gauntlet (needs anomaly monologue content), Shift Change (needs density stress content), Zone Gate (reserved). + +### Content Pack Structure + +``` +content/gauntlet/ + gauntlet.yaml # Master file: rooms, spawn order, map dimensions + rooms/ + hub.yaml # Hub room: layout, reset plate, corridor exits + pause_chamber.yaml # Pause room: entities, tick rate test scenarios + inventory_warehouse.yaml # Inventory room: 10 pickup items, crates + occlusion_corridor.yaml # Occlusion room: walls, NPCs at LOS boundaries + interaction_gallery.yaml # Interaction room: one per ObjectType + multi-verb NPC +``` + +### Master File: `gauntlet.yaml` + +```yaml +# content/gauntlet/gauntlet.yaml +# Gauntlet test world — Sprint 8 MVP +# Additive only: existing rooms never modified. New rooms appended. + +name: gauntlet +version: 1 +seed: 42 +map_dimensions: + width: 128 + height: 128 + z_levels: 1 + +# Canonical room ordering — determines entity spawn order → StableId assignment. +# DO NOT REORDER existing rooms. Append new rooms at the end. +rooms: + - hub + - pause_chamber + - inventory_warehouse + - occlusion_corridor + - interaction_gallery +``` + +### Room File: `pause_chamber.yaml` + +```yaml +# content/gauntlet/rooms/pause_chamber.yaml +# Tests: TickRate toggle, pause guard (Bug #3), state transitions + +room_id: pause_chamber +bounds: + top_left: [32, 0] + bottom_right: [47, 15] +observer_position: [40, 8] # Fixed position for golden file snapshots + +walls: + # Perimeter walls (bounds are inclusive) + - type: perimeter + +entities: + - id: pause_npc_1 + kind: npc + position: [40, 6] + interactable: true + components: + want: { primary: Safety, intensity: 3 } + contentment: 10 + tolerance: { stress: 10, threshold: 60 } + + - id: pause_crate_1 + kind: object + position: [38, 8] + object_type: Container + interactable: true + +reset_plate: [40, 15] # Bottom edge of room +``` + +### Room File: `inventory_warehouse.yaml` + +```yaml +# content/gauntlet/rooms/inventory_warehouse.yaml +# Tests: Pickup, CarriedBy, 9-slot limit, Take/Place verbs + +room_id: inventory_warehouse +bounds: + top_left: [48, 0] + bottom_right: [63, 15] +observer_position: [56, 8] + +walls: + - type: perimeter + +entities: + # 10 pickup items — first 9 fit, 10th tests overflow rejection + - id: inv_item_1 + kind: object + position: [50, 4] + object_type: Pickup + item_name: "Manifest Alpha" + interactable: true + - id: inv_item_2 + kind: object + position: [52, 4] + object_type: Pickup + item_name: "Docking Token" + interactable: true + - id: inv_item_3 + kind: object + position: [54, 4] + object_type: Pickup + item_name: "Sensor Array" + interactable: true + - id: inv_item_4 + kind: object + position: [56, 4] + object_type: Pickup + item_name: "Cargo Key" + interactable: true + - id: inv_item_5 + kind: object + position: [58, 4] + object_type: Pickup + item_name: "Power Cell" + interactable: true + - id: inv_item_6 + kind: object + position: [50, 8] + object_type: Pickup + item_name: "Data Chip" + interactable: true + - id: inv_item_7 + kind: object + position: [52, 8] + object_type: Pickup + item_name: "Repair Kit" + interactable: true + - id: inv_item_8 + kind: object + position: [54, 8] + object_type: Pickup + item_name: "Transit Pass" + interactable: true + - id: inv_item_9 + kind: object + position: [56, 8] + object_type: Pickup + item_name: "Field Journal" + interactable: true + - id: inv_item_10 + kind: object + position: [58, 8] + object_type: Pickup + item_name: "Overflow Item" + interactable: true + + # Non-pickup objects for mixed interaction testing + - id: inv_crate_1 + kind: object + position: [50, 12] + object_type: Container + interactable: true + +reset_plate: [56, 15] +``` + +### Gauntlet Constants Module + +```rust +// server/src/test_world/constants.rs + +use crate::simulation::movement::TilePosition; + +pub struct GauntletRoom { + pub name: &'static str, + pub bounds: (TilePosition, TilePosition), + pub observer_position: TilePosition, +} + +pub struct GauntletEntity { + pub name: &'static str, + pub room: &'static str, + /// Wire ID (StableId.0) assigned by deterministic spawn order. + /// Player = 0, then entities in room order (gauntlet.yaml rooms[]), + /// within each room in entity order (room.yaml entities[]). + pub wire_id: u64, + pub position: TilePosition, +} + +// --- Rooms --- + +pub const HUB: GauntletRoom = GauntletRoom { + name: "hub", + bounds: (TilePosition::new(0, 0, 0), TilePosition::new(31, 31, 0)), + observer_position: TilePosition::new(16, 16, 0), +}; + +pub const PAUSE_CHAMBER: GauntletRoom = GauntletRoom { + name: "pause_chamber", + bounds: (TilePosition::new(32, 0, 0), TilePosition::new(47, 15, 0)), + observer_position: TilePosition::new(40, 8, 0), +}; + +pub const INVENTORY_WAREHOUSE: GauntletRoom = GauntletRoom { + name: "inventory_warehouse", + bounds: (TilePosition::new(48, 0, 0), TilePosition::new(63, 15, 0)), + observer_position: TilePosition::new(56, 8, 0), +}; + +pub const OCCLUSION_CORRIDOR: GauntletRoom = GauntletRoom { + name: "occlusion_corridor", + bounds: (TilePosition::new(64, 0, 0), TilePosition::new(79, 15, 0)), + observer_position: TilePosition::new(72, 8, 0), +}; + +pub const INTERACTION_GALLERY: GauntletRoom = GauntletRoom { + name: "interaction_gallery", + bounds: (TilePosition::new(80, 0, 0), TilePosition::new(95, 15, 0)), + observer_position: TilePosition::new(88, 8, 0), +}; + +pub const ROOMS: &[&GauntletRoom] = &[ + &HUB, + &PAUSE_CHAMBER, + &INVENTORY_WAREHOUSE, + &OCCLUSION_CORRIDOR, + &INTERACTION_GALLERY, +]; + +/// Look up the room a position falls in. +pub fn room_at(pos: &TilePosition) -> Option<&'static GauntletRoom> { + ROOMS.iter().find(|r| { + pos.x >= r.bounds.0.x && pos.x <= r.bounds.1.x + && pos.y >= r.bounds.0.y && pos.y <= r.bounds.1.y + && pos.z >= r.bounds.0.z && pos.z <= r.bounds.1.z + }).copied() +} + +// --- Entities (wire IDs assigned by spawn order) --- +// Player is always wire_id 0. +// Room entities follow in gauntlet.yaml room order × room.yaml entity order. + +pub const PLAYER: GauntletEntity = GauntletEntity { + name: "player", + room: "hub", + wire_id: 0, + position: TilePosition::new(16, 16, 0), +}; + +pub const PAUSE_NPC_1: GauntletEntity = GauntletEntity { + name: "pause_npc_1", + room: "pause_chamber", + wire_id: 1, // First entity after player + position: TilePosition::new(40, 6, 0), +}; + +// ... etc for all entities, assigned sequentially +``` + +### Gauntlet Loader System + +```rust +// server/src/test_world/loader.rs + +/// Load the Gauntlet test world from YAML content pack. +/// Called when --test-mode is active and content/gauntlet/ exists. +/// +/// Spawn order guarantees: +/// 1. Player entity (always StableId 0) +/// 2. Room entities in gauntlet.yaml rooms[] order +/// 3. Within each room, entities in room.yaml entities[] order +/// +/// This order is the determinism contract for StableId assignment. +pub fn setup_gauntlet_world(app: &mut App, seed: u64) { + // 1. Load gauntlet.yaml → room list, map dimensions + // 2. Create WalkabilityMap(width, height, z_levels) + // 3. Spawn player entity at HUB.observer_position + // 4. For each room in canonical order: + // a. Load room.yaml + // b. Apply walls to WalkabilityMap + // c. Spawn entities in listed order + // d. Register each entity in EntityRegistry + // e. If --test-mode: spawn RoomResetTrigger at reset_plate position + // 5. Insert RoomSnapshots resource (capture tick-0 state per room) + // 6. Insert SimRng::new(seed) +} +``` + +**Implementation note:** The loader should be a Rust function that reads YAML via `serde_yaml`, not a bevy plugin startup system. This keeps it synchronous and testable — the function takes `&mut App` and populates it before the game loop starts. The YAML structure above maps directly to the ECS components we already have. + +--- + +## 4. Remaining Questions Answered + +### Q7: SetTickRate while paused — reject or unpause? + +**Question (Hoshe R2-OQ-01):** `SetTickRate(Half)` while paused — should this unpause? Current code at `input.rs:165-168` sets the rate unconditionally, meaning `paused()` returns false on next tick. + +**Answer: This is a bug. SetTickRate should be rejected while paused.** + +The current behavior allows a client to bypass the pause guard by sending `SetTickRate(Half)` instead of `Unpause`. The pause state should only be exited by an explicit `Unpause` action. + +**Fix:** + +```rust +// server/src/simulation/input.rs, line 165-168 — BEFORE: + PlayerAction::SetTickRate(rate) => { + time.tick_rate = rate; + tracing::debug!("Tick rate set to {:?} by player input", rate); + } + +// AFTER: + PlayerAction::SetTickRate(rate) => { + if paused { + tracing::debug!("SetTickRate({:?}) rejected while paused", rate); + } else { + time.tick_rate = rate; + tracing::debug!("Tick rate set to {:?} by player input", rate); + } + } +``` + +**Rationale:** `Pause` is a player-initiated state lock. The only valid exit is `Unpause`. `SetTickRate` is a tick-speed adjustment for gameplay pacing (Full/Half), not a pause override. If the player wants to go from Paused to Half, they send `Unpause` then `SetTickRate(Half)`. + +**Test:** + +```rust +#[test] +fn set_tick_rate_rejected_while_paused() { + let mut world = bevy_ecs::world::World::new(); + world.insert_resource(InputQueue::default()); + world.insert_resource(SimulationTime { + tick_rate: TickRate::Paused, + ..Default::default() + }); + world.init_resource::(); + world.spawn((PlayerCharacter, TilePosition::new(5, 5, 0))); + + world.resource_mut::().push(PlayerInput { + tick: 0, + action: PlayerAction::SetTickRate(TickRate::Half), + }); + + let mut schedule = bevy_ecs::schedule::Schedule::default(); + schedule.add_systems(process_player_input); + schedule.run(&mut world); + + assert_eq!( + world.resource::().tick_rate, + TickRate::Paused, + "SetTickRate should not override Paused state" + ); +} +``` + +--- + +### Q8: Entity index recycling — how does registry handle despawn+respawn? + +**Question (Hoshe R2-OQ-02):** When bevy recycles an Entity index, does the registry handle old StableId not being unregistered? + +**Answer: The current registry is safe because bevy Entity includes a generation counter.** + +In bevy_ecs, `Entity` is a combination of `(index, generation)`. When an entity is despawned, the index is recycled but the generation increments. So `Entity(index=3, gen=1)` and `Entity(index=3, gen=2)` are different keys in the BTreeMap. The registry uses `Entity` as the BTreeMap key, not the raw index. + +**However**, there IS a correctness concern: if an entity is despawned but `unregister()` is never called, the old `(Entity(3,gen1) → StableId(5))` mapping persists. When bevy respawns `Entity(3,gen2)` and we call `register(Entity(3,gen2))`, it correctly gets a NEW StableId (since `Entity(3,gen2)` is a different key). But the OLD mapping `StableId(5) → Entity(3,gen1)` still exists, pointing to a dead entity. + +**The fix is discipline, not code change:** `unregister()` MUST be called on despawn. The Gauntlet's room reset system avoids despawn entirely (it resets components, not entities), so this is not a Sprint 8 risk. For Sprint 10+ save/load, add a `DespawnCleanup` system that runs `registry.unregister()` for all despawned entities. + +**Test for the concern (Sprint 8):** + +```rust +#[test] +fn register_new_entity_after_unregister_gets_new_stable_id() { + let mut world = bevy_ecs::world::World::new(); + let e1 = world.spawn_empty().id(); + + let mut registry = EntityRegistry::new(0); + let id1 = registry.register(e1); + assert_eq!(id1, StableId(0)); + + registry.unregister(e1); + + // Simulate bevy recycling the index with new generation + let e2 = world.spawn_empty().id(); + let id2 = registry.register(e2); + + // New entity gets StableId(1), NOT StableId(0) — IDs are monotonic + assert_eq!(id2, StableId(1)); + // Old StableId(0) is gone + assert_eq!(registry.to_entity(&id1), None); + // New entity has new StableId + assert_eq!(registry.to_entity(&id2), Some(e2)); +} +``` + +--- + +### Q1: Canonical room ordering — does entity spawn order matter for StableId determinism? + +**Question (Gestalt R2-OQ-09):** Room ordering in Gauntlet YAML — canonical ordering affects entity StableId assignment. + +**Answer: Yes, entity spawn order determines StableId assignment and must be canonical.** + +The EntityRegistry assigns StableIds sequentially via `next_id` (registry.rs:47-48). The order entities are registered determines their wire IDs. If room ordering changes, all wire IDs shift, breaking golden file comparisons and all test assertions that reference wire IDs by value. + +**The contract:** + +1. `gauntlet.yaml` defines room order. Existing rooms are NEVER reordered. New rooms append. +2. Within each room YAML, entity order is the spawn order. Existing entities are NEVER reordered. New entities append. +3. Player is always spawned first (StableId 0). +4. The `GauntletEntity.wire_id` constants are derived from this ordering. + +**If someone reorders rooms or entities in YAML:** +- Golden files break (changed wire IDs in snapshots) +- Test assertions referencing `PAUSE_NPC_1.wire_id` break +- The fixture staleness check (`make fixtures-check`) catches this + +**This is the same additive-only constraint as the workshop brief's "existing rooms stay frozen."** It applies to entity lists within rooms too. + +--- + +### Q4: blocked_entities feasibility — can compute_observer_snapshot include blocked entities with blocking wall position? + +**Question (Ozzie via UQ-03):** Can the ObserverSnapshot include entities that are NOT visible, along with what wall blocks them? + +**Answer: Feasible but not trivial. Sprint 9 scope, gated behind `--test-mode`.** + +**What it requires:** + +The current `filter_visible_entities` (observer/mod.rs:207-276) only processes entities whose position is in `visible_positions`. Blocked entities are simply skipped. To include them, we need: + +1. **Iterate ALL entities** in range (not just visible ones) +2. **For each non-visible entity**, trace a line from observer to entity position +3. **Find the blocking wall** — the first position on the line that is non-walkable + +The line trace is essentially a raycast on the tile grid. We already have shadowcasting, but it doesn't expose per-entity blocking info — it computes a visibility set, not per-entity raycasts. + +**Implementation sketch:** + +```rust +/// A blocked entity with its blocking wall position. +/// Only included in --test-mode snapshots. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct BlockedEntity { + pub entity_id: u64, + pub entity_x: f32, + pub entity_y: f32, + pub entity_z: i32, + pub blocked_by_x: i32, + pub blocked_by_y: i32, + pub kind: EntityKind, +} + +// In ObserverSnapshot, add (gated behind #[serde(default)]): +#[serde(default)] +pub blocked_entities: Vec, +``` + +**Cost estimate:** For each non-visible entity within the forward range (~20 tiles), trace a Bresenham line (~20 tile checks per entity). With 15 NPCs, worst case is ~15 × 20 = 300 tile lookups per tick. This is negligible (microseconds) compared to shadowcasting. + +**Why Sprint 9:** The feature requires: +- New wire protocol field (protocol version bump) +- Bresenham line trace utility (doesn't exist yet) +- Test client code to display `✕ BLOCKED` entries +- Tests verifying the blocking wall is correct + +None of this blocks Sprint 8 testing. The text renderer can show "Entities: 5 visible" without blocked entities. The blocked entity display is a debugging enhancement for the human tester experience. + +**Recommendation:** Add the `blocked_entities` field as `Vec` with `#[serde(default)]` (always empty in non-test-mode). Populate it in Sprint 9 when the test client's enhanced display is built. + +--- + +## Summary + +| # | Deliverable | Status | Sprint 8? | +|---|-------------|--------|-----------| +| 1 | Determinism fixes A, B, D with regression tests | Final spec, copy-pasteable | YES | +| 2 | Server `--test-mode` + `--port 0` | Final spec, main.rs replacement | YES | +| 3 | Gauntlet loader MVP (5 rooms) | YAML structure + constants module + loader design | YES | +| 4a | Q7: SetTickRate while paused → reject | Bug fix + test | YES | +| 4b | Q8: Entity index recycling → safe (generation counter) | Analysis + test | YES (test only) | +| 4c | Q1: Canonical room ordering → required | Design constraint documented | YES (constraint) | +| 4d | Q4: blocked_entities → feasible, Sprint 9 | Design + cost estimate | Sprint 9 | + +--- + +## Addendum: Lead Override — Test Client Crate Location + +**Override (received during Round 3):** The test client binary moves from `server/src/bin/test_client.rs` to `tooling/test-client/` as a separate workspace crate. + +### Bridge Type Pub Export Verification + +The lead's action item for Dudley: "ensure bridge types are pub-exported for the tools crate." + +**Result: No server crate changes needed.** All bridge types are already publicly exported. + +Verification trace: + +| File | Line | Export | Status | +|------|------|--------|--------| +| `server/src/lib.rs` | 4 | `pub mod bridge;` | Public | +| `server/src/bridge/mod.rs` | 9 | `pub mod framing;` | Public | +| `server/src/bridge/mod.rs` | 11 | `pub mod tcp;` | Public | +| `server/src/bridge/mod.rs` | 12 | `pub mod types;` | Public | +| `server/src/bridge/mod.rs` | 13 | `pub use types::*;` | Re-exported at bridge level | +| `server/src/bridge/framing.rs` | 15 | `pub fn write_framed(...)` | Public | +| `server/src/bridge/framing.rs` | 36 | `pub fn read_framed(...)` | Public | + +The `tooling/test-client/` crate adds `settled-reach-server` as a workspace dependency and imports: + +```rust +// tooling/test-client/src/main.rs +use settled_reach_server::bridge::{ + ObserverSnapshot, PlayerInput, PlayerAction, // via pub use types::* + framing::{write_framed, read_framed}, + tcp::TcpBridge, // if reusing; or raw TcpStream with framing functions +}; +``` + +No changes to `server/` required for this override. diff --git a/docs/workshops/test-architecture/gestalt-round2.md b/docs/workshops/test-architecture/gestalt-round2.md new file mode 100644 index 000000000..7bc5527b1 --- /dev/null +++ b/docs/workshops/test-architecture/gestalt-round2.md @@ -0,0 +1,418 @@ +# Gestalt — Round 2: Coverage Audit, Anti-Tedium Mechanics, Transition Scenarios, Invariant Prioritization + +**Workshop:** QA Strategy & Test Architecture +**Track:** 1 (Test World Design) +**Date:** 2026-02-17 +**Inputs:** All Round 1 outputs (Gestalt, Ozzie, Tyre, Dudley, Stig, Hoshe, Justine) +**Lead decisions:** Room count rolling, anti-tedium full suite approved, determinism confirmed. + +--- + +## 1. Final Room Coverage Audit + +I mapped every mechanical system from `decisions/*.md` to the complete Gauntlet room list (8 original + 6 proposed in Round 1). The goal: every system that creates a player decision is tested by at least one room. + +### Complete Room List (14 rooms) + +| # | Room | Source | Primary Systems | +|---|------|--------|----------------| +| 1 | Inventory Warehouse | Brief | Pickup, CarriedBy, inventory grid, 9-slot limit | +| 2 | Occlusion Corridor | Brief | LOS, shadowcasting, vision cone, perception modes | +| 3 | Interaction Gallery | Brief | ObjectType verbs, multi-verb NPC, sprint suppression | +| 4 | Crowd Plaza | Brief | Entity density, relationship colors, cognitive delay overlap | +| 5 | Fog Theater | Brief | Fog layer transitions, peripheral dimming, exploration persistence | +| 6 | Dialogue Room | Brief | Trust tiers, contradiction, walk-away, monologue during dialogue | +| 7 | Pause Chamber | Brief | TickRate toggle, state transitions, pause/unpause | +| 8 | Zone Gate | Brief (reserved) | Zone transition (future contract) | +| 9 | Eavesdrop Alcove | Gestalt R1 | ListeningFocus, zone ambient, Careful stance, conversation eavesdrop | +| 10 | Confrontation Stage | Gestalt R1 | Cognitive vulnerability, audio dip, graduated SFX suppression | +| 11 | Sprint Gauntlet | Gestalt R1 | Sprint suppression, anomaly survival, stance transitions | +| 12 | Sound Lab | Gestalt R1 | Three-range sound, sound pings, recognition chime sequence | +| 13 | Decay Observatory | Gestalt R1 | Knowledge decay, stale state, fog representation degradation | +| 14 | Shift Change | Gestalt R1 | Stress test: all systems simultaneously at density | + +### Coverage Matrix: System -> Room -> Key Assertion + +Each row is a mechanical system drawn from a confirmed decision. The matrix shows which room exercises it and the primary assertion. + +#### Pillar 1: Characters & Information + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| Knowledge graph — confidence levels | D-041 | Dialogue Room, Decay Observatory | Suspects < KnowsOf < KnowsDetails < Direct. Each level gates different verbs/dialogue. | +| Knowledge graph — decay | D-041 | Decay Observatory | Confidence downgrades once per game-minute (10 ticks). Direct -> KnowsDetails -> KnowsOf -> Suspects on schedule. | +| Knowledge graph — contradiction | D-041 | Dialogue Room | Two conflicting entries produce Contradicted state. Monologue fires. D-033 color shifts to PersonOfInterest. | +| Knowledge graph — event queue | D-041 | Shift Change | 15 entities generating KnowledgeEvents simultaneously. Queue drains completely per tick. No dropped events. | +| Entity color = relationship | D-033 | Crowd Plaza, Dialogue Room | Color maps to RelationshipState, not objective property. Unknown=#4a9ebb, Friendly=#6bc9a6, POI=#e8c547, Hostile=#d45d5d. | +| Entity color transitions | D-033 | Dialogue Room | 0.5s fade on relationship change. Contradiction shifts green -> amber. | +| Monologue — core system | D-016 | Sprint Gauntlet, Confrontation Stage, Eavesdrop Alcove | Fires as perception bridge, atmosphere, diegetic hint. Character voice varies by archetype. | +| Monologue — separate pools | D-032 | Dialogue Room | Smuggler and detective monologue pools are hard-partitioned. Same trigger, different pool. | +| Monologue — trigger types | D-016/D-032 | Multiple (see sub-table) | Each trigger type exercised at least once. | +| Invisible locked options | D-062 | Dialogue Room | No grayed-out options. No lock icons. Options appear only when unlocked. | +| Confrontation — same box | D-063 | Confrontation Stage | Confrontation uses same dialogue UI. Weight from italics + pre-delivery beat + world response. | +| Walk-away consequences | D-064 | Dialogue Room, Confrontation Stage | 300ms fade, NPC reacts, KG records incompleteness. Per-seed tolerance variation. | +| Inventory — physical items | D-065 | Inventory Warehouse | 9-slot limit (smuggler 3-4, detective 2). Take/Place verbs. CarriedBy component. | + +**Monologue trigger sub-table:** + +| Trigger Type | Room | Scenario | +|-------------|------|----------| +| `enter_location` | Hub (entering any room) | Player crosses room threshold, location monologue fires | +| `observe_npc` | Crowd Plaza | First observation of NPC triggers character assessment monologue | +| `hear_sound` | Sound Lab | Sound from behind wall triggers "Footsteps behind me..." | +| `observe_anomaly` | Sprint Gauntlet, Confrontation Stage | NPC in wrong location triggers anomaly monologue | +| `post_conversation` | Dialogue Room | After dialogue ends, reflective monologue fires | +| `discover_evidence` | Dialogue Room (contradiction) | Contradiction discovery triggers "Wait — they said..." | +| `witness_interaction` | Eavesdrop Alcove | Witnessing NPC-to-NPC conversation triggers interpretation | +| `time_idle` | Eavesdrop Alcove (30+ tick wait) | Idle in alcove triggers reflective monologue | +| `return_visit` | Decay Observatory (return from Part B) | Returning to Part A triggers "Last time I was here..." | + +#### Pillar 2: Perception + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| Fog of perception | D-011 | Fog Theater, Occlusion Corridor | LOS-based, not radius. Walls block. Fog returns on departure. | +| Camera locked to character | D-015 | All rooms | No panning. Vision cone: forward=full, peripheral=reduced, behind=fog. | +| Vision cone — forward/peripheral/behind | D-015 | Fog Theater, Occlusion Corridor | Forward: full LOS, full detail. Peripheral: reduced range, dimmed. Behind: blind. | +| Perception modes | D-017 | Occlusion Corridor | Mode switch reveals different information. Same entity, different ObserverSnapshot per mode. | +| Three-range sound | D-018 | Sound Lab | Close=stereo, high accuracy. Medium=directional indicator+monologue. Long=insert alert (deferred). | +| Shadowcasting — symmetric | D-035 | Occlusion Corridor, Fog Theater | If A sees B, B sees A. Verified with entities on both sides of LOS boundary. | +| Fog — 5 layers | D-059 | Fog Theater | Clear, light fog, deep fog, unexplored+maps, unexplored. Each layer tested for correct visual properties. | +| Fog — sound pings | D-059 | Sound Lab | 2-3 concentric expanding rings. Loud=3 bright fast. Quiet=1 faint slow. Fade 1.5s. | +| Fog — recognized vs unrecognized entity | D-059 | Sound Lab, Decay Observatory | Known: D-033 color glow + silhouette + 0.8s pulse + position drift. Unknown: grey blob. | +| Cognitive delay | D-060 | Fog Theater, Sound Lab, Crowd Plaza | 0.6s base, 0.3s urgent. Monologue fires DURING delay. Visual: blob -> color+silhouette over ~0.3s. | +| Recognition chime timing | D-067 | Sound Lab | Chime at ONSET of delay (not completion). 300-400ms. Sequence: detect -> chime -> delay -> monologue -> visual transition -> done. | +| Cognitive vulnerability (confrontation) | D-070 | Confrontation Stage | Emotional focus reduces ambient perception. Audio dip. Loud events break through, quiet don't. Post-confrontation delayed monologue. | +| ListeningFocus | D-071 | Eavesdrop Alcove | Stationary 30+ ticks -> +2 to +3dB World SFX. Far conversations become audible. | +| Zone ambient vs conversation | D-072 | Eavesdrop Alcove | High ambient hides conversations. Low ambient makes them conspicuous. | +| Insert overlay independence | D-048 | Fog Theater | Insert data (z-layer 6) renders regardless of fog state (z-layer 5). | + +#### Pillar 3: Movement & Interaction + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| Stance toggle — Sprint/Walk/Careful/Crouch | D-053 | Sprint Gauntlet, Eavesdrop Alcove | Each stance: correct tick/tile speed, monologue rate, footstep volume, interaction availability. | +| Sprint — interaction suppression | D-055 | Sprint Gauntlet, Interaction Gallery | Interaction buffer EMPTY during sprint. Anomaly monologue survives (sprint double-take). | +| Sprint — monologue at 40% | D-053 | Sprint Gauntlet | Run 10 times: monologue fires ~4 times (statistical via SimRng). | +| Careful — tell notice bonus | D-053 | Eavesdrop Alcove | Careful stance monologue at 150%. Observation monologue fires for Tier 2 NPC behavior. | +| Tile-based movement | D-054 | All rooms | Discrete positions. Client Tween hides grid. TilePresence layers. | +| Same-tile occupancy | D-054 | Crowd Plaza | Multiple entities on one tile in different posture layers (Standing + Seated). | +| Dual-scale grid | D-066 | Occlusion Corridor, Fog Theater | 0.5m sim tiles, 1m visual tiles. 2x2 geometry minimum. Cover maps 1:1 at visual scale. | +| Entity interaction — vertical list | D-057 | Interaction Gallery | Two-phase verb computation. Phase 1 (ObjectType), Phase 2 (KG filter). Max 4 options. | +| Cursor states | D-056 | Interaction Gallery | Default/entity hover/object hover/(weapon aim future). 150ms transitions. D-033 color on hover. | +| World menu — radial | D-058 | Interaction Gallery | Right-click: 4 spokes. v0.1: 2 spokes (Observe + Insert). | +| Dialogue box | D-061 | Dialogue Room | Bottom screen, max 20%, no portraits. Max 3 response options. Monologue floats ABOVE. | + +#### Pillar 4: Audio + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| 5-bus audio | D-068 | Sound Lab, Eavesdrop Alcove | Music, Ambient, World SFX, Player Actions, UI Sounds — all routable, independent sliders. | +| Audio dip — dialogue | D-069 | Dialogue Room | Ambient -6 to -8dB during dialogue. 300ms ease-in, 500ms ease-out. | +| Audio dip — confrontation | D-069 | Confrontation Stage | Ambient -10 to -12dB + low-pass 20kHz->800Hz. World SFX graduated. 500ms in, 1000ms out. | +| ListeningFocus boost | D-069 | Eavesdrop Alcove | World SFX +2 to +3dB when stationary 30+ ticks. | +| Zone crossfade | D-073 | Hub corridors (between rooms) | Hard tile boundary, 1.5-2s audio crossfade tween. | +| Audio aesthetic — insert-tech vs organic | D-074 | Sound Lab | Chimes = synthetic/precise. Footsteps = warm/breathy. Two sonic families. | + +#### Pillar 5: Simulation & Architecture + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| Client-server separation | D-020 | All rooms (Layer 3 test) | ObserverSnapshot is the only data crossing boundary. PlayerInput is semantic actions. | +| Deterministic simulation | D-010 p4 | Shift Change (determinism test) | Same seed + same inputs = identical world state at tick N. INV-T01 master invariant. | +| Pause/unpause | D-031, Bug #3 | Pause Chamber | Movement discarded. No state changes. UI responsive. Round-trip preserves state. | +| Tick budget | D-026 | Shift Change | <=100ms per tick with 15+ active entities. Measured per tick, not averaged. | +| Snapshot delivery | D-020, Bug #1 | Shift Change (Layer 3) | Every tick produces exactly one ObserverSnapshot. No drops. No duplicates. In order. | +| SimRng determinism | D-030 #7 | Shift Change | RNG consumed in fixed system order. Explicit `.after()` constraints on all consumers. | +| Time system | D-031 | Decay Observatory | 10 ticks = 1 game-minute. Day phases drive routine transitions. | +| Simulation tiers | D-026 | Shift Change | Active tier: full sim. Entities within scope tags stay fully simulated. | +| MessagePack serialization | D-020 | Cross-language fixture tests (Hoshe) | Boundary values roundtrip. Bug #4 class prevented. | +| StableId/EntityRegistry | D-041 | All rooms with NPCs | Unique IDs. Bidirectional mapping. No dangling references. | + +#### Content Systems + +| System | Decision | Room(s) | Key Assertion | +|--------|----------|---------|---------------| +| NPC 10-axis model | D-024 | Crowd Plaza, Dialogue Room | Want, Secret, Relationships, Tolerance, Routine, Information, Contentment, Personality, Tell, Skill. | +| Social site as unit | D-025 | Crowd Plaza | 4-8 NPCs in connected space with sightlines. | +| Dialogue — tagged pools | D-028 | Dialogue Room | 4 layers: access tier, relationship history, trust-gated gossip, unprompted disclosure. | +| Tag taxonomy | D-035 | Dialogue Room | 6 structural + 3 selection tags per line. Tag filtering produces valid line selection. | +| Two-tier animation | D-047 | Eavesdrop Alcove, Confrontation Stage | Tier 1 (clear activities) instantly readable. Tier 2 (ambiguous behaviors) observable but intention unclear. | +| Population ratio | D-029 | Crowd Plaza | ~30% flat, ~50% mundane triangles, ~20% entangled. Signal requires noise floor. | + +### Coverage Gaps: Systems With NO Room Coverage + +| System | Decision | Gap | Recommendation | +|--------|----------|-----|----------------| +| ~~Weather affecting vision cones~~ | D-050 | v0.1 deferred | No action needed | +| ~~Perception mode switching cost~~ | D-017 soft degradation | D-059 scan-line interference at high load | Could test in Occlusion Corridor by toggling modes rapidly. Low priority — soft visual effect. | +| Favorite colors — object identification | D-052 | No room tests object-layer color as identification mechanic | **Add to Inventory Warehouse**: place personal objects (mug, cushion) with NPC favorite colors. Assert saturation below D-033 entity color. v0.1.2+ system — defer room addition. | +| News ticker | D-039 wow #5 | No room tests dual-character ticker reactions | Not a Gauntlet room — this is content-level, tested via the line previewer. No room needed. | +| Idle reflective monologue | D-039 wow #6 | Partially covered by Eavesdrop Alcove (time_idle trigger) | Eavesdrop Alcove covers the mechanic. The "wow moment" quality is a content concern, not a systems test. | +| Diegetic insert/POI navigation | D-013 | No room tests POI discovery or insert minimap | **Add to Hub**: place POI markers that appear/disappear based on knowledge state. Low priority — insert UI is v0.1 stretch. | +| Environmental neutrality | D-045 | No room verifies that world layer DOESN'T change with narrative state | **Assertion on Dialogue Room**: pre-confrontation and post-confrontation CanvasModulate values are identical. Zone lighting unchanged. Add as assertion, not a room. | + +**Verdict: 3 minor gaps, all addressable as assertions on existing rooms. No new rooms needed for coverage.** + +--- + +## 2. Anti-Tedium Feature Mechanics + +Ozzie proposed 6 anti-tedium features. All approved by the lead. Let me map each to concrete game mechanics. + +### 2.1 Room Reset Trigger + +**Mechanic: Reset Plate — a special TileKind at each room entrance.** + +**Implementation:** + +- **Tile type:** `TileKind::ResetPlate` — a new variant in the tile enum. Rendered as a floor plate with a distinct visual (pulsing outline, insert-style geometric pattern). +- **Trigger:** Player steps onto the ResetPlate tile AND presses the Interact key (not automatic — prevents accidental resets during sprint-throughs). +- **Server-side behavior:** + 1. Identify the room boundary (room_id from ResetPlate's associated metadata) + 2. For all entities within the room boundary: + - Reset position to tick-0 spawn position (stored in `GauntletSpawnPosition` component) + - Reset KnowledgeGraph to tick-0 state (stored in `GauntletKnowledgeBaseline` component) + - Reset relationship state to tick-0 baseline + - Reset inventory to tick-0 baseline (for Inventory Warehouse) + - Reset any active cognitive delays, monologue queues, dialogue state + 3. Reset fog exploration state for tiles within the room boundary to unexplored + 4. Reset the player's KnowledgeGraph entries that reference entities in this room only + 5. Do NOT reset: player position (they're standing on the plate), global game time, player inventory items from other rooms + +**What resets (per room type):** + +| Room | Entity positions | Entity KG | Player KG (room refs) | Player inventory | Fog | Dialogue state | +|------|-----------------|-----------|----------------------|-----------------|-----|---------------| +| Inventory Warehouse | Yes | N/A | Yes | YES (clear room items) | Yes | N/A | +| Occlusion Corridor | Yes | N/A | Yes | No | Yes | N/A | +| Interaction Gallery | Yes | N/A | Yes | No | Yes | N/A | +| Crowd Plaza | Yes | N/A | Yes | No | Yes | N/A | +| Fog Theater | Yes | N/A | Yes | No | YES (critical) | N/A | +| Dialogue Room | Yes | Yes (trust tiers) | Yes | No | Yes | YES (reset tree) | +| Confrontation Stage | Yes | Yes | Yes | No | Yes | YES | +| All others | Yes | If applicable | Yes | No | Yes | If applicable | + +**ECS implementation sketch:** + +```rust +// New components for Gauntlet entities +#[derive(Component)] +pub struct GauntletSpawnPosition(pub TilePosition); + +#[derive(Component)] +pub struct GauntletKnowledgeBaseline(pub KnowledgeGraph); + +#[derive(Component)] +pub struct GauntletRoomId(pub u8); + +// Reset system — triggered by player input on ResetPlate tile +fn reset_gauntlet_room( + room_id: u8, + entities: Query<(&GauntletRoomId, &GauntletSpawnPosition, &mut TilePosition, ...)>, + mut player_kg: Query<&mut KnowledgeGraph, With>, +) { ... } +``` + +**Why Interact, not automatic:** Automatic reset on step creates chaos during cross-room transitions. Sprint through the Hub into the Occlusion Corridor — you DON'T want the corridor to reset just because you crossed the plate. The deliberate Interact press is the "I want to try this again" signal. + +### 2.2 Hub Teleport + +**Mechanic: Debug command, not a gameplay mechanic.** + +**Implementation:** + +- **Trigger:** Keyboard shortcut (Home key or backtick). NOT an in-game interaction — this is a testing tool. +- **Behavior:** + 1. Immediately move player entity to Hub center position (`GAUNTLET.hub_center`) + 2. Clear any active dialogue/monologue state + 3. Clear interaction buffer + 4. Do NOT reset any room state (rooms maintain their current state — the tester left, they didn't reset) + 5. Do NOT affect game time or tick count +- **Server-side:** New `PlayerAction::TeleportToHub` variant. The server validates this is a Gauntlet-only action (rejected in non-Gauntlet maps). +- **Game state impact:** Position changes. That's it. No KG changes, no relationship changes, no inventory changes. The player just "walked really fast" back to the Hub. + +**Why debug command, not gameplay:** Hub teleport breaks the information model (instant position change violates D-015 camera lock and D-011 fog progression). It exists for testing efficiency, not gameplay. The `--gauntlet` server flag enables it; production builds reject it. + +### 2.3 WRONG Button (F12) + +**Mechanic: State capture system with minimal human input.** + +**What gets captured:** + +| Layer | Data | Size estimate | +|-------|------|--------------| +| **ObserverSnapshot** | Full current snapshot (what the client sees) | ~5-20 KB (MessagePack) | +| **World digest** (per Dudley's spec) | Entity count, registry count, RNG position, NPC positions | ~1-2 KB (JSON) | +| **Input history** | Last 100 ticks of PlayerInput | ~2-5 KB (JSON) | +| **Tick number + room ID** | Current tick, current room (from player position) | ~50 bytes | +| **Text render** | `format_snapshot_text()` output (per Tyre's spec) | ~2-5 KB (text) | +| **Human description** | One-line text from tester | ~100 bytes | + +**Minimum viable capture (Sprint 8):** + +1. ObserverSnapshot (already exists — just serialize current buffer) +2. Tick number + player position +3. Text render output (once Tyre's formatter exists) +4. Human description prompt + +**Full capture (Sprint 9+):** + +Add world digest, input history, and deterministic replay seed. + +**Output directory:** `tests/bug-reports/gauntlet-{tick}-{timestamp}/` + +``` +tests/bug-reports/gauntlet-0142-20260217T141523/ + snapshot.msgpack # ObserverSnapshot + snapshot.json # Human-readable JSON + text-render.txt # Formatted text output + digest.json # World digest (Sprint 9+) + inputs.json # Last 100 ticks of input (Sprint 9+) + description.txt # "NPC behind wall was visible" +``` + +**Why this is the minimum:** The snapshot + text render + human description gives a developer everything they need to understand WHAT was wrong. The input history + seed (Sprint 9+) gives them everything they need to REPRODUCE it. Minimum viable = understand. Full = reproduce. + +**Server-side:** New `PlayerAction::BugReport { description: String }`. Server serializes state to the output directory. This is a Gauntlet-only action. + +--- + +## 3. Cross-Room Transition Scenarios (Refined for Hub-and-Spoke) + +Ozzie's hub-and-spoke layout with cross-cuts defines which rooms physically connect. Let me refine my Round 1 transition scenarios based on the actual topology. + +### Physical Layout (from Ozzie R1) + +``` + [Fog Theater] + | +[Inventory] --- [CENTRAL HUB] --- [Occlusion Corridor] + | | | +[Interaction [Pause Chamber] [Crowd Plaza] + Gallery] | + [Dialogue Room] + | + [Zone Gate (reserved)] +``` + +**Cross-cut corridors (Ozzie R1):** +- Crowd Plaza <-> Occlusion Corridor +- Fog Theater <-> Dialogue Room +- Inventory Warehouse <-> Interaction Gallery + +**New rooms to add to layout:** + +``` + [Fog Theater]----[Sound Lab] + | | +[Inventory] --- [CENTRAL HUB] --- [Occlusion Corridor] + | | | +[Interaction [Pause Chamber] [Crowd Plaza] + Gallery] | | + | [Dialogue Room] [Shift Change] + | | +[Sprint [Confrontation [Eavesdrop + Gauntlet] Stage] Alcove] + | + [Decay Observatory] + | + [Zone Gate (reserved)] +``` + +**New connections (proposed cross-cuts for system combination testing):** +- Sprint Gauntlet <-> Interaction Gallery (sprint suppression into interaction range) +- Confrontation Stage <-> Eavesdrop Alcove (confrontation vulnerability into listening) +- Sound Lab <-> Fog Theater (sound pings across fog boundaries) +- Shift Change <-> Crowd Plaza (density stress into density observation) + +### Refined Transition Scenarios + +All scenarios now reference physically connected rooms via corridors or cross-cuts. + +| # | Scenario | Path | System Combination | Expected Behavior | Testable? | +|---|----------|------|-------------------|-------------------|-----------| +| T1 | Sprint Exit | Crowd Plaza -> (cross-cut) -> Occlusion Corridor | D-055 + D-035 | Interaction buffer cleared during sprint. LOS recalculated at corridor entrance. No stale verbs from Plaza NPCs leak into Corridor snapshot. | YES — direct cross-cut exists | +| T2 | Fog into Dialogue | Fog Theater -> (cross-cut) -> Dialogue Room | D-059 + D-061 + D-060 | Player in light fog, initiates dialogue. Fog state preserved (entity in fog still tracked). Cognitive delay for fog entity completes during dialogue. Monologue from recognition appears ABOVE dialogue box (z-layer 7 vs dialogue at bottom). | YES — direct cross-cut exists | +| T3 | Full Inventory Interact | Inventory Warehouse -> (cross-cut) -> Interaction Gallery | D-065 + D-057 | Carry 9/9 inventory into interaction range. Take verb still offered by server (Phase 1). Client greys it out (Phase 2 filter: inventory full). Other verbs (Talk, Observe) unaffected. | YES — direct cross-cut exists | +| T4 | Sprint into Interaction | Sprint Gauntlet -> (cross-cut) -> Interaction Gallery | D-055 + D-057 | Sprint clears buffer. Arriving at Interaction Gallery entity in Walk stance: verbs repopulate within 1 tick. No buffer corruption from stance transition. | YES — proposed cross-cut | +| T5 | Confrontation to Eavesdrop | Confrontation Stage -> (cross-cut) -> Eavesdrop Alcove | D-070 + D-071 | Post-confrontation: audio dip releasing (1000ms ease-out). Enter Eavesdrop Alcove before dip fully releases. ListeningFocus not yet active (need 30 ticks stationary). Verify: dip release and ListeningFocus activation don't conflict. | YES — proposed cross-cut | +| T6 | Pause Anywhere | Pause Chamber -> Hub -> any room | D-031 + any system | Pause in Pause Chamber. Hub teleport. Enter any room. Unpause. All room systems resume correctly. No state corruption from pause during room transition. | YES — Hub connects all rooms | +| T7 | Sound across Fog | Sound Lab -> (cross-cut) -> Fog Theater | D-018 + D-059 | Sound events from Sound Lab NPCs propagate through wall into Fog Theater. Sound pings render in fog. Cognitive delay fires for recognized entity based on sound alone. | YES — proposed cross-cut | +| T8 | Walk-away Sprint | Dialogue Room -> Hub -> Sprint Gauntlet | D-064 + D-053 | Walk-away from dialogue (300ms fade). Walk to Hub. Sprint to Sprint Gauntlet. KG incompleteness recorded. NPC reaction in Dialogue Room occurs during player's Hub transit. Sprint suppresses any post-dialogue monologue. | YES — Hub path | + +**Scenarios dropped from Round 1:** +- "Peripheral Interact" (Fog Theater -> Interaction Gallery): No direct cross-cut. Covered by Interaction Gallery's own peripheral-entity test. Peripheral interaction range is a geometry question, not a cross-room question. + +### Corridor Design for Transition Testing + +The corridors between rooms aren't just hallways — they're testing infrastructure: + +- **Length:** 10-15 sim tiles (~5-7.5 visual tiles) per Ozzie's spec. Long enough to test stance transitions, short enough to not waste time. +- **Zone boundaries:** Each corridor has a zone_id transition at its midpoint. Tests D-073 zone crossfade (1.5-2s audio tween). +- **Fog behavior:** Corridor enters deep fog within 5-6 tiles of leaving a room (D-059 fog gradient over 6-8 sim tiles). Tests fog re-engagement. +- **Reset plates:** At each room entrance (room side of corridor, not hub side). Tester walks corridor, hits plate, room resets. + +--- + +## 4. Invariant Prioritization: Top 10 for Sprint 8 + +Ranked by: (1) does it catch a bug class we've actually seen? (2) how many systems does it protect? (3) implementation cost. + +| Rank | ID | Invariant | Bug Catalogue Match | Systems Protected | Implementation | Sprint 8? | +|------|-----|-----------|-------------------|------------------|----------------|-----------| +| **1** | INV-T04 | Pause coherence | Bug #3 (movement while paused) | Pause, movement, state transitions | Dudley's 8 test cases exist. Just wire up. | YES — direct regression guard | +| **2** | INV-T01 | Deterministic replay | Bug #1 class (state divergence) | ALL systems | Tyre's determinism test spec + 4 fixes. Core value of the Gauntlet. | YES — Gauntlet's raison d'etre | +| **3** | INV-T03 | Snapshot delivery | Bug #1 (no snapshots), Bug #5 (overwrite) | IPC, bridge, client-server | Layer 3 subprocess test (Hoshe's spec). Catches the exact bug class. | YES — prevents #1 recurrence | +| **4** | INV-T05 | Input ordering | Bug #1 (blocking read stalls schedule) | IPC, input processing, tick ordering | Covered by Layer 3 test + Dudley's pause tests. | YES — part of Layer 3 | +| **5** | INV-S01 | Player spawn reachable | No bug yet, but SOFTLOCK risk | Pathfinding, map loading, content validation | Single pathfind check at map load. ~10 lines. | YES — trivial, high value | +| **6** | INV-C03 | StableId uniqueness | No bug yet, but CORRUPTION risk | Knowledge graph, entity lifecycle, save/load | Dudley's 4 registry lifecycle tests. | YES — direct from Dudley R1 | +| **7** | INV-C07 | Dialogue pool non-empty | No bug yet, but PLAYER-FACING failure | Dialogue, tag filtering, content pipeline | Load-time validation (Tyre's cross-reference validator). | YES — part of content validation | +| **8** | INV-T02 | Tick budget | Bug #6 class (rate mismatch) | Performance, entity density, all tick systems | Shift Change room perf assertion (p95 < 20ms per Tyre). | YES — performance gate | +| **9** | INV-P02 | LOS symmetry | No bug yet, but FAIRNESS guarantee | Shadowcasting, NPC perception, information boundaries | Property-based test: random positions, verify A sees B iff B sees A. | YES — D-035 mandates symmetry | +| **10** | INV-S05 | No entity inside geometry | No bug yet, but content scaling risk | Content loading, entity spawning, map geometry | Check at map load: every entity spawn position is walkable. | YES — trivial, prevents softlocks | + +### Deferred to Sprint 9+ + +| Rank | ID | Invariant | Reason for deferral | +|------|-----|-----------|-------------------| +| 11 | INV-T06 | SimRng consumption order | Depends on determinism fixes landing first (Sprint 8 P0) | +| 12 | INV-T07 | Cognitive delay monotonicity | Cognitive delay system not yet implemented | +| 13 | INV-T08 | Knowledge decay timing | Decay system implemented but not yet connected to fog rendering | +| 14 | INV-S06 | 2x2 geometry minimum | Requires map validation tooling not yet built | +| 15 | INV-P03 | Fog layer ordering | Fog shader is client-side; needs gdUnit4 test (Stig's domain) | +| 16 | INV-P04 | Insert independence | Insert overlay not yet implemented | +| 17 | INV-S07 | Zone boundary coherence | Zone system not yet implemented | +| 18 | INV-P05 | Sound propagation coherence | Sound propagation not yet implemented | +| 19 | INV-C06 | Monologue trigger reachability | Requires monologue system + content cross-reference | +| 20 | INV-S04 | NPC routine paths valid | Requires NPC routine system + pathfinding integration | + +### Bug Catalogue Mapping + +Every Sprint 6-7 bug is now covered: + +| Bug | Root Cause | Invariant(s) | Room(s) | Test Type | +|-----|-----------|-------------|---------|-----------| +| #1 Server never sends snapshots | read_framed() blocking | INV-T03, INV-T05 | Layer 3 subprocess test | Integration | +| #2 Camera doesn't center | No snapshot until keystroke | INV-T03 | Layer 3 + client gdUnit4 | Integration + unit | +| #3 Player moves while paused | process_player_input ignores TickRate | INV-T04 | Pause Chamber | Unit (Dudley's 8 tests) | +| #4 MessagePack -128 for 128 | Signed int8 branch boundary | Hoshe's BV matrix (41 values) | Fixture tests | Unit + cross-language | +| #5 Monologue lost on overwrite | Snapshot buffer drops one-shots | INV-T03 | Shift Change (density) | Integration | +| #6 Snapshot overwrite warning | Server faster than client | INV-T02, INV-T03 | Shift Change (perf) | Performance | + +--- + +## Summary + +### Deliverables + +1. **Coverage matrix**: 50+ system-to-room mappings. 3 minor gaps identified (favorite colors, POI navigation, environmental neutrality assertions) — all addressable without new rooms. +2. **Anti-tedium mechanics**: Reset Plate (Interact-triggered, per-room state rollback), Hub Teleport (debug command, position-only), WRONG Button (MVP: snapshot + text render + description; full: + input history + replay seed). +3. **8 transition scenarios**: Refined for hub-and-spoke layout with cross-cuts. All physically testable. Corridor design specified (10-15 tiles, zone boundary at midpoint, reset plate at room entrance). +4. **Top 10 invariants for Sprint 8**: Prioritized by bug catalogue match. Every Sprint 6-7 bug class covered. INV-T04 (pause) and INV-T01 (determinism) are #1-#2. + +### Open Questions for Round 3 + +1. **Room ordering in Gauntlet YAML**: Should rooms have a canonical ordering in the content pack? Affects entity StableId assignment (lower room = lower IDs). Determinism depends on consistent entity spawn order. +2. **Reset Plate vs full server restart**: Is per-room reset sufficient, or do some tests require a clean server state? If the latter, the Gauntlet needs a "full reset" command alongside per-room resets. +3. **Cross-cut corridor count**: I proposed 4 new cross-cuts (Sprint Gauntlet<->Interaction Gallery, Confrontation Stage<->Eavesdrop Alcove, Sound Lab<->Fog Theater, Shift Change<->Crowd Plaza). Is this too many? Each cross-cut adds map complexity. Could consolidate Sound Lab into Occlusion Corridor as a sub-area instead of a separate room+cross-cut. diff --git a/docs/workshops/test-architecture/gestalt-round3.md b/docs/workshops/test-architecture/gestalt-round3.md new file mode 100644 index 000000000..1036ab4f6 --- /dev/null +++ b/docs/workshops/test-architecture/gestalt-round3.md @@ -0,0 +1,764 @@ +# Gestalt — Round 3: Final Gauntlet Map Specification + +**Workshop:** QA Strategy & Test Architecture +**Track:** 1 (Test World Design) +**Date:** 2026-02-17 +**Purpose:** Produce a buildable map specification for Sprint 8. Dudley builds from this. + +--- + +## 1. Gauntlet MVP Room List — Sprint 8 + +7 rooms + Central Hub. From the 14 rooms designed in Rounds 1-2, this is the minimum set that covers every Sprint 6-7 bug class, the top 10 invariants, and the core system interactions. + +### Rooms Shipping Sprint 8 + +| # | Room | Why Sprint 8 | Bug Coverage | Invariants | +|---|------|-------------|--------------|------------| +| 0 | **Central Hub** | Connector. Required for hub-and-spoke. | — | INV-S01 (spawn reachable) | +| 1 | **Pause Chamber** | Bug #3 direct regression guard. Dudley's 8 pause tests need a room. | Bug #3 (movement while paused) | INV-T04 (pause coherence) | +| 2 | **Occlusion Corridor** | Core information boundary testing. LOS = the game's master mechanic. | — | INV-P02 (LOS symmetry) | +| 3 | **Inventory Warehouse** | Pickup/drop/slot system. Vertical slice requirement. | — | INV-S05 (no entity inside geometry) | +| 4 | **Interaction Gallery** | Verb computation, multi-verb selection, sprint suppression. | — | — | +| 5 | **Fog Theater** | Fog layer transitions. Distinct from Occlusion (fog = visual treatment, occlusion = LOS geometry). | — | INV-P02 (fog+LOS interaction) | +| 6 | **Dialogue Room** | KG state, trust tiers, contradiction detection. Content system anchor. | — | INV-C07 (dialogue pool non-empty), INV-C03 (StableId uniqueness) | +| 7 | **Crowd Plaza** | Entity density stress test. Doubles as determinism + tick budget test target. 15+ NPCs. | Bug #1 (snapshot delivery at load), Bug #5 (overwrite at density), Bug #6 (rate mismatch) | INV-T01 (deterministic replay), INV-T02 (tick budget), INV-T03 (snapshot delivery) | + +**Crowd Plaza is the stress room.** It replaces the separate Shift Change room for Sprint 8 by combining density testing, determinism verification, and tick budget measurement into one room with 15 NPCs. Running the same seed twice and comparing golden files tests INV-T01. Measuring tick time at 15+ entity density tests INV-T02. + +### Rooms Deferred to Sprint 9+ + +| # | Room | Deferral Reason | Ships When | +|---|------|----------------|------------| +| 8 | Zone Gate | Zone transitions not implemented | Multi-map system lands | +| 9 | Eavesdrop Alcove | Depends on ListeningFocus (D-071) | Audio system implementation | +| 10 | Confrontation Stage | Depends on confrontation vulnerability (D-070) | Confrontation system implementation | +| 11 | Sprint Gauntlet | Sprint suppression covered by Interaction Gallery + cross-room T1 | Stretch goal for Sprint 9 | +| 12 | Sound Lab | Depends on sound propagation system | Audio system implementation | +| 13 | Decay Observatory | KG decay not yet connected to fog rendering | KG decay wired to client | +| 14 | Shift Change | Merged into Crowd Plaza for Sprint 8 | When granular stress testing needed | + +**Deferral justification:** Every deferred room depends on a system that doesn't exist yet. The 7 MVP rooms cover every *implemented* system. When new systems land, new rooms get appended — the additive-only rule means Sprint 8 rooms never change. + +--- + +## 2. Physical Map Layout — Final + +### Topology + +Hub-and-spoke with 2 cross-cuts. All corridors are straight (orthogonal). Every room is reachable from the hub in 1-2 corridors. + +``` + ┌─────────────────────────────┐ + │ FOG THEATER │ + │ (44×32 tiles) │ + │ observer: (28,16) S │ + └────────────┬────────────────┘ + │ corridor-N (6×10) + │ +┌──────────────────┐ ┌────────────┴────────────┐ ┌──────────────────────────┐ +│ INVENTORY │ │ │ │ OCCLUSION CORRIDOR │ +│ WAREHOUSE ├────┤ CENTRAL HUB ├────┤ (42×22 tiles) │ +│ (30×28 tiles) │ │ (24×24 tiles) │ │ observer: (10,10) E │ +│ observer:(15,14)E│ │ spawn: (12,12) │ │ │ +└────────┬─────────┘ └───────┬─────────────────┘ └─────────────┬────────────┘ + │ │ │ + cross-cut-W (6×10) corridor-S (6×10) cross-cut-E (6×10) + │ │ │ +┌────────┴─────────┐ ┌──────┴──────────┐ ┌──────────┴─────────────┐ +│ INTERACTION │ │ PAUSE CHAMBER │ │ CROWD PLAZA │ +│ GALLERY │ │ (16×16 tiles) │ │ (32×32 tiles) │ +│ (24×20 tiles) │ │ observer:(8,8) N│ │ observer: (16,16) W │ +│ observer:(12,10)E│ └──────┬──────────┘ │ │ +└──────────────────┘ │ └────────────────────────┘ + corridor-S2 (6×10) + │ + ┌──────┴──────────┐ + │ DIALOGUE ROOM │ + │ (28×20 tiles) │ + │ observer:(14,10)│ + │ N │ + └─────────────────┘ +``` + +### Room Coordinate Table + +All coordinates in sim tiles (0.5m per tile, per D-066). Room bounds include 2-tile-thick walls (1 visual tile = D-066 minimum). Interior walkable area starts 2 tiles inside each boundary. + +| # | Room | Origin (x,y) | Size (w×h) | Interior | Observer Pos | Observer Facing | +|---|------|-------------|-----------|----------|-------------|----------------| +| 0 | Central Hub | (38, 46) | 24×24 | 20×20 | (50, 58) | — | +| 1 | Fog Theater | (28, 2) | 44×32 | 40×28 | (56, 18) | South | +| 2 | Occlusion Corridor | (74, 48) | 42×22 | 38×18 | (84, 58) | East | +| 3 | Inventory Warehouse | (2, 40) | 30×28 | 26×24 | (17, 54) | East | +| 4 | Interaction Gallery | (2, 82) | 24×20 | 20×16 | (14, 92) | East | +| 5 | Pause Chamber | (42, 78) | 16×16 | 12×12 | (50, 86) | North | +| 6 | Dialogue Room | (36, 104) | 28×20 | 24×16 | (50, 114) | North | +| 7 | Crowd Plaza | (80, 78) | 32×32 | 28×28 | (96, 94) | West | + +**Observer positions** are the fixed player positions used for golden file snapshots. Each room has exactly one canonical observer position. Golden files are generated by: place player at observer position → wait 1 tick → serialize ObserverSnapshot → that's the golden file. + +### Corridor Table + +Corridors are 6 sim tiles wide (3 visual tiles — player + 1 tile margin each side) and 10 sim tiles long (5 visual tiles). Walls on both sides. Zone boundary at midpoint (for D-073 zone crossfade testing when audio ships). + +| Corridor | Origin (x,y) | Size (w×h) | Connects | Reset Plate At | +|----------|-------------|-----------|----------|---------------| +| corridor-N | (47, 34) | 6×12 | Hub ↔ Fog Theater | (50, 34) | +| corridor-E | (62, 55) | 12×6 | Hub ↔ Occlusion Corridor | (73, 58) | +| corridor-W | (32, 55) | 6×6 | Hub ↔ Inventory Warehouse | (33, 58) | +| corridor-S | (47, 70) | 6×8 | Hub ↔ Pause Chamber | (50, 77) | +| corridor-S2 | (47, 94) | 6×10 | Pause Chamber ↔ Dialogue Room | (50, 103) | +| cross-cut-W | (26, 68) | 6×14 | Inventory ↔ Interaction Gallery | — | +| cross-cut-E | (80, 70) | 6×8 | Occlusion ↔ Crowd Plaza | — | + +**Reset plates are at room entrances** (corridor end closest to the room, not the hub). Cross-cuts do not have reset plates — they are shortcuts, not room entrances. + +### Overall Map Bounds + +- **Width:** 0 to 116 sim tiles (58 visual tiles, 29 meters) +- **Height:** 0 to 124 sim tiles (62 visual tiles, 31 meters) +- **Total walkable area:** ~2,400 sim tiles (estimated) +- **Total entities:** ~35 (see entity tables below) + +--- + +## 3. Entity Placement Per Room + +Every entity has a fixed position, a stable spawn order (determines StableId), and a defined initial state. Entities are listed in **canonical spawn order** — this is the order they appear in the Gauntlet YAML/builder, and thus the order they receive StableId assignments. Changing this order breaks golden files. + +### Room 0: Central Hub + +No test entities. The hub is a connector with directional markers. + +| Entity | Type | Position (relative) | Purpose | +|--------|------|-------------------|---------| +| sign_north | Object | (12, 2) | "Fog Theater →" marker | +| sign_east | Object | (22, 12) | "Occlusion Corridor →" marker | +| sign_south | Object | (12, 22) | "Pause Chamber →" marker | +| sign_west | Object | (2, 12) | "Inventory Warehouse →" marker | + +Hub spawn point: (12, 12) relative to room origin = (50, 58) absolute. + +### Room 1: Fog Theater + +Tests D-059 (5-layer fog), D-015 (vision cone), D-060 (cognitive delay in fog), D-035 (symmetric LOS). + +**Layout:** Three zones — open foyer (entry), narrowing corridor, enclosed back room. Fog thickens as player moves from foyer to back room. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_fog_near | NPC | (10, 14) | Neutral, visible from observer | Fog layer: Clear. Entity fully visible. | +| npc_fog_mid | NPC | (28, 14) | Unknown, in light fog from observer | Fog layer: Peripheral. Entity dimmed. | +| npc_fog_far | NPC | (38, 24) | Unknown, behind corridor wall | Fog layer: Deep fog. Entity not visible (wall blocks). | +| crate_fog | Object | (20, 10) | Floor item | Interaction in fog: can player interact in peripheral fog? | + +**Wall layout:** L-shaped wall at y=20, x=[24,40] creating a corridor. Back room behind the wall from x=30 to x=40, y=20 to y=28. + +**Golden file assertions:** +- From observer (28,16) facing South: npc_fog_near VISIBLE, npc_fog_mid VISIBLE (peripheral), npc_fog_far NOT VISIBLE (wall blocks), crate_fog VISIBLE. +- Fog tile counts: ~120 Clear, ~80 Peripheral, ~150 Deep, remainder Unexplored. + +### Room 2: Occlusion Corridor + +Tests D-035 (symmetric shadowcasting), D-017 (perception modes), D-015 (vision cone sectors). + +**Layout:** Long east-west corridor with perpendicular wall segments creating visibility pockets. North alcove with hidden NPC. South alcove with partially visible NPC. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_guard_visible | NPC | (18, 10) | Neutral | Clear LOS from observer. Baseline "visible entity." | +| npc_hidden_wall | NPC | (18, 4) | Unknown | Behind north wall segment. NOT visible in Visual mode. | +| npc_peripheral | NPC | (6, 16) | Unknown | In south alcove, peripheral sector from observer facing East. | +| npc_far_end | NPC | (36, 10) | Unknown | Far end of corridor. Tests LOS at range. | + +**Wall layout:** +- North wall segment: x=[14,22], y=[6,7] — blocks LOS to npc_hidden_wall from observer at (10,10). +- South alcove walls: x=[4,8], y=[12,13] — creates peripheral pocket. + +**Golden file assertions:** +- From observer (10,10) facing East: npc_guard_visible VISIBLE/Forward, npc_hidden_wall NOT VISIBLE (blocked), npc_peripheral VISIBLE/Peripheral, npc_far_end VISIBLE/Forward. +- LOS symmetry: from npc_guard_visible's position, the observer position should also be visible. + +### Room 3: Inventory Warehouse + +Tests D-065 (9-slot inventory), pickup/drop verbs, CarriedBy component. + +**Layout:** Open warehouse floor with 10 crates arranged in a grid. Player starts near the entrance with empty inventory. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| crate_01 | Object | (4, 4) | Item: "keycard" | First pickup. Slot assignment. | +| crate_02 | Object | (8, 4) | Item: "manifest" | Second pickup. | +| crate_03 | Object | (12, 4) | Item: "datapad" | Third pickup. | +| crate_04 | Object | (16, 4) | Item: "toolkit" | Fourth pickup. | +| crate_05 | Object | (4, 10) | Item: "badge" | Fifth pickup. | +| crate_06 | Object | (8, 10) | Item: "medkit" | Sixth pickup. | +| crate_07 | Object | (12, 10) | Item: "ration" | Seventh pickup. | +| crate_08 | Object | (16, 10) | Item: "cable" | Eighth pickup. | +| crate_09 | Object | (4, 16) | Item: "seal" | Ninth pickup — inventory now FULL (9/9). | +| crate_10 | Object | (8, 16) | Item: "chip" | Tenth — Take verb offered by server but should fail/grey-out. | +| npc_warehouse | NPC | (22, 14) | Neutral | Interaction target (Talk verb while carrying items). | + +**Golden file assertions:** +- From observer (15,14) facing East: all 10 crates visible, npc_warehouse visible. Inventory count 0/9. +- After taking 9 items: inventory count 9/9, all crates still in snapshot (Take verb still offered). + +### Room 4: Interaction Gallery + +Tests D-057 (vertical interaction list), D-056 (cursor states), D-058 (world menu), D-055 (sprint suppression). + +**Layout:** Gallery corridor with one entity per ObjectType in a line, plus a multi-verb NPC at the end. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_talker | NPC | (4, 8) | Neutral | Single verb: Talk. | +| obj_container | Object | (8, 8) | Container | Single verb: Take. | +| obj_terminal | Object | (12, 8) | Terminal | Single verb: Read. | +| obj_door | Object | (16, 8) | Door | Single verb: Open. | +| npc_multiverb | NPC | (4, 14) | Friendly | Multiple verbs: Talk(1), Observe(2), ExamineNpc(3). Tests verb priority ordering. | + +**Sprint entry zone:** Cross-cut-W enters at (20, 10). Player sprinting from Inventory through cross-cut arrives here. Interaction buffer must be empty during sprint, then repopulate within 1 tick of stance change to Walk. + +**Golden file assertions:** +- From observer (12,10) facing East: all 5 entities visible. Interaction list shows closest entities with correct verb sets. + +### Room 5: Pause Chamber + +Tests D-031 (pause/unpause), Bug #3 regression (movement while paused). + +**Layout:** Minimal open room with a single NPC. No visual complexity — this room is pure state machine testing. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_pause_target | NPC | (6, 6) | Neutral | Interaction target during pause (Talk should work). | + +**Test scenarios (Dudley's 8 tests):** +1. Movement discarded while paused +2. Unpause accepted while paused +3. Stance toggle allowed while paused +4. Interact allowed while paused +5. Multiple movements in paused batch all discarded +6. Pause → move (discarded) → unpause → move (accepted) roundtrip +7. SetTickRate(Half) while paused behavior +8. Perception mode toggle during pause (passthrough) + +**Golden file assertions:** +- From observer (8,8) facing North: npc_pause_target visible. TickRate: Full. +- Paused golden file: identical entity positions, TickRate: Paused. + +### Room 6: Dialogue Room + +Tests D-041 (knowledge graph), D-028 (tagged dialogue pools), D-033 (entity color = relationship), D-062 (invisible locked options), D-063 (confrontation same box), D-064 (walk-away consequences). + +**Layout:** Waiting room layout — NPCs spaced apart at different trust tiers. One NPC has pre-loaded contradictory KG entries. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_stranger | NPC | (4, 8) | Unknown, Trust Tier 1 | Dialogue: access-tier lines only. Color: Unknown (#4a9ebb). | +| npc_acquaintance | NPC | (12, 8) | Neutral, Trust Tier 2 | Dialogue: relationship-history lines available. | +| npc_trusted | NPC | (20, 8) | Friendly, Trust Tier 3 | Dialogue: trust-gated gossip available. Color: Friendly (#6bc9a6). | +| npc_contradiction | NPC | (12, 14) | Neutral, KG: 2 contradicting facts | Dialogue triggers contradiction detection. Color should shift to POI (#e8c547). Monologue fires: "Wait — they said..." | + +**KG injection for npc_contradiction:** +``` +knowledge_entries: + - fact_id: "test.manifests_clean" + confidence: KnowsDetails + source: npc_stranger + - fact_id: "test.manifests_tampered" + confidence: KnowsDetails + source: npc_trusted +``` +These contradict → Contradicted state after player learns both. + +**Golden file assertions:** +- From observer (14,10) facing North: all 4 NPCs visible with correct relationship colors. No active dialogue. +- After dialogue with npc_contradiction and discovering both facts: entity color transitions to POI amber. + +### Room 7: Crowd Plaza + +Tests entity density, D-033 (color spread at scale), D-060 (cognitive delay overlap), determinism (INV-T01), tick budget (INV-T02), snapshot delivery (INV-T03). + +**Layout:** Large open area with 15 NPCs at varied positions and relationship states. + +| Entity | Type | Rel. Position | Initial State | Test Purpose | +|--------|------|--------------|---------------|-------------| +| npc_crowd_01 | NPC | (4, 4) | Friendly | D-033 color: #6bc9a6 | +| npc_crowd_02 | NPC | (10, 4) | Hostile | D-033 color: #d45d5d | +| npc_crowd_03 | NPC | (16, 4) | Unknown | D-033 color: #4a9ebb | +| npc_crowd_04 | NPC | (22, 4) | Neutral | | +| npc_crowd_05 | NPC | (28, 4) | Friendly | | +| npc_crowd_06 | NPC | (4, 10) | Unknown | | +| npc_crowd_07 | NPC | (10, 10) | Neutral | | +| npc_crowd_08 | NPC | (16, 10) | Hostile | | +| npc_crowd_09 | NPC | (22, 10) | Friendly | | +| npc_crowd_10 | NPC | (28, 10) | Unknown | | +| npc_crowd_11 | NPC | (4, 16) | Neutral | | +| npc_crowd_12 | NPC | (10, 16) | Friendly | | +| npc_crowd_13 | NPC | (16, 16) | Hostile | D-033: 3 hostile in room, color spread check | +| npc_crowd_14 | NPC | (22, 16) | Unknown | | +| npc_crowd_15 | NPC | (28, 16) | Neutral | | + +**Relationship distribution:** 4 Friendly, 3 Hostile, 4 Unknown, 4 Neutral. Tests D-033 color spread — all 4 relationship colors visible simultaneously. + +**Stress test assertions:** +- **Determinism (INV-T01):** Run with seed 42, capture golden file at tick 10. Run again with seed 42 and identical inputs. Golden files must be byte-identical. +- **Tick budget (INV-T02):** 100 ticks with 15 NPCs. p95 tick time < 20ms (Tyre's threshold). p99 < 50ms (hard alarm). +- **Snapshot delivery (INV-T03):** Every tick produces exactly one ObserverSnapshot. No drops, no duplicates, monotonically increasing tick numbers. + +--- + +## 4. Cross-Room Transitions — Sprint 8 MVP + +3 transitions. Selected for maximum bug coverage with minimum test infrastructure. + +### T1: Sprint Exit (Crowd Plaza → Occlusion Corridor) + +**Path:** Crowd Plaza → cross-cut-E → Occlusion Corridor entrance + +**System combination:** D-055 (sprint suppression) + D-035 (LOS recalculation) + +**Test script:** +``` +spawn_at crowd_plaza.observer +set_stance Sprint +move_toward cross-cut-E entrance +# During sprint: verify +assert snapshot.nearby_interactions == [] # buffer cleared +assert snapshot.player_stance == Sprint +# Enter Occlusion Corridor +move_to occlusion_corridor.entrance +set_stance Walk +wait 1 tick +assert snapshot.nearby_interactions.len() > 0 # verbs repopulate +assert npc_guard_visible in snapshot.entities # LOS recomputed +``` + +**Why Sprint 8:** Both sprint and LOS are implemented. Tests the most common player behavior pattern. Uses cross-cut-E which physically connects these rooms. + +### T3: Full Inventory Interact (Inventory → Interaction Gallery) + +**Path:** Inventory Warehouse → cross-cut-W → Interaction Gallery + +**System combination:** D-065 (inventory limit) + D-057 (verb computation) + +**Test script:** +``` +spawn_at inventory_warehouse.observer +# Fill inventory to 9/9 +repeat 9: interact crate_N Take +assert snapshot.inventory.count == 9 +# Walk to Interaction Gallery +move_through cross-cut-W +move_to interaction_gallery.obj_container +# Verify Take verb behavior at 9/9 +assert "Take" in snapshot.nearby_interactions[obj_container].verbs +# Server still offers Take — it's the CLIENT that should grey it out +# (Server doesn't filter Phase 1 verbs based on inventory state) +``` + +**Why Sprint 8:** Inventory and interaction are vertical slice systems. This tests the boundary between "server offers" and "client filters" — a class of information boundary bugs. + +### T6: Pause Anywhere (Pause Chamber → Hub → any room) + +**Path:** Pause Chamber → corridor-S → Hub → corridor to any room → unpause + +**System combination:** D-031 (pause) + any room's systems + +**Test script:** +``` +spawn_at pause_chamber.observer +send Pause +assert snapshot.tick_rate == Paused +# Teleport to Hub (or walk) +send TeleportToHub +assert player_position == hub.spawn +# Move into Fog Theater +move_to fog_theater.entrance +# Still paused — fog state should be frozen +send MoveNorth +assert player_position unchanged # movement discarded +send Unpause +assert snapshot.tick_rate == Full +send MoveNorth +assert player_position changed # movement now works +# Verify fog theater state is coherent (fog tiles computed correctly) +assert snapshot.visible_tiles.len() > 0 +``` + +**Why Sprint 8:** Bug #3 regression guard. Tests that pause state persists across room transitions and teleports. Any state corruption during pause-while-transitioning would be caught. + +### Deferred Transitions + +| # | Transition | Deferral Reason | +|---|-----------|----------------| +| T2 | Fog into Dialogue | Cognitive delay during dialogue not yet wired | +| T4 | Sprint into Interaction | Covered by T1 + T3 | +| T5 | Confrontation to Eavesdrop | Both rooms deferred | +| T7 | Sound across Fog | Sound propagation not implemented | +| T8 | Walk-away Sprint | Partially covered by T6 + Dialogue Room unit tests | + +--- + +## 5. Anti-Tedium — Sprint 8 MVP + +### Ships Sprint 8 + +| Feature | Priority | Effort | Justification | +|---------|----------|--------|--------------| +| **Reset Plate** | Essential | ~1 day (Dudley) | Without it, every test iteration requires server restart. Ozzie's UX flow depends on it. Already fully specified in Round 2. | +| **Hub Teleport** | Essential | ~0.5 day (Dudley + Stig) | Saves 30+ seconds per room switch. PlayerAction::TeleportToHub + Home key binding. Already fully specified. | + +### Deferred + +| Feature | Priority | Effort | Deferral Reason | +|---------|----------|--------|----------------| +| **WRONG Button (F12)** | Sprint 9 | ~1.5 days | Requires ring buffer, file output system, and text renderer integration. Text renderer ships Sprint 8 but WRONG button integration is Sprint 9 scope. Test client `--json` provides partial coverage in the interim. | +| **Room Timer + PBs** | Sprint 9 | ~0.5 day | Nice-to-have gamification. Zero impact on test coverage. | +| **F3 Debug Overlay** | Sprint 10+ | ~2 days | Per Stig's recommendation: WRONG button captures same data on-demand. F3 as real-time overlay has measurable performance cost. | + +### Reset Plate — Final Spec Confirmation + +Per Round 2 consensus (Gestalt + Ozzie + Dudley + Stig): + +- **Trigger:** Step on ResetPlate tile + press Interact (NOT automatic) +- **Server:** `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 per room), `execute_room_reset` system +- **Resets:** Entity positions, entity KG, player KG (room refs only), fog (room tiles), room-sourced inventory items, dialogue state +- **Does NOT reset:** Other rooms, player position, session/tick counter, global SimRng state +- **Debounce:** 10-tick cooldown after trigger +- **Test-mode only:** `RoomResetTrigger` entities only added with `--test-mode` + +### Hub Teleport — Final Spec Confirmation + +- **Trigger:** Home key → `PlayerAction::TeleportToHub` +- **Server:** Move player to `GAUNTLET.hub_spawn`, clear dialogue/monologue/interaction buffer +- **Does NOT affect:** Room state, inventory, game time, knowledge graph +- **Gauntlet-only:** Server rejects TeleportToHub in non-Gauntlet maps + +--- + +## 6. Remaining Questions — Answers + +### Q2: Per-Room Reset vs Server Restart + +**Answer: Both. Different test types need different reset levels.** + +| Reset Level | Mechanism | When to Use | +|------------|-----------|-------------| +| **Room reset** (Reset Plate) | Restore room entities to tick-0 state. Player stays. Global state preserved. | Manual testing iteration. Checklist retries. Cross-room transitions (reset source room, re-test). | +| **Full restart** (`make test-world-headless`) | Kill server process, relaunch. Tick counter at 0, fresh SimRng, fresh TCP connection. | Determinism replay tests (INV-T01). Layer 3 subprocess tests. Golden file generation. Any test where SimRng position matters. | + +**Tests requiring full restart:** +- INV-T01 (deterministic replay) — SimRng position must match exactly +- INV-T03 (snapshot delivery) — tests TCP connection lifecycle +- Layer 3 subprocess test — spawns fresh server process by definition +- Golden file regeneration — must produce identical output from tick 0 +- Performance benchmarks — clean process, no accumulated state + +**Tests where room reset is sufficient:** +- All per-room checklist items +- All cross-room transition scenarios (T1, T3, T6) +- Manual exploratory testing +- Bug reproduction attempts (after capturing with WRONG button) + +**No separate "full reset" command needed.** `make test-world-headless` is the full restart. Reset plates handle in-session resets. These two mechanisms cover all use cases. + +### Q3: Cross-Cut Count — MVP + +**Answer: 2 cross-cuts for Sprint 8.** + +| Cross-Cut | Connects | Transition Scenario | Width | +|-----------|----------|-------------------|-------| +| cross-cut-W | Inventory ↔ Interaction Gallery | T3 (Full Inventory Interact) | 6×14 sim tiles | +| cross-cut-E | Occlusion ↔ Crowd Plaza | T1 (Sprint Exit) | 6×8 sim tiles | + +**Justification:** Each cross-cut serves a specific Sprint 8 transition scenario. Each connects rooms that test complementary systems (inventory+interaction, LOS+density). More cross-cuts add map complexity without adding test coverage for Sprint 8 systems. + +**Deferred cross-cuts (Sprint 9+):** + +| Cross-Cut | Connects | Ships When | +|-----------|----------|-----------| +| Sound Lab ↔ Fog Theater | Sound propagation through fog | Audio system implementation | +| Confrontation Stage ↔ Eavesdrop Alcove | Emotional state transitions | D-070/D-071 implementation | +| Sprint Gauntlet ↔ Interaction Gallery | Full sprint-to-interaction chain | Sprint Gauntlet room ships | + +**Sound Lab consolidation note:** When sound propagation ships, evaluate whether a "sound section" can be added to the Occlusion Corridor (as a sub-area with sound sources behind walls) rather than building a separate room. If the system is simple enough, the sub-area approach saves a room + cross-cut. If it needs dedicated space, add Sound Lab as room #8 with its own cross-cut. + +### Q6: Cross-Room Checklist Location + +**Answer: `content/gauntlet/cross_room_checks.yaml` at the Gauntlet root level.** + +Cross-room transitions don't belong in per-room checklists because they span multiple rooms and test system *combinations*, not individual systems. + +```yaml +# content/gauntlet/cross_room_checks.yaml +transitions: + - id: t1_sprint_exit + description: "Sprint from Crowd Plaza through cross-cut to Occlusion Corridor" + source_room: crowd_plaza + target_room: occlusion_corridor + path: [crowd_plaza, cross_cut_e, occlusion_corridor] + checks: + - id: t1_01_buffer_cleared + description: "Interaction buffer empty during sprint through cross-cut" + type: auto + condition: + player_stance: Sprint + expected_interactions: empty + if_wrong: | + Sprint suppression not clearing buffer on room transition. + Check: D-055 interaction buffer clear in bridge/mod.rs. + + - id: t1_02_los_recomputed + description: "LOS recomputed on entering Occlusion Corridor" + type: auto + condition: + player_room: occlusion_corridor + player_stance: Walk + entity: npc_guard_visible + expected: visible + if_wrong: | + LOS not recomputed after room transition. + Check: compute_observer_snapshot runs after validate_movement. + + - id: t3_full_inventory_interact + description: "Carry full inventory from Warehouse to Interaction Gallery" + source_room: inventory_warehouse + target_room: interaction_gallery + path: [inventory_warehouse, cross_cut_w, interaction_gallery] + checks: + - id: t3_01_take_still_offered + description: "Server still offers Take verb at 9/9 inventory" + type: auto + condition: + inventory_count: 9 + entity: obj_container + verb: Take + expected: present + if_wrong: | + Server filtering Take based on inventory state. + Phase 1 (ObjectType) verbs should not check inventory. + Check: interaction.rs verb_computation. + + - id: t6_pause_anywhere + description: "Pause in Pause Chamber, teleport, verify pause persists" + source_room: pause_chamber + target_room: fog_theater + path: [pause_chamber, hub, fog_theater] + checks: + - id: t6_01_pause_survives_teleport + description: "TickRate stays Paused after Hub teleport" + type: auto + condition: + tick_rate: Paused + player_room: hub + if_wrong: | + TeleportToHub resetting tick rate. Check: teleport handler + should NOT modify TickRate. + + - id: t6_02_movement_blocked_in_new_room + description: "Movement discarded in Fog Theater while paused" + type: auto + condition: + tick_rate: Paused + player_room: fog_theater + movement_input: discarded + if_wrong: | + Pause guard not checking TickRate after room change. + Check: process_player_input guard at input.rs:102-105. +``` + +`make checklist` generates both per-room and cross-room sections. The test client loads cross_room_checks.yaml when running transition scenarios. + +--- + +## 7. Gauntlet Constants Module + +For Dudley's implementation. This module is the single source of truth for room geometry, entity positions, and observer locations. + +```rust +// server/src/test_world/constants.rs + +use crate::simulation::movement::TilePosition; + +/// A Gauntlet room definition. +pub struct GauntletRoom { + pub name: &'static str, + pub origin: TilePosition, // top-left corner (includes walls) + pub size: (i32, i32), // (width, height) in sim tiles + pub observer: TilePosition, // golden file observer position + pub observer_facing: Facing, // direction observer faces + pub reset_plate: Option, // reset plate location (if any) +} + +/// A Gauntlet entity definition. +pub struct GauntletEntity { + pub name: &'static str, + pub room: &'static str, + pub position: TilePosition, // relative to room origin + pub kind: EntityKind, + pub relationship: RelationshipState, +} + +/// Canonical room definitions. +pub const HUB: GauntletRoom = GauntletRoom { + name: "central_hub", + origin: TilePosition::new(38, 46, 0), + size: (24, 24), + observer: TilePosition::new(50, 58, 0), + observer_facing: Facing::North, + reset_plate: None, +}; + +pub const FOG_THEATER: GauntletRoom = GauntletRoom { + name: "fog_theater", + origin: TilePosition::new(28, 2, 0), + size: (44, 32), + observer: TilePosition::new(56, 18, 0), + observer_facing: Facing::South, + reset_plate: Some(TilePosition::new(50, 34, 0)), +}; + +pub const OCCLUSION_CORRIDOR: GauntletRoom = GauntletRoom { + name: "occlusion_corridor", + origin: TilePosition::new(74, 48, 0), + size: (42, 22), + observer: TilePosition::new(84, 58, 0), + observer_facing: Facing::East, + reset_plate: Some(TilePosition::new(73, 58, 0)), +}; + +pub const INVENTORY_WAREHOUSE: GauntletRoom = GauntletRoom { + name: "inventory_warehouse", + origin: TilePosition::new(2, 40, 0), + size: (30, 28), + observer: TilePosition::new(17, 54, 0), + observer_facing: Facing::East, + reset_plate: Some(TilePosition::new(33, 58, 0)), +}; + +pub const INTERACTION_GALLERY: GauntletRoom = GauntletRoom { + name: "interaction_gallery", + origin: TilePosition::new(2, 82, 0), + size: (24, 20), + observer: TilePosition::new(14, 92, 0), + observer_facing: Facing::East, + reset_plate: Some(TilePosition::new(14, 82, 0)), +}; + +pub const PAUSE_CHAMBER: GauntletRoom = GauntletRoom { + name: "pause_chamber", + origin: TilePosition::new(42, 78, 0), + size: (16, 16), + observer: TilePosition::new(50, 86, 0), + observer_facing: Facing::North, + reset_plate: Some(TilePosition::new(50, 77, 0)), +}; + +pub const DIALOGUE_ROOM: GauntletRoom = GauntletRoom { + name: "dialogue_room", + origin: TilePosition::new(36, 104, 0), + size: (28, 20), + observer: TilePosition::new(50, 114, 0), + observer_facing: Facing::North, + reset_plate: Some(TilePosition::new(50, 103, 0)), +}; + +pub const CROWD_PLAZA: GauntletRoom = GauntletRoom { + name: "crowd_plaza", + origin: TilePosition::new(80, 78, 0), + size: (32, 32), + observer: TilePosition::new(96, 94, 0), + observer_facing: Facing::West, + reset_plate: Some(TilePosition::new(80, 86, 0)), +}; + +/// All rooms in canonical spawn order. +/// THIS ORDER DETERMINES STABLEID ASSIGNMENT. +/// Do not reorder existing entries. Append new rooms at the end. +pub const ROOMS: &[GauntletRoom] = &[ + HUB, + FOG_THEATER, + OCCLUSION_CORRIDOR, + INVENTORY_WAREHOUSE, + INTERACTION_GALLERY, + PAUSE_CHAMBER, + DIALOGUE_ROOM, + CROWD_PLAZA, +]; + +/// Look up which room a position falls in. +pub fn room_at(pos: &TilePosition) -> Option<&'static GauntletRoom> { + ROOMS.iter().find(|r| { + pos.x >= r.origin.x && pos.x < r.origin.x + r.size.0 + && pos.y >= r.origin.y && pos.y < r.origin.y + r.size.1 + && pos.z == r.origin.z + }) +} +``` + +**Canonical spawn order (answer to R2-OQ-09):** Rooms are spawned in the order listed in `ROOMS`. Within each room, entities are spawned in the order listed in the room's entity table (Section 3). This order is stable — appending new rooms or entities at the end does not change existing StableId assignments. Do not reorder. + +--- + +## 8. Room Ordering and StableId Assignment + +Entities receive StableIds in spawn order. The Gauntlet's spawn order is: + +1. Hub entities (signs): StableId 1-4 +2. Fog Theater entities: StableId 5-8 +3. Occlusion Corridor entities: StableId 9-12 +4. Inventory Warehouse entities: StableId 13-23 (10 crates + 1 NPC) +5. Interaction Gallery entities: StableId 24-28 +6. Pause Chamber entities: StableId 29 +7. Dialogue Room entities: StableId 30-33 +8. Crowd Plaza entities: StableId 34-48 + +**Total: 48 entities.** Player entity gets its own StableId assignment (typically 0 or assigned by the bridge). + +**Golden file implications:** Golden files reference entities by StableId. If spawn order changes, all golden files break. The constants module is the single source of truth — any golden file regeneration is triggered by changes to `constants.rs`. + +--- + +## 9. Sprint 8 Implementation Priorities + +What Dudley should build first, based on test dependency chains: + +| Priority | What | Depends On | Enables | +|----------|------|-----------|---------| +| **P0** | `--test-mode` + `--port 0` | Nothing | Layer 3 test, test client, all Gauntlet testing | +| **P0** | Gauntlet constants module (`constants.rs`) | Nothing | All room builders, golden files, test client room detection | +| **P1** | Hub + Pause Chamber builder | constants.rs | Pause guard tests (Bug #3 regression), first golden file | +| **P1** | 4 determinism fixes (Fix A-D) | Nothing | INV-T01, golden file stability | +| **P2** | Occlusion Corridor builder | constants.rs | LOS tests, perception mode tests | +| **P2** | Inventory Warehouse builder | constants.rs | Inventory tests, T3 transition | +| **P2** | Interaction Gallery builder | constants.rs | Verb tests, T3 transition | +| **P2** | Fog Theater builder | constants.rs | Fog tests | +| **P2** | Dialogue Room builder | constants.rs + KG injection | Dialogue tests, contradiction | +| **P2** | Crowd Plaza builder | constants.rs | Density tests, determinism, tick budget | +| **P3** | Reset plate system | Room builders | Manual testing iteration | +| **P3** | Hub teleport handler | constants.rs | Manual testing efficiency | +| **P3** | Corridor + cross-cut geometry | Room builders | Cross-room transitions | + +**Critical path:** `--test-mode` → constants module → Hub + Pause Chamber → first golden file. Everything else can parallelize after that. + +--- + +## Summary + +| Deliverable | Status | +|-------------|--------| +| MVP room list (7 rooms + hub) with justification | Complete | +| Physical map layout with ASCII art | Complete | +| Coordinate table (rooms, corridors, cross-cuts) | Complete | +| Entity placement per room (48 entities) | Complete | +| Observer positions for golden files | Complete | +| 3 MVP cross-room transitions with test scripts | Complete | +| Anti-tedium MVP (Reset Plate + Hub Teleport) | Confirmed | +| Q2 answer: room reset vs full restart | Complete | +| Q3 answer: 2 cross-cuts for MVP | Complete | +| Q6 answer: cross-room checks at Gauntlet root YAML | Complete | +| Constants module specification | Complete | +| StableId assignment order | Complete | +| Sprint 8 implementation priorities | Complete | diff --git a/docs/workshops/test-architecture/hoshe-round1.md b/docs/workshops/test-architecture/hoshe-round1.md new file mode 100644 index 000000000..f1af3a762 --- /dev/null +++ b/docs/workshops/test-architecture/hoshe-round1.md @@ -0,0 +1,747 @@ +# Hoshe — Round 1 Analysis: Boundary Values, Layer 3, CI, Content Validation + +**Workshop:** QA Strategy & Test Architecture +**Tracks:** 4 (Serialization & Integration) + 5 (Content Scaling & CI) +**Date:** 2026-02-17 +**Spec references:** D-030 (testability architecture), D-020 (IPC/MessagePack protocol) + +--- + +## TRACK 4 — Serialization & Integration + +### T4-H1: MessagePack Boundary Value Test Matrix + +The GDScript encoder (`messagepack.gd:69-95`) uses cascading `if/elif` branches to select the MessagePack integer format. Each branch boundary is a potential off-by-one bug site — exactly where bug #4 lived. + +I traced every branch transition against the MessagePack spec. The encoder's branch order determines the effective ranges: + +| Branch | Condition (GDScript) | Effective range (after earlier branches steal values) | MsgPack format | Header byte | +|--------|---------------------|------------------------------------------------------|---------------|-------------| +| 1 | `-32 <= v <= 127` | -32 to 127 | positive fixint / negative fixint | 0x00-0x7f / 0xe0-0xff | +| 2 | `-128 <= v < 128` | -128 to -33 | int 8 | 0xd0 | +| 3 | `0 <= v <= 255` | 128 to 255 | uint 8 | 0xcc | +| 4 | `-32768 <= v < 32768` | -32768 to -129 AND 256 to 32767 | int 16 | 0xd1 | +| 5 | `0 <= v <= 65535` | 32768 to 65535 | uint 16 | 0xcd | +| 6 | `-2^31 <= v < 2^31` | -2147483648 to -32769 AND 65536 to 2147483647 | int 32 | 0xd2 | +| 7 | `0 <= v <= 2^32-1` | 2147483648 to 4294967295 | uint 32 | 0xce | +| 8 | `-2^63 <= v < 2^63` | -9223372036854775808 to -2147483649 AND 4294967296 to 9223372036854775807 | int 64 | 0xd3 | +| 9 | else | Godot `int` is 64-bit signed, so this branch is unreachable for valid Godot ints | uint 64 | 0xcf | + +**Note on branch 4:** Positive values 256-32767 are encoded as int_16, not uint_16. This is spec-valid (MessagePack allows any format that fits the value) but differs from what rmp_serde produces for unsigned Rust types. Cross-language roundtrip tests MUST cover this divergence — rmp_serde must accept int_16-encoded positive values when deserializing into `u64`. + +#### Complete Boundary Value Matrix + +Every row is a test case. Values are chosen at format transition boundaries (value-1, value, value+1). + +| Test ID | Value | Expected format | Header byte | Payload bytes | Why this value matters | +|---------|-------|----------------|-------------|---------------|----------------------| +| **Positive fixint boundaries** | +| BV-P01 | 0 | pos fixint | 0x00 | (none) | Zero — minimum positive fixint | +| BV-P02 | 1 | pos fixint | 0x01 | (none) | Smallest nonzero positive | +| BV-P03 | 126 | pos fixint | 0x7e | (none) | One below boundary | +| BV-P04 | 127 | pos fixint | 0x7f | (none) | **MAX positive fixint** — Bug #4 was here | +| **fixint → uint_8 transition** | +| BV-P05 | 128 | uint 8 | 0xcc | 0x80 | **MIN uint_8** — Bug #4: this was encoded as -128 | +| BV-P06 | 129 | uint 8 | 0xcc | 0x81 | One above boundary | +| BV-P07 | 254 | uint 8 | 0xcc | 0xfe | One below max uint_8 | +| BV-P08 | 255 | uint 8 | 0xcc | 0xff | **MAX uint_8** | +| **uint_8 → int_16 transition** (NOTE: encoder uses int_16, not uint_16, for 256-32767) | +| BV-P09 | 256 | int 16 | 0xd1 | 0x01 0x00 | **MIN int_16 positive** — format widens to 2 bytes | +| BV-P10 | 257 | int 16 | 0xd1 | 0x01 0x01 | One above boundary | +| BV-P11 | 32766 | int 16 | 0xd1 | 0x7f 0xfe | One below max int_16 | +| BV-P12 | 32767 | int 16 | 0xd1 | 0x7f 0xff | **MAX int_16 positive** | +| **int_16 → uint_16 transition** | +| BV-P13 | 32768 | uint 16 | 0xcd | 0x80 0x00 | **MIN uint_16** — exceeds int_16 max | +| BV-P14 | 32769 | uint 16 | 0xcd | 0x80 0x01 | One above boundary | +| BV-P15 | 65534 | uint 16 | 0xcd | 0xff 0xfe | One below max uint_16 | +| BV-P16 | 65535 | uint 16 | 0xcd | 0xff 0xff | **MAX uint_16** | +| **uint_16 → int_32 transition** | +| BV-P17 | 65536 | int 32 | 0xd2 | 0x00 0x01 0x00 0x00 | **MIN int_32 positive** | +| BV-P18 | 65537 | int 32 | 0xd2 | 0x00 0x01 0x00 0x01 | One above boundary | +| BV-P19 | 2147483646 | int 32 | 0xd2 | 0x7f 0xff 0xff 0xfe | One below max int_32 | +| BV-P20 | 2147483647 | int 32 | 0xd2 | 0x7f 0xff 0xff 0xff | **MAX int_32** (2^31-1) | +| **int_32 → uint_32 transition** | +| BV-P21 | 2147483648 | uint 32 | 0xce | 0x80 0x00 0x00 0x00 | **MIN uint_32** (2^31) | +| BV-P22 | 4294967294 | uint 32 | 0xce | 0xff 0xff 0xff 0xfe | One below max uint_32 | +| BV-P23 | 4294967295 | uint 32 | 0xce | 0xff 0xff 0xff 0xff | **MAX uint_32** (2^32-1) | +| **uint_32 → int_64 transition** | +| BV-P24 | 4294967296 | int 64 | 0xd3 | 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 | **MIN int_64 positive** (2^32) | +| BV-P25 | 9223372036854775807 | int 64 | 0xd3 | 0x7f 0xff 0xff 0xff 0xff 0xff 0xff 0xff | **MAX int_64** (2^63-1, Godot max int) | +| **Negative fixint boundaries** | +| BV-N01 | -1 | neg fixint | 0xff | (none) | Most common negative value | +| BV-N02 | -31 | neg fixint | 0xe1 | (none) | One above min fixint | +| BV-N03 | -32 | neg fixint | 0xe0 | (none) | **MIN negative fixint** | +| **fixint → int_8 transition** | +| BV-N04 | -33 | int 8 | 0xd0 | 0xdf | **Exceeds fixint** — first value requiring int_8 | +| BV-N05 | -34 | int 8 | 0xd0 | 0xde | One below boundary | +| BV-N06 | -127 | int 8 | 0xd0 | 0x81 | One above min int_8 | +| BV-N07 | -128 | int 8 | 0xd0 | 0x80 | **MIN int_8** | +| **int_8 → int_16 transition** | +| BV-N08 | -129 | int 16 | 0xd1 | 0xff 0x7f | **Exceeds int_8** — first value requiring int_16 | +| BV-N09 | -130 | int 16 | 0xd1 | 0xff 0x7e | One below boundary | +| BV-N10 | -32767 | int 16 | 0xd1 | 0x80 0x01 | One above min int_16 | +| BV-N11 | -32768 | int 16 | 0xd1 | 0x80 0x00 | **MIN int_16** | +| **int_16 → int_32 transition** | +| BV-N12 | -32769 | int 32 | 0xd2 | 0xff 0xff 0x7f 0xff | **Exceeds int_16** | +| BV-N13 | -2147483647 | int 32 | 0xd2 | 0x80 0x00 0x00 0x01 | One above min int_32 | +| BV-N14 | -2147483648 | int 32 | 0xd2 | 0x80 0x00 0x00 0x00 | **MIN int_32** (-2^31) | +| **int_32 → int_64 transition** | +| BV-N15 | -2147483649 | int 64 | 0xd3 | 0xff 0xff 0xff 0xff 0x7f 0xff 0xff 0xff | **Exceeds int_32** (-2^31-1) | +| BV-N16 | -9223372036854775808 | int 64 | 0xd3 | 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00 | **MIN int_64** (-2^63, Godot min int) | + +**Total: 41 boundary values.** + +#### Game-relevant values (sanity check) + +These specific values appear in the protocol and must be tested even though they may not sit on format boundaries: + +| Value | Protocol field | Expected format | +|-------|---------------|----------------| +| 0 | `tick` at game start | pos fixint | +| 10 | `tick` at 1 second | pos fixint | +| 42 | `tick` in test fixtures | pos fixint | +| 128 | `tick` at ~13 seconds — **BUG #4 SITE** | uint 8 | +| 500 | `tick` in v2_full fixture | int 16 (via branch 4) | +| 999 | `tick` in multi_entity fixture | int 16 | +| 100 | `entity_id` in test fixtures | pos fixint | +| 720 | `time_of_day` (noon) | int 16 | +| 1440 | `time_of_day` (end of day) | int 16 | + +#### Observation: Asymmetric encoding between GDScript and Rust + +The GDScript encoder picks int_16 for values 256-32767 (positive values in the signed range). The Rust encoder (rmp_serde) picks uint_16 for the same values. Both are valid per the MessagePack spec, but this means GDScript-encoded bytes and Rust-encoded bytes for the same logical value will differ at the byte level. + +**This is NOT a bug**, but it means: +- Byte-for-byte golden file comparison between GDScript-encoded and Rust-encoded data will FAIL for values 256-32767 (and similarly for 65536-2^31-1). +- Golden files must be direction-specific: "Rust encodes this snapshot" (canonical) vs "GDScript encodes this input" (canonical). +- Decoders on both sides MUST accept both signed and unsigned encodings for the same value. Tests should verify this explicitly. + +--- + +### T4-H2: Where Should Boundary Tests Live? + +**Recommendation: Both, but with different scopes.** + +| Layer | Location | What it tests | Speed | Frequency | +|-------|----------|--------------|-------|-----------| +| **Encode-only** (GDScript) | `client/tests/test_msgpack_boundaries.gd` | GDScript encoder produces correct format for each boundary value | Fast (~100ms) | Every commit | +| **Encode-only** (Rust) | `server/tests/serialization.rs` (extend) | Rust encoder produces correct format for each boundary value | Fast (~50ms) | Every commit | +| **Decode cross-language** | Fixtures: Rust-generated `.msgpack` files decoded by GDScript | GDScript decoder handles Rust-encoded boundary values | Medium (~2s) | Every PR | +| **Decode cross-language** | Fixtures: GDScript-generated `.msgpack` files decoded by Rust | Rust decoder handles GDScript-encoded boundary values | Medium (~2s) | Every PR | +| **Full roundtrip** | `bridge_tcp.rs` extension: send boundary-valued snapshots over TCP, receive on GDScript side | End-to-end wire format agreement | Slow (~5s) | Nightly / pre-merge | + +**Rationale:** Bug #4 was an encode-side bug (GDScript encoder chose wrong format). A GDScript-only encode test would have caught it. But the WORST bugs are decode mismatches — where one side encodes a value that the other side decodes as a different value. Cross-language fixtures catch those. The roundtrip catches protocol-level issues (framing, batching). + +**Concrete test functions:** + +GDScript (`test_msgpack_boundaries.gd`): +```gdscript +func test_boundary_positive_fixint_max() -> void: + # BV-P04: 127 must encode as single byte 0x7f (positive fixint) + var result = Messagepack.encode(127) + assert_that(result.value.size()).is_equal(1) + assert_that(result.value[0]).is_equal(0x7f) + +func test_boundary_uint8_min() -> void: + # BV-P05: 128 must encode as uint_8 (0xcc, 0x80) — Bug #4 regression guard + var result = Messagepack.encode(128) + assert_that(result.value.size()).is_equal(2) + assert_that(result.value[0]).is_equal(0xcc) + assert_that(result.value[1]).is_equal(0x80) + +func test_boundary_roundtrip_all() -> void: + # Every boundary value must survive encode→decode roundtrip + var boundaries = [0, 1, 126, 127, 128, 129, 254, 255, 256, 257, + 32766, 32767, 32768, 32769, 65534, 65535, 65536, + -1, -31, -32, -33, -34, -127, -128, -129, -130, + -32767, -32768, -32769] + for v in boundaries: + var encoded = Messagepack.encode(v) + assert_that(encoded.status).is_null() + var decoded = Messagepack.decode(encoded.value) + assert_that(decoded.status).is_null() + assert_that(decoded.value).is_equal(v) +``` + +Rust (`serialization.rs` addition): +```rust +#[test] +fn boundary_values_roundtrip() { + let boundaries: Vec = vec![ + 0, 1, 126, 127, 128, 129, 254, 255, 256, 257, + 32766, 32767, 32768, 32769, 65534, 65535, 65536, 65537, + 2147483646, 2147483647, 2147483648, 4294967294, 4294967295, 4294967296, + -1, -31, -32, -33, -34, -127, -128, -129, -130, + -32767, -32768, -32769, + -2147483647, -2147483648, -2147483649, + ]; + for val in boundaries { + let bytes = rmp_serde::to_vec(&val).unwrap(); + let decoded: i64 = rmp_serde::from_slice(&bytes).unwrap(); + assert_eq!(decoded, val, "boundary value {} did not roundtrip", val); + } +} +``` + +--- + +### T4-H3: Should gen_fixtures.rs Generate Boundary Value Fixtures? + +**Yes. Concrete proposal:** + +Add a `generate_boundary_fixtures()` function to `gen_fixtures.rs` that generates two types of fixtures: + +1. **Raw integer boundary fixtures** — one `.msgpack` file per boundary value, containing just the encoded integer. Filename encodes the expected value: `boundary_int_127.msgpack`, `boundary_int_128.msgpack`, `boundary_int_neg128.msgpack`, etc. + +2. **Snapshot boundary fixtures** — snapshots with tick values at critical boundaries. These test the field-level encoding within a struct context: + - `snapshot_tick_127.msgpack` — tick at max positive fixint + - `snapshot_tick_128.msgpack` — tick at min uint_8 (Bug #4 regression) + - `snapshot_tick_32768.msgpack` — tick at min uint_16 + - `snapshot_tick_65536.msgpack` — tick at min int_32 + - `snapshot_entity_id_boundary.msgpack` — entity_id values at boundaries + +**Implementation sketch for gen_fixtures.rs:** + +```rust +#[test] +#[ignore] +fn generate_boundary_fixtures() { + // Raw integer boundary values + let boundaries: Vec<(i64, &str)> = vec![ + (127, "boundary_int_127"), + (128, "boundary_int_128"), + (255, "boundary_int_255"), + (256, "boundary_int_256"), + (32767, "boundary_int_32767"), + (32768, "boundary_int_32768"), + (65535, "boundary_int_65535"), + (65536, "boundary_int_65536"), + (-32, "boundary_int_neg32"), + (-33, "boundary_int_neg33"), + (-128, "boundary_int_neg128"), + (-129, "boundary_int_neg129"), + (-32768, "boundary_int_neg32768"), + (-32769, "boundary_int_neg32769"), + ]; + + for (value, name) in &boundaries { + write_fixture(name, &rmp_serde::to_vec(value).unwrap()); + } + + // Snapshot with tick at the Bug #4 boundary + let snapshot_128 = fixture_snapshot(128, vec![]); + write_fixture("snapshot_tick_128", &rmp_serde::to_vec_named(&snapshot_128).unwrap()); + + // Snapshot with large entity_id + let snapshot_large_id = fixture_snapshot(0, vec![VisibleEntity { + entity_id: 65536, // uint_32 boundary + x: 0.0, y: 0.0, z: 0, + kind: EntityKind::Npc, + visibility: VisibilitySector::Forward, + relationship: RelationshipState::Unknown, + observation: EntityVisibility::Visible, + }]); + write_fixture("snapshot_entity_id_65536", &rmp_serde::to_vec_named(&snapshot_large_id).unwrap()); +} +``` + +**Client-side verification** (`test_msgpack_boundaries.gd`): + +```gdscript +func test_boundary_fixture_int_128() -> void: + # Bug #4 regression: Rust encodes tick=128 as uint_8, GDScript must decode to 128 (not -128) + var bytes = _load_fixture("boundary_int_128") + var result = Messagepack.decode(bytes) + assert_that(result.value).is_equal(128) + +func test_boundary_fixture_snapshot_tick_128() -> void: + var bytes = _load_fixture("snapshot_tick_128") + var snapshot = Protocol.decode_snapshot(bytes) + assert_that(snapshot.tick).is_equal(128) +``` + +**Workflow:** `make fixtures` regenerates all fixtures (existing + boundary). Client tests verify them. This is D-030 Layer 1: fixture-based cross-language validation. + +--- + +### T4-H4: Golden File Approach Tied to the Gauntlet + +**Proposal: Gauntlet Snapshot Golden File Pipeline** + +The golden file flow: + +``` +[1. Server generates] [2. Committed to repo] [3. Client verifies] + +cargo test --test client/tests/fixtures/ make test-client + gen_gauntlet_golden gauntlet/ ↓ + -- --ignored tick_0.msgpack test_gauntlet_golden.gd + ↓ tick_10.msgpack verifies each fixture + Boots Gauntlet map tick_0.json (human-readable decodes to expected + with seed=42 diff companion) field values + Runs to tick 0, 10 + Serializes ObserverSnapshot + from fixed player position + Writes .msgpack + .json +``` + +**Key design decisions:** + +1. **Format: ObserverSnapshot, not full ECS world.** The snapshot is what the client sees — it's the contract surface. Internal ECS changes that don't affect the snapshot are allowed without golden file breakage. This keeps the golden file stable and focused. + +2. **JSON companion for diff.** Each `.msgpack` fixture gets a parallel `.json` file containing the same data in human-readable form. When the golden file breaks, `git diff` on the JSON shows exactly what changed (e.g., "entity_id changed from 100 to 101" or "new field `zone_id` appeared"). + +3. **Fixed player position.** The Gauntlet spec defines a named position per room (e.g., `GAUNTLET.occlusion_corridor.observer_position = (15, 10, 0)`). Golden files are generated from that exact position. Moving the observer changes what's visible. + +4. **Seed pinning.** `seed=42` (or whatever the Gauntlet uses). The golden file test MUST fail if the seed produces different output, which catches non-determinism bugs. + +**CI integration:** + +```yaml +# Pseudo-CI pipeline (Gitea Actions or equivalent) +golden-file-check: + steps: + - cargo build --release + - cargo test --test gen_gauntlet_golden -- --ignored + # Generates fresh .msgpack files in a temp directory + - diff client/tests/fixtures/gauntlet/tick_0.msgpack /tmp/gauntlet_fresh/tick_0.msgpack + # If diff is non-empty, the golden file is stale → FAIL + # Developer must regenerate and commit updated golden files +``` + +**When golden files legitimately change:** + +1. Developer changes server logic that affects snapshot output +2. Developer runs `make fixtures-gauntlet` (regenerates golden files) +3. Developer inspects the JSON diff: `git diff client/tests/fixtures/gauntlet/tick_0.json` +4. If the diff is expected, commit the updated golden files +5. PR review includes the golden file diff as mandatory review artifact + +**Breakage detection granularity:** + +The golden file test should not just do byte comparison. It should decode both the golden file and the fresh snapshot, then compare field by field: + +```rust +#[test] +fn gauntlet_tick_0_matches_golden() { + let golden: ObserverSnapshot = load_golden("tick_0"); + let fresh: ObserverSnapshot = run_gauntlet_to_tick(0); + + assert_eq!(golden.tick, fresh.tick); + assert_eq!(golden.entities.len(), fresh.entities.len()); + for (g, f) in golden.entities.iter().zip(fresh.entities.iter()) { + assert_eq!(g.entity_id, f.entity_id, "entity_id mismatch"); + assert!((g.x - f.x).abs() < 0.01, "x mismatch for entity {}", g.entity_id); + assert!((g.y - f.y).abs() < 0.01, "y mismatch for entity {}", g.entity_id); + } + // ... etc for all fields +} +``` + +This gives actionable error messages instead of "binary files differ." + +--- + +### T4-H5: Minimum Viable Layer 3 Test (Real Subprocess Integration) + +**Current state assessment:** + +| Layer | Status | What exists | +|-------|--------|------------| +| Layer 1 (fixtures) | **Complete** | `gen_fixtures.rs` generates, `test_protocol.gd` + `serialization.rs` verify | +| Layer 2 (bridge) | **Partial** | `bridge_tcp.rs` + `bridge_ipc.rs` test roundtrip. `game_loop.rs` tests full pipeline. But all run within one process (server as thread, not subprocess). | +| Layer 3 (real subprocess) | **Missing** | No test launches the actual `settled-reach-server` binary as a child process | + +**Why Layer 3 matters:** Bug #1 (server never sends snapshots in live mode) was caused by `read_framed()` blocking the bevy Update schedule. This bug only manifests with the real binary running as a subprocess — the in-process thread tests in `game_loop.rs` don't reproduce it because the thread shares memory and doesn't have the same blocking behavior. + +**Minimum viable Layer 3 test specification:** + +``` +Test name: server_subprocess_sends_snapshot_on_connect +File: tests/integration_subprocess.rs (or tests/layer3.rs) + +Setup: + 1. cargo build the server binary (or use pre-built from CI artifact) + 2. Launch server binary as child process: `Command::new("target/debug/settled-reach-server")` + with args: --test-mode --port 0 (random port, printed to stdout) + 3. Parse port from server stdout + 4. Connect to server via TCP + +Test steps: + 1. Send one PlayerInput (MoveNorth, tick 0) via write_framed + 2. Read one ObserverSnapshot via read_framed with 5-second timeout + 3. Assert: snapshot.version == PROTOCOL_VERSION + 4. Assert: snapshot.tick == 0 + 5. Assert: snapshot.entities.len() >= 1 (at least the player) + 6. Assert: player entity has kind == Player + +Teardown: + 1. Drop TCP connection + 2. Kill child process (SIGTERM) + 3. Wait for exit with timeout + +Duration budget: < 10 seconds (including server startup) +Frequency: Nightly + pre-merge (D-030 says "slow, daily/pre-merge") +``` + +**Implementation requirements:** + +1. **Server needs `--test-mode` flag.** This flag should: + - Load the proof room (or Gauntlet) instead of requiring content files + - Print the listening port to stdout: `LISTENING:9876` + - Use a fixed seed for determinism + - Exit after first client disconnects (or after timeout) + +2. **Server needs `--port 0` support.** Bind to random available port and print it. Prevents test flakiness from port conflicts. + +3. **Test function:** + +```rust +use std::process::{Command, Stdio}; +use std::io::{BufRead, BufReader as StdBufReader}; +use std::net::TcpStream; +use std::time::Duration; + +#[test] +#[ignore] // Layer 3: slow, run with --ignored or in CI nightly +fn server_subprocess_sends_snapshot_on_connect() { + // Build the server binary + let status = Command::new("cargo") + .args(["build", "--bin", "settled-reach-server"]) + .status() + .expect("cargo build failed"); + assert!(status.success(), "server build failed"); + + // Launch server as subprocess + let mut server = Command::new("target/debug/settled-reach-server") + .args(["--test-mode", "--port", "0"]) + .stdout(Stdio::piped()) + .stderr(Stdio::inherit()) + .spawn() + .expect("failed to launch server"); + + // Parse port from stdout + let stdout = server.stdout.take().unwrap(); + let mut lines = StdBufReader::new(stdout).lines(); + let port: u16 = loop { + let line = lines.next().expect("server stdout ended").expect("read line"); + if let Some(port_str) = line.strip_prefix("LISTENING:") { + break port_str.trim().parse().expect("parse port"); + } + }; + + // Connect as client + let stream = TcpStream::connect(format!("127.0.0.1:{}", port)) + .expect("connect to server"); + stream.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + + let mut reader = std::io::BufReader::new(stream.try_clone().unwrap()); + let mut writer = std::io::BufWriter::new(stream); + + // Send input + let inputs = vec![PlayerInput { + tick: 0, + action: PlayerAction::MoveNorth, + }]; + let payload = rmp_serde::to_vec_named(&inputs).expect("serialize"); + write_framed(&mut writer, &payload).expect("send input"); + + // Receive snapshot + let response = read_framed(&mut reader) + .expect("read snapshot") + .expect("not EOF — server must send snapshot after receiving input"); + + let snapshot: ObserverSnapshot = + rmp_serde::from_slice(&response).expect("deserialize snapshot"); + + // Assertions + assert_eq!(snapshot.version, PROTOCOL_VERSION, + "snapshot version mismatch — protocol incompatibility"); + assert_eq!(snapshot.tick, 0); + assert!(!snapshot.entities.is_empty(), + "snapshot must contain at least the player entity"); + + let player = snapshot.entities.iter() + .find(|e| matches!(e.kind, EntityKind::Player)) + .expect("no Player entity in snapshot"); + assert!(player.x > 0.0, "player x must be positive"); + assert!(player.y > 0.0, "player y must be positive"); + + // Cleanup + drop(reader); + drop(writer); + server.kill().ok(); + server.wait().ok(); +} +``` + +**What this catches that Layer 2 doesn't:** +- Server binary startup issues (missing resources, config, panic on init) +- Subprocess I/O blocking (Bug #1 class) +- Protocol version mismatch between compiled server and test expectations +- Real TCP behavior (Nagle's algorithm, buffer sizes, connection lifecycle) + +--- + +## TRACK 5 — Content Scaling & CI Pipeline + +### T5-H1: Why Don't Client Tests Run in CI? + +**Finding: There IS no CI pipeline at all.** No `.github/`, `.gitea/`, or other CI configuration files exist in the repository. The Makefile has `ci`, `ci-server`, `ci-client` targets, but these are designed for local execution only. + +The blockers for CI are, in order of priority: + +1. **No CI runner configured.** The project uses Gitea (`git.schweitz.internal`), which supports Gitea Actions (GitHub Actions compatible). But no workflow files exist. This is the primary blocker — it's wiring, not a technical limitation. + +2. **Godot headless availability.** `make test-client` uses `$(GODOT) --headless --path client -s res://addons/gdUnit4/bin/GdUnitCmdTool.gd --ignoreHeadlessMode`. This requires: + - Godot 4.6 binary on the CI runner + - The `--headless` flag (supported since Godot 4.0) + - No GPU required (headless mode uses software rendering) + - The `--ignoreHeadlessMode` flag for gdUnit4 (already present in Makefile) + + This is solvable: install Godot on the CI runner, or use the official Godot Docker image. + +3. **gdUnit4 test runner stability in headless.** gdUnit4 documents headless CI support via `GdUnitCmdTool.gd`. I haven't verified whether the current test suite actually passes headless — some tests may have implicit dependencies on window size, input events, or rendering state. This needs a verification run: `make test-client` on a headless machine. + +4. **Test output format.** D-030 specifies JSON summary output. gdUnit4 produces JUnit XML by default. The CI pipeline needs a format adapter — or we accept JUnit XML for CI (most CI systems parse it natively). + +**Recommended action:** The first step is just running `make ci` on the Gitea runner. Everything else is refinement. + +--- + +### T5-H2: Minimal CI Pipeline — Tier Proposal + +| Tier | Trigger | Tests | Duration budget | Rationale | +|------|---------|-------|----------------|-----------| +| **Commit (fast feedback)** | Every push to any branch | `make lint-server` (clippy + fmt), `make lint-client` (GDScript error check), `make validate-content` (YAML schema), `make check-fact-ids` | < 2 min | Catches syntax errors, formatting, broken content. No compilation needed for content checks. | +| **PR (merge gate)** | PR opened or updated | Everything in Commit tier + `make build` (both server + client), `make test-server` (cargo nextest), `make test-client` (gdUnit4 headless), `make fixtures` + verify no diff (fixture staleness check) | < 10 min | Full build + test. Fixture staleness check catches protocol changes that weren't regenerated. | +| **Nightly (deep validation)** | Scheduled, 1x/day on main | Everything in PR tier + Layer 3 subprocess test (`cargo test --test layer3 -- --ignored`), Gauntlet golden file regeneration + diff check, Content load-test (boot server with full content, tick 100 times), Performance benchmark (Gauntlet 100 ticks, assert < time budget) | < 30 min | Slow tests that catch subtle integration bugs. Performance regression detection. | + +**Pipeline configuration (Gitea Actions):** + +```yaml +# .gitea/workflows/ci.yaml + +name: CI +on: + push: + branches: ['*'] + pull_request: + branches: [main] + schedule: + - cron: '0 3 * * *' # Nightly at 03:00 + +jobs: + commit-checks: + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - run: make lint-server + - run: make lint-client + - run: make validate-content + - run: make check-fact-ids + + pr-checks: + if: github.event_name == 'pull_request' + needs: commit-checks + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - run: make build + - run: make test-server + - run: make test-client + - run: make fixtures + - run: git diff --exit-code client/tests/fixtures/ + # Fails if fixtures are stale + + nightly: + if: github.event_name == 'schedule' + needs: commit-checks + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - run: make build + - run: make test-server + - run: make test-client + - run: cargo test --test layer3 -- --ignored + - run: make test-gauntlet-golden # (new target, TBD) + - run: make test-content-load # (new target, TBD) +``` + +**Critical gate:** PR tier is the merge gate. A PR cannot merge if any PR-tier test fails. Nightly failures generate alerts but don't block work. + +--- + +### T5-H3: Minimum Runtime Content Validation + +**Current state:** `make validate-content` only checks YAML structure against JSON Schema. It does NOT verify that the content can actually be loaded and used by the server. + +**Minimum runtime validation — "boot and tick" test:** + +``` +Test name: content_loads_and_ticks_without_panic +File: server/tests/content_loading.rs (extend existing) + +Setup: + 1. Load content from content/campaigns/ using ContentLoader + 2. Build a minimal bevy App with SimulationPlugin + ContentPlugin + 3. Insert loaded content as resources + +Test steps: + 1. app.update() — tick once + 2. Assert: no panic (the test itself succeeding is the assertion) + 3. Assert: SimulationTime.tick == 1 + 4. Assert: at least 1 entity with PlayerCharacter component exists + 5. Assert: at least 1 entity with NPC-related components exists (if content defines NPCs) + +Duration: < 5 seconds +Frequency: Every PR +``` + +**Graduated runtime validation layers:** + +| Level | What it checks | Test type | Duration | +|-------|---------------|-----------|----------| +| 1. Boot | Content loads without panic | Unit test in `content_loading.rs` | < 2s | +| 2. Tick | 10 ticks complete without panic | Integration test | < 3s | +| 3. Snapshot | ObserverSnapshot generates with expected entity count | Integration test | < 5s | +| 4. Cross-reference | All `entity_ref` in dialogue resolves to spawned entities | Dedicated validation test | < 3s | +| 5. Stress | 100 ticks with max-NPC content pack, no tick exceeds 50ms | Performance test | < 30s | + +**Level 1 already partially exists** in `content_loading.rs`. We should extend it to Level 3 as the minimum for PR gating. + +**Concrete Level 3 test:** + +```rust +#[test] +fn content_produces_valid_snapshot() { + let mut app = build_app_with_content("content/campaigns/meridian"); + + // Tick 10 times + for _ in 0..10 { + app.update(); + } + + // Generate snapshot from player position + let snapshot = extract_observer_snapshot(&app); + + assert!(snapshot.tick > 0, "simulation must have advanced"); + assert!(!snapshot.entities.is_empty(), "snapshot must contain entities"); + + // Verify player exists in snapshot + let has_player = snapshot.entities.iter().any(|e| matches!(e.kind, EntityKind::Player)); + assert!(has_player, "player must be visible in own snapshot"); + + // Verify NPC count matches content expectation + let npc_count = snapshot.entities.iter().filter(|e| matches!(e.kind, EntityKind::Npc)).count(); + assert!(npc_count >= 1, "at least one NPC should be visible from spawn position"); +} +``` + +--- + +### T5-H4: Content Scaling Test — "Adding an NPC Doesn't Break Anything" + +**The test:** When a new NPC is added to a district, the game must still boot, tick, and produce valid snapshots. + +**Test specification:** + +``` +Test name: adding_npc_to_district_preserves_functionality +File: server/tests/content_scaling.rs (new) + +Approach: Comparative testing + 1. Load baseline content → boot → tick 10 → snapshot (baseline) + 2. Load baseline content + 1 extra NPC → boot → tick 10 → snapshot (modified) + 3. Compare: modified snapshot should be a SUPERSET of baseline + +Assertions: + - Modified boots without panic + - Modified ticks 10 times without panic + - Modified snapshot has >= baseline entity count + - Modified snapshot has exactly baseline.npcs + 1 NPCs + - All baseline NPCs still present (by entity name/stable_id) + - Player entity unchanged between baseline and modified + - No tick exceeds 50ms (performance regression check) + - All nearby_interactions for existing NPCs still present +``` + +**Concrete implementation:** + +```rust +#[test] +fn adding_npc_preserves_existing_entities() { + // Baseline: load district with known NPCs + let baseline_app = build_app_with_content("content/campaigns/meridian"); + let baseline_snapshot = run_and_snapshot(&baseline_app, 10); + let baseline_npc_ids: HashSet = baseline_snapshot.entities.iter() + .filter(|e| matches!(e.kind, EntityKind::Npc)) + .map(|e| e.entity_id) + .collect(); + + // Modified: same district + one extra NPC injected programmatically + let mut modified_app = build_app_with_content("content/campaigns/meridian"); + inject_test_npc(&mut modified_app, "test_extra_npc", TilePosition::new(20, 20, 0)); + let modified_snapshot = run_and_snapshot(&modified_app, 10); + let modified_npc_ids: HashSet = modified_snapshot.entities.iter() + .filter(|e| matches!(e.kind, EntityKind::Npc)) + .map(|e| e.entity_id) + .collect(); + + // All baseline NPCs still exist + assert!(baseline_npc_ids.is_subset(&modified_npc_ids), + "existing NPCs disappeared after adding new NPC"); + + // Exactly one more NPC + assert_eq!(modified_npc_ids.len(), baseline_npc_ids.len() + 1, + "expected exactly one additional NPC"); + + // Player still exists and at same position + let baseline_player = find_player(&baseline_snapshot); + let modified_player = find_player(&modified_snapshot); + assert_eq!(baseline_player.x, modified_player.x); + assert_eq!(baseline_player.y, modified_player.y); +} +``` + +**Scaling test matrix:** + +| Scenario | Baseline | Modified | Key assertion | +|----------|----------|----------|--------------| +| +1 NPC | 3 NPCs | 4 NPCs | All original NPCs present | +| +5 NPCs | 3 NPCs | 8 NPCs | No tick > 50ms | +| +15 NPCs | 3 NPCs | 18 NPCs (Crowd Plaza scale) | No tick > 100ms, no panic | +| +1 location | 1 district | 1 district + 1 location YAML | Content loads, entities spawn correctly | +| +1 item | 9 items | 10 items | Inventory interaction still works (9-slot limit not exceeded by world items) | + +**Content-level validation (complement to runtime tests):** + +Add to `make validate-content`: +- **Entity reference check:** Every `entity_ref` in dialogue/monologue YAML resolves to a defined NPC profile +- **Location slug check:** Every `location` field matches a defined location YAML +- **Fact ID check:** Already exists as `make check-fact-ids` +- **District capacity check:** Warn if a district exceeds 15 NPCs (Crowd Plaza performance boundary) + +--- + +## Summary of Deliverables + +| Deliverable | Track | Status | +|-------------|-------|--------| +| Boundary value test matrix (41 values) | T4-H1 | Complete | +| Boundary test placement recommendation | T4-H2 | Complete | +| gen_fixtures.rs extension specification | T4-H3 | Complete | +| Golden file pipeline specification | T4-H4 | Complete | +| Layer 3 test specification with code | T4-H5 | Complete | +| CI blocker analysis | T5-H1 | Complete | +| CI pipeline tier proposal (commit/PR/nightly) | T5-H2 | Complete | +| Runtime content validation specification | T5-H3 | Complete | +| Content scaling test specification | T5-H4 | Complete | + +## Open Questions for Round 2 + +1. **For Dudley:** Does the server binary support `--test-mode` and `--port 0` flags? If not, what's the minimum change to add them? Layer 3 depends on this. +2. **For Dudley:** The GDScript encoder uses int_16 for positive values 256-32767 while rmp_serde uses uint_16. Does `rmp_serde::from_slice::()` accept int_16-encoded positive values? I believe it does, but this needs explicit verification. +3. **For Tyre/Justine:** The Gitea instance at `git.schweitz.internal` — does it have Gitea Actions enabled? Is there a self-hosted runner available? CI pipeline design depends on this. +4. **For Tyre:** The golden file JSON companion format — should it be the full ObserverSnapshot serialized to JSON, or a reduced "diff-friendly" format with sorted keys and abbreviated fields? +5. **For Justine:** Fixture staleness check in CI (`make fixtures && git diff --exit-code`) — is this robust enough, or do we need a content-addressed hash approach? diff --git a/docs/workshops/test-architecture/hoshe-round2.md b/docs/workshops/test-architecture/hoshe-round2.md new file mode 100644 index 000000000..17ba71a4a --- /dev/null +++ b/docs/workshops/test-architecture/hoshe-round2.md @@ -0,0 +1,651 @@ +# Hoshe — Round 2 Analysis: Coverage Validation, Content Validation, Pre-PR, Encoding Asymmetry + +**Workshop:** QA Strategy & Test Architecture +**Track:** Cross-review (T4 coverage) + T5 (content validation, CI) +**Date:** 2026-02-17 +**Round:** 2 (Synthesis) + +--- + +## 1. Coverage Validation of Dudley's Server Proposals + +I reviewed the Round 1 notes (sections 4.6, 4.7, 4.8), the existing tests in `server/src/simulation/input.rs` (lines 364-1027), `server/src/knowledge/registry.rs` (lines 83-161), and the live implementation code. Here's my assessment. + +### 1.1 Pause Guard Tests — Gaps Found + +**Current state:** One test exists: `process_input_pause_sets_paused` (input.rs:436). It verifies that `Pause` action sets `TickRate::Paused`. It does NOT verify that movement is discarded while paused. + +**Dudley proposed 6 tests. Assessment:** + +| Dudley's proposed test | Priority | Gap? | Assessment | +|----------------------|----------|------|------------| +| `movement_discarded_while_paused` | P0 | **YES — critical gap** | Bug #3 regression. The actual pause guard (input.rs:102-105) is `if paused && input.action.is_movement() { continue; }` — this works, but there's ZERO test coverage. If someone refactors the match arms or changes `is_movement()`, this breaks silently. | +| `unpause_accepted_while_paused` | P0 | **YES — gap** | Must verify Unpause action reaches `time.tick_rate = TickRate::Full` even when `paused == true`. Current test only tests Pause, not Unpause-while-paused. | +| `stance_toggle_allowed_while_paused` | P1 | **YES — gap** | The pause guard only skips `is_movement()`. Stance toggles should pass through. No test verifies this. | +| `interact_allowed_while_paused` | P1 | **YES — gap** | Same logic — Interact is not movement, should be processed while paused. Especially important because D-052 says "UI stays responsive" during pause. | +| `multiple_movements_in_paused_batch_all_discarded` | P1 | **YES — gap** | Tests that a batch of 3+ movements ALL get discarded, not just the first. Guards against early-exit bugs in the loop. | +| `pause_unpause_roundtrip_with_movement` | P1 | **YES — gap** | Full cycle: Pause → move (discarded) → Unpause → move (accepted). Most important integration test of the set. | + +**Verdict: All 6 are genuine gaps. The pause guard has ZERO direct test coverage today.** + +The existing test (`process_input_pause_sets_paused`) tests the Pause action side effect, not the pause guard itself. If someone deleted lines 102-105 (the guard), all existing tests would still pass. That's the gap. + +**Additional gaps I found that Dudley didn't propose:** + +| Test | Priority | Rationale | +|------|----------|-----------| +| `set_tick_rate_while_paused` | P2 | What happens if SetTickRate(Half) is sent while paused? Code at input.rs:165-168 sets the rate unconditionally — should this unpause or stay paused? Current behavior: sets to Half, which means `paused()` returns false on next tick. Might be intentional but needs explicit test. | +| `perception_mode_while_paused` | P2 | UsePerceptionMode is currently a no-op (input.rs:193-195), but it should pass through the pause guard since it's not movement. When it becomes functional, this test prevents regression. | +| `interact_take_while_paused` | P2 | Can the player Take an item while paused? The pause guard only blocks `is_movement()`. Take/Place go through. Is this intentional? Needs a test that documents the expected behavior either way. | + +### 1.2 EntityRegistry Lifecycle Tests — Gaps Found + +**Current state:** 6 tests in `registry.rs:83-161`: sequential IDs, idempotent register, bidirectional lookup, unregister removes both directions, seed offset, unknown returns None. + +**Dudley proposed 5 tests. Assessment:** + +| Dudley's proposed test | Priority | Gap? | Assessment | +|----------------------|----------|------|------------| +| `old_stable_id_not_resolvable_after_unregister` | P0 | **Partially covered** | `unregister_removes_both_directions` tests this — after unregister, `to_entity(&id)` returns None. But Dudley's concern is more subtle: after unregister + re-spawn a new entity, does the OLD StableId point to the NEW entity? This is NOT tested. | +| `register_after_unregister_gets_new_id` | P1 | **YES — gap** | If entity E is registered (StableId=5), unregistered, then a NEW entity is registered, it should get StableId=6, not StableId=5. Guarantees ID monotonicity across the lifecycle. | +| `register_respawn_no_stale_mapping` | P0 | **YES — critical gap** | bevy_ecs recycles Entity indices. If Entity(index=3, gen=1) is despawned and Entity(index=3, gen=2) is spawned, the registry must NOT return the old StableId for the new entity. The existing test doesn't test this because it doesn't use a real World with despawn+respawn. | +| `concurrent_register_unregister` | P2 | **Not applicable** | EntityRegistry is a Resource accessed through ResMut (exclusive), so concurrent access is impossible in bevy_ecs. This test would be testing bevy's scheduler, not our code. Skip. | +| `bulk_register_performance` | P2 | Nice-to-have | Registry uses BTreeMap — O(log N) for insert. At 80 active NPCs + 2000 background, we're talking ~2000 entries. Not a performance concern. Skip for now. | + +**Additional gaps I found:** + +| Test | Priority | Rationale | +|------|----------|-----------| +| `unregister_unknown_entity_is_noop` | P1 | Calling `unregister(e)` on an entity that was never registered should not panic or corrupt state. The current code handles this (the `if let Some` guard in line 67), but there's no test. | +| `register_with_pre_existing_stable_id_component` | P2 | If an entity is spawned with `StableEntityId(StableId(42))` component but NOT yet in the registry, calling `register()` assigns a NEW StableId, not 42. Is this correct? The component and registry could diverge. Needs at minimum a documented test. | + +### 1.3 Determinism Fixes — Coverage Assessment + +**Proposed fixes from Round 1:** + +| Fix | Coverage status | Assessment | +|-----|----------------|------------| +| `visible_ids: HashSet → BTreeSet` | **Untested** | No test verifies that sprint anomaly detection selects a deterministic "first Contradicted match." The fix itself is correct (BTreeSet iterates in order), but a test should run the observer with 2 equidistant contradicted NPCs and assert the same one is selected on both runs. | +| Sort `visible_tiles` by `(x, y, z)` | **Untested** | No test checks ordering of `visible_tiles` in the ObserverSnapshot. Golden file tests will catch this implicitly, but a focused unit test is valuable: generate a snapshot with tiles added in random order, assert the output Vec is sorted. | +| Pin monologue system ordering | **Untested directly** | The determinism regression test (`gauntlet_deterministic_replay`) will catch this. But a focused test should verify that `.after()` constraints are respected — run 100 times, assert identical output. | +| Sort movers by Entity bits | **Untested** | Dudley notes "no test for which one wins" in equidistant movement. The fix (sort by bits) needs a test with two entities at the same distance attempting to move to the same tile, asserting deterministic winner. | + +**Recommendation:** Each determinism fix should ship with its own regression test, not just rely on the broad `gauntlet_deterministic_replay` test. The broad test is the safety net; individual tests are the documentation. + +### 1.4 Bridge Deserialization (Section 4.8) — Assessment + +Dudley raised the question: should batch deserialization skip-and-log bad inputs, or reject the entire batch? + +**My assessment: Keep batch-failure for now, but add a test that documents the behavior.** + +Rationale: +- Both sides are co-versioned (D-020). A malformed input is a programming error, not user input. +- The boundary value test matrix (my Round 1 deliverable) prevents the main class of encoding bugs. +- Skip-and-log adds complexity and could mask real bugs during development. +- The test I'd add: `malformed_input_in_batch_rejects_entire_batch` — send a Vec where one entry has an invalid action variant. Assert: server logs error, discards entire batch. + +--- + +## 2. Content Cross-Reference Validation — Sprint 8 Specification + +**Context:** `make validate-content` currently runs schema validation only (YAML structure against JSON Schema). `make check-fact-ids` validates fact_id references. Neither validates entity references, location slugs, or dialogue pool tags. + +### 2.1 Architecture Decision + +**Extend `tooling/validate-content` with a second pass**, not a separate script. The schema validation pass runs first (fails fast on malformed YAML). The cross-reference pass runs second (requires all files to be parseable). + +``` +make validate-content + │ + ├── Pass 1: Schema validation (existing, unchanged) + │ → Each YAML file against its JSON Schema + │ → FAILS FAST on schema errors (no point cross-referencing broken files) + │ + └── Pass 2: Cross-reference validation (NEW) + → Build index of all defined entities, locations, fact_ids, pools + → Walk all files, check every reference resolves + → Report ALL errors (don't fail on first) +``` + +### 2.2 Cross-Reference Checks — Complete Specification + +I audited the content directory structure. Here are all cross-reference relationships that exist in the content: + +#### Check 1: NPC `canonical_id` uniqueness + +**What:** Every NPC profile YAML has a `canonical_id` field (e.g., `"npc:kael-davan"`). These MUST be globally unique. + +**Where:** `content/campaigns/**/npcs/*.yaml` + +**How:** Build `Set` from all NPC profiles. Error on duplicate. + +**Error format:** +``` +XREF ERROR: duplicate canonical_id "npc:kael-davan" + Defined in: campaigns/main/.../npcs/kael-davan.yaml + Duplicate in: campaigns/main/.../npcs/kael-davan-copy.yaml +``` + +#### Check 2: NPC relationship `target` resolution + +**What:** Each NPC profile has a `relationships` array where each entry has a `target` field (e.g., `"npc:nils-davan"`). Every target MUST match a defined `canonical_id`. + +**Where:** `content/campaigns/**/npcs/*.yaml` → `relationships[].target` + +**How:** For each `target` value, check membership in the `canonical_id` set. + +**Error format:** +``` +XREF ERROR: unresolved relationship target "npc:unknown-person" + In: campaigns/main/.../npcs/kael-davan.yaml + Relationship to: "npc:unknown-person" (kind: "colleague") + Known canonical_ids: npc:kael-davan, npc:nils-davan, ... (21 defined) +``` + +#### Check 3: Location slug resolution + +**What:** Each `district.yaml` lists location slugs in its `locations` array (e.g., `["the-terminal", "the-last-shift", "maintenance-corridors"]`). Each slug MUST correspond to a location YAML file at `locations/{slug}.yaml` in the same district. + +**Where:** `content/campaigns/**/district.yaml` → `locations[]` + +**How:** For each slug in `locations`, check that `locations/{slug}.yaml` exists in the same directory. + +**Error format:** +``` +XREF ERROR: location slug "the-docks" not found + In: campaigns/main/.../transit/district.yaml + Expected file: campaigns/main/.../transit/locations/the-docks.yaml + Available locations: the-terminal, the-last-shift, maintenance-corridors +``` + +#### Check 4: Dialogue pool location resolution + +**What:** Each dialogue YAML file has a `location` field (e.g., `location: the-terminal`). This MUST match a location slug defined in the parent district's `district.yaml`. + +**Where:** `content/campaigns/**/dialogue/**/*.yaml` → `location` + +**How:** Walk up the directory tree to find the parent district's `district.yaml`. Check `location` value against the district's `locations` array. + +**Error format:** +``` +XREF ERROR: dialogue location "the-warehouse" not in district + In: campaigns/main/.../dialogue/the-terminal/kael-davan.yaml + Location: "the-warehouse" + District locations: the-terminal, the-last-shift, maintenance-corridors +``` + +#### Check 5: Dialogue `knowledge_grant.fact_id` resolution + +**What:** Some dialogue lines have a `knowledge_grant` with a `fact_id` (e.g., `fact_id: investigation.manifest_discrepancy`). These MUST be valid fact IDs. + +**Where:** `content/campaigns/**/dialogue/**/*.yaml` → `lines[].knowledge_grant.fact_id` + +**How:** Reuse the canonical fact_id set from `check-fact-ids` logic. This is a superset of what `check-fact-ids` already does, but integrated into the same pass. + +**Note:** This subsumes `make check-fact-ids` for dialogue files. We keep `check-fact-ids` as a standalone check because it also covers monologue `prerequisites.facts[].fact_id`. + +**Error format:** +``` +XREF ERROR: unknown fact_id "investigation.unknown_fact" + In: campaigns/main/.../dialogue/the-terminal/kael-davan.yaml + Line: the-terminal_d_099 + Canonical fact_ids: 42 defined in content/global/knowledge/ +``` + +#### Check 6: NPC `triangle_membership` resolution + +**What:** NPC profiles list `triangle_membership` (e.g., `["hub-power", "worried-partner"]`). Each MUST match a triangle YAML file in the same district. + +**Where:** `content/campaigns/**/npcs/*.yaml` → `triangle_membership[]` + +**How:** Check that `triangles/{slug}.yaml` exists in the same district. + +**Error format:** +``` +XREF ERROR: triangle "unknown-triangle" not found + In: campaigns/main/.../npcs/kael-davan.yaml + Expected file: campaigns/main/.../triangles/unknown-triangle.yaml + Available triangles: bar-tensions, hub-power, informant-question, worried-knowledge, worried-partner +``` + +#### Check 7: District `npc_count` accuracy + +**What:** Each `district.yaml` has `npc_count: N`. This SHOULD match the actual number of NPC profile YAML files in the `npcs/` subdirectory. + +**Where:** `content/campaigns/**/district.yaml` → `npc_count` + +**How:** Count files in `npcs/*.yaml` in the same district. Compare against declared `npc_count`. + +**Severity: WARNING, not ERROR.** The count might intentionally differ during content development. But a mismatch should be visible. + +**Warning format:** +``` +XREF WARNING: npc_count mismatch in transit district + Declared: 17 + Actual NPC files: 20 + In: campaigns/main/.../transit/district.yaml +``` + +#### Check 8: Dialogue line ID uniqueness within pool + +**What:** Each dialogue line has an `id` field (e.g., `the-terminal_d_001`). IDs MUST be unique within each dialogue file. + +**Where:** `content/campaigns/**/dialogue/**/*.yaml` → `lines[].id` + +**How:** Build `Set` per file. Error on duplicate. + +**Error format:** +``` +XREF ERROR: duplicate dialogue line id "the-terminal_d_015" + In: campaigns/main/.../dialogue/the-terminal/kael-davan.yaml + First occurrence: line 167 + Duplicate: line 215 +``` + +### 2.3 Implementation Plan + +```python +# Additions to tooling/validate-content (after schema validation pass) + +def cross_reference_validation(campaigns_dir: Path) -> int: + """Pass 2: Cross-reference validation across content files.""" + errors = 0 + warnings = 0 + + # Phase 1: Build indices + canonical_ids: dict[str, Path] = {} # canonical_id → defining file + location_files: dict[Path, set] = {} # district path → set of location slugs + triangle_files: dict[Path, set] = {} # district path → set of triangle slugs + fact_ids: set[str] = set() # canonical fact_ids + + # Phase 2: Walk and validate references + # ... (checks 1-8 as specified above) + + return errors +``` + +**Integration with existing script:** + +```python +def main() -> int: + # ... existing schema validation (Pass 1) ... + + if errors > 0: + print(f"\nSchema validation failed — skipping cross-reference checks") + return 1 + + # Pass 2: Cross-reference validation + xref_errors = cross_reference_validation(campaigns_dir) + errors += xref_errors + + print(f"\nValidated {validated} files, {skipped} skipped, {errors} errors, {warnings} warnings") + return 1 if errors else 0 +``` + +### 2.4 Summary Table + +| Check | Severity | What | Fields checked | +|-------|----------|------|---------------| +| 1 | ERROR | canonical_id uniqueness | `npcs/*.yaml → canonical_id` | +| 2 | ERROR | relationship target resolution | `npcs/*.yaml → relationships[].target` | +| 3 | ERROR | location slug existence | `district.yaml → locations[]` | +| 4 | ERROR | dialogue location matches district | `dialogue/**/*.yaml → location` | +| 5 | ERROR | knowledge_grant.fact_id validity | `dialogue/**/*.yaml → lines[].knowledge_grant.fact_id` | +| 6 | ERROR | triangle membership existence | `npcs/*.yaml → triangle_membership[]` | +| 7 | WARNING | npc_count matches file count | `district.yaml → npc_count` | +| 8 | ERROR | dialogue line ID uniqueness | `dialogue/**/*.yaml → lines[].id` | + +**Estimated effort:** 1-2 days. The script structure is straightforward — build indices in one pass, validate references in a second pass. The hardest part is the directory-tree walk logic for finding parent districts. + +--- + +## 3. Manual Testing Protocol — `make pre-pr` Target + +**Context:** No automated CI (lead decision). Developers need a clear checklist before submitting PRs. + +### 3.1 `make pre-pr` Target Specification + +```makefile +# Pre-PR checklist: run before submitting any PR +# Chains all checks in dependency order. Fails fast on first error. +pre-pr: lint build test validate-content check-fact-ids fixtures-check + @echo "" + @echo "Pre-PR checks PASSED. Safe to push." + +# Verify fixtures are not stale (protocol changes require regeneration) +fixtures-check: fixtures + @if git diff --quiet client/tests/fixtures/; then \ + echo "Fixtures: up to date"; \ + else \ + echo "FIXTURES STALE — run 'make fixtures' and commit the updated files"; \ + git diff --stat client/tests/fixtures/; \ + exit 1; \ + fi +``` + +**Execution order (sequential, fails fast):** + +| Step | Target | What it does | Duration | Catches | +|------|--------|-------------|----------|---------| +| 1 | `lint` | `lint-server` + `lint-client` | ~30s | Clippy warnings, fmt violations, GDScript errors | +| 2 | `build` | `build-server` + `build-client` | ~60s | Compilation errors both sides | +| 3 | `test` | `test-server` + `test-client` | ~30s | All unit + integration tests | +| 4 | `validate-content` | YAML schema + cross-references | ~5s | Broken content files | +| 5 | `check-fact-ids` | Fact ID resolution | ~2s | Dangling fact references | +| 6 | `fixtures-check` | Regenerate fixtures + git diff | ~10s | Stale protocol fixtures | + +**Total: ~2.5 minutes.** Fast enough to run before every PR push. + +### 3.2 Branch-Specific Variants + +Not all checks apply to all branches. Content-only PRs don't need server builds. + +```makefile +# Server branch pre-PR (no content checks needed) +pre-pr-server: lint-server build-server test-server fixtures-check + @echo "Server pre-PR checks PASSED." + +# Client branch pre-PR (no server build needed) +pre-pr-client: lint-client build-client test-client + @echo "Client pre-PR checks PASSED." + +# Copy/content branch pre-PR (no builds needed) +pre-pr-content: validate-content check-fact-ids + @echo "Content pre-PR checks PASSED." +``` + +### 3.3 Developer Documentation + +Add to `docs/DEVOPS.md`: + +```markdown +## Pre-PR Checklist + +Before pushing a PR, run: + + make pre-pr + +This runs all checks in order: lint → build → test → content validation → fixture staleness. + +For branch-specific checks: +- Server changes: `make pre-pr-server` +- Client changes: `make pre-pr-client` +- Content changes: `make pre-pr-content` + +If `fixtures-check` fails, your protocol changes require fixture regeneration: + + make fixtures + git add client/tests/fixtures/ + git commit -m "chore(fixtures): regenerate for protocol vN" +``` + +--- + +## 4. Encoding Asymmetry — Cross-Language Decode Tests + +### 4.1 The Problem + +From my Round 1 analysis: GDScript's `messagepack.gd` encoder uses **int_16** (signed, header 0xd1) for positive values 256-32767. Rust's `rmp_serde` uses **uint_16** (unsigned, header 0xcd) for the same values. Both are spec-valid, but they produce different bytes. + +This means: +- The GDScript decoder must accept uint_16 (what Rust sends) +- The Rust decoder must accept int_16 (what GDScript sends) +- Both must produce the same logical value from either encoding + +### 4.2 Concrete Cross-Language Tests + +#### Test A: GDScript decoder accepts Rust-style unsigned encodings + +These tests belong in `client/tests/test_msgpack_boundaries.gd`: + +```gdscript +# Verify GDScript decoder handles unsigned encodings (Rust-style) +# for values that GDScript would encode as signed + +func test_decode_uint16_256() -> void: + # Rust encodes 256 as uint_16: [0xcd, 0x01, 0x00] + var bytes = PackedByteArray([0xcd, 0x01, 0x00]) + var result = Messagepack.decode(bytes) + assert_that(result.status).is_null() + assert_that(result.value).is_equal(256) + +func test_decode_uint16_32767() -> void: + # Rust encodes 32767 as uint_16: [0xcd, 0x7f, 0xff] + var bytes = PackedByteArray([0xcd, 0x7f, 0xff]) + var result = Messagepack.decode(bytes) + assert_that(result.status).is_null() + assert_that(result.value).is_equal(32767) + +func test_decode_uint32_65536() -> void: + # Rust encodes 65536 as uint_32: [0xce, 0x00, 0x01, 0x00, 0x00] + var bytes = PackedByteArray([0xce, 0x00, 0x01, 0x00, 0x00]) + var result = Messagepack.decode(bytes) + assert_that(result.status).is_null() + assert_that(result.value).is_equal(65536) + +func test_decode_uint32_2147483647() -> void: + # Rust encodes 2147483647 as uint_32: [0xce, 0x7f, 0xff, 0xff, 0xff] + var bytes = PackedByteArray([0xce, 0x7f, 0xff, 0xff, 0xff]) + var result = Messagepack.decode(bytes) + assert_that(result.status).is_null() + assert_that(result.value).is_equal(2147483647) +``` + +**Why these specific values:** 256 and 32767 are in the int_16/uint_16 overlap zone. 65536 and 2147483647 are in the int_32/uint_32 overlap zone. These are the exact values where GDScript and Rust encode differently. + +#### Test B: Rust decoder accepts GDScript-style signed encodings + +These tests belong in `server/tests/serialization.rs`: + +```rust +#[test] +fn rust_decodes_gdscript_signed_encoding_for_positive_values() { + // GDScript encodes 256 as int_16: [0xd1, 0x01, 0x00] + // Rust must decode this into u64 correctly + let bytes: Vec = vec![0xd1, 0x01, 0x00]; + let value: u64 = rmp_serde::from_slice(&bytes) + .expect("Rust must accept int_16-encoded positive value as u64"); + assert_eq!(value, 256); +} + +#[test] +fn rust_decodes_gdscript_signed_encoding_for_tick() { + // GDScript sends tick=500 encoded as int_16 inside a PlayerInput map + // Verify the full struct deserializes correctly + let tick_500_int16 = vec![0xd1, 0x01, 0xf4]; // int_16(500) + let value: u64 = rmp_serde::from_slice(&tick_500_int16) + .expect("tick=500 as int_16 must deserialize into u64"); + assert_eq!(value, 500); +} + +#[test] +fn rust_decodes_all_overlap_zone_values() { + // Values where GDScript uses signed and Rust uses unsigned + let overlap_values: Vec<(Vec, u64)> = vec![ + (vec![0xd1, 0x01, 0x00], 256), // int_16(256) + (vec![0xd1, 0x7f, 0xff], 32767), // int_16(32767) + (vec![0xd2, 0x00, 0x01, 0x00, 0x00], 65536), // int_32(65536) + (vec![0xd2, 0x7f, 0xff, 0xff, 0xff], 2147483647), // int_32(2147483647) + ]; + + for (bytes, expected) in overlap_values { + let value: u64 = rmp_serde::from_slice(&bytes) + .unwrap_or_else(|e| panic!( + "Rust failed to decode signed-encoded {} from {:?}: {}", + expected, bytes, e + )); + assert_eq!(value, expected); + } +} +``` + +#### Test C: Fixture-based cross-language roundtrip for overlap values + +Add to `gen_fixtures.rs`: + +```rust +#[test] +#[ignore] +fn generate_encoding_asymmetry_fixtures() { + // Generate snapshots with tick values in the signed/unsigned overlap zones + // GDScript will encode these differently than Rust — but both must decode correctly + + let overlap_ticks = vec![ + (256, "snapshot_tick_256"), + (500, "snapshot_tick_500"), + (999, "snapshot_tick_999"), + (32767, "snapshot_tick_32767"), + (65536, "snapshot_tick_65536"), + ]; + + for (tick, name) in overlap_ticks { + let snapshot = fixture_snapshot(tick, vec![]); + write_fixture(name, &rmp_serde::to_vec_named(&snapshot).unwrap()); + } +} +``` + +And in `client/tests/test_msgpack_boundaries.gd`: + +```gdscript +func test_decode_overlap_fixtures() -> void: + # Rust-generated snapshots with tick values in the encoding overlap zone + var overlap_ticks = [ + ["snapshot_tick_256", 256], + ["snapshot_tick_500", 500], + ["snapshot_tick_999", 999], + ["snapshot_tick_32767", 32767], + ["snapshot_tick_65536", 65536], + ] + for pair in overlap_ticks: + var bytes = _load_fixture(pair[0]) + var snapshot = Protocol.decode_snapshot(bytes) + assert_that(snapshot).is_not_null() + assert_that(snapshot.tick).is_equal(pair[1]) +``` + +#### Test D: GDScript-encoded values decoded by Rust + +This is the reverse direction — GDScript encodes, Rust decodes. This requires generating fixtures from the GDScript side. + +**Proposal:** Add a `make fixtures-client` target that runs a GDScript test which generates `.msgpack` fixtures in `server/tests/fixtures/gdscript/`. The server test suite then verifies these decode correctly. + +```gdscript +# client/tests/test_gen_client_fixtures.gd (run via make fixtures-client) +func test_generate_client_fixtures() -> void: + var dir = "res://../../server/tests/fixtures/gdscript/" + DirAccess.make_dir_recursive_absolute(dir) + + # Encode tick=256 (GDScript uses int_16, Rust uses uint_16) + var input_256 = Protocol.encode_player_input(256, "MoveNorth") + _write_fixture(dir + "input_tick_256.msgpack", input_256) + + var input_32767 = Protocol.encode_player_input(32767, "MoveNorth") + _write_fixture(dir + "input_tick_32767.msgpack", input_32767) + + var input_65536 = Protocol.encode_player_input(65536, "MoveNorth") + _write_fixture(dir + "input_tick_65536.msgpack", input_65536) +``` + +```rust +// server/tests/serialization.rs (new test) +#[test] +fn gdscript_encoded_inputs_deserialize() { + let fixture_dir = std::path::Path::new("tests/fixtures/gdscript"); + if !fixture_dir.exists() { + eprintln!("GDScript fixtures not generated — run 'make fixtures-client' first"); + return; // Skip, don't fail + } + + for entry in std::fs::read_dir(fixture_dir).expect("read fixture dir") { + let path = entry.unwrap().path(); + if path.extension().and_then(|e| e.to_str()) != Some("msgpack") { + continue; + } + let bytes = std::fs::read(&path).unwrap(); + let input: PlayerInput = rmp_serde::from_slice(&bytes) + .unwrap_or_else(|e| panic!("Failed to decode GDScript fixture {:?}: {}", path, e)); + assert!(input.tick > 0, "tick should be positive"); + } +} +``` + +### 4.3 Summary of Encoding Asymmetry Test Coverage + +| Direction | What | Test location | When | +|-----------|------|--------------|------| +| Rust → GDScript (fixture) | Rust encodes snapshots with overlap-zone ticks, GDScript decodes | `test_msgpack_boundaries.gd` | Every PR | +| GDScript → Rust (fixture) | GDScript encodes inputs with overlap-zone ticks, Rust decodes | `serialization.rs` | Every PR | +| Rust → GDScript (raw bytes) | Hand-crafted uint_16/uint_32 bytes, GDScript decodes | `test_msgpack_boundaries.gd` | Every commit | +| GDScript → Rust (raw bytes) | Hand-crafted int_16/int_32 bytes, Rust decodes | `serialization.rs` | Every commit | + +--- + +## 5. Layer 3 Test Refinement + +Based on the lead decisions: +- `--test-mode` and `--port 0` are greenlit for Sprint 8 +- No CI pipeline, so Layer 3 runs manually via `make test-layer3` + +### 5.1 Refinements from Lead Decisions + +**Original spec (from Round 1):** `server_subprocess_sends_snapshot_on_connect` — launches server binary as child process, sends input, reads snapshot. + +**Refinements:** + +1. **Makefile target:** +```makefile +test-layer3: build-server + cd server && cargo test --test layer3 -- --ignored --nocapture +``` + +2. **Server startup protocol:** The `--test-mode` flag output format should be: +``` +SETTLED_REACH_SERVER_READY port=9876 +``` +This is both grep-parseable and human-readable. The test parses the port number. + +3. **Timeout handling:** Add a 10-second overall timeout on the test. If the server doesn't print the ready line within 5 seconds, fail with: +``` +Layer 3 FAILED: server did not become ready within 5 seconds + Stderr: [last 10 lines of server stderr] +``` + +4. **Cleanup robustness:** Use `Drop` guard on the child process to ensure cleanup even on panic: +```rust +struct ServerGuard(std::process::Child); +impl Drop for ServerGuard { + fn drop(&mut self) { + self.0.kill().ok(); + self.0.wait().ok(); + } +} +``` + +5. **No TCP port conflicts:** `--port 0` assigns a random port. The test reads the port from stdout. No hardcoded port numbers. + +### 5.2 Waiting for Dudley's Cross-Review + +Dudley's Round 2 may suggest changes to the `--test-mode` flag behavior (what content to load, what seed to use, whether to auto-exit). I'll incorporate those in Round 3 if needed. + +--- + +## Summary of Deliverables + +| Deliverable | Section | Status | +|-------------|---------|--------| +| Dudley's pause guard coverage audit (6 gaps + 3 additional) | 1.1 | Complete | +| Dudley's EntityRegistry coverage audit (3 real gaps + 2 additional) | 1.2 | Complete | +| Determinism fix coverage assessment | 1.3 | Complete | +| Bridge deserialization recommendation | 1.4 | Complete | +| Content cross-reference validation spec (8 checks) | 2 | Complete | +| `make pre-pr` target specification | 3 | Complete | +| Cross-language encoding asymmetry tests (4 directions) | 4 | Complete | +| Layer 3 test refinement | 5 | Complete | + +## Open Questions for Round 3 + +1. **For Dudley:** `SetTickRate(Half)` while paused — should this unpause the simulation? Current code sets the rate unconditionally (input.rs:165-168), which means `paused()` returns false on next tick. Is this intentional or a bug? +2. **For Dudley:** Entity respawn + registry — when bevy recycles an Entity index, does the current registry correctly handle the case where the old Entity's StableId was NOT unregistered before the new Entity spawns? (Registry keys use Entity, which includes generation — so this might be safe, but needs verification.) +3. **For Tyre:** The `make pre-pr` target — should it also include `make content-ron` (YAML→RON conversion)? This would catch content that passes schema validation but fails RON conversion. +4. **For Justine:** Fixture staleness check in `make pre-pr` — the `fixtures` target runs `cargo test --test gen_fixtures -- --ignored`, which requires a full server build. Should this be a separate `make pre-pr-full` target to keep the basic pre-PR fast? diff --git a/docs/workshops/test-architecture/hoshe-round3.md b/docs/workshops/test-architecture/hoshe-round3.md new file mode 100644 index 000000000..8c3037922 --- /dev/null +++ b/docs/workshops/test-architecture/hoshe-round3.md @@ -0,0 +1,677 @@ +# Hoshe — Round 3: Prioritized Test Backlog, Final Specifications + +**Workshop:** QA Strategy & Test Architecture +**Round:** 3 (Prioritization) +**Date:** 2026-02-17 +**Inputs:** All Round 1 + Round 2 outputs, round-1-notes.md, round-2-notes.md + +--- + +## 1. Prioritized Test Backlog + +Every test and infrastructure item proposed across all 5 tracks, ranked by priority. Format per the task description. + +### Sprint 8 P0 — Ship or the Gauntlet doesn't work + +These items directly prevent recurrence of Sprint 6-7 bugs, enable deterministic testing, or unblock all other test infrastructure. + +| # | Title | Track | Team | Dependencies | Effort | Bug class prevented | +|---|-------|-------|------|-------------|--------|-------------------| +| 1 | **Determinism Fix A: visible_ids HashSet -> BTreeSet + sort visible_tiles** | T2 | server | None | 0.5d | Bug #1 class (state divergence), non-deterministic sprint anomaly detection | +| 2 | **Determinism Fix B: Sort visible entities in snapshot by entity_id** | T2 | server | None | 0.25d | Non-deterministic golden file comparison | +| 3 | **Determinism Fix D: Sort movers by Entity bits in validate_movement** | T2 | server | None | 0.25d | Non-deterministic movement tie-breaking | +| 4 | **Server `--test-mode` and `--port 0` flags** | T3 | server | None | 1d | Unblocks Layer 3 tests, test client binary, Gauntlet headless testing | +| 5 | **`make pre-pr` target** | T5 | joint | Existing make targets | 0.5d | All bug classes (developer discipline enforcement) | +| 6 | **Pause guard: `movement_discarded_while_paused`** | T4 | server | None | 0.25d | Bug #3 (player moves while paused) | +| 7 | **Pause guard: `unpause_accepted_while_paused`** | T4 | server | None | 0.25d | Bug #3 class (pause state corruption) | +| 8 | **Pause guard: `pause_unpause_roundtrip_with_movement`** | T4 | server | None | 0.25d | Bug #3 class (full cycle regression) | +| 9 | **Content cross-reference validation (9 checks)** | T5 | joint | None | 1.5d | Content scaling regressions (dangling refs, missing files) | +| 10 | **Fixture staleness check in `make pre-pr`** | T5 | joint | #5 | 0.25d | Bug #4 class (protocol drift undetected) | + +**Sprint 8 P0 total: ~5d server + ~2d joint = ~7d combined effort** + +### Sprint 8 P1 — Core test infrastructure, high-value coverage + +| # | Title | Track | Team | Dependencies | Effort | Bug class prevented | +|---|-------|-------|------|-------------|--------|-------------------| +| 11 | **Determinism regression test: `gauntlet_deterministic_replay`** | T2 | server | #1, #2, #3 | 1d | All non-determinism bugs (master regression guard) | +| 12 | **Per-fix determinism unit tests (Fix A: equidistant NPCs, Fix D: same-tile movers)** | T2 | server | #1, #3 | 0.5d | Per-fix regression guards | +| 13 | **Remaining pause guard tests (stance_toggle, interact, batch_discard)** | T4 | server | None | 0.5d | Bug #3 class (pause guard edge cases) | +| 14 | **EntityRegistry lifecycle tests (stale mapping, register-after-unregister, unregister-unknown)** | T4 | server | None | 0.5d | Entity ID corruption, knowledge graph corruption | +| 15 | **Boundary value tests: GDScript encode-only (41 values)** | T4 | client | None | 1d | Bug #4 class (format boundary encoding errors) | +| 16 | **Boundary value tests: Rust encode-only + roundtrip** | T4 | server | None | 0.5d | Bug #4 class (Rust-side encoding errors) | +| 17 | **gen_fixtures.rs boundary extension (14 raw + 5 snapshot fixtures)** | T4 | joint | #16 | 0.5d | Bug #4 class (cross-language decode failures) | +| 18 | **Encoding asymmetry tests (4 directions: Rust->GDScript, GDScript->Rust, fixture + raw)** | T4 | joint | #17 | 1d | Encoding divergence between GDScript int_16 and Rust uint_16 for values 256-32767 | +| 19 | **`make fixtures-client` target (GDScript-generated fixtures for Rust)** | T4 | joint | #18 | 0.5d | Reverse-direction protocol validation | +| 20 | **Fog byte value constants (VIS_HIDDEN/PERIPHERAL/FORWARD, EXP_UNEXPLORED/EXPLORED/VISIBLE)** | T3 | client | None | 0.25d | Magic number drift, fog assertion targets | +| 21 | **Client P0 tests: monologue not lost on overwrite (Bug #5), camera static during pause (Bug #2)** | T3 | client | None | 0.5d | Bug #2, Bug #5 regression | +| 22 | **Client P1 tests: fog shader state (4 tests), entity lifecycle, pending recognition blob** | T3 | client | #20 | 1d | Information boundary enforcement, memory leaks | +| 23 | **`malformed_input_in_batch_rejects_entire_batch` test** | T4 | server | None | 0.25d | Documents batch-failure behavior (UQ-01 resolution) | +| 24 | **Test client binary scaffolding + CLI (`tooling/test-client/`)** | T3 | server | #4 | 0.5d | Unblocks text renderer, Layer 3, golden file comparison. Separate workspace crate, depends on server crate for shared types (ObserverSnapshot, PlayerInput, read_framed, write_framed). Lead override: NOT `server/src/bin/`. | +| 25 | **Text renderer library (`server/src/bridge/text_renderer.rs`)** | T3 | server | #24 | 0.5-1d | Unblocks human tester workflow, WRONG button. Pub-exported from server crate for `tooling/test-client/` consumption. | + +**Sprint 8 P1 total: ~5.5d server + ~3.25d client + ~2d joint = ~10.75d combined effort** + +### Sprint 9 P0 — Full test pipeline operational + +| # | Title | Track | Team | Dependencies | Effort | Bug class prevented | +|---|-------|-------|------|-------------|--------|-------------------| +| 26 | **Layer 3 subprocess test** | T4 | server | #4, #24 | 1d | Bug #1 (subprocess I/O blocking), binary startup issues | +| 27 | **Test client replay loading + tick-scheduled sending** | T3 | server | #24 | 0.5d | Unblocks scripted Gauntlet testing (`tooling/test-client/`) | +| 28 | **Test client golden file comparison (JSON diff)** | T3 | server | #24, #25 | 0.5d | Determinism regression, snapshot format drift (`tooling/test-client/`) | +| 29 | **Golden file test suite: `gauntlet_tick_10_matches_golden`** | T5 | server | #11, #28 | 0.5d | Any code change that alters snapshot output | +| 30 | **`make golden-diff` + `make golden-update` targets** | T5 | joint | #29 | 0.25d | Developer workflow for golden file updates | +| 31 | **Gauntlet first 3 rooms (Inventory Warehouse, Occlusion Corridor, Pause Chamber)** | T1 | server | #4 | 2-3d | Unblocks all room-specific testing | +| 32 | **Gauntlet room constants module (`server/src/test_world/constants.rs`)** | T1 | server | #31 | 0.5d | Stable coordinate references for all tests | +| 33 | **Content runtime validation: boot + tick 10 + snapshot** | T5 | server | None | 0.5d | Content that passes schema validation but panics at runtime | +| 34 | **Room reset trigger mechanism** | T1 | server | #31 | 1d | Unblocks sustainable human testing workflow | +| 35 | **Hub teleport action (`PlayerAction::TeleportToHub`)** | T1 | server | #31 | 0.5d | Unblocks efficient room navigation | + +**Sprint 9 P0 total: ~7-8d server + ~0.25d joint** + +### Sprint 9 P1 — Polish, UX, extended coverage + +| # | Title | Track | Team | Dependencies | Effort | Bug class prevented | +|---|-------|-------|------|-------------|--------|-------------------| +| 36 | **Client P2 tests: remaining camera (5), entity alpha/color (4), UI (7)** | T3 | client | None | 2d | Camera smoothing, entity rendering, UI elements | +| 37 | **Client P3 tests: z-layer ordering (4), entity lerp (3), Tyre additions (5)** | T3 | client | None | 1.5d | Z-layer constants, animation, recognition transition | +| 38 | **Client anti-tedium tests: bug report capture, progress hidden when not gauntlet** | T3 | client | None | 0.5d | Anti-tedium feature regression | +| 39 | **WRONG button (F12) MVP: snapshot + text render + description** | T1 | client + server | #25, #34 | 1.5d | Tester productivity (bug report capture) | +| 40 | **Room timer + personal bests** | T1 | client | #31 | 0.5d | Tester engagement | +| 41 | **Checklist YAML schema + `make checklist` generation** | T1 | joint | #31 | 0.5d | Tester workflow standardization | +| 42 | **Gauntlet next 3-4 rooms (Interaction Gallery, Fog Theater, Crowd Plaza, Dialogue Room)** | T1 | server | #31 | 2-3d | Extended system coverage | +| 43 | **Performance baseline tooling (`make perf-baseline`)** | T5 | joint | #31 | 1d | Performance regression detection | +| 44 | **Content scaling test (baseline + extra NPC comparative)** | T5 | server | #33 | 1d | Content additions breaking existing functionality | +| 45 | **Hub teleport client UX (Home key, fade transition)** | T1 | client | #35 | 0.5d | Anti-tedium UX | +| 46 | **Room reset client UX (reset plate tile, interaction verb, amber flash)** | T1 | client | #34 | 0.5d | Anti-tedium UX | +| 47 | **Auto-checklist progress tracking (client-side snapshot evaluation)** | T1 | client | #41 | 1d | Auto-tracking verification items | + +**Sprint 9 P1 total: ~3-4d server + ~8d client + ~1.5d joint** + +### Later (Sprint 10+) + +| # | Title | Track | Team | Dependencies | Effort | Bug class prevented | +|---|-------|-------|------|-------------|--------|-------------------| +| 48 | Gauntlet rooms: Sprint Gauntlet, Eavesdrop Alcove, Confrontation Stage | T1 | server | #31 | 3d | Interaction combination coverage | +| 49 | Gauntlet rooms: Sound Lab, Decay Observatory, Shift Change | T1 | server | Sound system, knowledge decay | 3d | Audio + decay + stress coverage | +| 50 | Cross-room transition test scenarios (T1-T8 from Gestalt) | T1 | server | #42, #48 | 2d | System combination bugs at room boundaries | +| 51 | WRONG button full captures (input history ring buffer, snapshot history, replay seed) | T1 | client + server | #39 | 1.5d | Full bug reproduction capability | +| 52 | Map-agnostic invariant tests (36 invariants from Gestalt) | T1 | server | Dynamic map system | 3d | Any-map structural guarantees | +| 53 | Fuzzy tests for dynamic/procedural maps | T5 | server | Dynamic map system | 1d | Procedural generation correctness | +| 54 | CI pipeline (Gitea Actions YAML, 3-tier) | T5 | joint | Lead greenlight | 1d | Automated merge gating | +| 55 | F3 debug overlay (deferred per Stig — WRONG button covers same data) | T1 | client | #39 | 1.5d | Real-time state inspection | +| 56 | Zone Gate room implementation | T1 | server | Multi-map system | 1d | Zone transition correctness | +| 57 | Content scaling stress test (100 ticks, max-NPC pack, tick budget) | T5 | server | #31, #43 | 1d | Performance at content scale | +| 58 | `blocked_entities` debug field on ObserverSnapshot | T3 | server | Feasibility TBD (R2-OQ-05) | 0.5d | LOS debugging in test client | +| 59 | Bidirectional relationship consistency warning (Check 9 in content validation) | T5 | joint | #9 | 0.25d | Asymmetric relationship detection | + +### Summary by Sprint + +| Sprint | P0 items | P1 items | Total effort | +|--------|----------|----------|-------------| +| **Sprint 8** | 10 items (~7d) | 15 items (~10.75d) | ~17.75d | +| **Sprint 9** | 10 items (~7.5d) | 12 items (~12.5d) | ~20d | +| **Sprint 10+** | — | 12 items (~18.75d) | ~18.75d | + +### Bug Catalogue Coverage Verification + +Every Sprint 6-7 bug class is covered by Sprint 8 P0/P1: + +| Bug | Root Cause | Backlog item(s) | +|-----|-----------|----------------| +| #1 Server never sends snapshots | read_framed() blocking | #4 (--test-mode), #26 (Layer 3 test) | +| #2 Camera doesn't center at startup | No snapshot until keystroke | #21 (client P0: camera static during pause) | +| #3 Player moves while paused | Pause guard not filtering | #6, #7, #8, #13 (pause guard suite) | +| #4 MessagePack -128 for 128 | Signed int8 boundary | #15, #16, #17, #18 (boundary value matrix) | +| #5 Monologue lost on overwrite | Snapshot buffer drops one-shots | #21 (client P0: monologue not lost) | +| #6 Snapshot overwrite warning spam | Server ticks faster than client | #1, #2 (determinism fixes make tick rate predictable), #29 (golden file catches drift) | + +--- + +## 2. Content Validation Final Spec + +Merged from Hoshe (8 checks, R2 section 2) and Justine (7 checks, R2 section 1). Reconciled per round-2-notes.md section 6. + +### Architecture + +Extend `tooling/validate-content` (Python) with a second pass after schema validation. No Rust dependency. Content authors validate without compiling the server. + +``` +make validate-content + | + +-- Pass 1: Schema validation (existing, unchanged) + | -> Each YAML file against its JSON Schema + | -> FAILS FAST on schema errors + | + +-- Pass 2: Cross-reference validation (NEW) + -> Build index of all defined entities + -> Walk all files, check every reference resolves + -> Report ALL errors (don't fail on first) +``` + +### The 9 Checks + +| # | Check name | Severity | Source files | Target | Error message template | Source | +|---|-----------|----------|-------------|--------|----------------------|--------| +| 1 | `canonical_id_uniqueness` | ERROR | `npcs/*.yaml` | `canonical_id` field | `XREF ERROR: duplicate canonical_id "{id}" -- Defined in: {file1} -- Duplicate in: {file2}` | Hoshe | +| 2 | `relationship_target_resolution` | ERROR | `npcs/*.yaml -> relationships[].target` | Other NPC canonical_ids | `XREF ERROR: unresolved relationship target "{target}" -- In: {file} -- Known canonical_ids: {list}` | Hoshe + Justine | +| 3 | `location_slug_resolution` | ERROR | `district.yaml -> locations[]` | `locations/{slug}.yaml` existence | `XREF ERROR: location slug "{slug}" not found -- In: {district_file} -- Expected file: {expected_path}` | Hoshe + Justine | +| 4 | `dialogue_location_resolution` | ERROR | `dialogue/**/*.yaml -> location` | District `locations[]` | `XREF ERROR: dialogue location "{location}" not in district -- In: {file} -- District locations: {list}` | Hoshe + Justine | +| 5 | `fact_id_resolution` | ERROR | `dialogue/**/*.yaml -> lines[].knowledge_grant.fact_id`, `npcs/*.yaml -> information.knows[]`, `monologue/**/*.yaml -> prerequisites.facts[].fact_id` | Knowledge catalogs | `XREF ERROR: unknown fact_id "{id}" -- In: {file} -- Canonical fact_ids: {count} defined` | Hoshe + Justine (absorbs `check-fact-ids`) | +| 6 | `triangle_membership_resolution` | ERROR | `npcs/*.yaml -> triangle_membership[]` | `triangles/{slug}.yaml` existence | `XREF ERROR: triangle "{slug}" not found -- In: {file} -- Available triangles: {list}` | Hoshe + Justine | +| 7 | `npc_count_accuracy` | WARNING | `district.yaml -> npc_count` | Actual count of `npcs/*.yaml` files | `XREF WARNING: npc_count mismatch -- Declared: {n} -- Actual NPC files: {m} -- In: {file}` | Hoshe | +| 8 | `dialogue_line_id_uniqueness` | ERROR | `dialogue/**/*.yaml -> lines[].id` | Per-file uniqueness | `XREF ERROR: duplicate dialogue line id "{id}" -- In: {file} -- First: line {n1} -- Duplicate: line {n2}` | Hoshe | +| 9 | `bidirectional_relationship_consistency` | WARNING | `npcs/*.yaml -> relationships[].target` | Reciprocal relationship exists | `XREF WARNING: {npc_a} has relationship to {npc_b} but no reciprocal found` | Justine | + +### Implementation Skeleton + +```python +class ContentIndex: + """Builds an index of all defined entities for cross-referencing.""" + + def __init__(self, content_dir: Path): + self.npcs: dict[str, Path] = {} # canonical_id -> defining file + self.locations: dict[Path, set] = {} # district path -> set of location slugs + self.triangles: dict[Path, set] = {} # district path -> set of triangle slugs + self.fact_ids: set[str] = set() # from knowledge catalogs + self.dialogue_files: list[tuple[Path, dict]] = [] # (file, parsed yaml) + self.npc_files: list[tuple[Path, dict]] = [] # (file, parsed yaml) + self.district_files: list[tuple[Path, dict]] = [] # (file, parsed yaml) + + def build(self): + """Scan all content files and populate the index.""" + self._scan_npcs() # Populates self.npcs + self._scan_locations() # Populates self.locations + self._scan_triangles() # Populates self.triangles + self._scan_knowledge() # Populates self.fact_ids + self._scan_districts() # Populates self.district_files + self._scan_dialogue() # Populates self.dialogue_files + + def validate_references(self) -> tuple[int, int]: + """Check all cross-references. Returns (error_count, warning_count).""" + errors = 0 + warnings = 0 + + errors += self._check_1_canonical_id_uniqueness() + errors += self._check_2_relationship_targets() + errors += self._check_3_location_slugs() + errors += self._check_4_dialogue_locations() + errors += self._check_5_fact_ids() + errors += self._check_6_triangle_membership() + warnings += self._check_7_npc_count() + errors += self._check_8_dialogue_line_ids() + warnings += self._check_9_bidirectional_relationships() + + return errors, warnings +``` + +### Integration with Existing Script + +```python +# In tooling/validate-content main() +def main() -> int: + # ... existing schema validation (Pass 1) ... + + if schema_errors > 0: + print(f"\nSchema validation failed ({schema_errors} errors) -- skipping cross-references") + return 1 + + # Pass 2: Cross-reference validation + index = ContentIndex(campaigns_dir) + index.build() + xref_errors, xref_warnings = index.validate_references() + + total_errors = schema_errors + xref_errors + print(f"\nValidated {file_count} files: {total_errors} errors, {xref_warnings} warnings") + return 1 if total_errors > 0 else 0 +``` + +### Phased Rollout + +| Phase | Checks | Sprint | Notes | +|-------|--------|--------|-------| +| 1 | #1, #2, #3, #6 (NPC + triangle + district) | Sprint 8 | Core entity graph validation | +| 2 | #4, #5, #8 (dialogue + fact_ids) | Sprint 8 | Absorbs `check-fact-ids` functionality | +| 3 | #7, #9 (warnings) | Sprint 9 | Advisory checks, non-blocking | + +### `check-fact-ids` Transition + +Once Check 5 is implemented in the Python validator, `check-fact-ids` becomes a thin backward-compatibility wrapper: + +```makefile +check-fact-ids: + @echo "NOTE: fact_id validation is now part of validate-content" + @tooling/validate-content --fact-ids-only +``` + +Keep the wrapper until all pre-commit hooks and developer muscle memory have been updated. + +--- + +## 3. Layer 3 Test Final Spec + +Incorporating Dudley's cross-review from R2 section 7: (1) use `to_vec` not `to_vec_named` for input serialization, (2) tolerate initial tick=0 snapshot before input is processed. + +### Test Identity + +``` +Name: server_subprocess_sends_snapshot_on_connect +File: server/tests/layer3.rs +Marker: #[test] #[ignore] -- run via `cargo test --test layer3 -- --ignored` +Makefile: make test-layer3 +Duration: <10 seconds +Tier: Nightly + pre-merge (manual via `make test-layer3`) +``` + +### Prerequisites + +- Server `--test-mode` flag (backlog #4) +- Server `--port 0` support (backlog #4) +- Server prints `LISTENING:{port}` to stdout after TCP bind, before accept (Dudley R2 section 2) +- Server crate pub-exports bridge types: `ObserverSnapshot`, `PlayerInput`, `PlayerAction`, `EntityKind`, `PROTOCOL_VERSION`, `read_framed`, `write_framed` (lead override: needed for both this test and `tooling/test-client/`) + +### Test Code + +```rust +use std::process::{Command, Stdio}; +use std::io::{BufRead, BufReader as StdBufReader, Write}; +use std::net::TcpStream; +use std::time::Duration; + +use settled_reach_server::bridge::framing::{read_framed, write_framed}; +use settled_reach_server::bridge::types::*; + +/// Drop guard: kill server on test exit (including panic). +struct ServerGuard(std::process::Child); +impl Drop for ServerGuard { + fn drop(&mut self) { + self.0.kill().ok(); + self.0.wait().ok(); + } +} + +#[test] +#[ignore] // Layer 3: slow, run with --ignored +fn server_subprocess_sends_snapshot_on_connect() { + // 1. Build server binary + let status = Command::new("cargo") + .args(["build", "--bin", "settled-reach-server"]) + .status() + .expect("cargo build failed"); + assert!(status.success(), "server build failed"); + + // 2. Launch server as subprocess + let mut server = Command::new("target/debug/settled-reach-server") + .args(["--test-mode", "--port", "0"]) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("failed to launch server"); + let _guard = ServerGuard(server); // kill on drop + + // 3. Parse port from stdout (5-second timeout) + let stdout = server.stdout.take().unwrap(); + let mut lines = StdBufReader::new(stdout).lines(); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + let port: u16 = loop { + if std::time::Instant::now() > deadline { + panic!("Layer 3 FAILED: server did not print LISTENING within 5 seconds"); + } + match lines.next() { + Some(Ok(line)) => { + if let Some(port_str) = line.strip_prefix("LISTENING:") { + break port_str.trim().parse().expect("parse port number"); + } + } + Some(Err(e)) => panic!("error reading server stdout: {}", e), + None => panic!("server stdout closed before printing LISTENING"), + } + }; + + // 4. Connect as client + let stream = TcpStream::connect(format!("127.0.0.1:{}", port)) + .expect("connect to server"); + stream.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); + + let mut reader = std::io::BufReader::new(stream.try_clone().unwrap()); + let mut writer = stream; + + // 5. Send input -- use to_vec (not to_vec_named) to match GDScript encoding + let inputs = vec![PlayerInput { + tick: 0, + action: PlayerAction::MoveNorth, + }]; + let payload = rmp_serde::to_vec(&inputs).expect("serialize input"); + write_framed(&mut writer, &payload).expect("send input"); + writer.flush().ok(); + + // 6. Read snapshot(s) -- tolerate initial tick=0 snapshot before input is processed + // (Dudley R2 adjustment: server may send snapshot before reading client input) + let response = read_framed(&mut reader) + .expect("read snapshot") + .expect("not EOF -- server must send snapshot"); + + let snapshot: ObserverSnapshot = + rmp_serde::from_slice(&response).expect("deserialize snapshot"); + + // 7. Assertions -- these hold for EITHER the initial snapshot or post-input snapshot + assert_eq!(snapshot.version, PROTOCOL_VERSION, + "snapshot version mismatch -- protocol incompatibility"); + // tick=0 is valid (initial snapshot before input processed) + assert!(snapshot.tick <= 1, + "unexpected tick {} -- expected 0 or 1", snapshot.tick); + assert!(!snapshot.entities.is_empty(), + "snapshot must contain at least the player entity"); + + let player = snapshot.entities.iter() + .find(|e| matches!(e.kind, EntityKind::Player)) + .expect("no Player entity in snapshot"); + assert!(player.x > 0.0, "player x must be positive (in proof room)"); + assert!(player.y > 0.0, "player y must be positive (in proof room)"); + + // 8. Cleanup -- ServerGuard kills on drop +} +``` + +### What Layer 3 Catches That Layer 2 Doesn't + +| Failure mode | Layer 2 (in-process) | Layer 3 (subprocess) | +|-------------|---------------------|---------------------| +| Binary startup panic | Not tested | Caught (server process exits non-zero) | +| Subprocess I/O blocking (Bug #1) | Cannot reproduce (shared memory) | Caught (real TCP, real blocking) | +| Port binding failure | Mocked | Real OS port allocation | +| Protocol version compiled into binary | Always matches (same compilation) | Catches stale binary vs test expectation | +| Server exit on client disconnect | Not tested | Tested (ServerGuard kills, server should exit cleanly) | + +### Makefile Target + +```makefile +test-layer3: build-server + cd server && cargo test --test layer3 -- --ignored --nocapture +``` + +--- + +## 4. Boundary Value Test Final Spec + +The 41-value matrix from Round 1, with placement decisions refined by Dudley's OQ-2 answer (asymmetric encoding is safe -- `rmp_serde` accepts `int_16`-encoded positive values for `u64` fields). + +### Resolved question: Is the encoding asymmetry a bug? + +**No.** Dudley traced through `rmp-serde 1.3.1` (R2 section 4): `Marker::I16 -> visit_i16 -> visit_i64 -> u64::try_from`. Positive `int_16` values deserialize correctly into `u64`. Negative values correctly rejected. The asymmetry is spec-valid and safe. + +**Implication:** We don't need to "fix" either encoder. We DO need explicit tests documenting that both decoders accept the other side's encoding. These tests are the insurance policy. + +### Test Placement Matrix + +| Test type | Location | Values tested | When run | Purpose | +|-----------|---------|---------------|----------|---------| +| **GDScript encode-only** | `client/tests/test_msgpack_boundaries.gd` | All 41 BV values: verify header byte + payload | Every commit (via `make test-client`) | Catches GDScript encoder regressions (Bug #4 class) | +| **GDScript encode-decode roundtrip** | `client/tests/test_msgpack_boundaries.gd` | All 41 BV values: encode -> decode -> assert equal | Every commit | Catches roundtrip failures | +| **Rust encode-decode roundtrip** | `server/tests/serialization.rs` | All 41 BV values as `i64`: `to_vec -> from_slice -> assert_eq` | Every commit (via `make test-server`) | Catches Rust-side encoding regressions | +| **Rust-generated fixtures decoded by GDScript** | `gen_fixtures.rs` generates -> `test_msgpack_boundaries.gd` verifies | 14 raw int fixtures + 5 snapshot fixtures at boundary ticks | Every PR (via `make pre-pr`) | Cross-language: Rust encoder -> GDScript decoder | +| **GDScript-generated fixtures decoded by Rust** | `test_gen_client_fixtures.gd` generates -> `serialization.rs` verifies | Inputs at ticks 256, 32767, 65536 | Every PR (via `make pre-pr`) | Cross-language: GDScript encoder -> Rust decoder | +| **Asymmetry-specific raw bytes** | Both `test_msgpack_boundaries.gd` and `serialization.rs` | Hand-crafted bytes: uint_16 decoded by GDScript, int_16 decoded by Rust | Every commit | Directly tests the encoding overlap zone | + +### The 41 Boundary Values (unchanged from R1) + +**Positive boundaries (25 values):** + +| ID | Value | Expected format | Header | Payload | +|----|-------|----------------|--------|---------| +| BV-P01 | 0 | pos fixint | 0x00 | - | +| BV-P02 | 1 | pos fixint | 0x01 | - | +| BV-P03 | 126 | pos fixint | 0x7e | - | +| BV-P04 | 127 | pos fixint | 0x7f | - | +| BV-P05 | 128 | uint 8 | 0xcc | 0x80 | +| BV-P06 | 129 | uint 8 | 0xcc | 0x81 | +| BV-P07 | 254 | uint 8 | 0xcc | 0xfe | +| BV-P08 | 255 | uint 8 | 0xcc | 0xff | +| BV-P09 | 256 | int 16 (GDScript) / uint 16 (Rust) | 0xd1 / 0xcd | varies | +| BV-P10 | 257 | int 16 / uint 16 | 0xd1 / 0xcd | varies | +| BV-P11 | 32766 | int 16 / uint 16 | 0xd1 / 0xcd | varies | +| BV-P12 | 32767 | int 16 / uint 16 | 0xd1 / 0xcd | varies | +| BV-P13 | 32768 | uint 16 | 0xcd | 0x80 0x00 | +| BV-P14 | 32769 | uint 16 | 0xcd | 0x80 0x01 | +| BV-P15 | 65534 | uint 16 | 0xcd | 0xff 0xfe | +| BV-P16 | 65535 | uint 16 | 0xcd | 0xff 0xff | +| BV-P17 | 65536 | int 32 (GDScript) / uint 32 (Rust) | 0xd2 / 0xce | varies | +| BV-P18 | 65537 | int 32 / uint 32 | 0xd2 / 0xce | varies | +| BV-P19 | 2147483646 | int 32 / uint 32 | 0xd2 / 0xce | varies | +| BV-P20 | 2147483647 | int 32 / uint 32 | 0xd2 / 0xce | varies | +| BV-P21 | 2147483648 | uint 32 | 0xce | 0x80 0x00 0x00 0x00 | +| BV-P22 | 4294967294 | uint 32 | 0xce | 0xff 0xff 0xff 0xfe | +| BV-P23 | 4294967295 | uint 32 | 0xce | 0xff 0xff 0xff 0xff | +| BV-P24 | 4294967296 | int 64 | 0xd3 | 0x00...0x01 0x00...0x00 | +| BV-P25 | 9223372036854775807 | int 64 | 0xd3 | 0x7f 0xff...0xff | + +**Negative boundaries (16 values):** + +| ID | Value | Expected format | Header | Payload | +|----|-------|----------------|--------|---------| +| BV-N01 | -1 | neg fixint | 0xff | - | +| BV-N02 | -31 | neg fixint | 0xe1 | - | +| BV-N03 | -32 | neg fixint | 0xe0 | - | +| BV-N04 | -33 | int 8 | 0xd0 | 0xdf | +| BV-N05 | -34 | int 8 | 0xd0 | 0xde | +| BV-N06 | -127 | int 8 | 0xd0 | 0x81 | +| BV-N07 | -128 | int 8 | 0xd0 | 0x80 | +| BV-N08 | -129 | int 16 | 0xd1 | 0xff 0x7f | +| BV-N09 | -130 | int 16 | 0xd1 | 0xff 0x7e | +| BV-N10 | -32767 | int 16 | 0xd1 | 0x80 0x01 | +| BV-N11 | -32768 | int 16 | 0xd1 | 0x80 0x00 | +| BV-N12 | -32769 | int 32 | 0xd2 | 0xff 0xff 0x7f 0xff | +| BV-N13 | -2147483647 | int 32 | 0xd2 | 0x80 0x00 0x00 0x01 | +| BV-N14 | -2147483648 | int 32 | 0xd2 | 0x80 0x00 0x00 0x00 | +| BV-N15 | -2147483649 | int 64 | 0xd3 | 0xff...0xff 0x7f 0xff...0xff | +| BV-N16 | -9223372036854775808 | int 64 | 0xd3 | 0x80 0x00...0x00 | + +### Encoding Overlap Zone Tests (new from R2) + +These 4 tests specifically target the asymmetric encoding between GDScript (`int_16` for 256-32767) and Rust (`uint_16` for same range). Each direction must pass. + +**Direction 1: GDScript decoder accepts Rust-style unsigned encodings** +```gdscript +# test_msgpack_boundaries.gd +func test_decode_rust_uint16_256() -> void: + var bytes = PackedByteArray([0xcd, 0x01, 0x00]) # uint_16(256) + var result = Messagepack.decode(bytes) + assert_that(result.value).is_equal(256) + +func test_decode_rust_uint16_32767() -> void: + var bytes = PackedByteArray([0xcd, 0x7f, 0xff]) # uint_16(32767) + var result = Messagepack.decode(bytes) + assert_that(result.value).is_equal(32767) + +func test_decode_rust_uint32_65536() -> void: + var bytes = PackedByteArray([0xce, 0x00, 0x01, 0x00, 0x00]) # uint_32(65536) + var result = Messagepack.decode(bytes) + assert_that(result.value).is_equal(65536) + +func test_decode_rust_uint32_2147483647() -> void: + var bytes = PackedByteArray([0xce, 0x7f, 0xff, 0xff, 0xff]) # uint_32(2^31-1) + var result = Messagepack.decode(bytes) + assert_that(result.value).is_equal(2147483647) +``` + +**Direction 2: Rust decoder accepts GDScript-style signed encodings** +```rust +#[test] +fn rust_decodes_gdscript_signed_overlap_values() { + let cases: Vec<(Vec, u64)> = vec![ + (vec![0xd1, 0x01, 0x00], 256), // int_16(256) + (vec![0xd1, 0x7f, 0xff], 32767), // int_16(32767) + (vec![0xd2, 0x00, 0x01, 0x00, 0x00], 65536), // int_32(65536) + (vec![0xd2, 0x7f, 0xff, 0xff, 0xff], 2147483647), // int_32(2^31-1) + ]; + for (bytes, expected) in cases { + let value: u64 = rmp_serde::from_slice(&bytes) + .unwrap_or_else(|e| panic!( + "Rust failed to decode signed-encoded {} from {:?}: {}", expected, bytes, e + )); + assert_eq!(value, expected); + } +} +``` + +**Direction 3: Rust-generated fixtures with overlap-zone ticks** +```rust +// In gen_fixtures.rs +fn generate_encoding_asymmetry_fixtures() { + let ticks = [(256, "snapshot_tick_256"), (500, "snapshot_tick_500"), + (32767, "snapshot_tick_32767"), (65536, "snapshot_tick_65536")]; + for (tick, name) in ticks { + let snapshot = fixture_snapshot(tick, vec![]); + write_fixture(name, &rmp_serde::to_vec_named(&snapshot).unwrap()); + } +} +``` + +**Direction 4: GDScript-generated fixtures decoded by Rust** + +New `make fixtures-client` target runs GDScript test that generates `.msgpack` files at `server/tests/fixtures/gdscript/`. Rust test verifies decode. + +### Fixture Regeneration After visible_tiles Sorting + +Per Stig R2 (OQ-1): two test files reference `visible_tiles[0]` by index (`test_protocol.gd:299-301`, `test_rendering.gd:66-69`). After determinism Fix A sorts `visible_tiles`, fixture regeneration via `make fixtures` will update these. The tests themselves don't need code changes -- just regenerated fixture data. Flag for whoever implements Fix A: run `make fixtures` and commit the updated fixture files. + +--- + +## 5. `make pre-pr` Final Spec + +Reconciled from Hoshe (6-step, R2 section 3) and Justine (5-step, R2 section 2). The proposals are compatible -- Justine's chain is a subset of Hoshe's. This spec uses Hoshe's branch-specific variants with Justine's Makefile structure. + +### Resolved Tensions + +| Question | Resolution | +|----------|-----------| +| Fixture staleness: BLOCKER or WARNING? | **BLOCKER** (Tyre's argument from R2 section 4.3). Stale fixtures = false positive client tests. `exit 1` on stale fixtures. | +| Include `check-fact-ids` separately? | **Yes, until Phase 2 of content validation** absorbs it. Then becomes alias. | +| Include `content-ron` (YAML->RON)? | **No.** RON conversion is a content loader concern, not a pre-PR concern. If RON conversion fails, the boot-and-tick test (backlog #33) catches it. | +| `pre-pr-full` vs `pre-pr`? | **No split.** The fixture staleness check adds ~10-15s. Total under 3 minutes. Not worth a separate target. | +| PR tier time budget? | **<3 minutes** for `make pre-pr` (local, incremental). **<15 minutes** for future CI (Tyre's revision, covers clean-cache worst case). | + +### Definitive Makefile + +```makefile +# ================================================================ +# Pre-PR verification -- run before submitting any PR +# ================================================================ + +pre-pr: pre-pr-lint pre-pr-build pre-pr-test pre-pr-validate pre-pr-fixtures + @echo "" + @echo "=== PRE-PR: ALL CHECKS PASSED ===" + @echo "Safe to create PR." + +pre-pr-lint: lint-server lint-client + @echo "--- Lint: PASS ---" + +pre-pr-build: build-server build-client + @echo "--- Build: PASS ---" + +pre-pr-test: test-server test-client + @echo "--- Tests: PASS ---" + +pre-pr-validate: validate-content check-fact-ids + @echo "--- Content validation: PASS ---" + +pre-pr-fixtures: + @echo "Checking fixture staleness..." + @cd server && cargo test --test gen_fixtures -- --ignored 2>/dev/null + @if git diff --quiet client/tests/fixtures/; then \ + echo "--- Fixtures: UP TO DATE ---"; \ + else \ + echo ""; \ + echo "--- FIXTURES STALE ---"; \ + echo " Protocol changed but fixtures not regenerated."; \ + echo " Stale fixtures make all client tests FALSE POSITIVES."; \ + echo ""; \ + echo " Changed files:"; \ + git diff --stat client/tests/fixtures/; \ + echo ""; \ + echo " Fix: commit the updated fixtures with your protocol change."; \ + exit 1; \ + fi + +# ================================================================ +# Branch-specific variants (faster, scope-appropriate) +# ================================================================ + +pre-pr-server: lint-server build-server test-server pre-pr-fixtures + @echo "=== Server pre-PR: PASSED ===" + +pre-pr-client: lint-client build-client test-client + @echo "=== Client pre-PR: PASSED ===" + +pre-pr-content: validate-content check-fact-ids + @echo "=== Content pre-PR: PASSED ===" +``` + +### Execution Chain + +``` +pre-pr (~2.5 min incremental, ~3 min clean) + | + +-- 1. lint-server + lint-client (~15s) Fast syntax/style check + +-- 2. build-server + build-client (~30-90s) Compilation errors + +-- 3. test-server + test-client (~15-30s) Logic errors + +-- 4. validate-content + check-fact-ids (~5s) Content correctness + +-- 5. fixture staleness check (~10-15s) Protocol drift +``` + +**Fail-fast behavior:** Make's default `&&` chaining. If lint fails, build is skipped. If build fails, tests are skipped. Error messages from each step are visible in terminal. + +### Developer Documentation (add to docs/DEVOPS.md) + +```markdown +## Pre-PR Checklist + +Before pushing a PR, run: + + make pre-pr + +This runs all checks in order: lint -> build -> test -> content validation -> fixture staleness. + +For branch-specific checks: +- Server changes: `make pre-pr-server` +- Client changes: `make pre-pr-client` +- Content changes: `make pre-pr-content` + +If `fixtures-check` fails, your protocol changes require fixture regeneration: + + make fixtures + git add client/tests/fixtures/ + git commit -m "chore(fixtures): regenerate for protocol vN" + +Total runtime: ~2.5 minutes (incremental build). Under 3 minutes clean. +``` + +### Future CI Mapping + +When the lead greenlights CI, the `make pre-pr` chain maps directly to the PR tier: + +| `make pre-pr` step | CI job | Duration | +|--------------------|--------|----------| +| lint-server + lint-client | `commit-checks` job | ~15s | +| build-server + build-client | `pr-checks` job (parallel) | ~90s | +| test-server + test-client | `pr-checks` job (parallel) | ~30s | +| validate-content + check-fact-ids | `commit-checks` job | ~5s | +| fixture staleness | `pr-checks` job (after build) | ~15s | + +No new work needed -- just wrap `make pre-pr` in a Gitea Actions workflow. + +--- + +## Appendix: Open Questions Deferred to Implementation + +These questions were raised in Round 2 but do not block the backlog. They should be resolved during implementation by the assigned team. + +| ID | Question | Raised by | Assigned to | When to resolve | +|----|----------|-----------|-------------|----------------| +| R2-OQ-01 | `SetTickRate(Half)` while paused -- should this unpause? | Hoshe | Dudley | During pause guard test implementation (#13) | +| R2-OQ-02 | Entity respawn + registry stale mapping | Hoshe | Dudley | During EntityRegistry test implementation (#14) | +| R2-OQ-05 | `blocked_entities` debug field feasibility | Ozzie | Dudley | During test client binary work (#24) | +| R2-OQ-09 | Gauntlet room ordering in YAML (StableId assignment) | Gestalt | Dudley | During Gauntlet first rooms (#31) | +| R2-OQ-10 | Per-room reset sufficient, or need full restart? | Gestalt | Dudley | During room reset implementation (#34) | +| R2-OQ-11 | Client tests headless stability | Justine | Stig | During client test expansion (#21, #22) | +| OQ-11 | Cross-room checklist YAML location | Ozzie | Gestalt | During checklist spec (#41) | diff --git a/docs/workshops/test-architecture/justine-round2.md b/docs/workshops/test-architecture/justine-round2.md new file mode 100644 index 000000000..aac1a1e53 --- /dev/null +++ b/docs/workshops/test-architecture/justine-round2.md @@ -0,0 +1,750 @@ +# Justine — Round 2: Content Validation Tooling + Performance Baseline + +**Workshop:** QA Strategy & Test Architecture +**Track:** 5 (Content Scaling & CI Pipeline) +**Date:** 2026-02-17 +**Context:** Lead decided: no Gitea Actions for now. Manual `make ci` stays. Focus shifts to local tooling: validation expansion, pre-PR checklist, perf baselines, golden file diffing. + +--- + +## Task 1: `make validate-content` Expansion — Cross-Reference Validation + +### Current State + +`tooling/validate-content` is a Python script (133 lines) that: +- Walks `content/campaigns/**/*.yaml` +- Maps each file to a JSON Schema based on directory name or filename +- Validates structure via `jsonschema.validate()` +- Reports errors with path and field info +- Does NOT check cross-references between files + +`tooling/check-fact-ids` is a bash script (90 lines) that: +- Extracts `fact_id:` values from knowledge catalogs (`content/global/knowledge/`) +- Extracts `fact_id:` references from campaign content +- Cross-checks references against canonical definitions +- Advisory mode when catalogs are unpopulated, enforcing mode when populated + +### Cross-Reference Inventory + +After reviewing the content schemas and actual YAML files, here are all cross-reference relationships: + +| Source File Type | Field | References | Target File Type | +|-----------------|-------|------------|-----------------| +| NPC profile | `relationships[].target` | `npc:{slug}` | Other NPC profiles | +| NPC profile | `triangle_membership[]` | triangle slug | Triangle definitions | +| NPC profile | `information.knows[]` | fact_id string | Knowledge catalogs | +| Dialogue pool | `location` | location slug | Location definitions | +| Dialogue pool | `role` | template role slug | Template definitions | +| Dialogue pool | `lines[].knowledge_grant.fact_id` | fact_id string | Knowledge catalogs | +| Monologue pool | `location` | location slug | Location definitions (or "general") | +| Monologue pool | `lines[].prerequisites.facts[].fact_id` | fact_id string | Knowledge catalogs | +| Monologue pool | `lines[].prerequisites.relationship.target` | entity ref | NPC profiles | +| Monologue pool | `lines[].prerequisites.entity_attributes[].entity` | entity ref | NPC profiles | +| Triangle | `members[].npc` | `npc:{slug}` | NPC profiles | +| District | `locations[]` | location slug | Location definitions | +| Routine | location references | location slug | Location definitions | + +### Recommendation: Extend the Python Validator + +**Extend `tooling/validate-content`, not a separate Rust step.** Reasons: + +1. The validator already loads and parses every YAML file. Adding cross-reference checks is O(1) additional passes over the same data. +2. Python is the right tool: string matching, file walking, error reporting. No compilation step. +3. `check-fact-ids` (bash) already handles fact_id validation. Absorb its logic into the Python validator to eliminate duplication and get consistent error reporting. +4. A Rust validation step would require building the server before validating content — that's a much heavier dependency chain. Content authors (copy team) should be able to validate without compiling Rust. + +### Implementation Design + +Add a second pass to `tooling/validate-content` after schema validation: + +```python +# Phase 1: Schema validation (existing) +# Phase 2: Cross-reference validation (new) + +class ContentIndex: + """Builds an index of all defined entities for cross-referencing.""" + + def __init__(self, content_dir: Path): + self.npcs: set[str] = set() # canonical_id values + self.locations: set[str] = set() # location slugs + self.triangles: set[str] = set() # triangle canonical_id values + self.fact_ids: set[str] = set() # from knowledge catalogs + self.roles: set[str] = set() # template role slugs + self.district_locations: dict[str, list[str]] = {} # district → listed locations + + def build(self): + """Scan all content files and populate the index.""" + self._scan_npcs() + self._scan_locations() + self._scan_triangles() + self._scan_knowledge() + self._scan_districts() + self._scan_templates() + + def validate_references(self) -> list[ValidationError]: + """Check all cross-references against the index.""" + errors = [] + errors += self._check_npc_relationships() + errors += self._check_npc_triangle_membership() + errors += self._check_npc_fact_ids() + errors += self._check_dialogue_locations() + errors += self._check_dialogue_roles() + errors += self._check_dialogue_fact_ids() + errors += self._check_monologue_locations() + errors += self._check_monologue_prerequisites() + errors += self._check_triangle_members() + errors += self._check_district_locations() + return errors +``` + +### Concrete Checks + +**Check 1: NPC relationship targets resolve** +``` +For each NPC profile: + For each relationship in relationships[]: + Assert relationship.target exists in npcs index + Error: "npc:kael-davan references unknown NPC npc:nonexistent in relationships" +``` + +**Check 2: Triangle members resolve** +``` +For each triangle: + For each member in members[]: + Assert member.npc exists in npcs index + Error: "triangle hub-power references unknown NPC npc:missing" +``` + +**Check 3: NPC triangle_membership matches triangle definitions** +``` +For each NPC profile: + For each triangle_slug in triangle_membership[]: + Assert triangle_slug exists in triangles index + Error: "npc:kael-davan claims membership in unknown triangle 'missing-triangle'" +``` + +**Check 4: Dialogue pool location resolves** +``` +For each dialogue pool: + Assert pool.location exists in locations index + Error: "dialogue pool kael-davan.yaml references unknown location 'nonexistent-bar'" +``` + +**Check 5: Fact IDs resolve (absorb check-fact-ids)** +``` +For each fact_id reference (NPC knows[], dialogue knowledge_grant, monologue prerequisites): + Assert fact_id exists in knowledge catalogs + Error: "npc:kael-davan references unknown fact_id 'contraband.nonexistent'" + (Advisory mode when catalogs are unpopulated, same as current check-fact-ids) +``` + +**Check 6: District location list matches actual location files** +``` +For each district: + For each location_slug in locations[]: + Assert a location YAML exists at locations/{slug}.yaml + Error: "district transit lists location 'ghost-alley' but no location file exists" + Also: warn if location files exist that aren't listed in the district +``` + +**Check 7: Bidirectional relationship consistency (WARNING, not ERROR)** +``` +For each NPC A with relationship to NPC B: + Warn if NPC B has no relationship back to NPC A + Warning: "npc:kael-davan has relationship to npc:naia-tamm but no reciprocal found" + (This is a warning because asymmetric relationships may be intentional) +``` + +### Makefile Change + +```makefile +validate-content: + @tooling/validate-content + +# Deprecate separate check-fact-ids once absorbed into validate-content +# Keep as alias for backward compatibility during transition +check-fact-ids: + @tooling/check-fact-ids +``` + +After the Python validator absorbs fact_id checking, `check-fact-ids` becomes a thin wrapper that calls `tooling/validate-content --fact-ids-only` for the pre-commit hook (fast path, <2 seconds). + +### Phased Rollout + +| Phase | Checks Added | Timeline | +|-------|-------------|----------| +| 1 | NPC relationship targets, triangle members, district locations | First implementation ticket | +| 2 | Dialogue/monologue location + fact_id (absorb check-fact-ids) | Follow-up ticket | +| 3 | Bidirectional relationship warnings, role validation | Polish ticket | + +--- + +## Task 2: `make pre-pr` Target + +Since there's no automated CI, developers need a single command that runs the full verification suite before creating a PR. This replaces the discipline of "remember to run lint, build, test, and validate." + +### Design + +```makefile +# --- Pre-PR verification (replaces CI until automated pipeline exists) --- + +pre-pr: pre-pr-lint pre-pr-build pre-pr-test pre-pr-validate pre-pr-fixtures + @echo "" + @echo "=== PRE-PR: ALL CHECKS PASSED ===" + @echo "Safe to create PR." + +pre-pr-lint: lint-server lint-client + @echo "--- Lint: PASS ---" + +pre-pr-build: build-server build-client + @echo "--- Build: PASS ---" + +pre-pr-test: test-server test-client + @echo "--- Tests: PASS ---" + +pre-pr-validate: validate-content + @echo "--- Content validation: PASS ---" + +pre-pr-fixtures: + @echo "Checking fixture staleness..." + @cd server && cargo test --test gen_fixtures -- --ignored 2>/dev/null + @if git diff --quiet client/tests/fixtures/; then \ + echo "--- Fixtures: UP TO DATE ---"; \ + else \ + echo "--- Fixtures: STALE ---"; \ + echo " Fixtures changed after regeneration. Commit the updated fixtures:"; \ + git diff --stat client/tests/fixtures/; \ + exit 1; \ + fi +``` + +### Chain Order and Rationale + +``` +pre-pr + ├── 1. lint-server + lint-client (fastest, catch formatting/style) + ├── 2. build-server + build-client (catch compilation errors) + ├── 3. test-server + test-client (catch logic errors) + ├── 4. validate-content (catch content errors) + └── 5. fixture staleness check (catch protocol drift) +``` + +**Order matters:** Lint is fastest and catches the cheapest errors. Build must succeed before tests can run. Content validation is independent of build but runs after to keep the fast-fail path clean. Fixtures are last because they require a server build + test run. + +### Duration Budget + +| Step | Expected Duration | Notes | +|------|------------------|-------| +| lint-server | ~10s | clippy + fmt check | +| lint-client | ~5s | headless Godot script check | +| build-server | ~30-90s | incremental Rust build | +| build-client | ~5s | Godot headless import | +| test-server | ~15-30s | cargo nextest | +| test-client | ~10-20s | gdUnit4 headless | +| validate-content | ~2s | Python YAML walk | +| fixture check | ~10-15s | build + gen_fixtures + git diff | +| **Total** | **~90-180s** | Under 3 minutes for clean incremental build | + +### Failure Behavior + +Each step uses Make's default behavior: fail-fast on non-zero exit. If `lint-server` fails, the chain stops immediately — no point building if there are lint errors. + +Output on failure: +``` +cd server && cargo clippy -- -D warnings +error: unused variable `x` + --> src/simulation/movement.rs:42:9 +make: *** [lint-server] Error 1 +``` + +Output on success: +``` +--- Lint: PASS --- +--- Build: PASS --- +--- Tests: PASS --- +--- Content validation: PASS --- +--- Fixtures: UP TO DATE --- + +=== PRE-PR: ALL CHECKS PASSED === +Safe to create PR. +``` + +### What pre-pr Does NOT Do + +- Does not run Layer 3 subprocess tests (too slow, nightly-tier) +- Does not run performance benchmarks (machine-dependent, separate target) +- Does not run Gauntlet golden file tests (depends on Gauntlet implementation) +- Does not push or create the PR (that's `make push-pr` or the `/push-pr` skill) + +These are separate targets for developers who want deeper verification: + +```makefile +# Optional deeper checks (not part of pre-pr) +pre-pr-deep: pre-pr test-perf test-gauntlet + @echo "=== DEEP VERIFICATION: ALL CHECKS PASSED ===" +``` + +--- + +## Task 3: Local Performance Baseline Tooling + +### Design: `make perf-baseline` + +Even without CI, developers need to track performance locally. The Gauntlet doesn't exist yet, but we can design the tooling now and wire it up when the Gauntlet lands. + +#### Baseline File Format + +`tests/perf/baseline.json` — checked into the repo: + +```json +{ + "_meta": { + "format_version": 1, + "updated_at": "2026-02-17T14:30:00Z", + "updated_by": "developer-name", + "commit": "abc123f", + "machine": "workstation-01", + "rust_version": "1.82.0", + "build_profile": "release" + }, + "benchmarks": { + "gauntlet_100_ticks": { + "median_ms": 142.3, + "min_ms": 138.1, + "max_ms": 156.7, + "runs": 5, + "description": "Gauntlet map, seed 42, 100 simulation ticks, release build" + }, + "shadowcast_150x150_open": { + "median_ms": 12.4, + "min_ms": 11.8, + "max_ms": 14.2, + "runs": 5, + "description": "Shadowcast benchmark, 150x150 open field, 1000 iterations" + } + } +} +``` + +#### Measurement Tooling + +New script: `tooling/perf-measure` + +```bash +#!/usr/bin/env bash +# Run performance benchmarks and compare against baseline. +# Usage: tooling/perf-measure [--update] +# --update: write results as new baseline (otherwise compare only) + +set -euo pipefail +REPO_ROOT="$(git rev-parse --show-toplevel)" +BASELINE="$REPO_ROOT/tests/perf/baseline.json" +RESULTS="$REPO_ROOT/.cache/perf_results.json" +THRESHOLD_WARN=15 # % regression = warning +THRESHOLD_FAIL=30 # % regression = failure + +# Build release (perf measurements on debug builds are meaningless) +echo "Building server (release)..." +cd "$REPO_ROOT/server" && cargo build --release 2>/dev/null + +# Run benchmark tests (--ignored = benchmark tests, --nocapture for timing output) +echo "Running benchmarks (5 iterations each)..." +cargo test --release --test gauntlet_perf -- --ignored --nocapture 2>&1 \ + | tee "$REPO_ROOT/.cache/perf_raw.txt" + +# Parse results into JSON (perf test outputs structured timing data) +python3 "$REPO_ROOT/tooling/parse-perf-output" \ + "$REPO_ROOT/.cache/perf_raw.txt" > "$RESULTS" + +# Compare against baseline +if [ -f "$BASELINE" ]; then + python3 "$REPO_ROOT/tooling/compare-perf" \ + "$BASELINE" "$RESULTS" \ + --warn-threshold "$THRESHOLD_WARN" \ + --fail-threshold "$THRESHOLD_FAIL" +else + echo "No baseline found at $BASELINE" + echo "Run 'make perf-baseline-update' to create initial baseline." +fi + +# Optionally update baseline +if [ "${1:-}" = "--update" ]; then + cp "$RESULTS" "$BASELINE" + echo "Baseline updated. Commit tests/perf/baseline.json to save." +fi +``` + +#### Rust-Side Benchmark Test + +The benchmark test itself lives in `server/tests/gauntlet_perf.rs`: + +```rust +#[test] +#[ignore] // Run with: cargo test --release --test gauntlet_perf -- --ignored +fn gauntlet_100_ticks_5_runs() { + let mut times_ms: Vec = Vec::new(); + + for run in 0..5 { + let mut app = build_gauntlet_app(42); // seed 42 + + let start = Instant::now(); + for _ in 0..100 { + app.update(); + } + let elapsed = start.elapsed().as_secs_f64() * 1000.0; + times_ms.push(elapsed); + eprintln!("PERF_RUN: gauntlet_100_ticks run={} ms={:.2}", run, elapsed); + } + + times_ms.sort_by(|a, b| a.partial_cmp(b).unwrap()); + let median = times_ms[2]; // middle of 5 + let min = times_ms[0]; + let max = times_ms[4]; + + eprintln!("PERF_RESULT: gauntlet_100_ticks median={:.2} min={:.2} max={:.2} runs=5", + median, min, max); +} +``` + +Output format is structured for machine parsing: +``` +PERF_RUN: gauntlet_100_ticks run=0 ms=141.23 +PERF_RUN: gauntlet_100_ticks run=1 ms=138.12 +PERF_RUN: gauntlet_100_ticks run=2 ms=142.87 +PERF_RUN: gauntlet_100_ticks run=3 ms=156.71 +PERF_RUN: gauntlet_100_ticks run=4 ms=139.44 +PERF_RESULT: gauntlet_100_ticks median=141.23 min=138.12 max=156.71 runs=5 +``` + +#### Human-Readable Comparison Output + +``` +=== Performance Comparison === +Baseline: abc123f (2026-02-15) on workstation-01 +Current: def456g (2026-02-17) on workstation-01 + +Benchmark Baseline Current Delta Status +───────────────────────────────────────────────────────────────────── +gauntlet_100_ticks 142.3ms 148.7ms +4.5% PASS +shadowcast_150x150_open 12.4ms 12.1ms -2.4% PASS (faster) + +Overall: 2/2 PASS, 0 WARN, 0 FAIL +``` + +Warning example: +``` +gauntlet_100_ticks 142.3ms 168.9ms +18.7% WARN ⚠ + Exceeds 15% threshold. Investigate recent changes. +``` + +Failure example: +``` +gauntlet_100_ticks 142.3ms 203.1ms +42.7% FAIL ✗ + Exceeds 30% threshold. Likely regression. +``` + +#### Makefile Targets + +```makefile +# Performance benchmarks (local only — no CI yet) +perf-baseline: + @tooling/perf-measure + +perf-baseline-update: + @tooling/perf-measure --update + +perf-quick: + cd server && cargo test --release --test shadowcast_bench -- --ignored --nocapture +``` + +#### Machine Variance Handling + +Local measurements are inherently noisy. Mitigation: +- **5 runs, take median**: Absorbs outliers from background processes +- **Release builds only**: Debug builds have 10-20x variance from unoptimized code paths +- **Machine tag in baseline**: Baseline records which machine it was measured on. Comparing across machines is meaningless — warn if machine tag differs +- **15% threshold**: Generous enough for laptop variance (thermal throttling, background apps) +- **Commit tag in baseline**: Developers can see which commit the baseline was set against + +--- + +## Task 4: Golden File Diff Tooling + +### Design: `make golden-diff` + +When the Gauntlet golden file changes, developers need to see what changed. The question: Rust test that prints the diff, or separate tool? + +### Recommendation: Rust Test That Prints the Diff + +**A Rust test, not a separate tool.** Reasons: + +1. The golden file is an `ObserverSnapshot` — a Rust struct. Rust code already knows how to deserialize and compare it field by field. +2. A Rust test naturally lives alongside the Gauntlet test suite. It's just a test that produces better output on failure. +3. No additional tooling dependency (no Python, no external diff tool). +4. The test can use `#[derive(Debug)]` to print readable struct output and custom assertion messages for each field. + +### Implementation: `server/tests/gauntlet_golden.rs` + +```rust +//! Golden file comparison for the Gauntlet. +//! Generates actual snapshot, loads expected golden file, compares field by field. +//! On mismatch: prints structured diff to stderr. +//! Run with: cargo test --test gauntlet_golden -- --nocapture + +use settled_reach_server::bridge::types::*; +use std::path::Path; + +const GOLDEN_DIR: &str = "../tests/golden"; + +fn load_golden(name: &str) -> ObserverSnapshot { + let path = Path::new(GOLDEN_DIR).join(format!("{}.json", name)); + let content = std::fs::read_to_string(&path) + .unwrap_or_else(|_| panic!("Golden file not found: {}", path.display())); + serde_json::from_str(&content) + .unwrap_or_else(|e| panic!("Failed to parse golden file {}: {}", path.display(), e)) +} + +fn run_gauntlet_to_tick(tick_count: u64) -> ObserverSnapshot { + // Build Gauntlet app, run N ticks, extract observer snapshot + let mut app = build_gauntlet_app(42); + for _ in 0..tick_count { + app.update(); + } + extract_observer_snapshot(&app) +} + +fn diff_snapshots(golden: &ObserverSnapshot, actual: &ObserverSnapshot) -> Vec { + let mut diffs = Vec::new(); + + if golden.tick != actual.tick { + diffs.push(format!(" tick: {} → {}", golden.tick, actual.tick)); + } + if golden.version != actual.version { + diffs.push(format!(" version: {} → {}", golden.version, actual.version)); + } + if golden.player_facing != actual.player_facing { + diffs.push(format!(" player_facing: {:?} → {:?}", + golden.player_facing, actual.player_facing)); + } + + // Entity comparison + if golden.entities.len() != actual.entities.len() { + diffs.push(format!(" entities.count: {} → {}", + golden.entities.len(), actual.entities.len())); + } + + let max_entities = golden.entities.len().max(actual.entities.len()); + for i in 0..max_entities { + match (golden.entities.get(i), actual.entities.get(i)) { + (Some(g), Some(a)) => { + if g.entity_id != a.entity_id { + diffs.push(format!(" entities[{}].entity_id: {} → {}", + i, g.entity_id, a.entity_id)); + } + if (g.x - a.x).abs() > 0.01 { + diffs.push(format!(" entities[{}].x: {} → {} ← POSITION", + i, g.x, a.x)); + } + if (g.y - a.y).abs() > 0.01 { + diffs.push(format!(" entities[{}].y: {} → {} ← POSITION", + i, g.y, a.y)); + } + if g.kind != a.kind { + diffs.push(format!(" entities[{}].kind: {:?} → {:?}", + i, g.kind, a.kind)); + } + if g.visibility != a.visibility { + diffs.push(format!(" entities[{}].visibility: {:?} → {:?} ← VISIBILITY", + i, g.visibility, a.visibility)); + } + if g.relationship != a.relationship { + diffs.push(format!(" entities[{}].relationship: {:?} → {:?}", + i, g.relationship, a.relationship)); + } + } + (Some(g), None) => { + diffs.push(format!(" REMOVED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}", + i, g.entity_id, g.kind, g.x, g.y)); + } + (None, Some(a)) => { + diffs.push(format!(" ADDED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}", + i, a.entity_id, a.kind, a.x, a.y)); + } + _ => {} + } + } + + // Visible tiles summary (don't diff each tile — too noisy) + if golden.visible_tiles.len() != actual.visible_tiles.len() { + diffs.push(format!(" visible_tiles.count: {} → {}", + golden.visible_tiles.len(), actual.visible_tiles.len())); + } + + // Inventory + if golden.player_inventory != actual.player_inventory { + diffs.push(format!(" player_inventory: {:?} → {:?}", + golden.player_inventory, actual.player_inventory)); + } + + // Monologue + if golden.current_monologue != actual.current_monologue { + diffs.push(format!(" current_monologue: {:?} → {:?}", + golden.current_monologue, actual.current_monologue)); + } + + diffs +} + +#[test] +fn gauntlet_tick_10_matches_golden() { + let golden = load_golden("gauntlet_tick_10"); + let actual = run_gauntlet_to_tick(10); + + let diffs = diff_snapshots(&golden, &actual); + if !diffs.is_empty() { + eprintln!("\n=== Gauntlet Golden File Diff ==="); + eprintln!("Golden: tests/golden/gauntlet_tick_10.json"); + eprintln!("Seed: 42, Ticks: 10\n"); + eprintln!("CHANGED FIELDS:"); + for d in &diffs { + eprintln!("{}", d); + } + // Count unchanged for context + let total_fields = count_comparable_fields(&golden); + eprintln!("\nUNCHANGED: {} of {} comparable fields", + total_fields - diffs.len(), total_fields); + eprintln!("\nTo update the golden file:"); + eprintln!(" make golden-update"); + eprintln!(); + panic!("Golden file mismatch: {} field(s) differ", diffs.len()); + } +} +``` + +### Golden File Update Workflow + +```makefile +# Golden file operations +golden-diff: + cd server && cargo test --test gauntlet_golden -- --nocapture 2>&1 || true + +golden-update: + cd server && cargo test --test gauntlet_golden_gen -- --ignored --nocapture + @echo "Golden files updated in tests/golden/. Review and commit." + @git diff --stat tests/golden/ +``` + +The `golden-update` target: +1. Runs the Gauntlet with the canonical seed +2. Serializes the ObserverSnapshot as sorted JSON +3. Writes to `tests/golden/gauntlet_tick_10.json` +4. Prints a git diff summary so the developer can review + +### Golden File JSON Format + +Sorted keys, pretty-printed, deterministic output: + +```json +{ + "current_monologue": null, + "entities": [ + { + "entity_id": 1, + "kind": "Player", + "observation": "Visible", + "relationship": "Unknown", + "visibility": "Forward", + "x": 16.5, + "y": 16.5, + "z": 0 + }, + { + "entity_id": 2, + "kind": "Npc", + "observation": "Visible", + "relationship": "Unknown", + "visibility": "Forward", + "x": 18.0, + "y": 10.0, + "z": 0 + } + ], + "game_time": { + "day": 0, + "day_phase": "Morning", + "tick_rate": "Full", + "time_of_day": 1 + }, + "nearby_interactions": [], + "pending_recognitions": [], + "player_facing": "North", + "player_inventory": [], + "player_stance": "Walk", + "tick": 10, + "version": 7, + "visible_tiles": [ + {"tile_kind": "Floor", "visibility": "Forward", "x": 15, "y": 15, "z": 0}, + {"tile_kind": "Floor", "visibility": "Forward", "x": 16, "y": 15, "z": 0} + ] +} +``` + +Why JSON, not MessagePack: +- Human-readable in `git diff` +- Sorted keys = deterministic output regardless of struct field order +- Standard format — no custom tooling needed for basic inspection +- `serde_json` with `#[serde(sort_maps)]` + `to_string_pretty()` handles this natively + +### Why Not a Separate Python/Bash Tool? + +A separate `tooling/diff-golden` script would need to: +1. Understand the ObserverSnapshot structure (duplicating the Rust type definitions) +2. Parse JSON with knowledge of which fields are positions (float tolerance), which are counts, etc. +3. Be kept in sync as the snapshot format evolves + +The Rust test already has the type definitions, already knows the structure, and can use `#[derive(PartialEq, Debug)]` for free. A Rust test with structured `eprintln!` output is the simplest correct solution. + +For developers who want a quick terminal diff without running the full test: +```bash +# Just see what changed (raw JSON diff) +diff tests/golden/gauntlet_tick_10.json .cache/gauntlet_actual.json +``` + +This is the fallback — not as pretty as the structured diff, but always available. + +--- + +## Answering Hoshe's Open Question (T5-H5) + +> **For Justine:** Fixture staleness check in CI (`make fixtures && git diff --exit-code`) — is this robust enough, or do we need a content-addressed hash approach? + +**`git diff --exit-code` is robust enough.** Reasons: + +1. It detects any byte-level change in the fixtures directory. Content-addressed hashing would catch the same thing with more complexity. +2. `git diff` already handles binary files (MessagePack fixtures are binary). It won't show a meaningful diff, but it will detect changes. +3. The pre-PR workflow is: regenerate fixtures, check for diff, fail if stale. This is deterministic — same source produces same fixtures. +4. Content-addressed hashing adds a build step (compute hashes, store/compare) for zero additional safety. + +**One caveat:** The fixture regeneration must be deterministic. If `gen_fixtures.rs` uses timestamps or random values, the diff will always show changes. Current code uses fixed values — verified. + +The implementation in `pre-pr-fixtures` above handles this correctly. + +--- + +## Summary of Deliverables + +| Deliverable | Status | +|-------------|--------| +| `validate-content` expansion design (7 cross-reference checks) | Complete | +| `make pre-pr` target specification (5-step chain, ~3 min) | Complete | +| `make perf-baseline` tooling design (median-of-5, relative delta) | Complete | +| Golden file diff design (Rust test, structured output) | Complete | +| Hoshe Q5 response (fixture staleness check) | Complete | + +## Dependencies on Other Tracks + +| Dependency | Owner | What I Need | +|-----------|-------|-------------| +| Gauntlet implementation | Dudley/Tyre | `build_gauntlet_app()` function for perf + golden file tests | +| `serde_json` derive on `ObserverSnapshot` | Dudley | `#[derive(Serialize, Deserialize)]` for JSON golden files (currently only `rmp_serde`) | +| Content validation phase 2 enums | Hoshe | Agreed list of valid location slugs, role slugs, and enum values for cross-reference validation | diff --git a/docs/workshops/test-architecture/justine-round3.md b/docs/workshops/test-architecture/justine-round3.md new file mode 100644 index 000000000..a2f2cdd2e --- /dev/null +++ b/docs/workshops/test-architecture/justine-round3.md @@ -0,0 +1,996 @@ +# Justine — Round 3: Final Tooling Specifications + +**Workshop:** QA Strategy & Test Architecture +**Track:** 5 (Content Scaling & CI Pipeline) +**Round:** 3 (Prioritization — build-ready specs) +**Date:** 2026-02-17 + +These specifications are implementation-ready. A developer should be able to build each target by reading this document alone. + +--- + +## 1. `make pre-pr` — Final Specification + +### Reconciliation + +Hoshe proposed 6 steps; I proposed 5 in Round 2. The difference: Hoshe lists `check-fact-ids` as a separate step; I proposed absorbing it into `validate-content`. Since absorption hasn't been implemented yet, **keep them separate for now**. When Phase 2 of the content validator ships, `check-fact-ids` becomes redundant and the step count drops to 5. + +Tyre argued fixture staleness should be BLOCKER, not WARNING. **Accepted.** Stale fixtures mean client tests run against outdated protocol data — every passing client test becomes a false positive. BLOCKER is correct. + +### Makefile Additions + +```makefile +# --- Pre-PR verification --- +# Run before pushing any PR. Chains all checks in dependency order. +# Fails fast on first error. Total: ~90-180s on incremental build. + +.PHONY: pre-pr pre-pr-server pre-pr-client pre-pr-content fixtures-check + +pre-pr: lint build test validate-content check-fact-ids fixtures-check + @echo "" + @echo "=== PRE-PR: ALL CHECKS PASSED ===" + @echo "Safe to create PR." + +# Branch-specific variants +pre-pr-server: lint-server build-server test-server fixtures-check + @echo "Server pre-PR checks PASSED." + +pre-pr-client: lint-client build-client test-client + @echo "Client pre-PR checks PASSED." + +pre-pr-content: validate-content check-fact-ids + @echo "Content pre-PR checks PASSED." + +# Fixture staleness check — BLOCKER (Tyre R2: stale fixtures = false positive client tests) +fixtures-check: fixtures + @if git diff --quiet client/tests/fixtures/; then \ + echo "Fixtures: up to date"; \ + else \ + echo ""; \ + echo "FIXTURES STALE — protocol changed but fixtures not committed."; \ + echo "The following fixture files differ from the committed version:"; \ + git diff --stat client/tests/fixtures/; \ + echo ""; \ + echo "To fix: stage and commit the updated fixtures:"; \ + echo " git add client/tests/fixtures/"; \ + echo " git commit -m \"chore(fixtures): regenerate for protocol changes\""; \ + exit 1; \ + fi +``` + +### Execution Chain + +``` +make pre-pr + │ + ├── 1. lint lint-server (clippy + fmt) + lint-client (GDScript check) + │ Duration: ~15s + │ Catches: clippy warnings, fmt violations, GDScript errors + │ Failure: exits immediately, no point building broken code + │ + ├── 2. build build-server (cargo build) + build-client (godot --headless --quit) + │ Duration: ~30-90s (incremental), ~5-7min (clean) + │ Catches: compilation errors both sides + │ + ├── 3. test test-server (cargo nextest) + test-client (gdUnit4 headless) + │ Duration: ~15-30s + │ Catches: unit + integration test failures + │ + ├── 4. validate-content YAML schema validation (existing Python script) + │ Duration: ~2-5s + │ Catches: malformed YAML, schema violations + │ Future: cross-reference validation (Phase 2) + │ + ├── 5. check-fact-ids Fact ID resolution against knowledge catalogs + │ Duration: ~2s + │ Catches: dangling fact_id references + │ Future: absorbed into validate-content Phase 2 + │ + └── 6. fixtures-check Regenerate fixtures + git diff --exit-code + Duration: ~10-15s + Catches: stale protocol fixtures (BLOCKER) + Requires: server build (already done in step 2) +``` + +### Failure Output Examples + +**Lint failure (step 1):** +``` +cd server && cargo clippy -- -D warnings +error: unused variable `x` + --> src/simulation/movement.rs:42:9 +make: *** [lint-server] Error 1 +``` + +**Fixture staleness (step 6):** +``` +FIXTURES STALE — protocol changed but fixtures not committed. +The following fixture files differ from the committed version: + client/tests/fixtures/msgpack/snapshot_one_npc.msgpack | Bin 45 -> 52 bytes + client/tests/fixtures/msgpack/snapshot_v2_full.msgpack | Bin 89 -> 96 bytes + +To fix: stage and commit the updated fixtures: + git add client/tests/fixtures/ + git commit -m "chore(fixtures): regenerate for protocol changes" +make: *** [fixtures-check] Error 1 +``` + +**Success:** +``` +--- lint: done --- +--- build: done --- +--- test: done --- +Validated 47 files, 3 skipped, 0 errors +check-fact-ids: OK — 42 references validated against 42 canonical facts +Fixtures: up to date + +=== PRE-PR: ALL CHECKS PASSED === +Safe to create PR. +``` + +### Duration Budget + +| Step | Incremental | Clean Cache | Notes | +|------|------------|-------------|-------| +| lint | ~15s | ~15s | No build dependency | +| build | ~30s | ~5-7min | Rust incremental build is fast | +| test | ~15-30s | ~15-30s | Tests compile quickly once build exists | +| validate-content | ~2-5s | ~2-5s | Python, no build dependency | +| check-fact-ids | ~2s | ~2s | Bash grep, no build dependency | +| fixtures-check | ~10-15s | ~10-15s | Runs gen_fixtures (server build already cached) | +| **Total** | **~90s** | **~8min** | Fast enough for every PR | + +### What pre-pr Does NOT Include + +- Layer 3 subprocess tests (slow, nightly-tier — `make test-layer3`) +- Performance benchmarks (machine-dependent — `make perf-baseline`) +- Gauntlet golden file tests (depends on Gauntlet implementation — `make golden-diff`) +- Content RON conversion (`make content-ron` — deferred per R2-OQ-03, not yet needed for validation) + +For deeper verification: +```makefile +# Optional: run everything including slow tests +pre-pr-deep: pre-pr test-layer3 perf-baseline golden-diff + @echo "=== DEEP VERIFICATION: ALL CHECKS PASSED ===" +``` + +--- + +## 2. `make perf-baseline` — Final Specification + +### Dependency + +**Blocked on Gauntlet implementation.** The perf benchmark needs `build_gauntlet_app(seed)` to construct the test world. Until the Gauntlet ships, the shadowcast benchmark (`shadowcast_bench.rs`) is the only available benchmark. The tooling is designed to accommodate both. + +### Baseline File: `tests/perf/baseline.json` + +Checked into the repo. Updated explicitly by the developer. + +```json +{ + "_meta": { + "format_version": 1, + "updated_at": "2026-02-17T14:30:00Z", + "updated_by": "developer-name", + "commit": "abc123f", + "machine": "workstation-01", + "rust_version": "1.82.0", + "build_profile": "release" + }, + "benchmarks": { + "gauntlet_100_ticks": { + "median_ms": 142.3, + "min_ms": 138.1, + "max_ms": 156.7, + "runs": 5, + "description": "Gauntlet map, seed 42, 100 simulation ticks" + }, + "shadowcast_150x150_30pct": { + "median_ms": 12.4, + "min_ms": 11.8, + "max_ms": 14.2, + "runs": 5, + "description": "Shadowcast, 150x150 map, 30% walls, 1000 iterations" + } + } +} +``` + +### Rust Benchmark Test: `server/tests/gauntlet_perf.rs` + +```rust +//! Performance benchmarks for the Gauntlet. +//! Run with: cargo test --release --test gauntlet_perf -- --ignored --nocapture +//! Outputs structured PERF_RESULT lines for tooling/perf-compare to parse. + +use std::time::Instant; + +#[test] +#[ignore] +fn gauntlet_100_ticks_5_runs() { + let mut times_ms: Vec = Vec::new(); + + for run in 0..5 { + let mut app = build_gauntlet_app(42); // seed 42, deterministic + + let start = Instant::now(); + for _ in 0..100 { + app.update(); + } + let elapsed = start.elapsed().as_secs_f64() * 1000.0; + times_ms.push(elapsed); + eprintln!("PERF_RUN: gauntlet_100_ticks run={} ms={:.2}", run, elapsed); + } + + times_ms.sort_by(|a, b| a.partial_cmp(b).unwrap()); + let median = times_ms[2]; // middle of 5 + let min = times_ms[0]; + let max = times_ms[4]; + + eprintln!("PERF_RESULT: gauntlet_100_ticks median={:.2} min={:.2} max={:.2} runs=5", + median, min, max); +} +``` + +Output format (machine-parseable): +``` +PERF_RUN: gauntlet_100_ticks run=0 ms=141.23 +PERF_RUN: gauntlet_100_ticks run=1 ms=138.12 +PERF_RUN: gauntlet_100_ticks run=2 ms=142.87 +PERF_RUN: gauntlet_100_ticks run=3 ms=156.71 +PERF_RUN: gauntlet_100_ticks run=4 ms=139.44 +PERF_RESULT: gauntlet_100_ticks median=141.23 min=138.12 max=156.71 runs=5 +``` + +### Comparison Script: `tooling/perf-compare` + +```bash +#!/usr/bin/env bash +# Compare performance results against committed baseline. +# Usage: tooling/perf-compare [baseline_file] +# results_file: raw output from cargo test (contains PERF_RESULT lines) +# baseline_file: defaults to tests/perf/baseline.json +# +# Exit codes: 0 = all pass, 1 = warning(s), 2 = failure(s) +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel)" +RESULTS="${1:?Usage: perf-compare [baseline_file]}" +BASELINE="${2:-$REPO_ROOT/tests/perf/baseline.json}" + +WARN_THRESHOLD=15 # % regression = warning +FAIL_THRESHOLD=30 # % regression = failure + +if [ ! -f "$BASELINE" ]; then + echo "No baseline at $BASELINE — run 'make perf-baseline-update' to create." + exit 0 +fi + +# Parse PERF_RESULT lines from test output +# Format: PERF_RESULT: median= min= max= runs= +EXIT_CODE=0 + +echo "" +echo "=== Performance Comparison ===" +echo "Baseline: $(python3 -c "import json; d=json.load(open('$BASELINE')); print(d['_meta']['commit'], '('+d['_meta']['updated_at'][:10]+')', 'on', d['_meta']['machine'])")" +echo "" +printf "%-30s %12s %12s %10s %8s\n" "Benchmark" "Baseline" "Current" "Delta" "Status" +printf "%s\n" "$(printf '%.0s─' {1..76})" + +while IFS= read -r line; do + NAME=$(echo "$line" | sed 's/.*PERF_RESULT: //' | awk '{print $1}') + CURRENT=$(echo "$line" | grep -oP 'median=\K[0-9.]+') + + # Look up baseline + BASELINE_VAL=$(python3 -c " +import json, sys +d = json.load(open('$BASELINE')) +b = d.get('benchmarks', {}).get('$NAME', {}) +print(b.get('median_ms', 'N/A')) +" 2>/dev/null) + + if [ "$BASELINE_VAL" = "N/A" ]; then + printf "%-30s %12s %10.1fms %10s %8s\n" "$NAME" "N/A" "$CURRENT" "—" "NEW" + continue + fi + + DELTA=$(python3 -c "print(f'{(($CURRENT - $BASELINE_VAL) / $BASELINE_VAL) * 100:.1f}')") + DELTA_ABS=$(python3 -c "print(abs(($CURRENT - $BASELINE_VAL) / $BASELINE_VAL) * 100)") + + if python3 -c "exit(0 if $CURRENT < $BASELINE_VAL else 1)" 2>/dev/null; then + STATUS="PASS" + elif python3 -c "exit(0 if $DELTA_ABS < $WARN_THRESHOLD else 1)" 2>/dev/null; then + STATUS="PASS" + elif python3 -c "exit(0 if $DELTA_ABS < $FAIL_THRESHOLD else 1)" 2>/dev/null; then + STATUS="WARN" + [ "$EXIT_CODE" -lt 1 ] && EXIT_CODE=1 + else + STATUS="FAIL" + EXIT_CODE=2 + fi + + printf "%-30s %10.1fms %10.1fms %+9.1f%% %8s\n" \ + "$NAME" "$BASELINE_VAL" "$CURRENT" "$DELTA" "$STATUS" + +done < <(grep "^PERF_RESULT:" "$RESULTS") + +echo "" +case $EXIT_CODE in + 0) echo "Overall: PASS" ;; + 1) echo "Overall: WARNING — investigate regressions above 15%" ;; + 2) echo "Overall: FAIL — regression(s) exceed 30% threshold" ;; +esac + +exit $EXIT_CODE +``` + +### Makefile Targets + +```makefile +# --- Performance benchmarks (local, release builds only) --- + +.PHONY: perf-baseline perf-baseline-update + +perf-baseline: + @echo "Building server (release)..." + @cd server && cargo build --release 2>&1 | tail -1 + @echo "Running benchmarks (5 runs each)..." + @cd server && cargo test --release --test gauntlet_perf -- --ignored --nocapture \ + 2>&1 | tee ../.cache/perf_raw.txt + @tooling/perf-compare .cache/perf_raw.txt + +perf-baseline-update: + @echo "Building server (release)..." + @cd server && cargo build --release 2>&1 | tail -1 + @echo "Running benchmarks (5 runs each)..." + @cd server && cargo test --release --test gauntlet_perf -- --ignored --nocapture \ + 2>&1 | tee ../.cache/perf_raw.txt + @tooling/perf-update .cache/perf_raw.txt tests/perf/baseline.json + @echo "" + @echo "Baseline updated. Review and commit tests/perf/baseline.json" + @git diff --stat tests/perf/baseline.json +``` + +### Baseline Update Script: `tooling/perf-update` + +```bash +#!/usr/bin/env bash +# Update the performance baseline file from benchmark results. +# Usage: tooling/perf-update +set -euo pipefail + +RESULTS="${1:?Usage: perf-update }" +BASELINE="${2:?Usage: perf-update }" + +python3 -c " +import json, sys, os, subprocess, datetime + +results_file = '$RESULTS' +baseline_file = '$BASELINE' + +# Parse PERF_RESULT lines +benchmarks = {} +with open(results_file) as f: + for line in f: + if 'PERF_RESULT:' not in line: + continue + parts = line.strip().split() + name = parts[1] + vals = {} + for p in parts[2:]: + k, v = p.split('=') + vals[k] = float(v) if '.' in v else int(v) + benchmarks[name] = { + 'median_ms': vals['median'], + 'min_ms': vals['min'], + 'max_ms': vals['max'], + 'runs': vals['runs'], + } + +# Get metadata +commit = subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).decode().strip() +rust_ver = subprocess.check_output(['rustc', '--version']).decode().strip().split()[1] +machine = os.uname().nodename + +baseline = { + '_meta': { + 'format_version': 1, + 'updated_at': datetime.datetime.now(datetime.timezone.utc).isoformat(), + 'updated_by': os.environ.get('USER', 'unknown'), + 'commit': commit, + 'machine': machine, + 'rust_version': rust_ver, + 'build_profile': 'release', + }, + 'benchmarks': benchmarks, +} + +with open(baseline_file, 'w') as f: + json.dump(baseline, f, indent=2) + f.write('\n') + +print(f'Wrote {len(benchmarks)} benchmark(s) to {baseline_file}') +" +``` + +### Human-Readable Output + +Normal run: +``` +=== Performance Comparison === +Baseline: abc123f (2026-02-15) on workstation-01 + +Benchmark Baseline Current Delta Status +──────────────────────────────────────────────────────────────────────────── +gauntlet_100_ticks 142.3ms 148.7ms +4.5% PASS +shadowcast_150x150_30pct 12.4ms 12.1ms -2.4% PASS + +Overall: PASS +``` + +Regression detected: +``` +gauntlet_100_ticks 142.3ms 203.1ms +42.7% FAIL + Exceeds 30% threshold. Likely regression. + +Overall: FAIL — regression(s) exceed 30% threshold +``` + +### Thresholds + +| Delta | Status | Action | +|-------|--------|--------| +| <15% or faster | PASS | Normal variance | +| 15-30% slower | WARN | Investigate. May be noise or real regression | +| >30% slower | FAIL | Almost certainly a regression. Profile before merging | + +### Machine Variance Protection + +- **Machine tag in baseline**: Baseline records which machine it was set on. If the machine differs, the comparison script prints a warning: `"WARNING: baseline was measured on workstation-01, current machine is laptop-02. Results may not be comparable."` +- **Release builds only**: Debug builds have 10-20x performance variance. The Makefile targets use `--release`. +- **Median of 5**: Absorbs outliers from background processes, thermal throttling. + +--- + +## 3. Golden File Workflow — Final Specification + +### Overview + +Golden files are canonical `ObserverSnapshot` outputs from the Gauntlet at fixed ticks with a fixed seed. They're checked into the repo as sorted JSON. When server behavior changes, the golden files change — and the diff shows exactly what changed. + +### Dependency + +**Blocked on Gauntlet implementation.** Requires `build_gauntlet_app(seed)` from Dudley/Tyre and `serde_json` derives on `ObserverSnapshot`. + +### File Layout + +``` +tests/ + golden/ + gauntlet_tick_0.json # ObserverSnapshot at tick 0, seed 42 + gauntlet_tick_10.json # ObserverSnapshot at tick 10, seed 42 + gauntlet_tick_100.json # ObserverSnapshot at tick 100, seed 42 +``` + +### JSON Format + +Sorted keys, pretty-printed, deterministic. Generated with `serde_json::to_string_pretty()` and sorted keys (`#[serde(sort_maps)]` or post-processing): + +```json +{ + "current_monologue": null, + "entities": [ + { + "entity_id": 1, + "kind": "Player", + "observation": "Visible", + "relationship": "Unknown", + "visibility": "Forward", + "x": 16.5, + "y": 16.5, + "z": 0 + } + ], + "game_time": { + "day": 0, + "day_phase": "Morning", + "tick_rate": "Full", + "time_of_day": 0 + }, + "nearby_interactions": [], + "pending_recognitions": [], + "player_facing": "North", + "player_inventory": [], + "player_stance": "Walk", + "tick": 0, + "version": 7, + "visible_tiles": [] +} +``` + +**Why JSON, not MessagePack:** +- Human-readable in `git diff` — PR reviewers see exactly what changed +- Sorted keys = deterministic output regardless of Rust struct field order +- `serde_json` is already a dev-dependency in the server crate +- Standard format — no custom tooling for basic inspection + +### Golden File Generator: `server/tests/gauntlet_golden_gen.rs` + +```rust +//! Regenerate Gauntlet golden files. +//! Run with: cargo test --test gauntlet_golden_gen -- --ignored --nocapture + +use settled_reach_server::bridge::types::ObserverSnapshot; +use std::fs; +use std::path::Path; + +const GOLDEN_DIR: &str = "../tests/golden"; +const SEED: u64 = 42; +const TICKS: &[u64] = &[0, 10, 100]; + +fn write_golden(name: &str, snapshot: &ObserverSnapshot) { + let dir = Path::new(GOLDEN_DIR); + fs::create_dir_all(dir).expect("create golden dir"); + let path = dir.join(format!("{}.json", name)); + let json = serde_json::to_string_pretty(snapshot).expect("serialize to JSON"); + fs::write(&path, &json).expect("write golden file"); + eprintln!("Wrote {} ({} bytes)", path.display(), json.len()); +} + +#[test] +#[ignore] +fn regenerate_golden_files() { + for &tick in TICKS { + let mut app = build_gauntlet_app(SEED); + for _ in 0..tick { + app.update(); + } + let snapshot = extract_observer_snapshot(&app); + write_golden(&format!("gauntlet_tick_{}", tick), &snapshot); + } +} +``` + +### Golden File Comparator: `server/tests/gauntlet_golden.rs` + +This is the test that runs during `make golden-diff`. It loads the checked-in golden file, runs the Gauntlet fresh, and compares field-by-field. + +```rust +//! Compare current Gauntlet output against committed golden files. +//! Run with: cargo test --test gauntlet_golden -- --nocapture +//! On mismatch: prints structured diff and fails. + +use settled_reach_server::bridge::types::*; +use std::path::Path; + +const GOLDEN_DIR: &str = "../tests/golden"; +const SEED: u64 = 42; + +fn load_golden(name: &str) -> ObserverSnapshot { + let path = Path::new(GOLDEN_DIR).join(format!("{}.json", name)); + let content = std::fs::read_to_string(&path) + .unwrap_or_else(|_| panic!("Golden file not found: {}. Run 'make golden-update'.", path.display())); + serde_json::from_str(&content) + .unwrap_or_else(|e| panic!("Failed to parse {}: {}", path.display(), e)) +} + +fn diff_snapshots(golden: &ObserverSnapshot, actual: &ObserverSnapshot) -> Vec { + let mut diffs = Vec::new(); + + // Scalar fields + if golden.version != actual.version { + diffs.push(format!(" version: {} -> {}", golden.version, actual.version)); + } + if golden.tick != actual.tick { + diffs.push(format!(" tick: {} -> {}", golden.tick, actual.tick)); + } + if golden.player_facing != actual.player_facing { + diffs.push(format!(" player_facing: {:?} -> {:?}", golden.player_facing, actual.player_facing)); + } + if golden.player_stance != actual.player_stance { + diffs.push(format!(" player_stance: {:?} -> {:?}", golden.player_stance, actual.player_stance)); + } + + // Game time + if golden.game_time != actual.game_time { + diffs.push(format!(" game_time: {:?} -> {:?}", golden.game_time, actual.game_time)); + } + + // Entity count + if golden.entities.len() != actual.entities.len() { + diffs.push(format!(" entities.count: {} -> {}", golden.entities.len(), actual.entities.len())); + } + + // Per-entity comparison (both sorted by entity_id per determinism fixes) + let max_len = golden.entities.len().max(actual.entities.len()); + for i in 0..max_len { + match (golden.entities.get(i), actual.entities.get(i)) { + (Some(g), Some(a)) => { + let label = format!("entities[{}](id:{})", i, g.entity_id); + if g.entity_id != a.entity_id { + diffs.push(format!(" {}.entity_id: {} -> {}", label, g.entity_id, a.entity_id)); + } + if (g.x - a.x).abs() > 0.01 || (g.y - a.y).abs() > 0.01 { + diffs.push(format!(" {}: ({},{}) -> ({},{}) <- POSITION", label, g.x, g.y, a.x, a.y)); + } + if g.z != a.z { + diffs.push(format!(" {}.z: {} -> {}", label, g.z, a.z)); + } + if g.kind != a.kind { + diffs.push(format!(" {}.kind: {:?} -> {:?}", label, g.kind, a.kind)); + } + if g.visibility != a.visibility { + diffs.push(format!(" {}.visibility: {:?} -> {:?} <- VISIBILITY", label, g.visibility, a.visibility)); + } + if g.relationship != a.relationship { + diffs.push(format!(" {}.relationship: {:?} -> {:?}", label, g.relationship, a.relationship)); + } + if g.observation != a.observation { + diffs.push(format!(" {}.observation: {:?} -> {:?}", label, g.observation, a.observation)); + } + } + (Some(g), None) => { + diffs.push(format!(" REMOVED: entities[{}] {{ id:{}, kind:{:?}, pos:({},{}) }}", i, g.entity_id, g.kind, g.x, g.y)); + } + (None, Some(a)) => { + diffs.push(format!(" ADDED: entities[{}] {{ id:{}, kind:{:?}, pos:({},{}) }}", i, a.entity_id, a.kind, a.x, a.y)); + } + (None, None) => {} + } + } + + // Visible tiles (summary only — per-tile diff is too noisy) + if golden.visible_tiles.len() != actual.visible_tiles.len() { + diffs.push(format!(" visible_tiles.count: {} -> {}", golden.visible_tiles.len(), actual.visible_tiles.len())); + } + + // Interactions + if golden.nearby_interactions.len() != actual.nearby_interactions.len() { + diffs.push(format!(" nearby_interactions.count: {} -> {}", golden.nearby_interactions.len(), actual.nearby_interactions.len())); + } + + // Inventory + if golden.player_inventory != actual.player_inventory { + diffs.push(format!(" player_inventory: {:?} -> {:?}", golden.player_inventory, actual.player_inventory)); + } + + // Monologue + if golden.current_monologue != actual.current_monologue { + diffs.push(format!(" current_monologue: {:?} -> {:?}", golden.current_monologue, actual.current_monologue)); + } + + diffs +} + +#[test] +fn gauntlet_tick_0_matches_golden() { + compare_golden("gauntlet_tick_0", 0); +} + +#[test] +fn gauntlet_tick_10_matches_golden() { + compare_golden("gauntlet_tick_10", 10); +} + +#[test] +fn gauntlet_tick_100_matches_golden() { + compare_golden("gauntlet_tick_100", 100); +} + +fn compare_golden(name: &str, ticks: u64) { + let golden = load_golden(name); + let mut app = build_gauntlet_app(SEED); + for _ in 0..ticks { + app.update(); + } + let actual = extract_observer_snapshot(&app); + + let diffs = diff_snapshots(&golden, &actual); + if !diffs.is_empty() { + eprintln!(); + eprintln!("=== Golden File Mismatch: {} ===", name); + eprintln!("Seed: {}, Ticks: {}", SEED, ticks); + eprintln!(); + eprintln!("CHANGED FIELDS ({}):", diffs.len()); + for d in &diffs { + eprintln!("{}", d); + } + eprintln!(); + eprintln!("To update golden files: make golden-update"); + eprintln!("Then review: git diff tests/golden/"); + panic!("{} field(s) differ from golden file", diffs.len()); + } +} +``` + +### Makefile Targets + +```makefile +# --- Golden file operations --- + +.PHONY: golden-diff golden-update + +# Compare current Gauntlet output against committed golden files. +# Prints structured diff on mismatch. +golden-diff: + cd server && cargo test --test gauntlet_golden -- --nocapture + +# Regenerate golden files from current server behavior. +# Review the diff before committing. +golden-update: + cd server && cargo test --test gauntlet_golden_gen -- --ignored --nocapture + @echo "" + @echo "Golden files regenerated. Review changes:" + @git diff --stat tests/golden/ + @echo "" + @echo "If changes are expected, commit:" + @echo " git add tests/golden/" + @echo " git commit -m \"chore(golden): update for \"" +``` + +### Developer Workflow + +1. Developer changes server logic +2. `make golden-diff` — shows structured diff if anything changed +3. Developer reviews the diff: "Yes, I moved NPC guard-1 by one tile, this is expected" +4. `make golden-update` — regenerates golden files +5. `git diff tests/golden/` — final review of JSON changes +6. Commit the updated golden files alongside the code change + +--- + +## 4. CI Pipeline Design — Deferred but Documented + +When the lead greenlights CI, this is the ready-to-implement specification. Based on my Round 1 proposal, Hoshe's 3-tier model, and Tyre's adjustments (15min PR budget, fixture staleness BLOCKER, content scaling in nightly). + +### Workflow File: `.gitea/workflows/ci.yaml` + +```yaml +name: CI +on: + push: + branches: ['*'] + pull_request: + branches: [main] + schedule: + - cron: '0 3 * * *' # Nightly at 03:00 UTC + +jobs: + # ────────────────────────────────────────────── + # TIER 1: Commit checks (every push, ~2 min) + # ────────────────────────────────────────────── + commit-checks: + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - name: Lint server + run: make lint-server + - name: Lint client + run: make lint-client + - name: Validate content (schema) + run: make validate-content + - name: Check fact IDs + run: make check-fact-ids + + # ────────────────────────────────────────────── + # TIER 2: PR checks (merge gate, <15 min) + # ────────────────────────────────────────────── + server-build-test: + if: github.event_name == 'pull_request' + needs: commit-checks + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - name: Cache cargo + uses: actions/cache@v4 + with: + path: | + ~/.cargo/registry + ~/.cargo/git + server/target + key: cargo-${{ hashFiles('server/Cargo.lock') }} + - name: Build server + run: make build-server + - name: Test server + run: make test-server + - name: Generate fixtures + run: make fixtures + - name: Check fixture staleness (BLOCKER) + run: | + if ! git diff --quiet client/tests/fixtures/; then + echo "::error::Fixtures are stale. Run 'make fixtures' and commit." + git diff --stat client/tests/fixtures/ + exit 1 + fi + - name: Upload fixtures + uses: actions/upload-artifact@v4 + with: + name: msgpack-fixtures + path: client/tests/fixtures/msgpack/ + + client-build-test: + if: github.event_name == 'pull_request' + needs: [commit-checks, server-build-test] + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - name: Download fixtures + uses: actions/download-artifact@v4 + with: + name: msgpack-fixtures + path: client/tests/fixtures/msgpack/ + - name: Build client + run: make build-client + - name: Test client + run: make test-client + + # ────────────────────────────────────────────── + # TIER 3: Nightly (deep validation, <30 min) + # ────────────────────────────────────────────── + nightly: + if: github.event_name == 'schedule' + runs-on: self-hosted + steps: + - uses: actions/checkout@v4 + - name: Full build + run: make build + - name: All tests + run: make test + - name: Layer 3 subprocess test + run: cd server && cargo test --test layer3 -- --ignored --nocapture + - name: Golden file check + run: | + make golden-diff || { + echo "::warning::Golden file mismatch detected" + cd server && cargo test --test gauntlet_golden -- --nocapture 2>&1 || true + } + - name: Performance benchmark + run: | + make perf-baseline 2>&1 | tee .cache/perf_output.txt + # Annotate but don't fail + grep "FAIL\|WARN" .cache/perf_output.txt && \ + echo "::warning::Performance regression detected" || true + - name: Content scaling stress test + run: cd server && cargo test --test content_scaling -- --ignored --nocapture +``` + +### Runner Requirements + +| Dependency | How to Provide | +|-----------|---------------| +| Rust stable + clippy + rustfmt | Pre-installed on self-hosted runner via `rustup` | +| cargo-nextest | Pre-installed: `cargo install cargo-nextest --locked` | +| Godot 4.6 headless | Pre-installed to `/usr/local/bin/godot4` on runner | +| Python 3 + jsonschema + pyyaml | Pre-installed: `pip install jsonschema pyyaml` | + +**Self-hosted runner is required** for: +- Pre-installed Godot (no download step) +- Stable performance baselines (no co-tenancy variance) +- Access to internal network (Gitea at `git.schweitz.internal`) + +### Merge-Blocking Policy (Gitea Branch Protection) + +| Job | Required for Merge? | Rationale | +|-----|-------------------|-----------| +| `commit-checks` | **Yes** | Lint + content validation are fast gates | +| `server-build-test` | **Yes** | Server tests + fixture staleness are correctness gates | +| `client-build-test` | **Yes** | Client tests verify the rendering contract | +| `nightly` | **No** | Deep tests are informational, not blocking | + +Gitea branch protection settings for `main`: +- Required status checks: `commit-checks`, `server-build-test`, `client-build-test` +- Require 1 approval +- Dismiss stale approvals on new pushes + +### Estimated Wiring Effort + +~1 day. The Makefile targets already exist. The workflow file is the only new artifact. Self-hosted runner setup is separate infrastructure work (~0.5 day). + +--- + +## 5. Fixture Staleness Check — Final Specification + +### Status: BLOCKER + +Confirmed BLOCKER, not WARNING. Tyre's Round 2 argument is definitive: stale fixtures mean client tests run against outdated protocol data. Every passing client test is a false positive. This is exactly the class of bug (wire format mismatch, like Bug #4) that the entire serialization testing track exists to prevent. + +### How It Works + +``` +make fixtures-check + │ + ├── 1. Run make fixtures + │ → cd server && cargo test --test gen_fixtures -- --ignored + │ → Writes .msgpack files to client/tests/fixtures/msgpack/ + │ + └── 2. Check for uncommitted changes + → git diff --quiet client/tests/fixtures/ + → Exit 0: fixtures match committed versions (PASS) + → Exit 1: fixtures differ from committed versions (FAIL) +``` + +### Where It Runs + +| Context | How | Blocking? | +|---------|-----|-----------| +| `make pre-pr` | Step 6 of 6 | Yes — pre-pr fails | +| `make pre-pr-server` | Final step | Yes — server changes affect fixtures | +| CI PR tier (future) | `server-build-test` job | Yes — merge blocked | +| CI nightly (future) | Not separately — covered by PR tier | N/A | +| `make pre-pr-client` | **Not included** | No — client doesn't generate fixtures | +| `make pre-pr-content` | **Not included** | No — content changes don't affect fixtures | + +### Makefile Target (repeated from Section 1 for standalone reference) + +```makefile +fixtures-check: fixtures + @if git diff --quiet client/tests/fixtures/; then \ + echo "Fixtures: up to date"; \ + else \ + echo ""; \ + echo "FIXTURES STALE — protocol changed but fixtures not committed."; \ + echo "The following fixture files differ from the committed version:"; \ + git diff --stat client/tests/fixtures/; \ + echo ""; \ + echo "To fix: stage and commit the updated fixtures:"; \ + echo " git add client/tests/fixtures/"; \ + echo " git commit -m \"chore(fixtures): regenerate for protocol changes\""; \ + exit 1; \ + fi +``` + +### Edge Cases + +1. **New fixture files (untracked):** `git diff --quiet` does NOT detect untracked files. If `gen_fixtures.rs` adds a new fixture, it won't be flagged by `git diff`. Mitigation: use `git diff --quiet client/tests/fixtures/ && git ls-files --others --exclude-standard client/tests/fixtures/ | grep -q . && exit 1 || true`. Or simpler: check for any untracked `.msgpack` files. + + Updated target: + ```makefile + fixtures-check: fixtures + @STALE=0; \ + if ! git diff --quiet client/tests/fixtures/; then \ + STALE=1; \ + echo "Modified fixtures:"; \ + git diff --stat client/tests/fixtures/; \ + fi; \ + UNTRACKED=$$(git ls-files --others --exclude-standard client/tests/fixtures/); \ + if [ -n "$$UNTRACKED" ]; then \ + STALE=1; \ + echo "New (untracked) fixtures:"; \ + echo "$$UNTRACKED"; \ + fi; \ + if [ "$$STALE" -eq 1 ]; then \ + echo ""; \ + echo "FIXTURES STALE — run 'make fixtures' and commit the results."; \ + exit 1; \ + fi; \ + echo "Fixtures: up to date" + ``` + +2. **Fixture path fragility:** `gen_fixtures.rs` uses `../client/tests/fixtures/msgpack` (relative to `server/`). This works in the monorepo checkout and CI checkout. If the path ever breaks, `make fixtures` itself will fail — which is caught before the diff check. + +3. **Determinism:** `gen_fixtures.rs` uses fixed values (no timestamps, no random data). Verified in Round 2 — same source always produces same output. + +### Answering Hoshe's R2-OQ-04 + +> Fixture staleness in `make pre-pr` — separate `make pre-pr-full` to keep basic pre-PR fast? + +**No.** The fixture check adds ~10-15 seconds and requires only a server build (which `make pre-pr` already does in step 2). The incremental cost is negligible. Separating it into `pre-pr-full` means developers skip it — defeating the purpose. Keep it in the standard `make pre-pr` chain. + +--- + +## Summary + +| Spec | Status | Blocked On | Implementable Now? | +|------|--------|-----------|-------------------| +| `make pre-pr` | Final | Nothing | **Yes** | +| `make perf-baseline` | Final | Gauntlet (`build_gauntlet_app`) | Tooling: yes. Benchmark test: after Gauntlet | +| Golden file workflow | Final | Gauntlet + `serde_json` on `ObserverSnapshot` | Tooling: yes. Tests: after Gauntlet | +| CI pipeline | Final, deferred | Lead greenlight + self-hosted runner | Documented, ~1 day to wire | +| Fixture staleness | Final | Nothing | **Yes** | + +**Immediate implementation order:** +1. `fixtures-check` target (prerequisite for pre-pr) +2. `make pre-pr` target (immediate developer value) +3. `tooling/perf-compare` + `tooling/perf-update` scripts (ready for when benchmarks exist) +4. `make golden-diff` / `make golden-update` targets (ready for when Gauntlet ships) +5. CI workflow file (ready for when lead greenlights) diff --git a/docs/workshops/test-architecture/ozzie-round2.md b/docs/workshops/test-architecture/ozzie-round2.md new file mode 100644 index 000000000..94267cf16 --- /dev/null +++ b/docs/workshops/test-architecture/ozzie-round2.md @@ -0,0 +1,562 @@ +# Ozzie — Round 2: Test Client UX + Anti-Tedium Specs + +**Workshop:** QA Strategy & Test Architecture +**Track:** 1 (Test World Design) — Human Tester Experience +**Date:** 2026-02-17 + +--- + +## 1. Test Client Text Output Format + +The test client is a Rust binary that connects to the server via TCP, receives ObserverSnapshots, and renders them as text in the terminal. This is the human tester's "eyes" when they can't (or don't want to) run the full Godot client. + +### Display Mode: Live-Updating Terminal (not file dumps) + +**Live-updating.** Not per-tick dumps to a file. + +Here's why: a tester watching text scroll by can react in real time. They see the NPC move, they see the fog update, they see the monologue fire. Dumping to a file means the tester has to stop the server, open the file, find the relevant tick, and compare. That's a context switch. Context switches kill testing momentum. + +**Architecture:** The test client uses a terminal UI library (crossterm or similar) to maintain a fixed-layout display that refreshes every tick. Not a TUI framework — just cursor positioning and ANSI escape codes. The display is split into sections that update in-place. + +**But ALSO log to file.** Every tick's text output is appended to `gauntlet-session-{timestamp}.log`. The live display is for the tester's eyes. The log is for the WRONG button's bug report (see Section 2.5). Both exist simultaneously. + +### Terminal Layout + +``` +╔══════════════════════════════════════════════════════════════════════╗ +║ GAUNTLET TEST CLIENT v0.1 Tick: 42 TickRate: Full ║ +║ Room: Occlusion Corridor Seed: 42 Session: 00:01:23 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ PLAYER (15,10) → East | Walk | HP: -- | Inventory: 2/9 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ ENTITIES ║ +║ ● npc:guard-1 (18,10) Fwd Neutral (#4a9ebb) VISIBLE d=3 ║ +║ ◐ npc:worker-2 (20,10) Fwd Unknown (#4a9ebb) REMEMBERED d=5 ║ +║ ✕ npc:hidden-1 (19,12) --- Unknown (#4a9ebb) BLOCKED ║ +║ └─ wall at (17,10) blocks LOS ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ FOG Clear:31 | Peripheral:12 | Deep:58 | Map:89 | Dark:312 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ SOUND ║ +║ 🔊 footsteps ~(22,10) — 2 entities, unhurried [MEDIUM RANGE] ║ +║ └─ visual indicator rendered at fog edge (no audio asset) ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ COGNITION ║ +║ ⏳ Recognizing entity at ~(22,10): 0.3s / 0.6s elapsed ║ +║ Pending monologue: "Those footsteps... that's—" ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ INTERACTIONS ║ +║ guard-1 [Talk(1) Observe(2)] d=3 in-LOS ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ MONOLOGUE: "Quiet shift. Too quiet." ║ +║ DIALOGUE: none ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ INVENTORY: [0:keycard] [3:manifest] [_] [_] [_] [_] [_] [_] [_] ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ CHECKLIST: Occlusion Corridor — 4/7 ██████░░░░ 57% ║ +║ TIMER: 00:47 (PB: 00:38) [F12:WRONG] [Home:Hub] [R:Reset]║ +╚══════════════════════════════════════════════════════════════════════╝ +``` + +### Section-by-Section Spec + +**Header Bar:** +- `Tick: N` — current simulation tick +- `TickRate: Full/Half/Paused` — simulation speed +- `Room: {name}` — detected from player position against Gauntlet room bounds +- `Seed: N` — the SimRng seed (for bug reproduction) +- `Session: HH:MM:SS` — wall-clock time since test client connected + +**Player Line:** +- Position `(x,y)`, facing arrow (`→ East`), stance (`Walk/Sprint/Careful/Crouch`), inventory count + +**Entities Section:** +Each entity on its own line. Symbols indicate visibility state: +- `●` VISIBLE (in clear vision cone) +- `◐` REMEMBERED (in fog, previously seen) +- `◌` FOGGED (detected in fog, not yet recognized — grey blob state) +- `✕` BLOCKED (exists but LOS blocked — debug info only, requires Dudley's `blocked_entities` field) +- `⚡` RECOGNIZING (mid-cognitive-delay transition) + +Per entity: name, position, sector (Fwd/Periph/Behind), relationship state with color hex AND semantic label, visibility state, distance. + +For BLOCKED entities: indented line showing WHICH wall blocks LOS. This is the "show your work" data that lets a tester say "the wall at (17,10) should NOT be blocking LOS here" vs "the NPC is correctly hidden." + +**Fog Section:** +Five numbers matching D-059's five fog layers. At a glance, the tester sees the fog distribution. If Clear drops to 0, something is very wrong. If Dark is 0, fog isn't working. + +**Sound Section:** +EVERY sound event that fired this tick. This is the critical missing piece from Stig's Round 1 spec. D-018 defines three information quality ranges — Close (stereo, identity possible), Medium (directional, imprecise), Long (insert alerts, delayed). Each sound event shows: +- Approximate source position +- Description (footsteps, conversation murmur, alarm) +- Range classification [CLOSE/MEDIUM/LONG] +- Whether an audio asset played or only a visual indicator rendered + +**Cognition Section:** +Active cognitive delays (D-060). Shows: +- Which entity is being recognized +- Time elapsed vs total delay +- Any pending monologue text that will fire during the delay +- Recognition chime state (D-067): "chime played at onset" / "chime complete" + +**Interactions Section:** +Available verbs per entity within range. Includes priority numbers, distance, and whether the entity is currently in LOS. + +**Monologue/Dialogue Section:** +- Monologue: the EXACT text currently displayed. Not "active" — the words. +- Dialogue: NPC name, current line, number of response options, walk-away state. + +**Inventory Section:** +Visual slot map. Filled slots show item name. Empty slots show `[_]`. Matches the 9-slot grid (D-065). + +**Status Bar:** +- Checklist progress for current room (auto-tracked, see Section 2.6) +- Room timer +- Hotkey reminders: F12 for WRONG, Home for Hub teleport, R for Reset + +### Data Requirements from ObserverSnapshot + +The test client needs these fields that Dudley identified as missing (Round 1, Track 3 Q1): + +1. **`blocked_entities: Vec`** — entities not visible with the blocking wall position. Essential for the `✕ BLOCKED` line. Populated only in debug/test mode to avoid production overhead. +2. **Entity display names** — `VisibleEntity` needs a `display_name: Option` or the test client resolves StableIds against a loaded name table from the Gauntlet content pack. +3. **Fog layer counts per type** — the snapshot has `visible_tiles` but not the 5-layer breakdown. Either the server computes this (preferred — it has the fog state) or the test client derives it from tile visibility data. +4. **Sound events** — a `Vec` with source position, type, range classification, and whether it triggered audio or visual-only feedback. + +Items 1, 3, and 4 should be gated behind a `--debug` flag or test-mode marker on the snapshot request to avoid production overhead. + +--- + +## 2. Anti-Tedium Feature Specs + +All approved by lead. Here are the full specs. + +### 2.1 Room Reset + +**Trigger:** Player steps on a reset plate at the room entrance. Reset plates are 2x2 tile areas (matching D-066 visual tile size) with a distinct floor pattern (striped hazard marking — fits the station maintenance aesthetic). + +**UX Flow:** +1. Player walks onto reset plate +2. Terminal displays: `⟳ RESET: Occlusion Corridor? [Enter to confirm / any move to cancel]` +3. Player presses Enter (or sends a dedicated ResetRoom action) +4. Server resets the room: + - All entities in the room return to tick-0 positions + - Entity knowledge graph entries related to this room are reverted + - Fog for room tiles reverts to initial state (unexplored for fresh rooms, or the room's default) + - Player inventory changes from this room are reverted (items taken are returned, items dropped are removed) + - Player position stays on the reset plate (they don't teleport) +5. Terminal displays: `✓ Occlusion Corridor reset to tick-0 state. Room timer reset.` +6. Room timer restarts from 0 + +**Server-side implementation hint:** Each room has a `RoomState` snapshot taken at Gauntlet load time. Reset = restore that snapshot for all entities within the room's bounding box. Player entity is excluded from the state restore (they stay where they are with their current stance/facing). + +**What does NOT reset:** +- Other rooms (isolation is critical — resetting the Occlusion Corridor doesn't touch the Dialogue Room) +- Player position (stays on reset plate) +- Session timer (wall-clock time keeps running) +- Checklist progress for OTHER rooms + +**Edge case:** What if the player is carrying an item from this room when they reset? The item is removed from inventory and returned to its tick-0 position. Terminal shows: `Items returned: keycard → crate_1` + +### 2.2 Hub Teleport + +**Hotkey:** `Home` key. Universal, always available, no modifier needed. + +**UX Flow:** +1. Player presses Home from any room +2. Terminal displays: `⟳ Teleporting to Central Hub...` +3. Server teleports player entity to `GAUNTLET.hub_spawn` position +4. Next tick's snapshot reflects new position +5. Terminal displays: `✓ Central Hub. Choose a room.` + +**No confirmation required.** Hub teleport is non-destructive — it doesn't reset any room state. The player's inventory, knowledge graph, and all room states are preserved. They're just... elsewhere now. + +**What happens to active states:** +- Active dialogue: walk-away triggers (D-064 300ms fade equivalent, dialogue fades) +- Active cognitive delay: cancelled (entity recognition interrupted by teleport) +- Active monologue: cleared (new room, new context) +- Interaction buffer: cleared (D-055 pattern — stance change clears, teleport should too) + +**In text mode:** The display refreshes instantly to show the Hub's snapshot. Room timer pauses (the tester left the room). + +### 2.3 WRONG Button + +**Hotkey:** `F12`. THE most important anti-tedium feature. This is how testing becomes sustainable. + +**Full Capture Flow:** + +1. Tester sees something wrong. Presses F12. +2. Test client immediately captures: + - **Current ObserverSnapshot** (the exact server state at this tick) + - **Last 60 ticks of snapshot history** (the test client buffers these in a ring buffer) + - **Last 60 ticks of player inputs** (what the tester was doing) + - **Current text renderer output** (what the tester was seeing) + - **Room metadata** (which room, expected behaviors, checklist state) + - **Gauntlet seed** (for reproduction) + - **Session timestamp** (wall-clock time) +3. Terminal pauses the live display and shows: + +``` +╔══════════════════════════════════════════════════════════════════╗ +║ 🚨 BUG REPORT — Tick 42 — Occlusion Corridor ║ +╠══════════════════════════════════════════════════════════════════╣ +║ What's wrong? (one line, then Enter): ║ +║ > _ ║ +╚══════════════════════════════════════════════════════════════════╝ +``` + +4. Tester types one sentence: `NPC behind wall was visible` +5. Presses Enter. +6. Test client writes bug report to disk. Terminal shows: + +``` +✓ Bug report saved: tests/bug-reports/gauntlet-2026-02-17T08-42-13/ + ├── report.md (human-readable summary) + ├── snapshot_current.json + ├── snapshot_history.jsonl (last 60 ticks) + ├── input_history.jsonl (last 60 inputs) + ├── text_output.txt (what the terminal showed) + └── room_metadata.json (room name, checklist, expected state) +``` + +7. Live display resumes. Tester continues testing. + +**Bug Report Format (`report.md`):** + +```markdown +# Bug Report — Gauntlet +- **Date:** 2026-02-17 08:42:13 +- **Room:** Occlusion Corridor +- **Tick:** 42 +- **Seed:** 42 +- **Tester description:** NPC behind wall was visible + +## State at time of report +Player (15,10) facing East | Walk | Inventory: 2/9 + +### Entities +| Name | Position | Sector | Relationship | Visibility | Distance | +|------|----------|--------|-------------|------------|----------| +| guard-1 | (18,10) | Forward | Neutral | VISIBLE | 3 | +| worker-2 | (20,10) | Forward | Unknown | REMEMBERED | 5 | +| hidden-1 | (19,12) | --- | Unknown | **VISIBLE** | 7 | + +### Expected (from room checklist) +- hidden-1 should be BLOCKED by wall at (17,10) + +### Fog State +Clear: 31 | Peripheral: 12 | Deep: 58 | Map: 89 | Dark: 312 + +## Reproduction +1. Start Gauntlet with seed 42 +2. Navigate to Occlusion Corridor +3. Stand at (15,10) facing East +4. Observe: hidden-1 at (19,12) is visible (should be blocked) +``` + +**File location:** `tests/bug-reports/gauntlet-{ISO-timestamp}/`. Each report gets its own directory. The reports accumulate across sessions — developers browse them, fix bugs, delete resolved reports. + +**Ring buffer size:** 60 ticks = ~6 seconds at 10 tps. Enough to capture "what just happened" without burning memory. Configurable via CLI flag (`--history-buffer 120` for longer captures). + +### 2.4 Room Timer + +**Visual in text mode:** Bottom status bar shows: + +``` +TIMER: 00:47 (PB: 00:38) +``` + +- `00:47` — time spent in this room this run (wall-clock, not tick time) +- `PB: 00:38` — personal best for this room across all sessions + +**Timer starts** when the player enters a room (crosses room bounding box). **Timer pauses** when the player leaves the room (hub teleport or walking to another room). **Timer resets** when the room resets (Section 2.1). + +**Personal bests** are stored in a local file: `tests/gauntlet-stats.json`. Simple key-value: room name → best time in seconds. Persists across sessions. Not committed to git (it's local QA data). + +```json +{ + "occlusion_corridor": { "best_seconds": 38, "runs": 12 }, + "fog_theater": { "best_seconds": 52, "runs": 8 }, + "inventory_warehouse": { "best_seconds": 25, "runs": 15 } +} +``` + +**Why this matters for anti-tedium:** "Can I beat my time?" turns testing into a personal challenge. The tester isn't just checking boxes — they're getting FASTER at checking boxes. That's intrinsic motivation. Free. No server changes needed. + +### 2.5 Auto-Checklist Progress + +**How it works:** Each room has a `checklist.yaml` co-located with its definition (matches Stig's proposal — see Section 3). The test client loads these at startup and tracks which items have been verified. + +**Tracking mechanism:** The test client watches the ObserverSnapshot for conditions that match checklist items. When a condition is met, the checklist item is marked as "observed" (not "passed" — the tester decides if the behavior is correct). + +Example checklist item: +```yaml +- id: occ_hidden_npc_not_visible + description: "NPC behind wall is NOT visible in Visual mode" + condition: + entity: hidden-1 + expected_visibility: blocked + player_position_near: [15, 10] + player_facing: East +``` + +When the player is near (15,10), facing East, and `hidden-1` is NOT in the visible entities list, this item transitions from `[ ]` to `[?]` (observed — was it correct?). The tester explicitly confirms with a keypress or it auto-confirms if the condition matches expectation. + +**Terminal display:** + +``` +CHECKLIST: Occlusion Corridor — 4/7 ██████░░░░ 57% + [✓] NPC behind wall: NOT visible + [✓] NPC in front: IS visible + [✓] Peripheral NPC: dimmed + [?] Sensor mode: hidden NPC detected ← auto-observed, awaiting confirm + [ ] Sound ping: directional indicator + [ ] Cognitive delay: ~0.6s timing + [ ] Recognition: monologue fires DURING delay +``` + +**Auto-confirm vs manual-confirm:** For objective conditions (entity visible/not-visible, inventory count, fog state), auto-confirm when the snapshot matches. For subjective conditions (timing "feels right", monologue text is appropriate), require manual confirmation (tester presses `Y` on the highlighted item). + +**Progress persists within a session** but resets when the room resets (Section 2.1). Cross-session progress is stored in `tests/gauntlet-stats.json` alongside timer data. + +--- + +## 3. Cross-Review: Stig's Checklist Proposal + +**Stig proposes:** Checklists co-located with room YAML definitions, `make checklist` generates `docs/workshops/test-architecture/gauntlet-checklist.md`. + +**My Layer 3 (Round 1) proposed:** Generated checklist from room metadata, lives at `docs/qa/gauntlet-checklist.md`, printable, checkboxes. + +**Verdict: Stig's approach is BETTER than mine. Here's why, and what's missing.** + +### What Stig Gets Right + +1. **Co-location with room YAML.** Checklist items live next to the things they test. When you add a new entity to the Occlusion Corridor YAML, the checklist is right there — you can't forget to add a test for it. My Round 1 proposed a separate checklist file, which would drift from the room definitions over time. + +2. **Auto-generation via `make checklist`.** Single source of truth. The markdown output is the artifact, not the source. Change the YAML, regenerate, done. + +3. **Example format is concrete.** Stig's examples ("Stand at (15,8) face East → guard-1 visible/Forward/full alpha") are actionable. A tester can follow them step-by-step. + +### What's Missing from Stig's Proposal + +1. **No machine-readable condition format.** Stig's checklists are human-readable prose. For auto-tracking in the test client (Section 2.5), we need structured conditions — entity name, expected visibility state, player position, etc. The YAML needs both: a `description` field (human prose) and a `condition` field (structured data). + +2. **No subjective vs objective distinction.** Some checklist items are objectively verifiable ("NPC not visible" — check the snapshot). Others are subjective ("cognitive delay feels like 0.6s" — tester judgment). The checklist YAML should tag items as `auto` (snapshot-verifiable) or `manual` (requires human judgment). This drives the auto-confirm vs manual-confirm behavior in the test client. + +3. **No cross-room checklist items.** Stig's examples are per-room. But Gestalt's Round 1 identified critical cross-room scenarios (sprint from Crowd Plaza into Occlusion Corridor). Where do those checklist items live? Proposal: a `cross_room_checks.yaml` at the Gauntlet root level, separate from per-room checklists. + +4. **No failure guidance.** Each checklist item should include a `if_wrong` field: "If this fails, the likely cause is X. Check Y." This is the "articulate failures" requirement from the brief. Example: + +```yaml +- id: occ_hidden_npc_not_visible + description: "NPC behind wall is NOT visible in Visual mode" + condition: + entity: hidden-1 + expected_visibility: blocked + type: auto + if_wrong: > + LOS is leaking through the wall at (17,10). + Check: symmetric shadowcasting treating wall tile as transparent? + Check: entity position exactly at (19,12)? Off-by-one puts them in LOS. + File: server/src/perception/shadowcast.rs +``` + +### Proposed Merged Format + +```yaml +# content/gauntlet/rooms/occlusion_corridor/checklist.yaml +room: occlusion_corridor +description: "Tests LOS, shadowcasting, vision cone, and perception modes" + +checks: + - id: occ_01_hidden_not_visible + description: "NPC behind wall is NOT visible in Visual mode" + type: auto + condition: + player_near: [15, 10] + player_facing: East + entity: hidden-1 + expected: blocked + if_wrong: | + LOS leaking through wall at (17,10). + Check symmetric shadowcasting in server/src/perception/shadowcast.rs. + + - id: occ_02_guard_visible + description: "NPC in front of wall IS visible with full alpha" + type: auto + condition: + player_near: [15, 10] + entity: guard-1 + expected: visible + expected_sector: Forward + if_wrong: | + Guard should be in clear LOS at distance 3. + Check entity spawn position in room YAML. + + - id: occ_03_cognitive_delay_timing + description: "Cognitive delay for fog recognition takes ~0.6s" + type: manual + guidance: | + Walk to the fog boundary. Wait for a sound event. + Count: does recognition take roughly 0.6 seconds? + The pending_recognitions field shows elapsed/total. + if_wrong: | + Cognitive delay timing off. Check D-060 values in + server/src/perception/recognition.rs. +``` + +This format serves BOTH Stig's generated markdown checklist AND my test client auto-tracking. `make checklist` renders the human-readable version. The test client loads the structured conditions. + +--- + +## 4. Human Tester Workflow — Full Step-by-Step + +### Pre-Session Setup + +``` +# Terminal 1: Start the Gauntlet server +make test-world-headless SEED=42 + +# Terminal 2: Connect test client +make test-client-connect +# or: cargo run --bin gauntlet-client -- --host 127.0.0.1 --port 9876 --history-buffer 60 +``` + +The test client connects, receives the first ObserverSnapshot, and displays the live terminal UI. The tester is in the Central Hub. + +### Typical Testing Flow + +**Step 1: Choose a room.** +Tester is in the Central Hub. The terminal shows room exits with labels: + +``` +EXITS: [N] Fog Theater | [E] Occlusion Corridor | [S] Dialogue Room | [W] Inventory +``` + +Tester sends movement input (arrow keys or WASD mapped to PlayerAction) toward the desired room. Or presses a room-select hotkey if quick-nav is implemented. + +**Step 2: Enter the room.** +Player walks through the corridor into the Occlusion Corridor. Terminal updates: +- Room name changes to "Occlusion Corridor" +- Timer starts +- Checklist section loads this room's checklist items +- Progress bar shows 0/7 + +**Step 3: Execute the checklist.** +Tester moves to the specified positions and observes. The test client auto-tracks objective items: + +``` +CHECKLIST: Occlusion Corridor — 0/7 + [ ] NPC behind wall: NOT visible ← move to (15,10), face East +``` + +Tester moves to (15,10), faces East. Test client checks snapshot: +- `hidden-1` is NOT in visible entities. Condition met. + +``` +CHECKLIST: Occlusion Corridor — 1/7 + [✓] NPC behind wall: NOT visible + [ ] NPC in front: IS visible ← auto-checked: guard-1 visible ✓ +``` + +Wait — that auto-checked too, because guard-1 IS visible from this position. Two items checked simultaneously. + +``` +CHECKLIST: Occlusion Corridor — 2/7 + [✓] NPC behind wall: NOT visible + [✓] NPC in front: IS visible + [?] Peripheral NPC: dimmed ← auto-observed, confirm? [Y/N] +``` + +Tester looks at the entity list: peripheral NPC shows `◐ PERIPHERAL`. They press Y to confirm. + +**Step 4: Encounter a bug.** +Tester switches perception mode (sends a TogglePerceptionMode action). The hidden NPC should now be detected in Sensor mode. But it's not. The entity list still shows `✕ BLOCKED`. + +Tester presses F12. + +``` +╔═══════════════════════════════════════════════╗ +║ 🚨 BUG REPORT — Tick 87 — Occlusion Corridor ║ +║ What's wrong? ║ +║ > Sensor mode doesn't detect NPC behind wall_ ║ +╚═══════════════════════════════════════════════╝ +``` + +Tester types description, presses Enter. Bug report saved. + +**Step 5: Reset and retry (or move on).** +Tester presses R to reset the room. All entities return to tick-0. Timer resets. Checklist resets. Tester retries the perception mode test. + +Or, tester presses Home to teleport to Hub and test a different room. + +**Step 6: Complete the session.** +After testing all target rooms, tester presses Ctrl+C to disconnect. + +Test client prints session summary: + +``` +═══════════════════════════════════════════════════ +SESSION SUMMARY — 2026-02-17 08:42 +═══════════════════════════════════════════════════ +Rooms visited: 4/7 +Checklist coverage: 22/38 items (58%) + + Occlusion Corridor: 6/7 ██████████░ 86% 00:47 (PB: 00:38) + Fog Theater: 5/8 ████████░░░ 63% 01:12 (PB: 00:52) + Inventory Warehouse: 7/7 ███████████ 100% 00:25 (PB: 00:25) ★ NEW PB + Crowd Plaza: 4/9 ██████░░░░░ 44% 01:35 (PB: --) + +Bug reports filed: 2 + → tests/bug-reports/gauntlet-2026-02-17T08-42-13/ + → tests/bug-reports/gauntlet-2026-02-17T08-43-57/ + +Stats saved to tests/gauntlet-stats.json +═══════════════════════════════════════════════════ +``` + +THAT'S a session that felt productive. The tester knows exactly what they covered, what they missed, and where the bugs are. They can hand the bug reports to a developer and say "here, everything you need is in the folder." + +### Quick-Test Workflow (developer fixing a specific bug) + +Not every session is a full walkthrough. A developer just fixed the fog shader and wants to verify: + +``` +# Start server + client +make test-world-headless SEED=42 +make test-client-connect + +# Press Home (already in Hub) +# Walk to Fog Theater (or press room-select hotkey) +# Run through fog-specific checklist items +# If it passes: done in 60 seconds +# If it fails: F12, bug report, fix, repeat +``` + +Total time: under 2 minutes. THAT'S why hub-and-spoke layout matters. THAT'S why room reset matters. The developer doesn't touch any room they don't care about. + +--- + +## Summary + +| Deliverable | Status | +|-------------|--------| +| Test client terminal layout specification | Complete | +| Live-updating display rationale | Complete | +| Missing ObserverSnapshot fields identified | Complete (4 fields) | +| Room reset full UX flow | Complete | +| Hub teleport spec | Complete | +| WRONG button full capture flow + file format | Complete | +| Room timer + personal bests spec | Complete | +| Auto-checklist progress tracking spec | Complete | +| Cross-review of Stig's checklist: gaps identified | Complete (4 gaps) | +| Merged checklist YAML format proposed | Complete | +| Full human tester workflow (step-by-step) | Complete | +| Quick-test developer workflow | Complete | + +## Open Questions + +1. **For Dudley:** The `blocked_entities` debug field on ObserverSnapshot — is this feasible within the current `compute_observer_snapshot` pipeline? Estimated cost per tick? +2. **For Stig:** Can the Godot client also render the checklist progress overlay (my Layer 2 debug overlay from Round 1)? Or is checklist tracking test-client-only? +3. **For Tyre:** The test client binary — should it live in `server/src/bin/gauntlet-client.rs` (alongside the server binary) or in a separate `tools/gauntlet-client/` crate? The former shares types easily. The latter keeps the server crate focused. +4. **For Gestalt:** Your cross-room transition scenarios (Round 1) — should these have their own checklist section? Where does "sprint from Crowd Plaza into Occlusion Corridor" live in the room YAML hierarchy? diff --git a/docs/workshops/test-architecture/ozzie-round3.md b/docs/workshops/test-architecture/ozzie-round3.md new file mode 100644 index 000000000..97f0e25a7 --- /dev/null +++ b/docs/workshops/test-architecture/ozzie-round3.md @@ -0,0 +1,520 @@ +# Ozzie — Round 3: Final Tester Workflow + UX Specs + +**Workshop:** QA Strategy & Test Architecture +**Track:** 1 (Test World Design) — Human Tester Experience +**Date:** 2026-02-17 +**Round:** 3 (Prioritization) +**Inputs:** All Round 2 outputs, round-2-notes.md + +--- + +## 1. Human Tester Walkthrough — Final + +This is the definitive step-by-step workflow. A human tester picks this up and knows exactly what to do from zero to bug report. + +### Prerequisites + +- Server binary built: `make build-server` +- Test client binary built: `make build-test-client` (builds `settled-reach-test-client`) +- Gauntlet content pack exists at `content/gauntlet/` +- Checklist generated: `make checklist` (produces `docs/qa/gauntlet-checklist.md`) + +### Session Start + +**Terminal 1 — Server:** +```bash +make test-world-headless +# Equivalent to: settled-reach-server --test-mode --port 0 --seed 42 +# Server prints: LISTENING:54321 +# Server waits for client connection +``` + +**Terminal 2 — Test Client:** +```bash +make test-client +# Equivalent to: settled-reach-test-client --connect 127.0.0.1:54321 --text +# Test client connects, receives first ObserverSnapshot +# Live terminal display appears +``` + +**What the tester sees on connect:** + +``` +╔══════════════════════════════════════════════════════════════════════╗ +║ GAUNTLET TEST CLIENT v0.1 Tick: 0 TickRate: Full ║ +║ Room: Central Hub Seed: 42 Session: 00:00:01 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ PLAYER (50,50) → North | Walk | Inventory: 0/9 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ ENTITIES (0 visible) ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ FOG Clear:24 | Periph:8 | Deep:0 | Map:0 | Dark:468 ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ EXITS: [N]Fog Theater [E]Occlusion [S]Dialogue [W]Inventory ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ MONOLOGUE: none | DIALOGUE: none ║ +╠══════════════════════════════════════════════════════════════════════╣ +║ CHECKLIST: Central Hub — N/A ║ +║ [F12:WRONG] [Home:Hub] [R:Reset] ║ +╚══════════════════════════════════════════════════════════════════════╝ +``` + +### Phase 1: Navigate to a Room + +Tester sends movement inputs (arrow keys or WASD via `--interactive` mode, or via scripted replay file). They walk East toward the Occlusion Corridor. + +As they enter the corridor connecting Hub to Occlusion Corridor: +- Room name changes: `Room: → Occlusion Corridor` +- Timer starts: `TIMER: 00:00` +- Checklist loads: `CHECKLIST: Occlusion Corridor — 0/7` +- Entities appear as they enter the player's vision cone + +### Phase 2: Execute the Checklist + +The tester has a printed copy of `docs/qa/gauntlet-checklist.md` taped to their monitor (or the test client shows checklist items in the status bar). + +**Step 1 from checklist:** "Stand at corridor entrance (45,3), face East" + +Tester moves to position. Terminal updates: + +``` +╠══════════════════════════════════════════════════════════════════════╣ +║ ENTITIES (3) ║ +║ ● npc:100 (48,3) Fwd Neutral(#4a9ebb) VISIBLE d=3 ║ +║ ● obj:200 (46,2) Fwd n/a VISIBLE d=1 ║ +║ ✕ npc:101 (50,7) --- --- BLOCKED ║ +║ └─ wall at (49,4) blocks LOS ║ +╠══════════════════════════════════════════════════════════════════════╣ +``` + +Tester checks against the checklist: +- [x] `npc:100` (guard-1) visible, Forward sector? YES +- [x] `npc:101` (hidden-1) NOT visible? YES — blocked by wall +- [x] Corridor tiles visible, room behind wall unexplored? Check FOG line + +Three items checked. Progress bar updates: `3/7 ██████░░░░░` + +**Step 2:** "Walk south to (45,8), observe peripheral vision" + +Tester moves. guard-1 shifts from Forward to Peripheral: + +``` +║ ◐ npc:100 (48,3) Periph Neutral(#4a9ebb) VISIBLE d=6 ║ +``` + +- [x] guard-1 in Peripheral sector? YES — symbol changed from `●` to `◐` +- [x] Fog visible_count decreased? Check FOG line: was 24, now 18 + +**Step 3:** "Switch to Sensor perception mode" + +Tester sends TogglePerceptionMode input. If working correctly, hidden-1 should appear: + +``` +║ ⚡ npc:101 (50,7) --- Unknown(#4a9ebb) DETECTED d=7 ║ +``` + +- [x] hidden-1 appears in entity list with sensor data? YES + +**Checklist complete: 7/7.** Timer shows `00:47`. If personal best was `00:38`, tester sees `(PB: 00:38)`. + +### Phase 3: Encounter a Bug + +What if step 3 fails? hidden-1 doesn't appear even in Sensor mode. The tester presses F12. + +**F12 Flow:** + +1. Game pauses immediately (Pause sent to server) +2. Terminal shows: + +``` +╔══════════════════════════════════════════════════════════════════╗ +║ BUG REPORT — Tick 87 — Occlusion Corridor ║ +╠══════════════════════════════════════════════════════════════════╣ +║ What's wrong? (type, then Enter): ║ +║ > Sensor mode doesn't detect NPC behind wall_ ║ +╚══════════════════════════════════════════════════════════════════╝ +``` + +3. Tester types one sentence, presses Enter +4. Terminal shows: + +``` +Bug report saved: tests/bug-reports/gauntlet-t87-20260217-084213/ + report.md | snapshot.json | text_output.txt | description.txt +Resume testing. [Enter] +``` + +5. Game unpauses. Tester continues. + +### Phase 4: Reset and Re-Test + +Tester walks back to the room entrance and steps on the reset plate (2x2 amber-striped floor tile). Interaction list shows `[Reset Room]`. Tester activates it. + +``` +Reset: Occlusion Corridor → tick-0 state. Timer reset. +``` + +All entities return to starting positions. Fog reverts. Timer restarts. Checklist resets to 0/7. Tester can re-run the checklist from step 1. + +### Phase 5: Move to Another Room + +Tester presses `Home` → teleports to Central Hub. Walks to next room. Repeat Phase 2-4. + +### Phase 6: End Session + +Tester presses Ctrl+C. Test client prints: + +``` +══════════════════════════════════════════════════════ +SESSION SUMMARY — 2026-02-17 08:55 +══════════════════════════════════════════════════════ +Rooms tested: 4 / 14 +Total time: 12:34 + + Occlusion Corridor: 7/7 ███████████ 100% 00:47 + Fog Theater: 5/8 ████████░░░ 63% 01:12 + Inventory Warehouse: 7/7 ███████████ 100% 00:25 ★ PB + Crowd Plaza: 4/9 ██████░░░░░ 44% 01:35 + +Bug reports filed: 1 + → tests/bug-reports/gauntlet-t87-20260217-084213/ + +Stats saved: tests/gauntlet-stats.json +══════════════════════════════════════════════════════ +``` + +Done. Developer picks up the bug report folder, has everything they need to reproduce. + +--- + +## 2. Test Client UX Final Spec + +Merges my Round 2 terminal layout with Tyre's `kind:entity_id` labels and the lead-confirmed architecture decisions. + +### Display Architecture + +- **Binary:** `settled-reach-test-client` at `tooling/test-client/` (separate crate, imports shared types from server) +- **Text renderer:** `tooling/test-client/src/text_renderer.rs` (or shared module within the crate) +- **Display mode:** Live-updating terminal via crossterm ANSI escape codes. NOT a TUI framework. Fixed-layout sections that refresh every tick. +- **Simultaneous logging:** Every tick appended to `gauntlet-session-{timestamp}.log` for the WRONG button and post-session review. + +### Entity Label Format + +Per Tyre's decision: `kind:entity_id`. Examples: +- `npc:100` — NPC with wire entity_id 100 +- `obj:200` — Object with wire entity_id 200 +- `player:0` — Player entity + +No display names on wire. Test assertions use Gauntlet constants: `assert_entity_visible(&snapshot, GUARD_1.wire_id)`. + +For human testers, the printed checklist maps IDs to names: "npc:100 = guard-1, npc:101 = hidden-1". This lives in the checklist header, not the terminal display. + +### Entity Visibility Symbols + +| Symbol | State | Meaning | +|--------|-------|---------| +| `●` | VISIBLE | In clear vision cone (Forward or Peripheral) | +| `◐` | REMEMBERED | In fog, previously seen, entity persists in snapshot | +| `◌` | FOGGED | Detected in fog, NOT yet recognized (grey blob state) | +| `✕` | BLOCKED | Exists but LOS blocked by wall (debug info, requires `blocked_entities` field) | +| `⚡` | RECOGNIZING | Mid-cognitive-delay (D-060), transitioning from blob to recognized | + +Peripheral entities show `●` but with the `Periph` sector label — the symbol indicates visibility state, the sector label indicates where in the cone. + +### Section Order (top to bottom) + +1. **Header** — tick, tick rate, room name, seed, session timer +2. **Player** — position, facing, stance, inventory count +3. **Entities** — one line per entity, sorted by distance (nearest first per Tyre's format) +4. **Fog** — 5-layer tile counts (Clear/Periph/Deep/Map/Dark) +5. **Sound** — events this tick (MVP: omit if no `SoundEvent` in snapshot yet) +6. **Cognition** — active cognitive delays (MVP: derive from `pending_recognitions`) +7. **Interactions** — available verbs per entity +8. **Monologue/Dialogue** — exact text content +9. **Inventory** — slot map +10. **Status** — checklist progress, timer, PB, hotkey reminders + +### MVP vs Full Display + +**Sprint 8 MVP** (what ships first): + +Sections 1, 2, 3, 4, 7, 8, 9, 10. These derive entirely from the existing ObserverSnapshot fields. No server changes needed. + +**Sprint 9+ Full** (requires new snapshot fields): + +Add sections 5 (Sound) and 6 (Cognition). These need: +- `Vec` in ObserverSnapshot (source position, type, range) +- Enhanced `pending_recognitions` with elapsed/total timing +- `blocked_entities` for the `✕ BLOCKED` display + +### Refresh Behavior + +- **Per-tick refresh.** Every ObserverSnapshot received triggers a full terminal redraw. +- **Cursor positioning.** Use ANSI escape `\x1b[H` (cursor home) + section rewrites. No full clear (avoids flicker). +- **Paused display.** When TickRate is Paused, display shows `[PAUSED]` in header. Still refreshes on snapshot receipt (pause state changes are snapshots). + +--- + +## 3. WRONG Button Final Spec + +### MVP — Sprint 8 + +The MINIMUM that makes bug reporting useful. Implementable without any server changes. + +**Hotkey:** F12 + +**What gets captured:** + +| Data | Source | Format | Size | +|------|--------|--------|------| +| Current ObserverSnapshot | Last received snapshot (already in memory) | JSON | ~2-5KB | +| Current text output | Last `format_snapshot_text()` result | Text | ~1KB | +| Tick + room + seed | From snapshot + Gauntlet constants | Part of report.md | Trivial | +| Tester description | One-line text prompt | Text | ~100 bytes | + +**What does NOT ship in MVP:** +- No snapshot history ring buffer (requires 60x snapshot storage) +- No input history (requires InputMapper ring buffer) +- No screenshot (test client is terminal-only) +- No world digest (requires server-side addition) + +**UX Flow (MVP):** + +1. F12 pressed +2. Test client sends Pause to server +3. Terminal shows one-line prompt: `BUG: What's wrong? > _` +4. Tester types, presses Enter +5. Test client writes files to `tests/bug-reports/gauntlet-t{tick}-{timestamp}/`: + +``` +tests/bug-reports/gauntlet-t87-20260217-084213/ +├── report.md # Human-readable summary (generated) +├── snapshot.json # ObserverSnapshot as pretty-printed JSON +├── text_output.txt # What the terminal was showing +└── description.txt # Tester's one-line description +``` + +6. Test client sends Unpause +7. Testing resumes + +**report.md Format (MVP):** + +```markdown +# Bug Report — Gauntlet +- **Tick:** 87 +- **Room:** Occlusion Corridor +- **Seed:** 42 +- **Date:** 2026-02-17 08:42:13 +- **Description:** Sensor mode doesn't detect NPC behind wall + +## Player State +Position: (45,8) facing East | Stance: Walk | Inventory: 0/9 + +## Entities at Time of Report +| ID | Position | Sector | Relationship | Visibility | Distance | +|----|----------|--------|-------------|------------|----------| +| npc:100 | (48,3) | Periph | Neutral | VISIBLE | 6 | +| npc:101 | (50,7) | --- | --- | BLOCKED | 7 | + +## Fog State +Clear: 18 | Peripheral: 6 | Deep: 12 | Map: 0 | Dark: 464 + +## Checklist State +Occlusion Corridor: 5/7 — item 'occ_hidden_sensor' FAILED + +## Reproduction +1. `make test-world-headless` (seed 42) +2. `make test-client` +3. Walk to Occlusion Corridor (45,8), face East +4. Switch to Sensor perception mode +5. Expected: npc:101 appears in entity list +6. Actual: npc:101 still BLOCKED +``` + +The reproduction steps are templated from the current checklist step + room position. A developer reads this and can reproduce in under a minute. + +### Full — Sprint 9+ + +Everything in MVP plus: + +| Addition | Source | Requires | +|----------|--------|----------| +| Snapshot history (60 ticks) | Ring buffer in test client | ~300KB memory, client-side only | +| Input history (60 inputs) | Ring buffer in test client | ~12KB memory, client-side only | +| Replay seed + input file | Combine seed + full input log | Can reproduce entire session | +| Room metadata + expected state | From checklist YAML | Client loads checklist at startup | + +**Additional files in Full mode:** + +``` +tests/bug-reports/gauntlet-t87-20260217-084213/ +├── report.md +├── snapshot.json +├── snapshot_history.jsonl # Last 60 snapshots, one per line +├── input_history.jsonl # Last 60 inputs, one per line +├── text_output.txt +├── description.txt +└── room_metadata.json # Checklist items + expected state +``` + +**The ring buffer is the key addition.** 60 ticks = ~6 seconds at 10 tps. When the tester presses F12, the last 6 seconds of game state are preserved. A developer can replay those 6 seconds to see exactly what happened leading up to the bug. + +### Implementation Notes for Dudley (server) and Stig (client) + +**Server (Dudley):** MVP requires ZERO server changes. The test client formats the existing ObserverSnapshot. Full mode also requires no server changes — the ring buffer and input history are client-side. The only server-side addition (deferred to Sprint 9+) is `blocked_entities` for the `✕ BLOCKED` display. + +**Client (Stig):** The Godot client gets its own F12 handler (`bug_report.gd` autoload per Stig's R2 spec). The Godot version captures a screenshot + scene tree dump that the test client can't. Both clients write to the same `tests/bug-reports/` directory. Reports from either client are useful. + +--- + +## 4. Quick-Test Developer Workflow + +A developer just fixed the fog shader. They want to verify the Occlusion Corridor works. Total time target: **under 2 minutes.** + +### The Flow + +``` +# Step 1: Build (incremental, ~5-10s) +make build-server + +# Step 2: Start server + client (split terminal or use tmux) +make test-world-headless & +# Wait for LISTENING:port output +make test-client + +# Step 3: Teleport to target room (~2s) +# Press Home (if not already in Hub) +# Walk East to Occlusion Corridor (or type room shortcut if implemented) + +# Step 4: Run the specific checklist items (~30-60s) +# Move to (45,3), face East — check entities +# Move to (45,8) — check peripheral +# Toggle Sensor mode — check hidden NPC + +# Step 5: Done +# If all good: Ctrl+C, session summary confirms 7/7 +# If bug found: F12, one sentence, resume or Ctrl+C +``` + +**Total wall-clock:** Build 10s + startup 3s + navigate 5s + test 45s + exit 2s = **~65 seconds.** + +### Why This Is Fast + +1. **Incremental build** — only recompiles changed files (~5-10s for a shader fix) +2. **Hub-and-spoke layout** — walk directly to target room, skip everything else +3. **No server restart for re-test** — room reset plate lets them retry without killing the server +4. **No report overhead** — F12 captures everything in one press, no manual note-taking + +### For Regression Verification After a Fix + +The developer fixed the bug from the report. Now they want to verify: + +```bash +# Option A: Manual verification (interactive) +make test-world-headless & +make test-client +# Navigate to the room, reproduce the steps from report.md +# Verify the bug is fixed + +# Option B: Automated verification (replay file) +make test-world-headless & +settled-reach-test-client \ + --connect 127.0.0.1:54321 \ + --replay tests/bug-reports/gauntlet-t87-20260217-084213/input_history.jsonl \ + --ticks 90 \ + --text +# Watch the replay, verify the entity appears correctly at tick 87 +``` + +Option B is available once the Full WRONG button ships (Sprint 9+) — it uses the captured input history as a replay file. The developer literally replays the exact sequence that triggered the bug and checks if it's fixed. + +--- + +## 5. Anti-Tedium Priority for Sprint 8 MVP + +Four features approved. Not all are equally important for Sprint 8. Here's the priority ranking. + +### Sprint 8: MUST SHIP (2 features) + +#### Priority 1: Room Reset Triggers + +**Why first:** Without room reset, a tester who wants to re-run a room's checklist must restart the server. That's 30+ seconds of downtime every retry. Over a sprint of testing, that's hours of wasted time. Room reset is the difference between "I'll re-test that" and "I'll skip it." + +**Implementation scope:** +- Server: `RoomResetTrigger` component, `RoomSnapshots` resource, `detect_room_reset` + `execute_room_reset` systems, 10-tick debounce. ~150 lines (Dudley's R2 spec). +- Client: `reset_plate` tile type in TileRenderer, "Reset Room" interaction verb. ~20 lines (Stig's R2 spec). +- Test mode only: systems only registered with `--test-mode`. + +**Estimated effort:** 1-1.5 days. + +#### Priority 2: WRONG Button (MVP) + +**Why second:** Without the WRONG button, a tester who finds a bug has to manually describe it — writing down the tick, room, entity positions, what they expected. That's slow, error-prone, and most testers won't bother. The WRONG button makes bug reporting a 5-second action. + +**Implementation scope:** +- Test client: capture current snapshot + text output, one-line prompt, write to disk. ~100 lines. +- Godot client: `bug_report.gd` autoload, F12 handler, modal prompt, snapshot + screenshot + scene tree dump. ~80 lines (Stig's R2 spec). +- No server changes. + +**Estimated effort:** 1 day. + +### Sprint 9: SHOULD SHIP (2 features) + +#### Priority 3: Hub Teleport + +**Why deferred:** Hub teleport saves walking time (~5-10 seconds per room transition). Useful, but the Gauntlet corridors are only ~10-15 tiles long. Walking is tolerable in Sprint 8. Hub teleport becomes more valuable as more rooms are added. + +**Implementation scope:** +- Server: `TeleportToHub` PlayerAction variant, process in `process_player_input`. ~20 lines. +- Client: `Home` key mapping, `TeleportHub` wire action, camera snap + fade. ~30 lines. + +**Estimated effort:** 0.5 days. + +#### Priority 4: Room Timer + Auto-Checklist Progress + +**Why deferred:** Timers and progress tracking are engagement features — they make testing more pleasant, not more possible. The Gauntlet works without them. They become valuable once the team is doing regular sprint-over-sprint testing. + +**Implementation scope:** +- Test client: room detection from bounds, wall-clock timer, stats file, session summary. ~150 lines. +- Test client: checklist YAML loading, condition evaluation against snapshot, progress display. ~200 lines. +- Godot client: `GauntletProgress` overlay. ~60 lines (Stig's R2 spec). +- Stats persistence: `tests/gauntlet-stats.json`. + +**Estimated effort:** 1.5-2 days. + +### Summary Table + +| Priority | Feature | Sprint | Effort | Why | +|----------|---------|--------|--------|-----| +| **P1** | Room Reset | **8** | 1-1.5d | Can't re-test without it | +| **P2** | WRONG Button (MVP) | **8** | 1d | Can't report bugs without it | +| P3 | Hub Teleport | 9 | 0.5d | Convenience, not necessity | +| P4 | Timer + Checklist | 9 | 1.5-2d | Engagement, not necessity | + +**Sprint 8 anti-tedium budget: ~2-2.5 days.** This fits alongside the other Sprint 8 priorities (test client binary, determinism fixes, server flags). + +### What About F3 Debug Overlay? + +**Defer indefinitely.** Stig is right — the WRONG button captures the same data on demand. F3 as a real-time overlay is a performance cost for a marginal convenience gain. If testers ask for it after using the WRONG button for a sprint, reconsider. + +--- + +## Summary of Round 3 Deliverables + +| # | Deliverable | Status | +|---|-------------|--------| +| 1 | Human tester walkthrough — final (start to finish, 6 phases) | Complete | +| 2 | Test client UX final spec (labels, symbols, sections, MVP vs full) | Complete | +| 3 | WRONG button final spec (MVP Sprint 8 + Full Sprint 9+) | Complete | +| 4 | Quick-test developer workflow (65-second flow) | Complete | +| 5 | Anti-tedium Sprint 8 priority (Room Reset + WRONG Button MVP) | Complete | + +### Build-Ready Specs + +These specs are implementation-ready for the assigned developers: + +- **Dudley:** Room reset server mechanism (Section 5, P1). Use your R2 `RoomResetTrigger` + `RoomSnapshots` design. No server changes needed for WRONG button MVP. +- **Stig:** Room reset tile type + WRONG button Godot client (Section 5, P1+P2). Use your R2 `reset_plate` tile + `bug_report.gd` autoload designs. +- **Tyre:** Test client binary text renderer + WRONG button integration (Section 2+3). Use your R2 `test_client.rs` design. Add the F12 capture flow from Section 3 MVP. + +The human tester picks up `docs/qa/gauntlet-checklist.md`, opens two terminals, and goes. That's the goal. That's what we're shipping. diff --git a/docs/workshops/test-architecture/round-1-notes.md b/docs/workshops/test-architecture/round-1-notes.md new file mode 100644 index 000000000..f37853a8a --- /dev/null +++ b/docs/workshops/test-architecture/round-1-notes.md @@ -0,0 +1,563 @@ +# QA Strategy & Test Architecture Workshop — Round 1 Notes + +**Workshop:** QA Strategy & Test Architecture +**Round:** 1 (Analysis) +**Date:** 2026-02-17 +**Documenter:** Qatux +**Participants:** Gestalt, Dudley, Tyre, Stig, Hoshe, Justine, Ozzie + +--- + +## Executive Summary + +Round 1 produced strong consensus on the major architectural questions and a wealth of concrete specifications. All 7 participants delivered detailed, code-level proposals. The key finding: **the simulation is remarkably close to deterministic already** — Tyre and Dudley independently audited the codebase and found only 2-4 targeted fixes needed, totaling ~15 lines of code. The workshop also surfaced 5 new Gauntlet rooms (from Gestalt), 36 map-agnostic invariants, a 41-value boundary test matrix (from Hoshe), and a concrete CI pipeline design (from Justine). + +**Five decisions require lead confirmation before Round 2 can proceed.** See [Decisions Requiring Confirmation](#decisions-requiring-confirmation) below. + +--- + +## Track 1: Test World Design (Gauntlet) + +### Participants: Gestalt (T1), Dudley (T1), Tyre (T1), Ozzie (T1) + +### 1.1 Gauntlet Implementation Strategy + +**Consensus: Hybrid YAML content pack + Rust ECS state injection.** + +Both Tyre and Dudley independently proposed the same hybrid approach — strong convergence. + +| Aspect | YAML content pack | Rust builder function | +|--------|-------------------|----------------------| +| Room geometry / walls | WalkabilityMap (Dudley: Rust builder; Tyre: "YAML for geometry") | — | +| NPC/item placement | `content/gauntlet/` directory, loaded by existing ContentPlugin | — | +| Knowledge graph states | — | Post-spawn injection in `inject_gauntlet_state()` | +| Trust tiers, contradictions | — | Programmatic: `player_kg.set_relationship(...)` | +| Named coordinates | `gauntlet_coords` constants module | Referenced by tests | + +**Minor tension on geometry:** Dudley proposes WalkabilityMap construction in Rust (`server/src/test_world/rooms.rs`). Tyre proposes "YAML for geometry + entity placement." The difference is whether room walls are YAML or code. **Both agree** the content loader handles NPCs; the question is map geometry specifically. + +**Proposed by:** Dudley and Tyre (independently converged) +**Estimated effort:** 2-3 days for first 3 rooms, ~0.5 day per additional room (Tyre) + +### 1.2 Gauntlet Layout: Hub-and-Spoke with Cross-Cuts + +**Proposed by:** Ozzie + +Ozzie argued strongly against both pure linear and pure hub-and-spoke layouts. Recommendation: **central hub** with connections to every room, plus **intentional cross-room corridors** for testing system combinations. + +Key cross-cut corridors proposed: +- Crowd Plaza <-> Occlusion Corridor (sprint through crowd into LOS testing) +- Fog Theater <-> Dialogue Room (cognitive delay + dialogue interaction) +- Inventory Warehouse <-> Interaction Gallery (full inventory + interaction verbs) +- Pause Chamber connects to all rooms + +No dissent on this from other participants. + +### 1.3 Missing Gauntlet Rooms + +**Proposed by:** Gestalt + +Gestalt identified 5 rooms missing from the workshop brief's proposed list, each targeting system *combinations* not tested by isolated rooms: + +| Room | Priority | Systems Combined | Key Mechanic Tested | +|------|----------|-----------------|---------------------| +| **Eavesdrop Alcove** | P1 | D-071 + D-072 + D-053 | ListeningFocus, zone ambient, Careful stance perception | +| **Confrontation Stage** | P0 | D-070 + D-069 + D-018 | Confrontation as cognitive vulnerability, audio dip, post-confrontation delayed monologue | +| **Sprint Gauntlet** | P1 | D-055 + D-053 + D-060 | Sprint suppression, anomaly survival, interaction buffer clearing | +| **Sound Lab** | P0 | D-018 + D-059 + D-067 | Three-range sound model, sound pings, recognition chime sequence | +| **Decay Observatory** | P2 | D-041 + D-059 + D-011 | Knowledge decay timing, fog representation degradation | + +Additionally, Gestalt proposed a **Shift Change Room** as the combined stress test — 15 NPCs simultaneously changing state (arriving, leaving, conversing, entering/exiting fog) within a 10-tick window. This replaces simple density testing with *simultaneous system activation* testing. + +Gestalt also defined 6 **cross-room transition scenarios** testing system combinations at room boundaries (sprint exit, peripheral interact, cognitive delay + dialogue, confrontation + sound, pause during delay, walk-away sprint). + +### 1.4 Gauntlet Signage and Tester UX + +**Proposed by:** Ozzie + +Three-layer signage system: +1. **In-world station signage** — diegetic labels ("PERCEPTION CALIBRATION BAY"), always visible +2. **Debug overlay (F3 toggle)** — floating entity labels with full state data +3. **Printed checklist** — auto-generated from room metadata, markdown format + +Stig's complementary proposal: checklists co-located with Gauntlet room YAML definitions. `make checklist` generates `gauntlet-checklist.md`. New rooms automatically extend the checklist. + +### 1.5 Anti-Tedium Features + +**Proposed by:** Ozzie (emphatic — "this is where the Gauntlet lives or dies") + +| Feature | Description | Priority | +|---------|-------------|----------| +| Room Reset Triggers | Floor plate at entrance resets room to tick-0 state | Mandatory | +| Instant Hub Teleport | Hotkey returns to Central Hub | Mandatory | +| Room Timer + PB | Gamified testing with speedrun incentive | Nice-to-have | +| State Inspector (F3) | Extended debug overlay with full ECS data | High | +| "WRONG" Button (F12) | One-press bug report: captures full game state + 60 ticks of history | High | +| Auto-Checklist Progress | Tracks which checklist items verified this session | Nice-to-have | + +### 1.6 Reserved Zone Gate + +**Proposed by:** Tyre + +Minimum provision: empty room in map YAML + `ZoneTransition` trait stub + future-contract test file (`zone_transition_contracts.rs` with `#[test] #[ignore]`). Zero runtime code. ~0.5 day effort. + +### 1.7 Performance Budget + +**Proposed by:** Tyre + +| Threshold | Tick Time | Action | +|-----------|-----------|--------| +| Target | ≤5ms | Normal operation | +| Warning | >8ms | Investigate | +| Error | >20ms | Test failure (>40% of frame budget) | +| Critical | >50ms | Tick overflow, simulation can't keep up | + +CI performance test: 100 ticks, measure p50/p95/max, fail if p95 > 20ms. + +--- + +## Track 2: Deterministic Gameplay + +### Participants: Tyre (T2), Dudley (T2) + +### 2.1 Determinism Audit — Independent Convergence + +Tyre and Dudley **independently** audited the critical path and reached **highly aligned conclusions**. The simulation is close to deterministic already. Both identified the same primary issue: `HashSet` iteration in `observer/mod.rs`. + +**Combined non-determinism findings:** + +| # | Location | Issue | Tyre | Dudley | Severity | +|---|----------|-------|------|--------|----------| +| 1 | `observer/mod.rs:134` | Sprint anomaly iterates `HashSet` — "first Contradicted match" is non-deterministic | CRITICAL — Fix #1 | Identified same pattern | CRITICAL | +| 2 | `observer/mod.rs:197` | `visible_tiles` Vec built from HashSet iteration, non-deterministic ordering | MEDIUM — Fix #2 | Identified as "cosmetically non-deterministic" | MEDIUM | +| 3 | `movement.rs:288` | `validate_movement` tie-breaking is implicit (archetype order) | LOW — Fix #3 (document) | "no test for which one wins" — proposes sort by Entity bits | LOW-MEDIUM | +| 4 | `interaction.rs:191` | Equidistant NPC ordering | Not flagged | Identified | LOW | + +**Tyre's Fix #4 (client audit):** Need to verify Godot fog/tile rendering doesn't depend on `visible_tiles` Vec ordering. Delegated to Stig for Round 2. + +**SimRng status:** Both agree the SimRng (ChaCha20) is correctly seeded and consumed in a deterministic order — IF system execution order is fixed. Dudley identified that monologue systems lack explicit `.after()` constraints and could run in either order. Tyre confirmed system ordering is explicit in `SimulationPlugin::build()` but monologue ordering needs pinning. + +### 2.2 Minimum Change Set for Determinism + +**Consensus: 4 targeted fixes, ~15-30 lines total.** + +| Fix | Change | Lines | Blocks | +|-----|--------|-------|--------| +| `visible_ids: HashSet → BTreeSet` | `observer/mod.rs` | ~3 | Deterministic sprint anomaly detection | +| Sort `visible_tiles` by `(x, y, z)` | `observer/mod.rs` | ~1 | Canonical wire format for golden files | +| Pin monologue system ordering | `simulation/mod.rs` | ~3 | Deterministic RNG consumption | +| Sort movers by Entity bits (or document) | `movement.rs` | ~5 | Deterministic tie-breaking | + +**Explicitly NOT required** (both agree): +- `WalkabilityMap.chunks: HashMap` — point lookups only, never iterated +- `validate_movement` occupied HashMap — containment checks only +- `visible_positions: HashSet` — membership checks only +- Fixed-point math — simulation is integer-only; f32 in vision_cone.rs is IEEE 754 deterministic same-platform + +### 2.3 Determinism Test Specification + +**Proposed by:** Tyre (with concrete code) + +``` +Test: gauntlet_deterministic_replay +Setup: Build headless App with SimulationPlugin + BridgePlugin (no TCP) +Input: Seed 42, fixed input sequence, run to tick 50 +Assert: Two identical runs produce identical ObserverSnapshot +``` + +Both agree the test is feasible once the 4 fixes are applied. Estimated 1-2 days. + +**Key requirement:** `ObserverSnapshot` must derive `PartialEq` or compare via canonical serialized form (since it contains f32 fields). + +--- + +## Track 3: Test Automation & Text Renderer + +### Participants: Tyre (T3), Dudley (T3), Stig (T3) + +### 3.1 Test Client Architecture + +**Consensus: Integration test harness in `server/tests/`, not a separate binary.** + +Tyre proposed extending the `game_loop.rs` pattern. The `GauntletRunner` helper wraps a headless bevy App, pre-loads inputs, ticks, and extracts snapshots. No TCP needed for most tests; TCP used only for Layer 3 IPC tests. + +``` +server/tests/gauntlet/ + mod.rs, helpers.rs, replay.rs + gauntlet_inventory.rs, gauntlet_occlusion.rs, etc. +``` + +### 3.2 Text Renderer + +**Consensus: Server-side formatter on ObserverSnapshot, output to stdout.** + +All three participants agree the text renderer should: +- Live server-side (no Godot dependency) +- Format ObserverSnapshot (not raw ECS state) +- Be activated via `--text-render` flag (Stig) or `--text-mode` flag (Tyre) +- Output to stdout for piping/diffing + +**Missing data for text rendering** (identified by Dudley): +1. Wall positions blocking LOS paths — not in snapshot (only visible tiles) +2. Entity display names — snapshot has `entity_id` but no name +3. Explored vs. unexplored tile counts — client-side accumulation, not in snapshot + +Dudley proposes an optional `blocked_entities: Vec` field, populated only when `debug_los: true`. + +**Ozzie's additions** — the text renderer MUST also show: +- Sound events (D-018 three-range model) — "40% of the perception system" +- Fog layer TYPE per entity (D-059 five layers, not just "remembered") +- Cognitive delay state with timing (D-060) +- Monologue content (not just "active") +- Entity colors with semantic labels ("#4a9ebb (Neutral)") +- What's blocked and WHY ("wall at (17,10) blocks LOS to npc:hidden-1") + +**Stig's "no" on client `--text-mode`:** Text rendering is purely server-side. Client gets `--verbose` flag for one-line-per-frame debug output only. + +### 3.3 Assertion Language + +**Consensus: Rust test macros with named helpers. No custom DSL.** + +Tyre argued against a custom DSL: "requires a parser, error handling, debugging tools, documentation. It's a language design project. Premature for v0.1." The same expressiveness comes from Rust helpers: + +```rust +fn assert_entity_visible(snapshot: &ObserverSnapshot, name: &str, pos: TilePosition); +fn assert_entity_not_visible(snapshot: &ObserverSnapshot, name: &str); +fn assert_inventory_count(snapshot: &ObserverSnapshot, expected: usize); +fn assert_interaction_available(snapshot: &ObserverSnapshot, entity: &str, verb: &str); +// ... ~10 assertion helpers total +``` + +DSL can be revisited as a sprint 10+ luxury if needed. + +### 3.4 Client-Side Assertion Targets + +**Proposed by:** Stig + +~32 new gdUnit4 test functions across 6 categories, all structural scene-tree assertions (no pixel comparison): + +| Category | Test Count | Key Assertions | +|----------|-----------|----------------| +| Camera system | 7 | Position after ready, anchored flag, smoothing, follows movement, static during pause | +| Entity rendering | 7 | Peripheral alpha=0.5, forward alpha=1.0, D-033 colors, lifecycle | +| Z-layer ordering | 4 | Fog rect z=900, fog entities z=950, insert canvas=10, UI canvas=20 | +| Fog shader state | 3 | Visibility texture updates, exploration persistence (255→128), peripheral dimming (180) | +| UI elements | 8 | Monologue consumed once, not lost on overwrite (bug #5), interaction list, dialogue, inventory grid | +| Entity lerp | 3 | Target set on update, snap on first appearance, convergence | + +### 3.5 Scripted Replay + +**Proposed by:** Dudley + +InputQueue pre-loading is straightforward — `push()` with tick-ordering enforcement. Streaming `ReplayFeeder` handles sequences >1000 inputs (InputQueue capacity). Wait conditions use tick-budget polling: + +```rust +enum WaitCondition { + RecognitionComplete { entity_name: String }, + MonologueFired { monologue_id: String }, + EntityVisible { entity_name: String }, + Ticks(u64), +} +``` + +Every wait condition gets a tick budget (e.g., 1000 ticks max) to prevent infinite loops. + +--- + +## Track 4: Serialization & Integration Testing + +### Participants: Hoshe (T4), Dudley (T4) + +### 4.1 MessagePack Boundary Value Matrix + +**Proposed by:** Hoshe (comprehensive specification) + +41 boundary test values covering every format transition in the MessagePack integer encoding. Traces every branch in `messagepack.gd:69-95` against the MessagePack spec. + +Key ranges: positive fixint (0-127), uint 8 (128-255), int 16 (256-32767), uint 16 (32768-65535), int 32 (65536-2^31-1), uint 32 (2^31-2^32-1), int 64 (2^32+), and corresponding negative boundaries. + +**Critical finding — asymmetric encoding:** GDScript encoder uses int_16 for positive values 256-32767; Rust encoder (rmp_serde) uses uint_16 for the same values. Both are spec-valid but produce different bytes. Implications: +- Byte-for-byte golden file comparison between GDScript and Rust will FAIL for values 256-32767 +- Golden files must be direction-specific (Rust-canonical vs GDScript-canonical) +- Decoders on both sides MUST accept both signed and unsigned encodings + +### 4.2 Boundary Test Placement + +**Proposed by:** Hoshe + +| Layer | Location | Speed | Frequency | +|-------|----------|-------|-----------| +| Encode-only (GDScript) | `test_msgpack_boundaries.gd` | ~100ms | Every commit | +| Encode-only (Rust) | `serialization.rs` extension | ~50ms | Every commit | +| Decode cross-language | Fixtures: Rust→GDScript and GDScript→Rust | ~2s | Every PR | +| Full roundtrip | `bridge_tcp.rs` extension over TCP | ~5s | Nightly/pre-merge | + +### 4.3 gen_fixtures.rs Extension + +**Proposed by:** Hoshe + +Add `generate_boundary_fixtures()` producing: +1. Raw integer boundary fixtures (one `.msgpack` per boundary value) +2. Snapshot boundary fixtures (snapshots with tick values at critical boundaries: 127, 128, 32768, 65536) + +Client verifies via `test_boundary_fixture_snapshot_tick_128()`. + +### 4.4 Golden File Pipeline + +**Consensus between Hoshe and Tyre:** ObserverSnapshot at fixed player positions, not full ECS world state. + +Hoshe's pipeline: +1. Server generates canonical snapshots (`cargo test --test gen_gauntlet_golden -- --ignored`) +2. `.msgpack` + `.json` companion (human-readable diff) +3. CI compares fresh generation against committed golden files +4. Field-by-field comparison with actionable error messages (not just "binary files differ") + +Tyre adds: **multi-position golden files** — one per room, from the room's designated observer position. ~4-6 golden files for the initial Gauntlet. + +**Justine proposes:** Golden files checked into repo (version-controlled alongside code). Changes visible in PR diffs. Sorted JSON for human readability. + +### 4.5 Layer 3 Test (Real Subprocess Integration) + +**Proposed by:** Hoshe (full code specification) + +``` +Test: server_subprocess_sends_snapshot_on_connect +Setup: cargo build server, launch as child process with --test-mode --port 0 +Steps: Send PlayerInput, read ObserverSnapshot with 5s timeout +Assert: version, tick, entity count, player entity kind +Duration: <10 seconds +``` + +Requires two server features: `--test-mode` flag (loads Gauntlet, fixed seed, exits after disconnect) and `--port 0` support (random available port printed to stdout). + +### 4.6 Pause Guard Tests + +**Proposed by:** Dudley + +8 test cases for `process_player_input` pause filtering — currently only 1 test exists (`process_input_pause_sets_paused`), which doesn't verify movement is actually discarded. + +| Test | Priority | +|------|----------| +| `movement_discarded_while_paused` | P0 — direct bug #3 regression | +| `unpause_accepted_while_paused` | P0 | +| `stance_toggle_allowed_while_paused` | P1 | +| `interact_allowed_while_paused` | P1 | +| `multiple_movements_in_paused_batch_all_discarded` | P1 | +| `pause_unpause_roundtrip_with_movement` | P1 | + +### 4.7 EntityRegistry Lifecycle Tests + +**Proposed by:** Dudley + +5 new test cases for entity lifecycle. Most critical: `old_stable_id_not_resolvable_after_unregister` — prevents a despawned entity's StableId from resolving to a recycled entity, which would corrupt the knowledge graph. + +### 4.8 Bridge Deserialization Robustness + +**Proposed by:** Dudley + +Current `tcp.rs` rejects the ENTIRE `Vec` batch if any single input fails to deserialize. Proposal: per-input deserialization with skip-and-log. However, Dudley notes the alternative: since both sides are co-versioned (D-020), batch failure is a hard programming error, and the current behavior is acceptable IF boundary value tests prevent malformed encoding from shipping. + +--- + +## Track 5: Content Scaling & CI Pipeline + +### Participants: Hoshe (T5), Justine (T5), Tyre (T5) + +### 5.1 CI Pipeline Design + +**Consensus between Hoshe and Justine on a 3-tier pipeline:** + +| Tier | Trigger | Duration Budget | Contents | +|------|---------|----------------|----------| +| **Commit** | Every push | <2 min | lint-server, lint-client, validate-content, check-fact-ids | +| **PR** (merge gate) | PR opened/updated | <10 min | Commit tier + build both, test-server, test-client, fixture staleness check | +| **Nightly** | Scheduled daily | <30 min | PR tier + Layer 3 subprocess, golden file regen + diff, content load-test, performance benchmark | + +Justine provided complete Gitea Actions YAML. Hoshe provided the same tier structure independently. + +**CI runner:** Self-hosted strongly recommended (Gitea already self-hosted at `git.schweitz.internal`). Enables stable performance baselines and pre-installed Godot. + +**Merge policy** (Justine): +- **BLOCKER:** Server tests, client tests, lint, content validation +- **WARNING:** Golden file changed (requires reviewer ack), fixture regeneration needed +- **INFO:** Performance delta, build size increase + +### 5.2 Godot in CI + +**Proposed by:** Justine + +Download official headless binary (not container). ~80MB cached between runs. `--headless` + `--ignoreHeadlessMode` for gdUnit4. Pre-install on self-hosted runner to eliminate download step. + +**Open concern:** Need to verify `test_fog_shader.gd` and `test_rendering.gd` pass headless before gating merges on them. + +### 5.3 Performance Regression Detection + +**Proposed by:** Justine + +Median of 5 runs, relative threshold against committed baseline (`tests/perf/baseline.json`): +- <15% delta: PASS +- 15-30% delta: WARNING +- >30% delta: FAIL + +Baseline updated manually in PRs that legitimately change performance. Self-hosted runner essential for reducing noise. + +### 5.4 Golden File Diff Format + +**Proposed by:** Justine + +Structured field-by-field diff with semantic grouping (not raw binary diff, not full dump). Categories: changed fields, added entities, removed entities, unchanged count. Posted as PR comment via `tea comment`. + +### 5.5 Content Validation Layers + +**Consensus across all three:** + +| Layer | What | Status | Owner | +|-------|------|--------|-------| +| 1. Schema validation | YAML structure | Exists (`make validate-content`) | — | +| 2. Cross-reference validation | Entity refs resolve, fact_ids exist | Partial | Tyre proposes `validate_cross_references()` | +| 3. Runtime "boot and tick" | Load content, tick 10, no panic | Missing | Hoshe proposes `content_produces_valid_snapshot()` | +| 4. Regression snapshots | Golden file comparison | Missing | Tied to Gauntlet golden file pipeline | +| 5. Stress test | 100 ticks with max-NPC pack | Missing | Nightly tier | + +Hoshe adds **district capacity check** (warn if >15 NPCs, the Crowd Plaza boundary). + +### 5.6 Content Scaling Test + +**Proposed by:** Hoshe + +Comparative testing: load baseline content → tick 10 → snapshot, then load baseline + 1 extra NPC → tick 10 → snapshot. Assert modified is superset of baseline, all original NPCs present, no tick >50ms. + +Scaling matrix covers: +1 NPC, +5 NPCs, +15 NPCs, +1 location, +1 item. + +--- + +## Map-Agnostic Invariants + +**Proposed by:** Gestalt (36 invariants) + +Organized into 4 categories. These must hold for ANY valid map — Gauntlet, procedural, or hand-crafted. + +### Structural (INV-S01 through INV-S08) +Player spawn reachable, NPC spawns reachable, NPC routine paths valid, no entity inside geometry, 2x2 geometry minimum (D-066), zone boundary coherence, door bidirectionality. + +### Perception (INV-P01 through INV-P05) +Vision cone at spawn (>0 visible tiles), LOS symmetry, fog layer ordering (clear ⊂ peripheral ⊂ deep ⊂ unexplored), insert independence (D-048), sound propagation coherence (sound range ≥ vision range). + +### Population (INV-C01 through INV-C08) +Minimum nearby NPC (≥1 within 10 tiles of spawn), social site minimum, StableId uniqueness, KG reference validity, D-033 color validity, monologue trigger reachability, dialogue pool non-empty, perception mode consistency. + +### Simulation (INV-T01 through INV-T08) +Deterministic replay, tick budget (≤100ms), snapshot delivery (one per tick, ordered), pause coherence, input ordering, SimRng consumption order, cognitive delay monotonicity, knowledge decay timing. + +Tyre proposed 5 of these independently for dynamic map fuzzy testing (reachability, NPC minimum, no overlap, walkability coherence, content reference integrity). Full alignment with Gestalt's larger set. + +--- + +## Decisions Requiring Confirmation + +These emerged as consensus positions but require explicit lead sign-off before Round 2 implementation planning. + +| # | Decision | Proposed By | Consensus | Dissent | +|---|----------|-------------|-----------|---------| +| **WS-D1** | Commit to determinism now (4 targeted fixes, ~15 lines) | Tyre, Dudley | Unanimous | None — both auditors agree cost is trivial, D-010 mandates it | +| **WS-D2** | Gauntlet as hybrid YAML + Rust inject (not pure YAML or pure Rust) | Tyre, Dudley | Strong (independently converged) | Minor: geometry in YAML (Tyre) vs. geometry in Rust (Dudley) — needs alignment | +| **WS-D3** | Assertion language as Rust macros with named helpers (no custom DSL) | Tyre | Unanimous | None — all agree DSL is premature | +| **WS-D4** | Golden file format as ObserverSnapshot at fixed positions (not full ECS world state) | Tyre, Hoshe, Justine | Unanimous | None | +| **WS-D5** | Text renderer lives server-side, formats ObserverSnapshot to stdout | Tyre, Stig, Dudley | Unanimous | None — Stig explicitly vetoed client-side `--text-mode` | + +--- + +## Open Questions for Round 2 + +### Cross-Agent Questions (raised during Round 1) + +| ID | From | To | Question | +|----|------|----|----------| +| OQ-01 | Tyre | Stig | Does the Godot client depend on `visible_tiles` Vec ordering? (Fix #2 severity) | +| OQ-02 | Tyre | Dudley | Which bevy queries beyond the critical path use iteration order that affects output? | +| OQ-03 | Hoshe | Tyre/Justine | Gitea at `git.schweitz.internal` — are Gitea Actions enabled? Is a self-hosted runner available? | +| OQ-04 | Hoshe | Tyre | Golden file JSON companion format — full ObserverSnapshot or reduced "diff-friendly" format? | +| OQ-05 | Hoshe | Justine | Fixture staleness check (`make fixtures && git diff --exit-code`) — robust enough, or need content-addressed hashing? | +| OQ-06 | Hoshe | Dudley | Does `rmp_serde::from_slice::()` accept int_16-encoded positive values (256-32767)? (GDScript/Rust encoding asymmetry) | +| OQ-07 | Hoshe | Dudley | Server binary — does it support `--test-mode` and `--port 0` flags? Layer 3 depends on this. | +| OQ-08 | Stig | Dudley | Text renderer needs room name from player coordinates. Does Gauntlet content include room bounds? | +| OQ-09 | Stig | Tyre | Text renderer should format from ObserverSnapshot (not raw ECS). Agree? | +| OQ-10 | Stig | Hoshe | Fog texture byte values (255/180/128/0) — should these be named constants in FogState? | +| OQ-11 | Justine | — | Client test stability in headless mode — do `test_fog_shader.gd` and `test_rendering.gd` pass headless? | +| OQ-12 | Dudley | Tyre | `WalkabilityMap.chunks: HashMap` — convert to BTreeMap preemptively, or wait until chunk iteration is needed? | + +### Unresolved Architectural Questions + +| ID | Question | Raised By | Impact | +|----|----------|-----------|--------| +| UQ-01 | Bridge deserialization: per-input skip-and-log vs. batch failure? | Dudley | Robustness vs. simplicity tradeoff | +| UQ-02 | Gauntlet geometry: YAML vs. Rust WalkabilityMap builder? | Tyre/Dudley | Minor — needs alignment in Round 2 | +| UQ-03 | Text renderer missing data: how to expose wall-blocking-LOS info? | Dudley | New optional field on ObserverSnapshot? Performance cost? | +| UQ-04 | Fixture path `../client/tests/fixtures/msgpack/` — fragile relative path? | Justine | CI robustness | + +--- + +## Gaps Identified + +Issues not fully addressed by Round 1 that need exploration in Round 2 or later: + +1. **Save/load interaction with determinism.** Deterministic replay enables "serialize inputs + seed" save format. Nobody addressed how this interacts with the save/load architecture (workshop brief listed but not yet held). + +2. **Multiplayer implications of determinism.** Tyre mentions "future multiplayer sync" as a benefit but the networking implications aren't explored. Oscar (networking) is not a workshop participant. + +3. **Content authoring workflow.** The Gauntlet needs content (NPCs, dialogue, monologue). Who writes it? How is it reviewed? Mellanie (copywriter) is not a participant. + +4. **Visual verification beyond text renderer.** Ozzie's debug overlay (F3) and "WRONG" button (F12) are excellent UX proposals but have no implementation specs. These are client-side features (Stig's domain) not yet designed. + +5. **Nightly test infrastructure.** The nightly tier (Layer 3, golden file, stress test) requires server binary artifacts in CI. The build/cache/reuse workflow is sketched (Justine) but not specified. + +6. **The 5 new rooms (Gestalt) lack entity placement specs.** Gestalt provided layout concepts and named entities but not precise tile coordinates. These need to be pinned before Gauntlet YAML can be written. + +--- + +## Priority-Ordered Action Items (from all participants) + +Combined priority ranking across all tracks. Items at the same priority are independent and can be parallelized. + +| Priority | Item | Track | Owner(s) | Effort | +|----------|------|-------|----------|--------| +| **P0** | Fix `visible_ids` HashSet → BTreeSet | T2 | Dudley | 3 lines | +| **P0** | Sort `visible_tiles` in ObserverSnapshot | T2 | Dudley | 1 line | +| **P0** | Pin monologue system ordering (`.after()`) | T2 | Dudley | 3 lines | +| **P0** | Bug #3 regression test: `movement_discarded_while_paused` | T4 | Dudley | ~30 lines | +| **P1** | Build Gauntlet hybrid loader (YAML + Rust inject) | T1 | Dudley, Tyre | 2-3 days | +| **P1** | Build `GauntletRunner` test harness | T3 | Tyre | 1-2 days | +| **P1** | Write determinism regression test | T2 | Tyre | 1 day | +| **P1** | Boundary value test matrix (41 values, both languages) | T4 | Hoshe | 1-2 days | +| **P1** | gen_fixtures.rs boundary extension | T4 | Hoshe | 1 day | +| **P1** | Pause guard test suite (8 tests) | T4 | Dudley | 1 day | +| **P1** | EntityRegistry lifecycle tests (5 tests) | T4 | Dudley | 0.5 day | +| **P2** | CI pipeline (Gitea Actions YAML) | T5 | Justine | 1-2 days | +| **P2** | Golden file generation + comparison tool | T5 | Hoshe, Tyre | 1 day | +| **P2** | Text renderer for ObserverSnapshot | T3 | Tyre | 0.5-1 day | +| **P2** | Cross-reference content validator | T5 | Tyre | 2 days | +| **P2** | Client-side assertion suite (32 tests) | T3 | Stig | 2-3 days | +| **P2** | Layer 3 subprocess test | T4 | Hoshe | 1-2 days | +| **P3** | Zone Gate architectural provision | T1 | Tyre | 0.5 day | +| **P3** | Fuzzy invariant tests for dynamic maps | T5 | Tyre | 1 day | +| **P3** | Content scaling test suite | T5 | Hoshe | 1-2 days | +| **P3** | Map-agnostic invariant tests (36 invariants) | T1 | Gestalt spec, impl TBD | 2-3 days | + +--- + +## Points of Agreement (strong consensus) + +1. **Determinism is cheap and necessary.** Both auditors confirm ~15 lines of fixes. No dissent. +2. **ObserverSnapshot is the right golden file format.** Unanimously preferred over full ECS world state. +3. **Text renderer lives server-side.** No client-side rendering path. Stig explicit. +4. **Rust assertions, not custom DSL.** Universal agreement that DSL is premature. +5. **Hub-and-spoke Gauntlet layout with cross-cuts.** Ozzie's proposal went unchallenged. +6. **3-tier CI pipeline (commit/PR/nightly).** Hoshe and Justine independently designed the same tiers. +7. **Content validation needs a runtime "boot and tick" layer.** Schema-only is insufficient. + +## Points of Tension (minor, resolvable in Round 2) + +1. **Gauntlet geometry: YAML vs. Rust.** Tyre says YAML; Dudley says Rust builder. Both agree on the hybrid approach for everything else. This is a narrow question about WalkabilityMap construction. +2. **validate_movement tie-breaking:** Tyre says "document as invariant" (accept either winner). Dudley says "sort by Entity bits" (force deterministic winner). Low stakes — the existing test accepts either. +3. **Bridge batch deserialization:** Dudley raises per-input skip-and-log as desirable but acknowledges the current batch-failure behavior is acceptable with test coverage. No strong disagreement. +4. **Gauntlet room count:** The workshop brief proposes 7 rooms + 1 reserved. Gestalt adds 5 more + 1 stress room. Total would be 13-14 rooms. Scope needs to be managed — which rooms are Sprint 7-8 vs. later? diff --git a/docs/workshops/test-architecture/round-2-notes.md b/docs/workshops/test-architecture/round-2-notes.md new file mode 100644 index 000000000..2b794c88f --- /dev/null +++ b/docs/workshops/test-architecture/round-2-notes.md @@ -0,0 +1,623 @@ +# QA Strategy & Test Architecture Workshop — Round 2 Notes + +**Workshop:** QA Strategy & Test Architecture +**Round:** 2 (Synthesis & Cross-Review) +**Date:** 2026-02-17 +**Documenter:** Qatux +**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie + +--- + +## Executive Summary + +Round 2 produced implementation-ready specifications across all 5 tracks. The lead's overrule on WS-D5 (separate test client binary instead of server-side text renderer) was accepted constructively by Tyre, who designed the binary architecture at `server/src/bin/test_client.rs`. Dudley delivered concrete code changes for all 4 determinism fixes (one was already done — a self-correction from Round 1). The cross-review process identified real coverage gaps: Hoshe found ZERO direct test coverage for the pause guard (all 6 of Dudley's proposed tests are genuine gaps), and Tyre added 6 tests to Stig's 32 for a total of 38 client assertions. The anti-tedium suite is fully specified across Ozzie, Stig, Gestalt, and Dudley. Content cross-reference validation is specified with 8 concrete checks. + +**Key convergence:** Hoshe and Justine independently designed compatible `make pre-pr` targets. Ozzie and Stig independently designed compatible checklist formats (with Ozzie identifying 4 additions). Tyre validates Hoshe's CI tiers with refinements. All cross-reviews confirmed prior proposals as feasible. + +**One disagreement:** fixture staleness should be BLOCKER (Tyre) or WARNING (Justine). Tyre's argument is stronger — stale fixtures mean client tests run against outdated protocol data, making all passing results false positives. + +--- + +## Lead Decisions — Agent Responses + +The lead confirmed/overruled the 5 workshop decisions from Round 1. Here is how each was received and incorporated. + +| R1 Decision | Lead Ruling | Agent Response | +|-------------|-------------|----------------| +| **WS-D1:** Commit to determinism | **ACCEPTED** | Dudley produced exact code changes for 4 fixes (see Section 2). Self-corrected Fix C (monologue ordering was already done). | +| **WS-D2:** Hybrid YAML + Rust inject | **ACCEPTED** | Dudley designed `GauntletRoom` constants module + room reset mechanism. Gestalt mapped 14 rooms to the layout. | +| **WS-D3:** Rust test macros | **ACCEPTED** | No further discussion needed. Assertion helpers referenced throughout all proposals. | +| **WS-D4:** ObserverSnapshot golden files | **ACCEPTED** | Justine designed JSON golden file format with field-by-field diff. Dudley confirmed multi-position golden files (one per room). | +| **WS-D5:** Text renderer | **OVERRULED** — separate test client binary | Tyre accepted: "The architectural purity argument wins." Designed `server/src/bin/test_client.rs` with full CLI, text format, Layer 3 integration. See Section 1. | + +**Additional lead decisions incorporated:** +- No Gitea Actions for now — manual `make ci` stays. All agents adapted: Hoshe and Justine designed `make pre-pr` as the developer-facing tool. Tyre validated the CI tier design as "future-ready for when the lead greenlights CI." +- Anti-tedium full suite approved. Ozzie, Stig, Gestalt, and Dudley each contributed implementation specs. +- `--test-mode` + `--port 0` confirmed for Sprint 8. Dudley designed the server flag changes (Section 3). +- Cross-reference validation added to `make validate-content`. Hoshe specified 8 checks; Justine specified 7 checks (compatible sets). + +--- + +## 1. Test Client Binary Architecture (Tyre) + +The lead overruled the Round 1 consensus (WS-D5: server-side text renderer) in favor of a separate Rust binary. Tyre designed the architecture. + +### Location + +**`server/src/bin/test_client.rs`** — a second binary target in the server crate. + +```toml +# server/Cargo.toml +[[bin]] +name = "settled-reach-test-client" +path = "src/bin/test_client.rs" +``` + +**Rationale:** Shares bridge types (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed`) with zero duplication. Rust compiles each binary independently — no server bloat. A separate crate would need to depend on the server crate anyway (or require premature extraction of a `protocol` crate). + +### CLI Interface + +``` +settled-reach-test-client [OPTIONS] + +Connection: + --connect Connect to running server (default: 127.0.0.1:9876) + +Input: + --replay Send inputs from file (one JSON PlayerInput per line) + --interactive Read inputs from stdin (future) + +Output: + --text Render each snapshot as structured text to stdout + --json Dump each snapshot as JSON (for golden files) + --quiet No output (assertions only, for CI) + +Assertions: + --golden Compare final snapshot against golden file, exit 1 on diff + --ticks Disconnect after N ticks (default: unlimited) +``` + +### Text Output Format + +``` +=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === +Game time: Day 0, 04:12 (Morning) +Entities (5): + npc:100 (18,10) Forward rel:Neutral vis:Visible + npc:101 (20,10) Forward rel:Unknown vis:Remembered + obj:200 (16,9) Forward rel:n/a vis:Visible +Tiles: 31 visible +Interactions (2): + npc:100 [Talk(1), ExamineNpc(2)] distance=3 +Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] +Monologue: "Something about this manifest doesn't add up." +=== +``` + +Entities labeled as `kind:entity_id` (e.g., `npc:100`). No display names on wire — test assertions use Gauntlet constants module: `assert_entity_visible(&snapshot, GUARD_1.wire_id, GUARD_1.position)`. + +### Text Renderer Lives in Library + +`server/src/bridge/text_renderer.rs` — library code callable by the test client binary AND integration tests. The server binary never references it. + +### Ozzie's Enhanced Terminal Layout + +Ozzie proposed a richer live-updating terminal display for human testers (crossterm-based, ANSI escape codes), adding: +- Sound events section (D-018 three-range model) +- Cognition section (cognitive delay progress) +- Checklist progress bar +- Hotkey reminders (F12: WRONG, Home: Hub, R: Reset) +- Session timer +- Symbols per visibility state: `●` VISIBLE, `◐` REMEMBERED, `◌` FOGGED, `✕` BLOCKED, `⚡` RECOGNIZING + +### Missing ObserverSnapshot Fields (Ozzie) + +4 fields needed for the full test client experience, gated behind `--debug`/test-mode: +1. `blocked_entities: Vec` — entities not visible with blocking wall position +2. Entity display names (or resolve via Gauntlet constants — Tyre recommends the latter) +3. Fog layer counts per type (5-layer breakdown) +4. `Vec` — source position, type, range classification + +### Effort Estimate + +~2-3 days total: binary scaffolding (0.5d), text renderer (0.5-1d), replay loading (0.5d), golden file comparison (0.5d), Layer 3 wiring (0.5d). + +--- + +## 2. Determinism Fixes — Concrete Code Changes (Dudley) + +Dudley produced exact code changes for all 4 fixes identified in Round 1. Total: ~40 lines changed across 4 files (revised up from Round 1's estimate of ~15 lines due to full context being included). + +### Fix A: BTreeSet for visible_ids + sort visible_tiles + +**Files:** `server/src/perception/query.rs`, `server/src/perception/observer/mod.rs` + +Changes: +- `visible_positions: HashSet<(i32, i32)>` → `BTreeSet<(i32, i32)>` in `VisibilityGeometry` +- `visible_ids: HashSet` → `BTreeSet` in `filter_visible_entities` +- Add `visible_tiles.sort_by_key(|t| (t.x, t.y))` after collection in `NaturalVision::compute_geometry()` +- Update `collect_remembered_entities` signature to accept `&BTreeSet` references + +`sector_lookup` HashMap stays — it's point-lookup only via `.get()`, never iterated. + +### Fix B: Sort visible entities in snapshot + +**File:** `server/src/perception/observer/mod.rs`, in `compute_observer_snapshot()` + +Add after `collect_remembered_entities`: +```rust +entities.sort_by_key(|e| e.entity_id); +``` + +### Fix C: Pin monologue system ordering — ALREADY DONE + +**Round 1 self-correction:** Dudley stated monologue systems lacked explicit ordering. After re-reading `bridge/mod.rs:168-175`, confirmed the ordering is already explicit via `.after()` constraints. **No change needed.** + +### Fix D: Sort movers in validate_movement + +**File:** `server/src/simulation/movement.rs` + +Collect movers into Vec, sort by `entity.to_bits()`, then process: +```rust +let mut mover_list: Vec<_> = movers.iter_mut().collect(); +mover_list.sort_by_key(|(entity, _, _, _)| entity.to_bits()); +``` + +Note: `entity.to_bits()` provides stable ordering within a single run. For cross-session determinism (save/load), would need `registry.to_stable(entity)` — deferred to Sprint 10+. + +### Determinism Fix Coverage (Hoshe cross-review) + +Hoshe assessed that **none of the 4 fixes have direct test coverage today**. Each fix should ship with its own regression test: +- Fix A: Test observer with 2 equidistant contradicted NPCs, assert same one selected +- Fix A (tiles): Generate snapshot with random-order tiles, assert output sorted +- Fix B: Not identified separately — covered by golden file tests +- Fix D: Two entities at same distance to same tile, assert deterministic winner + +--- + +## 3. Server `--test-mode` and `--port 0` Design (Dudley) + +### `--test-mode` Flag + +| Aspect | Behavior | +|--------|----------| +| Content | Loads Gauntlet content pack (`content/gauntlet/`), falls back to proof room | +| Seed | Fixed seed 42 | +| Stdout | Prints `LISTENING:{port}` after TCP bind, before accept | +| Shutdown | Exits after first client disconnect | +| Logging | Defaults to `warn` (override with `RUST_LOG`) | + +### `--port 0` Support + +Uses existing `TcpBridge::accept_on(listener)` (already in `tcp.rs:67`). The bind/accept split was added for test race condition prevention — exactly what `--port 0` needs. No TcpBridge changes required. + +Port discovery protocol: server prints `LISTENING:{port}` to stdout, test client parses it. + +--- + +## 4. Cross-Review Results + +### 4.1 Hoshe validates Dudley's server proposals + +**Pause guard:** All 6 of Dudley's proposed tests are genuine gaps. The pause guard has **ZERO direct test coverage today**. The existing test (`process_input_pause_sets_paused`) tests the Pause action side effect, not the guard itself. If someone deleted lines 102-105 (the guard), all existing tests would still pass. + +Hoshe found 3 additional gaps: +- `set_tick_rate_while_paused` (P2) — SetTickRate(Half) while paused unconditionally sets rate, making `paused()` return false +- `perception_mode_while_paused` (P2) — currently no-op but should pass through guard +- `interact_take_while_paused` (P2) — documents expected behavior for Take during pause + +**EntityRegistry:** 3 of 5 Dudley tests are genuine gaps. 2 are not applicable (concurrent access impossible in bevy, bulk performance is fine at 2000 entries). Hoshe found 2 additional gaps: +- `unregister_unknown_entity_is_noop` (P1) — code handles it but no test +- `register_with_pre_existing_stable_id_component` (P2) — component/registry divergence + +**Bridge deserialization:** Hoshe recommends keeping batch-failure (current behavior) with a test documenting it: `malformed_input_in_batch_rejects_entire_batch`. + +### 4.2 Tyre validates Stig's client proposals + +All 32 tests **approved**. No redundancies. Tyre adds 6 more (38 total): + +| # | Test | Category | Rationale | +|---|------|----------|-----------| +| 33 | Pending recognition blob rendering | Entity rendering | D-060 cognitive delay visual | +| 34 | Recognition transition animation | Entity lerp | Blob → full entity over ~0.3s | +| 35 | Tick rate HUD indicator | UI elements | Full/Half/Paused display | +| 36 | Inventory full visual state | UI elements | 9/9 feedback | +| 37 | Sprint interaction suppression | UI elements | D-055 interaction buffer empty during sprint | +| 38 | Entity modulate for Remembered | Entity rendering | Remembered vs Visible visual difference | + +**Priority ranking:** P0 = monologue not lost on overwrite (Bug #5), camera static during pause (Bug #2). P1 = fog shader state (3 tests), entity lifecycle, pending recognition blob. + +**Refinements:** +- Camera rapid snapshots test should verify convergence, not just "doesn't crash" +- Entity lifecycle test should verify node freed (not just hidden) — memory leak prevention + +### 4.3 Tyre validates Hoshe's CI tiers + +**Approved** with refinements: +- PR tier budget revised: <15 min (not <10 min) — clean cache server builds take 5-7 min +- Add content cross-reference validation to PR tier (<2s, negligible budget impact) +- Add content scaling stress test to Nightly tier +- **Fixture staleness should be BLOCKER, not WARNING** — stale fixtures make all client tests false positives + +### 4.4 Dudley validates Hoshe's Layer 3 test + +**Feasible** with 2 minor adjustments: +1. Use `rmp_serde::to_vec` (not `to_vec_named`) for input serialization — matches GDScript's encoding +2. Tolerate initial tick=0 snapshot before input is processed — server may send snapshot before reading client input + +Hoshe's assertions on tick=0 snapshot are fine — they verify snapshot delivery, not input processing. + +--- + +## 5. Resolved Open Questions + +| ID | Question | Answer | Answered By | +|----|----------|--------|-------------| +| **OQ-01** | Does client depend on visible_tiles ordering? | **No.** All consumers are position-indexed (Dict keyed by Vector2i, set_cell() idempotent). Fix #2 safe to ship. Two fixture tests reference `visible_tiles[0]` by index — flag for fixture regeneration. | Stig | +| **OQ-02** | Does rmp_serde accept int_16 for u64? | **Yes.** Traced through rmp-serde 1.3.1: `Marker::I16 → visit_i16 → visit_i64 → u64::try_from`. GDScript encoding 256 as int_16 deserializes correctly. Negative values correctly rejected. | Dudley | +| **OQ-04** | WalkabilityMap HashMap → BTreeMap? | **No. Leave as HashMap.** Point-lookup only, never iterated. HashMap is faster in the hot path. Document as known-safe usage. | Tyre | +| **OQ-05** | Fixture staleness: git diff robust enough? | **Yes.** `git diff --exit-code` detects any byte-level change. Content-addressed hashing adds complexity for zero additional safety. Fixture generation is deterministic (verified — no timestamps/random values). | Justine | +| **OQ-06** | = OQ-02 (rmp_serde int_16→u64) | See OQ-02 | Dudley | +| **OQ-07** | Server --test-mode and --port 0? | **Designed.** See Section 3. Uses existing `accept_on()`. `LISTENING:{port}` signal to stdout. | Dudley | +| **OQ-08** | Room name from coordinates? | **Gauntlet coordinate bounds constants** in `server/src/test_world/constants.rs`. `room_at(player_pos)` returns room name. No server API needed. | Dudley | +| **OQ-10** | Fog byte value constants? | **Yes, promote to named constants.** `VIS_HIDDEN=0`, `VIS_PERIPHERAL=180`, `VIS_FORWARD=255`, `EXP_UNEXPLORED=0`, `EXP_EXPLORED=128`, `EXP_VISIBLE=255`. Replace magic numbers in `fog_state.gd`. | Stig | +| **OQ-12** | = OQ-04 (WalkabilityMap HashMap) | See OQ-04 | Tyre | + +### Partially Resolved + +| ID | Question | Status | +|----|----------|--------| +| **OQ-03** | Gitea Actions available? | Moot — lead confirmed no CI for now. Tyre notes: ~1 day effort when greenlighted. | +| **OQ-05** | Entity display_name on wire? | **Deferred.** Tyre: use `kind:entity_id` labels now. Add `display_name: Option` when client needs name labels (Sprint 9-10 gameplay feature). | +| **OQ-09** | Text renderer from ObserverSnapshot? | **Yes**, confirmed. The text renderer formats ObserverSnapshot. Lives in server library, called by test client binary. | +| **OQ-11** | Client tests headless? | **Unresolved.** No agent tested this in Round 2. Remains a concern for CI. | + +### Resolved Unresolved Architectural Questions (from Round 1) + +| ID | Question | Resolution | +|----|----------|-----------| +| **UQ-01** | Bridge deserialization: skip-and-log vs batch failure? | **Keep batch failure.** Both sides are co-versioned (D-020). Add test documenting behavior: `malformed_input_in_batch_rejects_entire_batch`. (Hoshe) | +| **UQ-02** | Gauntlet geometry: YAML vs Rust? | **Not directly addressed in Round 2.** Dudley's code uses Rust constants for room bounds. Gestalt references YAML room definitions for checklists. The hybrid approach (WS-D2) remains — specific geometry format deferred to implementation. | +| **UQ-03** | Wall-blocking-LOS info? | **Optional `blocked_entities` field** gated behind `--debug`/test-mode. Ozzie specifies as a requirement for the test client's `✕ BLOCKED` display. Feasibility question to Dudley in Round 3. | +| **UQ-04** | Fixture path fragility? | **Addressed by `make pre-pr` fixture staleness check.** Regenerate + diff catches stale fixtures regardless of path. | + +--- + +## 6. Content Cross-Reference Validation + +Both Hoshe and Justine independently specified content validation extensions. Their proposals are compatible and complementary. + +### Hoshe's 8 Checks + +| Check | Severity | What | Target | +|-------|----------|------|--------| +| 1 | ERROR | `canonical_id` uniqueness across all NPCs | `npcs/*.yaml` | +| 2 | ERROR | Relationship `target` resolves to defined NPC | `npcs/*.yaml → relationships[].target` | +| 3 | ERROR | District `locations[]` slug matches location file | `district.yaml → locations/` | +| 4 | ERROR | Dialogue pool `location` matches district | `dialogue/**/*.yaml → location` | +| 5 | ERROR | `knowledge_grant.fact_id` validity | `dialogue/**/*.yaml → lines[]` | +| 6 | ERROR | NPC `triangle_membership` matches triangle file | `npcs/*.yaml` | +| 7 | WARNING | `npc_count` matches actual NPC file count | `district.yaml` | +| 8 | ERROR | Dialogue line ID uniqueness within pool | `dialogue/**/*.yaml → lines[].id` | + +### Justine's 7 Checks + +| Check | Severity | What | +|-------|----------|------| +| 1 | ERROR | NPC relationship targets resolve | +| 2 | ERROR | Triangle members resolve | +| 3 | ERROR | NPC triangle_membership matches triangles | +| 4 | ERROR | Dialogue pool location resolves | +| 5 | ERROR | Fact IDs resolve (absorb `check-fact-ids`) | +| 6 | ERROR | District location list matches files | +| 7 | WARNING | Bidirectional relationship consistency | + +### Merged View + +Hoshe and Justine agree on checks 1-6. Justine adds bidirectional relationship warnings (Check 7). Hoshe adds dialogue line ID uniqueness (Check 8) and `npc_count` accuracy (Check 7). Combined: **9 unique checks** (7 ERROR, 2 WARNING). + +### Architecture Agreement + +Both recommend extending `tooling/validate-content` (Python) with a second pass after schema validation. No Rust dependency — content authors validate without compiling the server. Phased rollout: NPC/triangle/district first, dialogue/monologue second, warnings third. + +--- + +## 7. `make pre-pr` Target + +Both Hoshe and Justine independently designed this target. Their proposals are compatible. + +### Agreed Chain + +``` +pre-pr + ├── 1. lint-server + lint-client (~15s) + ├── 2. build-server + build-client (~30-90s) + ├── 3. test-server + test-client (~15-30s) + ├── 4. validate-content (~2-5s) + ├── 5. check-fact-ids (~2s) + └── 6. fixture staleness check (~10-15s) +``` + +**Total: ~90-180s** (under 3 minutes for clean incremental build). Fast enough for every PR. + +### Branch-Specific Variants (Hoshe) + +- `make pre-pr-server` — lint-server, build-server, test-server, fixtures +- `make pre-pr-client` — lint-client, build-client, test-client +- `make pre-pr-content` — validate-content, check-fact-ids + +### Fixture Staleness Check + +```makefile +fixtures-check: fixtures + @if git diff --quiet client/tests/fixtures/; then \ + echo "Fixtures: up to date"; \ + else \ + echo "FIXTURES STALE"; exit 1; \ + fi +``` + +--- + +## 8. Anti-Tedium Suite — Full Specifications + +All features approved by lead. Four agents contributed implementation specs. + +### 8.1 Room Reset Trigger + +| Aspect | Spec | Source | +|--------|------|--------| +| Trigger | Player steps on ResetPlate tile + presses Interact (not automatic) | Gestalt, Ozzie agree | +| Server mechanism | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system | Dudley | +| What resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles only), inventory items from room, dialogue state | Gestalt | +| What does NOT reset | Other rooms, player position (stays on plate), session timer, other-room checklist progress | Ozzie, Gestalt | +| Edge case | Items carried from room returned to tick-0 position, terminal shows "Items returned: keycard → crate_1" | Ozzie | +| Client visual | Distinct tile type (`reset_plate`), amber outline, interaction verb "Reset Room", 0.15s amber flash + monologue "Systems recalibrated." | Stig | +| Debounce | 10-tick cooldown prevents re-trigger while walking across plate | Dudley | +| Test mode only | `RoomResetTrigger` entities only added with `--test-mode` | Dudley | + +### 8.2 Hub Teleport + +| Aspect | Spec | Source | +|--------|------|--------| +| Hotkey | `Home` key | Ozzie, Stig agree | +| Wire format | `PlayerAction::TeleportToHub` | Gestalt | +| Server behavior | Move player entity to `GAUNTLET.hub_center`, clear dialogue/monologue/interaction buffer | Dudley, Ozzie | +| Does NOT affect | Room state, inventory, game time, knowledge graph | Ozzie, Gestalt | +| Client visual | Instant camera snap, 0.3s fade-to-black-and-back, no monologue (meta action) | Stig | +| Gauntlet-only | Server rejects `TeleportToHub` in non-Gauntlet maps | Gestalt | + +### 8.3 WRONG Button (F12) + +| Aspect | Spec | Source | +|--------|------|--------| +| MVP captures (Sprint 8) | ObserverSnapshot, tick + position, text render output, human description | Gestalt | +| Full captures (Sprint 9+) | + input history (60 ticks), snapshot history (60 ticks), world digest, replay seed | Ozzie | +| Output directory | `tests/bug-reports/gauntlet-{tick}-{timestamp}/` | Gestalt, Ozzie | +| Bug report format | `report.md` (human-readable), `snapshot.json`, `text_output.txt`, `description.txt` | Ozzie | +| Client implementation | `BugReportCapture` autoload, F12 hotkey in `_unhandled_input()`, modal prompt, 6 data captures | Stig | +| Ring buffer | 60 ticks (configurable via `--history-buffer`) | Ozzie | + +### 8.4 Room Timer + Personal Bests + +| Aspect | Spec | Source | +|--------|------|--------| +| Display | `TIMER: 00:47 (PB: 00:38)` in status bar/overlay | Ozzie | +| Start | Player enters room (crosses bounding box) | Ozzie | +| Reset | Room reset trigger resets timer | Ozzie | +| Persistence | `tests/gauntlet-stats.json` — local, not committed | Ozzie | +| Session summary | Printed on disconnect: rooms visited, coverage %, times, PBs, bug reports filed | Ozzie | + +### 8.5 Auto-Checklist Progress (Ozzie + Stig merged) + +Ozzie identified 4 gaps in Stig's per-room checklist proposal and proposed a merged format: + +| Gap | Stig's Proposal | Ozzie's Addition | +|-----|----------------|-----------------| +| No machine-readable conditions | Human prose only | Add `condition:` field (structured data) for auto-tracking | +| No auto vs manual distinction | All items equal | Add `type: auto/manual` — auto-confirms from snapshot, manual requires tester input | +| No cross-room items | Per-room only | Add `cross_room_checks.yaml` at Gauntlet root | +| No failure guidance | Just "check X" | Add `if_wrong:` field with likely causes + file references | + +**Merged checklist YAML format** (Ozzie): +```yaml +checks: + - id: occ_01_hidden_not_visible + description: "NPC behind wall is NOT visible" + type: auto + condition: + player_near: [15, 10] + entity: hidden-1 + expected: blocked + if_wrong: | + LOS leaking through wall. Check shadowcast.rs. +``` + +`make checklist` generates markdown from YAML (Stig's `tooling/gen_checklist.py`). Test client loads structured conditions for auto-tracking. + +### 8.6 Stig's Client-Side Anti-Tedium UI + +- **Room reset:** Tile type `reset_plate` in TileRenderer, interaction verb "Reset Room", amber flash +- **Hub teleport:** `TELEPORT_HUB` in InputMapper → `Home` key, fade transition +- **WRONG button:** New `bug_report.gd` autoload, 60-entry `input_history` ring buffer +- **Progress overlay:** Top-right panel with room name, run counter, timer, checklist progress bar. Only visible when `gauntlet_mode == true`. + +### 8.7 Deferred: F3 Debug Overlay + +**Stig recommends deferring** the F3 State Inspector Overlay. The WRONG button captures the same data on demand. F3 as a real-time overlay requires per-frame string formatting of the entire ObserverSnapshot — measurable performance cost. Ship WRONG button first, F3 if testers ask for it. + +--- + +## 9. Gauntlet Room Coverage (Gestalt) + +### Final Room List: 14 Rooms + +| # | Room | Primary Systems | Source | +|---|------|----------------|--------| +| 1 | Inventory Warehouse | Pickup, CarriedBy, 9-slot limit | Brief | +| 2 | Occlusion Corridor | LOS, shadowcasting, perception modes | Brief | +| 3 | Interaction Gallery | ObjectType verbs, sprint suppression | Brief | +| 4 | Crowd Plaza | Entity density, relationship colors, cognitive delay | Brief | +| 5 | Fog Theater | Fog transitions, peripheral dimming, exploration persistence | Brief | +| 6 | Dialogue Room | Trust tiers, contradiction, walk-away, monologue during dialogue | Brief | +| 7 | Pause Chamber | TickRate toggle, state transitions | Brief | +| 8 | Zone Gate | Zone transition (reserved, future contract) | Brief | +| 9 | Eavesdrop Alcove | ListeningFocus, zone ambient, Careful stance | Gestalt R1 | +| 10 | Confrontation Stage | Cognitive vulnerability, audio dip | Gestalt R1 | +| 11 | Sprint Gauntlet | Sprint suppression, anomaly survival, stance transitions | Gestalt R1 | +| 12 | Sound Lab | Three-range sound, sound pings, recognition chime | Gestalt R1 | +| 13 | Decay Observatory | Knowledge decay, stale state, fog degradation | Gestalt R1 | +| 14 | Shift Change | Stress test: all systems at density | Gestalt R1 | + +### Coverage Matrix + +Gestalt mapped 50+ system-to-room relationships across 6 pillars: Characters & Information, Perception, Movement & Interaction, Audio, Simulation & Architecture, Content Systems. Every system from confirmed decisions has at least one room exercising it. + +**Coverage gaps (3 minor, all addressable without new rooms):** +1. Object-layer favorite colors (D-052) — add to Inventory Warehouse in v0.1.2+ +2. POI navigation (D-013) — add to Hub as POI markers. v0.1 stretch. +3. Environmental neutrality (D-045) — assertion on Dialogue Room (pre/post confrontation CanvasModulate identical) + +### 8 Transition Scenarios + +All scenarios reference physically connected rooms via hub paths or cross-cuts: + +| # | Scenario | Path | Key Test | +|---|----------|------|----------| +| T1 | Sprint Exit | Plaza → Occlusion Corridor | Buffer cleared during sprint, LOS recalculated | +| T2 | Fog into Dialogue | Fog Theater → Dialogue Room | Fog state preserved during dialogue, monologue above dialogue box | +| T3 | Full Inventory Interact | Inventory → Interaction Gallery | 9/9 inventory, Take still offered server-side, client greys out | +| T4 | Sprint into Interaction | Sprint Gauntlet → Interaction Gallery | Verbs repopulate within 1 tick after stance change | +| T5 | Confrontation to Eavesdrop | Confrontation → Eavesdrop | Audio dip release + ListeningFocus activation don't conflict | +| T6 | Pause Anywhere | Pause Chamber → Hub → any room | Pause-during-transition state corruption | +| T7 | Sound across Fog | Sound Lab → Fog Theater | Sound propagation through walls, cognitive delay from sound | +| T8 | Walk-away Sprint | Dialogue → Hub → Sprint Gauntlet | KG incompleteness recorded, sprint suppresses post-dialogue monologue | + +### Top 10 Invariants for Sprint 8 (Gestalt) + +| Rank | ID | Invariant | Bug Match | +|------|-----|-----------|-----------| +| 1 | INV-T04 | Pause coherence | Bug #3 | +| 2 | INV-T01 | Deterministic replay | Bug #1 class | +| 3 | INV-T03 | Snapshot delivery | Bug #1, #5 | +| 4 | INV-T05 | Input ordering | Bug #1 | +| 5 | INV-S01 | Player spawn reachable | Softlock prevention | +| 6 | INV-C03 | StableId uniqueness | Corruption prevention | +| 7 | INV-C07 | Dialogue pool non-empty | Player-facing failure | +| 8 | INV-T02 | Tick budget | Bug #6 class | +| 9 | INV-P02 | LOS symmetry | D-035 mandate | +| 10 | INV-S05 | No entity inside geometry | Content scaling safety | + +**Bug catalogue mapping: every Sprint 6-7 bug is now covered** by at least one invariant + room + test type. + +--- + +## 10. Encoding Asymmetry Tests (Hoshe) + +Hoshe specified 4-direction cross-language testing for the encoding asymmetry between GDScript (int_16 for 256-32767) and Rust (uint_16): + +| Direction | What | Test Location | When | +|-----------|------|---------------|------| +| Rust → GDScript (fixture) | Snapshots with overlap-zone ticks | `test_msgpack_boundaries.gd` | Every PR | +| GDScript → Rust (fixture) | Inputs with overlap-zone ticks | `serialization.rs` | Every PR | +| Rust → GDScript (raw bytes) | Hand-crafted uint_16/uint_32 bytes | `test_msgpack_boundaries.gd` | Every commit | +| GDScript → Rust (raw bytes) | Hand-crafted int_16/int_32 bytes | `serialization.rs` | Every commit | + +New `make fixtures-client` target generates GDScript-encoded fixtures for Rust to verify. Reverse direction of existing `make fixtures`. + +--- + +## 11. Performance & Golden File Tooling (Justine) + +### Performance Baseline + +- `tests/perf/baseline.json` — committed, records median/min/max from 5 runs +- `tooling/perf-measure` — builds release, runs benchmarks, compares against baseline +- Thresholds: <15% = PASS, 15-30% = WARNING, >30% = FAIL +- Machine tag prevents meaningless cross-machine comparisons +- `make perf-baseline` (compare) and `make perf-baseline-update` (update) + +### Golden File Diff + +- **Rust test, not separate tool** — golden file is an ObserverSnapshot, Rust code knows the structure +- JSON format with sorted keys, pretty-printed (`serde_json::to_string_pretty`) +- Field-by-field diff output on failure: changed fields, POSITION markers, added/removed entities +- `make golden-diff` (view diff) and `make golden-update` (regenerate) +- Why JSON not MessagePack: human-readable in `git diff`, sorted keys = deterministic output + +--- + +## 12. Client Test Suite — Final Count + +| Category | Round 1 (Stig) | Round 2 Additions | Total | +|----------|---------------|-------------------|-------| +| Camera system | 7 | — | 7 | +| Entity rendering | 7 | +2 (Tyre: blob, remembered modulate) | 9 | +| Z-layer ordering | 4 | — | 4 | +| Fog shader state | 3 | +1 (Stig: hidden state) | 4 | +| UI elements | 8 | +3 (Tyre: tick rate, inventory full, sprint suppression) | 11 | +| Entity lerp | 3 | +1 (Tyre: recognition transition) | 4 | +| Anti-tedium | — | +2 (Stig: bug report capture, progress hidden) | 2 | +| **Total** | **32** | **+9** (6 Tyre + 3 Stig) | **~38-41** | + +Note: Stig counts 35 (32+3), Tyre counts 38 (32+6). Combined unique total depends on overlap — upper bound is 41. + +--- + +## 13. Open Questions for Round 3 + +### New Questions (raised in Round 2) + +| ID | From | To | Question | +|----|------|----|----------| +| R2-OQ-01 | Hoshe | Dudley | `SetTickRate(Half)` while paused — should this unpause? Current code sets rate unconditionally (input.rs:165-168). Intentional or bug? | +| R2-OQ-02 | Hoshe | Dudley | Entity respawn + registry — when bevy recycles Entity index, does registry handle old StableId not being unregistered? | +| R2-OQ-03 | Hoshe | Tyre | `make pre-pr` — should it include `make content-ron` (YAML→RON conversion)? | +| R2-OQ-04 | Hoshe | Justine | Fixture staleness in `make pre-pr` — separate `make pre-pr-full` to keep basic pre-PR fast? | +| R2-OQ-05 | Ozzie | Dudley | `blocked_entities` debug field on ObserverSnapshot — feasible? Cost per tick? | +| R2-OQ-06 | Ozzie | Stig | Can Godot client render checklist progress overlay, or test-client-only? | +| R2-OQ-07 | Ozzie | Tyre | Test client binary location: `server/src/bin/` or separate `tools/` crate? | +| R2-OQ-08 | Ozzie | Gestalt | Cross-room transition scenarios: where in YAML hierarchy? Own checklist section? | +| R2-OQ-09 | Gestalt | All | Room ordering in Gauntlet YAML — canonical ordering affects entity StableId assignment. | +| R2-OQ-10 | Gestalt | All | Per-room reset sufficient, or need "full server restart" command? | +| R2-OQ-11 | Gestalt | All | 4 new cross-cuts — too many? Consolidate Sound Lab into Occlusion Corridor sub-area? | + +### Remaining Unresolved from Round 1 + +| ID | Question | Status | +|----|----------|--------| +| OQ-11 | Client tests headless stability | Unresolved — no agent tested in Round 2 | + +--- + +## 14. Points of Agreement (consensus) + +1. **Test client binary is the right call.** Tyre accepted the overrule and designed it properly. The binary shares types, exercises real TCP, and enables Layer 3 testing. +2. **Determinism fixes are small and well-understood.** Dudley's code changes are concrete. Hoshe's coverage assessment confirms no existing tests break. +3. **Pause guard has zero coverage.** Hoshe independently confirmed Dudley's gap analysis. All 6 tests are genuine needs. +4. **`make pre-pr` replaces CI discipline.** Both Hoshe and Justine converged on the same chain (lint → build → test → validate → fixtures). +5. **Content cross-reference validation extends Python validator.** Both Hoshe and Justine agree: no Rust dependency, two-pass architecture (schema then cross-refs). +6. **Anti-tedium is fully specified.** Room reset (Interact-triggered), hub teleport (Home key), WRONG button (F12 + MVP captures). No dissent on any feature. +7. **38 client tests are architecturally sound.** Tyre's cross-review found no redundancies in Stig's 32 and added 6 meaningful tests. +8. **Bug catalogue fully covered.** Gestalt's invariant mapping confirms every Sprint 6-7 bug class has a test + room + invariant. +9. **Checklist format merges Stig + Ozzie proposals.** YAML with structured conditions for auto-tracking AND human prose for markdown generation. + +## 15. Points of Tension + +1. **Fixture staleness: BLOCKER vs WARNING.** Tyre argues BLOCKER (stale fixtures = false positive client tests). Justine tagged it WARNING. Recommend resolving in Round 3 — Tyre's argument appears stronger. +2. **F3 debug overlay: defer or implement.** Ozzie wants it for the testing experience. Stig says defer due to per-frame performance cost. WRONG button captures same data on demand. Lean toward deferral. +3. **PR tier time budget: <10 min (Hoshe) vs <15 min (Tyre).** Tyre accounts for clean-cache builds. Recommend <15 min as the budget (covers worst case). +4. **Cross-room checklist location.** Ozzie identifies a real gap: cross-room transition scenarios (Gestalt's T1-T8) don't fit in per-room YAML. Needs `cross_room_checks.yaml` or equivalent. +5. **Number of Gauntlet cross-cuts.** Gestalt proposes 4 new cross-cuts. Potential scope concern. Could consolidate Sound Lab into Occlusion Corridor sub-area. + +--- + +## 16. Gaps Remaining + +1. **Test client binary doesn't exist yet.** Tyre estimated 2-3 days. Depends on `--test-mode` + `--port 0` (Dudley, Sprint 8). +2. **Gauntlet content doesn't exist yet.** Room YAML, entity placement, checklist definitions all need writing. No specific agent assigned. +3. **Client test headless stability unverified.** OQ-11 from Round 1 remains open. Must verify before making client tests a merge gate. +4. **Cross-room checklist format unresolved.** Ozzie identified the gap. Needs design in Round 3. +5. **Gauntlet room ordering (entity spawn order → StableId assignment).** Gestalt raised this. Affects determinism — canonical YAML ordering needed. +6. **Sound events not in ObserverSnapshot.** Ozzie requires `Vec` for full test client display. Not yet designed on server side. +7. **Cognitive delay visual not yet implemented.** Tyre's tests #33-34 (blob rendering, recognition transition) depend on D-060 implementation. diff --git a/docs/workshops/test-architecture/round-3-notes.md b/docs/workshops/test-architecture/round-3-notes.md new file mode 100644 index 000000000..b2de3fb96 --- /dev/null +++ b/docs/workshops/test-architecture/round-3-notes.md @@ -0,0 +1,590 @@ +# QA Strategy & Test Architecture Workshop — Round 3 Notes + +**Workshop:** QA Strategy & Test Architecture +**Round:** 3 (Prioritization & Final Specs) +**Date:** 2026-02-17 +**Documenter:** Qatux +**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie + +--- + +## Executive Summary + +Round 3 produced build-ready specifications across all tracks. Every agent delivered implementation-level detail: Dudley's determinism patches are copy-pasteable, Justine's Makefile targets are ready for `make`, Stig's 38 client tests have function names and assertion logic, Hoshe's 59-item prioritized backlog has effort estimates and dependency chains, Tyre's Sprint 8 plan has a critical path and dependency graph, Gestalt's Gauntlet map has 48 entities with coordinate positions, and Ozzie's human tester walkthrough covers start-to-finish with terminal mockups. + +**Key outcomes:** +- **Sprint 8 scope:** 10 implementation items, ~7-8 team-days. Infrastructure only — no Gauntlet rooms, no anti-tedium features. Critical path: `--test-mode` (0.5d) -> test client binary (2-3d) -> Layer 3 test (0.5d). +- **Gauntlet map:** 7 rooms + Central Hub, 48 entities, 2 cross-cuts, hub-and-spoke topology. Designed for Sprint 8, room content built Sprint 9+. +- **All Round 2 open questions resolved.** 11 questions answered: SetTickRate while paused = bug (reject), entity index recycling = safe (generation counter), canonical room ordering = required, blocked_entities = feasible Sprint 9, test client at `tooling/test-client/` (lead override), 0 cross-cuts Sprint 8 (infrastructure only), content-ron in pre-pr = yes, fixed spawn order in setup function, kill+relaunch for full restart. +- **One lead override during Round 3:** Test client binary moved from `server/src/bin/test_client.rs` to `tooling/test-client/` as a separate workspace crate. Dudley verified bridge types are already pub-exported — no server changes needed. + +**Scope tension resolved:** Tyre scopes Sprint 8 as infrastructure-only. Dudley designs a 5-room Gauntlet loader MVP. Gestalt designs the full 7+Hub map. These are compatible: Tyre's Sprint 8 ships the pipes (flags, binary, test harness), Gestalt's map is the design document, Dudley's loader is the Sprint 9 implementation target. The map design happens now; room building happens Sprint 9. + +--- + +## 1. Sprint 8 Implementation Plan (Tyre) + +### Dependency Graph + +``` +S8-1: Determinism fixes ────────────────────────────┐ + ├─> S8-5: make pre-pr +S8-2: Content cross-ref validation ─────────────────┤ + │ +S8-3: --test-mode + --port 0 ──> S8-4: Test client ──┤ + binary MVP │ + │ │ + └─> S8-6: Layer 3 test wiring + │ +S8-7: Pause guard tests ────────────────────────────┘ (parallel, no deps) +S8-8: Determinism regression tests ── (after S8-1) +S8-9: EntityRegistry lifecycle tests ── (parallel, no deps) +S8-10: Fixture staleness check ── (after S8-5) +``` + +### Items + +| # | Item | Owner | Effort | Depends On | +|---|------|-------|--------|------------| +| S8-1 | Determinism fixes (A, B, D) | Dudley | 0.5d | Nothing | +| S8-2 | Content cross-reference validation (9 checks) | Justine | 1d | Nothing | +| S8-3 | `--test-mode` + `--port 0` server flags | Dudley | 0.5d | Nothing | +| S8-4 | Test client binary MVP | Dudley | 2-3d | S8-3 | +| S8-5 | `make pre-pr` chain | Justine | 0.5d | S8-1, S8-2 | +| S8-6 | Layer 3 test wiring | Dudley | 0.5d | S8-3, S8-4 | +| S8-7 | Pause guard tests (6 tests) | Dudley | 0.5d | Nothing | +| S8-8 | Determinism regression tests | Dudley | 0.5d | S8-1 | +| S8-9 | EntityRegistry lifecycle tests (3 tests) | Dudley | 0.25d | Nothing | +| S8-10 | Fixture staleness check | Justine | 0.25d | S8-5 | + +**Critical path:** S8-3 (0.5d) -> S8-4 (2-3d) -> S8-6 (0.5d) = 3-4 days. +**Total:** Server (Dudley) ~5-6d, Tooling (Justine) ~1.75d = ~7-8 team-days. + +### What Explicitly Does NOT Ship Sprint 8 + +- Gauntlet room content (rooms 1-14) +- Anti-tedium features (room reset, hub teleport, WRONG button) +- Crossterm live terminal display +- CI automation (Gitea Actions) +- Performance baselines +- Encoding asymmetry cross-language tests +- Client tests (Stig's 38) +- Cross-room transition scenarios + +--- + +## 2. Determinism Fixes — Final Patch Specs (Dudley) + +Three fixes shipping Sprint 8. Fix C was already done (self-corrected in Round 2). Total: ~22 lines production code, 4 regression tests. + +### Fix A: BTreeSet for visible_positions + sort visible_tiles + +**Files:** `server/src/perception/query.rs`, `server/src/perception/observer/mod.rs` + +- `visible_positions: HashSet<(i32, i32)>` -> `BTreeSet<(i32, i32)>` in `VisibilityGeometry` +- `visible_ids: HashSet` -> `BTreeSet` in `filter_visible_entities` +- Add `visible_tiles.sort_by_key(|t| (t.x, t.y))` after collection +- Update `collect_remembered_entities` signature to accept `&BTreeSet` +- `sector_lookup: HashMap` stays as-is (point-lookup only, confirmed safe per OQ-4) + +**Regression tests:** `snapshot_visible_tiles_are_sorted`, `sprint_anomaly_picks_lowest_stable_id` + +### Fix B: Sort visible entities by entity_id + +**File:** `server/src/perception/observer/mod.rs` + +Insert `entities.sort_by_key(|e| e.entity_id);` after `collect_remembered_entities`. + +**Regression test:** `snapshot_entities_sorted_by_id` + +### Fix D: Sort movers in validate_movement + +**File:** `server/src/simulation/movement.rs` + +Collect movers into Vec, sort by `entity.to_bits()`, then process. Deterministic collision resolution — entity with lower `to_bits()` wins contested tiles. + +**Regression test:** `validate_movement_deterministic_collision_winner` + +--- + +## 3. Server `--test-mode` Final Spec (Dudley) + +Complete `main.rs` replacement provided. Key design: + +| Aspect | Behavior | +|--------|----------| +| Stdout | `LISTENING:{port}` after TCP bind (test client parses this) | +| Stderr | All tracing via `tracing_subscriber::fmt::layer().with_writer(std::io::stderr)` | +| Content | Gauntlet content pack (when ready), fallback to proof room | +| Seed | `--seed 42` default in test-mode | +| Shutdown | Exits after first client disconnect | +| Port | `--port 0` for OS-assigned port (uses existing `accept_on(listener)`) | + +`setup_proof_room()` extracted as reusable function for both modes. + +--- + +## 4. Test Client Binary Final Spec (Tyre + Dudley) + +### Lead Override: Crate Location + +**During Round 3, the lead overruled the Round 2 location.** Test client moves from `server/src/bin/test_client.rs` to `tooling/test-client/` as a separate workspace crate. + +**Note:** Tyre's Round 3 spec was written before this override and still references `server/src/bin/`. Dudley's addendum confirms the override and verifies all bridge types are already pub-exported — no server changes needed. + +### CLI (Sprint 8 MVP) + +``` +settled-reach-test-client [OPTIONS] + +Connection: + --connect (default: 127.0.0.1:9876) + +Input: + --replay JSONL input file (one JSON array per tick) + +Output (mutually exclusive): + --text Structured text to stdout (default) + --json JSON to stdout (for golden files) + --quiet No output (CI assertions only) + +Assertions: + --golden Compare final snapshot, exit 1 on diff + --ticks Disconnect after N ticks +``` + +### Text Output Format + +``` +=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === +Game time: Day 0, 04:12 (Morning) +Room: Occlusion Corridor +Entities (5): + npc:100 (18,10) Forward rel:Neutral vis:Visible d=3 + ... +Tiles: 31 visible +Interactions (2): + npc:100 [Talk(1), ExamineNpc(2)] distance=3 +Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] +Monologue: "Something about this manifest doesn't add up." +=== +``` + +Entity labels: `kind:entity_id`. Sorted by distance (nearest first), ties broken by entity_id. Sections with no data omitted. + +### Text Renderer + +`server/src/bridge/text_renderer.rs` — library function `format_snapshot_text()` callable by the test client crate and integration tests. Pub-exported from server crate. Full implementation (~100 lines) provided by Tyre. + +### Golden File Comparison + +JSON with sorted keys. Recursive `diff_json_values()` produces field-by-field diff on mismatch. Exit code 1 on diff. + +### Replay Format + +JSONL — one JSON array of `PlayerInput` per line. Empty array `[]` = idle tick. Uses `rmp_serde::to_vec` (NOT `to_vec_named`) to match GDScript encoding. + +### Effort Estimate + +~2-3 days: binary scaffolding (0.5d), text renderer (0.5-1d), replay loading (0.5d), golden file comparison (0.5d), Layer 3 wiring (0.5d). + +--- + +## 5. Gauntlet Map Specification (Gestalt) + +### MVP Room List: 7 rooms + Central Hub + +| # | Room | Size | Observer | Facing | Entities | +|---|------|------|----------|--------|----------| +| 0 | Central Hub | 24x24 | (50,58) | -- | 4 signs | +| 1 | Fog Theater | 44x32 | (56,18) | South | 4 (3 NPCs, 1 object) | +| 2 | Occlusion Corridor | 42x22 | (84,58) | East | 4 NPCs | +| 3 | Inventory Warehouse | 30x28 | (17,54) | East | 11 (10 objects, 1 NPC) | +| 4 | Interaction Gallery | 24x20 | (14,92) | East | 5 (2 NPCs, 3 objects) | +| 5 | Pause Chamber | 16x16 | (50,86) | North | 1 NPC | +| 6 | Dialogue Room | 28x20 | (50,114) | North | 4 NPCs | +| 7 | Crowd Plaza | 32x32 | (96,94) | West | 15 NPCs | + +**Total: 48 entities.** Map bounds: 0-116 x 0-124 sim tiles. + +### Topology + +Hub-and-spoke with 2 cross-cuts: +- **cross-cut-W:** Inventory Warehouse <-> Interaction Gallery (T3: Full Inventory Interact) +- **cross-cut-E:** Occlusion Corridor <-> Crowd Plaza (T1: Sprint Exit) + +7 corridors connecting rooms to hub and each other. All 6 tiles wide, 8-14 tiles long. + +### StableId Assignment Order + +Entities receive StableIds in canonical spawn order: +1. Hub entities (signs): StableId 1-4 +2. Fog Theater: StableId 5-8 +3. Occlusion Corridor: StableId 9-12 +4. Inventory Warehouse: StableId 13-23 +5. Interaction Gallery: StableId 24-28 +6. Pause Chamber: StableId 29 +7. Dialogue Room: StableId 30-33 +8. Crowd Plaza: StableId 34-48 + +**Additive-only rule:** Existing rooms and entities are NEVER reordered. New rooms and entities append. This preserves golden file stability. + +### Cross-Room Transitions (3 MVP) + +| # | Transition | Path | Systems Tested | +|---|-----------|------|---------------| +| T1 | Sprint Exit | Crowd Plaza -> cross-cut-E -> Occlusion | Sprint suppression + LOS recalculation | +| T3 | Full Inventory Interact | Inventory -> cross-cut-W -> Interaction Gallery | Inventory limit + verb computation | +| T6 | Pause Anywhere | Pause -> Hub -> any room | Pause persistence across teleport/room change | + +### Scope Note: Gestalt vs Tyre vs Dudley + +Gestalt designs 7+Hub rooms for the full Gauntlet. Tyre scopes Sprint 8 as infrastructure-only (no rooms). Dudley designs a 5-room loader MVP as the Sprint 9 implementation target. These are consistent: the map design is complete, room building follows infrastructure. + +--- + +## 6. Client Test Suite — Final 38 Tests (Stig) + +Deduplicated from 41 candidates (32 original + 6 Tyre additions + 3 Stig R2 additions). + +| Priority | Count | Sprint | Categories | +|----------|-------|--------|-----------| +| P0 | 2 | Sprint 8 | Monologue overwrite (Bug #5), camera static during pause (Bug #2) | +| P1 | 7 | Sprint 8-9 | Fog shader (4), entity lifecycle (2), pending recognition blob (1) | +| P2 | 24 | Sprint 9 | Camera (5), entity alpha/color (5), UI (12), lerp (1), teleport (1) | +| P3 | 5 | Sprint 10+ | Z-layer (4), lerp target (1) | + +Each test includes: function name, category, exact assertions, and setup requirements. + +### Fog Constants + +Added to `client/scripts/autoloads/fog_state.gd`: +- `VIS_HIDDEN = 0`, `VIS_PERIPHERAL = 180`, `VIS_FORWARD = 255` +- `EXP_UNEXPLORED = 0`, `EXP_EXPLORED = 128`, `EXP_VISIBLE = 255` + +5 magic number replacements in `fog_state.gd`. No shader changes needed. + +--- + +## 7. Prioritized Test Backlog — 59 Items (Hoshe) + +### Summary by Sprint + +| Sprint | P0 items | P1 items | Combined effort | +|--------|----------|----------|----------------| +| Sprint 8 | 10 (~7d) | 15 (~10.75d) | ~17.75d | +| Sprint 9 | 10 (~7.5d) | 12 (~12.5d) | ~20d | +| Sprint 10+ | -- | 12 (~18.75d) | ~18.75d | + +### Sprint 8 P0 (10 items, ~7d) + +1. Determinism Fix A (0.5d) +2. Determinism Fix B (0.25d) +3. Determinism Fix D (0.25d) +4. Server `--test-mode` + `--port 0` (1d) +5. `make pre-pr` target (0.5d) +6-8. Pause guard tests: movement_discarded, unpause_accepted, roundtrip (0.75d total) +9. Content cross-reference validation (1.5d) +10. Fixture staleness check (0.25d) + +### Bug Catalogue Coverage + +Every Sprint 6-7 bug has Sprint 8 P0/P1 coverage: +- Bug #1 (snapshot delivery): #4 (`--test-mode`), #26 (Layer 3) +- Bug #2 (camera startup): #21 (client P0: camera static during pause) +- Bug #3 (movement while paused): #6, #7, #8, #13 (pause guard suite) +- Bug #4 (MessagePack -128): #15, #16, #17, #18 (boundary value matrix) +- Bug #5 (monologue overwrite): #21 (client P0: monologue not lost) +- Bug #6 (snapshot spam): #1, #2 (determinism), #29 (golden files) + +--- + +## 8. Tooling Specifications (Justine) + +### `make pre-pr` — Final + +6-step chain: lint -> build -> test -> validate-content -> check-fact-ids -> fixtures-check. +~90s incremental, ~8min clean cache. Fixture staleness is BLOCKER (not WARNING). + +Branch-specific variants: `pre-pr-server`, `pre-pr-client`, `pre-pr-content`. + +Full Makefile provided with failure output examples. + +### `make perf-baseline` + +- Baseline file: `tests/perf/baseline.json` (committed) +- Median of 5 runs, thresholds: <15% PASS, 15-30% WARN, >30% FAIL +- Machine tag prevents cross-machine comparison +- `tooling/perf-compare` and `tooling/perf-update` scripts provided (full bash code) +- **Blocked on Gauntlet** — shadowcast bench is the only available benchmark until rooms ship + +### Golden File Workflow + +- Generator: `server/tests/gauntlet_golden_gen.rs` (generates JSON at ticks 0, 10, 100) +- Comparator: `server/tests/gauntlet_golden.rs` (field-by-field diff on mismatch) +- JSON format: sorted keys, pretty-printed, deterministic +- `make golden-diff` / `make golden-update` targets provided +- Full Rust code for both (~180 lines combined) + +### CI Pipeline Design (Deferred) + +3-tier pipeline documented and ready for when lead greenlights: +- **Commit tier** (<2min): lint + validate-content + check-fact-ids +- **PR tier** (<15min): build + test + fixtures staleness (BLOCKER) +- **Nightly tier** (<30min): Layer 3 + golden files + perf benchmarks + content scaling + +Complete `.gitea/workflows/ci.yaml` provided (~100 lines). Self-hosted runner required. + +--- + +## 9. Content Validation — 9 Checks (Hoshe + Justine merged) + +Extends `tooling/validate-content` (Python) with a second pass after schema validation. + +| # | Check | Severity | Source | +|---|-------|----------|--------| +| 1 | `canonical_id_uniqueness` | ERROR | Hoshe | +| 2 | `relationship_target_resolution` | ERROR | Both | +| 3 | `location_slug_resolution` | ERROR | Both | +| 4 | `dialogue_location_resolution` | ERROR | Both | +| 5 | `fact_id_resolution` | ERROR | Both (absorbs `check-fact-ids`) | +| 6 | `triangle_membership_resolution` | ERROR | Both | +| 7 | `npc_count_accuracy` | WARNING | Hoshe | +| 8 | `dialogue_line_id_uniqueness` | ERROR | Hoshe | +| 9 | `bidirectional_relationship_consistency` | WARNING | Justine | + +`ContentIndex` Python class skeleton provided. Phased rollout: Phase 1 (NPC/triangle/district) Sprint 8, Phase 2 (dialogue/fact_ids) Sprint 8, Phase 3 (warnings) Sprint 9. + +--- + +## 10. Anti-Tedium Specifications (Ozzie + Gestalt + Stig + Dudley) + +### Sprint 8 Priorities (Ozzie) + +| Priority | Feature | Effort | Justification | +|----------|---------|--------|--------------| +| P1 | Room Reset Triggers | 1-1.5d | Can't re-test without it | +| P2 | WRONG Button MVP | 1d | Can't report bugs without it | +| P3 (Sprint 9) | Hub Teleport | 0.5d | Convenience, not necessity | +| P4 (Sprint 9) | Timer + Checklist | 1.5-2d | Engagement, not necessity | + +**Note:** Tyre's Sprint 8 plan does NOT include anti-tedium features (infrastructure-only scope). Anti-tedium ships Sprint 9 per Tyre's roadmap, Sprint 8 per Ozzie/Gestalt's priority. + +### Room Reset — Final Spec + +- Trigger: Step on ResetPlate + Interact (NOT automatic) +- Server: `RoomResetTrigger` component, `RoomSnapshots` resource, 10-tick debounce +- Resets: entity positions, KG, fog (room tiles), room-sourced inventory, dialogue state +- Does NOT reset: other rooms, player position, session timer, SimRng state +- Test-mode only: entities only added with `--test-mode` + +### WRONG Button MVP — Final Spec + +- Hotkey: F12 +- Captures: current ObserverSnapshot (JSON), text output, tick/room/seed, tester description +- Output: `tests/bug-reports/gauntlet-t{tick}-{timestamp}/` (4 files: report.md, snapshot.json, text_output.txt, description.txt) +- Zero server changes needed for MVP +- Full version (Sprint 9+): 60-tick ring buffer, input history, replay seed + +### Hub Teleport + +- Home key -> `PlayerAction::TeleportToHub` +- Instant camera snap + 0.3s fade-to-black +- Gauntlet-only (server rejects in non-Gauntlet maps) + +### Client-Side UI (Stig) + +- `bug_report.gd` autoload: ~80 lines GDScript, F12 handler, ring buffer, modal prompt +- `InsertOverlay/FlashRect`: shared by reset flash and teleport fade +- `GauntletProgress` overlay: room name, run counter, timer, progress bar (Sprint 9) +- Fog constants migration: 6 named constants, 5 line replacements + +--- + +## 11. Human Tester Workflow (Ozzie) + +### 6-Phase Walkthrough + +1. **Navigate to a Room** — walk from Hub, room name updates, timer starts, checklist loads +2. **Execute the Checklist** — printed checklist or terminal display, auto-tracking for `type: auto` items +3. **Encounter a Bug** — F12 -> pause -> one-line prompt -> 4 files saved -> resume +4. **Reset and Re-Test** — step on reset plate -> room reverts to tick-0 state -> retry +5. **Move to Another Room** — Home -> Hub -> walk to next room +6. **End Session** — Ctrl+C -> session summary (rooms tested, coverage %, times, bug reports) + +### Quick-Test Developer Workflow + +Target: **65 seconds** from fix to verification. +Build (10s) -> start server+client (3s) -> navigate (5s) -> test checklist items (45s) -> exit (2s). + +### Entity Visibility Symbols + +| Symbol | State | Meaning | +|--------|-------|---------| +| `●` | VISIBLE | In clear vision cone | +| `◐` | REMEMBERED | In fog, previously seen | +| `◌` | FOGGED | Detected but not recognized | +| `✕` | BLOCKED | LOS blocked by wall (debug, requires `blocked_entities`) | +| `⚡` | RECOGNIZING | Mid-cognitive-delay | + +### Test Client Display + +10 sections: Header, Player, Entities, Fog, Sound (Sprint 9+), Cognition (Sprint 9+), Interactions, Monologue/Dialogue, Inventory, Status. MVP ships sections 1-4, 7-10. + +--- + +## 12. Checklist YAML Schema (Stig + Ozzie merged) + +### Per-Room Format + +```yaml +# content/gauntlet/rooms/{room_id}/checklist.yaml +room: occlusion_corridor +checks: + - id: occ_01_hidden_not_visible + description: "NPC behind wall is NOT visible" + type: auto # auto | manual + step: "Stand at (45,3), face East" + condition: + player_near: [45, 3] + entity: hidden-1 + expected: blocked + if_wrong: | + LOS leaking through wall. Check shadowcast.rs. +``` + +### Condition Grammar (7 types) + +`player_near`, `player_facing`, `entity` + `expected` (blocked/visible/remembered/recognizing), `expected_sector`, `perception_mode`, `fog_visible_count_min`/`max`, `inventory_count`, `dialogue_active`, `monologue_contains`. + +### Cross-Room Checklist + +`content/gauntlet/cross_room_checks.yaml` at Gauntlet root. Transition scenarios with multi-room paths and structured conditions. + +### Auto-Tracking Split + +| Feature | Godot Client | Test Client | +|---------|-------------|-------------| +| Room name + timer | Yes | Yes | +| Progress bar (X/Y) | Yes (total from YAML) | Yes | +| Auto-evaluate conditions | **No** | **Yes** (Rust, type-safe) | +| Per-item display | **No** | **Yes** | + +--- + +## 13. Boundary Value Tests — 41 Values (Hoshe) + +### Encoding Asymmetry (Resolved) + +GDScript uses `int_16` for values 256-32767; Rust uses `uint_16`. Both are spec-valid. `rmp_serde` accepts `int_16`-encoded positive values for `u64` fields (Dudley traced through rmp-serde 1.3.1 in Round 2). + +### 4-Direction Tests + +| Direction | What | Location | When | +|-----------|------|----------|------| +| Rust -> GDScript (fixture) | Snapshots with overlap-zone ticks | `test_msgpack_boundaries.gd` | Every PR | +| GDScript -> Rust (fixture) | Inputs with overlap-zone ticks | `serialization.rs` | Every PR | +| Rust -> GDScript (raw bytes) | Hand-crafted uint_16/uint_32 bytes | `test_msgpack_boundaries.gd` | Every commit | +| GDScript -> Rust (raw bytes) | Hand-crafted int_16/int_32 bytes | `serialization.rs` | Every commit | + +Full test code provided for all 4 directions. + +--- + +## 14. Layer 3 Test — Final Spec (Hoshe) + +`server/tests/layer3.rs`, `#[test] #[ignore]`, <10 seconds. + +- Build server binary +- Launch as subprocess with `--test-mode --port 0` +- Parse `LISTENING:{port}` from stdout (5-second timeout) +- Connect via TCP, send `PlayerInput` via `rmp_serde::to_vec` (not `to_vec_named`) +- Receive `ObserverSnapshot`, assert version, tick, entity count, player entity kind +- `ServerGuard` drop pattern for cleanup (kill on drop) +- Tolerates initial tick=0 snapshot before input processed (Dudley R2 adjustment) + +Full Rust code (~80 lines) provided. + +--- + +## 15. Resolved Questions + +### New Answers from Round 3 + +| ID | Question | Answer | By | +|----|----------|--------|-----| +| Q1 (R2-OQ-09) | Canonical room ordering for StableId? | Yes — append-only rule. YAML room order = spawn order = StableId order. | Dudley, Gestalt | +| Q2 (R2-OQ-10) | Room reset vs full restart? | Both. Room reset for iteration, kill+relaunch for determinism. | Gestalt | +| Q3 | Cross-cuts in Sprint 8? | 0 cross-cuts Sprint 8 (infrastructure only). 2 in map design. Build first 2 Sprint 10. | Tyre, Gestalt | +| Q4 (R2-OQ-05) | `blocked_entities` feasibility? | Feasible, ~300 tile lookups/tick. Sprint 9 scope, gated behind `--test-mode`. | Dudley | +| Q5 (R2-OQ-07) | Test client binary location? | `tooling/test-client/` (lead override). Bridge types already pub-exported. | Tyre, Dudley | +| Q6 (R2-OQ-08) | Cross-room checklist location? | `content/gauntlet/cross_room_checks.yaml` at Gauntlet root. | Gestalt | +| Q7 (R2-OQ-01) | SetTickRate while paused? | Bug. Reject. Pause exits only via explicit Unpause. 4-line fix in `input.rs`. | Dudley | +| Q8 (R2-OQ-02) | Entity index recycling safe? | Yes (bevy generation counter). Discipline: call `unregister()` on despawn. | Dudley | +| R2-OQ-03 | `content-ron` in pre-pr? | Yes, in full `pre-pr` only. Not in `pre-pr-server`. | Tyre | +| R2-OQ-06 | Checklist overlay in Godot? | Lightweight overlay (room name + timer). Full tracking test-client-only. | Stig | +| R2-OQ-11 (cross-cuts) | 4 cross-cuts too many? | Keep all 4 in design. Build 2 for Sprint 8 map. Don't consolidate Sound Lab. | Tyre, Gestalt | + +### Questions Deferred to Implementation + +| ID | Question | Assigned To | When | +|----|----------|-------------|------| +| R2-OQ-04 | Fixture staleness separate target? | Justine | Resolved: No. 10-15s cost negligible. | +| OQ-11 | Client tests headless stability | Stig | Before making client tests a CI merge gate | + +--- + +## 16. Sprint 9+ Roadmap (Tyre) + +### Tier 1: Sprint 9 (~10-12 team-days) + +R-01: Gauntlet rooms 1-4 (3-4d), R-02: Room reset trigger (1.5d), R-03: Hub teleport (0.5d), R-04: Client tests P0-P1 (2-3d), R-05: Determinism golden files (1d), R-06: Fog byte constants (0.25d), R-07: Encoding asymmetry tests (1.5d). + +### Tier 2: Sprint 10 (~15-17 team-days) + +R-08: WRONG button MVP (2d), R-09: Gauntlet rooms 5-8 (3-4d), R-10: Room timer + PBs (1d), R-11: Checklist auto-tracking (2d), R-12: Performance baselines (1.5d), R-13: Client tests P2 (3d), R-14: Enhanced test client terminal (2d). + +### Tier 3: Sprint 11+ (build when needed) + +Gauntlet rooms 9-14, cross-room transitions, CI automation, content scaling stress, `blocked_entities`, client test headless, client tests P3, additional pause guard tests, WRONG button full capture, bidirectional relationship warnings. + +### Risk Register (5 risks) + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| Test client takes >3 days | Medium | Delays Layer 3 | MVP scope intentionally minimal | +| Proof room too simple for golden files | Low | Golden files don't catch real bugs until rooms exist | Proves toolchain; value comes Sprint 9 | +| Content validation finds many existing errors | Medium | Sprint 8 effort bloat | Run as WARNING first sprint | +| Client test headless blocks CI | Low | Can't automate client tests | Manual `make pre-pr-client` interim | +| Fixture BLOCKER frustrates devs | Low | Devs skip pre-pr | Education; false positives are worse | + +--- + +## 17. Points of Agreement (consensus) + +1. **Sprint 8 is infrastructure.** All agents accept Tyre's framing: build the pipes, fill them Sprint 9+. +2. **Determinism fixes are complete.** Copy-pasteable code, 4 regression tests, ~22 lines total. +3. **`make pre-pr` is the developer discipline tool.** Replaces CI until CI is greenlighted. BLOCKER on fixture staleness. +4. **Gauntlet map design is complete.** 7+Hub rooms, 48 entities, 2 cross-cuts. Additive-only constraint. +5. **Test client binary architecture is settled.** `tooling/test-client/` crate (lead override), text renderer in server library, golden file JSON comparison. +6. **38 client tests are prioritized and build-ready.** Function names, assertions, setup — ready for Stig to implement. +7. **Anti-tedium is fully specified.** All 4 features have implementation details. Priority ranked. +8. **All open questions resolved.** Zero questions carry forward to implementation. + +## 18. Points of Tension (resolved or minor) + +1. **Fixture staleness: BLOCKER.** Tyre's argument accepted by all. Justine updated spec. +2. **Sprint 8 anti-tedium:** Ozzie/Gestalt prioritize room reset + WRONG button for Sprint 8. Tyre defers both to Sprint 9 (infrastructure-only scope). The implementation plan (Tyre) is authoritative for Sprint 8 scope; anti-tedium ships Sprint 9. +3. **Gauntlet room count for Sprint 8:** Dudley (5 rooms), Gestalt (7+Hub). Tyre (0 rooms — infrastructure only). Resolution: map design has 7+Hub, room building is Sprint 9. +4. **Test client location:** Tyre R3 says `server/src/bin/`. Lead overrides to `tooling/test-client/`. Dudley confirms no server changes needed. Lead override is authoritative. +5. **F3 debug overlay:** Deferred indefinitely per Stig. WRONG button captures same data. Ozzie accepts. + +--- + +## 19. Gaps Remaining + +1. **Gauntlet content authoring.** Room YAML, entity placement, checklist definitions need writing. No specific agent assigned for content creation. +2. **Client test headless stability (OQ-11).** Still unverified. Must be tested before making client tests a CI merge gate. +3. **Sound events not in ObserverSnapshot.** Required for full test client display (sections 5-6). Not yet designed server-side. +4. **Cognitive delay visual not implemented.** Tyre's tests #33-34 (blob rendering, recognition transition) depend on D-060 client implementation. +5. **Gauntlet map vs Dudley's loader room count.** Gestalt designs 8 rooms (7+Hub). Dudley's loader MVP has 5 rooms. Reconciliation needed during Sprint 9 implementation (which rooms build first). diff --git a/docs/workshops/test-architecture/stig-round2.md b/docs/workshops/test-architecture/stig-round2.md new file mode 100644 index 000000000..a4b2c5956 --- /dev/null +++ b/docs/workshops/test-architecture/stig-round2.md @@ -0,0 +1,510 @@ +# Stig — Round 2: Client Assertions, Anti-Tedium UI, Cross-Review + +## OQ-1: Does the Godot client depend on visible_tiles Vec ordering? + +**No. The client is order-independent. Fix #2 (sort visible_tiles server-side) has zero client impact.** + +Audited every consumer of `visible_tiles` in the client: + +| Consumer | File:Line | How it uses visible_tiles | Order-sensitive? | +|----------|-----------|--------------------------|------------------| +| `GameState.apply_snapshot()` | `game_state.gd:62-68` | Stores array as-is into `visible_tiles` | No — just assignment | +| `GameState.apply_snapshot()` | `game_state.gd:119-131` | Iterates to build `visibility_sectors` Dict and `visible_positions` Dict, keyed by `Vector2i` | **No** — Dict keyed by position, insertion order irrelevant | +| `FogState.update_from_state()` | `fog_state.gd:52-99` | Reads `GameState.visible_positions` (Dict) and `GameState.visibility_sectors` (Dict). Writes pixel values by position into byte arrays | **No** — position-indexed writes, any order produces same texture | +| `TileRenderer.update_tiles()` | `tile_renderer.gd:63-81` | Iterates tiles, calls `set_cell(coords, ...)` per tile | **No** — `set_cell()` is idempotent per coordinate. Same tilemap regardless of call order | +| `WorldRenderer.update_from_state()` | `world_renderer.gd:35-36` | Passes `visible_tiles` to `tile_renderer.update_tiles()` | Passthrough — see above | + +**Test fixtures that reference by index:** + +Two test files reference `visible_tiles[0]` by index: `test_protocol.gd:299-301` and `test_rendering.gd:66-69`. These assert against fixture data, not server output. If server-side sorting changes the fixture ordering, these tests need fixture updates — but they're not runtime bugs, just test data alignment. Flag for Hoshe to include in fixture regeneration. + +**Verdict: Fix #2 is safe to ship. No client code changes needed.** + +--- + +## OQ-6: Fog Byte Value Constants + +**Yes, promote to named constants.** These magic numbers appear in `fog_state.gd` and are assertion targets for 3+ tests. Named constants improve readability and testability. + +### Proposed constants (add to `fog_state.gd`) + +```gdscript +# Fog texture byte values — visibility and exploration layers. +# Used by fog shader to select visual treatment per tile. +# Assertable in tests: assert_that(vis_bytes[idx]).is_equal(FogState.VIS_FORWARD) +const VIS_HIDDEN: int = 0 # Not in LOS — fully fogged +const VIS_PERIPHERAL: int = 180 # In LOS, peripheral sector — light fog +const VIS_FORWARD: int = 255 # In LOS, forward sector — clear + +const EXP_UNEXPLORED: int = 0 # Never seen — dark +const EXP_EXPLORED: int = 128 # Previously seen, now out of LOS — deep fog +const EXP_VISIBLE: int = 255 # Currently in LOS — clear (matches VIS_FORWARD) +``` + +### Migration in fog_state.gd + +Replace the hardcoded values: + +| Current code | Replacement | +|-------------|-------------| +| `_vis_bytes[...] = 255 if sector == "Forward" else 180` | `_vis_bytes[...] = VIS_FORWARD if sector == "Forward" else VIS_PERIPHERAL` | +| `if _exp_bytes[idx] > 128:` / `_exp_bytes[idx] = 128` | `if _exp_bytes[idx] > EXP_EXPLORED:` / `_exp_bytes[idx] = EXP_EXPLORED` | +| `_exp_bytes[...] = 255` | `_exp_bytes[...] = EXP_VISIBLE` | +| `_vis_bytes.fill(0)` | `_vis_bytes.fill(VIS_HIDDEN)` (clearer intent) | + +Tests then assert: `assert_that(vis_byte).is_equal(FogState.VIS_FORWARD)` instead of magic `255`. + +--- + +## Anti-Tedium Client UI Specs + +Lead approved Ozzie's full anti-tedium suite. Here's the client-side implementation for each feature. + +### 1. Room Reset Trigger + +**What:** Player steps on a marked floor plate at a room entrance. Everything in that room resets to tick-0 state. + +**Client implementation:** + +- **Visual:** A 1-tile floor plate at each room entrance, rendered as a distinct tile type (`reset_plate`) in the TileRenderer. Color: subtle amber outline on floor tile (reuses `INSERT_COLOR_HOVER` amber at 30% alpha — visible but not distracting). +- **Interaction:** When the player entity occupies the reset plate tile, the interaction list shows a single verb: `"Reset Room"`. No auto-trigger — the player must press Interact. This prevents accidental resets while walking through. +- **Feedback on activation:** + 1. Brief screen flash (0.15s amber overlay at 10% opacity on the Insert CanvasLayer — diegetic, like a system pulse) + 2. Monologue-style text: `"Systems recalibrated."` (1.5s duration, using existing MonologueDisplay) + 3. Server handles the actual reset — client just renders the new snapshot as normal +- **No new UI elements needed.** The existing interaction list + monologue display handle everything. The reset plate is a tile type + an interaction verb. + +**Client code changes:** +- `tile_renderer.gd`: Add `"reset_plate"` to `TILE_TYPE_MAP` with atlas coords `(4,0)`, amber-tinted floor tile +- `constants.gd`: Add `TILE_COLOR_RESET_PLATE: Color` if needed for the atlas +- No new scene nodes, no new scripts + +### 2. Hub Teleport Hotkey + +**What:** One key press teleports the player back to the Central Hub. No walking. + +**Key: `Home`** — unmissable on standard keyboards, not used by any game action, mnemonic ("go home"). Alternative: `Backtick` (but that conflicts with console conventions). + +**Client implementation:** + +- **InputMapper addition:** New action `TELEPORT_HUB` mapped to `Home` key. Client-only in non-Gauntlet contexts (InputMapper can gate on a `gauntlet_mode` flag). +- **Wire format:** Sends `PlayerInput { action: "TeleportHub" }` to server. Server handles the actual teleport (sets player position to hub center, recomputes visibility). +- **Client feedback:** + 1. Instant camera snap to hub position (disable smoothing for 1 frame, same pattern as the startup camera anchor in `main.gd:34`) + 2. Brief fade-to-black-and-back (0.3s total) via a CanvasLayer overlay — gives the teleport a sense of transition without being slow + 3. No monologue — teleporting is a meta action, not diegetic + +**Client code changes:** +- `input_mapper.gd`: Add `TELEPORT_HUB` enum value, map to `Home` key +- `sim_bridge.gd`: Add `"TeleportHub"` to `_action_enum_to_wire()` +- `main.gd`: On next snapshot after teleport, reset camera anchor (same as `_camera_anchored = false` pattern) +- New scene: `UILayer/TeleportFade` — a `ColorRect` on the UI CanvasLayer, starts transparent, tweens to black and back + +### 3. WRONG Button (F12) — One-Press Bug Report + +**What:** Press F12. System captures full game state, opens a one-line prompt, saves everything to disk. + +**Client captures:** + +| Data | Source | Format | +|------|--------|--------| +| Current ObserverSnapshot | `GameState.current_snapshot` | JSON (via `JSON.stringify()`) | +| Client scene tree dump | `get_tree().root` recursive dump | Text: node paths, visibility, modulate, position | +| Last 60 ticks of input history | New `InputMapper.input_history` ring buffer (60 entries) | JSON array of `{tick, action, timestamp}` | +| Screenshot | `get_viewport().get_texture().get_image()` | PNG | +| Client console log (last 100 lines) | Godot's log file or custom ring buffer | Text | +| Player-entered description | One-line text input | Text | + +**Client implementation:** + +- **Hotkey:** `F12` mapped in `_unhandled_input()` on a new `BugReportCapture` autoload. Not in InputMapper — this is meta, not gameplay. +- **Flow on F12 press:** + 1. Game pauses immediately (sends `Pause` to server) + 2. All data captured in <100ms (snapshot is already in memory, screenshot is one API call, scene dump is a recursive traversal) + 3. A minimal text input appears center-screen on the Modal CanvasLayer: `"What's wrong? (one line)"` with a text field and `[Save]` / `[Cancel]` buttons + 4. On Save: writes all data to `tests/bug-reports/gauntlet-{YYYYMMDD-HHmmss}/` as individual files (`snapshot.json`, `scene_tree.txt`, `inputs.json`, `screenshot.png`, `console.txt`, `description.txt`) + 5. On Save or Cancel: unpauses + 6. Brief monologue: `"Noted."` (1s) — confirms to the tester it worked + +**Client code changes:** +- New autoload: `client/scripts/autoloads/bug_report.gd` (~80 lines) +- New scene node: `ModalLayer/BugReportPrompt` (Label + LineEdit + two Buttons) +- `input_mapper.gd`: Add `input_history: Array` ring buffer, append on every `queue_action()` +- New directory: `tests/bug-reports/` (gitignored) + +**Scene tree dump function:** +```gdscript +func _dump_scene_tree(node: Node, depth: int = 0) -> String: + var indent := " ".repeat(depth) + var line := "%s%s" % [indent, node.name] + if node is CanvasItem: + line += " visible=%s modulate=%s" % [node.visible, node.modulate] + if node is Node2D: + line += " pos=%s" % node.position + if node is Control: + line += " pos=%s size=%s" % [node.position, node.size] + var result := line + "\n" + for child in node.get_children(): + result += _dump_scene_tree(child, depth + 1) + return result +``` + +### 4. Room Timer + Progress Overlay + +**What:** Small overlay showing current room, run count, elapsed time, and checklist progress. + +**Position:** Top-right corner of the UI CanvasLayer. Small, semi-transparent, out of the way. Below the HUD's existing elements. + +**Layout:** +``` +┌─────────────────────────┐ +│ OCCLUSION CORRIDOR │ ← Room name (from player position + room bounds) +│ Run #12 · 0:47 │ ← Run counter + elapsed since room entry +│ ████░░░ 4/7 │ ← Checklist progress bar + fraction +└─────────────────────────┘ +``` + +**Visual style:** +- Background: `Color(0.05, 0.05, 0.08, 0.7)` — dark, semi-transparent, matching fog aesthetic +- Text: `INSERT_COLOR_TEXT` (#c8d0e0) — consistent with diegetic insert UI +- Progress bar: filled = `INSERT_COLOR_ACTIVE` (#6bc9a6), empty = `Color(0.2, 0.2, 0.25)` +- Font size: small (12px), monospace +- Only visible when `gauntlet_mode` is true + +**Data flow:** +- **Room name:** Server includes room metadata in the Gauntlet ObserverSnapshot (or client derives from player position + a room bounds lookup table loaded from the Gauntlet content pack) +- **Run counter:** Client-local. `Dictionary` incremented when entering a new room. Persisted to `user://gauntlet_stats.json` between sessions. +- **Timer:** Client-local. Starts when player enters a room (position crosses room bounds). Resets on room entry or room reset trigger. +- **Checklist progress:** Server-side. The Gauntlet tracks which assertions have been "witnessed" (e.g., "player stood at the right position and the correct entities were visible/hidden"). Comes in the ObserverSnapshot as `gauntlet_progress: {room: str, checked: int, total: int}` — or client-side if we prefer no server coupling. + +**Client code changes:** +- New scene node: `UILayer/GauntletProgress` (Panel with Labels + ProgressBar) +- New script: `client/scripts/ui/gauntlet_progress.gd` (~60 lines) +- Reads `gauntlet_mode` flag from a launch argument or environment variable +- Hidden when `gauntlet_mode == false` + +**Preference: client-side progress tracking.** The client knows which room the player is in and can track "did the player stand at the right position" locally. This avoids coupling the server to Gauntlet-specific UI state. The checklist YAML (see section 5) defines check conditions; the client evaluates them against GameState each tick. + +--- + +## Refined Test Functions (32 → 35) + +Based on Tyre's cross-review and Ozzie's anti-tedium additions, three new tests added and some refined. + +### Camera System (7 tests — unchanged) + +```gdscript +class_name TestCameraSystem extends GdUnitTestSuite + +# Existing (already in test_camera_anchor.gd): +func test_camera_position_after_ready() -> void +func test_camera_anchored_flag_after_ready() -> void +func test_camera_smoothing_off_after_ready() -> void +func test_camera_smoothing_reenabled_after_process() -> void +func test_camera_follows_player_movement() -> void + +# New: +func test_camera_static_during_pause() -> void + # Apply snapshot with tick_rate "Paused", send MoveNorth + # Assert camera.global_position unchanged + +func test_camera_position_after_rapid_snapshots() -> void + # Call receive_bytes() 3x (simulating server ticking faster than client) + # Call _process() once — camera should be at LAST snapshot's player position +``` + +### Entity Rendering (7 tests — unchanged) + +```gdscript +class_name TestEntityRendering extends GdUnitTestSuite + +func test_entity_peripheral_alpha() -> void + # visibility="Peripheral" → modulate.a == Constants.PERIPHERAL_ALPHA (0.5) + +func test_entity_forward_full_alpha() -> void + # visibility="Forward" → modulate.a == 1.0 + +func test_entity_color_player() -> void + # kind.variant="Player" → color == Constants.ENTITY_COLOR_PLAYER + +func test_entity_color_npc_unknown() -> void + # kind.variant="Npc" → color == Constants.ENTITY_COLOR_UNKNOWN (Phase 1) + +func test_entity_color_object() -> void + # kind.variant="Object" → color == Constants.ENTITY_COLOR_OBJECT + +func test_entity_removed_when_leaving_visibility() -> void + # update_entities with entity, then without → node removed, entity_nodes empty + +func test_entity_created_on_first_appearance() -> void + # Empty renderer, update_entities with 1 entity → 1 node in entity_nodes +``` + +### Z-Layer Ordering (4 tests — unchanged) + +```gdscript +class_name TestZLayerOrdering extends GdUnitTestSuite + +func test_fog_overlay_z_index() -> void + # FogOverlay node z_index == Constants.Z_FOG (900) + +func test_fog_entities_z_index() -> void + # FogEntities node z_index == Constants.Z_FOG_ENTITIES (950) + +func test_insert_overlay_canvas_layer() -> void + # InsertOverlay CanvasLayer.layer == Constants.CANVAS_INSERT (10) + +func test_ui_layer_canvas_layer() -> void + # UILayer CanvasLayer.layer == Constants.CANVAS_UI (20) +``` + +### Fog Shader State (4 tests — +1 new, uses proposed constants) + +```gdscript +class_name TestFogState extends GdUnitTestSuite + +func test_visibility_texture_forward_tile() -> void + # Set visible_positions with Forward sector + # Assert vis_bytes at that position == FogState.VIS_FORWARD (255) + +func test_visibility_texture_peripheral_tile() -> void + # Set visible_positions with Peripheral sector + # Assert vis_bytes at that position == FogState.VIS_PERIPHERAL (180) + +func test_exploration_persistence_after_los_exit() -> void + # Frame 1: tile visible (EXP_VISIBLE=255) + # Frame 2: tile not visible + # Assert exp_bytes at that position == FogState.EXP_EXPLORED (128) + +# NEW: test the hidden state explicitly +func test_visibility_texture_hidden_tile() -> void + # Tile not in visible_positions + # Assert vis_bytes at that position == FogState.VIS_HIDDEN (0) +``` + +### UI Elements (8 tests — unchanged) + +```gdscript +class_name TestUIElements extends GdUnitTestSuite + +# Monologue (bug #5 regression) +func test_monologue_consumed_once_per_tick() -> void + # Set current_monologue, call _consume_monologue() twice with same tick + # Assert monologue_display.show_monologue() called once + +func test_monologue_carried_forward_on_overwrite() -> void + # receive_bytes() with monologue, receive_bytes() without monologue + # Assert _last_snapshot still has monologue (carry-forward logic) + +# Interaction list (D-057) +func test_interaction_list_shows_verbs() -> void + # Set nearby_interactions with 2 verbs, call update_from_state() + # Assert interaction list visible, shows both verbs + +func test_interaction_list_hidden_when_empty() -> void + # Set nearby_interactions = [], call update_from_state() + # Assert interaction list not visible + +# Dialogue (D-061) +func test_dialogue_shows_on_snapshot() -> void + # Set current_dialogue, call _consume_dialogue() + # Assert dialogue_box.is_dialogue_active() == true + +func test_dialogue_dismissed_sends_end_input() -> void + # Emit dialogue_dismissed signal + # Assert SimBridge received DialogueEnd input + +func test_dialogue_option_sends_response() -> void + # Emit option_selected(1, "text") + # Assert SimBridge received DialogueResponse with index=1 + +# Stance indicator (D-053) +func test_stance_indicator_matches_game_state() -> void + # Set player_stance = "Sprint", call update_from_state() + # Assert indicator shows Sprint state +``` + +### Entity Lerp (3 tests — unchanged) + +```gdscript +class_name TestEntityLerp extends GdUnitTestSuite + +func test_entity_lerp_target_set_on_update() -> void + # update_entities with entity at (5,5) + # Assert _entity_targets[id] == Vector2(5*32+4, 5*32+4) (TILE_SIZE*pos + ENTITY_OFFSET) + +func test_entity_snap_on_first_appearance() -> void + # New entity → position == target immediately (no lerp from origin) + # Assert entity_nodes[id].position == _entity_targets[id] + +func test_entity_lerp_converges() -> void + # Set target to (10,10), call _process(0.016) 20 times + # Assert position.distance_to(target) < 1.0 (converged within ~0.3s) +``` + +### Anti-Tedium (2 new tests) + +```gdscript +class_name TestGauntletUI extends GdUnitTestSuite + +# NEW: Bug report capture +func test_bug_report_captures_snapshot() -> void + # Set GameState with known snapshot + # Call BugReport.capture() + # Assert output directory created with snapshot.json containing current_tick + +# NEW: Gauntlet progress overlay visibility +func test_gauntlet_progress_hidden_when_not_gauntlet() -> void + # gauntlet_mode = false + # Assert GauntletProgress node is not visible +``` + +**Final count: 35 test functions** across 7 categories. + +--- + +## Checklist Generation: `make checklist` Spec + +### YAML Schema (room definition `checklist:` block) + +Each Gauntlet room YAML file includes a `checklist` section: + +```yaml +# content/gauntlet/rooms/occlusion_corridor.yaml +room: + id: occlusion_corridor + name: "Occlusion Corridor" + systems: [LOS, shadowcasting, vision_cone, perception_modes] + bounds: + origin: {x: 40, y: 0} + size: {x: 20, y: 15} + +# ... entity placement, walls, etc ... + +checklist: + - step: "Stand at corridor entrance (45,3), face East" + verify: + - id: occ_guard_visible + text: "guard-1 at (48,3) is visible, Forward sector, full alpha" + condition: "entity guard-1 visible AND sector==Forward" + - id: occ_hidden_blocked + text: "hidden-1 at (50,7) is NOT in entity list (wall blocks LOS)" + condition: "entity hidden-1 NOT visible" + - id: occ_fog_corridor + text: "Corridor tiles ahead are visible, room behind wall is unexplored" + condition: "fog visible_count > 10" + + - step: "Walk south to (45,8), observe peripheral vision" + verify: + - id: occ_guard_peripheral + text: "guard-1 modulate.a ≈ 0.5 (Peripheral sector)" + condition: "entity guard-1 sector==Peripheral" + - id: occ_fog_decrease + text: "Fog visible_count decreases as corridor narrows" + condition: "fog visible_count < prev.visible_count" + + - step: "Switch to Sensor perception mode" + verify: + - id: occ_hidden_sensor + text: "hidden-1 appears in entity list with sensor-specific data" + condition: "entity hidden-1 visible AND perception==Sensor" +``` + +### YAML Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `step` | string | yes | Human-readable instruction for the tester | +| `verify` | array | yes | List of verification items for this step | +| `verify[].id` | string | yes | Unique identifier (used for progress tracking) | +| `verify[].text` | string | yes | Human-readable description of what to check | +| `verify[].condition` | string | no | Machine-parseable condition for auto-progress tracking (future) | + +The `condition` field is optional — it enables the client-side progress tracker (section 4 above) to automatically check off items. For v1, human testers check items manually. The conditions use a simple grammar: + +``` +condition := "entity" ("visible" | "NOT visible") [AND clause]* + | "fog" + | "inventory" +clause := field "==" value | field "!=" value +comparator := ">" | "<" | "==" | ">=" +``` + +Not a full DSL — just enough for the progress tracker to evaluate against GameState. Conditions that can't be expressed are omitted (tester checks manually). + +### Markdown Output Format + +`make checklist` generates `docs/qa/gauntlet-checklist.md`: + +```markdown +# Gauntlet QA Checklist +Generated from content/gauntlet/rooms/*.yaml +Date: {generation_date} + +## Occlusion Corridor +Systems: LOS, shadowcasting, vision_cone, perception_modes + +### Step 1: Stand at corridor entrance (45,3), face East +- [ ] guard-1 at (48,3) is visible, Forward sector, full alpha +- [ ] hidden-1 at (50,7) is NOT in entity list (wall blocks LOS) +- [ ] Corridor tiles ahead are visible, room behind wall is unexplored + +### Step 2: Walk south to (45,8), observe peripheral vision +- [ ] guard-1 modulate.a ≈ 0.5 (Peripheral sector) +- [ ] Fog visible_count decreases as corridor narrows + +### Step 3: Switch to Sensor perception mode +- [ ] hidden-1 appears in entity list with sensor-specific data + +--- + +## Inventory Warehouse +Systems: pickup, CarriedBy, inventory_grid, 9_slot_limit + +### Step 1: Enter warehouse, interact with crate_1 +... + +--- + +Total: {N} rooms, {M} steps, {K} verification items +``` + +### `make checklist` Implementation + +```makefile +checklist: + @echo "Generating Gauntlet QA checklist..." + python3 tooling/gen_checklist.py \ + --rooms content/gauntlet/rooms/ \ + --output docs/qa/gauntlet-checklist.md + @echo "Written to docs/qa/gauntlet-checklist.md" +``` + +The Python script (`tooling/gen_checklist.py`, ~50 lines): +1. Glob `content/gauntlet/rooms/*.yaml` +2. Sort alphabetically (deterministic output) +3. Parse each YAML file, extract `room.name`, `room.systems`, `checklist[]` +4. Emit markdown with `- [ ]` checkboxes per verify item +5. Footer with totals + +--- + +## Summary + +| Deliverable | Status | +|-------------|--------| +| OQ-1: visible_tiles ordering | **No client dependency** — Fix #2 safe to ship | +| OQ-6: Fog byte constants | **6 constants proposed** — VIS_HIDDEN/PERIPHERAL/FORWARD, EXP_UNEXPLORED/EXPLORED/VISIBLE | +| Room reset trigger | Reuses existing interaction list + monologue. New tile type only | +| Hub teleport | `Home` key, TeleportHub wire action, fade transition | +| WRONG button (F12) | New autoload + modal prompt, captures 6 data types to disk | +| Room timer + progress | Top-right overlay, client-local tracking, reads checklist YAML conditions | +| Test functions | 35 total (32 original + 1 fog hidden state + 2 anti-tedium) | +| Checklist spec | YAML schema with `step`/`verify`/`condition`, `make checklist` generates markdown | + +### Cross-Review Notes for Other Agents + +- **For Dudley:** The room reset trigger needs a server-side handler for `"ResetRoom"` action — resets all entities in the room's bounds to tick-0 state. Also needs `"TeleportHub"` action support. +- **For Tyre:** The bug report capture writes to disk from GDScript. Is `DirAccess.make_dir_recursive()` + `FileAccess.store_string()` sufficient, or should we use a dedicated output directory configured at launch? +- **For Hoshe:** Two fixture-based tests (`test_protocol.gd:299`, `test_rendering.gd:66`) reference `visible_tiles[0]` by index. If Fix #2 changes fixture ordering, these need updates during fixture regeneration. Low risk — just a heads up. +- **For Ozzie:** Your State Inspector Overlay (F3) is powerful but I'd defer it to a later sprint. The WRONG button captures the same data on demand. F3 as a real-time overlay requires per-frame string formatting of the entire ObserverSnapshot — measurable performance cost. Ship WRONG button first, F3 if testers ask for it. diff --git a/docs/workshops/test-architecture/stig-round3.md b/docs/workshops/test-architecture/stig-round3.md new file mode 100644 index 000000000..6753c7b8c --- /dev/null +++ b/docs/workshops/test-architecture/stig-round3.md @@ -0,0 +1,637 @@ +# Stig — Round 3: Final Client Test Spec, Anti-Tedium UI, Checklist, Fog Constants + +**Workshop:** QA Strategy & Test Architecture +**Round:** 3 (Prioritization) +**Date:** 2026-02-17 + +--- + +## 1. Client Test Suite — Final 38 Tests + +My original 32 + Tyre's 6 additions + my 3 from Round 2 = 41 candidates. Deduplicated to 38 (my "fog hidden state" test is subsumed by the existing 3 fog tests, and my 2 anti-tedium tests fold into the UI category). + +### P0 — Sprint 8 (ship with Gauntlet infrastructure) + +Must exist before any other client testing is meaningful. These guard shipped bug fixes. + +| # | Function Name | Category | Asserts | Setup | +|---|--------------|----------|---------|-------| +| 1 | `test_monologue_not_lost_on_snapshot_overwrite` | UI: Monologue | `receive_bytes()` with monologue, then `receive_bytes()` without → `_last_snapshot` still carries monologue (carry-forward logic) | SimBridge in live mode mock. Two PackedByteArrays: first with monologue dict, second without. | +| 2 | `test_camera_static_during_pause` | Camera | Apply snapshot with `tick_rate: "Paused"`, inject MoveNorth, call `_process()` → `camera.global_position` unchanged | Scene instance from main.tscn. Set `GameState.game_time.tick_rate = "Paused"` before process. | + +### P1 — Sprint 8-9 (information boundary + core rendering) + +These enforce the perception/fog contract that makes the game work. + +| # | Function Name | Category | Asserts | Setup | +|---|--------------|----------|---------|-------| +| 3 | `test_fog_visibility_forward_tile` | Fog state | `FogState._vis_bytes` at Forward-sector position == `FogState.VIS_FORWARD` (255) | Set `GameState.visible_positions` + `visibility_sectors` with one Forward tile, call `FogState.update_from_state()`. | +| 4 | `test_fog_visibility_peripheral_tile` | Fog state | `_vis_bytes` at Peripheral-sector position == `FogState.VIS_PERIPHERAL` (180) | Same as above with Peripheral sector. | +| 5 | `test_fog_exploration_persistence` | Fog state | Frame 1: tile visible (`EXP_VISIBLE`=255). Frame 2: tile not visible → `_exp_bytes` == `FogState.EXP_EXPLORED` (128) | Two `update_from_state()` calls with different visible_positions. | +| 6 | `test_fog_hidden_tile_value` | Fog state | Tile never in visible_positions → `_vis_bytes` == `FogState.VIS_HIDDEN` (0) | Default state after `_resize()`. | +| 7 | `test_entity_removed_when_leaving_visibility` | Entity rendering | `update_entities()` with entity, then without → `entity_nodes` empty, `get_node_or_null("Entity_N")` returns null (freed, not hidden) | EntityRenderer instance, two update calls. | +| 8 | `test_entity_created_on_first_appearance` | Entity rendering | Empty renderer → `update_entities()` with 1 entity → `entity_nodes.size() == 1`, child exists in tree | EntityRenderer instance. | +| 9 | `test_pending_recognition_blob_rendering` | Entity rendering | `GameState.pending_recognitions` with 1 entry → FogEntities node shows a blob child (not a full entity sprite) | Main scene or FogEntities subscene. Apply snapshot with `pending_recognitions` array. | + +### P2 — Sprint 9 (camera, entity visuals, UI) + +Comprehensive coverage of rendering transforms. + +| # | Function Name | Category | Asserts | Setup | +|---|--------------|----------|---------|-------| +| 10 | `test_camera_position_after_ready` | Camera | `camera.global_position == player_position * TILE_SIZE` after `_ready()` | Scene instantiate from main.tscn. SimBridge test mode. | +| 11 | `test_camera_anchored_flag_after_ready` | Camera | `_camera_anchored == true` | Same as above. | +| 12 | `test_camera_smoothing_off_after_ready` | Camera | `camera.position_smoothing_enabled == false` after `_ready()` | Same as above. | +| 13 | `test_camera_smoothing_reenabled_after_process` | Camera | After `_process(0.016)`: `camera.position_smoothing_enabled == true` | Scene instance, one process call. | +| 14 | `test_camera_follows_player_movement` | Camera | After MoveNorth + `_process()`: `camera.global_position == new player_position * TILE_SIZE` | Scene instance. Inject MoveNorth via SimBridge test queue. | +| 15 | `test_camera_position_after_rapid_snapshots` | Camera | Three `receive_bytes()` calls, one `_process()` → camera at LAST snapshot's player position. Verify convergence (not just no crash). | SimBridge live mock. Three snapshot byte arrays with different player positions. | +| 16 | `test_camera_position_after_hub_teleport` | Camera | After teleport (large position jump): camera snaps immediately (no slow lerp from old position) | Scene instance. Set `_camera_anchored = false` to trigger re-anchor. | +| 17 | `test_entity_peripheral_alpha` | Entity rendering | Entity with `visibility: "Peripheral"` → `modulate.a == Constants.PERIPHERAL_ALPHA` (0.5) | EntityRenderer instance, one entity with Peripheral visibility. | +| 18 | `test_entity_forward_full_alpha` | Entity rendering | Entity with `visibility: "Forward"` → `modulate.a == 1.0` | EntityRenderer instance, one entity with Forward visibility. | +| 19 | `test_entity_color_player` | Entity rendering | `kind.variant == "Player"` → `color == Constants.ENTITY_COLOR_PLAYER` | EntityRenderer instance with Player entity. | +| 20 | `test_entity_color_npc_unknown` | Entity rendering | `kind.variant == "Npc"` → `color == Constants.ENTITY_COLOR_UNKNOWN` | EntityRenderer instance with Npc entity. | +| 21 | `test_entity_color_object` | Entity rendering | `kind.variant == "Object"` → `color == Constants.ENTITY_COLOR_OBJECT` | EntityRenderer instance with Object entity. | +| 22 | `test_entity_modulate_remembered` | Entity rendering | Entity with visibility "Remembered" → visually distinct from "Visible" (different modulate or shader param) | EntityRenderer instance. Requires Remembered visibility in snapshot (not yet on wire — gate test on implementation). | +| 23 | `test_recognition_transition_animation` | Entity lerp | Entity moves from `pending_recognitions` to `entities` between ticks → visual transitions from blob to full entity over ~0.3s | FogEntities + EntityRenderer. Two sequential snapshots: first with pending_recognition, second with entity. | +| 24 | `test_monologue_consumed_once_per_tick` | UI: Monologue | Set `current_monologue`, call `_consume_monologue()` twice with same tick → `show_monologue()` called once | Main scene instance. Set `GameState.current_monologue` and `current_tick`. | +| 25 | `test_interaction_list_shows_verbs` | UI: Interaction | `nearby_interactions` with 2 verbs → interaction list visible, shows both verbs sorted by priority | Main scene or InteractionList subscene. Set GameState.nearby_interactions. | +| 26 | `test_interaction_list_hidden_when_empty` | UI: Interaction | `nearby_interactions == []` → interaction list not visible | Same, with empty array. | +| 27 | `test_dialogue_shows_on_snapshot` | UI: Dialogue | Set `current_dialogue` → `dialogue_box.is_dialogue_active() == true` | Main scene instance. Set GameState.current_dialogue. | +| 28 | `test_dialogue_dismissed_sends_end_input` | UI: Dialogue | Emit `dialogue_dismissed` signal → SimBridge receives `DialogueEnd` input | Main scene instance. Emit signal, check SimBridge test queue. | +| 29 | `test_dialogue_option_sends_response` | UI: Dialogue | Emit `option_selected(1, "text")` → SimBridge receives `DialogueResponse` with `index=1` | Main scene instance. Emit signal, check SimBridge test queue. | +| 30 | `test_stance_indicator_matches_game_state` | UI: Stance | Set `player_stance = "Sprint"`, call `update_from_state()` → indicator displays Sprint | StanceIndicator subscene or main scene. | +| 31 | `test_tick_rate_hud_indicator` | UI: HUD | `game_time.tick_rate` changes → HUD element reflects Full/Half/Paused | HUD subscene. Set GameState.game_time. | +| 32 | `test_inventory_full_visual_state` | UI: Inventory | 9/9 inventory → visual feedback (e.g., "FULL" indicator or slot highlight change) | InventoryGrid subscene. Set GameState.player_inventory with 9 items. | +| 33 | `test_sprint_interaction_suppression` | UI: Interaction | `player_stance == "Sprint"` → interaction list hidden/empty regardless of nearby_interactions content | Main scene or InteractionList. Set stance to Sprint, set nearby_interactions with data. | + +### P3 — Sprint 10+ (constants checks, animation polish, anti-tedium) + +Low regression risk or dependent on unimplemented features. + +| # | Function Name | Category | Asserts | Setup | +|---|--------------|----------|---------|-------| +| 34 | `test_fog_overlay_z_index` | Z-layer | FogOverlay node `z_index == Constants.Z_FOG` (900) | Main scene instance, traverse scene tree. | +| 35 | `test_fog_entities_z_index` | Z-layer | FogEntities node `z_index == Constants.Z_FOG_ENTITIES` (950) | Same. | +| 36 | `test_insert_overlay_canvas_layer` | Z-layer | InsertOverlay CanvasLayer `.layer == Constants.CANVAS_INSERT` (10) | Same. | +| 37 | `test_ui_layer_canvas_layer` | Z-layer | UILayer CanvasLayer `.layer == Constants.CANVAS_UI` (20) | Same. | +| 38 | `test_entity_lerp_target_set_on_update` | Entity lerp | After `update_entities()`: `_entity_targets[id] == Vector2(x * TILE_SIZE + ENTITY_OFFSET, y * TILE_SIZE + ENTITY_OFFSET)` | EntityRenderer instance, one entity. | + +**Deferred (not in the 38):** +- `test_entity_snap_on_first_appearance` — covered by #8 (entity created) + #38 (target set). Snap-vs-lerp is implicitly tested. +- `test_entity_lerp_converges` — animation polish, hard to assert deterministically across frame timings. Manual visual verification. +- `test_bug_report_captures_snapshot` — depends on BugReport autoload (Sprint 9+). Add when feature ships. +- `test_gauntlet_progress_hidden_when_not_gauntlet` — depends on GauntletProgress overlay (Sprint 9+). + +### Summary by Category + +| Category | Count | Sprint Range | +|----------|-------|-------------| +| Camera | 7 | 8-9 | +| Entity rendering | 7 (+ 2 Tyre) = 9 | 8-10 | +| Fog state | 4 | 8-9 | +| Z-layer | 4 | 10+ | +| UI: Monologue | 2 | 8-9 | +| UI: Interaction | 3 | 9 | +| UI: Dialogue | 3 | 9 | +| UI: Other (stance, tick rate, inventory, sprint suppress) | 4 | 9 | +| Entity lerp | 2 | 9-10 | +| **Total** | **38** | | + +--- + +## 2. Anti-Tedium UI — Build-Ready Specs + +### 2.1 Room Reset Trigger + +**Scene tree changes:** +``` +# No new nodes. Reuses existing systems: +# - TileRenderer: new tile type +# - InteractionList: shows "Reset Room" verb +# - MonologueDisplay: shows feedback text +``` + +**tile_renderer.gd changes:** +```gdscript +# Add to TILE_TYPE_MAP: +"reset_plate": TileType.RESET_PLATE + +# Add enum value: +enum TileType { FLOOR = 0, WALL = 1, DOOR = 2, OBJECT = 3, RESET_PLATE = 4 } +``` + +Atlas tile (4,0): floor-colored base with 1px amber (`INSERT_COLOR_HOVER`) inset border. Subtle — visible when you're looking, invisible when you're not. + +**Input handling:** None. The reset plate is an interaction target. When the player stands on it, the server includes it in `nearby_interactions` with verb `"ResetRoom"`. The existing interaction list + Interact input flow handles it. No new client input code. + +**Wire format:** Standard `Interact` action with `action_data: { target_entity_id: , verb: "ResetRoom" }`. Server handles the reset. Client just renders the next snapshot (which shows tick-0 state). + +**Visual feedback:** Server sends a monologue in the post-reset snapshot: `"Systems recalibrated."` (1.5s). The existing MonologueDisplay renders it. Additionally, a 0.15s amber screen flash: + +```gdscript +# In main.gd, after applying snapshot: +if _detected_room_reset(snapshot): + _flash_overlay(Constants.INSERT_COLOR_HOVER, 0.15) + +func _flash_overlay(color: Color, duration: float) -> void: + # Uses a ColorRect on InsertOverlay, tween alpha 0.1 -> 0.0 + var flash := $InsertOverlay/FlashRect # pre-existing ColorRect, normally transparent + flash.color = Color(color, 0.1) + var tween := create_tween() + tween.tween_property(flash, "color:a", 0.0, duration) +``` + +**Room reset detection:** Compare `snapshot.tick` — if it decreased or a special `room_reset: true` flag is present in the snapshot, trigger the flash. Simplest: server sets `room_reset: true` in the snapshot after a reset. Client checks once, fires flash, done. + +**New scene node:** `InsertOverlay/FlashRect` — a `ColorRect` covering the viewport, `color = Color.TRANSPARENT`, `mouse_filter = IGNORE`. Used by both reset flash and hub teleport fade. + +### 2.2 Hub Teleport + +**Input handling:** + +```gdscript +# input_mapper.gd — add to Action enum: +TELEPORT_HUB = 14 # (next available value) + +# input_mapper.gd — add to _input mapping: +if event.is_action_pressed("teleport_hub"): + queue_action(Action.TELEPORT_HUB) +``` + +```gdscript +# sim_bridge.gd — add to _action_enum_to_wire(): +InputMapper.Action.TELEPORT_HUB: return "TeleportToHub" +``` + +**Godot input map:** Add `teleport_hub` action mapped to `KEY_HOME` in `project.godot` or via code in `_ready()`. + +**Visual feedback — fade transition:** + +```gdscript +# main.gd — after detecting large position jump (teleport): +func _detect_teleport(old_pos: Vector2, new_pos: Vector2) -> bool: + return old_pos.distance_to(new_pos) > 5.0 # > 5 tiles = teleport, not walk + +# On teleport detected: +func _teleport_transition() -> void: + _camera_anchored = false # Force re-anchor (snap, no smooth) + var fade := $InsertOverlay/FlashRect + fade.color = Color(0, 0, 0, 1.0) # Instant black + var tween := create_tween() + tween.tween_property(fade, "color:a", 0.0, 0.3) # Fade back in over 0.3s +``` + +**Camera behavior:** Setting `_camera_anchored = false` triggers the re-anchor path in `_process()` — camera snaps to new position instantly (smoothing disabled for 1 frame, same as startup). No lerp from old position. + +**Gauntlet-only gate:** The `TELEPORT_HUB` action enum exists in InputMapper always, but the Godot input mapping is only added when `gauntlet_mode == true`. Or simpler: server rejects `TeleportToHub` in non-Gauntlet maps (server-side gate, client sends regardless). + +### 2.3 WRONG Button (F12) + +**New autoload:** `client/scripts/autoloads/bug_report.gd` + +```gdscript +extends Node + +const MAX_HISTORY: int = 60 +var input_history: Array[Dictionary] = [] # Ring buffer of {tick, action, timestamp} +var _report_dir: String = "user://bug-reports" + +func _ready() -> void: + # Create base directory + DirAccess.make_dir_recursive_absolute( + ProjectSettings.globalize_path(_report_dir)) + +func _unhandled_input(event: InputEvent) -> void: + if event.is_action_pressed("bug_report"): + capture_bug_report() + get_viewport().set_input_as_handled() + +func record_input(input: Dictionary) -> void: + input_history.append(input) + if input_history.size() > MAX_HISTORY: + input_history.pop_front() + +func capture_bug_report() -> void: + # 1. Pause + SimBridge.send_input({ + "action": InputMapper.Action.PAUSE, + "timestamp_msec": Time.get_ticks_msec(), + }) + + # 2. Capture data + var snapshot_json := JSON.stringify(GameState.current_snapshot, "\t") + var scene_dump := _dump_scene_tree(get_tree().root) + var inputs_json := JSON.stringify(input_history, "\t") + var screenshot := get_viewport().get_texture().get_image() + + # 3. Show prompt (modal) + var prompt := $"/root/Main/ModalLayer/BugReportPrompt" + prompt.show_prompt(func(description: String): + _save_report(snapshot_json, scene_dump, inputs_json, screenshot, description) + # 4. Unpause + SimBridge.send_input({ + "action": InputMapper.Action.UNPAUSE, + "timestamp_msec": Time.get_ticks_msec(), + }) + ) + +func _save_report(snapshot: String, scene: String, inputs: String, + screenshot: Image, description: String) -> void: + var timestamp := Time.get_datetime_string_from_system().replace(":", "-") + var dir_path := _report_dir + "/gauntlet-" + timestamp + DirAccess.make_dir_recursive_absolute( + ProjectSettings.globalize_path(dir_path)) + + _write_file(dir_path + "/snapshot.json", snapshot) + _write_file(dir_path + "/scene_tree.txt", scene) + _write_file(dir_path + "/inputs.json", inputs) + _write_file(dir_path + "/description.txt", description) + screenshot.save_png(ProjectSettings.globalize_path(dir_path + "/screenshot.png")) + +func _write_file(path: String, content: String) -> void: + var file := FileAccess.open(path, FileAccess.WRITE) + if file: + file.store_string(content) + +func _dump_scene_tree(node: Node, depth: int = 0) -> String: + var indent := " ".repeat(depth) + var line := "%s%s" % [indent, node.name] + if node is CanvasItem: + line += " visible=%s mod=%s" % [node.visible, node.modulate] + if node is Node2D: + line += " pos=%s" % node.position + if node is Control: + line += " pos=%s sz=%s" % [node.position, node.size] + var result := line + "\n" + for child in node.get_children(): + result += _dump_scene_tree(child, depth + 1) + return result +``` + +**Input map addition:** `bug_report` action → `KEY_F12` + +**Modal prompt scene:** `ModalLayer/BugReportPrompt` +``` +BugReportPrompt (PanelContainer, visible=false) + VBoxContainer + Label "What's wrong? (one line)" + LineEdit (placeholder: "Describe the issue...") + HBoxContainer + Button "Save" + Button "Cancel" +``` + +Style: dark panel (`Color(0.08, 0.08, 0.12, 0.95)`), text in `INSERT_COLOR_TEXT`, centered on screen. Modal — blocks input to game while visible. + +**Integration with InputMapper:** Call `BugReport.record_input(input_dict)` from `InputMapper.queue_action()` to feed the ring buffer. One line addition. + +**Confirmation feedback:** After save, show monologue `"Noted."` (1.0s) via existing MonologueDisplay. + +### 2.4 Room Timer + Progress Overlay + +**New scene node:** `UILayer/GauntletProgress` +``` +GauntletProgress (PanelContainer, visible=false) + VBoxContainer + RoomLabel (Label) — "OCCLUSION CORRIDOR" + RunLabel (Label) — "Run #12 · 0:47" + ProgressBar — 4/7 + ProgressLabel (Label) — "4/7" +``` + +**Position:** Anchored top-right, 8px margin. `anchor_left = 1.0, anchor_right = 1.0, anchor_top = 0.0`. Grows leftward from the right edge. + +**Style:** +- Panel: `Color(0.05, 0.05, 0.08, 0.7)`, 4px corner radius +- Room name: `INSERT_COLOR_TEXT` (#c8d0e0), 12px, bold +- Run/timer: `INSERT_COLOR_TEXT`, 10px, regular +- Progress bar fill: `INSERT_COLOR_ACTIVE` (#6bc9a6) +- Progress bar empty: `Color(0.2, 0.2, 0.25)` +- Max width: 220px + +**Script:** `client/scripts/ui/gauntlet_progress.gd` + +```gdscript +extends PanelContainer + +var gauntlet_mode: bool = false +var _current_room: String = "" +var _room_enter_time: float = 0.0 +var _run_counts: Dictionary = {} # room_name -> int +var _room_totals: Dictionary = {} # room_name -> total checklist items +var _room_checked: Dictionary = {} # room_name -> checked count + +func _ready() -> void: + gauntlet_mode = OS.get_environment("SR_GAUNTLET") == "1" + visible = false + +func update_from_state() -> void: + if not gauntlet_mode: + visible = false + return + + var room := _detect_room(GameState.player_position) + if room.is_empty(): + visible = false + return + + visible = true + + if room != _current_room: + _current_room = room + _room_enter_time = Time.get_ticks_msec() / 1000.0 + _run_counts[room] = _run_counts.get(room, 0) + 1 + + var elapsed := Time.get_ticks_msec() / 1000.0 - _room_enter_time + var mins := int(elapsed) / 60 + var secs := int(elapsed) % 60 + + $VBoxContainer/RoomLabel.text = room.to_upper() + $VBoxContainer/RunLabel.text = "Run #%d · %d:%02d" % [ + _run_counts.get(room, 1), mins, secs] + + var total: int = _room_totals.get(room, 0) + var checked: int = _room_checked.get(room, 0) + if total > 0: + $VBoxContainer/ProgressBar.value = float(checked) / float(total) + $VBoxContainer/ProgressLabel.text = "%d/%d" % [checked, total] + else: + $VBoxContainer/ProgressBar.value = 0.0 + $VBoxContainer/ProgressLabel.text = "" + +func _detect_room(_player_pos: Vector2) -> String: + # TODO: Load room bounds from Gauntlet content, match against player position + return "" +``` + +**Visibility gate:** `SR_GAUNTLET=1` environment variable. Not visible in normal gameplay. + +**Data persistence:** `_run_counts` saved to `user://gauntlet-stats.json` on room change and on `_notification(NOTIFICATION_WM_CLOSE_REQUEST)`. + +--- + +## 3. Checklist YAML — Final Merged Schema + +Merges my Round 1 co-located approach with Ozzie's 4 additions: auto/manual type tags, `if_wrong` field, structured conditions, cross-room items. + +### Per-Room Checklist Schema + +```yaml +# content/gauntlet/rooms/{room_id}/checklist.yaml + +room: occlusion_corridor +description: "Tests LOS, shadowcasting, vision cone, and perception modes" + +checks: + - id: occ_01_hidden_not_visible + description: "NPC behind wall is NOT visible in Visual mode" + type: auto # auto = snapshot-verifiable, manual = human judgment + step: "Stand at corridor entrance (45,3), face East" + condition: + player_near: [45, 3] + player_facing: East + entity: hidden-1 + expected: blocked # blocked | visible | remembered | recognizing + if_wrong: | + LOS leaking through wall. Check symmetric shadowcasting + in server/src/perception/shadowcast.rs. + Verify wall at designated blocking position exists in room YAML. + + - id: occ_02_guard_visible + description: "NPC in front of wall IS visible with full alpha" + type: auto + step: "Same position — guard should be in clear LOS" + condition: + player_near: [45, 3] + entity: guard-1 + expected: visible + expected_sector: Forward + if_wrong: | + Guard not visible at distance 3 in clear LOS. + Check entity spawn position in room YAML. + Check vision cone range in server/src/perception/query.rs. + + - id: occ_03_peripheral_dimmed + description: "Entity at edge of vision cone is dimmed (Peripheral sector)" + type: auto + step: "Walk south to (45,8), observe entity alpha" + condition: + player_near: [45, 8] + entity: guard-1 + expected_sector: Peripheral + if_wrong: | + Peripheral dimming not applied. Check Constants.PERIPHERAL_ALPHA + and entity_renderer.gd modulate.a assignment. + + - id: occ_04_sensor_detects_hidden + description: "Sensor perception mode detects NPC behind wall" + type: auto + step: "Switch to Sensor perception mode" + condition: + perception_mode: Sensor + entity: hidden-1 + expected: visible + if_wrong: | + Sensor mode not detecting through walls. Check perception mode + compute_geometry in server/src/perception/modes/. + + - id: occ_05_cognitive_delay_timing + description: "Cognitive delay for recognition takes approximately 0.6s" + type: manual # Subjective — tester judges timing + step: "Walk to fog boundary, wait for recognition" + guidance: | + Watch pending_recognitions — elapsed should reach ~6 ticks + (0.6s at 10 tps). The monologue should fire DURING the delay, + not after it completes. + if_wrong: | + Cognitive delay timing off. Check D-060 values in + server/src/perception/recognition.rs. + + - id: occ_06_fog_corridor_state + description: "Fog shows corridor tiles as visible, room behind wall as unexplored" + type: auto + step: "Same position as step 1" + condition: + player_near: [45, 3] + fog_visible_count_min: 10 + if_wrong: | + Fog not rendering corridor correctly. + Check FogState.update_from_state() and fog shader uniforms. + + - id: occ_07_fog_count_decreases_in_narrow + description: "Fog visible count decreases as corridor narrows" + type: manual + step: "Walk deeper into corridor, observe fog count" + guidance: | + Visible tile count should drop as walls close in. + Compare fog count at (45,3) vs (45,10). + if_wrong: | + Fog not responding to geometry. Check shadowcasting range + vs corridor width. +``` + +### Schema Field Reference + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `room` | string | yes | Room ID (matches room YAML) | +| `description` | string | yes | Human-readable room purpose | +| `checks` | array | yes | Checklist items | +| `checks[].id` | string | yes | Unique ID (prefix with room abbreviation) | +| `checks[].description` | string | yes | What to verify (one sentence) | +| `checks[].type` | `auto` or `manual` | yes | `auto`: test client evaluates from snapshot. `manual`: human judgment | +| `checks[].step` | string | yes | Action the tester takes | +| `checks[].condition` | object | for `auto` | Structured condition for auto-evaluation | +| `checks[].guidance` | string | for `manual` | Instructions for human judgment | +| `checks[].if_wrong` | string | yes | Likely causes + file references | + +### Condition Grammar + +```yaml +condition: + player_near: [x, y] # Player within 2 tiles of position + player_facing: Direction # N/NE/E/SE/S/SW/W/NW + entity: entity-name # Gauntlet entity constant name + expected: blocked|visible|remembered|recognizing + expected_sector: Forward|Peripheral + perception_mode: Visual|Sensor|... + fog_visible_count_min: N # Minimum visible tile count + fog_visible_count_max: N # Maximum visible tile count + inventory_count: N # Exact inventory count + inventory_count_min: N + dialogue_active: true|false + monologue_contains: "substring" +``` + +The test client evaluates conditions against the current ObserverSnapshot. The Godot client does NOT evaluate conditions — it only displays the progress count (see section 5). + +### Cross-Room Checklist + +```yaml +# content/gauntlet/cross_room_checks.yaml + +cross_room_checks: + - id: xr_01_sprint_exit_buffer + description: "Sprint from Crowd Plaza into Occlusion Corridor — interaction buffer cleared" + rooms: [crowd_plaza, occlusion_corridor] + type: manual + step: "Sprint through Plaza, enter Corridor. Check interaction list is empty during sprint." + if_wrong: | + Sprint suppression not clearing buffer on room transition. + Check D-055 interaction buffer clear in input.rs. + + - id: xr_02_fog_into_dialogue + description: "Start dialogue while partially fogged — fog state preserved" + rooms: [fog_theater, dialogue_room] + type: manual + step: "Walk from Fog Theater to Dialogue Room. Initiate dialogue. Fog should not reset." + if_wrong: | + Fog state being cleared on dialogue open. + Check fog_state.gd update_from_state() is not skipping during dialogue. + + - id: xr_03_full_inventory_interact + description: "Full inventory + interaction — Take verb greyed, server still offers it" + rooms: [inventory_warehouse, interaction_gallery] + type: auto + condition: + inventory_count: 9 + entity: interaction_gallery_crate + expected: visible + if_wrong: | + Server should offer Take even when inventory full (available=true). + Client should grey it out. Check interaction list rendering. +``` + +### `make checklist` Implementation + +```makefile +checklist: + @python3 tooling/gen_checklist.py \ + --rooms content/gauntlet/rooms/ \ + --cross content/gauntlet/cross_room_checks.yaml \ + --output docs/qa/gauntlet-checklist.md + @echo "Checklist: docs/qa/gauntlet-checklist.md" +``` + +**Output format:** Same as Round 2 spec — markdown with `- [ ]` checkboxes, grouped by room, steps as `###` headings, `if_wrong` in blockquotes under each item. Cross-room items get their own section at the bottom. + +--- + +## 4. Fog Constants — Definition and Migration + +### Constants to add to `client/scripts/autoloads/fog_state.gd` + +```gdscript +# Fog texture byte values — visibility and exploration layers. +# Used by fog shader to distinguish visual treatment per tile. +# Test assertions reference these: assert_that(byte).is_equal(FogState.VIS_FORWARD) +const VIS_HIDDEN: int = 0 # Not in LOS — fully fogged +const VIS_PERIPHERAL: int = 180 # In LOS, peripheral sector — light fog dimming +const VIS_FORWARD: int = 255 # In LOS, forward sector — clear vision + +const EXP_UNEXPLORED: int = 0 # Never seen — total darkness +const EXP_EXPLORED: int = 128 # Previously seen, now out of LOS — deep fog +const EXP_VISIBLE: int = 255 # Currently in LOS — clear (written each frame) +``` + +### Files to update + +**`client/scripts/autoloads/fog_state.gd`** — 5 replacements: + +| Line | Current | Replacement | +|------|---------|-------------| +| 68 | `_vis_bytes.fill(0)` | `_vis_bytes.fill(VIS_HIDDEN)` | +| 75 | `_vis_bytes[py * _width + px] = 255 if sector == "Forward" else 180` | `_vis_bytes[py * _width + px] = VIS_FORWARD if sector == "Forward" else VIS_PERIPHERAL` | +| 87 | `if _exp_bytes[idx] > 128:` | `if _exp_bytes[idx] > EXP_EXPLORED:` | +| 88 | `_exp_bytes[idx] = 128` | `_exp_bytes[idx] = EXP_EXPLORED` | +| 94 | `_exp_bytes[py * _width + px] = 255` | `_exp_bytes[py * _width + px] = EXP_VISIBLE` | + +**`client/tests/test_fog_shader.gd`** — use constants in any assertions that reference byte values. + +**No other files affected.** The fog shader (`shaders/fog.gdshader`) reads textures, not GDScript constants. The byte values flowing into the texture are the same — just named now. + +--- + +## 5. Answer: Checklist Overlay in Godot Client? + +**Ozzie's question (R2-OQ-06):** Can the Godot client render a checklist progress overlay, or is checklist tracking test-client-only? + +**Answer: The Godot client shows a lightweight progress overlay. Full auto-tracking is test-client-only.** + +Split: + +| Feature | Godot Client | Test Client Binary | +|---------|-------------|-------------------| +| Room name display | Yes — from room bounds | Yes | +| Run counter | Yes — client-local | Yes | +| Timer | Yes — client-local | Yes | +| Progress bar (X/Y) | Yes — loads total count from checklist YAML | Yes | +| Auto-evaluate conditions | **No** — too complex for GDScript, snapshot structure differs | **Yes** — Rust, same crate as ObserverSnapshot types | +| Auto/manual confirm | **No** | **Yes** | +| Per-item checklist display | **No** — just the bar | **Yes** — full `[✓]/[ ]/[?]` list | +| if_wrong guidance | **No** | **Yes** | + +**Why the split:** The Godot client's GameState holds the snapshot as a Dictionary. Evaluating structured conditions (`player_near`, `entity expected: blocked`) against Dictionary data is fragile GDScript — no type safety, no access to entity name constants. The test client binary has the same types as the server (`ObserverSnapshot`, entity ID constants), making condition evaluation clean and type-safe. + +The Godot client's progress overlay (section 2.4) gets its checked count from the test client via a simple mechanism: the test client writes `gauntlet-stats.json` with per-room progress, the Godot client reads it. Or more simply: the Godot client shows total items per room (loaded from YAML at startup) but doesn't track checked items. The progress bar shows "0/7" until the tester manually marks items done (or it stays as a room-entry indicator only). + +**Recommendation:** Ship the Godot overlay with room name + timer + run counter only (Sprint 9). Progress bar comes when the test client ships and can feed it data (Sprint 9-10). Don't block the overlay on checklist auto-tracking. + +--- + +## Summary + +| Deliverable | Count/Status | +|-------------|-------------| +| Client tests: P0 | 2 (monologue overwrite, camera pause) | +| Client tests: P1 | 7 (fog ×4, entity lifecycle ×2, recognition blob) | +| Client tests: P2 | 24 (camera ×5, entity ×5, UI ×12, lerp ×1, teleport ×1) | +| Client tests: P3 | 5 (z-layer ×4, lerp target ×1) | +| Client tests: total | **38** | +| Anti-tedium features | 4 specced (reset, teleport, WRONG, progress) | +| New autoloads | 1 (bug_report.gd, ~80 lines) | +| New scene nodes | 3 (FlashRect, BugReportPrompt, GauntletProgress) | +| Input additions | 2 (TELEPORT_HUB, bug_report) | +| Checklist YAML schema | Complete with 7 condition types | +| Fog constants | 6 constants, 5 line replacements in fog_state.gd | +| Ozzie's question | Answered: lightweight overlay in Godot, full tracking in test client | diff --git a/docs/workshops/test-architecture/test-architecture-workshop-brief.md b/docs/workshops/test-architecture/test-architecture-workshop-brief.md index d96e87ca9..7d84a52a5 100644 --- a/docs/workshops/test-architecture/test-architecture-workshop-brief.md +++ b/docs/workshops/test-architecture/test-architecture-workshop-brief.md @@ -1,115 +1,216 @@ -# Test Architecture & Tooling Workshop Brief +# QA Strategy & Test Architecture Workshop Brief -**Goal:** Design the test strategy that catches the class of bugs we've been hitting — blocking I/O, serialization boundary errors, state desync, off-by-one encoding — before they reach manual playtesting. Produce concrete test specifications, not aspirational test plans. +**Goal:** Design a comprehensive testing strategy — test world, automation framework, content scaling safety, and CI pipeline — that catches integration boundary bugs before they reach manual playtesting, and scales safely as content grows. Produce concrete specifications and a prioritized implementation backlog. -**Participants:** Tyre, Hoshe, Dudley, Stig +**Participants:** Tyre, Hoshe, Dudley, Stig, Justine, Gestalt, Ozzie -**Context:** Sprint 6-7 development exposed a pattern of bugs that unit tests don't catch and manual `make game` testing finds too late. The bugs share a common profile: they live at integration boundaries (client-server, encode-decode, ECS schedule ordering, pause/unpause state transitions). D-030 defined the test architecture but implementation has lagged — this workshop closes the gap. +**Note for Tyre:** You may spawn Troblum as a sparring partner for feasibility questions if you need a second opinion on architecture or technology choices. + +**Context:** Sprint 6-7 development exposed a pattern of bugs that unit tests don't catch and manual `make game` testing finds too late. The bugs live at integration boundaries (client-server, encode-decode, ECS schedule ordering, pause/unpause state transitions). D-030 defined the test architecture but implementation has lagged. Simultaneously, the project is about to scale content generation (more NPCs, locations, items, districts) — we need confidence that new content won't introduce regressions. This workshop designs the QA infrastructure to keep everything tight and under control. ## The Bug Catalogue -These are real bugs from the current sprint. Every test specification produced by this workshop must prevent at least one from recurring. +Real bugs from Sprint 6-7. Every test specification produced by this workshop must prevent at least one class of these from recurring. -| # | Bug | Root Cause | Category | -|---|-----|-----------|----------| -| 1 | Server never sends snapshots in live mode | `read_framed()` blocking TCP read stalls entire bevy Update schedule | IPC / schedule integration | -| 2 | Camera doesn't center on player at startup | Client receives no snapshot until first keystroke (consequence of #1) | Client-server startup sequencing | -| 3 | Player moves while game is paused | `process_player_input` processes movement regardless of TickRate | State transition / input filtering | -| 4 | MessagePack encodes tick 128 as -128 | Signed int8 branch uses `<=` instead of `<` for upper bound | Serialization boundary | -| 5 | Monologue lost when snapshots overwrite | Latest-wins snapshot buffer drops one-shot events | Event delivery reliability | -| 6 | Snapshot overwrite warning spam | Server ticks faster than client consumes | Rate mismatch (design, not bug) | +| # | Bug | Root Cause | Category | Status | +|---|-----|-----------|----------|--------| +| 1 | Server never sends snapshots in live mode | `read_framed()` blocking TCP read stalls entire bevy Update schedule | IPC / schedule integration | Fixed | +| 2 | Camera doesn't center on player at startup | Client receives no snapshot until first keystroke (consequence of #1) | Client-server startup sequencing | Fixed | +| 3 | Player moves while game is paused | `process_player_input` processes movement regardless of TickRate | State transition / input filtering | Fixed | +| 4 | MessagePack encodes tick 128 as -128 | Signed int8 branch uses `<=` instead of `<` for upper bound | Serialization boundary | Fixed (all int sizes) | +| 5 | Monologue lost when snapshots overwrite | Latest-wins snapshot buffer drops one-shot events | Event delivery reliability | Fixed (carry-forward) | +| 6 | Snapshot overwrite warning spam | Server ticks faster than client consumes | Rate mismatch | Fixed (carry-forward eliminates warning) | + +## The Test World — "The Gauntlet" + +A purpose-built, **permanently stable** server-side map designed as a QA playground. Not a game level — a mechanical test suite in map form. The Gauntlet is the golden file anchor: its layout never changes, so regression tests against it are reliable. When new systems are added, new rooms are added to the Gauntlet — existing rooms stay frozen. + +Dynamic/procedural maps get separate, fuzzier assertion-based tests (e.g., "player can see at least 3 NPCs" not "player sees kael at (12,7)"). + +### Proposed Rooms + +| Room | Systems Exercised | Key Assertions | +|------|-------------------|----------------| +| **Inventory Warehouse** | Pickup, CarriedBy, inventory grid, 9-slot limit | Take when full, drop, re-take, slot assignment stability | +| **Occlusion Corridor** | LOS, shadowcasting, vision cone, perception modes | NPC behind wall: invisible in Visual, detected in Sensor, heat signature in IR, footsteps in Sound. Each perception mode yields different ObserverSnapshot content | +| **Interaction Gallery** | One entity per ObjectType + one multi-verb NPC | Single-verb Interact, multi-verb scroll/selection, verb priority ordering, sprint suppression clears interaction buffer | +| **Crowd Plaza** | 15+ NPCs at various relationship states/distances | Entity rendering at density, cognitive delay overlap, D-033 color spread, performance (tick budget) | +| **Fog Theater** | Open area → corridor → room with controlled LOS transitions | Fog layer transitions, peripheral dimming, remembered entities after LOS exit, exploration texture persistence | +| **Dialogue Room** | NPCs at different trust tiers, one with contradiction | Full dialogue tree traversal, walk-away mid-dialogue, confrontation trigger, monologue during cognitive delay | +| **Pause Chamber** | TickRate toggle, state transitions | Movement discarded while paused, pause/unpause round-trip, UI state during pause, stance toggle during pause | +| **Zone Gate** (provisioned) | Zone transition, entity persistence, camera behavior | *Reserved space in the map layout. Implementation deferred until multi-map system exists. Test specifications written as future contracts — "when zone transitions ship, these assertions must pass."* | + +### Design Constraints + +- **Stable coordinates**: Every entity, wall, and item has a fixed position documented in the Gauntlet spec. Regression tests reference these positions by name (e.g., `GAUNTLET.occlusion_npc_behind_wall`), not raw coordinates. +- **Deterministic content**: The Gauntlet loads from a fixed content pack with a fixed seed. No procedural variation. Same seed → same world state at tick 0. +- **Additive only**: Existing rooms are never modified. New systems get new rooms appended to the map. This preserves all existing golden files. +- **Launchable**: `make test-world` boots server with the Gauntlet map and connects the client. `make test-world-headless` boots server only for automated tests. ## Workshop Tracks -### Track 1: Integration Test Architecture (Tyre, Dudley) +### Track 1: Test World Design (Gestalt, Dudley, Tyre, Ozzie) -D-030 defined three IPC test layers but only Layer 1 (fixture roundtrips) and partial Layer 2 (bridge tests) exist. Layer 3 (real subprocess integration) is entirely missing. This track designs the concrete test infrastructure. +Design the Gauntlet map — room layout, entity placement, scenario coverage, and the human tester walkthrough flow. -**Current state:** -- Server: 352 unit tests, 9 integration tests (`tests/` dir), all pass -- Client: 16 test files using gdUnit4, untested headless reliability -- Cross-boundary: `bridge_tcp.rs` tests roundtrip serialization, but nothing tests the full `client sends input → server processes → server sends snapshot → client receives` loop -- No test covers the bevy schedule execution order -- No test covers non-blocking socket behavior under load - -**Questions for Tyre:** -1. What's the minimum viable Layer 3 test? A Rust test that spawns a real server, connects a mock client over TCP, sends input, and verifies a snapshot comes back — or something lighter? -2. Should bevy schedule ordering be tested explicitly (e.g., assert that `compute_observer_snapshot` runs after `validate_movement`), or is the existing system ordering via `.after()/.before()` sufficient? -3. The server game loop (`loop { app.update() }`) is untestable as-is — it's in `main()`. Should we extract a `GameLoop` struct with `tick()` that integration tests can drive? -4. How do we test the non-blocking TCP behavior? The fix toggles between blocking/non-blocking per operation — what invariants should tests check? +**Questions for Gestalt:** +1. Review the proposed room list. What mechanical interactions are missing? Think about system *combinations* — what happens when the player sprints through the Crowd Plaza into the Occlusion Corridor? When they try to interact while in peripheral vision? +2. What makes a good "stress test" room? Maximum entity density? Maximum interaction depth? Both? +3. The Gauntlet is stable, but dynamic maps need fuzzy tests. What invariants should hold for ANY valid map? (e.g., "player spawn is always reachable", "at least one NPC is within interaction range within 10 tiles of spawn") **Questions for Dudley:** -1. The `smoke.rs` test boots the world and ticks — but doesn't verify snapshots are produced. What's the minimal extension to catch bug #1? -2. `process_player_input` has no test for pause-state filtering. Propose the test cases for the pause guard (movement discarded, unpause accepted, stance toggle during pause — allowed or not?). -3. Content loading fails silently (`Failed to load content: manifest not found`). Should this be a test failure in integration tests, or is graceful degradation correct? -4. The `EntityRegistry` ↔ `StableId` mapping is a correctness boundary — are there enough tests for entity lifecycle (spawn, register, lookup, despawn)? +1. The current proof room (`spawn_proof_room` in `content_loader.rs`) is a 30x30 box with 3 NPCs. How do we extend this into the Gauntlet? Separate content YAML, or a programmatic builder function? +2. Each room needs a known "tick 0 state" for golden file comparison. How do we snapshot this — serialize the full ECS world, or just the ObserverSnapshot from a fixed player position? +3. The Gauntlet needs entities with specific knowledge graph states (contradiction for Dialogue Room, different trust tiers). How do we inject these at load time? -### Track 2: Serialization Boundary Tests (Hoshe, Dudley) +**Questions for Tyre:** +1. Should the Gauntlet be a content pack (YAML/RON loaded by the content loader) or a Rust builder function (like the current proof room)? Content pack is more maintainable but harder to set up precise ECS state. +2. The "reserved Zone Gate" — what's the minimum architectural provision? A marker in the map? An empty module with trait stubs? +3. Performance budget: what tick time is acceptable for the Gauntlet with 15+ NPCs? When should we raise an alarm? -The MessagePack off-by-one (bug #4) is a classic boundary value error. The project has TWO independent serialization paths: Godot `messagepack.gd` (client encode) and Rust `rmp_serde` (server decode). They must agree on every value. +**Questions for Ozzie:** +1. A human tester walks through the Gauntlet room by room. What's the optimal flow? Linear corridor connecting rooms, or hub-and-spoke from a central area? +2. What should the tester see on screen that tells them "this room is testing X"? Signs? HUD overlay? A printed checklist they follow? +3. The text output mode (see Track 3) replaces visual rendering with structured text. What information does a human tester need in text form to verify "this looks correct"? + +### Track 2: Deterministic Gameplay (Tyre, Dudley) + +The project uses `SimRng` (ChaCha20, seeded) for randomness, but full determinism — same inputs + same seed = identical game state — has not been validated or hardened. This track evaluates whether to commit to determinism now. + +**The position:** Hardening determinism now is cheaper than retrofitting later. Every system added without determinism constraints makes the retrofit harder. But there may be gameplay or performance costs. + +**Questions for Tyre:** +1. **Pros and cons of committing to determinism now.** What breaks? What do we gain? What's the ongoing maintenance cost? Consider: HashMap iteration order (non-deterministic in Rust), floating-point operations (platform-dependent), bevy system ordering (parallel execution), external I/O timing. +2. Where is determinism already broken today? Audit the critical path: `process_player_input` → `validate_movement` → `compute_observer_snapshot`. Which of these use HashMap, f32 arithmetic, or unordered queries? +3. What would a "determinism test" look like? Run the Gauntlet with seed X and inputs [A, B, C], serialize world state at tick 50, compare against golden file. Is this feasible with bevy_ecs? +4. Performance implications: replacing HashMap with BTreeMap everywhere? Deterministic float alternatives? + +**Questions for Dudley:** +1. Which ECS queries are order-dependent today? If two NPCs are equidistant from the player, does the system process them in a stable order? +2. The `SimRng` is seeded but is it consumed in a deterministic order? If system execution order varies, RNG calls happen in different order → different outcomes. +3. What's the minimum change set to make the server deterministic for the Gauntlet? Can we scope it to "deterministic for single-player, single-thread" as a starting point? + +### Track 3: Test Automation & Text Renderer (Tyre, Dudley, Stig) + +Two automation tools: scripted input replay with snapshot assertions, and a text-based game state renderer for human verification without a GPU. + +**Scripted replay concept:** +``` +# test_inventory_full.replay +@gauntlet seed=42 +spawn_at inventory_warehouse_entrance +move_to crate_1 # pathfind or explicit directions +interact crate_1 Take +assert inventory.count == 1 +repeat 8: interact crate_N Take # fill all 9 slots +interact crate_10 Take +assert inventory.count == 9 # didn't overflow +assert snapshot.nearby_interactions[crate_10].verbs contains "Take" +assert snapshot.nearby_interactions[crate_10].verbs["Take"].available == true +# ^ server should still offer Take even when inventory full — client greys it out +``` + +**Text renderer concept:** +``` +=== Tick 42 | Gauntlet: Occlusion Corridor === +Player (15,10) facing East | Stance: Walk | TickRate: Full +Visible entities: + npc:guard-1 (18,10) Forward relationship:Neutral visible + npc:worker-2 (20,10) Forward relationship:Unknown remembered + [wall at (17,10) blocks LOS to npc:hidden-1 at (19,12)] +Fog: 31 visible, 58 explored, 412 unexplored +Interactions: guard-1 [Talk(1), Observe(2)] distance=3 +Inventory: 2/9 [keycard(slot-0), manifest(slot-3)] +Monologue: none | Dialogue: none +``` + +**Questions for Tyre:** +1. Scripted replay requires a "test client" — a Rust program that connects to the server over TCP, sends scripted inputs, and reads snapshots. Is this a new binary, a test harness in `tests/`, or an extension of the existing `game_loop` integration test? +2. The text renderer — should it live server-side (format the ObserverSnapshot before sending), client-side (format after receiving), or as a standalone tool that reads snapshots from a file/pipe? +3. What assertion language? Custom DSL (like above), Rust test macros, or a data-driven approach (YAML expected-state files compared against actual)? + +**Questions for Dudley:** +1. The server already produces ObserverSnapshot with all the data needed for text rendering. What's missing? (Hint: wall positions in the LOS path aren't in the snapshot — only visible tiles.) +2. The scripted replay needs to inject inputs at specific ticks. The current `InputQueue` accepts `PlayerInput { tick, action }`. Can we pre-load a sequence from a file? +3. How do we handle "wait for condition" in scripts? (e.g., "wait until cognitive delay completes" — variable tick count depending on entity distance) + +**Questions for Stig:** +1. The text renderer replaces visual output for automated testing. But human testers also need a **checklist** — "in the Occlusion Corridor, verify: NPC behind wall is NOT visible, NPC in front of wall IS visible, peripheral NPC is dimmed." Where does this checklist live? In the Gauntlet spec? Generated from room metadata? +2. Client-side text rendering: should the client have a `--text-mode` flag that replaces the Godot renderer with a terminal-output formatter? Or is this purely a server-side tool? +3. The client tests (gdUnit4) are structural — they check scene tree state, not pixels. What client-side properties are worth asserting beyond what the text renderer shows? (Camera position, z-layer ordering, node visibility, modulate alpha values?) + +### Track 4: Serialization & Integration Testing (Hoshe, Dudley) + +The MessagePack boundary bug (bug #4) was fixed across all int sizes. This track ensures it never recurs and extends coverage to the full wire protocol. **Current state:** - `test_protocol.gd` and `serialization.rs` test roundtrips within their respective languages - `bridge_tcp.rs` tests cross-language roundtrip for snapshots and inputs -- No test specifically exercises boundary values (128, 256, 32768, 2^31, etc.) -- The messagepack.gd encoder has unfixed boundary bugs at int16, int32, int64 boundaries (same pattern as the int8 bug) +- Boundary value bugs at int8/16/32/64 are now fixed, but no tests specifically exercise boundary values +- `gen_fixtures.rs` generates MessagePack fixtures but not boundary value fixtures **Questions for Hoshe:** -1. Design a boundary value test matrix for the MessagePack encoder. Which values must be tested? (Hint: every power-of-two boundary where the format changes, both positive and negative.) +1. Design a boundary value test matrix for the MessagePack encoder. Every power-of-two boundary where the format changes, both positive and negative. Include the exact values. 2. Should boundary tests live in the client (GDScript), server (Rust), or both? Cross-language roundtrip tests catch the real bugs but are slower. 3. The `gen_fixtures.rs` test generates MessagePack fixtures — should it generate boundary value fixtures that the client can verify? -4. Propose a "golden file" approach: server generates canonical MessagePack bytes for known inputs, client verifies it decodes to the same values. How would this work in CI? +4. Propose a "golden file" approach tied to the Gauntlet: server generates canonical MessagePack bytes for a known Gauntlet snapshot, client verifies it decodes to the same values. How would this work in CI? +5. D-030 defined three IPC test layers. Layer 1 (fixtures) and partial Layer 2 (bridge) exist. Layer 3 (real subprocess integration) is missing. What's the minimum viable Layer 3 test? **Questions for Dudley:** 1. The Rust `PlayerInput` uses `tick: u64` but the client sends Godot's signed `int` (63-bit range). What's the actual valid range, and should the server reject negative ticks explicitly? 2. `rmp_serde` silently rejects `-128` for a `u64` field. Should the bridge layer pre-validate and log, or is the current error-and-skip behavior acceptable? +3. `process_player_input` has no test for pause-state filtering. Propose the test cases for the pause guard (movement discarded, unpause accepted, stance toggle during pause — allowed or not?). +4. The `EntityRegistry` / `StableId` mapping is a correctness boundary. Are there enough tests for entity lifecycle (spawn, register, lookup, despawn)? -### Track 3: Client Test Infrastructure (Stig, Hoshe) +### Track 5: Content Scaling & CI Pipeline (Hoshe, Justine, Tyre) -The client has 16 test files but reliability is uncertain. gdUnit4 headless mode needs validation. Scene-level tests (camera, renderer, input pipeline) are the gap that let bugs #2, #3, and #5 through. +When content scales from 3 NPCs to 30, from 1 district to 5, from 9 items to 50 — what breaks? This track designs the safety net. -**Current state:** -- `make test-client` invokes gdUnit4 headless runner -- Tests use `GdUnitTestSuite` base class -- `test_camera_anchor.gd` was written this sprint (10 tests, all pass) but only tests test-mode (not live TCP mode) -- No test for the `_process()` loop in `main.gd` -- No test for the snapshot event carry-forward (bug #5 fix) -- No test for pause toggle (InputMapper → SimBridge → wire encoding) - -**Questions for Stig:** -1. `main.gd._process()` drives the entire client game loop. What's testable here? Can we use `GdUnitSceneRunner` to drive the scene, inject mock snapshots, and verify camera + renderer state? -2. The snapshot carry-forward logic (monologue/dialogue preserved across overwrites) is in `sim_bridge.gd`. Propose test cases. -3. InputMapper's pause toggle reads `GameState.game_time.tick_rate` to decide Pause vs Unpause. How do we test this state-dependent branching? -4. The cursor renderer does world-space hit detection via `get_canvas_transform()`. Is this testable in headless mode (no GPU)? +**Content validation layers:** +1. **Schema validation** (exists: `make validate-content`) — YAML structure correctness +2. **Cross-reference validation** (partial) — entity refs resolve, fact_ids exist, location slugs match, dialogue pool tags use valid enums +3. **Load-test validation** (doesn't exist) — boot the server with a stress content pack, tick 100 times, no panics, snapshots arrive within frame budget +4. **Regression snapshots** — the Gauntlet golden file: seed X produces this exact snapshot at tick 10. Any code/content change that breaks it is flagged. **Questions for Hoshe:** 1. The client tests don't run in CI yet. What's the blocker — headless Godot availability, test runner stability, or just wiring? -2. Propose a minimal CI pipeline: which tests are fast enough to run on every commit, which are PR-only, which are nightly? -3. gdUnit4 `GdUnitSceneRunner` can simulate frames. Design a test that catches bug #2: scene loads → no snapshot → first snapshot arrives → camera anchors to player position (not 0,0). +2. Propose a minimal CI pipeline: which tests run on every commit, which are PR-only, which are nightly? +3. The current `make validate-content` checks YAML schema but not runtime behavior. What's the minimum runtime validation — "boot server, load content, tick once, no panics"? +4. Content scaling: what's the test for "adding a new NPC to the transit district doesn't break anything"? Load the district, verify entity count, verify interactions still work? -### Track 4: Test Gaps & Priority (All) +**Questions for Justine:** +1. The server runs `cargo nextest` for Rust tests. The client runs `make test-client` for gdUnit4. Neither runs in CI. What's the minimum CI pipeline that runs both? +2. The Gauntlet golden file test needs both server build + test world content. How should CI manage this artifact? Build once, cache, reuse across test jobs? +3. Performance regression detection: the Gauntlet should complete 100 ticks within a time budget. How do we track this across commits without noise from CI machine variance? +4. When a golden file breaks, the developer needs to see *what* changed. Diff format for ObserverSnapshot comparison — structured diff, or just "expected vs actual" dump? -D-030 defined test phases but we're past the timeline. Recalibrate. - -**D-030 phases vs reality:** -- Phase 1 (sprint 1-2): test infra + collision/pathfinding/time — **DONE** (server-side) -- Phase 2 (sprint 3-4): monologue pipeline integration + info boundary negative tests — **NOT DONE** -- Phase 3 (sprint 5+): CauseChain verification + divergent snapshots — **NOT STARTED** - -**Questions for all:** -1. Given the bug catalogue above, what test would you write FIRST if you could only write one? -2. What's the highest-risk untested boundary in the codebase right now? -3. The server has 352 unit tests but zero tests for the observer pipeline producing correct snapshots from a known world state. Is this the biggest gap? -4. Should we invest in deterministic replay testing (D-030 #7) now, or is it still premature? +**Questions for Tyre:** +1. The Gauntlet golden file is the anchor for regression testing. What format — serialized ECS world state, or ObserverSnapshot at a fixed position? Snapshot is smaller and closer to what the client sees; world state catches server-only bugs. +2. Fuzzy tests for dynamic maps: what invariants should always hold regardless of procedural generation? (Reachability, minimum NPC count, spawn safety, no overlapping entities?) +3. How do we test content *combinations*? NPC A's dialogue references NPC B's secret — if B is removed from the content pack, A's dialogue breaks. Is this a schema problem, a load-time validation problem, or a runtime test? ## Workshop Format **Round 1 (analysis):** Each participant answers their track questions independently. Reference existing code, D-030, and the bug catalogue. Produce concrete test specifications (function names, assertions, setup), not general principles. -**Round 2 (synthesis):** Cross-review. Tyre validates Stig's client test proposals for architectural soundness. Hoshe validates Dudley's server test proposals for coverage completeness. All participants rank the proposed tests by bug-prevention value. +*Pause after Round 1 for lead review of summaries, open decisions, and questions.* -**Output:** -- Prioritized test backlog (tickets) covering the bug catalogue -- Concrete test specifications for the top 10 tests -- CI pipeline design (what runs when) -- Decision on Layer 3 integration test approach +**Round 2 (synthesis):** Cross-review and integration. Tyre validates Stig's client proposals for architectural soundness. Hoshe validates Dudley's server proposals for coverage completeness. All participants weigh in on the deterministic gameplay decision. Reconcile overlapping proposals across tracks. + +*Pause after Round 2 for lead review.* + +**Round 3 (prioritization):** Rank all proposed tests and infrastructure by implementation priority. Produce the final test backlog, concrete specs for top implementations, and the CI pipeline design. + +*Pause after Round 3 for lead review and closing.* + +## Output + +- Decision on deterministic gameplay commitment (with documented pros/cons) +- Gauntlet test world specification (room layout, entity placement, golden file format) +- Prioritized test backlog (tickets) covering the bug catalogue + content scaling +- Concrete test specifications for the top implementations +- Text renderer specification +- Human tester checklist template for the Gauntlet walkthrough +- CI pipeline design (what runs when, what gates what) +- Scripted replay framework specification +- Content validation pipeline design diff --git a/docs/workshops/test-architecture/tyre-round2.md b/docs/workshops/test-architecture/tyre-round2.md new file mode 100644 index 000000000..d0c4a9f09 --- /dev/null +++ b/docs/workshops/test-architecture/tyre-round2.md @@ -0,0 +1,405 @@ +# Tyre — Round 2 Analysis: Test Client Binary, Cross-Review, Open Questions + +**Workshop:** QA Strategy & Test Architecture +**Round:** 2 (Cross-review and integration) +**Date:** 2026-02-17 +**Inputs:** Stig Round 1 (32 client tests, text renderer), Dudley Round 1 (Gauntlet, determinism, replay, serialization), Hoshe Round 1 (boundary values, Layer 3, CI), Justine Round 1 (CI pipeline, perf regression, golden files) + +--- + +## 1. Test Client Binary Architecture + +*The lead overruled my Round 1 recommendation of a server-side text renderer. The reasoning is sound: a separate binary avoids server bloat, keeps the testing landscape clean, and — critically — exercises the real TCP boundary. I argued against it on simplicity grounds, but the architectural purity argument wins. Let me design it properly.* + +### Where It Lives + +**`server/src/bin/test_client.rs`** — a second binary target in the server crate. + +Add to `server/Cargo.toml`: +```toml +[[bin]] +name = "settled-reach-test-client" +path = "src/bin/test_client.rs" +``` + +**Rationale for same crate, separate binary:** + +| Option | Pros | Cons | +|--------|------|------| +| `server/src/bin/test_client.rs` | Shares bridge types + framing code. Zero new crate management. Built with `cargo build --bin settled-reach-test-client`. | "In the server crate" may look like server bloat at first glance. | +| New workspace crate `test-client/` | Clean separation. Can't accidentally use server internals. | New crate boilerplate, workspace config, dependency management. Types need to be either duplicated or extracted to a shared crate. | +| `tooling/test-client/` | Consistent with project layout. | Same dependency issues as workspace crate. Not under `cargo` workspace. | + +The second binary target wins because: +1. **Shared types are the point.** The test client needs `ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed` — all defined in the server crate's library. A separate crate would need to depend on the server crate anyway (or we'd extract a `protocol` crate, which is premature). +2. **No server bloat.** Rust compiles each binary target independently. Code in `test_client.rs` doesn't end up in the `settled-reach-server` binary. Dead code elimination handles shared library code that only the test client calls. +3. **Build integration.** `cargo build` builds both binaries. `cargo test` runs tests in both. `cargo nextest` picks up integration tests that use either binary. + +### How It Connects + +``` +┌────────────────────┐ TCP ┌────────────────────────┐ +│ settled-reach- │◄───────────►│ settled-reach-test- │ +│ server │ framed │ client │ +│ │ msgpack │ │ +│ --test-mode │ │ --connect host:port │ +│ --port 0 │ │ --text (stdout render) │ +│ --seed 42 │ │ --ticks 50 │ +└────────────────────┘ │ --replay inputs.jsonl │ + │ --golden tick50.json │ + └────────────────────────┘ +``` + +**Connection protocol:** +1. Test client connects to server's TCP port +2. Each tick: client sends `Vec` via `write_framed`, server responds with `ObserverSnapshot` via `send_bridge_snapshot` +3. Test client deserializes snapshot, formats text output, optionally compares against golden file +4. After `--ticks N`, client disconnects cleanly + +**CLI interface:** + +``` +settled-reach-test-client [OPTIONS] + +Connection: + --connect Connect to running server (default: 127.0.0.1:9876) + +Input: + --replay Send inputs from file (one JSON PlayerInput per line) + --interactive Read inputs from stdin (future: human-in-the-loop testing) + +Output: + --text Render each snapshot as structured text to stdout + --json Dump each snapshot as JSON to stdout (for golden files) + --quiet No output (assertions only, for CI) + +Assertions: + --golden Compare final snapshot against golden file, exit 1 on diff + --ticks Disconnect after N ticks (default: unlimited) + +Debugging: + --dump-raw Hex dump raw MessagePack bytes before deserializing +``` + +### Text Output Format + +``` +=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === +Game time: Day 0, 04:12 (Morning) +Entities (5): + npc:100 (18,10) Forward rel:Neutral vis:Visible + npc:101 (20,10) Forward rel:Unknown vis:Remembered + obj:200 (16,9) Forward rel:n/a vis:Visible + npc:102 (12,8) Periph rel:Hostile vis:Visible + npc:103 (22,14) Periph rel:Friendly vis:Visible +Tiles: 31 visible +Pending recognitions: 1 [npc:104 at (19,12) 3/8 ticks] +Interactions (2): + npc:100 [Talk(1), ExamineNpc(2)] distance=3 + obj:200 [Read(1), Observe(2)] distance=1 +Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] +Monologue: "Something about this manifest doesn't add up." +=== +``` + +**Format rules:** +- Entities identified by `kind:entity_id` (e.g., `npc:100`, `obj:200`). No display names on the wire — see OQ-5 answer below. +- One line per entity, sorted by distance from player (nearest first). +- `===` tick separators for clean `diff` output. +- Positions as integer tile coords (the f32 render offset is irrelevant for testing). +- All fields directly from ObserverSnapshot — no additional server data needed. + +### Text Renderer Implementation + +```rust +// server/src/bridge/text_renderer.rs (library code, used by test client binary) + +/// Format an ObserverSnapshot as structured text for human verification. +/// The test client binary calls this; the server binary does not. +pub fn format_snapshot_text(snapshot: &ObserverSnapshot) -> String { + let mut out = String::with_capacity(2048); + // Header + writeln!(out, "=== Tick {} | Player ({},{}) facing {:?} | Stance: {:?} | TickRate: {:?} ===", + snapshot.tick, + snapshot.entities.iter().find(|e| matches!(e.kind, EntityKind::Player)) + .map(|p| p.x as i32).unwrap_or(-1), + snapshot.entities.iter().find(|e| matches!(e.kind, EntityKind::Player)) + .map(|p| p.y as i32).unwrap_or(-1), + snapshot.player_facing, + snapshot.player_stance, + snapshot.game_time.tick_rate, + ).ok(); + // ... entities, tiles, interactions, inventory, monologue + writeln!(out, "===").ok(); + out +} +``` + +This lives in the server crate's *library* (not in any binary). The test client binary calls it. Integration tests can also call it for debug output. The server binary never references it — zero bloat. + +### Integration with cargo nextest (Layer 3) + +The test client binary enables proper Layer 3 subprocess testing. Integration tests spawn both binaries: + +```rust +// server/tests/layer3_subprocess.rs + +#[test] +#[ignore] // Slow — nightly/pre-merge only +fn server_and_test_client_subprocess_roundtrip() { + // 1. Build both binaries + // 2. Launch server: settled-reach-server --test-mode --port 0 --seed 42 + // 3. Parse port from server stdout + // 4. Launch test client: settled-reach-test-client --connect 127.0.0.1:{port} + // --replay gauntlet_basic.jsonl --ticks 10 --golden tick10.json --quiet + // 5. Test client exits 0 if golden file matches, 1 if not + // 6. Kill server, assert clean exit +} +``` + +This is the test that would have caught Bug #1 (blocking TCP read stalling bevy Update). The test client is a real subprocess exercising the real wire protocol — not an in-process mock. + +### Difficulty Tier + +**Feasible. ~2-3 days total.** +- Binary scaffolding + CLI parsing: 0.5 day +- Text renderer function: 0.5-1 day +- Replay file loading + tick-scheduled sending: 0.5 day +- Golden file comparison (JSON deserialize + field diff): 0.5 day +- Layer 3 integration test wiring: 0.5 day + +--- + +## 2. Stig's 32 Client Test Proposals — Architectural Validation + +*Stig proposed 32 tests across 6 categories. All structural scene tree assertions, no pixel comparison. Let me evaluate each category for soundness, redundancies, and gaps.* + +### Camera System (7 tests) — APPROVED + +All 7 are architecturally sound: +- Position after ready, anchored flag, smoothing lifecycle, follows movement, static during pause, handles rapid snapshots + +**No redundancies.** Each tests a distinct camera behavior. The "static during pause" test directly prevents Bug #2 class regressions (camera doesn't update when no snapshots arrive). + +**One refinement:** The "handles rapid snapshots" test should verify that the camera smoothing doesn't overshoot or oscillate when snapshots arrive faster than the lerp completes. Assert convergence within a frame budget, not just "doesn't crash." + +### Entity Rendering (7 tests) — APPROVED with note + +- Peripheral alpha == 0.5, forward alpha == 1.0, D-033 colors by kind, lifecycle (create/remove) + +**Sound.** These directly test the information-to-visual mapping that is the client's core responsibility. + +**Refinement:** The lifecycle test should explicitly verify that entities leaving LOS are removed from the scene tree (not just hidden). Godot nodes that are hidden but not freed accumulate memory. Assert `get_node_or_null()` returns null after LOS exit, not just `visible == false`. + +### Z-Layer Ordering (4 tests) — APPROVED, low priority + +- Fog rect z=900, fog entities z=950, insert canvas=10, UI canvas=20 + +**These are effectively constant checks.** They verify the z-layer conventions documented in the rendering architecture. Worth having as a safety net, but low regression risk — z-layer values don't change accidentally. + +**No redundancy** with entity rendering tests. Z-layers are about ordering between *categories* (fog vs entities vs UI), not about individual entity rendering. + +### Fog Shader State (3 tests) — APPROVED, P1 + +- Visibility texture updates from positions, exploration persistence (255 -> 128), peripheral dimming (180) + +**Critical for information boundaries.** The fog system is the primary visual enforcement of D-010's asymmetric information principle. If fog breaks, the player sees things they shouldn't. + +**The byte values (255/180/128/0) should be named constants in the fog shader code.** Stig's open question to Hoshe is correct — these are assertion targets and should be documented as such. Recommend: `const FOG_VISIBLE: int = 255`, `const FOG_PERIPHERAL: int = 180`, `const FOG_EXPLORED: int = 128`, `const FOG_UNEXPLORED: int = 0`. + +### UI Elements (8 tests) — APPROVED, P0 for monologue + +- Monologue consumed once per tick, monologue not lost on overwrite (Bug #5), interaction list, dialogue, inventory grid, stance indicator + +**Bug #5 regression guard (monologue not lost on overwrite) is the highest-priority client test in the entire suite.** This was a real bug that the carry-forward fix addressed, and the test should be the first one written. + +**Sound overall.** The interaction list test should verify sorting order (nearest first, matching server `nearby_interactions` order). + +### Entity Lerp (3 tests) — APPROVED + +- Target set on update, snap on first appearance, convergence after N frames + +**Sound.** The "snap on first appearance" test is subtle and important — entities should appear at their correct position immediately, not lerp from (0,0). This was a visual glitch pattern in early development. + +### Summary: Missing Coverage + +Stig's 32 tests cover the client's core rendering and UI responsibilities well. Gaps I'd add: + +| # | Missing Test | Category | Why | +|---|-------------|----------|-----| +| 33 | **Pending recognition blob rendering** | Entity rendering | Cognitive delay (D-060) renders entities as grey blobs. Test that `pending_recognitions` entries appear as blobs, not full entities. Bug #7 class prevention. | +| 34 | **Recognition transition animation** | Entity lerp | When recognition completes (entity moves from `pending_recognitions` to `entities`), verify visual transition from blob to full entity over ~0.3s. | +| 35 | **Tick rate HUD indicator** | UI elements | GameTime.tick_rate changes should update the HUD. Test Full/Half/Paused display. | +| 36 | **Inventory full visual state** | UI elements | When inventory is 9/9, verify visual feedback (e.g., slot highlight change, "Full" indicator). | +| 37 | **Sprint interaction suppression** | UI elements | During Sprint stance, interaction buffer is suppressed (D-055). Verify interaction list is hidden/empty during Sprint. | +| 38 | **Entity modulate for Remembered state** | Entity rendering | Entities with `observation: Remembered` should render differently from `Visible` (e.g., translucent, desaturated). Test the modulate/shader difference. | + +**Total with additions: 38 tests.** All structural, all scene tree assertions. No pixel comparison. + +### Priority Ranking for Implementation + +| Priority | Tests | Count | Rationale | +|----------|-------|-------|-----------| +| P0 | Monologue not lost on overwrite (Bug #5) | 1 | Direct regression guard for shipped fix | +| P0 | Camera static during pause (Bug #2) | 1 | Direct regression guard | +| P1 | Fog shader state (all 3) | 3 | Information boundary enforcement | +| P1 | Entity lifecycle (create/remove) | 1 | Memory + correctness | +| P1 | Pending recognition blob | 1 | D-060 cognitive delay visual | +| P2 | Remaining camera tests (5) | 5 | Camera behavior suite | +| P2 | Entity alpha + color (4) | 4 | Visual fidelity | +| P2 | UI elements (remaining 7) | 7 | HUD and interaction | +| P3 | Z-layer ordering (4) | 4 | Constants checks | +| P3 | Entity lerp (3) | 3 | Animation refinement | +| P3 | Added tests 34-38 | 5 | New coverage | + +--- + +## 3. OQ-4: WalkabilityMap.chunks HashMap -> BTreeMap? + +**Answer: No. Leave it as HashMap.** + +Dudley asked this because `WalkabilityMap.chunks: HashMap` could become a determinism hazard if we ever iterate chunks (e.g., for save/load serialization). + +**Current usage audit (confirmed in Round 1):** +- `can_move_to(pos)` — point lookup via `self.chunks.get(&chunk_coord)` → safe +- `set(pos, walkable)` — point insertion via `self.chunks.entry(coord).or_insert_with(...)` → safe +- `WalkabilityMap::new()` — constructs with pre-allocated chunks → deterministic (insertion order = grid iteration order, but irrelevant since we never iterate) +- Nowhere in the codebase does any code call `.iter()`, `.keys()`, or `.values()` on `chunks` + +**Why not convert preemptively:** +1. **HashMap is faster for point lookups.** The WalkabilityMap is queried in the movement validation hot path (`validate_movement` calls `can_move_to` for every mover every tick). HashMap lookup is O(1) amortized vs BTreeMap's O(log N). With N=25 chunks, that's ~1 comparison vs ~5 comparisons. Per-call it's nanoseconds, but it's called hundreds of times per tick during pathfinding. +2. **No iteration today, no iteration planned.** Save/load is Sprint 10+ scope. When it arrives, the save system should collect chunks into a `BTreeMap` or sorted `Vec` for serialization — the WalkabilityMap's internal HashMap doesn't need to change. +3. **The determinism hygiene rule covers this.** The rule is: "HashMap/HashSet for point lookups only, BTreeMap/BTreeSet where iteration order affects output." WalkabilityMap falls cleanly in the first category. + +**Action:** Document in the determinism hygiene section that WalkabilityMap.chunks is a known-safe HashMap usage. If anyone adds iteration over chunks, they must convert to BTreeMap first. + +--- + +## 4. OQ-5: Entity display_name — Wire Protocol or Server-Side Lookup? + +**Answer: Neither for now. Use `kind:entity_id` labels in the test client.** + +The lead's hint is correct: since the test client is Rust code in the same crate, it has access to content data. But accessing content data from the test client creates two sources of truth for entity identity (wire ID vs content canonical_id) and requires the test client to load content — which adds variables to the testing landscape (the exact thing the lead wants to avoid). + +**For the test client text renderer:** + +Entities are labeled as `kind:entity_id`, e.g., `npc:100`, `obj:200`, `player:0`. This is: +- Unambiguous: the wire entity_id is the canonical identifier in the snapshot +- Zero-cost: no additional data needed, no content loading +- Stable: doesn't break when content names change +- Sufficient: Gauntlet test assertions use named constants that map to known IDs + +```rust +// In text renderer +fn entity_label(entity: &VisibleEntity) -> String { + let kind = match entity.kind { + EntityKind::Player => "player", + EntityKind::Npc => "npc", + EntityKind::Object => "obj", + EntityKind::Terrain => "terrain", + }; + format!("{}:{}", kind, entity.entity_id) +} +``` + +**For test assertions, the Gauntlet coordinate constants provide the human-readable mapping:** + +```rust +// server/src/content/gauntlet/constants.rs +pub const GUARD_1: GauntletEntity = GauntletEntity { + wire_id: 100, // Assigned by EntityRegistry in deterministic spawn order + name: "guard-1", + position: TilePosition::new(18, 10, 0), +}; +``` + +Test code reads naturally: `assert_entity_visible(&snapshot, GUARD_1.wire_id, GUARD_1.position)`. + +**When to add display_name to the wire protocol:** + +Add `display_name: Option` to `VisibleEntity` when the Godot client needs to render entity name labels in the UI (e.g., hovering over an NPC shows their name). That's a gameplay feature (likely Sprint 9-10), not a testing requirement. When it ships, the test client gets it for free. + +**Why not add it now:** +1. The lead explicitly wants to avoid adding variables to the testing landscape. A wire protocol change is a variable. +2. display_name involves information boundary decisions: does the player see the name before recognition completes? Before first conversation? These are gameplay questions, not testing questions. +3. `kind:entity_id` is honest — it shows exactly what the wire protocol contains, which is what we're testing. + +--- + +## 5. Cross-Review: Hoshe's CI Tier Proposal + +*Hoshe proposed three tiers: Commit (lint + content, <2min), PR (build + test + fixtures, <10min), Nightly (Layer 3 + golden + perf, <30min). Justine proposed a compatible pipeline with parallel server/client jobs, fixture artifacts, and performance baselines. The lead confirmed CI is deferred (manual `make ci` stays), so this is future design validation.* + +### Tier Structure: APPROVED + +The three-tier model (Commit / PR / Nightly) is the right design. Every tier has a clear purpose: + +| Tier | Purpose | Gate? | Hoshe's Budget | My Assessment | +|------|---------|-------|---------------|---------------| +| Commit | Fast syntax + content feedback | No (advisory) | <2 min | Correct. Lint + content validation catches the most common errors with the lowest cost. | +| PR | Merge gate — build + test + fixture | Yes (blocks merge) | <10 min | Correct, but budget <15 min — full rebuilds from cache miss take 5-7 min for server alone. | +| Nightly | Deep integration + regression | No (alerts) | <30 min | Correct. Layer 3 subprocess + golden files + perf belong here. | + +### Specific Feedback + +**Commit tier — good as-is.** `make lint-server`, `make lint-client`, `make validate-content`, `make check-fact-ids`. Fast, catches the obvious stuff. + +**PR tier — one addition.** Hoshe's fixture staleness check (`make fixtures && git diff --exit-code client/tests/fixtures/`) is excellent. This catches protocol changes where the developer forgot to regenerate fixtures. Justine's parallel server/client jobs with fixture artifact passing is the correct CI implementation. + +**Add to PR tier:** Content cross-reference validation (the new `validate_cross_references()` function from the lead's confirmed decisions). This should run after build, before tests. Budget impact: <2s, negligible. + +**PR tier time budget:** Revise from <10 min to **<15 min**. Rationale: +- Server build (clean cache): ~5-7 min on a self-hosted runner +- Server tests (cargo nextest): ~1-2 min +- Client build: ~30s (Godot project scan) +- Client tests (gdUnit4 headless): ~1-2 min +- Fixture generation + staleness check: ~10s +- Content validation: ~5s +- With caching: ~3-5 min total. Without caching: ~12 min. +- 15 min budget covers the worst case without caching. + +**Nightly tier — one addition.** Add a **content scaling test**: boot the server with a stress content pack (Crowd Plaza density: 15+ NPCs), tick 100 times, verify no tick exceeds the p95 budget. Hoshe's "Level 5 stress" test from T5-H3 belongs here. + +### Merge-Blocking Policy — Alignment with Justine + +Hoshe and Justine are aligned on the merge policy: + +| Check | Hoshe | Justine | My Assessment | +|-------|-------|---------|---------------| +| Test failures | BLOCKER | BLOCKER | Correct | +| Lint failures | BLOCKER | BLOCKER | Correct | +| Content validation | BLOCKER | BLOCKER | Correct | +| Golden file diff | Not explicit | WARNING | **Should be WARNING**, not blocker. Golden files change legitimately when behavior changes. Require reviewer ack. | +| Fixture staleness | Not explicit | WARNING | **Should be BLOCKER.** Stale fixtures mean the client is testing against outdated protocol data. This is a real correctness bug, not a warning. | +| Performance delta | Not explicit | INFO | Correct — CI variance makes perf unreliable as a gate. | + +**Fixture staleness should be a BLOCKER, not a WARNING.** If `make fixtures && git diff --exit-code` fails, it means the server's wire format changed but the fixtures weren't regenerated. The client tests are running against stale data — any passing client test is a false positive. This is the exact class of bug (protocol mismatch) that the entire serialization testing track is designed to prevent. + +### CI Readiness When We Wire It Up + +Hoshe correctly identified that there is no CI pipeline at all — no `.gitea/workflows/`. The Gitea instance at `git.schweitz.internal` supports Gitea Actions (GitHub Actions compatible), and we have self-hosted infrastructure. When the lead greenlights CI: + +1. Start with the PR tier only (merge gate). This gives the most value for the least setup. +2. Add Commit tier for fast feedback on feature branches. +3. Add Nightly tier last — it requires the Gauntlet test world to exist first. + +**Estimated wiring effort:** ~1 day for a Gitea Actions workflow that runs `make ci`. The Makefile targets already exist. + +--- + +## Summary: Round 2 Deliverables + +| # | Deliverable | Status | +|---|------------|--------| +| 1 | Test client binary architecture (location, CLI, text format, Layer 3 integration) | Complete | +| 2 | Stig's 32 client tests validated + 6 additions = 38 total with priority ranking | Complete | +| 3 | OQ-4: WalkabilityMap HashMap stays (point-lookup only, no iteration) | Complete | +| 4 | OQ-5: No wire protocol change now. Use `kind:entity_id` labels. Add display_name when client needs name labels. | Complete | +| 5 | Hoshe's CI tier proposal cross-reviewed. APPROVED with: 15min PR budget, fixture staleness → BLOCKER, content scaling in nightly. | Complete | + +## Integration Points with Other Round 2 Outputs + +- **Dudley (Task #10):** The test client binary depends on `--test-mode` + `--port 0` server flags (confirmed for Sprint 8). Dudley's server-side test infrastructure and the test client binary are complementary — the server provides the `run_gauntlet()` headless helper for unit/integration tests, the test client binary provides Layer 3 subprocess testing. +- **Stig (Task #11):** The 6 additional client tests I proposed should be reviewed by Stig for client-side feasibility. The pending recognition blob test (#33) depends on the cognitive delay visual implementation. +- **Hoshe (Task #12):** Fixture staleness as BLOCKER (not WARNING) is a cross-review finding. Hoshe should confirm this aligns with the Layer 1/2/3 testing strategy. +- **Justine (Task #13):** Performance baseline strategy is sound. The self-hosted runner recommendation is critical for stable perf numbers. diff --git a/docs/workshops/test-architecture/tyre-round3.md b/docs/workshops/test-architecture/tyre-round3.md new file mode 100644 index 000000000..86587e422 --- /dev/null +++ b/docs/workshops/test-architecture/tyre-round3.md @@ -0,0 +1,956 @@ +# Tyre — Round 3: Sprint 8 Implementation Plan, Test Client Final Spec, Roadmap + +**Workshop:** QA Strategy & Test Architecture +**Round:** 3 (Prioritization & Final Specs) +**Date:** 2026-02-17 +**Inputs:** All 7 agent Round 2 outputs + Qatux round-2-notes.md synthesis +**OVERRIDE APPLIED:** Lead overruled `server/src/bin/` placement. Test client binary moves to `tooling/test-client/` as a separate Rust crate. + +--- + +## 0. Lead Override: Test Client Location — Architectural Analysis + +*The lead overruled my Round 2 recommendation of `server/src/bin/test_client.rs`. The test client binary goes in `tooling/test-client/` as a standalone Rust crate. Let me be honest about what this means technically.* + +### What Changes + +| Aspect | Round 2 (`server/src/bin/`) | Round 3 (`tooling/test-client/`) | +|--------|---------------------------|----------------------------------| +| Type sharing | Free — same crate | Path dependency: `settled-reach-server = { path = "../../server" }` | +| Build command | `cargo build --bin settled-reach-test-client` | `cd tooling/test-client && cargo build` | +| Binary location | `server/target/debug/settled-reach-test-client` | `tooling/test-client/target/debug/settled-reach-test-client` | +| Compile time | Zero incremental cost (shared compilation) | First build pulls in full server dep tree (~30-60s extra for bevy). Incremental ~2-5s. | +| Cross-binary tests | `cargo test` in server builds both binaries | Requires Makefile orchestration to build both | +| Crate management | None | New `Cargo.toml`, new `Cargo.lock` | + +### Why This Works + +The server crate already has a `lib.rs` that publicly exports everything the test client needs: + +```rust +// server/src/lib.rs — already exists +pub mod bridge; // → bridge::types::*, bridge::framing::* +pub mod knowledge; // → knowledge::types::RelationshipState, EntityVisibility, etc. +pub mod simulation; // → simulation::time::DayPhase, TickRate +``` + +The bridge types used by the test client (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed`) are already `pub`. No server-side changes needed for the test client to import them. + +### Dependency Cost + +The test client crate will transitively depend on the entire server dependency tree: + +``` +settled-reach-test-client + └─ settled-reach-server (path) + ├─ bevy_ecs 0.18 + ├─ bevy_app 0.18 + ├─ rmp-serde 1 + ├─ serde 1 + ├─ pathfinding 4 + ├─ rand 0.9 + └─ ... (~15 transitive deps) +``` + +The test client only needs `serde`, `rmp-serde`, and the wire types. The bevy dependency is dead weight — pulled in because `ObjectType` derives `Component` and `SnapshotBuffer` derives `Resource` in `bridge/types.rs`. + +**Sprint 8 pragmatic choice:** Accept the heavy dependency. Bevy compiles once and caches. Incremental test client builds are fast (~2-5s). Binary size is larger than necessary but DCE removes unused code. + +**Sprint 9+ option:** Extract a `settled-reach-protocol` crate containing only bridge types + framing (no bevy). Both server and test client depend on it. This is the right long-term answer but premature for Sprint 8. Added to roadmap as R-25. + +### Why the Override Is Architecturally Sound + +Despite my Round 2 preference, the lead's reasoning holds: + +1. **`tooling/` establishes a clear convention** — project tools that aren't the game server or game client live here. The test client is a development tool, not a game component. +2. **Separation prevents accidental coupling.** A `server/src/bin/` test client could accidentally use server internals (private modules, internal state). A separate crate can only use what `lib.rs` exports. +3. **Independent release cycle.** The test client can version independently and add its own dependencies (crossterm, clap) without affecting the server's `Cargo.toml`. +4. **Existing pattern.** `tooling/content-converter/` and `tooling/line-previewer/` are already standalone Rust crates in this directory. + +--- + +## 1. Sprint 8 Implementation Plan + +*Sprint 8 ships infrastructure — the plumbing. The Gauntlet rooms are content that flows through the plumbing. Sprint 8 ships the pipes, Sprint 9 fills them.* + +### Dependency Graph + +``` +S8-1: Determinism fixes ──────────────────────────────┐ + ├─→ S8-5: make pre-pr +S8-2: Content cross-ref validation ───────────────────┤ + │ +S8-3: --test-mode + --port 0 ──→ S8-4: Test client ───┤ + binary MVP │ + │ │ + └─→ S8-6: Layer 3 test wiring + │ +S8-7: Pause guard tests ─────────────────────────────┘ (parallel, no deps) +S8-8: Determinism regression tests ── (after S8-1) +S8-9: EntityRegistry lifecycle tests ── (parallel, no deps) +S8-10: Fixture staleness check ── (after S8-5) +``` + +### Ordered Implementation + +| # | Item | Owner | Effort | Depends On | Deliverable | +|---|------|-------|--------|------------|-------------| +| **S8-1** | Determinism fixes (A, B, D) | Dudley | 0.5 day | Nothing | 3 fixes, ~40 lines across 4 files. Fix C already done. | +| **S8-2** | Content cross-reference validation | Justine | 1 day | Nothing | 9 checks added to `tooling/validate-content`. Python only. | +| **S8-3** | `--test-mode` + `--port 0` server flags | Dudley | 0.5 day | Nothing | Modified `main.rs`: flag parsing, `LISTENING:{port}` stdout, fixed seed 42, `accept_on(listener)`. | +| **S8-4** | Test client binary MVP | Dudley | 2.5-3.5 days | S8-3 | `tooling/test-client/` — new crate, connect, receive, text render, golden compare, replay. See Section 2 for full spec. Increased 0.5d from R2 estimate for crate setup + dependency wiring. | +| **S8-5** | `make pre-pr` chain | Justine | 0.5 day | S8-1, S8-2 | Makefile target: lint → build → test → validate → fixtures. Plus branch-specific variants. | +| **S8-6** | Layer 3 test wiring | Dudley | 0.5 day | S8-3, S8-4 | `make test-layer3` — builds both binaries, runs subprocess integration test. | +| **S8-7** | Pause guard tests (6 tests) | Dudley | 0.5 day | Nothing | All 6 Hoshe-validated gaps. P0: `movement_discarded_while_paused`, `unpause_accepted_while_paused`. | +| **S8-8** | Determinism regression tests | Dudley | 0.5 day | S8-1 | Per-fix regression tests: sorted tiles, sorted entities, deterministic mover winner. | +| **S8-9** | EntityRegistry lifecycle tests (3 tests) | Dudley | 0.25 day | Nothing | P0: `register_respawn_no_stale_mapping`. P1: `register_after_unregister_gets_new_id`, `unregister_unknown_entity_is_noop`. | +| **S8-10** | Fixture staleness check | Justine | 0.25 day | S8-5 | `make fixtures-check` target. BLOCKER policy (not WARNING). | + +### Critical Path + +``` +S8-3 (0.5d) → S8-4 (2.5-3.5d) → S8-6 (0.5d) = 3.5-4.5 days +``` + +The test client binary remains the long pole, slightly longer due to new crate setup. Everything else proceeds in parallel. + +### Sprint 8 Total Effort + +| Track | Items | Effort | +|-------|-------|--------| +| Server (Dudley) | S8-1, S8-3, S8-4, S8-6, S8-7, S8-8, S8-9 | ~5.5-7 days | +| Tooling (Justine) | S8-2, S8-5, S8-10 | ~1.75 days | +| **Total** | **10 items** | **~7.5-9 team-days** | + +No client (Stig) work in Sprint 8 for this track. Client test writing starts Sprint 9 once fixtures are flowing through `make pre-pr`. + +### What Explicitly Does NOT Ship in Sprint 8 + +- Gauntlet room content (rooms 1-14) +- Anti-tedium features (room reset, hub teleport, WRONG button) +- Crossterm live terminal display +- CI automation (Gitea Actions) +- Performance baselines +- Encoding asymmetry cross-language tests +- Client tests (Stig's 38+) +- Cross-room transition scenarios +- Protocol crate extraction + +All Sprint 9+ scope. See Section 3. + +--- + +## 2. Test Client Binary — Final Build-Ready Specification + +*This is the definitive spec. It combines Tyre R2 architecture + Ozzie R2 UX research + Dudley R2 server-side requirements + the lead's override on binary location. Implementers should treat this as the contract.* + +### 2.1 Crate Location and Structure + +**Location:** `tooling/test-client/` + +``` +tooling/test-client/ +├── Cargo.toml +├── Cargo.lock +└── src/ + ├── main.rs # CLI parsing, connection loop, output dispatch + ├── golden.rs # Golden file comparison (JSON field diff) + └── replay.rs # JSONL replay file loading +``` + +**`tooling/test-client/Cargo.toml`:** + +```toml +[package] +name = "settled-reach-test-client" +version = "0.1.0" +edition = "2021" +description = "Headless test client for Gauntlet QA verification. Connects to the game server via TCP, receives ObserverSnapshots, renders as text, compares against golden files." + +[[bin]] +name = "settled-reach-test-client" +path = "src/main.rs" + +[dependencies] +# Server crate provides bridge types + framing +settled-reach-server = { path = "../../server" } + +# CLI +clap = { version = "4", features = ["derive"] } + +# Golden file comparison (JSON) +serde_json = "1" +serde = { version = "1", features = ["derive"] } + +# Wire protocol (shared with server — version must match) +rmp-serde = "1" +``` + +**Build:** `cd tooling/test-client && cargo build` + +Or from project root via Makefile: + +```makefile +build-test-client: + cd tooling/test-client && cargo build + +build-test-client-release: + cd tooling/test-client && cargo build --release +``` + +### 2.2 Import Paths + +The test client imports shared types from the server crate's library: + +```rust +// tooling/test-client/src/main.rs + +// Wire types +use settled_reach_server::bridge::types::{ + ObserverSnapshot, PlayerInput, PlayerAction, + VisibleEntity, EntityKind, VisibilitySector, + GameTime, MovementStance, NearbyInteraction, + MonologueEvent, PendingRecognitionWire, +}; +use settled_reach_server::knowledge::types::{ + RelationshipState, EntityVisibility, +}; +use settled_reach_server::simulation::time::{DayPhase, TickRate}; + +// Framing protocol +use settled_reach_server::bridge::framing::{read_framed, write_framed}; + +// Text renderer (library function in server crate) +use settled_reach_server::bridge::text_renderer::format_snapshot_text; +``` + +**Dudley action required:** Ensure `server/src/bridge/mod.rs` adds `pub mod text_renderer;` when the text renderer is implemented. All other exports already exist. + +### 2.3 CLI Interface (Sprint 8 MVP) + +``` +settled-reach-test-client [OPTIONS] + +CONNECTION: + --connect Server address (default: 127.0.0.1:9876) + +INPUT: + --replay Send inputs from file (one JSON PlayerInput per line) + Empty lines = idle tick (no input sent) + +OUTPUT (mutually exclusive): + --text Render each snapshot as structured text to stdout (default) + --json Dump each snapshot as JSON to stdout (for golden file generation) + --quiet No output, assertions only (for CI) + +ASSERTIONS: + --golden Compare FINAL snapshot against golden file, exit 1 on diff + --ticks Disconnect after N ticks (default: unlimited) + +EXIT CODES: + 0 Success (all assertions passed, or no assertions) + 1 Golden file mismatch (diff printed to stderr) + 2 Connection error or protocol error +``` + +**Sprint 9+ CLI additions (NOT Sprint 8):** +``` + --interactive Read inputs from stdin (human-in-the-loop) + --live Crossterm live-updating terminal display (Ozzie spec) + --log Append text output to session log + --history-buffer Ring buffer depth for WRONG captures (default: 60) + --checklist Load checklist for auto-tracking +``` + +### 2.4 Connection Protocol + +``` +┌─────────────────────────┐ ┌──────────────────────────────┐ +│ settled-reach-server │ TCP │ settled-reach-test-client │ +│ │◄────────────►│ │ +│ --test-mode --port 0 │ framed │ --connect 127.0.0.1:{port} │ +│ --seed 42 │ msgpack │ --replay inputs.jsonl │ +│ │ │ --ticks 50 --golden f.json │ +└─────────────────────────┘ └──────────────────────────────┘ + server/target/debug/ tooling/test-client/target/debug/ + settled-reach-server settled-reach-test-client +``` + +**Startup sequence:** + +1. Server binds TCP socket, prints `LISTENING:{port}\n` to stdout, flushes, blocks on accept +2. Test client reads `LISTENING:{port}` from server's stdout (when spawned by Layer 3 test) or connects to known port via `--connect` +3. Test client calls `TcpStream::connect(addr)` +4. Server's `accept_on(listener)` returns — game loop starts + +**Per-tick protocol:** + +1. Server runs one tick of simulation +2. Server sends `ObserverSnapshot` via `write_framed` (length-prefixed MessagePack) +3. Test client receives snapshot via `read_framed` + `rmp_serde::from_slice::` +4. Test client formats output (text/json/quiet) +5. If `--replay`: test client sends next line's `Vec` via `write_framed` + `rmp_serde::to_vec` +6. If no more replay lines: send empty `Vec` (idle tick) + +**Important:** Use `rmp_serde::to_vec` (array format), NOT `to_vec_named` (map format). This matches GDScript's encoding. Dudley confirmed in R2 cross-review. + +**Shutdown:** + +After `--ticks N` snapshots received: +1. If `--golden`: compare final snapshot, print diff to stderr if mismatch, exit 1 +2. Drop TCP connection (triggers server shutdown in `--test-mode`) +3. Exit 0 + +### 2.5 Text Output Format (Sprint 8 MVP) + +``` +=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === +Game time: Day 0, 04:12 (Morning) +Room: Occlusion Corridor +Entities (5): + npc:100 (18,10) Forward rel:Neutral vis:Visible d=3 + npc:101 (20,10) Forward rel:Unknown vis:Remembered d=5 + obj:200 (16,9) Forward rel:n/a vis:Visible d=1 + npc:102 (12,8) Periph rel:Hostile vis:Visible d=5 + npc:103 (22,14) Periph rel:Friendly vis:Visible d=9 +Pending recognitions: 1 [npc:104 at (19,12) 3/8 ticks] +Tiles: 31 visible +Interactions (2): + npc:100 [Talk(1), ExamineNpc(2)] distance=3 + obj:200 [Read(1), Observe(2)] distance=1 +Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] +Monologue: "Something about this manifest doesn't add up." +=== +``` + +**Format rules:** + +| Rule | Detail | +|------|--------| +| Entity labels | `kind:entity_id` (e.g., `npc:100`, `obj:200`). No display names on wire. | +| Entity sort | By distance from player (nearest first). Ties broken by entity_id. | +| Tick separators | `===` lines for clean `diff` between ticks. | +| Positions | Integer tile coords. f32 render offset is irrelevant for testing. | +| Room name | From `room_at(player_pos)` using Gauntlet coordinate constants. Shows `Room: (unknown)` if outside all room bounds. | +| Sector labels | `Forward`, `Periph` — mapped from `VisibilitySector` enum. (Behind entities are absent from snapshot.) | +| Relationship labels | `Unknown`, `Known`, `Friendly`, `PersonOfInterest`, `Hostile`. Mapped from `RelationshipState`. | +| Visibility labels | `Visible`, `Remembered`, `Fogged`. Mapped from `EntityVisibility`. | +| Distance | Manhattan distance in tiles from player to entity. | +| Missing data | Sections with no data are omitted (no `Interactions (0):` noise). | +| Pending recognitions | From `ObserverSnapshot.pending_recognitions` — shows entity, position, remaining/total ticks. | + +### 2.6 Text Renderer — Library Function in Server Crate + +The text renderer lives in the **server crate's library** (not in the test client). This is deliberate: + +- Server integration tests can call it for debug output +- The test client imports it via the path dependency +- The server binary never references it — zero bloat + +**File:** `server/src/bridge/text_renderer.rs` + +```rust +// server/src/bridge/text_renderer.rs +// Library code — callable by test client crate AND server integration tests. +// The server binary never references this module. + +use std::fmt::Write; +use crate::bridge::types::*; +use crate::knowledge::types::{EntityVisibility, RelationshipState}; + +/// Format an ObserverSnapshot as structured text for human verification. +pub fn format_snapshot_text(snapshot: &ObserverSnapshot) -> String { + let mut out = String::with_capacity(2048); + + // Find player entity for position reference + let player = snapshot.entities.iter() + .find(|e| matches!(e.kind, EntityKind::Player)); + let (px, py) = player + .map(|p| (p.x as i32, p.y as i32)) + .unwrap_or((-1, -1)); + + // Header + writeln!(out, "=== Tick {} | Player ({},{}) facing {:?} | Stance: {:?} | TickRate: {:?} ===", + snapshot.tick, px, py, + snapshot.player_facing, + snapshot.player_stance, + snapshot.game_time.tick_rate, + ).ok(); + + // Game time + let minutes = snapshot.game_time.time_of_day % 60; + let hours = (snapshot.game_time.time_of_day / 60) % 24; + writeln!(out, "Game time: Day {}, {:02}:{:02} ({:?})", + snapshot.game_time.day, hours, minutes, + snapshot.game_time.day_phase, + ).ok(); + + // Room name (from Gauntlet constants — returns "(unknown)" if not in any room) + #[cfg(feature = "test-world")] + { + let room_name = crate::test_world::constants::room_at_position(px, py) + .map(|r| r.name) + .unwrap_or("(unknown)"); + writeln!(out, "Room: {}", room_name).ok(); + } + + // Non-player entities sorted by distance then entity_id + let mut entities: Vec<&VisibleEntity> = snapshot.entities.iter() + .filter(|e| !matches!(e.kind, EntityKind::Player)) + .collect(); + entities.sort_by_key(|e| { + let dist = (e.x as i32 - px).unsigned_abs() + (e.y as i32 - py).unsigned_abs(); + (dist, e.entity_id) + }); + + if !entities.is_empty() { + writeln!(out, "Entities ({}):", entities.len()).ok(); + for e in &entities { + let dist = (e.x as i32 - px).unsigned_abs() + (e.y as i32 - py).unsigned_abs(); + writeln!(out, " {}:{:<8} ({},{}) {:<8} rel:{:<16} vis:{:<12} d={}", + kind_label(e.kind), e.entity_id, + e.x as i32, e.y as i32, + sector_label(e.visibility), + relationship_label(e.relationship), + observation_label(e.observation), + dist, + ).ok(); + } + } + + // Pending recognitions + if !snapshot.pending_recognitions.is_empty() { + write!(out, "Pending recognitions: {}", snapshot.pending_recognitions.len()).ok(); + for pr in &snapshot.pending_recognitions { + write!(out, " [npc:{} at ({},{}) {}/{} ticks]", + pr.entity_id, pr.x as i32, pr.y as i32, + pr.total_delay_ticks - pr.remaining_ticks, pr.total_delay_ticks, + ).ok(); + } + writeln!(out).ok(); + } + + // Tiles + writeln!(out, "Tiles: {} visible", snapshot.visible_tiles.len()).ok(); + + // Interactions + if !snapshot.nearby_interactions.is_empty() { + writeln!(out, "Interactions ({}):", snapshot.nearby_interactions.len()).ok(); + for ni in &snapshot.nearby_interactions { + let verbs: Vec = ni.verbs.iter() + .map(|v| format!("{}({})", v.label, v.priority)) + .collect(); + writeln!(out, " {}:{} [{}] distance={}", + kind_label(ni.entity_type), ni.entity_id, + verbs.join(", "), ni.distance, + ).ok(); + } + } + + // Inventory + if !snapshot.player_inventory.is_empty() { + let slots: Vec = snapshot.player_inventory.iter() + .map(|item| format!("{}(slot-{})", item.name, item.slot)) + .collect(); + writeln!(out, "Inventory: {}/9 [{}]", + snapshot.player_inventory.len(), slots.join(", "), + ).ok(); + } + + // Monologue + if let Some(ref mono) = snapshot.current_monologue { + writeln!(out, "Monologue: \"{}\"", mono.text).ok(); + } + + writeln!(out, "===").ok(); + out +} + +fn kind_label(kind: EntityKind) -> &'static str { + match kind { + EntityKind::Player => "player", + EntityKind::Npc => "npc", + EntityKind::Object => "obj", + EntityKind::Terrain => "terrain", + } +} + +fn sector_label(sector: VisibilitySector) -> &'static str { + match sector { + VisibilitySector::Forward => "Forward", + VisibilitySector::Peripheral => "Periph", + } +} + +fn relationship_label(rel: RelationshipState) -> &'static str { + match rel { + RelationshipState::Unknown => "Unknown", + RelationshipState::Known => "Known", + RelationshipState::Friendly => "Friendly", + RelationshipState::PersonOfInterest => "POI", + RelationshipState::Hostile => "Hostile", + } +} + +fn observation_label(obs: EntityVisibility) -> &'static str { + match obs { + EntityVisibility::Visible => "Visible", + EntityVisibility::Remembered => "Remembered", + EntityVisibility::Fogged => "Fogged", + } +} +``` + +**Dudley action items for server crate:** +1. Add `pub mod text_renderer;` to `server/src/bridge/mod.rs` +2. Add `pub mod test_world;` to `server/src/lib.rs` (for Gauntlet constants — can be feature-gated behind `#[cfg(feature = "test-world")]` if desired) + +### 2.7 Golden File Comparison + +**Format:** JSON with sorted keys, pretty-printed. + +**Generation:** +```bash +# Generate golden file for tick 5 of the proof room (no input) +cd tooling/test-client +cargo run -- --connect 127.0.0.1:9876 --ticks 5 --json | jq -S . > ../../tests/fixtures/golden/proof_room_tick5.json +``` + +**Comparison algorithm:** + +```rust +// tooling/test-client/src/golden.rs + +use settled_reach_server::bridge::types::ObserverSnapshot; +use serde_json::Value; +use std::path::Path; + +pub struct FieldDiff { + pub path: String, + pub expected: String, + pub actual: String, +} + +pub fn compare_golden(actual: &ObserverSnapshot, golden_path: &Path) -> Result, String> { + let golden_str = std::fs::read_to_string(golden_path) + .map_err(|e| format!("failed to read golden file: {}", e))?; + let golden: ObserverSnapshot = serde_json::from_str(&golden_str) + .map_err(|e| format!("failed to parse golden file: {}", e))?; + + let actual_json = serde_json::to_value(actual) + .map_err(|e| format!("failed to serialize actual: {}", e))?; + let golden_json = serde_json::to_value(&golden) + .map_err(|e| format!("failed to serialize golden: {}", e))?; + + Ok(diff_values(&actual_json, &golden_json, String::new())) +} + +fn diff_values(actual: &Value, expected: &Value, path: String) -> Vec { + if actual == expected { + return vec![]; + } + match (actual, expected) { + (Value::Object(a), Value::Object(e)) => { + let mut diffs = Vec::new(); + for (key, eval) in e { + let child_path = format!("{}.{}", path, key); + match a.get(key) { + Some(aval) => diffs.extend(diff_values(aval, eval, child_path)), + None => diffs.push(FieldDiff { + path: child_path, + expected: format!("{}", eval), + actual: "MISSING".into(), + }), + } + } + for key in a.keys() { + if !e.contains_key(key) { + diffs.push(FieldDiff { + path: format!("{}.{}", path, key), + expected: "ABSENT".into(), + actual: format!("{}", a[key]), + }); + } + } + diffs + } + (Value::Array(a), Value::Array(e)) => { + let mut diffs = Vec::new(); + let len = a.len().max(e.len()); + for i in 0..len { + let child_path = format!("{}[{}]", path, i); + match (a.get(i), e.get(i)) { + (Some(av), Some(ev)) => diffs.extend(diff_values(av, ev, child_path)), + (Some(av), None) => diffs.push(FieldDiff { + path: child_path, expected: "ABSENT".into(), actual: format!("{}", av), + }), + (None, Some(ev)) => diffs.push(FieldDiff { + path: child_path, expected: format!("{}", ev), actual: "MISSING".into(), + }), + (None, None) => {} + } + } + diffs + } + _ => vec![FieldDiff { + path, + expected: format!("{}", expected), + actual: format!("{}", actual), + }], + } +} +``` + +**Diff output format (to stderr on mismatch):** + +``` +GOLDEN FILE MISMATCH: proof_room_tick5.json + .entities[2].x: expected 18, got 19 (CHANGED) + .entities[2].relationship: expected "Neutral", got "Hostile" (CHANGED) + .visible_tiles: 31 expected, 29 actual + [29]: expected {"x":22,"y":10,...}, got MISSING + [30]: expected {"x":22,"y":11,...}, got MISSING + .current_monologue: expected {"text":"Quiet shift.",...}, got MISSING +``` + +**Why JSON not MessagePack:** Human-readable in `git diff`. Sorted keys = deterministic output. JSON round-trips through `serde_json` without precision loss for ObserverSnapshot types (u64, f32, strings, enums). + +### 2.8 Replay File Format + +**Format:** JSONL (one JSON line per tick) + +Each line is a JSON array of `PlayerInput` objects: + +```jsonl +[{"tick":0,"action":"MoveNorth"}] +[{"tick":1,"action":"MoveNorth"}] +[] +[{"tick":3,"action":{"Interact":{"target_entity_id":100,"verb":"Talk"}}}] +[{"tick":4,"action":"ToggleStanceUp"}] +``` + +Empty array `[]` = idle tick (no player input). + +**Generation:** Manual creation for targeted tests. Future: record from Godot client or interactive test client. + +### 2.9 Layer 3 Integration Test + +With the test client in a separate crate, Layer 3 testing requires building two independent binaries. A Makefile target handles orchestration: + +```makefile +# Makefile (project root) + +# Build both binaries needed for Layer 3 testing +build-layer3: build-server build-test-client + +# Run the Layer 3 subprocess integration test +test-layer3: build-layer3 + cd tooling/test-client && cargo test --test layer3 -- --ignored +``` + +The Layer 3 test lives in the test client crate (not the server crate) because the test client crate already depends on the server crate, giving access to shared types: + +```rust +// tooling/test-client/tests/layer3.rs + +use std::process::{Command, Stdio}; +use std::io::{BufRead, BufReader}; +use std::path::PathBuf; + +/// Locate the server binary. Built via `make build-server`. +fn server_binary() -> PathBuf { + // Navigate from tooling/test-client/ up to project root, then to server binary + let project_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("..").join(".."); + let binary = project_root.join("server/target/debug/settled-reach-server"); + assert!(binary.exists(), + "Server binary not found at {:?}. Run `make build-server` first.", binary); + binary +} + +/// Locate the test client binary. Built via `make build-test-client`. +fn test_client_binary() -> PathBuf { + let project_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("..").join(".."); + let binary = project_root + .join("tooling/test-client/target/debug/settled-reach-test-client"); + assert!(binary.exists(), + "Test client binary not found. Run `make build-test-client` first."); + binary +} + +#[test] +#[ignore] // Slow — run via `make test-layer3` +fn server_and_test_client_subprocess_roundtrip() { + // 1. Launch server with --test-mode --port 0 + let mut server = Command::new(server_binary()) + .args(["--test-mode", "--port", "0"]) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .expect("failed to launch server"); + + // 2. Read port from server stdout + let stdout = server.stdout.take().unwrap(); + let mut reader = BufReader::new(stdout); + let mut line = String::new(); + reader.read_line(&mut line).expect("failed to read LISTENING line"); + let port: u16 = line.trim() + .strip_prefix("LISTENING:") + .expect("expected LISTENING:{port}") + .parse() + .expect("invalid port"); + + // 3. Launch test client: connect, receive 5 ticks, compare golden + let project_root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("..").join(".."); + let golden = project_root.join("tests/fixtures/golden/proof_room_tick5.json"); + + let client_status = Command::new(test_client_binary()) + .args([ + "--connect", &format!("127.0.0.1:{}", port), + "--ticks", "5", + "--golden", golden.to_str().unwrap(), + "--quiet", + ]) + .status() + .expect("failed to launch test client"); + + // 4. Assert test client passed + assert!(client_status.success(), "test client golden file mismatch (exit code: {})", + client_status.code().unwrap_or(-1)); + + // 5. Server should exit after client disconnect (--test-mode behavior) + let server_exit = server.wait().expect("failed to wait for server"); + assert!(server_exit.success(), "server exited with error"); +} +``` + +### 2.10 File Inventory + +| File | Sprint | Description | +|------|--------|-------------| +| **`tooling/test-client/Cargo.toml`** | 8 | New crate: depends on server lib, clap, serde_json | +| **`tooling/test-client/src/main.rs`** | 8 | CLI parsing, TCP connection, tick loop, output dispatch | +| **`tooling/test-client/src/golden.rs`** | 8 | Golden file JSON comparison with field-by-field diff | +| **`tooling/test-client/src/replay.rs`** | 8 | JSONL replay file loading | +| **`tooling/test-client/tests/layer3.rs`** | 8 | Subprocess integration test (S8-6) | +| `server/src/bridge/text_renderer.rs` | 8 | `format_snapshot_text()` library function (callable by test client + server tests) | +| `server/src/bridge/mod.rs` | 8 | Add `pub mod text_renderer;` | +| `server/src/test_world/constants.rs` | 8 | `GauntletRoom` struct, `ROOMS` array, `room_at_position()` function | +| `tests/fixtures/golden/proof_room_tick5.json` | 8 | First golden file (proof room, no input, 5 ticks) | + +--- + +## 2A. `tooling/` Directory Structure + +The lead asked for a proposed structure. Here's the current state and the proposed additions. + +### Current State + +``` +tooling/ +├── .gitkeep +├── check-fact-ids # Shell script — validates fact IDs in content +├── content-converter/ # Standalone Rust crate — YAML→RON conversion +│ ├── Cargo.toml +│ ├── Cargo.lock +│ └── src/ +│ ├── main.rs +│ └── types.rs +├── db-backup # Shell script — database backup +├── db-install # Shell script — database setup +├── install-godot # Shell script — Godot installation +├── install-rust # Shell script — Rust toolchain setup +├── line-previewer/ # Standalone Rust crate — dialogue/monologue preview +│ ├── Cargo.toml +│ ├── Cargo.lock +│ └── src/ +│ ├── main.rs +│ ├── pipeline.rs +│ └── types.rs +├── synth_ui_sounds.py # Python script — audio synthesis +└── validate-content # Python script — content schema validation +``` + +### Convention (Already Established) + +- **Directories** = Rust crates (each with own `Cargo.toml` and `Cargo.lock`) +- **Files** = Standalone scripts (shell, Python — no build step) + +This convention is clean and consistent. No restructure needed. + +### Proposed Addition + +``` +tooling/ +├── ... (all existing files unchanged) +├── test-client/ # NEW — Rust crate, depends on server lib +│ ├── Cargo.toml +│ ├── Cargo.lock +│ └── src/ +│ ├── main.rs +│ ├── golden.rs +│ └── replay.rs +└── ... +``` + +The test client follows the same standalone-crate pattern as `content-converter/` and `line-previewer/`. It differs in one way: it has a **path dependency on the server crate** (`../../server`), while the other two crates are fully independent. This is the architectural trade-off of sharing bridge types instead of duplicating them. + +### Future Considerations + +| Milestone | Potential Change | +|-----------|-----------------| +| Sprint 9+ (if compile times hurt) | Extract `settled-reach-protocol` crate to `tooling/protocol/`. Both server and test-client depend on it. Eliminates bevy transitive dep for test client. | +| When Cargo workspace makes sense | Root `Cargo.toml` with `workspace.members = ["server", "tooling/test-client", "tooling/protocol"]`. Shared `target/` directory, shared `Cargo.lock`, faster builds. | +| When more Rust tools arrive | Consider if the standalone pattern still scales. Workspace becomes more attractive at 4+ crates. | + +No action needed now. The standalone pattern works for 3 Rust crates. + +--- + +## 3. Sprint 9+ Roadmap + +*Everything that's NOT Sprint 8, ranked by implementation value. Tier 1 = next sprint. Tier 2 = sprint after. Tier 3 = when needed.* + +### Tier 1: Sprint 9 (High Value — Unblocked by Sprint 8) + +| # | Item | Owner | Effort | Depends On | Value | +|---|------|-------|--------|------------|-------| +| R-01 | **Gauntlet rooms 1-4** (Inventory, Occlusion, Interaction, Crowd) | Content + Dudley | 3-4 days | S8-3 (--test-mode) | Unlocks room-specific testing. These 4 cover the broadest system range. | +| R-02 | **Room reset trigger** | Dudley | 1.5 days | R-01 | Dudley's R2 design is implementation-ready. `RoomResetTrigger` component, `RoomSnapshots` resource, `execute_room_reset` system. | +| R-03 | **Hub teleport** | Dudley | 0.5 day | R-01 | `PlayerAction::TeleportToHub`, instant camera snap. Simplest anti-tedium feature. | +| R-04 | **Client tests P0-P1** (10 tests) | Stig | 2-3 days | S8-10 (fixtures) | Bug #5 monologue regression, Bug #2 camera, fog shader (3), entity lifecycle, pending recognition blob. Highest-value client coverage. | +| R-05 | **Determinism golden files** (per-room) | Dudley | 1 day | R-01, S8-4 | One golden file per room: 10-tick idle snapshot. The "did anything break?" safety net. | +| R-06 | **Fog byte constants** | Stig | 0.25 day | Nothing | `VIS_HIDDEN=0`, `VIS_PERIPHERAL=180`, `VIS_FORWARD=255`, `EXP_UNEXPLORED=0`, `EXP_EXPLORED=128`, `EXP_VISIBLE=255`. Replace magic numbers in fog shader. | +| R-07 | **Encoding asymmetry tests** (4-direction) | Hoshe + Stig | 1.5 days | S8-10 | Hoshe's R2 spec: Rust→GDScript fixtures, GDScript→Rust fixtures, raw byte round-trips both directions. | + +**Sprint 9 total: ~10-12 team-days** + +### Tier 2: Sprint 10 (Medium Value) + +| # | Item | Owner | Effort | Value | +|---|------|-------|--------|-------| +| R-08 | **WRONG button MVP** | Stig + Dudley | 2 days | Bug capture during manual testing. F12 hotkey, snapshot + text dump, human description prompt. | +| R-09 | **Gauntlet rooms 5-8** (Fog Theater, Dialogue, Pause, Zone Gate) | Content + Dudley | 3-4 days | Perception + dialogue system testing. Zone Gate is reserved/stub. | +| R-10 | **Room timer + personal bests** | Ozzie spec, Stig impl | 1 day | Anti-tedium. Session stats to `tests/gauntlet-stats.json`. | +| R-11 | **Checklist auto-tracking** | Stig + Dudley | 2 days | Merged Ozzie+Stig YAML format. `make checklist` generates markdown. Test client loads conditions. | +| R-12 | **Performance baselines** | Justine | 1.5 days | `tests/perf/baseline.json`, `tooling/perf-measure`, 15%/30% thresholds. Machine-tagged. | +| R-13 | **Client tests P2** (16 tests) | Stig | 3 days | Remaining camera (5), entity alpha+color (4), UI elements (7). | +| R-14 | **Enhanced test client terminal** | Dudley | 2 days | Ozzie's crossterm live-updating layout. Sound, cognition, checklist sections. | + +**Sprint 10 total: ~15-17 team-days (may split across 2 sprints)** + +### Tier 3: Sprint 11+ (Build When Needed) + +| # | Item | Effort | Trigger | +|---|------|--------|---------| +| R-15 | Gauntlet rooms 9-14 (Eavesdrop, Confrontation, Sprint, Sound Lab, Decay, Shift Change) | 4-6 days | When audio + cognitive delay systems are implemented | +| R-16 | Cross-room transition scenarios (T1-T8) | 2 days | When cross-cuts are built (see Q3 answer below) | +| R-17 | CI automation (Gitea Actions) | 1 day | When lead greenlights. Workflow runs `make ci`. PR tier first. | +| R-18 | Content scaling stress tests | 1 day | When content volume exceeds proof-of-concept size | +| R-19 | `blocked_entities` debug field on ObserverSnapshot | 1 day | When LOS debugging becomes a bottleneck for testers | +| R-20 | Client test headless stability (OQ-11) | 0.5 day | Before making client tests a merge gate in CI | +| R-21 | Client tests P3 (12 tests) | 2 days | Z-layer ordering (4), entity lerp (3), remaining additions (5) | +| R-22 | Hoshe's 3 additional pause guard tests | 0.5 day | P2: `set_tick_rate_while_paused`, `perception_mode_while_paused`, `interact_take_while_paused` | +| R-23 | WRONG button full capture (ring buffer, replay seed, world digest) | 1.5 days | When human testers file enough bugs to justify the investment | +| R-24 | Content cross-reference bidirectional relationship warnings | 0.5 day | When relationship asymmetries cause real content bugs | +| **R-25** | **Extract `settled-reach-protocol` crate** | 1-2 days | When test client compile times become a pain point, or when a 4th Rust crate needs bridge types | + +--- + +## 4. Remaining Question Answers + +### Q5: Test Client Binary Location — OVERRULED, NEW ANSWER + +**Answer: `tooling/test-client/` as a standalone Rust crate.** + +The lead overruled my Round 2 recommendation of `server/src/bin/test_client.rs`. See Section 0 for the full architectural analysis of why the override works and the trade-offs involved. + +The test client crate: +- Lives at `tooling/test-client/` alongside `content-converter/` and `line-previewer/` +- Has a path dependency on the server crate: `settled-reach-server = { path = "../../server" }` +- Imports bridge types, framing functions, and text renderer from the server's library +- Has its own `Cargo.toml`, `Cargo.lock`, `target/` directory + +### Q3: Are 4 Cross-Cuts Too Many for Sprint 8 Gauntlet MVP? + +**Answer: Sprint 8 has zero cross-cuts. Zero Gauntlet rooms. The question is moot for Sprint 8 — but the map DESIGN should include them.** + +*Let me be honest about what this means technically.* + +**Sprint 8 scope:** Sprint 8 ships infrastructure only (see Section 1). The `--test-mode` flag falls back to the existing proof room. There are no Gauntlet rooms in Sprint 8. Therefore, there are no cross-cuts in Sprint 8. + +**Sprint 9 scope (first rooms):** Rooms 1-4 ship as hub-and-spoke. No cross-cuts needed. Each room tests its own systems independently. The hub is the only navigation between rooms. + +**Sprint 10+ scope (cross-cuts):** Cross-cuts become valuable when we test transitions (Gestalt's T1-T8 scenarios). At that point: + +| Gestalt's Proposed Cross-Cut | Value | Sprint | +|------------------------------|-------|--------| +| Plaza → Occlusion Corridor (T1: Sprint Exit) | HIGH — tests sprint buffer clear + LOS recalc | 10 | +| Fog Theater → Dialogue Room (T2: Fog into Dialogue) | HIGH — tests state preservation across modes | 10 | +| Confrontation → Eavesdrop (T5) | MEDIUM — audio system transition | 11+ | +| Sound Lab → Fog Theater (T7) | LOW — can be tested via hub + 2 teleports | 11+ | + +**Recommendation:** Design all 4 cross-cuts in the Gauntlet map layout NOW (Gestalt's job). Build the first 2 (Plaza→Occlusion, Fog→Dialogue) in Sprint 10 when rooms 5-8 ship. Build the remaining 2 when rooms 9-14 ship. Cross-cuts are additive — adding a doorway between two rooms doesn't change either room's layout. + +**Gestalt's sub-question about consolidating Sound Lab into Occlusion Corridor:** Don't. Sound Lab tests D-018's three-range sound model, which is orthogonal to LOS/shadowcasting. Consolidating them makes the room too complex to isolate failures. Keep them separate. + +### R2-OQ-03 (from Hoshe): Should `make pre-pr` include `make content-ron`? + +**Answer: Yes, add it to `make pre-pr` but NOT to `make pre-pr-server`.** + +```makefile +pre-pr: lint build test validate-content content-ron fixtures-check +pre-pr-server: lint-server build-server test-server fixtures-check +pre-pr-client: lint-client build-client test-client +pre-pr-content: validate-content content-ron +``` + +**Budget impact:** `content-ron` takes <5s for current content volume. Negligible. + +### R2-OQ-09 (from Gestalt): Room ordering affects entity StableId assignment? + +**Answer: Yes, and it's handled by spawn order in the setup function.** + +Gauntlet entity spawn order must be deterministic. Since `--test-mode` uses seed 42, the SimRng is fixed. Entity spawn order is determined by the `setup_gauntlet_world()` function's code order, not YAML file ordering. As long as the setup function spawns entities in a fixed order (room 1 entities, then room 2, etc.), StableIds are deterministic. + +**The rule:** Gauntlet setup spawns entities in room-number order, entities within a room in a canonical order defined in the constants module. YAML room files are loaded in alphabetical order. If YAML files define entity lists, those lists must be ordered (not sets/maps with non-deterministic iteration). + +### R2-OQ-10 (from Gestalt): Per-room reset sufficient, or need full server restart? + +**Answer: Per-room reset is sufficient for Sprint 9-10. Full restart is `kill + relaunch`, which already works.** + +Room reset restores entity state within a room. Full reset = restart the server (the tester kills the test client, server exits on disconnect, tester relaunches both). With `--test-mode --seed 42`, a fresh server launch is identical to the previous one. + +Don't build a "full server reset" command. The operating system already provides one: process restart. + +--- + +## 5. Risk Register + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| Test client crate setup + bevy compile takes longer than expected | Medium | +0.5-1 day on S8-4 | Estimate already padded (2.5-3.5d vs original 2-3d). First bevy compile is one-time cost. | +| Cross-binary Layer 3 test is flaky (race conditions, path issues) | Medium | Delays S8-6 | `LISTENING:{port}` protocol + explicit binary path discovery via `CARGO_MANIFEST_DIR`. | +| Content cross-ref validation finds many existing errors | Medium | Sprint 8 effort bloat fixing content bugs | Run validation as WARNING first sprint, promote to ERROR in Sprint 9. | +| Client test headless stability (OQ-11) blocks CI | Low | Can't automate client tests in CI | Deferred to Tier 3. Manual `make pre-pr-client` is the interim gate. | +| Fixture staleness BLOCKER policy frustrates developers | Low | Devs skip `make pre-pr` | Education. The alternative (false-positive client tests) is worse. | +| Standalone crate pattern doesn't scale past 4 Rust crates | Low | Build fragmentation, dep version drift | Evaluate Cargo workspace at R-25 (protocol crate extraction). | + +--- + +## Summary + +| Deliverable | Status | +|-------------|--------| +| Lead override analysis (Section 0) — why `tooling/test-client/` works, trade-offs, dependency cost | Complete | +| Sprint 8 implementation plan (10 items, ~7.5-9 team-days, dependency graph) | Complete | +| Test client binary final spec (new crate structure, imports, CLI, protocol, text format, golden diff, Layer 3, file inventory) | Complete | +| `tooling/` directory structure proposal (Section 2A) | Complete | +| Sprint 9+ roadmap (25 items across 3 tiers, including R-25 protocol extraction) | Complete | +| Q5 (test client location): `tooling/test-client/` — override applied | Updated | +| Q3 (cross-cuts): 0 in Sprint 8, design all 4 now, build first 2 in Sprint 10 | Answered | +| R2-OQ-03 (content-ron in pre-pr): Yes, in full pre-pr only | Answered | +| R2-OQ-09 (room ordering): Fixed spawn order in setup function | Answered | +| R2-OQ-10 (full restart): Kill + relaunch, don't build a command | Answered | diff --git a/docs/workshops/test-architecture/workshop-outcomes.md b/docs/workshops/test-architecture/workshop-outcomes.md new file mode 100644 index 000000000..84aee086d --- /dev/null +++ b/docs/workshops/test-architecture/workshop-outcomes.md @@ -0,0 +1,655 @@ +# QA Strategy & Test Architecture Workshop — Outcomes + +**Workshop:** QA Strategy & Test Architecture +**Rounds:** 3 (Analysis -> Synthesis -> Prioritization) +**Date:** 2026-02-17 +**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie +**Documenter:** Qatux + +This is the definitive output of the workshop. All specifications are build-ready. Round notes are at `docs/workshops/test-architecture/round-{1,2,3}-notes.md`. + +--- + +## Table of Contents + +1. [Decision Summary](#1-decision-summary) +2. [Sprint 8 Implementation Plan](#2-sprint-8-implementation-plan) +3. [Sprint 9+ Roadmap](#3-sprint-9-roadmap) +4. [Gauntlet Map Specification](#4-gauntlet-map-specification) +5. [Test Client Binary Specification](#5-test-client-binary-specification) +6. [Prioritized Test Backlog](#6-prioritized-test-backlog) +7. [Content Validation Specification](#7-content-validation-specification) +8. [Anti-Tedium Specifications](#8-anti-tedium-specifications) +9. [Human Tester Workflow](#9-human-tester-workflow) +10. [CI Pipeline Design](#10-ci-pipeline-design) + +--- + +## 1. Decision Summary + +### Workshop Decisions (confirmed by lead) + +| ID | Decision | Round | Dissent | +|----|----------|-------|---------| +| WS-D1 | **Commit to determinism now.** 3 targeted fixes (A, B, D), ~22 lines. Fix C already done. | R1, confirmed R2 | None | +| WS-D2 | **Gauntlet as hybrid YAML + Rust inject.** YAML for geometry + entity placement, Rust for KG state injection. | R1, confirmed R2 | None | +| WS-D3 | **Rust test macros with named helpers. No custom DSL.** DSL revisitable Sprint 10+. | R1, confirmed R2 | None | +| WS-D4 | **Golden files as ObserverSnapshot at fixed positions.** JSON format, sorted keys, field-by-field diff. | R1, confirmed R2 | None | +| WS-D5 | **Separate test client binary** (lead override of R1 consensus on server-side text renderer). Binary at `tooling/test-client/` (lead override of R2 location at `server/src/bin/`). | R1 overruled R2, location overruled R3 | Tyre accepted both overrules constructively | + +### Resolved Questions + +| ID | Question | Answer | +|----|----------|--------| +| OQ-01 | Client depends on visible_tiles ordering? | No. Fix safe to ship. | +| OQ-02 | rmp_serde accepts int_16 for u64? | Yes. Traced through rmp-serde 1.3.1. | +| OQ-04 | WalkabilityMap HashMap -> BTreeMap? | No. Point-lookup only. | +| OQ-05 | Fixture staleness: git diff robust enough? | Yes. Deterministic generation verified. | +| OQ-07 | Server --test-mode and --port 0? | Designed and specified. | +| OQ-08 | Room name from coordinates? | `room_at()` in constants module. | +| OQ-10 | Fog byte value constants? | Yes. 6 named constants. | +| R2-OQ-01 | SetTickRate while paused? | Bug. Reject. 4-line fix. | +| R2-OQ-02 | Entity index recycling? | Safe (bevy generation counter). | +| R2-OQ-03 | content-ron in pre-pr? | Yes, in full pre-pr only. | +| R2-OQ-05 | blocked_entities feasibility? | Feasible. Sprint 9. ~300 tile lookups/tick. | +| R2-OQ-06 | Checklist overlay in Godot? | Lightweight overlay. Full tracking test-client-only. | +| R2-OQ-08 | Cross-room checklist location? | `content/gauntlet/cross_room_checks.yaml` | +| R2-OQ-09 | Room ordering for StableId? | Canonical order required. Append-only. | +| R2-OQ-10 | Room reset vs full restart? | Both. Different test types need different levels. | + +### Fixture Staleness: BLOCKER + +Tyre's argument (R2): stale fixtures mean client tests run against outdated protocol data — every passing client test is a false positive. Accepted by all agents. `make pre-pr` exits 1 on stale fixtures. + +--- + +## 2. Sprint 8 Implementation Plan + +*Sprint 8 ships infrastructure — the plumbing. Sprint 9 fills the pipes.* (Tyre) + +### Items + +| # | Item | Owner | Effort | Depends On | +|---|------|-------|--------|------------| +| S8-1 | Determinism fixes (A, B, D) | Dudley | 0.5d | -- | +| S8-2 | Content cross-reference validation (9 checks) | Justine | 1d | -- | +| S8-3 | `--test-mode` + `--port 0` server flags | Dudley | 0.5d | -- | +| S8-4 | Test client binary MVP | Dudley | 2-3d | S8-3 | +| S8-5 | `make pre-pr` chain | Justine | 0.5d | S8-1, S8-2 | +| S8-6 | Layer 3 test wiring | Dudley | 0.5d | S8-3, S8-4 | +| S8-7 | Pause guard tests (6 tests) | Dudley | 0.5d | -- | +| S8-8 | Determinism regression tests | Dudley | 0.5d | S8-1 | +| S8-9 | EntityRegistry lifecycle tests (3 tests) | Dudley | 0.25d | -- | +| S8-10 | Fixture staleness check | Justine | 0.25d | S8-5 | + +### Critical Path + +``` +S8-3 (0.5d) -> S8-4 (2-3d) -> S8-6 (0.5d) = 3-4 days +``` + +### Effort Summary + +| Track | Items | Effort | +|-------|-------|--------| +| Server (Dudley) | S8-1, S8-3, S8-4, S8-6, S8-7, S8-8, S8-9 | ~5-6d | +| Tooling (Justine) | S8-2, S8-5, S8-10 | ~1.75d | +| **Total** | **10 items** | **~7-8 team-days** | + +### Not In Sprint 8 + +Gauntlet rooms, anti-tedium features, crossterm display, CI, perf baselines, encoding asymmetry tests, client tests, cross-room transitions. + +### Determinism Fixes + +| Fix | File(s) | Lines | What | +|-----|---------|-------|------| +| A | `query.rs`, `observer/mod.rs` | ~15 | `HashSet` -> `BTreeSet` for visible_positions/ids, sort visible_tiles by (x,y) | +| B | `observer/mod.rs` | 2 | Sort entities by entity_id in snapshot | +| C | (already done) | 0 | Monologue system ordering | +| D | `movement.rs` | ~5 | Sort movers by `Entity::to_bits()` for deterministic collision | + +Each fix has a copy-pasteable regression test. Total: ~22 lines production code. + +### Server `--test-mode` + +- Prints `LISTENING:{port}` to stdout after bind, before accept +- All tracing to stderr (stdout clean for port discovery) +- `--port 0` for OS-assigned port (uses existing `accept_on(listener)`) +- `--seed 42` default in test-mode +- Exits after first client disconnect +- `setup_proof_room()` extracted for both modes; Gauntlet content loads when ready + +### Bug Fix: SetTickRate While Paused + +`SetTickRate(Half)` while paused now rejected (was unconditionally setting rate, bypassing pause guard). 4-line fix in `input.rs`. Test: `set_tick_rate_rejected_while_paused`. + +--- + +## 3. Sprint 9+ Roadmap + +### Tier 1: Sprint 9 (~10-12 team-days) + +| # | Item | Owner | Effort | +|---|------|-------|--------| +| R-01 | Gauntlet rooms 1-4 | Content + Dudley | 3-4d | +| R-02 | Room reset trigger | Dudley | 1.5d | +| R-03 | Hub teleport | Dudley | 0.5d | +| R-04 | Client tests P0-P1 (10 tests) | Stig | 2-3d | +| R-05 | Determinism golden files (per-room) | Dudley | 1d | +| R-06 | Fog byte constants | Stig | 0.25d | +| R-07 | Encoding asymmetry tests (4-direction) | Hoshe + Stig | 1.5d | + +### Tier 2: Sprint 10 (~15-17 team-days) + +| # | Item | Owner | Effort | +|---|------|-------|--------| +| R-08 | WRONG button MVP | Stig + Dudley | 2d | +| R-09 | Gauntlet rooms 5-8 | Content + Dudley | 3-4d | +| R-10 | Room timer + personal bests | Ozzie spec, Stig impl | 1d | +| R-11 | Checklist auto-tracking | Stig + Dudley | 2d | +| R-12 | Performance baselines | Justine | 1.5d | +| R-13 | Client tests P2 (16 tests) | Stig | 3d | +| R-14 | Enhanced test client terminal | Dudley | 2d | + +### Tier 3: Sprint 11+ (build when needed) + +Gauntlet rooms 9-14, cross-room transitions (T1-T8), CI automation (~1d when greenlighted), content scaling stress, `blocked_entities`, client tests P3, WRONG button full capture, additional pause guard tests. + +--- + +## 4. Gauntlet Map Specification + +### Room List (7 rooms + Central Hub) + +| # | Room | Origin | Size | Observer | Facing | Entities | Key Systems | +|---|------|--------|------|----------|--------|----------|-------------| +| 0 | Central Hub | (38,46) | 24x24 | (50,58) | -- | 4 signs | Connector, spawn point | +| 1 | Fog Theater | (28,2) | 44x32 | (56,18) | South | 4 | D-059 fog layers, D-015 vision cone, D-060 cognitive delay | +| 2 | Occlusion Corridor | (74,48) | 42x22 | (84,58) | East | 4 | D-035 LOS, D-017 perception modes, shadowcasting | +| 3 | Inventory Warehouse | (2,40) | 30x28 | (17,54) | East | 11 | D-065 9-slot inventory, pickup/drop, CarriedBy | +| 4 | Interaction Gallery | (2,82) | 24x20 | (14,92) | East | 5 | D-057 verb system, D-055 sprint suppression | +| 5 | Pause Chamber | (42,78) | 16x16 | (50,86) | North | 1 | D-031 pause, Bug #3 regression | +| 6 | Dialogue Room | (36,104) | 28x20 | (50,114) | North | 4 | D-041 KG, D-028 dialogue pools, D-033 colors, D-062 locked options | +| 7 | Crowd Plaza | (80,78) | 32x32 | (96,94) | West | 15 | Density stress, INV-T01 determinism, INV-T02 tick budget | + +### Physical Layout + +``` + ┌─────────────────────────────┐ + │ FOG THEATER │ + │ (44x32 tiles) │ + └────────────┬────────────────┘ + │ corridor-N + │ +┌──────────────────┐ ┌─────────┴──────────┐ ┌──────────────────────────┐ +│ INVENTORY │ │ │ │ OCCLUSION CORRIDOR │ +│ WAREHOUSE ├──┤ CENTRAL HUB ├──┤ (42x22 tiles) │ +│ (30x28 tiles) │ │ (24x24 tiles) │ │ │ +└────────┬─────────┘ └────────┬───────────┘ └─────────────┬───────────┘ + │ │ │ + cross-cut-W corridor-S cross-cut-E + │ │ │ +┌────────┴─────────┐ ┌───────┴───────────┐ ┌─────────────┴───────────┐ +│ INTERACTION │ │ PAUSE CHAMBER │ │ CROWD PLAZA │ +│ GALLERY │ │ (16x16 tiles) │ │ (32x32 tiles) │ +│ (24x20 tiles) │ └───────┬───────────┘ │ │ +└──────────────────┘ │ └────────────────────────┘ + corridor-S2 + │ + ┌──────┴──────────┐ + │ DIALOGUE ROOM │ + │ (28x20 tiles) │ + └─────────────────┘ +``` + +### Total Entity Count: 48 + +| Room | StableId Range | Count | +|------|---------------|-------| +| Hub (signs) | 1-4 | 4 | +| Fog Theater | 5-8 | 4 | +| Occlusion Corridor | 9-12 | 4 | +| Inventory Warehouse | 13-23 | 11 | +| Interaction Gallery | 24-28 | 5 | +| Pause Chamber | 29 | 1 | +| Dialogue Room | 30-33 | 4 | +| Crowd Plaza | 34-48 | 15 | + +Player entity: StableId 0. Map bounds: 0-116 x 0-124 sim tiles. + +**Additive-only constraint:** Existing rooms and entities NEVER reordered. New entries append. This preserves golden file stability and StableId assignments. + +### Cross-Room Transitions (3 MVP) + +| # | Name | Path | Systems Tested | +|---|------|------|---------------| +| T1 | Sprint Exit | Crowd Plaza -> cross-cut-E -> Occlusion | Sprint buffer clear + LOS recalculation | +| T3 | Full Inventory Interact | Inventory -> cross-cut-W -> Interaction | Inventory limit + verb computation | +| T6 | Pause Anywhere | Pause -> Hub -> any room | Pause persistence across teleport | + +### Constants Module + +`server/src/test_world/constants.rs`: `GauntletRoom` struct, `GauntletEntity` struct, 8 room constants, `ROOMS` array, `room_at()` lookup function. Full Rust code (~100 lines) in `gestalt-round3.md`. + +### Deferred Rooms (Sprint 9+) + +Zone Gate, Eavesdrop Alcove, Confrontation Stage, Sprint Gauntlet, Sound Lab, Decay Observatory, Shift Change — all depend on systems not yet implemented. + +--- + +## 5. Test Client Binary Specification + +### Location + +`tooling/test-client/` — separate workspace crate. Imports bridge types from server crate (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed` — all already pub-exported). + +### CLI (Sprint 8 MVP) + +``` +settled-reach-test-client [OPTIONS] + + --connect Server address (default: 127.0.0.1:9876) + --replay JSONL input file (one JSON array per tick) + --text Structured text to stdout (default) + --json JSON to stdout (for golden files) + --quiet No output (CI assertions only) + --golden Compare final snapshot, exit 1 on diff + --ticks Disconnect after N ticks +``` + +Exit codes: 0 = success, 1 = golden file mismatch, 2 = connection/protocol error. + +### Connection Protocol + +1. Server binds, prints `LISTENING:{port}\n` to stdout, blocks on accept +2. Test client connects via TCP +3. Per-tick: server sends `ObserverSnapshot` (length-prefixed MessagePack), client receives, formats output, sends `Vec` from replay file +4. After `--ticks N`: compare golden file if specified, drop connection, exit + +Uses `rmp_serde::to_vec` (NOT `to_vec_named`) to match GDScript encoding. + +### Text Output Format + +``` +=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === +Game time: Day 0, 04:12 (Morning) +Room: Occlusion Corridor +Entities (5): + npc:100 (18,10) Forward rel:Neutral vis:Visible d=3 + npc:101 (20,10) Forward rel:Unknown vis:Remembered d=5 + obj:200 (16,9) Forward rel:n/a vis:Visible d=1 +Pending recognitions: 1 [npc:104 at (19,12) 3/8 ticks] +Tiles: 31 visible +Interactions (2): + npc:100 [Talk(1), ExamineNpc(2)] distance=3 +Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] +Monologue: "Something about this manifest doesn't add up." +=== +``` + +Entity labels: `kind:entity_id`. Sorted by distance (nearest first), ties by entity_id. Sections with no data omitted. Room name from `room_at()` constants function. + +### Text Renderer + +`server/src/bridge/text_renderer.rs`: library function `format_snapshot_text(&ObserverSnapshot) -> String`. Pub-exported from server crate. ~100 lines Rust. Full implementation in `tyre-round3.md`. + +### Golden File Comparison + +JSON with sorted keys, pretty-printed. Recursive `diff_json_values()` comparison. Output on mismatch: +``` +GOLDEN FILE MISMATCH: tick50.json + .entities[2].x: expected 18.0, got 19.0 (CHANGED) + .visible_tiles: 31 expected, 29 actual (2 REMOVED) +``` + +### Replay Format + +JSONL (one JSON line per tick). Each line: JSON array of `PlayerInput`. Empty array `[]` = idle tick. + +### Sprint 9+ CLI Additions + +`--interactive`, `--live` (crossterm), `--log`, `--history-buffer`, `--checklist`. + +### Layer 3 Integration Test + +`server/tests/layer3.rs` (or `layer3_subprocess.rs`): spawns server + test client as subprocesses, verifies end-to-end TCP roundtrip. `ServerGuard` drop pattern for cleanup. Full code in `hoshe-round3.md` and `tyre-round3.md`. + +### File Inventory + +| File | Sprint | Description | +|------|--------|-------------| +| `tooling/test-client/src/main.rs` | 8 | Binary entry point | +| `server/src/bridge/text_renderer.rs` | 8 | `format_snapshot_text()` library | +| `server/src/test_world/constants.rs` | 8 | Room/entity constants | +| `server/tests/layer3.rs` | 8 | Subprocess integration test | +| `server/src/bridge/mod.rs` | 8 | Add `pub mod text_renderer;` | + +--- + +## 6. Prioritized Test Backlog + +### Sprint 8 P0 — 10 items (~7d) + +| # | Title | Team | Effort | Bug Class | +|---|-------|------|--------|-----------| +| 1 | Determinism Fix A: BTreeSet + sort visible_tiles | server | 0.5d | Bug #1 (state divergence) | +| 2 | Determinism Fix B: Sort entities by entity_id | server | 0.25d | Golden file non-determinism | +| 3 | Determinism Fix D: Sort movers | server | 0.25d | Movement tie-breaking | +| 4 | Server `--test-mode` + `--port 0` | server | 1d | Unblocks all test infrastructure | +| 5 | `make pre-pr` target | joint | 0.5d | Developer discipline | +| 6 | Pause guard: movement_discarded_while_paused | server | 0.25d | Bug #3 | +| 7 | Pause guard: unpause_accepted_while_paused | server | 0.25d | Bug #3 class | +| 8 | Pause guard: roundtrip_with_movement | server | 0.25d | Bug #3 class | +| 9 | Content cross-reference validation (9 checks) | joint | 1.5d | Content scaling | +| 10 | Fixture staleness check | joint | 0.25d | Bug #4 class (protocol drift) | + +### Sprint 8 P1 — 15 items (~10.75d) + +| # | Title | Team | Effort | +|---|-------|------|--------| +| 11 | Determinism regression test: `gauntlet_deterministic_replay` | server | 1d | +| 12 | Per-fix determinism unit tests | server | 0.5d | +| 13 | Remaining pause guard tests (3 edge cases) | server | 0.5d | +| 14 | EntityRegistry lifecycle tests (3 tests) | server | 0.5d | +| 15 | Boundary value tests: GDScript encode-only (41 values) | client | 1d | +| 16 | Boundary value tests: Rust encode-only + roundtrip | server | 0.5d | +| 17 | gen_fixtures.rs boundary extension | joint | 0.5d | +| 18 | Encoding asymmetry tests (4 directions) | joint | 1d | +| 19 | `make fixtures-client` target | joint | 0.5d | +| 20 | Fog byte value constants | client | 0.25d | +| 21 | Client P0: monologue not lost, camera static during pause | client | 0.5d | +| 22 | Client P1: fog (4), entity lifecycle, recognition blob | client | 1d | +| 23 | `malformed_input_in_batch_rejects_entire_batch` | server | 0.25d | +| 24 | Test client binary scaffolding + CLI | server | 0.5d | +| 25 | Text renderer library | server | 0.5-1d | + +### Sprint 9 P0 — 10 items (~7.5d) + +Layer 3 test, test client replay/golden/comparison, golden file suite, Gauntlet first 3-4 rooms, constants module, content runtime validation, room reset, hub teleport. + +### Sprint 9 P1 — 12 items (~12.5d) + +Client tests P2 (16 tests), client P3 (12 tests), client anti-tedium tests, WRONG button MVP, room timer, checklist auto-tracking, Gauntlet rooms 5-8, performance baseline tooling, content scaling test, hub teleport UX, room reset UX, auto-checklist. + +### Sprint 10+ — 12 items (~18.75d) + +Gauntlet rooms 9-14, cross-room transitions, WRONG button full, map-agnostic invariants (36), fuzzy tests, CI pipeline, F3 debug overlay, zone gate, content scaling stress, blocked_entities, bidirectional relationship warnings. + +### Bug Catalogue Coverage + +| Bug | Covered By | +|-----|-----------| +| #1 (snapshot delivery) | `--test-mode` (#4), Layer 3 test (#26) | +| #2 (camera startup) | Client P0 (#21) | +| #3 (movement while paused) | Pause guard suite (#6-8, #13) | +| #4 (MessagePack -128) | Boundary value matrix (#15-18) | +| #5 (monologue overwrite) | Client P0 (#21) | +| #6 (snapshot spam) | Determinism fixes (#1-3), golden files (#29) | + +### Client Tests: 38 Total + +| Priority | Count | Sprint | Categories | +|----------|-------|--------|-----------| +| P0 | 2 | 8 | Monologue overwrite, camera pause | +| P1 | 7 | 8-9 | Fog (4), entity lifecycle (2), recognition blob | +| P2 | 24 | 9 | Camera (5), entity (5), UI (12), lerp (1), teleport (1) | +| P3 | 5 | 10+ | Z-layer (4), lerp target (1) | + +--- + +## 7. Content Validation Specification + +### Architecture + +Extends `tooling/validate-content` (Python). Two-pass: schema validation (existing) then cross-reference validation (new). No Rust dependency. + +### The 9 Checks + +| # | Check | Severity | What | +|---|-------|----------|------| +| 1 | `canonical_id_uniqueness` | ERROR | Duplicate NPC canonical_ids | +| 2 | `relationship_target_resolution` | ERROR | Relationship targets resolve to defined NPCs | +| 3 | `location_slug_resolution` | ERROR | District locations match location files | +| 4 | `dialogue_location_resolution` | ERROR | Dialogue pool locations match district | +| 5 | `fact_id_resolution` | ERROR | fact_ids resolve to knowledge catalogs (absorbs `check-fact-ids`) | +| 6 | `triangle_membership_resolution` | ERROR | Triangle references match triangle files | +| 7 | `npc_count_accuracy` | WARNING | Declared npc_count matches actual file count | +| 8 | `dialogue_line_id_uniqueness` | ERROR | No duplicate line IDs within a dialogue file | +| 9 | `bidirectional_relationship_consistency` | WARNING | Asymmetric relationships flagged | + +### Implementation + +`ContentIndex` Python class: scans NPCs, locations, triangles, knowledge catalogs, districts, dialogue. `validate_references()` runs all 9 checks, returns (error_count, warning_count). Schema errors fail-fast before cross-reference pass. + +### Phased Rollout + +| Phase | Checks | Sprint | +|-------|--------|--------| +| 1 | #1, #2, #3, #6 (NPC/triangle/district) | Sprint 8 | +| 2 | #4, #5, #8 (dialogue/fact_ids) | Sprint 8 | +| 3 | #7, #9 (warnings) | Sprint 9 | + +--- + +## 8. Anti-Tedium Specifications + +### Priority Ranking + +| Priority | Feature | Sprint | Effort | Justification | +|----------|---------|--------|--------|--------------| +| P1 | Room Reset Triggers | 9 | 1-1.5d | Can't re-test without server restart | +| P2 | WRONG Button MVP | 9 | 1d | Can't report bugs efficiently | +| P3 | Hub Teleport | 9 | 0.5d | Convenience for room navigation | +| P4 | Timer + Checklist | 9 | 1.5-2d | Engagement, not necessity | + +*Note: Ozzie/Gestalt prioritize P1-P2 for Sprint 8. Tyre's Sprint 8 plan (authoritative for scope) defers all anti-tedium to Sprint 9.* + +### Room Reset — Full Specification + +| Aspect | Spec | +|--------|------| +| Trigger | Step on ResetPlate tile + press Interact (NOT automatic) | +| Server | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system | +| Resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles), inventory items sourced from room, dialogue state | +| Does NOT reset | Other rooms, player position, session timer, global SimRng state | +| Debounce | 10-tick cooldown | +| Test-mode only | `RoomResetTrigger` entities only added with `--test-mode` | +| Client visual | Amber `reset_plate` tile, "Reset Room" interaction verb, 0.15s amber flash + monologue "Systems recalibrated." | + +### WRONG Button MVP (F12) + +| Aspect | Spec | +|--------|------| +| Hotkey | F12 | +| Captures | Current ObserverSnapshot (JSON), text render output, tick/room/seed, tester description (one line) | +| Output | `tests/bug-reports/gauntlet-t{tick}-{timestamp}/` with report.md, snapshot.json, text_output.txt, description.txt | +| Server changes | Zero (MVP) | +| UX flow | F12 -> pause -> one-line prompt -> save files -> unpause -> resume | + +**Full version (Sprint 9+):** 60-tick ring buffer, input history, replay seed, room metadata. + +**Godot client (`bug_report.gd`):** ~80 lines autoload, F12 handler, ring buffer, modal prompt, screenshot + scene tree dump. Full code in `stig-round3.md`. + +### Hub Teleport + +| Aspect | Spec | +|--------|------| +| Hotkey | Home | +| Wire format | `PlayerAction::TeleportToHub` | +| Server | Move player to hub_spawn, clear dialogue/monologue/interaction buffer | +| Does NOT affect | Room state, inventory, game time, knowledge graph | +| Client | Instant camera snap + 0.3s fade-to-black-and-back | +| Gauntlet-only | Server rejects in non-Gauntlet maps | + +### Room Timer + Personal Bests + +Display `TIMER: 00:47 (PB: 00:38)`. Timer starts on room entry, resets on room reset. Stats persisted to `tests/gauntlet-stats.json`. Session summary on disconnect. + +### Deferred: F3 Debug Overlay + +Deferred indefinitely per Stig's recommendation. WRONG button captures same data on demand. F3 real-time overlay has measurable per-frame performance cost. + +--- + +## 9. Human Tester Workflow + +### Prerequisites + +1. Build: `make build-server` + `make build-test-client` +2. Gauntlet content at `content/gauntlet/` +3. Checklist: `make checklist` -> `docs/qa/gauntlet-checklist.md` + +### Session Start + +**Terminal 1:** `make test-world-headless` (server with `--test-mode --port 0 --seed 42`) +**Terminal 2:** `make test-client` (connects, live terminal display) + +### 6-Phase Workflow + +1. **Navigate to a Room** — Walk from Hub. Room name, timer, and checklist update automatically. +2. **Execute the Checklist** — Follow printed checklist. Terminal shows entities with visibility symbols, fog counts, interactions. +3. **Encounter a Bug** — Press F12. Game pauses. Type one sentence. 4 files saved. Resume. +4. **Reset and Re-Test** — Walk to reset plate, interact. Room reverts to tick-0. Timer resets. Retry. +5. **Move to Another Room** — Press Home -> Hub. Walk to next room. Repeat. +6. **End Session** — Ctrl+C. Summary: rooms tested, coverage %, times, bug reports filed. + +### Quick-Test Developer Workflow + +**Target: 65 seconds.** Build (10s) -> start (3s) -> navigate (5s) -> test (45s) -> exit (2s). + +### Entity Visibility Symbols + +| Symbol | State | Meaning | +|--------|-------|---------| +| `●` | VISIBLE | In clear vision cone | +| `◐` | REMEMBERED | Previously seen, now in fog | +| `◌` | FOGGED | Detected but not recognized | +| `✕` | BLOCKED | LOS blocked by wall (debug, Sprint 9+) | +| `⚡` | RECOGNIZING | Mid-cognitive-delay | + +### Display Sections (10 total) + +1. Header (tick, rate, room, seed, timer) +2. Player (position, facing, stance, inventory count) +3. Entities (sorted by distance, visibility symbols) +4. Fog (5-layer tile counts) +5. Sound (Sprint 9+ — requires `Vec`) +6. Cognition (Sprint 9+ — requires enhanced `pending_recognitions`) +7. Interactions (available verbs per entity) +8. Monologue/Dialogue (exact text) +9. Inventory (slot map) +10. Status (checklist, timer, PB, hotkeys) + +**Sprint 8 MVP:** Sections 1-4, 7-10. Sprint 9+ adds sections 5-6. + +### Checklist YAML + +Per-room at `content/gauntlet/rooms/{room_id}/checklist.yaml`. Cross-room at `content/gauntlet/cross_room_checks.yaml`. Each check has: id, description, type (auto/manual), step, condition (structured), if_wrong (debug guidance). + +7 condition types: `player_near`, `player_facing`, `entity`+`expected`, `expected_sector`, `perception_mode`, `fog_visible_count_min`/`max`, `inventory_count`, `dialogue_active`, `monologue_contains`. + +--- + +## 10. CI Pipeline Design + +### Status: Deferred (documented, ready for when lead greenlights) + +### 3-Tier Pipeline + +| Tier | Trigger | Budget | Contents | +|------|---------|--------|----------| +| **Commit** | Every push | <2 min | lint-server, lint-client, validate-content, check-fact-ids | +| **PR** (merge gate) | PR opened/updated | <15 min | Commit tier + build, test, fixture staleness (BLOCKER) | +| **Nightly** | Scheduled 03:00 UTC | <30 min | PR tier + Layer 3, golden files, perf benchmarks, content scaling | + +### Merge-Blocking Policy + +| Job | Required? | Rationale | +|-----|-----------|-----------| +| `commit-checks` | **Yes** | Fast lint + content validation | +| `server-build-test` | **Yes** | Tests + fixture staleness = correctness | +| `client-build-test` | **Yes** | Client rendering contract | +| `nightly` | **No** | Deep tests are informational | + +### `make pre-pr` — Interim CI + +Until Gitea Actions are greenlighted, `make pre-pr` is the developer discipline tool: + +``` +pre-pr (~2.5 min incremental) + ├── 1. lint (server + client) ~15s + ├── 2. build (server + client) ~30-90s + ├── 3. test (server + client) ~15-30s + ├── 4. validate-content + fact-ids ~5s + └── 5. fixtures-check (BLOCKER) ~10-15s +``` + +Branch variants: `pre-pr-server`, `pre-pr-client`, `pre-pr-content`. + +### Runner Requirements + +Self-hosted runner required. Pre-installed: Rust + clippy + rustfmt, cargo-nextest, Godot 4.6 headless, Python 3 + jsonschema + pyyaml. + +### Wiring Effort + +~1 day. Makefile targets already exist. Workflow file is the only new artifact. Runner setup ~0.5 day. + +### Complete Workflow File + +`.gitea/workflows/ci.yaml` (~100 lines) provided in `justine-round3.md`. Ready to deploy. + +--- + +## Appendix A: Risk Register + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| Test client takes >3 days | Medium | Delays Layer 3 | MVP scope intentionally minimal | +| Proof room too simple for golden files | Low | Low value until rooms exist | Proves toolchain; value Sprint 9 | +| Content validation finds many errors | Medium | Sprint 8 effort bloat | Run as WARNING first sprint | +| Client test headless blocks CI | Low | Can't automate client tests | Manual `make pre-pr-client` interim | +| Fixture BLOCKER frustrates devs | Low | Devs skip pre-pr | Education; false positives worse | + +## Appendix B: Source Files + +| File | Agent | Lines | Content | +|------|-------|-------|---------| +| `ozzie-round3.md` | Ozzie | 521 | Tester walkthrough, UX spec, WRONG button, anti-tedium priority | +| `justine-round3.md` | Justine | 997 | pre-pr, perf-baseline, golden files, CI pipeline, fixture staleness | +| `stig-round3.md` | Stig | 638 | 38 client tests, anti-tedium UI, checklist YAML, fog constants | +| `dudley-round3.md` | Dudley | 1039 | Determinism fixes, --test-mode, Gauntlet loader, Q7/Q8 answers | +| `hoshe-round3.md` | Hoshe | 677 | 59-item backlog, content validation, Layer 3 test, boundary values | +| `tyre-round3.md` | Tyre | 646 | Sprint 8 plan, test client spec, Sprint 9+ roadmap, risk register | +| `gestalt-round3.md` | Gestalt | 765 | Map layout, 48 entities, cross-room transitions, constants module | + +## Appendix C: Boundary Value Matrix (41 values) + +### Positive Boundaries (25 values) + +| Range | Values | GDScript Format | Rust Format | +|-------|--------|----------------|-------------| +| 0-127 | 0, 1, 126, 127 | pos fixint | pos fixint | +| 128-255 | 128, 129, 254, 255 | uint 8 | uint 8 | +| 256-32767 | 256, 257, 32766, 32767 | **int 16** | **uint 16** | +| 32768-65535 | 32768, 32769, 65534, 65535 | uint 16 | uint 16 | +| 65536-2^31-1 | 65536, 65537, 2147483646, 2147483647 | **int 32** | **uint 32** | +| 2^31-2^32-1 | 2147483648, 4294967294, 4294967295 | uint 32 | uint 32 | +| 2^32+ | 4294967296, 2^63-1 | int 64 | int 64 | + +**Bold rows** = encoding asymmetry between GDScript and Rust. Both are spec-valid. Both decoders accept both encodings (verified). + +### Negative Boundaries (16 values) + +-1, -31, -32 (neg fixint); -33, -34, -127, -128 (int 8); -129, -130, -32767, -32768 (int 16); -32769, -2147483647, -2147483648 (int 32); -2147483649, -2^63 (int 64). + +## Appendix D: Map-Agnostic Invariants (from Round 1) + +36 invariants across 4 categories. These must hold for ANY valid map. + +- **Structural (INV-S01-S08):** Spawn reachable, NPC paths valid, no entity inside geometry, 2x2 minimum, door bidirectionality. +- **Perception (INV-P01-P05):** Vision >0 at spawn, LOS symmetry, fog layer ordering, insert independence, sound coherence. +- **Population (INV-C01-C08):** Minimum nearby NPC, StableId uniqueness, KG validity, D-033 colors, monologue reachability, dialogue pool non-empty. +- **Simulation (INV-T01-T08):** Deterministic replay, tick budget (<=100ms), snapshot delivery, pause coherence, input ordering, SimRng order, cognitive delay monotonicity, knowledge decay timing. + +Top 10 for Sprint 8 (from Gestalt): INV-T04 (pause), INV-T01 (determinism), INV-T03 (snapshot delivery), INV-T05 (input ordering), INV-S01 (spawn reachable), INV-C03 (StableId uniqueness), INV-C07 (dialogue pool), INV-T02 (tick budget), INV-P02 (LOS symmetry), INV-S05 (no entity in geometry).