# QA Strategy & Test Architecture Workshop — Outcomes **Workshop:** QA Strategy & Test Architecture **Rounds:** 3 (Analysis -> Synthesis -> Prioritization) **Date:** 2026-02-17 **Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie **Documenter:** Qatux This is the definitive output of the workshop. All specifications are build-ready. Round notes are at `docs/workshops/test-architecture/round-{1,2,3}-notes.md`. --- ## Table of Contents 1. [Decision Summary](#1-decision-summary) 2. [Sprint 8 Implementation Plan](#2-sprint-8-implementation-plan) 3. [Sprint 9+ Roadmap](#3-sprint-9-roadmap) 4. [Gauntlet Map Specification](#4-gauntlet-map-specification) 5. [Test Client Binary Specification](#5-test-client-binary-specification) 6. [Prioritized Test Backlog](#6-prioritized-test-backlog) 7. [Content Validation Specification](#7-content-validation-specification) 8. [Anti-Tedium Specifications](#8-anti-tedium-specifications) 9. [Human Tester Workflow](#9-human-tester-workflow) 10. [CI Pipeline Design](#10-ci-pipeline-design) --- ## 1. Decision Summary ### Workshop Decisions (confirmed by lead) | ID | Decision | Round | Dissent | |----|----------|-------|---------| | WS-D1 | **Commit to determinism now.** 3 targeted fixes (A, B, D), ~22 lines. Fix C already done. | R1, confirmed R2 | None | | WS-D2 | **Gauntlet as hybrid YAML + Rust inject.** YAML for geometry + entity placement, Rust for KG state injection. | R1, confirmed R2 | None | | WS-D3 | **Rust test macros with named helpers. No custom DSL.** DSL revisitable Sprint 10+. | R1, confirmed R2 | None | | WS-D4 | **Golden files as ObserverSnapshot at fixed positions.** JSON format, sorted keys, field-by-field diff. | R1, confirmed R2 | None | | WS-D5 | **Separate test client binary** (lead override of R1 consensus on server-side text renderer). Binary at `tooling/test-client/` (lead override of R2 location at `server/src/bin/`). | R1 overruled R2, location overruled R3 | Tyre accepted both overrules constructively | ### Resolved Questions | ID | Question | Answer | |----|----------|--------| | OQ-01 | Client depends on visible_tiles ordering? | No. Fix safe to ship. | | OQ-02 | rmp_serde accepts int_16 for u64? | Yes. Traced through rmp-serde 1.3.1. | | OQ-04 | WalkabilityMap HashMap -> BTreeMap? | No. Point-lookup only. | | OQ-05 | Fixture staleness: git diff robust enough? | Yes. Deterministic generation verified. | | OQ-07 | Server --test-mode and --port 0? | Designed and specified. | | OQ-08 | Room name from coordinates? | `room_at()` in constants module. | | OQ-10 | Fog byte value constants? | Yes. 6 named constants. | | R2-OQ-01 | SetTickRate while paused? | Bug. Reject. 4-line fix. | | R2-OQ-02 | Entity index recycling? | Safe (bevy generation counter). | | R2-OQ-03 | content-ron in pre-pr? | Yes, in full pre-pr only. | | R2-OQ-05 | blocked_entities feasibility? | Feasible. Sprint 9. ~300 tile lookups/tick. | | R2-OQ-06 | Checklist overlay in Godot? | Lightweight overlay. Full tracking test-client-only. | | R2-OQ-08 | Cross-room checklist location? | `content/gauntlet/cross_room_checks.yaml` | | R2-OQ-09 | Room ordering for StableId? | Canonical order required. Append-only. | | R2-OQ-10 | Room reset vs full restart? | Both. Different test types need different levels. | ### Fixture Staleness: BLOCKER Tyre's argument (R2): stale fixtures mean client tests run against outdated protocol data — every passing client test is a false positive. Accepted by all agents. `make pre-pr` exits 1 on stale fixtures. --- ## 2. Sprint 8 Implementation Plan *Sprint 8 ships infrastructure — the plumbing. Sprint 9 fills the pipes.* (Tyre) ### Items | # | Item | Owner | Effort | Depends On | |---|------|-------|--------|------------| | S8-1 | Determinism fixes (A, B, D) | Dudley | 0.5d | -- | | S8-2 | Content cross-reference validation (9 checks) | Justine | 1d | -- | | S8-3 | `--test-mode` + `--port 0` server flags | Dudley | 0.5d | -- | | S8-4 | Test client binary MVP | Dudley | 2-3d | S8-3 | | S8-5 | `make pre-pr` chain | Justine | 0.5d | S8-1, S8-2 | | S8-6 | Layer 3 test wiring | Dudley | 0.5d | S8-3, S8-4 | | S8-7 | Pause guard tests (6 tests) | Dudley | 0.5d | -- | | S8-8 | Determinism regression tests | Dudley | 0.5d | S8-1 | | S8-9 | EntityRegistry lifecycle tests (3 tests) | Dudley | 0.25d | -- | | S8-10 | Fixture staleness check | Justine | 0.25d | S8-5 | ### Critical Path ``` S8-3 (0.5d) -> S8-4 (2-3d) -> S8-6 (0.5d) = 3-4 days ``` ### Effort Summary | Track | Items | Effort | |-------|-------|--------| | Server (Dudley) | S8-1, S8-3, S8-4, S8-6, S8-7, S8-8, S8-9 | ~5-6d | | Tooling (Justine) | S8-2, S8-5, S8-10 | ~1.75d | | **Total** | **10 items** | **~7-8 team-days** | ### Not In Sprint 8 Gauntlet rooms, anti-tedium features, crossterm display, CI, perf baselines, encoding asymmetry tests, client tests, cross-room transitions. ### Determinism Fixes | Fix | File(s) | Lines | What | |-----|---------|-------|------| | A | `query.rs`, `observer/mod.rs` | ~15 | `HashSet` -> `BTreeSet` for visible_positions/ids, sort visible_tiles by (x,y) | | B | `observer/mod.rs` | 2 | Sort entities by entity_id in snapshot | | C | (already done) | 0 | Monologue system ordering | | D | `movement.rs` | ~5 | Sort movers by `Entity::to_bits()` for deterministic collision | Each fix has a copy-pasteable regression test. Total: ~22 lines production code. ### Server `--test-mode` - Prints `LISTENING:{port}` to stdout after bind, before accept - All tracing to stderr (stdout clean for port discovery) - `--port 0` for OS-assigned port (uses existing `accept_on(listener)`) - `--seed 42` default in test-mode - Exits after first client disconnect - `setup_proof_room()` extracted for both modes; Gauntlet content loads when ready ### Bug Fix: SetTickRate While Paused `SetTickRate(Half)` while paused now rejected (was unconditionally setting rate, bypassing pause guard). 4-line fix in `input.rs`. Test: `set_tick_rate_rejected_while_paused`. --- ## 3. Sprint 9+ Roadmap ### Tier 1: Sprint 9 (~10-12 team-days) | # | Item | Owner | Effort | |---|------|-------|--------| | R-01 | Gauntlet rooms 1-4 | Content + Dudley | 3-4d | | R-02 | Room reset trigger | Dudley | 1.5d | | R-03 | Hub teleport | Dudley | 0.5d | | R-04 | Client tests P0-P1 (10 tests) | Stig | 2-3d | | R-05 | Determinism golden files (per-room) | Dudley | 1d | | R-06 | Fog byte constants | Stig | 0.25d | | R-07 | Encoding asymmetry tests (4-direction) | Hoshe + Stig | 1.5d | ### Tier 2: Sprint 10 (~15-17 team-days) | # | Item | Owner | Effort | |---|------|-------|--------| | R-08 | WRONG button MVP | Stig + Dudley | 2d | | R-09 | Gauntlet rooms 5-8 | Content + Dudley | 3-4d | | R-10 | Room timer + personal bests | Ozzie spec, Stig impl | 1d | | R-11 | Checklist auto-tracking | Stig + Dudley | 2d | | R-12 | Performance baselines | Justine | 1.5d | | R-13 | Client tests P2 (16 tests) | Stig | 3d | | R-14 | Enhanced test client terminal | Dudley | 2d | ### Tier 3: Sprint 11+ (build when needed) Gauntlet rooms 9-14, cross-room transitions (T1-T8), CI automation (~1d when greenlighted), content scaling stress, `blocked_entities`, client tests P3, WRONG button full capture, additional pause guard tests. --- ## 4. Gauntlet Map Specification ### Room List (7 rooms + Central Hub) | # | Room | Origin | Size | Observer | Facing | Entities | Key Systems | |---|------|--------|------|----------|--------|----------|-------------| | 0 | Central Hub | (38,46) | 24x24 | (50,58) | -- | 4 signs | Connector, spawn point | | 1 | Fog Theater | (28,2) | 44x32 | (56,18) | South | 4 | D-059 fog layers, D-015 vision cone, D-060 cognitive delay | | 2 | Occlusion Corridor | (74,48) | 42x22 | (84,58) | East | 4 | D-035 LOS, D-017 perception modes, shadowcasting | | 3 | Inventory Warehouse | (2,40) | 30x28 | (17,54) | East | 11 | D-065 9-slot inventory, pickup/drop, CarriedBy | | 4 | Interaction Gallery | (2,82) | 24x20 | (14,92) | East | 5 | D-057 verb system, D-055 sprint suppression | | 5 | Pause Chamber | (42,78) | 16x16 | (50,86) | North | 1 | D-031 pause, Bug #3 regression | | 6 | Dialogue Room | (36,104) | 28x20 | (50,114) | North | 4 | D-041 KG, D-028 dialogue pools, D-033 colors, D-062 locked options | | 7 | Crowd Plaza | (80,78) | 32x32 | (96,94) | West | 15 | Density stress, INV-T01 determinism, INV-T02 tick budget | ### Physical Layout ``` ┌─────────────────────────────┐ │ FOG THEATER │ │ (44x32 tiles) │ └────────────┬────────────────┘ │ corridor-N │ ┌──────────────────┐ ┌─────────┴──────────┐ ┌──────────────────────────┐ │ INVENTORY │ │ │ │ OCCLUSION CORRIDOR │ │ WAREHOUSE ├──┤ CENTRAL HUB ├──┤ (42x22 tiles) │ │ (30x28 tiles) │ │ (24x24 tiles) │ │ │ └────────┬─────────┘ └────────┬───────────┘ └─────────────┬───────────┘ │ │ │ cross-cut-W corridor-S cross-cut-E │ │ │ ┌────────┴─────────┐ ┌───────┴───────────┐ ┌─────────────┴───────────┐ │ INTERACTION │ │ PAUSE CHAMBER │ │ CROWD PLAZA │ │ GALLERY │ │ (16x16 tiles) │ │ (32x32 tiles) │ │ (24x20 tiles) │ └───────┬───────────┘ │ │ └──────────────────┘ │ └────────────────────────┘ corridor-S2 │ ┌──────┴──────────┐ │ DIALOGUE ROOM │ │ (28x20 tiles) │ └─────────────────┘ ``` ### Total Entity Count: 48 | Room | StableId Range | Count | |------|---------------|-------| | Hub (signs) | 1-4 | 4 | | Fog Theater | 5-8 | 4 | | Occlusion Corridor | 9-12 | 4 | | Inventory Warehouse | 13-23 | 11 | | Interaction Gallery | 24-28 | 5 | | Pause Chamber | 29 | 1 | | Dialogue Room | 30-33 | 4 | | Crowd Plaza | 34-48 | 15 | Player entity: StableId 0. Map bounds: 0-116 x 0-124 sim tiles. **Additive-only constraint:** Existing rooms and entities NEVER reordered. New entries append. This preserves golden file stability and StableId assignments. ### Cross-Room Transitions (3 MVP) | # | Name | Path | Systems Tested | |---|------|------|---------------| | T1 | Sprint Exit | Crowd Plaza -> cross-cut-E -> Occlusion | Sprint buffer clear + LOS recalculation | | T3 | Full Inventory Interact | Inventory -> cross-cut-W -> Interaction | Inventory limit + verb computation | | T6 | Pause Anywhere | Pause -> Hub -> any room | Pause persistence across teleport | ### Constants Module `server/src/test_world/constants.rs`: `GauntletRoom` struct, `GauntletEntity` struct, 8 room constants, `ROOMS` array, `room_at()` lookup function. Full Rust code (~100 lines) in `gestalt-round3.md`. ### Deferred Rooms (Sprint 9+) Zone Gate, Eavesdrop Alcove, Confrontation Stage, Sprint Gauntlet, Sound Lab, Decay Observatory, Shift Change — all depend on systems not yet implemented. --- ## 5. Test Client Binary Specification ### Location `tooling/test-client/` — separate workspace crate. Imports bridge types from server crate (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed` — all already pub-exported). ### CLI (Sprint 8 MVP) ``` settled-reach-test-client [OPTIONS] --connect Server address (default: 127.0.0.1:9876) --replay JSONL input file (one JSON array per tick) --text Structured text to stdout (default) --json JSON to stdout (for golden files) --quiet No output (CI assertions only) --golden Compare final snapshot, exit 1 on diff --ticks Disconnect after N ticks ``` Exit codes: 0 = success, 1 = golden file mismatch, 2 = connection/protocol error. ### Connection Protocol 1. Server binds, prints `LISTENING:{port}\n` to stdout, blocks on accept 2. Test client connects via TCP 3. Per-tick: server sends `ObserverSnapshot` (length-prefixed MessagePack), client receives, formats output, sends `Vec` from replay file 4. After `--ticks N`: compare golden file if specified, drop connection, exit Uses `rmp_serde::to_vec` (NOT `to_vec_named`) to match GDScript encoding. ### Text Output Format ``` === Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full === Game time: Day 0, 04:12 (Morning) Room: Occlusion Corridor Entities (5): npc:100 (18,10) Forward rel:Neutral vis:Visible d=3 npc:101 (20,10) Forward rel:Unknown vis:Remembered d=5 obj:200 (16,9) Forward rel:n/a vis:Visible d=1 Pending recognitions: 1 [npc:104 at (19,12) 3/8 ticks] Tiles: 31 visible Interactions (2): npc:100 [Talk(1), ExamineNpc(2)] distance=3 Inventory: 2/9 [item:300(slot-0), item:301(slot-3)] Monologue: "Something about this manifest doesn't add up." === ``` Entity labels: `kind:entity_id`. Sorted by distance (nearest first), ties by entity_id. Sections with no data omitted. Room name from `room_at()` constants function. ### Text Renderer `server/src/bridge/text_renderer.rs`: library function `format_snapshot_text(&ObserverSnapshot) -> String`. Pub-exported from server crate. ~100 lines Rust. Full implementation in `tyre-round3.md`. ### Golden File Comparison JSON with sorted keys, pretty-printed. Recursive `diff_json_values()` comparison. Output on mismatch: ``` GOLDEN FILE MISMATCH: tick50.json .entities[2].x: expected 18.0, got 19.0 (CHANGED) .visible_tiles: 31 expected, 29 actual (2 REMOVED) ``` ### Replay Format JSONL (one JSON line per tick). Each line: JSON array of `PlayerInput`. Empty array `[]` = idle tick. ### Sprint 9+ CLI Additions `--interactive`, `--live` (crossterm), `--log`, `--history-buffer`, `--checklist`. ### Layer 3 Integration Test `server/tests/layer3.rs` (or `layer3_subprocess.rs`): spawns server + test client as subprocesses, verifies end-to-end TCP roundtrip. `ServerGuard` drop pattern for cleanup. Full code in `hoshe-round3.md` and `tyre-round3.md`. ### File Inventory | File | Sprint | Description | |------|--------|-------------| | `tooling/test-client/src/main.rs` | 8 | Binary entry point | | `server/src/bridge/text_renderer.rs` | 8 | `format_snapshot_text()` library | | `server/src/test_world/constants.rs` | 8 | Room/entity constants | | `server/tests/layer3.rs` | 8 | Subprocess integration test | | `server/src/bridge/mod.rs` | 8 | Add `pub mod text_renderer;` | --- ## 6. Prioritized Test Backlog ### Sprint 8 P0 — 10 items (~7d) | # | Title | Team | Effort | Bug Class | |---|-------|------|--------|-----------| | 1 | Determinism Fix A: BTreeSet + sort visible_tiles | server | 0.5d | Bug #1 (state divergence) | | 2 | Determinism Fix B: Sort entities by entity_id | server | 0.25d | Golden file non-determinism | | 3 | Determinism Fix D: Sort movers | server | 0.25d | Movement tie-breaking | | 4 | Server `--test-mode` + `--port 0` | server | 1d | Unblocks all test infrastructure | | 5 | `make pre-pr` target | joint | 0.5d | Developer discipline | | 6 | Pause guard: movement_discarded_while_paused | server | 0.25d | Bug #3 | | 7 | Pause guard: unpause_accepted_while_paused | server | 0.25d | Bug #3 class | | 8 | Pause guard: roundtrip_with_movement | server | 0.25d | Bug #3 class | | 9 | Content cross-reference validation (9 checks) | joint | 1.5d | Content scaling | | 10 | Fixture staleness check | joint | 0.25d | Bug #4 class (protocol drift) | ### Sprint 8 P1 — 15 items (~10.75d) | # | Title | Team | Effort | |---|-------|------|--------| | 11 | Determinism regression test: `gauntlet_deterministic_replay` | server | 1d | | 12 | Per-fix determinism unit tests | server | 0.5d | | 13 | Remaining pause guard tests (3 edge cases) | server | 0.5d | | 14 | EntityRegistry lifecycle tests (3 tests) | server | 0.5d | | 15 | Boundary value tests: GDScript encode-only (41 values) | client | 1d | | 16 | Boundary value tests: Rust encode-only + roundtrip | server | 0.5d | | 17 | gen_fixtures.rs boundary extension | joint | 0.5d | | 18 | Encoding asymmetry tests (4 directions) | joint | 1d | | 19 | `make fixtures-client` target | joint | 0.5d | | 20 | Fog byte value constants | client | 0.25d | | 21 | Client P0: monologue not lost, camera static during pause | client | 0.5d | | 22 | Client P1: fog (4), entity lifecycle, recognition blob | client | 1d | | 23 | `malformed_input_in_batch_rejects_entire_batch` | server | 0.25d | | 24 | Test client binary scaffolding + CLI | server | 0.5d | | 25 | Text renderer library | server | 0.5-1d | ### Sprint 9 P0 — 10 items (~7.5d) Layer 3 test, test client replay/golden/comparison, golden file suite, Gauntlet first 3-4 rooms, constants module, content runtime validation, room reset, hub teleport. ### Sprint 9 P1 — 12 items (~12.5d) Client tests P2 (16 tests), client P3 (12 tests), client anti-tedium tests, WRONG button MVP, room timer, checklist auto-tracking, Gauntlet rooms 5-8, performance baseline tooling, content scaling test, hub teleport UX, room reset UX, auto-checklist. ### Sprint 10+ — 12 items (~18.75d) Gauntlet rooms 9-14, cross-room transitions, WRONG button full, map-agnostic invariants (36), fuzzy tests, CI pipeline, F3 debug overlay, zone gate, content scaling stress, blocked_entities, bidirectional relationship warnings. ### Bug Catalogue Coverage | Bug | Covered By | |-----|-----------| | #1 (snapshot delivery) | `--test-mode` (#4), Layer 3 test (#26) | | #2 (camera startup) | Client P0 (#21) | | #3 (movement while paused) | Pause guard suite (#6-8, #13) | | #4 (MessagePack -128) | Boundary value matrix (#15-18) | | #5 (monologue overwrite) | Client P0 (#21) | | #6 (snapshot spam) | Determinism fixes (#1-3), golden files (#29) | ### Client Tests: 38 Total | Priority | Count | Sprint | Categories | |----------|-------|--------|-----------| | P0 | 2 | 8 | Monologue overwrite, camera pause | | P1 | 7 | 8-9 | Fog (4), entity lifecycle (2), recognition blob | | P2 | 24 | 9 | Camera (5), entity (5), UI (12), lerp (1), teleport (1) | | P3 | 5 | 10+ | Z-layer (4), lerp target (1) | --- ## 7. Content Validation Specification ### Architecture Extends `tooling/validate-content` (Python). Two-pass: schema validation (existing) then cross-reference validation (new). No Rust dependency. ### The 9 Checks | # | Check | Severity | What | |---|-------|----------|------| | 1 | `canonical_id_uniqueness` | ERROR | Duplicate NPC canonical_ids | | 2 | `relationship_target_resolution` | ERROR | Relationship targets resolve to defined NPCs | | 3 | `location_slug_resolution` | ERROR | District locations match location files | | 4 | `dialogue_location_resolution` | ERROR | Dialogue pool locations match district | | 5 | `fact_id_resolution` | ERROR | fact_ids resolve to knowledge catalogs (absorbs `check-fact-ids`) | | 6 | `triangle_membership_resolution` | ERROR | Triangle references match triangle files | | 7 | `npc_count_accuracy` | WARNING | Declared npc_count matches actual file count | | 8 | `dialogue_line_id_uniqueness` | ERROR | No duplicate line IDs within a dialogue file | | 9 | `bidirectional_relationship_consistency` | WARNING | Asymmetric relationships flagged | ### Implementation `ContentIndex` Python class: scans NPCs, locations, triangles, knowledge catalogs, districts, dialogue. `validate_references()` runs all 9 checks, returns (error_count, warning_count). Schema errors fail-fast before cross-reference pass. ### Phased Rollout | Phase | Checks | Sprint | |-------|--------|--------| | 1 | #1, #2, #3, #6 (NPC/triangle/district) | Sprint 8 | | 2 | #4, #5, #8 (dialogue/fact_ids) | Sprint 8 | | 3 | #7, #9 (warnings) | Sprint 9 | --- ## 8. Anti-Tedium Specifications ### Priority Ranking | Priority | Feature | Sprint | Effort | Justification | |----------|---------|--------|--------|--------------| | P1 | Room Reset Triggers | 9 | 1-1.5d | Can't re-test without server restart | | P2 | WRONG Button MVP | 9 | 1d | Can't report bugs efficiently | | P3 | Hub Teleport | 9 | 0.5d | Convenience for room navigation | | P4 | Timer + Checklist | 9 | 1.5-2d | Engagement, not necessity | *Note: Ozzie/Gestalt prioritize P1-P2 for Sprint 8. Tyre's Sprint 8 plan (authoritative for scope) defers all anti-tedium to Sprint 9.* ### Room Reset — Full Specification | Aspect | Spec | |--------|------| | Trigger | Step on ResetPlate tile + press Interact (NOT automatic) | | Server | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system | | Resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles), inventory items sourced from room, dialogue state | | Does NOT reset | Other rooms, player position, session timer, global SimRng state | | Debounce | 10-tick cooldown | | Test-mode only | `RoomResetTrigger` entities only added with `--test-mode` | | Client visual | Amber `reset_plate` tile, "Reset Room" interaction verb, 0.15s amber flash + monologue "Systems recalibrated." | ### WRONG Button MVP (F12) | Aspect | Spec | |--------|------| | Hotkey | F12 | | Captures | Current ObserverSnapshot (JSON), text render output, tick/room/seed, tester description (one line) | | Output | `tests/bug-reports/gauntlet-t{tick}-{timestamp}/` with report.md, snapshot.json, text_output.txt, description.txt | | Server changes | Zero (MVP) | | UX flow | F12 -> pause -> one-line prompt -> save files -> unpause -> resume | **Full version (Sprint 9+):** 60-tick ring buffer, input history, replay seed, room metadata. **Godot client (`bug_report.gd`):** ~80 lines autoload, F12 handler, ring buffer, modal prompt, screenshot + scene tree dump. Full code in `stig-round3.md`. ### Hub Teleport | Aspect | Spec | |--------|------| | Hotkey | Home | | Wire format | `PlayerAction::TeleportToHub` | | Server | Move player to hub_spawn, clear dialogue/monologue/interaction buffer | | Does NOT affect | Room state, inventory, game time, knowledge graph | | Client | Instant camera snap + 0.3s fade-to-black-and-back | | Gauntlet-only | Server rejects in non-Gauntlet maps | ### Room Timer + Personal Bests Display `TIMER: 00:47 (PB: 00:38)`. Timer starts on room entry, resets on room reset. Stats persisted to `tests/gauntlet-stats.json`. Session summary on disconnect. ### Deferred: F3 Debug Overlay Deferred indefinitely per Stig's recommendation. WRONG button captures same data on demand. F3 real-time overlay has measurable per-frame performance cost. --- ## 9. Human Tester Workflow ### Prerequisites 1. Build: `make build-server` + `make build-test-client` 2. Gauntlet content at `content/gauntlet/` 3. Checklist: `make checklist` -> `docs/qa/gauntlet-checklist.md` ### Session Start **Terminal 1:** `make test-world-headless` (server with `--test-mode --port 0 --seed 42`) **Terminal 2:** `make test-client` (connects, live terminal display) ### 6-Phase Workflow 1. **Navigate to a Room** — Walk from Hub. Room name, timer, and checklist update automatically. 2. **Execute the Checklist** — Follow printed checklist. Terminal shows entities with visibility symbols, fog counts, interactions. 3. **Encounter a Bug** — Press F12. Game pauses. Type one sentence. 4 files saved. Resume. 4. **Reset and Re-Test** — Walk to reset plate, interact. Room reverts to tick-0. Timer resets. Retry. 5. **Move to Another Room** — Press Home -> Hub. Walk to next room. Repeat. 6. **End Session** — Ctrl+C. Summary: rooms tested, coverage %, times, bug reports filed. ### Quick-Test Developer Workflow **Target: 65 seconds.** Build (10s) -> start (3s) -> navigate (5s) -> test (45s) -> exit (2s). ### Entity Visibility Symbols | Symbol | State | Meaning | |--------|-------|---------| | `●` | VISIBLE | In clear vision cone | | `◐` | REMEMBERED | Previously seen, now in fog | | `◌` | FOGGED | Detected but not recognized | | `✕` | BLOCKED | LOS blocked by wall (debug, Sprint 9+) | | `⚡` | RECOGNIZING | Mid-cognitive-delay | ### Display Sections (10 total) 1. Header (tick, rate, room, seed, timer) 2. Player (position, facing, stance, inventory count) 3. Entities (sorted by distance, visibility symbols) 4. Fog (5-layer tile counts) 5. Sound (Sprint 9+ — requires `Vec`) 6. Cognition (Sprint 9+ — requires enhanced `pending_recognitions`) 7. Interactions (available verbs per entity) 8. Monologue/Dialogue (exact text) 9. Inventory (slot map) 10. Status (checklist, timer, PB, hotkeys) **Sprint 8 MVP:** Sections 1-4, 7-10. Sprint 9+ adds sections 5-6. ### Checklist YAML Per-room at `content/gauntlet/rooms/{room_id}/checklist.yaml`. Cross-room at `content/gauntlet/cross_room_checks.yaml`. Each check has: id, description, type (auto/manual), step, condition (structured), if_wrong (debug guidance). 7 condition types: `player_near`, `player_facing`, `entity`+`expected`, `expected_sector`, `perception_mode`, `fog_visible_count_min`/`max`, `inventory_count`, `dialogue_active`, `monologue_contains`. --- ## 10. CI Pipeline Design ### Status: Deferred (documented, ready for when lead greenlights) ### 3-Tier Pipeline | Tier | Trigger | Budget | Contents | |------|---------|--------|----------| | **Commit** | Every push | <2 min | lint-server, lint-client, validate-content, check-fact-ids | | **PR** (merge gate) | PR opened/updated | <15 min | Commit tier + build, test, fixture staleness (BLOCKER) | | **Nightly** | Scheduled 03:00 UTC | <30 min | PR tier + Layer 3, golden files, perf benchmarks, content scaling | ### Merge-Blocking Policy | Job | Required? | Rationale | |-----|-----------|-----------| | `commit-checks` | **Yes** | Fast lint + content validation | | `server-build-test` | **Yes** | Tests + fixture staleness = correctness | | `client-build-test` | **Yes** | Client rendering contract | | `nightly` | **No** | Deep tests are informational | ### `make pre-pr` — Interim CI Until Gitea Actions are greenlighted, `make pre-pr` is the developer discipline tool: ``` pre-pr (~2.5 min incremental) ├── 1. lint (server + client) ~15s ├── 2. build (server + client) ~30-90s ├── 3. test (server + client) ~15-30s ├── 4. validate-content + fact-ids ~5s └── 5. fixtures-check (BLOCKER) ~10-15s ``` Branch variants: `pre-pr-server`, `pre-pr-client`, `pre-pr-content`. ### Runner Requirements Self-hosted runner required. Pre-installed: Rust + clippy + rustfmt, cargo-nextest, Godot 4.6 headless, Python 3 + jsonschema + pyyaml. ### Wiring Effort ~1 day. Makefile targets already exist. Workflow file is the only new artifact. Runner setup ~0.5 day. ### Complete Workflow File `.gitea/workflows/ci.yaml` (~100 lines) provided in `justine-round3.md`. Ready to deploy. --- ## Appendix A: Risk Register | Risk | Likelihood | Impact | Mitigation | |------|-----------|--------|------------| | Test client takes >3 days | Medium | Delays Layer 3 | MVP scope intentionally minimal | | Proof room too simple for golden files | Low | Low value until rooms exist | Proves toolchain; value Sprint 9 | | Content validation finds many errors | Medium | Sprint 8 effort bloat | Run as WARNING first sprint | | Client test headless blocks CI | Low | Can't automate client tests | Manual `make pre-pr-client` interim | | Fixture BLOCKER frustrates devs | Low | Devs skip pre-pr | Education; false positives worse | ## Appendix B: Source Files | File | Agent | Lines | Content | |------|-------|-------|---------| | `ozzie-round3.md` | Ozzie | 521 | Tester walkthrough, UX spec, WRONG button, anti-tedium priority | | `justine-round3.md` | Justine | 997 | pre-pr, perf-baseline, golden files, CI pipeline, fixture staleness | | `stig-round3.md` | Stig | 638 | 38 client tests, anti-tedium UI, checklist YAML, fog constants | | `dudley-round3.md` | Dudley | 1039 | Determinism fixes, --test-mode, Gauntlet loader, Q7/Q8 answers | | `hoshe-round3.md` | Hoshe | 677 | 59-item backlog, content validation, Layer 3 test, boundary values | | `tyre-round3.md` | Tyre | 646 | Sprint 8 plan, test client spec, Sprint 9+ roadmap, risk register | | `gestalt-round3.md` | Gestalt | 765 | Map layout, 48 entities, cross-room transitions, constants module | ## Appendix C: Boundary Value Matrix (41 values) ### Positive Boundaries (25 values) | Range | Values | GDScript Format | Rust Format | |-------|--------|----------------|-------------| | 0-127 | 0, 1, 126, 127 | pos fixint | pos fixint | | 128-255 | 128, 129, 254, 255 | uint 8 | uint 8 | | 256-32767 | 256, 257, 32766, 32767 | **int 16** | **uint 16** | | 32768-65535 | 32768, 32769, 65534, 65535 | uint 16 | uint 16 | | 65536-2^31-1 | 65536, 65537, 2147483646, 2147483647 | **int 32** | **uint 32** | | 2^31-2^32-1 | 2147483648, 4294967294, 4294967295 | uint 32 | uint 32 | | 2^32+ | 4294967296, 2^63-1 | int 64 | int 64 | **Bold rows** = encoding asymmetry between GDScript and Rust. Both are spec-valid. Both decoders accept both encodings (verified). ### Negative Boundaries (16 values) -1, -31, -32 (neg fixint); -33, -34, -127, -128 (int 8); -129, -130, -32767, -32768 (int 16); -32769, -2147483647, -2147483648 (int 32); -2147483649, -2^63 (int 64). ## Appendix D: Map-Agnostic Invariants (from Round 1) 36 invariants across 4 categories. These must hold for ANY valid map. - **Structural (INV-S01-S08):** Spawn reachable, NPC paths valid, no entity inside geometry, 2x2 minimum, door bidirectionality. - **Perception (INV-P01-P05):** Vision >0 at spawn, LOS symmetry, fog layer ordering, insert independence, sound coherence. - **Population (INV-C01-C08):** Minimum nearby NPC, StableId uniqueness, KG validity, D-033 colors, monologue reachability, dialogue pool non-empty. - **Simulation (INV-T01-T08):** Deterministic replay, tick budget (<=100ms), snapshot delivery, pause coherence, input ordering, SimRng order, cognitive delay monotonicity, knowledge decay timing. Top 10 for Sprint 8 (from Gestalt): INV-T04 (pause), INV-T01 (determinism), INV-T03 (snapshot delivery), INV-T05 (input ordering), INV-S01 (spawn reachable), INV-C03 (StableId uniqueness), INV-C07 (dialogue pool), INV-T02 (tick budget), INV-P02 (LOS symmetry), INV-S05 (no entity in geometry).