jpmschweitzer/settled-reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(workshops): complete QA test architecture workshop

Browse Source 
3-round workshop with 7 agents (Tyre, Dudley, Stig, Hoshe,
Justine, Gestalt, Ozzie) plus Qatux documenting. Produced:

- 59-item prioritized test backlog (60 tickets under epic #455)
- Gauntlet test world spec: 7 rooms + hub, 48 entities
- Test client binary spec (tooling/test-client/)
- Determinism fixes (3 patches, ~22 lines)
- Server --test-mode + --port 0 design
- Content cross-reference validation (9 checks)
- make pre-pr pipeline (6-step)
- 38 client tests prioritized
- Anti-tedium features (reset plate, hub teleport, WRONG button)
- Human tester walkthrough
- CI pipeline design (deferred but documented)

Sprint 8 scope: ~17.75 team-days across 26 tickets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2026-02-17 14:00:15 +01:00
co-authored by Claude Opus 4.6
parent 7043fd7e4f
commit a87c95a6eb
20 changed files with 12848 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/workshops/test-architecture/workshop-outcomes.md
+655
View File
@@ -0,0 +1,655 @@
# QA Strategy & Test Architecture Workshop — Outcomes
**Workshop:** QA Strategy & Test Architecture
**Rounds:** 3 (Analysis -> Synthesis -> Prioritization)
**Date:** 2026-02-17
**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie
**Documenter:** Qatux
This is the definitive output of the workshop. All specifications are build-ready. Round notes are at `docs/workshops/test-architecture/round-{1,2,3}-notes.md`.
---
## Table of Contents
1. [Decision Summary](#1-decision-summary)
2. [Sprint 8 Implementation Plan](#2-sprint-8-implementation-plan)
3. [Sprint 9+ Roadmap](#3-sprint-9-roadmap)
4. [Gauntlet Map Specification](#4-gauntlet-map-specification)
5. [Test Client Binary Specification](#5-test-client-binary-specification)
6. [Prioritized Test Backlog](#6-prioritized-test-backlog)
7. [Content Validation Specification](#7-content-validation-specification)
8. [Anti-Tedium Specifications](#8-anti-tedium-specifications)
9. [Human Tester Workflow](#9-human-tester-workflow)
10. [CI Pipeline Design](#10-ci-pipeline-design)
---
## 1. Decision Summary
### Workshop Decisions (confirmed by lead)
| ID | Decision | Round | Dissent |
|----|----------|-------|---------|
| WS-D1 | **Commit to determinism now.** 3 targeted fixes (A, B, D), ~22 lines. Fix C already done. | R1, confirmed R2 | None |
| WS-D2 | **Gauntlet as hybrid YAML + Rust inject.** YAML for geometry + entity placement, Rust for KG state injection. | R1, confirmed R2 | None |
| WS-D3 | **Rust test macros with named helpers. No custom DSL.** DSL revisitable Sprint 10+. | R1, confirmed R2 | None |
| WS-D4 | **Golden files as ObserverSnapshot at fixed positions.** JSON format, sorted keys, field-by-field diff. | R1, confirmed R2 | None |
| WS-D5 | **Separate test client binary** (lead override of R1 consensus on server-side text renderer). Binary at `tooling/test-client/` (lead override of R2 location at `server/src/bin/`). | R1 overruled R2, location overruled R3 | Tyre accepted both overrules constructively |
### Resolved Questions
| ID | Question | Answer |
|----|----------|--------|
| OQ-01 | Client depends on visible_tiles ordering? | No. Fix safe to ship. |
| OQ-02 | rmp_serde accepts int_16 for u64? | Yes. Traced through rmp-serde 1.3.1. |
| OQ-04 | WalkabilityMap HashMap -> BTreeMap? | No. Point-lookup only. |
| OQ-05 | Fixture staleness: git diff robust enough? | Yes. Deterministic generation verified. |
| OQ-07 | Server --test-mode and --port 0? | Designed and specified. |
| OQ-08 | Room name from coordinates? | `room_at()` in constants module. |
| OQ-10 | Fog byte value constants? | Yes. 6 named constants. |
| R2-OQ-01 | SetTickRate while paused? | Bug. Reject. 4-line fix. |
| R2-OQ-02 | Entity index recycling? | Safe (bevy generation counter). |
| R2-OQ-03 | content-ron in pre-pr? | Yes, in full pre-pr only. |
| R2-OQ-05 | blocked_entities feasibility? | Feasible. Sprint 9. ~300 tile lookups/tick. |
| R2-OQ-06 | Checklist overlay in Godot? | Lightweight overlay. Full tracking test-client-only. |
| R2-OQ-08 | Cross-room checklist location? | `content/gauntlet/cross_room_checks.yaml` |
| R2-OQ-09 | Room ordering for StableId? | Canonical order required. Append-only. |
| R2-OQ-10 | Room reset vs full restart? | Both. Different test types need different levels. |
### Fixture Staleness: BLOCKER
Tyre's argument (R2): stale fixtures mean client tests run against outdated protocol data — every passing client test is a false positive. Accepted by all agents. `make pre-pr` exits 1 on stale fixtures.
---
## 2. Sprint 8 Implementation Plan
*Sprint 8 ships infrastructure — the plumbing. Sprint 9 fills the pipes.* (Tyre)
### Items
| # | Item | Owner | Effort | Depends On |
|---|------|-------|--------|------------|
| S8-1 | Determinism fixes (A, B, D) | Dudley | 0.5d | -- |
| S8-2 | Content cross-reference validation (9 checks) | Justine | 1d | -- |
| S8-3 | `--test-mode` + `--port 0` server flags | Dudley | 0.5d | -- |
| S8-4 | Test client binary MVP | Dudley | 2-3d | S8-3 |
| S8-5 | `make pre-pr` chain | Justine | 0.5d | S8-1, S8-2 |
| S8-6 | Layer 3 test wiring | Dudley | 0.5d | S8-3, S8-4 |
| S8-7 | Pause guard tests (6 tests) | Dudley | 0.5d | -- |
| S8-8 | Determinism regression tests | Dudley | 0.5d | S8-1 |
| S8-9 | EntityRegistry lifecycle tests (3 tests) | Dudley | 0.25d | -- |
| S8-10 | Fixture staleness check | Justine | 0.25d | S8-5 |
### Critical Path
```
S8-3 (0.5d) -> S8-4 (2-3d) -> S8-6 (0.5d) = 3-4 days
```
### Effort Summary
| Track | Items | Effort |
|-------|-------|--------|
| Server (Dudley) | S8-1, S8-3, S8-4, S8-6, S8-7, S8-8, S8-9 | ~5-6d |
| Tooling (Justine) | S8-2, S8-5, S8-10 | ~1.75d |
| **Total** | **10 items** | **~7-8 team-days** |
### Not In Sprint 8
Gauntlet rooms, anti-tedium features, crossterm display, CI, perf baselines, encoding asymmetry tests, client tests, cross-room transitions.
### Determinism Fixes
| Fix | File(s) | Lines | What |
|-----|---------|-------|------|
| A | `query.rs`, `observer/mod.rs` | ~15 | `HashSet` -> `BTreeSet` for visible_positions/ids, sort visible_tiles by (x,y) |
| B | `observer/mod.rs` | 2 | Sort entities by entity_id in snapshot |
| C | (already done) | 0 | Monologue system ordering |
| D | `movement.rs` | ~5 | Sort movers by `Entity::to_bits()` for deterministic collision |
Each fix has a copy-pasteable regression test. Total: ~22 lines production code.
### Server `--test-mode`
- Prints `LISTENING:{port}` to stdout after bind, before accept
- All tracing to stderr (stdout clean for port discovery)
- `--port 0` for OS-assigned port (uses existing `accept_on(listener)`)
- `--seed 42` default in test-mode
- Exits after first client disconnect
- `setup_proof_room()` extracted for both modes; Gauntlet content loads when ready
### Bug Fix: SetTickRate While Paused
`SetTickRate(Half)` while paused now rejected (was unconditionally setting rate, bypassing pause guard). 4-line fix in `input.rs`. Test: `set_tick_rate_rejected_while_paused`.
---
## 3. Sprint 9+ Roadmap
### Tier 1: Sprint 9 (~10-12 team-days)
| # | Item | Owner | Effort |
|---|------|-------|--------|
| R-01 | Gauntlet rooms 1-4 | Content + Dudley | 3-4d |
| R-02 | Room reset trigger | Dudley | 1.5d |
| R-03 | Hub teleport | Dudley | 0.5d |
| R-04 | Client tests P0-P1 (10 tests) | Stig | 2-3d |
| R-05 | Determinism golden files (per-room) | Dudley | 1d |
| R-06 | Fog byte constants | Stig | 0.25d |
| R-07 | Encoding asymmetry tests (4-direction) | Hoshe + Stig | 1.5d |
### Tier 2: Sprint 10 (~15-17 team-days)
| # | Item | Owner | Effort |
|---|------|-------|--------|
| R-08 | WRONG button MVP | Stig + Dudley | 2d |
| R-09 | Gauntlet rooms 5-8 | Content + Dudley | 3-4d |
| R-10 | Room timer + personal bests | Ozzie spec, Stig impl | 1d |
| R-11 | Checklist auto-tracking | Stig + Dudley | 2d |
| R-12 | Performance baselines | Justine | 1.5d |
| R-13 | Client tests P2 (16 tests) | Stig | 3d |
| R-14 | Enhanced test client terminal | Dudley | 2d |
### Tier 3: Sprint 11+ (build when needed)
Gauntlet rooms 9-14, cross-room transitions (T1-T8), CI automation (~1d when greenlighted), content scaling stress, `blocked_entities`, client tests P3, WRONG button full capture, additional pause guard tests.
---
## 4. Gauntlet Map Specification
### Room List (7 rooms + Central Hub)
| # | Room | Origin | Size | Observer | Facing | Entities | Key Systems |
|---|------|--------|------|----------|--------|----------|-------------|
| 0 | Central Hub | (38,46) | 24x24 | (50,58) | -- | 4 signs | Connector, spawn point |
| 1 | Fog Theater | (28,2) | 44x32 | (56,18) | South | 4 | D-059 fog layers, D-015 vision cone, D-060 cognitive delay |
| 2 | Occlusion Corridor | (74,48) | 42x22 | (84,58) | East | 4 | D-035 LOS, D-017 perception modes, shadowcasting |
| 3 | Inventory Warehouse | (2,40) | 30x28 | (17,54) | East | 11 | D-065 9-slot inventory, pickup/drop, CarriedBy |
| 4 | Interaction Gallery | (2,82) | 24x20 | (14,92) | East | 5 | D-057 verb system, D-055 sprint suppression |
| 5 | Pause Chamber | (42,78) | 16x16 | (50,86) | North | 1 | D-031 pause, Bug #3 regression |
| 6 | Dialogue Room | (36,104) | 28x20 | (50,114) | North | 4 | D-041 KG, D-028 dialogue pools, D-033 colors, D-062 locked options |
| 7 | Crowd Plaza | (80,78) | 32x32 | (96,94) | West | 15 | Density stress, INV-T01 determinism, INV-T02 tick budget |
### Physical Layout
```
┌─────────────────────────────┐
│ FOG THEATER │
│ (44x32 tiles) │
└────────────┬────────────────┘
│ corridor-N
│
┌──────────────────┐ ┌─────────┴──────────┐ ┌──────────────────────────┐
│ INVENTORY │ │ │ │ OCCLUSION CORRIDOR │
│ WAREHOUSE ├──┤ CENTRAL HUB ├──┤ (42x22 tiles) │
│ (30x28 tiles) │ │ (24x24 tiles) │ │ │
└────────┬─────────┘ └────────┬───────────┘ └─────────────┬───────────┘
│ │ │
cross-cut-W corridor-S cross-cut-E
│ │ │
┌────────┴─────────┐ ┌───────┴───────────┐ ┌─────────────┴───────────┐
│ INTERACTION │ │ PAUSE CHAMBER │ │ CROWD PLAZA │
│ GALLERY │ │ (16x16 tiles) │ │ (32x32 tiles) │
│ (24x20 tiles) │ └───────┬───────────┘ │ │
└──────────────────┘ │ └────────────────────────┘
corridor-S2
│
┌──────┴──────────┐
│ DIALOGUE ROOM │
│ (28x20 tiles) │
└─────────────────┘
```
### Total Entity Count: 48
| Room | StableId Range | Count |
|------|---------------|-------|
| Hub (signs) | 1-4 | 4 |
| Fog Theater | 5-8 | 4 |
| Occlusion Corridor | 9-12 | 4 |
| Inventory Warehouse | 13-23 | 11 |
| Interaction Gallery | 24-28 | 5 |
| Pause Chamber | 29 | 1 |
| Dialogue Room | 30-33 | 4 |
| Crowd Plaza | 34-48 | 15 |
Player entity: StableId 0. Map bounds: 0-116 x 0-124 sim tiles.
**Additive-only constraint:** Existing rooms and entities NEVER reordered. New entries append. This preserves golden file stability and StableId assignments.
### Cross-Room Transitions (3 MVP)
| # | Name | Path | Systems Tested |
|---|------|------|---------------|
| T1 | Sprint Exit | Crowd Plaza -> cross-cut-E -> Occlusion | Sprint buffer clear + LOS recalculation |
| T3 | Full Inventory Interact | Inventory -> cross-cut-W -> Interaction | Inventory limit + verb computation |
| T6 | Pause Anywhere | Pause -> Hub -> any room | Pause persistence across teleport |
### Constants Module
`server/src/test_world/constants.rs`: `GauntletRoom` struct, `GauntletEntity` struct, 8 room constants, `ROOMS` array, `room_at()` lookup function. Full Rust code (~100 lines) in `gestalt-round3.md`.
### Deferred Rooms (Sprint 9+)
Zone Gate, Eavesdrop Alcove, Confrontation Stage, Sprint Gauntlet, Sound Lab, Decay Observatory, Shift Change — all depend on systems not yet implemented.
---
## 5. Test Client Binary Specification
### Location
`tooling/test-client/` — separate workspace crate. Imports bridge types from server crate (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed` — all already pub-exported).
### CLI (Sprint 8 MVP)
```
settled-reach-test-client [OPTIONS]
--connect <host:port> Server address (default: 127.0.0.1:9876)
--replay <file.jsonl> JSONL input file (one JSON array per tick)
--text Structured text to stdout (default)
--json JSON to stdout (for golden files)
--quiet No output (CI assertions only)
--golden <file.json> Compare final snapshot, exit 1 on diff
--ticks <N> Disconnect after N ticks
```
Exit codes: 0 = success, 1 = golden file mismatch, 2 = connection/protocol error.
### Connection Protocol
1. Server binds, prints `LISTENING:{port}\n` to stdout, blocks on accept
2. Test client connects via TCP
3. Per-tick: server sends `ObserverSnapshot` (length-prefixed MessagePack), client receives, formats output, sends `Vec<PlayerInput>` from replay file
4. After `--ticks N`: compare golden file if specified, drop connection, exit
Uses `rmp_serde::to_vec` (NOT `to_vec_named`) to match GDScript encoding.
### Text Output Format
```
=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full ===
Game time: Day 0, 04:12 (Morning)
Room: Occlusion Corridor
Entities (5):
npc:100 (18,10) Forward rel:Neutral vis:Visible d=3
npc:101 (20,10) Forward rel:Unknown vis:Remembered d=5
obj:200 (16,9) Forward rel:n/a vis:Visible d=1
Pending recognitions: 1 [npc:104 at (19,12) 3/8 ticks]
Tiles: 31 visible
Interactions (2):
npc:100 [Talk(1), ExamineNpc(2)] distance=3
Inventory: 2/9 [item:300(slot-0), item:301(slot-3)]
Monologue: "Something about this manifest doesn't add up."
===
```
Entity labels: `kind:entity_id`. Sorted by distance (nearest first), ties by entity_id. Sections with no data omitted. Room name from `room_at()` constants function.
### Text Renderer
`server/src/bridge/text_renderer.rs`: library function `format_snapshot_text(&ObserverSnapshot) -> String`. Pub-exported from server crate. ~100 lines Rust. Full implementation in `tyre-round3.md`.
### Golden File Comparison
JSON with sorted keys, pretty-printed. Recursive `diff_json_values()` comparison. Output on mismatch:
```
GOLDEN FILE MISMATCH: tick50.json
.entities[2].x: expected 18.0, got 19.0 (CHANGED)
.visible_tiles: 31 expected, 29 actual (2 REMOVED)
```
### Replay Format
JSONL (one JSON line per tick). Each line: JSON array of `PlayerInput`. Empty array `[]` = idle tick.
### Sprint 9+ CLI Additions
`--interactive`, `--live` (crossterm), `--log`, `--history-buffer`, `--checklist`.
### Layer 3 Integration Test
`server/tests/layer3.rs` (or `layer3_subprocess.rs`): spawns server + test client as subprocesses, verifies end-to-end TCP roundtrip. `ServerGuard` drop pattern for cleanup. Full code in `hoshe-round3.md` and `tyre-round3.md`.
### File Inventory
| File | Sprint | Description |
|------|--------|-------------|
| `tooling/test-client/src/main.rs` | 8 | Binary entry point |
| `server/src/bridge/text_renderer.rs` | 8 | `format_snapshot_text()` library |
| `server/src/test_world/constants.rs` | 8 | Room/entity constants |
| `server/tests/layer3.rs` | 8 | Subprocess integration test |
| `server/src/bridge/mod.rs` | 8 | Add `pub mod text_renderer;` |
---
## 6. Prioritized Test Backlog
### Sprint 8 P0 — 10 items (~7d)
| # | Title | Team | Effort | Bug Class |
|---|-------|------|--------|-----------|
| 1 | Determinism Fix A: BTreeSet + sort visible_tiles | server | 0.5d | Bug #1 (state divergence) |
| 2 | Determinism Fix B: Sort entities by entity_id | server | 0.25d | Golden file non-determinism |
| 3 | Determinism Fix D: Sort movers | server | 0.25d | Movement tie-breaking |
| 4 | Server `--test-mode` + `--port 0` | server | 1d | Unblocks all test infrastructure |
| 5 | `make pre-pr` target | joint | 0.5d | Developer discipline |
| 6 | Pause guard: movement_discarded_while_paused | server | 0.25d | Bug #3 |
| 7 | Pause guard: unpause_accepted_while_paused | server | 0.25d | Bug #3 class |
| 8 | Pause guard: roundtrip_with_movement | server | 0.25d | Bug #3 class |
| 9 | Content cross-reference validation (9 checks) | joint | 1.5d | Content scaling |
| 10 | Fixture staleness check | joint | 0.25d | Bug #4 class (protocol drift) |
### Sprint 8 P1 — 15 items (~10.75d)
| # | Title | Team | Effort |
|---|-------|------|--------|
| 11 | Determinism regression test: `gauntlet_deterministic_replay` | server | 1d |
| 12 | Per-fix determinism unit tests | server | 0.5d |
| 13 | Remaining pause guard tests (3 edge cases) | server | 0.5d |
| 14 | EntityRegistry lifecycle tests (3 tests) | server | 0.5d |
| 15 | Boundary value tests: GDScript encode-only (41 values) | client | 1d |
| 16 | Boundary value tests: Rust encode-only + roundtrip | server | 0.5d |
| 17 | gen_fixtures.rs boundary extension | joint | 0.5d |
| 18 | Encoding asymmetry tests (4 directions) | joint | 1d |
| 19 | `make fixtures-client` target | joint | 0.5d |
| 20 | Fog byte value constants | client | 0.25d |
| 21 | Client P0: monologue not lost, camera static during pause | client | 0.5d |
| 22 | Client P1: fog (4), entity lifecycle, recognition blob | client | 1d |
| 23 | `malformed_input_in_batch_rejects_entire_batch` | server | 0.25d |
| 24 | Test client binary scaffolding + CLI | server | 0.5d |
| 25 | Text renderer library | server | 0.5-1d |
### Sprint 9 P0 — 10 items (~7.5d)
Layer 3 test, test client replay/golden/comparison, golden file suite, Gauntlet first 3-4 rooms, constants module, content runtime validation, room reset, hub teleport.
### Sprint 9 P1 — 12 items (~12.5d)
Client tests P2 (16 tests), client P3 (12 tests), client anti-tedium tests, WRONG button MVP, room timer, checklist auto-tracking, Gauntlet rooms 5-8, performance baseline tooling, content scaling test, hub teleport UX, room reset UX, auto-checklist.
### Sprint 10+ — 12 items (~18.75d)
Gauntlet rooms 9-14, cross-room transitions, WRONG button full, map-agnostic invariants (36), fuzzy tests, CI pipeline, F3 debug overlay, zone gate, content scaling stress, blocked_entities, bidirectional relationship warnings.
### Bug Catalogue Coverage
| Bug | Covered By |
|-----|-----------|
| #1 (snapshot delivery) | `--test-mode` (#4), Layer 3 test (#26) |
| #2 (camera startup) | Client P0 (#21) |
| #3 (movement while paused) | Pause guard suite (#6-8, #13) |
| #4 (MessagePack -128) | Boundary value matrix (#15-18) |
| #5 (monologue overwrite) | Client P0 (#21) |
| #6 (snapshot spam) | Determinism fixes (#1-3), golden files (#29) |
### Client Tests: 38 Total
| Priority | Count | Sprint | Categories |
|----------|-------|--------|-----------|
| P0 | 2 | 8 | Monologue overwrite, camera pause |
| P1 | 7 | 8-9 | Fog (4), entity lifecycle (2), recognition blob |
| P2 | 24 | 9 | Camera (5), entity (5), UI (12), lerp (1), teleport (1) |
| P3 | 5 | 10+ | Z-layer (4), lerp target (1) |
---
## 7. Content Validation Specification
### Architecture
Extends `tooling/validate-content` (Python). Two-pass: schema validation (existing) then cross-reference validation (new). No Rust dependency.
### The 9 Checks
| # | Check | Severity | What |
|---|-------|----------|------|
| 1 | `canonical_id_uniqueness` | ERROR | Duplicate NPC canonical_ids |
| 2 | `relationship_target_resolution` | ERROR | Relationship targets resolve to defined NPCs |
| 3 | `location_slug_resolution` | ERROR | District locations match location files |
| 4 | `dialogue_location_resolution` | ERROR | Dialogue pool locations match district |
| 5 | `fact_id_resolution` | ERROR | fact_ids resolve to knowledge catalogs (absorbs `check-fact-ids`) |
| 6 | `triangle_membership_resolution` | ERROR | Triangle references match triangle files |
| 7 | `npc_count_accuracy` | WARNING | Declared npc_count matches actual file count |
| 8 | `dialogue_line_id_uniqueness` | ERROR | No duplicate line IDs within a dialogue file |
| 9 | `bidirectional_relationship_consistency` | WARNING | Asymmetric relationships flagged |
### Implementation
`ContentIndex` Python class: scans NPCs, locations, triangles, knowledge catalogs, districts, dialogue. `validate_references()` runs all 9 checks, returns (error_count, warning_count). Schema errors fail-fast before cross-reference pass.
### Phased Rollout
| Phase | Checks | Sprint |
|-------|--------|--------|
| 1 | #1, #2, #3, #6 (NPC/triangle/district) | Sprint 8 |
| 2 | #4, #5, #8 (dialogue/fact_ids) | Sprint 8 |
| 3 | #7, #9 (warnings) | Sprint 9 |
---
## 8. Anti-Tedium Specifications
### Priority Ranking
| Priority | Feature | Sprint | Effort | Justification |
|----------|---------|--------|--------|--------------|
| P1 | Room Reset Triggers | 9 | 1-1.5d | Can't re-test without server restart |
| P2 | WRONG Button MVP | 9 | 1d | Can't report bugs efficiently |
| P3 | Hub Teleport | 9 | 0.5d | Convenience for room navigation |
| P4 | Timer + Checklist | 9 | 1.5-2d | Engagement, not necessity |
*Note: Ozzie/Gestalt prioritize P1-P2 for Sprint 8. Tyre's Sprint 8 plan (authoritative for scope) defers all anti-tedium to Sprint 9.*
### Room Reset — Full Specification
| Aspect | Spec |
|--------|------|
| Trigger | Step on ResetPlate tile + press Interact (NOT automatic) |
| Server | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system |
| Resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles), inventory items sourced from room, dialogue state |
| Does NOT reset | Other rooms, player position, session timer, global SimRng state |
| Debounce | 10-tick cooldown |
| Test-mode only | `RoomResetTrigger` entities only added with `--test-mode` |
| Client visual | Amber `reset_plate` tile, "Reset Room" interaction verb, 0.15s amber flash + monologue "Systems recalibrated." |
### WRONG Button MVP (F12)
| Aspect | Spec |
|--------|------|
| Hotkey | F12 |
| Captures | Current ObserverSnapshot (JSON), text render output, tick/room/seed, tester description (one line) |
| Output | `tests/bug-reports/gauntlet-t{tick}-{timestamp}/` with report.md, snapshot.json, text_output.txt, description.txt |
| Server changes | Zero (MVP) |
| UX flow | F12 -> pause -> one-line prompt -> save files -> unpause -> resume |
**Full version (Sprint 9+):** 60-tick ring buffer, input history, replay seed, room metadata.
**Godot client (`bug_report.gd`):** ~80 lines autoload, F12 handler, ring buffer, modal prompt, screenshot + scene tree dump. Full code in `stig-round3.md`.
### Hub Teleport
| Aspect | Spec |
|--------|------|
| Hotkey | Home |
| Wire format | `PlayerAction::TeleportToHub` |
| Server | Move player to hub_spawn, clear dialogue/monologue/interaction buffer |
| Does NOT affect | Room state, inventory, game time, knowledge graph |
| Client | Instant camera snap + 0.3s fade-to-black-and-back |
| Gauntlet-only | Server rejects in non-Gauntlet maps |
### Room Timer + Personal Bests
Display `TIMER: 00:47 (PB: 00:38)`. Timer starts on room entry, resets on room reset. Stats persisted to `tests/gauntlet-stats.json`. Session summary on disconnect.
### Deferred: F3 Debug Overlay
Deferred indefinitely per Stig's recommendation. WRONG button captures same data on demand. F3 real-time overlay has measurable per-frame performance cost.
---
## 9. Human Tester Workflow
### Prerequisites
1. Build: `make build-server` + `make build-test-client`
2. Gauntlet content at `content/gauntlet/`
3. Checklist: `make checklist` -> `docs/qa/gauntlet-checklist.md`
### Session Start
**Terminal 1:** `make test-world-headless` (server with `--test-mode --port 0 --seed 42`)
**Terminal 2:** `make test-client` (connects, live terminal display)
### 6-Phase Workflow
1. **Navigate to a Room** — Walk from Hub. Room name, timer, and checklist update automatically.
2. **Execute the Checklist** — Follow printed checklist. Terminal shows entities with visibility symbols, fog counts, interactions.
3. **Encounter a Bug** — Press F12. Game pauses. Type one sentence. 4 files saved. Resume.
4. **Reset and Re-Test** — Walk to reset plate, interact. Room reverts to tick-0. Timer resets. Retry.
5. **Move to Another Room** — Press Home -> Hub. Walk to next room. Repeat.
6. **End Session** — Ctrl+C. Summary: rooms tested, coverage %, times, bug reports filed.
### Quick-Test Developer Workflow
**Target: 65 seconds.** Build (10s) -> start (3s) -> navigate (5s) -> test (45s) -> exit (2s).
### Entity Visibility Symbols
| Symbol | State | Meaning |
|--------|-------|---------|
| `●` | VISIBLE | In clear vision cone |
| `◐` | REMEMBERED | Previously seen, now in fog |
| `◌` | FOGGED | Detected but not recognized |
| `✕` | BLOCKED | LOS blocked by wall (debug, Sprint 9+) |
| `⚡` | RECOGNIZING | Mid-cognitive-delay |
### Display Sections (10 total)
1. Header (tick, rate, room, seed, timer)
2. Player (position, facing, stance, inventory count)
3. Entities (sorted by distance, visibility symbols)
4. Fog (5-layer tile counts)
5. Sound (Sprint 9+ — requires `Vec<SoundEvent>`)
6. Cognition (Sprint 9+ — requires enhanced `pending_recognitions`)
7. Interactions (available verbs per entity)
8. Monologue/Dialogue (exact text)
9. Inventory (slot map)
10. Status (checklist, timer, PB, hotkeys)
**Sprint 8 MVP:** Sections 1-4, 7-10. Sprint 9+ adds sections 5-6.
### Checklist YAML
Per-room at `content/gauntlet/rooms/{room_id}/checklist.yaml`. Cross-room at `content/gauntlet/cross_room_checks.yaml`. Each check has: id, description, type (auto/manual), step, condition (structured), if_wrong (debug guidance).
7 condition types: `player_near`, `player_facing`, `entity`+`expected`, `expected_sector`, `perception_mode`, `fog_visible_count_min`/`max`, `inventory_count`, `dialogue_active`, `monologue_contains`.
---
## 10. CI Pipeline Design
### Status: Deferred (documented, ready for when lead greenlights)
### 3-Tier Pipeline
| Tier | Trigger | Budget | Contents |
|------|---------|--------|----------|
| **Commit** | Every push | <2 min | lint-server, lint-client, validate-content, check-fact-ids |
| **PR** (merge gate) | PR opened/updated | <15 min | Commit tier + build, test, fixture staleness (BLOCKER) |
| **Nightly** | Scheduled 03:00 UTC | <30 min | PR tier + Layer 3, golden files, perf benchmarks, content scaling |
### Merge-Blocking Policy
| Job | Required? | Rationale |
|-----|-----------|-----------|
| `commit-checks` | **Yes** | Fast lint + content validation |
| `server-build-test` | **Yes** | Tests + fixture staleness = correctness |
| `client-build-test` | **Yes** | Client rendering contract |
| `nightly` | **No** | Deep tests are informational |
### `make pre-pr` — Interim CI
Until Gitea Actions are greenlighted, `make pre-pr` is the developer discipline tool:
```
pre-pr (~2.5 min incremental)
├── 1. lint (server + client) ~15s
├── 2. build (server + client) ~30-90s
├── 3. test (server + client) ~15-30s
├── 4. validate-content + fact-ids ~5s
└── 5. fixtures-check (BLOCKER) ~10-15s
```
Branch variants: `pre-pr-server`, `pre-pr-client`, `pre-pr-content`.
### Runner Requirements
Self-hosted runner required. Pre-installed: Rust + clippy + rustfmt, cargo-nextest, Godot 4.6 headless, Python 3 + jsonschema + pyyaml.
### Wiring Effort
~1 day. Makefile targets already exist. Workflow file is the only new artifact. Runner setup ~0.5 day.
### Complete Workflow File
`.gitea/workflows/ci.yaml` (~100 lines) provided in `justine-round3.md`. Ready to deploy.
---
## Appendix A: Risk Register
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Test client takes >3 days | Medium | Delays Layer 3 | MVP scope intentionally minimal |
| Proof room too simple for golden files | Low | Low value until rooms exist | Proves toolchain; value Sprint 9 |
| Content validation finds many errors | Medium | Sprint 8 effort bloat | Run as WARNING first sprint |
| Client test headless blocks CI | Low | Can't automate client tests | Manual `make pre-pr-client` interim |
| Fixture BLOCKER frustrates devs | Low | Devs skip pre-pr | Education; false positives worse |
## Appendix B: Source Files
| File | Agent | Lines | Content |
|------|-------|-------|---------|
| `ozzie-round3.md` | Ozzie | 521 | Tester walkthrough, UX spec, WRONG button, anti-tedium priority |
| `justine-round3.md` | Justine | 997 | pre-pr, perf-baseline, golden files, CI pipeline, fixture staleness |
| `stig-round3.md` | Stig | 638 | 38 client tests, anti-tedium UI, checklist YAML, fog constants |
| `dudley-round3.md` | Dudley | 1039 | Determinism fixes, --test-mode, Gauntlet loader, Q7/Q8 answers |
| `hoshe-round3.md` | Hoshe | 677 | 59-item backlog, content validation, Layer 3 test, boundary values |
| `tyre-round3.md` | Tyre | 646 | Sprint 8 plan, test client spec, Sprint 9+ roadmap, risk register |
| `gestalt-round3.md` | Gestalt | 765 | Map layout, 48 entities, cross-room transitions, constants module |
## Appendix C: Boundary Value Matrix (41 values)
### Positive Boundaries (25 values)
| Range | Values | GDScript Format | Rust Format |
|-------|--------|----------------|-------------|
| 0-127 | 0, 1, 126, 127 | pos fixint | pos fixint |
| 128-255 | 128, 129, 254, 255 | uint 8 | uint 8 |
| 256-32767 | 256, 257, 32766, 32767 | **int 16** | **uint 16** |
| 32768-65535 | 32768, 32769, 65534, 65535 | uint 16 | uint 16 |
| 65536-2^31-1 | 65536, 65537, 2147483646, 2147483647 | **int 32** | **uint 32** |
| 2^31-2^32-1 | 2147483648, 4294967294, 4294967295 | uint 32 | uint 32 |
| 2^32+ | 4294967296, 2^63-1 | int 64 | int 64 |
**Bold rows** = encoding asymmetry between GDScript and Rust. Both are spec-valid. Both decoders accept both encodings (verified).
### Negative Boundaries (16 values)
-1, -31, -32 (neg fixint); -33, -34, -127, -128 (int 8); -129, -130, -32767, -32768 (int 16); -32769, -2147483647, -2147483648 (int 32); -2147483649, -2^63 (int 64).
## Appendix D: Map-Agnostic Invariants (from Round 1)
36 invariants across 4 categories. These must hold for ANY valid map.
- **Structural (INV-S01-S08):** Spawn reachable, NPC paths valid, no entity inside geometry, 2x2 minimum, door bidirectionality.
- **Perception (INV-P01-P05):** Vision >0 at spawn, LOS symmetry, fog layer ordering, insert independence, sound coherence.
- **Population (INV-C01-C08):** Minimum nearby NPC, StableId uniqueness, KG validity, D-033 colors, monologue reachability, dialogue pool non-empty.
- **Simulation (INV-T01-T08):** Deterministic replay, tick budget (<=100ms), snapshot delivery, pause coherence, input ordering, SimRng order, cognitive delay monotonicity, knowledge decay timing.
Top 10 for Sprint 8 (from Gestalt): INV-T04 (pause), INV-T01 (determinism), INV-T03 (snapshot delivery), INV-T05 (input ordering), INV-S01 (spawn reachable), INV-C03 (StableId uniqueness), INV-C07 (dialogue pool), INV-T02 (tick budget), INV-P02 (LOS symmetry), INV-S05 (no entity in geometry).