3-round workshop with 7 agents (Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie) plus Qatux documenting. Produced: - 59-item prioritized test backlog (60 tickets under epic #455) - Gauntlet test world spec: 7 rooms + hub, 48 entities - Test client binary spec (tooling/test-client/) - Determinism fixes (3 patches, ~22 lines) - Server --test-mode + --port 0 design - Content cross-reference validation (9 checks) - make pre-pr pipeline (6-step) - 38 client tests prioritized - Anti-tedium features (reset plate, hub teleport, WRONG button) - Human tester walkthrough - CI pipeline design (deferred but documented) Sprint 8 scope: ~17.75 team-days across 26 tickets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
624 lines
35 KiB
Markdown
624 lines
35 KiB
Markdown
# QA Strategy & Test Architecture Workshop — Round 2 Notes
|
|
|
|
**Workshop:** QA Strategy & Test Architecture
|
|
**Round:** 2 (Synthesis & Cross-Review)
|
|
**Date:** 2026-02-17
|
|
**Documenter:** Qatux
|
|
**Participants:** Tyre, Dudley, Stig, Hoshe, Justine, Gestalt, Ozzie
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
Round 2 produced implementation-ready specifications across all 5 tracks. The lead's overrule on WS-D5 (separate test client binary instead of server-side text renderer) was accepted constructively by Tyre, who designed the binary architecture at `server/src/bin/test_client.rs`. Dudley delivered concrete code changes for all 4 determinism fixes (one was already done — a self-correction from Round 1). The cross-review process identified real coverage gaps: Hoshe found ZERO direct test coverage for the pause guard (all 6 of Dudley's proposed tests are genuine gaps), and Tyre added 6 tests to Stig's 32 for a total of 38 client assertions. The anti-tedium suite is fully specified across Ozzie, Stig, Gestalt, and Dudley. Content cross-reference validation is specified with 8 concrete checks.
|
|
|
|
**Key convergence:** Hoshe and Justine independently designed compatible `make pre-pr` targets. Ozzie and Stig independently designed compatible checklist formats (with Ozzie identifying 4 additions). Tyre validates Hoshe's CI tiers with refinements. All cross-reviews confirmed prior proposals as feasible.
|
|
|
|
**One disagreement:** fixture staleness should be BLOCKER (Tyre) or WARNING (Justine). Tyre's argument is stronger — stale fixtures mean client tests run against outdated protocol data, making all passing results false positives.
|
|
|
|
---
|
|
|
|
## Lead Decisions — Agent Responses
|
|
|
|
The lead confirmed/overruled the 5 workshop decisions from Round 1. Here is how each was received and incorporated.
|
|
|
|
| R1 Decision | Lead Ruling | Agent Response |
|
|
|-------------|-------------|----------------|
|
|
| **WS-D1:** Commit to determinism | **ACCEPTED** | Dudley produced exact code changes for 4 fixes (see Section 2). Self-corrected Fix C (monologue ordering was already done). |
|
|
| **WS-D2:** Hybrid YAML + Rust inject | **ACCEPTED** | Dudley designed `GauntletRoom` constants module + room reset mechanism. Gestalt mapped 14 rooms to the layout. |
|
|
| **WS-D3:** Rust test macros | **ACCEPTED** | No further discussion needed. Assertion helpers referenced throughout all proposals. |
|
|
| **WS-D4:** ObserverSnapshot golden files | **ACCEPTED** | Justine designed JSON golden file format with field-by-field diff. Dudley confirmed multi-position golden files (one per room). |
|
|
| **WS-D5:** Text renderer | **OVERRULED** — separate test client binary | Tyre accepted: "The architectural purity argument wins." Designed `server/src/bin/test_client.rs` with full CLI, text format, Layer 3 integration. See Section 1. |
|
|
|
|
**Additional lead decisions incorporated:**
|
|
- No Gitea Actions for now — manual `make ci` stays. All agents adapted: Hoshe and Justine designed `make pre-pr` as the developer-facing tool. Tyre validated the CI tier design as "future-ready for when the lead greenlights CI."
|
|
- Anti-tedium full suite approved. Ozzie, Stig, Gestalt, and Dudley each contributed implementation specs.
|
|
- `--test-mode` + `--port 0` confirmed for Sprint 8. Dudley designed the server flag changes (Section 3).
|
|
- Cross-reference validation added to `make validate-content`. Hoshe specified 8 checks; Justine specified 7 checks (compatible sets).
|
|
|
|
---
|
|
|
|
## 1. Test Client Binary Architecture (Tyre)
|
|
|
|
The lead overruled the Round 1 consensus (WS-D5: server-side text renderer) in favor of a separate Rust binary. Tyre designed the architecture.
|
|
|
|
### Location
|
|
|
|
**`server/src/bin/test_client.rs`** — a second binary target in the server crate.
|
|
|
|
```toml
|
|
# server/Cargo.toml
|
|
[[bin]]
|
|
name = "settled-reach-test-client"
|
|
path = "src/bin/test_client.rs"
|
|
```
|
|
|
|
**Rationale:** Shares bridge types (`ObserverSnapshot`, `PlayerInput`, `read_framed`, `write_framed`) with zero duplication. Rust compiles each binary independently — no server bloat. A separate crate would need to depend on the server crate anyway (or require premature extraction of a `protocol` crate).
|
|
|
|
### CLI Interface
|
|
|
|
```
|
|
settled-reach-test-client [OPTIONS]
|
|
|
|
Connection:
|
|
--connect <host:port> Connect to running server (default: 127.0.0.1:9876)
|
|
|
|
Input:
|
|
--replay <file.jsonl> Send inputs from file (one JSON PlayerInput per line)
|
|
--interactive Read inputs from stdin (future)
|
|
|
|
Output:
|
|
--text Render each snapshot as structured text to stdout
|
|
--json Dump each snapshot as JSON (for golden files)
|
|
--quiet No output (assertions only, for CI)
|
|
|
|
Assertions:
|
|
--golden <file.json> Compare final snapshot against golden file, exit 1 on diff
|
|
--ticks <N> Disconnect after N ticks (default: unlimited)
|
|
```
|
|
|
|
### Text Output Format
|
|
|
|
```
|
|
=== Tick 42 | Player (15,10) facing East | Stance: Walk | TickRate: Full ===
|
|
Game time: Day 0, 04:12 (Morning)
|
|
Entities (5):
|
|
npc:100 (18,10) Forward rel:Neutral vis:Visible
|
|
npc:101 (20,10) Forward rel:Unknown vis:Remembered
|
|
obj:200 (16,9) Forward rel:n/a vis:Visible
|
|
Tiles: 31 visible
|
|
Interactions (2):
|
|
npc:100 [Talk(1), ExamineNpc(2)] distance=3
|
|
Inventory: 2/9 [item:300(slot-0), item:301(slot-3)]
|
|
Monologue: "Something about this manifest doesn't add up."
|
|
===
|
|
```
|
|
|
|
Entities labeled as `kind:entity_id` (e.g., `npc:100`). No display names on wire — test assertions use Gauntlet constants module: `assert_entity_visible(&snapshot, GUARD_1.wire_id, GUARD_1.position)`.
|
|
|
|
### Text Renderer Lives in Library
|
|
|
|
`server/src/bridge/text_renderer.rs` — library code callable by the test client binary AND integration tests. The server binary never references it.
|
|
|
|
### Ozzie's Enhanced Terminal Layout
|
|
|
|
Ozzie proposed a richer live-updating terminal display for human testers (crossterm-based, ANSI escape codes), adding:
|
|
- Sound events section (D-018 three-range model)
|
|
- Cognition section (cognitive delay progress)
|
|
- Checklist progress bar
|
|
- Hotkey reminders (F12: WRONG, Home: Hub, R: Reset)
|
|
- Session timer
|
|
- Symbols per visibility state: `●` VISIBLE, `◐` REMEMBERED, `◌` FOGGED, `✕` BLOCKED, `⚡` RECOGNIZING
|
|
|
|
### Missing ObserverSnapshot Fields (Ozzie)
|
|
|
|
4 fields needed for the full test client experience, gated behind `--debug`/test-mode:
|
|
1. `blocked_entities: Vec<BlockedEntity>` — entities not visible with blocking wall position
|
|
2. Entity display names (or resolve via Gauntlet constants — Tyre recommends the latter)
|
|
3. Fog layer counts per type (5-layer breakdown)
|
|
4. `Vec<SoundEvent>` — source position, type, range classification
|
|
|
|
### Effort Estimate
|
|
|
|
~2-3 days total: binary scaffolding (0.5d), text renderer (0.5-1d), replay loading (0.5d), golden file comparison (0.5d), Layer 3 wiring (0.5d).
|
|
|
|
---
|
|
|
|
## 2. Determinism Fixes — Concrete Code Changes (Dudley)
|
|
|
|
Dudley produced exact code changes for all 4 fixes identified in Round 1. Total: ~40 lines changed across 4 files (revised up from Round 1's estimate of ~15 lines due to full context being included).
|
|
|
|
### Fix A: BTreeSet for visible_ids + sort visible_tiles
|
|
|
|
**Files:** `server/src/perception/query.rs`, `server/src/perception/observer/mod.rs`
|
|
|
|
Changes:
|
|
- `visible_positions: HashSet<(i32, i32)>` → `BTreeSet<(i32, i32)>` in `VisibilityGeometry`
|
|
- `visible_ids: HashSet<u64>` → `BTreeSet<u64>` in `filter_visible_entities`
|
|
- Add `visible_tiles.sort_by_key(|t| (t.x, t.y))` after collection in `NaturalVision::compute_geometry()`
|
|
- Update `collect_remembered_entities` signature to accept `&BTreeSet` references
|
|
|
|
`sector_lookup` HashMap stays — it's point-lookup only via `.get()`, never iterated.
|
|
|
|
### Fix B: Sort visible entities in snapshot
|
|
|
|
**File:** `server/src/perception/observer/mod.rs`, in `compute_observer_snapshot()`
|
|
|
|
Add after `collect_remembered_entities`:
|
|
```rust
|
|
entities.sort_by_key(|e| e.entity_id);
|
|
```
|
|
|
|
### Fix C: Pin monologue system ordering — ALREADY DONE
|
|
|
|
**Round 1 self-correction:** Dudley stated monologue systems lacked explicit ordering. After re-reading `bridge/mod.rs:168-175`, confirmed the ordering is already explicit via `.after()` constraints. **No change needed.**
|
|
|
|
### Fix D: Sort movers in validate_movement
|
|
|
|
**File:** `server/src/simulation/movement.rs`
|
|
|
|
Collect movers into Vec, sort by `entity.to_bits()`, then process:
|
|
```rust
|
|
let mut mover_list: Vec<_> = movers.iter_mut().collect();
|
|
mover_list.sort_by_key(|(entity, _, _, _)| entity.to_bits());
|
|
```
|
|
|
|
Note: `entity.to_bits()` provides stable ordering within a single run. For cross-session determinism (save/load), would need `registry.to_stable(entity)` — deferred to Sprint 10+.
|
|
|
|
### Determinism Fix Coverage (Hoshe cross-review)
|
|
|
|
Hoshe assessed that **none of the 4 fixes have direct test coverage today**. Each fix should ship with its own regression test:
|
|
- Fix A: Test observer with 2 equidistant contradicted NPCs, assert same one selected
|
|
- Fix A (tiles): Generate snapshot with random-order tiles, assert output sorted
|
|
- Fix B: Not identified separately — covered by golden file tests
|
|
- Fix D: Two entities at same distance to same tile, assert deterministic winner
|
|
|
|
---
|
|
|
|
## 3. Server `--test-mode` and `--port 0` Design (Dudley)
|
|
|
|
### `--test-mode` Flag
|
|
|
|
| Aspect | Behavior |
|
|
|--------|----------|
|
|
| Content | Loads Gauntlet content pack (`content/gauntlet/`), falls back to proof room |
|
|
| Seed | Fixed seed 42 |
|
|
| Stdout | Prints `LISTENING:{port}` after TCP bind, before accept |
|
|
| Shutdown | Exits after first client disconnect |
|
|
| Logging | Defaults to `warn` (override with `RUST_LOG`) |
|
|
|
|
### `--port 0` Support
|
|
|
|
Uses existing `TcpBridge::accept_on(listener)` (already in `tcp.rs:67`). The bind/accept split was added for test race condition prevention — exactly what `--port 0` needs. No TcpBridge changes required.
|
|
|
|
Port discovery protocol: server prints `LISTENING:{port}` to stdout, test client parses it.
|
|
|
|
---
|
|
|
|
## 4. Cross-Review Results
|
|
|
|
### 4.1 Hoshe validates Dudley's server proposals
|
|
|
|
**Pause guard:** All 6 of Dudley's proposed tests are genuine gaps. The pause guard has **ZERO direct test coverage today**. The existing test (`process_input_pause_sets_paused`) tests the Pause action side effect, not the guard itself. If someone deleted lines 102-105 (the guard), all existing tests would still pass.
|
|
|
|
Hoshe found 3 additional gaps:
|
|
- `set_tick_rate_while_paused` (P2) — SetTickRate(Half) while paused unconditionally sets rate, making `paused()` return false
|
|
- `perception_mode_while_paused` (P2) — currently no-op but should pass through guard
|
|
- `interact_take_while_paused` (P2) — documents expected behavior for Take during pause
|
|
|
|
**EntityRegistry:** 3 of 5 Dudley tests are genuine gaps. 2 are not applicable (concurrent access impossible in bevy, bulk performance is fine at 2000 entries). Hoshe found 2 additional gaps:
|
|
- `unregister_unknown_entity_is_noop` (P1) — code handles it but no test
|
|
- `register_with_pre_existing_stable_id_component` (P2) — component/registry divergence
|
|
|
|
**Bridge deserialization:** Hoshe recommends keeping batch-failure (current behavior) with a test documenting it: `malformed_input_in_batch_rejects_entire_batch`.
|
|
|
|
### 4.2 Tyre validates Stig's client proposals
|
|
|
|
All 32 tests **approved**. No redundancies. Tyre adds 6 more (38 total):
|
|
|
|
| # | Test | Category | Rationale |
|
|
|---|------|----------|-----------|
|
|
| 33 | Pending recognition blob rendering | Entity rendering | D-060 cognitive delay visual |
|
|
| 34 | Recognition transition animation | Entity lerp | Blob → full entity over ~0.3s |
|
|
| 35 | Tick rate HUD indicator | UI elements | Full/Half/Paused display |
|
|
| 36 | Inventory full visual state | UI elements | 9/9 feedback |
|
|
| 37 | Sprint interaction suppression | UI elements | D-055 interaction buffer empty during sprint |
|
|
| 38 | Entity modulate for Remembered | Entity rendering | Remembered vs Visible visual difference |
|
|
|
|
**Priority ranking:** P0 = monologue not lost on overwrite (Bug #5), camera static during pause (Bug #2). P1 = fog shader state (3 tests), entity lifecycle, pending recognition blob.
|
|
|
|
**Refinements:**
|
|
- Camera rapid snapshots test should verify convergence, not just "doesn't crash"
|
|
- Entity lifecycle test should verify node freed (not just hidden) — memory leak prevention
|
|
|
|
### 4.3 Tyre validates Hoshe's CI tiers
|
|
|
|
**Approved** with refinements:
|
|
- PR tier budget revised: <15 min (not <10 min) — clean cache server builds take 5-7 min
|
|
- Add content cross-reference validation to PR tier (<2s, negligible budget impact)
|
|
- Add content scaling stress test to Nightly tier
|
|
- **Fixture staleness should be BLOCKER, not WARNING** — stale fixtures make all client tests false positives
|
|
|
|
### 4.4 Dudley validates Hoshe's Layer 3 test
|
|
|
|
**Feasible** with 2 minor adjustments:
|
|
1. Use `rmp_serde::to_vec` (not `to_vec_named`) for input serialization — matches GDScript's encoding
|
|
2. Tolerate initial tick=0 snapshot before input is processed — server may send snapshot before reading client input
|
|
|
|
Hoshe's assertions on tick=0 snapshot are fine — they verify snapshot delivery, not input processing.
|
|
|
|
---
|
|
|
|
## 5. Resolved Open Questions
|
|
|
|
| ID | Question | Answer | Answered By |
|
|
|----|----------|--------|-------------|
|
|
| **OQ-01** | Does client depend on visible_tiles ordering? | **No.** All consumers are position-indexed (Dict keyed by Vector2i, set_cell() idempotent). Fix #2 safe to ship. Two fixture tests reference `visible_tiles[0]` by index — flag for fixture regeneration. | Stig |
|
|
| **OQ-02** | Does rmp_serde accept int_16 for u64? | **Yes.** Traced through rmp-serde 1.3.1: `Marker::I16 → visit_i16 → visit_i64 → u64::try_from`. GDScript encoding 256 as int_16 deserializes correctly. Negative values correctly rejected. | Dudley |
|
|
| **OQ-04** | WalkabilityMap HashMap → BTreeMap? | **No. Leave as HashMap.** Point-lookup only, never iterated. HashMap is faster in the hot path. Document as known-safe usage. | Tyre |
|
|
| **OQ-05** | Fixture staleness: git diff robust enough? | **Yes.** `git diff --exit-code` detects any byte-level change. Content-addressed hashing adds complexity for zero additional safety. Fixture generation is deterministic (verified — no timestamps/random values). | Justine |
|
|
| **OQ-06** | = OQ-02 (rmp_serde int_16→u64) | See OQ-02 | Dudley |
|
|
| **OQ-07** | Server --test-mode and --port 0? | **Designed.** See Section 3. Uses existing `accept_on()`. `LISTENING:{port}` signal to stdout. | Dudley |
|
|
| **OQ-08** | Room name from coordinates? | **Gauntlet coordinate bounds constants** in `server/src/test_world/constants.rs`. `room_at(player_pos)` returns room name. No server API needed. | Dudley |
|
|
| **OQ-10** | Fog byte value constants? | **Yes, promote to named constants.** `VIS_HIDDEN=0`, `VIS_PERIPHERAL=180`, `VIS_FORWARD=255`, `EXP_UNEXPLORED=0`, `EXP_EXPLORED=128`, `EXP_VISIBLE=255`. Replace magic numbers in `fog_state.gd`. | Stig |
|
|
| **OQ-12** | = OQ-04 (WalkabilityMap HashMap) | See OQ-04 | Tyre |
|
|
|
|
### Partially Resolved
|
|
|
|
| ID | Question | Status |
|
|
|----|----------|--------|
|
|
| **OQ-03** | Gitea Actions available? | Moot — lead confirmed no CI for now. Tyre notes: ~1 day effort when greenlighted. |
|
|
| **OQ-05** | Entity display_name on wire? | **Deferred.** Tyre: use `kind:entity_id` labels now. Add `display_name: Option<String>` when client needs name labels (Sprint 9-10 gameplay feature). |
|
|
| **OQ-09** | Text renderer from ObserverSnapshot? | **Yes**, confirmed. The text renderer formats ObserverSnapshot. Lives in server library, called by test client binary. |
|
|
| **OQ-11** | Client tests headless? | **Unresolved.** No agent tested this in Round 2. Remains a concern for CI. |
|
|
|
|
### Resolved Unresolved Architectural Questions (from Round 1)
|
|
|
|
| ID | Question | Resolution |
|
|
|----|----------|-----------|
|
|
| **UQ-01** | Bridge deserialization: skip-and-log vs batch failure? | **Keep batch failure.** Both sides are co-versioned (D-020). Add test documenting behavior: `malformed_input_in_batch_rejects_entire_batch`. (Hoshe) |
|
|
| **UQ-02** | Gauntlet geometry: YAML vs Rust? | **Not directly addressed in Round 2.** Dudley's code uses Rust constants for room bounds. Gestalt references YAML room definitions for checklists. The hybrid approach (WS-D2) remains — specific geometry format deferred to implementation. |
|
|
| **UQ-03** | Wall-blocking-LOS info? | **Optional `blocked_entities` field** gated behind `--debug`/test-mode. Ozzie specifies as a requirement for the test client's `✕ BLOCKED` display. Feasibility question to Dudley in Round 3. |
|
|
| **UQ-04** | Fixture path fragility? | **Addressed by `make pre-pr` fixture staleness check.** Regenerate + diff catches stale fixtures regardless of path. |
|
|
|
|
---
|
|
|
|
## 6. Content Cross-Reference Validation
|
|
|
|
Both Hoshe and Justine independently specified content validation extensions. Their proposals are compatible and complementary.
|
|
|
|
### Hoshe's 8 Checks
|
|
|
|
| Check | Severity | What | Target |
|
|
|-------|----------|------|--------|
|
|
| 1 | ERROR | `canonical_id` uniqueness across all NPCs | `npcs/*.yaml` |
|
|
| 2 | ERROR | Relationship `target` resolves to defined NPC | `npcs/*.yaml → relationships[].target` |
|
|
| 3 | ERROR | District `locations[]` slug matches location file | `district.yaml → locations/` |
|
|
| 4 | ERROR | Dialogue pool `location` matches district | `dialogue/**/*.yaml → location` |
|
|
| 5 | ERROR | `knowledge_grant.fact_id` validity | `dialogue/**/*.yaml → lines[]` |
|
|
| 6 | ERROR | NPC `triangle_membership` matches triangle file | `npcs/*.yaml` |
|
|
| 7 | WARNING | `npc_count` matches actual NPC file count | `district.yaml` |
|
|
| 8 | ERROR | Dialogue line ID uniqueness within pool | `dialogue/**/*.yaml → lines[].id` |
|
|
|
|
### Justine's 7 Checks
|
|
|
|
| Check | Severity | What |
|
|
|-------|----------|------|
|
|
| 1 | ERROR | NPC relationship targets resolve |
|
|
| 2 | ERROR | Triangle members resolve |
|
|
| 3 | ERROR | NPC triangle_membership matches triangles |
|
|
| 4 | ERROR | Dialogue pool location resolves |
|
|
| 5 | ERROR | Fact IDs resolve (absorb `check-fact-ids`) |
|
|
| 6 | ERROR | District location list matches files |
|
|
| 7 | WARNING | Bidirectional relationship consistency |
|
|
|
|
### Merged View
|
|
|
|
Hoshe and Justine agree on checks 1-6. Justine adds bidirectional relationship warnings (Check 7). Hoshe adds dialogue line ID uniqueness (Check 8) and `npc_count` accuracy (Check 7). Combined: **9 unique checks** (7 ERROR, 2 WARNING).
|
|
|
|
### Architecture Agreement
|
|
|
|
Both recommend extending `tooling/validate-content` (Python) with a second pass after schema validation. No Rust dependency — content authors validate without compiling the server. Phased rollout: NPC/triangle/district first, dialogue/monologue second, warnings third.
|
|
|
|
---
|
|
|
|
## 7. `make pre-pr` Target
|
|
|
|
Both Hoshe and Justine independently designed this target. Their proposals are compatible.
|
|
|
|
### Agreed Chain
|
|
|
|
```
|
|
pre-pr
|
|
├── 1. lint-server + lint-client (~15s)
|
|
├── 2. build-server + build-client (~30-90s)
|
|
├── 3. test-server + test-client (~15-30s)
|
|
├── 4. validate-content (~2-5s)
|
|
├── 5. check-fact-ids (~2s)
|
|
└── 6. fixture staleness check (~10-15s)
|
|
```
|
|
|
|
**Total: ~90-180s** (under 3 minutes for clean incremental build). Fast enough for every PR.
|
|
|
|
### Branch-Specific Variants (Hoshe)
|
|
|
|
- `make pre-pr-server` — lint-server, build-server, test-server, fixtures
|
|
- `make pre-pr-client` — lint-client, build-client, test-client
|
|
- `make pre-pr-content` — validate-content, check-fact-ids
|
|
|
|
### Fixture Staleness Check
|
|
|
|
```makefile
|
|
fixtures-check: fixtures
|
|
@if git diff --quiet client/tests/fixtures/; then \
|
|
echo "Fixtures: up to date"; \
|
|
else \
|
|
echo "FIXTURES STALE"; exit 1; \
|
|
fi
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Anti-Tedium Suite — Full Specifications
|
|
|
|
All features approved by lead. Four agents contributed implementation specs.
|
|
|
|
### 8.1 Room Reset Trigger
|
|
|
|
| Aspect | Spec | Source |
|
|
|--------|------|--------|
|
|
| Trigger | Player steps on ResetPlate tile + presses Interact (not automatic) | Gestalt, Ozzie agree |
|
|
| Server mechanism | `RoomResetTrigger` component, `RoomSnapshots` resource (tick-0 state per room), `execute_room_reset` system | Dudley |
|
|
| What resets | Entity positions, entity KG, player KG (room refs only), fog (room tiles only), inventory items from room, dialogue state | Gestalt |
|
|
| What does NOT reset | Other rooms, player position (stays on plate), session timer, other-room checklist progress | Ozzie, Gestalt |
|
|
| Edge case | Items carried from room returned to tick-0 position, terminal shows "Items returned: keycard → crate_1" | Ozzie |
|
|
| Client visual | Distinct tile type (`reset_plate`), amber outline, interaction verb "Reset Room", 0.15s amber flash + monologue "Systems recalibrated." | Stig |
|
|
| Debounce | 10-tick cooldown prevents re-trigger while walking across plate | Dudley |
|
|
| Test mode only | `RoomResetTrigger` entities only added with `--test-mode` | Dudley |
|
|
|
|
### 8.2 Hub Teleport
|
|
|
|
| Aspect | Spec | Source |
|
|
|--------|------|--------|
|
|
| Hotkey | `Home` key | Ozzie, Stig agree |
|
|
| Wire format | `PlayerAction::TeleportToHub` | Gestalt |
|
|
| Server behavior | Move player entity to `GAUNTLET.hub_center`, clear dialogue/monologue/interaction buffer | Dudley, Ozzie |
|
|
| Does NOT affect | Room state, inventory, game time, knowledge graph | Ozzie, Gestalt |
|
|
| Client visual | Instant camera snap, 0.3s fade-to-black-and-back, no monologue (meta action) | Stig |
|
|
| Gauntlet-only | Server rejects `TeleportToHub` in non-Gauntlet maps | Gestalt |
|
|
|
|
### 8.3 WRONG Button (F12)
|
|
|
|
| Aspect | Spec | Source |
|
|
|--------|------|--------|
|
|
| MVP captures (Sprint 8) | ObserverSnapshot, tick + position, text render output, human description | Gestalt |
|
|
| Full captures (Sprint 9+) | + input history (60 ticks), snapshot history (60 ticks), world digest, replay seed | Ozzie |
|
|
| Output directory | `tests/bug-reports/gauntlet-{tick}-{timestamp}/` | Gestalt, Ozzie |
|
|
| Bug report format | `report.md` (human-readable), `snapshot.json`, `text_output.txt`, `description.txt` | Ozzie |
|
|
| Client implementation | `BugReportCapture` autoload, F12 hotkey in `_unhandled_input()`, modal prompt, 6 data captures | Stig |
|
|
| Ring buffer | 60 ticks (configurable via `--history-buffer`) | Ozzie |
|
|
|
|
### 8.4 Room Timer + Personal Bests
|
|
|
|
| Aspect | Spec | Source |
|
|
|--------|------|--------|
|
|
| Display | `TIMER: 00:47 (PB: 00:38)` in status bar/overlay | Ozzie |
|
|
| Start | Player enters room (crosses bounding box) | Ozzie |
|
|
| Reset | Room reset trigger resets timer | Ozzie |
|
|
| Persistence | `tests/gauntlet-stats.json` — local, not committed | Ozzie |
|
|
| Session summary | Printed on disconnect: rooms visited, coverage %, times, PBs, bug reports filed | Ozzie |
|
|
|
|
### 8.5 Auto-Checklist Progress (Ozzie + Stig merged)
|
|
|
|
Ozzie identified 4 gaps in Stig's per-room checklist proposal and proposed a merged format:
|
|
|
|
| Gap | Stig's Proposal | Ozzie's Addition |
|
|
|-----|----------------|-----------------|
|
|
| No machine-readable conditions | Human prose only | Add `condition:` field (structured data) for auto-tracking |
|
|
| No auto vs manual distinction | All items equal | Add `type: auto/manual` — auto-confirms from snapshot, manual requires tester input |
|
|
| No cross-room items | Per-room only | Add `cross_room_checks.yaml` at Gauntlet root |
|
|
| No failure guidance | Just "check X" | Add `if_wrong:` field with likely causes + file references |
|
|
|
|
**Merged checklist YAML format** (Ozzie):
|
|
```yaml
|
|
checks:
|
|
- id: occ_01_hidden_not_visible
|
|
description: "NPC behind wall is NOT visible"
|
|
type: auto
|
|
condition:
|
|
player_near: [15, 10]
|
|
entity: hidden-1
|
|
expected: blocked
|
|
if_wrong: |
|
|
LOS leaking through wall. Check shadowcast.rs.
|
|
```
|
|
|
|
`make checklist` generates markdown from YAML (Stig's `tooling/gen_checklist.py`). Test client loads structured conditions for auto-tracking.
|
|
|
|
### 8.6 Stig's Client-Side Anti-Tedium UI
|
|
|
|
- **Room reset:** Tile type `reset_plate` in TileRenderer, interaction verb "Reset Room", amber flash
|
|
- **Hub teleport:** `TELEPORT_HUB` in InputMapper → `Home` key, fade transition
|
|
- **WRONG button:** New `bug_report.gd` autoload, 60-entry `input_history` ring buffer
|
|
- **Progress overlay:** Top-right panel with room name, run counter, timer, checklist progress bar. Only visible when `gauntlet_mode == true`.
|
|
|
|
### 8.7 Deferred: F3 Debug Overlay
|
|
|
|
**Stig recommends deferring** the F3 State Inspector Overlay. The WRONG button captures the same data on demand. F3 as a real-time overlay requires per-frame string formatting of the entire ObserverSnapshot — measurable performance cost. Ship WRONG button first, F3 if testers ask for it.
|
|
|
|
---
|
|
|
|
## 9. Gauntlet Room Coverage (Gestalt)
|
|
|
|
### Final Room List: 14 Rooms
|
|
|
|
| # | Room | Primary Systems | Source |
|
|
|---|------|----------------|--------|
|
|
| 1 | Inventory Warehouse | Pickup, CarriedBy, 9-slot limit | Brief |
|
|
| 2 | Occlusion Corridor | LOS, shadowcasting, perception modes | Brief |
|
|
| 3 | Interaction Gallery | ObjectType verbs, sprint suppression | Brief |
|
|
| 4 | Crowd Plaza | Entity density, relationship colors, cognitive delay | Brief |
|
|
| 5 | Fog Theater | Fog transitions, peripheral dimming, exploration persistence | Brief |
|
|
| 6 | Dialogue Room | Trust tiers, contradiction, walk-away, monologue during dialogue | Brief |
|
|
| 7 | Pause Chamber | TickRate toggle, state transitions | Brief |
|
|
| 8 | Zone Gate | Zone transition (reserved, future contract) | Brief |
|
|
| 9 | Eavesdrop Alcove | ListeningFocus, zone ambient, Careful stance | Gestalt R1 |
|
|
| 10 | Confrontation Stage | Cognitive vulnerability, audio dip | Gestalt R1 |
|
|
| 11 | Sprint Gauntlet | Sprint suppression, anomaly survival, stance transitions | Gestalt R1 |
|
|
| 12 | Sound Lab | Three-range sound, sound pings, recognition chime | Gestalt R1 |
|
|
| 13 | Decay Observatory | Knowledge decay, stale state, fog degradation | Gestalt R1 |
|
|
| 14 | Shift Change | Stress test: all systems at density | Gestalt R1 |
|
|
|
|
### Coverage Matrix
|
|
|
|
Gestalt mapped 50+ system-to-room relationships across 6 pillars: Characters & Information, Perception, Movement & Interaction, Audio, Simulation & Architecture, Content Systems. Every system from confirmed decisions has at least one room exercising it.
|
|
|
|
**Coverage gaps (3 minor, all addressable without new rooms):**
|
|
1. Object-layer favorite colors (D-052) — add to Inventory Warehouse in v0.1.2+
|
|
2. POI navigation (D-013) — add to Hub as POI markers. v0.1 stretch.
|
|
3. Environmental neutrality (D-045) — assertion on Dialogue Room (pre/post confrontation CanvasModulate identical)
|
|
|
|
### 8 Transition Scenarios
|
|
|
|
All scenarios reference physically connected rooms via hub paths or cross-cuts:
|
|
|
|
| # | Scenario | Path | Key Test |
|
|
|---|----------|------|----------|
|
|
| T1 | Sprint Exit | Plaza → Occlusion Corridor | Buffer cleared during sprint, LOS recalculated |
|
|
| T2 | Fog into Dialogue | Fog Theater → Dialogue Room | Fog state preserved during dialogue, monologue above dialogue box |
|
|
| T3 | Full Inventory Interact | Inventory → Interaction Gallery | 9/9 inventory, Take still offered server-side, client greys out |
|
|
| T4 | Sprint into Interaction | Sprint Gauntlet → Interaction Gallery | Verbs repopulate within 1 tick after stance change |
|
|
| T5 | Confrontation to Eavesdrop | Confrontation → Eavesdrop | Audio dip release + ListeningFocus activation don't conflict |
|
|
| T6 | Pause Anywhere | Pause Chamber → Hub → any room | Pause-during-transition state corruption |
|
|
| T7 | Sound across Fog | Sound Lab → Fog Theater | Sound propagation through walls, cognitive delay from sound |
|
|
| T8 | Walk-away Sprint | Dialogue → Hub → Sprint Gauntlet | KG incompleteness recorded, sprint suppresses post-dialogue monologue |
|
|
|
|
### Top 10 Invariants for Sprint 8 (Gestalt)
|
|
|
|
| Rank | ID | Invariant | Bug Match |
|
|
|------|-----|-----------|-----------|
|
|
| 1 | INV-T04 | Pause coherence | Bug #3 |
|
|
| 2 | INV-T01 | Deterministic replay | Bug #1 class |
|
|
| 3 | INV-T03 | Snapshot delivery | Bug #1, #5 |
|
|
| 4 | INV-T05 | Input ordering | Bug #1 |
|
|
| 5 | INV-S01 | Player spawn reachable | Softlock prevention |
|
|
| 6 | INV-C03 | StableId uniqueness | Corruption prevention |
|
|
| 7 | INV-C07 | Dialogue pool non-empty | Player-facing failure |
|
|
| 8 | INV-T02 | Tick budget | Bug #6 class |
|
|
| 9 | INV-P02 | LOS symmetry | D-035 mandate |
|
|
| 10 | INV-S05 | No entity inside geometry | Content scaling safety |
|
|
|
|
**Bug catalogue mapping: every Sprint 6-7 bug is now covered** by at least one invariant + room + test type.
|
|
|
|
---
|
|
|
|
## 10. Encoding Asymmetry Tests (Hoshe)
|
|
|
|
Hoshe specified 4-direction cross-language testing for the encoding asymmetry between GDScript (int_16 for 256-32767) and Rust (uint_16):
|
|
|
|
| Direction | What | Test Location | When |
|
|
|-----------|------|---------------|------|
|
|
| Rust → GDScript (fixture) | Snapshots with overlap-zone ticks | `test_msgpack_boundaries.gd` | Every PR |
|
|
| GDScript → Rust (fixture) | Inputs with overlap-zone ticks | `serialization.rs` | Every PR |
|
|
| Rust → GDScript (raw bytes) | Hand-crafted uint_16/uint_32 bytes | `test_msgpack_boundaries.gd` | Every commit |
|
|
| GDScript → Rust (raw bytes) | Hand-crafted int_16/int_32 bytes | `serialization.rs` | Every commit |
|
|
|
|
New `make fixtures-client` target generates GDScript-encoded fixtures for Rust to verify. Reverse direction of existing `make fixtures`.
|
|
|
|
---
|
|
|
|
## 11. Performance & Golden File Tooling (Justine)
|
|
|
|
### Performance Baseline
|
|
|
|
- `tests/perf/baseline.json` — committed, records median/min/max from 5 runs
|
|
- `tooling/perf-measure` — builds release, runs benchmarks, compares against baseline
|
|
- Thresholds: <15% = PASS, 15-30% = WARNING, >30% = FAIL
|
|
- Machine tag prevents meaningless cross-machine comparisons
|
|
- `make perf-baseline` (compare) and `make perf-baseline-update` (update)
|
|
|
|
### Golden File Diff
|
|
|
|
- **Rust test, not separate tool** — golden file is an ObserverSnapshot, Rust code knows the structure
|
|
- JSON format with sorted keys, pretty-printed (`serde_json::to_string_pretty`)
|
|
- Field-by-field diff output on failure: changed fields, POSITION markers, added/removed entities
|
|
- `make golden-diff` (view diff) and `make golden-update` (regenerate)
|
|
- Why JSON not MessagePack: human-readable in `git diff`, sorted keys = deterministic output
|
|
|
|
---
|
|
|
|
## 12. Client Test Suite — Final Count
|
|
|
|
| Category | Round 1 (Stig) | Round 2 Additions | Total |
|
|
|----------|---------------|-------------------|-------|
|
|
| Camera system | 7 | — | 7 |
|
|
| Entity rendering | 7 | +2 (Tyre: blob, remembered modulate) | 9 |
|
|
| Z-layer ordering | 4 | — | 4 |
|
|
| Fog shader state | 3 | +1 (Stig: hidden state) | 4 |
|
|
| UI elements | 8 | +3 (Tyre: tick rate, inventory full, sprint suppression) | 11 |
|
|
| Entity lerp | 3 | +1 (Tyre: recognition transition) | 4 |
|
|
| Anti-tedium | — | +2 (Stig: bug report capture, progress hidden) | 2 |
|
|
| **Total** | **32** | **+9** (6 Tyre + 3 Stig) | **~38-41** |
|
|
|
|
Note: Stig counts 35 (32+3), Tyre counts 38 (32+6). Combined unique total depends on overlap — upper bound is 41.
|
|
|
|
---
|
|
|
|
## 13. Open Questions for Round 3
|
|
|
|
### New Questions (raised in Round 2)
|
|
|
|
| ID | From | To | Question |
|
|
|----|------|----|----------|
|
|
| R2-OQ-01 | Hoshe | Dudley | `SetTickRate(Half)` while paused — should this unpause? Current code sets rate unconditionally (input.rs:165-168). Intentional or bug? |
|
|
| R2-OQ-02 | Hoshe | Dudley | Entity respawn + registry — when bevy recycles Entity index, does registry handle old StableId not being unregistered? |
|
|
| R2-OQ-03 | Hoshe | Tyre | `make pre-pr` — should it include `make content-ron` (YAML→RON conversion)? |
|
|
| R2-OQ-04 | Hoshe | Justine | Fixture staleness in `make pre-pr` — separate `make pre-pr-full` to keep basic pre-PR fast? |
|
|
| R2-OQ-05 | Ozzie | Dudley | `blocked_entities` debug field on ObserverSnapshot — feasible? Cost per tick? |
|
|
| R2-OQ-06 | Ozzie | Stig | Can Godot client render checklist progress overlay, or test-client-only? |
|
|
| R2-OQ-07 | Ozzie | Tyre | Test client binary location: `server/src/bin/` or separate `tools/` crate? |
|
|
| R2-OQ-08 | Ozzie | Gestalt | Cross-room transition scenarios: where in YAML hierarchy? Own checklist section? |
|
|
| R2-OQ-09 | Gestalt | All | Room ordering in Gauntlet YAML — canonical ordering affects entity StableId assignment. |
|
|
| R2-OQ-10 | Gestalt | All | Per-room reset sufficient, or need "full server restart" command? |
|
|
| R2-OQ-11 | Gestalt | All | 4 new cross-cuts — too many? Consolidate Sound Lab into Occlusion Corridor sub-area? |
|
|
|
|
### Remaining Unresolved from Round 1
|
|
|
|
| ID | Question | Status |
|
|
|----|----------|--------|
|
|
| OQ-11 | Client tests headless stability | Unresolved — no agent tested in Round 2 |
|
|
|
|
---
|
|
|
|
## 14. Points of Agreement (consensus)
|
|
|
|
1. **Test client binary is the right call.** Tyre accepted the overrule and designed it properly. The binary shares types, exercises real TCP, and enables Layer 3 testing.
|
|
2. **Determinism fixes are small and well-understood.** Dudley's code changes are concrete. Hoshe's coverage assessment confirms no existing tests break.
|
|
3. **Pause guard has zero coverage.** Hoshe independently confirmed Dudley's gap analysis. All 6 tests are genuine needs.
|
|
4. **`make pre-pr` replaces CI discipline.** Both Hoshe and Justine converged on the same chain (lint → build → test → validate → fixtures).
|
|
5. **Content cross-reference validation extends Python validator.** Both Hoshe and Justine agree: no Rust dependency, two-pass architecture (schema then cross-refs).
|
|
6. **Anti-tedium is fully specified.** Room reset (Interact-triggered), hub teleport (Home key), WRONG button (F12 + MVP captures). No dissent on any feature.
|
|
7. **38 client tests are architecturally sound.** Tyre's cross-review found no redundancies in Stig's 32 and added 6 meaningful tests.
|
|
8. **Bug catalogue fully covered.** Gestalt's invariant mapping confirms every Sprint 6-7 bug class has a test + room + invariant.
|
|
9. **Checklist format merges Stig + Ozzie proposals.** YAML with structured conditions for auto-tracking AND human prose for markdown generation.
|
|
|
|
## 15. Points of Tension
|
|
|
|
1. **Fixture staleness: BLOCKER vs WARNING.** Tyre argues BLOCKER (stale fixtures = false positive client tests). Justine tagged it WARNING. Recommend resolving in Round 3 — Tyre's argument appears stronger.
|
|
2. **F3 debug overlay: defer or implement.** Ozzie wants it for the testing experience. Stig says defer due to per-frame performance cost. WRONG button captures same data on demand. Lean toward deferral.
|
|
3. **PR tier time budget: <10 min (Hoshe) vs <15 min (Tyre).** Tyre accounts for clean-cache builds. Recommend <15 min as the budget (covers worst case).
|
|
4. **Cross-room checklist location.** Ozzie identifies a real gap: cross-room transition scenarios (Gestalt's T1-T8) don't fit in per-room YAML. Needs `cross_room_checks.yaml` or equivalent.
|
|
5. **Number of Gauntlet cross-cuts.** Gestalt proposes 4 new cross-cuts. Potential scope concern. Could consolidate Sound Lab into Occlusion Corridor sub-area.
|
|
|
|
---
|
|
|
|
## 16. Gaps Remaining
|
|
|
|
1. **Test client binary doesn't exist yet.** Tyre estimated 2-3 days. Depends on `--test-mode` + `--port 0` (Dudley, Sprint 8).
|
|
2. **Gauntlet content doesn't exist yet.** Room YAML, entity placement, checklist definitions all need writing. No specific agent assigned.
|
|
3. **Client test headless stability unverified.** OQ-11 from Round 1 remains open. Must verify before making client tests a merge gate.
|
|
4. **Cross-room checklist format unresolved.** Ozzie identified the gap. Needs design in Round 3.
|
|
5. **Gauntlet room ordering (entity spawn order → StableId assignment).** Gestalt raised this. Affects determinism — canonical YAML ordering needed.
|
|
6. **Sound events not in ObserverSnapshot.** Ozzie requires `Vec<SoundEvent>` for full test client display. Not yet designed on server side.
|
|
7. **Cognitive delay visual not yet implemented.** Tyre's tests #33-34 (blob rendering, recognition transition) depend on D-060 implementation.
|