# Hoshe — Round 1 Analysis: Boundary Values, Layer 3, CI, Content Validation **Workshop:** QA Strategy & Test Architecture **Tracks:** 4 (Serialization & Integration) + 5 (Content Scaling & CI) **Date:** 2026-02-17 **Spec references:** D-030 (testability architecture), D-020 (IPC/MessagePack protocol) --- ## TRACK 4 — Serialization & Integration ### T4-H1: MessagePack Boundary Value Test Matrix The GDScript encoder (`messagepack.gd:69-95`) uses cascading `if/elif` branches to select the MessagePack integer format. Each branch boundary is a potential off-by-one bug site — exactly where bug #4 lived. I traced every branch transition against the MessagePack spec. The encoder's branch order determines the effective ranges: | Branch | Condition (GDScript) | Effective range (after earlier branches steal values) | MsgPack format | Header byte | |--------|---------------------|------------------------------------------------------|---------------|-------------| | 1 | `-32 <= v <= 127` | -32 to 127 | positive fixint / negative fixint | 0x00-0x7f / 0xe0-0xff | | 2 | `-128 <= v < 128` | -128 to -33 | int 8 | 0xd0 | | 3 | `0 <= v <= 255` | 128 to 255 | uint 8 | 0xcc | | 4 | `-32768 <= v < 32768` | -32768 to -129 AND 256 to 32767 | int 16 | 0xd1 | | 5 | `0 <= v <= 65535` | 32768 to 65535 | uint 16 | 0xcd | | 6 | `-2^31 <= v < 2^31` | -2147483648 to -32769 AND 65536 to 2147483647 | int 32 | 0xd2 | | 7 | `0 <= v <= 2^32-1` | 2147483648 to 4294967295 | uint 32 | 0xce | | 8 | `-2^63 <= v < 2^63` | -9223372036854775808 to -2147483649 AND 4294967296 to 9223372036854775807 | int 64 | 0xd3 | | 9 | else | Godot `int` is 64-bit signed, so this branch is unreachable for valid Godot ints | uint 64 | 0xcf | **Note on branch 4:** Positive values 256-32767 are encoded as int_16, not uint_16. This is spec-valid (MessagePack allows any format that fits the value) but differs from what rmp_serde produces for unsigned Rust types. Cross-language roundtrip tests MUST cover this divergence — rmp_serde must accept int_16-encoded positive values when deserializing into `u64`. #### Complete Boundary Value Matrix Every row is a test case. Values are chosen at format transition boundaries (value-1, value, value+1). | Test ID | Value | Expected format | Header byte | Payload bytes | Why this value matters | |---------|-------|----------------|-------------|---------------|----------------------| | **Positive fixint boundaries** | | BV-P01 | 0 | pos fixint | 0x00 | (none) | Zero — minimum positive fixint | | BV-P02 | 1 | pos fixint | 0x01 | (none) | Smallest nonzero positive | | BV-P03 | 126 | pos fixint | 0x7e | (none) | One below boundary | | BV-P04 | 127 | pos fixint | 0x7f | (none) | **MAX positive fixint** — Bug #4 was here | | **fixint → uint_8 transition** | | BV-P05 | 128 | uint 8 | 0xcc | 0x80 | **MIN uint_8** — Bug #4: this was encoded as -128 | | BV-P06 | 129 | uint 8 | 0xcc | 0x81 | One above boundary | | BV-P07 | 254 | uint 8 | 0xcc | 0xfe | One below max uint_8 | | BV-P08 | 255 | uint 8 | 0xcc | 0xff | **MAX uint_8** | | **uint_8 → int_16 transition** (NOTE: encoder uses int_16, not uint_16, for 256-32767) | | BV-P09 | 256 | int 16 | 0xd1 | 0x01 0x00 | **MIN int_16 positive** — format widens to 2 bytes | | BV-P10 | 257 | int 16 | 0xd1 | 0x01 0x01 | One above boundary | | BV-P11 | 32766 | int 16 | 0xd1 | 0x7f 0xfe | One below max int_16 | | BV-P12 | 32767 | int 16 | 0xd1 | 0x7f 0xff | **MAX int_16 positive** | | **int_16 → uint_16 transition** | | BV-P13 | 32768 | uint 16 | 0xcd | 0x80 0x00 | **MIN uint_16** — exceeds int_16 max | | BV-P14 | 32769 | uint 16 | 0xcd | 0x80 0x01 | One above boundary | | BV-P15 | 65534 | uint 16 | 0xcd | 0xff 0xfe | One below max uint_16 | | BV-P16 | 65535 | uint 16 | 0xcd | 0xff 0xff | **MAX uint_16** | | **uint_16 → int_32 transition** | | BV-P17 | 65536 | int 32 | 0xd2 | 0x00 0x01 0x00 0x00 | **MIN int_32 positive** | | BV-P18 | 65537 | int 32 | 0xd2 | 0x00 0x01 0x00 0x01 | One above boundary | | BV-P19 | 2147483646 | int 32 | 0xd2 | 0x7f 0xff 0xff 0xfe | One below max int_32 | | BV-P20 | 2147483647 | int 32 | 0xd2 | 0x7f 0xff 0xff 0xff | **MAX int_32** (2^31-1) | | **int_32 → uint_32 transition** | | BV-P21 | 2147483648 | uint 32 | 0xce | 0x80 0x00 0x00 0x00 | **MIN uint_32** (2^31) | | BV-P22 | 4294967294 | uint 32 | 0xce | 0xff 0xff 0xff 0xfe | One below max uint_32 | | BV-P23 | 4294967295 | uint 32 | 0xce | 0xff 0xff 0xff 0xff | **MAX uint_32** (2^32-1) | | **uint_32 → int_64 transition** | | BV-P24 | 4294967296 | int 64 | 0xd3 | 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 | **MIN int_64 positive** (2^32) | | BV-P25 | 9223372036854775807 | int 64 | 0xd3 | 0x7f 0xff 0xff 0xff 0xff 0xff 0xff 0xff | **MAX int_64** (2^63-1, Godot max int) | | **Negative fixint boundaries** | | BV-N01 | -1 | neg fixint | 0xff | (none) | Most common negative value | | BV-N02 | -31 | neg fixint | 0xe1 | (none) | One above min fixint | | BV-N03 | -32 | neg fixint | 0xe0 | (none) | **MIN negative fixint** | | **fixint → int_8 transition** | | BV-N04 | -33 | int 8 | 0xd0 | 0xdf | **Exceeds fixint** — first value requiring int_8 | | BV-N05 | -34 | int 8 | 0xd0 | 0xde | One below boundary | | BV-N06 | -127 | int 8 | 0xd0 | 0x81 | One above min int_8 | | BV-N07 | -128 | int 8 | 0xd0 | 0x80 | **MIN int_8** | | **int_8 → int_16 transition** | | BV-N08 | -129 | int 16 | 0xd1 | 0xff 0x7f | **Exceeds int_8** — first value requiring int_16 | | BV-N09 | -130 | int 16 | 0xd1 | 0xff 0x7e | One below boundary | | BV-N10 | -32767 | int 16 | 0xd1 | 0x80 0x01 | One above min int_16 | | BV-N11 | -32768 | int 16 | 0xd1 | 0x80 0x00 | **MIN int_16** | | **int_16 → int_32 transition** | | BV-N12 | -32769 | int 32 | 0xd2 | 0xff 0xff 0x7f 0xff | **Exceeds int_16** | | BV-N13 | -2147483647 | int 32 | 0xd2 | 0x80 0x00 0x00 0x01 | One above min int_32 | | BV-N14 | -2147483648 | int 32 | 0xd2 | 0x80 0x00 0x00 0x00 | **MIN int_32** (-2^31) | | **int_32 → int_64 transition** | | BV-N15 | -2147483649 | int 64 | 0xd3 | 0xff 0xff 0xff 0xff 0x7f 0xff 0xff 0xff | **Exceeds int_32** (-2^31-1) | | BV-N16 | -9223372036854775808 | int 64 | 0xd3 | 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00 | **MIN int_64** (-2^63, Godot min int) | **Total: 41 boundary values.** #### Game-relevant values (sanity check) These specific values appear in the protocol and must be tested even though they may not sit on format boundaries: | Value | Protocol field | Expected format | |-------|---------------|----------------| | 0 | `tick` at game start | pos fixint | | 10 | `tick` at 1 second | pos fixint | | 42 | `tick` in test fixtures | pos fixint | | 128 | `tick` at ~13 seconds — **BUG #4 SITE** | uint 8 | | 500 | `tick` in v2_full fixture | int 16 (via branch 4) | | 999 | `tick` in multi_entity fixture | int 16 | | 100 | `entity_id` in test fixtures | pos fixint | | 720 | `time_of_day` (noon) | int 16 | | 1440 | `time_of_day` (end of day) | int 16 | #### Observation: Asymmetric encoding between GDScript and Rust The GDScript encoder picks int_16 for values 256-32767 (positive values in the signed range). The Rust encoder (rmp_serde) picks uint_16 for the same values. Both are valid per the MessagePack spec, but this means GDScript-encoded bytes and Rust-encoded bytes for the same logical value will differ at the byte level. **This is NOT a bug**, but it means: - Byte-for-byte golden file comparison between GDScript-encoded and Rust-encoded data will FAIL for values 256-32767 (and similarly for 65536-2^31-1). - Golden files must be direction-specific: "Rust encodes this snapshot" (canonical) vs "GDScript encodes this input" (canonical). - Decoders on both sides MUST accept both signed and unsigned encodings for the same value. Tests should verify this explicitly. --- ### T4-H2: Where Should Boundary Tests Live? **Recommendation: Both, but with different scopes.** | Layer | Location | What it tests | Speed | Frequency | |-------|----------|--------------|-------|-----------| | **Encode-only** (GDScript) | `client/tests/test_msgpack_boundaries.gd` | GDScript encoder produces correct format for each boundary value | Fast (~100ms) | Every commit | | **Encode-only** (Rust) | `server/tests/serialization.rs` (extend) | Rust encoder produces correct format for each boundary value | Fast (~50ms) | Every commit | | **Decode cross-language** | Fixtures: Rust-generated `.msgpack` files decoded by GDScript | GDScript decoder handles Rust-encoded boundary values | Medium (~2s) | Every PR | | **Decode cross-language** | Fixtures: GDScript-generated `.msgpack` files decoded by Rust | Rust decoder handles GDScript-encoded boundary values | Medium (~2s) | Every PR | | **Full roundtrip** | `bridge_tcp.rs` extension: send boundary-valued snapshots over TCP, receive on GDScript side | End-to-end wire format agreement | Slow (~5s) | Nightly / pre-merge | **Rationale:** Bug #4 was an encode-side bug (GDScript encoder chose wrong format). A GDScript-only encode test would have caught it. But the WORST bugs are decode mismatches — where one side encodes a value that the other side decodes as a different value. Cross-language fixtures catch those. The roundtrip catches protocol-level issues (framing, batching). **Concrete test functions:** GDScript (`test_msgpack_boundaries.gd`): ```gdscript func test_boundary_positive_fixint_max() -> void: # BV-P04: 127 must encode as single byte 0x7f (positive fixint) var result = Messagepack.encode(127) assert_that(result.value.size()).is_equal(1) assert_that(result.value[0]).is_equal(0x7f) func test_boundary_uint8_min() -> void: # BV-P05: 128 must encode as uint_8 (0xcc, 0x80) — Bug #4 regression guard var result = Messagepack.encode(128) assert_that(result.value.size()).is_equal(2) assert_that(result.value[0]).is_equal(0xcc) assert_that(result.value[1]).is_equal(0x80) func test_boundary_roundtrip_all() -> void: # Every boundary value must survive encode→decode roundtrip var boundaries = [0, 1, 126, 127, 128, 129, 254, 255, 256, 257, 32766, 32767, 32768, 32769, 65534, 65535, 65536, -1, -31, -32, -33, -34, -127, -128, -129, -130, -32767, -32768, -32769] for v in boundaries: var encoded = Messagepack.encode(v) assert_that(encoded.status).is_null() var decoded = Messagepack.decode(encoded.value) assert_that(decoded.status).is_null() assert_that(decoded.value).is_equal(v) ``` Rust (`serialization.rs` addition): ```rust #[test] fn boundary_values_roundtrip() { let boundaries: Vec = vec![ 0, 1, 126, 127, 128, 129, 254, 255, 256, 257, 32766, 32767, 32768, 32769, 65534, 65535, 65536, 65537, 2147483646, 2147483647, 2147483648, 4294967294, 4294967295, 4294967296, -1, -31, -32, -33, -34, -127, -128, -129, -130, -32767, -32768, -32769, -2147483647, -2147483648, -2147483649, ]; for val in boundaries { let bytes = rmp_serde::to_vec(&val).unwrap(); let decoded: i64 = rmp_serde::from_slice(&bytes).unwrap(); assert_eq!(decoded, val, "boundary value {} did not roundtrip", val); } } ``` --- ### T4-H3: Should gen_fixtures.rs Generate Boundary Value Fixtures? **Yes. Concrete proposal:** Add a `generate_boundary_fixtures()` function to `gen_fixtures.rs` that generates two types of fixtures: 1. **Raw integer boundary fixtures** — one `.msgpack` file per boundary value, containing just the encoded integer. Filename encodes the expected value: `boundary_int_127.msgpack`, `boundary_int_128.msgpack`, `boundary_int_neg128.msgpack`, etc. 2. **Snapshot boundary fixtures** — snapshots with tick values at critical boundaries. These test the field-level encoding within a struct context: - `snapshot_tick_127.msgpack` — tick at max positive fixint - `snapshot_tick_128.msgpack` — tick at min uint_8 (Bug #4 regression) - `snapshot_tick_32768.msgpack` — tick at min uint_16 - `snapshot_tick_65536.msgpack` — tick at min int_32 - `snapshot_entity_id_boundary.msgpack` — entity_id values at boundaries **Implementation sketch for gen_fixtures.rs:** ```rust #[test] #[ignore] fn generate_boundary_fixtures() { // Raw integer boundary values let boundaries: Vec<(i64, &str)> = vec![ (127, "boundary_int_127"), (128, "boundary_int_128"), (255, "boundary_int_255"), (256, "boundary_int_256"), (32767, "boundary_int_32767"), (32768, "boundary_int_32768"), (65535, "boundary_int_65535"), (65536, "boundary_int_65536"), (-32, "boundary_int_neg32"), (-33, "boundary_int_neg33"), (-128, "boundary_int_neg128"), (-129, "boundary_int_neg129"), (-32768, "boundary_int_neg32768"), (-32769, "boundary_int_neg32769"), ]; for (value, name) in &boundaries { write_fixture(name, &rmp_serde::to_vec(value).unwrap()); } // Snapshot with tick at the Bug #4 boundary let snapshot_128 = fixture_snapshot(128, vec![]); write_fixture("snapshot_tick_128", &rmp_serde::to_vec_named(&snapshot_128).unwrap()); // Snapshot with large entity_id let snapshot_large_id = fixture_snapshot(0, vec![VisibleEntity { entity_id: 65536, // uint_32 boundary x: 0.0, y: 0.0, z: 0, kind: EntityKind::Npc, visibility: VisibilitySector::Forward, relationship: RelationshipState::Unknown, observation: EntityVisibility::Visible, }]); write_fixture("snapshot_entity_id_65536", &rmp_serde::to_vec_named(&snapshot_large_id).unwrap()); } ``` **Client-side verification** (`test_msgpack_boundaries.gd`): ```gdscript func test_boundary_fixture_int_128() -> void: # Bug #4 regression: Rust encodes tick=128 as uint_8, GDScript must decode to 128 (not -128) var bytes = _load_fixture("boundary_int_128") var result = Messagepack.decode(bytes) assert_that(result.value).is_equal(128) func test_boundary_fixture_snapshot_tick_128() -> void: var bytes = _load_fixture("snapshot_tick_128") var snapshot = Protocol.decode_snapshot(bytes) assert_that(snapshot.tick).is_equal(128) ``` **Workflow:** `make fixtures` regenerates all fixtures (existing + boundary). Client tests verify them. This is D-030 Layer 1: fixture-based cross-language validation. --- ### T4-H4: Golden File Approach Tied to the Gauntlet **Proposal: Gauntlet Snapshot Golden File Pipeline** The golden file flow: ``` [1. Server generates] [2. Committed to repo] [3. Client verifies] cargo test --test client/tests/fixtures/ make test-client gen_gauntlet_golden gauntlet/ ↓ -- --ignored tick_0.msgpack test_gauntlet_golden.gd ↓ tick_10.msgpack verifies each fixture Boots Gauntlet map tick_0.json (human-readable decodes to expected with seed=42 diff companion) field values Runs to tick 0, 10 Serializes ObserverSnapshot from fixed player position Writes .msgpack + .json ``` **Key design decisions:** 1. **Format: ObserverSnapshot, not full ECS world.** The snapshot is what the client sees — it's the contract surface. Internal ECS changes that don't affect the snapshot are allowed without golden file breakage. This keeps the golden file stable and focused. 2. **JSON companion for diff.** Each `.msgpack` fixture gets a parallel `.json` file containing the same data in human-readable form. When the golden file breaks, `git diff` on the JSON shows exactly what changed (e.g., "entity_id changed from 100 to 101" or "new field `zone_id` appeared"). 3. **Fixed player position.** The Gauntlet spec defines a named position per room (e.g., `GAUNTLET.occlusion_corridor.observer_position = (15, 10, 0)`). Golden files are generated from that exact position. Moving the observer changes what's visible. 4. **Seed pinning.** `seed=42` (or whatever the Gauntlet uses). The golden file test MUST fail if the seed produces different output, which catches non-determinism bugs. **CI integration:** ```yaml # Pseudo-CI pipeline (Gitea Actions or equivalent) golden-file-check: steps: - cargo build --release - cargo test --test gen_gauntlet_golden -- --ignored # Generates fresh .msgpack files in a temp directory - diff client/tests/fixtures/gauntlet/tick_0.msgpack /tmp/gauntlet_fresh/tick_0.msgpack # If diff is non-empty, the golden file is stale → FAIL # Developer must regenerate and commit updated golden files ``` **When golden files legitimately change:** 1. Developer changes server logic that affects snapshot output 2. Developer runs `make fixtures-gauntlet` (regenerates golden files) 3. Developer inspects the JSON diff: `git diff client/tests/fixtures/gauntlet/tick_0.json` 4. If the diff is expected, commit the updated golden files 5. PR review includes the golden file diff as mandatory review artifact **Breakage detection granularity:** The golden file test should not just do byte comparison. It should decode both the golden file and the fresh snapshot, then compare field by field: ```rust #[test] fn gauntlet_tick_0_matches_golden() { let golden: ObserverSnapshot = load_golden("tick_0"); let fresh: ObserverSnapshot = run_gauntlet_to_tick(0); assert_eq!(golden.tick, fresh.tick); assert_eq!(golden.entities.len(), fresh.entities.len()); for (g, f) in golden.entities.iter().zip(fresh.entities.iter()) { assert_eq!(g.entity_id, f.entity_id, "entity_id mismatch"); assert!((g.x - f.x).abs() < 0.01, "x mismatch for entity {}", g.entity_id); assert!((g.y - f.y).abs() < 0.01, "y mismatch for entity {}", g.entity_id); } // ... etc for all fields } ``` This gives actionable error messages instead of "binary files differ." --- ### T4-H5: Minimum Viable Layer 3 Test (Real Subprocess Integration) **Current state assessment:** | Layer | Status | What exists | |-------|--------|------------| | Layer 1 (fixtures) | **Complete** | `gen_fixtures.rs` generates, `test_protocol.gd` + `serialization.rs` verify | | Layer 2 (bridge) | **Partial** | `bridge_tcp.rs` + `bridge_ipc.rs` test roundtrip. `game_loop.rs` tests full pipeline. But all run within one process (server as thread, not subprocess). | | Layer 3 (real subprocess) | **Missing** | No test launches the actual `settled-reach-server` binary as a child process | **Why Layer 3 matters:** Bug #1 (server never sends snapshots in live mode) was caused by `read_framed()` blocking the bevy Update schedule. This bug only manifests with the real binary running as a subprocess — the in-process thread tests in `game_loop.rs` don't reproduce it because the thread shares memory and doesn't have the same blocking behavior. **Minimum viable Layer 3 test specification:** ``` Test name: server_subprocess_sends_snapshot_on_connect File: tests/integration_subprocess.rs (or tests/layer3.rs) Setup: 1. cargo build the server binary (or use pre-built from CI artifact) 2. Launch server binary as child process: `Command::new("target/debug/settled-reach-server")` with args: --test-mode --port 0 (random port, printed to stdout) 3. Parse port from server stdout 4. Connect to server via TCP Test steps: 1. Send one PlayerInput (MoveNorth, tick 0) via write_framed 2. Read one ObserverSnapshot via read_framed with 5-second timeout 3. Assert: snapshot.version == PROTOCOL_VERSION 4. Assert: snapshot.tick == 0 5. Assert: snapshot.entities.len() >= 1 (at least the player) 6. Assert: player entity has kind == Player Teardown: 1. Drop TCP connection 2. Kill child process (SIGTERM) 3. Wait for exit with timeout Duration budget: < 10 seconds (including server startup) Frequency: Nightly + pre-merge (D-030 says "slow, daily/pre-merge") ``` **Implementation requirements:** 1. **Server needs `--test-mode` flag.** This flag should: - Load the proof room (or Gauntlet) instead of requiring content files - Print the listening port to stdout: `LISTENING:9876` - Use a fixed seed for determinism - Exit after first client disconnects (or after timeout) 2. **Server needs `--port 0` support.** Bind to random available port and print it. Prevents test flakiness from port conflicts. 3. **Test function:** ```rust use std::process::{Command, Stdio}; use std::io::{BufRead, BufReader as StdBufReader}; use std::net::TcpStream; use std::time::Duration; #[test] #[ignore] // Layer 3: slow, run with --ignored or in CI nightly fn server_subprocess_sends_snapshot_on_connect() { // Build the server binary let status = Command::new("cargo") .args(["build", "--bin", "settled-reach-server"]) .status() .expect("cargo build failed"); assert!(status.success(), "server build failed"); // Launch server as subprocess let mut server = Command::new("target/debug/settled-reach-server") .args(["--test-mode", "--port", "0"]) .stdout(Stdio::piped()) .stderr(Stdio::inherit()) .spawn() .expect("failed to launch server"); // Parse port from stdout let stdout = server.stdout.take().unwrap(); let mut lines = StdBufReader::new(stdout).lines(); let port: u16 = loop { let line = lines.next().expect("server stdout ended").expect("read line"); if let Some(port_str) = line.strip_prefix("LISTENING:") { break port_str.trim().parse().expect("parse port"); } }; // Connect as client let stream = TcpStream::connect(format!("127.0.0.1:{}", port)) .expect("connect to server"); stream.set_read_timeout(Some(Duration::from_secs(5))).unwrap(); let mut reader = std::io::BufReader::new(stream.try_clone().unwrap()); let mut writer = std::io::BufWriter::new(stream); // Send input let inputs = vec![PlayerInput { tick: 0, action: PlayerAction::MoveNorth, }]; let payload = rmp_serde::to_vec_named(&inputs).expect("serialize"); write_framed(&mut writer, &payload).expect("send input"); // Receive snapshot let response = read_framed(&mut reader) .expect("read snapshot") .expect("not EOF — server must send snapshot after receiving input"); let snapshot: ObserverSnapshot = rmp_serde::from_slice(&response).expect("deserialize snapshot"); // Assertions assert_eq!(snapshot.version, PROTOCOL_VERSION, "snapshot version mismatch — protocol incompatibility"); assert_eq!(snapshot.tick, 0); assert!(!snapshot.entities.is_empty(), "snapshot must contain at least the player entity"); let player = snapshot.entities.iter() .find(|e| matches!(e.kind, EntityKind::Player)) .expect("no Player entity in snapshot"); assert!(player.x > 0.0, "player x must be positive"); assert!(player.y > 0.0, "player y must be positive"); // Cleanup drop(reader); drop(writer); server.kill().ok(); server.wait().ok(); } ``` **What this catches that Layer 2 doesn't:** - Server binary startup issues (missing resources, config, panic on init) - Subprocess I/O blocking (Bug #1 class) - Protocol version mismatch between compiled server and test expectations - Real TCP behavior (Nagle's algorithm, buffer sizes, connection lifecycle) --- ## TRACK 5 — Content Scaling & CI Pipeline ### T5-H1: Why Don't Client Tests Run in CI? **Finding: There IS no CI pipeline at all.** No `.github/`, `.gitea/`, or other CI configuration files exist in the repository. The Makefile has `ci`, `ci-server`, `ci-client` targets, but these are designed for local execution only. The blockers for CI are, in order of priority: 1. **No CI runner configured.** The project uses Gitea (`git.schweitz.internal`), which supports Gitea Actions (GitHub Actions compatible). But no workflow files exist. This is the primary blocker — it's wiring, not a technical limitation. 2. **Godot headless availability.** `make test-client` uses `$(GODOT) --headless --path client -s res://addons/gdUnit4/bin/GdUnitCmdTool.gd --ignoreHeadlessMode`. This requires: - Godot 4.6 binary on the CI runner - The `--headless` flag (supported since Godot 4.0) - No GPU required (headless mode uses software rendering) - The `--ignoreHeadlessMode` flag for gdUnit4 (already present in Makefile) This is solvable: install Godot on the CI runner, or use the official Godot Docker image. 3. **gdUnit4 test runner stability in headless.** gdUnit4 documents headless CI support via `GdUnitCmdTool.gd`. I haven't verified whether the current test suite actually passes headless — some tests may have implicit dependencies on window size, input events, or rendering state. This needs a verification run: `make test-client` on a headless machine. 4. **Test output format.** D-030 specifies JSON summary output. gdUnit4 produces JUnit XML by default. The CI pipeline needs a format adapter — or we accept JUnit XML for CI (most CI systems parse it natively). **Recommended action:** The first step is just running `make ci` on the Gitea runner. Everything else is refinement. --- ### T5-H2: Minimal CI Pipeline — Tier Proposal | Tier | Trigger | Tests | Duration budget | Rationale | |------|---------|-------|----------------|-----------| | **Commit (fast feedback)** | Every push to any branch | `make lint-server` (clippy + fmt), `make lint-client` (GDScript error check), `make validate-content` (YAML schema), `make check-fact-ids` | < 2 min | Catches syntax errors, formatting, broken content. No compilation needed for content checks. | | **PR (merge gate)** | PR opened or updated | Everything in Commit tier + `make build` (both server + client), `make test-server` (cargo nextest), `make test-client` (gdUnit4 headless), `make fixtures` + verify no diff (fixture staleness check) | < 10 min | Full build + test. Fixture staleness check catches protocol changes that weren't regenerated. | | **Nightly (deep validation)** | Scheduled, 1x/day on main | Everything in PR tier + Layer 3 subprocess test (`cargo test --test layer3 -- --ignored`), Gauntlet golden file regeneration + diff check, Content load-test (boot server with full content, tick 100 times), Performance benchmark (Gauntlet 100 ticks, assert < time budget) | < 30 min | Slow tests that catch subtle integration bugs. Performance regression detection. | **Pipeline configuration (Gitea Actions):** ```yaml # .gitea/workflows/ci.yaml name: CI on: push: branches: ['*'] pull_request: branches: [main] schedule: - cron: '0 3 * * *' # Nightly at 03:00 jobs: commit-checks: runs-on: self-hosted steps: - uses: actions/checkout@v4 - run: make lint-server - run: make lint-client - run: make validate-content - run: make check-fact-ids pr-checks: if: github.event_name == 'pull_request' needs: commit-checks runs-on: self-hosted steps: - uses: actions/checkout@v4 - run: make build - run: make test-server - run: make test-client - run: make fixtures - run: git diff --exit-code client/tests/fixtures/ # Fails if fixtures are stale nightly: if: github.event_name == 'schedule' needs: commit-checks runs-on: self-hosted steps: - uses: actions/checkout@v4 - run: make build - run: make test-server - run: make test-client - run: cargo test --test layer3 -- --ignored - run: make test-gauntlet-golden # (new target, TBD) - run: make test-content-load # (new target, TBD) ``` **Critical gate:** PR tier is the merge gate. A PR cannot merge if any PR-tier test fails. Nightly failures generate alerts but don't block work. --- ### T5-H3: Minimum Runtime Content Validation **Current state:** `make validate-content` only checks YAML structure against JSON Schema. It does NOT verify that the content can actually be loaded and used by the server. **Minimum runtime validation — "boot and tick" test:** ``` Test name: content_loads_and_ticks_without_panic File: server/tests/content_loading.rs (extend existing) Setup: 1. Load content from content/campaigns/ using ContentLoader 2. Build a minimal bevy App with SimulationPlugin + ContentPlugin 3. Insert loaded content as resources Test steps: 1. app.update() — tick once 2. Assert: no panic (the test itself succeeding is the assertion) 3. Assert: SimulationTime.tick == 1 4. Assert: at least 1 entity with PlayerCharacter component exists 5. Assert: at least 1 entity with NPC-related components exists (if content defines NPCs) Duration: < 5 seconds Frequency: Every PR ``` **Graduated runtime validation layers:** | Level | What it checks | Test type | Duration | |-------|---------------|-----------|----------| | 1. Boot | Content loads without panic | Unit test in `content_loading.rs` | < 2s | | 2. Tick | 10 ticks complete without panic | Integration test | < 3s | | 3. Snapshot | ObserverSnapshot generates with expected entity count | Integration test | < 5s | | 4. Cross-reference | All `entity_ref` in dialogue resolves to spawned entities | Dedicated validation test | < 3s | | 5. Stress | 100 ticks with max-NPC content pack, no tick exceeds 50ms | Performance test | < 30s | **Level 1 already partially exists** in `content_loading.rs`. We should extend it to Level 3 as the minimum for PR gating. **Concrete Level 3 test:** ```rust #[test] fn content_produces_valid_snapshot() { let mut app = build_app_with_content("content/campaigns/meridian"); // Tick 10 times for _ in 0..10 { app.update(); } // Generate snapshot from player position let snapshot = extract_observer_snapshot(&app); assert!(snapshot.tick > 0, "simulation must have advanced"); assert!(!snapshot.entities.is_empty(), "snapshot must contain entities"); // Verify player exists in snapshot let has_player = snapshot.entities.iter().any(|e| matches!(e.kind, EntityKind::Player)); assert!(has_player, "player must be visible in own snapshot"); // Verify NPC count matches content expectation let npc_count = snapshot.entities.iter().filter(|e| matches!(e.kind, EntityKind::Npc)).count(); assert!(npc_count >= 1, "at least one NPC should be visible from spawn position"); } ``` --- ### T5-H4: Content Scaling Test — "Adding an NPC Doesn't Break Anything" **The test:** When a new NPC is added to a district, the game must still boot, tick, and produce valid snapshots. **Test specification:** ``` Test name: adding_npc_to_district_preserves_functionality File: server/tests/content_scaling.rs (new) Approach: Comparative testing 1. Load baseline content → boot → tick 10 → snapshot (baseline) 2. Load baseline content + 1 extra NPC → boot → tick 10 → snapshot (modified) 3. Compare: modified snapshot should be a SUPERSET of baseline Assertions: - Modified boots without panic - Modified ticks 10 times without panic - Modified snapshot has >= baseline entity count - Modified snapshot has exactly baseline.npcs + 1 NPCs - All baseline NPCs still present (by entity name/stable_id) - Player entity unchanged between baseline and modified - No tick exceeds 50ms (performance regression check) - All nearby_interactions for existing NPCs still present ``` **Concrete implementation:** ```rust #[test] fn adding_npc_preserves_existing_entities() { // Baseline: load district with known NPCs let baseline_app = build_app_with_content("content/campaigns/meridian"); let baseline_snapshot = run_and_snapshot(&baseline_app, 10); let baseline_npc_ids: HashSet = baseline_snapshot.entities.iter() .filter(|e| matches!(e.kind, EntityKind::Npc)) .map(|e| e.entity_id) .collect(); // Modified: same district + one extra NPC injected programmatically let mut modified_app = build_app_with_content("content/campaigns/meridian"); inject_test_npc(&mut modified_app, "test_extra_npc", TilePosition::new(20, 20, 0)); let modified_snapshot = run_and_snapshot(&modified_app, 10); let modified_npc_ids: HashSet = modified_snapshot.entities.iter() .filter(|e| matches!(e.kind, EntityKind::Npc)) .map(|e| e.entity_id) .collect(); // All baseline NPCs still exist assert!(baseline_npc_ids.is_subset(&modified_npc_ids), "existing NPCs disappeared after adding new NPC"); // Exactly one more NPC assert_eq!(modified_npc_ids.len(), baseline_npc_ids.len() + 1, "expected exactly one additional NPC"); // Player still exists and at same position let baseline_player = find_player(&baseline_snapshot); let modified_player = find_player(&modified_snapshot); assert_eq!(baseline_player.x, modified_player.x); assert_eq!(baseline_player.y, modified_player.y); } ``` **Scaling test matrix:** | Scenario | Baseline | Modified | Key assertion | |----------|----------|----------|--------------| | +1 NPC | 3 NPCs | 4 NPCs | All original NPCs present | | +5 NPCs | 3 NPCs | 8 NPCs | No tick > 50ms | | +15 NPCs | 3 NPCs | 18 NPCs (Crowd Plaza scale) | No tick > 100ms, no panic | | +1 location | 1 district | 1 district + 1 location YAML | Content loads, entities spawn correctly | | +1 item | 9 items | 10 items | Inventory interaction still works (9-slot limit not exceeded by world items) | **Content-level validation (complement to runtime tests):** Add to `make validate-content`: - **Entity reference check:** Every `entity_ref` in dialogue/monologue YAML resolves to a defined NPC profile - **Location slug check:** Every `location` field matches a defined location YAML - **Fact ID check:** Already exists as `make check-fact-ids` - **District capacity check:** Warn if a district exceeds 15 NPCs (Crowd Plaza performance boundary) --- ## Summary of Deliverables | Deliverable | Track | Status | |-------------|-------|--------| | Boundary value test matrix (41 values) | T4-H1 | Complete | | Boundary test placement recommendation | T4-H2 | Complete | | gen_fixtures.rs extension specification | T4-H3 | Complete | | Golden file pipeline specification | T4-H4 | Complete | | Layer 3 test specification with code | T4-H5 | Complete | | CI blocker analysis | T5-H1 | Complete | | CI pipeline tier proposal (commit/PR/nightly) | T5-H2 | Complete | | Runtime content validation specification | T5-H3 | Complete | | Content scaling test specification | T5-H4 | Complete | ## Open Questions for Round 2 1. **For Dudley:** Does the server binary support `--test-mode` and `--port 0` flags? If not, what's the minimum change to add them? Layer 3 depends on this. 2. **For Dudley:** The GDScript encoder uses int_16 for positive values 256-32767 while rmp_serde uses uint_16. Does `rmp_serde::from_slice::()` accept int_16-encoded positive values? I believe it does, but this needs explicit verification. 3. **For Tyre/Justine:** The Gitea instance at `git.schweitz.internal` — does it have Gitea Actions enabled? Is there a self-hosted runner available? CI pipeline design depends on this. 4. **For Tyre:** The golden file JSON companion format — should it be the full ObserverSnapshot serialized to JSON, or a reduced "diff-friendly" format with sorted keys and abbreviated fields? 5. **For Justine:** Fixture staleness check in CI (`make fixtures && git diff --exit-code`) — is this robust enough, or do we need a content-addressed hash approach?