settled-reach/docs/workshops/v01-gap-analysis/round1-hoshe.md

title, description, type, status, workshop, agent, round, created

title	description	type	status	workshop	agent	round	created
Round 1: Testability Deep-Dive (Track 4) — Hoshe	QA analysis of test architecture, Rust/Godot framework selection, IPC testing layers, and success criteria automation	workshop	archived	v01-gap-analysis	hoshe	1	2026-02-11

Round 1: Testability Deep-Dive (Track 4) — Hoshe

Workshop: v0.1 Gap Analysis Workshop Agent: Hoshe (QA Engineer) Date: 2026-02-11 Scope: Full testability initiative (#34, epics #61-#64, stories #200-#217) + cross-initiative testing needs

---

1. Test Architecture for Agent-Driven Development

The core constraint: Claude Code agents write tests, run tests, and interpret results. Everything flows from that.

What works for agents

Simple invocation. One command, no interactive prompts, no GUI. The ideal is:

simulation/test.sh              # runs all Rust tests
simulation/test.sh perception   # runs one module
client/test.sh                  # runs all Godot tests
test-integration.sh             # runs IPC + sync tests

Wrapper scripts, like the existing db/connectors/sqlite-* pattern. Agent types command, gets back structured output. No environment detective work.

Structured output with failure context. Agents need:

• Pass/fail per test (not just a summary count)
• Failure message with assertion details (expected vs actual)
• File path + line number of the failure
• Test name that maps to what was being tested (not test_17)

Deterministic results. Flaky tests are agent poison. An agent can't distinguish "test is flaky, retry" from "I broke something." Every test must produce the same result given the same inputs. This has implications for the simulation (see section on determinism below).

Fast feedback. Agents work in edit-test cycles. If the test suite takes 30 seconds, that's 30 seconds of wasted context window per iteration. Target: full Rust unit tests < 5s, individual test < 100ms.

Self-contained tests. No external service dependencies, no database state to set up, no network calls. Tests create their own world, run, assert, tear down. bevy_ecs is naturally good at this (create a World, add components, run system, inspect components). GDScript is harder — scene trees have lifecycle assumptions.

Recommended test structure

simulation/
  src/
    perception/
      mod.rs
      shadowcast.rs
      vision_cone.rs
    perception/tests/        # unit tests per module (or inline #[cfg(test)])
  tests/
    integration/             # multi-system integration tests
      perception_chain.rs    # shadowcast -> vision cone -> observer query
      npc_behavior.rs        # routine + mood + relationship interaction
      tier_transitions.rs    # active -> background -> state-saved
    determinism/             # replay-based determinism tests
    benchmarks/              # criterion benchmarks

client/
  test/                      # gdUnit4 tests (see section 3)
    unit/
    scene/
    integration/

protocol/
  tests/                     # serialization round-trip, schema compat

scripts/
  test-rust.sh               # wraps cargo test/nextest with output formatting
  test-godot.sh              # wraps godot --headless with gdUnit4
  test-integration.sh        # spins up real subprocess, runs protocol tests
  test-all.sh                # runs everything, returns combined report

---

2. Rust Testing: Inline vs External Test Crate

Recommendation: Hybrid approach

Inline #[cfg(test)] for unit tests within each module. Rationale:

• Direct access to private functions and internal state
• Tests live next to the code they test — agents can read both in one file
• Zero boilerplate: cargo test finds them automatically
• bevy_ecs systems typically have small, focused functions that test well inline

// simulation/src/perception/shadowcast.rs

pub fn compute_visibility(origin: IVec2, range: i32, map: &TileMap) -> HashSet<IVec2> {
    // ... implementation
}

#[cfg(test)]
mod tests {
    use super::*;
    use bevy::prelude::*;

    #[test]
    fn empty_room_full_visibility() {
        let map = TileMap::empty(10, 10);
        let visible = compute_visibility(IVec2::new(5, 5), 5, &map);
        // All tiles within range should be visible
        assert!(visible.contains(&IVec2::new(5, 6)));
        assert!(visible.contains(&IVec2::new(3, 3)));
    }

    #[test]
    fn wall_blocks_vision() {
        let mut map = TileMap::empty(10, 10);
        map.set_wall(5, 6);
        let visible = compute_visibility(IVec2::new(5, 5), 5, &map);
        assert!(visible.contains(&IVec2::new(5, 6))); // wall itself is visible
        assert!(!visible.contains(&IVec2::new(5, 7))); // behind wall is not
    }
}

Separate tests/ directory for integration tests. These test system interactions, multi-tick behavior, and cross-module concerns:

// simulation/tests/integration/perception_chain.rs

use bevy::prelude::*;
use simulation::perception::*;
use simulation::components::*;

#[test]
fn observer_sees_entity_in_los() {
    let mut world = World::new();

    // Set up minimal world
    let observer = world.spawn((
        Position(IVec2::new(5, 5)),
        VisionCone { range: 10, facing: Direction::North },
        Observer,
    )).id();

    let target = world.spawn((
        Position(IVec2::new(5, 8)),
        Visible,
        NpcMarker,
    )).id();

    // Run perception systems
    let mut schedule = Schedule::default();
    schedule.add_systems((
        compute_shadowcast_system,
        apply_vision_cone_system,
        generate_observer_snapshot_system,
    ).chain());
    schedule.run(&mut world);

    // Assert observer can see target
    let snapshot = world.get::<ObserverSnapshot>(observer).unwrap();
    assert!(snapshot.visible_entities.contains(&target));
}

Why not a separate test crate? For our project size (one developer + agents), the indirection of a separate crate adds complexity without benefit. Inline + integration directory covers all needs. A separate test crate makes sense when you need to test the public API boundary of a library — our simulation isn't a published crate, it's an application.

bevy_ecs-specific testing patterns

Pattern 1: World + Schedule for system tests Create a World, spawn test entities, build a Schedule with the systems under test, call schedule.run(&mut world), assert on component state. This is the bread-and-butter pattern.

Pattern 2: App for multi-tick tests For tests that need multiple simulation ticks (tier transitions, behavior over time):

let mut app = App::new();
app.add_plugins(MinimalPlugins);
app.add_systems(Update, (system_a, system_b).chain());
app.world_mut().spawn(/* test entities */);

// Simulate N ticks
for _ in 0..10 {
    app.update();
}

// Assert state after N ticks

Pattern 3: Resource injection for determinism Replace RNG, time, and external inputs with test-controlled resources:

// Production: reads real time
// Test: inject frozen or stepped time
app.insert_resource(SimulationTime::fixed(GameTime::new(8, 0, 0))); // 8 AM

Critical for #201 (deterministic replay): The simulation MUST consume time, randomness, and player input exclusively through injectable resources. No std::time::Instant, no rand::thread_rng() — everything goes through a SimRng resource seeded from the world seed. This isn't test infrastructure — it's a production architecture requirement (D-010 principle 4: deterministic simulation with input events).

---

3. Godot Testing: GUT vs gdUnit4

Recommendation: gdUnit4

Criterion	GUT	gdUnit4	Winner
CLI/headless	godot -d -s --path . addons/gut/gut_cmdln.gd	GdUnitCmdTool built-in CLI	gdUnit4 (cleaner interface)
Output format	JUnit XML, console text	JUnit XML, HTML, JSON summaries	gdUnit4 (JSON for agents)
Scene testing	Basic	GdUnitSceneRunner with input simulation, time control	gdUnit4 (much richer)
Mocking	Doubler system	Code-gen mocking + spy builder	gdUnit4 (more capable)
Assertions	Good basics	13 specialized assertion types including Godot-native	gdUnit4 (exhaustive)
CI integration	JUnit XML + exit codes	GitHub Action (gdunit4-action), JUnit XML + HTML	gdUnit4 (turnkey CI)
Maintenance	Single maintainer (bitwes)	Organization (godot-gdunit-labs)	gdUnit4 (bus factor)
Known issues	Headless mode issues (#491 on GitHub)	Stable headless support	gdUnit4
Agent friendliness	Good	Better (JSON output, structured reports)	gdUnit4

Can we realistically run Godot tests from Claude Code?

Yes, with caveats.

The invocation would be:

godot --headless --path client/ -s addons/gdUnit4/bin/GdUnitCmdTool.gd --test-suite test/unit/

Caveats:

1. Godot must be installed and on PATH on the dev machine. This is a tooling requirement for ticket #216.
2. Headless mode doesn't render. We can't visually verify fog rendering or sprite placement from automated tests. Rendering verification (#208) is limited to asserting shader parameters, visibility flags, node properties — not pixel output.
3. Scene lifecycle. GDScript tests that need _ready(), _process(), or signal connections require GdUnitSceneRunner to simulate the scene tree lifecycle. Direct function tests are simpler but limited.
4. Import cache. First run after project changes requires Godot to rebuild the import cache, which can take seconds. Subsequent runs are fast.

What we CAN test in Godot client (pure renderer, no game logic per D-020):

• MessagePack deserialization (GDScript side)
• ObserverSnapshot → scene tree mapping
• Fog overlay parameter calculation
• UI widget state from HUD data
• Input capture → PlayerInput serialization
• Sound event → audio playback trigger

What we CANNOT effectively test in headless Godot:

• Visual correctness of fog rendering (needs eyes or screenshot comparison)
• Audio output correctness (headless has no audio device)
• Frame rate / rendering performance

Wrapper script for agents:

#!/bin/bash
# client/test.sh - Godot test runner for agent use
godot --headless --path client/ \
  -s addons/gdUnit4/bin/GdUnitCmdTool.gd \
  --test-suite "${1:-test/}" \
  --report-format json \
  2>&1
exit $?

---

4. Integration Testing

IPC Protocol Tests (#210)

Recommendation: Both mock and real, layered.

Layer 1 — Serialization round-trip (mock, fast, in Rust + GDScript separately):

Rust:  ObserverSnapshot → MessagePack bytes → ObserverSnapshot (assert equality)
GDS:   MessagePack bytes → Dictionary → MessagePack bytes (assert equality)
Cross: Rust-serialized bytes → GDScript deserialize → assert field values

The cross-language test is the critical one. Run it by having Rust write test fixtures (MessagePack binary files), then GDScript tests read and verify them. No subprocess needed.

Layer 2 — Protocol sequence (mock subprocess): Test the protocol state machine: handshake → tick loop → snapshot delivery → input receipt → error recovery. Use a mock Rust process that sends predetermined sequences. This tests the GDScript LocalBridge without real simulation.

Layer 3 — Real subprocess (integration, slower): Launch actual simulation binary, send actual inputs, verify actual snapshots. This is the end-to-end truth. Run less frequently (not every edit cycle).

# test-integration.sh
# Layer 1: fixture-based (fast)
cargo test -p protocol --lib     # Rust serialization
godot --headless --path client/ -s ... --test-suite test/protocol/

# Layer 2: mock subprocess (medium)
cargo test -p protocol --test mock_bridge

# Layer 3: real subprocess (slow, nightly)
cargo test -p integration --test real_bridge -- --ignored

Sync Verification (#211) — Testing Determinism

This is the hardest testing problem in the project.

Deterministic simulation means: same seed + same input sequence = same state at every tick. Testing this requires:

1. Record & replay infrastructure (#201):

   • Record: capture (tick_number, input_event) pairs during a test run
   • Replay: feed the same inputs, compare state at each tick
   • Compare: serialize ECS world state to a canonical hash at each tick

2. Canonical state hashing:

   • Sort all entities deterministically (by entity ID or stable identifier)
   • Hash all component values in deterministic order
   • Compare hashes across runs

3. Known pitfalls that break determinism:

   • HashMap iteration order (use BTreeMap or sort before hashing)
   • Floating-point accumulation (prefer fixed-point or integer math for game logic)
   • System execution order (bevy schedules must be explicit, not ambiguous)
   • Entity allocation order after despawn/respawn cycles

4. Test pattern:

#[test]
fn deterministic_over_100_ticks() {
    let seed = 42u64;
    let inputs = load_test_inputs("basic_movement.inputs");

    let hash_1 = run_simulation(seed, &inputs, 100);
    let hash_2 = run_simulation(seed, &inputs, 100);

    assert_eq!(hash_1, hash_2, "Simulation diverged: non-deterministic behavior detected");
}

5. Regression strategy: Store golden hashes for known input sequences. If a code change alters a golden hash, the test fails. Developer must either fix the non-determinism or deliberately update the golden hash (with justification).

v0.1 scope: Record/replay for Rust simulation only. Godot client determinism is not required (it's a pure renderer — visual jitter doesn't affect game state).

Full Playthrough Automation (#212)

Is this realistic for v0.1? Partially.

What IS realistic:

• Scripted input sequences: "Walk north 10 tiles, wait 5 ticks, interact with NPC at (20, 15), select dialogue option 1." These exercise the full stack without AI.
• Smoke tests: Launch game, verify no crash after 60 seconds of simulated input. Verify NPC entities exist. Verify ObserverSnapshot contains expected fields.
• Milestone validation: "Start as smuggler. Walk to logistics hub. Verify hub NPCs are visible. Verify monologue triggers." These are semi-automated acceptance tests.

What is NOT realistic for v0.1:

• AI-driven playthroughs: We don't have an AI player that can navigate, make decisions, and evaluate outcomes.
• Full 30-minute playthroughs: Too slow for CI, too complex to script meaningfully.
• "Fun" automation: The D-027 success criteria include subjective measures (emotional attachment, emergent observation). These need human playtesters.

Recommendation: Build the scripted input infrastructure (#212) but scope it to smoke tests and milestone checkpoints, not full playthroughs. Full playthrough testing is manual playtesting with structured reports (see section 7).

---

5. Test Output Format

Recommendation: JUnit XML as interchange + JSON summaries for agents

Why not one format?

• JUnit XML is the universal CI standard. GitHub Actions, GitLab CI, Jenkins — they all render it natively. Both gdUnit4 and cargo-nextest can produce it. It's the right format for CI dashboards and historical tracking.
• JSON is what agents parse fastest. An agent reading JUnit XML needs to understand a slightly awkward schema. A simple JSON summary is immediately actionable.

Proposed wrapper output (what agents actually see):

{
  "suite": "simulation::perception",
  "timestamp": "2026-02-11T14:30:00Z",
  "duration_ms": 247,
  "total": 15,
  "passed": 14,
  "failed": 1,
  "skipped": 0,
  "failures": [
    {
      "test": "vision_cone_peripheral_range",
      "file": "simulation/src/perception/vision_cone.rs",
      "line": 142,
      "message": "assertion failed: `(left == right)`\n  left: 12\n right: 15",
      "expected": "peripheral range should be 15 tiles",
      "actual": "computed range was 12 tiles"
    }
  ]
}

Implementation: The wrapper scripts (test-rust.sh, test-godot.sh) run the native test runners, capture JUnit XML output, and post-process into this JSON summary format. This is a small Python or Rust CLI tool — ticket it under #216 (tooling requirements).

cargo-nextest is the recommended Rust test runner over plain cargo test:

• Runs tests in separate processes (better isolation)
• JUnit XML output with --message-format libtest-json (experimental) or nextest-run --junit-xml
• Faster parallel execution
• Better failure output formatting

---

6. Production Code Constraints (#217)

The principle: Test seams through architecture, not annotations

The D-020 architecture (subprocess/IPC, client-server separation) already creates the primary test seam: the protocol boundary. Anything that crosses that boundary is testable by mocking one side.

Rust/bevy_ecs: Naturally testable, minimal pollution needed

bevy_ecs is inherently test-friendly:

• Systems are functions. They take queries and resources as parameters. Inject different resources → different behavior. No special test hooks needed.
• Components are data. Spawn whatever entities you want in a test World. No factory pattern, no dependency injection framework.
• Resources are swappable. Replace Time with a test time. Replace SimRng with a seeded RNG. This is standard bevy, not test pollution.

Acceptable seams in Rust production code:

• SimRng resource wrapping RNG (required for determinism anyway — D-010 principle 4)
• SimulationTime resource wrapping time (required for determinism)
• SimBridge trait abstracting transport (required for multiplayer-ready architecture — D-010)
• #[derive(Debug, PartialEq)] on components (needed for assertions, zero runtime cost)
• Public visibility on types that tests need to construct (use pub(crate) not pub)

Unacceptable pollution:

• #[cfg(test)] conditional logic in production systems
• Test-only components in the production ECS
• Feature flags that alter simulation behavior for testing
• Mock traits wrapping concrete types just for testability

GDScript: Harder, needs clear boundaries

GDScript's scene tree model is less naturally testable than ECS. The key problems:

• Node lifecycle (_ready, _process) only runs inside the scene tree
• Signals are tied to the tree structure
• Autoloads create implicit global state

Strategy for testable GDScript:

1. Separate logic from nodes. Pure functions that take data and return data:

   # Good: testable without scene tree
   static func parse_observer_snapshot(data: PackedByteArray) -> Dictionary:
       return MessagePack.decode(data)
   
   # Bad: needs full scene tree
   func _on_snapshot_received():
       var data = bridge.read()
       update_entities(data)
       update_fog(data)
   

2. Data classes / Resources for state. GDScript Resource classes are serializable, inspectable, and don't need the scene tree.

3. Scene runner for integration tests. Use GdUnitSceneRunner when you MUST test node lifecycle — but keep these tests few and focused.

4. No game logic in GDScript. Per D-020, the client is a pure renderer. The less logic in GDScript, the less needs testing there. Push all game logic to Rust, where testing is natural.

The line: If you find yourself adding testability hooks to GDScript, you're probably putting logic in the wrong place. Move it to Rust and test it there.

---

7. Success Criteria Validation (#60 / D-027)

Criterion 1: "30-minute daily-life breathing room" (#196)

Can this be automated? Partially.

Automated test:

• Start simulation with test seed, smuggler character
• Advance simulation clock to 30 in-game minutes
• Assert: contamination module activation event has NOT fired
• Assert: at least N routine events have occurred (NPCs going to work, socializing)
• Assert: at least N monologue triggers have fired (character is experiencing daily life)

#[test]
fn contamination_does_not_activate_before_30_minutes() {
    let mut app = build_vertical_slice_app(Seed(42), Character::Smuggler);

    // Simulate 30 minutes of game time
    for _ in 0..(30 * 60 * TICKS_PER_SECOND) {
        app.update();
    }

    let storyteller = app.world().resource::<Storyteller>();
    assert!(!storyteller.contamination_activated(),
        "Contamination activated before 30-minute runway");
}

What automation CAN'T test: Whether those 30 minutes are interesting. That's a human playtest question. But we can verify the mechanical guarantee: the storyteller doesn't rush.

Playtest protocol needed: Structured form with timestamp markers: "At minute 5, what were you doing? At minute 15? At minute 25? Were you bored? When?"

Criterion 2: "Fundamentally different playthroughs" (#197)

Can this be automated? Yes, comparatively.

Run two simulations with the same seed but different characters. Compare:

Metric	Smuggler run	Detective run	Test assertion
Starting known NPCs	Set A	Set B	A != B (at least 50% different)
Starting known POIs	Set C	Set D	C != D
NPC relationship valences	Positive toward smuggling ring	Suspicious of smuggling ring	Inverted signs
Accessible dialogue lines (tick 0)	Insider lines available	Authority lines available	Non-overlapping access tiers
Monologue on same NPC	"Good old [name]"	"Subject of interest"	Different voice/content
Visible information (same location)	Knows cargo contents	Sees suspicious behavior	Different ObserverSnapshot fields

#[test]
fn smuggler_and_detective_see_different_worlds() {
    let seed = Seed(42);
    let smuggler_snapshot = run_initial_snapshot(seed, Character::Smuggler);
    let detective_snapshot = run_initial_snapshot(seed, Character::Detective);

    // Known NPCs should differ
    let overlap = smuggler_snapshot.known_npcs.intersection(&detective_snapshot.known_npcs);
    assert!(overlap.count() < smuggler_snapshot.known_npcs.len() / 2,
        "Characters know too many of the same NPCs — not divergent enough");

    // Relationship valences toward smuggling ring NPCs should be inverted
    for npc in smuggling_ring_npcs(seed) {
        let s_val = smuggler_snapshot.relationship_valence(npc);
        let d_val = detective_snapshot.relationship_valence(npc);
        assert!(s_val.signum() != d_val.signum(),
            "Smuggler and detective have same relationship valence toward {npc:?}");
    }
}

This is one of the most automatable success criteria. The divergence is structural, not subjective.

Criterion 3: "Emotional NPC attachment" (#198)

Can this be automated? No.

This is a subjective, emotional response. "Can you name an NPC you felt conflicted about?" requires a human brain that experienced narrative tension.

What we CAN automate as proxies:

• Verify NPCs have sufficient axis data to BE interesting (10 axes present, contradictions exist)
• Verify relationship dynamics change over time (not static)
• Verify monologue references NPCs by name with emotional coloring
• Verify dialogue layers unlock over time (surface → real → secret)
• Count unique NPC interactions per 30-minute session (minimum threshold)

What we MUST do manually:

• Post-playtest survey: "Name an NPC. What do you know about them? Did they surprise you?"
• Track if players remember NPC names unprompted
• Track if players hesitate before actions that affect specific NPCs

Playtest protocol: Minimum 5 playtesters (can be team members in different sessions). Structured debrief form. Key question: "Which NPC would you feel bad about betraying?"

Criterion 4: "Emergent observation mechanics" (#199)

Can this be automated? The chain can be verified; emergence cannot.

The required chain: observe → notice → follow → discover.

Automated verification:

• Observe: Player's ObserverSnapshot contains NPC entity at some tick. (Logged.)
• Notice: Monologue system fires a notice trigger about that NPC's behavior. (Logged event with cause chain.)
• Follow: Player moves toward NPC's location in subsequent ticks. (Input log analysis.)
• Discover: Player gains new information (information inventory changes). (State diff.)

#[test]
fn observation_chain_is_system_driven_not_scripted() {
    // Run simulation with scripted player input that follows an NPC
    let log = run_with_inputs(Seed(42), Character::Detective, &follow_suspicious_npc_inputs());

    // Verify the discovery event has a system-generated cause chain
    let discovery = log.events.iter()
        .find(|e| e.event_type == EventType::InformationGained)
        .expect("No discovery event occurred");

    // The cause chain should trace back to perception, not a script trigger
    assert!(!discovery.cause_chain.contains(&Cause::ScriptedReveal),
        "Discovery was scripted, not emergent");
    assert!(discovery.cause_chain.contains(&Cause::VisualObservation)
        || discovery.cause_chain.contains(&Cause::AudioObservation),
        "Discovery should originate from a perception event");
}

The "emergent" part is harder. Emergence means the sequence wasn't pre-authored. We can verify:

• No hard-coded "if player is at location X, trigger revelation Y"
• The NPC's suspicious behavior comes from their routine + secret, not a script
• The monologue notice comes from the monologue system's standard triggers, not a special case

Recommendation: Add a CauseChain component to information gain events. Every piece of new information tracks HOW it was learned. If any chain includes Cause::ScriptedReveal, that's a test failure for this criterion. This is a small production code addition that serves both debugging and criterion validation.

---

8. Cross-Initiative Testing Gaps

Systems NOT in the testability initiative (#34) that need test coverage:

Pathfinding (NOT IN ANY TICKET)

NPCs need to navigate. There's no pathfinding ticket anywhere in the 232-ticket catalog. This is a Track 3 gap, but from a testing perspective: pathfinding is one of the most testable systems (input: start + end + obstacles, output: path). We need the ticket before we can write tests.

Testing need: Path validity (no wall clipping), path optimality (no absurd routes), performance (A* on 150x150 within tick budget).

Collision Detection (NOT IN ANY TICKET)

Same gap. Player and NPC movement needs collision. Testable: entity at position A moves toward wall, verify entity stops.

Time System (Q-009 — UNRESOLVED)

Daily routines (#88) depend on a time system. The deterministic replay system (#201) depends on controllable time. We can't write tests for time-dependent behavior until Q-009 is resolved.

Testing need: Time advancement determinism, routine triggers at correct game-time, tier eviction timing (D-026).

Information Boundary Verification (#138-142)

These are in the Map & Navigation initiative, not testability. But information boundary correctness IS the most critical test target in the game — it's the core mechanic (D-010 principle 2, D-011).

Testing need: Exhaustive negative testing. "Entity X CANNOT see component Y of entity Z." Information leak = game-breaking bug. This should be the most heavily tested subsystem in the entire project.

Proposed test pattern:

#[test]
fn smuggler_cannot_see_detective_case_notes() {
    let world = setup_vertical_slice(Seed(42));
    let smuggler_snapshot = generate_snapshot(&world, Character::Smuggler);
    let detective = find_entity(&world, "detective_character");

    // Smuggler's snapshot should not contain detective's investigation data
    assert!(!smuggler_snapshot.has_component_data(detective, ComponentType::CaseNotes),
        "Information boundary violated: smuggler can see detective's case notes");
}

NPC Generation Validation (#92 / D-024)

NPC generation needs validation tests: do generated NPCs actually have all 10 axes? Do triangles form? Are contradictions present?

Testing need: Statistical validation over N generation runs. "Given seed X, do at least 2 triangles form per template? Does every NPC have a non-empty Secret?"

Dialogue System (#168-174 / D-028)

The line previewer (#193) is explicitly called out as a regression test harness. But we also need:

• Tag filtering tests: "Does an outsider character get insider dialogue?" (must fail)
• Layer progression tests: "After N positive interactions, does trust-gated content unlock?"
• Trait modifier tests: "Does the same base line read differently with different personality traits?"

Save/Load (NOT IN ANY TICKET)

Not testable until it exists, but flagging: save/load is a classic source of bugs. Serialization round-trip tests (save state → load state → compare) should be written the moment the system exists.

---

9. Priority Adjustments

Based on this analysis, I recommend the following priority changes:

Ticket	Current	Recommended	Reason
#214 (Refinement discussion)	Critical	Critical	Confirmed. This workshop IS the discussion. Unblocks everything else.
#201 (Deterministic replay)	High	Critical	Without determinism, sync tests (#211) and divergence tests (#197) are impossible. Blocks multiple success criteria.
#210 (IPC protocol tests)	High	High	Confirmed. Layer 1 (fixture-based) should be built alongside #76/#77.
#211 (Sync verification)	High	High	Confirmed, but blocked by #201.
#212 (Full playthrough automation)	Medium	Low for v0.1	Scope to smoke tests only. Full automation isn't realistic.
#213 (Performance profiling)	Low	Low	Confirmed. Nice to have, not blocking.
#208 (Rendering verification)	Medium	Low	Headless Godot can't meaningfully verify rendering. Defer to manual.
#207 (Input simulation)	Medium	High	Required for scripted smoke tests and integration testing.

New tickets needed (suggest to Si):

1. Pathfinding test plan — blocked by pathfinding implementation ticket (which doesn't exist)
2. Information boundary negative test suite — high priority, core mechanic validation
3. Test output formatter CLI — transforms JUnit XML to agent-friendly JSON
4. Playtest protocol definition — structured form for manual D-027 criteria validation
5. CauseChain component for information events — supports criterion 4 automation

---

10. Summary of Recommendations

Question	Recommendation
Rust test organization	Hybrid: inline #[cfg(test)] for unit tests, tests/ directory for integration
Rust test runner	cargo-nextest (parallel, isolated, JUnit XML output)
Godot test framework	gdUnit4 (better CLI, JSON output, scene runner, maintained)
IPC tests	Three layers: fixture (fast), mock subprocess (medium), real subprocess (slow)
Determinism testing	Record/replay with canonical state hashing. Promote #201 to critical.
Playthrough automation	Smoke tests only for v0.1. Full automation is premature.
Test output format	JUnit XML for CI + JSON summary for agents. Wrapper scripts handle conversion.
Production code constraints	Architecture provides seams naturally. No #[cfg(test)] in production. Move logic out of GDScript.
Criterion 1 (30-min runway)	Automated: verify no contamination before 30 min. Manual: verify it's interesting.
Criterion 2 (divergent plays)	Automated: compare snapshots across characters. Most automatable criterion.
Criterion 3 (emotional attachment)	Manual only. Automate proxies (axis presence, interaction count). Playtest protocol required.
Criterion 4 (emergent observation)	Semi-automated: verify cause chains are system-driven, not scripted. Add CauseChain component.

---

References

• Bevy System Testing Patterns
• gdUnit4 Repository
• GUT Command Line Documentation
• cargo-nextest Machine-Readable Output
• Automated Testing in Bevy