settled-reach/docs/workshops/test-architecture/justine-round2.md

Justine — Round 2: Content Validation Tooling + Performance Baseline

Workshop: QA Strategy & Test Architecture Track: 5 (Content Scaling & CI Pipeline) Date: 2026-02-17 Context: Lead decided: no Gitea Actions for now. Manual make ci stays. Focus shifts to local tooling: validation expansion, pre-PR checklist, perf baselines, golden file diffing.

---

Task 1: make validate-content Expansion — Cross-Reference Validation

Current State

tooling/validate-content is a Python script (133 lines) that:

• Walks content/campaigns/**/*.yaml
• Maps each file to a JSON Schema based on directory name or filename
• Validates structure via jsonschema.validate()
• Reports errors with path and field info
• Does NOT check cross-references between files

tooling/check-fact-ids is a bash script (90 lines) that:

• Extracts fact_id: values from knowledge catalogs (content/global/knowledge/)
• Extracts fact_id: references from campaign content
• Cross-checks references against canonical definitions
• Advisory mode when catalogs are unpopulated, enforcing mode when populated

Cross-Reference Inventory

After reviewing the content schemas and actual YAML files, here are all cross-reference relationships:

Source File Type	Field	References	Target File Type
NPC profile	relationships[].target	npc:{slug}	Other NPC profiles
NPC profile	triangle_membership[]	triangle slug	Triangle definitions
NPC profile	information.knows[]	fact_id string	Knowledge catalogs
Dialogue pool	location	location slug	Location definitions
Dialogue pool	role	template role slug	Template definitions
Dialogue pool	lines[].knowledge_grant.fact_id	fact_id string	Knowledge catalogs
Monologue pool	location	location slug	Location definitions (or "general")
Monologue pool	lines[].prerequisites.facts[].fact_id	fact_id string	Knowledge catalogs
Monologue pool	lines[].prerequisites.relationship.target	entity ref	NPC profiles
Monologue pool	lines[].prerequisites.entity_attributes[].entity	entity ref	NPC profiles
Triangle	members[].npc	npc:{slug}	NPC profiles
District	locations[]	location slug	Location definitions
Routine	location references	location slug	Location definitions

Recommendation: Extend the Python Validator

Extend tooling/validate-content, not a separate Rust step. Reasons:

1. The validator already loads and parses every YAML file. Adding cross-reference checks is O(1) additional passes over the same data.
2. Python is the right tool: string matching, file walking, error reporting. No compilation step.
3. check-fact-ids (bash) already handles fact_id validation. Absorb its logic into the Python validator to eliminate duplication and get consistent error reporting.
4. A Rust validation step would require building the server before validating content — that's a much heavier dependency chain. Content authors (copy team) should be able to validate without compiling Rust.

Implementation Design

Add a second pass to tooling/validate-content after schema validation:

# Phase 1: Schema validation (existing)
# Phase 2: Cross-reference validation (new)

class ContentIndex:
    """Builds an index of all defined entities for cross-referencing."""

    def __init__(self, content_dir: Path):
        self.npcs: set[str] = set()           # canonical_id values
        self.locations: set[str] = set()       # location slugs
        self.triangles: set[str] = set()       # triangle canonical_id values
        self.fact_ids: set[str] = set()        # from knowledge catalogs
        self.roles: set[str] = set()           # template role slugs
        self.district_locations: dict[str, list[str]] = {}  # district → listed locations

    def build(self):
        """Scan all content files and populate the index."""
        self._scan_npcs()
        self._scan_locations()
        self._scan_triangles()
        self._scan_knowledge()
        self._scan_districts()
        self._scan_templates()

    def validate_references(self) -> list[ValidationError]:
        """Check all cross-references against the index."""
        errors = []
        errors += self._check_npc_relationships()
        errors += self._check_npc_triangle_membership()
        errors += self._check_npc_fact_ids()
        errors += self._check_dialogue_locations()
        errors += self._check_dialogue_roles()
        errors += self._check_dialogue_fact_ids()
        errors += self._check_monologue_locations()
        errors += self._check_monologue_prerequisites()
        errors += self._check_triangle_members()
        errors += self._check_district_locations()
        return errors

Concrete Checks

Check 1: NPC relationship targets resolve

For each NPC profile:
  For each relationship in relationships[]:
    Assert relationship.target exists in npcs index
    Error: "npc:kael-davan references unknown NPC npc:nonexistent in relationships"

Check 2: Triangle members resolve

For each triangle:
  For each member in members[]:
    Assert member.npc exists in npcs index
    Error: "triangle hub-power references unknown NPC npc:missing"

Check 3: NPC triangle_membership matches triangle definitions

For each NPC profile:
  For each triangle_slug in triangle_membership[]:
    Assert triangle_slug exists in triangles index
    Error: "npc:kael-davan claims membership in unknown triangle 'missing-triangle'"

Check 4: Dialogue pool location resolves

For each dialogue pool:
  Assert pool.location exists in locations index
  Error: "dialogue pool kael-davan.yaml references unknown location 'nonexistent-bar'"

Check 5: Fact IDs resolve (absorb check-fact-ids)

For each fact_id reference (NPC knows[], dialogue knowledge_grant, monologue prerequisites):
  Assert fact_id exists in knowledge catalogs
  Error: "npc:kael-davan references unknown fact_id 'contraband.nonexistent'"
  (Advisory mode when catalogs are unpopulated, same as current check-fact-ids)

Check 6: District location list matches actual location files

For each district:
  For each location_slug in locations[]:
    Assert a location YAML exists at locations/{slug}.yaml
    Error: "district transit lists location 'ghost-alley' but no location file exists"
  Also: warn if location files exist that aren't listed in the district

Check 7: Bidirectional relationship consistency (WARNING, not ERROR)

For each NPC A with relationship to NPC B:
  Warn if NPC B has no relationship back to NPC A
  Warning: "npc:kael-davan has relationship to npc:naia-tamm but no reciprocal found"
  (This is a warning because asymmetric relationships may be intentional)

Makefile Change

validate-content:
	@tooling/validate-content

# Deprecate separate check-fact-ids once absorbed into validate-content
# Keep as alias for backward compatibility during transition
check-fact-ids:
	@tooling/check-fact-ids

After the Python validator absorbs fact_id checking, check-fact-ids becomes a thin wrapper that calls tooling/validate-content --fact-ids-only for the pre-commit hook (fast path, <2 seconds).

Phased Rollout

Phase	Checks Added	Timeline
1	NPC relationship targets, triangle members, district locations	First implementation ticket
2	Dialogue/monologue location + fact_id (absorb check-fact-ids)	Follow-up ticket
3	Bidirectional relationship warnings, role validation	Polish ticket

---

Task 2: make pre-pr Target

Since there's no automated CI, developers need a single command that runs the full verification suite before creating a PR. This replaces the discipline of "remember to run lint, build, test, and validate."

Design

# --- Pre-PR verification (replaces CI until automated pipeline exists) ---

pre-pr: pre-pr-lint pre-pr-build pre-pr-test pre-pr-validate pre-pr-fixtures
	@echo ""
	@echo "=== PRE-PR: ALL CHECKS PASSED ==="
	@echo "Safe to create PR."

pre-pr-lint: lint-server lint-client
	@echo "--- Lint: PASS ---"

pre-pr-build: build-server build-client
	@echo "--- Build: PASS ---"

pre-pr-test: test-server test-client
	@echo "--- Tests: PASS ---"

pre-pr-validate: validate-content
	@echo "--- Content validation: PASS ---"

pre-pr-fixtures:
	@echo "Checking fixture staleness..."
	@cd server && cargo test --test gen_fixtures -- --ignored 2>/dev/null
	@if git diff --quiet client/tests/fixtures/; then \
		echo "--- Fixtures: UP TO DATE ---"; \
	else \
		echo "--- Fixtures: STALE ---"; \
		echo "  Fixtures changed after regeneration. Commit the updated fixtures:"; \
		git diff --stat client/tests/fixtures/; \
		exit 1; \
	fi

Chain Order and Rationale

pre-pr
  ├── 1. lint-server + lint-client      (fastest, catch formatting/style)
  ├── 2. build-server + build-client    (catch compilation errors)
  ├── 3. test-server + test-client      (catch logic errors)
  ├── 4. validate-content               (catch content errors)
  └── 5. fixture staleness check        (catch protocol drift)

Order matters: Lint is fastest and catches the cheapest errors. Build must succeed before tests can run. Content validation is independent of build but runs after to keep the fast-fail path clean. Fixtures are last because they require a server build + test run.

Duration Budget

Step	Expected Duration	Notes
lint-server	~10s	clippy + fmt check
lint-client	~5s	headless Godot script check
build-server	~30-90s	incremental Rust build
build-client	~5s	Godot headless import
test-server	~15-30s	cargo nextest
test-client	~10-20s	gdUnit4 headless
validate-content	~2s	Python YAML walk
fixture check	~10-15s	build + gen_fixtures + git diff
Total	~90-180s	Under 3 minutes for clean incremental build

Failure Behavior

Each step uses Make's default behavior: fail-fast on non-zero exit. If lint-server fails, the chain stops immediately — no point building if there are lint errors.

Output on failure:

cd server && cargo clippy -- -D warnings
error: unused variable `x`
  --> src/simulation/movement.rs:42:9
make: *** [lint-server] Error 1

Output on success:

--- Lint: PASS ---
--- Build: PASS ---
--- Tests: PASS ---
--- Content validation: PASS ---
--- Fixtures: UP TO DATE ---

=== PRE-PR: ALL CHECKS PASSED ===
Safe to create PR.

What pre-pr Does NOT Do

• Does not run Layer 3 subprocess tests (too slow, nightly-tier)
• Does not run performance benchmarks (machine-dependent, separate target)
• Does not run Gauntlet golden file tests (depends on Gauntlet implementation)
• Does not push or create the PR (that's make pr-push or the /pr-push skill)

These are separate targets for developers who want deeper verification:

# Optional deeper checks (not part of pre-pr)
pre-pr-deep: pre-pr test-perf test-gauntlet
	@echo "=== DEEP VERIFICATION: ALL CHECKS PASSED ==="

---

Task 3: Local Performance Baseline Tooling

Design: make perf-baseline

Even without CI, developers need to track performance locally. The Gauntlet doesn't exist yet, but we can design the tooling now and wire it up when the Gauntlet lands.

Baseline File Format

tests/perf/baseline.json — checked into the repo:

{
  "_meta": {
    "format_version": 1,
    "updated_at": "2026-02-17T14:30:00Z",
    "updated_by": "developer-name",
    "commit": "abc123f",
    "machine": "workstation-01",
    "rust_version": "1.82.0",
    "build_profile": "release"
  },
  "benchmarks": {
    "gauntlet_100_ticks": {
      "median_ms": 142.3,
      "min_ms": 138.1,
      "max_ms": 156.7,
      "runs": 5,
      "description": "Gauntlet map, seed 42, 100 simulation ticks, release build"
    },
    "shadowcast_150x150_open": {
      "median_ms": 12.4,
      "min_ms": 11.8,
      "max_ms": 14.2,
      "runs": 5,
      "description": "Shadowcast benchmark, 150x150 open field, 1000 iterations"
    }
  }
}

Measurement Tooling

New script: tooling/perf-measure

#!/usr/bin/env bash
# Run performance benchmarks and compare against baseline.
# Usage: tooling/perf-measure [--update]
#   --update: write results as new baseline (otherwise compare only)

set -euo pipefail
REPO_ROOT="$(git rev-parse --show-toplevel)"
BASELINE="$REPO_ROOT/tests/perf/baseline.json"
RESULTS="$REPO_ROOT/.cache/perf_results.json"
THRESHOLD_WARN=15   # % regression = warning
THRESHOLD_FAIL=30   # % regression = failure

# Build release (perf measurements on debug builds are meaningless)
echo "Building server (release)..."
cd "$REPO_ROOT/server" && cargo build --release 2>/dev/null

# Run benchmark tests (--ignored = benchmark tests, --nocapture for timing output)
echo "Running benchmarks (5 iterations each)..."
cargo test --release --test gauntlet_perf -- --ignored --nocapture 2>&1 \
    | tee "$REPO_ROOT/.cache/perf_raw.txt"

# Parse results into JSON (perf test outputs structured timing data)
python3 "$REPO_ROOT/tooling/parse-perf-output" \
    "$REPO_ROOT/.cache/perf_raw.txt" > "$RESULTS"

# Compare against baseline
if [ -f "$BASELINE" ]; then
    python3 "$REPO_ROOT/tooling/compare-perf" \
        "$BASELINE" "$RESULTS" \
        --warn-threshold "$THRESHOLD_WARN" \
        --fail-threshold "$THRESHOLD_FAIL"
else
    echo "No baseline found at $BASELINE"
    echo "Run 'make perf-baseline-update' to create initial baseline."
fi

# Optionally update baseline
if [ "${1:-}" = "--update" ]; then
    cp "$RESULTS" "$BASELINE"
    echo "Baseline updated. Commit tests/perf/baseline.json to save."
fi

Rust-Side Benchmark Test

The benchmark test itself lives in server/tests/gauntlet_perf.rs:

#[test]
#[ignore] // Run with: cargo test --release --test gauntlet_perf -- --ignored
fn gauntlet_100_ticks_5_runs() {
    let mut times_ms: Vec<f64> = Vec::new();

    for run in 0..5 {
        let mut app = build_gauntlet_app(42); // seed 42

        let start = Instant::now();
        for _ in 0..100 {
            app.update();
        }
        let elapsed = start.elapsed().as_secs_f64() * 1000.0;
        times_ms.push(elapsed);
        eprintln!("PERF_RUN: gauntlet_100_ticks run={} ms={:.2}", run, elapsed);
    }

    times_ms.sort_by(|a, b| a.partial_cmp(b).unwrap());
    let median = times_ms[2]; // middle of 5
    let min = times_ms[0];
    let max = times_ms[4];

    eprintln!("PERF_RESULT: gauntlet_100_ticks median={:.2} min={:.2} max={:.2} runs=5",
        median, min, max);
}

Output format is structured for machine parsing:

PERF_RUN: gauntlet_100_ticks run=0 ms=141.23
PERF_RUN: gauntlet_100_ticks run=1 ms=138.12
PERF_RUN: gauntlet_100_ticks run=2 ms=142.87
PERF_RUN: gauntlet_100_ticks run=3 ms=156.71
PERF_RUN: gauntlet_100_ticks run=4 ms=139.44
PERF_RESULT: gauntlet_100_ticks median=141.23 min=138.12 max=156.71 runs=5

Human-Readable Comparison Output

=== Performance Comparison ===
Baseline: abc123f (2026-02-15) on workstation-01
Current:  def456g (2026-02-17) on workstation-01

Benchmark                    Baseline    Current     Delta    Status
─────────────────────────────────────────────────────────────────────
gauntlet_100_ticks           142.3ms     148.7ms     +4.5%    PASS
shadowcast_150x150_open       12.4ms      12.1ms     -2.4%    PASS (faster)

Overall: 2/2 PASS, 0 WARN, 0 FAIL

Warning example:

gauntlet_100_ticks           142.3ms     168.9ms    +18.7%    WARN ⚠
  Exceeds 15% threshold. Investigate recent changes.

Failure example:

gauntlet_100_ticks           142.3ms     203.1ms    +42.7%    FAIL ✗
  Exceeds 30% threshold. Likely regression.

Makefile Targets

# Performance benchmarks (local only — no CI yet)
perf-baseline:
	@tooling/perf-measure

perf-baseline-update:
	@tooling/perf-measure --update

perf-quick:
	cd server && cargo test --release --test shadowcast_bench -- --ignored --nocapture

Machine Variance Handling

Local measurements are inherently noisy. Mitigation:

• 5 runs, take median: Absorbs outliers from background processes
• Release builds only: Debug builds have 10-20x variance from unoptimized code paths
• Machine tag in baseline: Baseline records which machine it was measured on. Comparing across machines is meaningless — warn if machine tag differs
• 15% threshold: Generous enough for laptop variance (thermal throttling, background apps)
• Commit tag in baseline: Developers can see which commit the baseline was set against

---

Task 4: Golden File Diff Tooling

Design: make golden-diff

When the Gauntlet golden file changes, developers need to see what changed. The question: Rust test that prints the diff, or separate tool?

Recommendation: Rust Test That Prints the Diff

A Rust test, not a separate tool. Reasons:

1. The golden file is an ObserverSnapshot — a Rust struct. Rust code already knows how to deserialize and compare it field by field.
2. A Rust test naturally lives alongside the Gauntlet test suite. It's just a test that produces better output on failure.
3. No additional tooling dependency (no Python, no external diff tool).
4. The test can use #[derive(Debug)] to print readable struct output and custom assertion messages for each field.

Implementation: server/tests/gauntlet_golden.rs

//! Golden file comparison for the Gauntlet.
//! Generates actual snapshot, loads expected golden file, compares field by field.
//! On mismatch: prints structured diff to stderr.
//! Run with: cargo test --test gauntlet_golden -- --nocapture

use settled_reach_server::bridge::types::*;
use std::path::Path;

const GOLDEN_DIR: &str = "../tests/golden";

fn load_golden(name: &str) -> ObserverSnapshot {
    let path = Path::new(GOLDEN_DIR).join(format!("{}.json", name));
    let content = std::fs::read_to_string(&path)
        .unwrap_or_else(|_| panic!("Golden file not found: {}", path.display()));
    serde_json::from_str(&content)
        .unwrap_or_else(|e| panic!("Failed to parse golden file {}: {}", path.display(), e))
}

fn run_gauntlet_to_tick(tick_count: u64) -> ObserverSnapshot {
    // Build Gauntlet app, run N ticks, extract observer snapshot
    let mut app = build_gauntlet_app(42);
    for _ in 0..tick_count {
        app.update();
    }
    extract_observer_snapshot(&app)
}

fn diff_snapshots(golden: &ObserverSnapshot, actual: &ObserverSnapshot) -> Vec<String> {
    let mut diffs = Vec::new();

    if golden.tick != actual.tick {
        diffs.push(format!("  tick: {} → {}", golden.tick, actual.tick));
    }
    if golden.version != actual.version {
        diffs.push(format!("  version: {} → {}", golden.version, actual.version));
    }
    if golden.player_facing != actual.player_facing {
        diffs.push(format!("  player_facing: {:?} → {:?}",
            golden.player_facing, actual.player_facing));
    }

    // Entity comparison
    if golden.entities.len() != actual.entities.len() {
        diffs.push(format!("  entities.count: {} → {}",
            golden.entities.len(), actual.entities.len()));
    }

    let max_entities = golden.entities.len().max(actual.entities.len());
    for i in 0..max_entities {
        match (golden.entities.get(i), actual.entities.get(i)) {
            (Some(g), Some(a)) => {
                if g.entity_id != a.entity_id {
                    diffs.push(format!("  entities[{}].entity_id: {} → {}",
                        i, g.entity_id, a.entity_id));
                }
                if (g.x - a.x).abs() > 0.01 {
                    diffs.push(format!("  entities[{}].x: {} → {}  ← POSITION",
                        i, g.x, a.x));
                }
                if (g.y - a.y).abs() > 0.01 {
                    diffs.push(format!("  entities[{}].y: {} → {}  ← POSITION",
                        i, g.y, a.y));
                }
                if g.kind != a.kind {
                    diffs.push(format!("  entities[{}].kind: {:?} → {:?}",
                        i, g.kind, a.kind));
                }
                if g.visibility != a.visibility {
                    diffs.push(format!("  entities[{}].visibility: {:?} → {:?}  ← VISIBILITY",
                        i, g.visibility, a.visibility));
                }
                if g.relationship != a.relationship {
                    diffs.push(format!("  entities[{}].relationship: {:?} → {:?}",
                        i, g.relationship, a.relationship));
                }
            }
            (Some(g), None) => {
                diffs.push(format!("  REMOVED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}",
                    i, g.entity_id, g.kind, g.x, g.y));
            }
            (None, Some(a)) => {
                diffs.push(format!("  ADDED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}",
                    i, a.entity_id, a.kind, a.x, a.y));
            }
            _ => {}
        }
    }

    // Visible tiles summary (don't diff each tile — too noisy)
    if golden.visible_tiles.len() != actual.visible_tiles.len() {
        diffs.push(format!("  visible_tiles.count: {} → {}",
            golden.visible_tiles.len(), actual.visible_tiles.len()));
    }

    // Inventory
    if golden.player_inventory != actual.player_inventory {
        diffs.push(format!("  player_inventory: {:?} → {:?}",
            golden.player_inventory, actual.player_inventory));
    }

    // Monologue
    if golden.current_monologue != actual.current_monologue {
        diffs.push(format!("  current_monologue: {:?} → {:?}",
            golden.current_monologue, actual.current_monologue));
    }

    diffs
}

#[test]
fn gauntlet_tick_10_matches_golden() {
    let golden = load_golden("gauntlet_tick_10");
    let actual = run_gauntlet_to_tick(10);

    let diffs = diff_snapshots(&golden, &actual);
    if !diffs.is_empty() {
        eprintln!("\n=== Gauntlet Golden File Diff ===");
        eprintln!("Golden: tests/golden/gauntlet_tick_10.json");
        eprintln!("Seed: 42, Ticks: 10\n");
        eprintln!("CHANGED FIELDS:");
        for d in &diffs {
            eprintln!("{}", d);
        }
        // Count unchanged for context
        let total_fields = count_comparable_fields(&golden);
        eprintln!("\nUNCHANGED: {} of {} comparable fields",
            total_fields - diffs.len(), total_fields);
        eprintln!("\nTo update the golden file:");
        eprintln!("  make golden-update");
        eprintln!();
        panic!("Golden file mismatch: {} field(s) differ", diffs.len());
    }
}

Golden File Update Workflow

# Golden file operations
golden-diff:
	cd server && cargo test --test gauntlet_golden -- --nocapture 2>&1 || true

golden-update:
	cd server && cargo test --test gauntlet_golden_gen -- --ignored --nocapture
	@echo "Golden files updated in tests/golden/. Review and commit."
	@git diff --stat tests/golden/

The golden-update target:

1. Runs the Gauntlet with the canonical seed
2. Serializes the ObserverSnapshot as sorted JSON
3. Writes to tests/golden/gauntlet_tick_10.json
4. Prints a git diff summary so the developer can review

Golden File JSON Format

Sorted keys, pretty-printed, deterministic output:

{
  "current_monologue": null,
  "entities": [
    {
      "entity_id": 1,
      "kind": "Player",
      "observation": "Visible",
      "relationship": "Unknown",
      "visibility": "Forward",
      "x": 16.5,
      "y": 16.5,
      "z": 0
    },
    {
      "entity_id": 2,
      "kind": "Npc",
      "observation": "Visible",
      "relationship": "Unknown",
      "visibility": "Forward",
      "x": 18.0,
      "y": 10.0,
      "z": 0
    }
  ],
  "game_time": {
    "day": 0,
    "day_phase": "Morning",
    "tick_rate": "Full",
    "time_of_day": 1
  },
  "nearby_interactions": [],
  "pending_recognitions": [],
  "player_facing": "North",
  "player_inventory": [],
  "player_stance": "Walk",
  "tick": 10,
  "version": 7,
  "visible_tiles": [
    {"tile_kind": "Floor", "visibility": "Forward", "x": 15, "y": 15, "z": 0},
    {"tile_kind": "Floor", "visibility": "Forward", "x": 16, "y": 15, "z": 0}
  ]
}

Why JSON, not MessagePack:

• Human-readable in git diff
• Sorted keys = deterministic output regardless of struct field order
• Standard format — no custom tooling needed for basic inspection
• serde_json with #[serde(sort_maps)] + to_string_pretty() handles this natively

Why Not a Separate Python/Bash Tool?

A separate tooling/diff-golden script would need to:

1. Understand the ObserverSnapshot structure (duplicating the Rust type definitions)
2. Parse JSON with knowledge of which fields are positions (float tolerance), which are counts, etc.
3. Be kept in sync as the snapshot format evolves

The Rust test already has the type definitions, already knows the structure, and can use #[derive(PartialEq, Debug)] for free. A Rust test with structured eprintln! output is the simplest correct solution.

For developers who want a quick terminal diff without running the full test:

# Just see what changed (raw JSON diff)
diff tests/golden/gauntlet_tick_10.json .cache/gauntlet_actual.json

This is the fallback — not as pretty as the structured diff, but always available.

---

Answering Hoshe's Open Question (T5-H5)

For Justine: Fixture staleness check in CI (make fixtures && git diff --exit-code) — is this robust enough, or do we need a content-addressed hash approach?

git diff --exit-code is robust enough. Reasons:

1. It detects any byte-level change in the fixtures directory. Content-addressed hashing would catch the same thing with more complexity.
2. git diff already handles binary files (MessagePack fixtures are binary). It won't show a meaningful diff, but it will detect changes.
3. The pre-PR workflow is: regenerate fixtures, check for diff, fail if stale. This is deterministic — same source produces same fixtures.
4. Content-addressed hashing adds a build step (compute hashes, store/compare) for zero additional safety.

One caveat: The fixture regeneration must be deterministic. If gen_fixtures.rs uses timestamps or random values, the diff will always show changes. Current code uses fixed values — verified.

The implementation in pre-pr-fixtures above handles this correctly.

---

Summary of Deliverables

Deliverable	Status
validate-content expansion design (7 cross-reference checks)	Complete
make pre-pr target specification (5-step chain, ~3 min)	Complete
make perf-baseline tooling design (median-of-5, relative delta)	Complete
Golden file diff design (Rust test, structured output)	Complete
Hoshe Q5 response (fixture staleness check)	Complete

Dependencies on Other Tracks

Dependency	Owner	What I Need
Gauntlet implementation	Dudley/Tyre	build_gauntlet_app() function for perf + golden file tests
serde_json derive on ObserverSnapshot	Dudley	#[derive(Serialize, Deserialize)] for JSON golden files (currently only rmp_serde)
Content validation phase 2 enums	Hoshe	Agreed list of valid location slugs, role slugs, and enum values for cross-reference validation