settled-reach/docs/workshops/test-architecture/justine-round2.md

# Justine — Round 2: Content Validation Tooling + Performance Baseline

**Workshop:** QA Strategy & Test Architecture
**Track:** 5 (Content Scaling & CI Pipeline)
**Date:** 2026-02-17
**Context:** Lead decided: no Gitea Actions for now. Manual `make ci` stays. Focus shifts to local tooling: validation expansion, pre-PR checklist, perf baselines, golden file diffing.

---

## Task 1: `make validate-content` Expansion — Cross-Reference Validation

### Current State

`tooling/validate-content` is a Python script (133 lines) that:
- Walks `content/campaigns/**/*.yaml`
- Maps each file to a JSON Schema based on directory name or filename
- Validates structure via `jsonschema.validate()`
- Reports errors with path and field info
- Does NOT check cross-references between files

`tooling/check-fact-ids` is a bash script (90 lines) that:
- Extracts `fact_id:` values from knowledge catalogs (`content/global/knowledge/`)
- Extracts `fact_id:` references from campaign content
- Cross-checks references against canonical definitions
- Advisory mode when catalogs are unpopulated, enforcing mode when populated

### Cross-Reference Inventory

After reviewing the content schemas and actual YAML files, here are all cross-reference relationships:

| Source File Type | Field | References | Target File Type |
|-----------------|-------|------------|-----------------|
| NPC profile | `relationships[].target` | `npc:{slug}` | Other NPC profiles |
| NPC profile | `triangle_membership[]` | triangle slug | Triangle definitions |
| NPC profile | `information.knows[]` | fact_id string | Knowledge catalogs |
| Dialogue pool | `location` | location slug | Location definitions |
| Dialogue pool | `role` | template role slug | Template definitions |
| Dialogue pool | `lines[].knowledge_grant.fact_id` | fact_id string | Knowledge catalogs |
| Monologue pool | `location` | location slug | Location definitions (or "general") |
| Monologue pool | `lines[].prerequisites.facts[].fact_id` | fact_id string | Knowledge catalogs |
| Monologue pool | `lines[].prerequisites.relationship.target` | entity ref | NPC profiles |
| Monologue pool | `lines[].prerequisites.entity_attributes[].entity` | entity ref | NPC profiles |
| Triangle | `members[].npc` | `npc:{slug}` | NPC profiles |
| District | `locations[]` | location slug | Location definitions |
| Routine | location references | location slug | Location definitions |

### Recommendation: Extend the Python Validator

**Extend `tooling/validate-content`, not a separate Rust step.** Reasons:

1. The validator already loads and parses every YAML file. Adding cross-reference checks is O(1) additional passes over the same data.
2. Python is the right tool: string matching, file walking, error reporting. No compilation step.
3. `check-fact-ids` (bash) already handles fact_id validation. Absorb its logic into the Python validator to eliminate duplication and get consistent error reporting.
4. A Rust validation step would require building the server before validating content — that's a much heavier dependency chain. Content authors (copy team) should be able to validate without compiling Rust.

### Implementation Design

Add a second pass to `tooling/validate-content` after schema validation:

```python
# Phase 1: Schema validation (existing)
# Phase 2: Cross-reference validation (new)

class ContentIndex:
    """Builds an index of all defined entities for cross-referencing."""

    def __init__(self, content_dir: Path):
        self.npcs: set[str] = set()           # canonical_id values
        self.locations: set[str] = set()       # location slugs
        self.triangles: set[str] = set()       # triangle canonical_id values
        self.fact_ids: set[str] = set()        # from knowledge catalogs
        self.roles: set[str] = set()           # template role slugs
        self.district_locations: dict[str, list[str]] = {}  # district → listed locations

    def build(self):
        """Scan all content files and populate the index."""
        self._scan_npcs()
        self._scan_locations()
        self._scan_triangles()
        self._scan_knowledge()
        self._scan_districts()
        self._scan_templates()

    def validate_references(self) -> list[ValidationError]:
        """Check all cross-references against the index."""
        errors = []
        errors += self._check_npc_relationships()
        errors += self._check_npc_triangle_membership()
        errors += self._check_npc_fact_ids()
        errors += self._check_dialogue_locations()
        errors += self._check_dialogue_roles()
        errors += self._check_dialogue_fact_ids()
        errors += self._check_monologue_locations()
        errors += self._check_monologue_prerequisites()
        errors += self._check_triangle_members()
        errors += self._check_district_locations()
        return errors
```

### Concrete Checks

**Check 1: NPC relationship targets resolve**
```
For each NPC profile:
  For each relationship in relationships[]:
    Assert relationship.target exists in npcs index
    Error: "npc:kael-davan references unknown NPC npc:nonexistent in relationships"
```

**Check 2: Triangle members resolve**
```
For each triangle:
  For each member in members[]:
    Assert member.npc exists in npcs index
    Error: "triangle hub-power references unknown NPC npc:missing"
```

**Check 3: NPC triangle_membership matches triangle definitions**
```
For each NPC profile:
  For each triangle_slug in triangle_membership[]:
    Assert triangle_slug exists in triangles index
    Error: "npc:kael-davan claims membership in unknown triangle 'missing-triangle'"
```

**Check 4: Dialogue pool location resolves**
```
For each dialogue pool:
  Assert pool.location exists in locations index
  Error: "dialogue pool kael-davan.yaml references unknown location 'nonexistent-bar'"
```

**Check 5: Fact IDs resolve (absorb check-fact-ids)**
```
For each fact_id reference (NPC knows[], dialogue knowledge_grant, monologue prerequisites):
  Assert fact_id exists in knowledge catalogs
  Error: "npc:kael-davan references unknown fact_id 'contraband.nonexistent'"
  (Advisory mode when catalogs are unpopulated, same as current check-fact-ids)
```

**Check 6: District location list matches actual location files**
```
For each district:
  For each location_slug in locations[]:
    Assert a location YAML exists at locations/{slug}.yaml
    Error: "district transit lists location 'ghost-alley' but no location file exists"
  Also: warn if location files exist that aren't listed in the district
```

**Check 7: Bidirectional relationship consistency (WARNING, not ERROR)**
```
For each NPC A with relationship to NPC B:
  Warn if NPC B has no relationship back to NPC A
  Warning: "npc:kael-davan has relationship to npc:naia-tamm but no reciprocal found"
  (This is a warning because asymmetric relationships may be intentional)
```

### Makefile Change

```makefile
validate-content:
	@tooling/validate-content

# Deprecate separate check-fact-ids once absorbed into validate-content
# Keep as alias for backward compatibility during transition
check-fact-ids:
	@tooling/check-fact-ids
```

After the Python validator absorbs fact_id checking, `check-fact-ids` becomes a thin wrapper that calls `tooling/validate-content --fact-ids-only` for the pre-commit hook (fast path, <2 seconds).

### Phased Rollout

| Phase | Checks Added | Timeline |
|-------|-------------|----------|
| 1 | NPC relationship targets, triangle members, district locations | First implementation ticket |
| 2 | Dialogue/monologue location + fact_id (absorb check-fact-ids) | Follow-up ticket |
| 3 | Bidirectional relationship warnings, role validation | Polish ticket |

---

## Task 2: `make pre-pr` Target

Since there's no automated CI, developers need a single command that runs the full verification suite before creating a PR. This replaces the discipline of "remember to run lint, build, test, and validate."

### Design

```makefile
# --- Pre-PR verification (replaces CI until automated pipeline exists) ---

pre-pr: pre-pr-lint pre-pr-build pre-pr-test pre-pr-validate pre-pr-fixtures
	@echo ""
	@echo "=== PRE-PR: ALL CHECKS PASSED ==="
	@echo "Safe to create PR."

pre-pr-lint: lint-server lint-client
	@echo "--- Lint: PASS ---"

pre-pr-build: build-server build-client
	@echo "--- Build: PASS ---"

pre-pr-test: test-server test-client
	@echo "--- Tests: PASS ---"

pre-pr-validate: validate-content
	@echo "--- Content validation: PASS ---"

pre-pr-fixtures:
	@echo "Checking fixture staleness..."
	@cd server && cargo test --test gen_fixtures -- --ignored 2>/dev/null
	@if git diff --quiet client/tests/fixtures/; then \
		echo "--- Fixtures: UP TO DATE ---"; \
	else \
		echo "--- Fixtures: STALE ---"; \
		echo "  Fixtures changed after regeneration. Commit the updated fixtures:"; \
		git diff --stat client/tests/fixtures/; \
		exit 1; \
	fi
```

### Chain Order and Rationale

```
pre-pr
  ├── 1. lint-server + lint-client      (fastest, catch formatting/style)
  ├── 2. build-server + build-client    (catch compilation errors)
  ├── 3. test-server + test-client      (catch logic errors)
  ├── 4. validate-content               (catch content errors)
  └── 5. fixture staleness check        (catch protocol drift)
```

**Order matters:** Lint is fastest and catches the cheapest errors. Build must succeed before tests can run. Content validation is independent of build but runs after to keep the fast-fail path clean. Fixtures are last because they require a server build + test run.

### Duration Budget

| Step | Expected Duration | Notes |
|------|------------------|-------|
| lint-server | ~10s | clippy + fmt check |
| lint-client | ~5s | headless Godot script check |
| build-server | ~30-90s | incremental Rust build |
| build-client | ~5s | Godot headless import |
| test-server | ~15-30s | cargo nextest |
| test-client | ~10-20s | gdUnit4 headless |
| validate-content | ~2s | Python YAML walk |
| fixture check | ~10-15s | build + gen_fixtures + git diff |
| **Total** | **~90-180s** | Under 3 minutes for clean incremental build |

### Failure Behavior

Each step uses Make's default behavior: fail-fast on non-zero exit. If `lint-server` fails, the chain stops immediately — no point building if there are lint errors.

Output on failure:
```
cd server && cargo clippy -- -D warnings
error: unused variable `x`
  --> src/simulation/movement.rs:42:9
make: *** [lint-server] Error 1
```

Output on success:
```
--- Lint: PASS ---
--- Build: PASS ---
--- Tests: PASS ---
--- Content validation: PASS ---
--- Fixtures: UP TO DATE ---

=== PRE-PR: ALL CHECKS PASSED ===
Safe to create PR.
```

### What pre-pr Does NOT Do

- Does not run Layer 3 subprocess tests (too slow, nightly-tier)
- Does not run performance benchmarks (machine-dependent, separate target)
- Does not run Gauntlet golden file tests (depends on Gauntlet implementation)
- Does not push or create the PR (that's `make pr-push` or the `/pr-push` skill)

These are separate targets for developers who want deeper verification:

```makefile
# Optional deeper checks (not part of pre-pr)
pre-pr-deep: pre-pr test-perf test-gauntlet
	@echo "=== DEEP VERIFICATION: ALL CHECKS PASSED ==="
```

---

## Task 3: Local Performance Baseline Tooling

### Design: `make perf-baseline`

Even without CI, developers need to track performance locally. The Gauntlet doesn't exist yet, but we can design the tooling now and wire it up when the Gauntlet lands.

#### Baseline File Format

`tests/perf/baseline.json` — checked into the repo:

```json
{
  "_meta": {
    "format_version": 1,
    "updated_at": "2026-02-17T14:30:00Z",
    "updated_by": "developer-name",
    "commit": "abc123f",
    "machine": "workstation-01",
    "rust_version": "1.82.0",
    "build_profile": "release"
  },
  "benchmarks": {
    "gauntlet_100_ticks": {
      "median_ms": 142.3,
      "min_ms": 138.1,
      "max_ms": 156.7,
      "runs": 5,
      "description": "Gauntlet map, seed 42, 100 simulation ticks, release build"
    },
    "shadowcast_150x150_open": {
      "median_ms": 12.4,
      "min_ms": 11.8,
      "max_ms": 14.2,
      "runs": 5,
      "description": "Shadowcast benchmark, 150x150 open field, 1000 iterations"
    }
  }
}
```

#### Measurement Tooling

New script: `tooling/perf-measure`

```bash
#!/usr/bin/env bash
# Run performance benchmarks and compare against baseline.
# Usage: tooling/perf-measure [--update]
#   --update: write results as new baseline (otherwise compare only)

set -euo pipefail
REPO_ROOT="$(git rev-parse --show-toplevel)"
BASELINE="$REPO_ROOT/tests/perf/baseline.json"
RESULTS="$REPO_ROOT/.cache/perf_results.json"
THRESHOLD_WARN=15   # % regression = warning
THRESHOLD_FAIL=30   # % regression = failure

# Build release (perf measurements on debug builds are meaningless)
echo "Building server (release)..."
cd "$REPO_ROOT/server" && cargo build --release 2>/dev/null

# Run benchmark tests (--ignored = benchmark tests, --nocapture for timing output)
echo "Running benchmarks (5 iterations each)..."
cargo test --release --test gauntlet_perf -- --ignored --nocapture 2>&1 \
    | tee "$REPO_ROOT/.cache/perf_raw.txt"

# Parse results into JSON (perf test outputs structured timing data)
python3 "$REPO_ROOT/tooling/parse-perf-output" \
    "$REPO_ROOT/.cache/perf_raw.txt" > "$RESULTS"

# Compare against baseline
if [ -f "$BASELINE" ]; then
    python3 "$REPO_ROOT/tooling/compare-perf" \
        "$BASELINE" "$RESULTS" \
        --warn-threshold "$THRESHOLD_WARN" \
        --fail-threshold "$THRESHOLD_FAIL"
else
    echo "No baseline found at $BASELINE"
    echo "Run 'make perf-baseline-update' to create initial baseline."
fi

# Optionally update baseline
if [ "${1:-}" = "--update" ]; then
    cp "$RESULTS" "$BASELINE"
    echo "Baseline updated. Commit tests/perf/baseline.json to save."
fi
```

#### Rust-Side Benchmark Test

The benchmark test itself lives in `server/tests/gauntlet_perf.rs`:

```rust
#[test]
#[ignore] // Run with: cargo test --release --test gauntlet_perf -- --ignored
fn gauntlet_100_ticks_5_runs() {
    let mut times_ms: Vec<f64> = Vec::new();

    for run in 0..5 {
        let mut app = build_gauntlet_app(42); // seed 42

        let start = Instant::now();
        for _ in 0..100 {
            app.update();
        }
        let elapsed = start.elapsed().as_secs_f64() * 1000.0;
        times_ms.push(elapsed);
        eprintln!("PERF_RUN: gauntlet_100_ticks run={} ms={:.2}", run, elapsed);
    }

    times_ms.sort_by(|a, b| a.partial_cmp(b).unwrap());
    let median = times_ms[2]; // middle of 5
    let min = times_ms[0];
    let max = times_ms[4];

    eprintln!("PERF_RESULT: gauntlet_100_ticks median={:.2} min={:.2} max={:.2} runs=5",
        median, min, max);
}
```

Output format is structured for machine parsing:
```
PERF_RUN: gauntlet_100_ticks run=0 ms=141.23
PERF_RUN: gauntlet_100_ticks run=1 ms=138.12
PERF_RUN: gauntlet_100_ticks run=2 ms=142.87
PERF_RUN: gauntlet_100_ticks run=3 ms=156.71
PERF_RUN: gauntlet_100_ticks run=4 ms=139.44
PERF_RESULT: gauntlet_100_ticks median=141.23 min=138.12 max=156.71 runs=5
```

#### Human-Readable Comparison Output

```
=== Performance Comparison ===
Baseline: abc123f (2026-02-15) on workstation-01
Current:  def456g (2026-02-17) on workstation-01

Benchmark                    Baseline    Current     Delta    Status
─────────────────────────────────────────────────────────────────────
gauntlet_100_ticks           142.3ms     148.7ms     +4.5%    PASS
shadowcast_150x150_open       12.4ms      12.1ms     -2.4%    PASS (faster)

Overall: 2/2 PASS, 0 WARN, 0 FAIL
```

Warning example:
```
gauntlet_100_ticks           142.3ms     168.9ms    +18.7%    WARN ⚠
  Exceeds 15% threshold. Investigate recent changes.
```

Failure example:
```
gauntlet_100_ticks           142.3ms     203.1ms    +42.7%    FAIL ✗
  Exceeds 30% threshold. Likely regression.
```

#### Makefile Targets

```makefile
# Performance benchmarks (local only — no CI yet)
perf-baseline:
	@tooling/perf-measure

perf-baseline-update:
	@tooling/perf-measure --update

perf-quick:
	cd server && cargo test --release --test shadowcast_bench -- --ignored --nocapture
```

#### Machine Variance Handling

Local measurements are inherently noisy. Mitigation:
- **5 runs, take median**: Absorbs outliers from background processes
- **Release builds only**: Debug builds have 10-20x variance from unoptimized code paths
- **Machine tag in baseline**: Baseline records which machine it was measured on. Comparing across machines is meaningless — warn if machine tag differs
- **15% threshold**: Generous enough for laptop variance (thermal throttling, background apps)
- **Commit tag in baseline**: Developers can see which commit the baseline was set against

---

## Task 4: Golden File Diff Tooling

### Design: `make golden-diff`

When the Gauntlet golden file changes, developers need to see what changed. The question: Rust test that prints the diff, or separate tool?

### Recommendation: Rust Test That Prints the Diff

**A Rust test, not a separate tool.** Reasons:

1. The golden file is an `ObserverSnapshot` — a Rust struct. Rust code already knows how to deserialize and compare it field by field.
2. A Rust test naturally lives alongside the Gauntlet test suite. It's just a test that produces better output on failure.
3. No additional tooling dependency (no Python, no external diff tool).
4. The test can use `#[derive(Debug)]` to print readable struct output and custom assertion messages for each field.

### Implementation: `server/tests/gauntlet_golden.rs`

```rust
//! Golden file comparison for the Gauntlet.
//! Generates actual snapshot, loads expected golden file, compares field by field.
//! On mismatch: prints structured diff to stderr.
//! Run with: cargo test --test gauntlet_golden -- --nocapture

use settled_reach_server::bridge::types::*;
use std::path::Path;

const GOLDEN_DIR: &str = "../tests/golden";

fn load_golden(name: &str) -> ObserverSnapshot {
    let path = Path::new(GOLDEN_DIR).join(format!("{}.json", name));
    let content = std::fs::read_to_string(&path)
        .unwrap_or_else(|_| panic!("Golden file not found: {}", path.display()));
    serde_json::from_str(&content)
        .unwrap_or_else(|e| panic!("Failed to parse golden file {}: {}", path.display(), e))
}

fn run_gauntlet_to_tick(tick_count: u64) -> ObserverSnapshot {
    // Build Gauntlet app, run N ticks, extract observer snapshot
    let mut app = build_gauntlet_app(42);
    for _ in 0..tick_count {
        app.update();
    }
    extract_observer_snapshot(&app)
}

fn diff_snapshots(golden: &ObserverSnapshot, actual: &ObserverSnapshot) -> Vec<String> {
    let mut diffs = Vec::new();

    if golden.tick != actual.tick {
        diffs.push(format!("  tick: {} → {}", golden.tick, actual.tick));
    }
    if golden.version != actual.version {
        diffs.push(format!("  version: {} → {}", golden.version, actual.version));
    }
    if golden.player_facing != actual.player_facing {
        diffs.push(format!("  player_facing: {:?} → {:?}",
            golden.player_facing, actual.player_facing));
    }

    // Entity comparison
    if golden.entities.len() != actual.entities.len() {
        diffs.push(format!("  entities.count: {} → {}",
            golden.entities.len(), actual.entities.len()));
    }

    let max_entities = golden.entities.len().max(actual.entities.len());
    for i in 0..max_entities {
        match (golden.entities.get(i), actual.entities.get(i)) {
            (Some(g), Some(a)) => {
                if g.entity_id != a.entity_id {
                    diffs.push(format!("  entities[{}].entity_id: {} → {}",
                        i, g.entity_id, a.entity_id));
                }
                if (g.x - a.x).abs() > 0.01 {
                    diffs.push(format!("  entities[{}].x: {} → {}  ← POSITION",
                        i, g.x, a.x));
                }
                if (g.y - a.y).abs() > 0.01 {
                    diffs.push(format!("  entities[{}].y: {} → {}  ← POSITION",
                        i, g.y, a.y));
                }
                if g.kind != a.kind {
                    diffs.push(format!("  entities[{}].kind: {:?} → {:?}",
                        i, g.kind, a.kind));
                }
                if g.visibility != a.visibility {
                    diffs.push(format!("  entities[{}].visibility: {:?} → {:?}  ← VISIBILITY",
                        i, g.visibility, a.visibility));
                }
                if g.relationship != a.relationship {
                    diffs.push(format!("  entities[{}].relationship: {:?} → {:?}",
                        i, g.relationship, a.relationship));
                }
            }
            (Some(g), None) => {
                diffs.push(format!("  REMOVED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}",
                    i, g.entity_id, g.kind, g.x, g.y));
            }
            (None, Some(a)) => {
                diffs.push(format!("  ADDED: entities[{}] {{ id: {}, kind: {:?}, pos: ({}, {}) }}",
                    i, a.entity_id, a.kind, a.x, a.y));
            }
            _ => {}
        }
    }

    // Visible tiles summary (don't diff each tile — too noisy)
    if golden.visible_tiles.len() != actual.visible_tiles.len() {
        diffs.push(format!("  visible_tiles.count: {} → {}",
            golden.visible_tiles.len(), actual.visible_tiles.len()));
    }

    // Inventory
    if golden.player_inventory != actual.player_inventory {
        diffs.push(format!("  player_inventory: {:?} → {:?}",
            golden.player_inventory, actual.player_inventory));
    }

    // Monologue
    if golden.current_monologue != actual.current_monologue {
        diffs.push(format!("  current_monologue: {:?} → {:?}",
            golden.current_monologue, actual.current_monologue));
    }

    diffs
}

#[test]
fn gauntlet_tick_10_matches_golden() {
    let golden = load_golden("gauntlet_tick_10");
    let actual = run_gauntlet_to_tick(10);

    let diffs = diff_snapshots(&golden, &actual);
    if !diffs.is_empty() {
        eprintln!("\n=== Gauntlet Golden File Diff ===");
        eprintln!("Golden: tests/golden/gauntlet_tick_10.json");
        eprintln!("Seed: 42, Ticks: 10\n");
        eprintln!("CHANGED FIELDS:");
        for d in &diffs {
            eprintln!("{}", d);
        }
        // Count unchanged for context
        let total_fields = count_comparable_fields(&golden);
        eprintln!("\nUNCHANGED: {} of {} comparable fields",
            total_fields - diffs.len(), total_fields);
        eprintln!("\nTo update the golden file:");
        eprintln!("  make golden-update");
        eprintln!();
        panic!("Golden file mismatch: {} field(s) differ", diffs.len());
    }
}
```

### Golden File Update Workflow

```makefile
# Golden file operations
golden-diff:
	cd server && cargo test --test gauntlet_golden -- --nocapture 2>&1 || true

golden-update:
	cd server && cargo test --test gauntlet_golden_gen -- --ignored --nocapture
	@echo "Golden files updated in tests/golden/. Review and commit."
	@git diff --stat tests/golden/
```

The `golden-update` target:
1. Runs the Gauntlet with the canonical seed
2. Serializes the ObserverSnapshot as sorted JSON
3. Writes to `tests/golden/gauntlet_tick_10.json`
4. Prints a git diff summary so the developer can review

### Golden File JSON Format

Sorted keys, pretty-printed, deterministic output:

```json
{
  "current_monologue": null,
  "entities": [
    {
      "entity_id": 1,
      "kind": "Player",
      "observation": "Visible",
      "relationship": "Unknown",
      "visibility": "Forward",
      "x": 16.5,
      "y": 16.5,
      "z": 0
    },
    {
      "entity_id": 2,
      "kind": "Npc",
      "observation": "Visible",
      "relationship": "Unknown",
      "visibility": "Forward",
      "x": 18.0,
      "y": 10.0,
      "z": 0
    }
  ],
  "game_time": {
    "day": 0,
    "day_phase": "Morning",
    "tick_rate": "Full",
    "time_of_day": 1
  },
  "nearby_interactions": [],
  "pending_recognitions": [],
  "player_facing": "North",
  "player_inventory": [],
  "player_stance": "Walk",
  "tick": 10,
  "version": 7,
  "visible_tiles": [
    {"tile_kind": "Floor", "visibility": "Forward", "x": 15, "y": 15, "z": 0},
    {"tile_kind": "Floor", "visibility": "Forward", "x": 16, "y": 15, "z": 0}
  ]
}
```

Why JSON, not MessagePack:
- Human-readable in `git diff`
- Sorted keys = deterministic output regardless of struct field order
- Standard format — no custom tooling needed for basic inspection
- `serde_json` with `#[serde(sort_maps)]` + `to_string_pretty()` handles this natively

### Why Not a Separate Python/Bash Tool?

A separate `tooling/diff-golden` script would need to:
1. Understand the ObserverSnapshot structure (duplicating the Rust type definitions)
2. Parse JSON with knowledge of which fields are positions (float tolerance), which are counts, etc.
3. Be kept in sync as the snapshot format evolves

The Rust test already has the type definitions, already knows the structure, and can use `#[derive(PartialEq, Debug)]` for free. A Rust test with structured `eprintln!` output is the simplest correct solution.

For developers who want a quick terminal diff without running the full test:
```bash
# Just see what changed (raw JSON diff)
diff tests/golden/gauntlet_tick_10.json .cache/gauntlet_actual.json
```

This is the fallback — not as pretty as the structured diff, but always available.

---

## Answering Hoshe's Open Question (T5-H5)

> **For Justine:** Fixture staleness check in CI (`make fixtures && git diff --exit-code`) — is this robust enough, or do we need a content-addressed hash approach?

**`git diff --exit-code` is robust enough.** Reasons:

1. It detects any byte-level change in the fixtures directory. Content-addressed hashing would catch the same thing with more complexity.
2. `git diff` already handles binary files (MessagePack fixtures are binary). It won't show a meaningful diff, but it will detect changes.
3. The pre-PR workflow is: regenerate fixtures, check for diff, fail if stale. This is deterministic — same source produces same fixtures.
4. Content-addressed hashing adds a build step (compute hashes, store/compare) for zero additional safety.

**One caveat:** The fixture regeneration must be deterministic. If `gen_fixtures.rs` uses timestamps or random values, the diff will always show changes. Current code uses fixed values — verified.

The implementation in `pre-pr-fixtures` above handles this correctly.

---

## Summary of Deliverables

| Deliverable | Status |
|-------------|--------|
| `validate-content` expansion design (7 cross-reference checks) | Complete |
| `make pre-pr` target specification (5-step chain, ~3 min) | Complete |
| `make perf-baseline` tooling design (median-of-5, relative delta) | Complete |
| Golden file diff design (Rust test, structured output) | Complete |
| Hoshe Q5 response (fixture staleness check) | Complete |

## Dependencies on Other Tracks

| Dependency | Owner | What I Need |
|-----------|-------|-------------|
| Gauntlet implementation | Dudley/Tyre | `build_gauntlet_app()` function for perf + golden file tests |
| `serde_json` derive on `ObserverSnapshot` | Dudley | `#[derive(Serialize, Deserialize)]` for JSON golden files (currently only `rmp_serde`) |
| Content validation phase 2 enums | Hoshe | Agreed list of valid location slugs, role slugs, and enum values for cross-reference validation |