feat(simulation): believability test protocol — D-245 enforcer (T-1083)
Promote the throwaway aliveness probe into a committed, repeatable believability protocol — the instrument that found T-1080/T-1081/T-1082. - atlas/believability.rs (new): BelievabilityReport (serde) + analyze() computing CONTRAST (per-field min/max/distinct for moisture/elev/slope/ocean over all districts; distinct morphology zones / vegetation classes / terrain materials) and COHERENCE (water-renders-wet, drainage-monotonic, vegetation-present) — never marginal per-tile counts (the lesson: that called a broken uniform world ALIVE). evaluate_criteria() = advisory D-245 checks; cascade_for_body()/seed_to_u64() loader shared by the bin and the harness. Unit tests prove the metric tells a uniform world (fails) from a varied one (passes) + determinism. - bin/aliveness_probe.rs: refactored to a thin CLI over the module — prints the body-level report + advisory D-245 criteria, drops the naive per-tile verdict. - tests/believability_harness.rs (+ golden): runs the real cascade for Arbour (temperate/ocean) + Edict (frozen/ice), asserts determinism, golden-snapshots the reports (regression — updates when T-1080/T-1082 land), advisory criteria with BELIEVABILITY_STRICT=1 to fail on unmet D-245 criteria. Skips if committed data absent. x86_64 golden (cascade has f32 warp paths). Baseline @ believability-v1: Arbour 3/7, Edict 3/7 criteria pass — moisture gradient, water-renders-wet, drainage all FAIL (T-1080/T-1082); Edict vegetation 0/64 (blank tundra — the frozen 'reads dead' case D-245 targets). PNG layer maps (ticket item 3) deferred — explicitly optional; the contrast + coherence metrics are the core enforcer. clippy --all-targets -D warnings clean; 1575 lib tests + the harness pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,139 @@
|
||||
//! Believability regression harness (T-1083) — the D-245 acceptance-gate enforcer.
|
||||
//!
|
||||
//! D-245 makes "the nature layers read alive *anywhere*" the deliverable of the
|
||||
//! cascade's nature half. This harness is its repeatable instrument: for a set of
|
||||
//! validation bodies at fixed seeds it runs the real deterministic cascade, computes
|
||||
//! the [`BelievabilityReport`] (contrast + coherence — never marginal per-tile counts,
|
||||
//! the T-1083 lesson), and:
|
||||
//!
|
||||
//! 1. **Determinism** — derives each body twice and asserts the reports are identical
|
||||
//! (the D-010 contract; the strongest invariant, platform-independent).
|
||||
//! 2. **Golden baseline** — snapshots the reports so any future cascade change that
|
||||
//! moves believability (a regression *or* an improvement, e.g. when T-1080/T-1082
|
||||
//! land) is caught and must be acknowledged. Regenerate with
|
||||
//! `UPDATE_GOLDEN=1 cargo test --test believability_harness`.
|
||||
//! 3. **D-245 criteria (advisory → strict)** — prints each criterion's pass/fail per
|
||||
//! body. Per D-245 the gate starts *budgeted + advisory* (thresholds are Q-123-TBD);
|
||||
//! set `BELIEVABILITY_STRICT=1` to make unmet criteria *fail* the test (they do
|
||||
//! today — that is the point: T-1080/T-1081/T-1082 are the open work to flip them).
|
||||
//!
|
||||
//! Distinct from `derivation_harness` (T-1031), which checks the binding *laws*; this
|
||||
//! is its believability sibling — laws are necessary, this is the sufficiency gate.
|
||||
//!
|
||||
//! Bodies whose committed data (systems.db / heightmap) is absent are **skipped** with
|
||||
//! a logged note (matching `derivation_harness`), so the harness is robust in trimmed
|
||||
//! checkouts. The golden is an x86_64 reference (the cascade has f32 warp paths, per
|
||||
//! `cascade_golden`).
|
||||
|
||||
use settled_reach_server::atlas::believability::{
|
||||
analyze, cascade_for_body, evaluate_criteria, seed_to_u64, BelievabilityReport,
|
||||
};
|
||||
|
||||
const GOLDEN_FILE: &str = "tests/golden/believability.json";
|
||||
|
||||
/// `(label, body_id, seed)` — lore-anchored validation bodies spanning the climate
|
||||
/// extremes the gate must handle: a temperate ocean world (should read lush) and a
|
||||
/// frozen ice world (must read as a living *cold* landscape, not blank — D-245).
|
||||
const VALIDATION_BODIES: &[(&str, &str, &str)] = &[
|
||||
("Arbour (temperate/ocean)", "GJ338Bd", "believability-v1"),
|
||||
("Edict (frozen/ice)", "GJ244Ad", "believability-v1"),
|
||||
];
|
||||
|
||||
/// Run the cascade + analyze for one body, or `None` if its committed data is absent.
|
||||
fn report_for(body_id: &str, seed: &str) -> Option<BelievabilityReport> {
|
||||
let world_seed = seed_to_u64(seed);
|
||||
match cascade_for_body(world_seed, body_id) {
|
||||
Ok(bws) => Some(analyze(world_seed, body_id, &bws.districts)),
|
||||
Err(e) => {
|
||||
eprintln!("[believability] SKIP {body_id}: {e}");
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn believability_determinism_and_golden() {
|
||||
let mut reports: Vec<BelievabilityReport> = Vec::new();
|
||||
|
||||
for (label, body_id, seed) in VALIDATION_BODIES {
|
||||
let Some(first) = report_for(body_id, seed) else {
|
||||
continue;
|
||||
};
|
||||
// Determinism (D-010): a second full cascade + analyze must match exactly.
|
||||
let second = report_for(body_id, seed).expect("body resolved once, must resolve again");
|
||||
assert_eq!(
|
||||
first, second,
|
||||
"[{label}] non-deterministic believability report — D-010 broken"
|
||||
);
|
||||
|
||||
// Structural sanity that holds regardless of believability quality or platform.
|
||||
assert!(
|
||||
first.district_count > 0,
|
||||
"[{label}] cascade produced no districts"
|
||||
);
|
||||
assert!(
|
||||
first.coherence.water_districts_wet <= first.coherence.water_districts,
|
||||
"[{label}] wet water districts exceed total"
|
||||
);
|
||||
|
||||
// Advisory D-245 criteria report (strict via BELIEVABILITY_STRICT=1).
|
||||
let strict = std::env::var("BELIEVABILITY_STRICT").is_ok();
|
||||
let crit = evaluate_criteria(&first);
|
||||
let passes = crit.iter().filter(|c| c.pass).count();
|
||||
eprintln!(
|
||||
"[believability] {label}: {passes}/{} D-245 criteria pass",
|
||||
crit.len()
|
||||
);
|
||||
for c in &crit {
|
||||
eprintln!(
|
||||
" [{}] {} — {}",
|
||||
if c.pass { "PASS" } else { "FAIL" },
|
||||
c.name,
|
||||
c.detail
|
||||
);
|
||||
if strict {
|
||||
assert!(
|
||||
c.pass,
|
||||
"[{label}] D-245 criterion failed (strict): {}",
|
||||
c.name
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
reports.push(first);
|
||||
}
|
||||
|
||||
if reports.is_empty() {
|
||||
eprintln!("[believability] no validation bodies available — skipping golden");
|
||||
return;
|
||||
}
|
||||
|
||||
// ── Golden baseline ─────────────────────────────────────────────────────
|
||||
let golden_path = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(GOLDEN_FILE);
|
||||
let actual_json = serde_json::to_string_pretty(&reports).expect("serialize reports") + "\n";
|
||||
|
||||
if std::env::var("UPDATE_GOLDEN").is_ok() {
|
||||
std::fs::create_dir_all(golden_path.parent().unwrap()).expect("mkdir golden");
|
||||
std::fs::write(&golden_path, &actual_json).expect("write golden");
|
||||
eprintln!("Golden written: {}", golden_path.display());
|
||||
return;
|
||||
}
|
||||
|
||||
let golden_json = std::fs::read_to_string(&golden_path).unwrap_or_else(|e| {
|
||||
panic!(
|
||||
"Golden not found: {}.\n\
|
||||
First run: UPDATE_GOLDEN=1 cargo test --test believability_harness\n{e}",
|
||||
golden_path.display()
|
||||
)
|
||||
});
|
||||
|
||||
let actual_v: serde_json::Value = serde_json::from_str(&actual_json).expect("reparse actual");
|
||||
let golden_v: serde_json::Value = serde_json::from_str(&golden_json).expect("parse golden");
|
||||
assert!(
|
||||
actual_v == golden_v,
|
||||
"Believability golden mismatch — the cascade moved believability (regression OR \
|
||||
improvement). If intended (e.g. T-1080/T-1082 landed), update:\n\
|
||||
UPDATE_GOLDEN=1 cargo test --test believability_harness\n\nActual:\n{}",
|
||||
actual_json.trim()
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
[
|
||||
{
|
||||
"body_id": "GJ338Bd",
|
||||
"world_seed": 13200299156134074780,
|
||||
"district_count": 2048,
|
||||
"voxel_sampled_districts": 64,
|
||||
"contrast": {
|
||||
"moisture_q": {
|
||||
"min": 80,
|
||||
"max": 80,
|
||||
"distinct": 1
|
||||
},
|
||||
"elev_q": {
|
||||
"min": 0,
|
||||
"max": 98,
|
||||
"distinct": 99
|
||||
},
|
||||
"slope_q": {
|
||||
"min": 0,
|
||||
"max": 13,
|
||||
"distinct": 14
|
||||
},
|
||||
"ocean_fraction_q": {
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"distinct": 65
|
||||
},
|
||||
"morphology_zones": 9,
|
||||
"vegetation_classes": 3,
|
||||
"terrain_materials": 1
|
||||
},
|
||||
"coherence": {
|
||||
"water_districts": 64,
|
||||
"water_districts_wet": 4,
|
||||
"drainage_samples": 16,
|
||||
"drainage_monotonic": 4,
|
||||
"vegetation_samples": 64,
|
||||
"vegetated_districts": 64
|
||||
}
|
||||
},
|
||||
{
|
||||
"body_id": "GJ244Ad",
|
||||
"world_seed": 13200299156134074780,
|
||||
"district_count": 2048,
|
||||
"voxel_sampled_districts": 64,
|
||||
"contrast": {
|
||||
"moisture_q": {
|
||||
"min": 20,
|
||||
"max": 20,
|
||||
"distinct": 1
|
||||
},
|
||||
"elev_q": {
|
||||
"min": 0,
|
||||
"max": 99,
|
||||
"distinct": 100
|
||||
},
|
||||
"slope_q": {
|
||||
"min": 0,
|
||||
"max": 6,
|
||||
"distinct": 7
|
||||
},
|
||||
"ocean_fraction_q": {
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"distinct": 65
|
||||
},
|
||||
"morphology_zones": 9,
|
||||
"vegetation_classes": 1,
|
||||
"terrain_materials": 2
|
||||
},
|
||||
"coherence": {
|
||||
"water_districts": 20,
|
||||
"water_districts_wet": 1,
|
||||
"drainage_samples": 4,
|
||||
"drainage_monotonic": 1,
|
||||
"vegetation_samples": 64,
|
||||
"vegetated_districts": 0
|
||||
}
|
||||
}
|
||||
]
|
||||
Reference in New Issue
Block a user