feat(simulation): believability test protocol — D-245 enforcer (T-1083)

Promote the throwaway aliveness probe into a committed, repeatable believability
protocol — the instrument that found T-1080/T-1081/T-1082.

- atlas/believability.rs (new): BelievabilityReport (serde) + analyze() computing
  CONTRAST (per-field min/max/distinct for moisture/elev/slope/ocean over all
  districts; distinct morphology zones / vegetation classes / terrain materials)
  and COHERENCE (water-renders-wet, drainage-monotonic, vegetation-present) — never
  marginal per-tile counts (the lesson: that called a broken uniform world ALIVE).
  evaluate_criteria() = advisory D-245 checks; cascade_for_body()/seed_to_u64()
  loader shared by the bin and the harness. Unit tests prove the metric tells a
  uniform world (fails) from a varied one (passes) + determinism.
- bin/aliveness_probe.rs: refactored to a thin CLI over the module — prints the
  body-level report + advisory D-245 criteria, drops the naive per-tile verdict.
- tests/believability_harness.rs (+ golden): runs the real cascade for Arbour
  (temperate/ocean) + Edict (frozen/ice), asserts determinism, golden-snapshots
  the reports (regression — updates when T-1080/T-1082 land), advisory criteria
  with BELIEVABILITY_STRICT=1 to fail on unmet D-245 criteria. Skips if committed
  data absent. x86_64 golden (cascade has f32 warp paths).

Baseline @ believability-v1: Arbour 3/7, Edict 3/7 criteria pass — moisture
gradient, water-renders-wet, drainage all FAIL (T-1080/T-1082); Edict vegetation
0/64 (blank tundra — the frozen 'reads dead' case D-245 targets).

PNG layer maps (ticket item 3) deferred — explicitly optional; the contrast +
coherence metrics are the core enforcer.

clippy --all-targets -D warnings clean; 1575 lib tests + the harness pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-28 14:55:08 +02:00
co-authored by Claude Opus 4.8
parent 528cd5d5f5
commit 91a079718b
7 changed files with 905 additions and 218 deletions
+595
View File
@@ -0,0 +1,595 @@
//! Believability analysis — the D-245 acceptance-gate instrument (T-1083).
//!
//! D-245 makes "the nature layers read alive *anywhere*" the deliverable of the
//! cascade's nature half. This module is the **enforcer**: given the cascade's
//! per-district output for a body, it computes the metrics that decide whether the
//! generated world reads as a living, caused place — and exposes them as a
//! serialisable [`BelievabilityReport`] for both the interactive probe
//! (`bin/aliveness_probe`) and the regression harness (`tests/believability_harness`).
//!
//! ## The lesson baked in (T-1083)
//!
//! The first naive probe metric ("what fraction of tiles are vegetated?") reported a
//! *broken, uniform* world as ALIVE — every district was the same wet forest, but
//! "vegetated %" was high everywhere. A marginal per-tile count cannot tell "alive"
//! from "uniformly dead." So this module measures two things a count cannot fake:
//!
//! - **Contrast** ([`ContrastMetrics`]) — *variation across districts*. A living world
//! has gradients (moisture, elevation, slope) and a mix of morphologies, vegetation
//! classes and ground materials; a dead one is uniform. Spread + distinct-value
//! counts catch the uniform case (T-1080: `moisture_q` = 80 everywhere → distinct 1).
//! - **Coherence** ([`CoherenceMetrics`]) — *does it render as caused?* Water zones must
//! render wet (T-1082: oceans rendered as dry land), channels must sit at/below their
//! banks (drainage monotonic), and life must actually appear where conditions allow.
//!
//! ## District tier (today)
//!
//! Per T-1083, analysis is addressed in **district** space: no production code maps a
//! world chunk → its covering [`DistrictProfile`] yet (the voxel layer is a walking
//! skeleton). The `DistrictProfile` (2 km) is the finest *authoritative* nature unit the
//! cascade produces; the voxel-derived metrics sample a representative chunk per district.
//!
//! ## Determinism
//!
//! [`analyze`] is a pure, deterministic function of `(world_seed, body_id, districts)`:
//! scalar/categorical contrast is computed over **all** districts; the voxel-derived
//! metrics sample the first [`VOXEL_SAMPLE_DISTRICTS`] in `BTreeMap` order (sorted →
//! stable). Suitable for golden-snapshot regression.
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
use serde::{Deserialize, Serialize};
use crate::atlas::attractor_matching::CityRecord;
use crate::atlas::body_params_reader::BodyParamsReader;
use crate::atlas::body_world_state::BodyWorldState;
use crate::atlas::cascade::{run_cascade_from_heightmap, CascadeLayer};
use crate::atlas::chunk_context::derive_chunk_context;
use crate::atlas::district_profile::DistrictProfile;
use crate::atlas::heightmap::{load_heightmap_png, GRID_H, GRID_W};
use crate::atlas::scale::{ChunkPos, DistrictPos, CHUNKS_PER_DISTRICT, CHUNK_M};
use crate::atlas::voxel::{derive_voxel_column, Vegetation, Water};
use crate::seed::SeedChain;
use crate::simulation::generator::SettlementClass;
/// Fallback sea level when the heightmap PNG carries no `sea_level` tEXt chunk
/// (mirrors `layer_proxy::DEFAULT_SEA_LEVEL`).
const DEFAULT_SEA_LEVEL: f32 = 0.3;
/// `ocean_fraction_q` at or above this marks a district water-present (D-239 §10 uses
/// `ocean_fraction_q` as the chunk-scale water proxy).
const WATER_PRESENCE_Q: i32 = 10;
/// Districts voxel-sampled for the derived metrics — the first N in `BTreeMap` order
/// (deterministic). Scalar/categorical contrast uses *all* districts (cheap).
const VOXEL_SAMPLE_DISTRICTS: usize = 64;
/// Stride over a district's representative 64 m chunk (8 → an 8×8 = 64-voxel sample).
const VOXEL_SAMPLE_STRIDE: usize = 8;
// ---------------------------------------------------------------------------
// Report types (serde — golden-snapshot-able)
// ---------------------------------------------------------------------------
/// min / max / distinct-value count for one quantised district field (0..100).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct FieldStats {
pub min: i32,
pub max: i32,
pub distinct: usize,
}
impl FieldStats {
/// max − min. A spread of 0 means the field is flat across the whole body.
pub fn spread(&self) -> i32 {
self.max - self.min
}
}
/// Cross-district variation — the screen that catches uniform/dead worlds (T-1080).
/// Measures CONTRAST, never marginal per-tile counts (the T-1083 lesson).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct ContrastMetrics {
pub moisture_q: FieldStats,
pub elev_q: FieldStats,
pub slope_q: FieldStats,
pub ocean_fraction_q: FieldStats,
/// Distinct `MorphologyZone` values across all districts.
pub morphology_zones: usize,
/// Distinct `VegetationClass` values across all districts.
pub vegetation_classes: usize,
/// Distinct voxel `TerrainMaterial` values across the voxel sample.
pub terrain_materials: usize,
}
/// Coherence checks — does the world render as a *caused* place? Pass/fail counts
/// over the voxel-sampled districts.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct CoherenceMetrics {
/// Sampled districts with water present (`ocean_fraction_q` ≥ threshold or an
/// active channel).
pub water_districts: usize,
/// …that render at least one `Shallow`/`Deep` voxel (T-1082: oceans must be wet).
pub water_districts_wet: usize,
/// Sampled active-channel districts checked for drainage monotonicity.
pub drainage_samples: usize,
/// …where the wet voxels sit at/below the dry-land mean elevation (water runs low).
pub drainage_monotonic: usize,
/// Voxel-sampled districts checked for any vegetation.
pub vegetation_samples: usize,
/// …with at least one non-`Barren` voxel (life appears somewhere).
pub vegetated_districts: usize,
}
/// The full believability report for one body+seed — the D-245 gate's measurement.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct BelievabilityReport {
pub body_id: String,
pub world_seed: u64,
pub district_count: usize,
pub voxel_sampled_districts: usize,
pub contrast: ContrastMetrics,
pub coherence: CoherenceMetrics,
}
/// One advisory D-245 criterion result (thresholds are placeholders pending Q-123
/// calibration; the report exposes the raw numbers the thresholds judge).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Criterion {
pub name: &'static str,
pub pass: bool,
pub detail: String,
}
// ---------------------------------------------------------------------------
// Analysis
// ---------------------------------------------------------------------------
/// Compute the [`BelievabilityReport`] for a body's per-district cascade output.
///
/// Pure + deterministic (see module docs). `districts` is `BodyWorldState.districts`.
pub fn analyze(
world_seed: u64,
body_id: &str,
districts: &BTreeMap<DistrictPos, DistrictProfile>,
) -> BelievabilityReport {
// ── Contrast: scalar + categorical over ALL districts (cheap, no voxels) ──
let contrast_scalar = ContrastMetrics {
moisture_q: field_stats(districts.values().map(|d| d.moisture_q)),
elev_q: field_stats(districts.values().map(|d| d.elev_q)),
slope_q: field_stats(districts.values().map(|d| d.slope_q)),
ocean_fraction_q: field_stats(districts.values().map(|d| d.ocean_fraction_q)),
morphology_zones: distinct(
districts
.values()
.map(|d| format!("{:?}", d.morphology_zone)),
),
vegetation_classes: distinct(
districts
.values()
.map(|d| format!("{:?}", d.vegetation_class)),
),
terrain_materials: 0, // filled from the voxel sample below
};
// ── Voxel-derived metrics over a deterministic subset of districts ────────
let mut terrain_set: BTreeSet<String> = BTreeSet::new();
let mut coh = CoherenceMetrics::default();
let mut sampled = 0usize;
for (dp, prof) in districts.iter().take(VOXEL_SAMPLE_DISTRICTS) {
sampled += 1;
let chunk = district_centre_chunk(*dp);
let ctx = derive_chunk_context(world_seed, body_id, prof, chunk, None);
let mut any_wet = false;
let mut any_veg = false;
// Elevation accumulators for the drainage-monotonicity proxy.
let (mut wet_elev_sum, mut wet_n) = (0i64, 0i64);
let (mut dry_elev_sum, mut dry_n) = (0i64, 0i64);
for sx in (0..CHUNK_M as usize).step_by(VOXEL_SAMPLE_STRIDE) {
for sy in (0..CHUNK_M as usize).step_by(VOXEL_SAMPLE_STRIDE) {
let tx = chunk.0 * CHUNK_M + sx as i32;
let ty = chunk.1 * CHUNK_M + sy as i32;
let col = derive_voxel_column(world_seed, body_id, prof, &ctx, tx, ty);
terrain_set.insert(format!("{:?}", col.terrain));
if col.vegetation != Vegetation::Barren {
any_veg = true;
}
if col.water == Water::Dry {
dry_elev_sum += col.elevation_m as i64;
dry_n += 1;
} else {
any_wet = true;
wet_elev_sum += col.elevation_m as i64;
wet_n += 1;
}
}
}
// Water coherence (T-1082): water-present districts must render wet voxels.
if prof.ocean_fraction_q >= WATER_PRESENCE_Q || ctx.has_active_channel {
coh.water_districts += 1;
if any_wet {
coh.water_districts_wet += 1;
}
}
// Vegetation presence.
coh.vegetation_samples += 1;
if any_veg {
coh.vegetated_districts += 1;
}
// Drainage monotonicity (active-channel districts): a channel must render wet
// AND its water must sit at/below the dry-land mean elevation. A district whose
// channel renders no water (the T-1082 failure) fails this too — vacuously
// non-monotonic, which is the believability-correct verdict.
if ctx.has_active_channel {
coh.drainage_samples += 1;
let wet_below_land =
wet_n > 0 && (dry_n == 0 || wet_elev_sum / wet_n <= dry_elev_sum / dry_n.max(1));
if wet_below_land {
coh.drainage_monotonic += 1;
}
}
}
BelievabilityReport {
body_id: body_id.to_string(),
world_seed,
district_count: districts.len(),
voxel_sampled_districts: sampled,
contrast: ContrastMetrics {
terrain_materials: terrain_set.len(),
..contrast_scalar
},
coherence: coh,
}
}
/// Evaluate the report against the D-245 criteria (advisory — thresholds are
/// placeholders pending Q-123 calibration). Returns one [`Criterion`] per check.
pub fn evaluate_criteria(r: &BelievabilityReport) -> Vec<Criterion> {
let c = &r.contrast;
let h = &r.coherence;
let pct = |num: usize, den: usize| if den == 0 { 100 } else { num * 100 / den };
vec![
Criterion {
name: "moisture gradient",
pass: c.moisture_q.distinct >= 3,
detail: format!(
"moisture_q distinct={} spread={}",
c.moisture_q.distinct,
c.moisture_q.spread()
),
},
Criterion {
name: "elevation relief",
pass: c.elev_q.spread() >= 10,
detail: format!("elev_q spread={}", c.elev_q.spread()),
},
Criterion {
name: "morphology variety",
pass: c.morphology_zones >= 2,
detail: format!("{} distinct zones", c.morphology_zones),
},
Criterion {
name: "terrain-material variety",
pass: c.terrain_materials >= 2,
detail: format!("{} distinct materials", c.terrain_materials),
},
Criterion {
name: "water renders wet",
pass: pct(h.water_districts_wet, h.water_districts) >= 50,
detail: format!(
"{}/{} water districts render wet",
h.water_districts_wet, h.water_districts
),
},
Criterion {
name: "drainage monotonic",
pass: pct(h.drainage_monotonic, h.drainage_samples) >= 80,
detail: format!(
"{}/{} channel districts monotonic",
h.drainage_monotonic, h.drainage_samples
),
},
Criterion {
name: "vegetation present",
pass: pct(h.vegetated_districts, h.vegetation_samples) >= 25,
detail: format!(
"{}/{} sampled districts vegetated",
h.vegetated_districts, h.vegetation_samples
),
},
]
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
fn field_stats(vals: impl Iterator<Item = i32>) -> FieldStats {
let mut set: BTreeSet<i32> = BTreeSet::new();
for v in vals {
set.insert(v);
}
match (set.iter().next(), set.iter().next_back()) {
(Some(&min), Some(&max)) => FieldStats {
min,
max,
distinct: set.len(),
},
_ => FieldStats::default(),
}
}
fn distinct(vals: impl Iterator<Item = String>) -> usize {
vals.collect::<BTreeSet<String>>().len()
}
/// The chunk at the centre of a district (32 chunks/district) — the representative
/// chunk the voxel-derived metrics sample.
fn district_centre_chunk(dp: DistrictPos) -> ChunkPos {
(
dp.0 * CHUNKS_PER_DISTRICT + CHUNKS_PER_DISTRICT / 2,
dp.1 * CHUNKS_PER_DISTRICT + CHUNKS_PER_DISTRICT / 2,
)
}
// ---------------------------------------------------------------------------
// Loader — resolve committed data + run the real cascade (shared by the probe
// binary and the regression harness, so both measure the same thing)
// ---------------------------------------------------------------------------
/// FNV-1a 64-bit — a stable, dependency-free `seed string → u64` so `--seed yolo`
/// (and the harness's fixed seeds) map deterministically and reproducibly across
/// runs (unlike std's randomised `RandomState`). D-245 wants a string seed; this is it.
pub fn seed_to_u64(seed: &str) -> u64 {
let mut h: u64 = 0xcbf2_9ce4_8422_2325;
for &b in seed.as_bytes() {
h ^= b as u64;
h = h.wrapping_mul(0x0000_0100_0000_01b3);
}
h
}
/// Resolve the committed inputs for `body_id` and run the real deterministic cascade
/// (through the road graph), returning the per-body world state to [`analyze`].
///
/// `Err` if `systems.db` or the body's `heightmap.png` cannot be found, or the body
/// has no params — callers (the regression harness) may *skip* on that rather than
/// fail, so this is the one believability entry point that does I/O. Tries both the
/// repo-root and `server/`-relative paths so it works from either CWD.
pub fn cascade_for_body(world_seed: u64, body_id: &str) -> Result<BodyWorldState, String> {
let db = first_existing(&["server/data/systems.db", "data/systems.db"])
.ok_or_else(|| "systems.db not found".to_string())?;
let hm_path =
find_heightmap(body_id).ok_or_else(|| format!("no heightmap.png for {body_id}"))?;
let params = BodyParamsReader::open(&db)
.map_err(|e| format!("open systems.db: {e:?}"))?
.read_body_params(body_id)
.map_err(|e| format!("read body params: {e:?}"))?;
let cities = read_cities(&db, body_id)?;
let hm = load_heightmap_png(&hm_path, body_id, DEFAULT_SEA_LEVEL)
.map_err(|e| format!("load heightmap: {e:?}"))?;
let working = if hm.width > GRID_W || hm.height > GRID_H {
hm.downsample(GRID_W, GRID_H)
} else {
hm
};
let snapshot = run_cascade_from_heightmap(
SeedChain::for_body(world_seed, body_id),
working,
&cities,
None,
Some(&params),
CascadeLayer::RoadGraph,
);
Ok(snapshot.into_body_world_state())
}
/// Read a body's settlements from `atlas_city_names`. Per-city `settlement_class` is
/// unset in the committed pool, so it defaults to `PopulationBudget` (what the
/// city-context read-set assumes).
fn read_cities(db: &PathBuf, body_id: &str) -> Result<Vec<CityRecord>, String> {
let conn = rusqlite::Connection::open(db).map_err(|e| format!("open db: {e}"))?;
let mut stmt = conn
.prepare(
"SELECT id, name, COALESCE(economic_role,'service_mixed'), COALESCE(population,0)
FROM atlas_city_names WHERE body_id = ?1 ORDER BY id",
)
.map_err(|e| format!("prepare city query: {e}"))?;
let rows = stmt
.query_map([body_id], |r| {
Ok(CityRecord {
city_id: r.get::<_, i64>(0)? as u64,
name: r.get(1)?,
settlement_class: SettlementClass::PopulationBudget,
economic_role: r.get(2)?,
population: r.get(3)?,
})
})
.map_err(|e| format!("city query: {e}"))?
.filter_map(Result::ok)
.collect();
Ok(rows)
}
/// Glob `*/bodies/<body>/heightmap.png` under the committed wiki tree (either CWD).
fn find_heightmap(body_id: &str) -> Option<PathBuf> {
for base in ["wiki/star-systems", "../wiki/star-systems"] {
let Ok(systems) = std::fs::read_dir(PathBuf::from(base)) else {
continue;
};
for sys in systems.flatten() {
let cand = sys
.path()
.join("bodies")
.join(body_id)
.join("heightmap.png");
if cand.is_file() {
return Some(cand);
}
}
}
None
}
fn first_existing(paths: &[&str]) -> Option<PathBuf> {
paths.iter().map(PathBuf::from).find(|p| p.is_file())
}
// ---------------------------------------------------------------------------
// Tests — the instrument's own regression guard: it must tell alive from dead
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
use crate::atlas::district_profile::{
GlaciationGrade, PrecipitationClass, TectonicClass, VegetationClass,
};
use crate::atlas::scale::BasinDirection;
use crate::simulation::generator::MorphologyZone;
fn district(
zone: MorphologyZone,
elev_q: i32,
slope_q: i32,
moisture_q: i32,
ocean_fraction_q: i32,
veg: VegetationClass,
) -> DistrictProfile {
DistrictProfile {
morphology_zone: zone,
tectonic_class: TectonicClass::Stable,
glaciation_grade: GlaciationGrade::None,
precipitation_class: PrecipitationClass::Temperate,
slope_q,
elev_q,
ocean_fraction_q,
river_threshold: 200,
temperature_c: Some(15.0),
moisture_q,
vegetation_class: veg,
basin_direction: BasinDirection::South,
}
}
/// A uniform world — every district identical — must FAIL the contrast criteria.
/// This is the case the naive per-tile metric called ALIVE.
#[test]
fn uniform_world_fails_contrast_criteria() {
let mut districts: BTreeMap<DistrictPos, DistrictProfile> = BTreeMap::new();
for x in 0..8 {
for y in 0..8 {
districts.insert(
(x, y),
district(
MorphologyZone::AlluvialPlain,
20,
0,
80,
0,
VegetationClass::Forest,
),
);
}
}
let report = analyze(42, "uniform", &districts);
assert_eq!(report.contrast.moisture_q.distinct, 1, "moisture is flat");
assert_eq!(report.contrast.morphology_zones, 1);
assert_eq!(report.contrast.elev_q.spread(), 0);
let crit = evaluate_criteria(&report);
let failed = |name: &str| {
crit.iter()
.find(|c| c.name == name)
.is_some_and(|c| !c.pass)
};
assert!(failed("moisture gradient"), "uniform moisture must fail");
assert!(failed("morphology variety"), "single zone must fail");
assert!(failed("elevation relief"), "flat must fail");
}
/// A varied world — gradients + a mix of zones — passes the contrast criteria the
/// uniform world failed. Proves the metric discriminates (the T-1083 point).
#[test]
fn varied_world_passes_contrast_criteria() {
let zones = [
MorphologyZone::AlluvialPlain,
MorphologyZone::MeanderReach,
MorphologyZone::CliffCoast,
MorphologyZone::DuneStrand,
];
let vegs = [
VegetationClass::Barren,
VegetationClass::Scrub,
VegetationClass::Forest,
VegetationClass::RiparianThicket,
];
let mut districts: BTreeMap<DistrictPos, DistrictProfile> = BTreeMap::new();
for x in 0..8 {
for y in 0..8 {
let i = (x * 8 + y) as usize;
districts.insert(
(x, y),
district(
zones[i % zones.len()],
(i as i32 * 7) % 100, // varied elevation
(i as i32 * 3) % 60, // varied slope
(i as i32 * 11) % 100, // varied moisture
(i as i32 * 13) % 40,
vegs[i % vegs.len()],
),
);
}
}
let report = analyze(42, "varied", &districts);
assert!(report.contrast.moisture_q.distinct >= 3);
assert!(report.contrast.morphology_zones >= 2);
assert!(report.contrast.elev_q.spread() >= 10);
let crit = evaluate_criteria(&report);
let passed = |name: &str| crit.iter().find(|c| c.name == name).is_some_and(|c| c.pass);
assert!(passed("moisture gradient"));
assert!(passed("morphology variety"));
assert!(passed("elevation relief"));
}
#[test]
fn analyze_is_deterministic() {
let mut districts: BTreeMap<DistrictPos, DistrictProfile> = BTreeMap::new();
districts.insert(
(0, 0),
district(
MorphologyZone::MeanderReach,
30,
10,
55,
15,
VegetationClass::Forest,
),
);
districts.insert(
(1, 0),
district(
MorphologyZone::CliffCoast,
70,
40,
20,
60,
VegetationClass::Scrub,
),
);
let a = analyze(7, "GJ1c", &districts);
let b = analyze(7, "GJ1c", &districts);
assert_eq!(a, b);
}
}
+1
View File
@@ -4,6 +4,7 @@
//! populating BodyWorldState (D-203). They are never called on the main tick thread.
pub mod attractor_matching;
pub mod believability;
pub mod block_irregularity;
pub mod body_params_reader;
pub mod body_world_state;