feat(simulation): T-1137 windowed district layer — queue-served, coalesced, determinism-tested (D-226 T-1124 SS1-SS4)
AtlasLayerRequest gains window_center/window_n (serde-default, absent = whole-body, wire back-compat; demux untouched — up_to stays the discriminator). DistrictWindowLayer echoes center/n + six parallel arrays (morphology, elev_q, temp_dc i16 with the region sentinel, moisture_q, vegetation incl Marine=6, glaciation). Serving per the amendment's binding model: NEVER inline — GenWorkItem::DeriveWindow rides the Rayon queue, completion drain caches into DistrictWindowCache (bounded FIFO 256; no staleness by D-227 purity, capacity bound only), serve_district_window polls the cache and returns Pending-shaped None until derived. Per-connection coalescing: submit_window supersedes a still-pending item for the same (ConnectionId, body) — the surviving item is the newer one, proven by dedicated tests. TerrainAnalysis decision (option b, numbers in ticket/PR): re-derive via run_layer1 in the DeriveWindow branch rather than caching ~1.5MB x 50 LRU slots (~100MB permanent, the exact D-203 bloat T-1044's own text guarded against); ~45ms one-time on the Rayon path, invisible to the tick thread. T-1044 confirmed within-cascade-only (cascade.rs:358 still drops the analysis before BodyWorldState) — the fork was open. Window derive loop promoted from aliveness_probe::render_window_panels; determinism promoted from probe-only proof to a real test (two passes byte-identical). New fixture atlas_response_ready_with_window exercises all six arrays incl. the airless sentinel and Marine; three existing fixtures gain district_window: None. Server clamps window_n to 1..=DISTRICT_WINDOW_MAX_N=64 (never trust the wire). 1774/1774 lib + 19/19 bridge_tcp green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -14,6 +14,7 @@
|
||||
//! - `AnalyzeBody`: D8 drainage + attractor extraction for a body.
|
||||
//! - `GenerateSkeleton`: Phase 1 QuarterSkeleton for a city.
|
||||
//! - `FillChunk`: Phase 2 chunk fill for a pre-loaded quarter.
|
||||
//! - `DeriveWindow`: District-resolution window derive (D-226 T-1124, T-1137).
|
||||
//!
|
||||
//! Completion events are delivered to the main thread via
|
||||
//! `GenerationQueue::drain_completions()`, called once per tick from a Bevy
|
||||
@@ -30,11 +31,13 @@ use crossbeam_channel::{Receiver, Sender};
|
||||
use crate::atlas::attractor_matching::CityRecord;
|
||||
use crate::atlas::body_world_state::BodyWorldState;
|
||||
use crate::atlas::cascade::{run_cascade_from_heightmap, CascadeLayer};
|
||||
use crate::atlas::district_profile::BodyParams;
|
||||
use crate::atlas::district_profile::{BodyParams, ClimateConstants, DistrictPos};
|
||||
use crate::atlas::heightmap::{load_heightmap_png, GRID_H, GRID_W};
|
||||
use crate::atlas::layer_proxy::{build_district_window_layer, DistrictWindowLayer};
|
||||
use crate::atlas::shell::{fill_chunk, FilledChunk};
|
||||
use crate::atlas::skeleton_gen::{assign_all_block_tags, generate_quarter_skeleton};
|
||||
use crate::atlas::trait_catalog_reader::ExteriorCatalog;
|
||||
use crate::bridge::ConnectionId;
|
||||
use crate::seed::SeedChain;
|
||||
use crate::simulation::generator::{BuildingPropertyTag, CityGenerationContext, QuarterWorldState};
|
||||
|
||||
@@ -148,12 +151,71 @@ pub enum GenWorkItem {
|
||||
/// this variant needs no `Box` to stay clippy `large_enum_variant`-clean.
|
||||
block_tags: Vec<BuildingPropertyTag>,
|
||||
},
|
||||
/// Derive a district-resolution window (D-226 T-1124 amendment, T-1137).
|
||||
///
|
||||
/// **Binding serving model:** window derives ride this SAME Rayon queue as
|
||||
/// every other expensive atlas path — never inline on the `PreInput` drain
|
||||
/// (the amendment's §1 is explicit: a window derive at n=32/64 is
|
||||
/// ~7–29 ms, which would blow the "cheap channel drain" contract
|
||||
/// `drain_generation_completions` documents).
|
||||
///
|
||||
/// **TerrainAnalysis availability (T-1137 binding decision, with numbers):**
|
||||
/// `BodyWorldState` does NOT retain `TerrainAnalysis` after cascade
|
||||
/// completion (T-1044 scoped its transient-carry fix to *within-cascade*
|
||||
/// reuse only — `cascade.rs` drops it once `DistrictProfile`+`RoadGraph`
|
||||
/// finish; see the doc on `CascadeSnapshot::terrain_analysis`). Caching it
|
||||
/// alongside every `BodyWorldStateCache` entry would cost ~2 MB × 50-body
|
||||
/// capacity ≈ 100 MB of PERMANENT resident cost, paid by every cached body
|
||||
/// whether or not a window is ever requested for it — the exact D-203
|
||||
/// budget concern T-1044's own ticket text guarded against. Re-deriving via
|
||||
/// `run_layer1` per window-serving cache miss costs ~45 ms ONE-TIME, paid
|
||||
/// only when a window is actually requested, and — because this work item
|
||||
/// already runs off the tick thread on the Rayon queue — that cost is
|
||||
/// invisible to the main thread; it is the same order of magnitude as one
|
||||
/// window derive itself, not a multiplier on it. So: re-derive
|
||||
/// (`heightmap_path`/`sea_level` below), matching `aliveness_probe`'s
|
||||
/// existing `--render` workaround, NOT a `TerrainAnalysis` field cached on
|
||||
/// `BodyWorldState`.
|
||||
DeriveWindow {
|
||||
body_id: String,
|
||||
/// Coalescing/routing key (D-226 T-1124 amendment §1 "recommended"
|
||||
/// per-connection coalescing) — NOT used by `run_work_item` itself
|
||||
/// (the derive is connection-agnostic), only by
|
||||
/// `GenerationQueue::submit_window` to decide which still-pending item
|
||||
/// a new one for the same connection+body supersedes.
|
||||
conn_id: ConnectionId,
|
||||
/// Mod-resolved source heightmap PNG (mirrors `AnalyzeBody`).
|
||||
heightmap_path: PathBuf,
|
||||
sea_level: f32,
|
||||
/// This body's `SeedChain` position — `derive_district`'s seed input.
|
||||
body_seed: SeedChain,
|
||||
/// Body physical parameters. Boxed for the same `large_enum_variant`
|
||||
/// reason `AnalyzeBody.body_params` is boxed.
|
||||
body_params: Box<BodyParams>,
|
||||
/// Window centre + side length in districts. `n` is ALREADY clamped to
|
||||
/// `[1, DISTRICT_WINDOW_MAX_N]` by the caller (`handle_atlas_request`)
|
||||
/// before this item is built — never trusted from the wire again here.
|
||||
center: DistrictPos,
|
||||
n: u32,
|
||||
},
|
||||
}
|
||||
|
||||
impl GenWorkItem {
|
||||
pub fn body_id(&self) -> Option<&str> {
|
||||
if let GenWorkItem::AnalyzeBody { body_id, .. } = self {
|
||||
Some(body_id)
|
||||
match self {
|
||||
GenWorkItem::AnalyzeBody { body_id, .. } => Some(body_id),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Coalescing key for `DeriveWindow` items only — `(connection, body)`.
|
||||
/// `None` for every other variant (they don't coalesce this way).
|
||||
pub fn window_supersede_key(&self) -> Option<(ConnectionId, &str)> {
|
||||
if let GenWorkItem::DeriveWindow {
|
||||
body_id, conn_id, ..
|
||||
} = self
|
||||
{
|
||||
Some((*conn_id, body_id))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
@@ -187,6 +249,19 @@ pub enum GenCompletion {
|
||||
/// Boxed to keep `GenCompletion` variant sizes balanced.
|
||||
filled: Box<FilledChunk>,
|
||||
},
|
||||
/// A district window finished deriving (D-226 T-1124 amendment, T-1137).
|
||||
/// The main thread inserts `layer` into the `DistrictWindowCache` keyed by
|
||||
/// `(body_id, layer.center, layer.n)` — NOT pushed directly into any
|
||||
/// in-flight response (the window's requester re-polls per the existing
|
||||
/// D-225 loop and hits the now-populated cache on its next request; see
|
||||
/// `handle_atlas_request`'s window branch).
|
||||
WindowDerived {
|
||||
body_id: String,
|
||||
/// Boxed to keep `GenCompletion` variant sizes balanced (six
|
||||
/// `Vec`s — comparable to `SkeletonGenerated`/`ChunkFilled`'s own
|
||||
/// boxing rationale).
|
||||
layer: Box<DistrictWindowLayer>,
|
||||
},
|
||||
/// Work item failed — body_id or city_id for logging.
|
||||
Failed { item: GenWorkItem, reason: String },
|
||||
}
|
||||
@@ -303,6 +378,43 @@ impl GenerationQueue {
|
||||
self.dispatch_next();
|
||||
}
|
||||
|
||||
/// Submit a `DeriveWindow` item with per-connection coalescing (D-226
|
||||
/// T-1124 amendment §1, "recommended"): if a `DeriveWindow` item for the
|
||||
/// SAME `(connection, body)` is still sitting in the pending queue
|
||||
/// (not yet dispatched to a Rayon worker), it is replaced in place by the
|
||||
/// new one — a pan-burst that queues several window requests for the same
|
||||
/// connection+body before the first is dispatched collapses to one derive.
|
||||
///
|
||||
/// Deliberately does **not** attempt to cancel an item already dispatched
|
||||
/// to a Rayon worker (no cancellation channel exists, and the amendment
|
||||
/// does not require it — bounding queue buildup is the goal, not
|
||||
/// interrupting in-flight compute). `item` MUST be a `DeriveWindow`
|
||||
/// variant; any other variant is submitted via the plain `submit` path
|
||||
/// with no coalescing (this method still accepts it for caller
|
||||
/// convenience, but the supersede check is a no-op when
|
||||
/// `window_supersede_key()` returns `None`).
|
||||
pub fn submit_window(&self, item: GenWorkItem, priority: GenPriority) {
|
||||
if let Some(key) = item.window_supersede_key() {
|
||||
let key = (key.0, key.1.to_string());
|
||||
let mut pending = self.pending.lock().unwrap();
|
||||
pending.retain(|q| {
|
||||
q.item
|
||||
.window_supersede_key()
|
||||
.map(|k| (k.0, k.1.to_string()) != key)
|
||||
.unwrap_or(true)
|
||||
});
|
||||
let pos = pending
|
||||
.iter()
|
||||
.position(|q| q.priority > priority)
|
||||
.unwrap_or(pending.len());
|
||||
pending.insert(pos, QueuedWork { priority, item });
|
||||
drop(pending);
|
||||
self.dispatch_next();
|
||||
} else {
|
||||
self.submit(item, priority);
|
||||
}
|
||||
}
|
||||
|
||||
/// Drain all completed items from the channel and dispatch pending work.
|
||||
///
|
||||
/// Call once per tick from the main thread. Returns all completions
|
||||
@@ -491,6 +603,52 @@ fn run_work_item(item: &GenWorkItem) -> GenCompletion {
|
||||
filled: Box::new(filled),
|
||||
}
|
||||
}
|
||||
GenWorkItem::DeriveWindow {
|
||||
body_id,
|
||||
conn_id: _, // routing-only (queue-level coalescing); the derive itself is connection-agnostic
|
||||
heightmap_path,
|
||||
sea_level,
|
||||
body_seed,
|
||||
body_params,
|
||||
center,
|
||||
n,
|
||||
} => match load_heightmap_png(heightmap_path, body_id, *sea_level) {
|
||||
Ok(hm) => {
|
||||
// Same GRID_W×GRID_H downsample AnalyzeBody applies (D-202) — the
|
||||
// window derive must run on the SAME working-grid resolution the
|
||||
// whole-body cascade uses, or district positions between the two
|
||||
// views would disagree (derive_district maps DistrictPos through
|
||||
// ta.w/ta.h, T-1137 decision note).
|
||||
let working = if hm.width > GRID_W || hm.height > GRID_H {
|
||||
hm.downsample(GRID_W, GRID_H)
|
||||
} else {
|
||||
hm
|
||||
};
|
||||
// Re-derive TerrainAnalysis via run_layer1 (T-1137 binding
|
||||
// decision — see the DeriveWindow variant doc for the numbers).
|
||||
// This is the SAME workaround aliveness_probe --render already
|
||||
// uses when CascadeSnapshot.terrain_analysis is None.
|
||||
let (_, ta) = crate::atlas::layer1::run_layer1(&working);
|
||||
let climate = ClimateConstants::default();
|
||||
let layer = build_district_window_layer(
|
||||
*body_seed,
|
||||
body_id,
|
||||
body_params,
|
||||
&ta,
|
||||
*center,
|
||||
*n,
|
||||
&climate,
|
||||
);
|
||||
GenCompletion::WindowDerived {
|
||||
body_id: body_id.clone(),
|
||||
layer: Box::new(layer),
|
||||
}
|
||||
}
|
||||
Err(e) => GenCompletion::Failed {
|
||||
item: item.clone(),
|
||||
reason: format!("heightmap load failed: {e}"),
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -814,4 +972,145 @@ mod tests {
|
||||
"a chunk containing a building must derive shell voxels"
|
||||
);
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// DeriveWindow / submit_window coalescing (D-226 T-1124 amendment, T-1137)
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
/// Build a `DeriveWindow` work item pointing at a tiny test heightmap,
|
||||
/// mirroring `analyze()`'s fixture shape.
|
||||
fn derive_window(body_id: &str, conn_id: ConnectionId, center: DistrictPos) -> GenWorkItem {
|
||||
GenWorkItem::DeriveWindow {
|
||||
body_id: body_id.to_string(),
|
||||
conn_id,
|
||||
heightmap_path: test_heightmap_path(),
|
||||
sea_level: 0.3,
|
||||
body_seed: SeedChain::for_body(42, body_id),
|
||||
body_params: Box::new(BodyParams {
|
||||
hydrosphere: Some("ocean".into()),
|
||||
atmosphere: Some("breathable".into()),
|
||||
planet_class: Some("temperate".into()),
|
||||
body_radius_km: Some(6371.0),
|
||||
..Default::default()
|
||||
}),
|
||||
center,
|
||||
n: 4,
|
||||
}
|
||||
}
|
||||
|
||||
/// The full `DeriveWindow` → Rayon → `WindowDerived` round trip: the
|
||||
/// real re-derive-via-`run_layer1` path (T-1137 binding decision) runs
|
||||
/// end to end and produces a populated `DistrictWindowLayer`.
|
||||
#[test]
|
||||
fn derive_window_round_trip_produces_populated_layer() {
|
||||
let q = make_queue();
|
||||
q.submit(
|
||||
derive_window("TestBody", ConnectionId(0), (2, -1)),
|
||||
GenPriority::Immediate,
|
||||
);
|
||||
std::thread::sleep(Duration::from_millis(150));
|
||||
let completions = q.drain_completions();
|
||||
assert_eq!(completions.len(), 1);
|
||||
let GenCompletion::WindowDerived { body_id, layer } = &completions[0] else {
|
||||
panic!("expected WindowDerived, got {:?}", completions[0]);
|
||||
};
|
||||
assert_eq!(body_id, "TestBody");
|
||||
assert_eq!(layer.center, (2, -1));
|
||||
assert_eq!(layer.n, 4);
|
||||
assert_eq!(layer.morphology.len(), 16);
|
||||
assert_eq!(layer.elev_q.len(), 16);
|
||||
assert_eq!(layer.temp_dc.len(), 16);
|
||||
assert_eq!(layer.moisture_q.len(), 16);
|
||||
assert_eq!(layer.vegetation.len(), 16);
|
||||
assert_eq!(layer.glaciation.len(), 16);
|
||||
}
|
||||
|
||||
/// `submit_window` coalescing (D-226 T-1124 amendment §1 "recommended"):
|
||||
/// two `DeriveWindow` items for the SAME `(connection, body)` queued
|
||||
/// while the pool is saturated collapse to ONE pending entry — the
|
||||
/// second submission replaces the first rather than queuing alongside it.
|
||||
#[test]
|
||||
fn submit_window_coalesces_same_connection_and_body() {
|
||||
// Single-thread pool: the first item occupies the only worker, so
|
||||
// subsequent DeriveWindow submissions stay in `pending` long enough
|
||||
// to inspect (mirrors `priority_ordering_respected_under_saturation`'s
|
||||
// saturation trick). Uses `analyze()` (real cascade work — measurable
|
||||
// latency), NOT `FillChunk` (documented "trivially fast" — it can
|
||||
// complete before the next `submit_window` call even runs, which
|
||||
// would make `pending_count()` observe 0 instead of 1, a real race
|
||||
// this test hit before switching occupiers).
|
||||
let q = GenerationQueue::with_threads(1);
|
||||
q.submit(analyze("Occupier"), GenPriority::Low);
|
||||
|
||||
let conn = ConnectionId(7);
|
||||
q.submit_window(derive_window("Coal", conn, (0, 0)), GenPriority::Immediate);
|
||||
assert_eq!(
|
||||
q.pending_count(),
|
||||
1,
|
||||
"one DeriveWindow queued behind the saturating item"
|
||||
);
|
||||
|
||||
// A second DeriveWindow for the SAME (connection, body) supersedes
|
||||
// the first — pending count stays at 1, not 2.
|
||||
q.submit_window(derive_window("Coal", conn, (5, 5)), GenPriority::Immediate);
|
||||
assert_eq!(
|
||||
q.pending_count(),
|
||||
1,
|
||||
"same (connection, body) DeriveWindow must supersede, not queue alongside"
|
||||
);
|
||||
|
||||
// Drain everything and confirm exactly one WindowDerived for "Coal",
|
||||
// carrying the SECOND (superseding) center — not the first.
|
||||
std::thread::sleep(Duration::from_millis(150));
|
||||
let mut completions = q.drain_completions();
|
||||
std::thread::sleep(Duration::from_millis(150));
|
||||
completions.extend(q.drain_completions());
|
||||
|
||||
let window_completions: Vec<_> = completions
|
||||
.iter()
|
||||
.filter_map(|c| {
|
||||
if let GenCompletion::WindowDerived { body_id, layer } = c {
|
||||
if body_id == "Coal" {
|
||||
return Some(layer);
|
||||
}
|
||||
}
|
||||
None
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(
|
||||
window_completions.len(),
|
||||
1,
|
||||
"exactly one WindowDerived for the coalesced body, not two"
|
||||
);
|
||||
assert_eq!(
|
||||
window_completions[0].center,
|
||||
(5, 5),
|
||||
"the surviving item must be the SECOND (superseding) submission"
|
||||
);
|
||||
}
|
||||
|
||||
/// `submit_window` does NOT coalesce across different connections or
|
||||
/// different bodies — only an exact `(connection, body)` match supersedes.
|
||||
#[test]
|
||||
fn submit_window_does_not_coalesce_different_keys() {
|
||||
let q = GenerationQueue::with_threads(1);
|
||||
// See `submit_window_coalesces_same_connection_and_body`'s comment on
|
||||
// why the occupier must be `analyze()`, not `FillChunk`.
|
||||
q.submit(analyze("Occupier2"), GenPriority::Low);
|
||||
|
||||
// Different connections, same body — must NOT coalesce.
|
||||
q.submit_window(
|
||||
derive_window("Shared", ConnectionId(1), (0, 0)),
|
||||
GenPriority::Immediate,
|
||||
);
|
||||
q.submit_window(
|
||||
derive_window("Shared", ConnectionId(2), (1, 1)),
|
||||
GenPriority::Immediate,
|
||||
);
|
||||
assert_eq!(
|
||||
q.pending_count(),
|
||||
2,
|
||||
"different connections requesting the same body must NOT coalesce"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user