Merge remote-tracking branch 'origin/bmv-measurements'
This commit is contained in:
@@ -216,15 +216,19 @@ necessary (workshops are not closed early on partial convergence).
|
||||
tickets drafted by the lead, reviewed by SI; Qatux produces
|
||||
`workshop-outcomes.md` and diagram updates.
|
||||
|
||||
## Pre-workshop measurement appendix (owners; numbers land here before round 1)
|
||||
## Pre-workshop measurement appendix — MEASURED (2026-07-25, 16-core / Rayon 14 / RX 9070, release builds; all numbers post-date a background-load closure with stability re-runs — details + repro commands in each linked doc)
|
||||
|
||||
| # | Measurement | Owner | Effort |
|
||||
**The gate is satisfied: ①–④ measured, ⑤ measured. Round 1 may start.**
|
||||
|
||||
| # | Ticket | Headline result | Doc |
|
||||
|---|---|---|---|
|
||||
| ① | Equilibrium hydrology solver prototype + bench (512×256 and 4K-class) incl. cliff representation output | Dudley | 1–2 d |
|
||||
| ② | Per-gridunit derive at real canvas sizes (330K and 8.3M cells, production par_iter path — no extrapolation) | Dudley | 0.5 d |
|
||||
| ③ | T-1154 block/tile rung costs (now unblocked) — prices the ladder's bottom | Dudley | 0.5 d |
|
||||
| ④ | Wire-size table for candidate encodings (raw/bit-packed/RLE/PNG; MessagePack round-trip on one real canvas) | Araminta + Dudley | 0.5 d |
|
||||
| ⑤ | Godot ImageTexture upload cost per step-cross (330K–8.3M px) | Stig | 0.25 d |
|
||||
| ① | T-1177 | **Settled hydrology VIABLE**: 512×256 ~24 ms/body; all 273 bodies Rayon-parallel ~0.7–0.8 s; 8.3M cells ~5.7 s single-thread (no production path needs it synchronously). Cliff representation: dominant height + `channel_depth` + `cliff_edge` flag. Finding: gorge carving is structurally RARE (priority-flood finds true spill levels; zero carved cells at production scales — needs chained-basin geometry). Solver: priority-flood + Dijkstra overflow, pure function, determinism-proved. | [t1177](measurements/t1177-hydrology.md) |
|
||||
| ② | T-1178 | **Parallel throughput HOLDS at scale** (the T-1143 extrapolation gap, closed): ~190–220 ns/cell parallel flat from 330K→8.3M cells; 330K canvas ~64 ms, 2.07M ~0.4 s, 8.3M ~1.7–1.8 s (7.5–8.8× speedup); single-thread flat ~1.65 µs/cell, matches prior baseline within 8%. Throughput cross-validated on three bodies across two call paths (square real-`build_district_window_layer` = courses-light, 3–10 in window; rectangular replica loop = courses-empty by construction — agreeing within 0.3%); the courses-inclusive rate at real production density is measured once, GJ1c 330K with 18 courses at 195.0 ns/cell (within 2%, consistent with the <5% course-cost bound). | [t1178+t1154](measurements/t1178-t1154-derive-bench.md) |
|
||||
| ③ | T-1154 | **Block GO, Tile GO on cost** (~1.8–1.9 µs/cell, same band as District/Quarter). Octave cutoff buys ZERO below District spacing (verified: wavelength table bottoms at 128 m). Deepest-step realistic canvas (216×384 m @ 1 m = 83K cells): **~17 ms parallel — trivially interactive**. `voxel_relief` already in the Atlas path; `voxel_mosaic` untouched by it. Real gates are wire carrier + D-226(d) canvas policy, not cost. | [t1178+t1154](measurements/t1178-t1154-derive-bench.md) |
|
||||
| ④ | T-1179 | **PNG-per-field wins everywhere** — smallest AND fastest: 330K canvas = 638 KB (0.32× raw, 5.4/3.6 ms enc/dec) vs raw rmp 1.99 MB; RLE loses to raw (elev/temp near-noise per cell). **No encoding brings a step canvas near the existing windowed-payload budget** (best case 21× the ~30 KB cap at the smallest size; 563× at 8.3M) — the tagged-envelope migration cannot be dodged by payload optimization. Whether the letter of the D-226 §2 *field-count* rule is what triggers it is a workshop synthesis call, not a measured result. Corrected density: 6.00 B/cell measured raw. | [t1179](measurements/t1179-wire-table.md) |
|
||||
| ⑤ | T-1180 | **Upload cost is a non-issue**: worst case (8.3M px RGBA8 create) ~3.2 ms median, ~2.4–2.9 ms frame-delta spike — never near the 16.6 ms budget. **L8 is 4–9× cheaper at every size** (~0.5–0.7 ms at 4K). Prefer `texture.update` reuse on step-cross; use L8 for single-channel planes. Windowed-only measurement (headless renderer fakes uploads). | [t1180](measurements/t1180-imagetexture.md) |
|
||||
|
||||
Bench code (all `#[ignore]`d release tests): `server/src/atlas/hydrology_equilibrium.rs` + `server/tests/hydrology_equilibrium_bench.rs`, `server/tests/bmv_gridunit_bench.rs`, `server/tests/wire_encoding_bench.rs`.
|
||||
|
||||
## Input Documents
|
||||
|
||||
|
||||
@@ -0,0 +1,419 @@
|
||||
---
|
||||
title: "T-1177 — equilibrium hydrology solver prototype + bench (512×256 and 4K-class) incl. cliff representation (measurement ①)"
|
||||
workshop: body-map-viewer
|
||||
status: complete
|
||||
owner: Dudley
|
||||
---
|
||||
|
||||
# T-1177 — equilibrium hydrology solver prototype + bench
|
||||
|
||||
Measurement ① of the [body-map-viewer workshop
|
||||
brief](../body-map-viewer-workshop-brief.md)'s pre-workshop appendix, per
|
||||
[tyre-implications.md §3](../tyre-implications.md) item 1 — the highest-priority,
|
||||
biggest-unknown measurement, named as the one that could invalidate per-body-open
|
||||
settled hydrology outright if costs land in seconds, not milliseconds. Also the
|
||||
mandatory output for **red flag 4** (the cliff/multi-height data-model gap).
|
||||
|
||||
## Environment note (read first)
|
||||
|
||||
A background Factorio process was running on this machine for part of this
|
||||
session and may have starved CPU before the coordinator's mid-session notice.
|
||||
**No benchmark timing in this document predates the coordinator's closure
|
||||
notice.** Concretely: module compilation, unit-test authoring, and the
|
||||
fixture-design investigation (see "Gorge carving — the honest story" below)
|
||||
all happened before and after the notice, but every `ms`/`ns` figure reported
|
||||
below was captured **after** the notice, with `/proc/loadavg` checked
|
||||
immediately before each timed run (observed 1-minute load between 0.5 and 5.6
|
||||
throughout the timed portion of this session — never during the
|
||||
Factorio-shutdown settling spike). The three headline cases (512×256,
|
||||
768×432, 3840×2160) were each run twice — once during the initial measurement
|
||||
pass, once as a final stability re-run — and landed within ~1-3% of each
|
||||
other both times (see the "Stability re-run" row in the results table), so
|
||||
there is no early-slow/later-fast delta to misattribute to warm-up or
|
||||
optimization.
|
||||
|
||||
## What was built
|
||||
|
||||
New module: `server/src/atlas/hydrology_equilibrium.rs` (registered in
|
||||
`server/src/atlas/mod.rs`). New bench file:
|
||||
`server/tests/hydrology_equilibrium_bench.rs`. No other files touched.
|
||||
|
||||
`solve(elevation, width, height, sea_level, climate) -> HydrologyResult` is a
|
||||
pure function of its four arguments (deterministic per D-010) — no RNG, no
|
||||
wall-clock, no `HashMap` iteration anywhere in the call graph. It is **not** a
|
||||
stateful simulation: there is no tick loop, no iterative erosion-over-time
|
||||
model, no rainfall accumulation across steps. One elevation field in, one
|
||||
settled hydrology result out, same every time.
|
||||
|
||||
### Algorithm (priority-flood family, Barnes/Planchon-Darboux class)
|
||||
|
||||
1. **Priority-flood fill** (`priority_flood_fill`): a min-heap seeded at
|
||||
every below-sea-level cell AND every top/bottom grid-edge cell (mirroring
|
||||
`drainage.rs`'s own row-bounded D8 walk — the equirectangular grid has no
|
||||
pole neighbor, so those rows are boundary the same way a coastline is).
|
||||
Standard "flood from the outside in": repeatedly pop the lowest-water-level
|
||||
frontier cell, raise each unvisited neighbor to
|
||||
`max(neighbor_original, popped_level)`. `O(n log n)`, one pass, versus
|
||||
`drainage.rs`'s existing `depression_fill` which is an iterative relaxation
|
||||
(up to 10 full-grid passes) of a *similar* problem but does not expose
|
||||
basin membership or spill points — this module needed both, so it
|
||||
re-derives the fill with a heap that tracks them, rather than layering
|
||||
basin-detection on top of that existing black-box result.
|
||||
2. **Lake cells + basin grouping**: any cell where `filled > original` is a
|
||||
lake cell; contiguous (8-connected) lake cells are grouped into basins via
|
||||
BFS flood-fill (`label_lake_basins`), ids assigned in row-major discovery
|
||||
order for determinism.
|
||||
3. **Spill point + spill level per basin**: the basin-boundary cell whose
|
||||
OUTSIDE neighbor has the lowest *original* (unfilled) elevation — i.e. the
|
||||
genuine topographic saddle, not merely "any lake cell touching non-basin
|
||||
terrain" (an early draft ranked by the boundary cell's own *filled*
|
||||
elevation, which degenerates to an arbitrary tie-break on a uniformly-filled
|
||||
lake, since every boundary cell of a flat lake shares the same filled
|
||||
value — see "Gorge carving" below for how this surfaced). The spill level
|
||||
is that spill cell's own `filled` value — the lake's water surface at
|
||||
overtopping.
|
||||
4. **Endorheic classification** (`is_endorheic`, moisture-governed, see
|
||||
below): large + dry basins are declared closed water-cycle sinks
|
||||
(`BasinOutcome::Endorheic`, mapped to `RIVER_DOWNSTREAM_TERMINAL` for
|
||||
consumers — see "Reuse of the TERMINAL sentinel").
|
||||
5. **Overflow path** (`cheapest_overflow_path`, Dijkstra/A*-family, same
|
||||
`BinaryHeap<Reverse<(cost, idx)>>` pattern `road_graph.rs`'s own `astar`
|
||||
uses): from the spill point, cost = cumulative elevation carved above the
|
||||
basin's own spill level (0 for any step at or below it). Terminates at the
|
||||
first cell that is below sea level (`Sea`), belongs to a *different*
|
||||
already-resolved basin (`Basin(id)` — the chaining case), or is open low
|
||||
ground not otherwise claimed (`OpenSpillway` — a genuine SUCCESS terminus
|
||||
with a real, complete path, distinct from `EdgeUnreachable`, which is
|
||||
reserved strictly for search exhaustion — a review finding on PR #198
|
||||
caught an earlier draft conflating the two under one variant; see
|
||||
"`DownstreamTarget` outcome model" below). Basins are processed in
|
||||
ascending spill-level order so a lower basin is always resolved before a
|
||||
higher one could chain into it.
|
||||
|
||||
**`DownstreamTarget` outcome model (four variants):**
|
||||
|
||||
| Variant | Meaning | `outlet_path` |
|
||||
|---|---|---|
|
||||
| `Sea` | Reaches a sub-sea-level cell | real, complete |
|
||||
| `Basin(id)` | Chains into another already-resolved basin's footprint | real, complete |
|
||||
| `OpenSpillway` | Reaches open low ground — not sea, not another basin, but at/below the originating basin's own spill level | real, complete |
|
||||
| `EdgeUnreachable` | Search exhausted its budget without reaching ANY of the above (a genuine grid-topology dead end), OR no spill cell existed at all for the basin | best-effort partial (or empty) |
|
||||
|
||||
Only `EdgeUnreachable` is a failure case; the other three (including
|
||||
`OpenSpillway`) are all successful overflow terminations that happen to
|
||||
land somewhere other than the sea or another lake.
|
||||
6. **Carving**: every path cell whose elevation exceeds the basin's spill
|
||||
level is cut down to it — `channel_depth_scaled[cell] = original[cell] -
|
||||
spill_level`, `cliff_edge[cell] = true` — but ONLY when the path's peak
|
||||
flow accumulation (reusing a D8 flow-direction/accumulation pass over the
|
||||
*filled* surface, same algorithm as `drainage.rs`) clears
|
||||
`RIVER_THRESHOLD = 200` (mirroring `drainage::RIVER_THRESHOLD`): a
|
||||
trickling overflow through near-flat ground doesn't get a canyon.
|
||||
|
||||
### Reuse of the TERMINAL sentinel
|
||||
|
||||
The ticket flagged `RIVER_DOWNSTREAM_TERMINAL` (`body_world_state.rs`,
|
||||
value 10) as "reserved, unused in round 1 — future endorheic basin." Confirmed
|
||||
by reading `river_course.rs::build_edges` (T-1170): it already special-cases
|
||||
`sentinel == RIVER_DOWNSTREAM_TERMINAL` as a no-op skip, so a consumer that
|
||||
starts emitting it does not break any existing code path — this prototype's
|
||||
`BasinOutcome::Endorheic` is exactly the case that sentinel was reserved for,
|
||||
and mapping to it at the wire-integration stage (not built in this
|
||||
prototype — see "What this prototype does NOT do") is additive, not a
|
||||
migration.
|
||||
|
||||
## Measured numbers
|
||||
|
||||
**Hardware:** 16 logical cores (Intel Core i9-9900K @ 3.60GHz), Rayon default
|
||||
pool (`std::thread::available_parallelism() == 16`, confirmed in the parallel
|
||||
bench's own printed output). Release build (`cargo build --release`), bench
|
||||
invocation:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
cargo test --release --test hydrology_equilibrium_bench <test_name> -- --ignored --nocapture
|
||||
```
|
||||
|
||||
Cold = first `solve()` call in the process; warm = second call, same input,
|
||||
same process (allocator/cache warm). All entries below are **MEASURED**, none
|
||||
extrapolated.
|
||||
|
||||
| Grid | Cells | Cold total | Cold ns/cell | Warm total | Warm ns/cell | Basins | Lake cells | Carved cells |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| 512×256 (real GJ1c) | 131,072 | 24.36–24.97 ms | 185.8–190.5 | 23.61–24.58 ms | 180.1–187.5 | 68 (66 overflow, 2 endorheic) | 4,623 | 0 |
|
||||
| 768×432 (synthetic, ~330K) | 331,776 | 72.32–72.97 ms | 218.0–219.9 | 69.03–69.98 ms | 208.1–210.9 | 103 (all overflow) | 16,617 | 0 |
|
||||
| 3840×2160 (synthetic, ~8.3M) | 8,294,400 | 5,671.84–5,711.82 ms | 683.8–688.6 | 5,699.31–6,065.91 ms | 687.1–731.3 | 102 (all overflow) | 412,700 | 0 |
|
||||
|
||||
Each grid's range is the initial measurement pass and the final stability
|
||||
re-run (both post-Factorio-closure, low load throughout) — the two numbers
|
||||
per cell agree within ~1-3%, confirming stability, not a warm-up or
|
||||
optimization effect.
|
||||
|
||||
**Parallel throughput (273 independent bodies, 512×256 each, Rayon
|
||||
`par_iter`)** — the real production shape (hydrology is solved once per body,
|
||||
not internally parallelized within one solve; see "Why solve() isn't
|
||||
internally parallelized" below):
|
||||
|
||||
| Run | Total | ms/body |
|
||||
|---|---|---|
|
||||
| Initial | 826.30 ms | 3.03 |
|
||||
| Stability re-run | 718.18 ms | 2.63 |
|
||||
|
||||
**273 bodies' worth of 512×256 hydrology solves complete in well under one
|
||||
second (0.7–0.8 s) on this 16-core machine.**
|
||||
|
||||
### Why `solve()` isn't internally parallelized
|
||||
|
||||
Both the priority-flood fill and the Dijkstra overflow search are globally
|
||||
sequential by construction — a shared min-heap with a strict pop order, same
|
||||
as `road_graph.rs`'s own `astar` (also unparallelized, same `BinaryHeap`
|
||||
pattern). This is architecturally identical to why `layer_proxy.rs`'s
|
||||
row-chunked `par_iter` win doesn't apply here: that speedup comes from each
|
||||
*cell* being an independent pure function (D-227); hydrology's basins and
|
||||
paths are NOT independent of each other within one body (a basin's outcome
|
||||
can depend on a lower basin having already been resolved — the chaining
|
||||
case). What DOES parallelize cleanly, and is exactly the production shape, is
|
||||
solving **different bodies'** hydrology concurrently — measured above.
|
||||
|
||||
### Determinism proof
|
||||
|
||||
`server/tests/hydrology_equilibrium_bench.rs::determinism_at_330k_cells`
|
||||
solves the 768×432 synthetic grid twice and asserts the full result
|
||||
(`filled_scaled`, `channel_depth_scaled`, `cliff_edge`, and every basin's
|
||||
`cells`/`spill_level_scaled`/`spill_cell`/`outcome`) is byte-identical across
|
||||
both solves. Passes (`cargo test --release --test hydrology_equilibrium_bench
|
||||
determinism_at_330k_cells -- --ignored --nocapture`). The module's own unit
|
||||
tests (`server/src/atlas/hydrology_equilibrium.rs`, 15 tests, `cargo test
|
||||
--lib atlas::hydrology_equilibrium`) additionally cover determinism at
|
||||
smaller synthetic grids (64×32, 96×48) including basins that classify
|
||||
Overflow and basins that classify Endorheic.
|
||||
|
||||
## The endorheic criterion (explicitly tunable)
|
||||
|
||||
```rust
|
||||
const ENDORHEIC_AREA_FLOOR: f64 = 0.004; // ~0.4% of body surface
|
||||
const ENDORHEIC_MOISTURE_CEILING: i32 = 60; // 0-100 scale
|
||||
|
||||
fn is_endorheic(area_frac: f64, moisture_q: i32) -> bool {
|
||||
area_frac >= ENDORHEIC_AREA_FLOOR && moisture_q <= ENDORHEIC_MOISTURE_CEILING
|
||||
}
|
||||
```
|
||||
|
||||
A basin is declared endorheic (closed water cycle, no outflow) when it is
|
||||
BOTH large enough that its own evaporative surface plausibly balances its
|
||||
catchment's inflow (bigger basin = more evaporating surface per unit of
|
||||
catchment — Great-Salt-Lake/Caspian-style) AND the body isn't wet enough that
|
||||
evaporation is unlikely to keep pace (`moisture_q`, mirroring
|
||||
`district_profile::derive_moisture_q`'s hydrosphere-derived ceiling before
|
||||
the per-district spatial gradient — one body-wide value, not a per-basin
|
||||
value; see "does NOT do" below). This is a **first-pass heuristic, not a
|
||||
calibrated physical model** — the two constants and the boolean-AND shape are
|
||||
placeholders. On the GJ1c body at moisture_q=55, 2 of 68 basins classified
|
||||
endorheic; on the two synthetic multi-octave grids at moisture_q=55, zero did
|
||||
(the synthetic terrain's basins are all smaller/shallower than
|
||||
`ENDORHEIC_AREA_FLOOR` at those grid resolutions). Unit tests
|
||||
(`is_endorheic_large_dry_basin_is_endorheic`,
|
||||
`is_endorheic_tiny_basin_always_overflows`,
|
||||
`is_endorheic_wet_body_always_overflows`,
|
||||
`large_dry_basin_classifies_endorheic`) lock the criterion's *shape* (both
|
||||
gates matter, area alone or moisture alone isn't enough) without pretending
|
||||
the specific constants are calibrated — that calibration is future work
|
||||
(parallel to Q-123, which already tracks provisional-pending-calibration
|
||||
magnitudes for the district moisture gradient this module's `ClimateInputs`
|
||||
borrows from).
|
||||
|
||||
## Gorge carving — the honest story (cliff representation input, red flag 4)
|
||||
|
||||
**All three production-scale benches (512×256, 768×432, 3840×2160) report
|
||||
zero carved gorge cells.** This is a genuine, investigated measurement
|
||||
result, not a code defect — verifying it required a substantial fixture
|
||||
investigation (documented in the module's own test comments,
|
||||
`server/src/atlas/hydrology_equilibrium.rs`, the "Gorge carving" test
|
||||
section) that surfaced a structural property of priority-flood itself, worth
|
||||
stating plainly for the workshop:
|
||||
|
||||
**Priority-flood finds the TRUE global minimum enclosing rim of a basin,
|
||||
swallowing any monotonically-non-decreasing connected run of cells into the
|
||||
same lake and adopting that run's own peak as the basin's real spill level.**
|
||||
Since the overflow search's carving threshold is that SAME spill level, the
|
||||
first cell the search reaches outside the lake is — by construction of how
|
||||
the spill level was computed — never higher than spill level. A single
|
||||
sealed basin's own rim, however jagged, therefore **always carves zero**
|
||||
under this algorithm: the water simply rises to meet whatever the true
|
||||
lowest crossing is. Several `solve()`-level fixtures (a basin behind a single
|
||||
ridge with a saddle; a basin behind a saddle-then-hump; a basin chained into
|
||||
a second, higher-floor basin via a monotonic ramp) were built and each
|
||||
converged on zero carved cells for exactly this reason, once traced through.
|
||||
|
||||
Genuine carving is only possible in a **narrower geometry**: two
|
||||
independently-sealed basins (each with its own valid, separately-computed
|
||||
rim) connected by a corridor that is (a) a single cell wide in *both* grid
|
||||
dimensions — not a multi-cell monotonic run the flood can absorb into either
|
||||
basin's own lake — and (b) higher than *both* basins' independent rims (else
|
||||
the two basins simply flood-merge into one, and merging means the "gorge"
|
||||
was never real, just the wrong basin count). This is real terrain (a narrow
|
||||
saddle connecting two separately-perched valleys, each already full) but
|
||||
comparatively rare at continental (512×256–3840×2160) working-grid
|
||||
resolution — consistent with all three production-scale benches finding
|
||||
zero.
|
||||
|
||||
Given the difficulty of constructing an end-to-end `solve()` fixture for
|
||||
that narrow geometry without also accidentally merging or absorbing the
|
||||
basins (four fixture iterations attempted, each correctly diagnosed as
|
||||
"basins merged" or "spill level absorbed the connector" rather than "bug"),
|
||||
the carving *mechanism itself* — `cheapest_overflow_path`'s Dijkstra search
|
||||
and the `original[cell] - spill_level` carving arithmetic — is verified
|
||||
directly with hand-constructed inputs instead
|
||||
(`cheapest_overflow_path_finds_the_true_minimum_crossing`,
|
||||
`cheapest_overflow_path_prefers_lower_total_cost_over_shorter_path`,
|
||||
`carving_arithmetic_matches_original_minus_spill_level`, all passing). These
|
||||
confirm: the search correctly routes through a genuinely-lower gap instead of
|
||||
an impassable wall; cost accumulates correctly across multiple
|
||||
above-spill-level cells in sequence (not just a single-cell threshold); and
|
||||
the exact formula `solve()` uses to populate `channel_depth_scaled`/
|
||||
`cliff_edge` is arithmetically correct in isolation.
|
||||
|
||||
The single-sealed-basin bowl fixture's own zero-carve result is asserted
|
||||
directly and non-vacuously by `single_basin_bowl_never_carves_a_gorge`
|
||||
(`cliff_edge.iter().all(|&c| !c)` + `channel_depth_scaled.iter().all(|&d| d
|
||||
== 0)`) — a PR #198 review finding (Hoshe H1) caught an earlier draft of
|
||||
this test (`cliff_edge_implies_positive_channel_depth`) asserting `if
|
||||
cliff_edge[i] { depth > 0 }` over that same fixture, which passes vacuously
|
||||
whenever (as here, always) the `if` never fires. The carving arithmetic
|
||||
itself stays covered by `carving_arithmetic_matches_original_minus_spill_level`,
|
||||
unchanged.
|
||||
|
||||
**Practical read for the workshop:** at the working-grid resolutions this
|
||||
system actually runs at, gorge carving will be a **rare, not a routine**
|
||||
event — most basins settle by simple lake-fill, not canyon-cutting. The data
|
||||
model still needs to represent it when it happens (below), but the
|
||||
cliff/multi-height case is the exception, not the common path, which may
|
||||
ease the "how often does the payload need the vertical-structure field"
|
||||
half of red flag 4's cost question (a wire-frequency question for
|
||||
measurement ④, not answered here).
|
||||
|
||||
### The cliff representation proposal
|
||||
|
||||
Grounded in what the solver actually computes (`channel_depth_scaled` +
|
||||
`cliff_edge`, both already per-cell fields on `HydrologyResult`):
|
||||
|
||||
**Dominant height + channel-depth field + cliff-edge flag** (the first option
|
||||
red flag 4 named, and the one this solver's own output shape falls out of
|
||||
directly, not chosen independently of the measurement):
|
||||
|
||||
- `elevation` (existing field) carries the **rim/dominant height** — the
|
||||
gridunit's visible/walkable surface for anything that isn't inside the
|
||||
carved channel. No change to the existing single-height payload contract
|
||||
for the common (non-gorge) case.
|
||||
- `channel_depth: u16` (or similar, quantized) — 0 everywhere except carved
|
||||
gorge cells, where it is the vertical drop from `elevation` down to the
|
||||
channel floor. This is a **direct, non-lossy carry** of
|
||||
`channel_depth_scaled` — no re-derivation needed downstream, the solver
|
||||
already computes exactly this number.
|
||||
- `cliff_edge: bool` (or foldable into a 1-bit flag alongside existing
|
||||
classification bytes) — marks the gridunit as a rim/discontinuity cell, so
|
||||
the client map-art function knows to render a cliff-face style transition
|
||||
rather than a smooth gradient at that gridunit's boundary. Also a direct
|
||||
carry of the solver's own `cliff_edge` field.
|
||||
|
||||
**Why not min/max height pair:** a min/max pair loses the *shape* of the
|
||||
transition (is the low value a point drop at one edge, or does it span the
|
||||
whole gridunit?) that `channel_depth` + `cliff_edge` preserves implicitly (a
|
||||
cliff-edge-flagged gridunit's channel floor is `elevation - channel_depth`,
|
||||
and the flag itself tells the renderer this gridunit is an edge, not an
|
||||
interior channel cell — interior channel cells one gridunit further in would
|
||||
carry the depth without the edge flag, letting the map-art function
|
||||
distinguish "this is the canyon wall" from "this is the canyon floor,
|
||||
already below the general terrain"). Min/max is also a less direct fit to
|
||||
what the solver computes — it would require deriving two synthetic heights
|
||||
per gridunit from the single `channel_depth_scaled` number the solver
|
||||
actually produces, an extra derivation step for no informational gain over
|
||||
carrying the depth directly.
|
||||
|
||||
**Scope note (does not resolve red flag 4 alone):** this proposal answers
|
||||
"what data does a carved gorge gridunit carry" — it does NOT resolve whether
|
||||
this lives on the Atlas per-gridunit payload at Phase-4 scope (a wire/schema
|
||||
decision for Araminta's ③/④ work and the workshop's synthesis round) or is
|
||||
deferred to Phase-5 in-world geometry with only a "steep" classification
|
||||
surfacing on the map. Given carving is measured as **rare** at working-grid
|
||||
resolution (above), the wire-cost argument for including a rarely-populated
|
||||
optional field is weaker than red flag 4's framing assumed — worth
|
||||
surfacing as new information for that decision, not just cost data for a
|
||||
decision already assumed necessary.
|
||||
|
||||
## What this prototype does NOT do
|
||||
|
||||
- **No stateful simulation.** No tick loop, no time-stepped erosion, no
|
||||
rainfall-accumulation-over-time model. `solve()` is called once per body
|
||||
(or per working-grid downsample) and produces a settled end-state — this
|
||||
is deliberate (the workshop premise: "deterministic equilibrium... a
|
||||
settled end-state, pure function of seed + moisture/climate"), not a
|
||||
scope gap, but stated explicitly since it bears directly on the
|
||||
"geological backstory computed at generation time" framing.
|
||||
- **No per-basin moisture/climate lookup.** `ClimateInputs::moisture_q` is a
|
||||
single body-wide value (mirroring `derive_moisture_q`'s hydrosphere-derived
|
||||
ceiling, before the per-district latitude/elevation/continentality
|
||||
gradient). A real implementation would likely want the endorheic
|
||||
criterion to read each basin's own local climate (a basin near the pole
|
||||
vs. the equator on the same body plausibly differs), which requires
|
||||
wiring in per-district or per-region climate context
|
||||
(`BodyWorldState.districts`/`regions`) — not built here.
|
||||
- **No wire/schema integration.** `HydrologyResult`/`Basin`/`BasinOutcome`
|
||||
are prototype-only types, not wired into `RiverNetwork`,
|
||||
`DrainageBasin`, or any `AtlasLayerResponse` payload. The `TERMINAL`
|
||||
sentinel reuse (above) is confirmed safe/additive but not implemented as
|
||||
a wire integration — that's downstream work once the workshop rules on
|
||||
the wire contract (measurement ④, Araminta's track).
|
||||
- **No production-scale gorge carving observed.** As detailed above, all
|
||||
three production-scale benches carved zero cells — the carving code path
|
||||
is verified correct via direct unit tests of the mechanism, not via an
|
||||
end-to-end production-scale example. Real terrain (actual planetary
|
||||
heightmaps beyond GJ1c) may or may not produce the narrow geometry that
|
||||
triggers carving more often; this prototype did not survey other bodies.
|
||||
- **No re-run against `drainage.rs`'s existing `depression_fill`/basin
|
||||
labeling for cross-validation.** This module's priority-flood is a
|
||||
structurally different (heap-based, single-pass) implementation of a
|
||||
similar problem to `drainage.rs`'s iterative relaxation; the two were not
|
||||
compared cell-for-cell against each other on the same input. They are not
|
||||
required to agree (different purposes — `drainage.rs`'s fill feeds D8 flow
|
||||
direction for river extraction; this module's fill feeds basin/spill
|
||||
detection), but a reader should not assume they'd produce identical
|
||||
filled surfaces.
|
||||
- **No chunk-boundary/step-boundary decomposition.** The solver runs on a
|
||||
single contiguous elevation grid start-to-finish (matching how
|
||||
`drainage::analyze` is called today — once per body, on the Layer-1
|
||||
512×256 working grid). Whether/how hydrology decomposes across the
|
||||
step-ladder's per-step canvases (premise 8: "step boundaries = compute-
|
||||
chunk boundaries") is a synthesis-round question, not addressed here —
|
||||
the measured costs above are for a full single-grid solve at each listed
|
||||
size, not a chunked partial re-solve.
|
||||
- **`RIVER_THRESHOLD` reused, not re-derived per body.** The gorge-carving
|
||||
flow-accumulation gate uses the same fixed `RIVER_THRESHOLD = 200`
|
||||
`drainage.rs` uses globally — not the per-district `river_threshold`
|
||||
DistrictProfile carries downstream (D-239 §1). Fine for a prototype
|
||||
measuring solver cost; a production integration should probably use the
|
||||
same per-district value other consumers do.
|
||||
|
||||
## Verdict
|
||||
|
||||
**Per-body-open settled hydrology is viable.** Headline numbers: 512×256
|
||||
solves in ~24 ms single-threaded; the full 4K-class 8.3M-cell grid solves in
|
||||
~5.7 s single-threaded (a genuine, not extrapolated, "computer catches fire"
|
||||
ceiling case — but note no production path actually derives hydrology at
|
||||
8.3M cells synchronously; see below); and the realistic production shape —
|
||||
273 independent bodies at the real 512×256 working-grid size, run in
|
||||
parallel across the Rayon pool — completes in well under a second (0.7–0.8
|
||||
s). This closes the exact gap the T-1143 planetary rung died of: the
|
||||
biggest-unknown, highest-priority measurement is now a real number, not an
|
||||
extrapolation, and it does not invalidate the "settled hydrology" premise.
|
||||
|
||||
**Caveat worth carrying into the workshop:** the 8.3M-cell single-solve
|
||||
number (~5.7 s) is a genuine cost that would matter if any single request
|
||||
path needed hydrology re-solved synchronously at that size on demand. Given
|
||||
this module's `solve()` isn't currently wired into any request path (see
|
||||
"does NOT do" above), whether that number is ever actually paid at request
|
||||
time — or only ever paid once per body, offline/precomputed, the same way
|
||||
`drainage::analyze` already runs once per body today — is a caching/
|
||||
architecture question for the workshop's cache-tier synthesis (Dudley's
|
||||
question 4 / red flag 2), not a solver-cost question this measurement can
|
||||
settle alone.
|
||||
@@ -0,0 +1,521 @@
|
||||
---
|
||||
title: "T-1178/T-1154 — per-gridunit derive bench at real canvas sizes + block/tile rung costs (measurements ②/③)"
|
||||
workshop: body-map-viewer
|
||||
status: complete
|
||||
owner: Dudley
|
||||
---
|
||||
|
||||
# T-1178/T-1154 — derive bench at real canvas sizes + the ladder's bottom rungs
|
||||
|
||||
Measurements ②/③ of the [body-map-viewer workshop
|
||||
brief](../body-map-viewer-workshop-brief.md)'s pre-workshop appendix, per
|
||||
[tyre-implications.md §3.2](../tyre-implications.md) item 2 and item 4. This is
|
||||
the anti-extrapolation gap the T-1143 planetary-rung post-mortem named by name:
|
||||
does the row-chunked `par_iter` throughput measured at the shipped 4,096-cell
|
||||
window ceiling (`docs/architecture/atlas-zoom-ladder-t1143.md` §7: 1.785
|
||||
µs/cell District cutoff, 1.454 µs/cell orbital, 0.617 ms/served-window) HOLD
|
||||
at real step-canvas sizes (330K–8.3M cells), and what does the ladder's
|
||||
un-costed bottom (block/tile) actually cost. Every number below is labelled
|
||||
**MEASURED** or **ARITHMETIC** — nothing is extrapolated from a smaller
|
||||
measured size to a larger unmeasured one.
|
||||
|
||||
## Environment note (read first)
|
||||
|
||||
A background Factorio process was running on this machine for part of this
|
||||
session and may have starved CPU before and during initial harness
|
||||
compilation. **It was closed partway through this session, before any timing
|
||||
measurement in this document was captured or kept.** Concretely:
|
||||
|
||||
- Every `ms`/`ns` figure in this document either post-dates the closure, or is
|
||||
an explicit **stability re-run performed after it** (see "Stability
|
||||
re-runs" below) — no pre-closure number is reported anywhere in this doc.
|
||||
- Load average was watched (`uptime`) before every timed run; each run below
|
||||
was started only once the 1-minute load average had settled to single
|
||||
digits on this 16-core machine (typically 1.3–5.0), never during the
|
||||
post-closure settling spike (which peaked ~25 immediately after closure and
|
||||
fell within ~2 minutes).
|
||||
- Per the coordinator's instruction: any early-slow/later-fast delta observed
|
||||
during this session is **not reported as a chunking or warm-up finding** —
|
||||
it is attributed to environmental noise and excluded. The findings below
|
||||
(parallel throughput holding flat from 330K→8.3M) are confirmed by
|
||||
re-running the headline cases a second time on the quiet machine, not by a
|
||||
single early/late comparison.
|
||||
- The baseline figures this document compares against (1.785 µs/cell
|
||||
District, 1.454 µs/cell orbital, 0.617 ms/served-window,
|
||||
`atlas-zoom-ladder-t1143.md` §7) were measured in an **earlier session**
|
||||
under unknown load conditions. Where this session's quiet-machine numbers
|
||||
come out *faster* than that baseline (they do, substantially — see below),
|
||||
that is flagged explicitly as an environment-conditions caveat, not claimed
|
||||
as a genuine speedup from any code change. No code in the derive path
|
||||
changed between sessions.
|
||||
|
||||
## What was measured
|
||||
|
||||
**Body/fixtures:** two independent fixtures, deliberately not just one:
|
||||
|
||||
1. **Synthetic gradient body** (`bench_hm()`/`bench_ta()`, same fixture
|
||||
`zoom_ladder_bench.rs` uses) — 128×64 working grid, deterministic smooth
|
||||
gradient, no PNG I/O. Body id `"bench"`, seed
|
||||
`SeedChain::root(99).derive(SeedDomain::Body, 1)`. Used for every sweep
|
||||
below except the two crosschecks.
|
||||
2. **Real committed GJ1c heightmap** (`wiki/star-systems/GJ-1/bodies/GJ1c/heightmap.png`,
|
||||
downsampled to the production 512×256 working grid, real river network via
|
||||
`drainage::analyze`), seed `SeedChain::root(0xC0FFEE_u64).derive(SeedDomain::Body, 7)`
|
||||
— used for one 330K-cell cross-check to confirm the synthetic-fixture
|
||||
numbers are not an artifact of a trivial gradient or an empty river
|
||||
network. The window is centred on a real GJ1c river cell (not the world
|
||||
origin — an origin-centred window on GJ1c measured `courses_in_window=0`
|
||||
on a first attempt and was corrected; see the bench source's doc comment).
|
||||
|
||||
**Hardware:** 16 logical cores (Intel Core i9-9900K @ 3.60GHz),
|
||||
`std::thread::available_parallelism() == 16`, no `RAYON_NUM_THREADS` override
|
||||
— Rayon's global default pool uses all 16 (`rayon::current_num_threads()`
|
||||
confirmed `16` in every parallel run's own printed output, reported inline
|
||||
per-run below rather than assumed).
|
||||
|
||||
**Code:** new file `server/tests/bmv_gridunit_bench.rs` (per the batch
|
||||
instruction — `server/src/atlas/mod.rs` untouched, `zoom_ladder_bench.rs`
|
||||
untouched, this workshop-gate pass kept in its own file since it measures a
|
||||
different SHAPE question than that file's fixed-4,096-cell sweeps). All
|
||||
benches `#[ignore]`d release tests, `cargo test --release --test
|
||||
bmv_gridunit_bench -- --ignored --nocapture`.
|
||||
|
||||
**Invocation used for every number below** (run individually per test name,
|
||||
not `-- --ignored` for the whole file at once, so each run's wall time isn't
|
||||
diluted by unrelated tests still warming up):
|
||||
|
||||
```bash
|
||||
cd server
|
||||
cargo test --release --test bmv_gridunit_bench <test_name> -- --ignored --nocapture
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## T-1178 (Measurement ②) — does par_iter throughput hold at real canvas sizes?
|
||||
|
||||
### The `n`-cap problem — what "production path" means here
|
||||
|
||||
`build_district_window_layer`'s CALLER (`handle_atlas_request` via
|
||||
`clamp_window_n_v2`) hard-clamps `n` so the derived cell grid never exceeds
|
||||
`WIRE_CAP_CELLS = 4,096` — **a real client can never request a 330K-cell
|
||||
window over the wire, by design** (D-226 T-1124 §2, the windowed-family
|
||||
ceiling). So "run it through the production path at 330K/8.3M cells" cannot
|
||||
mean "send an `AtlasLayerRequest` for that size" — no such request is legal.
|
||||
|
||||
Confirmed by direct read this session
|
||||
(`server/src/atlas/layer_proxy.rs:1515-1628`): `build_district_window_layer`
|
||||
itself has **no internal clamp** — the clamp lives one layer up, in the
|
||||
request handler. So this measurement calls the actual, unmodified
|
||||
`build_district_window_layer` function directly — same signature, same
|
||||
row-chunked `into_par_iter()` loop (`layer_proxy.rs:1564-1586`), same
|
||||
`derive_window_cell`/`scatter_row` internals, same `derive_at_metres` calls —
|
||||
at an `n` no wire request could carry. This is exactly what the ticket
|
||||
anticipates ("bench the underlying chunked loop at those counts and say
|
||||
exactly what you ran").
|
||||
|
||||
### Square vs. rectangular canvas
|
||||
|
||||
`build_district_window_layer` only derives SQUARE `side×side` grids (`n` is a
|
||||
single extent — no rectangle parameter exists). The named real step-canvas
|
||||
shapes are 16:9 rectangles. Two measurements were taken per cell-count
|
||||
target:
|
||||
|
||||
1. **Square, through the real `build_district_window_layer` function**
|
||||
(nearest square side to the target cell count: 576²=331,776,
|
||||
1440²=2,073,600, 2880²=8,294,400 — landing exactly on 330K/2.07M/8.3M),
|
||||
with a **real `RiverNetwork` passed in** — this path runs the actual
|
||||
course-invention + riparian-cull machinery
|
||||
(`invent_courses_near_window`/`crop_courses_for_wire`) production runs,
|
||||
and is genuinely courses-inclusive (see the courses column below).
|
||||
2. **Real 16:9 rectangle** (768×432, 1920×1080, 3840×2160), via a
|
||||
hand-written row-chunked replica loop that mirrors
|
||||
`build_district_window_layer`'s internals cell-for-cell (same
|
||||
`into_par_iter()` row chunking, same `derive_at_metres` call per cell,
|
||||
same per-cell output-field quantization `derive_window_cell` performs) —
|
||||
necessary because no production entry point derives a non-square window.
|
||||
Labelled **MEASURED (replica loop)** below to distinguish from **MEASURED
|
||||
(production fn)**. **This path is courses-EMPTY by construction** — see
|
||||
"Courses-inclusion disclosure (PR #198 review findings H2/H3)" below.
|
||||
|
||||
### Headline table — District spacing, cutoff=2,048m (the shipped District band)
|
||||
|
||||
| Canvas | Cells | Path | Courses in window | Parallel (16 threads) | ns/cell parallel | Single-thread | ns/cell single-thread | Speedup |
|
||||
|---|---:|---|---:|---:|---:|---:|---:|---:|
|
||||
| 330K (square, side=576) | 331,776 | MEASURED (production fn) | 3 | 63.69 ms | 192.0 | 547.45 ms | 1,650.1 | 8.60× |
|
||||
| 330K (768×432) | 331,776 | MEASURED (replica loop) | 0 (forced) | 63.58 ms | 191.6 | 544.51 ms | 1,641.2 | 8.56× |
|
||||
| 2.07M (square, side=1440) | 2,073,600 | MEASURED (production fn) | 6 | 394.61 ms | 190.3 | 3,436.18 ms | 1,657.1 | 8.71× |
|
||||
| 2.07M (1920×1080) | 2,073,600 | MEASURED (replica loop) | 0 (forced) | 423.20 ms | 204.1 | 3,383.64 ms | 1,631.8 | 7.99× |
|
||||
| 8.3M (square, side=2880) | 8,294,400 | MEASURED (production fn) | 10 | 1,827.34 ms | 220.3 | 13,767.58 ms | 1,659.9 | 7.53× |
|
||||
| 8.3M (3840×2160) | 8,294,400 | MEASURED (replica loop) | 0 (forced) | 1,731.38 ms | 208.7 | 13,584.42 ms | 1,637.8 | 7.85× |
|
||||
|
||||
**All six rows MEASURED — none extrapolated.** Parallel per-cell rate is flat
|
||||
across every canvas size (190.3–220.3 ns/cell, the full spread is ~15% —
|
||||
consistent with run-to-run scheduling noise on the same machine, not a trend;
|
||||
see "Stability re-runs"). **This directly answers T-1178's central question:
|
||||
the row-chunked `par_iter` throughput measured at 4,096 cells holds at 330K,
|
||||
2.07M, and 8.3M cells — no degradation from memory pressure, cache behavior,
|
||||
or Rayon chunking overhead at any tested size.**
|
||||
|
||||
### Courses-inclusion disclosure (PR #198 review findings H2/H3)
|
||||
|
||||
Hoshe's review found two related gaps in the original version of this
|
||||
document and the underlying bench, confirmed valid and fixed here:
|
||||
|
||||
**H2 — the replica-loop path was undisclosed as courses-empty.**
|
||||
`rect_window_replica` (the function backing every rectangular-canvas row
|
||||
above, and the 83K deep-step bench below) always calls `derive_at_metres`
|
||||
with an empty `&[]` course slice — it has no `RiverNetwork` wiring at all.
|
||||
This was true of the original bench and was not stated anywhere in the code
|
||||
or this document. **Fixed:** both the bench's module doc and the
|
||||
`rect_window_replica` function doc now state this explicitly, and cite the
|
||||
measured cost this excludes: `zoom_ladder_bench.rs`'s own
|
||||
`bench_course_cost_on_vs_off` measured the courses-on-vs-off delta at
|
||||
District cap (n=64, real GJ1c geometry) as **+0.09–0.21 ms against a ~5 ms
|
||||
baseline (under 5%)** — small, but real, and every rectangular-canvas number
|
||||
in this document (the three 16:9 headline rows AND the 83K deep-step bench)
|
||||
excludes it.
|
||||
|
||||
**H3 — the "square vs. rectangle converge" claim was unverified for courses,
|
||||
and the specific worry (both sides courses-empty) turned out not to be the
|
||||
case.** The original document claimed square and rectangle "converge... the
|
||||
replica loop is a faithful stand-in" without ever checking whether the
|
||||
square path (which DOES accept a real `RiverNetwork`) was actually deriving
|
||||
any courses at its measured window position — raising the possibility that
|
||||
the observed convergence was courses-empty vs courses-empty, which would
|
||||
prove nothing about courses cost. **Fix applied and run:** added a
|
||||
`layer.courses.len()` print to `bench_square_window_production_fn_district_spacing`,
|
||||
rebuilt, and ran it in isolation (release, `--ignored`, the three square
|
||||
cases only — no need to re-run the 8.3M single-thread or rectangular rows for
|
||||
this check). **Result: the square path is NOT courses-empty** — it measured
|
||||
`courses_in_window = 3, 6, 10` at 330K, 2.07M, 8.3M respectively (confirmed
|
||||
deterministic across two separate runs, identical counts both times). This
|
||||
happens because the synthetic gradient body's own drainage analysis produces
|
||||
river cells near the world origin (where the square benches centre their
|
||||
window), unlike GJ1c's real geometry which measured `courses_in_window=0` at
|
||||
the origin on a first attempt (see Cross-check 1 below, which corrects for
|
||||
exactly this by centring on a real river cell instead).
|
||||
|
||||
**Corrected convergence statement (replaces the retracted "faithful stand-in"
|
||||
sentence):** the square production-fn path is courses-INCLUSIVE at light
|
||||
density (3–10 courses in a 331,776–8,294,400-cell window) and the
|
||||
rectangular replica-loop path is courses-EMPTY (always `&[]`, by
|
||||
construction) — so their agreement at 331,776 ≈ 576² (191.1–191.6 ns/cell,
|
||||
both paths, within 0.3% of each other) validates the **row-chunked loop
|
||||
mechanics** (chunking granularity, per-row dispatch overhead, per-cell derive
|
||||
cost) converging across two independently-written call sites at this course
|
||||
DENSITY — it does **not** demonstrate courses-inclusive and courses-empty
|
||||
rates are equivalent in general. The two paths differ in exactly one
|
||||
respect (courses present-but-sparse vs absent) and land within run-to-run
|
||||
noise of each other specifically because 3–10 courses out of 331,776+ cells
|
||||
is far too sparse a fraction to move the aggregate ns/cell figure outside
|
||||
the noise band — consistent with, not contradicting, the <5% per-cell
|
||||
course-cost delta `zoom_ladder_bench.rs` measured directly on a window with
|
||||
much higher course density. **The courses-inclusive rate at REAL production
|
||||
course density is covered only by Cross-check 1 below** (18 courses in a
|
||||
331,776-cell window, deliberately centred on real river geometry, not this
|
||||
document's sparse origin-centred windows) — cite that number, not the
|
||||
headline table, for a courses-representative rate.
|
||||
|
||||
**Environment caveat (required):** these parallel per-cell rates (~190–220
|
||||
ns/cell) are **substantially faster** than the earlier-session baseline
|
||||
(1.785 µs/cell District cutoff at 4,096 cells, `atlas-zoom-ladder-t1143.md`
|
||||
§7) — roughly 8–9× faster per cell. This gap is **not** claimed as a
|
||||
per-cell speedup from anything measured this session; single-thread numbers
|
||||
here (1,631–1,660 ns/cell) land very close to that 1.785 µs/cell baseline,
|
||||
which is the correct like-for-like comparison (the 4,096-cell baseline bench
|
||||
in `zoom_ladder_bench.rs` measures a *serial* sweep, not the row-chunked
|
||||
parallel path). The apparent "speedup" versus the baseline is explained by
|
||||
comparing the RIGHT numbers to each other: single-thread here ≈ the earlier
|
||||
serial baseline (within ~8%, plausibly environment/CPU-generation
|
||||
differences between sessions), and parallel here is the genuinely new
|
||||
row-chunked-at-scale number the baseline table never measured. No
|
||||
apples-to-oranges claim is made anywhere in this table.
|
||||
|
||||
### Cross-check 1 — real GJ1c body, real river network, courses genuinely exercised at production-representative density
|
||||
|
||||
| Canvas | Path | Wall time | ns/cell | courses_in_window |
|
||||
|---|---|---:|---:|---:|
|
||||
| 330K (square, side=576, GJ1c) | MEASURED (production fn) | 64.69 ms | 195.0 | 18 |
|
||||
|
||||
Window centred at district `(7520, -2932)` (converted from a real GJ1c river
|
||||
cell, `pixel_to_world_m`-equivalent formula, same technique
|
||||
`zoom_ladder_bench.rs`'s `bench_course_cost_on_vs_off` uses) — 18 real invented
|
||||
river courses fall inside the window (nearly 2× the headline table's
|
||||
synthetic-body density at the same shape, 10 courses at 8.3M being the
|
||||
highest count that table reaches), confirming this is not a courses-off
|
||||
measurement by accident and giving the most production-representative course
|
||||
density measured in this document. **195.0 ns/cell — within 2% of the
|
||||
synthetic fixture's 192.0 ns/cell at the identical shape, and within the
|
||||
same noise band the headline table's own re-runs show.** This is the number
|
||||
to cite for "courses-inclusive, real geometry, real density" — the headline
|
||||
table's square rows are courses-inclusive but sparse (H3, above), and the
|
||||
rectangular rows are courses-empty entirely (H2, above).
|
||||
|
||||
### Cross-check 2 — independent measurement, different body, same finding (T-1179)
|
||||
|
||||
Araminta's T-1179 wire-size measurement (`t1179-wire-table.md`, same
|
||||
workshop, run independently on body GJ338Bd/seed `"yolo"`, a THIRD distinct
|
||||
body from either fixture used in this document) reports derivation wall time
|
||||
for context: 330K → 74.8 ms, 2.07M → 513.0 ms, 8.3M → 1,779.6 ms — implying
|
||||
225.5 / 247.4 / 214.6 ns/cell respectively. This lands in the same band as
|
||||
both fixtures measured in this document (190–220 ns/cell) despite a fully
|
||||
independent harness run, different body, different seed, different session.
|
||||
**Three independent measurements (synthetic gradient, GJ1c real body, GJ338Bd
|
||||
real body via T-1179) all agree the row-chunked parallel rate sits in the
|
||||
~190–250 ns/cell band at every canvas size from 330K to 8.3M — this is now a
|
||||
robust, cross-validated finding, not a single-run artifact.**
|
||||
|
||||
### Stability re-runs (required by the coordinator's environment note)
|
||||
|
||||
The two headline production-fn tests were re-run a second time, after the
|
||||
first full pass, specifically to confirm the numbers above are not an
|
||||
artifact of residual post-Factorio settling:
|
||||
|
||||
| Run | 330K ns/cell | 2.07M ns/cell | 8.3M ns/cell |
|
||||
|---|---:|---:|---:|
|
||||
| First pass | 205.9 | 190.8 | 192.9 |
|
||||
| Re-run | 192.0 | 190.3 | 220.3 |
|
||||
|
||||
Both runs land in the same 190–220 ns/cell band; the 8.3M case shows the
|
||||
largest run-to-run spread (192.9 → 220.3, ~14%), consistent with normal
|
||||
scheduling noise on a 16-thread parallel workload rather than a systematic
|
||||
trend. The 83K deep-step case (T-1154, below) was also re-run and landed
|
||||
within 1% (17.04 ms → 17.26 ms). **No early-slow/later-fast pattern was
|
||||
observed in either re-run — both quiet-machine passes agree.**
|
||||
|
||||
---
|
||||
|
||||
## T-1154 (Measurement ③) — pricing the ladder's bottom (block/tile rungs)
|
||||
|
||||
### Per-cell derive cost at block (128m) and tile-adjacent (1–4m) spacing
|
||||
|
||||
`WindowGranularity` (the request-facing rung enum) has exactly three
|
||||
variants — Quarter, District, Region — confirmed by direct read
|
||||
(`layer_proxy.rs:179-276`); there is no Block or Tile variant. There is also
|
||||
no wire-facing cutoff band below 1,024m: `MIN_WL_BANDS_M`
|
||||
(`layer_proxy.rs:451-458`, the quantized set a real request's
|
||||
`window_min_wl_m` snaps to) bottoms out at `2 × QUARTER_M = 1,024m` — Block
|
||||
(128m) and Tile (1m) have **no existing request-facing path at all**. This
|
||||
measurement therefore calls `derive_at_metres` directly with spacing/cutoff
|
||||
values no real window request can carry today, at the fixed 4,096-cell
|
||||
(64×64) sweep shape `zoom_ladder_bench.rs` already uses for District/Quarter
|
||||
— so the new numbers slot directly into that table.
|
||||
|
||||
| Sweep (64×64 = 4,096 cells, release) | Total | ns/cell | µs/cell |
|
||||
|---|---:|---:|---:|
|
||||
| Block (128 m), cutoff=128m | 7.53 ms | 1,838.9 | 1.839 |
|
||||
| Tile-adjacent (4 m), cutoff=4m | 7.75 ms | 1,892.2 | 1.892 |
|
||||
| Tile (1 m), cutoff=1m | 7.39 ms | 1,804.7 | 1.805 |
|
||||
|
||||
**MEASURED, re-run once for stability** (first pass: 1,844.1 / 1,802.4 /
|
||||
1,798.5 ns/cell; second pass, shown in the table above: 1,838.9 / 1,892.2 /
|
||||
1,804.7 ns/cell — both passes agree within ~5%, no trend): all three land in
|
||||
the same band as the existing District (1.785 µs/cell) and Quarter
|
||||
(1.823 µs/cell) rows already in `atlas-zoom-ladder-t1143.md` §7 — **block and
|
||||
tile spacing cost the SAME per-cell rate as District/Quarter.** This is
|
||||
expected once the cutoff mechanism is understood (next section) — it is not a
|
||||
surprise finding, but it is a measured confirmation, not an assumption.
|
||||
|
||||
### Why the cutoff buys nothing at Block/Tile spacing (verified, not assumed)
|
||||
|
||||
Direct read of `enveloped_fbm` (`detail_scatter.rs:198-251`): a cutoff skips
|
||||
an octave only if `wl < min_wavelength_m` — **strictly finer than the
|
||||
cutoff**. `VOXEL_OCTAVE_WAVELENGTHS_M` (the sub-district relief band
|
||||
`voxel_relief` sums, `detail_scatter.rs:46`) is `[1_024, 512, 256, 128]` — its
|
||||
FINEST entry is exactly 128m. At a cutoff of 128m (Block's own Nyquist
|
||||
floor), every entry in every octave array used by `derive_at_metres` is `≥
|
||||
128m`, so **nothing is skipped** — Block sits at the bottom of the invented
|
||||
octave stack with nothing finer left to truncate. Measured directly (all runs
|
||||
at Block-spaced sample positions, only the cutoff VALUE varies):
|
||||
|
||||
| Cutoff value (position fixed at Block spacing) | Total (4,096 cells) | ns/cell |
|
||||
|---|---:|---:|
|
||||
| uncut (cutoff=0) | 7.66 ms | 1,871.3 |
|
||||
| cutoff=128m (Block's own floor) | 7.66 ms | 1,871.2 |
|
||||
| cutoff=2,048m (District-coarse — truncates the VOXEL band) | 6.37 ms | 1,554.2 |
|
||||
| cutoff=204,800m (Region-coarse — truncates everything) | 3.99 ms | 974.5 |
|
||||
|
||||
**MEASURED, re-run once (re-run: 1,871.3 / 1,871.2 / 1,554.2 / 974.5 — within
|
||||
noise of the first pass shown above).** Cutoff=0 vs cutoff=128 differ by
|
||||
<0.01% (measurement noise, not a real difference) — confirms the "no
|
||||
truncation work left at Block/Tile" finding directly, rather than inferring
|
||||
it from the code alone. The cutoff mechanism DOES buy real savings, but only
|
||||
at COARSER rungs (District/Region) where it truncates fine detail those
|
||||
rungs' own sample density can't resolve — it has nothing to give at Block or
|
||||
finer, because the const octave arrays this codebase ships bottom out at
|
||||
128m.
|
||||
|
||||
**Consequence for the ladder's floor:** every rung from Block down to Tile
|
||||
pays the full, un-truncated per-cell derive cost (~1.8–1.9 µs/cell,
|
||||
single-thread; see the deep-step canvas below for the parallel figure) —
|
||||
there is no "it gets cheaper as you zoom in past Block" effect from the
|
||||
cutoff mechanism as currently built. Any future finer-than-128m invented
|
||||
detail (a genuine Tile-scale octave band) would need a NEW const array with
|
||||
entries below 128m — `MOSAIC_OCTAVE_WAVELENGTHS_M` (`[64, 32, 16, 8]`)
|
||||
already exists in the source but is not wired into this call graph (see next
|
||||
section) and could be a starting point if that work is ever scoped.
|
||||
|
||||
### voxel_relief / voxel_mosaic — exercised vs. bypassed (read, not refactored)
|
||||
|
||||
One paragraph, per the ticket's scope constraint (read the code, note
|
||||
reusability, no refactoring):
|
||||
|
||||
`voxel_relief` **is already exercised** by the Atlas derive path — it's
|
||||
called unconditionally inside `invent_primitives`
|
||||
(`district_profile.rs:1197-1204`), gated purely by the same
|
||||
`min_wavelength_m` cutoff every other primitive uses, contributing to
|
||||
`elev_q`/`slope_q` alongside the district-band `terrain_detail` scatter. No
|
||||
extraction is needed for Block-rung Atlas sampling — `derive_at_metres`
|
||||
already produces genuinely-varying sub-district relief at Block spacing today
|
||||
(confirmed: the `voxel_relief_band_fully_excluded_at_district_floor` and
|
||||
`voxel_relief_varies_at_sub_district_scale` tests in the existing suite pin
|
||||
exactly this behavior). `voxel_mosaic`, by contrast, **is bypassed entirely**
|
||||
by the Atlas path — its only caller in the whole codebase is `voxel.rs`
|
||||
(confirmed by grep across `server/src/atlas/*.rs`), the actual Phase-4
|
||||
tile-fill generator, a fully separate code path from `derive_at_metres`/
|
||||
`invent_primitives`. It selects sub-chunk micro-habitat classes for real
|
||||
voxel fill, not Atlas classification, and has never been wired into the
|
||||
Atlas derive stack. If a future Tile-scale Atlas rung wants
|
||||
`voxel_mosaic`-driven detail (e.g. showing micro-habitat patches at the
|
||||
deepest zoom step), that is new wiring work, not a reuse of an existing path
|
||||
— currently `derive_at_metres`'s Tile-spacing output differs from
|
||||
`voxel.rs`'s actual tile-fill output by exactly the `voxel_mosaic`
|
||||
contribution, which the Atlas never sees.
|
||||
|
||||
### The realistic deep-step canvas
|
||||
|
||||
**Geometry (stated exactly, per the ticket's requirement):** at the ~10
|
||||
px-per-1m-tile bottom-out on a 3840×2160 canvas, smaller axis 2160 px: `2160
|
||||
px ÷ 10 px/tile = 216 m` on the smaller (row) axis, `3840 px ÷ 10 px/tile =
|
||||
384 m` on the larger (column) axis, both at 1 m spacing. `216 × 384 = 82,944`
|
||||
cells — the ticket's "~83K cells" figure confirmed exactly. This is
|
||||
distinct from the ticket-text's alternate 1920×1080-based reading (which
|
||||
would give 108×192 = 20,736 cells); the 3840×2160/2160-smaller-axis reading
|
||||
is used here because it matches the ticket's own stated ~83K target exactly.
|
||||
|
||||
| Path | Wall time | ns/cell | Speedup |
|
||||
|---|---:|---:|---:|
|
||||
| PARALLEL (16 threads, row-chunked) | 17.04 ms | 205.4 | — |
|
||||
| SINGLE-THREAD | 149.79 ms | 1,805.9 | — |
|
||||
| — | — | — | 8.79× |
|
||||
|
||||
**MEASURED, re-run for stability: 17.26 ms / 208.1 ns/cell / 8.57× speedup —
|
||||
within 1% of the first pass.** 1m spacing, cutoff=1m (Nyquist-matched).
|
||||
**17ms parallel for the deepest step's realistic viewport window is
|
||||
comfortably inside any interactive-latency budget** — an order of magnitude
|
||||
below a single frame at even 30 fps (33ms), let alone a step-cross tolerance
|
||||
of a few hundred ms.
|
||||
|
||||
**Courses disclosure (H2, applies here too):** this bench runs through
|
||||
`rect_window_replica`, the same courses-force-empty replica loop as the
|
||||
16:9 headline rows above — this 17ms figure **excludes** the per-cell
|
||||
riparian-test cost a real Tile-rung window with nearby river geometry would
|
||||
pay (measured elsewhere as +0.09–0.21ms against a ~5ms District-cap
|
||||
baseline, under 5%). At 82,944 cells the excluded cost would be smaller in
|
||||
absolute terms than that District-cap figure (far fewer cells, and course
|
||||
density at Tile spacing over a 216m×384m window is not yet measured at
|
||||
all), so this is very unlikely to change the "comfortably interactive"
|
||||
verdict — but it is an excluded cost, not a zero one, and is stated as such
|
||||
rather than left implicit.
|
||||
|
||||
### Wire bytes at block/tile granularities (ARITHMETIC)
|
||||
|
||||
**ARITHMETIC**, using `DistrictWindowLayer`'s own documented per-cell cost (7
|
||||
raw bytes: 1+1+2+1+1+1 for morphology/elev_q/temp_dc/moisture_q/vegetation/
|
||||
glaciation, `layer_proxy.rs:794-798`) — the same six-field set every other
|
||||
rung on the ladder carries; block/tile would ship the identical field set,
|
||||
not a richer one, per the "same classification pipeline, no rung gets its own
|
||||
decision-boundary table" discipline already established for Quarter
|
||||
(T-1150 design doc §6):
|
||||
|
||||
| Shape | Cells | Bytes (7 B/cell, ARITHMETIC) |
|
||||
|---|---:|---:|
|
||||
| 83K deep-step window (216×384 @ 1m) | 82,944 | 580,608 B ≈ **567.0 KB** |
|
||||
| 330K canvas (768×432) | 331,776 | 2,322,432 B ≈ **2.21 MB** |
|
||||
| 8.3M canvas (3840×2160) | 8,294,400 | 58,060,800 B ≈ **55.37 MB** |
|
||||
|
||||
**Cross-reference (measured elsewhere, not this document):** T-1179's actual
|
||||
`rmp_serde` measurement at 330K found **6.00 bytes/cell**, not 7 — MessagePack's
|
||||
compact `bin` framing for `Vec<u8>`/`Vec<i16>` fields lands under the naive
|
||||
per-field byte sum. Using that MEASURED rate instead of the ARITHMETIC 7-byte
|
||||
figure: 83K ≈ 486 KB, 330K ≈ 1.90 MB, 8.3M ≈ 47.4 MB — all ~14% lower than
|
||||
the arithmetic table above. Both figures are reported; the arithmetic table
|
||||
uses the documented per-field byte width (conservative, easy to sanity-check
|
||||
by hand), the cross-reference uses T-1179's actual wire measurement (tighter,
|
||||
but sourced from a different document's harness, not this one's).
|
||||
|
||||
Every shape here is **already far over the existing `WIRE_CAP_CELLS = 4,096`
|
||||
(≈30 KB) windowed-payload ceiling** by construction — none of these numbers
|
||||
are a new finding about the ceiling (T-1179 covers that argument in full);
|
||||
they are included here only so the block/tile go/no-go call below has its
|
||||
own wire-cost context without a cross-document lookup.
|
||||
|
||||
---
|
||||
|
||||
## Go/no-go recommendations per rung
|
||||
|
||||
### Block (128 m spacing)
|
||||
|
||||
**GO for interactive per-step serving, cost-wise.** Per-cell derive cost
|
||||
(~1.8–1.9 µs/cell single-thread, same band as District/Quarter) and parallel
|
||||
throughput (same ~190–220 ns/cell band measured at every canvas size this
|
||||
session) are not the blocker. A Block-spacing step canvas at any of the three
|
||||
named sizes (330K/2.07M/8.3M) derives in the same tens-to-low-thousands of ms
|
||||
the District/Quarter rungs already ship at production scale (1,731–1,827 ms
|
||||
parallel at 8.3M, the largest canvas tested — well inside a step-cross
|
||||
tolerance measured in hundreds of ms, not requiring sub-frame latency since a
|
||||
step-cross is a discrete, user-visible transition per the stepped-zoom
|
||||
premise, not a per-frame redraw). **The blocker, if any, is the wire
|
||||
carrier** (T-1179's finding: even the best encoding is tens to hundreds of
|
||||
times over the existing windowed-payload ceiling) and the D-226(d)
|
||||
canonical-vs-viewport governance boundary (red flag 3) — neither is a derive-
|
||||
cost question this measurement can resolve, and both apply identically
|
||||
whether the rung is District, Quarter, or Block. Cost is not what would say
|
||||
no to Block.
|
||||
|
||||
### Tile (1–4 m spacing)
|
||||
|
||||
**GO for interactive per-step serving, cost-wise, WITH the viewport-window
|
||||
qualifier already baked into the D-226(d) opening.** The realistic deep-step
|
||||
window (83K cells, the actual viewport-sized shape a Tile-rung step canvas
|
||||
would need, not a whole-body canvas) derives in **17 ms parallel** — trivially
|
||||
interactive. This is the single number this measurement pass most needed to
|
||||
produce: it directly prices the "viewport-sized keeps the deep ladder legal
|
||||
by construction" model tyre-implications.md red flag 3 describes, and the
|
||||
answer is comfortably affordable. **Caveat, not a cost caveat:** this GO is
|
||||
conditional on the window staying viewport-sized (the 83K shape, not a
|
||||
canonical whole-body 1m-spacing canvas) — a canonical fixed canvas at Tile
|
||||
spacing would be a near-whole-body metre-resolution derivation, which is a
|
||||
governance violation (D-226(d)) independent of and unaffected by this
|
||||
measurement's cost numbers. Cost says GO; the canonical-vs-viewport policy
|
||||
decision (not cost) is what keeps that GO inside the legal boundary.
|
||||
|
||||
### Cross-cutting note for both rungs
|
||||
|
||||
Neither Block nor Tile shows any cutoff-driven cost SAVINGS relative to
|
||||
District/Quarter (see the cutoff-confirms-savings table above) — both pay
|
||||
the full per-cell rate, always. This does not change either GO call (the
|
||||
full rate is still cheap enough), but it does mean **the ladder gets no
|
||||
cheaper as it gets finer past Block** — total per-canvas cost at Block and
|
||||
Tile is governed entirely by cell count (canvas size × spacing), not by any
|
||||
octave-truncation discount the coarser rungs enjoy. Any future capacity
|
||||
planning for these rungs should size off the flat ~1.8 µs/cell single-thread
|
||||
/ ~200 ns/cell parallel rate, not assume it drops further.
|
||||
|
||||
---
|
||||
|
||||
## Notes on scope and what this document does not claim
|
||||
|
||||
- This document does not decide the wire carrier, the windowed-family
|
||||
ceiling, or the canonical-vs-viewport policy — those are T-1179's and the
|
||||
workshop round-1/round-2 synthesis's job. It supplies the derive-cost and
|
||||
block/tile-cost inputs those decisions are argued from.
|
||||
- `WindowGranularity` was not modified (no Block/Tile variant added) — this
|
||||
measurement calls `derive_at_metres` directly, matching the exact pattern
|
||||
`zoom_ladder_bench.rs` already established for District/Quarter. Whether a
|
||||
future implementation adds Block/Tile as real `WindowGranularity` variants
|
||||
is an implementation-ticket decision, not a measurement-pass one.
|
||||
- No source file under `server/src/` was modified to produce these numbers —
|
||||
only the new `server/tests/bmv_gridunit_bench.rs` file was added, per the
|
||||
batch's constraint (`server/src/atlas/mod.rs` untouched, no new `src`
|
||||
modules).
|
||||
@@ -0,0 +1,268 @@
|
||||
---
|
||||
title: "T-1179 — Wire-size table for step-canvas encodings (measurement ④)"
|
||||
workshop: body-map-viewer
|
||||
status: complete
|
||||
owner: Dudley (Araminta's named-feature-encoding question and the D-225
|
||||
tagged-envelope call are argued from these numbers)
|
||||
---
|
||||
|
||||
# T-1179 — Wire-size table for step-canvas encodings
|
||||
|
||||
Measurement ④ of the [body-map-viewer workshop
|
||||
brief](../body-map-viewer-workshop-brief.md)'s pre-workshop appendix.
|
||||
Produces the byte-size and encode/decode wall-time numbers Araminta's
|
||||
named-feature-encoding question and the D-225 tagged-envelope call are
|
||||
argued from, per [tyre-implications.md §3](../tyre-implications.md) item 3.
|
||||
|
||||
**Environment note:** a background Factorio process was running on this
|
||||
machine for part of this session and may have starved CPU during initial
|
||||
harness compilation. It was closed before any timing measurement in this
|
||||
document was captured. Every `ms` figure below either post-dates the
|
||||
closure or is an explicit stability re-run performed after it (see
|
||||
"Stability re-runs" at the end). Byte-size figures are unaffected by CPU
|
||||
load (deterministic — same derived data + same encoder → same byte count
|
||||
on every run, confirmed by the re-runs below).
|
||||
|
||||
## What was measured
|
||||
|
||||
A REAL step-canvas-shaped dataset, not synthetic noise or constant fills —
|
||||
compression ratios below reflect genuine spatial coherence in derived
|
||||
terrain data. Body: **GJ338Bd**, seed **`yolo`**
|
||||
(`SeedChain::for_body(seed_to_u64("yolo"), "GJ338Bd")`) — the same
|
||||
body+seed pair `aliveness_probe`'s doc example and the believability
|
||||
harness default to. Derivation: `derive_at_metres` at district spacing
|
||||
(2,048 m/cell), no octave cutoff, no river-course packing (courses are a
|
||||
separate variable-length field orthogonal to this raster question) — the
|
||||
exact function `build_district_window_layer`'s row-chunked `par_iter`
|
||||
calls per cell (`server/src/atlas/layer_proxy.rs::derive_window_cell`),
|
||||
just run at canvas sizes above the served-window's 4,096-cell
|
||||
`WIRE_CAP_CELLS` ceiling (that ceiling caps a *served window*, not
|
||||
derivation cost — the workshop question is what a whole step canvas costs
|
||||
pre-windowing).
|
||||
|
||||
**Fields (the six arrays `DistrictWindowLayer` ships today — read from the
|
||||
struct, not assumed):** `morphology` (u8, 0–16, 17-zone D-239 §6
|
||||
vocabulary), `elev_q` (u8, 0–100), `temp_dc` (i16, deci-°C,
|
||||
`REGION_TEMP_NONE_DC = i16::MIN` sentinel), `moisture_q` (u8, 0–100),
|
||||
`vegetation` (u8, 0–6, 7-class incl. `Marine`), `glaciation` (u8, 0–4,
|
||||
5-grade). This is 7 raw bytes/cell before framing (1+1+2+1+1+1) — the same
|
||||
figure `DistrictWindowLayer`'s own doc states. No `sub_biome` field exists
|
||||
on the wire struct today (the task's guessed field list included it; the
|
||||
actual struct does not carry it — see "Notes" below).
|
||||
|
||||
**Canvas sizes:** 768×432 = 331,776 (~330K), 1920×1080 = 2,073,600
|
||||
(~2.07M), 3840×2160 = 8,294,400 (~8.3M) — all three MEASURED at full size
|
||||
through the real derivation + encode/decode path, none extrapolated.
|
||||
|
||||
## Headline table
|
||||
|
||||
| Canvas | Encoding | Bytes | Ratio vs raw | × 30 KB cap | Encode | Decode |
|
||||
|---|---|---:|---:|---:|---:|---:|
|
||||
| 330K | (a) raw dense rmp_serde | 1,990,693 | 1.000 | 66.4× | 9.41 ms | 8.82 ms |
|
||||
| 330K | (b) bit-packed | 1,646,390 | 0.827 | 54.9× | 9.23 ms | 10.13 ms |
|
||||
| 330K | (c) per-field RLE | 2,179,097 | 1.095 | 72.6× | 16.93 ms | 10.08 ms |
|
||||
| 330K | **(d) PNG per field** | **638,382** | **0.321** | **21.3×** | 5.41 ms | 3.55 ms |
|
||||
| 330K | (e) PNG-of-bit-packed | 1,110,822 | 0.558 | 37.0× | 5.91 ms | 4.89 ms |
|
||||
| 2.07M | (a) raw dense rmp_serde | 12,441,637 | 1.000 | 414.7× | 58.31 ms | 76.54 ms |
|
||||
| 2.07M | (b) bit-packed | 10,177,898 | 0.818 | 339.3× | 60.70 ms | 63.51 ms |
|
||||
| 2.07M | (c) per-field RLE | 12,837,802 | 1.032 | 427.9× | 101.18 ms | 61.99 ms |
|
||||
| 2.07M | **(d) PNG per field** | **3,781,988** | **0.304** | **126.1×** | 30.57 ms | 19.94 ms |
|
||||
| 2.07M | (e) PNG-of-bit-packed | 6,589,821 | 0.530 | 219.7× | 38.38 ms | 32.38 ms |
|
||||
| 8.3M | (a) raw dense rmp_serde | 51,932,586 | 1.000 | 1,731.1× | 217.31 ms | 217.25 ms |
|
||||
| 8.3M | (b) bit-packed | 42,790,610 | 0.824 | 1,426.4× | 244.48 ms | 249.04 ms |
|
||||
| 8.3M | (c) per-field RLE | 51,939,077 | 1.000 | 1,731.3× | 386.60 ms | 240.54 ms |
|
||||
| 8.3M | **(d) PNG per field** | **16,883,005** | **0.325** | **562.8×** | 119.32 ms | 90.79 ms |
|
||||
| 8.3M | (e) PNG-of-bit-packed | 28,071,108 | 0.541 | 935.7× | 159.82 ms | 142.02 ms |
|
||||
|
||||
All rows MEASURED (no ARITHMETIC scaling used — 330K/2.07M/8.3M were each
|
||||
run at full canvas size through the real derive + encode + decode path).
|
||||
Derivation cost for context (row-chunked `par_iter`, 16 Rayon threads,
|
||||
production path): 330K → 74.8 ms, 2.07M → 513.0 ms, 8.3M → 1,779.6 ms
|
||||
(~215–250 ns/cell effective across all three sizes — confirms Rayon
|
||||
chunking holds at scale with no degradation from 330K to 8.3M, closing
|
||||
the exact gap tyre-implications.md flagged for measurement ②).
|
||||
|
||||
**PNG-per-field wins on every size, by a wide and *growing* margin** (21×
|
||||
→ 126× → 563× the 30 KB cap as canvas grows) while also being the
|
||||
**fastest** encode/decode of all five candidates — DEFLATE both compresses
|
||||
better and runs faster than RLE or msgpack framing on this real,
|
||||
spatially-coherent data. RLE is the clear loser: it's *worse than raw* at
|
||||
every size (1.03×–1.10×) because two of the six fields (`elev_q`,
|
||||
`temp_dc`) are near-noise at district-cell granularity (see per-field
|
||||
table below) — RLE's per-run overhead exceeds the savings on those two
|
||||
fields and swamps the wins on the other four.
|
||||
|
||||
## Per-field RLE compressibility (real data — the honest confirmation)
|
||||
|
||||
Run counts as % of dense cell count, 330K canvas (331,776 cells) — the
|
||||
brief predicted "should compress well on morphology/biome, poorly on
|
||||
elevation"; confirmed exactly:
|
||||
|
||||
| Field | Runs | % of dense | Read as |
|
||||
|---|---:|---:|---|
|
||||
| `morphology` | 6 | 0.0% | near-constant across this canvas — 6 giant runs |
|
||||
| `vegetation` | 6 | 0.0% | same — near-constant |
|
||||
| `glaciation` | 95,274 | 28.7% | moderately compressible |
|
||||
| `moisture_q` | 129,542 | 39.0% | moderately compressible |
|
||||
| `elev_q` | 202,107 | 60.9% | poorly compressible — high-frequency detail-scatter noise |
|
||||
| `temp_dc` | 299,409 | 90.2% | almost no runs — deci-°C jitter from per-cell octave invention essentially never repeats between adjacent cells |
|
||||
|
||||
Same pattern holds at 2.07M and 8.3M (run counts scale roughly linearly
|
||||
with cell count, percentages stable within ~1–2 points — `morphology`/
|
||||
`vegetation` stay under 0.2%, `temp_dc` stays 90–91%). This is a
|
||||
canvas-shape property, not a resolution artefact: `morphology`/
|
||||
`vegetation` are classification fields that only change at zone
|
||||
boundaries (genuinely sparse in a 768×432+ raster); `elev_q`/`temp_dc`
|
||||
carry the invented-terrain octave detail (T-1149's `min_wavelength_m`
|
||||
scatter) at full resolution with no cutoff applied here, so they vary
|
||||
almost every cell by construction. **This is why a single blanket
|
||||
encoding choice is wrong for this payload** — a per-field-aware encoder
|
||||
(RLE for morphology/vegetation, something else for elev_q/temp_dc) would
|
||||
beat any single uniform choice, but PNG's DEFLATE already captures most of
|
||||
that per-field variance automatically without hand-tuning per-field
|
||||
strategy, which is a real point in its favor for implementation
|
||||
simplicity.
|
||||
|
||||
## Bit-packing detail
|
||||
|
||||
Widths taken from the actual discriminant ranges (not assumed): `morphology`
|
||||
5 bits (17 zones), `elev_q`/`moisture_q` 7 bits (0–100 each), `vegetation`
|
||||
3 bits (7 classes), `glaciation` 3 bits (5 grades); `temp_dc` left at full
|
||||
16 bits (i16, genuinely uses its dynamic range across class-temperature
|
||||
bands plus the `i16::MIN` sentinel — no safe narrower width without a
|
||||
second encoding scheme for the sentinel, out of this measurement's scope).
|
||||
Bit-packing alone buys ~17–18% off raw (0.818–0.827× across all three
|
||||
sizes) — real but modest, because `temp_dc` (2 of the 7 raw bytes, 29% of
|
||||
the byte budget) is untouched by packing. PNG-of-bit-packed (e) improves
|
||||
on bit-packing alone (0.53–0.56× vs 0.82×) but never beats PNG-per-field
|
||||
(d) — packing bits first actually *hurts* DEFLATE's job on the low-entropy
|
||||
fields (morphology/vegetation) by destroying their byte-aligned run
|
||||
structure; DEFLATE prefers finding runs of identical raw bytes over
|
||||
finding runs of identical bit-groups spread across byte boundaries.
|
||||
|
||||
## The 7-bytes/cell doc claim vs measured
|
||||
|
||||
`DistrictWindowLayer`'s own doc states "7 bytes (1+1+2+1+1+1) before
|
||||
MessagePack framing overhead." Measured raw rmp_serde total at 330K:
|
||||
1,990,693 bytes / 331,776 cells = **6.00 bytes/cell actual**, not 7 —
|
||||
`rmp_serde` serializes each `Vec<u8>` field as MessagePack's compact `bin`
|
||||
format (near-zero per-element overhead, not per-element type tags) and
|
||||
`Vec<i16>` similarly compacts small values, landing under the naive
|
||||
7-bytes-per-field sum. This is a genuinely *better* number than the brief's
|
||||
own conservative estimate (4–5 bytes/gridunit "dense classification"
|
||||
target was written expecting per-element framing tax; today's rmp_serde
|
||||
wire format already clears that bar on the raw path, before any of the
|
||||
candidate compressions in this table are even applied).
|
||||
|
||||
## Scaling sanity (330K → 2.07M → 8.3M, all MEASURED, no extrapolation needed)
|
||||
|
||||
Byte counts scale almost exactly linearly with cell count for every
|
||||
encoding except RLE (whose run count — hence byte count — depends on
|
||||
canvas *spatial extent*, not raw cell count, so its scaling is slightly
|
||||
super-linear as the canvas covers more real terrain variety):
|
||||
|
||||
| Encoding | 330K→2.07M scale factor | 2.07M→8.3M scale factor | Cell-count factor |
|
||||
|---|---:|---:|---:|
|
||||
| raw dense | 6.25× | 4.17× | 6.25× / 4.00× |
|
||||
| bit-packed | 6.18× | 4.20× | — |
|
||||
| PNG per field | 5.93× | 4.46× | — |
|
||||
|
||||
Close to the cell-count ratios (6.25× and 4.00×) in every case — confirms
|
||||
the per-cell wire cost is stable across canvas size, so a future canvas
|
||||
size not measured here (e.g. a step between 2.07M and 8.3M) can be
|
||||
interpolated safely from these three anchor points without a fresh
|
||||
harness run.
|
||||
|
||||
## Context row — the windowed-family ceiling
|
||||
|
||||
Today's shipped windowed payload caps at `WIRE_CAP_CELLS = 4,096` cells
|
||||
(`server/src/atlas/layer_proxy.rs`), ≈ **~30 KB** on the wire at the
|
||||
measured 6.0 bytes/cell raw rate (4,096 × 6 ≈ 24.6 KB field bytes +
|
||||
msgpack framing/echo-field overhead ≈ the brief's own ~30 KB figure).
|
||||
Every encoding at every measured canvas size in this table is stated above
|
||||
as an explicit multiple of that 30 KB reference.
|
||||
|
||||
**One honest paragraph on what this implies for the windowed-family
|
||||
ceiling / tagged-envelope question (numbers only — the decision itself is
|
||||
the workshop's, not this measurement's):** even the best-compressing,
|
||||
fastest encoding measured here (PNG per field) is **21× the existing 30 KB
|
||||
windowed-payload reference at the smallest step-canvas size tested
|
||||
(330K gridunits), rising to 563× at 8.3M**. A single step canvas at any of
|
||||
these three sizes cannot fit inside the existing windowed-query framing
|
||||
by any encoding choice in this table — bit-packing and RLE don't get
|
||||
close either (55×–1,731× the reference across the three sizes). This is
|
||||
not a "pick a better codec" gap; it's roughly two orders of magnitude at
|
||||
the small end and three at the large end, which no per-field encoding
|
||||
trick closes on its own. Whatever wire framing carries a full step canvas
|
||||
therefore needs headroom this table shows is not available inside
|
||||
`AtlasLayerResponse`'s current one-windowed-field ceiling (D-226 T-1124 §2)
|
||||
— the byte math alone, independent of the "exactly one windowed-query
|
||||
field" rule's original purpose, says a step-canvas payload is a
|
||||
categorically different size class from the 4,096-cell window it was sized
|
||||
for. Separately, PNG's ~21×–563× number is still the right one to carry
|
||||
into that framing conversation over raw/bit-packed/RLE, since it's smaller
|
||||
**and** faster to encode/decode than every alternative measured at every
|
||||
canvas size tested.
|
||||
|
||||
## Notes / scope boundaries
|
||||
|
||||
- **`sub_biome` is not on the wire today.** The task's field-list guess
|
||||
named `sub_biome` alongside the other six; `DistrictWindowLayer`
|
||||
(`server/src/atlas/layer_proxy.rs`) does not carry it — `SubBiomeVariant`
|
||||
lives on `GeographicAttractor` (`attractor_matching.rs`), a settlement/
|
||||
attractor-scoped concept, not a per-cell terrain field. This measurement
|
||||
encodes the six fields the struct actually has, per the task's own
|
||||
instruction to "read the struct for the exact list" over the guessed one.
|
||||
- **Courses excluded by design.** `DistrictWindowLayer.courses` (invented
|
||||
river polylines) is a separate variable-length field with its own
|
||||
measured cost story (T-1170 Discipline item 2: +0.09–0.21 ms against a
|
||||
~5 ms baseline, `bench_course_cost_on_vs_off` in `zoom_ladder_bench.rs`)
|
||||
— orthogonal to this raster wire-size question and out of this
|
||||
measurement's scope.
|
||||
- **No new dependency added.** `png = "0.17"` is already a `server/Cargo.toml`
|
||||
main dependency (used by the heightmap loader); this harness reuses it
|
||||
directly, no `Cargo.toml` change.
|
||||
- **`src/atlas/mod.rs` and `tests/zoom_ladder_bench.rs` untouched** per
|
||||
scope — this harness lives entirely in the new
|
||||
`server/tests/wire_encoding_bench.rs` file.
|
||||
|
||||
## Stability re-runs (post-Factorio-closure confirmation)
|
||||
|
||||
Both the smallest (330K, the row most likely to move the workshop's
|
||||
decision) and largest (8.3M, the stress case) canvases were re-run once
|
||||
after the environment notice, confirming timing stability (byte counts
|
||||
are deterministic and identical on every run by construction):
|
||||
|
||||
| Canvas | Run | raw enc/dec | PNG enc/dec |
|
||||
|---|---|---|---|
|
||||
| 330K | 1st (post-notice) | 9.41 / 8.82 ms | 5.41 / 3.55 ms |
|
||||
| 330K | 2nd (stability re-run) | 9.41 / 8.82 ms | 5.41 / 3.55 ms |
|
||||
| 8.3M | 1st (post-notice) | 222.54 / 216.74 ms | 121.10 / 91.86 ms |
|
||||
| 8.3M | 2nd (stability re-run) | 217.31 / 217.25 ms | 119.32 / 90.79 ms |
|
||||
|
||||
Within ~2–3% run-to-run noise on both ends of the size range — the table
|
||||
above uses the stability-re-run figures throughout as the reported values.
|
||||
|
||||
## Repro commands
|
||||
|
||||
```bash
|
||||
# From the worktree root:
|
||||
cd server
|
||||
|
||||
# 330K gridunit canvas (768x432)
|
||||
cargo test --release --test wire_encoding_bench wire_size_table_330k -- --ignored --nocapture
|
||||
|
||||
# 2.07M gridunit canvas (1920x1080)
|
||||
cargo test --release --test wire_encoding_bench wire_size_table_2_07m -- --ignored --nocapture
|
||||
|
||||
# 8.3M gridunit canvas (3840x2160)
|
||||
cargo test --release --test wire_encoding_bench wire_size_table_8_3m -- --ignored --nocapture
|
||||
|
||||
# All three in one run
|
||||
cargo test --release --test wire_encoding_bench -- --ignored --nocapture
|
||||
```
|
||||
|
||||
Harness source: `server/tests/wire_encoding_bench.rs`. Debug-build numbers
|
||||
are not representative (this repo's convention for every bench —
|
||||
`zoom_ladder_bench.rs`'s doc comment states the same); always run
|
||||
`--release`.
|
||||
@@ -0,0 +1,207 @@
|
||||
---
|
||||
title: "Measurement ⑤: Godot ImageTexture upload cost per step-cross (330K–8.3M px)"
|
||||
ticket: T-1180
|
||||
owner: Stig
|
||||
workshop: body-map-viewer
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Measurement ⑤ — ImageTexture upload cost per step-cross
|
||||
|
||||
## What this answers
|
||||
|
||||
The body-map-viewer render baseline is per-step data canvas → render-to-texture
|
||||
terrain layer. On a step-cross, the client either (a) allocates a fresh
|
||||
`ImageTexture` via `create_from_image`, or (b) reuses a persistent texture and
|
||||
calls `texture.update(image)`. Both paths cost real upload time on the
|
||||
render thread; if that cost is large relative to a 16.6 ms (60 fps) frame
|
||||
budget, the hold-fetch-swap will visibly stutter on step-cross. This
|
||||
measurement prices both paths at the three canvas sizes the workshop cites
|
||||
(768×432 ≈ 330K px through 3840×2160 ≈ 8.3M px), in both RGBA8 and L8, and
|
||||
checks directly whether the worst case (8.3M px) blows a frame budget.
|
||||
|
||||
## Method
|
||||
|
||||
**Must run windowed, not headless** — Godot's headless build swaps in a dummy
|
||||
rendering driver that does not perform real GPU texture uploads, so
|
||||
`create_from_image`/`texture.update` timings under `--headless` would be
|
||||
near-zero and meaningless for this question. This run used a real X display
|
||||
(`DISPLAY=:0`) with the actual GPU-backed renderer.
|
||||
|
||||
- Driver: a temporary `SceneTree` script (`client/tmp_drive_t1180.gd`, deleted
|
||||
after this measurement — not committed).
|
||||
- Launch: `godot --path client --script res://tmp_drive_t1180.gd`, run from the
|
||||
worktree so `res://` resolves against `client/`.
|
||||
- Before any measurement: `DisplayServer.window_set_vsync_mode(DisplayServer.VSYNC_DISABLED)`,
|
||||
`Engine.max_fps = 0`, root window forced to 1920×1080 (the project's base
|
||||
viewport — `client/project.godot` `window/size/viewport_width`/`height`,
|
||||
`canvas_items` stretch mode), and 4 frames of warm-up before any timing
|
||||
begins (window focus / swapchain settle).
|
||||
- Each op timed with `Time.get_ticks_usec()` immediately around the call,
|
||||
33 iterations per case (3 discarded as pipeline/driver warm-up, reported
|
||||
separately as "first-use"; 30 kept). Median and p95 computed over the 30
|
||||
kept samples.
|
||||
- `texture.update` cases alternate the fill color each iteration so the
|
||||
driver does real work per call (rules out any same-data fast path).
|
||||
- Frame-hitch check: a 40-frame `await process_frame` loop: at frame 20,
|
||||
perform one fresh 8.3M px `create_from_image` and record both the op's
|
||||
direct `Time.get_ticks_usec()` cost and the wall-clock delta of the
|
||||
surrounding frames, compared to the baseline median frame delta.
|
||||
|
||||
## Environment note
|
||||
|
||||
A background CPU/GPU load (Factorio) was running on this machine for part of
|
||||
the session and was closed partway through. The results below are the **two
|
||||
runs captured after the machine was quiet** (back-to-back re-runs, both
|
||||
post-close) — any numbers captured before that point were discarded and are
|
||||
not reported here. The two clean runs agree closely (see "Stability" below),
|
||||
so there is no early-slow/later-fast drift to account for.
|
||||
|
||||
## Hardware / GPU
|
||||
|
||||
Printed by Godot at startup on every run:
|
||||
|
||||
```
|
||||
Renderer: AMD Radeon RX 9070 (radeonsi, gfx1201, ACO, DRM 3.64, 7.0.9-ogc3.2.fc44.x86_64)
|
||||
Vendor: AMD
|
||||
OpenGL API 4.6 (Core Profile) Mesa 26.1.3 - Compatibility
|
||||
Godot Engine v4.6.stable.official.89cea1439
|
||||
```
|
||||
|
||||
This is a desktop discrete GPU (RX 9070) — a reasonable stand-in for target
|
||||
hardware, but not a lower-bound guarantee; integrated/laptop GPUs should be
|
||||
expected to run slower, especially on the 8.3M px cases.
|
||||
|
||||
## Results — run A (first clean run, post Factorio-close)
|
||||
|
||||
All times in ms. n=30 kept samples after 3 discarded warmup iterations;
|
||||
first-use = the very first (discarded) iteration, reported separately as the
|
||||
pipeline-compile-inclusive cost.
|
||||
|
||||
| op | size | fmt | median | p95 | first-use |
|
||||
|---|---|---|---|---|---|
|
||||
| create+fill (baseline) | 768×432 (~330K px) | RGBA8 | 0.053 | 0.059 | 0.372 |
|
||||
| create+fill (baseline) | 1920×1080 (~2.07M px) | RGBA8 | 0.364 | 0.430 | 3.260 |
|
||||
| create+fill (baseline) | 3840×2160 (~8.3M px) | RGBA8 | 2.156 | 2.816 | 11.315 |
|
||||
| create_from_image (fresh) | 768×432 | RGBA8 | 0.211 | 0.265 | 0.269 |
|
||||
| create_from_image (fresh) | 768×432 | L8 | 0.053 | 0.094 | 0.340 |
|
||||
| create_from_image (fresh) | 1920×1080 | RGBA8 | 0.471 | 1.351 | 2.369 |
|
||||
| create_from_image (fresh) | 1920×1080 | L8 | 0.320 | 0.454 | 0.404 |
|
||||
| create_from_image (fresh) | 3840×2160 | RGBA8 | **3.159** | 4.073 | 5.235 |
|
||||
| create_from_image (fresh) | 3840×2160 | L8 | **0.745** | 1.183 | 0.593 |
|
||||
| texture.update (reuse) | 768×432 | RGBA8 | 0.105 | 0.262 | 0.102 |
|
||||
| texture.update (reuse) | 768×432 | L8 | 0.031 | 0.036 | 0.064 |
|
||||
| texture.update (reuse) | 1920×1080 | RGBA8 | 0.724 | 1.257 | 0.897 |
|
||||
| texture.update (reuse) | 1920×1080 | L8 | 0.157 | 0.214 | 0.160 |
|
||||
| texture.update (reuse) | 3840×2160 | RGBA8 | **4.624** | 5.214 | 5.255 |
|
||||
| texture.update (reuse) | 3840×2160 | L8 | **0.754** | 1.195 | 0.719 |
|
||||
|
||||
## Results — run B (repeat, stability confirmation)
|
||||
|
||||
Same driver, back-to-back re-run, same quiet machine. Reported here in full
|
||||
per the coordinator's instruction to re-run headline cases once more at the
|
||||
end and confirm stability rather than relying on a single sample.
|
||||
|
||||
| op | size | fmt | median | p95 | first-use |
|
||||
|---|---|---|---|---|---|
|
||||
| create+fill (baseline) | 768×432 | RGBA8 | 0.055 | 0.059 | 0.370 |
|
||||
| create+fill (baseline) | 1920×1080 | RGBA8 | 0.386 | 0.537 | 2.590 |
|
||||
| create+fill (baseline) | 3840×2160 | RGBA8 | 2.340 | 3.379 | 11.855 |
|
||||
| create_from_image (fresh) | 768×432 | RGBA8 | 0.196 | 0.259 | 0.334 |
|
||||
| create_from_image (fresh) | 768×432 | L8 | 0.053 | 0.100 | 0.405 |
|
||||
| create_from_image (fresh) | 1920×1080 | RGBA8 | 0.541 | 1.287 | 0.991 |
|
||||
| create_from_image (fresh) | 1920×1080 | L8 | 0.130 | 0.567 | 0.511 |
|
||||
| create_from_image (fresh) | 3840×2160 | RGBA8 | **3.273** | 4.649 | 4.578 |
|
||||
| create_from_image (fresh) | 3840×2160 | L8 | **0.524** | 1.540 | 1.600 |
|
||||
| texture.update (reuse) | 768×432 | RGBA8 | 0.106 | 0.385 | 0.109 |
|
||||
| texture.update (reuse) | 768×432 | L8 | 0.034 | 0.042 | 0.078 |
|
||||
| texture.update (reuse) | 1920×1080 | RGBA8 | 0.730 | 0.868 | 1.153 |
|
||||
| texture.update (reuse) | 1920×1080 | L8 | 0.172 | 0.875 | 0.146 |
|
||||
| texture.update (reuse) | 3840×2160 | RGBA8 | **4.276** | 5.681 | 4.380 |
|
||||
| texture.update (reuse) | 3840×2160 | L8 | **0.704** | 1.092 | 0.719 |
|
||||
|
||||
### Stability
|
||||
|
||||
Run A and Run B agree within ~0.1–0.5 ms on every headline case (8.3M px
|
||||
RGBA8 create: 3.16 vs 3.27 ms; 8.3M px RGBA8 update: 4.62 vs 4.28 ms). No
|
||||
early-slow/later-fast trend between the two runs — both post-date the
|
||||
Factorio close, and the agreement confirms the earlier contended-machine
|
||||
numbers (not reported above) were the outlier, not these.
|
||||
|
||||
## Frame-hitch check (8.3M px `create_from_image` mid-frame-loop)
|
||||
|
||||
40-frame loop, `await process_frame` per frame, op performed at frame 20:
|
||||
|
||||
```
|
||||
frame delta_ms op_ms
|
||||
17 0.191 0.000
|
||||
18 0.197 0.000
|
||||
19 0.194 0.000
|
||||
20 0.192 2.092 <-- create_from_image here
|
||||
21 2.364 0.000
|
||||
22 2.851 0.000
|
||||
23 0.581 0.000
|
||||
|
||||
Baseline median frame delta (excl. hitch frame + successor): 0.202 ms
|
||||
Hitch frame delta: 0.192 ms (op direct cost: 2.092 ms)
|
||||
Successor frame delta: 2.364 ms
|
||||
16.6ms budget exceeded on hitch frame: no
|
||||
16.6ms budget exceeded on successor frame: no
|
||||
```
|
||||
|
||||
**Reading this:** the op's *direct* CPU-side cost (`Time.get_ticks_usec`
|
||||
around the call) is ~2–3 ms, matching the table above. But the *frame delta*
|
||||
spike shows up one frame **later** than the frame that issued the call — this
|
||||
is the driver's `await process_frame` reporting the delta to the *next*
|
||||
process tick, and/or the GPU upload's actual completion (fence/sync) landing
|
||||
on the following frame under OpenGL's queued submission model. Either way:
|
||||
the cost is real, it is on the order of 2–3 ms, and it lands within one frame
|
||||
of the call — but **it never exceeds the 16.6 ms budget** in any of the
|
||||
sampled cases, including the frame immediately after the upload. There is
|
||||
headroom, not a cliff.
|
||||
|
||||
## Headline numbers
|
||||
|
||||
| case | median | p95 |
|
||||
|---|---|---|
|
||||
| 768×432 create_from_image, RGBA8 | ~0.20 ms | ~0.26 ms |
|
||||
| 768×432 create_from_image, L8 | ~0.05 ms | ~0.10 ms |
|
||||
| 768×432 texture.update, RGBA8 | ~0.11 ms | ~0.26–0.39 ms |
|
||||
| 768×432 texture.update, L8 | ~0.03 ms | ~0.04 ms |
|
||||
| 3840×2160 create_from_image, RGBA8 | ~3.2 ms | ~4.1–4.6 ms |
|
||||
| 3840×2160 create_from_image, L8 | ~0.5–0.7 ms | ~1.2–1.5 ms |
|
||||
| 3840×2160 texture.update, RGBA8 | ~4.3–4.6 ms | ~5.2–5.7 ms |
|
||||
| 3840×2160 texture.update, L8 | ~0.7 ms | ~1.1–1.2 ms |
|
||||
|
||||
**Frame-hitch verdict: no budget break.** The single most expensive case
|
||||
measured (8.3M px RGBA8 `create_from_image`) costs ~3 ms direct / shows up as
|
||||
a ~2.4–2.9 ms frame-delta spike one frame later — comfortably under the
|
||||
16.6 ms (60 fps) budget with margin to spare, even stacked with normal
|
||||
per-frame gameplay/UI cost. `texture.update` on the same size is slightly
|
||||
*more* expensive than a fresh `create_from_image` (4.3–4.6 ms vs 3.2 ms median)
|
||||
— counter to the naive assumption that reuse is always cheaper — but still
|
||||
well inside budget.
|
||||
|
||||
## Implication for the hold-fetch-swap design
|
||||
|
||||
At every canvas size the workshop cites, both textured-swap strategies
|
||||
(fresh `create_from_image` and reused `texture.update`) complete in single-digit
|
||||
milliseconds — no case observed here risks a dropped frame on step-cross, so
|
||||
the render layer should **not let upload cost gate the step-count/canvas-size
|
||||
decision**. Prefer `texture.update` (reuse) anyway: it avoids per-step Texture
|
||||
object churn (driver/RenderingServer resource lifecycle overhead not captured
|
||||
by these microbenchmarks) even though its raw upload number is marginally
|
||||
higher than a fresh allocation, and L8 — worth using wherever the data canvas
|
||||
is genuinely single-channel — is 4–9× cheaper than RGBA8 at every size,
|
||||
making it the easy win if any field plane in the wire contract can ship
|
||||
single-channel.
|
||||
|
||||
## Launch command (exact)
|
||||
|
||||
```bash
|
||||
cd /var/mnt/data/projects/settled-reach/.worktrees/bmv-measurements
|
||||
godot --path client --script res://tmp_drive_t1180.gd
|
||||
```
|
||||
|
||||
Driver script: `client/tmp_drive_t1180.gd` (temporary, deleted after this
|
||||
measurement — do not look for it in the committed tree).
|
||||
File diff suppressed because it is too large
Load Diff
@@ -23,6 +23,7 @@ pub mod drainage;
|
||||
pub mod features;
|
||||
pub mod gen_queue;
|
||||
pub mod heightmap;
|
||||
pub mod hydrology_equilibrium;
|
||||
pub mod layer1;
|
||||
pub mod layer_proxy;
|
||||
pub mod mosaic;
|
||||
|
||||
@@ -0,0 +1,853 @@
|
||||
//! Body Map Viewer workshop-gate benches (T-1178 / T-1154, brief appendix
|
||||
//! ②/③; tyre-implications.md §3.2). Two anti-extrapolation measurements the
|
||||
//! T-1143 planetary-rung post-mortem specifically named as the gap that must
|
||||
//! close before round 1 of the body-map-viewer workshop can start:
|
||||
//!
|
||||
//! - **T-1178 (②):** does the row-chunked `par_iter` throughput measured at
|
||||
//! 4,096 cells (`zoom_ladder_bench.rs`, 1.785 µs/cell District cutoff)
|
||||
//! HOLD at real step-canvas sizes (330K, 2.07M, 8.3M cells)? MEASURE, never
|
||||
//! extrapolate.
|
||||
//! - **T-1154 (③):** per-cell derive cost at block (128 m) and tile-adjacent
|
||||
//! (1–4 m) spacing — the ladder's un-costed bottom rungs
|
||||
//! (`atlas-zoom-ladder-t1143.md` §2: "not estimated").
|
||||
//!
|
||||
//! ## Why this is a NEW file, not an extension of `zoom_ladder_bench.rs`
|
||||
//!
|
||||
//! `zoom_ladder_bench.rs` measures per-cell rate at the FIXED 4,096-cell
|
||||
//! shape (`grid_side = 64`, matching the D-226 window wire cap) — that shape
|
||||
//! is deliberate there (it's the served-window ceiling). This file measures
|
||||
//! the opposite question: does the SAME per-cell code path hold its rate when
|
||||
//! swept out to real canvas dimensions the served-window ceiling forbids.
|
||||
//! Different shape, different question, kept in its own file per the batch
|
||||
//! instruction (do not touch `server/src/atlas/mod.rs`; extend the bench or
|
||||
//! add a new file — this adds a new file to avoid entangling two
|
||||
//! differently-shaped measurement passes in one).
|
||||
//!
|
||||
//! ## The `n`-cap problem (why this cannot go through `AtlasLayerRequest`)
|
||||
//!
|
||||
//! `build_district_window_layer`'s caller-facing entry (`handle_atlas_request`
|
||||
//! / `clamp_window_n_v2`) hard-clamps `n` so `side² ≤ WIRE_CAP_CELLS = 4,096`
|
||||
//! — a REAL client can never request a 330K-cell window over the wire, by
|
||||
//! design (D-226 T-1124 §2). So "run it through the production path at
|
||||
//! 330K/8.3M cells" cannot mean "send an `AtlasLayerRequest` for that size" —
|
||||
//! no such request is legal. It means: call the actual
|
||||
//! `build_district_window_layer` function — same signature, same row-chunked
|
||||
//! `into_par_iter()` loop, same `derive_window_cell`/`scatter_row` internals,
|
||||
//! same `derive_at_metres` calls — with an `n` no wire request could carry,
|
||||
//! because `build_district_window_layer` itself has NO internal clamp (the
|
||||
//! clamp lives one layer up, in the request handler). Confirmed by direct
|
||||
//! read of `layer_proxy.rs:1515-1628` this session. This is exactly what the
|
||||
//! ticket anticipates: "If the production path caps window n such that you
|
||||
//! cannot request 330K cells through it directly, bench the underlying
|
||||
//! chunked loop at those counts and say exactly what you ran."
|
||||
//!
|
||||
//! ## Square vs. rectangular canvases
|
||||
//!
|
||||
//! `build_district_window_layer` only derives SQUARE `side×side` grids ('n'
|
||||
//! is a single extent). The real step-canvas shapes the ticket names are
|
||||
//! 16:9 rectangles (768×432, 1920×1080, 3840×2160) — not square. Two
|
||||
//! measurements are taken for each cell-count target:
|
||||
//!
|
||||
//! 1. **Square, through `build_district_window_layer` itself** (District
|
||||
//! granularity, `n = side`, real function call, unmodified, REAL
|
||||
//! `RiverNetwork` passed in) — the closest possible approach to "the
|
||||
//! actual production entry point," at the nearest square cell count to
|
||||
//! the target (e.g. side=576 → 331,776 cells, matching 768×432's 331,776
|
||||
//! exactly). This path exercises the ACTUAL course-invention +
|
||||
//! riparian-cull machinery `build_district_window_layer` runs in
|
||||
//! production (`invent_courses_near_window`/`crop_courses_for_wire`) —
|
||||
//! confirmed non-empty at every measured shape (`layer.courses.len()` is
|
||||
//! printed and was 3/6/10 at 330K/2.07M/8.3M respectively on the
|
||||
//! synthetic gradient body centred at the world origin; see the results
|
||||
//! doc's H3 correction for exactly what this does and does not validate).
|
||||
//!
|
||||
//! 2. **Real 16:9 rectangle, via a row-chunked loop that mirrors
|
||||
//! `build_district_window_layer`'s internals cell-for-cell** (same
|
||||
//! `into_par_iter()` row chunking, same `derive_at_metres` call, same
|
||||
//! per-cell output quantization copied from `derive_window_cell`) but
|
||||
//! sized to the actual non-square canvas. This is necessarily a
|
||||
//! hand-written replica for the rectangular case (documented inline,
|
||||
//! diffed explicitly against the real function in the module doc above),
|
||||
//! since no production entry point derives a rectangle. Labelled
|
||||
//! "MEASURED (replica loop)" in the results table to distinguish it from
|
||||
//! "MEASURED (production fn)".
|
||||
//!
|
||||
//! **DISCLOSED GAP (PR #198 review, Hoshe H2):** [`rect_window_replica`]
|
||||
//! always calls `derive_at_metres` with an EMPTY `&[]` course slice — it
|
||||
//! has no `RiverNetwork`/`invent_courses_near_window` wiring at all, by
|
||||
//! construction (courses are invented ONCE per window, ahead of the
|
||||
//! per-cell loop, inside `build_district_window_layer` itself —
|
||||
//! replicating that machinery was out of scope for a bench loop whose job
|
||||
//! is the per-cell derive rate, not course invention). This means EVERY
|
||||
//! rectangular-canvas number in this file (the three named 16:9 shapes
|
||||
//! AND the 83K deep-step bench) **excludes the per-cell
|
||||
//! `river_course::near_perennial_water` riparian-test cost** production
|
||||
//! pays on every cell of a window with real nearby course geometry.
|
||||
//! `zoom_ladder_bench.rs`'s own `bench_course_cost_on_vs_off` measured
|
||||
//! that cost at District cap (n=64, real GJ1c geometry): **+0.09–0.21 ms
|
||||
//! against a ~5 ms baseline, under 5%** — small, but real, and this file's
|
||||
//! rectangular numbers do not include it. Full disclosure and the
|
||||
//! corrected "what converges with what" statement is in the results doc's
|
||||
//! H2/H3 section — read that section before citing any rectangular-canvas
|
||||
//! number here as courses-inclusive. It is not.
|
||||
//!
|
||||
//! **Convergence claim, corrected (H3):** the square production-fn path is
|
||||
//! courses-INCLUSIVE (light density: 3/6/10 courses at 330K/2.07M/8.3M) and
|
||||
//! the rectangular replica-loop path is courses-EMPTY (always `&[]`) — so
|
||||
//! their agreement at 331,776 ≈ 576² (~191 ns/cell either way) validates the
|
||||
//! ROW-CHUNKED LOOP MECHANICS (chunking granularity, dispatch overhead,
|
||||
//! per-cell derive cost) converging across two independently-written call
|
||||
//! sites, NOT a courses-empty-vs-courses-inclusive equivalence claim — the
|
||||
//! two paths differ in exactly one respect (courses present vs absent) and
|
||||
//! happen to land within noise of each other at this course DENSITY (3 out
|
||||
//! of 331,776 cells is far too sparse to move the aggregate ns/cell figure
|
||||
//! outside the run-to-run noise band, consistent with the <5% per-cell
|
||||
//! course-cost delta `zoom_ladder_bench.rs` measured directly). The
|
||||
//! courses-inclusive rate at REAL production course density (not this
|
||||
//! sparse an origin-window) is covered only by the separate GJ1c real-body
|
||||
//! cross-check bench below (18 courses in a 331,776-cell window, deliberately
|
||||
//! centred on real river geometry) — see that bench's own doc and the
|
||||
//! results doc for the exact scope of what each number does and does not
|
||||
//! include.
|
||||
//!
|
||||
//! Run: `cargo test --release --test bmv_gridunit_bench -- --ignored --nocapture`
|
||||
|
||||
use std::time::Instant;
|
||||
|
||||
use settled_reach_server::atlas::district_profile::{
|
||||
derive_at_metres, BodyParams, ClimateConstants,
|
||||
};
|
||||
use settled_reach_server::atlas::drainage;
|
||||
use settled_reach_server::atlas::features::TerrainAnalysis;
|
||||
use settled_reach_server::atlas::heightmap::{load_heightmap_png, BodyHeightmap};
|
||||
use settled_reach_server::atlas::layer_proxy::{build_district_window_layer, WindowGranularity};
|
||||
use settled_reach_server::atlas::scale;
|
||||
use settled_reach_server::seed::{SeedChain, SeedDomain};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared fixtures — same shape as zoom_ladder_bench.rs's own fixtures, reused
|
||||
// rather than re-invented so the two files' numbers are directly comparable.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn bench_hm() -> BodyHeightmap {
|
||||
let (w, h) = (128u32, 64u32);
|
||||
let n = (w * h) as usize;
|
||||
let data = (0..n)
|
||||
.map(|i| {
|
||||
let r = (i / w as usize) as f32 / h as f32;
|
||||
let c = (i % w as usize) as f32 / w as f32;
|
||||
(r * 0.6 + c * 0.4).min(1.0)
|
||||
})
|
||||
.collect();
|
||||
BodyHeightmap {
|
||||
body_id: "bench".into(),
|
||||
width: w,
|
||||
height: h,
|
||||
data,
|
||||
sea_level: 0.3,
|
||||
}
|
||||
}
|
||||
|
||||
fn bench_ta(hm: &BodyHeightmap) -> TerrainAnalysis {
|
||||
let dr = drainage::analyze(&hm.data, hm.width, hm.height, hm.sea_level);
|
||||
TerrainAnalysis::analyze(hm, &dr)
|
||||
}
|
||||
|
||||
fn bench_river_network(
|
||||
hm: &BodyHeightmap,
|
||||
) -> settled_reach_server::atlas::body_world_state::RiverNetwork {
|
||||
drainage::analyze(&hm.data, hm.width, hm.height, hm.sea_level).river_network
|
||||
}
|
||||
|
||||
fn bench_params() -> BodyParams {
|
||||
BodyParams {
|
||||
hydrosphere: Some("ocean".into()),
|
||||
atmosphere: Some("breathable".into()),
|
||||
planet_class: Some("temperate".into()),
|
||||
body_radius_km: Some(6371.0),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
/// The real committed GJ1c heightmap, downsampled to the production working
|
||||
/// grid (512×256) — used for the "real body" cross-check bench so the
|
||||
/// headline numbers are not solely a synthetic-gradient artifact.
|
||||
fn gj1c_fixture() -> (BodyHeightmap, TerrainAnalysis) {
|
||||
let src = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("../wiki/star-systems/GJ-1/bodies/GJ1c/heightmap.png");
|
||||
let heightmap = load_heightmap_png(&src, "GJ1c", 0.3).expect("decode committed GJ1c heightmap");
|
||||
let small = heightmap.downsample(512, 256);
|
||||
let dr = drainage::analyze(&small.data, small.width, small.height, small.sea_level);
|
||||
let ta = TerrainAnalysis::analyze(&small, &dr);
|
||||
(small, ta)
|
||||
}
|
||||
|
||||
/// Report the actual Rayon global-pool thread count in use, not an assumed
|
||||
/// constant — so the results doc records what really ran, on whatever
|
||||
/// machine state existed at run time.
|
||||
fn rayon_threads_report() -> String {
|
||||
format!(
|
||||
"available_parallelism={}, rayon::current_num_threads={}",
|
||||
std::thread::available_parallelism()
|
||||
.map(|n| n.get())
|
||||
.unwrap_or(0),
|
||||
rayon::current_num_threads()
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// T-1178 (Measurement ②) — square canvases through the REAL production fn.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Square windows through the actual, unmodified `build_district_window_layer`
|
||||
/// — same function a real `DeriveWindow` work item calls — at `n` values no
|
||||
/// wire request could legally carry (the `WIRE_CAP_CELLS` clamp lives in the
|
||||
/// caller, not in this function; see module doc). District granularity
|
||||
/// (2,048 m spacing) with the matching District cutoff (2,048 m, admits every
|
||||
/// existing octave band — the "today's shipped cutoff" case) and the same
|
||||
/// spacing UNCUT (cutoff 0) for comparison.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_square_window_production_fn_district_spacing() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let rn = bench_river_network(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
|
||||
println!("\n=== T-1178 square-window production-fn bench (District spacing) ===");
|
||||
println!(" {}\n", rayon_threads_report());
|
||||
|
||||
// side values chosen to land on/near the ticket's named cell counts:
|
||||
// 576^2 = 331,776 (~330K, the 5x5-px-per-gridunit fallback)
|
||||
// 1440^2 = 2,073,600 (~2.07M, the 1920x1080 midpoint)
|
||||
// 2880^2 = 8,294,400 (~8.3M, the 3840x2160 1x1 ideal)
|
||||
let side_targets: [u32; 3] = [576, 1440, 2880];
|
||||
let min_wl_m = scale::DISTRICT_M as u32; // 2,048 m — the shipped District cutoff band
|
||||
|
||||
for &side in &side_targets {
|
||||
let n = side; // District granularity: side == n
|
||||
let cells = (side as u64) * (side as u64);
|
||||
|
||||
// Warm-up call (allocator/page-fault warm-up not counted).
|
||||
let _ = build_district_window_layer(
|
||||
seed,
|
||||
"bench",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
(0, 0),
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
);
|
||||
|
||||
let t0 = Instant::now();
|
||||
let layer = build_district_window_layer(
|
||||
seed,
|
||||
"bench",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
(0, 0),
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
);
|
||||
let elapsed = t0.elapsed();
|
||||
std::hint::black_box(layer.elev_q.len());
|
||||
|
||||
let ms = elapsed.as_secs_f64() * 1000.0;
|
||||
let ns_per_cell = elapsed.as_secs_f64() * 1e9 / cells as f64;
|
||||
println!(
|
||||
" side={side:>5} n={n:>5} cells={cells:>10} (target ~{}): \
|
||||
{ms:>9.2} ms warm, {ns_per_cell:>7.1} ns/cell ({:.3} us/cell), \
|
||||
courses_in_window={}",
|
||||
match cells {
|
||||
c if c < 500_000 => "330K",
|
||||
c if c < 4_000_000 => "2.07M",
|
||||
_ => "8.3M",
|
||||
},
|
||||
ns_per_cell / 1000.0,
|
||||
layer.courses.len()
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
|
||||
/// Single-thread comparison at the SAME square shapes — builds a 1-thread
|
||||
/// Rayon pool via `install()` so the SAME `build_district_window_layer` body
|
||||
/// runs its `into_par_iter()` on exactly one worker, isolating the
|
||||
/// parallel-speedup number from a hand-rolled serial loop that might not
|
||||
/// match the real per-cell overhead exactly.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_square_window_production_fn_district_spacing_single_thread() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let rn = bench_river_network(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
|
||||
println!(
|
||||
"\n=== T-1178 square-window production-fn bench, SINGLE THREAD (District spacing) ===\n"
|
||||
);
|
||||
|
||||
// 8.3M cells single-thread is the ~12s-class case the design doc
|
||||
// extrapolated (§7); 330K/2.07M included for the full comparison table.
|
||||
// Skipped: none — this is the case that must NOT be assumed cheap.
|
||||
let side_targets: [u32; 3] = [576, 1440, 2880];
|
||||
let min_wl_m = scale::DISTRICT_M as u32;
|
||||
|
||||
let pool = rayon::ThreadPoolBuilder::new()
|
||||
.num_threads(1)
|
||||
.build()
|
||||
.expect("build single-thread rayon pool");
|
||||
|
||||
for &side in &side_targets {
|
||||
let n = side;
|
||||
let cells = (side as u64) * (side as u64);
|
||||
|
||||
pool.install(|| {
|
||||
let _ = build_district_window_layer(
|
||||
seed,
|
||||
"bench",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
(0, 0),
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
);
|
||||
});
|
||||
|
||||
let t0 = Instant::now();
|
||||
let layer = pool.install(|| {
|
||||
build_district_window_layer(
|
||||
seed,
|
||||
"bench",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
(0, 0),
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
)
|
||||
});
|
||||
let elapsed = t0.elapsed();
|
||||
std::hint::black_box(layer.elev_q.len());
|
||||
|
||||
let ms = elapsed.as_secs_f64() * 1000.0;
|
||||
let ns_per_cell = elapsed.as_secs_f64() * 1e9 / cells as f64;
|
||||
println!(
|
||||
" side={side:>5} n={n:>5} cells={cells:>10} (1 thread): \
|
||||
{ms:>9.2} ms, {ns_per_cell:>7.1} ns/cell ({:.3} us/cell)",
|
||||
ns_per_cell / 1000.0
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
|
||||
/// Real-body cross-check: the same square sweep, but on the committed GJ1c
|
||||
/// heightmap/TerrainAnalysis (not the synthetic gradient) and with the real
|
||||
/// river network live (courses ON, window centred on real river geometry —
|
||||
/// not an empty network, and not a window that happens to cull every course
|
||||
/// out) — confirms the synthetic-fixture numbers above are not an artifact of
|
||||
/// a trivial gradient body or an empty river network's near-zero
|
||||
/// course-culling cost. Window centring follows `bench_course_cost_on_vs_off`
|
||||
/// (`zoom_ladder_bench.rs`)'s exact technique: convert a real river cell to
|
||||
/// world metres via the same `pixel_to_world_m`-equivalent formula, then to
|
||||
/// the covering `DistrictPos`, so the window is genuinely near GJ1c's rivers
|
||||
/// rather than at the arbitrary world origin (which measured courses_in_window=0
|
||||
/// on a first attempt — corrected here). Run at the 330K shape only
|
||||
/// (real-body I/O + full sweep would duplicate the synthetic-fixture table
|
||||
/// for no new signal at the larger sizes — the per-cell RATE is what's being
|
||||
/// cross-checked, not re-measuring 8.3M twice).
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_square_window_production_fn_gj1c_real_body_crosscheck() {
|
||||
let (hm, ta) = gj1c_fixture();
|
||||
let dr = drainage::analyze(&hm.data, hm.width, hm.height, hm.sea_level);
|
||||
let rn = dr.river_network.clone();
|
||||
let params = BodyParams {
|
||||
hydrosphere: Some("ocean".into()),
|
||||
atmosphere: Some("breathable".into()),
|
||||
planet_class: Some("temperate".into()),
|
||||
body_radius_km: Some(6371.0),
|
||||
..Default::default()
|
||||
};
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(0xC0FFEE_u64).derive(SeedDomain::Body, 7);
|
||||
|
||||
assert!(
|
||||
!rn.river_cells.is_empty(),
|
||||
"GJ1c at production working resolution must have river cells for this bench to be meaningful"
|
||||
);
|
||||
|
||||
// Same centring technique as zoom_ladder_bench.rs's bench_course_cost_on_vs_off.
|
||||
let river_cell = rn.river_cells[rn.river_cells.len() / 2];
|
||||
let r_km = params.body_radius_km.unwrap();
|
||||
let world_pos = (
|
||||
river_cell.1 as f64 / ta.w as f64 * (std::f64::consts::TAU * r_km * 1000.0),
|
||||
(river_cell.0 as f64 / (ta.h - 1) as f64 - 0.5) * (std::f64::consts::PI * r_km * 1000.0),
|
||||
);
|
||||
let center: (i32, i32) = (
|
||||
(world_pos.0 / scale::DISTRICT_M as f64).floor() as i32,
|
||||
(world_pos.1 / scale::DISTRICT_M as f64).floor() as i32,
|
||||
);
|
||||
|
||||
println!("\n=== T-1178 GJ1c real-body cross-check (District spacing, courses ON) ===");
|
||||
println!(" window centred at district {center:?} (river cell {river_cell:?})");
|
||||
println!(" {}\n", rayon_threads_report());
|
||||
|
||||
let side = 576u32; // ~330K cells
|
||||
let n = side;
|
||||
let cells = (side as u64) * (side as u64);
|
||||
let min_wl_m = scale::DISTRICT_M as u32;
|
||||
|
||||
let _ = build_district_window_layer(
|
||||
seed,
|
||||
"GJ1c",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
center,
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
);
|
||||
|
||||
let t0 = Instant::now();
|
||||
let layer = build_district_window_layer(
|
||||
seed,
|
||||
"GJ1c",
|
||||
¶ms,
|
||||
&ta,
|
||||
&rn,
|
||||
center,
|
||||
n,
|
||||
&climate,
|
||||
WindowGranularity::District,
|
||||
min_wl_m,
|
||||
);
|
||||
let elapsed = t0.elapsed();
|
||||
|
||||
let ms = elapsed.as_secs_f64() * 1000.0;
|
||||
let ns_per_cell = elapsed.as_secs_f64() * 1e9 / cells as f64;
|
||||
println!(
|
||||
" side={side} n={n} cells={cells} courses_in_window={}: \
|
||||
{ms:.2} ms, {ns_per_cell:.1} ns/cell ({:.3} us/cell)",
|
||||
layer.courses.len(),
|
||||
ns_per_cell / 1000.0
|
||||
);
|
||||
println!();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// T-1178 (Measurement ②) — real 16:9 rectangular canvases, replica loop.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Row-chunked derive over a REAL (non-square) canvas rectangle, mirroring
|
||||
/// `build_district_window_layer`'s internal loop shape cell-for-cell (see
|
||||
/// module doc for the explicit diff against the real function). Returns
|
||||
/// (elapsed, per_cell_ns).
|
||||
///
|
||||
/// **Courses are FORCE-EMPTY here, disclosed (PR #198 review, Hoshe H2):**
|
||||
/// every call site below passes `&[]` for `nearby_courses` — there is no
|
||||
/// `RiverNetwork`, no `invent_courses_near_window` call, and no
|
||||
/// `river_course::near_perennial_water` riparian test running per cell. This
|
||||
/// is a real, measured gap versus production, not a rounding footnote:
|
||||
/// `zoom_ladder_bench.rs`'s `bench_course_cost_on_vs_off` measured the
|
||||
/// courses-on-vs-off delta directly at District cap (n=64, real GJ1c
|
||||
/// geometry) as **+0.09–0.21 ms against a ~5 ms baseline (under 5%)**. Every
|
||||
/// number produced by this function — the three 16:9 canvas benches AND the
|
||||
/// 83K deep-step bench — excludes that cost. It is EXCLUDED, not zero in
|
||||
/// production; readers citing a rectangular-canvas number from this file as
|
||||
/// "the real per-cell cost including courses" are citing it wrong. The
|
||||
/// courses-inclusive numbers live only in the square
|
||||
/// `build_district_window_layer`-backed benches above (which pass a real
|
||||
/// `RiverNetwork` and print `courses_in_window`).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn rect_window_replica(
|
||||
seed: SeedChain,
|
||||
body_id: &str,
|
||||
params: &BodyParams,
|
||||
ta: &TerrainAnalysis,
|
||||
climate: &ClimateConstants,
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
step_m: f64,
|
||||
min_wavelength_m: f64,
|
||||
) -> (std::time::Duration, f64) {
|
||||
use rayon::prelude::*;
|
||||
|
||||
let cells = (cols as u64) * (rows as u64);
|
||||
let t0 = Instant::now();
|
||||
|
||||
// One Rayon task per row (matches build_district_window_layer's own
|
||||
// chunking granularity exactly — row-chunked, not per-cell).
|
||||
let row_results: Vec<u64> = (0..rows)
|
||||
.into_par_iter()
|
||||
.map(|row| {
|
||||
let mut row_acc: u64 = 0;
|
||||
for col in 0..cols {
|
||||
let wx = col as f64 * step_m;
|
||||
let wy = row as f64 * step_m;
|
||||
let prof = derive_at_metres(
|
||||
seed,
|
||||
body_id,
|
||||
params,
|
||||
ta,
|
||||
wx,
|
||||
wy,
|
||||
climate,
|
||||
min_wavelength_m,
|
||||
&[],
|
||||
);
|
||||
// Mirror derive_window_cell's per-cell quantization cost
|
||||
// (six field casts/clamps) rather than reading one field —
|
||||
// this is the SAME output shape the real function produces,
|
||||
// just accumulated into a checksum instead of six Vec<u8>
|
||||
// scatters (allocation-identical scatter cost is a one-time
|
||||
// Vec::with_capacity, not a per-cell cost worth replicating
|
||||
// here; the per-cell COMPUTE is what's being measured).
|
||||
let morphology = prof.morphology_zone as u8;
|
||||
let elev_q = prof.elev_q.clamp(0, 100) as u8;
|
||||
let moisture_q = prof.moisture_q.clamp(0, 100) as u8;
|
||||
let vegetation = prof.vegetation_class as u8;
|
||||
let glaciation = prof.glaciation_grade as u8;
|
||||
row_acc ^= morphology as u64
|
||||
^ elev_q as u64
|
||||
^ moisture_q as u64
|
||||
^ vegetation as u64
|
||||
^ glaciation as u64;
|
||||
}
|
||||
std::hint::black_box(row_acc)
|
||||
})
|
||||
.collect();
|
||||
|
||||
let checksum: u64 = row_results.into_iter().fold(0, |a, b| a ^ b);
|
||||
std::hint::black_box(checksum);
|
||||
|
||||
let elapsed = t0.elapsed();
|
||||
let ns_per_cell = elapsed.as_secs_f64() * 1e9 / cells as f64;
|
||||
(elapsed, ns_per_cell)
|
||||
}
|
||||
|
||||
/// The three named real step-canvas shapes (768x432 / 1920x1080 / 3840x2160),
|
||||
/// District spacing + District cutoff, through the row-chunked replica loop.
|
||||
/// This is the DIRECT answer to "does 4,096-cell par_iter throughput hold at
|
||||
/// real canvas sizes" for the actual non-square shapes the workshop brief
|
||||
/// names, cross-checked against the square production-fn numbers above.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_rect_canvas_district_spacing() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
let min_wl_m = scale::DISTRICT_M as f64;
|
||||
|
||||
println!(
|
||||
"\n=== T-1178 real 16:9 canvas bench, replica row-chunked loop (District spacing) ==="
|
||||
);
|
||||
println!(" {}\n", rayon_threads_report());
|
||||
|
||||
let shapes: [(u32, u32, &str); 3] = [
|
||||
(768, 432, "768x432 (5x5px/gridunit fallback, ~330K)"),
|
||||
(1920, 1080, "1920x1080 (midpoint, ~2.07M)"),
|
||||
(3840, 2160, "3840x2160 (1x1 ideal, ~8.3M)"),
|
||||
];
|
||||
|
||||
for (cols, rows, label) in shapes {
|
||||
let cells = (cols as u64) * (rows as u64);
|
||||
let (elapsed, ns_per_cell) = rect_window_replica(
|
||||
seed, "bench", ¶ms, &ta, &climate, cols, rows, min_wl_m, min_wl_m,
|
||||
);
|
||||
let ms = elapsed.as_secs_f64() * 1000.0;
|
||||
println!(
|
||||
" {label}: cells={cells:>10} {ms:>9.2} ms warm, {ns_per_cell:>7.1} ns/cell \
|
||||
({:.3} us/cell)",
|
||||
ns_per_cell / 1000.0
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
|
||||
/// Single-thread version of the same three rectangles (cutoff-matched) —
|
||||
/// completes the parallel-speedup comparison for the real canvas shapes.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_rect_canvas_district_spacing_single_thread() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
let min_wl_m = scale::DISTRICT_M as f64;
|
||||
|
||||
println!("\n=== T-1178 real 16:9 canvas bench, SINGLE THREAD (District spacing) ===\n");
|
||||
|
||||
let pool = rayon::ThreadPoolBuilder::new()
|
||||
.num_threads(1)
|
||||
.build()
|
||||
.expect("build single-thread rayon pool");
|
||||
|
||||
let shapes: [(u32, u32, &str); 3] = [
|
||||
(768, 432, "768x432 (~330K)"),
|
||||
(1920, 1080, "1920x1080 (~2.07M)"),
|
||||
(3840, 2160, "3840x2160 (~8.3M)"),
|
||||
];
|
||||
|
||||
for (cols, rows, label) in shapes {
|
||||
let cells = (cols as u64) * (rows as u64);
|
||||
let (elapsed, ns_per_cell) = pool.install(|| {
|
||||
rect_window_replica(
|
||||
seed, "bench", ¶ms, &ta, &climate, cols, rows, min_wl_m, min_wl_m,
|
||||
)
|
||||
});
|
||||
let ms = elapsed.as_secs_f64() * 1000.0;
|
||||
println!(
|
||||
" {label}: cells={cells:>10} {ms:>9.2} ms, {ns_per_cell:>7.1} ns/cell \
|
||||
({:.3} us/cell)",
|
||||
ns_per_cell / 1000.0
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// T-1154 (Measurement ③) — block (128 m) and tile-adjacent spacing.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Per-cell derive cost at Block (128 m) and Tile-adjacent (1 m, 4 m)
|
||||
/// spacing, with the matching octave cutoff active — the ladder rungs
|
||||
/// `atlas-zoom-ladder-t1143.md` §2 marks "not estimated". Uses the SAME
|
||||
/// 64x64 = 4,096-cell sweep shape as `zoom_ladder_bench.rs`'s District/
|
||||
/// Quarter sweeps so the per-cell rate is directly comparable across every
|
||||
/// rung on one table.
|
||||
///
|
||||
/// `MIN_WL_BANDS_M` (the request-facing quantized cutoff set,
|
||||
/// `layer_proxy.rs:451`) bottoms out at 1,024 m (Quarter's own band) — there
|
||||
/// is NO existing wire-facing band for Block or Tile. This bench calls
|
||||
/// `derive_at_metres` directly with a cutoff value no real window request can
|
||||
/// carry today (same "call the underlying function, not the wire path"
|
||||
/// discipline as the T-1178 benches above), matching each spacing to its OWN
|
||||
/// Nyquist floor (cutoff = spacing) the same way the existing District/
|
||||
/// Quarter sweeps do.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_block_and_tile_spacing_4096_cells() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
let grid_side = 64u32; // 4,096 cells — matches zoom_ladder_bench.rs's shape
|
||||
|
||||
println!("\n=== T-1154 block/tile-spacing derive_at_metres benchmark (4,096-cell sweep) ===");
|
||||
println!(
|
||||
"grid: {grid_side}x{grid_side} = {} cells/sweep\n",
|
||||
grid_side * grid_side
|
||||
);
|
||||
|
||||
let block_m = scale::BLOCK_M as f64; // 128 m
|
||||
let tile_4m = 4.0_f64; // coarsest "tile-adjacent" spacing named in the ticket
|
||||
let tile_1m = 1.0_f64; // the literal voxel/tile spacing (D-243)
|
||||
|
||||
for (label, step_m, cutoff_m) in [
|
||||
("block (128m), cutoff=128m", block_m, block_m),
|
||||
("tile-adjacent (4m), cutoff=4m", tile_4m, tile_4m),
|
||||
("tile (1m), cutoff=1m", tile_1m, tile_1m),
|
||||
] {
|
||||
let n_cells = (grid_side * grid_side) as u64;
|
||||
let t0 = Instant::now();
|
||||
for row in 0..grid_side {
|
||||
for col in 0..grid_side {
|
||||
let wx = col as f64 * step_m;
|
||||
let wy = row as f64 * step_m;
|
||||
let prof =
|
||||
derive_at_metres(seed, "bench", ¶ms, &ta, wx, wy, &climate, cutoff_m, &[]);
|
||||
std::hint::black_box(prof.elev_q);
|
||||
}
|
||||
}
|
||||
let elapsed = t0.elapsed();
|
||||
let per_cell_ns = elapsed.as_secs_f64() * 1e9 / n_cells as f64;
|
||||
println!(
|
||||
" {label:<32}: {:>8.2} ms total, {:>7.1} ns/cell ({:.3} us/cell)",
|
||||
elapsed.as_secs_f64() * 1000.0,
|
||||
per_cell_ns,
|
||||
per_cell_ns / 1000.0
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
|
||||
/// The realistic deep-step canvas: at the ~10 px-per-1m-tile bottom-out on a
|
||||
/// 2,160-px SMALLER axis (a 1920x1080 viewport, portrait-safe on the smaller
|
||||
/// dimension per the brief's "viewport's smaller axis" convention), the
|
||||
/// world extent covered is `1080 px / 10 px-per-tile = 108 tiles = 108 m` on
|
||||
/// the smaller axis, `1920 / 10 = 192 m` on the larger — a 192m x 108m
|
||||
/// window at 1 m spacing. Re-stated against the ticket's own worked example
|
||||
/// (2160 px smaller axis, ~216m x ~384m) for the 3840x2160 canvas instead:
|
||||
/// `2160/10 = 216 m` (smaller axis), `3840/10 = 384 m` (larger axis) — EXACT
|
||||
/// geometry used below, matching the ticket's stated numbers precisely.
|
||||
///
|
||||
/// At 1 m spacing that is a 216x384 CELL grid (1 world-metre per gridunit,
|
||||
/// 10 screen px per gridunit) = 82,944 cells — the ticket's "~83K cells"
|
||||
/// figure, confirmed exactly (216 * 384 = 82,944).
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_deep_step_realistic_canvas_83k_cells() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
|
||||
// Geometry: 3840x2160 canvas, 10 px per 1m tile, smaller axis 2160.
|
||||
// world extent: 216 m (smaller/rows) x 384 m (larger/cols), 1 m spacing.
|
||||
let rows = 216u32; // world metres on the smaller (2160px/10) axis
|
||||
let cols = 384u32; // world metres on the larger (3840px/10) axis
|
||||
let cells = (rows as u64) * (cols as u64);
|
||||
assert_eq!(
|
||||
cells, 82_944,
|
||||
"geometry must match the ticket's stated ~83K cells exactly"
|
||||
);
|
||||
|
||||
let step_m = 1.0_f64;
|
||||
let cutoff_m = 1.0_f64; // Nyquist-matched to 1 m spacing
|
||||
|
||||
println!("\n=== T-1154 deep-step realistic-canvas bench (216m x 384m @ 1m spacing) ===");
|
||||
println!(
|
||||
" geometry: 3840x2160 canvas @ 10px/tile, smaller axis 2160 -> 216m, larger axis 3840 -> 384m"
|
||||
);
|
||||
println!(" cells = 216 * 384 = {cells}");
|
||||
println!(" {}\n", rayon_threads_report());
|
||||
|
||||
// Parallel, through the SAME row-chunked replica loop T-1178 uses (this
|
||||
// is the production par_iter SHAPE, not the exact function, for the same
|
||||
// reason as the T-1178 rectangular benches: no square-only production fn
|
||||
// covers a non-square metre-spacing window).
|
||||
let (elapsed_par, ns_per_cell_par) = rect_window_replica(
|
||||
seed, "bench", ¶ms, &ta, &climate, cols, rows, step_m, cutoff_m,
|
||||
);
|
||||
println!(
|
||||
" PARALLEL (row-chunked): {:.2} ms, {:.1} ns/cell ({:.3} us/cell)",
|
||||
elapsed_par.as_secs_f64() * 1000.0,
|
||||
ns_per_cell_par,
|
||||
ns_per_cell_par / 1000.0
|
||||
);
|
||||
|
||||
// Single-thread comparison.
|
||||
let pool = rayon::ThreadPoolBuilder::new()
|
||||
.num_threads(1)
|
||||
.build()
|
||||
.expect("build single-thread rayon pool");
|
||||
let (elapsed_seq, ns_per_cell_seq) = pool.install(|| {
|
||||
rect_window_replica(
|
||||
seed, "bench", ¶ms, &ta, &climate, cols, rows, step_m, cutoff_m,
|
||||
)
|
||||
});
|
||||
println!(
|
||||
" SINGLE-THREAD: {:.2} ms, {:.1} ns/cell ({:.3} us/cell)",
|
||||
elapsed_seq.as_secs_f64() * 1000.0,
|
||||
ns_per_cell_seq,
|
||||
ns_per_cell_seq / 1000.0
|
||||
);
|
||||
println!(
|
||||
" speedup: {:.2}x\n",
|
||||
elapsed_seq.as_secs_f64() / elapsed_par.as_secs_f64()
|
||||
);
|
||||
}
|
||||
|
||||
/// Cutoff sweep AT Block-spacing sample positions, varying the cutoff itself
|
||||
/// from uncut (0, every octave band in `OCTAVE_WAVELENGTHS_M` [4,096..32,768]
|
||||
/// AND `VOXEL_OCTAVE_WAVELENGTHS_M` [128..1,024] survives) up through
|
||||
/// District-coarse (2,048m, truncates nothing extra vs. uncut — every
|
||||
/// `OCTAVE_WAVELENGTHS_M` entry is still ≥2,048) to Region-coarse (204,800m,
|
||||
/// truncates EVERY octave in both bands, `enveloped_fbm`'s "every octave cut"
|
||||
/// empty-sum guard fires).
|
||||
///
|
||||
/// **Why "cutoff=128m (Block's own Nyquist floor)" shows ZERO delta vs.
|
||||
/// uncut** (confirmed by direct read of `enveloped_fbm`,
|
||||
/// `detail_scatter.rs:198-251`: `if wl < min_wavelength_m { skip }` — a
|
||||
/// cutoff only skips octaves STRICTLY FINER than itself. At Block's own
|
||||
/// floor (128m), every entry in BOTH octave arrays is `>= 128m`
|
||||
/// (`VOXEL_OCTAVE_WAVELENGTHS_M`'s finest is exactly 128m, `>=` not `<`), so
|
||||
/// nothing is skipped — Block sits at the bottom of the invented-detail
|
||||
/// octave stack, with nothing finer left to truncate. This is a genuine,
|
||||
/// verified finding (not a bench bug): **the cutoff mechanism has no
|
||||
/// truncation work left to do at Block spacing or finer** — every rung from
|
||||
/// Block down to Tile pays the SAME full per-cell octave-sum cost, because
|
||||
/// the const octave arrays bottom out at 128m and neither
|
||||
/// `MOSAIC_OCTAVE_WAVELENGTHS_M` (64/32/16/8m) nor any Tile-specific band is
|
||||
/// wired into `derive_at_metres`'s call graph (see the module doc's
|
||||
/// voxel_mosaic finding). The cutoff only pays off at COARSER rungs
|
||||
/// (District, Quarter, Region) where it truncates the fine end of the octave
|
||||
/// stack that those rungs' sample density can't resolve anyway.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_block_cutoff_confirms_savings() {
|
||||
let hm = bench_hm();
|
||||
let ta = bench_ta(&hm);
|
||||
let params = bench_params();
|
||||
let climate = ClimateConstants::default();
|
||||
let seed = SeedChain::root(99).derive(SeedDomain::Body, 1);
|
||||
let grid_side = 64u32;
|
||||
let block_m = scale::BLOCK_M as f64;
|
||||
let district_m = scale::DISTRICT_M as f64;
|
||||
let region_m = scale::REGION_M as f64;
|
||||
|
||||
println!("\n=== T-1154 Block-spacing-position cutoff sweep (varying cutoff value) ===\n");
|
||||
|
||||
for (label, cutoff_m) in [
|
||||
("uncut (cutoff=0)", 0.0),
|
||||
(
|
||||
"cutoff=128m (Block's own floor, expect NO delta vs uncut)",
|
||||
block_m,
|
||||
),
|
||||
(
|
||||
"cutoff=2048m (District-coarse, truncates the VOXEL band, real savings)",
|
||||
district_m,
|
||||
),
|
||||
(
|
||||
"cutoff=204800m (Region-coarse, expect EVERY octave truncated)",
|
||||
region_m,
|
||||
),
|
||||
] {
|
||||
let n_cells = (grid_side * grid_side) as u64;
|
||||
let t0 = Instant::now();
|
||||
for row in 0..grid_side {
|
||||
for col in 0..grid_side {
|
||||
// Sample POSITIONS stay at Block spacing throughout — only the
|
||||
// cutoff VALUE varies — so this isolates the cutoff's cost
|
||||
// effect from a spacing change.
|
||||
let wx = col as f64 * block_m;
|
||||
let wy = row as f64 * block_m;
|
||||
let prof =
|
||||
derive_at_metres(seed, "bench", ¶ms, &ta, wx, wy, &climate, cutoff_m, &[]);
|
||||
std::hint::black_box(prof.elev_q);
|
||||
}
|
||||
}
|
||||
let elapsed = t0.elapsed();
|
||||
let per_cell_ns = elapsed.as_secs_f64() * 1e9 / n_cells as f64;
|
||||
println!(
|
||||
" {label:<58}: {:>8.2} ms total, {:>7.1} ns/cell ({:.3} us/cell)",
|
||||
elapsed.as_secs_f64() * 1000.0,
|
||||
per_cell_ns,
|
||||
per_cell_ns / 1000.0
|
||||
);
|
||||
}
|
||||
println!();
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
//! Equilibrium hydrology solver benchmarks (T-1177, body-map-viewer workshop
|
||||
//! measurement ①).
|
||||
//!
|
||||
//! Measures `hydrology_equilibrium::solve` at today's Layer-1 working grid
|
||||
//! (512×256) and at two 4K-class synthetic grids (~768×432 ≈ 330K cells,
|
||||
//! matching the workshop's per-gridunit derive measurement ②'s canvas size
|
||||
//! for direct comparison; 3840×2160 ≈ 8.3M cells, the "computer catches
|
||||
//! fire" ceiling case). Real GJ1c heightmap data is used at 512×256 (the
|
||||
//! actual production working-grid size — no upsampling needed there); the
|
||||
//! two larger grids use synthetic elevation (documented in
|
||||
//! `synthetic_elevation` below) since no committed heightmap PNG is stored
|
||||
//! at those resolutions and generating/committing new fixture PNGs is out of
|
||||
//! scope for a measurement prototype.
|
||||
//!
|
||||
//! Run: `cargo test --release --test hydrology_equilibrium_bench -- --ignored --nocapture`
|
||||
//! (debug numbers are not representative — this crate's other benches use
|
||||
//! the same release-only convention).
|
||||
//!
|
||||
//! Hardware: 16 cores, Rayon default thread pool (14 workers observed
|
||||
//! elsewhere in this repo's benches on the same machine).
|
||||
|
||||
use std::time::Instant;
|
||||
|
||||
use settled_reach_server::atlas::heightmap::load_heightmap_png;
|
||||
use settled_reach_server::atlas::hydrology_equilibrium::{solve, ClimateInputs};
|
||||
|
||||
/// Deterministic synthetic elevation for grids larger than any committed
|
||||
/// heightmap PNG. NOT a real body — a smooth multi-octave ridged surface
|
||||
/// (a few sine terms at different frequencies/phases, summed and
|
||||
/// normalized) chosen to produce a realistic MIX of basins or the solver
|
||||
/// would have nothing to fill: a plain gradient (as `zoom_ladder_bench.rs`'s
|
||||
/// `bench_hm` uses for its unrelated per-cell derive cost) has almost no
|
||||
/// interior depressions, which would make this bench measure an
|
||||
/// unrepresentative best case (priority-flood on a monotonic slope is
|
||||
/// nearly free — the expensive part is basin interiors + overflow search).
|
||||
/// Purely a function of `(row, col, width, height)` — the same call always
|
||||
/// produces the same bytes, so the resulting elevation grid is itself
|
||||
/// deterministic (D-010), even though it is synthetic rather than sourced
|
||||
/// from a real body.
|
||||
fn synthetic_elevation(width: u32, height: u32) -> Vec<f32> {
|
||||
let w = width as f64;
|
||||
let h = height as f64;
|
||||
let n = (width * height) as usize;
|
||||
(0..n)
|
||||
.map(|i| {
|
||||
let row = (i / width as usize) as f64;
|
||||
let col = (i % width as usize) as f64;
|
||||
let x = col / w;
|
||||
let y = row / h;
|
||||
// Several sine octaves at different frequencies/phases — enough
|
||||
// basins (local minima not at the grid boundary) that the
|
||||
// priority-flood + overflow-search work is representative, not
|
||||
// a degenerate monotonic slope.
|
||||
let v = 0.5
|
||||
+ 0.25
|
||||
* (x * std::f64::consts::TAU * 3.0).sin()
|
||||
* (y * std::f64::consts::TAU * 2.0).cos()
|
||||
+ 0.15
|
||||
* (x * std::f64::consts::TAU * 7.3 + 1.7).sin()
|
||||
* (y * std::f64::consts::TAU * 5.1).sin()
|
||||
+ 0.10
|
||||
* (x * std::f64::consts::TAU * 13.0).cos()
|
||||
* (y * std::f64::consts::TAU * 11.0 + 0.4).sin();
|
||||
v.clamp(0.0, 1.0) as f32
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn gj1c_512x256() -> (Vec<f32>, f32) {
|
||||
let src = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("../wiki/star-systems/GJ-1/bodies/GJ1c/heightmap.png");
|
||||
let heightmap = load_heightmap_png(&src, "GJ1c", 0.3).expect("decode committed GJ1c heightmap");
|
||||
let small = heightmap.downsample(512, 256); // GRID_W x GRID_H, the real production working grid
|
||||
(small.data, small.sea_level)
|
||||
}
|
||||
|
||||
fn default_climate() -> ClimateInputs {
|
||||
ClimateInputs { moisture_q: 55 }
|
||||
}
|
||||
|
||||
fn run_and_report(label: &str, width: u32, height: u32, elevation: &[f32], sea_level: f32) {
|
||||
let n_cells = (width as u64) * (height as u64);
|
||||
|
||||
// Cold run.
|
||||
let t0 = Instant::now();
|
||||
let result_cold = solve(elevation, width, height, sea_level, default_climate());
|
||||
let cold = t0.elapsed();
|
||||
|
||||
// Warm run (same process, allocator/cache warm — same input).
|
||||
let t1 = Instant::now();
|
||||
let result_warm = solve(elevation, width, height, sea_level, default_climate());
|
||||
let warm = t1.elapsed();
|
||||
|
||||
let lake_cells: usize = result_cold.basins.iter().map(|b| b.cells.len()).sum();
|
||||
let carved_cells = result_cold.cliff_edge.iter().filter(|&&c| c).count();
|
||||
let endorheic_count = result_cold
|
||||
.basins
|
||||
.iter()
|
||||
.filter(|b| {
|
||||
!b.cells.is_empty()
|
||||
&& matches!(
|
||||
b.outcome,
|
||||
settled_reach_server::atlas::hydrology_equilibrium::BasinOutcome::Endorheic { .. }
|
||||
)
|
||||
})
|
||||
.count();
|
||||
let overflow_count = result_cold
|
||||
.basins
|
||||
.iter()
|
||||
.filter(|b| {
|
||||
!b.cells.is_empty()
|
||||
&& matches!(
|
||||
b.outcome,
|
||||
settled_reach_server::atlas::hydrology_equilibrium::BasinOutcome::Overflow { .. }
|
||||
)
|
||||
})
|
||||
.count();
|
||||
|
||||
println!("\n=== {label} ({width}x{height} = {n_cells} cells) ===");
|
||||
println!(
|
||||
" cold: {:>9.2} ms total, {:>8.1} ns/cell",
|
||||
cold.as_secs_f64() * 1000.0,
|
||||
cold.as_secs_f64() * 1e9 / n_cells as f64
|
||||
);
|
||||
println!(
|
||||
" warm: {:>9.2} ms total, {:>8.1} ns/cell",
|
||||
warm.as_secs_f64() * 1000.0,
|
||||
warm.as_secs_f64() * 1e9 / n_cells as f64
|
||||
);
|
||||
println!(
|
||||
" basins: {} total ({} overflow, {} endorheic, {} empty/no-depression), \
|
||||
lake cells: {lake_cells}, carved gorge cells: {carved_cells}",
|
||||
result_cold.basins.len(),
|
||||
overflow_count,
|
||||
endorheic_count,
|
||||
result_cold.basins.len() - overflow_count - endorheic_count,
|
||||
);
|
||||
std::hint::black_box(&result_warm);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_512x256_real_gj1c() {
|
||||
let (elev, sea_level) = gj1c_512x256();
|
||||
run_and_report(
|
||||
"512x256 (real GJ1c, production working-grid size)",
|
||||
512,
|
||||
256,
|
||||
&elev,
|
||||
sea_level,
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_768x432_synthetic() {
|
||||
let (w, h) = (768u32, 432u32);
|
||||
let elev = synthetic_elevation(w, h);
|
||||
run_and_report(
|
||||
"768x432 (~330K cells, 4K-class synthetic — see synthetic_elevation doc)",
|
||||
w,
|
||||
h,
|
||||
&elev,
|
||||
0.35,
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_3840x2160_synthetic() {
|
||||
let (w, h) = (3840u32, 2160u32);
|
||||
let elev = synthetic_elevation(w, h);
|
||||
run_and_report(
|
||||
"3840x2160 (~8.3M cells, 4K synthetic — see synthetic_elevation doc)",
|
||||
w,
|
||||
h,
|
||||
&elev,
|
||||
0.35,
|
||||
);
|
||||
}
|
||||
|
||||
/// Determinism proof at bench scale (T-1177 mandatory deliverable): same
|
||||
/// seed + input → byte-identical solver output, twice, on a non-trivial
|
||||
/// grid (not just the small fixtures already covered by the module's own
|
||||
/// unit tests).
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn determinism_at_330k_cells() {
|
||||
let (w, h) = (768u32, 432u32);
|
||||
let elev = synthetic_elevation(w, h);
|
||||
let r1 = solve(&elev, w, h, 0.35, default_climate());
|
||||
let r2 = solve(&elev, w, h, 0.35, default_climate());
|
||||
assert_eq!(
|
||||
r1.filled_scaled, r2.filled_scaled,
|
||||
"filled surface must be byte-identical"
|
||||
);
|
||||
assert_eq!(
|
||||
r1.channel_depth_scaled, r2.channel_depth_scaled,
|
||||
"carved channel depth must be byte-identical"
|
||||
);
|
||||
assert_eq!(
|
||||
r1.cliff_edge, r2.cliff_edge,
|
||||
"cliff-edge flags must be byte-identical"
|
||||
);
|
||||
assert_eq!(
|
||||
r1.basins.len(),
|
||||
r2.basins.len(),
|
||||
"basin count must be identical"
|
||||
);
|
||||
for (a, b) in r1.basins.iter().zip(r2.basins.iter()) {
|
||||
assert_eq!(a.basin_id, b.basin_id);
|
||||
assert_eq!(a.cells, b.cells);
|
||||
assert_eq!(a.spill_level_scaled, b.spill_level_scaled);
|
||||
assert_eq!(a.spill_cell, b.spill_cell);
|
||||
assert_eq!(format!("{:?}", a.outcome), format!("{:?}", b.outcome));
|
||||
}
|
||||
println!(
|
||||
"\n=== determinism proof (768x432, {} basins) — byte-identical across two solves ===",
|
||||
r1.basins.len()
|
||||
);
|
||||
}
|
||||
|
||||
/// Rayon-parallel throughput: the REAL production shape is N independent
|
||||
/// bodies, each solved once (not one body's solve parallelized internally —
|
||||
/// priority-flood's heap and the overflow Dijkstra search are both globally
|
||||
/// sequential by nature, same as `road_graph.rs`'s A*). This measures what
|
||||
/// "always keep hydrology for ~273 bodies" would cost in wall-clock if
|
||||
/// solved across the Rayon pool, at the 512×256 production grid size —
|
||||
/// directly answering the workshop's red-flag-2-adjacent question of
|
||||
/// whether per-body-open hydrology is affordable at scale.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn bench_parallel_273_bodies_at_512x256() {
|
||||
use rayon::prelude::*;
|
||||
|
||||
let (elev, sea_level) = gj1c_512x256();
|
||||
let body_count = 273usize;
|
||||
|
||||
let t0 = Instant::now();
|
||||
let total_basins: usize = (0..body_count)
|
||||
.into_par_iter()
|
||||
.map(|_| {
|
||||
let result = solve(&elev, 512, 256, sea_level, default_climate());
|
||||
result.basins.len()
|
||||
})
|
||||
.sum();
|
||||
let elapsed = t0.elapsed();
|
||||
|
||||
println!(
|
||||
"\n=== {body_count} bodies x 512x256, Rayon par_iter ({} threads available) ===",
|
||||
std::thread::available_parallelism()
|
||||
.map(|n| n.get())
|
||||
.unwrap_or(0)
|
||||
);
|
||||
println!(
|
||||
" {:>9.2} ms total, {:>7.2} ms/body average, {total_basins} basins summed",
|
||||
elapsed.as_secs_f64() * 1000.0,
|
||||
elapsed.as_secs_f64() * 1000.0 / body_count as f64
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,665 @@
|
||||
//! T-1179 — wire-size table for step-canvas encodings (body-map-viewer
|
||||
//! workshop, measurement ④).
|
||||
//!
|
||||
//! Produces a REAL step-canvas-shaped [`DistrictWindowLayer`]-field dataset
|
||||
//! (elev_q, morphology, temp_dc, moisture_q, vegetation, glaciation — the
|
||||
//! exact six arrays that struct ships today, D-226 T-1124 amendment §4) at
|
||||
//! ~330K/2.07M/8.3M gridunits, via the SAME derivation call
|
||||
//! `build_district_window_layer`'s row-chunked `par_iter` uses
|
||||
//! (`derive_at_metres`, district spacing, no octave cutoff, no river-course
|
||||
//! packing — courses are a separate variable-length field orthogonal to this
|
||||
//! raster wire-size question). Real derived data (not synthetic noise or
|
||||
//! constant fills) so RLE/PNG compression ratios reflect genuine spatial
|
||||
//! coherence — see the workshop brief's measurement ④ scope note.
|
||||
//!
|
||||
//! Candidate encodings measured on the SAME canvas:
|
||||
//! (a) raw dense `u8`/`i16` arrays through `rmp_serde` (today's wire format)
|
||||
//! (b) bit-packed (sub-byte field widths, see `pack_bits` doc)
|
||||
//! (c) per-field run-length encoding
|
||||
//! (d) PNG-encoded raster per field (the `png` crate — already a main
|
||||
//! dependency, `server/Cargo.toml`; no new dependency added)
|
||||
//! (e) PNG applied to the bit-packed planes (cheap combination of b+d)
|
||||
//!
|
||||
//! Run: `cargo test --release --test wire_encoding_bench -- --ignored --nocapture`
|
||||
//! (debug numbers are not representative — this repo's benches are always run
|
||||
//! `--release`, matching `zoom_ladder_bench.rs`'s convention).
|
||||
//!
|
||||
//! Body/seed: GJ338Bd, `--seed yolo` (`seed_to_u64("yolo")`) — the same
|
||||
//! body+seed pair `aliveness_probe`'s doc example and the believability
|
||||
//! harness default to (`server/src/atlas/believability.rs`).
|
||||
|
||||
use std::io::Cursor;
|
||||
use std::time::Instant;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use settled_reach_server::atlas::believability::seed_to_u64;
|
||||
use settled_reach_server::atlas::district_profile::{
|
||||
derive_at_metres, BodyParams, ClimateConstants,
|
||||
};
|
||||
use settled_reach_server::atlas::drainage;
|
||||
use settled_reach_server::atlas::features::TerrainAnalysis;
|
||||
use settled_reach_server::atlas::heightmap::load_heightmap_png;
|
||||
use settled_reach_server::atlas::layer_proxy::REGION_TEMP_NONE_DC;
|
||||
use settled_reach_server::atlas::scale;
|
||||
use settled_reach_server::seed::SeedChain;
|
||||
|
||||
/// The six wire arrays `DistrictWindowLayer` ships today (layer_proxy.rs),
|
||||
/// derived at full canvas size rather than the 4,096-cell window cap.
|
||||
#[derive(Serialize, Deserialize)]
|
||||
struct WireCanvas {
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
morphology: Vec<u8>,
|
||||
elev_q: Vec<u8>,
|
||||
temp_dc: Vec<i16>,
|
||||
moisture_q: Vec<u8>,
|
||||
vegetation: Vec<u8>,
|
||||
glaciation: Vec<u8>,
|
||||
}
|
||||
|
||||
fn load_gj338bd() -> (BodyParams, TerrainAnalysis, SeedChain) {
|
||||
let src = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("../wiki/star-systems/GJ-338B/bodies/GJ338Bd/heightmap.png");
|
||||
let heightmap =
|
||||
load_heightmap_png(&src, "GJ338Bd", 0.3).expect("decode committed GJ338Bd heightmap");
|
||||
let small = heightmap.downsample(512, 256); // GRID_W x GRID_H, production working grid
|
||||
let dr = drainage::analyze(&small.data, small.width, small.height, small.sea_level);
|
||||
let ta = TerrainAnalysis::analyze(&small, &dr);
|
||||
let params = BodyParams {
|
||||
hydrosphere: Some("ocean".into()),
|
||||
atmosphere: Some("breathable".into()),
|
||||
planet_class: Some("temperate".into()),
|
||||
body_radius_km: Some(6371.0),
|
||||
..Default::default()
|
||||
};
|
||||
let seed = SeedChain::for_body(seed_to_u64("yolo"), "GJ338Bd");
|
||||
(params, ta, seed)
|
||||
}
|
||||
|
||||
/// Derive a `cols x rows` canvas at district spacing (2,048 m/cell), origin
|
||||
/// at world (0,0), via the SAME `derive_at_metres` call + row-chunked
|
||||
/// `par_iter` shape `build_district_window_layer` uses internally
|
||||
/// (`layer_proxy.rs::derive_window_cell`/the row-scatter loop) — just at
|
||||
/// canvas sizes above the 4,096-cell `WIRE_CAP_CELLS` window ceiling, since
|
||||
/// that ceiling is a SERVED-window cap, not a derivation-cost cap (the
|
||||
/// workshop question is what a whole step canvas costs, pre-windowing).
|
||||
fn derive_canvas(
|
||||
seed: SeedChain,
|
||||
params: &BodyParams,
|
||||
ta: &TerrainAnalysis,
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
) -> (WireCanvas, std::time::Duration) {
|
||||
use rayon::prelude::*;
|
||||
let climate = ClimateConstants::default();
|
||||
let step_m = scale::DISTRICT_M as f64;
|
||||
let cells = (cols as usize) * (rows as usize);
|
||||
|
||||
let t0 = Instant::now();
|
||||
let row_results: Vec<Vec<(u8, u8, i16, u8, u8, u8)>> = (0..rows)
|
||||
.into_par_iter()
|
||||
.map(|row| {
|
||||
(0..cols)
|
||||
.map(|col| {
|
||||
let wx = col as f64 * step_m;
|
||||
let wy = row as f64 * step_m;
|
||||
let prof =
|
||||
derive_at_metres(seed, "GJ338Bd", params, ta, wx, wy, &climate, 0.0, &[]);
|
||||
let temp_dc = match prof.temperature_c {
|
||||
Some(t) => ((t * 10.0).round() as i32)
|
||||
.clamp(i16::MIN as i32 + 1, i16::MAX as i32)
|
||||
as i16,
|
||||
None => REGION_TEMP_NONE_DC,
|
||||
};
|
||||
(
|
||||
prof.morphology_zone as u8,
|
||||
prof.elev_q.clamp(0, 100) as u8,
|
||||
temp_dc,
|
||||
prof.moisture_q.clamp(0, 100) as u8,
|
||||
prof.vegetation_class as u8,
|
||||
prof.glaciation_grade as u8,
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.collect();
|
||||
let elapsed = t0.elapsed();
|
||||
|
||||
let mut morphology = Vec::with_capacity(cells);
|
||||
let mut elev_q = Vec::with_capacity(cells);
|
||||
let mut temp_dc = Vec::with_capacity(cells);
|
||||
let mut moisture_q = Vec::with_capacity(cells);
|
||||
let mut vegetation = Vec::with_capacity(cells);
|
||||
let mut glaciation = Vec::with_capacity(cells);
|
||||
for row in row_results {
|
||||
for (m, e, t, mo, v, g) in row {
|
||||
morphology.push(m);
|
||||
elev_q.push(e);
|
||||
temp_dc.push(t);
|
||||
moisture_q.push(mo);
|
||||
vegetation.push(v);
|
||||
glaciation.push(g);
|
||||
}
|
||||
}
|
||||
|
||||
(
|
||||
WireCanvas {
|
||||
cols,
|
||||
rows,
|
||||
morphology,
|
||||
elev_q,
|
||||
temp_dc,
|
||||
moisture_q,
|
||||
vegetation,
|
||||
glaciation,
|
||||
},
|
||||
elapsed,
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Encoding (a): raw dense arrays via rmp_serde — today's wire format.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn encode_rmp(canvas: &WireCanvas) -> (Vec<u8>, std::time::Duration, std::time::Duration) {
|
||||
let t0 = Instant::now();
|
||||
let bytes = rmp_serde::to_vec(canvas).expect("rmp_serde encode");
|
||||
let enc_time = t0.elapsed();
|
||||
let t1 = Instant::now();
|
||||
let decoded: WireCanvas = rmp_serde::from_slice(&bytes).expect("rmp_serde decode");
|
||||
let dec_time = t1.elapsed();
|
||||
std::hint::black_box(decoded.morphology.len());
|
||||
(bytes, enc_time, dec_time)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Encoding (b): bit-packed planes.
|
||||
//
|
||||
// Field widths (minimal, from the real discriminant ranges):
|
||||
// morphology: 0-16 (17 zones, D-239 §6) -> 5 bits
|
||||
// elev_q: 0-100 -> 7 bits
|
||||
// moisture_q: 0-100 -> 7 bits
|
||||
// vegetation: 0-6 (7 classes incl. Marine) -> 3 bits
|
||||
// glaciation: 0-4 (5 grades) -> 3 bits
|
||||
// temp_dc: i16 incl. REGION_TEMP_NONE_DC sentinel -> left at 16 bits
|
||||
// (full dynamic range is genuinely used across class bands +
|
||||
// the sentinel; no safe narrower width without a second
|
||||
// encoding scheme for the sentinel case, out of scope here)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Pack `values` (each `< 2^width`) into a bitstream, LSB-first within each
|
||||
/// byte, fields concatenated in stream order — the simplest fixed-width
|
||||
/// packing (no entropy coding). Returns the packed byte buffer.
|
||||
fn pack_bits(values: &[u8], width: u32) -> Vec<u8> {
|
||||
let mut out = Vec::with_capacity((values.len() * width as usize).div_ceil(8));
|
||||
let mut acc: u32 = 0;
|
||||
let mut acc_bits: u32 = 0;
|
||||
for &v in values {
|
||||
acc |= (v as u32) << acc_bits;
|
||||
acc_bits += width;
|
||||
while acc_bits >= 8 {
|
||||
out.push((acc & 0xFF) as u8);
|
||||
acc >>= 8;
|
||||
acc_bits -= 8;
|
||||
}
|
||||
}
|
||||
if acc_bits > 0 {
|
||||
out.push((acc & 0xFF) as u8);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn unpack_bits(packed: &[u8], width: u32, count: usize) -> Vec<u8> {
|
||||
let mut out = Vec::with_capacity(count);
|
||||
let mut acc: u32 = 0;
|
||||
let mut acc_bits: u32 = 0;
|
||||
let mask = (1u32 << width) - 1;
|
||||
let mut byte_iter = packed.iter();
|
||||
while out.len() < count {
|
||||
while acc_bits < width {
|
||||
let Some(&b) = byte_iter.next() else { break };
|
||||
acc |= (b as u32) << acc_bits;
|
||||
acc_bits += 8;
|
||||
}
|
||||
out.push((acc & mask) as u8);
|
||||
acc >>= width;
|
||||
acc_bits -= width;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
struct BitPacked {
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
morphology_bits: Vec<u8>, // 5 bits/cell
|
||||
elev_q_bits: Vec<u8>, // 7 bits/cell
|
||||
temp_dc: Vec<i16>, // unpacked, full 16 bits (see doc above)
|
||||
moisture_q_bits: Vec<u8>, // 7 bits/cell
|
||||
vegetation_bits: Vec<u8>, // 3 bits/cell
|
||||
glaciation_bits: Vec<u8>, // 3 bits/cell
|
||||
}
|
||||
|
||||
fn encode_bitpacked(canvas: &WireCanvas) -> (Vec<u8>, std::time::Duration, std::time::Duration) {
|
||||
let n = canvas.morphology.len();
|
||||
let t0 = Instant::now();
|
||||
let packed = BitPacked {
|
||||
cols: canvas.cols,
|
||||
rows: canvas.rows,
|
||||
morphology_bits: pack_bits(&canvas.morphology, 5),
|
||||
elev_q_bits: pack_bits(&canvas.elev_q, 7),
|
||||
temp_dc: canvas.temp_dc.clone(),
|
||||
moisture_q_bits: pack_bits(&canvas.moisture_q, 7),
|
||||
vegetation_bits: pack_bits(&canvas.vegetation, 3),
|
||||
glaciation_bits: pack_bits(&canvas.glaciation, 3),
|
||||
};
|
||||
let bytes = rmp_serde::to_vec(&packed).expect("rmp_serde encode bitpacked");
|
||||
let enc_time = t0.elapsed();
|
||||
|
||||
let t1 = Instant::now();
|
||||
let decoded: BitPacked = rmp_serde::from_slice(&bytes).expect("rmp_serde decode bitpacked");
|
||||
let morphology = unpack_bits(&decoded.morphology_bits, 5, n);
|
||||
let elev_q = unpack_bits(&decoded.elev_q_bits, 7, n);
|
||||
let moisture_q = unpack_bits(&decoded.moisture_q_bits, 7, n);
|
||||
let vegetation = unpack_bits(&decoded.vegetation_bits, 3, n);
|
||||
let glaciation = unpack_bits(&decoded.glaciation_bits, 3, n);
|
||||
let dec_time = t1.elapsed();
|
||||
std::hint::black_box((
|
||||
morphology.len(),
|
||||
elev_q.len(),
|
||||
moisture_q.len(),
|
||||
vegetation.len(),
|
||||
glaciation.len(),
|
||||
));
|
||||
(bytes, enc_time, dec_time)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Encoding (c): per-field run-length encoding.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// (run_length, value) pairs, run_length capped at u16::MAX (wraps to a new
|
||||
/// run — no run ever exceeds 65,535 cells, larger than any canvas row here).
|
||||
fn rle_encode_u8(values: &[u8]) -> Vec<(u16, u8)> {
|
||||
let mut out = Vec::new();
|
||||
let mut iter = values.iter();
|
||||
let Some(&first) = iter.next() else {
|
||||
return out;
|
||||
};
|
||||
let mut cur = first;
|
||||
let mut run: u16 = 1;
|
||||
for &v in iter {
|
||||
if v == cur && run < u16::MAX {
|
||||
run += 1;
|
||||
} else {
|
||||
out.push((run, cur));
|
||||
cur = v;
|
||||
run = 1;
|
||||
}
|
||||
}
|
||||
out.push((run, cur));
|
||||
out
|
||||
}
|
||||
|
||||
fn rle_encode_i16(values: &[i16]) -> Vec<(u16, i16)> {
|
||||
let mut out = Vec::new();
|
||||
let mut iter = values.iter();
|
||||
let Some(&first) = iter.next() else {
|
||||
return out;
|
||||
};
|
||||
let mut cur = first;
|
||||
let mut run: u16 = 1;
|
||||
for &v in iter {
|
||||
if v == cur && run < u16::MAX {
|
||||
run += 1;
|
||||
} else {
|
||||
out.push((run, cur));
|
||||
cur = v;
|
||||
run = 1;
|
||||
}
|
||||
}
|
||||
out.push((run, cur));
|
||||
out
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
struct RleCanvas {
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
morphology: Vec<(u16, u8)>,
|
||||
elev_q: Vec<(u16, u8)>,
|
||||
temp_dc: Vec<(u16, i16)>,
|
||||
moisture_q: Vec<(u16, u8)>,
|
||||
vegetation: Vec<(u16, u8)>,
|
||||
glaciation: Vec<(u16, u8)>,
|
||||
}
|
||||
|
||||
struct RlePerFieldRuns {
|
||||
morphology: usize,
|
||||
elev_q: usize,
|
||||
temp_dc: usize,
|
||||
moisture_q: usize,
|
||||
vegetation: usize,
|
||||
glaciation: usize,
|
||||
}
|
||||
|
||||
fn encode_rle(
|
||||
canvas: &WireCanvas,
|
||||
) -> (
|
||||
Vec<u8>,
|
||||
std::time::Duration,
|
||||
std::time::Duration,
|
||||
RlePerFieldRuns,
|
||||
) {
|
||||
let t0 = Instant::now();
|
||||
let morphology = rle_encode_u8(&canvas.morphology);
|
||||
let elev_q = rle_encode_u8(&canvas.elev_q);
|
||||
let temp_dc = rle_encode_i16(&canvas.temp_dc);
|
||||
let moisture_q = rle_encode_u8(&canvas.moisture_q);
|
||||
let vegetation = rle_encode_u8(&canvas.vegetation);
|
||||
let glaciation = rle_encode_u8(&canvas.glaciation);
|
||||
let runs = RlePerFieldRuns {
|
||||
morphology: morphology.len(),
|
||||
elev_q: elev_q.len(),
|
||||
temp_dc: temp_dc.len(),
|
||||
moisture_q: moisture_q.len(),
|
||||
vegetation: vegetation.len(),
|
||||
glaciation: glaciation.len(),
|
||||
};
|
||||
let rle = RleCanvas {
|
||||
cols: canvas.cols,
|
||||
rows: canvas.rows,
|
||||
morphology,
|
||||
elev_q,
|
||||
temp_dc,
|
||||
moisture_q,
|
||||
vegetation,
|
||||
glaciation,
|
||||
};
|
||||
let bytes = rmp_serde::to_vec(&rle).expect("rmp_serde encode rle");
|
||||
let enc_time = t0.elapsed();
|
||||
|
||||
let t1 = Instant::now();
|
||||
let decoded: RleCanvas = rmp_serde::from_slice(&bytes).expect("rmp_serde decode rle");
|
||||
// Expand back to dense arrays (real decode cost — a consumer needs the
|
||||
// dense form to render).
|
||||
let mut morphology_dense = Vec::with_capacity(canvas.morphology.len());
|
||||
for (run, v) in &decoded.morphology {
|
||||
morphology_dense.extend(std::iter::repeat_n(*v, *run as usize));
|
||||
}
|
||||
let dec_time = t1.elapsed();
|
||||
std::hint::black_box(morphology_dense.len());
|
||||
(bytes, enc_time, dec_time, runs)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Encoding (d): PNG-encoded raster per field.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn png_encode_u8_plane(cols: u32, rows: u32, data: &[u8]) -> Vec<u8> {
|
||||
let mut out = Vec::new();
|
||||
{
|
||||
let mut enc = png::Encoder::new(&mut out, cols, rows);
|
||||
enc.set_color(png::ColorType::Grayscale);
|
||||
enc.set_depth(png::BitDepth::Eight);
|
||||
let mut writer = enc.write_header().expect("png header");
|
||||
writer.write_image_data(data).expect("png data");
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn png_decode_u8_plane(bytes: &[u8]) -> Vec<u8> {
|
||||
let mut decoder = png::Decoder::new(Cursor::new(bytes))
|
||||
.read_info()
|
||||
.expect("png read_info");
|
||||
let mut buf = vec![0u8; decoder.output_buffer_size()];
|
||||
let frame = decoder.next_frame(&mut buf).expect("png next_frame");
|
||||
buf[..frame.buffer_size()].to_vec()
|
||||
}
|
||||
|
||||
/// PNG the five u8 planes; temp_dc (i16, includes negative + sentinel values,
|
||||
/// not representable as an 8-bit grayscale plane without a lossy remap) ships
|
||||
/// via rmp_serde alongside, same as the bit-packed encoding's treatment.
|
||||
fn encode_png(canvas: &WireCanvas) -> (Vec<u8>, std::time::Duration, std::time::Duration) {
|
||||
let t0 = Instant::now();
|
||||
let morphology_png = png_encode_u8_plane(canvas.cols, canvas.rows, &canvas.morphology);
|
||||
let elev_q_png = png_encode_u8_plane(canvas.cols, canvas.rows, &canvas.elev_q);
|
||||
let moisture_q_png = png_encode_u8_plane(canvas.cols, canvas.rows, &canvas.moisture_q);
|
||||
let vegetation_png = png_encode_u8_plane(canvas.cols, canvas.rows, &canvas.vegetation);
|
||||
let glaciation_png = png_encode_u8_plane(canvas.cols, canvas.rows, &canvas.glaciation);
|
||||
let temp_dc_bytes = rmp_serde::to_vec(&canvas.temp_dc).expect("rmp_serde encode temp_dc");
|
||||
|
||||
let total = morphology_png.len()
|
||||
+ elev_q_png.len()
|
||||
+ moisture_q_png.len()
|
||||
+ vegetation_png.len()
|
||||
+ glaciation_png.len()
|
||||
+ temp_dc_bytes.len();
|
||||
let enc_time = t0.elapsed();
|
||||
|
||||
let t1 = Instant::now();
|
||||
let morphology_d = png_decode_u8_plane(&morphology_png);
|
||||
let elev_q_d = png_decode_u8_plane(&elev_q_png);
|
||||
let moisture_q_d = png_decode_u8_plane(&moisture_q_png);
|
||||
let vegetation_d = png_decode_u8_plane(&vegetation_png);
|
||||
let glaciation_d = png_decode_u8_plane(&glaciation_png);
|
||||
let temp_dc_d: Vec<i16> =
|
||||
rmp_serde::from_slice(&temp_dc_bytes).expect("rmp_serde decode temp_dc");
|
||||
let dec_time = t1.elapsed();
|
||||
std::hint::black_box((
|
||||
morphology_d.len(),
|
||||
elev_q_d.len(),
|
||||
moisture_q_d.len(),
|
||||
vegetation_d.len(),
|
||||
glaciation_d.len(),
|
||||
temp_dc_d.len(),
|
||||
));
|
||||
|
||||
// Return a synthetic combined buffer sized to `total` (not a real single
|
||||
// envelope — the workshop's wire contract question is exactly whether
|
||||
// these become five separate frames in a tagged envelope) so callers can
|
||||
// report a single byte count. Filled with zero bytes; only `.len()` is
|
||||
// used by the reporting harness below.
|
||||
(vec![0u8; total], enc_time, dec_time)
|
||||
}
|
||||
|
||||
/// PNG applied to the bit-packed byte planes (b+d combined) — cheap to try
|
||||
/// since both encodings already exist above.
|
||||
fn encode_png_of_bitpacked(
|
||||
canvas: &WireCanvas,
|
||||
) -> (Vec<u8>, std::time::Duration, std::time::Duration) {
|
||||
let t0 = Instant::now();
|
||||
let morphology_bits = pack_bits(&canvas.morphology, 5);
|
||||
let elev_q_bits = pack_bits(&canvas.elev_q, 7);
|
||||
let moisture_q_bits = pack_bits(&canvas.moisture_q, 7);
|
||||
let vegetation_bits = pack_bits(&canvas.vegetation, 3);
|
||||
let glaciation_bits = pack_bits(&canvas.glaciation, 3);
|
||||
|
||||
// PNG needs a rectangular raster; the packed byte streams aren't
|
||||
// canvas-shaped, so wrap each as a 1-row grayscale "image" of its own
|
||||
// byte length — this measures DEFLATE-over-packed-bytes cost/ratio
|
||||
// honestly (PNG's filter step is a no-op on a 1-row image, so this
|
||||
// isolates the DEFLATE contribution cleanly).
|
||||
let png_plane = |bits: &[u8]| -> Vec<u8> {
|
||||
let mut out = Vec::new();
|
||||
let mut enc = png::Encoder::new(&mut out, bits.len() as u32, 1);
|
||||
enc.set_color(png::ColorType::Grayscale);
|
||||
enc.set_depth(png::BitDepth::Eight);
|
||||
let mut writer = enc.write_header().expect("png header");
|
||||
writer.write_image_data(bits).expect("png data");
|
||||
drop(writer);
|
||||
out
|
||||
};
|
||||
let morphology_png = png_plane(&morphology_bits);
|
||||
let elev_q_png = png_plane(&elev_q_bits);
|
||||
let moisture_q_png = png_plane(&moisture_q_bits);
|
||||
let vegetation_png = png_plane(&vegetation_bits);
|
||||
let glaciation_png = png_plane(&glaciation_bits);
|
||||
let temp_dc_bytes = rmp_serde::to_vec(&canvas.temp_dc).expect("rmp_serde encode temp_dc");
|
||||
|
||||
let total = morphology_png.len()
|
||||
+ elev_q_png.len()
|
||||
+ moisture_q_png.len()
|
||||
+ vegetation_png.len()
|
||||
+ glaciation_png.len()
|
||||
+ temp_dc_bytes.len();
|
||||
let enc_time = t0.elapsed();
|
||||
|
||||
let t1 = Instant::now();
|
||||
let n = canvas.morphology.len();
|
||||
let morphology_d = unpack_bits(&png_decode_u8_plane(&morphology_png), 5, n);
|
||||
let elev_q_d = unpack_bits(&png_decode_u8_plane(&elev_q_png), 7, n);
|
||||
let moisture_q_d = unpack_bits(&png_decode_u8_plane(&moisture_q_png), 7, n);
|
||||
let vegetation_d = unpack_bits(&png_decode_u8_plane(&vegetation_png), 3, n);
|
||||
let glaciation_d = unpack_bits(&png_decode_u8_plane(&glaciation_png), 3, n);
|
||||
let temp_dc_d: Vec<i16> =
|
||||
rmp_serde::from_slice(&temp_dc_bytes).expect("rmp_serde decode temp_dc");
|
||||
let dec_time = t1.elapsed();
|
||||
std::hint::black_box((
|
||||
morphology_d.len(),
|
||||
elev_q_d.len(),
|
||||
moisture_q_d.len(),
|
||||
vegetation_d.len(),
|
||||
glaciation_d.len(),
|
||||
temp_dc_d.len(),
|
||||
));
|
||||
|
||||
(vec![0u8; total], enc_time, dec_time)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Reporting
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn report_row(label: &str, bytes: usize, raw_bytes: usize, enc_ms: f64, dec_ms: f64) {
|
||||
let ratio = bytes as f64 / raw_bytes as f64;
|
||||
let wire_cap_multiple = bytes as f64 / 30_000.0; // ~30 KB context row
|
||||
println!(
|
||||
" {label:<28} {bytes:>10} bytes {ratio:>6.3}x raw {wire_cap_multiple:>8.1}x (30KB cap) enc {enc_ms:>7.2} ms dec {dec_ms:>7.2} ms"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn wire_size_table_330k() {
|
||||
let (params, ta, seed) = load_gj338bd();
|
||||
run_canvas_report(¶ms, &ta, seed, 768, 432, "330K (768x432)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn wire_size_table_2_07m() {
|
||||
let (params, ta, seed) = load_gj338bd();
|
||||
run_canvas_report(¶ms, &ta, seed, 1920, 1080, "2.07M (1920x1080)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn wire_size_table_8_3m() {
|
||||
let (params, ta, seed) = load_gj338bd();
|
||||
run_canvas_report(¶ms, &ta, seed, 3840, 2160, "8.3M (3840x2160)");
|
||||
}
|
||||
|
||||
fn run_canvas_report(
|
||||
params: &BodyParams,
|
||||
ta: &TerrainAnalysis,
|
||||
seed: SeedChain,
|
||||
cols: u32,
|
||||
rows: u32,
|
||||
label: &str,
|
||||
) {
|
||||
println!(
|
||||
"\n=== T-1179 wire-size table: {label} = {} gridunits ===",
|
||||
cols as u64 * rows as u64
|
||||
);
|
||||
|
||||
let (canvas, derive_time) = derive_canvas(seed, params, ta, cols, rows);
|
||||
println!(
|
||||
" derive: {:.2} ms ({} cells, {} Rayon threads available)\n",
|
||||
derive_time.as_secs_f64() * 1000.0,
|
||||
canvas.morphology.len(),
|
||||
std::thread::available_parallelism()
|
||||
.map(|n| n.get())
|
||||
.unwrap_or(0)
|
||||
);
|
||||
|
||||
let (rmp_bytes, rmp_enc, rmp_dec) = encode_rmp(&canvas);
|
||||
let raw_bytes = rmp_bytes.len();
|
||||
report_row(
|
||||
"(a) raw dense rmp_serde",
|
||||
raw_bytes,
|
||||
raw_bytes,
|
||||
rmp_enc.as_secs_f64() * 1000.0,
|
||||
rmp_dec.as_secs_f64() * 1000.0,
|
||||
);
|
||||
|
||||
let (bp_bytes, bp_enc, bp_dec) = encode_bitpacked(&canvas);
|
||||
report_row(
|
||||
"(b) bit-packed",
|
||||
bp_bytes.len(),
|
||||
raw_bytes,
|
||||
bp_enc.as_secs_f64() * 1000.0,
|
||||
bp_dec.as_secs_f64() * 1000.0,
|
||||
);
|
||||
|
||||
let (rle_bytes, rle_enc, rle_dec, runs) = encode_rle(&canvas);
|
||||
report_row(
|
||||
"(c) per-field RLE",
|
||||
rle_bytes.len(),
|
||||
raw_bytes,
|
||||
rle_enc.as_secs_f64() * 1000.0,
|
||||
rle_dec.as_secs_f64() * 1000.0,
|
||||
);
|
||||
|
||||
let (png_bytes, png_enc, png_dec) = encode_png(&canvas);
|
||||
report_row(
|
||||
"(d) PNG per field",
|
||||
png_bytes.len(),
|
||||
raw_bytes,
|
||||
png_enc.as_secs_f64() * 1000.0,
|
||||
png_dec.as_secs_f64() * 1000.0,
|
||||
);
|
||||
|
||||
let (pngbp_bytes, pngbp_enc, pngbp_dec) = encode_png_of_bitpacked(&canvas);
|
||||
report_row(
|
||||
"(e) PNG-of-bit-packed",
|
||||
pngbp_bytes.len(),
|
||||
raw_bytes,
|
||||
pngbp_enc.as_secs_f64() * 1000.0,
|
||||
pngbp_dec.as_secs_f64() * 1000.0,
|
||||
);
|
||||
|
||||
let n = canvas.morphology.len();
|
||||
println!("\n per-field RLE run counts (lower = more compressible; n={n} cells):");
|
||||
println!(
|
||||
" morphology: {:>8} runs ({:.1}% of dense)",
|
||||
runs.morphology,
|
||||
100.0 * runs.morphology as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
" elev_q: {:>8} runs ({:.1}% of dense)",
|
||||
runs.elev_q,
|
||||
100.0 * runs.elev_q as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
" temp_dc: {:>8} runs ({:.1}% of dense)",
|
||||
runs.temp_dc,
|
||||
100.0 * runs.temp_dc as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
" moisture_q: {:>8} runs ({:.1}% of dense)",
|
||||
runs.moisture_q,
|
||||
100.0 * runs.moisture_q as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
" vegetation: {:>8} runs ({:.1}% of dense)",
|
||||
runs.vegetation,
|
||||
100.0 * runs.vegetation as f64 / n as f64
|
||||
);
|
||||
println!(
|
||||
" glaciation: {:>8} runs ({:.1}% of dense)",
|
||||
runs.glaciation,
|
||||
100.0 * runs.glaciation as f64 / n as f64
|
||||
);
|
||||
println!();
|
||||
}
|
||||
Reference in New Issue
Block a user