Files
settled-reach/docs/workshops/body-map-viewer/stig-round2.md
T
jpmschweitzerandClaude Fable 5 29c22cb728 docs(meta): body-map-viewer workshop — rounds, measurements, outcomes, as-built briefing
The complete workshop record: four round-1 positions, five round-2 syntheses
(incl. Troblum's adversarial pass with addendum + final scorecard — all seven
findings resolved), both lead interviews, Qatux's round notes and the 8-section
workshop-outcomes.md (the lakes message-crossing documented as process
history), measurement ⑥ (set_pixel/c1) + the population-survey and chunk/S2
addenda in the measurement docs, the brief's appendix updated through ⑥, and
architecture-briefing-final.md — Jeroen's outline written back as-built
(six-level ladder, lakes, ~9MB resident global tier). README row: Complete.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 10:57:45 +02:00

301 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Body Map Viewer — Stig's Round 2 Position"
description: "Measurement ⑥ (CPU-colorize cost), the final c1 shader-vs-CPU call, the cache-store spec with Jeroen's storage-eviction amendment, and the px-per-gridunit band recommendation"
type: workshop
status: active
workshop: body-map-viewer
created: 2026-07-25
owner: Stig
---
# Stig — Round 2 Position
Round 1 (`stig-round1.md`) flagged one open item: the c1 shader-vs-CPU call
was made CPU-first but explicitly **pending** the missing `Image.set_pixel`
cost number. Jeroen ruled at lead interview 1 to run it now — done, below,
plus the confirmed cache-store spec incorporating his new storage-eviction
amendment, and the px-per-gridunit band Dudley/Tyre need for the ladder
tables.
## (a) Measurement ⑥ — headline
Full doc: `measurements/t-setpixel-c1.md`. Appendix row added to the brief.
| size | `set_pixel` median | ns/cell | byte-buffer median | ns/cell |
|---|---:|---:|---:|---:|
| 330K | 25.7 ms | 77.5 | 51.0 ms | 153.8 |
| 2.07M | 165.2 ms | 79.7 | 319.2 ms | 154.0 |
| 8.3M | 643.5 ms | 77.6 | 1,261.4 ms | 152.1 |
Flat per-cell rate across the full 25× size range for both paths (no
cliff — same pattern every prior measurement in this appendix found).
Headless-valid: this is pure CPU `Image`/`PackedByteArray` manipulation, no
RenderingServer call in the path, unlike ⑤ which needed a real display to
avoid the headless dummy renderer's faked GPU uploads.
**Two findings, one expected, one not:**
1. Expected: cost is real and scales linearly, not free the way upload (⑤)
turned out to be.
2. Not expected: **`Image.set_pixel` beats a hand-rolled `PackedByteArray`
direct write by ~2×**, at every size. I went in assuming the opposite —
skip the per-pixel method call, skip `Color` object construction, write
raw bytes. Measured, GDScript's own per-element indexed `PackedByteArray`
write (four separate indexed writes per cell in the buffer path) costs
more than `set_pixel`'s single per-pixel call. This is worth stating
plainly for whoever implements the terrain layer: **don't hand-roll the
buffer write as a "faster" alternative in GDScript** — the intuition
that's usually right in a compiled language doesn't transfer here.
## (b) C1 final call — CONFIRMED, CPU-first, with an explicit sizing caveat
Round 1 said CPU-first-pending-the-number. The number is in. **I'm
confirming CPU coloring (`Image.set_pixel`) as the shipped default for the
terrain layer, not flipping to shaders — but the number changes *how* I'd
frame the recommendation, and it sharpens rather than weakens the case for
Dudley's viewport-sized-canvas policy.**
**Why confirm, not flip:**
- **At the sizes the ladder will actually request, colorize is cheap.**
Dudley's round-1 position (§2, "Question 2") is explicit that
viewport-sized canvases at a fixed pixel budget — not the 8.3M-cell figure
— are the real per-step request shape; 8.3M exists in the appendix purely
as the stress-ceiling case answering "does this degrade at scale" (no).
At 330K cells, the size that shape actually produces, `set_pixel` colorize
costs **25.7 ms** — well inside "the player is waiting for a step-cross to
resolve" tolerance, and it's happening once per arrived canvas inside the
hold-fetch-swap sequence (§(c) below refines that sequence's cost
ordering), not per frame.
- **Nothing in ①–⑤ forced shaders architecturally, and ⑥ doesn't either —
it just prices the CPU path honestly instead of leaving it an
unmeasured assumption.** The original round-1 reasoning stands: today's
code already does CPU coloring successfully (`_build_tile_texture`/
`_rebuild_texture_if_needed`, five toggle overlays shipped this way), and
the biggest structural change this workshop makes to the client (RTT as
universal terrain path, killing `_canvas.scale`) is large enough on its
own without also swapping the coloring mechanism in the same pass without
a forcing reason.
- **Where the number DOES change my framing:** round 1 called the gap
"a real gap" but didn't know if it would come back trivial or
meaningful. It came back **meaningful at the stress-ceiling size** (643 ms
at 8.3M is roughly a third of server derive time at that size — not a
rounding error) **and trivial at the realistic per-step size** (25.7 ms at
330K). That's not a wash — it's a data point that argues *for* Dudley's
viewport-sizing policy being load-bearing on the client side too, not just
the D-226(d) governance reason he named. If some future caller ever
requested a literal 8.3M-cell canvas as a real step payload (not the
stress-test shape), CPU colorize alone would eat ~640 ms of the arrival
budget — a real, nameable cost that the viewport-sized-canvas policy
structurally avoids by never generating that request in the first place.
I'd rather land this as "the policy is validated from a second angle" than
as a new risk, since Dudley already ruled viewport-sized canvases as the
policy on governance grounds independent of this number.
- **The shader path is not closed, it's sequenced.** Same as round 1: the
natural trigger for shader-side terrain compositing is T-1175's per-vertex
river tapering work (needs shader/Polygon2D-strip geometry regardless of
what the terrain raster does) landing and, separately, if a future step
count/canvas-size decision in round 2's ladder synthesis pushes the
*realistic* per-step canvas size meaningfully above 330K–2.07M (not the
8.3M stress case, an actual steady-state request shape), that's the
trigger to revisit — cite this measurement, don't re-guess.
**Net: CPU coloring ships. File a note on the terrain-layer implementation
ticket: use `Image.set_pixel`, not a hand-rolled buffer write (⑥'s 2×
finding), and treat "does the real step-count/size decision ever make 330K
the *small* end rather than the steady case" as the trigger to reopen c1,
not a speculative future pass.**
## (c) Cache-store spec — final, incorporating Jeroen's storage-eviction amendment
Lead interview 1 ruling 2 (verbatim, captured in `lead-interview-1.md`):
*"we still may also want to evict non global level geometry based on time to
save storage for planets the player visits but never goes back to"* →
**staleness-eviction and storage-eviction are distinct axes.** Geometry never
goes stale (D-227 determinism — re-derivable, byte-identical forever), but
sub-global geometry still gets evicted on **time-since-last-visit** as a
storage-budget policy. The global tier alone is keep-always.
This extends my round-1 two-tier design (retention-floor+LRU for geometry,
TTL for sim-state) with a **third eviction mechanism** for sub-global
geometry specifically — it was previously LRU-evict-only (matching
`atlas_window_tile_set.gd`'s existing behavior); it now also gets a
time-since-last-visit sweep. Three mechanisms, three different problems,
kept structurally separate rather than folded into one formula — this
mirrors Dudley's own multiplicative-TTL design for the server tier
(`BASE_TTL[rung] × time_decay × distance_decay`), except my two extra axes
answer different questions (capacity pressure vs. storage-budget thrift) so
I'm keeping them as two independent sweep passes rather than composing them
into one number, for the same legibility reason Dudley cites for keeping his
formula to one tunable per rung.
### Ratified from round 1, unchanged
- Shape: plain Godot `FileAccess` cache dir + index — no `godot-sqlite`
addon, no second SQLite file. Confirmed by lead interview 1: *"both
rejected SQLite in every shape independently"* (Dudley's server-side
answer converged on the same rejection from a different angle: a plain
D-203-shaped resource, not SQLite, for the global tier). **Composition,
not competition**, per the ruling: Dudley's server-side global tier
(~174 MB PNG-encoded, D-203-shaped) and my client-side `FileAccess` dir
are two tiers of the same cache stack, not alternatives to each other.
- Wire content = the already-decided wire encoding (PNG-per-field, T-1179's
winner) written to disk as-is — no re-encoding for storage.
- In-memory tier (the `atlas_window_tile_set.gd`-style Dictionary LRU)
remains the hot path in front of the disk tier, unchanged.
### Three-tier eviction — the concrete spec
**Tier 1 — Global/step-0 (region-spaced) geometry: retention floor, no
sweep at all.** Never touched by either the LRU-capacity sweep or the
time-since-visit sweep. The only way an entry leaves is an explicit
clear/uninstall action. This is the literal mechanism for "always keep the
global level." Matches Dudley's server-side answer (region-spaced, ~638 KB
PNG-encoded/body, ~174 MB across all ~273 bodies server-side) — the client
disk tier for this rung should be sized the same way, since a client that
has actually visited a body already has the same canvas the server cached,
and re-fetching it from the local warm server subprocess after a client-side
eviction would be needlessly wasteful when disk is this cheap.
**Tier 2 — Sub-global geometry (every step below global): two independent
sweep passes, both storage-motivated, distinct from staleness.**
- **(2a) Time-since-last-visit sweep** (Jeroen's new amendment) — a
periodic pass (triggered on body-open + a coarse background timer, never
per-frame) that walks the index and deletes any sub-global entry whose
`last_read_at` is older than a configurable threshold (proposed starting
point: on the order of days-to-weeks of real wall-clock time, tunable —
not a round-2 architecture call, a tuning pass once this ships). This
answers *"the player visited this body once, three sessions ago, and
hasn't been back"* — the entries are still byte-valid (D-227 determinism
means they're never wrong), they're just not worth the disk space for a
body the player has functionally abandoned.
- **(2b) LRU-capacity sweep** (unchanged from round 1) — if total disk usage
for the sub-global tier exceeds a configured budget, evict oldest-touched
entries first, same erase+reinsert-on-touch mechanism
`atlas_window_tile_set.gd` already implements, just backed by the index
file instead of an in-process Dictionary of raw window data.
- These are **two separate triggers checking two separate conditions**
(age-since-visit vs. total-bytes-over-budget), not one merged policy —
keeping them apart means either one can fire independently (a player who
visits many bodies briefly hits 2b before 2a; a player who stays on one
body for a long single session but never returns to old ones hits 2a
before 2b), and each is independently legible/tunable.
**Tier 3 — Sim-state-tagged planes (frozen/flooded, whatever fields the map
time axis ruling lands on — ruled at lead interview 1, ruling 2, as
"current state via TTL-split"): explicit TTL, staleness-motivated,
structurally separate from tiers 1/2.** These entries carry a real
expiry — re-requested as sim time advances, per Jeroen's original hint made
concrete by the ruling. Distinct index field, distinct sweep condition
(`now > written_at + ttl`), never touched by the LRU-capacity or
time-since-visit sweeps (a sim-state entry doesn't get to live longer just
because disk space is available — it goes stale on its own schedule
regardless of capacity pressure).
### Index file schema (concrete)
One manifest per body, `user://atlas_cache/<body_id>/index.dat` (a small
binary or JSON — binary preferred for parse cost at scale, but this is an
implementation-detail choice, not an architecture one), loaded once on
body-open and held in memory as a `Dictionary` for the session — matching
the same "erase+reinsert = move-to-MRU" idiom `atlas_window_tile_set.gd`
already uses, just now also carrying the two extra timestamp/tag fields
tiers 2 and 3 need:
```
IndexEntry {
key: String # same composite key shape as atlas_window_tile_set.gd's
# make_key(), extended with step index:
# "<body_id>:<step>:<center>:<n>:<granularity_v2>"
file_path: String # user://atlas_cache/<body_id>/<key_hash>.png (or
# one file per field, per the wire contract's own
# per-field-PNG framing — Araminta's call, mirrored
# here, not re-decided)
tier: Geometry | SimState
written_at: int # unix time, set once, never updated
last_read_at: int # unix time, updated on every cache hit (drives 2a/2b)
size_bytes: int # drives 2b's budget accounting without a stat() call
sim_ttl: int? # only present when tier == SimState; null/absent
# for Geometry entries (2a/2b apply, 3 never does)
retention_floor: bool # true only for tier-1 (global/step-0) entries;
# short-circuits both sweep passes unconditionally
}
```
Directory layout: one PNG-per-field file per cache entry (mirrors the wire
encoding exactly — no format translation between "on the wire" and "on
disk"), named by a hash of `key` to avoid filesystem-unsafe characters
(the `:`/`,` composite key string itself isn't a safe filename on every
target platform); the index maps the human-legible key to that hash.
### Sweep triggers
- **On body-open:** load the index; run 2a (time-since-visit) immediately —
cheap (a metadata scan, no file I/O beyond the index itself) and this is
the natural moment ("returning to a body") where stale-by-absence entries
are most likely to exist and least likely to be needed again in the next
few seconds.
- **On a coarse background timer** (not per-frame, not even per-step-cross —
proposed on the order of minutes, tunable): run 2b (LRU-capacity) if
total sub-global bytes exceed budget, and sweep tier 3 for expired
sim-state entries. Both are backgroundable (Rayon-queue-adjacent on the
server side; on the client this is a low-priority deferred call, never
blocking a frame or a step-cross — matches premise 1's "smart precache
allowed, never blocking user output" applied to eviction as much as to
fetch).
- **Never per-frame.** All three sweep mechanisms are explicitly excluded
from the render/input loop — this is bookkeeping, not gameplay-adjacent
work, and belongs nowhere near the 16.6 ms budget ⑤ and ⑥ both care about.
### Per-tier budgets
- **Tier 1 (global):** sized to match Dudley's server-side number directly
— ~638 KB/body PNG-encoded, ~174 MB across all ~273 bodies if the client
has visited every body (an upper bound, not a typical player's actual
footprint). No cap needed beyond that — it's the "keep always" tier by
definition, and 174 MB is a trivial disk allocation on any target
platform.
- **Tier 2 (sub-global):** a configurable byte budget (proposed starting
point: same order of magnitude as tier 1, e.g. a few hundred MB to
low-GB range — a tuning-pass number once real play-pattern data exists,
not an architecture call I'm locking here) that 2b enforces directly, with
2a doing most of the practical work of keeping it under budget without
ever hitting the hard cap in normal play (a player who keeps returning to
the same few bodies never triggers 2a for those bodies' entries; a player
who body-hops constantly generates entries 2a will clear out on its own
schedule before 2b's cap becomes the active constraint).
- **Tier 3 (sim-state):** no byte budget needed — TTL alone bounds it, and
these planes are a small fraction of a step canvas's total field set (per
Araminta's round-1 schema, frozen/flooded reads off the existing
`glaciation`/morphology-water-class fields rather than adding new dense
arrays, so tier 3's actual footprint is small regardless).
### What stays true from D-227 regardless of which tier
Every tier is still an evictable cache, never a source of truth — deleting
the entire `user://atlas_cache/` directory at any time, for any reason
(corruption, manual clear, platform storage pressure), changes client
behavior only by causing re-fetches. No tier, no sweep policy, no budget
number in this spec is allowed to become load-bearing for correctness. This
is the same discipline Dudley's server-side tier commits to explicitly in
his round-1 doc, restated here for the client side as it was in round 1.
## (d) Px-per-gridunit band — sent to Dudley + Tyre
See the accompanying `SendMessage`. Summary of the recommendation for
their ladder tables: **1×1 px/gridunit at the deepest step (Tile,
D-243-pinned per Tyre's snap ruling), relaxing toward the ~5×5 fallback only
at the shallower/coarser steps where canvas extent (not spacing) is what's
growing.** Reasoning grounded in ⑥ + ⑤ together: crisper display costs
nothing extra in upload (⑤ showed upload is flat-cheap regardless of pixel
density, it's a function of byte count not "how many real gridunits per
displayed pixel") and colorize cost (⑥) is driven by **canvas cell count**,
which is already bounded by Dudley's viewport-sizing policy independent of
the display ratio — so there's no cost reason to fall back to 5×5 at the
steps where fidelity matters most (the deep, ground-level steps where the
player is closest to visible detail). The 5×5 fallback earns its keep at the
*shallow* steps instead, where canvas extent needs to cover a large ground
area and cell count would otherwise balloon for comparatively little
perceptual gain (players aren't scrutinizing per-metre detail from orbital
view). Full band recommendation in the message.