The complete workshop record: four round-1 positions, five round-2 syntheses (incl. Troblum's adversarial pass with addendum + final scorecard — all seven findings resolved), both lead interviews, Qatux's round notes and the 8-section workshop-outcomes.md (the lakes message-crossing documented as process history), measurement ⑥ (set_pixel/c1) + the population-survey and chunk/S2 addenda in the measurement docs, the brief's appendix updated through ⑥, and architecture-briefing-final.md — Jeroen's outline written back as-built (six-level ladder, lakes, ~9MB resident global tier). README row: Complete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
301 lines
17 KiB
Markdown
301 lines
17 KiB
Markdown
---
|
||
title: "Body Map Viewer — Stig's Round 2 Position"
|
||
description: "Measurement ⑥ (CPU-colorize cost), the final c1 shader-vs-CPU call, the cache-store spec with Jeroen's storage-eviction amendment, and the px-per-gridunit band recommendation"
|
||
type: workshop
|
||
status: active
|
||
workshop: body-map-viewer
|
||
created: 2026-07-25
|
||
owner: Stig
|
||
---
|
||
|
||
# Stig — Round 2 Position
|
||
|
||
Round 1 (`stig-round1.md`) flagged one open item: the c1 shader-vs-CPU call
|
||
was made CPU-first but explicitly **pending** the missing `Image.set_pixel`
|
||
cost number. Jeroen ruled at lead interview 1 to run it now — done, below,
|
||
plus the confirmed cache-store spec incorporating his new storage-eviction
|
||
amendment, and the px-per-gridunit band Dudley/Tyre need for the ladder
|
||
tables.
|
||
|
||
## (a) Measurement ⑥ — headline
|
||
|
||
Full doc: `measurements/t-setpixel-c1.md`. Appendix row added to the brief.
|
||
|
||
| size | `set_pixel` median | ns/cell | byte-buffer median | ns/cell |
|
||
|---|---:|---:|---:|---:|
|
||
| 330K | 25.7 ms | 77.5 | 51.0 ms | 153.8 |
|
||
| 2.07M | 165.2 ms | 79.7 | 319.2 ms | 154.0 |
|
||
| 8.3M | 643.5 ms | 77.6 | 1,261.4 ms | 152.1 |
|
||
|
||
Flat per-cell rate across the full 25× size range for both paths (no
|
||
cliff — same pattern every prior measurement in this appendix found).
|
||
Headless-valid: this is pure CPU `Image`/`PackedByteArray` manipulation, no
|
||
RenderingServer call in the path, unlike ⑤ which needed a real display to
|
||
avoid the headless dummy renderer's faked GPU uploads.
|
||
|
||
**Two findings, one expected, one not:**
|
||
1. Expected: cost is real and scales linearly, not free the way upload (⑤)
|
||
turned out to be.
|
||
2. Not expected: **`Image.set_pixel` beats a hand-rolled `PackedByteArray`
|
||
direct write by ~2×**, at every size. I went in assuming the opposite —
|
||
skip the per-pixel method call, skip `Color` object construction, write
|
||
raw bytes. Measured, GDScript's own per-element indexed `PackedByteArray`
|
||
write (four separate indexed writes per cell in the buffer path) costs
|
||
more than `set_pixel`'s single per-pixel call. This is worth stating
|
||
plainly for whoever implements the terrain layer: **don't hand-roll the
|
||
buffer write as a "faster" alternative in GDScript** — the intuition
|
||
that's usually right in a compiled language doesn't transfer here.
|
||
|
||
## (b) C1 final call — CONFIRMED, CPU-first, with an explicit sizing caveat
|
||
|
||
Round 1 said CPU-first-pending-the-number. The number is in. **I'm
|
||
confirming CPU coloring (`Image.set_pixel`) as the shipped default for the
|
||
terrain layer, not flipping to shaders — but the number changes *how* I'd
|
||
frame the recommendation, and it sharpens rather than weakens the case for
|
||
Dudley's viewport-sized-canvas policy.**
|
||
|
||
**Why confirm, not flip:**
|
||
|
||
- **At the sizes the ladder will actually request, colorize is cheap.**
|
||
Dudley's round-1 position (§2, "Question 2") is explicit that
|
||
viewport-sized canvases at a fixed pixel budget — not the 8.3M-cell figure
|
||
— are the real per-step request shape; 8.3M exists in the appendix purely
|
||
as the stress-ceiling case answering "does this degrade at scale" (no).
|
||
At 330K cells, the size that shape actually produces, `set_pixel` colorize
|
||
costs **25.7 ms** — well inside "the player is waiting for a step-cross to
|
||
resolve" tolerance, and it's happening once per arrived canvas inside the
|
||
hold-fetch-swap sequence (§(c) below refines that sequence's cost
|
||
ordering), not per frame.
|
||
- **Nothing in ①–⑤ forced shaders architecturally, and ⑥ doesn't either —
|
||
it just prices the CPU path honestly instead of leaving it an
|
||
unmeasured assumption.** The original round-1 reasoning stands: today's
|
||
code already does CPU coloring successfully (`_build_tile_texture`/
|
||
`_rebuild_texture_if_needed`, five toggle overlays shipped this way), and
|
||
the biggest structural change this workshop makes to the client (RTT as
|
||
universal terrain path, killing `_canvas.scale`) is large enough on its
|
||
own without also swapping the coloring mechanism in the same pass without
|
||
a forcing reason.
|
||
- **Where the number DOES change my framing:** round 1 called the gap
|
||
"a real gap" but didn't know if it would come back trivial or
|
||
meaningful. It came back **meaningful at the stress-ceiling size** (643 ms
|
||
at 8.3M is roughly a third of server derive time at that size — not a
|
||
rounding error) **and trivial at the realistic per-step size** (25.7 ms at
|
||
330K). That's not a wash — it's a data point that argues *for* Dudley's
|
||
viewport-sizing policy being load-bearing on the client side too, not just
|
||
the D-226(d) governance reason he named. If some future caller ever
|
||
requested a literal 8.3M-cell canvas as a real step payload (not the
|
||
stress-test shape), CPU colorize alone would eat ~640 ms of the arrival
|
||
budget — a real, nameable cost that the viewport-sized-canvas policy
|
||
structurally avoids by never generating that request in the first place.
|
||
I'd rather land this as "the policy is validated from a second angle" than
|
||
as a new risk, since Dudley already ruled viewport-sized canvases as the
|
||
policy on governance grounds independent of this number.
|
||
- **The shader path is not closed, it's sequenced.** Same as round 1: the
|
||
natural trigger for shader-side terrain compositing is T-1175's per-vertex
|
||
river tapering work (needs shader/Polygon2D-strip geometry regardless of
|
||
what the terrain raster does) landing and, separately, if a future step
|
||
count/canvas-size decision in round 2's ladder synthesis pushes the
|
||
*realistic* per-step canvas size meaningfully above 330K–2.07M (not the
|
||
8.3M stress case, an actual steady-state request shape), that's the
|
||
trigger to revisit — cite this measurement, don't re-guess.
|
||
|
||
**Net: CPU coloring ships. File a note on the terrain-layer implementation
|
||
ticket: use `Image.set_pixel`, not a hand-rolled buffer write (⑥'s 2×
|
||
finding), and treat "does the real step-count/size decision ever make 330K
|
||
the *small* end rather than the steady case" as the trigger to reopen c1,
|
||
not a speculative future pass.**
|
||
|
||
## (c) Cache-store spec — final, incorporating Jeroen's storage-eviction amendment
|
||
|
||
Lead interview 1 ruling 2 (verbatim, captured in `lead-interview-1.md`):
|
||
*"we still may also want to evict non global level geometry based on time to
|
||
save storage for planets the player visits but never goes back to"* →
|
||
**staleness-eviction and storage-eviction are distinct axes.** Geometry never
|
||
goes stale (D-227 determinism — re-derivable, byte-identical forever), but
|
||
sub-global geometry still gets evicted on **time-since-last-visit** as a
|
||
storage-budget policy. The global tier alone is keep-always.
|
||
|
||
This extends my round-1 two-tier design (retention-floor+LRU for geometry,
|
||
TTL for sim-state) with a **third eviction mechanism** for sub-global
|
||
geometry specifically — it was previously LRU-evict-only (matching
|
||
`atlas_window_tile_set.gd`'s existing behavior); it now also gets a
|
||
time-since-last-visit sweep. Three mechanisms, three different problems,
|
||
kept structurally separate rather than folded into one formula — this
|
||
mirrors Dudley's own multiplicative-TTL design for the server tier
|
||
(`BASE_TTL[rung] × time_decay × distance_decay`), except my two extra axes
|
||
answer different questions (capacity pressure vs. storage-budget thrift) so
|
||
I'm keeping them as two independent sweep passes rather than composing them
|
||
into one number, for the same legibility reason Dudley cites for keeping his
|
||
formula to one tunable per rung.
|
||
|
||
### Ratified from round 1, unchanged
|
||
|
||
- Shape: plain Godot `FileAccess` cache dir + index — no `godot-sqlite`
|
||
addon, no second SQLite file. Confirmed by lead interview 1: *"both
|
||
rejected SQLite in every shape independently"* (Dudley's server-side
|
||
answer converged on the same rejection from a different angle: a plain
|
||
D-203-shaped resource, not SQLite, for the global tier). **Composition,
|
||
not competition**, per the ruling: Dudley's server-side global tier
|
||
(~174 MB PNG-encoded, D-203-shaped) and my client-side `FileAccess` dir
|
||
are two tiers of the same cache stack, not alternatives to each other.
|
||
- Wire content = the already-decided wire encoding (PNG-per-field, T-1179's
|
||
winner) written to disk as-is — no re-encoding for storage.
|
||
- In-memory tier (the `atlas_window_tile_set.gd`-style Dictionary LRU)
|
||
remains the hot path in front of the disk tier, unchanged.
|
||
|
||
### Three-tier eviction — the concrete spec
|
||
|
||
**Tier 1 — Global/step-0 (region-spaced) geometry: retention floor, no
|
||
sweep at all.** Never touched by either the LRU-capacity sweep or the
|
||
time-since-visit sweep. The only way an entry leaves is an explicit
|
||
clear/uninstall action. This is the literal mechanism for "always keep the
|
||
global level." Matches Dudley's server-side answer (region-spaced, ~638 KB
|
||
PNG-encoded/body, ~174 MB across all ~273 bodies server-side) — the client
|
||
disk tier for this rung should be sized the same way, since a client that
|
||
has actually visited a body already has the same canvas the server cached,
|
||
and re-fetching it from the local warm server subprocess after a client-side
|
||
eviction would be needlessly wasteful when disk is this cheap.
|
||
|
||
**Tier 2 — Sub-global geometry (every step below global): two independent
|
||
sweep passes, both storage-motivated, distinct from staleness.**
|
||
|
||
- **(2a) Time-since-last-visit sweep** (Jeroen's new amendment) — a
|
||
periodic pass (triggered on body-open + a coarse background timer, never
|
||
per-frame) that walks the index and deletes any sub-global entry whose
|
||
`last_read_at` is older than a configurable threshold (proposed starting
|
||
point: on the order of days-to-weeks of real wall-clock time, tunable —
|
||
not a round-2 architecture call, a tuning pass once this ships). This
|
||
answers *"the player visited this body once, three sessions ago, and
|
||
hasn't been back"* — the entries are still byte-valid (D-227 determinism
|
||
means they're never wrong), they're just not worth the disk space for a
|
||
body the player has functionally abandoned.
|
||
- **(2b) LRU-capacity sweep** (unchanged from round 1) — if total disk usage
|
||
for the sub-global tier exceeds a configured budget, evict oldest-touched
|
||
entries first, same erase+reinsert-on-touch mechanism
|
||
`atlas_window_tile_set.gd` already implements, just backed by the index
|
||
file instead of an in-process Dictionary of raw window data.
|
||
- These are **two separate triggers checking two separate conditions**
|
||
(age-since-visit vs. total-bytes-over-budget), not one merged policy —
|
||
keeping them apart means either one can fire independently (a player who
|
||
visits many bodies briefly hits 2b before 2a; a player who stays on one
|
||
body for a long single session but never returns to old ones hits 2a
|
||
before 2b), and each is independently legible/tunable.
|
||
|
||
**Tier 3 — Sim-state-tagged planes (frozen/flooded, whatever fields the map
|
||
time axis ruling lands on — ruled at lead interview 1, ruling 2, as
|
||
"current state via TTL-split"): explicit TTL, staleness-motivated,
|
||
structurally separate from tiers 1/2.** These entries carry a real
|
||
expiry — re-requested as sim time advances, per Jeroen's original hint made
|
||
concrete by the ruling. Distinct index field, distinct sweep condition
|
||
(`now > written_at + ttl`), never touched by the LRU-capacity or
|
||
time-since-visit sweeps (a sim-state entry doesn't get to live longer just
|
||
because disk space is available — it goes stale on its own schedule
|
||
regardless of capacity pressure).
|
||
|
||
### Index file schema (concrete)
|
||
|
||
One manifest per body, `user://atlas_cache/<body_id>/index.dat` (a small
|
||
binary or JSON — binary preferred for parse cost at scale, but this is an
|
||
implementation-detail choice, not an architecture one), loaded once on
|
||
body-open and held in memory as a `Dictionary` for the session — matching
|
||
the same "erase+reinsert = move-to-MRU" idiom `atlas_window_tile_set.gd`
|
||
already uses, just now also carrying the two extra timestamp/tag fields
|
||
tiers 2 and 3 need:
|
||
|
||
```
|
||
IndexEntry {
|
||
key: String # same composite key shape as atlas_window_tile_set.gd's
|
||
# make_key(), extended with step index:
|
||
# "<body_id>:<step>:<center>:<n>:<granularity_v2>"
|
||
file_path: String # user://atlas_cache/<body_id>/<key_hash>.png (or
|
||
# one file per field, per the wire contract's own
|
||
# per-field-PNG framing — Araminta's call, mirrored
|
||
# here, not re-decided)
|
||
tier: Geometry | SimState
|
||
written_at: int # unix time, set once, never updated
|
||
last_read_at: int # unix time, updated on every cache hit (drives 2a/2b)
|
||
size_bytes: int # drives 2b's budget accounting without a stat() call
|
||
sim_ttl: int? # only present when tier == SimState; null/absent
|
||
# for Geometry entries (2a/2b apply, 3 never does)
|
||
retention_floor: bool # true only for tier-1 (global/step-0) entries;
|
||
# short-circuits both sweep passes unconditionally
|
||
}
|
||
```
|
||
|
||
Directory layout: one PNG-per-field file per cache entry (mirrors the wire
|
||
encoding exactly — no format translation between "on the wire" and "on
|
||
disk"), named by a hash of `key` to avoid filesystem-unsafe characters
|
||
(the `:`/`,` composite key string itself isn't a safe filename on every
|
||
target platform); the index maps the human-legible key to that hash.
|
||
|
||
### Sweep triggers
|
||
|
||
- **On body-open:** load the index; run 2a (time-since-visit) immediately —
|
||
cheap (a metadata scan, no file I/O beyond the index itself) and this is
|
||
the natural moment ("returning to a body") where stale-by-absence entries
|
||
are most likely to exist and least likely to be needed again in the next
|
||
few seconds.
|
||
- **On a coarse background timer** (not per-frame, not even per-step-cross —
|
||
proposed on the order of minutes, tunable): run 2b (LRU-capacity) if
|
||
total sub-global bytes exceed budget, and sweep tier 3 for expired
|
||
sim-state entries. Both are backgroundable (Rayon-queue-adjacent on the
|
||
server side; on the client this is a low-priority deferred call, never
|
||
blocking a frame or a step-cross — matches premise 1's "smart precache
|
||
allowed, never blocking user output" applied to eviction as much as to
|
||
fetch).
|
||
- **Never per-frame.** All three sweep mechanisms are explicitly excluded
|
||
from the render/input loop — this is bookkeeping, not gameplay-adjacent
|
||
work, and belongs nowhere near the 16.6 ms budget ⑤ and ⑥ both care about.
|
||
|
||
### Per-tier budgets
|
||
|
||
- **Tier 1 (global):** sized to match Dudley's server-side number directly
|
||
— ~638 KB/body PNG-encoded, ~174 MB across all ~273 bodies if the client
|
||
has visited every body (an upper bound, not a typical player's actual
|
||
footprint). No cap needed beyond that — it's the "keep always" tier by
|
||
definition, and 174 MB is a trivial disk allocation on any target
|
||
platform.
|
||
- **Tier 2 (sub-global):** a configurable byte budget (proposed starting
|
||
point: same order of magnitude as tier 1, e.g. a few hundred MB to
|
||
low-GB range — a tuning-pass number once real play-pattern data exists,
|
||
not an architecture call I'm locking here) that 2b enforces directly, with
|
||
2a doing most of the practical work of keeping it under budget without
|
||
ever hitting the hard cap in normal play (a player who keeps returning to
|
||
the same few bodies never triggers 2a for those bodies' entries; a player
|
||
who body-hops constantly generates entries 2a will clear out on its own
|
||
schedule before 2b's cap becomes the active constraint).
|
||
- **Tier 3 (sim-state):** no byte budget needed — TTL alone bounds it, and
|
||
these planes are a small fraction of a step canvas's total field set (per
|
||
Araminta's round-1 schema, frozen/flooded reads off the existing
|
||
`glaciation`/morphology-water-class fields rather than adding new dense
|
||
arrays, so tier 3's actual footprint is small regardless).
|
||
|
||
### What stays true from D-227 regardless of which tier
|
||
|
||
Every tier is still an evictable cache, never a source of truth — deleting
|
||
the entire `user://atlas_cache/` directory at any time, for any reason
|
||
(corruption, manual clear, platform storage pressure), changes client
|
||
behavior only by causing re-fetches. No tier, no sweep policy, no budget
|
||
number in this spec is allowed to become load-bearing for correctness. This
|
||
is the same discipline Dudley's server-side tier commits to explicitly in
|
||
his round-1 doc, restated here for the client side as it was in round 1.
|
||
|
||
## (d) Px-per-gridunit band — sent to Dudley + Tyre
|
||
|
||
See the accompanying `SendMessage`. Summary of the recommendation for
|
||
their ladder tables: **1×1 px/gridunit at the deepest step (Tile,
|
||
D-243-pinned per Tyre's snap ruling), relaxing toward the ~5×5 fallback only
|
||
at the shallower/coarser steps where canvas extent (not spacing) is what's
|
||
growing.** Reasoning grounded in ⑥ + ⑤ together: crisper display costs
|
||
nothing extra in upload (⑤ showed upload is flat-cheap regardless of pixel
|
||
density, it's a function of byte count not "how many real gridunits per
|
||
displayed pixel") and colorize cost (⑥) is driven by **canvas cell count**,
|
||
which is already bounded by Dudley's viewport-sizing policy independent of
|
||
the display ratio — so there's no cost reason to fall back to 5×5 at the
|
||
steps where fidelity matters most (the deep, ground-level steps where the
|
||
player is closest to visible detail). The 5×5 fallback earns its keep at the
|
||
*shallow* steps instead, where canvas extent needs to cover a large ground
|
||
area and cell count would otherwise balloon for comparatively little
|
||
perceptual gain (players aren't scrutinizing per-metre detail from orbital
|
||
view). Full band recommendation in the message.
|