settled-reach/spikes/heightmap-pipeline/PIPELINE.md

Heightmap Pipeline — Spike Documentation

Ticket: #778 Author: Araminta Date: 2026-04-05 Status: Spike complete — awaiting review before batch (#794, Sprint 33) Outputs:

• GJ144d_heightmap.png (4096×2048px, equirectangular, cartographic)
• GJ144d_globe.png (2048×2048px, ray-traced sphere)

---

What This Spike Validates

This spike validates the full annotated heightmap + globe pipeline from wiki data through to deliverable PNGs. Both outputs are produced from a single simulation run.

• Wiki parsing → body_def dict (body_definition_parser)
• FBM+Voronoi tectonic simulation → elevation grid (no external dependencies)
• Temperature model (stellar physics + class clamping)
• Moisture model (Hadley cells + ocean proximity + rain shadow)
• Hillshade (gradient-based)
• River network (D8 steepest descent, polyline output)
• Extended Whittaker biome classification (absolute Kelvin — no frozen-world tropics)
• Annotated equirectangular heightmap render
• Ray-traced globe render with terrain-driven surface, PBR lighting, atmosphere

The pipeline is confirmed viable for batch production (#794).

---

Planet: Kallast (GJ144d)

Selected because it showcases temperate terrain variety and the wiki narrative provides a direct visual brief.

Property	Value	Source
Planet ID	GJ144d	wiki bodies table
System	Ran (GJ 144)	wiki
Star	K2V	wiki
Planet class	temperate	wiki
Hydrosphere	ocean	wiki
Atmosphere	breathable	wiki
Gravity	0.95g	wiki
Population	2,000,000,000	wiki
Terrain character	Amber continental shelves, irrigation channels	wiki narrative

---

Pipeline Architecture

Input: wiki/star-systems/GJ-144/index.md
         ↓ body_definition_parser.parse_system()

Stage 1: Body definition
         Parses bodies table → body_def dict per planet
         Derives: seed (MD5 of body_id), distance_au (Kepler),
                  land_fraction (hydrosphere→ HYDRO_LAND), polar_ice_lat,
                  axial_tilt (CLASS_TILT), tectonics, atmosphere
         "rand" sentinels → seeded randomization within class bounds
         Runtime: <0.1s

Stage 2: Elevation (planet_simulation.compute_elevation)
         Primary continent mask: FBM with domain warp (3 independent fields)
         Tectonic ridges: Voronoi plate boundaries + domain warp (curves)
         Detail noise: FBM high-frequency layer
         Dynamic sea level: np.percentile(elev, ocean_pct)
           CRITICAL: right-skewed FBM output requires percentile-derived threshold.
           Fixed fraction does not produce the target land coverage.
         Erosion: slope-weighted gaussian smoothing, N passes (varies by tectonics)
         Polar ice flattening at high latitudes
         All longitude noise uses 3D circle projection for seamless wrapping.
         Runtime: ~3-5s (512×256 grid)

Stage 3: Temperature (planet_simulation.compute_temperature)
         Stellar equilibrium temp (Stefan-Boltzmann) → greenhouse offset →
         CLASS_T_BAND clamp → latitude gradient → elevation lapse rate →
         class offset → geothermal boost
         CRITICAL: output is absolute Kelvin, not normalised.
           Biome classification uses raw K values to avoid frozen-world misclassification.
           Normalisation happens AFTER biome classification for renderer display.

Stage 4: Moisture (planet_simulation.compute_moisture)
         Hadley cell bands (ITCZ + subtropical high + polar) + ocean proximity +
         temperature contribution × rain shadow factor
         Class/hydrosphere scale factors applied per planet type

Stage 5: Hillshade (planet_simulation.compute_hillshade)
         Gradient-based normal → dot product with sun direction (315°az, 45°alt)
         Used for elevation shading in both heightmap and globe renders

Stage 6: Rivers (planet_simulation.compute_rivers)
         D8 steepest-descent flow from high-moisture local maxima
         Output: list of (row, col) polylines in simulation grid coordinates
         Max rivers capped per planet class (arid=3, frozen=2, default=12)
         River list stored in terrain dict; renderer scales coords to output resolution

Stage 7: Biome classification (planet_simulation.compute_biome)
         Extended Whittaker table lookup in absolute Kelvin × moisture [0,1]
         Ocean depth bands (0=deep, 1=mid, 2=shallow)
         Frozen ocean override (class 26 = ice shelf, distinct from land ice)
         Modifier stack: geothermal, chemosynthetic, UV, substrate overrides
         26 biome classes + 1 unused slot (0=deep ocean … 26=ice shelf)

Stage 8: Heightmap render (render_heightmap.render_heightmap)
         Layer compositing order:
           1. Biome base colour (cartographic or photographic palette)
           2. Ocean depth gradient (3-stop blend: shallow → mid → deep)
           3. Elevation shading on land  [0.88, 1.06] factor
           4. Hillshade blend (0.55 hs + 0.45 flat) — land only
           5. Coastline ring (binary_dilation XOR, 2px dark border)
           6. Rivers: PIL polylines scaled from grid coords to output pixels
              Width 1-3px scaled by path length (longer = wider)
           7. Lat/lon grid every 30° (12% white overlay, 2px)
           8. Title panel (name, class, star, orbit, atmo, hydro)
           9. Biome legend (present-only swatches, natural features only)
         Output resolution: configurable, default 4096×2048. UI_SCALE = W/1024.

Stage 9: Globe render (planet_renderer.render_globe)
         Ray-traced sphere (camera at z=3, looking at origin)
         Terrain-driven surface: biome→ photographic palette (27 classes, 0-26)
           Extended colour array (_EXTENDED_BIOME_COLORS) covers full class range
           including exotic classes 20-26. BIOME_COLORS (0-19) used only for
           procedural fallback when terrain=None.
         Elevation shading on land cells
         Full lighting: smoothstep diffuse, terminator warm scatter, ocean specular,
           atmospheric rim glow, night-side ambient
         Optional cloud layer: moisture-driven coverage + gaussian blur
         Star field background, atmosphere halo
         Output: RGBA PNG (alpha=255 on sphere+ring pixels, 0 on background)
         Default size: 2048×2048

---

Two-Layer Model

Heightmaps are geographic only. Human data lives in JSON sidecars.

GJ144d_heightmap.png              ← geographic render: terrain, rivers, biomes, grid
GJ144d_settlements.json           ← human layer: city names + grid coordinates (Phase 3)

The PNG renders: terrain classification colors, elevation shading, river network, lat/lon grid, title panel, biome legend.

The PNG does NOT render: settlements, roads, freight elevators, city labels, irrigation channels, or any human-activity markers. Those exist in the JSON sidecar and are overlaid by the atlas app (Phase 3) when the map is interactive.

Rationale: A geographic heightmap is a stable base layer. The human overlay changes as the simulation runs. Keeping them separate means the PNG can be regenerated from terrain data without recomputing settlement placement.

---

Configuration

The body_def drives all simulation parameters. Key fields parsed from wiki:

Field	Source	Effect
planet_class	wiki type column	CLASS_T_BAND, erosion passes, river cap
hydrosphere	wiki hydro column	land_fraction, moisture scale
atmosphere	wiki atmo column	greenhouse offset, moisture computation
gravity_g	wiki gravity column	informs max_elevation_km
star.type	system profile	luminosity, star tint on globe
orbit.distance_au	derived (Kepler)	equilibrium temperature

No manual per-planet configuration required for batch. All parameters derive from the wiki's bodies table.

---

Running

# Standard (4096×2048 heightmap + 2048×2048 globe)
python3 spikes/heightmap-pipeline/generate_kallast.py

# Fast iteration (1024×512 + 512×512)
python3 spikes/heightmap-pipeline/generate_kallast.py --small

# Photographic colour mode (orbital appearance, dark/muted)
python3 spikes/heightmap-pipeline/generate_kallast.py --render-mode photographic

# High resolution (8192×4096 heightmap)
python3 spikes/heightmap-pipeline/generate_kallast.py --out-w 8192 --out-h 4096

# Custom output paths
python3 spikes/heightmap-pipeline/generate_kallast.py \
    --output /tmp/kallast_hm.png \
    --globe-output /tmp/kallast_globe.png

Dependencies: scipy, numpy, Pillow (no external simulation engine required)

Individual prototype modules also have standalone CLIs:

# Parse body defs from wiki (inspect what the parser produces)
python3 prototype/body_definition_parser.py wiki/star-systems/GJ-144/index.md --out-dir /tmp/defs/

# Simulate only (inspect terrain grids)
python3 prototype/planet_simulation.py /tmp/defs/GJ144d_def.json --save-grids

# Render heightmap from body_def JSON
python3 prototype/render_heightmap.py /tmp/defs/GJ144d_def.json [--small]

---

Known Issues / Calibration Notes for Batch

1. Sea level uses percentile, not fixed fraction. FBM output is non-uniformly distributed — np.percentile(elev, ocean_pct) gives correct land coverage. Fixed fractions do NOT work reliably.

2. Temperature is absolute Kelvin throughout. The Whittaker biome table uses K, not normalised [0,1]. This is intentional: prevents a frozen world's "warm" pole from classifying as tropical. Normalisation happens after biome classification for renderer display only.

3. Class clamping (CLASS_T_BAND). If stellar physics puts a temperate world outside its expected band (e.g. wiki says "temperate" but distance_au makes it hotter), temperature is clamped. The script logs a warning when clamping occurs. Check these during batch review.

4. Rivers are polylines, not pixel masks. The simulation outputs (row, col) paths. The renderer scales to output resolution and draws with PIL's anti-aliased line tool. Width 1-3px scales with path length. No pre-upscale painting / post-LANCZOS issues (old pyplatec pipeline concern).

5. Globe uses photographic palette. When terrain data is provided, _EXTENDED_BIOME_COLORS (27 entries, photographic values) is used instead of BIOME_COLORS (19 entries, procedural-style values). The extended array covers exotic classes 20-26 (lava fields, chemosynthetic mats, ash fields, ice shelves) which the old array silently clipped to index 19.

---

Batch Estimate (#794)

Grid size: 512×256 (prototype default). Output: 4096×2048 heightmap + 2048 globe.

Timing per planet (single-threaded, approximate):

Stage	Time
Parse wiki	<0.1s
Simulation (elev+temp+moist+hs+rivers+biome)	~3–6s
Heightmap render (4096×2048)	~2–4s
Globe render (2048×2048)	~3–6s
Total per planet	~8–16s

Full batch of 301 planets: ~40–80 minutes single-threaded. Parallelisable across all CPU cores (no shared state) — ~10–20 min on 4 cores.

For faster batch: --globe-size 1024 halves globe render time with acceptable fidelity for thumbnail/wiki use. Full 2048 globe recommended for atlas app.

---

Open Questions for Review

Before starting batch (#794):

1. Output resolution. Default 4096×2048 for heightmap and 2048 for globe. Is this sufficient for the Phase 3 atlas app, or do we need 8192×4096? 8192×4096 is available via --out-w 8192 --out-h 4096 — adds ~4× render time.

2. Render mode for batch. Cartographic (NG map style) or photographic (orbital)? Cartographic reads more clearly as a map; photographic looks more realistic as a wiki thumbnail. Could produce both.

3. River calibration per class. max_rivers=12 is the default. Arid worlds get 3, frozen get 2. Is this density appropriate? Can compare against wiki narrative.

4. Cloud layer. Globe render supports moisture-driven clouds (enabled via body_def clouds.enabled: true). Should this be enabled for inhabited temperate worlds?