settled-reach/spikes/heightmap-pipeline/PIPELINE.md

# Heightmap Pipeline — Spike Documentation

**Ticket:** #778
**Author:** Araminta
**Date:** 2026-04-05
**Status:** Spike complete — awaiting review before batch (#794, Sprint 33)
**Outputs:**
- `GJ144d_heightmap.png` (4096×2048px, equirectangular, cartographic)
- `GJ144d_globe.png` (2048×2048px, ray-traced sphere)

---

## What This Spike Validates

This spike validates the full annotated heightmap + globe pipeline from wiki data
through to deliverable PNGs. Both outputs are produced from a single simulation run.

- Wiki parsing → body_def dict (body_definition_parser)
- FBM+Voronoi tectonic simulation → elevation grid (no external dependencies)
- Temperature model (stellar physics + class clamping)
- Moisture model (Hadley cells + ocean proximity + rain shadow)
- Hillshade (gradient-based)
- River network (D8 steepest descent, polyline output)
- Extended Whittaker biome classification (absolute Kelvin — no frozen-world tropics)
- Annotated equirectangular heightmap render
- Ray-traced globe render with terrain-driven surface, PBR lighting, atmosphere

**The pipeline is confirmed viable for batch production (#794).**

---

## Planet: Kallast (GJ144d)

Selected because it showcases temperate terrain variety and the wiki narrative
provides a direct visual brief.

| Property | Value | Source |
|----------|-------|--------|
| Planet ID | `GJ144d` | wiki bodies table |
| System | Ran (GJ 144) | wiki |
| Star | K2V | wiki |
| Planet class | temperate | wiki |
| Hydrosphere | ocean | wiki |
| Atmosphere | breathable | wiki |
| Gravity | 0.95g | wiki |
| Population | 2,000,000,000 | wiki |
| Terrain character | Amber continental shelves, irrigation channels | wiki narrative |

---

## Pipeline Architecture

```
Input: wiki/star-systems/GJ-144/index.md
         ↓ body_definition_parser.parse_system()

Stage 1: Body definition
         Parses bodies table → body_def dict per planet
         Derives: seed (MD5 of body_id), distance_au (Kepler),
                  land_fraction (hydrosphere→ HYDRO_LAND), polar_ice_lat,
                  axial_tilt (CLASS_TILT), tectonics, atmosphere
         "rand" sentinels → seeded randomization within class bounds
         Runtime: <0.1s

Stage 2: Elevation (planet_simulation.compute_elevation)
         Primary continent mask: FBM with domain warp (3 independent fields)
         Tectonic ridges: Voronoi plate boundaries + domain warp (curves)
         Detail noise: FBM high-frequency layer
         Dynamic sea level: np.percentile(elev, ocean_pct)
           CRITICAL: right-skewed FBM output requires percentile-derived threshold.
           Fixed fraction does not produce the target land coverage.
         Erosion: slope-weighted gaussian smoothing, N passes (varies by tectonics)
         Polar ice flattening at high latitudes
         All longitude noise uses 3D circle projection for seamless wrapping.
         Runtime: ~3-5s (512×256 grid)

Stage 3: Temperature (planet_simulation.compute_temperature)
         Stellar equilibrium temp (Stefan-Boltzmann) → greenhouse offset →
         CLASS_T_BAND clamp → latitude gradient → elevation lapse rate →
         class offset → geothermal boost
         CRITICAL: output is absolute Kelvin, not normalised.
           Biome classification uses raw K values to avoid frozen-world misclassification.
           Normalisation happens AFTER biome classification for renderer display.

Stage 4: Moisture (planet_simulation.compute_moisture)
         Hadley cell bands (ITCZ + subtropical high + polar) + ocean proximity +
         temperature contribution × rain shadow factor
         Class/hydrosphere scale factors applied per planet type

Stage 5: Hillshade (planet_simulation.compute_hillshade)
         Gradient-based normal → dot product with sun direction (315°az, 45°alt)
         Used for elevation shading in both heightmap and globe renders

Stage 6: Rivers (planet_simulation.compute_rivers)
         D8 steepest-descent flow from high-moisture local maxima
         Output: list of (row, col) polylines in simulation grid coordinates
         Max rivers capped per planet class (arid=3, frozen=2, default=12)
         River list stored in terrain dict; renderer scales coords to output resolution

Stage 7: Biome classification (planet_simulation.compute_biome)
         Extended Whittaker table lookup in absolute Kelvin × moisture [0,1]
         Ocean depth bands (0=deep, 1=mid, 2=shallow)
         Frozen ocean override (class 26 = ice shelf, distinct from land ice)
         Modifier stack: geothermal, chemosynthetic, UV, substrate overrides
         26 biome classes + 1 unused slot (0=deep ocean … 26=ice shelf)

Stage 8: Heightmap render (render_heightmap.render_heightmap)
         Layer compositing order:
           1. Biome base colour (cartographic or photographic palette)
           2. Ocean depth gradient (3-stop blend: shallow → mid → deep)
           3. Elevation shading on land  [0.88, 1.06] factor
           4. Hillshade blend (0.55 hs + 0.45 flat) — land only
           5. Coastline ring (binary_dilation XOR, 2px dark border)
           6. Rivers: PIL polylines scaled from grid coords to output pixels
              Width 1-3px scaled by path length (longer = wider)
           7. Lat/lon grid every 30° (12% white overlay, 2px)
           8. Title panel (name, class, star, orbit, atmo, hydro)
           9. Biome legend (present-only swatches, natural features only)
         Output resolution: configurable, default 4096×2048. UI_SCALE = W/1024.

Stage 9: Globe render (planet_renderer.render_globe)
         Ray-traced sphere (camera at z=3, looking at origin)
         Terrain-driven surface: biome→ photographic palette (27 classes, 0-26)
           Extended colour array (_EXTENDED_BIOME_COLORS) covers full class range
           including exotic classes 20-26. BIOME_COLORS (0-19) used only for
           procedural fallback when terrain=None.
         Elevation shading on land cells
         Full lighting: smoothstep diffuse, terminator warm scatter, ocean specular,
           atmospheric rim glow, night-side ambient
         Optional cloud layer: moisture-driven coverage + gaussian blur
         Star field background, atmosphere halo
         Output: RGBA PNG (alpha=255 on sphere+ring pixels, 0 on background)
         Default size: 2048×2048
```

---

## Two-Layer Model

Heightmaps are **geographic only**. Human data lives in JSON sidecars.

```
GJ144d_heightmap.png              ← geographic render: terrain, rivers, biomes, grid
GJ144d_settlements.json           ← human layer: city names + grid coordinates (Phase 3)
```

The PNG renders: terrain classification colors, elevation shading, river network,
lat/lon grid, title panel, biome legend.

The PNG does NOT render: settlements, roads, freight elevators, city labels,
irrigation channels, or any human-activity markers. Those exist in the JSON
sidecar and are overlaid by the atlas app (Phase 3) when the map is interactive.

**Rationale:** A geographic heightmap is a stable base layer. The human overlay
changes as the simulation runs. Keeping them separate means the PNG can be
regenerated from terrain data without recomputing settlement placement.

---

## Configuration

The body_def drives all simulation parameters. Key fields parsed from wiki:

| Field | Source | Effect |
|-------|--------|--------|
| `planet_class` | wiki type column | CLASS_T_BAND, erosion passes, river cap |
| `hydrosphere` | wiki hydro column | land_fraction, moisture scale |
| `atmosphere` | wiki atmo column | greenhouse offset, moisture computation |
| `gravity_g` | wiki gravity column | informs max_elevation_km |
| `star.type` | system profile | luminosity, star tint on globe |
| `orbit.distance_au` | derived (Kepler) | equilibrium temperature |

No manual per-planet configuration required for batch. All parameters derive
from the wiki's bodies table.

---

## Running

```bash
# Standard (4096×2048 heightmap + 2048×2048 globe)
python3 spikes/heightmap-pipeline/generate_kallast.py

# Fast iteration (1024×512 + 512×512)
python3 spikes/heightmap-pipeline/generate_kallast.py --small

# Photographic colour mode (orbital appearance, dark/muted)
python3 spikes/heightmap-pipeline/generate_kallast.py --render-mode photographic

# High resolution (8192×4096 heightmap)
python3 spikes/heightmap-pipeline/generate_kallast.py --out-w 8192 --out-h 4096

# Custom output paths
python3 spikes/heightmap-pipeline/generate_kallast.py \
    --output /tmp/kallast_hm.png \
    --globe-output /tmp/kallast_globe.png
```

Dependencies: `scipy`, `numpy`, `Pillow` (no external simulation engine required)

Individual prototype modules also have standalone CLIs:

```bash
# Parse body defs from wiki (inspect what the parser produces)
python3 prototype/body_definition_parser.py wiki/star-systems/GJ-144/index.md --out-dir /tmp/defs/

# Simulate only (inspect terrain grids)
python3 prototype/planet_simulation.py /tmp/defs/GJ144d_def.json --save-grids

# Render heightmap from body_def JSON
python3 prototype/render_heightmap.py /tmp/defs/GJ144d_def.json [--small]
```

---

## Known Issues / Calibration Notes for Batch

1. **Sea level uses percentile, not fixed fraction.**
   FBM output is non-uniformly distributed — `np.percentile(elev, ocean_pct)` gives
   correct land coverage. Fixed fractions do NOT work reliably.

2. **Temperature is absolute Kelvin throughout.**
   The Whittaker biome table uses K, not normalised [0,1]. This is intentional:
   prevents a frozen world's "warm" pole from classifying as tropical.
   Normalisation happens after biome classification for renderer display only.

3. **Class clamping (CLASS_T_BAND).**
   If stellar physics puts a temperate world outside its expected band (e.g. wiki
   says "temperate" but distance_au makes it hotter), temperature is clamped.
   The script logs a warning when clamping occurs. Check these during batch review.

4. **Rivers are polylines, not pixel masks.**
   The simulation outputs `(row, col)` paths. The renderer scales to output resolution
   and draws with PIL's anti-aliased line tool. Width 1-3px scales with path length.
   No pre-upscale painting / post-LANCZOS issues (old pyplatec pipeline concern).

5. **Globe uses photographic palette.**
   When terrain data is provided, `_EXTENDED_BIOME_COLORS` (27 entries, photographic
   values) is used instead of `BIOME_COLORS` (19 entries, procedural-style values).
   The extended array covers exotic classes 20-26 (lava fields, chemosynthetic mats,
   ash fields, ice shelves) which the old array silently clipped to index 19.

---

## Batch Estimate (#794)

Grid size: 512×256 (prototype default). Output: 4096×2048 heightmap + 2048 globe.

Timing per planet (single-threaded, approximate):

| Stage | Time |
|-------|------|
| Parse wiki | <0.1s |
| Simulation (elev+temp+moist+hs+rivers+biome) | ~3–6s |
| Heightmap render (4096×2048) | ~2–4s |
| Globe render (2048×2048) | ~3–6s |
| **Total per planet** | **~8–16s** |

Full batch of 301 planets: ~40–80 minutes single-threaded.
Parallelisable across all CPU cores (no shared state) — ~10–20 min on 4 cores.

For faster batch: `--globe-size 1024` halves globe render time with acceptable
fidelity for thumbnail/wiki use. Full 2048 globe recommended for atlas app.

---

## Open Questions for Review

Before starting batch (#794):

1. **Output resolution.** Default 4096×2048 for heightmap and 2048 for globe.
   Is this sufficient for the Phase 3 atlas app, or do we need 8192×4096?
   8192×4096 is available via `--out-w 8192 --out-h 4096` — adds ~4× render time.

2. **Render mode for batch.** Cartographic (NG map style) or photographic (orbital)?
   Cartographic reads more clearly as a map; photographic looks more realistic as a
   wiki thumbnail. Could produce both.

3. **River calibration per class.** max_rivers=12 is the default. Arid worlds get 3,
   frozen get 2. Is this density appropriate? Can compare against wiki narrative.

4. **Cloud layer.** Globe render supports moisture-driven clouds (enabled via body_def
   `clouds.enabled: true`). Should this be enabled for inhabited temperate worlds?