jpmschweitzerandClaude Sonnet 4.6
f64ee9dded
Validates full heightmap pipeline: pyplatec tectonics → erosion → dynamic sea level → terrain classification (13 classes) → D8 river network → settlement placement → geographic PNG + settlement JSON. Key technical decisions: - Dynamic sea level via np.percentile (pyplatec output is right-skewed; fixed fraction gives ~0.2% land, not 40%) - Terrain classes as fractions of land_range (not fixed offsets) - Two-layer model: geographic PNG + human-layer JSON sidecar - Rivers painted AFTER LANCZOS upscale via NEAREST neighbor mask (painting before blurs rivers into invisibility) - grain_belt removed — reclassified as grassland (natural terrain) - Irrigation overlay removed — human activity, lives in JSON sidecar Output: 4096×2048px (1024×512 simulation grid, 5.2s total runtime). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
252 lines
10 KiB
Markdown
252 lines
10 KiB
Markdown
# Heightmap Pipeline — Spike Documentation
|
||
|
||
**Ticket:** #778
|
||
**Author:** Araminta
|
||
**Date:** 2026-04-05
|
||
**Status:** Spike complete — awaiting review before batch (#794, Sprint 33)
|
||
**Output:** `heightmaps/GJ144d_kallast.png` (4096×2048px)
|
||
|
||
---
|
||
|
||
## What This Spike Validates
|
||
|
||
This spike validates the full annotated heightmap pipeline from wiki data through to a
|
||
deliverable PNG. Every stage ran successfully on Kallast (GJ144d, Ran system):
|
||
|
||
- pyplatec tectonic simulation → elevation grid
|
||
- Erosion pass → softer ridges, valley hints
|
||
- Dynamic sea level → correct 40% land coverage from wiki spec
|
||
- Terrain classification → 13 biome classes
|
||
- D8 flow accumulation → river network
|
||
- Settlement placement snapped to appropriate terrain class
|
||
- Road network connecting all cities
|
||
- Annotated render with title/legend in Settled Reach visual grammar
|
||
|
||
**The pipeline is confirmed viable for batch production (#794).**
|
||
|
||
---
|
||
|
||
## Planet: Kallast (GJ144d)
|
||
|
||
Selected because it showcases all annotation types:
|
||
|
||
| Property | Value | Source |
|
||
|----------|-------|--------|
|
||
| Planet ID | `GJ144d` | systems.db |
|
||
| System | Ran (GJ 144) | systems.db |
|
||
| Biome | temperate | systems.db |
|
||
| Hydrosphere | ocean | systems.db |
|
||
| Land coverage | 40% | wiki: "amber continental shelves" |
|
||
| Population | 2,000,000,000 | systems.db |
|
||
| Settlement wave | 1 (580y) | systems.db |
|
||
| Settlement pattern | urban_concentrated | systems.db |
|
||
| Industrial | Agricultural_Syndic | systems.db |
|
||
| Terrain character | Extensive temperate plains, amber-toned grassland | wiki narrative |
|
||
|
||
Kallast was chosen over higher-population worlds (Haodu, etc.) because the wiki narrative
|
||
explicitly describes the terrain features we need to annotate: "amber continental shelves
|
||
broken by irrigation channels wide enough to see from low orbit." That text is a direct
|
||
visual brief. The pipeline output should feel consistent with it.
|
||
|
||
---
|
||
|
||
## Pipeline Architecture
|
||
|
||
```
|
||
Input: Planet profile (wiki + systems.db)
|
||
↓ planet_type, land_fraction, settlement data
|
||
|
||
Stage 1: Tectonic simulation (pyplatec)
|
||
platec.create(seed, W, H, sea_level=land_fraction, …)
|
||
platec.step() × 200 [200 steps for mature, well-eroded world]
|
||
platec.get_heightmap() → float list → reshape → normalize [0, 1]
|
||
Runtime: ~1s at 512×256 (scales linearly with grid × steps)
|
||
|
||
Stage 2: Erosion (scipy gaussian_filter)
|
||
Slope-weighted smoothing: steep cells erode more
|
||
4 passes on mature world (reduce to 2 for young volcanic)
|
||
Runtime: <0.5s at 512×256
|
||
|
||
Stage 3: Dynamic sea level
|
||
sea_level = np.percentile(terrain, (1 - land_fraction) * 100)
|
||
CRITICAL: pyplatec output is heavily right-skewed (most cells at low
|
||
elevation). A fixed sea_level fraction (e.g. 0.40) does NOT produce
|
||
40% land — you get ~0.2% land. Always compute from actual distribution.
|
||
|
||
Stage 4: Terrain classification (13 classes)
|
||
Thresholds as fractions of the land elevation range [sea_level, max]
|
||
so classification scales correctly across different pyplatec outputs.
|
||
Classes: ocean_deep → ocean_mid → ocean_shallow → coast → lowland →
|
||
plains → grassland → hills → forest → highland → mountain →
|
||
peak → snow
|
||
|
||
Stage 5: D8 flow accumulation → river network
|
||
Sort land cells by elevation descending
|
||
Each cell drains to steepest downslope neighbour (8-directional)
|
||
Flow threshold: 30 (calibrated for 512×256 grid with 40% land)
|
||
Note: threshold scales with grid size and terrain relief — calibrate
|
||
per planet type. Very flat worlds (like Kallast) need lower threshold.
|
||
|
||
Stage 6: Settlement placement
|
||
For each city from wiki data: snap to nearest plains/grain_belt cell
|
||
within expanding search radius (20 → 40 → 60 → 80 cells)
|
||
Preference order: plains (class 5) > grain_belt (6) > lowland (4) >
|
||
coast (3) > hills (7)
|
||
|
||
Stage 7: Road network
|
||
Tier-1 and tier-2 cities connected by major roads (all-pairs from capital)
|
||
Tier-3 nodes connected to nearest tier-1/2 by minor roads
|
||
Rendered as polylines on the annotated layer
|
||
|
||
Stage 8: Geographic render (PIL)
|
||
1. Base terrain color layer (RGB from class colors)
|
||
2. Elevation shading on land (ambient occlusion proxy)
|
||
3. Dilate river mask at source grid resolution (2 iterations, preserves topology)
|
||
4. Scale up terrain to output resolution (4096×2048) via LANCZOS
|
||
5. Paint rivers AFTER upscale via NEAREST-neighbor upscaled mask
|
||
CRITICAL: painting before LANCZOS blurs rivers into invisibility.
|
||
Post-upscale NEAREST gives each source cell a 4×4px block — clearly legible.
|
||
6. Lat/lon grid lines (every 30°), scaled width
|
||
7. Title panel + legend — natural geographic features only
|
||
(ocean, coast, plains, grassland, mountain, river)
|
||
Text/panel sizes scale with UI_SCALE = OUTPUT_W / 1024
|
||
NOTE: settlements, roads, freight elevators are NOT rendered here.
|
||
They live in the JSON sidecar and are overlaid by the atlas app.
|
||
```
|
||
|
||
---
|
||
|
||
## Configuration per Planet Type
|
||
|
||
For the batch run (#794), per-planet config differs in:
|
||
|
||
| Parameter | Kallast | Young volcanic | Ice world | Desert | Ocean world |
|
||
|-----------|---------|----------------|-----------|--------|-------------|
|
||
| `plate_count` | 10 | 4 | 7 | 6 | 8 |
|
||
| `sim_steps` | 200 | 100 | 150 | 150 | 180 |
|
||
| `erosion_passes` | 4 | 1 | 3 (glacial) | 2 (aeolian) | 3 |
|
||
| `land_fraction` | 0.40 | 0.55 | 0.30 | 0.60 | 0.15 |
|
||
| `river_threshold` | 60 | 320 | 80 | 60 | 200 |
|
||
|
||
The `land_fraction` comes directly from the wiki's hydrosphere field:
|
||
- `ocean` → 0.30–0.45
|
||
- `liquid_water` → 0.40–0.60
|
||
- `ice` → 0.20–0.35
|
||
- `none` → 0.90–0.99
|
||
|
||
---
|
||
|
||
## Two-Layer Model
|
||
|
||
Heightmaps are **geographic only**. Human data lives in JSON sidecars.
|
||
|
||
```
|
||
kallast_heightmap.png ← geographic render: terrain, rivers, biomes, grid
|
||
kallast_heightmap_settlements.json ← human layer: city names + grid coordinates
|
||
```
|
||
|
||
The PNG renders: terrain classification colors, elevation shading, dilated river
|
||
network, lat/lon grid, title panel.
|
||
|
||
The PNG does NOT render: settlements, roads, freight elevators, city labels, irrigation
|
||
channels, or any human-activity markers. Those exist in the JSON sidecar and are overlaid
|
||
separately by the atlas app (Phase 3) when the map is interactive.
|
||
|
||
**Rationale:** A geographic heightmap is a stable base layer. The human overlay changes
|
||
as the simulation runs (cities grow, shrink, change character). Keeping them separate
|
||
means the PNG can be regenerated from terrain data without recomputing settlement
|
||
placement, and vice versa.
|
||
|
||
## Output Files
|
||
|
||
| File | Size | Description |
|
||
|------|------|-------------|
|
||
| `kallast_heightmap.png` | 4096×2048px | Geographic world map (deliverable) |
|
||
| `kallast_terrain.npy` | ~2MB | Raw normalised elevation grid (numpy float32, 1024×512) |
|
||
| `kallast_heightmap_settlements.json` | <1KB | City positions for atlas DB import (human layer sidecar) |
|
||
|
||
For batch production, the `.npy` and `.json` files are inputs to the Phase 3
|
||
atlas pipeline — they pre-seed the city layer rather than requiring re-computation.
|
||
|
||
---
|
||
|
||
## Known Issues / Calibration Notes for Batch
|
||
|
||
1. **River painting order is critical.** Painting river pixels into the source-resolution
|
||
array before LANCZOS upscaling blurs them into invisibility. Always dilate at source
|
||
resolution, then upscale with NEAREST neighbor and paint AFTER. Enforced in `render()`.
|
||
|
||
2. **Flat worlds produce sparse rivers.** Kallast has low terrain relief. Threshold=60
|
||
at 1024×512 gives 113 pre-dilation cells (1164 post). Scale threshold with grid area:
|
||
`threshold_1024 ≈ threshold_512 * 4`. For this flat world, halve the baseline to get
|
||
denser coverage.
|
||
|
||
3. **City placement uses wiki narrative coordinates, not astrophysical simulation.**
|
||
Relative positions (e.g. "Kallast Prime at 45% longitude, 48% latitude") are editorial
|
||
decisions. The snap algorithm finds nearest suitable terrain class within search radius.
|
||
This is intentional — settlement locations should reflect the world's narrative.
|
||
|
||
4. **Agricultural layer.** The wiki describes irrigation channels wide enough to see
|
||
from orbit. These are human infrastructure — they belong in the JSON sidecar, not the
|
||
geographic heightmap. Phase 3 atlas work should render irrigation channels as a
|
||
separate overlay from hydrology + settlement data.
|
||
|
||
---
|
||
|
||
## Batch Run Estimate (#794)
|
||
|
||
Grid size: 1024×512. Output: 4096×2048.
|
||
|
||
| Phase | Step | Time per planet | 301 planets |
|
||
|-------|------|-----------------|-------------|
|
||
| Tectonic (200 steps, 1024×512) | ~3.5s | 1054s |
|
||
| Erosion (4 passes) | ~1.0s | 301s |
|
||
| Hydrology | ~0.5s | 151s |
|
||
| Placement + roads | ~0.5s | 151s |
|
||
| Render + export | ~0.8s | 241s |
|
||
| **Total** | | **~6.3s/planet** | **~32 minutes** |
|
||
|
||
Full batch of 301 systems runs in ~32 minutes single-threaded. Parallelisable across all
|
||
CPU cores (no shared state) — realistically ~8 minutes on 4 cores.
|
||
|
||
Note: if batch time is a concern, `sim_steps=100` halves tectonic time with acceptable
|
||
terrain quality for most planet types. Only mature worlds (Kallast, old ocean worlds)
|
||
benefit meaningfully from 200 steps.
|
||
|
||
---
|
||
|
||
## Running the Spike
|
||
|
||
```bash
|
||
# Standard (200 tectonic steps, ~1.5s)
|
||
python3 spikes/heightmap-pipeline/generate_kallast.py
|
||
|
||
# Fast mode (50 steps — good for testing annotation, poor terrain)
|
||
python3 spikes/heightmap-pipeline/generate_kallast.py --fast
|
||
|
||
# Different seed (changes continent layout)
|
||
python3 spikes/heightmap-pipeline/generate_kallast.py --seed 42
|
||
|
||
# Custom output path
|
||
python3 spikes/heightmap-pipeline/generate_kallast.py --output heightmaps/GJ144d_kallast_v2.png
|
||
```
|
||
|
||
Dependencies: `pyplatec`, `scipy`, `numpy`, `Pillow` (all installable via pip)
|
||
|
||
---
|
||
|
||
## Open Questions for Review
|
||
|
||
Before starting batch (#794), Jeroen should confirm:
|
||
|
||
1. **Visual style.** Does the terrain color palette work? The grassland amber
|
||
(`#a59b4b`) reads as temperate plains — is this the right mood for Kallast?
|
||
2. **Annotation density.** 12 settlements for a 2B-population world — too sparse? too
|
||
many? For the batch, settlement count would be derived from wiki city data (if any)
|
||
or a formula from population + settlement_pattern.
|
||
3. **Output resolution.** 1024×512 adequate for wiki use? Or do we need 2048×1024
|
||
for the implant atlas app (Phase 3)?
|
||
4. **River threshold calibration.** The flat terrain of Kallast needed threshold=30.
|
||
Should we auto-calibrate per planet by targeting N river-mouth cells, rather than
|
||
a fixed threshold?
|