jpmschweitzer/settled-reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(tooling): address PR #129 review — atlas generator, schema, Makefile

Browse Source 
- ON DELETE CASCADE added to every atlas_* foreign key (atlas_body_grids,
  atlas_cities, atlas_roads, atlas_railroads, atlas_pois, atlas_rivers,
  atlas_oceans, atlas_mountain_ranges). Previously, deleting a body from
  the bodies table or NULL-ing its terrain_reference would leave orphan
  atlas rows forever — sync_markers_to_db only cleans up for bodies it
  re-processes. The existing atlas tables in systems.db were dropped and
  recreated with the new constraint; FK list now reports CASCADE.

- Atlas DDL deduplicated. systems-schema.sql is now the single source of
  truth, bracketed by `-- BEGIN ATLAS INDEX` / `-- END ATLAS INDEX`
  markers. generate_atlas.py reads that block via `_load_atlas_schema()`
  and applies it at runtime, so there is no second copy of the DDL to
  keep in sync. Adding a column requires one edit, not two.

- Uniqueness guard on city coordinates. `_enforce_unique_city_coords`
  runs at the end of `place_cities` and deterministically perturbs any
  duplicate (row, col) via a fixed spiral walk to the first free
  walkable land cell. Rare in practice but the MST collapses to a
  zero-distance edge otherwise, producing an empty A* path and silently
  dropping the road.

- Grid header validation. `load_markers` now raises `AtlasGridMismatch`
  if the loaded `grid: {w, h}` header does not match `GRID_W`/`GRID_H`.
  Both the incremental-skip path and the regenerate path route through
  this loader, so a hand-authored template shipping a different grid
  size fails loud with a per-body error rather than silently producing
  half-scale coordinates.

- Unused `seed_rng` parameter removed from `_analyse_terrain`. The
  function is RNG-free (continent flood-fill, habitability scoring,
  river-mouth dedup, cost grid — all pure functions of terrain). The
  false API contract made it look like terrain analysis consumed RNG
  state and had to be sequenced with downstream RNG use.

- `_score_capital_sites` river-mouth bonus now builds one sparse
  accumulator with all mouth points set at once and runs a single
  `gaussian_filter` call, instead of O(n_mouths) filter calls over
  single-point images.

- `binary_dilation(analysis["land_mask"] == False)` replaced with the
  idiomatic `~analysis["land_mask"]`, matching the convention used
  elsewhere in the file.

- `atlas-generate` Makefile target now guards on
  `SELECT COUNT(*) FROM bodies WHERE terrain_reference IS NOT NULL`.
  On a fresh DB that count is 0 and the generator previously exited
  "success" after processing zero bodies. The target now fails loud
  with a pointer to `populate_terrain_reference.py`.

- `main.rs` SimRng defensive re-insertion gains a long comment
  explaining the exact plugin-ordering hazard it guards against, so
  future readers don't treat the line as dead code. Tied to #826.
This commit is contained in:
jpmschweitzer
2026-04-15 09:25:46 +02:00
parent 64bb83f749
commit d21c690293
5 changed files with 209 additions and 152 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tooling/planet-gen/generate_atlas.py
+158 -127
View File
@@ -78,116 +78,51 @@ QUADRANT_SATURATION = 2
DB_PATH = REPO_ROOT / "server" / "data" / "systems.db"
WIKI_SYSTEMS = REPO_ROOT / "wiki" / "star-systems"
SYSTEMS_SCHEMA_PATH = REPO_ROOT / "server" / "data" / "systems-schema.sql"
# Delimiters for the single canonical atlas_* DDL block in
# server/data/systems-schema.sql. `_load_atlas_schema()` extracts everything
# between these markers at runtime so this file does not have to duplicate
# the schema (and drift from it).
_ATLAS_SCHEMA_BEGIN_MARKER = "-- BEGIN ATLAS INDEX"
_ATLAS_SCHEMA_END_MARKER = "-- END ATLAS INDEX"
# ---------------------------------------------------------------------------
# Atlas index tables (D-191 §8, #832) — scalar mirror of markers.json.
# Duplicated from server/data/systems-schema.sql so generate_atlas.py works
# against any DB state without needing a wiki_sync first. Keep in sync with
# the canonical schema when adding columns.
# ---------------------------------------------------------------------------
ATLAS_MIGRATION_SQL = """
CREATE TABLE IF NOT EXISTS atlas_body_grids (
body_id TEXT PRIMARY KEY REFERENCES bodies(body_id),
grid_w INTEGER NOT NULL,
grid_h INTEGER NOT NULL,
updated_at TEXT DEFAULT (datetime('now'))
);
def _load_atlas_schema() -> str:
"""Return the atlas_* DDL block from systems-schema.sql.
CREATE TABLE IF NOT EXISTS atlas_cities (
city_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
center_row INTEGER NOT NULL,
center_col INTEGER NOT NULL,
population INTEGER NOT NULL DEFAULT 0,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_roads (
road_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
point_count INTEGER NOT NULL,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_railroads (
railroad_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
point_count INTEGER NOT NULL,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_pois (
poi_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
center_row INTEGER NOT NULL,
center_col INTEGER NOT NULL,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_rivers (
river_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
point_count INTEGER NOT NULL,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_oceans (
water_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
kind TEXT NOT NULL,
center_row INTEGER NOT NULL,
center_col INTEGER NOT NULL,
area_fraction REAL NOT NULL DEFAULT 0.0,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS atlas_mountain_ranges (
range_id TEXT PRIMARY KEY,
body_id TEXT NOT NULL REFERENCES bodies(body_id),
local_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
center_row INTEGER NOT NULL,
center_col INTEGER NOT NULL,
peak_row INTEGER NOT NULL,
peak_col INTEGER NOT NULL,
area_cells INTEGER NOT NULL DEFAULT 0,
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_atlas_cities_body ON atlas_cities(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_cities_kind ON atlas_cities(kind);
CREATE INDEX IF NOT EXISTS idx_atlas_cities_population ON atlas_cities(population);
CREATE INDEX IF NOT EXISTS idx_atlas_cities_name ON atlas_cities(name);
CREATE INDEX IF NOT EXISTS idx_atlas_roads_body ON atlas_roads(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_railroads_body ON atlas_railroads(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_pois_body ON atlas_pois(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_pois_kind ON atlas_pois(kind);
CREATE INDEX IF NOT EXISTS idx_atlas_rivers_body ON atlas_rivers(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_oceans_body ON atlas_oceans(body_id);
CREATE INDEX IF NOT EXISTS idx_atlas_mountain_ranges_body ON atlas_mountain_ranges(body_id);
"""
systems-schema.sql is the single source of truth for the atlas index
tables (see the BEGIN ATLAS INDEX / END ATLAS INDEX markers). We extract
just that block and run it through `executescript` so the generator
works against any DB state (fresh or partially-migrated) without
requiring a prior `wiki_sync.ensure_schema()` call and without
maintaining a second copy of the DDL here.
"""
if not SYSTEMS_SCHEMA_PATH.exists():
raise RuntimeError(
f"systems-schema.sql not found at {SYSTEMS_SCHEMA_PATH} — "
"atlas generator cannot proceed without the canonical schema."
)
text = SYSTEMS_SCHEMA_PATH.read_text()
try:
start = text.index(_ATLAS_SCHEMA_BEGIN_MARKER)
end = text.index(_ATLAS_SCHEMA_END_MARKER, start)
except ValueError as e:
raise RuntimeError(
f"systems-schema.sql is missing the {_ATLAS_SCHEMA_BEGIN_MARKER}/"
f"{_ATLAS_SCHEMA_END_MARKER} block — has the schema been "
"restructured?"
) from e
return text[start:end]
def ensure_atlas_schema(conn: sqlite3.Connection) -> None:
"""Apply CREATE TABLE / INDEX IF NOT EXISTS for all atlas_* tables."""
conn.executescript(ATLAS_MIGRATION_SQL)
"""Apply the canonical atlas_* DDL from systems-schema.sql.
Idempotent: all statements inside the block use CREATE TABLE / INDEX
IF NOT EXISTS, so running this on an already-migrated DB is a no-op.
"""
conn.executescript(_load_atlas_schema())
def _first_int(values, default: int = 0) -> int:
@@ -438,9 +373,14 @@ def load_body_def(body_dir: Path) -> dict | None:
# Terrain analysis
# ---------------------------------------------------------------------------
def _analyse_terrain(terrain: dict, seed_rng: np.random.Generator) -> dict:
def _analyse_terrain(terrain: dict) -> dict:
"""Analyse terrain for atlas city placement.
Deterministic and RNG-free: continent flood-fill, habitability scoring,
river-mouth detection, and the terrain cost grid are all pure functions
of the terrain dict. Per-body variation comes from the RNG used later
in `_score_capital_sites` and `place_cities`, not from this function.
Returns:
continents: list of continent dicts {cells, area, id}
habitability: float32 (H, W) — 0=uninhabitable, 1=ideal
@@ -634,15 +574,15 @@ def _score_capital_sites(
# Base score
score = habitability.copy()
# River mouth bonus (makes ~50% of capital choices go there)
river_bonus = np.zeros((GRID_H, GRID_W), dtype=np.float32)
for r, c in river_mouths:
# Smooth bonus around the river mouth
from scipy.ndimage import gaussian_filter
tmp = np.zeros((GRID_H, GRID_W), np.float32)
tmp[r, c] = 1.0
river_bonus += gaussian_filter(tmp, sigma=8.0)
# River mouth bonus (makes ~50% of capital choices go there).
# Build a single sparse accumulator with all mouth points set, then
# run gaussian_filter once — O(1) filter calls rather than O(n_mouths).
if river_mouths:
from scipy.ndimage import gaussian_filter
mouth_field = np.zeros((GRID_H, GRID_W), dtype=np.float32)
for r, c in river_mouths:
mouth_field[r, c] = 1.0
river_bonus = gaussian_filter(mouth_field, sigma=8.0)
river_bonus = river_bonus / (river_bonus.max() + 1e-9)
score = score * 0.5 + river_bonus * 0.5
@@ -810,7 +750,7 @@ def place_cities(
cont_mask = continent_map == cont["id"]
# Coastal cells: continent land cells adjacent to water
from scipy.ndimage import binary_dilation
water_dilated = binary_dilation(analysis["land_mask"] == False) # noqa: E712
water_dilated = binary_dilation(~analysis["land_mask"])
coast_mask = cont_mask & water_dilated
coast_bonus[coast_mask] = 0.4
site_score = site_score + coast_bonus
@@ -849,6 +789,7 @@ def place_cities(
"_continent_id": cont_id,
})
_enforce_unique_city_coords(cities, analysis["land_mask"])
_assign_populations(cities, body_population)
return cities
@@ -860,6 +801,58 @@ def _assign_populations(cities: list[dict], body_population: int) -> None:
city["population"] = int(pop)
def _enforce_unique_city_coords(cities: list[dict], land: np.ndarray) -> None:
"""Guarantee no two cities share the same (row, col).
Rare on real heightmaps but possible when grids are small and the
quadrant-saturation penalty pushes candidates into tight corners. If
two cities land on identical pixels the MST treats them as zero-
distance nodes and A* produces an empty path, silently skipping the
edge. We deterministically perturb duplicates by walking outward in
a fixed spiral until a free, walkable land cell is found; the search
order is fully determined by `city_index` so the operation stays
byte-equivalent across runs.
"""
if len(cities) <= 1:
return
# Fixed spiral offsets — small radius first, then widen.
spiral: list[tuple[int, int]] = []
for radius in range(1, 12):
for dr in range(-radius, radius + 1):
for dc in range(-radius, radius + 1):
if abs(dr) == radius or abs(dc) == radius:
spiral.append((dr, dc))
occupied: set[tuple[int, int]] = set()
for idx, city in enumerate(cities):
center = (city["_row"], city["_col"])
if center not in occupied:
occupied.add(center)
continue
# Duplicate — walk the spiral for the first free land cell.
for dr, dc in spiral:
nr, nc = center[0] + dr, center[1] + dc
if 0 <= nr < GRID_H and 0 <= nc < GRID_W and land[nr, nc] and (nr, nc) not in occupied:
print(
f" warning: city {idx} at {center} collided with an "
f"earlier placement — deterministically perturbed to ({nr}, {nc})"
)
city["_row"] = nr
city["_col"] = nc
city["center"] = [nr, nc]
occupied.add((nr, nc))
break
else:
# Fallback: land is saturated — just keep the duplicate; the
# MST edge will collapse but the body is a degenerate case.
print(
f" warning: city {idx} at {center} has no free neighbor "
f"— duplicate allowed (degenerate body)"
)
occupied.add(center)
# ---------------------------------------------------------------------------
# Infrastructure generation (D-191 §3)
# ---------------------------------------------------------------------------
@@ -1052,12 +1045,40 @@ def place_gate_terminal(
# Markers.json update
# ---------------------------------------------------------------------------
class AtlasGridMismatch(Exception):
"""Raised when a markers.json grid header does not match generator constants.
If a hand-authored template ships with, say, `{"w": 1024, "h": 512}`
and the generator overlays new cities computed against the 512 × 256
cost grid, every coordinate is half-scale and every marker is broken.
Fail loud here rather than silently produce corrupt output.
"""
def load_markers(body_dir: Path) -> dict:
"""Load existing markers.json, return empty structure if missing."""
"""Load existing markers.json, return empty structure if missing.
Validates the grid header against the generator constants so the
caller can trust that overlaid city/road/poi coordinates are in
the same pixel space as the loaded geographic features. A mismatch
raises `AtlasGridMismatch` — regenerating a markers.json against
a different grid would corrupt every coordinate in it.
"""
markers_path = body_dir / "markers.json"
if markers_path.exists():
with open(markers_path) as f:
return json.load(f)
markers = json.load(f)
grid = markers.get("grid") or {}
g_w = grid.get("w")
g_h = grid.get("h")
if g_w != GRID_W or g_h != GRID_H:
raise AtlasGridMismatch(
f"{markers_path} has grid {{w: {g_w}, h: {g_h}}} but the "
f"atlas generator runs against {{w: {GRID_W}, h: {GRID_H}}}. "
"Either regenerate the heightmap pipeline at the generator "
"resolution, or update GRID_W / GRID_H to match the source."
)
return markers
return {
"grid": {"w": GRID_W, "h": GRID_H},
"rivers": [],
@@ -1141,19 +1162,23 @@ def process_body(
# Incremental check: if the markers file already has cities, we keep it as
# the source of truth and return it for DB sync instead of regenerating.
# `load_markers` validates the grid header; a mismatch on a hand-authored
# template is reported as an error so downstream DB sync doesn't silently
# index a file that disagrees with the generator's pixel space.
if not force:
markers_path = body_dir / "markers.json"
if markers_path.exists():
try:
with open(markers_path) as f:
existing = json.load(f)
if existing.get("cities"):
return {
"status": "already_populated",
"markers": existing,
}
existing = load_markers(body_dir)
except AtlasGridMismatch as e:
return {"status": "error", "message": str(e)}
except json.JSONDecodeError:
pass # corrupted markers.json → regenerate below
existing = None # corrupted markers.json → regenerate below
if existing and existing.get("cities"):
return {
"status": "already_populated",
"markers": existing,
}
bd = load_body_def(body_dir)
if not bd:
@@ -1171,7 +1196,7 @@ def process_body(
if not terrain:
return {"status": "gas_giant"}
analysis = _analyse_terrain(terrain, rng)
analysis = _analyse_terrain(terrain)
n_cities = compute_city_count(population, settlement_pattern)
@@ -1187,8 +1212,14 @@ def process_body(
pois = place_gate_terminal(cities, rng)
# Load existing markers (preserves rivers/oceans/mountain_ranges) and
# overlay the new generator output.
markers = load_markers(body_dir)
# overlay the new generator output. `load_markers` asserts the loaded
# grid header matches the generator constants — a mismatch would
# silently corrupt every coordinate in the file.
try:
markers = load_markers(body_dir)
except AtlasGridMismatch as e:
return {"status": "error", "message": str(e)}
output_cities = [
{k: v for k, v in city.items() if not k.startswith("_")}
for city in cities