Parallelism via two concurrent sr-voice subprocesses does not work on
this ROCm + llama-cpp-rs setup — launching a second instance poisons
the first one's GPU context (both fall back to 0% GPU / 50% CPU
busy-loop and stop making progress). Verified empirically: single
shard runs cleanly at ~1.2s/feature, two shards deadlock.
Without a working parallel path, --shard is dead weight. Resume
semantics were already free: the pipeline skips bodies whose
markers.json has non-empty name fields (preserved path), so a
killed run re-starts just by re-running the same command.
Simplifications:
- Remove --shard argument and all slicing logic.
- Remove banner_shard / shard_offset / shard_n / shard_m plumbing.
- Rename internal total_shard_systems → total_systems.
- Default --log path is now .tmp/gemma_naming.log (was conditional
on --shard). Pass `--log -` to disable file logging.
- Startup banner now prints a one-line resume reminder so the user
can see at a glance that a killed run is recoverable.
New end-to-end pipeline that walks every markers.json in the reach and
fills empty `name` fields using the Gemma 2 voice pipeline via
`sr-voice serve --stdio`. Per D-191 §4: the same Gemma 2 pipeline the
client uses for NPC voicing also produces the atlas content, which is
dual-purposed as a quality test of the LLM plumbing.
Pipeline per body (hop-ordered, core-first):
1. Load markers.json; identify feature records whose `name` is
blank (null or ""). Hand-authored names are never overwritten;
the 6 template bodies and any partial authoring stay put.
2. Look up body context (planet_class, settlement_pattern,
cultural_corridor, population, economic_role) from systems.db.
3. Build a short corridor-aware few-shot prompt per feature type.
Prompts carry 3 concrete `Style: X. Answer: Y` examples so
Gemma 2 2B completes a pattern instead of generating to an
open-ended instruction — this is the single biggest lever
against placeholder echoes on a small model.
4. Stream the prompt into a long-lived sr-voice subprocess, read
the JSONL response, post-process (strip markdown, label
prefixes, brackets, reject 5+ word outputs and placeholder
tokens), check the earth-name blocklist, check per-(corridor,
feature_type) + per-body dedup, check the per-stem cap, retry
up to 3 times with a bumped seed.
5. On persistent failure, fall back to a deterministic palette
generator so every feature ends up with a name.
6. Write markers.json atomically and refresh atlas_* DB rows via
sync_markers_to_db. Commit the DB per body so a crash loses
at most one body of state.
7. Restart the sr-voice subprocess every `--refresh` requests
(default: 200) to prevent KV-cache context bleed.
Core design decisions:
- Determinism: per-(world_seed, body_id, feature_local_id, attempt)
seed so the full run is reproducible.
- Ordering: bodies are processed in ascending `hop_distance_from_gateway`
so core bodies get first pick at every unique Gemma output and
outer sectors fall into the palette fallback when they lose the
dedup race.
- Dedup scope: (cultural_corridor, feature_type) across the run,
PLUS a per-body cross-type set so the same name can't be a river
AND an ocean AND a mountain on the same world. Hand-authored names
are seeded into both sets on load so templates win priority.
- Stem cap: each non-generic root token (e.g. 'Arcturus', 'Meridian')
may appear at most `--stem-cap` times across the full run (default
20), preventing single-word runaway. Fallback names bypass the cap.
- Earth blocklist: 181 curated entries covering major Earth cities,
mountains, rivers, oceans, historical/colonial spellings, and
Greek/Roman mythology that reads too literally. Prefixed variants
('Nouveau Paris', 'New Tokyo') explicitly allowed per the product
intent that Earth-echo names are fine but must not dominate.
Leading 'The ' is stripped before comparison so 'The Great Divide'
also matches.
Operational features:
- `--shard N/M` slices the body list into M partitions for parallel
runs. Two terminals × `--shard 0/2` + `--shard 1/2` fits the
~2.5 GB/instance VRAM footprint twice under the 50% cap on a
16 GB AMD GPU and roughly halves wall time.
- `--log PATH` writes a timestamped tee of every status line to a
file. Default: `.tmp/gemma_naming.shard{N}of{M}.log` when a
non-trivial shard is in use.
- SQLite `PRAGMA journal_mode=WAL` + `busy_timeout=15000` so two
concurrent shards serialize writes without lock errors.
- Per-body progress lines report `body K/N`, `sys K/N`, and
`hop=H` so the user can watch core sectors finish first.
- Each body logs the new names it produced per feature type so the
user can eyeball quality as the run progresses.
- Checkpoint summary every 25 bodies: cumulative names, rate,
ETA — gives the log regular scroll points.
- `--mock` uses `server/sr-voice/mock-stdio.sh` for dry-fire
pipeline validation without a model load (tested end-to-end).
Supporting files:
- `tooling/planet-gen/earth_blocklist.txt` — 181 curated entries.
- `tooling/db/backfill_cultural_corridor.py` — one-off migration
that fills the `cultural_corridor` column on both `star_systems`
and `bodies` from the `geographic_sector` values. Before this
pass, 99.4% of rows (3221/3240) had a NULL cultural_corridor
despite `wiki_sync.py` being aware of the column — the wiki
index.md files only carry the sector header, which was never
propagated to the DB column. Idempotent, safe to re-run after
any wiki_sync rebuild, explicit transaction wrapper with
rollback on failure.
Full batch runtime estimate: ~20 hours single-shard / ~10 hours
double-shard on this hardware. Smoke tests across five hardened
iterations (v1–v5) on GJ71b/c/d/d-1/e confirm the pipeline produces
clean, varied, culturally-coherent names with zero post-processing
residue.
- ON DELETE CASCADE added to every atlas_* foreign key (atlas_body_grids,
atlas_cities, atlas_roads, atlas_railroads, atlas_pois, atlas_rivers,
atlas_oceans, atlas_mountain_ranges). Previously, deleting a body from
the bodies table or NULL-ing its terrain_reference would leave orphan
atlas rows forever — sync_markers_to_db only cleans up for bodies it
re-processes. The existing atlas tables in systems.db were dropped and
recreated with the new constraint; FK list now reports CASCADE.
- Atlas DDL deduplicated. systems-schema.sql is now the single source of
truth, bracketed by `-- BEGIN ATLAS INDEX` / `-- END ATLAS INDEX`
markers. generate_atlas.py reads that block via `_load_atlas_schema()`
and applies it at runtime, so there is no second copy of the DDL to
keep in sync. Adding a column requires one edit, not two.
- Uniqueness guard on city coordinates. `_enforce_unique_city_coords`
runs at the end of `place_cities` and deterministically perturbs any
duplicate (row, col) via a fixed spiral walk to the first free
walkable land cell. Rare in practice but the MST collapses to a
zero-distance edge otherwise, producing an empty A* path and silently
dropping the road.
- Grid header validation. `load_markers` now raises `AtlasGridMismatch`
if the loaded `grid: {w, h}` header does not match `GRID_W`/`GRID_H`.
Both the incremental-skip path and the regenerate path route through
this loader, so a hand-authored template shipping a different grid
size fails loud with a per-body error rather than silently producing
half-scale coordinates.
- Unused `seed_rng` parameter removed from `_analyse_terrain`. The
function is RNG-free (continent flood-fill, habitability scoring,
river-mouth dedup, cost grid — all pure functions of terrain). The
false API contract made it look like terrain analysis consumed RNG
state and had to be sequenced with downstream RNG use.
- `_score_capital_sites` river-mouth bonus now builds one sparse
accumulator with all mouth points set at once and runs a single
`gaussian_filter` call, instead of O(n_mouths) filter calls over
single-point images.
- `binary_dilation(analysis["land_mask"] == False)` replaced with the
idiomatic `~analysis["land_mask"]`, matching the convention used
elsewhere in the file.
- `atlas-generate` Makefile target now guards on
`SELECT COUNT(*) FROM bodies WHERE terrain_reference IS NOT NULL`.
On a fresh DB that count is 0 and the generator previously exited
"success" after processing zero bodies. The target now fails loud
with a pointer to `populate_terrain_reference.py`.
- `main.rs` SimRng defensive re-insertion gains a long comment
explaining the exact plugin-ordering hazard it guards against, so
future readers don't treat the line as dead code. Tied to #826.
Implements the Phase 3 atlas content generator per D-191 §3, §8, and §9.
Pipeline per body (terrain-aware, deterministic per seed + body):
1. Simulate terrain via planet_simulation.simulate().
2. Analyse continents (flood-fill), habitability (temp/moisture/slope +
coastal bonus), river mouths, and a terrain A* cost grid.
3. Place cities sequentially — capital first (habitability + river-mouth
bias), then corridor growth via multi-source Dijkstra, quadrant-spread
penalty after 2 cities in a quadrant, port-on-new-continent bonus at
cities 3–4. ±25% noise for seed variation.
4. Generate roads and railroads as an MST over city positions, with
A* paths on the terrain cost grid (rail follows roads where possible).
5. Place a transit POI at the capital (15% chance to scatter to a
secondary city).
Output (canonical markers.json schema, pixel space per D-191 §8):
- cities: {id, name, kind, center:[r,c], population}
- roads: {id, name, kind, path:[[r,c],...]}
- railroads: {id, name, kind, path:[[r,c],...]}
- pois: {id, name, kind, center:[r,c]}
- existing rivers/oceans/mountain_ranges preserved untouched.
City names are left empty for gemma_naming.py (#833). Body population is
split across cities with geometric decay (capital ~50%, each subsequent
city half the previous). The 6 hand-authored bodies (Lendel, Edict,
Vuurkloof, Røros, Cairnside, Estrade) are detected by existing
`cities` and skipped for regeneration; their markers are still synced
to the DB index below.
Atlas index in systems.db (new):
- atlas_body_grids, atlas_cities, atlas_roads, atlas_railroads,
atlas_pois, atlas_rivers, atlas_oceans, atlas_mountain_ranges
- Scalar metadata mirror of every markers.json — the implant atlas app
and development queries can lookup cities/POIs/features without
scanning 267 JSON files. Polyline geometry stays in the markers.json
files next to the heightmaps (used by the renderer); the DB only
stores filterable scalar fields plus `point_count` as a length proxy.
- Schema lives in server/data/systems-schema.sql; generate_atlas.py
mirrors the CREATE TABLE IF NOT EXISTS block so it runs against any
DB state (matches the economy-db importer pattern).
- Populated and refreshed on every run. Each body's rows are deleted
and reinserted deterministically — no stale state.
Also fixes a pre-existing WIP bug in the quadrant-saturation penalty
loop (a stray outer `for r in range(GRID_H)` with unreachable breaks
meant only the NW quadrant was ever checked).
Runtime: 280s for all 267 inhabited bodies on a single core. 265 bodies
updated this run, 6 hand-authored bodies synced to DB without
regeneration.
Atlas index after run:
atlas_cities 329 (15 hand-authored + 314 awaiting #833)
atlas_roads 46
atlas_railroads 44
atlas_pois 287
atlas_rivers 2034
atlas_oceans 696
atlas_mountain_ranges 1953
atlas_body_grids 267
Adds tooling/planet-gen/populate_terrain_reference.py and runs it against
systems.db. Resolves each body's expected wiki heightmap path (repo-root
relative) and writes it into bodies.terrain_reference. Missing heightmaps
are logged for remediation.
Result: 2380/3240 bodies populated, 860 still missing heightmaps. This
unblocks generate_atlas.py (#832) for every body that has a heightmap.
Adds the brand layer per D-189 §5:
- Schema: brand_products, brand_inputs, system_fiscal, corp_financial_state,
corp_lifecycle_events (+ 5 indexes).
- Importer: reads wiki/economics/corporations/brands.toml, populates the
new tables, validates V-B01–V-B05 structural rules, and derives
system_fiscal for inhabited systems.
- Data: 8 brand_products, 16 brand_inputs, 301 system_fiscal rows.
Brand products are demand nodes — they consume commodities; they are not
commodities themselves (D-185). Depends on copy PR #127 for the corp
records referenced by brands.toml.
Add D-180 EconEvent struct (target, effect, duration, visibility enums)
with no-op handler to satisfy #809 spec. Import MARK_PRIMARY and MIXED
currency zone assignments from wiki/economics/currency_zones.toml (D-172).
All four D-179 stability tests now pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends import_economics.py from 6-step to 8-step pipeline:
- Loads wiki corporation markdown frontmatter as authoritative source
- Syncs corporations table (hard error on proper_name divergence per D-182)
- Populates corp_presence table (one row per corp × headquarters system)
- Splits validation: structural checks block commit; coverage checks post-commit
- D-175 Phase 2 gate: 3+ corps per commodity, 1+ corp per system with pop > 100K
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 5 new tables: gate_links (668 bidirectional edges from star-map.json),
commodities (36 types from commodities.toml), production_chains (21
Leontief recipes), chain_inputs (52 input requirements), corp_presence
(empty, populated by future pipeline). Add currency_zone column to
star_systems (D-172), archetype columns to corporations (D-175).
New import pipeline: tooling/economy-db/import_economics.py reads
TOML/JSON source files and populates the DB. Idempotent — safe to
rerun via `make economy-db`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
"Biome" describes per-zone vegetation classification (Whittaker table).
"Planet class" describes overall planetary character. The conflation
caused the planet generator to misclassify ~270 bodies as barren.
Scope: systems.db column, schema SQL, Rust atlas code, wiki table
headers (Biome → Class), atlas proposal JSONs, all docs/decisions,
tooling scripts. Also normalizes atmosphere vocabulary (breathable →
standard) and expands planet class mapping to all 26 wiki values.
Unknown classes default to temperate for modder safety.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
20 changes across 18 systems implementing the approved remediation plan
from the cultural diversity audit (docs/design/cultural-diversity-audit.md).
DB changes (15 bodies/stations in systems.db):
- Thai bodies: Saikaew (GJ 635B), Nakhon Sung (GJ 667C), Doi Kaeo (GJ 117)
- Khmer bodies: Banteay (GJ 434), Srok Thmei (GJ 866A)
- Ukrainian: Chornozem body + Svoboda station (GJ 729)
- Caribbean: Lavenille body (GJ 873), Accompong Works station (GJ 406)
- Malay: Pulau Baru body (GJ 555), Balai station (GJ 449)
- Scientists: Huygens Platform (GJ 251), Ramanujan Station (GJ 699),
van Leeuwenhoek Station (GJ 820B), Turing Survey (GJ 663A)
Wiki calibration note additions:
- GJ 635B/667C/117/434: Thai cultural presence (survey teams, founding cohorts)
- GJ 866A/873: Khmer/Caribbean community naming origin
- GJ 729: Ukrainian founding cooperative in west_reach
- GJ 1245A/75: Igbo/Nigerian heritage explicitly acknowledged
- GJ 172: Malay kelong dual-heritage in Kaelong name
- GJ 601A: Russian secondary founding stream in Ostmark
Horizon stations not renamed. No corridor reassignments.
Conservative: systems with existing identity untouched.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. GJ 111: spectral_class F5V → F5/F6V to match wiki (subgiant)
2. GJ 3943: spectral_class k-m → K5V+M3V (valid MK notation for binary)
3. GJ 903: write wiki prose — was empty stub with only body tables
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unsettled systems receive their GJ designation as proper_name, except
GJ 868 which is named "Encontro" per wiki lore (informal Portuguese
corridor name). 24 proposals updated, systems.db fully synced.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
commit-system was inserting bodies/stations but never writing proper_name
to star_systems. Bulk-updated all 276 authored system names. Fixed duplicate
body_id collisions in GJ 1156, GJ 139, GJ 34A, GJ 34B, GJ 661A, GJ 664,
GJ 780 (planet and gas giant sharing same letter suffix).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Also includes star_type DB fixes and proposal file updates from
the west_reach authoring session.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
systems.db: 43 North Reach body catalogs (hops 4-12)
tooling: atlas-verify, atlas-names, atlas-systems-done, atlas-helpers.sh
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GJ 702A was a deprecated binary system entry. Replace with GJ 1221,
a real catalog star — DXP9 peculiar magnetic white dwarf at 20.03 ly.
Dead-end topology connecting to EZ Aqr (GJ 866A).
Updates: star-map.json, systems.db, gj-catalog-real.json, catalog.md,
EZ Aqr topology reference. New wiki entry and Drifter's Guide entry.
System count restored to 301.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gate builders don't place two horizon stations in one star system.
Binary pairs (A/B components of same physical star) split to
independent stars. Each "move" system keeps its connections and
content, only the host star changes.
Migrations:
- GJ 559A (Rigil Kentaurus) -> GJ 768 (Altair) — rewritten for new star
- GJ 725A (Renaissance) -> GJ 251
- GJ 820A -> GJ 3877 (Tessera — compact/records world)
- GJ 570B -> GJ 1156
- GJ 783B -> GJ 299
- GJ 663B (Guniibuu B) -> GJ 1286 (Kensho — university world)
- GJ 338A -> GJ 4053
- GJ 280B -> GJ 3991
- GJ 15B -> GJ 4274
New system names and roles assigned:
- Tessera: founding compact + institutional records
- Kensho: university world, largest Institute presence
- Vargr: mining (DSMC, MVG/GC minority stakes)
- Parallax: media world (was Procyon)
- Lu Ban: failed corp / mixed industrial (was Groombridge 34)
- Altair: southern gateway (was Rigil Kentaurus)
- Keid: garden world / ecological reservation
- Ross 128: ocean world (fishing, research, tourism)
Also adds docs/design/gj-catalog-full.json (3801 GJ entries from
HYG v4.2) for future star assignment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces flat star_systems table with 7 normalized tables: star_systems,
system_gates (with hop_distance_from_gateway), system_history,
historical_events (1:N), system_economy, system_factions, system_culture.
Primary key is now the GJ astronomical ID (e.g., "GJ 244A") — S-number
identifiers removed entirely. The astronomical_id column is dropped as
redundant. Database moved to server/data/systems.db (ships with game).
10 core systems fully populated with structured data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
