diff --git a/CHANGELOG.md b/CHANGELOG.md index d72967b2c..b468b13eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,9 +16,12 @@ Format based on [Keep a Changelog](https://keepachangelog.com/). - **Regional atlas wiki loading** — heightmaps and markers load directly from the wiki filesystem, fixing "TERRAIN DATA PENDING" for all bodies with authored heightmaps - **Agent-team coordination tools** — `SendMessage` plus `TaskList`/`TaskUpdate`/`TaskGet` added to all 21 agent definitions so custom subagents work as full teammates in tmux split panes (message the lead, claim/complete tasks, shut down cleanly); `TaskCreate` deliberately withheld to keep task creation with the lead - **`git-centralize-guard` hook** — PreToolUse hook blocking `.git`-mutating commands for agent-team teammates, keying on the `agent_type` field present only in teammate hook input; version control stays centralized to the team lead +- **Gemma naming methodology doc** (`docs/gemma-naming-methodology.md`) — preserves the corridor-aware LLM place-naming approach (sector palettes, few-shot prompting, dedup, Earth-major blocklist, deterministic fallback) as institutional knowledge after the naming pipeline's retirement ### Changed - `teammateMode` set to `auto` — teammates spawn in tmux split panes when the lead runs inside tmux, with graceful in-process fallback +- **markers.json → names-only flavored pool** (D-223, #951) — all 2,398 bodies carry place *names* only, no positions or geometry; river/mountain positions derive from the heightmap + drainage, settlement positions from the economic sim + placement, with pooled names attached at generation time. The six hand-authored "templates" dissolve into ordinary named bodies +- **`import_economics` owns the atlas index** (#951) — now the sole `regen-db` generator that touches the atlas tables; loads the names pool into `atlas_city_names` and empties the geometry tables, which the Phase 4 server cascade fills ### Fixed - Godot 4.6 type inference parse errors in `main.gd` (`var zoom`, `var speed`) that prevented the main scene from loading @@ -26,6 +29,7 @@ Format based on [Keep a Changelog](https://keepachangelog.com/). ### Removed - Sprint CLI (`tooling/db/sprint`), sprint skills (`/sprint-start`, `/sprint-plan`, `/sprint-status`), team-scoped file restrictions, sprint briefing workflow +- **Atlas geometry generator + LLM naming pipeline** (#951, D-223) — `generate_atlas.py`, `gemma_naming.py`, `naming_core.py`, `apply_name_fixes.py`, the redundant `import_city_names.py`, their tests/QA/runner scripts, and the `fix_fewshot_bleed`/`prune_atlas_features` geometry tools. Procedural placement and authored names are superseded by the deterministic server cascade and the frozen name pool ## [v0.2.0] — 2026-05-03 diff --git a/decisions/architecture.md b/decisions/architecture.md index f04be03d3..7bb9b607c 100644 --- a/decisions/architecture.md +++ b/decisions/architecture.md @@ -1430,6 +1430,7 @@ Technical foundation decisions that constrain implementation: engine, client-ser - **Rationale:** Authored positions created a hand/procedural split that (per D-207's own rationale) was impossible to query, diff, or validate. Reducing authored content to a name pool removes that split entirely while preserving the flavor that makes places feel hand-made — names follow culture and region, geometry follows terrain and economics. The templates were leftover scaffolding from before fully-generative placement; removing them simplifies the pipeline without losing any canonical place. - **Implementation:** tracked under Phase 4 (epic #750) — strip markers to names, remove template machinery + `reserved` pinning, update the atlas pipeline/schema, regen. Historical archives (sprints, workshops, audits, CHANGELOG, atlas proposals) are left untouched. +- **Implementation status (#951, 2026-05-22): done.** All 2,398 `markers.json` reduced to a names-only pool (25,264 names; 349 city names across 271 bodies). The Python atlas **geometry generator** (`generate_atlas.py`) and the **LLM naming cluster** (`gemma_naming.py`, `naming_core.py`, `apply_name_fixes.py`, their tests/QA, the `fix_fewshot_bleed`/`prune_atlas_features` geometry tools, the redundant `import_city_names.py`, and the `run-atlas-naming.sh` runner) were retired — the procedural server cascade (Phase 4) supersedes them. The Gemma prompting methodology is preserved in [docs/gemma-naming-methodology.md](../docs/gemma-naming-methodology.md). Shared atlas-DB utilities moved to `tooling/planet-gen/atlas_common.py`; **`import_economics.py` is now the sole regen-db generator that owns the atlas index** — it loads the names pool into `atlas_city_names` and empties the 8 geometry tables (`atlas_cities/roads/railroads/pois/rivers/oceans/mountain_ranges/body_grids`), which the cascade fills. `population`/`kind`/`settlement_class` on `atlas_city_names` are **deferred to placement (#955)** — `attractor_matching` reads them at 0/default until then; that empty state is the gap the cascade closes. The `reserved=1` corp-HQ cross-reference stays (corp HQ names are real places). A latent duplicate-accumulation bug in name population (no clear + no unique constraint) was fixed with a deterministic rebuild. **Sol (system `GJ 0`) is permanently exempt from the normal generators:** it uses real Earth/Mars/Luna geography via the offline `sol_import.py` (left in place for future scripted integration), so its bodies keep geometry-bearing `markers.json` as preserved positional config and are skipped by the names-pool importer — Sol names will come from its own integration, not the cascade. `make regen-db` green. - **Raised by:** Jeroen, 2026-05-22 — resolving the open question on merging preconfigured content into the deterministic cascade. - **Cross-reference:** [D-207](#d-207-fully-generative-placement--markersjson-stripped-to-topographic-features) (superseded — names-only, no reserved pinning), [D-191](#d-191) §8 (markers format — names-only), [D-208](#d-208) (drainage → river courses), [D-211](#d-211) (settlement placement), [D-199](#d-199) (economic read set), [D-222](#d-222) (lore≠code names on generated geometry) - **Dissent:** None diff --git a/docs/gemma-naming-methodology.md b/docs/gemma-naming-methodology.md new file mode 100644 index 000000000..640a2579a --- /dev/null +++ b/docs/gemma-naming-methodology.md @@ -0,0 +1,132 @@ +# Gemma Naming Methodology (institutional knowledge) + +**Status:** the implementing code (`gemma_naming.py`, `naming_core.py`, and the +`test_batch_naming.py` / `qa_naming.py` harnesses) was **retired in #951 (D-223)** +when authored markers became a names-only flavoured pool and the per-feature +geometry generator was retired. This document preserves *how* the Reach's place +names were generated so the approach can be rebuilt for the names-only format if +new bodies ever need fresh names. The names it produced are now the frozen pool +in each `markers.json` (`names.{rivers,mountain_ranges,oceans,cities,pois,...}`). + +The pipeline named every empty `name` field across ~2,400 bodies' markers.json +files (cities, rivers, oceans, mountain ranges, gate terminals, landmarks) with +culturally-coherent, corridor-appropriate names — using a small local LLM, not a +hand-written name table. + +--- + +## 1. Model & runtime + +- **Model:** Gemma 2 2B (`gemma2.gguf`), later `gemma-4.gguf` — a *small* local + GGUF model, chosen so the whole Reach could be named offline on commodity + hardware (CPU fallback, ROCm/CUDA when available). +- **Serving:** `sr-voice serve --stdio` — a long-lived subprocess fed prompts over + stdin, replies over stdout. The same binary backs the in-game voice pipeline. +- **KV-cache bleed is the enemy.** A long-lived model accumulates context across + requests and starts echoing earlier completions (every river becomes "Aldren"). + The subprocess was **restarted every `--refresh` requests** to flush the cache. + This single knob mattered more than any prompt tweak for output diversity. + +## 2. Two-stage flow + +1. **Register selection** (`select_register`) — for each star system, Gemma is + shown the corridor's candidate cultural sub-styles (numbered) plus a compact + cultural excerpt from the system's wiki + GTTR text, and asked to **pick the + number** of the best-fitting register. This grounds naming in the *authored* + cultural identity rather than a blind hash. Falls back to `palette_for` + (hash-based pick) if the model fails or there's no cultural text. +2. **Name generation** (`_build_prompt`) — generate names in the selected + register, per feature, with retries and dedup. + +## 3. Corridor palettes — the cultural-design crux + +Names are biased by **geographic sector** (the real corridor column in +`systems.db` is `star_systems.geographic_sector`). Each sector has a list of +**sub-style inflections**, each with a register description and ~5 example stems. +A sub-style is picked per system (via `hash(system_id)`) so neighbouring systems +rotate registers and the model's narrow ~15-stem vocabulary stays fresh across +hundreds of bodies. + +> **Inflection is a dominant bias, not a hard lock.** A British surveyor on an +> east_reach moon still names a river after their aunt in Dorset. Each sub-style +> explicitly names its register *and invites diaspora variety.* This is what keeps +> the Reach feeling like blended-reality settlement rather than themed zones. + +| Sector | Sub-style registers | +|--------|---------------------| +| `core` | English countryside · British colonial · American frontier · American municipal · Classical/civic · ANZ settler | +| `north_reach` | English rural/parish · Scottish Highland/Lowland · Australian outback · Irish coastal · South African English | +| `south_reach` | Portuguese colonial/Iberian · Brazilian interior · East African Swahili · Cape Verdean/West African · Angolan/Mozambican | +| `east_reach` | Korean · Japanese rural/coastal · Taiwanese/Hakka · Filipino · Mixed East Asian diaspora | +| `west_reach` | German compound · Dutch low-country · Nordic/Scandinavian · Polish/Czech · Baltic/Finnish | +| `deep_frontier` | Founder-surname · Surveyor-descriptive · Functional/military outpost | + +Legacy aliases mapped to `core` (`sol-gateway-axis`, `inner_corridor`, +`inner_orbit`) and `deep_frontier` (`frontier`). Cross-cultural names in every +direction are expected and correct (see the corridor cultural-mixing principle). + +## 4. Few-shot prompting (the model-fit lesson) + +> Gemma 2 2B is **far better at pattern completion than instruction following.** + +So prompts were *worked examples*, not instructions: +- Show **2 examples from *different* corridors than the target** (teach the + *pattern* — "system description → register number" or "register → place name" — + without biasing toward the target's vocabulary), then present the target and let + the model complete. +- Register selection used a fixed preamble with two `System: … → Best: N` examples, + then the target system, then `Best (number only):`. +- Per-feature generation rotated through example **pools** picked deterministically + by `hash(body_id, local_id, attempt)` so neighbouring features on one body don't + all draw the same prompt and collapse to identical outputs. + +## 5. Context extraction (fitting 1024 tokens) + +Gemma 2 2B has a ~1024-token context, so the authored cultural signal had to be +compressed hard: +- **GTTR hook** — the 30–45 word "Drifter's Guide" characterisation of the system + is the single biggest lever for names that feel like *this* world. Title lines + and section headers were stripped; the first substantive paragraph was used. +- **Wiki cultural lines** — `_extract_cultural_lines` scanned for cultural-identity + keywords (heritage, founding, settler, surname, language, diaspora, and explicit + culture names) and kept the strongest hits, falling back to opening prose. +- Budget math reserved tokens for preamble + tail + output; context was truncated + to fit. + +## 6. Post-processing, retries, dedup, fallback + +- **Deterministic seeds:** `seed = sha256("{role}|{id}|{attempt}").hexdigest()[:8]` + — reproducible, and bumping `attempt` rotates the completion on retry. +- **Earth-major blocklist:** `earth_blocklist.txt` rejected real Earth majors + (Paris, Tokyo, …). Earth-*echo* names are fine; Earth *majors* are not. +- **Retry:** on collision or blocklist hit, retry with a bumped seed, up to 3 + attempts. +- **Deterministic fallback:** persistent failure fell back to a palette-driven + stem+suffix name (`fallback_name`, `_FALLBACK_STEMS`/`_FALLBACK_SUFFIXES`) so the + pipeline always produced *something* valid. +- **Dedup scope:** within `(geographic_sector, feature_type)` — no two bodies in + the same sector ship the same river name; **cross-corridor collisions are + allowed** (two "Aldren"s on opposite arcs is fine). Processing ran **core-first** + (`SECTOR_PRIORITY`) so core bodies won the dedup race and outer sectors took the + fallback path on collision. + +## 7. Per-feature prompt config + +Each feature type had its own subject framing, length hint, and example pools +(`_PROMPT_CONFIG`): rivers/oceans/seas/lakes/mountain ranges → "1–3 words"; +capital → "1–2 words" (+ planet context); secondary towns → "1–2 words"; +gate terminals → "2–3 words ending in 'Gate Terminal'/'Transit'/'Exchange'/ +'Concourse'"; institutional & cultural landmarks → "2–4 words". Capitals and +secondary towns included planet context so town names sat under the world's name. + +## 8. If you rebuild this for names-only markers + +The data model changes: there are no longer per-feature records with empty `name` +fields to fill — the target is the **flat name pool** per body +(`names.cities`, `names.rivers`, …). A rebuild would: +1. Decide pool sizes per body (how many city/river/etc. names to mint). +2. Keep stages 1–6 verbatim — register selection, corridor palettes, few-shot + completion, seed/dedup/blocklist/fallback are all format-independent. +3. Write strings into `names.` lists instead of into feature records, + and drop the atlas-DB sync entirely (the server cascade attaches pooled names + to computed features at placement — D-223, #955).