- Add docs/gemma-naming-methodology.md preserving the corridor-aware LLM place-naming approach (sector palettes, two-stage register selection + generation, few-shot prompting, KV-cache refresh, dedup, Earth-major blocklist, deterministic fallback) as institutional knowledge after the pipeline's retirement. - Add an implementation-status note to D-223 recording what #951 did (generator + naming cluster retired, import_economics owns the atlas index, population deferred to #955, Sol exempt, dup bug fixed). - CHANGELOG entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
133 lines
7.6 KiB
Markdown
133 lines
7.6 KiB
Markdown
# Gemma Naming Methodology (institutional knowledge)
|
||
|
||
**Status:** the implementing code (`gemma_naming.py`, `naming_core.py`, and the
|
||
`test_batch_naming.py` / `qa_naming.py` harnesses) was **retired in #951 (D-223)**
|
||
when authored markers became a names-only flavoured pool and the per-feature
|
||
geometry generator was retired. This document preserves *how* the Reach's place
|
||
names were generated so the approach can be rebuilt for the names-only format if
|
||
new bodies ever need fresh names. The names it produced are now the frozen pool
|
||
in each `markers.json` (`names.{rivers,mountain_ranges,oceans,cities,pois,...}`).
|
||
|
||
The pipeline named every empty `name` field across ~2,400 bodies' markers.json
|
||
files (cities, rivers, oceans, mountain ranges, gate terminals, landmarks) with
|
||
culturally-coherent, corridor-appropriate names — using a small local LLM, not a
|
||
hand-written name table.
|
||
|
||
---
|
||
|
||
## 1. Model & runtime
|
||
|
||
- **Model:** Gemma 2 2B (`gemma2.gguf`), later `gemma-4.gguf` — a *small* local
|
||
GGUF model, chosen so the whole Reach could be named offline on commodity
|
||
hardware (CPU fallback, ROCm/CUDA when available).
|
||
- **Serving:** `sr-voice serve --stdio` — a long-lived subprocess fed prompts over
|
||
stdin, replies over stdout. The same binary backs the in-game voice pipeline.
|
||
- **KV-cache bleed is the enemy.** A long-lived model accumulates context across
|
||
requests and starts echoing earlier completions (every river becomes "Aldren").
|
||
The subprocess was **restarted every `--refresh` requests** to flush the cache.
|
||
This single knob mattered more than any prompt tweak for output diversity.
|
||
|
||
## 2. Two-stage flow
|
||
|
||
1. **Register selection** (`select_register`) — for each star system, Gemma is
|
||
shown the corridor's candidate cultural sub-styles (numbered) plus a compact
|
||
cultural excerpt from the system's wiki + GTTR text, and asked to **pick the
|
||
number** of the best-fitting register. This grounds naming in the *authored*
|
||
cultural identity rather than a blind hash. Falls back to `palette_for`
|
||
(hash-based pick) if the model fails or there's no cultural text.
|
||
2. **Name generation** (`_build_prompt`) — generate names in the selected
|
||
register, per feature, with retries and dedup.
|
||
|
||
## 3. Corridor palettes — the cultural-design crux
|
||
|
||
Names are biased by **geographic sector** (the real corridor column in
|
||
`systems.db` is `star_systems.geographic_sector`). Each sector has a list of
|
||
**sub-style inflections**, each with a register description and ~5 example stems.
|
||
A sub-style is picked per system (via `hash(system_id)`) so neighbouring systems
|
||
rotate registers and the model's narrow ~15-stem vocabulary stays fresh across
|
||
hundreds of bodies.
|
||
|
||
> **Inflection is a dominant bias, not a hard lock.** A British surveyor on an
|
||
> east_reach moon still names a river after their aunt in Dorset. Each sub-style
|
||
> explicitly names its register *and invites diaspora variety.* This is what keeps
|
||
> the Reach feeling like blended-reality settlement rather than themed zones.
|
||
|
||
| Sector | Sub-style registers |
|
||
|--------|---------------------|
|
||
| `core` | English countryside · British colonial · American frontier · American municipal · Classical/civic · ANZ settler |
|
||
| `north_reach` | English rural/parish · Scottish Highland/Lowland · Australian outback · Irish coastal · South African English |
|
||
| `south_reach` | Portuguese colonial/Iberian · Brazilian interior · East African Swahili · Cape Verdean/West African · Angolan/Mozambican |
|
||
| `east_reach` | Korean · Japanese rural/coastal · Taiwanese/Hakka · Filipino · Mixed East Asian diaspora |
|
||
| `west_reach` | German compound · Dutch low-country · Nordic/Scandinavian · Polish/Czech · Baltic/Finnish |
|
||
| `deep_frontier` | Founder-surname · Surveyor-descriptive · Functional/military outpost |
|
||
|
||
Legacy aliases mapped to `core` (`sol-gateway-axis`, `inner_corridor`,
|
||
`inner_orbit`) and `deep_frontier` (`frontier`). Cross-cultural names in every
|
||
direction are expected and correct (see the corridor cultural-mixing principle).
|
||
|
||
## 4. Few-shot prompting (the model-fit lesson)
|
||
|
||
> Gemma 2 2B is **far better at pattern completion than instruction following.**
|
||
|
||
So prompts were *worked examples*, not instructions:
|
||
- Show **2 examples from *different* corridors than the target** (teach the
|
||
*pattern* — "system description → register number" or "register → place name" —
|
||
without biasing toward the target's vocabulary), then present the target and let
|
||
the model complete.
|
||
- Register selection used a fixed preamble with two `System: … → Best: N` examples,
|
||
then the target system, then `Best (number only):`.
|
||
- Per-feature generation rotated through example **pools** picked deterministically
|
||
by `hash(body_id, local_id, attempt)` so neighbouring features on one body don't
|
||
all draw the same prompt and collapse to identical outputs.
|
||
|
||
## 5. Context extraction (fitting 1024 tokens)
|
||
|
||
Gemma 2 2B has a ~1024-token context, so the authored cultural signal had to be
|
||
compressed hard:
|
||
- **GTTR hook** — the 30–45 word "Drifter's Guide" characterisation of the system
|
||
is the single biggest lever for names that feel like *this* world. Title lines
|
||
and section headers were stripped; the first substantive paragraph was used.
|
||
- **Wiki cultural lines** — `_extract_cultural_lines` scanned for cultural-identity
|
||
keywords (heritage, founding, settler, surname, language, diaspora, and explicit
|
||
culture names) and kept the strongest hits, falling back to opening prose.
|
||
- Budget math reserved tokens for preamble + tail + output; context was truncated
|
||
to fit.
|
||
|
||
## 6. Post-processing, retries, dedup, fallback
|
||
|
||
- **Deterministic seeds:** `seed = sha256("{role}|{id}|{attempt}").hexdigest()[:8]`
|
||
— reproducible, and bumping `attempt` rotates the completion on retry.
|
||
- **Earth-major blocklist:** `earth_blocklist.txt` rejected real Earth majors
|
||
(Paris, Tokyo, …). Earth-*echo* names are fine; Earth *majors* are not.
|
||
- **Retry:** on collision or blocklist hit, retry with a bumped seed, up to 3
|
||
attempts.
|
||
- **Deterministic fallback:** persistent failure fell back to a palette-driven
|
||
stem+suffix name (`fallback_name`, `_FALLBACK_STEMS`/`_FALLBACK_SUFFIXES`) so the
|
||
pipeline always produced *something* valid.
|
||
- **Dedup scope:** within `(geographic_sector, feature_type)` — no two bodies in
|
||
the same sector ship the same river name; **cross-corridor collisions are
|
||
allowed** (two "Aldren"s on opposite arcs is fine). Processing ran **core-first**
|
||
(`SECTOR_PRIORITY`) so core bodies won the dedup race and outer sectors took the
|
||
fallback path on collision.
|
||
|
||
## 7. Per-feature prompt config
|
||
|
||
Each feature type had its own subject framing, length hint, and example pools
|
||
(`_PROMPT_CONFIG`): rivers/oceans/seas/lakes/mountain ranges → "1–3 words";
|
||
capital → "1–2 words" (+ planet context); secondary towns → "1–2 words";
|
||
gate terminals → "2–3 words ending in 'Gate Terminal'/'Transit'/'Exchange'/
|
||
'Concourse'"; institutional & cultural landmarks → "2–4 words". Capitals and
|
||
secondary towns included planet context so town names sat under the world's name.
|
||
|
||
## 8. If you rebuild this for names-only markers
|
||
|
||
The data model changes: there are no longer per-feature records with empty `name`
|
||
fields to fill — the target is the **flat name pool** per body
|
||
(`names.cities`, `names.rivers`, …). A rebuild would:
|
||
1. Decide pool sizes per body (how many city/river/etc. names to mint).
|
||
2. Keep stages 1–6 verbatim — register selection, corridor palettes, few-shot
|
||
completion, seed/dedup/blocklist/fallback are all format-independent.
|
||
3. Write strings into `names.<category>` lists instead of into feature records,
|
||
and drop the atlas-DB sync entirely (the server cascade attaches pooled names
|
||
to computed features at placement — D-223, #955).
|