jpmschweitzer
1efbbcaea9
New end-to-end pipeline that walks every markers.json in the reach and
fills empty `name` fields using the Gemma 2 voice pipeline via
`sr-voice serve --stdio`. Per D-191 §4: the same Gemma 2 pipeline the
client uses for NPC voicing also produces the atlas content, which is
dual-purposed as a quality test of the LLM plumbing.
Pipeline per body (hop-ordered, core-first):
1. Load markers.json; identify feature records whose `name` is
blank (null or ""). Hand-authored names are never overwritten;
the 6 template bodies and any partial authoring stay put.
2. Look up body context (planet_class, settlement_pattern,
cultural_corridor, population, economic_role) from systems.db.
3. Build a short corridor-aware few-shot prompt per feature type.
Prompts carry 3 concrete `Style: X. Answer: Y` examples so
Gemma 2 2B completes a pattern instead of generating to an
open-ended instruction — this is the single biggest lever
against placeholder echoes on a small model.
4. Stream the prompt into a long-lived sr-voice subprocess, read
the JSONL response, post-process (strip markdown, label
prefixes, brackets, reject 5+ word outputs and placeholder
tokens), check the earth-name blocklist, check per-(corridor,
feature_type) + per-body dedup, check the per-stem cap, retry
up to 3 times with a bumped seed.
5. On persistent failure, fall back to a deterministic palette
generator so every feature ends up with a name.
6. Write markers.json atomically and refresh atlas_* DB rows via
sync_markers_to_db. Commit the DB per body so a crash loses
at most one body of state.
7. Restart the sr-voice subprocess every `--refresh` requests
(default: 200) to prevent KV-cache context bleed.
Core design decisions:
- Determinism: per-(world_seed, body_id, feature_local_id, attempt)
seed so the full run is reproducible.
- Ordering: bodies are processed in ascending `hop_distance_from_gateway`
so core bodies get first pick at every unique Gemma output and
outer sectors fall into the palette fallback when they lose the
dedup race.
- Dedup scope: (cultural_corridor, feature_type) across the run,
PLUS a per-body cross-type set so the same name can't be a river
AND an ocean AND a mountain on the same world. Hand-authored names
are seeded into both sets on load so templates win priority.
- Stem cap: each non-generic root token (e.g. 'Arcturus', 'Meridian')
may appear at most `--stem-cap` times across the full run (default
20), preventing single-word runaway. Fallback names bypass the cap.
- Earth blocklist: 181 curated entries covering major Earth cities,
mountains, rivers, oceans, historical/colonial spellings, and
Greek/Roman mythology that reads too literally. Prefixed variants
('Nouveau Paris', 'New Tokyo') explicitly allowed per the product
intent that Earth-echo names are fine but must not dominate.
Leading 'The ' is stripped before comparison so 'The Great Divide'
also matches.
Operational features:
- `--shard N/M` slices the body list into M partitions for parallel
runs. Two terminals × `--shard 0/2` + `--shard 1/2` fits the
~2.5 GB/instance VRAM footprint twice under the 50% cap on a
16 GB AMD GPU and roughly halves wall time.
- `--log PATH` writes a timestamped tee of every status line to a
file. Default: `.tmp/gemma_naming.shard{N}of{M}.log` when a
non-trivial shard is in use.
- SQLite `PRAGMA journal_mode=WAL` + `busy_timeout=15000` so two
concurrent shards serialize writes without lock errors.
- Per-body progress lines report `body K/N`, `sys K/N`, and
`hop=H` so the user can watch core sectors finish first.
- Each body logs the new names it produced per feature type so the
user can eyeball quality as the run progresses.
- Checkpoint summary every 25 bodies: cumulative names, rate,
ETA — gives the log regular scroll points.
- `--mock` uses `server/sr-voice/mock-stdio.sh` for dry-fire
pipeline validation without a model load (tested end-to-end).
Supporting files:
- `tooling/planet-gen/earth_blocklist.txt` — 181 curated entries.
- `tooling/db/backfill_cultural_corridor.py` — one-off migration
that fills the `cultural_corridor` column on both `star_systems`
and `bodies` from the `geographic_sector` values. Before this
pass, 99.4% of rows (3221/3240) had a NULL cultural_corridor
despite `wiki_sync.py` being aware of the column — the wiki
index.md files only carry the sector header, which was never
propagated to the DB column. Idempotent, safe to re-run after
any wiki_sync rebuild, explicit transaction wrapper with
rollback on failure.
Full batch runtime estimate: ~20 hours single-shard / ~10 hours
double-shard on this hardware. Smoke tests across five hardened
iterations (v1–v5) on GJ71b/c/d/d-1/e confirm the pipeline produces
clean, varied, culturally-coherent names with zero post-processing
residue.
217 lines
2.7 KiB
Plaintext
217 lines
2.7 KiB
Plaintext
# Earth-name blocklist for gemma_naming.py (#833, D-191 §4).
|
|
#
|
|
# Per memory feedback_earth_echo_names: Earth-sounding names are OK and
|
|
# expected — the setting is a blended reality. This file is a small,
|
|
# curated list of Earth *majors* whose raw names should not appear as
|
|
# settled-reach feature names because they are too on-the-nose. Prefixed
|
|
# variants like "Nouveau Paris", "Neu Berlin", or "New Tokyo" are allowed
|
|
# (they pass because the exact-match check does not strip prefixes).
|
|
#
|
|
# Match rule: case-insensitive whole-name equality between a generated
|
|
# feature name and any entry below. One entry per line, `#` comments are
|
|
# allowed, blank lines ignored.
|
|
#
|
|
# Expand this list if the generator starts producing raw-Earth-major
|
|
# names in frequent runs. Err on the side of short — we do NOT want to
|
|
# block every vaguely Earthy token.
|
|
|
|
# Capitals & megacities
|
|
Paris
|
|
London
|
|
Tokyo
|
|
Beijing
|
|
Moscow
|
|
Berlin
|
|
Madrid
|
|
Rome
|
|
Cairo
|
|
Mumbai
|
|
Delhi
|
|
Jakarta
|
|
Seoul
|
|
Bangkok
|
|
Istanbul
|
|
Lagos
|
|
Nairobi
|
|
Johannesburg
|
|
Sydney
|
|
Washington
|
|
Ottawa
|
|
Mexico City
|
|
Rio de Janeiro
|
|
Rio
|
|
Buenos Aires
|
|
Lima
|
|
Bogota
|
|
Dhaka
|
|
Karachi
|
|
Lahore
|
|
Tehran
|
|
Baghdad
|
|
Riyadh
|
|
Manila
|
|
Hanoi
|
|
Kuala Lumpur
|
|
Singapore
|
|
Vienna
|
|
Prague
|
|
Warsaw
|
|
Amsterdam
|
|
Brussels
|
|
Copenhagen
|
|
Stockholm
|
|
Oslo
|
|
Helsinki
|
|
Reykjavik
|
|
Lisbon
|
|
Athens
|
|
Budapest
|
|
Dublin
|
|
Edinburgh
|
|
Glasgow
|
|
|
|
# Iconic cities / regional majors
|
|
New York
|
|
Los Angeles
|
|
Chicago
|
|
Houston
|
|
Miami
|
|
Boston
|
|
Seattle
|
|
Toronto
|
|
Vancouver
|
|
Montreal
|
|
Barcelona
|
|
Munich
|
|
Hamburg
|
|
Frankfurt
|
|
Milan
|
|
Venice
|
|
Florence
|
|
Naples
|
|
Marseille
|
|
Lyon
|
|
Geneva
|
|
Zurich
|
|
Shanghai
|
|
Hong Kong
|
|
Taipei
|
|
Osaka
|
|
Kyoto
|
|
Yokohama
|
|
Busan
|
|
Ho Chi Minh
|
|
Kolkata
|
|
Chennai
|
|
Bangalore
|
|
Hyderabad
|
|
Addis Ababa
|
|
Casablanca
|
|
Accra
|
|
Kinshasa
|
|
Cape Town
|
|
Tel Aviv
|
|
Dubai
|
|
Doha
|
|
Damascus
|
|
Beirut
|
|
Jerusalem
|
|
Auckland
|
|
Wellington
|
|
Melbourne
|
|
Brisbane
|
|
Perth
|
|
|
|
# Mountain / natural majors that read as Earth
|
|
Everest
|
|
Kilimanjaro
|
|
Fuji
|
|
Matterhorn
|
|
Olympus
|
|
Etna
|
|
Vesuvius
|
|
Denali
|
|
Aconcagua
|
|
Ararat
|
|
K2
|
|
Annapurna
|
|
Elbrus
|
|
Kosciuszko
|
|
|
|
# Rivers
|
|
Amazon
|
|
Nile
|
|
Danube
|
|
Mississippi
|
|
Mekong
|
|
Ganges
|
|
Yangtze
|
|
Rhine
|
|
Volga
|
|
Thames
|
|
Seine
|
|
Tigris
|
|
Euphrates
|
|
Congo
|
|
Niger
|
|
Zambezi
|
|
|
|
# Oceans / seas
|
|
Atlantic
|
|
Pacific
|
|
Indian
|
|
Arctic
|
|
Mediterranean
|
|
Baltic
|
|
Caspian
|
|
Aegean
|
|
Adriatic
|
|
Caribbean
|
|
|
|
# Historical / colonial / alternate spellings of Earth cities — Gemma
|
|
# knows these from training data and emits them as if they were
|
|
# neutral names.
|
|
Calcutta
|
|
Bombay
|
|
Madras
|
|
Bangalore
|
|
Poona
|
|
Benares
|
|
Peking
|
|
Canton
|
|
Nanking
|
|
Chungking
|
|
Saigon
|
|
Rangoon
|
|
Ceylon
|
|
Batavia
|
|
Angora
|
|
Smyrna
|
|
Constantinople
|
|
Formosa
|
|
Tasmania
|
|
Rhodesia
|
|
Persia
|
|
Mesopotamia
|
|
|
|
# Earth natural-phenomenon names Gemma likes to reach for
|
|
Aurora Borealis
|
|
Aurora Australis
|
|
Great Divide
|
|
Great Lakes
|
|
Great Basin
|
|
Grand Canyon
|
|
Death Valley
|
|
Sahara
|
|
Gobi
|
|
Patagonia
|
|
Serengeti
|
|
Outback
|
|
Tundra
|
|
Siberia
|
|
Amazon Basin
|
|
|
|
# Greek/Roman mythology that reads too literally as Earth classical
|
|
Olympus Mons
|
|
Mount Olympus
|