jpmschweitzer/settled-reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
de8bcf4ebbcc8eaee1a11c8ade30fdcfdea10c71
settled-reach/tooling/economy-db/economy_import/specialization.py
T
jpmschweitzerandClaude Fable 5 4f73624eff refactor(db): split import_economics.py; single generator-source registry (T-1067) 
import_economics.py 2,620 → 309 lines — a thin orchestrator keeping the
exact CLI, single-transaction/rollback contract, and exit codes. The 16
import steps, MIGRATION_SQL, brands shell-out, validators, and stamp
write now live in tooling/economy-db/economy_import/ (db, migration,
economy, corporations, brands, bodies, atlas, specialization, traits,
validators, stamp, paths, errors). Full type hints throughout.

tooling/generator_sources.py replaces the triplicated source registry
(importer / stamp checker / pr-process watch list — the skill now derives
its list via --list). The registry stamps itself, and economy_import/
modules are globbed fail-closed, so a future module is stamped the moment
it exists — closing the silently-weakened-stamp failure mode.

Rider: connector config helpers centralized in tooling/db/common.py.

Byte-identical behavior proven: full-import table dump diff EMPTY over
107,843 lines / 37 tables (volatile timestamp fields excluded); dry-run
output parity; generated_brands.toml sha unchanged. make test-tooling
PASS; ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:13:28 +02:00

429 lines
19 KiB
Python
Raw Blame History

"""System specialization — D-237 authored layer (#1013, #1015)."""
import sqlite3
import tomllib
from collections import Counter
from .errors import ImportAborted
from .paths import SPECIALIZATION_VOCAB_TOML, SYSTEM_SPECIALIZATION_TOML
# Authoritative D-233 projected enums. The full CI guardrail suite (V-SES-*)
# lands in #1015; this module performs the FK + enum sanity the import itself
# needs to stay sound. NOTE for #1015: the D-237 "equal-or-higher" override rule
# must NOT hard-fail HUB specializations (shipbuilding, transit_hub) whose local
# production_ubiquity_projected is intentionally below their commodity's global
# default — see specialization_vocabulary.toml header.
_BULK_CLASSES: set[str] = {"BulkSolid", "BulkLiquid", "PrecisionDense", "Perishable", "NonPhysical"}
_PRODUCTION_UBIQUITY: set[str] = {"Ubiquitous", "Common", "Specialist", "MonopolySource"}
_FACTION_VOCAB: set[str] = {
"concord_assembly", "compact", "compact_sympathetic", "syndic_dominant",
"veil_institute", "independent", "disputed", "mixed",
}
# Combined cultural_specialization vocabulary (D-237; miri-round3 §2). Two value
# kinds in one column: activity/character and founding-heritage. EXTENSIBLE — the
# #1016 content pass adds heritage values here as more GTTR systems are reviewed;
# add the new value to this set and CI accepts it. V-SES-04 validates against it.
_CULTURAL_ACTIVITY: set[str] = {
"scholarly", "artistic", "institutional", "commercial", "agrarian",
"industrial_heritage", "medical_elite", "ecological", "military",
"financial_technocratic", "cosmopolitan", "compact_cooperative",
}
# Canonical 47-value heritage taxonomy (#1016, D-237). Source of truth:
# docs/workshops/system-economic-specialization/heritage-taxonomy-draft.md.
# Real-world people/nationality granularity, lowercase_snake. Heritage wins over
# activity values when both apply. Pin only when a system's founding heritage
# DIVERGES from its corridor baseline (D-167/D-232); corridor-typical systems
# stay NULL and take the corridor default.
_CULTURAL_HERITAGE: set[str] = {
# British Isles & Anglo-diaspora
"anglo", "scottish", "irish", "welsh",
# Iberian, Latin & Lusophone
"portuguese", "brazilian", "afro_brazilian", "cape_verdean", "angolan",
"sao_tomean", "spanish", "canarian", "italian", "french",
# Northern / Central / Eastern European
"german", "dutch", "nordic", "finnish", "polish", "czech", "russian",
"luxembourgish", "hungarian",
# Sub-Saharan African
"afrikaans", "cape_malay", "zulu", "xhosa", "herero", "shona", "swahili",
"igbo", "yoruba", "hausa", "akan",
# South Asian
"indian", "bengali", "punjabi", "konkan",
# East & Southeast Asian
"chinese", "korean", "japanese", "vietnamese", "tagalog",
# Pacific
"maori",
# Middle East / North Africa / Central Asia
"arab", "persian", "turkic",
}
_CULTURAL_VOCAB: set[str] = _CULTURAL_ACTIVITY | _CULTURAL_HERITAGE
# Catalog production_ubiquity concentration ranking for V-SES-03 (override may
# only be >= the commodity's global default). regional ≈ common tier.
_UBIQUITY_RANK: dict[str, int] = {
"ubiquitous": 0, "common": 1, "regional": 1, "concentrated": 2, "monopolistic": 3,
}
def import_system_specialization(conn: sqlite3.Connection, dry_run: bool,
strict: bool = False) -> dict:
"""Import + validate the D-237 authored specialization layer (#1013, #1015).
Reads two TOMLs:
- specialization_vocabulary.toml -> specialization_vocabulary table
(FK-validated against commodities; MUST run after import_commodities).
- system_specialization.toml -> UPSERTs economic_specialization +
cultural_specialization onto system_economy, and dominant_faction onto
system_factions, for authored (hero) systems only.
Authored lore wins; unauthored systems are left NULL for the generator's
heuristic fallback. economic_specialization / cultural_specialization are
owned exclusively by this importer, so they are cleared to NULL first for
idempotency (a removed stanza must not leave a stale value). dominant_faction
is shared with other derivation paths, so it is ONLY overwritten for systems
present in the TOML (per #1013) — never globally cleared.
CI guardrails (#1015):
Always-on hard errors (abort): V-SES-01 (econ value in vocab), V-SES-03
(override >= catalog concentration), V-SES-04 (cultural value in vocab),
V-SES-05 (faction in vocab), V-SES-06 (vocab commodity FK), plus unknown
system_id.
Completeness gates V-SES-02 (every inhabited system resolves a non-null
economic value) and V-FAC-01 (every inhabited named system has authored
dominant_faction) are HARD only under `strict` — their preconditions are
the #1014 fallback (blocked by #982) and the #1016 content pass. Until
those land, they emit warnings; flip `--strict-specialization` on once
both are complete so regen-db enforces them.
Soft warnings (W-SES-*, W-FAC-*) and the coverage report always print.
Returns a coverage dict for the caller's report. Raises ImportAborted on a
hard validation failure.
"""
with open(SPECIALIZATION_VOCAB_TOML, "rb") as f:
vocab = tomllib.load(f)
with open(SYSTEM_SPECIALIZATION_TOML, "rb") as f:
systems = tomllib.load(f)
commodity_pu = {
r[0]: r[1] for r in conn.execute(
"SELECT commodity_id, production_ubiquity FROM commodities"
).fetchall()
}
commodity_ids = set(commodity_pu)
system_ids = {
r[0] for r in conn.execute("SELECT system_id FROM star_systems").fetchall()
}
economy_system_ids = {
r[0] for r in conn.execute("SELECT system_id FROM system_economy").fetchall()
}
faction_system_ids = {
r[0] for r in conn.execute("SELECT system_id FROM system_factions").fetchall()
}
errors: list[str] = []
# --- Vocabulary: FK + enum validation (V-SES-06, V-SES-03) ----------
vocab_rows: list[tuple] = []
for spec_id, v in vocab.items():
cid = v.get("commodity_id")
if cid not in commodity_ids: # V-SES-06
errors.append(
f"V-SES-06: specialization_vocabulary '{spec_id}': commodity_id "
f"'{cid}' not in commodities catalog"
)
bc = v.get("bulk_class_projected")
if bc not in _BULK_CLASSES:
errors.append(
f"specialization_vocabulary '{spec_id}': bulk_class_projected "
f"'{bc}' invalid (expected one of {sorted(_BULK_CLASSES)})"
)
pu = v.get("production_ubiquity_projected")
if pu not in _PRODUCTION_UBIQUITY:
errors.append(
f"specialization_vocabulary '{spec_id}': "
f"production_ubiquity_projected '{pu}' invalid "
f"(expected one of {sorted(_PRODUCTION_UBIQUITY)})"
)
override = v.get("production_ubiquity_override") or None # "" -> NULL
# V-SES-03: a non-empty override may only raise (or equal) the
# commodity's global concentration — never claim a globally scarce good
# is locally more common. Empty override = HUB value (intentionally
# projects below catalog; exempt — see vocab TOML header).
if override is not None and cid in commodity_pu:
cat_rank = _UBIQUITY_RANK.get(commodity_pu[cid], -1)
ovr_rank = _UBIQUITY_RANK.get(override, -1)
if ovr_rank < 0:
errors.append(
f"V-SES-03: specialization_vocabulary '{spec_id}': "
f"production_ubiquity_override '{override}' not a catalog term"
)
elif ovr_rank < cat_rank:
errors.append(
f"V-SES-03: specialization_vocabulary '{spec_id}': override "
f"'{override}' is less concentrated than commodity "
f"'{cid}' catalog default '{commodity_pu[cid]}' — incoherent"
)
vocab_rows.append((spec_id, cid, override, bc, pu, v.get("description", "")))
valid_spec_ids = set(vocab.keys())
# --- System stanzas: id + value validation (V-SES-01/04/05) ---------
for sid, s in systems.items():
if sid not in system_ids:
errors.append(
f"system_specialization '{sid}': not a known star_systems.system_id"
)
es = s.get("economic_specialization")
if es is not None and es not in valid_spec_ids: # V-SES-01
errors.append(
f"V-SES-01: system_specialization '{sid}': economic_specialization "
f"'{es}' not in specialization_vocabulary"
)
cs = s.get("cultural_specialization")
if cs is not None and cs not in _CULTURAL_VOCAB: # V-SES-04
errors.append(
f"V-SES-04: system_specialization '{sid}': cultural_specialization "
f"'{cs}' not in the activity+heritage vocabulary "
f"(add new heritage values to _CULTURAL_HERITAGE)"
)
df = s.get("dominant_faction")
if df is not None and df not in _FACTION_VOCAB: # V-SES-05
errors.append(
f"V-SES-05: system_specialization '{sid}': dominant_faction "
f"'{df}' invalid (expected one of {sorted(_FACTION_VOCAB)})"
)
if errors:
print(f" SPECIALIZATION ERRORS ({len(errors)}):")
for e in errors:
print(f" - {e}")
raise ImportAborted()
coverage: dict = {
"vocab": len(vocab_rows),
"economic": 0,
"cultural": 0,
"faction": 0,
"missing_economy_row": [],
"missing_faction_row": [],
}
if dry_run:
for sid, s in systems.items():
coverage["economic"] += 1 if s.get("economic_specialization") else 0
coverage["cultural"] += 1 if s.get("cultural_specialization") else 0
coverage["faction"] += 1 if s.get("dominant_faction") else 0
_specialization_checks(conn, vocab, systems, strict)
return coverage
# --- Repopulate vocabulary table ------------------------------------
conn.execute("DELETE FROM specialization_vocabulary")
conn.executemany(
"""INSERT INTO specialization_vocabulary (
specialization_id, commodity_id, production_ubiquity_override,
bulk_class_projected, production_ubiquity_projected, description
) VALUES (?, ?, ?, ?, ?, ?)""",
vocab_rows,
)
# --- Clear importer-owned columns (idempotency) ---------------------
conn.execute(
"UPDATE system_economy SET economic_specialization = NULL, "
"cultural_specialization = NULL"
)
# --- UPSERT per-system authored fields ------------------------------
for sid, s in systems.items():
es = s.get("economic_specialization")
cs = s.get("cultural_specialization")
if sid in economy_system_ids:
conn.execute(
"UPDATE system_economy SET economic_specialization = ?, "
"cultural_specialization = ? WHERE system_id = ?",
(es, cs, sid),
)
coverage["economic"] += 1 if es else 0
coverage["cultural"] += 1 if cs else 0
else:
coverage["missing_economy_row"].append(sid)
df = s.get("dominant_faction")
if df is not None:
if sid in faction_system_ids:
conn.execute(
"UPDATE system_factions SET dominant_faction = ? "
"WHERE system_id = ?",
(df, sid),
)
coverage["faction"] += 1
else:
coverage["missing_faction_row"].append(sid)
# --- Completeness gates + soft warnings + coverage report -----------
# Run AFTER the UPSERTs so they see the freshly-written DB state.
_specialization_checks(conn, vocab, systems, strict)
return coverage
def _specialization_checks(
conn: sqlite3.Connection, vocab: dict, systems: dict, strict: bool
) -> None:
"""V-SES-02 / V-FAC-01 completeness gates, soft warnings, coverage report.
Reads the post-UPSERT DB state. Gates are warnings unless `strict` (their
preconditions — the #1014 fallback and the #1016 content pass — are not yet
in place). Raises ImportAborted only when strict and a gate fails.
"""
gate_failures: list[str] = []
warnings: list[str] = []
# Population: integer where present. Inhabited = population > 0.
inhabited = [
(r[0], r[1]) for r in conn.execute(
"SELECT system_id, population FROM system_economy "
"WHERE population IS NOT NULL AND population > 0"
).fetchall()
]
econ = {
r[0]: r[1] for r in conn.execute(
"SELECT system_id, economic_specialization FROM system_economy"
).fetchall()
}
cult = {
r[0]: r[1] for r in conn.execute(
"SELECT system_id, cultural_specialization FROM system_economy"
).fetchall()
}
faction = {
r[0]: r[1] for r in conn.execute(
"SELECT system_id, dominant_faction FROM system_factions"
).fetchall()
}
currency = {
r[0]: r[1] for r in conn.execute(
"SELECT system_id, currency_zone FROM star_systems"
).fetchall()
}
# "Named" = has authored GTTR identity (proper_name or gttr_hook).
named = {
r[0] for r in conn.execute(
"SELECT system_id FROM star_systems "
"WHERE (proper_name IS NOT NULL AND proper_name != '') "
" OR (gttr_hook IS NOT NULL AND gttr_hook != '')"
).fetchall()
}
# Tier-1 monopolist corp HQ presence per system (best-effort; tables may be
# sparse pre-#1016). primary_operation/headquarters live on corp_presence.
hq_systems: set[str] = set()
try:
hq_systems = {
r[0] for r in conn.execute(
"SELECT DISTINCT location_id FROM corp_presence "
"WHERE primary_operation IS NOT NULL"
).fetchall()
}
except sqlite3.OperationalError:
pass
vocab_pu = {k: (v.get("production_ubiquity_projected")) for k, v in vocab.items()}
vocab_commodity = {k: v.get("commodity_id") for k, v in vocab.items()}
# V-SES-02: every inhabited system must resolve a non-null economic value
# (authored here, or via the #1014 fallback once it exists).
for sid, _pop in inhabited:
if not econ.get(sid):
gate_failures.append(
f"V-SES-02: inhabited system '{sid}' has no economic_specialization "
f"(authored or fallback)"
)
# V-FAC-01: every inhabited NAMED system must have an authored faction.
for sid, _pop in inhabited:
if sid in named and not faction.get(sid):
gate_failures.append(
f"V-FAC-01: inhabited named system '{sid}' has no dominant_faction"
)
# --- Soft warnings --------------------------------------------------
# W-SES-01: MonopolySource systems for D-177 human review.
monopoly = [
sid for sid, e in econ.items()
if e and vocab_pu.get(e) == "MonopolySource"
]
if monopoly:
warnings.append(f"W-SES-01: MonopolySource systems [D-177 review]: {sorted(monopoly)}")
# W-SES-02: >25% of authored-economic systems share one value.
econ_counts = Counter(e for e in econ.values() if e)
n_authored_econ = sum(econ_counts.values())
if n_authored_econ:
for val, cnt in econ_counts.items():
if cnt > 0.25 * n_authored_econ and cnt > 2:
warnings.append(
f"W-SES-02: '{val}' covers {cnt}/{n_authored_econ} "
f"({100*cnt//n_authored_econ}%) of authored-economic systems"
)
# W-SES-08: estate_farming + large population (probably breadbasket).
for sid, pop in inhabited:
if econ.get(sid) == "estate_farming" and pop and pop > 5_000_000:
warnings.append(
f"W-SES-08: '{sid}' is estate_farming with population {pop} "
f"(probably breadbasket)"
)
# W-FAC-01: compact + tractus_primary currency (D-172 violation).
# W-FAC-04: compact_sympathetic + mark_primary (may be full member).
for sid, f in faction.items():
if not f:
continue
cz = (currency.get(sid) or "").upper()
if f == "compact" and cz == "TRACTUS_PRIMARY":
warnings.append(f"W-FAC-01: '{sid}' compact + TRACTUS_PRIMARY currency (D-172)")
if f == "compact_sympathetic" and cz == "MARK_PRIMARY":
warnings.append(f"W-FAC-04: '{sid}' compact_sympathetic + MARK_PRIMARY (may be full member)")
# W-FAC-02: compact + MonopolySource extraction (Compact self-sufficiency),
# excluding terroir_* / marble_monopoly (lore-sanctioned monopolies).
_exempt = {"marble_monopoly", "terroir_agriculture", "terroir_spirits", "terroir_organics"}
for sid, f in faction.items():
e = econ.get(sid)
if f == "compact" and e and vocab_pu.get(e) == "MonopolySource" and e not in _exempt:
warnings.append(f"W-FAC-02: '{sid}' compact + MonopolySource '{e}' (self-sufficiency doctrine)")
# W-FAC-03: syndic_dominant + no corp HQ presence (ungrounded pin).
for sid, f in faction.items():
if f == "syndic_dominant" and sid not in hq_systems:
warnings.append(f"W-FAC-03: '{sid}' syndic_dominant but no corp HQ in corp_presence (ungrounded)")
# --- Coverage report (always) ---------------------------------------
n_inhabited = len(inhabited)
n_named = len(named)
n_econ = sum(1 for e in econ.values() if e)
n_cult = sum(1 for c in cult.values() if c)
n_fac = sum(1 for f in faction.values() if f)
bulk_dist: Counter = Counter()
pu_dist: Counter = Counter()
for e in econ.values():
if e and e in vocab:
bulk_dist[vocab[e].get("bulk_class_projected")] += 1
pu_dist[vocab[e].get("production_ubiquity_projected")] += 1
print(" Specialization coverage (D-237):")
print(f" economic: {n_econ} authored ({n_inhabited} inhabited; rest on fallback once #1014 lands)")
print(f" cultural: {n_cult} authored ({n_named} named; rest on corridor default)")
print(f" faction: {n_fac} authored ({n_named} named; rest on derivation)")
print(" BulkClass: " + " | ".join(f"{k}={v}" for k, v in sorted(bulk_dist.items())))
print(" ProductionUbiquity: " + " | ".join(f"{k}={v}" for k, v in sorted(pu_dist.items())))
if warnings:
print(f" Specialization warnings ({len(warnings)}):")
for w in warnings:
print(f" - {w}")
if gate_failures:
if strict:
print(f" SPECIALIZATION COMPLETENESS FAILURES ({len(gate_failures)}) [strict]:")
for g in gate_failures:
print(f" - {g}")
raise ImportAborted()
else:
print(
f" Specialization completeness: {len(gate_failures)} gate item(s) "
f"pending (#1014 fallback / #1016 content pass) — warnings only until "
f"--strict-specialization"
)
Reference in New Issue View Git Blame Copy Permalink