diff --git a/docs/design/systems-framework.md b/docs/design/systems-framework.md index ab7a2b077..3168a3008 100644 --- a/docs/design/systems-framework.md +++ b/docs/design/systems-framework.md @@ -134,8 +134,7 @@ Columns are organized into eleven sections. Section order matches the wiki page | `founding_culture_primary` | enum | administrative_charter / syndic_company / nordic_diaspora / east_asian_diaspora / south_asian_diaspora / iberian_diaspora / west_african_diaspora / religious_refugee / separatist_charter / academic_scientific / military_frontier / asteroid_mining / agricultural_breadbasket / penal_exile / refugee_wave / speculative_venture / other | (G)(S)(C) | Primary founding cultural origin. Drives naming conventions, cultural contact networks, and some dialogue variation. | | `founding_culture_secondary` | enum or null | Same values as primary | (C) | Second culture if mixed founding. Null if single-culture. | | `cultural_persistence` | enum | standard / high / trauma_origin | (S)(G) | How strongly the founding culture persists across generations. standard: accent-level after ~3 generations. high: identity-primary for 6+ generations. trauma_origin: displacement identity persists 4–5 generations as organizing social force. | -| `historical_event` | enum or null | purge / company_abandonment / gate_silence / religious_schism / gyre_event / military_conflict / reformation / second_gate_activation / earth_adjacent_crisis / institutional_consolidation / none | (S)(G) | The most significant single historical event in this system's past. `institutional_consolidation` is the post-secession power reorganization that concentrated authority in existing institutional actors — not a conspiracy, a documented historical process. | -| `historical_event_age_years` | int or null | 0–600 | (S)(G) | Approximate years since the historical event. Null if none. Affects stability_index calculation. | +| `historical_events` | list | See below | (S)(G) | Ordered list of significant historical events, most recent first. Each entry has `type` (enum) and `age_years` (int or null). Most systems have 0–1 entries; older or more complex systems may have several. The most recent event drives `stability_index` calculation. Valid types: `purge` / `company_abandonment` / `gate_silence` / `religious_schism` / `gyre_event` / `military_conflict` / `reformation` / `second_gate_activation` / `earth_adjacent_crisis` / `institutional_consolidation` / `terraforming` / `civilizational_origin` / `colonial_charter` / `none`. `institutional_consolidation` is the post-secession power reorganization that concentrated authority in existing institutional actors — not a conspiracy, a documented historical process. | | `religious_status` | enum | none / early_covenant / mid_crisis / theocratic_consolidation / secular_cultural / radical_reformation / post_schism_accommodation / genuine_revival | (S)(C) | Compound field encoding religious community trajectory. `none` for non-religious-refugee founding. | | `religious_generation_count` | int or null | integer | (C) | Optional. Approximate generation count for religious communities where the generational arc matters for NPC dialogue. Nullable for non-religious founding and for systems where this level of detail is not authored. | @@ -230,7 +229,7 @@ Columns are organized into eleven sections. Section order matches the wiki page | Column | Type | Valid Values | Consumer | Notes | |---|---|---|---|---| -| `stability_index` | int | 1–5 | (S)(G)(D) | Derived synthesized pressure signal for the storyteller. 1–2 = hot system, crisis events elevated, instability-precondition Tier 1 modules available. 3 = moderate, normal distribution. 4–5 = cold, background-life events dominate. Calculated from governance_type + primary_fault_line + historical_event recency + distribution_index + dominant_faction / governance_type alignment mismatch. Stored for storyteller performance; recomputable from source columns. | +| `stability_index` | int | 1–5 | (S)(G)(D) | Derived synthesized pressure signal for the storyteller. 1–2 = hot system, crisis events elevated, instability-precondition Tier 1 modules available. 3 = moderate, normal distribution. 4–5 = cold, background-life events dominate. Calculated from governance_type + primary_fault_line + most recent historical_events entry recency + distribution_index + dominant_faction / governance_type alignment mismatch. Stored for storyteller performance; recomputable from source columns. | | `system_volatility` | enum | static / responsive / player_affected | (G)(S) | Which systems can have mutable state. static = core parameters are constants for the playthrough. responsive = dominant_faction, faction presence levels, and stability_index can shift in response to story arc events (player need not be the cause). player_affected = the player's actions directly affect core parameters. The implementation of change conditions is a separate design document; this column marks which systems the simulation is permitted to treat as mutable. | | `cultural_corridor` | string or null | corridor identifier | (C) | Identifier for the cultural corridor this system belongs to, if any. | | `calibration_note` | string or null | free text | (C) | Notes on any unusual column combination that required explicit justification during generation. | @@ -394,8 +393,7 @@ Every framework change is validated against Van Maanen's Star (D-036, S-TBD). Va | `founding_motivation` | pragmatist | | | `founding_culture_primary` | nordic_diaspora | | | `cultural_persistence` | standard | Accent-level after three generations. | -| `historical_event` | none | | -| `historical_event_age_years` | null | | +| `historical_events` | [] | No significant historical events. | | `religious_status` | none | | | `economic_tier` | 3 | Functional regional. | | `governance_type` | mixed_contested | | diff --git a/tooling/db/wiki_sync.py b/tooling/db/wiki_sync.py index f6b6855d8..dfe069ac6 100755 --- a/tooling/db/wiki_sync.py +++ b/tooling/db/wiki_sync.py @@ -1,16 +1,20 @@ #!/usr/bin/env python3 """ -Settled Reach Wiki Sync — parse wiki frontmatter into SQLite. +Settled Reach Wiki Sync — wiki frontmatter ↔ systems.db -Wiki pages are the single source of truth for all per-system data. -star-map.json only owns topology (edges). All per-node fields come from wiki. +Data flow: + - DB owns structured fields (identity, gates, history, economy, factions, culture) + - Wiki owns authored prose (supply_dependency, faction_notes, silence_topic, + narrative_hook, calibration_note) + - Wiki pages are generated from DB (structured infobox) + prose (authored body) -This module provides the sync logic. Called by tooling/process-wiki-changes. +This module provides: + - import_from_wiki(): parse wiki frontmatter into systems.db (migration/bootstrap) + - generate_wiki(): render wiki pages from DB + existing prose + - stats(): show completion stats -Usage: - python3 wiki_sync.py sync Parse wiki and upsert into SQLite - python3 wiki_sync.py stats Show completion stats - python3 wiki_sync.py --help Show this help +Database: server/data/systems.db +Schema: server/data/systems-schema.sql """ import json @@ -24,71 +28,92 @@ from pathlib import Path # --------------------------------------------------------------------------- SCRIPT_DIR = Path(__file__).resolve().parent -CONFIG_PATH = SCRIPT_DIR / "config.json" WORKTREE_ROOT = (SCRIPT_DIR / ".." / "..").resolve() -SCHEMA_PATH = WORKTREE_ROOT / "db" / "schema.sql" +SCHEMA_PATH = WORKTREE_ROOT / "server" / "data" / "systems-schema.sql" WIKI_DIR = WORKTREE_ROOT / "wiki" / "star-systems" -DB_PATH = (WORKTREE_ROOT / ".." / "settledreach.db").resolve() +DB_PATH = WORKTREE_ROOT / "server" / "data" / "systems.db" # --------------------------------------------------------------------------- -# Config / DB +# DB helpers # --------------------------------------------------------------------------- -def load_config(): - with open(CONFIG_PATH, "r") as f: - cfg = json.load(f) - cfg["sqlite_db_resolved"] = str(DB_PATH) - return cfg - - -def get_connection(cfg): - conn = sqlite3.connect(cfg["sqlite_db_resolved"]) +def get_connection(): + conn = sqlite3.connect(str(DB_PATH)) conn.execute("PRAGMA journal_mode=WAL;") conn.execute("PRAGMA foreign_keys=ON;") conn.row_factory = sqlite3.Row return conn +def ensure_schema(conn): + schema_sql = SCHEMA_PATH.read_text() + conn.executescript(schema_sql) + + # --------------------------------------------------------------------------- -# Column definitions (must match db/schema.sql star_systems table) +# Table → column mapping # --------------------------------------------------------------------------- -ALL_COLUMNS = [ - "system_id", "astronomical_id", "proper_name", "system_name", "star_type", - "geographic_sector", "geographic_band", "political_zone", - "habitable_planet_count", "inhabited_planet_count", "asteroid_belt", - "gas_giant", "habitability_profile", - "horizon_station", "aperture_count", "gate_connections", "gate_topology", - "span_gate_network", - "settlement_wave", "founding_motivation", "founding_culture_primary", - "founding_culture_secondary", "cultural_persistence", "historical_event", - "historical_event_age_years", "religious_status", "religious_generation_count", - "economic_tier", "population", "economic_base_primary", - "economic_base_secondary", "distribution_index", "imprint_access", - "supply_dependency", - "governance_type", "dominant_faction", "primary_fault_line", - "secondary_fault_line", - "assembly_presence", "commission_presence", "syndic_presence", "syndic_type", - "separatist_presence", "separatist_type", "institute_presence", - "guardians_presence", "faction_notes", - "cultural_register", "cultural_register_secondary", "ambient_anxiety", - "local_pride", "atmospheric_tone", "atmospheric_tone_secondary", - "active_situation", "silence_threshold", "silence_topic", - "earth_alignment", "earth_proximity", "earth_tension", - "primary_archetype", "secondary_archetype", "narrative_notable", - "narrative_hook", "generation_priority", - "stability_index", "system_volatility", "cultural_corridor", - "calibration_note", -] +# Maps each table to its columns (excluding system_id which is always the key) +TABLE_COLUMNS = { + "star_systems": [ + "proper_name", "system_name", "star_type", + "spectral_class", "dist_ly", "geographic_sector", "geographic_band", + "political_zone", + "habitable_planet_count", "inhabited_planet_count", "asteroid_belt", + "gas_giant", "habitability_profile", + "earth_alignment", "earth_proximity", "earth_tension", + "stability_index", "system_volatility", "cultural_corridor", + "generation_priority", + ], + "system_gates": [ + "horizon_station", "aperture_count", "gate_connections", + "gate_topology", "span_gate_network", + ], + "system_history": [ + "settlement_wave", "founding_motivation", "founding_culture_primary", + "founding_culture_secondary", "cultural_persistence", + "religious_status", "religious_generation_count", + ], + "system_economy": [ + "economic_tier", "population", "economic_base_primary", + "economic_base_secondary", "distribution_index", "imprint_access", + ], + "system_factions": [ + "governance_type", "dominant_faction", "primary_fault_line", + "secondary_fault_line", + "assembly_presence", "commission_presence", "syndic_presence", + "syndic_type", "separatist_presence", "separatist_type", + "institute_presence", "guardians_presence", + ], + "system_culture": [ + "cultural_register", "cultural_register_secondary", "ambient_anxiety", + "local_pride", "atmospheric_tone", "atmospheric_tone_secondary", + "active_situation", "silence_threshold", + "primary_archetype", "secondary_archetype", "narrative_notable", + ], +} + +# All structured columns (flat list for frontmatter parsing) +ALL_STRUCTURED_COLUMNS = set() +for cols in TABLE_COLUMNS.values(): + ALL_STRUCTURED_COLUMNS.update(cols) +ALL_STRUCTURED_COLUMNS.add("system_id") INTEGER_COLUMNS = { "habitable_planet_count", "inhabited_planet_count", - "horizon_station", "aperture_count", "gate_connections", - "historical_event_age_years", "religious_generation_count", + "asteroid_belt", "gas_giant", "horizon_station", + "aperture_count", "gate_connections", + "religious_generation_count", "economic_tier", "population", + "stability_index", "narrative_notable", } -BODY_SECTION_MAP = { +REAL_COLUMNS = {"dist_ly"} + +BOOLEAN_COLUMNS = {"asteroid_belt", "gas_giant", "horizon_station", "narrative_notable"} + +PROSE_SECTIONS = { "supply dependency": "supply_dependency", "faction notes": "faction_notes", "silence topic": "silence_topic", @@ -98,19 +123,22 @@ BODY_SECTION_MAP = { # --------------------------------------------------------------------------- -# Parsing +# Frontmatter parsing (for migration/bootstrap from existing wiki) # --------------------------------------------------------------------------- -def parse_frontmatter(text: str) -> dict[str, str]: - """Extract YAML frontmatter between --- delimiters, ignoring comments.""" +def parse_frontmatter(text: str) -> dict: + """Extract YAML frontmatter between --- delimiters.""" match = re.match(r"^---\n(.*?\n)---", text, re.DOTALL) if not match: return {} result = {} - for line in match.group(1).split("\n"): - line = line.strip() + lines = match.group(1).split("\n") + i = 0 + while i < len(lines): + line = lines[i].strip() + i += 1 if not line or line.startswith("#"): continue if ":" not in line: @@ -120,13 +148,64 @@ def parse_frontmatter(text: str) -> dict[str, str]: key = key.strip() value = value.strip() + # YAML list (historical_events) + if key == "historical_events": + items = [] + if value == "[]": + result[key] = items + continue + while i < len(lines): + next_line = lines[i] + stripped = next_line.strip() + if not stripped or (not next_line.startswith(" ") and not next_line.startswith("\t")): + break + if stripped.startswith("- "): + item = {} + item_content = stripped[2:].strip() + if ":" in item_content: + ik, _, iv = item_content.partition(":") + iv = iv.strip() + if iv == "~" or iv.lower() == "null": + iv = None + else: + try: + iv = int(iv) + except (ValueError, TypeError): + pass + item[ik.strip()] = iv + items.append(item) + i += 1 + elif stripped.startswith("#"): + break + elif ":" in stripped: + ik, _, iv = stripped.partition(":") + iv = iv.strip() + if iv == "~" or iv.lower() == "null": + iv = None + else: + try: + iv = int(iv) + except (ValueError, TypeError): + pass + if items: + items[-1][ik.strip()] = iv + i += 1 + else: + break + result[key] = items + continue + + # Strip quotes if value.startswith('"') and value.endswith('"'): value = value[1:-1] elif value.startswith("'") and value.endswith("'"): value = value[1:-1] if value == "~" or value.lower() == "null" or value == "": - value = "" + value = None + else: + result[key] = value + continue result[key] = value @@ -134,7 +213,7 @@ def parse_frontmatter(text: str) -> dict[str, str]: def parse_body_sections(text: str) -> dict[str, str]: - """Extract content from ## sections in the markdown body.""" + """Extract authored prose from ## sections in the markdown body.""" match = re.match(r"^---\n.*?\n---\n", text, re.DOTALL) body = text[match.end():] if match else text @@ -147,7 +226,7 @@ def parse_body_sections(text: str) -> dict[str, str]: if current_key is not None: sections[current_key] = "\n".join(current_lines).strip() header = line[3:].strip().lower() - current_key = BODY_SECTION_MAP.get(header) + current_key = PROSE_SECTIONS.get(header) current_lines = [] elif current_key is not None: if line.strip().startswith(""): @@ -160,65 +239,16 @@ def parse_body_sections(text: str) -> dict[str, str]: return sections -# --------------------------------------------------------------------------- -# Data loading -# --------------------------------------------------------------------------- - - -def load_wiki_pages(): - """Load all wiki pages. Returns (systems_dict, warnings, body_count).""" - if not WIKI_DIR.is_dir(): - return {}, [f"Wiki directory not found: {WIKI_DIR}"], 0 - - systems = {} - warnings = [] - body_count = 0 - - for d in sorted(WIKI_DIR.iterdir()): - if not d.is_dir(): - continue - index_file = d / "index.md" - if not index_file.exists(): - warnings.append(f"Missing index.md in {d.name}") - continue - - text = index_file.read_text(encoding="utf-8") - fm = parse_frontmatter(text) - body = parse_body_sections(text) - - sid = fm.get("system_id") - if not sid: - warnings.append(f"No system_id in {d.name}/index.md") - continue - - record = {} - for col in ALL_COLUMNS: - if col in fm and fm[col] != "": - record[col] = fm[col] - elif col in body and body[col] != "": - record[col] = body[col] - - for key in BODY_SECTION_MAP.values(): - if key in body and body[key]: - body_count += 1 - - systems[sid] = record - - return systems, warnings, body_count - - def coerce_value(col, val): """Coerce a value for SQLite insertion.""" if val is None: return None - if isinstance(val, (int, float)): - return val val_str = str(val).strip() if val_str == "" or val_str == "~" or val_str.lower() == "null": return None - if col == "horizon_station": + if col in BOOLEAN_COLUMNS: lower = val_str.lower() if lower in ("true", "yes", "1"): return 1 @@ -232,103 +262,416 @@ def coerce_value(col, val): except (ValueError, TypeError): return None + if col in REAL_COLUMNS: + try: + return float(val_str) + except (ValueError, TypeError): + return None + return val_str # --------------------------------------------------------------------------- -# Sync +# Import from wiki (migration/bootstrap) # --------------------------------------------------------------------------- -def sync(cfg): - """Parse wiki and upsert into star_systems table.""" - wiki_systems, warnings, body_count = load_wiki_pages() +def import_from_wiki(): + """Parse all wiki frontmatter and populate systems.db.""" + if not WIKI_DIR.is_dir(): + return {"ok": False, "error": f"Wiki directory not found: {WIKI_DIR}"} - if not wiki_systems: - return {"ok": False, "error": "No wiki pages found", "warnings": warnings} + conn = get_connection() + ensure_schema(conn) - conn = get_connection(cfg) + warnings = [] + imported = 0 + + for d in sorted(WIKI_DIR.iterdir()): + if not d.is_dir(): + continue + index_file = d / "index.md" + if not index_file.exists(): + warnings.append(f"Missing index.md in {d.name}") + continue + + text = index_file.read_text(encoding="utf-8") + fm = parse_frontmatter(text) + + sid = fm.get("system_id") + if not sid: + warnings.append(f"No system_id in {d.name}/index.md") + continue + + try: + # Upsert into each table + for table, columns in TABLE_COLUMNS.items(): + all_cols = ["system_id"] + columns + if table == "star_systems": + all_cols.append("updated_at") + + values = [sid] + for col in columns: + values.append(coerce_value(col, fm.get(col))) + if table == "star_systems": + values.append(None) # updated_at = DEFAULT + + placeholders = ", ".join(["?"] * len(all_cols)) + col_list = ", ".join(all_cols) + + if table == "star_systems": + update_set = ", ".join( + f"{c} = excluded.{c}" for c in columns + ) + ", updated_at = datetime('now')" + else: + update_set = ", ".join( + f"{c} = excluded.{c}" for c in columns + ) + + sql = ( + f"INSERT INTO {table} ({col_list}) " + f"VALUES ({placeholders}) " + f"ON CONFLICT(system_id) DO UPDATE SET {update_set}" + ) + conn.execute(sql, values) + + # Historical events + events = fm.get("historical_events", []) + if isinstance(events, list): + conn.execute( + "DELETE FROM historical_events WHERE system_id = ?", (sid,) + ) + for idx, event in enumerate(events): + if isinstance(event, dict): + conn.execute( + "INSERT INTO historical_events " + "(system_id, event_type, age_years, sort_order) " + "VALUES (?, ?, ?, ?)", + (sid, event.get("type", ""), event.get("age_years"), idx), + ) + + imported += 1 + + except sqlite3.Error as exc: + warnings.append(f"Error on {sid}: {exc}") + + conn.commit() + conn.close() + + return { + "ok": True, + "systems_imported": imported, + "warnings": warnings, + "summary": f"Imported {imported} systems into {DB_PATH}", + } + + +# --------------------------------------------------------------------------- +# Wiki page generation (DB → wiki infobox) +# --------------------------------------------------------------------------- + + +def _format_population(pop): + """Format population with commas.""" + if pop is None: + return "~" try: - schema_sql = SCHEMA_PATH.read_text() - conn.executescript(schema_sql) - - col_list = ", ".join(ALL_COLUMNS) - placeholders = ", ".join(["?"] * len(ALL_COLUMNS)) - update_set = ", ".join( - f"{col} = excluded.{col}" for col in ALL_COLUMNS if col != "system_id" - ) - - upsert_sql = ( - f"INSERT INTO star_systems ({col_list}, synced_at) " - f"VALUES ({placeholders}, datetime('now')) " - f"ON CONFLICT(system_id) DO UPDATE SET " - f"{update_set}, synced_at = datetime('now')" - ) - - upserted = 0 - for sid in sorted(wiki_systems.keys()): - record = wiki_systems[sid] - record["system_id"] = sid - values = [coerce_value(col, record.get(col)) for col in ALL_COLUMNS] - - if values[1] is None: # astronomical_id NOT NULL - warnings.append(f"Skipping {sid}: missing astronomical_id") - continue - - conn.execute(upsert_sql, values) - upserted += 1 - - conn.commit() - - return { - "ok": True, - "systems_synced": upserted, - "body_sections_filled": body_count, - "warnings": warnings, - "summary": f"Synced {upserted} systems from wiki ({body_count} body sections filled)", - } - - except sqlite3.Error as exc: - conn.rollback() - return {"ok": False, "error": str(exc), "warnings": warnings} - finally: - conn.close() + return f"{int(pop):,}" + except (ValueError, TypeError): + return str(pop) -def stats(cfg): - """Show per-column fill rates.""" - conn = get_connection(cfg) +def generate_infobox(conn, sid: str) -> str: + """Generate the READ-ONLY infobox markdown for a system.""" + s = conn.execute("SELECT * FROM star_systems WHERE system_id = ?", (sid,)).fetchone() + if not s: + return "" + + g = conn.execute("SELECT * FROM system_gates WHERE system_id = ?", (sid,)).fetchone() + h = conn.execute("SELECT * FROM system_history WHERE system_id = ?", (sid,)).fetchone() + e = conn.execute("SELECT * FROM system_economy WHERE system_id = ?", (sid,)).fetchone() + f = conn.execute("SELECT * FROM system_factions WHERE system_id = ?", (sid,)).fetchone() + c = conn.execute("SELECT * FROM system_culture WHERE system_id = ?", (sid,)).fetchone() + events = conn.execute( + "SELECT event_type, age_years FROM historical_events " + "WHERE system_id = ? ORDER BY sort_order", (sid,) + ).fetchall() + + lines = [ + "## System Profile", + "", + "", + "| | |", + "|---|---|", + ] + + # Star + spectral = s["spectral_class"] or s["star_type"] or "~" + dist = f"{s['dist_ly']:.1f} ly" if s["dist_ly"] is not None else "~" + lines.append(f"| **Star** | {spectral} · {dist} |") + + # Bodies + hab = s["habitable_planet_count"] + inh = s["inhabited_planet_count"] + if hab is not None or inh is not None: + hab_s = f"{hab} habitable" if hab is not None else "~" + inh_s = f"{inh} inhabited" if inh is not None else "~" + lines.append(f"| **Bodies** | {hab_s} · {inh_s} |") + + # Gates + if g: + ap = g["aperture_count"] if g["aperture_count"] is not None else "~" + topo = g["gate_topology"] or "~" + lines.append(f"| **Gates** | {ap} aperture · {topo} |") + + # Settlement + if h: + wave = h["settlement_wave"] or "~" + lines.append(f"| **Settlement** | {wave} |") + + # Historical events + if events: + ev_parts = [] + for ev in events: + age = f"{ev['age_years']}y" if ev["age_years"] is not None else "~" + ev_parts.append(f"{ev['event_type']} ({age})") + lines.append(f"| **History** | {', '.join(ev_parts)} |") + + # Economy + if e and (e["population"] is not None or e["economic_tier"] is not None): + pop = _format_population(e["population"]) + tier = f"Tier {e['economic_tier']}" if e["economic_tier"] is not None else "~" + base_parts = [] + if e["economic_base_primary"]: + base_parts.append(e["economic_base_primary"]) + if e["economic_base_secondary"]: + base_parts.append(e["economic_base_secondary"]) + base = " / ".join(base_parts) if base_parts else "~" + if e["population"] is not None: + lines.append(f"| **Population** | {pop} |") + lines.append(f"| **Economy** | {tier} · {base} |") + + # Governance + if f and (f["governance_type"] or f["dominant_faction"]): + gov = f["governance_type"] or "~" + dom = f["dominant_faction"] or "~" + lines.append(f"| **Governance** | {gov} · {dom} |") + + # Culture + if c and c["atmospheric_tone"]: + tone = c["atmospheric_tone"] + if c["atmospheric_tone_secondary"]: + tone += f" / {c['atmospheric_tone_secondary']}" + lines.append(f"| **Atmosphere** | {tone} |") + + # Stability + stab = s["stability_index"] + if stab is not None: + vol = s["system_volatility"] or "~" + lines.append(f"| **Stability** | {stab} · {vol} |") + + # Political + if s["earth_alignment"]: + lines.append(f"| **Earth Alignment** | {s['earth_alignment']} |") + + lines.append("") + return "\n".join(lines) + + +def generate_topology(conn, sid: str, star_map_path: Path) -> str: + """Generate the READ-ONLY topology section from star-map.json.""" + try: + with open(star_map_path) as f: + data = json.load(f) + except (FileNotFoundError, json.JSONDecodeError): + return "## Topology\n\n" + + # Find hop distance + hop = "~" + for node in data["nodes"]: + if node["system_id"] == sid: + hop = node.get("hop_distance_from_gateway", "~") + break + + # Find adjacent systems + adjacent = set() + for a, b in data["edges"]: + if a == sid: + adjacent.add(b) + elif b == sid: + adjacent.add(a) + + adj_str = ", ".join(sorted(adjacent)) if adjacent else "none" + + return ( + "## Topology\n" + "\n" + f"- **Hop Distance from Gateway:** {hop}\n" + f"- **Adjacent Systems:** {adj_str}\n" + ) + + +def generate_wiki_page(conn, sid: str, star_map_path: Path) -> str: + """Generate a complete wiki page from DB + existing prose.""" + s = conn.execute("SELECT * FROM star_systems WHERE system_id = ?", (sid,)).fetchone() + if not s: + return "" + + system_name = s["proper_name"] or s["system_name"] or f"System {sid}" + star_type = s["star_type"] or "~" + sector = s["geographic_sector"] or "~" + + # Title block + title = f"# {system_name}\n**{sid}** | {star_type}-type | {sector}\n\n---\n\n" + + # Infobox + infobox = generate_infobox(conn, sid) + + # Topology + topology = generate_topology(conn, sid, star_map_path) + + return title + infobox + "\n---\n\n" + topology + + +def generate_wiki(prose_only_update=False): + """Generate/update wiki pages from DB. + + For each system: + - Generates the title + infobox + topology (from DB/star-map) + - Preserves existing authored prose sections from the wiki file + - Writes the combined result back + """ + conn = get_connection() + star_map_path = WORKTREE_ROOT / "docs" / "design" / "star-map.json" + + systems = conn.execute("SELECT system_id FROM star_systems ORDER BY system_id").fetchall() + updated = 0 + created = 0 + + for row in systems: + sid = row["system_id"] + dirname = sid.replace(" ", "-") + outdir = WIKI_DIR / dirname + outfile = outdir / "index.md" + + # Read existing prose if file exists + existing_prose = {} + if outfile.exists(): + text = outfile.read_text(encoding="utf-8") + existing_prose = parse_body_sections(text) + + # Generate structured parts + s = conn.execute("SELECT * FROM star_systems WHERE system_id = ?", (sid,)).fetchone() + system_name = s["proper_name"] or s["system_name"] or f"System {sid}" + star_type = s["star_type"] or "~" + sector = s["geographic_sector"] or "~" + + infobox = generate_infobox(conn, sid) + topology = generate_topology(conn, sid, star_map_path) + + # Build page + page_lines = [ + f"# {system_name}", + f"**{sid}** | {star_type}-type | {sector}", + "", + "---", + "", + infobox, + "---", + "", + ] + + # Prose sections — preserve existing content or leave placeholder + for header, key in PROSE_SECTIONS.items(): + section_title = header.title() + content = existing_prose.get(key, "") + page_lines.append(f"## {section_title}") + if content: + page_lines.append("") + page_lines.append(content) + else: + comment_map = { + "supply_dependency": "", + "faction_notes": "", + "silence_topic": "", + "narrative_hook": "", + "calibration_note": "", + } + page_lines.append(comment_map.get(key, "")) + page_lines.append("") + + # Topology (always last) + page_lines.append(topology) + + outdir.mkdir(parents=True, exist_ok=True) + outfile.write_text("\n".join(page_lines), encoding="utf-8") + + if outfile.exists(): + updated += 1 + else: + created += 1 + + conn.close() + return { + "ok": True, + "updated": updated, + "created": created, + "summary": f"Generated {updated + created} wiki pages", + } + + +# --------------------------------------------------------------------------- +# Stats +# --------------------------------------------------------------------------- + + +def stats(): + """Show per-table fill rates.""" + conn = get_connection() try: row = conn.execute("SELECT COUNT(*) as cnt FROM star_systems").fetchone() total = row["cnt"] if total == 0: - return {"ok": True, "summary": "No systems in database. Run sync first."} - - fill = {} - for col in ALL_COLUMNS: - r = conn.execute( - f"SELECT COUNT(*) as cnt FROM star_systems WHERE {col} IS NOT NULL AND {col} != ''" - ).fetchone() - fill[col] = r["cnt"] - - total_cells = total * len(ALL_COLUMNS) - filled = sum(fill.values()) - pct = filled / total_cells * 100 + print("No systems in database. Run import first.") + return {"ok": True, "total": 0} print(f"Star systems: {total}") - print(f"Completion: {filled}/{total_cells} ({pct:.1f}%)") + print(f"Database: {DB_PATH}") print() - for col in ALL_COLUMNS: - c = fill[col] - p = c / total * 100 - bar = "#" * int(p / 5) + "." * (20 - int(p / 5)) - mark = "+" if p == 100 else " " - print(f" {mark} {col:35s} {c:3d}/{total} {bar} {p:.0f}%") - return {"ok": True, "total": total, "filled": filled, "pct": round(pct, 1)} + total_cells = 0 + filled_cells = 0 + + for table, columns in TABLE_COLUMNS.items(): + table_filled = 0 + table_total = total * len(columns) + for col in columns: + r = conn.execute( + f"SELECT COUNT(*) as cnt FROM {table} WHERE {col} IS NOT NULL" + ).fetchone() + table_filled += r["cnt"] + + total_cells += table_total + filled_cells += table_filled + pct = table_filled / table_total * 100 if table_total > 0 else 0 + bar = "#" * int(pct / 5) + "." * (20 - int(pct / 5)) + print(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%") + + # Historical events + ev_count = conn.execute("SELECT COUNT(*) as cnt FROM historical_events").fetchone()["cnt"] + ev_systems = conn.execute( + "SELECT COUNT(DISTINCT system_id) as cnt FROM historical_events" + ).fetchone()["cnt"] + print(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems") + + overall = filled_cells / total_cells * 100 if total_cells > 0 else 0 + print(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)") + + return {"ok": True, "total": total, "filled": filled_cells, "pct": round(overall, 1)} except sqlite3.OperationalError as exc: - return {"ok": False, "error": f"Table not found — run sync first: {exc}"} + print(f"Table not found — run import first: {exc}") + return {"ok": False, "error": str(exc)} finally: conn.close() @@ -337,45 +680,38 @@ def stats(cfg): # CLI # --------------------------------------------------------------------------- -HELP_TEXT = f"""\ -Wiki Sync — wiki frontmatter -> SQLite - -Usage: - wiki_sync.py sync Parse wiki and upsert into SQLite - wiki_sync.py stats Show completion stats - wiki_sync.py --help Show this help - -Source: {WIKI_DIR} -Schema: {SCHEMA_PATH} -DB: {DB_PATH} -""" - def main(): + help_text = f"""\ +Wiki Sync — systems.db ↔ wiki pages + +Usage: + wiki_sync.py import Parse wiki frontmatter into systems.db (bootstrap) + wiki_sync.py generate Generate wiki pages from systems.db + wiki_sync.py stats Show completion stats + wiki_sync.py --help Show this help + +Database: {DB_PATH} +Schema: {SCHEMA_PATH} +Wiki: {WIKI_DIR} +""" if len(sys.argv) < 2 or sys.argv[1] in ("--help", "-h", "help"): - print(HELP_TEXT) + print(help_text) sys.exit(0) cmd = sys.argv[1] - try: - cfg = load_config() - except (FileNotFoundError, json.JSONDecodeError) as exc: - print(json.dumps({"ok": False, "error": f"Config error: {exc}"}, indent=2)) - sys.exit(1) - - if cmd == "sync": - result = sync(cfg) + if cmd == "import": + result = import_from_wiki() + print(json.dumps(result, indent=2)) + elif cmd == "generate": + result = generate_wiki() print(json.dumps(result, indent=2)) elif cmd == "stats": - result = stats(cfg) - if not result.get("ok"): - print(json.dumps(result, indent=2)) + stats() else: - result = {"ok": False, "error": f"Unknown: {cmd}"} - print(json.dumps(result, indent=2)) - - sys.exit(0 if result.get("ok") else 1) + print(json.dumps({"ok": False, "error": f"Unknown: {cmd}"}, indent=2)) + sys.exit(1) if __name__ == "__main__": diff --git a/tooling/migrate-s-to-gj.py b/tooling/migrate-s-to-gj.py new file mode 100644 index 000000000..7d9bfb8c4 --- /dev/null +++ b/tooling/migrate-s-to-gj.py @@ -0,0 +1,251 @@ +#!/usr/bin/env python3 +"""Migrate all S-number references to GJ astronomical IDs. + +Replaces S-XXX system identifiers with GJ XXX astronomical IDs across: +- star-map.json (nodes and edges) +- systems.db (all tables) +- wiki pages (headers and topology sections) +- catalog-index.md + +Usage: python3 tooling/migrate-s-to-gj.py [--dry-run] +""" + +import json +import os +import re +import sqlite3 +import sys + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +STAR_MAP = os.path.join(ROOT, "docs", "design", "star-map.json") +SYSTEMS_DB = os.path.join(ROOT, "server", "data", "systems.db") +SCHEMA_SQL = os.path.join(ROOT, "server", "data", "systems-schema.sql") +WIKI_DIR = os.path.join(ROOT, "wiki", "star-systems") +CATALOG = os.path.join(ROOT, "wiki", "catalog-index.md") + +dry_run = "--dry-run" in sys.argv + + +def build_mapping(): + """Build S-number -> GJ ID mapping from systems.db or star-map.json.""" + # Try DB first (has both system_id and astronomical_id) + if os.path.exists(SYSTEMS_DB): + conn = sqlite3.connect(SYSTEMS_DB) + cur = conn.cursor() + cur.execute("SELECT system_id, astronomical_id FROM star_systems WHERE system_id LIKE 'S-%'") + rows = cur.fetchall() + conn.close() + if rows: + mapping = {r[0]: r[1] for r in rows} + with open(STAR_MAP) as f: + data = json.load(f) + return mapping, data + + # Fall back to star-map.json + with open(STAR_MAP) as f: + data = json.load(f) + mapping = {} + for node in data["nodes"]: + sid = node["system_id"] + aid = node.get("astronomical_id") + if aid and sid.startswith("S-"): + mapping[sid] = aid + return mapping, data + + +def migrate_star_map(mapping, data): + """Replace S-numbers with GJ IDs in star-map.json.""" + print(f"Migrating star-map.json ({len(data['nodes'])} nodes, {len(data['edges'])} edges)...") + + # Update nodes + for node in data["nodes"]: + old_id = node["system_id"] + new_id = mapping.get(old_id, old_id) + node["system_id"] = new_id + # Remove astronomical_id field (now redundant with system_id) + if "astronomical_id" in node: + del node["astronomical_id"] + + # Update edges + new_edges = [] + for edge in data["edges"]: + new_edge = [mapping.get(e, e) for e in edge] + new_edges.append(new_edge) + data["edges"] = new_edges + + # Update meta references + if "_meta" in data: + meta = data["_meta"] + # Replace S-number references in meta strings + for key, val in meta.items(): + if isinstance(val, str): + for sid, gj in mapping.items(): + val = val.replace(sid, gj) + meta[key] = val + + if not dry_run: + with open(STAR_MAP, "w") as f: + json.dump(data, f, indent=2) + print(" Written.") + else: + print(" (dry run)") + + +def migrate_database(mapping): + """Replace S-numbers with GJ IDs in systems.db.""" + if not os.path.exists(SYSTEMS_DB): + print("Database not found, skipping.") + return + + print(f"Migrating systems.db...") + conn = sqlite3.connect(SYSTEMS_DB) + cur = conn.cursor() + + # Get all tables with system_id column + cur.execute("SELECT name FROM sqlite_master WHERE type='table'") + tables = [r[0] for r in cur.fetchall()] + + tables_with_sid = [] + for table in tables: + cur.execute(f"PRAGMA table_info({table})") + cols = [r[1] for r in cur.fetchall()] + if "system_id" in cols: + tables_with_sid.append(table) + + if dry_run: + for table in tables_with_sid: + cur.execute(f"SELECT COUNT(*) FROM {table}") + count = cur.fetchone()[0] + print(f" Would migrate {table} ({count} rows)") + conn.close() + return + + # Must update in dependency order: children first, then parent + # Temporarily disable foreign key checks + cur.execute("PRAGMA foreign_keys = OFF") + + # star_systems is the parent table + parent = "star_systems" + children = [t for t in tables_with_sid if t != parent] + + # Update children first + for table in children: + cur.execute(f"SELECT DISTINCT system_id FROM {table}") + ids = [r[0] for r in cur.fetchall()] + for old_id in ids: + new_id = mapping.get(old_id, old_id) + if new_id != old_id: + cur.execute(f"UPDATE {table} SET system_id = ? WHERE system_id = ?", + (new_id, old_id)) + + # Update parent + cur.execute(f"SELECT system_id FROM {parent}") + ids = [r[0] for r in cur.fetchall()] + for old_id in ids: + new_id = mapping.get(old_id, old_id) + if new_id != old_id: + cur.execute(f"UPDATE {parent} SET system_id = ? WHERE system_id = ?", + (new_id, old_id)) + + cur.execute("PRAGMA foreign_keys = ON") + conn.commit() + conn.close() + print(f" Migrated {len(tables_with_sid)} tables.") + + +def migrate_wiki_pages(mapping): + """Replace S-numbers in wiki page headers and topology sections.""" + print(f"Migrating wiki pages...") + count = 0 + + # Build reverse: need to match S-numbers in text + # Sort by length descending to avoid partial matches (S-10 before S-1) + sorted_sids = sorted(mapping.keys(), key=lambda x: -len(x)) + + for gj_dir in os.listdir(WIKI_DIR): + index_path = os.path.join(WIKI_DIR, gj_dir, "index.md") + if not os.path.isfile(index_path): + continue + + with open(index_path) as f: + content = f.read() + + original = content + + # Replace S-numbers with GJ IDs + for sid in sorted_sids: + gj = mapping[sid] + # Replace in header line (e.g., "**GJ 570A** | S-235 | M-type | core") + # Remove the "| S-XXX " segment entirely since GJ is already in the header + content = re.sub( + r'\| ' + re.escape(sid) + r' \|', + '|', + content + ) + # Replace standalone S-number references (in topology, prose, etc.) + # But be careful not to match partial (S-10 in S-100) + content = re.sub( + r'\b' + re.escape(sid) + r'\b', + gj, + content + ) + + if content != original: + count += 1 + if not dry_run: + with open(index_path, "w") as f: + f.write(content) + + print(f" {'Would modify' if dry_run else 'Modified'} {count} wiki pages.") + + +def migrate_catalog(mapping): + """Replace S-numbers in catalog-index.md.""" + if not os.path.exists(CATALOG): + print("Catalog index not found, skipping.") + return + + print("Migrating catalog-index.md...") + with open(CATALOG) as f: + content = f.read() + + original = content + sorted_sids = sorted(mapping.keys(), key=lambda x: -len(x)) + for sid in sorted_sids: + gj = mapping[sid] + content = re.sub(r'\b' + re.escape(sid) + r'\b', gj, content) + + if content != original and not dry_run: + with open(CATALOG, "w") as f: + f.write(content) + print(" Written.") + elif content != original: + print(" (dry run)") + else: + print(" No changes needed.") + + +def main(): + if dry_run: + print("=== DRY RUN ===\n") + + mapping, star_map_data = build_mapping() + print(f"Built mapping: {len(mapping)} systems (S-000..S-{len(mapping)-1} -> GJ IDs)\n") + + # Show sample + samples = ["S-000", "S-001", "S-010", "S-067", "S-091", "S-120", "S-181", "S-213", "S-235"] + for s in samples: + if s in mapping: + print(f" {s} -> {mapping[s]}") + print() + + migrate_star_map(mapping, star_map_data) + migrate_database(mapping) + migrate_wiki_pages(mapping) + migrate_catalog(mapping) + + print("\nDone." + (" (dry run — no files changed)" if dry_run else "")) + + +if __name__ == "__main__": + main() diff --git a/tooling/process-wiki-system-changes b/tooling/process-wiki-system-changes index 28a9862d2..e4053c8c5 100755 --- a/tooling/process-wiki-system-changes +++ b/tooling/process-wiki-system-changes @@ -1,19 +1,18 @@ #!/usr/bin/env python3 """ process-wiki-system-changes -Propagates wiki changes to star-map.json and SQLite. +Pipeline for star systems data. -Wiki pages (wiki/star-systems/*/index.md) are the single source of truth. -This script: - 1. Reads all wiki frontmatter - 2. Updates star-map.json nodes (preserving edges/topology) - 3. Syncs all data into SQLite star_systems table - 4. Reports completion stats +Modes: + (default) Import wiki frontmatter → systems.db, update star-map.json, + then regenerate wiki pages from DB (infobox + topology). + --generate Generate wiki pages from existing DB (no import). + --rebuild-db Drop and recreate systems.db from wiki (fresh start). + --stats Show fill rates. + --dry-run Show what star-map.json changes would occur. -Usage: - tooling/process-wiki-system-changes Run full sync - tooling/process-wiki-system-changes --dry-run Show what would change - tooling/process-wiki-system-changes --stats Just show fill rates +Database: server/data/systems.db +Schema: server/data/systems-schema.sql """ import argparse @@ -24,111 +23,100 @@ from pathlib import Path SCRIPT_DIR = Path(__file__).resolve().parent WORKTREE_ROOT = (SCRIPT_DIR / "..").resolve() STAR_MAP_PATH = WORKTREE_ROOT / "docs" / "design" / "star-map.json" +DB_PATH = WORKTREE_ROOT / "server" / "data" / "systems.db" -# Import wiki_sync for DB operations +# Import wiki_sync sys.path.insert(0, str(SCRIPT_DIR / "db")) import wiki_sync # --------------------------------------------------------------------------- -# Star-map.json update +# Star-map.json update (topology fields only) # --------------------------------------------------------------------------- -# Fields that star-map.json nodes carry (topology + join keys) STAR_MAP_NODE_FIELDS = [ - "system_id", "astronomical_id", + "system_id", "gate_topology", "aperture_count", "gate_connections", "hop_distance_from_gateway", ] -def update_star_map(wiki_systems: dict) -> dict: - """Update star-map.json node fields from wiki data. Returns summary.""" +def update_star_map_from_db(): + """Update star-map.json node fields from systems.db.""" + conn = wiki_sync.get_connection() + with open(STAR_MAP_PATH, "r") as f: data = json.load(f) nodes = data["nodes"] updated = 0 - added = 0 - wiki_sids = set(wiki_systems.keys()) - map_sids = {n["system_id"] for n in nodes} - # Update existing nodes for node in nodes: sid = node["system_id"] - if sid not in wiki_systems: + row = conn.execute( + "SELECT g.gate_topology, g.aperture_count, g.gate_connections " + "FROM star_systems s LEFT JOIN system_gates g ON s.system_id = g.system_id " + "WHERE s.system_id = ?", (sid,) + ).fetchone() + + if not row: continue - wiki = wiki_systems[sid] changed = False - for field in STAR_MAP_NODE_FIELDS: - if field in wiki and wiki[field] != "": - val = wiki[field] - # Coerce integers - if field in ("aperture_count", "gate_connections", "hop_distance_from_gateway"): - try: - val = int(val) - except (ValueError, TypeError): - continue - if node.get(field) != val: - node[field] = val - changed = True + field_map = { + "gate_topology": row["gate_topology"], + "aperture_count": row["aperture_count"], + "gate_connections": row["gate_connections"], + } + + for field, val in field_map.items(): + if val is not None and node.get(field) != val: + node[field] = val + changed = True - # Preserve is_gateway flag if changed: updated += 1 - # Add nodes for wiki pages that aren't in star-map yet - for sid in wiki_sids - map_sids: - wiki = wiki_systems[sid] - new_node = {"system_id": sid} - for field in STAR_MAP_NODE_FIELDS: - if field in wiki and wiki[field] != "": - val = wiki[field] - if field in ("aperture_count", "gate_connections", "hop_distance_from_gateway"): - try: - val = int(val) - except (ValueError, TypeError): - continue - new_node[field] = val - nodes.append(new_node) - added += 1 + conn.close() with open(STAR_MAP_PATH, "w") as f: json.dump(data, f, indent=2) f.write("\n") - return { - "nodes": len(nodes), - "edges": len(data["edges"]), - "updated": updated, - "added": added, - } + return {"nodes": len(nodes), "edges": len(data["edges"]), "updated": updated} -def dry_run(wiki_systems: dict) -> dict: - """Show what would change in star-map.json without writing.""" +def dry_run_star_map(): + """Show what would change in star-map.json.""" + conn = wiki_sync.get_connection() + with open(STAR_MAP_PATH, "r") as f: data = json.load(f) changes = [] for node in data["nodes"]: sid = node["system_id"] - if sid not in wiki_systems: - continue - wiki = wiki_systems[sid] - for field in STAR_MAP_NODE_FIELDS: - if field in wiki and wiki[field] != "": - val = wiki[field] - if field in ("aperture_count", "gate_connections", "hop_distance_from_gateway"): - try: - val = int(val) - except (ValueError, TypeError): - continue - if node.get(field) != val: - changes.append(f" {sid}.{field}: {node.get(field)} -> {val}") + row = conn.execute( + "SELECT g.gate_topology, g.aperture_count, g.gate_connections " + "FROM star_systems s LEFT JOIN system_gates g ON s.system_id = g.system_id " + "WHERE s.system_id = ?", (sid,) + ).fetchone() - return {"changes": changes} + if not row: + continue + + field_map = { + "gate_topology": row["gate_topology"], + "aperture_count": row["aperture_count"], + "gate_connections": row["gate_connections"], + } + + for field, val in field_map.items(): + if val is not None and node.get(field) != val: + changes.append(f" {sid}.{field}: {node.get(field)} -> {val}") + + conn.close() + return changes # --------------------------------------------------------------------------- @@ -138,63 +126,73 @@ def dry_run(wiki_systems: dict) -> dict: def main(): parser = argparse.ArgumentParser( - description="Propagate wiki changes to star-map.json and SQLite" + description="Star systems data pipeline: wiki ↔ systems.db ↔ star-map.json" ) - parser.add_argument("--dry-run", action="store_true", help="Show changes without writing") - parser.add_argument("--stats", action="store_true", help="Just show fill rates") + parser.add_argument("--generate", action="store_true", help="Generate wiki from DB only") + parser.add_argument("--rebuild-db", action="store_true", help="Drop and recreate DB from wiki") + parser.add_argument("--stats", action="store_true", help="Show fill rates") + parser.add_argument("--dry-run", action="store_true", help="Show star-map changes without writing") args = parser.parse_args() - cfg = wiki_sync.load_config() - if args.stats: - wiki_sync.stats(cfg) + wiki_sync.stats() return - # Load wiki - wiki_systems, warnings, body_count = wiki_sync.load_wiki_pages() - if not wiki_systems: - print("ERROR: No wiki pages found") - for w in warnings: - print(f" {w}") - sys.exit(1) + if args.generate: + print("--- Generating wiki pages from DB ---") + result = wiki_sync.generate_wiki() + print(result["summary"]) + return - print(f"Loaded {len(wiki_systems)} wiki pages ({body_count} body sections filled)") - if warnings: - print(f"Warnings: {len(warnings)}") - for w in warnings[:5]: - print(f" {w}") - if len(warnings) > 5: - print(f" ... and {len(warnings) - 5} more") + if args.rebuild_db: + print(f"--- Rebuilding {DB_PATH} from wiki ---") + if DB_PATH.exists(): + DB_PATH.unlink() + print(" Deleted existing DB") if args.dry_run: - result = dry_run(wiki_systems) - if result["changes"]: - print(f"\nstar-map.json changes ({len(result['changes'])}):") - for c in result["changes"]: + # Need DB populated first for dry-run to work + if not DB_PATH.exists(): + print("No DB found. Run without --dry-run first.") + sys.exit(1) + changes = dry_run_star_map() + if changes: + print(f"star-map.json changes ({len(changes)}):") + for c in changes: print(c) else: - print("\nstar-map.json: no changes") - print("\n(dry run — nothing written)") + print("star-map.json: no changes") return - # Step 1: Update star-map.json - print("\n--- Updating star-map.json ---") - map_result = update_star_map(wiki_systems) - print(f"Nodes: {map_result['nodes']}, Edges: {map_result['edges']}") - print(f"Updated: {map_result['updated']}, Added: {map_result['added']}") + # Full pipeline: wiki → DB → star-map.json → wiki pages + print("=== Star Systems Pipeline ===\n") - # Step 2: Sync to SQLite - print("\n--- Syncing to SQLite ---") - db_result = wiki_sync.sync(cfg) - if db_result["ok"]: - print(db_result["summary"]) - else: - print(f"ERROR: {db_result.get('error')}") + # Step 1: Import wiki frontmatter into DB + print("--- Step 1: Wiki → DB ---") + result = wiki_sync.import_from_wiki() + if not result["ok"]: + print(f"ERROR: {result.get('error')}") sys.exit(1) + print(result["summary"]) + if result["warnings"]: + for w in result["warnings"][:5]: + print(f" ⚠ {w}") + if len(result["warnings"]) > 5: + print(f" ... and {len(result['warnings']) - 5} more") - # Step 3: Quick stats + # Step 2: Update star-map.json from DB + print("\n--- Step 2: DB → star-map.json ---") + map_result = update_star_map_from_db() + print(f"Nodes: {map_result['nodes']}, Edges: {map_result['edges']}, Updated: {map_result['updated']}") + + # Step 3: Generate wiki pages from DB + print("\n--- Step 3: DB → Wiki pages ---") + gen_result = wiki_sync.generate_wiki() + print(gen_result["summary"]) + + # Step 4: Stats print("\n--- Completion ---") - wiki_sync.stats(cfg) + wiki_sync.stats() if __name__ == "__main__": diff --git a/tooling/validate-ron b/tooling/validate-ron index 12304025a..74bbb2d95 100755 --- a/tooling/validate-ron +++ b/tooling/validate-ron @@ -6,7 +6,7 @@ # # Examples: # tooling/validate-ron content/global/zone-identity-spec.example.ron zone -# tooling/validate-ron content/global/culture-krenn.example.ron culture +# tooling/validate-ron content/global/culture-van-maanens-star.example.ron culture set -euo pipefail diff --git a/tooling/wiki/generate-stubs.py b/tooling/wiki/generate-stubs.py index 3233f7561..adee9badc 100644 --- a/tooling/wiki/generate-stubs.py +++ b/tooling/wiki/generate-stubs.py @@ -91,8 +91,7 @@ founding_motivation: ~ founding_culture_primary: ~ founding_culture_secondary: ~ cultural_persistence: ~ -historical_event: ~ -historical_event_age_years: ~ +historical_events: [] religious_status: ~ religious_generation_count: ~