refactor(tooling): T-1290 — the wiki domain, and the renderer that must not run
`reach wiki stats` and `reach wiki gttr-hook` replace tooling/db/wiki_sync.py and populate_gttr_hook.py. Both are output-identical to the originals: `stats` byte-for-byte, and all 301 extracted GTTR hooks line-for-line. wiki_sync.py moved whole, but generate_wiki() and import_from_wiki() are NOT verbs. Before porting, the old `--generate` was run against a clean tree to get a parity baseline. It changed all 301 system pages, +940 / -10,761, and was reverted at once. It deletes the Celestial Bodies / Stations blocks (owned by the Rust atlas sync, which it does not know about), deletes the Industries / Exports / Imports rows (nothing writes those any more), and rewrites star types where systems.db and the pages disagree. D-262, CLAUDE.md and the wiki skill all described it as the routine, prose-preserving render. CLAUDE.md and the skill now say not to run it; D-262 needs amending — T-1292. Provenance moves to tooling/archive/, with a README naming what each script did and why it is not run: - pql-migrate/ (the T-1271 ruling) - wiki-bootstrap/: assign-astro-ids + its catalog, migrate-s-to-gj, patch-core-sector (hardcodes a dead path), fill-missing-globes, generate-stubs and find-stubs (finds 0 stubs — Phase 1 is done), backfill_cultural_corridor (a raw systems.db patch script, outside D-262), and process-wiki-system-changes, whose last step is the destructive render Also: - stats() printed "run import first" and exited 0 when a table was missing; it now fails with a remedy. generate_wiki() counted created pages after writing them, so `created` was always 0. - tooling/godot-cold-parse and godot-parse-sweep were never retired after T-1283, and the pr-process skill still told agents to run them. Removed; the skill and parse_sweep.gd now name the reach verbs. - systems.db re-stamped: schema comments changed, and the stamp records the schema file's SHA for tamper detection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
+4
-4
@@ -5,8 +5,8 @@ before anything moves. This is the artefact the per-domain port tickets are
|
||||
written from — without it their boundaries would be guesses, renegotiated
|
||||
halfway through the move.
|
||||
|
||||
**Status:** map only. Nothing here has moved yet except `check client-version`,
|
||||
which shipped in T-1262 as the reference implementation.
|
||||
**Status:** the map was written before anything moved; the ✅ rows below record
|
||||
which domains have since been ported, and by which ticket.
|
||||
|
||||
## What the survey changed
|
||||
|
||||
@@ -90,11 +90,11 @@ invented.
|
||||
| ~~`starmap`~~ | **folded into `atlas map`** — the top rung of the same ladder | — |
|
||||
| ~~`planet`~~ | **folded into `atlas planet`** — the third rung of the same ladder | — |
|
||||
| `ledger` | the economics pipeline, named for the UI component that will aggregate it | ✅ ported (T-1289). `economy-db/` → `domains/ledger/` (`economy_import/` kept by name; the entrypoint became `service.py`), `schema_version.py` with it. `reach ledger import`; `make regen-db` survives as a one-line delegate, `make economy-db` retired. `generated_brands.toml` byte-identical across the move |
|
||||
| `wiki` | wiki sync and content maintenance | `wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, `db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, `fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, `process-wiki-system-changes` |
|
||||
| `wiki` | wiki sync and content maintenance | ✅ ported (T-1290): `reach wiki stats`, `reach wiki gttr-hook` (both output-identical to the originals). `wiki_sync.py` moved whole, but its renderer and importer are NOT verbs — re-rendering deletes ~10,700 lines of committed pages (T-1292). The seven one-shots (`assign-astro-ids`, `migrate-s-to-gj`, `patch-core-sector`, `fill-missing-globes`, `generate-stubs`/`find-stubs`, `backfill_cultural_corridor`) and the destructive `process-wiki-system-changes` went to `archive/wiki-bootstrap/` |
|
||||
| `assets` | connectors to the tower-of-joy generators | `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, `db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py` |
|
||||
| `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** |
|
||||
| `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` |
|
||||
| `godot` | Godot parse and cold-start checks | `godot-parse-sweep`, `godot-cold-parse` |
|
||||
| `godot` | Godot parse and cold-start checks | ✅ ported (T-1283); the two bash originals were left beside the port and retired in T-1290 |
|
||||
| `generate` | content generators not owned elsewhere | ✅ ported (T-1286). `generate-brands` + `generate-corporations` collapsed into `core.process.cargo_binary` — they were the same 24 lines of bash a third time |
|
||||
| `dev` | developer environment and workflow | ✅ ported (T-1286). The three environment scripts split decision from performing — `godot_plan`/`worktree_plan` are pure and pinned by `test_environment.py` |
|
||||
| `pr` | the PR/review loop | ✅ ported (T-1286). `watchlist-diff` now reads the watched set from `generator_sources.py` instead of restating it |
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# tooling/archive — provenance, not tooling
|
||||
|
||||
Scripts kept because they record HOW something in the repo came to be, not
|
||||
because anything should run them again. None is a `reach` verb, none is
|
||||
importable (no `__init__.py`, excluded from package discovery and from the
|
||||
D-263 conformance scope), and several would do damage if run today. Deleting
|
||||
them would lose the provenance; keeping them live would imply they still work
|
||||
(tooling/DOMAINS.md, "pql-migrate/ is provenance, not tooling").
|
||||
|
||||
| path | what it did | why it is here |
|
||||
|---|---|---|
|
||||
| `pql-migrate/` | the one-shot planning-store migration into pql | completed migration (T-1271 ruling) |
|
||||
| `wiki-bootstrap/assign-astro-ids.py` + `gj-catalog-real.json` | gave the 300 `S-NNN` systems real Gliese-Jahreiss ids | done; `S-` ids no longer exist |
|
||||
| `wiki-bootstrap/migrate-s-to-gj.py` | rewrote every `S-NNN` reference to its GJ id | done |
|
||||
| `wiki-bootstrap/patch-core-sector.py` | added 17 hand-balanced core-sector gate edges | done; hardcodes a `planning/` path that no longer exists |
|
||||
| `wiki-bootstrap/generate-stubs.py`, `find-stubs` | created and then tracked the Phase-1 system stubs | done; `find-stubs --all-sectors 0 99` finds 0 stubs |
|
||||
| `wiki-bootstrap/fill-missing-globes.py` | copied donor `globe.png` files into bodies that lacked one | self-described one-shot |
|
||||
| `wiki-bootstrap/backfill_cultural_corridor.py` | copied `geographic_sector` into `cultural_corridor` | a raw systems.db patch script, the kind `.claude/rules/asset-pipeline.md` forbids; absent from the D-262 flow |
|
||||
| `wiki-bootstrap/process-wiki-system-changes` | wiki → systems.db → star-map.json → re-rendered system pages | **destructive today** — its last step is `wiki_sync.generate_wiki()`, which no longer reproduces the committed pages (T-1290 finding, see the wiki domain's router) |
|
||||
|
||||
Archived in T-1250 / T-1290 (2026-09-23).
|
||||
Executable → Regular
@@ -0,0 +1,11 @@
|
||||
"""`wiki` — the wiki seed's tooling (D-263).
|
||||
|
||||
Mirrors an implant app: the wiki/GTTR popups the player reads. Before touching
|
||||
anything here, read `Skill(wiki)` — two ownership models run in opposite
|
||||
directions through this directory, and D-262's flow diagram
|
||||
(docs/diagrams/data-flow/wiki-generator-flow.d2) is the canonical map.
|
||||
|
||||
Formerly tooling/db/wiki_sync.py and tooling/db/populate_gttr_hook.py
|
||||
(T-1290). The one-shot bootstrap scripts that sat beside them went to
|
||||
tooling/archive/wiki-bootstrap/.
|
||||
"""
|
||||
@@ -1,6 +1,4 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Extract a short narrative hook from every wiki/star-systems/<slug>/gttr.md
|
||||
"""Extract a short narrative hook from every wiki/star-systems/<slug>/gttr.md
|
||||
and store it on `star_systems.gttr_hook`.
|
||||
|
||||
The GTTR ("Drifter's Guide to the Reach") files follow a consistent
|
||||
@@ -19,35 +17,32 @@ cultural signal available for each system, capturing things like:
|
||||
- Posto Avançado: "forward post — the place beyond the established
|
||||
line"
|
||||
|
||||
This script parses each gttr.md, regex-extracts the first `**NAME**`
|
||||
paragraph, normalises whitespace, and truncates at a soft word cap so
|
||||
the hook stays cheap to inject into prompts. Empty or unmatched files
|
||||
leave the column NULL.
|
||||
This parses each gttr.md, regex-extracts the first `**NAME**` paragraph,
|
||||
normalises whitespace, and truncates at a soft word cap so the hook stays
|
||||
cheap to inject into prompts. Empty or unmatched files leave the column NULL.
|
||||
|
||||
Idempotent, safe to re-run after any wiki update. Explicit transaction
|
||||
wrapper with rollback on exception.
|
||||
wrapper with rollback on exception. It writes systems.db directly, which the
|
||||
asset-pipeline rule otherwise forbids; this edge is sanctioned by the D-262
|
||||
flow diagram (`populate_gttr_hook -> systems.db`), and `gttr_hook` sits on
|
||||
`star_systems`, which `reach ledger import` does not clear.
|
||||
|
||||
Usage:
|
||||
tooling/db/populate_gttr_hook.py
|
||||
tooling/db/populate_gttr_hook.py --max-words 45
|
||||
tooling/db/populate_gttr_hook.py --dry-run
|
||||
tooling/db/populate_gttr_hook.py --system "GJ 71"
|
||||
Formerly tooling/db/populate_gttr_hook.py (T-1290).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
REPO_ROOT = (SCRIPT_DIR / ".." / "..").resolve()
|
||||
from tooling.core import config, console
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
REPO_ROOT = config.repo_root()
|
||||
DB_PATH = REPO_ROOT / "server" / "data" / "systems.db"
|
||||
WIKI_SYSTEMS = REPO_ROOT / "wiki" / "star-systems"
|
||||
|
||||
# Matches the first paragraph that opens with `**NAME**` at the start
|
||||
# of a line. Captures everything up to a blank line or the next H1/H2.
|
||||
# Non-greedy on the content.
|
||||
# Match a line that opens with `**NAME**` (any non-asterisk chars, since
|
||||
# names contain accents from many alphabets — Á, Ž, Ç, Ñ, etc) followed
|
||||
# by the paragraph body up to a blank line or the next markdown heading.
|
||||
@@ -86,10 +81,7 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None:
|
||||
# Collapse whitespace so multi-line paragraphs become one clean line.
|
||||
hook = re.sub(r"\s+", " ", hook).strip()
|
||||
|
||||
# Strip a leading orphan "is" that comes from `**NAME**` + " is …":
|
||||
# the regex captures the word "is" on its own because the opener is
|
||||
# typically `**GATEWAY** (known as Tau Ceti...) is the most connected…`.
|
||||
# Normal reading already works — this is just hygiene.
|
||||
# Drop an empty parenthetical left behind by the collapse — hygiene only.
|
||||
hook = re.sub(r"\s*\(\s*\)\s*", " ", hook)
|
||||
|
||||
# Hard cap at max_words. Truncate at the last word boundary before
|
||||
@@ -101,37 +93,20 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None:
|
||||
return hook
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Populate star_systems.gttr_hook from wiki gttr.md files"
|
||||
)
|
||||
parser.add_argument("--db", default=str(DB_PATH), help="Path to systems.db")
|
||||
parser.add_argument(
|
||||
"--system",
|
||||
help="Process only this system_id (e.g. 'GJ 71')",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-words",
|
||||
type=int,
|
||||
default=45,
|
||||
help="Soft cap on hook length (default: 45 words)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Extract and print but do not write to the DB",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
help="Print every extracted hook",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
db_path = Path(args.db)
|
||||
def run(
|
||||
db_path: Path = DB_PATH,
|
||||
*,
|
||||
system: str | None = None,
|
||||
max_words: int = 45,
|
||||
dry_run: bool = False,
|
||||
verbose: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Populate star_systems.gttr_hook. Returns updated/missing/unmatched counts."""
|
||||
if not db_path.exists():
|
||||
print(f"error: {db_path} not found", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
raise ReachError(
|
||||
f"{db_path} not found",
|
||||
fix="pass --db with an existing systems.db, or restore it: git restore server/data/systems.db",
|
||||
)
|
||||
|
||||
conn = sqlite3.connect(str(db_path), timeout=30.0)
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
@@ -144,22 +119,21 @@ def main():
|
||||
except sqlite3.OperationalError:
|
||||
pass # column already exists
|
||||
|
||||
print("\n populate_gttr_hook.py")
|
||||
print(f" DB: {db_path}")
|
||||
print(f" max words: {args.max_words}")
|
||||
if args.dry_run:
|
||||
print(" Mode: DRY RUN")
|
||||
print()
|
||||
console.event(
|
||||
f"gttr-hook into {db_path} (max {max_words} words)" + (" — DRY RUN" if dry_run else "")
|
||||
)
|
||||
|
||||
rows = conn.execute(
|
||||
"SELECT system_id, proper_name FROM star_systems ORDER BY system_id"
|
||||
).fetchall()
|
||||
if args.system:
|
||||
rows = [r for r in rows if r[0] == args.system]
|
||||
rows = conn.execute("SELECT system_id, proper_name FROM star_systems ORDER BY system_id").fetchall()
|
||||
if system:
|
||||
rows = [r for r in rows if r[0] == system]
|
||||
if not rows:
|
||||
conn.close()
|
||||
raise ReachError(
|
||||
f"no system {system!r} in star_systems",
|
||||
fix="pass a system_id with its space, e.g. --system 'GJ 71'",
|
||||
)
|
||||
|
||||
updated = 0
|
||||
missing = 0
|
||||
unmatched = 0
|
||||
counts = {"updated": 0, "missing": 0, "unmatched": 0}
|
||||
|
||||
conn.execute("BEGIN")
|
||||
try:
|
||||
@@ -167,31 +141,26 @@ def main():
|
||||
slug = system_id.replace(" ", "-", 1)
|
||||
gttr_path = WIKI_SYSTEMS / slug / "gttr.md"
|
||||
|
||||
hook = extract_hook(gttr_path, args.max_words)
|
||||
hook = extract_hook(gttr_path, max_words)
|
||||
if hook is None:
|
||||
if not gttr_path.exists():
|
||||
missing += 1
|
||||
if args.verbose:
|
||||
print(f" MISSING {system_id:10s} {gttr_path}")
|
||||
else:
|
||||
unmatched += 1
|
||||
if args.verbose:
|
||||
print(f" UNMATCHED {system_id:10s} {gttr_path}")
|
||||
kind = "missing" if not gttr_path.exists() else "unmatched"
|
||||
counts[kind] += 1
|
||||
if verbose:
|
||||
console.event(f"{kind.upper()} {system_id} {gttr_path}", level="warn")
|
||||
continue
|
||||
|
||||
if args.verbose:
|
||||
if verbose:
|
||||
display = f"{system_id} ({proper_name})" if proper_name else system_id
|
||||
print(f" {display}")
|
||||
print(f" → {hook}")
|
||||
console.out(f"{display}\n → {hook}")
|
||||
|
||||
if not args.dry_run:
|
||||
if not dry_run:
|
||||
conn.execute(
|
||||
"UPDATE star_systems SET gttr_hook = ? WHERE system_id = ?",
|
||||
(hook, system_id),
|
||||
)
|
||||
updated += 1
|
||||
counts["updated"] += 1
|
||||
|
||||
if args.dry_run:
|
||||
if dry_run:
|
||||
conn.rollback()
|
||||
else:
|
||||
conn.commit()
|
||||
@@ -201,15 +170,4 @@ def main():
|
||||
raise
|
||||
|
||||
conn.close()
|
||||
|
||||
print("\n Done:")
|
||||
print(f" updated: {updated}")
|
||||
print(f" missing: {missing}")
|
||||
print(f" unmatched: {unmatched}")
|
||||
if args.dry_run:
|
||||
print("\n Dry run — no DB writes.")
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
return counts
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Transport for the `wiki` domain — args in, delegate, format out.
|
||||
|
||||
Two verbs, deliberately not four. `wiki_sync` also holds `generate_wiki()`
|
||||
(systems.db → system pages) and `import_from_wiki()` (pages → systems.db), and
|
||||
neither is exposed: re-rendering today would delete ~10,700 lines of the
|
||||
committed pages (see wiki_sync's docstring, T-1290). A verb is a promise that
|
||||
running it is safe, and that one would not be. They come back when the
|
||||
renderer reproduces the committed pages byte-for-byte.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from tooling.core import cli, console
|
||||
from tooling.core.command import command
|
||||
|
||||
app = cli.domain("wiki", "The wiki seed — fill rates and the GTTR hook.")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _domain() -> None:
|
||||
"""Keeps `wiki` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@app.command("stats")
|
||||
@command
|
||||
def stats() -> None:
|
||||
"""Fill rates of the structured star-system columns in systems.db."""
|
||||
from tooling.domains.wiki import wiki_sync
|
||||
|
||||
wiki_sync.stats()
|
||||
|
||||
|
||||
@app.command("gttr-hook")
|
||||
@command
|
||||
def gttr_hook(
|
||||
db: Path = typer.Option(None, "--db", help="systems.db to write (default: server/data/systems.db)."),
|
||||
system: str = typer.Option(None, "--system", help="Only this system_id, e.g. 'GJ 71'."),
|
||||
max_words: int = typer.Option(45, "--max-words", help="Soft cap on hook length."),
|
||||
dry_run: bool = typer.Option(False, "--dry-run", help="Extract, write nothing."),
|
||||
verbose: bool = typer.Option(False, "--verbose", help="Print every extracted hook."),
|
||||
) -> None:
|
||||
"""Extract each system's GTTR opening line into star_systems.gttr_hook.
|
||||
|
||||
Reads wiki/star-systems/<GJ-id>/gttr.md, takes the first **NAME**
|
||||
paragraph, collapses and caps it. Re-run after editing any gttr.md.
|
||||
"""
|
||||
from tooling.domains.wiki import gttr_hook as service
|
||||
|
||||
counts = service.run(
|
||||
db or service.DB_PATH,
|
||||
system=system,
|
||||
max_words=max_words,
|
||||
dry_run=dry_run,
|
||||
verbose=verbose,
|
||||
)
|
||||
written = "extracted, nothing written (dry run)" if dry_run else "written"
|
||||
console.verdict(
|
||||
f"gttr-hook: {counts['updated']} {written}, "
|
||||
f"{counts['missing']} missing gttr.md, {counts['unmatched']} unmatched"
|
||||
)
|
||||
@@ -1,4 +1,3 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Settled Reach Wiki Sync — wiki frontmatter ↔ systems.db
|
||||
|
||||
@@ -13,6 +12,18 @@ This module provides:
|
||||
- generate_wiki(): render wiki pages from DB + existing prose
|
||||
- stats(): show completion stats
|
||||
|
||||
**generate_wiki() is NOT safe to run against the committed wiki** (T-1290).
|
||||
Re-rendering today deletes ~10,700 lines across all 301 system pages: the
|
||||
Celestial Bodies / Stations blocks (written by the Rust atlas CLI's
|
||||
sync_wiki.rs, which this renderer knows nothing about), the Industries /
|
||||
Exports / Imports rows (which nothing in the repo writes any more), and it
|
||||
rewrites star types where systems.db and the pages disagree. That is why
|
||||
`reach wiki` exposes only `stats` and `gttr-hook`. The functions stay, because
|
||||
fixing the renderer means fixing them, not rewriting them from nothing.
|
||||
|
||||
Formerly tooling/db/wiki_sync.py, run by hand or via the now-archived
|
||||
process-wiki-system-changes pipeline.
|
||||
|
||||
Database: server/data/systems.db
|
||||
Schema: server/data/systems-schema.sql
|
||||
"""
|
||||
@@ -20,15 +31,15 @@ Schema: server/data/systems-schema.sql
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from tooling.core import config, console
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
WORKTREE_ROOT = (SCRIPT_DIR / ".." / "..").resolve()
|
||||
WORKTREE_ROOT = config.repo_root()
|
||||
SCHEMA_PATH = WORKTREE_ROOT / "server" / "data" / "systems-schema.sql"
|
||||
WIKI_DIR = WORKTREE_ROOT / "wiki" / "star-systems"
|
||||
DB_PATH = WORKTREE_ROOT / "server" / "data" / "systems.db"
|
||||
@@ -613,14 +624,16 @@ def generate_wiki(prose_only_update=False):
|
||||
# Topology (always last)
|
||||
page_lines.append(topology)
|
||||
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
outfile.write_text("\n".join(page_lines), encoding="utf-8")
|
||||
|
||||
# Counted BEFORE the write. Checking exists() after writing (as this
|
||||
# did until T-1290) made every page an update and `created` always 0.
|
||||
if outfile.exists():
|
||||
updated += 1
|
||||
else:
|
||||
created += 1
|
||||
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
outfile.write_text("\n".join(page_lines), encoding="utf-8")
|
||||
|
||||
conn.close()
|
||||
return {
|
||||
"ok": True,
|
||||
@@ -635,19 +648,26 @@ def generate_wiki(prose_only_update=False):
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def stats():
|
||||
"""Show per-table fill rates."""
|
||||
def stats() -> dict:
|
||||
"""Per-table fill rates for the structured system columns.
|
||||
|
||||
The table is the command's output, so it goes to stdout (console.out).
|
||||
A missing table used to print a hint and exit 0 — a failure reading as
|
||||
success — and now raises with the remedy (T-1290).
|
||||
"""
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
conn = get_connection()
|
||||
try:
|
||||
row = conn.execute("SELECT COUNT(*) as cnt FROM star_systems").fetchone()
|
||||
total = row["cnt"]
|
||||
if total == 0:
|
||||
print("No systems in database. Run import first.")
|
||||
console.out("No systems in database.")
|
||||
return {"ok": True, "total": 0}
|
||||
|
||||
print(f"Star systems: {total}")
|
||||
print(f"Database: {DB_PATH}")
|
||||
print()
|
||||
console.out(f"Star systems: {total}")
|
||||
console.out(f"Database: {DB_PATH}")
|
||||
console.out()
|
||||
|
||||
total_cells = 0
|
||||
filled_cells = 0
|
||||
@@ -665,64 +685,24 @@ def stats():
|
||||
filled_cells += table_filled
|
||||
pct = table_filled / table_total * 100 if table_total > 0 else 0
|
||||
bar = "#" * int(pct / 5) + "." * (20 - int(pct / 5))
|
||||
print(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%")
|
||||
console.out(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%")
|
||||
|
||||
# Historical events
|
||||
ev_count = conn.execute("SELECT COUNT(*) as cnt FROM historical_events").fetchone()["cnt"]
|
||||
ev_systems = conn.execute(
|
||||
"SELECT COUNT(DISTINCT system_id) as cnt FROM historical_events"
|
||||
).fetchone()["cnt"]
|
||||
print(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems")
|
||||
console.out(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems")
|
||||
|
||||
overall = filled_cells / total_cells * 100 if total_cells > 0 else 0
|
||||
print(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)")
|
||||
console.out(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)")
|
||||
|
||||
return {"ok": True, "total": total, "filled": filled_cells, "pct": round(overall, 1)}
|
||||
|
||||
except sqlite3.OperationalError as exc:
|
||||
print(f"Table not found — run import first: {exc}")
|
||||
return {"ok": False, "error": str(exc)}
|
||||
raise ReachError(
|
||||
f"systems.db is missing a system table: {exc}",
|
||||
fix="make regen-db, or restore it: git restore server/data/systems.db",
|
||||
) from exc
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def main():
|
||||
help_text = f"""\
|
||||
Wiki Sync — systems.db ↔ wiki pages
|
||||
|
||||
Usage:
|
||||
wiki_sync.py import Parse wiki frontmatter into systems.db (bootstrap)
|
||||
wiki_sync.py generate Generate wiki pages from systems.db
|
||||
wiki_sync.py stats Show completion stats
|
||||
wiki_sync.py --help Show this help
|
||||
|
||||
Database: {DB_PATH}
|
||||
Schema: {SCHEMA_PATH}
|
||||
Wiki: {WIKI_DIR}
|
||||
"""
|
||||
if len(sys.argv) < 2 or sys.argv[1] in ("--help", "-h", "help"):
|
||||
print(help_text)
|
||||
sys.exit(0)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "import":
|
||||
result = import_from_wiki()
|
||||
print(json.dumps(result, indent=2))
|
||||
elif cmd == "generate":
|
||||
result = generate_wiki()
|
||||
print(json.dumps(result, indent=2))
|
||||
elif cmd == "stats":
|
||||
stats()
|
||||
else:
|
||||
print(json.dumps({"ok": False, "error": f"Unknown: {cmd}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,96 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/godot-cold-parse [--run-menu] — cold-cache headless parse check.
|
||||
#
|
||||
# Used by /pr-process step 1c before push. Deletes the cached script-class
|
||||
# registry so the parse simulates the cold-start ordering CI / fresh clones
|
||||
# see: Sprint 36 close caught a new `class_name MetaScreen` base class and
|
||||
# six extending scripts that parsed fine on warm developer caches but hit
|
||||
# `Could not find base class "MetaScreen"` post-merge, because the
|
||||
# autoload-vs-class_name registration order only resolves correctly once the
|
||||
# class cache is seeded (see CLAUDE.md -> GDScript conventions -> Autoload
|
||||
# parse-order rule).
|
||||
#
|
||||
# Godot's resource scanner emits category errors (e.g. "Export type can only
|
||||
# be built-in, a resource, a node, or an enum") that do NOT always prefix
|
||||
# with SCRIPT ERROR — they appear as plain ERROR lines. The filter below
|
||||
# catches both, then drops known pre-existing noise from the autoload
|
||||
# class_name parse-order trap. Sprint 36 shipped a scanner error the old
|
||||
# narrower grep missed; this is why the filter stays wide.
|
||||
#
|
||||
# --run-menu: also launch main_menu.tscn briefly (for branches with UI changes).
|
||||
#
|
||||
# Exit 0 + "clean" if no matches. Exit 1 + the matched lines if any are found.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
RUN_MENU=false
|
||||
[ "${1:-}" = "--run-menu" ] && RUN_MENU=true
|
||||
|
||||
rm -f "$REPO_ROOT/client/.godot/global_script_class_cache.cfg"
|
||||
|
||||
# A truly cold checkout (fresh clone or worktree — .godot/ is gitignored) has
|
||||
# no resource-import cache, and every imported asset (fonts, ogg) then "fails
|
||||
# loading" during the parse run: a wall of false positives. Seed the cache
|
||||
# with an import pass first; source assets are tracked, so this is always
|
||||
# reconstructible. (Found live: first run in a fresh worktree, 2026-07-13.)
|
||||
if [ ! -d "$REPO_ROOT/client/.godot/imported" ] || [ -z "$(ls -A "$REPO_ROOT/client/.godot/imported" 2>/dev/null)" ]; then
|
||||
echo "godot-cold-parse: no import cache — running one-time import pass..." >&2
|
||||
set +e
|
||||
IMPORT_OUT=$(godot --headless --path "$REPO_ROOT/client" --import 2>&1)
|
||||
IMPORT_EXIT=$?
|
||||
set -e
|
||||
if [ "$IMPORT_EXIT" -ne 0 ]; then
|
||||
echo "godot-cold-parse: import pass exited $IMPORT_EXIT" >&2
|
||||
printf '%s\n' "$IMPORT_OUT" | tail -20 >&2
|
||||
exit "$IMPORT_EXIT"
|
||||
fi
|
||||
fi
|
||||
|
||||
FILTER='^(SCRIPT )?ERROR|Parse Error|Export type'
|
||||
|
||||
# Capture the godot run separately from the filter pipeline: with the
|
||||
# trailing `|| true` on the greps, a nonzero exit from godot itself (crash,
|
||||
# missing binary, corrupted install) would otherwise report "clean". Nothing
|
||||
# downstream re-reads the raw output now that this is scripted, so fail loud.
|
||||
set +e
|
||||
RAW=$(godot --headless --path "$REPO_ROOT/client" --quit 2>&1)
|
||||
GODOT_EXIT=$?
|
||||
set -e
|
||||
if [ "$GODOT_EXIT" -ne 0 ]; then
|
||||
echo "godot-cold-parse: godot itself exited $GODOT_EXIT — not a parse verdict" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit "$GODOT_EXIT"
|
||||
fi
|
||||
|
||||
MATCHES=$(printf '%s\n' "$RAW" \
|
||||
| grep -iE "$FILTER" \
|
||||
| grep -v "Failed loading resource: res://assets" \
|
||||
| grep -v "Cannot infer the type" \
|
||||
| grep -vE '(Messagepack|LocalBridge|ServerProcess|Constants)" not declared' || true)
|
||||
|
||||
if [ "$RUN_MENU" = true ]; then
|
||||
# Deliberately no exit-code check here: `timeout` kills the menu after
|
||||
# 10s by design (exit 124 is the expected shutdown path); only the
|
||||
# scraped error lines carry signal for this bounded run.
|
||||
MENU_MATCHES=$(timeout 10 godot --path "$REPO_ROOT/client" res://scenes/main_menu.tscn 2>&1 \
|
||||
| grep -iE "$FILTER" || true)
|
||||
if [ -n "$MENU_MATCHES" ]; then
|
||||
MATCHES="$MATCHES
|
||||
$MENU_MATCHES"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -n "$MATCHES" ]; then
|
||||
echo "$MATCHES"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Restore a FULL class cache before exiting: the cold parse run re-seeds
|
||||
# global_script_class_cache.cfg only partially — addon classes (e.g. gdUnit4's
|
||||
# GdUnitTestCIRunner) are missing, which leaves tests/run-godot unable to even
|
||||
# start (0 tests in ~350ms; found live when the pre-push gate ran the suite
|
||||
# right after this script, 2026-07-14). The cold verdict above is already
|
||||
# decided; this restore just returns the tree to a runnable state.
|
||||
godot --headless --path "$REPO_ROOT/client" --import > /dev/null 2>&1 || true
|
||||
|
||||
echo "godot-cold-parse: clean"
|
||||
@@ -1,64 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/godot-parse-sweep — open every project .gd and fail on any that won't parse.
|
||||
#
|
||||
# Complements `godot-cold-parse`, which does NOT cover this. That script only
|
||||
# ever sees scripts on the STARTUP path (autoloads + the main scene chain) —
|
||||
# correct for the registration-ORDER bug it was built for, far narrower than
|
||||
# its name implies. Verified 2026-07-27 by breaking a non-startup UI script
|
||||
# and a test file in turn: cold-parse reported "clean", exit 0, for both.
|
||||
#
|
||||
# Division of labour, deliberate:
|
||||
# godot-cold-parse — cold-cache STARTUP ordering (class_name/autoload race)
|
||||
# godot-parse-sweep — does every file in the project parse at all
|
||||
#
|
||||
# The GDScript half (client/tools/parse_sweep.gd) only opens files; it makes no
|
||||
# verdict, because no Godot API reports GDScript parse failure reliably (see
|
||||
# that file's header — one segfaults, one false-positives 150/226, and plain
|
||||
# load() returns non-null for a broken script). The engine's own stderr is the
|
||||
# only honest signal, so the verdict is made here by scraping it.
|
||||
#
|
||||
# Exit 0 + "clean" if every script parsed. Exit 1 + the offending lines if not.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
|
||||
set +e
|
||||
RAW=$(godot --headless --path "$REPO_ROOT/client" -s res://tools/parse_sweep.gd 2>&1)
|
||||
GODOT_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ "$GODOT_EXIT" -ne 0 ]; then
|
||||
echo "godot-parse-sweep: godot itself exited $GODOT_EXIT — not a parse verdict" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit "$GODOT_EXIT"
|
||||
fi
|
||||
|
||||
# The sweep must actually have run. Without this, a future change that breaks
|
||||
# the walk (or renames the script) would produce zero error lines and read as
|
||||
# a clean sweep — the same false-green shape this tool exists to close.
|
||||
if ! printf '%s\n' "$RAW" | grep -q '^parse-sweep: opened'; then
|
||||
echo "godot-parse-sweep: the sweep did not report completion — no verdict possible" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit 1
|
||||
fi
|
||||
SUMMARY=$(printf '%s\n' "$RAW" | grep '^parse-sweep: opened')
|
||||
|
||||
# NOTE: unlike godot-cold-parse, "Cannot infer the type" is NOT filtered here.
|
||||
# That filter is why cold-parse stayed silent about tests/util/scene_helper.gd,
|
||||
# which genuinely does not parse — the suppressed class was hiding a real
|
||||
# failure, not noise.
|
||||
MATCHES=$(printf '%s\n' "$RAW" \
|
||||
| grep -E 'Parse Error|Failed to load script' \
|
||||
| grep -v "Failed loading resource: res://assets" || true)
|
||||
|
||||
if [ -n "$MATCHES" ]; then
|
||||
echo "PARSE SWEEP FAILED — at least one script does not parse." >&2
|
||||
printf '%s\n' "$MATCHES" | head -40 >&2
|
||||
echo "" >&2
|
||||
echo " ${SUMMARY}" >&2
|
||||
echo " An unparseable file cannot run. If it is a test suite, it did not" >&2
|
||||
echo " execute, and any pass count reported elsewhere excludes it." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "godot-parse-sweep: clean — ${SUMMARY#parse-sweep: }"
|
||||
@@ -63,6 +63,10 @@ DOMAINS: dict[str, tuple[str, str]] = {
|
||||
"tooling.domains.ledger.router:app",
|
||||
"The economics pipeline — the import that builds systems.db",
|
||||
),
|
||||
"wiki": (
|
||||
"tooling.domains.wiki.router:app",
|
||||
"The wiki seed — fill rates and the GTTR hook",
|
||||
),
|
||||
"godot": (
|
||||
"tooling.domains.godot.router:app",
|
||||
"Does the client parse, and does it parse cold",
|
||||
|
||||
Reference in New Issue
Block a user