refactor(tooling): T-1290 — the wiki domain, and the renderer that must not run
`reach wiki stats` and `reach wiki gttr-hook` replace tooling/db/wiki_sync.py and populate_gttr_hook.py. Both are output-identical to the originals: `stats` byte-for-byte, and all 301 extracted GTTR hooks line-for-line. wiki_sync.py moved whole, but generate_wiki() and import_from_wiki() are NOT verbs. Before porting, the old `--generate` was run against a clean tree to get a parity baseline. It changed all 301 system pages, +940 / -10,761, and was reverted at once. It deletes the Celestial Bodies / Stations blocks (owned by the Rust atlas sync, which it does not know about), deletes the Industries / Exports / Imports rows (nothing writes those any more), and rewrites star types where systems.db and the pages disagree. D-262, CLAUDE.md and the wiki skill all described it as the routine, prose-preserving render. CLAUDE.md and the skill now say not to run it; D-262 needs amending — T-1292. Provenance moves to tooling/archive/, with a README naming what each script did and why it is not run: - pql-migrate/ (the T-1271 ruling) - wiki-bootstrap/: assign-astro-ids + its catalog, migrate-s-to-gj, patch-core-sector (hardcodes a dead path), fill-missing-globes, generate-stubs and find-stubs (finds 0 stubs — Phase 1 is done), backfill_cultural_corridor (a raw systems.db patch script, outside D-262), and process-wiki-system-changes, whose last step is the destructive render Also: - stats() printed "run import first" and exited 0 when a table was missing; it now fails with a remedy. generate_wiki() counted created pages after writing them, so `created` was always 0. - tooling/godot-cold-parse and godot-parse-sweep were never retired after T-1283, and the pr-process skill still told agents to run them. Removed; the skill and parse_sweep.gd now name the reach verbs. - systems.db re-stamped: schema comments changed, and the stamp records the schema file's SHA for tamper detection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -17,7 +17,7 @@ wiki/ # THE SEED (11,864 files, the largest tree here). Authored
|
||||
economics/ # INPUT. TOML vocabularies + schema.md. Seeds systems.db
|
||||
# AND, from Phase 4 on, world content generation
|
||||
star-systems/ # OUTPUT, 93% of the wiki. Generated by
|
||||
# tooling/db/wiki_sync.py FROM systems.db + star-map.json
|
||||
# tooling/domains/wiki/wiki_sync.py FROM systems.db + star-map.json
|
||||
GJ-{id}/ # system dir (hyphenated id)
|
||||
index.md # no frontmatter; <!-- READ-ONLY --> blocks are
|
||||
# regenerated — authored prose lives in the named
|
||||
@@ -64,7 +64,7 @@ tooling/
|
||||
config.json # Endpoint configuration
|
||||
common.py # Shared venv/config helpers
|
||||
audio_connector.py # Stable Audio Open connector
|
||||
pql-migrate/ # One-shot scripts from the pql migration (provenance)
|
||||
archive/ # Provenance only, never run: pql-migrate/, wiki-bootstrap/
|
||||
.claude/
|
||||
agents/ # Agent personality files
|
||||
skills/ # Skill definitions
|
||||
|
||||
@@ -108,14 +108,14 @@ bugs (parse errors, depth sorting, scene tree failures).
|
||||
|
||||
**For client/visual branches:**
|
||||
|
||||
`tooling/godot-cold-parse` wipes the cached script-class registry (matching
|
||||
`reach godot cold-parse` wipes the cached script-class registry (matching
|
||||
the cold-start ordering CI / fresh clones see — Sprint 36 close caught a
|
||||
`class_name` base-class registration bug that warm caches masked) and runs a
|
||||
headless parse, filtering known pre-existing noise. Pass `--run-menu` if the
|
||||
branch has UI changes to also launch the main menu briefly:
|
||||
|
||||
```bash
|
||||
tooling/godot-cold-parse --run-menu
|
||||
reach godot cold-parse --run-menu
|
||||
```
|
||||
|
||||
If it reports a "Could not find base class X" error, the fix is almost
|
||||
@@ -123,7 +123,7 @@ always an autoload-order issue (see `CLAUDE.md` → GDScript conventions →
|
||||
Autoload parse-order rule) — fix the ordering, don't just rebuild the cache
|
||||
to mask it locally (the same error resurfaces post-merge).
|
||||
|
||||
**Scope, so you don't over-trust it:** `godot-cold-parse` only ever sees
|
||||
**Scope, so you don't over-trust it:** `reach godot cold-parse` only ever sees
|
||||
scripts on the **startup path** — autoloads and the main scene chain. That is
|
||||
exactly right for the registration-ORDER bug it exists to catch, and far
|
||||
narrower than the name suggests: it reports `clean` for a syntactically
|
||||
@@ -132,7 +132,7 @@ by breaking one of each). It also filters `"Cannot infer the type"`, which
|
||||
hid a genuinely unparseable file for five months.
|
||||
|
||||
**"Does every script parse?" is a different question, and the pre-push hook
|
||||
now answers it** — `tooling/godot-parse-sweep` opens all ~226 project scripts
|
||||
now answers it** — `reach godot parse-sweep` opens all ~226 project scripts
|
||||
and blocks the push on any that don't, running *before* the test suite so a
|
||||
parse failure costs 4s instead of 135s. It is enforced by the gate, so do
|
||||
**not** run it here as well (`team-patterns.md`: don't duplicate the push
|
||||
|
||||
@@ -32,8 +32,9 @@ the two disagree, the diagram is right.
|
||||
### `star-systems/` — DB owns structure, wiki owns prose (OUTPUT)
|
||||
|
||||
3,262 pages, 93% of the wiki. Generated FROM `server/data/systems.db` and
|
||||
`star-map.json` by `tooling/db/wiki_sync.py`, with `scaffold_bodies.py` and the
|
||||
atlas sync alongside.
|
||||
`star-map.json` by `tooling/domains/wiki/wiki_sync.py`, with `scaffold_bodies.py`
|
||||
and the atlas sync (`server/src/bin/atlas/sync_wiki.rs`, which owns the
|
||||
Celestial Bodies / Stations blocks) alongside.
|
||||
|
||||
**Never hand-edit:**
|
||||
- anything inside a `<!-- READ-ONLY -->` block (System Profile, Topology,
|
||||
@@ -42,9 +43,13 @@ atlas sync alongside.
|
||||
|
||||
**System pages and BODY pages behave differently. Do not merge them in your head.**
|
||||
|
||||
*System page* (`GJ-{id}/index.md`) — `wiki_sync.py::generate_wiki()` re-renders it
|
||||
from `systems.db` + `star-map.json`, preserving the authored prose sections. A
|
||||
hand-edit inside a `<!-- READ-ONLY -->` block IS reverted on the next sync.
|
||||
*System page* (`GJ-{id}/index.md`) — `wiki_sync.py::generate_wiki()` was written
|
||||
to re-render it from `systems.db` + `star-map.json`, preserving the authored
|
||||
prose sections. **It is broken against today's pages — do not run it**
|
||||
(measured 2026-09-23, T-1290): it deletes the Celestial Bodies / Stations blocks,
|
||||
the Industries / Exports / Imports rows (which nothing writes any more), and
|
||||
rewrites star types where the DB and the pages disagree — ~10,700 lines across
|
||||
all 301 pages. `reach wiki` therefore exposes only `stats` and `gttr-hook`.
|
||||
(`wiki_sync.py` also has `import_from_wiki()`, a bootstrap/migration path running
|
||||
the other way — wiki frontmatter into the DB. Know it exists before assuming any
|
||||
single direction is "the" direction.)
|
||||
@@ -199,10 +204,10 @@ session once already.
|
||||
|
||||
| tool | does |
|
||||
|---|---|
|
||||
| `tooling/db/wiki_sync.py` | systems.db → star-system page sections |
|
||||
| `tooling/domains/wiki/wiki_sync.py` | systems.db → star-system page sections (renderer broken — see above) |
|
||||
| `reach atlas planet scaffold` | creates body dirs/pages |
|
||||
| `tooling/domains/atlas/planet/body_definition_parser.py` | reads body frontmatter |
|
||||
| `tooling/db/populate_gttr_hook.py` | GTTR prose → `gttr_hook` |
|
||||
| `reach wiki gttr-hook` | GTTR prose → `gttr_hook` |
|
||||
| `reach atlas planet terrain-reference` | terrain asset paths |
|
||||
| `reach atlas planet audit` | audits atlas coherence |
|
||||
| `make regen-db` | economics TOML + corp frontmatter → systems.db |
|
||||
|
||||
@@ -403,3 +403,70 @@ Standard port acceptance as on T-1281/T-1286.
|
||||
|
||||
DONE 2026-09-23. tooling/economy-db/ + schema_version.py -> tooling/domains/ledger/ (economy_import/ kept by name, since server/src Rust comments cite economy_import/*.py; import_economics.py became service.py). reach ledger import [--db] [--dry-run] [--strict-specialization]. Evidence: generated_brands.toml sha256 identical before/after (e748531…); stamp gate reported STALE after the move and OK after regen; dry-run transcript carries every count and warning of the baseline; exit 2 (coverage gap, data committed) proven to relay through @command with a probe. Make: regen-db kept as a one-line delegate per D-263''s muscle-memory clause (~50 files name it, incl. generated TOML headers and gate remedies); economy-db retired (it ran generate brands, which the import already does first). Reconciled economy_import.errors (DOMAINS.md ask): ImportAborted stays internal rollback control flow, converted to ReachError at the service boundary. Also: regenerate_brands used to swallow cargo_binary''s ReachError into ImportAborted, dropping its remedy — now propagates (it runs before the transaction, nothing to roll back). Both sys.path bootstraps removed. Step labels were inconsistent ([1/10]..[10/13]..[17/19]); one 24-step counter now drives phase + progress. Stale doc pointers fixed while there: MIGRATION_SQL had lived in economy_import/migration.py since T-1067 but rules/DEVOPS still said import_economics.py; asset-pipeline rule and DEVOPS still named the check-systems-db script/target T-1281 retired.', NULL, '2026-09-23 14:18:49', '2026-09-23 14:18:49.681', '2026-09-23 14:18:49.681', NULL, '13300d7ea5bd79ae4857cd59399531a6', 2) ON CONFLICT(hash) DO NOTHING;
|
||||
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65T2N1WAT491WY8SV736J3G', 'status', 'backlog', 'done', NULL, '2026-09-23 14:18:50', '2026-09-23 14:18:50.093', '2026-09-23 14:18:50.093', NULL, '63d24743b229b1544d62104c4e68ca66', 2) ON CONFLICT(hash) DO NOTHING;
|
||||
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'description', 'The tail of the tree — three domains, ~26 files, batched because none of them
|
||||
entangles with the spatial ladder or the systems.db stamp.
|
||||
|
||||
`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`,
|
||||
`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`,
|
||||
`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`,
|
||||
`process-wiki-system-changes`.
|
||||
READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite
|
||||
directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is
|
||||
OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are
|
||||
INPUT. A verb that writes to the wrong side is silently reverted on the next sync.
|
||||
Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) —
|
||||
decide per file whether it is a live verb or provenance bound for `tooling/archive/`,
|
||||
the same call T-1271 made for `pql-migrate/`.
|
||||
|
||||
`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`,
|
||||
`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`.
|
||||
These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by
|
||||
IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json`
|
||||
— that config must keep working or move deliberately, not incidentally. Network
|
||||
failure is the normal case here, so `ReachError` messages need to distinguish
|
||||
"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter.
|
||||
|
||||
`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`,
|
||||
`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`,
|
||||
`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`,
|
||||
`setup_clothing_metadata.py`.
|
||||
Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender
|
||||
payloads that belong to the carve-out (T-1273) and MUST NOT import the package.
|
||||
Do T-1273 first or the boundary will blur.
|
||||
|
||||
Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots
|
||||
before moving anything — five silent failures so far.', 'The tail of the tree — three domains, ~26 files, batched because none of them
|
||||
entangles with the spatial ladder or the systems.db stamp.
|
||||
|
||||
`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`,
|
||||
`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`,
|
||||
`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`,
|
||||
`process-wiki-system-changes`.
|
||||
READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite
|
||||
directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is
|
||||
OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are
|
||||
INPUT. A verb that writes to the wrong side is silently reverted on the next sync.
|
||||
Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) —
|
||||
decide per file whether it is a live verb or provenance bound for `tooling/archive/`,
|
||||
the same call T-1271 made for `pql-migrate/`.
|
||||
|
||||
`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`,
|
||||
`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`.
|
||||
These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by
|
||||
IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json`
|
||||
— that config must keep working or move deliberately, not incidentally. Network
|
||||
failure is the normal case here, so `ReachError` messages need to distinguish
|
||||
"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter.
|
||||
|
||||
`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`,
|
||||
`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`,
|
||||
`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`,
|
||||
`setup_clothing_metadata.py`.
|
||||
Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender
|
||||
payloads that belong to the carve-out (T-1273) and MUST NOT import the package.
|
||||
Do T-1273 first or the boundary will blur.
|
||||
|
||||
Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots
|
||||
before moving anything — five silent failures so far.
|
||||
|
||||
PROGRESS 2026-09-23 — wiki half done (commit to follow): reach wiki stats + gttr-hook, both output-identical to the originals (stats byte-identical; all 301 gttr hooks identical). wiki_sync.generate_wiki/import_from_wiki moved but NOT exposed: a pre-port run of the old --generate deleted ~10,700 lines of committed pages -> T-1292. Archive: tooling/archive/{pql-migrate,wiki-bootstrap}/ with a README (one-shots + the destructive process-wiki-system-changes + backfill_cultural_corridor, a raw DB patch script outside D-262). find-stubs archived too: it finds 0 stubs, Phase-1 job done. Side fix: tooling/godot-cold-parse + godot-parse-sweep were never retired after T-1283 and pr-process still told agents to run them — removed, skill repointed. Remaining: assets, character.', NULL, '2026-09-23 14:25:42', '2026-09-23 14:25:42.440', '2026-09-23 14:25:42.440', NULL, '50093acafd657096ec7f86acfa339f78', 2) ON CONFLICT(hash) DO NOTHING;
|
||||
|
||||
@@ -6,3 +6,4 @@ INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_
|
||||
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65T2N1WAT491WY8SV736J3G', 'T-1289', '2026-09-02 16:16:28.431', '2026-09-02 16:16:28.431', NULL, '976a8d383d0581caafca5a4bc2e0fb61', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
|
||||
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'T-1290', '2026-09-02 16:17:45.770', '2026-09-02 16:17:45.770', NULL, '748d7ed02f4821a7ee5059a76d2fd3fe', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
|
||||
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX60QRMD7KJ1TVWTR7FWK28', 'T-1291', '2026-09-23 14:07:49.190', '2026-09-23 14:07:49.190', NULL, '20abfe457c23ea4e00d3c8109f2e39f2', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
|
||||
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX9TTBJW4NPECEN6JM0V4HM', 'T-1292', '2026-09-23 14:24:29.282', '2026-09-23 14:24:29.282', NULL, '45174fb519d2deed098a244229d543e1', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
|
||||
|
||||
@@ -561,3 +561,39 @@ a bug, not a regen.
|
||||
Standard port acceptance as on T-1281/T-1286.
|
||||
|
||||
DONE 2026-09-23. tooling/economy-db/ + schema_version.py -> tooling/domains/ledger/ (economy_import/ kept by name, since server/src Rust comments cite economy_import/*.py; import_economics.py became service.py). reach ledger import [--db] [--dry-run] [--strict-specialization]. Evidence: generated_brands.toml sha256 identical before/after (e748531…); stamp gate reported STALE after the move and OK after regen; dry-run transcript carries every count and warning of the baseline; exit 2 (coverage gap, data committed) proven to relay through @command with a probe. Make: regen-db kept as a one-line delegate per D-263''s muscle-memory clause (~50 files name it, incl. generated TOML headers and gate remedies); economy-db retired (it ran generate brands, which the import already does first). Reconciled economy_import.errors (DOMAINS.md ask): ImportAborted stays internal rollback control flow, converted to ReachError at the service boundary. Also: regenerate_brands used to swallow cargo_binary''s ReachError into ImportAborted, dropping its remedy — now propagates (it runs before the transaction, nothing to roll back). Both sys.path bootstraps removed. Step labels were inconsistent ([1/10]..[10/13]..[17/19]); one 24-step counter now drives phase + progress. Stale doc pointers fixed while there: MIGRATION_SQL had lived in economy_import/migration.py since T-1067 but rules/DEVOPS still said import_economics.py; asset-pipeline rule and DEVOPS still named the check-systems-db script/target T-1281 retired.', 'done', 'high', NULL, 'tooling', 'D-263', '2026-09-02 16:16:28.431', '2026-09-23 14:18:50.093', NULL, '579a9be46ac9f07e3b7a687fbe4772fe', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
||||
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX9TTBJW4NPECEN6JM0V4HM', 'bug', '06FBPPMZNNEV052DBYYY3A897C', 'wiki_sync.generate_wiki() deletes ~10,700 lines of committed system pages — renderer out of step with the pages', 'Found in T-1290 (2026-09-23) by running the pre-port tool (process-wiki-system-changes --generate) against a clean tree and diffing: 301 pages changed, +940 / -10,761. Reverted immediately; nothing committed. What the render does, e.g. wiki/star-systems/GJ-1002/index.md: (1) DELETES the Celestial Bodies + Stations & Facilities READ-ONLY blocks — those are written by the Rust atlas CLI (server/src/bin/atlas/sync_wiki.rs), which generate_wiki() does not know about; (2) DELETES the infobox Industries / Exports / Imports / Population rows and replaces them with a Specialization row — nothing in the repo writes the old rows any more, so a render destroys content with no generator; (3) REWRITES star type (GJ 1002: page says G-type, systems.db says M-type — GJ 1002 is a real M dwarf, so the DB is probably right and the pages are stale; needs a decision which is truth); (4) reorders Adjacent Systems (sorted vs star-map order). Consequences: reach wiki exposes only stats + gttr-hook until this is fixed; wiki_sync.generate_wiki/import_from_wiki are kept in tooling/domains/wiki/wiki_sync.py but unexposed; process-wiki-system-changes archived. The wiki skill and CLAUDE.md now warn not to run it. ALSO: D-262 and its diagram (docs/diagrams/data-flow/wiki-generator-flow.d2) present wiki_sync generate_wiki as the ROUTINE DB->wiki path; that edge''s meaning is no longer true and D-262 needs amending (the dataflow gate only checks paths, it cannot catch this — exactly the gap D-262 names). Acceptance: a render of the current DB reproduces the committed pages byte-for-byte (or the diff is a reviewed, intended content change), then expose reach wiki render.', 'backlog', 'high', NULL, 'tooling', NULL, '2026-09-23 14:24:29.276', '2026-09-23 14:24:29.276', NULL, '8c8194e2723412aced90777f689dc7b7', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
||||
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port wiki, assets and character — the three small remaining domains', 'The tail of the tree — three domains, ~26 files, batched because none of them
|
||||
entangles with the spatial ladder or the systems.db stamp.
|
||||
|
||||
`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`,
|
||||
`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`,
|
||||
`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`,
|
||||
`process-wiki-system-changes`.
|
||||
READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite
|
||||
directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is
|
||||
OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are
|
||||
INPUT. A verb that writes to the wrong side is silently reverted on the next sync.
|
||||
Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) —
|
||||
decide per file whether it is a live verb or provenance bound for `tooling/archive/`,
|
||||
the same call T-1271 made for `pql-migrate/`.
|
||||
|
||||
`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`,
|
||||
`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`.
|
||||
These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by
|
||||
IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json`
|
||||
— that config must keep working or move deliberately, not incidentally. Network
|
||||
failure is the normal case here, so `ReachError` messages need to distinguish
|
||||
"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter.
|
||||
|
||||
`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`,
|
||||
`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`,
|
||||
`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`,
|
||||
`setup_clothing_metadata.py`.
|
||||
Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender
|
||||
payloads that belong to the carve-out (T-1273) and MUST NOT import the package.
|
||||
Do T-1273 first or the boundary will blur.
|
||||
|
||||
Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots
|
||||
before moving anything — five silent failures so far.
|
||||
|
||||
PROGRESS 2026-09-23 — wiki half done (commit to follow): reach wiki stats + gttr-hook, both output-identical to the originals (stats byte-identical; all 301 gttr hooks identical). wiki_sync.generate_wiki/import_from_wiki moved but NOT exposed: a pre-port run of the old --generate deleted ~10,700 lines of committed pages -> T-1292. Archive: tooling/archive/{pql-migrate,wiki-bootstrap}/ with a README (one-shots + the destructive process-wiki-system-changes + backfill_cultural_corridor, a raw DB patch script outside D-262). find-stubs archived too: it finds 0 stubs, Phase-1 job done. Side fix: tooling/godot-cold-parse + godot-parse-sweep were never retired after T-1283 and pr-process still told agents to run them — removed, skill repointed. Remaining: assets, character.', 'backlog', 'medium', NULL, 'tooling', 'D-263', '2026-09-02 16:17:45.770', '2026-09-23 14:25:42.440', NULL, 'f5425885aa0e5f99f744ab4cdaaf5d63', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
||||
|
||||
@@ -25,9 +25,11 @@ Full annotated tree: `.claude/rules/project-structure.md`
|
||||
**`wiki/` is the seed, and it is mostly generated.** 11,864 files — the largest
|
||||
tree in the repo. Two ownership models run in OPPOSITE directions, and mixing
|
||||
them up costs work: `star-systems/` (3,262 pages, 93% of the wiki) is produced
|
||||
FROM `systems.db` by `tooling/db/wiki_sync.py`, so its `<!-- READ-ONLY -->`
|
||||
sections and body-page frontmatter are OUTPUT — hand-edits are silently reverted
|
||||
on the next sync. The flat categories (`corporations/`, `factions/`, `economics/`,
|
||||
FROM `systems.db` by `tooling/domains/wiki/wiki_sync.py` plus the Rust atlas
|
||||
CLI's wiki sync, so its `<!-- READ-ONLY -->` sections and body-page frontmatter
|
||||
are OUTPUT — do not hand-edit them. (Do not run `wiki_sync.generate_wiki()`
|
||||
either: it no longer reproduces the committed pages and would delete ~10,700
|
||||
lines — T-1290.) The flat categories (`corporations/`, `factions/`, `economics/`,
|
||||
…, 253 pages) are INPUT the generators read. Body frontmatter IS the body
|
||||
definition (`seed`, `land_fraction`, `axial_tilt_deg`); `wiki/economics/` seeds
|
||||
world content generation, not just DB rows. Read `Skill(wiki)` before touching
|
||||
|
||||
@@ -2,7 +2,7 @@ extends SceneTree
|
||||
|
||||
## Parse-sweep: open EVERY project .gd so the engine reports the broken ones.
|
||||
##
|
||||
## Why this exists. `tooling/godot-cold-parse` only ever sees scripts on the
|
||||
## Why this exists. `reach godot cold-parse` only ever sees scripts on the
|
||||
## STARTUP path — autoloads and the main scene chain. That is the correct
|
||||
## scope for the job it was built for (Sprint 36's `Could not find base class
|
||||
## "MetaScreen"`, an autoload-vs-class_name registration ORDER bug), but it is
|
||||
@@ -18,7 +18,7 @@ extends SceneTree
|
||||
##
|
||||
## **This script deliberately makes no verdict of its own.** It opens every
|
||||
## script and lets Godot's own front-end print the diagnosis; the wrapper
|
||||
## (`tooling/godot-parse-sweep`) greps for that. Two rejected alternatives,
|
||||
## (`reach godot parse-sweep`) greps for that. Two rejected alternatives,
|
||||
## both found the hard way while building this:
|
||||
##
|
||||
## - `ResourceLoader.load(path, "GDScript", CACHE_MODE_IGNORE)` **segfaults
|
||||
@@ -34,7 +34,7 @@ extends SceneTree
|
||||
## either. The engine's stderr is the only honest signal, which is why the
|
||||
## verdict lives in the wrapper.
|
||||
##
|
||||
## Run via `tooling/godot-parse-sweep` (never directly — this half cannot fail).
|
||||
## Run via `reach godot parse-sweep` (never directly — this half cannot fail).
|
||||
|
||||
## Vendor and generated trees are not ours to verify. `addons/` is gdUnit4,
|
||||
## whose own test corpus deliberately contains broken fixtures — sweeping it
|
||||
|
||||
@@ -13,7 +13,7 @@ tags: [migration, tooling, pql, decisions, ticketing]
|
||||
> the phases-as-hierarchy refactor + Phase-6 SQLite retirement in PR #155. It is kept
|
||||
> as a historical record of *why and how* the planning system moved off the binary
|
||||
> SQLite DB onto pql. **Current usage** is in `CLAUDE.md`, `.claude/rules/ticket-cli.md`,
|
||||
> and `governance/README.md`; the one-shot transforms live in `tooling/pql-migrate/`.
|
||||
> and `governance/README.md`; the one-shot transforms live in `tooling/archive/pql-migrate/`.
|
||||
> Open feedback for the pql team is in the root `pql-requirements.md`.
|
||||
|
||||
# pql Migration Plan — decisions + ticketing
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
-- Authored via tooling pipeline; wiki pages are generated views.
|
||||
-- Shipped with the game — read-only from the server's perspective.
|
||||
--
|
||||
-- Rebuild: tooling/process-wiki-system-changes --rebuild-db
|
||||
-- Rebuild: make regen-db (the wiki->DB rebuild script is archived, T-1290)
|
||||
|
||||
PRAGMA journal_mode=WAL;
|
||||
PRAGMA foreign_keys=ON;
|
||||
@@ -41,7 +41,7 @@ CREATE TABLE IF NOT EXISTS star_systems (
|
||||
-- Short narrative hook extracted from the wiki's gttr.md file — the
|
||||
-- first characterisation paragraph ("where the rules live", "forty
|
||||
-- years old and still in the draft", etc). Populated by
|
||||
-- tooling/db/populate_gttr_hook.py from wiki/star-systems/<slug>/gttr.md.
|
||||
-- `reach wiki gttr-hook` from wiki/star-systems/<slug>/gttr.md.
|
||||
-- Used as compact cultural context in the Gemma 2 naming pipeline
|
||||
-- (#833) so per-system feel is grounded in the canonical identity
|
||||
-- rather than generic corridor labels.
|
||||
|
||||
Binary file not shown.
+4
-4
@@ -5,8 +5,8 @@ before anything moves. This is the artefact the per-domain port tickets are
|
||||
written from — without it their boundaries would be guesses, renegotiated
|
||||
halfway through the move.
|
||||
|
||||
**Status:** map only. Nothing here has moved yet except `check client-version`,
|
||||
which shipped in T-1262 as the reference implementation.
|
||||
**Status:** the map was written before anything moved; the ✅ rows below record
|
||||
which domains have since been ported, and by which ticket.
|
||||
|
||||
## What the survey changed
|
||||
|
||||
@@ -90,11 +90,11 @@ invented.
|
||||
| ~~`starmap`~~ | **folded into `atlas map`** — the top rung of the same ladder | — |
|
||||
| ~~`planet`~~ | **folded into `atlas planet`** — the third rung of the same ladder | — |
|
||||
| `ledger` | the economics pipeline, named for the UI component that will aggregate it | ✅ ported (T-1289). `economy-db/` → `domains/ledger/` (`economy_import/` kept by name; the entrypoint became `service.py`), `schema_version.py` with it. `reach ledger import`; `make regen-db` survives as a one-line delegate, `make economy-db` retired. `generated_brands.toml` byte-identical across the move |
|
||||
| `wiki` | wiki sync and content maintenance | `wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, `db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, `fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, `process-wiki-system-changes` |
|
||||
| `wiki` | wiki sync and content maintenance | ✅ ported (T-1290): `reach wiki stats`, `reach wiki gttr-hook` (both output-identical to the originals). `wiki_sync.py` moved whole, but its renderer and importer are NOT verbs — re-rendering deletes ~10,700 lines of committed pages (T-1292). The seven one-shots (`assign-astro-ids`, `migrate-s-to-gj`, `patch-core-sector`, `fill-missing-globes`, `generate-stubs`/`find-stubs`, `backfill_cultural_corridor`) and the destructive `process-wiki-system-changes` went to `archive/wiki-bootstrap/` |
|
||||
| `assets` | connectors to the tower-of-joy generators | `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, `db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py` |
|
||||
| `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** |
|
||||
| `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` |
|
||||
| `godot` | Godot parse and cold-start checks | `godot-parse-sweep`, `godot-cold-parse` |
|
||||
| `godot` | Godot parse and cold-start checks | ✅ ported (T-1283); the two bash originals were left beside the port and retired in T-1290 |
|
||||
| `generate` | content generators not owned elsewhere | ✅ ported (T-1286). `generate-brands` + `generate-corporations` collapsed into `core.process.cargo_binary` — they were the same 24 lines of bash a third time |
|
||||
| `dev` | developer environment and workflow | ✅ ported (T-1286). The three environment scripts split decision from performing — `godot_plan`/`worktree_plan` are pure and pinned by `test_environment.py` |
|
||||
| `pr` | the PR/review loop | ✅ ported (T-1286). `watchlist-diff` now reads the watched set from `generator_sources.py` instead of restating it |
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# tooling/archive — provenance, not tooling
|
||||
|
||||
Scripts kept because they record HOW something in the repo came to be, not
|
||||
because anything should run them again. None is a `reach` verb, none is
|
||||
importable (no `__init__.py`, excluded from package discovery and from the
|
||||
D-263 conformance scope), and several would do damage if run today. Deleting
|
||||
them would lose the provenance; keeping them live would imply they still work
|
||||
(tooling/DOMAINS.md, "pql-migrate/ is provenance, not tooling").
|
||||
|
||||
| path | what it did | why it is here |
|
||||
|---|---|---|
|
||||
| `pql-migrate/` | the one-shot planning-store migration into pql | completed migration (T-1271 ruling) |
|
||||
| `wiki-bootstrap/assign-astro-ids.py` + `gj-catalog-real.json` | gave the 300 `S-NNN` systems real Gliese-Jahreiss ids | done; `S-` ids no longer exist |
|
||||
| `wiki-bootstrap/migrate-s-to-gj.py` | rewrote every `S-NNN` reference to its GJ id | done |
|
||||
| `wiki-bootstrap/patch-core-sector.py` | added 17 hand-balanced core-sector gate edges | done; hardcodes a `planning/` path that no longer exists |
|
||||
| `wiki-bootstrap/generate-stubs.py`, `find-stubs` | created and then tracked the Phase-1 system stubs | done; `find-stubs --all-sectors 0 99` finds 0 stubs |
|
||||
| `wiki-bootstrap/fill-missing-globes.py` | copied donor `globe.png` files into bodies that lacked one | self-described one-shot |
|
||||
| `wiki-bootstrap/backfill_cultural_corridor.py` | copied `geographic_sector` into `cultural_corridor` | a raw systems.db patch script, the kind `.claude/rules/asset-pipeline.md` forbids; absent from the D-262 flow |
|
||||
| `wiki-bootstrap/process-wiki-system-changes` | wiki → systems.db → star-map.json → re-rendered system pages | **destructive today** — its last step is `wiki_sync.generate_wiki()`, which no longer reproduces the committed pages (T-1290 finding, see the wiki domain's router) |
|
||||
|
||||
Archived in T-1250 / T-1290 (2026-09-23).
|
||||
Executable → Regular
@@ -0,0 +1,11 @@
|
||||
"""`wiki` — the wiki seed's tooling (D-263).
|
||||
|
||||
Mirrors an implant app: the wiki/GTTR popups the player reads. Before touching
|
||||
anything here, read `Skill(wiki)` — two ownership models run in opposite
|
||||
directions through this directory, and D-262's flow diagram
|
||||
(docs/diagrams/data-flow/wiki-generator-flow.d2) is the canonical map.
|
||||
|
||||
Formerly tooling/db/wiki_sync.py and tooling/db/populate_gttr_hook.py
|
||||
(T-1290). The one-shot bootstrap scripts that sat beside them went to
|
||||
tooling/archive/wiki-bootstrap/.
|
||||
"""
|
||||
@@ -1,6 +1,4 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Extract a short narrative hook from every wiki/star-systems/<slug>/gttr.md
|
||||
"""Extract a short narrative hook from every wiki/star-systems/<slug>/gttr.md
|
||||
and store it on `star_systems.gttr_hook`.
|
||||
|
||||
The GTTR ("Drifter's Guide to the Reach") files follow a consistent
|
||||
@@ -19,35 +17,32 @@ cultural signal available for each system, capturing things like:
|
||||
- Posto Avançado: "forward post — the place beyond the established
|
||||
line"
|
||||
|
||||
This script parses each gttr.md, regex-extracts the first `**NAME**`
|
||||
paragraph, normalises whitespace, and truncates at a soft word cap so
|
||||
the hook stays cheap to inject into prompts. Empty or unmatched files
|
||||
leave the column NULL.
|
||||
This parses each gttr.md, regex-extracts the first `**NAME**` paragraph,
|
||||
normalises whitespace, and truncates at a soft word cap so the hook stays
|
||||
cheap to inject into prompts. Empty or unmatched files leave the column NULL.
|
||||
|
||||
Idempotent, safe to re-run after any wiki update. Explicit transaction
|
||||
wrapper with rollback on exception.
|
||||
wrapper with rollback on exception. It writes systems.db directly, which the
|
||||
asset-pipeline rule otherwise forbids; this edge is sanctioned by the D-262
|
||||
flow diagram (`populate_gttr_hook -> systems.db`), and `gttr_hook` sits on
|
||||
`star_systems`, which `reach ledger import` does not clear.
|
||||
|
||||
Usage:
|
||||
tooling/db/populate_gttr_hook.py
|
||||
tooling/db/populate_gttr_hook.py --max-words 45
|
||||
tooling/db/populate_gttr_hook.py --dry-run
|
||||
tooling/db/populate_gttr_hook.py --system "GJ 71"
|
||||
Formerly tooling/db/populate_gttr_hook.py (T-1290).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
REPO_ROOT = (SCRIPT_DIR / ".." / "..").resolve()
|
||||
from tooling.core import config, console
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
REPO_ROOT = config.repo_root()
|
||||
DB_PATH = REPO_ROOT / "server" / "data" / "systems.db"
|
||||
WIKI_SYSTEMS = REPO_ROOT / "wiki" / "star-systems"
|
||||
|
||||
# Matches the first paragraph that opens with `**NAME**` at the start
|
||||
# of a line. Captures everything up to a blank line or the next H1/H2.
|
||||
# Non-greedy on the content.
|
||||
# Match a line that opens with `**NAME**` (any non-asterisk chars, since
|
||||
# names contain accents from many alphabets — Á, Ž, Ç, Ñ, etc) followed
|
||||
# by the paragraph body up to a blank line or the next markdown heading.
|
||||
@@ -86,10 +81,7 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None:
|
||||
# Collapse whitespace so multi-line paragraphs become one clean line.
|
||||
hook = re.sub(r"\s+", " ", hook).strip()
|
||||
|
||||
# Strip a leading orphan "is" that comes from `**NAME**` + " is …":
|
||||
# the regex captures the word "is" on its own because the opener is
|
||||
# typically `**GATEWAY** (known as Tau Ceti...) is the most connected…`.
|
||||
# Normal reading already works — this is just hygiene.
|
||||
# Drop an empty parenthetical left behind by the collapse — hygiene only.
|
||||
hook = re.sub(r"\s*\(\s*\)\s*", " ", hook)
|
||||
|
||||
# Hard cap at max_words. Truncate at the last word boundary before
|
||||
@@ -101,37 +93,20 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None:
|
||||
return hook
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Populate star_systems.gttr_hook from wiki gttr.md files"
|
||||
)
|
||||
parser.add_argument("--db", default=str(DB_PATH), help="Path to systems.db")
|
||||
parser.add_argument(
|
||||
"--system",
|
||||
help="Process only this system_id (e.g. 'GJ 71')",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-words",
|
||||
type=int,
|
||||
default=45,
|
||||
help="Soft cap on hook length (default: 45 words)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Extract and print but do not write to the DB",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
help="Print every extracted hook",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
db_path = Path(args.db)
|
||||
def run(
|
||||
db_path: Path = DB_PATH,
|
||||
*,
|
||||
system: str | None = None,
|
||||
max_words: int = 45,
|
||||
dry_run: bool = False,
|
||||
verbose: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Populate star_systems.gttr_hook. Returns updated/missing/unmatched counts."""
|
||||
if not db_path.exists():
|
||||
print(f"error: {db_path} not found", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
raise ReachError(
|
||||
f"{db_path} not found",
|
||||
fix="pass --db with an existing systems.db, or restore it: git restore server/data/systems.db",
|
||||
)
|
||||
|
||||
conn = sqlite3.connect(str(db_path), timeout=30.0)
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
@@ -144,22 +119,21 @@ def main():
|
||||
except sqlite3.OperationalError:
|
||||
pass # column already exists
|
||||
|
||||
print("\n populate_gttr_hook.py")
|
||||
print(f" DB: {db_path}")
|
||||
print(f" max words: {args.max_words}")
|
||||
if args.dry_run:
|
||||
print(" Mode: DRY RUN")
|
||||
print()
|
||||
console.event(
|
||||
f"gttr-hook into {db_path} (max {max_words} words)" + (" — DRY RUN" if dry_run else "")
|
||||
)
|
||||
|
||||
rows = conn.execute(
|
||||
"SELECT system_id, proper_name FROM star_systems ORDER BY system_id"
|
||||
).fetchall()
|
||||
if args.system:
|
||||
rows = [r for r in rows if r[0] == args.system]
|
||||
rows = conn.execute("SELECT system_id, proper_name FROM star_systems ORDER BY system_id").fetchall()
|
||||
if system:
|
||||
rows = [r for r in rows if r[0] == system]
|
||||
if not rows:
|
||||
conn.close()
|
||||
raise ReachError(
|
||||
f"no system {system!r} in star_systems",
|
||||
fix="pass a system_id with its space, e.g. --system 'GJ 71'",
|
||||
)
|
||||
|
||||
updated = 0
|
||||
missing = 0
|
||||
unmatched = 0
|
||||
counts = {"updated": 0, "missing": 0, "unmatched": 0}
|
||||
|
||||
conn.execute("BEGIN")
|
||||
try:
|
||||
@@ -167,31 +141,26 @@ def main():
|
||||
slug = system_id.replace(" ", "-", 1)
|
||||
gttr_path = WIKI_SYSTEMS / slug / "gttr.md"
|
||||
|
||||
hook = extract_hook(gttr_path, args.max_words)
|
||||
hook = extract_hook(gttr_path, max_words)
|
||||
if hook is None:
|
||||
if not gttr_path.exists():
|
||||
missing += 1
|
||||
if args.verbose:
|
||||
print(f" MISSING {system_id:10s} {gttr_path}")
|
||||
else:
|
||||
unmatched += 1
|
||||
if args.verbose:
|
||||
print(f" UNMATCHED {system_id:10s} {gttr_path}")
|
||||
kind = "missing" if not gttr_path.exists() else "unmatched"
|
||||
counts[kind] += 1
|
||||
if verbose:
|
||||
console.event(f"{kind.upper()} {system_id} {gttr_path}", level="warn")
|
||||
continue
|
||||
|
||||
if args.verbose:
|
||||
if verbose:
|
||||
display = f"{system_id} ({proper_name})" if proper_name else system_id
|
||||
print(f" {display}")
|
||||
print(f" → {hook}")
|
||||
console.out(f"{display}\n → {hook}")
|
||||
|
||||
if not args.dry_run:
|
||||
if not dry_run:
|
||||
conn.execute(
|
||||
"UPDATE star_systems SET gttr_hook = ? WHERE system_id = ?",
|
||||
(hook, system_id),
|
||||
)
|
||||
updated += 1
|
||||
counts["updated"] += 1
|
||||
|
||||
if args.dry_run:
|
||||
if dry_run:
|
||||
conn.rollback()
|
||||
else:
|
||||
conn.commit()
|
||||
@@ -201,15 +170,4 @@ def main():
|
||||
raise
|
||||
|
||||
conn.close()
|
||||
|
||||
print("\n Done:")
|
||||
print(f" updated: {updated}")
|
||||
print(f" missing: {missing}")
|
||||
print(f" unmatched: {unmatched}")
|
||||
if args.dry_run:
|
||||
print("\n Dry run — no DB writes.")
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
return counts
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Transport for the `wiki` domain — args in, delegate, format out.
|
||||
|
||||
Two verbs, deliberately not four. `wiki_sync` also holds `generate_wiki()`
|
||||
(systems.db → system pages) and `import_from_wiki()` (pages → systems.db), and
|
||||
neither is exposed: re-rendering today would delete ~10,700 lines of the
|
||||
committed pages (see wiki_sync's docstring, T-1290). A verb is a promise that
|
||||
running it is safe, and that one would not be. They come back when the
|
||||
renderer reproduces the committed pages byte-for-byte.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from tooling.core import cli, console
|
||||
from tooling.core.command import command
|
||||
|
||||
app = cli.domain("wiki", "The wiki seed — fill rates and the GTTR hook.")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _domain() -> None:
|
||||
"""Keeps `wiki` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@app.command("stats")
|
||||
@command
|
||||
def stats() -> None:
|
||||
"""Fill rates of the structured star-system columns in systems.db."""
|
||||
from tooling.domains.wiki import wiki_sync
|
||||
|
||||
wiki_sync.stats()
|
||||
|
||||
|
||||
@app.command("gttr-hook")
|
||||
@command
|
||||
def gttr_hook(
|
||||
db: Path = typer.Option(None, "--db", help="systems.db to write (default: server/data/systems.db)."),
|
||||
system: str = typer.Option(None, "--system", help="Only this system_id, e.g. 'GJ 71'."),
|
||||
max_words: int = typer.Option(45, "--max-words", help="Soft cap on hook length."),
|
||||
dry_run: bool = typer.Option(False, "--dry-run", help="Extract, write nothing."),
|
||||
verbose: bool = typer.Option(False, "--verbose", help="Print every extracted hook."),
|
||||
) -> None:
|
||||
"""Extract each system's GTTR opening line into star_systems.gttr_hook.
|
||||
|
||||
Reads wiki/star-systems/<GJ-id>/gttr.md, takes the first **NAME**
|
||||
paragraph, collapses and caps it. Re-run after editing any gttr.md.
|
||||
"""
|
||||
from tooling.domains.wiki import gttr_hook as service
|
||||
|
||||
counts = service.run(
|
||||
db or service.DB_PATH,
|
||||
system=system,
|
||||
max_words=max_words,
|
||||
dry_run=dry_run,
|
||||
verbose=verbose,
|
||||
)
|
||||
written = "extracted, nothing written (dry run)" if dry_run else "written"
|
||||
console.verdict(
|
||||
f"gttr-hook: {counts['updated']} {written}, "
|
||||
f"{counts['missing']} missing gttr.md, {counts['unmatched']} unmatched"
|
||||
)
|
||||
@@ -1,4 +1,3 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Settled Reach Wiki Sync — wiki frontmatter ↔ systems.db
|
||||
|
||||
@@ -13,6 +12,18 @@ This module provides:
|
||||
- generate_wiki(): render wiki pages from DB + existing prose
|
||||
- stats(): show completion stats
|
||||
|
||||
**generate_wiki() is NOT safe to run against the committed wiki** (T-1290).
|
||||
Re-rendering today deletes ~10,700 lines across all 301 system pages: the
|
||||
Celestial Bodies / Stations blocks (written by the Rust atlas CLI's
|
||||
sync_wiki.rs, which this renderer knows nothing about), the Industries /
|
||||
Exports / Imports rows (which nothing in the repo writes any more), and it
|
||||
rewrites star types where systems.db and the pages disagree. That is why
|
||||
`reach wiki` exposes only `stats` and `gttr-hook`. The functions stay, because
|
||||
fixing the renderer means fixing them, not rewriting them from nothing.
|
||||
|
||||
Formerly tooling/db/wiki_sync.py, run by hand or via the now-archived
|
||||
process-wiki-system-changes pipeline.
|
||||
|
||||
Database: server/data/systems.db
|
||||
Schema: server/data/systems-schema.sql
|
||||
"""
|
||||
@@ -20,15 +31,15 @@ Schema: server/data/systems-schema.sql
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from tooling.core import config, console
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
WORKTREE_ROOT = (SCRIPT_DIR / ".." / "..").resolve()
|
||||
WORKTREE_ROOT = config.repo_root()
|
||||
SCHEMA_PATH = WORKTREE_ROOT / "server" / "data" / "systems-schema.sql"
|
||||
WIKI_DIR = WORKTREE_ROOT / "wiki" / "star-systems"
|
||||
DB_PATH = WORKTREE_ROOT / "server" / "data" / "systems.db"
|
||||
@@ -613,14 +624,16 @@ def generate_wiki(prose_only_update=False):
|
||||
# Topology (always last)
|
||||
page_lines.append(topology)
|
||||
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
outfile.write_text("\n".join(page_lines), encoding="utf-8")
|
||||
|
||||
# Counted BEFORE the write. Checking exists() after writing (as this
|
||||
# did until T-1290) made every page an update and `created` always 0.
|
||||
if outfile.exists():
|
||||
updated += 1
|
||||
else:
|
||||
created += 1
|
||||
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
outfile.write_text("\n".join(page_lines), encoding="utf-8")
|
||||
|
||||
conn.close()
|
||||
return {
|
||||
"ok": True,
|
||||
@@ -635,19 +648,26 @@ def generate_wiki(prose_only_update=False):
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def stats():
|
||||
"""Show per-table fill rates."""
|
||||
def stats() -> dict:
|
||||
"""Per-table fill rates for the structured system columns.
|
||||
|
||||
The table is the command's output, so it goes to stdout (console.out).
|
||||
A missing table used to print a hint and exit 0 — a failure reading as
|
||||
success — and now raises with the remedy (T-1290).
|
||||
"""
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
conn = get_connection()
|
||||
try:
|
||||
row = conn.execute("SELECT COUNT(*) as cnt FROM star_systems").fetchone()
|
||||
total = row["cnt"]
|
||||
if total == 0:
|
||||
print("No systems in database. Run import first.")
|
||||
console.out("No systems in database.")
|
||||
return {"ok": True, "total": 0}
|
||||
|
||||
print(f"Star systems: {total}")
|
||||
print(f"Database: {DB_PATH}")
|
||||
print()
|
||||
console.out(f"Star systems: {total}")
|
||||
console.out(f"Database: {DB_PATH}")
|
||||
console.out()
|
||||
|
||||
total_cells = 0
|
||||
filled_cells = 0
|
||||
@@ -665,64 +685,24 @@ def stats():
|
||||
filled_cells += table_filled
|
||||
pct = table_filled / table_total * 100 if table_total > 0 else 0
|
||||
bar = "#" * int(pct / 5) + "." * (20 - int(pct / 5))
|
||||
print(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%")
|
||||
console.out(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%")
|
||||
|
||||
# Historical events
|
||||
ev_count = conn.execute("SELECT COUNT(*) as cnt FROM historical_events").fetchone()["cnt"]
|
||||
ev_systems = conn.execute(
|
||||
"SELECT COUNT(DISTINCT system_id) as cnt FROM historical_events"
|
||||
).fetchone()["cnt"]
|
||||
print(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems")
|
||||
console.out(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems")
|
||||
|
||||
overall = filled_cells / total_cells * 100 if total_cells > 0 else 0
|
||||
print(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)")
|
||||
console.out(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)")
|
||||
|
||||
return {"ok": True, "total": total, "filled": filled_cells, "pct": round(overall, 1)}
|
||||
|
||||
except sqlite3.OperationalError as exc:
|
||||
print(f"Table not found — run import first: {exc}")
|
||||
return {"ok": False, "error": str(exc)}
|
||||
raise ReachError(
|
||||
f"systems.db is missing a system table: {exc}",
|
||||
fix="make regen-db, or restore it: git restore server/data/systems.db",
|
||||
) from exc
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def main():
|
||||
help_text = f"""\
|
||||
Wiki Sync — systems.db ↔ wiki pages
|
||||
|
||||
Usage:
|
||||
wiki_sync.py import Parse wiki frontmatter into systems.db (bootstrap)
|
||||
wiki_sync.py generate Generate wiki pages from systems.db
|
||||
wiki_sync.py stats Show completion stats
|
||||
wiki_sync.py --help Show this help
|
||||
|
||||
Database: {DB_PATH}
|
||||
Schema: {SCHEMA_PATH}
|
||||
Wiki: {WIKI_DIR}
|
||||
"""
|
||||
if len(sys.argv) < 2 or sys.argv[1] in ("--help", "-h", "help"):
|
||||
print(help_text)
|
||||
sys.exit(0)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "import":
|
||||
result = import_from_wiki()
|
||||
print(json.dumps(result, indent=2))
|
||||
elif cmd == "generate":
|
||||
result = generate_wiki()
|
||||
print(json.dumps(result, indent=2))
|
||||
elif cmd == "stats":
|
||||
stats()
|
||||
else:
|
||||
print(json.dumps({"ok": False, "error": f"Unknown: {cmd}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,96 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/godot-cold-parse [--run-menu] — cold-cache headless parse check.
|
||||
#
|
||||
# Used by /pr-process step 1c before push. Deletes the cached script-class
|
||||
# registry so the parse simulates the cold-start ordering CI / fresh clones
|
||||
# see: Sprint 36 close caught a new `class_name MetaScreen` base class and
|
||||
# six extending scripts that parsed fine on warm developer caches but hit
|
||||
# `Could not find base class "MetaScreen"` post-merge, because the
|
||||
# autoload-vs-class_name registration order only resolves correctly once the
|
||||
# class cache is seeded (see CLAUDE.md -> GDScript conventions -> Autoload
|
||||
# parse-order rule).
|
||||
#
|
||||
# Godot's resource scanner emits category errors (e.g. "Export type can only
|
||||
# be built-in, a resource, a node, or an enum") that do NOT always prefix
|
||||
# with SCRIPT ERROR — they appear as plain ERROR lines. The filter below
|
||||
# catches both, then drops known pre-existing noise from the autoload
|
||||
# class_name parse-order trap. Sprint 36 shipped a scanner error the old
|
||||
# narrower grep missed; this is why the filter stays wide.
|
||||
#
|
||||
# --run-menu: also launch main_menu.tscn briefly (for branches with UI changes).
|
||||
#
|
||||
# Exit 0 + "clean" if no matches. Exit 1 + the matched lines if any are found.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
RUN_MENU=false
|
||||
[ "${1:-}" = "--run-menu" ] && RUN_MENU=true
|
||||
|
||||
rm -f "$REPO_ROOT/client/.godot/global_script_class_cache.cfg"
|
||||
|
||||
# A truly cold checkout (fresh clone or worktree — .godot/ is gitignored) has
|
||||
# no resource-import cache, and every imported asset (fonts, ogg) then "fails
|
||||
# loading" during the parse run: a wall of false positives. Seed the cache
|
||||
# with an import pass first; source assets are tracked, so this is always
|
||||
# reconstructible. (Found live: first run in a fresh worktree, 2026-07-13.)
|
||||
if [ ! -d "$REPO_ROOT/client/.godot/imported" ] || [ -z "$(ls -A "$REPO_ROOT/client/.godot/imported" 2>/dev/null)" ]; then
|
||||
echo "godot-cold-parse: no import cache — running one-time import pass..." >&2
|
||||
set +e
|
||||
IMPORT_OUT=$(godot --headless --path "$REPO_ROOT/client" --import 2>&1)
|
||||
IMPORT_EXIT=$?
|
||||
set -e
|
||||
if [ "$IMPORT_EXIT" -ne 0 ]; then
|
||||
echo "godot-cold-parse: import pass exited $IMPORT_EXIT" >&2
|
||||
printf '%s\n' "$IMPORT_OUT" | tail -20 >&2
|
||||
exit "$IMPORT_EXIT"
|
||||
fi
|
||||
fi
|
||||
|
||||
FILTER='^(SCRIPT )?ERROR|Parse Error|Export type'
|
||||
|
||||
# Capture the godot run separately from the filter pipeline: with the
|
||||
# trailing `|| true` on the greps, a nonzero exit from godot itself (crash,
|
||||
# missing binary, corrupted install) would otherwise report "clean". Nothing
|
||||
# downstream re-reads the raw output now that this is scripted, so fail loud.
|
||||
set +e
|
||||
RAW=$(godot --headless --path "$REPO_ROOT/client" --quit 2>&1)
|
||||
GODOT_EXIT=$?
|
||||
set -e
|
||||
if [ "$GODOT_EXIT" -ne 0 ]; then
|
||||
echo "godot-cold-parse: godot itself exited $GODOT_EXIT — not a parse verdict" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit "$GODOT_EXIT"
|
||||
fi
|
||||
|
||||
MATCHES=$(printf '%s\n' "$RAW" \
|
||||
| grep -iE "$FILTER" \
|
||||
| grep -v "Failed loading resource: res://assets" \
|
||||
| grep -v "Cannot infer the type" \
|
||||
| grep -vE '(Messagepack|LocalBridge|ServerProcess|Constants)" not declared' || true)
|
||||
|
||||
if [ "$RUN_MENU" = true ]; then
|
||||
# Deliberately no exit-code check here: `timeout` kills the menu after
|
||||
# 10s by design (exit 124 is the expected shutdown path); only the
|
||||
# scraped error lines carry signal for this bounded run.
|
||||
MENU_MATCHES=$(timeout 10 godot --path "$REPO_ROOT/client" res://scenes/main_menu.tscn 2>&1 \
|
||||
| grep -iE "$FILTER" || true)
|
||||
if [ -n "$MENU_MATCHES" ]; then
|
||||
MATCHES="$MATCHES
|
||||
$MENU_MATCHES"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -n "$MATCHES" ]; then
|
||||
echo "$MATCHES"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Restore a FULL class cache before exiting: the cold parse run re-seeds
|
||||
# global_script_class_cache.cfg only partially — addon classes (e.g. gdUnit4's
|
||||
# GdUnitTestCIRunner) are missing, which leaves tests/run-godot unable to even
|
||||
# start (0 tests in ~350ms; found live when the pre-push gate ran the suite
|
||||
# right after this script, 2026-07-14). The cold verdict above is already
|
||||
# decided; this restore just returns the tree to a runnable state.
|
||||
godot --headless --path "$REPO_ROOT/client" --import > /dev/null 2>&1 || true
|
||||
|
||||
echo "godot-cold-parse: clean"
|
||||
@@ -1,64 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/godot-parse-sweep — open every project .gd and fail on any that won't parse.
|
||||
#
|
||||
# Complements `godot-cold-parse`, which does NOT cover this. That script only
|
||||
# ever sees scripts on the STARTUP path (autoloads + the main scene chain) —
|
||||
# correct for the registration-ORDER bug it was built for, far narrower than
|
||||
# its name implies. Verified 2026-07-27 by breaking a non-startup UI script
|
||||
# and a test file in turn: cold-parse reported "clean", exit 0, for both.
|
||||
#
|
||||
# Division of labour, deliberate:
|
||||
# godot-cold-parse — cold-cache STARTUP ordering (class_name/autoload race)
|
||||
# godot-parse-sweep — does every file in the project parse at all
|
||||
#
|
||||
# The GDScript half (client/tools/parse_sweep.gd) only opens files; it makes no
|
||||
# verdict, because no Godot API reports GDScript parse failure reliably (see
|
||||
# that file's header — one segfaults, one false-positives 150/226, and plain
|
||||
# load() returns non-null for a broken script). The engine's own stderr is the
|
||||
# only honest signal, so the verdict is made here by scraping it.
|
||||
#
|
||||
# Exit 0 + "clean" if every script parsed. Exit 1 + the offending lines if not.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
|
||||
set +e
|
||||
RAW=$(godot --headless --path "$REPO_ROOT/client" -s res://tools/parse_sweep.gd 2>&1)
|
||||
GODOT_EXIT=$?
|
||||
set -e
|
||||
|
||||
if [ "$GODOT_EXIT" -ne 0 ]; then
|
||||
echo "godot-parse-sweep: godot itself exited $GODOT_EXIT — not a parse verdict" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit "$GODOT_EXIT"
|
||||
fi
|
||||
|
||||
# The sweep must actually have run. Without this, a future change that breaks
|
||||
# the walk (or renames the script) would produce zero error lines and read as
|
||||
# a clean sweep — the same false-green shape this tool exists to close.
|
||||
if ! printf '%s\n' "$RAW" | grep -q '^parse-sweep: opened'; then
|
||||
echo "godot-parse-sweep: the sweep did not report completion — no verdict possible" >&2
|
||||
printf '%s\n' "$RAW" | tail -20 >&2
|
||||
exit 1
|
||||
fi
|
||||
SUMMARY=$(printf '%s\n' "$RAW" | grep '^parse-sweep: opened')
|
||||
|
||||
# NOTE: unlike godot-cold-parse, "Cannot infer the type" is NOT filtered here.
|
||||
# That filter is why cold-parse stayed silent about tests/util/scene_helper.gd,
|
||||
# which genuinely does not parse — the suppressed class was hiding a real
|
||||
# failure, not noise.
|
||||
MATCHES=$(printf '%s\n' "$RAW" \
|
||||
| grep -E 'Parse Error|Failed to load script' \
|
||||
| grep -v "Failed loading resource: res://assets" || true)
|
||||
|
||||
if [ -n "$MATCHES" ]; then
|
||||
echo "PARSE SWEEP FAILED — at least one script does not parse." >&2
|
||||
printf '%s\n' "$MATCHES" | head -40 >&2
|
||||
echo "" >&2
|
||||
echo " ${SUMMARY}" >&2
|
||||
echo " An unparseable file cannot run. If it is a test suite, it did not" >&2
|
||||
echo " execute, and any pass count reported elsewhere excludes it." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "godot-parse-sweep: clean — ${SUMMARY#parse-sweep: }"
|
||||
@@ -63,6 +63,10 @@ DOMAINS: dict[str, tuple[str, str]] = {
|
||||
"tooling.domains.ledger.router:app",
|
||||
"The economics pipeline — the import that builds systems.db",
|
||||
),
|
||||
"wiki": (
|
||||
"tooling.domains.wiki.router:app",
|
||||
"The wiki seed — fill rates and the GTTR hook",
|
||||
),
|
||||
"godot": (
|
||||
"tooling.domains.godot.router:app",
|
||||
"Does the client parse, and does it parse cold",
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@
|
||||
> input. Where this text and the diagram disagree, the diagram is right.
|
||||
|
||||
The wiki has two kinds of content with different sources of truth (see
|
||||
`tooling/db/wiki_sync.py`):
|
||||
`tooling/domains/wiki/wiki_sync.py`):
|
||||
|
||||
- **DB owns structured fields, wiki owns prose** (`star-systems/`): identity,
|
||||
gates, history, economy, factions, and culture live in `server/data/systems.db`
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
System pages carry NO YAML frontmatter. Structured fields live in
|
||||
server/data/systems.db (DB owns structured, wiki owns prose — see
|
||||
tooling/db/wiki_sync.py and wiki/GOVERNANCE.md). Pages are produced by the
|
||||
tooling/domains/wiki/wiki_sync.py and wiki/GOVERNANCE.md). Pages are produced by the
|
||||
generator pipeline; the authoring surface is the five prose sections below.
|
||||
Sections marked READ-ONLY are regenerated from their source — never edit them
|
||||
by hand. -->
|
||||
|
||||
Reference in New Issue
Block a user