diff --git a/.claude/rules/project-structure.md b/.claude/rules/project-structure.md index 3b0aca491..aae4daf4e 100644 --- a/.claude/rules/project-structure.md +++ b/.claude/rules/project-structure.md @@ -17,7 +17,7 @@ wiki/ # THE SEED (11,864 files, the largest tree here). Authored economics/ # INPUT. TOML vocabularies + schema.md. Seeds systems.db # AND, from Phase 4 on, world content generation star-systems/ # OUTPUT, 93% of the wiki. Generated by - # tooling/db/wiki_sync.py FROM systems.db + star-map.json + # tooling/domains/wiki/wiki_sync.py FROM systems.db + star-map.json GJ-{id}/ # system dir (hyphenated id) index.md # no frontmatter; blocks are # regenerated — authored prose lives in the named @@ -64,7 +64,7 @@ tooling/ config.json # Endpoint configuration common.py # Shared venv/config helpers audio_connector.py # Stable Audio Open connector - pql-migrate/ # One-shot scripts from the pql migration (provenance) + archive/ # Provenance only, never run: pql-migrate/, wiki-bootstrap/ .claude/ agents/ # Agent personality files skills/ # Skill definitions diff --git a/.claude/skills/pr-process/SKILL.md b/.claude/skills/pr-process/SKILL.md index 0ad8f1533..38b801c99 100644 --- a/.claude/skills/pr-process/SKILL.md +++ b/.claude/skills/pr-process/SKILL.md @@ -108,14 +108,14 @@ bugs (parse errors, depth sorting, scene tree failures). **For client/visual branches:** -`tooling/godot-cold-parse` wipes the cached script-class registry (matching +`reach godot cold-parse` wipes the cached script-class registry (matching the cold-start ordering CI / fresh clones see — Sprint 36 close caught a `class_name` base-class registration bug that warm caches masked) and runs a headless parse, filtering known pre-existing noise. Pass `--run-menu` if the branch has UI changes to also launch the main menu briefly: ```bash -tooling/godot-cold-parse --run-menu +reach godot cold-parse --run-menu ``` If it reports a "Could not find base class X" error, the fix is almost @@ -123,7 +123,7 @@ always an autoload-order issue (see `CLAUDE.md` → GDScript conventions → Autoload parse-order rule) — fix the ordering, don't just rebuild the cache to mask it locally (the same error resurfaces post-merge). -**Scope, so you don't over-trust it:** `godot-cold-parse` only ever sees +**Scope, so you don't over-trust it:** `reach godot cold-parse` only ever sees scripts on the **startup path** — autoloads and the main scene chain. That is exactly right for the registration-ORDER bug it exists to catch, and far narrower than the name suggests: it reports `clean` for a syntactically @@ -132,7 +132,7 @@ by breaking one of each). It also filters `"Cannot infer the type"`, which hid a genuinely unparseable file for five months. **"Does every script parse?" is a different question, and the pre-push hook -now answers it** — `tooling/godot-parse-sweep` opens all ~226 project scripts +now answers it** — `reach godot parse-sweep` opens all ~226 project scripts and blocks the push on any that don't, running *before* the test suite so a parse failure costs 4s instead of 135s. It is enforced by the gate, so do **not** run it here as well (`team-patterns.md`: don't duplicate the push diff --git a/.claude/skills/wiki/SKILL.md b/.claude/skills/wiki/SKILL.md index 6cf86cee2..7114abf65 100644 --- a/.claude/skills/wiki/SKILL.md +++ b/.claude/skills/wiki/SKILL.md @@ -32,8 +32,9 @@ the two disagree, the diagram is right. ### `star-systems/` — DB owns structure, wiki owns prose (OUTPUT) 3,262 pages, 93% of the wiki. Generated FROM `server/data/systems.db` and -`star-map.json` by `tooling/db/wiki_sync.py`, with `scaffold_bodies.py` and the -atlas sync alongside. +`star-map.json` by `tooling/domains/wiki/wiki_sync.py`, with `scaffold_bodies.py` +and the atlas sync (`server/src/bin/atlas/sync_wiki.rs`, which owns the +Celestial Bodies / Stations blocks) alongside. **Never hand-edit:** - anything inside a `` block (System Profile, Topology, @@ -42,9 +43,13 @@ atlas sync alongside. **System pages and BODY pages behave differently. Do not merge them in your head.** -*System page* (`GJ-{id}/index.md`) — `wiki_sync.py::generate_wiki()` re-renders it -from `systems.db` + `star-map.json`, preserving the authored prose sections. A -hand-edit inside a `` block IS reverted on the next sync. +*System page* (`GJ-{id}/index.md`) — `wiki_sync.py::generate_wiki()` was written +to re-render it from `systems.db` + `star-map.json`, preserving the authored +prose sections. **It is broken against today's pages — do not run it** +(measured 2026-09-23, T-1290): it deletes the Celestial Bodies / Stations blocks, +the Industries / Exports / Imports rows (which nothing writes any more), and +rewrites star types where the DB and the pages disagree — ~10,700 lines across +all 301 pages. `reach wiki` therefore exposes only `stats` and `gttr-hook`. (`wiki_sync.py` also has `import_from_wiki()`, a bootstrap/migration path running the other way — wiki frontmatter into the DB. Know it exists before assuming any single direction is "the" direction.) @@ -199,10 +204,10 @@ session once already. | tool | does | |---|---| -| `tooling/db/wiki_sync.py` | systems.db → star-system page sections | +| `tooling/domains/wiki/wiki_sync.py` | systems.db → star-system page sections (renderer broken — see above) | | `reach atlas planet scaffold` | creates body dirs/pages | | `tooling/domains/atlas/planet/body_definition_parser.py` | reads body frontmatter | -| `tooling/db/populate_gttr_hook.py` | GTTR prose → `gttr_hook` | +| `reach wiki gttr-hook` | GTTR prose → `gttr_hook` | | `reach atlas planet terrain-reference` | terrain asset paths | | `reach atlas planet audit` | audits atlas coherence | | `make regen-db` | economics TOML + corp frontmatter → systems.db | diff --git a/.pql/changelog/ticket_history/2026-09.sql b/.pql/changelog/ticket_history/2026-09.sql index c7e1edf96..0a33c3578 100644 --- a/.pql/changelog/ticket_history/2026-09.sql +++ b/.pql/changelog/ticket_history/2026-09.sql @@ -403,3 +403,70 @@ Standard port acceptance as on T-1281/T-1286. DONE 2026-09-23. tooling/economy-db/ + schema_version.py -> tooling/domains/ledger/ (economy_import/ kept by name, since server/src Rust comments cite economy_import/*.py; import_economics.py became service.py). reach ledger import [--db] [--dry-run] [--strict-specialization]. Evidence: generated_brands.toml sha256 identical before/after (e748531…); stamp gate reported STALE after the move and OK after regen; dry-run transcript carries every count and warning of the baseline; exit 2 (coverage gap, data committed) proven to relay through @command with a probe. Make: regen-db kept as a one-line delegate per D-263''s muscle-memory clause (~50 files name it, incl. generated TOML headers and gate remedies); economy-db retired (it ran generate brands, which the import already does first). Reconciled economy_import.errors (DOMAINS.md ask): ImportAborted stays internal rollback control flow, converted to ReachError at the service boundary. Also: regenerate_brands used to swallow cargo_binary''s ReachError into ImportAborted, dropping its remedy — now propagates (it runs before the transaction, nothing to roll back). Both sys.path bootstraps removed. Step labels were inconsistent ([1/10]..[10/13]..[17/19]); one 24-step counter now drives phase + progress. Stale doc pointers fixed while there: MIGRATION_SQL had lived in economy_import/migration.py since T-1067 but rules/DEVOPS still said import_economics.py; asset-pipeline rule and DEVOPS still named the check-systems-db script/target T-1281 retired.', NULL, '2026-09-23 14:18:49', '2026-09-23 14:18:49.681', '2026-09-23 14:18:49.681', NULL, '13300d7ea5bd79ae4857cd59399531a6', 2) ON CONFLICT(hash) DO NOTHING; INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65T2N1WAT491WY8SV736J3G', 'status', 'backlog', 'done', NULL, '2026-09-23 14:18:50', '2026-09-23 14:18:50.093', '2026-09-23 14:18:50.093', NULL, '63d24743b229b1544d62104c4e68ca66', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'description', 'The tail of the tree — three domains, ~26 files, batched because none of them +entangles with the spatial ladder or the systems.db stamp. + +`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, +`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, +`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, +`process-wiki-system-changes`. +READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite +directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is +OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are +INPUT. A verb that writes to the wrong side is silently reverted on the next sync. +Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) — +decide per file whether it is a live verb or provenance bound for `tooling/archive/`, +the same call T-1271 made for `pql-migrate/`. + +`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, +`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`. +These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by +IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json` +— that config must keep working or move deliberately, not incidentally. Network +failure is the normal case here, so `ReachError` messages need to distinguish +"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter. + +`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, +`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, +`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, +`setup_clothing_metadata.py`. +Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender +payloads that belong to the carve-out (T-1273) and MUST NOT import the package. +Do T-1273 first or the boundary will blur. + +Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots +before moving anything — five silent failures so far.', 'The tail of the tree — three domains, ~26 files, batched because none of them +entangles with the spatial ladder or the systems.db stamp. + +`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, +`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, +`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, +`process-wiki-system-changes`. +READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite +directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is +OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are +INPUT. A verb that writes to the wrong side is silently reverted on the next sync. +Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) — +decide per file whether it is a live verb or provenance bound for `tooling/archive/`, +the same call T-1271 made for `pql-migrate/`. + +`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, +`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`. +These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by +IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json` +— that config must keep working or move deliberately, not incidentally. Network +failure is the normal case here, so `ReachError` messages need to distinguish +"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter. + +`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, +`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, +`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, +`setup_clothing_metadata.py`. +Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender +payloads that belong to the carve-out (T-1273) and MUST NOT import the package. +Do T-1273 first or the boundary will blur. + +Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots +before moving anything — five silent failures so far. + +PROGRESS 2026-09-23 — wiki half done (commit to follow): reach wiki stats + gttr-hook, both output-identical to the originals (stats byte-identical; all 301 gttr hooks identical). wiki_sync.generate_wiki/import_from_wiki moved but NOT exposed: a pre-port run of the old --generate deleted ~10,700 lines of committed pages -> T-1292. Archive: tooling/archive/{pql-migrate,wiki-bootstrap}/ with a README (one-shots + the destructive process-wiki-system-changes + backfill_cultural_corridor, a raw DB patch script outside D-262). find-stubs archived too: it finds 0 stubs, Phase-1 job done. Side fix: tooling/godot-cold-parse + godot-parse-sweep were never retired after T-1283 and pr-process still told agents to run them — removed, skill repointed. Remaining: assets, character.', NULL, '2026-09-23 14:25:42', '2026-09-23 14:25:42.440', '2026-09-23 14:25:42.440', NULL, '50093acafd657096ec7f86acfa339f78', 2) ON CONFLICT(hash) DO NOTHING; diff --git a/.pql/changelog/ticket_idmap/2026-09.sql b/.pql/changelog/ticket_idmap/2026-09.sql index 5b1e49a24..a11d205b1 100644 --- a/.pql/changelog/ticket_idmap/2026-09.sql +++ b/.pql/changelog/ticket_idmap/2026-09.sql @@ -6,3 +6,4 @@ INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_ INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65T2N1WAT491WY8SV736J3G', 'T-1289', '2026-09-02 16:16:28.431', '2026-09-02 16:16:28.431', NULL, '976a8d383d0581caafca5a4bc2e0fb61', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'T-1290', '2026-09-02 16:17:45.770', '2026-09-02 16:17:45.770', NULL, '748d7ed02f4821a7ee5059a76d2fd3fe', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX60QRMD7KJ1TVWTR7FWK28', 'T-1291', '2026-09-23 14:07:49.190', '2026-09-23 14:07:49.190', NULL, '20abfe457c23ea4e00d3c8109f2e39f2', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; +INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX9TTBJW4NPECEN6JM0V4HM', 'T-1292', '2026-09-23 14:24:29.282', '2026-09-23 14:24:29.282', NULL, '45174fb519d2deed098a244229d543e1', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; diff --git a/.pql/changelog/tickets/2026-09.sql b/.pql/changelog/tickets/2026-09.sql index 66115b3b6..75648ac16 100644 --- a/.pql/changelog/tickets/2026-09.sql +++ b/.pql/changelog/tickets/2026-09.sql @@ -561,3 +561,39 @@ a bug, not a regen. Standard port acceptance as on T-1281/T-1286. DONE 2026-09-23. tooling/economy-db/ + schema_version.py -> tooling/domains/ledger/ (economy_import/ kept by name, since server/src Rust comments cite economy_import/*.py; import_economics.py became service.py). reach ledger import [--db] [--dry-run] [--strict-specialization]. Evidence: generated_brands.toml sha256 identical before/after (e748531…); stamp gate reported STALE after the move and OK after regen; dry-run transcript carries every count and warning of the baseline; exit 2 (coverage gap, data committed) proven to relay through @command with a probe. Make: regen-db kept as a one-line delegate per D-263''s muscle-memory clause (~50 files name it, incl. generated TOML headers and gate remedies); economy-db retired (it ran generate brands, which the import already does first). Reconciled economy_import.errors (DOMAINS.md ask): ImportAborted stays internal rollback control flow, converted to ReachError at the service boundary. Also: regenerate_brands used to swallow cargo_binary''s ReachError into ImportAborted, dropping its remedy — now propagates (it runs before the transaction, nothing to roll back). Both sys.path bootstraps removed. Step labels were inconsistent ([1/10]..[10/13]..[17/19]); one 24-step counter now drives phase + progress. Stale doc pointers fixed while there: MIGRATION_SQL had lived in economy_import/migration.py since T-1067 but rules/DEVOPS still said import_economics.py; asset-pipeline rule and DEVOPS still named the check-systems-db script/target T-1281 retired.', 'done', 'high', NULL, 'tooling', 'D-263', '2026-09-02 16:16:28.431', '2026-09-23 14:18:50.093', NULL, '579a9be46ac9f07e3b7a687fbe4772fe', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GCX9TTBJW4NPECEN6JM0V4HM', 'bug', '06FBPPMZNNEV052DBYYY3A897C', 'wiki_sync.generate_wiki() deletes ~10,700 lines of committed system pages — renderer out of step with the pages', 'Found in T-1290 (2026-09-23) by running the pre-port tool (process-wiki-system-changes --generate) against a clean tree and diffing: 301 pages changed, +940 / -10,761. Reverted immediately; nothing committed. What the render does, e.g. wiki/star-systems/GJ-1002/index.md: (1) DELETES the Celestial Bodies + Stations & Facilities READ-ONLY blocks — those are written by the Rust atlas CLI (server/src/bin/atlas/sync_wiki.rs), which generate_wiki() does not know about; (2) DELETES the infobox Industries / Exports / Imports / Population rows and replaces them with a Specialization row — nothing in the repo writes the old rows any more, so a render destroys content with no generator; (3) REWRITES star type (GJ 1002: page says G-type, systems.db says M-type — GJ 1002 is a real M dwarf, so the DB is probably right and the pages are stale; needs a decision which is truth); (4) reorders Adjacent Systems (sorted vs star-map order). Consequences: reach wiki exposes only stats + gttr-hook until this is fixed; wiki_sync.generate_wiki/import_from_wiki are kept in tooling/domains/wiki/wiki_sync.py but unexposed; process-wiki-system-changes archived. The wiki skill and CLAUDE.md now warn not to run it. ALSO: D-262 and its diagram (docs/diagrams/data-flow/wiki-generator-flow.d2) present wiki_sync generate_wiki as the ROUTINE DB->wiki path; that edge''s meaning is no longer true and D-262 needs amending (the dataflow gate only checks paths, it cannot catch this — exactly the gap D-262 names). Acceptance: a render of the current DB reproduces the committed pages byte-for-byte (or the diff is a reviewed, intended content change), then expose reach wiki render.', 'backlog', 'high', NULL, 'tooling', NULL, '2026-09-23 14:24:29.276', '2026-09-23 14:24:29.276', NULL, '8c8194e2723412aced90777f689dc7b7', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G65TC359QGCVBEPXHJ4N0MRG', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port wiki, assets and character — the three small remaining domains', 'The tail of the tree — three domains, ~26 files, batched because none of them +entangles with the spatial ladder or the systems.db stamp. + +`reach wiki` — `tooling/wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, +`db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, +`fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, +`process-wiki-system-changes`. +READ `Skill(wiki)` FIRST. The wiki has two ownership models running in opposite +directions and these tools sit on the seam: `star-systems/` (93% of the wiki) is +OUTPUT generated FROM systems.db by `wiki_sync.py`, while the flat categories are +INPUT. A verb that writes to the wrong side is silently reverted on the next sync. +Several of these are one-shot migrations (`migrate-s-to-gj`, `patch-core-sector`) — +decide per file whether it is a live verb or provenance bound for `tooling/archive/`, +the same call T-1271 made for `pql-migrate/`. + +`reach assets` — `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, +`db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py`. +These call tower-of-joy over the network (Stable Audio :11500, Trellis :11510, by +IP per `.claude/rules/local-services.md`). Endpoints live in `tooling/db/config.json` +— that config must keep working or move deliberately, not incidentally. Network +failure is the normal case here, so `ReachError` messages need to distinguish +"the box is off" from "the request was wrong"; `missing_fix=` and `fix=` both matter. + +`reach character` — `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, +`convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, +`check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, +`setup_clothing_metadata.py`. +Much smaller than `garment-fit/`''s 23 files suggest — 22 of them are Blender +payloads that belong to the carve-out (T-1273) and MUST NOT import the package. +Do T-1273 first or the boundary will blur. + +Standard port acceptance as on T-1281/T-1286. Check `__file__`-relative roots +before moving anything — five silent failures so far. + +PROGRESS 2026-09-23 — wiki half done (commit to follow): reach wiki stats + gttr-hook, both output-identical to the originals (stats byte-identical; all 301 gttr hooks identical). wiki_sync.generate_wiki/import_from_wiki moved but NOT exposed: a pre-port run of the old --generate deleted ~10,700 lines of committed pages -> T-1292. Archive: tooling/archive/{pql-migrate,wiki-bootstrap}/ with a README (one-shots + the destructive process-wiki-system-changes + backfill_cultural_corridor, a raw DB patch script outside D-262). find-stubs archived too: it finds 0 stubs, Phase-1 job done. Side fix: tooling/godot-cold-parse + godot-parse-sweep were never retired after T-1283 and pr-process still told agents to run them — removed, skill repointed. Remaining: assets, character.', 'backlog', 'medium', NULL, 'tooling', 'D-263', '2026-09-02 16:17:45.770', '2026-09-23 14:25:42.440', NULL, 'f5425885aa0e5f99f744ab4cdaaf5d63', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; diff --git a/CLAUDE.md b/CLAUDE.md index 72071fe90..d22512b89 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,9 +25,11 @@ Full annotated tree: `.claude/rules/project-structure.md` **`wiki/` is the seed, and it is mostly generated.** 11,864 files — the largest tree in the repo. Two ownership models run in OPPOSITE directions, and mixing them up costs work: `star-systems/` (3,262 pages, 93% of the wiki) is produced -FROM `systems.db` by `tooling/db/wiki_sync.py`, so its `` -sections and body-page frontmatter are OUTPUT — hand-edits are silently reverted -on the next sync. The flat categories (`corporations/`, `factions/`, `economics/`, +FROM `systems.db` by `tooling/domains/wiki/wiki_sync.py` plus the Rust atlas +CLI's wiki sync, so its `` sections and body-page frontmatter +are OUTPUT — do not hand-edit them. (Do not run `wiki_sync.generate_wiki()` +either: it no longer reproduces the committed pages and would delete ~10,700 +lines — T-1290.) The flat categories (`corporations/`, `factions/`, `economics/`, …, 253 pages) are INPUT the generators read. Body frontmatter IS the body definition (`seed`, `land_fraction`, `axial_tilt_deg`); `wiki/economics/` seeds world content generation, not just DB rows. Read `Skill(wiki)` before touching diff --git a/client/tools/parse_sweep.gd b/client/tools/parse_sweep.gd index ef5d1f0b9..eff636bc0 100644 --- a/client/tools/parse_sweep.gd +++ b/client/tools/parse_sweep.gd @@ -2,7 +2,7 @@ extends SceneTree ## Parse-sweep: open EVERY project .gd so the engine reports the broken ones. ## -## Why this exists. `tooling/godot-cold-parse` only ever sees scripts on the +## Why this exists. `reach godot cold-parse` only ever sees scripts on the ## STARTUP path — autoloads and the main scene chain. That is the correct ## scope for the job it was built for (Sprint 36's `Could not find base class ## "MetaScreen"`, an autoload-vs-class_name registration ORDER bug), but it is @@ -18,7 +18,7 @@ extends SceneTree ## ## **This script deliberately makes no verdict of its own.** It opens every ## script and lets Godot's own front-end print the diagnosis; the wrapper -## (`tooling/godot-parse-sweep`) greps for that. Two rejected alternatives, +## (`reach godot parse-sweep`) greps for that. Two rejected alternatives, ## both found the hard way while building this: ## ## - `ResourceLoader.load(path, "GDScript", CACHE_MODE_IGNORE)` **segfaults @@ -34,7 +34,7 @@ extends SceneTree ## either. The engine's stderr is the only honest signal, which is why the ## verdict lives in the wrapper. ## -## Run via `tooling/godot-parse-sweep` (never directly — this half cannot fail). +## Run via `reach godot parse-sweep` (never directly — this half cannot fail). ## Vendor and generated trees are not ours to verify. `addons/` is gdUnit4, ## whose own test corpus deliberately contains broken fixtures — sweeping it diff --git a/docs/architecture/pql-migration.md b/docs/architecture/pql-migration.md index 263ae0ded..e720eb130 100644 --- a/docs/architecture/pql-migration.md +++ b/docs/architecture/pql-migration.md @@ -13,7 +13,7 @@ tags: [migration, tooling, pql, decisions, ticketing] > the phases-as-hierarchy refactor + Phase-6 SQLite retirement in PR #155. It is kept > as a historical record of *why and how* the planning system moved off the binary > SQLite DB onto pql. **Current usage** is in `CLAUDE.md`, `.claude/rules/ticket-cli.md`, -> and `governance/README.md`; the one-shot transforms live in `tooling/pql-migrate/`. +> and `governance/README.md`; the one-shot transforms live in `tooling/archive/pql-migrate/`. > Open feedback for the pql team is in the root `pql-requirements.md`. # pql Migration Plan — decisions + ticketing diff --git a/server/data/systems-schema.sql b/server/data/systems-schema.sql index 4ba45072a..959c7ab0e 100644 --- a/server/data/systems-schema.sql +++ b/server/data/systems-schema.sql @@ -3,7 +3,7 @@ -- Authored via tooling pipeline; wiki pages are generated views. -- Shipped with the game — read-only from the server's perspective. -- --- Rebuild: tooling/process-wiki-system-changes --rebuild-db +-- Rebuild: make regen-db (the wiki->DB rebuild script is archived, T-1290) PRAGMA journal_mode=WAL; PRAGMA foreign_keys=ON; @@ -41,7 +41,7 @@ CREATE TABLE IF NOT EXISTS star_systems ( -- Short narrative hook extracted from the wiki's gttr.md file — the -- first characterisation paragraph ("where the rules live", "forty -- years old and still in the draft", etc). Populated by - -- tooling/db/populate_gttr_hook.py from wiki/star-systems//gttr.md. + -- `reach wiki gttr-hook` from wiki/star-systems//gttr.md. -- Used as compact cultural context in the Gemma 2 naming pipeline -- (#833) so per-system feel is grounded in the canonical identity -- rather than generic corridor labels. diff --git a/server/data/systems.db b/server/data/systems.db index d027f030a..4ca489829 100644 Binary files a/server/data/systems.db and b/server/data/systems.db differ diff --git a/tooling/DOMAINS.md b/tooling/DOMAINS.md index a277563db..254d5b74f 100644 --- a/tooling/DOMAINS.md +++ b/tooling/DOMAINS.md @@ -5,8 +5,8 @@ before anything moves. This is the artefact the per-domain port tickets are written from — without it their boundaries would be guesses, renegotiated halfway through the move. -**Status:** map only. Nothing here has moved yet except `check client-version`, -which shipped in T-1262 as the reference implementation. +**Status:** the map was written before anything moved; the ✅ rows below record +which domains have since been ported, and by which ticket. ## What the survey changed @@ -90,11 +90,11 @@ invented. | ~~`starmap`~~ | **folded into `atlas map`** — the top rung of the same ladder | — | | ~~`planet`~~ | **folded into `atlas planet`** — the third rung of the same ladder | — | | `ledger` | the economics pipeline, named for the UI component that will aggregate it | ✅ ported (T-1289). `economy-db/` → `domains/ledger/` (`economy_import/` kept by name; the entrypoint became `service.py`), `schema_version.py` with it. `reach ledger import`; `make regen-db` survives as a one-line delegate, `make economy-db` retired. `generated_brands.toml` byte-identical across the move | -| `wiki` | wiki sync and content maintenance | `wiki/`, `db/wiki_sync.py`, `db/populate_gttr_hook.py`, `db/backfill_cultural_corridor.py`, `assign-astro-ids.py`, `fill-missing-globes.py`, `migrate-s-to-gj.py`, `patch-core-sector.py`, `process-wiki-system-changes` | +| `wiki` | wiki sync and content maintenance | ✅ ported (T-1290): `reach wiki stats`, `reach wiki gttr-hook` (both output-identical to the originals). `wiki_sync.py` moved whole, but its renderer and importer are NOT verbs — re-rendering deletes ~10,700 lines of committed pages (T-1292). The seven one-shots (`assign-astro-ids`, `migrate-s-to-gj`, `patch-core-sector`, `fill-missing-globes`, `generate-stubs`/`find-stubs`, `backfill_cultural_corridor`) and the destructive `process-wiki-system-changes` went to `archive/wiki-bootstrap/` | | `assets` | connectors to the tower-of-joy generators | `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, `db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py` | | `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** | | `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` | -| `godot` | Godot parse and cold-start checks | `godot-parse-sweep`, `godot-cold-parse` | +| `godot` | Godot parse and cold-start checks | ✅ ported (T-1283); the two bash originals were left beside the port and retired in T-1290 | | `generate` | content generators not owned elsewhere | ✅ ported (T-1286). `generate-brands` + `generate-corporations` collapsed into `core.process.cargo_binary` — they were the same 24 lines of bash a third time | | `dev` | developer environment and workflow | ✅ ported (T-1286). The three environment scripts split decision from performing — `godot_plan`/`worktree_plan` are pure and pinned by `test_environment.py` | | `pr` | the PR/review loop | ✅ ported (T-1286). `watchlist-diff` now reads the watched set from `generator_sources.py` instead of restating it | diff --git a/tooling/archive/README.md b/tooling/archive/README.md new file mode 100644 index 000000000..c36f9e59f --- /dev/null +++ b/tooling/archive/README.md @@ -0,0 +1,21 @@ +# tooling/archive — provenance, not tooling + +Scripts kept because they record HOW something in the repo came to be, not +because anything should run them again. None is a `reach` verb, none is +importable (no `__init__.py`, excluded from package discovery and from the +D-263 conformance scope), and several would do damage if run today. Deleting +them would lose the provenance; keeping them live would imply they still work +(tooling/DOMAINS.md, "pql-migrate/ is provenance, not tooling"). + +| path | what it did | why it is here | +|---|---|---| +| `pql-migrate/` | the one-shot planning-store migration into pql | completed migration (T-1271 ruling) | +| `wiki-bootstrap/assign-astro-ids.py` + `gj-catalog-real.json` | gave the 300 `S-NNN` systems real Gliese-Jahreiss ids | done; `S-` ids no longer exist | +| `wiki-bootstrap/migrate-s-to-gj.py` | rewrote every `S-NNN` reference to its GJ id | done | +| `wiki-bootstrap/patch-core-sector.py` | added 17 hand-balanced core-sector gate edges | done; hardcodes a `planning/` path that no longer exists | +| `wiki-bootstrap/generate-stubs.py`, `find-stubs` | created and then tracked the Phase-1 system stubs | done; `find-stubs --all-sectors 0 99` finds 0 stubs | +| `wiki-bootstrap/fill-missing-globes.py` | copied donor `globe.png` files into bodies that lacked one | self-described one-shot | +| `wiki-bootstrap/backfill_cultural_corridor.py` | copied `geographic_sector` into `cultural_corridor` | a raw systems.db patch script, the kind `.claude/rules/asset-pipeline.md` forbids; absent from the D-262 flow | +| `wiki-bootstrap/process-wiki-system-changes` | wiki → systems.db → star-map.json → re-rendered system pages | **destructive today** — its last step is `wiki_sync.generate_wiki()`, which no longer reproduces the committed pages (T-1290 finding, see the wiki domain's router) | + +Archived in T-1250 / T-1290 (2026-09-23). diff --git a/tooling/pql-migrate/add_workshop_provenance.py b/tooling/archive/pql-migrate/add_workshop_provenance.py similarity index 100% rename from tooling/pql-migrate/add_workshop_provenance.py rename to tooling/archive/pql-migrate/add_workshop_provenance.py diff --git a/tooling/pql-migrate/repath_references.py b/tooling/archive/pql-migrate/repath_references.py similarity index 100% rename from tooling/pql-migrate/repath_references.py rename to tooling/archive/pql-migrate/repath_references.py diff --git a/tooling/pql-migrate/restructure_decisions.py b/tooling/archive/pql-migrate/restructure_decisions.py similarity index 100% rename from tooling/pql-migrate/restructure_decisions.py rename to tooling/archive/pql-migrate/restructure_decisions.py diff --git a/tooling/pql-migrate/retag_ticket_refs.py b/tooling/archive/pql-migrate/retag_ticket_refs.py similarity index 100% rename from tooling/pql-migrate/retag_ticket_refs.py rename to tooling/archive/pql-migrate/retag_ticket_refs.py diff --git a/tooling/pql-migrate/seed_tickets.py b/tooling/archive/pql-migrate/seed_tickets.py similarity index 100% rename from tooling/pql-migrate/seed_tickets.py rename to tooling/archive/pql-migrate/seed_tickets.py diff --git a/tooling/assign-astro-ids.py b/tooling/archive/wiki-bootstrap/assign-astro-ids.py similarity index 100% rename from tooling/assign-astro-ids.py rename to tooling/archive/wiki-bootstrap/assign-astro-ids.py diff --git a/tooling/db/backfill_cultural_corridor.py b/tooling/archive/wiki-bootstrap/backfill_cultural_corridor.py similarity index 100% rename from tooling/db/backfill_cultural_corridor.py rename to tooling/archive/wiki-bootstrap/backfill_cultural_corridor.py diff --git a/tooling/fill-missing-globes.py b/tooling/archive/wiki-bootstrap/fill-missing-globes.py similarity index 100% rename from tooling/fill-missing-globes.py rename to tooling/archive/wiki-bootstrap/fill-missing-globes.py diff --git a/tooling/wiki/find-stubs b/tooling/archive/wiki-bootstrap/find-stubs similarity index 100% rename from tooling/wiki/find-stubs rename to tooling/archive/wiki-bootstrap/find-stubs diff --git a/tooling/wiki/generate-stubs.py b/tooling/archive/wiki-bootstrap/generate-stubs.py similarity index 100% rename from tooling/wiki/generate-stubs.py rename to tooling/archive/wiki-bootstrap/generate-stubs.py diff --git a/tooling/wiki/gj-catalog-real.json b/tooling/archive/wiki-bootstrap/gj-catalog-real.json similarity index 100% rename from tooling/wiki/gj-catalog-real.json rename to tooling/archive/wiki-bootstrap/gj-catalog-real.json diff --git a/tooling/migrate-s-to-gj.py b/tooling/archive/wiki-bootstrap/migrate-s-to-gj.py similarity index 100% rename from tooling/migrate-s-to-gj.py rename to tooling/archive/wiki-bootstrap/migrate-s-to-gj.py diff --git a/tooling/patch-core-sector.py b/tooling/archive/wiki-bootstrap/patch-core-sector.py similarity index 100% rename from tooling/patch-core-sector.py rename to tooling/archive/wiki-bootstrap/patch-core-sector.py diff --git a/tooling/process-wiki-system-changes b/tooling/archive/wiki-bootstrap/process-wiki-system-changes similarity index 100% rename from tooling/process-wiki-system-changes rename to tooling/archive/wiki-bootstrap/process-wiki-system-changes diff --git a/tooling/domains/ledger/service.py b/tooling/domains/ledger/service.py old mode 100755 new mode 100644 diff --git a/tooling/domains/wiki/__init__.py b/tooling/domains/wiki/__init__.py new file mode 100644 index 000000000..9f15a95f7 --- /dev/null +++ b/tooling/domains/wiki/__init__.py @@ -0,0 +1,11 @@ +"""`wiki` — the wiki seed's tooling (D-263). + +Mirrors an implant app: the wiki/GTTR popups the player reads. Before touching +anything here, read `Skill(wiki)` — two ownership models run in opposite +directions through this directory, and D-262's flow diagram +(docs/diagrams/data-flow/wiki-generator-flow.d2) is the canonical map. + +Formerly tooling/db/wiki_sync.py and tooling/db/populate_gttr_hook.py +(T-1290). The one-shot bootstrap scripts that sat beside them went to +tooling/archive/wiki-bootstrap/. +""" diff --git a/tooling/db/populate_gttr_hook.py b/tooling/domains/wiki/gttr_hook.py similarity index 53% rename from tooling/db/populate_gttr_hook.py rename to tooling/domains/wiki/gttr_hook.py index c353f3b81..574ce9f8e 100755 --- a/tooling/db/populate_gttr_hook.py +++ b/tooling/domains/wiki/gttr_hook.py @@ -1,6 +1,4 @@ -#!/usr/bin/env python3 -""" -Extract a short narrative hook from every wiki/star-systems//gttr.md +"""Extract a short narrative hook from every wiki/star-systems//gttr.md and store it on `star_systems.gttr_hook`. The GTTR ("Drifter's Guide to the Reach") files follow a consistent @@ -19,35 +17,32 @@ cultural signal available for each system, capturing things like: - Posto Avançado: "forward post — the place beyond the established line" -This script parses each gttr.md, regex-extracts the first `**NAME**` -paragraph, normalises whitespace, and truncates at a soft word cap so -the hook stays cheap to inject into prompts. Empty or unmatched files -leave the column NULL. +This parses each gttr.md, regex-extracts the first `**NAME**` paragraph, +normalises whitespace, and truncates at a soft word cap so the hook stays +cheap to inject into prompts. Empty or unmatched files leave the column NULL. Idempotent, safe to re-run after any wiki update. Explicit transaction -wrapper with rollback on exception. +wrapper with rollback on exception. It writes systems.db directly, which the +asset-pipeline rule otherwise forbids; this edge is sanctioned by the D-262 +flow diagram (`populate_gttr_hook -> systems.db`), and `gttr_hook` sits on +`star_systems`, which `reach ledger import` does not clear. -Usage: - tooling/db/populate_gttr_hook.py - tooling/db/populate_gttr_hook.py --max-words 45 - tooling/db/populate_gttr_hook.py --dry-run - tooling/db/populate_gttr_hook.py --system "GJ 71" +Formerly tooling/db/populate_gttr_hook.py (T-1290). """ -import argparse +from __future__ import annotations + import re import sqlite3 -import sys from pathlib import Path -SCRIPT_DIR = Path(__file__).resolve().parent -REPO_ROOT = (SCRIPT_DIR / ".." / "..").resolve() +from tooling.core import config, console +from tooling.core.errors import ReachError + +REPO_ROOT = config.repo_root() DB_PATH = REPO_ROOT / "server" / "data" / "systems.db" WIKI_SYSTEMS = REPO_ROOT / "wiki" / "star-systems" -# Matches the first paragraph that opens with `**NAME**` at the start -# of a line. Captures everything up to a blank line or the next H1/H2. -# Non-greedy on the content. # Match a line that opens with `**NAME**` (any non-asterisk chars, since # names contain accents from many alphabets — Á, Ž, Ç, Ñ, etc) followed # by the paragraph body up to a blank line or the next markdown heading. @@ -86,10 +81,7 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None: # Collapse whitespace so multi-line paragraphs become one clean line. hook = re.sub(r"\s+", " ", hook).strip() - # Strip a leading orphan "is" that comes from `**NAME**` + " is …": - # the regex captures the word "is" on its own because the opener is - # typically `**GATEWAY** (known as Tau Ceti...) is the most connected…`. - # Normal reading already works — this is just hygiene. + # Drop an empty parenthetical left behind by the collapse — hygiene only. hook = re.sub(r"\s*\(\s*\)\s*", " ", hook) # Hard cap at max_words. Truncate at the last word boundary before @@ -101,37 +93,20 @@ def extract_hook(gttr_path: Path, max_words: int) -> str | None: return hook -def main(): - parser = argparse.ArgumentParser( - description="Populate star_systems.gttr_hook from wiki gttr.md files" - ) - parser.add_argument("--db", default=str(DB_PATH), help="Path to systems.db") - parser.add_argument( - "--system", - help="Process only this system_id (e.g. 'GJ 71')", - ) - parser.add_argument( - "--max-words", - type=int, - default=45, - help="Soft cap on hook length (default: 45 words)", - ) - parser.add_argument( - "--dry-run", - action="store_true", - help="Extract and print but do not write to the DB", - ) - parser.add_argument( - "--verbose", - action="store_true", - help="Print every extracted hook", - ) - args = parser.parse_args() - - db_path = Path(args.db) +def run( + db_path: Path = DB_PATH, + *, + system: str | None = None, + max_words: int = 45, + dry_run: bool = False, + verbose: bool = False, +) -> dict[str, int]: + """Populate star_systems.gttr_hook. Returns updated/missing/unmatched counts.""" if not db_path.exists(): - print(f"error: {db_path} not found", file=sys.stderr) - sys.exit(1) + raise ReachError( + f"{db_path} not found", + fix="pass --db with an existing systems.db, or restore it: git restore server/data/systems.db", + ) conn = sqlite3.connect(str(db_path), timeout=30.0) conn.execute("PRAGMA journal_mode=WAL") @@ -144,22 +119,21 @@ def main(): except sqlite3.OperationalError: pass # column already exists - print("\n populate_gttr_hook.py") - print(f" DB: {db_path}") - print(f" max words: {args.max_words}") - if args.dry_run: - print(" Mode: DRY RUN") - print() + console.event( + f"gttr-hook into {db_path} (max {max_words} words)" + (" — DRY RUN" if dry_run else "") + ) - rows = conn.execute( - "SELECT system_id, proper_name FROM star_systems ORDER BY system_id" - ).fetchall() - if args.system: - rows = [r for r in rows if r[0] == args.system] + rows = conn.execute("SELECT system_id, proper_name FROM star_systems ORDER BY system_id").fetchall() + if system: + rows = [r for r in rows if r[0] == system] + if not rows: + conn.close() + raise ReachError( + f"no system {system!r} in star_systems", + fix="pass a system_id with its space, e.g. --system 'GJ 71'", + ) - updated = 0 - missing = 0 - unmatched = 0 + counts = {"updated": 0, "missing": 0, "unmatched": 0} conn.execute("BEGIN") try: @@ -167,31 +141,26 @@ def main(): slug = system_id.replace(" ", "-", 1) gttr_path = WIKI_SYSTEMS / slug / "gttr.md" - hook = extract_hook(gttr_path, args.max_words) + hook = extract_hook(gttr_path, max_words) if hook is None: - if not gttr_path.exists(): - missing += 1 - if args.verbose: - print(f" MISSING {system_id:10s} {gttr_path}") - else: - unmatched += 1 - if args.verbose: - print(f" UNMATCHED {system_id:10s} {gttr_path}") + kind = "missing" if not gttr_path.exists() else "unmatched" + counts[kind] += 1 + if verbose: + console.event(f"{kind.upper()} {system_id} {gttr_path}", level="warn") continue - if args.verbose: + if verbose: display = f"{system_id} ({proper_name})" if proper_name else system_id - print(f" {display}") - print(f" → {hook}") + console.out(f"{display}\n → {hook}") - if not args.dry_run: + if not dry_run: conn.execute( "UPDATE star_systems SET gttr_hook = ? WHERE system_id = ?", (hook, system_id), ) - updated += 1 + counts["updated"] += 1 - if args.dry_run: + if dry_run: conn.rollback() else: conn.commit() @@ -201,15 +170,4 @@ def main(): raise conn.close() - - print("\n Done:") - print(f" updated: {updated}") - print(f" missing: {missing}") - print(f" unmatched: {unmatched}") - if args.dry_run: - print("\n Dry run — no DB writes.") - print() - - -if __name__ == "__main__": - main() + return counts diff --git a/tooling/domains/wiki/router.py b/tooling/domains/wiki/router.py new file mode 100644 index 000000000..dcc59ced1 --- /dev/null +++ b/tooling/domains/wiki/router.py @@ -0,0 +1,64 @@ +"""Transport for the `wiki` domain — args in, delegate, format out. + +Two verbs, deliberately not four. `wiki_sync` also holds `generate_wiki()` +(systems.db → system pages) and `import_from_wiki()` (pages → systems.db), and +neither is exposed: re-rendering today would delete ~10,700 lines of the +committed pages (see wiki_sync's docstring, T-1290). A verb is a promise that +running it is safe, and that one would not be. They come back when the +renderer reproduces the committed pages byte-for-byte. +""" + +from __future__ import annotations + +from pathlib import Path + +import typer + +from tooling.core import cli, console +from tooling.core.command import command + +app = cli.domain("wiki", "The wiki seed — fill rates and the GTTR hook.") + + +@app.callback() +def _domain() -> None: + """Keeps `wiki` a group (Typer collapses a single-command app).""" + + +@app.command("stats") +@command +def stats() -> None: + """Fill rates of the structured star-system columns in systems.db.""" + from tooling.domains.wiki import wiki_sync + + wiki_sync.stats() + + +@app.command("gttr-hook") +@command +def gttr_hook( + db: Path = typer.Option(None, "--db", help="systems.db to write (default: server/data/systems.db)."), + system: str = typer.Option(None, "--system", help="Only this system_id, e.g. 'GJ 71'."), + max_words: int = typer.Option(45, "--max-words", help="Soft cap on hook length."), + dry_run: bool = typer.Option(False, "--dry-run", help="Extract, write nothing."), + verbose: bool = typer.Option(False, "--verbose", help="Print every extracted hook."), +) -> None: + """Extract each system's GTTR opening line into star_systems.gttr_hook. + + Reads wiki/star-systems//gttr.md, takes the first **NAME** + paragraph, collapses and caps it. Re-run after editing any gttr.md. + """ + from tooling.domains.wiki import gttr_hook as service + + counts = service.run( + db or service.DB_PATH, + system=system, + max_words=max_words, + dry_run=dry_run, + verbose=verbose, + ) + written = "extracted, nothing written (dry run)" if dry_run else "written" + console.verdict( + f"gttr-hook: {counts['updated']} {written}, " + f"{counts['missing']} missing gttr.md, {counts['unmatched']} unmatched" + ) diff --git a/tooling/db/wiki_sync.py b/tooling/domains/wiki/wiki_sync.py similarity index 92% rename from tooling/db/wiki_sync.py rename to tooling/domains/wiki/wiki_sync.py index 12a03c6b2..37d05cf3e 100755 --- a/tooling/db/wiki_sync.py +++ b/tooling/domains/wiki/wiki_sync.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python3 """ Settled Reach Wiki Sync — wiki frontmatter ↔ systems.db @@ -13,6 +12,18 @@ This module provides: - generate_wiki(): render wiki pages from DB + existing prose - stats(): show completion stats +**generate_wiki() is NOT safe to run against the committed wiki** (T-1290). +Re-rendering today deletes ~10,700 lines across all 301 system pages: the +Celestial Bodies / Stations blocks (written by the Rust atlas CLI's +sync_wiki.rs, which this renderer knows nothing about), the Industries / +Exports / Imports rows (which nothing in the repo writes any more), and it +rewrites star types where systems.db and the pages disagree. That is why +`reach wiki` exposes only `stats` and `gttr-hook`. The functions stay, because +fixing the renderer means fixing them, not rewriting them from nothing. + +Formerly tooling/db/wiki_sync.py, run by hand or via the now-archived +process-wiki-system-changes pipeline. + Database: server/data/systems.db Schema: server/data/systems-schema.sql """ @@ -20,15 +31,15 @@ Schema: server/data/systems-schema.sql import json import re import sqlite3 -import sys from pathlib import Path +from tooling.core import config, console + # --------------------------------------------------------------------------- # Paths # --------------------------------------------------------------------------- -SCRIPT_DIR = Path(__file__).resolve().parent -WORKTREE_ROOT = (SCRIPT_DIR / ".." / "..").resolve() +WORKTREE_ROOT = config.repo_root() SCHEMA_PATH = WORKTREE_ROOT / "server" / "data" / "systems-schema.sql" WIKI_DIR = WORKTREE_ROOT / "wiki" / "star-systems" DB_PATH = WORKTREE_ROOT / "server" / "data" / "systems.db" @@ -613,14 +624,16 @@ def generate_wiki(prose_only_update=False): # Topology (always last) page_lines.append(topology) - outdir.mkdir(parents=True, exist_ok=True) - outfile.write_text("\n".join(page_lines), encoding="utf-8") - + # Counted BEFORE the write. Checking exists() after writing (as this + # did until T-1290) made every page an update and `created` always 0. if outfile.exists(): updated += 1 else: created += 1 + outdir.mkdir(parents=True, exist_ok=True) + outfile.write_text("\n".join(page_lines), encoding="utf-8") + conn.close() return { "ok": True, @@ -635,19 +648,26 @@ def generate_wiki(prose_only_update=False): # --------------------------------------------------------------------------- -def stats(): - """Show per-table fill rates.""" +def stats() -> dict: + """Per-table fill rates for the structured system columns. + + The table is the command's output, so it goes to stdout (console.out). + A missing table used to print a hint and exit 0 — a failure reading as + success — and now raises with the remedy (T-1290). + """ + from tooling.core.errors import ReachError + conn = get_connection() try: row = conn.execute("SELECT COUNT(*) as cnt FROM star_systems").fetchone() total = row["cnt"] if total == 0: - print("No systems in database. Run import first.") + console.out("No systems in database.") return {"ok": True, "total": 0} - print(f"Star systems: {total}") - print(f"Database: {DB_PATH}") - print() + console.out(f"Star systems: {total}") + console.out(f"Database: {DB_PATH}") + console.out() total_cells = 0 filled_cells = 0 @@ -665,64 +685,24 @@ def stats(): filled_cells += table_filled pct = table_filled / table_total * 100 if table_total > 0 else 0 bar = "#" * int(pct / 5) + "." * (20 - int(pct / 5)) - print(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%") + console.out(f" {table:20s} {table_filled:4d}/{table_total:4d} {bar} {pct:.0f}%") # Historical events ev_count = conn.execute("SELECT COUNT(*) as cnt FROM historical_events").fetchone()["cnt"] ev_systems = conn.execute( "SELECT COUNT(DISTINCT system_id) as cnt FROM historical_events" ).fetchone()["cnt"] - print(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems") + console.out(f" {'historical_events':20s} {ev_count} events across {ev_systems} systems") overall = filled_cells / total_cells * 100 if total_cells > 0 else 0 - print(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)") + console.out(f"\nOverall: {filled_cells}/{total_cells} ({overall:.1f}%)") return {"ok": True, "total": total, "filled": filled_cells, "pct": round(overall, 1)} except sqlite3.OperationalError as exc: - print(f"Table not found — run import first: {exc}") - return {"ok": False, "error": str(exc)} + raise ReachError( + f"systems.db is missing a system table: {exc}", + fix="make regen-db, or restore it: git restore server/data/systems.db", + ) from exc finally: conn.close() - - -# --------------------------------------------------------------------------- -# CLI -# --------------------------------------------------------------------------- - - -def main(): - help_text = f"""\ -Wiki Sync — systems.db ↔ wiki pages - -Usage: - wiki_sync.py import Parse wiki frontmatter into systems.db (bootstrap) - wiki_sync.py generate Generate wiki pages from systems.db - wiki_sync.py stats Show completion stats - wiki_sync.py --help Show this help - -Database: {DB_PATH} -Schema: {SCHEMA_PATH} -Wiki: {WIKI_DIR} -""" - if len(sys.argv) < 2 or sys.argv[1] in ("--help", "-h", "help"): - print(help_text) - sys.exit(0) - - cmd = sys.argv[1] - - if cmd == "import": - result = import_from_wiki() - print(json.dumps(result, indent=2)) - elif cmd == "generate": - result = generate_wiki() - print(json.dumps(result, indent=2)) - elif cmd == "stats": - stats() - else: - print(json.dumps({"ok": False, "error": f"Unknown: {cmd}"}, indent=2)) - sys.exit(1) - - -if __name__ == "__main__": - main() diff --git a/tooling/godot-cold-parse b/tooling/godot-cold-parse deleted file mode 100755 index 613813f16..000000000 --- a/tooling/godot-cold-parse +++ /dev/null @@ -1,96 +0,0 @@ -#!/usr/bin/env bash -# tooling/godot-cold-parse [--run-menu] — cold-cache headless parse check. -# -# Used by /pr-process step 1c before push. Deletes the cached script-class -# registry so the parse simulates the cold-start ordering CI / fresh clones -# see: Sprint 36 close caught a new `class_name MetaScreen` base class and -# six extending scripts that parsed fine on warm developer caches but hit -# `Could not find base class "MetaScreen"` post-merge, because the -# autoload-vs-class_name registration order only resolves correctly once the -# class cache is seeded (see CLAUDE.md -> GDScript conventions -> Autoload -# parse-order rule). -# -# Godot's resource scanner emits category errors (e.g. "Export type can only -# be built-in, a resource, a node, or an enum") that do NOT always prefix -# with SCRIPT ERROR — they appear as plain ERROR lines. The filter below -# catches both, then drops known pre-existing noise from the autoload -# class_name parse-order trap. Sprint 36 shipped a scanner error the old -# narrower grep missed; this is why the filter stays wide. -# -# --run-menu: also launch main_menu.tscn briefly (for branches with UI changes). -# -# Exit 0 + "clean" if no matches. Exit 1 + the matched lines if any are found. -set -euo pipefail - -REPO_ROOT="$(git rev-parse --show-toplevel)" -RUN_MENU=false -[ "${1:-}" = "--run-menu" ] && RUN_MENU=true - -rm -f "$REPO_ROOT/client/.godot/global_script_class_cache.cfg" - -# A truly cold checkout (fresh clone or worktree — .godot/ is gitignored) has -# no resource-import cache, and every imported asset (fonts, ogg) then "fails -# loading" during the parse run: a wall of false positives. Seed the cache -# with an import pass first; source assets are tracked, so this is always -# reconstructible. (Found live: first run in a fresh worktree, 2026-07-13.) -if [ ! -d "$REPO_ROOT/client/.godot/imported" ] || [ -z "$(ls -A "$REPO_ROOT/client/.godot/imported" 2>/dev/null)" ]; then - echo "godot-cold-parse: no import cache — running one-time import pass..." >&2 - set +e - IMPORT_OUT=$(godot --headless --path "$REPO_ROOT/client" --import 2>&1) - IMPORT_EXIT=$? - set -e - if [ "$IMPORT_EXIT" -ne 0 ]; then - echo "godot-cold-parse: import pass exited $IMPORT_EXIT" >&2 - printf '%s\n' "$IMPORT_OUT" | tail -20 >&2 - exit "$IMPORT_EXIT" - fi -fi - -FILTER='^(SCRIPT )?ERROR|Parse Error|Export type' - -# Capture the godot run separately from the filter pipeline: with the -# trailing `|| true` on the greps, a nonzero exit from godot itself (crash, -# missing binary, corrupted install) would otherwise report "clean". Nothing -# downstream re-reads the raw output now that this is scripted, so fail loud. -set +e -RAW=$(godot --headless --path "$REPO_ROOT/client" --quit 2>&1) -GODOT_EXIT=$? -set -e -if [ "$GODOT_EXIT" -ne 0 ]; then - echo "godot-cold-parse: godot itself exited $GODOT_EXIT — not a parse verdict" >&2 - printf '%s\n' "$RAW" | tail -20 >&2 - exit "$GODOT_EXIT" -fi - -MATCHES=$(printf '%s\n' "$RAW" \ - | grep -iE "$FILTER" \ - | grep -v "Failed loading resource: res://assets" \ - | grep -v "Cannot infer the type" \ - | grep -vE '(Messagepack|LocalBridge|ServerProcess|Constants)" not declared' || true) - -if [ "$RUN_MENU" = true ]; then - # Deliberately no exit-code check here: `timeout` kills the menu after - # 10s by design (exit 124 is the expected shutdown path); only the - # scraped error lines carry signal for this bounded run. - MENU_MATCHES=$(timeout 10 godot --path "$REPO_ROOT/client" res://scenes/main_menu.tscn 2>&1 \ - | grep -iE "$FILTER" || true) - if [ -n "$MENU_MATCHES" ]; then - MATCHES="$MATCHES -$MENU_MATCHES" - fi -fi - -if [ -n "$MATCHES" ]; then - echo "$MATCHES" - exit 1 -fi - -# Restore a FULL class cache before exiting: the cold parse run re-seeds -# global_script_class_cache.cfg only partially — addon classes (e.g. gdUnit4's -# GdUnitTestCIRunner) are missing, which leaves tests/run-godot unable to even -# start (0 tests in ~350ms; found live when the pre-push gate ran the suite -# right after this script, 2026-07-14). The cold verdict above is already -# decided; this restore just returns the tree to a runnable state. -godot --headless --path "$REPO_ROOT/client" --import > /dev/null 2>&1 || true - -echo "godot-cold-parse: clean" diff --git a/tooling/godot-parse-sweep b/tooling/godot-parse-sweep deleted file mode 100755 index ee50c275c..000000000 --- a/tooling/godot-parse-sweep +++ /dev/null @@ -1,64 +0,0 @@ -#!/usr/bin/env bash -# tooling/godot-parse-sweep — open every project .gd and fail on any that won't parse. -# -# Complements `godot-cold-parse`, which does NOT cover this. That script only -# ever sees scripts on the STARTUP path (autoloads + the main scene chain) — -# correct for the registration-ORDER bug it was built for, far narrower than -# its name implies. Verified 2026-07-27 by breaking a non-startup UI script -# and a test file in turn: cold-parse reported "clean", exit 0, for both. -# -# Division of labour, deliberate: -# godot-cold-parse — cold-cache STARTUP ordering (class_name/autoload race) -# godot-parse-sweep — does every file in the project parse at all -# -# The GDScript half (client/tools/parse_sweep.gd) only opens files; it makes no -# verdict, because no Godot API reports GDScript parse failure reliably (see -# that file's header — one segfaults, one false-positives 150/226, and plain -# load() returns non-null for a broken script). The engine's own stderr is the -# only honest signal, so the verdict is made here by scraping it. -# -# Exit 0 + "clean" if every script parsed. Exit 1 + the offending lines if not. -set -euo pipefail - -REPO_ROOT="$(git rev-parse --show-toplevel)" - -set +e -RAW=$(godot --headless --path "$REPO_ROOT/client" -s res://tools/parse_sweep.gd 2>&1) -GODOT_EXIT=$? -set -e - -if [ "$GODOT_EXIT" -ne 0 ]; then - echo "godot-parse-sweep: godot itself exited $GODOT_EXIT — not a parse verdict" >&2 - printf '%s\n' "$RAW" | tail -20 >&2 - exit "$GODOT_EXIT" -fi - -# The sweep must actually have run. Without this, a future change that breaks -# the walk (or renames the script) would produce zero error lines and read as -# a clean sweep — the same false-green shape this tool exists to close. -if ! printf '%s\n' "$RAW" | grep -q '^parse-sweep: opened'; then - echo "godot-parse-sweep: the sweep did not report completion — no verdict possible" >&2 - printf '%s\n' "$RAW" | tail -20 >&2 - exit 1 -fi -SUMMARY=$(printf '%s\n' "$RAW" | grep '^parse-sweep: opened') - -# NOTE: unlike godot-cold-parse, "Cannot infer the type" is NOT filtered here. -# That filter is why cold-parse stayed silent about tests/util/scene_helper.gd, -# which genuinely does not parse — the suppressed class was hiding a real -# failure, not noise. -MATCHES=$(printf '%s\n' "$RAW" \ - | grep -E 'Parse Error|Failed to load script' \ - | grep -v "Failed loading resource: res://assets" || true) - -if [ -n "$MATCHES" ]; then - echo "PARSE SWEEP FAILED — at least one script does not parse." >&2 - printf '%s\n' "$MATCHES" | head -40 >&2 - echo "" >&2 - echo " ${SUMMARY}" >&2 - echo " An unparseable file cannot run. If it is a test suite, it did not" >&2 - echo " execute, and any pass count reported elsewhere excludes it." >&2 - exit 1 -fi - -echo "godot-parse-sweep: clean — ${SUMMARY#parse-sweep: }" diff --git a/tooling/main.py b/tooling/main.py index 9d4b7594e..f0da35e65 100644 --- a/tooling/main.py +++ b/tooling/main.py @@ -63,6 +63,10 @@ DOMAINS: dict[str, tuple[str, str]] = { "tooling.domains.ledger.router:app", "The economics pipeline — the import that builds systems.db", ), + "wiki": ( + "tooling.domains.wiki.router:app", + "The wiki seed — fill rates and the GTTR hook", + ), "godot": ( "tooling.domains.godot.router:app", "Does the client parse, and does it parse cold", diff --git a/wiki/GOVERNANCE.md b/wiki/GOVERNANCE.md index 7511a230c..0624d2b5d 100644 --- a/wiki/GOVERNANCE.md +++ b/wiki/GOVERNANCE.md @@ -15,7 +15,7 @@ > input. Where this text and the diagram disagree, the diagram is right. The wiki has two kinds of content with different sources of truth (see -`tooling/db/wiki_sync.py`): +`tooling/domains/wiki/wiki_sync.py`): - **DB owns structured fields, wiki owns prose** (`star-systems/`): identity, gates, history, economy, factions, and culture live in `server/data/systems.db` diff --git a/wiki/_templates/star-system.md b/wiki/_templates/star-system.md index 325c13344..1dbe21be7 100644 --- a/wiki/_templates/star-system.md +++ b/wiki/_templates/star-system.md @@ -2,7 +2,7 @@ System pages carry NO YAML frontmatter. Structured fields live in server/data/systems.db (DB owns structured, wiki owns prose — see -tooling/db/wiki_sync.py and wiki/GOVERNANCE.md). Pages are produced by the +tooling/domains/wiki/wiki_sync.py and wiki/GOVERNANCE.md). Pages are produced by the generator pipeline; the authoring surface is the five prose sections below. Sections marked READ-ONLY are regenerated from their source — never edit them by hand. -->