diff --git a/.claude/rules/tea-cli.md b/.claude/rules/tea-cli.md index 87eca3fba..afaa66b22 100644 --- a/.claude/rules/tea-cli.md +++ b/.claude/rules/tea-cli.md @@ -12,8 +12,8 @@ tea pr list --login schweitz --repo jpmschweitzer/settled-reach --state open --o tea pr --login schweitz --repo jpmschweitzer/settled-reach --comments -o simple # Post a comment on a PR (or issue) — use @filepath for long comments -tooling/tea-comment @/tmp/review.md -tooling/tea-comment "short inline comment" +reach pr comment @/tmp/review.md +reach pr comment "short inline comment" # Approve a PR tea pr approve --login schweitz --repo jpmschweitzer/settled-reach @@ -26,7 +26,7 @@ tea issue list --login schweitz --repo jpmschweitzer/settled-reach --state open - **All flags must be explicit** — omitting `--login` or `--repo` triggers interactive prompts that crash in Claude Code (no TTY). **One exception: `tea pr create` must OMIT `--repo`** — see Pull requests below. - **Use `--output simple`** for machine-readable output (no table borders) -- **For comments, use `tooling/tea-comment @/tmp/file.md`** for long comments (write to file first, then pass `@filepath`). Short inline strings also work: `tooling/tea-comment "body"`. The `@filepath` form avoids `$()` subshells which break permission matching. +- **For comments, use `reach pr comment @/tmp/file.md`** for long comments (write to file first, then pass `@filepath`). Short inline strings also work: `reach pr comment "body"`. The `@filepath` form avoids `$()` subshells which break permission matching. - **`tea pr reject` does not work on your own PRs** — use `tea comment` instead - **Run `tea` from the main checkout, never a linked worktree** — its go-git can't read a worktree's `.git` *file* and errors with "local/remote repository required". `tea pr close`/`comment` take explicit `--login`/`--repo` so cwd barely matters for them, but `tea pr create` resolves the repo from the **local checkout's remote** (see below) — that is why it must run from the main repo. - **Never delete protected branches:** `main`, `maintenance`, `server`, `client`, `copy`, `audio`, `visual`, `ci` are protected on Gitea. Do not use `tea pr clean`, `git push --delete`, or `git branch -D` on these branches. diff --git a/.claude/settings.json b/.claude/settings.json index 4c1b3aadb..7653998d1 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -35,8 +35,6 @@ "Bash(reach)", "Bash(reach *)", "Bash(tea *)", - "Bash(tooling/tea-comment *)", - "Bash(tooling/worktree-setup:*)", "Bash(cargo test *)", "Bash(cargo test)", "Bash(cargo build *)", @@ -50,7 +48,6 @@ "Bash(tests/run-*)", "Bash(godot4:*)", "Bash(gdformat:*)", - "Bash(tooling/pr-watchlist-diff:*)", "Bash(chmod *)", "Bash(ls *)", "Bash(find *)", diff --git a/.claude/skills/git-commit/SKILL.md b/.claude/skills/git-commit/SKILL.md index a76dfd74f..07fc7fefe 100644 --- a/.claude/skills/git-commit/SKILL.md +++ b/.claude/skills/git-commit/SKILL.md @@ -99,7 +99,7 @@ Use the project subsystem as scope. Examples: The pre-push clerk reviews each commit for D-record consistency. For a pure bulk content commit — e.g. shipping thousands of generated planetary description files — that review is moot and just burns agents on noise. Add a `Clerk-Skip:` **trailer -line** to the commit message and `tooling/clerk-review` auto-approves that commit +line** to the commit message and `reach dev clerk` auto-approves that commit without spawning an agent. Use it only for content/data dumps, never for commits that touch `governance/`, code, or ticket-bearing work. diff --git a/.claude/skills/pql-board/SKILL.md b/.claude/skills/pql-board/SKILL.md index 4f05cbab3..f78f869fa 100644 --- a/.claude/skills/pql-board/SKILL.md +++ b/.claude/skills/pql-board/SKILL.md @@ -26,7 +26,7 @@ so the page is a **snapshot** — this skill regenerates and redeploys it. run from the MAIN checkout, not a worktree): ```bash - tooling/pql-board-html /pql-board.html + reach pr board-html --output /pql-board.html ``` The script pulls `pql ticket list --full`, batch-fetches blocker edges for @@ -51,7 +51,7 @@ so the page is a **snapshot** — this skill regenerates and redeploys it. with `force` only if intentionally replacing. - If the canonical artifact was deleted and a new URL gets minted, **update the URL in this skill file** in the same turn and commit it. -- The generator is `tooling/pql-board-html` (python, pql-native JSON only). +- The generator is `reach pr board-html` (python, pql-native JSON only). Known scope limits vs clide: no decisions view, no kanban columns, no live editing. Dep edges cover non-terminal tickets only — if pql grows `ticket list --with-blockers`, simplify the script (candidate upstream FR diff --git a/.claude/skills/pr-process/SKILL.md b/.claude/skills/pr-process/SKILL.md index bbc0da74e..0ad8f1533 100644 --- a/.claude/skills/pr-process/SKILL.md +++ b/.claude/skills/pr-process/SKILL.md @@ -217,14 +217,14 @@ versus `origin/main`. This list covers generator code AND the data files that fe The stamped generator sources come from the shared registry `tooling/generator_sources.py` (T-1067) — the same module the stamp writer and -`tooling/check-systems-db-stamp` use, so the lists can no longer drift (PR #136 -review T7). `tooling/pr-watchlist-diff` wraps the comparison — its header comment -documents the non-stamped watch items it also checks (the surviving one-time +`reach check systems-db-stamp` use, so the lists can no longer drift (PR #136 +review T7). `reach pr watchlist-diff` wraps the comparison — it reads the watched +set from that registry and adds the non-stamped items (the surviving one-time planet-gen importers, the schema DDL whose SHA is stamped separately, and the wiki data directories that feed the generators): ```bash -tooling/pr-watchlist-diff origin/main HEAD +reach pr watchlist-diff origin/main HEAD ``` **If output is empty:** skip this step entirely. @@ -358,7 +358,7 @@ If comments exist and contain a review verdict (look for "CHANGES REQUESTED" or 6. Post a response comment on the PR summarizing: - Which issues were fixed (with commit refs) - Which issues were pushed back on (with rationale) - - Use `tooling/tea-comment @/tmp/pr-response.md` + - Use `reach pr comment @/tmp/pr-response.md` If no review comments exist, or the review is APPROVED, skip this step. diff --git a/.claude/skills/pr-review/SKILL.md b/.claude/skills/pr-review/SKILL.md index b14663e79..4cf58afc9 100644 --- a/.claude/skills/pr-review/SKILL.md +++ b/.claude/skills/pr-review/SKILL.md @@ -148,7 +148,7 @@ assumed — or via `git show`.** Topic branches from `/whats-next` may have a worktree at `/.worktrees/` (the kanban convention, D-221 — see -`tooling/worktree-setup`): +`reach dev worktree`): ```bash REPO_ROOT="$(git rev-parse --show-toplevel)" @@ -264,13 +264,13 @@ After presenting results to the user, post the review as a PR comment. Note: `tea pr reject` does not work on your own PRs. Use `tea comment` instead. -Post using the `tea-comment` wrapper (handles temp files and cleanup). +Post using `reach pr comment` (handles temp files and cleanup). **Two rules:** 1. **Use the Write tool** for the file content (no permission prompt, no heredoc parsing issues with markdown tables/pipes). Then call - `tooling/tea-comment` in a separate short Bash call. -2. **Run `tooling/tea-comment` in the FOREGROUND, never with + `reach pr comment` in a separate short Bash call. +2. **Run `reach pr comment` in the FOREGROUND, never with `run_in_background`.** The background execution path silently fails — the comment never reaches Gitea and the team never sees the review (Sprint 38 lost an entire review round this way). @@ -280,7 +280,7 @@ Post using the `tea-comment` wrapper (handles temp files and cleanup). Write({ file_path: "/tmp/pr-review-.md", content: "..." }) # Step 2: Post via short Bash call (foreground) -tooling/tea-comment @/tmp/pr-review-.md +reach pr comment @/tmp/pr-review-.md ``` Do NOT use `cat << 'EOF'` heredocs for review content — they create diff --git a/.claude/skills/whats-next/SKILL.md b/.claude/skills/whats-next/SKILL.md index 12a4dee07..45e73f68f 100644 --- a/.claude/skills/whats-next/SKILL.md +++ b/.claude/skills/whats-next/SKILL.md @@ -222,7 +222,7 @@ file access), symlinks the `.venv` (so `make test-tooling`/`regen-db` and python tooling resolve `.venv/bin/python`), and prints the in-worktree reminders: ```bash -tooling/worktree-setup # run from the main checkout +reach dev worktree # run from the main checkout ``` Report the worktree path so the user can open a terminal there. diff --git a/.config/hooks/pre-push b/.config/hooks/pre-push index 4c6cbe2ae..64c34a2b5 100755 --- a/.config/hooks/pre-push +++ b/.config/hooks/pre-push @@ -341,9 +341,12 @@ fi # diff check + a SHA-keyed verdict cache invalidated on decisions/ change, with # skip tracked in the cache rather than a commit-message trailer — see #965). # Force-run with SR_RUN_CLERK=1. -if [ -x "$REPO_ROOT/tooling/clerk-review" ] && [ "${SR_RUN_CLERK:-0}" = "1" ]; then +if command -v reach >/dev/null 2>&1 && [ "${SR_RUN_CLERK:-0}" = "1" ]; then echo "pre-push: running clerk review..." - CLERK_VERDICT=$("$REPO_ROOT/tooling/clerk-review" 2>&1 | tee /dev/stderr | tail -1) + # stdout carries the verdict, stderr carries progress. They must NOT be + # merged: under the streaming model (D-263) the last merged line would be a + # JSONL progress event, and an unrecognised verdict blocks the push. + CLERK_VERDICT=$(reach dev clerk | tail -1) || true if [ "$CLERK_VERDICT" = "APPROVED" ]; then echo "pre-push: clerk — APPROVED" elif [ "$CLERK_VERDICT" = "REJECTED" ]; then diff --git a/.pql/changelog/ticket_history/2026-09.sql b/.pql/changelog/ticket_history/2026-09.sql index c28e3ea43..1345874b7 100644 --- a/.pql/changelog/ticket_history/2026-09.sql +++ b/.pql/changelog/ticket_history/2026-09.sql @@ -66,3 +66,57 @@ ALSO CAUGHT: update_field now checks rowcount and fails when no row matched. The THE THREE PYTHON SCRIPTS moved with the usual treatment: prints to console events, argparse mains replaced by typed functions the router calls, __file__-relative roots to config.repo_root(). No root bug this time — checked before moving rather than after, which is the third domain running.', NULL, '2026-09-02 12:36:36', '2026-09-02 12:36:36.194', '2026-09-02 12:36:36.194', NULL, 'd804c766e461d4aaf6cdf4d8d1157125', 2) ON CONFLICT(hash) DO NOTHING; INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G642RKW8PNW7AP9BY9203SPG', 'status', 'in_progress', 'done', NULL, '2026-09-02 12:36:36', '2026-09-02 12:36:36.208', '2026-09-02 12:36:36.208', NULL, '8aa91d6e9cacaaa410b20ef6cf9703b9', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'description', 'Twelve small files, three domains, no ladder entanglement — deliberately batched to clear the tail before the two large ports (atlas planet, ledger). GENERATE: generate-brands (bash 23, wraps the Rust generate_brands binary), generate-corporations (bash 23), generate-character-manifest (py 111), generate_corp_stubs.py (py 413). PR: tea-comment (bash 41), pr-watchlist-diff (bash 37), pql-board-html (py). DEV, joining the existing selftest verb: install-rust (bash 33), install-godot (bash 108), worktree-setup (bash 49), perf-baseline (py), clerk-review (py). Standard port acceptance as on T-1281. SEVEN OF THE TWELVE ARE BASH so those are rewrites. THE THREE ENVIRONMENT SCRIPTS need the D-263 guarded-exec treatment specifically: install-godot downloads and unzips a pinned Godot build, install-rust drives rustup, worktree-setup manipulates git worktrees — port the DECISION logic (which version is wanted, is it already present, what does the tree look like) into a testable service and keep the irreducible external calls behind core/process.run. The test of a correct port is whether the decisions can be exercised WITHOUT performing them; a rewrite that cannot be tested has to be trusted instead, and trusting an installer is how a working environment becomes an unreproducible one. WATCH tea-comment: .claude/rules/tea-cli.md says tea prompts crash in Claude Code with no TTY and that tooling/tea-comment exists specifically to avoid subshells breaking permission matching — preserve both properties, and pass --no-input through.', 'Twelve small files, three domains, no ladder entanglement — deliberately batched to clear the tail before the two large ports (atlas planet, ledger). GENERATE: generate-brands (bash 23, wraps the Rust generate_brands binary), generate-corporations (bash 23), generate-character-manifest (py 111), generate_corp_stubs.py (py 413). PR: tea-comment (bash 41), pr-watchlist-diff (bash 37), pql-board-html (py). DEV, joining the existing selftest verb: install-rust (bash 33), install-godot (bash 108), worktree-setup (bash 49), perf-baseline (py), clerk-review (py). Standard port acceptance as on T-1281. SEVEN OF THE TWELVE ARE BASH so those are rewrites. THE THREE ENVIRONMENT SCRIPTS need the D-263 guarded-exec treatment specifically: install-godot downloads and unzips a pinned Godot build, install-rust drives rustup, worktree-setup manipulates git worktrees — port the DECISION logic (which version is wanted, is it already present, what does the tree look like) into a testable service and keep the irreducible external calls behind core/process.run. The test of a correct port is whether the decisions can be exercised WITHOUT performing them; a rewrite that cannot be tested has to be trusted instead, and trusting an installer is how a working environment becomes an unreproducible one. WATCH tea-comment: .claude/rules/tea-cli.md says tea prompts crash in Claude Code with no TTY and that tooling/tea-comment exists specifically to avoid subshells breaking permission matching — preserve both properties, and pass --no-input through. + +Ported. `generate`, `pr` and `dev` are registered; 12 scripts retired. + +**generate** — `generate-brands` and `generate-corporations` were the *third* +and second copies of the same 24-line build-if-missing-then-exec bash that +`tooling/atlas` carried, so both collapsed into `core.process.cargo_binary` +rather than being ported. `generate-character-manifest` and +`generate_corp_stubs.py` moved as modules; both had `__file__`-relative roots +that break on move (the fourth and fifth instances of that trap this epic) and +now use `config.repo_root()`. + +**pr** — `watchlist-diff` now derives its watched set from +`generator_sources.GENERATOR_SOURCES` instead of restating it, so it cannot +drift from the stamp check (185 registered paths + 5 non-stamped extras). +Verified with a positive control: silent on a commit touching only a test file, +reports `economy_import/atlas.py` + `import_economics.py` on 8da9670e0. + +**dev** — the risky third. Split per the ticket''s own test: `godot_plan()` and +`worktree_plan()` are pure and decide *what would happen*; `install_godot()`, +`install_rust()` and `setup_worktree()` perform it. `tooling/test_environment.py` +pins the version pin, the `--version` and `GODOT_VERSION` precedence, the +already-current skip decision, the unsupported-platform refusal, and both +worktree refusals — none of them performed. Every assertion was mutation-checked. + +## Found while porting + +1. **`make regen-db` was about to break.** `import_economics` shells out to + `tooling/generate-brands`, which this ticket deleted. Rerouted through + `cargo_binary`; `generated_brands.toml` came back byte-identical, and the + stamp registry swapped the retired wrapper for `core/process.py`. Regen run, + stamp fresh. +2. **The clerk has been grepping a directory that does not exist.** `load_index()` + read `decisions/README.md`; the DQR tree moved to `governance/` long ago, so + every clerk agent got "(not found)" as its domain index. The prompt told it to + grep `decisions/*.md` too. Both fixed. (Clerk is disabled by default per #965, + so this cost nothing yet — but it would have been invisible on re-enable.) +3. **The conformance exec-check fired on the wrong receiver.** It matched any + `x.system()`, so `platform.system()` read as `os.system()`. Narrowed to an + `os` receiver and re-proved against a real `os.system` mutant. +4. **The committed character manifest was stale** — it predates the 24-garment + wardrobe. Confirmed the port is not the cause: the original script, run from + its own directory, produces the identical new output. Regenerated. +5. **`make setup` had a bootstrap-order problem** once `setup-rust`/`setup-godot` + became reach verbs — they ran before `install-reach`. `install-reach` needs + only `uv`, so it now leads the chain. +6. `process.run` gained `input=` and `timeout=` (the clerk needs both) plus a + `ProcessTimeout` subclass, so a killed run is distinguishable from a verdict. + The pre-push hook''s clerk capture no longer merges stderr into stdout — under + streaming the last merged line is a JSONL event, which would have read as an + unrecognised verdict and blocked the push. + +Gate: `ruff check tooling/` clean, `make test-tooling` PASS (12 checks).', NULL, '2026-09-02 14:54:35', '2026-09-02 14:54:35.578', '2026-09-02 14:54:35.578', NULL, 'e0bd8f1d16b2dc7c7d1b8b1511633cc3', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'status', 'backlog', 'done', NULL, '2026-09-02 14:54:37', '2026-09-02 14:54:37.762', '2026-09-02 14:54:37.762', NULL, '5f9240c61aa2bf03924e4edaf4dd6f7c', 2) ON CONFLICT(hash) DO NOTHING; diff --git a/.pql/changelog/ticket_idmap/2026-09.sql b/.pql/changelog/ticket_idmap/2026-09.sql index e161d8b32..b34be2f2c 100644 --- a/.pql/changelog/ticket_idmap/2026-09.sql +++ b/.pql/changelog/ticket_idmap/2026-09.sql @@ -1,2 +1,3 @@ INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G63FEA51J4JKE3A7473D3QZ0', 'T-1284', '2026-09-02 10:50:23.145', '2026-09-02 10:50:23.145', NULL, '74113524e3a280bd06875305a547349a', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G642RKW8PNW7AP9BY9203SPG', 'T-1285', '2026-09-02 12:14:48.290', '2026-09-02 12:14:48.290', NULL, '020f491d1ae8fbde83d2aae0f8325011', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; +INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'T-1286', '2026-09-02 12:49:55.434', '2026-09-02 12:49:55.434', NULL, '7c65442cc261176c9c472fdbd635cc91', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at; diff --git a/.pql/changelog/tickets/2026-09.sql b/.pql/changelog/tickets/2026-09.sql index d08819f91..d67af1523 100644 --- a/.pql/changelog/tickets/2026-09.sql +++ b/.pql/changelog/tickets/2026-09.sql @@ -122,3 +122,110 @@ UPDATE-FIELD ported with the asset-pipeline distinction in its help, per the pre ALSO CAUGHT: update_field now checks rowcount and fails when no row matched. The original''s SQL would succeed silently against a system_id that does not exist — a no-op reported as success, which is the exact shape of failure this initiative keeps finding. THE THREE PYTHON SCRIPTS moved with the usual treatment: prints to console events, argparse mains replaced by typed functions the router calls, __file__-relative roots to config.repo_root(). No root bug this time — checked before moving rather than after, which is the third domain running.', 'done', 'high', NULL, NULL, 'D-263', '2026-09-02 12:14:48.290', '2026-09-02 12:36:36.208', NULL, '6eff7972f7304c334f347ae8598bdf6b', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port generate, pr, and the rest of dev', 'Twelve small files, three domains, no ladder entanglement — deliberately batched to clear the tail before the two large ports (atlas planet, ledger). GENERATE: generate-brands (bash 23, wraps the Rust generate_brands binary), generate-corporations (bash 23), generate-character-manifest (py 111), generate_corp_stubs.py (py 413). PR: tea-comment (bash 41), pr-watchlist-diff (bash 37), pql-board-html (py). DEV, joining the existing selftest verb: install-rust (bash 33), install-godot (bash 108), worktree-setup (bash 49), perf-baseline (py), clerk-review (py). Standard port acceptance as on T-1281. SEVEN OF THE TWELVE ARE BASH so those are rewrites. THE THREE ENVIRONMENT SCRIPTS need the D-263 guarded-exec treatment specifically: install-godot downloads and unzips a pinned Godot build, install-rust drives rustup, worktree-setup manipulates git worktrees — port the DECISION logic (which version is wanted, is it already present, what does the tree look like) into a testable service and keep the irreducible external calls behind core/process.run. The test of a correct port is whether the decisions can be exercised WITHOUT performing them; a rewrite that cannot be tested has to be trusted instead, and trusting an installer is how a working environment becomes an unreproducible one. WATCH tea-comment: .claude/rules/tea-cli.md says tea prompts crash in Claude Code with no TTY and that tooling/tea-comment exists specifically to avoid subshells breaking permission matching — preserve both properties, and pass --no-input through.', 'backlog', 'medium', NULL, NULL, 'D-263', '2026-09-02 12:49:55.434', '2026-09-02 12:49:55.434', NULL, '5676581f3cae08432c174d73b39a8cd5', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port generate, pr, and the rest of dev', 'Twelve small files, three domains, no ladder entanglement — deliberately batched to clear the tail before the two large ports (atlas planet, ledger). GENERATE: generate-brands (bash 23, wraps the Rust generate_brands binary), generate-corporations (bash 23), generate-character-manifest (py 111), generate_corp_stubs.py (py 413). PR: tea-comment (bash 41), pr-watchlist-diff (bash 37), pql-board-html (py). DEV, joining the existing selftest verb: install-rust (bash 33), install-godot (bash 108), worktree-setup (bash 49), perf-baseline (py), clerk-review (py). Standard port acceptance as on T-1281. SEVEN OF THE TWELVE ARE BASH so those are rewrites. THE THREE ENVIRONMENT SCRIPTS need the D-263 guarded-exec treatment specifically: install-godot downloads and unzips a pinned Godot build, install-rust drives rustup, worktree-setup manipulates git worktrees — port the DECISION logic (which version is wanted, is it already present, what does the tree look like) into a testable service and keep the irreducible external calls behind core/process.run. The test of a correct port is whether the decisions can be exercised WITHOUT performing them; a rewrite that cannot be tested has to be trusted instead, and trusting an installer is how a working environment becomes an unreproducible one. WATCH tea-comment: .claude/rules/tea-cli.md says tea prompts crash in Claude Code with no TTY and that tooling/tea-comment exists specifically to avoid subshells breaking permission matching — preserve both properties, and pass --no-input through. + +Ported. `generate`, `pr` and `dev` are registered; 12 scripts retired. + +**generate** — `generate-brands` and `generate-corporations` were the *third* +and second copies of the same 24-line build-if-missing-then-exec bash that +`tooling/atlas` carried, so both collapsed into `core.process.cargo_binary` +rather than being ported. `generate-character-manifest` and +`generate_corp_stubs.py` moved as modules; both had `__file__`-relative roots +that break on move (the fourth and fifth instances of that trap this epic) and +now use `config.repo_root()`. + +**pr** — `watchlist-diff` now derives its watched set from +`generator_sources.GENERATOR_SOURCES` instead of restating it, so it cannot +drift from the stamp check (185 registered paths + 5 non-stamped extras). +Verified with a positive control: silent on a commit touching only a test file, +reports `economy_import/atlas.py` + `import_economics.py` on 8da9670e0. + +**dev** — the risky third. Split per the ticket''s own test: `godot_plan()` and +`worktree_plan()` are pure and decide *what would happen*; `install_godot()`, +`install_rust()` and `setup_worktree()` perform it. `tooling/test_environment.py` +pins the version pin, the `--version` and `GODOT_VERSION` precedence, the +already-current skip decision, the unsupported-platform refusal, and both +worktree refusals — none of them performed. Every assertion was mutation-checked. + +## Found while porting + +1. **`make regen-db` was about to break.** `import_economics` shells out to + `tooling/generate-brands`, which this ticket deleted. Rerouted through + `cargo_binary`; `generated_brands.toml` came back byte-identical, and the + stamp registry swapped the retired wrapper for `core/process.py`. Regen run, + stamp fresh. +2. **The clerk has been grepping a directory that does not exist.** `load_index()` + read `decisions/README.md`; the DQR tree moved to `governance/` long ago, so + every clerk agent got "(not found)" as its domain index. The prompt told it to + grep `decisions/*.md` too. Both fixed. (Clerk is disabled by default per #965, + so this cost nothing yet — but it would have been invisible on re-enable.) +3. **The conformance exec-check fired on the wrong receiver.** It matched any + `x.system()`, so `platform.system()` read as `os.system()`. Narrowed to an + `os` receiver and re-proved against a real `os.system` mutant. +4. **The committed character manifest was stale** — it predates the 24-garment + wardrobe. Confirmed the port is not the cause: the original script, run from + its own directory, produces the identical new output. Regenerated. +5. **`make setup` had a bootstrap-order problem** once `setup-rust`/`setup-godot` + became reach verbs — they ran before `install-reach`. `install-reach` needs + only `uv`, so it now leads the chain. +6. `process.run` gained `input=` and `timeout=` (the clerk needs both) plus a + `ProcessTimeout` subclass, so a killed run is distinguishable from a verdict. + The pre-push hook''s clerk capture no longer merges stderr into stdout — under + streaming the last merged line is a JSONL event, which would have read as an + unrecognised verdict and blocked the push. + +Gate: `ruff check tooling/` clean, `make test-tooling` PASS (12 checks).', 'backlog', 'medium', NULL, NULL, 'D-263', '2026-09-02 12:49:55.434', '2026-09-02 14:54:35.577', NULL, 'c010e2f8bdc2cfe4843926351d87c6bb', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G64ASTXBBHMWMBQSVPM9MK1W', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port generate, pr, and the rest of dev', 'Twelve small files, three domains, no ladder entanglement — deliberately batched to clear the tail before the two large ports (atlas planet, ledger). GENERATE: generate-brands (bash 23, wraps the Rust generate_brands binary), generate-corporations (bash 23), generate-character-manifest (py 111), generate_corp_stubs.py (py 413). PR: tea-comment (bash 41), pr-watchlist-diff (bash 37), pql-board-html (py). DEV, joining the existing selftest verb: install-rust (bash 33), install-godot (bash 108), worktree-setup (bash 49), perf-baseline (py), clerk-review (py). Standard port acceptance as on T-1281. SEVEN OF THE TWELVE ARE BASH so those are rewrites. THE THREE ENVIRONMENT SCRIPTS need the D-263 guarded-exec treatment specifically: install-godot downloads and unzips a pinned Godot build, install-rust drives rustup, worktree-setup manipulates git worktrees — port the DECISION logic (which version is wanted, is it already present, what does the tree look like) into a testable service and keep the irreducible external calls behind core/process.run. The test of a correct port is whether the decisions can be exercised WITHOUT performing them; a rewrite that cannot be tested has to be trusted instead, and trusting an installer is how a working environment becomes an unreproducible one. WATCH tea-comment: .claude/rules/tea-cli.md says tea prompts crash in Claude Code with no TTY and that tooling/tea-comment exists specifically to avoid subshells breaking permission matching — preserve both properties, and pass --no-input through. + +Ported. `generate`, `pr` and `dev` are registered; 12 scripts retired. + +**generate** — `generate-brands` and `generate-corporations` were the *third* +and second copies of the same 24-line build-if-missing-then-exec bash that +`tooling/atlas` carried, so both collapsed into `core.process.cargo_binary` +rather than being ported. `generate-character-manifest` and +`generate_corp_stubs.py` moved as modules; both had `__file__`-relative roots +that break on move (the fourth and fifth instances of that trap this epic) and +now use `config.repo_root()`. + +**pr** — `watchlist-diff` now derives its watched set from +`generator_sources.GENERATOR_SOURCES` instead of restating it, so it cannot +drift from the stamp check (185 registered paths + 5 non-stamped extras). +Verified with a positive control: silent on a commit touching only a test file, +reports `economy_import/atlas.py` + `import_economics.py` on 8da9670e0. + +**dev** — the risky third. Split per the ticket''s own test: `godot_plan()` and +`worktree_plan()` are pure and decide *what would happen*; `install_godot()`, +`install_rust()` and `setup_worktree()` perform it. `tooling/test_environment.py` +pins the version pin, the `--version` and `GODOT_VERSION` precedence, the +already-current skip decision, the unsupported-platform refusal, and both +worktree refusals — none of them performed. Every assertion was mutation-checked. + +## Found while porting + +1. **`make regen-db` was about to break.** `import_economics` shells out to + `tooling/generate-brands`, which this ticket deleted. Rerouted through + `cargo_binary`; `generated_brands.toml` came back byte-identical, and the + stamp registry swapped the retired wrapper for `core/process.py`. Regen run, + stamp fresh. +2. **The clerk has been grepping a directory that does not exist.** `load_index()` + read `decisions/README.md`; the DQR tree moved to `governance/` long ago, so + every clerk agent got "(not found)" as its domain index. The prompt told it to + grep `decisions/*.md` too. Both fixed. (Clerk is disabled by default per #965, + so this cost nothing yet — but it would have been invisible on re-enable.) +3. **The conformance exec-check fired on the wrong receiver.** It matched any + `x.system()`, so `platform.system()` read as `os.system()`. Narrowed to an + `os` receiver and re-proved against a real `os.system` mutant. +4. **The committed character manifest was stale** — it predates the 24-garment + wardrobe. Confirmed the port is not the cause: the original script, run from + its own directory, produces the identical new output. Regenerated. +5. **`make setup` had a bootstrap-order problem** once `setup-rust`/`setup-godot` + became reach verbs — they ran before `install-reach`. `install-reach` needs + only `uv`, so it now leads the chain. +6. `process.run` gained `input=` and `timeout=` (the clerk needs both) plus a + `ProcessTimeout` subclass, so a killed run is distinguishable from a verdict. + The pre-push hook''s clerk capture no longer merges stderr into stdout — under + streaming the last merged line is a JSONL event, which would have read as an + unrecognised verdict and blocked the push. + +Gate: `ruff check tooling/` clean, `make test-tooling` PASS (12 checks).', 'done', 'medium', NULL, NULL, 'D-263', '2026-09-02 12:49:55.434', '2026-09-02 14:54:37.762', NULL, '678c0d0578c2255438d2e51829d87a64', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; diff --git a/Makefile b/Makefile index e36eceda8..8f47ef977 100644 --- a/Makefile +++ b/Makefile @@ -89,14 +89,17 @@ help: # --- Setup --- -setup: setup-rust setup-godot setup-tooling setup-venv install-reach setup-hooks decisions-sync +# install-reach leads: it needs only uv, and setup-rust/setup-godot are now +# reach verbs. Bootstrapping the CLI before the things it installs is the only +# order that works on a cold checkout. +setup: install-reach setup-rust setup-godot setup-tooling setup-venv setup-hooks decisions-sync @echo "Dev environment ready." setup-rust: - @tooling/install-rust + @reach dev install-rust setup-godot: - @GODOT_VERSION=$(GODOT_VERSION) tooling/install-godot + @reach dev install-godot --version $(GODOT_VERSION) setup-tooling: @echo "Checking tooling dependencies..." @@ -307,6 +310,10 @@ test-tooling: @mkdir -p .cache @$(VENV_PY) tooling/test_jobs.py 2> .cache/test-tooling-jobs.log || \ { echo " FAIL: jobs service — log follows:"; cat .cache/test-tooling-jobs.log; exit 1; } + @echo " [test-tooling] reach dev environment decisions (T-1286)..." + @mkdir -p .cache + @$(VENV_PY) tooling/test_environment.py 2> .cache/test-tooling-environment.log || \ + { echo " FAIL: environment decisions — log follows:"; cat .cache/test-tooling-environment.log; exit 1; } @echo " [test-tooling] reach D-263 conformance (T-1270)..." @mkdir -p .cache @$(VENV_PY) tooling/test_conformance.py 2> .cache/test-tooling-conformance.log || \ @@ -438,7 +445,7 @@ ci-client: lint-client build-client test-client economy-db: ## Import economics data (commodities, chains, gate links) into systems.db @echo " Generating minor brands (D-189 #829)..." - @tooling/generate-brands + @reach generate brands @python3 tooling/economy-db/import_economics.py regen-db: ## Regenerate systems.db from all sources and stamp meta table (#855, #856) @@ -523,7 +530,7 @@ star-map-data: @python3 tooling/generate-star-map-data.py perf-baseline: - @tooling/perf-baseline + @reach dev perf # --- Schedule debug (#346) --- @@ -579,7 +586,7 @@ test-voice-real: # --- Asset manifest --- manifest: - @tooling/generate-character-manifest + @reach generate character-manifest @echo "Manifest regenerated — commit client/assets/characters/manifest.json if changed." # --- Diagrams --- diff --git a/docs/DEVOPS.md b/docs/DEVOPS.md index 20b1c5f16..ed8ec8876 100644 --- a/docs/DEVOPS.md +++ b/docs/DEVOPS.md @@ -269,8 +269,8 @@ the next regeneration. ### Generator `import_economics` (`tooling/economy-db/import_economics.py`) is the sole -generator. As its first step it shells out to the Rust `generate_brands` binary -(via `tooling/generate-brands`) to refresh `generated_brands.toml`, then imports +generator. As its first step it runs the Rust `generate_brands` binary +(via `core.process.cargo_binary`) to refresh `generated_brands.toml`, then imports economics data and the atlas index (names-only city pool; geometry tables stay empty for the Phase 4 server cascade). The former atlas geometry generator (`generate_atlas`) was retired in #951 (D-223). diff --git a/tooling/DOMAINS.md b/tooling/DOMAINS.md index 420dd847e..9468273b0 100644 --- a/tooling/DOMAINS.md +++ b/tooling/DOMAINS.md @@ -95,9 +95,9 @@ invented. | `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** | | `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` | | `godot` | Godot parse and cold-start checks | `godot-parse-sweep`, `godot-cold-parse` | -| `generate` | content generators not owned elsewhere | `generate-brands`, `generate-corporations`, `generate-character-manifest`, `generate_corp_stubs.py` | -| `dev` | developer environment and workflow | `install-rust`, `install-godot`, `worktree-setup`, `perf-baseline`, `clerk-review` | -| `pr` | the PR/review loop | `tea-comment`, `pr-watchlist-diff`, `pql-board-html` | +| `generate` | content generators not owned elsewhere | ✅ ported (T-1286). `generate-brands` + `generate-corporations` collapsed into `core.process.cargo_binary` — they were the same 24 lines of bash a third time | +| `dev` | developer environment and workflow | ✅ ported (T-1286). The three environment scripts split decision from performing — `godot_plan`/`worktree_plan` are pure and pinned by `test_environment.py` | +| `pr` | the PR/review loop | ✅ ported (T-1286). `watchlist-diff` now reads the watched set from `generator_sources.py` instead of restating it | | `blender` | **carve-out** — payloads run by Blender | **35** `blender_*.py` (13 top-level + 22 in `garment-fit/`), `blender` wrapper → `tooling/scripts/blender/` | ## Judgment calls, with reasons diff --git a/tooling/core/process.py b/tooling/core/process.py index eb4cf3880..5304fa299 100644 --- a/tooling/core/process.py +++ b/tooling/core/process.py @@ -63,6 +63,16 @@ def meta_path(job_id: str) -> Path: return jobs_dir() / f"{job_id}.json" +class ProcessTimeout(ReachError): + """A run that exceeded its `timeout=`. + + Its own class because a timeout is not a failure verdict — a caller that + treats "the tool ran and said no" and "the tool never answered" alike will + turn a slow machine into a rejection. Uncaught it still reads as a + `ReachError` with a remedy. + """ + + def run( argv: list[str], *, @@ -70,6 +80,8 @@ def run( env: dict[str, str] | None = None, capture: bool = True, check: bool = True, + input: str | None = None, + timeout: float | None = None, fix: str | None = None, missing_fix: str | None = None, ) -> subprocess.CompletedProcess[str]: @@ -108,12 +120,20 @@ def run( capture_output=capture, text=True, shell=False, + input=input, + timeout=timeout, ) except FileNotFoundError as exc: raise ReachError( f"{argv[0]} is not installed or not on PATH", fix=missing_fix or f"install {argv[0]}, or check PATH in a non-interactive shell", ) from exc + except subprocess.TimeoutExpired as exc: + raise ProcessTimeout( + f"{argv[0]} did not finish within {timeout:g}s", + fix=f"re-run, or raise the timeout — `{' '.join(argv)}` was killed, " + "which says nothing about whether it would have succeeded", + ) from exc if check and result.returncode != 0: detail = (result.stderr or result.stdout or "").strip() @@ -126,6 +146,47 @@ def run( return result +def cargo_binary(name: str, *args: str, capture: bool = True) -> str: + """Run a Rust binary from `server/`, building it first if it is absent. + + Three scripts wrote this by hand — `tooling/atlas`, `generate-brands`, + `generate-corporations` — each ~24 lines of identical bash: check for the + debug binary, `cargo build --bin` it if missing, `exec` it with every + argument. Three copies of one idea is a substrate, so it lives here. + + Build-if-missing is kept because it is genuinely useful on a cold checkout, + but it is announced rather than silent: a first run that takes ninety + seconds with no explanation reads as a hang. + """ + binary = config.path("server", "target", "debug", name) + if not binary.is_file(): + from tooling.core import console + + console.event(f"building the {name} binary (first run)", level="warn") + build = run( + ["cargo", "build", "--bin", name], + cwd=config.path("server"), + check=False, + missing_fix="install Rust — make setup-rust", + ) + if build.returncode != 0: + raise ReachError( + f"could not build {name}\n{(build.stderr or '').strip()}", + fix=f"cd server && cargo build --bin {name} — for the full error", + exit_code=build.returncode, + ) + + result = run([str(binary), *args], check=False, capture=capture) + if result.returncode != 0: + raise ReachError( + f"{name} {' '.join(args)} exited {result.returncode}\n" + + (result.stderr or "").strip(), + fix=f"run `{binary} {' '.join(args)}` directly for the full output", + exit_code=result.returncode, + ) + return result.stdout or "" + + def spawn_detached(argv: list[str]) -> str: """Run `reach ` in a detached child. Returns the job id immediately. diff --git a/tooling/domains/atlas/binary.py b/tooling/domains/atlas/binary.py index e236b54da..b92b8358c 100644 --- a/tooling/domains/atlas/binary.py +++ b/tooling/domains/atlas/binary.py @@ -11,15 +11,18 @@ what exists, and an index that says "ask the binary" is not one. The cost is that a subcommand added on the Rust side is invisible here until someone adds a line — so `run()` also accepts anything, and an unknown verb reaches the binary rather than being rejected by a list that has fallen behind. + +The build-if-missing and invoke machinery moved to `core.process.cargo_binary` +once `generate-brands` and `generate-corporations` turned out to be the same +24 lines of bash. Three copies of one idea is a substrate. """ from __future__ import annotations -from tooling.core import config, console, process -from tooling.core.errors import ReachError +from tooling.core import process # What the binary offers today. Discovered from the wrapper's own usage text and -# from atlas-commit-and-sync, which calls three verbs the wrapper never +# from atlas-commit-and-sync, which calls four verbs the wrapper never # documented — list-stations, wipe-system, commit-system, sync-wiki. KNOWN_VERBS = ( "stats", @@ -33,34 +36,6 @@ KNOWN_VERBS = ( ) -def binary_path(): - return config.path("server", "target", "debug", "atlas") - - def run(*args: str, capture: bool = True) -> str: """Invoke the Rust atlas binary, building it first if it is absent.""" - binary = binary_path() - if not binary.is_file(): - console.event("building the atlas binary (first run)", level="warn") - build = process.run( - ["cargo", "build", "--bin", "atlas"], - cwd=config.path("server"), - check=False, - missing_fix="install Rust — make setup-rust", - ) - if build.returncode != 0: - raise ReachError( - f"could not build the atlas binary\n{(build.stderr or '').strip()}", - fix="cd server && cargo build --bin atlas — to see the full error", - exit_code=build.returncode, - ) - - result = process.run([str(binary), *args], check=False, capture=capture) - if result.returncode != 0: - raise ReachError( - f"atlas {' '.join(args)} exited {result.returncode}\n" - + (result.stderr or "").strip(), - fix=f"run `{binary} {' '.join(args)}` directly for the full output", - exit_code=result.returncode, - ) - return result.stdout or "" + return process.cargo_binary("atlas", *args, capture=capture) diff --git a/tooling/clerk-review b/tooling/domains/dev/clerk.py similarity index 70% rename from tooling/clerk-review rename to tooling/domains/dev/clerk.py index de136eed2..88b9213c6 100755 --- a/tooling/clerk-review +++ b/tooling/domains/dev/clerk.py @@ -40,20 +40,20 @@ Env knobs: SR_CLERK_TIMEOUT per-commit timeout seconds (default 300) Usage: - tooling/clerk-review run the review, print verdict - tooling/clerk-review --plan print the per-commit plan only (no clerk spawned) + reach dev clerk run the review, print verdict + reach dev clerk --plan print the per-commit plan only (no clerk spawned) """ +from __future__ import annotations + import os import re -import subprocess -import sys from concurrent.futures import ThreadPoolExecutor, as_completed -from pathlib import Path -REPO_ROOT = Path(subprocess.check_output( - ["git", "rev-parse", "--show-toplevel"], text=True -).strip()) +from tooling.core import config, console, process +from tooling.core.errors import ReachError + +REPO_ROOT = config.repo_root() CACHE_DIR = REPO_ROOT / ".cache" FINDINGS_FILE = CACHE_DIR / "pre-push-review.md" @@ -73,7 +73,7 @@ CLERK_PROMPT = """You are the CLERK, the institutional guardrail for The Settled You are reviewing ONE COMMIT ({label}) from a larger pre-push for D-record consistency, ticket drift, and decision contradictions. -## decisions/ domain index +## governance/ domain index {index} @@ -83,7 +83,7 @@ consistency, ticket drift, and decision contradictions. ## Your task -Be efficient — a few targeted greps of decisions/*.md, then conclude. Check: +Be efficient — a few targeted greps of governance/decisions/*.md, then conclude. Check: - Does this commit contradict any active D-record? - If the commit's code/text cites a D/Q/R-ID, does that ID exist and is it active? - If the commit message cites ticket #NNN, does the change match the ticket? @@ -101,29 +101,25 @@ APPROVED or REJECTED. """ -def base_range(): +def _git(*args: str) -> str: + return process.run(["git", *args], cwd=REPO_ROOT).stdout + + +def base_range() -> str: """The commit range that would be pushed (base..HEAD).""" - branch = subprocess.check_output( - ["git", "branch", "--show-current"], text=True - ).strip() + branch = _git("branch", "--show-current").strip() for ref in [f"origin/{branch}", "origin/main"]: - try: - subprocess.check_output( - ["git", "rev-parse", "--verify", ref], - stderr=subprocess.DEVNULL, text=True, - ) + probe = process.run( + ["git", "rev-parse", "--verify", ref], cwd=REPO_ROOT, check=False + ) + if probe.returncode == 0: return f"{ref}..HEAD" - except subprocess.CalledProcessError: - continue return "HEAD~1..HEAD" -def list_commits(rng): +def list_commits(rng: str) -> list[str]: """SHAs in the push range, oldest first.""" - out = subprocess.check_output( - ["git", "rev-list", "--reverse", rng], text=True - ) - return [s for s in out.splitlines() if s.strip()] + return [s for s in _git("rev-list", "--reverse", rng).splitlines() if s.strip()] def commit_unit(sha): @@ -132,16 +128,10 @@ def commit_unit(sha): `skip` is True when the commit message has a `Clerk-Skip:` trailer (the safety valve); `text` is the message + diff, truncated to COMMIT_BUDGET. """ - subject = subprocess.check_output( - ["git", "show", "-s", "--format=%h %s", sha], text=True - ).strip() - message = subprocess.check_output( - ["git", "show", "-s", "--format=%B", sha], text=True - ) + subject = _git("show", "-s", "--format=%h %s", sha).strip() + message = _git("show", "-s", "--format=%B", sha) skip = bool(SKIP_TRAILER.search(message)) - text = subprocess.check_output( - ["git", "show", "--format=fuller", sha], text=True - ) + text = _git("show", "--format=fuller", sha) if len(text) > COMMIT_BUDGET: text = text[:COMMIT_BUDGET] + f"\n\n... (commit diff truncated; {len(text)} total chars)\n" return subject, text, skip @@ -155,13 +145,16 @@ def run_clerk(label, diff_text, index): """ prompt = CLERK_PROMPT.format(label=label, index=index, diff=diff_text) try: - result = subprocess.run( + result = process.run( ["claude", "-p", "--model", "sonnet", "--max-turns", str(MAX_TURNS)], - input=prompt, capture_output=True, text=True, - timeout=TIMEOUT_SECONDS, cwd=str(REPO_ROOT), + cwd=REPO_ROOT, + input=prompt, + timeout=TIMEOUT_SECONDS, + check=False, + missing_fix="install the claude CLI, or set SR_CLERK=0 to skip the review", ) output = result.stdout.strip() - except subprocess.TimeoutExpired: + except process.ProcessTimeout: return "INCOMPLETE", f"{label}: review timed out after {TIMEOUT_SECONDS}s (not a contradiction)." if not output: @@ -175,39 +168,49 @@ def run_clerk(label, diff_text, index): return "INCOMPLETE", output + "\n\n(No clear verdict on last line — recorded as INCOMPLETE, not a contradiction.)" -def load_index(): - readme = REPO_ROOT / "decisions" / "README.md" - return readme.read_text()[:8000] if readme.exists() else "(decisions/README.md not found)" +def load_index() -> str: + """The decision-domain index the clerk greps from. + + `governance/README.md` since the DQR tree moved; the old `decisions/README.md` + path silently resolved to "not found", which handed every clerk an empty + index and made it grep blind. + """ + readme = REPO_ROOT / "governance" / "README.md" + return readme.read_text()[:8000] if readme.exists() else "(governance/README.md not found)" -def main(): - plan_only = "--plan" in sys.argv[1:] +def run(plan_only: bool = False) -> str: + """Review the push range commit by commit. Returns the overall verdict.""" CACHE_DIR.mkdir(parents=True, exist_ok=True) rng = base_range() commits = list_commits(rng) if not commits: FINDINGS_FILE.write_text("# Clerk Review\n\nNo commits to review.\n\nVerdict: APPROVED\n") - print(" clerk: no commits to review", file=sys.stderr) - sys.stderr.flush() - print("APPROVED") - return 0 + console.event("clerk: no commits to review") + console.out("APPROVED") + return "APPROVED" units = [commit_unit(sha) for sha in commits] # [(subject, text, skip), ...] if plan_only: - print(f"clerk plan: {len(commits)} commit(s) over {rng} " - f"(budget {COMMIT_BUDGET} chars, {WORKERS} workers, {MAX_TURNS} turns each)") + console.out(f"clerk plan: {len(commits)} commit(s) over {rng} " + f"(budget {COMMIT_BUDGET} chars, {WORKERS} workers, " + f"{MAX_TURNS} turns each)") for i, (subject, text, skip) in enumerate(units): tag = "SKIP (Clerk-Skip:)" if skip else "review" - print(f" commit {i + 1}/{len(commits)}: {len(text):>9} chars [{tag}] {subject}") - return 0 + console.out( + f" commit {i + 1}/{len(commits)}: {len(text):>9} chars [{tag}] {subject}" + ) + return "PLAN" index = load_index() reviewable = sum(1 for _, _, skip in units if not skip) - print(f" clerk: {len(commits)} commit(s) — {reviewable} to review, " - f"{len(commits) - reviewable} auto-approved (Clerk-Skip:); {WORKERS} parallel...", - file=sys.stderr) + console.event( + f"clerk: {len(commits)} commit(s) — {reviewable} to review, " + f"{len(commits) - reviewable} auto-approved (Clerk-Skip:); {WORKERS} parallel", + phase="clerk", + ) def task(i, subject, text, skip): label = f"commit {i + 1}/{len(commits)} ({subject})" @@ -222,7 +225,7 @@ def main(): for fut in as_completed(futures): i, label, verdict, findings = fut.result() results[i] = (label, verdict, findings) - print(f" clerk: {label} — {verdict}", file=sys.stderr) + console.event(f"clerk: {label} — {verdict}", phase="clerk") verdicts = [r[1] for r in results] if "REJECTED" in verdicts: @@ -243,11 +246,15 @@ def main(): parts.append(f"\n---\n\n## {label} — {verdict}\n\n{findings}\n") FINDINGS_FILE.write_text("\n".join(parts)) - print(f" clerk: overall verdict — {overall} (details: {FINDINGS_FILE})", file=sys.stderr) - sys.stderr.flush() - print(overall) - return 1 if overall == "REJECTED" else 0 + console.event( + f"clerk: overall verdict — {overall} (details: {FINDINGS_FILE})", phase="clerk" + ) + console.out(overall) - -if __name__ == "__main__": - sys.exit(main()) + if overall == "REJECTED": + raise ReachError( + f"clerk rejected {n_rej} commit(s) — a named, active D-record is contradicted", + fix=f"read {FINDINGS_FILE.relative_to(REPO_ROOT)}; amend the commit or " + "the decision, or add a `Clerk-Skip:` trailer if the clerk is wrong", + ) + return overall diff --git a/tooling/domains/dev/environment.py b/tooling/domains/dev/environment.py new file mode 100644 index 000000000..d642c15d2 --- /dev/null +++ b/tooling/domains/dev/environment.py @@ -0,0 +1,203 @@ +"""Environment setup — the decisions, separated from the performing (D-263). + +These three ported worst and mattered most to get right. `install-godot` +downloads and unzips a pinned build, `install-rust` drives `rustup`, +`worktree-setup` manipulates git worktrees — none of which can be exercised in +a gate without actually doing it. + +So the shape here is deliberate: **every function that decides is pure and +importable, and every function that acts is a thin call through +`core.process.run`.** `godot_plan()` answers "what would be downloaded, and is +it needed" without touching the network; `install_godot()` performs it. The +test of a correct port is whether the decisions can be exercised WITHOUT +performing them — a rewrite that cannot be tested has to be trusted instead, +and trusting an installer is how a working environment becomes an +unreproducible one. +""" + +from __future__ import annotations + +import os +import platform +import shutil +from dataclasses import dataclass +from pathlib import Path + +from tooling.core import config, console, process +from tooling.core.errors import ReachError + +DEFAULT_GODOT_VERSION = "4.6" +INSTALL_DIR = Path.home() / "bin" +GODOT_BINARY = INSTALL_DIR / "godot4" + +# Godot publishes one archive per platform triple; an unsupported pair is a +# clear failure rather than a download that 404s. +PLATFORMS = { + ("Linux", "x86_64"): "linux.x86_64", + ("Linux", "aarch64"): "linux.arm64", + ("Darwin", "x86_64"): "macos.universal", + ("Darwin", "arm64"): "macos.universal", +} + + +@dataclass(frozen=True) +class GodotPlan: + """What installing Godot would do — decided without doing any of it.""" + + wanted: str + installed: str | None + platform_tag: str + url: str + filename: str + + @property + def already_current(self) -> bool: + return self.installed == self.wanted + + +def godot_plan(wanted: str | None = None) -> GodotPlan: + """Decide what an install would fetch. Pure apart from reading the binary.""" + wanted = wanted or os.environ.get("GODOT_VERSION", DEFAULT_GODOT_VERSION) + system, machine = platform.system(), platform.machine() + tag = PLATFORMS.get((system, machine)) + if tag is None: + raise ReachError( + f"unsupported platform: {system} {machine}", + fix="install Godot manually from https://godotengine.org/download", + ) + + filename = f"Godot_v{wanted}-stable_{tag}.zip" + return GodotPlan( + wanted=wanted, + installed=installed_godot_version(), + platform_tag=tag, + filename=filename, + url=( + "https://github.com/godotengine/godot/releases/download/" + f"{wanted}-stable/{filename}" + ), + ) + + +def installed_godot_version() -> str | None: + """The major.minor already on disk, or None. Never raises.""" + if not GODOT_BINARY.is_file() or not os.access(GODOT_BINARY, os.X_OK): + return None + result = process.run([str(GODOT_BINARY), "--version"], check=False) + first = (result.stdout or "").splitlines() + if not first: + return None + return ".".join(first[0].split(".")[:2]) + + +def install_godot(wanted: str | None = None) -> GodotPlan: + """Perform the install. The decision is `godot_plan`; this is the doing.""" + plan = godot_plan(wanted) + if plan.already_current: + console.verdict(f"Godot {plan.wanted} already installed at {GODOT_BINARY}") + return plan + if plan.installed: + console.event( + f"Godot found but version is {plan.installed}, want {plan.wanted}", + level="warn", + ) + + INSTALL_DIR.mkdir(parents=True, exist_ok=True) + import tempfile + + with tempfile.TemporaryDirectory() as tmp: + archive = Path(tmp) / plan.filename + console.event(f"downloading Godot {plan.wanted} for {plan.platform_tag}", phase="godot") + process.run( + ["curl", "-fSL", "--progress-bar", "-o", str(archive), plan.url], + capture=False, + fix=f"check that GODOT_VERSION={plan.wanted} is a real release", + missing_fix="install curl", + ) + console.event("extracting", phase="godot") + process.run( + ["unzip", "-q", "-o", str(archive), "-d", tmp], + fix="the archive may be truncated — re-run to download it again", + missing_fix="install unzip", + ) + extracted = next( + (p for p in Path(tmp).iterdir() if p.is_file() and p.name.startswith("Godot")), + None, + ) + if extracted is None: + raise ReachError( + "the Godot archive contained no binary", + fix="the download may be corrupt — delete it and re-run", + ) + shutil.move(str(extracted), GODOT_BINARY) + GODOT_BINARY.chmod(0o755) + + console.verdict(f"Godot {plan.wanted} installed at {GODOT_BINARY}") + return plan + + +def install_rust() -> None: + """Ensure rustup, clippy and rustfmt are present.""" + if shutil.which("cargo") is None: + raise ReachError( + "cargo is not installed", + fix="install Rust from https://rustup.rs, then re-run — this command " + "adds the components but does not bootstrap the toolchain", + ) + for component in ("clippy", "rustfmt"): + console.event(f"ensuring {component}", phase="rust") + process.run( + ["rustup", "component", "add", component], + check=False, + missing_fix="install rustup from https://rustup.rs", + ) + console.verdict("Rust toolchain ready — clippy and rustfmt present") + + +def worktree_plan(branch: str) -> Path: + """Where a worktree for `branch` would go. Raises if it cannot be created. + + Both refusals are the original's and both are worth keeping: a worktree + created from inside a worktree nests confusingly, and silently reusing an + existing directory is how two branches end up sharing one tree. + """ + root = config.repo_root() + if (root / ".git").is_file(): + raise ReachError( + "this is a worktree, not the main checkout", + fix="run this from the main checkout — nesting worktrees confuses " + "both git and the tools that resolve the repo root", + ) + target = root / ".worktrees" / branch + if target.exists(): + raise ReachError( + f"{target} already exists", + fix=f"reuse it, or remove it first with: git worktree remove {target}", + ) + return target + + +def setup_worktree(branch: str, start: str = "HEAD") -> Path: + """Create a worktree and link the venv into it.""" + target = worktree_plan(branch) + root = config.repo_root() + + process.run( + ["git", "worktree", "add", str(target), "-b", branch, start], + cwd=root, + fix=f"check that {start} is a valid ref and {branch} is not already a branch", + ) + + venv = root / ".venv" + if venv.is_dir() and not (target / ".venv").exists(): + (target / ".venv").symlink_to(venv) + console.event(f"linked .venv -> {venv}") + + console.verdict( + f"worktree ready: {target}\n" + f" pql in this worktree needs --vault {target} (FR-4)\n" + " tea runs from the main checkout — its go-git cannot read a linked worktree\n" + " reach follows whichever checkout it was installed from; `make reach-repoint` " + "if you want it to follow this one" + ) + return target diff --git a/tooling/perf-baseline b/tooling/domains/dev/perf.py similarity index 58% rename from tooling/perf-baseline rename to tooling/domains/dev/perf.py index c5355f0af..96a6228ea 100755 --- a/tooling/perf-baseline +++ b/tooling/domains/dev/perf.py @@ -1,24 +1,21 @@ #!/usr/bin/env python3 -"""Performance baseline tooling. +"""Performance baseline — tick timing, memory, and shadowcast scaling. -Runs the server benchmark suite, captures tick timing, memory usage, and entity -count scaling metrics, outputs results to tests/perf/. - -Usage: - tooling/perf-baseline Run benchmarks and save baseline - tooling/perf-baseline --compare Compare current run against saved baseline (no save) - -Exit code 0 = success, 1 = failure or regression detected (--compare mode). +Builds the server in release, runs the benchmark suite, and either writes +tests/perf/baseline.json or compares against the committed one. The compare +path exits non-zero on a >20% regression or a p95 over the D-026 tick budget. """ +from __future__ import annotations + import json import re -import subprocess -import sys from datetime import datetime, timezone -from pathlib import Path -ROOT = Path(__file__).resolve().parent.parent +from tooling.core import config, console, process +from tooling.core.errors import ReachError + +ROOT = config.repo_root() PERF_DIR = ROOT / "tests" / "perf" BASELINE_FILE = PERF_DIR / "baseline.json" @@ -29,39 +26,35 @@ TICK_BUDGET_US = 100_000 # 100ms def run_command(cmd, **kwargs): """Run a command in the server directory and return the result.""" - return subprocess.run( - cmd, capture_output=True, text=True, cwd=ROOT / "server", **kwargs - ) + return process.run(cmd, cwd=ROOT / "server", check=False, **kwargs) def get_git_info(): - """Get current git commit and branch.""" - commit = subprocess.run( - ["git", "rev-parse", "--short", "HEAD"], - capture_output=True, text=True, cwd=ROOT, + """Current commit and branch. check=False — a shallow or detached tree is + not an error here, it just means the stamp carries less.""" + commit = process.run( + ["git", "rev-parse", "--short", "HEAD"], cwd=ROOT, check=False ).stdout.strip() - branch = subprocess.run( - ["git", "rev-parse", "--abbrev-ref", "HEAD"], - capture_output=True, text=True, cwd=ROOT, + branch = process.run( + ["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=ROOT, check=False ).stdout.strip() return {"commit": commit, "branch": branch} def run_tick_benchmark(): """Run perf_tick_timing test and parse PERF_RESULT JSON.""" - print(" Running tick timing benchmark (release mode)...") + console.event("running tick timing benchmark (release mode)", phase="bench") result = run_command([ "cargo", "test", "--release", "--test", "perf_bench", "--", "--ignored", "--nocapture", "perf_tick_timing", ]) if result.returncode != 0: - print(f" FAILED: tick benchmark exited {result.returncode}") + console.event(f"tick benchmark exited {result.returncode}", level="error") if result.stderr: - # Print last 20 lines of stderr for diagnostics - lines = result.stderr.strip().splitlines() - for line in lines[-20:]: - print(f" {line}") + # Last 20 lines of stderr are the diagnostic; the rest is build noise. + for line in result.stderr.strip().splitlines()[-20:]: + console.event(line, level="error") return None # Parse PERF_RESULT: line from stdout @@ -70,20 +63,22 @@ def run_tick_benchmark(): json_str = line[len("PERF_RESULT:"):] return json.loads(json_str) - print(" WARNING: No PERF_RESULT found in test output") + console.event("no PERF_RESULT found in test output", level="warn") return None def run_shadowcast_benchmark(): """Run shadowcast benchmark and parse structured output.""" - print(" Running shadowcast benchmark (release mode)...") + console.event("running shadowcast benchmark (release mode)", phase="bench") result = run_command([ "cargo", "test", "--release", "--test", "shadowcast_bench", "--", "--ignored", "--nocapture", "benchmark_symmetric_vs_recursive", ]) if result.returncode != 0: - print(f" FAILED: shadowcast benchmark exited {result.returncode}") + console.event( + f"shadowcast benchmark exited {result.returncode}", level="error" + ) return None configs = [] @@ -173,29 +168,28 @@ def compare_baselines(old, new): return regressions, improvements -def main(): - compare_mode = "--compare" in sys.argv - - print("=== Performance Baseline ===\n") - - # Build in release mode first - print("Building server (release)...") +def run(compare_mode: bool = False) -> None: + """Build release, run the benchmarks, then write or compare the baseline.""" + console.event("building server (release)", phase="build") build = run_command(["cargo", "build", "--release"]) if build.returncode != 0: - print("BUILD FAILED") - lines = build.stderr.strip().splitlines() - for line in lines[-20:]: - print(f" {line}") - return 1 - - print("\nRunning benchmarks...\n") + for line in build.stderr.strip().splitlines()[-20:]: + console.event(line, level="error") + raise ReachError( + "the release build failed, so there is nothing to benchmark", + fix="fix the build errors above, then re-run — a perf number from a " + "stale binary is worse than no number", + ) tick_results = run_tick_benchmark() shadowcast_results = run_shadowcast_benchmark() if not tick_results: - print("\nFATAL: tick benchmark failed -- no baseline generated") - return 1 + raise ReachError( + "the tick benchmark produced no result — no baseline generated", + fix="run `cargo test --release --test perf_bench -- --ignored " + "--nocapture perf_tick_timing` in server/ to see why it failed", + ) # Assemble baseline git_info = get_git_info() @@ -211,72 +205,70 @@ def main(): # Report tt = baseline["tick_timing"] - print(f"\n--- Results ---") - print(f"Git: {git_info['commit']} ({git_info['branch']})") - print(f"Tick timing ({tt.get('measured_ticks', '?')} ticks, " - f"{tt.get('warmup_ticks', '?')} warmup):") - print(f" min: {tt.get('min_us', '?')}us") - print(f" mean: {tt.get('mean_us', '?')}us") - print(f" p95: {tt.get('p95_us', '?')}us") - print(f" max: {tt.get('max_us', '?')}us") + console.out("--- Results ---") + console.out(f"Git: {git_info['commit']} ({git_info['branch']})") + console.out(f"Tick timing ({tt.get('measured_ticks', '?')} ticks, " + f"{tt.get('warmup_ticks', '?')} warmup):") + console.out(f" min: {tt.get('min_us', '?')}us") + console.out(f" mean: {tt.get('mean_us', '?')}us") + console.out(f" p95: {tt.get('p95_us', '?')}us") + console.out(f" max: {tt.get('max_us', '?')}us") ent = baseline["entities"] - print(f"Entities: avg {ent.get('avg_per_snapshot', '?')}, " - f"max {ent.get('max_per_snapshot', '?')}") + console.out(f"Entities: avg {ent.get('avg_per_snapshot', '?')}, " + f"max {ent.get('max_per_snapshot', '?')}") - mem = baseline["memory"] - rss = mem.get("rss_kb") + rss = baseline["memory"].get("rss_kb") if rss: - print(f"Memory: {rss} KB RSS ({rss / 1024:.1f} MB)") + console.out(f"Memory: {rss} KB RSS ({rss / 1024:.1f} MB)") if shadowcast_results: n = len(shadowcast_results.get("configs", [])) - print(f"Shadowcast: {n} configurations benchmarked") + console.out(f"Shadowcast: {n} configurations benchmarked") # Budget check p95 = tt.get("p95_us", 0) if p95 > TICK_BUDGET_US: - print(f"\nBUDGET EXCEEDED: p95 {p95}us > {TICK_BUDGET_US}us (D-026)") + console.out(f"BUDGET EXCEEDED: p95 {p95}us > {TICK_BUDGET_US}us (D-026)") else: budget_pct = p95 / TICK_BUDGET_US * 100 if TICK_BUDGET_US else 0 - print(f"\nBudget: {budget_pct:.1f}% of {TICK_BUDGET_US}us tick budget (D-026)") + console.out( + f"Budget: {budget_pct:.1f}% of {TICK_BUDGET_US}us tick budget (D-026)" + ) # Compare with previous baseline if it exists + regressions: list[str] = [] if BASELINE_FILE.exists(): - with open(BASELINE_FILE) as f: - old_baseline = json.load(f) + old_baseline = json.loads(BASELINE_FILE.read_text()) old_commit = old_baseline.get("git", {}).get("commit", "?") - print(f"\n--- Comparison vs {old_commit} ---") + console.out(f"--- Comparison vs {old_commit} ---") regressions, improvements = compare_baselines(old_baseline, baseline) for r in regressions: - print(f" REGRESSION: {r}") + console.out(f" REGRESSION: {r}") for i in improvements: - print(f" IMPROVEMENT: {i}") + console.out(f" IMPROVEMENT: {i}") if not regressions and not improvements: - print(" No significant changes.") - if compare_mode and regressions: - print(f"\n{len(regressions)} regression(s) detected.") - return 1 + console.out(" No significant changes.") elif compare_mode: - print(f"\nERROR: --compare requires a saved baseline at {BASELINE_FILE.relative_to(ROOT)}") - print("Run `make perf-baseline` first to create one.") - return 1 + raise ReachError( + f"--compare needs a saved baseline at {BASELINE_FILE.relative_to(ROOT)}", + fix="run `reach dev perf` once without --compare to create one", + ) if compare_mode: - return 0 + if regressions: + raise ReachError( + f"{len(regressions)} performance regression(s) detected", + fix="see the REGRESSION lines above; if the change is intended, " + "re-run without --compare to accept it as the new baseline", + ) + console.verdict("perf: no regressions against the saved baseline") + return # Save baseline (strip per-tick array — too noisy for git diffs) PERF_DIR.mkdir(parents=True, exist_ok=True) committed = json.loads(json.dumps(baseline)) committed["tick_timing"].pop("all_us", None) + BASELINE_FILE.write_text(json.dumps(committed, indent=2) + "\n") - with open(BASELINE_FILE, "w") as f: - json.dump(committed, f, indent=2) - f.write("\n") - - print(f"\nBaseline written to {BASELINE_FILE.relative_to(ROOT)}") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) + console.verdict(f"baseline written to {BASELINE_FILE.relative_to(ROOT)}") diff --git a/tooling/domains/dev/router.py b/tooling/domains/dev/router.py index 1745f9205..2d562fdfd 100644 --- a/tooling/domains/dev/router.py +++ b/tooling/domains/dev/router.py @@ -7,7 +7,7 @@ import typer from tooling.core import cli, console from tooling.core.command import command from tooling.core.errors import ReachError -from tooling.domains.dev import service +from tooling.domains.dev import environment, service app = cli.domain("dev", "Developer environment and self-diagnosis.") @@ -17,6 +17,79 @@ def _domain() -> None: """Keeps `dev` a group (Typer collapses a single-command app).""" +@app.command("install-godot") +@command +def install_godot( + version: str = typer.Option(None, "--version", help="Godot version to install."), + plan_only: bool = typer.Option( + False, "--plan", help="Report what would be downloaded without doing it." + ), +) -> None: + """Install the pinned Godot build to ~/bin/godot4, if it is not already there.""" + if plan_only: + plan = environment.godot_plan(version) + console.out(f"wanted {plan.wanted}") + console.out(f"installed {plan.installed or '(none)'}") + console.out(f"platform {plan.platform_tag}") + console.out(f"url {plan.url}") + console.verdict( + "already current — nothing to do" + if plan.already_current + else "would download and install" + ) + return + environment.install_godot(version) + + +@app.command("install-rust") +@command +def install_rust() -> None: + """Ensure clippy and rustfmt are present on the Rust toolchain.""" + environment.install_rust() + + +@app.command("worktree") +@command +def worktree( + branch: str = typer.Argument(..., help="Branch name for the new worktree."), + start: str = typer.Argument("HEAD", help="Start point."), + plan_only: bool = typer.Option( + False, "--plan", help="Report where it would go without creating it." + ), +) -> None: + """Create a worktree under .worktrees/ with the venv linked in.""" + if plan_only: + console.verdict(f"would create {environment.worktree_plan(branch)}") + return + environment.setup_worktree(branch, start) + + +@app.command("perf") +@command +def perf( + compare: bool = typer.Option( + False, "--compare", help="Compare against the saved baseline instead of writing one." + ), +) -> None: + """Benchmark the server and write or check tests/perf/baseline.json.""" + from tooling.domains.dev import perf as perf_module + + perf_module.run(compare) + + +@app.command("clerk") +@command +def clerk( + plan_only: bool = typer.Option( + False, "--plan", help="Show the per-commit plan without spawning any clerk." + ), +) -> None: + """Review the push range for decision contradictions, one agent per commit.""" + from tooling.domains.dev import clerk as clerk_module + + clerk_module.run(plan_only) + + @app.command("selftest") @command def selftest( diff --git a/tooling/domains/generate/__init__.py b/tooling/domains/generate/__init__.py new file mode 100644 index 000000000..c0beaf465 --- /dev/null +++ b/tooling/domains/generate/__init__.py @@ -0,0 +1,8 @@ +"""The `generate` domain — content generators not owned by another domain. + +Deliberately a small residual rather than a catch-all. Brands and corporations +live here because they are economic *content* generated ahead of the sim rather +than by it; the character manifest because it indexes asset directories. When +`ledger` lands, brands and corporations are worth revisiting — they may belong +under it. +""" diff --git a/tooling/generate-character-manifest b/tooling/domains/generate/character_manifest.py similarity index 78% rename from tooling/generate-character-manifest rename to tooling/domains/generate/character_manifest.py index eabbe8385..048d657d6 100755 --- a/tooling/generate-character-manifest +++ b/tooling/domains/generate/character_manifest.py @@ -2,7 +2,7 @@ """Generate client/assets/characters/manifest.json from the asset directories. Run this whenever artists add new assets so the manifest stays in sync: - tooling/generate-character-manifest + reach generate character-manifest The manifest is the single source of truth for CharacterCreation asset IDs. DirAccess.open() cannot enumerate res:// paths in exported PCK builds (#720). @@ -12,9 +12,13 @@ Output: client/assets/characters/manifest.json import json import os -import sys -ASSETS_ROOT = os.path.join(os.path.dirname(__file__), "..", "client", "assets", "characters") +from tooling.core import config, console + +# config.repo_root(), not __file__-relative: this file moved two directories +# deeper, and a relative root would resolve to nothing and report an empty +# manifest as success (the failure that bit validate-checklist in T-1282). +ASSETS_ROOT = str(config.path("client", "assets", "characters")) MANIFEST_PATH = os.path.join(ASSETS_ROOT, "manifest.json") @@ -92,20 +96,23 @@ def build_manifest() -> dict: } -def main() -> None: +def run() -> None: + """Rebuild the manifest from the asset directories and report the counts.""" manifest = build_manifest() output = json.dumps(manifest, indent=2) + "\n" with open(MANIFEST_PATH, "w", encoding="utf-8") as f: f.write(output) - print(f"Written: {MANIFEST_PATH}") - print(f" body_types : {len(manifest['body_types'])}") - print(f" heads : {len(manifest['heads'])}") - print(f" hair : {len(manifest['hair'])}") - print(f" facial_hair: {len(manifest['facial_hair'])}") - print(f" eyebrows : {len(manifest['eyebrows'])}") - print(f" clothing : {len(manifest['clothing'])}") - print(f" accessories: {len(manifest['accessories'])}") - - -if __name__ == "__main__": - main() + console.event(f"Written: {MANIFEST_PATH}") + for key in ( + "body_types", + "heads", + "hair", + "facial_hair", + "eyebrows", + "clothing", + "accessories", + ): + console.event(f" {key:<12}: {len(manifest[key])}") + console.verdict( + f"generate-character-manifest: OK — {sum(len(v) for v in manifest.values())} entries" + ) diff --git a/tooling/generate_corp_stubs.py b/tooling/domains/generate/corp_stubs.py similarity index 93% rename from tooling/generate_corp_stubs.py rename to tooling/domains/generate/corp_stubs.py index 507d34226..1e23571eb 100644 --- a/tooling/generate_corp_stubs.py +++ b/tooling/domains/generate/corp_stubs.py @@ -1,16 +1,16 @@ #!/usr/bin/env python3 -"""Generate minimal wiki stub pages for corps in brands.toml that don't yet have wiki pages. +"""Generate minimal wiki stub pages for corps in brands.toml with no wiki page yet. -Usage: - python3 tooling/generate_corp_stubs.py [--dry-run] - -In --dry-run mode, prints which stubs would be created without writing any files. +`--dry-run` reports which stubs would be created without writing any of them. """ -import argparse +from __future__ import annotations + from pathlib import Path -REPO = Path(__file__).parent.parent +from tooling.core import config, console + +REPO = config.repo_root() BRANDS_TOML = REPO / "wiki/economics/corporations/brands.toml" CORPS_DIR = REPO / "wiki/corporations" @@ -374,21 +374,12 @@ cross_refs: [] {title} is a {cat_desc} operating out of {sys_name} ({origin_system}). Details to be expanded. """ -def main(): - parser = argparse.ArgumentParser( - description="Generate minimal wiki stub pages for corps in brands.toml." - ) - parser.add_argument( - "--dry-run", - action="store_true", - help="Print which stubs would be created without writing any files.", - ) - args = parser.parse_args() - +def run(dry_run: bool = False) -> None: + """Write a stub page for every corp in brands.toml that lacks one.""" corps = parse_brands_toml(BRANDS_TOML) - print(f"Found {len(corps)} unique corp_ids in brands.toml") - if args.dry_run: - print("(dry-run — no files will be written)\n") + console.event(f"Found {len(corps)} unique corp_ids in brands.toml") + if dry_run: + console.event("(dry-run — no files will be written)") created = 0 skipped = 0 @@ -397,17 +388,13 @@ def main(): if stub_path.exists(): skipped += 1 continue - if args.dry_run: - print(f" (dry-run) Would create: {slug}.md") + if dry_run: + console.event(f" (dry-run) Would create: {slug}.md") else: content = generate_stub(slug, meta["origin_system"], meta["brand_category"]) stub_path.write_text(content, encoding="utf-8") - print(f" created: {slug}.md") + console.event(f" created: {slug}.md") created += 1 - action = "Would create" if args.dry_run else "Created" - print(f"\nDone. {action} {created} stubs, skipped {skipped} existing.") - - -if __name__ == "__main__": - main() + action = "Would create" if dry_run else "Created" + console.verdict(f"{action} {created} stubs, skipped {skipped} existing.") diff --git a/tooling/domains/generate/router.py b/tooling/domains/generate/router.py new file mode 100644 index 000000000..ace41b21a --- /dev/null +++ b/tooling/domains/generate/router.py @@ -0,0 +1,58 @@ +"""Transport for the `generate` domain — args in, delegate, format out.""" + +from __future__ import annotations + +import typer + +from tooling.core import cli, console, process +from tooling.core.command import command +from tooling.domains.generate import character_manifest, corp_stubs + +app = cli.domain("generate", "Content generators — brands, corporations, manifests.") + + +@app.callback() +def _domain() -> None: + """Keeps `generate` a group (Typer collapses a single-command app).""" + + +@app.command("brands") +@command +def brands(args: list[str] = typer.Argument(None, help="Passed to the binary.")) -> None: + """Refresh wiki/economics/corporations/generated_brands.toml. + + Note this is also a SUBROUTINE of import_economics rather than only a + standalone verb: `make regen-db` shells out to the same binary as its first + step, which is why changes to its Rust source invalidate the systems.db + stamp even though no Python changed (.claude/rules/asset-pipeline.md). + """ + output = process.cargo_binary("generate_brands", *(args or [])) + if output.strip(): + console.out(output.rstrip()) + + +@app.command("corporations") +@command +def corporations( + args: list[str] = typer.Argument(None, help="Passed to the binary."), +) -> None: + """Generate corporation records.""" + output = process.cargo_binary("generate_corporations", *(args or [])) + if output.strip(): + console.out(output.rstrip()) + + +@app.command("character-manifest") +@command +def manifest() -> None: + """Regenerate assets/characters/manifest.json from the asset directories.""" + character_manifest.run() + + +@app.command("corp-stubs") +@command +def stubs( + dry_run: bool = typer.Option(False, "--dry-run", help="Report without writing."), +) -> None: + """Write wiki stub pages for corporations that lack one.""" + corp_stubs.run(dry_run=dry_run) diff --git a/tooling/domains/pr/__init__.py b/tooling/domains/pr/__init__.py new file mode 100644 index 000000000..9096d185c --- /dev/null +++ b/tooling/domains/pr/__init__.py @@ -0,0 +1,7 @@ +"""The `pr` domain — the review loop. + +A domain the original map omitted. `tea-comment` and `pr-watchlist-diff` are +neither environment setup nor content generation; folding them into `dev` would +make `dev` the drawer everything ambiguous goes into, which is how `core/` rots +and the same argument applies here. +""" diff --git a/tooling/pql-board-html b/tooling/domains/pr/board.py similarity index 95% rename from tooling/pql-board-html rename to tooling/domains/pr/board.py index c3017fb7d..88eb89630 100755 --- a/tooling/pql-board-html +++ b/tooling/domains/pr/board.py @@ -15,26 +15,34 @@ Data comes from pql's native JSON output only (no output parsing): upstream feature request per the no-ad-hoc-wrappers rule) - `pql plan status` → header counts -Usage: tooling/pql-board-html [OUTPUT.html] +Usage: reach pr board-html [--output OUTPUT.html] (default output: /tmp/pql-board.html) The emitted file is an Artifact-ready fragment: no doctype/html/head/body wrapper (the Artifact pipeline adds those); it starts with + <style>. """ -import html import json -import subprocess -import sys from datetime import datetime, timezone +from tooling.core import config, console, process + NON_TERMINAL = ["in_progress", "review", "ready", "backlog"] SHOW_CHUNK = 40 def pql(*args): - out = subprocess.run( - ["pql", *args], capture_output=True, text=True, check=True + """Query the planning store. Goes through the guarded exec (D-263). + + Was a bare subprocess.run with check=True, which raised + CalledProcessError — a traceback at someone who wanted to know that pql + was not installed. + """ + out = process.run( + ["pql", *args], + cwd=config.repo_root(), + missing_fix="install pql, or check PATH in a non-interactive shell", + fix=f"run `pql {' '.join(args)}` directly to see what it reports", ).stdout return json.loads(out) @@ -337,8 +345,9 @@ if(initial&&byId[initial]) select(initial,false); """ -def main(): - out_path = sys.argv[1] if len(sys.argv) > 1 else "/tmp/pql-board.html" +def run(out_path=None) -> None: + """Render the board to `out_path`, defaulting to the repo's scratch dir.""" + out_path = str(out_path) if out_path else str(config.path(".cache", "pql-board.html")) tickets, blockers, plan = fetch() stamp = datetime.now(timezone.utc).astimezone().strftime("%Y-%m-%d %H:%M") data = json.dumps( @@ -349,12 +358,8 @@ def main(): page = PAGE.replace("__STAMP__", stamp).replace("__DATA__", data) with open(out_path, "w", encoding="utf-8") as f: f.write(page) - print( + console.verdict( f"wrote {out_path}: {len(tickets)} tickets, " f"{sum(len(v) for v in blockers.values())} dep edges, " f"{len(page) // 1024} KiB" ) - - -if __name__ == "__main__": - main() diff --git a/tooling/domains/pr/router.py b/tooling/domains/pr/router.py new file mode 100644 index 000000000..4c658ab1e --- /dev/null +++ b/tooling/domains/pr/router.py @@ -0,0 +1,63 @@ +"""Transport for the `pr` domain — args in, delegate, format out.""" + +from __future__ import annotations + +from pathlib import Path + +import typer + +from tooling.core import cli, console +from tooling.core.command import command +from tooling.domains.pr import service + +app = cli.domain("pr", "The review loop — comments and the stale-DB watchlist.") + + +@app.callback() +def _domain() -> None: + """Keeps `pr` a group (Typer collapses a single-command app).""" + + +@app.command("comment") +@command +def comment( + number: str = typer.Argument(..., help="PR or issue number."), + body: str = typer.Argument(..., help="The comment, or @path to read it from a file."), +) -> None: + """Post a comment on a Gitea PR or issue. + + Use `@path` for anything long. That form exists because an inline body would + need a `$(...)` subshell, and a subshell breaks the permission gate's prefix + matching (.claude/rules/tea-cli.md). + """ + service.comment(number, body) + console.verdict(f"commented on #{number}") + + +@app.command("watchlist-diff") +@command +def watchlist_diff( + base: str = typer.Argument(..., help="Base ref."), + head: str = typer.Argument("HEAD", help="Head ref."), +) -> None: + """List changed files in a range that could make the committed systems.db stale. + + The watched set is read from tooling/generator_sources.py rather than + restated here, so it cannot drift from the registry the stamp check uses. + """ + changed = service.watchlist_diff(base, head) + for path in changed: + console.out(path) + if not changed: + console.verdict(f"pr-watchlist-diff: nothing generator-relevant in {base}...{head}") + + +@app.command("board-html") +@command +def board_html( + output: Path = typer.Option(None, "--output", help="Where to write the board HTML."), +) -> None: + """Render the pql ticket board as a standalone HTML page.""" + from tooling.domains.pr import board + + board.run(output) diff --git a/tooling/domains/pr/service.py b/tooling/domains/pr/service.py new file mode 100644 index 000000000..c9dc05d40 --- /dev/null +++ b/tooling/domains/pr/service.py @@ -0,0 +1,93 @@ +"""Logic for the `pr` domain. Transport-agnostic (D-263).""" + +from __future__ import annotations + +import shutil +from pathlib import Path + +from tooling.core import config, process +from tooling.core.errors import ReachError + +LOGIN = "schweitz" +REPO = "jpmschweitzer/settled-reach" + +# Paths whose change means the committed systems.db may be stale. The generator +# set is read from the registry rather than restated, so this list cannot drift +# from tooling/generator_sources.py (T-1067). +EXTRA_WATCHED = ( + "tooling/planet-gen/import_heightmaps.py", + "tooling/planet-gen/import_province_boundaries.py", + "server/data/systems-schema.sql", + "wiki/star-systems/", + "wiki/economics/", +) + + +def comment(number: str, body_or_path: str) -> None: + """Post a comment on a Gitea PR or issue. + + `@path` reads the body from a file — the form that exists because a long + comment inline would need a `$(...)` subshell, and a subshell breaks the + permission gate's prefix matching (.claude/rules/tea-cli.md). + + Every flag is explicit for the same reason the rule gives: omitting + `--login` or `--repo` sends `tea` interactive, and an interactive prompt + with no TTY does not wait — it crashes. + """ + body = _resolve_body(body_or_path) + + tea = shutil.which("tea") + if tea is None: + raise ReachError( + "the tea CLI is not on PATH", + fix="tea is symlinked into ~/.local/bin — check that, rather than " + "calling the linuxbrew path directly, which breaks the Bash(tea *) rule", + ) + + process.run( + [tea, "comment", "--login", LOGIN, "--repo", REPO, number, body], + cwd=config.repo_root(), + fix=f"check that PR/issue {number} exists and the login is authorised", + ) + + +def watchlist_diff(base: str, head: str) -> list[str]: + """Files changed in the range that would make the committed systems.db stale.""" + from tooling import generator_sources + + root = config.repo_root() + registered = sorted( + { + str(path.relative_to(root)) + for paths in generator_sources.GENERATOR_SOURCES.values() + for path in paths + } + ) + watched = [*registered, *EXTRA_WATCHED] + if not watched: + # The original checked this too: an empty registry would diff nothing + # and report a clean watchlist, which is the answer that lets a stale DB + # through. + raise ReachError( + "the generator-source registry is empty", + fix="tooling/generator_sources.py should list every generator input", + ) + + result = process.run( + ["git", "diff", "--name-only", f"{base}...{head}", "--", *watched], + cwd=config.repo_root(), + check=False, + ) + return [line for line in (result.stdout or "").splitlines() if line] + + +def _resolve_body(body_or_path: str) -> str: + if not body_or_path.startswith("@"): + return body_or_path + path = Path(body_or_path[1:]) + if not path.is_file(): + raise ReachError( + f"comment body file not found: {path}", + fix="write the comment to a file first, then pass @that-path", + ) + return path.read_text(encoding="utf-8") diff --git a/tooling/economy-db/economy_import/brands.py b/tooling/economy-db/economy_import/brands.py index e08ce768a..e8d835234 100644 --- a/tooling/economy-db/economy_import/brands.py +++ b/tooling/economy-db/economy_import/brands.py @@ -1,13 +1,18 @@ """Brand layer (D-189, #827): generate_brands shell-out, TOML import, validation.""" import sqlite3 -import subprocess import sys import tomllib from pathlib import Path from .errors import ImportAborted -from .paths import BRANDS_TOML, GENERATE_BRANDS_WRAPPER, GENERATED_BRANDS_TOML, REPO_ROOT +from .paths import BRANDS_TOML, GENERATED_BRANDS_TOML, REPO_ROOT + +# economy-db/ is hyphenated, so it is not importable as a package and cannot +# reach `tooling.core` by normal import. The bootstrap goes away with T-1272 +# (the hyphen sweep); until then it is explicit rather than implied. +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) VALID_BRAND_CATEGORIES: set[str] = { "terroir", "heritage_craft", "tech_premium", "cultural", @@ -28,28 +33,28 @@ def regenerate_brands() -> None: This replaces the former split (tooling/generate-brands run separately by make regen-db) with a single, coherent brand pipeline owned by one stamp. - The wrapper script builds the binary on demand and runs it with the default - canonical seed=1; callers that need non-canonical seeds must still invoke - the wrapper directly (experimentation only — committed output must be seed=1). + The binary is built on demand and run with the default canonical seed=1; + callers that need non-canonical seeds invoke it directly (experimentation + only — committed output must be seed=1). + + This used to shell out to a `tooling/generate-brands` bash wrapper. The + wrapper was one of three identical copies of build-if-missing-then-exec, so + it was retired into `core.process.cargo_binary` (T-1286) and this calls that + helper instead. Same binary, same seed, same output. """ - if not GENERATE_BRANDS_WRAPPER.exists(): - raise FileNotFoundError( - f"generate_brands wrapper not found at {GENERATE_BRANDS_WRAPPER}" - ) + from tooling.core.errors import ReachError + from tooling.core.process import cargo_binary + print(" [pre/10] Running generate_brands (Rust) to refresh generated_brands.toml...") - result = subprocess.run( - [str(GENERATE_BRANDS_WRAPPER)], - cwd=str(REPO_ROOT), - capture_output=True, - text=True, - ) - if result.returncode != 0: - print(result.stdout, file=sys.stderr) - print(result.stderr, file=sys.stderr) - raise ImportAborted() + try: + stdout = cargo_binary("generate_brands") + except ReachError as exc: + print(str(exc), file=sys.stderr) + raise ImportAborted() from exc + # Print the Rust binary's own summary lines (brands generated, coverage). # Indent so they fold under the pre-step heading. - for line in result.stdout.splitlines(): + for line in stdout.splitlines(): if line.strip(): print(f" {line}") diff --git a/tooling/economy-db/economy_import/paths.py b/tooling/economy-db/economy_import/paths.py index fb9557227..204202912 100644 --- a/tooling/economy-db/economy_import/paths.py +++ b/tooling/economy-db/economy_import/paths.py @@ -15,7 +15,6 @@ from generator_sources import ( COLOR_REGISTER_BANDS_TOML, CORP_HQ_PLACEMENT_TOML, CORPORATIONS_DIR, - GENERATE_BRANDS_WRAPPER, OBJECT_TAG_VOCABULARY_TOML, REPO_ROOT, SETTLEMENT_NAME_LOCKED_TOML, @@ -36,7 +35,6 @@ __all__ = [ "CURRENCY_ZONES_TOML", "DB_PATH", "GENERATED_BRANDS_TOML", - "GENERATE_BRANDS_WRAPPER", "OBJECT_TAG_VOCABULARY_TOML", "REPO_ROOT", "SCHEMA_SQL", diff --git a/tooling/generate-brands b/tooling/generate-brands deleted file mode 100755 index fd9109206..000000000 --- a/tooling/generate-brands +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env bash -# Generate minor brand products for the Settled Reach economy. -# -# Usage: -# tooling/generate-brands -# tooling/generate-brands --seed 42 --min-brands 10000 -# tooling/generate-brands --output wiki/economics/corporations/generated_brands.toml -# -# Builds on first run if binary doesn't exist. - -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" -ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" -BIN="$ROOT_DIR/server/target/debug/generate_brands" - -# Build if needed -if [ ! -f "$BIN" ]; then - echo "Building generate_brands..." >&2 - (cd "$ROOT_DIR/server" && cargo build --bin generate_brands 2>&1 | tail -3) >&2 -fi - -exec "$BIN" "$@" diff --git a/tooling/generate-corporations b/tooling/generate-corporations deleted file mode 100755 index a638a45db..000000000 --- a/tooling/generate-corporations +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env bash -# Generate Tier-3 corporations for the Settled Reach economy. -# -# Usage: -# tooling/generate-corporations -# tooling/generate-corporations --seed 42 --min-corps 5000 -# tooling/generate-corporations --output path/to/output.toml -# -# Builds on first run if binary doesn't exist. - -set -euo pipefail - -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" -ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" -BIN="$ROOT_DIR/server/target/debug/generate_corporations" - -# Build if needed -if [ ! -f "$BIN" ]; then - echo "Building generate_corporations..." >&2 - (cd "$ROOT_DIR/server" && cargo build --bin generate_corporations 2>&1 | tail -3) >&2 -fi - -exec "$BIN" "$@" diff --git a/tooling/generator_sources.py b/tooling/generator_sources.py index 647394427..345edf8b6 100644 --- a/tooling/generator_sources.py +++ b/tooling/generator_sources.py @@ -50,8 +50,8 @@ ECONOMY_IMPORT_PACKAGE_DIR: Path = ( REPO_ROOT / "tooling" / "economy-db" / "economy_import" ) -# Rust sources for the generate_brands subroutine. import_economics shells out -# to tooling/generate-brands as part of its normal flow (see economy_import/ +# Rust sources for the generate_brands subroutine. import_economics runs the +# generate_brands binary as part of its normal flow (see economy_import/ # brands.py), so all of these contribute to its effective source SHA: any change # to them must invalidate the meta stamp even though Python hasn't changed. GENERATE_BRANDS_RS: Path = ( @@ -64,7 +64,11 @@ GENERATE_BRANDS_NAMES_RS: Path = ( GENERATE_BRANDS_SURNAMES_RS: Path = ( REPO_ROOT / "server" / "src" / "bin" / "shared" / "surname_corpus.rs" ) -GENERATE_BRANDS_WRAPPER: Path = REPO_ROOT / "tooling" / "generate-brands" +# The `tooling/generate-brands` bash wrapper used to be stamped here. It was one +# of three identical copies of build-if-missing-then-exec and was retired into +# core.process.cargo_binary (T-1286); the seed argument it carried now lives in +# economy_import/brands.py, which is already stamped via the package directory +# below. Nothing about the brand output stopped being covered. # D-237 authored specialization layer: these data TOMLs feed the DB, so a change # to either must flip the stamp and force a regen (#1013). @@ -170,7 +174,7 @@ IMPORT_ECONOMICS_SOURCES: tuple[Path, ...] = ( GENERATE_BRANDS_RS, GENERATE_BRANDS_NAMES_RS, GENERATE_BRANDS_SURNAMES_RS, - GENERATE_BRANDS_WRAPPER, + REPO_ROOT / "tooling" / "core" / "process.py", # runs the brand binary (T-1286) REPO_ROOT / "tooling" / "schema_version.py", SPECIALIZATION_VOCAB_TOML, SYSTEM_SPECIALIZATION_TOML, diff --git a/tooling/install-godot b/tooling/install-godot deleted file mode 100755 index a8e189f60..000000000 --- a/tooling/install-godot +++ /dev/null @@ -1,108 +0,0 @@ -#!/usr/bin/env bash -# tooling/install-godot — Download and install Godot to ~/bin/ -# -# Usage: -# GODOT_VERSION=4.6 ./tooling/install-godot -# ./tooling/install-godot # uses default version -# -# Installs the Godot binary to ~/bin/godot4. -# Skips download if correct version is already installed. - -set -euo pipefail - -GODOT_VERSION="${GODOT_VERSION:-4.6}" -INSTALL_DIR="$HOME/bin" -BINARY="$INSTALL_DIR/godot4" - -# Check if already installed at the right version -if [[ -x "$BINARY" ]]; then - INSTALLED="$("$BINARY" --version 2>/dev/null | head -1 | cut -d. -f1-2)" || true - if [[ "$INSTALLED" == "$GODOT_VERSION" ]]; then - echo "Godot $GODOT_VERSION already installed at $BINARY" - exit 0 - fi - echo "Godot found but version is $INSTALLED, want $GODOT_VERSION" -fi - -# Detect platform and architecture -OS="$(uname -s)" -ARCH="$(uname -m)" - -case "$OS" in - Linux) - case "$ARCH" in - x86_64) PLATFORM="linux.x86_64" ;; - aarch64) PLATFORM="linux.arm64" ;; - *) echo "Error: unsupported architecture: $ARCH"; exit 1 ;; - esac - ;; - Darwin) - PLATFORM="macos.universal" - ;; - *) - echo "Error: unsupported OS: $OS" - echo "Install Godot manually: https://godotengine.org/download" - exit 1 - ;; -esac - -FILENAME="Godot_v${GODOT_VERSION}-stable_${PLATFORM}.zip" -URL="https://github.com/godotengine/godot/releases/download/${GODOT_VERSION}-stable/${FILENAME}" - -echo "Downloading Godot $GODOT_VERSION for $PLATFORM..." -echo " $URL" - -mkdir -p "$INSTALL_DIR" - -# Download to temp directory -TMPDIR="$(mktemp -d)" -trap 'rm -rf "$TMPDIR"' EXIT - -if ! curl -fSL --progress-bar -o "$TMPDIR/$FILENAME" "$URL"; then - echo "Error: download failed. Check GODOT_VERSION=$GODOT_VERSION is a valid release." - echo "Available releases: https://github.com/godotengine/godot/releases" - exit 1 -fi - -# Extract -echo "Extracting..." -if ! command -v unzip >/dev/null 2>&1; then - echo "Error: unzip is required but not found. Install it and retry." - exit 1 -fi -unzip -q -o "$TMPDIR/$FILENAME" -d "$TMPDIR/extract" - -# Install binary -case "$OS" in - Linux) - SRC="$TMPDIR/extract/Godot_v${GODOT_VERSION}-stable_${PLATFORM}" - if [[ -f "$SRC" ]]; then - chmod +x "$SRC" - mv "$SRC" "$BINARY" - else - echo "Error: expected binary not found after extraction" - ls -la "$TMPDIR/extract/" - exit 1 - fi - ;; - Darwin) - SRC="$TMPDIR/extract/Godot.app/Contents/MacOS/Godot" - if [[ -f "$SRC" ]]; then - mv "$SRC" "$BINARY" - chmod +x "$BINARY" - else - echo "Error: expected Godot.app binary not found after extraction" - ls -la "$TMPDIR/extract/" - exit 1 - fi - ;; -esac - -# Verify -if "$BINARY" --version >/dev/null 2>&1; then - echo "Godot $GODOT_VERSION installed to $BINARY" - "$BINARY" --version -else - echo "Warning: installed to $BINARY but --version check failed." - echo "This may be a display server issue (expected in headless environments)." -fi diff --git a/tooling/install-rust b/tooling/install-rust deleted file mode 100755 index d8dd7ff98..000000000 --- a/tooling/install-rust +++ /dev/null @@ -1,33 +0,0 @@ -#!/usr/bin/env bash -# tooling/install-rust — Install Rust via rustup and add required components -# -# Usage: -# ./tooling/install-rust -# -# Installs rustup + stable toolchain if not present, then ensures -# clippy and rustfmt components are installed. Idempotent. - -set -euo pipefail - -if command -v rustup >/dev/null 2>&1; then - echo "Rust already installed: $(rustc --version)" - rustup component add clippy rustfmt 2>/dev/null - echo "Components verified: clippy, rustfmt" - exit 0 -fi - -echo "Installing Rust via rustup..." - -if ! command -v curl >/dev/null 2>&1; then - echo "Error: curl is required to install Rust." - exit 1 -fi - -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --component clippy rustfmt - -# Source cargo env for the rest of this script -# shellcheck source=/dev/null -. "$HOME/.cargo/env" - -echo "Rust installed: $(rustc --version)" -echo "Components: clippy, rustfmt" diff --git a/tooling/main.py b/tooling/main.py index 2af3be977..3d62a84d0 100644 --- a/tooling/main.py +++ b/tooling/main.py @@ -67,6 +67,14 @@ DOMAINS: dict[str, tuple[str, str]] = { "tooling.domains.visual.router:app", "Compare captures against goldens, and catch blank ones", ), + "generate": ( + "tooling.domains.generate.router:app", + "Producers — brands, corporations, manifests and stubs", + ), + "pr": ( + "tooling.domains.pr.router:app", + "The review loop — comments, watchlist and the board", + ), "jobs": ( "tooling.domains.jobs.router:app", "Detached runs — status, logs and outcomes", diff --git a/tooling/pr-watchlist-diff b/tooling/pr-watchlist-diff deleted file mode 100755 index 42ac88d56..000000000 --- a/tooling/pr-watchlist-diff +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash -# tooling/pr-watchlist-diff <base> <head> — list watch-list files changed -# between <base> and <head>. -# -# Used by /pr-process step 4a (T-858) to decide whether `make regen-db` must -# run before push. The stamped generator sources come from the shared -# registry tooling/generator_sources.py (T-1067) — imported live here so this -# list can't drift from the stamp writer / tooling/check-systems-db-stamp. -# -# The extra hardcoded paths below are non-stamped watch items: the surviving -# one-time planet-gen importers (import_heightmaps.py, import_province_ -# boundaries.py — not part of `make regen-db`, but their data feeds the -# committed DB), the schema DDL (stamped separately via schema_sha), and the -# wiki data directories that feed the generators. -set -euo pipefail - -BASE="${1:?usage: tooling/pr-watchlist-diff <base> <head>}" -HEAD="${2:?usage: tooling/pr-watchlist-diff <base> <head>}" - -# Load the registry via plain assignment (set -e sees its failure), not -# `mapfile < <(...)` — a failed process substitution is invisible to set -e -# and would silently yield an empty watch list, disabling the DB-staleness -# net exactly when the shared registry breaks. Guard the empty case too. -SOURCES_RAW="$(python3 tooling/generator_sources.py --list)" -if [ -z "$SOURCES_RAW" ]; then - echo "pr-watchlist-diff: generator_sources.py --list returned nothing" >&2 - exit 1 -fi -mapfile -t GENERATOR_SOURCES <<< "$SOURCES_RAW" - -git diff --name-only "$BASE...$HEAD" -- \ - "${GENERATOR_SOURCES[@]}" \ - tooling/planet-gen/import_heightmaps.py \ - tooling/planet-gen/import_province_boundaries.py \ - server/data/systems-schema.sql \ - wiki/star-systems/ \ - wiki/economics/ diff --git a/tooling/tea-comment b/tooling/tea-comment deleted file mode 100755 index a63d13729..000000000 --- a/tooling/tea-comment +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -# Post a comment to a Gitea PR or issue. -# Usage: tea-comment <number> "comment body" -# tea-comment <number> @/path/to/file.md -set -euo pipefail - -if [[ $# -lt 2 ]]; then - echo "Usage: tea-comment <number> <comment|@filepath>" >&2 - exit 1 -fi - -NUMBER="$1" -INPUT="$2" - -# Resolve tea: PATH first, then the known linuxbrew keg (agent shells often -# miss the brew shellenv — clide FR-1). -TEA="$(command -v tea || true)" -[[ -n "$TEA" ]] || TEA="/home/linuxbrew/.linuxbrew/bin/tea" -if [[ ! -x "$TEA" ]]; then - echo "Error: tea CLI not found (PATH or /home/linuxbrew/.linuxbrew/bin)" >&2 - exit 1 -fi - -# If the body starts with @, read from file -if [[ "$INPUT" == @* ]]; then - FILEPATH="${INPUT#@}" - if [[ ! -f "$FILEPATH" ]]; then - echo "Error: file not found: $FILEPATH" >&2 - exit 1 - fi - BODY=$(cat "$FILEPATH") -else - BODY="$INPUT" -fi - -HASH=$(echo -n "$BODY" | md5sum | cut -c1-8) -TMPFILE="/tmp/tea-comment-${NUMBER}-${HASH}.md" -trap 'rm -f "$TMPFILE"' EXIT - -printf '%s' "$BODY" > "$TMPFILE" -"$TEA" comment --login schweitz --repo jpmschweitzer/settled-reach "$NUMBER" "$(cat "$TMPFILE")" diff --git a/tooling/test_conformance.py b/tooling/test_conformance.py index de6b05d28..1c96a572a 100644 --- a/tooling/test_conformance.py +++ b/tooling/test_conformance.py @@ -167,9 +167,14 @@ def check_one_guarded_exec(failures: list[str]) -> None: "missing-binary message and the failure remedy live" ) for node in ast.walk(tree): + # The receiver has to be checked, not just the attribute name: + # `platform.system()` is a legitimate call and shares a name with + # `os.system()`. A check that fires on the wrong thing gets muted. if ( isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) + and isinstance(node.func.value, ast.Name) + and node.func.value.id == "os" and node.func.attr in {"system", "popen", "execv", "execvp"} ): failures.append( diff --git a/tooling/test_environment.py b/tooling/test_environment.py new file mode 100644 index 000000000..63cf78400 --- /dev/null +++ b/tooling/test_environment.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 +"""The `reach dev` environment decisions, exercised without performing them (T-1286). + +`install-godot`, `install-rust` and `worktree-setup` were the three scripts the +CLI port stood to gain least from and risked most on. Nothing about them can be +checked by running them: a passing test would download a 60 MB archive, mutate +`~/bin`, or leave a git worktree behind. So the port split each one into a pure +decision (`godot_plan`, `worktree_plan`) and a thin performing half, and this +file pins the decisions. + +That split is the whole claim of the port, which is why it gets a test rather +than a manual `--plan` run: an installer that cannot be tested has to be +trusted instead, and trusting an installer is how a working environment becomes +an unreproducible one. + +Run: python3 tooling/test_environment.py +""" + +from __future__ import annotations + +import os +import sys +import tempfile +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(REPO_ROOT)) + +from tooling.core.errors import ReachError # noqa: E402 +from tooling.domains.dev import environment # noqa: E402 + + +def test_plan_names_the_pinned_version(failures: list[str]) -> None: + """The default plan targets DEFAULT_GODOT_VERSION and builds a real URL.""" + plan = environment.godot_plan() + if plan.wanted != environment.DEFAULT_GODOT_VERSION: + failures.append( + f"godot_plan() wanted {plan.wanted}, expected the pinned " + f"{environment.DEFAULT_GODOT_VERSION}" + ) + if plan.wanted not in plan.url or plan.filename not in plan.url: + failures.append(f"the URL does not carry version and filename: {plan.url}") + if not plan.url.startswith("https://"): + failures.append(f"the download URL is not https: {plan.url}") + + +def test_explicit_version_overrides_the_pin(failures: list[str]) -> None: + plan = environment.godot_plan("4.9") + if plan.wanted != "4.9" or "4.9-stable" not in plan.url: + failures.append(f"an explicit version did not reach the URL: {plan.url}") + + +def test_env_var_overrides_the_pin(failures: list[str]) -> None: + """GODOT_VERSION is how make setup passes the version through.""" + before = os.environ.get("GODOT_VERSION") + os.environ["GODOT_VERSION"] = "4.7" + try: + if environment.godot_plan().wanted != "4.7": + failures.append("GODOT_VERSION was ignored by godot_plan()") + # An explicit argument still wins over the environment. + if environment.godot_plan("4.8").wanted != "4.8": + failures.append("an explicit version lost to GODOT_VERSION") + finally: + if before is None: + del os.environ["GODOT_VERSION"] + else: + os.environ["GODOT_VERSION"] = before + + +def test_already_current_is_decided_not_performed(failures: list[str]) -> None: + """already_current compares installed against wanted — the skip decision.""" + real = environment.godot_plan() + same = type(real)( + wanted="4.6", installed="4.6", platform_tag=real.platform_tag, + url=real.url, filename=real.filename, + ) + differs = type(real)( + wanted="4.6", installed="4.5", platform_tag=real.platform_tag, + url=real.url, filename=real.filename, + ) + absent = type(real)( + wanted="4.6", installed=None, platform_tag=real.platform_tag, + url=real.url, filename=real.filename, + ) + if not same.already_current: + failures.append("a matching installed version was not treated as current") + if differs.already_current: + failures.append("a mismatched version was treated as current — install skipped") + if absent.already_current: + failures.append("a missing install was treated as current — install skipped") + + +def test_unsupported_platform_names_a_remedy(failures: list[str]) -> None: + """An unknown platform must fail loudly, not build a URL that 404s.""" + saved = dict(environment.PLATFORMS) + environment.PLATFORMS.clear() + try: + environment.godot_plan() + failures.append("an unsupported platform produced a plan instead of an error") + except ReachError as exc: + if not exc.fix: + failures.append("the unsupported-platform error carries no fix") + finally: + environment.PLATFORMS.update(saved) + + +def test_worktree_plan_targets_the_convention(failures: list[str]) -> None: + """D-221: worktrees live at <root>/.worktrees/<branch>.""" + target = environment.worktree_plan("some-unused-branch-name") + expected = environment.config.repo_root() / ".worktrees" / "some-unused-branch-name" + if target != expected: + failures.append(f"worktree_plan gave {target}, expected {expected}") + + +def test_worktree_plan_refuses_an_existing_tree(failures: list[str]) -> None: + """Silently reusing a directory is how two branches share one tree.""" + root = environment.config.repo_root() + with tempfile.TemporaryDirectory(dir=root / ".worktrees") as existing: + name = Path(existing).name + try: + environment.worktree_plan(name) + failures.append("worktree_plan accepted a branch whose directory exists") + except ReachError as exc: + if not exc.fix: + failures.append("the existing-worktree error carries no fix") + + +def main() -> int: + # .worktrees/ must exist for the collision test to place a directory in it. + (environment.config.repo_root() / ".worktrees").mkdir(exist_ok=True) + + failures: list[str] = [] + test_plan_names_the_pinned_version(failures) + test_explicit_version_overrides_the_pin(failures) + test_env_var_overrides_the_pin(failures) + test_already_current_is_decided_not_performed(failures) + test_unsupported_platform_names_a_remedy(failures) + test_worktree_plan_targets_the_convention(failures) + test_worktree_plan_refuses_an_existing_tree(failures) + + if failures: + print("test_environment: FAIL", file=sys.stderr) + for failure in failures: + print(f" - {failure}", file=sys.stderr) + return 1 + print( + "test_environment: OK — version pin, overrides, skip decision, " + "platform refusal and worktree placement, none of them performed" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tooling/worktree-setup b/tooling/worktree-setup deleted file mode 100755 index 9e73686f5..000000000 --- a/tooling/worktree-setup +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env bash -# worktree-setup — create a git worktree that's immediately usable. -# -# Collapses the /whats-next §3c ritual into one command. A bare -# `git worktree add` leaves two things broken that bit the Layer-5 batch: -# 1. no `.venv` in the worktree, so `make test-tooling` / `make regen-db` / -# any python tooling can't find `.venv/bin/python`; -# 2. (fixed separately in .config/hooks/post-checkout) pql resolving the -# worktree's `.git` *file* to the MAIN checkout — the post-checkout hook -# now passes `--vault`, so this script relies on it for the pql rebuild. -# -# This script adds the worktree under the repo's gitignored `.worktrees/` -# (never the parent dir — outside the permission sandbox), symlinks the venv, -# and prints the worktree path + the in-worktree reminders. -# -# Usage: tooling/worktree-setup <branch-name> [<start-point>] -# (run from the MAIN checkout, not a linked worktree) -set -euo pipefail - -branch="${1:?usage: tooling/worktree-setup <branch-name> [<start-point>]}" -start="${2:-HEAD}" - -repo_root="$(git rev-parse --show-toplevel)" -if [ -f "$repo_root/.git" ]; then - echo "worktree-setup: run this from the MAIN checkout, not a worktree" >&2 - exit 1 -fi - -wt_dir="$repo_root/.worktrees/$branch" -if [ -e "$wt_dir" ]; then - echo "worktree-setup: $wt_dir already exists — reuse it or 'git worktree remove' it first" >&2 - exit 1 -fi - -# The post-checkout hook (.config/hooks/post-checkout) fires here and rebuilds -# the new worktree's pql.db via `pql --vault <worktree>`. -git worktree add "$wt_dir" -b "$branch" "$start" - -# Symlink the venv — worktrees don't copy it and a fresh per-worktree venv is -# wasteful (the main one is identical). Makes `.venv/bin/python` resolve so the -# Makefile's VENV_PY and every python tool work inside the worktree. -if [ -d "$repo_root/.venv" ] && [ ! -e "$wt_dir/.venv" ]; then - ln -s "$repo_root/.venv" "$wt_dir/.venv" - echo " linked .venv -> $repo_root/.venv" -fi - -echo "worktree ready: $wt_dir" -echo " reminders: pql in this worktree needs --vault \"$wt_dir\" (FR-4);" -echo " tea commands run from the main checkout (go-git can't read a linked worktree)."