refactor(tooling): T-1286 — generate, pr and dev become reach domains
Twelve scripts retired, three domains registered. `reach` now covers nine. generate: `generate-brands` and `generate-corporations` were the second and third copies of the same 24-line build-if-missing-then-exec bash `tooling/atlas` carried, so they collapsed into `core.process.cargo_binary` rather than being ported. `import_economics` shelled out to the first of those, so it now calls that helper — `generated_brands.toml` comes back byte-identical, and the stamp registry swaps the retired wrapper for `core/process.py`. pr: `watchlist-diff` derives its watched set from `generator_sources.py` instead of restating it, so it cannot drift from the stamp check. dev: the environment scripts split decision from performing, per D-263's guarded-exec rule. `godot_plan()` and `worktree_plan()` decide what would happen; `install_godot()`, `install_rust()` and `setup_worktree()` do it. `tooling/test_environment.py` pins the version pin, both override precedences, the already-current skip, the platform refusal and both worktree refusals — none of them performed. `make setup` now installs reach first, since the targets that install rust and godot are reach verbs. Two live bugs found while porting: - The clerk read its decision index from `decisions/README.md`, a path that stopped existing when the DQR tree moved to `governance/`. Every clerk agent has been grepping blind; its prompt pointed at the same dead directory. - The conformance exec-check matched any `x.system()` regardless of receiver, so `platform.system()` read as `os.system()`. Narrowed and re-proved against a real mutant. `process.run` gains `input=`, `timeout=` and a `ProcessTimeout` subclass so a killed run stays distinguishable from a verdict. The pre-push hook no longer merges the clerk's stderr into its stdout — under streaming the last merged line is a JSONL event, which would read as an unrecognised verdict and block. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
+3
-3
@@ -95,9 +95,9 @@ invented.
|
||||
| `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** |
|
||||
| `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` |
|
||||
| `godot` | Godot parse and cold-start checks | `godot-parse-sweep`, `godot-cold-parse` |
|
||||
| `generate` | content generators not owned elsewhere | `generate-brands`, `generate-corporations`, `generate-character-manifest`, `generate_corp_stubs.py` |
|
||||
| `dev` | developer environment and workflow | `install-rust`, `install-godot`, `worktree-setup`, `perf-baseline`, `clerk-review` |
|
||||
| `pr` | the PR/review loop | `tea-comment`, `pr-watchlist-diff`, `pql-board-html` |
|
||||
| `generate` | content generators not owned elsewhere | ✅ ported (T-1286). `generate-brands` + `generate-corporations` collapsed into `core.process.cargo_binary` — they were the same 24 lines of bash a third time |
|
||||
| `dev` | developer environment and workflow | ✅ ported (T-1286). The three environment scripts split decision from performing — `godot_plan`/`worktree_plan` are pure and pinned by `test_environment.py` |
|
||||
| `pr` | the PR/review loop | ✅ ported (T-1286). `watchlist-diff` now reads the watched set from `generator_sources.py` instead of restating it |
|
||||
| `blender` | **carve-out** — payloads run by Blender | **35** `blender_*.py` (13 top-level + 22 in `garment-fit/`), `blender` wrapper → `tooling/scripts/blender/` |
|
||||
|
||||
## Judgment calls, with reasons
|
||||
|
||||
@@ -63,6 +63,16 @@ def meta_path(job_id: str) -> Path:
|
||||
return jobs_dir() / f"{job_id}.json"
|
||||
|
||||
|
||||
class ProcessTimeout(ReachError):
|
||||
"""A run that exceeded its `timeout=`.
|
||||
|
||||
Its own class because a timeout is not a failure verdict — a caller that
|
||||
treats "the tool ran and said no" and "the tool never answered" alike will
|
||||
turn a slow machine into a rejection. Uncaught it still reads as a
|
||||
`ReachError` with a remedy.
|
||||
"""
|
||||
|
||||
|
||||
def run(
|
||||
argv: list[str],
|
||||
*,
|
||||
@@ -70,6 +80,8 @@ def run(
|
||||
env: dict[str, str] | None = None,
|
||||
capture: bool = True,
|
||||
check: bool = True,
|
||||
input: str | None = None,
|
||||
timeout: float | None = None,
|
||||
fix: str | None = None,
|
||||
missing_fix: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
@@ -108,12 +120,20 @@ def run(
|
||||
capture_output=capture,
|
||||
text=True,
|
||||
shell=False,
|
||||
input=input,
|
||||
timeout=timeout,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
raise ReachError(
|
||||
f"{argv[0]} is not installed or not on PATH",
|
||||
fix=missing_fix or f"install {argv[0]}, or check PATH in a non-interactive shell",
|
||||
) from exc
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise ProcessTimeout(
|
||||
f"{argv[0]} did not finish within {timeout:g}s",
|
||||
fix=f"re-run, or raise the timeout — `{' '.join(argv)}` was killed, "
|
||||
"which says nothing about whether it would have succeeded",
|
||||
) from exc
|
||||
|
||||
if check and result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout or "").strip()
|
||||
@@ -126,6 +146,47 @@ def run(
|
||||
return result
|
||||
|
||||
|
||||
def cargo_binary(name: str, *args: str, capture: bool = True) -> str:
|
||||
"""Run a Rust binary from `server/`, building it first if it is absent.
|
||||
|
||||
Three scripts wrote this by hand — `tooling/atlas`, `generate-brands`,
|
||||
`generate-corporations` — each ~24 lines of identical bash: check for the
|
||||
debug binary, `cargo build --bin` it if missing, `exec` it with every
|
||||
argument. Three copies of one idea is a substrate, so it lives here.
|
||||
|
||||
Build-if-missing is kept because it is genuinely useful on a cold checkout,
|
||||
but it is announced rather than silent: a first run that takes ninety
|
||||
seconds with no explanation reads as a hang.
|
||||
"""
|
||||
binary = config.path("server", "target", "debug", name)
|
||||
if not binary.is_file():
|
||||
from tooling.core import console
|
||||
|
||||
console.event(f"building the {name} binary (first run)", level="warn")
|
||||
build = run(
|
||||
["cargo", "build", "--bin", name],
|
||||
cwd=config.path("server"),
|
||||
check=False,
|
||||
missing_fix="install Rust — make setup-rust",
|
||||
)
|
||||
if build.returncode != 0:
|
||||
raise ReachError(
|
||||
f"could not build {name}\n{(build.stderr or '').strip()}",
|
||||
fix=f"cd server && cargo build --bin {name} — for the full error",
|
||||
exit_code=build.returncode,
|
||||
)
|
||||
|
||||
result = run([str(binary), *args], check=False, capture=capture)
|
||||
if result.returncode != 0:
|
||||
raise ReachError(
|
||||
f"{name} {' '.join(args)} exited {result.returncode}\n"
|
||||
+ (result.stderr or "").strip(),
|
||||
fix=f"run `{binary} {' '.join(args)}` directly for the full output",
|
||||
exit_code=result.returncode,
|
||||
)
|
||||
return result.stdout or ""
|
||||
|
||||
|
||||
def spawn_detached(argv: list[str]) -> str:
|
||||
"""Run `reach <argv>` in a detached child. Returns the job id immediately.
|
||||
|
||||
|
||||
@@ -11,15 +11,18 @@ what exists, and an index that says "ask the binary" is not one. The cost is
|
||||
that a subcommand added on the Rust side is invisible here until someone adds a
|
||||
line — so `run()` also accepts anything, and an unknown verb reaches the binary
|
||||
rather than being rejected by a list that has fallen behind.
|
||||
|
||||
The build-if-missing and invoke machinery moved to `core.process.cargo_binary`
|
||||
once `generate-brands` and `generate-corporations` turned out to be the same
|
||||
24 lines of bash. Three copies of one idea is a substrate.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from tooling.core import config, console, process
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.core import process
|
||||
|
||||
# What the binary offers today. Discovered from the wrapper's own usage text and
|
||||
# from atlas-commit-and-sync, which calls three verbs the wrapper never
|
||||
# from atlas-commit-and-sync, which calls four verbs the wrapper never
|
||||
# documented — list-stations, wipe-system, commit-system, sync-wiki.
|
||||
KNOWN_VERBS = (
|
||||
"stats",
|
||||
@@ -33,34 +36,6 @@ KNOWN_VERBS = (
|
||||
)
|
||||
|
||||
|
||||
def binary_path():
|
||||
return config.path("server", "target", "debug", "atlas")
|
||||
|
||||
|
||||
def run(*args: str, capture: bool = True) -> str:
|
||||
"""Invoke the Rust atlas binary, building it first if it is absent."""
|
||||
binary = binary_path()
|
||||
if not binary.is_file():
|
||||
console.event("building the atlas binary (first run)", level="warn")
|
||||
build = process.run(
|
||||
["cargo", "build", "--bin", "atlas"],
|
||||
cwd=config.path("server"),
|
||||
check=False,
|
||||
missing_fix="install Rust — make setup-rust",
|
||||
)
|
||||
if build.returncode != 0:
|
||||
raise ReachError(
|
||||
f"could not build the atlas binary\n{(build.stderr or '').strip()}",
|
||||
fix="cd server && cargo build --bin atlas — to see the full error",
|
||||
exit_code=build.returncode,
|
||||
)
|
||||
|
||||
result = process.run([str(binary), *args], check=False, capture=capture)
|
||||
if result.returncode != 0:
|
||||
raise ReachError(
|
||||
f"atlas {' '.join(args)} exited {result.returncode}\n"
|
||||
+ (result.stderr or "").strip(),
|
||||
fix=f"run `{binary} {' '.join(args)}` directly for the full output",
|
||||
exit_code=result.returncode,
|
||||
)
|
||||
return result.stdout or ""
|
||||
return process.cargo_binary("atlas", *args, capture=capture)
|
||||
|
||||
@@ -40,20 +40,20 @@ Env knobs:
|
||||
SR_CLERK_TIMEOUT per-commit timeout seconds (default 300)
|
||||
|
||||
Usage:
|
||||
tooling/clerk-review run the review, print verdict
|
||||
tooling/clerk-review --plan print the per-commit plan only (no clerk spawned)
|
||||
reach dev clerk run the review, print verdict
|
||||
reach dev clerk --plan print the per-commit plan only (no clerk spawned)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(subprocess.check_output(
|
||||
["git", "rev-parse", "--show-toplevel"], text=True
|
||||
).strip())
|
||||
from tooling.core import config, console, process
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
REPO_ROOT = config.repo_root()
|
||||
|
||||
CACHE_DIR = REPO_ROOT / ".cache"
|
||||
FINDINGS_FILE = CACHE_DIR / "pre-push-review.md"
|
||||
@@ -73,7 +73,7 @@ CLERK_PROMPT = """You are the CLERK, the institutional guardrail for The Settled
|
||||
You are reviewing ONE COMMIT ({label}) from a larger pre-push for D-record
|
||||
consistency, ticket drift, and decision contradictions.
|
||||
|
||||
## decisions/ domain index
|
||||
## governance/ domain index
|
||||
|
||||
{index}
|
||||
|
||||
@@ -83,7 +83,7 @@ consistency, ticket drift, and decision contradictions.
|
||||
|
||||
## Your task
|
||||
|
||||
Be efficient — a few targeted greps of decisions/*.md, then conclude. Check:
|
||||
Be efficient — a few targeted greps of governance/decisions/*.md, then conclude. Check:
|
||||
- Does this commit contradict any active D-record?
|
||||
- If the commit's code/text cites a D/Q/R-ID, does that ID exist and is it active?
|
||||
- If the commit message cites ticket #NNN, does the change match the ticket?
|
||||
@@ -101,29 +101,25 @@ APPROVED or REJECTED.
|
||||
"""
|
||||
|
||||
|
||||
def base_range():
|
||||
def _git(*args: str) -> str:
|
||||
return process.run(["git", *args], cwd=REPO_ROOT).stdout
|
||||
|
||||
|
||||
def base_range() -> str:
|
||||
"""The commit range that would be pushed (base..HEAD)."""
|
||||
branch = subprocess.check_output(
|
||||
["git", "branch", "--show-current"], text=True
|
||||
).strip()
|
||||
branch = _git("branch", "--show-current").strip()
|
||||
for ref in [f"origin/{branch}", "origin/main"]:
|
||||
try:
|
||||
subprocess.check_output(
|
||||
["git", "rev-parse", "--verify", ref],
|
||||
stderr=subprocess.DEVNULL, text=True,
|
||||
)
|
||||
probe = process.run(
|
||||
["git", "rev-parse", "--verify", ref], cwd=REPO_ROOT, check=False
|
||||
)
|
||||
if probe.returncode == 0:
|
||||
return f"{ref}..HEAD"
|
||||
except subprocess.CalledProcessError:
|
||||
continue
|
||||
return "HEAD~1..HEAD"
|
||||
|
||||
|
||||
def list_commits(rng):
|
||||
def list_commits(rng: str) -> list[str]:
|
||||
"""SHAs in the push range, oldest first."""
|
||||
out = subprocess.check_output(
|
||||
["git", "rev-list", "--reverse", rng], text=True
|
||||
)
|
||||
return [s for s in out.splitlines() if s.strip()]
|
||||
return [s for s in _git("rev-list", "--reverse", rng).splitlines() if s.strip()]
|
||||
|
||||
|
||||
def commit_unit(sha):
|
||||
@@ -132,16 +128,10 @@ def commit_unit(sha):
|
||||
`skip` is True when the commit message has a `Clerk-Skip:` trailer (the safety
|
||||
valve); `text` is the message + diff, truncated to COMMIT_BUDGET.
|
||||
"""
|
||||
subject = subprocess.check_output(
|
||||
["git", "show", "-s", "--format=%h %s", sha], text=True
|
||||
).strip()
|
||||
message = subprocess.check_output(
|
||||
["git", "show", "-s", "--format=%B", sha], text=True
|
||||
)
|
||||
subject = _git("show", "-s", "--format=%h %s", sha).strip()
|
||||
message = _git("show", "-s", "--format=%B", sha)
|
||||
skip = bool(SKIP_TRAILER.search(message))
|
||||
text = subprocess.check_output(
|
||||
["git", "show", "--format=fuller", sha], text=True
|
||||
)
|
||||
text = _git("show", "--format=fuller", sha)
|
||||
if len(text) > COMMIT_BUDGET:
|
||||
text = text[:COMMIT_BUDGET] + f"\n\n... (commit diff truncated; {len(text)} total chars)\n"
|
||||
return subject, text, skip
|
||||
@@ -155,13 +145,16 @@ def run_clerk(label, diff_text, index):
|
||||
"""
|
||||
prompt = CLERK_PROMPT.format(label=label, index=index, diff=diff_text)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
result = process.run(
|
||||
["claude", "-p", "--model", "sonnet", "--max-turns", str(MAX_TURNS)],
|
||||
input=prompt, capture_output=True, text=True,
|
||||
timeout=TIMEOUT_SECONDS, cwd=str(REPO_ROOT),
|
||||
cwd=REPO_ROOT,
|
||||
input=prompt,
|
||||
timeout=TIMEOUT_SECONDS,
|
||||
check=False,
|
||||
missing_fix="install the claude CLI, or set SR_CLERK=0 to skip the review",
|
||||
)
|
||||
output = result.stdout.strip()
|
||||
except subprocess.TimeoutExpired:
|
||||
except process.ProcessTimeout:
|
||||
return "INCOMPLETE", f"{label}: review timed out after {TIMEOUT_SECONDS}s (not a contradiction)."
|
||||
|
||||
if not output:
|
||||
@@ -175,39 +168,49 @@ def run_clerk(label, diff_text, index):
|
||||
return "INCOMPLETE", output + "\n\n(No clear verdict on last line — recorded as INCOMPLETE, not a contradiction.)"
|
||||
|
||||
|
||||
def load_index():
|
||||
readme = REPO_ROOT / "decisions" / "README.md"
|
||||
return readme.read_text()[:8000] if readme.exists() else "(decisions/README.md not found)"
|
||||
def load_index() -> str:
|
||||
"""The decision-domain index the clerk greps from.
|
||||
|
||||
`governance/README.md` since the DQR tree moved; the old `decisions/README.md`
|
||||
path silently resolved to "not found", which handed every clerk an empty
|
||||
index and made it grep blind.
|
||||
"""
|
||||
readme = REPO_ROOT / "governance" / "README.md"
|
||||
return readme.read_text()[:8000] if readme.exists() else "(governance/README.md not found)"
|
||||
|
||||
|
||||
def main():
|
||||
plan_only = "--plan" in sys.argv[1:]
|
||||
def run(plan_only: bool = False) -> str:
|
||||
"""Review the push range commit by commit. Returns the overall verdict."""
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
rng = base_range()
|
||||
commits = list_commits(rng)
|
||||
if not commits:
|
||||
FINDINGS_FILE.write_text("# Clerk Review\n\nNo commits to review.\n\nVerdict: APPROVED\n")
|
||||
print(" clerk: no commits to review", file=sys.stderr)
|
||||
sys.stderr.flush()
|
||||
print("APPROVED")
|
||||
return 0
|
||||
console.event("clerk: no commits to review")
|
||||
console.out("APPROVED")
|
||||
return "APPROVED"
|
||||
|
||||
units = [commit_unit(sha) for sha in commits] # [(subject, text, skip), ...]
|
||||
|
||||
if plan_only:
|
||||
print(f"clerk plan: {len(commits)} commit(s) over {rng} "
|
||||
f"(budget {COMMIT_BUDGET} chars, {WORKERS} workers, {MAX_TURNS} turns each)")
|
||||
console.out(f"clerk plan: {len(commits)} commit(s) over {rng} "
|
||||
f"(budget {COMMIT_BUDGET} chars, {WORKERS} workers, "
|
||||
f"{MAX_TURNS} turns each)")
|
||||
for i, (subject, text, skip) in enumerate(units):
|
||||
tag = "SKIP (Clerk-Skip:)" if skip else "review"
|
||||
print(f" commit {i + 1}/{len(commits)}: {len(text):>9} chars [{tag}] {subject}")
|
||||
return 0
|
||||
console.out(
|
||||
f" commit {i + 1}/{len(commits)}: {len(text):>9} chars [{tag}] {subject}"
|
||||
)
|
||||
return "PLAN"
|
||||
|
||||
index = load_index()
|
||||
reviewable = sum(1 for _, _, skip in units if not skip)
|
||||
print(f" clerk: {len(commits)} commit(s) — {reviewable} to review, "
|
||||
f"{len(commits) - reviewable} auto-approved (Clerk-Skip:); {WORKERS} parallel...",
|
||||
file=sys.stderr)
|
||||
console.event(
|
||||
f"clerk: {len(commits)} commit(s) — {reviewable} to review, "
|
||||
f"{len(commits) - reviewable} auto-approved (Clerk-Skip:); {WORKERS} parallel",
|
||||
phase="clerk",
|
||||
)
|
||||
|
||||
def task(i, subject, text, skip):
|
||||
label = f"commit {i + 1}/{len(commits)} ({subject})"
|
||||
@@ -222,7 +225,7 @@ def main():
|
||||
for fut in as_completed(futures):
|
||||
i, label, verdict, findings = fut.result()
|
||||
results[i] = (label, verdict, findings)
|
||||
print(f" clerk: {label} — {verdict}", file=sys.stderr)
|
||||
console.event(f"clerk: {label} — {verdict}", phase="clerk")
|
||||
|
||||
verdicts = [r[1] for r in results]
|
||||
if "REJECTED" in verdicts:
|
||||
@@ -243,11 +246,15 @@ def main():
|
||||
parts.append(f"\n---\n\n## {label} — {verdict}\n\n{findings}\n")
|
||||
FINDINGS_FILE.write_text("\n".join(parts))
|
||||
|
||||
print(f" clerk: overall verdict — {overall} (details: {FINDINGS_FILE})", file=sys.stderr)
|
||||
sys.stderr.flush()
|
||||
print(overall)
|
||||
return 1 if overall == "REJECTED" else 0
|
||||
console.event(
|
||||
f"clerk: overall verdict — {overall} (details: {FINDINGS_FILE})", phase="clerk"
|
||||
)
|
||||
console.out(overall)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
if overall == "REJECTED":
|
||||
raise ReachError(
|
||||
f"clerk rejected {n_rej} commit(s) — a named, active D-record is contradicted",
|
||||
fix=f"read {FINDINGS_FILE.relative_to(REPO_ROOT)}; amend the commit or "
|
||||
"the decision, or add a `Clerk-Skip:` trailer if the clerk is wrong",
|
||||
)
|
||||
return overall
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Environment setup — the decisions, separated from the performing (D-263).
|
||||
|
||||
These three ported worst and mattered most to get right. `install-godot`
|
||||
downloads and unzips a pinned build, `install-rust` drives `rustup`,
|
||||
`worktree-setup` manipulates git worktrees — none of which can be exercised in
|
||||
a gate without actually doing it.
|
||||
|
||||
So the shape here is deliberate: **every function that decides is pure and
|
||||
importable, and every function that acts is a thin call through
|
||||
`core.process.run`.** `godot_plan()` answers "what would be downloaded, and is
|
||||
it needed" without touching the network; `install_godot()` performs it. The
|
||||
test of a correct port is whether the decisions can be exercised WITHOUT
|
||||
performing them — a rewrite that cannot be tested has to be trusted instead,
|
||||
and trusting an installer is how a working environment becomes an
|
||||
unreproducible one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from tooling.core import config, console, process
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
DEFAULT_GODOT_VERSION = "4.6"
|
||||
INSTALL_DIR = Path.home() / "bin"
|
||||
GODOT_BINARY = INSTALL_DIR / "godot4"
|
||||
|
||||
# Godot publishes one archive per platform triple; an unsupported pair is a
|
||||
# clear failure rather than a download that 404s.
|
||||
PLATFORMS = {
|
||||
("Linux", "x86_64"): "linux.x86_64",
|
||||
("Linux", "aarch64"): "linux.arm64",
|
||||
("Darwin", "x86_64"): "macos.universal",
|
||||
("Darwin", "arm64"): "macos.universal",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GodotPlan:
|
||||
"""What installing Godot would do — decided without doing any of it."""
|
||||
|
||||
wanted: str
|
||||
installed: str | None
|
||||
platform_tag: str
|
||||
url: str
|
||||
filename: str
|
||||
|
||||
@property
|
||||
def already_current(self) -> bool:
|
||||
return self.installed == self.wanted
|
||||
|
||||
|
||||
def godot_plan(wanted: str | None = None) -> GodotPlan:
|
||||
"""Decide what an install would fetch. Pure apart from reading the binary."""
|
||||
wanted = wanted or os.environ.get("GODOT_VERSION", DEFAULT_GODOT_VERSION)
|
||||
system, machine = platform.system(), platform.machine()
|
||||
tag = PLATFORMS.get((system, machine))
|
||||
if tag is None:
|
||||
raise ReachError(
|
||||
f"unsupported platform: {system} {machine}",
|
||||
fix="install Godot manually from https://godotengine.org/download",
|
||||
)
|
||||
|
||||
filename = f"Godot_v{wanted}-stable_{tag}.zip"
|
||||
return GodotPlan(
|
||||
wanted=wanted,
|
||||
installed=installed_godot_version(),
|
||||
platform_tag=tag,
|
||||
filename=filename,
|
||||
url=(
|
||||
"https://github.com/godotengine/godot/releases/download/"
|
||||
f"{wanted}-stable/{filename}"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def installed_godot_version() -> str | None:
|
||||
"""The major.minor already on disk, or None. Never raises."""
|
||||
if not GODOT_BINARY.is_file() or not os.access(GODOT_BINARY, os.X_OK):
|
||||
return None
|
||||
result = process.run([str(GODOT_BINARY), "--version"], check=False)
|
||||
first = (result.stdout or "").splitlines()
|
||||
if not first:
|
||||
return None
|
||||
return ".".join(first[0].split(".")[:2])
|
||||
|
||||
|
||||
def install_godot(wanted: str | None = None) -> GodotPlan:
|
||||
"""Perform the install. The decision is `godot_plan`; this is the doing."""
|
||||
plan = godot_plan(wanted)
|
||||
if plan.already_current:
|
||||
console.verdict(f"Godot {plan.wanted} already installed at {GODOT_BINARY}")
|
||||
return plan
|
||||
if plan.installed:
|
||||
console.event(
|
||||
f"Godot found but version is {plan.installed}, want {plan.wanted}",
|
||||
level="warn",
|
||||
)
|
||||
|
||||
INSTALL_DIR.mkdir(parents=True, exist_ok=True)
|
||||
import tempfile
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
archive = Path(tmp) / plan.filename
|
||||
console.event(f"downloading Godot {plan.wanted} for {plan.platform_tag}", phase="godot")
|
||||
process.run(
|
||||
["curl", "-fSL", "--progress-bar", "-o", str(archive), plan.url],
|
||||
capture=False,
|
||||
fix=f"check that GODOT_VERSION={plan.wanted} is a real release",
|
||||
missing_fix="install curl",
|
||||
)
|
||||
console.event("extracting", phase="godot")
|
||||
process.run(
|
||||
["unzip", "-q", "-o", str(archive), "-d", tmp],
|
||||
fix="the archive may be truncated — re-run to download it again",
|
||||
missing_fix="install unzip",
|
||||
)
|
||||
extracted = next(
|
||||
(p for p in Path(tmp).iterdir() if p.is_file() and p.name.startswith("Godot")),
|
||||
None,
|
||||
)
|
||||
if extracted is None:
|
||||
raise ReachError(
|
||||
"the Godot archive contained no binary",
|
||||
fix="the download may be corrupt — delete it and re-run",
|
||||
)
|
||||
shutil.move(str(extracted), GODOT_BINARY)
|
||||
GODOT_BINARY.chmod(0o755)
|
||||
|
||||
console.verdict(f"Godot {plan.wanted} installed at {GODOT_BINARY}")
|
||||
return plan
|
||||
|
||||
|
||||
def install_rust() -> None:
|
||||
"""Ensure rustup, clippy and rustfmt are present."""
|
||||
if shutil.which("cargo") is None:
|
||||
raise ReachError(
|
||||
"cargo is not installed",
|
||||
fix="install Rust from https://rustup.rs, then re-run — this command "
|
||||
"adds the components but does not bootstrap the toolchain",
|
||||
)
|
||||
for component in ("clippy", "rustfmt"):
|
||||
console.event(f"ensuring {component}", phase="rust")
|
||||
process.run(
|
||||
["rustup", "component", "add", component],
|
||||
check=False,
|
||||
missing_fix="install rustup from https://rustup.rs",
|
||||
)
|
||||
console.verdict("Rust toolchain ready — clippy and rustfmt present")
|
||||
|
||||
|
||||
def worktree_plan(branch: str) -> Path:
|
||||
"""Where a worktree for `branch` would go. Raises if it cannot be created.
|
||||
|
||||
Both refusals are the original's and both are worth keeping: a worktree
|
||||
created from inside a worktree nests confusingly, and silently reusing an
|
||||
existing directory is how two branches end up sharing one tree.
|
||||
"""
|
||||
root = config.repo_root()
|
||||
if (root / ".git").is_file():
|
||||
raise ReachError(
|
||||
"this is a worktree, not the main checkout",
|
||||
fix="run this from the main checkout — nesting worktrees confuses "
|
||||
"both git and the tools that resolve the repo root",
|
||||
)
|
||||
target = root / ".worktrees" / branch
|
||||
if target.exists():
|
||||
raise ReachError(
|
||||
f"{target} already exists",
|
||||
fix=f"reuse it, or remove it first with: git worktree remove {target}",
|
||||
)
|
||||
return target
|
||||
|
||||
|
||||
def setup_worktree(branch: str, start: str = "HEAD") -> Path:
|
||||
"""Create a worktree and link the venv into it."""
|
||||
target = worktree_plan(branch)
|
||||
root = config.repo_root()
|
||||
|
||||
process.run(
|
||||
["git", "worktree", "add", str(target), "-b", branch, start],
|
||||
cwd=root,
|
||||
fix=f"check that {start} is a valid ref and {branch} is not already a branch",
|
||||
)
|
||||
|
||||
venv = root / ".venv"
|
||||
if venv.is_dir() and not (target / ".venv").exists():
|
||||
(target / ".venv").symlink_to(venv)
|
||||
console.event(f"linked .venv -> {venv}")
|
||||
|
||||
console.verdict(
|
||||
f"worktree ready: {target}\n"
|
||||
f" pql in this worktree needs --vault {target} (FR-4)\n"
|
||||
" tea runs from the main checkout — its go-git cannot read a linked worktree\n"
|
||||
" reach follows whichever checkout it was installed from; `make reach-repoint` "
|
||||
"if you want it to follow this one"
|
||||
)
|
||||
return target
|
||||
@@ -1,24 +1,21 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Performance baseline tooling.
|
||||
"""Performance baseline — tick timing, memory, and shadowcast scaling.
|
||||
|
||||
Runs the server benchmark suite, captures tick timing, memory usage, and entity
|
||||
count scaling metrics, outputs results to tests/perf/.
|
||||
|
||||
Usage:
|
||||
tooling/perf-baseline Run benchmarks and save baseline
|
||||
tooling/perf-baseline --compare Compare current run against saved baseline (no save)
|
||||
|
||||
Exit code 0 = success, 1 = failure or regression detected (--compare mode).
|
||||
Builds the server in release, runs the benchmark suite, and either writes
|
||||
tests/perf/baseline.json or compares against the committed one. The compare
|
||||
path exits non-zero on a >20% regression or a p95 over the D-026 tick budget.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
from tooling.core import config, console, process
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
ROOT = config.repo_root()
|
||||
PERF_DIR = ROOT / "tests" / "perf"
|
||||
BASELINE_FILE = PERF_DIR / "baseline.json"
|
||||
|
||||
@@ -29,39 +26,35 @@ TICK_BUDGET_US = 100_000 # 100ms
|
||||
|
||||
def run_command(cmd, **kwargs):
|
||||
"""Run a command in the server directory and return the result."""
|
||||
return subprocess.run(
|
||||
cmd, capture_output=True, text=True, cwd=ROOT / "server", **kwargs
|
||||
)
|
||||
return process.run(cmd, cwd=ROOT / "server", check=False, **kwargs)
|
||||
|
||||
|
||||
def get_git_info():
|
||||
"""Get current git commit and branch."""
|
||||
commit = subprocess.run(
|
||||
["git", "rev-parse", "--short", "HEAD"],
|
||||
capture_output=True, text=True, cwd=ROOT,
|
||||
"""Current commit and branch. check=False — a shallow or detached tree is
|
||||
not an error here, it just means the stamp carries less."""
|
||||
commit = process.run(
|
||||
["git", "rev-parse", "--short", "HEAD"], cwd=ROOT, check=False
|
||||
).stdout.strip()
|
||||
branch = subprocess.run(
|
||||
["git", "rev-parse", "--abbrev-ref", "HEAD"],
|
||||
capture_output=True, text=True, cwd=ROOT,
|
||||
branch = process.run(
|
||||
["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=ROOT, check=False
|
||||
).stdout.strip()
|
||||
return {"commit": commit, "branch": branch}
|
||||
|
||||
|
||||
def run_tick_benchmark():
|
||||
"""Run perf_tick_timing test and parse PERF_RESULT JSON."""
|
||||
print(" Running tick timing benchmark (release mode)...")
|
||||
console.event("running tick timing benchmark (release mode)", phase="bench")
|
||||
result = run_command([
|
||||
"cargo", "test", "--release", "--test", "perf_bench",
|
||||
"--", "--ignored", "--nocapture", "perf_tick_timing",
|
||||
])
|
||||
|
||||
if result.returncode != 0:
|
||||
print(f" FAILED: tick benchmark exited {result.returncode}")
|
||||
console.event(f"tick benchmark exited {result.returncode}", level="error")
|
||||
if result.stderr:
|
||||
# Print last 20 lines of stderr for diagnostics
|
||||
lines = result.stderr.strip().splitlines()
|
||||
for line in lines[-20:]:
|
||||
print(f" {line}")
|
||||
# Last 20 lines of stderr are the diagnostic; the rest is build noise.
|
||||
for line in result.stderr.strip().splitlines()[-20:]:
|
||||
console.event(line, level="error")
|
||||
return None
|
||||
|
||||
# Parse PERF_RESULT: line from stdout
|
||||
@@ -70,20 +63,22 @@ def run_tick_benchmark():
|
||||
json_str = line[len("PERF_RESULT:"):]
|
||||
return json.loads(json_str)
|
||||
|
||||
print(" WARNING: No PERF_RESULT found in test output")
|
||||
console.event("no PERF_RESULT found in test output", level="warn")
|
||||
return None
|
||||
|
||||
|
||||
def run_shadowcast_benchmark():
|
||||
"""Run shadowcast benchmark and parse structured output."""
|
||||
print(" Running shadowcast benchmark (release mode)...")
|
||||
console.event("running shadowcast benchmark (release mode)", phase="bench")
|
||||
result = run_command([
|
||||
"cargo", "test", "--release", "--test", "shadowcast_bench",
|
||||
"--", "--ignored", "--nocapture", "benchmark_symmetric_vs_recursive",
|
||||
])
|
||||
|
||||
if result.returncode != 0:
|
||||
print(f" FAILED: shadowcast benchmark exited {result.returncode}")
|
||||
console.event(
|
||||
f"shadowcast benchmark exited {result.returncode}", level="error"
|
||||
)
|
||||
return None
|
||||
|
||||
configs = []
|
||||
@@ -173,29 +168,28 @@ def compare_baselines(old, new):
|
||||
return regressions, improvements
|
||||
|
||||
|
||||
def main():
|
||||
compare_mode = "--compare" in sys.argv
|
||||
|
||||
print("=== Performance Baseline ===\n")
|
||||
|
||||
# Build in release mode first
|
||||
print("Building server (release)...")
|
||||
def run(compare_mode: bool = False) -> None:
|
||||
"""Build release, run the benchmarks, then write or compare the baseline."""
|
||||
console.event("building server (release)", phase="build")
|
||||
build = run_command(["cargo", "build", "--release"])
|
||||
if build.returncode != 0:
|
||||
print("BUILD FAILED")
|
||||
lines = build.stderr.strip().splitlines()
|
||||
for line in lines[-20:]:
|
||||
print(f" {line}")
|
||||
return 1
|
||||
|
||||
print("\nRunning benchmarks...\n")
|
||||
for line in build.stderr.strip().splitlines()[-20:]:
|
||||
console.event(line, level="error")
|
||||
raise ReachError(
|
||||
"the release build failed, so there is nothing to benchmark",
|
||||
fix="fix the build errors above, then re-run — a perf number from a "
|
||||
"stale binary is worse than no number",
|
||||
)
|
||||
|
||||
tick_results = run_tick_benchmark()
|
||||
shadowcast_results = run_shadowcast_benchmark()
|
||||
|
||||
if not tick_results:
|
||||
print("\nFATAL: tick benchmark failed -- no baseline generated")
|
||||
return 1
|
||||
raise ReachError(
|
||||
"the tick benchmark produced no result — no baseline generated",
|
||||
fix="run `cargo test --release --test perf_bench -- --ignored "
|
||||
"--nocapture perf_tick_timing` in server/ to see why it failed",
|
||||
)
|
||||
|
||||
# Assemble baseline
|
||||
git_info = get_git_info()
|
||||
@@ -211,72 +205,70 @@ def main():
|
||||
|
||||
# Report
|
||||
tt = baseline["tick_timing"]
|
||||
print(f"\n--- Results ---")
|
||||
print(f"Git: {git_info['commit']} ({git_info['branch']})")
|
||||
print(f"Tick timing ({tt.get('measured_ticks', '?')} ticks, "
|
||||
f"{tt.get('warmup_ticks', '?')} warmup):")
|
||||
print(f" min: {tt.get('min_us', '?')}us")
|
||||
print(f" mean: {tt.get('mean_us', '?')}us")
|
||||
print(f" p95: {tt.get('p95_us', '?')}us")
|
||||
print(f" max: {tt.get('max_us', '?')}us")
|
||||
console.out("--- Results ---")
|
||||
console.out(f"Git: {git_info['commit']} ({git_info['branch']})")
|
||||
console.out(f"Tick timing ({tt.get('measured_ticks', '?')} ticks, "
|
||||
f"{tt.get('warmup_ticks', '?')} warmup):")
|
||||
console.out(f" min: {tt.get('min_us', '?')}us")
|
||||
console.out(f" mean: {tt.get('mean_us', '?')}us")
|
||||
console.out(f" p95: {tt.get('p95_us', '?')}us")
|
||||
console.out(f" max: {tt.get('max_us', '?')}us")
|
||||
|
||||
ent = baseline["entities"]
|
||||
print(f"Entities: avg {ent.get('avg_per_snapshot', '?')}, "
|
||||
f"max {ent.get('max_per_snapshot', '?')}")
|
||||
console.out(f"Entities: avg {ent.get('avg_per_snapshot', '?')}, "
|
||||
f"max {ent.get('max_per_snapshot', '?')}")
|
||||
|
||||
mem = baseline["memory"]
|
||||
rss = mem.get("rss_kb")
|
||||
rss = baseline["memory"].get("rss_kb")
|
||||
if rss:
|
||||
print(f"Memory: {rss} KB RSS ({rss / 1024:.1f} MB)")
|
||||
console.out(f"Memory: {rss} KB RSS ({rss / 1024:.1f} MB)")
|
||||
|
||||
if shadowcast_results:
|
||||
n = len(shadowcast_results.get("configs", []))
|
||||
print(f"Shadowcast: {n} configurations benchmarked")
|
||||
console.out(f"Shadowcast: {n} configurations benchmarked")
|
||||
|
||||
# Budget check
|
||||
p95 = tt.get("p95_us", 0)
|
||||
if p95 > TICK_BUDGET_US:
|
||||
print(f"\nBUDGET EXCEEDED: p95 {p95}us > {TICK_BUDGET_US}us (D-026)")
|
||||
console.out(f"BUDGET EXCEEDED: p95 {p95}us > {TICK_BUDGET_US}us (D-026)")
|
||||
else:
|
||||
budget_pct = p95 / TICK_BUDGET_US * 100 if TICK_BUDGET_US else 0
|
||||
print(f"\nBudget: {budget_pct:.1f}% of {TICK_BUDGET_US}us tick budget (D-026)")
|
||||
console.out(
|
||||
f"Budget: {budget_pct:.1f}% of {TICK_BUDGET_US}us tick budget (D-026)"
|
||||
)
|
||||
|
||||
# Compare with previous baseline if it exists
|
||||
regressions: list[str] = []
|
||||
if BASELINE_FILE.exists():
|
||||
with open(BASELINE_FILE) as f:
|
||||
old_baseline = json.load(f)
|
||||
old_baseline = json.loads(BASELINE_FILE.read_text())
|
||||
old_commit = old_baseline.get("git", {}).get("commit", "?")
|
||||
print(f"\n--- Comparison vs {old_commit} ---")
|
||||
console.out(f"--- Comparison vs {old_commit} ---")
|
||||
regressions, improvements = compare_baselines(old_baseline, baseline)
|
||||
for r in regressions:
|
||||
print(f" REGRESSION: {r}")
|
||||
console.out(f" REGRESSION: {r}")
|
||||
for i in improvements:
|
||||
print(f" IMPROVEMENT: {i}")
|
||||
console.out(f" IMPROVEMENT: {i}")
|
||||
if not regressions and not improvements:
|
||||
print(" No significant changes.")
|
||||
if compare_mode and regressions:
|
||||
print(f"\n{len(regressions)} regression(s) detected.")
|
||||
return 1
|
||||
console.out(" No significant changes.")
|
||||
elif compare_mode:
|
||||
print(f"\nERROR: --compare requires a saved baseline at {BASELINE_FILE.relative_to(ROOT)}")
|
||||
print("Run `make perf-baseline` first to create one.")
|
||||
return 1
|
||||
raise ReachError(
|
||||
f"--compare needs a saved baseline at {BASELINE_FILE.relative_to(ROOT)}",
|
||||
fix="run `reach dev perf` once without --compare to create one",
|
||||
)
|
||||
|
||||
if compare_mode:
|
||||
return 0
|
||||
if regressions:
|
||||
raise ReachError(
|
||||
f"{len(regressions)} performance regression(s) detected",
|
||||
fix="see the REGRESSION lines above; if the change is intended, "
|
||||
"re-run without --compare to accept it as the new baseline",
|
||||
)
|
||||
console.verdict("perf: no regressions against the saved baseline")
|
||||
return
|
||||
|
||||
# Save baseline (strip per-tick array — too noisy for git diffs)
|
||||
PERF_DIR.mkdir(parents=True, exist_ok=True)
|
||||
committed = json.loads(json.dumps(baseline))
|
||||
committed["tick_timing"].pop("all_us", None)
|
||||
BASELINE_FILE.write_text(json.dumps(committed, indent=2) + "\n")
|
||||
|
||||
with open(BASELINE_FILE, "w") as f:
|
||||
json.dump(committed, f, indent=2)
|
||||
f.write("\n")
|
||||
|
||||
print(f"\nBaseline written to {BASELINE_FILE.relative_to(ROOT)}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
console.verdict(f"baseline written to {BASELINE_FILE.relative_to(ROOT)}")
|
||||
@@ -7,7 +7,7 @@ import typer
|
||||
from tooling.core import cli, console
|
||||
from tooling.core.command import command
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.domains.dev import service
|
||||
from tooling.domains.dev import environment, service
|
||||
|
||||
app = cli.domain("dev", "Developer environment and self-diagnosis.")
|
||||
|
||||
@@ -17,6 +17,79 @@ def _domain() -> None:
|
||||
"""Keeps `dev` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@app.command("install-godot")
|
||||
@command
|
||||
def install_godot(
|
||||
version: str = typer.Option(None, "--version", help="Godot version to install."),
|
||||
plan_only: bool = typer.Option(
|
||||
False, "--plan", help="Report what would be downloaded without doing it."
|
||||
),
|
||||
) -> None:
|
||||
"""Install the pinned Godot build to ~/bin/godot4, if it is not already there."""
|
||||
if plan_only:
|
||||
plan = environment.godot_plan(version)
|
||||
console.out(f"wanted {plan.wanted}")
|
||||
console.out(f"installed {plan.installed or '(none)'}")
|
||||
console.out(f"platform {plan.platform_tag}")
|
||||
console.out(f"url {plan.url}")
|
||||
console.verdict(
|
||||
"already current — nothing to do"
|
||||
if plan.already_current
|
||||
else "would download and install"
|
||||
)
|
||||
return
|
||||
environment.install_godot(version)
|
||||
|
||||
|
||||
@app.command("install-rust")
|
||||
@command
|
||||
def install_rust() -> None:
|
||||
"""Ensure clippy and rustfmt are present on the Rust toolchain."""
|
||||
environment.install_rust()
|
||||
|
||||
|
||||
@app.command("worktree")
|
||||
@command
|
||||
def worktree(
|
||||
branch: str = typer.Argument(..., help="Branch name for the new worktree."),
|
||||
start: str = typer.Argument("HEAD", help="Start point."),
|
||||
plan_only: bool = typer.Option(
|
||||
False, "--plan", help="Report where it would go without creating it."
|
||||
),
|
||||
) -> None:
|
||||
"""Create a worktree under .worktrees/ with the venv linked in."""
|
||||
if plan_only:
|
||||
console.verdict(f"would create {environment.worktree_plan(branch)}")
|
||||
return
|
||||
environment.setup_worktree(branch, start)
|
||||
|
||||
|
||||
@app.command("perf")
|
||||
@command
|
||||
def perf(
|
||||
compare: bool = typer.Option(
|
||||
False, "--compare", help="Compare against the saved baseline instead of writing one."
|
||||
),
|
||||
) -> None:
|
||||
"""Benchmark the server and write or check tests/perf/baseline.json."""
|
||||
from tooling.domains.dev import perf as perf_module
|
||||
|
||||
perf_module.run(compare)
|
||||
|
||||
|
||||
@app.command("clerk")
|
||||
@command
|
||||
def clerk(
|
||||
plan_only: bool = typer.Option(
|
||||
False, "--plan", help="Show the per-commit plan without spawning any clerk."
|
||||
),
|
||||
) -> None:
|
||||
"""Review the push range for decision contradictions, one agent per commit."""
|
||||
from tooling.domains.dev import clerk as clerk_module
|
||||
|
||||
clerk_module.run(plan_only)
|
||||
|
||||
|
||||
@app.command("selftest")
|
||||
@command
|
||||
def selftest(
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
"""The `generate` domain — content generators not owned by another domain.
|
||||
|
||||
Deliberately a small residual rather than a catch-all. Brands and corporations
|
||||
live here because they are economic *content* generated ahead of the sim rather
|
||||
than by it; the character manifest because it indexes asset directories. When
|
||||
`ledger` lands, brands and corporations are worth revisiting — they may belong
|
||||
under it.
|
||||
"""
|
||||
+23
-16
@@ -2,7 +2,7 @@
|
||||
"""Generate client/assets/characters/manifest.json from the asset directories.
|
||||
|
||||
Run this whenever artists add new assets so the manifest stays in sync:
|
||||
tooling/generate-character-manifest
|
||||
reach generate character-manifest
|
||||
|
||||
The manifest is the single source of truth for CharacterCreation asset IDs.
|
||||
DirAccess.open() cannot enumerate res:// paths in exported PCK builds (#720).
|
||||
@@ -12,9 +12,13 @@ Output: client/assets/characters/manifest.json
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
ASSETS_ROOT = os.path.join(os.path.dirname(__file__), "..", "client", "assets", "characters")
|
||||
from tooling.core import config, console
|
||||
|
||||
# config.repo_root(), not __file__-relative: this file moved two directories
|
||||
# deeper, and a relative root would resolve to nothing and report an empty
|
||||
# manifest as success (the failure that bit validate-checklist in T-1282).
|
||||
ASSETS_ROOT = str(config.path("client", "assets", "characters"))
|
||||
MANIFEST_PATH = os.path.join(ASSETS_ROOT, "manifest.json")
|
||||
|
||||
|
||||
@@ -92,20 +96,23 @@ def build_manifest() -> dict:
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
def run() -> None:
|
||||
"""Rebuild the manifest from the asset directories and report the counts."""
|
||||
manifest = build_manifest()
|
||||
output = json.dumps(manifest, indent=2) + "\n"
|
||||
with open(MANIFEST_PATH, "w", encoding="utf-8") as f:
|
||||
f.write(output)
|
||||
print(f"Written: {MANIFEST_PATH}")
|
||||
print(f" body_types : {len(manifest['body_types'])}")
|
||||
print(f" heads : {len(manifest['heads'])}")
|
||||
print(f" hair : {len(manifest['hair'])}")
|
||||
print(f" facial_hair: {len(manifest['facial_hair'])}")
|
||||
print(f" eyebrows : {len(manifest['eyebrows'])}")
|
||||
print(f" clothing : {len(manifest['clothing'])}")
|
||||
print(f" accessories: {len(manifest['accessories'])}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
console.event(f"Written: {MANIFEST_PATH}")
|
||||
for key in (
|
||||
"body_types",
|
||||
"heads",
|
||||
"hair",
|
||||
"facial_hair",
|
||||
"eyebrows",
|
||||
"clothing",
|
||||
"accessories",
|
||||
):
|
||||
console.event(f" {key:<12}: {len(manifest[key])}")
|
||||
console.verdict(
|
||||
f"generate-character-manifest: OK — {sum(len(v) for v in manifest.values())} entries"
|
||||
)
|
||||
@@ -1,16 +1,16 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate minimal wiki stub pages for corps in brands.toml that don't yet have wiki pages.
|
||||
"""Generate minimal wiki stub pages for corps in brands.toml with no wiki page yet.
|
||||
|
||||
Usage:
|
||||
python3 tooling/generate_corp_stubs.py [--dry-run]
|
||||
|
||||
In --dry-run mode, prints which stubs would be created without writing any files.
|
||||
`--dry-run` reports which stubs would be created without writing any of them.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).parent.parent
|
||||
from tooling.core import config, console
|
||||
|
||||
REPO = config.repo_root()
|
||||
BRANDS_TOML = REPO / "wiki/economics/corporations/brands.toml"
|
||||
CORPS_DIR = REPO / "wiki/corporations"
|
||||
|
||||
@@ -374,21 +374,12 @@ cross_refs: []
|
||||
{title} is a {cat_desc} operating out of {sys_name} ({origin_system}). Details to be expanded.
|
||||
"""
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate minimal wiki stub pages for corps in brands.toml."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Print which stubs would be created without writing any files.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
def run(dry_run: bool = False) -> None:
|
||||
"""Write a stub page for every corp in brands.toml that lacks one."""
|
||||
corps = parse_brands_toml(BRANDS_TOML)
|
||||
print(f"Found {len(corps)} unique corp_ids in brands.toml")
|
||||
if args.dry_run:
|
||||
print("(dry-run — no files will be written)\n")
|
||||
console.event(f"Found {len(corps)} unique corp_ids in brands.toml")
|
||||
if dry_run:
|
||||
console.event("(dry-run — no files will be written)")
|
||||
|
||||
created = 0
|
||||
skipped = 0
|
||||
@@ -397,17 +388,13 @@ def main():
|
||||
if stub_path.exists():
|
||||
skipped += 1
|
||||
continue
|
||||
if args.dry_run:
|
||||
print(f" (dry-run) Would create: {slug}.md")
|
||||
if dry_run:
|
||||
console.event(f" (dry-run) Would create: {slug}.md")
|
||||
else:
|
||||
content = generate_stub(slug, meta["origin_system"], meta["brand_category"])
|
||||
stub_path.write_text(content, encoding="utf-8")
|
||||
print(f" created: {slug}.md")
|
||||
console.event(f" created: {slug}.md")
|
||||
created += 1
|
||||
|
||||
action = "Would create" if args.dry_run else "Created"
|
||||
print(f"\nDone. {action} {created} stubs, skipped {skipped} existing.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
action = "Would create" if dry_run else "Created"
|
||||
console.verdict(f"{action} {created} stubs, skipped {skipped} existing.")
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Transport for the `generate` domain — args in, delegate, format out."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
from tooling.core import cli, console, process
|
||||
from tooling.core.command import command
|
||||
from tooling.domains.generate import character_manifest, corp_stubs
|
||||
|
||||
app = cli.domain("generate", "Content generators — brands, corporations, manifests.")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _domain() -> None:
|
||||
"""Keeps `generate` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@app.command("brands")
|
||||
@command
|
||||
def brands(args: list[str] = typer.Argument(None, help="Passed to the binary.")) -> None:
|
||||
"""Refresh wiki/economics/corporations/generated_brands.toml.
|
||||
|
||||
Note this is also a SUBROUTINE of import_economics rather than only a
|
||||
standalone verb: `make regen-db` shells out to the same binary as its first
|
||||
step, which is why changes to its Rust source invalidate the systems.db
|
||||
stamp even though no Python changed (.claude/rules/asset-pipeline.md).
|
||||
"""
|
||||
output = process.cargo_binary("generate_brands", *(args or []))
|
||||
if output.strip():
|
||||
console.out(output.rstrip())
|
||||
|
||||
|
||||
@app.command("corporations")
|
||||
@command
|
||||
def corporations(
|
||||
args: list[str] = typer.Argument(None, help="Passed to the binary."),
|
||||
) -> None:
|
||||
"""Generate corporation records."""
|
||||
output = process.cargo_binary("generate_corporations", *(args or []))
|
||||
if output.strip():
|
||||
console.out(output.rstrip())
|
||||
|
||||
|
||||
@app.command("character-manifest")
|
||||
@command
|
||||
def manifest() -> None:
|
||||
"""Regenerate assets/characters/manifest.json from the asset directories."""
|
||||
character_manifest.run()
|
||||
|
||||
|
||||
@app.command("corp-stubs")
|
||||
@command
|
||||
def stubs(
|
||||
dry_run: bool = typer.Option(False, "--dry-run", help="Report without writing."),
|
||||
) -> None:
|
||||
"""Write wiki stub pages for corporations that lack one."""
|
||||
corp_stubs.run(dry_run=dry_run)
|
||||
@@ -0,0 +1,7 @@
|
||||
"""The `pr` domain — the review loop.
|
||||
|
||||
A domain the original map omitted. `tea-comment` and `pr-watchlist-diff` are
|
||||
neither environment setup nor content generation; folding them into `dev` would
|
||||
make `dev` the drawer everything ambiguous goes into, which is how `core/` rots
|
||||
and the same argument applies here.
|
||||
"""
|
||||
@@ -15,26 +15,34 @@ Data comes from pql's native JSON output only (no output parsing):
|
||||
upstream feature request per the no-ad-hoc-wrappers rule)
|
||||
- `pql plan status` → header counts
|
||||
|
||||
Usage: tooling/pql-board-html [OUTPUT.html]
|
||||
Usage: reach pr board-html [--output OUTPUT.html]
|
||||
(default output: /tmp/pql-board.html)
|
||||
|
||||
The emitted file is an Artifact-ready fragment: no doctype/html/head/body
|
||||
wrapper (the Artifact pipeline adds those); it starts with <title> + <style>.
|
||||
"""
|
||||
|
||||
import html
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from tooling.core import config, console, process
|
||||
|
||||
NON_TERMINAL = ["in_progress", "review", "ready", "backlog"]
|
||||
SHOW_CHUNK = 40
|
||||
|
||||
|
||||
def pql(*args):
|
||||
out = subprocess.run(
|
||||
["pql", *args], capture_output=True, text=True, check=True
|
||||
"""Query the planning store. Goes through the guarded exec (D-263).
|
||||
|
||||
Was a bare subprocess.run with check=True, which raised
|
||||
CalledProcessError — a traceback at someone who wanted to know that pql
|
||||
was not installed.
|
||||
"""
|
||||
out = process.run(
|
||||
["pql", *args],
|
||||
cwd=config.repo_root(),
|
||||
missing_fix="install pql, or check PATH in a non-interactive shell",
|
||||
fix=f"run `pql {' '.join(args)}` directly to see what it reports",
|
||||
).stdout
|
||||
return json.loads(out)
|
||||
|
||||
@@ -337,8 +345,9 @@ if(initial&&byId[initial]) select(initial,false);
|
||||
"""
|
||||
|
||||
|
||||
def main():
|
||||
out_path = sys.argv[1] if len(sys.argv) > 1 else "/tmp/pql-board.html"
|
||||
def run(out_path=None) -> None:
|
||||
"""Render the board to `out_path`, defaulting to the repo's scratch dir."""
|
||||
out_path = str(out_path) if out_path else str(config.path(".cache", "pql-board.html"))
|
||||
tickets, blockers, plan = fetch()
|
||||
stamp = datetime.now(timezone.utc).astimezone().strftime("%Y-%m-%d %H:%M")
|
||||
data = json.dumps(
|
||||
@@ -349,12 +358,8 @@ def main():
|
||||
page = PAGE.replace("__STAMP__", stamp).replace("__DATA__", data)
|
||||
with open(out_path, "w", encoding="utf-8") as f:
|
||||
f.write(page)
|
||||
print(
|
||||
console.verdict(
|
||||
f"wrote {out_path}: {len(tickets)} tickets, "
|
||||
f"{sum(len(v) for v in blockers.values())} dep edges, "
|
||||
f"{len(page) // 1024} KiB"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Transport for the `pr` domain — args in, delegate, format out."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from tooling.core import cli, console
|
||||
from tooling.core.command import command
|
||||
from tooling.domains.pr import service
|
||||
|
||||
app = cli.domain("pr", "The review loop — comments and the stale-DB watchlist.")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _domain() -> None:
|
||||
"""Keeps `pr` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@app.command("comment")
|
||||
@command
|
||||
def comment(
|
||||
number: str = typer.Argument(..., help="PR or issue number."),
|
||||
body: str = typer.Argument(..., help="The comment, or @path to read it from a file."),
|
||||
) -> None:
|
||||
"""Post a comment on a Gitea PR or issue.
|
||||
|
||||
Use `@path` for anything long. That form exists because an inline body would
|
||||
need a `$(...)` subshell, and a subshell breaks the permission gate's prefix
|
||||
matching (.claude/rules/tea-cli.md).
|
||||
"""
|
||||
service.comment(number, body)
|
||||
console.verdict(f"commented on #{number}")
|
||||
|
||||
|
||||
@app.command("watchlist-diff")
|
||||
@command
|
||||
def watchlist_diff(
|
||||
base: str = typer.Argument(..., help="Base ref."),
|
||||
head: str = typer.Argument("HEAD", help="Head ref."),
|
||||
) -> None:
|
||||
"""List changed files in a range that could make the committed systems.db stale.
|
||||
|
||||
The watched set is read from tooling/generator_sources.py rather than
|
||||
restated here, so it cannot drift from the registry the stamp check uses.
|
||||
"""
|
||||
changed = service.watchlist_diff(base, head)
|
||||
for path in changed:
|
||||
console.out(path)
|
||||
if not changed:
|
||||
console.verdict(f"pr-watchlist-diff: nothing generator-relevant in {base}...{head}")
|
||||
|
||||
|
||||
@app.command("board-html")
|
||||
@command
|
||||
def board_html(
|
||||
output: Path = typer.Option(None, "--output", help="Where to write the board HTML."),
|
||||
) -> None:
|
||||
"""Render the pql ticket board as a standalone HTML page."""
|
||||
from tooling.domains.pr import board
|
||||
|
||||
board.run(output)
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Logic for the `pr` domain. Transport-agnostic (D-263)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from tooling.core import config, process
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
LOGIN = "schweitz"
|
||||
REPO = "jpmschweitzer/settled-reach"
|
||||
|
||||
# Paths whose change means the committed systems.db may be stale. The generator
|
||||
# set is read from the registry rather than restated, so this list cannot drift
|
||||
# from tooling/generator_sources.py (T-1067).
|
||||
EXTRA_WATCHED = (
|
||||
"tooling/planet-gen/import_heightmaps.py",
|
||||
"tooling/planet-gen/import_province_boundaries.py",
|
||||
"server/data/systems-schema.sql",
|
||||
"wiki/star-systems/",
|
||||
"wiki/economics/",
|
||||
)
|
||||
|
||||
|
||||
def comment(number: str, body_or_path: str) -> None:
|
||||
"""Post a comment on a Gitea PR or issue.
|
||||
|
||||
`@path` reads the body from a file — the form that exists because a long
|
||||
comment inline would need a `$(...)` subshell, and a subshell breaks the
|
||||
permission gate's prefix matching (.claude/rules/tea-cli.md).
|
||||
|
||||
Every flag is explicit for the same reason the rule gives: omitting
|
||||
`--login` or `--repo` sends `tea` interactive, and an interactive prompt
|
||||
with no TTY does not wait — it crashes.
|
||||
"""
|
||||
body = _resolve_body(body_or_path)
|
||||
|
||||
tea = shutil.which("tea")
|
||||
if tea is None:
|
||||
raise ReachError(
|
||||
"the tea CLI is not on PATH",
|
||||
fix="tea is symlinked into ~/.local/bin — check that, rather than "
|
||||
"calling the linuxbrew path directly, which breaks the Bash(tea *) rule",
|
||||
)
|
||||
|
||||
process.run(
|
||||
[tea, "comment", "--login", LOGIN, "--repo", REPO, number, body],
|
||||
cwd=config.repo_root(),
|
||||
fix=f"check that PR/issue {number} exists and the login is authorised",
|
||||
)
|
||||
|
||||
|
||||
def watchlist_diff(base: str, head: str) -> list[str]:
|
||||
"""Files changed in the range that would make the committed systems.db stale."""
|
||||
from tooling import generator_sources
|
||||
|
||||
root = config.repo_root()
|
||||
registered = sorted(
|
||||
{
|
||||
str(path.relative_to(root))
|
||||
for paths in generator_sources.GENERATOR_SOURCES.values()
|
||||
for path in paths
|
||||
}
|
||||
)
|
||||
watched = [*registered, *EXTRA_WATCHED]
|
||||
if not watched:
|
||||
# The original checked this too: an empty registry would diff nothing
|
||||
# and report a clean watchlist, which is the answer that lets a stale DB
|
||||
# through.
|
||||
raise ReachError(
|
||||
"the generator-source registry is empty",
|
||||
fix="tooling/generator_sources.py should list every generator input",
|
||||
)
|
||||
|
||||
result = process.run(
|
||||
["git", "diff", "--name-only", f"{base}...{head}", "--", *watched],
|
||||
cwd=config.repo_root(),
|
||||
check=False,
|
||||
)
|
||||
return [line for line in (result.stdout or "").splitlines() if line]
|
||||
|
||||
|
||||
def _resolve_body(body_or_path: str) -> str:
|
||||
if not body_or_path.startswith("@"):
|
||||
return body_or_path
|
||||
path = Path(body_or_path[1:])
|
||||
if not path.is_file():
|
||||
raise ReachError(
|
||||
f"comment body file not found: {path}",
|
||||
fix="write the comment to a file first, then pass @that-path",
|
||||
)
|
||||
return path.read_text(encoding="utf-8")
|
||||
@@ -1,13 +1,18 @@
|
||||
"""Brand layer (D-189, #827): generate_brands shell-out, TOML import, validation."""
|
||||
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
from .errors import ImportAborted
|
||||
from .paths import BRANDS_TOML, GENERATE_BRANDS_WRAPPER, GENERATED_BRANDS_TOML, REPO_ROOT
|
||||
from .paths import BRANDS_TOML, GENERATED_BRANDS_TOML, REPO_ROOT
|
||||
|
||||
# economy-db/ is hyphenated, so it is not importable as a package and cannot
|
||||
# reach `tooling.core` by normal import. The bootstrap goes away with T-1272
|
||||
# (the hyphen sweep); until then it is explicit rather than implied.
|
||||
if str(REPO_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
VALID_BRAND_CATEGORIES: set[str] = {
|
||||
"terroir", "heritage_craft", "tech_premium", "cultural",
|
||||
@@ -28,28 +33,28 @@ def regenerate_brands() -> None:
|
||||
This replaces the former split (tooling/generate-brands run separately by
|
||||
make regen-db) with a single, coherent brand pipeline owned by one stamp.
|
||||
|
||||
The wrapper script builds the binary on demand and runs it with the default
|
||||
canonical seed=1; callers that need non-canonical seeds must still invoke
|
||||
the wrapper directly (experimentation only — committed output must be seed=1).
|
||||
The binary is built on demand and run with the default canonical seed=1;
|
||||
callers that need non-canonical seeds invoke it directly (experimentation
|
||||
only — committed output must be seed=1).
|
||||
|
||||
This used to shell out to a `tooling/generate-brands` bash wrapper. The
|
||||
wrapper was one of three identical copies of build-if-missing-then-exec, so
|
||||
it was retired into `core.process.cargo_binary` (T-1286) and this calls that
|
||||
helper instead. Same binary, same seed, same output.
|
||||
"""
|
||||
if not GENERATE_BRANDS_WRAPPER.exists():
|
||||
raise FileNotFoundError(
|
||||
f"generate_brands wrapper not found at {GENERATE_BRANDS_WRAPPER}"
|
||||
)
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.core.process import cargo_binary
|
||||
|
||||
print(" [pre/10] Running generate_brands (Rust) to refresh generated_brands.toml...")
|
||||
result = subprocess.run(
|
||||
[str(GENERATE_BRANDS_WRAPPER)],
|
||||
cwd=str(REPO_ROOT),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
print(result.stdout, file=sys.stderr)
|
||||
print(result.stderr, file=sys.stderr)
|
||||
raise ImportAborted()
|
||||
try:
|
||||
stdout = cargo_binary("generate_brands")
|
||||
except ReachError as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
raise ImportAborted() from exc
|
||||
|
||||
# Print the Rust binary's own summary lines (brands generated, coverage).
|
||||
# Indent so they fold under the pre-step heading.
|
||||
for line in result.stdout.splitlines():
|
||||
for line in stdout.splitlines():
|
||||
if line.strip():
|
||||
print(f" {line}")
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@ from generator_sources import (
|
||||
COLOR_REGISTER_BANDS_TOML,
|
||||
CORP_HQ_PLACEMENT_TOML,
|
||||
CORPORATIONS_DIR,
|
||||
GENERATE_BRANDS_WRAPPER,
|
||||
OBJECT_TAG_VOCABULARY_TOML,
|
||||
REPO_ROOT,
|
||||
SETTLEMENT_NAME_LOCKED_TOML,
|
||||
@@ -36,7 +35,6 @@ __all__ = [
|
||||
"CURRENCY_ZONES_TOML",
|
||||
"DB_PATH",
|
||||
"GENERATED_BRANDS_TOML",
|
||||
"GENERATE_BRANDS_WRAPPER",
|
||||
"OBJECT_TAG_VOCABULARY_TOML",
|
||||
"REPO_ROOT",
|
||||
"SCHEMA_SQL",
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generate minor brand products for the Settled Reach economy.
|
||||
#
|
||||
# Usage:
|
||||
# tooling/generate-brands
|
||||
# tooling/generate-brands --seed 42 --min-brands 10000
|
||||
# tooling/generate-brands --output wiki/economics/corporations/generated_brands.toml
|
||||
#
|
||||
# Builds on first run if binary doesn't exist.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
BIN="$ROOT_DIR/server/target/debug/generate_brands"
|
||||
|
||||
# Build if needed
|
||||
if [ ! -f "$BIN" ]; then
|
||||
echo "Building generate_brands..." >&2
|
||||
(cd "$ROOT_DIR/server" && cargo build --bin generate_brands 2>&1 | tail -3) >&2
|
||||
fi
|
||||
|
||||
exec "$BIN" "$@"
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generate Tier-3 corporations for the Settled Reach economy.
|
||||
#
|
||||
# Usage:
|
||||
# tooling/generate-corporations
|
||||
# tooling/generate-corporations --seed 42 --min-corps 5000
|
||||
# tooling/generate-corporations --output path/to/output.toml
|
||||
#
|
||||
# Builds on first run if binary doesn't exist.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
BIN="$ROOT_DIR/server/target/debug/generate_corporations"
|
||||
|
||||
# Build if needed
|
||||
if [ ! -f "$BIN" ]; then
|
||||
echo "Building generate_corporations..." >&2
|
||||
(cd "$ROOT_DIR/server" && cargo build --bin generate_corporations 2>&1 | tail -3) >&2
|
||||
fi
|
||||
|
||||
exec "$BIN" "$@"
|
||||
@@ -50,8 +50,8 @@ ECONOMY_IMPORT_PACKAGE_DIR: Path = (
|
||||
REPO_ROOT / "tooling" / "economy-db" / "economy_import"
|
||||
)
|
||||
|
||||
# Rust sources for the generate_brands subroutine. import_economics shells out
|
||||
# to tooling/generate-brands as part of its normal flow (see economy_import/
|
||||
# Rust sources for the generate_brands subroutine. import_economics runs the
|
||||
# generate_brands binary as part of its normal flow (see economy_import/
|
||||
# brands.py), so all of these contribute to its effective source SHA: any change
|
||||
# to them must invalidate the meta stamp even though Python hasn't changed.
|
||||
GENERATE_BRANDS_RS: Path = (
|
||||
@@ -64,7 +64,11 @@ GENERATE_BRANDS_NAMES_RS: Path = (
|
||||
GENERATE_BRANDS_SURNAMES_RS: Path = (
|
||||
REPO_ROOT / "server" / "src" / "bin" / "shared" / "surname_corpus.rs"
|
||||
)
|
||||
GENERATE_BRANDS_WRAPPER: Path = REPO_ROOT / "tooling" / "generate-brands"
|
||||
# The `tooling/generate-brands` bash wrapper used to be stamped here. It was one
|
||||
# of three identical copies of build-if-missing-then-exec and was retired into
|
||||
# core.process.cargo_binary (T-1286); the seed argument it carried now lives in
|
||||
# economy_import/brands.py, which is already stamped via the package directory
|
||||
# below. Nothing about the brand output stopped being covered.
|
||||
|
||||
# D-237 authored specialization layer: these data TOMLs feed the DB, so a change
|
||||
# to either must flip the stamp and force a regen (#1013).
|
||||
@@ -170,7 +174,7 @@ IMPORT_ECONOMICS_SOURCES: tuple[Path, ...] = (
|
||||
GENERATE_BRANDS_RS,
|
||||
GENERATE_BRANDS_NAMES_RS,
|
||||
GENERATE_BRANDS_SURNAMES_RS,
|
||||
GENERATE_BRANDS_WRAPPER,
|
||||
REPO_ROOT / "tooling" / "core" / "process.py", # runs the brand binary (T-1286)
|
||||
REPO_ROOT / "tooling" / "schema_version.py",
|
||||
SPECIALIZATION_VOCAB_TOML,
|
||||
SYSTEM_SPECIALIZATION_TOML,
|
||||
|
||||
@@ -1,108 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/install-godot — Download and install Godot to ~/bin/
|
||||
#
|
||||
# Usage:
|
||||
# GODOT_VERSION=4.6 ./tooling/install-godot
|
||||
# ./tooling/install-godot # uses default version
|
||||
#
|
||||
# Installs the Godot binary to ~/bin/godot4.
|
||||
# Skips download if correct version is already installed.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
GODOT_VERSION="${GODOT_VERSION:-4.6}"
|
||||
INSTALL_DIR="$HOME/bin"
|
||||
BINARY="$INSTALL_DIR/godot4"
|
||||
|
||||
# Check if already installed at the right version
|
||||
if [[ -x "$BINARY" ]]; then
|
||||
INSTALLED="$("$BINARY" --version 2>/dev/null | head -1 | cut -d. -f1-2)" || true
|
||||
if [[ "$INSTALLED" == "$GODOT_VERSION" ]]; then
|
||||
echo "Godot $GODOT_VERSION already installed at $BINARY"
|
||||
exit 0
|
||||
fi
|
||||
echo "Godot found but version is $INSTALLED, want $GODOT_VERSION"
|
||||
fi
|
||||
|
||||
# Detect platform and architecture
|
||||
OS="$(uname -s)"
|
||||
ARCH="$(uname -m)"
|
||||
|
||||
case "$OS" in
|
||||
Linux)
|
||||
case "$ARCH" in
|
||||
x86_64) PLATFORM="linux.x86_64" ;;
|
||||
aarch64) PLATFORM="linux.arm64" ;;
|
||||
*) echo "Error: unsupported architecture: $ARCH"; exit 1 ;;
|
||||
esac
|
||||
;;
|
||||
Darwin)
|
||||
PLATFORM="macos.universal"
|
||||
;;
|
||||
*)
|
||||
echo "Error: unsupported OS: $OS"
|
||||
echo "Install Godot manually: https://godotengine.org/download"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
FILENAME="Godot_v${GODOT_VERSION}-stable_${PLATFORM}.zip"
|
||||
URL="https://github.com/godotengine/godot/releases/download/${GODOT_VERSION}-stable/${FILENAME}"
|
||||
|
||||
echo "Downloading Godot $GODOT_VERSION for $PLATFORM..."
|
||||
echo " $URL"
|
||||
|
||||
mkdir -p "$INSTALL_DIR"
|
||||
|
||||
# Download to temp directory
|
||||
TMPDIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMPDIR"' EXIT
|
||||
|
||||
if ! curl -fSL --progress-bar -o "$TMPDIR/$FILENAME" "$URL"; then
|
||||
echo "Error: download failed. Check GODOT_VERSION=$GODOT_VERSION is a valid release."
|
||||
echo "Available releases: https://github.com/godotengine/godot/releases"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract
|
||||
echo "Extracting..."
|
||||
if ! command -v unzip >/dev/null 2>&1; then
|
||||
echo "Error: unzip is required but not found. Install it and retry."
|
||||
exit 1
|
||||
fi
|
||||
unzip -q -o "$TMPDIR/$FILENAME" -d "$TMPDIR/extract"
|
||||
|
||||
# Install binary
|
||||
case "$OS" in
|
||||
Linux)
|
||||
SRC="$TMPDIR/extract/Godot_v${GODOT_VERSION}-stable_${PLATFORM}"
|
||||
if [[ -f "$SRC" ]]; then
|
||||
chmod +x "$SRC"
|
||||
mv "$SRC" "$BINARY"
|
||||
else
|
||||
echo "Error: expected binary not found after extraction"
|
||||
ls -la "$TMPDIR/extract/"
|
||||
exit 1
|
||||
fi
|
||||
;;
|
||||
Darwin)
|
||||
SRC="$TMPDIR/extract/Godot.app/Contents/MacOS/Godot"
|
||||
if [[ -f "$SRC" ]]; then
|
||||
mv "$SRC" "$BINARY"
|
||||
chmod +x "$BINARY"
|
||||
else
|
||||
echo "Error: expected Godot.app binary not found after extraction"
|
||||
ls -la "$TMPDIR/extract/"
|
||||
exit 1
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
# Verify
|
||||
if "$BINARY" --version >/dev/null 2>&1; then
|
||||
echo "Godot $GODOT_VERSION installed to $BINARY"
|
||||
"$BINARY" --version
|
||||
else
|
||||
echo "Warning: installed to $BINARY but --version check failed."
|
||||
echo "This may be a display server issue (expected in headless environments)."
|
||||
fi
|
||||
@@ -1,33 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/install-rust — Install Rust via rustup and add required components
|
||||
#
|
||||
# Usage:
|
||||
# ./tooling/install-rust
|
||||
#
|
||||
# Installs rustup + stable toolchain if not present, then ensures
|
||||
# clippy and rustfmt components are installed. Idempotent.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if command -v rustup >/dev/null 2>&1; then
|
||||
echo "Rust already installed: $(rustc --version)"
|
||||
rustup component add clippy rustfmt 2>/dev/null
|
||||
echo "Components verified: clippy, rustfmt"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Installing Rust via rustup..."
|
||||
|
||||
if ! command -v curl >/dev/null 2>&1; then
|
||||
echo "Error: curl is required to install Rust."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --component clippy rustfmt
|
||||
|
||||
# Source cargo env for the rest of this script
|
||||
# shellcheck source=/dev/null
|
||||
. "$HOME/.cargo/env"
|
||||
|
||||
echo "Rust installed: $(rustc --version)"
|
||||
echo "Components: clippy, rustfmt"
|
||||
@@ -67,6 +67,14 @@ DOMAINS: dict[str, tuple[str, str]] = {
|
||||
"tooling.domains.visual.router:app",
|
||||
"Compare captures against goldens, and catch blank ones",
|
||||
),
|
||||
"generate": (
|
||||
"tooling.domains.generate.router:app",
|
||||
"Producers — brands, corporations, manifests and stubs",
|
||||
),
|
||||
"pr": (
|
||||
"tooling.domains.pr.router:app",
|
||||
"The review loop — comments, watchlist and the board",
|
||||
),
|
||||
"jobs": (
|
||||
"tooling.domains.jobs.router:app",
|
||||
"Detached runs — status, logs and outcomes",
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# tooling/pr-watchlist-diff <base> <head> — list watch-list files changed
|
||||
# between <base> and <head>.
|
||||
#
|
||||
# Used by /pr-process step 4a (T-858) to decide whether `make regen-db` must
|
||||
# run before push. The stamped generator sources come from the shared
|
||||
# registry tooling/generator_sources.py (T-1067) — imported live here so this
|
||||
# list can't drift from the stamp writer / tooling/check-systems-db-stamp.
|
||||
#
|
||||
# The extra hardcoded paths below are non-stamped watch items: the surviving
|
||||
# one-time planet-gen importers (import_heightmaps.py, import_province_
|
||||
# boundaries.py — not part of `make regen-db`, but their data feeds the
|
||||
# committed DB), the schema DDL (stamped separately via schema_sha), and the
|
||||
# wiki data directories that feed the generators.
|
||||
set -euo pipefail
|
||||
|
||||
BASE="${1:?usage: tooling/pr-watchlist-diff <base> <head>}"
|
||||
HEAD="${2:?usage: tooling/pr-watchlist-diff <base> <head>}"
|
||||
|
||||
# Load the registry via plain assignment (set -e sees its failure), not
|
||||
# `mapfile < <(...)` — a failed process substitution is invisible to set -e
|
||||
# and would silently yield an empty watch list, disabling the DB-staleness
|
||||
# net exactly when the shared registry breaks. Guard the empty case too.
|
||||
SOURCES_RAW="$(python3 tooling/generator_sources.py --list)"
|
||||
if [ -z "$SOURCES_RAW" ]; then
|
||||
echo "pr-watchlist-diff: generator_sources.py --list returned nothing" >&2
|
||||
exit 1
|
||||
fi
|
||||
mapfile -t GENERATOR_SOURCES <<< "$SOURCES_RAW"
|
||||
|
||||
git diff --name-only "$BASE...$HEAD" -- \
|
||||
"${GENERATOR_SOURCES[@]}" \
|
||||
tooling/planet-gen/import_heightmaps.py \
|
||||
tooling/planet-gen/import_province_boundaries.py \
|
||||
server/data/systems-schema.sql \
|
||||
wiki/star-systems/ \
|
||||
wiki/economics/
|
||||
@@ -1,41 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Post a comment to a Gitea PR or issue.
|
||||
# Usage: tea-comment <number> "comment body"
|
||||
# tea-comment <number> @/path/to/file.md
|
||||
set -euo pipefail
|
||||
|
||||
if [[ $# -lt 2 ]]; then
|
||||
echo "Usage: tea-comment <number> <comment|@filepath>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
NUMBER="$1"
|
||||
INPUT="$2"
|
||||
|
||||
# Resolve tea: PATH first, then the known linuxbrew keg (agent shells often
|
||||
# miss the brew shellenv — clide FR-1).
|
||||
TEA="$(command -v tea || true)"
|
||||
[[ -n "$TEA" ]] || TEA="/home/linuxbrew/.linuxbrew/bin/tea"
|
||||
if [[ ! -x "$TEA" ]]; then
|
||||
echo "Error: tea CLI not found (PATH or /home/linuxbrew/.linuxbrew/bin)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# If the body starts with @, read from file
|
||||
if [[ "$INPUT" == @* ]]; then
|
||||
FILEPATH="${INPUT#@}"
|
||||
if [[ ! -f "$FILEPATH" ]]; then
|
||||
echo "Error: file not found: $FILEPATH" >&2
|
||||
exit 1
|
||||
fi
|
||||
BODY=$(cat "$FILEPATH")
|
||||
else
|
||||
BODY="$INPUT"
|
||||
fi
|
||||
|
||||
HASH=$(echo -n "$BODY" | md5sum | cut -c1-8)
|
||||
TMPFILE="/tmp/tea-comment-${NUMBER}-${HASH}.md"
|
||||
trap 'rm -f "$TMPFILE"' EXIT
|
||||
|
||||
printf '%s' "$BODY" > "$TMPFILE"
|
||||
"$TEA" comment --login schweitz --repo jpmschweitzer/settled-reach "$NUMBER" "$(cat "$TMPFILE")"
|
||||
@@ -167,9 +167,14 @@ def check_one_guarded_exec(failures: list[str]) -> None:
|
||||
"missing-binary message and the failure remedy live"
|
||||
)
|
||||
for node in ast.walk(tree):
|
||||
# The receiver has to be checked, not just the attribute name:
|
||||
# `platform.system()` is a legitimate call and shares a name with
|
||||
# `os.system()`. A check that fires on the wrong thing gets muted.
|
||||
if (
|
||||
isinstance(node, ast.Call)
|
||||
and isinstance(node.func, ast.Attribute)
|
||||
and isinstance(node.func.value, ast.Name)
|
||||
and node.func.value.id == "os"
|
||||
and node.func.attr in {"system", "popen", "execv", "execvp"}
|
||||
):
|
||||
failures.append(
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
#!/usr/bin/env python3
|
||||
"""The `reach dev` environment decisions, exercised without performing them (T-1286).
|
||||
|
||||
`install-godot`, `install-rust` and `worktree-setup` were the three scripts the
|
||||
CLI port stood to gain least from and risked most on. Nothing about them can be
|
||||
checked by running them: a passing test would download a 60 MB archive, mutate
|
||||
`~/bin`, or leave a git worktree behind. So the port split each one into a pure
|
||||
decision (`godot_plan`, `worktree_plan`) and a thin performing half, and this
|
||||
file pins the decisions.
|
||||
|
||||
That split is the whole claim of the port, which is why it gets a test rather
|
||||
than a manual `--plan` run: an installer that cannot be tested has to be
|
||||
trusted instead, and trusting an installer is how a working environment becomes
|
||||
an unreproducible one.
|
||||
|
||||
Run: python3 tooling/test_environment.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from tooling.core.errors import ReachError # noqa: E402
|
||||
from tooling.domains.dev import environment # noqa: E402
|
||||
|
||||
|
||||
def test_plan_names_the_pinned_version(failures: list[str]) -> None:
|
||||
"""The default plan targets DEFAULT_GODOT_VERSION and builds a real URL."""
|
||||
plan = environment.godot_plan()
|
||||
if plan.wanted != environment.DEFAULT_GODOT_VERSION:
|
||||
failures.append(
|
||||
f"godot_plan() wanted {plan.wanted}, expected the pinned "
|
||||
f"{environment.DEFAULT_GODOT_VERSION}"
|
||||
)
|
||||
if plan.wanted not in plan.url or plan.filename not in plan.url:
|
||||
failures.append(f"the URL does not carry version and filename: {plan.url}")
|
||||
if not plan.url.startswith("https://"):
|
||||
failures.append(f"the download URL is not https: {plan.url}")
|
||||
|
||||
|
||||
def test_explicit_version_overrides_the_pin(failures: list[str]) -> None:
|
||||
plan = environment.godot_plan("4.9")
|
||||
if plan.wanted != "4.9" or "4.9-stable" not in plan.url:
|
||||
failures.append(f"an explicit version did not reach the URL: {plan.url}")
|
||||
|
||||
|
||||
def test_env_var_overrides_the_pin(failures: list[str]) -> None:
|
||||
"""GODOT_VERSION is how make setup passes the version through."""
|
||||
before = os.environ.get("GODOT_VERSION")
|
||||
os.environ["GODOT_VERSION"] = "4.7"
|
||||
try:
|
||||
if environment.godot_plan().wanted != "4.7":
|
||||
failures.append("GODOT_VERSION was ignored by godot_plan()")
|
||||
# An explicit argument still wins over the environment.
|
||||
if environment.godot_plan("4.8").wanted != "4.8":
|
||||
failures.append("an explicit version lost to GODOT_VERSION")
|
||||
finally:
|
||||
if before is None:
|
||||
del os.environ["GODOT_VERSION"]
|
||||
else:
|
||||
os.environ["GODOT_VERSION"] = before
|
||||
|
||||
|
||||
def test_already_current_is_decided_not_performed(failures: list[str]) -> None:
|
||||
"""already_current compares installed against wanted — the skip decision."""
|
||||
real = environment.godot_plan()
|
||||
same = type(real)(
|
||||
wanted="4.6", installed="4.6", platform_tag=real.platform_tag,
|
||||
url=real.url, filename=real.filename,
|
||||
)
|
||||
differs = type(real)(
|
||||
wanted="4.6", installed="4.5", platform_tag=real.platform_tag,
|
||||
url=real.url, filename=real.filename,
|
||||
)
|
||||
absent = type(real)(
|
||||
wanted="4.6", installed=None, platform_tag=real.platform_tag,
|
||||
url=real.url, filename=real.filename,
|
||||
)
|
||||
if not same.already_current:
|
||||
failures.append("a matching installed version was not treated as current")
|
||||
if differs.already_current:
|
||||
failures.append("a mismatched version was treated as current — install skipped")
|
||||
if absent.already_current:
|
||||
failures.append("a missing install was treated as current — install skipped")
|
||||
|
||||
|
||||
def test_unsupported_platform_names_a_remedy(failures: list[str]) -> None:
|
||||
"""An unknown platform must fail loudly, not build a URL that 404s."""
|
||||
saved = dict(environment.PLATFORMS)
|
||||
environment.PLATFORMS.clear()
|
||||
try:
|
||||
environment.godot_plan()
|
||||
failures.append("an unsupported platform produced a plan instead of an error")
|
||||
except ReachError as exc:
|
||||
if not exc.fix:
|
||||
failures.append("the unsupported-platform error carries no fix")
|
||||
finally:
|
||||
environment.PLATFORMS.update(saved)
|
||||
|
||||
|
||||
def test_worktree_plan_targets_the_convention(failures: list[str]) -> None:
|
||||
"""D-221: worktrees live at <root>/.worktrees/<branch>."""
|
||||
target = environment.worktree_plan("some-unused-branch-name")
|
||||
expected = environment.config.repo_root() / ".worktrees" / "some-unused-branch-name"
|
||||
if target != expected:
|
||||
failures.append(f"worktree_plan gave {target}, expected {expected}")
|
||||
|
||||
|
||||
def test_worktree_plan_refuses_an_existing_tree(failures: list[str]) -> None:
|
||||
"""Silently reusing a directory is how two branches share one tree."""
|
||||
root = environment.config.repo_root()
|
||||
with tempfile.TemporaryDirectory(dir=root / ".worktrees") as existing:
|
||||
name = Path(existing).name
|
||||
try:
|
||||
environment.worktree_plan(name)
|
||||
failures.append("worktree_plan accepted a branch whose directory exists")
|
||||
except ReachError as exc:
|
||||
if not exc.fix:
|
||||
failures.append("the existing-worktree error carries no fix")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
# .worktrees/ must exist for the collision test to place a directory in it.
|
||||
(environment.config.repo_root() / ".worktrees").mkdir(exist_ok=True)
|
||||
|
||||
failures: list[str] = []
|
||||
test_plan_names_the_pinned_version(failures)
|
||||
test_explicit_version_overrides_the_pin(failures)
|
||||
test_env_var_overrides_the_pin(failures)
|
||||
test_already_current_is_decided_not_performed(failures)
|
||||
test_unsupported_platform_names_a_remedy(failures)
|
||||
test_worktree_plan_targets_the_convention(failures)
|
||||
test_worktree_plan_refuses_an_existing_tree(failures)
|
||||
|
||||
if failures:
|
||||
print("test_environment: FAIL", file=sys.stderr)
|
||||
for failure in failures:
|
||||
print(f" - {failure}", file=sys.stderr)
|
||||
return 1
|
||||
print(
|
||||
"test_environment: OK — version pin, overrides, skip decision, "
|
||||
"platform refusal and worktree placement, none of them performed"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,49 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# worktree-setup — create a git worktree that's immediately usable.
|
||||
#
|
||||
# Collapses the /whats-next §3c ritual into one command. A bare
|
||||
# `git worktree add` leaves two things broken that bit the Layer-5 batch:
|
||||
# 1. no `.venv` in the worktree, so `make test-tooling` / `make regen-db` /
|
||||
# any python tooling can't find `.venv/bin/python`;
|
||||
# 2. (fixed separately in .config/hooks/post-checkout) pql resolving the
|
||||
# worktree's `.git` *file* to the MAIN checkout — the post-checkout hook
|
||||
# now passes `--vault`, so this script relies on it for the pql rebuild.
|
||||
#
|
||||
# This script adds the worktree under the repo's gitignored `.worktrees/`
|
||||
# (never the parent dir — outside the permission sandbox), symlinks the venv,
|
||||
# and prints the worktree path + the in-worktree reminders.
|
||||
#
|
||||
# Usage: tooling/worktree-setup <branch-name> [<start-point>]
|
||||
# (run from the MAIN checkout, not a linked worktree)
|
||||
set -euo pipefail
|
||||
|
||||
branch="${1:?usage: tooling/worktree-setup <branch-name> [<start-point>]}"
|
||||
start="${2:-HEAD}"
|
||||
|
||||
repo_root="$(git rev-parse --show-toplevel)"
|
||||
if [ -f "$repo_root/.git" ]; then
|
||||
echo "worktree-setup: run this from the MAIN checkout, not a worktree" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
wt_dir="$repo_root/.worktrees/$branch"
|
||||
if [ -e "$wt_dir" ]; then
|
||||
echo "worktree-setup: $wt_dir already exists — reuse it or 'git worktree remove' it first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The post-checkout hook (.config/hooks/post-checkout) fires here and rebuilds
|
||||
# the new worktree's pql.db via `pql --vault <worktree>`.
|
||||
git worktree add "$wt_dir" -b "$branch" "$start"
|
||||
|
||||
# Symlink the venv — worktrees don't copy it and a fresh per-worktree venv is
|
||||
# wasteful (the main one is identical). Makes `.venv/bin/python` resolve so the
|
||||
# Makefile's VENV_PY and every python tool work inside the worktree.
|
||||
if [ -d "$repo_root/.venv" ] && [ ! -e "$wt_dir/.venv" ]; then
|
||||
ln -s "$repo_root/.venv" "$wt_dir/.venv"
|
||||
echo " linked .venv -> $repo_root/.venv"
|
||||
fi
|
||||
|
||||
echo "worktree ready: $wt_dir"
|
||||
echo " reminders: pql in this worktree needs --vault \"$wt_dir\" (FR-4);"
|
||||
echo " tea commands run from the main checkout (go-git can't read a linked worktree)."
|
||||
Reference in New Issue
Block a user