diff --git a/.pql/changelog/ticket_history/2026-08.sql b/.pql/changelog/ticket_history/2026-08.sql index c4a7e8517..779934e3f 100644 --- a/.pql/changelog/ticket_history/2026-08.sql +++ b/.pql/changelog/ticket_history/2026-08.sql @@ -1361,3 +1361,55 @@ NEGATIVE ACCEPTANCE CRITERION (added 2026-08-20), which is the one that will be Rationale: an absolute or interpreter-prefixed path breaks the Bash(reach *) permission rule and prompts every time — the exact failure already recorded for tea, where the fix was a bare name on PATH — and it reinstates the interpreter fork between a human shell and a hook that VENV_PY papers over today. The highest-risk site is E6: 84 make targets become reach wrappers, and a single VENV_PY-prefixed or python -m invocation among them silently undoes the initiative. Add a grep-shaped check to the E8 conformance suite: no python -m tooling and no .venv/bin/reach anywhere in the Makefile, hooks, skills or rules. FINDING from T-1258 (2026-08-20): make setup-venv is already stale and this ticket owns the fix. It runs python3 -m venv .venv then .venv/bin/pip install -e .[dev] --quiet, but the venv on disk was created by uv (pyvenv.cfg records uv = 0.10.2) and therefore has NO pip at all, so that second line cannot work against it. Replace the target with the uv path rather than patching pip back in — uv is the direction of travel for this initiative anyway, and installing pip into a uv venv just to satisfy a stale recipe recreates the interpreter fork the ticket exists to remove. Related: the project itself had never actually been installed before T-1258 — site-packages held the dependencies but no settled-reach-tooling dist-info, which is consistent with there being no __init__.py to expose.', NULL, '2026-08-20 16:53:12', '2026-08-20 16:53:12.179', '2026-08-20 16:53:12.179', NULL, '9c22d5535327a13116653673fd38dd38', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S66GG3RZ3V2F9EXNSFW2G4', 'status', 'in_progress', 'done', NULL, '2026-08-20 16:53:51', '2026-08-20 16:53:51.369', '2026-08-20 16:53:51.369', NULL, '65293f029457b2ff1fa6ee1ce31b5eba', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'status', 'backlog', 'in_progress', NULL, '2026-08-20 16:54:55', '2026-08-20 16:54:55.954', '2026-08-20 16:54:55.954', NULL, '191850edd28c79922d78c980bf18532e', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'status', 'in_progress', 'in_progress', NULL, '2026-08-20 16:55:17', '2026-08-20 16:55:17.084', '2026-08-20 16:55:17.084', NULL, '74b5d2d9f54f49c8e844a7918d10e355', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'description', 'pyproject.toml currently has NO [project.scripts] at all — no console entrypoint is defined anywhere. Adding one plus the two dependencies is the whole delta. Deliverables: [project.scripts] reach = tooling.main:cli; typer and pydantic added to [project] dependencies, both version-pinned and CVE-checked against NVD plus OSV the way scipy and Pillow already are in this file, with the check date in a comment since that is the established convention here. NOTE the entrypoint target is main:cli not main:app — see the lazy-registration task; the root is a click Group subclass rather than a typer.Typer instance, because a Typer root with zero eagerly-registered commands raises at build time and because keeping typer off the --help path is worth having. Acceptance: .venv/bin/reach --help runs. This is the venv-local proof only; the bare-name-on-PATH guarantee is a separate task and is the one that actually matters.', 'pyproject.toml currently has NO [project.scripts] at all — no console entrypoint is defined anywhere. Adding one plus the two dependencies is the whole delta. Deliverables: [project.scripts] reach = tooling.main:cli; typer and pydantic added to [project] dependencies, both version-pinned and CVE-checked against NVD plus OSV the way scipy and Pillow already are in this file, with the check date in a comment since that is the established convention here. NOTE the entrypoint target is main:cli not main:app — see the lazy-registration task; the root is a click Group subclass rather than a typer.Typer instance, because a Typer root with zero eagerly-registered commands raises at build time and because keeping typer off the --help path is worth having. Acceptance: .venv/bin/reach --help runs. This is the venv-local proof only; the bare-name-on-PATH guarantee is a separate task and is the one that actually matters. + +DONE 2026-08-23. Entrypoint works: .venv/bin/reach --help runs in 80 ms. But the design in the description above is WRONG in one load-bearing way and had to change. + +TYPER VENDORS CLICK — the click.Group root is impossible. +The description specifies main:cli as a click Group subclass, on the reasoning that it keeps typer off the --help path. That cannot be done. As of typer 0.26.0 Click is VENDORED into typer as typer._click: no top-level click package is installed at all, and the docs are explicit that extracting the internal Click app is no longer supported. A real click.Group root hosting typer sub-apps would mean two different Click implementations in one process. Discovered the direct way — main.py imported click and reach --help died with ModuleNotFoundError. +Replacement, and it is the supported surface: the root is a typer.Typer, and lazy registration goes through typer.Typer(cls=...) with a typer.core.TyperGroup subclass. Verified TyperGroup subclasses typer._click.core.Command and that cls= composes. T-1260 builds the lazy group on that surface, NOT on click.Group. + +THE ZERO-COMMAND CLAIM WAS HALF RIGHT. +The description says a Typer root with zero eagerly-registered commands raises at build time. Tested both ways: with no commands AND no callback it does raise (RuntimeError: Could not get a command for this Typer instance); with a callback and no commands it builds fine and prints help. So the empty root is legal as long as it carries a callback, which main.py now has. That matters because lazy registration means NO command is ever eager. + +rich_markup_mode=None IS LOAD-BEARING, NOT COSMETIC. +An empty --help costs 168 ms with typer default rich rendering and 74 ms with rich disabled — 94 ms, 56 percent of runtime, for an empty CLI. Verified rich and pygments are then absent from sys.modules entirely, so this removes them from the import path rather than merely skipping the render. It also stops typer drawing box-art help, which it does EVEN WHEN STDOUT IS A PIPE — that would have put box-drawing characters into every hook log and agent capture. One line to revert if the boxes are ever worth more than the milliseconds. + +typer-slim IS NOT THE ANSWER (checked, since rich is unused). +Deprecated as of typer 0.22.0 and now a shallow wrapper that installs all of typer, so depending on it buys nothing. Dependency stays plain typer; rich ships transitively but is never imported. + +SECURITY — both direct deps and the transitive set checked 2026-08-23 against NVD, OSV and the GitHub Advisory Database: +- typer 0.27.1: no advisories on record at all. +- pydantic 2.13.4: clean. PYSEC-2026-1812 / CVE-2024-3772 (email-regex ReDoS) is fixed in 2.4.0. NOTE the 2026 SSRF advisories CVE-2026-25580 and CVE-2026-54249 are against pydantic-ai, a DIFFERENT package that is not a dependency here — recorded in pyproject so the next sweep does not re-panic. +- transitive: pygments 2.21.0 clears CVE-2026-4539 (archetype-lexer ReDoS, fixed 2.20.0); rich and click have no advisories on record. + +TIMING, for the record: 80 ms for reach --help with zero domains registered. Interpreter startup is 27 ms of that and typer import is ~42 ms. Against the D-263 ceiling — raised to 500 ms on 2026-08-23 — that leaves real headroom for the domains to come. + +BARE `reach` EXITS 2. no_args_is_help=True prints help and exits 2 (Click usage-error convention). Flagging rather than changing it: that is standard for no-command-given, and D-263''s exit-code contract is about FAILURES carrying a remedy, which this is not. Revisit in T-1260 if the domain list should exit 0.', NULL, '2026-08-23 12:00:53', '2026-08-23 12:00:53.737', '2026-08-23 12:00:53.737', NULL, '279e4d8614ec5d62125a6105aee43ae0', 2) ON CONFLICT(hash) DO NOTHING; +INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S685APSPZRXSM9F62DTKGM', 'description', 'reach --help must list every domain WITHOUT importing any of them. This is not an optimisation; it is the mechanism that keeps the 104 ms push-gate budget reachable, and it has one specific trap. Click Group.format_commands calls get_command for every subcommand to render its short help — which imports all 123 modules just to print help. So format_commands MUST be overridden to read short help from a declaration table rather than from the loaded command. Deliverables: tooling/main.py holding a DOMAINS table mapping name to (import_path, short_help); a LazyDomainGroup(click.Group) overriding list_commands, get_command and format_commands; get_command resolving via importlib and typer.main.get_command only for the domain actually invoked. main.py is a router and nothing else — no logic, no I/O, no pydantic, no domain imports at module level. Acceptance is a TEST, not an eyeball: run reach --help in a subprocess and assert no tooling.domains..service module is in sys.modules afterwards. That test is the whole point of the ticket and belongs with it, not in E8. + +AMENDED 2026-08-20 — the justification changes, the requirement does not. Lazy registration is still mandatory, but NOT because of a 104 ms parity budget (withdrawn — see D-263 amendment and T-1251). It is mandatory because of what an eager entrypoint would actually cost: measured in the repo venv, scipy.ndimage 275 ms, pydantic 87 ms, numpy 72 ms, PIL.Image 29 ms — about 460 ms paid before executing a line of its own, growing every time a domain is added. The format_commands trap and the sys.modules acceptance test are unchanged and are the heart of this ticket.', 'reach --help must list every domain WITHOUT importing any of them. This is not an optimisation; it is the mechanism that keeps the 104 ms push-gate budget reachable, and it has one specific trap. Click Group.format_commands calls get_command for every subcommand to render its short help — which imports all 123 modules just to print help. So format_commands MUST be overridden to read short help from a declaration table rather than from the loaded command. Deliverables: tooling/main.py holding a DOMAINS table mapping name to (import_path, short_help); a LazyDomainGroup(click.Group) overriding list_commands, get_command and format_commands; get_command resolving via importlib and typer.main.get_command only for the domain actually invoked. main.py is a router and nothing else — no logic, no I/O, no pydantic, no domain imports at module level. Acceptance is a TEST, not an eyeball: run reach --help in a subprocess and assert no tooling.domains..service module is in sys.modules afterwards. That test is the whole point of the ticket and belongs with it, not in E8. + +AMENDED 2026-08-20 — the justification changes, the requirement does not. Lazy registration is still mandatory, but NOT because of a 104 ms parity budget (withdrawn — see D-263 amendment and T-1251). It is mandatory because of what an eager entrypoint would actually cost: measured in the repo venv, scipy.ndimage 275 ms, pydantic 87 ms, numpy 72 ms, PIL.Image 29 ms — about 460 ms paid before executing a line of its own, growing every time a domain is added. The format_commands trap and the sys.modules acceptance test are unchanged and are the heart of this ticket. + +AMENDED 2026-08-23 from T-1259 — the base class in the description is WRONG and this ticket cannot be built as written. + +The description specifies LazyDomainGroup(click.Group). There is no top-level click package: typer VENDORS click as of 0.26.0 (as typer._click), and the docs are explicit that extracting the internal Click app is unsupported. Adding click as a direct dependency would be worse, not better — two Click implementations in one process, with typer.main.get_command returning a typer._click Command that a real click.Group cannot host. + +CORRECTED DESIGN, verified working in T-1259: +- The root is `cli = typer.Typer(cls=LazyDomainGroup, ...)` in tooling/main.py — already in place. +- LazyDomainGroup subclasses **typer.core.TyperGroup**, not click.Group. Confirmed TyperGroup''s MRO is (TyperGroup, typer._click.core.Command, ABC) and that cls= composes. +- Override list_commands, get_command and format_commands on that class. The format_commands trap is UNCHANGED and is still the heart of this ticket: the base implementation calls get_command for every subcommand to render short help, which would import every domain just to print --help. It must read short help from the DOMAINS declaration table instead. +- get_command resolves the invoked domain via importlib and converts the domain''s Typer sub-app with typer.main.get_command. +- The sys.modules acceptance test is unchanged and is still the point: after `reach --help`, no domain service module may appear in sys.modules. Extend it to assert `rich` and `pygments` are absent too — that is currently true and is worth pinning, since it is one careless kwarg away from regressing. + +ALREADY DONE IN T-1259, do not redo: +- main.py exists with the Typer root and a callback. The callback is REQUIRED: a Typer root with no commands and no callback raises RuntimeError("Could not get a command for this Typer instance") at build time, and lazy registration means no command is ever eager. +- rich_markup_mode=None is set and is load-bearing — worth 94 ms of a 168 ms empty --help, and it keeps rich and pygments off the import path. Do not remove it while adding the lazy group. + +OPEN QUESTION carried here: bare `reach` currently exits 2 (Click''s usage-error convention via no_args_is_help). Once domains are registered and a bare `reach` prints a useful domain list, decide whether that should exit 0 instead. It is not a failure, and D-263''s exit-code contract concerns failures carrying a remedy. + +BUDGET NOTE: the D-263 ceiling was raised to ~500 ms on 2026-08-23. Current cost of `reach --help` with zero domains is 80 ms. Stay smart about what loads; do not contort for tightness.', NULL, '2026-08-23 12:01:12', '2026-08-23 12:01:12.824', '2026-08-23 12:01:12.824', NULL, 'ba4645e4d3bebf918819b0564ff7c1ff', 2) ON CONFLICT(hash) DO NOTHING; diff --git a/.pql/changelog/tickets/2026-08.sql b/.pql/changelog/tickets/2026-08.sql index 46244cb6f..c999fed5d 100644 --- a/.pql/changelog/tickets/2026-08.sql +++ b/.pql/changelog/tickets/2026-08.sql @@ -1334,3 +1334,86 @@ NEGATIVE ACCEPTANCE CRITERION (added 2026-08-20), which is the one that will be Rationale: an absolute or interpreter-prefixed path breaks the Bash(reach *) permission rule and prompts every time — the exact failure already recorded for tea, where the fix was a bare name on PATH — and it reinstates the interpreter fork between a human shell and a hook that VENV_PY papers over today. The highest-risk site is E6: 84 make targets become reach wrappers, and a single VENV_PY-prefixed or python -m invocation among them silently undoes the initiative. Add a grep-shaped check to the E8 conformance suite: no python -m tooling and no .venv/bin/reach anywhere in the Makefile, hooks, skills or rules. FINDING from T-1258 (2026-08-20): make setup-venv is already stale and this ticket owns the fix. It runs python3 -m venv .venv then .venv/bin/pip install -e .[dev] --quiet, but the venv on disk was created by uv (pyvenv.cfg records uv = 0.10.2) and therefore has NO pip at all, so that second line cannot work against it. Replace the target with the uv path rather than patching pip back in — uv is the direction of travel for this initiative anyway, and installing pip into a uv venv just to satisfy a stale recipe recreates the interpreter fork the ticket exists to remove. Related: the project itself had never actually been installed before T-1258 — site-packages held the dependencies but no settled-reach-tooling dist-info, which is consistent with there being no __init__.py to expose.', 'backlog', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:34.486', '2026-08-20 16:53:12.179', NULL, 'fe96cd641763396858e412c5af53b982', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S66GG3RZ3V2F9EXNSFW2G4', 'task', '06G1S392DEG41MRGEFSYWJWP2W', 'Package skeleton: tooling/ becomes an importable package with a bounded core/', 'Turn tooling/ into a real package without moving anything yet. Deliverables: tooling/__init__.py (deliberately empty — every import here is paid by every reach invocation including four per push); tooling/core/__init__.py stating the bound (only what has no domain; the moment a core module grows a service it moves out); tooling/core/config.py resolving the repo root from __file__ with a project.yaml sentinel check and an SR_REPO_ROOT override, no subprocess and no git call since this is on the gate path; tooling/core/console.py as the SINGLE output path, stdlib only, out() to stdout and err() to stderr, no rich and no typer import so it stays cheap; tooling/domains/__init__.py. Also configure setuptools explicitly rather than relying on flat-layout auto-discovery — [tool.setuptools.packages.find] with include tooling*. Note the four hyphenated directories planet-gen, economy-db, garment-fit and pql-migrate are invisible to package discovery because a hyphen is not a valid identifier, and tooling/econ-sim is a Rust crate; none need explicit exclusion until E3 renames them, at which point they need __init__.py. Acceptance: pip install -e . succeeds, python -c import tooling.core.config works, ruff clean. + +AMENDED 2026-08-20 — core/console.py changes shape BEFORE this ticket runs. The description above specs it as a print wrapper (stdlib only, out() to stdout, err() to stderr). D-263 was amended the same day with a streaming execution model, so console.py is the EVENT EMITTER instead: it emits JSONL events, one object per line, fields ts/level/phase/message and optional progress, to STDERR — stdout stays reserved for the command actual output so reach ... | jq keeps working. Still the single output path and still stdlib-only, since json is stdlib. Rendering to human-readable text happens at the sink, not at the emit site, which is what lets the same artefact serve a person tailing a log and a conformance test asserting on it. Two properties to get right here because everything downstream depends on them: (1) emitting is OPTIONAL — a command that emits no progress events must work normally, and the gates should emit none, so this is a channel not an obligation; (2) the event stream never replaces the final verdict — outcome, exit code and remedy are still printed once, last. See E9 (T-1264) for the job log and tail, which consume what this emits. + +DONE 2026-08-20. Package skeleton landed and verified. Findings and deviations below, since several affect later tickets. + +VERIFIED (each check could have failed, and the failure paths were exercised): +- uv pip install -e . succeeds; dist-info top_level.txt contains exactly "tooling" — discovery did NOT pull in client/, server/, docs/, wiki/, db/ or tests/, which was the risk that made explicit config necessary. +- import tooling.core.config, tooling.core.console, tooling.domains all succeed. +- console non-TTY: JSONL on stderr, command output alone on stdout (proved by redirecting stderr to /dev/null and seeing only DATA). +- console SR_OUTPUT_FORMAT=text: human rendering, phase prefix, percentage, and the verdict printing its Fix line last. +- debug-level events suppressed at the default threshold; verdict never suppressed. +- config sentinel REJECTS a bad override: SR_REPO_ROOT=/tmp raises and names both remedies. +- ruff check tooling/ clean (full tree, as the push gate runs it). +- No regression to the existing scripts: check-client-version, check-canvas-version and check-dataflow-graph all still pass with tooling/ now a package. +- Import cost: tooling.core.config 0.4 ms, tooling.core.console 3.6 ms — of which 1.9 ms is typing, which typer and click both import anyway, so it is free in any real invocation. + +DEVIATION FROM THE WRITTEN ACCEPTANCE — pip is not available. +The acceptance says "pip install -e . succeeds". There is no pip in .venv: pyvenv.cfg records uv = 0.10.2, so the venv was made by uv, which does not install pip. Acceptance was met with `uv pip install -e . --python .venv/bin/python` instead. This is the direction of travel per D-263 anyway. + +FINDING FOR T-1261 (the make target) — make setup-venv is STALE. +It runs `python3 -m venv .venv` then `.venv/bin/pip install -e ".[dev]" --quiet`. Neither matches the venv on disk, and the pip line cannot work against it. This is precisely the venv split Q-124 named: the recorded procedure and the actual state have already diverged. T-1261 owns the make target and should replace it with the uv path rather than patch pip back in. + +FINDING — the project had never actually been installed. +Before this ticket site-packages held the DEPENDENCIES but no settled-reach-tooling dist-info and no .pth for it. That is consistent: with no __init__.py anywhere, an editable install had no package to expose. So this is the first commit where `import tooling` means anything. + +DELIBERATE OMISSION — err() was dropped. +The original description asked for out() to stdout and err() to stderr. The amendment made console the event emitter, and a raw-text err() would then be a SECOND output path competing with the event stream — exactly the drift D-263 forbids by naming console the single one. event(level="error") and verdict(ok=False) cover every case err() would have. Flagging it because the word appears in the description and its absence is intentional, not an oversight. + +CARRIED TO T-1249 — config raises RuntimeError, not ReachError. +core/errors.py does not exist yet, so _validated raises RuntimeError with an instructional message that already follows the contract (it names the fix). One call site to convert when T-1249 lands; there is a comment at the raise saying so. + +INCIDENTAL — .gitignore. +Added *.egg-info/ (build output the install creates). Also added .venv/, which was only ignored via .git/info/exclude — machine-local, so a fresh clone or a new worktree would not have ignored it.', 'done', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:12.032', '2026-08-20 16:53:51.369', NULL, '90e67d766cb10aff69fbc9eaa35377f4', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'task', '06G1S392DEG41MRGEFSYWJWP2W', 'The reach entrypoint: [project.scripts] plus typer and pydantic', 'pyproject.toml currently has NO [project.scripts] at all — no console entrypoint is defined anywhere. Adding one plus the two dependencies is the whole delta. Deliverables: [project.scripts] reach = tooling.main:cli; typer and pydantic added to [project] dependencies, both version-pinned and CVE-checked against NVD plus OSV the way scipy and Pillow already are in this file, with the check date in a comment since that is the established convention here. NOTE the entrypoint target is main:cli not main:app — see the lazy-registration task; the root is a click Group subclass rather than a typer.Typer instance, because a Typer root with zero eagerly-registered commands raises at build time and because keeping typer off the --help path is worth having. Acceptance: .venv/bin/reach --help runs. This is the venv-local proof only; the bare-name-on-PATH guarantee is a separate task and is the one that actually matters.', 'in_progress', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:19.770', '2026-08-20 16:54:55.954', NULL, '737668978ade0231373d65c11a6bdfb1', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'task', '06G1S392DEG41MRGEFSYWJWP2W', 'The reach entrypoint: [project.scripts] plus typer and pydantic', 'pyproject.toml currently has NO [project.scripts] at all — no console entrypoint is defined anywhere. Adding one plus the two dependencies is the whole delta. Deliverables: [project.scripts] reach = tooling.main:cli; typer and pydantic added to [project] dependencies, both version-pinned and CVE-checked against NVD plus OSV the way scipy and Pillow already are in this file, with the check date in a comment since that is the established convention here. NOTE the entrypoint target is main:cli not main:app — see the lazy-registration task; the root is a click Group subclass rather than a typer.Typer instance, because a Typer root with zero eagerly-registered commands raises at build time and because keeping typer off the --help path is worth having. Acceptance: .venv/bin/reach --help runs. This is the venv-local proof only; the bare-name-on-PATH guarantee is a separate task and is the one that actually matters.', 'in_progress', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:19.770', '2026-08-20 16:55:17.084', NULL, '2ebb34aa80cde666bae90bc287c3cba7', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S67EQ8EQTT2E9E3CGTXS7W', 'task', '06G1S392DEG41MRGEFSYWJWP2W', 'The reach entrypoint: [project.scripts] plus typer and pydantic', 'pyproject.toml currently has NO [project.scripts] at all — no console entrypoint is defined anywhere. Adding one plus the two dependencies is the whole delta. Deliverables: [project.scripts] reach = tooling.main:cli; typer and pydantic added to [project] dependencies, both version-pinned and CVE-checked against NVD plus OSV the way scipy and Pillow already are in this file, with the check date in a comment since that is the established convention here. NOTE the entrypoint target is main:cli not main:app — see the lazy-registration task; the root is a click Group subclass rather than a typer.Typer instance, because a Typer root with zero eagerly-registered commands raises at build time and because keeping typer off the --help path is worth having. Acceptance: .venv/bin/reach --help runs. This is the venv-local proof only; the bare-name-on-PATH guarantee is a separate task and is the one that actually matters. + +DONE 2026-08-23. Entrypoint works: .venv/bin/reach --help runs in 80 ms. But the design in the description above is WRONG in one load-bearing way and had to change. + +TYPER VENDORS CLICK — the click.Group root is impossible. +The description specifies main:cli as a click Group subclass, on the reasoning that it keeps typer off the --help path. That cannot be done. As of typer 0.26.0 Click is VENDORED into typer as typer._click: no top-level click package is installed at all, and the docs are explicit that extracting the internal Click app is no longer supported. A real click.Group root hosting typer sub-apps would mean two different Click implementations in one process. Discovered the direct way — main.py imported click and reach --help died with ModuleNotFoundError. +Replacement, and it is the supported surface: the root is a typer.Typer, and lazy registration goes through typer.Typer(cls=...) with a typer.core.TyperGroup subclass. Verified TyperGroup subclasses typer._click.core.Command and that cls= composes. T-1260 builds the lazy group on that surface, NOT on click.Group. + +THE ZERO-COMMAND CLAIM WAS HALF RIGHT. +The description says a Typer root with zero eagerly-registered commands raises at build time. Tested both ways: with no commands AND no callback it does raise (RuntimeError: Could not get a command for this Typer instance); with a callback and no commands it builds fine and prints help. So the empty root is legal as long as it carries a callback, which main.py now has. That matters because lazy registration means NO command is ever eager. + +rich_markup_mode=None IS LOAD-BEARING, NOT COSMETIC. +An empty --help costs 168 ms with typer default rich rendering and 74 ms with rich disabled — 94 ms, 56 percent of runtime, for an empty CLI. Verified rich and pygments are then absent from sys.modules entirely, so this removes them from the import path rather than merely skipping the render. It also stops typer drawing box-art help, which it does EVEN WHEN STDOUT IS A PIPE — that would have put box-drawing characters into every hook log and agent capture. One line to revert if the boxes are ever worth more than the milliseconds. + +typer-slim IS NOT THE ANSWER (checked, since rich is unused). +Deprecated as of typer 0.22.0 and now a shallow wrapper that installs all of typer, so depending on it buys nothing. Dependency stays plain typer; rich ships transitively but is never imported. + +SECURITY — both direct deps and the transitive set checked 2026-08-23 against NVD, OSV and the GitHub Advisory Database: +- typer 0.27.1: no advisories on record at all. +- pydantic 2.13.4: clean. PYSEC-2026-1812 / CVE-2024-3772 (email-regex ReDoS) is fixed in 2.4.0. NOTE the 2026 SSRF advisories CVE-2026-25580 and CVE-2026-54249 are against pydantic-ai, a DIFFERENT package that is not a dependency here — recorded in pyproject so the next sweep does not re-panic. +- transitive: pygments 2.21.0 clears CVE-2026-4539 (archetype-lexer ReDoS, fixed 2.20.0); rich and click have no advisories on record. + +TIMING, for the record: 80 ms for reach --help with zero domains registered. Interpreter startup is 27 ms of that and typer import is ~42 ms. Against the D-263 ceiling — raised to 500 ms on 2026-08-23 — that leaves real headroom for the domains to come. + +BARE `reach` EXITS 2. no_args_is_help=True prints help and exits 2 (Click usage-error convention). Flagging rather than changing it: that is standard for no-command-given, and D-263''s exit-code contract is about FAILURES carrying a remedy, which this is not. Revisit in T-1260 if the domain list should exit 0.', 'in_progress', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:19.770', '2026-08-23 12:00:53.737', NULL, '1b39db8971c19cd69a747c87014ed058', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; +INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G1S685APSPZRXSM9F62DTKGM', 'task', '06G1S392DEG41MRGEFSYWJWP2W', 'Lazy domain registration — --help must not import the tree', 'reach --help must list every domain WITHOUT importing any of them. This is not an optimisation; it is the mechanism that keeps the 104 ms push-gate budget reachable, and it has one specific trap. Click Group.format_commands calls get_command for every subcommand to render its short help — which imports all 123 modules just to print help. So format_commands MUST be overridden to read short help from a declaration table rather than from the loaded command. Deliverables: tooling/main.py holding a DOMAINS table mapping name to (import_path, short_help); a LazyDomainGroup(click.Group) overriding list_commands, get_command and format_commands; get_command resolving via importlib and typer.main.get_command only for the domain actually invoked. main.py is a router and nothing else — no logic, no I/O, no pydantic, no domain imports at module level. Acceptance is a TEST, not an eyeball: run reach --help in a subprocess and assert no tooling.domains..service module is in sys.modules afterwards. That test is the whole point of the ticket and belongs with it, not in E8. + +AMENDED 2026-08-20 — the justification changes, the requirement does not. Lazy registration is still mandatory, but NOT because of a 104 ms parity budget (withdrawn — see D-263 amendment and T-1251). It is mandatory because of what an eager entrypoint would actually cost: measured in the repo venv, scipy.ndimage 275 ms, pydantic 87 ms, numpy 72 ms, PIL.Image 29 ms — about 460 ms paid before executing a line of its own, growing every time a domain is added. The format_commands trap and the sys.modules acceptance test are unchanged and are the heart of this ticket. + +AMENDED 2026-08-23 from T-1259 — the base class in the description is WRONG and this ticket cannot be built as written. + +The description specifies LazyDomainGroup(click.Group). There is no top-level click package: typer VENDORS click as of 0.26.0 (as typer._click), and the docs are explicit that extracting the internal Click app is unsupported. Adding click as a direct dependency would be worse, not better — two Click implementations in one process, with typer.main.get_command returning a typer._click Command that a real click.Group cannot host. + +CORRECTED DESIGN, verified working in T-1259: +- The root is `cli = typer.Typer(cls=LazyDomainGroup, ...)` in tooling/main.py — already in place. +- LazyDomainGroup subclasses **typer.core.TyperGroup**, not click.Group. Confirmed TyperGroup''s MRO is (TyperGroup, typer._click.core.Command, ABC) and that cls= composes. +- Override list_commands, get_command and format_commands on that class. The format_commands trap is UNCHANGED and is still the heart of this ticket: the base implementation calls get_command for every subcommand to render short help, which would import every domain just to print --help. It must read short help from the DOMAINS declaration table instead. +- get_command resolves the invoked domain via importlib and converts the domain''s Typer sub-app with typer.main.get_command. +- The sys.modules acceptance test is unchanged and is still the point: after `reach --help`, no domain service module may appear in sys.modules. Extend it to assert `rich` and `pygments` are absent too — that is currently true and is worth pinning, since it is one careless kwarg away from regressing. + +ALREADY DONE IN T-1259, do not redo: +- main.py exists with the Typer root and a callback. The callback is REQUIRED: a Typer root with no commands and no callback raises RuntimeError("Could not get a command for this Typer instance") at build time, and lazy registration means no command is ever eager. +- rich_markup_mode=None is set and is load-bearing — worth 94 ms of a 168 ms empty --help, and it keeps rich and pygments off the import path. Do not remove it while adding the lazy group. + +OPEN QUESTION carried here: bare `reach` currently exits 2 (Click''s usage-error convention via no_args_is_help). Once domains are registered and a bare `reach` prints a useful domain list, decide whether that should exit 0 instead. It is not a failure, and D-263''s exit-code contract concerns failures carrying a remedy. + +BUDGET NOTE: the D-263 ceiling was raised to ~500 ms on 2026-08-23. Current cost of `reach --help` with zero domains is 80 ms. Stay smart about what loads; do not contort for tightness.', 'backlog', 'high', NULL, NULL, 'D-263', '2026-08-20 00:36:25.557', '2026-08-23 12:01:12.824', NULL, '9f7e2efceadf8ec6d46fd5094d1564ed', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at; diff --git a/governance/decisions/architecture.md b/governance/decisions/architecture.md index 2b03da8bc..ef33ba7d9 100644 --- a/governance/decisions/architecture.md +++ b/governance/decisions/architecture.md @@ -2546,7 +2546,7 @@ Technical foundation decisions that constrain implementation: engine, client-ser | language | **Python.** The friction is packaging, not language — see below. | | command name | **`reach`.** Free on PATH; `Bash(reach *)` becomes the single allowlist entry. | | package home | **`tooling/` itself is the package** (`tooling/__init__.py`, imported as `tooling.*`), with every domain under `tooling/domains/`. Fewest path rewrites across the 160 markdown files that name `tooling/…`. | -| dependencies added | **`typer`** (transport) and **`pydantic`** (data shapes) join `pyproject.toml`. Neither may be imported at module level in `main.py`. | +| dependencies added | **`typer`** (transport) and **`pydantic`** (data shapes) join `pyproject.toml`. pydantic may not be imported at module level in `main.py`. **typer necessarily is** — see the vendoring note below. | | numerics | **They move too, and stay Python.** `planet-gen`, `garment-fit`, `economy-db` and `blender` all come inside. No numerical-equivalence problem is created because no numerical code is rewritten. | **Why not Rust — the friction Q-124 names is packaging, and Rust is not the cheapest fix for any of it.** @@ -2596,6 +2596,7 @@ tooling/ - `@logged` (`core/logging.py`) emits one structured line per invocation — command, arguments, duration, outcome — through the shared logger. **To stderr, never stdout**, so machine-readable output stays parseable, and quiet by default so hooks are not spammed. - They compose into a single `@command` decorator so no command can carry one without the other, and **the conformance test asserts every registered command carries it.** A cross-cutting concern applied by hand is a cross-cutting concern applied to 90% of cases. - **Pydantic is the data-shape vocabulary, and it lives in `domains/*/schemas.py`.** Available to every domain, including the gate domain. `main.py` and `core/` stay pydantic-free — not for the milliseconds, but because `main.py` is a router and `core/` is a substrate, and neither has data shapes of its own. +- **Typer vendors Click, so the root group is a Typer** *(found 2026-08-23 while building T-1259)*. As of **typer 0.26.0** Click is vendored: there is no top-level `click` package installed, and the docs state that "extracting the internal Click app" is no longer supported. The plan of a `click.Group` root hosting Typer sub-apps — which would have kept typer off the `--help` path — is therefore impossible; it would mean two Click implementations in one process. Lazy registration goes through the supported `typer.Typer(cls=...)` surface with a `TyperGroup` subclass instead. Two consequences worth recording: a Typer root with **no commands and no callback raises at build time** (`Could not get a command for this Typer instance`), so the empty root needs a callback; and **`rich_markup_mode=None` is load-bearing, not cosmetic** — it removes `rich` and `pygments` from the import path and takes an empty `--help` from 168 ms to 74 ms. Also: **`typer-slim` is not the answer** — deprecated as of 0.22.0 and now a shallow wrapper that installs all of typer. - **The import rule that actually matters: nothing heavy at module level in `main.py` or any `router.py`.** Measured 2026-08-20 in the repo venv: `scipy.ndimage` **275 ms**, `pydantic` 87 ms, `numpy` 72 ms, `PIL.Image` 29 ms. An entrypoint that eagerly imported the tree would pay **~460 ms before executing a line of its own** — and *that*, not pydantic, is what lazy registration exists to prevent. Heavy imports belong inside a service, or inside the function that needs them. - **Enforced, not asked for:** no `typer`/`click` import outside `main.py` and `router.py`; no bare `print` outside `core/console.py`; no heavy import (numpy, scipy, PIL, pydantic) reachable from `main.py`; every command carries `@command`. All four are grep-shaped or import-graph-shaped, and belong in the conformance test alongside the help/failure checks. The **import-graph** assertion is the one worth writing carefully — a wall-clock assertion is flaky on a loaded machine and tells you *that* something got slow rather than *what*, whereas asserting `sys.modules` after `reach --help` names the offender directly. - **Not every domain needs every file.** `schemas.py`/`dependencies.py` appear when a domain has data shapes or collaborators worth naming; `check/` may be a router and a service and nothing else. The layering is a vocabulary, not a quota — a folder of five empty modules is worse than a folder of two full ones. @@ -2625,7 +2626,7 @@ tooling/ **Constraints on execution (these are why the work is sequenced, not why it is hard):** - **The acceptance criterion is output parity, not timing parity** *(amended 2026-08-20, same day: the original text set the ported gates a hard ceiling of ~104 ms — the summed single-sample cost of the three unconditional checks — and that was wrong in kind. It imported "do not regress" as a requirement without asking who pays.)* **Who pays is the pre-push hook, and almost nobody else.** On a push touching `server/` or `client/` the hook runs `cargo test` or the gdUnit4 suite — minutes — so a few hundred milliseconds is invisible. On a governance-only push the whole hook is about a second. No human and no loop consumes these often enough for 100 ms versus 400 ms to register. **So a ported check must produce the same output and the same exit code as the script it replaces; it is not required to be as fast.** - **The budget that replaces it is a ceiling with headroom, not a ratchet:** a `reach` invocation should feel instant to a human — **under ~250 ms** — and the unconditional gate set should stay **comfortably under a second**. That is loose enough that pydantic, a subprocess, or a DB open are all affordable, and tight enough that nobody imports scipy at module level. **Lazy registration stays mandatory**, justified by the real threat rather than by parity: an eager entrypoint would pay ~460 ms of numpy + scipy + PIL + pydantic before executing a line of its own, and would grow every time a domain was added. + **The budget that replaces it is a ceiling with headroom, not a ratchet:** a `reach` invocation should stay **under ~500 ms** *(raised from 250 ms, 2026-08-23 — a ceiling is not a typical, and most invocations sit far below it; the tighter number was buying discipline that the import-graph assertion enforces better anyway)*. Stay smart about what gets loaded; stop worrying about tightness. That is loose enough that pydantic, a subprocess, or a DB open are all affordable, and tight enough that nobody imports scipy at module level. **Lazy registration stays mandatory**, justified by the real threat rather than by parity: an eager entrypoint would pay ~460 ms of numpy + scipy + PIL + pydantic before executing a line of its own, and would grow every time a domain was added. - **The `systems.db` stamp survives the move or the move does not land.** `tooling/generator_sources.py` SHAs the concatenated bytes of the generator's sources **sorted by path**, so renaming a file changes the stamp even when its content is byte-identical. The generator relocation must therefore land as **one commit** — registry paths updated, `make regen-db` run, stamp verified — never split across pushes, or the pre-push gate rejects an intermediate state that is in fact correct. - **Old paths are retired through a deprecation window, not deleted under the callers.** 160 markdown files under `.claude/` and `docs/`, 84 make targets, the pre-push hook and the skills all name `tooling/…` paths. Each retired path leaves a shim that prints the new command and exits non-zero — the failure contract applied to the migration itself — before the shims are removed. - **What this decision does not claim.** It does not make make-target invocation cheaper (already free), and it does not make the tooling faster to *run* — only to start, find, and be allowed to call. The wins are: ad-hoc and direct invocation stop prompting, `--help` answers "what tooling exists" without an `ls`, arguments become expressible where make could not express them, the venv split disappears, and failures carry their own remedy. diff --git a/pyproject.toml b/pyproject.toml index 36079e5f1..4a89f8bab 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -11,8 +11,38 @@ dependencies = [ # Pillow 12.2.0 — checked clean against NVD + OSV (2026-04-06) # CVE-2026-25990 fixed in 12.1.1, CVE-2025-48379 fixed in 11.3.0 "Pillow==12.2.0", + # typer 0.27.1 — the CLI transport (D-263). Checked clean against NVD, OSV + # and the GitHub Advisory Database: no advisories on record for typer at all + # (2026-08-20). Its transitive set was checked too, since typer pulls in + # rich -> pygments: pygments 2.21.0 clears CVE-2026-4539 (archetype-lexer + # ReDoS, fixed in 2.20.0); rich and click have no advisories on record. + # + # Plain `typer`, not `typer-slim`: slim is deprecated as of typer 0.22.0 and + # is now a shallow wrapper that installs all of typer, so it buys nothing. + # rich therefore ships as a transitive dependency — but `rich_markup_mode=None` + # in tooling/main.py keeps it off the import path entirely (verified: `rich` + # and `pygments` are absent from sys.modules after loading the CLI). + "typer==0.27.1", + # pydantic 2.13.4 — data shapes for domain schemas (D-263). Checked clean + # against NVD + OSV (2026-08-20). PYSEC-2026-1812 / CVE-2024-3772 (email + # regex ReDoS) is fixed in 2.4.0. NOTE the 2026 SSRF advisories + # CVE-2026-25580 and CVE-2026-54249 are against *pydantic-ai*, a different + # package that is not a dependency here — do not confuse the two on the + # next sweep. + "pydantic==2.13.4", ] +[project.scripts] +# The whole point of D-263: one bare command, so one permission-rule entry +# covers every tool. +# +# `cli` is a typer.Typer instance (callable), NOT the click.Group that T-1259 +# originally specified: typer vendors click as of 0.26.0, so there is no +# top-level `click` to import and no supported way to extract typer's internal +# one. Lazy domain registration therefore goes through `typer.Typer(cls=...)` +# with a TyperGroup subclass (T-1260) rather than a click Group. +reach = "tooling.main:cli" + [tool.setuptools.packages.find] # Explicit, not flat-layout auto-discovery. The repo root holds client/, server/, # docs/, wiki/, db/ and tests/ alongside tooling/, and auto-discovery either diff --git a/tooling/main.py b/tooling/main.py new file mode 100644 index 000000000..f71d296d4 --- /dev/null +++ b/tooling/main.py @@ -0,0 +1,50 @@ +"""`reach` — one command for every repo tool (D-263). + +**This file is a router and nothing else.** No logic, no I/O, no pydantic, no +domain imports at module level. It is the file most likely to accumulate "just +one small thing", and the only defence is that it stays short enough that an +addition is obvious in review. + +The root is a `typer.Typer`, and the reason is worth recording because the +original plan was different. T-1259 specified a `click.Group` root, on the +theory that it would keep typer off the `reach --help` path. **That is no longer +possible: typer vendors click as of 0.26.0** — there is no top-level `click` +package to import, and the docs are explicit that "extracting the internal Click +app" is unsupported. Mixing a real `click.Group` root with typer sub-apps would +mean two different Click implementations in one process. + +So the customisation surface is `typer.Typer(cls=...)` with a `TyperGroup` +subclass, which is the supported path and is what T-1260 uses to register +domains lazily. + +`rich_markup_mode=None` is not a style preference — it is worth 94 ms of the +168 ms an empty `--help` otherwise costs, and it keeps `rich` and `pygments` +off the import path entirely (verified absent from `sys.modules`). It also +stops typer drawing box-art help, which it does **even when stdout is a pipe**, +so hook logs and agent output stay readable. One line to revert if the boxes +are ever wanted more than the milliseconds. + +The callback below is not decoration. A `typer.Typer` with **no commands and no +callback** raises `RuntimeError: Could not get a command for this Typer +instance` at build time; with a callback it builds fine and prints help. Since +domains are registered lazily and none are eager, the callback is what makes an +empty root legal. +""" + +from __future__ import annotations + +import typer + +cli = typer.Typer( + name="reach", + help="Repo tooling for The Settled Reach.\n\nRun `reach --help` to see what a domain can do.", + no_args_is_help=True, + add_completion=False, + rich_markup_mode=None, + context_settings={"help_option_names": ["-h", "--help"], "max_content_width": 100}, +) + + +@cli.callback() +def root() -> None: + """Present so an empty root is legal — see the module docstring."""