test(config): T-1281 — canvas-version parity, using its own history as fixture
Five cases, including the failure path that matters — this is the gate with five shipped regressions behind it. A fixture repo does not work here, and finding out why exposed a real inconsistency in the port: canvas_sources globs from a __file__-derived root, so under SR_REPO_ROOT the service would diff the fixture while globbing the real tree. Git goes through config.repo_root(); the registry does not. Harmless in production since they are the same repo, but it is the same no-root-override asymmetry the domain map noted about the old scripts, now inside the new code. Not fixed here — making it dynamic means restructuring six module-level constants in a module the old script still imports. Real history is the better fixture anyway: both implementations see identical input, nothing is mutated, and nothing can drift from the thing it models. Two of the cases are genuine historical instances of the regression this gate exists to catch — T-1237 and T-1194 both changed canvas generation and were bumped only after the fact. The history that produced the check, used as its own test. The test also asserts no changed file is dropped from the failure message. That list is the actionable half; "something changed" without saying what leaves the reader to re-derive the intersection by hand. Proven to fail by truncating the touched-file set, which reported both the exit-code divergence and all three omitted filenames by name. Each case also asserts the OLD script still behaves as the case claims, so a rewritten history would say so rather than silently checking nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -2204,3 +2204,91 @@ A BUG CAUGHT BY RUNNING IT: the check router had dropped its `import typer` in T
|
||||
STILL OPEN on this ticket:
|
||||
- No parity tests for canvas-version, dataflow-graph or systems-db-stamp. All three were verified side by side on the live tree, which proves the happy path only. canvas-version''s failure path is the valuable one and needs a git fixture — a temp repo with a commit touching a registry file and no version bump.
|
||||
- Retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.', NULL, '2026-08-31 16:22:11', '2026-08-31 16:22:11.006', '2026-08-31 16:22:11.006', NULL, 'ffbaaabe80ba8e200132ab7fcd391775', 2) ON CONFLICT(hash) DO NOTHING;
|
||||
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G5GWCTS89VSST07DFVBVG064', 'description', 'Finish what T-1262 started. domains/check/ currently has one verb (client-version); the other four gates join it: check-canvas-version, check-systems-db-stamp, check-fact-ids and check-dataflow-graph.py. Per the domain map, canvas_sources.py and generator_sources.py move alongside as module-level REGISTRIES, not verbs — nothing types them, and putting them in reach --help would answer a question nobody has. schema_version.py goes to the db domain where it is defined; check imports it, which is what a service layer is for. STANDARD PORT ACCEPTANCE, applying to every per-domain ticket under this epic: (a) service.py holds the logic and is transport-agnostic — no printing, no sys.exit, no typer; (b) router.py holds no logic and every command wears @command; (c) a parity test per gate, copying the pattern in tooling/test_check_parity.py, which builds a throwaway fixture repo and runs BOTH implementations — exit codes must match exactly and no fact the old message carried may be lost; (d) the old scripts are RETIRED once parity passes, not left alongside; (e) that domain''s tooling make targets are retired too rather than left as wrappers, per the D-263 make/reach split — build orchestration stays with make. Note check-dataflow-graph.py and check-canvas-version both parse git output, so their fixtures need a git repo rather than a bare directory; that is the one thing here harder than client-version was.
|
||||
|
||||
PROGRESS 2026-08-31 — fact-ids ported, the first bash-to-Python rewrite under the D-263 decision. It was 89 lines of grep/sed pipeline; it is now service.fact_ids() returning a FactIdCheck, with a router that renders it. PARITY IS EXACT ON THE LIVE TREE: both implementations print ''check-fact-ids: OK — 6 references validated against 61 canonical facts'' and exit 0. The matching COUNTS are the real evidence — if the line-matching regex differed from the grep chain even slightly, 6 and 61 would diverge. Kept line-matched rather than YAML-parsed on purpose: parsing properly would change which lines count (anchors, merge keys, multi-document files would start contributing ids the old check never saw), which is a different check wearing the same name. Three parity cases added to test_check_parity.py — ok, unknown fact_id, and the advisory mode where catalogs are unpopulated. Proven to fail: removing the entity-attributes.yaml exclusion was caught, and interestingly by the ADVISORY case rather than the exclusion assertion — including that file made the catalog non-empty, so the new implementation went to enforcing mode while the old stayed advisory. A real behavioural divergence, caught by exit code. RETIREMENT DEFERRED to the end of this ticket, and there is a reason worth recording. The rule is retire per DOMAIN, and three gates remain. It also resolves a tension: the parity test COPIES the old script into its fixture, so deleting the script early would delete the test''s own subject. A parity test is scaffolding with a defined lifetime — when the old script goes, its parity case goes with it and is replaced by a unit test of the new behaviour. Same will apply to every port under T-1250. NOTE the pre-commit hook calls tooling/check-fact-ids directly, so retirement means editing that hook — and a mistake there is felt on every commit rather than every push. Do it together with validate-content (T-1282), which is in the same hook, so it is edited once.
|
||||
|
||||
PROGRESS 2026-08-31, second increment — dataflow-graph and systems-db-stamp ported. Both were already Python, so these are moves rather than rewrites, and both produce BYTE-IDENTICAL output to their originals on the live tree: "check-dataflow-graph: OK — 1 diagram(s)" and "check-systems-db-stamp: OK — 1 generator(s) up to date", same exit codes.
|
||||
|
||||
CORRECTION TO THIS TICKET''S OWN NOTE: it says check-dataflow-graph.py parses git output. It does not — it reads .d2 files and globs the filesystem. Only check-canvas-version parses git, so only that one needs a git repo in its fixture.
|
||||
|
||||
THE E402 DEBT EVAPORATED, exactly as T-1274 predicted. check-systems-db-stamp reached generator_sources through sys.path.insert plus a noqa: E402, because tooling/ was not a package. It now imports as `from tooling import generator_sources` with no hack and no suppression. That is the first concrete evidence for T-1274''s premise; the map''s plan to move generator_sources into the db domain still stands, but it is already importable where it sits.
|
||||
|
||||
SIX FAILURE MODES PRESERVED in the stamp port, modelled as a StampState enum rather than collapsed to pass/fail, because they carry different remedies and one carries a different EXIT CODE: UNSTAMPED exits 2 while every other failure exits 1, and the pre-push hook has relied on that distinction since T-857. The states are ok / absent / unstamped / bad_version / conflict / unknown / broken / stale, each with its own remedy string.
|
||||
|
||||
ONE DELIBERATE BEHAVIOURAL DIFFERENCE, flagged rather than hidden: the old stamp script was SILENT on success unless given --verbose; the new one always prints its verdict. That is not a lost fact, so it does not violate the parity definition, and it makes the gate consistent with client-version and dataflow-graph, which both always print. The old script was the odd one out. Its per-command --verbose also disappears in favour of the global one, which is the consolidation the whole initiative is for.
|
||||
|
||||
STILL OPEN on this ticket, and honest about it:
|
||||
- check-canvas-version is not ported. It is the one that genuinely parses git, so its fixture needs a real git repo — the hardest of the four.
|
||||
- NO PARITY TESTS yet for dataflow-graph or systems-db-stamp. Both were verified by running the two implementations side by side on the live tree, which proves the happy path only. The stamp gate''s six failure modes are entirely unexercised, and its fixture is awkward because the old script imports generator_sources from its own directory, so a fixture must copy both files and build a sqlite DB with a meta table.
|
||||
- Retirement (scripts, parity cases, make targets, the pre-commit hook entry for fact-ids) waits for the whole domain, per the per-domain rule.
|
||||
|
||||
PROGRESS 2026-08-31, third increment — canvas-version ported. ALL FOUR GATES ARE NOW IN THE DOMAIN: client-version, canvas-version, systems-db-stamp, dataflow-graph, fact-ids. Byte-identical output to the original on the live tree.
|
||||
|
||||
FIRST REAL CONSUMER OF core/process.run. The git calls go through the guarded exec with check=False, deliberately: a git failure here is not an error to report but a signal that there is nothing to compare — a fresh clone with no remote is a legitimate state, not a broken one. The argv-list and missing-binary guards still apply.
|
||||
|
||||
TWO SKIPS KEPT DISTINCT FROM A PASS. NO_BASE (base ref absent) and DIFF_FAILED both exit 0, as does CLEAN (nothing relevant changed) — but only CLEAN means the gate actually looked at something. Collapsing them would hide a gate that had silently stopped running, which for this particular check is the exact failure mode it exists to prevent.
|
||||
|
||||
A SECOND RICH PATH FOUND, and it is worth knowing about. typer''s PRETTY-EXCEPTION handler renders unhandled errors as box-art with syntax highlighting, and it is a DIFFERENT mechanism from rich_markup_mode — setting one does nothing about the other. It surfaced when a NameError produced a full-width box-drawn traceback. That is the same log pollution rich_markup_mode=None was set to prevent (T-1259), arriving through another door, and landing in the worst possible place: a hook log at the moment something has already gone wrong. pretty_exceptions_enable=False now set on both the root app and every domain built by cli.domain().
|
||||
|
||||
THE EXISTING TEST MOVED WITH THE CODE. tooling/test_canvas_version_check.py loaded the extensionless script through a SourceFileLoader and reached canvas_sources via sys.path.insert — both only because tooling/ was not an importable package. It now does `from tooling.domains.check import service as check` and `from tooling.canvas_sources import relative_paths`. Second instance of the E402/sys.path debt evaporating on contact (T-1274). The test passes unchanged in what it ASSERTS, which is the point: diff_has_version_bump was kept pure in the port precisely so those six properties still hold without constructing git history.
|
||||
|
||||
A BUG CAUGHT BY RUNNING IT: the check router had dropped its `import typer` in T-1267 when it moved to cli.domain(), and canvas-version needs typer.Option for --base/--head. Restored.
|
||||
|
||||
STILL OPEN on this ticket:
|
||||
- No parity tests for canvas-version, dataflow-graph or systems-db-stamp. All three were verified side by side on the live tree, which proves the happy path only. canvas-version''s failure path is the valuable one and needs a git fixture — a temp repo with a commit touching a registry file and no version bump.
|
||||
- Retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.', 'Finish what T-1262 started. domains/check/ currently has one verb (client-version); the other four gates join it: check-canvas-version, check-systems-db-stamp, check-fact-ids and check-dataflow-graph.py. Per the domain map, canvas_sources.py and generator_sources.py move alongside as module-level REGISTRIES, not verbs — nothing types them, and putting them in reach --help would answer a question nobody has. schema_version.py goes to the db domain where it is defined; check imports it, which is what a service layer is for. STANDARD PORT ACCEPTANCE, applying to every per-domain ticket under this epic: (a) service.py holds the logic and is transport-agnostic — no printing, no sys.exit, no typer; (b) router.py holds no logic and every command wears @command; (c) a parity test per gate, copying the pattern in tooling/test_check_parity.py, which builds a throwaway fixture repo and runs BOTH implementations — exit codes must match exactly and no fact the old message carried may be lost; (d) the old scripts are RETIRED once parity passes, not left alongside; (e) that domain''s tooling make targets are retired too rather than left as wrappers, per the D-263 make/reach split — build orchestration stays with make. Note check-dataflow-graph.py and check-canvas-version both parse git output, so their fixtures need a git repo rather than a bare directory; that is the one thing here harder than client-version was.
|
||||
|
||||
PROGRESS 2026-08-31 — fact-ids ported, the first bash-to-Python rewrite under the D-263 decision. It was 89 lines of grep/sed pipeline; it is now service.fact_ids() returning a FactIdCheck, with a router that renders it. PARITY IS EXACT ON THE LIVE TREE: both implementations print ''check-fact-ids: OK — 6 references validated against 61 canonical facts'' and exit 0. The matching COUNTS are the real evidence — if the line-matching regex differed from the grep chain even slightly, 6 and 61 would diverge. Kept line-matched rather than YAML-parsed on purpose: parsing properly would change which lines count (anchors, merge keys, multi-document files would start contributing ids the old check never saw), which is a different check wearing the same name. Three parity cases added to test_check_parity.py — ok, unknown fact_id, and the advisory mode where catalogs are unpopulated. Proven to fail: removing the entity-attributes.yaml exclusion was caught, and interestingly by the ADVISORY case rather than the exclusion assertion — including that file made the catalog non-empty, so the new implementation went to enforcing mode while the old stayed advisory. A real behavioural divergence, caught by exit code. RETIREMENT DEFERRED to the end of this ticket, and there is a reason worth recording. The rule is retire per DOMAIN, and three gates remain. It also resolves a tension: the parity test COPIES the old script into its fixture, so deleting the script early would delete the test''s own subject. A parity test is scaffolding with a defined lifetime — when the old script goes, its parity case goes with it and is replaced by a unit test of the new behaviour. Same will apply to every port under T-1250. NOTE the pre-commit hook calls tooling/check-fact-ids directly, so retirement means editing that hook — and a mistake there is felt on every commit rather than every push. Do it together with validate-content (T-1282), which is in the same hook, so it is edited once.
|
||||
|
||||
PROGRESS 2026-08-31, second increment — dataflow-graph and systems-db-stamp ported. Both were already Python, so these are moves rather than rewrites, and both produce BYTE-IDENTICAL output to their originals on the live tree: "check-dataflow-graph: OK — 1 diagram(s)" and "check-systems-db-stamp: OK — 1 generator(s) up to date", same exit codes.
|
||||
|
||||
CORRECTION TO THIS TICKET''S OWN NOTE: it says check-dataflow-graph.py parses git output. It does not — it reads .d2 files and globs the filesystem. Only check-canvas-version parses git, so only that one needs a git repo in its fixture.
|
||||
|
||||
THE E402 DEBT EVAPORATED, exactly as T-1274 predicted. check-systems-db-stamp reached generator_sources through sys.path.insert plus a noqa: E402, because tooling/ was not a package. It now imports as `from tooling import generator_sources` with no hack and no suppression. That is the first concrete evidence for T-1274''s premise; the map''s plan to move generator_sources into the db domain still stands, but it is already importable where it sits.
|
||||
|
||||
SIX FAILURE MODES PRESERVED in the stamp port, modelled as a StampState enum rather than collapsed to pass/fail, because they carry different remedies and one carries a different EXIT CODE: UNSTAMPED exits 2 while every other failure exits 1, and the pre-push hook has relied on that distinction since T-857. The states are ok / absent / unstamped / bad_version / conflict / unknown / broken / stale, each with its own remedy string.
|
||||
|
||||
ONE DELIBERATE BEHAVIOURAL DIFFERENCE, flagged rather than hidden: the old stamp script was SILENT on success unless given --verbose; the new one always prints its verdict. That is not a lost fact, so it does not violate the parity definition, and it makes the gate consistent with client-version and dataflow-graph, which both always print. The old script was the odd one out. Its per-command --verbose also disappears in favour of the global one, which is the consolidation the whole initiative is for.
|
||||
|
||||
STILL OPEN on this ticket, and honest about it:
|
||||
- check-canvas-version is not ported. It is the one that genuinely parses git, so its fixture needs a real git repo — the hardest of the four.
|
||||
- NO PARITY TESTS yet for dataflow-graph or systems-db-stamp. Both were verified by running the two implementations side by side on the live tree, which proves the happy path only. The stamp gate''s six failure modes are entirely unexercised, and its fixture is awkward because the old script imports generator_sources from its own directory, so a fixture must copy both files and build a sqlite DB with a meta table.
|
||||
- Retirement (scripts, parity cases, make targets, the pre-commit hook entry for fact-ids) waits for the whole domain, per the per-domain rule.
|
||||
|
||||
PROGRESS 2026-08-31, third increment — canvas-version ported. ALL FOUR GATES ARE NOW IN THE DOMAIN: client-version, canvas-version, systems-db-stamp, dataflow-graph, fact-ids. Byte-identical output to the original on the live tree.
|
||||
|
||||
FIRST REAL CONSUMER OF core/process.run. The git calls go through the guarded exec with check=False, deliberately: a git failure here is not an error to report but a signal that there is nothing to compare — a fresh clone with no remote is a legitimate state, not a broken one. The argv-list and missing-binary guards still apply.
|
||||
|
||||
TWO SKIPS KEPT DISTINCT FROM A PASS. NO_BASE (base ref absent) and DIFF_FAILED both exit 0, as does CLEAN (nothing relevant changed) — but only CLEAN means the gate actually looked at something. Collapsing them would hide a gate that had silently stopped running, which for this particular check is the exact failure mode it exists to prevent.
|
||||
|
||||
A SECOND RICH PATH FOUND, and it is worth knowing about. typer''s PRETTY-EXCEPTION handler renders unhandled errors as box-art with syntax highlighting, and it is a DIFFERENT mechanism from rich_markup_mode — setting one does nothing about the other. It surfaced when a NameError produced a full-width box-drawn traceback. That is the same log pollution rich_markup_mode=None was set to prevent (T-1259), arriving through another door, and landing in the worst possible place: a hook log at the moment something has already gone wrong. pretty_exceptions_enable=False now set on both the root app and every domain built by cli.domain().
|
||||
|
||||
THE EXISTING TEST MOVED WITH THE CODE. tooling/test_canvas_version_check.py loaded the extensionless script through a SourceFileLoader and reached canvas_sources via sys.path.insert — both only because tooling/ was not an importable package. It now does `from tooling.domains.check import service as check` and `from tooling.canvas_sources import relative_paths`. Second instance of the E402/sys.path debt evaporating on contact (T-1274). The test passes unchanged in what it ASSERTS, which is the point: diff_has_version_bump was kept pure in the port precisely so those six properties still hold without constructing git history.
|
||||
|
||||
A BUG CAUGHT BY RUNNING IT: the check router had dropped its `import typer` in T-1267 when it moved to cli.domain(), and canvas-version needs typer.Option for --base/--head. Restored.
|
||||
|
||||
STILL OPEN on this ticket:
|
||||
- No parity tests for canvas-version, dataflow-graph or systems-db-stamp. All three were verified side by side on the live tree, which proves the happy path only. canvas-version''s failure path is the valuable one and needs a git fixture — a temp repo with a commit touching a registry file and no version bump.
|
||||
- Retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.
|
||||
|
||||
PROGRESS 2026-08-31, fourth increment — canvas-version parity, five cases, including its failure path.
|
||||
|
||||
A FIXTURE REPO DOES NOT WORK FOR THIS GATE, and finding out why exposed a real inconsistency in the port. canvas_sources.relative_paths() globs from a __file__-derived REPO_ROOT, so under SR_REPO_ROOT the new service would DIFF the fixture while GLOBBING the real tree. The service resolves git through config.repo_root() but the registry does not — the two disagree about which repo is being checked. Harmless in production (they are the same repo) but it makes fixture-based parity impossible, and it is the same asymmetry the domain map already noted about old scripts having no root override, showing up inside the new code.
|
||||
NOT FIXED HERE: making it dynamic means restructuring six module-level path constants in a module the old script also imports, which is more churn than this ticket should carry. Filed as a follow-up consideration for when canvas_sources moves into the domain per the map.
|
||||
|
||||
SO PARITY USES REAL HISTORY INSTEAD, which is better anyway: both implementations see byte-identical input, nothing is mutated, and no fixture can drift from the thing it models. Five cases:
|
||||
- canvas-needs-bump: 4e503c356 (T-1237 one-course-per-river) — changed canvas generation, did NOT bump. Both exit 1.
|
||||
- canvas-needs-bump-2: 566b56651 (T-1194 relief/vegetation texture) — same. Both exit 1.
|
||||
- canvas-bumped: 9b146f9e1 — changed canvas AND bumped. Both exit 0.
|
||||
- canvas-clean: 6b31111cd — governance only. Both exit 0.
|
||||
- canvas-no-base: a nonexistent base ref — a SKIP, not a failure. Both exit 0.
|
||||
The two failure cases are GENUINE historical instances of the regression this gate exists to catch, both bumped after the fact. That is the history that produced the check, used as its own test.
|
||||
|
||||
THE TEST ALSO ASSERTS NO CHANGED FILE IS DROPPED from the failure message. The list of touched files is the actionable half — a failure saying "something changed" without saying what leaves the reader to re-derive the intersection by hand.
|
||||
|
||||
PROVEN TO FAIL: truncating the touched-file set made canvas-needs-bump report both the exit-code divergence AND all three omitted filenames by name.
|
||||
|
||||
SHAs are hardcoded because history is immutable, and each case asserts the OLD script still behaves as the case claims — so if a rebase ever invalidated a range, the test says so by name rather than silently checking nothing.
|
||||
|
||||
STILL OPEN on this ticket: parity tests for dataflow-graph and systems-db-stamp (happy path verified side by side only), and retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.', NULL, '2026-08-31 18:07:14', '2026-08-31 18:07:14.123', '2026-08-31 18:07:14.123', NULL, 'eacbb029d58e1ea83aae262aed0c8e89', 2) ON CONFLICT(hash) DO NOTHING;
|
||||
|
||||
@@ -2514,3 +2514,58 @@ A BUG CAUGHT BY RUNNING IT: the check router had dropped its `import typer` in T
|
||||
STILL OPEN on this ticket:
|
||||
- No parity tests for canvas-version, dataflow-graph or systems-db-stamp. All three were verified side by side on the live tree, which proves the happy path only. canvas-version''s failure path is the valuable one and needs a git fixture — a temp repo with a commit touching a registry file and no version bump.
|
||||
- Retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.', 'in_progress', 'high', NULL, NULL, 'D-263', '2026-08-31 15:30:35.338', '2026-08-31 16:22:11.006', NULL, '3e7813e253aead4835ce00f1f3533abd', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
||||
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06G5GWCTS89VSST07DFVBVG064', 'task', '06G1S3D0M1TQW0GMFBBPQZG3ZM', 'Port the check domain — the four remaining gates', 'Finish what T-1262 started. domains/check/ currently has one verb (client-version); the other four gates join it: check-canvas-version, check-systems-db-stamp, check-fact-ids and check-dataflow-graph.py. Per the domain map, canvas_sources.py and generator_sources.py move alongside as module-level REGISTRIES, not verbs — nothing types them, and putting them in reach --help would answer a question nobody has. schema_version.py goes to the db domain where it is defined; check imports it, which is what a service layer is for. STANDARD PORT ACCEPTANCE, applying to every per-domain ticket under this epic: (a) service.py holds the logic and is transport-agnostic — no printing, no sys.exit, no typer; (b) router.py holds no logic and every command wears @command; (c) a parity test per gate, copying the pattern in tooling/test_check_parity.py, which builds a throwaway fixture repo and runs BOTH implementations — exit codes must match exactly and no fact the old message carried may be lost; (d) the old scripts are RETIRED once parity passes, not left alongside; (e) that domain''s tooling make targets are retired too rather than left as wrappers, per the D-263 make/reach split — build orchestration stays with make. Note check-dataflow-graph.py and check-canvas-version both parse git output, so their fixtures need a git repo rather than a bare directory; that is the one thing here harder than client-version was.
|
||||
|
||||
PROGRESS 2026-08-31 — fact-ids ported, the first bash-to-Python rewrite under the D-263 decision. It was 89 lines of grep/sed pipeline; it is now service.fact_ids() returning a FactIdCheck, with a router that renders it. PARITY IS EXACT ON THE LIVE TREE: both implementations print ''check-fact-ids: OK — 6 references validated against 61 canonical facts'' and exit 0. The matching COUNTS are the real evidence — if the line-matching regex differed from the grep chain even slightly, 6 and 61 would diverge. Kept line-matched rather than YAML-parsed on purpose: parsing properly would change which lines count (anchors, merge keys, multi-document files would start contributing ids the old check never saw), which is a different check wearing the same name. Three parity cases added to test_check_parity.py — ok, unknown fact_id, and the advisory mode where catalogs are unpopulated. Proven to fail: removing the entity-attributes.yaml exclusion was caught, and interestingly by the ADVISORY case rather than the exclusion assertion — including that file made the catalog non-empty, so the new implementation went to enforcing mode while the old stayed advisory. A real behavioural divergence, caught by exit code. RETIREMENT DEFERRED to the end of this ticket, and there is a reason worth recording. The rule is retire per DOMAIN, and three gates remain. It also resolves a tension: the parity test COPIES the old script into its fixture, so deleting the script early would delete the test''s own subject. A parity test is scaffolding with a defined lifetime — when the old script goes, its parity case goes with it and is replaced by a unit test of the new behaviour. Same will apply to every port under T-1250. NOTE the pre-commit hook calls tooling/check-fact-ids directly, so retirement means editing that hook — and a mistake there is felt on every commit rather than every push. Do it together with validate-content (T-1282), which is in the same hook, so it is edited once.
|
||||
|
||||
PROGRESS 2026-08-31, second increment — dataflow-graph and systems-db-stamp ported. Both were already Python, so these are moves rather than rewrites, and both produce BYTE-IDENTICAL output to their originals on the live tree: "check-dataflow-graph: OK — 1 diagram(s)" and "check-systems-db-stamp: OK — 1 generator(s) up to date", same exit codes.
|
||||
|
||||
CORRECTION TO THIS TICKET''S OWN NOTE: it says check-dataflow-graph.py parses git output. It does not — it reads .d2 files and globs the filesystem. Only check-canvas-version parses git, so only that one needs a git repo in its fixture.
|
||||
|
||||
THE E402 DEBT EVAPORATED, exactly as T-1274 predicted. check-systems-db-stamp reached generator_sources through sys.path.insert plus a noqa: E402, because tooling/ was not a package. It now imports as `from tooling import generator_sources` with no hack and no suppression. That is the first concrete evidence for T-1274''s premise; the map''s plan to move generator_sources into the db domain still stands, but it is already importable where it sits.
|
||||
|
||||
SIX FAILURE MODES PRESERVED in the stamp port, modelled as a StampState enum rather than collapsed to pass/fail, because they carry different remedies and one carries a different EXIT CODE: UNSTAMPED exits 2 while every other failure exits 1, and the pre-push hook has relied on that distinction since T-857. The states are ok / absent / unstamped / bad_version / conflict / unknown / broken / stale, each with its own remedy string.
|
||||
|
||||
ONE DELIBERATE BEHAVIOURAL DIFFERENCE, flagged rather than hidden: the old stamp script was SILENT on success unless given --verbose; the new one always prints its verdict. That is not a lost fact, so it does not violate the parity definition, and it makes the gate consistent with client-version and dataflow-graph, which both always print. The old script was the odd one out. Its per-command --verbose also disappears in favour of the global one, which is the consolidation the whole initiative is for.
|
||||
|
||||
STILL OPEN on this ticket, and honest about it:
|
||||
- check-canvas-version is not ported. It is the one that genuinely parses git, so its fixture needs a real git repo — the hardest of the four.
|
||||
- NO PARITY TESTS yet for dataflow-graph or systems-db-stamp. Both were verified by running the two implementations side by side on the live tree, which proves the happy path only. The stamp gate''s six failure modes are entirely unexercised, and its fixture is awkward because the old script imports generator_sources from its own directory, so a fixture must copy both files and build a sqlite DB with a meta table.
|
||||
- Retirement (scripts, parity cases, make targets, the pre-commit hook entry for fact-ids) waits for the whole domain, per the per-domain rule.
|
||||
|
||||
PROGRESS 2026-08-31, third increment — canvas-version ported. ALL FOUR GATES ARE NOW IN THE DOMAIN: client-version, canvas-version, systems-db-stamp, dataflow-graph, fact-ids. Byte-identical output to the original on the live tree.
|
||||
|
||||
FIRST REAL CONSUMER OF core/process.run. The git calls go through the guarded exec with check=False, deliberately: a git failure here is not an error to report but a signal that there is nothing to compare — a fresh clone with no remote is a legitimate state, not a broken one. The argv-list and missing-binary guards still apply.
|
||||
|
||||
TWO SKIPS KEPT DISTINCT FROM A PASS. NO_BASE (base ref absent) and DIFF_FAILED both exit 0, as does CLEAN (nothing relevant changed) — but only CLEAN means the gate actually looked at something. Collapsing them would hide a gate that had silently stopped running, which for this particular check is the exact failure mode it exists to prevent.
|
||||
|
||||
A SECOND RICH PATH FOUND, and it is worth knowing about. typer''s PRETTY-EXCEPTION handler renders unhandled errors as box-art with syntax highlighting, and it is a DIFFERENT mechanism from rich_markup_mode — setting one does nothing about the other. It surfaced when a NameError produced a full-width box-drawn traceback. That is the same log pollution rich_markup_mode=None was set to prevent (T-1259), arriving through another door, and landing in the worst possible place: a hook log at the moment something has already gone wrong. pretty_exceptions_enable=False now set on both the root app and every domain built by cli.domain().
|
||||
|
||||
THE EXISTING TEST MOVED WITH THE CODE. tooling/test_canvas_version_check.py loaded the extensionless script through a SourceFileLoader and reached canvas_sources via sys.path.insert — both only because tooling/ was not an importable package. It now does `from tooling.domains.check import service as check` and `from tooling.canvas_sources import relative_paths`. Second instance of the E402/sys.path debt evaporating on contact (T-1274). The test passes unchanged in what it ASSERTS, which is the point: diff_has_version_bump was kept pure in the port precisely so those six properties still hold without constructing git history.
|
||||
|
||||
A BUG CAUGHT BY RUNNING IT: the check router had dropped its `import typer` in T-1267 when it moved to cli.domain(), and canvas-version needs typer.Option for --base/--head. Restored.
|
||||
|
||||
STILL OPEN on this ticket:
|
||||
- No parity tests for canvas-version, dataflow-graph or systems-db-stamp. All three were verified side by side on the live tree, which proves the happy path only. canvas-version''s failure path is the valuable one and needs a git fixture — a temp repo with a commit touching a registry file and no version bump.
|
||||
- Retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.
|
||||
|
||||
PROGRESS 2026-08-31, fourth increment — canvas-version parity, five cases, including its failure path.
|
||||
|
||||
A FIXTURE REPO DOES NOT WORK FOR THIS GATE, and finding out why exposed a real inconsistency in the port. canvas_sources.relative_paths() globs from a __file__-derived REPO_ROOT, so under SR_REPO_ROOT the new service would DIFF the fixture while GLOBBING the real tree. The service resolves git through config.repo_root() but the registry does not — the two disagree about which repo is being checked. Harmless in production (they are the same repo) but it makes fixture-based parity impossible, and it is the same asymmetry the domain map already noted about old scripts having no root override, showing up inside the new code.
|
||||
NOT FIXED HERE: making it dynamic means restructuring six module-level path constants in a module the old script also imports, which is more churn than this ticket should carry. Filed as a follow-up consideration for when canvas_sources moves into the domain per the map.
|
||||
|
||||
SO PARITY USES REAL HISTORY INSTEAD, which is better anyway: both implementations see byte-identical input, nothing is mutated, and no fixture can drift from the thing it models. Five cases:
|
||||
- canvas-needs-bump: 4e503c356 (T-1237 one-course-per-river) — changed canvas generation, did NOT bump. Both exit 1.
|
||||
- canvas-needs-bump-2: 566b56651 (T-1194 relief/vegetation texture) — same. Both exit 1.
|
||||
- canvas-bumped: 9b146f9e1 — changed canvas AND bumped. Both exit 0.
|
||||
- canvas-clean: 6b31111cd — governance only. Both exit 0.
|
||||
- canvas-no-base: a nonexistent base ref — a SKIP, not a failure. Both exit 0.
|
||||
The two failure cases are GENUINE historical instances of the regression this gate exists to catch, both bumped after the fact. That is the history that produced the check, used as its own test.
|
||||
|
||||
THE TEST ALSO ASSERTS NO CHANGED FILE IS DROPPED from the failure message. The list of touched files is the actionable half — a failure saying "something changed" without saying what leaves the reader to re-derive the intersection by hand.
|
||||
|
||||
PROVEN TO FAIL: truncating the touched-file set made canvas-needs-bump report both the exit-code divergence AND all three omitted filenames by name.
|
||||
|
||||
SHAs are hardcoded because history is immutable, and each case asserts the OLD script still behaves as the case claims — so if a rebase ever invalidated a range, the test says so by name rather than silently checking nothing.
|
||||
|
||||
STILL OPEN on this ticket: parity tests for dataflow-graph and systems-db-stamp (happy path verified side by side only), and retirement of all five old scripts, their make targets, and the pre-commit hook entry for fact-ids.', 'in_progress', 'high', NULL, NULL, 'D-263', '2026-08-31 15:30:35.338', '2026-08-31 18:07:14.123', NULL, 'b6d293132ca2394ec8e11441243d174f', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
||||
|
||||
@@ -195,6 +195,81 @@ def check_fact_ids(failures: list[str]) -> None:
|
||||
)
|
||||
|
||||
|
||||
# Ranges from the repo's own history, chosen so both implementations see
|
||||
# identical input with nothing mutated. A fixture repo would not work here: the
|
||||
# canvas registry globs from a __file__-derived root, so under SR_REPO_ROOT the
|
||||
# new service would diff the fixture while globbing the real tree. Real history
|
||||
# sidesteps that asymmetry entirely — and the failure cases below are genuine
|
||||
# instances of the regression this gate exists to catch, not constructed ones.
|
||||
#
|
||||
# SHAs are hardcoded because history is immutable. If a rebase ever invalidates
|
||||
# one, the test says so by name rather than silently checking nothing.
|
||||
CANVAS_CASES = [
|
||||
# (label, base, head, expect non-zero)
|
||||
("canvas-needs-bump", "4e503c356~1", "4e503c356", True), # T-1237: bumped later
|
||||
("canvas-needs-bump-2", "566b56651~1", "566b56651", True), # T-1194: same
|
||||
("canvas-bumped", "9b146f9e1~1", "9b146f9e1", False), # changed AND bumped
|
||||
("canvas-clean", "6b31111cd~1", "6b31111cd", False), # governance only
|
||||
("canvas-no-base", "no-such-ref-anywhere", "HEAD", False), # skip, not fail
|
||||
]
|
||||
|
||||
|
||||
def check_canvas_version(failures: list[str]) -> None:
|
||||
"""Parity for the gate with the worst track record — five shipped regressions.
|
||||
|
||||
Its failure path is the one that matters, so two of the five cases are real
|
||||
commits that changed canvas generation without bumping the version. Both
|
||||
were bumped after the fact, which is precisely the history that produced
|
||||
this check.
|
||||
"""
|
||||
for label, base, head, expect_failure in CANVAS_CASES:
|
||||
old = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
str(REPO_ROOT / "tooling" / "check-canvas-version"),
|
||||
"--base",
|
||||
base,
|
||||
"--head",
|
||||
head,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd=REPO_ROOT,
|
||||
)
|
||||
new = subprocess.run(
|
||||
["reach", "check", "canvas-version", "--base", base, "--head", head],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd=REPO_ROOT,
|
||||
env={**os.environ, "SR_OUTPUT_FORMAT": "text"},
|
||||
)
|
||||
|
||||
if (old.returncode != 0) != expect_failure:
|
||||
failures.append(
|
||||
f"[{label}] the OLD script exited {old.returncode}; this range no "
|
||||
"longer sets up the case it claims to — history may have been rewritten"
|
||||
)
|
||||
if old.returncode != new.returncode:
|
||||
failures.append(
|
||||
f"[{label}] exit differs: old={old.returncode} new={new.returncode}"
|
||||
)
|
||||
|
||||
if expect_failure:
|
||||
# The changed files are the actionable half of this failure; a
|
||||
# message that says "something changed" without saying what leaves
|
||||
# the reader to re-derive the intersection by hand.
|
||||
for stream in (old.stdout + old.stderr,):
|
||||
for line in stream.splitlines():
|
||||
token = line.strip()
|
||||
if token.endswith((".rs", ".gd")) and token not in (
|
||||
new.stdout + new.stderr
|
||||
):
|
||||
failures.append(
|
||||
f"[{label}] new output omits a changed file the old one "
|
||||
f"named: {token}"
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if shutil.which("reach") is None:
|
||||
print(
|
||||
@@ -269,6 +344,7 @@ def main() -> int:
|
||||
)
|
||||
|
||||
check_fact_ids(failures)
|
||||
check_canvas_version(failures)
|
||||
|
||||
if failures:
|
||||
print("test_check_parity: FAIL", file=sys.stderr)
|
||||
@@ -278,7 +354,8 @@ def main() -> int:
|
||||
|
||||
print(
|
||||
f"test_check_parity: OK — client-version {len(CASES)} paths, "
|
||||
f"fact-ids {len(FACT_CASES)} paths, all agreeing on exit code and content"
|
||||
f"fact-ids {len(FACT_CASES)} paths, canvas-version {len(CANVAS_CASES)} paths, "
|
||||
"all agreeing on exit code and content"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user