jpmschweitzer/clide
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add a serial test lane for concurrency-vulnerable tests

Browse Source 
Generalizes the pty split into an explicit "parallel=false" opt-out: a
`serial` tag (declared in dart_test.yaml). The parallel flutter run now
excludes `pty || serial`; a separate `flutter test --tags serial
--concurrency=1` pass runs the vulnerable ones. For the coverage gate the
two passes are real-merged by ci/merge_lcov.py (union DA, max hits, recompute
LF/LH) — a plain concat would double-count and corrupt the total.

Tag transcript_publisher's bus-republish test serial (it flaked in the
parallel pool). Gate verified green end-to-end at 95.08%.

T-193.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2026-05-31 19:26:17 +02:00
co-authored by Claude
parent 5b739ef6a7
commit b6eef3fc2b
6 changed files with 81 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.pql/changelog/ticket_history/2026-05.sql
+3
View File
@@ -2322,3 +2322,6 @@ INSERT INTO ticket_history (ticket_id, field, old_value, new_value, changed_by,
INSERT INTO ticket_history (ticket_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-191', 'status', 'backlog', 'in_progress', NULL, '2026-05-31 16:13:39', '2026-05-31 16:13:39', '2026-05-31 16:13:39', NULL, '76e72b3d04c0456e8a39d8ca7dba7644', 1) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-190', 'status', 'backlog', 'in_progress', NULL, '2026-05-31 16:13:39', '2026-05-31 16:13:39', '2026-05-31 16:13:39', NULL, 'cc7072933e46f402bf65c02595843185', 1) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-186', 'status', 'backlog', 'done', NULL, '2026-05-31 16:40:19', '2026-05-31 16:40:19', '2026-05-31 16:40:19', NULL, '8f5241e0d76f6cef282f5a04d274345a', 1) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-193', 'description', 'test/builtin/claude/transcript_publisher_test.dart ''TranscriptPublisher republishes reader items onto the bus (lead channel + item key)'' fails intermittently during ''make test-coverage'' / push-check (the full parallel flutter test run) but PASSES reliably in isolation (flutter test <file> = 3/3 green). Observed 2026-05-31 — one run failed it, the immediately-preceding agent run passed all 2221. Not caused by T-192 (the coverage path uses default concurrency, unchanged) — it''s a pre-existing timing/ordering flake, likely a MessageBus delivery race: the test asserts a republished item arrived on the lead channel, and under concurrent load the broadcast-stream delivery may not have been awaited deterministically. Fix: make the test await delivery deterministically (pump/await the bus microtask, or expectLater on the stream) rather than relying on timing; check TranscriptPublisher for any real ordering bug too. Impact: intermittently red push-check → tempts --no-verify (see T-192). Acceptance: the test passes deterministically across repeated full-suite runs; root cause (test-only vs a real publisher race) identified.', 'test/builtin/claude/transcript_publisher_test.dart ''TranscriptPublisher republishes reader items onto the bus (lead channel + item key)'' fails intermittently during ''make test-coverage'' / push-check (the full parallel flutter test run) but PASSES reliably in isolation (flutter test <file> = 3/3 green). Observed 2026-05-31 — one run failed it, the immediately-preceding agent run passed all 2221. Not caused by T-192 (the coverage path uses default concurrency, unchanged) — it''s a pre-existing timing/ordering flake, likely a MessageBus delivery race: the test asserts a republished item arrived on the lead channel, and under concurrent load the broadcast-stream delivery may not have been awaited deterministically. Fix: make the test await delivery deterministically (pump/await the bus microtask, or expectLater on the stream) rather than relying on timing; check TranscriptPublisher for any real ordering bug too. Impact: intermittently red push-check → tempts --no-verify (see T-192). Acceptance: the test passes deterministically across repeated full-suite runs; root cause (test-only vs a real publisher race) identified.
MECHANISM (the systemic answer, per discussion): tests are parallel by default (--concurrency global = invisible parallel=true); express ''parallel=false'' for a vulnerable test via a TAG + a serial pass. The ''pty'' tag is the half-built precedent: it splits PTY tests into a separate ''dart test'' pass but did NOT serialize it (and leaned on retry:2). DONE 2026-05-31: that pty pass now runs --concurrency=1 and the retry:2 band-aids are removed — PTY/registry flake verified stable 4/4 runs. REMAINING: generalize to a ''serial'' tag for FLUTTER-side vulnerable tests (transcript_publisher''s ''republishes reader items onto the bus''): declare ''serial'' in dart_test.yaml; the parallel flutter run adds --exclude-tags serial; add a serial flutter pass (flutter test --tags serial --concurrency=1). WRINKLE for the gate: two flutter --coverage passes can''t just concatenate lcov (duplicate SF records double-count) — must real-merge (package:coverage format_coverage / lcov -a), OR run the serial pass without coverage if those tests'' source lines are covered elsewhere (check transcript_publisher.dart coverage from other tests first). ALTERNATIVE for the single transcript_publisher case: fix its await to be deterministic (await the bus delivery / expectLater on the stream) rather than quarantine — cheaper if it''s a test-timing bug vs a real bus race. Acceptance: no flaky test runs in the parallel pool; vulnerable tests are tagged + run serially; gate is deterministic across repeated full runs.', NULL, '2026-05-31 17:01:58', '2026-05-31 17:01:58', '2026-05-31 17:01:58', NULL, 'efff36d6e6969000e1085c073788425c', 1) ON CONFLICT(hash) DO NOTHING;
.pql/changelog/tickets/2026-05.sql
+3
View File
@@ -2960,3 +2960,6 @@ INSERT INTO tickets (id, type, parent_id, title, description, status, priority,
INSERT INTO tickets (id, type, parent_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-189', 'story', 'T-186', 'Back/forward navigation on the markdown & decision readers', 'Add Back/Forward buttons to both sidebar readers so you can return to what you were reading. Neither reader keeps history today: _MarkdownViewerState (markdown_viewer.dart) holds only _path/_content; _DecisionDetailViewState (decision_detail_view.dart) holds only _decision. Add a history stack + index to each, pushing on each new open; Back/Forward walk the stack and load the entry WITHOUT re-publishing a ''selection'' (in-widget navigation). For the decision reader, back/forward must not trigger the extension tab re-contribution — coordinate with the decision-open bug fix (T-188), which should stop re-contributing per selection. Custom chrome only (no Material). Acceptance: Back/Forward controls on both readers walk reading history and disable at the ends; widget tests. Sequence after the reader-open bug fixes (T-187/T-188) since it shares those widgets.', 'in_progress', 'medium', NULL, NULL, NULL, '2026-05-31 11:38:36', '2026-05-31 16:13:39', NULL, '8e8e53cd7d4bb35535281c433e8d122d', 1) ON CONFLICT(id) DO UPDATE SET type=excluded.type, parent_id=excluded.parent_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at > tickets.updated_at OR (excluded.updated_at = tickets.updated_at AND excluded.hash > tickets.hash);
INSERT INTO tickets (id, type, parent_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-190', 'story', 'T-186', 'Pin a doc in the markdown & decision readers (single-slot quick-return)', 'Add a one-click PIN to both sidebar readers. Semantics per the reporter: a SINGLE pin slot per reader — clicking pin remembers the CURRENT doc; setting a new pin REPLACES the previous one (no list to manage); the pinned doc is always one click away to return to, so it is easy to use inline as you work. It is a quick-return bookmark, NOT a freeze-incoming toggle. Add a pinned-doc ref to _MarkdownViewerState and _DecisionDetailViewState; a pin button (set/replace) plus a jump-to-pin affordance in the reader chrome; persist for at least the session. Custom chrome only (no Material). Acceptance: pinning a doc lets you one-click return to it later; pinning another replaces it; works on both readers; widget tests. Sequence after the reader-open bug fixes (T-187/T-188); pairs naturally with back/forward (T-189).', 'in_progress', 'medium', NULL, NULL, NULL, '2026-05-31 11:38:42', '2026-05-31 16:13:39', NULL, 'd0ec78c056f0d7f8e35e521f13d229ed', 1) ON CONFLICT(id) DO UPDATE SET type=excluded.type, parent_id=excluded.parent_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at > tickets.updated_at OR (excluded.updated_at = tickets.updated_at AND excluded.hash > tickets.hash);
INSERT INTO tickets (id, type, parent_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-186', 'epic', NULL, 'Sidebar readers: fix message-bus open + add back/forward + pin', 'The right-side sidebar READERS — the markdown reader (builtin.markdown, lib/builtin/markdown/src/markdown_viewer.dart) and the decision reader (builtin.decisions, lib/builtin/decisions/src/decision_detail_view.dart) — are separate from the editor and open via the kernel MessageBus, the same pattern as the still-working ticket panel (lib/builtin/tickets): a click publishes (publisher, channel ''selection''); an extension activates the context-panel tab; the reader widget subscribes and renders. Two open paths are currently broken and we also want reading-navigation (back/forward) plus a single-slot pin on BOTH readers. Grouped as one set because all four tickets touch the same two reader widgets. Root-cause investigation done 2026-05-31 (see child tickets).', 'done', 'high', NULL, NULL, NULL, '2026-05-31 11:37:34', '2026-05-31 16:40:19', NULL, 'f32bbd391b6fd1d8448af994fae28a13', 1) ON CONFLICT(id) DO UPDATE SET type=excluded.type, parent_id=excluded.parent_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at > tickets.updated_at OR (excluded.updated_at = tickets.updated_at AND excluded.hash > tickets.hash);
INSERT INTO tickets (id, type, parent_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('T-193', 'bug', NULL, 'Flaky: transcript_publisher_test ''republishes reader items onto the bus'' fails under the full parallel run', 'test/builtin/claude/transcript_publisher_test.dart ''TranscriptPublisher republishes reader items onto the bus (lead channel + item key)'' fails intermittently during ''make test-coverage'' / push-check (the full parallel flutter test run) but PASSES reliably in isolation (flutter test <file> = 3/3 green). Observed 2026-05-31 — one run failed it, the immediately-preceding agent run passed all 2221. Not caused by T-192 (the coverage path uses default concurrency, unchanged) — it''s a pre-existing timing/ordering flake, likely a MessageBus delivery race: the test asserts a republished item arrived on the lead channel, and under concurrent load the broadcast-stream delivery may not have been awaited deterministically. Fix: make the test await delivery deterministically (pump/await the bus microtask, or expectLater on the stream) rather than relying on timing; check TranscriptPublisher for any real ordering bug too. Impact: intermittently red push-check → tempts --no-verify (see T-192). Acceptance: the test passes deterministically across repeated full-suite runs; root cause (test-only vs a real publisher race) identified.
MECHANISM (the systemic answer, per discussion): tests are parallel by default (--concurrency global = invisible parallel=true); express ''parallel=false'' for a vulnerable test via a TAG + a serial pass. The ''pty'' tag is the half-built precedent: it splits PTY tests into a separate ''dart test'' pass but did NOT serialize it (and leaned on retry:2). DONE 2026-05-31: that pty pass now runs --concurrency=1 and the retry:2 band-aids are removed — PTY/registry flake verified stable 4/4 runs. REMAINING: generalize to a ''serial'' tag for FLUTTER-side vulnerable tests (transcript_publisher''s ''republishes reader items onto the bus''): declare ''serial'' in dart_test.yaml; the parallel flutter run adds --exclude-tags serial; add a serial flutter pass (flutter test --tags serial --concurrency=1). WRINKLE for the gate: two flutter --coverage passes can''t just concatenate lcov (duplicate SF records double-count) — must real-merge (package:coverage format_coverage / lcov -a), OR run the serial pass without coverage if those tests'' source lines are covered elsewhere (check transcript_publisher.dart coverage from other tests first). ALTERNATIVE for the single transcript_publisher case: fix its await to be deterministic (await the bus delivery / expectLater on the stream) rather than quarantine — cheaper if it''s a test-timing bug vs a real bus race. Acceptance: no flaky test runs in the parallel pool; vulnerable tests are tagged + run serially; gate is deterministic across repeated full runs.', 'backlog', 'medium', NULL, NULL, NULL, '2026-05-31 16:12:42', '2026-05-31 17:01:58', NULL, 'a53bd052ae3024be526c4a6d54da23ef', 1) ON CONFLICT(id) DO UPDATE SET type=excluded.type, parent_id=excluded.parent_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at > tickets.updated_at OR (excluded.updated_at = tickets.updated_at AND excluded.hash > tickets.hash);