The suite is 12.1k tests in a single CI job — nearly six minutes of pytest
that every push and every PR waits on in one block, on top of a setup step
that already installs npm, Playwright, FFmpeg and bubblewrap. Split it into
four sections the matrix runs in parallel: 352s becomes a 98s longest pole
locally.
Shards partition by test *file*, not by the area_* taxonomy markers. Those
markers do not partition the suite - a file can carry a hand-applied area_*
mark on top of the one conftest derives from its filename, so a marker-based
split would run those tests in more than one section. Assignment is a total
function of the file path instead, so every file lands in exactly one shard
and the four together run every test exactly once.
Sharding deselects rather than narrowing collection, so every test module is
still imported, in the same order, in every shard. The import-time stubbing
in conftest and the session-scoped static server behave identically whether
the suite runs whole or in sections - this suite has known collection-order
coupling and splitting by path would have walked into it.
Balance uses the existing `slow` marker as its weight signal rather than a
committed duration table that would go stale unnoticed. Files pack
heaviest-first into the lightest shard, which is deterministic for a given
file set, so every parallel job computes the same plan from the same commit.
Verified: the four shards together reproduce the full run exactly - 12141
tests selected across the four, and the same 59 failures, 78 skips and 2
xfails, by node ID and not merely by count.
