docs(governance): D-263 — output parity over timing, and commands that stream

Three amendments, all from pressure-testing the record against how the CLI
will actually be used.

The ~104 ms push-gate ceiling is withdrawn. It was the summed cost of three
single-sample timings, imported as a requirement without asking who pays —
and who pays is the pre-push hook, which already runs cargo test or the
gdUnit4 suite on any code push. A few hundred milliseconds is invisible
there, and on a governance-only push the whole hook is about a second. The
criterion is OUTPUT parity: a ported check must produce the same output and
the same exit code as the script it replaces, and is not required to be as
fast. What replaces the ratchet is a ceiling with headroom — under ~250 ms to
feel instant. Lazy registration stays mandatory, justified by the real
threat rather than by parity: scipy.ndimage alone is 275 ms, and an eager
entrypoint would pay ~460 ms before executing a line of its own.

That budget change removed the only argument for keeping pydantic out of the
gate domain, so the carve-out goes with it. One fewer exception, and the
reference implementation is now the normal pattern rather than a footnote.

Commands also stream. The gates are milliseconds but the generators are
minutes, and an agent Bash call gives up at two and sends nothing. Detaching
alone would fix the timeout and keep the silence; streaming fixes the part
that costs real time — you learn a generator is wedged at minute one instead
of minute nine. JSONL events on stderr, stdout reserved for actual output,
rendering at the sink so a job log and a live terminal are one artefact in
two presentations. Reattach is a byte offset into an append-only file, which
is why there is deliberately no daemon.

The trap, recorded because it would quietly undo the thing this record cares
most about: streaming is ADDITIVE to the failure contract. A remedy emitted
at line 400 of 900 is printed and invisible, so the verdict still prints
once, last.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

governance/decisions/architecture.md

@@ -2546,7 +2546,7 @@ Technical foundation decisions that constrain implementation: engine, client-ser
 | language | **Python.** The friction is packaging, not language — see below. |
 | command name | **`reach`.** Free on PATH; `Bash(reach *)` becomes the single allowlist entry. |
 | package home | **`tooling/` itself is the package** (`tooling/__init__.py`, imported as `tooling.*`), with every domain under `tooling/domains/`. Fewest path rewrites across the 160 markdown files that name `tooling/…`. |
-| dependencies added | **`typer`** (transport) and **`pydantic`** (data shapes) join `pyproject.toml`. Both are placement-constrained by the startup budget below — pydantic in particular. |
+| dependencies added | **`typer`** (transport) and **`pydantic`** (data shapes) join `pyproject.toml`. Neither may be imported at module level in `main.py`. |
 | numerics | **They move too, and stay Python.** `planet-gen`, `garment-fit`, `economy-db` and `blender` all come inside. No numerical-equivalence problem is created because no numerical code is rewritten. |
 
 **Why not Rust — the friction Q-124 names is packaging, and Rust is not the cheapest fix for any of it.**
@@ -2574,12 +2574,13 @@ tooling/
   core/                     # shared substrate — only what has no domain
     logging.py              #   the shared logger + the @logged decorator
     errors.py               #   ReachError + the @handle_errors decorator
-    console.py              #   all output; nothing else prints
+    console.py              #   the event emitter — all output; nothing else
+                            #     prints. JSONL to stderr, rendered at the sink.
     config.py               #   repo paths, project.yaml, endpoint config
     process.py              #   subprocess + the Blender launcher
   domains/                  # every domain lives here, one directory each
     atlas/  check/  generate/  validate/  planet/  db/  wiki/  visual/
-    garment/  godot/  blender/  dev/
+    garment/  godot/  blender/  jobs/  dev/
       router.py             #   CONTROLLER — args in, delegate, format out. No logic.
       service.py            #   LOGIC — transport-agnostic, importable by anything.
       schemas.py            #   pydantic models for this domain's data
@@ -2594,8 +2595,9 @@ tooling/
   - `@handle_errors` (`core/errors.py`) catches `ReachError(message, fix="…")` and renders it as the instructional failure below — message and remedy to stderr, **non-zero exit preserved**. An uncaught exception it does not recognise still exits non-zero, with the traceback behind `--verbose`.
   - `@logged` (`core/logging.py`) emits one structured line per invocation — command, arguments, duration, outcome — through the shared logger. **To stderr, never stdout**, so machine-readable output stays parseable, and quiet by default so hooks are not spammed.
   - They compose into a single `@command` decorator so no command can carry one without the other, and **the conformance test asserts every registered command carries it.** A cross-cutting concern applied by hand is a cross-cutting concern applied to 90% of cases.
-- **Pydantic is the data-shape vocabulary, and it is confined to `domains/*/schemas.py`.** Measured 2026-08-20: `import pydantic` costs **87 ms**, against a *whole current gate check* of 20–46 ms (`check-client-version` 20 ms, `check-dataflow-graph` 38 ms, `check-canvas-version` 46 ms — ~104 ms for the three unconditional ones). Put pydantic on the import path of `main.py` or `core/` and the push gate goes to ~365 ms, a 3.5× regression bought for nothing, four times per push. **So: `main.py` and `core/` are pydantic-free; a domain's `schemas.py` is imported by that domain's service, never by its router; and `domains/check/` — the push-gate domain — carries no pydantic at all.** This is the concrete reason lazy registration is load-bearing rather than tidy.
-- **Enforced, not asked for:** no `typer`/`click` import outside `main.py` and `router.py`; no bare `print` outside `core/console.py`; no `pydantic` import reachable from `main.py`; every command carries `@command`. All four are grep-shaped or import-graph-shaped, and belong in the conformance test alongside the help/failure checks — plus a wall-clock assertion on `reach check …` so the budget is a test, not an intention.
+- **Pydantic is the data-shape vocabulary, and it lives in `domains/*/schemas.py`.** Available to every domain, including the gate domain. `main.py` and `core/` stay pydantic-free — not for the milliseconds, but because `main.py` is a router and `core/` is a substrate, and neither has data shapes of its own.
+- **The import rule that actually matters: nothing heavy at module level in `main.py` or any `router.py`.** Measured 2026-08-20 in the repo venv: `scipy.ndimage` **275 ms**, `pydantic` 87 ms, `numpy` 72 ms, `PIL.Image` 29 ms. An entrypoint that eagerly imported the tree would pay **~460 ms before executing a line of its own** — and *that*, not pydantic, is what lazy registration exists to prevent. Heavy imports belong inside a service, or inside the function that needs them.
+- **Enforced, not asked for:** no `typer`/`click` import outside `main.py` and `router.py`; no bare `print` outside `core/console.py`; no heavy import (numpy, scipy, PIL, pydantic) reachable from `main.py`; every command carries `@command`. All four are grep-shaped or import-graph-shaped, and belong in the conformance test alongside the help/failure checks. The **import-graph** assertion is the one worth writing carefully — a wall-clock assertion is flaky on a loaded machine and tells you *that* something got slow rather than *what*, whereas asserting `sys.modules` after `reach --help` names the offender directly.
 - **Not every domain needs every file.** `schemas.py`/`dependencies.py` appear when a domain has data shapes or collaborators worth naming; `check/` may be a router and a service and nothing else. The layering is a vocabulary, not a quota — a folder of five empty modules is worse than a folder of two full ones.
 - **`core/` is bounded on day one, because its failure mode is gradual and invisible.** It holds **only what has no domain**: config, paths, errors, console output, process launching. **The moment something in `core/` grows a service — its own logic, its own data store, its own verbs — it is a domain and it moves out.** A `core/` that accumulates services becomes a package every other package imports and nobody can change, which is the worst possible shape for the one directory that is supposed to be stable. There is no gate that catches this; it is a review question, asked every time a file is added to `core/`.
 - **The Blender scripts are a physically-enforced exception.** `tooling/blender` is a bash wrapper resolving flatpak/native/brew installs, and the 35 `blender_*.py` / `blender_author_*.py` files run **under Blender's own bundled interpreter** via `--background --python`, which cannot import this package. They stay standalone payload files. **`reach` fronts them; it does not absorb them** — `reach blender process-bodies` builds and executes the Blender command line. Any claim that "everything is one package" must carry this exception or it is false.
@@ -2608,16 +2610,29 @@ tooling/
 - **Never literally interactive by default.** Any prompt is TTY-gated and suppressible with `--no-input`, which hooks pass unconditionally. `tea`'s interactive prompts *"crash in Claude Code (no TTY)"*; a helpful prompt that hangs a hook is worse than a terse exit code.
 - **This is enforced by a conformance test, not by discipline** — every registered command must have help at its own level, and every declared failure path must name a next command. A contract nothing checks is a style guide.
 
+**Commands stream, they do not go quiet and return a verdict** *(added 2026-08-20)*. The gates are milliseconds; the generators are not. `make regen-db`, the planet-gen pipeline, the Blender batches and the Trellis/audio connectors run for minutes, and the callers that matter have ceilings — an agent `Bash` call gives up at two minutes and **sends nothing**, already a recorded scar here for `git push` under the full-`cargo test` hook. Detaching alone would fix the timeout and keep the silence. Streaming fixes the thing that actually costs time: **you learn a generator is wedged at minute one instead of minute nine.**
+
+- **Every command emits a stream of structured progress events as it runs.** Not a requirement to instrument everything — a command that emits nothing still works, and the gates should emit nothing. It is a requirement that the *channel exists* and is the same channel everywhere, so a long command has somewhere to say what it is doing.
+- **The stream is JSONL — one object per line** (`ts`, `level`, `phase`, `message`, optional `progress`), rendered human-readably at the sink. Machine-parseable and human-readable are then the **same artefact** rather than two that drift: `reach jobs log` renders it for a person, and the conformance suite asserts against it directly.
+- **The stream goes to stderr; stdout carries the command's actual output.** Same rule already stated for `@logged`, for the same reason — `reach … | jq` must keep working.
+- **`core/console.py` is therefore an event emitter, not a print wrapper.** It remains the single output path and stays stdlib-only (`json` is stdlib). This is the module every command depends on, so its shape is fixed here rather than discovered per-domain.
+- **Detached runs append to a per-job log; tailing it is how you attach.** `.cache/reach/jobs/<id>.jsonl` (gitignored), `reach jobs log --follow` to tail, and reattach is **reading from a byte offset** — a caller can attach, drop off, and come back without losing anything. **Deliberately no daemon:** a session process that survives between calls is state that can be stale, orphaned, or wrong, and an append-only file buys ~90% of the value with no lifecycle to get wrong. If that proves insufficient, a session layer is a follow-on — to be *found* necessary, not assumed.
+- **Streaming is additive to the failure contract, never a replacement for it.** A stream has no single moment of truth: a remedy emitted at line 400 of 900 is technically printed and practically invisible. **The verdict — outcome, exit code, and the command that fixes it — is still printed once, last, where it cannot be missed.** A stream that dissolved the summary would quietly undo the requirement this record cares most about.
+- **The split follows the `core/` bound, and this is its first real test.** The primitives — emit, spawn, detach, redirect, record — are substrate and live in `core/`. The verbs `list`, `status`, `log`, `wait` have logic and state of their own, so they are a **domain**: `reach jobs …`. A job store in `core/` would be exactly the drift this record warns about.
+- **Non-negotiable: a detached job's exit code must survive.** A runner that reports "started" and loses the failure is the exit-0 trap from the top of this record, relocated somewhere nothing is watching — which is worse. `reach jobs wait` exits with the job's code, and an unwaited failed job is visible in `reach jobs list`.
+- **Where this overlaps the harness, prefer the harness.** Claude Code's `Bash` tool already has a background mode that solves the timeout *for agents*. What `reach` adds is for the callers with no such escape — a human terminal, a Makefile, a git hook — plus durable logs and job history. Scope it there rather than rebuilding what one caller already provides.
+
 **Constraints on execution (these are why the work is sequenced, not why it is hard):**
 
-- **The push-gate total may not regress, and the baseline is already measured.** 2026-08-20, this machine: `check-client-version` **20 ms**, `check-dataflow-graph` **38 ms**, `check-canvas-version` **46 ms** — **~104 ms** for the three unconditional checks (`check-systems-db-stamp` runs only when `systems.db` is in the push). That is the number the ported gates must not exceed. Lazy registration is mandatory, not an optimisation: a single entrypoint that eagerly imported 123 modules — or merely imported pydantic — would multiply this several-fold, four times per push, forever.
+- **The acceptance criterion is output parity, not timing parity** *(amended 2026-08-20, same day: the original text set the ported gates a hard ceiling of ~104 ms — the summed single-sample cost of the three unconditional checks — and that was wrong in kind. It imported "do not regress" as a requirement without asking who pays.)* **Who pays is the pre-push hook, and almost nobody else.** On a push touching `server/` or `client/` the hook runs `cargo test` or the gdUnit4 suite — minutes — so a few hundred milliseconds is invisible. On a governance-only push the whole hook is about a second. No human and no loop consumes these often enough for 100 ms versus 400 ms to register. **So a ported check must produce the same output and the same exit code as the script it replaces; it is not required to be as fast.**
+  **The budget that replaces it is a ceiling with headroom, not a ratchet:** a `reach` invocation should feel instant to a human — **under ~250 ms** — and the unconditional gate set should stay **comfortably under a second**. That is loose enough that pydantic, a subprocess, or a DB open are all affordable, and tight enough that nobody imports scipy at module level. **Lazy registration stays mandatory**, justified by the real threat rather than by parity: an eager entrypoint would pay ~460 ms of numpy + scipy + PIL + pydantic before executing a line of its own, and would grow every time a domain was added.
 - **The `systems.db` stamp survives the move or the move does not land.** `tooling/generator_sources.py` SHAs the concatenated bytes of the generator's sources **sorted by path**, so renaming a file changes the stamp even when its content is byte-identical. The generator relocation must therefore land as **one commit** — registry paths updated, `make regen-db` run, stamp verified — never split across pushes, or the pre-push gate rejects an intermediate state that is in fact correct.
 - **Old paths are retired through a deprecation window, not deleted under the callers.** 160 markdown files under `.claude/` and `docs/`, 84 make targets, the pre-push hook and the skills all name `tooling/…` paths. Each retired path leaves a shim that prints the new command and exits non-zero — the failure contract applied to the migration itself — before the shims are removed.
 - **What this decision does not claim.** It does not make make-target invocation cheaper (already free), and it does not make the tooling faster to *run* — only to start, find, and be allowed to call. The wins are: ad-hoc and direct invocation stop prompting, `--help` answers "what tooling exists" without an `ls`, arguments become expressible where make could not express them, the venv split disappears, and failures carry their own remedy.
 - **Raised by:** Jeroen, 2026-08-20 — *"those python files are a pain… maybe make it into an actual cli of the quality level of pql"*, then the shape: *"moving all python into a separate dir with proper domain split so one door answers all options we have with help and instructions/help when there is an error: a new prompt not an error code"*, and the principle behind it: *"I have this in pql as well: errors become instructions."*
-- **Cross-reference:** [Q-124](../questions/architecture.md#q-124-should-the-python-tooling-be-retooled-into-a-single-rust-cli) (the question, and the costing that got here), [R-014](../rejected/architecture.md#r-014-rust-rewrite-of-the-python-tooling) (the Rust option), [D-223](#d-223) + `.claude/rules/asset-pipeline.md` (the stamp contract the move must preserve), [D-262](#d-262) (`check-dataflow-graph`, the newest member of the per-push Python set), `.claude/rules/ticket-cli.md` (`pql` as the quality bar), T-1247 (the initiative implementing this, eight epics).
+- **Cross-reference:** [Q-124](../questions/architecture.md#q-124-should-the-python-tooling-be-retooled-into-a-single-rust-cli) (the question, and the costing that got here), [R-014](../rejected/architecture.md#r-014-rust-rewrite-of-the-python-tooling) (the Rust option), [D-223](#d-223) + `.claude/rules/asset-pipeline.md` (the stamp contract the move must preserve), [D-262](#d-262) (`check-dataflow-graph`, the newest member of the per-push Python set), `.claude/rules/ticket-cli.md` (`pql` as the quality bar), T-1247 (the initiative implementing this) — epics T-1248 the door, T-1249 decorators, T-1250 the domain move, T-1251 the gate family, T-1252 generators and numerics, T-1253 retiring old paths, T-1256 adoption, T-1257 the test surface, T-1264 streaming and jobs.
 - **Dissent:** None recorded. The Rust option was preferred by the raiser at filing time and was costed down rather than argued down — see [R-014](../rejected/architecture.md#r-014-rust-rewrite-of-the-python-tooling).
 
 ---
 
-*117 decisions (D-001 through D-263, excluding gaps). Last updated: 2026-08-20 (D-263 — `tooling/` becomes one installable Python package behind the `reach` command: a routing-only `main.py`, `domains/<name>/{router,service,schemas,helpers}.py`, a bounded `core/`, logging + error handling as decorators, pydantic kept off the gate path; Rust rejected as R-014 because the friction is packaging, not language).*
+*117 decisions (D-001 through D-263, excluding gaps). Last updated: 2026-08-20 (D-263 — `tooling/` becomes one installable Python package behind the `reach` command: a routing-only `main.py`, `domains/<name>/{router,service,schemas,helpers}.py`, a bounded `core/`, logging + error handling as decorators; amended same day twice — the timing-parity budget dropped for output parity plus a ~250 ms feels-instant ceiling, and a streaming execution model added: JSONL progress events to stderr, per-job logs tailed for reattach, no daemon; Rust rejected as R-014 because the friction is packaging, not language).*