feat(config): T-1280 — job log retention, and the corpse that would never die

Pruning happens at spawn time rather than on a schedule: a retention pass that
depends on someone remembering to run it is one that silently never happens.
reach jobs prune is the explicit escape hatch for reclaiming space now.

The cap was measured rather than guessed, which is why this ticket ran last. A
chatty short job writes ~1.8 KB across its three files, so 100 jobs is
single-digit megabytes even if a generator emits per-body progress — inside
.cache/, where being wrong costs disk and never data. SR_JOB_KEEP overrides it.

The interesting part is what "a running job is never pruned" has to mean. Not
"the file says running" — a process killed outright never updates its own
status, so that reading would make every crashed job immortal. Those are
exactly the ones that accumulate, so the naive rule produces the opposite of
retention: the only logs that never go away are the ones nobody wants. The
check consults the process table instead.

Verified both directions. Live, a running 30-second job survived a prune to
--keep 1. Pinned with a fixture holding a finished job, a corpse (record says
running, pid gone), and a genuinely live one — asserting the live one survives
and the corpse does not. Proven to fail by dropping the liveness check.

One false alarm worth recording: my first live test looked exactly like the bug,
showing a running job pruned. It was not — my commands ran two minutes apart, so
the "20-second" job had finished long before. The test was invalid, not the
guard. A timing-sensitive check across separate shell turns proves nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-31 17:27:25 +02:00
co-authored by Claude Opus 5
parent 3b211a5450
commit de69bd70b4
6 changed files with 207 additions and 1 deletions
+45
View File
@@ -111,9 +111,54 @@ def spawn_detached(argv: list[str]) -> str:
"status": "running",
},
)
# At write time, not on a schedule. A retention pass that depends on someone
# remembering to run it is one that silently never happens.
prune()
return job_id
# Measured 2026-08-31: a chatty short job (10 progress events) writes ~1.8 KB
# across its three files. A generator emitting per-body progress might reach
# 100 KB. 100 jobs is therefore single-digit megabytes at worst, inside .cache/
# which is gitignored scratch — so the failure mode of this number being wrong
# is disk, never data. Raise it freely if a real batch proves it tight.
DEFAULT_KEEP = 100
ENV_KEEP = "SR_JOB_KEEP"
def prune(keep: int | None = None) -> list[str]:
"""Delete all but the `keep` most recent jobs. Returns the ids removed.
**A genuinely running job is never pruned** — but note what that has to
mean. Skipping anything whose *file* says "running" would make every
crashed job immortal, because a process killed outright never gets to
update its own status. Those are precisely the jobs that accumulate, so the
check is against the process table, not the record.
Called at spawn time rather than by a sweep: a cleanup nothing invokes is a
cleanup that does not happen.
"""
if keep is None:
raw = os.environ.get(ENV_KEEP)
keep = int(raw) if raw and raw.isdigit() else DEFAULT_KEEP
directory = jobs_dir()
# Ids sort chronologically because they are timestamp-first (T-1276).
ids = sorted((path.stem for path in directory.glob("*.json")), reverse=True)
removed: list[str] = []
for job_id in ids[keep:]:
meta = read_meta(job_id)
if meta and meta.get("status") == "running" and is_alive(int(meta.get("pid", -1))):
continue # actually running — a long generator may outlive the window
for path in (log_path(job_id), output_path(job_id), meta_path(job_id)):
path.unlink(missing_ok=True)
removed.append(job_id)
return removed
def finish_if_detached(exit_code: int) -> None:
"""Record completion — called by the CHILD, from the outermost decorator.