Files
settled-reach/tooling/check-systems-db-stamp
T
jpmschweitzerandClaude Opus 4.6 3d9dd7d909 chore(ci): merge brand pipeline into import_economics + harden review findings
Addresses all blocking + minor items from PR #136 review.

Architectural change (T2/H3 — the review's main complaint):

  generate_brands was previously a separate Rust binary that produced a TOML
  artifact, with its stamp written "on behalf" by import_economics.py at the
  end of its own run.  Reviewers flagged the invisible coupling: two sources
  of truth in a system designed to have one, and no way to tell from the
  stamp that one "generator" was really a subroutine of the other.

  import_economics now invokes tooling/generate-brands as the first step of
  its main() flow, before opening its own DB connection.  The TOML artefact
  is still produced and still committed (useful for diff-review of brand
  changes), but there's now one pipeline owner.  The meta table carries two
  rows (import_economics, generate_atlas) not three; the Rust binary's
  source SHA folds into import_economics' stamp via IMPORT_ECONOMICS_SOURCES.
  A MIGRATION_SQL DELETE cleans up pre-merge DBs that still have the
  orphan generate_brands row.

Other review items addressed in-line:

  H1  generate_atlas._write_stamp no longer commits — transaction ownership
      stays with the caller (matches import_economics pattern).  Stamp +
      atlas data now commit atomically; a failed stamp rolls back the
      atlas data rather than leaving a stamp-missing-data intermediate.

  H2  _file_sha1 (in both import_economics, generate_atlas,
      check-systems-db-stamp) raises FileNotFoundError on missing sources
      instead of silently contributing an empty-bytes hash.  A ghost-SHA
      convergence could otherwise produce vacuous "fresh" passes.

  H4  pre-push no-meta-table warning rephrased — was "run after next
      regeneration", now "run now if this DB was generated by you".

  T1  asset-pipeline.md determinism claim softened: the stamp is
      deterministic (same source → same recorded SHA), the DB binary is
      not (generated_at + SQLite rowids/freelist churn).

  T3  asset-pipeline.md gains a "migration escape hatch" section naming
      MIGRATION_SQL in import_economics.py as the only sanctioned path
      for direct writes, and forbidding hand-run sqlite-exec / one-off
      patch scripts / SQLite-GUI edits.

  T4  Makefile regen-db now runs as a single shell with `set -e`.  A
      failure in one generator halts the pipeline immediately, preventing
      the "stale data, fresh stamp" state where a later step stamped a
      DB whose earlier step had failed.  import_economics' exit code 2
      (coverage gate warning) remains explicitly tolerated.

  T5  pre-push stamp check now runs on a branch's first push too —
      compares against origin/main instead of origin/$BRANCH, closing
      the gap where a new branch could ship a stale DB via the first push.

  T6  check-systems-db-stamp fails closed on unknown generator_names in
      meta — a future branch adding a new generator without registering
      it in GENERATOR_SOURCES will now be rejected, not silently skipped.

  T7  /pr-push watch list gains a mutual cross-reference comment with
      GENERATOR_SOURCES in check-systems-db-stamp, plus the missing
      names.rs source file, so the two lists cannot silently drift.

Follow-up tickets created:

  #887 T8  decisions-orphan-tickets CLI — surfaces tickets whose
           decision_ref points at a non-existent D-record.
  #888 T9  meta.schema_version monotonic semver — for savegame migration
           lineage in Phase 5+ (SHA comparison can't be ordered).

Verified:

  make regen-db end-to-end — OK
  make check-systems-db    — OK, 2 generator(s) up to date
  STALE detection          — OK, verified by touching generate_atlas.py
  /pr-push watch list      — OK, flags this branch's changed sources
  decision show D-159      — OK, structured output with tickets + refs

Refs: #855 #856 #857 #858 #859 PR #136

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-22 08:38:41 +02:00

155 lines
5.1 KiB
Python

#!/usr/bin/env python3
"""
check-systems-db-stamp — verify that server/data/systems.db is up to date.
Reads the meta table from systems.db and checks that the stored SHA-1 of each
generator's source file(s) matches the current file content on disk.
Exit codes:
0 — DB is stamped and all generator SHAs match current sources
1 — DB is stale, has an unknown generator, or references a missing source file
2 — DB does not have a meta table (treat as unstamped — run make regen-db)
Usage (called by .config/hooks/pre-push):
tooling/check-systems-db-stamp
Usage (interactive):
tooling/check-systems-db-stamp --verbose
Decision refs: #855 (generator versioning), #857 (pre-push hook)
"""
import hashlib
import sqlite3
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
DB_PATH = REPO_ROOT / "server" / "data" / "systems.db"
# Maps generator_name (as stored in meta.generator_name) to the source
# file(s) whose SHA is stamped. The SHA is computed as SHA-1 of the
# concatenated bytes of all files in sorted order.
#
# import_economics' source set includes the Rust generate_brands binary it now
# invokes as a subroutine (#136 review T2/H3). Keep this list in sync with
# IMPORT_ECONOMICS_SOURCES in tooling/economy-db/import_economics.py.
GENERATOR_SOURCES: dict[str, list[Path]] = {
"import_economics": [
REPO_ROOT / "tooling" / "economy-db" / "import_economics.py",
REPO_ROOT / "server" / "src" / "bin" / "generate_brands" / "main.rs",
REPO_ROOT / "server" / "src" / "bin" / "generate_brands" / "names.rs",
REPO_ROOT / "tooling" / "generate-brands",
],
"generate_atlas": [
REPO_ROOT / "tooling" / "planet-gen" / "generate_atlas.py",
],
}
def file_sha1(*paths: Path) -> str:
"""SHA-1 of concatenated file contents (sorted paths).
Missing files raise FileNotFoundError rather than silently contributing
an empty-string hash (H2): a ghost SHA could mask real breakage when
stored and current SHAs converge on the empty-bytes digest.
"""
h = hashlib.sha1()
for p in sorted(paths):
if not p.exists():
raise FileNotFoundError(f"generator source not found: {p}")
h.update(p.read_bytes())
return h.hexdigest()
def check(verbose: bool = False) -> int:
"""Return exit code: 0 = fresh, 1 = stale, 2 = no meta table."""
if not DB_PATH.exists():
if verbose:
print(f"check-systems-db-stamp: {DB_PATH} not found — skipping check")
return 0
try:
conn = sqlite3.connect(str(DB_PATH))
rows = conn.execute(
"SELECT generator_name, generator_sha FROM meta"
).fetchall()
conn.close()
except sqlite3.OperationalError:
# meta table does not exist
if verbose:
print("check-systems-db-stamp: no meta table — systems.db has not been stamped")
print(" Run: make regen-db")
return 2
if not rows:
if verbose:
print("check-systems-db-stamp: meta table is empty — systems.db has not been stamped")
print(" Run: make regen-db")
return 2
stale: list[str] = []
unknown: list[str] = []
for generator_name, stored_sha in rows:
sources = GENERATOR_SOURCES.get(generator_name)
if sources is None:
# Unknown generator — fail closed (T6). A future branch adding a
# new generator without registering it here must update this map
# before the check will pass, preventing the "silent no-op" trap.
unknown.append(generator_name)
continue
try:
current_sha = file_sha1(*sources)
except FileNotFoundError as exc:
# Source file moved/deleted — explicit failure instead of
# silent empty-hash (H2).
print(
f"check-systems-db-stamp: BROKEN — {generator_name}: {exc}",
file=sys.stderr,
)
return 1
if current_sha != stored_sha:
stale.append(generator_name)
if verbose:
print(
f"check-systems-db-stamp: STALE — {generator_name}"
f"\n stored: {stored_sha}"
f"\n current: {current_sha}"
)
if unknown:
print(
"check-systems-db-stamp: UNKNOWN generator(s) in meta table: "
f"{unknown}",
file=sys.stderr,
)
print(
" Update GENERATOR_SOURCES in tooling/check-systems-db-stamp to "
"register them before pushing.",
file=sys.stderr,
)
return 1
if stale:
if not verbose:
print(
"systems.db is stale — run `make regen-db` before pushing.",
file=sys.stderr,
)
print(f" Stale generators: {stale}", file=sys.stderr)
return 1
if verbose:
print(f"check-systems-db-stamp: OK — {len(rows)} generator(s) up to date")
return 0
def main() -> None:
verbose = "--verbose" in sys.argv or "-v" in sys.argv
sys.exit(check(verbose=verbose))
if __name__ == "__main__":
main()