refactor(db): split import_economics.py; single generator-source registry (T-1067)
import_economics.py 2,620 → 309 lines — a thin orchestrator keeping the exact CLI, single-transaction/rollback contract, and exit codes. The 16 import steps, MIGRATION_SQL, brands shell-out, validators, and stamp write now live in tooling/economy-db/economy_import/ (db, migration, economy, corporations, brands, bodies, atlas, specialization, traits, validators, stamp, paths, errors). Full type hints throughout. tooling/generator_sources.py replaces the triplicated source registry (importer / stamp checker / pr-process watch list — the skill now derives its list via --list). The registry stamps itself, and economy_import/ modules are globbed fail-closed, so a future module is stamped the moment it exists — closing the silently-weakened-stamp failure mode. Rider: connector config helpers centralized in tooling/db/common.py. Byte-identical behavior proven: full-import table dump diff EMPTY over 107,843 lines / 37 tables (volatile timestamp fields excluded); dry-run output parity; generated_brands.toml sha unchanged. make test-tooling PASS; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -35,10 +35,11 @@ invalidate the `import_economics` meta stamp even though the Python file
|
||||
itself didn't change.
|
||||
|
||||
The full set of source files contributing to the meta stamp SHA (the Python
|
||||
importer, the Rust brand binary sources, `tooling/schema_version.py`, and the
|
||||
authored economics TOMLs) is registered in the `GENERATOR_SOURCES` dict at the
|
||||
top of `tooling/check-systems-db-stamp` — that dict is the single source of
|
||||
truth, mirrored by `IMPORT_ECONOMICS_SOURCES` in `import_economics.py`.
|
||||
importer entrypoint + its `economy_import` module package, the Rust brand
|
||||
binary sources, `tooling/schema_version.py`, the authored economics TOMLs, and
|
||||
the registry itself) is defined once in `tooling/generator_sources.py` (T-1067)
|
||||
— imported by both the importer's stamp writer and `check-systems-db-stamp`,
|
||||
and listed via `python3 tooling/generator_sources.py --list`.
|
||||
|
||||
The surviving planet-gen importers (`import_heightmaps`,
|
||||
`import_province_boundaries`) are one-time build imports baked into the
|
||||
@@ -122,9 +123,11 @@ automatically before pushing.
|
||||
|
||||
The check script is `tooling/check-systems-db-stamp`. Run it interactively with
|
||||
`make check-systems-db` or `python3 tooling/check-systems-db-stamp --verbose`. The
|
||||
`GENERATOR_SOURCES` dict at the top of that script is the single registry — when
|
||||
you add a new generator or source file, update it there and mirror the change in
|
||||
the `/pr-push` skill's source-file watch list.
|
||||
`GENERATOR_SOURCES` dict in `tooling/generator_sources.py` is the single registry
|
||||
(T-1067) — the check script and the importer's stamp writer both import it, and
|
||||
the `/pr-push` skill derives its source-file watch list from
|
||||
`python3 tooling/generator_sources.py --list`. When you add a new generator or
|
||||
source file, register it there and nowhere else.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -219,24 +219,23 @@ If clean, continue.
|
||||
Check whether any file in the **source-file watch list** was modified on this branch
|
||||
versus `origin/main`. This list covers generator code AND the data files that feed them.
|
||||
|
||||
The generator-source paths below **must stay in sync** with `GENERATOR_SOURCES` in
|
||||
`tooling/check-systems-db-stamp` (PR #136 review T7) — if you add a new source file
|
||||
to the stamp, add it here too, and vice versa. Drift between the two lists reintroduces
|
||||
exactly the silent-stale-DB class of bug this skill exists to prevent.
|
||||
The stamped generator sources come from the shared registry
|
||||
`tooling/generator_sources.py` (T-1067) — the same module the stamp writer and
|
||||
`tooling/check-systems-db-stamp` use, so the lists can no longer drift (PR #136
|
||||
review T7). The CLI call below expands to one repo-relative path per line; the
|
||||
extra hardcoded entries are non-stamped watch items (retired/one-time planet-gen
|
||||
importers, the schema DDL whose SHA is stamped separately, and the data
|
||||
directories that feed the generators).
|
||||
|
||||
```bash
|
||||
git diff --name-only origin/main...HEAD -- \
|
||||
tooling/economy-db/import_economics.py \
|
||||
$(python3 tooling/generator_sources.py --list) \
|
||||
tooling/planet-gen/generate_atlas.py \
|
||||
tooling/planet-gen/gemma_naming.py \
|
||||
tooling/planet-gen/naming_core.py \
|
||||
tooling/planet-gen/import_city_names.py \
|
||||
tooling/planet-gen/import_heightmaps.py \
|
||||
tooling/planet-gen/import_province_boundaries.py \
|
||||
server/src/bin/generate_brands/main.rs \
|
||||
server/src/bin/generate_brands/names.rs \
|
||||
server/src/bin/shared/surname_corpus.rs \
|
||||
tooling/generate-brands \
|
||||
server/data/systems-schema.sql \
|
||||
wiki/star-systems/ \
|
||||
wiki/economics/ \
|
||||
|
||||
Reference in New Issue
Block a user