Rework the regen-db atlas path for the names-only marker pool:
- populate_atlas_city_names now reads the names.cities pool instead of
the retired geometry-bearing cities[] records; population/kind are
deferred to placement (#955). Adds a deterministic clear-then-insert
(no UNIQUE on (body_id, name)) that fixes a latent duplicate-
accumulation bug — atlas_city_names dropped from an inflated 3276 to a
clean 329 pooled names + 134 corp-HQ rows.
- ensure_atlas_index_schema applies the canonical ATLAS INDEX block from
systems-schema.sql and empties the 8 geometry tables every regen; the
Phase 4 server cascade fills them (they start empty — the revealed gap).
- Sol (system 'GJ 0') is permanently exempt from the normal generators:
skipped in both the name-pool importer and the corp-HQ cross-ref.
- MIGRATION_SQL drops the retired generate_atlas meta stamp row so the
fail-closed stamp checker accepts older committed DBs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deterministic ±scatter on body radii seeded by body_id hash — no two
bodies share the same radius. Gas giants 40k-60k km, moons 200-2600 km,
rocky planets ±15% from class base. Oort/asteroid skip radius (NULL).
Sol system gets real planetary radii. body_radius_km exported to
star_map_data.json for client orbital diagram sizing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New tables: atlas_body_heightmaps (D-202), atlas_city_names (D-207),
atlas_feature_names, atlas_province_boundaries (D-205), body_radius_km
column (D-204). Three new importers: heightmap BLOBs, city names from
wiki markers.json, province boundaries via D8 watershed analysis.
economic_role normalized to 7 canonical values (D-194). Stamp fix in
generate_atlas.py to hash all tracked source files. systems.db regenerated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. decisions_sync.py: fix refs_created inflation (check rowcount),
remove dead IntegrityError except block
2. Extract SCHEMA_VERSION to shared tooling/schema_version.py —
both generators import from single source of truth
3. generate_atlas.py: narrow bare except to OperationalError +
"duplicate column" check
4. check-systems-db-stamp: add cross-generator schema_version
agreement assertion (defense-in-depth)
5. decision wrapper: add show + orphan-tickets to usage text
6. Add schema_version.py to all three source watch lists
(GENERATOR_SOURCES, IMPORT_ECONOMICS_SOURCES, generate_atlas
_write_stamp) — prevents silent staleness on version bump
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace SHA-1 hash in meta.schema_version with an orderable semver
string ("1.0.0"). SHA preserved in new schema_sha column for tamper
detection. Enables savegame migration lineage in Phase 5+ — saves can
record their schema version and determine which migrations to apply.
Updated both generators, check-systems-db-stamp validation (rejects
old SHA-hex values), schema DDL, and asset-pipeline docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Addresses all blocking + minor items from PR #136 review.
Architectural change (T2/H3 — the review's main complaint):
generate_brands was previously a separate Rust binary that produced a TOML
artifact, with its stamp written "on behalf" by import_economics.py at the
end of its own run. Reviewers flagged the invisible coupling: two sources
of truth in a system designed to have one, and no way to tell from the
stamp that one "generator" was really a subroutine of the other.
import_economics now invokes tooling/generate-brands as the first step of
its main() flow, before opening its own DB connection. The TOML artefact
is still produced and still committed (useful for diff-review of brand
changes), but there's now one pipeline owner. The meta table carries two
rows (import_economics, generate_atlas) not three; the Rust binary's
source SHA folds into import_economics' stamp via IMPORT_ECONOMICS_SOURCES.
A MIGRATION_SQL DELETE cleans up pre-merge DBs that still have the
orphan generate_brands row.
Other review items addressed in-line:
H1 generate_atlas._write_stamp no longer commits — transaction ownership
stays with the caller (matches import_economics pattern). Stamp +
atlas data now commit atomically; a failed stamp rolls back the
atlas data rather than leaving a stamp-missing-data intermediate.
H2 _file_sha1 (in both import_economics, generate_atlas,
check-systems-db-stamp) raises FileNotFoundError on missing sources
instead of silently contributing an empty-bytes hash. A ghost-SHA
convergence could otherwise produce vacuous "fresh" passes.
H4 pre-push no-meta-table warning rephrased — was "run after next
regeneration", now "run now if this DB was generated by you".
T1 asset-pipeline.md determinism claim softened: the stamp is
deterministic (same source → same recorded SHA), the DB binary is
not (generated_at + SQLite rowids/freelist churn).
T3 asset-pipeline.md gains a "migration escape hatch" section naming
MIGRATION_SQL in import_economics.py as the only sanctioned path
for direct writes, and forbidding hand-run sqlite-exec / one-off
patch scripts / SQLite-GUI edits.
T4 Makefile regen-db now runs as a single shell with `set -e`. A
failure in one generator halts the pipeline immediately, preventing
the "stale data, fresh stamp" state where a later step stamped a
DB whose earlier step had failed. import_economics' exit code 2
(coverage gate warning) remains explicitly tolerated.
T5 pre-push stamp check now runs on a branch's first push too —
compares against origin/main instead of origin/$BRANCH, closing
the gap where a new branch could ship a stale DB via the first push.
T6 check-systems-db-stamp fails closed on unknown generator_names in
meta — a future branch adding a new generator without registering
it in GENERATOR_SOURCES will now be rejected, not silently skipped.
T7 /pr-push watch list gains a mutual cross-reference comment with
GENERATOR_SOURCES in check-systems-db-stamp, plus the missing
names.rs source file, so the two lists cannot silently drift.
Follow-up tickets created:
#887 T8 decisions-orphan-tickets CLI — surfaces tickets whose
decision_ref points at a non-existent D-record.
#888 T9 meta.schema_version monotonic semver — for savegame migration
lineage in Phase 5+ (SHA comparison can't be ordered).
Verified:
make regen-db end-to-end — OK
make check-systems-db — OK, 2 generator(s) up to date
STALE detection — OK, verified by touching generate_atlas.py
/pr-push watch list — OK, flags this branch's changed sources
decision show D-159 — OK, structured output with tickets + refs
Refs: #855#856#857#858#859 PR #136
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds systems.db regeneration discipline (#855) via a `meta` table (#856)
stamped by every generator, a pre-push hook that rejects stale DBs (#857),
and the top-level `make regen-db` / `make check-systems-db` targets that
drive the whole pipeline.
The stamp stores SHA-1 of generator source + schema, so the pre-push hook
can cheaply detect "you changed a generator but forgot to regen the DB"
before a binary merge conflict lands. Sprint 36 hit that class of conflict
on two branches touching systems.db simultaneously — this is the systemic
fix.
Regenerated systems.db is stamped; `make check-systems-db` passes.
Refs: #855#856#857
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
price_tier was documented in D-189 §5 and present in brands.toml but
silently discarded on import — absent from schema, INSERT, and V-B06.
- Add price_tier TEXT column to brand_products CREATE TABLE
- Add COLUMN_MIGRATIONS entry for ALTER TABLE on existing DBs
- Add VALID_PRICE_TIERS constant (mass/premium/luxury/flagship/institutional)
- Include price_tier in product_rows tuple and INSERT OR REPLACE
- Add price_tier to V-B06 enum checks; skip NULL (nullable column)
- Backfill 4 pre-amendment anchor brands (8 entries): Calloway flagship/premium,
VGV luxury/premium, thrds luxury/premium, Bífröst flagship/luxury
V-B01..V-B06 all pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- V-B06 enum validation: the five VALID_* sets
(VALID_BRAND_CATEGORIES, VALID_VALUE_TRAJECTORIES, VALID_SCARCITY_CLASSES,
VALID_BRAND_TIERS, VALID_CURRENCY_DENOMINATIONS) were defined but never
referenced. brand_products.brand_category etc. are plain TEXT with no
CHECK constraints, so a typo like `brand_category = "terrior"` silently
imported. `validate_brands` now runs a V-B06 pass that asserts every
enum column is a member of its VALID_* set. V-B01..V-B05 + V-B06 all
reported together on import failure.
- Explicit transaction wrapper: the clear-then-reimport cycle (10 DELETEs
followed by 9 imports and structural validation) used to depend on
Python's implicit-deferred-transaction semantics and sys.exit() on
validation failure. A crash mid-import could leave the DB with some
tables empty and others intact. The body now runs inside
`conn.execute("BEGIN")` + try/except with an explicit `_ImportAborted`
for validation failures and a `BaseException` catch-all for
KeyboardInterrupt / programmer errors. All failure paths rollback
before exit; the commit only fires after structural validation
passes. Dry-run leaves the transaction open so the coverage check
below can still SELECT against in-memory state.
- system_fiscal docstring: previously cited the D-189 §6 derived
formula (`collection_efficiency = 1.0 - shadow_economy_intensity × 0.6`)
while the implementation hardcodes `collection_efficiency = 0.85` for
every system. The docstring now explicitly states these are Phase 2
placeholder values (with named constants PHASE2_CORP_TAX_RATE and
PHASE2_COLLECTION_EFFICIENCY) and calls out the shadow_economy.toml
pipeline as the Phase 3 follow-up.
Adds the brand layer per D-189 §5:
- Schema: brand_products, brand_inputs, system_fiscal, corp_financial_state,
corp_lifecycle_events (+ 5 indexes).
- Importer: reads wiki/economics/corporations/brands.toml, populates the
new tables, validates V-B01–V-B05 structural rules, and derives
system_fiscal for inhabited systems.
- Data: 8 brand_products, 16 brand_inputs, 301 system_fiscal rows.
Brand products are demand nodes — they consume commodities; they are not
commodities themselves (D-185). Depends on copy PR #127 for the corp
records referenced by brands.toml.
Add D-180 EconEvent struct (target, effect, duration, visibility enums)
with no-op handler to satisfy #809 spec. Import MARK_PRIMARY and MIXED
currency zone assignments from wiki/economics/currency_zones.toml (D-172).
All four D-179 stability tests now pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
import_corp_presence() was inserting system IDs with location_type='system',
violating the schema which expects body/station IDs (location_type='body'|'station').
Fix:
- import_economics.py: add _resolve_hq_location() that picks the most-populated
body in the HQ system (falling back to any body, then any station)
- econ-sim/db.rs: load_corp_presences() now JOINs bodies/stations to recover
system_id from body/station location_ids, dropping the 'system' filter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends import_economics.py from 6-step to 8-step pipeline:
- Loads wiki corporation markdown frontmatter as authoritative source
- Syncs corporations table (hard error on proper_name divergence per D-182)
- Populates corp_presence table (one row per corp × headquarters system)
- Splits validation: structural checks block commit; coverage checks post-commit
- D-175 Phase 2 gate: 3+ corps per commodity, 1+ corp per system with pop > 100K
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 5 new tables: gate_links (668 bidirectional edges from star-map.json),
commodities (36 types from commodities.toml), production_chains (21
Leontief recipes), chain_inputs (52 input requirements), corp_presence
(empty, populated by future pipeline). Add currency_zone column to
star_systems (D-172), archetype columns to corporations (D-175).
New import pipeline: tooling/economy-db/import_economics.py reads
TOML/JSON source files and populates the DB. Idempotent — safe to
rerun via `make economy-db`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
