import_economics.py 2,620 → 309 lines — a thin orchestrator keeping the
exact CLI, single-transaction/rollback contract, and exit codes. The 16
import steps, MIGRATION_SQL, brands shell-out, validators, and stamp
write now live in tooling/economy-db/economy_import/ (db, migration,
economy, corporations, brands, bodies, atlas, specialization, traits,
validators, stamp, paths, errors). Full type hints throughout.
tooling/generator_sources.py replaces the triplicated source registry
(importer / stamp checker / pr-process watch list — the skill now derives
its list via --list). The registry stamps itself, and economy_import/
modules are globbed fail-closed, so a future module is stamped the moment
it exists — closing the silently-weakened-stamp failure mode.
Rider: connector config helpers centralized in tooling/db/common.py.
Byte-identical behavior proven: full-import table dump diff EMPTY over
107,843 lines / 37 tables (volatile timestamp fields excluded); dry-run
output parity; generated_brands.toml sha unchanged. make test-tooling
PASS; ruff clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- make test-tooling: planet-gen determinism guard + import_economics
--dry-run, wired into pre-push on TOOLING_CHANGED; ruff widened to
E4/E7/E9/F/W (90 safe auto-fixes applied; E402/E702/F841 ignored with
documented counts)
- one-generator reality fixed in DEVOPS.md, asset-pipeline rule, CLAUDE.md
(import_economics sole generator since #951/D-223); dead check-protocol
target deleted; DEVOPS hook/config sections rewritten from the actual
hook sources; team-patterns gate description updated (client+tooling)
- project.yaml: 0.2.0 → 0.4.0 per the 0.{phase}.{n} scheme, description
refreshed from the v0.1 Sova narration to cascade reality
- stale comment sweep: voxel.rs stub claims (all 8 families implemented),
cascade.rs TODO recited to T-1044, main.rs D-192 handshake claim,
relationships.rs/chunk_streaming.rs version targets → phase language
- gitignore: client/settings.db* e2e-run artifacts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pql cutover is stable, so remove the superseded SQLite planning tooling. Surgical
— only the ticket/decision/raw-SQL scripts (all settledreach.db-bound and replaced by
pql) are deleted; the asset/audio/wiki connectors and shared common.py stay.
Removed:
- tooling/db/{ticket,decision,decisions-sync,decisions_sync.py,sqlite-query,sqlite-exec,
sqlite-init,sqlite-seed,sqlite_connector.py}
- tooling/{db-backup,db-install} + docs/backups/settledreach.db.backup (the binary-DB
backup ritual; tickets now live in the git-tracked .pql/changelog/)
- tooling/check-decision-ids (dead stub, superseded by `pql decisions validate`)
- Makefile db-backup/db-install targets; SR_DB_PATH + tooling/db/{ticket,sqlite-*,
decision*} entries from .claude/settings.json (audio entries kept)
Updated docs to pql: DEVOPS.md (SQLite Access + Decisions System → pql), project
structure, ticket-cli closing note, asset-pipeline raw-SQL warning.
Kept (verified still imported by the asset connectors via common.ensure_venv): common.py,
config.json, audio/image/trellis/wiki connectors. The live settledreach.db file
(gitignored, repo-parent) is left on disk as a cold rollback only.
ruff clean; pql decisions validate ok (357 decisions / 1013 tickets).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopts the T-NNN convention (T-N == old #N == pql ticket id) across the active
operational layer: governance/ decision records, .claude/{rules,agents,skills},
CLAUDE.md, DECISIONS.md. 283 references rewritten.
Guarded against false positives (17 correctly skipped, each logged):
- PR references kept (PR #136/#138/... — PRs are a separate #-namespace)
- non-ticket numbers kept (#4122; the "#1 process failure" idiom; "task #3")
- only #N where N is an actual ticket id is rewritten; the 1-4 digit word-bounded
match also excludes 6-digit hex colours in the visual decision records
Git history is NOT rewritten (a commit's #N already equals T-N numerically), and
historical archives (docs/sprints, docs/discussions, docs/workshops) keep their
point-in-time #N. The /pr-process ticket-ID extraction logic moves to T-NNN in the
Phase 4 consumer cutover.
Verified: pql decisions validate ok; sync 357 records / 1057 refs / broken 0 (the
prose edits don't affect decision parsing or the tickets.decision_ref linkage).
Transform committed at tooling/pql-migrate/retag_ticket_refs.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
asset-pipeline.md (2 locations) and systems-schema.sql still pointed
at import_economics.py as the SCHEMA_VERSION home after the extraction
to a shared module.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. decisions_sync.py: fix refs_created inflation (check rowcount),
remove dead IntegrityError except block
2. Extract SCHEMA_VERSION to shared tooling/schema_version.py —
both generators import from single source of truth
3. generate_atlas.py: narrow bare except to OperationalError +
"duplicate column" check
4. check-systems-db-stamp: add cross-generator schema_version
agreement assertion (defense-in-depth)
5. decision wrapper: add show + orphan-tickets to usage text
6. Add schema_version.py to all three source watch lists
(GENERATOR_SOURCES, IMPORT_ECONOMICS_SOURCES, generate_atlas
_write_stamp) — prevents silent staleness on version bump
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace SHA-1 hash in meta.schema_version with an orderable semver
string ("1.0.0"). SHA preserved in new schema_sha column for tamper
detection. Enables savegame migration lineage in Phase 5+ — saves can
record their schema version and determine which migrations to apply.
Updated both generators, check-systems-db-stamp validation (rejects
old SHA-hex values), schema DDL, and asset-pipeline docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Addresses all blocking + minor items from PR #136 review.
Architectural change (T2/H3 — the review's main complaint):
generate_brands was previously a separate Rust binary that produced a TOML
artifact, with its stamp written "on behalf" by import_economics.py at the
end of its own run. Reviewers flagged the invisible coupling: two sources
of truth in a system designed to have one, and no way to tell from the
stamp that one "generator" was really a subroutine of the other.
import_economics now invokes tooling/generate-brands as the first step of
its main() flow, before opening its own DB connection. The TOML artefact
is still produced and still committed (useful for diff-review of brand
changes), but there's now one pipeline owner. The meta table carries two
rows (import_economics, generate_atlas) not three; the Rust binary's
source SHA folds into import_economics' stamp via IMPORT_ECONOMICS_SOURCES.
A MIGRATION_SQL DELETE cleans up pre-merge DBs that still have the
orphan generate_brands row.
Other review items addressed in-line:
H1 generate_atlas._write_stamp no longer commits — transaction ownership
stays with the caller (matches import_economics pattern). Stamp +
atlas data now commit atomically; a failed stamp rolls back the
atlas data rather than leaving a stamp-missing-data intermediate.
H2 _file_sha1 (in both import_economics, generate_atlas,
check-systems-db-stamp) raises FileNotFoundError on missing sources
instead of silently contributing an empty-bytes hash. A ghost-SHA
convergence could otherwise produce vacuous "fresh" passes.
H4 pre-push no-meta-table warning rephrased — was "run after next
regeneration", now "run now if this DB was generated by you".
T1 asset-pipeline.md determinism claim softened: the stamp is
deterministic (same source → same recorded SHA), the DB binary is
not (generated_at + SQLite rowids/freelist churn).
T3 asset-pipeline.md gains a "migration escape hatch" section naming
MIGRATION_SQL in import_economics.py as the only sanctioned path
for direct writes, and forbidding hand-run sqlite-exec / one-off
patch scripts / SQLite-GUI edits.
T4 Makefile regen-db now runs as a single shell with `set -e`. A
failure in one generator halts the pipeline immediately, preventing
the "stale data, fresh stamp" state where a later step stamped a
DB whose earlier step had failed. import_economics' exit code 2
(coverage gate warning) remains explicitly tolerated.
T5 pre-push stamp check now runs on a branch's first push too —
compares against origin/main instead of origin/$BRANCH, closing
the gap where a new branch could ship a stale DB via the first push.
T6 check-systems-db-stamp fails closed on unknown generator_names in
meta — a future branch adding a new generator without registering
it in GENERATOR_SOURCES will now be rejected, not silently skipped.
T7 /pr-push watch list gains a mutual cross-reference comment with
GENERATOR_SOURCES in check-systems-db-stamp, plus the missing
names.rs source file, so the two lists cannot silently drift.
Follow-up tickets created:
#887 T8 decisions-orphan-tickets CLI — surfaces tickets whose
decision_ref points at a non-existent D-record.
#888 T9 meta.schema_version monotonic semver — for savegame migration
lineage in Phase 5+ (SHA comparison can't be ordered).
Verified:
make regen-db end-to-end — OK
make check-systems-db — OK, 2 generator(s) up to date
STALE detection — OK, verified by touching generate_atlas.py
/pr-push watch list — OK, flags this branch's changed sources
decision show D-159 — OK, structured output with tickets + refs
Refs: #855#856#857#858#859 PR #136
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds `.claude/rules/asset-pipeline.md` describing the full pipeline —
generators, meta stamp, pre-push hook, how to make DB/schema changes,
and why direct systems.db edits are forbidden. CLAUDE.md and DEVOPS.md
point at the rules file; sprint-start template now reminds teammates
to read it before touching DB sources.
CLAUDE.md also gains the missing `decision show` row in the CLI tool
table (companion to #723's `decision show` subcommand).
Refs: #859#723
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jpmschweitzerandClaude Fable 5