docs(ci): document source-canonical asset pipeline rule
Adds `.claude/rules/asset-pipeline.md` describing the full pipeline — generators, meta stamp, pre-push hook, how to make DB/schema changes, and why direct systems.db edits are forbidden. CLAUDE.md and DEVOPS.md point at the rules file; sprint-start template now reminds teammates to read it before touching DB sources. CLAUDE.md also gains the missing `decision show` row in the CLI tool table (companion to #723's `decision show` subcommand). Refs: #859 #723 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.6
@@ -0,0 +1,121 @@
|
||||
# Asset Pipeline — Source-Canonical Rule
|
||||
|
||||
`server/data/systems.db` is a **read-only, deterministic snapshot** produced by the
|
||||
generator pipeline. It is checked in to the repo as a build artefact so the Godot
|
||||
client can ship it without a build step, but **it is never the source of truth**.
|
||||
|
||||
---
|
||||
|
||||
## The Golden Rule
|
||||
|
||||
> **Edit sources, not the DB.**
|
||||
|
||||
If you need to change economics data, modify the TOML/JSON source files.
|
||||
If you need to change atlas markers, modify the `markers.json` files.
|
||||
Never run `UPDATE` or `INSERT` directly on `server/data/systems.db` outside of a
|
||||
migration — those changes will be silently overwritten by the next `make regen-db`.
|
||||
|
||||
---
|
||||
|
||||
## What produces systems.db
|
||||
|
||||
Three generators write to `systems.db` in sequence:
|
||||
|
||||
| Generator | Command | Source files |
|
||||
|-----------|---------|--------------|
|
||||
| `generate_brands` | `tooling/generate-brands` | `server/src/bin/generate_brands/main.rs` |
|
||||
| `import_economics` | `python3 tooling/economy-db/import_economics.py` | `tooling/economy-db/import_economics.py` |
|
||||
| `generate_atlas` | `python3 tooling/planet-gen/generate_atlas.py --seed 42` | `tooling/planet-gen/generate_atlas.py` |
|
||||
|
||||
`make regen-db` runs all three in the correct order.
|
||||
|
||||
---
|
||||
|
||||
## The meta table stamp (#855, #856)
|
||||
|
||||
After every successful non-dry-run, each generator writes a row to the `meta` table:
|
||||
|
||||
```sql
|
||||
CREATE TABLE meta (
|
||||
generator_name TEXT PRIMARY KEY, -- 'import_economics' | 'generate_atlas' | 'generate_brands'
|
||||
schema_version TEXT NOT NULL, -- SHA-1 of server/data/systems-schema.sql at generation time
|
||||
generator_sha TEXT NOT NULL, -- SHA-1 of the generator source file(s)
|
||||
generated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
```
|
||||
|
||||
The `generator_sha` is the SHA-1 of the generator source file content. If the source
|
||||
changes and `regen-db` is not re-run, the stamped SHA will differ from the current file
|
||||
SHA — this is what the pre-push hook detects.
|
||||
|
||||
---
|
||||
|
||||
## How to make a DB change
|
||||
|
||||
### Normal data changes (economics, atlas markers)
|
||||
|
||||
1. Edit the source files (TOML, JSON, markers.json).
|
||||
2. Run `make regen-db`.
|
||||
3. Run `make check-systems-db` to confirm the stamp is fresh.
|
||||
4. Stage and commit:
|
||||
```bash
|
||||
git add server/data/systems.db
|
||||
git commit -m "chore(db): regen systems.db — <what changed>"
|
||||
```
|
||||
|
||||
### Schema changes (new tables or columns)
|
||||
|
||||
1. Add the DDL to `server/data/systems-schema.sql`.
|
||||
2. Add migration SQL to `MIGRATION_SQL` in `import_economics.py` if the change
|
||||
affects existing DBs (idempotent `CREATE TABLE IF NOT EXISTS` or `ALTER TABLE`).
|
||||
3. Run `make regen-db`.
|
||||
4. Stage `server/data/systems-schema.sql` and `server/data/systems.db` together.
|
||||
|
||||
---
|
||||
|
||||
## Pre-push hook (#857)
|
||||
|
||||
`.config/hooks/pre-push` (installed via `make install-hooks`) checks that whenever
|
||||
`server/data/systems.db` is in the push, its meta stamp matches the current generator
|
||||
source SHAs. If not, the push is rejected with:
|
||||
|
||||
```
|
||||
systems.db is stale — run `make regen-db` before pushing.
|
||||
Stale generators: ['import_economics']
|
||||
```
|
||||
|
||||
Fix: run `make regen-db`, stage `server/data/systems.db`, amend or add a commit.
|
||||
Or use `/pr-push` — it detects stale generator sources and reruns `make regen-db`
|
||||
automatically before pushing.
|
||||
|
||||
The check script is `tooling/check-systems-db-stamp`. Run it interactively with
|
||||
`make check-systems-db` or `python3 tooling/check-systems-db-stamp --verbose`.
|
||||
|
||||
---
|
||||
|
||||
## /pr-push integration (#858)
|
||||
|
||||
The `/pr-push` skill checks whether any generator source files are modified on the
|
||||
branch. If they are, it automatically runs `make regen-db` and stages the updated
|
||||
`server/data/systems.db` before pushing — preventing pre-push hook rejections on
|
||||
branches that modify generators without regenerating.
|
||||
|
||||
---
|
||||
|
||||
## Why direct DB edits are forbidden
|
||||
|
||||
Two branches that both commit `server/data/systems.db` changes produce a binary
|
||||
merge conflict. Git cannot diff or merge binary SQLite files. Sprint 36 hit this
|
||||
exact class of problem. The meta stamp + pre-push hook is the systematic fix:
|
||||
|
||||
- Regeneration is deterministic (same sources → same DB, byte-for-byte)
|
||||
- Only one branch modifies generator sources at a time (per team scope rules)
|
||||
- The pre-push hook is a hard blocker before the binary conflict can land
|
||||
|
||||
---
|
||||
|
||||
## Future: savegame migration lineage
|
||||
|
||||
The `meta.schema_version` field records the schema SHA at generation time. When the
|
||||
savegame system is built (Phase 5+), a save file can record which systems.db snapshot
|
||||
it derives from, enabling forward migration without branching the DB file itself.
|
||||
@@ -423,6 +423,9 @@ Task(
|
||||
|
||||
1. Read the sprint briefing: docs/sprints/sprint-{N}/{team}.md
|
||||
2. Read the decision files referenced in the briefing.
|
||||
If your work touches systems.db sources (markers.json, TOML files,
|
||||
or generator code), read .claude/rules/asset-pipeline.md before
|
||||
modifying anything.
|
||||
3. Check TaskList for available work.
|
||||
4. Claim an unblocked task (TaskUpdate with owner: your name),
|
||||
mark it in_progress, and implement it.
|
||||
|
||||
@@ -25,6 +25,12 @@ Full annotated tree: `.claude/rules/project-structure.md`
|
||||
|
||||
See [docs/DEVOPS.md](docs/DEVOPS.md) for build, test, lint, and CI procedures. All development operations go through the top-level `Makefile` — run `make` for a summary of targets.
|
||||
|
||||
### Asset pipeline
|
||||
|
||||
`server/data/systems.db` is a read-only canonical snapshot produced by the generator
|
||||
pipeline — never edit it directly. To regenerate: `make regen-db`. Full rules in
|
||||
`.claude/rules/asset-pipeline.md`.
|
||||
|
||||
## Development Cascade — First Things First
|
||||
|
||||
Development follows a strict cascade. Each phase has a concrete deliverable. **Do NOT discuss, design, or implement detail from a later phase while an earlier phase is incomplete.** If you encounter references to later-phase detail (room grammar, NPC bundles, heritage tokens, etc.) in documents or decisions, either ignore them silently and stay at the correct level, or flag that the reference is dragging attention to the wrong scope level and suggest it be rephrased or moved.
|
||||
@@ -89,7 +95,7 @@ The ticketing database (`settledreach.db`) is accessed via `SR_DB_PATH` env var
|
||||
| Sprints | `tooling/db/sprint status`, `start-work`, `prepare` | `/sprint-start` skill |
|
||||
| SQL queries | `tooling/db/sqlite-query "SELECT ..."` | — |
|
||||
| SQL writes | `tooling/db/sqlite-exec "UPDATE ..."` | — |
|
||||
| Decisions | `tooling/db/decision next`, `claim`, `check-dupes` | — |
|
||||
| Decisions | `tooling/db/decision show`, `next`, `claim`, `check-dupes` | — |
|
||||
|
||||
### Testing preferences
|
||||
|
||||
|
||||
+52
-5
@@ -207,13 +207,55 @@ Schema: `content/_schema/checklist.schema.json`. The checklist format feeds into
|
||||
- **Advisory** — when knowledge catalogs (`content/global/knowledge/*.yaml`) have no fact definitions yet: lists referenced fact_ids and exits cleanly.
|
||||
- **Enforcing** — when catalogs are populated: fails on any `fact_id` reference that doesn't match a canonical definition.
|
||||
|
||||
## Pre-commit Hooks
|
||||
## Asset Pipeline — Generator-Driven DB (#855, #856, #857)
|
||||
|
||||
`server/data/systems.db` is a **read-only canonical snapshot** produced by three
|
||||
generators. It is committed to the repo so the client can ship it, but it is never
|
||||
the source of truth. Direct edits are forbidden — they are silently overwritten by
|
||||
the next regeneration.
|
||||
|
||||
### Generators
|
||||
|
||||
| Generator | Source | Runs via |
|
||||
|-----------|--------|----------|
|
||||
| `generate_brands` | `server/src/bin/generate_brands/main.rs` | `tooling/generate-brands` |
|
||||
| `import_economics` | `tooling/economy-db/import_economics.py` | `python3 tooling/economy-db/import_economics.py` |
|
||||
| `generate_atlas` | `tooling/planet-gen/generate_atlas.py` | `python3 tooling/planet-gen/generate_atlas.py --seed 42` |
|
||||
|
||||
Run all three at once with:
|
||||
|
||||
```bash
|
||||
make regen-db
|
||||
```
|
||||
|
||||
### Meta table stamp
|
||||
|
||||
After every successful non-dry-run, each generator writes a row to the `meta` table in
|
||||
`systems.db` recording the SHA-1 of its source file(s) and the schema file.
|
||||
|
||||
```bash
|
||||
make check-systems-db # Verify the stamp is fresh (exit 1 = stale)
|
||||
```
|
||||
|
||||
### Making a DB change
|
||||
|
||||
1. Edit source files (TOML, JSON, `markers.json`).
|
||||
2. `make regen-db`
|
||||
3. `git add server/data/systems.db`
|
||||
4. Commit with `chore(db): regen systems.db — <reason>`
|
||||
|
||||
For schema changes, also update `server/data/systems-schema.sql` and add migration DDL
|
||||
to `MIGRATION_SQL` in `import_economics.py`.
|
||||
|
||||
See `.claude/rules/asset-pipeline.md` for the full rule set.
|
||||
|
||||
## Pre-commit and Pre-push Hooks
|
||||
|
||||
Git hooks are stored in `.config/hooks/` (version-controlled). Activate them with:
|
||||
|
||||
```bash
|
||||
make setup # Includes hook installation
|
||||
make setup-hooks # Just hooks
|
||||
make install-hooks # Just hooks (also makes them executable)
|
||||
```
|
||||
|
||||
Or manually:
|
||||
@@ -224,9 +266,14 @@ git config core.hooksPath .config/hooks
|
||||
|
||||
Active checks:
|
||||
|
||||
| Check | Script | Behavior |
|
||||
|-------|--------|----------|
|
||||
| fact_id validation | `tooling/check-fact-ids` | Warns if catalogs are stubs; fails on unknown fact_ids when populated |
|
||||
| Hook | Check | Script | Behavior |
|
||||
|------|-------|--------|----------|
|
||||
| pre-commit | fact_id validation | `tooling/check-fact-ids` | Warns if catalogs are stubs; fails on unknown fact_ids when populated |
|
||||
| pre-push | GDScript parse | internal | Fails on any SCRIPT ERROR |
|
||||
| pre-push | Rust lint | internal | fmt + clippy |
|
||||
| pre-push | Python lint | internal | ruff |
|
||||
| pre-push | JSON syntax | internal | python3 -m json.tool |
|
||||
| pre-push | systems.db stamp | `tooling/check-systems-db-stamp` | Rejects stale DB when pushed (#857) |
|
||||
|
||||
The `core.hooksPath` setting uses a relative path (`.config/hooks`) that resolves per worktree, so it works correctly across all worktrees in the repository.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user