jpmschweitzer/settled-reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(simulation): sprint 38 server — generation pipeline Phase 1 #142

Closed
jpmschweitzer wants to merge 0 commits from sprint-38/server into main
pull from: sprint-38/server
merge into: :main
Conversation 3 Commits - Files Changed -
jpmschweitzer commented 2026-05-03 09:48:29 +02:00
Owner
Copy Link
Copy Source

Summary

  • 25 D-records (D-194–D-218) formalizing the full generation cascade from heightmap to walkable tile
  • 10-module server/src/atlas/ package (3,326 lines): heightmap loader, BodyWorldState LRU cache, D8 drainage routing, background generation queue (Rayon), five-phase attractor matching, three-component district mix, block irregularity, tile conditions, Phase 1 skeleton generator
  • Atlas data pipeline — 5 new schema tables, 3 new Python importers (heightmaps, city names, province boundaries), economic_role normalization
  • Generation pipeline Rust types — WorldTier fix (D-218), 10 enum implementations, CityGenerationContext, SystemNameIndex (Aho-Corasick)
  • Bug fixes — 6 Bevy baseline test panics resolved (#885), suffix monotony auto-fix (#886), stamp expansion (#892)

31 files changed, ~5,800 insertions. 1,211 tests passing, zero failures. Closes #885 #886 #892 #899 #900 #901 #902 #903 #904 #905 #906 #907 #908 #909 #910 #911 #912 #913 #914 #916 #917 #918 #919 #920 #921 #922 #923 #924 #925 #926.

Test plan

  • cargo clippy -- -D warnings — zero warnings
  • cargo fmt --check — clean
  • cargo test --lib — 1,211 tests pass (62 new atlas tests)
  • make check-systems-db — 2 generators up to date
  • ruff check tooling/ — clean
  • Review D-records (D-194–D-218) for correctness against workshop outcomes
  • Verify atlas importers against real planet data (heightmaps, city names, province boundaries)
  • Spot-check skeleton generator output for a sample city

🤖 Generated with Claude Code

## Summary - **25 D-records** (D-194–D-218) formalizing the full generation cascade from heightmap to walkable tile - **10-module `server/src/atlas/` package** (3,326 lines): heightmap loader, BodyWorldState LRU cache, D8 drainage routing, background generation queue (Rayon), five-phase attractor matching, three-component district mix, block irregularity, tile conditions, Phase 1 skeleton generator - **Atlas data pipeline** — 5 new schema tables, 3 new Python importers (heightmaps, city names, province boundaries), economic_role normalization - **Generation pipeline Rust types** — WorldTier fix (D-218), 10 enum implementations, CityGenerationContext, SystemNameIndex (Aho-Corasick) - **Bug fixes** — 6 Bevy baseline test panics resolved (#885), suffix monotony auto-fix (#886), stamp expansion (#892) 31 files changed, ~5,800 insertions. 1,211 tests passing, zero failures. Closes #885 #886 #892 #899 #900 #901 #902 #903 #904 #905 #906 #907 #908 #909 #910 #911 #912 #913 #914 #916 #917 #918 #919 #920 #921 #922 #923 #924 #925 #926. ## Test plan - [x] `cargo clippy -- -D warnings` — zero warnings - [x] `cargo fmt --check` — clean - [x] `cargo test --lib` — 1,211 tests pass (62 new atlas tests) - [x] `make check-systems-db` — 2 generators up to date - [x] `ruff check tooling/` — clean - [ ] Review D-records (D-194–D-218) for correctness against workshop outcomes - [ ] Verify atlas importers against real planet data (heightmaps, city names, province boundaries) - [ ] Spot-check skeleton generator output for a sample city 🤖 Generated with [Claude Code](https://claude.com/claude-code)
jpmschweitzer added 13 commits 2026-05-03 09:48:30 +02:00
feat(simulation): add server/src/atlas/ — full Phase 1 generation pipeline b9fd7a3fc8 
Ten-module atlas package implementing the D-194–D-218 district generation
stack: heightmap loader, BodyWorldState LRU cache, D8 drainage routing,
background generation queue, five-phase attractor-matching, three-component
district mix, block irregularity, tile condition thresholds, and the Phase 1
skeleton generator that wires them into DistrictSkeleton.

Closes #916 #917 #918 #919 #920 #922 #923 #924 #899.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(simulation): resolve Bevy baseline test panics (#885) 38f9002844 
Three root causes: SnapshotBuffer hard-dependency in economy.rs
(Option-wrapped), TickPhase::configure missing from SimulationPlugin
(added idempotent call), and stale golden file after D-192 dropped
the version field (regenerated).

All 6 previously-failing tests now pass with zero regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
docs(decisions): formalize D-194 through D-218 generation cascade records 8abb8e4ec1 
25 D-records defining the full generation pipeline from heightmap to
walkable tile: WorldTier taxonomy (D-218), settlement classification
(D-196), city generation context (D-200), drainage routing (D-208),
attractor matching (D-211), district mix (D-194), and supporting
enums/types. Produced by workshop #897, formalized from ticket specs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(db): expand stamp GENERATOR_SOURCES for naming helpers (#892) 91066e4be6 
Add gemma_naming.py and naming_core.py to generate_atlas source
tracking in check-systems-db-stamp. Mirror in pr-push watch list.
Prevents naming helper changes from bypassing stale-stamp detection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix(atlas): suffix monotony auto-fix + cultural-history prompting (#886) d57f0566d2 
gemma_naming.py now re-queries affected bodies when >40% suffix
clustering is detected. naming_core.py build_batch_prompt accepts
cultural_history param threading secondary corridor substyles into
the few-shot prompt for richer cross-cultural name blending.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat(db): atlas data pipeline — schema, importers, and normalization (#901-#911) b6ca785f74 
New tables: atlas_body_heightmaps (D-202), atlas_city_names (D-207),
atlas_feature_names, atlas_province_boundaries (D-205), body_radius_km
column (D-204). Three new importers: heightmap BLOBs, city names from
wiki markers.json, province boundaries via D8 watershed analysis.
economic_role normalized to 7 canonical values (D-194). Stamp fix in
generate_atlas.py to hash all tracked source files. systems.db regenerated.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat(simulation): generation pipeline Rust types + SystemNameIndex (#900, #912-#926) ca9ab189e0 
WorldTier enum fixed to Epicenter/Regional/Backwater/Passage/Waypoint
(D-218). Full enum implementations for ComplexityTier, SettingType,
SettlementClass, DistrictType, PoliticalArchetype, FoundingOrientation,
TerritorialStatus, GeographicAttractor, AttractorType, and
CompatibilityMatrix. SystemNameIndex with Aho-Corasick text scanning
for background pre-generation queue integration (D-206).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(meta): update changelog 718fcd0510 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix(simulation): resolve clippy warnings in atlas modules 6582ee4cca 
Replace HashMap/HashSet with BTreeMap/BTreeSet per D-030 determinism
rule. Fix while_let_loop, map_or simplification, collapsible if,
unsigned_abs casting, iterator indexing, and redundant wildcard arms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merge remote-tracking branch 'origin/main' into sprint-38/server 64a873fc1a 
# Conflicts:
#	CHANGELOG.md
#	server/data/systems.db
#	tooling/check-systems-db-stamp
#	tooling/economy-db/import_economics.py
#	tooling/planet-gen/generate_atlas.py
chore(simulation): cargo fmt atlas modules f1278f88fd 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix(atlas): remove unused imports in import_province_boundaries.py 5ad8754b85 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(db): regen systems.db against merged sources 1a2bc58e27 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jpmschweitzer commented 2026-05-03 12:24:55 +02:00
Author
Owner
Copy Link
Copy Source

Review: sprint-38/server → main (type: code)

Hoshe (Code Quality): REQUEST_CHANGES

Well-structured, idiomatic Rust with strong test coverage. Six issues:

# Sev File Issue
1 Med gen_queue.rs Priority guarantee hollow — dispatch_next() fires on every submit(), pending Vec always empty, sort never consulted. Items run in submission order.
2 Med name_index.rs:~83 panic! in startup path that otherwise returns Option<Self>. Convert to tracing::error! + return None.
3 Low skeleton_gen.rs:~267 "Safety: BlockPlacement is Copy-able" — type is Clone not Copy, and Safety: implies unsafe. Remove/rewrite.
4 Med district_mix.rs + skeleton_gen.rs Duplicated LCG with divergent seed init. Unify into shared atlas::rng module.
5 Low import_province_boundaries.py Pure-Python loops on 131k-cell grid. Document expected runtime or vectorize.
6 Med import_province_boundaries.py No per-body transaction boundary. Interrupted run loses all progress.

Tyre (Architecture + D-Records): REQUEST_CHANGES

Clean module separation, solid determinism discipline. 20/25 D-records fully pass. Three required fixes:

# Sev File Issue
1 High skeleton_gen.rs — derive_complexity() D-218 violation: Backwater grouped with Passage (→ Moderate). D-218 says ceiling = Full. Fix: move to Epicenter/Regional arm + add test.
2 High systems-schema.sql Missing atlas_city_positions table. D-211 output has no persistence path.
3 Med systems-schema.sql atlas_city_names missing settlement_class column for Tier A routing.
4 Low attractor_matching.rs D-210 (SubBiomeVariant) unimplemented. Needs impl or deferral ticket.
5 Low check-systems-db-stamp import_city_names.py not in GENERATOR_SOURCES.

Cascade completeness: Infrastructure complete (heightmap → drainage → attractors → district mix → skeleton → tiles). End-to-end wiring intentionally stubbed at run_work_item() — acceptable Phase 1 boundary. Missing persistence for city placements (issue #2) is the key gap before the pipeline can produce output.

Verdict: CHANGES REQUESTED

7 required fixes: D-218 Backwater complexity, missing atlas_city_positions table, priority queue dispatch logic, panic→soft-failure in name_index, settlement_class column, per-body transactions, LCG unification.

4 lower-severity (fix or ticket): Safety comment, D-210 impl, stamp registration, Python perf docs.

## Review: sprint-38/server → main (type: code) ### Hoshe (Code Quality): REQUEST_CHANGES Well-structured, idiomatic Rust with strong test coverage. Six issues: | # | Sev | File | Issue | |---|-----|------|-------| | 1 | Med | `gen_queue.rs` | Priority guarantee hollow — `dispatch_next()` fires on every `submit()`, pending Vec always empty, sort never consulted. Items run in submission order. | | 2 | Med | `name_index.rs:~83` | `panic!` in startup path that otherwise returns `Option<Self>`. Convert to `tracing::error!` + `return None`. | | 3 | Low | `skeleton_gen.rs:~267` | "Safety: BlockPlacement is Copy-able" — type is Clone not Copy, and `Safety:` implies unsafe. Remove/rewrite. | | 4 | Med | `district_mix.rs` + `skeleton_gen.rs` | Duplicated LCG with divergent seed init. Unify into shared `atlas::rng` module. | | 5 | Low | `import_province_boundaries.py` | Pure-Python loops on 131k-cell grid. Document expected runtime or vectorize. | | 6 | Med | `import_province_boundaries.py` | No per-body transaction boundary. Interrupted run loses all progress. | ### Tyre (Architecture + D-Records): REQUEST_CHANGES Clean module separation, solid determinism discipline. 20/25 D-records fully pass. Three required fixes: | # | Sev | File | Issue | |---|-----|------|-------| | 1 | High | `skeleton_gen.rs` — `derive_complexity()` | **D-218 violation:** Backwater grouped with Passage (→ Moderate). D-218 says ceiling = Full. Fix: move to Epicenter/Regional arm + add test. | | 2 | High | `systems-schema.sql` | Missing `atlas_city_positions` table. D-211 output has no persistence path. | | 3 | Med | `systems-schema.sql` | `atlas_city_names` missing `settlement_class` column for Tier A routing. | | 4 | Low | `attractor_matching.rs` | D-210 (SubBiomeVariant) unimplemented. Needs impl or deferral ticket. | | 5 | Low | `check-systems-db-stamp` | `import_city_names.py` not in GENERATOR_SOURCES. | **Cascade completeness:** Infrastructure complete (heightmap → drainage → attractors → district mix → skeleton → tiles). End-to-end wiring intentionally stubbed at `run_work_item()` — acceptable Phase 1 boundary. Missing persistence for city placements (issue #2) is the key gap before the pipeline can produce output. ### Verdict: CHANGES REQUESTED **7 required fixes:** D-218 Backwater complexity, missing atlas_city_positions table, priority queue dispatch logic, panic→soft-failure in name_index, settlement_class column, per-body transactions, LCG unification. **4 lower-severity (fix or ticket):** Safety comment, D-210 impl, stamp registration, Python perf docs.
jpmschweitzer added 1 commit 2026-05-03 15:30:59 +02:00
fix(simulation): address PR #142 review — 11 issues resolved 332b9404be 
Rust fixes: D-218 Backwater complexity (moved to Full arm), priority
queue dispatch gated on thread saturation, panic→soft-fail in
name_index, duplicated LCG unified into atlas::rng module, misleading
Safety comment removed, D-210 SubBiomeVariant deferred to #948.

Schema/data fixes: atlas_city_positions table (D-211 persistence),
settlement_class column on atlas_city_names, per-body transaction
boundaries in import_province_boundaries, import_city_names.py added
to GENERATOR_SOURCES, Python perf documented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jpmschweitzer commented 2026-05-03 15:40:29 +02:00
Author
Owner
Copy Link
Copy Source

Review Round 2: sprint-38/server → main

Tyre (Architecture): APPROVE

All 5 previous issues resolved correctly. D-218 Backwater fixed with tests, atlas_city_positions table added, settlement_class column present, D-210 deferred with ticket #948, import_city_names registered in stamp. No new issues.

Hoshe (Code Quality): REQUEST_CHANGES

Previous 6 issues addressed but priority queue fix is incomplete:

# Sev File Issue
A Med gen_queue.rs in_flight only tracks AnalyzeBody (via item.body_id()). GenerateSkeleton/FillChunk never enter it → saturation gate is inert for those types → all non-body tasks dispatch immediately, bypassing priority. Fix: track ALL in-flight tasks with a separate counter (not keyed to body_id).
B Med gen_queue.rs priority_ordering_respected_under_saturation test passes for wrong reason. Items never enter in_flight, so observed order is Rayon FIFO, not priority. Either use AnalyzeBody items in the test, or fix A first so all types are tracked.
C Low import_province_boundaries.py:~447 Comment says "per-body savepoint" — actually uses BEGIN/COMMIT via with conn:, not SAVEPOINT. Fix the comment.
D Low server/src/atlas/rng.rs new() and new_mixed() encode per-caller seeding conventions in the shared module. Callers should apply their own seed transform before passing to a single constructor.

Verdict: CHANGES REQUESTED

Fix A is the key item — add a general in_flight_count: usize alongside the body-specific dedup set. Increment on dispatch, decrement on completion. Gate dispatch_next() on in_flight_count >= n_threads.

## Review Round 2: sprint-38/server → main ### Tyre (Architecture): APPROVE All 5 previous issues resolved correctly. D-218 Backwater fixed with tests, atlas_city_positions table added, settlement_class column present, D-210 deferred with ticket #948, import_city_names registered in stamp. No new issues. ### Hoshe (Code Quality): REQUEST_CHANGES Previous 6 issues addressed but priority queue fix is incomplete: | # | Sev | File | Issue | |---|-----|------|-------| | A | Med | `gen_queue.rs` | `in_flight` only tracks `AnalyzeBody` (via `item.body_id()`). GenerateSkeleton/FillChunk never enter it → saturation gate is inert for those types → all non-body tasks dispatch immediately, bypassing priority. Fix: track ALL in-flight tasks with a separate counter (not keyed to body_id). | | B | Med | `gen_queue.rs` | `priority_ordering_respected_under_saturation` test passes for wrong reason. Items never enter `in_flight`, so observed order is Rayon FIFO, not priority. Either use `AnalyzeBody` items in the test, or fix A first so all types are tracked. | | C | Low | `import_province_boundaries.py:~447` | Comment says "per-body savepoint" — actually uses BEGIN/COMMIT via `with conn:`, not SAVEPOINT. Fix the comment. | | D | Low | `server/src/atlas/rng.rs` | `new()` and `new_mixed()` encode per-caller seeding conventions in the shared module. Callers should apply their own seed transform before passing to a single constructor. | ### Verdict: CHANGES REQUESTED Fix A is the key item — add a general `in_flight_count: usize` alongside the body-specific dedup set. Increment on dispatch, decrement on completion. Gate `dispatch_next()` on `in_flight_count >= n_threads`.
jpmschweitzer added 2 commits 2026-05-03 15:52:23 +02:00
fix(simulation): address PR #142 review round 2 0d09d905e4 
gen_queue: add in_flight_count tracking for all work item types (not
just AnalyzeBody). Rewrite saturation test with AnalyzeBody items.
Fix priority_ordering test thread count to match new gate.

rng: collapse to single AtlasRng::new(seed) constructor — callers
own their seed transform.

import_province_boundaries: fix "savepoint" comment to "transaction".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(db): regen systems.db after stamp source update 61daafe9ad 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jpmschweitzer commented 2026-05-03 15:58:53 +02:00
Author
Owner
Copy Link
Copy Source

Review Round 3: sprint-38/server → main

All R2 issues: FIXED

# Issue Status
A in_flight_count tracking all work types FIXED — Arc<Mutex> gates dispatch for all item types
B Priority test uses wrong items FIXED — now uses AnalyzeBody with distinct body_ids
C "savepoint" comment incorrect FIXED — now says "per-body transaction (BEGIN/COMMIT)"
D RNG carries per-caller seeding FIXED — single AtlasRng::new(seed), callers pre-mix

Clippy clean. No new issues.

Verdict: APPROVED

Tyre approved in R2. Hoshe approves in R3. Both reviewers satisfied — merge when ready.

## Review Round 3: sprint-38/server → main ### All R2 issues: FIXED | # | Issue | Status | |---|-------|--------| | A | in_flight_count tracking all work types | FIXED — Arc<Mutex<usize>> gates dispatch for all item types | | B | Priority test uses wrong items | FIXED — now uses AnalyzeBody with distinct body_ids | | C | "savepoint" comment incorrect | FIXED — now says "per-body transaction (BEGIN/COMMIT)" | | D | RNG carries per-caller seeding | FIXED — single AtlasRng::new(seed), callers pre-mix | Clippy clean. No new issues. ### Verdict: APPROVED Tyre approved in R2. Hoshe approves in R3. Both reviewers satisfied — merge when ready.
jpmschweitzer closed this pull request 2026-05-03 16:01:30 +02:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
jpmschweitzer (Jeroen Schweitzer)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: jpmschweitzer/settled-reach#142