settled-reach/docs/workshops/commodity-catalog/tyre-round1.md

Tyre — #801 TOML Schema & Data Pipeline Analysis

cracks knuckles — Let me be honest about what this means technically. The schema design here is the contract between the content authors and the simulation. Get it right and everything downstream compiles cleanly. Get it wrong and we're refactoring TOML files after 30 commodities and 150 production chains are already authored.

---

1. ID Scheme

Recommendation: flat string slugs. No namespace prefixing.

ore, rare_minerals, timber, agricultural_produce, water, energy, organics
refined_metals, structural_panels, processed_food, ...
legal_services, banking, tourism, ...

Why not namespaced IDs like raw.ore or intermediate.refined_metals?

• The tier is a field on the record, not an identity property. If we ever reclassify something (unlikely but possible), namespaced IDs break all references.
• Flat slugs are consistent with existing corp_id convention in systems.db ("gate-corporation", "vethara" — flat slugs, no hierarchy).
• TOML table keys with dots create nested tables in TOML syntax, which would fight us. [raw.ore] in TOML means raw is a table with key ore inside it — that's not what we want.
• Production chain references become simpler: inputs = [{ commodity = "ore", ... }] vs inputs = [{ commodity = "raw.ore", ... }].

Separator convention: underscores (refined_metals), matching the Rust/TOML convention. Hyphens (refined-metals) would match corp_id but underscores are more natural in TOML keys and Rust identifiers. Pick one, enforce it. I'd go underscores for the economics TOML domain since it's Rust-consumed.

---

2. TOML Schema: wiki/economics/commodities.toml

# Every commodity in the simulation. Source of truth — compiled to systems.db
# via `make economy-db`. Do NOT edit the .db directly.
#
# Fields:
#   name              — display name (English, Title Case)
#   tier              — raw | intermediate | final | service_professional | service_luxury
#   elasticity        — perfectly_inelastic | inelastic | unit_elastic | elastic | perfectly_elastic
#   base_price        — equilibrium price in Tractus (positive float)
#   bulk_class        — heavy | standard | light | non_physical
#   political_sensitivity — boolean; if true, price signals emit to political layer
#   unit              — tonnes | units | contracts
#   description       — short flavor text (optional, for wiki/UI)

[ore]
name = "Metallic Ore"
tier = "raw"
elasticity = "inelastic"
base_price = 12.0
bulk_class = "heavy"
political_sensitivity = false
unit = "tonnes"
description = "Unrefined ferrous and non-ferrous metallic ore. Extracted from planetary mines and asteroid operations."

[energy]
name = "Energy"
tier = "raw"
elasticity = "perfectly_inelastic"
base_price = 8.0
bulk_class = "non_physical"
political_sensitivity = true
unit = "units"
description = "Fusion output, solar collection, and geothermal harvest. Non-transportable through gates; consumed locally or converted to stored fuel."

[legal_services]
name = "Legal Services"
tier = "service_professional"
elasticity = "inelastic"
base_price = 45.0
bulk_class = "non_physical"
political_sensitivity = true
unit = "contracts"
description = "Commercial law, contract arbitration, Commission compliance certification. Location-bound."

Field rationale

Field	Type	Why
name	string	Display name. TOML key is the ID — name is for humans.
tier	enum (5 values)	Maps to D-173 categories. 5 tiers, not 3 — services are NOT in the raw→intermediate→final chain.
elasticity	enum (5 values)	Per D-173: "5 categories." These map to numeric coefficients in the sim (Burnelli-Sheldon's tâtonnement parameters).
base_price	float	Equilibrium price in Tractus. The tâtonnement starts here and adjusts. Must be positive.
bulk_class	enum (4 values)	Determines transport cost multiplier. non_physical = cannot traverse gates (services, energy).
political_sensitivity	bool	Flags commodities whose price movements emit signals to the political/event layer. Energy, food, legal services — yes. Hull plates — no.
unit	enum (3 values)	Keeps production chain quantities unambiguous. Tonnes for physical bulk goods, units for manufactured items, contracts for services.
description	string (optional)	Flavor text. Not consumed by the sim. Useful for wiki generation and UI tooltips.

What I deliberately excluded

• No transportable field. Redundant — derivable from bulk_class == "non_physical". Services can't traverse gates. Everything else can.
• No category sub-grouping within tiers (e.g., "extraction" vs "agriculture" within raw). The tier IS the grouping. Finer categorization is the production chain graph's job.
• No shadow_economy flag. D-174 says shadow economy is per-node intensity, not per-commodity. Contraband isn't a commodity property — it's a node-level enforcement gap. Any commodity can be traded in the shadow economy; what changes is the premium.
• No currency field. All base prices are Tractus-denominated (D-171: Tractus is the simulation numeraire). Mark and Sol conversions are applied by the sim at runtime.

---

3. TOML Schema: wiki/economics/production_chains.toml

# Production chain recipes. Each chain transforms input commodities into
# one output commodity. Multiple chains MAY produce the same output —
# this models structural substitution (D-173).
#
# Fields:
#   output            — commodity_id of the product
#   output_quantity   — units produced per production cycle (float, default 1.0)
#   inputs            — array of { commodity, quantity } pairs
#   location_bound    — if true, chain can only operate where specific
#                       resources exist (optional, default false)
#   description       — human-readable recipe description (optional)

[refined_metals_from_ore]
output = "refined_metals"
output_quantity = 1.0
inputs = [
    { commodity = "ore", quantity = 3.0 },
    { commodity = "energy", quantity = 0.5 },
]
description = "Standard smelting: 3t ore + 0.5 energy units → 1t refined metal."

[refined_metals_from_rare_minerals]
output = "refined_metals"
output_quantity = 0.5
inputs = [
    { commodity = "rare_minerals", quantity = 1.0 },
    { commodity = "energy", quantity = 1.0 },
]
description = "Specialty refining: rare minerals yield less bulk metal but higher value. Structural substitution route."

[structural_panels]
output = "structural_panels"
output_quantity = 1.0
inputs = [
    { commodity = "refined_metals", quantity = 2.0 },
    { commodity = "chemicals", quantity = 0.5 },
]
description = "Composite paneling for construction and hull repair."

[heavy_equipment]
output = "heavy_equipment"
output_quantity = 1.0
inputs = [
    { commodity = "structural_panels", quantity = 2.0 },
    { commodity = "components", quantity = 3.0 },
    { commodity = "drive_assemblies", quantity = 1.0 },
]
description = "Industrial machinery, mining rigs, construction equipment."

Key design decisions in the chain schema

1. Chain ID is the TOML key, output is a field. This allows multiple chains to produce the same output (structural substitution per D-173). The chain ID is descriptive (refined_metals_from_ore) but the sim uses output + inputs — the chain ID is just for human readability and DB primary key.

2. output_quantity defaults to 1.0. Most chains produce 1 unit. The field exists for substitution routes where yields differ (the rare minerals route above yields 0.5t per cycle).

3. location_bound is optional. Most chains are not location-bound. When true, the sim checks whether the corporation operating this chain has access to the required resource at its site. This connects to D-177's productivity constraints.

4. Services have NO production chains. Per D-173: "Services sit outside the production chain (Raw → Intermediate → Final). Services consume goods but do not produce them." Services are consumed by population/corporations but don't appear as chain inputs or outputs. Their supply is driven by service_throughput and service_capacity seeds (D-176).

---

4. Mapping to systems.db Tables

Here's how these TOML files compile to the #804 schema extensions:

-- Compiled from wiki/economics/commodities.toml
CREATE TABLE commodities (
    commodity_id TEXT PRIMARY KEY,        -- TOML key
    name TEXT NOT NULL,
    tier TEXT NOT NULL CHECK(tier IN (
        'raw', 'intermediate', 'final',
        'service_professional', 'service_luxury'
    )),
    elasticity TEXT NOT NULL CHECK(elasticity IN (
        'perfectly_inelastic', 'inelastic', 'unit_elastic',
        'elastic', 'perfectly_elastic'
    )),
    base_price REAL NOT NULL CHECK(base_price > 0),
    bulk_class TEXT NOT NULL CHECK(bulk_class IN (
        'heavy', 'standard', 'light', 'non_physical'
    )),
    political_sensitivity INTEGER NOT NULL DEFAULT 0,
    unit TEXT NOT NULL CHECK(unit IN ('tonnes', 'units', 'contracts')),
    description TEXT
);

-- Compiled from wiki/economics/production_chains.toml
CREATE TABLE production_chains (
    chain_id TEXT PRIMARY KEY,            -- TOML key
    output_commodity TEXT NOT NULL REFERENCES commodities(commodity_id),
    output_quantity REAL NOT NULL DEFAULT 1.0 CHECK(output_quantity > 0),
    location_bound INTEGER NOT NULL DEFAULT 0,
    description TEXT
);

CREATE TABLE chain_inputs (
    chain_id TEXT NOT NULL REFERENCES production_chains(chain_id) ON DELETE CASCADE,
    commodity_id TEXT NOT NULL REFERENCES commodities(commodity_id),
    quantity REAL NOT NULL CHECK(quantity > 0),
    PRIMARY KEY (chain_id, commodity_id)
);

-- Indexes
CREATE INDEX idx_commodities_tier ON commodities(tier);
CREATE INDEX idx_chains_output ON production_chains(output_commodity);
CREATE INDEX idx_chain_inputs_commodity ON chain_inputs(commodity_id);

Alignment notes for the server team (#804):

• Every column maps 1:1 from a TOML field. No computed columns in the compiled DB.
• political_sensitivity becomes INTEGER (0/1) — standard SQLite boolean convention, consistent with existing schema (inhabited, asteroid_belt, etc.).
• FK constraints ensure referential integrity at the DB level too, not just the build-step validator.
• The chain_inputs composite PK means one commodity appears at most once per chain. If a recipe needs the same input twice at different stages... it doesn't. Leontief fixed-coefficient means one coefficient per input per chain.

---

5. Validation Rules for make economy-db

The build step must enforce these invariants. Fail the build on violation — no warnings, no "best effort."

Hard failures (build aborts)

Rule	Rationale
Every inputs[].commodity in production_chains.toml must exist as a key in commodities.toml	Dangling reference = broken simulation
Every output in production_chains.toml must exist as a key in commodities.toml	Same
Every production chain must have ≥1 input	A chain with zero inputs is free production — breaks Leontief
No commodity may appear as both input and output in the same chain	Self-referential production is nonsensical
No circular dependencies in the chain graph	Detect via topological sort. If A→B→C→A, the sim will infinite-loop. Note: this is across chains, not within a chain.
base_price > 0 for all commodities	Zero or negative prices break tâtonnement
quantity > 0 for all chain inputs and outputs	Zero-quantity inputs are dead weight
All enum fields match their allowed values	Typo protection
No duplicate commodity IDs	TOML enforces this at parse level, but validate anyway
Services (tier = service_*) must NOT appear as chain inputs or outputs	D-173: services sit outside the production chain
bulk_class = "non_physical" commodities must NOT appear as chain inputs (except energy)	Non-physical goods can't be freighted. Energy is the exception — it's consumed locally in production but listed as non_physical because it can't traverse gates.

Soft warnings (build succeeds but prints warnings)

Rule	Rationale
Every non-service commodity SHOULD appear as either a chain input or chain output	Orphan commodities are suspicious — might be a typo or missing chain
Raw tier commodities SHOULD NOT appear as chain outputs	Raws are extracted, not produced. If a chain produces a raw, the tier classification may be wrong.
Final tier commodities SHOULD NOT appear as chain inputs	Finals are end-of-chain. If a final is consumed by another chain, the tier classification may be wrong.
≥3 production chains should exist per intermediate commodity	Ensures supply resilience in the sim — but this is a content completeness check, not a structural invariant

Coverage warnings (gated by corp data availability)

Rule	Rationale
3+ corporations per major commodity type (D-175)	Only enforceable once corp TOML data exists. Warn, don't fail.
1+ corporation per inhabited system >100K population (D-175)	Same — future gate

---

6. Extensibility

The schema is designed to grow without restructuring:

Adding commodities: Add a new [slug] block to commodities.toml. No schema change. No file restructuring. The TOML file is a flat list of records — 30 or 300 is the same structure.

Adding production chains: Add a new [chain_id] block to production_chains.toml. Multiple chains producing the same output is already supported (substitution routes).

Adding Tier 2/3 corporations: Corporation TOML files (separate from commodities) reference commodity IDs. Adding corps doesn't touch the commodity schema at all.

Splitting files if they get large: If commodities.toml grows past ~100 entries (unlikely for commodities, possible for chains), we can split into production_chains_raw.toml, production_chains_intermediate.toml, etc. The build step can glob wiki/economics/production_chains*.toml. But I'd defer this — ~30 commodities and ~40 chains fit comfortably in single files.

Future fields: If the sim needs new per-commodity data (e.g., spoilage rate, contraband premium), add the field to the TOML spec and the DB table. Existing records get a default. No migration needed — the DB is rebuilt from TOML every time.

---

7. Energy: A Design Note

Energy is the one commodity that doesn't fit neatly into the schema. It's:

• tier = "raw" — produced from natural sources
• bulk_class = "non_physical" — can't be freighted through gates
• But it IS a production chain input (smelting, manufacturing, etc.)

My validation rule above says "non_physical commodities must not appear as chain inputs except energy." This is an explicit carve-out. The alternative — making energy bulk_class = "standard" — would be wrong because energy genuinely cannot be shipped between systems.

The sim handles this correctly if energy is produced and consumed locally at each node. The transport graph simply never routes energy. The non_physical bulk class already means "transport cost = infinity."

Burnelli-Sheldon should confirm whether this carve-out is clean from the economic model's perspective, or whether energy should be modeled as a derived capacity constraint rather than a tradeable commodity.

---

8. What I Need From Other Proposers

• Burnelli-Sheldon: Validate the 5 elasticity class names. Confirm energy-as-commodity vs energy-as-constraint. Review the base_price field — is a single equilibrium price sufficient or does the tâtonnement need a price range (floor/ceiling)?
• Gestalt: Confirm services have NO production chain participation. If services "consume goods" (D-173), how is that modeled? Separate demand table? Or is service demand just a population-driven consumption rate with no chain?
• Miri: The description field is optional flavor text — but it'll appear in wiki/UI. Should Miri or Mellanie author those, or are they placeholder-quality for now?
• Paula: Which commodities get political_sensitivity = true? I've flagged energy and legal services. Food almost certainly. What else?

---

Difficulty Assessment

• TOML schema design: Tier 1 (straightforward). The schema is a flat record list with a secondary chain file. Nothing exotic.
• Build step (make economy-db): Tier 2 (moderate). TOML parsing + graph validation + SQLite emission. A Python script, ~200 lines. The circular dependency check is the trickiest part (topological sort on the chain graph).
• Schema alignment with #804: Tier 1 (straightforward). The SQL above is the target. Direct 1:1 mapping from TOML fields.
• Validation rules: Tier 2 (moderate). The hard failures are simple assertions. The coverage warnings require cross-referencing corp data that may not exist yet.

Total: this is a clean, well-bounded data engineering task. Feasible. The risk is in content completeness, not technical complexity.