settled-reach/docs/workshops/llm-voice-pipeline/miri-round2.md

title, description, type, status, workshop, agent, round, created

title	description	type	status	workshop	agent	round	created
Miri Round 2: World Consistency Evaluation	Convergent evaluation of world consistency safeguards and lore contamination risks	workshop	archived	llm-voice-pipeline	miri	2	2026-03-07

Round 2: World Consistency Evaluation — Miri

Workshop: LLM Voice Pipeline Domain: Worldbuilding / Setting Consistency / IP Originality Round: 2 (Convergent Evaluation)

---

Resolution Matrix

Question	Answer
Which proposal do you recommend?	A, with one named condition
Are there blockers in your recommended proposal?	Yes — one: the 150-token injector budget needs a hybrid instruction+example structure, not instruction-only. Details below.
Can you live with Proposal B?	Yes, conditional on spike proving >98% semantic core preservation before deployment
Can you live with Proposal C?	No for v0.2. Architecturally sound but wrong sequencing. Reasons below.
Minimum change to make B acceptable	Define the spike success threshold for tell semantic preservation explicitly (≥98%) and treat failure as automatic fallback to Proposal A passthrough
Minimum change to make C acceptable	Defer dialogue re-voicing to a follow-up sprint; treat C as A + "extend to dialogue after validation"

---

Q-R1-04: Is 150 Tokens Sufficient for Cultural Injector Clauses?

This is the question I raised in Round 1. Now that I have a concrete budget number (150 tokens ≈ 100-120 words of English), I can give a concrete answer.

What 150 tokens can encode

An aggressive compression of the Van Maanen's Star cultural injector:

---

Van Maanen's Star culture. Direct-informal register — short because time is scarce, not unfriendly. Competence earns respect; showing up matters more than rank. Community references: crew, shift, street. Exclamations ONLY from: "void take it" / "stars" / "blood and void" / "void's sake" / "cold vacuum." Greetings: hey, morning, shift treating you alright. Farewells: shift's calling, gotta move. NO religious oaths, NO quips, NO sir/ma'am deference. Filler words: look, right, yeah, so.

Word count: ~85 words. Token count: ~100-115 tokens. This fits within the 150-token budget.

What this encodes at 150 tokens:

• Speech register descriptor ✓
• Oath vocabulary (explicit list, mandatory constraint) ✓
• Greeting/farewell pool ✓
• Filler word pool ✓
• Minimal NOT-list (3 items) ✓
• Community anchor vocabulary ✓

What 150 tokens cannot encode

What is missing at 150 tokens:

1. The reason behind the register. "Direct-informal" describes the surface; it doesn't explain that Van Maanen's Star directness is compressed purposefulness — short because the work is real and time is genuinely scarce — not shortness-as-personality. A 2B model interpreting "direct-informal" without this context will produce casual American working-class dialogue. Van Maanen's Star is not casual American. It's earned competence under material constraint.

2. Void-oath philosophy. The culture RON describes Van Maanen's Star oaths as "void-adjacent — space is real here, and hostile. They don't swear by gods or governments. They swear by what kills you." This is the reason the oath vocabulary is what it is. A model that has the vocabulary list but not this context will use "void take it" as a rule, not as an instinct — and the register difference is visible in how and when the oath appears.

3. Deep NOT-list. 150 tokens gives room for 3-4 exclusions. The full exclusion set needed to prevent franchise bleed is 8-10 items. See Section 4 for the full universal negative injector set.

4. Social calibration. Van Maanen's Star culture is community-oriented but not warm-in-the-American-sense. Outsiders are "tolerated but watched." Loyalty "runs narrow and deep — to your crew, your shift, your street." This social topology shapes how NPCs interact with the player in ways that a register descriptor cannot convey.

150 tokens vs. 300 tokens: what changes

At 300 tokens, you can add:

• A worldview sentence: "Settled Reach workers live in sealed environments — space is the hostile reality outside the hull, not a romantic backdrop. Void-oaths reflect proximity to vacuum, not metaphor."
• Behavioral guidance: "Van Maanen's Star workers show competence through visible action, not through claiming status. They answer questions with the minimum needed and add context only when it affects the work."
• Extended NOT-list: Full set of 8-10 exclusion categories rather than 3.
• Social calibration: "Outsiders are politely watched, not warmly welcomed. Trust is earned through reliable work, not through friendliness."

The difference between 150 and 300 tokens is the difference between following rules and embodying a voice. At 150 tokens, the model follows a vocabulary list. At 300 tokens, it has enough philosophical context to make sensible judgment calls in edge cases the list doesn't cover.

Instruction-only vs. few-shot examples at this budget

Instruction-only at 150 tokens: Viable for vocabulary preservation. Risky for register. The model applies rules without understanding the cultural context behind them.

Few-shot only at 150 tokens: Not viable. A single example pair costs ~50-70 tokens. With 150 tokens, you can fit 2 example pairs and nothing else. A 2B model inferring cultural rules from 2 examples alone will generalize poorly.

Hybrid (instructions + examples) at 200-250 tokens: This is the recommendation. Specifically:

• ~80-90 tokens: minimal instruction set (register, oath vocabulary list, 3-4 NOT-items)
• ~120-140 tokens: 2 brief example pairs showing Van Maanen's Star voice in practice

Example pair format:

BASE: "checks the gate without looking at you"
VMS: "runs the check, eyes on the panel — gives you a nod when it clears"
---
BASE: "works on the conduit"
VMS: "traces the run with a handheld light, finds the splice, fixes it without ceremony"

Each pair: ~35-40 tokens. Two pairs: ~70-80 tokens. Adding these to the 100-token instruction core produces ~170-180 total — still under 250 tokens.

Why examples outperform instructions at 2B model size:

Small models are pattern matchers before they are instruction-followers. An example that demonstrates Van Maanen's Star register (spare phrasing, visible competence, no ceremony) is more reliably reproduced than an instruction describing the same properties in the abstract. The instruction tells the model what to do; the example shows it what the output looks like.

Verdict on Q-R1-04

150 tokens is sufficient for vocabulary preservation only. It is insufficient for register philosophy and provides a minimal NOT-list. The spike should test 150-token instruction-only against 200-token hybrid (instructions + 2 examples) and measure:

1. Oath vocabulary correct usage rate (target: >95%)
2. Register accuracy (blind review: "does this sound like the Settled Reach or generic SF?")
3. Franchise bleed rate (target: <2% of outputs contain recognizably non-Settled-Reach vocabulary or tone)

If the 150-token instruction-only version meets those thresholds, it's acceptable. My prediction: it meets criterion 1 but struggles with criteria 2 and 3. The hybrid version at 200 tokens is the recommendation.

---

Evaluating the Three Proposals Against World Consistency

Proposal A: Conservative — Behaviors Only, Tells Locked

World consistency assessment: Strongest of the three.

Proposal A's tell passthrough is the correct worldbuilding decision. Tells are not flavor — they are the observable surface of the information asymmetry mechanic (D-007 pillar 1). The base-text tell strings are authored to be precise. Passthrough preserves that precision absolutely.

The behaviors-only scope is also correct sequencing. Observable behaviors are short-form (5-15 words), the failure mode is bounded (a poorly re-voiced behavior is aesthetic damage, not mechanical damage), and the quality bar is clear (the current zone RON strings are the reference).

Specific world consistency concerns for Proposal A:

The 150-token injector budget (all three proposals share this for ambient behaviors) is adequate for vocabulary but requires the hybrid instruction+example structure I described above. If the injector is instruction-only at 150 tokens, the output will be Van Maanen's Star-vocabulary but not necessarily Van Maanen's Star-register.

The NOT-list in the injector needs the universal negative injectors (see Section 4). These are not culture-specific — they prevent franchise bleed for any culture, including cultures we haven't authored yet.

Lore contamination surface: Small. 5-15 word behaviors. Wrong vocabulary is visible and correctable. The specific cultural tell that a behavior uses wrong oath vocabulary is immediately audible.

My blocker for Proposal A: The injector architecture must use the hybrid instruction+example format at ~200 tokens, not instruction-only at 150 tokens. If Troblum can confirm 200 tokens is within throughput tolerance for the ambient behavior use case (shorter strings, higher volume), this is resolved.

Verdict: Recommended. Cleanest risk profile. Tell safety is absolute. Pipeline is testable. If the injector hybrid is confirmed viable, no remaining blockers.

---

Proposal B: Two-Track — Behaviors + Tells with Semantic Core

World consistency assessment: Appealing in theory, risk in practice.

I want to give Gestalt's semantic_core proposal credit — it's architecturally elegant and the theory is correct. A tell that reads "avoidance_behavior" in Van Maanen's Star dialect should express differently than one in a different culture. The player who has spent 20 hours in the Reach should learn to read Van Maanen's Star avoidance as distinct from other cultures' avoidance. That cultural specificity is worldbuilding-good.

The risk is asymmetric failure. Proposal A fails visibly and audibly (wrong vocabulary in a behavior line). Proposal B fails invisibly and mechanically — a tell that sounds fine but does not preserve the phenomenon it was authored to signal. That is a corrupted gameplay-information path that may not be detected in testing because it reads as acceptable prose.

The specific risk surface I'm watching:

The current tell grammar contains behaviors like:

• "affects exaggerated calm" — this is a precise observation: the NPC is performing composure, not naturally composed
• "becomes evasive and avoids eye contact" — two behaviors combined into one tell, which is what makes it readable
• "checks surroundings repeatedly" — frequency ("repeatedly") is load-bearing; "checks surroundings" is a different tell

Can a 2B model, given semantic_core = "suppression_behavior", preserve the "exaggerated" quality that distinguishes performed calm from natural calm? Can it preserve the "repeatedly" that makes the second tell a tell rather than a normal behavior? My concern is that the model preserves the category (avoidance, suppression, vigilance) but loses the specific qualifier that makes each tell readable as a tell rather than readable as neutral behavior.

Setting note: The void-oath vocabulary issue is more dangerous in tells than in ambient behaviors. An ambient behavior that uses wrong vocabulary is a minor lore break. A tell that uses wrong vocabulary may read as a different tell entirely — wrong signal, wrong player inference. If "becomes evasive and avoids eye contact" is re-voiced with a Van Maanen's Star register that produces "keeps to themselves, moves through the space quiet" — that is NOT a strong avoidance tell. It could be an introversion tell, a neutral behavior, or nothing. The vocabulary change produced a semantic shift.

Condition for acceptability: The spike must test Proposal B's constrained re-voicing on tell strings explicitly and measure whether the phenomenon survives at >98% accuracy. "Phenomenon survives" means: a blind reviewer, shown the base text tell and the re-voiced tell, identifies them as expressing the same observable pattern. Below 98%, fall back to Proposal A passthrough for tells.

Lore contamination surface: Medium. Same as A for ambient behaviors. Additional surface in tell re-voicing where cultural register change could corrupt signal.

Verdict: Acceptable with spike threshold condition. Not recommended over A for v0.2.

---

Proposal C: Full Pipeline — Behaviors + Dialogue, Tells Locked

World consistency assessment: Architecturally correct, wrong sequencing.

Dialogue is where cultural voice matters most to the player. What NPCs say is where Van Maanen's Star identity is most legible — their speech register, their void-oaths used naturally in conversation, their working-class pragmatism in how they respond to the player. I agree with Paula that dialogue is the highest-value target for re-voicing.

The problem is dialogue is also the highest-risk target for lore contamination at 2B model size.

Why dialogue is harder for small models:

Dialogue is longer (15-40 words), more contextually demanding (relationship state, conversation topic, access tier, trust tier), and more culturally legible — a player listens to an NPC speak for several sentences and forms a detailed cultural read. A single behavioral mis-register is a blip. A dialogue mis-register persists across the conversation.

The 400-500 token total prompt for dialogue means the cultural injector (~100-200 tokens) competes with dialogue context (~80 tokens) for the model's effective attention window. At 2B model size, longer prompts can dilute adherence to specific constraints — the model pays more attention to the most recent context and less to constraints stated earlier in the prompt. This means the cultural injector may get less weight in a 400-token dialogue prompt than in a 200-token behavior prompt.

The franchise bleed failure mode at dialogue scale:

For behaviors: "wrong vocabulary once" is the failure mode — detectable, bounded.

For dialogue: "sounds like the wrong franchise for the whole conversation" is the failure mode — immersive, corrosive. An NPC whose ambient behaviors are correctly Van Maanen's Star-voiced but whose dialogue sounds like a Mass Effect NPC creates a cognitive dissonance that damages trust in the setting. The player will notice "this feels like I've heard this before" more readily in dialogue than in brief behavioral observations.

The access/trust tier constraint adds surface:

Paula's observation that dialogue has access/trust tier tags is correct, and it's a structural advantage for information safety. But from a worldbuilding perspective, those tiers also change the register of the dialogue — a high-trust conversation with a Van Maanen's Star worker sounds different from a low-trust first encounter. A 2B model given cultural injectors + trust tier needs to combine two constraint sets simultaneously without collapsing either. That's a harder instruction-following task.

How to make C acceptable:

Treat C as "Proposal A + planned dialogue extension after spike validation." The architecture is the same. The sequencing is: ship A with behaviors re-voiced, validate quality in v0.2, extend to dialogue once the pipeline is proven. This de-risks the v0.2 quality bar while preserving the full-pipeline vision.

Lore contamination surface: Large. Behaviors + dialogue = two content types, two failure modes. The dialogue failure mode is higher-stakes and harder to catch in testing.

Verdict: Not recommended for v0.2. Recommend as explicit v0.3 target.

---

Lore Contamination Ranking

From smallest to largest contamination surface, across all three proposals:

A < B < C

Proposal	Contamination surface	Primary failure mode
A	Small	Wrong vocabulary in ambient behavior (aesthetic, catchable)
B	Medium	Wrong register in tell re-voicing (mechanical, subtle)
C	Large	Franchise bleed in extended dialogue (immersive, corrosive)

Specific guards each proposal needs:

Proposal A:

1. Universal negative injectors (see Section 4) — required for all cultures
2. Hybrid instruction+example injector format (200 tokens, not 150) — required for register accuracy
3. CulturalMarkers struct as injector source — ensures per-NPC markers are consistent with blueprint
4. Build-time validation pass on baked content: oath vocabulary check, franchise vocabulary blocklist

Proposal B (in addition to A's guards): 5. Semantic core vocabulary for every tell type authored before spike — "avoidance_behavior", "suppression_behavior", "vigilance_behavior", etc. 6. Spike success threshold: >98% phenomenon preservation on blind review before constrained re-voicing deploys 7. Semantic core reviewer: a post-re-voicing validation pass that checks whether the phenomenon category survives

Proposal C (in addition to A's guards): 8. Separate quality bar for dialogue vs. behaviors — dialogue must pass a longer blind review 9. Trust/access tier constraints authored as explicit injector components, not implicit from context 10. Dialogue-specific franchise bleed check: blocklist for recognizable genre dialogue patterns ("Commander, I've been expecting you", etc.) 11. Defer to v0.3 spike after A has been validated in production

---

Section 4: Universal Negative Injectors

These apply to re-voicing prompts for ALL cultures. They prevent the most common 2B-model failure modes regardless of culture-specific injector content.

Draft negative injectors:

NI-1: "Do not use religious language, prayer, references to deities, or spiritual practices of any kind. This universe's cultures do not have canonical religious expression in this context."

NI-2: "Do not use military rank titles (Commander, Captain, Sergeant, General, Admiral, Lieutenant, Officer). Authority structures in this setting use civilian and occupational titles only."

NI-3: "Do not reference technology that does not exist in this setting. Valid technology terms: insert (neural implant worn in the base of the skull), span gate (faster-than-light transit gate), the Reach (the interstellar network of settled systems). Do not use: holoscreens, blasters, force fields, teleporters, mind-reading, jump drives, warp speed."

NI-4: "Do not produce wit, banter, or jokes intended to be entertaining to the reader. Humor in this setting is dry, incidental, and rare. Do not add levity that was not present in the original text."

NI-5: "Do not reference Earth, nations, sports, politics, or other contemporary Earth-origin social structures. These characters have no memory of Earth; it is irrelevant to them."

Usage notes:

These five injectors total approximately 130-150 tokens. They cannot all be included in the 150-token cultural injector budget without crowding out culture-specific content. These should be encoded in the system prompt or a shared prompt prefix, not in the per-culture injector. This separates universal constraints (pipeline-level) from culture-specific constraints (authoring-level) and leaves the full 150-token budget for culture-specific content.

The split:

• System/prefix prompt: NI-1 through NI-5 (~130-150 tokens, shared across all cultures)
• Culture injector: register, vocabulary, NOT-lists that are culture-specific (~100-150 tokens)
• Trait modifier: personality expression in this culture (~20-30 tokens)
• Mood tag: current state (~5-10 tokens)

Total prompt overhead (before base text): ~270-340 tokens. Combined with base text (~20-40 tokens) and format instructions (~20-30 tokens), this is a ~310-410 token prompt for ambient behaviors. Within the behavior use case budget.

---

The CulturalMarkers Source-of-Truth Issue (Restatement)

I raised this in Round 1 and it survives into Round 2. All three proposals assemble cultural injectors. The question is: where do the per-NPC vocabulary items come from?

The NpcBlueprint.cultural_markers struct already contains per-NPC vocabulary assignments:

CulturalMarkers {
    speech_register: "direct, minimal pleasantries, gets to the point",
    filler_words: ["look", "yeah"],  // randomized subset from culture pool
    greeting: "shift treating you alright?",  // randomized from culture pool
}

These are per-NPC, generated once, seeded deterministically. If the injector re-derives vocabulary from the culture RON at prompt time, a given NPC might have filler_words: ["look", "yeah"] in their blueprint but the prompt might say "use filler words: look, right, yeah, so, listen, well" — a different set.

The fix is simple and I want it confirmed as architectural principle: The injector prompt assembler reads npc.cultural_markers.filler_words, npc.cultural_markers.greeting, and npc.cultural_markers.speech_register to construct per-NPC injector content, not the culture RON directly. The culture RON is the source for the template; the blueprint is the source for the instance.

This is not a blocker for any proposal, but it must be decided before the sr-voice crate is implemented. Otherwise we have two sources of truth for per-NPC vocabulary.

---

Summary Position

Recommended: Proposal A with:

1. Universal negative injectors in system/prefix prompt (shared, ~130-150 tokens)
2. Hybrid instruction+example format for culture injectors (~200 tokens, not 150 instruction-only)
3. NpcBlueprint.cultural_markers as the source for per-NPC vocabulary in injector assembly
4. Tell passthrough as absolute — no tell re-voicing in v0.2 under any circumstances
5. Proposal C (dialogue) as explicitly-stated v0.3 target in the D-record

The corrected Van Maanen's Star injector (placeholder, for Mellanie to refine):

[SYSTEM/SHARED] Do not use religious language. No military ranks. Technology terms
only: insert, span gate, the Reach. No wit or banter. No Earth references.

[VAN MAANEN'S STAR CULTURE] Direct-informal register — short because time is scarce, not
unfriendly. Competence earns respect. Community: crew, shift, street.
Exclamations ONLY: "void take it" / "stars" / "blood and void" / "cold vacuum."
Greetings: hey, morning, shift treating you alright.
Farewells: shift's calling, gotta move.
Fillers: [NPC-specific, from blueprint].
SHOW: BASE → VOICED
"checks the gate" → "runs the check, nods when it clears"
"works on the equipment" → "traces the fault, finds it, fixes it without ceremony"

This is a working draft, not final copy. The two examples demonstrate Van Maanen's Star's visible-competence-without-commentary register in practice. The full injector with the example pairs runs ~220-240 tokens — above 150 but within the 250-token soft ceiling I'd recommend for behavior re-voicing.

Setting note — the Proposal C vision is correct, just not for v0.2. Dialogue re-voicing with culture-vector injectors is the right long-term architecture for scaling Van Maanen's Star voice (and every other culture we build) across the full Reach. The risk profile for v0.2 is too high because we haven't yet proven the behavior pipeline. The correct statement is: build A, validate, extend to C in the next spike.