settled-reach/docs/workshops/llm-voice-pipeline/gestalt-round3.md

title, description, type, status, workshop, agent, round, created

title	description	type	status	workshop	agent	round	created
Gestalt Round 3: D-record, Tell-as-Context Design, Spike Criteria	D-record drafting, tell-as-context design, and spike 1 acceptance criteria	workshop	archived	llm-voice-pipeline	gestalt	3	2026-03-07

LLM Voice Pipeline Workshop — Round 3: D-record, Tell-as-Context Design, Spike 1 Criteria

Author: Gestalt Round: 3 — Decision and Commitment Date: 2026-03-07

---

1. D-138 Draft: LLM Re-voicing Pipeline for NPC Voice

Claimed ID: D-138 Domain: content Status: DRAFT — requires SI to formally record in decisions/content.md

---

D-138: LLM re-voicing pipeline for NPC voice

• Date: 2026-03-07
• Decision: NPC observable behaviors and dialogue are processed through an LLM re-voicing pipeline that translates culture-neutral semantic base text into character-voiced output. The pipeline is a background runtime enhancement, not a live generation system. Tell behaviors are base-text passthrough — always. Active tell state influences the re-voicing prompt for surrounding content without the tell text itself being re-voiced. The game is complete and functional without the pipeline; it is an enhancement that elevates voice quality for players with sufficient hardware.

Architecture:

Layer	What	How
Semantic base text	Culture-neutral behaviors and dialogue	Authored in RON files; serves as LLM seed, graceful fallback, and LLM-off experience simultaneously
Tell behaviors	Mechanical signals (avoidance, suppression, surveillance, etc.)	Base-text passthrough — NEVER sent to LLM. Always served as authored.
Tell context injectors	Active tell state influence on surrounding content	Per-TellCategory tone instructions that shape how behaviors/dialogue are re-voiced; tells inform without being re-voiced
Culture injectors	Culture-specific voice (register, oath vocabulary, negatives)	150–250 tokens per culture; sourced from CultureProfile.speech; negative injectors in shared prefix
Trait + mood modifiers	Personality and current emotional state	~10 tokens each; layered atop culture injector
Re-voiced output	Cached, player-facing voiced content	Generated per (NPC × tell_state × culture); cached at generation time; served at runtime by cache lookup

Content tiers:

1. Baked — Hub zones (Sova Transit District and other major locations) ship with pre-voiced content generated at build time, human-reviewed before shipping. This is the quality reference and the player's first-hours experience.
2. Pre-voiced — Background queue generates voiced content for adjacent zones before the player arrives. Priority: plot-critical NPCs first, then semi-unique, then ambient. Queue processes in a separate thread pool at below-normal priority.
3. Base text fallback — If pre-voicing hasn't completed, base text is served. Designed to be neutral, not broken. Pre-voicing catches up in the background; voiced content is ready on the player's next visit.

Tell-state variant caching: For each behavior and dialogue line, the pipeline pre-voices one version per TellCategory state (Neutral + Nervous + Angry + Friendly + Guarded + RoutineDeviation = 6 variants). At runtime, the game reads the NPC's current active tell state and serves the matching pre-voiced variant. No runtime inference is triggered by tell-state changes — it is a cache lookup.

Data model changes:

// NpcBlueprint — tell_behaviors as first-class field, routing by field not content
pub struct NpcBlueprint {
    pub observable_behaviors: Vec<String>,   // → free re-voicing queue
    pub tell_behaviors: Vec<TellBehavior>,   // → base-text passthrough always
    // ...
}

pub struct TellBehavior {
    pub category: TellCategory,     // Nervous | Angry | Friendly | Guarded | RoutineDeviation
    pub base_text: String,          // base text — also the final shipped text
}

// Individual lines — anchor line protection (Paula, N-2)
pub struct VoicedLine {
    pub base_text: String,
    pub anchor_line: bool,   // true = passthrough regardless of field; protects Tier 1/2 notable NPC lines
}

Model provenance: Gemma 2B (Google) primary, quantized Q4_K_M (~1.5GB). Phi-3 (Microsoft) as fallback if Gemma 2B fails quality bar in the spike. No Chinese-origin models (Qwen/Alibaba excluded). Reconsider only if both candidates fail benchmarks.

Inference runtime: llama-cpp-rs with GGUF Q4_K_M quantization. Separate thread pool from world generation to prevent memory bandwidth contention.

Hardware detection (layered, no hard floor):

1. RAM check — can the model load alongside the game?
2. Time-per-token benchmark on first enable — background inference latency estimate
3. Recommendation to disable if below threshold; player can always override
4. "AI-Enhanced Dialogue" toggle always present — OFF delivers base text everywhere

Distribution: Model bundled in the game install (~1.5GB added). No optional download step for the base model.

Two-spike delivery plan:

• Spike 1: Rust llama-cpp-rs wrapper (plumbing only) + manual prompt experiments (Jeroen, Mellanie, Paula). Validates model choice and prompt architecture. No game integration.

• Spike 2: Full integration — pre-voicing queue, cache-as-determinism, thread pool isolation, baked content generation, hardware detection, fallback behavior.

• Rationale: D-122 (all NPCs generated) and D-128 (culture implicit in starting location) require NPC voice to scale across zones and cultures without O(R×Z×C) hand-authoring. The re-voicing model — translate culture-neutral semantic base text into character voice — is the only architecture that scales while preserving content quality. The base-text fallback ensures the game is complete without the pipeline. Tell-as-passthrough with context influence preserves the information asymmetry mechanic (D-010) while giving tells cultural texture through their influence on surrounding content. Tells are READ-ONLY inputs; the LLM never owns tell text.

• Raised by: LLM Voice Pipeline Workshop (2026-03-07), full team. Jeroen's decisions are the binding inputs.

• Dissent: Miri flagged concern about cultural philosophy at 2B model size — addressed via hybrid injector format (instruction + example pairs) and spike validation.

• Amends: D-123 — see amendment text below.

• Supersedes: D-124 (in-game AI deferred — the door is now open and entered).

• Cross-reference: D-010 (information boundaries — tells are READ-ONLY inputs; LLM cannot produce game state), D-121 (culture-primary voice), D-122 (all NPCs generated), D-128 (culture implicit in location), D-029 (NPC tier model — tier mapping for re-voicing priority)

D-123 Amendment text:

D-123 is amended as follows: The AI pipeline operates in two modes. Baked mode: content is generated at build time and reviewed by humans before shipping — this preserves D-123's authorial control constraint. Runtime mode: content is generated in the background during gameplay without per-line human review, when "AI-Enhanced Dialogue" is enabled. All other D-123 constraints remain binding in both modes: culture vectors are the primary prompt constraint, the AI does not default to genre conventions, and authorial control governs what the LLM may and may not produce. The AI pipeline does not drive live narrative decisions — it applies voice to authored semantic content. Culture profiles (Miri) remain the primary authoring deliverable.

---

2. Tell-as-Context Design

The Core Insight

Tells are READ-ONLY inputs to the LLM. The tell text is never sent to the LLM for re-voicing — it is always served as authored base text. But when an NPC's tell state is active, that state flows into the re-voicing prompt for the NPC's observable behaviors and dialogue as a tone injector.

The distinction matters mechanically: the tell communicates NPC internal state to the observant player. If the LLM re-voices the tell, the phenomenon might shift and the mechanical signal corrupts. If the tell influences surrounding content, the player perceives a coherent character — their dialogue and movement feel consistent with their internal state — without the game explicitly labeling that state.

The effect we're producing: An NPC under Guarded tell state should feel guarded. Their base-text tell ("becomes evasive and avoids eye contact") is unchanged. But their re-voiced dialogue ("All in one piece. What do you need?") comes out differently than when they're in neutral state — more clipped, more words chosen, a slight sense of something unsaid. The player who has learned the tell grammar sees the tell AND hears it echoed in the surrounding content. The player who hasn't learned the grammar yet just notices the NPC feels slightly off — which is the right experience.

The Five Tell Context Injectors

One injector per TellCategory. These are tone instructions — they describe HOW to phrase the content, not WHAT to add. They must not name the tell state. They must not introduce new information. They modulate expression.

TellCategory	Tone Injector
Neutral	(no injector — free re-voicing with culture + trait only)
Nervous	"This NPC's words come slightly faster than usual, briefer. They don't elaborate. A phrase drops off before it's finished. Do not say they seem nervous or afraid."
Angry	"This NPC's words are measured and deliberate — not shouting, containing. A word hits harder than the context requires. Do not say they seem angry."
Friendly	"This NPC offers slightly more than asked. A word of genuine warmth lands casually. They don't perform friendliness — it just shows. Do not add compliments or over-warmth."
Guarded	"This NPC chooses each word with a half-second more care than normal. They answer what was asked, no more. There is nothing wrong here. Do not say they seem guarded or evasive."
RoutineDeviation	"This NPC is elsewhere in their mind. They are present but preoccupied — answers are on track but land a beat late, like they're half attending. Do not explain why or name what they're thinking about."

Critical constraints on all tone injectors:

• Do not name the internal state ("nervous", "angry", "hiding", "guarded", "distracted")
• Do not add information not in the base text
• Do not change the content — only the texture of expression
• The resulting output must pass the deniability test: could the player explain this phrasing without knowing the tell was active?

Prompt Assembly with Tell Context

The re-voicing prompt for a behavior or dialogue line in a given tell state assembles as:

[SYSTEM/PREFIX — universal negative injectors]
You are re-voicing NPC dialogue for a game set in the Settled Reach, a gritty working-class
science fiction setting. Never reference: religion, military titles, fantasy elements,
Earth geography, banter/wit unearned by context, or anachronistic technology. Never invent
new facts, locations, or relationships. Output only the re-voiced line.

[CULTURE INJECTOR — per CultureProfile, ~150-250 tokens]
Van Maanen's Star culture: direct, working-class, minimal pleasantries. Vocabulary markers:
void-oaths ("void take it", "blood and void"), clipped greetings ("hey", "all good?"),
no contractions avoided — they use contractions naturally. Register is not formal.
Example of Van Maanen's Star register: [brief paired example demonstrating Van Maanen's Star voice]

[TRAIT MODIFIER — per NPC's PersonalityTraits, ~10 tokens each]
This character is Bold: confident, speaks their mind directly.

[TELL CONTEXT INJECTOR — per active TellCategory, ~30-40 tokens]
This NPC chooses each word with a half-second more care than normal. They answer what
was asked, no more. There is nothing wrong here. Do not say they seem guarded or evasive.

[TASK — base text]
Re-voice in this character's voice: "All in one piece. What do you need?"

Total prompt for behavior with tell context: ~200-350 tokens (well within the 150-token culture + 150-token tell/other budget).

Caching Architecture

The 6-variant per line model (Neutral + 5 TellCategory states) enables runtime determinism:

Cache key: (npc_stable_id, line_id, tell_state, culture_id)
Cache value: voiced_text: String

// At pre-voicing time (generation or background queue):
for each NPC in zone:
    for each behavior/dialogue line:
        for each TellCategory in [Neutral, Nervous, Angry, Friendly, Guarded, RoutineDeviation]:
            voiced = llm.revoice(base_text, culture_injector, trait_modifier, tell_injector)
            cache.insert((npc_id, line_id, tell_state, culture_id), voiced)

// At runtime (zero inference):
fn get_voiced_line(npc_id, line_id, current_tell_state, culture_id) -> String {
    cache.get((npc_id, line_id, current_tell_state, culture_id))
        .unwrap_or_else(|| base_text(line_id))  // graceful fallback
}

Cost: 6× inference per line at generation time. At runtime: pure cache lookups, zero inference triggered by tell-state changes.

Why this is the right model: D-010 principle 4 (deterministic simulation with input events). The voiced content is determined at generation time by (seed + culture + NPC traits). Tell state is a runtime variable that selects from pre-computed variants. This keeps the pre-voicing pipeline in the background where it belongs and the gameplay loop fast and deterministic.

Fallback order:

1. Pre-voiced variant for current tell state → serve it
2. Pre-voiced neutral variant → serve it (content matches, tone is neutral — acceptable degradation)
3. Base text → always present, always correct

This means a player will almost never see raw base text once the pre-voicing pipeline has completed for a zone. The neutral variant is a sufficient fallback that sounds intentional.

What the Player Experiences

The player who has learned the tell grammar:

1. Sees the base-text tell ("becomes evasive and avoids eye contact") — mechanical signal, unchanged
2. Hears the Guarded-influenced dialogue — coherent with the tell, amplifying the read
3. Pattern: "this NPC's words are as guarded as their eyes"

The player who hasn't yet learned the tell grammar:

1. Sees the base-text tell — may not yet know what it means
2. Hears the Guarded-influenced dialogue — senses something slightly off
3. The monologue system (Q-052) may flag it: "The Character's Eye" moment
4. Next time they encounter this pattern on a different NPC, they recognize it

Both experiences are correct. The tell-context architecture serves both simultaneously.

What the LLM Must Never Do with Tells

These are absolute constraints, tested in Spike 1:

1. Never name the state: "seems nervous" / "appears guarded" / "is hiding something" — explicit tell labeling destroys the signal's mechanical value
2. Never add knowledge: "carefully, as if worried about the patrol" — the LLM cannot introduce narrative content not in the base text or injectors
3. Never replace phenomenon with inference: "says nothing" instead of "answers briefly" — the base-text content must survive re-voicing
4. Never overplay: exaggerating tone injectors into theatrical performance destroys the deniability that makes tells work

---

3. Spike 1 Success Criteria: "Does This Even Play?"

What Spike 1 Is

Spike 1 is: build the Rust inference wrapper, load Gemma 2B and Phi-3, then Jeroen + Mellanie + Paula manually craft prompts and run them by hand. No game integration. The goal is to answer "does this even play?" before committing to Spike 2 integration.

From my domain (systems design), "does this even play?" means five specific things:

---

Criterion 1: Information Preservation

What it tests: Does re-voiced content preserve the mechanical information the player needs?

Observable behaviors and dialogue carry game-relevant information: what an NPC is doing, what they know, what they want. Re-voicing must modulate expression without removing or distorting content.

Protocol:

• Select 10 behaviors and 10 dialogue lines covering a range of mechanical content (actions, facts, offers, refusals)
• Re-voice each through Gemma 2B, culture + trait injectors only (Neutral state)
• Give a reader unfamiliar with the base texts ONLY the re-voiced versions
• Ask: "what is this NPC doing / saying?" for each
• Compare reader's summary to what the base text communicates

Success bar: 9/10 for both behaviors and dialogue — the reader's summary matches the mechanical content of the base text. The one failure is examined for pattern (is it a prompt issue, a model issue, a base-text issue?).

---

Criterion 2: Tell-Context Tone Without Naming

What it tests: Does the tell context injector modulate tone without the LLM naming or inferring the tell state?

This is the core mechanical test for D-138's tell-as-context design.

Protocol:

• Take 5 behaviors and 5 dialogue lines, re-voice each in Neutral and Guarded states
• Blind review: show reviewer ONLY the re-voiced Guarded outputs (no Neutral comparison, no context about tells)
• Ask two questions:
  • "Does this NPC feel like they're being careful about something?" (yes/no)
  • "Does this text explicitly say or imply what they're being careful about?" (yes/no)
• Also scan all 10 outputs for forbidden phrases: "nervous", "guarded", "hiding", "evasive", "worried", or any inference about NPC internal state

Success bar:

• ≥4/5 behaviors and ≥4/5 dialogue lines: reviewers sense the undertone
• 0/10 outputs: explicit state naming or inference. This is a HARD requirement — any explicit naming fails the test regardless of tone success rate
• Repeat for Nervous and RoutineDeviation (the two most distinct tone profiles)

---

Criterion 3: Cultural Grammar Survival

What it tests: Does culture remain legible after re-voicing? Culture is a tell — hearing Van Maanen's Star speech should tell the player something about where this NPC is from.

Protocol:

• Re-voice 10 Van Maanen's Star base texts (mix of behaviors and dialogue)
• Check for oath vocabulary: do void-oaths appear in outputs where they're appropriate?
• Check register: does the output read working-class, direct, minimal pleasantries?
• Compare 5 Van Maanen's Star re-voiced outputs against 5 "generic sci-fi NPC" sentences that could come from any game
• Blind reviewer: can they identify which 5 are Van Maanen's Star-flavored vs. generic?

Success bar:

• Void-oath vocabulary appears in ≥3/5 appropriate outputs (where an exclamation is called for)
• Blind reviewer correctly identifies Van Maanen's Star-vs-generic at ≥8/10 (they shouldn't be guessing)
• Zero outputs that sound like fantasy, military, Earth-based, or comedic-banter registers

---

Criterion 4: No False Information

What it tests: Does the LLM stay within the information the base text and injectors provide?

This is a D-010 constraint (information boundaries). The LLM cannot introduce facts the NPC doesn't know, locations that don't exist, relationships that aren't authored.

Protocol:

• Review ALL outputs from Criteria 1-3 for false information
• Flag anything the LLM added that isn't in: (a) base text, (b) culture injector, (c) trait modifier, (d) tell context injector

Success bar: 0 false information introductions. This is a hard requirement. Any false information in any output is a spike finding that must be addressed before Spike 2 integration, regardless of how good the output otherwise is.

---

Criterion 5: The "Does This Feel Like a Place?" Test

What it tests: Does the re-voiced output produce the experience of encountering a real inhabitant of the Settled Reach, not a generated NPC?

This is the qualitative gut check that can't be fully quantified. It's the question the whole workshop has been building toward.

Protocol:

• After running all manual prompts, Jeroen, Mellanie, and Paula each answer independently:
  1. "Pick one output from today that felt like a real person. Read it aloud. Does it sound like the Settled Reach?"
  2. "Pick one output that fell flat. What made it fall flat? Model failure, prompt failure, or base-text failure?"
  3. "If you read this NPC's dialogue in a playtest and didn't know it was LLM-generated, would you think it was authored?"

Success bar: This is not a pass/fail criterion — it's a diagnostic that informs Spike 2. But:

• If all three reviewers struggle to find an output that "felt like a real person," the spike has failed qualitatively regardless of quantitative scores
• The "fell flat" analysis is as important as the passes — it tells us where to invest prompt engineering effort

---

Summary of Spike 1 Go/No-Go

Criterion	Hard requirement?	Target
Information preservation (behaviors)	No	≥9/10
Information preservation (dialogue)	No	≥9/10
Tell-context tone (undertone sensed)	No	≥8/10
Tell-context: zero explicit naming	YES	0/20
Cultural grammar (Van Maanen's Star legible)	No	≥8/10 correct identifications
No false information	YES	0 instances
Qualitative "real person"	No	At least 1 convincing output per reviewer

Go decision rule: Both hard requirements met + at least 4/5 soft criteria pass = proceed to Spike 2 with the winning model. Hard requirement failure = fix the prompt architecture before Spike 2, regardless of soft criteria scores.

Model selection: Run all criteria against both Gemma 2B and Phi-3. The winning model is the one that passes both hard requirements and scores higher across soft criteria. If both fail hard requirements, that's a prompt architecture problem — fix the prompts and re-run before concluding the models are insufficient.

---

Appendix: Tell Category → Tone Injector Authoring Notes

For Mellanie and Paula's reference during Spike 1 prompt crafting:

The tone injectors above are v1 drafts. The spike should test them and refine. Key authoring principles:

1. Describe the EXPRESSION, not the internal state: "words come slightly faster" not "seems nervous"
2. Anchor the instruction to observable behavior: what the reader hears/sees, not what the NPC feels
3. Include a negative constraint: every injector ends with "Do not say they [explicit state label]"
4. Keep it short: 30-40 tokens. The model doesn't need an essay. Precision > length.
5. Test the deniability: after seeing an output, could you explain the phrasing without knowing the injector was active? If yes, the injector is working correctly.