settled-reach/docs/workshops/llm-voice-pipeline/gestalt-round2.md

title, description, type, status, workshop, agent, round, created

title	description	type	status	workshop	agent	round	created
Gestalt Round 2: Systems Design Evaluation	Convergent evaluation of LLM voice pipeline proposals from systems design perspective	workshop	archived	llm-voice-pipeline	gestalt	2	2026-03-07

LLM Voice Pipeline Workshop — Round 2: Systems Design Evaluation

Author: Gestalt Round: 2 — Convergent Evaluation Date: 2026-03-07

---

Resolving Q-R1-01: The Tell Literacy Model

Question: Is the player's tell literacy model cross-NPC grammar (players learn "avoidance = hiding something" across all NPCs) or fresh-each-time (each NPC's tells are unique)?

Answer: Cross-NPC grammar — specifically, phenomenon-class grammar.

Here's the argument from the evidence.

Evidence in the codebase and design decisions

gen_tells() in generate.rs generates tells from trait+trigger combinations. The behavior strings are constants — same trait, same trigger, always the same string:

Deceptive + StressAboveThreshold → "affects exaggerated calm"  (always)
Cautious + StressAboveThreshold  → "checks surroundings repeatedly" (always)
Major secret + stress            → "becomes evasive and avoids eye contact" (always)

The generator produces at most ~12 distinct tell behavior strings in the entire game. This is not an accident — it's a grammar. The design intent is that these strings represent recognizable classes of observable behavior that the player can learn to associate with internal states.

Q-052 makes the learning model explicit: "Hours 1-5: full hints. Hours 15+: player reads the world by behavioral tells alone. Not harder combat — a quieter, more trusting world." The game has a teacher that backs off as the player develops tell literacy. That only works if there IS a tell literacy to develop — a learnable grammar, not random case-by-case observation.

D-039 wow moment #2 ("The Character's Eye") is the tell literacy game stated directly: "My character is smarter than me." The character's internal monologue flags something the player missed. This moment only works if (a) there was a tell in the observable space and (b) the player hadn't yet learned to see it. The game is explicitly modeling a skill gap the player closes over time.

The grammar works at the phenomenon class level, not the phrasing level.

This is the crucial nuance. The player doesn't learn "when I see the exact string 'affects exaggerated calm' that means Deceptive+stress." They learn "when I see suppression behavior — exaggerated stillness, forced normalcy — that NPC is hiding something consciously." The phenomenon class (suppression, avoidance, surveillance, fidgeting) is the unit of pattern recognition.

Implications for Proposal B

This is directly relevant to whether Proposal B's constrained re-voicing is safe.

Constrained re-voicing is safe IF the constraint preserves phenomenon class membership.

The failure mode is phenomenon class migration:

• "affects exaggerated calm" → "seems composed and unhurried" — still suppression class, SAFE
• "affects exaggerated calm" → "looks away when you approach" — avoidance class instead of suppression, BROKEN
• "affects exaggerated calm" → "moves with unusual speed" — hurried class, completely broken signal

The semantic_core constraint must be written at the phenomenon-class level, not just as an abstract label. This matters for implementation:

Too abstract (unreliable)	Precise (reliable for 2B model)
"PRESERVE: suppression_behavior"	"PRESERVE: forced calm. The NPC appears deliberately composed and unhurried. Must not show avoidance, fidgeting, or hurry."
"PRESERVE: avoidance_behavior"	"PRESERVE: eye contact avoidance. The NPC avoids holding your gaze. Must not show aggression or forced calm."
"PRESERVE: surveillance_behavior"	"PRESERVE: environmental scanning. The NPC checks their surroundings and aware of exits. Must not show avoidance or stillness."

The abstract label is a human-readable tag. The precise constraint is what actually guides a 2B model reliably. If we implement Proposal B, semantic_core should store the precise constraint language, not just the category name.

For 5-15 word tells, constrained re-voicing is actually EASIER for the 2B model than free re-voicing of ambient behaviors. The input is short, the output should be short, the constraint is explicit. This is the regime where small instruction-following models perform most reliably.

Conclusion on Q-R1-01: Cross-NPC grammar at the phenomenon-class level. Proposal B's constrained re-voicing is safe with precise semantic_core language. Proposal A (passthrough) is also safe — it's the conservative floor, not the optimum.

---

Resolving Q-R1-03: The Tell Data Model Separation

Question: Is separating tell_behaviors from observable_behaviors in the data model implementable and correct?

Answer: Yes, and the pipeline architecture makes it natural. But the implementation requires understanding where tells actually live.

Where tells currently live

The codebase has a structural split I need to be clear about, because it affects how "implementable" this is:

In the blueprint pipeline (npc/blueprint.rs, generator_spike.rs):

pub struct NpcBlueprint {
    pub observable_behaviors: Vec<String>,  // from RoleSpec.typical_behaviors
    // No tell_behaviors field — tells are not currently in the blueprint
}

In the ECS pipeline (npc/generate.rs, npc/mod.rs):

// TellSystem is a separate ECS component — generated from traits + secret at spawn time
fn gen_tells(traits: &PersonalityTraits, secret: &Secret) -> TellSystem { ... }

Tells are not authored — they're generated by gen_tells() from trait + secret combinations. They don't appear in RON files. They're computed at entity spawn time.

For the re-voicing pipeline to work cleanly, tells need to be accessible before they hit the ObserverSnapshot. The correct implementation:

Proposed data model change

Step 1: Add semantic_core to Tell (in npc/mod.rs):

pub struct Tell {
    pub trigger: TellTrigger,
    pub behavior: String,           // base text / passthrough / re-voiced
    pub semantic_core: String,      // re-voicing constraint (precise phenomenon description)
}

The semantic_core for each tell is authored alongside the behavior strings in gen_tells(). There are ~12 distinct tell types — this is a one-time authoring task of 12 constraint sentences.

Step 2: Add tell_behaviors to NpcBlueprint (in npc/blueprint.rs):

pub struct NpcBlueprint {
    pub observable_behaviors: Vec<String>,  // ambient — route to free re-voicing
    pub tell_behaviors: Vec<TellBehavior>,  // mechanical — route to constrained/passthrough
}

pub struct TellBehavior {
    pub behavior: String,        // base text
    pub semantic_core: String,   // re-voicing constraint
    pub trigger_type: String,    // "always" or "stress" (for documentation, not gameplay use)
}

Step 3: Produce tells at blueprint time via a standalone function that mirrors gen_tells() without ECS:

// New function in generator pipeline (not requiring World)
fn gen_tell_behaviors_for_blueprint(
    traits: &[PersonalityTrait],
    secret_severity: SecretSeverity,
) -> Vec<TellBehavior> { ... }

This mirrors the existing pattern in generator_spike.rs, which already reimplements ECS-level logic as standalone functions for blueprint generation.

Why this separation is correct

1. The pipeline routing is by field, not content inference.

Paula and Mellanie both flagged this requirement. The re-voicing pipeline must route based on the structural location of the string (which field it came from), not by analyzing whether the string looks like a mechanical signal. Content analysis is fragile; field routing is deterministic.

With observable_behaviors and tell_behaviors as separate fields, the routing rule is trivial:

NpcBlueprint.observable_behaviors → free re-voicing queue
NpcBlueprint.tell_behaviors → locked/constrained queue

No content parsing. No heuristics. The data model enforces the distinction.

2. It respects D-010 (information boundaries) structurally.

D-010 principle 2: "every piece of game state is tagged with who knows it." Tell behaviors are a specific class of observable state — they're what the player can observe about the NPC's internal state. Tagging them as a distinct field makes the information class explicit in the data model, not implicit in editorial convention.

3. The ECS TellSystem is unaffected.

The existing TellSystem ECS component remains as the runtime authoritative source. The blueprint's tell_behaviors is the pre-generation staging ground. At entity spawn time, the ECS pipeline can:

• Load tell behaviors from the pre-voiced cache (if available)
• Fall back to the gen_tells() generated base text (if not)
• The TellSystem struct may optionally store both the base text and the voiced text for graceful fallback

This is the same pattern as the broader base-text/voiced-text architecture. Tells get the same fallback model as everything else.

4. The authoring burden is minimal.

gen_tells() currently has ~12 tell types, each with a one-line behavior string. Adding semantic_core is 12 additional sentences written once by Gestalt or Tyre at implementation time. The copy team doesn't author tells (they're generated algorithmically) — so this doesn't create copy team overhead.

Conclusion on Q-R1-03: Implementable and correct. The separation exists already at the ECS level (TellSystem as a distinct component). Adding it to the blueprint and routing by field (not content) is the right architectural expression of that separation. The semantic_core field on Tell is the mechanism that makes Proposal B work.

---

Proposal Evaluation

Proposal A: Conservative — Behaviors Only, Tells Locked

Recommendation: Acceptable, not preferred.

Does it work mechanically? Yes. Base text passthrough for tells is absolutely safe. The phenomenon-class grammar works in base text form — the current tell strings ("becomes evasive and avoids eye contact") are clear enough that cross-NPC pattern recognition is possible.

What it misses: Cultural texture on tells. A Van Maanen's Star person avoiding eye contact should look different from whatever a high-register culture's evasion looks like. D-121 (voice is culture-driven) applies to tells too — a Van Maanen's Star NPC's deception tell should read like a Van Maanen's Star person hiding something, not like a generic sci-fi NPC hiding something. Passthrough surrenders this.

Ozzie's "contrast is a feature" argument deserves consideration. Base text standing out against voiced ambient behaviors might make tells MORE immediately recognizable — they read differently precisely because they weren't culture-voiced. This is a legitimate UX argument, not just a consolation prize for the conservative choice. I don't know if it's empirically correct; it's a testable hypothesis.

Can I live with Proposal A? Yes.

Minimum change to make it acceptable: None needed — it's already acceptable. If we choose A, I'd request we commit to revisiting tell culture-voicing as a v0.3 task once we've validated the base system. The phenomenon-class grammar holds in base text; we're just leaving cultural texture on the table.

---

Proposal B: Two-Track — Behaviors + Constrained Tell Re-voicing

Recommendation: Preferred.

Does it work mechanically? Yes, with the semantic_core precision requirement from Q-R1-01 above. The constrained re-voicing task is well-suited to a 2B model: short input, explicit constraint, short output. This is easier than free re-voicing of longer ambient behaviors.

The spike test for this proposal must answer: Does Gemma 2B reliably stay within the phenomenon class when given precise constraint language? This is a concrete, measurable success criterion: author 12 tell re-voicing prompts (one per tell type), run 20 completions each, score by phenomenon-class preservation. If ≥18/20 stay in the correct class for each tell type, the approach is viable. If not, fall back to passthrough (Proposal A) for tells.

Interaction with the data model (Q-R1-03): Proposal B requires tell_behaviors as a first-class field AND semantic_core on each tell. The data model change described in Q-R1-03 is a prerequisite for B, not optional.

The one complexity: The semantic_core language must be authored carefully. 12 sentences, but they're precision-critical. I'd recommend reviewing them against actual 2B model outputs during the spike before committing them as the canonical constraint text.

Can I live with Proposal B? Yes — it's my recommendation.

---

Proposal C: Full Pipeline — Behaviors + Dialogue, Tells Locked

Recommendation: Conditionally acceptable. Depends on Tyre and Troblum's Round 2 assessment.

Does it work mechanically? Probably, but with meaningful scope risk. Dialogue has structural protection (access tier, trust tier tags — Paula's correct observation) that ambient behaviors don't. The dialogue system's existing tag model is actually better-suited to constrained re-voicing than ambient behaviors are.

But: The scope increase is real. Two content types, two prompt templates, two validation passes, two quality bars. The spike becomes more complex. If dialogue quality at 2B is insufficient, we're forced into a larger model that may violate the hardware budget (C-6 established Q4 as hard requirement on 8GB shared RAM).

From a systems standpoint: The information-safety question for dialogue is different from tells. Dialogue can leak game state in ways that tells don't — an NPC who "shouldn't know" something could be prompted to say it via a poorly constrained re-voicing prompt. The access tier and trust tier tags mitigate this structurally, but the LLM can still hallucinate knowledge beyond the tag constraints. This is a lore contamination risk class that doesn't exist for observable behaviors.

My preference on sequencing: A or B first (validate the simpler problem), C after the spike proves the pipeline. Proposal C is the right end state. Getting there via A→C or B→C is safer than attempting C in the first spike.

Can I live with Proposal C? Yes, if Tyre confirms 2B quality is sufficient for dialogue AND Troblum confirms the RAM budget holds for the additional prompt context.

Minimum change to make C acceptable: Define a failure mode and fallback for dialogue quality. If the 2B model doesn't meet bar for dialogue, the fallback shouldn't be "abandon the whole pipeline" — it should be "scope to behaviors only (falling back to Proposal A or B)." This needs to be built into the spike design.

---

Resolution Matrix

Question	Answer
Which proposal do you recommend?	B (Two-Track)
Blockers in Proposal B?	tell_behaviors field + semantic_core on Tell required before spike design. Scope is well-defined and achievable.
Can you live with Proposal A?	Yes
Can you live with Proposal C?	Yes, conditionally (see above)
Minimum change to Proposal A?	None needed for acceptability. Commit to tell culture-voicing as future work.
Minimum change to Proposal C?	Define explicit fallback to behaviors-only if dialogue quality fails the spike.

---

Additional Systems Notes for Round 2

On the T-3 composable primitives question

Round 1 notes document T-3 as "artifact or reject?" for composable primitives. My Round 2 position: not needed in the architecture.

The two-track re-voicing system (base text + injector clauses) subsumes what composable primitives were trying to achieve. Composable primitives were an attempt to make the authoring generative without LLM help — assemble behaviors from components at runtime. The LLM re-voicing approach does the same job more cleanly: authors write complete, evocative base lines (which they already have from the Sprint 25 spike), and the LLM applies cultural voice. The composition step IS the LLM.

The authoring benefit of composable primitives (structured thinking about role + culture + context) is real but can be captured in the injector clause design without building a composition engine. Injector clause authoring guides authors toward the same structured thinking without requiring a formal grammar system.

Composable primitives: closed as a rendering layer, not needed as schema.

On D-123 amendment language

All three proposals amend D-123 ("authoring tool, not runtime system"). The correct framing:

D-123 is amended: "NPC content (dialogue pools, voice, vocabulary) is generated using generative AI with culture vectors as primary constraints. The AI pipeline functions as both an authoring tool for batch content generation AND a background runtime system for on-demand pre-voicing. The runtime component generates content in the background before the player arrives, caches the result, and uses base text as graceful fallback. Culture profiles remain the primary authoring deliverable."

This preserves the spirit of D-123 (culture vectors as primary constraint, culture profiles as the deliverable) while being honest that the runtime system is in-game. D-124 is superseded.

On the spike success criteria

For my domain (systems design), the spike must answer:

1. Does behavior re-voicing preserve cultural identity? (Run Van Maanen's Star-voiced output past Miri's criteria)
2. For Proposal B: does constrained tell re-voicing preserve phenomenon class? (12 tell types × 20 completions, score by class preservation rate — target ≥90%)
3. Does the tell literacy grammar hold after re-voicing? (Can a naive reader identify "avoidance" vs. "suppression" vs. "surveillance" from the re-voiced tells?)

Criterion 3 is the real test. The spike should include a blind evaluation: show 10 re-voiced tell strings to someone who hasn't read the original base text, ask them to categorize the behavior. If they can reliably assign to the correct phenomenon class, the grammar survived re-voicing.

---

Summary

Q-R1-01 resolved: Cross-NPC grammar at the phenomenon-class level. Proposal B is safe with precise semantic_core language that specifies the phenomenon concretely ("the NPC avoids holding eye contact") not just abstractly ("avoidance_behavior").

Q-R1-03 resolved: Correct and implementable. Tells are already separated from ambient behaviors at the ECS level. Adding tell_behaviors: Vec<TellBehavior> to NpcBlueprint and semantic_core: String to Tell is the natural expression of that existing separation. Field routing (not content analysis) is the correct implementation model.

Proposal recommendation: B, with the understanding that A is acceptable and C is the right long-term destination.