settled-reach/docs/architecture/proposed-llm-voice.md

title, description, type, status, ticket, decision_refs, author, created, updated

title	description	type	status	ticket	decision_refs	author	created	updated
Proposed Architecture: LLM-Powered Voice Synthesis	Re-voicing architecture for dynamic NPC dialogue using a local LLM as a stylistic enhancement layer — injector system, priority queue, and optional AI-enhanced toggle	architecture	superseded		D-138	Gemini	2026-03-07	2026-03-07

Proposed Architecture: LLM-Powered Voice Synthesis

Status: Superseded by D-138 and LLM Voice Pipeline Workshop Author: Gemini (synthesizing a design sparring session with Jeroen) Date: 2026-03-07

---

1. Executive Summary

This document proposes a "Re-voicing" architecture for dynamic NPC dialogue. This system uses a small, locally-run LLM as a stylistic enhancement layer, akin to a localization engine, that "translates" functional, base dialogue into rich, in-character performances.

This design elegantly solves the combinatorial complexity of traditional dialogue systems while retaining full authorial control over gameplay-critical information. Furthermore, it is architected to be a player-facing, optional feature ("AI-Enhanced Dialogue"), allowing the game to run on a wide range of hardware by providing a lightweight, non-LLM fallback that is a core part of the pipeline itself.

The implementation strategy involves on-demand, background pre-generation of dialogue managed by a prioritized queue, ensuring a smooth player experience with no real-time latency.

2. Problem Statement

A rich, reactive world requires NPCs whose dialogue reflects their personality, culture, mood, and the current game state. Authoring this manually via a traditional template tree leads to a combinatorial explosion of content that is brittle, difficult to maintain, and often fails to capture the desired nuance, feeling robotic despite its complexity.

3. Proposed Architecture: The "Re-voicing" Model

Our proposed solution is to treat dynamic dialogue not as a generation task, but as a stylistic localization task.

Analogy: Dialogue as an i18n System

The core of this design is to think of character voice as a "language." Our simple, non-LLM template system provides the default "language" (en-US)—a clear, functional line of text that serves the gameplay. The LLM's job is to "translate" this line into a specific character's "language" (en-VMS-RUTHLESS).

This immediately enables a powerful player-facing feature:

The "AI-Enhanced Dialogue" Toggle

This architecture allows for a setting in the game menu:

• OFF: The game uses the fast, lightweight, default "semantic lines." The experience is 100% complete and functional on any hardware.
• ON: The game uses the LLM to "translate" the dialogue into the richer, in-character "voices," providing a premium experience for players with capable hardware.

This de-risks all performance concerns and makes the innovative dialogue system an optional enhancement rather than a mandatory hardware requirement.

The Two-Step Pipeline

1. Step 1: Generate the Semantic Core: The existing simple template system generates a functional, gameplay-serving "semantic line." This is our i18n default string. It guarantees that gameplay-critical information is always present.

   Semantic Line: "You need a keycard for that door."

2. Step 2: Perform the "Re-voicing": The LLM receives this semantic line with a prompt to rephrase it in the voice of a specific character persona.

   Final Stylized Line: "I suspect you'll find that door won't open without the proper authorization."

4. Core Component: The "Injector" System

The character persona is constructed for the LLM using a manageable library of "Injector Clauses"—dozens at most. These clauses are assembled on-the-fly to guide the re-voicing task.

• Personal Injectors (~10-20 clauses): Mapped to personality traits, defining the manner of speech.

  • Example [Bold]: "Your delivery is direct and confident."

• Cultural Injectors (~5-10 clauses): Mapped to origin, defining the cultural "flavor" or dialect.

  • Example [Van Maanen's Star Culture]: "Your speech is formal and avoids contractions."

5. The Composition Engine: Priority & Blending

To prevent conflicting instructions (e.g., a [Social] but [Angry] character), the prompt assembler will act as a small rule engine, composing injectors based on a priority hierarchy:

1. Mood as an Override: A strong, temporary emotional state (e.g., [Angry]) takes highest priority, suppressing conflicting personality traits.
2. Personality as Flavor: The one or two most relevant personality traits for the situation are chosen.
3. Culture as Baseline: The cultural injector is almost always applied, establishing the foundational dialect.

6. Implementation Strategy: The Dialogue Generation Queue

To eliminate real-time latency and manage performance, all LLM generation will happen in the background, managed by a prioritized queue.

1. On-Demand Trigger: When the player takes an action that signals intent to enter a new area (e.g., accepts a mission), the system populates a queue with all dialogue generation tasks for that area.
2. Prioritized Queue: Tasks are prioritized to ensure the best possible experience upon arrival.
   • P0 (Critical): Plot-essential NPCs.
   • P1 (High): Important secondary characters.
   • P2 (Standard): Background flavor NPCs (the "enhancement" tier).
3. Background Worker: A low-priority CPU thread works through this queue. On high-end machines, the entire area may be pre-generated quickly. On low-end machines, only critical dialogue may be ready.
4. Pre-warmed Cache: To guarantee a high-quality initial experience, the game will ship with a pre-generated cache of all dialogue for the first few hours of gameplay.

7. Next Steps: The A/B Prompt Spike

Before implementation, a spike is required to validate our choice of model and the creative viability of the injector system.

Test Candidates

Given the project's constraints (no Meta/Chinese models, Mistral 7B is too large), the two leading candidates are:

• Candidate A (The Performance Play): Google Gemma 2B
• Candidate B (The Balanced Play): Microsoft Phi-3-mini

Spike Methodology

The spike will be a standalone script to test the core trade-off between these models.

1. Author Assets: Create 3-5 structured "payloads" (semantic line + character context) for different scenarios, including at least one with conflicting injectors.
2. A/B Test: Run the same set of composed prompts through both Gemma 2B and Phi-3-mini.
3. Evaluate: Compare the outputs on two axes:
   • Creative Quality: How reliably does each model handle the stylistic instructions and conflicting constraints?
   • Performance Cost: What is the measured CPU-only inference latency and RAM usage for each model?

The outcome will determine which model provides the best balance of quality and performance for our needs, and will validate the "complexity ceiling" of our chosen technology.

8. Long-Term Risks

• Localization: While this architecture is more localization-friendly than pure generation, a full strategy for translating prompts and handling different linguistic nuances will be a significant future task.
• Performance Tuning: The background worker's impact on game performance, especially on CPU-bound laptops, will require careful tuning to prevent stuttering or system slowdown.