settled-reach/docs/workshops/llm-voice-pipeline/tyre-round3.md

---
title: "Tyre Round 3: Implementation Specification"
description: "Spike 1 implementation spec, model test plan, spike 2 outline, and hardware detection design"
type: workshop
status: archived
workshop: llm-voice-pipeline
agent: "tyre"
round: 3
created: 2026-03-07
---

# Tyre Round 3: Implementation Specification

**Domain:** Technical architecture
**Round:** 3 — Decision & Implementation Plan
**Deliverables:** Spike 1 implementation spec, model test plan, Spike 2 outline, hardware detection design

---

## 1. Spike 1 Implementation Spec: `sr-voice` CLI Tool

### Purpose

A standalone Rust CLI that loads a GGUF model, accepts prompts, and returns generated text. No game integration, no queue, no cache. This is the plumbing that Jeroen, Mellanie, and Paula will feed manually-crafted prompts through to answer: "does this even play?"

### Crate structure

```
server/
  sr-voice/
    Cargo.toml
    src/
      main.rs          # CLI entry point
      inference.rs     # Model loading and generation wrapper
      prompt.rs        # Prompt file parsing and construction
```

`sr-voice` is a separate crate in the server workspace, not compiled into the game binary. It depends on `llama-cpp-rs` (or `llama-cpp-2` — see build notes) and produces a standalone binary: `sr-voice`.

### Cargo.toml dependencies

```toml
[package]
name = "sr-voice"
version = "0.1.0"
edition = "2021"

[dependencies]
llama-cpp-2 = { version = "0.1", features = ["metal", "vulkan"] }
# Note: "metal" and "vulkan" are optional features, compile-time gated.
# CPU-only is the default and always available.
clap = { version = "4", features = ["derive"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"

[features]
default = []
gpu-metal = ["llama-cpp-2/metal"]
gpu-vulkan = ["llama-cpp-2/vulkan"]
```

**Build note on `llama-cpp-rs` vs `llama-cpp-2`:** Both wrap the same C library. `llama-cpp-2` is the more actively maintained fork as of early 2026 and has cleaner safe Rust wrappers. Evaluate both at spike start; pick whichever compiles cleanly on Linux + macOS + Windows without manual C++ toolchain intervention. Pin the llama.cpp commit hash in Cargo.toml to prevent upstream API breaks.

**Build dependency:** Requires a C/C++ compiler (gcc/clang/MSVC). CMake is pulled in by the llama.cpp build system. This is a compile-time dependency, not a runtime dependency — the final binary is self-contained.

### CLI interface

```
sr-voice --model <path-to-gguf> [OPTIONS] [PROMPT_FILE]

Options:
  --model <path>        Path to GGUF model file (required)
  --threads <n>         CPU threads for inference (default: physical_cores - 1)
  --ctx-size <n>        Context window size in tokens (default: 512)
  --max-tokens <n>      Maximum output tokens (default: 64)
  --temperature <f>     Sampling temperature (default: 0.7)
  --top-p <f>           Top-p sampling (default: 0.9)
  --seed <n>            RNG seed for sampling (default: random)
  --json                Output as JSON: {"input": "...", "output": "...", "tokens_per_sec": N}
  --batch               Process multiple prompts from a JSONL file (one per line)
  --benchmark           Run 5 inference calls and report avg tokens/sec

PROMPT_FILE:
  Read prompt from file (plain text). If omitted, reads from stdin.
```

### Core function signatures

```rust
// inference.rs

/// Configuration for the inference engine.
pub struct InferenceConfig {
    pub model_path: PathBuf,
    pub n_threads: u32,
    pub ctx_size: u32,
    pub seed: Option<u32>,
}

/// A loaded model ready for inference.
pub struct InferenceEngine {
    // Wraps llama_model + llama_context from llama-cpp-2.
    // Model is loaded once; context is reused across calls.
    model: LlamaModel,
    ctx: LlamaContext,
}

impl InferenceEngine {
    /// Load a GGUF model from disk. Returns an error if the model
    /// doesn't fit in available RAM or the file is invalid.
    ///
    /// Typical load time: 2-5 seconds for a 2B Q4 model from SSD.
    pub fn load(config: &InferenceConfig) -> Result<Self, VoiceError>;

    /// Run inference on a prompt string. Returns the generated text.
    ///
    /// `max_tokens`: maximum output tokens (stops early on EOS).
    /// `temperature`: sampling temperature (0.0 = greedy, 1.0 = creative).
    /// `top_p`: nucleus sampling threshold.
    pub fn generate(
        &mut self,
        prompt: &str,
        max_tokens: u32,
        temperature: f32,
        top_p: f32,
    ) -> Result<GenerationResult, VoiceError>;
}

/// Result of a single inference call.
pub struct GenerationResult {
    /// Generated text (stripped of prompt echo).
    pub text: String,
    /// Number of tokens generated.
    pub tokens_generated: u32,
    /// Wall-clock time for generation (excludes prompt processing).
    pub generation_time_ms: u64,
    /// Tokens per second (generation phase only).
    pub tokens_per_sec: f32,
    /// Wall-clock time for prompt processing (prefill).
    pub prefill_time_ms: u64,
}

pub enum VoiceError {
    ModelLoadFailed(String),
    InferenceFailed(String),
    OutOfMemory,
    InvalidModel(String),
}
```

```rust
// prompt.rs

/// A structured prompt payload for the spike test matrix.
/// Parsed from JSON files that Mellanie/Paula/Jeroen prepare.
#[derive(Debug, Deserialize)]
pub struct PromptPayload {
    /// Unique ID for tracking results.
    pub id: String,
    /// Content type being re-voiced.
    pub content_type: ContentType,
    /// The fully assembled prompt string (system + injectors + base text).
    pub prompt: String,
    /// The original base text (for output comparison).
    pub base_text: String,
    /// Expected semantic core (for tell payloads — optional).
    pub semantic_core: Option<String>,
}

#[derive(Debug, Deserialize)]
pub enum ContentType {
    Behavior,
    Tell,
    Dialogue,
}
```

### Batch mode for the test matrix

The `--batch` flag processes a JSONL file where each line is a `PromptPayload` JSON object. Output is JSONL with the original payload + generated text + timing:

```jsonl
{"id":"rural-farmer-1","content_type":"Behavior","base_text":"tends crops in the field","output":"works the irrigation channels before morning rotation","tokens_per_sec":8.2,"prefill_ms":340,"generation_ms":3650}
{"id":"nervous-tell-1","content_type":"Tell","base_text":"shifts weight and checks the time without reason","output":"shifts from foot to foot, void-ward glances at the clock","tokens_per_sec":7.9,"prefill_ms":380,"generation_ms":3800}
```

This enables Mellanie and Paula to prepare prompt files, run them through both models, and compare output side by side. The JSON output feeds directly into a comparison spreadsheet or diff tool.

### What Spike 1 does NOT include

- No game integration
- No queue or priority system
- No cache
- No thread pool management
- No prompt construction logic (prompts are hand-crafted by the content team for the spike)
- No save/load of voiced content
- No Godot interaction

---

## 2. Model Test Plan

### Candidates

| Model | Parameters | Q4_K_M Size | Why it's here |
|-------|-----------|-------------|---------------|
| **Gemma 2 2B** | 2.6B | ~1.5 GB | Primary candidate. Google origin. Good instruction following for size. |
| **Phi-3-mini** | **3.8B** | ~2.2 GB | Fallback candidate. Microsoft origin. Better quality, larger footprint. |

**Addressing the Phi-3 size discrepancy:** The original proposal called both "2B class." This is incorrect. Phi-3-mini is 3.8B parameters — nearly 50% larger. This matters for:
- **RAM:** +700MB at Q4 (+47% over Gemma 2B)
- **Throughput:** ~30% slower decode due to larger weight matrix
- **Install size:** +700MB in the distribution bundle

Phi-3-mini is the quality fallback, not a peer candidate. If Gemma 2B passes the quality bar, Phi-3 is unnecessary. If Gemma 2B fails, Phi-3 tells us whether more parameters solve the problem or the task itself is wrong for small models.

### Test matrix

**Prompt payloads** (prepared by Mellanie/Paula/Jeroen — Tyre provides the structure):

| ID | Content type | Base text | Zone | Culture | Traits | Mood | TellCategory | Notes |
|----|-------------|-----------|------|---------|--------|------|-------------|-------|
| B-01 | Behavior | "tends crops in the field" | rural | van-maanens-star | Bold, Honest | Neutral | — | Simple role action |
| B-02 | Behavior | "checks a manifest against a handheld scanner, lips moving" | industrial | van-maanens-star | Curious, Social | Neutral | — | Detailed role action |
| B-03 | Behavior | "catches {target}'s eye and nods across the room" | industrial | van-maanens-star | Social, Compassionate | Neutral | — | Relationship behavior — named target preservation |
| B-04 | Behavior | "talks past {target} without making eye contact" | industrial | van-maanens-star | Deceptive, Bold | Neutral | — | Negative relationship — social signal preservation |
| B-05 | Behavior | "sits alone in the break room rubbing the back of her neck, datapad face-down on the table" | industrial | van-maanens-star | — | Stressed | — | Long-form atmospheric behavior |
| T-01 | Tell | "shifts weight and checks the time without reason" | any | van-maanens-star | — | — | Nervous | Nervous fidget — phenomenon must survive |
| T-02 | Tell | "affects exaggerated calm" | any | van-maanens-star | Deceptive | — | Guarded | Suppression — constrained re-voicing test |
| T-03 | Tell | "checks surroundings repeatedly" | any | van-maanens-star | Cautious | — | Nervous | Surveillance — must not become avoidance |
| D-01 | Dialogue | "You need a keycard for that door." | industrial | van-maanens-star | Bold | Neutral | — | Simple informational dialogue |
| D-02 | Dialogue | "I haven't seen Kael since second shift. Why?" | industrial | van-maanens-star | Suspicious, Cautious | Guarded | Guarded | Dialogue with active tell context — tell shapes tone |
| D-03 | Dialogue | "The cargo manifest doesn't match what's in bay seven." | industrial | van-maanens-star | Honest, Curious | Alert | — | Information-bearing dialogue — must preserve factual content |

**11 payloads total.** Each run through both models = 22 outputs per prompt template variant.

### Prompt template variants to test

For each payload, test 2-3 prompt template variants to find the optimal instruction format:

**Variant 1 — Instruction-first:**
```
[System instruction]
[Culture injector with closed vocabulary]
[Personality + mood]
[Tell context if applicable]
Rephrase: "[base text]"
```

**Variant 2 — Few-shot:**
```
[System instruction]
[Culture injector]

Examples:
Base: "repairs equipment by hand" → Voiced: "strips the housing down and rebuilds it, no manual needed"
Base: "arranges goods on a portable display" → Voiced: "squares the goods on the fold-out, everything where it should be"

[Personality + mood]
[Tell context if applicable]
Rephrase: "[base text]"
```

**Variant 3 — Negative-constraint-heavy (Miri's recommendation):**
```
[System instruction]
[Culture injector]
DO NOT use: military ranks, sir/ma'am, religious references, quips, banter.
DO NOT reference: religion, sports, nationality, Earth-origin social structures.
Technology terms ONLY: insert, span gate, horizon gate, void, the Reach.
Exclamations ONLY: "void take it", "stars", "blood and void", "cold vacuum", "damn all".

[Personality + mood]
Rephrase: "[base text]"
```

### Evaluation criteria

Each output is scored on 5 axes (1-5 scale, scored by Paula and Mellanie independently):

| Criterion | What it measures | Pass threshold |
|-----------|-----------------|----------------|
| **Register accuracy** | Does it sound like Van Maanen's Star working-class? Not formal, not quippy, not military. | >= 3 |
| **Oath preservation** | If an exclamation appears, is it from the canonical list? No franchise bleed? | >= 4 (hard requirement) |
| **Semantic preservation** | Does the output preserve the action/information from the base text? | >= 4 (hard requirement) |
| **Named target survival** | For B-03/B-04: does the `{target}` name survive in the output? | Pass/Fail |
| **Tell phenomenon class** | For T-01/T-02/T-03: does the tell still express the same TellCategory? | Pass/Fail |

**Spike 1 success criteria:**
- At least one model achieves >= 3 average on register accuracy across all payloads
- Both hard requirements (oath, semantic) pass on >= 9/11 payloads
- Named target survival: pass on both relationship payloads
- Tell phenomenon class: pass on all 3 tell payloads

If Gemma 2B meets these criteria, it's the selected model. If only Phi-3 meets them, we accept the RAM/size tradeoff and document why. If neither meets them, the spike has failed and we fall back to base-text-only (Proposal A without re-voicing, which is the current system with scaling managed by content authoring).

### Benchmark protocol

On Spike 1 hardware (developer machine), record for each model:
- Tokens/sec at Q4_K_M with `--threads` set to physical_cores - 1
- RAM usage during inference (peak RSS)
- Model load time from SSD

On a representative minimum-spec machine (if available — otherwise note the hardware used and extrapolate using Troblum's bandwidth formula):
- Same metrics
- Thermal behavior during 5-minute sustained inference run

---

## 3. Spike 2 Outline: Integration Architecture

Spike 2 wires the validated Spike 1 runner into the game. This outline is ticket-ready, not full implementation design.

### 3.1 Inference thread pool

```
┌──────────────────────────────────────────┐
│ Game Process                              │
│                                           │
│  ┌───────────────┐   ┌────────────────┐  │
│  │ Simulation    │   │ Voice Pipeline │  │
│  │ (server)      │   │ (separate pool)│  │
│  │               │   │                │  │
│  │ World gen ────┼──>│ Work queue     │  │
│  │ NPC spawn     │   │ InferenceEngine│  │
│  │ Tick loop     │   │ Voice cache    │  │
│  └───────────────┘   └────────────────┘  │
│                                           │
│  ┌───────────────┐                        │
│  │ Godot client  │ <── reads cache ──┘    │
│  └───────────────┘                        │
└──────────────────────────────────────────┘
```

**Thread count:** 1 dedicated inference thread. llama.cpp uses its own internal threading (set to `physical_cores - world_gen_threads - 1`). The inference thread owns the `InferenceEngine` instance — no model sharing across threads.

**Priority:** Below-normal OS thread priority. Inference yields to simulation and rendering.

### 3.2 Work queue

```rust
/// A single unit of work for the voice pipeline.
pub struct VoiceWorkItem {
    /// Cache key for storing the result.
    pub cache_key: VoiceCacheKey,
    /// Priority tier (lower number = higher priority).
    pub priority: VoicePriority,
    /// The fully constructed prompt string.
    pub prompt: String,
    /// Maximum output tokens.
    pub max_tokens: u32,
}

pub enum VoicePriority {
    /// P0: Plot-critical NPCs the player is about to interact with.
    Critical = 0,
    /// P1: NPCs in the current zone the player may interact with.
    High = 1,
    /// P2: NPCs in adjacent/anticipated zones.
    Standard = 2,
    /// P3: Ambient NPCs in distant zones (opportunistic).
    Background = 3,
}
```

**Queue implementation:** `crossbeam-channel` bounded channel (capacity: 256). Items sorted by priority. Producer: the world generation system, triggered by `ZonePopulated` event. Consumer: the inference thread.

**Backpressure:** If the queue is full, new items are dropped silently — the game continues with base text. No blocking the simulation thread.

**Zone transition pause:** When the simulation emits a `ZoneTransitionStart` event, the inference thread pauses (drains current item, then waits). Resumes on `ZoneTransitionComplete`. This prevents CPU contention during the loading spike.

### 3.3 Voice cache

```rust
pub struct VoiceCacheKey {
    pub world_seed: u64,
    pub culture_id: String,       // "van-maanens-star"
    pub npc_stable_id: StableId,
    pub content_type: ContentType, // Behavior | Dialogue
    pub content_index: u8,         // which behavior/dialogue line
}

pub struct VoiceCache {
    /// In-memory cache for current session.
    entries: BTreeMap<VoiceCacheKey, String>,
    /// Model identifier used to generate these entries.
    model_id: String,
}
```

**Persistence:** Written to `user://voice_cache/{seed}.msgpack` on zone transition or autosave. Loaded on game start if seed matches. Format version tag for migration.

**Invalidation:** Full cache invalidation on: seed change, model update (game patch). Per-NPC invalidation on: relationship change that affects behavior text.

**Baked content:** Hub zone voiced content ships as a game asset at `res://voice_baked/{zone_id}.msgpack`. Loaded into the cache on zone entry. Never regenerated at runtime.

### 3.4 Tell-as-context prompt construction

Per Jeroen's decision: tells are passthrough (never re-voiced), but they INFORM the re-voicing prompt for behaviors and dialogue.

```rust
/// Build the re-voicing prompt for an NPC's behavior or dialogue.
fn build_prompt(
    base_text: &str,
    content_type: ContentType,
    culture: &CultureProfile,
    npc: &NpcBlueprint,
    active_tell: Option<TellCategory>,  // from DerivedTellState
) -> String {
    let mut prompt = String::with_capacity(512);

    // System instruction
    prompt.push_str(SYSTEM_INSTRUCTION);

    // Culture injector (from NpcBlueprint.cultural_markers — per Miri's recommendation)
    prompt.push_str(&format_culture_injector(&npc.cultural_markers, culture));

    // Personality injector
    prompt.push_str(&format_personality(&npc.traits));

    // Tell-as-context: if the NPC has an active tell, inject it as mood/state context
    // The tell itself is NOT being re-voiced — it's informing the tone
    if let Some(tell) = active_tell {
        prompt.push_str(&format_tell_context(tell));
        // e.g., "The character is currently guarded and evasive.
        //        Their dialogue should reflect this state without stating it directly."
    }

    // Negative constraints (franchise bleed prevention)
    prompt.push_str(NEGATIVE_CONSTRAINTS);

    // Base text to re-voice
    match content_type {
        ContentType::Behavior => {
            prompt.push_str(&format!("\nRephrase this action: \"{}\"", base_text));
        }
        ContentType::Dialogue => {
            prompt.push_str(&format!("\nRephrase this dialogue line: \"{}\"", base_text));
        }
    }

    prompt
}

fn format_tell_context(tell: TellCategory) -> String {
    match tell {
        TellCategory::Nervous => {
            "\nState: The character is anxious. Their speech is clipped, distracted.\n".into()
        }
        TellCategory::Angry => {
            "\nState: The character is angry. Their speech is terse, barely controlled.\n".into()
        }
        TellCategory::Friendly => {
            "\nState: The character is warm and open. Their speech is relaxed.\n".into()
        }
        TellCategory::Guarded => {
            "\nState: The character is guarded. They deflect and keep things vague.\n".into()
        }
        TellCategory::RoutineDeviation => {
            "\nState: The character is preoccupied. Something else is on their mind.\n".into()
        }
    }
}
```

This is the critical design: the `TellCategory` enum flows into the prompt as a mood/state modifier, not as content to re-voice. The tell behavior string stays untouched. The dialogue and ambient behaviors around the tell are colored by the NPC's state.

### 3.5 Observer integration

The observer snapshot system already reads `DerivedTellState` and `observable_behaviors`. Integration point:

```
Observer reads NPC behavior string:
  1. Check voice cache for (seed, culture, npc_id, behavior_index)
  2. If cache hit → use voiced string
  3. If cache miss → use base text string (fallback)
  4. Tell state → always from DerivedTellState (passthrough, never from cache)
```

No changes to the wire format (`ObserverSnapshot`). The client doesn't know or care whether the behavior string was voiced or base text.

### 3.6 Baked content generation

A build-time step that runs the inference engine on all hub zone NPCs:

```bash
# Build tool (not the game binary)
sr-voice-bake \
  --model models/gemma-2b-q4.gguf \
  --zones content/global/van-maanens-star-*.ron \
  --culture content/global/culture-van-maanens-star.ron \
  --output client/assets/voice_baked/ \
  --seed 0  # baked content uses seed 0 as the canonical reference
```

Output: one `.msgpack` file per zone containing all voiced behavior and dialogue strings. Checked into the repository (text-only, compresses to ~20-50KB per zone). Human-reviewed by Paula/Mellanie before ship.

---

## 4. Hardware Detection Design

### Layer 1: RAM check (can the model load?)

On first toggle of "AI-Enhanced Dialogue":

```rust
fn check_ram_available() -> RamCheckResult {
    let available_mb = get_available_system_ram_mb();
    let model_size_mb = 1600; // Gemma 2B Q4 + KV cache overhead

    if available_mb < model_size_mb {
        RamCheckResult::InsufficientRam {
            available_mb,
            required_mb: model_size_mb,
        }
    } else {
        RamCheckResult::Ok
    }
}
```

**User-facing message if insufficient:**
> "AI-Enhanced Dialogue requires approximately 1.6 GB of free RAM. Your system currently has {available_mb} MB available. The feature may cause instability. Enable anyway?"

Player can always override. No hard block.

### Layer 2: Time-per-token benchmark (is inference useful?)

If RAM check passes, run a 5-token benchmark on first enable:

```rust
fn benchmark_inference(engine: &mut InferenceEngine) -> BenchmarkResult {
    let test_prompt = "Rephrase: \"walks down the corridor.\"";
    let result = engine.generate(test_prompt, 5, 0.7, 0.9)?;
    let tpt_ms = result.generation_time_ms as f32 / result.tokens_generated as f32;

    BenchmarkResult {
        tokens_per_sec: result.tokens_per_sec,
        time_per_token_ms: tpt_ms,
    }
}
```

**Thresholds:**

| Tokens/sec | Recommendation | User message |
|-----------|---------------|--------------|
| >= 5 t/s | Full enable | "AI-Enhanced Dialogue is active." |
| 2-5 t/s | Enable with warning | "AI-Enhanced Dialogue is active. On your hardware, voiced content will generate slowly. Some NPCs may show plain text until generation catches up." |
| < 2 t/s | Recommend disable | "Your hardware generates voiced content very slowly. We recommend disabling AI-Enhanced Dialogue for the best experience. Enable anyway?" |

**No hard floor.** Player can always choose to run it. The benchmark runs once, result is cached in user settings. Player can re-run from the settings menu.

### Layer 3: Runtime monitoring

During gameplay, the inference thread monitors its own throughput:

```rust
// In the inference thread main loop:
if current_tokens_per_sec < 1.0 {
    // Sustained very-slow inference — likely thermal throttle or power saver
    pause_inference();
    notify_ui("AI dialogue generation paused — system is running slowly.");
    // Resume after 60 seconds or on user action
}
```

**Battery/power-saver detection:** On Windows, check `GetSystemPowerStatus()`. If on battery with power saver active, auto-pause inference and show notification. On Linux/macOS, check `/sys/class/power_supply/` or equivalent. Resume when plugged in or power mode changes.

### Settings UI

```
[Settings > Audio & Dialogue]

AI-Enhanced Dialogue: [ON / OFF]
  Status: Active (8.2 tokens/sec)

  [Re-run benchmark]

  Note: When enabled, NPC dialogue and behaviors are enhanced with
  culture-specific voice. This uses additional CPU resources.
  Disable if you experience performance issues.
```

---

## Effort Estimates

| Work item | Sprints | Dependencies |
|-----------|---------|-------------|
| Spike 1: `sr-voice` CLI tool | 1 | None — can start immediately |
| Spike 1: Prompt crafting + model testing | 1 | sr-voice CLI (Mellanie/Paula/Jeroen run the tests) |
| Spike 2: Queue + cache + thread pool | 1.5 | Spike 1 model selection |
| Spike 2: Tell-as-context prompt construction | 0.5 | Queue infrastructure |
| Spike 2: Observer integration | 0.5 | Cache system |
| Spike 2: Baked content generation tool | 0.5 | Queue + cache |
| Hardware detection system | 0.5 | InferenceEngine (from Spike 1) |
| **Total** | **5.5** | Spike 1 and 2 are sequential; sub-items within each spike are partially parallelizable |

Spike 1 can start next sprint. The CLI tool is self-contained Rust with no game dependencies. While the content team runs manual prompt tests, Spike 2 infrastructure design can begin in parallel.