jpmschweitzer/settled-reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(voice): complete Spike 2 voice pipeline with quality-tested prompt engine

Browse Source 
Spike 2 delivers the full voice pipeline: queue → worker pool → sr-voice
child process (stdio JSONL) → cache → disk. Three rounds of quality testing
with Paula, Mellanie, and Gestalt produced iterative prompt improvements.

Prompt engine (prompt_builder.rs):
- Example-based epistemic marker integration (not keyword lists)
- Length-aware Angry tell variant (preserves facts on long content)
- Double-prompt technique: REMEMBER block repeats constraints near OUTPUT:
- Imperative injection framing (composition engine controls frequency)
- Anti-invention constraint ("do not add information not in the input")
- Universal RULES cleaned: worldbuilding moved to culture personas

Worker pool (worker.rs):
- Output post-processor strips after first newline (prevents prompt leakage)
- Watchdog poll loop (1s ticks) replaces blocking sleep for cancel
- Child health check before writing (try_wait)

Test infrastructure:
- voice_pipeline.rs: end-to-end test, auto-detects real sr-voice or mock
- voice_quality_batch.rs: 39 edge-case prompts for quality review
- mock-stdio.sh: Python JSONL mock for CI (no model needed)
- Makefile targets: test-voice-mock, test-voice-real

Quality results (Gemma 2B Q4_K_M, CPU ~13 t/s):
- Epistemic markers: naturally integrated (round 1 comma-lists fixed)
- Tell differentiation: 3/5 working (Nervous, Guarded, Angry)
- Information preservation: ~90% (up from ~70%)
- Prompt leakage: eliminated
- Open: Friendly/RoutineDeviation tells inert (#651), Factual bypass (#650)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2026-03-07 19:20:06 +01:00
co-authored by Claude Opus 4.6
parent e93a9e8b70
commit 9f34d030d7
6 changed files with 1201 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Makefile
+15 -1
View File
@@ -7,7 +7,7 @@ GODOT := $(shell command -v godot4 2>/dev/null || command -v godot 2>/dev/null)
pre-pr-server pre-pr-client pre-pr-content \
fixtures-client fixtures-gauntlet golden-diff golden-update \
checklist-validate checklist-generate \
build-sr-voice run-sr-voice \
build-sr-voice run-sr-voice test-voice-mock test-voice-real \
perf-baseline debug-schedule \
test-ipc-fixtures test-ipc-protocol test-ipc-integration test-ipc-benchmark \
screenshot visual-movie test-visual visual-update
@@ -70,6 +70,8 @@ help:
@echo " make serve-sr-voice Start sr-voice server (ARGS='--model <path>')"
@echo " make run-sr-voice Submit to sr-voice server (ARGS='generate|batch|benchmark ...')"
@echo " make stop-sr-voice Stop sr-voice server"
@echo " make test-voice-mock Test voice pipeline with mock sr-voice"
@echo " make test-voice-real Test voice pipeline with real sr-voice + Gemma 2B"
@echo " make debug-schedule Print bevy_ecs schedule graph (diff for PR artifacts)"
@echo ""
@echo " GODOT_VERSION=4.6 make setup Override Godot version"
@@ -368,6 +370,18 @@ stop-sr-voice:
@lsof -ti :$(SR_VOICE_PORT) | xargs -r kill 2>/dev/null || true
@echo "Stopped sr-voice on port $(SR_VOICE_PORT)"
test-voice-mock:
@echo "Running voice pipeline test (mock sr-voice)..."
cd server && SR_VOICE_MOCK=1 cargo test --test voice_pipeline -- --nocapture
@echo "Results: .tmp/voice-test/results.txt"
test-voice-real:
@echo "Running voice pipeline test (real sr-voice + Gemma 2B)..."
@test -f server/sr-voice/target/release/sr-voice || { echo "Build sr-voice first: make build-sr-voice"; exit 1; }
@test -f server/models/gemma2.gguf || { echo "Model not found: server/models/gemma2.gguf"; exit 1; }
cd server && cargo test --test voice_pipeline -- --nocapture
@echo "Results: .tmp/voice-test/results.txt"
content-ron:
cd tooling/content-converter && cargo build --release
tooling/content-converter/target/release/content-converter --input content --output content-ron --verbose
server/sr-voice/mock-stdio.sh
Executable
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Mock sr-voice stdio mode for pipeline testing (D-138).
Reads JSONL from stdin, writes JSONL to stdout. Simulates inference
by uppercasing the base text portion of the prompt as the "voiced" output.
Usage: echo '{"prompt":"Re-voice this.","seed":42}' | ./mock-stdio.sh serve --stdio
"""
import json
import sys
print("mock-sr-voice: stdio mode (no model loaded)", file=sys.stderr, flush=True)
# Use readline() loop — Python's `for line in sys.stdin` has an internal
# read-ahead buffer that blocks on piped stdin until 8KB is available.
while True:
line = sys.stdin.readline()
if not line:
break
line = line.strip()
if not line:
continue
try:
req = json.loads(line)
except json.JSONDecodeError as e:
print(json.dumps({"error": f"invalid JSON: {e}"}), flush=True)
continue
prompt = req.get("prompt", "")
if not prompt:
print(json.dumps({"error": "missing prompt field"}), flush=True)
continue
# Extract the INPUT line from the prompt (last INPUT: before OUTPUT:)
input_text = ""
for pline in prompt.split("\n"):
if pline.startswith("INPUT: "):
input_text = pline[7:]
if not input_text:
input_text = prompt[:80]
voiced = f"[VOICED] {input_text.upper()}"
result = {
"text": voiced,
"tokens_generated": len(voiced.split()),
"generation_time_ms": 50,
"tokens_per_sec": 240.0,
"prefill_time_ms": 10,
}
print(json.dumps(result), flush=True)
print("mock-sr-voice: stdin closed, exiting", file=sys.stderr, flush=True)
server/src/voice/prompt_builder.rs
+141 -56
View File
@@ -33,50 +33,73 @@ pub struct BuiltPrompt {
pub injections_fired: Vec<usize>,
}
/// Universal rules prefix — format constraints and negative injectors.
/// Universal rules prefix — format and output constraints only.
/// Worldbuilding and style constraints belong in the culture persona or
/// the TASK section, not here.
const RULES: &str = "\
RULES: Output exactly one line of voiced text. \
No explanation. No options. No markdown. No labels. Stop after one line.\n\n\
CONSTRAINTS:\n\
- Use occupational titles (shift lead, supervisor, foreman), not military ranks.\n\
- Technology: insert (neural implant), span gate (FTL transit), \
horizon gate (alien gate), the Reach (settled systems).\n\
- No wit, quips, or wordplay. Humor is dry and rare.\n\
- Do not reference Earth as a current place. Cultural heritage markers are natural.";
OUTPUT CONSTRAINTS:\n\
- Speak in complete sentences. Short ones. Cut words that don't pull weight \
— but keep the sentence structure.\n\
- Do not summarize. Keep all facts from the input. Shorten phrasing, not content.\n\
- Do not add information that is not in the input. No invented details.\n\
- NOT: \"Parts late. Behind.\" YES: \"Parts never came. We're a shift behind.\"";
/// Tell-state tone injectors (D-024 tell taxonomy).
///
/// Each tell category has a carefully worded tone modifier that influences
/// the LLM output without naming the emotion. The model shows, not tells.
/// Each tell category has a concrete syntactic instruction with an example,
/// tuned for 2B model capacity. Behavioral descriptions alone ("a beat late")
/// don't produce differentiated output at this model size — concrete surface
/// patterns are needed.
///
/// Angry has a length-aware variant: on long content (16+ words), the default
/// "make sentences shorter" instruction causes destructive compression that
/// strips facts. Long-Angry instead preserves the full claim and focuses
/// intensity on one sentence.
fn tell_injector(category: TellCategory) -> &'static str {
match category {
TellCategory::Nervous => {
"TELL-STATE: This character's words come slightly faster than usual, briefer. \
They don't elaborate. A phrase drops off before it's finished. \
Do not say they seem nervous or afraid."
"TONE: Cut one clause from the sentence. Let a phrase trail off with a dash or ellipsis. \
Example: \"Yeah, it's just — doesn't matter.\" \
Do not say they seem nervous."
}
TellCategory::Angry => {
"TELL-STATE: This character's words are measured and deliberate — not shouting, containing. \
A word hits harder than the context requires. Do not say they seem angry."
"TONE: Make sentences shorter and more deliberate. One word should hit harder than expected. \
Example: \"Supervisor wants me in early.\" where \"wants\" carries weight. \
Do not say they seem angry."
}
TellCategory::Friendly => {
"TELL-STATE: This character offers slightly more than asked. \
A word of genuine warmth lands casually. They don't perform friendliness — it just shows. \
Do not add compliments or over-warmth."
"TONE: Add one small extra detail or aside that wasn't strictly necessary. \
Example: \"Inspection's tomorrow — should be fine, though.\" \
Do not add compliments or forced warmth."
}
TellCategory::Guarded => {
"TELL-STATE: This character chooses each word with a half-second more care than normal. \
They answer what was asked, no more. There is nothing wrong here. \
Do not say they seem guarded or evasive."
"TONE: Use formal, precise words. Answer exactly what was asked, nothing extra. \
Example: \"That's correct.\" instead of \"Yeah, exactly.\" \
Do not say they seem guarded."
}
TellCategory::RoutineDeviation => {
"TELL-STATE: This character is elsewhere in their mind. \
They are present but preoccupied — answers are on track but land a beat late. \
Do not explain why or name what they're thinking about."
"TONE: Start the sentence on topic, then add a brief unfinished thought about something else. \
Example: \"Pressure's fine. I was going to — anyway, it's logged.\" \
Do not explain what they were thinking about."
}
}
}
/// Long-content variant for Angry tell. Used when base_text is 16+ words
/// to prevent destructive compression that strips facts.
const ANGRY_LONG: &str = "\
TONE: Keep the full claim intact — do not cut facts. \
Make one sentence land harder than the rest. \
Example: \"Three reports filed. No budget. But they found half a million for the lounge.\" \
Do not say they seem angry.";
/// Whether to use the long-content Angry variant.
fn is_long_content(base_text: &str) -> bool {
word_count(base_text) >= 16
}
/// Known epistemic markers that must be preserved through re-voicing.
///
/// When the base text contains these phrases, the LLM is instructed to
@@ -131,10 +154,15 @@ fn extract_epistemic_markers(base_text: &str) -> Vec<&'static str> {
/// 1. Universal RULES prefix (format constraints, negative injectors)
/// 2. Culture-specific PERSONA block (from `culture.voice_persona`)
/// 3. Culture-specific examples
/// 4. Occasional injections (rolled per-prompt via seeded RNG)
/// 5. Tell-state tone modifier (only for medium/long content)
/// 6. Epistemic marker protection
/// 7. TASK + INPUT + OUTPUT: stop token
/// 4. Tell-state tone modifier (only for medium/long content)
/// 5. Epistemic marker protection (example-based, not keyword-list)
/// 6. Occasional injections (imperative, positioned near TASK for 2B attention)
/// 7. TASK + INPUT + repeated RULES reminder + OUTPUT: stop token
///
/// The prompt is structured so that the most important instructions appear
/// both at the start and immediately before OUTPUT: (double-prompt technique).
/// 2B models de-weight early prompt sections; repeating near the end anchors
/// the instructions in the attention window.
///
/// `seed` should be deterministic per (npc_id, content_index, world_seed)
/// so that the same prompt produces the same injection pattern on re-run.
@@ -145,7 +173,7 @@ pub fn build_prompt(
tell_state: Option<TellCategory>,
seed: u64,
) -> BuiltPrompt {
let mut parts: Vec<String> = Vec::with_capacity(10);
let mut parts: Vec<String> = Vec::with_capacity(16);
let mut injections_fired: Vec<usize> = Vec::new();
// 1. Universal rules
@@ -167,7 +195,47 @@ pub fn build_prompt(
}
}
// 4. Occasional injections — rolled by composition engine, not model
// 4. Tell-state tone modifier (skip for short content — 2B model can't differentiate)
if !is_short_content(base_text) {
if let Some(tell) = tell_state {
parts.push(String::new());
// Angry on long content uses a special variant that preserves facts
if tell == TellCategory::Angry && is_long_content(base_text) {
parts.push(ANGRY_LONG.to_string());
} else {
parts.push(tell_injector(tell).to_string());
}
}
}
// 5. Epistemic marker protection — example-based, not keyword-list.
// The old keyword-list approach ("must appear in the output: i heard, might have")
// caused 2B models to emit markers as comma-separated lists. Example-based
// integration teaches the model how to weave them into natural speech.
let markers = extract_epistemic_markers(base_text);
if !markers.is_empty() {
parts.push(String::new());
if markers.len() == 1 {
parts.push(format!(
"PRESERVE: The phrase \"{}\" carries specific meaning. \
Use it naturally in the output as part of a sentence, not as a label. \
Example: \"I heard they stopped the line — twice, apparently.\"",
markers[0]
));
} else {
let marker_list = markers.join("\", \"");
parts.push(format!(
"PRESERVE: The phrases \"{}\" carry specific meaning. \
Weave them naturally into the output sentence. Do not list them. \
Example: \"I heard they rerouted it. Might have been last cycle.\"",
marker_list
));
}
}
// 6. Occasional injections — imperative, positioned near TASK for 2B attention.
// The composition engine already controls frequency — once an injection fires,
// the model must execute it without discretion.
let mut rng = ChaCha8Rng::seed_from_u64(seed);
for (i, injection) in culture.occasional_injections.iter().enumerate() {
// Gate off for suppressive tells
@@ -179,42 +247,59 @@ pub fn build_prompt(
if rng.random::<f32>() < injection.frequency {
parts.push(String::new());
parts.push(injection.clause.clone());
// Imperative framing — no "when" conditional, just "include this"
parts.push(format!("INJECT: Include the phrase from this example in your output."));
if let Some(ref example) = injection.example {
parts.push(format!("INPUT: {}", example.input));
parts.push(format!("OUTPUT: {}", example.output));
} else {
parts.push(injection.clause.clone());
}
injections_fired.push(i);
}
}
// 5. Tell-state tone modifier (skip for short content — 2B model can't differentiate)
if !is_short_content(base_text) {
if let Some(tell) = tell_state {
parts.push(String::new());
parts.push(tell_injector(tell).to_string());
}
}
// 6. Epistemic marker protection
let markers = extract_epistemic_markers(base_text);
if !markers.is_empty() {
parts.push(String::new());
let marker_list = markers.join(", ");
parts.push(format!(
"PRESERVE: The following phrases must appear in the output: {}",
marker_list
));
}
// 7. Task + input + output stop token
// 7. Task + input + repeated rules reminder + output stop token
let task_verb = match content_type {
ContentType::Dialogue => "Re-voice",
ContentType::Behavior => "Describe",
ContentType::Dialogue => {
"Re-voice the following in this character's voice. \
Keep all facts. Use complete sentences"
}
ContentType::Behavior => {
"Describe the following action as a third-person observer. \
Preserve all individual actions in sequence. Do not extract conclusions"
}
};
parts.push(String::new());
parts.push(format!("TASK: {} the following in this character's voice.", task_verb));
parts.push(format!("TASK: {}.", task_verb));
parts.push(format!("INPUT: {}", base_text));
// Double-prompt: repeat the critical constraints immediately before OUTPUT:
// to anchor them in the 2B model's attention window.
let mut reminder = String::from("REMEMBER:");
reminder.push_str(" Output exactly one line.");
reminder.push_str(" Keep all facts from the input. Do not invent new details.");
reminder.push_str(" Complete sentences, not fragments.");
if !markers.is_empty() {
reminder.push_str(&format!(
" Use \"{}\" naturally in the sentence.",
markers[0]
));
}
if !injections_fired.is_empty() {
// Remind about the first fired injection
if let Some(inj) = culture
.occasional_injections
.get(*injections_fired.first().unwrap())
{
if let Some(ref ex) = inj.example {
// Extract the key phrase from the example output
let phrase = ex.output.split('.').next().unwrap_or(&ex.output);
reminder.push_str(&format!(" Include a phrase like \"{}\".", phrase));
}
}
}
parts.push(reminder);
parts.push("OUTPUT:".to_string());
BuiltPrompt {
@@ -379,7 +464,7 @@ mod tests {
42,
);
// 3 words — should skip tell injector
assert!(!result.prompt.contains("TELL-STATE:"));
assert!(!result.prompt.contains("TONE:"));
}
#[test]
@@ -392,8 +477,8 @@ mod tests {
Some(TellCategory::Nervous),
42,
);
assert!(result.prompt.contains("TELL-STATE:"));
assert!(result.prompt.contains("slightly faster than usual"));
assert!(result.prompt.contains("TONE:"));
assert!(result.prompt.contains("trail off"));
}
#[test]
server/src/voice/worker.rs
+22 -8
View File
@@ -205,18 +205,23 @@ impl VoicePipe {
// Watchdog: kill the child if it doesn't respond within the timeout.
// This unblocks the read_line below (stdout closes → read returns empty).
// Uses a poll loop (1s ticks) so the watchdog exits promptly on cancel.
let child_id = self.child.id();
let cancel = Arc::new(AtomicBool::new(false));
let cancel_clone = Arc::clone(&cancel);
let timeout_secs = INFERENCE_TIMEOUT.as_secs();
let watchdog = std::thread::spawn(move || {
std::thread::sleep(INFERENCE_TIMEOUT);
if !cancel_clone.load(Ordering::Relaxed) {
tracing::warn!(pid = child_id, "sr-voice inference timeout — killing child");
let _ = Command::new("kill")
.arg("-9")
.arg(child_id.to_string())
.output();
for _ in 0..timeout_secs {
std::thread::sleep(Duration::from_secs(1));
if cancel_clone.load(Ordering::Relaxed) {
return;
}
}
tracing::warn!(pid = child_id, "sr-voice inference timeout — killing child");
let _ = Command::new("kill")
.arg("-9")
.arg(child_id.to_string())
.output();
});
let mut response_line = String::new();
@@ -241,7 +246,16 @@ impl VoicePipe {
body["text"]
.as_str()
.map(|s| s.trim().to_string())
.map(|s| {
// Post-processor: strip everything after the first newline.
// Prevents prompt leakage (survey bleed, example continuation)
// that 2B models sometimes produce after the first valid line.
let trimmed = s.trim();
match trimmed.find('\n') {
Some(pos) => trimmed[..pos].trim().to_string(),
None => trimmed.to_string(),
}
})
.ok_or_else(|| "response missing 'text' field".to_string())
}
}
server/tests/voice_pipeline.rs
+321
View File
@@ -0,0 +1,321 @@
//! Voice pipeline integration test (D-138, Spike 2).
//!
//! Exercises the full pipeline: queue → worker → sr-voice child (mock or real)
//! → cache → disk. Outputs voiced results to `.tmp/voice-test/`.
//!
//! Run with mock: cargo test --test voice_pipeline -- --nocapture
//! Run with real: SR_VOICE_BIN=sr-voice/target/release/sr-voice \
//! SR_VOICE_MODEL=models/gemma2.gguf \
//! cargo test --test voice_pipeline -- --nocapture
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use settled_reach_server::npc::blueprint::{
CulturalValues, CultureProfile, NamingConventions, OccasionalInjection, SpeechPatterns,
VoiceExample,
};
use settled_reach_server::npc::PersonalityTrait;
use settled_reach_server::npc::tell_state::TellCategory;
use settled_reach_server::voice::cache::VoiceCacheStore;
use settled_reach_server::voice::prompt_builder::ContentType;
use settled_reach_server::voice::queue::{Priority, VoiceQueue, VoiceRequest};
use settled_reach_server::voice::worker::{VoiceProcessConfig, WorkerPool};
/// Output directory for test results (relative to repo root).
const OUTPUT_DIR: &str = ".tmp/voice-test";
fn output_dir() -> PathBuf {
let manifest = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into());
PathBuf::from(manifest).join("..").join(OUTPUT_DIR)
}
/// Known paths relative to CARGO_MANIFEST_DIR (server/).
const SR_VOICE_BIN: &str = "sr-voice/target/release/sr-voice";
const SR_VOICE_MODEL: &str = "models/gemma2.gguf";
fn voice_config() -> VoiceProcessConfig {
let manifest = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into());
let manifest = PathBuf::from(manifest);
let bin_path = manifest.join(SR_VOICE_BIN);
let model_path = manifest.join(SR_VOICE_MODEL);
// SR_VOICE_MOCK=1 forces mock mode (used by `make test-voice-mock`)
let force_mock = std::env::var("SR_VOICE_MOCK").is_ok();
// Use real sr-voice if both binary and model exist on disk
if !force_mock && bin_path.exists() && model_path.exists() {
eprintln!("Using real sr-voice: {}", bin_path.display());
eprintln!("Model: {}", model_path.display());
VoiceProcessConfig {
binary_path: bin_path.to_string_lossy().into(),
model_path: model_path.to_string_lossy().into(),
threads: 2,
ctx_size: 512,
}
} else {
// Fall back to mock script — no model needed
let mock = manifest.join("sr-voice/mock-stdio.sh");
assert!(
mock.exists(),
"Mock script not found: {}",
mock.display()
);
eprintln!("Using mock sr-voice: {}", mock.display());
if !bin_path.exists() {
eprintln!(" (real binary not found: {})", bin_path.display());
}
if !model_path.exists() {
eprintln!(" (model not found: {})", model_path.display());
}
VoiceProcessConfig {
binary_path: mock.to_string_lossy().into(),
model_path: "unused".into(),
threads: 1,
ctx_size: 512,
}
}
}
fn krenn_culture() -> CultureProfile {
CultureProfile {
id: "krenn".into(),
name: "Krenn System Culture".into(),
description: "Working-class pragmatic".into(),
naming: NamingConventions {
style: "compact".into(),
given_names: vec!["Kael".into()],
family_names: vec!["Davan".into()],
family_name_used_socially: false,
},
speech: SpeechPatterns {
register: "direct".into(),
filler_words: vec!["look".into()],
greetings: vec!["hey".into()],
farewells: vec!["shift's calling".into()],
exclamations: vec!["void take it".into()],
},
values: CulturalValues {
description: "Pragmatic".into(),
favored_traits: vec![PersonalityTrait::Bold],
disfavored_traits: vec![PersonalityTrait::Reclusive],
},
voice_persona: Some(
"PERSONA: You are a Krenn station worker.\n\
1. Be direct. No pleasantries.\n\
2. You're working-class and pragmatic."
.into(),
),
voice_examples: vec![VoiceExample {
input: "declines to answer a question".into(),
output: "Look, that's not mine to say.".into(),
}],
occasional_injections: vec![OccasionalInjection {
kind: "oath".into(),
clause: "Use an oath like \"void take it.\"".into(),
example: Some(VoiceExample {
input: "discovers a critical part is missing".into(),
output: "Void take it. The coupling's not here.".into(),
}),
frequency: 0.25,
suppress_on_tells: vec![
TellCategory::Guarded,
TellCategory::RoutineDeviation,
TellCategory::Friendly,
],
}],
}
}
fn test_requests() -> Vec<VoiceRequest> {
vec![
VoiceRequest {
priority: Priority::High,
npc_stable_id: 1,
zone_id: 100,
culture_id: "krenn".into(),
base_text: "The coupling is faulty.".into(),
content_type: ContentType::Dialogue,
content_index: 0,
tell_state: None,
seed: 42,
},
VoiceRequest {
priority: Priority::High,
npc_stable_id: 1,
zone_id: 100,
culture_id: "krenn".into(),
base_text: "I heard the night crew had to stop the line twice because the coupling was faulty and nobody had flagged it in the log.".into(),
content_type: ContentType::Dialogue,
content_index: 1,
tell_state: Some(TellCategory::Nervous),
seed: 43,
},
VoiceRequest {
priority: Priority::Standard,
npc_stable_id: 2,
zone_id: 100,
culture_id: "krenn".into(),
base_text: "checks the pressure gauge and writes in a logbook".into(),
content_type: ContentType::Behavior,
content_index: 0,
tell_state: None,
seed: 44,
},
VoiceRequest {
priority: Priority::High,
npc_stable_id: 3,
zone_id: 100,
culture_id: "unknown_culture".into(),
base_text: "Unknown culture fallback test.".into(),
content_type: ContentType::Dialogue,
content_index: 0,
tell_state: None,
seed: 45,
},
]
}
#[test]
fn voice_pipeline_end_to_end() {
// Setup output dir
let out = output_dir();
let cache_dir = out.join("cache");
let _ = std::fs::remove_dir_all(&cache_dir);
std::fs::create_dir_all(&cache_dir).expect("failed to create cache dir");
let config = voice_config();
// Build cultures map
let mut cultures = BTreeMap::new();
cultures.insert("krenn".to_string(), krenn_culture());
let cultures = Arc::new(cultures);
// Create cache
let cache = Arc::new(Mutex::new(VoiceCacheStore::new(
cache_dir.clone(),
12345,
"test-v1".into(),
"test-i1".into(),
)));
// Create queue and worker pool
let queue = VoiceQueue::new();
let paused = queue.paused_flag();
let mut pool = WorkerPool::spawn(
2, // 2 workers
&config,
queue.receiver(),
Arc::clone(&cache),
Arc::clone(&cultures),
paused,
);
eprintln!("Worker pool started: {} workers", pool.worker_count());
// Submit test requests
let requests = test_requests();
let request_count = requests.len();
for req in requests {
let submitted = queue.submit(req);
assert!(submitted, "queue should accept request");
}
eprintln!("Submitted {} requests", request_count);
// Wait for processing (mock is instant, real LLM takes seconds per request)
let manifest = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into());
let is_real = std::env::var("SR_VOICE_MOCK").is_err()
&& PathBuf::from(&manifest).join(SR_VOICE_BIN).exists()
&& PathBuf::from(&manifest).join(SR_VOICE_MODEL).exists();
let max_wait = if is_real {
Duration::from_secs(120)
} else {
Duration::from_secs(10)
};
let start = std::time::Instant::now();
loop {
std::thread::sleep(Duration::from_millis(200));
let pending = queue.pending_count();
let active = pool.active_workers();
if pending == 0 && active == 0 {
break;
}
if start.elapsed() > max_wait {
eprintln!(
"Timeout waiting for pipeline (pending={}, active={})",
pending, active
);
break;
}
}
let elapsed = start.elapsed();
eprintln!("Pipeline drained in {:.1}s", elapsed.as_secs_f64());
// Shutdown workers
pool.shutdown();
eprintln!("Workers shut down");
// Inspect cache
let mut cache_guard = cache.lock().unwrap();
let zone_cache = cache_guard.zone_cache(100);
let entry_count = zone_cache.len();
eprintln!("Cache entries for zone 100: {}", entry_count);
// Write results to a human-readable file
let results_path = out.join("results.txt");
let mut results = String::new();
results.push_str(&format!(
"Voice Pipeline Test Results\n\
==========================\n\
Workers: {}\n\
Requests: {}\n\
Cache entries: {}\n\
Time: {:.1}s\n\
Mode: {}\n\n",
2,
request_count,
entry_count,
elapsed.as_secs_f64(),
if is_real { "real sr-voice" } else { "mock" },
));
for (key, text) in &zone_cache.entries {
results.push_str(&format!(
"--- npc={} type={:?} idx={} tell={:?} ---\n{}\n\n",
key.npc_stable_id, key.content_type, key.content_index, key.tell_state, text
));
}
// Save cache to disk
drop(cache_guard);
// Drop triggers save_all via the Drop impl — but the cache is behind
// Arc<Mutex<>>, so we can't drop it here. Explicitly save instead.
cache.lock().unwrap().save_all().expect("failed to save cache");
std::fs::write(&results_path, &results).expect("failed to write results");
eprintln!("Results written to {}", results_path.display());
eprintln!("\n{}", results);
// Assertions
assert!(
entry_count >= 3,
"Expected at least 3 cache entries (3 valid culture requests), got {}",
entry_count
);
// The unknown culture request should still cache (with base text as fallback)
// Total should be 4 (3 krenn + 1 unknown fallback)
assert!(
entry_count == 4,
"Expected 4 cache entries (3 krenn + 1 unknown fallback), got {}",
entry_count
);
}
server/tests/voice_quality_batch.rs
+647
View File
@@ -0,0 +1,647 @@
//! Voice quality batch test (D-138, Spike 2).
//!
//! Runs a comprehensive set of edge-case prompts through the voice pipeline
//! and writes results to `.tmp/voice-test/quality-batch.txt` for human review.
//!
//! Run: make test-voice-quality
//! Or: cd server && cargo test --test voice_quality_batch -- --nocapture --ignored
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use settled_reach_server::npc::blueprint::{
CulturalValues, CultureProfile, NamingConventions, OccasionalInjection, SpeechPatterns,
VoiceExample,
};
use settled_reach_server::npc::PersonalityTrait;
use settled_reach_server::npc::tell_state::TellCategory;
use settled_reach_server::voice::cache::{CacheKey, VoiceCacheStore};
use settled_reach_server::voice::prompt_builder::ContentType;
use settled_reach_server::voice::queue::{Priority, VoiceQueue, VoiceRequest};
use settled_reach_server::voice::worker::{VoiceProcessConfig, WorkerPool};
// ---------------------------------------------------------------------------
// Config
// ---------------------------------------------------------------------------
const SR_VOICE_BIN: &str = "sr-voice/target/release/sr-voice";
const SR_VOICE_MODEL: &str = "models/gemma2.gguf";
const OUTPUT_DIR: &str = ".tmp/voice-test";
fn output_dir() -> PathBuf {
let manifest = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into());
PathBuf::from(manifest).join("..").join(OUTPUT_DIR)
}
fn voice_config() -> VoiceProcessConfig {
let manifest = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into());
let manifest = PathBuf::from(manifest);
let bin_path = manifest.join(SR_VOICE_BIN);
let model_path = manifest.join(SR_VOICE_MODEL);
assert!(
bin_path.exists(),
"sr-voice binary not found: {} — run `make build-sr-voice`",
bin_path.display()
);
assert!(
model_path.exists(),
"Model not found: {}",
model_path.display()
);
VoiceProcessConfig {
binary_path: bin_path.to_string_lossy().into(),
model_path: model_path.to_string_lossy().into(),
threads: 2,
ctx_size: 512,
}
}
// ---------------------------------------------------------------------------
// Culture profiles
// ---------------------------------------------------------------------------
fn krenn_culture() -> CultureProfile {
CultureProfile {
id: "krenn".into(),
name: "Krenn System Culture".into(),
description: "Working-class pragmatic".into(),
naming: NamingConventions {
style: "compact".into(),
given_names: vec!["Kael".into(), "Dren".into(), "Sira".into()],
family_names: vec!["Davan".into(), "Voss".into()],
family_name_used_socially: false,
},
speech: SpeechPatterns {
register: "direct".into(),
filler_words: vec!["look".into()],
greetings: vec!["hey".into()],
farewells: vec!["shift's calling".into()],
exclamations: vec!["void take it".into()],
},
values: CulturalValues {
description: "Pragmatic".into(),
favored_traits: vec![PersonalityTrait::Bold],
disfavored_traits: vec![PersonalityTrait::Reclusive],
},
voice_persona: Some(
"PERSONA: You are a Krenn station worker.\n\
1. Be direct. No pleasantries.\n\
2. You're working-class and pragmatic."
.into(),
),
voice_examples: vec![
VoiceExample {
input: "declines to answer a question".into(),
output: "Look, that's not mine to say.".into(),
},
VoiceExample {
input: "confirms a task is complete".into(),
output: "Done. Logged it.".into(),
},
],
occasional_injections: vec![OccasionalInjection {
kind: "oath".into(),
clause: "Use an oath like \"void take it\" when something is surprising or frustrating."
.into(),
example: Some(VoiceExample {
input: "discovers a critical part is missing".into(),
output: "Void take it. The coupling's not here.".into(),
}),
frequency: 0.25,
suppress_on_tells: vec![
TellCategory::Guarded,
TellCategory::RoutineDeviation,
TellCategory::Friendly,
],
}],
}
}
// ---------------------------------------------------------------------------
// Test cases
// ---------------------------------------------------------------------------
struct TestCase {
label: &'static str,
category: &'static str,
base_text: &'static str,
content_type: ContentType,
tell_state: Option<TellCategory>,
seed: u64,
}
fn test_cases() -> Vec<TestCase> {
vec![
// === CATEGORY: Length tiers ===
TestCase {
label: "ultra-short (2 words)",
category: "length",
base_text: "It's broken.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 100,
},
TestCase {
label: "short (5 words)",
category: "length",
base_text: "The coupling is beyond repair.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 101,
},
TestCase {
label: "medium (12 words)",
category: "length",
base_text: "The pressure readings have been unstable all week and nobody filed a report.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 102,
},
TestCase {
label: "long (28 words)",
category: "length",
base_text: "I was checking the manifests from last quarter and it looks like three shipments \
never arrived at the depot, which means someone either lost them or diverted \
them deliberately.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 103,
},
// === CATEGORY: Tell states on medium content ===
TestCase {
label: "medium + Nervous",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Nervous),
seed: 200,
},
TestCase {
label: "medium + Angry",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Angry),
seed: 200,
},
TestCase {
label: "medium + Friendly",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Friendly),
seed: 200,
},
TestCase {
label: "medium + Guarded",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Guarded),
seed: 200,
},
TestCase {
label: "medium + RoutineDeviation",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::RoutineDeviation),
seed: 200,
},
TestCase {
label: "medium + neutral (baseline)",
category: "tell-medium",
base_text: "The supervisor asked me to come in early tomorrow for an unscheduled inspection.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 200,
},
// === CATEGORY: Tell states on short content (should NOT differentiate) ===
TestCase {
label: "short + Nervous (expect same as neutral)",
category: "tell-short",
base_text: "Inspection's next week.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Nervous),
seed: 300,
},
TestCase {
label: "short + Angry (expect same as neutral)",
category: "tell-short",
base_text: "Inspection's next week.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Angry),
seed: 300,
},
TestCase {
label: "short + neutral (baseline)",
category: "tell-short",
base_text: "Inspection's next week.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 300,
},
// === CATEGORY: Epistemic markers ===
TestCase {
label: "\"I heard\" — must preserve",
category: "epistemic",
base_text: "I heard the night crew had to stop the line twice because the coupling was faulty.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 400,
},
TestCase {
label: "\"someone told me\" — must preserve",
category: "epistemic",
base_text: "Someone told me the foreman is transferring out next cycle.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 401,
},
TestCase {
label: "\"apparently\" — must preserve",
category: "epistemic",
base_text: "Apparently three containers went missing from the last shipment and nobody noticed until the audit.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 402,
},
TestCase {
label: "\"I think\" + \"might have\" — two markers",
category: "epistemic",
base_text: "I think the wiring might have been rerouted during the last maintenance cycle without anyone logging it.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 403,
},
TestCase {
label: "\"supposedly\" — hedged rumor",
category: "epistemic",
base_text: "Supposedly the company is cutting the night shift entirely after the audit results come back.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Nervous),
seed: 404,
},
// === CATEGORY: Occasional injection (oath) ===
// seed 7 fires oath for krenn (frequency 0.25, ChaCha8)
TestCase {
label: "oath should fire (seed 7)",
category: "injection",
base_text: "The replacement parts never showed up and now we're a full shift behind schedule.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 7,
},
TestCase {
label: "oath suppressed by Guarded tell",
category: "injection",
base_text: "The replacement parts never showed up and now we're a full shift behind schedule.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Guarded),
seed: 7,
},
TestCase {
label: "oath suppressed by Friendly tell",
category: "injection",
base_text: "The replacement parts never showed up and now we're a full shift behind schedule.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Friendly),
seed: 7,
},
// === CATEGORY: Behavior descriptions ===
TestCase {
label: "behavior — simple action",
category: "behavior",
base_text: "checks the pressure gauge and writes in a logbook",
content_type: ContentType::Behavior,
tell_state: None,
seed: 500,
},
TestCase {
label: "behavior — multi-step action",
category: "behavior",
base_text: "opens a maintenance panel, inspects the wiring, shakes head, then reseals the panel without making changes",
content_type: ContentType::Behavior,
tell_state: None,
seed: 501,
},
TestCase {
label: "behavior — subtle body language",
category: "behavior",
base_text: "glances toward the corridor, hesitates, then continues working",
content_type: ContentType::Behavior,
tell_state: Some(TellCategory::Nervous),
seed: 502,
},
TestCase {
label: "behavior — routine",
category: "behavior",
base_text: "sweeps the floor of the cargo bay in a slow, methodical pattern",
content_type: ContentType::Behavior,
tell_state: None,
seed: 503,
},
// === CATEGORY: Edge cases ===
TestCase {
label: "named entities — must preserve names",
category: "edge",
base_text: "Kael told me that Dren Voss moved the shipment to Bay Seven without logging it.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 600,
},
TestCase {
label: "technology terms — insert, span gate",
category: "edge",
base_text: "The insert feed has been glitching since they updated the span gate routing tables last cycle.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 601,
},
TestCase {
label: "emotional content + no tell (flat delivery)",
category: "edge",
base_text: "My partner didn't come home last night and nobody on the station will tell me what happened.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 602,
},
TestCase {
label: "Friendly tell + bad news (contradiction test)",
category: "edge",
base_text: "The contract fell through and we're going to lose half the crew by end of quarter.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Friendly),
seed: 603,
},
TestCase {
label: "question form — NPC asking",
category: "edge",
base_text: "Have you seen the shift roster for next week? I can't find it anywhere.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 604,
},
TestCase {
label: "imperative — NPC giving instruction",
category: "edge",
base_text: "Check the seals on bay four before you clock out. Last person forgot and we had a pressure drop.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 605,
},
TestCase {
label: "long + Angry — emotional pressure on length",
category: "edge",
base_text: "I've filed three reports about the ventilation in section nine and every single time the response \
comes back saying there's no budget, while they just spent half a million credits refitting the \
executive lounge on deck two.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Angry),
seed: 606,
},
TestCase {
label: "epistemic + tell — compound modifiers",
category: "edge",
base_text: "I heard the safety team might have flagged section twelve but I think somebody pulled the report \
before it went up the chain.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Nervous),
seed: 607,
},
// === CATEGORY: Paula's targeted tests (round 3) ===
// #32 without Angry — does neutral preserve more facts?
TestCase {
label: "long + neutral (compare to #32 Angry)",
category: "paula",
base_text: "I've filed three reports about the ventilation in section nine and every single time the response \
comes back saying there's no budget, while they just spent half a million credits refitting the \
executive lounge on deck two.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 606,
},
// Specific numbers — must preserve quantities
TestCase {
label: "specific numbers — 14 crates, bay 7",
category: "paula",
base_text: "We counted 14 crates in bay seven but the manifest says 18, so four are missing somewhere between \
the loading dock and here.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 700,
},
// Causal chain — A caused B caused C
TestCase {
label: "causal chain — must preserve cause-effect",
category: "paula",
base_text: "The pressure seal on bulkhead nine failed because Torren skipped the inspection, which caused \
the atmosphere leak that put three people in medical.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 701,
},
// Multiple named entities in a relationship
TestCase {
label: "named entities — Sira, Dren, relationship",
category: "paula",
base_text: "Sira told Dren about the missing cargo but Dren went straight to the shift lead instead of \
reporting it through proper channels.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 702,
},
// Specific time reference — must not shift tense
TestCase {
label: "past event — must stay past tense",
category: "paula",
base_text: "Last Thursday the cooling system failed for six hours and we lost two full batches of \
pharmaceutical stock worth about forty thousand credits.",
content_type: ContentType::Dialogue,
tell_state: None,
seed: 703,
},
// Contradiction/denial — NPC denying something
TestCase {
label: "denial — must preserve what is denied",
category: "paula",
base_text: "I wasn't anywhere near section twelve that night and whoever said they saw me there is either \
confused or lying.",
content_type: ContentType::Dialogue,
tell_state: Some(TellCategory::Guarded),
seed: 704,
},
]
}
// ---------------------------------------------------------------------------
// Test runner
// ---------------------------------------------------------------------------
#[test]
#[ignore] // Run explicitly: cargo test --test voice_quality_batch -- --ignored --nocapture
fn voice_quality_batch() {
let out = output_dir();
let cache_dir = out.join("quality-cache");
let _ = std::fs::remove_dir_all(&cache_dir);
std::fs::create_dir_all(&cache_dir).expect("failed to create cache dir");
let config = voice_config();
let mut cultures = BTreeMap::new();
cultures.insert("krenn".to_string(), krenn_culture());
let cultures = Arc::new(cultures);
let cases = test_cases();
let total = cases.len();
let cache = Arc::new(Mutex::new(VoiceCacheStore::new(
cache_dir.clone(),
99999,
"quality-v1".into(),
"quality-i1".into(),
)));
let queue = VoiceQueue::new();
let paused = queue.paused_flag();
let mut pool = WorkerPool::spawn(
2,
&config,
queue.receiver(),
Arc::clone(&cache),
Arc::clone(&cultures),
paused,
);
eprintln!("Quality batch: {} test cases, 2 workers", total);
// Submit all requests — use zone_id = npc_stable_id = index for easy lookup
for (i, case) in cases.iter().enumerate() {
let req = VoiceRequest {
priority: Priority::High,
npc_stable_id: i as u64,
zone_id: 1, // all same zone for cache lookup
culture_id: "krenn".into(),
base_text: case.base_text.into(),
content_type: case.content_type,
content_index: case.tell_state.map(|t| t as u16).unwrap_or(0),
tell_state: case.tell_state,
seed: case.seed,
};
assert!(queue.submit(req), "queue rejected request {}", i);
}
eprintln!("Submitted {} requests", total);
// Wait for all to complete
let start = Instant::now();
let max_wait = Duration::from_secs(300); // 5 min for ~35 prompts at ~4s each
loop {
std::thread::sleep(Duration::from_millis(500));
let pending = queue.pending_count();
let active = pool.active_workers();
if pending == 0 && active == 0 {
break;
}
if start.elapsed() > max_wait {
eprintln!(
"TIMEOUT after {}s (pending={}, active={})",
max_wait.as_secs(),
pending,
active
);
break;
}
}
let elapsed = start.elapsed();
eprintln!("Pipeline drained in {:.1}s", elapsed.as_secs_f64());
pool.shutdown();
// Collect results
let mut cache_guard = cache.lock().unwrap();
let zone_cache = cache_guard.zone_cache(1);
let mut output = String::new();
output.push_str("Voice Quality Batch Results (Spike 2)\n");
output.push_str("=====================================\n");
output.push_str(&format!(
"Model: Gemma 2B Q4_K_M | Workers: 2 | Time: {:.1}s\n",
elapsed.as_secs_f64()
));
output.push_str(&format!(
"Test cases: {} | Cache entries: {}\n\n",
total,
zone_cache.len()
));
let mut current_category = "";
for (i, case) in cases.iter().enumerate() {
if case.category != current_category {
current_category = case.category;
let sep = "=".repeat(60);
output.push_str(&format!(
"\n{}\n== CATEGORY: {} ==\n{}\n\n",
sep, current_category, sep
));
}
let key = CacheKey {
culture_id: "krenn".into(),
npc_stable_id: i as u64,
content_type: case.content_type,
content_index: case.tell_state.map(|t| t as u16).unwrap_or(0),
tell_state: case.tell_state,
};
let voiced = zone_cache
.entries
.get(&key)
.map(|s| s.as_str())
.unwrap_or("[MISSING — not cached]");
output.push_str(&format!("--- #{}: {} ---\n", i + 1, case.label));
output.push_str(&format!(" tell: {:?} | seed: {} | type: {:?}\n", case.tell_state, case.seed, case.content_type));
output.push_str(&format!(" BASE: {}\n", case.base_text));
output.push_str(&format!(" VOICED: {}\n\n", voiced));
}
drop(cache_guard);
cache.lock().unwrap().save_all().expect("failed to save cache");
let results_path = out.join("quality-batch.txt");
std::fs::write(&results_path, &output).expect("failed to write results");
eprintln!("Results written to {}", results_path.display());
eprintln!("\n{}", output);
// Basic sanity: we should have cached something for every test case
let mut cache_guard = cache.lock().unwrap();
let zone_cache = cache_guard.zone_cache(1);
assert!(
zone_cache.len() >= total - 1, // allow 1 miss for flaky edge cases
"Expected at least {} cache entries, got {}",
total - 1,
zone_cache.len()
);
}