jpmschweitzerandClaude Opus 4.6
e93a9e8b70
Critical fixes: - Pause mechanism: workers now hold requests during pause instead of dropping them. Queue and worker pool share the same AtomicBool flag via VoiceQueue::paused_flag(). Submit() rejects while paused. - Seed type: sr-voice accepts u64 seeds over IPC (explicit u32 truncation for llama.cpp sampler, documented). Warning fixes: - HashMap → BTreeMap in cache.rs and worker.rs (D-010 determinism mandate). Added Ord derives to CacheKey, ContentType, TellCategory. - VoicePipe::generate() watchdog kills child after 120s timeout to prevent indefinite blocking on read_line. - VoiceCacheStore Drop impl calls save_all() on shutdown. - trait-modifiers.ron: fixed 3 wrong trait names (Impulsive→Compassionate, Methodical→Incurious, Stubborn→Ruthless) to match PersonalityTrait enum. Suggestion fixes: - Worker spawn: log error + reduce pool instead of panic on thread failure. - on_battery(): added macOS detection via pmset. - Epistemic markers: lowercased constants, removed redundant to_lowercase(). - cache.rs: documented non-atomic write tradeoff. - queue.rs: reprioritize() bypasses pause check (it runs during pause). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
72 lines
2.3 KiB
Rust
72 lines
2.3 KiB
Rust
//! Stdio JSONL mode for sr-voice (D-138, Spike 2).
|
|
//!
|
|
//! Reads one JSON object per line from stdin, runs inference, writes one JSON
|
|
//! object per line to stdout. No network port is opened — the model is only
|
|
//! reachable through the parent process's pipe (Gemma 2 T&C compliance).
|
|
//!
|
|
//! Request format: {"prompt": "...", "seed": 42}
|
|
//! Response format: {"text": "...", "tokens_generated": N, ...}
|
|
//! or {"error": "..."}
|
|
|
|
use std::io::{self, BufRead, Write};
|
|
|
|
use crate::inference::InferenceEngine;
|
|
|
|
const MAX_TOKENS: u32 = 64;
|
|
const TEMPERATURE: f32 = 0.7;
|
|
const TOP_P: f32 = 0.9;
|
|
|
|
#[derive(serde::Deserialize)]
|
|
struct StdioRequest {
|
|
prompt: String,
|
|
seed: Option<u64>,
|
|
}
|
|
|
|
pub fn run_stdio(engine: InferenceEngine) -> Result<(), Box<dyn std::error::Error>> {
|
|
let stdin = io::stdin().lock();
|
|
let mut stdout = io::stdout().lock();
|
|
|
|
for line in stdin.lines() {
|
|
let line = match line {
|
|
Ok(l) => l,
|
|
Err(e) => {
|
|
eprintln!("stdin read error: {}", e);
|
|
break;
|
|
}
|
|
};
|
|
|
|
if line.trim().is_empty() {
|
|
continue;
|
|
}
|
|
|
|
let response = match serde_json::from_str::<StdioRequest>(&line) {
|
|
Ok(req) => {
|
|
eprintln!(" stdio: {} chars", req.prompt.len());
|
|
// Truncate u64 seed to u32 for llama.cpp sampler (deterministic
|
|
// within same process, seed space reduction is acceptable).
|
|
match engine.generate(&req.prompt, MAX_TOKENS, TEMPERATURE, TOP_P, req.seed.map(|s| s as u32)) {
|
|
Ok(result) => {
|
|
eprintln!(
|
|
" -> {} tokens, {:.1} t/s",
|
|
result.tokens_generated, result.tokens_per_sec
|
|
);
|
|
serde_json::to_string(&result).unwrap()
|
|
}
|
|
Err(e) => {
|
|
serde_json::json!({"error": e.to_string()}).to_string()
|
|
}
|
|
}
|
|
}
|
|
Err(e) => {
|
|
serde_json::json!({"error": format!("invalid JSON: {}", e)}).to_string()
|
|
}
|
|
};
|
|
|
|
writeln!(stdout, "{}", response)?;
|
|
stdout.flush()?;
|
|
}
|
|
|
|
eprintln!("sr-voice stdio mode — stdin closed, exiting");
|
|
Ok(())
|
|
}
|