diff --git a/.claude/skills/audio-gen/SKILL.md b/.claude/skills/audio-gen/SKILL.md index 69459be32..ef6a17526 100644 --- a/.claude/skills/audio-gen/SKILL.md +++ b/.claude/skills/audio-gen/SKILL.md @@ -27,13 +27,19 @@ workflow, and quality validation. # Check API health db/connectors/audio-health -# Generate audio +# Generate a single asset (WAV only) db/connectors/audio-generate "prompt text" \ - --duration 10 \ - --steps 100 \ - --cfg 7 \ - --output path/to/output.wav \ - --timeout 600 + --duration 10 --steps 100 --cfg 7 \ + --output path/to/output.wav + +# Generate + post-process in one command (WAV → trim → normalize → OGG) +db/connectors/audio-generate "prompt text" \ + --duration 10 --steps 100 --cfg 7 \ + --output path/to/gen/intermediate.wav \ + --output-ogg client/assets/audio/final.ogg + +# Batch-generate from a manifest (preferred for multiple assets) +db/connectors/audio-batch docs/assets/audio/batch-s10-327.json ``` ### Parameters @@ -43,7 +49,9 @@ db/connectors/audio-generate "prompt text" \ | `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. | | `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. | | `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. | -| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. | +| `--output` | auto | — | Output WAV file path. Auto-names from prompt if omitted. | +| `--post` | off | — | Run trim + normalize + convert after generation. | +| `--output-ogg` | auto | — | OGG output path (implies `--post`). Defaults to same basename as WAV. | | `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. | ### Critical Constraints @@ -55,8 +63,6 @@ db/connectors/audio-generate "prompt text" \ Be patient. The timeout default (600s) is generous. - **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s clips and crossfade-stitch in post-processing. -- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import: - `ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg` ## Prompt Assembly @@ -74,19 +80,121 @@ family prefix and matching category template. asset type (ambient, sfx, ui) - **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md` +## Batch Workflow (Preferred) + +For generating multiple assets, use a manifest file. This reduces prompt +approvals to 2: one Write (manifest) + one Bash (batch run). + +### 1. Create the manifest + +Write a JSON manifest to `docs/assets/audio/batch-{sprint}-{ticket}.json`: + +```json +{ + "description": "Sprint 10 ambient + world SFX batch", + "output_dir": "client/assets/audio", + "gen_dir": "client/assets/audio/gen", + "defaults": { + "steps": 100, + "cfg": 7, + "lufs": -16, + "quality": 6 + }, + "assets": [ + { + "id": "AMB-001", + "filename": "amb_station_base.ogg", + "method": "sao", + "duration": 45, + "steps": 150, + "cfg": 5, + "prompt": "[sonic family prefix] + [template] + [description]" + }, + { + "id": "UI-005", + "filename": "sfx_monologue_chime.ogg", + "method": "synth", + "synth": { + "type": "harmonic", + "duration": 0.8, + "fundamental": 1200, + "harmonics": [ + {"freq": 2400, "db": -12}, + {"freq": 3600, "db": -24} + ], + "attack_ms": 15, + "sustain_ratio": 0.2, + "decay": "exponential" + } + } + ] +} +``` + +Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g., +AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory. + +### 2. Run the batch + +```bash +# Full run +db/connectors/audio-batch docs/assets/audio/batch-s10-327.json + +# Dry run — preview what would be generated +db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --dry-run + +# Generate only specific assets +db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --only AMB-001,AMB-002 + +# Skip assets that already have OGG files +db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --skip-existing +``` + +### 3. Update asset docs with prompts + +After the batch completes, write the exact prompts used back into the +Prompt/Notes column of `docs/assets/audio/{category}.md`. The manifest records +what was generated; the asset docs record what we have. + +### Manifest fields + +| Field | Required | Notes | +|-------|----------|-------| +| `id` | yes | Asset ID from docs (AMB-001, SFX-002, UI-005) | +| `filename` | yes | Output filename (must match asset doc) | +| `method` | yes | `sao` (Stable Audio Open) or `synth` (harmonic synthesis) | +| `duration` | SAO only | Duration in seconds | +| `prompt` | SAO only | Full assembled prompt | +| `steps` | no | Override default steps | +| `cfg` | no | Override default CFG | +| `synth` | synth only | Synthesis parameters (see below) | + +### Synth parameters + +| Field | Default | Notes | +|-------|---------|-------| +| `type` | harmonic | Only `harmonic` supported currently | +| `duration` | — | Duration in seconds | +| `fundamental` | — | Fundamental frequency in Hz | +| `harmonics` | [] | List of `{"freq": Hz, "db": dB}` objects | +| `attack_ms` | 10 | Attack time in milliseconds | +| `sustain_ratio` | 0.2 | Fraction of duration at full level before decay | +| `decay` | exponential | `exponential` or `linear` | + ## Single Asset Workflow +For one-off generation or iteration on a specific asset: + 1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename, duration, bus, method, and design intent. 2. Read `references/sonic-palette.md` for the sonic family prefix. 3. Read `references/category-templates.md` for the matching template. 4. Assemble the full prompt. 5. Run `db/connectors/audio-health` to verify the API is up. -6. Run `db/connectors/audio-generate` with the assembled prompt. **One request - at a time. Wait for completion.** -7. Listen to the output (or describe it based on file size/duration). -8. If acceptable, convert to .ogg and place in `client/assets/audio/`. -9. Update the asset status in `docs/assets/audio/{category}.md`. +6. Run `db/connectors/audio-generate` with `--post` or `--output-ogg` to + generate and post-process in one step. +7. Verify the output (file size, duration). +8. Update the asset status and prompt in `docs/assets/audio/{category}.md`. ## Iteration Workflow @@ -101,62 +209,38 @@ For each asset, generate 4-6 candidates: 4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a jarring repeat? 5. **Close-your-eyes test** — does it create a mental image or sensation? -6. Select the best candidate, trim, normalize, convert. +6. Select the best candidate (post-processing is already done if `--post` was + used). -## Post-Processing +## Post-Processing (Standalone) -After selecting the best generation: +If you need to post-process separately (e.g., re-normalizing an existing file): ```bash -# Trim silence from start/end -ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav +# Full pipeline: trim → normalize → convert +db/connectors/audio-post pipeline input.wav --output output.ogg -# LUFS normalize to -16 LUFS (broadcast standard, good for game audio) -ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav - -# Convert to .ogg for Godot -ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg - -# For loops: verify loop point -ffplay -loop 0 output.ogg -``` - -For ambient loops, create crossfade overlap: -```bash -# Create a 45s loop with 3s crossfade overlap -# (manual: export 48s, crossfade first 3s with last 3s in Audacity) +# Individual steps +db/connectors/audio-post trim input.wav +db/connectors/audio-post normalize input.wav --lufs -16 +db/connectors/audio-post convert input.wav --output output.ogg ``` ## Manual Synthesis (Insert-Tech Sounds) For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot -produce meaningful output. Use manual synthesis instead: +produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py` +or the batch manifest's `method: "synth"` with harmonic parameters. -```python -# Example: 50ms cursor hover tick -import numpy as np -import wave - -sr = 44100 -duration = 0.05 # 50ms -t = np.linspace(0, duration, int(sr * duration), endpoint=False) -freq = 3200 # Hz -signal = np.sin(2 * np.pi * freq * t) -envelope = np.exp(-t * 80) # exponential decay -audio = (signal * envelope * 32767).astype(np.int16) - -with wave.open("cursor_hover.wav", "w") as f: - f.setnchannels(1) - f.setsampwidth(2) - f.setframerate(sr) - f.writeframes(audio.tobytes()) -``` +For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass +impulse), write a custom script in `tooling/` following the pattern in +`tooling/synth_ui_sounds.py`. ## Quality Checklist After generating, verify: - Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural) -- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md) +- Frequency range doesn't mask other layers (check docs/assets/audio/) - Duration matches spec - No unwanted artifacts (clicks, pops, digital noise at start/end) - Loop point is clean (ambient loops only) @@ -166,22 +250,6 @@ After generating, verify: ## File Placement Generated assets go to `client/assets/audio/` with exact filenames from the -asset docs: +asset docs. Intermediates go to `client/assets/audio/gen/` (gitignored). -``` -client/assets/audio/ - amb_station_base.ogg # Ambient bus - amb_workplace_layer.ogg # Ambient bus - amb_bar_layer.ogg # Ambient bus - amb_corridor_layer.ogg # Ambient bus - sfx_footstep_metal.ogg # Player Actions bus - sfx_footstep_metal_run.ogg # Player Actions bus - cursor_hover.ogg # UI Sounds bus - implant_open.ogg # UI Sounds bus - fog_recognition.ogg # UI Sounds bus - weapon_aim.ogg # UI Sounds bus - sfx_monologue_chime.ogg # UI Sounds bus - sfx_monologue_chime_urgent.ogg # UI Sounds bus -``` - -AudioManager discovers these by directory scan — filenames must match exactly. +AudioManager discovers assets by directory scan — filenames must match exactly. diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a500126d..635a47822 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,12 +25,17 @@ Format based on [Keep a Changelog](https://keepachangelog.com/). - Mouse-relative facing and movement (#526, D-054) — mouse position determines facing direction (client-side float), WASD remapped to cursor-relative (W=toward, S=away, A/D=strafe), SET_FACING action sends octant to server, smooth facing indicator rotation - Room reset client UX (#502) — amber reset_plate tile type, 0.15s screen flash on room reset, 'Reset Room' interaction verb - Auto-checklist progress tracking (#503) — ChecklistEvaluator parses room YAML and evaluates 7 condition types against GameState with latching, ChecklistOverlay renders progress in gauntlet mode only, 48 new tests +- 4 ambient zone loops: station base, workplace, bar, corridor — SAO-generated organic soundscape with crossfade loop points (#327) +- 2 footstep SFX: metal walk and run — SAO hybrid with best-transient extraction (#327) +- `audio-batch` command — batch audio generation from JSON manifests, supports SAO and harmonic synthesis, with `--dry-run`, `--only`, and `--skip-existing` flags +- `--post` and `--output-ogg` flags on `audio-generate` — chain post-processing (trim, normalize, convert) into a single command ### Changed - `push-pr` skill now runs `/commit` first when uncommitted changes are detected - Insert open/close now sends explicit PauseSimulation/ResumeSimulation (#518, D-058) — replaces toggle-style pause with idempotent pair - Interaction list colors reference Constants.IMPLANT_TEXT_COLOR instead of hardcoded values - World radial menu uses theme font instead of ThemeDB.fallback_font +- Monologue chimes replaced with production-quality manual synthesis — insert-tech aesthetic per D-074, pure sine harmonics with mathematical envelopes (#327) ### Fixed - Bidirectional relationship check (#515) — Check 9 tested `target in npc_rels` which missed NPCs with no relationship entries; changed to `target in self.npcs` diff --git a/client/assets/audio/amb_bar_layer.ogg b/client/assets/audio/amb_bar_layer.ogg new file mode 100644 index 000000000..e57d3be93 Binary files /dev/null and b/client/assets/audio/amb_bar_layer.ogg differ diff --git a/client/assets/audio/amb_corridor_layer.ogg b/client/assets/audio/amb_corridor_layer.ogg new file mode 100644 index 000000000..1a8b3e262 Binary files /dev/null and b/client/assets/audio/amb_corridor_layer.ogg differ diff --git a/client/assets/audio/amb_station_base.ogg b/client/assets/audio/amb_station_base.ogg new file mode 100644 index 000000000..35115eee8 Binary files /dev/null and b/client/assets/audio/amb_station_base.ogg differ diff --git a/client/assets/audio/amb_workplace_layer.ogg b/client/assets/audio/amb_workplace_layer.ogg new file mode 100644 index 000000000..1654dad5c Binary files /dev/null and b/client/assets/audio/amb_workplace_layer.ogg differ diff --git a/client/assets/audio/sfx_footstep_metal_run.ogg b/client/assets/audio/sfx_footstep_metal_run.ogg new file mode 100644 index 000000000..2e8f4e9a7 Binary files /dev/null and b/client/assets/audio/sfx_footstep_metal_run.ogg differ diff --git a/client/assets/audio/sfx_footstep_metal_walk.ogg b/client/assets/audio/sfx_footstep_metal_walk.ogg new file mode 100644 index 000000000..2bdbbd026 Binary files /dev/null and b/client/assets/audio/sfx_footstep_metal_walk.ogg differ diff --git a/client/assets/audio/sfx_monologue_chime.ogg b/client/assets/audio/sfx_monologue_chime.ogg index a638d4621..5eaa29ba5 100644 Binary files a/client/assets/audio/sfx_monologue_chime.ogg and b/client/assets/audio/sfx_monologue_chime.ogg differ diff --git a/client/assets/audio/sfx_monologue_chime_urgent.ogg b/client/assets/audio/sfx_monologue_chime_urgent.ogg index 026e780de..b9a0f1742 100644 Binary files a/client/assets/audio/sfx_monologue_chime_urgent.ogg and b/client/assets/audio/sfx_monologue_chime_urgent.ogg differ diff --git a/db/connectors/audio-batch b/db/connectors/audio-batch new file mode 100755 index 000000000..ba8468a0c --- /dev/null +++ b/db/connectors/audio-batch @@ -0,0 +1,4 @@ +#!/usr/bin/env bash +# Batch audio generation from a manifest file. Whitelistable command. +# Usage: audio-batch manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing] +exec python3 "$(dirname "$0")/audio_batch.py" "$@" diff --git a/db/connectors/audio_batch.py b/db/connectors/audio_batch.py new file mode 100644 index 000000000..419bce4f6 --- /dev/null +++ b/db/connectors/audio_batch.py @@ -0,0 +1,309 @@ +#!/usr/bin/env python3 +"""Batch audio generation from a manifest file. + +Processes multiple assets sequentially: SAO generation or harmonic synthesis, +followed by post-processing (trim, normalize, convert to OGG). + +Usage: + python3 audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing] +""" + +import json +import math +import os +import struct +import subprocess +import sys +import wave + +import numpy as np + + +def load_manifest(path): + with open(path) as f: + return json.load(f) + + +def resolve_paths(manifest, manifest_dir): + """Resolve output_dir and gen_dir relative to the git root.""" + # Find git root by walking up from manifest_dir + # Check for .git as file (worktree) or directory (regular repo) + git_root = manifest_dir + while git_root != "/": + if os.path.exists(os.path.join(git_root, ".git")): + break + git_root = os.path.dirname(git_root) + else: + git_root = manifest_dir + + output_dir = os.path.join(git_root, manifest.get("output_dir", "client/assets/audio")) + gen_dir = os.path.join(git_root, manifest.get("gen_dir", "client/assets/audio/gen")) + os.makedirs(output_dir, exist_ok=True) + os.makedirs(gen_dir, exist_ok=True) + return output_dir, gen_dir, git_root + + +def get_default(manifest, asset, key): + """Get a value from the asset, falling back to manifest defaults.""" + defaults = manifest.get("defaults", {}) + return asset.get(key, defaults.get(key)) + + +def run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir): + """Generate audio via Stable Audio Open + post-processing.""" + filename = asset["filename"] + base_name = os.path.splitext(filename)[0] + wav_path = os.path.join(gen_dir, base_name + ".wav") + ogg_path = os.path.join(output_dir, filename) + + prompt = asset["prompt"] + duration = asset.get("duration", 10) + steps = get_default(manifest, asset, "steps") or 100 + cfg = get_default(manifest, asset, "cfg") or 7 + timeout = get_default(manifest, asset, "timeout") or 600 + + # Run audio-generate with --post + cmd = [ + sys.executable, os.path.join(script_dir, "audio_connector.py"), + "generate", prompt, + "--duration", str(duration), + "--steps", str(steps), + "--cfg", str(cfg), + "--output", wav_path, + "--output-ogg", ogg_path, + "--timeout", str(timeout), + ] + + result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout + 60) + if result.returncode != 0: + stderr = result.stderr.strip() + try: + err = json.loads(result.stdout) + return {"ok": False, "error": err.get("error", stderr)} + except (json.JSONDecodeError, ValueError): + return {"ok": False, "error": stderr or "generation failed"} + + try: + return json.loads(result.stdout) + except json.JSONDecodeError: + return {"ok": False, "error": f"Unexpected output: {result.stdout[:200]}"} + + +def synthesize_harmonic(params, wav_path): + """Synthesize audio from harmonic parameters.""" + sr = 44100 + duration = params["duration"] + fundamental = params["fundamental"] + harmonics = params.get("harmonics", []) + attack_ms = params.get("attack_ms", 10) + sustain_ratio = params.get("sustain_ratio", 0.2) + decay = params.get("decay", "exponential") + + n = int(sr * duration) + t = np.linspace(0, duration, n, endpoint=False) + + # Fundamental + signal = np.sin(2 * np.pi * fundamental * t) + + # Add harmonics + for h in harmonics: + freq = h["freq"] + db = h["db"] + amplitude = 10 ** (db / 20) + signal = signal + amplitude * np.sin(2 * np.pi * freq * t) + + # Envelope: attack + sustain + decay + attack_s = attack_ms / 1000 + attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n) + + sustain_end = duration * sustain_ratio + if decay == "exponential": + # Decay rate: reach -60dB by end of duration + decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10 + decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end))) + else: + # Linear decay + decay_env = np.where(t < sustain_end, 1.0, + 1.0 - (t - sustain_end) / (duration - sustain_end)) + + envelope = attack_env * decay_env + signal = signal * envelope + + # Normalize to peak + peak = np.max(np.abs(signal)) + if peak > 0: + signal = signal / peak * 0.9 + + # Write WAV + int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16) + with wave.open(wav_path, "w") as f: + f.setnchannels(1) + f.setsampwidth(2) + f.setframerate(sr) + f.writeframes(int_samples.tobytes()) + + return wav_path + + +def run_synth(asset, manifest, gen_dir, output_dir, script_dir): + """Synthesize audio from harmonic parameters + post-process.""" + filename = asset["filename"] + base_name = os.path.splitext(filename)[0] + wav_path = os.path.join(gen_dir, base_name + "_synth.wav") + ogg_path = os.path.join(output_dir, filename) + + synth_params = asset.get("synth") + if not synth_params: + return {"ok": False, "error": "No synth parameters provided"} + + synth_type = synth_params.get("type", "harmonic") + if synth_type != "harmonic": + return {"ok": False, "error": f"Unknown synth type: {synth_type}"} + + try: + synthesize_harmonic(synth_params, wav_path) + except Exception as e: + return {"ok": False, "error": f"Synthesis failed: {e}"} + + # Post-process: normalize + convert (skip trim for synth — no silence to trim) + post_script = os.path.join(script_dir, "audio_post.py") + lufs = get_default(manifest, asset, "lufs") or -16 + quality = get_default(manifest, asset, "quality") or 6 + + # Normalize + norm_path = os.path.join(gen_dir, base_name + "_norm.wav") + cmd = [sys.executable, post_script, "normalize", wav_path, "--output", norm_path, + "--lufs", str(lufs)] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + return {"ok": False, "error": f"Normalize failed: {result.stderr.strip()}"} + + # Convert to OGG + cmd = [sys.executable, post_script, "convert", norm_path, "--output", ogg_path, + "--quality", str(quality)] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + return {"ok": False, "error": f"Convert failed: {result.stderr.strip()}"} + + # Clean up intermediate + try: + os.remove(norm_path) + except OSError: + pass + + ogg_size = os.path.getsize(ogg_path) + return { + "ok": True, + "file": wav_path, + "ogg_file": ogg_path, + "ogg_size_bytes": ogg_size, + "synth_params": synth_params, + "post_processed": True, + } + + +def main(): + if len(sys.argv) < 2: + print("Usage: audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]", + file=sys.stderr) + sys.exit(1) + + manifest_path = sys.argv[1] + dry_run = "--dry-run" in sys.argv + skip_existing = "--skip-existing" in sys.argv + + only_ids = None + for i, arg in enumerate(sys.argv): + if arg == "--only" and i + 1 < len(sys.argv): + only_ids = set(sys.argv[i + 1].split(",")) + + manifest = load_manifest(manifest_path) + manifest_dir = os.path.dirname(os.path.abspath(manifest_path)) + output_dir, gen_dir, git_root = resolve_paths(manifest, manifest_dir) + script_dir = os.path.dirname(os.path.abspath(__file__)) + + assets = manifest.get("assets", []) + if only_ids: + assets = [a for a in assets if a["id"] in only_ids] + + # Health check if any SAO assets + sao_assets = [a for a in assets if a.get("method") == "sao"] + if sao_assets and not dry_run: + print(f"Checking SAO API health...", file=sys.stderr) + health_cmd = [sys.executable, os.path.join(script_dir, "audio_connector.py"), "health"] + result = subprocess.run(health_cmd, capture_output=True, text=True) + if result.returncode != 0: + print(json.dumps({"ok": False, "error": "SAO API health check failed", + "details": result.stdout.strip()})) + sys.exit(1) + print(f" SAO API is up.", file=sys.stderr) + + total = len(assets) + results = [] + success = 0 + failed = 0 + skipped = 0 + + print(f"Processing {total} assets from {os.path.basename(manifest_path)}...", file=sys.stderr) + if dry_run: + print(" (dry run — no generation will occur)", file=sys.stderr) + + for i, asset in enumerate(assets, 1): + asset_id = asset["id"] + filename = asset["filename"] + method = asset.get("method", "sao") + + print(f"\n[{i}/{total}] {asset_id}: {filename} ({method})", file=sys.stderr) + + if skip_existing: + ogg_path = os.path.join(output_dir, filename) + if os.path.exists(ogg_path): + print(f" Skipping — already exists", file=sys.stderr) + results.append({"id": asset_id, "status": "skipped", "reason": "exists"}) + skipped += 1 + continue + + if dry_run: + print(f" Would generate: {filename}", file=sys.stderr) + if method == "sao": + print(f" Prompt: {asset.get('prompt', '(none)')[:80]}...", file=sys.stderr) + elif method == "synth": + synth = asset.get("synth", {}) + print(f" Synth: {synth.get('fundamental')}Hz, {synth.get('duration')}s", + file=sys.stderr) + results.append({"id": asset_id, "status": "dry_run"}) + continue + + if method == "sao": + result = run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir) + elif method == "synth": + result = run_synth(asset, manifest, gen_dir, output_dir, script_dir) + else: + result = {"ok": False, "error": f"Unknown method: {method}"} + + result["id"] = asset_id + if result.get("ok"): + success += 1 + result["status"] = "success" + print(f" OK → {result.get('ogg_file', filename)}", file=sys.stderr) + else: + failed += 1 + result["status"] = "failed" + print(f" FAILED: {result.get('error', 'unknown')}", file=sys.stderr) + + results.append(result) + + # Summary + summary = { + "ok": failed == 0, + "total": total, + "success": success, + "failed": failed, + "skipped": skipped, + "results": results, + } + print(json.dumps(summary, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/db/connectors/audio_connector.py b/db/connectors/audio_connector.py index 1323a8e70..5e01260ca 100644 --- a/db/connectors/audio_connector.py +++ b/db/connectors/audio_connector.py @@ -6,12 +6,13 @@ Talks to the Stable Audio Open Gradio app at tower-of-joy:11500. Uses the async Gradio API pattern: POST to submit, SSE stream for results. Usage: - python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav] + python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav] [--post] python3 audio_connector.py health """ import json import os +import subprocess import sys import time import urllib.error @@ -56,7 +57,31 @@ def health(): }, indent=2)) sys.exit(1) -def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600): +def post_process(wav_path, ogg_path=None, lufs=-16, quality=6, threshold=-50): + """Run trim + normalize + convert on a WAV file via audio-post pipeline.""" + script_dir = os.path.dirname(os.path.abspath(__file__)) + post_script = os.path.join(script_dir, "audio_post.py") + if ogg_path is None: + ogg_path = os.path.splitext(wav_path)[0] + ".ogg" + cmd = [ + sys.executable, post_script, "pipeline", wav_path, + "--output", ogg_path, + "--lufs", str(lufs), + "--quality", str(quality), + "--threshold", str(threshold), + ] + print(f" Post-processing → {os.path.basename(ogg_path)}...", file=sys.stderr) + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + return {"ok": False, "error": f"Post-processing failed: {result.stderr.strip()}"} + try: + return json.loads(result.stdout) + except json.JSONDecodeError: + return {"ok": True, "output": ogg_path} + + +def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600, + post=False, output_ogg=None): """ Generate audio from a text prompt. @@ -67,6 +92,8 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600 cfg: Classifier-free guidance scale (default 7) output: Output file path (default: auto-named in current directory) timeout: Maximum wait time in seconds (default 600 = 10 minutes) + post: If True, run trim+normalize+convert after generation + output_ogg: OGG output path when post=True (default: same basename .ogg) """ base = get_base_url() api_url = f"{base}/gradio_api/call/generate_audio" @@ -216,7 +243,7 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600 shutil.copyfileobj(resp, f) file_size = os.path.getsize(output) - print(json.dumps({ + result_json = { "ok": True, "file": output, "size_bytes": file_size, @@ -225,7 +252,20 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600 "cfg": cfg, "prompt": prompt, "generation_time_s": elapsed - }, indent=2)) + } + + if post: + post_result = post_process(output, ogg_path=output_ogg) + if not post_result.get("ok"): + result_json["post_processed"] = False + result_json["post_error"] = post_result.get("error", "unknown") + else: + result_json["post_processed"] = True + result_json["ogg_file"] = post_result.get("output", output_ogg) + ogg_size = os.path.getsize(result_json["ogg_file"]) + result_json["ogg_size_bytes"] = ogg_size + + print(json.dumps(result_json, indent=2)) except Exception as e: print(json.dumps({ @@ -258,6 +298,8 @@ def main(): cfg = 7.0 output = None timeout = 600 + post = False + output_ogg = None # Parse optional args i = 3 @@ -277,11 +319,19 @@ def main(): elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv): timeout = int(sys.argv[i + 1]) i += 2 + elif sys.argv[i] == "--post": + post = True + i += 1 + elif sys.argv[i] == "--output-ogg" and i + 1 < len(sys.argv): + output_ogg = sys.argv[i + 1] + post = True # --output-ogg implies --post + i += 2 else: print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr) sys.exit(1) - generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, timeout=timeout) + generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, + timeout=timeout, post=post, output_ogg=output_ogg) else: print(f"Unknown command: {cmd}", file=sys.stderr) sys.exit(1) diff --git a/docs/assets/audio/ambient.md b/docs/assets/audio/ambient.md index 7643e2add..45cafbd5e 100644 --- a/docs/assets/audio/ambient.md +++ b/docs/assets/audio/ambient.md @@ -14,10 +14,10 @@ Stable Audio Open primary. Prompt strategy: describe the SPACE, not the sound. | ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint | |----|----------|--------|-----|--------|----------|-------------|--------| -| AMB-001 | `amb_station_base.ogg` | planned | Ambient | SAO | 45-60s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. | S8+ | -| AMB-002 | `amb_workplace_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. | S8+ | -| AMB-003 | `amb_bar_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. | S8+ | -| AMB-004 | `amb_corridor_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. | S8+ | +| AMB-001 | `amb_station_base.ogg` | done | Ambient | SAO | 43.3s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 | +| AMB-002 | `amb_workplace_layer.ogg` | done | Ambient | SAO | 42.9s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 | +| AMB-003 | `amb_bar_layer.ogg` | done | Ambient | SAO | 43.6s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 | +| AMB-004 | `amb_corridor_layer.ogg` | done | Ambient | SAO | 42.7s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 | ## Detailed Entries diff --git a/docs/assets/audio/sfx.md b/docs/assets/audio/sfx.md index c385559ba..58903680e 100644 --- a/docs/assets/audio/sfx.md +++ b/docs/assets/audio/sfx.md @@ -13,8 +13,8 @@ Mixed methods depending on duration and character: | ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint | |----|----------|--------|-----|--------|----------|-------------|--------| -| SFX-001 | `sfx_footstep_metal.ogg` | planned | Player Actions | hybrid | 0.3-0.5s | Single footstep on metal grating. Walk pace. Player character proprioceptive feedback. | S8+ | -| SFX-002 | `sfx_footstep_metal_run.ogg` | planned | Player Actions | hybrid | 0.2-0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). | S8+ | +| SFX-001 | `sfx_footstep_metal_walk.ogg` | done | Player Actions | SAO hybrid | 0.3s | Single footstep on metal grating. Walk pace. SAO 5s/100 steps/CFG 8, best transient trimmed at 2.42s, LUFS -16 normalized. | S10 #327 | +| SFX-002 | `sfx_footstep_metal_run.ogg` | done | Player Actions | SAO hybrid | 0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). SAO 5s/100 steps/CFG 8, best transient trimmed at 1.03s, LUFS -16 normalized. | S10 #327 | ## Detailed Entries diff --git a/docs/assets/audio/ui.md b/docs/assets/audio/ui.md index b222173cf..90d71750e 100644 --- a/docs/assets/audio/ui.md +++ b/docs/assets/audio/ui.md @@ -5,7 +5,7 @@ Interface sounds triggered by player interaction, insert systems, and cognitive ## Generation Approach - **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`. -- **Monologue chimes:** Re-generated in Sprint 9 (#453). Insert-tech prefix + Chimes template, 5s SAO generation trimmed to 0.8s. +- **Monologue chimes:** Replaced in Sprint 10 (#327) with manual synthesis. Insert-tech aesthetic: pure sine harmonics, mathematical envelope, no SAO. Previous S9 SAO versions were acknowledged placeholders per D-038 amendment. ## Assets @@ -15,8 +15,8 @@ Interface sounds triggered by player interaction, insert systems, and cognitive | UI-002 | `implant_open.ogg` | done | UI Sounds | SAO | 217ms | Rising tone when insert UI opens (radial menu, inventory, stance). Neural lattice powering up. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 | | UI-003 | `fog_recognition.ogg` | done | UI Sounds | SAO | 400ms | Warm organic chime at onset of cognitive delay (D-060). Re-generated in S9 — original was silent. Organic prefix + Chimes template. | S7 #440, S9 #453 | | UI-004 | `weapon_aim.ogg` | done | UI Sounds | SAO | 127ms | Harder click/lock for weapon aim state. Mechanical, deliberate. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 | -| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template. Crystalline tone, neural lattice surfacing a thought. See design brief below. | S7 #440, S9 #453 | -| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template (sharper/brighter variant). CFG 8. See design brief below. | S7 #440, S9 #453 | +| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1200Hz fundamental + harmonics (2400Hz -12dB, 3600Hz -24dB, 6000Hz -30dB). 15ms attack, 0.2 sustain ratio, exp decay. Peak -15dB. No SAO — pure mathematical precision. | S7 #440, S9 #453, S10 #327 | +| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1220Hz fundamental (detuned +20Hz for tension) + stronger harmonics (2440Hz -6dB, 3660Hz -18dB, 5087Hz -24dB, inharmonic 5087Hz -28dB). 8ms attack, 0.3 sustain ratio. Peak -6.3dB. | S7 #440, S9 #453, S10 #327 | | UI-007 | `sfx_weapon_aim_lock.ogg` | done | UI Sounds | SAO | 505ms | Sharp targeting lock-on confirmation for weapon aim. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 | | UI-008 | `sfx_stance_change.ogg` | done | UI Sounds | SAO | 349ms | Soft mechanical mode-switch click for stance toggle. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 |