Merge remote-tracking branch 'origin/audio'
This commit is contained in:
@@ -27,13 +27,19 @@ workflow, and quality validation.
|
||||
# Check API health
|
||||
db/connectors/audio-health
|
||||
|
||||
# Generate audio
|
||||
# Generate a single asset (WAV only)
|
||||
db/connectors/audio-generate "prompt text" \
|
||||
--duration 10 \
|
||||
--steps 100 \
|
||||
--cfg 7 \
|
||||
--output path/to/output.wav \
|
||||
--timeout 600
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/output.wav
|
||||
|
||||
# Generate + post-process in one command (WAV → trim → normalize → OGG)
|
||||
db/connectors/audio-generate "prompt text" \
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/gen/intermediate.wav \
|
||||
--output-ogg client/assets/audio/final.ogg
|
||||
|
||||
# Batch-generate from a manifest (preferred for multiple assets)
|
||||
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json
|
||||
```
|
||||
|
||||
### Parameters
|
||||
@@ -43,7 +49,9 @@ db/connectors/audio-generate "prompt text" \
|
||||
| `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. |
|
||||
| `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. |
|
||||
| `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. |
|
||||
| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. |
|
||||
| `--output` | auto | — | Output WAV file path. Auto-names from prompt if omitted. |
|
||||
| `--post` | off | — | Run trim + normalize + convert after generation. |
|
||||
| `--output-ogg` | auto | — | OGG output path (implies `--post`). Defaults to same basename as WAV. |
|
||||
| `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. |
|
||||
|
||||
### Critical Constraints
|
||||
@@ -55,8 +63,6 @@ db/connectors/audio-generate "prompt text" \
|
||||
Be patient. The timeout default (600s) is generous.
|
||||
- **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s
|
||||
clips and crossfade-stitch in post-processing.
|
||||
- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import:
|
||||
`ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg`
|
||||
|
||||
## Prompt Assembly
|
||||
|
||||
@@ -74,19 +80,121 @@ family prefix and matching category template.
|
||||
asset type (ambient, sfx, ui)
|
||||
- **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md`
|
||||
|
||||
## Batch Workflow (Preferred)
|
||||
|
||||
For generating multiple assets, use a manifest file. This reduces prompt
|
||||
approvals to 2: one Write (manifest) + one Bash (batch run).
|
||||
|
||||
### 1. Create the manifest
|
||||
|
||||
Write a JSON manifest to `docs/assets/audio/batch-{sprint}-{ticket}.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"description": "Sprint 10 ambient + world SFX batch",
|
||||
"output_dir": "client/assets/audio",
|
||||
"gen_dir": "client/assets/audio/gen",
|
||||
"defaults": {
|
||||
"steps": 100,
|
||||
"cfg": 7,
|
||||
"lufs": -16,
|
||||
"quality": 6
|
||||
},
|
||||
"assets": [
|
||||
{
|
||||
"id": "AMB-001",
|
||||
"filename": "amb_station_base.ogg",
|
||||
"method": "sao",
|
||||
"duration": 45,
|
||||
"steps": 150,
|
||||
"cfg": 5,
|
||||
"prompt": "[sonic family prefix] + [template] + [description]"
|
||||
},
|
||||
{
|
||||
"id": "UI-005",
|
||||
"filename": "sfx_monologue_chime.ogg",
|
||||
"method": "synth",
|
||||
"synth": {
|
||||
"type": "harmonic",
|
||||
"duration": 0.8,
|
||||
"fundamental": 1200,
|
||||
"harmonics": [
|
||||
{"freq": 2400, "db": -12},
|
||||
{"freq": 3600, "db": -24}
|
||||
],
|
||||
"attack_ms": 15,
|
||||
"sustain_ratio": 0.2,
|
||||
"decay": "exponential"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g.,
|
||||
AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory.
|
||||
|
||||
### 2. Run the batch
|
||||
|
||||
```bash
|
||||
# Full run
|
||||
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json
|
||||
|
||||
# Dry run — preview what would be generated
|
||||
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --dry-run
|
||||
|
||||
# Generate only specific assets
|
||||
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --only AMB-001,AMB-002
|
||||
|
||||
# Skip assets that already have OGG files
|
||||
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --skip-existing
|
||||
```
|
||||
|
||||
### 3. Update asset docs with prompts
|
||||
|
||||
After the batch completes, write the exact prompts used back into the
|
||||
Prompt/Notes column of `docs/assets/audio/{category}.md`. The manifest records
|
||||
what was generated; the asset docs record what we have.
|
||||
|
||||
### Manifest fields
|
||||
|
||||
| Field | Required | Notes |
|
||||
|-------|----------|-------|
|
||||
| `id` | yes | Asset ID from docs (AMB-001, SFX-002, UI-005) |
|
||||
| `filename` | yes | Output filename (must match asset doc) |
|
||||
| `method` | yes | `sao` (Stable Audio Open) or `synth` (harmonic synthesis) |
|
||||
| `duration` | SAO only | Duration in seconds |
|
||||
| `prompt` | SAO only | Full assembled prompt |
|
||||
| `steps` | no | Override default steps |
|
||||
| `cfg` | no | Override default CFG |
|
||||
| `synth` | synth only | Synthesis parameters (see below) |
|
||||
|
||||
### Synth parameters
|
||||
|
||||
| Field | Default | Notes |
|
||||
|-------|---------|-------|
|
||||
| `type` | harmonic | Only `harmonic` supported currently |
|
||||
| `duration` | — | Duration in seconds |
|
||||
| `fundamental` | — | Fundamental frequency in Hz |
|
||||
| `harmonics` | [] | List of `{"freq": Hz, "db": dB}` objects |
|
||||
| `attack_ms` | 10 | Attack time in milliseconds |
|
||||
| `sustain_ratio` | 0.2 | Fraction of duration at full level before decay |
|
||||
| `decay` | exponential | `exponential` or `linear` |
|
||||
|
||||
## Single Asset Workflow
|
||||
|
||||
For one-off generation or iteration on a specific asset:
|
||||
|
||||
1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename,
|
||||
duration, bus, method, and design intent.
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `db/connectors/audio-health` to verify the API is up.
|
||||
6. Run `db/connectors/audio-generate` with the assembled prompt. **One request
|
||||
at a time. Wait for completion.**
|
||||
7. Listen to the output (or describe it based on file size/duration).
|
||||
8. If acceptable, convert to .ogg and place in `client/assets/audio/`.
|
||||
9. Update the asset status in `docs/assets/audio/{category}.md`.
|
||||
6. Run `db/connectors/audio-generate` with `--post` or `--output-ogg` to
|
||||
generate and post-process in one step.
|
||||
7. Verify the output (file size, duration).
|
||||
8. Update the asset status and prompt in `docs/assets/audio/{category}.md`.
|
||||
|
||||
## Iteration Workflow
|
||||
|
||||
@@ -101,62 +209,38 @@ For each asset, generate 4-6 candidates:
|
||||
4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a
|
||||
jarring repeat?
|
||||
5. **Close-your-eyes test** — does it create a mental image or sensation?
|
||||
6. Select the best candidate, trim, normalize, convert.
|
||||
6. Select the best candidate (post-processing is already done if `--post` was
|
||||
used).
|
||||
|
||||
## Post-Processing
|
||||
## Post-Processing (Standalone)
|
||||
|
||||
After selecting the best generation:
|
||||
If you need to post-process separately (e.g., re-normalizing an existing file):
|
||||
|
||||
```bash
|
||||
# Trim silence from start/end
|
||||
ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav
|
||||
# Full pipeline: trim → normalize → convert
|
||||
db/connectors/audio-post pipeline input.wav --output output.ogg
|
||||
|
||||
# LUFS normalize to -16 LUFS (broadcast standard, good for game audio)
|
||||
ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav
|
||||
|
||||
# Convert to .ogg for Godot
|
||||
ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg
|
||||
|
||||
# For loops: verify loop point
|
||||
ffplay -loop 0 output.ogg
|
||||
```
|
||||
|
||||
For ambient loops, create crossfade overlap:
|
||||
```bash
|
||||
# Create a 45s loop with 3s crossfade overlap
|
||||
# (manual: export 48s, crossfade first 3s with last 3s in Audacity)
|
||||
# Individual steps
|
||||
db/connectors/audio-post trim input.wav
|
||||
db/connectors/audio-post normalize input.wav --lufs -16
|
||||
db/connectors/audio-post convert input.wav --output output.ogg
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis instead:
|
||||
produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py`
|
||||
or the batch manifest's `method: "synth"` with harmonic parameters.
|
||||
|
||||
```python
|
||||
# Example: 50ms cursor hover tick
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
sr = 44100
|
||||
duration = 0.05 # 50ms
|
||||
t = np.linspace(0, duration, int(sr * duration), endpoint=False)
|
||||
freq = 3200 # Hz
|
||||
signal = np.sin(2 * np.pi * freq * t)
|
||||
envelope = np.exp(-t * 80) # exponential decay
|
||||
audio = (signal * envelope * 32767).astype(np.int16)
|
||||
|
||||
with wave.open("cursor_hover.wav", "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(audio.tobytes())
|
||||
```
|
||||
For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass
|
||||
impulse), write a custom script in `tooling/` following the pattern in
|
||||
`tooling/synth_ui_sounds.py`.
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
After generating, verify:
|
||||
- Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural)
|
||||
- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md)
|
||||
- Frequency range doesn't mask other layers (check docs/assets/audio/)
|
||||
- Duration matches spec
|
||||
- No unwanted artifacts (clicks, pops, digital noise at start/end)
|
||||
- Loop point is clean (ambient loops only)
|
||||
@@ -166,22 +250,6 @@ After generating, verify:
|
||||
## File Placement
|
||||
|
||||
Generated assets go to `client/assets/audio/` with exact filenames from the
|
||||
asset docs:
|
||||
asset docs. Intermediates go to `client/assets/audio/gen/` (gitignored).
|
||||
|
||||
```
|
||||
client/assets/audio/
|
||||
amb_station_base.ogg # Ambient bus
|
||||
amb_workplace_layer.ogg # Ambient bus
|
||||
amb_bar_layer.ogg # Ambient bus
|
||||
amb_corridor_layer.ogg # Ambient bus
|
||||
sfx_footstep_metal.ogg # Player Actions bus
|
||||
sfx_footstep_metal_run.ogg # Player Actions bus
|
||||
cursor_hover.ogg # UI Sounds bus
|
||||
implant_open.ogg # UI Sounds bus
|
||||
fog_recognition.ogg # UI Sounds bus
|
||||
weapon_aim.ogg # UI Sounds bus
|
||||
sfx_monologue_chime.ogg # UI Sounds bus
|
||||
sfx_monologue_chime_urgent.ogg # UI Sounds bus
|
||||
```
|
||||
|
||||
AudioManager discovers these by directory scan — filenames must match exactly.
|
||||
AudioManager discovers assets by directory scan — filenames must match exactly.
|
||||
|
||||
@@ -25,12 +25,17 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
- Mouse-relative facing and movement (#526, D-054) — mouse position determines facing direction (client-side float), WASD remapped to cursor-relative (W=toward, S=away, A/D=strafe), SET_FACING action sends octant to server, smooth facing indicator rotation
|
||||
- Room reset client UX (#502) — amber reset_plate tile type, 0.15s screen flash on room reset, 'Reset Room' interaction verb
|
||||
- Auto-checklist progress tracking (#503) — ChecklistEvaluator parses room YAML and evaluates 7 condition types against GameState with latching, ChecklistOverlay renders progress in gauntlet mode only, 48 new tests
|
||||
- 4 ambient zone loops: station base, workplace, bar, corridor — SAO-generated organic soundscape with crossfade loop points (#327)
|
||||
- 2 footstep SFX: metal walk and run — SAO hybrid with best-transient extraction (#327)
|
||||
- `audio-batch` command — batch audio generation from JSON manifests, supports SAO and harmonic synthesis, with `--dry-run`, `--only`, and `--skip-existing` flags
|
||||
- `--post` and `--output-ogg` flags on `audio-generate` — chain post-processing (trim, normalize, convert) into a single command
|
||||
|
||||
### Changed
|
||||
- `push-pr` skill now runs `/commit` first when uncommitted changes are detected
|
||||
- Insert open/close now sends explicit PauseSimulation/ResumeSimulation (#518, D-058) — replaces toggle-style pause with idempotent pair
|
||||
- Interaction list colors reference Constants.IMPLANT_TEXT_COLOR instead of hardcoded values
|
||||
- World radial menu uses theme font instead of ThemeDB.fallback_font
|
||||
- Monologue chimes replaced with production-quality manual synthesis — insert-tech aesthetic per D-074, pure sine harmonics with mathematical envelopes (#327)
|
||||
|
||||
### Fixed
|
||||
- Bidirectional relationship check (#515) — Check 9 tested `target in npc_rels` which missed NPCs with no relationship entries; changed to `target in self.npcs`
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
# Batch audio generation from a manifest file. Whitelistable command.
|
||||
# Usage: audio-batch manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
|
||||
exec python3 "$(dirname "$0")/audio_batch.py" "$@"
|
||||
@@ -0,0 +1,309 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Batch audio generation from a manifest file.
|
||||
|
||||
Processes multiple assets sequentially: SAO generation or harmonic synthesis,
|
||||
followed by post-processing (trim, normalize, convert to OGG).
|
||||
|
||||
Usage:
|
||||
python3 audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
|
||||
"""
|
||||
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def load_manifest(path):
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def resolve_paths(manifest, manifest_dir):
|
||||
"""Resolve output_dir and gen_dir relative to the git root."""
|
||||
# Find git root by walking up from manifest_dir
|
||||
# Check for .git as file (worktree) or directory (regular repo)
|
||||
git_root = manifest_dir
|
||||
while git_root != "/":
|
||||
if os.path.exists(os.path.join(git_root, ".git")):
|
||||
break
|
||||
git_root = os.path.dirname(git_root)
|
||||
else:
|
||||
git_root = manifest_dir
|
||||
|
||||
output_dir = os.path.join(git_root, manifest.get("output_dir", "client/assets/audio"))
|
||||
gen_dir = os.path.join(git_root, manifest.get("gen_dir", "client/assets/audio/gen"))
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(gen_dir, exist_ok=True)
|
||||
return output_dir, gen_dir, git_root
|
||||
|
||||
|
||||
def get_default(manifest, asset, key):
|
||||
"""Get a value from the asset, falling back to manifest defaults."""
|
||||
defaults = manifest.get("defaults", {})
|
||||
return asset.get(key, defaults.get(key))
|
||||
|
||||
|
||||
def run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir):
|
||||
"""Generate audio via Stable Audio Open + post-processing."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
wav_path = os.path.join(gen_dir, base_name + ".wav")
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
|
||||
prompt = asset["prompt"]
|
||||
duration = asset.get("duration", 10)
|
||||
steps = get_default(manifest, asset, "steps") or 100
|
||||
cfg = get_default(manifest, asset, "cfg") or 7
|
||||
timeout = get_default(manifest, asset, "timeout") or 600
|
||||
|
||||
# Run audio-generate with --post
|
||||
cmd = [
|
||||
sys.executable, os.path.join(script_dir, "audio_connector.py"),
|
||||
"generate", prompt,
|
||||
"--duration", str(duration),
|
||||
"--steps", str(steps),
|
||||
"--cfg", str(cfg),
|
||||
"--output", wav_path,
|
||||
"--output-ogg", ogg_path,
|
||||
"--timeout", str(timeout),
|
||||
]
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout + 60)
|
||||
if result.returncode != 0:
|
||||
stderr = result.stderr.strip()
|
||||
try:
|
||||
err = json.loads(result.stdout)
|
||||
return {"ok": False, "error": err.get("error", stderr)}
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return {"ok": False, "error": stderr or "generation failed"}
|
||||
|
||||
try:
|
||||
return json.loads(result.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return {"ok": False, "error": f"Unexpected output: {result.stdout[:200]}"}
|
||||
|
||||
|
||||
def synthesize_harmonic(params, wav_path):
|
||||
"""Synthesize audio from harmonic parameters."""
|
||||
sr = 44100
|
||||
duration = params["duration"]
|
||||
fundamental = params["fundamental"]
|
||||
harmonics = params.get("harmonics", [])
|
||||
attack_ms = params.get("attack_ms", 10)
|
||||
sustain_ratio = params.get("sustain_ratio", 0.2)
|
||||
decay = params.get("decay", "exponential")
|
||||
|
||||
n = int(sr * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
# Fundamental
|
||||
signal = np.sin(2 * np.pi * fundamental * t)
|
||||
|
||||
# Add harmonics
|
||||
for h in harmonics:
|
||||
freq = h["freq"]
|
||||
db = h["db"]
|
||||
amplitude = 10 ** (db / 20)
|
||||
signal = signal + amplitude * np.sin(2 * np.pi * freq * t)
|
||||
|
||||
# Envelope: attack + sustain + decay
|
||||
attack_s = attack_ms / 1000
|
||||
attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n)
|
||||
|
||||
sustain_end = duration * sustain_ratio
|
||||
if decay == "exponential":
|
||||
# Decay rate: reach -60dB by end of duration
|
||||
decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10
|
||||
decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end)))
|
||||
else:
|
||||
# Linear decay
|
||||
decay_env = np.where(t < sustain_end, 1.0,
|
||||
1.0 - (t - sustain_end) / (duration - sustain_end))
|
||||
|
||||
envelope = attack_env * decay_env
|
||||
signal = signal * envelope
|
||||
|
||||
# Normalize to peak
|
||||
peak = np.max(np.abs(signal))
|
||||
if peak > 0:
|
||||
signal = signal / peak * 0.9
|
||||
|
||||
# Write WAV
|
||||
int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16)
|
||||
with wave.open(wav_path, "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
|
||||
return wav_path
|
||||
|
||||
|
||||
def run_synth(asset, manifest, gen_dir, output_dir, script_dir):
|
||||
"""Synthesize audio from harmonic parameters + post-process."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
wav_path = os.path.join(gen_dir, base_name + "_synth.wav")
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
|
||||
synth_params = asset.get("synth")
|
||||
if not synth_params:
|
||||
return {"ok": False, "error": "No synth parameters provided"}
|
||||
|
||||
synth_type = synth_params.get("type", "harmonic")
|
||||
if synth_type != "harmonic":
|
||||
return {"ok": False, "error": f"Unknown synth type: {synth_type}"}
|
||||
|
||||
try:
|
||||
synthesize_harmonic(synth_params, wav_path)
|
||||
except Exception as e:
|
||||
return {"ok": False, "error": f"Synthesis failed: {e}"}
|
||||
|
||||
# Post-process: normalize + convert (skip trim for synth — no silence to trim)
|
||||
post_script = os.path.join(script_dir, "audio_post.py")
|
||||
lufs = get_default(manifest, asset, "lufs") or -16
|
||||
quality = get_default(manifest, asset, "quality") or 6
|
||||
|
||||
# Normalize
|
||||
norm_path = os.path.join(gen_dir, base_name + "_norm.wav")
|
||||
cmd = [sys.executable, post_script, "normalize", wav_path, "--output", norm_path,
|
||||
"--lufs", str(lufs)]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Normalize failed: {result.stderr.strip()}"}
|
||||
|
||||
# Convert to OGG
|
||||
cmd = [sys.executable, post_script, "convert", norm_path, "--output", ogg_path,
|
||||
"--quality", str(quality)]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Convert failed: {result.stderr.strip()}"}
|
||||
|
||||
# Clean up intermediate
|
||||
try:
|
||||
os.remove(norm_path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
ogg_size = os.path.getsize(ogg_path)
|
||||
return {
|
||||
"ok": True,
|
||||
"file": wav_path,
|
||||
"ogg_file": ogg_path,
|
||||
"ogg_size_bytes": ogg_size,
|
||||
"synth_params": synth_params,
|
||||
"post_processed": True,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]",
|
||||
file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
manifest_path = sys.argv[1]
|
||||
dry_run = "--dry-run" in sys.argv
|
||||
skip_existing = "--skip-existing" in sys.argv
|
||||
|
||||
only_ids = None
|
||||
for i, arg in enumerate(sys.argv):
|
||||
if arg == "--only" and i + 1 < len(sys.argv):
|
||||
only_ids = set(sys.argv[i + 1].split(","))
|
||||
|
||||
manifest = load_manifest(manifest_path)
|
||||
manifest_dir = os.path.dirname(os.path.abspath(manifest_path))
|
||||
output_dir, gen_dir, git_root = resolve_paths(manifest, manifest_dir)
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
assets = manifest.get("assets", [])
|
||||
if only_ids:
|
||||
assets = [a for a in assets if a["id"] in only_ids]
|
||||
|
||||
# Health check if any SAO assets
|
||||
sao_assets = [a for a in assets if a.get("method") == "sao"]
|
||||
if sao_assets and not dry_run:
|
||||
print(f"Checking SAO API health...", file=sys.stderr)
|
||||
health_cmd = [sys.executable, os.path.join(script_dir, "audio_connector.py"), "health"]
|
||||
result = subprocess.run(health_cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(json.dumps({"ok": False, "error": "SAO API health check failed",
|
||||
"details": result.stdout.strip()}))
|
||||
sys.exit(1)
|
||||
print(f" SAO API is up.", file=sys.stderr)
|
||||
|
||||
total = len(assets)
|
||||
results = []
|
||||
success = 0
|
||||
failed = 0
|
||||
skipped = 0
|
||||
|
||||
print(f"Processing {total} assets from {os.path.basename(manifest_path)}...", file=sys.stderr)
|
||||
if dry_run:
|
||||
print(" (dry run — no generation will occur)", file=sys.stderr)
|
||||
|
||||
for i, asset in enumerate(assets, 1):
|
||||
asset_id = asset["id"]
|
||||
filename = asset["filename"]
|
||||
method = asset.get("method", "sao")
|
||||
|
||||
print(f"\n[{i}/{total}] {asset_id}: {filename} ({method})", file=sys.stderr)
|
||||
|
||||
if skip_existing:
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
if os.path.exists(ogg_path):
|
||||
print(f" Skipping — already exists", file=sys.stderr)
|
||||
results.append({"id": asset_id, "status": "skipped", "reason": "exists"})
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
if dry_run:
|
||||
print(f" Would generate: {filename}", file=sys.stderr)
|
||||
if method == "sao":
|
||||
print(f" Prompt: {asset.get('prompt', '(none)')[:80]}...", file=sys.stderr)
|
||||
elif method == "synth":
|
||||
synth = asset.get("synth", {})
|
||||
print(f" Synth: {synth.get('fundamental')}Hz, {synth.get('duration')}s",
|
||||
file=sys.stderr)
|
||||
results.append({"id": asset_id, "status": "dry_run"})
|
||||
continue
|
||||
|
||||
if method == "sao":
|
||||
result = run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir)
|
||||
elif method == "synth":
|
||||
result = run_synth(asset, manifest, gen_dir, output_dir, script_dir)
|
||||
else:
|
||||
result = {"ok": False, "error": f"Unknown method: {method}"}
|
||||
|
||||
result["id"] = asset_id
|
||||
if result.get("ok"):
|
||||
success += 1
|
||||
result["status"] = "success"
|
||||
print(f" OK → {result.get('ogg_file', filename)}", file=sys.stderr)
|
||||
else:
|
||||
failed += 1
|
||||
result["status"] = "failed"
|
||||
print(f" FAILED: {result.get('error', 'unknown')}", file=sys.stderr)
|
||||
|
||||
results.append(result)
|
||||
|
||||
# Summary
|
||||
summary = {
|
||||
"ok": failed == 0,
|
||||
"total": total,
|
||||
"success": success,
|
||||
"failed": failed,
|
||||
"skipped": skipped,
|
||||
"results": results,
|
||||
}
|
||||
print(json.dumps(summary, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -6,12 +6,13 @@ Talks to the Stable Audio Open Gradio app at tower-of-joy:11500.
|
||||
Uses the async Gradio API pattern: POST to submit, SSE stream for results.
|
||||
|
||||
Usage:
|
||||
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav]
|
||||
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav] [--post]
|
||||
python3 audio_connector.py health
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
@@ -56,7 +57,31 @@ def health():
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600):
|
||||
def post_process(wav_path, ogg_path=None, lufs=-16, quality=6, threshold=-50):
|
||||
"""Run trim + normalize + convert on a WAV file via audio-post pipeline."""
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
post_script = os.path.join(script_dir, "audio_post.py")
|
||||
if ogg_path is None:
|
||||
ogg_path = os.path.splitext(wav_path)[0] + ".ogg"
|
||||
cmd = [
|
||||
sys.executable, post_script, "pipeline", wav_path,
|
||||
"--output", ogg_path,
|
||||
"--lufs", str(lufs),
|
||||
"--quality", str(quality),
|
||||
"--threshold", str(threshold),
|
||||
]
|
||||
print(f" Post-processing → {os.path.basename(ogg_path)}...", file=sys.stderr)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Post-processing failed: {result.stderr.strip()}"}
|
||||
try:
|
||||
return json.loads(result.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return {"ok": True, "output": ogg_path}
|
||||
|
||||
|
||||
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600,
|
||||
post=False, output_ogg=None):
|
||||
"""
|
||||
Generate audio from a text prompt.
|
||||
|
||||
@@ -67,6 +92,8 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
|
||||
cfg: Classifier-free guidance scale (default 7)
|
||||
output: Output file path (default: auto-named in current directory)
|
||||
timeout: Maximum wait time in seconds (default 600 = 10 minutes)
|
||||
post: If True, run trim+normalize+convert after generation
|
||||
output_ogg: OGG output path when post=True (default: same basename .ogg)
|
||||
"""
|
||||
base = get_base_url()
|
||||
api_url = f"{base}/gradio_api/call/generate_audio"
|
||||
@@ -216,7 +243,7 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
file_size = os.path.getsize(output)
|
||||
print(json.dumps({
|
||||
result_json = {
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
@@ -225,7 +252,20 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
|
||||
"cfg": cfg,
|
||||
"prompt": prompt,
|
||||
"generation_time_s": elapsed
|
||||
}, indent=2))
|
||||
}
|
||||
|
||||
if post:
|
||||
post_result = post_process(output, ogg_path=output_ogg)
|
||||
if not post_result.get("ok"):
|
||||
result_json["post_processed"] = False
|
||||
result_json["post_error"] = post_result.get("error", "unknown")
|
||||
else:
|
||||
result_json["post_processed"] = True
|
||||
result_json["ogg_file"] = post_result.get("output", output_ogg)
|
||||
ogg_size = os.path.getsize(result_json["ogg_file"])
|
||||
result_json["ogg_size_bytes"] = ogg_size
|
||||
|
||||
print(json.dumps(result_json, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
@@ -258,6 +298,8 @@ def main():
|
||||
cfg = 7.0
|
||||
output = None
|
||||
timeout = 600
|
||||
post = False
|
||||
output_ogg = None
|
||||
|
||||
# Parse optional args
|
||||
i = 3
|
||||
@@ -277,11 +319,19 @@ def main():
|
||||
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
|
||||
timeout = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--post":
|
||||
post = True
|
||||
i += 1
|
||||
elif sys.argv[i] == "--output-ogg" and i + 1 < len(sys.argv):
|
||||
output_ogg = sys.argv[i + 1]
|
||||
post = True # --output-ogg implies --post
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, timeout=timeout)
|
||||
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output,
|
||||
timeout=timeout, post=post, output_ogg=output_ogg)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -14,10 +14,10 @@ Stable Audio Open primary. Prompt strategy: describe the SPACE, not the sound.
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| AMB-001 | `amb_station_base.ogg` | planned | Ambient | SAO | 45-60s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. | S8+ |
|
||||
| AMB-002 | `amb_workplace_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. | S8+ |
|
||||
| AMB-003 | `amb_bar_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. | S8+ |
|
||||
| AMB-004 | `amb_corridor_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. | S8+ |
|
||||
| AMB-001 | `amb_station_base.ogg` | done | Ambient | SAO | 43.3s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
|
||||
| AMB-002 | `amb_workplace_layer.ogg` | done | Ambient | SAO | 42.9s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
|
||||
| AMB-003 | `amb_bar_layer.ogg` | done | Ambient | SAO | 43.6s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
|
||||
| AMB-004 | `amb_corridor_layer.ogg` | done | Ambient | SAO | 42.7s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
|
||||
@@ -13,8 +13,8 @@ Mixed methods depending on duration and character:
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| SFX-001 | `sfx_footstep_metal.ogg` | planned | Player Actions | hybrid | 0.3-0.5s | Single footstep on metal grating. Walk pace. Player character proprioceptive feedback. | S8+ |
|
||||
| SFX-002 | `sfx_footstep_metal_run.ogg` | planned | Player Actions | hybrid | 0.2-0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). | S8+ |
|
||||
| SFX-001 | `sfx_footstep_metal_walk.ogg` | done | Player Actions | SAO hybrid | 0.3s | Single footstep on metal grating. Walk pace. SAO 5s/100 steps/CFG 8, best transient trimmed at 2.42s, LUFS -16 normalized. | S10 #327 |
|
||||
| SFX-002 | `sfx_footstep_metal_run.ogg` | done | Player Actions | SAO hybrid | 0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). SAO 5s/100 steps/CFG 8, best transient trimmed at 1.03s, LUFS -16 normalized. | S10 #327 |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ Interface sounds triggered by player interaction, insert systems, and cognitive
|
||||
## Generation Approach
|
||||
|
||||
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`.
|
||||
- **Monologue chimes:** Re-generated in Sprint 9 (#453). Insert-tech prefix + Chimes template, 5s SAO generation trimmed to 0.8s.
|
||||
- **Monologue chimes:** Replaced in Sprint 10 (#327) with manual synthesis. Insert-tech aesthetic: pure sine harmonics, mathematical envelope, no SAO. Previous S9 SAO versions were acknowledged placeholders per D-038 amendment.
|
||||
|
||||
## Assets
|
||||
|
||||
@@ -15,8 +15,8 @@ Interface sounds triggered by player interaction, insert systems, and cognitive
|
||||
| UI-002 | `implant_open.ogg` | done | UI Sounds | SAO | 217ms | Rising tone when insert UI opens (radial menu, inventory, stance). Neural lattice powering up. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 |
|
||||
| UI-003 | `fog_recognition.ogg` | done | UI Sounds | SAO | 400ms | Warm organic chime at onset of cognitive delay (D-060). Re-generated in S9 — original was silent. Organic prefix + Chimes template. | S7 #440, S9 #453 |
|
||||
| UI-004 | `weapon_aim.ogg` | done | UI Sounds | SAO | 127ms | Harder click/lock for weapon aim state. Mechanical, deliberate. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 |
|
||||
| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template. Crystalline tone, neural lattice surfacing a thought. See design brief below. | S7 #440, S9 #453 |
|
||||
| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template (sharper/brighter variant). CFG 8. See design brief below. | S7 #440, S9 #453 |
|
||||
| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1200Hz fundamental + harmonics (2400Hz -12dB, 3600Hz -24dB, 6000Hz -30dB). 15ms attack, 0.2 sustain ratio, exp decay. Peak -15dB. No SAO — pure mathematical precision. | S7 #440, S9 #453, S10 #327 |
|
||||
| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1220Hz fundamental (detuned +20Hz for tension) + stronger harmonics (2440Hz -6dB, 3660Hz -18dB, 5087Hz -24dB, inharmonic 5087Hz -28dB). 8ms attack, 0.3 sustain ratio. Peak -6.3dB. | S7 #440, S9 #453, S10 #327 |
|
||||
| UI-007 | `sfx_weapon_aim_lock.ogg` | done | UI Sounds | SAO | 505ms | Sharp targeting lock-on confirmation for weapon aim. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 |
|
||||
| UI-008 | `sfx_stance_change.ogg` | done | UI Sounds | SAO | 349ms | Soft mechanical mode-switch click for stance toggle. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user