Merge remote-tracking branch 'origin/audio'

This commit is contained in:
2026-02-19 13:10:26 +01:00
16 changed files with 522 additions and 86 deletions
+140 -72
View File
@@ -27,13 +27,19 @@ workflow, and quality validation.
# Check API health
db/connectors/audio-health
# Generate audio
# Generate a single asset (WAV only)
db/connectors/audio-generate "prompt text" \
--duration 10 \
--steps 100 \
--cfg 7 \
--output path/to/output.wav \
--timeout 600
--duration 10 --steps 100 --cfg 7 \
--output path/to/output.wav
# Generate + post-process in one command (WAV → trim → normalize → OGG)
db/connectors/audio-generate "prompt text" \
--duration 10 --steps 100 --cfg 7 \
--output path/to/gen/intermediate.wav \
--output-ogg client/assets/audio/final.ogg
# Batch-generate from a manifest (preferred for multiple assets)
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json
```
### Parameters
@@ -43,7 +49,9 @@ db/connectors/audio-generate "prompt text" \
| `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. |
| `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. |
| `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. |
| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. |
| `--output` | auto | — | Output WAV file path. Auto-names from prompt if omitted. |
| `--post` | off | — | Run trim + normalize + convert after generation. |
| `--output-ogg` | auto | — | OGG output path (implies `--post`). Defaults to same basename as WAV. |
| `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. |
### Critical Constraints
@@ -55,8 +63,6 @@ db/connectors/audio-generate "prompt text" \
Be patient. The timeout default (600s) is generous.
- **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s
clips and crossfade-stitch in post-processing.
- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import:
`ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg`
## Prompt Assembly
@@ -74,19 +80,121 @@ family prefix and matching category template.
asset type (ambient, sfx, ui)
- **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md`
## Batch Workflow (Preferred)
For generating multiple assets, use a manifest file. This reduces prompt
approvals to 2: one Write (manifest) + one Bash (batch run).
### 1. Create the manifest
Write a JSON manifest to `docs/assets/audio/batch-{sprint}-{ticket}.json`:
```json
{
"description": "Sprint 10 ambient + world SFX batch",
"output_dir": "client/assets/audio",
"gen_dir": "client/assets/audio/gen",
"defaults": {
"steps": 100,
"cfg": 7,
"lufs": -16,
"quality": 6
},
"assets": [
{
"id": "AMB-001",
"filename": "amb_station_base.ogg",
"method": "sao",
"duration": 45,
"steps": 150,
"cfg": 5,
"prompt": "[sonic family prefix] + [template] + [description]"
},
{
"id": "UI-005",
"filename": "sfx_monologue_chime.ogg",
"method": "synth",
"synth": {
"type": "harmonic",
"duration": 0.8,
"fundamental": 1200,
"harmonics": [
{"freq": 2400, "db": -12},
{"freq": 3600, "db": -24}
],
"attack_ms": 15,
"sustain_ratio": 0.2,
"decay": "exponential"
}
}
]
}
```
Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g.,
AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory.
### 2. Run the batch
```bash
# Full run
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json
# Dry run — preview what would be generated
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --dry-run
# Generate only specific assets
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --only AMB-001,AMB-002
# Skip assets that already have OGG files
db/connectors/audio-batch docs/assets/audio/batch-s10-327.json --skip-existing
```
### 3. Update asset docs with prompts
After the batch completes, write the exact prompts used back into the
Prompt/Notes column of `docs/assets/audio/{category}.md`. The manifest records
what was generated; the asset docs record what we have.
### Manifest fields
| Field | Required | Notes |
|-------|----------|-------|
| `id` | yes | Asset ID from docs (AMB-001, SFX-002, UI-005) |
| `filename` | yes | Output filename (must match asset doc) |
| `method` | yes | `sao` (Stable Audio Open) or `synth` (harmonic synthesis) |
| `duration` | SAO only | Duration in seconds |
| `prompt` | SAO only | Full assembled prompt |
| `steps` | no | Override default steps |
| `cfg` | no | Override default CFG |
| `synth` | synth only | Synthesis parameters (see below) |
### Synth parameters
| Field | Default | Notes |
|-------|---------|-------|
| `type` | harmonic | Only `harmonic` supported currently |
| `duration` | — | Duration in seconds |
| `fundamental` | — | Fundamental frequency in Hz |
| `harmonics` | [] | List of `{"freq": Hz, "db": dB}` objects |
| `attack_ms` | 10 | Attack time in milliseconds |
| `sustain_ratio` | 0.2 | Fraction of duration at full level before decay |
| `decay` | exponential | `exponential` or `linear` |
## Single Asset Workflow
For one-off generation or iteration on a specific asset:
1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename,
duration, bus, method, and design intent.
2. Read `references/sonic-palette.md` for the sonic family prefix.
3. Read `references/category-templates.md` for the matching template.
4. Assemble the full prompt.
5. Run `db/connectors/audio-health` to verify the API is up.
6. Run `db/connectors/audio-generate` with the assembled prompt. **One request
at a time. Wait for completion.**
7. Listen to the output (or describe it based on file size/duration).
8. If acceptable, convert to .ogg and place in `client/assets/audio/`.
9. Update the asset status in `docs/assets/audio/{category}.md`.
6. Run `db/connectors/audio-generate` with `--post` or `--output-ogg` to
generate and post-process in one step.
7. Verify the output (file size, duration).
8. Update the asset status and prompt in `docs/assets/audio/{category}.md`.
## Iteration Workflow
@@ -101,62 +209,38 @@ For each asset, generate 4-6 candidates:
4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a
jarring repeat?
5. **Close-your-eyes test** — does it create a mental image or sensation?
6. Select the best candidate, trim, normalize, convert.
6. Select the best candidate (post-processing is already done if `--post` was
used).
## Post-Processing
## Post-Processing (Standalone)
After selecting the best generation:
If you need to post-process separately (e.g., re-normalizing an existing file):
```bash
# Trim silence from start/end
ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav
# Full pipeline: trim → normalize → convert
db/connectors/audio-post pipeline input.wav --output output.ogg
# LUFS normalize to -16 LUFS (broadcast standard, good for game audio)
ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav
# Convert to .ogg for Godot
ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg
# For loops: verify loop point
ffplay -loop 0 output.ogg
```
For ambient loops, create crossfade overlap:
```bash
# Create a 45s loop with 3s crossfade overlap
# (manual: export 48s, crossfade first 3s with last 3s in Audacity)
# Individual steps
db/connectors/audio-post trim input.wav
db/connectors/audio-post normalize input.wav --lufs -16
db/connectors/audio-post convert input.wav --output output.ogg
```
## Manual Synthesis (Insert-Tech Sounds)
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
produce meaningful output. Use manual synthesis instead:
produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py`
or the batch manifest's `method: "synth"` with harmonic parameters.
```python
# Example: 50ms cursor hover tick
import numpy as np
import wave
sr = 44100
duration = 0.05 # 50ms
t = np.linspace(0, duration, int(sr * duration), endpoint=False)
freq = 3200 # Hz
signal = np.sin(2 * np.pi * freq * t)
envelope = np.exp(-t * 80) # exponential decay
audio = (signal * envelope * 32767).astype(np.int16)
with wave.open("cursor_hover.wav", "w") as f:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(sr)
f.writeframes(audio.tobytes())
```
For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass
impulse), write a custom script in `tooling/` following the pattern in
`tooling/synth_ui_sounds.py`.
## Quality Checklist
After generating, verify:
- Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural)
- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md)
- Frequency range doesn't mask other layers (check docs/assets/audio/)
- Duration matches spec
- No unwanted artifacts (clicks, pops, digital noise at start/end)
- Loop point is clean (ambient loops only)
@@ -166,22 +250,6 @@ After generating, verify:
## File Placement
Generated assets go to `client/assets/audio/` with exact filenames from the
asset docs:
asset docs. Intermediates go to `client/assets/audio/gen/` (gitignored).
```
client/assets/audio/
amb_station_base.ogg # Ambient bus
amb_workplace_layer.ogg # Ambient bus
amb_bar_layer.ogg # Ambient bus
amb_corridor_layer.ogg # Ambient bus
sfx_footstep_metal.ogg # Player Actions bus
sfx_footstep_metal_run.ogg # Player Actions bus
cursor_hover.ogg # UI Sounds bus
implant_open.ogg # UI Sounds bus
fog_recognition.ogg # UI Sounds bus
weapon_aim.ogg # UI Sounds bus
sfx_monologue_chime.ogg # UI Sounds bus
sfx_monologue_chime_urgent.ogg # UI Sounds bus
```
AudioManager discovers these by directory scan — filenames must match exactly.
AudioManager discovers assets by directory scan — filenames must match exactly.
+5
View File
@@ -25,12 +25,17 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
- Mouse-relative facing and movement (#526, D-054) — mouse position determines facing direction (client-side float), WASD remapped to cursor-relative (W=toward, S=away, A/D=strafe), SET_FACING action sends octant to server, smooth facing indicator rotation
- Room reset client UX (#502) — amber reset_plate tile type, 0.15s screen flash on room reset, 'Reset Room' interaction verb
- Auto-checklist progress tracking (#503) — ChecklistEvaluator parses room YAML and evaluates 7 condition types against GameState with latching, ChecklistOverlay renders progress in gauntlet mode only, 48 new tests
- 4 ambient zone loops: station base, workplace, bar, corridor — SAO-generated organic soundscape with crossfade loop points (#327)
- 2 footstep SFX: metal walk and run — SAO hybrid with best-transient extraction (#327)
- `audio-batch` command — batch audio generation from JSON manifests, supports SAO and harmonic synthesis, with `--dry-run`, `--only`, and `--skip-existing` flags
- `--post` and `--output-ogg` flags on `audio-generate` — chain post-processing (trim, normalize, convert) into a single command
### Changed
- `push-pr` skill now runs `/commit` first when uncommitted changes are detected
- Insert open/close now sends explicit PauseSimulation/ResumeSimulation (#518, D-058) — replaces toggle-style pause with idempotent pair
- Interaction list colors reference Constants.IMPLANT_TEXT_COLOR instead of hardcoded values
- World radial menu uses theme font instead of ThemeDB.fallback_font
- Monologue chimes replaced with production-quality manual synthesis — insert-tech aesthetic per D-074, pure sine harmonics with mathematical envelopes (#327)
### Fixed
- Bidirectional relationship check (#515) — Check 9 tested `target in npc_rels` which missed NPCs with no relationship entries; changed to `target in self.npcs`
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Batch audio generation from a manifest file. Whitelistable command.
# Usage: audio-batch manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
exec python3 "$(dirname "$0")/audio_batch.py" "$@"
+309
View File
@@ -0,0 +1,309 @@
#!/usr/bin/env python3
"""Batch audio generation from a manifest file.
Processes multiple assets sequentially: SAO generation or harmonic synthesis,
followed by post-processing (trim, normalize, convert to OGG).
Usage:
python3 audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
"""
import json
import math
import os
import struct
import subprocess
import sys
import wave
import numpy as np
def load_manifest(path):
with open(path) as f:
return json.load(f)
def resolve_paths(manifest, manifest_dir):
"""Resolve output_dir and gen_dir relative to the git root."""
# Find git root by walking up from manifest_dir
# Check for .git as file (worktree) or directory (regular repo)
git_root = manifest_dir
while git_root != "/":
if os.path.exists(os.path.join(git_root, ".git")):
break
git_root = os.path.dirname(git_root)
else:
git_root = manifest_dir
output_dir = os.path.join(git_root, manifest.get("output_dir", "client/assets/audio"))
gen_dir = os.path.join(git_root, manifest.get("gen_dir", "client/assets/audio/gen"))
os.makedirs(output_dir, exist_ok=True)
os.makedirs(gen_dir, exist_ok=True)
return output_dir, gen_dir, git_root
def get_default(manifest, asset, key):
"""Get a value from the asset, falling back to manifest defaults."""
defaults = manifest.get("defaults", {})
return asset.get(key, defaults.get(key))
def run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir):
"""Generate audio via Stable Audio Open + post-processing."""
filename = asset["filename"]
base_name = os.path.splitext(filename)[0]
wav_path = os.path.join(gen_dir, base_name + ".wav")
ogg_path = os.path.join(output_dir, filename)
prompt = asset["prompt"]
duration = asset.get("duration", 10)
steps = get_default(manifest, asset, "steps") or 100
cfg = get_default(manifest, asset, "cfg") or 7
timeout = get_default(manifest, asset, "timeout") or 600
# Run audio-generate with --post
cmd = [
sys.executable, os.path.join(script_dir, "audio_connector.py"),
"generate", prompt,
"--duration", str(duration),
"--steps", str(steps),
"--cfg", str(cfg),
"--output", wav_path,
"--output-ogg", ogg_path,
"--timeout", str(timeout),
]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout + 60)
if result.returncode != 0:
stderr = result.stderr.strip()
try:
err = json.loads(result.stdout)
return {"ok": False, "error": err.get("error", stderr)}
except (json.JSONDecodeError, ValueError):
return {"ok": False, "error": stderr or "generation failed"}
try:
return json.loads(result.stdout)
except json.JSONDecodeError:
return {"ok": False, "error": f"Unexpected output: {result.stdout[:200]}"}
def synthesize_harmonic(params, wav_path):
"""Synthesize audio from harmonic parameters."""
sr = 44100
duration = params["duration"]
fundamental = params["fundamental"]
harmonics = params.get("harmonics", [])
attack_ms = params.get("attack_ms", 10)
sustain_ratio = params.get("sustain_ratio", 0.2)
decay = params.get("decay", "exponential")
n = int(sr * duration)
t = np.linspace(0, duration, n, endpoint=False)
# Fundamental
signal = np.sin(2 * np.pi * fundamental * t)
# Add harmonics
for h in harmonics:
freq = h["freq"]
db = h["db"]
amplitude = 10 ** (db / 20)
signal = signal + amplitude * np.sin(2 * np.pi * freq * t)
# Envelope: attack + sustain + decay
attack_s = attack_ms / 1000
attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n)
sustain_end = duration * sustain_ratio
if decay == "exponential":
# Decay rate: reach -60dB by end of duration
decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10
decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end)))
else:
# Linear decay
decay_env = np.where(t < sustain_end, 1.0,
1.0 - (t - sustain_end) / (duration - sustain_end))
envelope = attack_env * decay_env
signal = signal * envelope
# Normalize to peak
peak = np.max(np.abs(signal))
if peak > 0:
signal = signal / peak * 0.9
# Write WAV
int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16)
with wave.open(wav_path, "w") as f:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(sr)
f.writeframes(int_samples.tobytes())
return wav_path
def run_synth(asset, manifest, gen_dir, output_dir, script_dir):
"""Synthesize audio from harmonic parameters + post-process."""
filename = asset["filename"]
base_name = os.path.splitext(filename)[0]
wav_path = os.path.join(gen_dir, base_name + "_synth.wav")
ogg_path = os.path.join(output_dir, filename)
synth_params = asset.get("synth")
if not synth_params:
return {"ok": False, "error": "No synth parameters provided"}
synth_type = synth_params.get("type", "harmonic")
if synth_type != "harmonic":
return {"ok": False, "error": f"Unknown synth type: {synth_type}"}
try:
synthesize_harmonic(synth_params, wav_path)
except Exception as e:
return {"ok": False, "error": f"Synthesis failed: {e}"}
# Post-process: normalize + convert (skip trim for synth — no silence to trim)
post_script = os.path.join(script_dir, "audio_post.py")
lufs = get_default(manifest, asset, "lufs") or -16
quality = get_default(manifest, asset, "quality") or 6
# Normalize
norm_path = os.path.join(gen_dir, base_name + "_norm.wav")
cmd = [sys.executable, post_script, "normalize", wav_path, "--output", norm_path,
"--lufs", str(lufs)]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
return {"ok": False, "error": f"Normalize failed: {result.stderr.strip()}"}
# Convert to OGG
cmd = [sys.executable, post_script, "convert", norm_path, "--output", ogg_path,
"--quality", str(quality)]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
return {"ok": False, "error": f"Convert failed: {result.stderr.strip()}"}
# Clean up intermediate
try:
os.remove(norm_path)
except OSError:
pass
ogg_size = os.path.getsize(ogg_path)
return {
"ok": True,
"file": wav_path,
"ogg_file": ogg_path,
"ogg_size_bytes": ogg_size,
"synth_params": synth_params,
"post_processed": True,
}
def main():
if len(sys.argv) < 2:
print("Usage: audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]",
file=sys.stderr)
sys.exit(1)
manifest_path = sys.argv[1]
dry_run = "--dry-run" in sys.argv
skip_existing = "--skip-existing" in sys.argv
only_ids = None
for i, arg in enumerate(sys.argv):
if arg == "--only" and i + 1 < len(sys.argv):
only_ids = set(sys.argv[i + 1].split(","))
manifest = load_manifest(manifest_path)
manifest_dir = os.path.dirname(os.path.abspath(manifest_path))
output_dir, gen_dir, git_root = resolve_paths(manifest, manifest_dir)
script_dir = os.path.dirname(os.path.abspath(__file__))
assets = manifest.get("assets", [])
if only_ids:
assets = [a for a in assets if a["id"] in only_ids]
# Health check if any SAO assets
sao_assets = [a for a in assets if a.get("method") == "sao"]
if sao_assets and not dry_run:
print(f"Checking SAO API health...", file=sys.stderr)
health_cmd = [sys.executable, os.path.join(script_dir, "audio_connector.py"), "health"]
result = subprocess.run(health_cmd, capture_output=True, text=True)
if result.returncode != 0:
print(json.dumps({"ok": False, "error": "SAO API health check failed",
"details": result.stdout.strip()}))
sys.exit(1)
print(f" SAO API is up.", file=sys.stderr)
total = len(assets)
results = []
success = 0
failed = 0
skipped = 0
print(f"Processing {total} assets from {os.path.basename(manifest_path)}...", file=sys.stderr)
if dry_run:
print(" (dry run — no generation will occur)", file=sys.stderr)
for i, asset in enumerate(assets, 1):
asset_id = asset["id"]
filename = asset["filename"]
method = asset.get("method", "sao")
print(f"\n[{i}/{total}] {asset_id}: {filename} ({method})", file=sys.stderr)
if skip_existing:
ogg_path = os.path.join(output_dir, filename)
if os.path.exists(ogg_path):
print(f" Skipping — already exists", file=sys.stderr)
results.append({"id": asset_id, "status": "skipped", "reason": "exists"})
skipped += 1
continue
if dry_run:
print(f" Would generate: {filename}", file=sys.stderr)
if method == "sao":
print(f" Prompt: {asset.get('prompt', '(none)')[:80]}...", file=sys.stderr)
elif method == "synth":
synth = asset.get("synth", {})
print(f" Synth: {synth.get('fundamental')}Hz, {synth.get('duration')}s",
file=sys.stderr)
results.append({"id": asset_id, "status": "dry_run"})
continue
if method == "sao":
result = run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir)
elif method == "synth":
result = run_synth(asset, manifest, gen_dir, output_dir, script_dir)
else:
result = {"ok": False, "error": f"Unknown method: {method}"}
result["id"] = asset_id
if result.get("ok"):
success += 1
result["status"] = "success"
print(f" OK → {result.get('ogg_file', filename)}", file=sys.stderr)
else:
failed += 1
result["status"] = "failed"
print(f" FAILED: {result.get('error', 'unknown')}", file=sys.stderr)
results.append(result)
# Summary
summary = {
"ok": failed == 0,
"total": total,
"success": success,
"failed": failed,
"skipped": skipped,
"results": results,
}
print(json.dumps(summary, indent=2))
if __name__ == "__main__":
main()
+55 -5
View File
@@ -6,12 +6,13 @@ Talks to the Stable Audio Open Gradio app at tower-of-joy:11500.
Uses the async Gradio API pattern: POST to submit, SSE stream for results.
Usage:
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav]
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav] [--post]
python3 audio_connector.py health
"""
import json
import os
import subprocess
import sys
import time
import urllib.error
@@ -56,7 +57,31 @@ def health():
}, indent=2))
sys.exit(1)
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600):
def post_process(wav_path, ogg_path=None, lufs=-16, quality=6, threshold=-50):
"""Run trim + normalize + convert on a WAV file via audio-post pipeline."""
script_dir = os.path.dirname(os.path.abspath(__file__))
post_script = os.path.join(script_dir, "audio_post.py")
if ogg_path is None:
ogg_path = os.path.splitext(wav_path)[0] + ".ogg"
cmd = [
sys.executable, post_script, "pipeline", wav_path,
"--output", ogg_path,
"--lufs", str(lufs),
"--quality", str(quality),
"--threshold", str(threshold),
]
print(f" Post-processing → {os.path.basename(ogg_path)}...", file=sys.stderr)
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
return {"ok": False, "error": f"Post-processing failed: {result.stderr.strip()}"}
try:
return json.loads(result.stdout)
except json.JSONDecodeError:
return {"ok": True, "output": ogg_path}
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600,
post=False, output_ogg=None):
"""
Generate audio from a text prompt.
@@ -67,6 +92,8 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
cfg: Classifier-free guidance scale (default 7)
output: Output file path (default: auto-named in current directory)
timeout: Maximum wait time in seconds (default 600 = 10 minutes)
post: If True, run trim+normalize+convert after generation
output_ogg: OGG output path when post=True (default: same basename .ogg)
"""
base = get_base_url()
api_url = f"{base}/gradio_api/call/generate_audio"
@@ -216,7 +243,7 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
shutil.copyfileobj(resp, f)
file_size = os.path.getsize(output)
print(json.dumps({
result_json = {
"ok": True,
"file": output,
"size_bytes": file_size,
@@ -225,7 +252,20 @@ def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600
"cfg": cfg,
"prompt": prompt,
"generation_time_s": elapsed
}, indent=2))
}
if post:
post_result = post_process(output, ogg_path=output_ogg)
if not post_result.get("ok"):
result_json["post_processed"] = False
result_json["post_error"] = post_result.get("error", "unknown")
else:
result_json["post_processed"] = True
result_json["ogg_file"] = post_result.get("output", output_ogg)
ogg_size = os.path.getsize(result_json["ogg_file"])
result_json["ogg_size_bytes"] = ogg_size
print(json.dumps(result_json, indent=2))
except Exception as e:
print(json.dumps({
@@ -258,6 +298,8 @@ def main():
cfg = 7.0
output = None
timeout = 600
post = False
output_ogg = None
# Parse optional args
i = 3
@@ -277,11 +319,19 @@ def main():
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
timeout = int(sys.argv[i + 1])
i += 2
elif sys.argv[i] == "--post":
post = True
i += 1
elif sys.argv[i] == "--output-ogg" and i + 1 < len(sys.argv):
output_ogg = sys.argv[i + 1]
post = True # --output-ogg implies --post
i += 2
else:
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
sys.exit(1)
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, timeout=timeout)
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output,
timeout=timeout, post=post, output_ogg=output_ogg)
else:
print(f"Unknown command: {cmd}", file=sys.stderr)
sys.exit(1)
+4 -4
View File
@@ -14,10 +14,10 @@ Stable Audio Open primary. Prompt strategy: describe the SPACE, not the sound.
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|----|----------|--------|-----|--------|----------|-------------|--------|
| AMB-001 | `amb_station_base.ogg` | planned | Ambient | SAO | 45-60s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. | S8+ |
| AMB-002 | `amb_workplace_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. | S8+ |
| AMB-003 | `amb_bar_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. | S8+ |
| AMB-004 | `amb_corridor_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. | S8+ |
| AMB-001 | `amb_station_base.ogg` | done | Ambient | SAO | 43.3s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
| AMB-002 | `amb_workplace_layer.ogg` | done | Ambient | SAO | 42.9s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
| AMB-003 | `amb_bar_layer.ogg` | done | Ambient | SAO | 43.6s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. SAO 45s/150 steps/CFG 6, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
| AMB-004 | `amb_corridor_layer.ogg` | done | Ambient | SAO | 42.7s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. SAO 45s/150 steps/CFG 5, equal-power crossfade loop, LUFS -16 normalized. | S10 #327 |
## Detailed Entries
+2 -2
View File
@@ -13,8 +13,8 @@ Mixed methods depending on duration and character:
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|----|----------|--------|-----|--------|----------|-------------|--------|
| SFX-001 | `sfx_footstep_metal.ogg` | planned | Player Actions | hybrid | 0.3-0.5s | Single footstep on metal grating. Walk pace. Player character proprioceptive feedback. | S8+ |
| SFX-002 | `sfx_footstep_metal_run.ogg` | planned | Player Actions | hybrid | 0.2-0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). | S8+ |
| SFX-001 | `sfx_footstep_metal_walk.ogg` | done | Player Actions | SAO hybrid | 0.3s | Single footstep on metal grating. Walk pace. SAO 5s/100 steps/CFG 8, best transient trimmed at 2.42s, LUFS -16 normalized. | S10 #327 |
| SFX-002 | `sfx_footstep_metal_run.ogg` | done | Player Actions | SAO hybrid | 0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). SAO 5s/100 steps/CFG 8, best transient trimmed at 1.03s, LUFS -16 normalized. | S10 #327 |
## Detailed Entries
+3 -3
View File
@@ -5,7 +5,7 @@ Interface sounds triggered by player interaction, insert systems, and cognitive
## Generation Approach
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`.
- **Monologue chimes:** Re-generated in Sprint 9 (#453). Insert-tech prefix + Chimes template, 5s SAO generation trimmed to 0.8s.
- **Monologue chimes:** Replaced in Sprint 10 (#327) with manual synthesis. Insert-tech aesthetic: pure sine harmonics, mathematical envelope, no SAO. Previous S9 SAO versions were acknowledged placeholders per D-038 amendment.
## Assets
@@ -15,8 +15,8 @@ Interface sounds triggered by player interaction, insert systems, and cognitive
| UI-002 | `implant_open.ogg` | done | UI Sounds | SAO | 217ms | Rising tone when insert UI opens (radial menu, inventory, stance). Neural lattice powering up. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 |
| UI-003 | `fog_recognition.ogg` | done | UI Sounds | SAO | 400ms | Warm organic chime at onset of cognitive delay (D-060). Re-generated in S9 — original was silent. Organic prefix + Chimes template. | S7 #440, S9 #453 |
| UI-004 | `weapon_aim.ogg` | done | UI Sounds | SAO | 127ms | Harder click/lock for weapon aim state. Mechanical, deliberate. Trimmed + normalized in S9 #453. | S7 #440, S9 #453 |
| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template. Crystalline tone, neural lattice surfacing a thought. See design brief below. | S7 #440, S9 #453 |
| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | SAO | 800ms | Re-generated in S9. Insert-tech prefix + Chimes template (sharper/brighter variant). CFG 8. See design brief below. | S7 #440, S9 #453 |
| UI-005 | `sfx_monologue_chime.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1200Hz fundamental + harmonics (2400Hz -12dB, 3600Hz -24dB, 6000Hz -30dB). 15ms attack, 0.2 sustain ratio, exp decay. Peak -15dB. No SAO — pure mathematical precision. | S7 #440, S9 #453, S10 #327 |
| UI-006 | `sfx_monologue_chime_urgent.ogg` | done | UI Sounds | manual synthesis | 800ms | Production-quality manual synthesis in S10. Insert-tech: 1220Hz fundamental (detuned +20Hz for tension) + stronger harmonics (2440Hz -6dB, 3660Hz -18dB, 5087Hz -24dB, inharmonic 5087Hz -28dB). 8ms attack, 0.3 sustain ratio. Peak -6.3dB. | S7 #440, S9 #453, S10 #327 |
| UI-007 | `sfx_weapon_aim_lock.ogg` | done | UI Sounds | SAO | 505ms | Sharp targeting lock-on confirmation for weapon aim. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 |
| UI-008 | `sfx_stance_change.ogg` | done | UI Sounds | SAO | 349ms | Soft mechanical mode-switch click for stance toggle. Downsampled to 44.1kHz + normalized in S9 #453. | S8 #440, S9 #453 |