--- name: audio-gen description: > Generate audio assets for The Settled Reach using the Stable Audio Open API (self-hosted Gradio app at tower-of-joy:11500). Use when generating any game audio: ambient loops, SFX, UI sounds, monologue chimes, footsteps, or any sound asset from docs/assets/audio/. Also use when the user asks about audio generation, sound design pipeline, or audio asset iteration. Triggers on: "generate audio", "make sounds", "create ambient", "audio pipeline", "generate sfx", "stable audio", "gen audio", "sound design". --- # Audio Generation — The Settled Reach Generate sonically consistent audio assets using the Stable Audio Open API via wrapper scripts at `tooling/db/audio-*`. Asset descriptions, filenames, bus routing, and design intent are documented in `docs/assets/audio/`. This skill provides the prompt system, generation workflow, and quality validation. ## API Access **Never call the API directly.** Use the wrapper scripts: ```bash # Check API health tooling/db/audio-health # Generate a single asset (WAV only) tooling/db/audio-generate "prompt text" \ --duration 10 --steps 100 --cfg 7 \ --output path/to/output.wav # Generate + post-process in one command (WAV → trim → normalize → OGG) tooling/db/audio-generate "prompt text" \ --duration 10 --steps 100 --cfg 7 \ --output path/to/gen/intermediate.wav \ --output-ogg client/assets/audio/final.ogg # Batch-generate from a manifest (preferred for multiple assets) tooling/db/audio-batch docs/assets/audio/batch-s10-327.json ``` ### Parameters | Parameter | Default | Range | Notes | |-----------|---------|-------|-------| | `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. | | `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. | | `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. | | `--output` | auto | — | Output WAV file path. Auto-names from prompt if omitted. | | `--post` | off | — | Run trim + normalize + convert after generation. | | `--output-ogg` | auto | — | OGG output path (implies `--post`). Defaults to same basename as WAV. | | `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. | ### Critical Constraints - **NEVER parallelize requests.** The server has 11GB VRAM and runs one generation at a time. Always wait for a generation to complete before starting the next. Sequential only. - **Generation takes 2-5 minutes** per clip depending on duration and steps. Be patient. The timeout default (600s) is generous. - **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s clips and crossfade-stitch in post-processing. ## Prompt Assembly Every generation uses three parts: ``` [SONIC FAMILY PREFIX] + [CATEGORY TEMPLATE] + [ASSET DESCRIPTION from docs/assets/audio/] ``` Never call the API with just the asset description. Always prepend the sonic family prefix and matching category template. - **Sonic palette and families:** Read `references/sonic-palette.md` - **Category templates:** Read `references/category-templates.md` and match by asset type (ambient, sfx, ui) - **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md` ## Batch Workflow (Preferred) For generating multiple assets, use a manifest file. This reduces prompt approvals to 2: one Write (manifest) + one Bash (batch run). ### 1. Create the manifest Write a JSON manifest to `docs/assets/audio/batch-{sprint}-{ticket}.json`: ```json { "description": "Sprint 10 ambient + world SFX batch", "output_dir": "client/assets/audio", "gen_dir": "client/assets/audio/gen", "defaults": { "steps": 100, "cfg": 7, "lufs": -16, "quality": 6 }, "assets": [ { "id": "AMB-001", "filename": "amb_station_base.ogg", "method": "sao", "duration": 45, "steps": 150, "cfg": 5, "prompt": "[sonic family prefix] + [template] + [description]" }, { "id": "UI-005", "filename": "sfx_monologue_chime.ogg", "method": "synth", "synth": { "type": "harmonic", "duration": 0.8, "fundamental": 1200, "harmonics": [ {"freq": 2400, "db": -12}, {"freq": 3600, "db": -24} ], "attack_ms": 15, "sustain_ratio": 0.2, "decay": "exponential" } } ] } ``` Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g., AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory. ### 2. Run the batch ```bash # Full run tooling/db/audio-batch docs/assets/audio/batch-s10-327.json # Dry run — preview what would be generated tooling/db/audio-batch docs/assets/audio/batch-s10-327.json --dry-run # Generate only specific assets tooling/db/audio-batch docs/assets/audio/batch-s10-327.json --only AMB-001,AMB-002 # Skip assets that already have OGG files tooling/db/audio-batch docs/assets/audio/batch-s10-327.json --skip-existing ``` ### 3. Update asset docs with prompts After the batch completes, write the exact prompts used back into the Prompt/Notes column of `docs/assets/audio/{category}.md`. The manifest records what was generated; the asset docs record what we have. ### Manifest fields | Field | Required | Notes | |-------|----------|-------| | `id` | yes | Asset ID from docs (AMB-001, SFX-002, UI-005) | | `filename` | yes | Output filename (must match asset doc) | | `method` | yes | `sao` (Stable Audio Open) or `synth` (harmonic synthesis) | | `duration` | SAO only | Duration in seconds | | `prompt` | SAO only | Full assembled prompt | | `steps` | no | Override default steps | | `cfg` | no | Override default CFG | | `synth` | synth only | Synthesis parameters (see below) | ### Synth parameters | Field | Default | Notes | |-------|---------|-------| | `type` | harmonic | Only `harmonic` supported currently | | `duration` | — | Duration in seconds | | `fundamental` | — | Fundamental frequency in Hz | | `harmonics` | [] | List of `{"freq": Hz, "db": dB}` objects | | `attack_ms` | 10 | Attack time in milliseconds | | `sustain_ratio` | 0.2 | Fraction of duration at full level before decay | | `decay` | exponential | `exponential` or `linear` | ## Single Asset Workflow For one-off generation or iteration on a specific asset: 1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename, duration, bus, method, and design intent. 2. Read `references/sonic-palette.md` for the sonic family prefix. 3. Read `references/category-templates.md` for the matching template. 4. Assemble the full prompt. 5. Run `tooling/db/audio-health` to verify the API is up. 6. Run `tooling/db/audio-generate` with `--post` or `--output-ogg` to generate and post-process in one step. 7. Verify the output (file size, duration). 8. Update the asset status and prompt in `docs/assets/audio/{category}.md`. ## Iteration Workflow For each asset, generate 4-6 candidates: 1. **Generate candidates** — vary the prompt slightly (add/remove descriptors, adjust CFG between 5-9). Run each generation sequentially — never in parallel. 2. **Solo test** — does each candidate sound right alone? 3. **Stack test** — play the candidate alongside other layers. Does it mask or clash? 4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a jarring repeat? 5. **Close-your-eyes test** — does it create a mental image or sensation? 6. Select the best candidate (post-processing is already done if `--post` was used). ## Post-Processing (Standalone) If you need to post-process separately (e.g., re-normalizing an existing file): ```bash # Full pipeline: trim → normalize → convert tooling/db/audio-post pipeline input.wav --output output.ogg # Individual steps tooling/db/audio-post trim input.wav tooling/db/audio-post normalize input.wav --lufs -16 tooling/db/audio-post convert input.wav --output output.ogg ``` ## Manual Synthesis (Insert-Tech Sounds) For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py` or the batch manifest's `method: "synth"` with harmonic parameters. For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass impulse), write a custom script in `tooling/` following the pattern in `tooling/synth_ui_sounds.py`. ## Quality Checklist After generating, verify: - Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural) - Frequency range doesn't mask other layers (check docs/assets/audio/) - Duration matches spec - No unwanted artifacts (clicks, pops, digital noise at start/end) - Loop point is clean (ambient loops only) - Volume sits well relative to other assets (LUFS normalized) - Passes the close-your-eyes test ## File Placement Generated assets go to `client/assets/audio/` with exact filenames from the asset docs. Intermediates go to `client/assets/audio/gen/` (gitignored). AudioManager discovers assets by directory scan — filenames must match exactly.