feat(skills): add gen-audio skill, rename asset-gen to gen-image
gen-audio: prompt assembly system (sonic palette + category templates + asset descriptions), SAO generation workflow, post-processing pipeline, manual synthesis guide for sub-200ms insert-tech sounds. Rename asset-gen to gen-image for consistent gen-* naming. Add audio connector permissions to settings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.6
@@ -36,6 +36,10 @@
|
||||
"Bash(db/connectors/sqlite-init)",
|
||||
"Bash(db/connectors/decisions-sync)",
|
||||
|
||||
"Bash(db/connectors/audio-generate *)",
|
||||
"Bash(db/connectors/audio-health)",
|
||||
"Bash(db/connectors/audio-post *)",
|
||||
|
||||
"Bash(make *)",
|
||||
"Bash(make)",
|
||||
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
---
|
||||
name: gen-audio
|
||||
description: >
|
||||
Generate audio assets for The Settled Reach using the Stable Audio Open API
|
||||
(self-hosted Gradio app at tower-of-joy:11500). Use when generating any game
|
||||
audio: ambient loops, SFX, UI sounds, monologue chimes, footsteps, or any
|
||||
sound asset from docs/assets/audio/. Also use when the user asks about audio
|
||||
generation, sound design pipeline, or audio asset iteration. Triggers on:
|
||||
"generate audio", "make sounds", "create ambient", "audio pipeline",
|
||||
"generate sfx", "stable audio", "gen audio", "sound design".
|
||||
---
|
||||
|
||||
# Audio Generation — The Settled Reach
|
||||
|
||||
Generate sonically consistent audio assets using the Stable Audio Open API via
|
||||
wrapper scripts at `db/connectors/audio-*`.
|
||||
|
||||
Asset descriptions, filenames, bus routing, and design intent are documented in
|
||||
`docs/assets/audio/`. This skill provides the prompt system, generation
|
||||
workflow, and quality validation.
|
||||
|
||||
## API Access
|
||||
|
||||
**Never call the API directly.** Use the wrapper scripts:
|
||||
|
||||
```bash
|
||||
# Check API health
|
||||
db/connectors/audio-health
|
||||
|
||||
# Generate audio
|
||||
db/connectors/audio-generate "prompt text" \
|
||||
--duration 10 \
|
||||
--steps 100 \
|
||||
--cfg 7 \
|
||||
--output path/to/output.wav \
|
||||
--timeout 600
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Default | Range | Notes |
|
||||
|-----------|---------|-------|-------|
|
||||
| `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. |
|
||||
| `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. |
|
||||
| `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. |
|
||||
| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. |
|
||||
| `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. |
|
||||
|
||||
### Critical Constraints
|
||||
|
||||
- **NEVER parallelize requests.** The server has 11GB VRAM and runs one
|
||||
generation at a time. Always wait for a generation to complete before
|
||||
starting the next. Sequential only.
|
||||
- **Generation takes 2-5 minutes** per clip depending on duration and steps.
|
||||
Be patient. The timeout default (600s) is generous.
|
||||
- **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s
|
||||
clips and crossfade-stitch in post-processing.
|
||||
- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import:
|
||||
`ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg`
|
||||
|
||||
## Prompt Assembly
|
||||
|
||||
Every generation uses three parts:
|
||||
|
||||
```
|
||||
[SONIC FAMILY PREFIX] + [CATEGORY TEMPLATE] + [ASSET DESCRIPTION from docs/assets/audio/]
|
||||
```
|
||||
|
||||
Never call the API with just the asset description. Always prepend the sonic
|
||||
family prefix and matching category template.
|
||||
|
||||
- **Sonic palette and families:** Read `references/sonic-palette.md`
|
||||
- **Category templates:** Read `references/category-templates.md` and match by
|
||||
asset type (ambient, sfx, ui)
|
||||
- **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md`
|
||||
|
||||
## Single Asset Workflow
|
||||
|
||||
1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename,
|
||||
duration, bus, method, and design intent.
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `db/connectors/audio-health` to verify the API is up.
|
||||
6. Run `db/connectors/audio-generate` with the assembled prompt. **One request
|
||||
at a time. Wait for completion.**
|
||||
7. Listen to the output (or describe it based on file size/duration).
|
||||
8. If acceptable, convert to .ogg and place in `client/assets/audio/`.
|
||||
9. Update the asset status in `docs/assets/audio/{category}.md`.
|
||||
|
||||
## Iteration Workflow
|
||||
|
||||
For each asset, generate 4-6 candidates:
|
||||
|
||||
1. **Generate candidates** — vary the prompt slightly (add/remove descriptors,
|
||||
adjust CFG between 5-9). Run each generation sequentially — never in
|
||||
parallel.
|
||||
2. **Solo test** — does each candidate sound right alone?
|
||||
3. **Stack test** — play the candidate alongside other layers. Does it mask or
|
||||
clash?
|
||||
4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a
|
||||
jarring repeat?
|
||||
5. **Close-your-eyes test** — does it create a mental image or sensation?
|
||||
6. Select the best candidate, trim, normalize, convert.
|
||||
|
||||
## Post-Processing
|
||||
|
||||
After selecting the best generation:
|
||||
|
||||
```bash
|
||||
# Trim silence from start/end
|
||||
ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav
|
||||
|
||||
# LUFS normalize to -16 LUFS (broadcast standard, good for game audio)
|
||||
ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav
|
||||
|
||||
# Convert to .ogg for Godot
|
||||
ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg
|
||||
|
||||
# For loops: verify loop point
|
||||
ffplay -loop 0 output.ogg
|
||||
```
|
||||
|
||||
For ambient loops, create crossfade overlap:
|
||||
```bash
|
||||
# Create a 45s loop with 3s crossfade overlap
|
||||
# (manual: export 48s, crossfade first 3s with last 3s in Audacity)
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis instead:
|
||||
|
||||
```python
|
||||
# Example: 50ms cursor hover tick
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
sr = 44100
|
||||
duration = 0.05 # 50ms
|
||||
t = np.linspace(0, duration, int(sr * duration), endpoint=False)
|
||||
freq = 3200 # Hz
|
||||
signal = np.sin(2 * np.pi * freq * t)
|
||||
envelope = np.exp(-t * 80) # exponential decay
|
||||
audio = (signal * envelope * 32767).astype(np.int16)
|
||||
|
||||
with wave.open("cursor_hover.wav", "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(audio.tobytes())
|
||||
```
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
After generating, verify:
|
||||
- Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural)
|
||||
- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md)
|
||||
- Duration matches spec
|
||||
- No unwanted artifacts (clicks, pops, digital noise at start/end)
|
||||
- Loop point is clean (ambient loops only)
|
||||
- Volume sits well relative to other assets (LUFS normalized)
|
||||
- Passes the close-your-eyes test
|
||||
|
||||
## File Placement
|
||||
|
||||
Generated assets go to `client/assets/audio/` with exact filenames from the
|
||||
asset docs:
|
||||
|
||||
```
|
||||
client/assets/audio/
|
||||
amb_station_base.ogg # Ambient bus
|
||||
amb_workplace_layer.ogg # Ambient bus
|
||||
amb_bar_layer.ogg # Ambient bus
|
||||
amb_corridor_layer.ogg # Ambient bus
|
||||
sfx_footstep_metal.ogg # Player Actions bus
|
||||
sfx_footstep_metal_run.ogg # Player Actions bus
|
||||
cursor_hover.ogg # UI Sounds bus
|
||||
implant_open.ogg # UI Sounds bus
|
||||
fog_recognition.ogg # UI Sounds bus
|
||||
weapon_aim.ogg # UI Sounds bus
|
||||
sfx_monologue_chime.ogg # UI Sounds bus
|
||||
sfx_monologue_chime_urgent.ogg # UI Sounds bus
|
||||
```
|
||||
|
||||
AudioManager discovers these by directory scan — filenames must match exactly.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Category Templates — Audio Generation
|
||||
|
||||
Match the asset's category to the right template. Append after the sonic family
|
||||
prefix and before the specific asset description.
|
||||
|
||||
## Ambient Loops
|
||||
|
||||
**Template:**
|
||||
> Continuous ambient soundscape loop, seamless looping audio, no distinct
|
||||
> beginning or end, steady background atmosphere, [DURATION]s duration
|
||||
|
||||
**Notes:**
|
||||
- Generate at 45s (the SAO sweet spot below the 47s ceiling)
|
||||
- Add 3-5s of overlap material for crossfade looping in post
|
||||
- These play continuously — they must be boring enough to fade into the
|
||||
background but rich enough to reward attention
|
||||
- Station baseline prefix is required (see sonic-palette.md)
|
||||
|
||||
**Steps:** 100-150 (higher quality for loops — artifacts are more noticeable)
|
||||
**CFG:** 5-7 (lower guidance = more natural variation, less mechanical)
|
||||
|
||||
## Sound Effects (SFX)
|
||||
|
||||
**Template:**
|
||||
> Single isolated sound effect, clean recording, [DURATION] duration,
|
||||
> clear attack and natural decay, no background noise, no reverb tail
|
||||
> beyond natural
|
||||
|
||||
**Notes:**
|
||||
- For footsteps: specify surface material, pace, weight
|
||||
- For environmental: specify the physical mechanism (door hinge, cargo latch)
|
||||
- Keep short and punchy — these are event-driven, not continuous
|
||||
|
||||
**Steps:** 80-100 (shorter sounds need fewer steps)
|
||||
**CFG:** 7-9 (more guidance = more precise sound matching)
|
||||
|
||||
## UI Sounds
|
||||
|
||||
**Template:**
|
||||
> Interface feedback sound, digital UI element, [DURATION] duration,
|
||||
> immediate attack, clean decay, isolated sound with no background
|
||||
|
||||
**Notes:**
|
||||
- Most UI sounds are under 200ms — use manual synthesis, not SAO
|
||||
- For SAO-generated UI sounds (>200ms like fog_recognition, chimes):
|
||||
generate at 3-5s duration, then trim to the best portion
|
||||
- Insert-tech prefix is required for lattice/interface sounds
|
||||
- Organic prefix for fog_recognition (cognitive, not technological)
|
||||
|
||||
**Steps:** 80-100
|
||||
**CFG:** 7-9 (precision matters for UI sounds)
|
||||
|
||||
## Chimes / Notification Sounds
|
||||
|
||||
**Template:**
|
||||
> Single musical tone, crystalline quality, [DURATION] duration, gentle
|
||||
> attack, resonant sustain, natural fade out, no accompaniment, no rhythm
|
||||
|
||||
**Notes:**
|
||||
- Generate at 3-5s to give SAO enough temporal context
|
||||
- Trim to the best 0.5-1.0s section
|
||||
- The attack transient is where the character lives — select for that
|
||||
- Normal vs urgent variants: same base prompt, vary "gentle/soft" vs
|
||||
"brighter/sharper/more present"
|
||||
|
||||
**Steps:** 100 (tonal quality matters)
|
||||
**CFG:** 6-8 (some freedom for natural harmonic content)
|
||||
@@ -0,0 +1,45 @@
|
||||
# Sonic Palette — Prompt Prefixes
|
||||
|
||||
Two sonic families. Every generated sound uses one of these as a prompt prefix.
|
||||
|
||||
## Insert-Tech (Synthetic)
|
||||
|
||||
For UI sounds, neural lattice interface, augmented cognition.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Clean digital audio, synthetic electronic tone, precise and clinical sound
|
||||
> design, no reverb, no room ambience, studio-dry recording, futuristic
|
||||
> interface sound
|
||||
|
||||
**Character:** Mathematical precision. The sound of well-designed technology.
|
||||
No organic texture, no room reflections. Exists "inside the head," not in
|
||||
physical space.
|
||||
|
||||
**Frequency range:** 800Hz-4kHz primary. Sharp attack, controlled decay.
|
||||
|
||||
**Used for:** cursor_hover, implant_open, weapon_aim, sfx_monologue_chime,
|
||||
sfx_monologue_chime_urgent
|
||||
|
||||
## Organic (Biological/Environmental)
|
||||
|
||||
For environmental sounds, human cognition, physical world.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Warm organic audio, natural sound recording, slight room ambience,
|
||||
> lived-in industrial space, realistic acoustic properties, authentic
|
||||
> environmental sound
|
||||
|
||||
**Character:** Human warmth. Natural attack and decay. Reverberant — exists
|
||||
in physical space. The sound of a real place with real materials.
|
||||
|
||||
**Frequency range:** 60Hz-3kHz primary. Soft or natural attack, room tail.
|
||||
|
||||
**Used for:** fog_recognition, all amb_* ambient loops, all sfx_footstep_*
|
||||
|
||||
## Station Baseline
|
||||
|
||||
For ambient loops specifically, add the station context:
|
||||
|
||||
> Interior of a large industrial space station, metal and composite
|
||||
> construction, background mechanical hum from power systems, air circulation
|
||||
> audible, no music, no prominent voices unless specified
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
name: asset-gen
|
||||
name: gen-image
|
||||
description: >
|
||||
Generate themed visual assets for the Lords of Ash CK3 total conversion mod
|
||||
using the generate_image MCP tool (Nano Banana / Gemini 2.5 Flash Image).
|
||||
|
||||
Reference in New Issue
Block a user