Merge remote-tracking branch 'origin/audio'
# Conflicts: # CHANGELOG.md
This commit is contained in:
@@ -36,6 +36,10 @@
|
||||
"Bash(db/connectors/sqlite-init)",
|
||||
"Bash(db/connectors/decisions-sync)",
|
||||
|
||||
"Bash(db/connectors/audio-generate *)",
|
||||
"Bash(db/connectors/audio-health)",
|
||||
"Bash(db/connectors/audio-post *)",
|
||||
|
||||
"Bash(make *)",
|
||||
"Bash(make)",
|
||||
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
---
|
||||
name: gen-audio
|
||||
description: >
|
||||
Generate audio assets for The Settled Reach using the Stable Audio Open API
|
||||
(self-hosted Gradio app at tower-of-joy:11500). Use when generating any game
|
||||
audio: ambient loops, SFX, UI sounds, monologue chimes, footsteps, or any
|
||||
sound asset from docs/assets/audio/. Also use when the user asks about audio
|
||||
generation, sound design pipeline, or audio asset iteration. Triggers on:
|
||||
"generate audio", "make sounds", "create ambient", "audio pipeline",
|
||||
"generate sfx", "stable audio", "gen audio", "sound design".
|
||||
---
|
||||
|
||||
# Audio Generation — The Settled Reach
|
||||
|
||||
Generate sonically consistent audio assets using the Stable Audio Open API via
|
||||
wrapper scripts at `db/connectors/audio-*`.
|
||||
|
||||
Asset descriptions, filenames, bus routing, and design intent are documented in
|
||||
`docs/assets/audio/`. This skill provides the prompt system, generation
|
||||
workflow, and quality validation.
|
||||
|
||||
## API Access
|
||||
|
||||
**Never call the API directly.** Use the wrapper scripts:
|
||||
|
||||
```bash
|
||||
# Check API health
|
||||
db/connectors/audio-health
|
||||
|
||||
# Generate audio
|
||||
db/connectors/audio-generate "prompt text" \
|
||||
--duration 10 \
|
||||
--steps 100 \
|
||||
--cfg 7 \
|
||||
--output path/to/output.wav \
|
||||
--timeout 600
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Default | Range | Notes |
|
||||
|-----------|---------|-------|-------|
|
||||
| `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. |
|
||||
| `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. |
|
||||
| `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. |
|
||||
| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. |
|
||||
| `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. |
|
||||
|
||||
### Critical Constraints
|
||||
|
||||
- **NEVER parallelize requests.** The server has 11GB VRAM and runs one
|
||||
generation at a time. Always wait for a generation to complete before
|
||||
starting the next. Sequential only.
|
||||
- **Generation takes 2-5 minutes** per clip depending on duration and steps.
|
||||
Be patient. The timeout default (600s) is generous.
|
||||
- **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s
|
||||
clips and crossfade-stitch in post-processing.
|
||||
- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import:
|
||||
`ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg`
|
||||
|
||||
## Prompt Assembly
|
||||
|
||||
Every generation uses three parts:
|
||||
|
||||
```
|
||||
[SONIC FAMILY PREFIX] + [CATEGORY TEMPLATE] + [ASSET DESCRIPTION from docs/assets/audio/]
|
||||
```
|
||||
|
||||
Never call the API with just the asset description. Always prepend the sonic
|
||||
family prefix and matching category template.
|
||||
|
||||
- **Sonic palette and families:** Read `references/sonic-palette.md`
|
||||
- **Category templates:** Read `references/category-templates.md` and match by
|
||||
asset type (ambient, sfx, ui)
|
||||
- **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md`
|
||||
|
||||
## Single Asset Workflow
|
||||
|
||||
1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename,
|
||||
duration, bus, method, and design intent.
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `db/connectors/audio-health` to verify the API is up.
|
||||
6. Run `db/connectors/audio-generate` with the assembled prompt. **One request
|
||||
at a time. Wait for completion.**
|
||||
7. Listen to the output (or describe it based on file size/duration).
|
||||
8. If acceptable, convert to .ogg and place in `client/assets/audio/`.
|
||||
9. Update the asset status in `docs/assets/audio/{category}.md`.
|
||||
|
||||
## Iteration Workflow
|
||||
|
||||
For each asset, generate 4-6 candidates:
|
||||
|
||||
1. **Generate candidates** — vary the prompt slightly (add/remove descriptors,
|
||||
adjust CFG between 5-9). Run each generation sequentially — never in
|
||||
parallel.
|
||||
2. **Solo test** — does each candidate sound right alone?
|
||||
3. **Stack test** — play the candidate alongside other layers. Does it mask or
|
||||
clash?
|
||||
4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a
|
||||
jarring repeat?
|
||||
5. **Close-your-eyes test** — does it create a mental image or sensation?
|
||||
6. Select the best candidate, trim, normalize, convert.
|
||||
|
||||
## Post-Processing
|
||||
|
||||
After selecting the best generation:
|
||||
|
||||
```bash
|
||||
# Trim silence from start/end
|
||||
ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav
|
||||
|
||||
# LUFS normalize to -16 LUFS (broadcast standard, good for game audio)
|
||||
ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav
|
||||
|
||||
# Convert to .ogg for Godot
|
||||
ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg
|
||||
|
||||
# For loops: verify loop point
|
||||
ffplay -loop 0 output.ogg
|
||||
```
|
||||
|
||||
For ambient loops, create crossfade overlap:
|
||||
```bash
|
||||
# Create a 45s loop with 3s crossfade overlap
|
||||
# (manual: export 48s, crossfade first 3s with last 3s in Audacity)
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis instead:
|
||||
|
||||
```python
|
||||
# Example: 50ms cursor hover tick
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
sr = 44100
|
||||
duration = 0.05 # 50ms
|
||||
t = np.linspace(0, duration, int(sr * duration), endpoint=False)
|
||||
freq = 3200 # Hz
|
||||
signal = np.sin(2 * np.pi * freq * t)
|
||||
envelope = np.exp(-t * 80) # exponential decay
|
||||
audio = (signal * envelope * 32767).astype(np.int16)
|
||||
|
||||
with wave.open("cursor_hover.wav", "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(audio.tobytes())
|
||||
```
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
After generating, verify:
|
||||
- Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural)
|
||||
- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md)
|
||||
- Duration matches spec
|
||||
- No unwanted artifacts (clicks, pops, digital noise at start/end)
|
||||
- Loop point is clean (ambient loops only)
|
||||
- Volume sits well relative to other assets (LUFS normalized)
|
||||
- Passes the close-your-eyes test
|
||||
|
||||
## File Placement
|
||||
|
||||
Generated assets go to `client/assets/audio/` with exact filenames from the
|
||||
asset docs:
|
||||
|
||||
```
|
||||
client/assets/audio/
|
||||
amb_station_base.ogg # Ambient bus
|
||||
amb_workplace_layer.ogg # Ambient bus
|
||||
amb_bar_layer.ogg # Ambient bus
|
||||
amb_corridor_layer.ogg # Ambient bus
|
||||
sfx_footstep_metal.ogg # Player Actions bus
|
||||
sfx_footstep_metal_run.ogg # Player Actions bus
|
||||
cursor_hover.ogg # UI Sounds bus
|
||||
implant_open.ogg # UI Sounds bus
|
||||
fog_recognition.ogg # UI Sounds bus
|
||||
weapon_aim.ogg # UI Sounds bus
|
||||
sfx_monologue_chime.ogg # UI Sounds bus
|
||||
sfx_monologue_chime_urgent.ogg # UI Sounds bus
|
||||
```
|
||||
|
||||
AudioManager discovers these by directory scan — filenames must match exactly.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Category Templates — Audio Generation
|
||||
|
||||
Match the asset's category to the right template. Append after the sonic family
|
||||
prefix and before the specific asset description.
|
||||
|
||||
## Ambient Loops
|
||||
|
||||
**Template:**
|
||||
> Continuous ambient soundscape loop, seamless looping audio, no distinct
|
||||
> beginning or end, steady background atmosphere, [DURATION]s duration
|
||||
|
||||
**Notes:**
|
||||
- Generate at 45s (the SAO sweet spot below the 47s ceiling)
|
||||
- Add 3-5s of overlap material for crossfade looping in post
|
||||
- These play continuously — they must be boring enough to fade into the
|
||||
background but rich enough to reward attention
|
||||
- Station baseline prefix is required (see sonic-palette.md)
|
||||
|
||||
**Steps:** 100-150 (higher quality for loops — artifacts are more noticeable)
|
||||
**CFG:** 5-7 (lower guidance = more natural variation, less mechanical)
|
||||
|
||||
## Sound Effects (SFX)
|
||||
|
||||
**Template:**
|
||||
> Single isolated sound effect, clean recording, [DURATION] duration,
|
||||
> clear attack and natural decay, no background noise, no reverb tail
|
||||
> beyond natural
|
||||
|
||||
**Notes:**
|
||||
- For footsteps: specify surface material, pace, weight
|
||||
- For environmental: specify the physical mechanism (door hinge, cargo latch)
|
||||
- Keep short and punchy — these are event-driven, not continuous
|
||||
|
||||
**Steps:** 80-100 (shorter sounds need fewer steps)
|
||||
**CFG:** 7-9 (more guidance = more precise sound matching)
|
||||
|
||||
## UI Sounds
|
||||
|
||||
**Template:**
|
||||
> Interface feedback sound, digital UI element, [DURATION] duration,
|
||||
> immediate attack, clean decay, isolated sound with no background
|
||||
|
||||
**Notes:**
|
||||
- Most UI sounds are under 200ms — use manual synthesis, not SAO
|
||||
- For SAO-generated UI sounds (>200ms like fog_recognition, chimes):
|
||||
generate at 3-5s duration, then trim to the best portion
|
||||
- Insert-tech prefix is required for lattice/interface sounds
|
||||
- Organic prefix for fog_recognition (cognitive, not technological)
|
||||
|
||||
**Steps:** 80-100
|
||||
**CFG:** 7-9 (precision matters for UI sounds)
|
||||
|
||||
## Chimes / Notification Sounds
|
||||
|
||||
**Template:**
|
||||
> Single musical tone, crystalline quality, [DURATION] duration, gentle
|
||||
> attack, resonant sustain, natural fade out, no accompaniment, no rhythm
|
||||
|
||||
**Notes:**
|
||||
- Generate at 3-5s to give SAO enough temporal context
|
||||
- Trim to the best 0.5-1.0s section
|
||||
- The attack transient is where the character lives — select for that
|
||||
- Normal vs urgent variants: same base prompt, vary "gentle/soft" vs
|
||||
"brighter/sharper/more present"
|
||||
|
||||
**Steps:** 100 (tonal quality matters)
|
||||
**CFG:** 6-8 (some freedom for natural harmonic content)
|
||||
@@ -0,0 +1,45 @@
|
||||
# Sonic Palette — Prompt Prefixes
|
||||
|
||||
Two sonic families. Every generated sound uses one of these as a prompt prefix.
|
||||
|
||||
## Insert-Tech (Synthetic)
|
||||
|
||||
For UI sounds, neural lattice interface, augmented cognition.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Clean digital audio, synthetic electronic tone, precise and clinical sound
|
||||
> design, no reverb, no room ambience, studio-dry recording, futuristic
|
||||
> interface sound
|
||||
|
||||
**Character:** Mathematical precision. The sound of well-designed technology.
|
||||
No organic texture, no room reflections. Exists "inside the head," not in
|
||||
physical space.
|
||||
|
||||
**Frequency range:** 800Hz-4kHz primary. Sharp attack, controlled decay.
|
||||
|
||||
**Used for:** cursor_hover, implant_open, weapon_aim, sfx_monologue_chime,
|
||||
sfx_monologue_chime_urgent
|
||||
|
||||
## Organic (Biological/Environmental)
|
||||
|
||||
For environmental sounds, human cognition, physical world.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Warm organic audio, natural sound recording, slight room ambience,
|
||||
> lived-in industrial space, realistic acoustic properties, authentic
|
||||
> environmental sound
|
||||
|
||||
**Character:** Human warmth. Natural attack and decay. Reverberant — exists
|
||||
in physical space. The sound of a real place with real materials.
|
||||
|
||||
**Frequency range:** 60Hz-3kHz primary. Soft or natural attack, room tail.
|
||||
|
||||
**Used for:** fog_recognition, all amb_* ambient loops, all sfx_footstep_*
|
||||
|
||||
## Station Baseline
|
||||
|
||||
For ambient loops specifically, add the station context:
|
||||
|
||||
> Interior of a large industrial space station, metal and composite
|
||||
> construction, background mechanical hum from power systems, air circulation
|
||||
> audible, no music, no prominent voices unless specified
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
name: asset-gen
|
||||
name: gen-image
|
||||
description: >
|
||||
Generate themed visual assets for the Lords of Ash CK3 total conversion mod
|
||||
using the generate_image MCP tool (Nano Banana / Gemini 2.5 Flash Image).
|
||||
@@ -13,6 +13,13 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
- YAML content loader with hot-reload (#326, D-028) — LinePool system parsing dialogue/monologue YAML into BTreeMap-indexed pools, 4-layer query filtering (access > situation > trust > topic+mood), timestamp-polling hot-reload (dev-only), graceful failure preserves previous content
|
||||
- Line pool format specification (#308) — formal spec at docs/architecture/line-pool-format.md defining YAML structure, tag enums, 4-layer filtering pipeline, prerequisite-to-KG mapping, ID format, and Rust loader interface
|
||||
- InteractionMemory KG schema design (#442, D-064) — design doc at docs/architecture/interaction-memory-schema.md extending FactKnowledge with interaction tracking, 5-state InteractionState enum, monologue prerequisite extension, NpcTolerance reconciliation
|
||||
- Audio discussion decisions D-067 through D-074: recognition chime timing, 5-bus architecture, audio dip profiles, confrontation as cognitive vulnerability, monologue chime placeholder strategy, universal conversation murmur, zone crossfade, hybrid audio generation
|
||||
- Asset pipeline documentation system (docs/assets/) with category-based index, sonic palette, and generation templates for audio, visual, and video pipelines
|
||||
- Stable Audio Open connector and post-processing wrappers (audio-generate, audio-health, audio-post) with timeout handling for 11GB VRAM constraint
|
||||
- gen-audio skill with prompt assembly system (sonic palette prefixes + category templates + asset descriptions)
|
||||
- 6 interaction UI audio assets (#440): cursor_hover, weapon_aim, implant_open, fog_recognition, sfx_monologue_chime, sfx_monologue_chime_urgent — generated via SAO, needs duration trimming (#453)
|
||||
- Dialogue/confrontation ambient dip implementation spec with full Godot AudioBus tween code
|
||||
- Synthesis tooling (tooling/synth_ui_sounds.py) for programmatic insert-tech sound generation
|
||||
|
||||
### Changed
|
||||
- Protocol version bumped from 6 to 7 (pending_recognitions field in ObserverSnapshot)
|
||||
@@ -20,6 +27,7 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
- Monologue trigger system uses .values() iterator (clippy fix)
|
||||
- Monologue schema: relationship prerequisite now requires target and state fields
|
||||
- 389 tests total (70 new) — cognitive delay pipeline, line pool loader, ListeningFocus, content watching, serialization
|
||||
- Renamed asset-gen skill to gen-image for consistent gen-* naming pattern
|
||||
- Client protocol v6 bridge — decode player_stance (4 variants) and player_inventory from ObserverSnapshot, TOGGLE_STANCE_UP/DOWN input actions, 25 gdUnit4 tests
|
||||
- Three-scope z-layer rendering pipeline (D-049) — world z:0-900 inside CanvasGroup, insert overlay CanvasLayer 10, UI CanvasLayer 20, modal CanvasLayer 30. Y-sort contract enforced, reserved VFX/airborne/lower-floor ranges documented
|
||||
- Cursor state machine (#429, D-056) — 4 states (Default/EntityHover/ObjectHover/WeaponAim), 150ms transitions, insert-styled geometric shapes
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generate audio via Stable Audio Open. Whitelistable command.
|
||||
# Usage: audio-generate "prompt text" [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" generate "$@"
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
# Check if the Stable Audio Open API is reachable. Whitelistable command.
|
||||
# Usage: audio-health
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" health
|
||||
Executable
+7
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Audio post-processing (ffmpeg wrapper). Whitelistable command.
|
||||
# Usage: audio-post convert input.wav [--output output.ogg]
|
||||
# audio-post normalize input.wav [--lufs -16]
|
||||
# audio-post trim input.wav [--threshold -50]
|
||||
# audio-post pipeline input.wav [--output output.ogg]
|
||||
exec python3 "$(dirname "$0")/audio_post.py" "$@"
|
||||
@@ -0,0 +1,290 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Stable Audio Open connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Stable Audio Open Gradio app at tower-of-joy:11500.
|
||||
Uses the async Gradio API pattern: POST to submit, SSE stream for results.
|
||||
|
||||
Usage:
|
||||
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav]
|
||||
python3 audio_connector.py health
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import shutil
|
||||
|
||||
CONFIG_PATH = os.path.join(os.path.dirname(__file__), "config.json")
|
||||
|
||||
def load_config():
|
||||
with open(CONFIG_PATH) as f:
|
||||
return json.load(f)
|
||||
|
||||
def get_base_url():
|
||||
config = load_config()
|
||||
return config.get("stable_audio_url", "http://tower-of-joy:11500")
|
||||
|
||||
def health():
|
||||
"""Check if the Stable Audio API is reachable."""
|
||||
base = get_base_url()
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/config", method="GET")
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
version = data.get("version", "unknown")
|
||||
# Extract component info for the generate endpoint
|
||||
api_names = []
|
||||
for dep in data.get("dependencies", []):
|
||||
name = dep.get("api_name", "")
|
||||
if name and not name.startswith("js_"):
|
||||
api_names.append(name)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"url": base,
|
||||
"gradio_version": version,
|
||||
"api_endpoints": api_names
|
||||
}, indent=2))
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"url": base,
|
||||
"error": str(e)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600):
|
||||
"""
|
||||
Generate audio from a text prompt.
|
||||
|
||||
Args:
|
||||
prompt: Text description of the audio to generate
|
||||
duration: Duration in seconds (0-47, default 10)
|
||||
steps: Number of diffusion steps (default 100, lower = faster but lower quality)
|
||||
cfg: Classifier-free guidance scale (default 7)
|
||||
output: Output file path (default: auto-named in current directory)
|
||||
timeout: Maximum wait time in seconds (default 600 = 10 minutes)
|
||||
"""
|
||||
base = get_base_url()
|
||||
api_url = f"{base}/gradio_api/call/generate_audio"
|
||||
|
||||
if output is None:
|
||||
# Auto-name: sanitize prompt to a filename
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
output = f"{safe}_{int(duration)}s.wav"
|
||||
|
||||
# Step 1: Submit the generation request
|
||||
payload = json.dumps({"data": [prompt, duration, steps, cfg]})
|
||||
req = urllib.request.Request(
|
||||
api_url,
|
||||
data=payload.encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print(f"Submitting generation request...", file=sys.stderr)
|
||||
print(f" Prompt: {prompt}", file=sys.stderr)
|
||||
print(f" Duration: {duration}s, Steps: {steps}, CFG: {cfg}", file=sys.stderr)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
result = json.loads(resp.read())
|
||||
event_id = result.get("event_id")
|
||||
if not event_id:
|
||||
print(json.dumps({"ok": False, "error": "No event_id returned", "response": result}, indent=2))
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Submit failed: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
print(f" Event ID: {event_id}", file=sys.stderr)
|
||||
print(f" Waiting for generation (timeout: {timeout}s)...", file=sys.stderr)
|
||||
|
||||
# Step 2: Poll the SSE stream for results
|
||||
stream_url = f"{api_url}/{event_id}"
|
||||
start_time = time.time()
|
||||
result_data = None
|
||||
last_status = None
|
||||
|
||||
while time.time() - start_time < timeout:
|
||||
try:
|
||||
req = urllib.request.Request(stream_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
# Read SSE events
|
||||
current_event = None
|
||||
for line_bytes in resp:
|
||||
line = line_bytes.decode("utf-8").strip()
|
||||
|
||||
if line.startswith("event: "):
|
||||
current_event = line[7:]
|
||||
elif line.startswith("data: ") and current_event:
|
||||
data_str = line[6:]
|
||||
|
||||
if current_event == "heartbeat":
|
||||
elapsed = int(time.time() - start_time)
|
||||
if elapsed % 30 == 0 and elapsed > 0:
|
||||
print(f" Still generating... ({elapsed}s elapsed)", file=sys.stderr)
|
||||
continue
|
||||
|
||||
if current_event == "error":
|
||||
error_msg = data_str
|
||||
try:
|
||||
error_msg = json.loads(data_str)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
print(json.dumps({"ok": False, "error": "Generation failed", "details": error_msg}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if current_event == "complete":
|
||||
try:
|
||||
result_data = json.loads(data_str)
|
||||
except json.JSONDecodeError:
|
||||
result_data = data_str
|
||||
break
|
||||
|
||||
if current_event == "progress":
|
||||
try:
|
||||
progress = json.loads(data_str)
|
||||
# Gradio progress events vary; log what we get
|
||||
status = str(progress)[:80]
|
||||
if status != last_status:
|
||||
print(f" Progress: {status}", file=sys.stderr)
|
||||
last_status = status
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
continue
|
||||
|
||||
if result_data is not None:
|
||||
break
|
||||
|
||||
except urllib.error.URLError as e:
|
||||
# Connection dropped — retry after brief pause
|
||||
time.sleep(2)
|
||||
continue
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Stream error: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if result_data is None:
|
||||
print(json.dumps({"ok": False, "error": f"Generation timed out after {timeout}s"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Step 3: Download the audio file
|
||||
# Gradio returns file info in the data array
|
||||
elapsed = round(time.time() - start_time, 1)
|
||||
print(f" Generation complete ({elapsed}s)", file=sys.stderr)
|
||||
|
||||
try:
|
||||
# result_data is typically [{"path": "...", "url": "...", ...}] or similar
|
||||
if isinstance(result_data, list) and len(result_data) > 0:
|
||||
audio_info = result_data[0]
|
||||
elif isinstance(result_data, dict) and "data" in result_data:
|
||||
audio_info = result_data["data"][0] if result_data["data"] else None
|
||||
else:
|
||||
audio_info = result_data
|
||||
|
||||
# Extract the file URL
|
||||
file_url = None
|
||||
if isinstance(audio_info, dict):
|
||||
file_url = audio_info.get("url") or audio_info.get("path")
|
||||
elif isinstance(audio_info, str):
|
||||
file_url = audio_info
|
||||
|
||||
if not file_url:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "Could not extract audio URL from response",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Handle relative URLs
|
||||
if file_url.startswith("/"):
|
||||
file_url = f"{base}{file_url}"
|
||||
elif not file_url.startswith("http"):
|
||||
file_url = f"{base}/file={file_url}"
|
||||
|
||||
# Download the file
|
||||
print(f" Downloading to {output}...", file=sys.stderr)
|
||||
req = urllib.request.Request(file_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
with open(output, "wb") as f:
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
file_size = os.path.getsize(output)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
"duration_requested": duration,
|
||||
"steps": steps,
|
||||
"cfg": cfg,
|
||||
"prompt": prompt,
|
||||
"generation_time_s": elapsed
|
||||
}, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"Download failed: {e}",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage:")
|
||||
print(" audio_connector.py health")
|
||||
print(" audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]")
|
||||
sys.exit(1)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "health":
|
||||
health()
|
||||
elif cmd == "generate":
|
||||
if len(sys.argv) < 3:
|
||||
print("Error: prompt required", file=sys.stderr)
|
||||
print("Usage: audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
prompt = sys.argv[2]
|
||||
duration = 10.0
|
||||
steps = 100
|
||||
cfg = 7.0
|
||||
output = None
|
||||
timeout = 600
|
||||
|
||||
# Parse optional args
|
||||
i = 3
|
||||
while i < len(sys.argv):
|
||||
if sys.argv[i] == "--duration" and i + 1 < len(sys.argv):
|
||||
duration = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--steps" and i + 1 < len(sys.argv):
|
||||
steps = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--cfg" and i + 1 < len(sys.argv):
|
||||
cfg = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--output" and i + 1 < len(sys.argv):
|
||||
output = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
|
||||
timeout = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, timeout=timeout)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,171 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audio post-processing wrapper around ffmpeg.
|
||||
|
||||
Subcommands:
|
||||
convert — WAV to OGG (libvorbis, quality 6)
|
||||
normalize — LUFS normalize to -16 LUFS (broadcast standard)
|
||||
trim — Remove leading/trailing silence
|
||||
pipeline — trim + normalize + convert (full post-processing chain)
|
||||
|
||||
All operations write to a new file (never overwrites input).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import shutil
|
||||
import json
|
||||
|
||||
|
||||
def check_ffmpeg():
|
||||
if not shutil.which("ffmpeg"):
|
||||
print(json.dumps({"ok": False, "error": "ffmpeg not found in PATH"}))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run_ffmpeg(args, description):
|
||||
"""Run ffmpeg, capture output, return success."""
|
||||
cmd = ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error"] + args
|
||||
print(f" {description}", file=sys.stderr)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"ffmpeg failed: {result.stderr.strip()}",
|
||||
"command": " ".join(cmd)
|
||||
}))
|
||||
sys.exit(1)
|
||||
return True
|
||||
|
||||
|
||||
def cmd_convert(args):
|
||||
"""Convert WAV to OGG (libvorbis)."""
|
||||
output = args.output or args.input.rsplit(".", 1)[0] + ".ogg"
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
f"converting {os.path.basename(args.input)} → {os.path.basename(output)}"
|
||||
)
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def cmd_normalize(args):
|
||||
"""LUFS normalize audio file."""
|
||||
output = args.output or _suffixed(args.input, "_norm")
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
output],
|
||||
f"normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_trim(args):
|
||||
"""Trim leading/trailing silence."""
|
||||
output = args.output or _suffixed(args.input, "_trimmed")
|
||||
af = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af, output],
|
||||
f"trimming silence (threshold: {args.threshold}dB)"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_pipeline(args):
|
||||
"""Full post-processing: trim → normalize → convert to OGG."""
|
||||
base = args.input.rsplit(".", 1)[0]
|
||||
trimmed = base + "_trimmed.wav"
|
||||
normalized = base + "_norm.wav"
|
||||
output = args.output or base + ".ogg"
|
||||
|
||||
# Step 1: trim
|
||||
af_trim = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af_trim, trimmed],
|
||||
"step 1/3: trimming silence"
|
||||
)
|
||||
|
||||
# Step 2: normalize
|
||||
run_ffmpeg(
|
||||
["-i", trimmed, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
normalized],
|
||||
f"step 2/3: normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
|
||||
# Step 3: convert
|
||||
run_ffmpeg(
|
||||
["-i", normalized, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
f"step 3/3: converting to OGG"
|
||||
)
|
||||
|
||||
# Clean up intermediates
|
||||
os.remove(trimmed)
|
||||
os.remove(normalized)
|
||||
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def _suffixed(path, suffix):
|
||||
base, ext = os.path.splitext(path)
|
||||
return base + suffix + ext
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Audio post-processing (ffmpeg wrapper)")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# convert
|
||||
p = sub.add_parser("convert", help="WAV → OGG")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output file (default: same name .ogg)")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.set_defaults(func=cmd_convert)
|
||||
|
||||
# normalize
|
||||
p = sub.add_parser("normalize", help="LUFS normalize")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.set_defaults(func=cmd_normalize)
|
||||
|
||||
# trim
|
||||
p = sub.add_parser("trim", help="Trim silence")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_trim)
|
||||
|
||||
# pipeline
|
||||
p = sub.add_parser("pipeline", help="Full post-processing: trim + normalize + convert")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output OGG file")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_pipeline)
|
||||
|
||||
args = parser.parse_args()
|
||||
check_ffmpeg()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
|
||||
"qdrant_url": "http://tower-of-joy:6333",
|
||||
"ollama_url": "http://tower-of-joy:11434",
|
||||
"stable_audio_url": "http://tower-of-joy:11500",
|
||||
"collection": "commonwealth",
|
||||
"embed_model": "nomic-embed-text",
|
||||
"embed_dimensions": 768
|
||||
|
||||
+3
-3
@@ -10,9 +10,9 @@ Cross-domain decisions live in one file with cross-reference notes in related fi
|
||||
|
||||
| File | Domain | Decisions |
|
||||
|------|--------|-----------|
|
||||
| [architecture.md](architecture.md) | Technical foundation | D-008, D-009, D-010, D-012, D-020, D-026, D-030, D-031, D-041, D-042, D-054, D-055, D-066 |
|
||||
| [perception.md](perception.md) | Player observation | D-011, D-015, D-016, D-017, D-018, D-019, D-033, D-035, D-043, D-044, D-045, D-046, D-047, D-048, D-049, D-052, D-056, D-057, D-058, D-059, D-060, D-061 |
|
||||
| [content.md](content.md) | NPC, dialogue, templates | D-023, D-024, D-025, D-028, D-029, D-032, D-034, D-035, D-036, D-037, D-050, D-062, D-063, D-064 |
|
||||
| [architecture.md](architecture.md) | Technical foundation | D-008, D-009, D-010, D-012, D-020, D-026, D-030, D-031, D-041, D-042, D-054, D-055, D-066, D-068, D-073 |
|
||||
| [perception.md](perception.md) | Player observation | D-011, D-015, D-016, D-017, D-018, D-019, D-033, D-035, D-043, D-044, D-045, D-046, D-047, D-048, D-049, D-052, D-056, D-057, D-058, D-059, D-060, D-061, D-067, D-069, D-070, D-071, D-072 |
|
||||
| [content.md](content.md) | NPC, dialogue, templates | D-023, D-024, D-025, D-028, D-029, D-032, D-034, D-035, D-036, D-037, D-050, D-062, D-063, D-064, D-074 |
|
||||
| [scope.md](scope.md) | Game concept, prototype | D-001, D-003, D-005, D-006, D-007, D-013, D-014, D-027, D-038, D-039, D-051, D-053, D-065 |
|
||||
| [process.md](process.md) | Team, workflow | D-004, D-021, D-022, D-040 |
|
||||
| [questions.md](questions.md) | Open questions | Q-001 through Q-026 |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Architecture Decisions
|
||||
|
||||
Technical foundation decisions that constrain implementation: engine, client-server, ECS, simulation, testability, performance budgets.
|
||||
Technical foundation decisions that constrain implementation: engine, client-server, ECS, simulation, testability, performance budgets, audio architecture.
|
||||
|
||||
---
|
||||
|
||||
@@ -178,6 +178,36 @@ Technical foundation decisions that constrain implementation: engine, client-ser
|
||||
- **Raised by:** Team Leader (Jeroen) — proposed retina scaling analogy and 2x2 geometry constraint. Tyre (feasibility: trivial, half-day integration). Gestalt (approved with 2x2 constraint resolving LOS readability concern). Ozzie (approved: solves sprite scale without uncanny mismatch).
|
||||
- **Dissent:** Gestalt initially objected to dual-scale (mental model mismatch for cover/LOS). Resolved by the 2x2 geometry minimum constraint — all cover maps 1:1 at visual scale.
|
||||
|
||||
### D-068: 5-bus audio architecture
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Audio uses a 5-bus architecture with player-facing volume sliders:
|
||||
1. **Music** — future/empty in v0.1. Reserved for diegetic Meridian music in social spaces.
|
||||
2. **Ambient** — station hum ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) asset 1) + zone overlays (assets 2-4). Continuous soundscape.
|
||||
3. **World SFX** — NPC footsteps, doors, environmental events. Diegetic world sounds not caused by player.
|
||||
4. **Player Actions** — player footsteps (assets 5-6), future: combat sounds, item interactions. Sounds player directly causes.
|
||||
5. **UI Sounds** — cursor hover, implant open, monologue chimes (assets 7-8), fog recognition. Interface feedback.
|
||||
- **Client implementation:**
|
||||
- AudioManager GDScript autoload singleton on client branch (lives with rendering, per [D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc)).
|
||||
- Audio assets committed to audio branch (content, not code).
|
||||
- Directory-scan registry pattern: AudioManager scans `res://audio/` on startup, maps filenames to AudioStream resources. No hardcoded asset list.
|
||||
- If directory empty or asset missing, all play methods no-op with visual fallback (per [D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) architecture).
|
||||
- 5 player-facing volume sliders, one per bus. Accessible via settings.
|
||||
- **Bus routing dropped:**
|
||||
- **Dialogue bus removed** — no voice acting in v0.1. NPC conversation murmur goes on World SFX (event-driven, per [D-072](perception.md#d-072-universal-event-driven-conversation-murmur)).
|
||||
- **Rationale:** 5 buses provide player control granularity (disable UI sounds, boost World SFX for eavesdropping, mute Ambient for focus) without over-segmentation. Directory-scan registry eliminates hardcoded asset paths — audio branch can add files without touching client code. No-op fallback means client works identically with or without audio.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), audio dip ([D-069](perception.md#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), client-server architecture ([D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc))
|
||||
- **Raised by:** Team Leader (channel split directive), Tyre (architecture), Inigo and Gestalt (bus refinement)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-073: Zone crossfade approach — hard boundary, soft audio transition
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Zone audio transitions use hard tile boundary triggers with 1.5-2s audio crossfade tweens. Server sends zone_id per tile in ObserverSnapshot (server-authoritative zone assignment). AudioManager receives zone changes and tweens between ambient layers. No blended overlap zones — the transition smoothness comes from audio fade duration, not spatial blending.
|
||||
- **Implementation:** AudioManager stub in Sprint 7 (5-bus setup, directory registry). Full zone crossfade implementation deferred to Sprint 8+.
|
||||
- **Rationale:** Hard boundaries with soft audio = predictable for simulation, pleasant for player. Avoids complex overlap zone geometry. Crossfade duration (1.5-2s) is long enough to feel smooth, short enough that walking back-and-forth across boundary doesn't create audio chaos.
|
||||
- **Cross-reference:** Audio architecture ([D-068](#d-068-5-bus-audio-architecture)), client-server ([D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc)), ambient assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open))
|
||||
- **Raised by:** Tyre
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*13 decisions. Last updated: 2026-02-14*
|
||||
*16 decisions. Last updated: 2026-02-16*
|
||||
|
||||
+18
-2
@@ -1,6 +1,6 @@
|
||||
# Content Decisions
|
||||
|
||||
How narrative, NPCs, and world content are created: content tiers, NPC generation, templates, dialogue, population.
|
||||
How narrative, NPCs, and world content are created: content tiers, NPC generation, templates, dialogue, population, audio aesthetic identity.
|
||||
|
||||
---
|
||||
|
||||
@@ -142,6 +142,22 @@ How narrative, NPCs, and world content are created: content tiers, NPC generatio
|
||||
- **Raised by:** Paula (three phases + KG recording), Stig (WASD mechanic + 300ms fade), Ozzie (consequences), Nigel (tolerance per seed)
|
||||
- **Dissent:** None.
|
||||
|
||||
### D-074: Audio aesthetic identity — insert-tech vs organic
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Audio design uses two sonic families with distinct aesthetic identities:
|
||||
- **Insert-tech sounds:** Synthetic, precise, clinical, no reverb. Monologue chimes ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) assets 7-8), UI feedback, future: lattice interface sounds, biometric scan confirmations. Represents neural lattice interface — artificial, computational, clinical.
|
||||
- **Organic sounds:** Warm, breathy, natural decay, environmental reverb. Footsteps, ambient layers, NPC movement, environmental events. Represents human cognition and physical reality — warm, imperfect, lived-in.
|
||||
- **Thematic mapping:** Insert-tech = neural lattice (computational precision, trustworthy but inhuman). Organic = human cognition and environment (warm, ambiguous, interpretive). Maps to [D-018](perception.md#d-018-three-range-sound-model) trust model: close/organic sounds = reliable, long-range/insert sounds = potentially compromised.
|
||||
- **Station palette — "functional warmth":**
|
||||
- Low-freq foundation (60-120Hz): station hum, span gate vibration, machinery drone.
|
||||
- Mid-freq texture (200-800Hz): footsteps, object handling, conversation murmur.
|
||||
- High-freq detail (2-8kHz, sparse): scanner pings, metal impacts, environmental detail.
|
||||
- No harsh frequencies, no industrial brutality. The station is SETTLED — lived-in, maintained, comfortable enough to be complacent.
|
||||
- **Environmental neutrality per [D-045](perception.md#d-045-art-direction--environmental-neutrality-strict-zero-shift):** Ambient audio does NOT shift with narrative state. Same station hum before and after conspiracy discovery. No music stingers for investigation progress. Emotional weight comes from monologue and player knowledge, not audio cues.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), sound model ([D-018](perception.md#d-018-three-range-sound-model)), environmental neutrality ([D-045](perception.md#d-045-art-direction--environmental-neutrality-strict-zero-shift)), setting ([D-036](#d-036-sova-transit-district--krenn-system-as-v01-setting))
|
||||
- **Raised by:** Inigo (insert-tech/organic split), Paula (cognitive architecture framing and trust model connection)
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*14 decisions. Last updated: 2026-02-13*
|
||||
*15 decisions. Last updated: 2026-02-16*
|
||||
|
||||
+65
-2
@@ -1,6 +1,6 @@
|
||||
# Perception Decisions
|
||||
|
||||
How the player observes and interacts with the world: camera, fog, line-of-sight, sound, monologue, perception modes.
|
||||
How the player observes and interacts with the world: camera, fog, line-of-sight, sound, monologue, perception modes, audio dip profiles, cognitive delays.
|
||||
|
||||
---
|
||||
|
||||
@@ -273,6 +273,69 @@ How the player observes and interacts with the world: camera, fog, line-of-sight
|
||||
- **Raised by:** Stig (UI spec + no portraits), Lead (20% height constraint + max-width directive)
|
||||
- **Dissent:** Stig initially proposed 25% height and 50% width centered. Lead constrained to 20% height and max-width.
|
||||
|
||||
### D-067: Recognition chime fires at onset of cognitive delay
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** The monologue recognition chime ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) assets 7-8) fires at the ONSET of the cognitive delay ([D-060](#d-060-cognitive-delay-for-fog-recognition)), not at its completion. Chime duration 300-400ms, overlapping with the start of the delay. The chime is "unresolved" — it opens a question, doesn't answer one. Full sequence when recognizing an entity in fog: (1) hear/sense something → (2) chime plays → (3) 0.6s cognitive delay begins (concurrent with chime tail) → (4) monologue text appears during delay ("Those footsteps... that's Kael's walk") → (5) blob transitions to [D-033](#d-033-entity-color--relationship-to-player) color + silhouette feature → (6) recognition complete.
|
||||
- **Rationale:** The chime marks the character's attention shifting, not the recognition completing. It creates an "unresolved" sensation — something is happening in the character's mind. Monologue text provides the resolution. This prevents the chime from feeling like a UI notification and instead makes it part of the character's cognitive process.
|
||||
- **Resolves:** Q-014
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), cognitive delay ([D-060](#d-060-cognitive-delay-for-fog-recognition)), fog recognition ([D-059](#d-059-fog--shader-based-five-layers-knowledge-graph-driven)), monologue ([D-016](#d-016-internal-monologue-as-core-perceptionatmosphere-system))
|
||||
- **Raised by:** Gestalt (proposal), confirmed by Team Leader (Jeroen)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-069: Audio dip profiles for dialogue and confrontation
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Three audio dip profiles modify bus volumes during player focus states, all relative to player slider settings (proportional, not absolute):
|
||||
- **Dialogue dip:** Ambient -6 to -8dB, World SFX 0dB, Player Actions 0dB, UI 0dB. 300ms ease-in, 500ms ease-out. Reduces environmental noise to foreground conversation without muting world events.
|
||||
- **Confrontation dip:** Ambient -10 to -12dB + low-pass filter (20kHz → 800Hz), World SFX -4 to -6dB (graduated — loud events like sprinting footsteps break through, quiet sneaking doesn't), Player Actions 0dB, UI 0dB. 500ms ease-in, 1000ms ease-out with filter sweep. Creates emotional/cognitive muffling per [D-070](#d-070-confrontation-as-cognitive-vulnerability).
|
||||
- **ListeningFocus boost:** World SFX +2 to +3dB when player is stationary for 30+ ticks (eavesdrop bonus, per [D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost)). Expresses heightened attention as audio gain.
|
||||
- **Key behaviors:**
|
||||
- All dB values are proportional to player slider setting, not absolute. If player sets World SFX to 50%, the -6dB dip applies to that 50% base.
|
||||
- Dips are interruptible — walk-away or stance change kills current tween, starts new tween to neutral.
|
||||
- Graduated World SFX dip during confrontation: loud, urgent events (sprinting, doors slamming, alarms) break through at reduced volume. Quiet, careful movement may not be noticed.
|
||||
- **Rationale:** Dips model finite attention. Dialogue suppresses ambient noise (focus on conversation). Confrontation suppresses both ambient and peripheral sounds (emotional tunnel vision per [D-070](#d-070-confrontation-as-cognitive-vulnerability)). ListeningFocus boost rewards deliberate eavesdropping. All effects proportional to user volume settings respects player accessibility choices.
|
||||
- **Cross-reference:** Audio architecture ([D-068](architecture.md#d-068-5-bus-audio-architecture)), confrontation vulnerability ([D-070](#d-070-confrontation-as-cognitive-vulnerability)), eavesdropping ([D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost)), audio aesthetic ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic))
|
||||
- **Raised by:** Inigo (initial spec), Gestalt (graduated approach + ListeningFocus boost), Tyre (implementation details), Ozzie (proportional dip to respect player settings)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-070: Confrontation as cognitive vulnerability
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Confrontation creates perceptual vulnerability through reduced ambient awareness. Design principle: "Emotional focus creates perceptual vulnerability." Player actions requiring cognitive focus (confrontation dialogue, future: deep terminal reading, complex insert queries) mechanically reduce ambient perception via audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)) and may suppress peripheral monologue triggers. This is NOT a punishment — it's a realistic consequence of finite attention. The player mitigates risk by choosing WHERE and WHEN to confront (safe location vs exposed corridor, early shift vs busy period).
|
||||
- **Mechanical expression:**
|
||||
- During confrontation: ambient sounds muffled, quiet NPC movement may go undetected (graduated World SFX dip means careful footsteps suppressed, sprinting footsteps still audible).
|
||||
- Post-confrontation: delayed monologue may fire for missed events ("Wait — did someone just pass by?").
|
||||
- World continues per [D-045](#d-045-art-direction--environmental-neutrality-strict-zero-shift) environmental indifference — no scripted ambushes, but NPCs on routine may coincidentally pass while player is focused.
|
||||
- Vulnerability is probabilistic and emergent, not scripted.
|
||||
- **Consistency principle:** Connects to [D-053](scope.md#d-053-movement-as-stance-toggle-system) sprint suppressing monologue (physical focus limits interpretation). Confrontation suppresses ambient perception (emotional focus limits peripheral awareness). High-focus activities suppress peripheral cognition — this is the shared rule.
|
||||
- **Must feel "felt, not computed" (Paula):** Muffling is psychological tunnel vision, not a debuff tooltip. No UI indicator. Player realizes in retrospect: "I was so focused on Sera I didn't hear Kael walk past."
|
||||
- **Rationale:** Makes WHERE and WHEN to confront meaningful decisions. Confronting in a private corner = safer. Confronting in a busy corridor during shift change = riskier. Player agency through spatial and temporal choice, not RNG.
|
||||
- **Cross-reference:** Audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), confrontation UI ([D-063](content.md#d-063-confrontation--same-box-different-weight)), environmental indifference ([D-045](#d-045-art-direction--environmental-neutrality-strict-zero-shift)), stance system ([D-053](scope.md#d-053-movement-as-stance-toggle-system)), monologue ([D-016](#d-016-internal-monologue-as-core-perceptionatmosphere-system))
|
||||
- **Raised by:** Gestalt (vulnerability principle), Ozzie (graduated dip approach), Paula ("felt not computed" framing)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-071: No ambient dip for eavesdropping — ListeningFocus boost
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Overheard conversation (eavesdropping) does NOT apply dialogue dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)). Cognitive state is opposite of direct conversation: extracting signal from noise requires MORE ambient awareness, not less. Instead, eavesdrop quality improves via ListeningFocus boost (+2 to +3dB World SFX when stationary 30+ ticks). Eavesdrop information quality is: distance-dependent, stance-modified ([D-053](scope.md#d-053-movement-as-stance-toggle-system) Careful stance grants "tell notice bonus"), ListeningFocus-boosted, and ambient-noise-penalized.
|
||||
- **Ambient zone affects eavesdrop difficulty:**
|
||||
- Bar (high ambient) = conversations hidden in murmur, requires proximity + ListeningFocus.
|
||||
- Workplace (moderate ambient) = conversations semi-conspicuous, distance-dependent.
|
||||
- Corridor (low ambient) = conversations conspicuous, easy to overhear from distance.
|
||||
- **Information confidence:** Overheard information enters knowledge graph at lower confidence (KnowsOf, not KnowsDetails per [D-041](architecture.md#d-041-knowledge-graph-data-model)) unless ListeningFocus + proximity allows high-quality capture.
|
||||
- **Rationale:** Direct conversation = character focuses, ambient dips. Overheard conversation = character strains to hear, ambient stays or boosts. Opposite cognitive states produce opposite audio profiles. Creates meaningful difference between talking TO someone vs listening IN on someone.
|
||||
- **Cross-reference:** Audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), stance system ([D-053](scope.md#d-053-movement-as-stance-toggle-system)), knowledge graph ([D-041](architecture.md#d-041-knowledge-graph-data-model)), zone audio ([D-072](#d-072-universal-event-driven-conversation-murmur))
|
||||
- **Raised by:** Gestalt (cognitive state framing), Paula (zone-conspicuousness model), unanimously endorsed
|
||||
- **Dissent:** None
|
||||
|
||||
### D-072: Universal event-driven conversation murmur
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** NPC-to-NPC conversations use a single universal murmur sound asset, not zone-specific variants. Zone ambient determines conspicuousness: bar ambient (high murmur density) hides conversations, corridor ambient (low background) makes conversations conspicuous. Maps to [D-047](#d-047-art-direction--two-tier-animation-system) two-tier behavior visibility: bar conversations are Tier 1 (clearly happening, content obscured), corridor conversations are Tier 2 (ambiguous — are they talking or just standing near each other?).
|
||||
- **Asset split:**
|
||||
- Bar ambient murmur: baked into `amb_bar_layer.ogg` (continuous background per [D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)).
|
||||
- NPC proximity murmur: separate event-driven asset, deferred to future sprint (not in Sprint 7 scope). When implemented, same asset plays everywhere; zone ambient determines audibility.
|
||||
- **Rationale:** One murmur asset + zone-dependent conspicuousness creates the signal/noise dynamic naturally. Bar = high noise floor, conversations blend in. Corridor = low noise floor, conversations stand out. Simpler content production, richer emergent behavior.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), two-tier animation ([D-047](#d-047-art-direction--two-tier-animation-system)), eavesdropping ([D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost))
|
||||
- **Raised by:** Paula (zone-conspicuousness model), Inigo (scoping to future sprint)
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*23 decisions. Last updated: 2026-02-14*
|
||||
*29 decisions. Last updated: 2026-02-16*
|
||||
|
||||
@@ -65,10 +65,9 @@ Tracked questions awaiting discussion or resolution.
|
||||
- **Source:** Content Gap Analysis Workshop (Mellanie R2)
|
||||
|
||||
### Q-014: Audio timing with monologue chime
|
||||
- **Status:** Open
|
||||
- **Question:** When does the monologue chime fire relative to text appearance? Before, simultaneous, or overlapping? Affects monologue line writing rhythm.
|
||||
- **Assigned to:** Gestalt, Ozzie
|
||||
- **Source:** Content Gap Analysis Workshop (Mellanie R2)
|
||||
- **Status:** Resolved → [D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay)
|
||||
- **Resolution:** Chime fires at ONSET of cognitive delay, not completion. 300-400ms duration, overlapping delay start. Chime is "unresolved" — opens a question, doesn't answer one. Sequence: hear/sense → chime plays → 0.6s delay begins → monologue text during delay → blob transitions to D-033 color → recognition complete.
|
||||
- **Date resolved:** 2026-02-16
|
||||
|
||||
### Q-015: Generation expansion for THE FRIEND content
|
||||
- **Status:** Open
|
||||
|
||||
+5
-2
@@ -114,8 +114,11 @@ What we're building: game concept, design pillars, prototype definition, map spe
|
||||
7. `sfx_monologue_chime.ogg` — soft crystalline tone, "neural lattice firing" feel (0.5-1.0s, monologue appearance)
|
||||
8. `sfx_monologue_chime_urgent.ogg` — sharper variant for contradiction/anomaly observations (0.5-1.0s)
|
||||
- **Architecture:** Event-driven with asset registry + visual fallback. Simulation emits typed sound events; client renders as audio (if asset exists) or visual indicator + monologue trigger (if not). Ambient loops managed separately from event-driven sounds. Monologue chime is a UI sound, not a simulation sound.
|
||||
- **Cross-reference:** Three-range sound model ([D-018](perception.md#d-018-three-range-sound-model)), sound event architecture (Gestalt R2 section 6)
|
||||
- **Raised by:** Ozzie (Round 1 minimum viable proposal, Round 2 full spec), project lead (confirmed, directives #3 and #9)
|
||||
- **Amendment (2026-02-16, Audio Pipeline Kickoff):**
|
||||
- **Hybrid audio generation approach:** Stable Audio Open (SAO) for sounds >200ms with organic character (footsteps, ambient layers). Manual synthesis for sounds <200ms with precise/digital character (UI clicks, scanner beeps). SAO ceiling ~47s; accept 45s loops with crossfade for ambient layers. The insert-tech/organic split ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic)) maps to synthesis/generation split.
|
||||
- **Sprint 7 scope expansion:** Ticket #440 expanded from 6 to 8 assets, adding monologue chimes (assets 7-8) as deliberate placeholders with Sprint 8 redo mandate. Chimes are critical for [D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay) cognitive delay feel but acknowledged as difficult to generate correctly — manual synthesis required for production quality.
|
||||
- **Cross-reference:** Three-range sound model ([D-018](perception.md#d-018-three-range-sound-model)), sound event architecture (Gestalt R2 section 6), audio aesthetic ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic)), recognition chime ([D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay))
|
||||
- **Raised by:** Ozzie (Round 1 minimum viable proposal, Round 2 full spec), project lead (confirmed, directives #3 and #9). Amendment raised by Inigo (hybrid approach), endorsed by Tyre.
|
||||
- **Dissent:** Mellanie and Araminta both proposed deferring audio; project lead overruled. Visual sound indicators remain complementary to audio (not replacement).
|
||||
|
||||
### D-039: v0.1 wow moment scope — all 6 moments
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# Asset Pipeline Index
|
||||
|
||||
Centralized tracking for all production assets across pipelines. Each pipeline has its own index, palette (style bible), and per-category asset tables.
|
||||
|
||||
## Pipelines
|
||||
|
||||
| Pipeline | Index | Palette | Status |
|
||||
|----------|-------|---------|--------|
|
||||
| [Audio](audio/README.md) | [audio/README.md](audio/README.md) | [audio/palette.md](audio/palette.md) | Active — Sprint 7 |
|
||||
| [Visual](visual/README.md) | [visual/README.md](visual/README.md) | [visual/palette.md](visual/palette.md) | Stub |
|
||||
| [Video](video/README.md) | [video/README.md](video/README.md) | [video/palette.md](video/palette.md) | Stub |
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
docs/assets/
|
||||
README.md # This file — master index
|
||||
_templates/
|
||||
audio.md # Row template + detailed entry template for audio
|
||||
sprite.md # Row template + detailed entry template for sprites
|
||||
video.md # Row template + detailed entry template for video
|
||||
audio/
|
||||
README.md # Audio pipeline index — categories, status summary
|
||||
palette.md # Sonic palette, aesthetic guide, generation approach
|
||||
ambient.md # All ambient loops (station, planet, biome, weather)
|
||||
sfx.md # All SFX (footsteps, doors, impacts, events)
|
||||
ui.md # All UI sounds (cursor, chimes, insert-tech)
|
||||
visual/
|
||||
README.md # Visual pipeline index
|
||||
palette.md # Art direction, "the angle", color relationships
|
||||
sprites.md # Entities (NPCs, player, creatures)
|
||||
furniture.md # Furniture items (hundreds, by location/type)
|
||||
tilesets.md # Floor/wall tiles by biome/planet
|
||||
icons.md # UI icons, status indicators
|
||||
effects.md # Fog shaders, particles, overlays
|
||||
video/
|
||||
README.md # Video pipeline index
|
||||
palette.md # Motion/editing style guide
|
||||
trailers.md # Marketing / Steam page trailers
|
||||
cutscenes.md # In-game narrative moments
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One .md per category** — asset table with one row per asset, detailed entries below for complex items
|
||||
- **palette.md** per pipeline — the style bible. Aesthetic baseline, theme, generation approach.
|
||||
- **Status values:** `planned` | `in-progress` | `placeholder` | `final`
|
||||
- **Templates:** `_templates/{type}.md` — row format + detailed entry format
|
||||
- **Asset files** live in `client/assets/{type}/` — these docs track design and production metadata
|
||||
- **Category splitting:** When a category file grows too long, split by domain (e.g., `furniture-station.md`, `furniture-colony.md`)
|
||||
|
||||
## Cross-References
|
||||
|
||||
- Decision files: `decisions/` (D-038 audio scope, D-033 entity colors, D-019 camera angle)
|
||||
- Art direction workshop: `docs/workshops/art-direction-mood-board/`
|
||||
- Sprint briefings: `docs/sprints/sprint-N/{team}.md`
|
||||
@@ -0,0 +1,33 @@
|
||||
# Audio Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.ogg` | {status} | {bus} | {method} | {duration} | {prompt/notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (AMB-001, SFX-001, UI-001)
|
||||
- **Filename:** Target filename in `client/assets/audio/`
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Bus:** Ambient / World SFX / Player Actions / UI Sounds / Music
|
||||
- **Method:** SAO (Stable Audio Open) / synth (manual synthesis) / hybrid
|
||||
- **Duration:** Target duration
|
||||
- **Prompt/Notes:** Generation prompt or design description (keep brief — expand in detailed entry if needed)
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
Use below the asset table for assets that need expanded documentation (complex generation, iteration history, special integration notes).
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Sonic family:** insert-tech / organic
|
||||
- **Frequency range:** {primary range}
|
||||
- **Full prompt:** `{complete generation prompt}`
|
||||
- **Post-processing:** {trim, normalize, EQ, crossfade, loop points}
|
||||
- **Integration:** {which system triggers this, bus routing, special behavior}
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,31 @@
|
||||
# Sprite Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.png` | {status} | {layer} | {method} | {directions} | {notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (SPR-001, FUR-001, TILE-001, ICO-001, FX-001)
|
||||
- **Filename:** Target filename in `client/assets/sprites/` (or tilesets/, etc.)
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Layer:** Entity / tile / effect / UI
|
||||
- **Method:** render pipeline / hand-drawn / AI-assisted
|
||||
- **Directions:** N/E/S/W (4-dir), S-only, or N/A
|
||||
- **Notes:** Source model, special render settings, design description
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Source model:** `{path to .glb/.blend}`
|
||||
- **Render settings:** Camera angle, lighting, outline width
|
||||
- **Color relationships:** D-033 mapping if entity
|
||||
- **Resolutions:** 1024px (working) / 256px (game) / 64px (minimap)
|
||||
- **Post-processing:** Outline pass, palette adjustment, special effects
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,30 @@
|
||||
# Video Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.{ext}` | {status} | {type} | {duration} | {notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (TRL-001, CUT-001)
|
||||
- **Filename:** Target filename
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Type:** trailer / cutscene / intro
|
||||
- **Duration:** Target duration
|
||||
- **Notes:** Purpose, audience, key scenes
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Resolution:** 1920x1080 / 3840x2160
|
||||
- **Frame rate:** 30 / 60 fps
|
||||
- **Audio track:** {link to audio asset or composition notes}
|
||||
- **Script/storyboard:** {link or inline}
|
||||
- **Production method:** In-engine capture / external editing / mixed
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,54 @@
|
||||
# Audio Asset Pipeline
|
||||
|
||||
Status: **Active** — Sprint 7 (first audio sprint)
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | File | Total | Planned | In-Progress | Placeholder | Final |
|
||||
|----------|------|-------|---------|-------------|-------------|-------|
|
||||
| Ambient | [ambient.md](ambient.md) | 4 | 4 | 0 | 0 | 0 |
|
||||
| SFX | [sfx.md](sfx.md) | 2 | 2 | 0 | 0 | 0 |
|
||||
| UI | [ui.md](ui.md) | 6 | 4 | 0 | 0 | 0 |
|
||||
| **Total** | | **12** | **10** | **0** | **0** | **0** |
|
||||
|
||||
Update counts when asset status changes.
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for the sonic identity: two sonic families (insert-tech / organic), station palette (functional warmth), generation approach, bus architecture.
|
||||
|
||||
## Generation Tools
|
||||
|
||||
- **Stable Audio Open:** Self-hosted, >200ms organic sounds. ~47s ceiling per generation.
|
||||
- **Manual synthesis:** <200ms insert-tech sounds. Audacity / Python oscillator.
|
||||
- **Post-processing:** LUFS normalization, EQ carving, crossfade loops (3-5s overlap).
|
||||
|
||||
## Bus Architecture
|
||||
|
||||
5 player-facing buses (see palette.md for full spec):
|
||||
|
||||
| Bus | Content |
|
||||
|-----|---------|
|
||||
| Music | Future — empty |
|
||||
| Ambient | Station hum, zone overlays |
|
||||
| World SFX | NPC footsteps, doors, conversation murmur |
|
||||
| Player Actions | Player footsteps, combat |
|
||||
| UI Sounds | Cursor, implant, chimes, recognition |
|
||||
|
||||
## Sprint Scope
|
||||
|
||||
### Sprint 7 (#440 — 8 assets)
|
||||
- 4 UI sounds: cursor hover, fog recognition, implant open, weapon aim (sketch)
|
||||
- 2 UI sounds: monologue chime normal + urgent (PLACEHOLDER — Sprint 8 redo)
|
||||
- 2 mix specs: dialogue dip, confrontation dip (documentation, not audio files)
|
||||
|
||||
### Sprint 8+ (D-038 — 6 remaining assets)
|
||||
- 4 ambient loops: station base, workplace layer, bar layer, corridor layer
|
||||
- 2 SFX: footstep metal walk, footstep metal run
|
||||
- Monologue chime redo (priority #1)
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-038: Audio in v0.1 scope
|
||||
- D-018: Three-range sound model
|
||||
- D-045: Environmental neutrality
|
||||
@@ -0,0 +1,43 @@
|
||||
# Ambient — Audio Assets
|
||||
|
||||
Zone-based ambient loops. Station base plays globally; overlays crossfade based on player zone (hard tile boundary, 1.5-2s audio tween).
|
||||
|
||||
## Generation Approach
|
||||
|
||||
Stable Audio Open primary. Prompt strategy: describe the SPACE, not the sound.
|
||||
- Target: 45s loops with 3-5s overlap crossfade (equal-power)
|
||||
- Validation: solo test → stack test → fatigue test (5+ minutes)
|
||||
- Post-processing: LUFS normalize, EQ carve per-layer to avoid masking, crossfade loop points
|
||||
- Environmental neutrality (D-045): warmth/character is baked into the asset, does not react to narrative state
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| AMB-001 | `amb_station_base.ogg` | planned | Ambient | SAO | 45-60s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. | S8+ |
|
||||
| AMB-002 | `amb_workplace_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. | S8+ |
|
||||
| AMB-003 | `amb_bar_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. | S8+ |
|
||||
| AMB-004 | `amb_corridor_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. | S8+ |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### AMB-001: amb_station_base
|
||||
- **Sonic family:** organic (environmental)
|
||||
- **Frequency range:** 60-120Hz foundation, 200-800Hz texture, 2-8kHz sparse detail
|
||||
- **Role:** Global ambient — always playing, never stops. The acoustic reality of Station Sova. If zone overlays fade, this is "silence" — should feel empty, not quiet.
|
||||
- **Layering:** Plays on AmbientBase sub-channel. Zone overlays play on AmbientLayer, additive.
|
||||
- **Integration:** Starts on game load, loops indefinitely. Volume on Ambient bus.
|
||||
- **Cross-reference:** D-038 (#1), D-039 wow moment #1 (Arrival — station hum is first thing player hears)
|
||||
|
||||
### AMB-003: amb_bar_layer
|
||||
- **Sonic family:** organic (social space)
|
||||
- **Murmur note:** Conversation murmur is baked into this asset, not triggered by NPC presence. The bar sounds like a crowd whether 2 or 15 NPCs are present. This is the acoustic character of the space. Separate NPC proximity murmur (event-driven, point-source) is a future asset.
|
||||
- **Integration:** Crossfades in when player enters bar zone, out when leaving. 1.5-2s tween.
|
||||
- **Cross-reference:** D-038 (#3), D-045 (warmth is indifference — bar sounds warm always)
|
||||
|
||||
## Future Expansion
|
||||
|
||||
As the game expands beyond Station Sova (multiple planets, biomes, weather), ambient categories will grow significantly. When this file exceeds ~50 entries, split by location:
|
||||
- `ambient-station.md` (Station Sova zones)
|
||||
- `ambient-planet-{name}.md` (planetary biomes)
|
||||
- `ambient-weather.md` (weather overlay loops)
|
||||
@@ -0,0 +1,93 @@
|
||||
# Audio Palette
|
||||
|
||||
The sonic identity of The Settled Reach. Living document — updated as the audio language evolves.
|
||||
|
||||
## Core Principle: Audio as Cognitive Architecture
|
||||
|
||||
Every sound maps to one of two cognitive sources:
|
||||
|
||||
| Source | Sound Type | What it means | Sonic character |
|
||||
|--------|-----------|---------------|-----------------|
|
||||
| Biological | Ambient layers, footsteps, fog recognition | "I am a body in a space, and my body knows things" | Organic, warm, natural decay |
|
||||
| Technological | Cursor hover, implant open, monologue chime | "I am augmented, and my lattice processes things my body can't" | Synthetic, precise, clinical |
|
||||
| Absence | Dialogue dip, confrontation dip | "I am choosing to focus, and the world continues without my attention" | Reduced ambient, muffled world |
|
||||
|
||||
This split maps to D-018's trust model: organic perception is slow but trustworthy, technological perception is fast but its data can be manipulated.
|
||||
|
||||
## Two Sonic Families
|
||||
|
||||
### Insert-Tech (Synthetic)
|
||||
- **Character:** Clean waveforms, no reverb (exists in your head, not in space). Digital. Precise. Mathematical.
|
||||
- **Frequency range:** 800Hz-4kHz primary. Sharp attack, controlled decay.
|
||||
- **Feel:** "A well-designed touchscreen" — tonal confirmation, not notification beeps.
|
||||
- **Used for:** Cursor hover, implant open, weapon aim, monologue chimes.
|
||||
- **Generation method:** Manual synthesis for <200ms sounds. SAO for >200ms.
|
||||
|
||||
### Organic (Biological)
|
||||
- **Character:** Warm, breathy, slightly resonant. Natural attack and decay. Reverberant (exists in the world).
|
||||
- **Frequency range:** 300Hz-1.5kHz primary. Soft attack, natural tail.
|
||||
- **Feel:** "The sound you 'hear' internally when something clicks into place."
|
||||
- **Used for:** Fog recognition, future character-cognition sounds.
|
||||
- **Generation method:** Stable Audio Open primary — organic texture is what generative audio does best.
|
||||
|
||||
## Station Sonic Palette: Functional Warmth
|
||||
|
||||
The Settled Reach is a working station. Not military, not luxury — **industrial infrastructure that people made livable.**
|
||||
|
||||
### Base Layer (always present)
|
||||
- **Low-frequency foundation (60-120Hz):** Generator hum, structural resonance. The span gate's mass holding the station together. Always present, barely noticed.
|
||||
- **Mid-frequency texture (200-800Hz):** Ventilation, distant pipe flow, air cycling systems. Life support you forget about until it stops.
|
||||
- **High-frequency detail (2-8kHz):** Occasional scanner pings, distant PA crackle, metal-on-metal contact. Sparse. These break through the baseline — something happened.
|
||||
|
||||
### Zone Overlays (additive to base)
|
||||
- **Bar (The Last Shift):** More mid-range (voice murmur 300Hz-3kHz), softer high end (glass filtered, not sharp), faint Meridian music barely melodic — someone left a radio on two rooms away. Social warmth.
|
||||
- **Corridor:** Emptier mid-range, more prominent low-end (gate hum louder — closer to structure), high-frequency echoes (reverb tail on ventilation whistle). Liminal emptiness.
|
||||
- **Workplace (Terminal):** Rhythmic mechanical elements (cargo cadence), cleaner scanner pings, distant procedural voices (cargo designations, not conversation). Institutional precision.
|
||||
|
||||
### What Silence Sounds Like
|
||||
Silence on a station isn't silence — it's the base hum with nothing on top. If a zone overlay fades out and you're left with just station base, that should feel **empty**. Not quiet. Empty. Like a room where people were just a moment ago.
|
||||
|
||||
## Environmental Neutrality (D-045)
|
||||
|
||||
The ambient palette **does not shift with narrative state**. The bar sounds warm whether THE FRIEND is honest or lying. The corridor hum doesn't become ominous when you discover the conspiracy. The world is indifferent to your investigation.
|
||||
|
||||
The warmth is the indifference. The bartender's radio doesn't know.
|
||||
|
||||
## Audio Bus Architecture
|
||||
|
||||
5 player-facing buses, each with independent volume slider:
|
||||
|
||||
| Bus | Content | Diegetic source |
|
||||
|-----|---------|-----------------|
|
||||
| Music | Future — empty for now | Non-diegetic |
|
||||
| Ambient | Station hum, zone overlays, global loops | Environment |
|
||||
| World SFX | NPC footsteps, doors, machinery, conversation murmur | Other entities |
|
||||
| Player Actions | Player footsteps, combat, weapon sounds | Player character |
|
||||
| UI Sounds | Cursor, implant, monologue chimes, fog recognition | Neural lattice / cognition |
|
||||
|
||||
## Generation Approach
|
||||
|
||||
### Stable Audio Open (>200ms, organic character)
|
||||
- Model: ~1.2B parameters, 44.1kHz stereo output, ~47s ceiling
|
||||
- Prompting: Describe the SPACE, not the sound. "Industrial space station interior, low frequency machinery hum" not "sci-fi ambient drone in D minor."
|
||||
- Iteration: Generate 4-6 candidates per asset. Test: solo → stack → fatigue (5+ minutes for loops).
|
||||
- Post-processing: Trim, normalize (LUFS), EQ to carve frequency space, crossfade loops (3-5s overlap, equal-power).
|
||||
|
||||
### Manual Synthesis (<200ms, insert-tech character)
|
||||
- Tools: Audacity / Python synthesis (sine tone + envelope + harmonic layering)
|
||||
- The precision IS the design — insert-tech sounds should feel mathematically exact.
|
||||
- A 50ms cursor tick is a few lines of oscillator + exponential decay.
|
||||
|
||||
### Quality Validation
|
||||
- **Solo test:** Does it sound right alone?
|
||||
- **Stack test:** Does it sit in the mix with other layers without masking?
|
||||
- **Fatigue test:** Can you listen for 5+ minutes without jarring repeat? (loops only)
|
||||
- **Close-your-eyes test (Ozzie):** Does the sound create a mental image / sensation?
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-018: Three-range sound model (close/medium/long)
|
||||
- D-038: Audio in v0.1 scope — 8 files via Stable Audio Open
|
||||
- D-045: Environmental neutrality
|
||||
- D-048: Neural insert overlay (insert-styled UI aesthetic)
|
||||
- D-053: Movement stances (acoustic footprint implications)
|
||||
@@ -0,0 +1,41 @@
|
||||
# SFX — Audio Assets
|
||||
|
||||
World sound effects: footsteps, environmental events, impacts. Event-driven, often positional (AudioStreamPlayer2D).
|
||||
|
||||
## Generation Approach
|
||||
|
||||
Mixed methods depending on duration and character:
|
||||
- Footsteps: SAO generation (>200ms) or hybrid (generate longer clip, extract best transient)
|
||||
- Future environmental SFX (doors, machinery events): SAO or foley recording
|
||||
- Post-processing: normalize, trim, ensure clean attack/release
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| SFX-001 | `sfx_footstep_metal.ogg` | planned | Player Actions | hybrid | 0.3-0.5s | Single footstep on metal grating. Walk pace. Player character proprioceptive feedback. | S8+ |
|
||||
| SFX-002 | `sfx_footstep_metal_run.ogg` | planned | Player Actions | hybrid | 0.2-0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). | S8+ |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### SFX-001: sfx_footstep_metal
|
||||
- **Sonic family:** organic (physical world)
|
||||
- **Bus rationale:** Player Actions — this is feedback on YOUR movement, not world information. NPC footsteps (future) go on World SFX.
|
||||
- **Stance implications (D-053):** Walk = normal volume. Sprint variant (SFX-002) = louder. Careful = quieter variant (future asset). Crouch = near-silent variant (future asset).
|
||||
- **Integration:** Triggered by player movement tick. Positional (AudioStreamPlayer2D at player position) — maps to camera-relative stereo per D-018.
|
||||
- **Cross-reference:** D-038 (#5), D-053 (movement stances as acoustic decisions)
|
||||
|
||||
## Future Expansion
|
||||
|
||||
SFX will grow significantly with:
|
||||
- NPC footstep variants (by surface type, stance) → World SFX bus
|
||||
- Door open/close, machinery activation → World SFX bus
|
||||
- Combat impacts, weapon sounds → Player Actions bus
|
||||
- Environmental events per planet/biome
|
||||
- NPC conversation murmur (event-driven, point-source) → World SFX bus
|
||||
|
||||
When this file exceeds ~50 entries, split by domain:
|
||||
- `sfx-movement.md` (footsteps, stance variants)
|
||||
- `sfx-environment.md` (doors, machinery, weather)
|
||||
- `sfx-combat.md` (weapons, impacts)
|
||||
- `sfx-conversation.md` (murmur, vocal events)
|
||||
@@ -0,0 +1,73 @@
|
||||
# UI — Audio Assets
|
||||
|
||||
Interface sounds triggered by player interaction, insert systems, and cognitive events. Non-positional (AudioStreamPlayer). Routes to UI Sounds bus.
|
||||
|
||||
## Generation Approach
|
||||
|
||||
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`.
|
||||
- **Monologue chimes:** Currently PLACEHOLDER (Sprint 7). Redo scheduled next audio sprint (#453). Design brief below captures target register.
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| UI-001 | `cursor_hover.ogg` | draft | UI Sounds | SAO | 50-100ms | Subtle tick/ping on entity hover (D-056). Insert-tech: clean, precise, synthetic. Almost subliminal — notice if missing, not if present. Needs trim to target duration (#453). | S7 #440 |
|
||||
| UI-002 | `implant_open.ogg` | draft | UI Sounds | SAO | 150-250ms | Rising tone when insert UI opens (radial menu, inventory, stance). Neural lattice powering up. Frequency sweep, harmonic shimmer. Needs trim (#453). | S7 #440 |
|
||||
| UI-003 | `fog_recognition.ogg` | draft | UI Sounds | SAO | 300-400ms | Warm organic chime at onset of cognitive delay (D-060). Character recognizing a person. Unresolved tone. Needs trim (#453). | S7 #440 |
|
||||
| UI-004 | `weapon_aim.ogg` | draft | UI Sounds | SAO | 100-150ms | Harder click/lock for weapon aim state. Mechanical, deliberate. SKETCH ONLY — no weapons in Sprint 7. Needs trim (#453). | S7 #440 |
|
||||
| UI-005 | `sfx_monologue_chime.ogg` | draft | UI Sounds | SAO | 500-1000ms | PLACEHOLDER. Soft crystalline tone, neural lattice surfacing a thought. Quiet, non-intrusive. Redo in next audio sprint (#453). See design brief below. | S7 #440 |
|
||||
| UI-006 | `sfx_monologue_chime_urgent.ogg` | draft | UI Sounds | SAO | 500-1000ms | PLACEHOLDER. Sharper variant for contradiction/anomaly. Brighter, more insistent. Redo in next audio sprint (#453). See design brief below. | S7 #440 |
|
||||
|
||||
## Mix Specs (not audio files)
|
||||
|
||||
These are AudioBus parameter changes, documented in `docs/audio/dialogue-ambient-dip.md`:
|
||||
|
||||
| Spec | Trigger | Effect | Sprint |
|
||||
|------|---------|--------|--------|
|
||||
| Dialogue dip | Dialogue box opens | Ambient -6 to -8dB, 300ms ease-in, 500ms ease-out | S7 #440 |
|
||||
| Confrontation dip | Confrontation dialogue | Ambient -10 to -12dB + LP (800Hz), World SFX -4 to -6dB, 500ms ease-in, 1000ms ease-out | S7 #440 |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### UI-001: cursor_hover
|
||||
- **Sonic family:** insert-tech
|
||||
- **Frequency range:** 3-4kHz primary
|
||||
- **Design principle:** "Like breathing — present, functional, invisible." Player shouldn't describe it if asked, but would notice absence. Fatigue test critical — plays 300+ times per session.
|
||||
- **Generation:** Pure synthesis. Sine tone at 3.2kHz, exponential decay over 40-50ms. No SAO — precision and purity are the point.
|
||||
- **Debouncing:** AudioManager enforces 100ms cooldown between plays (rapid mouse movement).
|
||||
- **Integration:** CursorRenderer.cursor_state_changed signal → AudioManager.play_ui("cursor_hover")
|
||||
- **Cross-reference:** D-056 (cursor states)
|
||||
|
||||
### UI-003: fog_recognition
|
||||
- **Sonic family:** organic
|
||||
- **Frequency range:** 300Hz-1.5kHz
|
||||
- **Design principle:** "The sound you hear internally when something clicks into place." Not a chime — more like a soft exhalation of tone. Sits BETWEEN insert sounds and ambient — not UI, not world, cognition.
|
||||
- **Timing (Q-014 resolved):** Fires at ONSET of cognitive delay. Sequence: hear something in fog → chime plays → 0.6s delay → monologue during delay → blob transitions to D-033 color. Chime is "your character is processing," color transition is "recognition complete."
|
||||
- **Integration:** Cognitive delay onset event → AudioManager.play_ui("fog_recognition")
|
||||
- **Cross-reference:** D-060 (cognitive delay), D-018 (three-range sound, medium range)
|
||||
|
||||
### UI-005 / UI-006: Monologue Chime Design Brief (PLACEHOLDER)
|
||||
|
||||
**Target emotional register for Sprint 8 redo:**
|
||||
|
||||
The monologue chimes are the two poles of augmented cognition made audible:
|
||||
|
||||
| Variant | Meaning | Feel | Target |
|
||||
|---------|---------|------|--------|
|
||||
| Normal | "My lattice surfaced this" | A thought arriving — not a notification. Like your own attention shifting. | Player never consciously notices it. Pavlovian: text appeared, brain registered, moved on. |
|
||||
| Urgent | "My lattice flagged a contradiction" | Productive discomfort. "Wait, what did I miss?" | Micro-spike of alertness. Not alarm. The delta from normal must be SMALL but unmistakable. |
|
||||
|
||||
**Relationship to fog recognition chime:**
|
||||
- Fog recognition = organic (biological cognition, warm, 300-800Hz)
|
||||
- Monologue normal = insert-tech (crystalline, clean, higher register, ~1.2kHz)
|
||||
- Monologue urgent = insert-tech (sharper, brighter, adds second harmonic ~2.4kHz)
|
||||
- Same cognitive family, different branches. Cousins, not twins.
|
||||
|
||||
**Quality bar:** If the urgent chime feels like a quest marker ping, it has failed. If it feels like Ubisoft, it has failed. The delta between normal and urgent = "someone thinking quietly" vs "someone's eyes going slightly wide."
|
||||
|
||||
**Placeholder spec (Sprint 7):**
|
||||
- Normal: pure sine 1.2kHz, 600ms, gentle attack, natural decay. Conservative volume.
|
||||
- Urgent: sine 1.2kHz + harmonic 2.4kHz, 400ms, sharper attack, 15% louder.
|
||||
- Deliberately undersized — obviously temporary. Ugly placeholders get replaced.
|
||||
|
||||
**Evaluation gate:** First internal playtest after Sprint 7: "Does the normal chime feel like a thought arriving? Does the urgent chime create a physiological response?" If no → trigger chime redesign.
|
||||
@@ -0,0 +1,18 @@
|
||||
# Video Asset Pipeline
|
||||
|
||||
Status: **Stub** — no video assets planned for v0.1.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | File | Count | Description |
|
||||
|----------|------|-------|-------------|
|
||||
| Trailers | [trailers.md](trailers.md) | 0 | Marketing / Steam page trailers |
|
||||
| Cutscenes | [cutscenes.md](cutscenes.md) | 0 | In-game narrative moments (D-019: deferred to milestones) |
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for motion/editing style guide (to be established).
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-019: 3D cutscenes for key moments — deferred to milestones, not v0.1
|
||||
@@ -0,0 +1,5 @@
|
||||
# Video Palette
|
||||
|
||||
Status: **Stub** — no video production planned for v0.1.
|
||||
|
||||
To be established when cutscene or trailer work begins.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Visual Asset Pipeline
|
||||
|
||||
Status: **Stub** — awaiting first visual sprint.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | File | Count | Description |
|
||||
|----------|------|-------|-------------|
|
||||
| Sprites | [sprites.md](sprites.md) | 0 | Entity sprites via 3D render pipeline |
|
||||
| Furniture | [furniture.md](furniture.md) | 0 | Furniture items by location/type |
|
||||
| Tilesets | [tilesets.md](tilesets.md) | 0 | Floor/wall tiles by biome/planet |
|
||||
| Icons | [icons.md](icons.md) | 0 | UI icons, status indicators |
|
||||
| Effects | [effects.md](effects.md) | 0 | Fog shaders, particles, overlays |
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for art direction, "the angle", color relationships, and render pipeline specs.
|
||||
|
||||
## Pipeline
|
||||
|
||||
- **3D render pipeline:** Blender model → Godot Camera3D at -72.5deg → orthographic render → outline pass → resolution downscale
|
||||
- **Render skill:** `/render-sprite` produces 12 PNGs (4 directions x 3 resolutions)
|
||||
- **Art direction workshop:** `docs/workshops/art-direction-mood-board/`
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-019: Top-down camera, 15-20deg from vertical ("the angle")
|
||||
- D-033: Entity color = relationship to player
|
||||
- D-045: Environmental neutrality
|
||||
@@ -0,0 +1,16 @@
|
||||
# Visual Palette
|
||||
|
||||
Status: **Stub** — reference art direction workshop output for current guidance.
|
||||
|
||||
See `docs/workshops/art-direction-mood-board/` for the established visual identity.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Camera angle:** 15-20deg from vertical ("the angle"), rendered via orthographic Camera3D at -72.5deg from horizontal (D-019)
|
||||
- **Entity colors:** Relationship-based per D-033 (green = known/friendly, amber = neutral, red = hostile, etc.)
|
||||
- **Environmental neutrality:** Spaces don't visually shift with narrative state (D-045)
|
||||
- **Functional warmth:** Industrial infrastructure that people made livable — not military, not luxury
|
||||
|
||||
## Style Guide
|
||||
|
||||
To be populated from art direction workshop synthesis and first visual sprint.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Dialogue & Confrontation Ambient Dip — Implementation Spec
|
||||
|
||||
Audio bus volume/filter changes for dialogue and confrontation states. These are NOT audio files — they're mix parameter changes applied to AudioBus volumes and effects in Godot's AudioServer.
|
||||
|
||||
**Ticket:** #440
|
||||
**Decisions:** D-068 (5-bus architecture), D-069 (dip profiles), D-070 (confrontation as cognitive vulnerability)
|
||||
**Branch:** client (AudioManager implementation), audio (this spec)
|
||||
|
||||
## Bus Architecture Reference
|
||||
|
||||
| Bus Index | Name | Purpose |
|
||||
|-----------|------|---------|
|
||||
| 0 | Master | Final mix output |
|
||||
| 1 | Music | Score (empty for now) |
|
||||
| 2 | Ambient | amb_* loops, environmental background |
|
||||
| 3 | World SFX | Positional sounds in physical space |
|
||||
| 4 | Player Actions | Combat, footsteps, interaction SFX |
|
||||
| 5 | UI Sounds | Non-positional interface feedback |
|
||||
|
||||
## Dialogue Dip
|
||||
|
||||
**Trigger:** Dialogue box opens (client #434)
|
||||
**Release:** Dialogue box closes
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Ambient bus volume | -6 to -8 dB (from current) |
|
||||
| Ease-in duration | 300ms |
|
||||
| Ease-out duration | 500ms |
|
||||
| Easing curve | Cubic ease-in-out |
|
||||
| Affected buses | Ambient only |
|
||||
|
||||
### Godot Implementation
|
||||
|
||||
```gdscript
|
||||
# In AudioManager (autoload singleton)
|
||||
|
||||
const DIALOGUE_DIP_DB := -7.0 # midpoint of -6 to -8 range
|
||||
const DIALOGUE_DIP_IN_MS := 300.0
|
||||
const DIALOGUE_DIP_OUT_MS := 500.0
|
||||
|
||||
var _ambient_bus_idx: int
|
||||
var _ambient_base_volume_db: float
|
||||
var _dip_tween: Tween
|
||||
|
||||
func _ready() -> void:
|
||||
_ambient_bus_idx = AudioServer.get_bus_index("Ambient")
|
||||
_ambient_base_volume_db = AudioServer.get_bus_volume_db(_ambient_bus_idx)
|
||||
|
||||
func dialogue_dip_start() -> void:
|
||||
_cancel_dip_tween()
|
||||
var target := _ambient_base_volume_db + DIALOGUE_DIP_DB
|
||||
_dip_tween = create_tween()
|
||||
_dip_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
target,
|
||||
DIALOGUE_DIP_IN_MS / 1000.0
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func dialogue_dip_end() -> void:
|
||||
_cancel_dip_tween()
|
||||
_dip_tween = create_tween()
|
||||
_dip_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
_ambient_base_volume_db,
|
||||
DIALOGUE_DIP_OUT_MS / 1000.0
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func _set_ambient_volume(db: float) -> void:
|
||||
AudioServer.set_bus_volume_db(_ambient_bus_idx, db)
|
||||
|
||||
func _cancel_dip_tween() -> void:
|
||||
if _dip_tween and _dip_tween.is_valid():
|
||||
_dip_tween.kill()
|
||||
```
|
||||
|
||||
### Signal Wiring
|
||||
|
||||
```gdscript
|
||||
# In DialogueBox or wherever dialogue state is managed:
|
||||
func _open_dialogue() -> void:
|
||||
# ... show dialogue UI ...
|
||||
AudioManager.dialogue_dip_start()
|
||||
|
||||
func _close_dialogue() -> void:
|
||||
# ... hide dialogue UI ...
|
||||
AudioManager.dialogue_dip_end()
|
||||
```
|
||||
|
||||
## Confrontation Dip
|
||||
|
||||
**Trigger:** Confrontation dialogue begins (client #434, confrontation variant)
|
||||
**Release:** Confrontation dialogue ends
|
||||
**Design intent (D-070):** "Felt, not computed." The player's focus narrows — the world acoustically recedes. This is cognitive vulnerability made audible.
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Ambient bus volume | -11 dB (midpoint of -10 to -12) |
|
||||
| Ambient bus low-pass filter | 800 Hz cutoff, 6 dB resonance |
|
||||
| World SFX bus volume | -5 dB (midpoint of -4 to -6) |
|
||||
| Ease-in duration | 500ms |
|
||||
| Ease-out duration | 1000ms |
|
||||
| Easing curve | Cubic ease-in-out |
|
||||
| Affected buses | Ambient, World SFX |
|
||||
|
||||
### Godot Implementation
|
||||
|
||||
The confrontation dip adds a low-pass filter effect to the Ambient bus. This must be set up in the Godot AudioBus layout (Project → Audio Bus Layout):
|
||||
|
||||
**Bus setup (audio bus layout .tres):**
|
||||
1. Add `AudioEffectLowPassFilter` to the Ambient bus
|
||||
2. Set it to **bypassed by default** (effect is inactive until confrontation)
|
||||
3. Default cutoff: 20500 Hz (fully open)
|
||||
|
||||
```gdscript
|
||||
const CONFRONTATION_AMBIENT_DIP_DB := -11.0
|
||||
const CONFRONTATION_SFX_DIP_DB := -5.0
|
||||
const CONFRONTATION_LP_CUTOFF_HZ := 800.0
|
||||
const CONFRONTATION_LP_OPEN_HZ := 20500.0
|
||||
const CONFRONTATION_DIP_IN_MS := 500.0
|
||||
const CONFRONTATION_DIP_OUT_MS := 1000.0
|
||||
|
||||
var _world_sfx_bus_idx: int
|
||||
var _world_sfx_base_volume_db: float
|
||||
var _ambient_lp_effect_idx: int # index of the LP filter on Ambient bus
|
||||
var _confrontation_tween: Tween
|
||||
|
||||
func _ready() -> void:
|
||||
# ... (ambient bus setup from dialogue dip above) ...
|
||||
_world_sfx_bus_idx = AudioServer.get_bus_index("World SFX")
|
||||
_world_sfx_base_volume_db = AudioServer.get_bus_volume_db(_world_sfx_bus_idx)
|
||||
# Find the LP filter effect index on the Ambient bus
|
||||
for i in range(AudioServer.get_bus_effect_count(_ambient_bus_idx)):
|
||||
if AudioServer.get_bus_effect(_ambient_bus_idx, i) is AudioEffectLowPassFilter:
|
||||
_ambient_lp_effect_idx = i
|
||||
break
|
||||
|
||||
func confrontation_dip_start() -> void:
|
||||
_cancel_confrontation_tween()
|
||||
|
||||
# Enable the LP filter
|
||||
AudioServer.set_bus_effect_enabled(_ambient_bus_idx, _ambient_lp_effect_idx, true)
|
||||
|
||||
var amb_target := _ambient_base_volume_db + CONFRONTATION_AMBIENT_DIP_DB
|
||||
var sfx_target := _world_sfx_base_volume_db + CONFRONTATION_SFX_DIP_DB
|
||||
var dur := CONFRONTATION_DIP_IN_MS / 1000.0
|
||||
|
||||
_confrontation_tween = create_tween()
|
||||
_confrontation_tween.set_parallel(true)
|
||||
|
||||
# Ambient volume dip
|
||||
_confrontation_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
amb_target, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# World SFX volume dip
|
||||
_confrontation_tween.tween_method(
|
||||
_set_world_sfx_volume,
|
||||
AudioServer.get_bus_volume_db(_world_sfx_bus_idx),
|
||||
sfx_target, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Low-pass filter sweep
|
||||
var lp_effect: AudioEffectLowPassFilter = AudioServer.get_bus_effect(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx
|
||||
)
|
||||
_confrontation_tween.tween_property(
|
||||
lp_effect, "cutoff_hz",
|
||||
CONFRONTATION_LP_CUTOFF_HZ, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func confrontation_dip_end() -> void:
|
||||
_cancel_confrontation_tween()
|
||||
var dur := CONFRONTATION_DIP_OUT_MS / 1000.0
|
||||
|
||||
_confrontation_tween = create_tween()
|
||||
_confrontation_tween.set_parallel(true)
|
||||
|
||||
# Restore ambient volume
|
||||
_confrontation_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
_ambient_base_volume_db, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Restore world SFX volume
|
||||
_confrontation_tween.tween_method(
|
||||
_set_world_sfx_volume,
|
||||
AudioServer.get_bus_volume_db(_world_sfx_bus_idx),
|
||||
_world_sfx_base_volume_db, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Open LP filter back up
|
||||
var lp_effect: AudioEffectLowPassFilter = AudioServer.get_bus_effect(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx
|
||||
)
|
||||
_confrontation_tween.tween_property(
|
||||
lp_effect, "cutoff_hz",
|
||||
CONFRONTATION_LP_OPEN_HZ, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Disable LP filter after tween completes
|
||||
_confrontation_tween.chain().tween_callback(func():
|
||||
AudioServer.set_bus_effect_enabled(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx, false
|
||||
)
|
||||
)
|
||||
|
||||
func _set_world_sfx_volume(db: float) -> void:
|
||||
AudioServer.set_bus_volume_db(_world_sfx_bus_idx, db)
|
||||
|
||||
func _cancel_confrontation_tween() -> void:
|
||||
if _confrontation_tween and _confrontation_tween.is_valid():
|
||||
_confrontation_tween.kill()
|
||||
```
|
||||
|
||||
## Edge Cases
|
||||
|
||||
### Confrontation during dialogue
|
||||
Confrontation dip supersedes dialogue dip (it's deeper). If dialogue is active when confrontation starts, skip directly to confrontation levels. When confrontation ends, restore to dialogue dip levels (not base), then to base when dialogue ends.
|
||||
|
||||
### Rapid open/close
|
||||
The tween-kill-and-restart pattern handles this — a new dip start/end always kills the current tween and starts from the current actual volume, preventing jarring jumps.
|
||||
|
||||
### Player volume slider interaction
|
||||
Dips are RELATIVE to `_ambient_base_volume_db`. If the player adjusts their Ambient slider mid-dip, update `_ambient_base_volume_db` and recalculate the target. The `AudioManager` settings save/load system should call a method to refresh base volumes.
|
||||
|
||||
## ListeningFocus Boost (D-069)
|
||||
|
||||
When the player is in active listening mode (future sprint), World SFX gets a +2-3 dB boost instead of a dip. This is the inverse of confrontation — the character is paying MORE attention to the environment.
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| World SFX bus volume | +2.5 dB (midpoint of +2 to +3) |
|
||||
| Ease-in | 200ms |
|
||||
| Ease-out | 300ms |
|
||||
|
||||
Implementation follows the same tween pattern. Mutually exclusive with confrontation dip.
|
||||
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Synthesize insert-tech UI sounds for Sprint 7 #440.
|
||||
|
||||
Each sound uses a DIFFERENT synthesis technique to ensure distinct character:
|
||||
- cursor_hover: impulse → resonant bandpass (digital click)
|
||||
- weapon_aim: filtered noise + sub thump (mechanical)
|
||||
- monologue_chime: FM synthesis (crystalline bell)
|
||||
- monologue_chime_urgent: FM synthesis + beating/dissonance (tense bell)
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import wave
|
||||
import os
|
||||
|
||||
SR = 44100
|
||||
OUTPUT_DIR = os.path.join(os.path.dirname(__file__), "..", "client", "assets", "audio")
|
||||
|
||||
|
||||
def write_wav(filename, samples, channels=1):
|
||||
"""Write float samples [-1, 1] to 16-bit WAV."""
|
||||
path = os.path.join(OUTPUT_DIR, filename)
|
||||
# Normalize to peak if it exceeds 1.0
|
||||
peak = np.max(np.abs(samples))
|
||||
if peak > 1.0:
|
||||
samples = samples / peak
|
||||
int_samples = np.clip(samples * 32767, -32767, 32767).astype(np.int16)
|
||||
with wave.open(path, "w") as f:
|
||||
f.setnchannels(channels)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(SR)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
dur_ms = len(int_samples) / SR * 1000
|
||||
print(f" wrote {path} ({dur_ms:.0f}ms, {channels}ch)")
|
||||
return path
|
||||
|
||||
|
||||
def simple_lowpass(signal, cutoff_hz, sr=SR):
|
||||
"""Single-pole IIR low-pass filter."""
|
||||
rc = 1.0 / (2 * np.pi * cutoff_hz)
|
||||
dt = 1.0 / sr
|
||||
alpha = dt / (rc + dt)
|
||||
out = np.zeros_like(signal)
|
||||
out[0] = alpha * signal[0]
|
||||
for i in range(1, len(signal)):
|
||||
out[i] = out[i - 1] + alpha * (signal[i] - out[i - 1])
|
||||
return out
|
||||
|
||||
|
||||
def simple_highpass(signal, cutoff_hz, sr=SR):
|
||||
"""Single-pole IIR high-pass filter."""
|
||||
rc = 1.0 / (2 * np.pi * cutoff_hz)
|
||||
dt = 1.0 / sr
|
||||
alpha = rc / (rc + dt)
|
||||
out = np.zeros_like(signal)
|
||||
out[0] = signal[0]
|
||||
for i in range(1, len(signal)):
|
||||
out[i] = alpha * (out[i - 1] + signal[i] - signal[i - 1])
|
||||
return out
|
||||
|
||||
|
||||
def cursor_hover():
|
||||
"""UI-001: Digital click/tick on entity hover.
|
||||
|
||||
Technique: Short noise impulse bandpass-filtered to ~4kHz.
|
||||
Sounds like a tiny digital snap — no sustained tone at all.
|
||||
The "click" of a selection appearing on a HUD.
|
||||
"""
|
||||
duration = 0.035 # 35ms — shorter than before
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
rng = np.random.default_rng(77)
|
||||
|
||||
# White noise impulse
|
||||
impulse = rng.uniform(-1, 1, n)
|
||||
|
||||
# Bandpass around 4kHz: high-pass at 3kHz, low-pass at 5kHz
|
||||
filtered = simple_highpass(impulse, 3000)
|
||||
filtered = simple_lowpass(filtered, 5500)
|
||||
|
||||
# Very fast decay — done in 25ms
|
||||
envelope = np.exp(-t * 140)
|
||||
|
||||
return write_wav("cursor_hover.wav", filtered * envelope * 0.3)
|
||||
|
||||
|
||||
def weapon_aim():
|
||||
"""UI-004: Mechanical latch for weapon aim.
|
||||
|
||||
Technique: Low-pass filtered noise burst (the clack) + sub-bass
|
||||
thump (the weight). No musical pitch — this is a MECHANICAL sound.
|
||||
Think: safety clicking off, bolt sliding home.
|
||||
"""
|
||||
duration = 0.15 # 150ms
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
rng = np.random.default_rng(42)
|
||||
|
||||
# Component 1: Low-pass noise burst — the metallic clack
|
||||
noise = rng.uniform(-1, 1, n)
|
||||
# Low-pass at 1.5kHz — dull, heavy impact, not bright
|
||||
clack = simple_lowpass(noise, 1500)
|
||||
clack_env = np.exp(-t * 60) # fast decay
|
||||
clack = clack * clack_env
|
||||
|
||||
# Component 2: Sub-bass thump — weight of the mechanism
|
||||
# 80Hz sine, very fast decay
|
||||
thump = np.sin(2 * np.pi * 80 * t)
|
||||
thump_env = np.exp(-t * 40)
|
||||
thump = thump * thump_env
|
||||
|
||||
# Component 3: High metallic click at the very start (2ms)
|
||||
click = rng.uniform(-1, 1, n)
|
||||
click = simple_highpass(click, 4000)
|
||||
click_env = np.zeros(n)
|
||||
click_mask = t < 0.003
|
||||
click_env[click_mask] = np.exp(-t[click_mask] * 800)
|
||||
click = click * click_env
|
||||
|
||||
signal = clack * 0.4 + thump * 0.35 + click * 0.25
|
||||
return write_wav("weapon_aim.wav", signal * 0.45)
|
||||
|
||||
|
||||
def monologue_chime():
|
||||
"""UI-005: Crystalline thought-chime. PLACEHOLDER.
|
||||
|
||||
Technique: FM synthesis — carrier modulated by a lower frequency
|
||||
creates rich, evolving harmonics that sound like struck glass or
|
||||
crystal. Fundamentally different timbre from additive sine waves.
|
||||
"""
|
||||
duration = 0.75 # 750ms
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
# FM synthesis: carrier at 1200Hz, modulator at 420Hz (ratio ~2.86:1)
|
||||
# Inharmonic ratio = bell-like quality
|
||||
f_carrier = 1200
|
||||
f_mod = 420
|
||||
# Mod index decays over time — bright attack, mellow sustain
|
||||
mod_index = 3.0 * np.exp(-t * 6)
|
||||
# Modulator signal
|
||||
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
|
||||
# Carrier with FM
|
||||
signal = np.sin(2 * np.pi * f_carrier * t + modulator)
|
||||
|
||||
# Gentle attack (20ms), slow decay
|
||||
attack = np.minimum(t / 0.02, 1.0)
|
||||
decay = np.exp(-t * 3.0)
|
||||
envelope = attack * decay
|
||||
|
||||
return write_wav("sfx_monologue_chime.wav", signal * envelope * 0.22)
|
||||
|
||||
|
||||
def monologue_chime_urgent():
|
||||
"""UI-006: Urgent thought-chime. PLACEHOLDER.
|
||||
|
||||
Technique: FM synthesis with higher mod index (brighter/harsher) +
|
||||
a second detuned carrier that creates beating/tension. The beating
|
||||
is what makes it feel "urgent" — not louder, but unsettled.
|
||||
"""
|
||||
duration = 0.5 # 500ms — shorter
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
f_carrier = 1200
|
||||
f_mod = 420
|
||||
|
||||
# Higher mod index = more sidebands = brighter, more aggressive
|
||||
mod_index = 5.0 * np.exp(-t * 7)
|
||||
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
|
||||
|
||||
# Primary carrier
|
||||
carrier1 = np.sin(2 * np.pi * f_carrier * t + modulator)
|
||||
|
||||
# Second carrier, detuned +8Hz — creates beating at 8Hz (nervous flicker)
|
||||
carrier2 = np.sin(2 * np.pi * (f_carrier + 8) * t + modulator * 0.8)
|
||||
|
||||
signal = carrier1 * 0.6 + carrier2 * 0.4
|
||||
|
||||
# Sharp attack (6ms), faster decay
|
||||
attack = np.minimum(t / 0.006, 1.0)
|
||||
decay = np.exp(-t * 4.0)
|
||||
envelope = attack * decay
|
||||
|
||||
# 15% louder than normal
|
||||
return write_wav("sfx_monologue_chime_urgent.wav", signal * envelope * 0.25)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
print("Synthesizing UI sounds for Sprint 7 #440...")
|
||||
print()
|
||||
|
||||
print("[UI-001] cursor_hover — bandpass noise impulse (digital click)")
|
||||
cursor_hover()
|
||||
|
||||
print("[UI-004] weapon_aim — filtered noise + sub thump (mechanical latch)")
|
||||
weapon_aim()
|
||||
|
||||
print("[UI-005] sfx_monologue_chime — FM synthesis bell (PLACEHOLDER)")
|
||||
monologue_chime()
|
||||
|
||||
print("[UI-006] sfx_monologue_chime_urgent — FM + beating (PLACEHOLDER)")
|
||||
monologue_chime_urgent()
|
||||
|
||||
print()
|
||||
print("Done. Convert with: db/connectors/audio-post convert <file.wav>")
|
||||
Reference in New Issue
Block a user