Merge remote-tracking branch 'origin/main' into client
This commit is contained in:
@@ -36,6 +36,10 @@
|
||||
"Bash(db/connectors/sqlite-init)",
|
||||
"Bash(db/connectors/decisions-sync)",
|
||||
|
||||
"Bash(db/connectors/audio-generate *)",
|
||||
"Bash(db/connectors/audio-health)",
|
||||
"Bash(db/connectors/audio-post *)",
|
||||
|
||||
"Bash(make *)",
|
||||
"Bash(make)",
|
||||
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
---
|
||||
name: gen-audio
|
||||
description: >
|
||||
Generate audio assets for The Settled Reach using the Stable Audio Open API
|
||||
(self-hosted Gradio app at tower-of-joy:11500). Use when generating any game
|
||||
audio: ambient loops, SFX, UI sounds, monologue chimes, footsteps, or any
|
||||
sound asset from docs/assets/audio/. Also use when the user asks about audio
|
||||
generation, sound design pipeline, or audio asset iteration. Triggers on:
|
||||
"generate audio", "make sounds", "create ambient", "audio pipeline",
|
||||
"generate sfx", "stable audio", "gen audio", "sound design".
|
||||
---
|
||||
|
||||
# Audio Generation — The Settled Reach
|
||||
|
||||
Generate sonically consistent audio assets using the Stable Audio Open API via
|
||||
wrapper scripts at `db/connectors/audio-*`.
|
||||
|
||||
Asset descriptions, filenames, bus routing, and design intent are documented in
|
||||
`docs/assets/audio/`. This skill provides the prompt system, generation
|
||||
workflow, and quality validation.
|
||||
|
||||
## API Access
|
||||
|
||||
**Never call the API directly.** Use the wrapper scripts:
|
||||
|
||||
```bash
|
||||
# Check API health
|
||||
db/connectors/audio-health
|
||||
|
||||
# Generate audio
|
||||
db/connectors/audio-generate "prompt text" \
|
||||
--duration 10 \
|
||||
--steps 100 \
|
||||
--cfg 7 \
|
||||
--output path/to/output.wav \
|
||||
--timeout 600
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
| Parameter | Default | Range | Notes |
|
||||
|-----------|---------|-------|-------|
|
||||
| `--duration` | 10 | 0-47s | Max 47s per generation. For longer loops, generate 45s with crossfade overlap. |
|
||||
| `--steps` | 100 | 10-200 | More steps = better quality, slower. Use 50 for quick previews, 100-150 for final. |
|
||||
| `--cfg` | 7 | 1-15 | Classifier-free guidance. Higher = more prompt-adherent but less natural. 5-9 is the sweet spot. |
|
||||
| `--output` | auto | — | Output file path. Auto-names from prompt if omitted. |
|
||||
| `--timeout` | 600 | — | Max wait in seconds. Generation can take 2-5 minutes on 11GB VRAM. |
|
||||
|
||||
### Critical Constraints
|
||||
|
||||
- **NEVER parallelize requests.** The server has 11GB VRAM and runs one
|
||||
generation at a time. Always wait for a generation to complete before
|
||||
starting the next. Sequential only.
|
||||
- **Generation takes 2-5 minutes** per clip depending on duration and steps.
|
||||
Be patient. The timeout default (600s) is generous.
|
||||
- **Max 47 seconds** per generation. For 60-90s ambient loops, generate 45s
|
||||
clips and crossfade-stitch in post-processing.
|
||||
- **Output is WAV at 44.1kHz stereo.** Convert to .ogg for Godot import:
|
||||
`ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg`
|
||||
|
||||
## Prompt Assembly
|
||||
|
||||
Every generation uses three parts:
|
||||
|
||||
```
|
||||
[SONIC FAMILY PREFIX] + [CATEGORY TEMPLATE] + [ASSET DESCRIPTION from docs/assets/audio/]
|
||||
```
|
||||
|
||||
Never call the API with just the asset description. Always prepend the sonic
|
||||
family prefix and matching category template.
|
||||
|
||||
- **Sonic palette and families:** Read `references/sonic-palette.md`
|
||||
- **Category templates:** Read `references/category-templates.md` and match by
|
||||
asset type (ambient, sfx, ui)
|
||||
- **Asset description:** Look up the specific asset in `docs/assets/audio/{category}.md`
|
||||
|
||||
## Single Asset Workflow
|
||||
|
||||
1. Find the asset in `docs/assets/audio/{ambient,sfx,ui}.md` — note filename,
|
||||
duration, bus, method, and design intent.
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `db/connectors/audio-health` to verify the API is up.
|
||||
6. Run `db/connectors/audio-generate` with the assembled prompt. **One request
|
||||
at a time. Wait for completion.**
|
||||
7. Listen to the output (or describe it based on file size/duration).
|
||||
8. If acceptable, convert to .ogg and place in `client/assets/audio/`.
|
||||
9. Update the asset status in `docs/assets/audio/{category}.md`.
|
||||
|
||||
## Iteration Workflow
|
||||
|
||||
For each asset, generate 4-6 candidates:
|
||||
|
||||
1. **Generate candidates** — vary the prompt slightly (add/remove descriptors,
|
||||
adjust CFG between 5-9). Run each generation sequentially — never in
|
||||
parallel.
|
||||
2. **Solo test** — does each candidate sound right alone?
|
||||
3. **Stack test** — play the candidate alongside other layers. Does it mask or
|
||||
clash?
|
||||
4. **Fatigue test** (loops only) — can you listen for 5+ minutes without a
|
||||
jarring repeat?
|
||||
5. **Close-your-eyes test** — does it create a mental image or sensation?
|
||||
6. Select the best candidate, trim, normalize, convert.
|
||||
|
||||
## Post-Processing
|
||||
|
||||
After selecting the best generation:
|
||||
|
||||
```bash
|
||||
# Trim silence from start/end
|
||||
ffmpeg -i input.wav -af "silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse,silenceremove=start_periods=1:start_silence=0.1:start_threshold=-50dB,areverse" trimmed.wav
|
||||
|
||||
# LUFS normalize to -16 LUFS (broadcast standard, good for game audio)
|
||||
ffmpeg -i trimmed.wav -af loudnorm=I=-16:LRA=11:TP=-1 normalized.wav
|
||||
|
||||
# Convert to .ogg for Godot
|
||||
ffmpeg -i normalized.wav -c:a libvorbis -q:a 6 output.ogg
|
||||
|
||||
# For loops: verify loop point
|
||||
ffplay -loop 0 output.ogg
|
||||
```
|
||||
|
||||
For ambient loops, create crossfade overlap:
|
||||
```bash
|
||||
# Create a 45s loop with 3s crossfade overlap
|
||||
# (manual: export 48s, crossfade first 3s with last 3s in Audacity)
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis instead:
|
||||
|
||||
```python
|
||||
# Example: 50ms cursor hover tick
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
sr = 44100
|
||||
duration = 0.05 # 50ms
|
||||
t = np.linspace(0, duration, int(sr * duration), endpoint=False)
|
||||
freq = 3200 # Hz
|
||||
signal = np.sin(2 * np.pi * freq * t)
|
||||
envelope = np.exp(-t * 80) # exponential decay
|
||||
audio = (signal * envelope * 32767).astype(np.int16)
|
||||
|
||||
with wave.open("cursor_hover.wav", "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(audio.tobytes())
|
||||
```
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
After generating, verify:
|
||||
- Sound matches the sonic family (insert-tech = synthetic/precise, organic = warm/natural)
|
||||
- Frequency range doesn't mask other layers (check docs/assets/audio/palette.md)
|
||||
- Duration matches spec
|
||||
- No unwanted artifacts (clicks, pops, digital noise at start/end)
|
||||
- Loop point is clean (ambient loops only)
|
||||
- Volume sits well relative to other assets (LUFS normalized)
|
||||
- Passes the close-your-eyes test
|
||||
|
||||
## File Placement
|
||||
|
||||
Generated assets go to `client/assets/audio/` with exact filenames from the
|
||||
asset docs:
|
||||
|
||||
```
|
||||
client/assets/audio/
|
||||
amb_station_base.ogg # Ambient bus
|
||||
amb_workplace_layer.ogg # Ambient bus
|
||||
amb_bar_layer.ogg # Ambient bus
|
||||
amb_corridor_layer.ogg # Ambient bus
|
||||
sfx_footstep_metal.ogg # Player Actions bus
|
||||
sfx_footstep_metal_run.ogg # Player Actions bus
|
||||
cursor_hover.ogg # UI Sounds bus
|
||||
implant_open.ogg # UI Sounds bus
|
||||
fog_recognition.ogg # UI Sounds bus
|
||||
weapon_aim.ogg # UI Sounds bus
|
||||
sfx_monologue_chime.ogg # UI Sounds bus
|
||||
sfx_monologue_chime_urgent.ogg # UI Sounds bus
|
||||
```
|
||||
|
||||
AudioManager discovers these by directory scan — filenames must match exactly.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Category Templates — Audio Generation
|
||||
|
||||
Match the asset's category to the right template. Append after the sonic family
|
||||
prefix and before the specific asset description.
|
||||
|
||||
## Ambient Loops
|
||||
|
||||
**Template:**
|
||||
> Continuous ambient soundscape loop, seamless looping audio, no distinct
|
||||
> beginning or end, steady background atmosphere, [DURATION]s duration
|
||||
|
||||
**Notes:**
|
||||
- Generate at 45s (the SAO sweet spot below the 47s ceiling)
|
||||
- Add 3-5s of overlap material for crossfade looping in post
|
||||
- These play continuously — they must be boring enough to fade into the
|
||||
background but rich enough to reward attention
|
||||
- Station baseline prefix is required (see sonic-palette.md)
|
||||
|
||||
**Steps:** 100-150 (higher quality for loops — artifacts are more noticeable)
|
||||
**CFG:** 5-7 (lower guidance = more natural variation, less mechanical)
|
||||
|
||||
## Sound Effects (SFX)
|
||||
|
||||
**Template:**
|
||||
> Single isolated sound effect, clean recording, [DURATION] duration,
|
||||
> clear attack and natural decay, no background noise, no reverb tail
|
||||
> beyond natural
|
||||
|
||||
**Notes:**
|
||||
- For footsteps: specify surface material, pace, weight
|
||||
- For environmental: specify the physical mechanism (door hinge, cargo latch)
|
||||
- Keep short and punchy — these are event-driven, not continuous
|
||||
|
||||
**Steps:** 80-100 (shorter sounds need fewer steps)
|
||||
**CFG:** 7-9 (more guidance = more precise sound matching)
|
||||
|
||||
## UI Sounds
|
||||
|
||||
**Template:**
|
||||
> Interface feedback sound, digital UI element, [DURATION] duration,
|
||||
> immediate attack, clean decay, isolated sound with no background
|
||||
|
||||
**Notes:**
|
||||
- Most UI sounds are under 200ms — use manual synthesis, not SAO
|
||||
- For SAO-generated UI sounds (>200ms like fog_recognition, chimes):
|
||||
generate at 3-5s duration, then trim to the best portion
|
||||
- Insert-tech prefix is required for lattice/interface sounds
|
||||
- Organic prefix for fog_recognition (cognitive, not technological)
|
||||
|
||||
**Steps:** 80-100
|
||||
**CFG:** 7-9 (precision matters for UI sounds)
|
||||
|
||||
## Chimes / Notification Sounds
|
||||
|
||||
**Template:**
|
||||
> Single musical tone, crystalline quality, [DURATION] duration, gentle
|
||||
> attack, resonant sustain, natural fade out, no accompaniment, no rhythm
|
||||
|
||||
**Notes:**
|
||||
- Generate at 3-5s to give SAO enough temporal context
|
||||
- Trim to the best 0.5-1.0s section
|
||||
- The attack transient is where the character lives — select for that
|
||||
- Normal vs urgent variants: same base prompt, vary "gentle/soft" vs
|
||||
"brighter/sharper/more present"
|
||||
|
||||
**Steps:** 100 (tonal quality matters)
|
||||
**CFG:** 6-8 (some freedom for natural harmonic content)
|
||||
@@ -0,0 +1,45 @@
|
||||
# Sonic Palette — Prompt Prefixes
|
||||
|
||||
Two sonic families. Every generated sound uses one of these as a prompt prefix.
|
||||
|
||||
## Insert-Tech (Synthetic)
|
||||
|
||||
For UI sounds, neural lattice interface, augmented cognition.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Clean digital audio, synthetic electronic tone, precise and clinical sound
|
||||
> design, no reverb, no room ambience, studio-dry recording, futuristic
|
||||
> interface sound
|
||||
|
||||
**Character:** Mathematical precision. The sound of well-designed technology.
|
||||
No organic texture, no room reflections. Exists "inside the head," not in
|
||||
physical space.
|
||||
|
||||
**Frequency range:** 800Hz-4kHz primary. Sharp attack, controlled decay.
|
||||
|
||||
**Used for:** cursor_hover, implant_open, weapon_aim, sfx_monologue_chime,
|
||||
sfx_monologue_chime_urgent
|
||||
|
||||
## Organic (Biological/Environmental)
|
||||
|
||||
For environmental sounds, human cognition, physical world.
|
||||
|
||||
**Prompt prefix:**
|
||||
> Warm organic audio, natural sound recording, slight room ambience,
|
||||
> lived-in industrial space, realistic acoustic properties, authentic
|
||||
> environmental sound
|
||||
|
||||
**Character:** Human warmth. Natural attack and decay. Reverberant — exists
|
||||
in physical space. The sound of a real place with real materials.
|
||||
|
||||
**Frequency range:** 60Hz-3kHz primary. Soft or natural attack, room tail.
|
||||
|
||||
**Used for:** fog_recognition, all amb_* ambient loops, all sfx_footstep_*
|
||||
|
||||
## Station Baseline
|
||||
|
||||
For ambient loops specifically, add the station context:
|
||||
|
||||
> Interior of a large industrial space station, metal and composite
|
||||
> construction, background mechanical hum from power systems, air circulation
|
||||
> audible, no music, no prominent voices unless specified
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
name: asset-gen
|
||||
name: gen-image
|
||||
description: >
|
||||
Generate themed visual assets for the Lords of Ash CK3 total conversion mod
|
||||
using the generate_image MCP tool (Nano Banana / Gemini 2.5 Flash Image).
|
||||
@@ -23,6 +23,13 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
- YAML content loader with hot-reload (#326, D-028) — LinePool system parsing dialogue/monologue YAML into BTreeMap-indexed pools, 4-layer query filtering (access > situation > trust > topic+mood), timestamp-polling hot-reload (dev-only), graceful failure preserves previous content
|
||||
- Line pool format specification (#308) — formal spec at docs/architecture/line-pool-format.md defining YAML structure, tag enums, 4-layer filtering pipeline, prerequisite-to-KG mapping, ID format, and Rust loader interface
|
||||
- InteractionMemory KG schema design (#442, D-064) — design doc at docs/architecture/interaction-memory-schema.md extending FactKnowledge with interaction tracking, 5-state InteractionState enum, monologue prerequisite extension, NpcTolerance reconciliation
|
||||
- Audio discussion decisions D-067 through D-074: recognition chime timing, 5-bus architecture, audio dip profiles, confrontation as cognitive vulnerability, monologue chime placeholder strategy, universal conversation murmur, zone crossfade, hybrid audio generation
|
||||
- Asset pipeline documentation system (docs/assets/) with category-based index, sonic palette, and generation templates for audio, visual, and video pipelines
|
||||
- Stable Audio Open connector and post-processing wrappers (audio-generate, audio-health, audio-post) with timeout handling for 11GB VRAM constraint
|
||||
- gen-audio skill with prompt assembly system (sonic palette prefixes + category templates + asset descriptions)
|
||||
- 6 interaction UI audio assets (#440): cursor_hover, weapon_aim, implant_open, fog_recognition, sfx_monologue_chime, sfx_monologue_chime_urgent — generated via SAO, needs duration trimming (#453)
|
||||
- Dialogue/confrontation ambient dip implementation spec with full Godot AudioBus tween code
|
||||
- Synthesis tooling (tooling/synth_ui_sounds.py) for programmatic insert-tech sound generation
|
||||
|
||||
### Changed
|
||||
- Protocol version bumped from 6 to 7 (pending_recognitions field in ObserverSnapshot)
|
||||
@@ -30,6 +37,7 @@ Format based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
- Monologue trigger system uses .values() iterator (clippy fix)
|
||||
- Monologue schema: relationship prerequisite now requires target and state fields
|
||||
- 389 tests total (70 new) — cognitive delay pipeline, line pool loader, ListeningFocus, content watching, serialization
|
||||
- Renamed asset-gen skill to gen-image for consistent gen-* naming pattern
|
||||
- Client protocol v6 bridge — decode player_stance (4 variants) and player_inventory from ObserverSnapshot, TOGGLE_STANCE_UP/DOWN input actions, 25 gdUnit4 tests
|
||||
- Three-scope z-layer rendering pipeline (D-049) — world z:0-900 inside CanvasGroup, insert overlay CanvasLayer 10, UI CanvasLayer 20, modal CanvasLayer 30. Y-sort contract enforced, reserved VFX/airborne/lower-floor ranges documented
|
||||
- Cursor state machine (#429, D-056) — 4 states (Default/EntityHover/ObjectHover/WeaponAim), 150ms transitions, insert-styled geometric shapes
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generate audio via Stable Audio Open. Whitelistable command.
|
||||
# Usage: audio-generate "prompt text" [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" generate "$@"
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
# Check if the Stable Audio Open API is reachable. Whitelistable command.
|
||||
# Usage: audio-health
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" health
|
||||
Executable
+7
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Audio post-processing (ffmpeg wrapper). Whitelistable command.
|
||||
# Usage: audio-post convert input.wav [--output output.ogg]
|
||||
# audio-post normalize input.wav [--lufs -16]
|
||||
# audio-post trim input.wav [--threshold -50]
|
||||
# audio-post pipeline input.wav [--output output.ogg]
|
||||
exec python3 "$(dirname "$0")/audio_post.py" "$@"
|
||||
@@ -0,0 +1,290 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Stable Audio Open connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Stable Audio Open Gradio app at tower-of-joy:11500.
|
||||
Uses the async Gradio API pattern: POST to submit, SSE stream for results.
|
||||
|
||||
Usage:
|
||||
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav]
|
||||
python3 audio_connector.py health
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import shutil
|
||||
|
||||
CONFIG_PATH = os.path.join(os.path.dirname(__file__), "config.json")
|
||||
|
||||
def load_config():
|
||||
with open(CONFIG_PATH) as f:
|
||||
return json.load(f)
|
||||
|
||||
def get_base_url():
|
||||
config = load_config()
|
||||
return config.get("stable_audio_url", "http://tower-of-joy:11500")
|
||||
|
||||
def health():
|
||||
"""Check if the Stable Audio API is reachable."""
|
||||
base = get_base_url()
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/config", method="GET")
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
version = data.get("version", "unknown")
|
||||
# Extract component info for the generate endpoint
|
||||
api_names = []
|
||||
for dep in data.get("dependencies", []):
|
||||
name = dep.get("api_name", "")
|
||||
if name and not name.startswith("js_"):
|
||||
api_names.append(name)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"url": base,
|
||||
"gradio_version": version,
|
||||
"api_endpoints": api_names
|
||||
}, indent=2))
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"url": base,
|
||||
"error": str(e)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600):
|
||||
"""
|
||||
Generate audio from a text prompt.
|
||||
|
||||
Args:
|
||||
prompt: Text description of the audio to generate
|
||||
duration: Duration in seconds (0-47, default 10)
|
||||
steps: Number of diffusion steps (default 100, lower = faster but lower quality)
|
||||
cfg: Classifier-free guidance scale (default 7)
|
||||
output: Output file path (default: auto-named in current directory)
|
||||
timeout: Maximum wait time in seconds (default 600 = 10 minutes)
|
||||
"""
|
||||
base = get_base_url()
|
||||
api_url = f"{base}/gradio_api/call/generate_audio"
|
||||
|
||||
if output is None:
|
||||
# Auto-name: sanitize prompt to a filename
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
output = f"{safe}_{int(duration)}s.wav"
|
||||
|
||||
# Step 1: Submit the generation request
|
||||
payload = json.dumps({"data": [prompt, duration, steps, cfg]})
|
||||
req = urllib.request.Request(
|
||||
api_url,
|
||||
data=payload.encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print(f"Submitting generation request...", file=sys.stderr)
|
||||
print(f" Prompt: {prompt}", file=sys.stderr)
|
||||
print(f" Duration: {duration}s, Steps: {steps}, CFG: {cfg}", file=sys.stderr)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
result = json.loads(resp.read())
|
||||
event_id = result.get("event_id")
|
||||
if not event_id:
|
||||
print(json.dumps({"ok": False, "error": "No event_id returned", "response": result}, indent=2))
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Submit failed: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
print(f" Event ID: {event_id}", file=sys.stderr)
|
||||
print(f" Waiting for generation (timeout: {timeout}s)...", file=sys.stderr)
|
||||
|
||||
# Step 2: Poll the SSE stream for results
|
||||
stream_url = f"{api_url}/{event_id}"
|
||||
start_time = time.time()
|
||||
result_data = None
|
||||
last_status = None
|
||||
|
||||
while time.time() - start_time < timeout:
|
||||
try:
|
||||
req = urllib.request.Request(stream_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
# Read SSE events
|
||||
current_event = None
|
||||
for line_bytes in resp:
|
||||
line = line_bytes.decode("utf-8").strip()
|
||||
|
||||
if line.startswith("event: "):
|
||||
current_event = line[7:]
|
||||
elif line.startswith("data: ") and current_event:
|
||||
data_str = line[6:]
|
||||
|
||||
if current_event == "heartbeat":
|
||||
elapsed = int(time.time() - start_time)
|
||||
if elapsed % 30 == 0 and elapsed > 0:
|
||||
print(f" Still generating... ({elapsed}s elapsed)", file=sys.stderr)
|
||||
continue
|
||||
|
||||
if current_event == "error":
|
||||
error_msg = data_str
|
||||
try:
|
||||
error_msg = json.loads(data_str)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
print(json.dumps({"ok": False, "error": "Generation failed", "details": error_msg}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if current_event == "complete":
|
||||
try:
|
||||
result_data = json.loads(data_str)
|
||||
except json.JSONDecodeError:
|
||||
result_data = data_str
|
||||
break
|
||||
|
||||
if current_event == "progress":
|
||||
try:
|
||||
progress = json.loads(data_str)
|
||||
# Gradio progress events vary; log what we get
|
||||
status = str(progress)[:80]
|
||||
if status != last_status:
|
||||
print(f" Progress: {status}", file=sys.stderr)
|
||||
last_status = status
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
continue
|
||||
|
||||
if result_data is not None:
|
||||
break
|
||||
|
||||
except urllib.error.URLError as e:
|
||||
# Connection dropped — retry after brief pause
|
||||
time.sleep(2)
|
||||
continue
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Stream error: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if result_data is None:
|
||||
print(json.dumps({"ok": False, "error": f"Generation timed out after {timeout}s"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Step 3: Download the audio file
|
||||
# Gradio returns file info in the data array
|
||||
elapsed = round(time.time() - start_time, 1)
|
||||
print(f" Generation complete ({elapsed}s)", file=sys.stderr)
|
||||
|
||||
try:
|
||||
# result_data is typically [{"path": "...", "url": "...", ...}] or similar
|
||||
if isinstance(result_data, list) and len(result_data) > 0:
|
||||
audio_info = result_data[0]
|
||||
elif isinstance(result_data, dict) and "data" in result_data:
|
||||
audio_info = result_data["data"][0] if result_data["data"] else None
|
||||
else:
|
||||
audio_info = result_data
|
||||
|
||||
# Extract the file URL
|
||||
file_url = None
|
||||
if isinstance(audio_info, dict):
|
||||
file_url = audio_info.get("url") or audio_info.get("path")
|
||||
elif isinstance(audio_info, str):
|
||||
file_url = audio_info
|
||||
|
||||
if not file_url:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "Could not extract audio URL from response",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Handle relative URLs
|
||||
if file_url.startswith("/"):
|
||||
file_url = f"{base}{file_url}"
|
||||
elif not file_url.startswith("http"):
|
||||
file_url = f"{base}/file={file_url}"
|
||||
|
||||
# Download the file
|
||||
print(f" Downloading to {output}...", file=sys.stderr)
|
||||
req = urllib.request.Request(file_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
with open(output, "wb") as f:
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
file_size = os.path.getsize(output)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
"duration_requested": duration,
|
||||
"steps": steps,
|
||||
"cfg": cfg,
|
||||
"prompt": prompt,
|
||||
"generation_time_s": elapsed
|
||||
}, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"Download failed: {e}",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage:")
|
||||
print(" audio_connector.py health")
|
||||
print(" audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]")
|
||||
sys.exit(1)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "health":
|
||||
health()
|
||||
elif cmd == "generate":
|
||||
if len(sys.argv) < 3:
|
||||
print("Error: prompt required", file=sys.stderr)
|
||||
print("Usage: audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
prompt = sys.argv[2]
|
||||
duration = 10.0
|
||||
steps = 100
|
||||
cfg = 7.0
|
||||
output = None
|
||||
timeout = 600
|
||||
|
||||
# Parse optional args
|
||||
i = 3
|
||||
while i < len(sys.argv):
|
||||
if sys.argv[i] == "--duration" and i + 1 < len(sys.argv):
|
||||
duration = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--steps" and i + 1 < len(sys.argv):
|
||||
steps = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--cfg" and i + 1 < len(sys.argv):
|
||||
cfg = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--output" and i + 1 < len(sys.argv):
|
||||
output = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
|
||||
timeout = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output, timeout=timeout)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,171 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audio post-processing wrapper around ffmpeg.
|
||||
|
||||
Subcommands:
|
||||
convert — WAV to OGG (libvorbis, quality 6)
|
||||
normalize — LUFS normalize to -16 LUFS (broadcast standard)
|
||||
trim — Remove leading/trailing silence
|
||||
pipeline — trim + normalize + convert (full post-processing chain)
|
||||
|
||||
All operations write to a new file (never overwrites input).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import shutil
|
||||
import json
|
||||
|
||||
|
||||
def check_ffmpeg():
|
||||
if not shutil.which("ffmpeg"):
|
||||
print(json.dumps({"ok": False, "error": "ffmpeg not found in PATH"}))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run_ffmpeg(args, description):
|
||||
"""Run ffmpeg, capture output, return success."""
|
||||
cmd = ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error"] + args
|
||||
print(f" {description}", file=sys.stderr)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"ffmpeg failed: {result.stderr.strip()}",
|
||||
"command": " ".join(cmd)
|
||||
}))
|
||||
sys.exit(1)
|
||||
return True
|
||||
|
||||
|
||||
def cmd_convert(args):
|
||||
"""Convert WAV to OGG (libvorbis)."""
|
||||
output = args.output or args.input.rsplit(".", 1)[0] + ".ogg"
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
f"converting {os.path.basename(args.input)} → {os.path.basename(output)}"
|
||||
)
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def cmd_normalize(args):
|
||||
"""LUFS normalize audio file."""
|
||||
output = args.output or _suffixed(args.input, "_norm")
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
output],
|
||||
f"normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_trim(args):
|
||||
"""Trim leading/trailing silence."""
|
||||
output = args.output or _suffixed(args.input, "_trimmed")
|
||||
af = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af, output],
|
||||
f"trimming silence (threshold: {args.threshold}dB)"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_pipeline(args):
|
||||
"""Full post-processing: trim → normalize → convert to OGG."""
|
||||
base = args.input.rsplit(".", 1)[0]
|
||||
trimmed = base + "_trimmed.wav"
|
||||
normalized = base + "_norm.wav"
|
||||
output = args.output or base + ".ogg"
|
||||
|
||||
# Step 1: trim
|
||||
af_trim = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af_trim, trimmed],
|
||||
"step 1/3: trimming silence"
|
||||
)
|
||||
|
||||
# Step 2: normalize
|
||||
run_ffmpeg(
|
||||
["-i", trimmed, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
normalized],
|
||||
f"step 2/3: normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
|
||||
# Step 3: convert
|
||||
run_ffmpeg(
|
||||
["-i", normalized, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
f"step 3/3: converting to OGG"
|
||||
)
|
||||
|
||||
# Clean up intermediates
|
||||
os.remove(trimmed)
|
||||
os.remove(normalized)
|
||||
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def _suffixed(path, suffix):
|
||||
base, ext = os.path.splitext(path)
|
||||
return base + suffix + ext
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Audio post-processing (ffmpeg wrapper)")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# convert
|
||||
p = sub.add_parser("convert", help="WAV → OGG")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output file (default: same name .ogg)")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.set_defaults(func=cmd_convert)
|
||||
|
||||
# normalize
|
||||
p = sub.add_parser("normalize", help="LUFS normalize")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.set_defaults(func=cmd_normalize)
|
||||
|
||||
# trim
|
||||
p = sub.add_parser("trim", help="Trim silence")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_trim)
|
||||
|
||||
# pipeline
|
||||
p = sub.add_parser("pipeline", help="Full post-processing: trim + normalize + convert")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output OGG file")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_pipeline)
|
||||
|
||||
args = parser.parse_args()
|
||||
check_ffmpeg()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
|
||||
"qdrant_url": "http://tower-of-joy:6333",
|
||||
"ollama_url": "http://tower-of-joy:11434",
|
||||
"stable_audio_url": "http://tower-of-joy:11500",
|
||||
"collection": "commonwealth",
|
||||
"embed_model": "nomic-embed-text",
|
||||
"embed_dimensions": 768
|
||||
|
||||
+3
-3
@@ -10,9 +10,9 @@ Cross-domain decisions live in one file with cross-reference notes in related fi
|
||||
|
||||
| File | Domain | Decisions |
|
||||
|------|--------|-----------|
|
||||
| [architecture.md](architecture.md) | Technical foundation | D-008, D-009, D-010, D-012, D-020, D-026, D-030, D-031, D-041, D-042, D-054, D-055, D-066 |
|
||||
| [perception.md](perception.md) | Player observation | D-011, D-015, D-016, D-017, D-018, D-019, D-033, D-035, D-043, D-044, D-045, D-046, D-047, D-048, D-049, D-052, D-056, D-057, D-058, D-059, D-060, D-061 |
|
||||
| [content.md](content.md) | NPC, dialogue, templates | D-023, D-024, D-025, D-028, D-029, D-032, D-034, D-035, D-036, D-037, D-050, D-062, D-063, D-064 |
|
||||
| [architecture.md](architecture.md) | Technical foundation | D-008, D-009, D-010, D-012, D-020, D-026, D-030, D-031, D-041, D-042, D-054, D-055, D-066, D-068, D-073 |
|
||||
| [perception.md](perception.md) | Player observation | D-011, D-015, D-016, D-017, D-018, D-019, D-033, D-035, D-043, D-044, D-045, D-046, D-047, D-048, D-049, D-052, D-056, D-057, D-058, D-059, D-060, D-061, D-067, D-069, D-070, D-071, D-072 |
|
||||
| [content.md](content.md) | NPC, dialogue, templates | D-023, D-024, D-025, D-028, D-029, D-032, D-034, D-035, D-036, D-037, D-050, D-062, D-063, D-064, D-074 |
|
||||
| [scope.md](scope.md) | Game concept, prototype | D-001, D-003, D-005, D-006, D-007, D-013, D-014, D-027, D-038, D-039, D-051, D-053, D-065 |
|
||||
| [process.md](process.md) | Team, workflow | D-004, D-021, D-022, D-040 |
|
||||
| [questions.md](questions.md) | Open questions | Q-001 through Q-026 |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Architecture Decisions
|
||||
|
||||
Technical foundation decisions that constrain implementation: engine, client-server, ECS, simulation, testability, performance budgets.
|
||||
Technical foundation decisions that constrain implementation: engine, client-server, ECS, simulation, testability, performance budgets, audio architecture.
|
||||
|
||||
---
|
||||
|
||||
@@ -178,6 +178,36 @@ Technical foundation decisions that constrain implementation: engine, client-ser
|
||||
- **Raised by:** Team Leader (Jeroen) — proposed retina scaling analogy and 2x2 geometry constraint. Tyre (feasibility: trivial, half-day integration). Gestalt (approved with 2x2 constraint resolving LOS readability concern). Ozzie (approved: solves sprite scale without uncanny mismatch).
|
||||
- **Dissent:** Gestalt initially objected to dual-scale (mental model mismatch for cover/LOS). Resolved by the 2x2 geometry minimum constraint — all cover maps 1:1 at visual scale.
|
||||
|
||||
### D-068: 5-bus audio architecture
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Audio uses a 5-bus architecture with player-facing volume sliders:
|
||||
1. **Music** — future/empty in v0.1. Reserved for diegetic Meridian music in social spaces.
|
||||
2. **Ambient** — station hum ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) asset 1) + zone overlays (assets 2-4). Continuous soundscape.
|
||||
3. **World SFX** — NPC footsteps, doors, environmental events. Diegetic world sounds not caused by player.
|
||||
4. **Player Actions** — player footsteps (assets 5-6), future: combat sounds, item interactions. Sounds player directly causes.
|
||||
5. **UI Sounds** — cursor hover, implant open, monologue chimes (assets 7-8), fog recognition. Interface feedback.
|
||||
- **Client implementation:**
|
||||
- AudioManager GDScript autoload singleton on client branch (lives with rendering, per [D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc)).
|
||||
- Audio assets committed to audio branch (content, not code).
|
||||
- Directory-scan registry pattern: AudioManager scans `res://audio/` on startup, maps filenames to AudioStream resources. No hardcoded asset list.
|
||||
- If directory empty or asset missing, all play methods no-op with visual fallback (per [D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) architecture).
|
||||
- 5 player-facing volume sliders, one per bus. Accessible via settings.
|
||||
- **Bus routing dropped:**
|
||||
- **Dialogue bus removed** — no voice acting in v0.1. NPC conversation murmur goes on World SFX (event-driven, per [D-072](perception.md#d-072-universal-event-driven-conversation-murmur)).
|
||||
- **Rationale:** 5 buses provide player control granularity (disable UI sounds, boost World SFX for eavesdropping, mute Ambient for focus) without over-segmentation. Directory-scan registry eliminates hardcoded asset paths — audio branch can add files without touching client code. No-op fallback means client works identically with or without audio.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), audio dip ([D-069](perception.md#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), client-server architecture ([D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc))
|
||||
- **Raised by:** Team Leader (channel split directive), Tyre (architecture), Inigo and Gestalt (bus refinement)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-073: Zone crossfade approach — hard boundary, soft audio transition
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Zone audio transitions use hard tile boundary triggers with 1.5-2s audio crossfade tweens. Server sends zone_id per tile in ObserverSnapshot (server-authoritative zone assignment). AudioManager receives zone changes and tweens between ambient layers. No blended overlap zones — the transition smoothness comes from audio fade duration, not spatial blending.
|
||||
- **Implementation:** AudioManager stub in Sprint 7 (5-bus setup, directory registry). Full zone crossfade implementation deferred to Sprint 8+.
|
||||
- **Rationale:** Hard boundaries with soft audio = predictable for simulation, pleasant for player. Avoids complex overlap zone geometry. Crossfade duration (1.5-2s) is long enough to feel smooth, short enough that walking back-and-forth across boundary doesn't create audio chaos.
|
||||
- **Cross-reference:** Audio architecture ([D-068](#d-068-5-bus-audio-architecture)), client-server ([D-020](#d-020-engine-and-architecture-selection--godot-client--rust-simulation-via-subprocessipc)), ambient assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open))
|
||||
- **Raised by:** Tyre
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*13 decisions. Last updated: 2026-02-14*
|
||||
*16 decisions. Last updated: 2026-02-16*
|
||||
|
||||
+18
-2
@@ -1,6 +1,6 @@
|
||||
# Content Decisions
|
||||
|
||||
How narrative, NPCs, and world content are created: content tiers, NPC generation, templates, dialogue, population.
|
||||
How narrative, NPCs, and world content are created: content tiers, NPC generation, templates, dialogue, population, audio aesthetic identity.
|
||||
|
||||
---
|
||||
|
||||
@@ -142,6 +142,22 @@ How narrative, NPCs, and world content are created: content tiers, NPC generatio
|
||||
- **Raised by:** Paula (three phases + KG recording), Stig (WASD mechanic + 300ms fade), Ozzie (consequences), Nigel (tolerance per seed)
|
||||
- **Dissent:** None.
|
||||
|
||||
### D-074: Audio aesthetic identity — insert-tech vs organic
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Audio design uses two sonic families with distinct aesthetic identities:
|
||||
- **Insert-tech sounds:** Synthetic, precise, clinical, no reverb. Monologue chimes ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) assets 7-8), UI feedback, future: lattice interface sounds, biometric scan confirmations. Represents neural lattice interface — artificial, computational, clinical.
|
||||
- **Organic sounds:** Warm, breathy, natural decay, environmental reverb. Footsteps, ambient layers, NPC movement, environmental events. Represents human cognition and physical reality — warm, imperfect, lived-in.
|
||||
- **Thematic mapping:** Insert-tech = neural lattice (computational precision, trustworthy but inhuman). Organic = human cognition and environment (warm, ambiguous, interpretive). Maps to [D-018](perception.md#d-018-three-range-sound-model) trust model: close/organic sounds = reliable, long-range/insert sounds = potentially compromised.
|
||||
- **Station palette — "functional warmth":**
|
||||
- Low-freq foundation (60-120Hz): station hum, span gate vibration, machinery drone.
|
||||
- Mid-freq texture (200-800Hz): footsteps, object handling, conversation murmur.
|
||||
- High-freq detail (2-8kHz, sparse): scanner pings, metal impacts, environmental detail.
|
||||
- No harsh frequencies, no industrial brutality. The station is SETTLED — lived-in, maintained, comfortable enough to be complacent.
|
||||
- **Environmental neutrality per [D-045](perception.md#d-045-art-direction--environmental-neutrality-strict-zero-shift):** Ambient audio does NOT shift with narrative state. Same station hum before and after conspiracy discovery. No music stingers for investigation progress. Emotional weight comes from monologue and player knowledge, not audio cues.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), sound model ([D-018](perception.md#d-018-three-range-sound-model)), environmental neutrality ([D-045](perception.md#d-045-art-direction--environmental-neutrality-strict-zero-shift)), setting ([D-036](#d-036-sova-transit-district--krenn-system-as-v01-setting))
|
||||
- **Raised by:** Inigo (insert-tech/organic split), Paula (cognitive architecture framing and trust model connection)
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*14 decisions. Last updated: 2026-02-13*
|
||||
*15 decisions. Last updated: 2026-02-16*
|
||||
|
||||
+65
-2
@@ -1,6 +1,6 @@
|
||||
# Perception Decisions
|
||||
|
||||
How the player observes and interacts with the world: camera, fog, line-of-sight, sound, monologue, perception modes.
|
||||
How the player observes and interacts with the world: camera, fog, line-of-sight, sound, monologue, perception modes, audio dip profiles, cognitive delays.
|
||||
|
||||
---
|
||||
|
||||
@@ -273,6 +273,69 @@ How the player observes and interacts with the world: camera, fog, line-of-sight
|
||||
- **Raised by:** Stig (UI spec + no portraits), Lead (20% height constraint + max-width directive)
|
||||
- **Dissent:** Stig initially proposed 25% height and 50% width centered. Lead constrained to 20% height and max-width.
|
||||
|
||||
### D-067: Recognition chime fires at onset of cognitive delay
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** The monologue recognition chime ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open) assets 7-8) fires at the ONSET of the cognitive delay ([D-060](#d-060-cognitive-delay-for-fog-recognition)), not at its completion. Chime duration 300-400ms, overlapping with the start of the delay. The chime is "unresolved" — it opens a question, doesn't answer one. Full sequence when recognizing an entity in fog: (1) hear/sense something → (2) chime plays → (3) 0.6s cognitive delay begins (concurrent with chime tail) → (4) monologue text appears during delay ("Those footsteps... that's Kael's walk") → (5) blob transitions to [D-033](#d-033-entity-color--relationship-to-player) color + silhouette feature → (6) recognition complete.
|
||||
- **Rationale:** The chime marks the character's attention shifting, not the recognition completing. It creates an "unresolved" sensation — something is happening in the character's mind. Monologue text provides the resolution. This prevents the chime from feeling like a UI notification and instead makes it part of the character's cognitive process.
|
||||
- **Resolves:** Q-014
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), cognitive delay ([D-060](#d-060-cognitive-delay-for-fog-recognition)), fog recognition ([D-059](#d-059-fog--shader-based-five-layers-knowledge-graph-driven)), monologue ([D-016](#d-016-internal-monologue-as-core-perceptionatmosphere-system))
|
||||
- **Raised by:** Gestalt (proposal), confirmed by Team Leader (Jeroen)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-069: Audio dip profiles for dialogue and confrontation
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Three audio dip profiles modify bus volumes during player focus states, all relative to player slider settings (proportional, not absolute):
|
||||
- **Dialogue dip:** Ambient -6 to -8dB, World SFX 0dB, Player Actions 0dB, UI 0dB. 300ms ease-in, 500ms ease-out. Reduces environmental noise to foreground conversation without muting world events.
|
||||
- **Confrontation dip:** Ambient -10 to -12dB + low-pass filter (20kHz → 800Hz), World SFX -4 to -6dB (graduated — loud events like sprinting footsteps break through, quiet sneaking doesn't), Player Actions 0dB, UI 0dB. 500ms ease-in, 1000ms ease-out with filter sweep. Creates emotional/cognitive muffling per [D-070](#d-070-confrontation-as-cognitive-vulnerability).
|
||||
- **ListeningFocus boost:** World SFX +2 to +3dB when player is stationary for 30+ ticks (eavesdrop bonus, per [D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost)). Expresses heightened attention as audio gain.
|
||||
- **Key behaviors:**
|
||||
- All dB values are proportional to player slider setting, not absolute. If player sets World SFX to 50%, the -6dB dip applies to that 50% base.
|
||||
- Dips are interruptible — walk-away or stance change kills current tween, starts new tween to neutral.
|
||||
- Graduated World SFX dip during confrontation: loud, urgent events (sprinting, doors slamming, alarms) break through at reduced volume. Quiet, careful movement may not be noticed.
|
||||
- **Rationale:** Dips model finite attention. Dialogue suppresses ambient noise (focus on conversation). Confrontation suppresses both ambient and peripheral sounds (emotional tunnel vision per [D-070](#d-070-confrontation-as-cognitive-vulnerability)). ListeningFocus boost rewards deliberate eavesdropping. All effects proportional to user volume settings respects player accessibility choices.
|
||||
- **Cross-reference:** Audio architecture ([D-068](architecture.md#d-068-5-bus-audio-architecture)), confrontation vulnerability ([D-070](#d-070-confrontation-as-cognitive-vulnerability)), eavesdropping ([D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost)), audio aesthetic ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic))
|
||||
- **Raised by:** Inigo (initial spec), Gestalt (graduated approach + ListeningFocus boost), Tyre (implementation details), Ozzie (proportional dip to respect player settings)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-070: Confrontation as cognitive vulnerability
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Confrontation creates perceptual vulnerability through reduced ambient awareness. Design principle: "Emotional focus creates perceptual vulnerability." Player actions requiring cognitive focus (confrontation dialogue, future: deep terminal reading, complex insert queries) mechanically reduce ambient perception via audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)) and may suppress peripheral monologue triggers. This is NOT a punishment — it's a realistic consequence of finite attention. The player mitigates risk by choosing WHERE and WHEN to confront (safe location vs exposed corridor, early shift vs busy period).
|
||||
- **Mechanical expression:**
|
||||
- During confrontation: ambient sounds muffled, quiet NPC movement may go undetected (graduated World SFX dip means careful footsteps suppressed, sprinting footsteps still audible).
|
||||
- Post-confrontation: delayed monologue may fire for missed events ("Wait — did someone just pass by?").
|
||||
- World continues per [D-045](#d-045-art-direction--environmental-neutrality-strict-zero-shift) environmental indifference — no scripted ambushes, but NPCs on routine may coincidentally pass while player is focused.
|
||||
- Vulnerability is probabilistic and emergent, not scripted.
|
||||
- **Consistency principle:** Connects to [D-053](scope.md#d-053-movement-as-stance-toggle-system) sprint suppressing monologue (physical focus limits interpretation). Confrontation suppresses ambient perception (emotional focus limits peripheral awareness). High-focus activities suppress peripheral cognition — this is the shared rule.
|
||||
- **Must feel "felt, not computed" (Paula):** Muffling is psychological tunnel vision, not a debuff tooltip. No UI indicator. Player realizes in retrospect: "I was so focused on Sera I didn't hear Kael walk past."
|
||||
- **Rationale:** Makes WHERE and WHEN to confront meaningful decisions. Confronting in a private corner = safer. Confronting in a busy corridor during shift change = riskier. Player agency through spatial and temporal choice, not RNG.
|
||||
- **Cross-reference:** Audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), confrontation UI ([D-063](content.md#d-063-confrontation--same-box-different-weight)), environmental indifference ([D-045](#d-045-art-direction--environmental-neutrality-strict-zero-shift)), stance system ([D-053](scope.md#d-053-movement-as-stance-toggle-system)), monologue ([D-016](#d-016-internal-monologue-as-core-perceptionatmosphere-system))
|
||||
- **Raised by:** Gestalt (vulnerability principle), Ozzie (graduated dip approach), Paula ("felt not computed" framing)
|
||||
- **Dissent:** None
|
||||
|
||||
### D-071: No ambient dip for eavesdropping — ListeningFocus boost
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** Overheard conversation (eavesdropping) does NOT apply dialogue dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)). Cognitive state is opposite of direct conversation: extracting signal from noise requires MORE ambient awareness, not less. Instead, eavesdrop quality improves via ListeningFocus boost (+2 to +3dB World SFX when stationary 30+ ticks). Eavesdrop information quality is: distance-dependent, stance-modified ([D-053](scope.md#d-053-movement-as-stance-toggle-system) Careful stance grants "tell notice bonus"), ListeningFocus-boosted, and ambient-noise-penalized.
|
||||
- **Ambient zone affects eavesdrop difficulty:**
|
||||
- Bar (high ambient) = conversations hidden in murmur, requires proximity + ListeningFocus.
|
||||
- Workplace (moderate ambient) = conversations semi-conspicuous, distance-dependent.
|
||||
- Corridor (low ambient) = conversations conspicuous, easy to overhear from distance.
|
||||
- **Information confidence:** Overheard information enters knowledge graph at lower confidence (KnowsOf, not KnowsDetails per [D-041](architecture.md#d-041-knowledge-graph-data-model)) unless ListeningFocus + proximity allows high-quality capture.
|
||||
- **Rationale:** Direct conversation = character focuses, ambient dips. Overheard conversation = character strains to hear, ambient stays or boosts. Opposite cognitive states produce opposite audio profiles. Creates meaningful difference between talking TO someone vs listening IN on someone.
|
||||
- **Cross-reference:** Audio dip ([D-069](#d-069-audio-dip-profiles-for-dialogue-and-confrontation)), stance system ([D-053](scope.md#d-053-movement-as-stance-toggle-system)), knowledge graph ([D-041](architecture.md#d-041-knowledge-graph-data-model)), zone audio ([D-072](#d-072-universal-event-driven-conversation-murmur))
|
||||
- **Raised by:** Gestalt (cognitive state framing), Paula (zone-conspicuousness model), unanimously endorsed
|
||||
- **Dissent:** None
|
||||
|
||||
### D-072: Universal event-driven conversation murmur
|
||||
- **Date:** 2026-02-16
|
||||
- **Decision:** NPC-to-NPC conversations use a single universal murmur sound asset, not zone-specific variants. Zone ambient determines conspicuousness: bar ambient (high murmur density) hides conversations, corridor ambient (low background) makes conversations conspicuous. Maps to [D-047](#d-047-art-direction--two-tier-animation-system) two-tier behavior visibility: bar conversations are Tier 1 (clearly happening, content obscured), corridor conversations are Tier 2 (ambiguous — are they talking or just standing near each other?).
|
||||
- **Asset split:**
|
||||
- Bar ambient murmur: baked into `amb_bar_layer.ogg` (continuous background per [D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)).
|
||||
- NPC proximity murmur: separate event-driven asset, deferred to future sprint (not in Sprint 7 scope). When implemented, same asset plays everywhere; zone ambient determines audibility.
|
||||
- **Rationale:** One murmur asset + zone-dependent conspicuousness creates the signal/noise dynamic naturally. Bar = high noise floor, conversations blend in. Corridor = low noise floor, conversations stand out. Simpler content production, richer emergent behavior.
|
||||
- **Cross-reference:** Audio assets ([D-038](scope.md#d-038-audio-in-v01-scope--8-files-via-stable-audio-open)), two-tier animation ([D-047](#d-047-art-direction--two-tier-animation-system)), eavesdropping ([D-071](#d-071-no-ambient-dip-for-eavesdropping--listeningfocus-boost))
|
||||
- **Raised by:** Paula (zone-conspicuousness model), Inigo (scoping to future sprint)
|
||||
- **Dissent:** None
|
||||
|
||||
---
|
||||
|
||||
*23 decisions. Last updated: 2026-02-14*
|
||||
*29 decisions. Last updated: 2026-02-16*
|
||||
|
||||
@@ -65,10 +65,9 @@ Tracked questions awaiting discussion or resolution.
|
||||
- **Source:** Content Gap Analysis Workshop (Mellanie R2)
|
||||
|
||||
### Q-014: Audio timing with monologue chime
|
||||
- **Status:** Open
|
||||
- **Question:** When does the monologue chime fire relative to text appearance? Before, simultaneous, or overlapping? Affects monologue line writing rhythm.
|
||||
- **Assigned to:** Gestalt, Ozzie
|
||||
- **Source:** Content Gap Analysis Workshop (Mellanie R2)
|
||||
- **Status:** Resolved → [D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay)
|
||||
- **Resolution:** Chime fires at ONSET of cognitive delay, not completion. 300-400ms duration, overlapping delay start. Chime is "unresolved" — opens a question, doesn't answer one. Sequence: hear/sense → chime plays → 0.6s delay begins → monologue text during delay → blob transitions to D-033 color → recognition complete.
|
||||
- **Date resolved:** 2026-02-16
|
||||
|
||||
### Q-015: Generation expansion for THE FRIEND content
|
||||
- **Status:** Open
|
||||
|
||||
+5
-2
@@ -114,8 +114,11 @@ What we're building: game concept, design pillars, prototype definition, map spe
|
||||
7. `sfx_monologue_chime.ogg` — soft crystalline tone, "neural lattice firing" feel (0.5-1.0s, monologue appearance)
|
||||
8. `sfx_monologue_chime_urgent.ogg` — sharper variant for contradiction/anomaly observations (0.5-1.0s)
|
||||
- **Architecture:** Event-driven with asset registry + visual fallback. Simulation emits typed sound events; client renders as audio (if asset exists) or visual indicator + monologue trigger (if not). Ambient loops managed separately from event-driven sounds. Monologue chime is a UI sound, not a simulation sound.
|
||||
- **Cross-reference:** Three-range sound model ([D-018](perception.md#d-018-three-range-sound-model)), sound event architecture (Gestalt R2 section 6)
|
||||
- **Raised by:** Ozzie (Round 1 minimum viable proposal, Round 2 full spec), project lead (confirmed, directives #3 and #9)
|
||||
- **Amendment (2026-02-16, Audio Pipeline Kickoff):**
|
||||
- **Hybrid audio generation approach:** Stable Audio Open (SAO) for sounds >200ms with organic character (footsteps, ambient layers). Manual synthesis for sounds <200ms with precise/digital character (UI clicks, scanner beeps). SAO ceiling ~47s; accept 45s loops with crossfade for ambient layers. The insert-tech/organic split ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic)) maps to synthesis/generation split.
|
||||
- **Sprint 7 scope expansion:** Ticket #440 expanded from 6 to 8 assets, adding monologue chimes (assets 7-8) as deliberate placeholders with Sprint 8 redo mandate. Chimes are critical for [D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay) cognitive delay feel but acknowledged as difficult to generate correctly — manual synthesis required for production quality.
|
||||
- **Cross-reference:** Three-range sound model ([D-018](perception.md#d-018-three-range-sound-model)), sound event architecture (Gestalt R2 section 6), audio aesthetic ([D-074](content.md#d-074-audio-aesthetic-identity--insert-tech-vs-organic)), recognition chime ([D-067](perception.md#d-067-recognition-chime-fires-at-onset-of-cognitive-delay))
|
||||
- **Raised by:** Ozzie (Round 1 minimum viable proposal, Round 2 full spec), project lead (confirmed, directives #3 and #9). Amendment raised by Inigo (hybrid approach), endorsed by Tyre.
|
||||
- **Dissent:** Mellanie and Araminta both proposed deferring audio; project lead overruled. Visual sound indicators remain complementary to audio (not replacement).
|
||||
|
||||
### D-039: v0.1 wow moment scope — all 6 moments
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# Asset Pipeline Index
|
||||
|
||||
Centralized tracking for all production assets across pipelines. Each pipeline has its own index, palette (style bible), and per-category asset tables.
|
||||
|
||||
## Pipelines
|
||||
|
||||
| Pipeline | Index | Palette | Status |
|
||||
|----------|-------|---------|--------|
|
||||
| [Audio](audio/README.md) | [audio/README.md](audio/README.md) | [audio/palette.md](audio/palette.md) | Active — Sprint 7 |
|
||||
| [Visual](visual/README.md) | [visual/README.md](visual/README.md) | [visual/palette.md](visual/palette.md) | Stub |
|
||||
| [Video](video/README.md) | [video/README.md](video/README.md) | [video/palette.md](video/palette.md) | Stub |
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
docs/assets/
|
||||
README.md # This file — master index
|
||||
_templates/
|
||||
audio.md # Row template + detailed entry template for audio
|
||||
sprite.md # Row template + detailed entry template for sprites
|
||||
video.md # Row template + detailed entry template for video
|
||||
audio/
|
||||
README.md # Audio pipeline index — categories, status summary
|
||||
palette.md # Sonic palette, aesthetic guide, generation approach
|
||||
ambient.md # All ambient loops (station, planet, biome, weather)
|
||||
sfx.md # All SFX (footsteps, doors, impacts, events)
|
||||
ui.md # All UI sounds (cursor, chimes, insert-tech)
|
||||
visual/
|
||||
README.md # Visual pipeline index
|
||||
palette.md # Art direction, "the angle", color relationships
|
||||
sprites.md # Entities (NPCs, player, creatures)
|
||||
furniture.md # Furniture items (hundreds, by location/type)
|
||||
tilesets.md # Floor/wall tiles by biome/planet
|
||||
icons.md # UI icons, status indicators
|
||||
effects.md # Fog shaders, particles, overlays
|
||||
video/
|
||||
README.md # Video pipeline index
|
||||
palette.md # Motion/editing style guide
|
||||
trailers.md # Marketing / Steam page trailers
|
||||
cutscenes.md # In-game narrative moments
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **One .md per category** — asset table with one row per asset, detailed entries below for complex items
|
||||
- **palette.md** per pipeline — the style bible. Aesthetic baseline, theme, generation approach.
|
||||
- **Status values:** `planned` | `in-progress` | `placeholder` | `final`
|
||||
- **Templates:** `_templates/{type}.md` — row format + detailed entry format
|
||||
- **Asset files** live in `client/assets/{type}/` — these docs track design and production metadata
|
||||
- **Category splitting:** When a category file grows too long, split by domain (e.g., `furniture-station.md`, `furniture-colony.md`)
|
||||
|
||||
## Cross-References
|
||||
|
||||
- Decision files: `decisions/` (D-038 audio scope, D-033 entity colors, D-019 camera angle)
|
||||
- Art direction workshop: `docs/workshops/art-direction-mood-board/`
|
||||
- Sprint briefings: `docs/sprints/sprint-N/{team}.md`
|
||||
@@ -0,0 +1,33 @@
|
||||
# Audio Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.ogg` | {status} | {bus} | {method} | {duration} | {prompt/notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (AMB-001, SFX-001, UI-001)
|
||||
- **Filename:** Target filename in `client/assets/audio/`
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Bus:** Ambient / World SFX / Player Actions / UI Sounds / Music
|
||||
- **Method:** SAO (Stable Audio Open) / synth (manual synthesis) / hybrid
|
||||
- **Duration:** Target duration
|
||||
- **Prompt/Notes:** Generation prompt or design description (keep brief — expand in detailed entry if needed)
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
Use below the asset table for assets that need expanded documentation (complex generation, iteration history, special integration notes).
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Sonic family:** insert-tech / organic
|
||||
- **Frequency range:** {primary range}
|
||||
- **Full prompt:** `{complete generation prompt}`
|
||||
- **Post-processing:** {trim, normalize, EQ, crossfade, loop points}
|
||||
- **Integration:** {which system triggers this, bus routing, special behavior}
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,31 @@
|
||||
# Sprite Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.png` | {status} | {layer} | {method} | {directions} | {notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (SPR-001, FUR-001, TILE-001, ICO-001, FX-001)
|
||||
- **Filename:** Target filename in `client/assets/sprites/` (or tilesets/, etc.)
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Layer:** Entity / tile / effect / UI
|
||||
- **Method:** render pipeline / hand-drawn / AI-assisted
|
||||
- **Directions:** N/E/S/W (4-dir), S-only, or N/A
|
||||
- **Notes:** Source model, special render settings, design description
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Source model:** `{path to .glb/.blend}`
|
||||
- **Render settings:** Camera angle, lighting, outline width
|
||||
- **Color relationships:** D-033 mapping if entity
|
||||
- **Resolutions:** 1024px (working) / 256px (game) / 64px (minimap)
|
||||
- **Post-processing:** Outline pass, palette adjustment, special effects
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,30 @@
|
||||
# Video Asset Template
|
||||
|
||||
## Table Row Format
|
||||
|
||||
```
|
||||
| {ID} | `{filename}.{ext}` | {status} | {type} | {duration} | {notes} | {sprint/ticket} |
|
||||
```
|
||||
|
||||
**Columns:**
|
||||
- **ID:** Category prefix + sequential number (TRL-001, CUT-001)
|
||||
- **Filename:** Target filename
|
||||
- **Status:** planned / in-progress / placeholder / final
|
||||
- **Type:** trailer / cutscene / intro
|
||||
- **Duration:** Target duration
|
||||
- **Notes:** Purpose, audience, key scenes
|
||||
- **Sprint/Ticket:** Sprint number and ticket ID
|
||||
|
||||
## Detailed Entry Format
|
||||
|
||||
```markdown
|
||||
### {ID}: {filename}
|
||||
- **Resolution:** 1920x1080 / 3840x2160
|
||||
- **Frame rate:** 30 / 60 fps
|
||||
- **Audio track:** {link to audio asset or composition notes}
|
||||
- **Script/storyboard:** {link or inline}
|
||||
- **Production method:** In-engine capture / external editing / mixed
|
||||
- **Iteration history:**
|
||||
| Date | Version | Notes |
|
||||
|------|---------|-------|
|
||||
```
|
||||
@@ -0,0 +1,54 @@
|
||||
# Audio Asset Pipeline
|
||||
|
||||
Status: **Active** — Sprint 7 (first audio sprint)
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | File | Total | Planned | In-Progress | Placeholder | Final |
|
||||
|----------|------|-------|---------|-------------|-------------|-------|
|
||||
| Ambient | [ambient.md](ambient.md) | 4 | 4 | 0 | 0 | 0 |
|
||||
| SFX | [sfx.md](sfx.md) | 2 | 2 | 0 | 0 | 0 |
|
||||
| UI | [ui.md](ui.md) | 6 | 4 | 0 | 0 | 0 |
|
||||
| **Total** | | **12** | **10** | **0** | **0** | **0** |
|
||||
|
||||
Update counts when asset status changes.
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for the sonic identity: two sonic families (insert-tech / organic), station palette (functional warmth), generation approach, bus architecture.
|
||||
|
||||
## Generation Tools
|
||||
|
||||
- **Stable Audio Open:** Self-hosted, >200ms organic sounds. ~47s ceiling per generation.
|
||||
- **Manual synthesis:** <200ms insert-tech sounds. Audacity / Python oscillator.
|
||||
- **Post-processing:** LUFS normalization, EQ carving, crossfade loops (3-5s overlap).
|
||||
|
||||
## Bus Architecture
|
||||
|
||||
5 player-facing buses (see palette.md for full spec):
|
||||
|
||||
| Bus | Content |
|
||||
|-----|---------|
|
||||
| Music | Future — empty |
|
||||
| Ambient | Station hum, zone overlays |
|
||||
| World SFX | NPC footsteps, doors, conversation murmur |
|
||||
| Player Actions | Player footsteps, combat |
|
||||
| UI Sounds | Cursor, implant, chimes, recognition |
|
||||
|
||||
## Sprint Scope
|
||||
|
||||
### Sprint 7 (#440 — 8 assets)
|
||||
- 4 UI sounds: cursor hover, fog recognition, implant open, weapon aim (sketch)
|
||||
- 2 UI sounds: monologue chime normal + urgent (PLACEHOLDER — Sprint 8 redo)
|
||||
- 2 mix specs: dialogue dip, confrontation dip (documentation, not audio files)
|
||||
|
||||
### Sprint 8+ (D-038 — 6 remaining assets)
|
||||
- 4 ambient loops: station base, workplace layer, bar layer, corridor layer
|
||||
- 2 SFX: footstep metal walk, footstep metal run
|
||||
- Monologue chime redo (priority #1)
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-038: Audio in v0.1 scope
|
||||
- D-018: Three-range sound model
|
||||
- D-045: Environmental neutrality
|
||||
@@ -0,0 +1,43 @@
|
||||
# Ambient — Audio Assets
|
||||
|
||||
Zone-based ambient loops. Station base plays globally; overlays crossfade based on player zone (hard tile boundary, 1.5-2s audio tween).
|
||||
|
||||
## Generation Approach
|
||||
|
||||
Stable Audio Open primary. Prompt strategy: describe the SPACE, not the sound.
|
||||
- Target: 45s loops with 3-5s overlap crossfade (equal-power)
|
||||
- Validation: solo test → stack test → fatigue test (5+ minutes)
|
||||
- Post-processing: LUFS normalize, EQ carve per-layer to avoid masking, crossfade loop points
|
||||
- Environmental neutrality (D-045): warmth/character is baked into the asset, does not react to narrative state
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| AMB-001 | `amb_station_base.ogg` | planned | Ambient | SAO | 45-60s loop | Station hum, span gate vibration, ventilation. Low-freq foundation (60-120Hz), mid texture (200-800Hz). Always playing — acoustic foundation of being on a station. | S8+ |
|
||||
| AMB-002 | `amb_workplace_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Cargo machinery, scanner pings, distant procedural voices (designations, not conversation). Rhythmic mechanical cadence. Terminal zone overlay. | S8+ |
|
||||
| AMB-003 | `amb_bar_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Conversation murmur (300Hz-3kHz), softened glass sounds, faint Meridian music (barely melodic, distant radio). Social warmth. Murmur baked in — bar IS the crowd. Last Shift zone overlay. | S8+ |
|
||||
| AMB-004 | `amb_corridor_layer.ogg` | planned | Ambient | SAO | 45-60s loop | Echoing footsteps, ventilation whistle with reverb tail, louder gate hum (closer to structure). Emptier mid-range, prominent low-end. Liminal emptiness. Corridor zone overlay. | S8+ |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### AMB-001: amb_station_base
|
||||
- **Sonic family:** organic (environmental)
|
||||
- **Frequency range:** 60-120Hz foundation, 200-800Hz texture, 2-8kHz sparse detail
|
||||
- **Role:** Global ambient — always playing, never stops. The acoustic reality of Station Sova. If zone overlays fade, this is "silence" — should feel empty, not quiet.
|
||||
- **Layering:** Plays on AmbientBase sub-channel. Zone overlays play on AmbientLayer, additive.
|
||||
- **Integration:** Starts on game load, loops indefinitely. Volume on Ambient bus.
|
||||
- **Cross-reference:** D-038 (#1), D-039 wow moment #1 (Arrival — station hum is first thing player hears)
|
||||
|
||||
### AMB-003: amb_bar_layer
|
||||
- **Sonic family:** organic (social space)
|
||||
- **Murmur note:** Conversation murmur is baked into this asset, not triggered by NPC presence. The bar sounds like a crowd whether 2 or 15 NPCs are present. This is the acoustic character of the space. Separate NPC proximity murmur (event-driven, point-source) is a future asset.
|
||||
- **Integration:** Crossfades in when player enters bar zone, out when leaving. 1.5-2s tween.
|
||||
- **Cross-reference:** D-038 (#3), D-045 (warmth is indifference — bar sounds warm always)
|
||||
|
||||
## Future Expansion
|
||||
|
||||
As the game expands beyond Station Sova (multiple planets, biomes, weather), ambient categories will grow significantly. When this file exceeds ~50 entries, split by location:
|
||||
- `ambient-station.md` (Station Sova zones)
|
||||
- `ambient-planet-{name}.md` (planetary biomes)
|
||||
- `ambient-weather.md` (weather overlay loops)
|
||||
@@ -0,0 +1,93 @@
|
||||
# Audio Palette
|
||||
|
||||
The sonic identity of The Settled Reach. Living document — updated as the audio language evolves.
|
||||
|
||||
## Core Principle: Audio as Cognitive Architecture
|
||||
|
||||
Every sound maps to one of two cognitive sources:
|
||||
|
||||
| Source | Sound Type | What it means | Sonic character |
|
||||
|--------|-----------|---------------|-----------------|
|
||||
| Biological | Ambient layers, footsteps, fog recognition | "I am a body in a space, and my body knows things" | Organic, warm, natural decay |
|
||||
| Technological | Cursor hover, implant open, monologue chime | "I am augmented, and my lattice processes things my body can't" | Synthetic, precise, clinical |
|
||||
| Absence | Dialogue dip, confrontation dip | "I am choosing to focus, and the world continues without my attention" | Reduced ambient, muffled world |
|
||||
|
||||
This split maps to D-018's trust model: organic perception is slow but trustworthy, technological perception is fast but its data can be manipulated.
|
||||
|
||||
## Two Sonic Families
|
||||
|
||||
### Insert-Tech (Synthetic)
|
||||
- **Character:** Clean waveforms, no reverb (exists in your head, not in space). Digital. Precise. Mathematical.
|
||||
- **Frequency range:** 800Hz-4kHz primary. Sharp attack, controlled decay.
|
||||
- **Feel:** "A well-designed touchscreen" — tonal confirmation, not notification beeps.
|
||||
- **Used for:** Cursor hover, implant open, weapon aim, monologue chimes.
|
||||
- **Generation method:** Manual synthesis for <200ms sounds. SAO for >200ms.
|
||||
|
||||
### Organic (Biological)
|
||||
- **Character:** Warm, breathy, slightly resonant. Natural attack and decay. Reverberant (exists in the world).
|
||||
- **Frequency range:** 300Hz-1.5kHz primary. Soft attack, natural tail.
|
||||
- **Feel:** "The sound you 'hear' internally when something clicks into place."
|
||||
- **Used for:** Fog recognition, future character-cognition sounds.
|
||||
- **Generation method:** Stable Audio Open primary — organic texture is what generative audio does best.
|
||||
|
||||
## Station Sonic Palette: Functional Warmth
|
||||
|
||||
The Settled Reach is a working station. Not military, not luxury — **industrial infrastructure that people made livable.**
|
||||
|
||||
### Base Layer (always present)
|
||||
- **Low-frequency foundation (60-120Hz):** Generator hum, structural resonance. The span gate's mass holding the station together. Always present, barely noticed.
|
||||
- **Mid-frequency texture (200-800Hz):** Ventilation, distant pipe flow, air cycling systems. Life support you forget about until it stops.
|
||||
- **High-frequency detail (2-8kHz):** Occasional scanner pings, distant PA crackle, metal-on-metal contact. Sparse. These break through the baseline — something happened.
|
||||
|
||||
### Zone Overlays (additive to base)
|
||||
- **Bar (The Last Shift):** More mid-range (voice murmur 300Hz-3kHz), softer high end (glass filtered, not sharp), faint Meridian music barely melodic — someone left a radio on two rooms away. Social warmth.
|
||||
- **Corridor:** Emptier mid-range, more prominent low-end (gate hum louder — closer to structure), high-frequency echoes (reverb tail on ventilation whistle). Liminal emptiness.
|
||||
- **Workplace (Terminal):** Rhythmic mechanical elements (cargo cadence), cleaner scanner pings, distant procedural voices (cargo designations, not conversation). Institutional precision.
|
||||
|
||||
### What Silence Sounds Like
|
||||
Silence on a station isn't silence — it's the base hum with nothing on top. If a zone overlay fades out and you're left with just station base, that should feel **empty**. Not quiet. Empty. Like a room where people were just a moment ago.
|
||||
|
||||
## Environmental Neutrality (D-045)
|
||||
|
||||
The ambient palette **does not shift with narrative state**. The bar sounds warm whether THE FRIEND is honest or lying. The corridor hum doesn't become ominous when you discover the conspiracy. The world is indifferent to your investigation.
|
||||
|
||||
The warmth is the indifference. The bartender's radio doesn't know.
|
||||
|
||||
## Audio Bus Architecture
|
||||
|
||||
5 player-facing buses, each with independent volume slider:
|
||||
|
||||
| Bus | Content | Diegetic source |
|
||||
|-----|---------|-----------------|
|
||||
| Music | Future — empty for now | Non-diegetic |
|
||||
| Ambient | Station hum, zone overlays, global loops | Environment |
|
||||
| World SFX | NPC footsteps, doors, machinery, conversation murmur | Other entities |
|
||||
| Player Actions | Player footsteps, combat, weapon sounds | Player character |
|
||||
| UI Sounds | Cursor, implant, monologue chimes, fog recognition | Neural lattice / cognition |
|
||||
|
||||
## Generation Approach
|
||||
|
||||
### Stable Audio Open (>200ms, organic character)
|
||||
- Model: ~1.2B parameters, 44.1kHz stereo output, ~47s ceiling
|
||||
- Prompting: Describe the SPACE, not the sound. "Industrial space station interior, low frequency machinery hum" not "sci-fi ambient drone in D minor."
|
||||
- Iteration: Generate 4-6 candidates per asset. Test: solo → stack → fatigue (5+ minutes for loops).
|
||||
- Post-processing: Trim, normalize (LUFS), EQ to carve frequency space, crossfade loops (3-5s overlap, equal-power).
|
||||
|
||||
### Manual Synthesis (<200ms, insert-tech character)
|
||||
- Tools: Audacity / Python synthesis (sine tone + envelope + harmonic layering)
|
||||
- The precision IS the design — insert-tech sounds should feel mathematically exact.
|
||||
- A 50ms cursor tick is a few lines of oscillator + exponential decay.
|
||||
|
||||
### Quality Validation
|
||||
- **Solo test:** Does it sound right alone?
|
||||
- **Stack test:** Does it sit in the mix with other layers without masking?
|
||||
- **Fatigue test:** Can you listen for 5+ minutes without jarring repeat? (loops only)
|
||||
- **Close-your-eyes test (Ozzie):** Does the sound create a mental image / sensation?
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-018: Three-range sound model (close/medium/long)
|
||||
- D-038: Audio in v0.1 scope — 8 files via Stable Audio Open
|
||||
- D-045: Environmental neutrality
|
||||
- D-048: Neural insert overlay (insert-styled UI aesthetic)
|
||||
- D-053: Movement stances (acoustic footprint implications)
|
||||
@@ -0,0 +1,41 @@
|
||||
# SFX — Audio Assets
|
||||
|
||||
World sound effects: footsteps, environmental events, impacts. Event-driven, often positional (AudioStreamPlayer2D).
|
||||
|
||||
## Generation Approach
|
||||
|
||||
Mixed methods depending on duration and character:
|
||||
- Footsteps: SAO generation (>200ms) or hybrid (generate longer clip, extract best transient)
|
||||
- Future environmental SFX (doors, machinery events): SAO or foley recording
|
||||
- Post-processing: normalize, trim, ensure clean attack/release
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| SFX-001 | `sfx_footstep_metal.ogg` | planned | Player Actions | hybrid | 0.3-0.5s | Single footstep on metal grating. Walk pace. Player character proprioceptive feedback. | S8+ |
|
||||
| SFX-002 | `sfx_footstep_metal_run.ogg` | planned | Player Actions | hybrid | 0.2-0.3s | Faster footstep, sprint pace. Louder — player is broadcasting position (D-053 acoustic footprint). | S8+ |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### SFX-001: sfx_footstep_metal
|
||||
- **Sonic family:** organic (physical world)
|
||||
- **Bus rationale:** Player Actions — this is feedback on YOUR movement, not world information. NPC footsteps (future) go on World SFX.
|
||||
- **Stance implications (D-053):** Walk = normal volume. Sprint variant (SFX-002) = louder. Careful = quieter variant (future asset). Crouch = near-silent variant (future asset).
|
||||
- **Integration:** Triggered by player movement tick. Positional (AudioStreamPlayer2D at player position) — maps to camera-relative stereo per D-018.
|
||||
- **Cross-reference:** D-038 (#5), D-053 (movement stances as acoustic decisions)
|
||||
|
||||
## Future Expansion
|
||||
|
||||
SFX will grow significantly with:
|
||||
- NPC footstep variants (by surface type, stance) → World SFX bus
|
||||
- Door open/close, machinery activation → World SFX bus
|
||||
- Combat impacts, weapon sounds → Player Actions bus
|
||||
- Environmental events per planet/biome
|
||||
- NPC conversation murmur (event-driven, point-source) → World SFX bus
|
||||
|
||||
When this file exceeds ~50 entries, split by domain:
|
||||
- `sfx-movement.md` (footsteps, stance variants)
|
||||
- `sfx-environment.md` (doors, machinery, weather)
|
||||
- `sfx-combat.md` (weapons, impacts)
|
||||
- `sfx-conversation.md` (murmur, vocal events)
|
||||
@@ -0,0 +1,73 @@
|
||||
# UI — Audio Assets
|
||||
|
||||
Interface sounds triggered by player interaction, insert systems, and cognitive events. Non-positional (AudioStreamPlayer). Routes to UI Sounds bus.
|
||||
|
||||
## Generation Approach
|
||||
|
||||
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`.
|
||||
- **Monologue chimes:** Currently PLACEHOLDER (Sprint 7). Redo scheduled next audio sprint (#453). Design brief below captures target register.
|
||||
|
||||
## Assets
|
||||
|
||||
| ID | Filename | Status | Bus | Method | Duration | Prompt/Notes | Sprint |
|
||||
|----|----------|--------|-----|--------|----------|-------------|--------|
|
||||
| UI-001 | `cursor_hover.ogg` | draft | UI Sounds | SAO | 50-100ms | Subtle tick/ping on entity hover (D-056). Insert-tech: clean, precise, synthetic. Almost subliminal — notice if missing, not if present. Needs trim to target duration (#453). | S7 #440 |
|
||||
| UI-002 | `implant_open.ogg` | draft | UI Sounds | SAO | 150-250ms | Rising tone when insert UI opens (radial menu, inventory, stance). Neural lattice powering up. Frequency sweep, harmonic shimmer. Needs trim (#453). | S7 #440 |
|
||||
| UI-003 | `fog_recognition.ogg` | draft | UI Sounds | SAO | 300-400ms | Warm organic chime at onset of cognitive delay (D-060). Character recognizing a person. Unresolved tone. Needs trim (#453). | S7 #440 |
|
||||
| UI-004 | `weapon_aim.ogg` | draft | UI Sounds | SAO | 100-150ms | Harder click/lock for weapon aim state. Mechanical, deliberate. SKETCH ONLY — no weapons in Sprint 7. Needs trim (#453). | S7 #440 |
|
||||
| UI-005 | `sfx_monologue_chime.ogg` | draft | UI Sounds | SAO | 500-1000ms | PLACEHOLDER. Soft crystalline tone, neural lattice surfacing a thought. Quiet, non-intrusive. Redo in next audio sprint (#453). See design brief below. | S7 #440 |
|
||||
| UI-006 | `sfx_monologue_chime_urgent.ogg` | draft | UI Sounds | SAO | 500-1000ms | PLACEHOLDER. Sharper variant for contradiction/anomaly. Brighter, more insistent. Redo in next audio sprint (#453). See design brief below. | S7 #440 |
|
||||
|
||||
## Mix Specs (not audio files)
|
||||
|
||||
These are AudioBus parameter changes, documented in `docs/audio/dialogue-ambient-dip.md`:
|
||||
|
||||
| Spec | Trigger | Effect | Sprint |
|
||||
|------|---------|--------|--------|
|
||||
| Dialogue dip | Dialogue box opens | Ambient -6 to -8dB, 300ms ease-in, 500ms ease-out | S7 #440 |
|
||||
| Confrontation dip | Confrontation dialogue | Ambient -10 to -12dB + LP (800Hz), World SFX -4 to -6dB, 500ms ease-in, 1000ms ease-out | S7 #440 |
|
||||
|
||||
## Detailed Entries
|
||||
|
||||
### UI-001: cursor_hover
|
||||
- **Sonic family:** insert-tech
|
||||
- **Frequency range:** 3-4kHz primary
|
||||
- **Design principle:** "Like breathing — present, functional, invisible." Player shouldn't describe it if asked, but would notice absence. Fatigue test critical — plays 300+ times per session.
|
||||
- **Generation:** Pure synthesis. Sine tone at 3.2kHz, exponential decay over 40-50ms. No SAO — precision and purity are the point.
|
||||
- **Debouncing:** AudioManager enforces 100ms cooldown between plays (rapid mouse movement).
|
||||
- **Integration:** CursorRenderer.cursor_state_changed signal → AudioManager.play_ui("cursor_hover")
|
||||
- **Cross-reference:** D-056 (cursor states)
|
||||
|
||||
### UI-003: fog_recognition
|
||||
- **Sonic family:** organic
|
||||
- **Frequency range:** 300Hz-1.5kHz
|
||||
- **Design principle:** "The sound you hear internally when something clicks into place." Not a chime — more like a soft exhalation of tone. Sits BETWEEN insert sounds and ambient — not UI, not world, cognition.
|
||||
- **Timing (Q-014 resolved):** Fires at ONSET of cognitive delay. Sequence: hear something in fog → chime plays → 0.6s delay → monologue during delay → blob transitions to D-033 color. Chime is "your character is processing," color transition is "recognition complete."
|
||||
- **Integration:** Cognitive delay onset event → AudioManager.play_ui("fog_recognition")
|
||||
- **Cross-reference:** D-060 (cognitive delay), D-018 (three-range sound, medium range)
|
||||
|
||||
### UI-005 / UI-006: Monologue Chime Design Brief (PLACEHOLDER)
|
||||
|
||||
**Target emotional register for Sprint 8 redo:**
|
||||
|
||||
The monologue chimes are the two poles of augmented cognition made audible:
|
||||
|
||||
| Variant | Meaning | Feel | Target |
|
||||
|---------|---------|------|--------|
|
||||
| Normal | "My lattice surfaced this" | A thought arriving — not a notification. Like your own attention shifting. | Player never consciously notices it. Pavlovian: text appeared, brain registered, moved on. |
|
||||
| Urgent | "My lattice flagged a contradiction" | Productive discomfort. "Wait, what did I miss?" | Micro-spike of alertness. Not alarm. The delta from normal must be SMALL but unmistakable. |
|
||||
|
||||
**Relationship to fog recognition chime:**
|
||||
- Fog recognition = organic (biological cognition, warm, 300-800Hz)
|
||||
- Monologue normal = insert-tech (crystalline, clean, higher register, ~1.2kHz)
|
||||
- Monologue urgent = insert-tech (sharper, brighter, adds second harmonic ~2.4kHz)
|
||||
- Same cognitive family, different branches. Cousins, not twins.
|
||||
|
||||
**Quality bar:** If the urgent chime feels like a quest marker ping, it has failed. If it feels like Ubisoft, it has failed. The delta between normal and urgent = "someone thinking quietly" vs "someone's eyes going slightly wide."
|
||||
|
||||
**Placeholder spec (Sprint 7):**
|
||||
- Normal: pure sine 1.2kHz, 600ms, gentle attack, natural decay. Conservative volume.
|
||||
- Urgent: sine 1.2kHz + harmonic 2.4kHz, 400ms, sharper attack, 15% louder.
|
||||
- Deliberately undersized — obviously temporary. Ugly placeholders get replaced.
|
||||
|
||||
**Evaluation gate:** First internal playtest after Sprint 7: "Does the normal chime feel like a thought arriving? Does the urgent chime create a physiological response?" If no → trigger chime redesign.
|
||||
@@ -0,0 +1,18 @@
|
||||
# Video Asset Pipeline
|
||||
|
||||
Status: **Stub** — no video assets planned for v0.1.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | File | Count | Description |
|
||||
|----------|------|-------|-------------|
|
||||
| Trailers | [trailers.md](trailers.md) | 0 | Marketing / Steam page trailers |
|
||||
| Cutscenes | [cutscenes.md](cutscenes.md) | 0 | In-game narrative moments (D-019: deferred to milestones) |
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for motion/editing style guide (to be established).
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-019: 3D cutscenes for key moments — deferred to milestones, not v0.1
|
||||
@@ -0,0 +1,5 @@
|
||||
# Video Palette
|
||||
|
||||
Status: **Stub** — no video production planned for v0.1.
|
||||
|
||||
To be established when cutscene or trailer work begins.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Visual Asset Pipeline
|
||||
|
||||
Status: **Stub** — awaiting first visual sprint.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | File | Count | Description |
|
||||
|----------|------|-------|-------------|
|
||||
| Sprites | [sprites.md](sprites.md) | 0 | Entity sprites via 3D render pipeline |
|
||||
| Furniture | [furniture.md](furniture.md) | 0 | Furniture items by location/type |
|
||||
| Tilesets | [tilesets.md](tilesets.md) | 0 | Floor/wall tiles by biome/planet |
|
||||
| Icons | [icons.md](icons.md) | 0 | UI icons, status indicators |
|
||||
| Effects | [effects.md](effects.md) | 0 | Fog shaders, particles, overlays |
|
||||
|
||||
## Palette
|
||||
|
||||
See [palette.md](palette.md) for art direction, "the angle", color relationships, and render pipeline specs.
|
||||
|
||||
## Pipeline
|
||||
|
||||
- **3D render pipeline:** Blender model → Godot Camera3D at -72.5deg → orthographic render → outline pass → resolution downscale
|
||||
- **Render skill:** `/render-sprite` produces 12 PNGs (4 directions x 3 resolutions)
|
||||
- **Art direction workshop:** `docs/workshops/art-direction-mood-board/`
|
||||
|
||||
## Decision References
|
||||
|
||||
- D-019: Top-down camera, 15-20deg from vertical ("the angle")
|
||||
- D-033: Entity color = relationship to player
|
||||
- D-045: Environmental neutrality
|
||||
@@ -0,0 +1,16 @@
|
||||
# Visual Palette
|
||||
|
||||
Status: **Stub** — reference art direction workshop output for current guidance.
|
||||
|
||||
See `docs/workshops/art-direction-mood-board/` for the established visual identity.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Camera angle:** 15-20deg from vertical ("the angle"), rendered via orthographic Camera3D at -72.5deg from horizontal (D-019)
|
||||
- **Entity colors:** Relationship-based per D-033 (green = known/friendly, amber = neutral, red = hostile, etc.)
|
||||
- **Environmental neutrality:** Spaces don't visually shift with narrative state (D-045)
|
||||
- **Functional warmth:** Industrial infrastructure that people made livable — not military, not luxury
|
||||
|
||||
## Style Guide
|
||||
|
||||
To be populated from art direction workshop synthesis and first visual sprint.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Dialogue & Confrontation Ambient Dip — Implementation Spec
|
||||
|
||||
Audio bus volume/filter changes for dialogue and confrontation states. These are NOT audio files — they're mix parameter changes applied to AudioBus volumes and effects in Godot's AudioServer.
|
||||
|
||||
**Ticket:** #440
|
||||
**Decisions:** D-068 (5-bus architecture), D-069 (dip profiles), D-070 (confrontation as cognitive vulnerability)
|
||||
**Branch:** client (AudioManager implementation), audio (this spec)
|
||||
|
||||
## Bus Architecture Reference
|
||||
|
||||
| Bus Index | Name | Purpose |
|
||||
|-----------|------|---------|
|
||||
| 0 | Master | Final mix output |
|
||||
| 1 | Music | Score (empty for now) |
|
||||
| 2 | Ambient | amb_* loops, environmental background |
|
||||
| 3 | World SFX | Positional sounds in physical space |
|
||||
| 4 | Player Actions | Combat, footsteps, interaction SFX |
|
||||
| 5 | UI Sounds | Non-positional interface feedback |
|
||||
|
||||
## Dialogue Dip
|
||||
|
||||
**Trigger:** Dialogue box opens (client #434)
|
||||
**Release:** Dialogue box closes
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Ambient bus volume | -6 to -8 dB (from current) |
|
||||
| Ease-in duration | 300ms |
|
||||
| Ease-out duration | 500ms |
|
||||
| Easing curve | Cubic ease-in-out |
|
||||
| Affected buses | Ambient only |
|
||||
|
||||
### Godot Implementation
|
||||
|
||||
```gdscript
|
||||
# In AudioManager (autoload singleton)
|
||||
|
||||
const DIALOGUE_DIP_DB := -7.0 # midpoint of -6 to -8 range
|
||||
const DIALOGUE_DIP_IN_MS := 300.0
|
||||
const DIALOGUE_DIP_OUT_MS := 500.0
|
||||
|
||||
var _ambient_bus_idx: int
|
||||
var _ambient_base_volume_db: float
|
||||
var _dip_tween: Tween
|
||||
|
||||
func _ready() -> void:
|
||||
_ambient_bus_idx = AudioServer.get_bus_index("Ambient")
|
||||
_ambient_base_volume_db = AudioServer.get_bus_volume_db(_ambient_bus_idx)
|
||||
|
||||
func dialogue_dip_start() -> void:
|
||||
_cancel_dip_tween()
|
||||
var target := _ambient_base_volume_db + DIALOGUE_DIP_DB
|
||||
_dip_tween = create_tween()
|
||||
_dip_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
target,
|
||||
DIALOGUE_DIP_IN_MS / 1000.0
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func dialogue_dip_end() -> void:
|
||||
_cancel_dip_tween()
|
||||
_dip_tween = create_tween()
|
||||
_dip_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
_ambient_base_volume_db,
|
||||
DIALOGUE_DIP_OUT_MS / 1000.0
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func _set_ambient_volume(db: float) -> void:
|
||||
AudioServer.set_bus_volume_db(_ambient_bus_idx, db)
|
||||
|
||||
func _cancel_dip_tween() -> void:
|
||||
if _dip_tween and _dip_tween.is_valid():
|
||||
_dip_tween.kill()
|
||||
```
|
||||
|
||||
### Signal Wiring
|
||||
|
||||
```gdscript
|
||||
# In DialogueBox or wherever dialogue state is managed:
|
||||
func _open_dialogue() -> void:
|
||||
# ... show dialogue UI ...
|
||||
AudioManager.dialogue_dip_start()
|
||||
|
||||
func _close_dialogue() -> void:
|
||||
# ... hide dialogue UI ...
|
||||
AudioManager.dialogue_dip_end()
|
||||
```
|
||||
|
||||
## Confrontation Dip
|
||||
|
||||
**Trigger:** Confrontation dialogue begins (client #434, confrontation variant)
|
||||
**Release:** Confrontation dialogue ends
|
||||
**Design intent (D-070):** "Felt, not computed." The player's focus narrows — the world acoustically recedes. This is cognitive vulnerability made audible.
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| Ambient bus volume | -11 dB (midpoint of -10 to -12) |
|
||||
| Ambient bus low-pass filter | 800 Hz cutoff, 6 dB resonance |
|
||||
| World SFX bus volume | -5 dB (midpoint of -4 to -6) |
|
||||
| Ease-in duration | 500ms |
|
||||
| Ease-out duration | 1000ms |
|
||||
| Easing curve | Cubic ease-in-out |
|
||||
| Affected buses | Ambient, World SFX |
|
||||
|
||||
### Godot Implementation
|
||||
|
||||
The confrontation dip adds a low-pass filter effect to the Ambient bus. This must be set up in the Godot AudioBus layout (Project → Audio Bus Layout):
|
||||
|
||||
**Bus setup (audio bus layout .tres):**
|
||||
1. Add `AudioEffectLowPassFilter` to the Ambient bus
|
||||
2. Set it to **bypassed by default** (effect is inactive until confrontation)
|
||||
3. Default cutoff: 20500 Hz (fully open)
|
||||
|
||||
```gdscript
|
||||
const CONFRONTATION_AMBIENT_DIP_DB := -11.0
|
||||
const CONFRONTATION_SFX_DIP_DB := -5.0
|
||||
const CONFRONTATION_LP_CUTOFF_HZ := 800.0
|
||||
const CONFRONTATION_LP_OPEN_HZ := 20500.0
|
||||
const CONFRONTATION_DIP_IN_MS := 500.0
|
||||
const CONFRONTATION_DIP_OUT_MS := 1000.0
|
||||
|
||||
var _world_sfx_bus_idx: int
|
||||
var _world_sfx_base_volume_db: float
|
||||
var _ambient_lp_effect_idx: int # index of the LP filter on Ambient bus
|
||||
var _confrontation_tween: Tween
|
||||
|
||||
func _ready() -> void:
|
||||
# ... (ambient bus setup from dialogue dip above) ...
|
||||
_world_sfx_bus_idx = AudioServer.get_bus_index("World SFX")
|
||||
_world_sfx_base_volume_db = AudioServer.get_bus_volume_db(_world_sfx_bus_idx)
|
||||
# Find the LP filter effect index on the Ambient bus
|
||||
for i in range(AudioServer.get_bus_effect_count(_ambient_bus_idx)):
|
||||
if AudioServer.get_bus_effect(_ambient_bus_idx, i) is AudioEffectLowPassFilter:
|
||||
_ambient_lp_effect_idx = i
|
||||
break
|
||||
|
||||
func confrontation_dip_start() -> void:
|
||||
_cancel_confrontation_tween()
|
||||
|
||||
# Enable the LP filter
|
||||
AudioServer.set_bus_effect_enabled(_ambient_bus_idx, _ambient_lp_effect_idx, true)
|
||||
|
||||
var amb_target := _ambient_base_volume_db + CONFRONTATION_AMBIENT_DIP_DB
|
||||
var sfx_target := _world_sfx_base_volume_db + CONFRONTATION_SFX_DIP_DB
|
||||
var dur := CONFRONTATION_DIP_IN_MS / 1000.0
|
||||
|
||||
_confrontation_tween = create_tween()
|
||||
_confrontation_tween.set_parallel(true)
|
||||
|
||||
# Ambient volume dip
|
||||
_confrontation_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
amb_target, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# World SFX volume dip
|
||||
_confrontation_tween.tween_method(
|
||||
_set_world_sfx_volume,
|
||||
AudioServer.get_bus_volume_db(_world_sfx_bus_idx),
|
||||
sfx_target, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Low-pass filter sweep
|
||||
var lp_effect: AudioEffectLowPassFilter = AudioServer.get_bus_effect(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx
|
||||
)
|
||||
_confrontation_tween.tween_property(
|
||||
lp_effect, "cutoff_hz",
|
||||
CONFRONTATION_LP_CUTOFF_HZ, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
func confrontation_dip_end() -> void:
|
||||
_cancel_confrontation_tween()
|
||||
var dur := CONFRONTATION_DIP_OUT_MS / 1000.0
|
||||
|
||||
_confrontation_tween = create_tween()
|
||||
_confrontation_tween.set_parallel(true)
|
||||
|
||||
# Restore ambient volume
|
||||
_confrontation_tween.tween_method(
|
||||
_set_ambient_volume,
|
||||
AudioServer.get_bus_volume_db(_ambient_bus_idx),
|
||||
_ambient_base_volume_db, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Restore world SFX volume
|
||||
_confrontation_tween.tween_method(
|
||||
_set_world_sfx_volume,
|
||||
AudioServer.get_bus_volume_db(_world_sfx_bus_idx),
|
||||
_world_sfx_base_volume_db, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Open LP filter back up
|
||||
var lp_effect: AudioEffectLowPassFilter = AudioServer.get_bus_effect(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx
|
||||
)
|
||||
_confrontation_tween.tween_property(
|
||||
lp_effect, "cutoff_hz",
|
||||
CONFRONTATION_LP_OPEN_HZ, dur
|
||||
).set_ease(Tween.EASE_IN_OUT).set_trans(Tween.TRANS_CUBIC)
|
||||
|
||||
# Disable LP filter after tween completes
|
||||
_confrontation_tween.chain().tween_callback(func():
|
||||
AudioServer.set_bus_effect_enabled(
|
||||
_ambient_bus_idx, _ambient_lp_effect_idx, false
|
||||
)
|
||||
)
|
||||
|
||||
func _set_world_sfx_volume(db: float) -> void:
|
||||
AudioServer.set_bus_volume_db(_world_sfx_bus_idx, db)
|
||||
|
||||
func _cancel_confrontation_tween() -> void:
|
||||
if _confrontation_tween and _confrontation_tween.is_valid():
|
||||
_confrontation_tween.kill()
|
||||
```
|
||||
|
||||
## Edge Cases
|
||||
|
||||
### Confrontation during dialogue
|
||||
Confrontation dip supersedes dialogue dip (it's deeper). If dialogue is active when confrontation starts, skip directly to confrontation levels. When confrontation ends, restore to dialogue dip levels (not base), then to base when dialogue ends.
|
||||
|
||||
### Rapid open/close
|
||||
The tween-kill-and-restart pattern handles this — a new dip start/end always kills the current tween and starts from the current actual volume, preventing jarring jumps.
|
||||
|
||||
### Player volume slider interaction
|
||||
Dips are RELATIVE to `_ambient_base_volume_db`. If the player adjusts their Ambient slider mid-dip, update `_ambient_base_volume_db` and recalculate the target. The `AudioManager` settings save/load system should call a method to refresh base volumes.
|
||||
|
||||
## ListeningFocus Boost (D-069)
|
||||
|
||||
When the player is in active listening mode (future sprint), World SFX gets a +2-3 dB boost instead of a dip. This is the inverse of confrontation — the character is paying MORE attention to the environment.
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| World SFX bus volume | +2.5 dB (midpoint of +2 to +3) |
|
||||
| Ease-in | 200ms |
|
||||
| Ease-out | 300ms |
|
||||
|
||||
Implementation follows the same tween pattern. Mutually exclusive with confrontation dip.
|
||||
Binary file not shown.
@@ -0,0 +1,65 @@
|
||||
# Sprint 8: React — Audio Tasks
|
||||
|
||||
**Goal:** Activate the dialogue pipeline end-to-end — player talks to NPCs, NPCs respond, the world reacts.
|
||||
|
||||
**Branch:** `audio`
|
||||
**Agents:** Inigo (sound design)
|
||||
|
||||
## Carry-over from Sprint 7
|
||||
|
||||
| # | Title | Status | Notes |
|
||||
|---|-------|--------|-------|
|
||||
| #440 | 6 interaction audio assets — cursor hover, weapon aim, etc. | in_progress | 4/6 assets delivered, 2 remaining |
|
||||
|
||||
## New Tickets
|
||||
|
||||
None for Sprint 8. Focus on completing carry-over #440.
|
||||
|
||||
Use `db/connectors/ticket show <id>` for full details.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- `decisions/scope.md` — D-038 (audio in v0.1 scope: 8 files via Stable Audio Open)
|
||||
- `decisions/perception.md` — D-018 (three-range sound model)
|
||||
- `decisions/content.md` — D-063 (confrontation weight via audio dip)
|
||||
|
||||
## Notes
|
||||
|
||||
### #440: 6 interaction audio assets (MEDIUM, carry-over)
|
||||
**What exists:**
|
||||
- 4/6 assets delivered in Sprint 7:
|
||||
- `sfx_cursor_hover.ogg` — soft tick on entity hover
|
||||
- `sfx_cursor_entity_lock.ogg` — short pulse when locking to entity
|
||||
- `sfx_interaction_confirm.ogg` — gentle chime on [E] press
|
||||
- `sfx_weapon_aim_enter.ogg` — tense metallic slide when entering aim mode
|
||||
|
||||
**What the ticket needs to deliver (2 remaining assets):**
|
||||
- `sfx_weapon_aim_lock.ogg` — sharper variant of cursor_entity_lock for weapon targeting. Distinct from general entity lock — should feel more clinical/dangerous. 0.3-0.5s. Weapon-selected + entity in sights = this sound plays.
|
||||
- `sfx_stance_change.ogg` — soft mechanical click for stance toggle (Walk ↔ Sprint ↔ Careful ↔ Crouch). Subtle, not intrusive. Neural insert feedback aesthetic (D-048). 0.2-0.4s. Plays on every stance change (Shift/Ctrl toggle).
|
||||
|
||||
**Integration:**
|
||||
- Both assets use Stable Audio Open (per D-038: self-hosted, free commercial use)
|
||||
- Client integration: audio events emitted by cursor_state_machine.gd (weapon_aim_lock) and stance_indicator.gd (stance_change)
|
||||
- Audio autoload (SoundManager) handles playback
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- weapon_aim_lock should NOT be loud or startling — it's UI feedback, not a combat sound. Think "confirmation beep" not "alarm."
|
||||
- stance_change fires frequently (player may toggle stances multiple times while positioning). Keep it short and pleasant — no long tails or reverb.
|
||||
- Both sounds are UI/insert layer (z-layer 7), not diegetic world sounds. They don't spatialize or attenuate with distance.
|
||||
|
||||
**Sprint goal:**
|
||||
Deliver final 2 assets. Audio team unblocked after this — no new audio tickets planned for Sprint 9 unless design requests.
|
||||
|
||||
## Dependency Chain
|
||||
|
||||
```
|
||||
#440 (interaction audio) → standalone, completes Sprint 7 audio scope
|
||||
```
|
||||
|
||||
## PR Workflow
|
||||
|
||||
When ready to submit, create a PR with `tea` CLI. **All flags are required** to avoid TTY prompts (see CLAUDE.md "Gitea access" section):
|
||||
|
||||
```bash
|
||||
tea pr create --repo jpmschweitzer/settled-reach --login schweitz --title "feat(audio): interaction audio completion (Sprint 8)" --description "Delivers final 2/6 interaction audio assets (#440) — weapon aim lock, stance change" --base main --head audio
|
||||
```
|
||||
@@ -0,0 +1,145 @@
|
||||
# Sprint 8: React — Client Tasks
|
||||
|
||||
**Goal:** Activate the dialogue pipeline end-to-end — player talks to NPCs, NPCs respond, the world reacts.
|
||||
|
||||
**Branch:** `client`
|
||||
**Agents:** Stig (dev), Tyre (arch), Hoshe (QA)
|
||||
|
||||
## Carry-over from Sprint 7
|
||||
|
||||
| # | Title | Status | Notes |
|
||||
|---|-------|--------|-------|
|
||||
| #431 | Fog entity visualization — sound pings, cognitive delay anim | in_progress | Protocol v7 bump + visualization implementation |
|
||||
|
||||
## New Tickets
|
||||
|
||||
| # | Title | Blocked by |
|
||||
|---|-------|------------|
|
||||
| #435 | Dialogue response selection — max 3 visible, invisible locks | #434 (done) |
|
||||
| #437 | Walk-away mechanic — WASD detect during dialogue, 300ms fade | — |
|
||||
| #436 | Confrontation text styling — italic voice, monologue beat | #435 |
|
||||
|
||||
Use `db/connectors/ticket show <id>` for full details.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- `decisions/content.md` — D-062 (invisible locked dialogue options), D-063 (confrontation same box different weight), D-064 (walk-away 3-phase consequences)
|
||||
- `decisions/perception.md` — D-059 (fog shader 5 layers), D-060 (cognitive delay), D-061 (dialogue box max 20% height)
|
||||
- `decisions/architecture.md` — D-066 (dual-scale grid 0.5m sim / 1m visual)
|
||||
|
||||
## Open Questions to Resolve Early
|
||||
|
||||
None.
|
||||
|
||||
## Notes
|
||||
|
||||
### #431: Fog entity visualization — sound pings, cognitive delay anim (HIGH, carry-over)
|
||||
**What exists:**
|
||||
- `client/scripts/rendering/fog_shader.gd` — FogShader node with 5-layer shader (D-059), noise animation
|
||||
- `client/scripts/protocol/protocol.gd` — MessagePack decoder for ObserverSnapshot (protocol v6)
|
||||
- Fog layers 1-3 (clear, light fog, deep fog) render via shader, no entity visualization yet
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Protocol v7 bump:** Add `pending_recognitions` and `sound_events` fields to ObserverSnapshot in protocol.gd decode logic
|
||||
- **Sound ping rendering:** On FogState layer 6 (insert overlay, NOT affected by fog shader), render 2-3 thin concentric expanding rings from sound source direction. Sonar-style. Insert white-blue color. Loud = 3 rings bright fast. Quiet = 1 ring faint slow. Fade over 1.5s. Use simple Sprite2D or Line2D, not particles.
|
||||
- **Recognized entity in fog:** Entity with `pending: false` renders on layer 6 as: D-033 color glow (via PointLight2D ~40% blend) + faint silhouette feature (cached sprite at ~20% opacity) + 0.8s breathing pulse (shader uniform or tween) + ±0.5 tile position drift (approx position, not exact). Silhouette feature: Kael's vest, Lera's apron (sprite metadata from entity.silhouette_feature field).
|
||||
- **Unrecognized entity in fog:** Entity with `pending: true` renders as neutral grey #555566 blob (simple circle sprite, no identifying features). No silhouette. No D-033 color.
|
||||
- **Recognition transition:** When `pending` flips from true → false, tween grey blob → color+silhouette over 0.3s (within the 0.6s cognitive delay window per D-060). Monologue fires during this tween.
|
||||
|
||||
**Integration points:**
|
||||
- Pairs with server #423 (cognitive delay system, done Sprint 7)
|
||||
- Protocol v7 wire format must match server ObserverSnapshot struct additions
|
||||
- Builds on fog shader from Sprint 6
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Sound pings and fog entities render on layer 6 (insert overlay), NOT layer 5 (fog shader). Insert data is computational, not perceptual — not affected by vision cone.
|
||||
- ±0.5 tile drift is VISUAL only (client-side randomness for "approximate" feel). Server sends exact position; client adds noise.
|
||||
- Silhouette feature sprite is entity metadata, not a full sprite. Think "identifying accessory" not "full character render."
|
||||
- Tween recognition over 0.3s to align with monologue timing (D-060: monologue fires DURING delay, not after)
|
||||
|
||||
---
|
||||
|
||||
### #435: Dialogue response selection — max 3 visible, invisible locks (HIGH)
|
||||
**What exists:**
|
||||
- `client/ui/interaction_list.gd` — Vertical list for entity interaction verbs (Talk, Observe, etc.)
|
||||
- `client/scripts/protocol/protocol.gd` — Decodes NearbyInteraction from snapshot
|
||||
- D-061 dialogue box spec: bottom screen, max 20% height, max-width, no portraits
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **New scene:** `dialogue_box.tscn` — Bottom-screen container with two sections: NPC speech (Label, top), player response options (VBoxContainer, bottom). Max 20% viewport height, max-width (exact value: viewport width minus 40px margin). Left-aligned text. Insert-styled per D-057/D-061.
|
||||
- **Response option rendering:** Each response is a Button (or custom Control with hover state). Show max 3 visible options. Sort by priority (server provides sorted list). D-062: locked options are invisible (not rendered at all). No grayed-out options. No lock icons. Player doesn't know what they don't know.
|
||||
- **New protocol field:** `dialogue_response` in ObserverSnapshot — array of {text: String, response_id: String, priority: int}. Decode in protocol.gd.
|
||||
- **Integration:** When player selects Talk verb from interaction_list, server sends snapshot with dialogue_response populated. Client shows dialogue_box, hides interaction_list. Player clicks response → send PlayerInput::DialogueResponse{response_id}.
|
||||
- **Monologue interaction:** Monologue floats ABOVE dialogue box on z-layer 7 (spatial separation, D-061). Both can be visible simultaneously — character thinks one thing while NPC says another.
|
||||
|
||||
**Integration points:**
|
||||
- Blocked by #434 (interaction verb execution, done Sprint 7)
|
||||
- Pairs with server #305 (dialogue selection pipeline)
|
||||
- Blocks #436 (confrontation styling)
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Max 3 VISIBLE means if server sends 5 options, show top 3 by priority. But D-062 invisible locks mean server should never send locked options at all (server filters before protocol). Client just renders what it receives.
|
||||
- Player response text can be variable length (confrontation lines are longer than casual small talk). Use word wrap, auto-expand button height.
|
||||
- No close button per D-064 — walk-away is the exit. Dialogue box stays until player walks away or server sends empty dialogue_response (conversation ended).
|
||||
|
||||
---
|
||||
|
||||
### #437: Walk-away mechanic — WASD detect during dialogue, 300ms fade (MEDIUM)
|
||||
**What exists:**
|
||||
- Input handling in `client/scripts/main.gd` captures WASD for movement, sends PlayerInput::Move
|
||||
- Dialogue box (#435) renders dialogue but doesn't handle walk-away yet
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Input override:** When dialogue_box is visible, WASD input triggers walk-away instead of movement. Send PlayerInput::WalkAway{npc_id} to server.
|
||||
- **Visual fade:** Dialogue box fades out over 300ms (Tween modulate alpha 1.0 → 0.0). No sound. Silence. D-064 phase 1.
|
||||
- **NPC may react:** Server may send updated NPC animation or monologue after walk-away (D-064 phase 2). Client renders these as normal snapshot updates — no special walk-away animation on client.
|
||||
- **KG recording:** Server handles D-064 phase 3 (#427). Client just sends WalkAway input.
|
||||
|
||||
**Integration points:**
|
||||
- Pairs with server #427 (walk-away KG recording)
|
||||
- Depends on #435 (dialogue box exists)
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Walk-away is ANY WASD press during dialogue, not just "walk away from NPC." Even pressing W (toward NPC) triggers walk-away. The direction doesn't matter — the intent to move ends the conversation.
|
||||
- After WalkAway input sent, dialogue box fades but player can't move until fade completes (300ms input block). Prevents "walk through NPC" edge case.
|
||||
- If server sends new dialogue_response during fade (NPC talks again), cancel fade and re-show dialogue. Rare edge case but possible (NPC calls after player).
|
||||
|
||||
---
|
||||
|
||||
### #436: Confrontation text styling — italic voice, monologue beat (MEDIUM)
|
||||
**What exists:**
|
||||
- Dialogue box (#435) renders NPC speech + player responses
|
||||
- D-063 spec: confrontation uses same UI, different weight via styling + monologue beat
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Confrontation option styling:** Response options marked as `confrontation: true` (new protocol field) render in italic + first-person voice. Example: regular = "Shift schedule", confrontation = *"I saw you in corridor B-7"* (italic BBCode tag).
|
||||
- **Pre-delivery monologue beat:** When player clicks confrontation option, 1-2 second pause before sending DialogueResponse input. During pause, show monologue: *"This changes things. No taking it back."* (hardcoded v0.1 line, future from content pools). Monologue displays on layer 7, dialogue box dims slightly (reduce opacity to 70%).
|
||||
- **Audio dip:** Ambient audio volume drops to 50% during confrontation selection (via AudioServer bus manipulation). Restore after response sent.
|
||||
- **No camera change:** D-063 defers camera tighten to post-v0.1. Same camera position throughout.
|
||||
|
||||
**Integration points:**
|
||||
- Blocked by #435 (dialogue box)
|
||||
- Pairs with server #305 (dialogue pipeline marks confrontation lines)
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Monologue beat is client-side delay (pause before sending input). Server doesn't know about the pause — from server's view, DialogueResponse arrives 1-2s after Talk verb.
|
||||
- Italic rendering: use BBCode `[i]...[/i]` in RichTextLabel. Standard Label doesn't support italics.
|
||||
- Don't apply italic to NPC speech — only to player confrontation responses. Regular dialogue responses stay non-italic.
|
||||
- Audio dip is temporary (duration of monologue beat only). Restore ambient volume after response sent, not after NPC replies.
|
||||
|
||||
## Dependency Chain
|
||||
|
||||
```
|
||||
#431 (fog visualization) → standalone, protocol v7 prerequisite for other features
|
||||
#435 (dialogue box) → #437 (walk-away) depends on this, #436 (confrontation styling) depends on this
|
||||
#437 (walk-away) → standalone once #435 exists
|
||||
#436 (confrontation styling) → depends on #435
|
||||
```
|
||||
|
||||
## PR Workflow
|
||||
|
||||
When ready to submit, create a PR with `tea` CLI. **All flags are required** to avoid TTY prompts (see CLAUDE.md "Gitea access" section):
|
||||
|
||||
```bash
|
||||
tea pr create --repo jpmschweitzer/settled-reach --login schweitz --title "feat(ui): dialogue pipeline client (Sprint 8 client)" --description "Implements #431 #435 #437 #436 — fog entity viz, dialogue response selection, walk-away, confrontation styling" --base main --head client
|
||||
```
|
||||
@@ -0,0 +1,125 @@
|
||||
# Sprint 8: React — Joint / Integration Tasks
|
||||
|
||||
**Goal:** Activate the dialogue pipeline end-to-end — player talks to NPCs, NPCs respond, the world reacts.
|
||||
|
||||
**Teams:** Server + Client
|
||||
**Agents:** All implementation agents
|
||||
|
||||
## Pre-Sprint
|
||||
|
||||
None. All required decisions (D-028, D-060, D-062, D-063, D-064) are confirmed.
|
||||
|
||||
## Sprint Completion Proof
|
||||
|
||||
Concrete observable criteria when Sprint 8 is done:
|
||||
|
||||
1. **Dialogue flow works end-to-end:**
|
||||
- Player walks to Kael (or any NPC), presses [E], selects Talk
|
||||
- NPC greeting appears in dialogue box (bottom 20% screen, max-width)
|
||||
- 2-3 player response options visible below NPC text
|
||||
- Player clicks response → NPC replies with new text + new options
|
||||
- Conversation continues until player walks away (WASD)
|
||||
|
||||
2. **Walk-away triggers visual + server recording:**
|
||||
- Player in dialogue with NPC, presses WASD (any direction)
|
||||
- Dialogue box fades out over 300ms, silence
|
||||
- Server logs incomplete interaction in KG (verify via debug print or db query)
|
||||
- Player can walk away mid-conversation without crash
|
||||
|
||||
3. **Confrontation option appears and feels different:**
|
||||
- Detective character approaches Sera Venn after observing corridor contradiction
|
||||
- Talk → response options include at least one confrontation line (italic, first-person: *"I saw you avoiding Torek..."*)
|
||||
- Click confrontation → 1-2s pause, monologue appears ("This changes things..."), ambient audio dips to 50%
|
||||
- After pause, NPC responds (text changes, relationship may shift color on next observation)
|
||||
|
||||
4. **Fog entity recognition timing:**
|
||||
- NPC enters fog (outside vision cone but within sound/thermal range)
|
||||
- Grey blob appears for 0.6s (6 ticks normal, or 0.3s/3 ticks if anomaly)
|
||||
- During delay, monologue fires ("Those footsteps... that's Kael's walk")
|
||||
- After delay, grey blob transitions to D-033 color + faint silhouette over 0.3s
|
||||
|
||||
5. **Sound ping rendering:**
|
||||
- NPC makes loud sound outside LOS (door slam, footsteps, voice)
|
||||
- 2-3 concentric rings expand from sound direction on insert overlay (white-blue, sonar aesthetic)
|
||||
- Rings fade over 1.5s
|
||||
- No crash, no visual glitches
|
||||
|
||||
6. **Invisible locks work:**
|
||||
- Player talks to Sera with Surface trust level
|
||||
- Gets 2-3 response options (small talk, routine questions)
|
||||
- Player CANNOT see that Real/Secret tier options exist (no grayed-out text, no lock icons, no hint)
|
||||
- After trust increase (via repeated positive interactions or evidence discovery), new options appear on next Talk — player realizes "wait, there's more now"
|
||||
|
||||
## Test Plan Alignment (D-030 phases)
|
||||
|
||||
**Phase 2 (Sprint 3-4): Integration tests**
|
||||
- Sprint 8 falls in integration test phase
|
||||
- Required: End-to-end dialogue pipeline integration test (server sends dialogue_response, client renders, player input returns, server logs)
|
||||
- Required: Walk-away flow integration test (client sends WalkAway, server records KG, dialogue state clears)
|
||||
- Negative test: Locked dialogue options are NOT sent to client (server filters before protocol, client never sees them)
|
||||
|
||||
## Cross-Team Dependencies
|
||||
|
||||
| Ticket | Team | Depends on | Delivers to |
|
||||
|--------|------|------------|-------------|
|
||||
| #305 | server | (none) | Client #435 consumes dialogue_response protocol field |
|
||||
| #427 | server | #452 (ContentSlug) | Client #437 triggers via WalkAway input |
|
||||
| #435 | client | #434 (done) | Server #305 sends data to render |
|
||||
| #436 | client | #435 | Uses dialogue box from #435 |
|
||||
| #437 | client | #435 | Pairs with server #427 |
|
||||
| #450 | server | (none) | Enables urgent recognition for #431 |
|
||||
| #451 | server | #450 | Pairs with client #431 fog visualization |
|
||||
| #431 | client | Protocol v7 | Consumes pending_recognitions from server |
|
||||
|
||||
**Critical path:** Server #305 (dialogue pipeline) + Client #435 (dialogue box) are the headline integration. Everything else is enhancement/polish around that core.
|
||||
|
||||
**Parallel tracks:**
|
||||
- Track 1 (dialogue): #305 (server) ↔ #435 (client) → #436 (confrontation styling) → #437 (walk-away) ↔ #427 (server KG)
|
||||
- Track 2 (perception): #450 (server anomaly) → #451 (server monologue) → #431 (client fog viz)
|
||||
- Track 3 (infrastructure): #452 (server ContentSlug) → used by #427
|
||||
|
||||
## Integration Risks
|
||||
|
||||
1. **Protocol version mismatch:** Client #431 bumps protocol to v7 (pending_recognitions field). Server must match. Verify protocol.gd PROTOCOL_VERSION == server bridge/types.rs PROTOCOL_VERSION after merge.
|
||||
|
||||
2. **Dialogue response format:** Server #305 populates `dialogue_response: Vec<DialogueOption>` in ObserverSnapshot. Client #435 must decode this correctly (MessagePack array of maps). Test with 0 options (no dialogue), 1 option, 3 options, and 5+ options (client shows top 3).
|
||||
|
||||
3. **Walk-away during network lag:** Client sends WalkAway input. Server may send 1-2 more dialogue snapshots in flight before WalkAway is processed. Client should handle: fade dialogue box, ignore late dialogue_response updates.
|
||||
|
||||
4. **Cognitive delay + monologue timing:** Server #451 fires monologue at delay START (when grey blob appears). Client #431 must show monologue DURING the 0.6s grey-to-color transition. If monologue arrives after transition completes, timing feels wrong. Verify monologue timestamp is within delay window.
|
||||
|
||||
5. **Confrontation italic rendering:** Client uses BBCode `[i]...[/i]` for italic. If NPC name or response text contains special BBCode chars (`[`, `]`), text will break. Server should escape BBCode in dialogue text, or client should use `bbcode_text.c_escape()` before rendering.
|
||||
|
||||
## Notes
|
||||
|
||||
**Server-side dialogue filtering (D-062 invisible locks):**
|
||||
- Server filters dialogue options BEFORE sending to client. Client never sees locked options in protocol.
|
||||
- Access tier filter: RelationshipState → AccessTier mapping must be shared understanding (or document in decisions/content.md amendment)
|
||||
- Trust tier filter: KG.relationship → TrustTier. Unknown = Surface, Known/Friend = Real, PersonOfInterest+Secret fact = Secret.
|
||||
- Situation filter: Game state (time, location, NPC routine) → active Situation tags. Document the mapping function (or make it data-driven YAML config).
|
||||
|
||||
**Client-side dialogue rendering (max 3 visible, D-061 20% height):**
|
||||
- If NPC sends 5 valid options (all unlocked, all pass filters), client shows top 3 by priority.
|
||||
- If later in conversation NPC sends 2 options, client shows 2.
|
||||
- Dialogue box height scales with content (NPC text + options) but never exceeds 20% viewport height. If content overflows, scroll NPC text, keep options visible.
|
||||
|
||||
**Monologue + dialogue spatial separation (D-061, D-063):**
|
||||
- Monologue floats above dialogue box on z-layer 7.
|
||||
- During confrontation beat: monologue visible ("This changes things..."), dialogue box dimmed to 70% opacity.
|
||||
- After beat: monologue fades, dialogue box returns to 100% opacity, NPC response appears.
|
||||
|
||||
**Audio dip during confrontation (D-063):**
|
||||
- Client-side: When player clicks confrontation option, AudioServer.set_bus_volume_db(ambient_bus, -6.0) (50% volume).
|
||||
- After monologue beat + response sent, restore: AudioServer.set_bus_volume_db(ambient_bus, 0.0).
|
||||
- Duration: ~1-2 seconds (duration of monologue beat). Not tied to NPC response arrival.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. All design decisions confirmed. If blockers arise during implementation, escalate to Team Leader (Jeroen).
|
||||
|
||||
## Files Written
|
||||
|
||||
- /var/home/jeroenschweitzer/Projects/settled-reach/main/docs/sprints/sprint-8/server.md
|
||||
- /var/home/jeroenschweitzer/Projects/settled-reach/main/docs/sprints/sprint-8/client.md
|
||||
- /var/home/jeroenschweitzer/Projects/settled-reach/main/docs/sprints/sprint-8/audio.md
|
||||
- /var/home/jeroenschweitzer/Projects/settled-reach/main/docs/sprints/sprint-8/joint.md
|
||||
@@ -0,0 +1,168 @@
|
||||
# Sprint 8: React — Server Tasks
|
||||
|
||||
**Goal:** Activate the dialogue pipeline end-to-end — player talks to NPCs, NPCs respond, the world reacts.
|
||||
|
||||
**Branch:** `server`
|
||||
**Agents:** Dudley (dev), Tyre (arch), Hoshe (QA)
|
||||
|
||||
## Carry-over from Sprint 7
|
||||
|
||||
None. Sprint 7 server tickets (#308, #326, #427 partial) completed.
|
||||
|
||||
## New Tickets
|
||||
|
||||
| # | Title | Blocked by |
|
||||
|---|-------|------------|
|
||||
| #305 | Dialogue selection pipeline — 4-layer filtering engine | — |
|
||||
| #427 | Walk-away KG recording | — |
|
||||
| #450 | Wire RecognitionTrigger::Urgent to anomaly detection | — |
|
||||
| #451 | Monologue fires during cognitive delay window | — |
|
||||
| #452 | Add ContentSlug component | — |
|
||||
|
||||
Use `db/connectors/ticket show <id>` for full details.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- `decisions/content.md` — D-028 (dialogue architecture: tagged line pools with 4 relational layers), D-035 (tag taxonomy), D-062 (invisible locked options), D-063 (confrontation same box), D-064 (walk-away 3-phase consequences)
|
||||
- `decisions/architecture.md` — D-041 (knowledge graph data model)
|
||||
- `decisions/perception.md` — D-060 (cognitive delay for fog recognition), D-061 (dialogue box layout)
|
||||
|
||||
## Open Questions to Resolve Early
|
||||
|
||||
None.
|
||||
|
||||
## Notes
|
||||
|
||||
### #305: Dialogue selection pipeline — 4-layer filtering engine (CRITICAL)
|
||||
**What exists:**
|
||||
- `server/src/content/line_pool.rs` — LinePoolIndex with full D-028 Layer 1-3 filtering already implemented:
|
||||
- `query_dialogue()` filters by access tier, situation, and trust
|
||||
- All tag enums defined (AccessTier, TrustTier, Situation, Topic, Mood, Trigger, Character)
|
||||
- Indexed pools built from ContentStore at startup
|
||||
- `server/src/content/mod.rs` — ContentPlugin loads YAML content and builds LinePoolIndex resource
|
||||
- `server/src/simulation/interaction.rs` — compute_nearby_interactions() generates verb options but doesn't yet invoke dialogue selection
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Layer 4 (topic + mood weighted selection):** New system `select_dialogue_line()` that takes the filtered lines from `query_dialogue()` and scores them by topic/mood match
|
||||
- **NPC mood component:** Add `CurrentMood(Mood)` component to NPCs, computed from NPC axes (Tolerance, Contentment, recent events)
|
||||
- **Situation derivation:** Map current game state (time of day, location, NPC routine phase, interaction history) → active Situation tags
|
||||
- **Integration with Talk verb:** When player selects Talk from interaction list, trigger dialogue pipeline: access tier (from KG RelationshipState) → situations (from context) → trust tier (from KG) → topic+mood scoring → select line → send to client via ObserverSnapshot.dialogue_response
|
||||
- **Deliverable:** Full pipeline from Talk verb → selected line text on client dialogue box. Test with 1 NPC (Kael or Sera), 3-5 lines per situation.
|
||||
|
||||
**Integration points:**
|
||||
- Reads LinePoolIndex resource (already exists)
|
||||
- Reads KnowledgeGraph component for access/trust tier
|
||||
- Writes to new DialogueResponse buffer component for snapshot inclusion
|
||||
- Pairs with client #435 (dialogue response selection UI)
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Access tier mapping: RelationshipState → AccessTier needs a lookup table (Unknown=public, Known=peer, PersonOfInterest=peer+authority for detective, Friend=insider)
|
||||
- Situation activation: NPC's current RoutinePhase + game time (D-031 phases) + location → 1-3 active situations. Don't hardcode — use a mapping table/function.
|
||||
- Layer 4 is weighted, not hard filter. Empty topic/mood tags on lines should NOT exclude them — they're topic-neutral.
|
||||
- Cooldown tracking per line ID to prevent repeating the same line within a session (MonologueState pattern, but for dialogue)
|
||||
|
||||
---
|
||||
|
||||
### #427: Walk-away KG recording (medium)
|
||||
**What exists:**
|
||||
- `server/src/knowledge/graph.rs` — KnowledgeGraph component with entity/fact maps
|
||||
- Client #437 detects WASD during dialogue, sends WalkAway input event
|
||||
- D-064 specifies 3-phase consequence model
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **KnowledgeEvent::IncompleteInteraction:** New event type carrying NPC stable_id, interaction_type (Talk/Confront), tick
|
||||
- **process_walk_away_input():** System that handles PlayerInput::WalkAway during active dialogue. Emits IncompleteInteraction event, clears dialogue state.
|
||||
- **KG recording:** process_knowledge_events() adds entry to entities[npc_id].known_attributes["incomplete_interactions"] = array of {tick, type}
|
||||
- **Future query hook:** Stub function `has_incomplete_interaction(npc_id) -> bool` for dialogue/monologue to check (not called in v0.1, but scaffolds D-064 phase 3)
|
||||
|
||||
**Integration points:**
|
||||
- Pairs with client #437 (WASD detect during dialogue)
|
||||
- Builds on KnowledgeGraph infrastructure from Sprint 2
|
||||
- Future: monologue system (#451) will check this for post-conversation triggers
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Walk-away during confrontation vs casual talk should both record, but the `interaction_type` field differentiates them (future dialogue may react differently)
|
||||
- Don't emit KnowledgeEvent if no dialogue was active (player hits WASD while near NPC but not talking)
|
||||
|
||||
---
|
||||
|
||||
### #450: Wire RecognitionTrigger::Urgent to anomaly detection (medium)
|
||||
**What exists:**
|
||||
- `server/src/perception/cognitive_delay.rs` — Full cognitive delay system with RecognitionTrigger enum (Normal = 0.6s, Urgent = 0.3s)
|
||||
- `server/src/perception/observation.rs` — emit_observation_events() queues PendingRecognition but hardcodes RecognitionTrigger::Normal (line 82: `// TODO(#450)`)
|
||||
- ObserveAnomaly monologue trigger defined in line_pool.rs Trigger enum
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Anomaly detection:** Add system `detect_anomalies()` that marks entities as anomalous when KG.relationship == PersonOfInterest OR KG.state == Contradicted
|
||||
- **Component:** `AnomalyMarker` — transient marker on entities flagged as anomalous this tick (cleared each tick)
|
||||
- **Wire to cognitive delay:** In emit_observation_events(), if target has AnomalyMarker, use RecognitionTrigger::Urgent instead of Normal
|
||||
- **Verify:** Contradicted entity in fog resolves in 0.3s (3 ticks) instead of 0.6s (6 ticks)
|
||||
|
||||
**Integration points:**
|
||||
- Builds on Sprint 7 cognitive delay system (#423 done)
|
||||
- Future: monologue system will also check AnomalyMarker for ObserveAnomaly trigger priority
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- AnomalyMarker is per-tick — clear it at tick start, recompute each tick based on KG state
|
||||
- Don't mark Player entity as anomalous (causes unnecessary self-checks)
|
||||
- Sprint double-take (#428 done) already fires monologue for sprint-past anomalies; this ticket affects fog recognition timing only
|
||||
|
||||
---
|
||||
|
||||
### #451: Monologue fires during cognitive delay window (medium)
|
||||
**What exists:**
|
||||
- `server/src/simulation/monologue.rs` — Trigger enum includes ObserveAnomaly
|
||||
- `server/src/perception/cognitive_delay.rs` — CognitiveDelay component tracks pending recognitions with tick countdown
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Trigger during delay:** When a PendingRecognition is queued (entity enters fog), immediately check monologue pools for `trigger: observe_anomaly` + character match
|
||||
- **Recognition monologue:** New system `trigger_recognition_monologue()` that fires when cognitive delay starts (NOT when it completes). Line text should match D-060 spec: "Those footsteps... that's Kael's walk" — character is thinking during the delay.
|
||||
- **Pair with visual:** Monologue appears DURING the grey-blob → color+silhouette transition (client shows both simultaneously)
|
||||
|
||||
**Integration points:**
|
||||
- Reads CognitiveDelay.pending() to detect new recognitions
|
||||
- Writes to MonologueBuffer (existing pattern from #414)
|
||||
- Pairs with client #431 (fog entity visualization with delay animation)
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- Fire monologue at delay START (when grey blob appears), not at delay END (when color resolves). The D-060 design is "monologue IS the recognition process."
|
||||
- Don't fire recognition monologue for entities already in KG (only for new entities entering perception)
|
||||
- Cooldown still applies — don't spam recognition monologue every tick if the same entity blinks in/out of fog
|
||||
|
||||
---
|
||||
|
||||
### #452: Add ContentSlug component (medium)
|
||||
**What exists:**
|
||||
- Content loading in `server/src/content/spawn.rs` spawns NPCs from YAML but doesn't attach content IDs
|
||||
- KnowledgeGraph interaction memory (#427) will need to reference "which NPC" for dialogue history
|
||||
|
||||
**What the ticket needs to deliver:**
|
||||
- **Component:** `ContentSlug(String)` — stable content identifier from YAML (e.g. "kael-davan", "sera-venn")
|
||||
- **Attach during spawn:** content/spawn.rs reads `npc.slug` field from YAML and attaches ContentSlug component to spawned NPC entity
|
||||
- **Use in KG:** Interaction memory records ContentSlug instead of Entity (Entity is unstable across save/load; ContentSlug is stable)
|
||||
|
||||
**Integration points:**
|
||||
- Used by #427 (walk-away recording)
|
||||
- Future: dialogue Layer 2 (relationship history) will query past interactions by ContentSlug
|
||||
|
||||
**Non-obvious gotchas:**
|
||||
- ContentSlug is independent of StableId. StableId is runtime entity tracking (KG references). ContentSlug is authoring/content identity (which authored NPC template).
|
||||
- Not all entities have ContentSlugs (e.g., procedurally spawned NPCs, furniture). Component is optional.
|
||||
|
||||
## Dependency Chain
|
||||
|
||||
```
|
||||
#450 (anomaly detection) → standalone, enables urgent recognition
|
||||
#451 (monologue during delay) → depends on #450 for anomaly context
|
||||
#452 (ContentSlug) → #427 (walk-away KG) depends on this
|
||||
#305 (dialogue pipeline) → standalone, headline feature
|
||||
#427 (walk-away KG) → depends on #452, pairs with client #437
|
||||
```
|
||||
|
||||
## PR Workflow
|
||||
|
||||
When ready to submit, create a PR with `tea` CLI. **All flags are required** to avoid TTY prompts (see CLAUDE.md "Gitea access" section):
|
||||
|
||||
```bash
|
||||
tea pr create --repo jpmschweitzer/settled-reach --login schweitz --title "feat(content): dialogue selection pipeline (Sprint 8 server)" --description "Implements #305 #427 #450 #451 #452 — dialogue 4-layer filtering, walk-away KG, urgent recognition, delay monologue, ContentSlug component" --base main --head server
|
||||
```
|
||||
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Synthesize insert-tech UI sounds for Sprint 7 #440.
|
||||
|
||||
Each sound uses a DIFFERENT synthesis technique to ensure distinct character:
|
||||
- cursor_hover: impulse → resonant bandpass (digital click)
|
||||
- weapon_aim: filtered noise + sub thump (mechanical)
|
||||
- monologue_chime: FM synthesis (crystalline bell)
|
||||
- monologue_chime_urgent: FM synthesis + beating/dissonance (tense bell)
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import wave
|
||||
import os
|
||||
|
||||
SR = 44100
|
||||
OUTPUT_DIR = os.path.join(os.path.dirname(__file__), "..", "client", "assets", "audio")
|
||||
|
||||
|
||||
def write_wav(filename, samples, channels=1):
|
||||
"""Write float samples [-1, 1] to 16-bit WAV."""
|
||||
path = os.path.join(OUTPUT_DIR, filename)
|
||||
# Normalize to peak if it exceeds 1.0
|
||||
peak = np.max(np.abs(samples))
|
||||
if peak > 1.0:
|
||||
samples = samples / peak
|
||||
int_samples = np.clip(samples * 32767, -32767, 32767).astype(np.int16)
|
||||
with wave.open(path, "w") as f:
|
||||
f.setnchannels(channels)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(SR)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
dur_ms = len(int_samples) / SR * 1000
|
||||
print(f" wrote {path} ({dur_ms:.0f}ms, {channels}ch)")
|
||||
return path
|
||||
|
||||
|
||||
def simple_lowpass(signal, cutoff_hz, sr=SR):
|
||||
"""Single-pole IIR low-pass filter."""
|
||||
rc = 1.0 / (2 * np.pi * cutoff_hz)
|
||||
dt = 1.0 / sr
|
||||
alpha = dt / (rc + dt)
|
||||
out = np.zeros_like(signal)
|
||||
out[0] = alpha * signal[0]
|
||||
for i in range(1, len(signal)):
|
||||
out[i] = out[i - 1] + alpha * (signal[i] - out[i - 1])
|
||||
return out
|
||||
|
||||
|
||||
def simple_highpass(signal, cutoff_hz, sr=SR):
|
||||
"""Single-pole IIR high-pass filter."""
|
||||
rc = 1.0 / (2 * np.pi * cutoff_hz)
|
||||
dt = 1.0 / sr
|
||||
alpha = rc / (rc + dt)
|
||||
out = np.zeros_like(signal)
|
||||
out[0] = signal[0]
|
||||
for i in range(1, len(signal)):
|
||||
out[i] = alpha * (out[i - 1] + signal[i] - signal[i - 1])
|
||||
return out
|
||||
|
||||
|
||||
def cursor_hover():
|
||||
"""UI-001: Digital click/tick on entity hover.
|
||||
|
||||
Technique: Short noise impulse bandpass-filtered to ~4kHz.
|
||||
Sounds like a tiny digital snap — no sustained tone at all.
|
||||
The "click" of a selection appearing on a HUD.
|
||||
"""
|
||||
duration = 0.035 # 35ms — shorter than before
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
rng = np.random.default_rng(77)
|
||||
|
||||
# White noise impulse
|
||||
impulse = rng.uniform(-1, 1, n)
|
||||
|
||||
# Bandpass around 4kHz: high-pass at 3kHz, low-pass at 5kHz
|
||||
filtered = simple_highpass(impulse, 3000)
|
||||
filtered = simple_lowpass(filtered, 5500)
|
||||
|
||||
# Very fast decay — done in 25ms
|
||||
envelope = np.exp(-t * 140)
|
||||
|
||||
return write_wav("cursor_hover.wav", filtered * envelope * 0.3)
|
||||
|
||||
|
||||
def weapon_aim():
|
||||
"""UI-004: Mechanical latch for weapon aim.
|
||||
|
||||
Technique: Low-pass filtered noise burst (the clack) + sub-bass
|
||||
thump (the weight). No musical pitch — this is a MECHANICAL sound.
|
||||
Think: safety clicking off, bolt sliding home.
|
||||
"""
|
||||
duration = 0.15 # 150ms
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
rng = np.random.default_rng(42)
|
||||
|
||||
# Component 1: Low-pass noise burst — the metallic clack
|
||||
noise = rng.uniform(-1, 1, n)
|
||||
# Low-pass at 1.5kHz — dull, heavy impact, not bright
|
||||
clack = simple_lowpass(noise, 1500)
|
||||
clack_env = np.exp(-t * 60) # fast decay
|
||||
clack = clack * clack_env
|
||||
|
||||
# Component 2: Sub-bass thump — weight of the mechanism
|
||||
# 80Hz sine, very fast decay
|
||||
thump = np.sin(2 * np.pi * 80 * t)
|
||||
thump_env = np.exp(-t * 40)
|
||||
thump = thump * thump_env
|
||||
|
||||
# Component 3: High metallic click at the very start (2ms)
|
||||
click = rng.uniform(-1, 1, n)
|
||||
click = simple_highpass(click, 4000)
|
||||
click_env = np.zeros(n)
|
||||
click_mask = t < 0.003
|
||||
click_env[click_mask] = np.exp(-t[click_mask] * 800)
|
||||
click = click * click_env
|
||||
|
||||
signal = clack * 0.4 + thump * 0.35 + click * 0.25
|
||||
return write_wav("weapon_aim.wav", signal * 0.45)
|
||||
|
||||
|
||||
def monologue_chime():
|
||||
"""UI-005: Crystalline thought-chime. PLACEHOLDER.
|
||||
|
||||
Technique: FM synthesis — carrier modulated by a lower frequency
|
||||
creates rich, evolving harmonics that sound like struck glass or
|
||||
crystal. Fundamentally different timbre from additive sine waves.
|
||||
"""
|
||||
duration = 0.75 # 750ms
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
# FM synthesis: carrier at 1200Hz, modulator at 420Hz (ratio ~2.86:1)
|
||||
# Inharmonic ratio = bell-like quality
|
||||
f_carrier = 1200
|
||||
f_mod = 420
|
||||
# Mod index decays over time — bright attack, mellow sustain
|
||||
mod_index = 3.0 * np.exp(-t * 6)
|
||||
# Modulator signal
|
||||
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
|
||||
# Carrier with FM
|
||||
signal = np.sin(2 * np.pi * f_carrier * t + modulator)
|
||||
|
||||
# Gentle attack (20ms), slow decay
|
||||
attack = np.minimum(t / 0.02, 1.0)
|
||||
decay = np.exp(-t * 3.0)
|
||||
envelope = attack * decay
|
||||
|
||||
return write_wav("sfx_monologue_chime.wav", signal * envelope * 0.22)
|
||||
|
||||
|
||||
def monologue_chime_urgent():
|
||||
"""UI-006: Urgent thought-chime. PLACEHOLDER.
|
||||
|
||||
Technique: FM synthesis with higher mod index (brighter/harsher) +
|
||||
a second detuned carrier that creates beating/tension. The beating
|
||||
is what makes it feel "urgent" — not louder, but unsettled.
|
||||
"""
|
||||
duration = 0.5 # 500ms — shorter
|
||||
n = int(SR * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
f_carrier = 1200
|
||||
f_mod = 420
|
||||
|
||||
# Higher mod index = more sidebands = brighter, more aggressive
|
||||
mod_index = 5.0 * np.exp(-t * 7)
|
||||
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
|
||||
|
||||
# Primary carrier
|
||||
carrier1 = np.sin(2 * np.pi * f_carrier * t + modulator)
|
||||
|
||||
# Second carrier, detuned +8Hz — creates beating at 8Hz (nervous flicker)
|
||||
carrier2 = np.sin(2 * np.pi * (f_carrier + 8) * t + modulator * 0.8)
|
||||
|
||||
signal = carrier1 * 0.6 + carrier2 * 0.4
|
||||
|
||||
# Sharp attack (6ms), faster decay
|
||||
attack = np.minimum(t / 0.006, 1.0)
|
||||
decay = np.exp(-t * 4.0)
|
||||
envelope = attack * decay
|
||||
|
||||
# 15% louder than normal
|
||||
return write_wav("sfx_monologue_chime_urgent.wav", signal * envelope * 0.25)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
print("Synthesizing UI sounds for Sprint 7 #440...")
|
||||
print()
|
||||
|
||||
print("[UI-001] cursor_hover — bandpass noise impulse (digital click)")
|
||||
cursor_hover()
|
||||
|
||||
print("[UI-004] weapon_aim — filtered noise + sub thump (mechanical latch)")
|
||||
weapon_aim()
|
||||
|
||||
print("[UI-005] sfx_monologue_chime — FM synthesis bell (PLACEHOLDER)")
|
||||
monologue_chime()
|
||||
|
||||
print("[UI-006] sfx_monologue_chime_urgent — FM + beating (PLACEHOLDER)")
|
||||
monologue_chime_urgent()
|
||||
|
||||
print()
|
||||
print("Done. Convert with: db/connectors/audio-post convert <file.wav>")
|
||||
Reference in New Issue
Block a user