refactor(tooling): T-1290 — the assets domain, where OFF is the normal case

tooling/db/ (a misnamed directory: connectors, not database work),
trellis-batch.sh and synth_ui_sounds.py become `reach assets`:
audio {health,generate,batch,post {convert,normalize,trim,pipeline}},
image {health,generate}, trellis {health,generate,batch}, and synth-ui.
The four audio bash wrappers are retired, and tooling/db/ is gone.

Parity, from baselines taken before anything moved:

- the four UI-sound WAVs and the harmonic-synth WAVs (exponential and linear
  decay) are byte-identical
- the ffmpeg pipeline's decoded PCM is identical. Its .ogg bytes are not,
  even between two runs of the OLD code: Ogg picks a random stream serial,
  so the encoded file was never the right thing to compare
- the network success paths can't be run in a gate (Stable Audio and Trellis
  are kept off, Gemini costs money), so tooling/test_assets.py stands up a
  fake Gradio and pins every payload: the audio submit, Trellis's six-call
  session sequence with its 9-input image_to_3d, and the Gemini body. It
  failed when one Trellis value was mutated (7.5 → 7.0)

Failure classification, in endpoints.py, is the point of the port. The
services are OFF by design (VRAM on tower-of-joy, D-17), and the topology doc
warns against "fixing" one by restarting it. So a refused connection says OFF
and asks for the service to be turned on rather than restarted; a 4xx/5xx says
the request was rejected; 401/403 says credentials; 429 says quota; and an
unreachable Gemini blames the network, not VRAM.

Behaviour changes, each a failure that used to read as success or crash:

- audio batch and trellis batch exited 0 with failures in their summaries;
  they now print the summary and exit 1
- trellis generate on a missing image crashed with a TypeError
  (print(..., indent=2)); it now names the file, and checks it before the
  service so a typo is not reported as an outage
- the ffmpeg pipeline left its intermediates behind when a step failed

Structure: the connectors called each other as subprocesses (batch spawned
the connector, which spawned audio_post) and parsed each other's stdout. They
are now function calls, and ffmpeg is the only exec, through core/process.
ensure_venv() is removed: it os.execv'd into .venv, which D-263's exec rule
forbids, and reach declares the dependencies itself. config.json moved into
the domain deliberately, and the local-services rule follows it.

Output contract: results are still JSON on stdout with the same keys, so skill
readers keep working. Failures are an exit status with a Fix line, never
{"ok": false}. The audio-gen, glb-gen and image-gen skills, Araminta's agent
file and the allow-list are updated to match. glb-gen's "trellis-batch.sh is
hardcoded to one category" caveat is gone: batch takes --input-dir or --names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-23 19:49:20 +02:00
co-authored by Claude Opus 5.5
parent 4537b71b92
commit ddce4441a9
36 changed files with 1929 additions and 1754 deletions
+13
View File
@@ -0,0 +1,13 @@
"""`assets` — connectors to the generators that make game assets (D-263).
Stable Audio Open and Trellis run on tower-of-joy and are kept switched OFF to
save VRAM; Gemini is a paid cloud API. So in this domain network failure is
the normal case, and every connector reports it through endpoints.py, which
says whether the service is off or rejected the request.
Formerly tooling/db/ (a misnamed directory: it held connectors, not database
work) plus tooling/trellis-batch.sh and tooling/synth_ui_sounds.py (T-1290).
Output contract: a verb's result goes to stdout as JSON — the same keys the
old scripts printed, so the skills' readers keep working — and a failure is a
non-zero exit with a remedy, not `{"ok": false}` on stdout.
"""
+247
View File
@@ -0,0 +1,247 @@
"""Stable Audio Open connector — Gradio API wrapper.
Talks to the Stable Audio Open Gradio app on tower-of-joy :11500 (URL from
config.json). Uses the async Gradio pattern: POST to submit, then read the SSE
stream for the result, then download the file.
Formerly tooling/db/audio_connector.py, fronted by the audio-generate and
audio-health bash wrappers (T-1290). Behaviour is unchanged except where a
failure used to print `{"ok": false}` on stdout and exit 1: it now raises a
ReachError that says whether the service is OFF or rejected the request, and
post-processing is a function call rather than a second Python process.
"""
from __future__ import annotations
import json
import os
import shutil
import time
import urllib.error
import urllib.request
from tooling.core import console
from tooling.core.errors import ReachError
from tooling.domains.assets import audio_post, endpoints
SERVICE = "audio"
def get_base_url() -> str:
return endpoints.base_url(SERVICE)
def health() -> dict:
"""Is the Stable Audio API reachable, and what does it expose?"""
base = get_base_url()
data = endpoints.call_json(
urllib.request.Request(f"{base}/config", method="GET"),
service=SERVICE,
what="the health check",
timeout=10,
)
api_names = [
dep.get("api_name", "")
for dep in data.get("dependencies", [])
if dep.get("api_name", "") and not dep.get("api_name", "").startswith("js_")
]
return {
"ok": True,
"url": base,
"gradio_version": data.get("version", "unknown"),
"api_endpoints": api_names,
}
def post_process(wav_path: str, ogg_path: str | None = None, lufs=-16, quality=6, threshold=-50) -> dict:
"""trim + normalize + convert a generated WAV (the `--post` step)."""
if ogg_path is None:
ogg_path = os.path.splitext(wav_path)[0] + ".ogg"
console.event(f"Post-processing → {os.path.basename(ogg_path)}...")
try:
return audio_post.pipeline(wav_path, ogg_path, quality=quality, lufs=lufs, threshold=threshold)
except ReachError as exc:
# Not fatal to the generation: the WAV exists and is reported, and the
# failure is carried in the result exactly as before.
return {"ok": False, "error": f"Post-processing failed: {exc.message}"}
def default_output(prompt: str, duration: float) -> str:
"""Auto-name: the first 40 prompt characters, sanitised, plus the duration."""
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
safe = safe.strip().replace(" ", "_").lower()
return f"{safe}_{int(duration)}s.wav"
def extract_file_url(result_data, base: str) -> str | None:
"""Find the audio file URL in Gradio's `complete` payload, made absolute."""
if isinstance(result_data, list) and len(result_data) > 0:
audio_info = result_data[0]
elif isinstance(result_data, dict) and "data" in result_data:
audio_info = result_data["data"][0] if result_data["data"] else None
else:
audio_info = result_data
file_url = None
if isinstance(audio_info, dict):
file_url = audio_info.get("url") or audio_info.get("path")
elif isinstance(audio_info, str):
file_url = audio_info
if not file_url:
return None
if file_url.startswith("/"):
return f"{base}{file_url}"
if not file_url.startswith("http"):
return f"{base}/file={file_url}"
return file_url
def generate(
prompt: str,
duration: float = 10.0,
steps: int = 100,
cfg: float = 7.0,
output: str | None = None,
timeout: int = 600,
post: bool = False,
output_ogg: str | None = None,
) -> dict:
"""Generate audio from a text prompt; returns the result dict.
duration is 0-47 s. Fewer steps is faster and worse. `post` (implied by
`output_ogg`) runs trim + normalize + convert on the result.
"""
base = get_base_url()
# Fail fast, and say OFF rather than "submit failed", before building anything.
endpoints.call(
urllib.request.Request(f"{base}/config", method="GET"),
service=SERVICE,
what="the availability check",
timeout=5,
)
api_url = f"{base}/gradio_api/call/generate_audio"
output = output or default_output(prompt, duration)
# 1. Submit.
console.event(f"Submitting: {prompt!r} ({duration}s, {steps} steps, cfg {cfg})")
submitted = endpoints.call_json(
urllib.request.Request(
api_url,
data=json.dumps({"data": [prompt, duration, steps, cfg]}).encode(),
headers={"Content-Type": "application/json"},
method="POST",
),
service=SERVICE,
what="the generation request",
timeout=30,
)
event_id = submitted.get("event_id") if isinstance(submitted, dict) else None
if not event_id:
raise ReachError(
f"Stable Audio accepted the request but returned no event_id: {submitted}",
fix="the Gradio app's API may have changed — check `reach assets audio health` endpoints",
)
console.event(f"Event {event_id} — waiting for generation (timeout {timeout}s)...")
# 2. Read the SSE stream. A dropped connection is retried until the
# deadline, exactly as before; only an `error` event is fatal.
started = time.time()
result_data = _await_result(f"{api_url}/{event_id}", timeout, started)
elapsed = round(time.time() - started, 1)
console.event(f"Generation complete ({elapsed}s)")
# 3. Download.
file_url = extract_file_url(result_data, base)
if not file_url:
raise ReachError(
f"could not find the audio URL in the response: {str(result_data)[:300]}",
fix="the Gradio app's response shape may have changed — see audio.extract_file_url",
)
console.event(f"Downloading to {output}...")
try:
with urllib.request.urlopen(urllib.request.Request(file_url, method="GET"), timeout=60) as resp:
with open(output, "wb") as f:
shutil.copyfileobj(resp, f)
except urllib.error.URLError as exc:
raise ReachError(
f"download failed from {file_url}: {exc}",
fix="the file may have expired on the server — re-run the generation",
) from exc
result = {
"ok": True,
"file": output,
"size_bytes": os.path.getsize(output),
"duration_requested": duration,
"steps": steps,
"cfg": cfg,
"prompt": prompt,
"generation_time_s": elapsed,
}
if post or output_ogg:
post_result = post_process(output, ogg_path=output_ogg)
if not post_result.get("ok"):
result["post_processed"] = False
result["post_error"] = post_result.get("error", "unknown")
else:
result["post_processed"] = True
result["ogg_file"] = post_result.get("output", output_ogg)
result["ogg_size_bytes"] = os.path.getsize(result["ogg_file"])
return result
def _await_result(stream_url: str, timeout: int, start: float):
"""Poll the Gradio SSE stream until `complete`, an `error`, or the deadline."""
last_status = None
while time.time() - start < timeout:
try:
with urllib.request.urlopen(urllib.request.Request(stream_url, method="GET"), timeout=timeout) as resp:
current_event = None
for line_bytes in resp:
line = line_bytes.decode("utf-8").strip()
if line.startswith("event: "):
current_event = line[7:]
continue
if not (line.startswith("data: ") and current_event):
continue
data_str = line[6:]
if current_event == "heartbeat":
elapsed = int(time.time() - start)
if elapsed % 30 == 0 and elapsed > 0:
console.event(f"Still generating... ({elapsed}s elapsed)")
elif current_event == "error":
try:
detail = json.loads(data_str)
except (json.JSONDecodeError, TypeError):
detail = data_str
raise ReachError(
f"Stable Audio reported a generation error: {detail}",
fix="the service is up — check the prompt and duration (0-47 s), then re-run",
)
elif current_event == "complete":
try:
return json.loads(data_str)
except json.JSONDecodeError:
return data_str
elif current_event == "progress":
try:
status = str(json.loads(data_str))[:80]
except (json.JSONDecodeError, TypeError):
continue
if status != last_status:
console.event(f"Progress: {status}")
last_status = status
except urllib.error.URLError:
time.sleep(2) # connection dropped mid-stream — retry until the deadline
raise ReachError(
f"generation timed out after {timeout}s",
fix="raise --timeout, or lower --steps / --duration",
)
+232
View File
@@ -0,0 +1,232 @@
"""Batch audio generation from a manifest file.
Processes assets in order: Stable Audio generation (`method: sao`) or harmonic
synthesis (`method: synth`), each followed by post-processing to OGG.
Formerly tooling/db/audio_batch.py behind the audio-batch wrapper (T-1290). It
used to re-launch audio_connector.py and audio_post.py as subprocesses and
parse their stdout; it now calls them. The synthesis itself is unchanged —
same parameters give byte-identical WAVs. One behaviour change: a batch with
failures used to print `"ok": false` and exit 0; the router now fails it.
Manifest schema (docs/assets/audio/, see the audio-gen skill):
{"output_dir": ..., "gen_dir": ..., "defaults": {steps, cfg, timeout, lufs, quality},
"assets": [{"id", "filename", "method": "sao"|"synth", "prompt"?, "duration"?,
"synth"?: {"type": "harmonic", "duration", "fundamental",
"harmonics": [{"freq", "db"}], "attack_ms",
"sustain_ratio", "decay": "exponential"|"linear"}}]}
"""
from __future__ import annotations
import json
import os
import wave
import numpy as np
from tooling.core import config, console
from tooling.core.errors import ReachError
from tooling.domains.assets import audio, audio_post
SAMPLE_RATE = 44100
def load_manifest(path: str) -> dict:
try:
with open(path) as f:
return json.load(f)
except FileNotFoundError as exc:
raise ReachError(f"manifest not found: {path}", fix="pass a manifest .json path") from exc
except json.JSONDecodeError as exc:
raise ReachError(f"manifest is not valid JSON: {path}: {exc}", fix="fix the JSON, then re-run") from exc
def resolve_paths(manifest: dict) -> tuple[str, str]:
"""output_dir and gen_dir, relative to the repo root (created if missing).
The old script found the root by walking up from the manifest looking for
.git; config.repo_root() asks git, which gives the same answer in a
worktree and does not silently fall back to the manifest's own directory.
"""
root = str(config.repo_root())
output_dir = os.path.join(root, manifest.get("output_dir", "client/assets/audio"))
gen_dir = os.path.join(root, manifest.get("gen_dir", "client/assets/audio/gen"))
os.makedirs(output_dir, exist_ok=True)
os.makedirs(gen_dir, exist_ok=True)
return output_dir, gen_dir
def get_default(manifest: dict, asset: dict, key: str):
"""An asset value, falling back to the manifest's defaults."""
return asset.get(key, manifest.get("defaults", {}).get(key))
def run_sao_generate(asset: dict, manifest: dict, gen_dir: str, output_dir: str) -> dict:
"""Generate via Stable Audio, post-processing straight to the final OGG."""
filename = asset["filename"]
base_name = os.path.splitext(filename)[0]
return audio.generate(
asset["prompt"],
duration=asset.get("duration", 10),
steps=get_default(manifest, asset, "steps") or 100,
cfg=get_default(manifest, asset, "cfg") or 7,
output=os.path.join(gen_dir, base_name + ".wav"),
timeout=get_default(manifest, asset, "timeout") or 600,
output_ogg=os.path.join(output_dir, filename),
)
def synthesize_harmonic(params: dict, wav_path: str) -> str:
"""Synthesize a tone from harmonic parameters and write a 16-bit mono WAV."""
sr = SAMPLE_RATE
duration = params["duration"]
fundamental = params["fundamental"]
harmonics = params.get("harmonics", [])
attack_ms = params.get("attack_ms", 10)
sustain_ratio = params.get("sustain_ratio", 0.2)
decay = params.get("decay", "exponential")
n = int(sr * duration)
t = np.linspace(0, duration, n, endpoint=False)
signal = np.sin(2 * np.pi * fundamental * t)
for h in harmonics:
amplitude = 10 ** (h["db"] / 20)
signal = signal + amplitude * np.sin(2 * np.pi * h["freq"] * t)
# Envelope: attack + sustain + decay.
attack_s = attack_ms / 1000
attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n)
sustain_end = duration * sustain_ratio
if decay == "exponential":
# Reach -60 dB by the end of the duration.
decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10
decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end)))
else:
decay_env = np.where(t < sustain_end, 1.0, 1.0 - (t - sustain_end) / (duration - sustain_end))
signal = signal * attack_env * decay_env
peak = np.max(np.abs(signal))
if peak > 0:
signal = signal / peak * 0.9
int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16)
with wave.open(wav_path, "w") as f:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(sr)
f.writeframes(int_samples.tobytes())
return wav_path
def run_synth(asset: dict, manifest: dict, gen_dir: str, output_dir: str) -> dict:
"""Synthesize, then normalize + convert (no trim: synth has no silence to trim)."""
filename = asset["filename"]
base_name = os.path.splitext(filename)[0]
wav_path = os.path.join(gen_dir, base_name + "_synth.wav")
norm_path = os.path.join(gen_dir, base_name + "_norm.wav")
ogg_path = os.path.join(output_dir, filename)
synth_params = asset.get("synth")
if not synth_params:
return {"ok": False, "error": "No synth parameters provided"}
if synth_params.get("type", "harmonic") != "harmonic":
return {"ok": False, "error": f"Unknown synth type: {synth_params.get('type')}"}
try:
synthesize_harmonic(synth_params, wav_path)
audio_post.normalize(wav_path, norm_path, lufs=get_default(manifest, asset, "lufs") or -16)
audio_post.convert(norm_path, ogg_path, quality=get_default(manifest, asset, "quality") or 6)
except ReachError as exc:
return {"ok": False, "error": exc.message}
finally:
if os.path.exists(norm_path):
os.remove(norm_path)
return {
"ok": True,
"file": wav_path,
"ogg_file": ogg_path,
"ogg_size_bytes": os.path.getsize(ogg_path),
"synth_params": synth_params,
"post_processed": True,
}
def run(
manifest_path: str,
*,
dry_run: bool = False,
only: set[str] | None = None,
skip_existing: bool = False,
) -> dict:
"""Process a manifest; returns the summary. The router fails it on any failure."""
manifest = load_manifest(manifest_path)
output_dir, gen_dir = resolve_paths(manifest)
assets = manifest.get("assets", [])
if only:
assets = [a for a in assets if a["id"] in only]
# One health check up front if anything needs Stable Audio — so an OFF
# service fails once, with the right remedy, not once per asset.
if not dry_run and any(a.get("method") == "sao" for a in assets):
console.event("Checking Stable Audio health...")
audio.health()
total = len(assets)
results = []
counts = {"success": 0, "failed": 0, "skipped": 0}
console.event(
f"Processing {total} assets from {os.path.basename(manifest_path)}"
+ (" — dry run, nothing generated" if dry_run else "")
)
for i, asset in enumerate(assets, 1):
asset_id, filename = asset["id"], asset["filename"]
method = asset.get("method", "sao")
phase = f"{i}/{total}"
console.event(f"{asset_id}: {filename} ({method})", phase=phase, progress=i / total if total else None)
if skip_existing and os.path.exists(os.path.join(output_dir, filename)):
console.event("skipping — already exists", phase=phase)
results.append({"id": asset_id, "status": "skipped", "reason": "exists"})
counts["skipped"] += 1
continue
if dry_run:
if method == "sao":
console.event(f"would generate — prompt: {asset.get('prompt', '(none)')[:80]}...", phase=phase)
elif method == "synth":
synth = asset.get("synth", {})
console.event(
f"would synthesize — {synth.get('fundamental')}Hz, {synth.get('duration')}s", phase=phase
)
results.append({"id": asset_id, "status": "dry_run"})
continue
if method == "sao":
try:
result = run_sao_generate(asset, manifest, gen_dir, output_dir)
except ReachError as exc:
result = {"ok": False, "error": exc.message}
elif method == "synth":
result = run_synth(asset, manifest, gen_dir, output_dir)
else:
result = {"ok": False, "error": f"Unknown method: {method}"}
result["id"] = asset_id
if result.get("ok"):
counts["success"] += 1
result["status"] = "success"
console.event(f"OK → {result.get('ogg_file', filename)}", phase=phase)
else:
counts["failed"] += 1
result["status"] = "failed"
console.event(f"FAILED: {result.get('error', 'unknown')}", phase=phase, level="warn")
results.append(result)
return {"ok": counts["failed"] == 0, "total": total, **counts, "results": results}
+113
View File
@@ -0,0 +1,113 @@
"""Audio post-processing around ffmpeg.
convert — WAV to OGG (libvorbis, quality 6)
normalize — LUFS normalize to -16 LUFS (broadcast standard)
trim — remove leading/trailing silence
pipeline — trim + normalize + convert (the full post-processing chain)
Every operation writes a new file and never overwrites its input. Each returns
a result dict — the same keys the old script printed as JSON — so callers read
data rather than parse output. ffmpeg runs through `core/process.run`, the one
guarded exec (D-263): a non-zero exit becomes a ReachError naming the command,
and a missing ffmpeg says what to install.
The ffmpeg argument lists are unchanged from tooling/db/audio_post.py (T-1290);
the decoded audio of a pipeline run is identical before and after the port.
"""
from __future__ import annotations
import os
from tooling.core import console, process
FFMPEG_MISSING = "install ffmpeg (brew install ffmpeg), or check PATH in a non-interactive shell"
def trim_filter(threshold: int) -> str:
"""The silence-removal filter chain: trim the start, reverse, trim, reverse."""
return (
"silenceremove=start_periods=1:start_silence=0.05"
f":start_threshold={threshold}dB,"
"areverse,"
"silenceremove=start_periods=1:start_silence=0.05"
f":start_threshold={threshold}dB,"
"areverse"
)
def ffmpeg_argv(args: list[str]) -> list[str]:
"""The full argv for one ffmpeg call — pure, so it can be tested unrun."""
return ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error", *args]
def run_ffmpeg(args: list[str], description: str) -> None:
console.event(description)
process.run(ffmpeg_argv(args), missing_fix=FFMPEG_MISSING)
def convert(input_path: str, output: str | None = None, quality: int = 6) -> dict:
"""WAV → OGG (libvorbis)."""
output = output or input_path.rsplit(".", 1)[0] + ".ogg"
run_ffmpeg(
["-i", input_path, "-c:a", "libvorbis", "-q:a", str(quality), output],
f"converting {os.path.basename(input_path)} → {os.path.basename(output)}",
)
return {"ok": True, "output": output, "size_bytes": os.path.getsize(output)}
def normalize(input_path: str, output: str | None = None, lufs: float = -16) -> dict:
"""LUFS-normalize an audio file."""
output = output or _suffixed(input_path, "_norm")
run_ffmpeg(
["-i", input_path, "-af", f"loudnorm=I={lufs}:LRA=11:TP=-1", output],
f"normalizing to {lufs} LUFS",
)
return {"ok": True, "output": output}
def trim(input_path: str, output: str | None = None, threshold: int = -50) -> dict:
"""Trim leading and trailing silence."""
output = output or _suffixed(input_path, "_trimmed")
run_ffmpeg(
["-i", input_path, "-af", trim_filter(threshold), output],
f"trimming silence (threshold: {threshold}dB)",
)
return {"ok": True, "output": output}
def pipeline(
input_path: str,
output: str | None = None,
quality: int = 6,
lufs: float = -16,
threshold: int = -50,
) -> dict:
"""Full post-processing: trim → normalize → convert to OGG."""
base = input_path.rsplit(".", 1)[0]
trimmed = base + "_trimmed.wav"
normalized = base + "_norm.wav"
output = output or base + ".ogg"
try:
run_ffmpeg(["-i", input_path, "-af", trim_filter(threshold), trimmed], "step 1/3: trimming silence")
run_ffmpeg(
["-i", trimmed, "-af", f"loudnorm=I={lufs}:LRA=11:TP=-1", normalized],
f"step 2/3: normalizing to {lufs} LUFS",
)
run_ffmpeg(
["-i", normalized, "-c:a", "libvorbis", "-q:a", str(quality), output],
"step 3/3: converting to OGG",
)
finally:
# The old script left the intermediates behind when a step failed.
for leftover in (trimmed, normalized):
if os.path.exists(leftover):
os.remove(leftover)
return {"ok": True, "output": output, "size_bytes": os.path.getsize(output)}
def _suffixed(path: str, suffix: str) -> str:
base, ext = os.path.splitext(path)
return base + suffix + ext
+4
View File
@@ -0,0 +1,4 @@
{
"stable_audio_url": "http://192.168.86.149:11500",
"trellis_url": "http://192.168.86.149:11510"
}
+151
View File
@@ -0,0 +1,151 @@
"""Where the generators live, and how a call to one fails.
Two jobs, both shared by every connector in the domain:
1. **Endpoints and keys.** Base URLs come from `config.json` beside this file
(moved deliberately from tooling/db/config.json in T-1290). Keys come from the
environment only: config.json is tracked, so a config fallback for a paid
API key is how one gets committed.
2. **Failure classification.** Network failure is the NORMAL case here, not an
exception. Stable Audio and Trellis are kept stopped because VRAM on
tower-of-joy is scarce (system-admin-danoontje docs/topology.md, D-17), so
"connection refused" means *switched off*, and the remedy is to get it
turned on — never to restart it blindly, which takes VRAM from whatever is
running. That is a different answer from "the service rejected what you
sent", and a caller that cannot tell the two apart wastes a round trip
either way. `call()` maps every urllib failure to one of the two.
`ensure_venv()` (formerly in tooling/db/common.py) is gone: it re-exec'd the
script under .venv/bin/python via os.execv. Under reach the dependencies are
declared by the package itself, and an execv carrying reach's argv into another
interpreter would relaunch something that is not the command at all.
"""
from __future__ import annotations
import json
import os
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any
from tooling.core.errors import ReachError
CONFIG_PATH = Path(__file__).resolve().parent / "config.json"
TOPOLOGY = "/var/mnt/data/projects/system-admin-danoontje/docs/topology.md"
# service key -> (display name, config key, fallback URL)
SERVICES: dict[str, tuple[str, str, str]] = {
"audio": ("Stable Audio Open", "stable_audio_url", "http://tower-of-joy:11500"),
"trellis": ("Trellis", "trellis_url", "http://tower-of-joy:11510"),
}
def load_config() -> dict:
"""The tracked endpoint configuration (URLs only, never secrets)."""
with open(CONFIG_PATH) as f:
return json.load(f)
def get_base_url(key: str, default: str) -> str:
"""Resolve a service base URL from config.json, with a fallback default."""
return load_config().get(key, default)
def base_url(service: str) -> str:
"""The configured base URL for one of SERVICES."""
_name, config_key, default = SERVICES[service]
return get_base_url(config_key, default)
def get_api_key(env_var: str) -> str:
"""An API key from the environment — environment-only, by design."""
key = os.environ.get(env_var)
if key:
return key
raise ReachError(
f"{env_var} is not set",
fix=(
f"export {env_var}, or add it to the machine-local "
".claude/settings.local.json env block (untracked). Never put keys in "
"tooling/domains/assets/config.json — it is tracked."
),
)
def unreachable(service: str, url: str, detail: str) -> ReachError:
"""The service did not answer at all.
For the tower-of-joy generators that is their normal resting state. For
anything else (the Gemini API) it means the network, not the service.
"""
if service not in SERVICES:
return ReachError(
f"{service} is not reachable at {url} ({detail})",
fix="check this machine's network connection, then re-run",
)
name = SERVICES[service][0]
return ReachError(
f"{name} is not reachable at {url} ({detail}). It is kept switched off to "
"save VRAM on tower-of-joy, so this usually means OFF, not broken.",
fix=(
f"ask for {name} to be turned on (something else may need to stop first "
f"to free VRAM — {TOPOLOGY}); do not restart it blindly. "
f"Then: reach assets {service} health"
),
)
def rejected(name: str, what: str, code: int, body: str) -> ReachError:
"""The service answered, and said no — the request is what needs changing."""
snippet = body.strip()[:500]
if code in (401, 403):
fix = "the service is up but refused the credentials — check the API key"
elif code == 429:
fix = "rate-limited or out of quota — wait, or check the account's quota"
else:
fix = "the service is up — check the arguments and inputs, then re-run"
return ReachError(
f"{name} rejected {what} (HTTP {code})" + (f": {snippet}" if snippet else ""),
fix=fix,
)
def call(
request: urllib.request.Request,
*,
service: str,
what: str,
timeout: float,
) -> bytes:
"""Perform one HTTP request, classifying any failure. Returns the body."""
name = SERVICES[service][0] if service in SERVICES else service
try:
with urllib.request.urlopen(request, timeout=timeout) as resp:
return resp.read()
except urllib.error.HTTPError as exc:
raise rejected(name, what, exc.code, exc.read().decode("utf-8", errors="replace")) from exc
except (urllib.error.URLError, TimeoutError, ConnectionError) as exc:
reason = getattr(exc, "reason", exc)
raise unreachable(service, _origin(request), str(reason)) from exc
def call_json(request: urllib.request.Request, *, service: str, what: str, timeout: float) -> Any:
"""`call()`, then parse JSON — an unparseable body is the service's fault, named as such."""
body = call(request, service=service, what=what, timeout=timeout)
try:
return json.loads(body)
except json.JSONDecodeError as exc:
raise ReachError(
f"{what}: the service answered with something that is not JSON: {body[:200]!r}",
fix="the service may be starting up or misconfigured — check its logs on tower-of-joy",
) from exc
def _origin(request: urllib.request.Request) -> str:
"""scheme://host:port of a request — the part that identifies the box."""
parts = request.full_url.split("/")
return "/".join(parts[:3])
+149
View File
@@ -0,0 +1,149 @@
"""Gemini image-generation connector — direct API wrapper.
Generates images through Google's gemini-2.5-flash-image model. The key comes
from GEMINI_API_KEY in the environment only (endpoints.get_api_key). **Every
generate call costs real money**; `health` only lists models and is free.
Formerly tooling/db/image_connector.py (T-1290). Unchanged except that
failures raise a ReachError instead of printing `{"ok": false}` and exiting 1,
and a network failure is reported as the network rather than as the API.
The key travels in the URL, so no error message ever includes the URL's query.
"""
from __future__ import annotations
import base64
import json
import os
import urllib.request
from tooling.core import console
from tooling.core.errors import ReachError
from tooling.domains.assets import endpoints
SERVICE = "Gemini API"
MODEL = "gemini-2.5-flash-image"
API = "https://generativelanguage.googleapis.com/v1beta"
DEFAULT_OUTPUT_DIR = os.path.expanduser("~/Pictures/mcp-images")
# Real API values for imageConfig.aspectRatio (not a prompt hint).
# https://ai.google.dev/gemini-api/docs/image-generation
ASPECT_RATIOS = ("1:1", "3:2", "2:3", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9")
MIME_TYPES = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp"}
def get_api_key() -> str:
return endpoints.get_api_key("GEMINI_API_KEY")
def health() -> dict:
"""Is the Gemini API reachable with the configured key? (Free — lists models.)"""
key = get_api_key()
data = endpoints.call_json(
urllib.request.Request(f"{API}/models?key={key}", method="GET"),
service=SERVICE,
what="the model listing",
timeout=10,
)
models = [
m.get("name", "")
for m in data.get("models", [])
if "imagen" in m.get("name", "").lower() or "flash" in m.get("name", "").lower()
]
return {"ok": True, "api": "gemini", "image_capable_models": models[:5]}
def default_output(prompt: str) -> str:
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
safe = safe.strip().replace(" ", "_").lower()
return os.path.join(DEFAULT_OUTPUT_DIR, f"{safe}.png")
def build_request_body(
prompt: str,
aspect_ratio: str | None = "1:1",
image_size: str | None = None,
input_image: str | None = None,
) -> dict:
"""The generateContent payload — pure, so it can be tested without a call."""
parts = []
if input_image:
if not os.path.isfile(input_image):
raise ReachError(
f"input image not found: {input_image}",
fix="pass --input with an existing .png/.jpg/.webp",
)
with open(input_image, "rb") as f:
data = base64.b64encode(f.read()).decode("utf-8")
mime = MIME_TYPES.get(os.path.splitext(input_image)[1].lower(), "image/png")
parts.append({"inlineData": {"mimeType": mime, "data": data}})
# The size is a best-effort prompt hint only: this model has no resolution
# parameter, unlike the aspect ratio below.
parts.append({"text": prompt + (f" Resolution: {image_size}." if image_size else "")})
generation_config: dict = {"responseModalities": ["TEXT", "IMAGE"]}
if aspect_ratio:
generation_config["imageConfig"] = {"aspectRatio": aspect_ratio}
return {"contents": [{"parts": parts}], "generationConfig": generation_config}
def generate(
prompt: str,
output: str | None = None,
aspect_ratio: str = "1:1",
image_size: str | None = None,
input_image: str | None = None,
) -> dict:
"""Generate one image. COSTS MONEY. Returns the result dict."""
key = get_api_key()
body = build_request_body(prompt, aspect_ratio, image_size, input_image)
output = output or default_output(prompt)
console.event(f"Generating image: {prompt!r}" + (f" (from {input_image})" if input_image else ""))
result = endpoints.call_json(
urllib.request.Request(
f"{API}/models/{MODEL}:generateContent?key={key}",
data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"},
method="POST",
),
service=SERVICE,
what="the generation request",
timeout=120,
)
candidates = result.get("candidates", [])
if not candidates:
raise ReachError(
f"Gemini returned no candidates: {json.dumps(result)[:500]}",
fix="the prompt may have been blocked by safety filters — rephrase it and re-run",
)
image_saved = False
text_response = ""
for candidate in candidates:
for part in candidate.get("content", {}).get("parts", []):
if "inlineData" in part:
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
with open(output, "wb") as f:
f.write(base64.b64decode(part["inlineData"]["data"]))
image_saved = True
elif "text" in part:
text_response += part["text"]
if not image_saved:
raise ReachError(
f"Gemini answered with text but no image: {text_response[:500]!r}",
fix="make the prompt ask for an image explicitly, then re-run",
)
return {
"ok": True,
"file": output,
"size_bytes": os.path.getsize(output),
"prompt": prompt,
"aspect_ratio": aspect_ratio,
}
+306
View File
@@ -0,0 +1,306 @@
"""Transport for the `assets` domain — args in, delegate, format out.
A verb's result is printed to stdout as JSON with the same keys the old
connector scripts printed, so a skill that reads `file`, `ogg_file` or
`size_bytes` keeps working. What changed is failure: it is a non-zero exit
with a remedy on the event stream, never `{"ok": false}` on stdout — so a
caller checks the exit status, not a field.
"""
from __future__ import annotations
import json
from pathlib import Path
import typer
from tooling.core import cli, console
from tooling.core.command import command
from tooling.core.errors import ReachError, unknown_choice
app = cli.domain("assets", "Asset generators — Stable Audio, Gemini images, Trellis 3D.")
audio_app = cli.domain("audio", "Stable Audio Open on tower-of-joy :11500 (kept OFF — VRAM).")
post_app = cli.domain("post", "ffmpeg post-processing: trim, normalize, convert.")
image_app = cli.domain("image", "Gemini image generation — every generate COSTS MONEY.")
trellis_app = cli.domain("trellis", "Trellis image-to-3D on tower-of-joy :11510 (kept OFF — VRAM).")
@app.callback()
def _domain() -> None:
"""Keeps `assets` a group (Typer collapses a single-command app)."""
@audio_app.callback()
def _audio() -> None:
"""Keeps `audio` a group."""
@post_app.callback()
def _post() -> None:
"""Keeps `post` a group."""
@image_app.callback()
def _image() -> None:
"""Keeps `image` a group."""
@trellis_app.callback()
def _trellis() -> None:
"""Keeps `trellis` a group."""
audio_app.add_typer(post_app, name="post")
app.add_typer(audio_app, name="audio")
app.add_typer(image_app, name="image")
app.add_typer(trellis_app, name="trellis")
def _emit(result: dict) -> None:
console.out(json.dumps(result, indent=2))
def _emit_summary(summary: dict, what: str) -> None:
"""Print a batch summary, then fail the command if any item failed."""
_emit(summary)
if summary.get("failed"):
raise ReachError(
f"{what}: {summary['failed']} of {summary['total']} failed (details in the summary above)",
fix="fix the failing items, then re-run with --skip-existing to leave the rest alone",
)
console.verdict(f"{what}: {summary.get('success', 0)} done, {summary.get('skipped', 0)} skipped")
# --- audio ----------------------------------------------------------------
@audio_app.command("health")
@command
def audio_health() -> None:
"""Is Stable Audio up? OFF is its normal resting state — this says which."""
from tooling.domains.assets import audio
_emit(audio.health())
@audio_app.command("generate")
@command
def audio_generate(
prompt: str = typer.Argument(..., help="What the audio should sound like."),
duration: float = typer.Option(10.0, "--duration", help="Seconds, 0-47."),
steps: int = typer.Option(100, "--steps", help="Diffusion steps; fewer is faster and worse."),
cfg: float = typer.Option(7.0, "--cfg", help="Classifier-free guidance scale."),
output: Path = typer.Option(None, "--output", help="WAV path (default: named from the prompt)."),
timeout: int = typer.Option(600, "--timeout", help="Seconds to wait for the result."),
post: bool = typer.Option(False, "--post", help="Also trim + normalize + convert to OGG."),
output_ogg: Path = typer.Option(None, "--output-ogg", help="OGG path; implies --post."),
) -> None:
"""Generate audio from a prompt; prints the result JSON."""
from tooling.domains.assets import audio
_emit(
audio.generate(
prompt,
duration=duration,
steps=steps,
cfg=cfg,
output=str(output) if output else None,
timeout=timeout,
post=post,
output_ogg=str(output_ogg) if output_ogg else None,
)
)
@audio_app.command("batch")
@command
def audio_batch(
manifest: Path = typer.Argument(..., help="Manifest .json (see the audio-gen skill)."),
dry_run: bool = typer.Option(False, "--dry-run", help="List what would be generated."),
only: str = typer.Option(None, "--only", help="Comma-separated asset ids."),
skip_existing: bool = typer.Option(False, "--skip-existing", help="Leave existing OGGs alone."),
) -> None:
"""Generate every asset in a manifest; fails if any asset failed."""
from tooling.domains.assets import audio_batch as service
summary = service.run(
str(manifest),
dry_run=dry_run,
only=set(only.split(",")) if only else None,
skip_existing=skip_existing,
)
_emit_summary(summary, "audio batch")
@post_app.command("convert")
@command
def post_convert(
input: Path = typer.Argument(..., help="Input WAV."),
output: Path = typer.Option(None, "--output", "-o", help="Output (default: same name .ogg)."),
quality: int = typer.Option(6, "--quality", "-q", help="Vorbis quality 0-10."),
) -> None:
"""WAV → OGG (libvorbis)."""
from tooling.domains.assets import audio_post
_emit(audio_post.convert(str(input), str(output) if output else None, quality))
@post_app.command("normalize")
@command
def post_normalize(
input: Path = typer.Argument(..., help="Input audio file."),
output: Path = typer.Option(None, "--output", "-o", help="Output (default: <name>_norm)."),
lufs: float = typer.Option(-16, "--lufs", help="Target loudness, LUFS."),
) -> None:
"""LUFS-normalize an audio file."""
from tooling.domains.assets import audio_post
_emit(audio_post.normalize(str(input), str(output) if output else None, lufs))
@post_app.command("trim")
@command
def post_trim(
input: Path = typer.Argument(..., help="Input audio file."),
output: Path = typer.Option(None, "--output", "-o", help="Output (default: <name>_trimmed)."),
threshold: int = typer.Option(-50, "--threshold", help="Silence threshold, dB."),
) -> None:
"""Trim leading and trailing silence."""
from tooling.domains.assets import audio_post
_emit(audio_post.trim(str(input), str(output) if output else None, threshold))
@post_app.command("pipeline")
@command
def post_pipeline(
input: Path = typer.Argument(..., help="Input WAV."),
output: Path = typer.Option(None, "--output", "-o", help="Output OGG (default: same name .ogg)."),
quality: int = typer.Option(6, "--quality", "-q", help="Vorbis quality 0-10."),
lufs: float = typer.Option(-16, "--lufs", help="Target loudness, LUFS."),
threshold: int = typer.Option(-50, "--threshold", help="Silence threshold, dB."),
) -> None:
"""trim → normalize → convert to OGG, the full chain."""
from tooling.domains.assets import audio_post
_emit(audio_post.pipeline(str(input), str(output) if output else None, quality, lufs, threshold))
# --- image ----------------------------------------------------------------
@image_app.command("health")
@command
def image_health() -> None:
"""Is the Gemini API reachable with GEMINI_API_KEY? Free — lists models."""
from tooling.domains.assets import image
_emit(image.health())
@image_app.command("generate")
@command
def image_generate(
prompt: str = typer.Argument(..., help="What to draw."),
output: Path = typer.Option(None, "--output", help="PNG path (default: ~/Pictures/mcp-images/)."),
aspect: str = typer.Option("1:1", "--aspect", help="Aspect ratio, e.g. 1:1, 16:9, 3:4."),
size: str = typer.Option(None, "--size", help="Resolution hint (1K/2K/4K) — may be ignored."),
input: Path = typer.Option(None, "--input", help="Source image, for image-to-image."),
) -> None:
"""Generate one image. COSTS MONEY per call."""
from tooling.domains.assets import image
if aspect not in image.ASPECT_RATIOS:
raise unknown_choice("aspect ratio", aspect, image.ASPECT_RATIOS)
_emit(
image.generate(
prompt,
output=str(output) if output else None,
aspect_ratio=aspect,
image_size=size,
input_image=str(input) if input else None,
)
)
# --- trellis --------------------------------------------------------------
@trellis_app.command("health")
@command
def trellis_health() -> None:
"""Is Trellis up? OFF is its normal resting state — this says which."""
from tooling.domains.assets import trellis
_emit(trellis.health())
@trellis_app.command("generate")
@command
def trellis_generate(
image: Path = typer.Argument(..., help="Input image (PNG recommended)."),
output: Path = typer.Option(None, "--output", help=".glb path (default: named from the image)."),
simplify: float = typer.Option(0.95, "--simplify", help="Mesh simplification, 0.9-0.98."),
texture_size: int = typer.Option(1024, "--texture-size", help="Texture resolution, 512-2048."),
seed: int = typer.Option(0, "--seed", help="Random seed."),
timeout: int = typer.Option(600, "--timeout", help="Seconds for the 3D step."),
) -> None:
"""Image → .glb; prints the result JSON."""
from tooling.domains.assets import trellis
_emit(
trellis.generate(
str(image),
output=str(output) if output else None,
simplify=simplify,
texture_size=texture_size,
seed=seed,
timeout=timeout,
)
)
@trellis_app.command("batch")
@command
def trellis_batch(
input_dir: Path = typer.Option(None, "--input-dir", help="Directory of .png inputs (default: the character bodies)."),
output_dir: Path = typer.Option(None, "--output-dir", help="Where the .glb files go."),
names: str = typer.Option(None, "--names", help="Comma-separated stems; default: every .png in --input-dir."),
simplify: float = typer.Option(0.95, "--simplify", help="Mesh simplification."),
texture_size: int = typer.Option(1024, "--texture-size", help="Texture resolution."),
cooldown: float = typer.Option(15, "--cooldown", help="Seconds between successful jobs."),
retries: int = typer.Option(3, "--retries", help="Attempts per item."),
retry_delay: float = typer.Option(60, "--retry-delay", help="Seconds before a retry."),
) -> None:
"""One .glb per image, gently — long; consider --detach. Fails if any item failed."""
from tooling.domains.assets import trellis
summary = trellis.batch(
str(input_dir) if input_dir else trellis.BODIES_INPUT,
str(output_dir) if output_dir else trellis.BODIES_OUTPUT,
names=names.split(",") if names else None,
simplify=simplify,
texture_size=texture_size,
cooldown=cooldown,
max_retries=retries,
retry_delay=retry_delay,
)
_emit_summary(summary, "trellis batch")
# --- local synthesis ------------------------------------------------------
@app.command("synth-ui")
@command
def synth_ui(
output_dir: Path = typer.Option(None, "--output-dir", help="Where the WAVs go (default: client/assets/audio)."),
) -> None:
"""Synthesize the four insert-tech UI sounds locally — no service needed."""
from tooling.domains.assets import synth_ui as service
paths = service.run(str(output_dir) if output_dir else service.OUTPUT_DIR)
console.out("\n".join(paths))
console.verdict(
f"synth-ui: wrote {len(paths)} WAVs — convert with `reach assets audio post convert <file.wav>`"
)
+209
View File
@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Synthesize insert-tech UI sounds for Sprint 7 #440.
Each sound uses a DIFFERENT synthesis technique to ensure distinct character:
- cursor_hover: impulse → resonant bandpass (digital click)
- weapon_aim: filtered noise + sub thump (mechanical)
- monologue_chime: FM synthesis (crystalline bell)
- monologue_chime_urgent: FM synthesis + beating/dissonance (tense bell)
Formerly tooling/synth_ui_sounds.py (T-1290). The synthesis is untouched —
the four WAVs are byte-identical before and after the port. The output
directory became a parameter; the default is still client/assets/audio.
"""
import os
import wave
import numpy as np
from tooling.core import config, console
SR = 44100
OUTPUT_DIR = str(config.repo_root() / "client" / "assets" / "audio")
_output_dir = OUTPUT_DIR
def write_wav(filename, samples, channels=1):
"""Write float samples [-1, 1] to 16-bit WAV."""
path = os.path.join(_output_dir, filename)
# Normalize to peak if it exceeds 1.0
peak = np.max(np.abs(samples))
if peak > 1.0:
samples = samples / peak
int_samples = np.clip(samples * 32767, -32767, 32767).astype(np.int16)
with wave.open(path, "w") as f:
f.setnchannels(channels)
f.setsampwidth(2)
f.setframerate(SR)
f.writeframes(int_samples.tobytes())
dur_ms = len(int_samples) / SR * 1000
console.event(f"wrote {path} ({dur_ms:.0f}ms, {channels}ch)")
return path
def simple_lowpass(signal, cutoff_hz, sr=SR):
"""Single-pole IIR low-pass filter."""
rc = 1.0 / (2 * np.pi * cutoff_hz)
dt = 1.0 / sr
alpha = dt / (rc + dt)
out = np.zeros_like(signal)
out[0] = alpha * signal[0]
for i in range(1, len(signal)):
out[i] = out[i - 1] + alpha * (signal[i] - out[i - 1])
return out
def simple_highpass(signal, cutoff_hz, sr=SR):
"""Single-pole IIR high-pass filter."""
rc = 1.0 / (2 * np.pi * cutoff_hz)
dt = 1.0 / sr
alpha = rc / (rc + dt)
out = np.zeros_like(signal)
out[0] = signal[0]
for i in range(1, len(signal)):
out[i] = alpha * (out[i - 1] + signal[i] - signal[i - 1])
return out
def cursor_hover():
"""UI-001: Digital click/tick on entity hover.
Technique: Short noise impulse bandpass-filtered to ~4kHz.
Sounds like a tiny digital snap — no sustained tone at all.
The "click" of a selection appearing on a HUD.
"""
duration = 0.035 # 35ms — shorter than before
n = int(SR * duration)
t = np.linspace(0, duration, n, endpoint=False)
rng = np.random.default_rng(77)
# White noise impulse
impulse = rng.uniform(-1, 1, n)
# Bandpass around 4kHz: high-pass at 3kHz, low-pass at 5kHz
filtered = simple_highpass(impulse, 3000)
filtered = simple_lowpass(filtered, 5500)
# Very fast decay — done in 25ms
envelope = np.exp(-t * 140)
return write_wav("cursor_hover.wav", filtered * envelope * 0.3)
def weapon_aim():
"""UI-004: Mechanical latch for weapon aim.
Technique: Low-pass filtered noise burst (the clack) + sub-bass
thump (the weight). No musical pitch — this is a MECHANICAL sound.
Think: safety clicking off, bolt sliding home.
"""
duration = 0.15 # 150ms
n = int(SR * duration)
t = np.linspace(0, duration, n, endpoint=False)
rng = np.random.default_rng(42)
# Component 1: Low-pass noise burst — the metallic clack
noise = rng.uniform(-1, 1, n)
# Low-pass at 1.5kHz — dull, heavy impact, not bright
clack = simple_lowpass(noise, 1500)
clack_env = np.exp(-t * 60) # fast decay
clack = clack * clack_env
# Component 2: Sub-bass thump — weight of the mechanism
# 80Hz sine, very fast decay
thump = np.sin(2 * np.pi * 80 * t)
thump_env = np.exp(-t * 40)
thump = thump * thump_env
# Component 3: High metallic click at the very start (2ms)
click = rng.uniform(-1, 1, n)
click = simple_highpass(click, 4000)
click_env = np.zeros(n)
click_mask = t < 0.003
click_env[click_mask] = np.exp(-t[click_mask] * 800)
click = click * click_env
signal = clack * 0.4 + thump * 0.35 + click * 0.25
return write_wav("weapon_aim.wav", signal * 0.45)
def monologue_chime():
"""UI-005: Crystalline thought-chime. PLACEHOLDER.
Technique: FM synthesis — carrier modulated by a lower frequency
creates rich, evolving harmonics that sound like struck glass or
crystal. Fundamentally different timbre from additive sine waves.
"""
duration = 0.75 # 750ms
n = int(SR * duration)
t = np.linspace(0, duration, n, endpoint=False)
# FM synthesis: carrier at 1200Hz, modulator at 420Hz (ratio ~2.86:1)
# Inharmonic ratio = bell-like quality
f_carrier = 1200
f_mod = 420
# Mod index decays over time — bright attack, mellow sustain
mod_index = 3.0 * np.exp(-t * 6)
# Modulator signal
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
# Carrier with FM
signal = np.sin(2 * np.pi * f_carrier * t + modulator)
# Gentle attack (20ms), slow decay
attack = np.minimum(t / 0.02, 1.0)
decay = np.exp(-t * 3.0)
envelope = attack * decay
return write_wav("sfx_monologue_chime.wav", signal * envelope * 0.22)
def monologue_chime_urgent():
"""UI-006: Urgent thought-chime. PLACEHOLDER.
Technique: FM synthesis with higher mod index (brighter/harsher) +
a second detuned carrier that creates beating/tension. The beating
is what makes it feel "urgent" — not louder, but unsettled.
"""
duration = 0.5 # 500ms — shorter
n = int(SR * duration)
t = np.linspace(0, duration, n, endpoint=False)
f_carrier = 1200
f_mod = 420
# Higher mod index = more sidebands = brighter, more aggressive
mod_index = 5.0 * np.exp(-t * 7)
modulator = mod_index * np.sin(2 * np.pi * f_mod * t)
# Primary carrier
carrier1 = np.sin(2 * np.pi * f_carrier * t + modulator)
# Second carrier, detuned +8Hz — creates beating at 8Hz (nervous flicker)
carrier2 = np.sin(2 * np.pi * (f_carrier + 8) * t + modulator * 0.8)
signal = carrier1 * 0.6 + carrier2 * 0.4
# Sharp attack (6ms), faster decay
attack = np.minimum(t / 0.006, 1.0)
decay = np.exp(-t * 4.0)
envelope = attack * decay
# 15% louder than normal
return write_wav("sfx_monologue_chime_urgent.wav", signal * envelope * 0.25)
def run(output_dir: str = OUTPUT_DIR) -> list[str]:
"""Write all four UI sounds into output_dir; returns their paths."""
global _output_dir
_output_dir = output_dir
os.makedirs(output_dir, exist_ok=True)
console.event("[UI-001] cursor_hover — bandpass noise impulse (digital click)")
paths = [cursor_hover()]
console.event("[UI-004] weapon_aim — filtered noise + sub thump (mechanical latch)")
paths.append(weapon_aim())
console.event("[UI-005] sfx_monologue_chime — FM synthesis bell (PLACEHOLDER)")
paths.append(monologue_chime())
console.event("[UI-006] sfx_monologue_chime_urgent — FM + beating (PLACEHOLDER)")
paths.append(monologue_chime_urgent())
return paths
+315
View File
@@ -0,0 +1,315 @@
"""Trellis 3D model generator connector — Gradio API wrapper.
Talks to the Trellis Gradio app on tower-of-joy :11510 (URL from config.json).
Pipeline: start session → upload + preprocess → seed → image_to_3d →
extract_glb → download .glb.
Formerly tooling/db/trellis_connector.py plus the tooling/trellis-batch.sh loop
(T-1290). `batch` is that loop in Python, and no longer hardcoded to the 16
character bodies — it takes a directory, or explicit names.
Gradio API parameter reference (TRELLIS v1, microsoft/TRELLIS):
/image_to_3d — 9 inputs:
0: image (Image) preprocessed image from /preprocess_image_1
1: multiimages (Gallery) [] for single-image mode
2: is_multiimage (State) False for single-image, True for multi-image
3: seed (Slider) int, 0-2147483647
4: ss_guidance (Slider) float, sparse structure guidance strength (default 7.5)
5: ss_steps (Slider) int, sparse structure sampling steps (default 12)
6: slat_guidance (Slider) float, structured latent guidance strength (default 3.0)
7: slat_steps (Slider) int, structured latent sampling steps (default 12)
8: multiimage_algo (Radio) "stochastic" or "multidiffusion"
/extract_glb — 3 inputs:
0: output_buf (State) None — server uses internal state from image_to_3d
1: simplify (Slider) float, mesh simplification ratio (default 0.95)
2: texture_size (Slider) int, texture resolution (default 1024)
Common failure modes:
- "needed 9, got 8": missing is_multiimage (position 2) — must pass False
- "needed 3, got 2": missing output_buf (position 0) — must pass None
- "'float' cannot be interpreted as int": numpy version issue on server,
Gradio Sliders send all values as float. Fix: patch flow_euler.py on
the server to cast steps to int, or pin numpy < 2.0
- CUDA device mismatch after crash: restart the container to clear GPU state
"""
from __future__ import annotations
import json
import os
import random
import string
import time
import urllib.request
from pathlib import Path
from tooling.core import config, console
from tooling.core.errors import ReachError
from tooling.domains.assets import endpoints
SERVICE = "trellis"
# The batch defaults are the character-body run the bash script was written for.
BODIES_INPUT = ".tmp/image-gen/characters/bodies"
BODIES_OUTPUT = ".tmp/glb-gen/characters/bodies"
def get_base_url() -> str:
return endpoints.base_url(SERVICE)
def health() -> dict:
"""Is the Trellis API reachable, and what endpoints does it name?"""
base = get_base_url()
data = endpoints.call_json(
urllib.request.Request(f"{base}/info", method="GET"),
service=SERVICE,
what="the health check",
timeout=10,
)
return {"ok": True, "url": base, "endpoints": list(data.get("named_endpoints", {}).keys())}
def _call_api(base: str, endpoint: str, data: list, timeout: float = 600, session_hash: str | None = None):
"""POST one Gradio endpoint, sharing a session so gr.State survives between calls.
image_to_3d stores its output in server-side State keyed by session_hash;
extract_glb reads it back. Without a shared session the state is lost.
"""
body: dict = {"data": data}
if session_hash:
body["session_hash"] = session_hash
console.event(f"Calling {endpoint}...")
result = endpoints.call_json(
urllib.request.Request(
f"{base}/api{endpoint}",
data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"},
method="POST",
),
service=SERVICE,
what=endpoint,
timeout=timeout,
)
if isinstance(result, dict) and "data" in result:
return result["data"]
return result
def multipart_body(filename: str, payload: bytes, boundary: str = "----TrellisConnectorBoundary") -> bytes:
"""The multipart/form-data body Gradio's /upload expects — a 'files' field."""
head = (
f"--{boundary}\r\n"
f'Content-Disposition: form-data; name="files"; filename="{filename}"\r\n'
"Content-Type: image/png\r\n"
"\r\n"
).encode()
return head + payload + f"\r\n--{boundary}--\r\n".encode()
def _upload_image(base: str, image_path: str):
filename = os.path.basename(image_path)
with open(image_path, "rb") as f:
body = multipart_body(filename, f.read())
console.event(f"Uploading {filename}...")
result = endpoints.call_json(
urllib.request.Request(
f"{base}/upload",
data=body,
headers={"Content-Type": "multipart/form-data; boundary=----TrellisConnectorBoundary"},
method="POST",
),
service=SERVICE,
what="the image upload",
timeout=30,
)
if isinstance(result, list) and result:
return result[0]
raise ReachError(
f"Trellis accepted the upload but returned no file reference: {result}",
fix="the Gradio app's upload API may have changed — check `reach assets trellis health`",
)
def _download_file(url: str, output_path: str, base: str) -> int:
if url.startswith("/"):
url = f"{base}{url}"
elif not url.startswith("http"):
url = f"{base}/file={url}"
console.event(f"Downloading to {output_path}...")
body = endpoints.call(
urllib.request.Request(url, method="GET"), service=SERVICE, what="the GLB download", timeout=120
)
with open(output_path, "wb") as f:
f.write(body)
return os.path.getsize(output_path)
def glb_url_from(glb_result) -> str | None:
"""extract_glb returns [model_viewer_data, download_button_data]; take the first URL."""
if isinstance(glb_result, list):
for item in glb_result:
if isinstance(item, dict):
url = item.get("url") or item.get("path")
if url:
return url
return None
def generate(
image_path: str,
output: str | None = None,
simplify: float = 0.95,
texture_size: int = 1024,
seed: int = 0,
timeout: int = 600,
) -> dict:
"""Image → .glb. Returns the result dict."""
# The input is checked before the service, so a typo does not read as an
# outage. (The old check also crashed: it passed indent= to print().)
if not os.path.isfile(image_path):
raise ReachError(f"image not found: {image_path}", fix="pass an existing .png")
base = get_base_url()
endpoints.call(
urllib.request.Request(f"{base}/info", method="GET"),
service=SERVICE,
what="the availability check",
timeout=5,
)
start_time = time.time()
output = output or f"{os.path.splitext(os.path.basename(image_path))[0]}.glb"
session = "".join(random.choices(string.ascii_lowercase + string.digits, k=12))
console.event(f"Session: {session}")
console.event("Starting session...", phase="1/5")
_call_api(base, "/start_session", [], timeout=30, session_hash=session)
console.event("Uploading and preprocessing image...", phase="2/5")
uploaded = _upload_image(base, image_path)
file_ref = {"path": uploaded, "meta": {"_type": "gradio.FileData"}}
preprocessed = _call_api(base, "/preprocess_image_1", [file_ref], timeout=60, session_hash=session)
preprocessed_ref = preprocessed[0] if isinstance(preprocessed, list) and preprocessed else preprocessed
console.event("Generating 3D model...", phase="3/5")
seed_result = _call_api(base, "/get_seed", [True, seed], timeout=10, session_hash=session)
actual_seed = seed_result[0] if isinstance(seed_result, list) and seed_result else seed
# Nine inputs, positions per the reference above. The server needs
# flow_euler.py patched to cast steps to int (numpy >= 2).
_call_api(
base,
"/image_to_3d",
[preprocessed_ref, [], False, actual_seed, 7.5, 12, 3.0, 12, "stochastic"],
timeout=timeout,
session_hash=session,
)
console.event("Extracting GLB...", phase="4/5")
glb_result = _call_api(base, "/extract_glb", [None, simplify, texture_size], timeout=120, session_hash=session)
glb_url = glb_url_from(glb_result)
if not glb_url:
raise ReachError(
f"could not find the GLB URL in the extract_glb response: {str(glb_result)[:300]}",
fix="the Gradio app's response shape may have changed — see trellis.glb_url_from",
)
console.event("Downloading GLB...", phase="5/5")
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
size = _download_file(glb_url, output, base)
return {
"ok": True,
"file": output,
"size_bytes": size,
"simplify": simplify,
"texture_size": texture_size,
"seed": actual_seed,
"generation_time_s": round(time.time() - start_time, 1),
"source_image": image_path,
}
def batch_plan(input_dir: Path, output_dir: Path, names: list[str] | None) -> list[tuple[str, Path, Path]]:
"""What a batch would do: (name, input png, output glb) per item — pure."""
if names:
chosen = names
else:
chosen = sorted(p.stem for p in input_dir.glob("*.png"))
return [(n, input_dir / f"{n}.png", output_dir / f"{n}.glb") for n in chosen]
def batch(
input_dir: str = BODIES_INPUT,
output_dir: str = BODIES_OUTPUT,
names: list[str] | None = None,
simplify: float = 0.95,
texture_size: int = 1024,
cooldown: float = 15,
max_retries: int = 3,
retry_delay: float = 60,
) -> dict:
"""Generate one .glb per input image, one at a time, gently.
Skips outputs that already exist. Retries each failure, and cools the GPU
down between successful jobs — the Trellis box is shared (VRAM is scarce).
Returns the summary; the router prints it and then fails the command if
anything failed. The bash original exited 0 regardless.
"""
root = config.repo_root()
in_dir = (root / input_dir) if not Path(input_dir).is_absolute() else Path(input_dir)
out_dir = (root / output_dir) if not Path(output_dir).is_absolute() else Path(output_dir)
plan = batch_plan(in_dir, out_dir, names)
if not plan:
raise ReachError(f"no .png inputs in {in_dir}", fix="pass --input-dir with images, or --names")
out_dir.mkdir(parents=True, exist_ok=True)
total = len(plan)
results = []
console.event(f"Trellis batch: {total} item(s), cooldown {cooldown}s, {max_retries} tries each")
for i, (name, src, dst) in enumerate(plan, 1):
phase = f"{i}/{total}"
if dst.exists():
console.event(f"{name} — already exists, skipping", phase=phase)
results.append({"name": name, "status": "skipped"})
continue
if not src.exists():
console.event(f"{name} — input not found: {src}", phase=phase, level="warn")
results.append({"name": name, "status": "failed", "error": f"input not found: {src}"})
continue
error = None
for attempt in range(1, max_retries + 1):
console.event(f"{name} (attempt {attempt}/{max_retries})...", phase=phase, progress=i / total)
try:
generate(str(src), output=str(dst), simplify=simplify, texture_size=texture_size)
error = None
break
except ReachError as exc:
error = exc.message
console.event(f"{name} failed: {exc.message}", phase=phase, level="warn")
if attempt < max_retries:
console.event(f"waiting {retry_delay}s before retry...", phase=phase)
time.sleep(retry_delay)
if error is None:
results.append({"name": name, "status": "success", "file": str(dst)})
if i < total:
console.event(f"cooling down {cooldown}s...", phase=phase)
time.sleep(cooldown)
else:
results.append({"name": name, "status": "failed", "error": error})
summary = {
"ok": not any(r["status"] == "failed" for r in results),
"total": total,
"success": sum(r["status"] == "success" for r in results),
"skipped": sum(r["status"] == "skipped" for r in results),
"failed": sum(r["status"] == "failed" for r in results),
"results": results,
}
return summary