refactor(tooling): T-1290 — the assets domain, where OFF is the normal case
tooling/db/ (a misnamed directory: connectors, not database work),
trellis-batch.sh and synth_ui_sounds.py become `reach assets`:
audio {health,generate,batch,post {convert,normalize,trim,pipeline}},
image {health,generate}, trellis {health,generate,batch}, and synth-ui.
The four audio bash wrappers are retired, and tooling/db/ is gone.
Parity, from baselines taken before anything moved:
- the four UI-sound WAVs and the harmonic-synth WAVs (exponential and linear
decay) are byte-identical
- the ffmpeg pipeline's decoded PCM is identical. Its .ogg bytes are not,
even between two runs of the OLD code: Ogg picks a random stream serial,
so the encoded file was never the right thing to compare
- the network success paths can't be run in a gate (Stable Audio and Trellis
are kept off, Gemini costs money), so tooling/test_assets.py stands up a
fake Gradio and pins every payload: the audio submit, Trellis's six-call
session sequence with its 9-input image_to_3d, and the Gemini body. It
failed when one Trellis value was mutated (7.5 → 7.0)
Failure classification, in endpoints.py, is the point of the port. The
services are OFF by design (VRAM on tower-of-joy, D-17), and the topology doc
warns against "fixing" one by restarting it. So a refused connection says OFF
and asks for the service to be turned on rather than restarted; a 4xx/5xx says
the request was rejected; 401/403 says credentials; 429 says quota; and an
unreachable Gemini blames the network, not VRAM.
Behaviour changes, each a failure that used to read as success or crash:
- audio batch and trellis batch exited 0 with failures in their summaries;
they now print the summary and exit 1
- trellis generate on a missing image crashed with a TypeError
(print(..., indent=2)); it now names the file, and checks it before the
service so a typo is not reported as an outage
- the ffmpeg pipeline left its intermediates behind when a step failed
Structure: the connectors called each other as subprocesses (batch spawned
the connector, which spawned audio_post) and parsed each other's stdout. They
are now function calls, and ffmpeg is the only exec, through core/process.
ensure_venv() is removed: it os.execv'd into .venv, which D-263's exec rule
forbids, and reach declares the dependencies itself. config.json moved into
the domain deliberately, and the local-services rule follows it.
Output contract: results are still JSON on stdout with the same keys, so skill
readers keep working. Failures are an exit status with a Fix line, never
{"ok": false}. The audio-gen, glb-gen and image-gen skills, Araminta's agent
file and the allow-list are updated to match. glb-gen's "trellis-batch.sh is
hardcoded to one category" caveat is gone: batch takes --input-dir or --names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -113,7 +113,7 @@ Synthesize findings.
|
||||
|
||||
### Araminta (Visual Designer)
|
||||
- Joins discussions only when visual consistency decisions are needed
|
||||
- Drives image-gen/sprite-gen/glb-gen/audio-gen via Bash — no Skill/MCP tool grant, so she invokes the connector scripts directly, e.g. `python3 tooling/db/image_connector.py generate ...`
|
||||
- Drives image-gen/sprite-gen/glb-gen/audio-gen via Bash — no Skill/MCP tool grant, so she invokes the connectors directly, e.g. `reach assets image generate ...`
|
||||
- **image-gen calls the paid Gemini API (`GEMINI_API_KEY`) — always ask Team Leader for permission before generating. glb-gen/audio-gen run against self-hosted tower-of-joy infrastructure and sprite-gen renders locally, so they don't carry the same per-call cost, but confirm intent before large batch jobs.**
|
||||
|
||||
### SI (Refinement Manager)
|
||||
|
||||
@@ -48,7 +48,7 @@ PBR assets from these sources go through our `toon_masked` shader and come out m
|
||||
|
||||
You have `Bash` but no `Skill`/MCP tool grant, so asset generation runs through the project's connector scripts directly, not a slash-skill invocation:
|
||||
|
||||
- **image-gen** (`.claude/skills/image-gen/`) — concept art, icons, UI mockups, reference images via the Gemini API: `python3 tooling/db/image_connector.py generate "prompt" --output .tmp/image-gen/[category]/[name].png --aspect 1:1`. Requires `GEMINI_API_KEY` (set in `.claude/settings.local.json`) — this is the one that costs real money per call.
|
||||
- **image-gen** (`.claude/skills/image-gen/`) — concept art, icons, UI mockups, reference images via the Gemini API: `reach assets image generate "prompt" --output .tmp/image-gen/[category]/[name].png --aspect 1:1`. Requires `GEMINI_API_KEY` (set in `.claude/settings.local.json`) — this is the one that costs real money per call.
|
||||
- **sprite-gen** (`.claude/skills/sprite-gen/`) — flat 2D artwork (paintings, flags, billboards, signage) rendered as PNG textures/decals, via `scripts/render.sh`.
|
||||
- **glb-gen** (`.claude/skills/glb-gen/`) — converts a concept PNG to a game-ready `.glb` via Trellis (self-hosted on tower-of-joy) plus Blender post-processing.
|
||||
- **audio-gen** (`.claude/skills/audio-gen/`) — ambient loops, SFX, and UI sounds via the self-hosted Stable Audio Open Gradio app (tower-of-joy).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Local Services
|
||||
|
||||
Endpoints are also preconfigured in `tooling/db/config.json`.
|
||||
Endpoints are also preconfigured in `tooling/domains/assets/config.json` (`reach assets …`).
|
||||
|
||||
- **Gitea:** `https://git.schweitz.net` (login: `schweitz`; LAN-hairpinned via
|
||||
AdGuard — resolves to tower-of-joy directly on the home network). The old
|
||||
|
||||
@@ -59,12 +59,20 @@ docs/
|
||||
workshops/ # Workshop briefs and outputs
|
||||
db/
|
||||
schema.sql # Database schema
|
||||
tooling/
|
||||
db/ # Asset/connector scripts (audio, image, trellis, wiki)
|
||||
config.json # Endpoint configuration
|
||||
common.py # Shared venv/config helpers
|
||||
audio_connector.py # Stable Audio Open connector
|
||||
tooling/ # ONE package behind the `reach` CLI (D-263) — `reach --help`
|
||||
main.py # routing only; the domain registry
|
||||
core/ # config, console (the single output path), errors,
|
||||
# process (the single guarded exec), jobs, command
|
||||
domains/<name>/ # router.py (transport) + service/helper modules (logic)
|
||||
atlas/ planet/ # the spatial ladder; `atlas planet` is its rung 3
|
||||
ledger/ # economics import — the sole systems.db generator
|
||||
wiki/ # wiki fill rates + GTTR hook
|
||||
assets/ # Stable Audio / Gemini / Trellis connectors
|
||||
config.json # endpoint URLs (never keys — tracked file)
|
||||
… # check, validate, godot, visual, generate, blender, pr, jobs, dev
|
||||
scripts/blender/ # Blender payloads — run by Blender's Python, never imported
|
||||
archive/ # Provenance only, never run: pql-migrate/, wiki-bootstrap/
|
||||
test_*.py # gate tests, run by `make test-tooling`
|
||||
.claude/
|
||||
agents/ # Agent personality files
|
||||
skills/ # Skill definitions
|
||||
|
||||
@@ -24,10 +24,6 @@
|
||||
"Bash(git rm *)",
|
||||
"Bash(git ls-tree *)",
|
||||
"Bash(git rev-parse --show-toplevel)",
|
||||
"Bash(tooling/db/audio-generate *)",
|
||||
"Bash(tooling/db/audio-health)",
|
||||
"Bash(tooling/db/audio-post *)",
|
||||
"Bash(tooling/db/audio-batch *)",
|
||||
"Bash(make *)",
|
||||
"Bash(make)",
|
||||
"Bash(pql)",
|
||||
|
||||
@@ -13,39 +13,44 @@ description: >
|
||||
# Audio Generation — The Settled Reach
|
||||
|
||||
Generate sonically consistent audio assets using the Stable Audio Open API via
|
||||
wrapper scripts at `tooling/db/audio-*`.
|
||||
`reach assets audio …` (`tooling/domains/assets/`, T-1290).
|
||||
|
||||
Asset descriptions, filenames, bus routing, and design intent are documented in
|
||||
`docs/assets/audio/`. This skill provides the prompt system, generation
|
||||
workflow, and quality validation.
|
||||
|
||||
`audio-health`, `audio-generate`, and `audio-batch` re-exec into the project
|
||||
`.venv` on startup (`audio-post` doesn't need to — it only shells out to
|
||||
ffmpeg). On a fresh clone with no `.venv` yet, they fail fast with `error:
|
||||
.venv not found — run make setup-venv first.` — run that once before using
|
||||
this skill.
|
||||
**Stable Audio is kept switched OFF** on tower-of-joy — VRAM is scarce, so
|
||||
the service is turned on only when needed, and restarting it blindly takes VRAM
|
||||
from whatever else is running. `reach assets audio health` tells you which:
|
||||
OFF exits 1 with a remedy that says so. Get it turned on before a session.
|
||||
|
||||
**Read the exit status, not an `ok` field.** Each verb prints its result as
|
||||
JSON on stdout (`file`, `size_bytes`, `ogg_file`, … — the same keys as before)
|
||||
and fails with a non-zero exit and a `Fix:` line. There is no `{"ok": false}`
|
||||
on stdout any more, and a batch with any failed asset now exits 1.
|
||||
|
||||
## API Access
|
||||
|
||||
**Never call the API directly.** Use the wrapper scripts:
|
||||
**Never call the API directly.** Use the reach verbs:
|
||||
|
||||
```bash
|
||||
# Check API health
|
||||
tooling/db/audio-health
|
||||
# Check API health (OFF is the normal resting state)
|
||||
reach assets audio health
|
||||
|
||||
# Generate a single asset (WAV only)
|
||||
tooling/db/audio-generate "prompt text" \
|
||||
reach assets audio generate "prompt text" \
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/output.wav
|
||||
|
||||
# Generate + post-process in one command (WAV → trim → normalize → OGG)
|
||||
tooling/db/audio-generate "prompt text" \
|
||||
reach assets audio generate "prompt text" \
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/gen/intermediate.wav \
|
||||
--output-ogg client/assets/audio/final.ogg
|
||||
|
||||
# Batch-generate from a manifest (preferred for multiple assets)
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json
|
||||
# Batch-generate from a manifest (preferred for multiple assets) — long:
|
||||
# add --detach and follow it with `reach jobs log <id>`
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json
|
||||
```
|
||||
|
||||
### Parameters
|
||||
@@ -142,9 +147,9 @@ Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g.,
|
||||
AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory.
|
||||
|
||||
**`lufs`/`quality` defaults apply to `synth` assets only.** `audio_batch.py`'s
|
||||
SAO path (`run_sao_generate`) only forwards `steps`/`cfg`/`timeout` to the
|
||||
connector — `--lufs`/`--quality` aren't even exposed as CLI flags on
|
||||
`audio_connector.py generate`, so a manifest's `defaults.lufs`/`defaults.quality`
|
||||
SAO path (`run_sao_generate`) only forwards `steps`/`cfg`/`timeout` to
|
||||
`audio.generate` — `--lufs`/`--quality` aren't exposed on
|
||||
`reach assets audio generate` either, so a manifest's `defaults.lufs`/`defaults.quality`
|
||||
are silently ignored for `method: "sao"` assets. SAO post-processing is fixed
|
||||
at -16 LUFS / quality 6 regardless of what the manifest says.
|
||||
|
||||
@@ -152,16 +157,16 @@ at -16 LUFS / quality 6 regardless of what the manifest says.
|
||||
|
||||
```bash
|
||||
# Full run
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json
|
||||
|
||||
# Dry run — preview what would be generated
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --dry-run
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --dry-run
|
||||
|
||||
# Generate only specific assets
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --only AMB-001,AMB-002
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --only AMB-001,AMB-002
|
||||
|
||||
# Skip assets that already have OGG files
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --skip-existing
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --skip-existing
|
||||
```
|
||||
|
||||
### 3. Update asset docs with prompts
|
||||
@@ -204,8 +209,8 @@ For one-off generation or iteration on a specific asset:
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `tooling/db/audio-health` to verify the API is up.
|
||||
6. Run `tooling/db/audio-generate` with `--post` or `--output-ogg` to
|
||||
5. Run `reach assets audio health` to verify the API is up.
|
||||
6. Run `reach assets audio generate` with `--post` or `--output-ogg` to
|
||||
generate and post-process in one step.
|
||||
7. Verify the output (file size, duration).
|
||||
8. Update the asset status and prompt in `docs/assets/audio/{category}.md`.
|
||||
@@ -232,23 +237,23 @@ If you need to post-process separately (e.g., re-normalizing an existing file):
|
||||
|
||||
```bash
|
||||
# Full pipeline: trim → normalize → convert
|
||||
tooling/db/audio-post pipeline input.wav --output output.ogg
|
||||
reach assets audio post pipeline input.wav --output output.ogg
|
||||
|
||||
# Individual steps
|
||||
tooling/db/audio-post trim input.wav
|
||||
tooling/db/audio-post normalize input.wav --lufs -16
|
||||
tooling/db/audio-post convert input.wav --output output.ogg
|
||||
reach assets audio post trim input.wav
|
||||
reach assets audio post normalize input.wav --lufs -16
|
||||
reach assets audio post convert input.wav --output output.ogg
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py`
|
||||
produce meaningful output. Use manual synthesis via `reach assets synth-ui`
|
||||
or the batch manifest's `method: "synth"` with harmonic parameters.
|
||||
|
||||
For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass
|
||||
impulse), write a custom script in `tooling/` following the pattern in
|
||||
`tooling/synth_ui_sounds.py`.
|
||||
impulse), add a function beside the four in
|
||||
`tooling/domains/assets/synth_ui.py` and call it from its `run()`.
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
|
||||
@@ -17,15 +17,19 @@ Convert approved concept images to game-ready .glb models.
|
||||
|
||||
| Service | Check |
|
||||
|---------|-------|
|
||||
| Trellis | `tooling/db/trellis_connector.py health` |
|
||||
| Trellis | `reach assets trellis health` |
|
||||
| Blender | `reach blender which` |
|
||||
|
||||
Trellis runs on tower-of-joy and may be switched off. Check before batching.
|
||||
Trellis runs on tower-of-joy and is kept **switched off** to save VRAM — that is
|
||||
its normal state, and `health` exits 1 saying so. Get it turned on before a
|
||||
session; do not restart it blindly (that takes VRAM from whatever is running).
|
||||
Each verb prints its result JSON on stdout and fails with a non-zero exit —
|
||||
check the exit status, not an `ok` field.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 tooling/db/trellis_connector.py generate input.png \
|
||||
reach assets trellis generate input.png \
|
||||
--output .tmp/glb-gen/[name].glb \
|
||||
--simplify 0.95 \
|
||||
--texture-size 1024
|
||||
@@ -41,26 +45,27 @@ python3 tooling/db/trellis_connector.py generate input.png \
|
||||
|
||||
## Batch Usage
|
||||
|
||||
`tooling/trellis-batch.sh` exists but is **hardcoded to one category**: it
|
||||
takes no arguments and always processes the 16 character body types from
|
||||
`.tmp/image-gen/characters/bodies` into `.tmp/glb-gen/characters/bodies`.
|
||||
Running it for any other asset category (furniture, props, etc.) does
|
||||
nothing useful — it will just re-run (or skip, if outputs already exist) the
|
||||
same 16 character bodies regardless of what you intended.
|
||||
|
||||
For any other category, loop `trellis_connector.py` calls yourself with a
|
||||
cooldown between jobs:
|
||||
`reach assets trellis batch` runs a whole directory, one job at a time, with a
|
||||
cooldown between jobs and retries on failure. (It replaced `trellis-batch.sh`,
|
||||
which was hardcoded to the 16 character bodies — T-1290.) It skips outputs that
|
||||
already exist and exits 1 if any item failed. It is long; add `--detach`.
|
||||
|
||||
```bash
|
||||
for f in .tmp/image-gen/furniture/tables/*.png; do
|
||||
name=$(basename "$f" .png)
|
||||
python3 tooling/db/trellis_connector.py generate "$f" \
|
||||
--output ".tmp/glb-gen/furniture/tables/${name}.glb" \
|
||||
--simplify 0.95 --texture-size 1024
|
||||
sleep 15
|
||||
done
|
||||
# every .png in a directory
|
||||
reach --detach assets trellis batch \
|
||||
--input-dir .tmp/image-gen/furniture/tables \
|
||||
--output-dir .tmp/glb-gen/furniture/tables
|
||||
|
||||
# only some of them
|
||||
reach assets trellis batch --input-dir <dir> --output-dir <dir> --names table_a,table_b
|
||||
|
||||
# no --input-dir: the character bodies, as the bash script did
|
||||
reach assets trellis batch
|
||||
```
|
||||
|
||||
`--cooldown` (15 s), `--retries` (3) and `--retry-delay` (60 s) keep the
|
||||
original script's pacing.
|
||||
|
||||
**Never run Trellis jobs in parallel** — it uses the full GPU and concurrent
|
||||
jobs will OOM and corrupt the CUDA state.
|
||||
|
||||
@@ -143,7 +148,7 @@ For best Trellis results:
|
||||
- **Do NOT force isometric angle** — Trellis reconstructs full 3D, the game camera handles the view
|
||||
- To maintain consistency across a batch, generate the concept images with
|
||||
`/image-gen` using its `--input` style anchor flag — `--input` is an
|
||||
/image-gen flag, not a Trellis one; `trellis_connector.py` has no `--input`
|
||||
/image-gen flag, not a Trellis one; `reach assets trellis generate` has no `--input`
|
||||
argument and exits on unrecognized flags.
|
||||
|
||||
These match `/image-gen` output with the Settled Reach style guide.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Trellis API Reference
|
||||
|
||||
The connector (`tooling/db/trellis_connector.py`) talks to a Gradio API. The
|
||||
The connector (`tooling/domains/assets/trellis.py`, `reach assets trellis`) talks to a Gradio API. The
|
||||
parameter layout is fragile — document changes here when the container is
|
||||
updated. This is the canonical copy; the connector's module docstring carries
|
||||
a duplicate for at-a-glance reference when reading the script directly — keep
|
||||
|
||||
@@ -19,18 +19,21 @@ not something this skill triggers itself. Output is a PNG file.
|
||||
|
||||
Requires `GEMINI_API_KEY` in environment (set in `.claude/settings.local.json`).
|
||||
|
||||
Uses the canonical connector at `tooling/db/image_connector.py` (see
|
||||
`.claude/rules/project-structure.md` — `tooling/db/` is the documented home
|
||||
for asset/connector scripts; this skill has no local fork of it).
|
||||
Uses the canonical connector behind `reach assets image`
|
||||
(`tooling/domains/assets/image.py`, T-1290; this skill has no local fork of it).
|
||||
`health` only lists models and is free; **every `generate` costs money**.
|
||||
A verb prints its result JSON on stdout and fails with a non-zero exit — check
|
||||
the exit status, not an `ok` field. An invalid `--aspect` is rejected up front
|
||||
with the accepted list.
|
||||
|
||||
```bash
|
||||
python3 tooling/db/image_connector.py health
|
||||
reach assets image health
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 tooling/db/image_connector.py generate \
|
||||
reach assets image generate \
|
||||
"prompt text" \
|
||||
--output path/to/output.png \
|
||||
--aspect 1:1
|
||||
|
||||
Reference in New Issue
Block a user