refactor(tooling): T-1290 — the assets domain, where OFF is the normal case
tooling/db/ (a misnamed directory: connectors, not database work),
trellis-batch.sh and synth_ui_sounds.py become `reach assets`:
audio {health,generate,batch,post {convert,normalize,trim,pipeline}},
image {health,generate}, trellis {health,generate,batch}, and synth-ui.
The four audio bash wrappers are retired, and tooling/db/ is gone.
Parity, from baselines taken before anything moved:
- the four UI-sound WAVs and the harmonic-synth WAVs (exponential and linear
decay) are byte-identical
- the ffmpeg pipeline's decoded PCM is identical. Its .ogg bytes are not,
even between two runs of the OLD code: Ogg picks a random stream serial,
so the encoded file was never the right thing to compare
- the network success paths can't be run in a gate (Stable Audio and Trellis
are kept off, Gemini costs money), so tooling/test_assets.py stands up a
fake Gradio and pins every payload: the audio submit, Trellis's six-call
session sequence with its 9-input image_to_3d, and the Gemini body. It
failed when one Trellis value was mutated (7.5 → 7.0)
Failure classification, in endpoints.py, is the point of the port. The
services are OFF by design (VRAM on tower-of-joy, D-17), and the topology doc
warns against "fixing" one by restarting it. So a refused connection says OFF
and asks for the service to be turned on rather than restarted; a 4xx/5xx says
the request was rejected; 401/403 says credentials; 429 says quota; and an
unreachable Gemini blames the network, not VRAM.
Behaviour changes, each a failure that used to read as success or crash:
- audio batch and trellis batch exited 0 with failures in their summaries;
they now print the summary and exit 1
- trellis generate on a missing image crashed with a TypeError
(print(..., indent=2)); it now names the file, and checks it before the
service so a typo is not reported as an outage
- the ffmpeg pipeline left its intermediates behind when a step failed
Structure: the connectors called each other as subprocesses (batch spawned
the connector, which spawned audio_post) and parsed each other's stdout. They
are now function calls, and ffmpeg is the only exec, through core/process.
ensure_venv() is removed: it os.execv'd into .venv, which D-263's exec rule
forbids, and reach declares the dependencies itself. config.json moved into
the domain deliberately, and the local-services rule follows it.
Output contract: results are still JSON on stdout with the same keys, so skill
readers keep working. Failures are an exit status with a Fix line, never
{"ok": false}. The audio-gen, glb-gen and image-gen skills, Araminta's agent
file and the allow-list are updated to match. glb-gen's "trellis-batch.sh is
hardcoded to one category" caveat is gone: batch takes --input-dir or --names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -113,7 +113,7 @@ Synthesize findings.
|
||||
|
||||
### Araminta (Visual Designer)
|
||||
- Joins discussions only when visual consistency decisions are needed
|
||||
- Drives image-gen/sprite-gen/glb-gen/audio-gen via Bash — no Skill/MCP tool grant, so she invokes the connector scripts directly, e.g. `python3 tooling/db/image_connector.py generate ...`
|
||||
- Drives image-gen/sprite-gen/glb-gen/audio-gen via Bash — no Skill/MCP tool grant, so she invokes the connectors directly, e.g. `reach assets image generate ...`
|
||||
- **image-gen calls the paid Gemini API (`GEMINI_API_KEY`) — always ask Team Leader for permission before generating. glb-gen/audio-gen run against self-hosted tower-of-joy infrastructure and sprite-gen renders locally, so they don't carry the same per-call cost, but confirm intent before large batch jobs.**
|
||||
|
||||
### SI (Refinement Manager)
|
||||
|
||||
@@ -48,7 +48,7 @@ PBR assets from these sources go through our `toon_masked` shader and come out m
|
||||
|
||||
You have `Bash` but no `Skill`/MCP tool grant, so asset generation runs through the project's connector scripts directly, not a slash-skill invocation:
|
||||
|
||||
- **image-gen** (`.claude/skills/image-gen/`) — concept art, icons, UI mockups, reference images via the Gemini API: `python3 tooling/db/image_connector.py generate "prompt" --output .tmp/image-gen/[category]/[name].png --aspect 1:1`. Requires `GEMINI_API_KEY` (set in `.claude/settings.local.json`) — this is the one that costs real money per call.
|
||||
- **image-gen** (`.claude/skills/image-gen/`) — concept art, icons, UI mockups, reference images via the Gemini API: `reach assets image generate "prompt" --output .tmp/image-gen/[category]/[name].png --aspect 1:1`. Requires `GEMINI_API_KEY` (set in `.claude/settings.local.json`) — this is the one that costs real money per call.
|
||||
- **sprite-gen** (`.claude/skills/sprite-gen/`) — flat 2D artwork (paintings, flags, billboards, signage) rendered as PNG textures/decals, via `scripts/render.sh`.
|
||||
- **glb-gen** (`.claude/skills/glb-gen/`) — converts a concept PNG to a game-ready `.glb` via Trellis (self-hosted on tower-of-joy) plus Blender post-processing.
|
||||
- **audio-gen** (`.claude/skills/audio-gen/`) — ambient loops, SFX, and UI sounds via the self-hosted Stable Audio Open Gradio app (tower-of-joy).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Local Services
|
||||
|
||||
Endpoints are also preconfigured in `tooling/db/config.json`.
|
||||
Endpoints are also preconfigured in `tooling/domains/assets/config.json` (`reach assets …`).
|
||||
|
||||
- **Gitea:** `https://git.schweitz.net` (login: `schweitz`; LAN-hairpinned via
|
||||
AdGuard — resolves to tower-of-joy directly on the home network). The old
|
||||
|
||||
@@ -59,12 +59,20 @@ docs/
|
||||
workshops/ # Workshop briefs and outputs
|
||||
db/
|
||||
schema.sql # Database schema
|
||||
tooling/
|
||||
db/ # Asset/connector scripts (audio, image, trellis, wiki)
|
||||
config.json # Endpoint configuration
|
||||
common.py # Shared venv/config helpers
|
||||
audio_connector.py # Stable Audio Open connector
|
||||
tooling/ # ONE package behind the `reach` CLI (D-263) — `reach --help`
|
||||
main.py # routing only; the domain registry
|
||||
core/ # config, console (the single output path), errors,
|
||||
# process (the single guarded exec), jobs, command
|
||||
domains/<name>/ # router.py (transport) + service/helper modules (logic)
|
||||
atlas/ planet/ # the spatial ladder; `atlas planet` is its rung 3
|
||||
ledger/ # economics import — the sole systems.db generator
|
||||
wiki/ # wiki fill rates + GTTR hook
|
||||
assets/ # Stable Audio / Gemini / Trellis connectors
|
||||
config.json # endpoint URLs (never keys — tracked file)
|
||||
… # check, validate, godot, visual, generate, blender, pr, jobs, dev
|
||||
scripts/blender/ # Blender payloads — run by Blender's Python, never imported
|
||||
archive/ # Provenance only, never run: pql-migrate/, wiki-bootstrap/
|
||||
test_*.py # gate tests, run by `make test-tooling`
|
||||
.claude/
|
||||
agents/ # Agent personality files
|
||||
skills/ # Skill definitions
|
||||
|
||||
@@ -24,10 +24,6 @@
|
||||
"Bash(git rm *)",
|
||||
"Bash(git ls-tree *)",
|
||||
"Bash(git rev-parse --show-toplevel)",
|
||||
"Bash(tooling/db/audio-generate *)",
|
||||
"Bash(tooling/db/audio-health)",
|
||||
"Bash(tooling/db/audio-post *)",
|
||||
"Bash(tooling/db/audio-batch *)",
|
||||
"Bash(make *)",
|
||||
"Bash(make)",
|
||||
"Bash(pql)",
|
||||
|
||||
@@ -13,39 +13,44 @@ description: >
|
||||
# Audio Generation — The Settled Reach
|
||||
|
||||
Generate sonically consistent audio assets using the Stable Audio Open API via
|
||||
wrapper scripts at `tooling/db/audio-*`.
|
||||
`reach assets audio …` (`tooling/domains/assets/`, T-1290).
|
||||
|
||||
Asset descriptions, filenames, bus routing, and design intent are documented in
|
||||
`docs/assets/audio/`. This skill provides the prompt system, generation
|
||||
workflow, and quality validation.
|
||||
|
||||
`audio-health`, `audio-generate`, and `audio-batch` re-exec into the project
|
||||
`.venv` on startup (`audio-post` doesn't need to — it only shells out to
|
||||
ffmpeg). On a fresh clone with no `.venv` yet, they fail fast with `error:
|
||||
.venv not found — run make setup-venv first.` — run that once before using
|
||||
this skill.
|
||||
**Stable Audio is kept switched OFF** on tower-of-joy — VRAM is scarce, so
|
||||
the service is turned on only when needed, and restarting it blindly takes VRAM
|
||||
from whatever else is running. `reach assets audio health` tells you which:
|
||||
OFF exits 1 with a remedy that says so. Get it turned on before a session.
|
||||
|
||||
**Read the exit status, not an `ok` field.** Each verb prints its result as
|
||||
JSON on stdout (`file`, `size_bytes`, `ogg_file`, … — the same keys as before)
|
||||
and fails with a non-zero exit and a `Fix:` line. There is no `{"ok": false}`
|
||||
on stdout any more, and a batch with any failed asset now exits 1.
|
||||
|
||||
## API Access
|
||||
|
||||
**Never call the API directly.** Use the wrapper scripts:
|
||||
**Never call the API directly.** Use the reach verbs:
|
||||
|
||||
```bash
|
||||
# Check API health
|
||||
tooling/db/audio-health
|
||||
# Check API health (OFF is the normal resting state)
|
||||
reach assets audio health
|
||||
|
||||
# Generate a single asset (WAV only)
|
||||
tooling/db/audio-generate "prompt text" \
|
||||
reach assets audio generate "prompt text" \
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/output.wav
|
||||
|
||||
# Generate + post-process in one command (WAV → trim → normalize → OGG)
|
||||
tooling/db/audio-generate "prompt text" \
|
||||
reach assets audio generate "prompt text" \
|
||||
--duration 10 --steps 100 --cfg 7 \
|
||||
--output path/to/gen/intermediate.wav \
|
||||
--output-ogg client/assets/audio/final.ogg
|
||||
|
||||
# Batch-generate from a manifest (preferred for multiple assets)
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json
|
||||
# Batch-generate from a manifest (preferred for multiple assets) — long:
|
||||
# add --detach and follow it with `reach jobs log <id>`
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json
|
||||
```
|
||||
|
||||
### Parameters
|
||||
@@ -142,9 +147,9 @@ Asset `id` values must match IDs in `docs/assets/audio/{category}.md` (e.g.,
|
||||
AMB-001, SFX-002, UI-005). This couples the manifest to the asset inventory.
|
||||
|
||||
**`lufs`/`quality` defaults apply to `synth` assets only.** `audio_batch.py`'s
|
||||
SAO path (`run_sao_generate`) only forwards `steps`/`cfg`/`timeout` to the
|
||||
connector — `--lufs`/`--quality` aren't even exposed as CLI flags on
|
||||
`audio_connector.py generate`, so a manifest's `defaults.lufs`/`defaults.quality`
|
||||
SAO path (`run_sao_generate`) only forwards `steps`/`cfg`/`timeout` to
|
||||
`audio.generate` — `--lufs`/`--quality` aren't exposed on
|
||||
`reach assets audio generate` either, so a manifest's `defaults.lufs`/`defaults.quality`
|
||||
are silently ignored for `method: "sao"` assets. SAO post-processing is fixed
|
||||
at -16 LUFS / quality 6 regardless of what the manifest says.
|
||||
|
||||
@@ -152,16 +157,16 @@ at -16 LUFS / quality 6 regardless of what the manifest says.
|
||||
|
||||
```bash
|
||||
# Full run
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json
|
||||
|
||||
# Dry run — preview what would be generated
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --dry-run
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --dry-run
|
||||
|
||||
# Generate only specific assets
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --only AMB-001,AMB-002
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --only AMB-001,AMB-002
|
||||
|
||||
# Skip assets that already have OGG files
|
||||
tooling/db/audio-batch docs/assets/audio/<manifest>.json --skip-existing
|
||||
reach assets audio batch docs/assets/audio/<manifest>.json --skip-existing
|
||||
```
|
||||
|
||||
### 3. Update asset docs with prompts
|
||||
@@ -204,8 +209,8 @@ For one-off generation or iteration on a specific asset:
|
||||
2. Read `references/sonic-palette.md` for the sonic family prefix.
|
||||
3. Read `references/category-templates.md` for the matching template.
|
||||
4. Assemble the full prompt.
|
||||
5. Run `tooling/db/audio-health` to verify the API is up.
|
||||
6. Run `tooling/db/audio-generate` with `--post` or `--output-ogg` to
|
||||
5. Run `reach assets audio health` to verify the API is up.
|
||||
6. Run `reach assets audio generate` with `--post` or `--output-ogg` to
|
||||
generate and post-process in one step.
|
||||
7. Verify the output (file size, duration).
|
||||
8. Update the asset status and prompt in `docs/assets/audio/{category}.md`.
|
||||
@@ -232,23 +237,23 @@ If you need to post-process separately (e.g., re-normalizing an existing file):
|
||||
|
||||
```bash
|
||||
# Full pipeline: trim → normalize → convert
|
||||
tooling/db/audio-post pipeline input.wav --output output.ogg
|
||||
reach assets audio post pipeline input.wav --output output.ogg
|
||||
|
||||
# Individual steps
|
||||
tooling/db/audio-post trim input.wav
|
||||
tooling/db/audio-post normalize input.wav --lufs -16
|
||||
tooling/db/audio-post convert input.wav --output output.ogg
|
||||
reach assets audio post trim input.wav
|
||||
reach assets audio post normalize input.wav --lufs -16
|
||||
reach assets audio post convert input.wav --output output.ogg
|
||||
```
|
||||
|
||||
## Manual Synthesis (Insert-Tech Sounds)
|
||||
|
||||
For sounds under 200ms (cursor hover, weapon aim), Stable Audio Open cannot
|
||||
produce meaningful output. Use manual synthesis via `tooling/synth_ui_sounds.py`
|
||||
produce meaningful output. Use manual synthesis via `reach assets synth-ui`
|
||||
or the batch manifest's `method: "synth"` with harmonic parameters.
|
||||
|
||||
For complex synthesis beyond the `harmonic` type (FM, filtered noise, bandpass
|
||||
impulse), write a custom script in `tooling/` following the pattern in
|
||||
`tooling/synth_ui_sounds.py`.
|
||||
impulse), add a function beside the four in
|
||||
`tooling/domains/assets/synth_ui.py` and call it from its `run()`.
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
|
||||
@@ -17,15 +17,19 @@ Convert approved concept images to game-ready .glb models.
|
||||
|
||||
| Service | Check |
|
||||
|---------|-------|
|
||||
| Trellis | `tooling/db/trellis_connector.py health` |
|
||||
| Trellis | `reach assets trellis health` |
|
||||
| Blender | `reach blender which` |
|
||||
|
||||
Trellis runs on tower-of-joy and may be switched off. Check before batching.
|
||||
Trellis runs on tower-of-joy and is kept **switched off** to save VRAM — that is
|
||||
its normal state, and `health` exits 1 saying so. Get it turned on before a
|
||||
session; do not restart it blindly (that takes VRAM from whatever is running).
|
||||
Each verb prints its result JSON on stdout and fails with a non-zero exit —
|
||||
check the exit status, not an `ok` field.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 tooling/db/trellis_connector.py generate input.png \
|
||||
reach assets trellis generate input.png \
|
||||
--output .tmp/glb-gen/[name].glb \
|
||||
--simplify 0.95 \
|
||||
--texture-size 1024
|
||||
@@ -41,26 +45,27 @@ python3 tooling/db/trellis_connector.py generate input.png \
|
||||
|
||||
## Batch Usage
|
||||
|
||||
`tooling/trellis-batch.sh` exists but is **hardcoded to one category**: it
|
||||
takes no arguments and always processes the 16 character body types from
|
||||
`.tmp/image-gen/characters/bodies` into `.tmp/glb-gen/characters/bodies`.
|
||||
Running it for any other asset category (furniture, props, etc.) does
|
||||
nothing useful — it will just re-run (or skip, if outputs already exist) the
|
||||
same 16 character bodies regardless of what you intended.
|
||||
|
||||
For any other category, loop `trellis_connector.py` calls yourself with a
|
||||
cooldown between jobs:
|
||||
`reach assets trellis batch` runs a whole directory, one job at a time, with a
|
||||
cooldown between jobs and retries on failure. (It replaced `trellis-batch.sh`,
|
||||
which was hardcoded to the 16 character bodies — T-1290.) It skips outputs that
|
||||
already exist and exits 1 if any item failed. It is long; add `--detach`.
|
||||
|
||||
```bash
|
||||
for f in .tmp/image-gen/furniture/tables/*.png; do
|
||||
name=$(basename "$f" .png)
|
||||
python3 tooling/db/trellis_connector.py generate "$f" \
|
||||
--output ".tmp/glb-gen/furniture/tables/${name}.glb" \
|
||||
--simplify 0.95 --texture-size 1024
|
||||
sleep 15
|
||||
done
|
||||
# every .png in a directory
|
||||
reach --detach assets trellis batch \
|
||||
--input-dir .tmp/image-gen/furniture/tables \
|
||||
--output-dir .tmp/glb-gen/furniture/tables
|
||||
|
||||
# only some of them
|
||||
reach assets trellis batch --input-dir <dir> --output-dir <dir> --names table_a,table_b
|
||||
|
||||
# no --input-dir: the character bodies, as the bash script did
|
||||
reach assets trellis batch
|
||||
```
|
||||
|
||||
`--cooldown` (15 s), `--retries` (3) and `--retry-delay` (60 s) keep the
|
||||
original script's pacing.
|
||||
|
||||
**Never run Trellis jobs in parallel** — it uses the full GPU and concurrent
|
||||
jobs will OOM and corrupt the CUDA state.
|
||||
|
||||
@@ -143,7 +148,7 @@ For best Trellis results:
|
||||
- **Do NOT force isometric angle** — Trellis reconstructs full 3D, the game camera handles the view
|
||||
- To maintain consistency across a batch, generate the concept images with
|
||||
`/image-gen` using its `--input` style anchor flag — `--input` is an
|
||||
/image-gen flag, not a Trellis one; `trellis_connector.py` has no `--input`
|
||||
/image-gen flag, not a Trellis one; `reach assets trellis generate` has no `--input`
|
||||
argument and exits on unrecognized flags.
|
||||
|
||||
These match `/image-gen` output with the Settled Reach style guide.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Trellis API Reference
|
||||
|
||||
The connector (`tooling/db/trellis_connector.py`) talks to a Gradio API. The
|
||||
The connector (`tooling/domains/assets/trellis.py`, `reach assets trellis`) talks to a Gradio API. The
|
||||
parameter layout is fragile — document changes here when the container is
|
||||
updated. This is the canonical copy; the connector's module docstring carries
|
||||
a duplicate for at-a-glance reference when reading the script directly — keep
|
||||
|
||||
@@ -19,18 +19,21 @@ not something this skill triggers itself. Output is a PNG file.
|
||||
|
||||
Requires `GEMINI_API_KEY` in environment (set in `.claude/settings.local.json`).
|
||||
|
||||
Uses the canonical connector at `tooling/db/image_connector.py` (see
|
||||
`.claude/rules/project-structure.md` — `tooling/db/` is the documented home
|
||||
for asset/connector scripts; this skill has no local fork of it).
|
||||
Uses the canonical connector behind `reach assets image`
|
||||
(`tooling/domains/assets/image.py`, T-1290; this skill has no local fork of it).
|
||||
`health` only lists models and is free; **every `generate` costs money**.
|
||||
A verb prints its result JSON on stdout and fails with a non-zero exit — check
|
||||
the exit status, not an `ok` field. An invalid `--aspect` is rejected up front
|
||||
with the accepted list.
|
||||
|
||||
```bash
|
||||
python3 tooling/db/image_connector.py health
|
||||
reach assets image health
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python3 tooling/db/image_connector.py generate \
|
||||
reach assets image generate \
|
||||
"prompt text" \
|
||||
--output path/to/output.png \
|
||||
--aspect 1:1
|
||||
|
||||
@@ -296,6 +296,9 @@ test-tooling:
|
||||
@echo " [test-tooling] reach atlas planet options vs module parsers (T-1288)..."
|
||||
@$(VENV_PY) tooling/test_planet_router.py 2> .cache/test-tooling-planet-router.log || \
|
||||
{ echo " FAIL: planet router drift — log follows:"; cat .cache/test-tooling-planet-router.log; exit 1; }
|
||||
@echo " [test-tooling] reach assets connectors against a fake Gradio (T-1290)..."
|
||||
@$(VENV_PY) tooling/test_assets.py 2> .cache/test-tooling-assets.log || \
|
||||
{ echo " FAIL: assets connectors — log follows:"; cat .cache/test-tooling-assets.log; exit 1; }
|
||||
@echo " [test-tooling] canvas-generation version gate units (T-1242)..."
|
||||
@mkdir -p .cache
|
||||
@python3 tooling/test_canvas_version_check.py 2> .cache/test-tooling-canvas-version.log || \
|
||||
|
||||
@@ -11,7 +11,7 @@ Interface sounds triggered by player interaction, insert systems, and cognitive
|
||||
|
||||
## Generation Approach
|
||||
|
||||
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `audio-post pipeline`.
|
||||
- **All UI sounds:** Generated via Stable Audio Open with sonic family prefix prompts, then trimmed/normalized/converted via `reach assets audio post pipeline`.
|
||||
- **Monologue chimes:** Replaced in Sprint 10 (#327) with manual synthesis. Insert-tech aesthetic: pure sine harmonics, mathematical envelope, no SAO. Previous S9 SAO versions were acknowledged placeholders per D-038 amendment.
|
||||
|
||||
## Assets
|
||||
|
||||
@@ -110,5 +110,5 @@ models/props/*.glb + *_mask.png # Post-processed Trellis props + masks
|
||||
## Dependencies
|
||||
|
||||
- Godot 4.6+ (gl_compatibility renderer)
|
||||
- Models generated by: `tooling/db/trellis_connector.py` + `.claude/skills/glb-gen/`
|
||||
- Models generated by: `reach assets trellis` (formerly `tooling/db/trellis_connector.py`) + `.claude/skills/glb-gen/`
|
||||
- Concept images generated by: `.claude/skills/image-gen/`
|
||||
|
||||
+1
-1
@@ -91,7 +91,7 @@ invented.
|
||||
| ~~`planet`~~ | **folded into `atlas planet`** — the third rung of the same ladder | — |
|
||||
| `ledger` | the economics pipeline, named for the UI component that will aggregate it | ✅ ported (T-1289). `economy-db/` → `domains/ledger/` (`economy_import/` kept by name; the entrypoint became `service.py`), `schema_version.py` with it. `reach ledger import`; `make regen-db` survives as a one-line delegate, `make economy-db` retired. `generated_brands.toml` byte-identical across the move |
|
||||
| `wiki` | wiki sync and content maintenance | ✅ ported (T-1290): `reach wiki stats`, `reach wiki gttr-hook` (both output-identical to the originals). `wiki_sync.py` moved whole, but its renderer and importer are NOT verbs — re-rendering deletes ~10,700 lines of committed pages (T-1292). The seven one-shots (`assign-astro-ids`, `migrate-s-to-gj`, `patch-core-sector`, `fill-missing-globes`, `generate-stubs`/`find-stubs`, `backfill_cultural_corridor`) and the destructive `process-wiki-system-changes` went to `archive/wiki-bootstrap/` |
|
||||
| `assets` | connectors to the tower-of-joy generators | `db/audio_*.py`, `db/audio-*`, `db/image_connector.py`, `db/trellis_connector.py`, `db/common.py`, `trellis-batch.sh`, `synth_ui_sounds.py` |
|
||||
| `assets` | connectors to the tower-of-joy generators | ✅ ported (T-1290): `reach assets {audio,image,trellis} …` + `synth-ui`. The four audio bash wrappers and `trellis-batch.sh` retired; connectors call each other instead of spawning each other; `tooling/db/` is gone. Synth WAVs byte-identical, ffmpeg pipeline decode-identical; network paths pinned by `test_assets.py` against a fake Gradio. Unreachable reads as OFF (VRAM, D-17), never "restart it" |
|
||||
| `character` | bodies, garments, GLB handling | `garment-fit/make_logo.py`, `garment-qa/analyze_captures.py`, `convert_outfit.py`, `glb_strip_utility_nodes.py`, `inspect_glb.py`, `check_hair_symmetry.py`, `check_icosphere.py`, `render_quaternius_test.py`, `setup_clothing_metadata.py` — **note this is far smaller than `garment-fit/`'s file count suggests; 22 of its 23 files are Blender payloads and belong to the carve-out** |
|
||||
| `visual` | screenshot and render comparison | `visual-diff`, `visual-thumbnail`, `visual-blank-check` |
|
||||
| `godot` | Godot parse and cold-start checks | ✅ ported (T-1283); the two bash originals were left beside the port and retired in T-1290 |
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Batch audio generation from a manifest file. Whitelistable command.
|
||||
# Usage: audio-batch manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
|
||||
exec python3 "$(dirname "$0")/audio_batch.py" "$@"
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generate audio via Stable Audio Open. Whitelistable command.
|
||||
# Usage: audio-generate "prompt text" [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" generate "$@"
|
||||
@@ -1,4 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Check if the Stable Audio Open API is reachable. Whitelistable command.
|
||||
# Usage: audio-health
|
||||
exec python3 "$(dirname "$0")/audio_connector.py" health
|
||||
@@ -1,7 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Audio post-processing (ffmpeg wrapper). Whitelistable command.
|
||||
# Usage: audio-post convert input.wav [--output output.ogg]
|
||||
# audio-post normalize input.wav [--lufs -16]
|
||||
# audio-post trim input.wav [--threshold -50]
|
||||
# audio-post pipeline input.wav [--output output.ogg]
|
||||
exec python3 "$(dirname "$0")/audio_post.py" "$@"
|
||||
@@ -1,315 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Batch audio generation from a manifest file.
|
||||
|
||||
Processes multiple assets sequentially: SAO generation or harmonic synthesis,
|
||||
followed by post-processing (trim, normalize, convert to OGG).
|
||||
|
||||
Usage:
|
||||
python3 audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from common import ensure_venv # noqa: E402
|
||||
|
||||
ensure_venv()
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def load_manifest(path):
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def resolve_paths(manifest, manifest_dir):
|
||||
"""Resolve output_dir and gen_dir relative to the git root."""
|
||||
# Find git root by walking up from manifest_dir
|
||||
# Check for .git as file (worktree) or directory (regular repo)
|
||||
git_root = manifest_dir
|
||||
while git_root != "/":
|
||||
if os.path.exists(os.path.join(git_root, ".git")):
|
||||
break
|
||||
git_root = os.path.dirname(git_root)
|
||||
else:
|
||||
git_root = manifest_dir
|
||||
|
||||
output_dir = os.path.join(git_root, manifest.get("output_dir", "client/assets/audio"))
|
||||
gen_dir = os.path.join(git_root, manifest.get("gen_dir", "client/assets/audio/gen"))
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(gen_dir, exist_ok=True)
|
||||
return output_dir, gen_dir, git_root
|
||||
|
||||
|
||||
def get_default(manifest, asset, key):
|
||||
"""Get a value from the asset, falling back to manifest defaults."""
|
||||
defaults = manifest.get("defaults", {})
|
||||
return asset.get(key, defaults.get(key))
|
||||
|
||||
|
||||
def run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir):
|
||||
"""Generate audio via Stable Audio Open + post-processing."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
wav_path = os.path.join(gen_dir, base_name + ".wav")
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
|
||||
prompt = asset["prompt"]
|
||||
duration = asset.get("duration", 10)
|
||||
steps = get_default(manifest, asset, "steps") or 100
|
||||
cfg = get_default(manifest, asset, "cfg") or 7
|
||||
timeout = get_default(manifest, asset, "timeout") or 600
|
||||
|
||||
# Run audio-generate with --post
|
||||
cmd = [
|
||||
sys.executable, os.path.join(script_dir, "audio_connector.py"),
|
||||
"generate", prompt,
|
||||
"--duration", str(duration),
|
||||
"--steps", str(steps),
|
||||
"--cfg", str(cfg),
|
||||
"--output", wav_path,
|
||||
"--output-ogg", ogg_path,
|
||||
"--timeout", str(timeout),
|
||||
]
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout + 60)
|
||||
if result.returncode != 0:
|
||||
stderr = result.stderr.strip()
|
||||
try:
|
||||
err = json.loads(result.stdout)
|
||||
return {"ok": False, "error": err.get("error", stderr)}
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return {"ok": False, "error": stderr or "generation failed"}
|
||||
|
||||
try:
|
||||
return json.loads(result.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return {"ok": False, "error": f"Unexpected output: {result.stdout[:200]}"}
|
||||
|
||||
|
||||
def synthesize_harmonic(params, wav_path):
|
||||
"""Synthesize audio from harmonic parameters."""
|
||||
sr = 44100
|
||||
duration = params["duration"]
|
||||
fundamental = params["fundamental"]
|
||||
harmonics = params.get("harmonics", [])
|
||||
attack_ms = params.get("attack_ms", 10)
|
||||
sustain_ratio = params.get("sustain_ratio", 0.2)
|
||||
decay = params.get("decay", "exponential")
|
||||
|
||||
n = int(sr * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
# Fundamental
|
||||
signal = np.sin(2 * np.pi * fundamental * t)
|
||||
|
||||
# Add harmonics
|
||||
for h in harmonics:
|
||||
freq = h["freq"]
|
||||
db = h["db"]
|
||||
amplitude = 10 ** (db / 20)
|
||||
signal = signal + amplitude * np.sin(2 * np.pi * freq * t)
|
||||
|
||||
# Envelope: attack + sustain + decay
|
||||
attack_s = attack_ms / 1000
|
||||
attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n)
|
||||
|
||||
sustain_end = duration * sustain_ratio
|
||||
if decay == "exponential":
|
||||
# Decay rate: reach -60dB by end of duration
|
||||
decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10
|
||||
decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end)))
|
||||
else:
|
||||
# Linear decay
|
||||
decay_env = np.where(t < sustain_end, 1.0,
|
||||
1.0 - (t - sustain_end) / (duration - sustain_end))
|
||||
|
||||
envelope = attack_env * decay_env
|
||||
signal = signal * envelope
|
||||
|
||||
# Normalize to peak
|
||||
peak = np.max(np.abs(signal))
|
||||
if peak > 0:
|
||||
signal = signal / peak * 0.9
|
||||
|
||||
# Write WAV
|
||||
int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16)
|
||||
with wave.open(wav_path, "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
|
||||
return wav_path
|
||||
|
||||
|
||||
def run_synth(asset, manifest, gen_dir, output_dir, script_dir):
|
||||
"""Synthesize audio from harmonic parameters + post-process."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
wav_path = os.path.join(gen_dir, base_name + "_synth.wav")
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
|
||||
synth_params = asset.get("synth")
|
||||
if not synth_params:
|
||||
return {"ok": False, "error": "No synth parameters provided"}
|
||||
|
||||
synth_type = synth_params.get("type", "harmonic")
|
||||
if synth_type != "harmonic":
|
||||
return {"ok": False, "error": f"Unknown synth type: {synth_type}"}
|
||||
|
||||
try:
|
||||
synthesize_harmonic(synth_params, wav_path)
|
||||
except Exception as e:
|
||||
return {"ok": False, "error": f"Synthesis failed: {e}"}
|
||||
|
||||
# Post-process: normalize + convert (skip trim for synth — no silence to trim)
|
||||
post_script = os.path.join(script_dir, "audio_post.py")
|
||||
lufs = get_default(manifest, asset, "lufs") or -16
|
||||
quality = get_default(manifest, asset, "quality") or 6
|
||||
|
||||
# Normalize
|
||||
norm_path = os.path.join(gen_dir, base_name + "_norm.wav")
|
||||
cmd = [sys.executable, post_script, "normalize", wav_path, "--output", norm_path,
|
||||
"--lufs", str(lufs)]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Normalize failed: {result.stderr.strip()}"}
|
||||
|
||||
# Convert to OGG
|
||||
cmd = [sys.executable, post_script, "convert", norm_path, "--output", ogg_path,
|
||||
"--quality", str(quality)]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Convert failed: {result.stderr.strip()}"}
|
||||
|
||||
# Clean up intermediate
|
||||
try:
|
||||
os.remove(norm_path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
ogg_size = os.path.getsize(ogg_path)
|
||||
return {
|
||||
"ok": True,
|
||||
"file": wav_path,
|
||||
"ogg_file": ogg_path,
|
||||
"ogg_size_bytes": ogg_size,
|
||||
"synth_params": synth_params,
|
||||
"post_processed": True,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: audio_batch.py manifest.json [--dry-run] [--only ID,ID,...] [--skip-existing]",
|
||||
file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
manifest_path = sys.argv[1]
|
||||
dry_run = "--dry-run" in sys.argv
|
||||
skip_existing = "--skip-existing" in sys.argv
|
||||
|
||||
only_ids = None
|
||||
for i, arg in enumerate(sys.argv):
|
||||
if arg == "--only" and i + 1 < len(sys.argv):
|
||||
only_ids = set(sys.argv[i + 1].split(","))
|
||||
|
||||
manifest = load_manifest(manifest_path)
|
||||
manifest_dir = os.path.dirname(os.path.abspath(manifest_path))
|
||||
output_dir, gen_dir, git_root = resolve_paths(manifest, manifest_dir)
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
assets = manifest.get("assets", [])
|
||||
if only_ids:
|
||||
assets = [a for a in assets if a["id"] in only_ids]
|
||||
|
||||
# Health check if any SAO assets
|
||||
sao_assets = [a for a in assets if a.get("method") == "sao"]
|
||||
if sao_assets and not dry_run:
|
||||
print("Checking SAO API health...", file=sys.stderr)
|
||||
health_cmd = [sys.executable, os.path.join(script_dir, "audio_connector.py"), "health"]
|
||||
result = subprocess.run(health_cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(json.dumps({"ok": False, "error": "SAO API health check failed",
|
||||
"details": result.stdout.strip()}))
|
||||
sys.exit(1)
|
||||
print(" SAO API is up.", file=sys.stderr)
|
||||
|
||||
total = len(assets)
|
||||
results = []
|
||||
success = 0
|
||||
failed = 0
|
||||
skipped = 0
|
||||
|
||||
print(f"Processing {total} assets from {os.path.basename(manifest_path)}...", file=sys.stderr)
|
||||
if dry_run:
|
||||
print(" (dry run — no generation will occur)", file=sys.stderr)
|
||||
|
||||
for i, asset in enumerate(assets, 1):
|
||||
asset_id = asset["id"]
|
||||
filename = asset["filename"]
|
||||
method = asset.get("method", "sao")
|
||||
|
||||
print(f"\n[{i}/{total}] {asset_id}: {filename} ({method})", file=sys.stderr)
|
||||
|
||||
if skip_existing:
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
if os.path.exists(ogg_path):
|
||||
print(" Skipping — already exists", file=sys.stderr)
|
||||
results.append({"id": asset_id, "status": "skipped", "reason": "exists"})
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
if dry_run:
|
||||
print(f" Would generate: {filename}", file=sys.stderr)
|
||||
if method == "sao":
|
||||
print(f" Prompt: {asset.get('prompt', '(none)')[:80]}...", file=sys.stderr)
|
||||
elif method == "synth":
|
||||
synth = asset.get("synth", {})
|
||||
print(f" Synth: {synth.get('fundamental')}Hz, {synth.get('duration')}s",
|
||||
file=sys.stderr)
|
||||
results.append({"id": asset_id, "status": "dry_run"})
|
||||
continue
|
||||
|
||||
if method == "sao":
|
||||
result = run_sao_generate(asset, manifest, gen_dir, output_dir, script_dir)
|
||||
elif method == "synth":
|
||||
result = run_synth(asset, manifest, gen_dir, output_dir, script_dir)
|
||||
else:
|
||||
result = {"ok": False, "error": f"Unknown method: {method}"}
|
||||
|
||||
result["id"] = asset_id
|
||||
if result.get("ok"):
|
||||
success += 1
|
||||
result["status"] = "success"
|
||||
print(f" OK → {result.get('ogg_file', filename)}", file=sys.stderr)
|
||||
else:
|
||||
failed += 1
|
||||
result["status"] = "failed"
|
||||
print(f" FAILED: {result.get('error', 'unknown')}", file=sys.stderr)
|
||||
|
||||
results.append(result)
|
||||
|
||||
# Summary
|
||||
summary = {
|
||||
"ok": failed == 0,
|
||||
"total": total,
|
||||
"success": success,
|
||||
"failed": failed,
|
||||
"skipped": skipped,
|
||||
"results": results,
|
||||
}
|
||||
print(json.dumps(summary, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,356 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Stable Audio Open connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Stable Audio Open Gradio app at tower-of-joy:11500.
|
||||
Uses the async Gradio API pattern: POST to submit, SSE stream for results.
|
||||
|
||||
Usage:
|
||||
python3 audio_connector.py generate "prompt text" [--duration 10] [--steps 100] [--cfg 7] [--output file.wav] [--post]
|
||||
python3 audio_connector.py health
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from common import ensure_venv, get_base_url as _get_base_url # noqa: E402
|
||||
|
||||
ensure_venv()
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
def get_base_url() -> str:
|
||||
return _get_base_url("stable_audio_url", "http://tower-of-joy:11500")
|
||||
|
||||
def health():
|
||||
"""Check if the Stable Audio API is reachable."""
|
||||
base = get_base_url()
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/config", method="GET")
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
version = data.get("version", "unknown")
|
||||
# Extract component info for the generate endpoint
|
||||
api_names = []
|
||||
for dep in data.get("dependencies", []):
|
||||
name = dep.get("api_name", "")
|
||||
if name and not name.startswith("js_"):
|
||||
api_names.append(name)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"url": base,
|
||||
"gradio_version": version,
|
||||
"api_endpoints": api_names
|
||||
}, indent=2))
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"url": base,
|
||||
"error": str(e)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def post_process(wav_path, ogg_path=None, lufs=-16, quality=6, threshold=-50):
|
||||
"""Run trim + normalize + convert on a WAV file via audio-post pipeline."""
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
post_script = os.path.join(script_dir, "audio_post.py")
|
||||
if ogg_path is None:
|
||||
ogg_path = os.path.splitext(wav_path)[0] + ".ogg"
|
||||
cmd = [
|
||||
sys.executable, post_script, "pipeline", wav_path,
|
||||
"--output", ogg_path,
|
||||
"--lufs", str(lufs),
|
||||
"--quality", str(quality),
|
||||
"--threshold", str(threshold),
|
||||
]
|
||||
print(f" Post-processing → {os.path.basename(ogg_path)}...", file=sys.stderr)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return {"ok": False, "error": f"Post-processing failed: {result.stderr.strip()}"}
|
||||
try:
|
||||
return json.loads(result.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return {"ok": True, "output": ogg_path}
|
||||
|
||||
|
||||
def _check_available(base):
|
||||
"""Quick check if Stable Audio is reachable. Fail fast with a clear message."""
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/config", method="GET")
|
||||
urllib.request.urlopen(req, timeout=5)
|
||||
except Exception:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"Stable Audio is not available at {base}. The service may be switched off to save system resources. Start it before generating audio."
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def generate(prompt, duration=10.0, steps=100, cfg=7.0, output=None, timeout=600,
|
||||
post=False, output_ogg=None):
|
||||
"""
|
||||
Generate audio from a text prompt.
|
||||
|
||||
Args:
|
||||
prompt: Text description of the audio to generate
|
||||
duration: Duration in seconds (0-47, default 10)
|
||||
steps: Number of diffusion steps (default 100, lower = faster but lower quality)
|
||||
cfg: Classifier-free guidance scale (default 7)
|
||||
output: Output file path (default: auto-named in current directory)
|
||||
timeout: Maximum wait time in seconds (default 600 = 10 minutes)
|
||||
post: If True, run trim+normalize+convert after generation
|
||||
output_ogg: OGG output path when post=True (default: same basename .ogg)
|
||||
"""
|
||||
base = get_base_url()
|
||||
_check_available(base)
|
||||
api_url = f"{base}/gradio_api/call/generate_audio"
|
||||
|
||||
if output is None:
|
||||
# Auto-name: sanitize prompt to a filename
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
output = f"{safe}_{int(duration)}s.wav"
|
||||
|
||||
# Step 1: Submit the generation request
|
||||
payload = json.dumps({"data": [prompt, duration, steps, cfg]})
|
||||
req = urllib.request.Request(
|
||||
api_url,
|
||||
data=payload.encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print("Submitting generation request...", file=sys.stderr)
|
||||
print(f" Prompt: {prompt}", file=sys.stderr)
|
||||
print(f" Duration: {duration}s, Steps: {steps}, CFG: {cfg}", file=sys.stderr)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
result = json.loads(resp.read())
|
||||
event_id = result.get("event_id")
|
||||
if not event_id:
|
||||
print(json.dumps({"ok": False, "error": "No event_id returned", "response": result}, indent=2))
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Submit failed: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
print(f" Event ID: {event_id}", file=sys.stderr)
|
||||
print(f" Waiting for generation (timeout: {timeout}s)...", file=sys.stderr)
|
||||
|
||||
# Step 2: Poll the SSE stream for results
|
||||
stream_url = f"{api_url}/{event_id}"
|
||||
start_time = time.time()
|
||||
result_data = None
|
||||
last_status = None
|
||||
|
||||
while time.time() - start_time < timeout:
|
||||
try:
|
||||
req = urllib.request.Request(stream_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
# Read SSE events
|
||||
current_event = None
|
||||
for line_bytes in resp:
|
||||
line = line_bytes.decode("utf-8").strip()
|
||||
|
||||
if line.startswith("event: "):
|
||||
current_event = line[7:]
|
||||
elif line.startswith("data: ") and current_event:
|
||||
data_str = line[6:]
|
||||
|
||||
if current_event == "heartbeat":
|
||||
elapsed = int(time.time() - start_time)
|
||||
if elapsed % 30 == 0 and elapsed > 0:
|
||||
print(f" Still generating... ({elapsed}s elapsed)", file=sys.stderr)
|
||||
continue
|
||||
|
||||
if current_event == "error":
|
||||
error_msg = data_str
|
||||
try:
|
||||
error_msg = json.loads(data_str)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
print(json.dumps({"ok": False, "error": "Generation failed", "details": error_msg}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if current_event == "complete":
|
||||
try:
|
||||
result_data = json.loads(data_str)
|
||||
except json.JSONDecodeError:
|
||||
result_data = data_str
|
||||
break
|
||||
|
||||
if current_event == "progress":
|
||||
try:
|
||||
progress = json.loads(data_str)
|
||||
# Gradio progress events vary; log what we get
|
||||
status = str(progress)[:80]
|
||||
if status != last_status:
|
||||
print(f" Progress: {status}", file=sys.stderr)
|
||||
last_status = status
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
continue
|
||||
|
||||
if result_data is not None:
|
||||
break
|
||||
|
||||
except urllib.error.URLError as e:
|
||||
# Connection dropped — retry after brief pause
|
||||
time.sleep(2)
|
||||
continue
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": f"Stream error: {e}"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
if result_data is None:
|
||||
print(json.dumps({"ok": False, "error": f"Generation timed out after {timeout}s"}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Step 3: Download the audio file
|
||||
# Gradio returns file info in the data array
|
||||
elapsed = round(time.time() - start_time, 1)
|
||||
print(f" Generation complete ({elapsed}s)", file=sys.stderr)
|
||||
|
||||
try:
|
||||
# result_data is typically [{"path": "...", "url": "...", ...}] or similar
|
||||
if isinstance(result_data, list) and len(result_data) > 0:
|
||||
audio_info = result_data[0]
|
||||
elif isinstance(result_data, dict) and "data" in result_data:
|
||||
audio_info = result_data["data"][0] if result_data["data"] else None
|
||||
else:
|
||||
audio_info = result_data
|
||||
|
||||
# Extract the file URL
|
||||
file_url = None
|
||||
if isinstance(audio_info, dict):
|
||||
file_url = audio_info.get("url") or audio_info.get("path")
|
||||
elif isinstance(audio_info, str):
|
||||
file_url = audio_info
|
||||
|
||||
if not file_url:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "Could not extract audio URL from response",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Handle relative URLs
|
||||
if file_url.startswith("/"):
|
||||
file_url = f"{base}{file_url}"
|
||||
elif not file_url.startswith("http"):
|
||||
file_url = f"{base}/file={file_url}"
|
||||
|
||||
# Download the file
|
||||
print(f" Downloading to {output}...", file=sys.stderr)
|
||||
req = urllib.request.Request(file_url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
with open(output, "wb") as f:
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
file_size = os.path.getsize(output)
|
||||
result_json = {
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
"duration_requested": duration,
|
||||
"steps": steps,
|
||||
"cfg": cfg,
|
||||
"prompt": prompt,
|
||||
"generation_time_s": elapsed
|
||||
}
|
||||
|
||||
if post:
|
||||
post_result = post_process(output, ogg_path=output_ogg)
|
||||
if not post_result.get("ok"):
|
||||
result_json["post_processed"] = False
|
||||
result_json["post_error"] = post_result.get("error", "unknown")
|
||||
else:
|
||||
result_json["post_processed"] = True
|
||||
result_json["ogg_file"] = post_result.get("output", output_ogg)
|
||||
ogg_size = os.path.getsize(result_json["ogg_file"])
|
||||
result_json["ogg_size_bytes"] = ogg_size
|
||||
|
||||
print(json.dumps(result_json, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"Download failed: {e}",
|
||||
"response": result_data
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage:")
|
||||
print(" audio_connector.py health")
|
||||
print(" audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav] [--timeout N]")
|
||||
sys.exit(1)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "health":
|
||||
health()
|
||||
elif cmd == "generate":
|
||||
if len(sys.argv) < 3:
|
||||
print("Error: prompt required", file=sys.stderr)
|
||||
print("Usage: audio_connector.py generate 'prompt' [--duration N] [--steps N] [--cfg N] [--output file.wav]", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
prompt = sys.argv[2]
|
||||
duration = 10.0
|
||||
steps = 100
|
||||
cfg = 7.0
|
||||
output = None
|
||||
timeout = 600
|
||||
post = False
|
||||
output_ogg = None
|
||||
|
||||
# Parse optional args
|
||||
i = 3
|
||||
while i < len(sys.argv):
|
||||
if sys.argv[i] == "--duration" and i + 1 < len(sys.argv):
|
||||
duration = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--steps" and i + 1 < len(sys.argv):
|
||||
steps = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--cfg" and i + 1 < len(sys.argv):
|
||||
cfg = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--output" and i + 1 < len(sys.argv):
|
||||
output = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
|
||||
timeout = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--post":
|
||||
post = True
|
||||
i += 1
|
||||
elif sys.argv[i] == "--output-ogg" and i + 1 < len(sys.argv):
|
||||
output_ogg = sys.argv[i + 1]
|
||||
post = True # --output-ogg implies --post
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(prompt, duration=duration, steps=steps, cfg=cfg, output=output,
|
||||
timeout=timeout, post=post, output_ogg=output_ogg)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,171 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Audio post-processing wrapper around ffmpeg.
|
||||
|
||||
Subcommands:
|
||||
convert — WAV to OGG (libvorbis, quality 6)
|
||||
normalize — LUFS normalize to -16 LUFS (broadcast standard)
|
||||
trim — Remove leading/trailing silence
|
||||
pipeline — trim + normalize + convert (full post-processing chain)
|
||||
|
||||
All operations write to a new file (never overwrites input).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import shutil
|
||||
import json
|
||||
|
||||
|
||||
def check_ffmpeg():
|
||||
if not shutil.which("ffmpeg"):
|
||||
print(json.dumps({"ok": False, "error": "ffmpeg not found in PATH"}))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run_ffmpeg(args, description):
|
||||
"""Run ffmpeg, capture output, return success."""
|
||||
cmd = ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error"] + args
|
||||
print(f" {description}", file=sys.stderr)
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"ffmpeg failed: {result.stderr.strip()}",
|
||||
"command": " ".join(cmd)
|
||||
}))
|
||||
sys.exit(1)
|
||||
return True
|
||||
|
||||
|
||||
def cmd_convert(args):
|
||||
"""Convert WAV to OGG (libvorbis)."""
|
||||
output = args.output or args.input.rsplit(".", 1)[0] + ".ogg"
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
f"converting {os.path.basename(args.input)} → {os.path.basename(output)}"
|
||||
)
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def cmd_normalize(args):
|
||||
"""LUFS normalize audio file."""
|
||||
output = args.output or _suffixed(args.input, "_norm")
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
output],
|
||||
f"normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_trim(args):
|
||||
"""Trim leading/trailing silence."""
|
||||
output = args.output or _suffixed(args.input, "_trimmed")
|
||||
af = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af, output],
|
||||
f"trimming silence (threshold: {args.threshold}dB)"
|
||||
)
|
||||
print(json.dumps({"ok": True, "output": output}))
|
||||
|
||||
|
||||
def cmd_pipeline(args):
|
||||
"""Full post-processing: trim → normalize → convert to OGG."""
|
||||
base = args.input.rsplit(".", 1)[0]
|
||||
trimmed = base + "_trimmed.wav"
|
||||
normalized = base + "_norm.wav"
|
||||
output = args.output or base + ".ogg"
|
||||
|
||||
# Step 1: trim
|
||||
af_trim = (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={args.threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", args.input, "-af", af_trim, trimmed],
|
||||
"step 1/3: trimming silence"
|
||||
)
|
||||
|
||||
# Step 2: normalize
|
||||
run_ffmpeg(
|
||||
["-i", trimmed, "-af",
|
||||
f"loudnorm=I={args.lufs}:LRA=11:TP=-1",
|
||||
normalized],
|
||||
f"step 2/3: normalizing to {args.lufs} LUFS"
|
||||
)
|
||||
|
||||
# Step 3: convert
|
||||
run_ffmpeg(
|
||||
["-i", normalized, "-c:a", "libvorbis", "-q:a", str(args.quality), output],
|
||||
"step 3/3: converting to OGG"
|
||||
)
|
||||
|
||||
# Clean up intermediates
|
||||
os.remove(trimmed)
|
||||
os.remove(normalized)
|
||||
|
||||
size = os.path.getsize(output)
|
||||
print(json.dumps({"ok": True, "output": output, "size_bytes": size}))
|
||||
|
||||
|
||||
def _suffixed(path, suffix):
|
||||
base, ext = os.path.splitext(path)
|
||||
return base + suffix + ext
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Audio post-processing (ffmpeg wrapper)")
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# convert
|
||||
p = sub.add_parser("convert", help="WAV → OGG")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output file (default: same name .ogg)")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.set_defaults(func=cmd_convert)
|
||||
|
||||
# normalize
|
||||
p = sub.add_parser("normalize", help="LUFS normalize")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.set_defaults(func=cmd_normalize)
|
||||
|
||||
# trim
|
||||
p = sub.add_parser("trim", help="Trim silence")
|
||||
p.add_argument("input", help="Input audio file")
|
||||
p.add_argument("--output", "-o", help="Output file")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_trim)
|
||||
|
||||
# pipeline
|
||||
p = sub.add_parser("pipeline", help="Full post-processing: trim + normalize + convert")
|
||||
p.add_argument("input", help="Input WAV file")
|
||||
p.add_argument("--output", "-o", help="Output OGG file")
|
||||
p.add_argument("--quality", "-q", type=int, default=6, help="Vorbis quality 0-10 (default: 6)")
|
||||
p.add_argument("--lufs", type=float, default=-16, help="Target LUFS (default: -16)")
|
||||
p.add_argument("--threshold", type=int, default=-50, help="Silence threshold in dB (default: -50)")
|
||||
p.set_defaults(func=cmd_pipeline)
|
||||
|
||||
args = parser.parse_args()
|
||||
check_ffmpeg()
|
||||
args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,99 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared utilities for Settled Reach asset/connector scripts.
|
||||
|
||||
Provides `ensure_venv`, used by the asset connectors (audio_connector,
|
||||
audio_batch, image_connector, trellis_connector) to re-exec into the project
|
||||
.venv before their third-party imports, plus the connector config helpers
|
||||
(`load_config`, `get_base_url`, `get_api_key`) that were formerly copy-pasted
|
||||
across the connectors (T-1067 rider, S-25). The former settledreach.db
|
||||
connection helpers were removed when the ticket/decision tooling was retired
|
||||
(pql migration Phase 6); planning now lives in pql (`.pql/`).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
WORKTREE_ROOT = (SCRIPT_DIR / ".." / "..").resolve()
|
||||
CONFIG_PATH = SCRIPT_DIR / "config.json"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def ensure_venv() -> None:
|
||||
"""Re-exec into the project .venv Python if not already running there.
|
||||
|
||||
Call this at the top of any script that uses third-party packages,
|
||||
before those imports.
|
||||
|
||||
Usage::
|
||||
|
||||
from common import ensure_venv
|
||||
ensure_venv()
|
||||
import numpy as np # third-party import follows
|
||||
"""
|
||||
venv_python = WORKTREE_ROOT / ".venv" / "bin" / "python"
|
||||
|
||||
# Already running inside the venv — nothing to do.
|
||||
if Path(sys.executable).resolve() == venv_python.resolve():
|
||||
return
|
||||
|
||||
# .venv not set up yet — fail with a helpful message.
|
||||
if not venv_python.exists():
|
||||
print(
|
||||
"error: .venv not found — run `make setup-venv` first.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Re-exec into the venv Python, preserving all arguments.
|
||||
os.execv(str(venv_python), [str(venv_python)] + sys.argv)
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
"""Load the shared endpoint configuration from tooling/db/config.json."""
|
||||
with open(CONFIG_PATH) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def get_base_url(key: str, default: str) -> str:
|
||||
"""Resolve a service base URL from config.json, with a fallback default.
|
||||
|
||||
Usage::
|
||||
|
||||
base = get_base_url("trellis_url", "http://tower-of-joy:11510")
|
||||
"""
|
||||
return load_config().get(key, default)
|
||||
|
||||
|
||||
def get_api_key(env_var: str, config_key: str = "") -> str:
|
||||
"""Get an API key from the environment — environment-only, by design.
|
||||
|
||||
``tooling/db/config.json`` is a *tracked* file and holds endpoints only;
|
||||
it must never carry secrets, so there is deliberately no config.json
|
||||
fallback here (the old one steered users toward committing paid API keys).
|
||||
``config_key`` is retained in the signature for caller compatibility but
|
||||
is ignored. Prints a JSON error and exits 1 if the variable is unset —
|
||||
connector scripts emit machine-readable JSON on all paths.
|
||||
"""
|
||||
key = os.environ.get(env_var)
|
||||
if key:
|
||||
return key
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": (
|
||||
f"{env_var} not set. Export it in your shell or add it to the "
|
||||
"machine-local .claude/settings.local.json env block (untracked). "
|
||||
"Never put keys in tooling/db/config.json — it is tracked."
|
||||
)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
@@ -1,247 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Gemini image generator connector — direct API wrapper.
|
||||
|
||||
Generates images via Google's gemini-2.5-flash-image generation API.
|
||||
API key from GEMINI_API_KEY env var or config.json.
|
||||
|
||||
Usage:
|
||||
python3 image_connector.py health
|
||||
python3 image_connector.py generate "prompt" [--output file.png] [--aspect 1:1] [--size 1K] [--input image.png]
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from common import ensure_venv, get_api_key as _get_api_key # noqa: E402
|
||||
|
||||
ensure_venv()
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
DEFAULT_OUTPUT_DIR = os.path.expanduser("~/Pictures/mcp-images")
|
||||
|
||||
|
||||
def get_api_key() -> str:
|
||||
"""Get Gemini API key from env or config."""
|
||||
return _get_api_key("GEMINI_API_KEY", "gemini_api_key")
|
||||
|
||||
|
||||
def health():
|
||||
"""Check if the Gemini API is reachable with the configured key."""
|
||||
key = get_api_key()
|
||||
url = f"https://generativelanguage.googleapis.com/v1beta/models?key={key}"
|
||||
try:
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
models = [m.get("name", "") for m in data.get("models", [])
|
||||
if "imagen" in m.get("name", "").lower()
|
||||
or "flash" in m.get("name", "").lower()]
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"api": "gemini",
|
||||
"image_capable_models": models[:5],
|
||||
}, indent=2))
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": str(e)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def generate(prompt, output=None, aspect_ratio="1:1", image_size=None,
|
||||
input_image=None):
|
||||
"""
|
||||
Generate an image from a text prompt using Gemini.
|
||||
|
||||
Args:
|
||||
prompt: Text description of the image to generate
|
||||
output: Output file path (default: auto-named in ~/Pictures/mcp-images/)
|
||||
aspect_ratio: Aspect ratio (1:1, 16:9, 3:2, etc.)
|
||||
image_size: Resolution hint (1K, 2K, 4K) - may not be honored
|
||||
input_image: Optional input image path for image-to-image generation
|
||||
"""
|
||||
key = get_api_key()
|
||||
|
||||
# Gemini image generation model
|
||||
model = "gemini-2.5-flash-image"
|
||||
url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={key}"
|
||||
|
||||
if output is None:
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
os.makedirs(DEFAULT_OUTPUT_DIR, exist_ok=True)
|
||||
output = os.path.join(DEFAULT_OUTPUT_DIR, f"{safe}.png")
|
||||
|
||||
# Build the request
|
||||
parts = []
|
||||
|
||||
# Add input image if provided (image-to-image)
|
||||
if input_image:
|
||||
if not os.path.isfile(input_image):
|
||||
print(json.dumps({"ok": False, "error": f"Input image not found: {input_image}"}, indent=2))
|
||||
sys.exit(1)
|
||||
with open(input_image, "rb") as f:
|
||||
image_data = base64.b64encode(f.read()).decode("utf-8")
|
||||
# Detect mime type
|
||||
ext = os.path.splitext(input_image)[1].lower()
|
||||
mime = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
|
||||
".webp": "image/webp"}.get(ext, "image/png")
|
||||
parts.append({
|
||||
"inlineData": {
|
||||
"mimeType": mime,
|
||||
"data": image_data
|
||||
}
|
||||
})
|
||||
|
||||
# Build enhanced prompt with a size hint. Unlike aspect ratio below, Gemini
|
||||
# has no dedicated resolution parameter for this model — this is a
|
||||
# best-effort prompt hint only and may not be honored.
|
||||
enhanced_prompt = prompt
|
||||
if image_size:
|
||||
enhanced_prompt += f" Resolution: {image_size}."
|
||||
|
||||
parts.append({"text": enhanced_prompt})
|
||||
|
||||
generation_config = {"responseModalities": ["TEXT", "IMAGE"]}
|
||||
if aspect_ratio:
|
||||
# Real API parameter (not a prompt hint). Valid values: 1:1, 3:2,
|
||||
# 2:3, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9.
|
||||
# https://ai.google.dev/gemini-api/docs/image-generation
|
||||
generation_config["imageConfig"] = {"aspectRatio": aspect_ratio}
|
||||
|
||||
payload = json.dumps({
|
||||
"contents": [{"parts": parts}],
|
||||
"generationConfig": generation_config
|
||||
})
|
||||
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=payload.encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print("Generating image...", file=sys.stderr)
|
||||
print(f" Prompt: {prompt}", file=sys.stderr)
|
||||
if input_image:
|
||||
print(f" Input image: {input_image}", file=sys.stderr)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
result = json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode("utf-8", errors="replace")
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"API error {e.code}: {e.reason}",
|
||||
"details": body[:500]
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(json.dumps({"ok": False, "error": str(e)}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Extract image data from response
|
||||
candidates = result.get("candidates", [])
|
||||
if not candidates:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "No candidates in response",
|
||||
"response": json.dumps(result)[:500]
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
image_saved = False
|
||||
text_response = ""
|
||||
|
||||
for candidate in candidates:
|
||||
content = candidate.get("content", {})
|
||||
for part in content.get("parts", []):
|
||||
if "inlineData" in part:
|
||||
# Image data
|
||||
image_b64 = part["inlineData"]["data"]
|
||||
image_bytes = base64.b64decode(image_b64)
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
|
||||
with open(output, "wb") as f:
|
||||
f.write(image_bytes)
|
||||
image_saved = True
|
||||
elif "text" in part:
|
||||
text_response += part["text"]
|
||||
|
||||
if not image_saved:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "No image data in response",
|
||||
"text_response": text_response[:500],
|
||||
"response": json.dumps(result)[:500]
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
file_size = os.path.getsize(output)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
"prompt": prompt,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
}, indent=2))
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage:")
|
||||
print(" image_connector.py health")
|
||||
print(" image_connector.py generate 'prompt' [--output file.png] [--aspect 1:1] [--size 1K] [--input image.png]")
|
||||
sys.exit(1)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "health":
|
||||
health()
|
||||
elif cmd == "generate":
|
||||
if len(sys.argv) < 3:
|
||||
print("Error: prompt required", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
prompt = sys.argv[2]
|
||||
output = None
|
||||
aspect_ratio = "1:1"
|
||||
image_size = None
|
||||
input_image = None
|
||||
|
||||
i = 3
|
||||
while i < len(sys.argv):
|
||||
if sys.argv[i] == "--output" and i + 1 < len(sys.argv):
|
||||
output = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--aspect" and i + 1 < len(sys.argv):
|
||||
aspect_ratio = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--size" and i + 1 < len(sys.argv):
|
||||
image_size = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--input" and i + 1 < len(sys.argv):
|
||||
input_image = sys.argv[i + 1]
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(prompt, output=output, aspect_ratio=aspect_ratio,
|
||||
image_size=image_size, input_image=input_image)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,360 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Trellis 3D model generator connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Trellis Gradio app at tower-of-joy:11510.
|
||||
Pipeline: upload image → start session → image_to_3d → extract_glb → download .glb
|
||||
|
||||
Usage:
|
||||
python3 trellis_connector.py health
|
||||
python3 trellis_connector.py generate image.png [--output model.glb] [--simplify 0.95] [--texture-size 1024] [--seed 42] [--timeout 600]
|
||||
|
||||
Gradio API Parameter Reference (TRELLIS v1, microsoft/TRELLIS):
|
||||
|
||||
/image_to_3d — 9 inputs:
|
||||
0: image (Image) preprocessed image from /preprocess_image_1
|
||||
1: multiimages (Gallery) [] for single-image mode
|
||||
2: is_multiimage (State) False for single-image, True for multi-image
|
||||
3: seed (Slider) int, 0-2147483647
|
||||
4: ss_guidance (Slider) float, sparse structure guidance strength (default 7.5)
|
||||
5: ss_steps (Slider) int, sparse structure sampling steps (default 12)
|
||||
6: slat_guidance (Slider) float, structured latent guidance strength (default 3.0)
|
||||
7: slat_steps (Slider) int, structured latent sampling steps (default 12)
|
||||
8: multiimage_algo (Radio) "stochastic" or "multidiffusion"
|
||||
|
||||
/extract_glb — 3 inputs:
|
||||
0: output_buf (State) None — server uses internal state from image_to_3d
|
||||
1: simplify (Slider) float, mesh simplification ratio (default 0.95)
|
||||
2: texture_size (Slider) int, texture resolution (default 1024)
|
||||
|
||||
Common failure modes:
|
||||
- "needed 9, got 8": missing is_multiimage (position 2) — must pass False
|
||||
- "needed 3, got 2": missing output_buf (position 0) — must pass None
|
||||
- "'float' cannot be interpreted as int": numpy version issue on server,
|
||||
Gradio Sliders send all values as float. Fix: patch flow_euler.py on
|
||||
the server to cast steps to int, or pin numpy < 2.0
|
||||
- CUDA device mismatch after crash: restart the container to clear GPU state
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from common import ensure_venv, get_base_url as _get_base_url # noqa: E402
|
||||
|
||||
ensure_venv()
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
|
||||
def get_base_url() -> str:
|
||||
return _get_base_url("trellis_url", "http://tower-of-joy:11510")
|
||||
|
||||
|
||||
def health():
|
||||
"""Check if the Trellis API is reachable."""
|
||||
base = get_base_url()
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/info", method="GET")
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
data = json.loads(resp.read())
|
||||
endpoints = list(data.get("named_endpoints", {}).keys())
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"url": base,
|
||||
"endpoints": endpoints
|
||||
}, indent=2))
|
||||
except Exception as e:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"url": base,
|
||||
"error": str(e)
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _call_api(base, endpoint, data, timeout=600, session_hash=None):
|
||||
"""Call a Gradio API endpoint with optional session tracking.
|
||||
|
||||
Gradio gr.State components are stored per session_hash on the server.
|
||||
All calls in a pipeline (image_to_3d → extract_glb) must share the same
|
||||
session_hash so the server can pass state between them.
|
||||
"""
|
||||
api_url = f"{base}/api{endpoint}"
|
||||
|
||||
body_dict = {"data": data}
|
||||
if session_hash:
|
||||
body_dict["session_hash"] = session_hash
|
||||
|
||||
payload = json.dumps(body_dict)
|
||||
req = urllib.request.Request(
|
||||
api_url,
|
||||
data=payload.encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print(f" Calling {endpoint}...", file=sys.stderr)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
result = json.loads(resp.read())
|
||||
if isinstance(result, dict) and "data" in result:
|
||||
return result["data"]
|
||||
return result
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode("utf-8", errors="replace")
|
||||
raise RuntimeError(f"{endpoint} failed ({e.code}): {body[:300]}")
|
||||
|
||||
|
||||
def _upload_image(base, image_path):
|
||||
"""Upload an image file to the Gradio server and return the file reference."""
|
||||
upload_url = f"{base}/upload"
|
||||
|
||||
with open(image_path, "rb") as f:
|
||||
image_data = f.read()
|
||||
|
||||
filename = os.path.basename(image_path)
|
||||
|
||||
# Gradio upload expects multipart/form-data with a 'files' field
|
||||
boundary = "----TrellisConnectorBoundary"
|
||||
body = (
|
||||
f"--{boundary}\r\n"
|
||||
f'Content-Disposition: form-data; name="files"; filename="{filename}"\r\n'
|
||||
f"Content-Type: image/png\r\n"
|
||||
f"\r\n"
|
||||
).encode() + image_data + f"\r\n--{boundary}--\r\n".encode()
|
||||
|
||||
req = urllib.request.Request(
|
||||
upload_url,
|
||||
data=body,
|
||||
headers={
|
||||
"Content-Type": f"multipart/form-data; boundary={boundary}",
|
||||
},
|
||||
method="POST"
|
||||
)
|
||||
|
||||
print(f" Uploading {filename}...", file=sys.stderr)
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
result = json.loads(resp.read())
|
||||
# Gradio returns a list of uploaded file paths
|
||||
if isinstance(result, list) and len(result) > 0:
|
||||
return result[0]
|
||||
raise RuntimeError(f"Upload failed: {result}")
|
||||
|
||||
|
||||
def _download_file(url, output_path, base):
|
||||
"""Download a file from the Gradio server."""
|
||||
if url.startswith("/"):
|
||||
url = f"{base}{url}"
|
||||
elif not url.startswith("http"):
|
||||
url = f"{base}/file={url}"
|
||||
|
||||
print(f" Downloading to {output_path}...", file=sys.stderr)
|
||||
req = urllib.request.Request(url, method="GET")
|
||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
||||
with open(output_path, "wb") as f:
|
||||
shutil.copyfileobj(resp, f)
|
||||
|
||||
return os.path.getsize(output_path)
|
||||
|
||||
|
||||
def _check_available(base):
|
||||
"""Quick check if Trellis is reachable. Fail fast with a clear message."""
|
||||
try:
|
||||
req = urllib.request.Request(f"{base}/info", method="GET")
|
||||
urllib.request.urlopen(req, timeout=5)
|
||||
except Exception:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": f"Trellis is not available at {base}. The service may be switched off to save system resources. Start it before generating 3D models."
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def generate(image_path, output=None, simplify=0.95, texture_size=1024,
|
||||
seed=0, timeout=600):
|
||||
"""
|
||||
Generate a 3D model from an image.
|
||||
|
||||
Pipeline:
|
||||
1. Start session
|
||||
2. Upload and preprocess image
|
||||
3. Generate 3D from image
|
||||
4. Extract GLB
|
||||
5. Download GLB file
|
||||
|
||||
Args:
|
||||
image_path: Path to the input image (PNG recommended)
|
||||
output: Output .glb file path (default: auto-named)
|
||||
simplify: Mesh simplification factor (0.9-0.98, default 0.95)
|
||||
texture_size: Texture resolution (512-2048, default 1024)
|
||||
seed: Random seed (default 0)
|
||||
timeout: Max wait time per step in seconds
|
||||
"""
|
||||
base = get_base_url()
|
||||
_check_available(base)
|
||||
start_time = time.time()
|
||||
|
||||
if not os.path.isfile(image_path):
|
||||
print(json.dumps({"ok": False, "error": f"Image not found: {image_path}"}), indent=2)
|
||||
sys.exit(1)
|
||||
|
||||
if output is None:
|
||||
name = os.path.splitext(os.path.basename(image_path))[0]
|
||||
output = f"{name}.glb"
|
||||
|
||||
# Generate a session hash — Gradio uses this to maintain gr.State between
|
||||
# separate API calls. Without it, image_to_3d's output state is lost before
|
||||
# extract_glb can read it.
|
||||
import random
|
||||
import string
|
||||
session = ''.join(random.choices(string.ascii_lowercase + string.digits, k=12))
|
||||
print(f" Session: {session}", file=sys.stderr)
|
||||
|
||||
# Step 1: Start session
|
||||
print("Step 1/5: Starting session...", file=sys.stderr)
|
||||
session_result = _call_api(base, "/start_session", [], timeout=30, session_hash=session)
|
||||
|
||||
# Step 2: Upload and preprocess image
|
||||
print("Step 2/5: Uploading and preprocessing image...", file=sys.stderr)
|
||||
uploaded_path = _upload_image(base, image_path)
|
||||
file_ref = {
|
||||
"path": uploaded_path,
|
||||
"meta": {"_type": "gradio.FileData"}
|
||||
}
|
||||
preprocess_result = _call_api(base, "/preprocess_image_1", [file_ref], timeout=60, session_hash=session)
|
||||
|
||||
# _call_api returns the "data" array directly
|
||||
if isinstance(preprocess_result, list) and len(preprocess_result) > 0:
|
||||
preprocessed_ref = preprocess_result[0]
|
||||
else:
|
||||
preprocessed_ref = preprocess_result
|
||||
|
||||
# Step 3: Get seed
|
||||
print("Step 3/5: Generating 3D model...", file=sys.stderr)
|
||||
seed_result = _call_api(base, "/get_seed", [True, seed], timeout=10, session_hash=session)
|
||||
if isinstance(seed_result, list) and seed_result:
|
||||
actual_seed = seed_result[0]
|
||||
else:
|
||||
actual_seed = seed
|
||||
|
||||
# Step 4: Image to 3D
|
||||
# Gradio app has 9 inputs: Image, Gallery, State, Slider×5, Radio
|
||||
# State is a hidden session component at position 3 — pass None.
|
||||
# Slider order (from /info): seed, ss_guidance, ss_steps, slat_guidance, slat_steps
|
||||
# Note: server-side numpy bug requires patching flow_euler.py to cast steps to int.
|
||||
gen_result = _call_api(base, "/image_to_3d", [
|
||||
preprocessed_ref, # image
|
||||
[], # multiimages (empty)
|
||||
False, # is_multiimage (boolean flag, not session state)
|
||||
actual_seed, # seed (Slider, 0-2147483647)
|
||||
7.5, # ss_guidance_strength
|
||||
12, # ss_sampling_steps
|
||||
3.0, # slat_guidance_strength
|
||||
12, # slat_sampling_steps
|
||||
"stochastic", # multiimage_algo
|
||||
], timeout=timeout, session_hash=session)
|
||||
|
||||
# Step 5: Extract GLB
|
||||
# State (output_buf) is maintained server-side via session_hash.
|
||||
# extract_glb reads it automatically. Pass None as placeholder for the State component.
|
||||
print("Step 4/5: Extracting GLB...", file=sys.stderr)
|
||||
glb_result = _call_api(base, "/extract_glb", [None, simplify, texture_size], timeout=120, session_hash=session)
|
||||
|
||||
# _call_api returns the "data" array: [model_viewer_data, download_button_data]
|
||||
glb_url = None
|
||||
if isinstance(glb_result, list):
|
||||
for item in glb_result:
|
||||
if isinstance(item, dict):
|
||||
url = item.get("url") or item.get("path")
|
||||
if url:
|
||||
glb_url = url
|
||||
break
|
||||
|
||||
if not glb_url:
|
||||
print(json.dumps({
|
||||
"ok": False,
|
||||
"error": "Could not extract GLB URL from response",
|
||||
"response": glb_result
|
||||
}, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
# Step 6: Download
|
||||
print("Step 5/5: Downloading GLB...", file=sys.stderr)
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
|
||||
file_size = _download_file(glb_url, output, base)
|
||||
|
||||
elapsed = round(time.time() - start_time, 1)
|
||||
print(json.dumps({
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": file_size,
|
||||
"simplify": simplify,
|
||||
"texture_size": texture_size,
|
||||
"seed": actual_seed,
|
||||
"generation_time_s": elapsed,
|
||||
"source_image": image_path
|
||||
}, indent=2))
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage:")
|
||||
print(" trellis_connector.py health")
|
||||
print(" trellis_connector.py generate image.png [--output model.glb] [--simplify 0.95] [--texture-size 1024] [--seed N] [--timeout N]")
|
||||
sys.exit(1)
|
||||
|
||||
cmd = sys.argv[1]
|
||||
|
||||
if cmd == "health":
|
||||
health()
|
||||
elif cmd == "generate":
|
||||
if len(sys.argv) < 3:
|
||||
print("Error: image path required", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
image_path = sys.argv[2]
|
||||
output = None
|
||||
simplify = 0.95
|
||||
texture_size = 1024
|
||||
seed = 0
|
||||
timeout = 600
|
||||
|
||||
i = 3
|
||||
while i < len(sys.argv):
|
||||
if sys.argv[i] == "--output" and i + 1 < len(sys.argv):
|
||||
output = sys.argv[i + 1]
|
||||
i += 2
|
||||
elif sys.argv[i] == "--simplify" and i + 1 < len(sys.argv):
|
||||
simplify = float(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--texture-size" and i + 1 < len(sys.argv):
|
||||
texture_size = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--seed" and i + 1 < len(sys.argv):
|
||||
seed = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
elif sys.argv[i] == "--timeout" and i + 1 < len(sys.argv):
|
||||
timeout = int(sys.argv[i + 1])
|
||||
i += 2
|
||||
else:
|
||||
print(f"Unknown argument: {sys.argv[i]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
generate(image_path, output=output, simplify=simplify,
|
||||
texture_size=texture_size, seed=seed, timeout=timeout)
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,13 @@
|
||||
"""`assets` — connectors to the generators that make game assets (D-263).
|
||||
|
||||
Stable Audio Open and Trellis run on tower-of-joy and are kept switched OFF to
|
||||
save VRAM; Gemini is a paid cloud API. So in this domain network failure is
|
||||
the normal case, and every connector reports it through endpoints.py, which
|
||||
says whether the service is off or rejected the request.
|
||||
|
||||
Formerly tooling/db/ (a misnamed directory: it held connectors, not database
|
||||
work) plus tooling/trellis-batch.sh and tooling/synth_ui_sounds.py (T-1290).
|
||||
Output contract: a verb's result goes to stdout as JSON — the same keys the
|
||||
old scripts printed, so the skills' readers keep working — and a failure is a
|
||||
non-zero exit with a remedy, not `{"ok": false}` on stdout.
|
||||
"""
|
||||
@@ -0,0 +1,247 @@
|
||||
"""Stable Audio Open connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Stable Audio Open Gradio app on tower-of-joy :11500 (URL from
|
||||
config.json). Uses the async Gradio pattern: POST to submit, then read the SSE
|
||||
stream for the result, then download the file.
|
||||
|
||||
Formerly tooling/db/audio_connector.py, fronted by the audio-generate and
|
||||
audio-health bash wrappers (T-1290). Behaviour is unchanged except where a
|
||||
failure used to print `{"ok": false}` on stdout and exit 1: it now raises a
|
||||
ReachError that says whether the service is OFF or rejected the request, and
|
||||
post-processing is a function call rather than a second Python process.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from tooling.core import console
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.domains.assets import audio_post, endpoints
|
||||
|
||||
SERVICE = "audio"
|
||||
|
||||
|
||||
def get_base_url() -> str:
|
||||
return endpoints.base_url(SERVICE)
|
||||
|
||||
|
||||
def health() -> dict:
|
||||
"""Is the Stable Audio API reachable, and what does it expose?"""
|
||||
base = get_base_url()
|
||||
data = endpoints.call_json(
|
||||
urllib.request.Request(f"{base}/config", method="GET"),
|
||||
service=SERVICE,
|
||||
what="the health check",
|
||||
timeout=10,
|
||||
)
|
||||
api_names = [
|
||||
dep.get("api_name", "")
|
||||
for dep in data.get("dependencies", [])
|
||||
if dep.get("api_name", "") and not dep.get("api_name", "").startswith("js_")
|
||||
]
|
||||
return {
|
||||
"ok": True,
|
||||
"url": base,
|
||||
"gradio_version": data.get("version", "unknown"),
|
||||
"api_endpoints": api_names,
|
||||
}
|
||||
|
||||
|
||||
def post_process(wav_path: str, ogg_path: str | None = None, lufs=-16, quality=6, threshold=-50) -> dict:
|
||||
"""trim + normalize + convert a generated WAV (the `--post` step)."""
|
||||
if ogg_path is None:
|
||||
ogg_path = os.path.splitext(wav_path)[0] + ".ogg"
|
||||
console.event(f"Post-processing → {os.path.basename(ogg_path)}...")
|
||||
try:
|
||||
return audio_post.pipeline(wav_path, ogg_path, quality=quality, lufs=lufs, threshold=threshold)
|
||||
except ReachError as exc:
|
||||
# Not fatal to the generation: the WAV exists and is reported, and the
|
||||
# failure is carried in the result exactly as before.
|
||||
return {"ok": False, "error": f"Post-processing failed: {exc.message}"}
|
||||
|
||||
|
||||
def default_output(prompt: str, duration: float) -> str:
|
||||
"""Auto-name: the first 40 prompt characters, sanitised, plus the duration."""
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
return f"{safe}_{int(duration)}s.wav"
|
||||
|
||||
|
||||
def extract_file_url(result_data, base: str) -> str | None:
|
||||
"""Find the audio file URL in Gradio's `complete` payload, made absolute."""
|
||||
if isinstance(result_data, list) and len(result_data) > 0:
|
||||
audio_info = result_data[0]
|
||||
elif isinstance(result_data, dict) and "data" in result_data:
|
||||
audio_info = result_data["data"][0] if result_data["data"] else None
|
||||
else:
|
||||
audio_info = result_data
|
||||
|
||||
file_url = None
|
||||
if isinstance(audio_info, dict):
|
||||
file_url = audio_info.get("url") or audio_info.get("path")
|
||||
elif isinstance(audio_info, str):
|
||||
file_url = audio_info
|
||||
if not file_url:
|
||||
return None
|
||||
|
||||
if file_url.startswith("/"):
|
||||
return f"{base}{file_url}"
|
||||
if not file_url.startswith("http"):
|
||||
return f"{base}/file={file_url}"
|
||||
return file_url
|
||||
|
||||
|
||||
def generate(
|
||||
prompt: str,
|
||||
duration: float = 10.0,
|
||||
steps: int = 100,
|
||||
cfg: float = 7.0,
|
||||
output: str | None = None,
|
||||
timeout: int = 600,
|
||||
post: bool = False,
|
||||
output_ogg: str | None = None,
|
||||
) -> dict:
|
||||
"""Generate audio from a text prompt; returns the result dict.
|
||||
|
||||
duration is 0-47 s. Fewer steps is faster and worse. `post` (implied by
|
||||
`output_ogg`) runs trim + normalize + convert on the result.
|
||||
"""
|
||||
base = get_base_url()
|
||||
# Fail fast, and say OFF rather than "submit failed", before building anything.
|
||||
endpoints.call(
|
||||
urllib.request.Request(f"{base}/config", method="GET"),
|
||||
service=SERVICE,
|
||||
what="the availability check",
|
||||
timeout=5,
|
||||
)
|
||||
api_url = f"{base}/gradio_api/call/generate_audio"
|
||||
output = output or default_output(prompt, duration)
|
||||
|
||||
# 1. Submit.
|
||||
console.event(f"Submitting: {prompt!r} ({duration}s, {steps} steps, cfg {cfg})")
|
||||
submitted = endpoints.call_json(
|
||||
urllib.request.Request(
|
||||
api_url,
|
||||
data=json.dumps({"data": [prompt, duration, steps, cfg]}).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
),
|
||||
service=SERVICE,
|
||||
what="the generation request",
|
||||
timeout=30,
|
||||
)
|
||||
event_id = submitted.get("event_id") if isinstance(submitted, dict) else None
|
||||
if not event_id:
|
||||
raise ReachError(
|
||||
f"Stable Audio accepted the request but returned no event_id: {submitted}",
|
||||
fix="the Gradio app's API may have changed — check `reach assets audio health` endpoints",
|
||||
)
|
||||
console.event(f"Event {event_id} — waiting for generation (timeout {timeout}s)...")
|
||||
|
||||
# 2. Read the SSE stream. A dropped connection is retried until the
|
||||
# deadline, exactly as before; only an `error` event is fatal.
|
||||
started = time.time()
|
||||
result_data = _await_result(f"{api_url}/{event_id}", timeout, started)
|
||||
|
||||
elapsed = round(time.time() - started, 1)
|
||||
console.event(f"Generation complete ({elapsed}s)")
|
||||
|
||||
# 3. Download.
|
||||
file_url = extract_file_url(result_data, base)
|
||||
if not file_url:
|
||||
raise ReachError(
|
||||
f"could not find the audio URL in the response: {str(result_data)[:300]}",
|
||||
fix="the Gradio app's response shape may have changed — see audio.extract_file_url",
|
||||
)
|
||||
console.event(f"Downloading to {output}...")
|
||||
try:
|
||||
with urllib.request.urlopen(urllib.request.Request(file_url, method="GET"), timeout=60) as resp:
|
||||
with open(output, "wb") as f:
|
||||
shutil.copyfileobj(resp, f)
|
||||
except urllib.error.URLError as exc:
|
||||
raise ReachError(
|
||||
f"download failed from {file_url}: {exc}",
|
||||
fix="the file may have expired on the server — re-run the generation",
|
||||
) from exc
|
||||
|
||||
result = {
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": os.path.getsize(output),
|
||||
"duration_requested": duration,
|
||||
"steps": steps,
|
||||
"cfg": cfg,
|
||||
"prompt": prompt,
|
||||
"generation_time_s": elapsed,
|
||||
}
|
||||
|
||||
if post or output_ogg:
|
||||
post_result = post_process(output, ogg_path=output_ogg)
|
||||
if not post_result.get("ok"):
|
||||
result["post_processed"] = False
|
||||
result["post_error"] = post_result.get("error", "unknown")
|
||||
else:
|
||||
result["post_processed"] = True
|
||||
result["ogg_file"] = post_result.get("output", output_ogg)
|
||||
result["ogg_size_bytes"] = os.path.getsize(result["ogg_file"])
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _await_result(stream_url: str, timeout: int, start: float):
|
||||
"""Poll the Gradio SSE stream until `complete`, an `error`, or the deadline."""
|
||||
last_status = None
|
||||
|
||||
while time.time() - start < timeout:
|
||||
try:
|
||||
with urllib.request.urlopen(urllib.request.Request(stream_url, method="GET"), timeout=timeout) as resp:
|
||||
current_event = None
|
||||
for line_bytes in resp:
|
||||
line = line_bytes.decode("utf-8").strip()
|
||||
if line.startswith("event: "):
|
||||
current_event = line[7:]
|
||||
continue
|
||||
if not (line.startswith("data: ") and current_event):
|
||||
continue
|
||||
data_str = line[6:]
|
||||
|
||||
if current_event == "heartbeat":
|
||||
elapsed = int(time.time() - start)
|
||||
if elapsed % 30 == 0 and elapsed > 0:
|
||||
console.event(f"Still generating... ({elapsed}s elapsed)")
|
||||
elif current_event == "error":
|
||||
try:
|
||||
detail = json.loads(data_str)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
detail = data_str
|
||||
raise ReachError(
|
||||
f"Stable Audio reported a generation error: {detail}",
|
||||
fix="the service is up — check the prompt and duration (0-47 s), then re-run",
|
||||
)
|
||||
elif current_event == "complete":
|
||||
try:
|
||||
return json.loads(data_str)
|
||||
except json.JSONDecodeError:
|
||||
return data_str
|
||||
elif current_event == "progress":
|
||||
try:
|
||||
status = str(json.loads(data_str))[:80]
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
if status != last_status:
|
||||
console.event(f"Progress: {status}")
|
||||
last_status = status
|
||||
except urllib.error.URLError:
|
||||
time.sleep(2) # connection dropped mid-stream — retry until the deadline
|
||||
|
||||
raise ReachError(
|
||||
f"generation timed out after {timeout}s",
|
||||
fix="raise --timeout, or lower --steps / --duration",
|
||||
)
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
"""Batch audio generation from a manifest file.
|
||||
|
||||
Processes assets in order: Stable Audio generation (`method: sao`) or harmonic
|
||||
synthesis (`method: synth`), each followed by post-processing to OGG.
|
||||
|
||||
Formerly tooling/db/audio_batch.py behind the audio-batch wrapper (T-1290). It
|
||||
used to re-launch audio_connector.py and audio_post.py as subprocesses and
|
||||
parse their stdout; it now calls them. The synthesis itself is unchanged —
|
||||
same parameters give byte-identical WAVs. One behaviour change: a batch with
|
||||
failures used to print `"ok": false` and exit 0; the router now fails it.
|
||||
|
||||
Manifest schema (docs/assets/audio/, see the audio-gen skill):
|
||||
{"output_dir": ..., "gen_dir": ..., "defaults": {steps, cfg, timeout, lufs, quality},
|
||||
"assets": [{"id", "filename", "method": "sao"|"synth", "prompt"?, "duration"?,
|
||||
"synth"?: {"type": "harmonic", "duration", "fundamental",
|
||||
"harmonics": [{"freq", "db"}], "attack_ms",
|
||||
"sustain_ratio", "decay": "exponential"|"linear"}}]}
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
|
||||
from tooling.core import config, console
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.domains.assets import audio, audio_post
|
||||
|
||||
SAMPLE_RATE = 44100
|
||||
|
||||
|
||||
def load_manifest(path: str) -> dict:
|
||||
try:
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
except FileNotFoundError as exc:
|
||||
raise ReachError(f"manifest not found: {path}", fix="pass a manifest .json path") from exc
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ReachError(f"manifest is not valid JSON: {path}: {exc}", fix="fix the JSON, then re-run") from exc
|
||||
|
||||
|
||||
def resolve_paths(manifest: dict) -> tuple[str, str]:
|
||||
"""output_dir and gen_dir, relative to the repo root (created if missing).
|
||||
|
||||
The old script found the root by walking up from the manifest looking for
|
||||
.git; config.repo_root() asks git, which gives the same answer in a
|
||||
worktree and does not silently fall back to the manifest's own directory.
|
||||
"""
|
||||
root = str(config.repo_root())
|
||||
output_dir = os.path.join(root, manifest.get("output_dir", "client/assets/audio"))
|
||||
gen_dir = os.path.join(root, manifest.get("gen_dir", "client/assets/audio/gen"))
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(gen_dir, exist_ok=True)
|
||||
return output_dir, gen_dir
|
||||
|
||||
|
||||
def get_default(manifest: dict, asset: dict, key: str):
|
||||
"""An asset value, falling back to the manifest's defaults."""
|
||||
return asset.get(key, manifest.get("defaults", {}).get(key))
|
||||
|
||||
|
||||
def run_sao_generate(asset: dict, manifest: dict, gen_dir: str, output_dir: str) -> dict:
|
||||
"""Generate via Stable Audio, post-processing straight to the final OGG."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
return audio.generate(
|
||||
asset["prompt"],
|
||||
duration=asset.get("duration", 10),
|
||||
steps=get_default(manifest, asset, "steps") or 100,
|
||||
cfg=get_default(manifest, asset, "cfg") or 7,
|
||||
output=os.path.join(gen_dir, base_name + ".wav"),
|
||||
timeout=get_default(manifest, asset, "timeout") or 600,
|
||||
output_ogg=os.path.join(output_dir, filename),
|
||||
)
|
||||
|
||||
|
||||
def synthesize_harmonic(params: dict, wav_path: str) -> str:
|
||||
"""Synthesize a tone from harmonic parameters and write a 16-bit mono WAV."""
|
||||
sr = SAMPLE_RATE
|
||||
duration = params["duration"]
|
||||
fundamental = params["fundamental"]
|
||||
harmonics = params.get("harmonics", [])
|
||||
attack_ms = params.get("attack_ms", 10)
|
||||
sustain_ratio = params.get("sustain_ratio", 0.2)
|
||||
decay = params.get("decay", "exponential")
|
||||
|
||||
n = int(sr * duration)
|
||||
t = np.linspace(0, duration, n, endpoint=False)
|
||||
|
||||
signal = np.sin(2 * np.pi * fundamental * t)
|
||||
for h in harmonics:
|
||||
amplitude = 10 ** (h["db"] / 20)
|
||||
signal = signal + amplitude * np.sin(2 * np.pi * h["freq"] * t)
|
||||
|
||||
# Envelope: attack + sustain + decay.
|
||||
attack_s = attack_ms / 1000
|
||||
attack_env = np.minimum(t / attack_s, 1.0) if attack_s > 0 else np.ones(n)
|
||||
|
||||
sustain_end = duration * sustain_ratio
|
||||
if decay == "exponential":
|
||||
# Reach -60 dB by the end of the duration.
|
||||
decay_rate = 6.9 / (duration - sustain_end) if duration > sustain_end else 10
|
||||
decay_env = np.where(t < sustain_end, 1.0, np.exp(-decay_rate * (t - sustain_end)))
|
||||
else:
|
||||
decay_env = np.where(t < sustain_end, 1.0, 1.0 - (t - sustain_end) / (duration - sustain_end))
|
||||
|
||||
signal = signal * attack_env * decay_env
|
||||
|
||||
peak = np.max(np.abs(signal))
|
||||
if peak > 0:
|
||||
signal = signal / peak * 0.9
|
||||
|
||||
int_samples = np.clip(signal * 32767, -32767, 32767).astype(np.int16)
|
||||
with wave.open(wav_path, "w") as f:
|
||||
f.setnchannels(1)
|
||||
f.setsampwidth(2)
|
||||
f.setframerate(sr)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
return wav_path
|
||||
|
||||
|
||||
def run_synth(asset: dict, manifest: dict, gen_dir: str, output_dir: str) -> dict:
|
||||
"""Synthesize, then normalize + convert (no trim: synth has no silence to trim)."""
|
||||
filename = asset["filename"]
|
||||
base_name = os.path.splitext(filename)[0]
|
||||
wav_path = os.path.join(gen_dir, base_name + "_synth.wav")
|
||||
norm_path = os.path.join(gen_dir, base_name + "_norm.wav")
|
||||
ogg_path = os.path.join(output_dir, filename)
|
||||
|
||||
synth_params = asset.get("synth")
|
||||
if not synth_params:
|
||||
return {"ok": False, "error": "No synth parameters provided"}
|
||||
if synth_params.get("type", "harmonic") != "harmonic":
|
||||
return {"ok": False, "error": f"Unknown synth type: {synth_params.get('type')}"}
|
||||
|
||||
try:
|
||||
synthesize_harmonic(synth_params, wav_path)
|
||||
audio_post.normalize(wav_path, norm_path, lufs=get_default(manifest, asset, "lufs") or -16)
|
||||
audio_post.convert(norm_path, ogg_path, quality=get_default(manifest, asset, "quality") or 6)
|
||||
except ReachError as exc:
|
||||
return {"ok": False, "error": exc.message}
|
||||
finally:
|
||||
if os.path.exists(norm_path):
|
||||
os.remove(norm_path)
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"file": wav_path,
|
||||
"ogg_file": ogg_path,
|
||||
"ogg_size_bytes": os.path.getsize(ogg_path),
|
||||
"synth_params": synth_params,
|
||||
"post_processed": True,
|
||||
}
|
||||
|
||||
|
||||
def run(
|
||||
manifest_path: str,
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
only: set[str] | None = None,
|
||||
skip_existing: bool = False,
|
||||
) -> dict:
|
||||
"""Process a manifest; returns the summary. The router fails it on any failure."""
|
||||
manifest = load_manifest(manifest_path)
|
||||
output_dir, gen_dir = resolve_paths(manifest)
|
||||
|
||||
assets = manifest.get("assets", [])
|
||||
if only:
|
||||
assets = [a for a in assets if a["id"] in only]
|
||||
|
||||
# One health check up front if anything needs Stable Audio — so an OFF
|
||||
# service fails once, with the right remedy, not once per asset.
|
||||
if not dry_run and any(a.get("method") == "sao" for a in assets):
|
||||
console.event("Checking Stable Audio health...")
|
||||
audio.health()
|
||||
|
||||
total = len(assets)
|
||||
results = []
|
||||
counts = {"success": 0, "failed": 0, "skipped": 0}
|
||||
console.event(
|
||||
f"Processing {total} assets from {os.path.basename(manifest_path)}"
|
||||
+ (" — dry run, nothing generated" if dry_run else "")
|
||||
)
|
||||
|
||||
for i, asset in enumerate(assets, 1):
|
||||
asset_id, filename = asset["id"], asset["filename"]
|
||||
method = asset.get("method", "sao")
|
||||
phase = f"{i}/{total}"
|
||||
console.event(f"{asset_id}: {filename} ({method})", phase=phase, progress=i / total if total else None)
|
||||
|
||||
if skip_existing and os.path.exists(os.path.join(output_dir, filename)):
|
||||
console.event("skipping — already exists", phase=phase)
|
||||
results.append({"id": asset_id, "status": "skipped", "reason": "exists"})
|
||||
counts["skipped"] += 1
|
||||
continue
|
||||
|
||||
if dry_run:
|
||||
if method == "sao":
|
||||
console.event(f"would generate — prompt: {asset.get('prompt', '(none)')[:80]}...", phase=phase)
|
||||
elif method == "synth":
|
||||
synth = asset.get("synth", {})
|
||||
console.event(
|
||||
f"would synthesize — {synth.get('fundamental')}Hz, {synth.get('duration')}s", phase=phase
|
||||
)
|
||||
results.append({"id": asset_id, "status": "dry_run"})
|
||||
continue
|
||||
|
||||
if method == "sao":
|
||||
try:
|
||||
result = run_sao_generate(asset, manifest, gen_dir, output_dir)
|
||||
except ReachError as exc:
|
||||
result = {"ok": False, "error": exc.message}
|
||||
elif method == "synth":
|
||||
result = run_synth(asset, manifest, gen_dir, output_dir)
|
||||
else:
|
||||
result = {"ok": False, "error": f"Unknown method: {method}"}
|
||||
|
||||
result["id"] = asset_id
|
||||
if result.get("ok"):
|
||||
counts["success"] += 1
|
||||
result["status"] = "success"
|
||||
console.event(f"OK → {result.get('ogg_file', filename)}", phase=phase)
|
||||
else:
|
||||
counts["failed"] += 1
|
||||
result["status"] = "failed"
|
||||
console.event(f"FAILED: {result.get('error', 'unknown')}", phase=phase, level="warn")
|
||||
results.append(result)
|
||||
|
||||
return {"ok": counts["failed"] == 0, "total": total, **counts, "results": results}
|
||||
@@ -0,0 +1,113 @@
|
||||
"""Audio post-processing around ffmpeg.
|
||||
|
||||
convert — WAV to OGG (libvorbis, quality 6)
|
||||
normalize — LUFS normalize to -16 LUFS (broadcast standard)
|
||||
trim — remove leading/trailing silence
|
||||
pipeline — trim + normalize + convert (the full post-processing chain)
|
||||
|
||||
Every operation writes a new file and never overwrites its input. Each returns
|
||||
a result dict — the same keys the old script printed as JSON — so callers read
|
||||
data rather than parse output. ffmpeg runs through `core/process.run`, the one
|
||||
guarded exec (D-263): a non-zero exit becomes a ReachError naming the command,
|
||||
and a missing ffmpeg says what to install.
|
||||
|
||||
The ffmpeg argument lists are unchanged from tooling/db/audio_post.py (T-1290);
|
||||
the decoded audio of a pipeline run is identical before and after the port.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from tooling.core import console, process
|
||||
|
||||
FFMPEG_MISSING = "install ffmpeg (brew install ffmpeg), or check PATH in a non-interactive shell"
|
||||
|
||||
|
||||
def trim_filter(threshold: int) -> str:
|
||||
"""The silence-removal filter chain: trim the start, reverse, trim, reverse."""
|
||||
return (
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={threshold}dB,"
|
||||
"areverse,"
|
||||
"silenceremove=start_periods=1:start_silence=0.05"
|
||||
f":start_threshold={threshold}dB,"
|
||||
"areverse"
|
||||
)
|
||||
|
||||
|
||||
def ffmpeg_argv(args: list[str]) -> list[str]:
|
||||
"""The full argv for one ffmpeg call — pure, so it can be tested unrun."""
|
||||
return ["ffmpeg", "-y", "-hide_banner", "-loglevel", "error", *args]
|
||||
|
||||
|
||||
def run_ffmpeg(args: list[str], description: str) -> None:
|
||||
console.event(description)
|
||||
process.run(ffmpeg_argv(args), missing_fix=FFMPEG_MISSING)
|
||||
|
||||
|
||||
def convert(input_path: str, output: str | None = None, quality: int = 6) -> dict:
|
||||
"""WAV → OGG (libvorbis)."""
|
||||
output = output or input_path.rsplit(".", 1)[0] + ".ogg"
|
||||
run_ffmpeg(
|
||||
["-i", input_path, "-c:a", "libvorbis", "-q:a", str(quality), output],
|
||||
f"converting {os.path.basename(input_path)} → {os.path.basename(output)}",
|
||||
)
|
||||
return {"ok": True, "output": output, "size_bytes": os.path.getsize(output)}
|
||||
|
||||
|
||||
def normalize(input_path: str, output: str | None = None, lufs: float = -16) -> dict:
|
||||
"""LUFS-normalize an audio file."""
|
||||
output = output or _suffixed(input_path, "_norm")
|
||||
run_ffmpeg(
|
||||
["-i", input_path, "-af", f"loudnorm=I={lufs}:LRA=11:TP=-1", output],
|
||||
f"normalizing to {lufs} LUFS",
|
||||
)
|
||||
return {"ok": True, "output": output}
|
||||
|
||||
|
||||
def trim(input_path: str, output: str | None = None, threshold: int = -50) -> dict:
|
||||
"""Trim leading and trailing silence."""
|
||||
output = output or _suffixed(input_path, "_trimmed")
|
||||
run_ffmpeg(
|
||||
["-i", input_path, "-af", trim_filter(threshold), output],
|
||||
f"trimming silence (threshold: {threshold}dB)",
|
||||
)
|
||||
return {"ok": True, "output": output}
|
||||
|
||||
|
||||
def pipeline(
|
||||
input_path: str,
|
||||
output: str | None = None,
|
||||
quality: int = 6,
|
||||
lufs: float = -16,
|
||||
threshold: int = -50,
|
||||
) -> dict:
|
||||
"""Full post-processing: trim → normalize → convert to OGG."""
|
||||
base = input_path.rsplit(".", 1)[0]
|
||||
trimmed = base + "_trimmed.wav"
|
||||
normalized = base + "_norm.wav"
|
||||
output = output or base + ".ogg"
|
||||
|
||||
try:
|
||||
run_ffmpeg(["-i", input_path, "-af", trim_filter(threshold), trimmed], "step 1/3: trimming silence")
|
||||
run_ffmpeg(
|
||||
["-i", trimmed, "-af", f"loudnorm=I={lufs}:LRA=11:TP=-1", normalized],
|
||||
f"step 2/3: normalizing to {lufs} LUFS",
|
||||
)
|
||||
run_ffmpeg(
|
||||
["-i", normalized, "-c:a", "libvorbis", "-q:a", str(quality), output],
|
||||
"step 3/3: converting to OGG",
|
||||
)
|
||||
finally:
|
||||
# The old script left the intermediates behind when a step failed.
|
||||
for leftover in (trimmed, normalized):
|
||||
if os.path.exists(leftover):
|
||||
os.remove(leftover)
|
||||
|
||||
return {"ok": True, "output": output, "size_bytes": os.path.getsize(output)}
|
||||
|
||||
|
||||
def _suffixed(path: str, suffix: str) -> str:
|
||||
base, ext = os.path.splitext(path)
|
||||
return base + suffix + ext
|
||||
@@ -0,0 +1,151 @@
|
||||
"""Where the generators live, and how a call to one fails.
|
||||
|
||||
Two jobs, both shared by every connector in the domain:
|
||||
|
||||
1. **Endpoints and keys.** Base URLs come from `config.json` beside this file
|
||||
(moved deliberately from tooling/db/config.json in T-1290). Keys come from the
|
||||
environment only: config.json is tracked, so a config fallback for a paid
|
||||
API key is how one gets committed.
|
||||
|
||||
2. **Failure classification.** Network failure is the NORMAL case here, not an
|
||||
exception. Stable Audio and Trellis are kept stopped because VRAM on
|
||||
tower-of-joy is scarce (system-admin-danoontje docs/topology.md, D-17), so
|
||||
"connection refused" means *switched off*, and the remedy is to get it
|
||||
turned on — never to restart it blindly, which takes VRAM from whatever is
|
||||
running. That is a different answer from "the service rejected what you
|
||||
sent", and a caller that cannot tell the two apart wastes a round trip
|
||||
either way. `call()` maps every urllib failure to one of the two.
|
||||
|
||||
`ensure_venv()` (formerly in tooling/db/common.py) is gone: it re-exec'd the
|
||||
script under .venv/bin/python via os.execv. Under reach the dependencies are
|
||||
declared by the package itself, and an execv carrying reach's argv into another
|
||||
interpreter would relaunch something that is not the command at all.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from tooling.core.errors import ReachError
|
||||
|
||||
CONFIG_PATH = Path(__file__).resolve().parent / "config.json"
|
||||
|
||||
TOPOLOGY = "/var/mnt/data/projects/system-admin-danoontje/docs/topology.md"
|
||||
|
||||
# service key -> (display name, config key, fallback URL)
|
||||
SERVICES: dict[str, tuple[str, str, str]] = {
|
||||
"audio": ("Stable Audio Open", "stable_audio_url", "http://tower-of-joy:11500"),
|
||||
"trellis": ("Trellis", "trellis_url", "http://tower-of-joy:11510"),
|
||||
}
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
"""The tracked endpoint configuration (URLs only, never secrets)."""
|
||||
with open(CONFIG_PATH) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def get_base_url(key: str, default: str) -> str:
|
||||
"""Resolve a service base URL from config.json, with a fallback default."""
|
||||
return load_config().get(key, default)
|
||||
|
||||
|
||||
def base_url(service: str) -> str:
|
||||
"""The configured base URL for one of SERVICES."""
|
||||
_name, config_key, default = SERVICES[service]
|
||||
return get_base_url(config_key, default)
|
||||
|
||||
|
||||
def get_api_key(env_var: str) -> str:
|
||||
"""An API key from the environment — environment-only, by design."""
|
||||
key = os.environ.get(env_var)
|
||||
if key:
|
||||
return key
|
||||
raise ReachError(
|
||||
f"{env_var} is not set",
|
||||
fix=(
|
||||
f"export {env_var}, or add it to the machine-local "
|
||||
".claude/settings.local.json env block (untracked). Never put keys in "
|
||||
"tooling/domains/assets/config.json — it is tracked."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def unreachable(service: str, url: str, detail: str) -> ReachError:
|
||||
"""The service did not answer at all.
|
||||
|
||||
For the tower-of-joy generators that is their normal resting state. For
|
||||
anything else (the Gemini API) it means the network, not the service.
|
||||
"""
|
||||
if service not in SERVICES:
|
||||
return ReachError(
|
||||
f"{service} is not reachable at {url} ({detail})",
|
||||
fix="check this machine's network connection, then re-run",
|
||||
)
|
||||
name = SERVICES[service][0]
|
||||
return ReachError(
|
||||
f"{name} is not reachable at {url} ({detail}). It is kept switched off to "
|
||||
"save VRAM on tower-of-joy, so this usually means OFF, not broken.",
|
||||
fix=(
|
||||
f"ask for {name} to be turned on (something else may need to stop first "
|
||||
f"to free VRAM — {TOPOLOGY}); do not restart it blindly. "
|
||||
f"Then: reach assets {service} health"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def rejected(name: str, what: str, code: int, body: str) -> ReachError:
|
||||
"""The service answered, and said no — the request is what needs changing."""
|
||||
snippet = body.strip()[:500]
|
||||
if code in (401, 403):
|
||||
fix = "the service is up but refused the credentials — check the API key"
|
||||
elif code == 429:
|
||||
fix = "rate-limited or out of quota — wait, or check the account's quota"
|
||||
else:
|
||||
fix = "the service is up — check the arguments and inputs, then re-run"
|
||||
return ReachError(
|
||||
f"{name} rejected {what} (HTTP {code})" + (f": {snippet}" if snippet else ""),
|
||||
fix=fix,
|
||||
)
|
||||
|
||||
|
||||
def call(
|
||||
request: urllib.request.Request,
|
||||
*,
|
||||
service: str,
|
||||
what: str,
|
||||
timeout: float,
|
||||
) -> bytes:
|
||||
"""Perform one HTTP request, classifying any failure. Returns the body."""
|
||||
name = SERVICES[service][0] if service in SERVICES else service
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=timeout) as resp:
|
||||
return resp.read()
|
||||
except urllib.error.HTTPError as exc:
|
||||
raise rejected(name, what, exc.code, exc.read().decode("utf-8", errors="replace")) from exc
|
||||
except (urllib.error.URLError, TimeoutError, ConnectionError) as exc:
|
||||
reason = getattr(exc, "reason", exc)
|
||||
raise unreachable(service, _origin(request), str(reason)) from exc
|
||||
|
||||
|
||||
def call_json(request: urllib.request.Request, *, service: str, what: str, timeout: float) -> Any:
|
||||
"""`call()`, then parse JSON — an unparseable body is the service's fault, named as such."""
|
||||
body = call(request, service=service, what=what, timeout=timeout)
|
||||
try:
|
||||
return json.loads(body)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ReachError(
|
||||
f"{what}: the service answered with something that is not JSON: {body[:200]!r}",
|
||||
fix="the service may be starting up or misconfigured — check its logs on tower-of-joy",
|
||||
) from exc
|
||||
|
||||
|
||||
def _origin(request: urllib.request.Request) -> str:
|
||||
"""scheme://host:port of a request — the part that identifies the box."""
|
||||
parts = request.full_url.split("/")
|
||||
return "/".join(parts[:3])
|
||||
Executable
+149
@@ -0,0 +1,149 @@
|
||||
"""Gemini image-generation connector — direct API wrapper.
|
||||
|
||||
Generates images through Google's gemini-2.5-flash-image model. The key comes
|
||||
from GEMINI_API_KEY in the environment only (endpoints.get_api_key). **Every
|
||||
generate call costs real money**; `health` only lists models and is free.
|
||||
|
||||
Formerly tooling/db/image_connector.py (T-1290). Unchanged except that
|
||||
failures raise a ReachError instead of printing `{"ok": false}` and exiting 1,
|
||||
and a network failure is reported as the network rather than as the API.
|
||||
The key travels in the URL, so no error message ever includes the URL's query.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import urllib.request
|
||||
|
||||
from tooling.core import console
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.domains.assets import endpoints
|
||||
|
||||
SERVICE = "Gemini API"
|
||||
MODEL = "gemini-2.5-flash-image"
|
||||
API = "https://generativelanguage.googleapis.com/v1beta"
|
||||
DEFAULT_OUTPUT_DIR = os.path.expanduser("~/Pictures/mcp-images")
|
||||
|
||||
# Real API values for imageConfig.aspectRatio (not a prompt hint).
|
||||
# https://ai.google.dev/gemini-api/docs/image-generation
|
||||
ASPECT_RATIOS = ("1:1", "3:2", "2:3", "3:4", "4:3", "4:5", "5:4", "9:16", "16:9", "21:9")
|
||||
|
||||
MIME_TYPES = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp"}
|
||||
|
||||
|
||||
def get_api_key() -> str:
|
||||
return endpoints.get_api_key("GEMINI_API_KEY")
|
||||
|
||||
|
||||
def health() -> dict:
|
||||
"""Is the Gemini API reachable with the configured key? (Free — lists models.)"""
|
||||
key = get_api_key()
|
||||
data = endpoints.call_json(
|
||||
urllib.request.Request(f"{API}/models?key={key}", method="GET"),
|
||||
service=SERVICE,
|
||||
what="the model listing",
|
||||
timeout=10,
|
||||
)
|
||||
models = [
|
||||
m.get("name", "")
|
||||
for m in data.get("models", [])
|
||||
if "imagen" in m.get("name", "").lower() or "flash" in m.get("name", "").lower()
|
||||
]
|
||||
return {"ok": True, "api": "gemini", "image_capable_models": models[:5]}
|
||||
|
||||
|
||||
def default_output(prompt: str) -> str:
|
||||
safe = "".join(c if c.isalnum() or c in "-_ " else "" for c in prompt[:40])
|
||||
safe = safe.strip().replace(" ", "_").lower()
|
||||
return os.path.join(DEFAULT_OUTPUT_DIR, f"{safe}.png")
|
||||
|
||||
|
||||
def build_request_body(
|
||||
prompt: str,
|
||||
aspect_ratio: str | None = "1:1",
|
||||
image_size: str | None = None,
|
||||
input_image: str | None = None,
|
||||
) -> dict:
|
||||
"""The generateContent payload — pure, so it can be tested without a call."""
|
||||
parts = []
|
||||
if input_image:
|
||||
if not os.path.isfile(input_image):
|
||||
raise ReachError(
|
||||
f"input image not found: {input_image}",
|
||||
fix="pass --input with an existing .png/.jpg/.webp",
|
||||
)
|
||||
with open(input_image, "rb") as f:
|
||||
data = base64.b64encode(f.read()).decode("utf-8")
|
||||
mime = MIME_TYPES.get(os.path.splitext(input_image)[1].lower(), "image/png")
|
||||
parts.append({"inlineData": {"mimeType": mime, "data": data}})
|
||||
|
||||
# The size is a best-effort prompt hint only: this model has no resolution
|
||||
# parameter, unlike the aspect ratio below.
|
||||
parts.append({"text": prompt + (f" Resolution: {image_size}." if image_size else "")})
|
||||
|
||||
generation_config: dict = {"responseModalities": ["TEXT", "IMAGE"]}
|
||||
if aspect_ratio:
|
||||
generation_config["imageConfig"] = {"aspectRatio": aspect_ratio}
|
||||
|
||||
return {"contents": [{"parts": parts}], "generationConfig": generation_config}
|
||||
|
||||
|
||||
def generate(
|
||||
prompt: str,
|
||||
output: str | None = None,
|
||||
aspect_ratio: str = "1:1",
|
||||
image_size: str | None = None,
|
||||
input_image: str | None = None,
|
||||
) -> dict:
|
||||
"""Generate one image. COSTS MONEY. Returns the result dict."""
|
||||
key = get_api_key()
|
||||
body = build_request_body(prompt, aspect_ratio, image_size, input_image)
|
||||
output = output or default_output(prompt)
|
||||
|
||||
console.event(f"Generating image: {prompt!r}" + (f" (from {input_image})" if input_image else ""))
|
||||
result = endpoints.call_json(
|
||||
urllib.request.Request(
|
||||
f"{API}/models/{MODEL}:generateContent?key={key}",
|
||||
data=json.dumps(body).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
),
|
||||
service=SERVICE,
|
||||
what="the generation request",
|
||||
timeout=120,
|
||||
)
|
||||
|
||||
candidates = result.get("candidates", [])
|
||||
if not candidates:
|
||||
raise ReachError(
|
||||
f"Gemini returned no candidates: {json.dumps(result)[:500]}",
|
||||
fix="the prompt may have been blocked by safety filters — rephrase it and re-run",
|
||||
)
|
||||
|
||||
image_saved = False
|
||||
text_response = ""
|
||||
for candidate in candidates:
|
||||
for part in candidate.get("content", {}).get("parts", []):
|
||||
if "inlineData" in part:
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
|
||||
with open(output, "wb") as f:
|
||||
f.write(base64.b64decode(part["inlineData"]["data"]))
|
||||
image_saved = True
|
||||
elif "text" in part:
|
||||
text_response += part["text"]
|
||||
|
||||
if not image_saved:
|
||||
raise ReachError(
|
||||
f"Gemini answered with text but no image: {text_response[:500]!r}",
|
||||
fix="make the prompt ask for an image explicitly, then re-run",
|
||||
)
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": os.path.getsize(output),
|
||||
"prompt": prompt,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
}
|
||||
@@ -0,0 +1,306 @@
|
||||
"""Transport for the `assets` domain — args in, delegate, format out.
|
||||
|
||||
A verb's result is printed to stdout as JSON with the same keys the old
|
||||
connector scripts printed, so a skill that reads `file`, `ogg_file` or
|
||||
`size_bytes` keeps working. What changed is failure: it is a non-zero exit
|
||||
with a remedy on the event stream, never `{"ok": false}` on stdout — so a
|
||||
caller checks the exit status, not a field.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from tooling.core import cli, console
|
||||
from tooling.core.command import command
|
||||
from tooling.core.errors import ReachError, unknown_choice
|
||||
|
||||
app = cli.domain("assets", "Asset generators — Stable Audio, Gemini images, Trellis 3D.")
|
||||
audio_app = cli.domain("audio", "Stable Audio Open on tower-of-joy :11500 (kept OFF — VRAM).")
|
||||
post_app = cli.domain("post", "ffmpeg post-processing: trim, normalize, convert.")
|
||||
image_app = cli.domain("image", "Gemini image generation — every generate COSTS MONEY.")
|
||||
trellis_app = cli.domain("trellis", "Trellis image-to-3D on tower-of-joy :11510 (kept OFF — VRAM).")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _domain() -> None:
|
||||
"""Keeps `assets` a group (Typer collapses a single-command app)."""
|
||||
|
||||
|
||||
@audio_app.callback()
|
||||
def _audio() -> None:
|
||||
"""Keeps `audio` a group."""
|
||||
|
||||
|
||||
@post_app.callback()
|
||||
def _post() -> None:
|
||||
"""Keeps `post` a group."""
|
||||
|
||||
|
||||
@image_app.callback()
|
||||
def _image() -> None:
|
||||
"""Keeps `image` a group."""
|
||||
|
||||
|
||||
@trellis_app.callback()
|
||||
def _trellis() -> None:
|
||||
"""Keeps `trellis` a group."""
|
||||
|
||||
|
||||
audio_app.add_typer(post_app, name="post")
|
||||
app.add_typer(audio_app, name="audio")
|
||||
app.add_typer(image_app, name="image")
|
||||
app.add_typer(trellis_app, name="trellis")
|
||||
|
||||
|
||||
def _emit(result: dict) -> None:
|
||||
console.out(json.dumps(result, indent=2))
|
||||
|
||||
|
||||
def _emit_summary(summary: dict, what: str) -> None:
|
||||
"""Print a batch summary, then fail the command if any item failed."""
|
||||
_emit(summary)
|
||||
if summary.get("failed"):
|
||||
raise ReachError(
|
||||
f"{what}: {summary['failed']} of {summary['total']} failed (details in the summary above)",
|
||||
fix="fix the failing items, then re-run with --skip-existing to leave the rest alone",
|
||||
)
|
||||
console.verdict(f"{what}: {summary.get('success', 0)} done, {summary.get('skipped', 0)} skipped")
|
||||
|
||||
|
||||
# --- audio ----------------------------------------------------------------
|
||||
|
||||
|
||||
@audio_app.command("health")
|
||||
@command
|
||||
def audio_health() -> None:
|
||||
"""Is Stable Audio up? OFF is its normal resting state — this says which."""
|
||||
from tooling.domains.assets import audio
|
||||
|
||||
_emit(audio.health())
|
||||
|
||||
|
||||
@audio_app.command("generate")
|
||||
@command
|
||||
def audio_generate(
|
||||
prompt: str = typer.Argument(..., help="What the audio should sound like."),
|
||||
duration: float = typer.Option(10.0, "--duration", help="Seconds, 0-47."),
|
||||
steps: int = typer.Option(100, "--steps", help="Diffusion steps; fewer is faster and worse."),
|
||||
cfg: float = typer.Option(7.0, "--cfg", help="Classifier-free guidance scale."),
|
||||
output: Path = typer.Option(None, "--output", help="WAV path (default: named from the prompt)."),
|
||||
timeout: int = typer.Option(600, "--timeout", help="Seconds to wait for the result."),
|
||||
post: bool = typer.Option(False, "--post", help="Also trim + normalize + convert to OGG."),
|
||||
output_ogg: Path = typer.Option(None, "--output-ogg", help="OGG path; implies --post."),
|
||||
) -> None:
|
||||
"""Generate audio from a prompt; prints the result JSON."""
|
||||
from tooling.domains.assets import audio
|
||||
|
||||
_emit(
|
||||
audio.generate(
|
||||
prompt,
|
||||
duration=duration,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
output=str(output) if output else None,
|
||||
timeout=timeout,
|
||||
post=post,
|
||||
output_ogg=str(output_ogg) if output_ogg else None,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@audio_app.command("batch")
|
||||
@command
|
||||
def audio_batch(
|
||||
manifest: Path = typer.Argument(..., help="Manifest .json (see the audio-gen skill)."),
|
||||
dry_run: bool = typer.Option(False, "--dry-run", help="List what would be generated."),
|
||||
only: str = typer.Option(None, "--only", help="Comma-separated asset ids."),
|
||||
skip_existing: bool = typer.Option(False, "--skip-existing", help="Leave existing OGGs alone."),
|
||||
) -> None:
|
||||
"""Generate every asset in a manifest; fails if any asset failed."""
|
||||
from tooling.domains.assets import audio_batch as service
|
||||
|
||||
summary = service.run(
|
||||
str(manifest),
|
||||
dry_run=dry_run,
|
||||
only=set(only.split(",")) if only else None,
|
||||
skip_existing=skip_existing,
|
||||
)
|
||||
_emit_summary(summary, "audio batch")
|
||||
|
||||
|
||||
@post_app.command("convert")
|
||||
@command
|
||||
def post_convert(
|
||||
input: Path = typer.Argument(..., help="Input WAV."),
|
||||
output: Path = typer.Option(None, "--output", "-o", help="Output (default: same name .ogg)."),
|
||||
quality: int = typer.Option(6, "--quality", "-q", help="Vorbis quality 0-10."),
|
||||
) -> None:
|
||||
"""WAV → OGG (libvorbis)."""
|
||||
from tooling.domains.assets import audio_post
|
||||
|
||||
_emit(audio_post.convert(str(input), str(output) if output else None, quality))
|
||||
|
||||
|
||||
@post_app.command("normalize")
|
||||
@command
|
||||
def post_normalize(
|
||||
input: Path = typer.Argument(..., help="Input audio file."),
|
||||
output: Path = typer.Option(None, "--output", "-o", help="Output (default: <name>_norm)."),
|
||||
lufs: float = typer.Option(-16, "--lufs", help="Target loudness, LUFS."),
|
||||
) -> None:
|
||||
"""LUFS-normalize an audio file."""
|
||||
from tooling.domains.assets import audio_post
|
||||
|
||||
_emit(audio_post.normalize(str(input), str(output) if output else None, lufs))
|
||||
|
||||
|
||||
@post_app.command("trim")
|
||||
@command
|
||||
def post_trim(
|
||||
input: Path = typer.Argument(..., help="Input audio file."),
|
||||
output: Path = typer.Option(None, "--output", "-o", help="Output (default: <name>_trimmed)."),
|
||||
threshold: int = typer.Option(-50, "--threshold", help="Silence threshold, dB."),
|
||||
) -> None:
|
||||
"""Trim leading and trailing silence."""
|
||||
from tooling.domains.assets import audio_post
|
||||
|
||||
_emit(audio_post.trim(str(input), str(output) if output else None, threshold))
|
||||
|
||||
|
||||
@post_app.command("pipeline")
|
||||
@command
|
||||
def post_pipeline(
|
||||
input: Path = typer.Argument(..., help="Input WAV."),
|
||||
output: Path = typer.Option(None, "--output", "-o", help="Output OGG (default: same name .ogg)."),
|
||||
quality: int = typer.Option(6, "--quality", "-q", help="Vorbis quality 0-10."),
|
||||
lufs: float = typer.Option(-16, "--lufs", help="Target loudness, LUFS."),
|
||||
threshold: int = typer.Option(-50, "--threshold", help="Silence threshold, dB."),
|
||||
) -> None:
|
||||
"""trim → normalize → convert to OGG, the full chain."""
|
||||
from tooling.domains.assets import audio_post
|
||||
|
||||
_emit(audio_post.pipeline(str(input), str(output) if output else None, quality, lufs, threshold))
|
||||
|
||||
|
||||
# --- image ----------------------------------------------------------------
|
||||
|
||||
|
||||
@image_app.command("health")
|
||||
@command
|
||||
def image_health() -> None:
|
||||
"""Is the Gemini API reachable with GEMINI_API_KEY? Free — lists models."""
|
||||
from tooling.domains.assets import image
|
||||
|
||||
_emit(image.health())
|
||||
|
||||
|
||||
@image_app.command("generate")
|
||||
@command
|
||||
def image_generate(
|
||||
prompt: str = typer.Argument(..., help="What to draw."),
|
||||
output: Path = typer.Option(None, "--output", help="PNG path (default: ~/Pictures/mcp-images/)."),
|
||||
aspect: str = typer.Option("1:1", "--aspect", help="Aspect ratio, e.g. 1:1, 16:9, 3:4."),
|
||||
size: str = typer.Option(None, "--size", help="Resolution hint (1K/2K/4K) — may be ignored."),
|
||||
input: Path = typer.Option(None, "--input", help="Source image, for image-to-image."),
|
||||
) -> None:
|
||||
"""Generate one image. COSTS MONEY per call."""
|
||||
from tooling.domains.assets import image
|
||||
|
||||
if aspect not in image.ASPECT_RATIOS:
|
||||
raise unknown_choice("aspect ratio", aspect, image.ASPECT_RATIOS)
|
||||
_emit(
|
||||
image.generate(
|
||||
prompt,
|
||||
output=str(output) if output else None,
|
||||
aspect_ratio=aspect,
|
||||
image_size=size,
|
||||
input_image=str(input) if input else None,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# --- trellis --------------------------------------------------------------
|
||||
|
||||
|
||||
@trellis_app.command("health")
|
||||
@command
|
||||
def trellis_health() -> None:
|
||||
"""Is Trellis up? OFF is its normal resting state — this says which."""
|
||||
from tooling.domains.assets import trellis
|
||||
|
||||
_emit(trellis.health())
|
||||
|
||||
|
||||
@trellis_app.command("generate")
|
||||
@command
|
||||
def trellis_generate(
|
||||
image: Path = typer.Argument(..., help="Input image (PNG recommended)."),
|
||||
output: Path = typer.Option(None, "--output", help=".glb path (default: named from the image)."),
|
||||
simplify: float = typer.Option(0.95, "--simplify", help="Mesh simplification, 0.9-0.98."),
|
||||
texture_size: int = typer.Option(1024, "--texture-size", help="Texture resolution, 512-2048."),
|
||||
seed: int = typer.Option(0, "--seed", help="Random seed."),
|
||||
timeout: int = typer.Option(600, "--timeout", help="Seconds for the 3D step."),
|
||||
) -> None:
|
||||
"""Image → .glb; prints the result JSON."""
|
||||
from tooling.domains.assets import trellis
|
||||
|
||||
_emit(
|
||||
trellis.generate(
|
||||
str(image),
|
||||
output=str(output) if output else None,
|
||||
simplify=simplify,
|
||||
texture_size=texture_size,
|
||||
seed=seed,
|
||||
timeout=timeout,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@trellis_app.command("batch")
|
||||
@command
|
||||
def trellis_batch(
|
||||
input_dir: Path = typer.Option(None, "--input-dir", help="Directory of .png inputs (default: the character bodies)."),
|
||||
output_dir: Path = typer.Option(None, "--output-dir", help="Where the .glb files go."),
|
||||
names: str = typer.Option(None, "--names", help="Comma-separated stems; default: every .png in --input-dir."),
|
||||
simplify: float = typer.Option(0.95, "--simplify", help="Mesh simplification."),
|
||||
texture_size: int = typer.Option(1024, "--texture-size", help="Texture resolution."),
|
||||
cooldown: float = typer.Option(15, "--cooldown", help="Seconds between successful jobs."),
|
||||
retries: int = typer.Option(3, "--retries", help="Attempts per item."),
|
||||
retry_delay: float = typer.Option(60, "--retry-delay", help="Seconds before a retry."),
|
||||
) -> None:
|
||||
"""One .glb per image, gently — long; consider --detach. Fails if any item failed."""
|
||||
from tooling.domains.assets import trellis
|
||||
|
||||
summary = trellis.batch(
|
||||
str(input_dir) if input_dir else trellis.BODIES_INPUT,
|
||||
str(output_dir) if output_dir else trellis.BODIES_OUTPUT,
|
||||
names=names.split(",") if names else None,
|
||||
simplify=simplify,
|
||||
texture_size=texture_size,
|
||||
cooldown=cooldown,
|
||||
max_retries=retries,
|
||||
retry_delay=retry_delay,
|
||||
)
|
||||
_emit_summary(summary, "trellis batch")
|
||||
|
||||
|
||||
# --- local synthesis ------------------------------------------------------
|
||||
|
||||
|
||||
@app.command("synth-ui")
|
||||
@command
|
||||
def synth_ui(
|
||||
output_dir: Path = typer.Option(None, "--output-dir", help="Where the WAVs go (default: client/assets/audio)."),
|
||||
) -> None:
|
||||
"""Synthesize the four insert-tech UI sounds locally — no service needed."""
|
||||
from tooling.domains.assets import synth_ui as service
|
||||
|
||||
paths = service.run(str(output_dir) if output_dir else service.OUTPUT_DIR)
|
||||
console.out("\n".join(paths))
|
||||
console.verdict(
|
||||
f"synth-ui: wrote {len(paths)} WAVs — convert with `reach assets audio post convert <file.wav>`"
|
||||
)
|
||||
@@ -6,19 +6,27 @@ Each sound uses a DIFFERENT synthesis technique to ensure distinct character:
|
||||
- weapon_aim: filtered noise + sub thump (mechanical)
|
||||
- monologue_chime: FM synthesis (crystalline bell)
|
||||
- monologue_chime_urgent: FM synthesis + beating/dissonance (tense bell)
|
||||
|
||||
Formerly tooling/synth_ui_sounds.py (T-1290). The synthesis is untouched —
|
||||
the four WAVs are byte-identical before and after the port. The output
|
||||
directory became a parameter; the default is still client/assets/audio.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import wave
|
||||
import os
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
|
||||
from tooling.core import config, console
|
||||
|
||||
SR = 44100
|
||||
OUTPUT_DIR = os.path.join(os.path.dirname(__file__), "..", "client", "assets", "audio")
|
||||
OUTPUT_DIR = str(config.repo_root() / "client" / "assets" / "audio")
|
||||
_output_dir = OUTPUT_DIR
|
||||
|
||||
|
||||
def write_wav(filename, samples, channels=1):
|
||||
"""Write float samples [-1, 1] to 16-bit WAV."""
|
||||
path = os.path.join(OUTPUT_DIR, filename)
|
||||
path = os.path.join(_output_dir, filename)
|
||||
# Normalize to peak if it exceeds 1.0
|
||||
peak = np.max(np.abs(samples))
|
||||
if peak > 1.0:
|
||||
@@ -30,7 +38,7 @@ def write_wav(filename, samples, channels=1):
|
||||
f.setframerate(SR)
|
||||
f.writeframes(int_samples.tobytes())
|
||||
dur_ms = len(int_samples) / SR * 1000
|
||||
print(f" wrote {path} ({dur_ms:.0f}ms, {channels}ch)")
|
||||
console.event(f"wrote {path} ({dur_ms:.0f}ms, {channels}ch)")
|
||||
return path
|
||||
|
||||
|
||||
@@ -185,22 +193,17 @@ def monologue_chime_urgent():
|
||||
return write_wav("sfx_monologue_chime_urgent.wav", signal * envelope * 0.25)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
print("Synthesizing UI sounds for Sprint 7 #440...")
|
||||
print()
|
||||
|
||||
print("[UI-001] cursor_hover — bandpass noise impulse (digital click)")
|
||||
cursor_hover()
|
||||
|
||||
print("[UI-004] weapon_aim — filtered noise + sub thump (mechanical latch)")
|
||||
weapon_aim()
|
||||
|
||||
print("[UI-005] sfx_monologue_chime — FM synthesis bell (PLACEHOLDER)")
|
||||
monologue_chime()
|
||||
|
||||
print("[UI-006] sfx_monologue_chime_urgent — FM + beating (PLACEHOLDER)")
|
||||
monologue_chime_urgent()
|
||||
|
||||
print()
|
||||
print("Done. Convert with: tooling/db/audio-post convert <file.wav>")
|
||||
def run(output_dir: str = OUTPUT_DIR) -> list[str]:
|
||||
"""Write all four UI sounds into output_dir; returns their paths."""
|
||||
global _output_dir
|
||||
_output_dir = output_dir
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
console.event("[UI-001] cursor_hover — bandpass noise impulse (digital click)")
|
||||
paths = [cursor_hover()]
|
||||
console.event("[UI-004] weapon_aim — filtered noise + sub thump (mechanical latch)")
|
||||
paths.append(weapon_aim())
|
||||
console.event("[UI-005] sfx_monologue_chime — FM synthesis bell (PLACEHOLDER)")
|
||||
paths.append(monologue_chime())
|
||||
console.event("[UI-006] sfx_monologue_chime_urgent — FM + beating (PLACEHOLDER)")
|
||||
paths.append(monologue_chime_urgent())
|
||||
return paths
|
||||
Executable
+315
@@ -0,0 +1,315 @@
|
||||
"""Trellis 3D model generator connector — Gradio API wrapper.
|
||||
|
||||
Talks to the Trellis Gradio app on tower-of-joy :11510 (URL from config.json).
|
||||
Pipeline: start session → upload + preprocess → seed → image_to_3d →
|
||||
extract_glb → download .glb.
|
||||
|
||||
Formerly tooling/db/trellis_connector.py plus the tooling/trellis-batch.sh loop
|
||||
(T-1290). `batch` is that loop in Python, and no longer hardcoded to the 16
|
||||
character bodies — it takes a directory, or explicit names.
|
||||
|
||||
Gradio API parameter reference (TRELLIS v1, microsoft/TRELLIS):
|
||||
|
||||
/image_to_3d — 9 inputs:
|
||||
0: image (Image) preprocessed image from /preprocess_image_1
|
||||
1: multiimages (Gallery) [] for single-image mode
|
||||
2: is_multiimage (State) False for single-image, True for multi-image
|
||||
3: seed (Slider) int, 0-2147483647
|
||||
4: ss_guidance (Slider) float, sparse structure guidance strength (default 7.5)
|
||||
5: ss_steps (Slider) int, sparse structure sampling steps (default 12)
|
||||
6: slat_guidance (Slider) float, structured latent guidance strength (default 3.0)
|
||||
7: slat_steps (Slider) int, structured latent sampling steps (default 12)
|
||||
8: multiimage_algo (Radio) "stochastic" or "multidiffusion"
|
||||
|
||||
/extract_glb — 3 inputs:
|
||||
0: output_buf (State) None — server uses internal state from image_to_3d
|
||||
1: simplify (Slider) float, mesh simplification ratio (default 0.95)
|
||||
2: texture_size (Slider) int, texture resolution (default 1024)
|
||||
|
||||
Common failure modes:
|
||||
- "needed 9, got 8": missing is_multiimage (position 2) — must pass False
|
||||
- "needed 3, got 2": missing output_buf (position 0) — must pass None
|
||||
- "'float' cannot be interpreted as int": numpy version issue on server,
|
||||
Gradio Sliders send all values as float. Fix: patch flow_euler.py on
|
||||
the server to cast steps to int, or pin numpy < 2.0
|
||||
- CUDA device mismatch after crash: restart the container to clear GPU state
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import string
|
||||
import time
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from tooling.core import config, console
|
||||
from tooling.core.errors import ReachError
|
||||
from tooling.domains.assets import endpoints
|
||||
|
||||
SERVICE = "trellis"
|
||||
|
||||
# The batch defaults are the character-body run the bash script was written for.
|
||||
BODIES_INPUT = ".tmp/image-gen/characters/bodies"
|
||||
BODIES_OUTPUT = ".tmp/glb-gen/characters/bodies"
|
||||
|
||||
|
||||
def get_base_url() -> str:
|
||||
return endpoints.base_url(SERVICE)
|
||||
|
||||
|
||||
def health() -> dict:
|
||||
"""Is the Trellis API reachable, and what endpoints does it name?"""
|
||||
base = get_base_url()
|
||||
data = endpoints.call_json(
|
||||
urllib.request.Request(f"{base}/info", method="GET"),
|
||||
service=SERVICE,
|
||||
what="the health check",
|
||||
timeout=10,
|
||||
)
|
||||
return {"ok": True, "url": base, "endpoints": list(data.get("named_endpoints", {}).keys())}
|
||||
|
||||
|
||||
def _call_api(base: str, endpoint: str, data: list, timeout: float = 600, session_hash: str | None = None):
|
||||
"""POST one Gradio endpoint, sharing a session so gr.State survives between calls.
|
||||
|
||||
image_to_3d stores its output in server-side State keyed by session_hash;
|
||||
extract_glb reads it back. Without a shared session the state is lost.
|
||||
"""
|
||||
body: dict = {"data": data}
|
||||
if session_hash:
|
||||
body["session_hash"] = session_hash
|
||||
console.event(f"Calling {endpoint}...")
|
||||
result = endpoints.call_json(
|
||||
urllib.request.Request(
|
||||
f"{base}/api{endpoint}",
|
||||
data=json.dumps(body).encode(),
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
),
|
||||
service=SERVICE,
|
||||
what=endpoint,
|
||||
timeout=timeout,
|
||||
)
|
||||
if isinstance(result, dict) and "data" in result:
|
||||
return result["data"]
|
||||
return result
|
||||
|
||||
|
||||
def multipart_body(filename: str, payload: bytes, boundary: str = "----TrellisConnectorBoundary") -> bytes:
|
||||
"""The multipart/form-data body Gradio's /upload expects — a 'files' field."""
|
||||
head = (
|
||||
f"--{boundary}\r\n"
|
||||
f'Content-Disposition: form-data; name="files"; filename="{filename}"\r\n'
|
||||
"Content-Type: image/png\r\n"
|
||||
"\r\n"
|
||||
).encode()
|
||||
return head + payload + f"\r\n--{boundary}--\r\n".encode()
|
||||
|
||||
|
||||
def _upload_image(base: str, image_path: str):
|
||||
filename = os.path.basename(image_path)
|
||||
with open(image_path, "rb") as f:
|
||||
body = multipart_body(filename, f.read())
|
||||
console.event(f"Uploading {filename}...")
|
||||
result = endpoints.call_json(
|
||||
urllib.request.Request(
|
||||
f"{base}/upload",
|
||||
data=body,
|
||||
headers={"Content-Type": "multipart/form-data; boundary=----TrellisConnectorBoundary"},
|
||||
method="POST",
|
||||
),
|
||||
service=SERVICE,
|
||||
what="the image upload",
|
||||
timeout=30,
|
||||
)
|
||||
if isinstance(result, list) and result:
|
||||
return result[0]
|
||||
raise ReachError(
|
||||
f"Trellis accepted the upload but returned no file reference: {result}",
|
||||
fix="the Gradio app's upload API may have changed — check `reach assets trellis health`",
|
||||
)
|
||||
|
||||
|
||||
def _download_file(url: str, output_path: str, base: str) -> int:
|
||||
if url.startswith("/"):
|
||||
url = f"{base}{url}"
|
||||
elif not url.startswith("http"):
|
||||
url = f"{base}/file={url}"
|
||||
console.event(f"Downloading to {output_path}...")
|
||||
body = endpoints.call(
|
||||
urllib.request.Request(url, method="GET"), service=SERVICE, what="the GLB download", timeout=120
|
||||
)
|
||||
with open(output_path, "wb") as f:
|
||||
f.write(body)
|
||||
return os.path.getsize(output_path)
|
||||
|
||||
|
||||
def glb_url_from(glb_result) -> str | None:
|
||||
"""extract_glb returns [model_viewer_data, download_button_data]; take the first URL."""
|
||||
if isinstance(glb_result, list):
|
||||
for item in glb_result:
|
||||
if isinstance(item, dict):
|
||||
url = item.get("url") or item.get("path")
|
||||
if url:
|
||||
return url
|
||||
return None
|
||||
|
||||
|
||||
def generate(
|
||||
image_path: str,
|
||||
output: str | None = None,
|
||||
simplify: float = 0.95,
|
||||
texture_size: int = 1024,
|
||||
seed: int = 0,
|
||||
timeout: int = 600,
|
||||
) -> dict:
|
||||
"""Image → .glb. Returns the result dict."""
|
||||
# The input is checked before the service, so a typo does not read as an
|
||||
# outage. (The old check also crashed: it passed indent= to print().)
|
||||
if not os.path.isfile(image_path):
|
||||
raise ReachError(f"image not found: {image_path}", fix="pass an existing .png")
|
||||
|
||||
base = get_base_url()
|
||||
endpoints.call(
|
||||
urllib.request.Request(f"{base}/info", method="GET"),
|
||||
service=SERVICE,
|
||||
what="the availability check",
|
||||
timeout=5,
|
||||
)
|
||||
start_time = time.time()
|
||||
output = output or f"{os.path.splitext(os.path.basename(image_path))[0]}.glb"
|
||||
|
||||
session = "".join(random.choices(string.ascii_lowercase + string.digits, k=12))
|
||||
console.event(f"Session: {session}")
|
||||
|
||||
console.event("Starting session...", phase="1/5")
|
||||
_call_api(base, "/start_session", [], timeout=30, session_hash=session)
|
||||
|
||||
console.event("Uploading and preprocessing image...", phase="2/5")
|
||||
uploaded = _upload_image(base, image_path)
|
||||
file_ref = {"path": uploaded, "meta": {"_type": "gradio.FileData"}}
|
||||
preprocessed = _call_api(base, "/preprocess_image_1", [file_ref], timeout=60, session_hash=session)
|
||||
preprocessed_ref = preprocessed[0] if isinstance(preprocessed, list) and preprocessed else preprocessed
|
||||
|
||||
console.event("Generating 3D model...", phase="3/5")
|
||||
seed_result = _call_api(base, "/get_seed", [True, seed], timeout=10, session_hash=session)
|
||||
actual_seed = seed_result[0] if isinstance(seed_result, list) and seed_result else seed
|
||||
|
||||
# Nine inputs, positions per the reference above. The server needs
|
||||
# flow_euler.py patched to cast steps to int (numpy >= 2).
|
||||
_call_api(
|
||||
base,
|
||||
"/image_to_3d",
|
||||
[preprocessed_ref, [], False, actual_seed, 7.5, 12, 3.0, 12, "stochastic"],
|
||||
timeout=timeout,
|
||||
session_hash=session,
|
||||
)
|
||||
|
||||
console.event("Extracting GLB...", phase="4/5")
|
||||
glb_result = _call_api(base, "/extract_glb", [None, simplify, texture_size], timeout=120, session_hash=session)
|
||||
glb_url = glb_url_from(glb_result)
|
||||
if not glb_url:
|
||||
raise ReachError(
|
||||
f"could not find the GLB URL in the extract_glb response: {str(glb_result)[:300]}",
|
||||
fix="the Gradio app's response shape may have changed — see trellis.glb_url_from",
|
||||
)
|
||||
|
||||
console.event("Downloading GLB...", phase="5/5")
|
||||
os.makedirs(os.path.dirname(os.path.abspath(output)), exist_ok=True)
|
||||
size = _download_file(glb_url, output, base)
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"file": output,
|
||||
"size_bytes": size,
|
||||
"simplify": simplify,
|
||||
"texture_size": texture_size,
|
||||
"seed": actual_seed,
|
||||
"generation_time_s": round(time.time() - start_time, 1),
|
||||
"source_image": image_path,
|
||||
}
|
||||
|
||||
|
||||
def batch_plan(input_dir: Path, output_dir: Path, names: list[str] | None) -> list[tuple[str, Path, Path]]:
|
||||
"""What a batch would do: (name, input png, output glb) per item — pure."""
|
||||
if names:
|
||||
chosen = names
|
||||
else:
|
||||
chosen = sorted(p.stem for p in input_dir.glob("*.png"))
|
||||
return [(n, input_dir / f"{n}.png", output_dir / f"{n}.glb") for n in chosen]
|
||||
|
||||
|
||||
def batch(
|
||||
input_dir: str = BODIES_INPUT,
|
||||
output_dir: str = BODIES_OUTPUT,
|
||||
names: list[str] | None = None,
|
||||
simplify: float = 0.95,
|
||||
texture_size: int = 1024,
|
||||
cooldown: float = 15,
|
||||
max_retries: int = 3,
|
||||
retry_delay: float = 60,
|
||||
) -> dict:
|
||||
"""Generate one .glb per input image, one at a time, gently.
|
||||
|
||||
Skips outputs that already exist. Retries each failure, and cools the GPU
|
||||
down between successful jobs — the Trellis box is shared (VRAM is scarce).
|
||||
Returns the summary; the router prints it and then fails the command if
|
||||
anything failed. The bash original exited 0 regardless.
|
||||
"""
|
||||
root = config.repo_root()
|
||||
in_dir = (root / input_dir) if not Path(input_dir).is_absolute() else Path(input_dir)
|
||||
out_dir = (root / output_dir) if not Path(output_dir).is_absolute() else Path(output_dir)
|
||||
plan = batch_plan(in_dir, out_dir, names)
|
||||
if not plan:
|
||||
raise ReachError(f"no .png inputs in {in_dir}", fix="pass --input-dir with images, or --names")
|
||||
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
total = len(plan)
|
||||
results = []
|
||||
console.event(f"Trellis batch: {total} item(s), cooldown {cooldown}s, {max_retries} tries each")
|
||||
|
||||
for i, (name, src, dst) in enumerate(plan, 1):
|
||||
phase = f"{i}/{total}"
|
||||
if dst.exists():
|
||||
console.event(f"{name} — already exists, skipping", phase=phase)
|
||||
results.append({"name": name, "status": "skipped"})
|
||||
continue
|
||||
if not src.exists():
|
||||
console.event(f"{name} — input not found: {src}", phase=phase, level="warn")
|
||||
results.append({"name": name, "status": "failed", "error": f"input not found: {src}"})
|
||||
continue
|
||||
|
||||
error = None
|
||||
for attempt in range(1, max_retries + 1):
|
||||
console.event(f"{name} (attempt {attempt}/{max_retries})...", phase=phase, progress=i / total)
|
||||
try:
|
||||
generate(str(src), output=str(dst), simplify=simplify, texture_size=texture_size)
|
||||
error = None
|
||||
break
|
||||
except ReachError as exc:
|
||||
error = exc.message
|
||||
console.event(f"{name} failed: {exc.message}", phase=phase, level="warn")
|
||||
if attempt < max_retries:
|
||||
console.event(f"waiting {retry_delay}s before retry...", phase=phase)
|
||||
time.sleep(retry_delay)
|
||||
|
||||
if error is None:
|
||||
results.append({"name": name, "status": "success", "file": str(dst)})
|
||||
if i < total:
|
||||
console.event(f"cooling down {cooldown}s...", phase=phase)
|
||||
time.sleep(cooldown)
|
||||
else:
|
||||
results.append({"name": name, "status": "failed", "error": error})
|
||||
|
||||
summary = {
|
||||
"ok": not any(r["status"] == "failed" for r in results),
|
||||
"total": total,
|
||||
"success": sum(r["status"] == "success" for r in results),
|
||||
"skipped": sum(r["status"] == "skipped" for r in results),
|
||||
"failed": sum(r["status"] == "failed" for r in results),
|
||||
"results": results,
|
||||
}
|
||||
return summary
|
||||
@@ -67,6 +67,10 @@ DOMAINS: dict[str, tuple[str, str]] = {
|
||||
"tooling.domains.wiki.router:app",
|
||||
"The wiki seed — fill rates and the GTTR hook",
|
||||
),
|
||||
"assets": (
|
||||
"tooling.domains.assets.router:app",
|
||||
"Asset generators — Stable Audio, Gemini images, Trellis 3D",
|
||||
),
|
||||
"godot": (
|
||||
"tooling.domains.godot.router:app",
|
||||
"Does the client parse, and does it parse cold",
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
#!/usr/bin/env python3
|
||||
"""`reach assets` connectors against a fake Gradio server (T-1290).
|
||||
|
||||
The real services cannot be used in a gate: Stable Audio and Trellis are kept
|
||||
switched OFF (VRAM on tower-of-joy), and every Gemini call costs money. So the
|
||||
decisions are exercised without performing them (D-263): a local HTTP server
|
||||
plays the Gradio apps, and the tests assert on what each connector SENDS and
|
||||
on how it classifies each kind of failure.
|
||||
|
||||
What is pinned:
|
||||
- audio: submit payload [prompt, duration, steps, cfg], SSE `complete` → file
|
||||
download, SSE `error` → fails with the error, relative file URL resolution.
|
||||
- trellis: the 5-call session sequence with one shared session_hash, the
|
||||
9-input image_to_3d payload, the 3-input extract_glb payload, the download.
|
||||
- image: the generateContent body (aspect ratio as imageConfig, size as a
|
||||
prompt hint, inlineData for image-to-image) — built, never sent.
|
||||
- endpoints: refused connection → "OFF" remedy; HTTP 500 → "rejected";
|
||||
401 → credentials; an unreachable non-tower service → network remedy.
|
||||
|
||||
Run: .venv/bin/python tooling/test_assets.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import urllib.request
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from tooling.core.errors import ReachError # noqa: E402
|
||||
from tooling.domains.assets import audio, endpoints, image, trellis # noqa: E402
|
||||
|
||||
AUDIO_BYTES = b"RIFF-fake-wav"
|
||||
GLB_BYTES = b"glTF-fake-glb"
|
||||
|
||||
|
||||
class FakeGradio(BaseHTTPRequestHandler):
|
||||
"""Just enough of both Gradio apps. Records every request it sees."""
|
||||
|
||||
seen: list[tuple[str, str, object]] = []
|
||||
sse_event = "complete"
|
||||
|
||||
def log_message(self, *args): # silence the default stderr access log
|
||||
pass
|
||||
|
||||
def _send(self, code: int, body: bytes, ctype: str = "application/json") -> None:
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", ctype)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def _json(self, obj, code: int = 200) -> None:
|
||||
self._send(code, json.dumps(obj).encode())
|
||||
|
||||
def do_GET(self):
|
||||
FakeGradio.seen.append(("GET", self.path, None))
|
||||
if self.path == "/config":
|
||||
return self._json({"version": "4.44", "dependencies": [{"api_name": "generate_audio"}, {"api_name": "js_x"}]})
|
||||
if self.path == "/info":
|
||||
return self._json({"named_endpoints": {"/image_to_3d": {}, "/extract_glb": {}}})
|
||||
if self.path.startswith("/gradio_api/call/generate_audio/"):
|
||||
if FakeGradio.sse_event == "error":
|
||||
body = b"event: error\ndata: \"CUDA out of memory\"\n\n"
|
||||
else:
|
||||
body = b"event: heartbeat\ndata: null\n\nevent: complete\ndata: [{\"url\": \"/file=out.wav\"}]\n\n"
|
||||
return self._send(200, body, "text/event-stream")
|
||||
if self.path == "/file=out.wav":
|
||||
return self._send(200, AUDIO_BYTES, "audio/wav")
|
||||
if self.path == "/file=model.glb":
|
||||
return self._send(200, GLB_BYTES, "model/gltf-binary")
|
||||
if self.path == "/boom":
|
||||
return self._json({"error": "kaboom"}, 500)
|
||||
if self.path == "/denied":
|
||||
return self._json({"error": "bad key"}, 401)
|
||||
return self._json({"error": "not found"}, 404)
|
||||
|
||||
def do_POST(self):
|
||||
length = int(self.headers.get("Content-Length", 0))
|
||||
raw = self.rfile.read(length)
|
||||
try:
|
||||
body = json.loads(raw)
|
||||
except (json.JSONDecodeError, UnicodeDecodeError):
|
||||
body = raw
|
||||
FakeGradio.seen.append(("POST", self.path, body))
|
||||
if self.path == "/gradio_api/call/generate_audio":
|
||||
return self._json({"event_id": "ev123"})
|
||||
if self.path == "/upload":
|
||||
return self._json(["/tmp/gradio/uploaded.png"])
|
||||
if self.path == "/api/preprocess_image_1":
|
||||
return self._json({"data": [{"path": "/tmp/pre.png"}]})
|
||||
if self.path == "/api/get_seed":
|
||||
return self._json({"data": [4242]})
|
||||
if self.path == "/api/extract_glb":
|
||||
return self._json({"data": [{"url": "/file=model.glb"}, {"url": "/file=model.glb"}]})
|
||||
if self.path.startswith("/api/"):
|
||||
return self._json({"data": []})
|
||||
return self._json({"error": "not found"}, 404)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def fake_server():
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), FakeGradio)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
base = f"http://127.0.0.1:{server.server_address[1]}"
|
||||
real = endpoints.load_config
|
||||
endpoints.load_config = lambda: {"stable_audio_url": base, "trellis_url": base}
|
||||
FakeGradio.seen = []
|
||||
FakeGradio.sse_event = "complete"
|
||||
try:
|
||||
yield base
|
||||
finally:
|
||||
endpoints.load_config = real
|
||||
server.shutdown()
|
||||
|
||||
|
||||
def _closed_port() -> int:
|
||||
with socket.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
|
||||
def quiet(fn, *args, **kwargs):
|
||||
with contextlib.redirect_stderr(io.StringIO()):
|
||||
return fn(*args, **kwargs)
|
||||
|
||||
|
||||
def check(failures: list[str], cond: bool, message: str) -> None:
|
||||
if not cond:
|
||||
failures.append(message)
|
||||
|
||||
|
||||
def test_audio(failures: list[str]) -> None:
|
||||
with fake_server(), tempfile.TemporaryDirectory() as tmp:
|
||||
out = str(Path(tmp) / "x.wav")
|
||||
result = quiet(audio.generate, "wind over dunes", duration=5, steps=40, cfg=6.5, output=out, timeout=20)
|
||||
posts = [s for s in FakeGradio.seen if s[0] == "POST"]
|
||||
check(failures, posts == [("POST", "/gradio_api/call/generate_audio", {"data": ["wind over dunes", 5, 40, 6.5]})],
|
||||
f"audio: submit payload changed: {posts}")
|
||||
check(failures, Path(out).read_bytes() == AUDIO_BYTES, "audio: downloaded bytes differ")
|
||||
check(failures, result["size_bytes"] == len(AUDIO_BYTES) and result["file"] == out,
|
||||
f"audio: result keys wrong: {result}")
|
||||
|
||||
FakeGradio.sse_event = "error"
|
||||
try:
|
||||
quiet(audio.generate, "x", output=out, timeout=20)
|
||||
failures.append("audio: an SSE error event did not fail the generation")
|
||||
except ReachError as exc:
|
||||
check(failures, "CUDA out of memory" in exc.message, f"audio: error detail lost: {exc.message}")
|
||||
|
||||
health = quiet(audio.health)
|
||||
check(failures, health["api_endpoints"] == ["generate_audio"], f"audio: js_ endpoints not filtered: {health}")
|
||||
|
||||
|
||||
def test_audio_urls(failures: list[str]) -> None:
|
||||
base = "http://h:1"
|
||||
cases = {
|
||||
"abs": ([{"url": "http://other/f.wav"}], "http://other/f.wav"),
|
||||
"rooted": ([{"path": "/file=a.wav"}], "http://h:1/file=a.wav"),
|
||||
"bare": (["tmp/a.wav"], "http://h:1/file=tmp/a.wav"),
|
||||
"wrapped": ({"data": [{"url": "/x.wav"}]}, "http://h:1/x.wav"),
|
||||
"empty": ([], None),
|
||||
}
|
||||
for name, (payload, want) in cases.items():
|
||||
got = audio.extract_file_url(payload, base)
|
||||
check(failures, got == want, f"audio url {name}: {got!r} != {want!r}")
|
||||
|
||||
|
||||
def test_trellis(failures: list[str]) -> None:
|
||||
with fake_server(), tempfile.TemporaryDirectory() as tmp:
|
||||
png = Path(tmp) / "crate.png"
|
||||
png.write_bytes(b"\x89PNG-fake")
|
||||
out = str(Path(tmp) / "crate.glb")
|
||||
result = quiet(trellis.generate, str(png), output=out, simplify=0.9, texture_size=512, seed=7)
|
||||
|
||||
posts = [(p, b) for m, p, b in FakeGradio.seen if m == "POST"]
|
||||
paths = [p for p, _ in posts]
|
||||
want = ["/api/start_session", "/upload", "/api/preprocess_image_1", "/api/get_seed",
|
||||
"/api/image_to_3d", "/api/extract_glb"]
|
||||
check(failures, paths == want, f"trellis: call sequence changed: {paths}")
|
||||
sessions = {b.get("session_hash") for p, b in posts if p != "/upload" and isinstance(b, dict)}
|
||||
check(failures, len(sessions) == 1 and None not in sessions, f"trellis: session not shared: {sessions}")
|
||||
by_path = dict(posts)
|
||||
check(failures, by_path["/api/get_seed"]["data"] == [True, 7], "trellis: get_seed payload changed")
|
||||
check(failures, by_path["/api/image_to_3d"]["data"] ==
|
||||
[{"path": "/tmp/pre.png"}, [], False, 4242, 7.5, 12, 3.0, 12, "stochastic"],
|
||||
f"trellis: image_to_3d payload changed: {by_path['/api/image_to_3d']['data']}")
|
||||
check(failures, by_path["/api/extract_glb"]["data"] == [None, 0.9, 512],
|
||||
"trellis: extract_glb payload changed")
|
||||
check(failures, Path(out).read_bytes() == GLB_BYTES, "trellis: downloaded bytes differ")
|
||||
check(failures, result["seed"] == 4242, "trellis: server seed not reported")
|
||||
|
||||
# batch: one present, one missing input -> summary counts, no sleeping
|
||||
plan = trellis.batch_plan(Path(tmp), Path(tmp) / "out", ["crate", "ghost"])
|
||||
check(failures, [p[0] for p in plan] == ["crate", "ghost"], "trellis: batch plan ignores --names")
|
||||
summary = quiet(trellis.batch, tmp, str(Path(tmp) / "out"), names=["crate", "ghost"],
|
||||
cooldown=0, max_retries=1, retry_delay=0)
|
||||
check(failures, (summary["success"], summary["failed"]) == (1, 1),
|
||||
f"trellis: batch counts wrong: {summary}")
|
||||
|
||||
|
||||
def test_image_body(failures: list[str]) -> None:
|
||||
body = image.build_request_body("a lighthouse", aspect_ratio="16:9", image_size="2K")
|
||||
check(failures, body["contents"][0]["parts"] == [{"text": "a lighthouse Resolution: 2K."}],
|
||||
f"image: prompt/size hint changed: {body}")
|
||||
check(failures, body["generationConfig"] == {"responseModalities": ["TEXT", "IMAGE"],
|
||||
"imageConfig": {"aspectRatio": "16:9"}},
|
||||
f"image: generationConfig changed: {body['generationConfig']}")
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
src = Path(tmp) / "ref.jpg"
|
||||
src.write_bytes(b"jpegdata")
|
||||
parts = image.build_request_body("x", input_image=str(src))["contents"][0]["parts"]
|
||||
check(failures, parts[0]["inlineData"]["mimeType"] == "image/jpeg" and len(parts) == 2,
|
||||
f"image: image-to-image part wrong: {parts}")
|
||||
try:
|
||||
image.build_request_body("x", input_image="/nope/missing.png")
|
||||
failures.append("image: a missing input image did not fail")
|
||||
except ReachError:
|
||||
pass
|
||||
|
||||
|
||||
def test_classification(failures: list[str]) -> None:
|
||||
port = _closed_port()
|
||||
real = endpoints.load_config
|
||||
endpoints.load_config = lambda: {"stable_audio_url": f"http://127.0.0.1:{port}"}
|
||||
try:
|
||||
quiet(audio.health)
|
||||
failures.append("endpoints: a refused connection did not fail")
|
||||
except ReachError as exc:
|
||||
check(failures, "OFF" in exc.message and "do not restart" in (exc.fix or ""),
|
||||
f"endpoints: refused connection not reported as OFF: {exc.message} / {exc.fix}")
|
||||
finally:
|
||||
endpoints.load_config = real
|
||||
|
||||
try:
|
||||
endpoints.call(urllib.request.Request(f"http://127.0.0.1:{port}/x"), service="Gemini API",
|
||||
what="t", timeout=2)
|
||||
failures.append("endpoints: unreachable cloud API did not fail")
|
||||
except ReachError as exc:
|
||||
check(failures, "network" in (exc.fix or ""), f"endpoints: cloud outage blamed on VRAM: {exc.fix}")
|
||||
|
||||
with fake_server() as base:
|
||||
for path, needle in (("/boom", "check the arguments"), ("/denied", "credentials")):
|
||||
try:
|
||||
endpoints.call(urllib.request.Request(base + path), service="audio", what="t", timeout=5)
|
||||
failures.append(f"endpoints: HTTP error at {path} did not fail")
|
||||
except ReachError as exc:
|
||||
check(failures, needle in (exc.fix or "") and "rejected" in exc.message,
|
||||
f"endpoints: {path} misclassified: {exc.message} / {exc.fix}")
|
||||
|
||||
try:
|
||||
endpoints.get_api_key("SR_TEST_SURELY_UNSET_KEY")
|
||||
failures.append("endpoints: a missing API key did not fail")
|
||||
except ReachError as exc:
|
||||
check(failures, "tracked" in (exc.fix or ""), "endpoints: missing-key remedy lost the never-commit warning")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
failures: list[str] = []
|
||||
for test in (test_audio, test_audio_urls, test_trellis, test_image_body, test_classification):
|
||||
test(failures)
|
||||
if failures:
|
||||
print("test_assets: FAIL", file=sys.stderr)
|
||||
for failure in failures:
|
||||
print(f" - {failure}", file=sys.stderr)
|
||||
return 1
|
||||
print("test_assets: OK — audio, trellis and image payloads unchanged; OFF, rejected, "
|
||||
"credentials and network failures each named correctly; nothing sent to a real service")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,93 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Batch Trellis generation — one at a time, gently.
|
||||
# Usage: ./tooling/trellis-batch.sh
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
INPUT_DIR=".tmp/image-gen/characters/bodies"
|
||||
OUTPUT_DIR=".tmp/glb-gen/characters/bodies"
|
||||
COOLDOWN=15 # seconds between jobs
|
||||
MAX_RETRIES=3
|
||||
RETRY_DELAY=60 # seconds before retry
|
||||
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
TYPES=(
|
||||
slim_m slim_f
|
||||
average_m average_f
|
||||
stocky_m stocky_f
|
||||
tall_lean_m tall_lean_f
|
||||
short_stout_m short_stout_f
|
||||
athletic_m athletic_f
|
||||
heavyset_m heavyset_f
|
||||
petite_m petite_f
|
||||
)
|
||||
|
||||
TOTAL=${#TYPES[@]}
|
||||
SUCCESS=0
|
||||
FAILED=0
|
||||
|
||||
echo "=== Trellis Batch: $TOTAL bodies ==="
|
||||
echo " Cooldown: ${COOLDOWN}s between jobs"
|
||||
echo " Retries: $MAX_RETRIES with ${RETRY_DELAY}s delay"
|
||||
echo ""
|
||||
|
||||
for i in "${!TYPES[@]}"; do
|
||||
TYPE="${TYPES[$i]}"
|
||||
NUM=$((i + 1))
|
||||
INPUT="$INPUT_DIR/${TYPE}.png"
|
||||
OUTPUT="$OUTPUT_DIR/${TYPE}.glb"
|
||||
|
||||
# Skip if already generated
|
||||
if [ -f "$OUTPUT" ]; then
|
||||
echo "[$NUM/$TOTAL] $TYPE — already exists, skipping"
|
||||
SUCCESS=$((SUCCESS + 1))
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ ! -f "$INPUT" ]; then
|
||||
echo "[$NUM/$TOTAL] $TYPE — input not found: $INPUT"
|
||||
FAILED=$((FAILED + 1))
|
||||
continue
|
||||
fi
|
||||
|
||||
ATTEMPT=0
|
||||
DONE=false
|
||||
while [ "$ATTEMPT" -lt "$MAX_RETRIES" ] && [ "$DONE" = "false" ]; do
|
||||
ATTEMPT=$((ATTEMPT + 1))
|
||||
echo "[$NUM/$TOTAL] $TYPE (attempt $ATTEMPT/$MAX_RETRIES)..."
|
||||
|
||||
if python3 tooling/db/trellis_connector.py generate \
|
||||
"$INPUT" \
|
||||
--output "$OUTPUT" \
|
||||
--simplify 0.95 \
|
||||
--texture-size 1024 \
|
||||
2>&1 | tee /dev/stderr | grep -q '"ok": true'; then
|
||||
echo " OK"
|
||||
SUCCESS=$((SUCCESS + 1))
|
||||
DONE=true
|
||||
else
|
||||
echo " FAILED"
|
||||
if [ "$ATTEMPT" -lt "$MAX_RETRIES" ]; then
|
||||
echo " Waiting ${RETRY_DELAY}s before retry..."
|
||||
sleep "$RETRY_DELAY"
|
||||
else
|
||||
echo " Giving up on $TYPE after $MAX_RETRIES attempts"
|
||||
FAILED=$((FAILED + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Cooldown between successful jobs
|
||||
if [ "$DONE" = "true" ] && [ "$NUM" -lt "$TOTAL" ]; then
|
||||
echo " Cooling down ${COOLDOWN}s..."
|
||||
sleep "$COOLDOWN"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=== Results: $SUCCESS/$TOTAL succeeded, $FAILED failed ==="
|
||||
ls -la "$OUTPUT_DIR"/*.glb 2>/dev/null || echo "No GLB files generated"
|
||||
Reference in New Issue
Block a user