jpmschweitzerandClaude Opus 5.5
ddce4441a9
tooling/db/ (a misnamed directory: connectors, not database work),
trellis-batch.sh and synth_ui_sounds.py become `reach assets`:
audio {health,generate,batch,post {convert,normalize,trim,pipeline}},
image {health,generate}, trellis {health,generate,batch}, and synth-ui.
The four audio bash wrappers are retired, and tooling/db/ is gone.
Parity, from baselines taken before anything moved:
- the four UI-sound WAVs and the harmonic-synth WAVs (exponential and linear
decay) are byte-identical
- the ffmpeg pipeline's decoded PCM is identical. Its .ogg bytes are not,
even between two runs of the OLD code: Ogg picks a random stream serial,
so the encoded file was never the right thing to compare
- the network success paths can't be run in a gate (Stable Audio and Trellis
are kept off, Gemini costs money), so tooling/test_assets.py stands up a
fake Gradio and pins every payload: the audio submit, Trellis's six-call
session sequence with its 9-input image_to_3d, and the Gemini body. It
failed when one Trellis value was mutated (7.5 → 7.0)
Failure classification, in endpoints.py, is the point of the port. The
services are OFF by design (VRAM on tower-of-joy, D-17), and the topology doc
warns against "fixing" one by restarting it. So a refused connection says OFF
and asks for the service to be turned on rather than restarted; a 4xx/5xx says
the request was rejected; 401/403 says credentials; 429 says quota; and an
unreachable Gemini blames the network, not VRAM.
Behaviour changes, each a failure that used to read as success or crash:
- audio batch and trellis batch exited 0 with failures in their summaries;
they now print the summary and exit 1
- trellis generate on a missing image crashed with a TypeError
(print(..., indent=2)); it now names the file, and checks it before the
service so a typo is not reported as an outage
- the ffmpeg pipeline left its intermediates behind when a step failed
Structure: the connectors called each other as subprocesses (batch spawned
the connector, which spawned audio_post) and parsed each other's stdout. They
are now function calls, and ffmpeg is the only exec, through core/process.
ensure_venv() is removed: it os.execv'd into .venv, which D-263's exec rule
forbids, and reach declares the dependencies itself. config.json moved into
the domain deliberately, and the local-services rule follows it.
Output contract: results are still JSON on stdout with the same keys, so skill
readers keep working. Failures are an exit status with a Fix line, never
{"ok": false}. The audio-gen, glb-gen and image-gen skills, Araminta's agent
file and the allow-list are updated to match. glb-gen's "trellis-batch.sh is
hardcoded to one category" caveat is gone: batch takes --input-dir or --names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
115 lines
4.3 KiB
Markdown
115 lines
4.3 KiB
Markdown
# 3D Pipeline Spike
|
|
|
|
Proof of concept for the Trellis image-to-3D asset pipeline with runtime
|
|
recoloring in Godot 4.6.
|
|
|
|
## What This Proves
|
|
|
|
1. **Trellis generates usable 3D assets** from 2D concept art (via /image-gen)
|
|
2. **Blender post-processing** normalizes scale, centers models, generates
|
|
recolor masks, and adjusts materials for toon rendering
|
|
3. **Godot renders textured GLBs** with embedded textures when
|
|
`gltf/embedded_image_handling=3` (embed uncompressed) is set
|
|
4. **Luminance-preserving recoloring** via shader + mask sidecar: tint the
|
|
dominant color region while keeping texture detail (shadows, highlights,
|
|
edges)
|
|
|
|
## Pipeline
|
|
|
|
```
|
|
concept.png ──→ Trellis ──→ raw.glb ──→ Blender postprocess ──→ model.glb + mask.png
|
|
│
|
|
Godot import
|
|
(embedded textures)
|
|
│
|
|
toon_masked.gdshader
|
|
(texture + mask + tint)
|
|
```
|
|
|
|
## Key Findings
|
|
|
|
### GLB Import Settings (Critical)
|
|
|
|
Godot's GLTF importer `gltf/embedded_image_handling` values:
|
|
|
|
| Value | Mode | Result |
|
|
|-------|------|--------|
|
|
| 0 | Discard textures | Models render without any texture |
|
|
| 1 | Extract textures | **Silently fails** — materials get null texture refs |
|
|
| 2 | Embed as Basis Universal | Compressed, may lose quality |
|
|
| 3 | Embed uncompressed | **Works correctly** — textures preserved in .scn |
|
|
|
|
**Always use value 3** for Trellis-generated GLBs. Set in project.godot under
|
|
`[gltf]` so new imports pick it up automatically.
|
|
|
|
### Recolor Shader
|
|
|
|
The `toon_masked.gdshader` uses luminance-preserving blending:
|
|
|
|
```glsl
|
|
float luma = dot(original.rgb, vec3(0.299, 0.587, 0.114));
|
|
vec3 tinted = tint_color.rgb * (luma * 1.5 + 0.2);
|
|
vec3 base = mix(original.rgb, tinted, mask);
|
|
```
|
|
|
|
- Where `mask = 0` (black): original Trellis texture preserved
|
|
- Where `mask = 1` (white): tint color applied, scaled by original luminance
|
|
- This keeps shadows, highlights, and detail lines even in recolored regions
|
|
|
|
The mask default is `hint_default_black` (replace nothing when no mask loaded).
|
|
|
|
### Toon Shadow
|
|
|
|
Gentle 15% darkening with `smoothstep` transition — not the harsh binary
|
|
shadow/lit split. This preserves texture readability at isometric camera angles
|
|
where many faces would otherwise be fully in shadow.
|
|
|
|
### Alpha / See-Through Fix
|
|
|
|
Trellis texture atlases sometimes have alpha artifacts at UV seams. The shader
|
|
does NOT output ALPHA — all models render fully opaque. If transparency is
|
|
needed for specific assets (glass, holograms), use a separate shader variant.
|
|
|
|
## Running
|
|
|
|
```bash
|
|
cd spikes/3dpipeline
|
|
godot --headless --import # first time only — generates .godot/imported/
|
|
godot --path . # run the spike
|
|
```
|
|
|
|
### Controls
|
|
|
|
| Key | Action |
|
|
|-----|--------|
|
|
| WASD | Pan camera (screen-aligned) |
|
|
| Mouse wheel | Zoom in/out |
|
|
| T | Cycle camera angle (top-down / 45 iso / 30 dramatic) |
|
|
| R | Randomize character colors |
|
|
|
|
## Files
|
|
|
|
```
|
|
project.godot # Minimal Godot project config
|
|
scenes/spike_3d.tscn # Main scene (Camera3D + script)
|
|
scripts/spike/spike_main.gd # Camera, model loading, shader application
|
|
shaders/spike/toon_masked.gdshader # Texture + recolor mask + toon shadow
|
|
shaders/spike/toon.gdshader # Flat color toon (characters)
|
|
shaders/spike/outline.gdshader # Inverted hull outline (not used yet)
|
|
models/furniture/*.glb + *_mask.png # Post-processed Trellis models + masks
|
|
models/props/*.glb + *_mask.png # Post-processed Trellis props + masks
|
|
```
|
|
|
|
## Known Issues
|
|
|
|
- Baroque table mask is too aggressive — tints the whole model bright white
|
|
- Character scale needs tuning relative to furniture
|
|
- Trellis mesh quality varies — some models have holes visible at close zoom
|
|
- No outline shader applied yet (inverted hull ready but not wired up)
|
|
|
|
## Dependencies
|
|
|
|
- Godot 4.6+ (gl_compatibility renderer)
|
|
- Models generated by: `reach assets trellis` (formerly `tooling/db/trellis_connector.py`) + `.claude/skills/glb-gen/`
|
|
- Concept images generated by: `.claude/skills/image-gen/`
|