settled-reach/spikes/quaternius-aesthetic/reviews/auto-rig-research.md

Auto-Rig Clothing Meshes to Character Skeletons — Research

Author: TYRE Date: 2026-03-18 Context: Trellis-generated clothing meshes (GLB) need to be rigged to Quaternius-style character skeletons (GLTF with armature) via blender --background --python. Must be fully headless, zero manual intervention, run as part of the asset pipeline.

---

The Problem

• Input A: An unskinned clothing mesh (from Trellis — expect non-manifold geometry, possible interior faces, loose vertices, inconsistent normals)
• Input B: A skinned character body mesh with skeleton/armature (Quaternius-style, low-poly, clean topology)
• Output: The clothing mesh with bone weights, parented to the skeleton, exported as GLB
• Must run headless: blender --background --python auto_rig_clothing.py

---

1. What Exists

Tier 1: Purpose-Built Weight Transfer (Best Fit)

A. Robust Weight Transfer (sentfromspacevr)

• Repo: https://github.com/sentfromspacevr/robust-weight-transfer
• License: GPL-2.0+
• Based on: SIGGRAPH Asia 2023 paper "Robust Skin Weights Transfer via Weight Inpainting" (Abdrashitov et al.)
• Academic code: https://github.com/rin-23/RobustSkinWeightsTransferCode (MIT)
• Blender versions: 2.93 through 4.1

How it works:

1. Extracts vertices, faces, normals from both meshes in world space
2. For each target vertex, finds closest point on source surface via igl.point_mesh_squared_distance()
3. Interpolates source weights at that point using barycentric coordinates
4. Validates matches with dual thresholds: distance AND normal angle
5. Unmatched vertices get weights via Laplacian inpainting — solves a quadratic minimization with matched vertices as boundary conditions
6. Optional smoothing pass via adjacency-matrix flood fill from unmatched regions
7. Bone count limiter (default 4 per vertex, useful for game engines)

Edge case handling:

• Flipped normals: supports 180-degree mirror matching
• Non-manifold / loose geometry: the "Point" inpainting mode remeshes internally, allowing weights to flow outside the mesh boundary
• World-space transform: applies matrix_world before any distance calculations
• Group bounds checking: guards against out-of-range vertex group indices
• Lock-aware: skips locked vertex groups

Dependencies: scipy, igl (libigl python bindings), robust_laplacian

• These are pip-installable but NOT bundled with Blender's Python
• Must install into Blender's Python: blender --background --python -c "import subprocess; subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'scipy', 'libigl', 'robust_laplacian'])"

Headless viability: The core algorithm (weighttransfer.py, util.py) is pure numpy/scipy/igl with bpy for mesh data extraction. The UI layer (__init__.py) is separable. The actual transfer logic can be extracted and called directly without operator registration. This is the strongest candidate.

Concern: GPL-2.0 license on the Blender addon. However, the underlying algorithm's reference implementation (rin-23/RobustSkinWeightsTransferCode) is MIT. We could reimplement the algorithm from the MIT code + paper rather than copying GPL code.

---

B. Blender Built-in: Data Transfer Modifier

• License: Blender's own (GPL-2.0)
• Dependencies: None — ships with Blender
• Blender API: bpy.types.DataTransferModifier or bpy.ops.object.data_transfer()

How it works:

import bpy

# Assume body_obj is skinned, clothing_obj is unskinned
# Both already loaded and positioned

# Parent clothing to armature with empty groups
clothing_obj.parent = armature_obj
mod = clothing_obj.modifiers.new(name="Armature", type='ARMATURE')
mod.object = armature_obj

# Add Data Transfer modifier
dt = clothing_obj.modifiers.new(name="WeightTransfer", type='DATA_TRANSFER')
dt.object = body_obj
dt.use_vert_data = True
dt.data_types_verts = {'VGROUP_WEIGHTS'}
dt.vert_mapping = 'POLYINTERP_NEAREST'  # nearest face interpolated
dt.layers_vgroup_select_src = 'ALL'
dt.layers_vgroup_select_dst = 'NAME'

# Generate destination layers, then apply
bpy.ops.object.datalayout_transfer(modifier=dt.name)
bpy.context.view_layer.objects.active = clothing_obj
bpy.ops.object.modifier_apply(modifier=dt.name)

Strengths:

• Zero external dependencies
• Well-tested, ships with Blender
• Multiple vertex mapping modes: NEAREST, EDGE_NEAREST, POLYINTERP_NEAREST, etc.
• Works headless out of the box

Weaknesses:

• No inpainting for unmatched vertices — they get zero weights
• No normal-angle validation — can bleed weights through thin surfaces
• No smoothing of transfer boundaries
• Struggles with meshes that don't closely conform to the source (loose clothing, capes, dangling parts)

Verdict: Good enough for tight-fitting clothing on low-poly meshes where the clothing closely follows the body surface. Will fail on loose geometry or parts that extend far from the body (cloaks, wide skirts, big sleeves). For Quaternius-style low-poly with simple silhouettes, this might actually be sufficient for v0.1.

---

C. Easy Weights (Blender Studio)

• Repo: https://studio.blender.org/tools/addons/easy_weight
• License: GPL (Blender Studio addon)
• Dependencies: None

Wraps Blender's built-in weight transfer with a nicer UI. Under the hood uses the same Data Transfer approach. Adds batch transfer to collections and zero-weight cleanup. Not substantially different from rolling our own with the Data Transfer modifier API. Skip — no algorithmic advantage over built-in.

---

Tier 2: Full Auto-Skinning (From Scratch, No Source Weights)

D. Voxel Heat Diffuse Skinning

• Repo: https://github.com/meshonline/Surface-Heat-Diffuse-Skinning
• Commercial addon: https://addons.cgdive.com/tools/voxel-heat-diffuse-skinning
• License: MIT (open source version)
• Blender versions: up to 4.3

How it works: Voxelizes the mesh, runs heat diffusion from each bone through the voxel grid, produces per-vertex weights. Unlike Blender's built-in "Automatic Weights" (which uses envelope-based heat mapping and frequently fails with the "Bone Heat Weighting failed" error), VHDS never fails to produce a result.

Strengths:

• Works on non-watertight meshes (no "bone heat failed" errors)
• Good for overlapping geometry (clothing over body)
• MIT licensed

Weaknesses:

• Skins from scratch — doesn't transfer from an existing skinned mesh
• Tends to fuse areas that should be separate (fingers, mouth)
• We already HAVE good weights on the body — transferring them is better than recomputing from scratch

Verdict: Wrong tool for the job. We want weight transfer, not weight generation. However, VHDS could be a fallback for clothing pieces that are too far from the body for transfer to work (standalone accessories, hats, etc.).

---

E. UniRig (SIGGRAPH 2025, VAST-AI-Research)

• Repo: https://github.com/VAST-AI-Research/UniRig
• License: MIT
• Requirements: Python 3.11, PyTorch >= 2.3.1, CUDA GPU >= 8GB VRAM

How it works: Large autoregressive model that predicts both skeleton structure AND per-vertex skinning weights from raw mesh geometry. Uses a Bone-Point Cross Attention mechanism. Trained on Rig-XL dataset (14K+ rigged models).

Strengths:

• State of the art accuracy (215% improvement over prior methods)
• Fast inference (1-5 seconds)
• Works on arbitrary mesh categories (not just humanoids)

Weaknesses:

• Generates its OWN skeleton — does not transfer weights to an existing skeleton
• Heavy dependency stack (PyTorch, CUDA, flash_attn, spconv)
• Overkill for our case — we have a known skeleton, we just need weights on new meshes
• GPU requirement adds infrastructure complexity to the pipeline

Verdict: Fascinating technology but wrong use case. We have a skeleton. We need clothing to follow it. UniRig solves a harder problem we don't have.

---

F. Mesh Data Transfer Addon (Furionk)

• Repo: https://github.com/Furionk/mesh-data-transfer
• License: Not specified
• Purpose: Transfers vertex groups, UVs, shape keys, vertex colors between meshes

Similar to the built-in Data Transfer modifier but with a Maya-like "transfer attributes" workflow. No significant algorithmic advantage. Skip.

---

Tier 3: Not Directly Applicable But Worth Noting

G. Auto-Rig Pro ($40, commercial)

Commercial Blender addon. Excellent for rigging characters from scratch but not for transferring weights to clothing from an existing rig. Also not scriptable in the way we need. Skip.

H. Rigify (Blender built-in)

Auto-rigging addon for creating complex control rigs from metarigs. Generates skeletons, doesn't transfer weights between meshes. Wrong tool.

---

2. Recommended Approach

For v0.1: Built-in Data Transfer Modifier (Zero Dependencies)

cracks knuckles — Let me be honest about what this means technically.

Our clothing meshes are low-poly Quaternius-style. The body meshes are low-poly with clean topology. The clothing will sit close to the body surface in most cases. For this combination, the built-in Data Transfer modifier with POLYINTERP_NEAREST mapping will produce acceptable results with zero external dependencies.

Complete headless pipeline script structure:

"""
auto_rig_clothing.py — Run via: blender --background --python auto_rig_clothing.py -- body.glb clothing.glb output.glb

Pipeline:
1. Import body (skinned, with armature)
2. Import clothing (unskinned)
3. Normalize clothing scale/position to body
4. Transfer bone weights from body to clothing
5. Parent clothing to armature
6. Export clothing with weights
"""
import bpy
import sys

def clear_scene():
    bpy.ops.object.select_all(action='SELECT')
    bpy.ops.object.delete()

def import_glb(path):
    bpy.ops.import_scene.gltf(filepath=path)
    return [obj for obj in bpy.context.selected_objects]

def find_armature(objects):
    for obj in objects:
        if obj.type == 'ARMATURE':
            return obj
    return None

def find_meshes(objects):
    return [obj for obj in objects if obj.type == 'MESH']

def transfer_weights(source_mesh, target_mesh, armature):
    """Transfer bone weights from source body mesh to target clothing mesh."""
    # Ensure target has armature modifier
    arm_mod = target_mesh.modifiers.new(name="Armature", type='ARMATURE')
    arm_mod.object = armature

    # Set up data transfer
    bpy.context.view_layer.objects.active = target_mesh
    dt = target_mesh.modifiers.new(name="WeightTransfer", type='DATA_TRANSFER')
    dt.object = source_mesh
    dt.use_vert_data = True
    dt.data_types_verts = {'VGROUP_WEIGHTS'}
    dt.vert_mapping = 'POLYINTERP_NEAREST'
    dt.layers_vgroup_select_src = 'ALL'
    dt.layers_vgroup_select_dst = 'NAME'

    # Generate vertex groups on target
    bpy.ops.object.datalayout_transfer(modifier=dt.name)

    # Apply the modifier to bake weights
    bpy.ops.object.modifier_apply(modifier=dt.name)

    # Parent to armature (keep transform)
    target_mesh.parent = armature
    target_mesh.matrix_parent_inverse = armature.matrix_world.inverted()

def normalize_weights(mesh):
    """Normalize all vertex groups so weights sum to 1.0 per vertex."""
    bpy.context.view_layer.objects.active = mesh
    mesh.select_set(True)
    bpy.ops.object.mode_set(mode='WEIGHT_PAINT')
    bpy.ops.object.vertex_group_normalize_all(lock_active=False)
    bpy.ops.object.mode_set(mode='OBJECT')

def export_glb(objects, path):
    bpy.ops.object.select_all(action='DESELECT')
    for obj in objects:
        obj.select_set(True)
    bpy.ops.export_scene.gltf(
        filepath=path,
        use_selection=True,
        export_format='GLB'
    )

# -- Main --
argv = sys.argv
argv = argv[argv.index("--") + 1:]
body_path, clothing_path, output_path = argv[0], argv[1], argv[2]

clear_scene()
body_objects = import_glb(body_path)
armature = find_armature(body_objects)
body_meshes = find_meshes(body_objects)

clothing_objects = import_glb(clothing_path)
clothing_meshes = find_meshes(clothing_objects)

for clothing_mesh in clothing_meshes:
    transfer_weights(body_meshes[0], clothing_mesh, armature)
    normalize_weights(clothing_mesh)

export_glb([armature] + clothing_meshes, output_path)

For v0.2+ (If Quality Needs Improvement): Robust Weight Transfer Algorithm

If the Data Transfer approach produces artifacts on loose clothing (capes, robes, wide sleeves), upgrade to the Robust Weight Transfer algorithm. Two options:

Option A — Reimplement from MIT reference code The academic reference implementation (rin-23/RobustSkinWeightsTransferCode) is MIT licensed. The algorithm is documented in the SIGGRAPH Asia 2023 paper. We can implement the core transfer + inpainting in ~200 lines of Python using:

• scipy (sparse matrices, optimization)
• libigl Python bindings (closest point queries, Laplacian)
• robust_laplacian (mesh Laplacian construction)

These would need to be pip-installed into Blender's bundled Python.

Option B — Use the Blender addon directly (GPL) The sentfromspacevr addon wraps the algorithm with Blender operators. If we're okay with GPL-2.0 for tooling scripts (which don't ship in the game binary), we can use it directly. The weighttransfer.py module is cleanly separable from the UI.

Recommendation: Start with Option A if we need it. The algorithm is well-documented and the MIT reference code is clean. The key functions to port:

1. find_closest_point_on_surface() — igl closest point query
2. interpolate_attribute_from_bary() — barycentric weight interpolation
3. find_matches_closest_surface() — distance + normal threshold matching
4. inpaint() — Laplacian solve for unmatched vertices

---

3. Trellis-Specific Edge Cases

Trellis output is notoriously messy. Before weight transfer, the clothing mesh needs cleanup. This should be a pre-processing step in the pipeline:

def cleanup_trellis_mesh(mesh_obj):
    """Fix common Trellis output issues before weight transfer."""
    bpy.context.view_layer.objects.active = mesh_obj
    mesh_obj.select_set(True)
    bpy.ops.object.mode_set(mode='EDIT')

    # Select all geometry
    bpy.ops.mesh.select_all(action='SELECT')

    # Remove interior faces (Trellis often generates these)
    bpy.ops.mesh.select_interior_faces()
    bpy.ops.mesh.delete(type='FACE')
    bpy.ops.mesh.select_all(action='SELECT')

    # Merge by distance (remove duplicate vertices)
    bpy.ops.mesh.remove_doubles(threshold=0.001)

    # Recalculate normals (Trellis normals are often inconsistent)
    bpy.ops.mesh.normals_make_consistent(inside=False)

    # Remove loose vertices/edges
    bpy.ops.mesh.select_all(action='DESELECT')
    bpy.ops.mesh.select_loose()
    bpy.ops.mesh.delete(type='VERT')

    bpy.ops.object.mode_set(mode='OBJECT')

Additional Trellis concerns:

• Scale: Trellis outputs at arbitrary scale. Normalize to body mesh bounding box.
• Origin: May not be centered. Use body mesh origin as reference.
• Material slots: Trellis assigns a single material. May need material reassignment.
• Triangle-only topology: Trellis outputs triangulated meshes. This is fine for weight transfer (both algorithms work on triangles) but may want a limited dissolve pass for cleaner geometry if doing any manual edits later.

---

4. The Full Pipeline (Recommended)

Trellis GLB ──> Cleanup ──> Scale/Position ──> Weight Transfer ──> Normalize ──> Export GLB
                  |              |                    |
                  v              v                    v
            remove interior   match body        Data Transfer
            merge doubles     bounding box      (v0.1: built-in)
            fix normals       apply transform   (v0.2: robust inpainting)
            remove loose

Difficulty assessment:

• Built-in Data Transfer pipeline: Feasible. Straightforward. ~100 lines of Python, zero dependencies, testable in an afternoon.
• Robust Weight Transfer integration: Challenging but doable. ~300 lines of Python, requires pip-installing scipy/igl/robust_laplacian into Blender's Python. The algorithm is well-documented. 2-3 days of work.
• UniRig / ML-based approaches: Overkill. Skip for now.

---

5. Key Links

• Robust Weight Transfer addon — GPL-2.0, Blender 2.93-4.1
• SIGGRAPH Asia 2023 reference code — MIT, the algorithm we'd reimplement
• Voxel Heat Diffuse Skinning — MIT, fallback for accessories
• UniRig — MIT, SIGGRAPH 2025, wrong use case but impressive tech
• Easy Weights — weight transfer + cleanup, no license specified
• Blender Data Transfer Modifier API
• Blender Data Transfer Modifier docs

---

6. Recommendation Summary

Approach	Difficulty	Dependencies	Quality	Use When
Built-in Data Transfer	Easy	None	Good for tight-fit	v0.1, low-poly close-fitting clothing
Robust Weight Transfer (MIT reimplement)	Medium	scipy, igl, robust_laplacian	Excellent	v0.2, loose clothing, capes, robes
VHDS from-scratch skinning	Medium	None (built-in variant)	Decent	Standalone accessories (hats, bags)
UniRig ML	Hard	PyTorch, CUDA	Overkill	Never (wrong problem)

Start with Built-in Data Transfer. It's zero-dependency, well-tested, runs headless, and for Quaternius-style low-poly clothing that sits close to the body, it will work. If we hit quality issues on specific garment types, upgrade to the Robust Weight Transfer algorithm — the MIT reference implementation gives us a clean path to do so without GPL concerns.