Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Training

• Existing methods struggle to compose and control complex creative content, including creatures assembled from parts of different animals and machines.Part-level approaches face difficult granularity control and incoherent interfaces, while 2D-to-3D methods depend on image quality and 3D data scale.Source: §1 · pp. 1–2
• Muses is a training-free framework that uses 3D skeletons to connect creature design and feed-forward generation.Its pipeline designs a skeleton, assembles content in structured 3D latent space, and generates style-consistent texture.Source: §1 · p. 2
• Muses designs a coherent skeleton through graph-constrained reasoning, then uses it to guide voxel-based content assembly and harmonious texture generation.The method fuses skeletal regions from different objects in structured 3D latent space.Source: §1 · p. 2
• Extensive qualitative and quantitative evaluations report superior generation fidelity and efficiency compared with state-of-the-art methods.The framework is also described as adaptable to different modern 3D generation models.Source: §1 · p. 2

2.1. 3D object generation

• Feed-forward 3D generation has accelerated, but current pipelines remain inadequate for genuinely out-of-distribution concepts.Prior approaches include optimized 3D representations, multi-view image synthesis followed by reconstruction, and feed-forward models trained on large 3D datasets.Source: §2.1 · pp. 2–3

2.2. Part-aware 3D generation

• Part-aware methods synthesize and compose semantic components, but part granularity remains difficult to control and direct composition often yields suboptimal results.DreamBeast [22] supports only three parts and incurs per-instance SDS optimization cost.Source: §2.2 · p. 3

2.3. Creative 2D image generation

• Fine-grained creative image methods recombine components, yet composites can be misaligned or inconsistent and difficult to lift into harmonious 3D results.Muses mines 3D skeletal information rather than relying on part-level assembly or 2D-driven creativity.Source: §2.3 · p. 3

• Native 3D generation models include VecSet-based and structured-latent approaches; Muses adopts Trellis’s SLAT framework for its explicit representations and decoupling.SLAT pairs local latent features with active voxel positions.Source: §3 · p. 3
• Trellis first predicts a low-resolution grid and reconstructs dense occupancy, then predicts latent features at the resulting sparse voxel positions.The local features encode fine-grained geometry and texture.Source: §3 · p. 3
• Directly creating nonexistent creatures with Trellis leads to suboptimal results, motivating an explicit skeleton to link design and generation.Source: §3 · p. 3

4. Methodology

• The pipeline designs a skeletal structure, composes SLAT representations according to the creative concept, then generates style-consistent texture while keeping geometry unchanged.It comprises skeleton-guided concept design, SLAT-based content composition, and style-consistent texture generation.Source: §4 · p. 3

4.1. Skeleton-guided concept design

• Graph-based heuristics clean and partition source skeletons into semantic parts, then LLM-guided reasoning arranges those parts into a geometrically consistent layout.The classification labels body, wings, legs, head, and tail; assembly can apply primitive operators and prompt-specified multiplicities.Source: §4.1 · pp. 4–5

4.2. SLAT-based content composition

• Muses maps skeleton-defined regions to SLAT using skinning weights, then interpolates and merges voxel content to create a structurally aligned coarse creature.Interpolation uses a more compact voxel space to bridge combination gaps associated with sparse active voxels.Source: §4.2 · p. 5

4.3. Style-consistent texture generation

• A rendered frame of the coarse creature guides geometry-invariant texture editing, followed by style-conscious refinement for harmonious, geometry-consistent appearance.Positive and negative prompt embeddings emphasize preserving input geometry while matching an artistic style.Source: Figure 3 · p. 4, §4.3 · p. 5

5.1. Setup

• Evaluation uses 30 randomly selected samples for CLIPScore and VQAScore, plus 10 examples rated by 60 volunteers for generation quality, text–image consistency, and preference.The paper notes CLIPScore is less reliable for highly compositional or combinatorial text.Source: §5.1 · p. 6

5.2. Comparison with the state-of-the-art

• Muses consistently outperforms all baselines across all reported metrics, with stronger fidelity and structural harmony on complex creature prompts.The comparison covers 2D-prior optimization, 2D-image lifting, feed-forward 3D generation, and manually assembled part-level outputs.Source: §5.2 · pp. 6–7

5.3. Ablation study

• Without LLM-guided design, layouts have improper scale, orientation, or position; without skinning weights or interpolation, compositions risk segmentation errors, seams, voids, and misalignment.Without geometrically invariant texture editing, textures can mismatch semantic regions; without style consistency, source assets remain disharmonious.Source: Figure 6 · p. 7, §5.3 · pp. 7–8

5.4. Robustness

• Failure rates rise with skeleton complexity, but remain around 10% overall across graph-based classification and LLM-guided assembly.The tests use joint-count and branching-topology bins, with 100 sampled cases per bin.Source: §5.4 · p. 8

5.5. Application

• Skeleton-aware design supports disentangled part-level editing while leaving other regions unchanged, and texture editing enables structure-aligned appearance changes across styles.The reported applications are potential extensions beyond creating novel creature models.Source: §5.5 · p. 8

5.6. Limitation

• The method is constrained by failed Trellis generations and improper initial skeletons, which can prevent meaningful part extraction or the design stage.The authors state these cases may be resolved by stronger 3D generation and skeleton-modeling approaches.Source: §5.6 · p. 8

• Muses uses skeleton-guided design, SLAT-based content composition, and style-consistent texture generation to create geometrically coherent fantasy creatures.Source: §6 · p. 8
• Experiments on challenging compositional prompts show state-of-the-art performance in visual fidelity and textual alignment.Source: §6 · p. 8
• The authors aim to expand Muses into a flexible 3D object-editing tool for interactive applications.Source: §6 · p. 8