MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generation

1 INTRODUCTION

• Compositional 3D representations support component reuse, targeted editing, animation, and physically plausible interaction modeling across applications such as virtual production, games, robotics, and CAD.Source: §1 · p. 1
• Prior compositional generators face O(N^2L^2) global-attention cost for N components with L tokens each, while uniform attention also spends resources on low-value interactions.These limitations become increasingly problematic for complex scenes and fine-grained object decompositions.Source: §1 · p. 2
• MoCA is a native compositional 3D generative model designed for efficient, scalable, and accurate modeling of both 3D objects and scenes.Its motivation follows limitations of monolithic 3D generators, which restrict component-level editing and materials customization.Source: §1 · p. 2
• MoCA routes each component to its top-k important components for sparse global attention and compresses unselected components into compact tokens, preserving coarse-grained context while reducing attention key/value tokens.With typical settings k ≪ N and σ ≫ 1, the context length is much smaller than naive global attention, enabling efficient compositional 3D generation.Source: §1 · p. 2
• Experiments cover object-level generation from single images with automatically decomposed parts and scene-level generation conditioned on scene images and per-instance masks.The authors report clear-margin improvements over prior works, particularly for fine-grained component generation.Source: §1 · p. 2

2 RELATED WORKS

• Vecset-based and sparse-voxel-based latent spaces support native 3D generation, with sparse voxels capturing fine-grained geometry; both require iso-surface extraction to obtain meshes.Vecset methods emphasize compact latent representations and high-resolution shape generation.Source: §2 · pp. 2–3
• MoCA’s architecture uses a compositional DiT with interleaved local and global attention blocks for jointly generating component latents.The overview also includes component-latent packing, random component IDs, and separate decoding into the global space.Source: Figure 2 · p. 3
• Part-aware object methods generate separate components through image-based reconstruction, segmentation and completion, or preprocessed 3D part datasets, but assembly and scaling to many parts remain challenging.Recent methods denoise N part-token sequences concurrently with global attention, whose quadratic cost grows as N increases.Source: §2 · p. 3
• Multi-instance scene generators must model object geometry and spatial relationships, while sequential pipelines are inefficient and end-to-end global-attention methods typically handle fewer than 20 instances.These constraints motivate more scalable attention mechanisms for scene generation.Source: §2 · p. 3

3.1 PRELIMINARY: VECSET DIFFUSION MODELS FOR 3D SHAPE GENERATION

• Vecset diffusion encodes sampled surface points into unordered latent vectors, decodes them into implicit fields, and extracts explicit surfaces with iso-surface methods.Source: §3.1 · p. 4

3.2 MODEL ARCHITECTURE

• MoCA jointly generates component-wise vecset latents with interleaved local and global attention blocks, then decodes each latent into a 3D component with a frozen vecset decoder.Source: §3.2 · p. 4

3.2.1 LEARNABLE TOKENS AND COMPONENT COMPRESSION

• Learnable query tokens cross-attend to each component’s noisy vecset tokens, producing compressed tokens and an anchor token used for later component-level importance estimation.Source: §3.2.1 · p. 4

3.2.2 LOCAL BLOCK: LOCAL FEATURE UPDATING

• Local blocks update component features with a partially blocked mask that preserves vecset-token self-attention while allowing compressed and anchor tokens to aggregate broader local information.Source: §3.2.2 · p. 5

3.2.3 GLOBAL BLOCK: MIXTURE-OF-COMPONENTS ATTENTION

• Importance scores are computed from anchor-token query-key interactions with sigmoid activation, then gate corresponding key vectors so routing remains trainable end-to-end and can vary across attention heads.Source: §3.2.3 · pp. 5–6
• MoC attention routes each component to full vecset tokens from its top-k important components and compressed tokens from the remaining components, reducing the global context.Source: §3.2.3 · pp. 5–6

3.3.1 LOAD BALANCE CONSIDERATION

• Stochastic sampling from normalized importance logits provides auxiliary-loss-free load balancing during training, while deterministic top-k routing is restored at test time.Source: §3.3.1 · p. 7

3.3.2 TRAINING OBJECTIVE

• MoCA trains with rectified flow matching by perturbing clean component latents toward shared Gaussian noise and predicting the velocity from the noisy latents, timestep, and conditioning image.Source: §3.3.2 · p. 7

4.1 IMPLEMENTATION DETAILS

• MoCA uses alternating self-attention and MoC attention, selecting 25% of components as important and compressing the rest with σ = 8.The implementation also uses a random ID embedding codebook of size 50.Source: §4.1 · p. 7

4.2 PART-COMPOSED 3D OBJECT GENERATION

• MoCA achieves superior inter-part independence and overall fidelity compared with PartPacker, PartCrafter, and HoloPart in part-composed object generation.PartCrafter is evaluated only on front-view renderings because it is trained exclusively on front-view images; other methods use random-view renderings.Source: §4.2 · p. 8
• MoCA generates clean part-level decompositions for real-world objects across different categories when conditioned on real images.The results are qualitative and are shown in Figure 5.Source: §4.2 · p. 8, Figure 5 · p. 8

4.3 INSTANCE-COMPOSED 3D SCENE GENERATION

• MoCA outperforms MIDI and PartCrafter at both scene and instance levels, producing more accurate instance placement and higher-quality geometry.PartCrafter can generate duplicate or incorrectly placed instances because it does not use instance masks.Source: §4.3 · p. 8, Table 2 · p. 8
• MoCA generates scenes with more than 16 instances, whereas the baseline methods are limited to fewer than 8 instances per scene.Figure 6 presents complex-scene examples, while Figure 7 compares geometry and placement on simpler scenes.Source: §4.3 · p. 8, Figure 6 · p. 9, Figure 7 · p. 9

4.4 ABLATIONS

• Routing important components is necessary: removing full-feature interaction with selected components causes a large degradation in generation quality.This ablation is reported in Table 3A.Source: §4.4 · p. 9
• Compressed distant components are also necessary, because removing their coarse information significantly reduces performance and eliminates global spatial context.The model benefits from combining coarse layout information from all components with full local features from important components.Source: §4.4 · p. 9
• Ablations show that key routing choices matter: direct value weighting, Softmax router activation, omitted load balancing, and removed multi-head routing each reduce performance.The reported explanations attribute these effects to post-softmax numerical instability, vanishing gate values, routing imbalance, and reduced representational capacity, respectively.Source: §4.4 · pp. 9–10, Table 3 · p. 10

• MoCA consistently outperforms baseline methods on both part-composed object and instance-composed scene generation tasks.Source: §5 · p. 10
• MoC attention enables training with up to 32 components per sample, twice the capacity of previous methods.Source: §5 · p. 10
• The frozen vecset VAE may bottleneck reconstruction when assets contain many components with very small global volumes.Source: §5 · p. 10

• The authors do not anticipate direct negative societal consequences from this research.Source: ETHICS STATEMENT · p. 11
• Potential ethical concerns may arise in downstream applications, motivating careful and responsible use of the methods.Source: ETHICS STATEMENT · p. 11
• The training dataset combines public and professional sources with rule-based filtering and contains no human subjects, personally identifiable information, or sensitive data.Source: ETHICS STATEMENT · p. 11

• ChatGPT was used exclusively for grammar checking and stylistic refinement of the manuscript.Source: §A.1 · p. 15
• The authors state that the scientific ideas, experiments, data work, analysis, and conclusions were entirely their original work.Source: §A.1 · p. 15
• The language model was not used to generate hypotheses, conduct experiments, interpret findings, or draw conclusions.Source: §A.1 · p. 15

Dataset Curation

• The object dataset contains nearly 2 million watertight meshes with part annotations, using objects with up to 32 parts.Source: §A.2 · p. 15
• Scene training adapts 3D-FRONT to scenes with up to 32 instances and uses four isometric views with randomly textured floors and walls.Source: §A.2 · p. 15

Training

• Object training uses staged pretraining and finetuning, followed by scene adaptation with image and instance-mask conditioning, LoRA, CFG, AdamW, and BFloat16.Source: §A.2 · p. 15

• The analysis measures local attention, MoCA’s routing procedure, global attention, and total single-forward-pass runtime on one H20 GPU with 1024 tokens per part.Source: §A.3 · p. 15, Table 4 · p. 15
• At N = 32, MoCA’s global attention runs at approximately half the runtime of PartCrafter.Source: Table 4 · p. 15, §A.3 · p. 16
• PyTorch sorting and indexing in MoCA’s routing procedure currently adds substantial runtime overhead, motivating further optimization.Source: §A.3 · p. 16

• Figure 10 presents qualitative results for part-level object generation and instance-level complex scene generation.Source: §A.4 · p. 16, Figure 10 · p. 17