PaperlayerView exact revision on arXiv

Source-linked AI summary

3DRAG: 3D-Native Object Drag Editing with Generative Models

Kuixiang Shao, Yuyang Jiao, Chenghao Zhang, Wenbo Zhao, Yinuo Bai, Jingyi Yu

arXiv:2610.13216v1cs.GR

TL;DR

Precise, interactive point-based editing remains challenging for 3D assets, as existing drag methods rely on costly 2D priors or geometry warping with consistency and structural risks. 3DRAG performs training-free editing in native 3D latent space, and experiments report state-of-the-art performance across drag accuracy, geometric fidelity, and unedited-region preservation.

  • Problem

    Existing 3D drag methods rely on 2D diffusion priors or geometry warping, with costly optimization, multi-view inconsistencies, and geometric artifacts limiting precise manipulation.

    Source: §1 · p. 2

  • Method

    3DRAG performs training-free drag editing in native 3D space using an occupancy-aware displacement field and an identity-preserving generative editing pipeline.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    3DRAG achieves state-of-the-art performance across drag accuracy, geometric fidelity, and unedited-region preservation, outperforming mesh-based and 3DGS-based baselines.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    3DRAG supports intuitive 3D content creation through spatial manipulations that preserve object identity and unedited regions.

    Source: §5 · p. 8

    Drag precision is bounded by the sparse active voxel grid resolution (currently 64^3), limiting fine-grained edits on extremely thin or intricate structures.

    Source: §5 · p. 8

Abstract

from arXiv · show

The rapid progress of 3D generative models has made 3D asset creation increasingly accessible. However, achieving intuitive, precise, and interactive user control, such as point-based drag editing, remains challenging. Existing 3D drag methods either rely on slow 2D diffusion priors, such as Score Distillation Sampling, or directly warp geometry, often causing multi-view inconsistency and topological artifacts. In this paper, we present 3DRAG, a training-free framework that executes precise drag editing natively within the explicit 3D latent space of large-scale generative models, without SDS-based optimization or iterative point tracking. To ensure geometrically plausible deformations, 3DRAG explicitly extends the drag displacement field into 3D space, incorporating an Occupancy-Aware Winner-Takes-All (OA-WTA) strategy to robustly handle sparse geometries. Furthermore, we introduce a generative editing pipeline that explicitly transforms latent representations via the 3D displacement field. By fully leveraging the model's generative capabilities, it achieves smooth geometric deformations while maintaining the object's identity. To further transfer original geometry and texture with high fidelity, we propose Logits-Level Token Fusion, which accurately injects source information into the destination region. To evaluate 3D point-based editing, we construct a novel dataset featuring diverse 3D assets and complex multi-point drag instructions. Quantitative and qualitative experiments demonstrate that 3DRAG achieves state-of-the-art performance, surpassing existing baselines in drag accuracy, geometric fidelity, and unedited region preservation.

1 Introduction

Precise, point-driven 3D manipulation remains difficult because existing drag methods rely on costly 2D priors or geometry warping that can damage consistency and structure. 3DRAG edits in native 3D space and combines displacement fields with generative editing to preserve object details; evaluations report state-of-the-art results against mesh-based and 3DGS-based baselines.

1 Introduction

  • Drag-based editing offers point-driven spatial control for users who need fine-grained manipulation beyond localized replacement or stylization.Source: §1 · p. 2
  • Existing 3D drag pipelines rely on 2D diffusion priors, leading to costly iterative optimization and risks of multi-view inconsistency and geometric artifacts.Source: §1 · p. 2
  • 3DRAG is a training-free framework for precise, high-fidelity drag editing in native 3D space, without SDS-based optimization or iterative point tracking.Source: §1 · p. 2
  • The generative editing pipeline uses Logits-Level Token Fusion to transfer source geometry and texture to new coordinates while preserving unedited regions and object identity.Source: §1 · p. 2
  • On a novel dataset of diverse 3D assets with complex multi-point drag instructions, quantitative and qualitative evaluations show 3DRAG’s clear superiority over representative mesh-based and 3DGS-based baselines.It achieves state-of-the-art performance in drag accuracy, unedited-region preservation, and overall 3D quality.Source: §1 · p. 2
  • Its 3D displacement field uses Occupancy-Aware Winner-Takes-All to handle sparse 3D latents and avoid topological tearing and geometric inconsistencies.Source: §1 · p. 2

2 Related Works

Related work moves from 2D-prior-driven 3D generation toward native 3D generative models, which improve efficiency but lack precise interactive manipulation. Drag editing spans 2D point-based methods and 3D representations, yet structural preservation and cross-view consistency remain challenges.

2.1 3D generative models

  • Native 3D models such as GET3D, CLAY, TRELLIS, and TripoSG improve generation efficiency and quality, but lack interactive mechanisms for fine-grained geometric manipulation.Source: §2.1 · pp. 2–3

2.2 Drag-based Editing

  • 2D drag-editing methods use point supervision, diffusion feature guidance, robustness constraints, or faster approximations, but remain prone to inconsistent geometry in multi-view settings.Source: §2.2 · p. 3

2.3 3D Editing

  • 3D editing progresses from NeRF manipulation to explicit 3DGS and native latent editing, while drag-based methods still face large-deformation and structural-preservation challenges.Source: §2.3 · p. 3

3 Method

3DRAG applies user-directed deformation in native 3D space, combining an occupancy-aware displacement field with generative editing designed to preserve source identity and details.

3 Method

  • 3DRAG applies a derived 3D displacement field to latents and attention representations to deform geometry while preserving original object details.The method describes editing as a displacement-field construction stage followed by identity-preserving generative editing.Source: §3 · p. 3

3.1 Preliminary

  • TRELLIS represents assets with sparse active-voxel coordinates and latent features; exact rectified-flow inversion caches latents and attention tokens for editing.The cached tokens provide geometric features and material details during editing.Source: §3.1 · p. 4

3.2 Construction of the 3D Displacement Field

  • The 3D elastic field approaches each handle's displacement near the handle and smoothly decays to zero at the deformation boundary.The field is constructed from user handle-target pairs over the editable region.Source: §3.2 · pp. 4–5
  • OA-WTA makes occupied source voxels dominate unoccupied ones when multiple points map to a destination, improving mesh topological consistency and reducing artifacts.The occupancy-aware rule addresses sparse-grid collisions that can otherwise create cavities or topological tearing.Source: §3.2 · p. 5

3.3 Identity-Preserving Generative Editing

  • The displacement field partitions space into destination, inpainting, preserving, and transition regions to guide editing and blending.Destination latents are reverse-mapped from the source, preserving and transition regions retain the inverted latent, and inpainting regions receive Gaussian noise.Source: §3.2 · p. 5, §3.3 · p. 5
  • Logits-Level Token Fusion appends mapped cached source Values and duplicates destination attention logits so source features receive destination-aligned attention.This addresses the negligible attention that destination tokens otherwise assign to source keys when 3D coordinates are entangled with features.Source: §3.3 · pp. 5–6

4 Experiments

Experiments report strong performance across editing accuracy, preservation, and 3D quality, supported by qualitative comparisons, ablations, and a user study. The evaluation uses a curated dataset of diverse 3D objects with multi-point drag instructions.

4.1 Setup

  • Implementation uses TRELLIS and Voxhammer on one NVIDIA H20 GPU, with sparse SLAT at resolution 643 and 25 inversion and denoising steps.Logits-Level Token Fusion and Unedited Region Preservation are activated across all self-attention layers.Source: §4.1 · p. 6
  • The benchmark uses 100 diverse objects with predefined multi-point instructions, comparing 3DRAG with mesh- and Gaussian-based methods across preservation, accuracy, quality, and runtime.Baselines include ARAP, APAP, 3DGS-Drag, and Drag Your Gaussian.Source: §4.1 · pp. 6–7

4.2 Main Results

  • 3DRAG achieves the best reported performance across unedited-region preservation, drag accuracy, and overall 3D quality, and is significantly faster than diffusion-based baselines.The reported metrics include Chamfer Distance, PSNR, SSIM, LPIPS, DAI, MD, FID, FVD, and GPTEval3D.Source: §4.2 · pp. 7–8
  • Qualitatively, 3DRAG preserves original geometry and texture, while ARAP, APAP, and SDS-based Gaussian methods exhibit distinct deformation or appearance artifacts.ARAP can stretch unnaturally, APAP can distort the global mesh, and Gaussian methods can produce color shifts, blur, and scattered Gaussians.Source: §4.2 · p. 8

4.3 Ablation Study

  • Removing occupancy awareness causes tearing, disconnected arm components, and floating artifacts; OA-WTA prioritizes occupied sources to protect valid geometry.The structural improvement is also quantitatively validated in Tab. 2.Source: Figure 7 · p. 7, §4.3 · p. 8, Table 2 · p. 8
  • Disabling SLAT-stage fusion impairs texture retrieval, while removing fusion in both stages also degrades geometric detail and produces over-smoothed blends.The full method preserves identity; Tab. 2 quantitatively corroborates improvements across metrics.Source: Figure 8 · p. 7, §4.3 · p. 8, Table 2 · p. 8

4.4 User Study

  • In a randomized blind study of 30 participants, 3DRAG outperforms all baselines in edit effectiveness, unedited-region preservation, and overall object quality.Thirty percent of participants were artists or 3D researchers; rankings were aggregated with Borda count.Source: §4.4 · p. 8, Table 3 · p. 8

5 Conclusion

3DRAG performs precise 3D-native drag editing without costly per-edit optimization, combining an occupancy-aware displacement field with generative editing and token fusion. Experiments report state-of-the-art visual quality and editing accuracy, while the method remains bounded by VAE color consistency and sparse-grid resolution.

5 Conclusion

  • 3DRAG executes complex spatial manipulations without costly per-edit optimization or reliance on 2D diffusion priors.Its 3D displacement field uses OA-WTA to preserve topological integrity, while exact diffusion inversion and Logits-Level Token Fusion support plausible deformation, texture blending, and unedited-region preservation.Source: §5 · p. 8
  • Experiments report state-of-the-art performance in visual quality and editing accuracy.The conclusion frames this as supporting intuitive 3D content creation.Source: §5 · p. 8
  • Occasional color shifts stem primarily from TRELLIS VAE reconstruction, and drag precision is bounded by the sparse voxel grid’s current 643 resolution.Fine-grained instructions on extremely thin structures or intricate details may therefore yield suboptimal deformations.Source: §5 · p. 8

Related papers

Loading 2610.13216v1…