Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting

Motivation

• DYG targets the geometric-editing and spatial-control limitations of text-guided 3DGS methods, which mainly support texture or style changes.Text struggles to specify the desired editing extent, while geometric edits such as structural deformation remain difficult.Source: §1 · p. 2

Approach

• DYG lets users specify editing regions and dragging directions with 3D masks and paired 3D control points.This extends the intuitive control-point paradigm from 2D drag-based editing to 3D scenes.Source: §1 · p. 2
• MTP encoding, RSP decoding, and Soft Local Edit combine implicit triplanes with 3DGS to construct smooth geometry while preserving surrounding regions.The design addresses uneven Gaussian distributions and limits edits to the desired area.Source: §1 · p. 2
• Drag-SDS uses a 2D drag-based latent diffusion model to provide perceptually plausible, fine-grained, multi-view-consistent scene dragging.The method uses diffusion supervision after establishing the geometric scaffold.Source: §1 · p. 2

Results

• Extensive qualitative and quantitative experiments report state-of-the-art 3D scene editing results for deformation, transformation, and morphing.The reported evaluation supports DYG’s flexibility and generalization across geometric editing tasks.Source: §1 · p. 2

2.1 Drag-based Image Editing

• Drag-based image editing evolved from DragGAN’s control-point latent optimization to LDM-based methods with broader generalization but costly latent optimization.Lightning-Drag injects point embeddings into Stable Diffusion’s inpainting attention modules to enable faster interactive editing.Source: §2.1 · pp. 2–3
• DYG adopts Lightning-Drag [31] as its guiding model because it provides rapid, high-quality drag-based image editing.The 2D model supplies the generative guidance used for 3DGS editing.Source: §2.1 · p. 3

2.2 3D Editing for Radiance Fields

• 3D editing methods use object-compositional fields, explicit geometry, or learned anchors, but remain limited by coarse manipulation, reconstruction dependence, editing diversity, or tearing artifacts.These approaches include object-level editing, cages or point clouds, and sampling-based anchor representations.Source: §2.2 · p. 3
• Text-guided latent diffusion methods such as GS-Editor [6] and GS-Ctrl [37] support texture or style changes but often fail at geometric edits and precise spatial extent control.DYG instead targets flexible, fine-grained geometric editing while maintaining plausible scenes.Source: §2.2 · p. 3

3.1 3D Gaussian Splatting

• 3DGS represents scenes with anisotropic Gaussians containing position, scale, rotation, opacity, and spherical-harmonics coefficients.The Gaussians are projected to 2D and rendered in real time with a tiled rasterizer.Source: §3.1 · p. 3

3.2 Score Distillation Sampling (SDS)

• SDS uses a pretrained 2D latent diffusion model to guide optimization of a differentiable 3D representation through rendered images.For representation G and camera pose c, rendering produces x = R(G,c), which is encoded into latent z = E(x).Source: §3.2 · pp. 3–4
• The SDS objective compares ground-truth noise with diffusion-model-predicted noise at timestep t, weighted by a timestep-dependent function.The reformulated objective also uses the dynamic weighting term ᾱ(t).Source: §3.2 · p. 4

4.1 Problem Definition and Method Overview

• 3D masks and paired 3D control points specify the editing region, dragging direction, and view-projected guidance used to optimize the Gaussian scene.The method preserves an initial copy of the scene to render view-specific images and masks for the 2D drag-based diffusion model.Source: §4.1 · p. 4
• DYG combines explicit 3DGS with implicit triplanes to overcome sparse target regions and support localized drag-based geometric editing.The method is designed for deformation, transformation, and morphing tasks while preserving unmasked regions as much as possible.Source: §4.1 · p. 4, §4.2 · p. 4

4.2 Smooth Geometric Editing

• The MTP encoder represents Gaussian positions with multi-resolution features, while the RSP decoder predicts position shifts separately for masked and unmasked regions.The triplane uses three orthogonal learnable planes, and the RSP design adds correction for unintended movement outside the edit mask.Source: §4.2 · pp. 4–5
• Two-stage dragging first optimizes the geometric scaffold with the MTP encoder and RSP decoder, then refines Gaussian appearance while reactivating densification and pruning.The first stage freezes Gaussian attributes and structural operations; the second primarily optimizes color, opacity, rotation, and scale.Source: §4.2 · p. 5
• Soft local editing optimizes K-nearest unmasked neighbors at a reduced learning rate, smoothing transitions that strict freezing can make disjoint.A region regularization loss further encourages unmasked Gaussians to remain unchanged.Source: §4.2 · p. 5

4.3 Score Distillation Guidance

• Drag-SDS combines target and source noise predictions from separate diffusion UNets with latent-space, image-space, and LoRA objectives.The inpainting UNet predicts target noise from masked image and control-point conditions, while the original Stable Diffusion UNet with LoRA estimates source noise.Source: §4.3 · pp. 5–6

• Approximately 10 minutes are required for one dragging operation on an A100-40G GPU, whereas GS-Editor requires over 20 minutes.Source: §5 · p. 7

6.1 Evaluation Setup

• Evaluation uses six representative scenes from Mip-NeRF360 [1] and Instruct-NeRF2NeRF [9], covering more than 20 tasks involving faces, indoor objects, and outdoor scenes without directly comparable 3DGS drag-editing baselines.The study compares against GS-Editor [6], GS-Ctrl [37], SC-GS [13], and a 2D-Lifting baseline, with additional efforts to guide these methods fairly.Source: §6.1 · p. 7, §6.2 · p. 7

6.2 Qualitative Evaluation

• DYG turns a face to one side while preserving details and multi-view consistency, whereas competing methods produce distortions, ineffective geometric changes, or blurry inconsistent scenes.The comparison includes SC-GS, GS-Editor, GS-Ctrl, and 2D-Lifting; additional scenes show plausible, high-quality edits guided by control points.Source: §6.2 · p. 7

6.3 Quantitative Evaluation

• User studies found that 86.1% of users favored DYG for editing effect and 62% favored it for scene quality, based on 1,500 votes.The study included 75 users comparing 10 edited scenes per questionnaire.Source: §6.3 · p. 7, Figure 9 · p. 8
• DYG achieves the best performance across Scene Quality, Editing Effect, Retention of Initial Features, and GPT-Overall, while 2D-Lifting suffers from multi-view inconsistencies and low scene quality.Table 1 reports these metrics across various scenes; GS-Editor and GS-Ctrl show weaker editing effectiveness, while 2D-Lifting generally achieves geometric edits but produces blurry results.Source: §6.3 · p. 8, Table 1 · p. 8

6.4 Ablation Study

• The full Smooth Geometric Editing design progressively adds MTPE, RPD, two-stage optimization, Local Edit, and Soft Local Edit, with Soft Local Edit achieving the most harmonious and consistent visual results.Earlier variants leave Gaussian artifacts, inadequately support fine-grained dragging, modify backgrounds, or introduce cracks around masked regions.Source: §6.4 · pp. 8–9, Figure 7 · p. 8, Figure 10 · p. 9
• Drag-SDS enables target editing more effectively than SDS, while the original SD UNet avoids the inconsistent color layers produced by an inpainting UNet.SDS blurs the target region, whereas inpainting-based Drag-SDS can over-focus on the mask and create disharmonious color layers.Source: Figure 8 · p. 8, §6.4 · p. 9

6.5 Multi-round Dragging

• DYG remains stable across multi-round edits to different or repeated targets and generalizes to high-quality, precise dragging in 3D generative scenes.The generative-scene experiments use Director3D [16] scenes, while multi-round edits include sequential limb changes and gradually raising the same arm.Source: §6.5 · p. 9, §6.6 · p. 9, Figure 11 · p. 9

7 LIMITATION AND CONCLUSION

• DYG’s 3D editing capabilities are inherently limited by the performance of the 2D drag-based generative models from which it distills prior knowledge.Source: §7 · p. 9
• DYG enables convenient, flexible, fine-grained, high-quality editing of 3D Gaussian scenes using 3D masks and control points.Source: §7 · pp. 9–10
• Extensive experiments demonstrate the effectiveness and generalization of DYG.Source: §7 · p. 10

A.1 Training Setup

• Gaussian reconstruction is optimized for 7,000 iterations with spherical harmonics set to degree 0 and batch size 4.Source: §A.1 · p. 12
• The first stage freezes Gaussian attributes and trains only the MTP encoder and RSP decoder for positional shifts.Source: §A.1 · p. 12
• The second stage lowers the MTP learning rate to 1 × 10^-4, stabilizes geometry, and begins training the other Gaussian attributes.Source: §A.1 · p. 12

A.2 Two-Stage Dragging

• Diffusion timesteps follow a cosine annealing schedule over the range [0.98, 0.02].Source: §A.2 · p. 12
• A threshold of 0.7T yields a first-stage geometric reconstruction epoch ratio of s = 0.36.Source: §A.2 · p. 12

A.3 Classifier-Free Guidance

• CFG uses inverse-square annealing to avoid over-saturation, with guidance applied to target noise prediction but not source noise prediction.Source: §A.3 · p. 12

A.4 Total Loss

• The total loss uses λd-sds = 1 and λrr = 2500.Source: §A.4 · p. 12
• A failure case occurs when Lightning-Drag cannot produce the desired eye-closing edit, because DYG uses Lightning-Drag as guidance.Source: Figure 14 · p. 12

B DETAILS OF THE RSP DECODER

• The RSP decoder uses two MLPs that predict positional shifts for desired and undesired editing areas from multi-scale features.Source: B DETAILS OF THE RSP DECODER · p. 12, Figure 13 · p. 12
• Equation (16) combines the positional-shift predictions to prevent the severe geometric tearing caused by directly using the undesired-area predictor.Source: B DETAILS OF THE RSP DECODER · p. 12

• DYG and Lightning-Drag both fail when the user expects an eye to close.Source: C FAILURE CASE · p. 12