PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Scalable and Consistent 3D Editing

Ruihao Xia, Yang Tang, Pan Zhou

arXiv:2510.02994v1cs.CV

TL;DR

3D editing remains more challenging than 2D editing because it must simultaneously ensure cross-view geometric consistency, global structural fidelity, and fine-grained controllability. The paper builds 3DEditVerse, a paired 3D editing benchmark, and proposes 3DEditFormer, which uses dual-guidance attention and time-adaptive gating to preserve source structure during editing. 3DEditFormer achieves state-of-the-art 3D editing performance, with a +13% improvement on 3D metrics over VoxHammer without auxiliary 3D masks.

  • Problem

    3D editing remains more challenging than 2D editing because it must simultaneously ensure cross-view geometric consistency, global structural fidelity, and fine-grained controllability.

    Source: §1 · p. 1

  • Method

    The paper builds 3DEditVerse, a paired 3D editing benchmark, and proposes 3DEditFormer, which uses dual-guidance attention and time-adaptive gating to preserve source structure during editing.

    Source: §1 · p. 2

  • Results

    3DEditFormer achieves state-of-the-art 3D editing performance, with a +13% improvement on 3D metrics over VoxHammer without auxiliary 3D masks.

    Source: §1 · p. 2, §5.1 · p. 8, §6 · p. 9

  • Takeaways & Limitations

    The benchmark and model combine high-fidelity editing with practicality by enabling consistent edits without relying on 3D masks.

    Source: §6 · p. 9

    Latent-space editing may lose precision on high-resolution 3D assets and degrade fine geometric details.

    Source: §6 · p. 9

Abstract

from arXiv · show

3D editing - the task of locally modifying the geometry or appearance of a 3D asset - has wide applications in immersive content creation, digital entertainment, and AR/VR. However, unlike 2D editing, it remains challenging due to the need for cross-view consistency, structural fidelity, and fine-grained controllability. Existing approaches are often slow, prone to geometric distortions, or dependent on manual and accurate 3D masks that are error-prone and impractical. To address these challenges, we advance both the data and model fronts. On the data side, we introduce 3DEditVerse, the largest paired 3D editing benchmark to date, comprising 116,309 high-quality training pairs and 1,500 curated test pairs. Built through complementary pipelines of pose-driven geometric edits and foundation model-guided appearance edits, 3DEditVerse ensures edit locality, multi-view consistency, and semantic alignment. On the model side, we propose 3DEditFormer, a 3D-structure-preserving conditional transformer. By enhancing image-to-3D generation with dual-guidance attention and time-adaptive gating, 3DEditFormer disentangles editable regions from preserved structure, enabling precise and consistent edits without requiring auxiliary 3D masks. Extensive experiments demonstrate that our framework outperforms state-of-the-art baselines both quantitatively and qualitatively, establishing a new standard for practical and scalable 3D editing. Dataset and code will be released. Project: https://www.lv-lab.org/3DEditFormer/

1 INTRODUCTION

3D editing must balance cross-view consistency, structural fidelity, and fine-grained control, while existing approaches can be slow, distort geometry, or require error-prone manual masks. The paper introduces a paired editing benchmark and a structure-preserving transformer, reporting state-of-the-art performance without auxiliary 3D masks.

1 INTRODUCTION

  • Practical 3D editing must preserve geometry across views, maintain global structure, and offer fine-grained control, while existing methods can be slow, distort geometry, or depend on error-prone masks.The paper frames the challenge as enabling precise, localized edits from intuitive prompts while maintaining structural fidelity across views.Source: §1 · pp. 1–2
  • 3DEditVerse provides 116,309 paired assets for training and 1,500 curated test pairs, combining pose-driven geometric edits with foundation-model-guided appearance edits.The benchmark is designed for localized edits, multi-view consistency, and semantic harmony.Source: §1 · p. 2
  • 3DEditFormer injects source-asset features through dual-guidance attention and time-adaptive gating to balance early semantic edits with later structural fidelity.This enables localized, consistent edits without manual 3D masks or external constraints.Source: §1 · p. 2
  • When trained on 3DEditVerse, 3DEditFormer achieves state-of-the-art 3D editing performance and, unlike VoxHammer (Li et al., 2025a), requires no auxiliary 3D masks while yielding an average +13% improvement on 3D metrics.Source: §1 · p. 2

2 RELATED WORK

The paper reviews limitations in 3D editing and presents paired-data pipelines alongside a source-guided transformer for localized, structure-preserving edits. Its method combines complementary features and timestep-dependent gating, with a two-stage generation and training setup.

2 RELATED WORK

  • Prior 3D editing methods are slow, inconsistent across views, or reliant on manual 3D masks, while paired training data remains scarce.SDS methods can take tens of minutes per edit; multi-view editing can distort geometry, and existing datasets have limited scale, correspondences, or realism.Source: §2 · p. 3, §3 · p. 3

3.1 CHARACTER–ANIMATION COMPOSITIONS FOR GEOMETRIC EDITS

  • DINOv2 similarity pruning selects 4,998 distinct poses, which are paired across 108 characters to form 54,000 pose-driven assets.The pose pairs use different articulations of the same character to capture pose and shape alterations.Source: §3.1 · p. 4

3.2 TEXT-GUIDED GENERATIVE PIPELINE FOR APPEARANCE EDITS

  • Text-to-image editing creates source–prompt–target pairs, while additional Objaverse-XL examples diversify instructions and align them with real 3D geometries.The pipeline combines these data with pose-driven pairs to produce about 118K paired assets, including 64,123 appearance-based edits and 54,000 structural edits.Source: §3.2 · pp. 4–5
  • Appearance edits are localized with an open-set detector and multi-view masks, then applied through masked latent repainting and filtered by rendered-view feature similarity.The pipeline projects evidence across views to form a 3D mask and discards samples whose mean cosine similarity falls below a threshold.Source: §3.2 · pp. 4–5

4.1 DUAL-GUIDANCE ATTENTION BLOCK

  • Dual-Guidance Attention adds parallel source-feature cross-attention branches to the frozen Trellis backbone, injecting structural priors into each layer.The fused representation then proceeds through the original target-image cross-attention and feed-forward network.Source: §4.1 · p. 6

4.2 MULTI-STAGE FEATURE EXTRACTION

  • Multi-stage features separate late-step structural detail for preserving unedited regions from early-step semantic cues for reflecting the requested edit.The feature streams are extracted from the frozen Trellis network at different denoising timesteps.Source: §4.2 · p. 6

4.3 TIME-ADAPTIVE GATING

  • Time-adaptive gating weights semantic-transition features more at early timesteps and structural features more at later timesteps.The paper describes this feature disentanglement as enabling localized, consistent, structure-preserving edits.Source: §4.3 · p. 7

4.4 TRAINING AND INFERENCE

  • Training and inference follow Trellis’s two-stage generation paradigm, with independent transformers trained under the same Conditional Flow Matching objective.The first transformer generates coarse voxelized shapes; the second refines texture and appearance before decoding to 3D representations.Source: §4.4 · p. 7

5 EXPERIMENTS

3DEditFormer outperforms prior methods on 3DEditVerse, including without auxiliary 3D masks. Ablations report gains from structural and semantic guidance and time-adaptive gating.

5 EXPERIMENTS

  • Evaluation uses 100,000 sampled mesh points for 3D metrics and renders each mesh from 10 fixed viewpoints for 2D metrics.The reported metrics include CD, NC, and F10.01 for 3D, and PSNR, SSIM, LPIPS, and DINO-I for 2D; training fine-tunes 252M parameters for 40k iterations with batch size 16.Source: §5 · p. 7

5.1 MAIN RESULTS

  • Qualitative comparisons report that 3DEditFormer preserves structure and consistency, while EditP23 and Instant3dit show fidelity problems and VoxHammer depends on accurate masks.Examples include adding a secondary ship without distorting the original vessel and removing a soldier’s rifle while maintaining uniform integrity.Source: §5.1 · pp. 7–8, Figure 4 · p. 8
  • 3DEditFormer achieves a +13% improvement of 3D Metrics than VoxHammer without any auxiliary 3D mask.It surpasses prior methods on CD, NC, F1, PSNR, LPIPS, and DINO-I while remaining competitive on SSIM.Source: §5.1 · p. 8, Table 2 · p. 9

5.2 ABLATION STUDY

  • Ablations report weak baseline fidelity, gains from fine-grained structural features on CD, NC, and F1, further 2D and 3D gains from semantic transition features, and the strongest all-metric performance with adaptive gating.The gating dynamically balances the two feature contributions across denoising steps.Source: §5.2 · p. 9, Table 3 · p. 9

6 CONCLUSION

The paper introduces a paired 3D editing benchmark and a structure-preserving transformer, reporting state-of-the-art performance. It also notes a precision limitation for high-resolution assets edited in latent space.

6 CONCLUSION

  • 3DEditVerse provides about 118K paired assets with geometry- and appearance-driven edits, designed for scalability, consistency, and semantic alignment.The benchmark supports training and evaluating 3D editing models.Source: §6 · p. 9
  • 3DEditFormer uses dual-guidance attention and time-adaptive gating to separate editable regions from preserved structure, achieving state-of-the-art 3D editing performance without 3D masks.The conclusion describes the results as combining high fidelity with strong practicality.Source: §6 · p. 9
  • Latent-space editing may lose precision on high-resolution assets, degrading fine geometric details.The paper identifies editing directly in the original 3D domain as a possible direction for better preserving fine-grained mesh fidelity.Source: §6 · p. 9

A VISUALIZATION OF OUR 3DEDITVERSE DATASET

Figure 5 presents examples from two kinds of 3DEditVerse data: character–animation compositions and text-guided generative edits.

A VISUALIZATION OF OUR 3DEDITVERSE DATASET

  • Figure 5 shows character–animation compositions alongside generative data from text-guided editing.The caption identifies these as examples from the proposed 3DEditVerse dataset.Source: Figure 5 · p. 14

B DETAILS OF INSTRUCTIONAL PROMPTS

The appendix illustrates prompts and outputs for generating diverse object descriptions, creating source images, producing shape edits, and localizing edited regions.

B.1 PROMPT EXAMPLE OF DEEPSEEK-R1 FOR VOCABULARY TO TEXT

  • Each vocabulary word receives 20 diverse textual descriptions to increase edit diversity.The example uses “plane” and asks for object attributes while excluding background, environment, and action descriptions.Source: §B.1 · p. 15

B.2 PROMPT EXAMPLE OF FLUX.1-DEV FOR SOURCE IMAGE GENERATION

  • For each textual description, a template places the object at the center of a pure white background in Flux.1-Dev images, creating a clean, unambiguous layout for reliable image-to-3D lifting.Source: §B.2 · p. 15

B.3 PROMPT EXAMPLE OF QWEN-VL FOR EDIT PROMPT GENERATION

  • Qwen-VL prompts request five substantial, visible-part shape edits limited to adding, removing, or replacing geometry.They exclude changes to color, texture, material, and lighting.Source: §B.3 · p. 16
  • Generated source images are sent to Qwen2.5-VL-7B for edit instructions, then paired with those instructions for Flux.1-Kontext image editing.The example airplane edits include removing a stabilizer and adding winglets.Source: §B.3 · p. 16

B.4 PROMPT EXAMPLE OF QWEN-VL FOR EDIT REGION LOCALIZATION

  • For edit-region localization, Qwen-VL-Max receives side-by-side source and target images and predicts bounding boxes for the modification.The example prompt asks for coordinates in the 512 × 512 left image, or an empty list if the region cannot be found.Source: §B.4 · pp. 16–17
  • Figure 6 examples place the generation prompt below each source image and the editing instruction below its target image.The figure caption describes source–target image pairs.Source: Figure 6 · p. 17

C DETAILS OF 3D EDITING VIA REPAINT

The Repaint-based editing derivation adds noise to the source latent and uses a binary 3D mask to confine target changes to the edit region.

C DETAILS OF 3D EDITING VIA REPAINT

D VISUALIZATION OF COMPARISON WITH SOTA METHODS

Figure 7 provides further qualitative results comparing the proposed approach with state-of-the-art methods.

D VISUALIZATION OF COMPARISON WITH SOTA METHODS

  • Figure 7 presents additional qualitative results compared with state-of-the-art methods.Source: Figure 7 · p. 18

E VISUALIZATION ON CHARACTER–ANIMATION TEST SET

On the Character–Animation Test Set, 3DEditFormer generates target 3D assets that capture articulated geometry and preserve texture fidelity and consistency across views.

Related papers

Loading 2510.02994v1…