Source-linked AI summary
SpatialEdit: Benchmarking Fine-Grained Image Spatial Editing
Yicheng Xiao, Wenhu Zhang, Lin Song, Yukang Chen, Wenbo Li, Nan Jiang, Tianhe Ren, Haokun Lin, Wei Huang, Haoyang Huang, Xiu Li, Nan Duan, Xiaojuan Qi
TL;DR
Fine-grained spatial editing remains brittle because existing systems often miss geometric intent and evaluation is underdeveloped. The paper introduces a geometry-aware benchmark, synthetic training dataset, and baseline model, which remains competitive on general editing while substantially improving spatial tasks.
Problem
Precise spatial control remains brittle because existing editors often miss metric or viewpoint intent, while spatial-editing evaluation cannot reliably distinguish visual plausibility from geometric correctness.
Method
The paper introduces SpatialEdit-Bench, SpatialEdit-500k, and SpatialEdit-16B using geometry-aware evaluation and controllable Blender-generated paired supervision.
Results
0.300 points: SpatialEdit surpasses LongCatImage-Edit on moving scores while maintaining comparable general-editing performance at 7.52 on GEdit-Bench.
Takeaways & Limitations
The benchmark, dataset, and model support reproducible progress on challenging object-manipulation and camera-control tasks.
Abstract
from arXiv · showhide
Image spatial editing performs geometry-driven transformations, allowing precise control over object layout and camera viewpoints. Current models are insufficient for fine-grained spatial manipulations, motivating a dedicated assessment suite. Our contributions are listed: (i) We introduce SpatialEdit-Bench, a complete benchmark that evaluates spatial editing by jointly measuring perceptual plausibility and geometric fidelity via viewpoint reconstruction and framing analysis. (ii) To address the data bottleneck for scalable training, we construct SpatialEdit-500k, a synthetic dataset generated with a controllable Blender pipeline that renders objects across diverse backgrounds and systematic camera trajectories, providing precise ground-truth transformations for both object- and camera-centric operations. (iii) Building on this data, we develop SpatialEdit-16B, a baseline model for fine-grained spatial editing. Our method achieves competitive performance on general editing while substantially outperforming prior methods on spatial manipulation tasks. All resources will be made public at https://github.com/EasonXiao-888/SpatialEdit.
1 Introduction
SpatialEdit targets fine-grained geometric transformations in images, addressing brittle spatial control and underdeveloped evaluation. The paper introduces a geometry-aware benchmark, a scalable synthetic training dataset, and a model that improves spatial editing while remaining competitive on general editing.
- Spatial editing covers camera-centric viewpoint changes and object-centric transformations such as translation, scaling, and rotation.These operations are intended to support world-modeling, embodied perception, interactive content creation, simulation, and 3D reasoning.
- Existing systems often require expert camera trajectories, miss metric or viewpoint intent, or generalize narrowly across spatial operations.These failure modes create a gap between semantic alignment and faithful geometric compliance.
- SpatialEdit-Bench evaluates object- and camera-level editing using framing analysis and reconstructed 3D viewpoint error.The authors report higher reliability than vision-language judging in controlled fine-grained pose validation.
- SpatialEdit-500k uses a controllable Blender pipeline to generate paired spatial supervision with geometric transformations and textual instructions.The pipeline combines rendered GLB assets, viewpoint coverage, VLM verification, SAM3 masks, generated backgrounds, and inpainting.
- SpatialEdit-16B remains competitive on general editing while substantially improving spatial tasks across object manipulation and camera control.The reported gains target the regime where prior systems most often fail.
- 0.300 and 0.127 points: SpatialEdit surpasses LongCatImage-Edit on moving and rotation scores, respectively, while achieving the lowest camera-control error.It maintains comparable general-editing performance at 7.52 on GEdit-Bench.
2 Related Work
Prior work includes instruction-based image editing and spatially conditioned generative modeling, while SpatialEdit-500k provides dataset statistics spanning camera operations, object bounding boxes, and categories.
- Instruction-based editing modifies images from natural-language commands while preserving overall semantics, but depends on large-scale instruction-faithful supervision.
- Spatially conditioned generative models explore camera-motion, 6-DoF, camera-aware, and geometric-control interfaces for explicit spatial manipulation.
- SpatialEdit-500k reports camera-level distributions across seven subtasks, object-level bounding-box aspect ratios, and object-category statistics.The camera statistics distinguish outdoor and indoor scenes and denote yaw, pitch, and distance with Y, P, and D.
3 Image Spatial Editing
Image spatial editing unifies object-centric manipulation and camera-centric view control, but existing methods often produce plausible images with incorrect geometry. SpatialEdit addresses this gap through a geometry-focused benchmark and a controllable synthetic-data pipeline.
- Task Definition: Spatial editing covers object translation, scaling, and rotation alongside camera yaw, pitch, and zoom manipulation.The framework distinguishes object-centric and camera-centric operations while preserving scene coherence.
- Data Generation: The object-centric pipeline curates and validates 3D assets, renders canonical views, and generates controlled viewpoint variations.Assets are filtered for frontal-view correctness before multiple viewpoints are rendered.
- Data Generation: The camera-centric pipeline samples yaw, pitch, and distance around salient scene objects in Blender, then filters invalid renderings.Detection and vision-language-model checks remove missing, occluded, truncated, or implausible scenes.
- SpatialEdit-Benchmark: SpatialEdit-Bench evaluates object scaling, translation, rotation, and camera viewpoint adjustment using geometric accuracy, semantic consistency, and structural preservation.The benchmark is designed for spatial transformations rather than appearance-only edits.
- SpatialEdit-Benchmark: Moving Score combines bounding-box IoU with object consistency through a geometric mean, penalizing imbalance between spatial accuracy and semantic fidelity.The object consistency score assesses subject integrity and environmental coherence after translation.
- SpatialEdit-Benchmark: Viewpoint Error reconstructs camera parameters with VGGT, while Framing Error measures object-layout alignment through matched ray directions.VGGT provides globally consistent 3D estimates, and ray-angle differences quantify predicted-target framing alignment.
4 Image Spatial Editing Model
SpatialEdit uses a cascaded multimodal editing pipeline that conditions latent denoising on instruction and image representations. Training first adapts general image editing, then specializes spatial control with LoRA post-tuning.
- Model Architecture: The pipeline combines instruction embeddings from a vision-language model with VAE image latents and MMDiT denoising to produce edited images.The denoised latent is decoded by the VAE into the final output.
- Training Strategy: Training adapts the model to image editing on public editing data before specializing it for spatial editing on a curated dataset.The two stages are general editing fine-tuning followed by spatial-editing specialization.
- Training Strategy: LoRA post-tuning improves transformation control while preserving general editing priors.
5 Experiments
Experiments evaluate SpatialEdit on spatial and general editing, training combinations, camera metrics, qualitative manipulation, and single-view reconstruction. The model achieves strong spatial-editing results while remaining competitive on general editing.
- Quantitative Results: 0.673 is SpatialEdit’s object moving score, while its object rotation score is 0.632 on SpatialEdit-Bench.It also attains viewpoint error 0.243 and framing error 0.527.
- Quantitative Results: 0.358 lower overall camera error than LongCatImage-Edit establishes SpatialEdit’s improvement on camera control.
- General Editing: SpatialEdit achieves competitive general-editing performance on GEdit-Bench, scoring 7.52 while providing a foundation for spatial fine-tuning.
- Ablation Studies: Training all three tasks yields the best reported trade-off: Mov 0.673, Rot 0.632, and Cam 0.385.The combinations are object moving, object rotation, and camera operation tasks.
- Camera Metrics: Viewpoint Error has the highest Spearman correlation among camera metrics, followed by Framing Error, and both outperform GPT.The evaluation ranks rendered views against their known ordering.
- Qualitative Results: Qualitatively, SpatialEdit follows camera shifts more faithfully and produces cleaner object edits with stronger background preservation than prior baselines.The comparisons cover zoom, yaw, pitch, rotation, tilt, object movement, and object reorientation.
- Single-view Reconstruction: Synthesizing novel camera viewpoints with SpatialEdit adds geometric constraints that improve single-view 3D reconstruction.The pipeline targets settings without multi-view observations.
6 Conclusion
The paper presents fine-grained spatial editing, its benchmark and controlled dataset, and SpatialEdit-16B as a strong baseline. Together, these resources target reproducible progress in geometric image editing.
- Contributions: The paradigm combines object manipulation with explicit geometric control of camera viewpoints.
- Contributions: SpatialEdit-Bench evaluates perceptual plausibility and geometric fidelity through viewpoint reconstruction and compositional analysis.
- Contributions: SpatialEdit-500k is a controlled Blender-based dataset with diverse scenes and 3D assets for spatial-editing training.
- Contributions: SpatialEdit-16B remains competitive on general editing while substantially advancing performance on challenging spatial manipulation tasks.
A Implementation Details
Implementation uses pre-training on general editing data followed by fine-tuning on SpatialEdit-500k. The supplied implementation passages specify optimization settings and identify SpatialEdit-Bench as the evaluation benchmark.
- Training: Pre-training uses general editing data while explicitly excluding spatially edited samples.The model is then fine-tuned on SpatialEdit-500k using LoRA.
- Optimization: AdamW uses β1 = 0.9, β2 = 0.95, and a learning rate of 1 × 10^-4 during pre-training.
- Optimization: A linear warm-up runs for the first 1000 iterations before standard decay.
- Evaluation: Table 6 reports performance on the proposed SpatialEdit-Bench.
B More World Model Results
The paper compares multiple video world models for precise camera viewpoint editing, including both closed-source and open-source systems.
- The comparison covers ViduQ2-Turbo, Kling-V2.5, Veo3.1, LingBot-World, and ReCamMaster for camera viewpoint editing.The first three are closed-source, while the latter two are open-source.
C Metrics in SpatialEdit-Bench
SpatialEdit-Bench uses algorithmic procedures to clarify how its evaluation metrics are calculated.
- Algorithms 1 and 2 provide the metric calculation process used in SpatialEdit-Bench.
D More Qualitative
The qualitative and evaluation materials distinguish object-level from camera-level spatial editing and define the inputs, outputs, and metric components used for each.
- D More Qualitative: Object-level evaluation compares open-source and closed-source editing models, while camera-level evaluation focuses on different world models.
- D More Qualitative: Algorithm 1 evaluates object-level spatial editing using source and predicted images, a ground-truth bounding box, and an object detector.
- D More Qualitative: Moving Score combines bounding-box IoU with object consistency, while Rotation Score combines view correctness with appearance consistency.The Object Overall Score is the sum of Moving Score and Rotation Score.
- D More Qualitative: Algorithm 2 evaluates camera-level editing from source, ground-truth, and predicted images using a command, detector, and VGGT pose model.
- D More Qualitative: Camera evaluation reports viewpoint error, framing error, and Camera Overall Error based on pose differences and matched object framing.Framing uses ray directions, object matching, and area-ratio differences.
- D More Qualitative: Figures 8 and 9 compare methods separately for object-level and camera-level manipulation.