PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Aware 3D Gaussian Editing for Driving Scene Generation

Feng Zhou, Jian Zhang, Yuhang Sun, He Wang, Qiong Wen, Debao Kong, Tieru Wu, Rui Ma

arXiv:2605.25373v1cs.CV

TL;DR

Existing 3DGS editing methods offer limited road-geometry editing coupled to plausible vehicle-road dynamics, while many optimization-based edits take minutes. RoVES combines single-image road insertion with half-car-model pose correction; Waymo experiments report practical efficiency and competitive visual consistency.

  • Problem

    Existing 3DGS editing methods lack road-height irregularity editing coupled to vehicle dynamics, and optimization-based editing can require minutes per edit.

    Source: §I · p. 1

  • Method

    RoVES inserts road geometry from a single image and couples edited road profiles to a half-car model for framewise vehicle pose correction.

    Source: §I · p. 1

  • Results

    Waymo experiments report practical efficiency and competitive geometric and visual quality; road insertion takes 1.84 s and the full pipeline takes 6.24 s.

    Source: §I · p. 1

  • Takeaways & Limitations

    RoVES offers a practical step toward physics-aware driving-scene generation, particularly for road editing, vehicle-response modeling, and efficient batch generation.

    Source: §V · p. 8

    The physics module models only vertical displacement and pitch and requires manually specified vehicle parameters, limiting direct use across vehicle types without per-vehicle tuning.

    Source: §VI · p. 8

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) has shown great potential in autonomous driving simulation and data generation, enabling photorealistic reconstruction and flexible scene manipulation. However, existing 3DGS scene editing methods have limited support for road geometry editing (e.g., inserting speed humps or sunken roads), and generally do not couple such edits with plausible vehicle-road interaction dynamics. Such editing is essential for generating training data under extreme driving scenarios or evaluating system reliability under these road irregularities. Moreover, many optimization-based methods require minutes of per-edit refinement, while existing efficient alternatives mainly focus on appearance-level or object-level manipulation rather than physics-aware road irregularity editing. To address these limitations, we propose RoVES, a Road-and-Vehicle Editing System for physics-aware 3D Gaussian editing in driving scenes. RoVES enables single-image-driven road geometry insertion and couples the edited road profile with a 4-DOF half-car vehicle dynamics model to achieve physics-aware vehicle pose correction in vertical displacement and pitch. RoVES inserts road elements in a one-shot, optimization-free pipeline (1.84s), and the full pipeline (including color transfer and vehicle-dynamics-based pose correction) completes in 6.24s; it edits dynamic vehicles via pose editing and corrects poses frame-by-frame to approximate dynamics-consistent vertical displacement and pitch responses. Experiments on the Waymo dataset show that RoVES provides practical efficiency and competitive visual consistency for physics-aware driving scene generation.

I. INTRODUCTION

The paper targets road-geometry editing that couples road irregularities to plausible vehicle responses, and introduces RoVES as an optimization-free 3DGS editing framework. Its Waymo experiments report practical efficiency and competitive geometric and visual quality.

I. INTRODUCTION

  • Road irregularities such as humps, potholes, and sunken roads can induce vehicle displacement and pitch, contributing to sensor offsets and challenging long-tail scenarios.Source: §I · p. 1
  • Existing editing methods lack metric road-height excitation for vehicle dynamics, while iterative methods take minutes and efficient alternatives mainly support object-level edits.Source: §I · p. 1
  • RoVES couples edited road geometry with a half-car dynamics model to correct vehicle poses frame by frame for vertical displacement and pitch.Source: §I · p. 1
  • Its optimization-free framework separates static road insertion from dynamic vehicle pose adjustment, requiring no backpropagation or fine-tuning.Object insertion takes 1.84 s and the full pipeline takes 6.24 s.Source: §I · p. 1
  • Road elements are inserted from a single image and text prompt using KNN adaptive scale and Lab color transfer.Source: §I · p. 1
  • Waymo comparisons and ablations indicate practical efficiency, physics-aware vehicle responses, and competitive geometric and visual quality.Source: §I · p. 1

A. 3D Gaussian Splatting Scene Editing Methods

Prior 3DGS editing methods trade flexibility for speed, while reconstructed-scene simulation often omits road-induced vertical vehicle dynamics. RoVES addresses the joint need for road editing, rapid generation, and physics-aware responses.

A. 3D Gaussian Splatting Scene Editing Methods

B. Vehicle-Road Interaction Physics and Data Generation

III. METHODOLOGY

RoVES edits a frozen, pose-decoupled 3DGS scene through separate static-road and dynamic-vehicle branches. Road geometry can be inserted in a one-shot pass, while vehicle poses can be directly edited or corrected using road-driven dynamics.

III. METHODOLOGY

  • RoVES uses edited road height as excitation for a half-car model and superimposes solved body displacement and pitch onto vehicle poses.The framework separates static road insertion from dynamic pose updates and requires no retraining.Source: §III · p. 3

A. 3DGS Scene Representation

  • After one-time scene reconstruction, Gaussian positions, covariances, colors, and opacities are frozen while per-frame object poses remain independently editable.Source: A. 3DGS Scene Representation · p. 3

B. Geometry–Pose Decoupled Editing Framework

C. Static Branch: Single-Image Driven Road Geometry Insertion

  • The static insertion branch uses a single reference image and text prompt, with texture extraction, monocular depth estimation, local point-cloud generation, and scale-corrected Gaussian merging.Source: C. Static Branch: Single-Image Driven… · p. 4

1) Object Texture Extraction:

RoVES extracts a road element from an image, lifts its texture into metric 3D geometry, and inserts it as Gaussians without retraining. Scale correction addresses the dense point cloud produced by monocular depth.

1) Object Texture Extraction:

  • Grounding DINO and SAM localize a prompted road structure and produce a foreground mask for cropping its texture.Removing the background reduces edge-color mixing during later geometric and visual fusion.Source: 1) Object Texture Extraction · p. 4

2) Depth Estimation and Local 3D Point Cloud Generation:

  • Depth Anything predicts dense depth for the texture and scene image, which is masked and scaled to preset real-world dimensions for local 3D point-cloud generation.The depth values are normalized over the foreground using their minimum and maximum; preset dimensions specify the target element’s physical length, width, and height.Source: 2) Depth Estimation and Local 3D Point… · p. 4

3) Scale Correction and Gaussian Merging for Monocular Depth–derived Point Clouds:

  • A global scale factor tightens Gaussian scales to limit overlap and color bleeding in dense depth-derived point clouds, after which the Gaussians are merged into the static background without retraining.The inserted primitives use texture RGB for the zero-order spherical-harmonic coefficient, set higher-order coefficients to zero, and use opacity 0.95.Source: 3) Scale Correction and Gaussian… · p. 4

4) Statistical Color Transfer in Lab Space:

RoVES aligns inserted road-element colors with the surrounding road using statistical transfer in Lab space, brightness adjustment, and linear blending.

4) Statistical Color Transfer in Lab Space:

D. Dynamic Branch: Physics-Aware Pose Correction via Vehicle Dynamics

RoVES couples edited road geometry to vehicle poses through a half-car dynamics model, correcting vertical displacement and pitch. The model’s state is solved and sampled at video-frame timestamps for pose updates.

D. Dynamic Branch: Physics-Aware Pose Correction via Vehicle Dynamics

  • Edited road geometry supplies excitation for vehicle dynamics, whose solved body pose is applied to the 3DGS rendering for physics-aware pose correction.The system also supports direct dynamic-vehicle pose edits such as deletion, translation, and rotation.Source: D. Dynamic Branch: Physics-Aware Pose… · p. 5

1) Standard Half-Car 4-DOF Dynamics Model:

  • The classic 4-DOF half-car model represents sprung-mass vertical displacement and pitch alongside front and rear unsprung-mass vertical displacements.Its differential equations are established from Newton’s second law and rigid-body rotation; the model is described as balancing accuracy and efficiency.Source: 1) Standard Half-Car 4-DOF Dynamics… · p. 5
  • RK4 solves the vehicle-state equations, and the resulting curves are sampled at video-frame timestamps and applied as pose corrections to the corresponding Gaussians.Source: 1) Standard Half-Car 4-DOF Dynamics… · p. 5

2) Road Height Field and Pose Correction:

RoVES converts edited road Gaussians into a height field, samples front and rear contact-point road heights, and uses them to correct vehicle vertical displacement and pitch.

2) Road Height Field and Pose Correction:

IV. EXPERIMENTS

The experiments evaluate road-structure insertion on Waymo scenes through editing comparisons, vehicle-response validation, rendering-realism assessment, and ablations. The setup combines several image, color, sharpness, and dynamics metrics with a user study.

A. Experimental Setup

  • Experiments use 10 Waymo scenes to insert speed humps and sunken roads, with runs conducted on an NVIDIA RTX 4090 GPU using PyTorch.Source: A. Experimental Setup · p. 5
  • Evaluation metrics include CLIPdir, CIEDE2000, two sharpness measures, pitch and vertical-displacement response errors, and 1–5 user ratings.CLIPdir, sharpness, and user-study scores are higher-is-better; CIEDE2000 and dynamic-response errors are lower-is-better.Source: A. Experimental Setup · p. 6

B. Comparative Experiments

  • Comparative experiments cover representative 3D editing methods, CarSim response consistency, and rendered-scene realism assessed by user study.Source: B. Comparative Experiments · p. 6

1) Reference-Level Comparison with 3D Editing Methods:

RoVES is compared with representative editing methods at reference level, then evaluated for runtime, vehicle-response agreement with CarSim, and rendering realism. Its half-car response closely tracks CarSim, while pose correction improves perceived plausibility.

1) Reference-Level Comparison with 3D Editing Methods:

  • Table I is a reference-level comparison, not a controlled benchmark, and several methods are omitted for unavailable metrics or different edit settings.The compared methods differ in datasets, edit types, hardware, prompts, and supported operations.Source: 1) Reference-Level Comparison with 3D… · p. 6
  • Road-element insertion takes 1.84 s and the full pipeline 6.24 s, completing road editing and vehicle-dynamics pose correction within seconds.Cross-paper runtimes are not directly comparable because hardware varies.Source: 1) Reference-Level Comparison with 3D… · p. 6, Table II · p. 6

2) Evaluation of Dynamic Response and Rendering Realism:

C. Ablation Experiments

The ablations examine Gaussian scale control and color-transfer strategies. The KNN adaptive scale factor improves sharpness over the original initialization, while the reported visual comparisons characterize the selected scale’s rendering effects.

C. Ablation Experiments

  • The ablation study targets KNN adaptive scale control and color transfer because both components affect rendering sharpness or visual consistency.Source: C. Ablation Experiments · p. 7

1) Ablation Experiment on KNN Adaptive Scale Control Scaling Factor:

  • Across all three speed-hump instances, σ = 0.01 scores highest on Laplacian variance and Tenengrad gradient, outperforming the original σ = 1.0 initialization.The authors attribute the improvement to alleviating scale mismatch from dense monocular point-cloud insertion.Source: 1) Ablation Experiment on KNN Adaptive… · p. 7, Table V · p. 7
  • With σ = 1.0 edges blur and texture details are lost; σ = 0.5 retains slight blur, while σ = 0.01 and 0.1 appear similarly clear.The authors choose σ = 0.01 as default based on numerical improvement and parameter robustness.Source: 1) Ablation Experiment on KNN Adaptive… · p. 7, Figure 7 · p. 7

2) Ablation Experiment on Color Transfer:

The color-transfer ablation compares reference sources and color spaces, with Img-Lab reported as the best balance across three metrics and as preserving texture in the rendered insertion. The paper concludes with a second-level editing pipeline and notes limits in vehicle coverage and large-pose rendering.

2) Ablation Experiment on Color Transfer:

  • Img-Lab obtains the best balance across all three metrics among the four strategies, while PC-RGB performs worst overall.PC-Lab receives higher user ratings but has larger color difference; Img-RGB has the lowest CIEDE2000 but weaker semantic alignment.Source: 2) Ablation Experiment on Color Transfer · pp. 7–8
  • Img-Lab naturally blends the inserted object with the road while preserving local contrast and texture; the alternatives show color discrepancy or lose texture detail.Source: 2) Ablation Experiment on Color Transfer · p. 8

V. CONCLUSION

  • RoVES decouples static Gaussian insertion from dynamic pose updates and completes the full pipeline in 6.24 s while producing dynamics-consistent vertical displacement and pitch responses.Source: §V · p. 8

VI. LIMITATIONS AND FUTURE WORK

  • The physics-aware module models only vertical displacement and pitch and requires manually specified vehicle parameters, limiting direct use across vehicle types without per-vehicle tuning.Large pose deviations can also move viewpoints outside well-reconstructed 3DGS regions and cause rendering artifacts.Source: §VI · p. 8
Loading 2605.25373v1…