PaperlayerView exact revision on arXiv

Source-linked AI summary

As-Rigid-As-Possible Deformation of Gaussian Radiance Fields

Xinhao Tong, Tianjia Shao, Yanlin Weng, Yin Yang, Kun Zhou

arXiv:2608.29538v1cs.GR

TL;DR

Deforming 3DGS geometrically can make the Gaussians inconsistent with the radiance field that renders the image, causing artifacts. The paper introduces interactive ARAP deformation with radiance-field-aware Gaussian optimization, radial features, and adaptive filtering, and reports higher rendering quality for complex deformations while retaining 3DGS efficiency. Its scope is limited by unmodeled changes in shadows and specular reflections, as well as sampling constraints.

  • Problem

    Geometric deformation of 3DGS can mismatch the deformed radiance field and produce artifacts, particularly under large or localized deformations.

  • Method

    The method performs geometric Gaussian deformation, then optimizes Gaussians using radial radiance-field features and an adaptive anisotropic spatial low-pass filter.

  • Results

    The method reports higher rendering quality for complex, nonlinear, and large deformations while maintaining 3DGS rendering quality and efficiency.

  • Takeaways & Limitations

    Users can interactively perform large-scale ARAP deformation of Gaussian radiance fields without explicit surface extraction.

  • Takeaways & Limitations

    The method does not properly handle deformation-related changes in shadows and specular reflections, and sampling is constrained by memory capacity and processing speed.

Abstract

from arXiv · show

3D Gaussian Splatting (3DGS) models radiance fields as sparsely distributed 3D Gaussians, providing a compelling solution to novel view synthesis at high resolutions and real-time frame rates. However, deforming objects represented by 3D Gaussians remains a challenging task. Existing methods deform a 3DGS object by editing Gaussians geometrically. These approaches ignore the fact that it is the radiance field that rasterizes and renders the final image. The inconsistency between the deformed 3D Gaussians and the desired radiance field inevitably leads to artifacts in the final results. In this paper, we propose an interactive method for as-rigid-as-possible (ARAP) deformation of the Gaussian radiance fields. Specifically, after performing geometric edits on the Gaussians, we further optimize Gaussians to ensure its rasterization yields a similar result as the deformed radiance field. To facilitate this objective, we design radial features to mathematically describe the radial difference before and after the deformation, which are densely sampled across the radiance field. Additionally, we propose an adaptive anisotropic spatial low-pass filter to prevent aliasing issues during sampling and to preserve the field with the varying non-uniform sampling intervals. Users can interactively employ this tool to achieve large-scale ARAP deformations of the radiance field. Since our method maintains the consistency of the Gaussian radiance field before and after deformation, it avoids artifacts that are common in existing 3DGS deformation frameworks. Meanwhile, our method keeps the high quality and efficiency of 3DGS in rendering.

1 INTRODUCTION

3DGS enables efficient, high-quality rendering, but geometric Gaussian edits can diverge from the radiance field and produce artifacts. The proposed interactive method combines geometric deformation with radiance-field-aware optimization, radial features, and adaptive filtering.

  • 3DGS combines explicit Gaussian representations with differentiable rasterization for efficient training and rendering while capturing high-frequency details.
  • Geometric edits can mismatch the spatially varying deformation of the radiance field, especially under nonlinear, localized, or large deformations.The mismatch is particularly visible at model boundaries and can reduce image-space rendering quality.
  • The method first applies embedded deformation to Gaussians, then optimizes them against the radiance field they represent.This two-stage design targets consistency between per-Gaussian edits and field-wise deformation.
  • Radial features describe differences across the deformed radiance field, while an adaptive anisotropic spatial low-pass filter addresses changing non-uniform sampling intervals.The filter is designed to prevent aliasing during deformation.
  • The approach supports interactive, large-scale ARAP deformation without surface extraction and maintains 3DGS rendering quality and efficiency.The authors report higher rendering quality for complex, nonlinear, and large deformations.

2 RELATED WORK

Prior work deforms explicit, implicit, and Gaussian-based representations through geometric structures or control mechanisms, but often leaves a mismatch between deformed Gaussians and the radiance field. The proposed pipeline addresses this mismatch through a two-stage optimization process.

  • 2.1 Radiance Fields: Radiance-field research has pursued 3D reconstruction and novel view synthesis, with NeRF representing color and opacity through an MLP and volume rendering.
  • 2.1 Radiance Fields: 3DGS uses explicit 3D Gaussian kernels, spherical harmonics for anisotropic color, and differentiable rasterization to provide high-quality, efficient rendering.
  • 2.2 3D Shape Deformation: Embedded Deformation supports ARAP editing across geometric representations, while existing implicit methods use cages, ray bending, or extracted meshes.
  • 2.2 3D Shape Deformation: The proposed pipeline geometrically deforms Gaussians first, then optimizes their configuration to reduce discrepancy with the deformed radiance field.
  • 2.2 3D Shape Deformation: Some prior shape-deformation approaches exhibit suboptimal rendering performance when supporting interactive deformations.
  • 2.2 3D Shape Deformation: Prior Gaussian editing methods bind Gaussians to meshes, triangle patches, control points, frosting layers, or physical particle clouds.These approaches modify Gaussian positions, rotations, scales, or shapes through their associated structures.
  • 2.2 3D Shape Deformation: Existing Gaussian deformation methods do not account for the mismatch between deformed Gaussians and the deformed radiance field, producing artifacts under large deformations.

3 METHODOLOGY

The method first applies embedded-deformation-based geometric edits, then optimizes Gaussian configurations so rasterization remains consistent with the deformed radiance field. It uses radial-feature errors, adaptive anisotropic filtering, and boundary-aware deformation to support large-scale ARAP edits while reducing artifacts.

  • Stage I: Geometrical Deformation of Gaussians: Embedded deformation uses affine transformations on a deformation graph to apply spatially varying ARAP edits to Gaussian-based geometry.Each control node carries a 3 × 3 matrix and translation vector; Gaussian displacements inherit influences from neighboring nodes.
  • Stage I: Geometrical Deformation of Gaussians: The geometric objective combines rotation, regularization, and user-constraint energies to keep deformation as rigid as possible.Erot measures deviation from an ideal rigid rotation, Ereg penalizes abrupt translational deformation, and Econ measures constraint satisfaction.
  • Stage I: Geometrical Deformation of Gaussians: Six axis-aligned endpoints placed two standard deviations from each Gaussian center capture per-Gaussian deformation for estimating local transformations.The endpoints are used to estimate the best-fitting local deformation of each Gaussian.
  • Stage II: Radiance-Field Consistency: Directly rasterized deformed Gaussians can produce artifacts because their rendering may not align with the radiance field warped by the deformation graph.The second stage therefore finetunes each Gaussian configuration to reduce this field-wise discrepancy.
  • Stage II: Radiance-Field Consistency: The method represents the radiance field with radial features F and O, encoding rotated-harmonics color and transparency at arbitrary positions.Desired post-deformation features are compared with features produced by the deformed Gaussians; the integral of their differences measures field discrepancy and is approximated by samples.
  • Adaptive Anisotropic Spatial Low-Pass Filter: An adaptive anisotropic low-pass filter estimates post-deformation sampling intervals from Ak/ndim to address aliasing under changing non-uniform sampling.The filter bandwidth parameter λlpf is empirically set to 0.2; overly conservative filtering can undersample, whereas overly aggressive filtering can blur results.

4 EXPERIMENTS & EVALUATIONS

The method was validated across diverse 3DGS-based models using visual, quantitative, and ablation experiments.

  • Experiments used C++ and CUDA on a single NVIDIA RTX 4090 24GB GPU.
  • The method outperformed the baseline and existing methods in visual quality and quantitative metrics.
  • Ablation studies examined several core design elements and their individual contributions.

4.1 Datasets & Metrics

The evaluation combines real-world and synthetic datasets with multi-view rendering metrics to assess deformation quality across viewpoints.

  • Datasets: Experiments used two real-world datasets and six synthetic datasets from NeRF-Synthetic, Sketchfab, and an internally created model.Camera poses came from Blender for synthetic data and COLMAP for real-world data.
  • Metrics: Synthetic evaluations compared Gaussian renderings against deformed-mesh ground truth using PSNR, SSIM, and LPIPS.The comparison used 200 novel views per dataset.
  • View Sampling: The evaluation included 200 novel views sampled along a 360-degree circular camera trajectory.This trajectory observes both deformation regions and the overall object.
  • Comparative Evaluation: Fig. 6 compares deformed results from existing methods and ours under the same deformation.The figure assesses preservation of Gaussian radiance fields after deformation.

4.2 Comparisons & Evaluations

Across baseline, 3DGS, flat-Gaussian, and efficiency comparisons, the method improves rendered deformation quality while supporting interactive optimization without reconstruction-pipeline restrictions.

  • Comparisons with the Baseline: The method mitigated spikes and blurring relative to geometric-only deformation and exceeded the baseline across synthetic-dataset rendering metrics.
  • Comparisons with Existing Methods: The method outperformed GaussianMesh and Gaussian Frosting qualitatively and quantitatively under identical inputs and spatial deformations.Its rendered results were closer to deformed-mesh ground truth and used fewer Gaussians.
  • Comparisons with Existing Methods: The method produced better rendered results than GaMeS on flat Gaussian radiance fields under the same deformation.Table 3 also reports quantitative superiority over GaMeS.
  • Comparisons with Existing Methods: Unlike the compared methods, the approach accepts any reconstructed Gaussians without restricting the reconstruction process or Gaussian type.
  • Efficiency: Optimization converges within 100 iterations, with optimization time defined as the total duration of those iterations.
  • Efficiency: Users generally wait within two seconds for optimization after deformation, supporting smooth interactive deformation interactions.
  • Efficiency: Precomputation typically requires 1 to 3 seconds for most datasets.This one-time stage constructs the graph, samples the radiance field, and determines nearest control points.

4.3 Ablation Study

Ablations show that shape, boundary, and adaptive low-pass components stabilize optimization, preserve object boundaries, and prevent undersampling artifacts.

  • Shape Energy: Removing shape energy can destabilize optimization because feature energy alone insufficiently preserves opacity and spherical-harmonic colors.
  • Shape Energy: Adding shape energy constrains gray Gaussians with near-zero spherical-harmonic values and mitigates uncontrolled size artifacts.
  • Boundary Energy: The method preserves object boundaries better with boundary energy than without it.
  • Boundary Energy: Boundary energy addresses large Gaussians extending beyond object boundaries when feature and opacity energies are insufficient.
  • Spatial Low-Pass Filter: Without spatial low-pass filtering, undersampling can create holes and produce results worse than the baseline in some regions.The designed filter resolves this issue.
  • Spatial Low-Pass Filter: An adaptive anisotropic low-pass filter compensates for changing spatial sampling intervals during deformation.

4.4 Selection of Parameters and Optimization Steps

The method examines how KNN neighborhood size affects deformation smoothness and evaluates optimization stability across iterations. Larger neighborhoods smooth regular geometries, while 100 iterations produce stable novel-view rendering results.

  • KNN parameter selection: Larger k-values in KNN produce smoother deformations for some regular Gaussian radiance-field geometries.This agrees with prior experimental results on meshes using Embedded Deformation.
  • Optimization steps: Optimization converges to a good result after 100 iterations across the six synthetic datasets.The study compares unoptimized results with results after 100 and 1000 iterations against ground truth.
  • Filter selection: The adaptive anisotropic low-pass filter better preserves high-frequency radiance-field signals when deformation changes sampling intervals.A fixed isotropic filter can instead introduce blurring under non-uniform spatial sampling.
  • KNN parameter selection: The KNN experiments compare different k-values using the same Gaussian object and identical control-point position edits.The comparison isolates the effect of neighborhood size on deformation smoothness.
  • Optimization steps: The designed energy terms and optimization approach produce stable novel-view rendering results.The convergence analysis reports average quantitative results across all six synthetic datasets.

4.5 Implementation Details

Implementation uses active-cell updates and fixed energy weights, with specified optimization settings and convergence comparisons across six synthetic datasets. The NeRF Synthetic objects are converted to meshes and re-rendered for ground-truth experiments.

  • Optimization scope: Only active grid cells and moving Gaussians are updated, limiting computation to regions affected by local deformation.This prevents the optimization region from spreading across the entire domain.
  • Energy settings: The experiments use λ1 = 1 and λ2 = 99, with equal weights for feature and opacity preservation and a larger boundary-energy weight.The boundary term is emphasized because it operates in empty regions.
  • Evaluation setup: Table 5 reports average results across six synthetic datasets for no optimization, 100 iterations, and 1000 iterations.These rows are used to assess optimization convergence and stability for 3DGS representations.
  • Filter and KNN settings: The adaptive anisotropic low-pass filter uses λ_lpf = 0.2, and KNN uses k = 8 for the experimental datasets except the stripes dataset.The filter parameter matches the spatial low-pass-filter parameter used in Mip-Splatting.
  • Learning rates: Gaussian optimization uses separate learning rates for rotation, scaling, opacity, spherical harmonics, and position.Position learning rates decay exponentially from 3.2 × 10^-4 to 6.4 × 10^-6.
  • Acceleration: Opacity is truncated below representation-specific thresholds, and Gaussian-to-sample coverage is updated every 10 steps.The thresholds are 1/255 for 3DGS objects and 1 × 10^-3 for ship ropes and GaMeS flat Gaussians.
  • Evaluation setup: NeRF Synthetic objects are converted to meshes and re-rendered with Blender because their Bézier curves complicate deformation-based ground-truth creation.Both training and testing sets are re-rendered for the experiments.

4.6 More Deformation Results

Additional novel-view synthesis results demonstrate the method on bending, twisting, and scaling deformations, including experiments on the real-world flower dataset.

  • Deformation types: The additional novel-view synthesis experiments cover bending, twisting, and scaling deformations.The flower dataset used in these results is a real-world dataset.

5 CONCLUSION & FUTURE WORK

The paper presents an interactive ARAP deformation method for Gaussian radiance fields that uses spatial features, energy preservation, and adaptive filtering. It reports broad deformation results while identifying limitations involving appearance changes, sampling capacity, extreme deformation, and overlap.

  • Conclusion: The conclusion presents a method for ARAP deformation of Gaussian radiance fields.The method is designed for deformation directly over the Gaussian radiance field.
  • Results: Additional results cover twisting, scaling, and bending deformations in novel-view synthesis.The flower dataset among these results is real-world.
  • Contributions: Spatial features describe the Gaussian radiance field, and Gaussians are optimized through an energy-preservation formulation.The approach also studies spatial sampling during deformation.
  • Contributions: An adaptive anisotropic spatial low-pass filter is proposed to better maintain the radiance field during spatial sampling changes.The method is presented as part of an interactive deformation system.
  • Limitations: The method fails to properly handle shadows and specular reflections because deformation-time color variations from occlusions, anisotropic materials, and lighting changes are not modeled.The radiance field encapsulates intrinsic color, material, environmental lighting, and occlusion together.
  • Limitations: The framework is bounded by sampling capacity and may fail under extreme deformations or overlapping sampling regions.Memory capacity and processing speed limit the number of sample points and representable detail frequency.
  • Reproducibility: The implementation is provided for further research and reproducibility.The paper gives a public repository for the method.
Loading 2608.29538v1…