PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocation

Rui Wang, Quentin Lohmeyer, Siyu Tang, Mirko Meboldt

arXiv:2606.22197v1cs.CV

TL;DR

Dynamic Gaussian splatting must balance motion consistency with fine visual detail, which existing deformation-based and 4D-primitive approaches represent imperfectly. Multi4D allocates capacity across static structure, persistent dynamic geometry, and transient appearance primitives. The authors report high-fidelity rendering with fewer dynamic primitives and state-of-the-art 4D segmentation with faster inference.

  • Problem

    Dynamic Gaussian reconstruction must balance motion consistency with fine visual detail, while deformation-based and 4D-primitive approaches have distinct representational drawbacks.

    Source: §1 · pp. 2–3

  • Method

    Multi4D competitively allocates modeling capacity among static structure, persistent dynamic geometry, and transient appearance primitives under a unified renderer.

    Source: §1 · p. 3

  • Results

    Multi4D reports coherent motion and fine-grained detail with fewer primitives and faster rendering, alongside state-of-the-art 4D segmentation with significantly faster inference.

    Source: §1 · pp. 3–4

  • Takeaways & Limitations

    The compact persistent representation supports efficient, highly accurate downstream 4D segmentation while preserving coherent motion and high-frequency appearance variation.

    Source: §6 · p. 14

    Multi4D does not currently incorporate explicit attribute compression, limiting further improvements in storage efficiency from its structural compactness.

    Source: §6 · p. 14

Abstract

from arXiv · show

Dynamic 3D Gaussian splatting faces a fundamental tension between motion consistency and visual fidelity. Deformation-based approaches preserve temporal correspondence but suffer from motion over-factorization, oversmoothing high-frequency dynamics. In contrast, 4D-primitive methods capture fine visual details yet incur temporal overparameterization, breaking object identity and leading to severe storage overhead. To resolve this, we introduce Multi4D, a framework for high-fidelity dynamic Gaussian Splatting based on multi-level competitive allocation. Instead of a monolithic representation, we distribute modeling capacity across three structured levels: static structure, persistent dynamic geometry, and transient appearance primitives. Through shared rasterization and residual-driven optimization, these levels dynamically compete to explain photometric error, enabling adaptive specialization without pre-assigned decomposition. This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving state-of-the-art rendering quality and real-time performance with significantly fewer dynamic primitives. Furthermore, because our representation explicitly tracks compact persistent Gaussians over time, semantic features can be embedded afterward, enabling Multi4D to achieve state-of-the-art 4D segmentation accuracy with an order-of-magnitude speedup. Project page: https://batfacewayne.github.io/Multi4D.io/

1 Introduction

Multi4D addresses the tension between motion consistency and visual fidelity by distributing dynamic-scene modeling across static structure, persistent dynamic geometry, and transient appearance. Its shared, residual-driven optimization specializes these levels while supporting high-fidelity reconstruction and compact, fast 4D segmentation.

1 Introduction

  • Deformation-based methods preserve Gaussian identity but can over-factorize motion, oversmooth high-frequency dynamics, mistake appearance changes for motion, and incur per-primitive deformation-query overhead.Increasing primitive count to capture fine detail compromises real-time rendering and memory efficiency.Source: §1 · pp. 1–2
  • 4D-primitive methods capture transient appearance and fine detail, but temporal over-parameterization can represent physical motion as short-lived primitives, breaking geometry in fast-motion regions.Prior pruning approaches focus mainly on static regions and remain limited in simplifying dynamic regions; these methods also generalize less well to sparse-camera and monocular settings than deformation-based approaches.Source: §1 · pp. 2–3
  • Multi4D jointly optimizes static Gaussians for structural backbone, persistent deformable Gaussians for long-term identity, and transient Gaussians for high-frequency appearance residuals.The framework separates static structure, persistent dynamic geometry, and transient appearance to reconcile motion consistency with photometric fidelity.Source: §1 · p. 3
  • Multi4D reports state-of-the-art dynamic novel-view synthesis with higher fidelity, fewer dynamic primitives, improved runtime, and state-of-the-art 4D segmentation with an order-of-magnitude speedup.For segmentation, optimization uses only the persistent subset, Gs ∪ Gd, to preserve temporal identity while discarding transient appearance noise.Source: §1 · pp. 3–4
  • Shared rasterization couples subset gradients, allowing stable geometry to suppress redundant modeling while transient primitives explain irreducible appearance variation without deforming geometry to fit photometric noise.Residual-driven allocation across levels occurs under a unified differentiable renderer.Source: §1 · p. 3
  • Multi4D introduces bottom-up, self-regularized training with velocity-aware lifting and mask-aware pruning to enable structured specialization and compact representations.The introduction also states that the framework improves generalization under sparse or monocular supervision.Source: §1 · pp. 3–4

2 Related Work

Related work spans deformation-based and 4D-primitive dynamic Gaussian methods, while dynamic semantic extensions inherit deformation costs. Their limitations motivate persistent geometry, sparse parameterization, and high-fidelity rendering.

2 Related Work

  • Deformation-based methods track canonical Gaussians but incur per-frame query costs; 4D-primitive methods render high fidelity [47] but risk short-lived primitives and weak persistent tracking.Extensions improve deformation granularity and 4D-primitive compactness, but redundancy remains problematic in highly dynamic scenes, especially under sparse or monocular supervision.Source: §2 · p. 4
  • Dynamic semantic Gaussian methods build on deformation frameworks [10] [21], requiring high-dimensional features to be evaluated and rendered for every primitive at each frame.Semantic embedding combines Gaussian representations with foundation models.Source: §2 · p. 4
  • These limitations motivate representations that preserve persistent geometry, maintain sparse parameterization, and enable high-fidelity dynamic rendering.Source: §2 · p. 4

3 Method

Multi4D allocates scene modeling across static, persistent dynamic, and transient Gaussian subsets that specialize through shared rendering and optimization. Its method adds adaptive subset formation, pruning, and a persistent representation for downstream 4D segmentation.

3 Method

  • Multi4D assigns stable structure to static Gaussians, identity-preserving rigid motion to persistent Gaussians, and high-frequency appearance or transient geometry to 4D primitives.The persistent subset uses a geometric-only deformation field, while transient primitives model time-varying appearance and geometry.Source: §3 · pp. 4–5

3.2 Initialization and Adaptive Rescaling

  • Initialization biases specialization by densely seeding static Gaussians from COLMAP, sparsely seeding persistent dynamics, and leaving transient primitives empty until periodic lifting.Spatial and temporal domains are normalized using camera-distribution and video-duration scaling factors to stabilize optimization.Source: §3.2 · p. 6

3.3 Unified Multi4D Rendering

  • A unified differentiable rasterizer jointly sorts and renders all subsets, coupling their gradients through shared transmittance to induce competition and suppress redundant densification.It extracts full, persistent, and transient renders in one pass, with an SSIM-based diversity loss encouraging specialization.Source: §3.3 · pp. 6–7

3.4 Self-Supervised Dynamic-Static Decomposition

  • A learned time-dependent mask separates dynamic actors from static content without tracking labels, while a spatially aware opacity penalty suppresses persistent Gaussians in static regions.Dense static and sparse dynamic initialization biases the mask toward regions with temporal variation.Source: §3.4 · p. 7

3.5 Velocity-Aware Periodical Lifting

  • After separation stabilizes, active persistent Gaussians are periodically lifted into the transient subset using inherited motion to initialize their 4D trajectories.Finite-difference velocity aligns the new primitive’s principal axis with its spatiotemporal trajectory, while a camera-facing offset prevents immediate occlusion.Source: §3.5 · p. 8

3.6 Mask-Aware Utility-Based Pruning

  • Mask-aware pruning scores visible rendering contributions within subset-specific regions, accounts for cross-set occlusion, and removes low-utility primitives; stochastic dropout discourages view-dependent overfitting.The score balances preserving occasional high contributions against removing consistently unused primitives.Source: §3.6 · pp. 8–9

3.7 Training Strategy and Objectives

  • Two-stage training first forms specialized subsets with separation and lifting, then disables both mechanisms and refines rendering while pruning remains active.The objective combines color supervision, separation, regularization, and diversity losses.Source: §3.7 · p. 9

3.8 Downstream Application: Efficient 4D Segmentation

  • For 4D segmentation, Multi4D freezes persistent geometry after appearance convergence, adds learnable semantic features, and supervises rendered maps against SAM masks with a contrastive objective.KNN feature smoothing in canonical 3D space encourages spatial coherence within actors.Source: §3.8 · p. 9

4 Experiments

Multi4D is evaluated on multi-view and monocular novel-view synthesis and 4D segmentation. The reported results include improved rendering quality and speed, stronger sparse-view reconstruction, and state-of-the-art segmentation accuracy.

4.1 Experimental Setup

  • The evaluation covers multi-view Technicolor and Neu3D, monocular NeRF-DS, and Neu3D-Mask segmentation, reporting PSNR, DSSIM, FPS, mIoU, and mAcc.Experiments ran on a single NVIDIA RTX 4090 GPU.Source: §4 · p. 9, §4.1 · p. 10

4.2 Experimental Results

  • On Technicolor, Multi4D improves PSNR by 0.95 dB over the strongest Gaussian baseline [47] while rendering at 161 FPS.Qualitative comparisons show stronger modeling of high-frequency dynamic details.Source: Figure 3 · p. 10, §4.2 · p. 11, Table 1 · p. 11
  • On Neu3D, Multi4D achieves state-of-the-art reconstruction quality at 217 FPS.It better preserves high-frequency dynamic details, mitigating temporal blurring in deformation-based methods and fragmented geometry in 4D-primitive approaches.Source: §4.2 · p. 11, Table 2 · p. 11, Figure 4 · p. 12
  • On sparse-view NeRF-DS, Multi4D reconstructs more strongly than 4D-primitive frameworks [47], using persistent Gaussians Gd for coherent motion and transient Gaussians Gt for localized specular highlights.The compared frameworks often degrade and produce floating artifacts because they lack a holistic motion prior.Source: §4.2 · p. 11
  • On Neu3D-Mask, Multi4D reaches 0.9142 mIoU; semantic optimization on persistent Gp preserves temporal identity with 13k dynamic Gaussians versus TRASE’s 624k.It renders 32-dimensional features at 204 FPS, nearly 10× faster than the 21 FPS baseline.Source: §4.2 · pp. 11–12

5 Ablation Study & Efficiency Analysis

Ablations show that persistent motion, transient appearance, and the regularization mechanisms play complementary roles in Multi4D. The final model combines higher PSNR with fewer dynamic Gaussians, smaller storage, and faster training than 4DGS.

Role of Dynamic Subsets Gd, Gt

  • Removing Gd yields 32.78 PSNR, while removing Gt yields 32.86 PSNR with 25k dynamic Gaussians, showing their respective roles in coherent motion and residual appearance modeling.Without stable structural initialization, 4D modeling becomes difficult, producing fragmented geometry and excessive primitive growth.Source: §5 · pp. 12–13

Velocity-Aware Periodical Lifting

  • Random initialization instead of velocity-aware lifting reduces PSNR by 0.70 dB (33.92 → 33.22), as transient primitives lack inherited motion priors and under-model high-frequency dynamics.Source: §5 · p. 13

Diversity Loss

  • Removing Ldiversity increases dynamic primitives by 67% (276k vs. 165k), enlarges storage, and reduces fidelity through redundant modeling across persistent and transient sets.Source: §5 · p. 13

Mask-Aware Pruning

  • Without mask-aware utility-based pruning, dynamic Gaussians grow to 729k and storage increases by 145%, while unregularized Gaussians degrade rendering quality.Source: §5 · p. 14

Efficiency and Compactness

  • 25× fewer dynamic Gaussians than 4DGS [47] (165k vs. 4.2M) accompanies 33.92 PSNR, 214.7 MB vs. 2.6 GB, and 1.2 vs. 5.5 training hours (4.6× faster).Training times were measured on a single RTX 4090.Source: §5 · p. 14

6 Discussion and Conclusion · A Supplementary Material · A.1 Overview of the Multi4D Pipeline

Multi4D uses competitive allocation across static, persistent dynamic, and transient Gaussian subsets to balance motion consistency with photometric fidelity. Its persistent subset supports efficient, accurate 4D segmentation, though explicit attribute compression remains future work.

6 Discussion and Conclusion

  • Multi4D resolves the tension between motion consistency and photometric fidelity through a multi-level framework for dynamic 3D Gaussian Splatting.It uses a structured representation rather than a monolithic one.Source: §6 · p. 14
  • Under a unified objective, static, persistent, and transient Gaussian subsets competitively allocate capacity for residual-driven specialization.This allocation suppresses motion over-factorization and temporal over-parameterization.Source: §6 · p. 14
  • The structured decomposition preserves coherent motion while capturing high-frequency appearance variation in compact, high-fidelity reconstructions.Source: §6 · p. 14
  • Semantic reasoning on the persistent Gaussian subset enables efficient and highly accurate downstream 4D segmentation.Source: §6 · p. 14
  • Multi4D reduces dynamic primitives through optimization-driven compactness but does not yet incorporate explicit attribute compression.Future work could explore post-training deformation distillation, Gaussian quantization, or lightweight deformation parameterizations to improve storage efficiency.Source: §6 · p. 14

A.1 Overview of the Multi4D Pipeline

  • The pipeline jointly optimizes static, persistent dynamic, and transient Gaussian subsets through a shared renderer and bottom-up, self-regularized multi-level allocation.The subsets retain independent optimization and density-control dynamics, enabling competitive allocation and emergent specialization.Source: Figure 7 · p. 19

B Analysis on Subset Specialization · C Detailed Implementation and Methodology · C.1 Independent Optimization and Densification per Subset

Multi4D’s competitive allocation specializes static, persistent-dynamic, and transient subsets to preserve coherent geometry while capturing fine dynamics. Independent optimization and densification policies support this specialization and compact representations.

B Analysis on Subset Specialization

  • Subset visualizations across Neu3D, Technicolor, and NeRF-DS show ground truth, full renders, individual subset renders, and learned dynamic–static separation masks.The figure presents these visualizations as an empirical analysis of subset specialization across diverse dynamic view synthesis benchmarks.Source: B Analysis on Subset Specialization · p. 19, Figure 8 · p. 20
  • Static Gaussians model time-invariant structure, persistent Gaussians maintain coherent motion and long-term geometric identity, and transient primitives capture high-frequency appearance changes unexplained by deformation.Examples of transient changes include fire, smoke, and specular highlights.Source: Figure 8 · p. 20, B Analysis on Subset Specialization · p. 21
  • Shared-renderer optimization makes subsets compete to explain photometric residuals under their own inductive biases, suppressing redundant modeling once a subset explains a region.Subsets are initialized inversely to representational capacity and progressively optimized with cross-level self-regularization.Source: B Analysis on Subset Specialization · p. 21
  • Multi4D avoids motion over-factorization and temporal over-parameterization by distributing capacity across complementary regimes, preserving coherent geometry while capturing high-frequency dynamics in compact representations.This structured specialization contributes to the improved rendering fidelity, reduced primitive counts, and higher runtime efficiency reported in the main experiments.Source: Figure 8 · p. 20, B Analysis on Subset Specialization · p. 21

C.1 Independent Optimization and Densification per Subset

  • Unlike global optimization and densification, Multi4D gives each subset its own Adam optimizer, learning-rate schedule, and densification–pruning policy.This lets each subset grow and adapt according to its representational role while cross-subset self-regularization supports fine-detail specialization.Source: §C.1 · p. 21
  • Static Gaussians update canonical parameters with Phase-I screen-space-gradient densification and relaxed pruning; persistent dynamics update Gaussians and HexPlane deformation, densifying from deformed gradients but cloning or splitting canonical Gaussians.Applying clone and split operations to canonical Gaussians provides additional regularization and improves training efficiency, but makes fine details more challenging to model.Source: §C.1 · pp. 21–22
  • Transient primitives use spatial and temporal gradients for densification and progressive opacity pruning to stay sparse, emerging where persistent models cannot explain observed appearance changes.Their high expressivity is tightly controlled to limit excessive primitive growth and maintain a compact representation.Source: §C.1 · p. 22

C.2 Training Schedule and Optimization Stages … D Downstream 4D segmentation

Multi4D trains its static, persistent-dynamic, and transient subsets through staged specialization, self-supervised dynamic–static separation, and unified rendering refinement. Its objective combines photometric supervision with separation, diversity, and geometric regularization, with implementation details specifying the training schedule and optimization settings.

C.2 Training Schedule and Optimization Stages

  • Freezing deformation for the first 2,000 iterations stabilizes canonical geometry before motion modeling; after subset specialization, training disables explicit separation and refines the unified renderer.The late-stage objective retains photometric loss, depth-ordering regularization, persistent-render supervision, and persistent-path depth smoothness.Source: §C.2 · p. 22
  • Gaussians are cloned or split every 100 iterations, while subset-specific utility pruning and contribution-based transient pruning limit primitive growth.Opacities are partially reset periodically to stabilize foreground–background decomposition and promote compact representations.Source: §C.2 · p. 22

C.3 Self-Supervised Dynamic–Static Decomposition Details

  • Self-supervised mask-guided decomposition separates persistent dynamics from static background, using dynamic-Gaussian mask logits and a time-dependent MLP offset to render a dynamic probability map.Dynamic Gaussians start from sparse random initialization while the static subset is densely initialized from COLMAP points; the dynamic mask contracts around moving actors during training. Fig. 9 depicts the decomposition and opacity regularization.Source: §C.3 · pp. 23–24, Figure 9 · p. 23
  • The separation loss Lsep enforces dynamic–static decomposition by aggregating compositing and regional supervision, with γ = 0.9 during the first 2,000 iterations and γ = 1 afterwards.This slightly attenuates the target signal in early training to prevent the high-capacity foreground model from overfitting background regions before the static geometry stabilizes.Source: §C.3 · pp. 23–24

C.4 Exact Loss Formulations

  • The total objective combines color, separation, cross-subset diversity, and geometric regularization losses; diversity penalizes transient–persistent similarity and also weakly encourages dynamic–static separation.Geometric constraints include mask-aware dynamic opacity, one-sided depth ordering, scale and aspect-ratio limits, edge-aware depth smoothness, and temporal smoothness on persistent deformation grids.Source: §C.4 · pp. 24–25

C.5 Implementation Details

  • Experiments use PyTorch on one RTX 4090 for 20,000 iterations, with deformation frozen for 2,000 iterations and unified-rendering refinement beginning at 10,000.Velocity-aware lifting runs every 50 iterations from 6,000 to 10,000; for monocular datasets, transient densification is disabled and transient primitives are introduced only through lifting.Source: §C.5 · p. 26
  • Adam optimizes all parameters, using exponentially decayed learning rates for Gaussian positions, the deformation MLP, and HexPlane grids, and fixed rates for appearance and Gaussian attributes.The deformation representation uses a [64, 64, 64, 150] HexPlane grid and a one-hidden-layer MLP of width 128; the release configuration also specifies regularization weights.Source: §C.5 · p. 26

D.1 4D Semantic Tracking … E.1 Compactness and Persistent Motion

Multi4D learns semantics on persistent Gaussians while discarding transient primitives, enabling efficient semantic tracking and temporally consistent segmentation. The analysis links excessive transient parameterization to worse storage, rendering quality, and persistent motion.

D.1 4D Semantic Tracking

  • After reconstruction converges, Multi4D freezes geometry and deformation, then optimizes randomly initialized 32D semantic features on persistent Gaussians using mask-driven contrastive learning.The method follows TRASE [21] and discards transient primitives, which the passage says break temporal identity.Source: §D.1 · pp. 26–27
  • Approximately 48× fewer primitives and nearly 10× faster semantic training and inference result from learning only on the compact persistent subset.The comparison is to monolithic 4DGS representations; the persistent-only semantic optimization preserves temporal identity.Source: §D.1 · p. 27

D.2 4D Semantic Clustering

  • DBSCAN clusters a random 2% sample of normalized persistent-Gaussian features, and cosine similarity assigns all persistent Gaussians to cluster labels.Normalized cluster centroids are computed from the sampled features.Source: §D.2 · pp. 27–28
  • Propagating cluster labels through the deformation field preserves object identities over time and enables rendering objects at arbitrary timestamps.Objects are rendered by isolating their corresponding persistent primitives.Source: §D.2 · p. 28

D.3 Open-Vocabulary 4D Segmentation

  • A text prompt is grounded in the first frame with Grounding DINO [24] and SAM, then matched to deformed persistent Gaussians to select a semantic cluster.The method unprojects masked pixels using rendered depth and identifies the cluster by majority voting.Source: §D.3 · p. 28
  • Selected primitives render temporally consistent object masks and RGB across timestamps without per-frame tracking or optical flow.The deformation field propagates the selected primitives through time.Source: §D.3 · p. 28

E.1 Compactness and Persistent Motion

  • Excessive transient parameterization disrupts decomposition, increases storage, lowers PSNR, and degrades persistent motion.Multi4D treats compactness as part of decomposition quality; its stable persistent subset supports downstream semantics without transient-induced inconsistency.Source: §E.1 · pp. 28–29, Figure 11 · p. 29

E.2 Sensitivity Analysis

Multi4D remains robust to tested hyperparameter changes, while competition, branch-specific updates, and utility-based lifting adapt allocation rather than relying on fixed motion scores.

E.2 Sensitivity Analysis

  • Sweeps over regularizers, lifting count, and mask-score threshold remain close to the reference setting on Neu3D, indicating robustness to exact hyperparameters.Table 6 reports Δmean peak PSNR relative to the reference setting; supplied passages do not provide the sweep values.Source: §E.2 · p. 29, Table 6 · p. 29
  • Appearance competition, branch-specific densification and pruning, and utility-based lifting specialize compact subsets instead of assigning Gaussians by fixed motion scores.Pruning, regrowth, and cross-branch lifting progressively correct misallocations, enabling one fixed parameter set for all scenes.Source: §E.2 · p. 29
  • The lifting threshold is only a coarse indicator of activated foreground Gaussians, not a transient classifier; shared competition, densification, and pruning determine final allocation.Source: §E.2 · p. 29

E.3 LPIPS Evaluation … F.1 Additional Discussion on Related Work

Multi4D reports strong perceptual quality and higher, more stable Neu3D PSNR under first-frame initialization. Its explicit Gaussian representation supports efficient transfer and intrinsic decomposition, separating persistent motion from transient detail across multi-view and monocular settings.

E.3 LPIPS Evaluation

  • Multi4D evaluates LPIPS-Alex on multi-view Technicolor and Neu3D, and LPIPS-VGG on monocular NeRF-DS, following each dataset's protocol.The reported results indicate strong perceptual quality across all three datasets.Source: §E.3 · pp. 29–30, Table 7 · p. 30

F.1 Additional Discussion on Related Work

  • Compared with NeRF-based decomposition methods D2-NeRF and NeRFPlayer, Gaussian-based DeGauss and Multi4D offer superior training and rendering efficiency and explicit densification.Explicit densification supports compact, self-regularized scene modeling.Source: §F.1 · p. 30

E.4 Initialization Robustness

  • 34.17 mean PSNR with first-frame initialization exceeds STG first-frame (32.94), STG all-frame (33.58), and FreeTimeGS first-frame (32.95) on Neu3D test views.FreeTimeGS was reproduced because its official code was unavailable; unlike FreeTimeGS's reported dense 4D position/velocity initialization, Multi4D uses only first-frame initialization.Source: §E.4 · p. 30, Figure 12 · p. 31

F.1 Additional Discussion on Related Work

  • Multi4D's explicit Gaussian representation enables direct cross-subset transfer and lifting, incorporating motion-aware primitives for faster modeling and better geometry-motion consistency.Source: §F.1 · p. 31
  • Unlike Swift4D [42], MAPo, and motion-layering methods, which rely on external cues and static-camera multiview data, Multi4D derives decomposition through competing Gaussian branches.Static, persistent dynamic, and transient branches share a renderer and perform branch-specific densification and pruning, continually revising the decomposition and improving robustness to floaters.Source: §F.1 · p. 31
  • By decoupling persistent motion from transient residuals, Multi4D regularizes coherent motion while compactly capturing short-lived details, addressing the expressiveness-regularization trade-off in multiview and monocular reconstruction.Prior work increases deformation capacity for multiview fidelity, which can weaken regularization for stable monocular or sparse-view reconstruction.Source: §F.1 · p. 31

F.2 Additional Qualitative Analysis and Limitations · F.3 All-Point Tracking and Feed-Forward Dynamic Scene Representation

Multi4D’s hybrid representation separates static structure, persistent motion, and transient appearance to retain correspondence-aware motion and high-quality reconstruction. The analysis also identifies challenging cases and proposes persistent–transient decomposition as a structural prior for feed-forward dynamic reconstruction.

F.2 Additional Qualitative Analysis and Limitations

  • Newly appearing content is first captured by the transient branch and can become persistent when repeatedly observed, as illustrated by monocular-dataset success and failure cases.Very fast camera or object motion remains challenging.Source: §F.2 · p. 31, Figure 13 · p. 32
  • In low-texture regions such as Painter, appearance-only separation cues are weak despite high PSNR, while boundary confusions mainly reflect the smoothness bias of persistent deformation.Source: §F.2 · p. 32

F.3 All-Point Tracking and Feed-Forward Dynamic Scene Representation

  • Multi4D disentangles static structure from dynamic content and persistent motion from transient appearance, using bottom-up self-regularized optimization to preserve correspondence-aware motion and high-quality reconstruction.The paper frames this as addressing the tension between rendering fidelity and temporally consistent correspondence.Source: §F.3 · p. 32
  • Existing feed-forward dynamic Gaussian representations based on per-frame 3DGS [23] or dense 4DGS can be over-parameterized and offer limited explicit motion correspondence.Transient effects and disocclusions make stable point identities over time difficult to maintain.Source: §F.3 · p. 32
  • A hybrid decomposition into persistent and transient components is proposed as a useful structural prior for feed-forward dynamic Gaussian models.Source: §F.3 · p. 32
  • Extending Multi4D to feed-forward reconstruction and evaluating point or Gaussian tracking on diverse egocentric and dynamic-scene benchmarks are identified as future work.Source: §F.3 · p. 32

Related papers

Loading 2606.22197v1…