PaperlayerView exact revision on arXiv

Source-linked AI summary

ChronoFuseGS: Multi-Temporal Gaussian Fusion with Per-Splat Persistence and Change Visualization

Tobias Batik, Diana Marin, Peter Kán, Hannes Kaufmann

arXiv:2609.31339v1cs.GRcs.CV

TL;DR

Reconstructing environments from image sets captured at different times is challenging when scene parts change between captures. ChronoFuseGS merges separately trained Gaussian models, encodes per-Gaussian persistence, and visualizes change; it improves novel-view synthesis over individual timestep models and highlights fine-grained changes.

  • Problem

    Reconstruction across changing image sets is challenging because the scene may no longer represent one unchanged state.

    Source: §1 · pp. 1–2

  • Method

    ChronoFuseGS merges separately trained timestep models, uses per-Gaussian persistence to share persistent scene information, and renders changes for selected timesteps.

    Source: §1 · p. 2

  • Results

    The combined model consistently improves novel-view synthesis over individually trained models and reliably highlights changes in fine details and subparts.

    Source: §10 · pp. 11–12, §8 · p. 11

  • Takeaways & Limitations

    The approach supports incremental extension and sub-object change visualization across captured timesteps without switching between separate reconstructions.

    Source: §8 · p. 11, §10 · p. 11

    If the scene changes dramatically between timesteps, SfM registration may fail, preventing reconstruction-quality improvement.

    Source: §9 · p. 11

Abstract

from arXiv · show

Reconstructing environments where parts of the scene change between captured image sets poses a challenge for 3D scene reconstruction. We present ChronoFuseGS, a multi-temporal Gaussian Splatting approach that addresses this issue by taking multiple separately trained Gaussian Splatting models, each representing a distinct timestep and partially overlapping in geographic coverage, and merging them into a single combined model. By allowing Gaussians from one timestep to contribute to the reconstruction at other timesteps, our approach leverages data across all captured timesteps to refine persistent parts of the scene. The model supports incremental extension, allowing new timesteps to be added while preserving the existing merged reconstruction. It encodes, for each Gaussian primitive, at which timesteps it contributes to the reconstruction. To support visual exploration of the reconstructed scene, we present a change-aware visualization approach that highlights the parts of the scene that have changed across a user-defined time selection, while preserving the color of persistent parts. Since the persistence encoding operates at the Gaussian primitive level, changes are visualized at sub-object granularity rather than being limited to object-level changes. We evaluate our approach on a real-world outdoor dataset of a flood management area, captured over 7 months across eight recording days and covering seasonal vegetation changes, snow cover, and flooding events, which we make publicly available. Our results demonstrate that the combined model consistently outperforms individually trained single-timestep models in novel-view synthesis quality, recovers structural details absent in the individual reconstructions, and reliably highlights changes in fine details and sub-parts of objects and natural structures.

1. Introduction

Reconstructing large, evolving environments is difficult because captures may span periods during which the scene changes. ChronoFuseGS merges separately trained timestep models, encodes per-Gaussian persistence, and visualizes changes across selected timesteps.

1. Introduction

  • Large or partially unknown environments may require repeated captures, so the scene can change and no longer represent a single unchanged state.The approach targets, among other settings, evolving natural disasters such as floods and landslides.Source: §1 · pp. 1–2
  • ChronoFuseGS merges individually pretrained models for separate timesteps and lets persistent Gaussians contribute across timesteps, encoding each Gaussian’s timestep contributions.The paper reports improved novel-view synthesis quality over independently trained single-timestep models.Source: §1 · p. 2
  • Its change-aware rendering highlights parts absent from a user-selected time period while retaining persistent parts’ true appearance.The visualization indicates when and where scene parts, objects, and details are present.Source: §1 · p. 2
  • The paper contributes a quantitative comparison and visualization user study, alongside an open dataset from eight recording days across seven months.The dataset includes seasonal vegetation changes, snow cover, and flooding events.Source: §1 · p. 2

2. Related Work

Related work spans explicit Gaussian scene representations, temporal reconstruction, and change detection. ChronoFuseGS distinguishes itself by fusing independently trained timestep models and learning change encoding through cross-timestep image comparisons.

2.1. Gaussian Splatting

  • 3D Gaussian Splatting represents scenes with explicit anisotropic Gaussians optimized from posed images and rendered with a differentiable rasterizer.Its explicit representation supports high-quality view synthesis and real-time rendering.Source: §2.1 · p. 2

2.2. Multi-Timestep and Dynamic Reconstruction

  • Dynamic 3DGS methods model continuous temporal change, while ChronoFuseGS targets repeated independent captures and supports incremental fusion of pretrained timestep models.Unlike ChronoGS, it allows models to be trained, pruned, and optimized in parallel before merging, and new timesteps to be added without rebuilding the full model.Source: §2.2 · p. 2

2.3. Change Detection in Gaussian Splatting

  • Prior change-detection methods derive post-hoc labels, whereas ChronoFuseGS uses cross-timestep image comparisons to jointly update geometry and per-Gaussian persistence.The related work also positions the approach at the intersection of temporal reconstruction, map reuse, and change detection.Source: §2.3 · p. 3

3. Multi-Temporal Gaussian Fusion

ChronoFuseGS merges individually trained timestep models into one refined model, where per-Gaussian parameters encode timestep contributions and support incremental updates. Cross-timestep initialization and combined refinement establish and preserve this encoding while fitting all timesteps.

3. Multi-Temporal Gaussian Fusion

  • The pipeline registers and merges separately trained timestep models, initializes cross-timestep persistence and light compensation, then jointly refines the combined model.A new pre-trained model can be fused into an existing reconstruction to add a timestep.Source: §3 · p. 3

3.1. Multi-Temporal Model

  • Each Gaussian stores timestep-dependent opacity and light compensation alongside its standard 3DGS parameters, allowing its contribution to vary across timesteps.Opacity near 1 indicates visibility at a timestep, while opacity near 0 makes the Gaussian invisible there.Source: §3.1 · p. 3

3.2. Cross-Timestep Initialization

  • Cross-timestep initialization optimizes each timestep’s Gaussian subset against other timesteps’ images to estimate persistence and light compensation.The method uses LPIPS spatial feature differences as per-pixel similarity weights and randomizes the background to decouple light compensation from persistence.Source: §3.2 · p. 4

3.3. Combined Refinement

  • Combined refinement trains on images from all timesteps while penalizing changes to the sequential differences in the initialized persistence values.For overlapping Gaussians, opacity is initially halved to encourage resolving overlap through opacity while preserving persistence; the change penalty is MAE between initialized and refined sequential differences.Source: §3.3 · p. 5

4. Multi-Temporal Change Visualization

The multi-temporal renderer supports user-defined timestep selections and highlights non-persistent Gaussians while preserving the appearance of persistent scene parts. Customization options help distinguish changes from true color without altering fully persistent Gaussians.

4. Multi-Temporal Change Visualization

  • The renderer highlights non-persistent areas at Gaussian-level granularity across user-defined time ranges, including scenes with partial and varying spatial coverage.Persistent parts retain their appearance while the selected non-persistent parts are highlighted.Source: §4 · p. 5, §4.1 · p. 6
  • Binary timestep selections are normalized for rendering; blended true-color views can make removed objects semi-transparent, while a max-based alternative avoids that effect.The authors use binary selections because intermediate values were hard for users to interpret; true-color views can also make small changes difficult to spot.Source: §4 · p. 5

4.1. Change Visualization

  • Because Gaussian presence is continuous and combinations multiply across timesteps, a lookup table would require thresholds and quickly become impractical; filtering absent-timestep colors leaves persistent Gaussians unaltered.Four timesteps would already require 14 colors, exceeding the cited recommendation for categorical color scales.Source: §4.1 · p. 6

4.2. Increasing Visual Distance

  • A sigmoid can optionally increase visual contrast between persistent and non-persistent Gaussians, with parameters controlling contrast and the transition midpoint.The rendering can also use grayscale or reduced contrast to separate the highlights from true colors, depending on whether clear change highlighting or overall scene appearance is preferred.Source: §4.2 · p. 6

4.3. Custom Highlight Colors

  • Custom gradients remap the persistence signal’s direction to indicate change timing while retaining its magnitude and saturation, leaving fully persistent Gaussian colors unaffected.The remapped gradient color is interpolated between adjacent samples.Source: §4.3 · pp. 6–7

5. Implementation Details

The implementation combines separately trained Gaussian sets through cross-timestep initialization and joint refinement, then renders time selections in an interactive visualization system. Training choices vary with scene scale and detail requirements.

5. Implementation Details

  • Initial Gaussian sets use PUP 3D-GS for large scenes and original Gaussian Splatting for small scenes, with pre-computed colors and random backgrounds during optimization.The stated position learning rate is 1.6×10−5 for large outdoor scenes and 1.6×10−4 for other test scenes.Source: §5 · p. 7
  • Cross-timestep initialization compares each Gaussian set against the other timestep image sets for 3,000 iterations, optimizing opacity manipulation and light compensation.The procedure starts with random background colors and gradually limits the background to black.Source: §5 · p. 7
  • Combined refinement optimizes Gaussian parameters except light compensation; opacity manipulation uses a lower learning rate than opacity to preserve a strong persistence encoding.Gaussians that contribute to no training image for a timestep have that timestep’s opacity manipulation set to zero, and Gaussians never visible at any timestep are pruned.Source: §5 · p. 7
  • The interactive renderer uses Unreal 5.6 and a GPU Niagara Particle System, representing each splat as a camera-facing particle sprite and allowing binary user-defined time selections.All renderings and FPS measurements use the highest material settings; the code and dataset are publicly available.Source: §5 · p. 7
  • Figure 8 arranges five dataset subsets by row and compares the individual baseline, the 30k-iteration ChronoFuseGS result, and ground truth by column; Figure 9 shows results across datasets.The Autumn subset has four timesteps and Complete NWM has nine.Source: Figure 8 · p. 8, Figure 9 · p. 8

6. Dataset

The evaluation dataset captures a flood-control area across changing seasons and events, alongside synthetic scenes designed to represent object appearance changes. Subsets isolate flooding, snow, and vegetation change to model more focused data collection.

6. Dataset

  • No Wolf in the Meadow records a roughly 100 × 200 meter flood-control area in Vienna on eight days from August 2025 to March 2026, with seasonal, lighting, snow, flooding, and renovation changes.The outdoor scene combines natural and built elements and was recorded with a consumer drone using pre-recorded flight plans.Source: §6 · p. 8
  • The NWM dataset includes subsets focused on flooding, snow, and vegetation change to simulate more realistic collection processes, including during a natural disaster.Some subsets reuse timesteps, while the snow subset also includes two timesteps absent from the complete dataset.Source: §6 · p. 8
  • Two synthetic datasets provide controlled changes: Baluster Vase adds cracks over four timesteps, while Coral shows objects appearing and disappearing across four timesteps under constant lighting.Both contain 3D-scanned and textured objects rendered in Blender.Source: §6 · p. 8

7. Evaluation

Across five NWM subsets, the fused model consistently outperformed individually trained baselines, while incremental extension reduced the time needed to add a timestep. In the user study, interactive highlighting matched baseline accuracy, whereas static highlighting performed worse.

7.1. Reconstruction Quality

  • Across all five datasets and three metrics, Ours 30k achieved the highest scores, and Figure 8 shows finer detail including structures absent from single-timestep reconstructions.The 6k model already beat the best individual baseline on LPIPS across all datasets, with improved or comparable PSNR and SSIM; overall metric differences were small.Source: §7.1 · p. 9, Table 2 · p. 9
  • On Autumn, combined optimization scored 21.2/0.70/0.46 for PSNR/SSIM/LPIPS, outperforming freezing o_i (21.0/0.68/0.47) or α_i (20.2/0.66/0.49).Source: §7.1 · p. 9
  • Adding timesteps increased detail across all timesteps at equal refinement iterations, suggesting Gaussians are reused rather than each timestep being refined independently.The complete reconstruction contained more detail than the subset despite fewer iterations per timestep.Source: §7.1 · p. 9

7.2. Training Times & Rendering Frame Rates

  • Adding t8 incrementally took 42 minutes versus 136 minutes for recomputation from scratch; merging accounted for 22 and 75 minutes, respectively.Source: §7.2 · p. 10, Figure 10 · p. 10

7.3. User Study

  • Interactive highlighting (83% mean success) did not differ significantly from baseline (93%), while static highlighting (72%) scored significantly below both.Interactive completion time was significantly longer than baseline; Figure 12 presents the condition results.Source: §7.3 · p. 10, Figure 12 · p. 10
  • Participants rated static highlighting highest for quickly spotting changes (5.6/7), but reported difficulty decomposing mixed colors into individual timesteps.Participants often selected several timesteps simultaneously; for persistent areas, static highlighting performed on par with the other methods.Source: §7.3 · pp. 10–11

8. Discussion

The fused reconstruction gains detail through cross-timestep optimization while retaining per-Gaussian persistence information. Its change visualization supports sub-object exploration across natural and built environments, including sparsely covered areas.

8. Discussion

  • Cross-timestep optimization increases detail without increasing the combined Gaussian count, and incremental fusion improves the reconstruction without reprocessing previously merged timesteps.Source: §8 · p. 11
  • Per-Gaussian persistence enables change highlighting at sub-object granularity and indicates when scene parts are present across vegetation, snow, water-level, and built-environment changes.Source: §8 · p. 11
  • The visualization also highlights structures or parts not captured in recent timesteps, supporting capture planning.Source: §8 · p. 11

9. Limitations and Future Work

ChronoFuseGS has scope and registration constraints, and its user study does not yet evaluate 3D navigation or field deployment.

9. Limitations and Future Work

  • The method does not account for structural changes or lighting shifts occurring within a single recording period.Source: §9 · p. 11
  • Merging depends on accurate SfM registration; dramatic inter-timestep changes such as snow cover may cause registration failure and prevent reconstruction improvement.SfM registration is also a time-consuming preprocessing step.Source: §9 · p. 11
  • The optimization currently preserves approximately the same Gaussian count because it lacks cross-timestep pruning; adding pruning remains future work.Source: §9 · p. 11
  • The user study used prerendered 2D images, isolating perception from 3D navigation and interface familiarity rather than testing field deployment.Source: §9 · p. 11

10. Conclusion

ChronoFuseGS merges individually trained Gaussian models into a refined multi-timestep reconstruction, using per-Gaussian persistence to support cross-timestep reconstruction and change visualization. On a real-world dataset spanning seven months, it showed consistent novel-view synthesis improvements and effective change highlighting.

10. Conclusion

  • ChronoFuseGS merges individually pretrained Gaussian models into one refined reconstruction supporting novel-view synthesis across multiple timesteps.Source: §10 · p. 11
  • Per-Gaussian persistence supports reconstruction improvement and incremental extension, while change visualization marks scene changes at sub-object granularity without switching between reconstructions.Source: §10 · p. 11
  • The real-world outdoor dataset spanning seven months showed consistent novel-view synthesis improvements and effective change highlighting.Source: §10 · p. 12
Loading 2609.31339v1…