PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Automatic 3D Gaussian Splatting Edits for Online Bayesian Scene Change Detection

Changrui Zhu, Simon Julier

arXiv:2610.28490v2cs.CVcs.GR

TL;DR

Online scene change detection must combine incoming observations despite unknown change priors and evidence models. The paper learns scene-specific models from automatic 3DGS edits and unchanged observations, then uses Bayesian evidence fusion and closed-form Gaussian-field updates. It reports state-of-the-art accuracy on PASLCD and CCS3D, with 67.64% mIoU on PASLCD and 17.2 FPS for the complete pipeline.

  • Problem

    Online change detection needs to combine incoming evidence, but the prior probability of change and models mapping image differences to evidence are unknown beforehand, and numerical Bayesian inference can be costly.

    Source: §1 · p. 1

  • Method

    Automatic edits to a reference 3DGS and unchanged observations provide offline supervision for scene-specific evidence models and response statistics used in online Bayesian prediction and closed-form Gaussian-field updates.

    Source: §1 · p. 1, §4.3 · p. 5

  • Results

    67.64% mIoU on PASLCD and 75.82% mIoU on CCS3D accompany state-of-the-art accuracy; on PASLCD the complete method runs at 17.2 FPS.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The learned models support online evidence accumulation without online backpropagation or past-image replay, while object-guided refinement sharpens boundaries and completes missed object regions.

    Source: §1 · p. 1

    Direct object edits and simple relighting may not cover real changes or unusual lighting, and accumulation on unchanged reference geometry limits representation of newly introduced objects and occlusion boundaries.

    Source: §6 · p. 9

Abstract

from arXiv · show

Online scene change detection requires integrating incoming observations to determine which regions have changed relative to a reference scene. We update change beliefs as new images arrive using generalised Bayesian evidence fusion. However, the prior probability of change and evidence models are unknown beforehand, and numerical Bayesian inference can be costly. We address these challenges by learning offline from automatic edits to a reference 3D Gaussian Splatting (3DGS) model. These edits generate labelled training views without manual annotations. Together with unchanged reference observations, they provide supervision for learning scene-specific evidence models and estimating the effective pixel prior and response statistics. Online prediction combines current evidence with historical support, while closed-form local Bayes factors, parameterized by the learned response statistics, update an accumulated 3D Gaussian field. Object-guided refinement produces the final change masks. Our method achieves state-of-the-art accuracy on PASLCD and CCS3D, with $67.64\%$ and $75.82\%$ mIoU, respectively. On PASLCD, the complete pipeline runs at $17.2$\,FPS over complete query sequences, the fastest among the evaluated online methods. Project page: https://changruizhu96.github.io/work/lego-scd/

1 INTRODUCTION

Scene change detection must compare different-time images despite viewpoint and visibility differences, while existing online Gaussian methods optimize change parameters using current and replayed observations. This paper learns scene-specific evidence models offline from automatic reference edits and unchanged observations, then accumulates evidence efficiently for online prediction.

Motivation

  • Different camera trajectories make direct comparisons difficult, while a reference 3DGS can render the earlier scene at query viewpoints for a common view.Prior methods integrate multi-view cues by optimizing per-Gaussian change parameters in batch or online.Source: §1 · p. 1

Approach

  • Automatic reference 3DGS edits and unchanged observations supervise scene-specific evidence models and estimate the effective pixel prior and response statistics without manual change annotations.This reduces dependence on large labelled datasets for one evidence model spanning diverse appearances and changes.Source: §1 · p. 1
  • Learned response statistics parameterize closed-form local Bayes factors, enabling evidence accumulation without numerical integration, online backpropagation, or past-image replay.Current evidence is combined with historical support for prediction, and object-guided refinement sharpens boundaries and completes missed regions.Source: §1 · pp. 1–2

Results

  • 67.64% mIoU on PASLCD and 75.82% mIoU on CCS3D accompany state-of-the-art accuracy; on PASLCD the complete method runs at 17.2 FPS versus 14.4 FPS for O-SCD Online.The method also outperforms O-SCD Online by 18.40 mIoU points.Source: §1 · p. 2

2 RELATED WORK

Prior work spans image-pair comparison, multi-view 3D change detection, synthetic supervision, and Bayesian evidence accumulation. This method combines scene-specific learning from edited 3DGS references with online evidence accumulation on reference Gaussians.

Scene change detection

  • Image-pair methods compare correspondence and appearance, while 3D representations enable matched-viewpoint comparisons and integration across observations.Examples include SAM-feature and mask matching, object-level semantic/geometric comparison, multi-view voting, and Gaussian change channels.Source: §2 · p. 2

Learning from synthetic changes and 3D editing

  • Synthetic-supervision methods construct paired data from segmentation or synthetic 3D scenes, while this work generates labelled changes through edits to a prebuilt reference 3DGS.Its scene-specific fitting focuses on each reference scene’s appearance and response statistics rather than one model covering many environments.Source: §2 · p. 2

Bayesian evidence accumulation

  • Generalised Bayesian updates connect observations to belief changes, and this work uses offline-learned response statistics for closed-form local Bayes factors on reference Gaussians.Related approaches include probabilistic object-state updates and closed-form changepoint posteriors.Source: §2 · p. 2

3 PRELIMINARIES

The method renders a fixed reference 3DGS from each query viewpoint, then compares the query with that rendering using photometric and semantic cues. Gaussian rendering contributions also connect pixels to the reference representation for later evidence aggregation.

Reference rendering

  • 18 query/reference-symmetric features are formed by matching XFeat keypoints to registered reference images and estimating each query pose with PnP-RANSAC.At that pose, the fixed reference 3DGS renders the earlier scene for comparison.Source: §3 · pp. 2–3
  • Each Gaussian contributes view-dependent colour through projected opacity and front-to-back alpha compositing, with its rendering contribution determined by preceding Gaussians along the ray.The reference reconstruction remains fixed during online inference.Source: §3 · p. 3

Change cues

  • RGB L1 and D-SSIM provide photometric differences, while frozen SAM2 encoder features provide semantic differences whose normalized, upsampled responses are averaged into the initial cue.The initial difference response serves as input to evidence learning and online inference.Source: §3 · p. 3

4 METHOD

The method treats pixel change prediction as generalised Bayesian fusion of current evidence, historical support, and an effective prior, with five scene-specific components learned offline. Online prediction and Gaussian-field updates use causal evidence accumulation, while automatic reference edits and unchanged views provide the fitting supervision.

4.1 PROBLEM FORMULATION AND OVERVIEW

  • An effective pixel prior and four scene-specific models construct current evidence and history, while a contour filter supports mask refinement; all five components are fitted offline and fixed during inference.The response suppressor and LLR mapping construct current evidence, and unchanged-response statistics parameterize closed-form accumulation.Source: §4.1 · p. 3

4.2 CAUSAL ONLINE CHANGE DETECTION

  • Each query’s current-view evidence supports both pixel prediction with historical support and local Bayes-factor updates to the Gaussian field for later queries.Figure 1 summarizes evidence fusion, history accumulation, and refinement.Source: §4.2 · p. 3, Figure 1 · p. 4
  • Current evidence is corrected to suppress view-synthesis artifacts, mapped to a shared evidence scale, and combined with Gaussian-aggregated support to retain pixel detail and spatially shared evidence.The method combines image evidence and Gaussian support before prediction.Source: §4.2 · pp. 3–4
  • For accumulation, neighbourhood responses beyond the centre’s expected unchanged co-response form a residual used in a closed-form local Bayes factor; adding its log evidence updates the field for subsequent queries.Evidence is neutral when no neighbours are observed, and the method avoids numerical integration and iterative optimization.Source: §4.2 · pp. 4–5
  • Historical support is rendered from the previous accumulated field and fused with current evidence and the offline-fitted prior to produce pixel-level change scores.Unobserved Gaussians provide neutral support on covered pixels.Source: §4.2 · p. 5
  • Object-guided refinement uses prompted SAM2 masks to penalize blurred boundaries and recover missed object parts, then thresholds the refined scores at 0.5.The contour refinement retains original scores for unprompted pixels.Source: §4.2 · p. 5

4.3 LEARNING FROM AUTOMATIC 3DGS EDITS

  • Automatic object-level edits to the reference 3DGS, paired with relighting, generate labelled training views without manual annotations; reference cameras are split between editing and unchanged-only statistics.Matched masks across views estimate Gaussian object membership, and duplicate models preserve the original reference during edits.Source: §4.3 · pp. 5–6
  • The edited views fit response suppression, the monotonic image-evidence LLR mapping, the unchanged-response statistics, the effective pixel prior, and the contour filter.The prior is selected by reliability-weighted foreground IoU; unchanged observations estimate Gaussian response means, uncertainty, and spatial co-response.Source: Figure 2 · p. 6, §4.3 · pp. 6–7

5 EXPERIMENTS

The method outperforms evaluated baselines on PASLCD and O-SCD(O) on CCS3D, while also achieving higher PASLCD throughput than O-SCD(O). Ablations, reuse tests, and visual comparisons examine the method’s components and scope.

Protocol

  • Evaluation covers 20 real-world PASLCD scene-instances and four synthetic CCS3D scenes; comparisons average two complete runs per dataset.The method and O-SCD are evaluated locally, while other published baseline results are taken from GS-Diff for PASLCD and SCaR-3D for CCS3D.Source: §5 · p. 7, Table 1 · p. 8, Table 2 · p. 8

Complete evaluation

  • 67.64% mIoU and 78.87% F1 on PASLCD exceed O-SCD(O) by 18.40 mIoU points and GS-Diff by 3.24 points; on CCS3D, the method exceeds O-SCD(O) by 25.88 mIoU points.The GS-Diff comparison does not require a post-change reconstruction.Source: §5 · p. 7, Table 1 · p. 8, Table 2 · p. 8

Visual results

  • Visual comparisons show more complete object masks in Porch, fewer boundary false positives in Pots, and fewer missed changes in Playground than O-SCD(R).Figure 3 columns show reference renderings, queries, ground truth, offline O-SCD(R), and the complete online method; per-frame foreground IoU is shown for one sequence run.Source: §5 · p. 8, Figure 3 · p. 8

Cross-instance reference reuse

  • Reusing fitted parameters for the other instance of the same PASLCD scene keeps pixel mIoU at 59.93% versus 60.36%, and full-output mIoU at 67.15% versus 67.64%.The reuse test does not refit the reference reconstruction or its parameters.Source: §5 · p. 8

Independent component ablations

  • Removing the pixel prior causes a 16.47-point drop, while removing historical support reduces mean pixel mIoU by 2.56 points; temporal averaging instead of Bayes-factor updates reduces it by 4.42 points.Gaussian-aggregated support contributes 1.89 points; response suppression and likelihood-ratio mapping also provide gains.Source: §5 · p. 8, Table 3 · p. 9

Runtime

  • 17.2 FPS for the complete method exceeds O-SCD(O)’s 14.4 FPS; without refinement, the pixel pipeline reaches 27.9 FPS at 60.36% mIoU.The complete method improves mIoU by 18.40 points over O-SCD(O); offline preparation averages 215.82 seconds per instance.Source: Table 4 · p. 9, §5 · p. 9

Independent component ablations

  • Removing contour filtering and object completion reduces mean mIoU by 4.49 and 5.08 points, respectively, while contour image weights add 0.26 points.Removing relighting during data generation reduces full-output mIoU by 0.66 points; doubling its strength gives similar aggregate results.Source: §5 · p. 9

6 CONCLUSION AND LIMITATIONS

The paper learns scene-specific evidence models and statistics from automatic reference 3DGS edits, then uses them for online Bayesian evidence accumulation. It reports benchmark gains while identifying limits in generated changes, reference geometry, and probabilistic assumptions.

Conclusion

  • Automatic reference 3DGS edits and unchanged observations provide supervision for scene-specific evidence models and statistics.The learned quantities support closed-form local Bayes factors and generalised Bayesian history accumulation without online backpropagation or past-image replay.Source: §6 · p. 9
  • The method outperforms O-SCD on PASLCD and CCS3D, achieving 67.64% mIoU at 17.2 FPS on PASLCD.Source: §6 · p. 9

Limitations

  • Direct object edits and simple relighting may not cover real changes or unusual lighting; unchanged reference geometry limits representation of new objects and occlusion boundaries.Response uncertainty and spatial covariance models are working approximations, and fused scores are not guaranteed to be calibrated probabilities of real-world changes.Source: §6 · p. 9

AI USE STATEMENT

The authors report using generative AI tools for early methodological work, implementation, drafting, and textual refinement. They state that authors reviewed and verified the contributions and take responsibility for the paper.

AI use

  • Generative AI assisted initial methodology discussions, exploratory brainstorming, algorithm and code implementation, preliminary drafting, and textual refinement.Source: AI USE STATEMENT · p. 10
  • The authors state that generative AI was not used to formulate theoretical claims, derive mathematical formulations, collect benchmark observations, or interpret empirical results.Source: AI USE STATEMENT · p. 10
  • The authors report reviewing and testing AI-assisted contributions, independently executing and verifying implementations and evaluation pipelines, and manually checking derivations.They assume full responsibility for the paper and its claims.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The work uses existing scene-change benchmarks and identifies two potential consequences of incorrect predictions: missed changes and unnecessary follow-up inspections.

Ethics

REPRODUCIBILITY STATEMENT

The reproducibility statement records scene-specific inputs, fitted models, parameters, and per-image results. It also specifies that scene-specific parameters are frozen before evaluation.

REPRODUCIBILITY STATEMENT

A.1 AUTOMATIC REFERENCE EDITING AND DATA GENERATION

This section describes how automatic 3DGS edits and unchanged reference views generate scene-specific training evidence and calibrate unchanged-scene response statistics without manual change annotations. It also specifies how pixel evidence and accumulated Gaussian support are combined through a conditional neighbourhood Bayes factor, with causal decoding and coverage rules constraining change detection.

A.1 AUTOMATIC REFERENCE EDITING AND DATA GENERATION

  • Automatic camera assignment separates unchanged-only views for response estimation from editing views that provide labelled examples for learning.The editing subset supports response suppression, likelihood-ratio mapping, effective pixel-prior estimation, and contour filtering; the unchanged-only subset supports fitting the unchanged-response model and spatial co-response. Cross-view mask association and edit validation use reference-derived data, without manually annotated change masks.Source: §A.1 · pp. 13–14
  • 11,384 edited training records span 20 instances: 5,692 identity and 5,692 relighting records.Synthetic interventions are transferred to real reference images while retaining image–render discrepancies; unchanged controls use the unedited rendering under the same lighting condition.Source: §A.1 · p. 16

A.2 IMAGE EVIDENCE PROCESSING AND AGGREGATION

  • The image response fuses photometric and feature cues, then renderer-conditioned logistic models suppress responses associated with rendering conditions.The full model adds renderer features to local response statistics, and its correction is bounded and disabled at very low rendered coverage.Source: §A.2 · pp. 16–17
  • A monotone response-to-evidence mapping is learned from changed, edited-image unchanged, and unedited-reference histograms; its equal prior weights set the image-evidence scale.The scene-specific effective pixel prior is fitted separately using causal prediction and field update.Source: §A.2 · pp. 17–18

B UNCHANGED-SCENE RESPONSE STATISTICS

  • The frozen image-evidence pipeline is applied to unchanged-only images, whose Gaussian-aggregated evidence defines the normalized response coordinate for unchanged-response statistics.Those statistics use observations from the unchanged-only camera subset.Source: B UNCHANGED-SCENE RESPONSE STATISTICS · pp. 18–19, §A.2 · p. 18
  • The unchanged-response model uses Beta soft counts for per-Gaussian response moments, with observed means unshrunk and unobserved Gaussians retaining scene-based moments.Its variance represents uncertainty in the soft-count construction, not predictive variance for a new image; empirical standardized responses are separately used to estimate spatial correlation.Source: B UNCHANGED-SCENE RESPONSE STATISTICS · p. 19

C CONDITIONAL NEIGHBOURHOOD BAYES FACTOR

  • The conditional neighbourhood Bayes factor analytically integrates a half-normal shared-increase model over the residual response, avoiding numerical integration and optimization.The residual subtracts the anchor response using the estimated spatial co-response; when no neighbour has positive observation weight, evidence is neutral (B = 1).Source: C CONDITIONAL NEIGHBOURHOOD BAYES FACTOR · pp. 20–21

D ACCUMULATED SUPPORT AND CAUSAL DECODING

  • Pixel change posteriors combine current evidence, rendered historical support, and the effective prior, while each sufficiently supported observation adds its log Bayes factor once to the Gaussian accumulator.History is decoded by applying the sigmoid to each Gaussian before spatial averaging; object-guided refinement changes the image prediction without modifying history.Source: D ACCUMULATED SUPPORT AND CAUSAL… · pp. 21–22
  • Pixels with reference coverage at most 10^-3 receive zero change probability and are excluded from object completion, so changes in uncovered regions may be missed.These coverage rules avoid unsupported updates but cannot recover missing geometry; the observation indicator measures rendering contribution, not geometric accuracy.Source: D ACCUMULATED SUPPORT AND CAUSAL… · p. 22

E RELIABILITY-WEIGHTED EFFECTIVE-PRIOR ESTIMATION

The section describes reliability-weighted fitting of an effective pixel prior, object-guided output refinement, and additional analyses of priors, relighting, and historical support. The reported tests include a gain from Bayes-factor history over temporal averaging and a modest benefit from the default relighting strength.

E RELIABILITY-WEIGHTED EFFECTIVE-PRIOR ESTIMATION

  • The effective prior is selected by reliability-weighted training IoU, using unchanged-response tails to weight images and branch-and-bound search certified within 0.01 percentage points.The weights are fixed before threshold search, and synthetic labels provide unchanged pixels for reliability weighting and changed pixels for IoU.Source: E RELIABILITY-WEIGHTED EFFECTIVE-PRIOR… · p. 23

F OBJECT-GUIDED MASK REFINEMENT

  • Object-guided refinement uses SAM2 proposals to filter unsupported detections and complete retained object regions, modifying only the final output.Connected components from pixel predictions provide prompts; synthetic edit masks supply the fitting labels.Source: F OBJECT-GUIDED MASK REFINEMENT · p. 24

G.1 CROSS-INSTANCE REFERENCE REUSE

  • Cross-instance reuse tests 20 directed PASLCD pairs without target-data refitting, but similar query sets make it a within-scene reuse test, not cross-scene generalization.Each scene’s fitted parameters and reference are reused for the other instance’s queries in both directions.Source: §G.1 · p. 24, Table 6 · p. 25

G.2 FITTED PRIORS AND CHANGE AREA

  • 0.00588–0.30471: fitted PASLCD priors have median 0.02368 and interquartile range [0.01487, 0.05489], with each instance’s prior fixed online.The priors select decision operating points, not the fraction of changed pixels in a query.Source: §G.2 · p. 25, Figure 5 · p. 26
  • Frames with less than 1% change have lower mIoU and higher FNR than the 1–5% group; scene and viewpoint differences prevent isolating change-area effects.Evaluated change fractions span 0.17–20.13%; only one frame exceeds 20%, and none reaches 30%.Source: §G.2 · p. 25, Table 8 · p. 27

G.3 RELIGHTING STRENGTH

  • 0.66 points: default relighting improves full-output mIoU over K = 0, with gains in 13 of 20 instances.Doubling relighting strength changes pixel and full mIoU by +0.03 and −0.07 points, respectively, relative to the default.Source: §G.3 · p. 27

G.4 CONSTRUCTION OF HISTORICAL SUPPORT

  • 4.42 points: Bayes-factor historical support improves mean mIoU over temporal averaging and performs better in 16 of 20 PASLCD instances.The comparison uses pixel outputs on the same 20 instances and 500 frames.Source: §G.4 · p. 27

G.5 EVALUATION AND RUNTIME DETAILS

The section reports repeatability and runtime measurements, offline preparation time, and additional qualitative comparisons on PASLCD and CCS3D. The full pipeline’s measured throughput is reported across complete PASLCD query sequences.

G.5 EVALUATION AND RUNTIME DETAILS

  • 17.15 and 17.19 FPS: the full pipeline’s two complete PASLCD runs process 1,000 frames, while pixel runs achieve 27.88 and 27.98 FPS.Both full runs achieve 67.64% mIoU, with a largest per-instance difference of 0.019 percentage points.Source: §G.5 · pp. 27–28

G.6 OFFLINE PREPARATION AND COMPONENT TIMING BREAKDOWN

  • 215.82 seconds: offline preparation and fitting average about 3.6 minutes per PASLCD instance, given an existing reference reconstruction.The workflow includes synthetic edit generation and scene-component fitting.Source: §G.6 · p. 28, Table 12 · p. 29

H ADDITIONAL VISUAL RESULTS

  • PASLCD examples show refinement recovering a cabinet region, while false positives near a ceiling and missed object regions remain in other views.Thin structures remain partly missed in Garden.Source: Figure 6 · p. 30, Figure 7 · p. 31

H.1 COMPARISON ON CCS3D

  • On CCS3D, qualitative examples show reduced excess responses around displaced objects and unchanged regions, while boundary errors remain in Bookcase.The examples cover Livingroom, Desk, Bookcase, and Bedroom.Source: Figure 8 · p. 32

H.2 EVIDENCE FUSION AND COMPONENT ABLATIONS

Evidence fusion combines current responses with historical support and a fitted prior, improving mean IoU in the illustrated Cantina sequence. The accompanying figure presents qualitative component ablations using the same queries.

H.2 EVIDENCE FUSION AND COMPONENT ABLATIONS

  • 57.84%: mean IoU after fusion exceeds the 47.31% achieved with current-view evidence alone across the 25-frame Cantina sequence.Fusion can reduce false positives but also suppress true changes, as frame 25 illustrates.Source: §H.2 · p. 33, Figure 9 · p. 33
  • Figure 9 compares Ct-only and fused masks across frames 1, 13, 18, and 25, with log-odds maps saturated at ±6 for display.Both masks threshold pixel probability at 0.5; the fused version also uses historical support and the fitted prior.Source: Figure 9 · p. 33, §H.2 · p. 33
  • Figure 10 groups five pixel-pipeline and three output-refinement ablations against the full method, using the same queries and per-frame foreground IoU.The no-prior variant sets π = 0.5; the image-weight variant omits weighting during contour fitting.Source: Figure 10 · p. 34

Related papers

Loading 2610.28490v2…