Source-linked AI summary
Towards Benchmarking Scene Background Initialization
Lucia Maddalena, Alfredo Petrosino
TL;DR
Background initialization seeks a foreground-free scene model from images that may contain occlusions, but comparisons of existing methods lack a common ground-truthed dataset and metric set. The paper assembles and publishes the SBI dataset, ground truths, and evaluation metrics, and compares existing methods. Its sequence-level findings vary: different methods handle persistent or widespread foreground clutter better on different sequences.
Problem
Background-initialization methods lack a common ground-truthed dataset and common metrics for comparing performance.
Source: §I · p. 1Method
Results
Sequence-level outcomes vary: WS2006, RSL2011, and Photomontage handle the walking man in Hall&Monitor but include the briefcase, while other cluttered sequences favor different methods.
Source: B. Qualitative and Quantitative… · pp. 3–4Takeaways & Limitations
The SBI dataset, ground truths, and metric-computation tool provide a common basis for reproducible comparisons of background-initialization methods.
Source: §V · p. 5
Abstract
from arXiv · showhide
Given a set of images of a scene taken at different times, the availability of an initial background model that describes the scene without foreground objects is the prerequisite for a wide range of applications, ranging from video surveillance to computational photography. Even though several methods have been proposed for scene background initialization, the lack of a common groundtruthed dataset and of a common set of metrics makes it difficult to compare their performance. To move first steps towards an easy and fair comparison of these methods, we assembled a dataset of sequences frequently adopted for background initialization, selected or created ground truths for quantitative evaluation through a selected suite of metrics, and compared results obtained by some existing methods, making all the material publicly available.
I. INTRODUCTION
Background modeling comprises representation, initialization, and update, but existing methods have focused more on representation and update than initialization. This paper frames initialization as recovering a foreground-free background from images taken at different times and presents a dataset and evaluation resources for comparing methods.
I. INTRODUCTION
- The background-modeling tasks are representation, initialization, and update, with initialization receiving comparatively limited attention.Representation describes the background model, initialization establishes it, and update adapts it to background changes.Source: §I · p. 1
- Initialization seeks a model of the scene background without foreground objects, given images in which foreground objects may occlude it.The input image set can be initial training frames, non-time-sequence photographs, or an entire sequence.Source: §I · p. 1
- The SBI dataset, ground truths, selected metrics, and comparisons of existing methods were assembled and made publicly available to support fairer comparisons.The paper presents these resources as first steps toward easy and fair comparisons of existing and future methods.Source: §I · p. 1
II. SEQUENCES
The SBI dataset contains seven bootstrap sequences drawn from datasets frequently used in background-initialization research, with curated testing subsets and ground truths.
II. SEQUENCES
- The SBI dataset comprises seven bootstrap sequences drawn from COST 211, ATON, and PBI source datasets.The sequences were selected from sequences frequently used in the literature to evaluate background-initialization algorithms.Source: §II · p. 1
- Testing subsets exclude empty frames, and final resolutions were chosen to avoid boundary-patch computation problems for block-based methods.Source: §II · p. 1
- Ground truths were manually selected from foreground-free frames or stitched from empty background regions across frames, and the dataset and references were made public.Source: §II · p. 1
III. METRICS
The study evaluates computed backgrounds against ground truth with eight literature-based metrics. The metrics assess pixel-level error, structural similarity, and color-image quality, with metric direction indicating better estimates.
III. METRICS
- Eight metrics were selected from those used in background-estimation literature to compare computed backgrounds (CB) with ground truth (GT).Source: §III · p. 1
- AGE, EPs, pEPs, CEPs, and pCEPs quantify gray-level differences or error-pixel counts and proportions, with lower values indicating better estimates.EPs use a threshold τ=20 in the experiments, and CEPs count error pixels whose four-connected neighbors are also errors.Source: §III · p. 2
- PSNR, MS-SSIM, and CQM are higher-is-better measures, while CQM is defined only for color images.For color-image measurements, the study applies metrics 1–7 to the luminance component Y of YCbCr.Source: §III · p. 2
- The scripts for computing the selected metrics were made publicly available.Source: §III · p. 2
A. Compared Methods
The comparison covers five methods using different methodological schemes, including temporal statistics, stable subsequences, block-level completion, and optimal labeling. Their background estimates are obtained through distinct selection or optimization procedures.
A. Compared Methods
- Temporal Median is the baseline, assigning each background pixel the same-location median across the bootstrap sequence.For color sequences, the reported implementation selects the pixel minimizing the sum of L∞ distances to the other pixels.Source: A. Compared Methods · p. 2
- SC-SOBS supplies a spatially coherent temporal-statistics comparison, with its estimate extracted by choosing the modeled weight vector closest to ground truth.The method is not explicitly devoted to initialization, and this ground-truth-based extraction is applicable only for comparison purposes.Source: A. Compared Methods · pp. 2–3
- WS2006 selects the most reliable stable intensity subsequence per pixel or block using RANSAC, then estimates the background by its temporal mean.It assumes that a background value has the longest stable value.Source: A. Compared Methods · p. 3
- RSL2011 maintains distinct candidate blocks over time and estimates the background in a Markov Random Field framework using neighborhood response and spatial continuity.Source: A. Compared Methods · p. 3
- Photomontage frames background estimation as optimal labeling with graph cuts, combining seam-penalty and data terms.For background estimation, the data term uses a maximum-likelihood image objective for visual smoothness.Source: A. Compared Methods · p. 3
B. Qualitative and Quantitative Evaluation
Across the benchmark, sequence difficulty varied with foreground steadiness more than object size, while region-based or hybrid methods and selective pixel choice characterized many of the best performers. Results ranged from strong performance by all methods on the highway sequences to persistent-clutter failures on harder sequences.
B. Qualitative and Quantitative Evaluation
- On Hall&Monitor, WS2006, RSL2011, and Photomontage handled the walking man well but included the abandoned briefcase in the background; highway estimates were faithful across methods.The man occupied the same region for over 65% of Hall&Monitor frames, and the briefcase remained for the last 60%; highway backgrounds were visible for at least 50% of the bootstrap sequences, with no stationary cars.Source: B. Qualitative and Quantitative… · p. 3
- Only RSL2011 perfectly handled CaVignal’s persistent clutter and achieved the best accuracy results across all metrics.Most methods included the man who stood in the left scene area for the first 60% of frames; only WS2006 partially included him on the right.Source: B. Qualitative and Quantitative… · p. 3
- On Foliage, only Median produced a greenish halo over almost the entire scene and had the worst accuracy results across all metrics.Many other methods represented the background quite well despite moving leaves occupying most of it for most of the time.Source: B. Qualitative and Quantitative… · p. 3
- Only Photomontage and RSL2011 handled People&Foliage’s widespread clutter well and achieved the best accuracy results across all metrics.Artificially added leaves and people occupied almost all the scene area in almost all frames.Source: B. Qualitative and Quantitative… · p. 4
- RSL2011 produced Snellen’s best qualitative result, confirmed by quantitative analysis across all adopted metrics.Foreground leaves occupied almost all the scene area in almost all frames, leading most methods to include them in the background.Source: B. Qualitative and Quantitative… · p. 4
- Many top performers were region-based or hybrid and selected candidate pixels, but the compared methodological schemes showed no preferred scheme.The authors identify spatiotemporal inter-pixel relations as important to account for.Source: B. Qualitative and Quantitative… · p. 4
- HighwayI and HighwayII were easiest, while Snellen was hardest; foreground speed or steadiness mattered more than object size.Median pEPs and MS-SSIM were strongly indicative of sequence difficulty.Source: B. Qualitative and Quantitative… · p. 4, Table III · p. 5
V. CONCLUDING REMARKS
The study benchmarks scene background initialization using a common dataset, ground truths, metrics, and reproducible results. It finds that region-based or hybrid selective methods generally perform best, while pEPs and MS-SSIM are strongly indicative metrics.
V. CONCLUDING REMARKS
- The study provides a common groundtruthed dataset, metrics, and reproducible results, with the dataset, ground truths, and metric tool publicly available.Source: §V · p. 5
- Low speed or steadiness of foreground objects in the bootstrap sequence, rather than their size, burdens most methods.Source: §V · p. 5
- Region-based or hybrid selective methods generally achieve the best results, while no common methodological scheme or recursivity is preferred.The study recommends preferring methods that are region-based or hybrid and selective.Source: §V · pp. 5–6
- Among eight selected metrics, pEPs and MS-SSIM are strongly indicative of background initialization performance.The authors note their potential interest for evaluating future methods.Source: §V · p. 6