PaperlayerView exact revision on arXiv

Source-linked AI summary

Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset

Thierry Bouwmans, Andrews Sobral, Sajid Javed, Soon Ki Jung, El-Hadi Zahzah

arXiv:1511.01245v3cs.CV

TL;DR

Background/foreground separation lacks a rigorous large-scale quantitative evaluation spanning robust subspace formulations and real-world video challenges. This paper reviews these formulations through the DLAM framework, examines their optimization and implementation properties, and evaluates 32 methods on BMC 2012, finding that DLAM provides a suitable framework and can outperform MOG and KDE in detection state.

  • Problem

    Existing foreground-detection research lacks rigorous quantitative evaluation on large-scale datasets with accurate ground truth covering real-world video challenges.

  • Method

    The paper reviews RPCA, RNMF, RMC, RSR, RST, and RLRM under the unified DLAM view, examining decompositions, losses, optimization, solvers, incremental algorithms, and real-time implementations.

  • Results

    DLAM models provide a suitable framework for background/foreground separation and, in their fundamental version, outperform MOG and KDE in detection state.

  • Takeaways & Limitations

    Structured spatial and temporal constraints can improve robustness to illumination changes and dynamic backgrounds, while specialized robust methods can achieve performance comparable to SubSENSE.

  • Takeaways & Limitations

    Sparse decompositions unified under DSAM are explicitly outside this paper's scope.

Abstract

from arXiv · show

Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate moving objects from the background. The most representative problem formulation is the Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit (PCP) which decomposes a data matrix in a low-rank matrix and a sparse matrix. However, similar robust implicit or explicit decompositions can be made in the following problem formulations: Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal of these similar problem formulations is to obtain explicitly or implicitly a decomposition into low-rank matrix plus additive matrices. In this context, this work aims to initiate a rigorous and comprehensive review of the similar problem formulations in robust subspace learning and tracking based on decomposition into low-rank plus additive matrices for testing and ranking existing algorithms for background/foreground separation. For this, we first provide a preliminary review of the recent developments in the different problem formulations which allows us to define a unified view that we called Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine carefully each method in each robust subspace learning/tracking frameworks with their decomposition, their loss functions, their optimization problem and their solvers. Furthermore, we investigate if incremental algorithms and real-time implementations can be achieved for background/foreground separation. Finally, experimental results on a large-scale dataset called Background Models Challenge (BMC 2012) show the comparative performance of 32 different robust subspace learning/tracking methods.

1 Introduction

The paper reviews robust subspace formulations that decompose video data into low-rank background and additive foreground or corruption components, unifying them as DLAM. It evaluates their suitability for background/foreground separation, including robustness, spatial and temporal constraints, incremental processing, real-time implementation, and performance on BMC 2012.

  • Background and motivation: Foreground detection separates moving objects from static background, motivating subspace-learning models for video analysis.PCA-based background modeling established an earlier subspace-learning approach for this task.
  • Unified decomposition framework: RPCA models the observed matrix A as A = L+S, with low-rank L representing background and sparse S representing outliers or foreground.Related formulations include RNMF, RMC, RSR, RST, and RLRM, with differences in constraints and decomposition structure.
  • Unified decomposition framework: The review unifies robust low-rank-plus-additive decompositions as Decomposition into Low-rank plus Additive Matrices (DLAM).Its comparison tables organize decompositions, minimization objectives, constraints, convexity, loss functions, and regularization choices.
  • Video-specific constraints: Spatial coherence can be modeled with mixed, structured, group, or dynamic-tree sparsity, while temporal coherence is treated as an additional design requirement.These constraints are intended to address properties of foreground objects and evolving video sequences.
  • Evaluation and limitations: The survey compares 32 robust subspace learning and tracking methods on the BMC 2012 dataset and identifies methods addressing many background-separation requirements.MAGRPCA, SLMC, and FRMC address many background-initialization requirements, while SMD-RPCA, SRPCA, and incPCP-TI address background/foreground separation requirements.
  • Evaluation and limitations: No reviewed algorithm simultaneously addresses all key challenges of real-world videos, including changing backgrounds, illumination variation, noise, and computational demands.The paper attributes the lack of a clear leading algorithm partly to limited rigorous quantitative evaluation on large-scale datasets with accurate ground truth.

2 Robust Principal Component Analysis

RPCA methods decompose video data into low-rank background and sparse or structured foreground components, while later variants target scalability, changing subspaces, noise, and application-specific constraints.

  • RPCA via PCP: Principal Component Pursuit perfectly recovers low-rank and sparse matrices under minimal assumptions.
  • Algorithms for solving PCP: AltProj has complexity O(r^2mnlog(1/ε)), runs around 19 times faster than IALM, and produces visually better background extraction.
  • Incremental PCP: ReProCS-cPCA can exactly recover the support of the sparse component at all times with high probability under mild assumptions.
  • Algorithms for solving PCP: Large-matrix singular value decomposition remains a major computational bottleneck, motivating factorized, thresholding, and other SVD-reducing solvers.Partial SVD reduces complexity to O(r × m × n), but this remains high for large datasets.
  • Real-time implementations: GPU CAQR achieved a 30× speedup, while PCP was demonstrated as suitable for real-time surveillance separation with corresponding hardware.
  • Modified PCP formulations: Modified and structured RPCA variants address changing background subspaces, block-structured foregrounds, nonconvex rank approximations, and spatial image information.Reported variants include block-sparse RPCA, capped-norm formulations, p,q-PCP, robust 2D-PCA, and low-rank-plus-dual models.

2.2 RPCA via Stable Principal Component Pursuit

Stable PCP extends PCP to account for entry-wise noise, while its variants modify the decomposition or optimization to improve robustness, scalability, or practical video separation.

  • Stable Principal Component Pursuit: Stable PCP models observations with entry-wise noise and seeks accurate recovery of the low-rank and sparse components.The noise term is bounded in Frobenius norm, and recovery is formulated as a relaxed version of PCP.
  • Algorithms for solving SPCP: ASALM, VASALM, and PSALM exploit separable objective and constraint structures to solve the stable PCP optimization efficiently.These methods split the augmented Lagrangian into subproblems.
  • Modified-SPCP: GreBsmo accelerates noisy robust PCA, running 30-100 times faster than most existing algorithms such as IALM.It replaces the low-rank matrix with a bilateral factorization and uses l1 regularization for soft-thresholding sparse entries.
  • Modified-SPCP: The nuclear-norm-free model extracts background more accurately than original SPCP while requiring only one parameter to tune.Its blurred extension extracts background more robustly than original SPCP with the blur option.
  • Modified-SPCP: SPCP variants also include dictionary-based decompositions, robust penalties, three-term turbulence models, and scalable real-time factorization frameworks.The reviewed approaches cover alternating minimization, quasi-Newton optimization, and divide-factor-combine implementations.

2.3 RPCA via Quantization based Principal Component Pursuit

Quantization-based PCP incorporates pixel quantization error by relaxing exact equality between the observed matrix and its low-rank-plus-sparse decomposition.

  • Quantization-based PCP: QPCP replaces the exact decomposition constraint with an l∞-norm bound that captures pixel quantization error.Pixel values are quantized between 0 and 255, while the underlying real observations lie in the continuous interval [0,255].
  • Quantization-based PCP: The inequality constraint permits RPCA to operate on real-valued observations rather than requiring a clean split of quantized observations.This formulation is designed to account for quantization error in each pixel value.
  • Algorithms for solving QPCP: TFOCS solves QPCP by converting the problem to a conic formulation, deriving its dual, smoothing it, and applying an optimal first-order method.The approach is also applied to compressed sensing.

2.4 RPCA via Block based Principal Component Pursuit

Block-based PCP replaces entry-wise sparsity with block sparsity to model low-rank structure alongside grouped outliers.

  • Block-based PCP: BPCP decomposes A into a low-rank matrix L and a block-sparse matrix S.The optimization uses the nuclear norm for low-rankness and the l2,1-norm for block sparsity.
  • Block-based PCP: The l2,1-norm is the l1-norm of columnwise l2 norms, encouraging grouped nonzero columns in the outlier matrix.An additional term encourages exact zero columns in the recovered low-rank matrix at outlier locations.
  • Block-based PCP: RPCA-LBD produced better background/foreground results than original RPCA-PCP, but assumes that outliers occupy mostly zero columns.That assumption is unsuitable when columns correspond to foreground objects across video frames.
  • Algorithms for solving BPCP: Tang and Nehorai solve the BPCP convex problem with an Augmented Lagrangian Method.The algorithm decomposes A into low-rank and block-sparse matrices under the specified norm terms.

2.5 RPCA via Local Principal Component Pursuit

Local PCP broadens robust decomposition beyond a single low-dimensional subspace, while related formulations address column outliers, sparsity control, identifiability, and dense corruption.

  • Local Principal Component Pursuit: PCP is unsuitable when data are not well modeled by a single low-dimensional subspace, motivating a union-of-subspaces decomposition.Wohlberg et al. replace explicit low-rank modeling with self-representability and sparse deviations.
  • Local Principal Component Pursuit: The local PCP formulation uses sparse self-representation, row sparsity in U, and entrywise sparsity in S to model locally low-dimensional structure and deviations.A data-derived dictionary D produces the local approximation DU of A − S.
  • Algorithms for solving LPCP: Split Bregman solves local PCP by alternating among five subproblems, including linear-system solves and additional subproblems for the nonsmooth terms.The method introduces Bregman variables after relaxing equality constraints with auxiliary variables.
  • Related robust PCA formulations: Outlier Pursuit targets column-corrupted data and recovers the uncorrupted matrix’s column space rather than the exact matrix.Its stable variant extends the model to A = L + S + E.
  • Sparse Corruption: Identifiable sparse-corruption recovery requires S not to be too dense in rows or columns and L’s singular vectors not to be too sparse.These conditions are used to guarantee recovery.
  • Log-sum Heuristic Recovery: Log-sum Heuristic Recovery uses a nonconvex log-sum penalty and can remove denser errors than PCP.Its majorization-minimization solver alternates between constructing a convex upper bound and minimizing it.

2.10 RPCA via Iteratively Reweighted Least Squares Minimization

This section reviews RPCA variants that model background and foreground through low-rank and sparse components, including iterative, online, Bayesian, and factorized formulations. The methods differ in optimization strategy, structural priors, and computational suitability for video processing.

  • IRLS-based RPCA: IRLS models the background with a low-rank matrix plus noise and reconstructs it through a factorized representation, while S captures moving objects.The formulation uses weighted fitting and a smoothed error function to emphasize guessed background regions.
  • IRLS-based RPCA: Local penalties and the l2,1 mixed norm encourage spatially homogeneous sparse components, while the first part tracks a dominant 1-Rank decomposition.These choices target coherent object regions rather than isolated pixel outliers.
  • Online RPCA: OR-PCA performs stochastic optimization one sample per time instance, updating a low-dimensional basis and coefficients incrementally after each frame.Unlike batch PCP schemes, this design supports real-time processing without batch optimization.
  • Online RPCA: MRF-constrained OR-PCA substantially improves foreground segmentation, especially for highly dynamic backgrounds, with parameters tuned to different scenarios.The reported work provides a large set of experimental results and recommended parameter ranges.
  • Bayesian RPCA: Bayesian RPCA variants infer low-rank, sparse, and noise components using probabilistic priors, including Markov dependencies across space and time.Factorized variational Bayesian RPCA performs slightly better than IALM-based RPCA, VBRPCA, and GoDec in reported experiments.
  • Bayesian RPCA: FVBRPCA performs slightly better than RPCA solved via IALM, VBRPCA, and GoDec in the reported experimental comparison.The method estimates posterior expectations of the latent factors and noise-related variables.

2.13 Approximated RPCA

Approximated RPCA methods estimate low-rank and sparse components through alternating nonconvex updates rather than exact convex optimization. GoDec and its variants reduce computational cost through thresholding and randomized approximations, including incremental processing for new video frames.

  • GoDec: GoDec alternates low-rank approximation of A − S and sparse approximation of A − L to minimize decomposition error.The method estimates both components using the same low-rank-plus-sparse decomposition structure as SPCP.
  • GoDec: GoDec updates L by singular-value hard thresholding and S by entry-wise hard thresholding, with global solutions for both subproblems.Although the constraints are nonconvex, each alternating subproblem has an available global solution.
  • GoDec: Bilateral Random Projection replaces the SVD in GoDec to significantly reduce computation time.The SVD is the main computational cost in the low-rank update sequence.
  • Semi-Soft GoDec: Semi-Soft GoDec uses soft thresholding for S, automatically determining sparsity and reducing leakage of noise into the sparse component.The soft threshold τ replaces manual selection of the cardinality parameter k.
  • Incremental processing: Incremental low-rank and sparse decomposition extends Semi-Soft GoDec to process new matrix columns with a given low-rank structure.This extension targets low-rank and sparse decomposition for video coding and background/foreground separation.

2.14 Sparse Additive Matrix Factorization (SAMF)

SAMF generalizes RPCA by allowing multiple structured sparse components, including row-wise and column-wise sparsity. The reviewed applications report improved background/foreground separation for segmented imagery and transformed noisy or blurred videos.

  • SAMF formulation: SAMF extends RPCA beyond element-wise and low-rank sparsity to handle multiple types of sparse noise, including row-wise and column-wise sparsity.The number of sparse matrices K determines the additive sparse structure; original RPCA uses K = 2.
  • SAMF formulation: For background/foreground separation, SAMF assigns the low-rank term to static background and the element-wise sparse term to moving foreground.The method uses the assumption that image regions with similar intensities tend to belong to the same object.
  • SAMF results: Image-segmentation-based sSAMF outperforms PCP via IALM on the CAVIAR dataset.The comparison identifies PCP via IALM as the LE-SAMF baseline.
  • SAMF optimization: SAMF inference reduces the partial problem to standard matrix factorization and uses variational Bayesian mean updates for iterative estimation.The mean-update procedure gives a global optimum for a large subset of parameters at each step.
  • VBSE: VBSE applies a known linear transformation to outlier-corrupted data and estimates sparse outlier amplitudes together with the low-rank component.The formulation is designed for background/foreground separation in blurred and noisy video sequences.
  • VBSE results: VBSE outperforms PCP via APG and PCA via IALM on the CAVIAR dataset.The method uses approximate Bayesian inference with hierarchical priors for the model components.

3 Robust Non-negative Matrix Factorization

Robust NMF methods impose non-negativity while separating low-rank structure from outliers through robust losses, separability assumptions, or grouped-outlier models. The reviewed results report advantages for MahNMF, RobustXray, and RANMF on heavy-tailed or contaminated video-related data.

  • MahNMF: MahNMF minimizes Manhattan distance between a non-negative matrix and its non-negative low-rank factorization to model heavy-tailed Laplacian noise.Its residual estimates the sparse component while the factor product estimates the non-negative low-rank component.
  • MahNMF: MahNMF uses RRI and Nesterov smoothing algorithms, but RRI is neither scalable to large matrices nor flexible enough for all optimization settings.The RRI method updates factors in closed form by approximating residuals with rank-one outer products.
  • Near-separable NMF: Under separability, anchor columns from the data generate the cone containing the factorization, enabling efficient and exact NMF recovery.The anchor columns correspond to columns of the basis matrix and form an identity submatrix in the right factor.
  • RobustXray: RobustXray identifies one anchor column per iteration and projects all data points onto the expanding cone using l1 minimization.The procedure is parallelizable across at most r iterations and avoids normalization or preprocessing requirements described for other methods.
  • RobustXray: RobustXray outperforms robust NMF local search, robust low-rank local search, and PCP via IALM on the I2R dataset.The method replaces Frobenius-norm approximations with robust l1-based factorization to address outliers and varied noise.
  • RANMF: RANMF controls factor denseness through asymmetric constraints while separating a low nonnegative-rank matrix from grouped outliers.Its objective additionally uses sparsity enforcement, rank constraints, and total variation for foreground denoising and segmentation.
  • RANMF: RANMF outperforms PCP via IALM and DECOLOR on the I2R dataset.The reported model treats the grouped-outlier component as distinct from the final foreground mask and applies total variation to obtain that mask.

4 Robust Matrix Completion

Robust matrix completion recovers low-rank and sparse components from incomplete or corrupted observations, with variants incorporating nonconvex losses, matrix factorization, motion information, and noise modeling. Reported results emphasize improved recovery details, accuracy, speed, and robustness for background/foreground separation.

  • RMC differs from RPCA because the support of missing entries is known, whereas corrupted entries in RPCA are unknown.
  • RMC-lσ uses a nonconvex loss for contaminated matrix-completion entries and recovers background details better than PCP solved via IALM.RMC-lσ is reported to be more than 3 times faster, although neither method recovers missing foreground entries.
  • RMC jointly recovers low-rank and sparse components from incomplete and grossly corrupted observations using projected constraints on observed entries.
  • RMC-MF factorizes the low-rank matrix into smaller matrices, making its solver more than 7 times faster than convex RMC and more accurate in low-rank recovery.It is also reported as more than 4 times faster than GRASTA and more than 2 times faster than UNN-BF.
  • FRMC uses a nuclear-norm surrogate and supports batch and online operation for videos with moving backgrounds.Its online mode completes recovery 7 to 9 times faster than GRASTA with comparable separation quality.
  • Motion-assisted matrix completion weights observations using dense motion fields, while robust MAMC additionally models noise and reportedly outperforms RPCA solved via IALM.The weighting suppresses slowly moving objects and improves recovery opportunities for backgrounds appearing in few frames.

5 Robust Subspace Recovery

Robust subspace recovery methods extend low-rank-plus-sparse decomposition to structured subspaces, orthonormal coefficient sparsity, and orthogonal outliers. The reviewed methods report foreground separation across camera settings and improved recovery efficiency against several PCP baselines.

  • RoSuRe: RoSuRe decomposes data into a low-rank union-of-subspaces component and sparse errors, while its sparse coefficient matrix records relations among data points.These relations can support clustering images with similar backgrounds into individual subspaces.
  • RoSuRe: RoSuRe separates foreground from background with static and moving cameras, including periodic camera motion through block-diagonal coefficient structure.
  • ROSL: ROSL represents the low-rank matrix as Dα and bounds its rank through the number of nonzero coefficient rows, while separately promoting sparse errors.
  • ROSL: ROSL replaces row-wise l0 sparsity with a row-wise l1 surrogate and solves the resulting nonconvex problem under orthonormality constraints.
  • ROSL: ROSL is slightly more accurate and efficient than four PCP variants on I2R and more than 10 times faster than RPCA solved via IALM.
  • ROC-PCA: ROC-PCA targets orthogonal outliers by projecting observations onto the orthogonal complement of the principal subspace and decomposing projected data into mean, outlier, and noise terms.It uses row-wise outlier penalties and alternating optimization on the Stiefel manifold.

6 Robust Subspace Tracking

Robust subspace tracking processes streaming observations without storing all input data, modeling evolving low-rank subspaces alongside sparse outliers, noise, transformations, or structured perturbations. The reviewed trackers trade robustness, computational efficiency, and adaptability for changing backgrounds, motion, and outlier structure.

  • Subspace tracking processes and discards progressively arriving observations, enabling online learning when the full data history cannot fit in memory.
  • GRASTA: GRASTA estimates nonstationary subspaces from streaming vectors corrupted by outliers using an l1 cost and Grassmannian augmented-Lagrangian optimization.Its sparse outlier vector models foreground pixels, while subsampling uses only observed indices.
  • GRASTA: GRASTA is more robust than RPCA solved via IALM and ReProCS on I2R while requiring less time.
  • t-GRASTA: t-GRASTA adds image transformations to jointly estimate low-rank structure, sparse occlusions or foreground, and rotation or translation in batch and online modes.
  • GASG21: GASG21 uses l2,1-norm minimization for column outliers and slightly outperforms OP with less computation time.
  • pROST: pROST replaces GRASTA’s l1-based approach with a smoothed weighted Lp quasi-norm and uses spatio-temporal pixel-label dependencies for online tracking.It directly optimizes the cost function with fewer than half as many unknowns as GRASTA.
  • pROST: pROST handles bootstrapping, large foreground objects, jittery cameras, and higher-rank underlying matrices, outperforming GRASTA for multi-modal backgrounds.
  • FARST: FARST uses recursive least squares for subspace tracking, making it adapted to background change and dynamics while separating streaming frames into background and foreground.

7 Robust Low Rank Minimization

Robust low-rank minimization methods extend low-rank modeling with explicit outlier, noise, probabilistic, spatial, or factorization structures. The reviewed methods show varied accuracy and computational trade-offs across background/foreground separation experiments.

  • DECOLOR: DECOLOR estimates low-rank background structure and contiguous foreground support using a Markov Random Field regularizer.Its optimization alternates convex background estimation with combinatorial foreground-support estimation via SOFT-IMPUTE and graph cuts.
  • DECOLOR: DECOLOR outperforms PCP via IALM, the mean model, and MOG on the I2R dataset, but has prohibitive computation time.
  • Robust matrix factorization: DRMF excludes sufficiently sparse outliers from low-rank estimation, while DRMF-R counts structured outlier rows using the l2,0-norm.DRMF uses block coordinate descent and partial SVD for its alternating updates.
  • Probabilistic factorization: PRMF uses an l1-norm loss with l2 regularization and an expectation-maximization algorithm, achieving visual results similar to several baselines with less computation time.The corresponding implementation is available in batch and online modes.
  • Bayesian factorization: MBRMF models spatial or temporal outlier proximity and outperforms PCP via IALM, BRPCA, VBRPCA, DECOLOR, and PRMF on SABS.
  • Unified factorization: UNNBF unifies bilinear factorization and nuclear-norm regularization, outperforming PCP via IALM, GRASTA, and PRMF with less computation time on I2R.

8 Experimental Evaluation

The evaluation uses the BMC dataset to compare 32 decomposition-based algorithms under synthetic and real video conditions. Results show that OR-PCA variants, particularly with MRF, achieve the strongest F-Measure performance, while different methods lead recall and precision.

  • Challenges: The evaluation considers camera, illumination, background, noise, motion, and foreground-related challenges that violate ideal subtraction conditions.Listed issues include camera jitter, automatic adjustments, illumination changes, bootstrapping, camouflage, shadows, and dynamic backgrounds.
  • Dataset: The BMC dataset combines synthetic and real videos for rigorous background-subtraction comparison under challenging surveillance conditions.Synthetic sequences include fog, sun, and acquisition noise; real videos include long-term illumination changes and difficult outdoor settings.
  • Metrics: Performance is measured from TP, TN, FP, and FN using detection rate, specificity, error rates, wrong classifications, precision, recall, and F-Measure.F-Measure is maximized toward one, while good performance requires high detection rate without altering precision.
  • Synthetic Videos: On synthetic videos, FAM, MahNMF, ROSL, and DECOLOR achieve recall above 0.9, while OR-PCA with MRF achieves the highest F-Measure.Lag-SCP-QN, OR-PCA without MRF, DECOLOR, and R2PCP follow OR-PCA with MRF in F-Measure.
  • Real Videos: On real videos, OR-PCA with MRF again achieves the best F-Measure, followed by OR-PCA without MRF, ROSL, PRMF, and DRMF.OR-PCA with and without MRF maintain stable performance across synthetic and real videos, whereas ROSL, PRMF, and DRMF are more robust on real videos.
  • Overall comparison: Most RPCA algorithms outperform adaptive MOG on both synthetic and real videos.

9 Implementation and Computational Cost

The implementation study examines runtime and singular-value-decomposition costs across algorithm families. Partial SVD substantially reduces computation for extremely low-rank problems, but runtime rankings vary with image size and resource requirements.

  • Evaluation setup: The study implements algorithms in Matlab through LRSLibrary and measures SVD usage and CPU time for each sequence.
  • RPCA runtime: Full-SVD methods EALM and ADM are computationally expensive, whereas partial SVD makes IALM, LADMAP, LSADM, and LADM substantially faster.Partial SVD is effective because these problems have extremely low ranks and require only a few leading singular values.
  • Runtime comparison: GoDec requires the least computation among the compared methods in the reported runtime analysis.
  • Other categories: GOSUS requires less computation time than GRASTA and pROST, while DRMF is the best-performing method within the PRMF category.
  • Computational boundary: Runtime rankings change with image size because algorithms differ in complexity and memory requirements, and some are suitable only for low-resolution inputs.

10 Experimental Results on I2R dataset and CD.net dataset

Additional evaluations on I2R and ChangeDetection.net compare robust DLAM methods with decomposition and background-subtraction baselines. Methods incorporating spatial and temporal constraints show strong reported performance on dynamic scenarios.

  • I2R dataset: The I2R dataset contains nine 176×144 video sequences with dynamic backgrounds, illumination changes, and bootstrapping issues.
  • I2R results: On I2R, RFDSA and MODSM outperform RPCA-PCP via EALM, GRASTA, DECOLOR, MOG, and SOBS.These are described as robust DLAM models with spatial and temporal constraints.
  • ChangeDetection.net dataset: ChangeDetection.net contains nearly 90,000 frames in 31 sequences spanning six categories and color and thermal-IR modalities.
  • ChangeDetection.net results: On dynamic ChangeDetection.net scenarios, SRPCA with graph-based spatial and temporal constraints outperforms GoDec, GRASTA, DECOLOR, TVRPCA, MAMC, SLMC, and SOBS.

11 Conclusion

The review unifies robust low-rank-plus-additive formulations as DLAM, evaluates their suitability for background/foreground separation, and compares recent methods on BMC. DLAM methods can perform strongly, but original batch formulations remain computationally expensive and often discard spatial and temporal image structure.

  • Review scope: The paper proposes DLAM as a unified view of robust PCA, RNMF, RMC, RSR, RST, and robust low-rank minimization formulations.It evaluates their decompositions, solvers, incremental algorithms, and real-time implementations for background/foreground separation.
  • Comparative evaluation: 32 robust subspace learning and tracking methods are comparatively evaluated on the Background Models Challenge dataset.
  • Supported performance: DLAM provides a suitable framework for background/foreground separation and can outperform state-of-the-art MOG and KDE detection models.Structured norms, group sparsity, dynamic tree sparsity, or MRFs can improve robustness to illumination changes and dynamic backgrounds.
  • Supported performance: Specialized robust background-separation methods achieve performance largely comparable to SubSENSE and better than SOBS, ViBe, and PBAS.
  • Limitations and future work: Original DLAM formulations often use batch algorithms whose computation time is too expensive for real-time requirements.Incremental and real-time variants have therefore been pursued.
  • Limitations and future work: Original DLAM models often lose spatial and temporal constraints when images are converted into vectors.The review points to MRFs, structured norms, or two-dimensional formulations as ways to preserve image structure and reduce computation.
  • Limitations and future work: Future work includes less computationally intensive SVD methods, DLAM algorithms that are both incremental and real-time, and tensor extensions.
Loading 1511.01245v3…