Source-linked AI summary
Correlation Filters with Limited Boundaries
Hamed Kiani Galoogahi, Terence Sim, Simon Lucey
TL;DR
Correlation filters are computationally efficient but suffer from boundary effects caused by circularly shifted training examples, limiting their representativeness. The paper introduces an objective optimized with an Augmented Lagrangian Method to reduce these effects while retaining frequency-domain efficiency, and reports superior detection and tracking results.
Problem
Circularly shifted training patches are generally unrepresentative because of boundary effects, while spatially removing those effects sacrifices correlation filters’ computational efficiency.
Method
The paper introduces a correlation-filter objective that reduces boundary effects and optimizes it iteratively with an Augmented Lagrangian Method using frequency-domain redundancies.
Results
The method reports superior empirical results for object detection and real-time tracking compared with current state-of-the-art approaches.
Takeaways & Limitations
The approach limits circular boundary effects while preserving many computational advantages of canonical frequency-domain correlation filters.
Takeaways & Limitations
Completely removing boundary effects by summing only unaffected shifts cannot be combined with efficient frequency-domain solution of the augmented objective.
Abstract
from arXiv · showhide
Correlation filters take advantage of specific properties in the Fourier domain allowing them to be estimated efficiently: O(NDlogD) in the frequency domain, versus O(D^3 + ND^2) spatially where D is signal length, and N is the number of signals. Recent extensions to correlation filters, such as MOSSE, have reignited interest of their use in the vision community due to their robustness and attractive computational properties. In this paper we demonstrate, however, that this computational efficiency comes at a cost. Specifically, we demonstrate that only 1/D proportion of shifted examples are unaffected by boundary effects which has a dramatic effect on detection/tracking performance. In this paper, we propose a novel approach to correlation filter estimation that: (i) takes advantage of inherent computational redundancies in the frequency domain, and (ii) dramatically reduces boundary effects. Impressive object tracking and detection results are presented in terms of both accuracy and computational efficiency.
1 INTRODUCTION
Correlation filters offer efficient frequency-domain estimation and have regained interest through robust vision applications, but their circularly shifted training examples create boundary effects. The paper proposes an efficiently optimized objective that reduces these effects while preserving computational advantages and improving detection and tracking results.
- Correlation filters learn frequency-domain templates whose correlation with training signals produces a desired response, typically a peak at the object and suppressed elsewhere.
- MOSSE renewed interest in correlation filters after performing well under rotation, scale, lighting, and partial occlusion while retaining low memory and computation costs.
- 1.1 The Problem: One real-world example is paired with D−1 synthetic circularly shifted examples, whose boundary effects make them unrepresentative of normal shifted image patches.
- 1.1 The Problem: Boundary effects can dramatically affect estimated-template performance, while removing them spatially sacrifices the computational efficiency that motivates correlation filters.
- 1.2 Contribution: The proposed objective reduces boundary-affected examples, and an Augmented Lagrangian Method exploits frequency-domain redundancies to optimize it efficiently.
- 1.2 Contribution: The paper reports impressive object-detection and tracking results that outperform MOSSE and leading non-correlation-filter tracking methods.
2 CORRELATION FILTERS
Correlation filters learn templates in the frequency domain efficiently, but circularly shifted training examples create boundary effects and translation sensitivity. Spatial masking reduces these effects, while frequency-domain reformulation preserves computational efficiency only through additional optimization machinery.
- Correlation filters learn templates by correlating training signals to produce a desired response, typically a peak at the object location.
- O(D^3 + ND^2) spatial optimization becomes intractable with signal length D, whereas frequency-domain estimation costs O(ND log D).
- 2.1 Correlation Filters: Circular shifts make the learned filter highly sensitive to translation and poorly representative of normally encountered image patches.
- 2.2 Boundary Effects: A masking matrix P with T > D enlarges the training signal and increases the proportion of examples unaffected by boundary effects.
- 2.2 Boundary Effects: Summing only T − D + 1 shifts could remove boundary effects completely, but this modification cannot be solved efficiently in the frequency domain.
- 2.3 Efficiency in the Frequency Domain: Fourier-domain correlation uses Hadamard products and spectral energies, retaining the spatial solution while reducing estimation cost to O(ND log D).
3 OUR APPROACH
The proposed approach addresses the spatial constraint through an auxiliary Fourier-domain variable and an Augmented Lagrangian optimization scheme. Its iterative decomposition retains FFT-based efficiency, with total cost O([N + K]T log T).
- 3 OUR APPROACH: The spatially constrained objective has cost O(D^3 + ND^2) because h must be solved in the spatial domain.
- 3.1 Augmented Lagrangian: An auxiliary variable ˆg is introduced to solve the constrained objective while exploiting frequency-domain redundancies.
- 3.1 Augmented Lagrangian: An Augmented Lagrangian Method handles the equality constraints, with ADMM interpreted as Gauss-Seidel optimization over efficiently decomposed subproblems.
- 3.2 Optimization: FFT-based updates estimate ˆh, g, and l efficiently using zero padding, inverse FFTs, and lookup tables implied by the masking matrix.
- 3.3 Computational Complexity: The penalty factor µ is updated during ADMM, and experiments found µ(0) = 10^-2, β = 1.1, and µmax = 20 effective.
- 3.3 Computational Complexity: The iterative algorithm costs O([N + K]T log T), combining O(NT log T) spectral precomputation with FFT-dominated ADMM iterations.
4 EXPERIMENTS
Experiments evaluated the method on eye localization, convergence, and real-time tracking. It outperformed comparison methods in localization and tracking while retaining efficient iterative optimization.
- Localization Performance: Our method outperformed prior correlation filters across all eye-localization thresholds and training-set sizes.Results averaged across 10 random runs.
- Localization Performance: Increasing training-patch size relative to the 32 × 32 filter improved localization, with the lowest performance when T equaled D.Larger training patches reduce the portion affected by boundary effects.
- Runtime Performance: ADMM convergence was largely independent of training-image count and required relatively few iterations for good convergence.Precomputed auto- and cross-spectral energy vectors prevent each iteration from revisiting the training set.
- Tracking Performance: Four ADMM iterations were selected as a tradeoff between tracking performance and computation.The position-error experiment varied the iteration count from 1 to 64.
- Tracking Performance: Our method achieved maximum average tracking precision and minimum average localization error, followed by STRUCK.Tracking evaluation also reported speed in frames per second.
- Tracking Performance: Our method achieved higher precision on all tested videos except Tiger1 and Twinings and suffered less drift in selected sequences.Online adaptation supported robustness across pose, scale, illumination, appearance, and partial occlusion variations.
5 CONCLUSIONS
The paper presents a correlation-filter estimator that limits circular boundary effects while preserving computational advantages, with superior empirical detection and tracking results.
- Conclusions: The proposed estimator limits circular boundary effects while preserving many computational advantages of frequency-domain correlation filters.The method is evaluated for object detection and real-time tracking.
Cliffbar
Tracking results on Cliffbar are presented with predicted positions compared against ground truth under challenging variations.
- Cliffbar: The figure compares ground-truth and predicted target positions while showing the target, trained filter, and correlation output for each frame.The tracking sequence includes challenging variations of pose, scale, illumination, and partial occlusion.