PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking

Tianyang Xu, Zhen-Hua Feng, Xiao-Jun Wu, Josef Kittler

arXiv:1807.11348v3cs.CV

TL;DR

DCF trackers face spatial boundary effects and temporal filter degradation. The paper introduces adaptive spatial feature selection with temporal consistency and unified optimization, and reports superior performance across multiple tracking benchmarks.

  • Problem

    DCF tracking suffers from spatial boundary effects and temporal filter degradation, limiting spatial modeling and historical appearance integration.

  • Method

    The method jointly learns discriminative filters and selects structured spatial features with lasso regularization and temporal consistency on a low-dimensional manifold.

  • Results

    The method outperforms state-of-the-art trackers across benchmark datasets, with LADCF achieving 86.4% DP and 66.4% AUC on OTB100 using hand-crafted features.

  • Takeaways & Limitations

    Adaptive feature selection and temporal consistency support real-time discriminative filter learning with both hand-crafted and deep features.

Abstract

from arXiv · show

With efficient appearance learning models, Discriminative Correlation Filter (DCF) has been proven to be very successful in recent video object tracking benchmarks and competitions. However, the existing DCF paradigm suffers from two major issues, i.e., spatial boundary effect and temporal filter degradation. To mitigate these challenges, we propose a new DCF-based tracking method. The key innovations of the proposed method include adaptive spatial feature selection and temporal consistent constraints, with which the new tracker enables joint spatial-temporal filter learning in a lower dimensional discriminative manifold. More specifically, we apply structured spatial sparsity constraints to multi-channel filers. Consequently, the process of learning spatial filters can be approximated by the lasso regularisation. To encourage temporal consistency, the filter model is restricted to lie around its historical value and updated locally to preserve the global structure in the manifold. Last, a unified optimisation framework is proposed to jointly select temporal consistency preserving spatial features and learn discriminative filters with the augmented Lagrangian method. Qualitative and quantitative evaluations have been conducted on a number of well-known benchmarking datasets such as OTB2013, OTB50, OTB100, Temple-Colour, UAV123 and VOT2018. The experimental results demonstrate the superiority of the proposed method over the state-of-the-art approaches.

I. INTRODUCTION

The introduction identifies spatial boundary effects and temporal filter degradation as major weaknesses of DCF tracking, then presents LADCF as a joint spatial-feature-selection and temporally consistent filter-learning approach.

  • Motivation: DCF tracking is challenged by spatial boundary effects and temporal filter degradation, which reduce training-example quality and modeling effectiveness.Circular shifts create boundary discontinuities, while insufficient historical appearance information weakens the learned filter.
  • Adaptive spatial feature selection: Group lasso adaptively selects spatial features, including stable-motion background regions, to preserve a discriminative manifold and reduce boundary distortion.The approach uses target and surrounding regions rather than activating all spatial features.
  • Temporal consistency: Temporal consistency interacts with spatial feature selection to capture global dynamic appearance information and prevent filter degradation.The proposed online adaptation strategy is designed to preserve temporal smoothness while accommodating appearance variation.
  • Unified optimization: An augmented-Lagrangian framework jointly performs spatial feature selection and discriminative filter learning, using spatial-domain selection and frequency-domain optimization.FFT connects the two domains, and both hand-crafted and deep features can be used.

II. RELATED WORK

The related work reviews generative, discriminative, deep, DCF-based, scale-aware, spatially regularized, and embedded feature-selection trackers, positioning the proposed method within DCF and structured feature selection.

  • Tracking paradigms: Tracking methods include generative models that compare candidates with target appearance and discriminative models that distinguish targets from backgrounds.The review also covers online classifiers, multiple-instance learning, and tracking-learning-detection approaches.
  • Deep tracking: Deep end-to-end trackers use offline learning and online tracking, including regression, verification, and Siamese-network formulations.The cited examples include GOTURN, SINT, SiamFC and CFNet.
  • DCF methods: DCF trackers exploit circulant structure and frequency-domain operations, while later work improves feature representation, memory, kernels and computational efficiency.C-COT and ECO are described as continuous-convolution approaches with efficiency and dimensionality-reduction mechanisms.
  • Scale detection: Scale-aware trackers estimate scale either after tracking from a scale pool or jointly with location during tracking.SAMF, DSST and fDSST exemplify these alternatives.
  • Spatial regularization and feature selection: Spatial regularization methods predefine weights or masks, whereas lasso enables sparse feature activation but basic forms ignore variable structure.The proposed method addresses this setting with embedded temporal consistency-preserving spatial feature selection.

III. TRACKING FORMULATION

The tracking formulation represents candidate patches through circulant shifts, evaluates them with a discriminative correlation filter, then learns and updates the model using regularized objectives.

  • DCF data construction: A base n × n image patch generates a circulant training matrix containing all cyclic shifts and Gaussian-shaped regression labels.The filter learns a discriminative function f(x_i; θ) to distinguish the target from background.
  • Tracking: During tracking, the candidate maximizing the discriminative function is selected from cyclic shifts of the previous-frame target patch.The circulant structure enables efficient calculation in the frequency domain.
  • Learning: After tracking, the model is learned by minimizing a regularized loss over labeled samples generated from the current-frame target-centered patch.Traditional DCF uses quadratic loss with an ℓ2-norm penalty, forming a ridge-regression problem.
  • Updating: Incremental updating combines current and historical model information, with α controlling their trade-off.This update addresses potential target-appearance variation within the DCF framework.

IV. THE PROPOSED LADCF ALGORITHM

The LADCF appearance model jointly selects spatial features and learns filters while enforcing temporal consistency around a historical template. This preserves a compact discriminative manifold and supports locally changing spatial selections.

  • Temporal Consistency Preserving Spatial Feature Selection Model: The method jointly performs adaptive spatial feature selection and discriminative filter learning through a spatially structured appearance formulation.The selected features preserve the discriminative manifold structure of target and background variation.
  • Temporal Consistency Preserving Spatial Feature Selection Model: An indicator vector φ activates or disables filter elements, reducing dimensionality while preserving spatial structure across training samples.The selected spatial features are shared by the input and filter representations.
  • Temporal Consistency Preserving Spatial Feature Selection Model: The nonconvex ℓ0 sparsity constraint is approximated with ℓ1 regularisation, while temporal consistency restricts successive estimates to local changes around the current template.This constraint keeps selected spatial features on a low-dimensional manifold across frames.
  • Temporal Consistency Preserving Spatial Feature Selection Model: The relaxed temporal term uses λ2 to keep the learned filter close to the sparse template, with larger λ2 imposing stronger proximity.The ℓ2-norm ball around the template guides estimates toward the low-dimensional manifold.
  • Temporal Consistency Preserving Spatial Feature Selection Model: The resulting model preserves both dynamic and static appearance while adaptively selecting discriminative spatial configurations.Target and background regions can be activated together to form a robust pattern.

B. Generalising to Multi-channel Features

LADCF extends spatial feature selection from single-channel filters to multi-channel representations using a shared spatial layout and structured sparsity. Group sparsity jointly evaluates spatial locations across channels and can adaptively weight channel contributions.

  • Generalising to Multi-channel Features: Multi-channel features share a common spatial layout, so the method applies one feature-selection index across all channels.The input is represented as X = {x1, x2, ..., xL} with corresponding filters θ = {θ1, θ2, ..., θL}.
  • Generalising to Multi-channel Features: Structured sparsity computes an ℓ2-norm across channels at each spatial location and applies an ℓ1-norm to induce joint sparsity.This extends the single-channel objective to multi-channel features.
  • Generalising to Multi-channel Features: Joint sparsity selects spatial locations according to their combined contribution across all feature channels.The selection reflects feature-map contributions at the same spatial position rather than treating channels independently.
  • Generalising to Multi-channel Features: Structured sparsity can adaptively weight different channels while unifying them during feature selection.The weighting is incorporated into the spatial selection process.

C. Optimisation

The optimisation reformulates the constrained multi-channel objective with slack variables and augmented Lagrange multipliers. ADMM alternates convex subproblem updates, using frequency-domain DCF computation for filters and closed-form spatial updates.

  • Optimisation: Slack variables convert the constrained objective into a convex optimisation formulation suitable for augmented-Lagrangian treatment.The equality constraint is incorporated through Lagrange multipliers and a penalty parameter.
  • Optimisation: ADMM iteratively optimises the filter, auxiliary filter, and multiplier-related subproblems with guaranteed convergence for the convex Lagrangian.The penalty parameter controls convergence and is varied during iterations.
  • Optimisation: The filter update exploits circulant structure and Parseval’s formula to solve channel-wise convex problems in the frequency domain.The resulting solution for the transformed filter has a closed form with element-wise division.
  • Optimisation: The auxiliary update shrinks each spatial feature collaboratively across channels, integrating the structured constraints into the solution.The update can be separated by spatial feature and admits a closed-form optimal solution.
  • Optimisation: The penalty parameter is updated as μ = min(ρμ, μmax), while iterations are capped at K in practical implementation.ρ > 1 accelerates the penalty and μmax prevents excessive values.

D. Tracking Framework

The LADCF tracking framework combines multi-scale detection with temporal-consistency-preserving spatial feature selection and iterative filter learning. It extracts multi-channel features, scores candidate scales in the frequency domain, then updates the filter model.

  • Tracking Framework: LADCF performs simultaneous target position and scale detection using search windows extracted at multiple scales.The search windows are centred around the previous target position.
  • Tracking Framework: The algorithm tracks by extracting scaled search-window features, computing response scores, and selecting the position and scale with the maximum response.The selected parameters define the current target bounding box.
  • Tracking Framework: The filter model is updated after learning, and the updated model is used for tracking in the next frame.Algorithm 1 outputs both the current target bounding box and updated filter model.
  • Tracking Framework: During learning, the padded target patch supplies multi-channel features for temporal-consistency-preserving spatial feature selection and filter optimisation.The filter is optimised for K iterations to control the selected spatial features.
  • Tracking Framework: The proposed feature representations are documented in a dedicated feature-representation table.The supplied framework identifies this representation table but does not specify its row or column contents.

V. PERFORMANCE EVALUATION

The evaluation section describes qualitative and quantitative experiments validating LADCF across benchmark datasets, metrics, and comparisons with state-of-the-art trackers.

  • LADCF is evaluated qualitatively and quantitatively using benchmark datasets, evaluation metrics, and state-of-the-art trackers.The section also analyzes results across datasets and discusses the method's advantages.

A. Implementation Details

The implementation uses hand-crafted and deep features, compares LADCF variants under matched feature settings, and reports evaluations through figures and tables.

  • LADCF uses hand-crafted features, while LADCF∗ combines hand-crafted and deep features for comparisons with deep-feature trackers.The two variants are designed to support fair comparisons across tracker groups.
  • Figure 3 compares LADCF with state-of-the-art trackers on OTB100 using precision and success plots for hand-crafted and deep features.
  • Tables II and III compare hand-crafted and deep-feature trackers on OTB2013, OTB50, and OTB100 using overlap precision.
  • Figure 4 reports success plots for OTB2013, OTB50, Temple-Colour, and UAV123, with AUC scores shown in the legend.
  • The implementation runs in MATLAB 2016a on an Intel i5 CPU and NVIDIA GeForce GTX 960M GPU.

B. Experimental Setup

The experiments cover six tracking benchmarks, use OPE metrics based on location error and overlap, and analyze OTB100 across 11 sequence attributes.

  • The evaluation uses OTB2013, OTB50, OTB100, Temple-Colour, UAV123, and VOT2018.The datasets contain annotated sequences spanning general, colour, aerial, and short-term tracking settings.
  • One Pass Evaluation reports AUC, overlap precision, distance precision, and tracking speed from success, overlap, precision, and frame-rate measurements.
  • Figure 5 analyzes OTB100 success plots across 11 attributes, showing only the top 10 trackers for each attribute.
  • Table IV reports tracking results on VOT2018, highlighting the three best results.
  • The comparison includes 13 state-of-the-art trackers, with starred trackers equipped with deep features or structures.

C. Results and Analysis

LADCF and LADCF∗ outperform many state-of-the-art trackers across datasets, feature settings, and sequence attributes, while performance is comparatively weaker on long UAV123 sequences.

  • Overall tracking performance: 86.4% DP and 66.4% AUC make LADCF the best hand-crafted-feature tracker on OTB100; LADCF∗ reaches 69.6% AUC with deep features.LADCF∗ also achieves 90.6% DP, ranking second and trailing ECO∗ by 0.4%.
  • Overall tracking performance: LADCF gains 1%, 4.1%, and 3.3% OP over the second-best hand-crafted trackers on OTB2013, OTB50, and OTB100.
  • Overall tracking performance: LADCF∗ exceeds ECO∗ by 2%, 1.5%, and 1.8% OP on OTB2013, OTB50, and OTB100, respectively.
  • Cross-dataset results: LADCF beats all hand-crafted trackers across OTB2013, OTB50, Temple-Colour, and UAV123 success plots.LADCF∗ is better than ECO∗ and C-COT∗ except on UAV123, while reaching 60.6% AUC on Temple-Colour.
  • Limitations: ECO∗ leads LADCF∗ by 1% on UAV123, possibly because its historical-sample clusters better support the longer sequences.UAV123 averages 915 frames per sequence, compared with 578, 591, and 429 for OTB2013, OTB50, and Temple-Colour.
  • Sequence-attribute analysis: LADCF∗ outperforms other trackers on five OTB100 attributes and ranks among the top three on the remaining six.LADCF achieves the best performance on nine attributes among hand-crafted-feature trackers.

D. Self Analysis

The self-analysis examines feature configurations, temporal-consistency sensitivity, and feature-selection ratios, while benchmark results support the method’s effectiveness and robustness.

  • Feature configurations: Combining Colour-Names with HOG improves LADCF’s OTB100 AUC by 2.1%.
  • Feature configurations: Middle convolutional layers Conv-3 and Conv-4 significantly outperform low and high convolutional layers.
  • Temporal consistency: Tracking results vary smoothly across learning rates, indicating stable performance under the proposed temporal-consistency formulation.
  • Feature selection: Hand-crafted and deep features achieve improvements with selection ratios of 2% ∼20% and 3% ∼40%, respectively.
  • Conclusion: The proposed approach combines temporal consistency and embedded spatial feature selection for adaptive discriminative filter learning on a low-dimensional manifold.
Loading 1807.11348v3…