PaperlayerView exact revision on arXiv

Source-linked AI summary

Fast Tracking via Spatio-Temporal Context Learning

Kaihua Zhang, Lei Zhang, Ming-Hsuan Yang, David Zhang

arXiv:1311.1939v1cs.CV

TL;DR

Visual tracking requires robustness to changing target and background appearance without sacrificing efficiency. The paper introduces a Bayesian spatio-temporal context tracker using FFT-based learning and detection, and reports 350 frames per second in MATLAB with favorable accuracy, efficiency, and robustness.

  • Problem

    Visual tracking remains challenging because target and background appearance can change substantially while effective appearance models often increase computational cost.

  • Method

    The tracker models spatial and temporal relationships between the target and local context, then uses FFT-based learning and detection to maximize a confidence map.

  • Results

    The MATLAB implementation runs at 350 frames per second and performs favorably against state-of-the-art methods in accuracy, efficiency, and robustness.

  • Takeaways & Limitations

    Spatio-temporal local context provides the basis for a fast and robust visual tracking algorithm.

Abstract

from arXiv · show

In this paper, we present a simple yet fast and robust algorithm which exploits the spatio-temporal context for visual tracking. Our approach formulates the spatio-temporal relationships between the object of interest and its local context based on a Bayesian framework, which models the statistical correlation between the low-level features (i.e., image intensity and position) from the target and its surrounding regions. The tracking problem is posed by computing a confidence map, and obtaining the best target location by maximizing an object location likelihood function. The Fast Fourier Transform is adopted for fast learning and detection in this work. Implemented in MATLAB without code optimization, the proposed tracker runs at 350 frames per second on an i7 machine. Extensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods in terms of efficiency, accuracy and robustness.

I. INTRODUCTION

Visual tracking must handle substantial target and background appearance changes while balancing robustness against computational efficiency. This paper addresses that challenge with a fast tracker that models spatio-temporal local context and reports favorable accuracy, efficiency, and robustness.

  • I. INTRODUCTION: The approach retains surrounding context that generative methods may discard and avoids the high feature-extraction loads associated with context-based tracking approaches.The paper identifies computational cost in both complex appearance modeling and large-sample feature extraction as efficiency challenges.
  • I. INTRODUCTION: The proposed tracker exploits spatio-temporal local context to improve robustness during large target and background appearance changes.It uses the target’s surrounding scene information alongside temporal relationships between consecutive frames.
  • I. INTRODUCTION: The method learns a spatial context model, updates a spatio-temporal context model, and estimates the next location by maximizing a confidence map.Spatial correlations are learned through deconvolution, while tracking is formulated as a convolution-based confidence-map computation.
  • I. INTRODUCTION: Experiments on numerous challenging sequences show favorable performance against state-of-the-art methods in accuracy, efficiency, and robustness.The reported evaluation targets the three-way trade-off identified as a central challenge for tracking algorithms.

II. PROBLEM FORMULATION

The tracker formulates object localization as estimating a confidence map from the target and its local spatial context. It learns the conditional relationship between object location and context to resolve ambiguous image measurements.

  • The tracking problem is posed as estimating an object location likelihood through a confidence map.
  • The context feature set uses image intensity and position over a neighborhood around the current object location.
  • Context information helps resolve ambiguities when image measurements support multiple interpretations.
  • The model learns P(x|c(z), o) to connect object location with spatial context.

A. Spatial Context Model

The spatial context model represents directional relationships between the object and surrounding locations, while weighting nearby context more strongly. Its non-radial structure helps distinguish similar nearby objects.

  • A. Spatial Context Model: When similar objects appear close together, spatial relationships can distinguish them even when their appearances are similar.
  • A. Spatial Context Model: The non-radially symmetric hsc models relative distance and direction, helping resolve ambiguities between nearby similar objects.
  • A. Spatial Context Model: The context prior assigns greater importance to locations closer to the currently tracked target.
  • A. Spatial Context Model: The method samples more detailed context near the object center, whereas other methods use spatial sampling techniques to focus attention.

C. Confidence Map

The confidence map uses a parameterized spatial profile and convolution of the spatial context model with weighted image context. FFT-based frequency-domain computation supports fast learning and detection.

  • C. Confidence Map: The object location ambiguity problem can arise when several nearby predicted locations are plausible.
  • C. Confidence Map: β = 2 oversmooths confidence near the object center, whereas β = 0.5 creates a sharp peak that can overfit; β = 1 gives robust results in experiments.
  • C. Confidence Map: The confidence map combines a spatial profile with weighted context through convolution to estimate object locations.
  • C. Confidence Map: FFT transforms the convolution into frequency-domain operations for fast computation.
  • C. Confidence Map: The tracker initializes the target location, updates the spatial context model, and maximizes the next frame’s confidence map to detect the object.

A. Update of Spatio-Temporal Context

The spatio-temporal context model is updated through temporal filtering in the frequency domain. Its low-pass behavior filters appearance noise and produces more stable tracking results.

  • A. Update of Spatio-Temporal Context: The update performs temporal filtering in the frequency domain.
  • A. Update of Spatio-Temporal Context: The frequency-domain filter is low-pass, suppressing image noise from appearance variations and leading to more stable results.

B. Update of Scale

The method updates the target scale from confidence-map estimates between consecutive frames, then filters those estimates to reduce oversensitivity and noise.

  • B. Update of Scale: The scale parameter σ is updated because the target scale changes over time.
  • B. Update of Scale: The proposed scale update estimates scale between consecutive frames from the confidence map.
  • B. Update of Scale: Filtering combines scale estimates from consecutive frames to avoid oversensitive adaptation and estimation noise.
  • FFT computation gives the tracker low computational complexity, with each FFT requiring O(MN log(MN)) operations.
  • The tracker runs at 350 frames per second in MATLAB on an i7 machine while using local context regions.
  • Spatio-temporal context helps distinguish distractors and supports robustness under occlusion and similar target-background appearances.

IV. EXPERIMENTS

The experiments evaluate STC on 18 challenging video sequences against 18 state-of-the-art trackers, using fixed proposed parameters and repeated trials for randomized methods.

  • IV. EXPERIMENTS: STC is evaluated on 18 video sequences containing heavy occlusion, illumination, pose and scale variation, deformation, clutter, and motion blur.
  • IV. EXPERIMENTS: The comparison includes 18 state-of-the-art trackers spanning appearance-based, context-based, correlation-filter, and related approaches.
  • IV. EXPERIMENTS: The proposed tracker uses fixed parameters across experiments and runs at 350 FPS on an i7 3.40 GHz machine with 8 GB RAM.
  • IV. EXPERIMENTS: Randomized trackers are run 10 times per sequence, with averaged results reported.

A. Experimental Setup

The setup specifies context, normalization, windowing, scale, and learning parameters before evaluating tracking with center location error and success rate.

  • A. Experimental Setup: The context region is initially set to twice the target-object size.
  • A. Experimental Setup: Context intensities are mean-normalized and multiplied by a Hamming window to reduce illumination and boundary-frequency effects.
  • A. Experimental Setup: Tracking is quantitatively evaluated with center location error and success rate, alongside average FPS in Table II.

B. Experimental Results

Across challenging sequences, STC generally achieves the best or second-best accuracy while remaining the most efficient evaluated tracker, with robustness attributed to spatio-temporal context.

  • B. Experimental Results: STC achieves the best or second-best performance on most sequences for both center location error and success rate, while averaging 350 FPS.
  • B. Experimental Results: FFT-based CST and MOS trackers are less efficient because they perform kernel operations or compute several correlation filters per frame.
  • B. Experimental Results: STC performs favorably under illumination, scale, and pose variation, including the full davidindoor sequence.
  • B. Experimental Results: On partially occluded, rotated, or deforming targets, STC tracks most frames successfully and is surpassed only by Struck in the woman sequence.
  • B. Experimental Results: STC achieves the best success rate and center location error on the abruptly moving chasing sequence.
  • B. Experimental Results: Spatio-temporal context separates targets from similar backgrounds by exploiting differences in local spatial relationships and appearances.

V. CONCLUSION

The paper presents a simple, fast, and robust visual tracker based on spatial and spatio-temporal context models, using FFT for efficient learning and detection. Experiments report favorable accuracy, robustness, and speed, including 350 frames per second in MATLAB.

  • The tracker uses spatial and spatio-temporal context models to remain robust to appearance variations from occlusion, illumination changes, and pose changes.
  • FFT-based online learning and detection produce an efficient tracker running at 350 frames per second with MATLAB implementation.
  • Experiments on challenging sequences show favorable accuracy, robustness, and speed compared with state-of-the-art algorithms.

Appendex

The appendix develops scale estimation for the tracker, using spatial scaling relationships and iterative updates. Because the relevant equation is nonlinear in the Gaussian weight function, the scale is approximately estimated and stabilized across frames.

  • The appendix illustrates the Gaussian weight function and change-of-variables relationships used in scale estimation.
  • The derivation assumes proximity between consecutive frames and uses spatially scaled versions of the image and confidence-map terms.
  • Scale estimation is difficult because the Gaussian weight function makes the relevant equation nonlinear, so the method uses an iterative approximation.
  • The scale estimate is stabilized by averaging estimates from the former n consecutive frames and incrementally updating the result.
Loading 1311.1939v1…