PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Linear Discriminant Analysis on Fisher Networks: A Hybrid Architecture for Person Re-identification

Lin Wu, Chunhua Shen, Anton van den Hengel

arXiv:1606.01595v1cs.CV

TL;DR

Person re-identification must handle dramatic appearance changes with limited matched training data while reducing intra-person variation and increasing inter-person differences. This paper combines Fisher vectors, deep neural networks, and an LDA objective to learn linearly separable representations end to end, achieving state-of-the-art results across four benchmarks.

  • Problem

    Person re-identification has limited matched training samples, while single-image losses may enlarge inter-person differences without reducing intra-person variation.

  • Method

    A hybrid network combines Fisher vectors with supervised fully connected layers and trains them end to end using modified LDA gradients back-propagated into Fisher vector encoding.

  • Results

    The method achieves state-of-the-art results across four person re-identification benchmarks, including 67.12% rank-1 matching on CUHK01.

  • Takeaways & Limitations

    LDA-trained Fisher-vector hybrids produce linearly separable hidden representations while using substantially fewer training parameters than comparable VGG-based models on CUHK03.

Abstract

from arXiv · show

Person re-identification is to seek a correct match for a person of interest across views among a large number of imposters. It typically involves two procedures of non-linear feature extractions against dramatic appearance changes, and subsequent discriminative analysis in order to reduce intra- personal variations while enlarging inter-personal differences. In this paper, we introduce a hybrid architecture which combines Fisher vectors and deep neural networks to learn non-linear representations of person images to a space where data can be linearly separable. We reinforce a Linear Discriminant Analysis (LDA) on top of the deep neural network such that linearly separable latent representations can be learnt in an end-to-end fashion. By optimizing an objective function modified from LDA, the network is enforced to produce feature distributions which have a low variance within the same class and high variance between classes. The objective is essentially derived from the general LDA eigenvalue problem and allows to train the network with stochastic gradient descent and back-propagate LDA gradients to compute the gradients involved in Fisher vector encoding. For evaluation we test our approach on four benchmark data sets in person re-identification (VIPeR [1], CUHK03 [2], CUHK01 [3], and Market1501 [4]). Extensive experiments on these benchmarks show that our model can achieve state-of-the-art results.

I. INTRODUCTION

The paper addresses person re-identification under substantial cross-view appearance variation and limited training data by combining Fisher vectors, supervised layers, and an LDA objective. The resulting network is trained end-to-end to produce linearly separable representations while reducing within-identity variation and increasing between-identity differences.

  • Person re-identification matches pedestrian images across disjoint camera views despite pose, illumination, viewpoint, and background changes.
  • Deep re-identification models require many matching training pairs, whereas only hundreds of matched samples are typically available.Their image-level cross-entropy losses may enlarge inter-person differences without reducing intra-person variation.
  • Fisher vector encoding offers a CNN-like sequence of gradient filtering, pooling, and normalization operations, while requiring less structural complexity.Prior studies reported Fisher vectors as competitive with learned representations.
  • The proposed hybrid architecture combines Fisher vectors with multiple supervised layers and an LDA criterion that maximizes between-class relative to within-class scatter.This objective is intended to yield linearly separable latent features with low intra-class and high inter-class variance.
  • LDA-derived gradients are back-propagated through the network to update the GMM parameters in Fisher vector encoding.The architecture is trained as a feed-forward network with Fisher encoding, fully connected layers, and a modified LDA objective.
  • Extensive experiments on benchmark datasets report state-of-the-art performance for the proposed model.

II. RELATED WORK

Related work develops robust descriptors, cross-view metrics, and deep or Fisher-based representations for person re-identification. The proposed direction builds on these strands by using a Fisher-vector network with supervised layers and LDA-based class separation.

  • Person re-identification research primarily develops feature descriptors, cross-view distance metrics, or methods combining both.
  • The architecture overview combines PCA-projected SIFT patches, Fisher encoding, aggregation, square-rooting, ℓ2-normalization, supervised layers, and an LDA objective.
  • Appearance descriptors face ambiguity from view orientation, occlusion, illumination, and background clutter, and some methods depend on high-quality foreground segmentation.
  • Metric-learning methods first extract image features and then learn metrics that increase inter-class differences while increasing intra-class similarity.
  • CNN-based methods learn hierarchical representations, including architectures that jointly process image pairs and compute cross-input neighborhood differences.
  • Deep Fisher networks stack Fisher-vector encoding into multiple layers and can achieve competitive results with deep convolutional networks at lower computational cost.

III. OUR ARCHITECTURE

The architecture combines Fisher-vector encoding with supervised fully connected layers, replacing sample-wise softmax classification with an LDA objective on hidden representations.

  • A. Fisher vector encoding: Fisher vectors encode local descriptors through GMM log-likelihood derivatives, capturing first- and second-order differences from Gaussian centers.Descriptors are PCA-reduced before diagonal-covariance GMM encoding.
  • A. Fisher vector encoding: Image-level Fisher vectors are formed by pooling descriptor encodings, then applying signed square-root and ℓ2 normalization.The pooled representation has dimension 2KD for K Gaussians and descriptor dimension D.
  • B. Supervised layers: The network feeds PCA-reduced Fisher vectors into L fully connected supervised layers using linear projections followed by ReLU activations.Each layer computes x_l = σ(W_lx_{l−1} + b_l), with σ(x) = max(0, x).
  • B. Supervised layers: Unlike softmax likelihood optimization, the final network learns nonlinear transforms intended to place samples in a linearly separable space.The LDA objective is imposed on hidden representations rather than defining cross-entropy on individual samples.

C. Linear discriminant analysis on deep Fisher networks

The paper applies LDA to deep Fisher-network representations, using scatter-based discrimination to seek low within-class and high between-class variation in a lower-dimensional space.

  • C. Linear discriminant analysis on deep Fisher networks: LDA transforms supervised-layer outputs into an l-dimensional subspace, with l = C − 1, to improve feature discrimination.The discriminant features are z_i = Ax_i.
  • C. Linear discriminant analysis on deep Fisher networks: Scatter matrices are computed from mean-centered class observations and corresponding mean-centered population observations.Class means m_c and the overall mean define the centered observations used in the matrices.
  • C. Linear discriminant analysis on deep Fisher networks: The Fisher-vector generative stage can model inputs continuously and provide representations amenable to linear classifiers before supervised learning.The paper views these first layers as an initially unsupervised stage that is subsequently retrained for data-dependent features.
  • C. Linear discriminant analysis on deep Fisher networks: The LDA criterion maximizes the ratio of between-class to within-class scatter, producing low same-class variance and high different-class variance after projection.Between scatter is defined as S_b = S_t − S_w.
  • C. Linear discriminant analysis on deep Fisher networks: The optimal projection is obtained from the generalized eigenvalue problem S_be = vS_we, whose eigenvectors form the projection matrix.Each eigenvalue quantifies separation along its corresponding eigenvector direction.

D. Optimization

The paper contrasts conventional sample-wise cross-entropy training with an objective designed to constrain properties of hidden representations.

  • D. Optimization: Existing person-re-identification networks optimize predicted class probabilities with sample-wise cross-entropy loss.The formulation uses a DNN with hidden layers and class-probability outputs.
  • D. Optimization: Cross-entropy maximizes the likelihood of each target class for its individual sample.The target indicator y_i,j equals one for the sample’s class and zero otherwise.
  • D. Optimization: The paper motivates LDA-based optimization because cross-entropy does not directly constrain latent representations to be linearly separable.The proposed alternative optimizes hidden representations under a separation-oriented criterion.

2) Optimization objective with linear discriminant analysis:

The modified LDA objective optimizes the least-separated directions, supports end-to-end backpropagation through Fisher vectors, and faces computational and non-convex optimization constraints.

  • 2) Optimization objective with linear discriminant analysis:: Adding λI to the within-class scatter matrix stabilizes small eigenvalues in the generalized eigenvalue problem.The regularized matrix is S_w + λI.
  • 2) Optimization objective with linear discriminant analysis:: The objective focuses on the smallest eigenvalues to avoid rewarding only already-separated class directions.The selected eigenvalues satisfy v_i < min(v_1, ..., v_{C−1}) + ϵ.
  • 2) Optimization objective with linear discriminant analysis:: Optimizing the modified objective encourages discriminative capability in each of the C − 1 feature dimensions and enables end-to-end training.Derivatives with respect to the last hidden representation allow gradients to update Fisher-vector parameters through the chain rule.
  • 2) Optimization objective with linear discriminant analysis:: Numerical gradient evaluation has O(D^2P) complexity, compared with O(DP) for Fisher-vector computation, making GMM updates potentially orders of magnitude slower.Efficient approximate gradients are proposed as alternatives.
  • 2) Optimization objective with linear discriminant analysis:: Optimization with respect to G is non-convex, so Algorithm 1 finds only a local optimum whose quality depends on initialization.Unsupervised expectation maximization is used to initialize the GMM.

E. Implementation details

The architecture extracts SIFT and color descriptors from dense image patches, then combines their Fisher-vector representations into a high-dimensional feature vector.

  • Local feature extraction: SIFT and color-histogram descriptors are extracted from 14 dense overlapping 32 × 32-pixel regions in 48 × 128-pixel person images.The regions use a 16-pixel stride in both directions.
  • Implementation expression: The supplied implementation passage includes a fragmented conditional expression without a clearly supported operational interpretation.
  • Fisher-vector representation: Each image is represented by two concatenated Fisher vectors forming a 256K-dimensional representation.

IV. COMPARISON WITH CNNS [32] AND DEEP FISHER NETWORKS [29]

The paper positions Fisher-vector processing as conceptually similar to CNN layers, while distinguishing its supervised layers and LDA-based objective from deep Fisher kernel learning.

  • Comparison with CNNs: Fisher-vector encoding alternates gradient filtering, pooling, and normalization, paralleling the linear and nonlinear operations of CNN convolutional layers.
  • Comparison with deep Fisher kernel learning: Unlike deep Fisher kernel learning, the proposed model adds stacked supervised fully connected layers to capture long-range and complex structure in person images.
  • Training objective: The proposed LDA objective learns linearly separable representations with low intra-personal and high inter-personal variation, unlike identity classification supervision.

V. EXPERIMENTS

Experiments evaluate the model on four person re-identification benchmarks using single-shot CMC recognition rates and comparisons with established methods.

  • Datasets: The evaluation uses VIPeR, CUHK03, CUHK01, and Market-1501, covering datasets with different identity, camera, and image-count scales.
  • Evaluation protocol: Single-shot evaluation matches each probe against a gallery and records rank-k recognition rates as cumulative matching characteristic results.The protocol is repeated ten times and averaged.
  • Competitors: The model is compared with handcrafted, metric-learning, ranking, and deep-learning approaches, including FPNN, DeepRanking, NullReid, and XQDA.
  • Training: Training uses Fisher vectors based on a 256-component GMM and a network with three supervised hidden layers of 4096, 1024, and 1024 units.Dropout is 0.2, and batch normalization follows the last supervised layer.

B. Experimental results

Experiments compare LDA- and cross-entropy-trained Fisher-vector and VGG models, then report recognition results across VIPeR, CUHK03, CUHK01, and Market-1501.

  • Model comparison: DeepFV+LDA achieves performance close to VGG+LDA on CUHK03 while using substantially fewer training parameters.Deep models trained with LDA outperform corresponding models trained with cross-entropy.
  • VIPeR: 44.11% rank-1 recognition on VIPeR surpasses NullReid’s previous best 42.28%, while NullReid leads at ranks 15 and 20.The proposed method performs best at ranks 1 and 5.
  • CUHK03: 63.23% rank-1 recognition on CUHK03 outperforms the previous best JointRe-id result, and the proposed method outperforms all competitors across reported ranks.The authors attribute the improvement to more training data and more shots per identity.
  • CUHK01 and Market-1501: The reported comparisons also include rank-based recognition rates on CUHK01 and rank-1 and mAP on Market-1501.

3) Evaluation on the CUHK01 data set:

On CUHK01, the proposed network outperforms the state of the art in rank-1 matching, while comparisons examine kernel-based representations and latent-space separability.

  • CUHK01 evaluation: 67.12% rank-1 matching rate surpasses the next best method’s 65% on the reported CUHK01 evaluation.The experiment uses 100 test identities and 871 training identities.
  • Kernel-method comparison: The study also compares the proposed network with kernel methods using CMC curves on the three benchmarks.
  • State-of-the-art comparison: The method is compared with state-of-the-art approaches using CMC curves on VIPeR, CUHK03, and CUHK01.
  • Kernel-method comparison: SVM classifiers trained on features from three hidden layers consistently achieve superior performance because that latent space is the most linearly separable.
  • Eigenvalue analysis: Eigenvalue analysis tracks rank-1 recognition and discriminative separation in latent representations across training epochs.

VII. APPENDIX

The appendix derives gradients for Fisher-vector encoding and shows how these expressions support end-to-end backpropagation through the hybrid network.

  • Gradient derivation: The appendix provides explicit derivatives of the loss with respect to the GMM components.
  • Gradient derivation: Derivatives of Fisher vectors are indexed over GMM components and vector dimensions, then stacked and averaged over image descriptors.
  • Appendix context: The appendix’s gradient expressions are connected to the LDA eigenvalue structure used to study recognition and discriminative separation.
  • Gradient derivation: The normalized Fisher-vector gradient is obtained by the chain rule, with gradients taken with respect to GMM parameters G = (π, µ, Σ).
  • End-to-end training: The resulting gradients enable backpropagation through Fisher-vector encoding and supervised layers to train the hybrid network end to end.
Loading 1606.01595v1…