PaperlayerView exact revision on arXiv

Source-linked AI summary

Image Set based Collaborative Representation for Face Recognition

Pengfei Zhu, Wangmeng Zuo, Lei Zhang, Simon C. K. Shiu, David Zhang

arXiv:1308.6687v1cs.CV

TL;DR

Prior ISFR methods compare query and gallery sets separately, overlooking relationships among gallery sets. ISCRC represents a convex or regularized query-set hull collaboratively over all gallery sets. On benchmark databases, it outperformed state-of-the-art ISFR methods in recognition rates and efficiency.

  • Problem

    Prior ISFR classifiers compare each query set separately with gallery sets, leaving relationships among gallery sets unmodeled.

    Source: §I · p. 3

  • Method

    ISCRC models the query set as a convex or regularized hull and represents that hull collaboratively over all gallery sets.

    Source: §VI · p. 25

  • Results

    ISCRC was superior to state-of-the-art ISFR methods in recognition rates and efficiency.

    Source: §VI · p. 25

  • Takeaways & Limitations

    The conclusion identifies query-hull representation over gallery sets as the basis of the proposed image-set collaborative classifier.

    Source: §VI · p. 25

Abstract

from arXiv · show

With the rapid development of digital imaging and communication technologies, image set based face recognition (ISFR) is becoming increasingly important. One key issue of ISFR is how to effectively and efficiently represent the query face image set by using the gallery face image sets. The set-to-set distance based methods ignore the relationship between gallery sets, while representing the query set images individually over the gallery sets ignores the correlation between query set images. In this paper, we propose a novel image set based collaborative representation and classification method for ISFR. By modeling the query set as a convex or regularized hull, we represent this hull collaboratively over all the gallery sets. With the resolved representation coefficients, the distance between the query set and each gallery set can then be calculated for classification. The proposed model naturally and effectively extends the image based collaborative representation to an image set based one, and our extensive experiments on benchmark ISFR databases show the superiority of the proposed method to state-of-the-art ISFR methods under different set sizes in terms of both recognition rate and efficiency.

I. INTRODUCTION

Prior ISFR methods either compare query and gallery sets separately, missing gallery-set relationships, or represent query images individually, missing their correlations. ISCRC instead represents a query-set hull collaboratively over all gallery sets and classifies by representation residual.

  • Parametric models face estimation and fit problems; subspace or manifold methods may degrade with few samples and high variation; closest-point models are outlier-sensitive.Source: §I · pp. 2–3
  • Prior set-to-set classifiers compare the query set with each gallery set separately, ignoring relationships among gallery sets.Source: §I · p. 3
  • Applying SRC or CRC per query image ignores within-set correlations; averaging weakens discrimination, minima expose outliers, and redundancy raises costs while hurting recognition.Source: §I · p. 4
  • ISCRC models the query set as a convex or regularized hull, represents it collaboratively over all gallery sets, and classifies by representation residual.Regularization can be imposed on the query and gallery representation coefficients for a stable solution.Source: §I · p. 4
  • Compressing gallery sets can reduce ISCRC's time complexity without sacrificing recognition rate.Source: §I · pp. 4–5
  • On three benchmark ISFR databases, ISCRC outperformed state-of-the-art methods in recognition rate and efficiency.Source: §I · p. 5

II. COLLABORATIVE REPRESENTATION BASED SET TO SETS DISTANCE

Section II introduces hull-based set-to-set distance, develops its collaborative representation variant, and uses CRSSD to frame ISCRC, including convex- and regularized-hull versions.

  • The section first introduces a hull-based set-to-set distance.Source: §II · p. 5
  • It then proposes collaborative representation based set-to-sets distance, or CRSSD.Source: §II · p. 5
  • CRSSD supports the ISCRC scheme, with convex-hull and regularized-hull versions presented separately.Source: §II · p. 5

A. Hull based set to set distance

Hull-based distances represent sets through combinations of their samples, while ISCRC represents a query-set hull collaboratively over all gallery sets and classifies using the resulting representation residuals.

A. Hull based set to set distance

  • Hull modeling represents a set as linear combinations of its samples, with coefficient constraints distinguishing convex from reduced convex hulls.The coefficients sum to one and are bounded; τ = 1 gives a convex hull, while τ < 1 gives a reduced convex hull.Source: A. Hull based set to set distance · p. 6
  • For non-intersecting sets, convex-hull set distance is the distance between their nearest hull points, but it can be sensitive to outliers.The passage also relates this distance to the distance computed by an SVM.Source: A. Hull based set to set distance · p. 6

B. Collaborative representation based set to sets distance and classification

  • Set-to-set distances do not use correlations between gallery sets, motivating collaborative representation and classification for image sets.ISCRC represents the query-set hull over gallery sets and uses the resulting coefficients to determine residuals for classification.Source: B. Collaborative representation based… · p. 7
  • Regularizing the query and gallery coefficient vectors is proposed to obtain more stable solutions.The framework discusses convex-hull and regularized-hull instantiations of the collaborative set-to-set distance.Source: B. Collaborative representation based… · p. 7

C. Convex hull based CRSSD

Convex-hull CRSSD searches for close points between the query hull and the gallery representation; an lp-norm regularized hull is offered as a more stable alternative when outliers affect convex-hull modeling.

C. Convex hull based CRSSD

  • The convex-hull formulation requires both the query and gallery hulls to be convex, with τ ≤ 1.Figure 3 provides a geometric illustration of convex-hull CRSSD.Source: C. Convex hull based CRSSD · p. 8, Figure 3 · p. 8
  • Convex-hull CRSSD finds close points between the query hull and the whole gallery set, rather than modeling each gallery set as its own hull as CHISD does.The method aims to find the closest points in a large-margin manner.Source: C. Convex hull based CRSSD · p. 8

D. lp-norm regularized hull based CRSSD

  • Because convex-hull modeling can be strongly affected by outliers, the paper proposes lp-norm regularized hulls for more stable CRSSD.The regularized hull is defined separately for the query and gallery sets.Source: D. lp-norm regularized hull based CRSSD · pp. 8–9

III. REGULARIZED HULL BASED ISCRC

The regularized-hull ISCRC model can use compact gallery dictionaries and either l1 or l2 regularization. Dictionary compression significantly improves speed with almost the same ISFR rate, and the l2 variant has a closed-form solution.

III. REGULARIZED HULL BASED ISCRC

  • The section develops regularized-hull CRSSD minimization as the RH-ISCRC classification scheme for image-set face recognition.The framework describes both convex-hull and regularized-hull instantiations of CRSSD.Source: §III · p. 9

A. Main model

  • Compressing each gallery set into a dictionary significantly improves speed with almost the same ISFR rate.The model replaces the full gallery data with compact dictionaries to reduce computation where gallery sets contain many redundant images.Source: A. Main model · pp. 9–10
  • The model regularizes query and gallery coefficients with either l1 or l2 norms; l1 yields sparser solutions but costs more computation.The paper notes sparsity can enhance classification when features are not informative enough; with one query sample, the model reduces to SRC for l1 or CRC for l2.Source: A. Main model · p. 10

B. l2-norm regularized hull based ISCRC

C. l1-norm regularized hull based ISCRC

The l1-regularized hull model is optimized by alternating between variables, with convex subproblems solvable by LARS. The paper reports convergence and gives an iteration-dependent complexity bound.

C. l1-norm regularized hull based ISCRC

  • RH-ISCRC-l1 alternates optimization of its variables, and the resulting subproblems are convex and can be solved with LARS.The paper presents an augmented Lagrangian formulation for this alternating minimization.Source: C. l1-norm regularized hull based ISCRC · pp. 11–12
  • Figure 4 plots the objective value against iteration number and reports convergence for RH-ISCRC-l1.The illustrated experiment uses the Honda/USCD1 database with 200 frames per query and gallery set, compressing each gallery set to 20 atoms.Source: Figure 4 · p. 12, C. l1-norm regularized hull based ISCRC · p. 12
  • RH-ISCRC-l1 has stated time complexity O(lm^2(na^ε + nβ^ε)), where l is the iteration number.Here m is feature dimension, na is the number of query samples, nβ is the number of dictionary atoms, and ε ≥ 1.2.Source: C. l1-norm regularized hull based ISCRC · p. 12

D. Examples and discussions

The Honda/UCSD example illustrates how collaborative hull representation selects a gallery set through residual distance. Its coefficients also show how regularization affects which samples contribute to the representation.

D. Examples and discussions

  • With l1 regularization, coefficients for same-class set X10 stand out, and the coefficient vectors are sparser, so few samples dominate representation and classification.The coefficients for the query Y and gallery data D are shown in Fig. 5.Source: Figure 5 · p. 13, D. Examples and discussions · p. 13
  • The distance r10 between reconstructed Y ˆa and D10 ˆβ10 is 0.03, the minimum among gallery sets, so ISCRC will make the correct classification.Source: D. Examples and discussions · p. 13
  • With l2 regularization, the coefficient vectors for the query and gallery data are also shown, while the example’s classification still selects the minimum-residual gallery set.The vectors appear in Fig. 6; the example’s residual comparison identifies set 10 as the minimum.Source: D. Examples and discussions · p. 13, Figure 6 · p. 14
  • ISCRC’s comparison is between a local subspace selected from the query set and local subspaces represented across the gallery sets, rather than distances between whole sets.The paper relates this construction to manifold-based methods that model sets through local subspaces.Source: D. Examples and discussions · p. 14

IV. KERNELIZED CONVEX HULL BASED ISCRC

The kernelized method addresses instability and nonlinear relationships in hull-based collaborative representation by mapping data into a higher-dimensional space. The paper presents KCH-ISCRC as a kernel-based variant and illustrates its coefficients and classification behavior.

IV. KERNELIZED CONVEX HULL BASED ISCRC

  • Kernel mapping addresses weak convex-hull constraints for stable solutions and difficult nonlinear query–gallery relationships by mapping data to a higher-dimensional space.The mapped gallery data may be overdetermined, making the convex-hull constraint strong enough for a stable solution.Source: §IV · p. 15
  • KCH-ISCRC solves the kernelized convex-hull CRSSD with quadratic programming and can use linear or Gaussian kernels.The paper reports global and quadratic convergence for the standard QP method.Source: §IV · p. 15
  • To reduce computation, KCH-ISCRC can store the gallery kernel matrix and calculate only query–query and query–gallery kernels when a query arrives.The stated time complexity is O((n_β + n_a)^3).Source: §IV · p. 15
  • In the Gaussian-kernel example, coefficients for gallery set D10 are larger than those for other sets, yielding a smaller residual and correct recognition.The example uses 50 atoms per compressed gallery set.Source: §IV · p. 15, Figure 8 · p. 16

V. EXPERIMENTAL ANALYSIS

  • The experiments compare RH-ISCRC with non-kernel methods and KCH-ISCRC with kernel methods; SRC and CRC use average query-image residuals for classification.The comparison methods include subspace/manifold, hull-based, representation-based, and kernel methods.Source: §V · p. 17

A. Parameter setting

Experiments use 50-, 100-, and 200-frame sets across three databases, with parameters configured separately for the proposed methods and competitors. On Honda/UCSD, RH-ISCRC performs strongly among non-kernel methods, while KCH-ISCRC leads kernel methods except at 100 frames per set.

A. Parameter setting

  • For competing methods, parameters follow recommendations from their original literature, with selected values empirically tuned for fair comparison.The proposed methods’ parameter sensitivity is discussed separately in Section V-F.Source: A. Parameter setting · p. 17

B. Honda/UCSD

  • The Honda/UCSD split uses 20 training sequences and 39 test sequences, with image intensity as the feature.Faces are detected, resized to 20 × 20, and histogram-equalized.Source: B. Honda/UCSD · p. 18
  • On Honda/UCSD, RH-ISCRC outperforms nearly all non-kernel methods, and KCH-ISCRC performs best among kernel methods except at 100 frames per set.The results are reported in Table IV.Source: B. Honda/UCSD · p. 18
  • On Honda/UCSD, RH-ISCRC-l1 and RH-ISCRC-l2 achieve the same recognition rate, indicating that l2 regularization is sufficient for a good regularized-hull solution on this dataset.Source: B. Honda/UCSD · p. 18

C. CMU MoBo

Across the CMU MoBo and YouTube Celebrities evaluations, the proposed methods outperform competitors under the reported set sizes, with results varying by dataset and kernel choice. The reported timing comparison also finds high efficiency for the proposed methods.

C. CMU MoBo

  • On CMU MoBo, the proposed methods outperform the other methods across different frames per set, while RH-ISCRC and KCH-ISCRC have similar ISFR rates.Table V reports recognition rates; the dataset contains 96 sequences from 24 subjects.Source: C. CMU MoBo · pp. 18–19, Table V · p. 20

D. YouTube Celebrities

  • Under YouTube’s uncontrolled conditions, l1 regularization is described as helpful for solution stability and discrimination, and RH-ISCRC-l1 performs much better than RH-ISCRC-l2.The dataset has significant variations among query and gallery sets.Source: D. YouTube Celebrities · p. 21

E. Time comparison

  • On CMU MoBo at 200 frames per set, RH-ISCRC-l2 is fastest except for RNP, RH-ISCRC-l1 is also fast, and KCH-ISCRC is much faster than other kernel methods.The timing comparison uses average running time per set on an Intel Core i7-2600K PC.Source: E. Time comparison · p. 21
  • Overall, the authors report that RH-ISCRC and KCH-ISCRC combine high ISFR accuracy with higher efficiency than competing methods.Source: E. Time comparison · p. 21

D. YouTube Celebrities

  • On YouTube, RH-ISCRC-l1 has the highest recognition rate among non-kernel methods, while KCH-ISCRC leads kernel methods; smaller frame sets yield more significant ISCRC improvement.The recognition rates are presented in Table VI.Source: D. YouTube Celebrities · p. 21, Table VI · p. 22

F. Parameter sensitivity analysis

Across three benchmark ISFR databases, ISCRC outperforms state-of-the-art methods in recognition rates and efficiency. Parameter tests report RH-ISCRC-l2 accuracy variation within 0.5% in the 100- and 200-frame experiments and insensitivity of KCH-ISCRC to τ.

F. Parameter sensitivity analysis

VI. CONCLUSION

  • ISCRC models the query set as a convex or regularized hull and represents it collaboratively over all gallery sets to compute classification residuals.The CRSSD considers correlations and distinctions among query-set images and relationships between gallery sets.Source: §VI · p. 25
  • ISCRC outperformed state-of-the-art ISFR methods on three benchmark databases in both recognition rates and efficiency.Source: §VI · p. 25
Loading 1308.6687v1…