PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition

Changxing Ding, Jonghyun Choi, Dacheng Tao, Larry S. Davis

arXiv:1401.5311v2cs.CV

TL;DR

Unconstrained face recognition is difficult under degraded quality and variations in illumination, pose, expression, and occlusion. The paper introduces DCP and the MDML-DCPs scheme, combining directional filtering with holistic- and component-level features. Across four databases and both identification and verification tasks, DCP and MDML-DCPs report strong results, including the best performance on challenging FRGC 2.0 and LFW evaluations.

  • Problem

    Unconstrained face recognition must handle degraded image quality and large variations in illumination, pose, expression, and occlusion.

  • Method

    MDML-DCPs applies multi-directional first-derivative Gaussian filtering and computes DCP features at holistic and component levels.

  • Results

    MDML-DCPs achieves 93.39% verification rate on ROC III of FRGC 2.0 and outperforms the state-of-the-art LFW result by 2.4%.

  • Takeaways & Limitations

    DCP provides superior discriminative power and robustness to pose, expression, and moderate illumination variations across four face databases.

  • Takeaways & Limitations

    The paper reports MDML-DCPs without score normalization because score normalization may not suit general face verification with only a target-query image pair.

Abstract

from arXiv · show

To perform unconstrained face recognition robust to variations in illumination, pose and expression, this paper presents a new scheme to extract "Multi-Directional Multi-Level Dual-Cross Patterns" (MDML-DCPs) from face images. Specifically, the MDMLDCPs scheme exploits the first derivative of Gaussian operator to reduce the impact of differences in illumination and then computes the DCP feature at both the holistic and component levels. DCP is a novel face image descriptor inspired by the unique textural structure of human faces. It is computationally efficient and only doubles the cost of computing local binary patterns, yet is extremely robust to pose and expression variations. MDML-DCPs comprehensively yet efficiently encodes the invariant characteristics of a face image from multiple levels into patterns that are highly discriminative of inter-personal differences but robust to intra-personal variations. Experimental results on the FERET, CAS-PERL-R1, FRGC 2.0, and LFW databases indicate that DCP outperforms the state-of-the-art local descriptors (e.g. LBP, LTP, LPQ, POEM, tLBP, and LGXP) for both face identification and face verification tasks. More impressively, the best performance is achieved on the challenging LFW and FRGC 2.0 databases by deploying MDML-DCPs in a simple recognition scheme.

1 INTRODUCTION

Unconstrained face recognition remains difficult because real-world images contain degraded quality and wide variations in pose, illumination, expression, and occlusion. The paper addresses limitations of existing descriptors with DCP and MDML-DCPs, reporting strong performance across identification and verification benchmarks.

  • Unconstrained face recognition is challenged by degraded image quality and variations in pose, illumination, expression, and occlusion.
  • Effective face representations must discriminate inter-personal differences while remaining robust to intra-personal variations.
  • Existing descriptors largely overlook facial textural characteristics and face difficulties adopting large sampling sizes with reasonable encoding complexity and feature size.
  • DCP encodes second-order information in facially informative directions, samples twice as many pixels as LBP, and keeps feature size reasonable through entropy-based grouping.DCP requires only twice the computational cost of LBP, while its DCP-1 and DCP-2 sub-descriptors match LBP's time and memory costs.
  • MDML-DCPs combines multi-directional Gaussian-derivative filtering with holistic- and component-level DCP features to encode multi-level invariant face characteristics.
  • DCP consistently achieves superior performance for face identification and verification, while MDML-DCPs achieves the best performance on FRGC 2.0 and LFW.The evaluation covers FERET, CAS-PEAL-R1, FRGC 2.0, and LFW.

2 DUAL-CROSS PATTERNS

DCP samples facial neighborhoods along informative directions, independently encodes directional texture, and combines the results through two cross-shaped encoders. Regional histograms of these codes form a compact face representation that can be compared between images.

  • 2.1 Local Sampling: DCP local sampling follows eight directions aligned with major facial textures, using two symmetrically sampled pixels per direction.The directions are 0, π/4, π/2, 3π/4, π, 5π/4, 3π/2, and 7π/4.
  • 2.2 Pattern Encoding: Each sampling direction is encoded independently before the directional patterns are combined into DCP codes.
  • 2.2 Pattern Encoding: DCPi compares gray-value differences between the center O, inner sample Ai, and outer sample Bi to encode directional second-order texture.
  • 2.2 Pattern Encoding: Grouping eight directions into two four-direction encoders reduces the code count from 4^8 = 65536 to 4^4×2 = 512, improving compactness and robustness despite information loss.
  • 2.3 Dual-Cross Grouping: The two cross encoders use alternating directions because maximum spacing is treated as maximizing statistical independence and joint Shannon entropy.The grouping is {DCP0, DCP2, DCP4, DCP6} and {DCP1, DCP3, DCP5, DCP7}.
  • 2.4 DCP Face Image Descriptor: Two DCP code maps are partitioned into non-overlapping regions, whose code histograms are concatenated into the holistic face representation.The resulting representation can measure image similarity using chi-squared distance or histogram intersection.

3 MULTI-DIRECTIONAL MULTI-LEVEL DUAL-CROSS PATTERNS

MDML-DCPs combines directional gradient filtering with holistic and component-level features to address illumination, pose, and expression variation in unconstrained face recognition. The scheme uses two geometric rectifications and nine normalized feature vectors.

  • Multi-directional filtering: MDML-DCPs applies first-derivative Gaussian filtering in multiple directions to produce gradient images more robust to illumination variation.The filter enhances facial texture while suppressing noise, consistent with signal-to-noise, edge-location, and single-response criteria.
  • Geometric rectification: The method uses similarity and affine transformations, with the latter reducing intra-personal appearance differences caused by pose variation.Similarity rectification preserves facial contours, components, and their configuration.
  • Multi-level representation: Holistic-level features capture facial components, contours, and configuration, whereas component-level features describe individual components independently of changes elsewhere.Fusing these complementary levels promotes robustness to interference from occlusion, pose, and expression changes.
  • Feature construction: The representation contains nine normalized feature vectors: H1 and H2 from similarity-rectified images, plus H3 and C1–C6 from affine-transformed images.The framework first locates 49 facial feature points before extracting the feature vectors.
  • Feature construction: H1 encodes the external cropped face including contour, while H2 encodes internal facial components without facial contour or background interference.H3 uses points across components, and C1–C6 focus on individual facial components.

4 FACE RECOGNITION ALGORITHM

Face matching builds one classifier for each of nine MDML-DCPs feature vectors and fuses their similarity scores. Depending on dataset supervision, it uses WPCA or PLDA after dimensionality reduction.

  • Score fusion: One classifier is built for each of the nine feature vectors, and their similarity scores are fused using a linear SVM or simple averaging.This provides the matching procedure for the complete MDML-DCPs representation.
  • Classifier selection: PLDA is selected when training data contain multiple images per subject; otherwise, WPCA is used.PLDA is supervised, whereas WPCA is unsupervised.
  • WPCA: WPCA projects high-dimensional feature vector x into a low-dimensional vector y using an orthogonal projection learned by PCA.Whitening normalizes principal-component contributions because leading eigenvectors mainly encode illumination and expression variation.
  • PCA combined with PLDA: Before PLDA, the dimensionality of each of the nine high-dimensional feature vectors is reduced by PCA.PLDA models face-data generation using between-individual and within-individual subspaces, identity variables, and noise terms.
  • PCA combined with PLDA: PLDA similarity is computed through a log-likelihood ratio testing whether two observed images share the same latent identity variable.The paper calls this log-likelihood ratio the similarity score.

5 EXPERIMENTS

The paper evaluates DCP and MDML-DCPs across identification and verification tasks on four large-scale public face databases. Experiments examine grouping, descriptor comparisons, multi-directional filtering, and the complete representation scheme.

  • Evaluation setup: Experiments cover face identification and verification on FERET, CAS-PEAL-R1, FRGC 2.0, and LFW.Example database images are shown in Figs. 5 and 6.
  • Evaluation setup: FERET uses one gallery set and four probe sets under its standard face-identification protocol.The probe sets are Fb, Fc, Dup1, and Dup2.
  • Evaluation setup: CAS-PEAL-R1 provides nine probe sets isolating expression, accessory, lighting, time, background, distance, and pose variations.The standard identification protocol is followed.
  • Evaluation setup: FRGC 2.0 evaluation focuses on Experiments 1 and 4 and reports verification rates at 0.1% FAR.Their query sets contain controlled and uncontrolled-condition images, respectively.
  • Evaluation setup: LFW contains 13,233 unconstrained images, with View 1 used for model selection and View 2 for performance reporting.Both image-restricted and image-unrestricted training paradigms are considered.
  • Experimental design: The fourth experiment demonstrates the MDML-DCPs representation after similarity- and affine-based normalization, with photometric normalization applied to resampled images.Earlier descriptor experiments use 128 × 128 crops and the Tan–Triggs operator.

5.1 Empirical Justification for Dual-Cross Grouping

The dual-cross grouping mode is evaluated against all 35 possible partitions of DCP’s eight sampling directions using joint Shannon entropy. It achieves the highest entropy across all tested sampling-radius settings.

  • Experimental protocol: 35 grouping modes are evaluated by partitioning the eight DCP directions into two subsets and summing their joint Shannon entropies over 1,196 FERET gallery images.The experiment tests the entropy-based rationale for dual-cross grouping.
  • Results: The dual-cross grouping mode achieves the highest joint Shannon entropy among all 35 grouping modes.Figure 8 compares it with two representative alternatives as a function of the inner and outer sampling radii.
  • Results: The dual-cross grouping mode maintains the highest entropy under every tested set of sampling radii and is therefore empirically optimal.Smaller radii increase dependence among sampled points and reduce joint Shannon entropy.

5.2 Evaluation of the Performance of DCP

DCP is evaluated against state-of-the-art descriptors across identification and verification benchmarks, including controlled and unconstrained conditions. It generally performs strongly, especially under pose, expression, and moderate illumination variation, while retaining low computational cost.

  • FERET identification: DCP performs best on FERET’s Dup1 and Dup2 probe sets, with a 1.28% margin over the second-best descriptor on Dup2.
  • CAS-PEAL-R1 identification: DCP produces the highest mean identification rate on CAS-PEAL-R1’s generally favorable probe sets and shows pose robustness of up to 3.44% on PU, PM, and PD.
  • FRGC 2.0 verification: DCP’s mean verification rate in FRGC 2.0 Experiment 4 exceeds MsLBP and LTP by 2.64% and 1.09%, respectively, but trails filtering-based descriptors.
  • LFW verification: Among manually designed descriptors on LFW, DCP performs best and DCP-1 ranks second, while DCP has lower time and memory costs than DFD.
  • Discussion: DCP’s strong performance is attributed to large sampling, second-order statistics in informative facial directions, and complementary dual-cross encoders.

5.3 The Contribution of Multi-directional Filtering

Multi-directional filtering addresses DCP’s degradation under serious illumination variation and blur. MD-DCPs applies DCP to FDG-filtered images and improves results across challenging identification and verification settings.

  • FDG filtering converts face images into gradient maps and provides high SNR, targeting illumination variation and low-quality images.
  • MD-DCPs raises FERET identification from 17.12% with DCP to 33.56% on a newly constructed challenging probe set.
  • MD-DCPs raises CAS-PEAL-R1 PL identification from 50.25% with DCP to 65.23%, exceeding LPQ, POEM, and LGXP results.
  • MD-DCPs achieves a 30.74% mean verification rate at 0.1% FAR on FRGC 2.0 Experiment 4 across ROC I, ROC II, and ROC III.

5.4 Performance Evaluation of MDML-DCPs

MDML-DCPs combines holistic- and component-level DCP features with recognition classifiers and score fusion. It achieves strong results across FERET, CAS-PEAL-R1, FRGC 2.0, and LFW, including challenging unconstrained settings.

  • Representation: The scheme uses holistic-level features for facial contours and component configuration alongside component-level features focused on individual facial components.
  • CAS-PEAL-R1: MDML-DCPs + WPCA achieves the best performance on eight of nine CAS-PEAL-R1 probe sets and improves over DCP by 32.67% on the challenging PL set.
  • FRGC 2.0: MDML-DCPs achieves superior performance on both FRGC 2.0 Experiments 1 and 4 using the single DCP descriptor.
  • Scope and evaluation: MDML-DCPs results are reported without similarity-matrix score normalization because that operation may not suit verification settings with only one target-query pair.
  • LFW: MDML-DCPs outperforms the current state-of-the-art method on LFW by over 2.2% and another PLDA-based method by over 5%.
  • LFW: Linear SVM score fusion yields a 95.58% mean verification rate on LFW, outperforming score averaging.

6 CONCLUSION

The paper presents DCP and the MDML-DCPs representation scheme for robust unconstrained face recognition, combining descriptor design with holistic- and component-level features. Across four databases, MDML-DCPs achieves strong recognition performance, including 93.39% verification on FRGC 2.0 ROC III and a 2.4% improvement over the state-of-the-art result on LFW.

  • DCP encodes second-order statistics in the most informative directions and is robust to pose, expression, and moderate illumination variations.
  • MDML-DCPs combines holistic-level and component-level DCP features into a comprehensive face representation scheme.
  • 93.39% verification rate was achieved on ROC III in Experiment 4 of FRGC 2.0.
  • 2.4% improvement over the state-of-the-art result was achieved on LFW.
  • The work aims to expedite the design of practical face image descriptors and representation schemes.

APPENDIX A PARAMETER SELECTION OF DCP

The appendix examines how DCP parameters affect FERET identification performance and reports settings that optimize results. DCP consistently outperforms LBP across the tested parameter settings, while remaining effective with either of two histogram similarity metrics.

  • DCP performance depends on radii Rin and Rex, which capture information at different scales, and region number N, which preserves spatial information but increases misalignment sensitivity.
  • N = 9, Rin = 4, and Rex = 6 produce optimal DCP performance in the FERET parameter experiment.
  • DCP consistently outperforms LBP across all tested parameter settings, with a 10% to 30% advantage when N ≤2.
  • 92.80% mean identification is obtained with chi-squared similarity and 92.66% with histogram intersection.
Loading 1401.5311v2…