PaperlayerView exact revision on arXiv

Source-linked AI summary

A Fast and Accurate Unconstrained Face Detector

Shengcai Liao, Anil K. Jain, Stan Z. Li

arXiv:1408.1656v3cs.CV

TL;DR

Unconstrained face detection remains difficult because arbitrary pose and occlusion challenge existing multiview and frontal-face methods. The paper combines normalized pixel-difference features with a deep quadratic tree and a single soft-cascade classifier; evaluations on three public databases report state-of-the-art unconstrained detection, with the detector about 6 times faster than Viola-Jones.

  • Problem

    Existing approaches struggle to detect faces under arbitrary pose and occlusion, while multiview methods require pose-specific classifiers or manual pose labels.

  • Method

    The method uses scale-invariant, bounded NPD features and a deep quadratic tree that learns feature combinations for one soft-cascade AdaBoost classifier.

  • Results

    Evaluations on three public face databases show state-of-the-art performance for unconstrained face detection, and the detector is about 6 times faster than Viola-Jones.

  • Takeaways & Limitations

    A single learned classifier can address unconstrained faces with large pose variations and occlusions without pose labeling or clustering during training.

  • Takeaways & Limitations

    Prior multiview methods cited by the paper cannot simultaneously handle other challenges such as occlusion and require partitioning data into known poses.

Abstract

from arXiv · show

We propose a method to address challenges in unconstrained face detection, such as arbitrary pose variations and occlusions. First, a new image feature called Normalized Pixel Difference (NPD) is proposed. NPD feature is computed as the difference to sum ratio between two pixel values, inspired by the Weber Fraction in experimental psychology. The new feature is scale invariant, bounded, and is able to reconstruct the original image. Second, we propose a deep quadratic tree to learn the optimal subset of NPD features and their combinations, so that complex face manifolds can be partitioned by the learned rules. This way, only a single soft-cascade classifier is needed to handle unconstrained face detection. Furthermore, we show that the NPD features can be efficiently obtained from a look up table, and the detection template can be easily scaled, making the proposed face detector very fast. Experimental results on three public face datasets (FDDB, GENKI, and CMU-MIT) show that the proposed method achieves state-of-the-art performance in detecting unconstrained faces with arbitrary pose variations and occlusions in cluttered scenes.

1 INTRODUCTION

The paper targets unconstrained face detection, where pose, occlusion, illumination, blur, and low resolution challenge conventional detectors. It introduces NPD features and a deep quadratic tree with one soft-cascade classifier to address these variations efficiently.

  • Unconstrained scenes combine arbitrary pose, illumination, occlusion, expression, blur, and low resolution, causing conventional detectors to fail.The paper motivates applications including video surveillance and hand-held-device imagery.
  • NPD represents two pixel intensities by their difference-to-sum ratio and provides scale invariance, boundedness, and image reconstruction ability.The feature is inspired by the Weber Fraction in experimental psychology.
  • NPD evaluation uses a look-up table, while pre-scaled detection templates support multiscale detection.These design choices are presented as efficiency advantages.
  • A deep quadratic tree learns and combines an optimal subset of NPD features, partitioning complex face manifolds within a single soft-cascade AdaBoost classifier.The approach avoids pose labeling or clustering during training.
  • The detector does not require pose-specific cascade design, pose labeling, or clustering in training.This contrasts with multiview approaches that learn separate classifiers for specific facial views.
  • The detector is designed to handle illumination and pose variations, occlusions, blur, and low-resolution faces in unconstrained scenarios.The stated robustness target covers the main variations identified in the introduction.

2 RELATED WORK

Related work advances face detection through richer features, alternative cascade structures, and occlusion-specific methods. However, prior approaches commonly require many classifiers or pose labels and do not jointly solve arbitrary pose and occlusion variation.

  • Richer features can improve detection but may produce hundreds of thousands of features and increase training time.
  • The paper integrates NPD features with a deep quadratic tree to learn optimal ordinal or contrastive feature combinations.This approach is presented as automatically partitioning unconstrained face variations into tree leaves.
  • Multiview cascade methods use separate classifiers or branches for specific facial views, increasing computational cost as the number of classifiers grows.
  • Existing multiview approaches generally require manual pose labeling, while automatically learned clusters are not always related to pose variations.
  • Occlusion methods typically address frontal faces, whereas multiview methods do not simultaneously handle arbitrary pose and occlusion.The combination remains described as an unresolved challenge.

3 NORMALIZED PIXEL DIFFERENCE FEATURE SPACE

The Normalized Pixel Difference (NPD) compares two pixel intensities by their relative difference, producing a scale-invariant, bounded representation that retains the information needed to reconstruct the image up to scale.

  • NPD definition: NPD is computed as the difference between two nonnegative pixel values divided by their sum, with f(0,0) defined as 0.The feature measures relative rather than absolute difference.
  • Feature interpretation: The sign of NPD encodes the ordinal relationship between two pixels, while its magnitude measures their relative difference.Using only the sign can be noise-sensitive when pixel values are similar.
  • Feature space: For an s × s patch with p=s·s pixels, antisymmetry leaves d=p(p−1)/2 pairwise NPD features; a 20×20 patch yields 79,800 features.Only one ordering of each pixel pair is needed because f(x,y) and f(y,x) are antisymmetric.
  • Feature properties: NPD is scale invariant, helping make the representation robust to illumination changes.Intensity-scaled images map to the same point in the NPD feature space.
  • Feature properties: NPD values are bounded in [-1,1], enabling histogram binning or threshold learning in tree-based classifiers.The bounded function defines a nonlinear surface.
  • Feature properties: The complete NPD feature vector can reconstruct the original image up to a scale factor, while collapsing intensity-scaled images to one feature-space point.A linear-time reconstruction approach is given.

4 NPD FOR FACE DETECTION

The detector replaces shallow stump decisions with deep quadratic trees that learn and combine NPD features, then uses one soft-cascade classifier for unconstrained faces. NPD’s pairwise structure and implementation choices address pose, occlusion, illumination, blur, and multiscale speed requirements.

  • 4.1 Deep Quadratic Tree: Quadratic splitting uses (ax^2 + bx + c) < t, allowing learned range tests that incorporate first- and second-order information.With suitable coefficients, the rule checks whether x lies inside or outside a learned interval [θ1, θ2].
  • 4.1 Deep Quadratic Tree: Deep quadratic trees learn ordinal and contrastive NPD structures, including darker-than, brighter-than, and polarity-uncertain edge relationships.The latter captures notable contrast when the polarity between pixels is uncertain.
  • 4.1 Deep Quadratic Tree: The method quantizes bounded NPD values into L bins and exhaustively searches for two thresholds using weighted mean square error.The paper uses L=256 and accelerates threshold search with sample-weight histograms.
  • 4.1 Deep Quadratic Tree: Depth-eight trees combine several selected NPD features, automatically partitioning different face views into leaves for face/nonface prediction.Gentle AdaBoost selects discriminative features and constructs strong classifiers.
  • 4 NPD for Face Detection: A single soft-cascade classifier handles unconstrained pose and occlusion without pose labeling or pose clustering during training.The trees automatically divide the face manifold into sub-manifolds.
  • 4 NPD for Face Detection: Pairwise NPD features are designed for occlusion, illumination, blur, and low-resolution conditions because they use only two pixel values and are scale invariant.Unlike large-support Haar-like features, pairwise computations limit the support of each feature.
  • 4.2.1 Implementation Details: Training uses AFLW with 217,300 augmented face images, a 24×24 template, depth-eight trees, 1,226 trees, and 46,401 NPD features.Nonface bootstrapping masks facial regions with random nonface images.
  • 4.2.1 Implementation Details: A 256×256 lookup table reduces each 8-bit NPD computation to one memory access, while precomputed templates avoid iterative image rescaling for multiscale detection.The learned template can be scaled to detect faces at various sizes.

5 EXPERIMENTS

The NPD face detector was evaluated across FDDB, GENKI, and CMU-MIT, including challenging pose, occlusion, illumination, blur, and crowding conditions. It generally offered strong detection accuracy with substantially faster runtime than several competing methods.

  • Evaluation setup: The evaluation covered FDDB, GENKI, and CMU-MIT, including unconstrained conditions such as pose variation, occlusion, illumination variation, and blur.The experiments also analyzed detection speed and performance on challenge-specific FDDB subsets.
  • FDDB results: 54% of annotated FDDB faces were detected at FP=0 using the discrete metric with 50% overlap against ground truth.The proposed detector ranked second at FP=0 for FDDB’s discrete metric and third for its continuous metric.
  • FDDB results: The NPD detector was among the top performers on FDDB’s discrete metric but trailed four recent methods on the continuous metric.Square detections with horizontal and vertical expansion were less suitable for FDDB’s elliptical continuous annotations than elliptical detections.
  • GENKI and CMU-MIT results: On CMU-MIT, NPD detected about 80% of frontal faces without false positives, although Soft cascade performed better and SURF surpassed NPD at higher false positives on GENKI.SURF used a 40 × 40 template, whereas NPD used a 20 × 20 template; larger templates can be more computationally expensive and less effective for blurred faces.
  • Feature analysis: NPD improved FDDB detection over Haar, LBP, and POF by about 6%, 19%, and 15% for the discrete metric at FP=1.For the continuous metric at FP=1, the corresponding improvements were about 4%, 13%, and 10%.
  • Detection speed: The detector achieved 29.6 FPS on 1920 × 1080 video, 177.6 FPS on VGA desktop video, and 70 FPS for unconstrained VGA detection using a multithreaded i5 CPU.Look-up-table evaluation and pre-scaled templates avoid repeated image rescaling and contribute to the reported speed.

6 SUMMARY AND FUTURE WORK

The paper proposes a fast, accurate unconstrained face detector based on NPD features and a deep quadratic tree with a single soft-cascade AdaBoost classifier. Evaluations report state-of-the-art performance across three public databases, with efficiency and remaining challenges also identified.

  • NPD features provide scale invariance, boundedness, and reconstruction ability for face detection.
  • A deep quadratic tree learns an optimal subset and combinations of NPD features.
  • A single soft-cascade AdaBoost classifier handles faces with large pose variations and occlusions.
  • Evaluations on three public face databases report state-of-the-art performance for unconstrained face detection.
  • The detector is about 6 times faster than the Viola-Jones detector implemented in OpenCV 2.4.
  • Occlusions and blur remain two major challenges, while the proposed features and classifier may also apply to face attributes and pedestrian detection.

APPENDIX A BOUNDEDNESS OF NPD

The boundedness proof establishes that the NPD feature remains within [-1,1] for nonnegative pixel intensities. It also identifies the exact conditions for attaining either boundary.

  • For all nonnegative x and y, the NPD feature f(x,y) is bounded in [-1,1].
  • When x > 0 and y = 0, the NPD feature equals 1.
  • When x = 0 and y > 0, the NPD feature equals -1.
  • The inequalities follow from the nonnegativity of the pixel intensities.

APPENDIX B PROOF OF THEOREM 1

The proof shows that the NPD constraints have a one-dimensional nullspace and admit a nonnegative reconstruction of the original image up to a scale factor. A special zero-image case requires separate treatment.

  • The nullspace of F contains only one nonzero direction, so every solution to Eq. (e) has the form cˆx.
  • The matrix F is sparse, with at most two nonzero entries per row and at least one nonzero entry in every row.
  • Every row of F can be expressed as a linear combination of the first p−1 rows, establishing rank(F) = p−1.
  • The proof guarantees a nonnegative solution because (fij−1)(fij+1) ≤ 0 implies xixj ≥ 0 for all i and j.
  • A specific reconstruction can be obtained by setting x1 = 1 and solving the remaining variables from the first p−1 equations in linear time.
  • For the all-zero original image, reconstruction is possible with c = 0, but a positive-scale solution is not generally equivalent to the zero image.
Loading 1408.1656v3…