PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Self-Paced Learning for Cost-Effective and Progressive Face Identification

Liang Lin, Keze Wang, Deyu Meng, Wangmeng Zuo, Lei Zhang

arXiv:1701.03555v2cs.CV

TL;DR

The paper tackles incremental face identification with scarce initial labels, changing appearances, and costly annotation. It proposes ASPL, which combines active learning and self-paced learning to automate confident-sample labeling while querying uncertain samples. On CACD and CASIA-WebFace-Sub, ASPL reached state-of-the-art performance with about 40% and 45% annotations, respectively.

  • Problem

    Incremental face identification must learn from few initial labels while handling unpredictable appearance changes and minimizing user annotation effort.

  • Method

    ASPL combines self-paced learning for high-confidence samples with active user annotation for low-confidence samples, progressively updating CNN-based classifiers.

  • Results

    ASPL required about 40% of annotations on CACD and 45% on CASIA-WebFace-Sub to achieve state-of-the-art performance.

  • Takeaways & Limitations

    The framework reduces annotation requirements while maintaining competitive or better performance and improving robustness against noisy samples.

Abstract

from arXiv · show

This paper aims to develop a novel cost-effective framework for face identification, which progressively maintains a batch of classifiers with the increasing face images of different individuals. By naturally combining two recently rising techniques: active learning (AL) and self-paced learning (SPL), our framework is capable of automatically annotating new instances and incorporating them into training under weak expert re-certification. We first initialize the classifier using a few annotated samples for each individual, and extract image features using the convolutional neural nets. Then, a number of candidates are selected from the unannotated samples for classifier updating, in which we apply the current classifiers ranking the samples by the prediction confidence. In particular, our approach utilizes the high-confidence and low-confidence samples in the self-paced and the active user-query way, respectively. The neural nets are later fine-tuned based on the updated classifiers. Such heuristic implementation is formulated as solving a concise active SPL optimization problem, which also advances the SPL development by supplementing a rational dynamic curriculum constraint. The new model finely accords with the "instructor-student-collaborative" learning mode in human education. The advantages of this proposed framework are two-folds: i) The required number of annotated samples is significantly decreased while the comparable performance is guaranteed. A dramatic reduction of user effort is also achieved over other state-of-the-art active learning techniques. ii) The mixture of SPL and AL effectively improves not only the classifier accuracy compared to existing AL/SPL methods but also the robustness against noisy data. We evaluate our framework on two challenging datasets, and demonstrate very promising results. (http://hcp.sysu.edu.cn/projects/aspl/)

I. INTRODUCTION

The paper addresses incremental face identification when appearance changes, initial labels are scarce, and user effort must remain low. It proposes ASPL, combining active learning with self-paced learning to use confident samples automatically and query uncertain samples selectively.

  • Problem: Incremental face identification must handle large appearance variations, unpredictable new images, and only a few initially labeled samples.The paper also targets minimizing user interventions as learning continues.
  • Problem: Conventional incremental methods can fail in complex, large-scale environments and degrade with insufficient initial data, noise, or outliers.These limitations motivate combining active learning and self-paced learning.
  • Background: Active learning selects informative low-confidence samples for annotation, whereas self-paced learning incorporates high-confidence samples first.The paper treats these opposing selection criteria as complementary learning regimes.
  • Method: High-confidence samples reduce annotation burden, while low-confidence user-labeled samples provide informative and adaptive updates during early learning.The framework is designed to scale while retaining human input where classifier uncertainty is highest.
  • Contributions: The framework combines cost reduction with improved classifier accuracy and robustness against noisy samples.The paper attributes these benefits to automatic incorporation of confident samples and active intervention on uncertain or diverse samples.
  • Method: ASPL initializes CNN-based classifiers with few annotations, then ranks unlabeled samples and updates classifiers through self-paced inclusion or active user queries.The framework divides labor by assigning reliable high-confidence recognition to computers and ambiguous low-confidence cases to users.

II. RELATED WORK

Related work covers incremental face recognition, active learning, and self-paced learning. The paper motivates combining active user annotation with easy-to-hard sample selection to reduce labeling effort while improving accuracy and robustness.

  • Incremental Face Recognition: Incremental face-recognition methods update representations or classifiers as new face images arrive, but existing approaches have limited large-scale validation and often require labels for both initial and incremental data.The cited related work notes that prior face recognizers were not successfully validated on datasets exceeding 500 individuals.
  • Active Learning: Active learning selects informative unlabeled samples for user annotation, commonly prioritizing low-confidence or decision-boundary instances.Certainty-based selection can be sensitive to outliers when it ignores the broader unlabeled set.
  • Self-Paced Learning: Self-paced learning gradually incorporates samples from easy to complex by weighting training instances and increasing the pace parameter.Its formulation uses a weighted loss and a self-paced regularizer whose sample weights decrease with loss and with the pace parameter.
  • Complementarity: The proposed framework combines both schemes so high-confidence samples can be incorporated automatically while low-confidence samples receive active user annotation.Figure 2 depicts initialization, classifier updating, SPL labeling, AL annotation, and CNN fine-tuning as the workflow.
  • Complementarity: Active learning and self-paced learning are complementary because they target low-confidence boundary samples and high-confidence intra-class samples, respectively.The paper frames their combination as a way to reduce human labor while improving classifier accuracy and robustness to noisy samples.

III. FRAMEWORK OVERVIEW

The framework initializes face classifiers from a small labeled set, alternates automatic and user-assisted sample incorporation, and fine-tunes the CNN as the classifiers are updated.

  • Initialization: The pipeline begins with CNN pretraining for face representation and initialization of classifiers using a few manually annotated images per individual.The initial samples are feature-extracted with a pretrained CNN before classifier training.
  • Classifier Updating: Classifier updating retrains one-vs-all linear SVMs as manually annotated and pseudo-labeled samples accumulate.The classifiers initially use only a small labeled subset and are retrained as the framework matures.
  • Self-Paced Labeling: High-confidence unlabeled samples are ranked by current classifier weights and automatically assigned pseudo-labels.This self-paced step avoids requiring additional human labor for selected samples.
  • Active Annotation: Low-confidence, informative unlabeled samples are selected using an active-learning criterion and manually annotated by users.The selected samples are generally those with low confidence under the current classifiers.
  • CNN Fine-Tuning: After interaction steps, the CNN is fine-tuned using both self-labeled and manually annotated samples with softmax loss.The parameters are optimized through stochastic gradient descent.

IV. FORMULATION AND OPTIMIZATION

The paper formulates the entire active self-paced learning pipeline as an optimization problem, providing a theoretical interpretation of its alternating learning procedure.

  • Optimization Formulation: The proposed framework is theoretically justified as solving an active self-paced learning optimization model.This formulation is intended to clarify the intrinsic mechanism underlying the ASPL system.

A. Active Self-paced Learning

The active self-paced learning formulation jointly represents classifiers, sample labels, sample importance, and curriculum constraints for incrementally growing face data. It alternates weighted classifier learning, pseudo-labeling, self-paced weighting, and active annotation while dynamically updating the curriculum.

  • Problem Setup: The formulation models n facial photos from m subjects using feature vectors and m one-vs-all classifiers.Classifier parameters, labels, and sample weights are represented for each subject-specific decision function.
  • Incremental Setting: ASPL is designed for settings where most samples begin unlabeled and the data scale can continue growing incrementally.A minority of samples is manually annotated while a majority is pseudo-labeled during learning.
  • Model Variables: The optimization treats classifier parameters, sample labels, importance weights, and active annotated-sample constraints as jointly learned variables.The sample weights reflect training importance, while pace ages control each classifier’s learning pace.
  • Curriculum Constraints: Pseudo-labeled samples receive learnable weights in [0, 1], whereas samples annotated through active learning receive deterministically fixed weights of 1.These two curriculum cases distinguish self-paced weighting from user-confirmed supervision.
  • Curriculum Constraints: The curriculum constraint changes dynamically with the pace ages of all classifiers, unlike previous self-paced learning settings.This dynamic constraint is presented as a central advantage of the model.
  • Optimization Procedure: The algorithm alternates weighted SVM classifier updates, closed-form pseudo-label updates, SPL importance weighting, and active annotation of low-confidence samples.Increasing the pace parameter also updates the curriculum constraint and feature representation through CNN fine-tuning.
  • Optimization Procedure: The self-paced weighting favors high-confidence samples with smaller losses, implementing a learning progression from easy to hard.The paper uses a linear soft-weighting regularizer in its experiments and derives an analytical weight solution.
  • Pseudo-Labeling: Positive-weight high-confidence samples are selected by reranking and assigned pseudo-labels according to the current classifiers.Only samples with positive importance weights are meaningful for this pseudo-labeling step.

1) Annotated Sample Verifying:

ASPL verifies uncertain annotations through active user intervention and progressively updates classifiers, curriculum variables, and CNN feature representations.

  • Annotated Sample Verifying: Lower-confidence annotated samples are flagged for active-user verification because they are more likely to be incorrectly labeled.The verification step uses current-classifier prediction scores to identify annotations needing review.
  • Annotated Sample Verifying: Samples predicted with more than two positive class labels are treated as ambiguous low-confidence samples requiring manual annotation.The criterion is based on disagreement or ambiguity among the current one-vs-all classifiers.
  • Annotated Sample Verifying: Verified samples receive curriculum value 1, updated labels, and membership in the currently annotated set.These assignments preserve the curriculum constraints while incorporating newly corrected supervision.
  • Annotated Sample Verifying: Newly identified classes are initialized independently using nearest-neighbor sample expansion and the same ASPL updating process.Active annotation enriches positive samples before classifiers for unseen classes are initialized and updated.
  • Annotated Sample Verifying: After several ASPL iterations, manually labeled and self-annotated samples fine-tune the CNN to improve feature representations.The implementation generally performs CNN fine-tuning after about 50 SPL and AL rounds with learning rate 0.001.
  • Annotated Sample Verifying: The algorithm alternates weighted-SVM classifier updates, pseudo-labeling, SPL importance weighting, and AL annotation updates.The first three updates use global optima of corresponding subproblems, while low-confidence annotations are supplied through active learning.
  • Annotated Sample Verifying: Pace parameters increase heuristically using classifier accuracy and stop updating after an early-iteration threshold.The reported settings are λ0=0.2, α=0.08, and τ=12.

B. Relationship with Other SPL/AL Models

ASPL contains traditional SPL and AL regimes as special cases while combining their complementary sample-selection strategies.

  • Relationship with Other SPL/AL Models: Fixing curriculum and feature representations reduces ASPL to a traditional SPL model, while fixing SPL parameters and omitting pseudo-labels yields an AL regime.The model is therefore presented as a generalization of both prior learning formulations.
  • Relationship with Other SPL/AL Models: High-confidence self-paced samples reduce annotation burden, while low-confidence active annotations support learning from fewer initial labels.The paper attributes these complementary benefits to the two sampling mechanisms.

V. EXPERIMENTS

The experiments evaluate ASPL using datasets, implementation settings, and comparisons with existing approaches.

  • V. EXPERIMENTS: The experimental section introduces datasets and implementation settings before discussing results and comparisons with existing methods.This section provides the organization for the evaluation rather than reporting a result itself.

A. Datasets and Setting

ASPL is evaluated on two challenging face datasets using incremental training, rank-one identification accuracy, and annotation effort as key evaluation dimensions.

  • A. Datasets and Setting: The experiments use CACD and CASIA-WebFace-Sub, two public datasets for challenging face-recognition evaluation.The datasets contain Internet-collected images with variation in pose, illumination, age, occlusion, viewpoints, and scenes.
  • A. Datasets and Setting: CACD contains 2,000 celebrities, while the evaluated augmented subset contains 56,138 images from 500 annotated individuals.The subset combines 200 previously annotated celebrities with 300 additionally labeled individuals.
  • A. Datasets and Setting: For each dataset, 80% of every individual’s images form unlabeled training data and the remainder forms the test set.A random number n of training samples per person is annotated to initialize the incremental process.
  • A. Datasets and Setting: Performance is evaluated by rank-one face-identification accuracy and the amount of user annotation during incremental learning.ASPL is compared with active-learning algorithms and baseline methods under the same setting.
  • A. Datasets and Setting: The implementation applies CNN fine-tuning to ASPL and comparison methods including CPAL, CCAL, AL RAND, and AL ALL.The cited setting specifies that CNN fine-tuning is also used to improve the feature extractor for these methods.
  • A. Datasets and Setting: AlexNet is used for feature extraction, with only a few CNN updates during ASPL iterations and learning rate 0.001 for all layers.CNN updates generally occur after about five rounds of SPL and AL updating, with no more than five steps per update.

B. Experimental Comparisons

ASPL reaches competitive recognition accuracy with substantially fewer annotations than competing active-learning methods on CACD and CASIA-WebFace-Sub. It also remains competitive as more unlabeled samples are incorporated.

  • 40% and 45% annotations suffice for ASPL to achieve state-of-the-art performance on CACD and CASIA-WebFace-Sub, respectively.AL RAND, CCAL, and CPAL require about 81% and 65% annotations, respectively, for the corresponding datasets.
  • At equal annotation amounts, ASPL outperforms competing methods in recognition accuracy on both datasets.The comparison includes AL RAND, CCAL, CPAL, and AL ALL across increasing annotation levels.
  • ASPL maintains superior performance as iterations continue, with similar trends on CACD and CASIA-WebFace-Sub.The experiments track recognition accuracy while progressively incorporating unlabeled samples.
  • RAND and CCAL achieve relatively close performance because many samples have low prediction confidence and are dispersed in feature space.This distribution makes randomized selection comparable to certainty-based CCAL selection.

C. Component Analysis

Component analyses show that active selection, self-paced learning, and CNN fine-tuning each contribute to ASPL, while the framework also handles initialization variation, new classes, and noisy samples. The experiments further examine annotation demand and pseudo-labeling behavior across CASIA-WebFace-Sub settings.

  • Component contributions: Each of active learning, self-paced learning, and CNN fine-tuning improves recognition accuracy in the ASPL variants.The comparison uses ASPL, ASPL (w/o FT), ASPL (w/o AL), and ASPL (w/o SPL), with AL ALL and AL ALL (w/o FT) as baselines.
  • Component contributions: The SPL component significantly improves accuracy and reduces annotations by automatically exploiting high-confidence samples for feature learning.These samples are incorporated through self-paced selection rather than additional user annotation.
  • Component contributions: CNN fine-tuning dramatically improves recognition accuracy during the early training steps.The paper attributes this to information gain from individual appearance diversity decreasing as new samples are progressively introduced.
  • Initialization analysis: ASPL reaches stable results across different initial samples, whereas SPL varies substantially as training continues.The comparison indicates that the active-learning component helps handle poor initialization, and the model is insensitive to initial-sample diversity and quantity.
  • New-class handling: ASPL handles unseen new classes without substantial performance loss, and can achieve slightly better performance than ASPL (ALL).ASPL (ALL) initializes all person-class classifiers at the beginning, whereas ASPL incrementally receives classes.
  • Annotation demand: ASPL requires relatively more annotations in early iterations, but annotation needs decrease as training continues and the model matures.This large-scale analysis is conducted on CASIA-WebFace-Sub, with iterations on the horizontal axis and required user annotation on the vertical axis.
  • Robustness analysis: With verification, ASPL recovers from noisy images slightly faster and eventually approaches accuracy obtained on clean data.Noise is introduced either in initial samples or during training; without verification, recovery is slower.

VI. CONCLUSIONS

The paper introduces an active self-paced framework for incremental face identification and provides an optimization-based interpretation of its pipeline. Evaluation on challenging scenarios reports very promising results.

  • The framework progressively annotates and selects unlabeled samples in an active self-paced process for incremental face identification.

APPENDIX

The appendix develops lemmas and a theorem to characterize solutions of the paper's optimization problem. The proof considers classifier confidence cases and the resulting label assignments.

  • The hinge loss of x_i is used in the optimization analysis for classifier j.
  • The appendix introduces two lemmas before giving the solution of Eqn. (11).
  • For a classifier score below zero, the associated label solution is negative; above zero, it is positive, with both labels allowed at zero.
  • The theorem separates cases where all classifier scores are negative, one score is positive, or multiple scores are positive.
  • The proof concludes that the constructed solution is globally optimal under the stated constraint.
  • When multiple positive-confidence scores occur, the solution selects the classifier index producing the minimum objective value.
Loading 1701.03555v2…