Source-linked AI summary
Nuclear Norm based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes
Jian Yang, Jianjun Qian, Lei Luo, Fanlong Zhang, Yicheng Gao
TL;DR
Existing regression methods model face-image errors pixel by pixel, neglecting low-rank structure associated with occlusion and illumination. The paper proposes Nuclear Norm based Matrix Regression, which performs matrix regression using nuclear-norm residual minimization and an alternating direction method. NMR is reported as more robust for illumination changes and more powerful for structural occlusion noise, while computational speed remains a constraint for real-world applications.
Problem
Existing regression methods use one-dimensional pixel-based errors and neglect low-rank error-image structure arising from occlusion and illumination.
Method
NMR uses two-dimensional matrix regression, minimizes the nuclear norm of the representation residual image, and computes coefficients with an alternating direction method.
Results
NMR is reported as more robust than state-of-the-art regression methods under occlusions and illumination changes, and more powerful than structured sparse methods for occlusion.
Takeaways & Limitations
NMR uses low-rank structural information to support face recognition under occlusions and illumination changes, with satisfying results also reported for general face recognition.
Takeaways & Limitations
Although faster than many robust regression methods, NMR remains too slow for some real-world applications.
Abstract
from arXiv · showhide
Recently regression analysis becomes a popular tool for face recognition. The existing regression methods all use the one-dimensional pixel-based error model, which characterizes the representation error pixel by pixel individually and thus neglects the whole structure of the error image. We observe that occlusion and illumination changes generally lead to a low-rank error image. To make use of this low-rank structural information, this paper presents a two-dimensional image matrix based error model, i.e. matrix regression, for face representation and classification. Our model uses the minimal nuclear norm of representation error image as a criterion, and the alternating direction method of multipliers method to calculate the regression coefficients. Compared with the current regression methods, the proposed Nuclear Norm based Matrix Regression (NMR) model is more robust for alleviating the effect of illumination, and more intuitive and powerful for removing the structural noise caused by occlusion. We experiment using four popular face image databases, the Extended Yale B database, the AR database, the Multi-PIE and the FRGC database. Experimental results demonstrate the performance advantage of NMR over the state-of-the-art regression based face recognition methods.
1. Introduction
Existing regression-based face recognition methods model errors pixel by pixel, overlooking correlated and low-rank structure caused by occlusion and illumination. NMR preserves this image structure by using matrix regression and nuclear-norm residual minimization.
- Limitations of Existing Models: Existing robust regression methods use a one-dimensional pixel-based error model that characterizes representation errors independently.This model assumes pixel errors are independent and identically distributed, which may hold for random corruption but not many occlusions.
- Limitations of Existing Models: Correlated errors arise in occluded image regions because neighboring pixels in local image areas are generally highly correlated.Experiments with randomly located 45x45 occlusions show that recovered occluded pixels are not independent.
- Low-Rank Error Structure: Pixelwise error modeling neglects structural information in the error image, such as its rank, making it theoretically questionable for occluded-image classification.The omitted structure is relevant because illumination changes and occlusions generally produce low-rank error images.
- Proposed NMR Model: The paper introduces a two-dimensional matrix regression model that avoids matrix-to-vector conversion and uses image residual structure directly.The model minimizes the rank-related structure of the representation residual and is named nuclear norm based matrix regression.
- Proposed NMR Model: NMR uses the nuclear norm of the representation residual image as its criterion and calculates regression coefficients with an alternating direction method.Its framework integrates error detection and error support into one model with one easily tuned, relatively database-insensitive parameter.
- Claimed Advantages: NMR is described as more robust for illumination changes and more powerful for structural occlusion noise than state-of-the-art regression-based classifiers.The paper contrasts nuclear-norm robustness with methods using Euclidean norms and highlights direct noise recovery without a dictionary.
2. Nuclear Norm Based Matrix Regression
NMR formulates face representation as matrix regression, modeling the residual image explicitly and replacing rank minimization with nuclear-norm minimization plus regularization. ADMM solves the resulting problem through alternating updates, with convergence to a Lagrangian saddle point under μ>0.
- Problem formulation: NMR represents a test image as a linear combination of training image matrices plus a residual image, preserving two-dimensional structure.The matrix model contrasts with vector regression, whose equivalent formulation applies Vec to the image matrices.
- Problem formulation: Because representation residuals are typically low rank, NMR replaces rank minimization with nuclear-norm minimization.The nuclear norm provides a tractable surrogate for the residual image's rank.
- Problem formulation: The regularized model combines the nuclear norm of the residual with an ℓ2 penalty on the regression coefficients.The coefficient regularizer is introduced by analogy with Ridge regression.
- Convergence analysis: For μ>0, the sequence generated by Algorithm 1 converges to a saddle point of the Lagrangian.The paper presents this as the convergence guarantee for the ADMM procedure.
3. NMR based Classification
The NMR classifier uses nuclear norms of class-specific residual images to measure similarity and exploits low-rank residual structure for occlusion recovery. Experiments show improved robustness under difficult illumination and larger or real-world occlusions.
- Similarity observations: NMR classifies a test face by comparing nuclear norms of residual images reconstructed from different classes.The class with the smaller residual norm supplies the similarity-based classification criterion.
- Similarity observations: The nuclear norm is presented as less sensitive to lighting changes than the Euclidean norm used by several regression classifiers.The paper links this difference to the residual-image structure under illumination variation.
- Occlusion observations: NMR provides a mechanism for removing or alleviating occlusion effects in test images by treating the residual as the recovered occluded part.This follows the paper's occlusion assumption that the residual image is low rank.
- Similarity observations: For an extreme illumination example, NMR preserves the correct class ordering, with ||E22*||_NMR = 11.02 < ||E21*||_NMR = 11.34.Ridge reverses this ordering, producing ||E22||_Ridge = 4.54 > ||E21||_Ridge = 4.46.
- Occlusion observations: For 25x25 block occlusion affecting 31.25% of pixels, NMR performs as well as SR and better than Ridge and robust regression.The comparison concerns recovery of both the clean face and the occluded region.
- Occlusion observations: At 43.75% occlusion from a 35x25 block, NMR still recovers the clean face well, whereas the other methods fail to achieve good results.For scarf occlusion, NMR significantly outperforms the other methods in recovering the clean image.
4. Experiments
Experiments evaluate NMR on four face databases against established regression-based classifiers using original images without preprocessing or feature extraction. Across illumination, occlusion, and general recognition settings, NMR generally achieves the strongest robustness and recognition performance, while offering favorable runtime and parameter behavior.
- Experimental setup: Experiments use the Extended Yale B, AR, Multi-PIE, and FRGC databases to compare NMR with state-of-the-art linear representation classifiers.Compared methods include LRC, CRC, SRC, CESR, RSC, SSEC, HQ_A, and HQ_M.
- Experimental setup: All experiments use original face images without image preprocessing or feature extraction, with method parameters selected by tuning or author recommendations.The default NMR regression parameter is 1.
- Recognition with occlusion: Under contiguous occlusion, NMR significantly outperforms SRC, RSC, HQ_M, and SSEC when occlusion reaches at least 50%.The test images contain randomly located square baboon-image blocks ranging from 10% to 60% occlusion.
- Recognition with occlusion: At 60% occlusion in one test setting, NMR reaches 57.3%, 6.2%, and 4.0% higher recognition rates than SRC, RSC, and SSEC, respectively.In another setting, NMR reaches 86.4%, exceeding SSEC by 4.1% and RSC by 22.8%.
- Recognition with different illumination: NMR achieves the best results on extreme-illumination subsets, while CESR, HQ_A, HQ_M, and SSEC are less robust in those conditions.SSEC is designed for contiguous occlusion and is not suitable for extreme illumination changes.
5. Conclusions and Future Work
The paper presents NMR with an augmented Lagrange multipliers method and evaluates it on four face databases. Results indicate robustness across occlusion, illumination changes, and general face recognition, while computational efficiency and general-noise applicability remain open issues.
- NMR uses the augmented Lagrange multipliers method to calculate regression coefficients, with convergence analysis provided.
- The NMR classifier is examined on the Extended Yale B, AR, Multi-PIE, and FRGC face image databases.
- NMR is more robust than state-of-the-art regression methods for face recognition with occlusions and illumination changes.
- NMR is more powerful than the structured sparse error coding model, which was designed for occlusion but is sensitive to illumination changes.
- NMR achieves satisfying results for general face recognition tasks without occlusion.
- NMR remains constrained by speed for real-world applications, and computational efficiency and effectiveness for more complex or general noise require further investigation.