Source-linked AI summary
An Enhanced Deep Feature Representation for Person Re-identification
Shangxuan Wu, Ying-Cong Chen, Xiang Li, An-Cong Wu, Jin-Jie You, Wei-Shi Zheng
TL;DR
Person re-identification requires representations that remain useful despite large cross-view appearance changes, while hand-crafted and CNN features may offer complementary information. The paper proposes FFN, which uses hand-crafted color and texture features to regularize CNN feature learning, and reports higher Rank-1 accuracy than state-of-the-art models on VIPeR, CUHK01, and PRID450s.
Problem
Large changes in view angle, lighting, background clutter, and occlusion challenge reliable person re-identification, while hand-crafted and CNN features provide potentially complementary information.
Method
FFN jointly processes CNN and hand-crafted color and texture features, using back propagation to regularize CNN learning toward complementary features and extracting a 4096D image representation.
Results
8.09%, 7.98%, and 11.2% higher Rank-1 accuracy than state-of-the-art models is reported on VIPeR, CUHK01, and PRID450s, respectively.
Takeaways & Limitations
The experiments support using hand-crafted features to improve CNN feature extraction and produce a more robust person image representation.
Abstract
from arXiv · showhide
Feature representation and metric learning are two critical components in person re-identification models. In this paper, we focus on the feature representation and claim that hand-crafted histogram features can be complementary to Convolutional Neural Network (CNN) features. We propose a novel feature extraction model called Feature Fusion Net (FFN) for pedestrian image representation. In FFN, back propagation makes CNN features constrained by the handcrafted features. Utilizing color histogram features (RGB, HSV, YCbCr, Lab and YIQ) and texture features (multi-scale and multi-orientation Gabor features), we get a new deep feature representation that is more discriminative and compact. Experiments on three challenging datasets (VIPeR, CUHK01, PRID450s) validates the effectiveness of our proposal.
1. Introduction
Person re-identification must handle substantial cross-view appearance variation, motivating a representation that combines complementary hand-crafted and CNN features. The proposed FFN uses hand-crafted features to regularize CNN learning and improves Rank-1 matching rates across three datasets.
- Cross-view changes in view angle, lighting, background clutter, and occlusion make person appearance difficult to match reliably.
- Hand-crafted appearance features such as RGB, HSV, and LBP can sometimes provide more distinctive and reliable representations under these variations.
- FFN combines multi-colorspace hand-crafted features with CNN features, using the former to regularize CNN learning toward complementary representations.
- 8.09%, 7.98% and 11.2% Rank-1 matching-rate improvements are reported on VIPeR, CUHK01, and PRID450s, respectively, over state-of-the-art methods.
2. Related Works
Prior work combines handcrafted descriptors and deep networks for person re-identification, often using pairwise inputs for matching. This paper contrasts those approaches with FFN, which extracts deep features from a single image for use with conventional classifiers.
- Hand-crafted Features: Color and texture descriptors, including HSV/LAB histograms, LBP histograms, and Gabor filters, are widely combined to improve image representations.These features measure image color and texture information.
- Hand-crafted Features: Person re-identification methods use specialized descriptors such as Fisher Vectors, color distributions, and symmetry-driven local-feature accumulation.The cited methods target improved matching or recognition performance through distinct handcrafted representations.
- Deep Learning: CNN-based re-identification research includes architectures for pose and viewpoint variation, cross-input neighborhood differences, and siamese deep metric learning.FPNN uses patch matching and maxout pooling, while later methods introduce cross-input comparison and cosine-based metric learning.
- Deep Learning: Existing deep methods combine feature extraction with image-pair classification, relying on pairwise comparison and symmetry structures.These approaches require forming many pairs for each probe image and performing deep convolution on the pairs.
- Novelty: FFN directly extracts deep features from a single image rather than using pairwise inputs, allowing its architecture to be followed by conventional classifiers.The paper presents this single-image feature-extraction design as a contrast with existing deep learning works.
3. Methodology
FFN combines CNN processing with ELF16 hand-crafted features so back propagation regularizes CNN feature extraction toward complementary representations. The resulting fused 4096D description supports conventional metric learning and is trained with softmax loss.
- Network architecture: FFN links CNN processing with a hand-crafted-feature subnetwork to produce a fused 4096D image description.The hand-crafted branch regularizes the CNN branch during learning.
- CNN features: CNN features are extracted through convolution and pooling, with the fifth pooling layer producing a 4096D vector.The network uses local response normalization after pooling layers except the third layer.
- Hand-crafted features: ELF16 partitions images into 16 horizontal stripes and concatenates normalized color histograms with Gabor, Schmid, and LBP texture features.The color spaces are RGB, HSV, LAB, XYZ, YCbCr, and NTSC; each channel uses a 16D histogram normalized by L1-norm.
- Fusion and optimization: The fusion layer uses full connections, while 4096D buffer layers bridge the CNN and ELF16 representations and are described as essential for convergence.The fusion computation applies an activation function, ReLU, and dropout with ratio 0.5.
- Training objective: Softmax loss trains FFN for discriminative single-image feature extraction rather than pairwise comparison.The number of output nodes varies with the training set.
- Feature interaction: Back propagation allows ELF16 features to influence CNN parameters through both parameter-gradient propagation and the softmax-loss output.This design adapts convolution filters according to ELF16 information while optimizing the final objective.
4. Settings for Feature Fusion Network
The Feature Fusion Network was trained on Market-1501 using mini-batch SGD and hard negative mining, then evaluated with CMC curves alongside metric-learning methods.
- 4.1. Training Dataset: Market-1501 supplied the training images for the Feature Fusion Network used to extract features.The dataset contains 38,195 images from 1,501 identities.
- 4.2. Training Strategies: Mini-batch stochastic gradient descent trained the network with 25 images per iteration and a small initial learning rate of 1e−5.The learning rate was multiplied by 0.1 every 20,000 iterations, and the network was fine-tuned from an ImageNet model.
- 4.2. Training Strategies: Hard negative mining selected difficult samples for further fine-tuning of the Feature Fusion Network.Approximately 12,000 images from 630 identities were manually selected after being wrongly labeled by the previous network.
- 4.3. Evaluation: CMC curves compared features on three datasets using L1-norm, LFDA, and Mirror KMFA evaluation methods.The yellow curves in the last row represent the final model used in Section 5.4.
5. Experiments
Experiments evaluate FFN features across three person re-identification datasets, using repeated train/test splits, unsupervised distance evaluation, metric learning, and comparisons with established features and models.
- Datasets and protocols: Experiments use VIPeR, CUHK01, and PRID450s, each containing disjoint camera views with misalignment, lighting changes, and body-part distortion.Half of the identities were used for training and half for testing; results were averaged over 10 repetitions.
- Unsupervised method: FFN features significantly outperform other stand-alone features under L1-norm evaluation across the three datasets.This evaluates representation quality without supervised metric learning.
- Unsupervised method: ELF16+CNN-FC7 ranks second and exceeds either component alone, supporting complementarity between handcrafted and CNN features.The proposed features outperform this simple concatenation, while Buffer and Fusion layers automatically tune feature weights.
- Metric learning methods: Ours+LOMO achieves the strongest performance with Mirror KMFA and is used as the final 31056D image representation.The concatenation is reported to have strong discriminative ability and to outperform all other compared features on Mirror KMFA.
- Comparison with state-of-the-art: The complete model beats several state-of-the-art methods by about 10% in Rank-1 matching rate and exceeds deep-learning competitors by 7.98% on CUHK01 and 11.2% on PRID450s.The model combines FFN features with normalized LOMO features and uses Mirror KMFA.
6. Conclusion
FFN jointly uses CNN and hand-crafted features, with back propagation regularizing CNN extraction toward complementary representations. On three datasets, the resulting features significantly outperform state-of-the-art models in Rank-1 accuracy.
- FFN jointly utilizes CNN and hand-crafted color and Gabor texture features for person re-identification.
- Back propagation automatically adjusts information weights and regularizes CNN extraction toward complementary features.
- 8.09%, 7.98%, and 11.2% Rank-1 accuracy improvements are achieved on VIPeR, CUHK01, and PRID450s, respectively.