Source-linked AI summary
Region-based Quality Estimation Network for Large-scale Person Re-identification
Guanglu Song, Biao Leng, Yu Liu, Congrui Hetang, Shaofan Cai
TL;DR
Video person re-identification is challenged by noisy, partially occluded regions and by the scarcity of clean, large-scale datasets. The paper proposes RQEN to estimate regional quality and aggregate complementary information across frames, and introduces LPW as a large, clean benchmark. RQEN reports state-of-the-art results on PRID 2011 and iLIDS-VID, while LPW contains 7,694 tracklets and over 590,000 images.
Problem
Video person re-identification must handle noisy regional information from occlusion and other disturbances, while existing datasets are often small or insufficiently clean.
Method
RQEN jointly learns regional quality and aggregates complementary high-quality regions across frames into a video-level representation, alongside the LPW dataset contribution.
Results
RQEN reports state-of-the-art results on PRID 2011 and iLIDS-VID and comparable results on MARS; LPW contains 7,694 tracklets and over 590,000 images.
Takeaways & Limitations
The work provides a regional aggregation approach for noisy video sequences and a large, clean, challenging benchmark for further person re-identification research.
Abstract
from arXiv · showhide
One of the major restrictions on the performance of video-based person re-id is partial noise caused by occlusion, blur and illumination. Since different spatial regions of a single frame have various quality, and the quality of the same region also varies across frames in a tracklet, a good way to address the problem is to effectively aggregate complementary information from all frames in a sequence, using better regions from other frames to compensate the influence of an image region with poor quality. To achieve this, we propose a novel Region-based Quality Estimation Network (RQEN), in which an ingenious training mechanism enables the effective learning to extract the complementary region-based information between different frames. Compared with other feature extraction methods, we achieved comparable results of 92.4%, 76.1% and 77.83% on the PRID 2011, iLIDS-VID and MARS, respectively. In addition, to alleviate the lack of clean large-scale person re-id datasets for the community, this paper also contributes a new high-quality dataset, named "Labeled Pedestrian in the Wild (LPW)" which contains 7,694 tracklets with over 590,000 images. Despite its relatively large scale, the annotations also possess high cleanliness. Moreover, it's more challenging in the following aspects: the age of characters varies from childhood to elderhood; the postures of people are diverse, including running and cycling in addition to the normal walking state.
Introduction
Video person re-identification must handle noisy, partially occluded frames and limited clean, large-scale datasets. RQEN addresses both by aggregating complementary regional information and introducing the LPW benchmark.
- Motivation: Video re-identification performance is affected by occlusion, body movement, blur, and other noise that can weaken extracted features.Sequence frames provide complementary information, but simple averaging can also incorporate undesirable noise.
- RQEN: RQEN uses complementary information from one frame’s high-quality region to compensate for the same region’s poor information in other frames.This region-level aggregation is more selective than assigning one quality score to an entire image.
- RQEN: RQEN estimates the quality of different image regions and jointly trains its regional components to emphasize reliable regions across frames.Higher-confidence regions contribute more to the sequence representation, while occluded or noisy regions contribute less.
- Dataset: Existing person re-identification datasets are often either small, insufficiently clean, or based on unrealistically aligned bounding boxes.The paper motivates a dataset that is large, clean, and closer to realistic detection settings.
- Dataset: LPW contains 2,731 pedestrians, 7,694 tracklets, and over 590,000 images captured across three scenes and multiple cameras.Its stated characteristics include clean tracklets, automatically detected bounding boxes, crowded scenes, and greater age variation.
- Contributions: The paper contributes RQEN, a jointly trained regional aggregation pipeline, and the large, clean LPW person re-identification dataset.The stated RQEN contribution targets regional quality estimation and complementary information aggregation in sequences.
Related Works
Prior person re-identification work develops feature extractors, metric-learning methods, or sequence aggregation strategies, but does not selectively combine complementary information from corresponding sub-regions across frames. RQEN is presented as an end-to-end alternative that learns regional quality and aggregates those regions into a video-level representation.
- Existing approaches: Person re-identification research has mainly focused on improving feature extraction or learning more discriminative distance metrics.These approaches address representation and matching from complementary methodological directions.
- Research gap: Prior sequence methods use multiple images or selected subsets but do not exploit complementary information from corresponding sub-regions across different frames.This identifies the regional aggregation gap targeted by RQEN.
- RQEN: RQEN learns regional quality and selectively aggregates high-quality sub-regions from a sequence through an end-to-end architecture.The resulting representation is described as having smaller intra-class variance and larger inter-class distance.
- RQEN: RQEN’s inference pipeline processes an image set through convolutional features, regional feature generation, quality prediction, and set aggregation.The final output is a video-level representation formed from region scores and features.
Proposed Method
RQEN divides pedestrian images into fixed body regions, predicts region quality, and aggregates sequence features with higher-quality regions weighted more strongly. Joint supervision of frame-level identity features and video-level triplet features trains the model to exploit complementary regional information while reducing noise.
- Architecture Overview: RQEN generates upper, middle, and lower regional features from a fully convolutional representation using landmark-assisted average pooling.Because individual landmarks can be unreliable, the method uses a generally fixed three-region division derived from landmark distributions.
- Learning Complementary Information Via Joint Training Frame-level and Video-level Features: RQEN jointly trains frame-level and video-level features using Softmax loss for pedestrian identities and Triplet loss for video-level representations.Identity supervision brings frame-level features closer to pedestrian centers, while triplet supervision promotes effective regional information and stronger inter-class separation.
- Architecture Overview: The region-based quality predictor assigns scores to regional representations so noisy or occluded areas contribute less during sequence aggregation.The predictor is designed to preserve useful regional information rather than discard an entire frame affected by localized noise.
- Set Aggregation Unit: The set aggregation unit maps variable-length image sequences to fixed-dimensional representations by weighting each region according to its predicted quality score.Highly scored regions contribute more information to the video-level representation, with scores normalized across frames for each region.
- Learning Complementary Information Via Joint Training Frame-level and Video-level Features: The architecture uses complementary information from different image regions across frames to weaken noise and improve pedestrian recognition.This design addresses the problem that simple sequence aggregation can incorporate unwanted noise instead of compensating for poor-quality regions.
The Labeled Pedestrian in the Wild Dataset
LPW is a clean, large-scale person re-identification dataset collected across crowded scenes and multiple cameras. Its construction combines automatic detection, tracking, and manual cleanup, while preserving challenging real-world variation.
- LPW was collected in three crowded scenes using three cameras in the first scene and four in the other two.
- Table 1 compares LPW with MARS, Market, iLIDS-VID, PRID2011, and CUHK03, including dataset aspects and detection or tracking failure.The table uses # for corresponding aspect counts and DT failure for detection or tracking failure in sequences.
- 2,731 pedestrians were captured across at least two cameras, enabling cross-camera person searches.
- 7,694 image sequences were produced through pedestrian detection, consecutive-frame overlap tracking, and manual removal of detection and tracking errors.Each sequence contains an average of 77 frames.
- LPW combines large scale and clean sequences with automatically detected bounding boxes, crowded scenes, broad age variation, frequent occlusion, and diverse postures.These characteristics are intended to reflect real-world conditions and support deep-network training.
Experiments
Experiments evaluate RQEN across video re-identification datasets, ablations, transfer learning, cross-dataset testing, and regional quality visualization. RQEN improves over strong baselines and uses quality scores to emphasize reliable, complementary regions.
- Ablation Study on iLIDS-VID and PRID2011: Joint training improves performance, while adding region-based quality produces a greater increase than joint image-level and video-level training alone.The parameter-matched +MP comparison examines whether the improvement is explained only by the increased number of parameters.
- Comparison with Published Results: RQEN increases top-1 accuracy by 1.5% on PRID 2011 and by 9.1% on iLIDS-VID compared with the state-of-the-art.The larger iLIDS-VID improvement is attributed to its frequent partial occlusion and deformation, which favor selecting complementary high-confidence regions.
- Comparison with Published Results: RQEN increases top-1 accuracy by 15.6% over the baseline on the challenging LPW dataset.The paper reports stable performance in the LPW ablation study.
- Cross-Dataset Testing: RQEN outperforms average pooling and other state-of-the-art methods in cross-dataset testing.The evaluation follows the protocol of McLaughlin, Martinez del Rincon, and Miller (2016).
- Transfer Learning Ability of LPW: Pretraining on LPW improves rank-1 accuracy by 4% on iLIDS-VID and by 1% on PRID2011 compared with ImageNet pretraining.The results support LPW’s usefulness for initial pedestrian re-identification training.
- Complementary Information Extraction: RQEN assigns lower regional quality scores to image parts affected by occlusion or blur.Figure 7 maps scores in [0,1] to colors from blue to red.
- Complementary Information Extraction: RQEN aggregates complementary information by allowing reliable regions in one frame to compensate for missing information in the same region of other frames.This produces a more robust sequence representation than average pooling.
Conclusion
The paper proposes RQEN to learn regional quality and aggregate complementary information across frames for video-based person re-identification. It reports strong results on PRID 2011 and iLIDS-VID and introduces the large, clean, challenging LPW dataset.
- Conclusion: RQEN learns each image region’s quality and aggregates complementary regional information across frames in a sequence.The network uses an end-to-end training strategy with jointly trained classification and verification signals.
- Conclusion: RQEN leads to state-of-the-art results on PRID 2011 and iLIDS-VID.
- Conclusion: LPW contains 7,694 tracklets and over 590,000 images, with occlusion, large posture changes, and a wide age span.The dataset is presented as large, clean, challenging, and suitable for practical use.