Source-linked AI summary
Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-Free Approach
Lingxiao He, Jian Liang, Haiqing Li, Zhenan Sun
TL;DR
Partial person re-identification must match images containing arbitrary body parts, but existing approaches offer limited flexible solutions and can require alignment or incur computational costs. The paper combines FCN spatial feature maps with Deep Spatial feature Reconstruction to compare different-sized images through sparse reconstruction. Experiments report effective and efficient performance on two partial-person datasets, while the supplied evidence also identifies limitations associated with prior methods and the training setup.
Problem
Partial person re-identification lacks flexible solutions for identifying people from images containing arbitrary body parts, especially under occlusion.
Method
The method uses an FCN to generate spatial feature maps and DSR to sparsely reconstruct probe features from gallery features without explicit alignment.
Results
The proposed DSR achieves impressive accuracy and efficiency on Partial-REID and Partial-iLIDS compared with state-of-the-art partial person re-identification approaches.
Takeaways & Limitations
DSR provides an alignment-free and flexible framework for matching arbitrary-sized partial person images.
Takeaways & Limitations
Prior sliding-window and part-based methods require strict alignment and repeated feature extraction, while the DSR training setup assumes paired holistic and arbitrary-patch images.
Abstract
from arXiv · showhide
Partial person re-identification (re-id) is a challenging problem, where only several partial observations (images) of people are available for matching. However, few studies have provided flexible solutions to identifying a person in an image containing arbitrary part of the body. In this paper, we propose a fast and accurate matching method to address this problem. The proposed method leverages Fully Convolutional Network (FCN) to generate fix-sized spatial feature maps such that pixel-level features are consistent. To match a pair of person images of different sizes, a novel method called Deep Spatial feature Reconstruction (DSR) is further developed to avoid explicit alignment. Specifically, DSR exploits the reconstructing error from popular dictionary learning models to calculate the similarity between different spatial feature maps. In that way, we expect that the proposed FCN can decrease the similarity of coupled images from different persons and increase that from the same person. Experimental results on two partial person datasets demonstrate the efficiency and effectiveness of the proposed method in comparison with several state-of-the-art partial person re-id approaches. Additionally, DSR achieves competitive results on a benchmark person dataset Market1501 with 83.58\% Rank-1 accuracy.
1. Introduction
Partial person re-identification addresses matching images that contain only arbitrary body regions, a setting inadequately handled by methods designed for full-person images. The proposed DSR framework uses FCN features and sparse reconstruction to avoid explicit alignment while targeting both accuracy and efficiency.
- Motivation: Partial person re-identification is needed because occlusion in real-world surveillance produces images containing only parts of a person.Occlusion can arise from moving obstacles such as cars and other people, or static obstacles such as trees and barriers.
- Existing limitations: Resizing arbitrary person patches to a fixed size can cause undesired deformation and degrade matching performance.The resizing limitation is illustrated as one of the approaches compared in Figure 2.
- Existing limitations: Sliding-window and part-based approaches require substantial computation, strict alignment, or repeated sub-region extraction without sharing intermediate results.These limitations lead to unsatisfactory computation efficiency.
- Proposed approach: DSR uses a Fully Convolutional Network to produce spatial feature maps and sparsely reconstructs probe pixels from gallery spatial maps.The framework matches person images of different sizes without explicit alignment.
- Proposed approach: The training objective minimizes reconstruction error for same-identity pairs and maximizes it for different-identity pairs.This integrates sparse reconstruction learning and deep learning in an end-to-end model.
- Results: Experimental results on Partial-REID and Partial-iLIDS show strong accuracy and efficiency compared with existing partial person re-identification approaches.The introduction reports impressive results on both databases without giving numerical values here.
2. Related Work
The paper builds on Fully Convolutional Networks for spatially dense, arbitrary-size outputs and Sparse Representation Classification for reconstruction-based recognition. Related partial person re-identification methods commonly warp arbitrary patches to fixed-size images, motivating a more flexible approach.
- Fully Convolutional Network: Fully Convolutional Networks use convolutional and pooling layers to produce spatial outputs corresponding to input dimensions.They have been applied to semantic segmentation, object detection, and visual recognition.
- Fully Convolutional Network: FCN-based methods can produce fixed-length representations from inputs of arbitrary sizes through spatial pyramid pooling.The related-work discussion identifies this as an example of FCN use in visual recognition.
- Sparse Representation Classification: Sparse Representation Classification was introduced for face recognition and has also been applied to signal classification, visual tracking, and visual classification.The paper uses this reconstruction-based family of methods as part of its deep feature-learning foundation.
- Partial Person Re-identification: Partial person re-identification remains relatively underexplored for matching arbitrary-sized images that show only part of the human body.Existing approaches commonly warp arbitrary patches to fixed-size images before extracting fixed-length features.
3. The Proposed Approach
The proposed approach uses an FCN to produce spatial feature maps from arbitrary-sized person images and DSR to compare them without explicit alignment. DSR is trained end-to-end, with multi-scale feature blocks improving robustness to scale variation while sharing computed features.
- 3.1. Fully Convolutional Network: FCN removes fully connected layers so arbitrary-sized inputs produce spatial feature maps while retaining coordinate information.This addresses fixed-size input requirements caused by fully connected layers.
- 3.2. Deep Spatial Feature Reconstruction: DSR divides probe and gallery feature maps into blocks and reconstructs each probe block as a sparse combination of gallery blocks.The reconstruction coefficients are constrained with an ℓ1-norm because few gallery blocks are expected to reconstruct each probe block.
- 3.2. Deep Spatial Feature Reconstruction: The reconstruction error between spatial feature maps provides a similarity measure for matching person images of different sizes without additional person alignment.DSR computes a matching distance from the reconstruction error using the sparse coefficient matrix.
- 3.3. Fine-tuning on Pre-trained FCN with DSR: DSR fine-tunes a pre-trained FCN through a verification signal that brings same-identity feature maps closer and separates different-identity maps.The FCN is first trained with an identification signal and then fine-tuned using DSR.
- 3.4. Multi-scale Block Representation: Multi-scale block representation uses 1×1, 2×2, and 3×3 sliding-window blocks, with larger blocks average-pooled to 1×1 dimensions.The resulting blocks are pooled into one block set to improve robustness against scale variation.
- 3.4. Multi-scale Block Representation: Feature-level multi-scale processing computes features once and shares them across block-dividing patterns, reducing the computational complexity of repeated image-level extraction.This contrasts with region-based models that repeatedly calculate features for different scales.
4. Experiments
The experiments evaluate DSR across partial person re-id datasets, representation scales, competing methods, efficiency, fine-tuning, and holistic re-id. Results support strong accuracy, multi-scale benefits, and fast matching.
- Experiment scope: Experiments cover deformation, multi-scale representation, state-of-the-art comparisons, computational time, DSR fine-tuning, and holistic person re-id.The evaluation includes Partial REID, Partial-iLIDS, and Market1501 settings.
- Multi-scale representation: DSR achieves its best performance when gallery and probe block sets combine 1 × 1, 2 × 2, and 3 × 3 blocks.The comparison evaluates three fusion choices for multi-scale block representation.
- State-of-the-art comparison: DSR outperforms AMC, SWM, AMC+SWM, and the Resizing model in single-shot experiments on Partial REID and Partial-iLIDS.The authors attribute this to FCN feature maps produced without deformation and sliding-window operation.
- Multi-shot experiments: 39.33% to 49.33% on Partial REID and 51.06% to 54.67% on Partial-iLIDS are the reported DSR improvements from single-shot to multi-shot settings.The multi-shot evaluation uses N=3 gallery images per individual.
- Computational efficiency: 0.269s and 0.278s are the reported identification times for DSR with single-scale and multi-scale block representations, respectively.The computational comparison is conducted on the Partial REID dataset.
- Fine-tuning and holistic re-id: Fine-tuning FCN with DSR is evaluated against a pre-trained FCN using ROC and CMC curves on Partial REID.The paper also reports an additional holistic person re-id evaluation on Market1501.
5. Conclusion
The paper concludes that DSR addresses partial person re-identification without fixed input size or trivial alignment. Experiments validate its effectiveness and efficiency on partial datasets and competitiveness on Market1501.
- Conclusion: DSR reconstructs probe spatial-feature channels from gallery spatial-image-map channels, avoiding fixed input size and trivial alignment-free matching.The method is embedded into FCN to learn more discriminative features.
- Conclusion: DSR minimizes reconstruction error for same-person image pairs and maximizes it for different-person pairs.The conclusion reports significant advantages over various partial person re-id approaches.
- Conclusion: Experimental results validate DSR's effectiveness and efficiency on Partial REID and Partial-iLIDS and its competitiveness on Market1501.The conclusion summarizes results across partial and holistic person re-id settings.