Source-linked AI summary
Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation
Beibei Lin, Shunli Zhang, Xin Yu
TL;DR
Gait recognition needs representations that retain both whole-body context and local gait details, because global-only and local-only features have complementary weaknesses. The paper introduces GLFE with GLConv layers and LTA in a 3D CNN framework, and reports state-of-the-art performance on CASIA-B and OUMVLP.
Problem
Global features may neglect gait details, while local features may lose global context and relations among local regions.
Method
The method combines global and local feature extraction through GLFE and GLConv layers, and uses LTA to aggregate temporal information while preserving spatial information.
Results
The proposed method achieves state-of-the-art performance on the CASIA-B and OUMVLP public datasets, especially in complex conditions.
Takeaways & Limitations
The framework produces more comprehensive and discriminative gait representations by combining global and local information with spatial-information-preserving temporal aggregation.
Takeaways & Limitations
The weighted-sum feature-mapping strategy is inflexible because its trade-off parameters are manually predefined.
Abstract
from arXiv · showhide
Gait recognition is one of the most important biometric technologies and has been applied in many fields. Recent gait recognition frameworks represent each gait frame by descriptors extracted from either global appearances or local regions of humans. However, the representations based on global information often neglect the details of the gait frame, while local region based descriptors cannot capture the relations among neighboring regions, thus reducing their discriminativeness. In this paper, we propose a novel feature extraction and fusion framework to achieve discriminative feature representations for gait recognition. Towards this goal, we take advantage of both global visual information and local region details and develop a Global and Local Feature Extractor (GLFE). Specifically, our GLFE module is composed of our newly designed multiple global and local convolutional layers (GLConv) to ensemble global and local features in a principle manner. Furthermore, we present a novel operation, namely Local Temporal Aggregation (LTA), to further preserve the spatial information by reducing the temporal resolution to obtain higher spatial resolution. With the help of our GLFE and LTA, our method significantly improves the discriminativeness of our visual features, thus improving the gait recognition performance. Extensive experiments demonstrate that our proposed method outperforms state-of-the-art gait recognition methods on two popular datasets.
1. Introduction
Gait recognition faces performance challenges under changing external conditions, while existing global-only, local-only, and fixed-length 3D representations have complementary limitations. The paper proposes GLFE and LTA within a 3D CNN framework to learn more discriminative representations.
- Changing clothing, carrying conditions, viewpoint, speed, and resolution make robust gait recognition in complex environments desirable.
- Global representations may miss gait details, whereas local representations may lose global context and relations among local regions.
- The proposed GLFE extracts and combines global and local gait information within a 3D CNN framework to obtain discriminative representations.
- GLConv uses separate global and local branches, with the global branch modeling entire gait appearance and the local branch emphasizing gait details.
- LTA aggregates temporal information in local clips while preserving spatial information by replacing a traditional spatial pooling layer.
- The method is evaluated on CASIA-B and OUMVLP, with experiments reported as achieving state-of-the-art performance, especially in complex conditions.
2. Related Work
Earlier gait recognition work includes 3D human modeling, view normalization or transformation, and CNN-based global or local feature extraction. These approaches provide useful information but face practical, information-loss, coverage, or relational limitations.
- 3D human models use multiple cameras to reconstruct gait across views, but they are difficult to build in real scenes and computationally complex.
- Hand-crafted view-invariant features normalize different views but can lose gait information during transformation.
- View Transformation Models learn correlations between views, but fully collecting all human gait view angles is difficult in reality.
- CNN-based gait recognition uses 2D or 3D convolutional networks to improve feature representation ability.
- Local-feature methods capture more detailed information than global features, but they do not attend to relations among local regions.
3. Proposed Method
The proposed method combines local temporal aggregation with global-local feature extraction and adaptive spatial pooling to preserve gait information and produce discriminative representations. It is trained with combined triplet and cross-entropy losses, then evaluated by matching probe features against gallery features.
- Local Temporal Aggregation: LTA replaces the first spatial pooling layer to aggregate temporal information in local clips while preserving more spatial information.It leverages temporal resolution to attain higher spatial resolution in gait features.
- Global and Local Feature Extractor: The GLFE module uses multiple GLConv layers to ensemble global gait appearance and local gait details.Global and local feature maps are extracted with 3D convolutions and combined by element-wise addition or concatenation.
- Global and Local Feature Extractor: GLConv partitions global feature maps into n local parts, applies shared convolutional weights to local maps, and combines global and local outputs.The first GLConv blocks use addition, while the last block uses concatenation.
- Feature Mapping: GeM pooling integrates spatial information adaptively using a learnable parameter p before separate fully connected layers aggregate channel information.p = 1 gives average pooling, while p →∞ gives max pooling.
- Loss Function: The model jointly uses triplet and cross-entropy losses, applying the combined loss independently to each horizontal output feature.Triplet loss increases inter-class distance and reduces intra-class distance, while cross entropy identifies human IDs.
- Testing: At testing, flattened output features from probe sequences are matched against gallery features using Euclidean distance to calculate Rank-1 accuracy.The gallery is treated as the standard view to be retrieved.
4. Experiments
Experiments on CASIA-B and OUMVLP evaluate recognition accuracy, dataset-scale effects, feature components, spatial mapping, and downsampling choices. The proposed method achieves strong results, especially under challenging conditions, while ablations support combining global-local features, GeM pooling, and LTA+SP.
- Datasets and implementation: The experiments use CASIA-B and OUMVLP, with silhouettes extracted using the same preprocessing approach as prior work.CASIA-B contains 124 subjects, 10 groups per subject, and 11 views per sequence; OUMVLP uses 10,307 subjects split into training and test groups.
- Comparison with state-of-the-art methods: 91.8% average accuracy on CASIA-B surpasses GaitSet by 7.6% and GaitPart by 3.0% across NM, BG, and CL conditions.The proposed method achieves 97.4%, 94.5%, and 83.6% in NM, BG, and CL, respectively, exceeding GaitPart by 1.2%, 3.0%, and 4.9%.
- Comparison with state-of-the-art methods: 86.0%, 95.9%, and 97.4% accuracy under CASIA-B NM at ST, MT, and LT exceed GaitSet by 6.5%, 3.9%, and 2.4%.The largest improvement occurs with the small-scale dataset setting.
- Comparison with state-of-the-art methods: The proposed method achieves the best recognition performance in most OUMVLP cases under the same protocols used by GaitSet and GaitPart.OUMVLP evaluation uses 5,153 subjects for training and the remaining subjects for testing, with Seq#00 as probe and Seq#01 as gallery.
- Ablation study: 97.4% accuracy with both GLConv global and local branches exceeds using only the global branch by 0.3% and only the local branch by 1.2%.The global-only and local-only accuracies are 97.1% and 96.2% in NM; GLConv also improves BG and CL performance.
- Ablation study: 91.8% accuracy with GeM pooling exceeds Max pooling at 91.3%, while LTA+SP reaches 97.4% versus 96.3% with two spatial pooling layers.Replacing both spatial pooling layers with LTA causes a heavy accuracy decrease, so LTA+SP is selected as the final downsampling strategy.
5. Conclusion
The proposed 3D-convolutional gait recognition framework combines global and local feature extraction with temporal aggregation to produce discriminative representations. Experiments on public datasets verify its effectiveness.
- The framework uses 3D convolutions to generate discriminative gait feature representations.
- Its Global and Local Feature Extractor combines global visual information with local gait details through GLConv layers.
- Local Temporal Aggregation replaces traditional spatial pooling, while Generalized-Mean pooling adaptively aggregates spatial information.