Source-linked AI summary
GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition
Hanqing Chao, Yiwei He, Junping Zhang, Jianfeng Feng
TL;DR
Existing gait recognition methods either compress silhouettes into templates that can lose temporal information or process ordered sequences with unnecessary sequential constraints. GaitSet instead represents gait as an unordered set of independent silhouettes and learns identity information through set-based aggregation. It reports state-of-the-art accuracy on CASIA-B and OU-MVLP, robustness across complex walking conditions, and satisfactory performance with few frames.
Problem
Existing gait recognition methods use templates that can lose temporal information or sequences that impose unnecessary sequential constraints, limiting flexibility.
Method
GaitSet regards gait as a set of independent silhouettes, extracts frame-level features, aggregates them with permutation-invariant Set Pooling, and learns a discriminative representation.
Results
GaitSet achieves the highest recognition accuracy among compared methods on two benchmark gait datasets and shows robustness across complex environments.
Takeaways & Limitations
The set perspective provides flexibility for combining silhouettes from different sequences and conditions while supporting recognition with limited or non-consecutive frames.
Takeaways & Limitations
The formulation assumes silhouettes of a person are subject to a distribution related only to that person’s identity.
Abstract
from arXiv · showhide
As a unique biometric feature that can be recognized at a distance, gait has broad applications in crime prevention, forensic identification and social security. To portray a gait, existing gait recognition methods utilize either a gait template, where temporal information is hard to preserve, or a gait sequence, which must keep unnecessary sequential constraints and thus loses the flexibility of gait recognition. In this paper we present a novel perspective, where a gait is regarded as a set consisting of independent frames. We propose a new network named GaitSet to learn identity information from the set. Based on the set perspective, our method is immune to permutation of frames, and can naturally integrate frames from different videos which have been filmed under different scenarios, such as diverse viewing angles, different clothes/carrying conditions. Experiments show that under normal walking conditions, our single-model method achieves an average rank-1 accuracy of 95.0% on the CASIA-B gait dataset and an 87.1% accuracy on the OU-MVLP gait dataset. These results represent new state-of-the-art recognition accuracy. On various complex scenarios, our model exhibits a significant level of robustness. It achieves accuracies of 87.2% and 70.4% on CASIA-B under bag-carrying and coat-wearing walking conditions, respectively. These outperform the existing best methods by a large margin. The method presented can also achieve a satisfactory accuracy with a small number of frames in a test sample, e.g., 82.5% on CASIA-B with only 7 frames. The source code has been released at https://github.com/AbnerHqC/GaitSet.
1 Introduction
Gait recognition must handle external variations while balancing information preservation and input flexibility. GaitSet treats gait as an unordered set of silhouettes, enabling flexible aggregation and strong recognition performance.
- Gait recognition is affected by walking speed, dressing, carrying condition, camera viewpoint, and frame rate.
- Template-based methods compress silhouettes into one image but can lose temporal and fine-grained spatial information.
- GaitSet regards gait as a set of silhouettes and assumes each silhouette’s appearance contains its position information.
- The model independently extracts frame features, aggregates them with Set Pooling, and maps the result into a discriminative space using Horizontal Pyramid Mapping.
- GaitSet accepts non-consecutive silhouettes from different viewpoints and walking conditions, computes each sample representation once, and compares representations with Euclidean distance.
- GaitSet improves performance on CASIA-B and OU-MVLP while showing robustness to view and walking-condition variations.
2 Related Work
Prior gait recognition methods primarily use compressed templates or ordered silhouette sequences, while set-based deep learning emerged mainly in computer vision tasks such as point-cloud processing.
- Gait recognition methods are grouped into template-based and sequence-based categories.
- Template-based approaches generate a single gait image before extracting and matching gait representations.
- Sequence-based approaches directly process silhouette videos using LSTM-based or 3D CNN-based methods to extract temporal information.
- Unordered sets were introduced into computer vision through PointNet for point-cloud tasks and were subsequently used broadly in that field.
3 GaitSet
GaitSet treats gait silhouettes as an unordered set, then extracts frame-level, set-level, and discriminative representations through permutation-invariant pooling and horizontal pyramid mapping. Its design combines multiple statistical summaries, attention refinement, spatial strip mappings, and multilevel features before Euclidean-distance recognition.
- 3.1 Problem Formulation: GaitSet formulates gait recognition as learning identity information from a set of silhouettes through three stages: frame extraction, set pooling, and discriminative mapping.The representation is f_i = H(G(F(X_i))), where F extracts frame-level features, G maps them to a set-level feature, and H learns a discriminative representation.
- 3.2 Set Pooling: Set Pooling combines max, mean, and median statistics across the set dimension to produce permutation-invariant set-level features.An enhanced variant concatenates these statistics and applies a 1 × 1 convolutional layer to learn how to combine them.
- 3.2 Set Pooling: Attention-based Set Pooling uses statistical summaries with the original feature map, then applies pointwise refinement and MAX aggregation to obtain the final set-level feature.The residual structure is described as accelerating and stabilizing convergence.
- 3.3 Horizontal Pyramid Mapping: Horizontal Pyramid Mapping splits set-level feature maps into strips at multiple scales, pools each strip, and uses independent fully connected layers for discriminative mapping.For strip z_s,t, global max pooling and average pooling are added before the resulting features are mapped into a discriminative space.
- 3.4 Multilayer Global Pipeline: The Multilayer Global Pipeline extracts set-level information from different convolutional layers, while its following HPM uses parameters separate from the main pipeline.The framework caption states that configurations may match across pipelines, but parameter sharing is restricted to blocks within the main pipeline.
- 3.5 Training And Testing: During testing, query and gallery sets are converted into concatenated multiscale representations and compared with Euclidean distance to calculate Rank 1 recognition accuracy.Training uses Batch All triplet loss with batches sampled by person and training sample.
4 Experiments
Experiments evaluate GaitSet on CASIA-B and OU-MVLP, then examine its components and practicality under limited silhouettes, multiple views, and varied walking conditions. Results show strong recognition performance, benefits from set-based representation and architectural components, and robustness to flexible inputs.
- Experimental design: Experiments compare GaitSet with state-of-the-art methods on CASIA-B and OU-MVLP, alongside ablations and practicality tests.The practicality tests vary silhouette count, viewpoints, and walking conditions without retraining the model.
- CASIA-B results: With only 24 training subjects, GaitSet exceeds the previously reported best performance by over 10% on the views evaluated.Set-based training exposes the convolutional network to many more silhouettes and generates multiple random training sets per sequence.
- CASIA-B results: CASIA-B results show over 90% accuracy on every normal-walking view except 180° under large-sample training, while bag and coat conditions exceed competing models by at least 18.8% in mean accuracy.Identical-view cases are excluded from the averaged rank-1 accuracies.
- OU-MVLP results: 87.1% average rank-1 accuracy on OU-MVLP demonstrates generalization across its large subject population and wide view variation.If probes without corresponding gallery samples are ignored, the average rises to 93.3%; testing 133780 sequences takes 7 minutes on 8 NVIDIA 1080TI GPUs.
- Ablation experiments: Replacing GEI with a set improves accuracy by more than 10% on normal walking and more than 25% on coat-wearing walking under identical networks.The GEI variant averages gait silhouettes into one image before network input.
- Practicality: 82% accuracy with 7 silhouettes shows useful recognition from sparse inputs, while combining views or complementary bag-and-coat conditions can further improve performance.Accuracy rises with silhouette count; multiple views provide parallel and vertical gait information, whereas normal-walking silhouettes can be degraded by noisy conditions.
5 Conclusion
GaitSet treats gait as a set and extracts spatial and temporal information more effectively and efficiently than template- or sequence-based methods. Experiments indicate leading recognition accuracy and flexibility across complex environments.
- GaitSet regards gait as a set and extracts spatial and temporal information more effectively and efficiently than template- or sequence-based methods.
- GaitSet provides a way to aggregate valuable information from different sequences to enhance recognition accuracy.
- Experiments on two benchmark gait datasets indicate that GaitSet achieves the highest recognition accuracy compared with other state-of-the-art algorithms.
- GaitSet shows flexibility across various complex environments, indicating potential for practical applications.