PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges

Chuanfu Shen, Shiqi Yu, Jilong Wang, George Q. Huang, Liang Wang

arXiv:2206.13732v2cs.CV

TL;DR

Gait recognition offers distant pedestrian identification with less active cooperation, while deep-learning methods have advanced across challenging settings. This survey organizes methods through representation-learning and architecture perspectives, reviews datasets and evaluations, and discusses privacy, security, and open challenges.

  • Problem

    Deep gait recognition has advanced, but new settings and modalities, limitations in existing taxonomies, and biometric privacy and security concerns remain central research issues.

    Source: §1 · pp. 1–2

  • Method

    The survey reviews deep gait methods through a taxonomy of representation-learning dimensions and network architectures, alongside datasets, evaluations, privacy, security, and challenges.

    Source: §1 · p. 2

  • Results

    The survey documents strong deep-gait results across settings, including 98.3% rank-1 accuracy on OUMVLP [45] and over 70% rank-1 accuracy on GREW, [47].

    Source: §1 · p. 2, §9 · p. 19

  • Takeaways & Limitations

    The proposed taxonomy offers a systematic view of deep gait methods, while the survey identifies privacy, security, and diverse-scenario evaluation as important topics.

    Source: §1 · p. 2

    Collecting large-scale in-the-wild gait datasets is extremely difficult because gait is ambiguous to annotate across viewpoints.

    Source: §8.1 · p. 18

Abstract

from arXiv · show

Gait recognition aims to identify a person at a distance, serving as a promising solution for long-distance and less-cooperation pedestrian recognition. Recently, significant advancements in gait recognition have achieved inspiring success in many challenging scenarios by utilizing deep learning techniques. Against the backdrop that deep gait recognition has achieved almost perfect performance in laboratory datasets, much recent research has introduced new challenges for gait recognition, including robust deep representation modeling, in-the-wild gait recognition, and even recognition from new visual sensors such as infrared and depth cameras. Meanwhile, the increasing performance of gait recognition might also reveal concerns about biometrics security and privacy prevention for society. We provide a comprehensive survey on recent literature using deep learning and a discussion on the privacy and security of gait biometrics. This survey reviews the existing deep gait recognition methods through a novel view based on our proposed taxonomy. The proposed taxonomy differs from the conventional taxonomy of categorizing available gait recognition methods into the model- or appearance-based methods, while our taxonomic hierarchy considers deep gait recognition from two perspectives: deep representation learning and deep network architectures, illustrating the current approaches from both micro and macro levels. We also include up-to-date reviews of datasets and performance evaluations on diverse scenarios. Finally, we introduce privacy and security concerns on gait biometrics and discuss outstanding challenges and potential directions for future research.

1 INTRODUCTION

Gait recognition identifies pedestrians from walking patterns at a distance, with deep learning expanding performance and the range of settings studied. This survey proposes a two-perspective taxonomy and reviews methods, datasets, evaluations, privacy, and challenges.

1 INTRODUCTION

  • Gait recognition can identify people at a distance with less active cooperation, supporting potential surveillance, forensic, and healthcare applications.The introduction presents gait recognition as a vision-based person-retrieval problem using walking patterns,.Source: §1 · p. 1
  • 98.3% rank-1 accuracy on OUMVLP [45], beyond 93% overall recognition accuracy on CASIA-B in an appearance-changing setting [34], and over 70% rank-1 accuracy on GREW, [47].The results span a dataset with over 10 thousand subjects, a difficult appearance-changing setting, and an outdoor dataset with over 20 thousand subjects.Source: §1 · p. 2
  • It reviews recent algorithms, modalities, datasets, and challenges, including in-the-wild, no-label, cloth-changing, and 3D-space research absent from earlier surveys,.The survey also compares performance across cross-view, in-the-wild, cloth-changing, and 3D-space scenarios and discusses biometric security and privacy.Source: §1 · p. 2
  • The survey proposes a taxonomy spanning deep representation learning and network architectures, with six representation-learning comparisons and a discriminative-versus-generative architecture split.The representation dimensions are 2D/3D, template/frames, shuffled/ordered, global/local, single/multi-scale, and long/short-term.Source: §1 · p. 2

2 OVERVIEW OF GAIT RECOGNITION

The overview defines gait-recognition settings, surveys task variants and visual inputs, and contrasts handcrafted with deep feature learning. It describes the pipeline from captured sequences to ranked identity retrieval.

2 OVERVIEW OF GAIT RECOGNITION

  • Gait recognition is framed as vision-based pedestrian identification from sequential streams; identification retrieves an identity from a gallery, unlike one-to-one verification.The survey distinguishes closed-set tests on known identities from open-set recognition of unseen subjects.Source: §2 · p. 3

2.1 Tasks in Gait Recognition

  • Tasks include changing-gait, in-the-wild, limited-label, cross-modality, and end-to-end recognition, alongside less-studied out-of-distribution detection and anonymous gait.The survey notes that changing dynamics receive less attention than viewpoint changes, and that in-the-wild studies have primarily focused on daytime settings.Source: §2.1 · p. 4

2.2 Various Input Data

  • Input representations range from RGB images, silhouettes, gait templates, optical flow, skeletons, and meshes, each retaining different appearance, motion, or body-structure information.Silhouettes can lose internal body structure and be affected by clothing and views; gait templates reduce sequence dimensionality, while optical flow has high computational cost.Source: §2.2 · pp. 4–5
  • Depth cameras are difficult to deploy outdoors beyond 10 meters, while event streams achieved nearly 96% real-world recognition accuracy and LiDAR supports long-range recognition in poor illumination.The cited event-camera work also reports performance comparable to state-of-the-art RGB methods on CASIA-B; LiDAR is reported to outperform camera-based methods in poor illumination and cross-view conditions.Source: §2.2 · p. 5

2.3.1 Handcrafted feature-based gait recognition

  • Gait recognition methods are broadly categorized as model-based or appearance-based according to whether they explicitly model human-body structure; appearance-based methods directly extract gait-relevant features without constructing human models.Model-based methods include structural models using static body parameters such as stride length and cadence, and motion models focused on dynamic motion features. Some works combine static body structure and dynamic motion,.Source: §2.3.1 · pp. 5–6

2.3.2 Deep feature-based gait recognition

  • Deep feature learning shifts feature extraction toward neural-network architectures and loss functions, progressing from gait templates toward frame-level features and models that learn the pipeline end to end,,, [50].The described typical pipeline includes foreground segmentation, gait alignment, and feature extraction.Source: §2.3.2 · p. 6, Figure 3 · p. 6

3 A GLIMPSE OF DEEP GAIT RECOGNITION

Deep gait models combine feature extraction, spatial-temporal aggregation, and representation mapping, with design choices spanning backbones, bottlenecks, heads, losses, and evaluation protocols.

3 A GLIMPSE OF DEEP GAIT RECOGNITION

  • Deep gait models use a backbone for feature extraction, a bottleneck for spatial-temporal aggregation, and an optional head for representation mapping.Evaluation then depends on the recognition mode, metrics, and dataset-specific protocols.Source: §3 · p. 6, §3.3 · p. 7, §3.5 · p. 8

3.1 Backbone Networks

  • CNN backbones evolved from 2D models with limited temporal capture to 3D convolutions and gait-specific focal, local, and graph convolutions.Backbones are often only 4–8 layers, which the survey relates to the lower information entropy of silhouettes and skeletons.Source: §3.1 · p. 6

3.2 Bottleneck Networks

  • Bottlenecks aggregate temporal information and enhance discriminative features through spatial manipulation, using part-based pooling and temporal attention or recurrent modeling.These approaches address overfitting and noise sensitivity associated with fully connected feature reduction on poorly segmented data.Source: §3.2 · pp. 6–7

3.3 Head Networks

  • Verification and identification heads are common, while compact blocks reduce identification-feature dimensionality and memory use.Quality-, view-, condition-, and gender-aware heads also support multi-task gait recognition.Source: §3.3 · p. 7

3.4 Loss Functions

  • Cross-entropy, contrastive, and triplet losses are common, while gait-specific losses target challenges such as cross-view and changing appearance.The survey notes that cross-entropy may be suboptimal with very large identity sets and substantial inter-class variation.Source: §3.4 · pp. 7–8

3.5 Evaluation

  • Verification is typically measured with ROC curves, whereas identification uses CMC, mAP, or mINP, with protocols varying by dataset.The survey points readers to Hou et al. for a fuller account of evaluation protocols.Source: §3.5 · p. 8

4 DEEP REPRESENTATIONS LEARNING

The survey organizes deep gait representation learning by both input-level and feature-level choices, which determine what spatial, temporal, and identity information models capture.

4.1.1 2D/3D representation learning

  • 2D representations are efficient and widely used, but silhouettes omit inter-frame motion; 3D representations can improve viewpoint handling, with practical constraints on estimated pose and depth sensing.The survey reports LiDAR-based gait recognition outperforming 2D methods by a large margin in cross-view performance.Source: §4.1.1 · p. 8

4.1.2 Template/frames-based representation learning

  • Template methods such as GEI compress a silhouette cycle into a computationally efficient representation, while frames-based methods retain sequences for spatial and temporal feature learning.GEI preserves static information but partially loses gait motion information.Source: §4.1.2 · pp. 8–9

4.1.3 Shuffled/ordered representation learning

  • Shuffled inputs are robust to frame permutations and outperform ordered inputs on outdoor datasets, while ordered motion learning works well in laboratory settings but has underperformed in-the-wild.Residual learning and deeper models can improve ordered-input accuracy in the wild.Source: §4.1.3 · p. 9

4.2.1 Global/local representation learning

  • Global features encode holistic body information, whereas local representations target fine-grained body regions; combining both is presented as a reasonable approach.Local methods include body-part partitions, masks, pose-driven regions, attention regions, parsing, and patches.Source: §4.2.1 · p. 9

4.2.2 Single/multi-scale representation learning

  • Single-scale learning is simple and computationally inexpensive, while multi-scale methods capture features at multiple levels and can strengthen part-level discrimination.Examples include horizontal pyramid pooling and top-down feature merging.Source: §4.2.2 · pp. 9–10

4.2.3 Long/short-term representation learning

  • Long-term representations capture broad motion cues and are reported robust in complex outdoor settings, whereas short-term methods model adjacent-frame micro-motion but underperform long-term approaches on in-the-wild datasets.Long-term approaches include temporal pooling, attention, LSTMs, and stacked 3D convolutions; short-term strategies include LSTMs and temporal convolutions.Source: §4.2.3 · p. 10

5 DEEP MODELS FOR GAIT RECOGNITION

The survey distinguishes deep gait models by whether they learn identity decision boundaries or model the data distribution.

5 DEEP MODELS FOR GAIT RECOGNITION

  • Discriminative models learn identity decision boundaries, whereas generative models model the data distribution and can generate new samples.The section introduces these as two broad deep-model categories.Source: §5 · p. 10, Figure 5 · p. 10

5.1 Discriminative Model

Discriminative gait models learn decision boundaries, and single-modal designs are categorized by where spatial and temporal information is fused.

5.1 Discriminative Model

  • Discriminative models directly learn decision boundaries for gait recognition and include single-modal and multi-modal approaches.Source: §5.1 · p. 10

5.1.1 Single-modal architecture

  • Single-modal models are grouped into divided early fusion, divided late fusion, and joint fusion according to where spatial and temporal information is combined.Single-modal models dominate the field in part because they are often more computationally efficient than multi-modal models.Source: §5.1.1 · pp. 10–11

Divided Early Fusion (T+S) Model

Divided early fusion combines gait inputs at the outset to extract spatiotemporal representations, often using a gait template such as the gait energy image (GEI). The survey describes both the efficiency and limitations of this approach, including clothes-change performance degradation.

Divided Early Fusion (T+S) Model

  • Early fusion combines gait inputs at the initial stage and then extracts spatiotemporal representations; it is also called template-based architecture and is computationally friendly.The cited works also indicate that this design can obstruct deep feature extraction in space and time.Source: Divided Early Fusion (T+S) Model · p. 11
  • GEI averages gait silhouettes over time, and GEINet applied a compact CNN to GEIs, marginally outperforming an earlier silhouette-frame method.GEI became a common template for deep learning after its use in cross-view datasets with over 10,000 subjects.Source: Divided Early Fusion (T+S) Model · p. 11
  • GEI-based methods can degrade on clothes-changing subsets, while skeleton gait energy images average sequential skeletons and can perform better across changing views and clothes.Later work reported that feature-level spatiotemporal fusion captures more informative gait features than input-level fusion.Source: Divided Early Fusion (T+S) Model · p. 11

Divided Late Fusion (S+T) Model

Divided late fusion learns appearance before modeling and aggregating motion, separating spatial and temporal processing. The survey describes spatial encoders, temporal aggregation choices, and trade-offs in accuracy, robustness, and computational cost.

Divided Late Fusion (S+T) Model

  • Divided late fusion extracts appearance from frame-level inputs before modeling and aggregating motion, preserving detailed spatial gait patterns.Separating spatial and temporal features also supports studying gait spatiotemporally.Source: Divided Late Fusion (S+T) Model · p. 11
  • GaitSet first achieved 70.3% cross-view recognition accuracy on CASIA-B under clothes-changing conditions, while deeper GaitBase and DeepGaitV2-2D improved performance on in-the-wild datasets.Shallow models can degrade substantially outdoors amid complex backgrounds, occlusion, illumination, viewpoints, and clothing changes.Source: Divided Late Fusion (S+T) Model · p. 11
  • Temporal modeling uses parameter-free statistical pooling or parameter-based learnable dependencies, including recurrent networks, 1D convolutions, and attention.Multi-scale temporal features can combine long-term, short-term, and frame-level clues to distinguish gait variants.Source: Divided Late Fusion (S+T) Model · pp. 11–12
  • Parameter-based temporal methods increase computational complexity and can perform worse than parameter-free methods, limiting their practicality in some real-world applications.The reported comparison includes works [41],,.Source: Divided Late Fusion (S+T) Model · p. 12

Joint Fusion (ST) Model

Joint-fusion models learn spatial and temporal dimensions simultaneously, while multimodal architectures combine distinct input modalities. The survey reviews their processing designs and how they use complementary gait information.

Joint Fusion (ST) Model

  • Joint-fusion models learn spatial and temporal dimensions simultaneously and can potentially capture more complex spatiotemporal patterns.Unlike divided-fusion models, they do not learn the two dimensions independently.Source: Joint Fusion (ST) Model · p. 12
  • 3D CNNs process structural gait sequences as C × T × H × W tensors, but C3D is computationally intensive and difficult to optimize on small datasets.Factorized P3D convolutions achieved comparative performance to R3D with fewer parameters (11.1 vs. 27.5M) and lower cost (2.9 vs. 6.8GFLOPs); DeepGaitV2-P3D improved DeepGaitV2-2D accuracy by up to 9.1%.Source: Joint Fusion (ST) Model · p. 12
  • Temporal shift operations and frame-difference methods offer alternatives to 3D convolution, while transformers and spatial-temporal graph convolution model skeleton and point-cloud inputs.Temporal shift is described as computationally efficient and requiring no additional parameter tuning.Source: Joint Fusion (ST) Model · p. 12

5.1.2 Multi-modal architecture

  • Here, multimodal models use multiple modalities rather than merely multiple pathways, and some combine modality-specialized branches with a gate to handle missing test-time inputs.UGaitNet’s gate can disable the branch corresponding to a missing modality so that it does not penalize performance.Source: §5.1.2 · p. 12
  • Silhouettes preserve body shape but are sensitive to clothing and viewpoint changes, whereas skeletons are viewpoint-robust but omit body-shape detail.Hybrid silhouette-skeleton methods combine these representations; other methods use 3D meshes, depth images, or point clouds.Source: §5.1.2 · p. 12

5.2 Generative Model

Generative gait models address appearance variation through autoencoders and adversarial networks that learn or synthesize gait data. The survey describes their applications, design approaches, and limitations.

5.2 Generative Model

  • Generative gait models use autoencoders or generative adversarial networks to model data distributions and synthesize samples for challenges such as carrying, clothing changes, and viewpoint variation.The survey distinguishes these generative approaches from discriminative models that learn decision boundaries.Source: §5.2 · pp. 12–13

5.2.1 Deep autoencoders

  • Deep autoencoders encode inputs into compact latent representations and decode them to reconstruct the original data, with reconstruction error guiding learning.SPAE learns invariant gait features from GEIs across conditions, while ICDNet disentangles identity and covariant features; autoencoders can suffer posterior collapse and imperfect reconstruction.Source: §5.2.1 · p. 13

5.2.2 Generative adversarial network

  • GAN-based methods transform gait across viewpoints and appearance conditions, but preserving identity during transformation remains a challenge.GaitGAN uses an identification discriminator to constrain identity preservation, and GaitGANv2 adds a multi-loss strategy; other methods address spatial misalignment or low frame rates.Source: §5.2.2 · p. 13
  • Most generative methods rely heavily on supervised labels, while GaitEditor uses GAN inversion for semantic attribute manipulation and is presented as an unsupervised approach.Its described attributes include viewpoints, clothing, walking speed, gender, and age.Source: §5.2.2 · p. 13

6 DATASETS AND EVALUATION

The survey reviews public gait datasets and compares deep recognition methods across cross-view, in-the-wild, clothes-changing, and 3D scenarios. The comparisons reveal both strong results and limits tied to setting, modality, and dataset characteristics.

6.1 Datasets for Gait Recognition

  • Large-scale datasets support statistical evaluation but typically offer fewer attributes; collecting them requires more time, storage, and cost and faces privacy restrictions.The survey notes that privacy laws add difficulty to large-scale gait data collection.Source: §6.1 · pp. 13–14

6.2 Performance Comparisons

  • With over 60 datasets and varying evaluation protocols, the survey selects representative datasets across four scenarios for performance comparison.The scenarios are cross-view, in-the-wild, clothes-changing, and point-cloud recognition.Source: §6.2 · p. 14

6.2.1 Evaluation on cross-view gait recognition

  • Over 90% rank-1 accuracy is achieved on CASIA-B and OUMVLP; silhouette methods dominate, while RGB and skeleton studies remain relatively scarce.The table covers cross-view recognition on both datasets.Source: Table 2 · p. 14, §6.2.1 · p. 14

6.2.2 Evaluation on in-the-wild gait recognition

  • Outdoor results drop versus indoor; silhouettes outperform estimated skeletons, and GREW generalizes better than Gait3D, possibly because its viewpoints are less diverse.The authors hypothesize that viewpoint diversity contributes to GREW’s stronger performance.Source: Table 3 · p. 15, §6.2.2 · p. 15

6.2.3 Evaluation on clothes-changing gait recognition

  • Over 70% recognition is reported on all three clothes-changing datasets, while RGB inputs bring gains that may partly use face or shoe cues.The cited table compares FVG, CASIA-E, and CCPG.Source: Table 4 · p. 15, §6.2.3 · p. 15

6.2.4 Evaluation on 3D gait recognition

  • LiDAR-based methods show promise under poor illumination and occlusion, with 3D geometry supporting more informative gait representations.The performance comparison covers LiCamGait and SUSTech1K.Source: Table 5 · p. 15, §6.2.4 · p. 15

7 SECURITY AND PRIVACY OF GAIT RECOGNITION

The survey outlines security threats and privacy concerns specific to gait recognition. It highlights underexplored attack types and unresolved questions about identifying people in shared videos.

7.1 Security

  • Gait systems face presentation, adversarial, and template attacks, as in other biometric systems.Template attacks target extracted features and may draw on insights from face, fingerprint, or iris biometrics.Source: §7.1 · p. 17
  • Gait presentation attacks remain relatively unexplored, with the cited vision-based investigation described as pioneering.The survey calls for further research on this attack type.Source: §7.1 · p. 17
  • Gait adversarial attacks require sequential synthetic data with both fidelity and temporal consistency, which the survey identifies as needing further study.Source: §7.1 · p. 17

7.2 Privacy

  • Gait may be captured farther away than facial features and can reveal identity, gender, and health conditions.The survey presents these factors as contributing to gait recognition’s privacy implications.Source: §7.2 · p. 17
  • Biometric privacy laws commonly provide rights to know, delete, and opt out, while government-use exceptions may lack transparency,,.Source: §7.2 · p. 17
  • Shared videos raise unresolved questions about whether identifiable pedestrians need permission and whether their videos should be blurred or removed.The figure depicts a shared social-media video containing many pedestrians.Source: Figure 7 · p. 17, §7.2 · p. 17

8 CHALLENGES AND DIRECTIONS

The section identifies dataset, practicality, and trustworthiness challenges that remain despite progress in gait recognition. It outlines research directions spanning real-world data, recognition methods, and privacy protection.

8 CHALLENGES AND DIRECTIONS

  • The authors frame future improvement around datasets, practicality, and trustworthiness.Source: §8 · p. 18

8.1 Datasets

  • Real-world gait datasets remain difficult to scale because viewpoint ambiguity complicates identity annotation across clothing and status variations.Synthetic data offers large-scale attributes but raises questions about when virtual bodies count as distinct identities; unlabelled data may support self-supervised learning.Source: §8.1 · p. 18

8.2 Gait Recognition Toward Reality

  • Aging and disease can change gait motion itself, a factor often overlooked alongside appearance changes from viewpoint, clothing, and occlusion.Disentanglement and motion modeling have been explored to address appearance changes,,.Source: §8.2 · p. 18
  • End-to-end gait recognition learns directly from raw inputs to use whole-body information and simplify pipelines affected by off-the-shelf segmentation.The approach includes work by Liang et al. [50], Song et al., and Zhang et al..Source: §8.2 · p. 18
  • Infrared cameras and LiDAR offer alternatives for low-light recognition, while multimodal fusion and cross-modal retrieval remain challenges.Source: §8.2 · p. 18
  • Practical surveillance still requires real-time feature extraction and retrieval, including searches across galleries of millions or billions of samples.The section calls for efficient methods and lightweight models with reduced computational costs.Source: §8.2 · p. 18
  • Transfer learning can carry knowledge between gait domains, such as GREW to CASIA-B, and from human gait recognition to animal recognition.The section associates transfer learning with improved performance and generalization.Source: §8.2 · pp. 18–19

8.3 Trustworthy Gait Recognition

  • Gait systems raise privacy and bias concerns around sensitive attributes, while practical gait-video encryption that preserves visual appeal remains underexplored.The section notes that masking pedestrians can degrade video quality and calls for effective privacy-protection techniques.Source: §8.3 · p. 19

9 CONCLUSIONS

The conclusion presents the survey as an overview of deep gait recognition, its datasets, challenges, and future directions. It highlights a taxonomy organized around feature learning and architectures as a way to understand the field.

9 CONCLUSIONS

  • The survey reviews deep gait recognition through feature-learning and architecture perspectives, alongside datasets, privacy, security, alternative modalities, challenges, and future directions.Its proposed taxonomy is intended to provide a clear understanding of the field's big picture.Source: §9 · p. 19
Loading 2206.13732v2…