A data augmentation methodology for training machine/deep learning gait recognition algorithms

1 Introduction

• Environmental, imaging, and subject-appearance factors can alter gait features and reduce their usefulness for identity recognition.Examples include walking surface, lighting, camera angle, clothing, footwear, and walking speed.Source: §1 · pp. 1–2
• Training supervised gait-recognition systems requires large, high-quality labelled datasets that are expensive or infeasible to collect across subjects and conditions.Collecting enough data for deep networks could require participants to walk for hours and experiments to be repeated under many conditions.Source: §1 · p. 2
• The authors present this as an initial feasibility step, conditional on sufficiently high-quality motion capture, rendering, and imaging models.They aim to enable synthetic video generation with many controllable conditions and report that identity characteristics are preserved in generated motion.Source: §1 · p. 2
• The paper introduces a simulation-based method and multimodal dataset with more than 6.5 million frames, based on motion capture from 26 subjects.The simulation files allow several confounding factors to be controlled, and the motion capture includes treadmill walking and running at different speeds.Source: §1 · p. 2

2 Literature Review

• Appearance-based gait methods compress video into image representations, including MEI, MHI, GEI, KFDs, MSI, GHI, and GEnI.GEI averages binary silhouettes across a gait cycle; other representations use motion history, contour frequencies, or pixelwise entropy.Source: §2 · pp. 2–3
• Appearance-based methods can discriminate subjects in controlled imaging environments, but confounding factors can substantially reduce recognition performance in less controlled settings.Viewpoint variation is a common confound; the View Transformation Model maps Fourier-based features between camera-subject viewpoints.Source: §2 · p. 3
• Prior studies address individual challenges such as clothing variation, occlusion, and walking speed using specialized features or datasets,,.The clothing study uses GEnI-derived features, while the occlusion work introduces a dataset and baselines.Source: §2 · p. 3
• Image augmentations such as flipping, cropping, resizing, rotations, and translations have been used in computer vision and gait recognition,.The paper treats these transformations as only a small subset of the conditions for which gait recognition should be invariant.Source: §2 · p. 4

3 Data Collection

• Subjects were motion-captured while walking at 3–7 km/h and jogging or running at 8–12 km/h on a treadmill.Each range includes five speeds.Source: §3 · p. 4
• Figure 1 traces data collection from reflective body markers through a 3D marker representation and estimated skeleton to a mocap-driven Blender skeleton and avatar.Its final panel shows the 3D mesh deformed according to the collected motion capture data.Source: Figure 1 · p. 4
• Ten infrared cameras recorded 38 body markers at 200 Hz, and the marker coordinates and anthropometric measurements were used to fit a 19-joint 3D skeleton.Vicon Nexus pre-processed the data and estimated the skeleton; its additional kinetics were not used.Source: §3 · p. 4

Data Preparation

• MakeHuman avatars were created using each volunteer’s anthropometry, with adjustable characteristics including gender, age, muscle percentage, weight, height, and body proportions.Appearance changes such as clothing and hair can also be applied; the six-subject sample shows variation in anthropometry and materials.Source: Data Preparation · p. 5, Figure 2 · p. 5
• Skinning attaches an avatar to a rig so its mesh can move with the skeleton, and retargetting transfers captured motion to the avatar rig.Retargetting required adjustments for inconsistencies between the motion-capture skeleton and MakeHuman armature.Source: Data Preparation · p. 5
• Combining motion capture, data preparation, avatar construction, and scripted rendering enables synthetic frames with extensive variation in confounding factors.The dataset also includes deformed 3D meshes for subjects at 5 km/h, supporting subject-, camera-, and pose-specific image sequences; Figure 3 shows video frames, silhouettes, and GEI features.Source: Data Preparation · p. 6, Figure 3 · p. 6

5 Experiments

• Recognition experiments used GEI features to train SVMs for subject identification, assessing identity preservation in generated data.GEIs were selected because they are easy to extract and relatively immune to movement jitter and silhouette-segmentation noise.Source: §5 · pp. 6–7
• Real silhouettes were produced by background subtraction and intensity clustering, while synthetic silhouettes used color separation against a uniform background.The real-image process thresholded the clustered foreground and background intensities.Source: §5 · pp. 6–7
• Gait-cycle boundaries were defined from every second peak in a periodic signal of lower-body white-pixel counts.The count peaks occur at extreme leg positions, with the signal having half the period of a gait cycle.Source: §5 · p. 7
• Silhouettes were resized to 50×30 pixels and averaged across each gait cycle to create GEIs, then augmented by flipping and systematic cropping.Figure 4 shows seven cycle frames and the resulting GEI for subject 3, plus real, synthetic, and fused GEIs for six subjects.Source: §5 · p. 7, Figure 4 · p. 7
• The evaluation compared real-subject and avatar silhouettes, then examined GEI principal components and recognition with real, synthetic, or combined training and test data.The GEI analysis included visualization of real and synthetic principal components and experiments using each dataset separately or together.Source: §5 · p. 7

Results & Discussion

• Synthetic silhouettes aligned well with real sequences relative to the same-subject, different-cycle real-silhouette baseline in the walking trial.Figure 6 reports Jaccard-index similarity for subjects 1–14; the baseline is shown in Figure 5.Source: Results & Discussion · p. 8, Figure 6 · p. 8, Figure 5 · p. 8
• Jaccard scores were reduced by unmodelled hairstyle, small rotation or translation offsets, and segmentation noise in real images.The index is not rotation- or translation-invariant, and the noise lowers reported average similarity.Source: Results & Discussion · p. 9
• More than 95% accuracy was achieved with limited real data augmented by synthetic data in the 70%R+S–30%R and 30%R+S–70%R conditions, given sufficient principal components.Same-mode training and testing performed best, while cross-mode R–S and S–R conditions performed worst; Figure 7(a) shows similar but non-identical real and synthetic feature spaces.Source: Figure 7 · p. 9, Results & Discussion · p. 9
• The treadmill setting enabled long motion-capture sequences but may alter natural gait.The cited biomechanics study found surprisingly small differences between treadmill and overground walking in healthy individuals.Source: Results & Discussion · p. 10
• Manually entered anthropometry left some body-appearance details out of the avatar meshes, and mesh deformation may also fail to transfer real gait kinematics perfectly.Silhouette calculations depend on the meshes, so these construction issues may introduce detail loss.Source: Results & Discussion · p. 10

• The method generates gait sequences with multiple confounding factors, enabling exploration of fully invariant recognition with more synthetic training and test data than otherwise possible.Source: §7 · p. 10
• Sampling viewing angles at 5° steps yields 703 azimuth-elevation pairs and expands the dataset to more than 4.5 billion frames, though synthesis remains bounded by collected mocap data.The underlying multi-modal dataset contains more than 6.5 million real-mocap frames.Source: §7 · p. 10
• Synthetic video contains identity information that discriminates subjects, while visually replicating some appearance variability for exploring more robust gait features.The conclusion cites qualitative evidence in Figure 4 and quantitative evidence in Figure 7(b).Source: §7 · p. 10
• Future work would broaden coverage to age, gender, body shape, fitness, and more subjects, while improving camera and garment modeling to close the synthetic-real gap.Source: §7 · p. 10