Domain Generalization for Smartphone-Based Human Activity Recognition: A Systematic Analysis of Components and Interactions

1 Introduction

• Conventional i.i.d. evaluations provide limited evidence of robustness when deployment data come from different users, devices, sensor placements, environments, or acquisition protocols.DG evaluates models trained on source domains on an unseen target domain.Source: §1 · pp. 1–2
• Prior HAR studies primarily compare objective-based DG methods, leaving initialization and architectural modifications less systematically evaluated despite evidence that these components interact.The paper organizes DG approaches by whether they modify representation initialization, architecture, or the training objective.Source: §1 · p. 2
• The benchmark covers more than 410,000 experiments across four HAR backbones, thirteen training objectives, five initialization strategies, four architectural modules, and six cross-dataset and four cross-position folds.It evaluates initialization, architectural modifications, and objective-based DG as distinct pipeline components.Source: §1 · p. 2
• Joint DG configurations can outperform isolated components through complementary or super-additive interactions, while architectural modifications—particularly DDG—provide the clearest standalone gains.The effects depend on the model and distribution shift; some class-level gains concentrate on difficult, shift-sensitive boundaries involving walking, running, and stair activities.Source: §1 · p. 2
• Source-validation selection recovers only 53% of available oracle gain in cross-dataset and 26% in cross-position settings at the highest compositional level.The analysis identifies source-only model selection as a major bottleneck.Source: §1 · p. 2

2 The Domain Generalization Problem

• DG trains on multiple related source domains and evaluates on an unavailable target domain with a different distribution, assuming shared latent structure enables knowledge transfer.The source domains differ in their distributions while sharing the label space.Source: §2 · p. 3
• A domain is defined through the input space, label space, and joint distribution over inputs and labels.A dataset for that domain is a finite collection of i.i.d. samples from the joint distribution, with each sample pairing an input vector and its label.Source: §2 · p. 3
• Meaningful domain shifts alter the data-generating process, as with changes in users, devices, sensor positions, environments, or acquisition protocols; numerical preprocessing transformations need not do so.Changing the semantic label space defines a different task even if sensor observations remain unchanged.Source: §2 · p. 3

2.1 Related Concepts

• Unlike domain adaptation, DG uses no target-domain training samples and assumes a fixed shared label space, unlike paradigms addressing unseen or unknown classes.The paper assumes a closed label set, so no new classes appear in the target domain.Source: §2.1 · p. 3

3 Distribution Shifts in Human Activity Recognition

• CD permits multiple acquisition factors to change, whereas CP isolates sensor position with subjects, device, and acquisition protocol fixed.The CD scenario uses six smartphone datasets; CP uses four RealWorld sensor placements.Source: §3 · p. 4
• Activity type remains the dominant organization, but same-activity distributions vary across domains, with clear CD shifts for walking and stair-related classes.The Figure 1 UMAP visualization is qualitative and should not be treated as a direct measure of shift magnitude.Source: §3 · p. 4, Figure 1 · p. 4
• Cross-position shifts substantially alter locomotion distributions even with subjects, device, and acquisition protocol fixed, while static activities remain comparatively stable.Source: §3 · pp. 4–5

3.1 Shift Magnitude and Potential Sources

• 0.912 mean CP separability versus 0.814 for CD; changing sensor position alone can produce stronger domain differences than heterogeneous dataset changes.Across dataset pairs, changing position within RealWorld yields 0.912 mean domain-classification accuracy, versus 0.781 for changing dataset family while approximately preserving position.Source: §3.1 · p. 5, Figure 2 · p. 5
• Changing both dataset family and position yields 0.876 mean domain-classification accuracy, below position-only changes at 0.912.The comparison spans 28 dataset pairs and uses mean normalized Proxy A-Distance with FFT features.Source: §3.1 · p. 5

• The taxonomy addresses a gap in HAR studies, which often evaluate DG components in isolation despite evidence that initialization, architecture, and optimization interact.Source: §4 · pp. 5–6
• DG methods modify initialization, architecture, or supervised objectives, corresponding to pretraining, internal feature operations, or training algorithms.Architectural changes may be trainable or parameter-free; objective-based methods alter losses, regularization, gradients, sampling, or optimization without changing feature-extraction operations.Source: §4 · p. 6
• These categories are non-exclusive, enabling controlled analysis of complementary, redundant, and model-dependent interactions among pipeline components.A pretrained model can be combined with an architectural modification and an alternative DG objective.Source: §4 · p. 6
• The evaluated methods span risk-based optimization, feature alignment, gradient regularization, feature augmentation or suppression, architectural adaptation, and self-supervised initialization.ERM is the supervised baseline; MixStyle, EFD-Mix, and DDG are architectural modifications, while TF-C and LFR are initialization strategies.Source: §4 · p. 6

• DG performance comparisons are sensitive to datasets, preprocessing, architectures, hyperparameter optimization, and checkpoint selection.DomainBed standardized evaluation, and Gulrajani and Lopez-Paz found that many specialized methods do not consistently outperform ERM.Source: §5 · p. 6
• Modern ERM training practices can strengthen the baseline, and DG method rankings depend on the broader learning pipeline, not only the objective.Teterwak et al. reported this in ERM++; the paper uses it to motivate joint evaluation of objectives, architectures, and initialization.Source: §5 · p. 6
• Existing HAR studies cover several shifts but often compare new methods with limited baselines in task-specific settings, hindering cross-study comparison.Napoli et al. standardized smartphone-HAR preprocessing but focused primarily on ERM; Lu et al. [3] introduced HAROOD.Source: §5 · pp. 6–7
• HAROOD standardizes OOD evaluation but focuses on standalone objectives, uses a limited backbone set, and omits SSL and recent architectural DG mechanisms.It also does not focus on smartphone-based HAR, leaving combined component effects in that setting largely unexplored.Source: §5 · p. 7
• The paper addresses the gap with a controlled smartphone-HAR evaluation jointly crossing initialization, architectural DG modifications, and objective-based methods.This design aims to isolate individual contributions and quantify interactions and combined effects.Source: §5 · p. 7

6 Materials and Methods

• The protocol standardizes smartphone IMU data and evaluates cross-dataset and cross-position scenarios with a held-out target domain.Samples are 3-second windows at 20 Hz, represented as 6×60 tensors of accelerometer and gyroscope signals.Source: §6 · p. 8, Figure 3 · p. 8

6.1 Evaluation Protocol and Data Partitioning

• Six cross-dataset folds and four RealWorld position folds use leave-one-domain-out evaluation, with the benchmark restricted to smartphone-based HAR.Source training and validation partitions are merged for supervised training; the target domain is excluded from pretraining, optimization, tuning, and checkpoint selection.Source: §6.1 · p. 8

6.2 Models and DG Strategy Families

• The full-factorial benchmark varies four backbones, five initialization strategies, architectural modifications, and thirteen objectives including ERM.The four models are CNN-PFF, ResNet-SE-5, TS2Vec, and IMU-Transformer; initialization includes random and SSL strategies, while MixStyle, EFD-Mix, and DDG are architectural options.Source: §6.2 · p. 9

6.3 Training, Model Selection and Evaluation

• Models train for 10,000 steps, and checkpoint and hyperparameter selection use source-validation accuracy before held-out target evaluation.Runs use three seeds and are evaluated every 100 steps; target accuracy is the primary metric, with paired Wilcoxon tests at p < 0.05.Source: §6.3 · p. 9

6.4 Research Questions

• The four research questions examine individual component contributions, interactions, consistency across models and shifts, and source-validation checkpoint selection versus oracle selection.Source: §6.4 · p. 9

7.1 Objective-Based DG Methods

• In cross-dataset evaluation, no alternative objective significantly outperforms ERM; ERM++ has the largest positive mean difference (+0.8 pp), which is not significant.In cross-position evaluation, no objective differs significantly from ERM; ERM++ improves IMU-Transformer by +2.3 pp and TS2Vec by +2.0 pp, but effects remain model-dependent.Source: §7.1 · p. 10, Figure 4 · p. 10

7.2 Representation Initialization

• TF-C (LODO) significantly improves cross-position accuracy by +1.9 pp, while self-supervised initialization yields limited aggregate gains in cross-dataset evaluation.It is the only initialization reported to improve consistently across models and scenarios, with positive rates up to 55% across the hyperparameter search.Source: §7.2 · p. 10, Figure 5 · p. 11

7.3 Architectural Modifications

• DDG significantly improves aggregate cross-dataset accuracy by +1.4 pp and is the most robust architectural modification, while EFD-Mix and MixStyle can reduce it.DDG’s mean gains are positive across scenarios; EFD-Mix and MixStyle reduce aggregate cross-dataset accuracy by 3.4 and 1.9 pp, respectively.Source: §7.3 · pp. 11–12, Figure 6 · p. 12

7.4 Summary of Individual DG Components

• No individual DG component is universally superior; objective methods are weakest, TF-C (LODO) is the most reliable initialization, and DDG has the clearest architectural gains.Cross-position is generally more receptive to DG interventions than cross-dataset.Source: §7.4 · p. 12

8.1 Overall Performance Landscape

• The best cross-dataset configuration, ERM++ + DDG, reaches 70.5% accuracy (+2.9 pp over ERM); the best cross-position recipe combines ERM++, LFR (LODO), and DDG for +4.9 pp.All ten leading cross-dataset configurations include DDG; nine of ten cross-position configurations include DDG and nine use SSL initialization.Source: §8.1 · p. 12, Table 7 · p. 13

8.2 Complementarity Across DG Dimensions

• Mean gains rise from k = 1 to k = 2, then dip at k = 3: cross-dataset changes from +0.82 to +2.52 to +2.31 pp, and cross-position from +1.37 to +2.13 to +2.00 pp.IMU-Transformer reaches +7.05 pp in cross-dataset and +5.77 pp in cross-position at k = 3, while CNN-PFF and ResNet-SE-5 are weaker or negative in some settings.Source: §8.2 · p. 13, Figure 7 · p. 13
• Five of eight model–scenario pairs are super-additive; IMU-Transformer cross-dataset reaches +7.05 pp despite isolated effects totaling +1.86 pp.TS2Vec cross-dataset improves ERM by +2.44 pp even though isolated effects sum to −4.82 pp; ResNet-SE-5 is the main exception.Source: §8.2 · p. 14, Figure 8 · p. 14

8.3 Component Attribution of Winning Configurations

• Winning full pipelines gain +2.1 to +9.5 pp in cross-dataset and +5.8 to +9.6 pp in cross-position; interactions can make individually neutral or detrimental components beneficial.For CNN-PFF cross-position, ERM++ plus DDG yields +5.3 pp and the full pipeline +5.8 pp although all three components are neutral or detrimental alone.Source: §8.3 · p. 14, Table 8 · p. 15

8.4 Robustness Across Distribution-Shift Scenarios

• Cross-shift robustness varies by model: 45% of IMU-Transformer configurations improve both shifts, versus 4.2% for ResNet-SE-5; the strongest balanced IMU-Transformer recipe gains +9.1 pp in cross-dataset and +8.3 pp in cross-position.Across models, improving cross-position while degrading cross-dataset is more common than the opposite trade-off.Source: §8.4 · p. 15, Figure 9 · p. 16, Table 9 · p. 17

8.5 How Winning Configurations Improve Performance

• 13.3 pp: ERM++ + DDG raises IMU-Transformer target accuracy from 64.58% to 77.91%; class-level gains concentrate on stair recognition and walk/run confusions, while some class pairs worsen.The strongest significant changes include stair down recall (+11.0 pp), stair up recall (+7.5 pp), and reduced walk→run confusion (+7.6 pp).Source: §8.5.1 · p. 15, §8.5.2 · p. 16, Figure 10 · p. 17

8.6 Checkpoint-Selection Headroom Under Domain Shift

• At k = 3, source-validation selection recovers about 53% of available oracle gain in CD and 26% in CP; in CD, source selection is −1.03 pp versus +2.80 pp with oracle selection.Oracle performance improves monotonically as DG dimensions are added, whereas source-selected performance saturates or declines.Source: §8.6 · p. 19

9 Discussion

• ERM is difficult to outperform consistently in isolation; ERM++, TF-C (LODO), and DDG are the strongest alternatives in their respective component categories, but gains depend on model and shift.DDG provides the clearest standalone architectural gains.Source: §9 · p. 19
• Five of eight model–scenario pairs show empirical superadditivity, although additional components can also introduce redundancy or interference.Standalone performance often poorly predicts joint behavior; neutral or detrimental components can become beneficial in compatible combinations.Source: §9 · p. 19
• Cross-position shifts have stronger domain separability than cross-dataset shifts, with sensor placement identified as the strongest source of separability in the benchmark.The discussion links this to greater difficulty and larger headroom for suitable DG configurations in cross-position evaluation.Source: §9 · pp. 19–20
• The largest gains concentrate on difficult, shift-sensitive boundaries among walking, running, and stair activities, rather than uniformly across the decision space.The discussion suggests focusing on specific behavioral patterns; some class pairs worsen even as others improve.Source: §9 · p. 20
• At three active dimensions, source validation recovers only 53% of the oracle gain in cross-dataset and 26% in cross-position settings.Oracle performance rises as DG dimensions are added, while source-selected performance can saturate or decline.Source: §9 · p. 20

9.1 Limitations and Future Work

• The benchmark covers four model architectures, twelve objective-based alternatives to ERM, three architectural modifications, five initialization strategies, and two shift scenarios.Larger foundation models, generative approaches, and other sensing modalities remain outside its scope.Source: §9.1 · p. 20
• The study evaluates training-time DG only; test-time, continual, and online adaptation using unlabeled target data are left for future evaluation.The paper notes these paradigms may achieve additional performance gains.Source: §9.1 · p. 20
• The evaluation uses domain-balanced sampling and a fixed source-validation split, not repeated resampling or leave-one-source-domain-out validation.The authors identify source weighting, adaptive sampling, and more robust source-only selection as future directions.Source: §9.1 · p. 20

10 Conclusions

• No individual DG component is universally superior: ERM remains a strong objective baseline, TF-C (LODO) is the most reliable initialization, and DDG provides the clearest architectural gains.Compatible components can produce complementary or super-additive gains, while others introduce redundancy or interference.Source: §10 · p. 20
• DG effects remain model- and shift-dependent: DDG is particularly effective with IMU-Transformer, while the largest class-level gains involve difficult locomotion and stair-related boundaries.Cross-position shifts show stronger domain separability, consistent with sensor placement being a major source of shift.Source: §10 · p. 20
• At the highest compositional level, source-validation selection recovers only 53% of available oracle gain in cross-dataset and 26% in cross-position settings.Oracle performance increases as additional DG dimensions are combined.Source: §10 · p. 20
• Effective smartphone HAR DG depends on jointly designing and selecting compatible architecture, initialization, objective, and model-selection combinations.The conclusion recommends considering these components jointly rather than relying on any one in isolation.Source: §10 · p. 20

A Training Procedure and Hyperparameters

• The appendix section supplies further training-procedure and hyperparameter-search details for the evaluation.Source: A Training Procedure and Hyperparameters · p. 24

A.1 Number of Training Steps

• Oracle selection is earlier than source validation; TS2Vec shifts from approximately 4,700 source-validation steps to 3,500 oracle steps, reflecting different source- and target-performance peaks.The 10,000-step budget captures the evaluated optimization behavior; the remaining generalization gap is more strongly tied to checkpoint selection than insufficient optimization.Source: Figure 13 · p. 24, §A.1 · p. 25
• Under source validation, mean selected steps are approximately 6,600 for CNN-PFF, 6,800 for IMU-Transformer, 6,200 for ResNet-SE-5, and 4,700 for TS2Vec.At least 25% of runs for the first three backbones select checkpoints after roughly 8,300 steps; TS2Vec is selected earlier.Source: §A.1 · p. 24

A.2 Hyperparameter Search Grids

• Method-specific search grids draw on original-paper recommendations, DomainBed configurations when applicable, and preliminary experiments; Adam uses learning rates 10−3 and 10−4.Up to five configurations per method and learning rate are uniformly sampled, or all are evaluated if fewer than five exist.Source: §A.2 · p. 25, Table 10 · p. 26, Table 11 · p. 27
• Objective configurations are crossed with each initialization and the original or modified backbone, while hyperparameters are selected using source validation without target labels.Each run uses three seeds, and the reported target performance is the mean and standard deviation across those runs.Source: §A.2 · p. 25

A.3 Number of Configurations for Complementarity Analysis

• Complementarity analysis averages the ten best source-validation-selected configurations to reduce sensitivity to one unusually favorable configuration without diluting results with weak combinations.The fixed k = 10 choice is examined against the ranked distribution of configurations improving over ERM.Source: §A.3 · p. 25, Figure 14 · p. 27
• For CD, k = 10 covers approximately the top 20.4% of configurations outperforming ERM; for CP, it covers the top 4.4%.The authors describe this choice as focusing on the high-performing region while avoiding the long tail of progressively smaller gains.Source: §A.3 · pp. 25, 27