PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion-Based Synthetic Data Pretraining for Enhancing Activity Recognition

E. Riveros, D. Vega-Oliveros, A. Soriano-Vargas, A. Rocha

arXiv:2610.02292v1cs.LGcs.AI

TL;DR

Fine-grained eating and drinking recognition is difficult amid class imbalance and subtle wrist motion. The paper pre-trains CABiGRU on diffusion-generated smartwatch data, then fine-tunes on real data. Diffusion pretraining achieves 0.9055 test balanced accuracy, ahead of PCF-GAN pretraining at 0.8802 and the from-scratch baseline.

  • Problem

    The authors identify limited fine-grained alimentary recognition and limited evaluation of synthetic time-series as downstream pretraining signals.

    Source: §II · p. 2

  • Method

    The pipeline pre-trains CABiGRU on generated multivariate smartwatch windows, then fine-tunes it on real DEO data.

    Source: §III · p. 2

  • Results

    0.9055 test balanced accuracy with diffusion pretraining exceeds PCF-GAN pretraining at 0.8802 and the from-scratch baseline.

    Source: D. Comparative Discussion and Error… · p. 4

  • Takeaways & Limitations

    The authors conclude that minority-class diffusion samples improve generalization and robustness for subtle, underrepresented activities such as eating and drinking.

    Source: §V · p. 5

Abstract

from arXiv · show

Human activity recognition (HAR) is increasingly important for healthcare, well-being, and daily monitoring ap- plications, for which detecting alimentary activities such as eating and drinking can provide actionable insight into dietary habits and chronic disease management. HAR systems, however, often underperform on subtle and underrepresented classes, limiting their utility in real-world dietary monitoring. This work builds upon CABiGRU, a convolutional architecture with Bidirectional GRU layers, multi-head attention, and residual connections, designed to capture discriminative temporal patterns from smart- watch accelerometer, gyroscope, and magnetometer data. To improve CaBiGRU's generalization and reduce underfitting in the minority class, we leverage synthetic sensor data windows using a diffusion model and adopt a two-stage training strategy: pre-training CABiGRU on synthetic data, followed by fine-tuning on the real-world data. On the DEO (drinking/eating/other) dataset, the proposed pipeline achieves a balanced accuracy of 90.6%, improving over a strong supervised baseline and showing the benefits of diffusion-based synthetic pre-training for recognizing alimentary activities and representing a step forward dealing with unbalanced classes. These results suggest that combining diffusion-generated data with targeted fine-tuning enhances robust recognition of dietary behaviors, supporting more reliable deployment in healthcare and nutrition-monitoring settings.

I. INTRODUCTION

Alimentary activity recognition is challenged by class imbalance and subtle wrist-motion patterns. The paper adapts diffusion-based synthetic pretraining for smartwatch data and reports improved DEO classification performance.

I. INTRODUCTION

  • Alimentary activities combine class imbalance with subtle wrist movements and short temporal patterns, while minority-class scarcity limits generalization.Prior work introduced CABiGRU and reported 86.6% balanced accuracy.Source: §I · p. 1
  • The authors identify diffusion as an alternative to augmentation approaches including SMOTE, autoencoders, and GANs for fine-grained time-series discrimination.They adapt PaD-TS [6] by replacing encoder–decoder attention with Bidirectional GRU blocks for short inertial sequences.Source: §I · p. 1
  • 90.6% balanced accuracy on DEO exceeds the 86.6% scratch-trained baseline and adversarial synthetic pretraining.The recurrent redesign replaces attention in the generator with Bidirectional GRU blocks.Source: §I · p. 1
  • A two-stage pipeline pre-trains CABiGRU on synthetic multivariate smartwatch samples, then fine-tunes it on real data.The authors describe the approach as addressing minority-class underfitting and generalization.Source: §I · p. 1
  • The stated contributions are diffusion-based pretraining for fine-grained HAR, improved temporal modeling with bidirectional GRUs, and validation on real wearable data.The recurrent redesign is reported to reduce overfitting.Source: §I · p. 1

II. RELATED WORK

Prior work spans wearable HAR architectures and synthetic time-series generation, but the authors identify gaps in fine-grained alimentary recognition and downstream evaluation of synthetic data. Their study evaluates diffusion-based pretraining for smartwatch alimentary HAR.

II. RELATED WORK

  • Wearable HAR uses classical and deep-learning methods, but most work targets common activities; eating and drinking remain difficult because of subtle wrist motion and fine-grained temporal structure.Related examples include AutoHydrate and multi-sensor drinking detection approaches.Source: §II · pp. 1–2
  • Synthetic time-series work includes PCF-GAN [7] and PaD-TS [6], but is typically evaluated for sample realism rather than downstream recognition utility.Source: §II · p. 2
  • The authors identify three gaps: limited fine-grained eating–drinking recognition, constrained alimentary-monitoring setups, and synthetic data rarely evaluated as a downstream pretraining signal.They address these by adapting a generator for multivariate smartwatch windows and evaluating under a subject-wise DEO protocol against scratch training and GAN pretraining.Source: §II · p. 2

III. PROPOSED METHODOLOGY

The methodology generates multivariate smartwatch windows for CABiGRU pretraining, then fine-tunes the model on real DEO data. The dataset uses three activity classes and overlapping sensor windows.

III. PROPOSED METHODOLOGY

  • Signals are segmented into 5-second windows with 50% overlap, and a diffusion model generates multivariate samples while preserving per-channel distributions and cross-sensor dependencies.Source: §III · p. 2
  • CABiGRU is pre-trained on generated windows and fine-tuned on real DEO windows for task-specific adaptation.The authors report improved balanced accuracy and reduced overfitting versus training from scratch.Source: §III · p. 2

A. Data Collection

  • The study defines DEO as drinking, eating, and other, with other grouping non-alimentary activities; 90 participants wore a Samsung GSW4 smartwatch.Signals were recorded at 100 Hz and represented in overlapping 5-second windows of 500 time steps and 9 features.Source: A. Data Collection · p. 2

B. Diffusion-Based Synthetic Pretraining

The synthetic pretraining approach generates multivariate smartwatch windows with diffusion and adapts PaD-TS using Bidirectional GRU blocks. Generated samples are limited to minority classes during training.

B. Diffusion-Based Synthetic Pretraining

  • The diffusion model learns the joint distribution of accelerometer, gyroscope, and magnetometer samples while preserving channel statistics and cross-sensor dependencies.Source: B. Diffusion-Based Synthetic Pretraining · p. 2
  • The approach modifies PaD-TS [6] by replacing its attention-based encoder–decoder with Bidirectional GRU blocks for short multivariate inertial windows.The redesign maintains the population-aware objective while better capturing temporal dependencies and reducing overfitting.Source: B. Diffusion-Based Synthetic Pretraining · p. 2
  • Generated samples are used only during training and only for eating and drinking, increasing realistic motion-pattern diversity in underrepresented classes.The authors contrast full multivariate windows with SMOTE or autoencoders for preserving temporal dependencies needed to distinguish eating and drinking.Source: B. Diffusion-Based Synthetic Pretraining · p. 2

C. CABiGRU Backbone

CABiGRU combines convolutional feature extraction, bidirectional GRUs, multi-head attention, and residual connections for smartwatch alimentary activity recognition. The proposed training process pre-trains its convolutional and GRU layers on generated data, then fine-tunes the full model on real data.

C. CABiGRU Backbone

  • CABiGRU combines convolutional feature extraction, bidirectional GRUs, multi-head attention, and residual connections for subtle wrist-motion classification.It is used as a previously validated downstream backbone for smartwatch alimentary activity recognition.Source: C. CABiGRU Backbone · p. 2
  • The paper describes diffusion-based synthetic pre-training as conceptually independent of CABiGRU and potentially integrable with other multivariate inertial sequence models.CABiGRU is selected as a previously validated, relevant backbone for assessing the pre-training strategy.Source: C. CABiGRU Backbone · p. 3

D. Two-Stage Training (Pre-train + Fine-tune)

  • Pre-training on generated sensor data trains CABiGRU’s convolutional and GRU layers; fine-tuning introduces the classification head and optimizes the full model on real data at a lower learning rate (1e-4).The two-stage procedure is described as self-supervised pre-training followed by adaptation to real sensor data.Source: D. Two-Stage Training (Pre-train +… · p. 3
  • GELU with the deep classifier head achieved the best tested balanced accuracy, 0.9055.The tested design combinations varied four activation functions and four classifier-head configurations.Source: D. Two-Stage Training (Pre-train +… · p. 3
  • The diffusion model is trained on raw multivariate time-series windows using Bidirectional GRU layers.The pipeline then uses its generated windows to pre-train CABiGRU before real-data fine-tuning.Source: Figure 2 · p. 3, Figure 1 · p. 3

IV. EXPERIMENTAL RESULTS

The experimental section evaluates the pipeline on the DEO multiclass dataset using a subject-wise protocol. Balanced accuracy is the primary metric, with per-class and curve-based measures also reported.

A. Experimental Setup and Evaluation Protocol

  • Evaluation uses a subject-wise split of 60% training, 10% validation, and 30% testing.The test set is subject-disjoint from training and validation.Source: A. Experimental Setup and Evaluation… · p. 3
  • Synthetic samples are used for self-supervised pre-training, while downstream classification is fine-tuned on the original DEO data splits.The passage states this protocol avoids information leakage because synthetic data is generated from eating and drinking classes.Source: A. Experimental Setup and Evaluation… · p. 3
  • Balanced accuracy, defined as the mean of per-class sensitivity and specificity, is the primary metric because the alimentary classes are imbalanced.The evaluation also reports per-class sensitivity and specificity, confusion matrices, ROC curves with per-class AUC, and learning curves.Source: A. Experimental Setup and Evaluation… · pp. 3–4

B. Diffusion-Based Synthetic Pretraining

The two-stage strategy pre-trains CABiGRU on diffusion-generated windows before fine-tuning on real data. It achieves 0.9055 test balanced accuracy, while learning curves show more stable validation behavior with pre-training.

B. Diffusion-Based Synthetic Pretraining

  • 0.9055 test balanced accuracy was achieved by CABiGRU pre-trained on diffusion-generated windows and then fine-tuned on real data.Per-class sensitivities were 0.8849 for Drink, 0.9014 for Eat, and 0.9304 for Other.Source: B. Diffusion-Based Synthetic Pretraining · p. 4
  • Most misclassifications occur between Drink and Eat, while Other remains highly separable; ROC AUC values range from 0.92 to 0.94 across the three classes.The passage attributes the Drink–Eat confusion to overlapping wrist-motion dynamics.Source: B. Diffusion-Based Synthetic Pretraining · p. 4
  • With synthetic pre-training, training and validation curves converge more consistently and validation loss remains lower and stable than without pre-training.The curves qualitatively support the quantitative gains reported in Table I.Source: B. Diffusion-Based Synthetic Pretraining · p. 4, Figure 3 · p. 4

C. PCF-GAN Synthetic Pretraining

Diffusion pre-training achieves the best test balanced accuracy among the compared strategies, ahead of PCF-GAN and scratch training. PCF-GAN improves minority-class detection over scratch but remains below diffusion on reported comparisons.

C. PCF-GAN Synthetic Pretraining

  • PCF-GAN pre-training generates extra windows only for Drinking and Eating during initial pre-training, before CABiGRU is fine-tuned on the original real DEO training split.Validation and test sets remain fully real.Source: C. PCF-GAN Synthetic Pretraining · p. 4

D. Comparative Discussion and Error Analysis

  • 0.9055 test balanced accuracy with diffusion pre-training exceeds PCF-GAN’s 0.8802 and the from-scratch baseline.The comparison uses CABiGRU under the same downstream setting.Source: D. Comparative Discussion and Error… · p. 4
  • Diffusion’s iterative denoising is described as enforcing local temporal consistency and cross-sensor dependencies, whereas adversarial methods may miss precise temporal structure.The passage links those structures to distinguishing fine-grained activities such as eating versus drinking.Source: D. Comparative Discussion and Error… · p. 4
  • GELU with the deep classifier head achieved the best balanced accuracy among the tested activation and head combinations, at 0.9055.This configuration was used in the main comparative experiments.Source: D. Comparative Discussion and Error… · p. 4
  • PCF-GAN pre-training reaches 0.8802 balanced accuracy, with sensitivities of 0.8139 (Drink), 0.9216 (Eat), and 0.9051 (Other).PCF-GAN improves minority-class detection over the from-scratch baseline, but its gains remain below diffusion.Source: D. Comparative Discussion and Error… · p. 4
  • Diffusion achieves higher macro F1 than PCF-GAN, 0.8243 versus 0.7764.The passage describes the gains as more consistent across classes, particularly for underrepresented alimentary categories.Source: D. Comparative Discussion and Error… · p. 4
  • Diffusion achieves higher weighted F1 than PCF-GAN, 0.9315 versus 0.9089.The passage interprets the result as stronger overall performance across the full test set.Source: D. Comparative Discussion and Error… · p. 4

E. Effect of Synthetic Data Volume

The section tests whether increasing diffusion-generated pretraining data improves generalization. Under the same subject-wise protocol, increasing the volume from N to 2N reduced test balanced accuracy.

E. Effect of Synthetic Data Volume

  • The scaling experiment examined whether increasing synthetic data volume improves generalization.Source: E. Effect of Synthetic Data Volume · p. 4
  • CABiGRU was pretrained on diffusion-generated windows and fine-tuned on the original real DEO training split for both volume settings.The experiment compared N synthetic windows per minority class with 2N under the same subject-wise protocol.Source: E. Effect of Synthetic Data Volume · p. 5
  • 0.9055 to 0.8338: increasing synthetic pretraining volume from N to 2N reduced test balanced accuracy.The authors suggest the larger set may contain redundant or synthetic-specific patterns, reducing transferability to real DEO data.Source: E. Effect of Synthetic Data Volume · p. 5

V. CONCLUSIONS AND FUTURE WORK

The conclusion presents two-stage diffusion pretraining and real-data fine-tuning as a way to improve recognition of subtle, underrepresented eating and drinking activities. It also cautions that larger synthetic sets can reduce balanced accuracy and outlines future extensions to populations, sensors, and explainability.

V. CONCLUSIONS AND FUTURE WORK

  • The proposed pipeline pretrains CABiGRU on diffusion-generated samples, then fine-tunes it on real smartwatch signals.Synthetic samples are generated for minority classes, giving eating and drinking greater representation during pretraining.Source: §V · p. 5
  • Fine-tuning on real data lets the model adapt to specific sensor signals; the authors report a balanced-accuracy boost, especially for eating and drinking.They report using a learning rate of 1e−4.Source: §V · p. 5
  • 0.9055 to 0.8338: increasing synthetic pretraining volume from N to 2N reduced balanced accuracy, so augmentation should not be overused.Source: §V · p. 5
  • Excessive synthetic data may introduce redundant or less transferable patterns, weakening pretraining for real downstream recognition.Source: §V · p. 5
  • The authors report that diffusion models preserve temporal continuity and cross-sensor dependencies, especially for subtle, underrepresented target classes.Source: §V · p. 5
  • Future work will broaden evaluation to older adults and people with health conditions, add sensor modalities, refine feature extraction, and develop explainability methods.Source: §V · p. 5
Loading 2610.02292v1…