Source-linked AI summary
Class-Conditioned Gaussian Mixture Modeling for Imbalanced Time Series Quantification
Md Shahriar Kabir, Mayesha Maliha R. Mithila, Anne H. H. Ngu, Mylène C. Q. Farias, Byron Gao
TL;DR
The paper addresses prevalence estimation from unlabeled bags when time-series data are highly imbalanced and class priors may shift. It introduces CC-GMNet-TS, combining Transformer-based segment embeddings with per-class Gaussian mixtures in a bounded latent space. Across EMG, SmartFallMM, and UCI-HAR, it reports lower quantification error than classical and recent deep baselines, with ablations supporting the Transformer and PShift components.
Problem
Quantification estimates class prevalences in unlabeled bags, but imbalanced time series contain rare patterns and may experience deployment prior shifts.
Method
CC-GMNet-TS combines a Transformer feature extractor with separate class-conditioned Gaussian mixtures in a bounded latent space for bag-level quantification.
Results
CC-GMNet-TS achieves lower MRAE, NRAE, and KLD than classical aggregators and contemporary deep quantifiers across EMG, SmartFallMM, and UCI-HAR.
Takeaways & Limitations
Ablations support the effectiveness of the Transformer backbone and PShift, particularly for imbalanced distributions.
Takeaways & Limitations
The evaluation is limited to offline, pre-segmented recordings with a closed, fixed class set.
Abstract
from arXiv · showhide
Quantification, estimating class prevalences in bags of unlabeled instances is vital in domains where aggregate statistics are more important than individual instance labels, such as biosignal monitoring, fall detection, and activity recognition. We investigate this issue in the challenging setting of imbalanced time series data and develop CC-GMNet-TS, a class-conditioned Gaussian mixture quantifier that combines a Transformer-based feature extractor with per-class latent mixtures. Unlike previous mixture-based quantifiers, which use a single Gaussian mixture shared by all classes, CC-GMNet-TS assigns each class its own compact mixture in a bounded latent space and scores segment embeddings against these class-specific components to create bag-level representations that emphasize rare but informative patterns. Bags are constructed from labeled pools using the Artificial Prevalence Protocol (APP) and prior shift bag sampling (PShift) to cover a wide range of class prevalence scenarios, and the model is trained end-to-end with a quantification-oriented loss. Experiments on three benchmarks: EMG Data for Gestures, SmartFallMM, and UCI-HAR show that CC-GMNet-TS achieves lower error across the three benchmarks compared to traditional aggregators and recent deep quantifiers, while ablations confirm the contributions of both the Transformer backbone and class-conditioned mixtures during PShift.
1 Introduction
Quantification estimates class prevalences in unlabeled bags, a difficult task for imbalanced, variable time series under changing class priors. CC-GMNet-TS addresses this with Transformer embeddings and class-conditioned latent Gaussian mixtures, achieving lower error than established baselines across three benchmarks.
- Motivation: Quantification forecasts the class distribution within an unlabeled bag rather than assigning labels to individual instances.It matters when aggregate prevalences are more useful than detailed annotations, including biomedical monitoring, fall-risk assessment, and behaviour analysis.
- Motivation: Rare events may comprise less than 1% of observations, while temporal variability and deployment prior shifts make prevalence estimation especially challenging.The motivating domains include biomedical and human-activity signals with multi-channel, non-stationary, and subject-variable time series.
- Research gap: Class-agnostic mixture quantifiers model the marginal latent distribution, allowing majority modes to absorb minority patterns and weakening sensitivity to rare-class prevalence changes.Class-conditioned mixtures instead model each p(z | y = c) separately, reducing cross-class leakage and aligning representations with the prior-shift formulation.
- Proposed method: CC-GMNet-TS combines a Transformer feature extractor with per-class Gaussian mixtures in a bounded latent space for bag-level prevalence estimation.The framework uses a lightweight Transformer, a learnable class token, attention pooling, and class-specific mixture components.
- Evaluation: The model is evaluated on EMG, SmartFallMM, and UCI-HAR using APP and PShift bags, achieving lower error than classical aggregators and recent deep quantifiers.The reported advantage is especially pronounced on heavily imbalanced datasets.
2 Related Work
Prior work spans post-hoc classifier corrections, recurrent and regression-based deep quantifiers, and Transformer-based time-series modeling. However, classification-oriented imbalance methods do not directly match prevalence-estimation objectives.
- Quantification methods: Classical quantifiers such as CC, ACC, PCC, and PACC correct classifier outputs, while EM-based methods iteratively adjust scores to new class priors.These approaches treat quantification as a correction or adjustment stage rather than direct end-to-end bag-level learning.
- Deep quantification: Deep quantifiers learn prevalences directly from bags using architectures including QuaNet and Deep Quantification Networks.QuaNet aggregates instance embeddings recurrently, whereas DQN uses generic encoders and a regression head for quantification-specific losses.
- Time-series modeling: Transformers have demonstrated strong time-series modeling capabilities through long-range dependency capture and variants such as TFT, Informer, and PatchTST.PatchTST supports variable-length inputs using masking and has been applied to forecasting and sensor analysis.
- Rare-event estimation: Imbalanced time-series methods commonly use class-balanced or focal losses and data augmentation, but classification-centric objectives are not immediately compatible with prevalence estimation.This mismatch can produce inaccurate estimates for rare classes.
3 Methodology
CC-GMNet-TS estimates bag-level class prevalences from weakly supervised, imbalanced time-series segments using Transformer embeddings and class-specific Gaussian mixtures. APP and PShift construct varied prevalence scenarios, while an end-to-end quantification module converts class-conditioned bag representations into prevalence estimates.
- Problem formulation: The framework aggregates variable-length time-series segments into bags and learns prevalence estimates without using instance-level labels during training.Each bag contains segments with associated prevalence vectors, and the goal is reliable estimation when test-time class priors differ from training.
- Transformer-based feature extraction: A Transformer feature extraction module maps each segment to a latent representation using self-attention and attention pooling.The encoder is designed to capture long-range temporal dependencies, handle different segment lengths, and focus on informative time steps.
- Transformer-based feature extraction: Latent features are bounded in [0, 1]^d before Gaussian modeling to improve numerical stability, regularize feature scale, and reduce outlier sensitivity.The bounded representation also stabilizes covariance learning in the mixture module.
- Bag construction: APP and PShift generate bags with controlled prevalence distributions ranging from uniform to highly imbalanced, for consistent training and evaluation.PShift biases sampled prevalence vectors toward more realistic or extreme class mixtures while using the same underlying labeled pool.
- Class-conditioned bag representation: The bag representation module assigns each class its own Gaussian mixture and averages segment likelihoods across mixture components to form a fixed-dimensional representation.This permutation-invariant representation explicitly encodes class-dependent latent structure and gives minority classes dedicated mixture components.
- Quantification module and loss: A small multilayer perceptron maps the bag representation to prevalence estimates, and end-to-end training uses MRAE to penalize relative prevalence errors.The loss includes a small constant to avoid division by zero and control the influence of extremely rare classes.
4 Experimental Results
The experiments evaluate CC-GMNet-TS on three imbalanced time-series benchmarks using prevalence-shifted bags and compare feature extractors, samplers, and mixture capacities. Results favor the Transformer with PShift and a moderate mixture configuration, while the evaluation remains limited to offline, pre-segmented data with fixed classes.
- Datasets and protocol: The evaluation covers EMG, SmartFallMM, and UCI-HAR using MRAE, NRAE, and KLD on bags of short time-series segments.The primary experiments create 200 training bags and 50 validation/test bags per dataset, each containing 20 segments.
- Comparative evaluation: CC-GMNet-TS is compared with classical aggregators and recent deep quantifiers under a common bag structure and training protocol.The baselines include CC, ACC, PCC, HistNetQ, DQN, and QuaNet.
- Ablation study: Replacing LSTM with a Transformer reduces both MRAE and KLD across all three datasets, while PShift improves results over APP for a fixed encoder.On EMG, MRAE drops from 0.242 to 0.223 when moving from LSTM+APP to Transformer+APP.
- Ablation study: Transformer+PShift consistently achieves the lowest MRAE and KLD on EMG, SmartFallMM, and UCI-HAR.PShift generates more skewed and diversified prevalence distributions than APP, exposing the model to broader imbalance scenarios.
- Hyperparameter study: K = 64 and d = 256 provide the most consistent moderate-capacity configuration across datasets.EMG reaches its lowest MRAE at 0.215, and SmartFallMM at 0.211, both with K = 64 and d = 256; UCI-HAR also minimizes error there.
- Discussion and limitations: The study assumes a closed, fixed class set and offline, pre-segmented data generated through synthetic bag procedures.These restrictions exclude streaming, online, and uncertain-label circumstances from the evaluation.
5 Conclusion
CC-GMNet-TS combines a Transformer encoder with per-class latent Gaussian mixtures for quantifying imbalanced time series. Across three benchmarks, it reports lower quantification errors than classical and contemporary deep quantifiers, while offline pre-segmented recordings and closed class sets limit the evaluation scope.
- Conclusion: CC-GMNet-TS combines a Transformer-based encoder with per-class latent mixtures and a quantification-oriented loss.The framework is designed for bag-level prevalence estimation and emphasizes unusual but meaningful patterns.
- Conclusion: Across EMG, SmartFallMM, and UCI-HAR, CC-GMNet-TS achieves lower MRAE, NRAE, and KLD than classical aggregators and contemporary deep quantifiers.The paper reports notable improvements on highly imbalanced datasets.
- Conclusion: Ablations support the effectiveness of both the Transformer backbone and PShift.The conclusion identifies both components as effective design choices.
- Limitations: The evaluation is limited to offline, pre-segmented recordings with closed class sets.The framework is therefore assessed outside streaming, online, and uncertain-label settings.