Source-linked AI summary
Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding
Shohreh Deldari, Daniel V. Smith, Hao Xue, Flora D. Salim
TL;DR
CPD must identify meaningful transitions in time series, but existing methods often specialise in particular signal properties and may generalise poorly. TS-CP^2 learns compact representations with self-supervised contrastive predictive coding, and experiments across three datasets show it outperforms five CPD baselines. Its main scope boundaries include false negative pairs from recurring patterns and false positive pairs caused by dense change points.
Problem
Existing CPD methods often focus on one time-series property, limiting their ability to generalise across different types of change points.
Method
TS-CP^2 uses self-supervised contrastive learning to learn compact embeddings that relate adjacent intervals and temporally separated intervals.
Results
TS-CP^2 achieved one of the two highest results across each dataset, significantly improving over five baselines on Yahoo! and USC-HAD and outperforming each baseline on average across HASC window sizes.
Takeaways & Limitations
Contrastive representation learning provides a CPD approach that performs strongly across datasets with differing transition characteristics.
Takeaways & Limitations
Recurring patterns can create false negative pairs, while dense change points can create false positive pairs that degrade self-supervised training.
Abstract
from arXiv · showhide
Change Point Detection (CPD) methods identify the times associated with changes in the trends and properties of time series data in order to describe the underlying behaviour of the system. For instance, detecting the changes and anomalies associated with web service usage, application usage or human behaviour can provide valuable insights for downstream modelling tasks. We propose a novel approach for self-supervised Time Series Change Point detection method based onContrastivePredictive coding (TS-CP^2). TS-CP^2 is the first approach to employ a contrastive learning strategy for CPD by learning an embedded representation that separates pairs of embeddings of time adjacent intervals from pairs of interval embeddings separated across time. Through extensive experiments on three diverse, widely used time series datasets, we demonstrate that our method outperforms five state-of-the-art CPD methods, which include unsupervised and semi-supervisedapproaches. TS-CP^2 is shown to improve the performance of methods that use either handcrafted statistical or temporal features by 79.4% and deep learning-based methods by 17.0% with respect to the F1-score averaged across the three datasets.
1 INTRODUCTION
The paper addresses CPD methods’ limited generalisation across different time-series properties by proposing TS-CP^2, a self-supervised contrastive representation-learning approach. It frames the method as a way to detect broader change points without relying on data-distribution assumptions.
- Motivation: The approach targets non-annotated data, for which conventional supervised annotation can be expensive, unwieldy, and inaccurate.This motivates self-supervised and unsupervised learning for extracting value from raw data.
- Motivation: Existing CPD methods often target one property, such as temporal shape or statistical distribution, limiting their generalisation across applications.The paper contrasts shape-focused FLOSS with distribution-focused RuLSIF and aHSIC.
- Approach: TS-CP^2 learns representations that maximise shared information between adjacent intervals while separating temporally distant intervals.The method hypothesises that large differences between representations of adjacent intervals indicate likely change points.
- Contributions: The paper proposes contrastive learning as an unsupervised CPD objective and learns compact latent embeddings for historical and future intervals.It presents this as the first use of contrastive learning for CPD and compares the method with five state-of-the-art baselines.
- Contributions: The paper makes its code, data, and experiments available online to support reproducibility.
2 RELATED WORK AND BACKGROUND
The related work covers feature-specific CPD, deep-learning and representation-learning approaches, and contrastive learning. It positions the paper as applying contrastive learning to time-series CPD, extending techniques previously used mainly in other modalities.
- Time-series change point detection: Existing CPD methods are grouped by the time-series features they exploit, including statistical properties and temporal shape patterns.Statistical approaches use parametric or non-parametric comparisons or segmentation costs, while shape-based methods detect salient pattern changes.
- Time-series change point detection: Deep-learning CPD methods include autoencoder reconstruction-error approaches and KL-CPD, which learns kernel parameters and combines multiple kernels.
- Representation learning: Representation learning has been applied across video, image, text, and time-series data, while most prior work focuses on natural language and computer vision.A prior multivariate time-series approach used WaveNet with an unsupervised triplet loss.
- Representation learning: Contrastive learning brings similar sample pairs together and separates dissimilar pairs in an embedding space.The background discusses contrastive, triplet, and multiple-negative learning objectives.
- Representation learning: Contrastive predictive coding uses autoregressive models to learn compact, abstract representations for downstream modelling tasks.
3 METHOD
TS-CP^2 learns compact representations of adjacent time windows with contrastive predictive coding, then detects change points from shifts in embedding similarity. Its training contrasts contiguous positive pairs with temporally separated negative pairs.
- TS-CP^2 defines change points as times when future behavior cannot be anticipated from preceding data, using dissimilarity between future and anticipated representations for detection.
- Representation learning: The encoder maps contiguous history and future windows into compact embeddings and maximizes their mutual information using an auto-regressive convolutional network.
- Representation learning: The encoder uses two TCN blocks with kernel size 4, dilation rates 1, 4, and 16, followed by a three-layer projection head.
- Representation learning: Contrastive training uses adjacent intervals as positive pairs and temporally separated intervals as negative pairs to learn a representation that separates local continuity from cross-time differences.
- Negative sampling: Positive pairs are randomly sampled with a minimum temporal separation between pairs, allowing other batch future windows to form negatives at relatively low complexity.
- Change point detection: Change points are detected from local minima in the difference between consecutive-window cosine similarity and its recent moving average.
4 EXPERIMENTS
The experiments evaluate TS-CP^2 across datasets and compare its change-point detection behavior using cosine similarity between consecutive windows. The evaluation includes implementation details and visual examples of detected change points.
- The evaluation presents datasets, baseline comparisons, and sensitivity analysis for TS-CP^2.
- Figure 6 illustrates temporally separated negative pairs whose history or future frame contains a change point or anomaly.
- Figure 7 uses consecutive-window cosine similarity, its moving average, and highlighted intervals to illustrate change-point detection on Yahoo! Benchmark-4.
4.1 Datasets
The experiments use three application-diverse time-series datasets covering web-service traffic, human activity, and mobile-application usage. The datasets differ in sample counts, sequence counts, channel dimensionality, and change-point counts.
- The evaluation spans web-service traffic analysis, human activity recognition, and mobile-application usage analysis.
- Yahoo! Benchmark-4 provides 100 time series containing varying trend, seasonality, noise, and random anomaly change points.
- HASC contributes three-axis accelerometer recordings for transitions among activities such as staying, walking, jogging, skipping, and stair movement.
- Table 1 organizes dataset properties by total samples, number of sequences, channel dimensionality, and total change points.
4.2 Baseline Methods
TS-CP^2 is compared with five state-of-the-art unsupervised CPD methods using publicly available implementations and an F1-score evaluation procedure. Baseline settings include dataset-specific windows and method-specific hyperparameters.
- The comparison includes ESPRESSO, FLOSS, aHSIC, RuLSIF, and KL-CPD as five state-of-the-art unsupervised CPD baselines.
- Baseline implementations use publicly available source code to reduce inconsistencies and implementation errors in the comparison.
- Detection performance is evaluated across dataset-specific window sizes, with F1-score calculated using the KL-CPD evaluation approach.
- KL-CPD training is unsupervised but still uses ground-truth labels to fine-tune hyperparameters during validation.
4.3 Evaluation Metrics
The evaluation uses F1-score while varying the allowable detection error margin, with each estimate matched to ground-truth change points under explicit true- and false-positive rules.
- F1-score is reported for each dataset under three detection margins because localization tolerance affects CPD evaluation.Performance is evaluated using the F1-score across the three error margins specified in Table 2.
- A prediction is a true positive when it falls within the specified error margin of a ground-truth change point.When multiple predictions fall within the margin, only the closest is counted as a true positive.
4.4 Fine-Tuning and Sensitivity Analysis
The sensitivity analysis examines window size, batch size, and code size, showing that longer windows and larger batches generally improve detection while compact intermediate embeddings can outperform the largest codes.
- Sensitivity Analysis: Figure 8 summarizes sensitivity to code size, batch size, and four Yahoo!Benchmark window sizes.The four windows shown are 24, 50, 75, and 100 samples.
- Window Size: Longer windows produced monotonically higher detection performance averaged across code size and batch size.The tested Yahoo!Benchmark windows ranged from 24 to 100 hourly samples, or one to four days.
- Batch Size: Larger batch sizes generally improved detection performance averaged across code size and window size.The tested batch sizes were 4, 8, 16, 32, 64, and 128, although the largest batches sometimes underperformed for the smallest code size.
- Contrastive Learning Caveat: Contrastive training can be degraded by negative pairs that are temporally separated but similarly distributed because time-series patterns may repeat.Such pairs are treated as false negatives and are pushed apart in the embedding space.
- Code Size: Code size had a non-monotonic relationship with detection performance, with embeddings of 8–12 dimensions often outperforming the largest codes.The smallest code size of 4 was relatively weak, while the optimal size depended on window and batch size.
4.5 Baseline Comparison
TS-CP^2 generally outperformed the baseline CPD methods across Yahoo!, USC-HAD, and HASC, while performance varied with dataset difficulty, detection margin, and training configuration.
- Yahoo! Benchmark Dataset Evaluation: On Yahoo!, TS-CP^2 strongly outperformed every baseline, whereas FLOSS and aHSIC missed subtle statistical changes that challenged the other methods.The Yahoo! Benchmark contains anomalies involving changes in seasonality, trend, and noise.
- USC-HAD Dataset Evaluation: Larger batch sizes performed better with longer windows, while shorter batch sizes performed better with smaller windows for TS-CP^2.This batch-size pattern was reported for the USC-HAD dataset.
- HASC Dataset Evaluation: On HASC, TS-CP^2 achieved a 19.2%, 54.8%, 10.1%, 11.1%, and 3.8% average F1-score improvement over FLOSS, aHSIC, RuLSIF, ESPRESSO, and KL-CPD, respectively.HASC was the most challenging dataset, with roughly 39K samples and 65 change points.
- Limitations and future work: High-frequency change points can create positive training pairs that cross change points, degrading self-supervised training because ground-truth labels cannot correct them.The authors suggest light negative mining and additional positive pairs through augmentation as remedies.
- Overall comparison: TS-CP^2 achieved either the first or second highest result across each dataset, with significant gains on Yahoo! and USC-HAD and superior average window-size performance on HASC.The method strongly outperformed every baseline on Yahoo! and achieved the highest average F1-score on HASC despite being second-best at each window size.
- Implementation: TS-CP^2 uses a shared compact representation and compares history and future embeddings, enabling faster convergence and potentially online operation on low-resource devices.After representation training, change-point detection only requires comparing learned representations.
5 CONCLUSION
The conclusion presents TS-CP^2 as a self-supervised contrastive method for time-series change-point detection and reports evaluation across three datasets. It outperformed baselines on two datasets and achieved a comparable score on the third.
- 5 CONCLUSION: TS-CP^2 learns an embedded representation that predicts a future time-series interval from historical samples and detects change points using embedding agreement.The method employs contrastive learning to produce a compact representation for each frame.
- 5 CONCLUSION: TS-CP^2 outperformed other baselines on two datasets and reached a comparable score on the remaining dataset.The method was evaluated against six state-of-the-art methods across three datasets.
- 5 CONCLUSION: Future work will extend the pretrained method toward continuously learning changes, anomalies, and drifts in data.The conclusion also notes that the pretrained method can detect changes in online applications.