Source-linked AI summary
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
Stephan Rabanser, Stephan Günnemann, Zachary C. Lipton
TL;DR
Machine-learning systems can fail silently when input distributions shift, but practical high-dimensional detection methods remain unsettled. The paper empirically combines dimensionality reduction with two-sample testing across varied shifts and datasets, finding that soft label-classifier outputs perform best overall while domain classifiers help characterize shift malignancy. Its conclusions are bounded by the image-focused experiments and unresolved online-shift and cross-domain extensions.
Problem
Machine-learning pipelines rarely inspect incoming data for distribution shift, and best practices for detecting shifts in high-dimensional real-world data have not been established.
Method
The paper empirically evaluates dimensionality-reduction representations combined with statistical two-sample tests, alongside domain classifiers for shift characterization.
Results
BBSDs was the best-performing dimensionality-reduction method overall, while domain classifiers helped characterize shifts and distinguish malignant from benign shifts.
Takeaways & Limitations
Soft black-box shift detection works across varied shifts, and domain-discriminating predictions provide useful qualitative and malignancy information.
Takeaways & Limitations
The experiments mostly use standard image classification, leaving online data and other domains such as language and graphs as open directions.
Abstract
from arXiv · showhide
We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings. Machine learning (ML) systems, however, which depend strongly on properties of their inputs (e.g. the i.i.d. assumption), tend to fail silently. This paper explores the problem of building ML systems that fail loudly, investigating methods for detecting dataset shift, identifying exemplars that most typify the shift, and quantifying shift malignancy. We focus on several datasets and various perturbations to both covariates and label distributions with varying magnitudes and fractions of data affected. Interestingly, we show that across the dataset shifts that we explore, a two-sample-testing-based approach, using pre-trained classifiers for dimensionality reduction, performs best. Moreover, we demonstrate that domain-discriminating approaches tend to be helpful for characterizing shifts qualitatively and determining if they are harmful.
1 Introduction
Machine-learning systems can fail silently under distribution shift, while practical detection methods for high-dimensional data remain unsettled. The paper studies two-sample-testing pipelines to detect shifts, characterize them, and assess their harm.
- Subtle changes in data distributions can severely damage machine-learning performance, including through covariate and label-distribution shifts.
- The paper targets detecting shifts from few examples, identifying over-represented test samples, and assessing whether shifts harm predictive performance.
- High-dimensional two-sample testing remains an open practical question because kernel methods scale poorly and lose statistical power with ambient dimension.
- The proposed framework combines dimensionality reduction with statistical hypothesis testing and compares multiple representations and two-sample tests.
- BBSD performs well across a broad set of shifts, including settings where its label-shift assumption is not satisfied.
2 Related work
Related work connects dataset-shift detection to anomaly detection, change-point detection, domain adaptation, and out-of-distribution detection. Existing domain-adaptation methods commonly rely on assumptions such as covariate or label shift.
- With one test example, dataset-shift detection reduces to anomaly detection, while time-dependent abrupt changes correspond to change-point detection.
- Domain-adaptation research often assumes covariate shift or label shift because correcting distribution shift without assumptions is impossible.
- Out-of-distribution methods use classifier confidence, calibrated or perturbed inputs, adapted losses, and ensembles to improve sample-level detection.
3 Shift Detection Techniques
The paper formulates shift detection as testing equality between source and target distributions after dimensionality reduction. It compares representations, statistical tests, and aggregation strategies, while using domain classifiers for sample characterization.
- The testing problem is H0: p(x) = q(x′) versus HA: p(x) ≠ q(x′), with design choices spanning representations, tests, dimensionality, and result aggregation.
- Dimensionality Reduction: Dimensionality-reduction methods map D-dimensional inputs to K dimensions, but can produce false negatives when shifted inputs retain unchanged latent distributions.
- Dimensionality Reduction: The evaluated representations include raw features, PCA, sparse random projections, autoencoders, label-classifier outputs, and domain-classifier predictions.
- Statistical Hypothesis Testing: The domain classifier distinguishes source from target samples and tests whether held-out accuracy differs statistically from random chance.
- Statistical Hypothesis Testing: Multivariate representations are tested with Maximum Mean Discrepancy, while continuous latent dimensions use separate KS tests with Bonferroni correction.
- Most Anomalous Samples: Non-domain detectors compare whole distributions rather than assigning individual samples, and initial attempts to identify samples driving shifts were unremarkable.
- Shift Malignancy: Because target labels are unavailable, the paper seeks to distinguish malignant from benign shifts without directly computing target error.
4 Experiments
Experiments evaluate shift detectors on image datasets under varied covariate and label perturbations. The design varies shift type, intensity, affected-data fraction, and dataset domain while controlling significance testing and repeated splits.
- Datasets: Experiments use MNIST and CIFAR-10 as main datasets, with additional original-split and domain-adaptation settings including Fashion-MNIST, SVHN, USPS, and COIL-100.
- Methods: The study compares PCA, SRP, UAE, TAE, BBSDs, BBSDh, and Classif representations, using K = 32 latent dimensions for PCA, SRP, UAE, and TAE.
- Shift Conditions: Shifts affect covariates or label proportions and include adversarial samples, class knock-outs, Gaussian noise, image transformations, and combined perturbations.
- Evaluation: Detection is evaluated at α = 0.05, includes no-shift false-positive checks, averages results over five random splits, and varies affected fractions δ ∈ {0.1, 0.5, 1.0}.
- Additional Settings: Experiments also include only-zero plus image shifts and provide a Python implementation of the experimental pipeline.
5 Discussion
Across the evaluated shifts, BBSDs was the strongest dimensionality-reduction method for univariate testing, while UAE performed best for multivariate testing. Detection difficulty increased as shifts became smaller or affected fewer samples, whereas domain classifiers helped characterize exemplars and shift malignancy.
- Univariate VS Multivariate Tests: Multiple univariate tests with Bonferroni correction offered performance comparable to multivariate kernel tests.This result held despite the heavy correction applied to the univariate tests.
- Dimensionality Reduction Methods: BBSDs performed best overall for multiple univariate testing, while UAE performed best for multivariate testing.Both methods consistently outperformed alternatives across sample sizes.
- Shift Strength: Large shifts were detectable with better-than-chance accuracy using 20 target samples, whereas medium and small shifts required orders of magnitude more samples.The reported ordering was small shifts, then medium shifts, then large shifts in increasing detectability.
- Shift Strength: With 50 target samples, perturbation levels of 50% and 100% were detectable above chance, while 10% anomalous data remained difficult to detect.The paper suggests that the 10% setting might be better addressed via outlier detection.
- Most Anomalous Samples and Shift Malignancy: Domain-classifier exemplars helped characterize shifts, and the framework distinguished malignant from benign shifts.Two to three orders of magnitude fewer labeled examples than the total sample size sufficed to approximate target accuracy in the experiments.
- Individual Examples: BBSDs was quickest for all tested perturbation percentages in the medium MNIST image-shift experiment.The experiment also estimated target accuracy using the most anomalous labeled instances returned by a domain classifier.
- Original Splits: The MNIST original split exhibited a statistically significant difference between training and test sixes, with a KS-test p-value of 2.7 · 10^-10.The difference was judged harmless by the malignancy detector despite showing that the split was not i.i.d.
6 Conclusions
The paper finds that soft-prediction black-box shift detection performs well across varied shifts, while domain classifiers help characterize shift type and malignancy. It also identifies future work in online data and non-image domains.
- Soft-prediction black-box shift detection works well across a wide variety of shifts, even when some assumptions do not hold.
- Aggregated univariate tests on latent dimensions offer comparable shift-detection performance to multivariate two-sample tests.
- Domain-discriminating classifiers enable qualitative characterization of shift type and assessment of its malignancy.
- Future work includes online shift detection and applying the framework to natural language processing or graphs.
A Detailed Shift Detection Results
The appendix documents complete shift-detection results for MNIST and CIFAR-10 under artificial shifts and original dataset splits.
- Complete shift-detection results evaluate different target shifts on MNIST and CIFAR-10.
- The experiments include artificially generated shifts.
- The procedure is also evaluated on original splits from MNIST, Fashion MNIST, CIFAR-10, and SVHN.
A.1.1 MNIST
The MNIST appendix presents shift-detection figures across adversarial, knock-out, Gaussian-noise, image, combined, and domain-adaptation shifts. It compares univariate tests with Bonferroni aggregation against multivariate two-sample tests.
- MNIST results include adversarial and knock-out shifts, including perturbation fractions and removal of class 0.
- Gaussian-noise and image shifts are evaluated at large, medium, and small magnitudes with univariate and multivariate tests.
- Additional MNIST figures examine combined shifts, top different or similar samples, and MNIST-to-USPS domain adaptation.
A.1.2 CIFAR-10
The CIFAR-10 appendix presents adversarial, knock-out, Gaussian-noise, image, and combined shifts. Across these settings, figures compare univariate tests with Bonferroni aggregation and multivariate two-sample tests.
- CIFAR-10 results include adversarial shifts and knock-out shifts that remove varying proportions of class 0.
- Gaussian-noise and image shifts are evaluated at large, medium, and small magnitudes using both testing strategies.
- Combined medium-image and knock-out shifts, and only-zero plus medium-image shifts, receive separate univariate and multivariate analyses.
A.2 Original Splits
This section presents figures comparing randomized and original splits for MNIST, Fashion MNIST, CIFAR-10, and SVHN. The comparisons use univariate and multivariate two-sample tests, with Bonferroni aggregation for the univariate analyses.
- The randomized datasets are described as having the same split proportions as the original datasets.
- The displayed method labels include NoRed, PCA, SRP, UAE, TAE, BBSDs, BBSDh, and Classif.
- Figures compare randomized and original splits for MNIST, Fashion MNIST, CIFAR-10, and SVHN.
- Univariate analyses use two-sample tests with Bonferroni aggregation across the four datasets.
- Multivariate analyses use two-sample tests for MNIST, Fashion MNIST, CIFAR-10, and SVHN.