Source-linked AI summary
A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns
Ashesh Chattopadhyay, Pedram Hassanzadeh, Saba Pasha
TL;DR
Climate data are complex spatio-temporal inputs, while CNNs require substantial labeled data and dataset-specific design. The paper proposes unsupervised K-means auto-labeling and evaluates CNN re-identification of four weather-pattern clusters, achieving over 90% test accuracy with at least 2000 training samples per cluster.
Problem
Climate and environmental data are complex, while supervised CNNs require large labeled datasets whose creation demands expert time and resources.
Method
The paper uses K-means clustering to auto-label spatio-temporal data, then trains CNNs to identify four clustered North American weather-pattern regimes and examines architecture, hyperparameters, and training-set size.
Results
Over 90% test accuracy is achieved for four-cluster classification with at least 2000 training samples per cluster.
Takeaways & Limitations
The auto-labeling strategy facilitates evaluating CNN capabilities and performance scaling for spatio-temporal climate and environmental data.
Takeaways & Limitations
K-means assigns each pattern to one cluster even when chaotic spatio-temporal patterns resemble multiple clusters, creating potentially inconsistent labels.
Abstract
from arXiv · showhide
Convolutional neural networks (CNNs) can potentially provide powerful tools for classifying and identifying patterns in climate and environmental data. However, because of the inherent complexities of such data, which are often spatio-temporal, chaotic, and non-stationary, the CNN algorithms must be designed/evaluated for each specific dataset and application. Yet to start, CNN, a supervised technique, requires a large labeled dataset. Labeling demands (human) expert time, which combined with the limited number of relevant examples in this area, can discourage using CNNs for new problems. To address these challenges, here we (1) Propose an effective auto-labeling strategy based on using an unsupervised clustering algorithm and evaluating the performance of CNNs in re-identifying these clusters; (2) Use this approach to label thousands of daily large-scale weather patterns over North America in the outputs of a fully-coupled climate model and show the capabilities of CNNs in re-identifying the 4 clustered regimes. The deep CNN trained with $1000$ samples or more per cluster has an accuracy of $90\%$ or better. Accuracy scales monotonically but nonlinearly with the size of the training set, e.g. reaching $94\%$ with $3000$ training samples per cluster. Effects of architecture and hyperparameters on the performance of CNNs are examined and discussed.
Introduction
CNNs could expand climate-pattern classification, but spatio-temporal complexity and the need for large expert-labeled datasets require application-specific design. This paper proposes unsupervised auto-labeling and tests CNN re-identification of four weather-pattern clusters.
- Prior work: CNNs have previously identified tropical cyclones, weather fronts, and atmospheric rivers in large labeled climate datasets.These studies motivate extending CNN applications to additional climate and environmental problems.
- Challenges: Climate and environmental data are spatio-temporal, nonlinear, chaotic, high-dimensional, non-stationary, multi-scale, and correlated.Observational datasets can also be short, sparse, and noisy.
- Challenges: CNN algorithms must be designed and evaluated for each specific climate dataset and application.Their supervised training requires large labeled datasets, making expert labeling a practical barrier.
- Contributions: The paper proposes K-means-based auto-labeling of spatio-temporal climate and environmental data.The approach uses an unsupervised classification technique to assign labels without manually labeling every pattern.
- Contributions: The study labels thousands of large-scale North American weather patterns from a state-of-the-art climate model and tests CNN identification of four clusters.It also examines how architecture, hyperparameters, and training-set size affect performance.
Methodology
The method clusters climate patterns, uses cluster indices as labels, and trains CNNs to re-identify them. Applied to four seasonal weather regimes, deeper CNNs achieve about 93% test accuracy, with higher accuracy requiring greater computational cost.
- Methodology: The approach first clusters spatio-temporal data into n classes, then uses cluster indices to label patterns for CNN training and testing.CNN test performance evaluates re-identification of the cluster indices for a specific dataset.
- Methodology: Seasonal North American patterns are classified separately into n = 4 clusters using K-means, with full Z500 fields labeled by the resulting indices.Clustering uses zonal-mean-removed Z500 anomalies projected onto 22 EOFs retaining approximately 95% of variance.
- Methodology: CNN inputs are full Z500 fields converted to images and down-sampled, while convolutional kernels are learned through backpropagation.Pooling reduces feature-map dimensions and helps control overfitting while inducing translation and scale invariance.
- Experimental design: Two CNN architectures, CNN2 and CNN4, are used to examine architecture, hyperparameters, practical issues, and training-set scaling.The comparison directly evaluates how network depth affects re-identification performance.
Results
CNN4 achieved robust test accuracy of about 93% on clustered summer and winter weather patterns, outperforming CNN2 while requiring greater computational resources. Accuracy increased monotonically but nonlinearly with training-set size, while imperfect clustering and labeling contributed to some errors.
- Architecture and accuracy: 93.3%±0.2% summer and 93.8%±0.1% winter test accuracy were achieved by CNN4, compared with 89.0%±0.3% and 86.6%±0.3% for CNN2.The deeper network improved accuracy by 4%–7% but required more time and memory.
- Overfitting: Training and test accuracies were approximately equal, and small test-set standard deviations indicated that the networks learned rather than overfit the classes.For N ≤12000, overfitting occurred when more than four convolutional layers were used.
- Training-set size: Test accuracy scaled monotonically but nonlinearly with training-set size, rising from around 64% at N = 500 to above 90% at N = 8000 for CNN4.Increasing N to 12000 slightly raised accuracy to 93%.
- Misclassification patterns: CNN4 still misidentified a few hundred of 4000 testing samples, and its overall improvement did not increase accuracy for every cluster.Misclassification patterns were asymmetric across clusters, and some cluster pairs showed no misclassifications.
- Training-set size: 94.1% was the best summer test accuracy at N = 18000, up from 93.3% at N = 12000, indicating only slow gains from additional samples.Further systematic hyperparameter optimization might still improve accuracy.
- Clustering and labeling: 11% of poorly clustered patterns with s < 0 were misclassified, compared with 4.8% of well-clustered patterns with s > 0.4.This suggests part of CNN4’s 6.7% summer testing error arose from poor clustering and improper labeling.
- Clustering and labeling: Soft clustering could assign patterns to multiple clusters, potentially addressing inaccuracies caused by borderline cases when poor labeling is a significant source of error.The labels must still form a learnable set for the CNN.
Discussion
The proposed auto-labeling strategy supports CNN evaluation on spatio-temporal climate data while exposing performance limits and training-size effects. The case study reports over 90% accuracy for four-cluster classification with sufficient training samples.
- The auto-labeling strategy facilitates exploring CNN capabilities in climate and environmental science.
- The method can be applied to spatio-temporal data and used to examine CNN architecture and training-dataset-size effects before further labeling investment.
- Over 90% test accuracy is achieved for four-cluster classification with at least 2000 training samples per cluster.
- The reported CNN performance on chaotic, multi-scale, non-stationary weather data requires minimal pre-processing.
Data from the Large Ensemble (LENS) Community Project
The study uses the publicly available LENS project, a 40-member fully coupled CESM1 ensemble, to analyze North American Z500 patterns across extended summer and winter periods.
- The dataset contains 40 fully coupled atmosphere-ocean-land-ice CESM1 simulations at approximately 1° horizontal resolution.
- Summer analysis uses June-August data from 1980-2005 across 40 ensemble members, totaling 95,680 days over North America.
Clustering of weather patterns
Daily North American Z500 patterns are clustered separately by season into four classes using EOF-reduced anomalies and repeatedly initialized K-means clustering.
- Daily Z500 patterns are clustered separately for each season into n = 4 classes.
- The clustering uses the first 22 principal components, which explain 95% of variance after zonal-mean removal and EOF analysis.
- K-means is repeated 1000 times with new initial centroids before assigning each daily pattern a cluster index.
- Four clusters are chosen as a challenging multi-class task with distinct centers and enough samples per cluster for training.
Labeling and up/down-samplings
The study labels full Z500 fields using cluster indices derived from anomaly patterns and preprocesses them through spatial up- and down-sampling. RGB channels encode the single scalar field.
- Cluster indices computed from anomaly patterns are used to label the corresponding full Z500 daily fields.
- Full Z500 fields provide a more challenging test because their cluster differences are subtler and retain temporal variability and non-stationarity.
- Down-sampling removes small-scale transient circulation features so the CNN focuses on large-scale cluster differences rather than mostly random details.
- The scalar Z500 field is represented through all three RGB channels because the study focuses on one variable.
Convolutional Neural Network (CNN)
The study compares shallow and deep CNN architectures for re-identifying weather-pattern clusters, using regularized training, validation, and independent testing. It also examines how training-set size and direct matrix inputs affect performance.
- Training: The networks use cross-entropy with L2 regularization, dropout, and ADAM-based adaptive learning-rate optimization.The final softmax output assigns each input pattern to the cluster with the highest probability.
- Architectures: CNN2 uses two convolutional layers, whereas CNN4 uses four layers with 8, 16, 32, and 64 filters.CNN4 differs from CNN2 mainly by adding two convolutional layers.
- Data partitioning: For N = 12000, training, validation, and testing sets are separated, with equal samples from all four clusters to prevent class imbalance.The training set contains 3000 images per cluster, while validation and testing use separate samples.
- Evaluation: Hyperparameters are tuned using validation sets before the best trained CNN is evaluated once on five independent testing datasets.Reported testing results are the mean and standard deviation across those five datasets.
- Training-set scaling: For N = 500 to 8000, training uses N/4 labeled images per cluster and testing uses approximately N/8 images per class.This setup evaluates how CNN performance changes with training-set size while preserving equal representation across clusters.
- Alternative input representation: CNNs can also operate directly on labeled Z500 data matrices, achieving the same accuracy as image-based CNNs with a somewhat different architecture.Each matrix column contains the full Z500 field over the 97×66 grid for one day.