PaperlayerView exact revision on arXiv

Source-linked AI summary

Imbalanced Continual Learning with Partitioning Reservoir Sampling

Chris Dongjoo Kim, Jinseo Jeong, Gunhee Kim

arXiv:2009.03632v1cs.LGcs.AIcs.CVstat.ML

TL;DR

Continual-learning research had not yet addressed multi-label classification, where long-tailed labels add to catastrophic forgetting. The paper creates COCOseq and NUS-WIDEseq and proposes PRS; PRS outperforms CRS, GSS, and ExStream on comparable C-F1, O-F1, and mAP metrics across both datasets.

  • Problem

    Continual learning had not yet been studied for multi-label classification, where catastrophic forgetting coincides with long-tailed input distributions.

    Source: §1 · pp. 1–2

  • Method

    The paper creates the COCOseq and NUS-WIDEseq benchmarks and proposes PRS, which allocates memory across classes between equal and frequency-proportional allocation.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    PRS outperforms CRS, GSS, and ExStream on comparable C-F1, O-F1, and mAP metrics across COCOseq and NUS-WIDEseq.

    Source: §6.2 · p. 12

  • Takeaways & Limitations

    The work provides datasets and a memory-maintenance algorithm for studying multi-label continual learning under imbalanced data streams.

    Source: §1 · p. 2, §7 · p. 14

    The forgetting metric is self-relative, so comparisons between different methods can be misleading.

    Source: §6.1 · p. 10

Abstract

from arXiv · show

Continual learning from a sequential stream of data is a crucial challenge for machine learning research. Most studies have been conducted on this topic under the single-label classification setting along with an assumption of balanced label distribution. This work expands this research horizon towards multi-label classification. In doing so, we identify unanticipated adversity innately existent in many multi-label datasets, the long-tailed distribution. We jointly address the two independently solved problems, Catastropic Forgetting and the long-tailed label distribution by first empirically showing a new challenge of destructive forgetting of the minority concepts on the tail. Then, we curate two benchmark datasets, COCOseq and NUS-WIDEseq, that allow the study of both intra- and inter-task imbalances. Lastly, we propose a new sampling strategy for replay-based approach named Partitioning Reservoir Sampling (PRS), which allows the model to maintain a balanced knowledge of both head and tail classes. We publicly release the dataset and the code in our project page.

1 Introduction

The introduction frames multi-label continual learning as an unaddressed setting and identifies catastrophic forgetting alongside naturally long-tailed labels as joint obstacles. It contributes benchmark sequences and PRS.

1 Introduction

  • The work identifies catastrophic forgetting and long-tailed input distributions as simultaneous obstacles, including intra- and inter-task imbalance.The authors say large-scale multi-label datasets naturally exhibit long-tailed distributions.Source: §1 · p. 2
  • The authors adapt MSCOCO and NUS-WIDE into mutually exclusive task sequences called COCOseq and NUS-WIDEseq.Source: §1 · pp. 1–2
  • This work takes on continual learning for multi-label classification, a setting the introduction says prior continual-learning studies had not addressed.The authors also identify intra- and inter-task imbalance as challenges alongside catastrophic forgetting.Source: §1 · pp. 1–2
  • They propose Partitioning Reservoir Sampling (PRS), a replay method for heterogeneous, long-tailed data streams.Source: §1 · p. 2
  • The authors identify allocating sufficient memory to moderate and minority classes as key to retaining balanced knowledge of present and past experiences.Source: §1 · p. 2

2 Motivation: Fatal Forgetting on the Tail Classes

Experiments show that minority classes experience more forgetting than majority classes under both reservoir replay and EWC. The authors connect this pattern to memory imbalance in long-tailed data.

2 Motivation: Fatal Forgetting on the Tail Classes

  • The authors report that long-tailed distributions aggravate continual-learning forgetting, with destructive forgetting on tail classes.Source: §2 · p. 2
  • Both EWC [43] and reservoir replay [7,73,56] show more forgetting on minority classes than majority classes across tasks.Source: §2 · p. 3, Figure 1 · p. 3
  • In COCOseq, forgetting is measured as the difference between peak and end-of-sequence performance, with classes sorted by within-task size.Source: §2 · p. 3
  • The authors attribute the pattern to memory sample imbalance mirroring the input distribution.Source: §2 · p. 3

3 Approach

The approach targets catastrophic forgetting and long-tailed inputs in task-free multi-label continual learning with fixed-memory replay. PRS uses running class statistics to set target memory partitions, then admits and removes samples to move memory toward those targets.

3.1 Problem Formulation

  • The task-free stream supplies online multi-hot examples without revealing task boundaries or the number of classes.The model must allocate a fixed memory across classes observed so far.Source: §3.1 · pp. 3–4

3.2 Conventional Reservoir Sampling

  • Uniform conventional reservoir sampling makes memory long-tailed when the input stream is long-tailed, reproducing imbalanced-training problems.CRS assigns each datapoint sampling probability m/n.Source: §3.2 · p. 4

3.3 Fundamental Problems in Imbalanced Learning

  • Imbalance can let majority classes dominate minibatch gradients, while minority classes remain under-represented in learned features.The paper reports that minority features are sparsely distributed under conventional methods.Source: §3.3 · p. 4

3.4 Partitioning Reservoir Sampling

  • PRS partitions a fixed memory using running class frequencies and maintains those targets through sample-in and sample-out operations.The allocation power ρ interpolates between equal class allocation at ρ = 0 and frequency-proportional allocation at ρ = 1.Source: §3.4 · p. 5
  • PRS biases sample-in probability toward minority classes while seeking compliance with target partitions.Its normalized class weights are computed by applying softmax to negative running class frequencies.Source: §3.4 · p. 6
  • For sample-out, PRS selects an over-quota class, filters candidates to avoid removing under-quota or satisfied classes, then removes the candidate that best advances the target partition.The memory-to-target distance uses current class counts and target partition ratios; datapoints may carry multiple labels.Source: §3.4 · p. 6

4 Related Work

Prior work spans replay, regularization, and expansion approaches, while online replay methods differ in their information requirements and treatment of imbalanced task sequences. The paper places its work in online, task-free continual multi-label learning, unlike the cited offline multi-label methods.

Replay-based approaches

  • Replay methods retain fixed-size memories or generative weights for rehearsal; GEM constrains gradients, while AGEM makes that idea more efficient.Other replay work studies tiny episodic memories and combines rehearsal with meta-learning to balance transfer and interference.Source: §4 · p. 7

Online Sequential Learning

  • ExStream requires the number of classes in advance, while GSS's cited MNIST setup has one large task and smaller tasks, limiting its fit to practical long-tailed settings.The cited GSS experiments use 2,000 instances for one task and 200 for each other task.Source: §4 · pp. 7–8

Multi-label Classification

  • Earlier multi-label methods address offline classification, whereas this work studies online task-free continual learning and can be integrated with those methods.The authors select some prior multi-label methods as base models in their experiments.Source: §4 · p. 8

5 The Multi-label Sequential Datasets

The paper constructs COCOseq and NUS-WIDEseq by splitting multi-label datasets into sequential tasks with exclusive class labels. Both benchmarks support imbalance analysis, with COCOseq using balanced per-class test counts and NUS-WIDEseq exhibiting imbalance within and across tasks.

5 The Multi-label Sequential Datasets

  • The datasets convert multi-label classification data into continual versions split into tasks with exclusive class labels despite each datapoint having multiple labels.The paper names the resulting benchmarks COCOseq and NUS-WIDEseq.Source: §5 · p. 8

5.1 The COCOseq

  • COCOseq has four tasks, 70 object classes, 35,072 training examples, and 6,346 test examples, with identical test-set size per class despite long-tailed training counts.The test set contains 100 images per class; its total is not 70 × 100 because images can have multiple labels.Source: §5.1 · p. 9

5.2 The NUS-WIDEseq

  • NUS-WIDEseq has 49 classes, 48,724 training examples, and 2,367 test examples, with 50 test samples per class and imbalance within and across tasks.Its 6 mutually exclusive tasks are increasingly difficult; NUS-WIDE's top 20% of classes account for 76.3% of the data.Source: §5.2 · p. 9

6 Experiments

Experiments evaluate online, task-free multi-label continual learning on COCOseq and NUS-WIDEseq, comparing PRS with continual-learning baselines across overall performance and imbalance-focused analyses.

6.1 Experimental Design

  • For task-free online learning, the datasets contain four and six tasks, and task labels are unavailable during training and testing.The evaluation follows continuous input streams without a separate offline batch-training stage.Source: §6.1 · p. 10, §6 · p. 10

Baselines.

  • Experiments use ImageNet-pretrained ResNet101 and compare six continual-learning baselines plus offline Multitask and online Finetune, reporting O-F1, C-F1, mAP, and forgetting.Forgetting is self-relative, so comparisons between methods can be misleading.Source: §6.1 · p. 10, Baselines · p. 11

6.2 Results

  • PRS outperforms CRS, GSS, and ExStream on C-F1, O-F1, and mAP across COCOseq and NUS-WIDEseq.ExStream uses prior task information to pre-allocate memory.Source: §6.2 · p. 12
  • PRS remains comparatively robust across task-schedule permutations and outperforms the baselines at memory sizes of 1,000, 2,000, and 3,000.CRS, GSS, and ExStream fluctuate with schedule permutations.Source: §6.2 · p. 12, Table 2 · p. 12
  • PRS performs best on moderate and minority classes, is competitive on majority classes, and outperforms baselines across metrics for the smallest NUS-WIDEseq task.Task 3 is the smallest task in NUS-WIDEseq.Source: §6.2 · pp. 12–13, Figure 4 · p. 12
  • CRS allocates most memory to majority classes, whereas PRS balances memory across classes without auxiliary task information.GSS allocates more memory to moderate classes than CRS but still leaves minority classes insufficiently represented.Source: §6.2 · p. 13, Figure 5 · p. 13
  • Performance declines across metrics as ρ moves away from the vicinity of balance, even with a highly imbalanced input stream.Figure 6 reports averages over five random seeds.Source: §6.2 · p. 13, Figure 6 · p. 13
  • On MNIST and COCOseq, PRS represents minority classes more discriminatively than CRS in t-SNE feature projections.The projections use penultimate-layer features.Source: §6.2 · p. 13, Figure 7 · p. 14

7 Conclusion

The paper concludes that balanced replay memory matters for multi-label continual learning on imbalanced streams. It identifies automatic online tuning of target partitions as future work.

7 Conclusion

  • The paper contributes COCOseq and NUS-WIDEseq and proposes Partitioning Reservoir Sampling to maintain memory toward desired partition ratios.The conclusion frames multi-label continual learning as learning from imbalanced data streams.Source: §7 · p. 14
  • The results underscore the importance of maintaining well-rounded knowledge through balanced replay memory.Source: §7 · p. 14
  • Automatically tuning target partitions while learning online is identified as a future research direction.Source: §7 · p. 14

8 Appendix

The appendix collects supplementary material on related work, dataset curation, experiments, and results. It also reports PRS results on imbalanced single-label datasets.

8 Appendix

  • Supplementary material covers dataset curation, experiment details, and additional results and analyses, including single-label experiments and gradient-variance analysis.It also includes related work and details for imbalanced multi-label and single-label sequential datasets.Source: §8 · p. 19
  • PRS also performs superior to the other baselines in the appendix's imbalanced single-label experiments.Source: §8 · p. 19

8.1 Related Work

Continual Learning.

The related-work discussion reviews replay, regularization, and expansion approaches to continual learning alongside the long-tailed imbalance problem. It describes mechanisms used to preserve or extend learned knowledge across tasks.

Generative Replay Methods

  • Generative replay trains a task solver on pseudo-data sampled from a deep generative model; FearNet combines a small reservoir with a generative network.Other generative replay approaches are also noted.Source: Continual Learning · p. 19

Regularization

  • Regularization penalizes parameter changes or estimates weight importance to mitigate forgetting, while Learning without Forgetting uses knowledge distillation.The discussion notes that learned parameters can drift from earlier tasks, especially over long sequences.Source: Continual Learning · p. 20

Expansion

  • Expansion methods freeze parameters for earlier tasks and add capacity for new tasks, as in Progressive Networks [74] and Dynamically Expandable Networks.NDL adds neural units to an autoencoder.Source: Continual Learning · p. 20

Imbalance

  • Substantial imbalance can bias models toward majority concepts and, at extreme levels, lead them to ignore minority concepts [80].The discussion distinguishes data-based and algorithm-based approaches.Source: Imbalance · p. 20

Imbalance.

Imbalanced-learning approaches adjust data or learning algorithms, each with trade-offs; this work also documents how it curates sequential datasets. The section surveys sampling, cost-sensitive, hybrid, and newer learning approaches.

Imbalance.

  • Oversampling duplicates minority examples but can increase training time and overfitting, while undersampling discards majority-class information.Synthetic minority data generation is difficult because few samples are available to model minority concepts.Source: Imbalance · pp. 20–21
  • Algorithmic alternatives include heuristic cost-sensitive weighting, ensemble or episodic hybrids, and transfer, metric, few-shot, or meta-learning approaches.Cost-sensitive methods require a suitable cost matrix, which entails significant manual work.Source: Imbalance · p. 21

8.2 Dataset Curation

  • The supplemental curation details cover COCOseq and NUS-WIDEseq, as well as sequential single-label datasets derived from MNIST and SVHN.Source: §8.2 · p. 21

Multi-Label Sequential Dataset Curation Details.

The multi-label split procedure uses hierarchical clustering to form non-overlapping tasks while managing data loss and, optionally, task-size balance. Its merge score prioritizes attainable images and uses β to control inter-task balance.

Multi-Label Sequential Dataset Curation Details.

  • The clustering procedure merges classes bottom-up to minimize data loss from overlapping labels and optionally balance the sizes of the resulting tasks.It continues until the designated number of groups remains.Source: Multi-Label Sequential Dataset… · pp. 21–22
  • At each iteration, it counts images belonging only to each group, compares attainable image counts for candidate merges, and merges the pair with the highest score.The score favors merges with more images and penalizes groups that are already large.Source: Multi-Label Sequential Dataset… · pp. 22–23
  • COCOseq uses β = 1 to promote intra-task imbalance with inter-task balance, whereas NUS-WIDEseq uses β = 0.0001 and requires at least four classes per task.Source: Multi-Label Sequential Dataset… · p. 23

Single-Label Sequential Dataset Curation Details.

The supplemental experiments examine PRS on single-label streams and across architectures, evaluation settings, task schedules, and memory sizes. They also report task-by-task performance and memory-gradient variance.

Single-Label Sequential Dataset Curation Details.

  • The single-label benchmarks use Pareto-distributed MNIST and SVHN streams, while supplemental experiments also cover additional architectures and analyses.The normalized forgetting measure is adjusted because class-performance differences can make the original measure imprecise; its j term denotes forgetting on task j after training through task k, and m denotes the chosen metric.Source: Single-Label Sequential Dataset… · p. 23, §8.3 · p. 23, Base Architectures · p. 25

Base Architectures.

  • Plain ResNet performs best online after one epoch, and PRS outperforms the other baselines across the evaluated architectures.On balanced test sets, all three base architectures perform worse than on imbalanced test sets, whether trained for one or ten epochs.Source: Base Architectures · p. 23, Table 5 · p. 28

Single-Labeled Classification Results.

Imbalanced test-set.

  • With ρ = 0.5, PRS outperforms the other baselines on the imbalanced COCOseq test set while remaining competitive on majority classes.The test set includes 50.5% of MSCOCO test examples from majority classes.Source: Imbalanced test-set · p. 26, Table 6 · p. 28

Permuted NUSWIDEseq Schedules.

Different Memory Sizes on NUSWIDEseq.

Task Progression Results.

Gradient Variance.

  • PRS has relatively higher memory-gradient variance than other baselines on Imbal-MNISTseq and COCOseq, possibly reflecting class diversity maintained in memory.The authors describe higher variance as a possible side effect of preventing forgetting; the COCOseq ExStream comparison is complicated by its different training method and pre-featurized inputs.Source: Gradient Variance · p. 27, Figure 9 · p. 27, Figure 10 · p. 27
Loading 2009.03632v1…