PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory Efficient Experience Replay for Streaming Learning

Tyler L. Hayes, Nathan D. Cahill, Christopher Kanan

arXiv:1809.05922v2cs.LGcs.AIstat.ML

TL;DR

Streaming learning requires DNNs to learn online in one pass from potentially non-iid data, challenging conventional training and causing catastrophic forgetting. The paper evaluates rehearsal, introduces ExStream for memory-efficient rehearsal, and finds that full rehearsal can stop forgetting while ExStream performs well with less memory and computation.

  • Problem

    Conventional DNNs require multiple passes through fixed datasets and can suffer catastrophic forgetting, while full rehearsal's effectiveness in modern DNNs had not been rigorously evaluated.

  • Method

    The paper rigorously evaluates rehearsal and compares six memory-efficient methods, introducing ExStream as a streaming-learning framework for memory-efficient rehearsal.

  • Results

    Full rehearsal stops catastrophic forgetting, while ExStream performs well across experiments using less memory and computation.

  • Takeaways & Limitations

    Memory-efficient rehearsal can support streaming classification with DNNs across iid and non-iid scenarios.

  • Takeaways & Limitations

    The study assumes a fixed buffer size per class, so adding more classes increases memory requirements.

Abstract

from arXiv · show

In supervised machine learning, an agent is typically trained once and then deployed. While this works well for static settings, robots often operate in changing environments and must quickly learn new things from data streams. In this paradigm, known as streaming learning, a learner is trained online, in a single pass, from a data stream that cannot be assumed to be independent and identically distributed (iid). Streaming learning will cause conventional deep neural networks (DNNs) to fail for two reasons: 1) they need multiple passes through the entire dataset; and 2) non-iid data will cause catastrophic forgetting. An old fix to both of these issues is rehearsal. To learn a new example, rehearsal mixes it with previous examples, and then this mixture is used to update the DNN. Full rehearsal is slow and memory intensive because it stores all previously observed examples, and its effectiveness for preventing catastrophic forgetting has not been studied in modern DNNs. Here, we describe the ExStream algorithm for memory efficient rehearsal and compare it to alternatives. We find that full rehearsal can eliminate catastrophic forgetting in a variety of streaming learning settings, with ExStream performing well using far less memory and computation.

I. INTRODUCTION

Streaming classification trains DNNs online from single-pass, potentially non-iid data, but conventional models learn slowly and may catastrophically forget. The paper studies rehearsal and introduces memory-efficient methods, including ExStream.

  • Conventional DNNs are ill-suited to streaming classification because they require multiple passes through fixed datasets and suffer catastrophic forgetting during updates.
  • Streaming classification lets learners use new knowledge immediately, see each labeled instance once, process non-iid streams, and limit memory usage.
  • Rehearsal mitigates catastrophic forgetting by mixing past experience with new data before fine-tuning the network.
  • The paper rigorously examines rehearsal using benchmarks designed to induce catastrophic forgetting in DNNs.
  • The authors propose new streaming-classification metrics and study six memory-efficient rehearsal methods, including streaming clustering.
  • ExStream is introduced as a streaming-learning framework for memory-efficient rehearsal.

II. RELATED WORK

Catastrophic forgetting arises when DNNs are incrementally trained on non-iid data, motivating methods that protect prior knowledge. Streaming learning imposes stricter constraints than incremental batch learning, including one-example batches and a single pass.

  • A. Catastrophic Forgetting: Catastrophic forgetting is the dramatic loss of previously learned knowledge during incremental DNN training with non-iid data.
  • A. Catastrophic Forgetting: The stability-plasticity dilemma requires networks to balance acquiring new knowledge against preserving previously learned information.
  • A. Catastrophic Forgetting: Regularization, ensembling, sparse coding, and dual-memory models are four major alternatives for mitigating catastrophic forgetting in DNNs.
  • A. Catastrophic Forgetting: Prior incremental-batch-learning work found a large gap between these methods and an offline baseline, with rehearsal-based GeppNet performing best.
  • B. Incremental Batch Learning: Incremental batch learning organizes labeled data into T possibly non-iid batches and restricts learning at time t to batch B_t.
  • B. Incremental Batch Learning: Streaming learning further requires batch size N_t = 1, a single pass through the labeled dataset, and evaluation at any time during training.

C. Streaming Learning

Streaming classification applies single-pass learning to temporally ordered data, creating challenges for DNNs that must learn quickly while limiting memory. The paper uses rehearsal buffers, including stream clustering and replacement methods, to support online DNN updates under these constraints.

  • Streaming classification requires learning quickly from individual instances rather than using incremental batches.It imposes a batch size of one, a single pass through labeled data, and possible evaluation during training.
  • Full rehearsal mixes all older examples with new examples but is inefficient in storage and computation for long-term resource-constrained robots.The paper motivates compressing redundant training examples instead of retaining every prior example.
  • Memory-limited rehearsal maintains K class-specific buffers containing at most b prototypes, which are used to fine-tune a DNN.Buffers are updated in a streaming fashion as labeled examples arrive.
  • Because stream order affects stored prototypes and classification results, experiments evaluate models under different ordering scenarios.
  • The study compares six fixed-capacity buffers: ExStream, Online k-means, CluStream, HPStream, reservoir sampling, and FIFO.The experiments vary the buffer capacity b across these methods.

A. Stream Clustering Buffers

The paper introduces ExStream as a partitioning-based online stream-clustering method for maintaining memory-efficient rehearsal buffers. It contrasts ExStream with an online k-means heuristic that updates the nearest stored exemplar.

  • 1) ExStream:: ExStream is a partitioning-based method for stream clustering designed for memory-efficient rehearsal.
  • 1) ExStream:: When a class buffer is full, ExStream merges its two closest clusters using Euclidean distance before inserting the new example.
  • 1) ExStream:: ExStream stores counts for clusters, updating the retained cluster count when two clusters are merged.The new point replaces one cluster location while the combined count is preserved in the other.
  • 2) Online k-means:: Online k-means finds the closest exemplar to a new example using Euclidean distance and updates that exemplar after the buffer is full.It also stores a counter for each exemplar or cluster.

3) CluStream:

The evaluated buffer alternatives use online clustering or prototype replacement to maintain rehearsal data under fixed memory. CluStream and HPStream cluster incoming points, while reservoir sampling and FIFO replace stored examples.

  • 3) CluStream:: CluStream assigns each incoming point to the closest micro-cluster when it lies within that cluster’s boundary.
  • 3) CluStream:: When a point falls outside the boundary, CluStream creates a new cluster and removes the least recently updated cluster or merges the two closest clusters.
  • 3) CluStream:: CluStream is not designed to handle high-dimensional data streams.
  • 4) HPStream:: HPStream uses projected micro-clusters with bit vectors indicating relevant dimensions for each cluster.
  • Replacement baselines store new inputs by replacing prototypes rather than compressing them with clustering.
  • Reservoir sampling replaces an existing class-specific sample with probability b/M after the buffer reaches capacity, whereas FIFO replaces the oldest example.

C. Baselines

The evaluation compares memory-limited streaming methods with single-pass, unbounded-rehearsal, and offline-DNN baselines across several stream-ordering paradigms. Datasets include streaming-specific object-recognition data and a 200-class fine-grained dataset.

  • The study compares memory-limited streaming methods against three baselines.
  • The single-pass baseline trains the DNN sample by sample with one pass through the labeled dataset.
  • The unbounded-rehearsal baseline stores all training examples as they arrive and uses them to fine-tune the DNN.
  • The offline baseline trains a conventional DNN from scratch on all training data and serves as an approximate upper bound.
  • Experiments use iid shuffled, class-organized, object-instance-ordered non-iid, and class-by-object-instance-ordered non-iid streams.All methods learn sample by sample with one pass, and the model is evaluated on all test data after each learning step.

A. Performance Metrics

Streaming methods are evaluated against an optimized offline baseline across testing events and buffer sizes, using normalized metrics that support cross-dataset comparisons.

  • Performance metric: Ωb measures streaming-classifier performance relative to an optimized offline baseline across T testing events.Ωb = 1 means parity with the offline model; values above 1 indicate better streaming performance.
  • Performance metric: The buffer-size aggregate µtotal averages performance across all tested buffer sizes.µtotal = 1 means the model matched the offline model at every tested buffer size.
  • Datasets: Experiments use iCub World 1.0 and CORe50 for temporally dependent streaming videos, plus CUB-200-2011 to test scaling across 200 classes.The datasets cover near-real-time, non-iid video streams and fine-grained recognition with few examples per class.
  • Implementation: Input features are 2048-dimensional unit-normalized embeddings from a ResNet-50 CNN pretrained on ImageNet-1K.These embeddings come from the network’s final mean-pooling layer.
  • Datasets: iCub World 1.0 contains 10 object categories with roughly 200 images per instance, forming temporally ordered image sequences.The experiments use 600–602 training images and 200–201 testing images per class.

2) CORe50:

The experiments include CORe50, a realistic 10-class video dataset, and CUB-200-2011, which tests scaling to many fine-grained categories and compares methods over buffer sizes.

  • CORe50: CORe50 contains 10 classes, five object instances per class, and 11 recording sessions with varied environments.Videos are sampled at 1 fps from roughly 15-second clips recorded at 20 fps.
  • CORe50: CORe50 provides 591–600 training images and 221–225 testing images per class under the suggested train/test split.
  • CUB-200-2011: CUB-200-2011 contains 200 bird species with roughly 30 training images per class and tests scaling to larger category counts.
  • Evaluation: The evaluation summarizes µtotal for each model over all tested buffer sizes, while clustering methods may require different memory per cluster.CluStream and HPStream require twice the memory of other prototype methods to maintain the same number of clusters.

A. Streaming iid Data

Across increasingly structured streaming scenarios, ExStream is consistently among the strongest methods and performs best overall in several settings, especially on CUB-200 and with small buffers.

  • Experimental scenarios: The experiments evaluate randomly shuffled streams, class-ordered streams, temporally ordered object instances, and combined class-and-instance ordering.The latter settings are designed to expose catastrophic forgetting in conventional DNNs.
  • Streaming iid Data: Queue replacement performs best on the two streaming datasets, while ExStream performs best overall and on the harder CUB-200 dataset.Stream-clustering methods generally provide significant CUB-200 advantages, especially with small buffers, except CluStream.
  • Learning new classes: Online k-means is the best overall model for learning new classes over time, closely followed by ExStream and HPStream.ExStream performs best on CUB-200, works well on iCub1, and performs fairly well with small CORe50 buffers.
  • Temporally ordered instances: ExStream, Online k-means, and HPStream outperform the other models on iCub1 with small buffers in the temporally ordered instance setting.CluStream exceeds full rehearsal on CORe50 at buffer size 2^8, suggesting it may need larger buffers.
  • Combined ordering: ExStream performs best overall in the combined class-and-instance experiment, though Online k-means performs slightly better on CORe50.

VI. DISCUSSION

The study finds that rehearsal mitigates catastrophic forgetting in streaming learning, while stream-clustering buffers provide strong memory-efficient performance across data orderings. ExStream performs well overall under memory constraints, but its fixed per-class buffer limits scalability as classes increase.

  • Full rehearsal suffices to mitigate catastrophic forgetting in streaming learning paradigms designed to induce it with high-resolution image datasets.
  • ExStream performs best across experiments on average and works well across iid and non-iid scenarios.
  • ExStream requires half the memory of CluStream and HPStream and does not require hyper-parameter tuning.
  • Stream-clustering methods perform best for non-iid data and maintain more consistent performance across iid and non-iid orderings.
  • Stream clustering improves performance on CUB-200, especially with smaller buffers, reducing memory and computational time needed for rehearsal.
  • The fixed buffer-size-per-class assumption means adding classes increases memory requirements, motivating a shared fixed-capacity buffer for larger datasets.

APPENDIX

The appendix documents implementation settings for the rehearsal DNNs and comparison algorithms, including parameter choices and buffer-dependent batching.

  • The rehearsal DNNs use batch normalization, with batch size reduced to min(batch size,num samples in buffer) when the buffer has fewer samples than the nominal batch size.
  • CluStream uses horizon=1000 and maximal boundary=2, with k-means initialized from 2×(buffer size) data points.
  • HPStream uses decay rate=0.5, spread radius factor=2, and speed=200; all other algorithms were implemented in Python 3.6.
  • Optimal parameters for each offline DNN are reported in Table A1.
Loading 1809.05922v2…