PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Lifelong Learning with A-GEM

Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny

arXiv:1812.00420v2cs.LGstat.ML

TL;DR

The paper studies efficient lifelong learning when training proceeds in a single pass over task streams, with limited samples, computation, and memory. It proposes a disjoint-task evaluation protocol and A-GEM, finding that A-GEM offers the best accuracy–efficiency trade-off, while task descriptors can further improve few-shot learning.

  • Problem

    Existing lifelong-learning methods require evaluation of efficiency under realistic single-pass task streams with constraints on training samples, computation, and memory.

  • Method

    The paper introduces disjoint-task cross-validation with single-pass evaluation, proposes A-GEM as a faster GEM variant, and evaluates compositional task descriptors for transfer.

  • Results

    A-GEM has the best trade-off between average accuracy and computational/memory cost; it is about 100 times faster and uses 10 times less memory than GEM, while task descriptors improve few-shot performance.

  • Takeaways & Limitations

    A-GEM provides an efficient alternative to GEM for single-pass lifelong learning, and compositional task descriptors can further improve learning of new tasks.

  • Takeaways & Limitations

    A substantial performance gap remains between sequential lifelong-learning methods, including A-GEM, and non-sequential multi-task training, while current methods show poor knowledge transfer even when forgetting is eliminated.

Abstract

from arXiv · show

In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task. In this work, we investigate the efficiency of current lifelong approaches, in terms of sample complexity, computational and memory cost. Towards this end, we first introduce a new and a more realistic evaluation protocol, whereby learners observe each example only once and hyper-parameter selection is done on a small and disjoint set of tasks, which is not used for the actual learning experience and evaluation. Second, we introduce a new metric measuring how quickly a learner acquires a new skill. Third, we propose an improved version of GEM (Lopez-Paz & Ranzato, 2017), dubbed Averaged GEM (A-GEM), which enjoys the same or even better performance as GEM, while being almost as computationally and memory efficient as EWC (Kirkpatrick et al., 2016) and other regularization-based methods. Finally, we show that all algorithms including A-GEM can learn even more quickly if they are provided with task descriptors specifying the classification tasks under consideration. Our experiments on several standard lifelong learning benchmarks demonstrate that A-GEM has the best trade-off between accuracy and efficiency.

1 INTRODUCTION

Lifelong learning targets rapid adaptation from limited data by reusing experience, but existing protocols and algorithms often violate constraints on samples, memory, and computation. The paper proposes a stricter evaluation protocol, A-GEM, task descriptors, and a learning-speed metric.

  • Lifelong learning aims to acquire new skills quickly from small amounts of training data by reusing knowledge from related past tasks.The learner receives a stream of tasks whose relatedness is not known in advance.
  • Current lifelong-learning protocols often use multiple passes over data, conflicting with the goal of learning from one-pass streams and few examples.The stricter setting also limits memory and computation.
  • Single-pass algorithms can still impose substantial costs: some require heavy computation, while others use memory that scales with the number of tasks.These costs impede practical deployment.
  • The paper introduces disjoint-task cross-validation, A-GEM, compositional task descriptors, and a metric for measuring learning speed.A-GEM changes GEM’s loss function to reduce training time while maintaining similar performance.

2 LEARNING PROTOCOL

The proposed protocol separates hyper-parameter tuning from final learning and evaluation: models may replay cross-validation tasks, but observe evaluation-stream training examples exactly once. It represents each example with an input, task descriptor, and target.

  • The task streams are ordered dataset sequences, with T_CV < T; experiments use T_CV = 3 and T = 20.The datasets are assumed to be drawn from the same distribution over tasks.
  • The protocol uses separate task streams for cross-validation and final development, so hyper-parameters are selected on tasks disjoint from final learning and evaluation.The final stream contains entirely new tasks under the protocol.
  • Cross-validation may replay samples multiple times, whereas the final development stream exposes each training example once and reports metrics on its test sets.This separation addresses the sensitivity of regularization methods to hyper-parameter choices.
  • Each example consists of an input x, task descriptor t, and task-specific target y, and the predictor maps an input-descriptor pair to a target.The predictor is parameterized by θ.

3 METRICS

The paper evaluates lifelong-learning methods with average accuracy, forgetting, and Learning Curve Area. LCA complements final accuracy by measuring how quickly performance improves from few examples.

  • Average Accuracy measures performance after continual training through task k, with A_T giving average accuracy after the final task.A_T is the most commonly used lifelong-learning metric.
  • Forgetting Measure quantifies accuracy loss on past tasks after subsequent continual training.Forgetting also provides an indirect indication of how much knowledge remains available for transfer to new tasks.
  • Learning Curve Area measures the area under the convergence curve over b-shot performance for b from 0 to β.It is designed for models that learn from few examples.
  • LCA_0 equals average zero-shot performance, while larger LCA_β reflects stronger zero-shot performance and faster learning.Two models with equal eventual accuracy can have different LCA values if one learns faster.

4 AVERAGED GRADIENT EPISODIC MEMORY (A-GEM)

A-GEM reduces GEM’s per-step constraints to one average-loss constraint, preserving strong single-pass performance while substantially lowering computational and memory costs.

  • GEM prevents increases in individual previous-task losses by constraining the current gradient against each task’s episodic-memory gradient.
  • GEM solves a dual quadratic program with t − 1 variables, but repeatedly computing the memory-gradient matrix and solving this program becomes costly as tasks and memory grow.
  • When the current gradient violates the average-loss constraint, A-GEM projects it using an inner-product-based update rather than solving GEM’s quadratic program.
  • A-GEM replaces GEM’s t − 1 constraints with one constraint based on the average gradient from a random subset of episodic-memory examples.
  • A-GEM is memory efficient and orders of magnitude faster because it avoids storing G, computes fewer gradients, avoids quadratic programming, and incurs fewer violations as task count grows.
  • A-GEM offers better average-accuracy guarantees, whereas GEM provides stronger worst-case protection against forgetting on individual tasks.

5 JOINT EMBEDDING MODEL USING COMPOSITIONAL TASK DESCRIPTORS

The joint embedding model uses compositional task descriptors to transfer knowledge across tasks and accelerate learning of unseen class combinations.

  • Compositional task descriptors specify shared task components, such as object attributes, enabling transfer to new tasks with unseen combinations.
  • The model learns a joint embedding space between input features and attribute-based task descriptors.
  • The attribute embedding matrix acts as a lookup table, combining attribute vectors into class embeddings and concatenating them into task-descriptor embeddings.
  • The architecture and loss functions are general and can be applied to A-GEM and other lifelong-learning models.

6 EXPERIMENTS

Experiments evaluate lifelong-learning methods across four task streams using fixed architectures and a cross-validation protocol followed by a single pass over evaluation tasks. A-GEM generally offers the strongest accuracy–efficiency balance, while task descriptors further accelerate learning.

  • Experimental setup: Experiments use Permuted MNIST, Split CIFAR, Split CUB, and Split AWA with shared architectures and mini-batches of 10.Permuted MNIST uses a two-hidden-layer fully connected network; the other streams use ResNet18 variants.
  • Experimental setup: The protocol cross-validates on the first 3 tasks and evaluates on the remaining 17 after one training pass through each task.The cross-validation and evaluation task streams are disjoint.
  • Overall results: PROG-NN runs out of memory on Split CUB and Split AWA, whereas A-GEM and GEM show the lowest forgetting among fixed-capacity methods.PROG-NN’s parameter count grows super-linearly with the number of tasks.
  • Overall results: A-GEM achieves the best average accuracy on all datasets except Permuted MNIST, where PROG-NN performs better.PROG-NN benefits from many training examples per task on Permuted MNIST.
  • Overall results: A-GEM and GEM have comparable average accuracy, while A-GEM is about 100 times faster and uses about 10 times less memory.A-GEM’s time and memory costs are comparable to regularization-based methods such as EWC.
  • Learning speed and task descriptors: A-GEM is always top-performing or tied for best in learning-curve performance, and task descriptors greatly speed learning in the few-shot regime.Joint-embedding models show a clear zero-shot advantage on Split CUB and Split AWA, especially with A-GEM.

7 RELATED WORK

Related work includes modular compositional models that reduce task interference by selecting different modules, as well as task descriptors used to support lifelong learning and reinforcement learning. These approaches face architectural-search or computational limitations, motivating efficient deep-network applications.

  • Modular compositional models: Modular compositional lifelong-learning models limit task interference by using different subsets of modules for each task.Such methods require searching over the space of architectures.
  • Task descriptors: Prior work advocated task descriptors for lifelong learning, but sparse coding was not obviously computationally efficient for deep networks and compositional descriptors were not explored.Task descriptors have also been used in reinforcement learning for related motivations.

8 CONCLUSION

A-GEM offers a strong accuracy–efficiency trade-off for single-pass lifelong learning, while task descriptors can further improve few-shot performance. However, sequential lifelong learning remains substantially behind non-sequential multi-task learning despite using the same samples.

  • A-GEM provides the best trade-off between end-of-learning average accuracy and computational and memory cost.
  • A-GEM is about 100 times faster and requires 10 times less memory than GEM, while achieving higher average accuracy than regularization-based approaches.
  • Task descriptors improve few-shot performance across methods, with A-GEM often performing best.
  • Sequential lifelong-learning methods, including A-GEM, retain a substantial performance gap from the same network trained non-sequentially in a multi-task setting despite seeing identical samples.

A DATASET STATISTICS

The appendix introduces dataset-statistics material alongside implementation and proof details for A-GEM. The supplied passages primarily document the training procedure, evaluation loop, episodic-memory sampling, and update-rule derivation.

  • A-GEM training proceeds over sequential datasets, processing each task’s training examples and evaluating the model on corresponding test data.
  • At each training step, A-GEM samples a reference example from episodic memory, computes current and reference gradients, and uses the reference gradient to determine the update.
  • The algorithm stores episodic-memory examples after training each task and returns the trained model together with evaluation results.
  • The appendix derives the A-GEM update rule by formulating a constrained optimization problem, constructing its dual, and recovering the update from the optimal multiplier.

D ANALYSIS OF GEM AND A-GEM

The analysis compares GEM, A-GEM, and S-GEM in terms of constraint violations, forgetting, accuracy, and computational behavior. A-GEM reduces violations and favors average accuracy, while GEM better controls worst-case task forgetting and S-GEM performs slightly worse than GEM.

  • D.1 FREQUENCY OF CONSTRAINT VIOLATIONS: As task count increases, GEM violates constraints at nearly every training step, whereas A-GEM plateaus at a much lower violation frequency.The comparison covers Permuted MNIST and Split CIFAR, with 5500 and 250 training steps per task, respectively.
  • D.1 FREQUENCY OF CONSTRAINT VIOLATIONS: A-GEM’s efficiency comes from replacing GEM’s quadratic-program solve with an inner product and from incurring fewer constraint violations as tasks accumulate.
  • D.2 AVERAGE ACCURACY AND WORST-CASE FORGETTING: GEM achieves lower worst-case task forgetting, whereas A-GEM achieves better overall average accuracy, particularly on episodic-memory training examples.Test-set comparisons are confounded by generalization error.
  • D.3 STOCHASTIC GEM (S-GEM): S-GEM randomly samples one previous-task constraint at each step and projects the gradient only when that sampled constraint is violated.
  • D.3 STOCHASTIC GEM (S-GEM): S-GEM performs slightly worse than GEM despite being closer to GEM’s constraint-sampling formulation than A-GEM’s averaged-gradient constraint.

E RESULT TABLES

The result tables provide detailed benchmark comparisons and complexity information underlying the paper’s reported figures. They cover baseline performance, dataset-specific accuracy and forgetting, and computational and memory costs.

  • Detailed results in Tables 4–7 support the main-paper results shown in Figures 1 and 2.
  • Table 4 compares different baselines on Permuted MNIST and Split CIFAR using results averaged across five runs.A metric receives ∞ when the model fails to train with cross-validated hyper-parameters.
  • Tables 5 and 6 report average accuracy and forgetting for standard and joint-embedding models on Split CUB and Split AWA.The numbers are averaged across ten runs, and ‘OoM’ denotes failure to fit in memory.
  • Table 7 compares computational cost and memory complexity across lifelong-learning approaches, including timing, network state, and episodic-memory size.

F ANALYSIS OF EWC

The analysis examines whether EWC's performance depends on architecture size and repeated passes over task data. Across Permuted MNIST and Split CIFAR, increasing epochs substantially improves average accuracy only for EWC with bigger networks.

  • The experiments compare smaller and bigger architectures on Permuted MNIST and Split CIFAR.Permuted MNIST uses 256-unit versus 2000-unit fully connected networks; CIFAR uses reduced-width versus standard ResNet-18 models.
  • The evaluation reports Average Accuracy and Forgetting as the number of training epochs increases.Each epoch presents a task's training samples once before examples from the next task appear.
  • In the single-epoch setting, EWC performs similarly to the baseline VAN on Split CIFAR.The comparison is notable because VAN uses fewer training examples per task.
  • Hyper-parameters are selected using three cross-validation tasks to preserve the requirement that evaluation-task data is not seen more than once.The best setting for each experiment is reported in parentheses.
Loading 1812.00420v2…