PaperlayerView exact revision on arXiv

Source-linked AI summary

Recasting Gradient-Based Meta-Learning as Hierarchical Bayes

Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, Thomas Griffiths

arXiv:1801.08930v1cs.LG

TL;DR

The paper addresses the lack of a probabilistic interpretation for MAML's empirically motivated inductive bias and the tractability challenges of hierarchical Bayes for expressive models. It recasts MAML as empirical Bayes, then proposes a Laplace and K-FAC-based extension that improves few-shot benchmark performance. The analysis also identifies a scope boundary in its point-estimate treatment of predictive distributions.

  • Problem

    MAML's learned initialization was empirically motivated, while hierarchical-Bayes inference can be intractable for expressive models such as deep neural networks.

  • Method

    The paper interprets MAML as empirical Bayes in a hierarchical Bayesian model and extends it with a Laplace posterior approximation estimated using K-FAC curvature.

  • Results

    The analysis shows that MAML estimates hierarchical-Bayes prior parameters, and the proposed extension improves performance on a few-shot learning benchmark.

  • Takeaways & Limitations

    The hierarchical-Bayes interpretation provides statistical motivation for MAML and opens the door to improvements based on probabilistic inference machinery.

  • Takeaways & Limitations

    The analysis assumes that the predictive distribution over new data points is narrow and well approximated by a point estimate.

Abstract

from arXiv · show

Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm's operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.

1 INTRODUCTION

Meta-learning extracts information from prior tasks to improve efficient adaptation to novel tasks with limited experience. The paper gives MAML a hierarchical Bayesian interpretation and uses it to motivate an extension that improves few-shot benchmark performance.

  • Meta-learning extracts domain-general information that serves as an inductive bias for improving learning efficiency on novel tasks.
  • MAML learns a shared initialization so gradient descent can provide a favorable inductive bias for fast adaptation to new tasks.Prior work evaluated this inductive bias only empirically.
  • The paper derives MAML as inference for the parameters of a prior distribution in a hierarchical Bayesian model.The learned prior supports quick adaptation through an implicit predictive density over task-specific parameters.
  • The hierarchical-Bayes reinterpretation provides a principled statistical motivation for MAML and motivates improvements based on Bayesian posterior estimation.
  • The proposed MAML improvement enables better performance on a few-shot learning benchmark.

2 META-LEARNING FORMULATION

Meta-learning uses shared knowledge across related tasks to enable fast adaptation, while MAML learns a shared initialization optimized for generalization after task-specific gradient updates. The section then formulates this process as hierarchical Bayesian inference, where task parameters depend on shared meta-level parameters estimated by empirical Bayes.

  • 2 META-LEARNING FORMULATION: A meta-learner extracts task-general knowledge from related tasks to support rapid adaptation when novel-task data or computation is limited.
  • 2 META-LEARNING FORMULATION: The formulation considers related supervised-learning tasks whose shared structure allows solving one task to aid solving another.
  • 2 META-LEARNING FORMULATION: The paper presents gradient-based hyperparameter optimization and hierarchical Bayesian inference as two formulations of meta-learning, then connects them.
  • 2.1 META-LEARNING AS GRADIENT-BASED HYPERPARAMETER OPTIMIZATION: MAML uses the same parameter space for meta-learning and fast adaptation while adding only the meta-learning rate as an additional parameter.
  • 2.1 META-LEARNING AS GRADIENT-BASED HYPERPARAMETER OPTIMIZATION: MAML learns shared parameters θ such that one or a few gradient steps on small task samples yield good generalization on additional data from the same task.
  • 2.2 META-LEARNING AS HIERARCHICAL BAYESIAN INFERENCE: In the hierarchical model, task-specific parameters are conditionally dependent through a shared meta-level parameter θ, which constrains their estimation.
  • 2.2 META-LEARNING AS HIERARCHICAL BAYESIAN INFERENCE: Estimating θ by integrating out task-specific parameters forms the marginal likelihood of multi-task data.
  • 2.2 META-LEARNING AS HIERARCHICAL BAYESIAN INFERENCE: Maximizing the marginal likelihood with respect to θ gives an empirical-Bayes point estimate of the prior parameters.

3 LINKING GRADIENT-BASED META-LEARNING & HIERARCHICAL BAYES

The paper connects MAML to empirical Bayes in a hierarchical model, interpreting fast adaptation as posterior inference under an implicit task-specific prior. This connection also shows how gradient-descent choices determine that prior.

  • MAML can be understood as empirical Bayes that estimates a shared prior parameter while using task-specific point estimates from gradient descent.
  • The MAML objective is recovered by setting each task-specific point estimate to one gradient step from the shared initialization θ.
  • In linear regression with squared error, k gradient-descent iterations exactly compute the MAP estimate under a Gaussian prior centered at θ.
  • In nonlinear models, truncated gradient descent yields a point estimate corresponding to a mode of an implicit posterior shaped by regularization and early stopping.
  • Preconditioning gradient descent with a meta-learned curvature matrix can encode parameter correlations in the covariance of the task-specific prior.

4 IMPROVING MODEL-AGNOSTIC META-LEARNING

The paper improves MAML by replacing task-specific point estimates with a Laplace approximation that incorporates posterior uncertainty. K-FAC makes the required curvature and determinant computations tractable for neural networks, yielding the LLAMA method.

  • 4.1 LAPLACE’S METHOD OF INTEGRATION: Laplace approximation incorporates task-specific parameter uncertainty by replacing a point estimate of an integral with the volume of a local Gaussian around a mode.
  • 4.1 LAPLACE’S METHOD OF INTEGRATION: The resulting objective combines implicit early-stopping regularization with a log-determinant term that penalizes model complexity.
  • 4.2 USING CURVATURE INFORMATION TO IMPROVE MAML: Exact Hessians are difficult to form and may fail to be positive definite, so the method uses an approximate curvature matrix instead.
  • 4.2 USING CURVATURE INFORMATION TO IMPROVE MAML: K-FAC approximates neural-network curvature with block-diagonal Kronecker factors, enabling determinant computation in O(d3) time for a d-dimensional factor.
  • 4.2 USING CURVATURE INFORMATION TO IMPROVE MAML: The authors call the Laplace- and K-FAC-based replacement for point-estimate marginal NLL computation Lightweight Laplace Approximation for Meta-Adaptation (LLAMA).
  • 4.2 USING CURVATURE INFORMATION TO IMPROVE MAML: On sinusoid tasks, posterior samples remain sinusoidal and show greater uncertainty when the observed datapoints are less informative.

5 EXPERIMENTAL EVALUATION

Experiments evaluate probabilistic sampling and the LLAMA extension on sinusoid tasks and miniImageNet, showing uncertainty-aware adapted models and practical large-scale training.

  • 5.1 WARMUP: TOY NONLINEAR MODEL: After observing 10 datapoints from an unseen sinusoid, LLAMA samples task-specific models whose uncertainty increases when the datapoints are ambiguous.The sinusoid tasks vary amplitude and phase, with training inputs sampled uniformly and evaluated by mean squared error.
  • 5.2 LARGE-SCALE EXPERIMENT: miniIMAGENET: LLAMA evaluates one-shot, five-way classification on miniImageNet, which contains 64 training, 12 validation, and 24 test classes.The benchmark presents one instance of each of five unseen classes and evaluates classification of new instances from those classes.
  • 5.2 LARGE-SCALE EXPERIMENT: miniIMAGENET: LLAMA trains for 60,000 iterations on one TITAN Xp GPU in 9 hours, compared with 5 hours for MAML.The implementation uses a standard four-layer convolutional architecture for few-shot classification.
  • 5.2 LARGE-SCALE EXPERIMENT: miniIMAGENET: LLAMA achieves comparable performance to Triantafillou et al. (2017) on miniImageNet, while the Laplace approximation improves over MAML by approximating task-parameter marginalization more accurately.The reported gap between MAML and LLAMA is small, and the authors identify more accurate marginalization as a possible route to further improvement.

6 RELATED WORK

The paper situates gradient-based meta-learning within a broad few-shot and hierarchical-Bayesian literature, then identifies its contribution as deriving MAML as hierarchical Bayesian inference.

  • 6 RELATED WORK: Few-shot and meta-learning research includes hierarchical Bayesian, transfer-learning, neural-architecture, learned-update-rule, and memory-augmentation approaches.The cited literature spans both classical Bayesian formulations and modern neural methods for learning from few examples.
  • 6 RELATED WORK: Reported miniImageNet improvements from other works use architectures with significantly more parameters than the methods compared in Table 1.Those architecture variations are omitted because this work does not explore neural-network architecture changes.
  • 6 RELATED WORK: Unlike prior hierarchical-Bayes formulations that do not automatically provide tractable inference for expressive models, this work formulates a gradient-based meta-learner as hierarchical Bayesian inference.The formulation is intended to retain model-agnostic posterior inference through gradient-based procedures.
  • 6 RELATED WORK: The paper does not merely formulate meta-learning as a hierarchical Bayesian model; it connects MAML directly to that model and provides an efficient inference perspective.This distinction is the stated novelty relative to hierarchical-Bayes formulations of meta-learning.

7 CONCLUSION

The conclusion recasts MAML as hierarchical Bayesian inference and proposes Laplace- and K-FAC-based improvements, while noting predictive and approximation limitations.

  • 7 CONCLUSION: MAML estimates the parameters of a prior in a hierarchical Bayesian model, opening opportunities for improvements inspired by probabilistic machinery.The conclusion characterizes MAML as an empirical-Bayes procedure using point estimates for task-specific parameters.
  • 7 CONCLUSION: The proposed extension uses a Laplace approximation for task-specific posterior distributions instead of MAML’s point-estimate approximation of the integral.K-FAC estimates the curvature quantity required by the Laplace approximation for neural-network objectives.
  • 7 CONCLUSION: The analysis assumes the predictive distribution over new datapoints is narrow and well approximated by a point estimate.The authors suggest using task-parameter variance in future methods to model predictive densities for novel-task examples.
  • 7 CONCLUSION: Laplace approximation is inaccurate for highly skewed or non-unimodal integrals that cannot be represented by a single Gaussian mode.The conclusion proposes finite Gaussian mixtures as one possible future improvement.
Loading 1801.08930v1…