Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory

Ron Amit; Ron Meir

Source-linked AI summary

Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory

Ron Amit, Ron Meir

arXiv:1711.01244v8Nov 3, 2017stat.MLcs.AIcs.LG

TL;DR

Learning new tasks can benefit from knowledge extracted from related observed tasks, but that knowledge must preserve flexibility for novel task aspects. The paper extends PAC-Bayes bounds into meta-learning by learning a distribution over priors and optimizing a bound-derived objective, then demonstrates improved performance and learned prior structure in neural networks.

Problem
Meta-learning needs to extract shared structure from observed tasks while retaining flexibility to adapt to novel aspects of new tasks.
Method
The paper extends PAC-Bayes bounds to meta-learning by learning a distribution over priors and minimizing the resulting objective with gradient-based neural-network procedures.
Results
The experiments demonstrate performance enhancement over naive approaches and recent methods, with transfer improving after observing at least three permuted-label tasks.
Takeaways & Limitations
The framework provides an efficient task-level procedure in which prior information can be manifested at different levels of deep neural networks.
Takeaways & Limitations
The experimental results are preliminary, and the current method learns all available tasks in parallel rather than sequentially.

Abstract

from arXiv · show

In meta-learning an agent extracts knowledge from observed tasks, aiming to facilitate learning of novel future tasks. Under the assumption that future tasks are 'related' to previous tasks, the accumulated knowledge should be learned in a way which captures the common structure across learned tasks, while allowing the learner sufficient flexibility to adapt to novel aspects of new tasks. We present a framework for meta-learning that is based on generalization error bounds, allowing us to extend various PAC-Bayes bounds to meta-learning. Learning takes place through the construction of a distribution over hypotheses based on the observed tasks, and its utilization for learning a new task. Thus, prior knowledge is incorporated through setting an experience-dependent prior for novel tasks. We develop a gradient-based algorithm which minimizes an objective function derived from the bounds and demonstrate its effectiveness numerically with deep neural networks. In addition to establishing the improved performance available through meta-learning, we demonstrate the intuitive way by which prior information is manifested at different levels of the network.

1. Introduction

Meta-learning extracts knowledge from observed tasks to facilitate learning related, previously unobserved tasks. This work frames that process through PAC-Bayes bounds, learned priors, and practical neural-network algorithms.

Prior information can be learned from previous interactions when the relevant task-specific prior is unclear.
Meta-learning extracts knowledge from several observed tasks to facilitate learning new tasks that are unavailable during meta-training.
A learned prior may fix generic lower-level image features while allowing higher layers to adapt to new classes.
The framework automatically infers priors from observed tasks rather than manually inserting them into the algorithm.
The paper extends PAC-Bayes generalization bounds to meta-learning and develops a practical method for probabilistic neural networks.
Most recent meta-learning methods discussed here were not based on generalization error bounds, while prior PAC-Bayes meta-learning lacked a practical deep-network algorithm.

2. Preliminaries: PAC-Bayes Learning

PAC-Bayes learning represents uncertainty with a posterior distribution over hypotheses and controls generalization through a data-independent prior. Its bound trades empirical fit against a KL-based complexity term.

Learning from finite samples requires selecting a hypothesis using prior assumptions because the data-generating distribution is unknown.
PAC-Bayes learning outputs a posterior distribution Q over the hypothesis space rather than selecting only one classifier.
The PAC-Bayes prior P is a reference distribution that must be specified before observing the dataset.
The generalization bound upper-bounds expected error by empirical error plus a complexity term involving KL divergence D(Q||P).
Because the bound holds uniformly over posteriors, minimizing it yields a learning algorithm with generalization guarantees.

3. PAC-Bayes Meta-Learning

The meta-learning formulation learns a distribution over task-level priors from observed tasks, then uses sampled priors to learn new tasks. A new PAC-Bayes transfer bound supports this procedure and accounts separately for task and environment complexity.

3. PAC-Bayes Meta-Learning: The meta-learner extracts common knowledge from observed tasks as a learned prior for new tasks from the same task environment.
3. PAC-Bayes Meta-Learning: A base learner combines a new task’s data with a sampled prior to produce a posterior distribution over hypotheses.
3. PAC-Bayes Meta-Learning: The transfer error measures expected loss on new environment tasks when priors are drawn from the hyper-posterior.
3. PAC-Bayes Meta-Learning: Meta-learning replaces a single-task hyperparameter prior with a hyper-prior over priors and learns a hyper-posterior after observing all training tasks.
3.2. Meta-Learning PAC-Bayes Bound: The new bound controls transfer error using empirical multi-task error plus task-complexity and environment-complexity terms.
3.2. Meta-Learning PAC-Bayes Bound: Task complexity vanishes with more samples per task, while environment complexity vanishes as more tasks are observed.
3.2. Meta-Learning PAC-Bayes Bound: The proof uses separate single-task bounds and a union bound, accounting for each observed task’s sample count instead of their harmonic mean.

4. Meta-Learning Algorithm

The algorithm turns a PAC-Bayes meta-learning bound into a differentiable objective, jointly learning task posteriors and a shared prior through stochastic optimization. Gaussian distributions, Monte Carlo estimation, reparameterization, and SGD make the procedure applicable to deep neural networks.

Objective: The bound can be evaluated on training data and minimized as a principled objective for differentiable meta-learning models.This objective is intended to approximately minimize transfer risk when the bound is sufficiently tight.
Distribution model: The hyper-posterior is modeled with isotropic Gaussian distributions, while the hyper-prior contributes both divergence regularization and expectations in the bound.A zero-mean isotropic Gaussian hyper-prior yields an L2-norm regularization term favoring small-norm solutions.
Joint optimization: Joint optimization couples each task posterior to its dataset and shared hyper-posterior, while coupling the hyper-posterior to task posteriors and the hyper-prior.The dataset enters through empirical error, task complexity connects task posteriors to the hyper-posterior, and environment complexity connects the hyper-posterior to the hyper-prior.
Distribution model: Each task posterior is parameterized as a factorized Gaussian, whose analytic KL divergence to the shared prior supplies a differentiable complexity term.The task parameters contain means and log-variances, while the shared prior has the same structure.
Optimization: SGD estimates gradients using mini-batches, Monte Carlo samples, and reparameterization, making expectations over Gaussian weights tractable.The local reparameterization trick samples per data point and performs randomization at activations to reduce variance and improve neural-network efficiency.
Two-phase algorithm: MLAP uses meta-training tasks to learn a prior, then uses that prior during meta-testing on an unobserved task.The meta-training phase can also function independently as a multi-task learning method.

5. Experimental Demonstration

The experiments evaluate MLAP on related MNIST task environments and compare its transfer performance with scratch learning, warm-start, oracle, and alternative objectives. MLAP improves substantially in the tested settings, while additional tasks improve transfer and the learned prior assigns flexibility differently across network layers.

Experimental setup: The study uses permuted-label and permuted-pixel MNIST environments, with meta-training on 60,000 examples and meta-testing on new tasks using 2,000 examples.The pixel environment varies task relatedness through a limited number of pixel-location swaps.
Results: MLAP-M and MLAP-S improve considerably over learning from scratch and naive warm-start transfer, and outperform the oracle in the permuted-pixels experiment.MLAP-PL performs much worse because its complexity terms are disproportionately large relative to empirical error terms.
Results: The MLAP results are slightly better than MAML because the learned prior acts both as an initialization and as a data-dependent regularizer.Unlike MAML’s initial point, the prior can fix some weights while allowing variation in others depending on the amount of data.
Effect of training-task count: After observing 3 or more permuted-label tasks, transfer becomes beneficial, whereas one or two tasks produce negative transfer relative to the 2.27% scratch error.In permuted-pixel environments, more swaps require more training tasks before the prior yields positive transfer; scratch learning achieves 7.9% test error.
Learned prior: The learned prior assigns low variance to lower layers and high variance to the output layer in the permuted-label experiment, expressing layer-specific flexibility.Higher log variance means weights are more flexible to change and less strongly penalized for deviating from the nominal prior.

6. Discussion and Future Work

The paper presents PAC-Bayes-motivated meta-learning through adjustment of a learned prior and reports preliminary empirical support for deriving algorithms from tighter performance bounds. It identifies sequential learning, stable stochastic optimization, and reinforcement-learning extensions as open directions.

Conclusions: The framework adjusts a learned prior from encountered tasks and combines gradient-based learning with deep neural networks to implement meta-learning.The approach is motivated by extended PAC-Bayes bounds and operates at the task level rather than the sample level.
Conclusions: The authors describe the experimental results as preliminary but report that tighter bounds lead to improved performance.They present this as evidence for the utility of using rigorous performance bounds to derive learning algorithms.
Future work: The current method learns all available tasks in parallel rather than sequentially, although the prior could be updated after each novel task.The authors identify sequential learning as an open issue rather than an implemented capability.
Future work: Training stochastic models requires high-variance gradients, motivating methods with more stable convergence and easier application to larger-scale problems.Extending the techniques to reinforcement-learning algorithms based on performance bounds is another stated challenge.

A.1. Proof of the Meta-Learning Bound

The proof applies the classical PAC-Bayes bound twice: first to finite samples within observed tasks, then to the finite set of observed tasks from the environment.

First step: The first proof step bounds each observed task’s generalization error using a tuple hypothesis containing both a prior P and hypothesis h.The posterior samples P from Q and h from the task-specific posterior Q(S_i, P).
First step: The KL term compares the joint posterior over priors and hypotheses with a joint prior that first samples P and then h.This construction applies the single-task PAC-Bayes theorem to each observed task.
Second step: The second proof step bounds environment-level generalization caused by observing only finitely many tasks.Tasks, sample sizes, and datasets are treated as i.i.d. samples from the task environment, with the loss averaging over posterior hypotheses and task data.
Combining the bounds: The final result combines the within-task and environment-level events with a union bound over their failure probabilities.The proof sets the task-level failure probabilities to δ/(2n) and intersects the resulting events.

A.2. Meta-Learning Bound Based on Alternative Single-Task Bounds

The framework can incorporate alternative single-task PAC-Bayes bounds, producing a meta-learning bound that may be tighter when empirical training error is very small. The paper also identifies stronger bounds left for future work.

Alternative bounds: The proof technique extends different single-task PAC-Bayes bounds to derive alternative meta-learning bounds.The section specifically considers Seeger’s bound as an alternative to the classical McAllester bound.
Alternative bounds: The alternative intra-task bound replaces the classical bound in the first proof step.Using Seeger’s theorem yields a corresponding meta-learning bound.
Tightness: O(1/m) convergence applies when empirical error is negligibly small relative to D(Q||P)/m, compared with O(1/√m) for the classical bound.The paper expects the alternative bound to be tighter in this regime, which it describes as common in modern deep learning.
Scope: More recent variance-sensitive, deep-neural-network-specific, and non-KL PAC-Bayes bounds are identified but not incorporated.The paper leaves their integration into the framework for future work.

A.3. Hierarchical Variational Bayes

A hierarchical Bayesian formulation introduces shared and task-specific latent variables, then approximates the intractable posterior through variational optimization. Its resulting objective is similar to the PAC-Bayes-derived meta-learning objective.

Model assumptions: The hierarchical model uses a shared random variable ψ and task-specific variables w_i for the observed task datasets.The model assumes conditional independence relationships among tasks, variables, and samples.
Model assumptions: The formulation assumes known priors over ψ and w_i conditioned on ψ, a known likelihood, and independent samples given each task variable.These assumptions define the probabilistic structure used for variational inference.
Posterior approximation: The joint posterior factorizes into a posterior over ψ and a conditional posterior over task-specific variables given ψ and all observed datasets.The factorization follows from conditional probability and the stated conditional independence assumption.
Posterior approximation: Because the exact posterior is intractable, the method uses a parameterized variational family and optimizes its parameters for the best approximation.The parameters are denoted θ and φ_i.
Connection to PAC-Bayes: The resulting variational objective is similar to the meta-learning generalization-bound objective, while the paper’s algorithm is derived from a bound rather than formulated as Bayesian inference.The paper reports similar experimental results for the two objectives.

A.4. Pseudo Code

The MLAP algorithm learns prior parameters across observed tasks and initializes adaptation to a new task from those learned parameters.

Meta-training: During meta-training, MLAP takes datasets from observed tasks and optimizes shared prior parameters together with task-specific parameters.The objective gradient is evaluated with backpropagation and updated iteratively.
Meta-training: For large task collections, each training iteration can use a sampled subset of tasks as a meta mini-batch.This estimates the objective while reducing the tasks processed per iteration.
Meta-testing: During meta-testing, the learned prior parameters initialize posterior parameters for a new task.The new-task parameters are then optimized using mini-batches, Monte Carlo averaging, and backpropagation.

A.5. Classification Example Implementation Details

The toy experiment illustrates how meta-training learns a prior from multiple tasks while task-specific posteriors remain near their respective task means. The implementation also specifies a small CNN setup and compares the learned-prior procedure with MAML settings.

Classification implementation: The permuted-labels experiment uses a small CNN with two convolutional layers, a 50-unit hidden layer, dropout, max pooling, and ELU activations.The convolutional layers use 10 and 20 filters with 5 × 5 kernels, while dropout before the output layer has p = 0.5.
Classification implementation: MAML results were selected over combinations of 1–3 meta-training steps, 1–20 meta-testing steps, 300 iterations, and α ∈ {0.01, 0.1, 0.4}.The best reported setting used α = 0.01, two meta-training steps, and 18 meta-testing steps.
Toy example: The toy setup estimates a two-dimensional distribution mean from samples using Euclidean-distance loss and Gaussian prior and posterior distributions.
Toy example: The learned prior lies between the two task posteriors and has larger covariance in the first dimension.This reflects shared task structure near 1 in dimension 2 and 3 in dimension 1, with greater variation along the first dimension.
Toy example: New similar tasks can be learned using the learned prior with fewer samples.