PaperlayerView exact revision on arXiv

Source-linked AI summary

Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation

Risto Vuorio, Shao-Hua Sun, Hexiang Hu, Joseph J. Lim

arXiv:1910.13616v1cs.LGcs.AIstat.ML

TL;DR

Model-agnostic meta-learners struggle to use one initialization across multimodal task distributions. MMAML infers task modes and modulates the meta-learned prior before gradient adaptation, achieving superior performance across regression, image classification, and reinforcement learning. The paper concludes that task-mode recognition and prior modulation improve adaptation to multimodal tasks.

  • Problem

    A single shared initialization can be inadequate for multimodal task distributions whose tasks require substantially different parameters.

  • Method

    MMAML uses a modulation network to infer a task’s mode from task data, generate task-specific parameters, modulate the task network’s prior, and then adapt by gradient updates.

  • Results

    MMAML achieves superior performance in multimodal few-shot regression, image classification, and reinforcement learning while effectively recognizing task modes.

  • Takeaways & Limitations

    Training and adapting with mode-aware prior parameters enables the model to exploit multimodal task structure without extra supervision for task-mode identification.

  • Takeaways & Limitations

    The general model-agnostic framework omits comparisons with methods that achieve strong performance only on an individual domain.

Abstract

from arXiv · show

Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those frameworks demonstrate appealing performance on a variety of domains such as few-shot image classification and reinforcement learning. However, one important limitation of such frameworks is that they seek a common initialization shared across the entire task distribution, substantially limiting the diversity of the task distributions that they are able to learn from. In this paper, we augment MAML with the capability to identify the mode of tasks sampled from a multimodal task distribution and adapt quickly through gradient updates. Specifically, we propose a multimodal MAML (MMAML) framework, which is able to modulate its meta-learned prior parameters according to the identified mode, allowing more efficient fast adaptation. We evaluate the proposed model on a diverse set of few-shot learning tasks, including regression, image classification, and reinforcement learning. The results not only demonstrate the effectiveness of our model in modulating the meta-learned prior in response to the characteristics of tasks but also show that training on a multimodal distribution can produce an improvement over unimodal training.

1 Introduction

Existing model-agnostic meta-learners typically seek one initialization across tasks, which is limiting when task distributions contain substantially different modes. MMAML addresses this by inferring task modes, modulating prior parameters, and evaluating multimodal adaptation across several domains.

  • Motivation: A single initialization can be inadequate when multimodal tasks require substantially different parameters.Separate mode-specific learners could better cover disjoint modes, but task assignment may require unavailable or ambiguous identity information and reduce cross-mode transfer.
  • Proposed approach: MMAML augments MAML by inferring a sampled task’s mode from task data and conditioning its initialization through modulation.The framework combines model-based task identification with model-agnostic gradient adaptation.
  • Evaluation: Experiments cover regression, image classification, and reinforcement learning, with results demonstrating effectiveness against other systems.Further analysis indicates that MMAML identifies task modes without extra supervision.
  • Contributions: The proposed framework generates mode-aware prior parameters and adapts quickly on tasks from multimodal distributions.Its contributions include identifying the single-initialization limitation, proposing an algorithmic remedy, and designing multimodal problems.

2 Related Work

Related meta-learning approaches differ in what they learn: metrics, stateful mappings, or model-agnostic initializations. MMAML extends model-agnostic meta-learning to complex multimodal task distributions by identifying modes before rapid fine-tuning.

  • Few-shot and metric-based learning: Metric-based methods learn distances for comparing samples in few-shot image classification.Related approaches also perform task-specific adaptation through different type transformations.
  • Model-based meta-learning: Model-based meta-learners adjust model states or external memory and output learned-model parameters or predictions from datasets and test samples.Their flexible mappings can suffer from overfitting and limited generalization ability.
  • Model-agnostic meta-learning: Model-agnostic meta-learners learn parameter initializations that provide favorable inductive bias for fast gradient-based adaptation.Their model-agnostic nature supports varied learning problems, but common initialization is tied to concentrated task distributions.
  • MMAML: MMAML identifies the mode of sampled tasks, modulates meta-learned prior parameters, and rapidly fine-tunes on the target task.This directly targets complex multimodal task distributions.

3 Preliminaries

Meta-learning seeks rapid task-specific function learning from few examples, with tasks defined by data-generating and conditional label distributions. MAML learns an initialization optimized for few-step adaptation and validation generalization.

  • Task formulation: A task comprises input-output examples sampled from a task-specific data-generating distribution and conditional probability.The number of examples K_t is small.
  • Task distributions: A unimodal task distribution contains one input and label domain, whereas a multimodal distribution combines domains such as digits and birds.The distinction is defined by the domains represented across classification tasks.
  • Meta-learning setup: Meta-training and meta-testing divide the task dataset into evaluation stages, with task-specific training sets used for learning structure and testing sets for evaluation.The meta-learner learns the underlying task-distribution structure from meta-training tasks.
  • MAML: MAML learns an initialization θ so a new task can be optimized with few gradient steps on training data and generalize to validation data.Meta-updates use validation losses computed after adapting each sampled task.

4 Method

MMAML uses a modulation network to infer task characteristics and generate task-specific parameters that alter a task network’s meta-learned prior. The modulated network then adapts through gradient updates, with FiLM selected as the default operator.

  • Architecture: The modulation network reads target-task data, predicts its mode, and generates task-specific parameters for the task network.The task network is then adapted with gradient-based optimization.
  • Architecture: Figure 1 depicts task embedding υ producing parameters {τ_i} that modulate the task network before target-task adaptation.The task network adapts the modulated parameters to fit the target task.
  • Modulation network: The modulation network encodes K labeled task examples into an embedding υ representing task characteristics.Task-specific parameters τ are computed from this embedding.
  • Modulation network: Each block receives parameters τ_i generated from υ, and the modulated prior is φ_i = θ_i ⊙ τ_i.N denotes the number of task-network blocks, while ⊙ is a general modulation operator.
  • Modulation operators: MMAML explores attention-based softmax modulation and FiLM, selecting FiLM because it performs better and more stably in experiments.The task network may use arbitrary parameterized blocks such as convolutional or recurrent layers.
  • Adaptation: After modulation, gradient descent updates only the task network’s meta-learned prior parameters while keeping task-specific τ_i fixed.The same modulation and optimization procedure is used during meta-training and meta-testing.
  • Qualitative behavior: Before updates, MMAML qualitatively fits five-mode regression functions better than MAML and Multi-MAML; after five updates, its predictions improve further.Task-embedding visualization separates input data from different tasks, supporting mode identification.

5 Experiments

Experiments evaluate MMAML against MAML, Multi-MAML, and related variants across multimodal regression, image classification, and reinforcement learning. Across these domains, results support task-mode identification, modulation, and improved adaptation to multimodal task distributions.

  • Experimental setup: MMAML is evaluated against MAML, Multi-MAML, and domain-specific variants across regression, image classification, and reinforcement learning.The experiments use multimodal task distributions in all three domains.
  • Regression experiments: MMAML combines an LSTM modulation network with gradient-based adaptation of a modulated task network.The modulation network processes task data to generate task-specific parameters; FiLM and Softmax operators are explored.
  • Regression experiments: MMAML with FiLM outperforms other methods on multimodal 5-shot regression, while FiLM consistently outperforms Softmax.MAML has the highest error across settings, and MMAML improves significantly over the LSTM learner.
  • Image classification: MMAML outperforms MAML and performs comparably to Multi-MAML in multimodal few-shot image classification.The performance gap between MMAML and MAML increases as the number of modes grows; MMAML slightly exceeds Multi-MAML in the reported comparison.
  • Image classification: tSNE visualizations show task embeddings clustering by dataset or task mode, supporting MMAML’s ability to identify modes from few samples.In few-shot image classification, each dataset forms an individual cluster; reinforcement-learning task modes also cluster clearly.
  • Reinforcement learning: In reinforcement learning, MMAML consistently outperforms unmodulated ProMP across multimodal continuous-control environments.The reported analysis suggests that distinct task modes, rather than mode count alone, influence adaptation difficulty.

6 Conclusion

The approach combines task-mode identification with gradient-based meta-learning to modulate prior parameters for multimodal task distributions. It is evaluated across several domains, with methodological scope bounded by the comparison choices described.

  • 6 Conclusion: MMAML identifies a task’s mode from a few samples, modulates the meta-learned prior, and then adapts through gradient updates.The modulation network combines model-based task identification with model-agnostic gradient adaptation.
  • 6 Conclusion: MMAML was evaluated on multimodal few-shot regression, image classification, and reinforcement learning, where it achieved superior generalization performance on sampled multimodal tasks.The experiments target a general model-agnostic framework rather than methods specialized to one individual domain.
  • 6 Conclusion: The modulation operators use attention or FiLM to condition neural-network activations or parameters on task-specific information.Attention produces Softmax or Sigmoid modulation vectors, while FiLM uses scale and shift components.
  • 6 Conclusion: The reinforcement-learning comparison with ProMP isolates the potential advantage of adding a separate modulation network to the task network.In that setting, MMAML uses ProMP as its optimization algorithm.

D Additional Experimental Details

The regression experiments construct multimodal task distributions from several function families and evaluate models under noisy, few-shot conditions. The setup includes visualization of task embeddings and a recurrent modulation component for MMAML.

  • D Additional Experimental Details: Regression tasks combine sinusoidal, linear, quadratic, ℓ1 norm, and hyperbolic tangent functions with sampled parameters and Gaussian observation noise.The noise has µ = 0 and ϵ = 0.3.
  • D Additional Experimental Details: tSNE visualizations show task embeddings for regression distributions containing two, three, and five modes.The plots visualize embeddings produced from randomly sampled tasks.
  • D Additional Experimental Details: The regression evaluation uses K = 5, L = 10, and mean squared error as its criterion.The multimodal and noisy setting is described as more challenging than the referenced MAML setting.
  • D Additional Experimental Details: MMAML uses a four-layer fully connected ReLU base network and a bidirectional LSTM with hidden size 40 to generate modulation vectors.The base network has hidden dimensions of 100, and the LSTM modulates each base-model layer.

D.1.3 Evaluation Protocol

The evaluation samples large numbers of tasks across regression modes and measures how modulation and gradient adaptation progressively improve performance. Modulation provides a rough shape-matching solution, while adaptation refines it.

  • D.1.3 Evaluation Protocol: The evaluation samples 25,000 tasks per mode and applies five gradient steps during adaptation.The resulting totals are 50,000, 75,000, and 125,000 tasks for two-, three-, and five-mode experiments, respectively.
  • D.1.3 Evaluation Protocol: Regression models are evaluated over the function range from -5 to 5.The passage states that the reported evaluation covers this range.
  • D.1.3 Evaluation Protocol: Prior parameters alone produce high error, while modulation and subsequent adaptation significantly reduce the error on five-mode regression.The comparison is reported using qualitative curves and quantitative MSE results in Figure 8 and Table 4.
  • D.1.3 Evaluation Protocol: Modulation seeks a rough solution capturing the target curve’s shape, and gradient-based adaptation refines the resulting curve.MMAML sequentially applies modulation and adaptation from learned prior parameters.

D.2.1 Meta-dataset

The image meta-dataset merges five diverse datasets into a common resized-image setting to create multimodal classification tasks. The supplementary materials document the associated figures, table, hyperparameters, and training conventions.

  • D.2.1 Meta-dataset: The meta-dataset merges OMNIGLOT, MINI-IMAGENET, FC100, CUB, and AIRCRAFT.These five datasets provide the source modes for the multimodal image-classification setting.
  • D.2.1 Meta-dataset: All images are resized to 84 × 84 before being fitted to the model.Figure 9 displays randomly sampled images from the merged datasets.
  • D.2.1 Meta-dataset: Figure 8 presents five-mode regression with linear and quadratic functions.Table 4 reports five-mode regression performance using mean squared error.
  • D.2.1 Meta-dataset: The experiments generally use shared hyperparameters for the model and MAML, with a smaller meta-batch size and more iterations for 20-way tasks.The stated exception addresses limited GPU memory.
  • D.2.1 Meta-dataset: Post-update meta-gradients use 15 examples per class, and training uses Adam with default hyperparameters.This convention follows the cited prior works.
  • D.2.1 Meta-dataset: Multi-MAML permits different hyperparameter sets because it trains a separate MAML model for each dataset.The authors report trying to find the best hyperparameters for each dataset.

D.2.3 Network Architectures

The image-classification architecture uses parallel task and task-encoding networks, with modulation MLPs generating layer-specific parameters for the task network.

  • Task Network: The task network follows MAML’s convolutional architecture, using four convolutional layers with channel sizes 32, 64, 128, and 256.Each convolution uses a 3×3 kernel and stride 2, followed by batch normalization and ReLU; a final linear layer produces n-way predictions.
  • Task Encoder: The task encoder uses the same four-layer convolutional architecture to process n-way k-shot inputs of size (n · k) × 84 × 84 × 3.Its layers use valid padding and are followed by batch normalization and ReLU.
  • Modulation MLPs: Four linear modulation layers map the task embedding to layer-specific τγ and τβ vectors matching channel dimensions 32, 64, 128, and 256.These dimensions correspond to the four convolutional layers of the task network.
  • Experimental Settings: The multimodal image-classification experiments use dataset-specific hyperparameters across grayscale and RGB dataset groups.The grayscale group includes OMNIGLOT, while the RGB group includes MINI-IMAGENET, FC100, CUB, and AIRCRAFT.

D.3.1 Environments

The reinforcement-learning evaluation uses multimodal goal-reaching environments with point-mass, robotic-arm, and ant agents, trained through repeated trajectory sampling and gradient updates.

  • Training Procedure: Training curves report average return per episode after gradient-based updates and modulation, with smoothing and variability described in Figure 11.The figure uses a ten-step averaging window and shaded standard deviation across three random seeds.
  • POINT MASS: POINT MASS includes 2-, 4-, and 6-mode variants in which velocity-controlled agents pursue goals sampled from multimodal distributions.Rewards are the negative Euclidean distance to the goal, and the 4-mode goals are located at the four combinations of ±5 coordinates.
  • REACHER: REACHER includes 2-, 4-, and 6-mode variants with a three-link, two-dimensional robot arm simulated in MuJoCo.Its goal distributions use mode-dependent locations and noise scale 0.1; the 4-mode centers lie at combinations of ±0.225 coordinates.
  • ANT: ANT includes two- and four-mode variants in which a four-limbed MuJoCo ant controls movement toward multimodally sampled goals.The four-mode centers are at (−4, 0), (−2, 3.46), (2, 3.46), and (4.0, 0), with goal-noise scale 0.8.
  • Training Procedure: Each reinforcement-learning update samples 40 tasks, with 20 trajectories sampled for each gradient step and task.The associated hyperparameter settings are provided in Table 10.

E Additional Experimental Results

Additional experiments examine task-embedding structure, dataset-specific image-classification behavior, and qualitative regression and reinforcement-learning results across multimodal settings.

  • E.1 Regression: Regression embeddings show linear separation for two- and three-mode tasks, while five-mode embeddings clutter similar function families under output noise.Linear, transformed ℓ1-norm, and hyperbolic-tangent functions are reported as especially similar in the five-mode visualization.
  • E.2 Image Classification: The appendix provides per-dataset classification results for 2-, 3-, and 5-mode experiments, while the main paper reports overall performance columns.The corresponding detailed results are organized in Tables 7, 8, and 9.
  • E.2 Image Classification: OMNIGLOT and MINI-IMAGENET results follow the tendency reported in prior work, while results may differ because of preprocessing, hyperparameters, and training-iteration choices.The cited preprocessing includes resizing and spatial tiling of images.
  • E.2 Image Classification: Additional classification visualizations show tSNE task embeddings for 2-mode, 3-mode, and 5-mode settings.Figure 10 covers 5-way 1-shot tasks for the 2- and 3-mode cases and a 5-way 5-shot task for the 5-mode case.
  • E.3 Reinforcement Learning: Additional reinforcement-learning figures compare MMAML and ProMP through smoothed training curves and sampled trajectories after zero, one, and two updates.The trajectory figure covers six point-mass tasks and places modulation between the initial policy and the policy after one update.
  • E.1 Regression: Additional regression figures compare MMAML after adaptation with other posterior models.These results are presented as qualitative comparisons.
Loading 1910.13616v1…