PaperlayerView exact revision on arXiv

Source-linked AI summary

Meta-Learning Update Rules for Unsupervised Representation Learning

Luke Metz, Niru Maheswaranathan, Brian Cheung, Jascha Sohl-Dickstein

arXiv:1804.00222v3cs.LGcs.NEstat.ML

TL;DR

Unsupervised objectives often optimize proxies and leave downstream usefulness to emerge indirectly. This paper meta-learns a neuron-local unsupervised update rule for later tasks, and reports useful features, competitive held-out performance, and generalization across architectures and domains.

  • Problem

    Many unsupervised methods optimize generative likelihood or reconstruction, with useful downstream representations emerging only as a side effect.

    Source: §1 · p. 1

  • Method

    The paper meta-learns a neuron-local, label-free update rule to produce representations directly useful for subsequent tasks.

    Source: §1 · pp. 1–2, §3.2 · p. 5

  • Results

    The update rule matches or exceeds existing unsupervised learning on held-out tasks and trains networks with varied widths, depths, and activation functions.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The work offers a proof of principle for learning complex optimization algorithms from data rather than relying only on hand-designed techniques.

    Source: §6 · p. 10

Abstract

from arXiv · show

A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect. In this work, we propose instead to directly target later desired tasks by meta-learning an unsupervised learning rule which leads to representations useful for those tasks. Specifically, we target semi-supervised classification performance, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations useful for this task. Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to different neural network architectures, datasets, and data modalities. We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques. We further show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities. It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.

1 INTRODUCTION

Many unsupervised objectives optimize proxies such as likelihood or reconstruction, leaving downstream usefulness to emerge as a side effect. This work instead meta-learns an unsupervised update rule directly for useful representations in later tasks.

1 INTRODUCTION

  • Many unsupervised objectives optimize generative likelihood or reconstruction, so useful representations arise only as a side effect.The motivation is to produce representations that expose high-level attributes and work well in semi-supervised settings.Source: §1 · p. 1
  • The paper meta-learns an unsupervised update rule with a meta-objective that directly targets the usefulness of representations for later supervised tasks.Unlike hand-designed learning rules, it evaluates representations by their utility for candidate tasks.Source: §1 · p. 1
  • The work recasts unsupervised representation learning as transfer learning of a learning rule rather than of transferable features.The learned rule does not require labels and is intended to generalize across data domains and neural network architectures.Source: §1 · pp. 1–2

2 RELATED WORK

Prior work learns representations through manually specified objectives, losses, or algorithms, while this paper learns the unsupervised algorithm itself. The related work also situates the approach within meta-learning and its comparisons across prior methods.

2.1 UNSUPERVISED REPRESENTATION LEARNING

  • Prior approaches use reconstruction objectives, generative modeling, or self-supervised proxy tasks to obtain unsupervised features.Examples include autoencoders and GANs, as well as image-jigsaw and video-temporal-ordering tasks.Source: §2.1 · p. 2
  • Other methods build representations through clustering or by optimizing desired latent properties, whereas this work learns the algorithm that creates useful representations.The desired properties include predictability, encoding complexity, independence, and sparsity.Source: §2.1 · p. 2

2.2 META LEARNING

  • Meta-learning optimizes an inner-loop computation through an outer loop, using a meta-objective to evaluate its performance.Approaches differ in inner-loop computation, domain, meta-parameters, and outer-loop optimization.Source: §2.2 · p. 2
  • Prior meta-learning work includes learned neuron-local rules and few-shot methods, while the paper frames its inner loop as unsupervised representation learning.The paper distinguishes its approach from transfer learning, which trains on a similar dataset before fine-tuning or post-processing on the target dataset.Source: §2.2 · pp. 2–3
  • Figure 1 depicts repeated application of the update rule and a base-model update using decoupled backward weights and neuron-level learning signals.The signals are generated by meta-learned MLPs rather than an explicit loss error signal.Source: Figure 1 · p. 3
  • Table 1 compares published meta-learning approaches.The comparison highlights task, meta-learning structure, meta-architecture, and domain.Source: §2.2 · p. 3

3 MODEL DESIGN

The model uses a learned, label-free update rule to train a base network, with meta-parameters optimized for downstream representation utility. Its neuron-local update design and labeled meta-objective define how the rule is trained and evaluated.

3 MODEL DESIGN

  • The inner loop trains a base MLP by repeatedly applying a learned update, while SGD trains the update rule’s meta-parameters against a task-distributed meta-objective.The update rule is label-free and generalizes in form across unsupervised algorithms; Table 1 compares published meta-learning approaches.Source: §3 · pp. 3, 5, Table 1 · p. 4

3.1 BASE MODEL

  • The base model is a fully connected MLP with batch normalization and ReLU, selected to limit convolutional inductive bias.Its layer weights and biases define the forward computation, with corresponding weights used in the backward pass.Source: §3.1 · p. 5

3.2 LEARNED UPDATE RULE

  • The update rule is designed to be neuron-local, using pre- and post-synaptic activity so it can apply across architectures of varying widths, depths, and topologies.Per-neuron MLPs share meta-parameters, receive feedforward activations and feedback information, and produce learning signals; the full weight update includes normalization and decorrelation.Source: §3.2 · pp. 5–6
  • The update networks accumulate activation statistics across minibatches, then propagate generated signals backward using learned weights separate from the forward weights.The backward weights are used instead of the transpose of the forward weights, unlike standard backpropagation.Source: §3.2 · p. 5

3.3 META-OBJECTIVE

  • The meta-objective fits a linear regression on one labeled minibatch and evaluates classification on another, using cosine distance for stability.Labels, including possible augmentation targets, are used only for meta-training; the inner-loop update itself runs without labels.Source: §3.3 · p. 6

4 TRAINING THE UPDATE RULE

The update rule is meta-trained with approximate gradients and a varied distribution of datasets and architectures. The implementation distributes partial inner-loop unrolls across workers.

4.1 APPROXIMATE GRADIENT BASED TRAINING

  • The authors use SGD for meta-optimization and truncated backpropagation through time to approximate gradients through long recurrent computations.Batch normalization and restricting update-step norms are among the additional choices used to improve stability and convergence.Source: §4.1 · p. 6

4.2 META-TRAINING DISTRIBUTION AND GENERALIZATION

  • Meta-training samples CIFAR10, ImageNet class subsets, and rendered fonts, with varied MLP depths and widths; evaluation includes MNIST, Fashion MNIST, IMDB, and held-out ImageNet classes.Inputs are resized to 16x16 pixels or less during meta-training, and all input dimensions are randomly permuted to encourage a permutation-invariant rule; augmentations include shifts, rotations, and noise.Source: §4.2 · p. 6

4.3 DISTRIBUTED IMPLEMENTATION

  • Training uses 512 workers running partial inner-loop unrolls and asynchronous meta-objective gradients, taking about eight days and 200,000 updates to θ.The minibatch size is 256.Source: §4.3 · p. 7

5 EXPERIMENTAL RESULTS

The experiments show that the learned update rule can produce useful representations across held-out datasets and settings, while also revealing domain-specific limits and how learned features develop.

5.1 OBJECTIVE FUNCTION MISMATCH AND EXISTING META-LEARNING METHODS

  • On 16×16 CIFAR10, a VAE’s few-shot classification accuracy drops sharply even as its objective continues improving; a prototypical network is also hampered by pixel shuffling.Figure 2 contrasts VAE training with few-shot accuracy and compares prototypical-network performance on intact versus shuffled inputs.Source: §5.1 · p. 7, Figure 2 · p. 7

5.2 META-OPTIMIZATION

  • Meta-objective training loss continues to decrease after 200 hours, and evaluation loss keeps decreasing on held-out image datasets, while IMDB shows meta-overfitting.The 200-hour checkpoint supplies the meta-parameters used for subsequent experiments unless otherwise stated.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 GENERALIZATION

  • On held-out MNIST and Fashion MNIST at 14×14 and 28×28, the learned update rule outperforms a VAE, supervised learning on the labeled examples, and random initialization with a trained readout.Evaluations use 10 examples per class and the same base-model architecture.Source: Figure 4 · p. 8, §5.3 · p. 8
  • After 30 hours of meta-training, the image-trained rule improves IMDB performance over random initialization by almost 10%, but longer meta-training leads to poor performance from image-domain meta-overfitting.The authors note that absolute performance on this text task remains quite low.Source: §5.3 · p. 8, Figure 4 · p. 8
  • A rule trained on networks with 2–5 layers and 64–512 units per layer generalizes to networks with 11 layers and 10,000 units per layer.Figure 5 presents performance as model size increases.Source: Figure 5 · p. 9, §5.3 · p. 9
  • Although trained only with ReLU, the rule improves over random initialization for all tested activations and reaches twice random-initialization performance with step activations.The rule does not use base-model gradients, enabling optimization despite the step activation’s lack of a useful gradient signal.Source: §5.3 · p. 9, Figure 5 · p. 9

5.4 HOW IT LEARNS AND HOW IT LEARNS TO LEARN

  • As meta-training progresses, first-layer features become more structured: MNIST filters look template-like, while CIFAR10 features become higher-frequency and more spatially localized.The filters shown were produced after 10,000 update applications.Source: §5.4 · p. 9, Figure 6 · p. 10
  • Without labels, the learned rule partially separates the two-moons data manifold; its representation variance is concentrated in fewer dimensions than MNIST’s.Both representations occupy only a fraction of the base network’s 32-dimensional space.Source: §5.4 · p. 9, Figure 6 · p. 10

6 DISCUSSION

The learned unsupervised update rule matches or exceeds existing unsupervised methods on held-out tasks and trains models with varied architectures and activation functions. The system diagram shows how meta-training optimizes the rule and how that rule updates a base model using unlabeled data.

6 DISCUSSION

  • The meta-learned update rule matches or exceeds existing unsupervised learning on held-out tasks and trains models with varying widths, depths, and activation functions.The work frames this as a proof of principle for meta-learning complex optimization tasks without an explicitly defined objective.Source: §6 · p. 10
  • The authors present meta-learning as a way to learn transferable update algorithms rather than hand-designed techniques, with the algorithm itself learned from data.They suggest the meta-objective could target representations for subsequent tasks beyond semi-supervised classification.Source: §6 · p. 10

A MORE DETAILED SYSTEM DIAGRAM

  • Meta-training updates θ by backpropagating through unrolled applications of UnsupervisedUpdate, which changes base-model parameters φ using unlabeled minibatches.The base model's backward pass uses a meta-learned error signal, and a convolutional network computes weight updates from pre- and post-synaptic activations and other terms.Source: A MORE DETAILED SYSTEM DIAGRAM · p. 15

B STABILIZING GRADIENT BASED META-LEARNING TRAINING

Training the learned update rule involves stabilizing gradients through recurrent computation and truncated unrolls. The authors use parallel workers, constrained inner-loop step sizes, and batch normalization to address instability.

B STABILIZING GRADIENT BASED META-LEARNING TRAINING

  • Standard measures such as gradient clipping, small learning rates, and Adam were insufficient to train most UnsupervisedUpdate architectures, so additional stabilization techniques were needed.The passage describes recurrent derivative computation as difficult and notes that training recurrent systems can lead to chaos.Source: B STABILIZING GRADIENT BASED… · pp. 15–16
  • Parallel workers maintain diverse base-model weight states while computing gradients, addressing the non-i.i.d. state distribution that changes as the learned optimizer is meta-optimized.The authors also sample truncation lengths and total unsupervised training steps to limit bias introduced by truncation.Source: B STABILIZING GRADIENT BASED… · p. 16
  • Constraining the maximum inner-loop step size was crucial for stability because unconstrained learning rates rapidly grew into a chaotic regime.Batch normalization in both the base model and UnsupervisedUpdate also helps by making more of the weight space usable when learned updates produce high-variance weights.Source: B STABILIZING GRADIENT BASED… · p. 16

C DISTRIBUTED IMPLEMENTATION

The implementation computes meta-objective gradients across 512 asynchronous workers, each sampling a task and unrolling the learned update rule. Gradients are batched for asynchronous SGD, and training uses multicore CPUs.

C DISTRIBUTED IMPLEMENTATION

  • A cluster of 512 workers asynchronously computes meta-objective gradients, with each worker sampling a dataset, architecture, and number of training steps for its task.Workers apply the UnsupervisedUpdate for k unrolling steps and compute the meta-objective at each new state.Source: C DISTRIBUTED IMPLEMENTATION · p. 16
  • Gradients from completed unrolls are batched for asynchronous SGD, reducing most gradient staleness while retaining compute efficiency across heterogeneous workloads.Each worker continues unrolling from the final base-model state until the specified training steps are reached.Source: C DISTRIBUTED IMPLEMENTATION · p. 16
  • Training used multicore CPUs rather than GPUs and ran for about 8 days, with about 200 thousand updates to θ and minibatch size 256.The authors cite small base models and sequential compute workloads as the reason for using CPUs.Source: C DISTRIBUTED IMPLEMENTATION · p. 16

D MORE FILTERS OVER META-TRAINING

Filters extracted during meta-training shift from noise at initialization toward more local features. The figure samples filters after 10,000 inner-loop steps across five datasets, but its columns do not correspond to shared training iterations.

D MORE FILTERS OVER META-TRAINING

E META-TRAINING CURVES

The section presents training and evaluation curves for meta-training and meta-evaluation task distributions, and outlines the distributed procedure used to train the update rule.

E META-TRAINING CURVES

  • The curves cover Mini Imagenet, Alphabet, and MiniCIFAR10 for meta-training, and Mini Imagenet Test, Tiny Fashion MNIST, Tiny MNIST, and IMDB for meta-evaluation.Error bars show standard deviations across evaluations from a fixed window of samples evaluated from one model.Source: E META-TRAINING CURVES · p. 18

F LEARNING ALGORITHM

  • Training initializes the update-rule parameters and runs partial inner-loop unrolls in parallel across 512 workers.Each worker samples a supervised task, base model, and truncated number of iterations before applying UnsupervisedUpdate to sampled data.Source: F LEARNING ALGORITHM · p. 18
  • The procedure evaluates a meta-objective on two data batches, accumulates gradients over the last U steps, and updates the shared parameters with averaged gradients and Adam.Workers store gradients in shared state until it exceeds the meta-batch size.Source: F LEARNING ALGORITHM · p. 18

G MODEL SPECIFICATION

The model applies a learned unsupervised update to a base network and trains that update rule using few-shot linear-regression performance. Its update computations use layer activity, learned top-down signals, and local weight-update terms.

G MODEL SPECIFICATION

  • The authors describe the system's architecture space as massive at the time of writing and identify simplifying and improving algorithmic abstractions as future work.They state they were unaware of similar systems at that time.Source: G MODEL SPECIFICATION · p. 19

G.1 INNER LOOP

  • The base model is an L-layer MLP whose parameters include forward weights, biases, and backward weights used during inner-loop training.The learned rule iteratively updates these parameters using unlabeled batches; its updates include a decay term and are computed by meta-learned functions.Source: §G.2 · p. 19, §G.1 · p. 19

G.3 METAOBJECTIVE

  • The meta-objective evaluates representations through few-shot linear regression, fitting on one batch and computing loss on another.The predicted targets are centered and normalized, and the resulting normalized prediction loss corresponds to cosine distance up to an offset and multiplicative factor.Source: §G.3 · pp. 20–21

G.4.1 BACKWARD ERROR PROPAGATION

  • The update rule shares its parameters across layers and uses a learned vector-valued top-down signal rather than a single backpropagated scalar error.Separate backward weights carry the signal down the network, and the signal is normalized to a fixed second moment.Source: §G.4 · p. 21, §G.4.1 · p. 21

G.5.1 LOCAL TERMS

  • Weight updates combine low-rank readouts from hidden states of adjacent layers, with local terms expressing how weights change with pre- and post-synaptic activity and current weight values.Additional decorrelation planes are intended to help units avoid near-identical representations.Source: §G.5 · p. 22, §G.5.1 · p. 22, §G.5.2 · pp. 22–23

G.6 APPLICATION OF THE WEIGHT TERMS IN THE OPTIMIZER

  • The optimizer normalizes and merges update planes, then constrains weight and bias updates to prevent training pathologies.The stated risks include increasing the effective learning rate and pushing all units into the linear regime.Source: §G.6 · p. 23

G.7 TopD

  • TopD processes the top-layer representation using batch-dimension convolutions and unit convolutions, with the latter rewriting information across the batch.This unit-convolution design restricts the model to a fixed-size batch.Source: §G.7 · p. 24

G.8 ComputeH

  • ComputeH combines layer activations, positional embeddings, the top-down signal, and statistics of activations and weights before applying convolutions across batch and unit dimensions.Its output is used directly for weight updates.Source: §G.8 · pp. 25–26

H.1.1 TRAINING DATA DISTRIBUTION

The appendix specifies the task distribution and experimental setups used to train and evaluate the learned update rule, including meta-optimization choices and comparison conditions.

H.1.1 TRAINING DATA DISTRIBUTION

  • Half of the training tasks use rendered glyphs, with varied classification ways and selected letter, math-symbol, and currency-symbol tasks; the remaining tasks include CIFAR10 and ImageNet subsets.Glyphs are rendered as 14x14 black-and-white images; CIFAR10 and ImageNet inputs are resized to 16x16.Source: §H.1.1 · p. 26
  • The training distribution also applies task-level and per-example dropout, class permutations, and per-image horizontal shifts as augmentations.The passage states these augmentations help with larger domain transfer.Source: §H.1.1 · p. 26

H.2 META-OPTIMIZATION

  • Meta-training uses Adam with a staged learning-rate schedule, gradient clipping, and an objective averaged over five linear-regression evaluations.Truncated unroll lengths increase from [2,4] to [8,15] over 50,000 steps; the authors report this schedule improves speed and stability.Source: §H.2 · p. 26

H.3.1 OBJECTIVE FUNCTION MISMATCH

  • The VAE objective-mismatch experiment uses a three-layer, 128-unit ReLU model and evaluates labeled training-set few-shot classification while training on the unlabeled training set.This setup is described as isolating objective mismatch and overfitting.Source: §H.3.1 · p. 27

H.3.2 GENERALIZATION: DATASET AND DOMAIN

  • Dataset and domain experiments use four-layer, 128-unit models with 32-dimensional embeddings; the IMDB input encodes presence or absence of the top 1,000 words and special tokens.The supervised baseline uses 10 labeled examples per class and selects its learning rate using test-set performance.Source: §H.3.2 · p. 27

H.3.3 GENERALIZATION: NETWORK ARCHITECTURE

  • Unless an experiment specifies otherwise, architecture-generalization setups use four 128-unit layers, ReLU activations, and a 32-unit embedding.Source: §H.3.3 · p. 27

H.3.4 EXISTING META LEARNING MODELS

  • The comparison trains prototypical networks on intact or shuffled Mini-ImageNet images using a fully connected base model with batch normalization.Shuffling uses a separately sampled fixed input permutation for each base-network episode.Source: §H.3.4 · p. 27
Loading 1804.00222v3…