Modular meta-learning

• The motivation is to reduce new-task training-data requirements by extracting information from related tasks, especially where training experience is expensive.The paper describes this as transfer learning, also called meta-learning or “learning to learn”.Source: §1 · p. 1
• Rather than transfer parameter values as earlier meta-learning approaches do [3, 4, 5], the method reuses modules to pursue combinatorial generalization [6, 7, 8].The paper likens this reuse to combining words into many sentences.Source: §1 · p. 1
• For tasks lacking task-level input that reveals a suitable decomposition, BOUNCEGRAD uses simulated annealing to search for one.It combines a modular structure and parameter adaptation method to learn reusable modules.Source: §1 · p. 1
• BOUNCEGRAD achieves better prediction from a few training examples and can be much faster to train than MAML.The comparison uses regression tasks representing prediction problems faced by robotics systems.Source: §1 · p. 1

• Figure 1 contrasts MAML-style representation adaptation, modular recombination, and a hybrid that combines modules and adapts their initial weights.Source: Figure 1 · p. 2
• Meta-learning differs from finding good starting weights [4, 5]: this work finds modules for later structural combination, distinguishing structural from parameter change.Source: §2 · p. 2
• Neural module networks recombine modules when the input signals a suitable decomposition [9]; this work targets settings where such structural cues are insufficient.Prior work trained structure controllers with reinforcement learning [18], or used similar frameworks for interpretability [19] and robotic-task generalization [20].Source: §2 · p. 2
• Prior structured-network approaches include fixed robot/task composition [21], master-policy sub-policies [22], PATHNET [23], and soft layer ordering or DAG routing [24] [25].The paper presents a general framework that includes the latter two compositional schemes as sub-cases.Source: §2 · p. 2

3 Modular meta-learning

• The setting uses supervised input-output tasks, with the aim of finding a hypothesis that has low expected loss on unseen data from a new task.Each task is a joint distribution over input-output pairs and supplies training and test data.Source: §3 · pp. 2–3

3.1 Structured hypotheses

• A structured hypothesis combines basis modules through a compositional scheme, ranging from single modules and fixed compositions to weighted ensembles and function-composition trees.The offline phase learns module parameters; the online phase selects a structure for the new task, with optional parameter tuning.Source: §3.1 · p. 3

3.2 Learning algorithm

• BOUNCEGRAD addresses the mixed continuous-discrete optimization by combining simulated-annealing searches over structures with gradient descent over module parameters.The objective is difficult because it has many local optima.Source: §3.2 · p. 4, §3.2.2 · p. 4

3.2.1 Meta-test learning phase

• At meta-test time, parameters are fixed while simulated annealing searches for a structure, accepting improvements and sometimes worse moves to avoid local optima.The temperature schedule is practically effective but decreases too quickly to satisfy theoretical convergence guarantees.Source: §3.2.1 · p. 4

3.2.2 Meta-learning phase

• During meta-learning, BOUNCEGRAD optimizes parameters against an expectation over annealing-sampled structures, smoothing the search objective while temperature cools.Gradient updates adjust module parameters; as temperature falls, modules can specialize for the roles they are selected to play.Source: §3.2.2 · pp. 4–5, Figure 2 · p. 5

3.3 Parameter tuning in online phase

• MOMA applies an MAML-style error after gradient adaptation and takes additional parameter steps once ONLINE has selected a structure.The optimized parameters are obtained by a gradient update on the task data.Source: §3.3 · p. 6

4 Experiments

• Experiments cover function prediction, object pushing, and skeleton prediction, comparing pooled, MAML, modular BOUNCEGRAD, and modular-plus-adaptation MOMA under matched network shapes.Source: §4 · p. 6
• BOUNCEGRAD avoids MAML’s meta-training inner-loop gradients and parallelizes passes across tasks, though structural search generally makes online training slower, taking at most 10 seconds in these examples.A structure requires only a few integers to store, rather than a full set of weights.Source: §4 · p. 6

4.1 Functions

• On function tasks, MAML outperforms BOUNCEGRAD on sine, MOMA improves on both, and BOUNCEGRAD wins with little data on sums while BOUNCEGRAD and MOMA improve on MAML with 16 examples.BOUNCEGRAD’s learned modules nearly recover the function primitives despite having no information about them.Source: Table 1 · p. 7, §4.1 · p. 7, Figure 3 · p. 7

4.2 Learning to model results of pushing actions

• For pushing, MOMA performs best on previously encountered objects and BOUNCEGRAD outperforms MAML, while all three methods perform roughly equivalently on unknown objects.Source: §4.2 · p. 8
• Shared-module patterns cluster pushing objects by shape and show no surface dependence, while motion patterns track action differences rather than subjects.Source: §4.2 · p. 8, Figure 4 · p. 8

4.3 Predicting skeleton configurations

• For skeleton prediction, BOUNCEGRAD and MOMA perform best on known activities; on unknown activities none performs very well, though MOMA outperforms the others.Source: §4.3 · p. 8

4.4 Conclusion

• Modular composition transfers competence and exposes domain structure; the authors view combinatorial generalization as promising for scaling to many tasks and continual learning without forgetting learned concepts.Source: §4.4 · p. 8

A More insight into the difficulty of meta-training

• The problem is circular: the best structure depends on good modules, while training modules requires knowing their roles.With structures fixed, parameters can be optimized by gradient descent; with modules fixed, structure selection is a search problem.Source: A More insight into the difficulty of… · p. 11
• Greedy optimization of structure risks premature optimization and local optima, motivating an objective that lets modules and structures adapt gradually.Figure 5 contrasts local steps that never update the blue module with a smoothed criterion that eventually does.Source: A More insight into the difficulty of… · p. 11, Figure 5 · p. 11

B More results on functions dataset

• The authors claim to find the basis set by plotting each of 16 functions alongside its closest learned module.The claim allows that some modules have no close-by function because there are 20 modules for 16 basis functions.Source: B More results on functions dataset · p. 11

C Experimental details

• The functions dataset includes 16 listed functions, including absolute value, trigonometric functions, sinc, square, tanh, and identity.The passage says datasets were created by picking pairs of functions, but its description is incomplete.Source: C Experimental details · p. 11
• POOLED and BOUNCEGRAD used 1000 rather than 500 epochs on MIT push and Berkeley, yet took less time because they trained four times faster.Learning rates and epochs were generally the same across methods; MAML used five inner updates and a 0.001 step size.Source: C Experimental details · p. 11
• All 16 basis functions were recovered, with all but the function near zero close to a learned module.The basis functions were never seen alone; the learned modules are distinct except for row 2, columns 3 and 4.Source: Figure 6 · p. 12
• Datasets sharing a function generally show a dominant diagonal in module sharing, near a predictable 50% rate for most functions.Exceptions involve sinc(x), |x|, and the relation between arctan x and 3√x.Source: Figure 7 · p. 13
• The learned set contains 20 modules: 4 useless, 2 encoding |x|, 13 encoding one function, and 1 encoding both 3√x and arctan(4*x)/π.Source: Figure 8 · p. 13