Interleaved Multitask Learning with Energy Modulated Learning Progress

1. Introduction

• Task order affects learning and transfer, and human learners switch tasks dynamically rather than following a static schedule.The introduction frames task switching as a response to environmental and cognitive constraints.Source: §1 · p. 2
• The approach draws on learning progress as a signal of improvement and on energy efficiency as a constraint on task selection.The introduction relates these ideas to human learning and brain energy efficiency.Source: §1 · pp. 2–3
• Energy-modulated learning progress reduces neural energy expenditure without a significant drop in learning efficiency.The experiments also report improved learning performance across tasks with suitable task switching.Source: §1 · p. 3
• The framework combines learning-progress and neural-energy task arbitration with shared attention layers for bidirectional skill transfer.It is evaluated on effect-prediction tasks in a simulated robotic environment.Source: §1 · p. 3

2.1. Multi-task Learning

• Multi-task learning shares knowledge across tasks; prior work addresses representations and negative interference, while this paper selects one task per step through dynamic arbitration.The paper contrasts this schedule with approaches that train tasks jointly or sequentially.Source: §2.1 · pp. 4–6
• Adaptive loss weighting, gradient balancing, Pareto optimization, and mixture-of-experts methods address task balance, but generally assume simultaneous or sequential schedules.The proposed arbitration instead uses learning progress and energy consumption to choose a single task at each training step.Source: §2.1 · pp. 5–6

2.2. Interleaved Learning in Humans

• Human interleaving is associated with improved problem solving, retention, and transfer, while machine-learning research on interleaved schedules remains limited.Prior work also reports memory retention effects in standard neural networks when tasks are interleaved.Source: §2.2 · pp. 6–7

2.3. Intrinsic Motivation and Learning Progress

• Learning progress has guided exploration, region, and goal selection; this study applies it at the task level and combines it with energy consumption.The paper distinguishes task-level arbitration from prior uses of learning progress for exploration or curriculum selection.Source: §2.3 · pp. 7–8

2.4. Energy Conservation in the Brain

• Research on effort discounting and cognitive energy costs motivates the paper's energy-efficient task-selection mechanism.The cited background describes people as tending to avoid highly demanding tasks when given a choice.Source: §2.4 · p. 8

2.5. Effect Prediction and its Applications

• Robotic effect prediction maps actions to subsequent states and supports manipulation, planning, and learned dynamics, but cited work did not address multi-task learning or intrinsic-motivation-guided action selection.The paper positions its approach at that intersection.Source: §2.5 · pp. 8–10

3. Problem Statement

• The framework seeks to dynamically arbitrate learning across tasks to improve overall learning and skill-transfer performance.The stated objective is to minimize average prediction error over the tasks.Source: §3 · p. 10
• When a task is selected, it is engaged for its allocated interactions and used for execution, data collection, and learning.The selected task's experience is represented as input-output pairs.Source: §3 · p. 10
• After gathering a minibatch, the agent updates the selected task's parameters and shared parameters, continuing online learning to minimize prediction error across tasks.The objective minimizes total loss across tasks.Source: §3 · pp. 10–11
• Each task learns an action-effect prediction function mapping a state-action pair to the resulting effect.The prediction is expected to approximate the actual effect.Source: §3 · p. 11

4.1. Neural Network Architecture

• The architecture combines task-specific state and action projections, task-specific encoders and effect decoders, and shared encoding and attention components.Only the selected task's parameters are updated during its training step.Source: §4.1 · pp. 11, 14
• Task-specific state and action projections map inputs to fixed-dimensional spaces for shared processing across tasks.The state projection is linear in this work.Source: §4.1 · p. 12
• The shared encoder extracts general features, while task-specific encoders capture state knowledge unique to each task.The task-specific encoders process the shared representation.Source: §4.1 · pp. 13–14
• The active task's representation supplies the attention query, while representations from all tasks supply keys and values to inform that task.A training flag marks whether each representation belongs to the active task.Source: §4.1 · pp. 14–15
• The active task's attention output and projected action are used by its task-specific decoder to predict the resulting effect.Only the selected task's decoder produces the output.Source: §4.1 · pp. 15–16

4.2. Task Arbitration

• Task arbitration selects the next task using recent learning progress, or a score that discounts progress by recent neural energy cost.Progress is derived from the recent error trend; an exploration rate gives other tasks a chance to be selected.Source: §4.2 · pp. 16–17
• The energy-modulated score balances learning gains and energy efficiency, favoring tasks with high progress and low recent energy costs.The sensitivity constant k controls the influence of energy on the combined score.Source: §4.2 · p. 18

5. Experiments and Results

• Experiments assess learning efficiency, final task accuracy, and knowledge transfer while comparing learning-progress selection with energy-aware task arbitration.IMTL-EMLP extends IMTL-LP by incorporating energy consumption into task selection.Source: §5 · p. 19

5.1. Simulation Environment

• A simulated UR10 arm with a Franka Panda end effector interacts with six object configurations randomly placed on a PyBullet tabletop.The objects are a sphere, cube, square prism, and cylinder; the latter two can be vertical or horizontal.Source: §5.1 · p. 19

5.2. Effect Prediction Tasks

• The robot predicts action effects in Push, Hit, and Stack tasks, with Stack representing two objects and 36 possible object-pair combinations.Effects are changes between pre-action and post-action object states; Figure 2 depicts object trajectories for Push and Hit and the end-effector trajectory for Stack.Source: §5.2 · pp. 20–21, Figure 2 · p. 20

5.3. Compared Models and Baselines

• IMTL-LP is evaluated against single-task learning, random task selection, and blocked training across all six task orders.The blocked baseline uses uninterrupted task blocks and tests each permutation of the three tasks.Source: §5.3 · pp. 21–22
• SINGLE uses dedicated task-specific modules without shared parameters, while IMTL-RAND retains the shared architecture but selects each task with equal probability.The shared architecture in SINGLE is adapted to maintain structural comparability with the multitask setup.Source: §5.3 · p. 22
• IMTL-EMLP extends IMTL-LP with energy sensitivity, and variants examine how coefficient k changes the accuracy–computational-cost balance.The models are compared alongside the baselines and IMTL-LP.Source: §5.3 · p. 22

5.4. Network Architecture and Initialization

• All models train for 3,000 epochs with learning rate 0.0001, batch size 100, and hidden dimension 4.Training uses AdamW with AMSGrad and ReLU activations in two-layer encoder and decoder modules.Source: §5.4 · p. 23
• Robot-environment interactions are cached before learning, avoiding repeated simulator calls during training.The reported experience cache size is approximately 10K.Source: §5.4 · p. 23
• IMTL-LP reduces prediction error faster than SINGLE and IMTL-RAND, while random selection performs slightly better than single-task learning.Figure 3 compares isolated learning, random selection, and LP-based selection.Source: Figure 3 · p. 23

5.5. LP-based Interleaved Learning (IMTL-LP) Results

• IMTL-LP reduces prediction error faster than SINGLE and IMTL-RAND, outperforming the baselines on Push and Stack, with a larger advantage on Stack.For Hit, improvement is reported around learning steps 400–1000; Figure 4 presents the task-specific comparison.Source: §5.5 · p. 24, Figure 4 · p. 24
• IMTL-LP achieves higher Stack performance than IMTL-RAND despite receiving fewer total training iterations for that task.The authors report that total training time allocated is not a direct determinant of task learning performance.Source: §5.5 · p. 25, Table 2 · p. 25

5.6. Effect of Network Complexity on Learning Performance

• IMTL-LP outperforms SINGLE and IMTL-RAND at low and medium network complexity; at high complexity, model performance differences further decrease.The compared configurations have 800, 2,000, and 5,200 parameters, and the figure reports midtraining performance.Source: Figure 5 · p. 25, §5.6 · p. 26

5.7. Interleaved vs. Blocked Learning

• Across all six task orders, IMTL-LP outperforms blocked learning; BLOCK shows error spikes at task-switch boundaries and varies with training order.The rightmost plot region shows performance after both models have seen all tasks.Source: §5.7 · pp. 26–27, Figure 6 · p. 27

5.8. Modulation of LP with Computational Energy (IMTL-EMLP)

• IMTL-EMLP discounts learning progress by energy cost, and the comparison sets its sensitivity constant to k = 1.The section compares this setting with IMTL-LP and baseline models, then varies k to assess prediction error and energy expenditure.Source: §5.8 · p. 27, Figure 7 · p. 28
• IMTL-EMLP and IMTL-LP converge faster and reach lower prediction errors than SINGLE and IMTL-RAND, with IMTL-EMLP slightly better through 1000 epochs.After 1000 epochs, IMTL-EMLP and IMTL-LP show similar performance.Source: §5.8 · p. 28, Figure 7 · p. 28
• At the training midpoint, IMTL-LP has the lowest prediction error overall, while IMTL-EMLP-K=1.2 closely follows.Figure 8a reports MAE at approximately 1500 training epochs.Source: §5.8 · pp. 28–29, Figure 8 · p. 29
• At k = 1.2 and k = 1, IMTL-EMLP offers a favorable trade-off, reducing energy consumption with only a small sacrifice in prediction accuracy.Lower k reduces energy use but degrades prediction performance; k = 0.4 consumes the least energy, almost matching SINGLE, while IMTL-LP is most energy-intensive.Source: §5.8 · p. 29, Figure 8 · p. 29

5.9.1. Self-Attention and the Task Flag Ablations

• The full model with both shared attention and task flags outperforms the ablated configurations from early training, while removing both yields the poorest performance.The four configurations are the full model, without task flags, without attention, and without both components.Source: §5.9.1 · pp. 30–31

5.9.2. Object-wise Skill Transfer Analysis

• Transfer is measured after training as ∆L = Lablate − Lfull: a positive ∆L indicates positive transfer from the removed source task to the target task.The analysis repeats IMTL-LP training and testing across 10 random seeds and reports mean ablation impact and standard deviation.Source: §5.9.2 · p. 31, Figure 10 · p. 32
• For sphere interactions, Stack→Hit has the highest ∆L at 0.86, while Stack→Push has the lowest at 0.22; rollable objects generally show lower transfer.The horizontal cylinder follows a similar Stack-to-Hit-over-Push pattern.Source: §5.9.2 · pp. 32–33
• Push→Hit transfer varies with object orientation: it is 0.55 for a vertical cylinder and 0.28 for a vertical prism, versus 1.55 and 3.26 when horizontal.The passage attributes these differences to affordance and orientation.Source: §5.9.2 · p. 33
• In Stack interactions, Hit is strongly utilized when the base object remains intact, with ∆L around 3.7–5.2; Push contributes a small non-negative amount.Sphere and horizontal-cylinder bases show patterns similar to the single-object interactions.Source: §5.9.2 · p. 33
• Push largely learns its effect-prediction task independently early on, while Hit and Stack co-learn and benefit from each other throughout training.Ablation-effect standard deviation is about 14%, and the authors conclude that task-agnostic selection shapes an emergent transfer landscape for these tasks.Source: §5.9.2 · pp. 33–34

6. Conclusion

• The framework combines dynamic task selection based on learning progress and neural energy cost with a shared encoder-decoder architecture for bidirectional skill transfer.IMTL-EMLP discounts learning progress by the energy needed to achieve it.Source: §6 · p. 34
• In simulated robotic manipulation, learning-progress arbitration improves performance and convergence speed over random interleaving, single-task learning, and blocked training, while facilitating beneficial transfer.The conclusion also reports complementary roles for attention and task flags.Source: §6 · p. 34
• Energy-aware selection enables a tunable trade-off between learning accuracy and resource efficiency, which the authors identify as valuable in energy-constrained settings.The evaluation uses a controlled simulation with a limited number of tasks; proposed future extensions include real-world robotics and larger, more diverse task sets.Source: §6 · pp. 34–35

Appendix A. Task Settings and Hyperparameters

• The multi-task model increases the number of neurons in F so both architectures have the same total number of parameters.Table A.1 lists network architecture details for both models.Source: Table A.1 · p. 36

• Figure B.1 tracks task-selection regimes during training using a sliding window of 50 and a step size of 10.Source: Figure B.1 · p. 37