Learning Options for Compositional Motor Control with Adapter Banks

Sreejan Kumar; Marcelo Mattar; Lea Duncker

Source-linked AI summary

Learning Options for Compositional Motor Control with Adapter Banks

Sreejan Kumar, Marcelo Mattar, Lea Duncker

arXiv:2609.17042v1Sep 15, 2026cs.LGcs.ROq-bio.NC

TL;DR

Flexible motor control requires reusable primitives that can be recombined into novel sequences, yet prior low-rank-adapter theories do not specify how to learn them end-to-end. This paper learns discrete adapter-selected skills around a shared recurrent core, yielding emergent low-rank structure and out-of-distribution motor-sequence transfer. The approach improves generalization error over a task-input-conditioned multitask baseline by up to an order of magnitude.

Problem
Existing low-rank perturbation theories leave open how to learn compositional motor primitives end-to-end from demonstrations for out-of-distribution transfer.
Method
A shared recurrent core is modulated by residual adapters selected through discrete latent codes inferred from compound expert demonstrations.
Results
The adapters develop emergent low-rank perturbations, task representations occupy separated recurrent subspaces, and novel sequences outperform a task-input-conditioned multitask baseline by up to an order of magnitude.
Takeaways & Limitations
A frozen high-level policy can sequence learned adapter options to produce novel out-of-distribution movements in closed-loop biomechanical control.
Takeaways & Limitations
Training uses task-identity supervision for segment-1 representations, so unsupervised code and boundary learning may produce a different adapter organization.

Abstract

from arXiv · show

Learning flexible motor primitives is a hallmark of skilled motor control. Recent neuroscience theory proposes that motor primitives may be implemented as low-rank perturbations of a shared recurrent network, but leaves open how such a system is learned. We translate this principle into a novel architecture for learning motor skills end-to-end: a shared recurrent core modulated by a bank of residual adapters, each selected by a discrete latent code. Trained on closed-loop biomechanical control, the adapters develop emergent low-rank perturbations of the recurrent dynamics despite no architectural rank constraint, placing task representations in disparate subspaces of the shared core network. A simple high-level policy over the learned options, optimized while the whole network is frozen, sequences the low-rank adapters to produce novel out-of-distribution movements. We demonstrate the ability to generalize to novel motor sequences within the closed-loop control setting, improving on the generalization error of a task-input-conditioned multitask baseline by upto order of magnitude.

1 Introduction

Skilled motor control depends on reusable primitives that can be recombined without interference, but learning such compositional options end-to-end remains unresolved. The paper proposes shared recurrent dynamics modulated by learned adapters and demonstrates out-of-distribution transfer.

The paper targets compositional motor control: discovering reusable behavioral building blocks that can be recombined into new tasks without catastrophic interference.
Low-rank perturbations of a shared recurrent network are proposed as a substrate for compositional computation, paralleling adapter-based parameter-efficient finetuning.
Prior work establishes low-rank perturbations as a compositional substrate but does not explain how to learn such architectures end-to-end from demonstrations for out-of-distribution transfer.
The proposed architecture uses a shared recurrent core, residual adapter bank, and discrete latent codes inferred from expert demonstrations.
Training yields emergent low-rank adapter perturbations, separated task subspaces, and novel-sequence transfer that outperforms a task-input-conditioned multitask baseline by up to an order of magnitude.

2 Low-rank Adapters via Thalamocortical Architectures in Neuroscience

Thalamocortical-loop theory explains motor primitives as low-rank perturbations that alter shared cortical dynamics. The paper uses this framework as motivation for learning and selecting such motifs in nonlinear motor-control systems.

Thalamocortical channels combine cortical readouts and projections to perturb a shared cortical recurrent network.
In linear dynamics, tuning these perturbations can move effective-system eigenvalues and thereby change activity behavior such as oscillation frequency.
The teacher–student framework instantiates the learning problem with a closed-loop arm controller, an encoder, and an adapter-selecting decoder.
Sequentially engaging different channels provides a mechanism for constructing diverse movements from a shared recurrent network.
The theoretical framework specifies perturbation design and basal-ganglia selection, but leaves general learning and motif-selection mechanisms open.

3 Learning Adapters for Flexible Motor Control

CABRA learns discrete latent skills from compound demonstrations and uses them to route residual adapters around a shared recurrent core. The encoder supplies segmentation and code objectives while the decoder executes closed-loop control through adapter-modulated dynamics.

The student learns to segment expert demonstrations into discrete latent skills and uses the inferred decomposition to select adapters for a shared recurrent core.
Exactly one of ten latent codes is active per segment, with five codes reserved for extension and five for retraction; task assignments and code reuse are learned.
The decoder shares the encoder’s core, freezes recurrent-parameter updates during decoder loss, and trains adapters, readout, and input projection through closed-loop rollouts.
Adapters are unconstrained in rank, perturb hidden states residually, and become sharply selected within segments with brief soft mixing near boundaries.
The decoder uses proprioceptive feedback and differentiable physics, while removing adapters or collapsing them to one shared modulator prevents effective learning.
End-to-end training combines trajectory, KL, activity, muscle-effort, task-identity, discriminator, and excitation-matching objectives.
Training uses compound trajectories only, so reusable half-movement primitives must emerge rather than being explicitly demonstrated.

4 Results

The student reproduces compound movements, discovers task-structured adapter codes and emergent low-rank perturbations, and separates task representations while retaining a shared recurrent core. A frozen high-level policy then composes learned options into novel trajectories, outperforming the teacher baseline by roughly an order of magnitude.

4.1 Compound performance and meaningful code routing: The student reproduces all five trained compound movements with hand L1 errors of approximately 10^-2 m, despite never receiving the teacher’s rule input.Reach is essentially exact, while figure-eights show closed-loop wobble but retain their general shape.
4.1 Compound performance and meaningful code routing: Extension codes specialize one-to-one by task, whereas retraction codes are partially shared and have weaker effects on in-distribution trajectory error.Natural extension codes are z2 for Reach, z0 for Circle CW, z3 for Circle CCW, z1 for Figure-8, and z4 for Figure-8 Inverse; retraction codes select z5 or z7 and swapping to z9 causes only modest changes.
4.2 Emergent Low-Rank Structure: Without an imposed rank constraint, individual adapter perturbations converge to effective rank ∼5–6 while the recurrent core’s hidden-state manifold reaches ∼30 dimensions.The adapter bank therefore develops low-dimensional perturbations relative to the available H = 256 dimensions.
4.3 Shared core, separated representations: Compared with the teacher’s shared geometry, the student places task representations in separated subspaces and organizes dynamical differences along behavioral axes.The teacher has off-diagonal principal-subspace overlap 0.69 and DSA distance 0.12, whereas the student has overlap 0.18 and DSA distance 0.25.
4.4 Compositional transfer to a novel two-petal trajectory: On novel splice-compositions, the student’s policy mixes adapter codes around transitions and preserves its advantage under hard discrete selection, with a 2.5× gap.The teacher’s optimized policy tends toward one rule at a time, limiting transition handling; the student advantage also persists on a harder out-of-distribution shape.

5 Discussion

The architecture operationalizes a neuroscience-inspired account of motor compositionality using a shared recurrent core, residual adapters, and discrete selection. Its separated task representations support compositional generalization, while task-identity supervision remains an important limitation.

5 Discussion: The shared recurrent core, adapter bank, and discrete code selection implement a cortex–thalamus–basal ganglia-inspired architecture for learning motor options.Adapters provide low-rank perturbations, while a policy selects which adapter is active.
5 Discussion: Task representations occupy highly separated recurrent subspaces while the underlying dynamics retain structured sharing organized by behavioral kinematics.
5 Discussion: The adapter library provides learned neural primitives without committing to a prespecified factor basis or generative model.
5 Discussion: Training uses task-identity supervision to organize codes, so fully unsupervised discovery of code numbers and boundaries remains unresolved.Future work must determine whether minimal-supervision training recovers biological motor primitives.
5 Discussion: The identified motor-compositionality principles may also bear on broader compositional intelligence, including cognitive flexibility.

A.1 Architecture details

The student combines a shared 256-wide LSTM with encoder heads and ten residual adapters selected by discrete, phase-specific codes. The architecture imposes no rank constraint, allowing learned perturbations to acquire their structure empirically.

A.1 Architecture details: The student uses one shared LSTM core of hidden dimension H = 256 for encoder and decoder processing.
A.1 Architecture details: The encoder receives sliced observations and teacher muscle commands, whereas the decoder receives sliced observations without muscle commands or rule input.
A.1 Architecture details: Encoder heads infer segment boundaries, sample K = 10 discrete codes with straight-through Gumbel-softmax, and predict task identity from the pooled segment-1 state.
A.1 Architecture details: Each adapter is a nonlinear residual modulator, and no rank constraint is imposed on its perturbation.The rank of the learned perturbation is analyzed empirically rather than fixed architecturally.
A.1 Architecture details: Hard phase masking reserves five codes for extension and five for retraction, while task assignments within each phase emerge during training.
A.1 Architecture details: A muscle readout maps the adapter-perturbed hidden state to six muscle excitations through a sigmoid.
A.1 Architecture details: The DIAYN discriminator predicts active codes from pooled muscle commands rather than cortical state.

A.2 Training procedure

Training jointly infers boundaries and adapter codes from demonstrations while optimizing closed-loop motor imitation and auxiliary objectives. Decoder motor gradients update adapters and readout components, but not the shared recurrent parameters.

A.2 Training procedure: The system trains end-to-end for 30,000 iterations with Adam at learning rate 10^-4, batch size 32, gradient clipping, and no weight decay.Each run took approximately 48 hours on one GTX 1080 Ti GPU.
A.2 Training procedure: The encoder reads teacher demonstrations to infer boundaries, codes, and task identity, while the decoder rolls out the arm in closed loop using body-state feedback.Student trajectories are generated afresh during training and compared with recorded teacher trajectories.
A.2 Training procedure: Decoder gradients update the input projection, adapter bank, and muscle readout, while encoder objectives update the shared recurrence through segmentation and code inference.
A.2 Training procedure: The total objective combines encoder inference, decoder closed-loop control, and auxiliary losses.
A.2 Training procedure: The encoder regularizes code distributions toward valid-subset uniform priors and boundaries toward midpoint-centered Poisson priors.
A.2 Training procedure: The decoder minimizes hand-trajectory L1 error with hidden-activity and muscle-excitation regularizers.
A.2 Training procedure: DIAYN, excitation-matching, and task-classification auxiliary losses can each be ablated without losing the architectural transfer advantage.

A.3 Task and data details

Experiments use a two-link, six-muscle arm and five compound extension–retraction trajectories. Half-task primitives are withheld, requiring any reusable decomposition to emerge from compound demonstrations.

A.3 Task and data details: The biomechanical environment is a two-link arm with six muscles controlled through the MujocoHillMuscle model.
A.3 Task and data details: The student trains on five compound trajectories: straight reach, clockwise and counter-clockwise circles, and two figure-eight variants.
A.3 Task and data details: Each training batch samples 32 reach conditions per task, and every trial lasts 200 timesteps.
A.3 Task and data details: The isolated extension and retraction primitives are never shown during training.
A.3 Task and data details: Reusable extension and retraction codes must therefore emerge by segmenting compound demonstrations rather than from explicitly demonstrated primitives.

A.4 Analysis methodology

The analysis measures adapter perturbation rank, task-subspace overlap, dynamical similarity, transfer performance, and isolated adapter behavior using frozen-network evaluations and trajectory-based analyses.

Representation analyses: Effective rank counts singular values explaining at least 95% of each adapter’s held-out cortical perturbation variance, with uncertainty from 10 bootstrap resamples.The metric is computed separately for each adapter’s perturbation Δk(ht).
Representation analyses: Principal-subspace overlap compares task activity subspaces built from top principal components explaining at least 95% of task variance, against random-subspace chance baselines.Teacher and student analyses use effective ranks of 12 and 25, respectively; trajectories pool 128 trials per task.
Dynamical analyses: Dynamical Similarity Analysis compares task dynamics after PCA reduction, delay embedding with lag p = 90, and reduced-rank DMD at rank r = 150.The protocol follows prior work and uses the euclidean DSA score with matched hyperparameters.
Transfer optimization: Transfer optimization freezes each network and learns a time-varying soft policy over its 10 task interfaces using temperature-0.5 softmax logits.The teacher uses rule-input weights, whereas the student uses adapter-code weights; optimization targets hand-trajectory L1 loss.
Qualitative analyses: Forced-code analysis overrides the encoder with one fixed one-hot code per segment to inspect each adapter’s isolated closed-loop behavior.This factors adapter behavior away from the encoder’s learned code-inference policy.

B Supplementary Figures

Supplementary experiments evaluate compositional transfer on butterfly and two-petal trajectories, test hard discrete execution, and ablate auxiliary supervision and adapter-bank structure.

Butterfly transfer: Butterfly transfer combines four unseen primitive pairings, and the student reaches hand L1 ≈0.011 versus the teacher’s ≈0.058, a ∼5× gap.The student recovers the butterfly’s figure-8-like crossing structure, whereas the teacher trajectory is qualitatively distorted.
Auxiliary-supervision ablation: The full configuration achieves hand L1 of 0.005 on two-petal transfer and 0.011 on butterfly transfer, versus teacher baselines of 0.04 and 0.058.All ablated configurations remain well below the teacher baseline, while the full model is numerically best on both trajectories.
Auxiliary-supervision ablation: The auxiliary-supervision ablation removes DIAYN alone or DIAYN plus excitation matching while keeping all other architectural and training choices unchanged.The architectural transfer advantage persists without these losses, although they provide modest refinement to transfer accuracy.
Adapter-bank ablation: A further ablation tests whether the adapter bank matters by routing all tasks through one shared adapter or removing adapters entirely.These configurations test whether task dynamics can be learned within a single shared recurrence rather than through separate residual modulators.
Hard discrete execution: Hard-discrete transfer uses argmax one-hot interface selection during the forward pass while retaining softmax gradients through a straight-through estimator.This provides a complementary test of compositional transfer under genuinely discrete policy execution.