Residual Pathway Priors for Soft Equivariance Constraints

1 Introduction

• Equivariance reduces meaningless shortcut features by ruling out hypotheses tied to irrelevant coordinate systems, node orderings, or set orderings.The introduction situates equivariance among inductive biases used for structured data.Source: §1 · p. 1
• Exact constraints can fail when reality includes non-local interactions, preferred orientations, discontinuities, wind, or uneven terrain.These examples motivate priors that admit uncertainty about whether assumed symmetries are exact.Source: §1 · p. 1
• RPPs convert hard architectural constraints into soft priors, favoring structured solutions without constraining model flexibility.The approach is motivated by the need for interpretable priors over high-level properties such as locality, independence, and symmetry.Source: §1 · pp. 1–2
• RPPs combine constrained-model inductive biases with MLP-like flexibility, favoring equivariant solutions while supporting deviations from symmetry.This positions RPPs between weakly biased flexible models and strongly constrained equivariant models.Source: §1 · p. 2
• The paper reports that RPPs perform well under exact, approximate, or misspecified symmetries and improve model-free reinforcement learning on approximate Mujoco symmetries.The contributions also include converting hard constraints into soft equivariance priors and applying them broadly.Source: §1 · p. 2

2 Related Work

• Several studies address the inability of strictly equivariant models to fit data by breaking or weakening symmetries, such as through coordinate inputs, subgroup equivariance, or axis indicators.These examples motivate methods that preserve useful structure without enforcing symmetry everywhere.Source: §2 · p. 3
• RPP differs from ConViT by providing a general soft-prior mechanism applicable beyond locality in self-attention, including other layers, symmetries, and architectural constraints.ConViT instead targets a soft locality bias specifically within Vision Transformers.Source: §2 · p. 3
• Residual modeling also combines a restrictive base model with a flexible correction, appearing in ResNets, residual reinforcement learning, and physics- or biology-informed dynamical models.RPP draws on this broader pattern while applying it to architectural restriction priors.Source: §2 · p. 3
• Prior equivariant methods often require exact symmetries, whereas RPP targets the complexity of real data that can violate those symmetries.Related approaches include regularization, data augmentation, and theoretical work on approximate MDP symmetries.Source: §2 · p. 3

3 Background

• A symmetry group is a set of invertible transformations under which an object is considered the same, including reflections and rotations.Such transformations can act on vectors, images, robot states, classification problems, or MDPs.Source: §3 · p. 3
• A representation maps each group element to a matrix that specifies how vectors or embedded objects transform under the symmetry.Different components can use different representations within a composite structured object.Source: §3 · pp. 3–4
• Composite representations combine component-specific transformations, allowing velocity vectors, orientation quaternions, joint angles, and control torques to be modeled together.The components are combined through block-diagonal composition of their representations.Source: §3 · p. 4
• A function is equivariant when transforming its input and then applying the function equals applying the function and then transforming its output.Constraining models this way can improve learning and generalization when the data relationship has the symmetry.Source: §3 · p. 4

4 Residual Pathway Priors

• RPP layers sum a restrictive layer encoding the architectural constraint with a generic flexible layer that explains residual complexity.The same construction can encode priors for linearity, locality, Markovian structure, or equivariance.Source: §4 · p. 4
• The RPP construction is motivated by residual networks, where a simpler identity path receives higher prior likelihood and the residual function models the remaining difference.This provides an interpretation of residual pathways as prior-weighted alternatives with different complexity.Source: §4 · p. 4
• With suitable priors, MAP optimization favors the structured pathway and uses the flexible pathway only for what the structured model cannot explain.The resulting model retains the expressivity of fully flexible layers while inheriting the inductive bias of constrained layers.Source: §4 · p. 4
• EMLP supplies the hard equivariant layer by solving linear constraints and parameterizing the solution subspace with an orthonormal basis Q.RPP replaces direct equivariant parameterization with an equivariant component plus an unconstrained component.Source: §4 · p. 5
• RPP-EMLP assigns higher prior mass to equivariant solutions while retaining support for non-equivariant solutions, enabling accommodation of approximate or incorrectly specified symmetries.The method can also be applied to convolutional, graph-convolutional, and subgroup-equivariant architectures.Source: §4 · p. 5

5 How and Why RPPs Work

• The experiments cover settings where constraints help, cannot fully describe the problem, or are misspecified.This scope tests whether RPPs preserve useful inductive biases without enforcing unsuitable structure.Source: §5 · p. 5

5.1 Dynamical Systems and Levels of Equivariance

• RPP-EMLP matches EMLP under exact symmetries, outperforms both EMLP and MLP under approximate symmetries, and matches MLP when symmetries are misspecified.These experiments use inertia and double pendulum datasets spanning exact, approximate, and intentionally incorrect symmetry settings.Source: §5.1 · pp. 6–7

5.2 Prior Levels of Equivariance

• Broad priors over equivariant weights produce better and more stable modified-inertia test MSE than overly concentrated equivariant priors.The modified inertia setting breaks perfect equivariance through external forces, and excessive regularization of equivariant weights causes instability.Source: §5.2 · p. 7

5.3 Posterior Levels of Equivariance

• Equivariance error is measured by relative disagreement between transforming the prediction and predicting from the transformed input.The metric is defined as EquivErr(f, x) = RelErr(ρ2(g)f(x), f(ρ1(g)x)).Source: §5.3 · p. 7
• RPP posteriors converge toward higher equivariance error on partially symmetric modified inertia than on perfectly symmetric inertia.Deep ensembles of 10 models show the posterior adapting its equivariance level during training.Source: §5.3 · p. 7

5.4 RPPs and Convolutional Structure

• RPP-Conv performs similarly to the model with the correct complexity on CIFAR-10 and four UCI regression tasks, whether convolutional structure helps or not.The model retains MLP flexibility while assigning larger prior variance to the convolutional subspace.Source: Table 1 · p. 8, §5.4 · p. 8

6 Approximate Symmetries in Reinforcement Learning

• Mujoco state and action spaces contain diverse components and symmetries, so exploiting representations can expose structure that standard RL methods otherwise ignore.The environments include exact reflections, approximate front-back reflections, D4 leg symmetries, and continuous rotations that external forces or rewards can break.Source: §6 · p. 8

6.1 Approximate Symmetries in Model Free Reinforcement Learning

• Figure 5 compares maximum average reward over training for RPP-SAC and SAC, with means and one-standard-deviation regions computed over four trials.The caption states that incorporating approximate symmetries improves model-free RL agent efficiency.Source: Figure 5 · p. 9
• RPP-EMLP consistently improves model-free RL data efficiency when exact and approximate equivariances are incorporated into both policy and Q-function, except on Humanoid-v2 where it destabilizes SAC.Using the RPP-EMLP for the policy alone is not very helpful with Actor Critic.Source: §6.1 · pp. 8, 10

6.2 Better Transition Models for Model Based Reinforcement Learning

• Table 3 reports transition-model relative error averaged over 10-, 30-, and 100-step rollouts, with error bars from three random seeds.It also reports equivariance error as a geometric mean over the 100-step rollout.Source: Table 3 · p. 10
• RPP transition models outperform MLPs on Swimmer and Hopper, especially for long rollouts, while retaining a smaller but non-negligible equivariance error.The evaluation uses relative state error across rollout horizons, and the transition dataset contains 50,000 replay-buffer transitions.Source: §6.2 · p. 10

7 Limitations

• RPP-EMLP requires error-prone identification of component meanings and can have unbounded Lipschitz constants, potentially destabilizing training when inputs are poorly normalized.These factors may contribute to the instability observed on Humanoid-v2.Source: §7 · p. 10

8 Conclusion

• RPPs match constrained equivariant models under exact symmetries, match unstructured MLPs when specified symmetries are absent, and outperform both under approximate symmetries.The conclusion highlights applications to complex state and action spaces in reinforcement learning.Source: §8 · p. 11
• RPPs aim to express high-level assumptions directly rather than relying on difficult-to-interpret low-level parameter characteristics.The authors connect this goal to lowering the cost of incorporating prior knowledge while accommodating data that violate expectations.Source: §8 · p. 11

Appendix Outline

• The appendix covers potential negative impacts, policy-only RPP-EMLP experiments, datasets and methodology, and Mujoco state, action, and representation details.These topics are organized across Sections 7, B, C, D, and E.Source: Appendix Outline · p. 15

A Potential Negative Impacts

• Model-free RL policies learned in Mujoco may transfer poorly to real robots, and deploying neural controllers can cause injury when failures affect robots or humans.The authors therefore call for responsible and cautious deployment.Source: A Potential Negative Impacts · p. 15

B Benefit of Equivariant Value Functions

• Using an RPP for the value function has a stronger performance impact because an RPP policy can otherwise fit non-equivariant parts of a standard MLP critic.The policy update learns from the value function, making the critic's equivariance relevant to the policy's learned behavior.Source: B Benefit of Equivariant Value Functions · p. 15

• The initial and final reinforcement-learning experiments used approximately 500 GPU hours, with dynamical-systems, CIFAR-10, and UCI experiments requiring 200 additional GPU hours.Source: C Experimental Details · p. 15

• The windy pendulum adds a constant wind force with ϵ = 0.01, breaking SO(2) symmetry about the z axis.Source: §C.1 · p. 15
• The models use three-layer, 128-hidden-unit Hamiltonian neural networks trained on five-step ODE rollouts with Δt = 0.2 and mean squared error.Source: §C.1 · p. 16
• The inertia experiments modify the target with an additional z-axis-dependent term, producing a deliberately non-equivariant matrix prediction task.Source: §C.1 · p. 16
• The inertia datasets use 1,000 training and test examples and train for 500 epochs with Adam at learning rate 0.003.Source: §C.1 · p. 16

C.2 Image and UCI experiments (5.4)

• C.2 Image and UCI experiments use convolutional networks and equivalent MLPs, trained with Adam and cosine learning-rate schedules.CIFAR-10 models have nine convolutional layers; UCI models have three.Source: §C.2 · pp. 16–17

C.3 Model Free RL

• C.3 Model Free RL required reduced critic-moving-average speeds for SAC stability, and RPP critics could not train reliably on Humanoid.Source: §C.3 · p. 17, Table 4 · p. 17

C.4 Transition Models for Mujoco

• C.4 Transition Models for Mujoco predict state changes from states and actions using xt+1 = xt + NN(xt, ut).RPP transition models require about 45 minutes to train versus 15 minutes for standard MLPs.Source: §C.4 · p. 17

D Mujoco State and Action Representations

• D Mujoco State and Action Representations define group representations using permutations, rotations, scalar components, and pseudoscalar components.The construction converts Humanoid and Ant quaternions to rotation matrices and groups left/right pairs for Walker2d and Swimmer.Source: D Mujoco State and Action… · p. 17, Table 5 · p. 18

E Mujoco State and Action Spaces

• E Mujoco State and Action Spaces documents the meanings of environment state vectors and action spaces for the locomotion tasks.Source: E Mujoco State and Action Spaces · p. 18, Table 6 · p. 18, Table 7 · p. 18