Tunable Soft Equivariance with Guarantees

Md Ashiqur Rahman; Lim Jun Hao; Jeremiah Jiang; Teck-Yian Lim; Raymond A. Yeh

Source-linked AI summary

Tunable Soft Equivariance with Guarantees

Md Ashiqur Rahman, Lim Jun Hao, Jeremiah Jiang, Teck-Yian Lim, Raymond A. Yeh

arXiv:2603.26657v1Mar 27, 2026cs.CVcs.LG

TL;DR

Real-world data only approximately satisfies strict equivariance, and enforcing it can reduce expressiveness, motivating controllable soft-equivariant models. The paper projects weights into designed subspaces for pre-trained architectures and derives bounds on equivariance error. Across classification, segmentation, trajectory prediction, and synthetic invariant tasks, the method improves performance while reducing equivariance error, including on ImageNet.

Problem
Real-world data only approximately satisfies equivariance, while strict enforcement can reduce model expressiveness and existing soft-equivariance methods lack guarantees.
Method
The framework constructs soft-equivariant layers by projecting parameters into designed subspaces and applies them to pre-trained architectures with theoretically bounded error.
Results
The method improves task performance while reducing equivariance error across image classification, segmentation, trajectory prediction, and synthetic invariant regression, including ImageNet.
Takeaways & Limitations
The framework provides a practical way to adapt existing non-equivariant pre-trained models into soft-equivariant models with controllable softness and guarantees.
Takeaways & Limitations
The relative soft-equivariance formulation assumes ∥JF (x)∥F > 0 and ∥x∥ > 0 throughout the domain to avoid degeneracy.

Abstract

from arXiv · show

Equivariance is a fundamental property in computer vision models, yet strict equivariance is rarely satisfied in real-world data, which can limit a model's performance. Controlling the degree of equivariance is therefore desirable. We propose a general framework for constructing soft equivariant models by projecting the model weights into a designed subspace. The method applies to any pre-trained architecture and provides theoretical bounds on the induced equivariance error. Empirically, we demonstrate the effectiveness of our method on multiple pre-trained backbones, including ViT and ResNet, across image classification, semantic segmentation, and human-trajectory prediction tasks. Notably, our approach improves the performance while simultaneously reducing equivariance error on the competitive ImageNet benchmark.

1. Introduction

Strict equivariance is uncommon because real-world data only approximately satisfies it, while enforcing it can reduce expressiveness. The paper proposes projecting weights into designed subspaces to build tunable soft-equivariant models with guarantees.

Equivariance makes transformations of inputs produce predictable transformations at outputs, such as shifted segmentation masks following shifted objects.
Real-world data only approximately satisfies equivariance, and strict enforcement can reduce model expressiveness.
Existing augmentation and regularization methods lack guarantees on equivariance after training, while component-based approaches also lack guarantees.
The proposed generalized blurring filters apply to any pre-trained model, extend beyond shift equivariance, and provide bounds on equivariance error.
Experiments evaluate tunability across pre-trained ViT, DINOv2, ResNet, and Segformer backbones on classification, segmentation, trajectory prediction, and synthetic invariant regression.

2. Related Work

Prior work develops equivariant architectures and methods for discovering or adapting symmetries, but applicability to modern models remains limited. The paper positions its framework as architecture-agnostic with explicit control over equivariance.

Equivariance research has expanded from group convolutions to transformers, graph neural networks, and transformations including permutations, scaling, and reflections.
Methods that learn group representations or discover symmetries often depend on equivariant architectures or kernels defined directly on symmetry groups.
The proposed framework is architecture-agnostic, extends soft equivariance beyond exact equivariance, integrates with pre-trained vision models, and explicitly controls equivariance.

3. Preliminaries

The preliminaries define group actions, Lie-group representations, and equivariance. They establish the mathematical language for describing transformations and their local approximations.

A group representation maps each group element to a linear transformation on a vector space describing the group action.
Lie groups are smooth manifolds with smooth multiplication and inversion, while their Lie algebras are tangent spaces at the identity with a Lie bracket.
The exponential map connects Lie-algebra elements to group elements, with a real parameter controlling transformation magnitude.
Differentiating a group representation at the identity yields an infinitesimal generator describing first-order action near the identity.
A function is equivariant when transforming its input and then applying the function matches applying the corresponding output transformation.

4. Towards Soft Equivariant Networks

The framework defines relative soft equivariance and constructs invariant or equivariant layers by projecting learnable parameters into selected subspaces. Cut-off values control softness while theoretical claims bound the resulting error.

Soft equivariance relaxes exact equality into an inequality measuring transformation inconsistency by an equivariance error.
The paper proposes a relative soft-equivariance notion because absolute output differences are sensitive to the scale of F(x).
An η-soft-equivariant function bounds the normalized Jacobian-based equivariance measure by η, with η serving as the soft-equivariance constant.
Soft invariant layers: Invariant fully connected layers constrain weights with a projection derived from Lie-algebra representations, filtering directions strongly affected by group actions.
Soft invariant layers: Claim 1 bounds soft-invariance error by ηb = b√nG rG + εG, where b is the projection cut-off and εG is Taylor residual.
Soft equivariant layers: Equivariant fully connected layers use Kronecker-product representations and SVD-based projections that remove components above cut-off b.
Soft equivariant layers: Claim 2 gives ηb = b√nG d′ rG + εG for projected equivariant layers, explicitly linking the bound to cut-off, output dimension, group properties, and Taylor residual.
Efficient implementation: A Schur-decomposition alternative reduces projection construction complexity to O(max(d, d′)^3) for groups with normal Lie-algebra representations.

5. Experiments

Experiments show that the proposed soft-equivariance approach is tunable and can improve performance while reducing equivariance error across classification, segmentation, and trajectory prediction. It integrates with pretrained backbones and offers favorable runtime and projection-thresholding behavior.

Tunable softness: Compared with RPP, the method achieves comparable performance with better invariance error across rotation and roto-reflection groups.RPP introduces a non-equivariant residual branch that roughly doubles model size.
Tunable softness: The method controls equivariance error through a softness parameter, where 0 is fully equivariant and 1 is non-equivariant.The parameter adjusts the cutoff value and the fraction of basis vectors retained by the projection operator.
Image classification: On ImageNet, the method is best across evaluated backbones for accuracy and invariance error, improving both simultaneously without a performance–invariance trade-off.It achieves the best aAcc and cAcc in every case, matches or improves iErr, ties ViT, and outperforms DINOv2 and ResNet on iErr.
Semantic segmentation: On PASCAL VOC segmentation, the method improves aIoU and cIoU while reducing eErr across settings, with mIoU increasing for ViT and SegFormer but mildly dropping for DINOv2.The canonicalization baseline significantly reduces mIoU and worsens consistency.
Human trajectory prediction: For trajectory prediction, the method outperforms standard and equivariant transformer baselines on cADE and cFDE across most scenes, achieving the best results in four scenes.Full equivariance in EqAuto does not necessarily produce better performance.
Ablation studies: Schur decomposition computes the projection operator in under one second, whereas SVD reaches nearly 15 minutes at input size 14 × 14.Smooth projection cutoffs also improve segmentation performance and lower eErr compared with hard thresholding.

6. Conclusion

The framework provides tunable soft invariant and equivariant layers with theoretically bounded error, adapts pre-trained models, and improves performance while reducing equivariance error across several tasks.

The framework designs soft invariant and equivariant layers with tunable, theoretically bounded equivariance error.
The method adapts existing non-equivariant pre-trained models to soft-equivariant ones.
Experiments cover image classification, segmentation, trajectory prediction, and synthetic O(5) invariant tasks.
The approach improves task performance while reducing equivariance error, including gains on ImageNet.
Visualizations show weights and features transitioning smoothly between perfectly equivariant and fully non-equivariant behavior as softness changes.

C.2. Details on Schur Equivariance prediction

Schur equivariance projection enforces blockwise constraints based on representation equivalence, while soft projection replaces a sharp cutoff with a smooth transition; the resulting metric is scale-invariant and empirically stable.

Equivariant transformed weights can be nonzero only between representation blocks sharing eigenvalues; other blocks must be zero.
A 3 × 4 equivariant design reduces learnable parameters from 12 in a dense layer to 2 while guaranteeing the desired symmetry.
For a 4D-to-3D map, Schur structure reduces learnable parameters from 16 to 4 by retaining diagonal-block parameters and enforcing zeros elsewhere.
For non-commutative generators, composing per-generator Schur projections is an approximation rather than the exact projection operator.
A smooth cutoff controls decay through a hyperparameter and avoids the sharp-cutoff behavior associated with undesirable time-domain ripples.
The relative equivariance metric is unaffected by local rescaling, and ResNet-18 experiments show that it neither explodes nor collapses.

D.2.1. Using SVD

The SVD construction identifies exact and approximate equivariant weight directions through singular values, then projects weights according to a cutoff; Schur decomposition yields the same operator more efficiently.

The 3-dimensional null space produces a 3-parameter family of exactly equivariant weights.
For b = 1.5, the projection retains seven singular vectors with σ ∈ {0, 1}, yielding a 7-dimensional projected subspace.
The retained basis contains three exactly equivariant directions and four directions that mildly break equivariance by coupling the xy-plane to the z-axis.
With b = 1.5, blocks with λT_l + λS_k > 1.5 are projected, while lower-threshold blocks remain unconstrained.
SVD and Schur decomposition produce the same projection operator, but Schur decomposition computes it more efficiently.

E.1. Proof of Claim 1

The proof bounds soft equivariance error by combining singular-value truncation with a first-order Taylor remainder, and extends the result to multiple Lie algebra generators.

Claim 1 states that the projected linear function is η_b-soft invariant for compact, connected Lie groups.
The bound is η_b = b√n_G r_G + ε_G, where b is the projection cutoff and ε_G is the Taylor residual.
Exact invariance occurs when the weight lies in the left null space of every Lie algebra generator representation.
The proof bounds contributions from singular directions above the cutoff and then adds the Taylor residual error.
A combined decomposition of multiple generators yields the same soft-invariance claim for the designed projection operator.
Per-generator projection compositions are exact for commuting generators but only approximate for non-commutative generators.

E.3. Proof of Claim 2

The proof establishes a bound on the equivariance error of projected weights using first-order Taylor expansion and representation-specific spectral structure. The resulting error depends on the projection cutoff, group geometry, output dimension, and Taylor residual.

Claim 2 states that the constructed fully connected layer is η_b-soft equivariant for compact, connected Lie groups.
η_b = b√n_G d′ r_G + ε_G, combining the projection cutoff b, generator count n_G, output dimension d′, injective radius r_G, and Taylor residual ε_G.
The proof expands the equivariance error around the identity and retains linear terms involving Lie-algebra generators.
The weight-space analysis uses singular-vector coordinates and block decompositions of the Lie-algebra representations to characterize equivariance constraints.
For finite discrete groups, the first-order approximation error is bounded by 2h · d_S(e, g) when f is h-Lipschitz under the word metric.

F.1. Synthetic O(5) benchmark

The synthetic O(5) benchmark varies the target function’s deviation from strict invariance through γ. The proposed method outperforms the reported baselines on this benchmark.

The benchmark predicts an O(5)-invariant quantity from two vectors in R^5, then adds a non-invariant component to make the task more challenging.
γ controls deviation from strict invariance: γ = 0 gives a strictly invariant target, while increasing γ makes the target less invariant.
The proposed method outperforms the baselines, with EMLP-RPP reported as the second-best method.

F.2. Additional ablation studies and extended results

Additional experiments compare the method with fully equivariant and group-convolution-based alternatives and extend evaluation to trajectory prediction. Across the reported tasks, the method maintains strong performance against these baselines.

Full equivariant models: The method is compared with fully equivariant models on classification and semantic segmentation under the same experimental setups.
Group-convolution comparison: On MNIST, the method maintains stable performance across softness levels and outperforms Wang et al.’s group-convolution-based baseline.
Human trajectory prediction: On human trajectory prediction, the method outperforms strict-equivariant and non-equivariant baselines on ADE, FDE, aADE, and aFDE across most datasets.
Reporting conventions: The segmentation and classification tables report mean ± std results, with eErr and iErr expressed in ×10^-2.

G. Experiment Details

The experiments fine-tune pretrained vision backbones and evaluate soft equivariance across classification, segmentation, and trajectory prediction. For pretrained models, higher softness levels are used to preserve pretrained performance.

Image experiments: Experiments use pretrained ViT-B/16, DINOv2-Base, and ResNet-50 backbones, with task-specific classification or segmentation heads.
Optimization: Models are fine-tuned with rotation augmentation, task-specific learning rates, AdamW, cosine scheduling, and validation-based model selection.
Trajectory prediction: The trajectory experiment uses ETH and UCY subsets with an autoregressive transformer containing four layers and four attention heads.
Softness level selection: For pretrained models, softness below 0.6 can sharply reduce SegFormer performance, so the experiments use softness levels above 0.7.