PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalars are universal: Equivariant machine learning, structured like classical physics

Soledad Villar, David W. Hogg, Kate Storey-Fisher, Weichi Yao, Ben Blum-Smith

arXiv:2106.06610v4cs.LGmath-phstat.ML

TL;DR

Many equivariant-learning methods rely on irreducible representations, tensor objects, or constraints, while a broad class of classical-physics symmetries calls for tractable universal parameterizations. This paper characterizes invariant and equivariant functions using scalar products and contractions, showing that scalar-based constructions provide a simple route to universal approximation across many relevant groups and dimensions. The approach is broad but remains limited for some groups, local gauge symmetries, and dimensional or units symmetries.

  • Problem

    Existing symmetry-aware methods can require irreducible-representation decompositions or other specialized constructions, motivating a tractable characterization for broad classes of physical invariant and equivariant functions.

  • Method

    The paper constructs invariant and equivariant functions from invariant scalar products and scalar contractions, with equivariant vectors expressed using invariant scalar-function coefficients multiplying input vectors.

  • Results

    The paper gives a complete, computationally tractable characterization for scalar and vector functions obeying many classical-physics symmetries, including Euclidean, Lorentz, and Poincaré groups.

  • Takeaways & Limitations

    Invariant scalars can support universally approximating symmetry-constrained neural networks without requiring Clebsch–Gordan solutions for the relevant broad class of groups.

  • Takeaways & Limitations

    The results do not cover all physical symmetries, including some groups without tractable invariant characterizations, local gauge symmetries, and dimensional or units symmetry.

Abstract

from arXiv · show

There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law. Some of these frameworks make use of irreducible representations, some make use of high-order tensor objects, and some apply symmetry-enforcing constraints. Different physical laws obey different combinations of fundamental symmetries, but a large fraction (possibly all) of classical physics is equivariant to translation, rotation, reflection (parity), boost (relativity), and permutations. Here we show that it is simple to parameterize universally approximating polynomial functions that are equivariant under these symmetries, or under the Euclidean, Lorentz, and Poincaré groups, at any dimensionality $d$. The key observation is that nonlinear O($d$)-equivariant (and related-group-equivariant) functions can be universally expressed in terms of a lightweight collection of scalars -- scalar products and scalar contractions of the scalar, vector, and tensor inputs. We complement our theory with numerical examples that show that the scalar-based method is simple, efficient, and scalable.

1 Introduction

The paper develops a scalar-based characterization of invariant and equivariant functions for many classical-physics symmetries, including Euclidean, Lorentz, and Poincaré groups. Its approach uses invariant scalar products and contractions to provide computationally tractable universal parameterizations across dimensions.

  • Motivation: Exact physical symmetries motivate neural architectures that encode translation, rotation, reflection, boosts, and related transformations directly.The motivation includes exact symmetries in astronomy, physics, and chemistry, where generalizable laws must obey them.
  • Method: The scalar-product approach bypasses Clebsch–Gordan decomposition while constructing universally approximating functions for a broad class of equivariant problems.The construction uses complete subsets of scalar products and scalar contractions of input vectors and tensors.
  • Contributions: The work characterizes all scalar and vector functions satisfying the symmetries of classical physics through physically principled invariant scalars.The stated domains include f : (R^d)^n → R and h : (R^d)^n → R^d.
  • Method: Invariant theory shows that O(d)- and Lorentz-equivariant vector functions are linear combinations of input vectors with invariant scalar-function coefficients.The coefficients can be arbitrary functions, or polynomials when the equivariant function is polynomial; scalar products may be Euclidean or Minkowski.
  • Scope: The framework covers orthogonal, rotational, translational, Euclidean, Lorentz, Poincaré, and permutation groups, with their actions specified in the paper’s tables.The Euclidean and Poincaré groups combine translations with orthogonal or Lorentz transformations, respectively.
  • Limitation: The characterization applies to global symmetries but does not obviously generalize to local gauge symmetries without position-dependent metrics and parallel transport.The paper presents extending the method to local gauge symmetries as future work.

2 Related work

Related work has encoded symmetry through irreducible representations, parameter sharing, data augmentation, and invariant-theoretic constructions. The paper positions its contribution as broader, covering invariant and equivariant functions for many physically relevant groups, including non-reductive Euclidean and Poincaré groups.

  • Symmetry-aware architectures: Existing architectures enforce translation, rotation, scale, permutation, and other symmetries in images, point clouds, graphs, weather models, turbulence, and quantum systems.Approaches include convolutional networks, point-cloud models, graph networks, and gauge-equivariant neural networks.
  • Permutation methods: Parameter sharing offers a simpler implementation of permutation equivariance, but characterizing the functions it can express remains an open problem.The approach has been successful in applications including autonomous driving.
  • Representation-based methods: Irreducible-representation methods require Clebsch–Gordan decompositions, whose explicit parameterizations are available for only some groups.The text identifies SO(3) as one group with an implemented solution while noting active research for other Lie groups.
  • Universal approximation: Group averaging can universally approximate invariant functions but is impractical for large finite groups and infinite groups such as O(d).This motivates alternative architectures based on invariant or equivariant layers.
  • Physics applications: Many physics applications still rely on data to encode exact symmetries, such as rotational augmentation for galaxy-image classification.The paper contrasts this practice with architectural enforcement of symmetry.
  • Positioning: Prior invariant-theoretic work often targets specific representations or invariant functions, whereas this paper treats invariant and equivariant functions across broader groups.The stated scope includes non-reductive Euclidean and Poincaré groups whose invariant theory remains tractable.

3 Equivariant maps

The section characterizes invariant and equivariant functions using scalar products and scalar contractions, extending the construction across orthogonal, special-orthogonal, translation, Lorentz, Poincaré, and permutation symmetries.

  • O(d) and SO(d) invariance and equivariance: O(d)-invariant scalar functions depend only on input scalar products, while O(d)-equivariant vector functions lie in the span of the input vectors.For polynomial functions, the scalar coefficients can also be chosen polynomial.
  • O(d) and SO(d) invariance and equivariance: SO(d)-equivariant vector functions require generalized cross-product terms when the inputs span a (d−1)-dimensional subspace.Outside that case, a characterization analogous to the O(d) result applies with SO(d)-invariant scalar coefficients.
  • E(d) invariance and equivariance: Translation-invariant functions can be rewritten uniquely using differences from one input, reducing the problem to n−1 relative vectors.The same reduction preserves equivariance under a subgroup of GL(n, R).
  • E(d) invariance and equivariance: Translation-invariant O(d)-equivariant vector functions use invariant scalar coefficients multiplying input vectors, with coefficient sums constrained to zero.The section also describes corresponding translation-equivariant functions.
  • Lorentz symmetry: Lorentz-equivariant functions admit an analogous scalar-based characterization, with coefficients determined by pairwise Minkowski inner products.The result applies to continuous functions and preserves polynomiality when the original function is polynomial.
  • Poincaré and permutation symmetries: The same framework extends to Poincaré-equivariant functions and to functions combining orthogonal or Lorentz equivariance with permutation invariance or equivariance.Permutation-equivariant vector outputs decompose into component functions characterized by the corresponding single-output result.

4 Examples

The examples express classical mechanics and electromagnetism using invariant scalars and equivariant vectors, illustrating how physical formulas fit the scalar-based framework.

  • Total mechanical energy: Newtonian total mechanical energy is constructed from scalar masses and scalar products of velocities and position differences.The energy is a scalar in Newtonian physics, while the positions enter only through differences.
  • Electromagnetic force law: The electromagnetic force expression is O(3)-equivariant because its nested cross products combine to produce an ordinary vector rather than a parity-violating pseudovector.The vector triple-product identity rewrites the nested cross product using scalar products and input vectors.
  • Electromagnetic force law: After expansion, the electromagnetic force consists of invariant scalar products multiplying input vectors.The velocity inner products and related quantities are scalar-based coefficients in the resulting vector expression.

5 How many scalars are needed?

The scalar parameterization uses pairwise scalar products to express invariant and equivariant functions, but the full collection can become large when the number of inputs greatly exceeds the dimension.

  • Invariant and equivariant functions of vector inputs can be expressed using scalar products ⟨v_i, v_j⟩.This provides the scalar feature set used by the characterization.
  • When n ≫ d, using all scalar products substantially increases the number of features.The feature count can be reduced because a subset of approximately (d + 1)n scalars uniquely determines the full collection.
  • The reduced-scalar observation is related to Gram-matrix rigidity and low-rank matrix completion.The paper also connects scalar recovery to robust reconstruction from random subsets and optimization on Gram matrices.

6 Limitations and caveats

The scalar characterization covers many physically relevant groups but does not address every equivariant-learning setting, and scalar representations can be cumbersome or require additional constraints.

  • The results do not provide a similarly simple characterization for every group, especially many discrete groups used in physical and chemical systems.The paper specifically identifies the GNNs discussed in Appendix A as an example.
  • Scalar-only communication may be impractical for multipole expansions, where high-order tensors provide concise linear aggregation.The authors describe this as an open research direction involving fast multipole methods and n-body networks.
  • Although electromagnetism and general relativity can be rewritten using scalars and vectors, higher-order or non-equivariant forms may be more concise or clear.The paper contrasts the scalar form with electromagnetic tensors, pseudo-vectors, and curvature tensors.
  • The framework does not encode dimensional or units symmetry, so valid polynomials and nonlinear-function inputs require additional architectural or constraint considerations.In particular, nonlinear functions such as exponentials or sigmoids can take only dimensionless scalar quantities as arguments.
  • The characterization concerns global symmetries and does not obviously generalize to local gauge symmetries.The authors suggest that a position-dependent metric might extend the model, but defer this development to future work.

7 Numerical experiments

The paper demonstrates scalar-based MLPs on O(5)-invariant and O(3)-equivariant toy tasks, constructing the latter to preserve both positional equivariance and permutation invariance.

  • The experiments use scalar-based multilayer perceptrons on two toy tasks: one O(5)-invariant and one O(3)-equivariant.The paper notes that further dynamical-system experiments appear elsewhere and that code is available on GitHub.
  • The O(5)-invariant task learns a scalar function of inner-product features from pairs of vectors in R^5.The target is represented through the scalar quantities x_1^T x_1, x_1^T x_2, and x_2^T x_2.
  • The O(3)-equivariant task uses five point masses and positions, with the target equivariant under O(3) and invariant under permutations of the five points.The construction expresses the output through permutation-invariant functions modeled by MLPs on deep sets.
  • The broader related-work approaches enforce symmetry through tensor representations, linear equivariant maps, and nonlinearities, often relying on irreducible-representation decompositions.For permutation groups, tensor order can support universality, but the dimension of linear equivariant maps grows super-exponentially with tensor order.

B Equivariant functions under rotations and the orthogonal group

The appendix proves that orthogonal-group equivariant vector functions are generated by input vectors with invariant scalar coefficients, while rotation-only equivariance additionally permits generalized cross-product terms.

  • SO(d) extension: For SO(d), a generalized cross product is needed when the input span has dimension d − 1 because it supplies a vector outside that span.This cross product is defined through the Hodge dual and satisfies an inner-product/determinant identity.
  • O(d) equivariance: For O(d)-equivariant vector functions, each output can be written as a sum of input vectors multiplied by invariant functions.The polynomial case follows by converting the vector output into an invariant scalar function using an auxiliary vector and applying invariant theory.
  • Proof strategy: The proofs establish the resulting identities first on orbit representatives and then everywhere by equivariance.For the polynomial statements, the same conclusion follows from invariant-theory expansions and the nondegeneracy of the bilinear form.
  • SO(d) extension: Polynomial SO(d)-invariant scalarizations may involve inner products, determinants of d input vectors, and determinants containing an auxiliary vector.The latter determinant terms become inner products with generalized cross products, yielding the additional equivariant generators.

C Translation-invariant functions

Translation invariance is handled by replacing inputs with differences relative to one reference vector. This reduces the problem to orthogonal-equivariant functions of n−1 vectors, with the original vector recovered when the output is translation-equivariant.

  • Translation reduction: The difference map (v1, …, vn) → (v2−v1, …, vn−v1) has fibers equal to translation orbits.Polynomiality and equivariance properties are preserved through the map and its section.
  • Translation reduction: A translation-invariant function therefore descends to a well-defined function of the n−1 relative vectors.
  • Invariant outputs: Translation-invariant O(d)-equivariant vector functions are generated by the relative input vectors with O(d)- and translation-invariant scalar coefficients.
  • Equivariant outputs: For translation-equivariant vector functions, the output is the corresponding relative-vector function plus the reference vector v1.

D Invariant and equivariant functions under the Lorentz group

Lorentz-equivariant constructions parallel the orthogonal-group case but require special treatment of lightlike vectors and orbit structure. Continuity supplies key extensions in degenerate cases, while Minkowski inner products characterize continuous invariant functions despite Lorentz-specific subtleties.

  • Invariant-theory foundation: The Lorentz-group proof follows the orthogonal-group argument because both groups inherit the relevant invariant-theory result from O(d,C).
  • Continuous invariants: Minkowski inner products distinguish closed Lorentz orbits, so every continuous invariant function is a function of those inner products.They do not distinguish every pair of distinct orbits, including a nonzero lightlike configuration and zero.
  • Equivariant vectors: Lorentz-equivariant vector functions are constrained to the span of their input vectors by transformations that fix the span and move vectors outside it.
  • Lightlike cases: The Lorentz orthogonalization construction must avoid lightlike basis vectors because Minkowski norms can vanish for nonzero vectors.The procedure can restart with alternative linear combinations or complementary bases in higher-dimensional subspaces.
  • Lightlike cases: When a one-dimensional input is lightlike, continuity is required to extend the span argument from dense non-lightlike vectors.Without continuity, the Lorentz-invariant lightlike set prevents this argument.

E Permutation-invariant and equivariant functions that are also orthogonal or Lorentz-equivariant

Permutation symmetry is imposed by averaging and relating component functions across permuted inputs. This yields shared scalar functions that are invariant over the non-target inputs and supports permutation-equivariant vector constructions.

  • Permutation averaging: Permutation invariance is obtained by averaging an equivariant representation over the symmetric-group orbit.
  • Permutation-invariant functions: For a fixed output index, the associated scalar function is invariant under permutations of the other n−1 inputs.
  • Shared component functions: A second averaging step allows all component scalar functions to be chosen as the same function.
  • Result: The resulting construction proves the claimed permutation-invariant and equivariant forms.

F Einstein summation notation

Einstein summation notation encodes equivariance through index-contraction rules. Scalar expressions reduce to products of scalar products, while vector expressions become input vectors multiplied by invariant scalar coefficients.

  • Index rules: Einstein notation produces equivariant objects when indices obey the rule that each index appears once or twice.
  • Vector outputs: An expression with one unsummed index produces an O(d)-equivariant vector.
  • Example: A cross product can be rewritten using scalar products and input vectors, but only with the antisymmetric tensor.
  • Scalar outputs: O(d)-invariant scalar polynomials use even numbers of vectors and can be rearranged into products of scalar products.
  • Vector outputs: O(d)-equivariant vector polynomials use odd numbers of vectors and can be rearranged into input vectors times scalar-product polynomials.
  • Nontrivial metrics: For nontrivial metrics, contractions require distinguishing covariant and contravariant components through the metric tensor.The Lorentz metric is diagonal, whereas a general-relativistic metric can depend on position and time.
  • Scope: The framework is expected to generalize to scalar, vector, and tensor functions with scalar, vector, and tensor inputs.

G Connections with low-rank matrix completion

The paper connects low-rank matrix structure to compact scalar parameterizations of invariant and equivariant functions, then applies these ideas to physics model design. The construction supports Lorentz and orthogonal symmetries but has a permutation-invariance limitation.

  • Low-rank matrix completion: Low-rank matrix structure lets invariant functions be represented using a small subset of matrix entries derived from scalar products.The subset Ω(M) is sufficient to determine M almost everywhere when M has rank d.
  • Low-rank matrix completion: The relevant matrix is either V^⊤V or V^⊤ΛV, covering orthogonal and Lorentz-invariant settings.The same parameterization applies even when the matrix is not positive semidefinite, as in the Minkowski case.
  • Low-rank matrix completion: Learning can use Ω(M) directly without computing every scalar in the full matrix.This reduces the scalar information required by the parameterization.
  • Low-rank matrix completion: The proposed scalar subset is not permutation invariant, motivating future work on universally expressive permutation-invariant scalars.This is the stated limitation of the sampling procedure Ω(M).
  • Model-building considerations: The causal structure and group actions must match the physics inputs and outputs, including distinct actions on positions, displacements, velocities, and other vectors.Particle-wise tuples such as charge, position, and velocity are the objects on which permutations typically act.
  • Model-building considerations: For n-body models, rotation symmetry uses invariant scalar products, while translation symmetry uses positional differences before scalar construction.Permutation symmetries are handled with set- or graph-based networks and shared functions h_i = h.
Loading 2106.06610v4…