PaperlayerView exact revision on arXiv

Source-linked AI summary

Coordinate-Free Automatic Differentiation

Alex Guerra, Joshua Aduol, Yaqian Tang, Daniel Williams, Ryan P. Adams, Peter Orbanz

arXiv:2610.12776v1cs.LGcs.MSmath.DG

TL;DR

Coordinate-array AD does not directly carry the geometric structure needed to express many manifold operators, and coordinate-dependent representations must be re-derived for new spaces. The paper implements a coordinate-free multivector derivative using JAX primitives and demonstrates its reuse across geometric workflows. Its conclusion is that geometric differential operators can be reusable AD transformations rather than being re-derived for each coordinate system or application.

  • Problem

    Coordinate-array AD does not directly cover geometric objects and operators requiring metric, orientation, or tangent-space structure, and representation-dependent operators must be re-derived for new spaces.

    Source: §1 · pp. 1–2

  • Method

    The framework implements a coordinate-free multivector derivative with JAX primitives and composes it with other JAX transformations.

    Source: §1 · p. 2

  • Results

    The framework demonstrates feasibility and reuse across manifold operators, electromagnetic inverse problems, and rotor-group optimization.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Geometric differential operators can be exposed as reusable AD transformations rather than re-derived for each coordinate system or application.

    Source: §5 · pp. 11–12

    The mass norm has no known closed form for 3 ≤ k ≤ n − 3, so these experiments are limited to the n = 4, k = 2 case.

    Source: §4.2 · p. 9

Abstract

from arXiv · show

Automatic differentiation (AD) is a cornerstone of modern machine learning. Since all AD frameworks implicitly fix a basis of Euclidean space to define gradients and Jacobians, they natively process mathematical objects that are arrays of a fixed set of coordinates. This makes it awkward to express objects such as rotations, volume forms, and curvatures that arise, for example, in applications of AI to science. Bespoke workarounds lack reusability. We show that (i) a single, basis-independent derivative operator can represent many geometric differential operations, and (ii) this operator can be expressed entirely in terms of existing AD primitives. This reusable transformation can be composed with JAX batching, compilation, and hardware acceleration. Geometry enters as algebraic data. We experimentally demonstrate its reuse across different manifold operators, its composition through a spacetime inverse-problem pipeline, and its use in optimization over different rotor subgroups.

1 Introduction

Coordinate-array AD handles Euclidean derivatives but does not directly carry the geometric structure needed for operators such as curvature or divergence. The paper introduces a coordinate-free multivector derivative implemented with JAX primitives and applies it across geometric operations and workflows.

1 Introduction

  • Coordinate arrays do not directly represent geometric objects and operators that require metric, orientation, or tangent-space structure, so these operators must be re-derived for new representations.The introduction names curvature, volume forms, divergence, curl, and covariant derivatives among the examples.Source: §1 · pp. 1–2
  • The framework uses geometric algebra to define a coordinate-free multivector derivative that incorporates metric and orientation information algebraically.The paper also notes that geometric-algebra representations are already used in parts of machine learning.Source: §1 · p. 2
  • The multivector derivative is represented with JAX primitives as a composable transformation supporting forward and reverse modes, higher-order differentiation, and batching.Source: §1 · p. 2
  • Specializing the derivative yields curl, divergence, covariant derivatives, shape operators, and curvature tensors on (pseudo-)Riemannian manifolds of any dimension.Source: §1 · p. 2
  • The experiments demonstrate reuse across manifold flows, electromagnetic inverse problems, and rotor-group optimization.Source: §1 · p. 2
  • Built entirely from JAX primitives, the framework composes with other JAX transformations and models.Source: §1 · p. 2

2 Related Work

Prior approaches extend differentiation beyond Euclidean arrays, while geometric algebra has also been used as a machine-learning representation. This paper distinguishes its approach by defining the derivative algebraically before incorporating manifold structure through tangent projection.

2 Related Work

  • Existing AD extensions for structured spaces commonly introduce geometry through a chosen local representation, such as charts or retractions [50].Source: §2 · p. 2
  • Here the derivative is defined algebraically on multivectors, and manifold structure enters afterward through tangent-space projection.Source: §2 · p. 2
  • Geometric algebra already appears in machine-learning representations, including equivariant architectures, but those works do not compute geometric differential operators [10] [11] [15] [47] [48] [52] [56].The paper presents those representations as natural downstream settings for its AD operators.Source: §2 · p. 3

3 The Multivector Derivative

The section defines a multivector derivative and expresses it through AD primitives, then specializes it to vector and manifold operations. Tangent projection brings curved spaces into this coordinate-free framework.

3 The Multivector Derivative

  • The approach defines the multivector derivative, translates it into AD primitives, and generalizes to manifolds through tangent-space projections.Source: §3 · p. 3

3.1. Definitions

  • Geometric algebra extends vectors with oriented area and volume elements and their sums; its full space has dimension 2^n, making computation potentially prohibitive in high dimensions.Source: §3.1 · p. 3

3.2. The Multivector Derivative

  • The multivector derivative is basis-independent, intrinsic to Λ(V), and encompasses the differential operators considered in the paper.Its definition depends on the inner product, the spanned subspace, and the geometric product, not the chosen basis.Source: §3.2 · p. 4

3.3. From the Definition to AD primitives

  • Jacobian-vector and vector-Jacobian products implement the derivative, with forward-mode cost scaling in input directions and reverse-mode cost in output directions.For scalar outputs, one reverse-mode call suffices, recovering the usual advantage of jax.grad.Source: §3.3 · pp. 4–5

3.4. The Vector Derivative

  • The vector derivative splits into grade-raising and grade-lowering parts: the former gives the exterior derivative, while the latter yields divergence-like terms.For scalar functions the grade-raising part is the ordinary gradient; for vector functions it is curl-like.Source: §3.4 · p. 5

3.5. From Flat Spaces to Manifolds

  • Projecting the derivative onto a manifold’s tangent space gives intrinsic and covariant derivatives; their compositions express standard operators and curvature without charts or Christoffel symbols.The projected curl and divergence are nilpotent, and the resulting framework also provides a grade-preserving Hodge–De Rham Laplacian.Source: §3.5 · pp. 5–6

4 Experiments

The experiments are end-to-end demonstrations of reusing and composing a small set of AD transformations, rather than performance benchmarks. The explicit manifold-learning experiments deform known starting manifolds toward target shapes.

4 Experiments

  • The experiments demonstrate reuse and composition of a small collection of AD transformations across four end-to-end workflows, not performance benchmarks.Source: §4 · p. 6

4.1. Explicit Manifold Learning

  • Explicit manifold learning starts from a known simple manifold and applies a smooth transformation to solve a learning problem.Source: §4.1 · p. 6
  • The illustrated targets are a Z5-symmetric pinwheel on S2 and a D7-symmetric gear on H2, starting from an icosidodecahedron and a rectified {3, 7} tiling, respectively.Source: Figure 1 · p. 6, Figure 2 · p. 6

Non-Euclidean tilings.

The examples reuse geometric-algebra differential operations for manifold learning, spacetime field inversion, and rotor-group optimization. They demonstrate divergence-free tiling flows, implicitly learned minimal surfaces, differentiable field fitting, and subgroup-specific alignment.

Non-Euclidean tilings.

  • A projected 2-form followed by ∇M⌋ yields divergence-free manifold flows; on S2 and H2, these deform tilings toward chiral pinwheel and D7-symmetric gear targets.The H2 flow preserves area with respect to the induced hyperbolic metric.Source: Non-Euclidean tilings · pp. 6–7
  • With the Gauss–Bonnet term, the square flow approaches an approximately closed surface; curvature-only training learns a small section of a sphere.The added loss guides the surface toward an Euler characteristic estimate of approximately 2, despite the square not being closed.Source: Figure 3 · p. 7, Non-Euclidean tilings · p. 7

4.2. Implicit Manifold Learning

  • The Biot–Savart parameterization makes ∂Mθ=Γ structural for any (n, k), while mass-norm optimization targets minimal surfaces but is attempted only for n=4, k=2.The restriction follows because no closed form is known for the mass norm when 3 ≤ k ≤ n−3.Source: §4.2 · pp. 8–9

4.3. Inverse Design in Spacetime

  • A bounded-velocity MLP worldline is differentiated through retarded-time bisection and the Liénard–Wiechert field pipeline, letting sparse Faraday-bivector observations recover the target helix.The pipeline backpropagates relative mean squared error between predicted and observed fields.Source: Figure 7 · p. 9, §4.3 · pp. 9–10

4.4. Optimization on Rotor Groups

  • Rotor subgroups are specified through bivector Lie-algebra subspaces; left-trivialized derivatives and a positive-definite inner product produce descent updates valid for Lie groups embedded in Cl(p, q).The positive-definite inner product is needed to map the covector into an update direction.Source: §4.4 · pp. 10–11
  • On four nested CGA(3) rotor groups, alignment reaches each subgroup’s best fit and the conformal fit is exact; Lorentz optimization also recovers the correct transformation.The groups are translation, rigid, similarity, and conformal; the optimized Lorentz rotor also recovers the ship’s velocity.Source: Figure 9 · p. 10, §4.4 · p. 11, Figure 10 · p. 11

5 Discussion and Conclusion

The same geometric operators support workflows across manifold learning, spacetime inverse problems, and rotor-group optimization. Geometry enters through algebraic data, making these operators reusable AD transformations rather than operations re-derived for each coordinate system or application.

5 Discussion and Conclusion

  • The same implementation of the multivector derivative, exterior derivative, and tangent projection supports manifold learning, spacetime inverse problems, and rotor-group optimization.These workflows use a small set of geometric operators.Source: §5 · p. 11
  • Geometric structure enters through the selected index set, the multivector field being differentiated, and, on manifolds, the tangent pseudoscalar.Source: §5 · p. 11
  • Geometric differential operators can be exposed as reusable AD transformations instead of being re-derived for each coordinate system or application.Source: §5 · p. 12

AI use statement

The authors used generative AI tools to iterate on experiments, identify relevant literature, and revise the paper. They state that they reviewed the sources and take responsibility for the paper's claims.

AI use statement

  • Generative AI helped the authors iterate on experiments and find training settings after early experiment versions were written by humans.Source: AI use statement · p. 12
  • The authors used generative AI to find relevant literature sources and say they reviewed them to check that citations were not hallucinated.Source: AI use statement · p. 12
  • Generative AI assisted with revising the paper, refining mathematical statements, and formatting, while the authors take responsibility for all claims.Source: AI use statement · p. 12

A Proofs

The appendix establishes basis independence for the multivector derivative and gives proof details for its AD implementation. It also discusses mode costs and the role of the multivector inner product in reverse mode.

A Proofs

  • The reciprocal basis is computed from the Gram matrix, which need not be the identity even for an orthonormal Euclidean basis of vectors.The appendix notes that nondegeneracy of the inner product ensures the Gram matrix is invertible.Source: A Proofs · p. 15

A.1. Basis Independence of ∂I

  • The multivector derivative indexed by I depends on the spanned subspace and pairing, not on the chosen basis for that subspace.The proof establishes this by rewriting the derivative in an alternative basis.Source: §A.1 · pp. 15–16

A.2. Proof of Lemma 2

  • The metric-aware VJP computes the adjoint using the multivector inner product rather than the Euclidean inner product.Source: §A.2 · p. 16
  • Forward mode uses |I| JVP calls and reverse mode uses |O| VJP calls, making reverse mode efficient for scalar outputs and forward mode often preferable for vector outputs.For vector derivatives, the passage specifies n JVPs.Source: §A.2 · p. 17

B Software API and Implementation

The software represents geometric algebras and multivectors in JAX, then builds multivector and manifold derivatives from AD transformations and projection operators. The appendix describes the associated geometric API, examples, and experimental runtime setup.

B Software API and Implementation

B.1. Algebras and Multivectors

  • Algebras are specified by positive, negative, and zero basis-vector counts; many AD primitives require a nondegenerate metric, so the implementation often assumes r = 0.The examples use real vector spaces.Source: §B.1 · p. 17
  • Algebra operators are stored as sparse tensors and implemented as tensor contractions, enabling GPU execution and native differentiation.Source: §B.1 · p. 17

B.2. The Multivector Derivative as a Function Transformation

  • The library implements the multivector derivative with forward- or reverse-mode AD, and generalizes both methods to pytrees of multivectors.The two methods compute extensionally equal values; the choice depends on the derivative context.Source: §B.2 · p. 18

B.3. Vector Manifolds and their Transformations

  • Manifold derivative functions are compositions of the multivector derivative and projection operators, with APIs for intrinsic or covariant derivatives, Laplacians, curvature, integration, and transformations.Source: §B.3 · pp. 19–20
  • The shape operator supplies curvature information; its grade-raising part yields the curl tensor used to construct Riemann, Ricci, and scalar curvature.The described curvature construction requires no coordinate charts or Christoffel symbols and is coherent across dimensions and signatures.Source: §B.3 · p. 19
  • The toolkit approximates both sides of a multivector-calculus fundamental theorem whose grade components recover Stokes', Green's, and the divergence theorem.Source: §B.3 · p. 21

C.1. Shared Setup

  • Experiments use 64-bit floats; all but the implicit-surface networks run on CPU, while those networks use one Nvidia L40 GPU.Quoted wall-clock times are from an Apple M3 laptop unless stated otherwise.Source: §C.1 · p. 21

C.2. Explicit Manifold Learning

The experiments use symmetry-aware divergence-free flows to deform spherical and hyperbolic tilings, and curvature-based training to close a square into a sphere. The Gauss–Bonnet term is associated with a near-spherical result, unlike curvature-only training.

C.2.A. Non-Euclidean Tilings

  • A divergence-free autonomous field deforms pentagons into a chiral Z5 pinwheel and heptagons into a D7 gear; equivariance on H2 is approximate.The flow is symmetrized for the tiling’s symmetry group; on H2, the group uses a finite patch of 96 elements.Source: §C.2 · pp. 21–22, Figure 12 · p. 23
  • The tiling experiment’s scalar potential parameterization yields divergence-free fields by construction, with the same trained field moving the tiles and triangles.On a 2-surface, the 2-field is parameterized as ϕθIM and the field is ∇M⌋(ϕθIM).Source: Figure 12 · p. 23

C.2.B. Backpropagation Through the Euler Characteristic

  • A diffeomorphic neural ODE flow maps the square to a compact manifold, enabling differential-operator computations and Euler-characteristic estimation on the deformed surface.The experiment estimates Euler characteristic from uniform samples and evaluates scalar curvature using pushed-forward tangent vectors.Source: §C.2 · p. 23
  • Scalar curvature R equals 2K on a 2-manifold, but is defined across dimensions and signatures and computed without dimension-specific code.The experiment matches the unit sphere’s scalar curvature R*=2.Source: §C.2 · p. 23
  • With the Gauss–Bonnet term, the learned flow closes the square into a sphere and its Monte Carlo Euler-characteristic estimate approaches 2; without it, the square collapses to a small spherical cap (χ̂≈0.05, area 0.34, mean curvature 0.98).Source: §C.2 · p. 24, Figure 14 · p. 25

C.3. Implicit Manifold Learning

Implicit manifold learning represents a surface with boundary through a differential-form field, then minimizes its mass norm and extracts the surface from that field. The experiments report close matches to exact minimal surfaces, within the tested codimensions.

Surface Representation

  • A differential-form field ωθ=ωΓ+∂I1∧ηθ parameterizes a surface with boundary Γ, with (∂I1∧)²=0 ensuring ∂I1∧ωθ=δΓ.The extraction uses ωθ alone and does not access the closed-form surface.Source: §C.3 · pp. 25–26

The Mass Norm

  • The mass-norm objective encourages ωθ to be approximately simple, but no closed form is known for its infimum when 3≤k≤n−3.The experiments therefore focus on (n,k)=(3,1) and (4,2).Source: §C.3 · pp. 25–26

Extraction

  • The extractor reads the learned field rather than the known surface: codimension-one surfaces are level sets, while codimension-two surfaces are ridges of |ωθ|.Codimension-one extraction filters and cleans a mesh; codimension-two extraction locates local maxima in normal planes and meshes the resulting points for rendering.Source: §C.3 · p. 26

Training & Metrics

  • The two main codimension-two ground truths are holomorphic curves and hence mass-minimizing; Table 1 compares extracted point clouds with dense exact-surface samples.The stated exact areas are 5.205 for the polynomial graph and 5.435 for the branched disk.Source: §C.3 · p. 26, Table 1 · p. 26
  • The three additional codimension-two boundaries’ extracted surfaces match their exact surfaces closely.The reported cases include a holomorphic graph, an annulus, and a pair of anti-holomorphic disks; the table compares point-cloud distances, coverage, spurious points, and mesh area.Source: Table 1 · p. 26, §C.3 · p. 27, Figure 16 · p. 28

C.4. Inverse Design in Spacetime

The spacetime inverse problem learns a particle trajectory from observed Faraday fields using a differentiable forward model. The recovered worldline closely matches the target, while observation timing affects how well its tail is constrained.

Forward Simulator

  • A differentiable retarded-time solve feeds the Liénard–Wiechert potential and Faraday bivector, allowing field-error backpropagation to update the trajectory network.The retarded time is found by solving the past-light-cone condition; the field loss is a relative mean squared error over observation events.Source: §C.4 · p. 29

Training & Metrics

  • 6·10−4 mean and 1.2·10−3 maximum position errors recover the helical worldline over t∈[1,4], equal to 0.2% and 0.3% of the orbit radius.Training takes 6 minutes on an Apple M3 laptop CPU.Source: §C.4 · p. 29
  • Restricting observations to [1] leaves the tail poorly constrained: mean error on [3] is 4.6·10−2 versus 5·10−4 with the extended observation window.The paper attributes this difference to causal coverage: no observation in the restricted window can see the tail.Source: §C.4 · p. 31
  • The final model reproduces the target field, including the sharp spiral wavefront emitted by the orbiting charge.During training, the learned trajectory moves from near rest toward the target helix as its field develops a spiral pattern.Source: Figure 17 · p. 30, §C.4 · p. 31

C.5. Optimization on Rotor Groups

The appendix develops multiplicative descent for rotor groups using a generator-space differential and an explicit metric choice, then evaluates it on conformal and Lorentz-group optimization tasks.

C.5. Optimization on Rotor Groups

  • The update pairs the left-trivialized differential with Lie-algebra generators, feeds their coordinates to Adam, and exponentiates each step to keep iterates on the rotor group.Chart updates break down where their chart differential is singular; the corrected chart differs from multiplicative descent only in its retraction.Source: §C.5 · pp. 31–32
  • Descent requires a positive-definite metric on the Lie algebra; the multivector inner product fails for the considered groups, and non-compact groups have no canonical bi-invariant metric.The chosen generators are declared orthonormal, making the metric explicit and replaceable.Source: §C.5 · p. 31
  • Four conformal subgroups of dimensions 3, 6, 7, and 10 are optimized in one vectorized run against a shared target, with 500 multiplicative Adam steps.The groups are translations, rigid motions, similarities, and the full conformal group.Source: §C.5 · p. 32
  • 6·10^-12 relative error recovers the conformal teacher noise-free; from a 4 rad start, multiplicative descent reaches loss 9·10^-21 while the static chart stalls at 0.41 and the corrected chart diverges.The descent crosses the conjugate point g = −1; the table reports final subgroup losses and Figure 18 shows the training-loss comparison.Source: §C.5 · p. 32, Table 2 · p. 33, Figure 18 · p. 33
  • Every on-group Lorentz update reaches the noise floor; the penalized matrix reaches it with constraint violation 1.9·10^-3, while polar projection stalls at 48σ^2 with violation 1.7.The polar projection stalls because O(4) contains no boosts.Source: Table 3 · p. 34, §C.5 · p. 34

Related papers

Loading 2610.12776v1…