PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Continuity of Rotation Representations in Neural Networks

Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, Hao Li

arXiv:1812.07035v4cs.LGstat.ML

TL;DR

Neural networks often regress rotations through representations such as quaternions or Euler angles, but discontinuities can make full rotation spaces difficult to learn. The paper defines continuity using topology, analyzes rotation representations across dimensions, and introduces continuous alternatives. It shows continuous 5D and 6D representations for 3D rotations and reports better performance than discontinuous representations across several graphics and vision tasks.

  • Problem

    Discontinuous quaternion and Euler-angle representations can cause learning problems when neural networks regress the full 3D rotation space.

  • Method

    The paper defines continuous representations through mappings between an original space and a Euclidean representation space, then analyzes and constructs representations for rotation groups.

  • Results

    Continuous 5D and 6D representations for 3D rotations outperform discontinuous representations across autoencoding, point-cloud rotation estimation, and human-pose inverse kinematics tasks.

  • Takeaways & Limitations

    Continuous rotation representations are more suitable for neural-network learning than the commonly used discontinuous quaternion and Euler-angle representations.

  • Takeaways & Limitations

    The decoding map f is specifically beneficial when defined almost everywhere on the region where network outputs are expected to lie.

Abstract

from arXiv · show

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

1. Introduction

The paper identifies continuity as a central issue in neural-network rotation regression and develops continuous representations to address it. It establishes dimensional limits for 3D rotations and reports empirical advantages for 5D and 6D alternatives.

  • 3D rotation representations in 3D or 4D spaces can produce large errors at certain angles because of discontinuities.The paper connects these discontinuities to deeper topological problems and argues that discontinuous representations can be harder for neural networks to approximate.
  • The paper defines continuity for rotation representations and connects the definition to homeomorphism and embedding.This definition is designed for neural-network use.
  • All representations of 3D rotations are discontinuous in real Euclidean spaces of four or fewer dimensions.The paper presents continuous representations for SO(n), including 5D and 6D representations for 3D rotations, and extends the constructions to O(n) and similarity transforms.
  • 6D and 5D continuous representations outperform discontinuous representations across rotation autoencoding, point-cloud rotation estimation, and human-pose inverse kinematics.In rotation autoencoder experiments, discontinuous representations have up to 6 to 14 times higher mean errors and tend to converge more slowly.
  • Direct regression of 3x3 rotation matrices produces larger errors than the 6D representation in the reported experiments.For applications requiring orthogonal outputs, Gram-Schmidt orthogonalization effectively yields the 6D representation.

2. Related Work

Prior work studied neural-network approximation, rotation-representation continuity, and learned 3D pose and inverse-kinematics tasks. These studies commonly used quaternions, Euler angles, axis-angle representations, or direct matrix regression.

  • Neural-network approximation theory provides accuracy results for functions in Lp spaces and bounds involving Fourier-transform properties.The reviewed results include an O(ϵ−2) neuron bound for achieving order-ϵ approximation under specified Fourier conditions.
  • Earlier work identified discontinuities in Euler angles and quaternions as problematic for orientation operations or neural-network learning.Exponential maps and direct 3x3 matrix regression were proposed in related settings, but not general continuous rotation representations.
  • Learned 3D pose-estimation systems commonly use compact quaternion or axis-angle representations instead of directly predicting rotation matrices.Direct matrix predictions may not correspond to valid rotations.
  • Neural networks for inverse kinematics are studied because they can be faster than traditional methods and differentiable for use in complex learning tasks.Most cited approaches represent rotations with quaternions or axis-angle representations.

3. Definition of Continuous Representation

The paper defines a representation as a pair of mappings between an original space and a Euclidean representation space, requiring reconstruction and continuity of the encoding map. A 2D example shows why angular coordinates can be discontinuous, while topology explains when continuous representations are possible.

  • The paper develops its continuity definition from a 2D rotation example and relates it to topology.The section introduces terminology, analyzes the motivating example, and connects continuity to topological concepts.
  • SO(n) denotes the special orthogonal group of n-dimensional rotations, represented by matrices satisfying orthogonality and determinant-one constraints.Its group operation is matrix multiplication.
  • An angular representation of SO(2) using [0, 2π] is discontinuous at the identity because directional limits yield 0 and 2π.The corresponding first-column vector representation on the unit circle is continuous.
  • A representation consists of f: R → X and g: X → R with f(g(x)) = x for every x; it is continuous when g is continuous.The network produces an intermediate representation in R, which f maps back to the original space X.
  • The decoding map f is beneficially defined almost everywhere on the region where network outputs are expected to lie.This allows arbitrary network representations in that region to be mapped back to X.
  • For a continuous representation, restricting the codomain of g to g(X) makes g a homeomorphism; if X cannot embed in the representation space, continuity is impossible.The paper applies this topological condition to show that 3D rotations lack continuous representations in four or fewer dimensions.

4. Rotation Representation Analysis

The paper analyzes discontinuous and continuous rotation representations, showing topological limits for low-dimensional 3D representations and constructing continuous alternatives for SO(n). For 3D rotations, its constructions yield 6D and 5D representations, with dimension reduction retaining continuity but not outperforming the Gram-Schmidt-like representation empirically.

  • Discontinuous representations: Euler-angle representations are discontinuous because the identity rotation has directional limits at 0 and 2π.The same discontinuity pattern applies to the azimuth angle in 3D rotations.
  • Discontinuous representations: Quaternion representations are also discontinuous around 180° rotations, despite reconstructing every rotation matrix through normalization.The discontinuity arises under the Euclidean topology on the quaternion representation space.
  • Discontinuous representations: SO(3) has no continuous representation in R^d for d < 5 because it embeds in R^5 but not in any lower-dimensional Euclidean space.The argument uses SO(3)'s homeomorphism with RP^3 and the requirement that a continuous representation provide an embedding.
  • Continuous representations: The n^2 − n-dimensional construction represents SO(n) using the first n − 1 columns and reconstructs the final column with a Gram-Schmidt-like process.The process normalizes successive vectors and computes the last column using a generalized cross product.
  • Continuous representations: For 3D rotations, the n^2 − n construction becomes a 6D representation whose Gram-Schmidt mapping ensures an orthogonal 3×3 output.This differs from direct matrix prediction, which may require orthogonalization separately.
  • Continuous representations: Dimension reduction can make representations less redundant, but experiments found it did not outperform the Gram-Schmidt-like representation from Case 3.The paper notes that projection nonlinearities from square terms and division may make learning more difficult.
  • Continuous representations: Stereographic projections reduce the SO(n) representation to n^2 − 2n + 2 dimensions while preserving continuity and one-to-one behavior.For SO(3), one normalized projection turns the 6D representation into a 5D representation; up to n − 2 projections are possible.

5. Empirical Results

The experiments compare continuous and discontinuous rotation representations across autoencoding, point-cloud rotation estimation, and human-pose inverse kinematics. Continuous representations generally train faster and achieve lower errors, while direct matrix regression and discontinuous representations show larger errors in several settings.

  • Continuous representations produced better learning behavior across a sanity test and two real-world rotation-regression applications.
  • 5.1. Sanity Test: In the autoencoder test, 6D and 5D representations converged faster and had the smallest mean, maximum, and standard-deviation errors.Their errors stayed below 2°, whereas quaternion, axis-angle, and Euler-angle representations reached errors up to 180°.
  • 5.2. Pose Estimation for 3D Point Clouds: In point-cloud rotation estimation, the 6D representation had the lowest mean and standard-deviation errors, with around 95% of errors below 5°.Euler representations had around 10% of errors above 25°, while 5D outperformed the 3D and 4D representations.
  • 5.2. Pose Estimation for 3D Point Clouds: The 3×3 matrix performed worse than the 6D representation after Gram-Schmidt orthogonalization, which the authors hypothesize introduces errors.
  • 5.3. Human Pose Inverse Kinematics: In human-pose inverse kinematics, predicted rotations are converted through forward kinematics and evaluated using joint-position reconstruction error.Hip-adjacent joints receive ten times higher loss weight because hip orientation critically affects the overall skeleton pose.
  • 5.3. Human Pose Inverse Kinematics: The inverse-kinematics experiments use 865 motion clips from 37 categories, with 1.14 × 10^6 training frames and 1.07 × 10^5 test frames.
  • 5.3. Human Pose Inverse Kinematics: Matrix regression without training-time orthogonalization produced huge errors in the human-pose experiment.

6. Conclusion

The paper concludes that neural networks fit continuous rotation representations better than discontinuous ones. It presents continuous 5D and 6D representations for 3D rotations and demonstrates their advantages in synthetic and practical tasks.

  • Neural networks can better fit continuous rotation representations than discontinuous ones.
  • Quaternion and Euler-angle representations have discontinuities that can cause problems during learning.
  • Continuous 5D and 6D representations outperform discontinuous alternatives in autoencoding, 3D pose estimation, and human inverse kinematics.

A. Overview of the Supplemental Document

The supplemental document provides explicit 6D formulas, a formal proof for the 5D representation, deeper quaternion analysis, connections between continuous and discontinuous representations, visualizations, and additional experiments.

  • Section B gives the explicit 6D representation for 3D rotations.
  • Section C formally proves that the 5D representation from Case 4 satisfies the properties of a continuous representation.
  • Section D establishes that unit quaternions are also discontinuous representations for 3D rotations.
  • Section E explains how continuous 5D and 6D representations interact with discontinuous angle representations such as quaternions.
  • Sections F and G provide visualizations of discontinuities and additional empirical results.
  • The supplemental formulas map between SO(3) and the proposed 6D representation.

C. Proof that Case 4 gives a Continuous Representation

The proof establishes that the Case 4 functions form a continuous representation by showing that the reconstruction map is defined and continuous, and that composing the maps recovers every M ∈ SO(n).

  • The proof targets three properties: gP is defined on SO(n), gP is continuous, and fP(gP(M)) = M for every M ∈ SO(n).
  • The construction vectorizes M by dropping its last column before applying the normalized projection.
  • gP is defined because v1 = 1 would force the remaining part of a unit-length column of M to have zero length, contradicting M ∈ SO(n).
  • gP is continuous because it is a composition of functions that are continuous on their domains.
  • The lemma Q(P(u)) = u shows that the normalized projection and reconstruction recover the original vector when ||u2:|| = 1.
  • Using the lemma, the proof derives fP(gP(M)) = M by reducing the composition to the corresponding Gram-Schmidt representation.

D. The Unit Quaternions are a Discontinuous Representation for the 3D Rotations

Unit quaternions are discontinuous representations of 3D rotations, and visualization exposes incorrect topology for quaternion, axis-angle, and Euler-angle mappings.

  • The representation maps SO(3) into the unit quaternions, treated as the Euclidean space R4.
  • As θ approaches π from below, a quaternion component reaches −1, establishing the discontinuity.
  • Unit quaternions are not a continuous representation for 3D rotations.
  • The Cayley transformation diverges at 180°, so it is not a representation for SO(3).
  • A continuous representation should map each axis-rotation curve to a 2D curve homeomorphic to a circle with nearby colors remaining spatially nearby.
  • The visualization shows incorrect topology for unit quaternion, axis-angle, and Euler-angle representations.

G. Additional Empirical Results

Additional experiments visualize worst-case inverse-kinematics frames for networks trained with quaternion and 6D rotation representations.

  • The additional results compare the two highest-pose-error test frames for quaternion and 6D networks in both directions.
  • The figure pairs each network’s worst frames with corresponding outputs from the other representation.

G.1. Visualization of Inverse Kinematics Test Result

Worst-case inverse-kinematics frames show qualitatively poorer quaternion outputs than 6D outputs.

  • For the worst error frames, the quaternion representation produces bad qualitative results while the 6D representation still creates a reasonable pose.

G.2. Additional Sanity test

Additional sanity tests compare alternative rotation representations and training configurations, again favoring the continuous 5D and 6D representations.

  • Geodesic-loss and alternative-sampling sanity tests produce results similar to those in the main paper.
  • The tested alternatives include the 3D Rodriguez vector and quaternions constrained to one hemisphere.
  • Errors for the Rodriguez vector and hemisphere-constrained quaternions are significantly worse than for the 5D and 6D representations.
  • Figure 8 labels Quat, Quat-hemi, AxisA, and Rodriguez for the compared representations.
Loading 1812.07035v4…