PaperlayerView exact revision on arXiv

Source-linked AI summary

Quaternion Gradient and Hessian

Dongpo Xu, Danilo P. Mandic

arXiv:1406.3587v1math.NAcs.LGcs.NA

TL;DR

Real-valued functions of quaternion variables are non-analytic, complicating optimization that needs gradients and Hessians. The paper defines these using GHR calculus, enabling direct quaternion-field algorithm derivation; the proposed gradient and Hessian correspond to real counterparts through simple linear transforms, and the framework derives optimization algorithms without increasing problem dimensionality.

  • Problem

    Real-valued objectives of quaternion variables are non-analytic, so current optimization procedures often reformulate them over four real components to calculate pseudo-derivatives.

    Source: §1 · p. 2

  • Method

    The paper defines quaternion gradients and Hessians using GHR calculus, whose product and chain rules support direct derivations in the quaternion field.

    Source: §1 · pp. 2–3

  • Results

    The proposed quaternion gradient and Hessian relate to their real counterparts by simple invertible linear transforms, and the derived optimization algorithms operate directly in the quaternion field without increasing problem dimensionality.

    Source: §3 · p. 8, §6 · p. 18

  • Takeaways & Limitations

    The GHR framework simplifies the derivation of first- and second-order iterative optimization procedures in the quaternion field.

    Source: §6 · p. 18

    The upper bound on the approximation error between the quaternion Newton algorithm and its approximation is left unestimated.

    Source: §5.1 · p. 15

Abstract

from arXiv · show

The optimization of real scalar functions of quaternion variables, such as the mean square error or array output power, underpins many practical applications. Solutions often require the calculation of the gradient and Hessian, however, real functions of quaternion variables are essentially non-analytic. To address this issue, we propose new definitions of quaternion gradient and Hessian, based on the novel generalized HR (GHR) calculus, thus making possible efficient derivation of optimization algorithms directly in the quaternion field, rather than transforming the problem to the real domain, as is current practice. In addition, unlike the existing quaternion gradients, the GHR calculus allows for the product and chain rule, and for a one-to-one correspondence of the proposed quaternion gradient and Hessian with their real counterparts. Properties of the quaternion gradient and Hessian relevant to numerical applications are elaborated, and the results illuminate the usefulness of the GHR calculus in greatly simplifying the derivation of the quaternion least mean squares, and in quaternion least square and Newton algorithm. The proposed gradient and Hessian are also shown to enable the same generic forms as the corresponding real- and complex-valued algorithms, further illustrating the advantages in algorithm design and evaluation.

1. Introduction.

Real-valued objectives of quaternion variables are non-analytic, so optimization commonly splits quaternions into real components for differentiation. The paper introduces GHR-based gradient and Hessian tools for direct quaternion-field algorithm derivation and reviews the algebra underlying those tools.

1. Introduction.

  • Real-valued objectives of quaternion variables are non-analytic, making real-component reformulation and gradient calculation awkward and tedious.Current procedures use pseudo-derivatives with respect to the four independent real components.Source: §1 · p. 2
  • GHR calculus equips quaternion analysis with product and chain rules, enabling optimization methods to be derived directly in the quaternion field.The paper contrasts this with the tedious pseudo-derivative approach.Source: §1 · p. 2
  • The proposed quaternion gradient and Hessian relate to their real counterparts through invertible linear transforms and support first- and second-order algorithm derivations.The paper also proposes a way to derive algorithms directly in HN×1 rather than the redundant augmented space H4N×1.Source: §1 · p. 3

1.1. Quaternion Algebra

  • Quaternions form an associative but noncommutative algebra over R, represented using the basis {1, i, j, k}.Their imaginary units satisfy i² = j² = k² = ijk = −1.Source: §1.1 · p. 3
  • Quaternion multiplication is noncommutative, while the conjugate reverses product order: (pq)* = q*p*.A quaternion also has real and vector parts, and can be written in polar form using a pure unit quaternion and an angle.Source: §1.1 · p. 3
  • Quaternion rotation describes a 3D rotation of a quaternion’s vector part by angle 2θ about the vector part of µ.For pure unit µ, this rotation becomes quaternion involution.Source: §1.1 · p. 4

2. Quaternion Derivatives

Quaternion differentiation approaches have made gradient-based optimization cumbersome, and HR calculus still lacks the traditional product rule. The section presents GHR derivatives and their product and chain rules, then distinguishes left and right derivatives.

2. Quaternion Derivatives

  • Quaternion pseudo-derivatives are cumbersome and make the derivation of gradient-based optimization algorithms prone to difficulty.HR calculus offers a more elegant alternative, but does not support the traditional product rule.Source: §2 · pp. 4–5
  • GHR calculus addresses HR calculus’s product-rule limitation with product and chain rules for quaternion derivatives.The GHR derivatives generalize HR derivatives, which are a special case of the right GHR derivative for µ in {1, i, j, k}.Source: §2 · pp. 5–6
  • The left GHR derivatives are defined for f with respect to qµ and qµ*, with component derivatives taken with respect to the four real coordinates in an orthogonal quaternion basis.The passage also distinguishes right GHR derivatives.Source: §2 · p. 5
  • Left and right GHR derivatives differ because quaternion products are noncommutative, but coincide when f is real-valued.The paper focuses on left GHR derivatives for their convenient properties.Source: §2 · p. 6

3. Quaternion Gradient

The paper defines quaternion gradients using GHR calculus, whose product and chain rules support optimization derivations. An invertible transform relates the augmented quaternion gradient to the real gradient, and this correspondence is used to derive quaternion gradient descent.

3. Quaternion Gradient

  • The GHR-based quaternion gradient addresses existing gradient limitations by providing product and chain rules for quaternion-valued functions.Source: §3 · pp. 6–7
  • The augmented quaternion gradient and real gradient are related by a simple invertible linear transformation, yielding equivalent necessary-and-sufficient stationary-point conditions.Source: §3 · pp. 8–9

3.1. Quaternion Gradient Descent Algorithm

  • Quaternion gradient descent is derived by translating the real-coordinate gradient-descent update through the correspondence between real coordinates and the augmented quaternion vector.Source: §3.1 · p. 9
  • For a real-valued scalar, the quaternion gradient is taken with respect to q* rather than q, extending the complex CR calculus.Source: §3.1 · p. 9

4. Quaternion Hessian

The section defines a quaternion Hessian and uses it to develop second-order Taylor expansions directly in quaternion space. It relates the augmented quaternion Hessian to the real Hessian, including shared definiteness and conditioning properties.

4. Quaternion Hessian

  • The quaternion Hessian is defined from second-order GHR derivatives, and the resulting Hessian is Hermitian.For quaternion matrices, the Hermitian operator cannot be replaced by transpose.Source: §4 · pp. 10–12
  • The Taylor-series derivation uses an augmented quaternion space before combining expansions to obtain the form directly in quaternion space.The augmented Hessian is 4N × 4N.Source: §4 · pp. 11, 13
  • The Taylor expansion for a non-analytic real scalar function includes augmented terms involving quaternion involutions, unlike the complex expansion for an analytic function.The section also expresses the expansion directly in the quaternion domain.Source: §4 · p. 13
  • The eigenvalues of the quadrivariate real Hessian are quadruple those of the augmented quaternion Hessian, while both have the same positive-definiteness properties and condition number.The section identifies these shared properties as relevant to quaternion Newton minimization.Source: §4 · p. 13

5. Application Examples

The application examples use quaternion gradients and Hessians in Newton, least-mean-square, and least-squares procedures. They derive quaternion-domain updates and solutions, with the Newton approximation's error bound left for future estimation.

5. Application Examples

  • Quaternion gradients and Hessians are presented as useful for analytic and numerical solutions of quaternion-valued parameter-estimation problems.The Newton method uses the Hessian but can be expensive to calculate and store.Source: §5 · p. 14, §5.1 · p. 14

5.1. Quaternion Newton Algorithm

  • The Newton derivation yields an update directly in quaternion space, and the approximation avoids inverting the Schur complement.The authors leave estimation of the approximation error's upper bound for future work.Source: §5.1 · p. 15

5.2. Quaternion Least Mean Square

  • The GHR-derived QLMS differs from earlier QLMS variants through its treatment of quaternion noncommutativity and the novel product rule.The resulting update is stated to equal the pseudo-gradient-derived QLMS and generalize complex LMS to quaternion vectors.Source: §5.2 · pp. 16–17

5.3. Quaternion Least Squares

  • For nonsingular AᴴA, the quaternion least-squares problem has a unique solution, and the approximation error vanishes in this case.The solution is described as formally equivalent to a real representation, while the quaternion matrix equation can be solved directly in the quaternion field.Source: §5.3 · pp. 17–18

6. Conclusions

The paper introduces quaternion gradients and Hessians based on GHR calculus and relates them to real counterparts through simple linear transforms. It derives direct quaternion-domain optimization algorithms and presents the framework as a basis for generic real- and complex-algorithm extensions.

6. Conclusions

  • The proposed quaternion gradient and Hessian for smooth real functions are related to their real counterparts by simple linear transforms.The framework derives gradient descent, least-squares, and Newton algorithms directly in the quaternion field without increasing problem dimensionality.Source: §6 · p. 18
  • GHR calculus supplies product and chain rules for quaternion optimization and simplifies deriving first- and second-order procedures.The paper identifies conjugate-gradient and quasi-Newton methods as readily obtainable extensions.Source: §6 · p. 18

Appendix: The QLMS derivation applied component-wise

The appendix gives a componentwise QLMS derivation using gradients with respect to the real-valued weight components. It applies the traditional product rule to those component gradients before forming the update expression.

Appendix: The QLMS derivation applied component-wise

Loading 1406.3587v1…