PaperlayerView exact revision on arXiv

Source-linked AI summary

The Theory of Quaternion Matrix Derivatives

Dongpo Xu, Danilo P. Mandic

arXiv:1410.0563v1math.GM

TL;DR

Quaternion matrix optimization needs derivatives that handle non-analytic functions, while traditional product-rule use is incorrect for quaternion products. The paper develops a GHR-based matrix calculus with product and chain rules, and presents its use in signal-processing optimization.

  • Problem

    Quaternion signal-processing objectives are often non-analytic, and existing methods incorrectly use the traditional product rule for quaternion products.

    Source: §I · p. 1

  • Method

    The paper develops quaternion matrix derivatives from GHR calculus, with matrix product and chain rules and direct calculation without function differentials.

    Source: §I · pp. 1–2, A. Product Rule · p. 6, B. Chain Rule · p. 6

  • Results

    The proposed framework provides quaternion matrix product and chain rules and supports gradient-based optimization results, including stationary-point and maximum-change conditions.

    Source: A. Product Rule · p. 6, B. Chain Rule · p. 6, §VIII · p. 11

  • Takeaways & Limitations

    The framework is illustrated on typical gradient-based signal-processing optimization problems, with key results summarized in tables.

    Source: §VIII · p. 11

Abstract

from arXiv · show

A systematic theory is introduced for calculating the derivatives of quaternion matrix function with respect to quaternion matrix variables. The proposed methodology is equipped with the matrix product rule and chain rule and it is able to handle both analytic and nonanalytic functions. This corrects a flaw in the existing methods, that is, the incorrect use of the traditional product rule. In the framework introduced, the derivatives of quaternion matrix functions can be calculated directly without the differential of this function. Key results are summarized in tables. Several examples show how the quaternion matrix derivatives can be used as an important tool for solving problems related to signal processing.

I. INTRODUCTION

Quaternion matrix optimization needs derivatives that work beyond restrictive analytic-function conditions and account for noncommutative multiplication. The paper develops a GHR-based matrix calculus, while introducing the quaternion notation and algebra used throughout.

I. INTRODUCTION

  • Quaternion analyticity conditions limit usable functions, while real-valued optimization objectives are often non-analytic and pseudo-derivatives can be cumbersome.The introduction notes that real-component differentiation becomes tedious even for simple algorithms.Source: §I · p. 1
  • The paper extends GHR calculus to quaternion matrix functions, deriving matrix product and chain rules and direct derivatives without first taking function differentials.The vec operator and Jacobian support the calculus, whose rules reduce to scalar rules for order-one matrices.Source: §I · pp. 1–2
  • Quaternion matrix derivatives support optimization problems, with the Hermitian of the derivative giving the direction of maximum rate of change.The paper presents this as a generalization of results for scalar functions of vector variables.Source: §I · p. 2

A. Notations

  • The notation section distinguishes matrices, column vectors, and scalars, and defines vec as stacking a matrix’s columns.It also specifies conjugate, transpose, Hermitian, Kronecker, and Hadamard notation.Source: A. Notations · p. 2

II. QUATERNION ALGEBRA

  • A quaternion has real and three imaginary components; its product is noncommutative, so generally pq ≠ qp.The paper defines its scalar and vector parts and states the conjugation and product properties used in later calculus.Source: §II · p. 2

III. ANALYTIC VERSUS NON-ANALYTIC FUNCTIONS

Traditional quaternion derivatives apply only to analytic functions, but engineering objectives are often non-analytic. The paper uses GHR calculus to handle these functions with product and chain rules suited to quaternion multiplication.

III. ANALYTIC VERSUS NON-ANALYTIC FUNCTIONS

  • Quaternion analytic-function conditions are restrictive: the traditional derivative conditions allow only linear functions and constants, while CRF excludes even the identity function.Slice monogenic functions include polynomials and power series with right-sided quaternion coefficients, but their products and compositions are generally not slice monogenic.Source: §III · p. 3
  • Engineering objectives are often real-valued and non-analytic, so the traditional quaternion derivative cannot be used for them.The passage gives minimization or maximization objectives as the context.Source: §III · p. 3
  • HR calculus extends classical complex CR calculus to quaternions, but its traditional product rule is invalid: for f(q) = |q|^2, the cited calculation gives 2q + q*, a difficulty GHR calculus addresses with product and chain rules.Source: §III · p. 3
  • GHR derivatives have distinct left and right forms because quaternion products are noncommutative, though the forms agree for real-valued functions.The paper focuses mainly on left GHR derivatives because they have convenient properties consistent with common sense.Source: §III · p. 4

IV. QUATERNION DIFFERENTIALS

The section develops quaternion matrix differentials from scalar differentials and first-order terms, then uses their independence properties to identify derivatives. It summarizes key differential results in tables.

IV. QUATERNION DIFFERENTIALS

  • The differential of a matrix function is obtained by retaining the first-order terms in its change.The procedure first calculates the difference and separates first-order from higher-order terms.Source: §IV · p. 4
  • For product-conforming quaternion matrix variables P and Q, their product differential follows a generalized product relation.The paper presents key quaternion matrix differential results in Table II.Source: §IV · p. 4, Table II · p. 5
  • The paper also summarizes the connection between quaternion differentials and derivatives for different function types in Table III.The differential results generalize corresponding complex matrix differentials.Source: §IV · p. 4, Table III · p. 5
  • The independence of the real quaternion components and their differentials supports identifying the separate GHR derivatives from a matrix differential.Lemma 1 concludes that if the stated relation holds for all dQ, then each A_n is zero.Source: §IV · p. 4

V. DEFINITION OF QUATERNION MATRIX DERIVATIVES

This section defines quaternion matrix derivatives as Jacobian matrices and extends derivative and gradient concepts from vector functions to matrix functions. The resulting derivatives are unique and connect matrix differentials to derivatives across function types.

V. DEFINITION OF QUATERNION MATRIX DERIVATIVES

  • For an M × 1 vector function of q, the M × N matrices are called its derivatives or Jacobian matrices.This extends the scalar-function derivatives of a vector variable to vector-valued functions.Source: §V · p. 5
  • For a matrix function F of Q, the GHR derivatives with respect to Q and its involutions are MP × NS Jacobian matrices.The paper assumes the elements of Q are linearly independent.Source: §V · p. 5
  • The transposes of the Jacobian matrices are called gradients.The paper extends this derivative framework from vector functions to matrix functions.Source: §V · p. 5
  • The matrix derivatives are unique, and Table III connects differentials and derivatives for the listed function types.The matrix-variable case generalizes the complex-valued case studied in.Source: §V · pp. 5–6

A. Product Rule

The section generalizes quaternion scalar product rules to matrix variables and states a chain rule for composite matrix functions. The rules provide a calculus framework for quaternion matrix derivatives.

A. Product Rule

  • Theorem 5.1 generalizes the quaternion scalar product rules to products of quaternion matrix functions.The stated setup is H = FG, with F mapping to HM×R and G mapping to HR×P.Source: A. Product Rule · p. 6

B. Chain Rule

  • Theorem 5.2 gives GHR derivatives for a composite function H(Q) = F(G(Q)) when the inner and outer functions have GHR derivatives.The theorem specifies interior-point conditions for Q and G(Q).Source: B. Chain Rule · p. 6

VI. QUATERNION OPTIMIZATION THEORY

For real-valued functions of quaternion matrices, the section gives necessary stationary-point conditions and identifies the gradient direction of maximum change. These results connect quaternion matrix derivatives to steepest-descent methods.

A. Stationary Points

  • Theorem 6.1 gives five equivalent necessary conditions for finding stationary points of a real-valued function of a quaternion matrix.One condition follows from the real-vector representation, where a stationary point has zero derivative.Source: A. Stationary Points · p. 6

B. Direction of maximum rate of change

  • The Hermitian of the derivative with respect to the quaternion matrix defines the direction of maximum rate of change with respect to vec(Q).The section notes that this direction is used in steepest-descent methods, including quaternion adaptive filters.Source: B. Direction of maximum rate of change · p. 7
  • The steepest-descent update uses that maximum-change direction, with η > 0 as the step size.Qn+1 is the unknown matrix after n iterations.Source: B. Direction of maximum rate of change · p. 7

VII. DEVOLOPMENT OF QUATERNION DERIVATIVES

The section collects scalar quaternion derivative results and applies the matrix-derivative framework to quaternion filtering. It derives a QLMS update and relates it to complex-valued LMS.

A. Derivatives of f(q)

  • Scalar quaternion-function derivative results are collected in Table IV for quaternion constants α, β, and λ and a quaternion-valued variable q.The scalar-function case is discussed in relation to [38].Source: A. Derivatives of f(q) · p. 7

B. Derivatives of f(q)

  • The function q^H A q arises in quaternion filter optimization and array signal processing, with q as filter coefficients and A as the input covariance matrix.The cited passage assumes A^H = A and points to derivative results in Table V.Source: B. Derivatives of f(q) · p. 7

1) Quaternion Least Mean Square:

  • The QLMS algorithm is a generic generalization of complex-valued LMS to quaternion vectors.The section derives it using GHR calculus for a real-valued cost function.Source: 1) Quaternion Least Mean Square · p. 7

2) Quaternion Widely Linear Least Mean Square:

This section derives a widely linear QLMS algorithm from a quaternion widely linear model. It calculates gradients for the model’s weights and uses them to form the weight update.

2) Quaternion Widely Linear Least Mean Square:

3) Quaternion Affine Projection Algorithm:

The quaternion affine projection algorithm adapts the weight vector by minimizing its squared change subject to a signal constraint. The section derives the update using Lagrange multipliers and notes regularization for the normalization matrix.

3) Quaternion Affine Projection Algorithm:

  • QAPA minimizes the squared Euclidean norm of the weight-vector change subject to d^T(n) = w^H(n+1)Q(n).Here d(n) is the desired-signal vector and Q(n) contains S past input vectors.Source: 3) Quaternion Affine Projection… · p. 9
  • Lagrange multipliers are used to solve the constrained optimization, yielding a weight update for QAPA.The update is obtained by setting the cost-function derivative to zero and solving for λ.Source: 3) Quaternion Affine Projection… · p. 9
  • A small regularization term εI is added to prevent Q^H(n)Q(n) from becoming singular, while η controls convergence and steady-state performance.Source: 3) Quaternion Affine Projection… · p. 9
  • Table VI presents derivatives of functions of the type f(Q).Source: Table VI · p. 10

C. Derivatives of f(Q)

The section develops quaternion-matrix gradients and applies them to reformulate steepest descent and derive a gradient for an overdetermined least-squares problem.

C. Derivatives of f(Q)

  • The scalar-function gradient notation uses derivatives with respect to Q and its involutions, extending real- and complex-valued cases to quaternions.The gradients are with respect to Qµ and Qµ∗.Source: C. Derivatives of f(Q) · p. 10
  • Steepest descent is reformulated with step size η > 0, and Table VI summarizes results for functions of the type f(Q).The matrices and constants in the table are chosen so the functions are well defined.Source: C. Derivatives of f(Q) · p. 10

1) Quaternion Matrix Least Squares:

2(BHQHAHA)T BH

The section derives matrix-function derivatives and applies them to a least-squares normal equation. It concludes that the GHR-based framework supplies product and chain rules and supports gradient optimization in signal-processing problems.

2(BHQHAHA)T BH

  • When AᴴA and BBᴴ are invertible, the least-squares system has a unique solution after setting the derivative condition to zero and obtaining a normal equation.The normal equation follows from setting (95) to zero.Source: 2(BHQHAHA)T BH · p. 11

D. Derivatives of F (Q)

  • Elementary matrix-function derivatives, beginning with Q^n, are developed for functions used in nonlinear adaptive filters and neural networks.Other useful examples are summarized in Table VII, with matrices and constants chosen so the functions are well defined.Source: D. Derivatives of F (Q) · p. 11

1) Derivatives of Power Function:

2) Derivatives of Exponential Function:

  • The exponential examples extend the quaternion scalar-variable case to matrix variables; trigonometric and hyperbolic derivatives can likewise be derived from the exponential function.Source: 2) Derivatives of Exponential Function · p. 11

VIII. CONCLUSIONS

  • The GHR-based framework introduces matrix product and chain rules and develops gradient-optimization results, illustrated with signal-processing problems.The results include stationary-point conditions, maximum-change directions, and steepest-descent methods.Source: §VIII · p. 11

Related papers

Loading 1410.0563v1…