PaperlayerView exact revision on arXiv

Source-linked AI summary

The Moore-Penrose Pseudoinverse. A Tutorial Review of the Theory

J. C. A. Barata, M. S. Hussein

arXiv:1110.6882v1math-ph

TL;DR

The paper addresses the need for a general treatment of pseudoinverses applicable when ordinary matrix inversion fails and when least-squares solutions are required. It develops a self-contained review using existence proofs, Tikhonov regularization, and the Spectral Theorem, yielding explicit computation expressions that the authors regard as algorithmically simpler than the usual SVD-based approach. The pseudoinverse supplies the minimizing set for least-squares problems, while the paper notes that (AB)+ is not always B+A+.

  • Problem

    Ordinary inversion does not cover singular or non-square matrices, and linear systems may lack solutions or have non-unique solutions despite requiring least-squares approximations.

  • Method

    The paper gives two existence proofs and derives explicit formulas using A, A∗, eigenvalues of AA∗ or A∗A, Tikhonov regularization, and the Spectral Theorem.

  • Results

    The Moore-Penrose pseudoinverse provides all vectors minimizing ∥Ax′ − y∥, and the paper presents expressions it considers algorithmically simpler than the usual SVD-based computation.

  • Takeaways & Limitations

    The review connects pseudoinverse theory with least-squares problems arising in applications such as imaging, data analysis, and linear integral equations.

  • Takeaways & Limitations

    The identity (AB)+ = B+A+ does not always hold for the Moore-Penrose pseudoinverse.

Abstract

from arXiv · show

In the last decades the Moore-Penrose pseudoinverse has found a wide range of applications in many areas of Science and became a useful tool for physicists dealing, for instance, with optimization problems, with data analysis, with the solution of linear integral equations, etc. The existence of such applications alone should attract the interest of students and researchers in the Moore-Penrose pseudoinverse and in related sub jects, like the singular values decomposition theorem for matrices. In this note we present a tutorial review of the theory of the Moore-Penrose pseudoinverse. We present the first definitions and some motivations and, after obtaining some basic results, we center our discussion on the Spectral Theorem and present an algorithmically simple expression for the computation of the Moore-Penrose pseudoinverse of a given matrix. We do not claim originality of the results. We rather intend to present a complete and self-contained tutorial review, useful for those more devoted to applications, for those more theoretically oriented and for those who already have some working knowledge of the sub ject.

1 Introduction, Motivation and Notation

The section introduces the Moore-Penrose pseudoinverse as a generalization of matrix inversion for singular or non-square matrices and motivates it through least-squares problems. It also previews a self-contained treatment using the Spectral Theorem and an algorithmically simple computation formula, alongside applications and notation.

  • Motivation: The Moore-Penrose pseudoinverse generalizes the inverse of a square matrix to singular and non-square matrices.It is presented as a tool for optimization and least-squares problems.
  • Least-squares motivation: When Ax = y has no solution or multiple solutions, the minimizing set contains vectors that minimize the Euclidean residual norm ∥Ax′ − y∥.The pseudoinverse provides this minimizing set, as established later in Theorem 6.1.
  • Computational approach: The paper uses the Spectral Theorem and Tikhonov regularization to derive an algorithmically simpler formula for computing A+.The formula is expressed using A∗ and the distinct eigenvalues of A∗A, identified in the paper as singular values.
  • Applications: Applications include imaging, seismic inversion, data analysis, principal component analysis, image compression, image restoration, and linear Fredholm integral equations.Examples include MRI, fMRI, PET, MSI, electroencephalographic source localization, astronomy, and gene-expression analysis.
  • Paper scope: The paper is organized around definitions and basic properties, further theoretical results, Tikhonov regularization, the Spectral Theorem, and least-squares applications.The stated objective is a self-contained introduction useful to both application-oriented and theoretically oriented readers.
  • Notation: The section establishes notation for complex vector spaces, matrices, scalar products, orthogonality, transposes, adjoints, diagonal matrices, subspaces, and spectra.These definitions support the subsequent development of pseudoinverse theory.

2 The Moore-Penrose Pseudoinverse. Definition and First Properties

The Moore-Penrose pseudoinverse narrows generalized inverses through self-adjointness conditions, yielding a unique inverse-like matrix for arbitrary rectangular or singular matrices. The section introduces its existence, basic special cases, and explicit computational approaches.

  • A generalized inverse satisfies algebraic conditions such as BAB = B, but these conditions alone do not ensure uniqueness.
  • The Moore-Penrose pseudoinverse extends the usual inverse to singular square and non-square matrices.
  • The Moore-Penrose pseudoinverse adds self-adjointness conditions on AA+ and A+A to narrow the generalized-inverse definition.
  • The Moore-Penrose pseudoinverse is unique whenever it exists, equals A^-1 for nonsingular square matrices, and maps the zero matrix to the corresponding zero matrix.
  • For arbitrary matrices, the paper gives two existence proofs and explicit formulas using A, A* and eigenvalues of AA* or A*A.
  • Special-case formulas cover matrices for which AA* or A*A is invertible, while the general construction combines the Spectral Theorem with Tikhonov regularization.

3 Further Properties of the Moore-Penrose Pseudoinverse

The section develops structural properties of the Moore-Penrose pseudoinverse, including scaling, composition constraints, projector identities, and relationships among kernels and ranges. These results support later least-squares optimization and computation.

  • The pseudoinverse obeys the nonzero scalar-scaling rule (zA)+ = z^-1A+ for z ∈ C.
  • Unlike ordinary inverses, (AB)+ is not generally equal to B+A+ for compatible matrices.
  • The identities established in Proposition 3.1 provide properties used in subsequent theory and in the proof of least-squares optimization results.
  • The formulas A+ = A*(AA*)+ and A+ = (A*A)+A* generalize inverse formulas when AA* or A*A is not invertible.
  • P1 = 1n − A+A and P2 = 1m − AA+ are orthogonal projectors.
  • These projectors connect the kernel and range spaces: Ker(A) = Ran(P1), Ran(A) = Ker(P2), and Ker(A+) = Ran(P2).

4 Tikhonov’s Regularization and Existence Theorem for the Moore-Penrose Pseudoinverse

The section uses Tikhonov regularization to replace potentially singular matrices with invertible perturbations, then proves that the resulting limits exist and equal the Moore-Penrose pseudoinverse. This establishes existence for every matrix, including cases with zero eigenvalues.

  • Regularization: Tikhonov regularization replaces AA* or A*A by AA* + µ1 or A*A + µ1, making the perturbed matrices invertible for suitable nonzero µ.The procedure originated in approximations to Fredholm equations and is applied here to finite matrix problems.
  • Regularization: The regularized equation (A*A + µ1)xµ = A*y is used because A*A may not be invertible.The paper asks whether xµ converges as |µ| approaches zero.
  • Existence: The limits of the regularized expressions exist and coincide, defining an element of Mat(C,n,m).Lemmas 4.1 and 4.2 establish equality of the relevant limits.
  • Proof strategy: The spectral-projector proof handles singular cases by isolating the zero-eigenvalue projector and verifying the generalized-inverse relations.The argument establishes ABA = A and BAB = B in the zero-eigenvalue case.
  • Existence: Theorem 4.3 proves that these limits converge to A+ for every matrix A.The proof treats separately the cases where zero is absent from or present in the spectrum of AA*.

5 The Moore-Penrose Pseudoinverse and the Spectral Theorem

This section derives spectral formulas for the Moore-Penrose pseudoinverse from the spectral representations of AA* and A*A. The resulting formulas provide an algorithm based on eigenvalues, spectral projectors, and polynomial evaluation.

  • Spectral formulas: The proof of Theorem 4.3 establishes the spectral formulas collected in Theorem 5.1.The formulas are obtained from the spectral decomposition of AA* and analogously for A*A.
  • Spectral formulas: The nonzero eigenvalues of AA* and A*A coincide, although the two matrices may have different zero-eigenspaces.This connects the two spectral representations used for computation.
  • Algorithm: Expressions (39) and (40) compute the pseudoinverse from the eigenvalues of AA* or A*A and polynomials of those matrices.The paper presents this as a general algorithm for every nonzero matrix A.
  • Algorithm: The spectral-theorem approach is algorithmically simpler than the usual singular-values-decomposition method.The section’s formulas are based on spectral projections and polynomial expressions.

6 The Moore-Penrose Pseudoinverse and Least Squares

The Moore-Penrose pseudoinverse solves linear least-squares problems when exact solutions may be absent or nonunique. The complete minimizing set is obtained by adding arbitrary vectors from the kernel of A to A+y.

  • Least-squares problem: When Ax = y has no solution or lacks uniqueness, least squares minimizes the Euclidean residual norm ||Ax − y||.The minimizing set contains the best approximants to the linear equation.
  • Minimizing set: Theorem 6.1 identifies the minimizing set as A+y + Ker(A).Thus A+y is one minimizing solution, while kernel vectors generate the remaining minimizing solutions.
  • Exact solutions: The residual vanishes exactly when y belongs to Ran(A), recovering the exact-solvability condition.This condition is expressed through the projection onto the range of A.
  • Geometric foundation: A unique vector y0 in Ran(A) minimizes the distance to y, and y0 − y is orthogonal to Ran(A).This follows from the Best Approximant and Orthogonal Decomposition Theorems.
  • Minimizing set: Every least-squares minimizer differs from A+y by a vector annihilated by A.The kernel term accounts for possible nonuniqueness.

A A Brief Review of Hilbert Space Theory and Linear Algebra

The appendix reviews the Hilbert-space and linear-algebra results used in the paper, including approximation, orthogonal decomposition, orthogonal complements, spectra, and eigenvalues. These results supply the geometric and spectral foundations for the pseudoinverse theory.

  • Hilbert spaces: A Hilbert space is a complete complex inner-product space equipped with the norm induced by its scalar product.The appendix also records the parallelogram identity for this norm.
  • Approximation: The Best Approximant Theorem guarantees a unique nearest point in a convex subset of a Hilbert space.Its proof constructs a minimizing sequence, proves it is Cauchy, and uses completeness and closedness.
  • Orthogonality: The orthogonal complement of any subset of a Hilbert space is a closed linear subspace.This property supports the orthogonal-decomposition arguments used for least squares.
  • Orthogonal decomposition: Every vector decomposes uniquely into a component in a closed subspace and a component in its orthogonal complement.The component in the subspace is the best approximant.
  • Linear algebra: The spectrum of a matrix consists of the scalars for which the shifted matrix lacks an inverse, equivalently the roots of its characteristic polynomial.Such scalars are eigenvalues when they admit nonzero eigenvectors.

The neighborhood of singular matrices

The section develops matrix results around invertibility, similarity, spectra, and diagonalizability. These results support the spectral-theoretic framework used later for pseudoinverse computation.

  • Invertibility and similarity: A + µB is invertible for sufficiently small nonzero |µ| and sufficiently large |µ| when B is nonsingular.The bounds M1 and M2 depend on A and B.
  • Invertibility and similarity: Similar matrices have identical characteristic polynomials, spectra, and geometric multiplicities of eigenvalues.
  • Invertibility and similarity: The products AB and BA have the same characteristic polynomial, spectrum, and geometric eigenvalue multiplicities for square matrices.
  • Invertibility and similarity: For rectangular products, AB and BA share the same nonzero eigenvalues, while their characteristic polynomials differ by powers of x.
  • Diagonalizability: A matrix is diagonalizable exactly when it has n linearly independent eigenvectors.
  • Diagonalizability: A diagonalizable matrix admits a spectral decomposition into distinct eigenvalues and nonzero mutually orthogonal projectors summing to the identity.

Getting the spectral projections

The paper expresses spectral projections through polynomial functional calculus. This gives a constructive way to isolate each distinct eigenspace and establishes uniqueness of the resulting decomposition.

  • Polynomial functional calculus: Polynomial interpolation produces pj(A), the spectral projector associated with eigenvalue αj, because pj(αk) = δj,k.
  • Uniqueness: The spectral decomposition of a diagonalizable matrix is unique.
  • Adjoints and self-adjoint matrices: The adjoint is defined through the inner-product relation and is represented by the conjugate transpose in the canonical basis.
  • Adjoints and self-adjoint matrices: Self-adjoint matrices have real eigenvalues.

Projectors and orthogonal projectors

The section develops orthogonal projectors and the spectral theorem for self-adjoint matrices, then connects these ideas to polar decomposition and singular values.

  • Projectors: An orthogonal projector is a self-adjoint idempotent matrix, and rank-one projectors are constructed from normalized vectors.
  • Projectors: Projectors onto orthogonal unit vectors annihilate one another, so mutually orthogonal eigenspaces can be represented independently.
  • Spectral theorem for self-adjoint matrices: A self-adjoint matrix has an orthonormal eigenbasis with real eigenvalues and a spectral representation as a sum of rank-one projectors.
  • Polar decomposition: The eigenvalues of A∗A are nonnegative, and the resulting square roots are the singular values of A.
  • Polar decomposition: The polar decomposition writes A as a unitary factor times the positive square root of A∗A; the unitary factor is unique when A is nonsingular.

Singular values decomposition

The singular value decomposition follows from polar decomposition and represents a matrix using unitary factors and a diagonal matrix of singular values. The construction extends to rectangular matrices.

  • Square-matrix decomposition: Every square complex matrix has a singular value decomposition with unitary matrices V and W and diagonal S.
  • Square-matrix decomposition: The diagonal entries of S are the singular values, given by the eigenvalues of the positive matrix derived from A∗A.
  • Rectangular matrices: The singular value decomposition generalizes to rectangular matrices by embedding them into square matrices of dimension m + n.

B Singular Values Decomposition and Existence of the Moore-Penrose Pseudoinverse

The section proves existence and identifies the Moore-Penrose pseudoinverse for square and rectangular complex matrices. It first handles diagonal and square matrices, then extends the construction to rectangular matrices through an associated square matrix.

  • Square matrices: For a diagonal matrix, the pseudoinverse uses the reciprocal of each nonzero diagonal entry and zero for each zero entry.The resulting products satisfy the Moore-Penrose relations, with diagonal entries of DD+ equal to either 0 or 1.
  • Square matrices: For a square matrix represented by its singular values decomposition, the pseudoinverse is constructed from the pseudoinverse of its diagonal singular-value matrix.Self-adjointness of the relevant products follows from the diagonal structure of SS+ and the analogous relation for A+A.
  • Rectangular matrices: A rectangular matrix is embedded into an associated square matrix A′, whose Moore-Penrose pseudoinverse exists uniquely.The construction then extracts the rectangular pseudoinverse from (A′)+ using the matrices defined with A′.
  • Rectangular matrices: The extracted matrix satisfies AA+A = A and A+AA+ = A+ through multiplication of the corresponding identities for A′.These are two of the defining Moore-Penrose relations used in the proof.
  • Rectangular matrices: The remaining self-adjointness conditions for AA+ and A+A follow from the self-adjointness of products involving (A′)+.Together with the two algebraic relations, this establishes that the constructed A+ is the Moore-Penrose pseudoinverse.
  • Rectangular matrices: The proof concludes that the expression given for A+ is the Moore-Penrose pseudoinverse of the rectangular matrix A.The result completes the construction for general complex matrices of size m × n.
Loading 1110.6882v1…