In mathematics, in particular functional analysis, the singular values of a compact operator acting between Hilbert spaces and , are the square roots of the (necessarily non-negative) eigenvalues of the self-adjoint operator (where denotes the adjoint of ).

The singular values are non-negative real numbers, usually listed in decreasing order (σ1(T), σ2(T), …). The largest singular value σ1(T) is equal to the operator norm of T (see Min-max theorem).

Visualization of a singular value decomposition (SVD) of a 2-dimensional, real shearing matrix M. First, we see the unit disc in blue together with the two canonical unit vectors. We then see the action of M, which distorts the disc to an ellipse. The SVD decomposes M into three simple transformations: a rotation V*, a scaling Σ along the rotated coordinate axes and a second rotation U. Σ is a (square, in this example) diagonal matrix containing in its diagonal the singular values of M, which represent the lengths σ1 and σ2 of the semi-axes of the ellipse.

If T acts on Euclidean space , there is a simple geometric interpretation for the singular values: Consider the image by of the unit sphere; this is an ellipsoid, and the lengths of its semi-axes are the singular values of (the figure provides an example in ).

The singular values are the absolute values of the eigenvalues of a normal matrix A, because the spectral theorem can be applied to obtain unitary diagonalization of as . Therefore, .

Most norms on Hilbert space operators studied are defined using singular values. For example, the Ky Fan-k-norm is the sum of first k singular values, the trace norm is the sum of all singular values, and the Schatten norm is the pth root of the sum of the pth powers of the singular values. Note that each norm is defined only on a special class of operators, hence singular values can be useful in classifying different operators.

In the finite-dimensional case, a matrix can always be decomposed in the form , where and are unitary matrices and is a rectangular diagonal matrix with the singular values lying on the diagonal. This is the singular value decomposition.

Basic properties

edit

For  , and  .

Min-max theorem for singular values. Here   is a subspace of   of dimension  .

 

Matrix transpose and conjugate do not alter singular values.

 

For any unitary  

 

Relation to eigenvalues:

 

Relation to trace:

 .

If   is full rank, the product of singular values is  .

If   is full rank, the product of singular values is  .

If   is full rank, the product of singular values is  .

The smallest singular value

edit

The smallest singular value of a matrix A is σn(A). It has the following properties for a non-singular matrix A:

  • The 2-norm of the inverse matrix (A-1) equals the inverse σn-1(A).[1]: Thm.3.3 
  • The absolute values of all elements in the inverse matrix (A-1) are at most the inverse σn-1(A).[1]: Thm.3.3 

Intuitively, if σn(A) is small, then the rows of A are "almost" linearly dependent. If it is σn(A) = 0, then the rows of A are linearly dependent and A is not invertible.

Inequalities about singular values

edit

See also.[2]

Singular values of sub-matrices

edit

For  

  1. Let   denote   with one of its rows or columns deleted. Then  
  2. Let   denote   with one of its rows and columns deleted. Then  
  3. Let   denote an   submatrix of  . Then  

Singular values of A + B

edit

For  

  1.  
  2.  

Singular values of AB

edit

For  

  1.  
  2.  

For  [3]  

Singular values and eigenvalues

edit

For  .

  1. See[4]  
  2. Assume  . Then for  :
    1. Weyl's theorem  
    2. For  .  

History

edit

This concept was introduced by Erhard Schmidt in 1907. Schmidt called singular values "eigenvalues" at that time. The name "singular value" was first quoted by Smithies in 1937. In 1957, Allahverdiev proved the following characterization of the nth singular number:[5]

 

This formulation made it possible to extend the notion of singular values to operators in Banach space. Note that there is a more general concept of s-numbers, which also includes Gelfand and Kolmogorov width.

See also

edit

References

edit
  1. ^ a b Demmel, James W. (January 1997). Applied Numerical Linear Algebra. Society for Industrial and Applied Mathematics. doi:10.1137/1.9781611971446. ISBN 978-0-89871-389-3.
  2. ^ R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, Cambridge, 1991. Chap. 3
  3. ^ X. Zhan. Matrix Inequalities. Springer-Verlag, Berlin, Heidelberg, 2002. p.28
  4. ^ R. Bhatia. Matrix Analysis. Springer-Verlag, New York, 1997. Prop. III.5.1
  5. ^ I. C. Gohberg and M. G. Krein. Introduction to the Theory of Linear Non-selfadjoint Operators. American Mathematical Society, Providence, R.I.,1969. Translated from the Russian by A. Feinstein. Translations of Mathematical Monographs, Vol. 18.