6.4 Normal and Self-Adjoint Operators – Linear Algebra, 5th Edition

6.4 Normal and Self-Adjoint Operators

We have seen the importance of diagonalizable operators in Chapter 5. For an operator on a vector space V to be diagonalizble, it is necessary and sufficient for V to contain a basis of eigenvectors for this operator. As V is an inner product space in this chapter, it is reasonable to seek conditions that guarantee that V has an orthonormal basis of eigenvectors. A very important result that helps achieve our goal is Schur’s theorem (Theorem 6.14). The formulation that follows is in terms of linear operators. The next section contains the more familiar matrix form. We begin with a lemma.

Lemma. Let T be a linear operator on a finite-dimensional inner product space V. If T has an eigenvector, then so does T*.

Proof.

Suppose that v is an eigenvector of T with corresponding eigenvalue λ. Then for any xV,

0 = 0 ,   x = ( T λ I ) ( v ) ,   x = v ,   ( T λ I ) * ( x ) = v ,   ( T* λ ¯ I ) ( x ) ,

and hence v is orthogonal to the range of T*λ¯I. So T*λ¯I is not onto and hence is not one-to-one. Thus T*λ¯I has a nonzero null space, and any nonzero vector in this null space is an eigenvector of T* with corresponding eigenvalue λ¯.

Recall (see the exercises of Section 2.1 and see Section 5.4) that a subspace W of V is said to be T-invariant if T(W) is contained in W. If W is T-invariant, we may define the restriction TW: WW by TW(x)=T(x) for all xW. It is clear that TW is a linear operator on W. Recall from Section 5.2 that a polynomial is said to split if it factors into linear polynomials.

Theorem 6.14 (Schur).

Let T be a linear operator on a finite-dimensional inner product space V. Suppose that the characteristic polynomial of T splits. Then there exists an orthonormal basis γ for V such that the matrix [T]γ is upper triangular.

Proof.

By Exercise 12(a) of Section 5.2, there exists an ordered basis β={w1, w2, , wn} for V such that [T]β is upper triangular. Now apply the Gram-Schmidt process to β to obtain an orthogonal basis β={v1, v2, , vn} for V. For each k, 1kn, let

S k = { w 1 ,   w 2 ,   ,   w k } and S k = { v 1 ,   v 2 ,   ,   v k } .

As in the proof of Theorem 6.4, span(Sk)=span(Sk) for all k. By Exercise 12 of Section 2.2, T(wk)span(Sk) for all k. Hence T(vk)span(Sk) for all k, and so [T]β is upper triangular by the same exercise. Finally, let zi=1||vi||vi for all 1in and γ={z1, z2, , zn}. Then γ is an orthonormal basis for V, and [T]γ is upper triangular.

We now return to our original goal of finding an orthonormal basis of eigenvectors of a linear operator T on a finite-dimensional inner product space V. Note that if such an orthonormal basis β exists, then [T]β is a diagonal matrix, and hence [T*]β=[T]β* is also a diagonal matrix. Because diagonal matrices commute, we conclude that T and T* commute. Thus if V possesses an orthonormal basis of eigenvectors of T, then TT*=T*T.

Definitions.

Let V be an inner product space, and let T be a linear operator on V. We say that T is normal if TT*=T*T. An n×n real or complex matrix A is normal if AA*=A*A.

It follows immediately from Theorem 6.10 (p. 356) that T is normal if and only if [T]β is normal, where β is an orthonormal basis.

Example 1

Let T: R2R2 be rotation by θ, where 0<θ<π. The matrix representation of T in the standard ordered basis is given by

A = ( cos   θ sin   θ sin   θ cos   θ ) .

Note that AA*=I=A*A; so A, and hence T, is normal.

Example 2

Suppose that A is a real skew-symmetric matrix; that is, At=A. Then A is normal because both AAt and AtA are equal to A2.

Clearly, the operator T in Example 1 does not even possess one eigenvector. So in the case of a real inner product space, we see that normality is not sufficient to guarantee an orthonormal basis of eigenvectors. All is not lost, however. We show that normality suffices if V is a complex inner product space.

Before we prove the promised result for normal operators, we need some general properties of normal operators.

Theorem 6.15.

Let V be an inner product space, and let T be a normal operator on V. Then the following statements are true.

  1. (a) ||T(x)||=||T*(x)|| for all xV.

  2. (b) TcI is normal for every cF.

  3. (c) If x is an eigenvector of T corresponding to eigenvalue λ, then x is also an eigenvector of T* corresponding to eigenvalue λ¯. That is, if T(x)=λx, then T*(x)=λ¯x.

  4. (d) If λ1 and λ2 are distinct eigenvalues of T with corresponding eigenvectors x1 and x2, then x1 and x2 are orthogonal.

Proof.

Proof. (a) For any xV, we have

| | T ( x ) | | 2 = T ( x ) ,  T ( x ) = T*T ( x ) ,   x = TT* ( x ) ,   x = T* ( x ) ,  T* ( x ) = | | T* ( x ) | | 2 .

The proof of (b) is left as an exercise.

(c) Suppose that T(x)=λx for some xV. Let U=TλI. Then U(x)=0, and U is normal by (b). Thus (a) implies that

0 = | | U ( x ) | | = | | U* ( x ) | | = | | ( T* λ ¯ I ) ( x ) | | = | | ( T* ( x ) λ ¯ x | | .

Hence T*(x)=λ¯x. So x is an eigenvector of T*.

(d) Let λ1 and λ2 be distinct eigenvalues of T with corresponding eigenvectors x1 and x2. Then, using (c), we have

λ 1 x 1 ,   x 2 = λ 1 x 1 ,   x 2 = T ( x 1 ) ,   x 2 = x 1 ,  T* ( x 2 ) = x 1 ,   λ ¯ 2 x 2 = λ 2 x 1 ,   x 2 .

Since λ1λ2, we conclude that x1, x2 =0.

Theorem 6.16.

Let T be a linear operator on a finite-dimensional complex inner product space V. Then T is normal if and only if there exists an orthonormal basis for V consisting of eigenvectors of T.

Proof.

Suppose that T is normal. By the fundamental theorem of algebra (Theorem D.4), the characteristic polynomial of T splits. So we may apply Schur’s theorem to obtain an orthonormal basis β={v1, v2, , vn} for V such that [T]β=A is upper triangular. We know that v1 is an eigenvector of T because A is upper triangular. Assume that v1, v2, , vk1 are eigenvectors of T. We claim that vk is also an eigenvector of T. It then follows by mathematical induction on k that all of the vi’s are eigenvectors of T. Consider any j<k, and let λj denote the eigenvalue of T corresponding to vj. By Theorem 6.15, T*(vj)=λ¯jvj. Since A is upper triangular,

T ( v k ) = A 1 k v 1 + A 2 k v 2 + + A j k v j + + A k k v k .

Furthermore, by the corollary to Theorem 6.5 (p. 345),

A j k = T ( v k ) ,   v j = v k ,  T* ( v j ) = v k ,   λ ¯ j v j = λ j v k ,   v j = 0.

It follows that T(vk)=Akkvk, and hence vk is an eigenvector of T. So by induction, all the vectors in β are eigenvectors of T.

The converse was already proved on page 367.

Interestingly, as the next example shows, Theorem 6.16 does not extend to infinite-dimensional complex inner product spaces.

Example 3

Consider the inner product space H with the orthonormal set S from Example 9 in Section 6.1. Let V=span(S), and let T and U be the linear operators on V defined by T(f)=f1f and U(f)=f1f. Then

T ( f n ) = f n + 1 and U ( f n ) = f n 1

for all integers n. Thus

T ( f m ) ,   f n = f m + 1 ,   f n = δ ( m + 1 ) ,   n = δ m ,   ( n 1 ) = f m ,   f n 1 = f m ,  U ( f n ) .

It follows that U=T*. Furthermore, TT*=I=T*T; so T is normal.

We show that T has no eigenvectors. Suppose that f is an eigenvector of T, say, T(f)=λf for some λ. Since V equals the span of S, we may write

f = i = n m a i f i , where  a m 0.

Hence

i = n m a i f i + 1 = T ( f ) = λ f = i = n m λ a i f i .

Since am0, we can write fm+1 as a linear combination of fn, fn+1, , fm. But this is a contradiction because S is linearly independent.

Example 1 illustrates that normality is not sufficient to guarantee the existence of an orthonormal basis of eigenvectors for real inner product spaces. For real inner product spaces, we must replace normality by the stronger condition that T=T* in order to guarantee such a basis.

Definitions.

Let T be a linear operator on an inner product space V. We say that T is self-adjoint (or Hermitian) if T=T*. An n×n real or complex matrix A is self-adjoint (or Hermitian) if A=A*.

It follows immediately that if β is an orthonormal basis, then T is self-adjoint if and only if [T]β is self-adjoint. For real matrices, this condition reduces to the requirement that A be symmetric.

Before we state our main result for self-adjoint operators, we need some preliminary work.

By definition, a linear operator on a real inner product space has only real eigenvalues. The lemma that follows shows that the same can be said for self-adjoint operators on a complex inner product space. Similarly, the characteristic polynomial of every linear operator on a complex inner product space splits, and the same is true for self-adjoint operators on a real inner product space.

Lemma. Let T be a self-adjoint operator on a finite-dimensional inner product space V. Then

  1. (a) Every eigenvalue of T is real.

  2. (b) Suppose that V is a real inner product space. Then the characteristic polynomial of T splits.

Proof.

(a) Suppose that T(x)=λx for x0. Because a self-adjoint operator is also normal, we can apply Theorem 6.15(c) to obtain

λ x = T ( x ) = T* ( x ) = λ ¯ x .

So λ=λ¯ that is, λ is real.

(b) Let n=dim(V), β be an orthonormal basis for V, and A=[T]β. Then A is self-adjoint. Let TA be the linear operator on Cn defined by TA(x)=Ax for all xCn. Note that TA is self-adjoint because [TA]γ=A, where γ is the standard ordered (orthonormal) basis for Cn. So, by (a), the eigenvalues of TA are real. By the fundamental theorem of algebra, the characteristic polynomial of TA splits into factors of the form tλ. Since each λ is real, the characteristic polynomial splits over R. But TA has the same characteristic polynomial as A, which has the same characteristic polynomial as T. Therefore the characteristic polynomial of T splits.

We are now able to establish one of the major results of this chapter.

Theorem 6.17.

Let T be a linear operator on a finite-dimensional real inner product space V. Then T is self-adjoint if and only if there exists an orthonormal basis β for V consisting of eigenvectors of T.

Proof.

Suppose that T is self-adjoint. By the lemma, we may apply Schur’s theorem to obtain an orthonormal basis β for V such that the matrix A=[T]β is upper triangular. But

A * = [ T ] β * = [ T* ] β = [ T ] β = A .

So A and A* are both upper triangular, and therefore A is a diagonal matrix. Thus β must consist of eigenvectors of T.

The converse is left as an exercise.

We restate this theorem in matrix form in the next section (as Theorem 6.20 on p. 381).

Example 4

As we noted earlier, real symmetric matrices are self-adjoint, and self-adjoint matrices are normal. The following matrix A is complex and symmetric:

A = ( i i i 1 ) and A *= ( i i i 1 ) .

But A is not normal, because (AA*)12=1+i and (A*A)12=1i. Therefore complex symmetric matrices need not be normal.

Exercises

  1. Label the following statements as true or false. Assume that the underlying inner product spaces are finite-dimensional.

    1. (a) Every self-adjoint operator is normal.

    2. (b) Operators and their adjoints have the same eigenvectors.

    3. (c) If T is an operator on an inner product space V, then T is normal if and only if [T]β is normal, where β is any ordered basis for V.

    4. (d) A real or complex matrix A is normal if and only if LA is normal.

    5. (e) The eigenvalues of a self-adjoint operator must all be real.

    6. (f) The identity and zero operators are self-adjoint.

    7. (g) Every normal operator is diagonalizable.

    8. (h) Every self-adjoint operator is diagonalizable.

  2. For each linear operator T on an inner product space V, determine whether T is normal, self-adjoint, or neither. If possible, produce an orthonormal basis of eigenvectors of T for V and list the corresponding eigenvalues.

    1. (a) V=R2 and T is defined by T(a, b)=(2a2b, 2a+5b).

    2. (b) V=R3 and T is defined by T(a, b, c)=(a+b, 5b, 4a2b+5c).

    3. (c) V=C2 and T is defined by T(a, b)=(2a+ib, a+2b).

    4. (d) V=P2(R) and T is defined by T(f)=f, where

      f ( x ) ,   g ( x ) = 0 1 f ( t ) g ( t )   d t .
    5. (e) V=M2×2(R) and T is defined by T(A)=At.

    6. (f) V=M2×2(R) and T is defined by T(abcd)=(cdab).

  3. Give an example of a linear operator T on R2 and an ordered basis for R2 that provides a counterexample to the statement in Exercise 1(c).

  4. Let T and U be self-adjoint operators on an inner product space V. Prove that TU is self-adjoint if and only if TU=UT.

  5. Prove (b) of Theorem 6.15.

  6. Let V be a complex inner product space, and let T be a linear operator on V. Define

    T 1 = 1 2 ( T + T * ) and T 2 = 1 2 i ( T T * ) .
    1. (a) Prove that T1 and T2 are self-adjoint and that T=T1+iT2.

    2. (b) Suppose also that T=U1+iU2, where U1 and U2 are self-adjoint. Prove that U1=T1 and U2=T2.

    3. (c) Prove that T is normal if and only if T1T2=T2T1.

  7. Let T be a linear operator on an inner product space V, and let W be a T-invariant subspace of V. Prove the following results.

    1. (a) If T is self-adjoint, then TW is self-adjoint.

    2. (b) W is T*-invariant.

    3. (c) If W is both T- and T*-invariant, then (TW)*=(T*)W.

    4. (d) If W is both T- and T*-invariant and T is normal, then TW is normal.

  8. Let T be a normal operator on a finite-dimensional complex inner product space V, and let W be a subspace of V. Prove that if W is T-invariant, then W is also T*-invariant. Hint: Use Exercise 24 of Section 5.4.

  9. Let T be a normal operator on a finite-dimensional inner product space V. Prove that N(T)=N(T*) and R(T)=R(T*). Hint: Use Theorem 6.15 and Exercise 12 of Section 6.3.

  10. Let T be a self-adjoint operator on a finite-dimensional inner product space V. Prove that for all xV

    | | T ( x ) ± i x | | 2 = | | T ( x ) | | 2 + | | x | | 2 .

    Deduce that TiI is invertible and that the adjoint of (TiI)1 is (TiI)1.

  11. Assume that T is a linear operator on a complex (not necessarily finite-dimensional) inner product space V with an adjoint T*. Prove the following results.

    1. (a) If T is self-adjoint, then T(x), x is real for all xV.

    2. (b) If T satisfies T(x), x =0 for all xV, then T=T0. Hint: Replace x by x+y and then by x+iy, and expand the resulting inner products.

    3. (c) If T(x), x is real for all xV, then T is self-adjoint.

  12. Let T be a normal operator on a finite-dimensional real inner product space V whose characteristic polynomial splits. Prove that V has an orthonormal basis of eigenvectors of T. Hence prove that T is self-adjoint.

  13. An n×n real matrix A is said to be a Gramian matrix if there exists a real (square) matrix B such that A=BtB. Prove that A is a Gramian matrix if and only if A is symmetric and all of its eigenvalues are non-negative. Hint: Apply Theorem 6.17 to T=LA to obtain an orthonormal basis {v1, v2, , vn} of eigenvectors with the associated eigenvalues λ1, λ2, , λn. Define the linear operator U by U(vi)=λivi.

  14. Simultaneous Diagonalization. Let V be a finite-dimensional real inner product space, and let U and T be self-adjoint linear operators on V such that UT=TU. Prove that there exists an orthonormal basis for V consisting of vectors that are eigenvectors of both U and T. (The complex version of this result appears as Exercise 10 of Section 6.6.) Hint: For any eigenspace W=Eλ of T, we have that W is both T- and U-invariant. By Exercise 7, we have that W is both T- and U-invariant. Apply Theorem 6.17 and Theorem 6.6 (p. 347).

  15. Let A and B be symmetric n×n matrices such that AB=BA. Use Exercise 14 to prove that there exists an orthogonal matrix P such that PtAP and PtBP are both diagonal matrices.

  16. Prove the Cayley-Hamilton theorem for a complex n×n matrix A. That is, if f(t) is the characteristic polynomial of A, prove that f(A)=O. Hint: Use Schur’s theorem to show that A may be assumed to be upper triangular, in which case

    f ( t ) = i = 1 n ( A i i t ) .

    Now if T=LA, we have (AjjIT)(ej)span({e1, e2, , ej1}) for j2, where {e1, e2, , en} is the standard ordered basis for Cn. (The general case is proved in Section 5.4.)

The following definitions are used in Exercises 17 through 23.

Definitions.

A linear operator T on a finite-dimensional inner product space is called positive definite [positive semidefinite] if T is self-adjoint and T(x), x >0 [ T(x), x 0] for all x0.

An n×n matrix A with entries from R or C is called positive definite [positive semidefinite] if LA is positive definite [positive semidefinite].

  1. Let T and U be self-adjoint linear operators on an n-dimensional inner product space V, and let A=[T]β, where β is an orthonormal basis for V. Prove the following results.

    1. (a) T is positive definite [semidefinite] if and only if all of its eigenvalues are positive [nonnegative].

    2. (b) T is positive definite if and only if

      i ,   j A i j a j a ¯ i > 0  for all nonzero  n -tuples  ( a 1 ,   a 2 ,   ,   a n ) .
    3. (c) T is positive semidefinite if and only if A=B*B for some square matrix B.

    4. (d) If T and U are positive semidefinite operators such that T2=U2, then T=U.

    5. (e) If T and U are positive definite operators such that TU=UT, then TU is positive definite.

    6. (f) T is positive definite [semidefinite] if and only if A is positive definite [semidefinite].

    Because of (f), results analogous to items (a) through (d) hold for matrices as well as operators.

  2. Let T: VW be a linear transformation, where V and W are finite-dimensional inner product spaces. Prove the following results.

    1. (a) T*T and TT* are positive semidefinite. (See Exercise 15 of Section 6.3.)

    2. (b) rank(T*T)=rank(TT*)=rank(T).

  3. Let T and U be positive definite operators on an inner product space V. Prove the following results.

    1. (a) T+U is positive definite.

    2. (b) If c>0, then cT is positive definite.

    3. (c) T1 is positive definite.

    Visit goo.gl/cQch7i for a solution.

  4. Let V be an inner product space with inner product ,  , and let T be a positive definite linear operator on V. Prove that x, y = T(x), y defines another inner product on V.

  5. Let V be a finite-dimensional inner product space, and let T and U be self-adjoint operators on V such that T is positive definite. Prove that both TU and UT are diagonalizable linear operators that have only real eigenvalues. Hint: Show that UT is self-adjoint with respect to the inner product x, y = T(x), y . To show that TU is self-adjoint, repeat the argument with T1 in place of T.

  6. This exercise provides a converse to Exercise 20. Let V be a finite-dimensional inner product space with inner product ,  , and let ,  be any other inner product on V.

    1. (a) Prove that there exists a unique linear operator T on V such that x, y = T(x), y for all x and y in V. Hint: Let β={v1, v2, , vn} be an orthonormal basis for V with respect to ,  , and define a matrix A by Aij= vj, vi for all i and j. Let T be the unique linear operator on V such that [T]β=A.

    2. (b) Prove that the operator T of (a) is positive definite with respect to both inner products.

  7. Let U be a diagonalizable linear operator on a finite-dimensional inner product space V such that all of the eigenvalues of U are real. Prove that there exist positive definite linear operators T1 and T1 and self-adjoint linear operators T2 and T2 such that U=T2T1=T1T2. Hint: Let ,  be the inner product associated with V, β a basis of eigenvectors for U, ,  the inner product on V with respect to which β is orthonormal (see Exercise 22(a) of Section 6.1), and T1 the positive definite operator according to Exercise 22. Show that U is self-adjoint with respect to ,  and U=T11U*T1 (the adjoint is with respect to ,  ). Let T2=T11U*.